Merge the array and object branches of the BON8 iterative writer

write_bon8_iterative and write_bon8_value_or_push handled arrays and
objects in separate branches that repeated the end-of-container check,
the 0xFE closer and the marker computation. Share those parts and branch
only where arrays and objects really differ. The output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hz7VJi1FTKr6gpseLErbbS
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/docset/generate_docset.py

@@ -337,14 +337,17 @@ def is_remote(url) -> bool:
 def download(url, docs) -> str:
     """Download url into assets/external and return the path relative to docs."""
     u = urllib.parse.urlparse(url if not url.startswith('//') else 'https:' + url)
+    if u.scheme.lower() not in ('http', 'https'):
+        raise ValueError(f'not an http(s) URL: {url}')
     req = urllib.request.Request(u.geturl(), headers={'User-Agent': USER_AGENT})
-    with urllib.request.urlopen(req, timeout=20) as r:
+    # (the scheme is checked above)
+    with urllib.request.urlopen(req, timeout=20) as r:  # nosec B310
         data = r.read()
         ctype = r.headers.get_content_type()
     path = urllib.parse.unquote(u.path).lstrip('/')
     ext = os.path.splitext(path)[1]
     if u.query or not ext or path.endswith('/'):
-        digest = hashlib.sha1(url.encode()).hexdigest()[:12]
+        digest = hashlib.sha1(url.encode(), usedforsecurity=False).hexdigest()[:12]
         path = os.path.join(os.path.dirname(path), digest + CONTENT_TYPE_EXT.get(ctype, ext or '.bin'))
     rel = os.path.normpath(os.path.join('assets', 'external', u.hostname, path))
     out = os.path.join(docs, rel)
@@ -385,7 +388,8 @@ def localize_images(docs) -> None:
 def load_mkdocs_yml() -> dict:
     """Load mkdocs.yml, ignoring tags like !ENV and !!python/name."""
     with open(MKDOCS_YML, encoding='utf-8') as f:
-        return yaml.load(f, Loader=Loader)
+        # (Loader is a yaml.SafeLoader)
+        return yaml.load(f, Loader=Loader)  # nosec B506
 
 
 def localize_site_urls(docs, site_url) -> None:

docs/mkdocs/docs/api/basic_json/at.md

@@ -241,6 +241,4 @@ Strong exception safety: if an exception occurs, the original value stays intact
 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as
    already supported by [`operator[]`](operator[].md), [`value`](value.md), [`find`](find.md), and other lookup
    functions.
-4. Added in version 2.0.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
-[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is not
-a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.
+4. Added in version 2.0.0.

docs/mkdocs/docs/api/basic_json/operator[].md

@@ -286,6 +286,4 @@ Strong exception safety: if an exception occurs, the original value stays intact
 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as
    already supported by [`at`](at.md), [`value`](value.md), [`find`](find.md), and other lookup functions.
 4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
-   version 3.13.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
-[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is not
-a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.
+   version 3.13.0.

docs/mkdocs/docs/api/basic_json/patch.md

@@ -112,6 +112,3 @@ is thrown. In any case, the original value is not changed: the patch is applied
   location has a non-object/non-array parent in version 3.13.0.
 - Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper
   prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0.
-- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
-  [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is
-  not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.

docs/mkdocs/docs/api/basic_json/patch_inplace.md

@@ -110,6 +110,3 @@ function throws an exception.
   location has a non-object/non-array parent in version 3.13.0.
 - Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper
   prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0.
-- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
-  [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is
-  not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.

docs/mkdocs/docs/api/basic_json/unflatten.md

@@ -36,9 +36,8 @@ The function can throw the following exceptions:
 - Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) if an array index in a key is not a
   number; example: `"array index 'one' is not a number"`
 - Throws [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) if a level becomes an array
-  (because one of its keys is `0`) and another key at that level begins with a digit but is not a valid array index
-  (such as `1a`), or is `-`; example:
-  `"unresolved reference token '-'"`
+  (because one of its keys is `0`) and another key at that level cannot be an array index; example:
+  `"unresolved reference token 'x'"`
 
 ## Complexity
 
@@ -81,6 +80,3 @@ Apart from these two cases, for a JSON value `j`, the following is always true:
 
 - Added in version 2.0.0.
 - Made the array/object decision independent of the object's iteration order in version 3.13.0.
-- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
-  [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is
-  not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.

docs/mkdocs/docs/api/basic_json/value.md

@@ -227,6 +227,4 @@ changes to any JSON value.
    [`operator[]`](operator[].md), [`at`](at.md), [`find`](find.md), and other lookup functions.
 3. Added in version 2.0.2. Extended to work with arrays in version 3.13.0, including fixing an issue where resolving
    `ptr` through an array unexpectedly threw `out_of_range` instead of returning the resolved element (or
-   `default_value`, as documented). Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109)
-   instead of returning `default_value` for a one-character array index that is not a digit (e.g., `/x`) in version
-   3.13.0, as it already did for longer ones.
+   `default_value`, as documented).

docs/mkdocs/docs/home/exceptions.md

@@ -278,9 +278,7 @@ In a JSON Pointer, only `~0` and `~1` are valid escape sequences.
 
 ### json.exception.parse_error.109
 
-A JSON Pointer array index must be a number. This exception is thrown for an array index that does not begin with a
-digit, except for `-` and the empty reference token, which throw [`out_of_range.404`](#jsonexceptionout_of_range404)
-where they cannot be resolved.
+A JSON Pointer array index must be a number.
 
 !!! failure "Example messages"
 
@@ -291,11 +289,6 @@ where they cannot be resolved.
     [json.exception.parse_error.109] parse error: array index '+1' is not a number
     ```
 
-!!! note
-
-    Before version 3.13.0, a one-character array index that is not a digit (e.g., `x`) threw
-    [`out_of_range.404`](#jsonexceptionout_of_range404) instead.
-
 ### json.exception.parse_error.110
 
 When parsing a [binary format](../features/binary_formats/index.md), the byte vector ends before the complete value has
@@ -878,8 +871,7 @@ The provided key was not found in the JSON object.
 
 ### json.exception.out_of_range.404
 
-A reference token in a JSON Pointer could not be resolved, for instance an array index that begins with a digit but
-contains other characters (e.g., `1a`), or `-` where it cannot be used.
+A reference token in a JSON Pointer could not be resolved.
 
 !!! failure "Example message"
 

include/nlohmann/detail/json_pointer.hpp

@@ -255,7 +255,7 @@ class json_pointer
     {
         ok,                ///< @a s is a valid, representable array index
         leading_zero,      ///< @a s begins with '0' but has more than one character
-        not_a_number,      ///< @a s is neither empty nor "-" and does not begin with a digit
+        not_a_number,      ///< @a s does not begin with a digit
         unresolved,        ///< @a s could not be converted to an integer
         exceeds_size_type  ///< @a s converts to an integer that exceeds size_type
     };
@@ -282,10 +282,8 @@ class json_pointer
             return array_index_status::leading_zero;
         }
 
-        // error condition (cf. RFC 6901, Sect. 4); this also covers single-
-        // character tokens, so "/x" and "/xy" fail alike; "-" and the empty
-        // token are left to the conversion below and are reported as unresolved
-        if (JSON_HEDLEY_UNLIKELY(!s.empty() && s != "-" && !(s[0] >= '0' && s[0] <= '9')))
+        // error condition (cf. RFC 6901, Sect. 4)
+        if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && !(s[0] >= '1' && s[0] <= '9')))
         {
             return array_index_status::not_a_number;
         }

include/nlohmann/detail/output/binary_writer.hpp

@@ -878,9 +878,9 @@ class binary_writer
         }
     }
 
-    /// @brief a CBOR or MessagePack array or object whose elements
-    ///        @ref write_cbor_iterative or @ref write_msgpack_iterative is
-    ///        still writing
+    /// @brief a CBOR, MessagePack, or BON8 array or object whose elements
+    ///        @ref write_cbor_iterative, @ref write_msgpack_iterative, or
+    ///        @ref write_bon8_iterative is still writing
     struct binary_container_frame
     {
         explicit binary_container_frame(const BasicJsonType* value_) noexcept
@@ -2570,9 +2570,26 @@ class binary_writer
     @param[in] j                JSON value to serialize
     @param[in,out] string_open  whether the output ends with a non-empty
                                 string that has not been terminated with 0xFF
+    @param[in] depth            nesting level of @a j, counted from the
+                                top-level value passed to @ref basic_json::to_bon8
+    @throw type_error.316 if a string value or an object key is not valid
+           UTF-8
+    @throw out_of_range.407 if an unsigned integer does not fit int64
+
+    Values nested deeper than @ref recursion_depth_limit are written by
+    @ref write_bon8_iterative without the call stack.
+
+    @sa @ref write_cbor
+    @sa https://github.com/nlohmann/json/issues/5392
     */
-    void write_bon8_value(const BasicJsonType& j, bool& string_open)
+    void write_bon8_value(const BasicJsonType& j, bool& string_open, const std::size_t depth = 0)
     {
+        if (JSON_HEDLEY_UNLIKELY(depth >= recursion_depth_limit()) && (j.is_array() || j.is_object()))
+        {
+            write_bon8_iterative(j, string_open);
+            return;
+        }
+
         switch (j.type())
         {
             case value_t::null:
@@ -2626,7 +2643,7 @@ class binary_writer
 
                 for (const auto& el : *j.m_data.m_value.array)
                 {
-                    write_bon8_value(el, string_open);
+                    write_bon8_value(el, string_open, depth + 1);
                 }
 
                 if (N > 4)
@@ -2645,7 +2662,7 @@ class binary_writer
                 for (const auto& el : *j.m_data.m_value.object)
                 {
                     write_bon8_string(el.first, string_open, j);
-                    write_bon8_value(el.second, string_open);
+                    write_bon8_value(el.second, string_open, depth + 1);
                 }
 
                 if (N > 4)
@@ -2682,6 +2699,83 @@ class binary_writer
         }
     }
 
+    /*!
+    @brief write @a j with @ref write_bon8_value, or write its marker and push
+           a frame for @ref write_bon8_iterative to continue with its elements
+
+    A scalar, and an empty array or object, are written out in full and not
+    pushed.
+
+    @sa @ref write_cbor_value_or_push
+    */
+    void write_bon8_value_or_push(const BasicJsonType& j, bool& string_open, std::vector<binary_container_frame>& stack)
+    {
+        if (j.is_array() || j.is_object())
+        {
+            // arrays use the markers 0x80..0x85, objects 0x86..0x8B; the last
+            // one stands for more than four elements, closed later with 0xFE
+            const auto N = j.size();
+            const std::size_t base = j.is_array() ? 0x80 : 0x86;
+            write_bon8_marker(static_cast<std::uint8_t>(base + (N <= 4 ? N : 5)), string_open);
+            if (N != 0)
+            {
+                stack.emplace_back(&j);
+            }
+            return;
+        }
+
+        write_bon8_value(j, string_open);
+    }
+
+    /*!
+    @brief write out @a root and everything below it without the call stack
+
+    A container with more than four elements is closed with 0xFE once its
+    last element is written, as in @ref write_bon8_value.
+
+    @sa @ref write_cbor_iterative
+    */
+    void write_bon8_iterative(const BasicJsonType& root, bool& string_open)
+    {
+        // only a container with elements is ever pushed; see write_bon8_value_or_push
+        std::vector<binary_container_frame> stack;
+        write_bon8_value_or_push(root, string_open, stack);
+
+        while (!stack.empty())
+        {
+            const binary_container_frame current = stack.back();
+            const bool is_array = current.value->is_array();
+            const bool at_end = is_array
+                                ? current.array_it == current.value->m_data.m_value.array->cend()
+                                : current.object_it == current.value->m_data.m_value.object->cend();
+
+            if (at_end)
+            {
+                if (current.value->size() > 4)
+                {
+                    write_bon8_marker(0xFE, string_open);
+                }
+                stack.pop_back();
+                continue;
+            }
+
+            // read the child before pushing: entering it can move every frame
+            const BasicJsonType* child = nullptr;
+            if (is_array)
+            {
+                child = &(*current.array_it);
+                ++stack.back().array_it;
+            }
+            else
+            {
+                write_bon8_string(current.object_it->first, string_open, *current.value);
+                child = &(current.object_it->second);
+                ++stack.back().object_it;
+            }
+            write_bon8_value_or_push(*child, string_open, stack);
+        }
+    }
+
     /*!
     @brief write a single byte that is not part of a string
 

include/nlohmann/ordered_map.hpp

@@ -247,14 +247,6 @@ public:
         //               ^        ^
         //             first    last
 
-        // Note on conformance: before C++20, [basic.life]/8 did not allow an
-        // object of a type with a const member (like value_type's const Key)
-        // to transparently replace the destroyed one, so strictly, accessing
-        // it through the vector's existing pointers would have required
-        // std::launder (which does not exist before C++17). C++20 dropped that
-        // condition (P1971R0, NB comment US 041). Compilers have always treated
-        // this pattern as intended, so it is kept deliberately.
-
         // Since we cannot move const Keys, we re-construct them in place.
         // We start at first and re-construct (viz. copy) the elements from
         // the back of the vector. Example for the first iteration:

single_include/nlohmann/json.hpp

@@ -20173,7 +20173,7 @@ class json_pointer
     {
         ok,                ///< @a s is a valid, representable array index
         leading_zero,      ///< @a s begins with '0' but has more than one character
-        not_a_number,      ///< @a s is neither empty nor "-" and does not begin with a digit
+        not_a_number,      ///< @a s does not begin with a digit
         unresolved,        ///< @a s could not be converted to an integer
         exceeds_size_type  ///< @a s converts to an integer that exceeds size_type
     };
@@ -20200,10 +20200,8 @@ class json_pointer
             return array_index_status::leading_zero;
         }
 
-        // error condition (cf. RFC 6901, Sect. 4); this also covers single-
-        // character tokens, so "/x" and "/xy" fail alike; "-" and the empty
-        // token are left to the conversion below and are reported as unresolved
-        if (JSON_HEDLEY_UNLIKELY(!s.empty() && s != "-" && !(s[0] >= '0' && s[0] <= '9')))
+        // error condition (cf. RFC 6901, Sect. 4)
+        if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && !(s[0] >= '1' && s[0] <= '9')))
         {
             return array_index_status::not_a_number;
         }
@@ -22497,9 +22495,9 @@ class binary_writer
         }
     }
 
-    /// @brief a CBOR or MessagePack array or object whose elements
-    ///        @ref write_cbor_iterative or @ref write_msgpack_iterative is
-    ///        still writing
+    /// @brief a CBOR, MessagePack, or BON8 array or object whose elements
+    ///        @ref write_cbor_iterative, @ref write_msgpack_iterative, or
+    ///        @ref write_bon8_iterative is still writing
     struct binary_container_frame
     {
         explicit binary_container_frame(const BasicJsonType* value_) noexcept
@@ -24189,9 +24187,26 @@ class binary_writer
     @param[in] j                JSON value to serialize
     @param[in,out] string_open  whether the output ends with a non-empty
                                 string that has not been terminated with 0xFF
+    @param[in] depth            nesting level of @a j, counted from the
+                                top-level value passed to @ref basic_json::to_bon8
+    @throw type_error.316 if a string value or an object key is not valid
+           UTF-8
+    @throw out_of_range.407 if an unsigned integer does not fit int64
+
+    Values nested deeper than @ref recursion_depth_limit are written by
+    @ref write_bon8_iterative without the call stack.
+
+    @sa @ref write_cbor
+    @sa https://github.com/nlohmann/json/issues/5392
     */
-    void write_bon8_value(const BasicJsonType& j, bool& string_open)
+    void write_bon8_value(const BasicJsonType& j, bool& string_open, const std::size_t depth = 0)
     {
+        if (JSON_HEDLEY_UNLIKELY(depth >= recursion_depth_limit()) && (j.is_array() || j.is_object()))
+        {
+            write_bon8_iterative(j, string_open);
+            return;
+        }
+
         switch (j.type())
         {
             case value_t::null:
@@ -24245,7 +24260,7 @@ class binary_writer
 
                 for (const auto& el : *j.m_data.m_value.array)
                 {
-                    write_bon8_value(el, string_open);
+                    write_bon8_value(el, string_open, depth + 1);
                 }
 
                 if (N > 4)
@@ -24264,7 +24279,7 @@ class binary_writer
                 for (const auto& el : *j.m_data.m_value.object)
                 {
                     write_bon8_string(el.first, string_open, j);
-                    write_bon8_value(el.second, string_open);
+                    write_bon8_value(el.second, string_open, depth + 1);
                 }
 
                 if (N > 4)
@@ -24301,6 +24316,83 @@ class binary_writer
         }
     }
 
+    /*!
+    @brief write @a j with @ref write_bon8_value, or write its marker and push
+           a frame for @ref write_bon8_iterative to continue with its elements
+
+    A scalar, and an empty array or object, are written out in full and not
+    pushed.
+
+    @sa @ref write_cbor_value_or_push
+    */
+    void write_bon8_value_or_push(const BasicJsonType& j, bool& string_open, std::vector<binary_container_frame>& stack)
+    {
+        if (j.is_array() || j.is_object())
+        {
+            // arrays use the markers 0x80..0x85, objects 0x86..0x8B; the last
+            // one stands for more than four elements, closed later with 0xFE
+            const auto N = j.size();
+            const std::size_t base = j.is_array() ? 0x80 : 0x86;
+            write_bon8_marker(static_cast<std::uint8_t>(base + (N <= 4 ? N : 5)), string_open);
+            if (N != 0)
+            {
+                stack.emplace_back(&j);
+            }
+            return;
+        }
+
+        write_bon8_value(j, string_open);
+    }
+
+    /*!
+    @brief write out @a root and everything below it without the call stack
+
+    A container with more than four elements is closed with 0xFE once its
+    last element is written, as in @ref write_bon8_value.
+
+    @sa @ref write_cbor_iterative
+    */
+    void write_bon8_iterative(const BasicJsonType& root, bool& string_open)
+    {
+        // only a container with elements is ever pushed; see write_bon8_value_or_push
+        std::vector<binary_container_frame> stack;
+        write_bon8_value_or_push(root, string_open, stack);
+
+        while (!stack.empty())
+        {
+            const binary_container_frame current = stack.back();
+            const bool is_array = current.value->is_array();
+            const bool at_end = is_array
+                                ? current.array_it == current.value->m_data.m_value.array->cend()
+                                : current.object_it == current.value->m_data.m_value.object->cend();
+
+            if (at_end)
+            {
+                if (current.value->size() > 4)
+                {
+                    write_bon8_marker(0xFE, string_open);
+                }
+                stack.pop_back();
+                continue;
+            }
+
+            // read the child before pushing: entering it can move every frame
+            const BasicJsonType* child = nullptr;
+            if (is_array)
+            {
+                child = &(*current.array_it);
+                ++stack.back().array_it;
+            }
+            else
+            {
+                write_bon8_string(current.object_it->first, string_open, *current.value);
+                child = &(current.object_it->second);
+                ++stack.back().object_it;
+            }
+            write_bon8_value_or_push(*child, string_open, stack);
+        }
+    }
+
     /*!
     @brief write a single byte that is not part of a string
 
@@ -27782,14 +27874,6 @@ public:
         //               ^        ^
         //             first    last
 
-        // Note on conformance: before C++20, [basic.life]/8 did not allow an
-        // object of a type with a const member (like value_type's const Key)
-        // to transparently replace the destroyed one, so strictly, accessing
-        // it through the vector's existing pointers would have required
-        // std::launder (which does not exist before C++17). C++20 dropped that
-        // condition (P1971R0, NB comment US 041). Compilers have always treated
-        // this pattern as intended, so it is kept deliberately.
-
         // Since we cannot move const Keys, we re-construct them in place.
         // We start at first and re-construct (viz. copy) the elements from
         // the back of the vector. Example for the first iteration:

tests/src/unit-element_access2.cpp

@@ -535,8 +535,6 @@ TEST_CASE_TEMPLATE("element access 2", Json, nlohmann::json, nlohmann::ordered_j
                     // Test malformed index (non-numeric) throws parse_error
                     CHECK_THROWS_WITH_AS(j_array.value("/foo"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'foo' is not a number", typename Json::parse_error&);
                     CHECK_THROWS_WITH_AS(j_array_const.value("/foo"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'foo' is not a number", typename Json::parse_error&);
-                    CHECK_THROWS_WITH_AS(j_array.value("/x"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'x' is not a number", typename Json::parse_error&);
-                    CHECK_THROWS_WITH_AS(j_array_const.value("/x"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'x' is not a number", typename Json::parse_error&);
 
                     // Test leading-zero index throws parse_error
                     CHECK_THROWS_WITH_AS(j_array.value("/01"_json_pointer, 1), "[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", typename Json::parse_error&);

tests/src/unit-json_patch.cpp

@@ -1713,21 +1713,6 @@ TEST_CASE("JSON patch - move where 'from' is a proper prefix of 'path' (regressi
         CHECK(doc.patch(patch) == R"({"b": 1})"_json);
     }
 
-    SECTION("array index tokens that are not a number")
-    {
-        json const doc = R"({"a": [1, 2]})"_json;
-
-        // "-" cannot be removed and is reported as unresolved
-        json const patch_dash = {{{"op", "remove"}, {"path", "/a/-"}}};
-        CHECK_THROWS_WITH_AS(doc.patch(patch_dash), "[json.exception.out_of_range.404] unresolved reference token '-'", json::out_of_range&);
-
-        // a single character is reported like a longer token
-        json const patch_x = {{{"op", "remove"}, {"path", "/a/x"}}};
-        CHECK_THROWS_WITH_AS(doc.patch(patch_x), "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&);
-        json const patch_xy = {{{"op", "remove"}, {"path", "/a/xy"}}};
-        CHECK_THROWS_WITH_AS(doc.patch(patch_xy), "[json.exception.parse_error.109] parse error: array index 'xy' is not a number", json::parse_error&);
-    }
-
     SECTION("the array-append token '-' is an ordinary child token")
     {
         // "-" (append-to-array) addresses a location *inside* the array,

tests/src/unit-json_pointer.cpp

@@ -421,27 +421,6 @@ TEST_CASE("JSON pointers")
             CHECK_THROWS_WITH_AS(json({{"/list/0", 1}, {"/list/1", 2}, {"/list/three", 3}}).unflatten(),
             "[json.exception.parse_error.109] parse error: array index 'three' is not a number", json::parse_error&);
 
-            // a single-character token that is not a digit is reported like a
-            // longer one (parse_error.109), not as unresolved (out_of_range.404)
-            CHECK_THROWS_WITH_AS(j["/x"_json_pointer] = 1,
-                                 "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&);
-            CHECK_THROWS_WITH_AS(j_const["/x"_json_pointer] == 1,
-                                 "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&);
-            CHECK_THROWS_WITH_AS(j.at("/x"_json_pointer) = 1,
-                                 "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&);
-            CHECK_THROWS_WITH_AS(j_const.at("/x"_json_pointer) == 1,
-                                 "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&);
-            CHECK_THROWS_WITH_AS(j.at("/+"_json_pointer),
-                                 "[json.exception.parse_error.109] parse error: array index '+' is not a number", json::parse_error&);
-            CHECK(!j.contains("/x"_json_pointer));
-            CHECK(!j_const.contains("/x"_json_pointer));
-            CHECK_THROWS_WITH_AS(json({{"/list/0", 1}, {"/list/x", 2}}).unflatten(),
-            "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&);
-
-            // "-" is a valid reference token, so it is still reported as unresolved
-            CHECK_THROWS_WITH_AS(json({{"/list/0", 1}, {"/list/-", 2}}).unflatten(),
-            "[json.exception.out_of_range.404] unresolved reference token '-'", json::out_of_range&);
-
             // assign to "-"
             j["/-"_json_pointer] = 99;
             CHECK(j == json({1, 13, 3, 33, nullptr, 55, 99}));

tests/src/unit-large_json.cpp

@@ -400,26 +400,31 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
         CHECK(json::from_ubjson(json::to_ubjson(deep_array, true, false)) == deep_array);
         CHECK(json::from_ubjson(json::to_ubjson(deep_array, true, true)) == deep_array);
         CHECK(json::from_bjdata(json::to_bjdata(deep_array)) == deep_array);
+        CHECK(json::from_bon8(json::to_bon8(deep_array)) == deep_array);
 
         CHECK(json::from_cbor(json::to_cbor(deep_object)) == deep_object);
         CHECK(json::from_msgpack(json::to_msgpack(deep_object)) == deep_object);
         CHECK(json::from_ubjson(json::to_ubjson(deep_object)) == deep_object);
         CHECK(json::from_ubjson(json::to_ubjson(deep_object, true, true)) == deep_object);
         CHECK(json::from_bjdata(json::to_bjdata(deep_object)) == deep_object);
+        CHECK(json::from_bon8(json::to_bon8(deep_object)) == deep_object);
 
         CHECK(json::from_cbor(json::to_cbor(empty_array)) == empty_array);
         CHECK(json::from_msgpack(json::to_msgpack(empty_array)) == empty_array);
         CHECK(json::from_ubjson(json::to_ubjson(empty_array)) == empty_array);
         CHECK(json::from_ubjson(json::to_ubjson(empty_array, true, true)) == empty_array);
+        CHECK(json::from_bon8(json::to_bon8(empty_array)) == empty_array);
 
         CHECK(json::from_cbor(json::to_cbor(empty_object)) == empty_object);
         CHECK(json::from_msgpack(json::to_msgpack(empty_object)) == empty_object);
         CHECK(json::from_ubjson(json::to_ubjson(empty_object)) == empty_object);
+        CHECK(json::from_bon8(json::to_bon8(empty_object)) == empty_object);
 
         CHECK(json::from_cbor(json::to_cbor(mixed)) == mixed);
         CHECK(json::from_msgpack(json::to_msgpack(mixed)) == mixed);
         CHECK(json::from_ubjson(json::to_ubjson(mixed)) == mixed);
         CHECK(json::from_bjdata(json::to_bjdata(mixed)) == mixed);
+        CHECK(json::from_bon8(json::to_bon8(mixed)) == mixed);
     }
 
     SECTION("the two ways of writing a value meet at the bound")
@@ -432,11 +437,13 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
             CHECK(json::from_cbor(json::to_cbor(array)) == array);
             CHECK(json::from_msgpack(json::to_msgpack(array)) == array);
             CHECK(json::from_ubjson(json::to_ubjson(array, true, true)) == array);
+            CHECK(json::from_bon8(json::to_bon8(array)) == array);
 
             const json object = nested_object(depth, json(7));
             CHECK(json::from_cbor(json::to_cbor(object)) == object);
             CHECK(json::from_msgpack(json::to_msgpack(object)) == object);
             CHECK(json::from_bjdata(json::to_bjdata(object)) == object);
+            CHECK(json::from_bon8(json::to_bon8(object)) == object);
         }
     }
 
@@ -481,6 +488,10 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
             expected_ubjson.append(depth, ']');
             const auto packed_ubjson = json::to_ubjson(array);
             CHECK(std::string(packed_ubjson.begin(), packed_ubjson.end()) == expected_ubjson);
+
+            std::vector<std::uint8_t> expected_bon8(depth, 0x81);
+            expected_bon8.push_back(0x90);
+            CHECK(json::to_bon8(array) == expected_bon8);
         }
     }
 
@@ -493,6 +504,7 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
         CHECK(json::from_msgpack(json::to_msgpack(object)) == object);
         CHECK(json::from_ubjson(json::to_ubjson(object, true, true)) == object);
         CHECK(json::from_bjdata(json::to_bjdata(object)) == object);
+        CHECK(json::from_bon8(json::to_bon8(object)) == object);
 
         const json ndarray = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
         const json deep_ndarray = nested_array(depth, ndarray);
@@ -523,6 +535,27 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
         CHECK_THROWS_WITH_AS(json::to_bjdata(deep_discarded), (prefix + "cannot serialize discarded value to BJData").c_str(), json::type_error);
     }
 
+    SECTION("BON8 past the recursion bound: discarded values and errors")
+    {
+        const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
+
+        // BON8 writes nothing for a discarded value, deep or not
+        const json deep_discarded = nested_array(depth, json(json::value_t::discarded));
+        CHECK(json::to_bon8(deep_discarded) == std::vector<std::uint8_t>(depth, 0x81));
+
+        // errors are thrown from below the bound as from above it
+        const json deep_invalid_string = nested_array(depth, json(std::string("\x80")));
+        CHECK_THROWS_AS(json::to_bon8(deep_invalid_string), json::type_error);
+
+        json deep_invalid_key = json::object();
+        deep_invalid_key[std::string("\x80")] = 1;
+        deep_invalid_key = nested_object(depth, deep_invalid_key);
+        CHECK_THROWS_AS(json::to_bon8(deep_invalid_key), json::type_error);
+
+        const json deep_too_large = nested_array(depth, json(9223372036854775808u));
+        CHECK_THROWS_AS(json::to_bon8(deep_too_large), json::out_of_range);
+    }
+
     SECTION("does not overflow the C++ stack")
     {
         const std::size_t depth = 100000;
@@ -543,6 +576,27 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
 
         CHECK_NOTHROW(packed = json::to_bjdata(j));
         CHECK(json::from_bjdata(packed) == j);
+
+        CHECK_NOTHROW(packed = json::to_bon8(j));
+        CHECK(json::from_bon8(packed) == j);
+    }
+
+    SECTION("BON8 containers with more than four elements past the recursion bound")
+    {
+        // such containers are closed with 0xFE, which the iterative writer
+        // must emit when it leaves them; strings next to each other must
+        // still be separated with 0xFF
+        const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 50;
+        json wide_array = json::array({1, "a", "b", json::array(), 5});
+        json wide_object = json({{"k1", "v"}, {"k2", 2}, {"k3", ""}, {"k4", json::object()}, {"k5", nullptr}});
+        for (std::size_t i = 0; i < depth; ++i)
+        {
+            wide_array = json::array({"s", wide_array, "t", wide_object, true});
+            wide_object = json({{"a", wide_object}, {"b", "x"}, {"c", "y"}, {"d", wide_array.size()}, {"e", "z"}});
+        }
+
+        CHECK(json::from_bon8(json::to_bon8(wide_array)) == wide_array);
+        CHECK(json::from_bon8(json::to_bon8(wide_object)) == wide_object);
     }
 
     SECTION("regression test for https://issues.oss-fuzz.com/issues/566583014")
@@ -561,5 +615,14 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
         const json j_msgpack = json::from_msgpack(v_msgpack);
         CHECK(json::to_msgpack(j_msgpack) == v_msgpack);
     }
+
+    SECTION("regression test for https://issues.oss-fuzz.com/issues/572238015")
+    {
+        // the BON8 analogue: 200000 nested one-element arrays down to null
+        std::vector<std::uint8_t> v(200000, 0x81);
+        v.push_back(0xFA);
+        const json j = json::from_bon8(v);
+        CHECK(json::to_bon8(j) == v);
+    }
 }