Merge branch 'develop' into issue4552-ignore

develop already implements error_handler_t::keep for dump() (#5741,
#5746), including the ensure_ascii interaction, so the serializer and the
error_handler_t documentation follow develop; this branch keeps its
additional tests and the FAQ mention.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

README.md

@@ -361,7 +361,7 @@ std::cout << j_string << " == " << serialized_string << std::endl;
 
 [`.dump()`](https://json.nlohmann.me/api/basic_json/dump/) returns the originally stored string value.
 
-Note the library only supports UTF-8. When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
+Note the library only supports UTF-8. When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace`, `json::error_handler_t::ignore`, or `json::error_handler_t::keep` are used as error handlers.
 
 #### To/from streams (e.g., files, string streams)
 
@@ -1914,7 +1914,7 @@ The library supports **Unicode input** as follows:
 - [Unicode noncharacters](https://www.unicode.org/faq/private_use.html#nonchar1) will not be replaced by the library.
 - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
 - The strings stored in the library are UTF-8 encoded. When using the default string type (`std::string`), note that its length/size functions return the number of stored bytes rather than the number of characters or glyphs.
-- When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
+- When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace`, `json::error_handler_t::ignore`, or `json::error_handler_t::keep` are used as error handlers.
 - To store wide strings (e.g., `std::wstring`), you need to convert them to a UTF-8 encoded `std::string` before, see [an example](https://json.nlohmann.me/home/faq/#wide-string-handling).
 
 ### Comments in JSON

docs/mkdocs/docs/features/serialization.md

@@ -64,6 +64,7 @@ serialization fails by default. The fourth argument of `dump` selects an
 - `strict` (default) — throw a [`type_error.316`](../home/exceptions.md#jsonexceptiontype_error316) exception.
 - `replace` — replace invalid bytes with the Unicode replacement character U+FFFD (`�`).
 - `ignore` — silently drop invalid bytes.
+- `keep` — copy invalid bytes to the output unchanged; the result is not valid UTF-8.
 
 ??? example "Example: serialize invalid UTF-8 with different error handlers"
 

docs/mkdocs/docs/home/exceptions.md

@@ -771,6 +771,7 @@ as well for a string value or object key that is not valid UTF-8 if their `error
     - Pass an error handler as last parameter to the `dump()` function to avoid this exception:
         - `json::error_handler_t::replace` will replace invalid bytes sequences with `U+FFFD` 
         - `json::error_handler_t::ignore` will silently ignore invalid byte sequences
+        - `json::error_handler_t::keep` will copy invalid byte sequences to the output unchanged
 
 ### json.exception.type_error.317
 

docs/mkdocs/docs/home/faq.md

@@ -85,7 +85,7 @@ The library supports **Unicode input** as follows:
 - The library will not replace [Unicode noncharacters](http://www.unicode.org/faq/private_use.html#nonchar1).
 - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
 - The strings stored in the library are UTF-8 encoded. When using the default string type (`std::string`), note that its length/size functions return the number of stored bytes rather than the number of characters or glyphs.
-- When you store strings with different encodings in the library, calling [`dump()`](../api/basic_json/dump.md) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
+- When you store strings with different encodings in the library, calling [`dump()`](../api/basic_json/dump.md) may throw an exception unless `json::error_handler_t::replace`, `json::error_handler_t::ignore`, or `json::error_handler_t::keep` are used as error handlers.
 
 In most cases, the parser is right to complain, because the input is not UTF-8 encoded. This is especially true for Microsoft Windows, where Latin-1 or ISO 8859-1 is often the standard encoding.
 

include/nlohmann/detail/conversions/to_json.hpp

@@ -42,10 +42,6 @@ namespace detail
  * j.m_data.m_value.destroy(j.m_data.m_type) to avoid a memory leak in case j contains an
  * allocated value (e.g., a string). See bug issue
  * https://github.com/nlohmann/json/issues/2865 for more information.
- *
- * A value that has to be allocated is created before the old one is destroyed:
- * were it the other way around, an exception while creating the new value would
- * leave j with the type of the new value, but the pointer to the destroyed old one.
  */
 
 template<value_t> struct external_constructor;
@@ -69,20 +65,18 @@ struct external_constructor<value_t::string>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::string_t& s)
     {
-        const typename BasicJsonType::json_value value(s);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::string;
-        j.m_data.m_value = value;
+        j.m_data.m_value = s;
         j.assert_invariant();
     }
 
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::string_t&& s)
     {
-        const typename BasicJsonType::json_value value(std::move(s));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::string;
-        j.m_data.m_value = value;
+        j.m_data.m_value = std::move(s);
         j.assert_invariant();
     }
 
@@ -91,10 +85,9 @@ struct external_constructor<value_t::string>
                              int > = 0 >
     static void construct(BasicJsonType& j, const CompatibleStringType& str)
     {
-        auto* created = j.template create<typename BasicJsonType::string_t>(str);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::string;
-        j.m_data.m_value.string = created;
+        j.m_data.m_value.string = j.template create<typename BasicJsonType::string_t>(str);
         j.assert_invariant();
     }
 };
@@ -105,20 +98,18 @@ struct external_constructor<value_t::binary>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::binary_t& b)
     {
-        const typename BasicJsonType::json_value value(b);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::binary;
-        j.m_data.m_value = value;
+        j.m_data.m_value = typename BasicJsonType::binary_t(b);
         j.assert_invariant();
     }
 
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::binary_t&& b)
     {
-        const typename BasicJsonType::json_value value(std::move(b));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::binary;
-        j.m_data.m_value = value;
+        j.m_data.m_value = typename BasicJsonType::binary_t(std::move(b));
         j.assert_invariant();
     }
 };
@@ -168,10 +159,9 @@ struct external_constructor<value_t::array>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::array_t& arr)
     {
-        const typename BasicJsonType::json_value value(arr);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = arr;
         j.set_parents();
         j.assert_invariant();
     }
@@ -179,10 +169,9 @@ struct external_constructor<value_t::array>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::array_t&& arr)
     {
-        const typename BasicJsonType::json_value value(std::move(arr));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = std::move(arr);
         j.set_parents();
         j.assert_invariant();
     }
@@ -198,10 +187,9 @@ struct external_constructor<value_t::array>
         using std::begin;
         using std::end;
 
-        auto* created = j.template create<typename BasicJsonType::array_t>(begin(arr), end(arr));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value.array = created;
+        j.m_data.m_value.array = j.template create<typename BasicJsonType::array_t>(begin(arr), end(arr));
         j.set_parents();
         j.assert_invariant();
     }
@@ -209,17 +197,15 @@ struct external_constructor<value_t::array>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const std::vector<bool>& arr)
     {
-        typename BasicJsonType::array_t elements;
-        elements.reserve(arr.size());
-        for (const bool x : arr)
-        {
-            elements.push_back(x);
-        }
-        const typename BasicJsonType::json_value value(std::move(elements));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = value_t::array;
-        j.set_parents();
+        j.m_data.m_value.array->reserve(arr.size());
+        for (const bool x : arr)
+        {
+            j.m_data.m_value.array->push_back(x);
+            j.set_parent(j.m_data.m_value.array->back());
+        }
         j.assert_invariant();
     }
 
@@ -227,12 +213,11 @@ struct external_constructor<value_t::array>
              enable_if_t<std::is_convertible<T, BasicJsonType>::value, int> = 0>
     static void construct(BasicJsonType& j, const std::valarray<T>& arr)
     {
-        typename BasicJsonType::array_t elements(arr.size());
-        std::copy(std::begin(arr), std::end(arr), elements.begin());
-        const typename BasicJsonType::json_value value(std::move(elements));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = value_t::array;
+        j.m_data.m_value.array->resize(arr.size());
+        std::copy(std::begin(arr), std::end(arr), j.m_data.m_value.array->begin());
         j.set_parents();
         j.assert_invariant();
     }
@@ -242,15 +227,13 @@ struct external_constructor<value_t::array>
              enable_if_t<is_compatible_range_view<std::remove_cvref_t<CompatibleArrayType>>::value, int> = 0>
     static void construct(BasicJsonType& j, CompatibleArrayType && arr)
     {
-        typename BasicJsonType::array_t elements;
-        for (auto&& x : std::forward<CompatibleArrayType>(arr))
-        {
-            elements.push_back(x);
-        }
-        const typename BasicJsonType::json_value value(std::move(elements));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = value_t::array;
+        for (auto&& x : std::forward<CompatibleArrayType>(arr))
+        {
+            j.m_data.m_value.array->push_back(x);
+        }
         // set the parents only once all elements are in place: a push_back
         // that reallocates moves the earlier elements, which does not keep
         // their parent pointers
@@ -266,10 +249,9 @@ struct external_constructor<value_t::object>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::object_t& obj)
     {
-        const typename BasicJsonType::json_value value(obj);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::object;
-        j.m_data.m_value = value;
+        j.m_data.m_value = obj;
         j.set_parents();
         j.assert_invariant();
     }
@@ -277,10 +259,9 @@ struct external_constructor<value_t::object>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
     {
-        const typename BasicJsonType::json_value value(std::move(obj));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::object;
-        j.m_data.m_value = value;
+        j.m_data.m_value = std::move(obj);
         j.set_parents();
         j.assert_invariant();
     }
@@ -292,10 +273,9 @@ struct external_constructor<value_t::object>
         using std::begin;
         using std::end;
 
-        auto* created = j.template create<typename BasicJsonType::object_t>(begin(obj), end(obj));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::object;
-        j.m_data.m_value.object = created;
+        j.m_data.m_value.object = j.template create<typename BasicJsonType::object_t>(begin(obj), end(obj));
         j.set_parents();
         j.assert_invariant();
     }

include/nlohmann/json.hpp

@@ -2056,8 +2056,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         if (is_an_object)
         {
             // the initializer list is a list of pairs -> create an object
-            m_data.m_value = value_t::object;
             m_data.m_type = value_t::object;
+            m_data.m_value = value_t::object;
 
             for (auto& element_ref : init)
             {
@@ -2079,8 +2079,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             }
 #endif
             // the initializer list describes an array -> create an array
-            m_data.m_value.array = create<array_t>(init.begin(), init.end());
             m_data.m_type = value_t::array;
+            m_data.m_value.array = create<array_t>(init.begin(), init.end());
         }
 
         set_parents();
@@ -2093,8 +2093,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(const typename binary_t::container_type& init)
     {
         auto res = basic_json();
-        res.m_data.m_value = init;
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = init;
         return res;
     }
 
@@ -2104,8 +2104,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(const typename binary_t::container_type& init, typename binary_t::subtype_type subtype)
     {
         auto res = basic_json();
-        res.m_data.m_value = binary_t(init, subtype);
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = binary_t(init, subtype);
         return res;
     }
 
@@ -2115,8 +2115,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(typename binary_t::container_type&& init)
     {
         auto res = basic_json();
-        res.m_data.m_value = std::move(init);
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = std::move(init);
         return res;
     }
 
@@ -2126,8 +2126,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(typename binary_t::container_type&& init, typename binary_t::subtype_type subtype)
     {
         auto res = basic_json();
-        res.m_data.m_value = binary_t(std::move(init), subtype);
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = binary_t(std::move(init), subtype);
         return res;
     }
 

single_include/nlohmann/json.hpp

@@ -6747,10 +6747,6 @@ namespace detail
  * j.m_data.m_value.destroy(j.m_data.m_type) to avoid a memory leak in case j contains an
  * allocated value (e.g., a string). See bug issue
  * https://github.com/nlohmann/json/issues/2865 for more information.
- *
- * A value that has to be allocated is created before the old one is destroyed:
- * were it the other way around, an exception while creating the new value would
- * leave j with the type of the new value, but the pointer to the destroyed old one.
  */
 
 template<value_t> struct external_constructor;
@@ -6774,20 +6770,18 @@ struct external_constructor<value_t::string>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::string_t& s)
     {
-        const typename BasicJsonType::json_value value(s);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::string;
-        j.m_data.m_value = value;
+        j.m_data.m_value = s;
         j.assert_invariant();
     }
 
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::string_t&& s)
     {
-        const typename BasicJsonType::json_value value(std::move(s));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::string;
-        j.m_data.m_value = value;
+        j.m_data.m_value = std::move(s);
         j.assert_invariant();
     }
 
@@ -6796,10 +6790,9 @@ struct external_constructor<value_t::string>
                              int > = 0 >
     static void construct(BasicJsonType& j, const CompatibleStringType& str)
     {
-        auto* created = j.template create<typename BasicJsonType::string_t>(str);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::string;
-        j.m_data.m_value.string = created;
+        j.m_data.m_value.string = j.template create<typename BasicJsonType::string_t>(str);
         j.assert_invariant();
     }
 };
@@ -6810,20 +6803,18 @@ struct external_constructor<value_t::binary>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::binary_t& b)
     {
-        const typename BasicJsonType::json_value value(b);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::binary;
-        j.m_data.m_value = value;
+        j.m_data.m_value = typename BasicJsonType::binary_t(b);
         j.assert_invariant();
     }
 
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::binary_t&& b)
     {
-        const typename BasicJsonType::json_value value(std::move(b));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::binary;
-        j.m_data.m_value = value;
+        j.m_data.m_value = typename BasicJsonType::binary_t(std::move(b));
         j.assert_invariant();
     }
 };
@@ -6873,10 +6864,9 @@ struct external_constructor<value_t::array>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::array_t& arr)
     {
-        const typename BasicJsonType::json_value value(arr);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = arr;
         j.set_parents();
         j.assert_invariant();
     }
@@ -6884,10 +6874,9 @@ struct external_constructor<value_t::array>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::array_t&& arr)
     {
-        const typename BasicJsonType::json_value value(std::move(arr));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = std::move(arr);
         j.set_parents();
         j.assert_invariant();
     }
@@ -6903,10 +6892,9 @@ struct external_constructor<value_t::array>
         using std::begin;
         using std::end;
 
-        auto* created = j.template create<typename BasicJsonType::array_t>(begin(arr), end(arr));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value.array = created;
+        j.m_data.m_value.array = j.template create<typename BasicJsonType::array_t>(begin(arr), end(arr));
         j.set_parents();
         j.assert_invariant();
     }
@@ -6914,17 +6902,15 @@ struct external_constructor<value_t::array>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const std::vector<bool>& arr)
     {
-        typename BasicJsonType::array_t elements;
-        elements.reserve(arr.size());
-        for (const bool x : arr)
-        {
-            elements.push_back(x);
-        }
-        const typename BasicJsonType::json_value value(std::move(elements));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = value_t::array;
-        j.set_parents();
+        j.m_data.m_value.array->reserve(arr.size());
+        for (const bool x : arr)
+        {
+            j.m_data.m_value.array->push_back(x);
+            j.set_parent(j.m_data.m_value.array->back());
+        }
         j.assert_invariant();
     }
 
@@ -6932,12 +6918,11 @@ struct external_constructor<value_t::array>
              enable_if_t<std::is_convertible<T, BasicJsonType>::value, int> = 0>
     static void construct(BasicJsonType& j, const std::valarray<T>& arr)
     {
-        typename BasicJsonType::array_t elements(arr.size());
-        std::copy(std::begin(arr), std::end(arr), elements.begin());
-        const typename BasicJsonType::json_value value(std::move(elements));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = value_t::array;
+        j.m_data.m_value.array->resize(arr.size());
+        std::copy(std::begin(arr), std::end(arr), j.m_data.m_value.array->begin());
         j.set_parents();
         j.assert_invariant();
     }
@@ -6947,15 +6932,13 @@ struct external_constructor<value_t::array>
              enable_if_t<is_compatible_range_view<std::remove_cvref_t<CompatibleArrayType>>::value, int> = 0>
     static void construct(BasicJsonType& j, CompatibleArrayType && arr)
     {
-        typename BasicJsonType::array_t elements;
-        for (auto&& x : std::forward<CompatibleArrayType>(arr))
-        {
-            elements.push_back(x);
-        }
-        const typename BasicJsonType::json_value value(std::move(elements));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::array;
-        j.m_data.m_value = value;
+        j.m_data.m_value = value_t::array;
+        for (auto&& x : std::forward<CompatibleArrayType>(arr))
+        {
+            j.m_data.m_value.array->push_back(x);
+        }
         // set the parents only once all elements are in place: a push_back
         // that reallocates moves the earlier elements, which does not keep
         // their parent pointers
@@ -6971,10 +6954,9 @@ struct external_constructor<value_t::object>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, const typename BasicJsonType::object_t& obj)
     {
-        const typename BasicJsonType::json_value value(obj);
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::object;
-        j.m_data.m_value = value;
+        j.m_data.m_value = obj;
         j.set_parents();
         j.assert_invariant();
     }
@@ -6982,10 +6964,9 @@ struct external_constructor<value_t::object>
     template<typename BasicJsonType>
     static void construct(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
     {
-        const typename BasicJsonType::json_value value(std::move(obj));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::object;
-        j.m_data.m_value = value;
+        j.m_data.m_value = std::move(obj);
         j.set_parents();
         j.assert_invariant();
     }
@@ -6997,10 +6978,9 @@ struct external_constructor<value_t::object>
         using std::begin;
         using std::end;
 
-        auto* created = j.template create<typename BasicJsonType::object_t>(begin(obj), end(obj));
         j.m_data.m_value.destroy(j.m_data.m_type);
         j.m_data.m_type = value_t::object;
-        j.m_data.m_value.object = created;
+        j.m_data.m_value.object = j.template create<typename BasicJsonType::object_t>(begin(obj), end(obj));
         j.set_parents();
         j.assert_invariant();
     }
@@ -29025,8 +29005,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         if (is_an_object)
         {
             // the initializer list is a list of pairs -> create an object
-            m_data.m_value = value_t::object;
             m_data.m_type = value_t::object;
+            m_data.m_value = value_t::object;
 
             for (auto& element_ref : init)
             {
@@ -29048,8 +29028,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             }
 #endif
             // the initializer list describes an array -> create an array
-            m_data.m_value.array = create<array_t>(init.begin(), init.end());
             m_data.m_type = value_t::array;
+            m_data.m_value.array = create<array_t>(init.begin(), init.end());
         }
 
         set_parents();
@@ -29062,8 +29042,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(const typename binary_t::container_type& init)
     {
         auto res = basic_json();
-        res.m_data.m_value = init;
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = init;
         return res;
     }
 
@@ -29073,8 +29053,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(const typename binary_t::container_type& init, typename binary_t::subtype_type subtype)
     {
         auto res = basic_json();
-        res.m_data.m_value = binary_t(init, subtype);
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = binary_t(init, subtype);
         return res;
     }
 
@@ -29084,8 +29064,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(typename binary_t::container_type&& init)
     {
         auto res = basic_json();
-        res.m_data.m_value = std::move(init);
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = std::move(init);
         return res;
     }
 
@@ -29095,8 +29075,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static basic_json binary(typename binary_t::container_type&& init, typename binary_t::subtype_type subtype)
     {
         auto res = basic_json();
-        res.m_data.m_value = binary_t(std::move(init), subtype);
         res.m_data.m_type = value_t::binary;
+        res.m_data.m_value = binary_t(std::move(init), subtype);
         return res;
     }
 

tests/CMakeLists.txt

@@ -140,11 +140,6 @@ json_test_set_test_options(test-unicode4 TEST_PROPERTIES TIMEOUT 3000)
 # only the #972 regression test needs thirdparty/fifo_map on its include path
 json_test_set_test_options(test-regression1 LINK_LIBRARIES fifo_map_include)
 
-# GCC's false -Warray-bounds error with JSON_DIAGNOSTICS only shows up when optimizing (#5742)
-json_test_set_test_options(test-diagnostics-optimized
-    COMPILE_OPTIONS $<$<CXX_COMPILER_ID:GNU>:-O3 -Werror=array-bounds>
-)
-
 #############################################################################
 # add unit tests
 #############################################################################

tests/src/unit-allocator.cpp

@@ -12,11 +12,6 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
-#include <valarray>
-#if JSON_HAS_RANGES
-    #include <ranges>
-#endif
-
 namespace
 {
 // special test case to check if memory is leaked if constructor throws
@@ -607,173 +602,3 @@ TEST_CASE("bad my_allocator::construct")
         j["test"].push_back("should not leak");
     }
 }
-
-// the no-exceptions CI job skips every CHECK_THROWS_AS, which would leave
-// next_construct_fails set for the next allocation outside a check
-#if !defined(JSON_NOEXCEPTION)
-TEST_CASE("a failed allocation leaves the value unchanged")
-{
-    // create JSON type using the throwing allocator
-    using my_json = nlohmann::basic_json<std::map,
-          std::vector,
-          std::string,
-          bool,
-          std::int64_t,
-          std::uint64_t,
-          double,
-          my_allocator>;
-
-    // Each of these creates a string, array, object, or binary value. The
-    // value must be created before the type is changed: otherwise, a failed
-    // creation left a value of the new type without anything behind it (an
-    // assertion in its destructor, a null pointer everywhere else) or, when
-    // an old value was destroyed first, with a pointer to that destroyed one.
-
-    SECTION("creating a binary value")
-    {
-        const std::vector<std::uint8_t> bytes = {1, 2, 3};
-        my_json _;
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(_ = my_json::binary(bytes), std::bad_alloc&);
-        next_construct_fails = true;
-        CHECK_THROWS_AS(_ = my_json::binary(bytes, 42), std::bad_alloc&);
-        next_construct_fails = true;
-        CHECK_THROWS_AS(_ = my_json::binary(std::vector<std::uint8_t>(bytes)), std::bad_alloc&);
-        next_construct_fails = true;
-        CHECK_THROWS_AS(_ = my_json::binary(std::vector<std::uint8_t>(bytes), 42), std::bad_alloc&);
-        next_construct_fails = false;
-    }
-
-    SECTION("turning a null value into an array or object")
-    {
-        my_json j;
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j[0], std::bad_alloc&);
-        CHECK(j.is_null());
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j["key"], std::bad_alloc&);
-        CHECK(j.is_null());
-
-#ifdef JSON_HAS_CPP_17
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j[std::string_view("key")], std::bad_alloc&);
-        CHECK(j.is_null());
-#endif
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j.push_back(my_json(1)), std::bad_alloc&);
-        CHECK(j.is_null());
-
-        const my_json one = 1;
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j.push_back(one), std::bad_alloc&);
-        CHECK(j.is_null());
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j.push_back(my_json::object_t::value_type("key", 1)), std::bad_alloc&);
-        CHECK(j.is_null());
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j.emplace_back(1), std::bad_alloc&);
-        CHECK(j.is_null());
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j.emplace("key", 1), std::bad_alloc&);
-        CHECK(j.is_null());
-
-        const my_json object = {{"key", 1}};
-        next_construct_fails = true;
-        CHECK_THROWS_AS(j.update(object), std::bad_alloc&);
-        CHECK(j.is_null());
-
-        next_construct_fails = false;
-    }
-
-    // With iterator debugging, VS 2015's containers construct a proxy with the
-    // allocator in constructors that cannot report its failure, so a failing
-    // allocator crashes this section there (SIGSEGV with VS 2015 Debug x86).
-#if !(defined(_MSC_VER) && _MSC_VER < 1910 && defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
-    SECTION("converting into an existing value")
-    {
-        // to_json replaces the value it is given; the old one must survive a
-        // failed creation of the new one
-        my_json j = "old";
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::string("new")), std::bad_alloc&);
-        CHECK(j == "old");
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::vector<int> {1, 2}), std::bad_alloc&);
-        CHECK(j == "old");
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::vector<bool> {true, false}), std::bad_alloc&);
-        CHECK(j == "old");
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::map<std::string, int> {{"a", 1}}), std::bad_alloc&);
-        CHECK(j == "old");
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, my_json::binary_t({1, 2})), std::bad_alloc&);
-        CHECK(j == "old");
-
-        // the overloads for lvalues of the value types, for the value types
-        // themselves, and for the remaining compatible types
-        const std::string string = "new";
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, string), std::bad_alloc&);
-        CHECK(j == "old");
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, "new"), std::bad_alloc&);
-        CHECK(j == "old");
-
-        // to_json only moves a binary value that it converted from another
-        // container type, which my_json's std::vector<std::uint8_t> is not
-        using binary_constructor = nlohmann::detail::external_constructor<nlohmann::detail::value_t::binary>;
-        next_construct_fails = true;
-        CHECK_THROWS_AS(binary_constructor::construct(j, my_json::binary_t({1, 2})), std::bad_alloc&);
-        CHECK(j == "old");
-
-        my_json::array_t array = {1, 2};
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, array), std::bad_alloc&);
-        CHECK(j == "old");
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::move(array)), std::bad_alloc&);
-        CHECK(j == "old");
-
-        my_json::object_t object = {{"a", 1}};
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, object), std::bad_alloc&);
-        CHECK(j == "old");
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::move(object)), std::bad_alloc&);
-        CHECK(j == "old");
-
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, std::valarray<int> {1, 2}), std::bad_alloc&);
-        CHECK(j == "old");
-
-#if JSON_HAS_RANGES && !defined(__MINGW32__)
-        const std::vector<int> numbers = {1, 2};
-        next_construct_fails = true;
-        CHECK_THROWS_AS(nlohmann::to_json(j, numbers | std::views::filter([](int /*unused*/)
-        {
-            return true;
-        })), std::bad_alloc&);
-        CHECK(j == "old");
-#endif
-
-        next_construct_fails = false;
-        nlohmann::to_json(j, std::vector<int> {1, 2});
-        CHECK(j == my_json({1, 2}));
-    }
-#endif
-}
-#endif

tests/src/unit-diagnostics-optimized.cpp

@@ -1,80 +0,0 @@
-//     __ _____ _____ _____
-//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
-// |  |  |__   |  |  | | | |  version 3.12.0
-// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
-//
-// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
-// SPDX-License-Identifier: MIT
-
-// Regression test for https://github.com/nlohmann/json/issues/5742: with
-// JSON_DIAGNOSTICS, GCC (12 to at least 16) reported a false -Warray-bounds
-// error in the inlined set_parents() at -O3. The type of a new string was set
-// before the string was allocated, so GCC had to assume that operator new
-// could change it again and checked the object branch of set_parents()
-// against the string's allocation. Setting the type after creating the value
-// avoids this. The warning depends on GCC's inlining decisions, so the
-// sections cover two patterns that trigger it on different GCC versions
-// (#4819 and #5742).
-// On GCC, this file is compiled with -O3 -Werror=array-bounds (see
-// tests/CMakeLists.txt), so the test fails to build if the warning returns.
-
-#include "doctest_compatibility.h"
-
-#ifdef JSON_DIAGNOSTICS
-    #undef JSON_DIAGNOSTICS
-#endif
-
-#define JSON_DIAGNOSTICS 1
-
-#include <nlohmann/json.hpp>
-using nlohmann::json;
-
-#include <algorithm>
-#include <iterator>
-#include <utility>
-#include <vector>
-
-namespace
-{
-enum class diag_color
-{
-    red,
-    green,
-    blue
-};
-
-void to_json(json& j, const diag_color& c)
-{
-    static const std::pair<diag_color, json> m[] = // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
-    {
-        {diag_color::red, "r"},
-        {diag_color::green, "g"},
-        {diag_color::blue, "b"},
-    };
-    const auto* it = std::find_if(std::begin(m), std::end(m), [c](const std::pair<diag_color, json>& p)
-    {
-        return p.first == c;
-    });
-    j = it->second;
-}
-} // namespace
-
-TEST_CASE("diagnostics with optimization")
-{
-    SECTION("issue #4819 - object in vector")
-    {
-        std::vector<json> jsons{};
-        jsons.emplace_back(json({{"key", "value"}}));
-        CHECK(jsons.back()["key"] == "value");
-    }
-
-    SECTION("issue #5742 - string values from a static table")
-    {
-        json j = json::array();
-        j.push_back(diag_color::red);
-        j.push_back(diag_color::green);
-        j.push_back(diag_color::blue);
-        CHECK(j.dump() == R"(["r","g","b"])");
-        CHECK_THROWS_WITH_AS(j[1].get<int>(), "[json.exception.type_error.302] (/1) type must be number, but is string", json::type_error);
-    }
-}

tests/src/unit-regression2.cpp

@@ -808,6 +808,15 @@ TEST_CASE("regression tests 2")
         CHECK(j == k);
     }
 
+    SECTION("issue #4552 - UTF-8 invalid characters are not always ignored when dumping with error_handler_t::ignore")
+    {
+        json node;
+        node["test"] = "test\334\005";
+        CHECK(node.dump(-1, ' ', false, json::error_handler_t::ignore) == "{\"test\":\"test\\u0005\"}");
+        CHECK(node.dump(-1, ' ', false, json::error_handler_t::keep) == "{\"test\":\"test\334\\u0005\"}");
+        CHECK(node.dump(-1, ' ', true, json::error_handler_t::keep) == "{\"test\":\"test\334\\u0005\"}");
+    }
+
 #ifdef JSON_HAS_CPP_17
     SECTION("issue #5066 - MSVC converts json to std::variant<json> via the conversion operator")
     {

tests/src/unit-serialization.cpp

@@ -92,6 +92,8 @@ TEST_CASE("serialization")
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"äü\"");
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"ä\xEF\xBF\xBDü\"");
             CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"\\u00e4\\ufffd\\u00fc\"");
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"ä\xA9ü\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"\\u00e4\xA9\\u00fc\"");
         }
 
         SECTION("invalid character (regression guard for shared UTF-8 decoder, see #5529)")
@@ -114,6 +116,8 @@ TEST_CASE("serialization")
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123\"");
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\"");
             CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd\"");
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"123\xC2\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"123\xC2\"");
         }
 
         SECTION("unexpected character")
@@ -126,6 +130,39 @@ TEST_CASE("serialization")
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123456\"");
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\x34\x35\x36\"");
             CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd456\"");
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"123\xF1\xB0\x34\x35\x36\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"123\xF1\xB0\x34\x35\x36\"");
+        }
+
+        SECTION("keep: valid characters are still escaped")
+        {
+            // an invalid byte followed by characters that must be escaped
+            const json j = "\xC2\"\\\n\xFF\x05";
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"\xC2\\\"\\\\\\n\xFF\\u0005\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"\xC2\\\"\\\\\\n\xFF\\u0005\"");
+        }
+
+        SECTION("keep: truncated multibyte sequences")
+        {
+            CHECK(json("\xF0\x9F\x98").dump(-1, ' ', false, json::error_handler_t::keep) == "\"\xF0\x9F\x98\"");
+            CHECK(json("\xF0\x9F\x98").dump(-1, ' ', true, json::error_handler_t::keep) == "\"\xF0\x9F\x98\"");
+            CHECK(json("\xF0\x9F\x98" "a").dump(-1, ' ', false, json::error_handler_t::keep) == "\"\xF0\x9F\x98" "a\"");
+            CHECK(json("\xF0\x9F\x98" "a").dump(-1, ' ', true, json::error_handler_t::keep) == "\"\xF0\x9F\x98" "a\"");
+        }
+
+        SECTION("keep: long string with many invalid bytes")
+        {
+            // exceeds the internal string buffer several times
+            std::string input;
+            std::string expected = "\"";
+            for (int i = 0; i < 2000; ++i)
+            {
+                input += "\xFF\xE2\x82\n\xC3\xA4";
+                expected += "\xFF\xE2\x82\\n\xC3\xA4";
+            }
+            expected += "\"";
+            const json j = input;
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == expected);
         }
 
         SECTION("U+FFFD Substitution of Maximal Subparts")

tests/src/unit-unicode2.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tests/src/unit-unicode3.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tests/src/unit-unicode4.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tests/src/unit-unicode5.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);