Merge branch 'develop' into issue4552-ignore

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

README.md

@@ -363,7 +363,7 @@ std::cout << j_string << " == " << serialized_string << std::endl;
 
 [`.dump()`](https://json.nlohmann.me/api/basic_json/dump/) returns the originally stored string value.
 
-Note the library only supports UTF-8. When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
+Note the library only supports UTF-8. When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace`, `json::error_handler_t::ignore`, or `json::error_handler_t::keep` are used as error handlers.
 
 #### To/from streams (e.g., files, string streams)
 
@@ -1915,7 +1915,7 @@ The library supports **Unicode input** as follows:
 - [Unicode noncharacters](https://www.unicode.org/faq/private_use.html#nonchar1) will not be replaced by the library.
 - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
 - The strings stored in the library are UTF-8 encoded. When using the default string type (`std::string`), note that its length/size functions return the number of stored bytes rather than the number of characters or glyphs.
-- When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
+- When you store strings with different encodings in the library, calling [`dump()`](https://json.nlohmann.me/api/basic_json/dump/) may throw an exception unless `json::error_handler_t::replace`, `json::error_handler_t::ignore`, or `json::error_handler_t::keep` are used as error handlers.
 - To store wide strings (e.g., `std::wstring`), you need to convert them to a UTF-8 encoded `std::string` before, see [an example](https://json.nlohmann.me/home/faq/#wide-string-handling).
 
 ### Comments in JSON

docs/mkdocs/docs/api/basic_json/dump.md

@@ -25,10 +25,15 @@ and `ensure_ascii` parameters.
     result consists of ASCII characters only.
 
 `error_handler` (in)
-:   how to react on decoding errors; there are three possible values (see [`error_handler_t`](error_handler_t.md):
-    `strict` (throws an exception in case a decoding error occurs; default), `replace` (replace invalid UTF-8 sequences
-    with U+FFFD), and `ignore` (ignore invalid UTF-8 sequences during serialization; all valid bytes are copied to the
-    output unchanged, and invalid bytes are dropped)).
+:   how to react on decoding errors; there are four possible values (see [`error_handler_t`](error_handler_t.md)):
+
+    - `strict`: throw a [`type_error`](../../home/exceptions.md#type-errors) exception in case a decoding error occurs
+      (default),
+    - `replace`: replace invalid UTF-8 sequences with U+FFFD (� REPLACEMENT CHARACTER),
+    - `ignore`: ignore invalid UTF-8 sequences during serialization; all valid bytes are copied to the output unchanged,
+      and invalid bytes are dropped, and
+    - `keep`: keep invalid UTF-8 sequences during serialization; all bytes are copied to the output unchanged, so the
+      result is not valid UTF-8.
     
 ## Return value
 
@@ -94,3 +99,4 @@ Binary values are serialized as an object containing two keys:
 - Indentation character `indent_char`, option `ensure_ascii` and exceptions added in version 3.0.0.
 - Error handlers added in version 3.4.0.
 - Serialization of binary values added in version 3.8.0.
+- Error handler value `keep` added in version 3.13.0.

docs/mkdocs/docs/api/basic_json/error_handler_t.md

@@ -4,12 +4,13 @@
 enum class error_handler_t {
     strict,
     replace,
-    ignore
+    ignore,
+    keep
 };
 ```
 
 This enumeration is used in the [`dump`](dump.md) function to choose how to treat decoding errors while serializing a
-`basic_json` value. Three values are differentiated:
+`basic_json` value. Four values are differentiated:
 
 strict
 :   throw a `type_error` exception in case of invalid UTF-8
@@ -20,6 +21,12 @@ replace
 ignore
 :   ignore invalid UTF-8 sequences; all valid bytes are copied to the output unchanged, and invalid bytes are dropped
 
+keep
+:   keep invalid UTF-8 sequences; all bytes are copied to the output unchanged. Valid characters are still escaped as
+    usual (e.g., `"`, `\\`, and control characters), so the result has valid JSON syntax, but it is not valid UTF-8.
+    In particular, [`parse`](parse.md) rejects it, and with `ensure_ascii` set to `true`, the invalid bytes are the
+    only non-ASCII bytes of the output.
+
 ## Examples
 
 ??? example
@@ -40,3 +47,4 @@ ignore
 ## Version history
 
 - Added in version 3.4.0.
+- Added value `keep` in version 3.13.0.

docs/mkdocs/docs/api/macros/index.md

@@ -53,8 +53,6 @@ header. See also the [macro overview page](../../features/macros.md).
 - [**JSON_BRACE_INIT_COPY_SEMANTICS**](json_brace_init_copy_semantics.md) - opt in to copy/move semantics for single-element brace initialization
 - [**JSON_DISABLE_ENUM_SERIALIZATION**](json_disable_enum_serialization.md) - switch off default serialization/deserialization functions for enums
 - [**JSON_USE_IMPLICIT_CONVERSIONS**](json_use_implicit_conversions.md) - control implicit conversions
-- [**JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS**](json_use_objects_for_enum_keyed_maps.md) - opt in to storing maps with enum
-  keys as objects
 
 ## Comparison behavior
 

docs/mkdocs/docs/api/macros/json_use_objects_for_enum_keyed_maps.md

@@ -1,139 +0,0 @@
-# JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-
-```cpp
-#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS /* value */
-```
-
-When defined to `1`, maps whose keys are enums (such as `std::map<E, T>` or `std::unordered_map<E, T>`) are stored as
-JSON objects, using the enum's own conversion for the keys. By default, they are stored as arrays of `[key, value]`
-pairs.
-
-## Default definition
-
-The default value is `0` (disabled — existing behavior is preserved).
-
-```cpp
-#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
-```
-
-## Notes
-
-!!! note "Background"
-
-    JSON object keys are strings, so a map is only stored as an object if its keys can be converted to a string type.
-    Enums are not, even if [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md) maps them to strings, so a
-    map with enum keys becomes an array of `[key, value]` pairs:
-
-    ```json
-    [["stopped", "aa"], ["completed", "bb"]]
-    ```
-
-    With this macro, the same map becomes an object
-    (see [#4378](https://github.com/nlohmann/json/issues/4378)):
-
-    ```json
-    {"completed": "bb", "stopped": "aa"}
-    ```
-
-!!! note "Maps with non-unique keys"
-
-    Maps that allow duplicate keys, such as `std::multimap<E, T>` or `std::unordered_multimap<E, T>`, are not affected
-    by the macro and are still stored as arrays of `[key, value]` pairs, as an object cannot hold duplicate keys.
-
-!!! note "Reading"
-
-    Reading is not affected by the macro: a map with enum keys can always be read from both an array of pairs and an
-    object. For the latter, each key is converted to the enum with its `from_json` function, e.g., the one defined by
-    [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md). Data written without the macro can therefore
-    still be read after enabling it.
-
-!!! warning "Keys must serialize to distinct strings"
-
-    Each key is converted with the enum's `to_json` function. If a key is not converted to a string (for instance, an
-    enum without [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md), which is stored as an integer, or an
-    enumerator mapped to `nullptr`), [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) is thrown.
-    If two keys are converted to the same string (for instance, because
-    [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md) maps an unlisted enumerator to the first entry),
-    [`type_error.318`](../../home/exceptions.md#jsonexceptiontype_error318) is thrown. In both cases, the target value
-    is not changed.
-
-!!! warning "Opt-in only"
-
-    This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no effect.
-
-!!! note "ABI compatibility"
-
-    The value of this macro is encoded in the [namespace](../../features/namespace.md) (tag `_ekmo`), resulting in
-    distinct symbol names. Translation units compiled with and without it can therefore be linked into the same program
-    without One Definition Rule (ODR) violations, but they cannot exchange instances of library types.
-
-## Examples
-
-??? example "Default behavior (macro not defined)"
-
-    Without the macro, a map with enum keys is stored as an array of pairs:
-
-    ```cpp
-    #include <map>
-    #include <nlohmann/json.hpp>
-
-    using json = nlohmann::json;
-
-    enum TaskState { TS_STOPPED, TS_RUNNING, TS_COMPLETED };
-
-    NLOHMANN_JSON_SERIALIZE_ENUM(TaskState, {
-        {TS_STOPPED, "stopped"},
-        {TS_RUNNING, "running"},
-        {TS_COMPLETED, "completed"},
-    })
-
-    int main()
-    {
-        std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
-
-        json j = m;
-        // j is [["stopped","aa"],["completed","bb"]]
-    }
-    ```
-
-??? example "Objects for enum-keyed maps (macro defined to 1)"
-
-    With the macro, the same map is stored as an object:
-
-    ```cpp
-    #define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 1
-    #include <map>
-    #include <nlohmann/json.hpp>
-
-    using json = nlohmann::json;
-
-    enum TaskState { TS_STOPPED, TS_RUNNING, TS_COMPLETED };
-
-    NLOHMANN_JSON_SERIALIZE_ENUM(TaskState, {
-        {TS_STOPPED, "stopped"},
-        {TS_RUNNING, "running"},
-        {TS_COMPLETED, "completed"},
-    })
-
-    int main()
-    {
-        std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
-
-        json j = m;
-        // j is {"completed":"bb","stopped":"aa"}
-
-        auto m2 = j.get<std::map<TaskState, std::string>>();
-        // m2 == m
-    }
-    ```
-
-## See also
-
-- [Specializing enum conversion](../../features/enum_conversion.md)
-- [**NLOHMANN_JSON_SERIALIZE_ENUM**](nlohmann_json_serialize_enum.md) - serialize/deserialize an enum
-- [**NLOHMANN_JSON_SERIALIZE_ENUM_STRICT**](nlohmann_json_serialize_enum_strict.md) - serialize/deserialize an enum with
-  exceptions
-
-## Version history
-
-- Added in version 3.13.0.

docs/mkdocs/docs/api/macros/nlohmann_json_serialize_enum.md

@@ -41,9 +41,6 @@ inline void from_json(const BasicJsonType& j, type& e);
       conversion. Select this default pair carefully. See example 1 below.
     - If an enum or JSON value is specified in multiple conversions, the first matching conversion from the top of the
       list will be returned when converting to or from JSON. See example 2 below.
-    - Maps with enum keys (e.g., `std::map<ENUM_TYPE, T>`) are stored as arrays of `[key, value]` pairs by default.
-      Define [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md) to store them as objects
-      with the converted keys. Such maps can be read from both forms.
 
 ## Examples
 
@@ -83,7 +80,6 @@ inline void from_json(const BasicJsonType& j, type& e);
 - [Specializing enum conversion](../../features/enum_conversion.md)
 - [`NLOHMANN_JSON_SERIALIZE_ENUM_STRICT`](./nlohmann_json_serialize_enum_strict.md)
 - [`JSON_DISABLE_ENUM_SERIALIZATION`](json_disable_enum_serialization.md)
-- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md)
 
 ## Version history
 

docs/mkdocs/docs/api/macros/nlohmann_json_serialize_enum_strict.md

@@ -44,9 +44,6 @@ inline void from_json(const BasicJsonType& j, type& e);
       `"enum value out of range for <type>"`.
     - If an enum or JSON value is specified in multiple conversions, the first matching conversion from the top of the
       list will be returned when converting to or from JSON. See example 2 below.
-    - Maps with enum keys (e.g., `std::map<ENUM_TYPE, T>`) are stored as arrays of `[key, value]` pairs by default.
-      Define [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md) to store them as objects
-      with the converted keys. Such maps can be read from both forms.
 
 ## Examples
 
@@ -102,7 +99,6 @@ inline void from_json(const BasicJsonType& j, type& e);
 - [Specializing enum conversion](../../features/enum_conversion.md)
 - [`NLOHMANN_JSON_SERIALIZE_ENUM`](./nlohmann_json_serialize_enum.md)
 - [`JSON_DISABLE_ENUM_SERIALIZATION`](json_disable_enum_serialization.md)
-- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md)
 
 ## Version history
 

docs/mkdocs/docs/examples/error_handler_t.cpp

@@ -1,3 +1,4 @@
+#include <iomanip>
 #include <iostream>
 #include <nlohmann/json.hpp>
 
@@ -21,4 +22,12 @@ int main()
               << "\nstring with ignored invalid characters: "
               << j_invalid.dump(-1, ' ', false, json::error_handler_t::ignore)
               << '\n';
+
+    // the invalid byte is kept; print the result byte-wise to make it visible
+    std::cout << "string with kept invalid characters:";
+    for (const unsigned char c : j_invalid.dump(-1, ' ', false, json::error_handler_t::keep))
+    {
+        std::cout << ' ' << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(c);
+    }
+    std::cout << '\n';
 }

docs/mkdocs/docs/examples/error_handler_t.output

@@ -1,3 +1,4 @@
 [json.exception.type_error.316] invalid UTF-8 byte at index 2: 0xA9
 string with replaced invalid characters: "ä�ü"
 string with ignored invalid characters: "äü"
+string with kept invalid characters: 22 c3 a4 a9 c3 bc 22

docs/mkdocs/docs/features/enum_conversion.md

@@ -43,23 +43,6 @@ json jPi = 3.14;
 assert(jPi.get<TaskState>() == TS_INVALID );
 ```
 
-## Maps with enum keys
-
-By default, maps with enum keys, such as `std::map<TaskState, std::string>`, are stored as arrays of `[key, value]`
-pairs, because JSON object keys must be strings. Define
-[`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) before including the
-library to store them as objects, with the keys converted by the enum's `to_json()` function:
-
-```cpp
-std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
-
-json j = m;
-// default:                                   [["stopped","aa"],["completed","bb"]]
-// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS: {"completed":"bb","stopped":"aa"}
-```
-
-Either form can be read back, with or without the macro.
-
 ## Notes
 
 Just as in [Arbitrary Type Conversions](arbitrary_types.md) above,

docs/mkdocs/docs/features/macros.md

@@ -167,13 +167,6 @@ behavior is deprecated and switched off (`0`) by default.
 
 See [full documentation of `JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md).
 
-## `JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`
-
-When defined to `1`, maps with enum keys (e.g., `std::map<E, T>`) are stored as objects, using the enum's conversion for
-the keys, instead of arrays of `[key, value]` pairs. It is switched off (`0`) by default.
-
-See [full documentation of `JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md).
-
 ## `JSON_USE_SIMDUTF`
 
 When defined, UTF-8 validation of JSON strings read from contiguous byte input is delegated to the

docs/mkdocs/docs/features/namespace.md

@@ -20,8 +20,6 @@ The complete default namespace name is derived as follows:
       `_bics`.
     - [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) defined non-zero appends `_psp`.
     - [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) defined non-zero appends `_snul`.
-    - [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) defined non-zero
-      appends `_ekmo`.
 - The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
   underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
   below.

docs/mkdocs/docs/features/serialization.md

@@ -64,6 +64,7 @@ serialization fails by default. The fourth argument of `dump` selects an
 - `strict` (default) — throw a [`type_error.316`](../home/exceptions.md#jsonexceptiontype_error316) exception.
 - `replace` — replace invalid bytes with the Unicode replacement character U+FFFD (`�`).
 - `ignore` — silently drop invalid bytes.
+- `keep` — copy invalid bytes to the output unchanged; the result is not valid UTF-8.
 
 ??? example
 

docs/mkdocs/docs/home/exceptions.md

@@ -589,9 +589,6 @@ During implicit or explicit value conversion, the JSON type must be compatible w
     [json.exception.type_error.302] type must be string, but is object
     ```
 
-This exception is also thrown with [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md)
-if a key of a map with enum keys is not converted to a string, for instance, because the enum is stored as an integer.
-
 ### json.exception.type_error.303
 
 To retrieve a reference to a value stored in a `basic_json` object with `get_ref`, the type of the reference must match the value type. For instance, for a JSON array, the `ReferenceType` must be `array_t &`.
@@ -758,6 +755,7 @@ The `dump()` function only works with UTF-8 encoded strings; that is, if you ass
     - Pass an error handler as last parameter to the `dump()` function to avoid this exception:
         - `json::error_handler_t::replace` will replace invalid bytes sequences with `U+FFFD` 
         - `json::error_handler_t::ignore` will silently ignore invalid byte sequences
+        - `json::error_handler_t::keep` will copy invalid byte sequences to the output unchanged
 
 ### json.exception.type_error.317
 
@@ -778,19 +776,6 @@ The dynamic type of the object cannot be represented in the requested serializat
 
     Encapsulate the JSON value in an object. That is, instead of serializing `#!json true`, serialize `#!json {"value": true}`
 
-### json.exception.type_error.318
-
-With [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md), a map with enum
-keys is stored as an object. This exception is thrown if two of its keys are converted to the same string, so one of the
-entries would be lost. This happens, for instance, if [`NLOHMANN_JSON_SERIALIZE_ENUM`](../api/macros/nlohmann_json_serialize_enum.md)
-does not list an enumerator and it is therefore converted like the first listed one.
-
-!!! failure "Example message"
-
-    ```
-    [json.exception.type_error.318] duplicate object key 'red'
-    ```
-
 ## Out of range
 
 This exception is thrown in case a library function is called on an input parameter that exceeds the expected range, for instance, in the case of array indices or nonexisting object keys.

docs/mkdocs/docs/home/faq.md

@@ -85,7 +85,7 @@ The library supports **Unicode input** as follows:
 - The library will not replace [Unicode noncharacters](http://www.unicode.org/faq/private_use.html#nonchar1).
 - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
 - The strings stored in the library are UTF-8 encoded. When using the default string type (`std::string`), note that its length/size functions return the number of stored bytes rather than the number of characters or glyphs.
-- When you store strings with different encodings in the library, calling [`dump()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a50ec80b02d0f3f51130d4abb5d1cfdc5.html#a50ec80b02d0f3f51130d4abb5d1cfdc5) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
+- When you store strings with different encodings in the library, calling [`dump()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a50ec80b02d0f3f51130d4abb5d1cfdc5.html#a50ec80b02d0f3f51130d4abb5d1cfdc5) may throw an exception unless `json::error_handler_t::replace`, `json::error_handler_t::ignore`, or `json::error_handler_t::keep` are used as error handlers.
 
 In most cases, the parser is right to complain, because the input is not UTF-8 encoded. This is especially true for Microsoft Windows, where Latin-1 or ISO 8859-1 is often the standard encoding.
 

docs/mkdocs/mkdocs.yml

@@ -303,7 +303,6 @@ nav:
           - 'JSON_USE_GLOBAL_UDLS': api/macros/json_use_global_udls.md
           - 'JSON_USE_IMPLICIT_CONVERSIONS': api/macros/json_use_implicit_conversions.md
           - 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
-          - 'JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS': api/macros/json_use_objects_for_enum_keyed_maps.md
           - 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
           - 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
           - 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md

include/nlohmann/detail/abi_macros.hpp

@@ -46,10 +46,6 @@
     #define JSON_STRICT_NUL_HANDLING 0
 #endif
 
-#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
-#endif
-
 #if JSON_DIAGNOSTICS
     #define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
 #else
@@ -86,20 +82,14 @@
     #define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
 #endif
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
-#else
-    #define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
-#endif
-
 #ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
     #define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
 #endif
 
 // Construct the namespace ABI tags component
-#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
-#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
-    NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
+#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f) json_abi ## a ## b ## c ## d ## e ## f
+#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f) \
+    NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f)
 
 #define NLOHMANN_JSON_ABI_TAGS                                       \
     NLOHMANN_JSON_ABI_TAGS_CONCAT(                                   \
@@ -108,8 +98,7 @@
             NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS,              \
             NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS,         \
             NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION,           \
-            NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING,               \
-            NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
+            NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING)
 
 // Construct the namespace version component
 #define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \

include/nlohmann/detail/conversions/from_json.hpp

@@ -550,40 +550,11 @@ auto from_json(BasicJsonType&& j, TupleRelated&& t)
     return from_json_tuple_impl(std::forward<BasicJsonType>(j), std::forward<TupleRelated>(t), priority_tag<3> {});
 }
 
-// read a map with enum keys from an object, using the enum's own from_json for
-// the keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); this is the form written
-// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-template<typename BasicJsonType, typename Map>
-inline bool from_json_enum_keyed_object(const BasicJsonType& j, Map& m, std::true_type /*key is enum*/)
-{
-    if (!j.is_object())
-    {
-        return false;
-    }
-    m.clear();
-    for (const auto& p : *j.template get_ptr<const typename BasicJsonType::object_t*>())
-    {
-        m.emplace(BasicJsonType(p.first).template get<typename Map::key_type>(), p.second.template get<typename Map::mapped_type>());
-    }
-    return true;
-}
-
-template<typename BasicJsonType, typename Map>
-inline bool from_json_enum_keyed_object(const BasicJsonType& /*j*/, Map& /*m*/, std::false_type /*key is enum*/)
-{
-    return false;
-}
-
 template < typename BasicJsonType, typename Key, typename Value, typename Compare, typename Allocator,
            typename = enable_if_t < !std::is_constructible <
                                         typename BasicJsonType::string_t, Key >::value >>
 inline void from_json(const BasicJsonType& j, std::map<Key, Value, Compare, Allocator>& m)
 {
-    // NOLINTNEXTLINE(modernize-type-traits) we use C++11
-    if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
-    {
-        return;
-    }
     if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
     {
         JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
@@ -604,11 +575,6 @@ template < typename BasicJsonType, typename Key, typename Value, typename Hash,
                                         typename BasicJsonType::string_t, Key >::value >>
 inline void from_json(const BasicJsonType& j, std::unordered_map<Key, Value, Hash, KeyEqual, Allocator>& m)
 {
-    // NOLINTNEXTLINE(modernize-type-traits) we use C++11
-    if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
-    {
-        return;
-    }
     if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
     {
         JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));

include/nlohmann/detail/conversions/to_json.hpp

@@ -23,7 +23,6 @@
 #include <valarray> // valarray
 #include <vector> // vector
 
-#include <nlohmann/detail/exceptions.hpp>
 #include <nlohmann/detail/iterators/iteration_proxy.hpp>
 #include <nlohmann/detail/meta/cpp_future.hpp>
 #include <nlohmann/detail/meta/std_fs.hpp>
@@ -382,9 +381,6 @@ template < typename BasicJsonType, typename CompatibleArrayType,
                          !is_basic_json<CompatibleArrayType>::value
 #if JSON_HAS_RANGES && !defined(__MINGW32__)
     && !is_compatible_range_view<CompatibleArrayType>::value
-#endif
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    && !is_enum_keyed_map<CompatibleArrayType>::value
 #endif
                          ,
                          int > = 0 >
@@ -439,33 +435,6 @@ inline void to_json(BasicJsonType& j, const CompatibleObjectType& obj)
     external_constructor<value_t::object>::construct(j, obj);
 }
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-// store a map with enum keys as an object, using the enum's own to_json for the
-// keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); without the macro, such maps
-// are stored as arrays of [key, value] pairs
-template < typename BasicJsonType, typename EnumKeyedMap,
-           enable_if_t < is_enum_keyed_map<EnumKeyedMap>::value&& !is_basic_json<EnumKeyedMap>::value, int > = 0 >
-inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
-{
-    typename BasicJsonType::object_t obj;
-    for (const auto& p : map)
-    {
-        BasicJsonType key = p.first;
-        if (JSON_HEDLEY_UNLIKELY(!key.is_string()))
-        {
-            JSON_THROW(type_error::create(302, concat("type must be string, but is ", key.type_name()), &key));
-        }
-
-        auto& key_string = *key.template get_ptr<typename BasicJsonType::string_t*>();
-        if (JSON_HEDLEY_UNLIKELY(!obj.emplace(key_string, BasicJsonType(p.second)).second))
-        {
-            JSON_THROW(type_error::create(318, concat("duplicate object key '", key_string, "'"), &key));
-        }
-    }
-    external_constructor<value_t::object>::construct(j, std::move(obj));
-}
-#endif
-
 template<typename BasicJsonType>
 inline void to_json(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
 {

include/nlohmann/detail/macro_unscope.hpp

@@ -46,7 +46,6 @@
     #undef JSON_BRACE_INIT_COPY_SEMANTICS
     #undef JSON_PRECISE_STREAM_POSITION
     #undef JSON_STRICT_NUL_HANDLING
-    #undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
 #endif
 
 #include <nlohmann/thirdparty/hedley/hedley_undef.hpp>

include/nlohmann/detail/meta/type_traits.hpp

@@ -400,30 +400,6 @@ template<typename BasicJsonType, typename CompatibleObjectType>
 struct is_compatible_object_type
     : is_compatible_object_type_impl<BasicJsonType, CompatibleObjectType> {};
 
-template<typename T>
-using insert_result_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()));
-
-template<typename T>
-using insert_result_second_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()).second);
-
-// a map-like type (std::map, std::unordered_map, ...) whose keys are enums; see
-// JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-template<typename T, typename = void>
-struct is_enum_keyed_map : std::false_type {};
-
-template<typename T>
-struct is_enum_keyed_map <
-    T, enable_if_t < is_detected<mapped_type_t, T>::value&&
-    is_detected<key_type_t, T>::value >>
-{
-    // maps with non-unique keys (std::multimap, std::unordered_multimap, ...)
-    // are excluded, because an object cannot hold duplicate keys; they are
-    // detected by insert() returning an iterator instead of a pair<iterator, bool>
-    // NOLINTNEXTLINE(modernize-type-traits) we use C++11
-    static constexpr bool value = std::is_enum<typename T::key_type>::value &&
-                                  !(is_detected<insert_result_t, T>::value && !is_detected<insert_result_second_t, T>::value);
-};
-
 template<typename BasicJsonType, typename ConstructibleObjectType,
          typename = void>
 struct is_constructible_object_type_impl : std::false_type {};

include/nlohmann/detail/output/serializer.hpp

@@ -48,7 +48,8 @@ enum class error_handler_t
 {
     strict,  ///< throw a type_error exception in case of invalid UTF-8
     replace, ///< replace invalid UTF-8 sequences with U+FFFD
-    ignore   ///< ignore invalid UTF-8 sequences
+    ignore,  ///< ignore invalid UTF-8 sequences
+    keep     ///< keep invalid UTF-8 sequences; their bytes are copied unchanged
 };
 
 template<typename BasicJsonType>
@@ -1019,6 +1020,47 @@ class serializer
                             break;
                         }
 
+                        case error_handler_t::keep:
+                        {
+                            // drop whatever the incomplete sequence left in
+                            // the buffer (only copied if !EnsureAscii) and copy
+                            // the ill-formed bytes from the input instead
+                            bytes = bytes_after_last_accept;
+
+                            if (undumped_chars > 0)
+                            {
+                                // the pending bytes of the incomplete sequence
+                                // are ill-formed; the current byte may be OK for
+                                // itself, so we would like to read it again
+                                for (std::size_t j = i - undumped_chars; j < i; ++j)
+                                {
+                                    string_buffer[bytes++] = s[j];
+                                }
+                                --i;
+                            }
+                            else
+                            {
+                                // the current byte cannot start any sequence
+                                string_buffer[bytes++] = s[i];
+                            }
+
+                            // write buffer and reset index; there must be 13 bytes
+                            // left, as this is the maximal number of bytes to be
+                            // written ("\uxxxx\uxxxx\0") for one code point
+                            if (string_buffer.size() - bytes < 13)
+                            {
+                                put_buffer(string_buffer, bytes);
+                                bytes = 0;
+                            }
+
+                            bytes_after_last_accept = bytes;
+                            undumped_chars = 0;
+
+                            // continue processing the string
+                            state = UTF8_ACCEPT;
+                            break;
+                        }
+
                         default:            // LCOV_EXCL_LINE
                             JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
                     }
@@ -1064,6 +1106,15 @@ class serializer
                     break;
                 }
 
+                case error_handler_t::keep:
+                {
+                    // write all accepted bytes
+                    put_buffer(string_buffer, bytes_after_last_accept);
+                    // copy the bytes of the incomplete sequence unchanged
+                    put_string(s, s.size() - undumped_chars, s.size());
+                    break;
+                }
+
                 case error_handler_t::replace:
                 {
                     // write all accepted bytes

single_include/nlohmann/json.hpp

@@ -103,10 +103,6 @@
     #define JSON_STRICT_NUL_HANDLING 0
 #endif
 
-#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
-#endif
-
 #if JSON_DIAGNOSTICS
     #define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
 #else
@@ -143,20 +139,14 @@
     #define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
 #endif
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
-#else
-    #define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
-#endif
-
 #ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
     #define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
 #endif
 
 // Construct the namespace ABI tags component
-#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
-#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
-    NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
+#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f) json_abi ## a ## b ## c ## d ## e ## f
+#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f) \
+    NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f)
 
 #define NLOHMANN_JSON_ABI_TAGS                                       \
     NLOHMANN_JSON_ABI_TAGS_CONCAT(                                   \
@@ -165,8 +155,7 @@
             NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS,              \
             NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS,         \
             NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION,           \
-            NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING,               \
-            NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
+            NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING)
 
 // Construct the namespace version component
 #define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
@@ -4421,30 +4410,6 @@ template<typename BasicJsonType, typename CompatibleObjectType>
 struct is_compatible_object_type
     : is_compatible_object_type_impl<BasicJsonType, CompatibleObjectType> {};
 
-template<typename T>
-using insert_result_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()));
-
-template<typename T>
-using insert_result_second_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()).second);
-
-// a map-like type (std::map, std::unordered_map, ...) whose keys are enums; see
-// JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-template<typename T, typename = void>
-struct is_enum_keyed_map : std::false_type {};
-
-template<typename T>
-struct is_enum_keyed_map <
-    T, enable_if_t < is_detected<mapped_type_t, T>::value&&
-    is_detected<key_type_t, T>::value >>
-{
-    // maps with non-unique keys (std::multimap, std::unordered_multimap, ...)
-    // are excluded, because an object cannot hold duplicate keys; they are
-    // detected by insert() returning an iterator instead of a pair<iterator, bool>
-    // NOLINTNEXTLINE(modernize-type-traits) we use C++11
-    static constexpr bool value = std::is_enum<typename T::key_type>::value &&
-                                  !(is_detected<insert_result_t, T>::value && !is_detected<insert_result_second_t, T>::value);
-};
-
 template<typename BasicJsonType, typename ConstructibleObjectType,
          typename = void>
 struct is_constructible_object_type_impl : std::false_type {};
@@ -6082,40 +6047,11 @@ auto from_json(BasicJsonType&& j, TupleRelated&& t)
     return from_json_tuple_impl(std::forward<BasicJsonType>(j), std::forward<TupleRelated>(t), priority_tag<3> {});
 }
 
-// read a map with enum keys from an object, using the enum's own from_json for
-// the keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); this is the form written
-// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-template<typename BasicJsonType, typename Map>
-inline bool from_json_enum_keyed_object(const BasicJsonType& j, Map& m, std::true_type /*key is enum*/)
-{
-    if (!j.is_object())
-    {
-        return false;
-    }
-    m.clear();
-    for (const auto& p : *j.template get_ptr<const typename BasicJsonType::object_t*>())
-    {
-        m.emplace(BasicJsonType(p.first).template get<typename Map::key_type>(), p.second.template get<typename Map::mapped_type>());
-    }
-    return true;
-}
-
-template<typename BasicJsonType, typename Map>
-inline bool from_json_enum_keyed_object(const BasicJsonType& /*j*/, Map& /*m*/, std::false_type /*key is enum*/)
-{
-    return false;
-}
-
 template < typename BasicJsonType, typename Key, typename Value, typename Compare, typename Allocator,
            typename = enable_if_t < !std::is_constructible <
                                         typename BasicJsonType::string_t, Key >::value >>
 inline void from_json(const BasicJsonType& j, std::map<Key, Value, Compare, Allocator>& m)
 {
-    // NOLINTNEXTLINE(modernize-type-traits) we use C++11
-    if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
-    {
-        return;
-    }
     if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
     {
         JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
@@ -6136,11 +6072,6 @@ template < typename BasicJsonType, typename Key, typename Value, typename Hash,
                                         typename BasicJsonType::string_t, Key >::value >>
 inline void from_json(const BasicJsonType& j, std::unordered_map<Key, Value, Hash, KeyEqual, Allocator>& m)
 {
-    // NOLINTNEXTLINE(modernize-type-traits) we use C++11
-    if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
-    {
-        return;
-    }
     if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
     {
         JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
@@ -6236,8 +6167,6 @@ NLOHMANN_JSON_NAMESPACE_END
 #include <valarray> // valarray
 #include <vector> // vector
 
-// #include <nlohmann/detail/exceptions.hpp>
-
 // #include <nlohmann/detail/iterators/iteration_proxy.hpp>
 //     __ _____ _____ _____
 //  __|  |   __|     |   | |  JSON for Modern C++
@@ -6981,9 +6910,6 @@ template < typename BasicJsonType, typename CompatibleArrayType,
                          !is_basic_json<CompatibleArrayType>::value
 #if JSON_HAS_RANGES && !defined(__MINGW32__)
     && !is_compatible_range_view<CompatibleArrayType>::value
-#endif
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    && !is_enum_keyed_map<CompatibleArrayType>::value
 #endif
                          ,
                          int > = 0 >
@@ -7038,33 +6964,6 @@ inline void to_json(BasicJsonType& j, const CompatibleObjectType& obj)
     external_constructor<value_t::object>::construct(j, obj);
 }
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-// store a map with enum keys as an object, using the enum's own to_json for the
-// keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); without the macro, such maps
-// are stored as arrays of [key, value] pairs
-template < typename BasicJsonType, typename EnumKeyedMap,
-           enable_if_t < is_enum_keyed_map<EnumKeyedMap>::value&& !is_basic_json<EnumKeyedMap>::value, int > = 0 >
-inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
-{
-    typename BasicJsonType::object_t obj;
-    for (const auto& p : map)
-    {
-        BasicJsonType key = p.first;
-        if (JSON_HEDLEY_UNLIKELY(!key.is_string()))
-        {
-            JSON_THROW(type_error::create(302, concat("type must be string, but is ", key.type_name()), &key));
-        }
-
-        auto& key_string = *key.template get_ptr<typename BasicJsonType::string_t*>();
-        if (JSON_HEDLEY_UNLIKELY(!obj.emplace(key_string, BasicJsonType(p.second)).second))
-        {
-            JSON_THROW(type_error::create(318, concat("duplicate object key '", key_string, "'"), &key));
-        }
-    }
-    external_constructor<value_t::object>::construct(j, std::move(obj));
-}
-#endif
-
 template<typename BasicJsonType>
 inline void to_json(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
 {
@@ -24032,7 +23931,8 @@ enum class error_handler_t
 {
     strict,  ///< throw a type_error exception in case of invalid UTF-8
     replace, ///< replace invalid UTF-8 sequences with U+FFFD
-    ignore   ///< ignore invalid UTF-8 sequences
+    ignore,  ///< ignore invalid UTF-8 sequences
+    keep     ///< keep invalid UTF-8 sequences; their bytes are copied unchanged
 };
 
 template<typename BasicJsonType>
@@ -25003,6 +24903,47 @@ class serializer
                             break;
                         }
 
+                        case error_handler_t::keep:
+                        {
+                            // drop whatever the incomplete sequence left in
+                            // the buffer (only copied if !EnsureAscii) and copy
+                            // the ill-formed bytes from the input instead
+                            bytes = bytes_after_last_accept;
+
+                            if (undumped_chars > 0)
+                            {
+                                // the pending bytes of the incomplete sequence
+                                // are ill-formed; the current byte may be OK for
+                                // itself, so we would like to read it again
+                                for (std::size_t j = i - undumped_chars; j < i; ++j)
+                                {
+                                    string_buffer[bytes++] = s[j];
+                                }
+                                --i;
+                            }
+                            else
+                            {
+                                // the current byte cannot start any sequence
+                                string_buffer[bytes++] = s[i];
+                            }
+
+                            // write buffer and reset index; there must be 13 bytes
+                            // left, as this is the maximal number of bytes to be
+                            // written ("\uxxxx\uxxxx\0") for one code point
+                            if (string_buffer.size() - bytes < 13)
+                            {
+                                put_buffer(string_buffer, bytes);
+                                bytes = 0;
+                            }
+
+                            bytes_after_last_accept = bytes;
+                            undumped_chars = 0;
+
+                            // continue processing the string
+                            state = UTF8_ACCEPT;
+                            break;
+                        }
+
                         default:            // LCOV_EXCL_LINE
                             JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
                     }
@@ -25048,6 +24989,15 @@ class serializer
                     break;
                 }
 
+                case error_handler_t::keep:
+                {
+                    // write all accepted bytes
+                    put_buffer(string_buffer, bytes_after_last_accept);
+                    // copy the bytes of the incomplete sequence unchanged
+                    put_string(s, s.size() - undumped_chars, s.size());
+                    break;
+                }
+
                 case error_handler_t::replace:
                 {
                     // write all accepted bytes
@@ -32650,7 +32600,6 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
     #undef JSON_BRACE_INIT_COPY_SEMANTICS
     #undef JSON_PRECISE_STREAM_POSITION
     #undef JSON_STRICT_NUL_HANDLING
-    #undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
 #endif
 
 // #include <nlohmann/thirdparty/hedley/hedley_undef.hpp>

single_include/nlohmann/json_fwd.hpp

@@ -64,10 +64,6 @@
     #define JSON_STRICT_NUL_HANDLING 0
 #endif
 
-#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
-#endif
-
 #if JSON_DIAGNOSTICS
     #define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
 #else
@@ -104,20 +100,14 @@
     #define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
 #endif
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
-#else
-    #define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
-#endif
-
 #ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
     #define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
 #endif
 
 // Construct the namespace ABI tags component
-#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
-#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
-    NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
+#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f) json_abi ## a ## b ## c ## d ## e ## f
+#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f) \
+    NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f)
 
 #define NLOHMANN_JSON_ABI_TAGS                                       \
     NLOHMANN_JSON_ABI_TAGS_CONCAT(                                   \
@@ -126,8 +116,7 @@
             NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS,              \
             NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS,         \
             NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION,           \
-            NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING,               \
-            NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
+            NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING)
 
 // Construct the namespace version component
 #define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \

tests/abi/config/default.cpp

@@ -44,10 +44,6 @@ TEST_CASE("default namespace")
         expected += "_snul";
 #endif
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-        expected += "_ekmo";
-#endif
-
         expected += "_v" STRINGIZE(NLOHMANN_JSON_VERSION_MAJOR);
         expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_MINOR);
         expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_PATCH) "::basic_json";

tests/abi/config/noversion.cpp

@@ -45,10 +45,6 @@ TEST_CASE("default namespace without version component")
         expected += "_snul";
 #endif
 
-#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-        expected += "_ekmo";
-#endif
-
         expected += "::basic_json";
 
         // fallback for Clang

tests/src/unit-enum_keyed_maps.cpp

@@ -1,246 +0,0 @@
-//     __ _____ _____ _____
-//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
-// |  |  |__   |  |  | | | |  version 3.12.0
-// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
-//
-// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
-// SPDX-License-Identifier: MIT
-
-#include "doctest_compatibility.h"
-
-// skip tests if JSON_DisableEnumSerialization=ON (#4384)
-#if defined(JSON_DISABLE_ENUM_SERIALIZATION) && (JSON_DISABLE_ENUM_SERIALIZATION == 1)
-    #define SKIP_TESTS_FOR_ENUM_SERIALIZATION
-#endif
-
-// This file tests the opt-in JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS, so it defines
-// the macro itself rather than relying on a -D flag, and runs in every build.
-// The default behavior is tested in unit-enum_keyed_maps_default.cpp.
-#ifdef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    #undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-#endif
-
-#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 1
-
-#include <nlohmann/json.hpp>
-using nlohmann::json;
-using nlohmann::ordered_json;
-
-#include <cstddef>
-#include <functional>
-#include <map>
-#include <string>
-#include <unordered_map>
-#include <utility>
-#include <vector>
-
-#define STRINGIZE_EX(x) #x
-#define STRINGIZE(x) STRINGIZE_EX(x)
-
-// NLOHMANN_JSON_SERIALIZE_ENUM uses a static std::pair
-DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
-DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors")
-
-namespace
-{
-// std::hash is only required for enums since C++14
-struct enum_hash
-{
-    template<typename T>
-    std::size_t operator()(T t) const noexcept
-    {
-        return static_cast<std::size_t>(t);
-    }
-};
-} // namespace
-
-// the example from #4378
-enum TaskState // NOLINT(cert-int09-c,readability-enum-initial-value,cppcoreguidelines-use-enum-class)
-{
-    TS_STOPPED,
-    TS_RUNNING,
-    TS_COMPLETED,
-    TS_INVALID = -1,
-};
-
-// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM(TaskState,
-{
-    {TS_INVALID, nullptr},
-    {TS_STOPPED, "stopped"},
-    {TS_RUNNING, "running"},
-    {TS_COMPLETED, "completed"},
-})
-
-enum class color {red, green, blue}; // blue is not mapped and falls back to "red"
-
-// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM(color,
-{
-    {color::red, "red"},
-    {color::green, "green"},
-})
-
-enum class strict_color {red, green, blue}; // blue is not mapped
-
-// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM_STRICT(strict_color,
-{
-    {strict_color::red, "red"},
-    {strict_color::green, "green"},
-})
-
-enum class digit {zero, one};
-
-// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM(digit,
-{
-    {digit::zero, 0},
-    {digit::one, 1},
-})
-
-#ifndef SKIP_TESTS_FOR_ENUM_SERIALIZATION
-enum class plain {zero, one}; // serialized as integer
-#endif
-
-TEST_CASE("JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS")
-{
-    SECTION("the macro is part of the ABI tag")
-    {
-        const std::string ns = STRINGIZE(NLOHMANN_JSON_NAMESPACE);
-        CHECK(ns.find("_ekmo") != std::string::npos);
-    }
-
-    SECTION("std::map (#4378)")
-    {
-        using task_map = std::map<TaskState, std::string>;
-        const task_map m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
-        const json j = m;
-        CHECK(j == json::parse(R"({"stopped":"aa","completed":"bb"})"));
-        CHECK(j.get<task_map>() == m);
-
-        json j2;
-        j2["x"] = m;
-        CHECK(j2.dump() == R"({"x":{"completed":"bb","stopped":"aa"}})");
-    }
-
-    SECTION("std::map with custom comparator")
-    {
-        using task_map = std::map<TaskState, int, std::greater<TaskState>>;
-        const task_map m = {{TS_STOPPED, 1}, {TS_RUNNING, 2}};
-        const json j = m;
-        CHECK(j == json::parse(R"({"stopped":1,"running":2})"));
-        CHECK(j.get<task_map>() == m);
-    }
-
-    SECTION("std::unordered_map")
-    {
-        using task_map = std::unordered_map<TaskState, int, enum_hash>;
-        const task_map m = {{TS_STOPPED, 1}, {TS_RUNNING, 2}};
-        const json j = m;
-        CHECK(j == json::parse(R"({"stopped":1,"running":2})"));
-        CHECK(j.get<task_map>() == m);
-    }
-
-    SECTION("nested maps")
-    {
-        using nested_map = std::map<color, std::map<TaskState, int>>;
-        const nested_map m = {{color::green, {{TS_RUNNING, 1}}}, {color::red, {}}};
-        const json j = m;
-        CHECK(j == json::parse(R"({"green":{"running":1},"red":{}})"));
-        CHECK(j.get<nested_map>() == m);
-    }
-
-    SECTION("ordered_json keeps the order of the map")
-    {
-        using task_map = std::map<TaskState, int>;
-        const task_map m = {{TS_STOPPED, 1}, {TS_RUNNING, 2}, {TS_COMPLETED, 3}};
-        const ordered_json j = m;
-        CHECK(j.dump() == R"({"stopped":1,"running":2,"completed":3})");
-        CHECK(j.get<task_map>() == m);
-    }
-
-    SECTION("empty map")
-    {
-        const json j = std::map<TaskState, int>();
-        CHECK(j.is_object());
-        CHECK(j.empty());
-    }
-
-    SECTION("NLOHMANN_JSON_SERIALIZE_ENUM_STRICT")
-    {
-        using color_map = std::map<strict_color, int>;
-        const color_map m = {{strict_color::red, 1}, {strict_color::green, 2}};
-        const json j = m;
-        CHECK(j == json::parse(R"({"red":1,"green":2})"));
-        CHECK(j.get<color_map>() == m);
-
-        const color_map unmapped = {{strict_color::blue, 1}};
-        json _;
-        CHECK_THROWS_WITH_AS(_ = unmapped,
-                             "[json.exception.out_of_range.410] enum value out of range for strict_color", json::out_of_range&);
-    }
-
-    SECTION("arrays of [key, value] pairs are still read")
-    {
-        using task_map = std::map<TaskState, int>;
-        const task_map m = {{TS_STOPPED, 1}};
-        CHECK(json::parse(R"([["stopped",1]])").get<task_map>() == m);
-    }
-
-    SECTION("other containers are not affected")
-    {
-        const std::vector<std::pair<TaskState, int>> pairs = {{TS_STOPPED, 1}};
-        const std::map<std::string, TaskState> string_keys = {{"a", TS_STOPPED}};
-        const std::map<int, int> int_keys = {{1, 2}};
-        CHECK(json(pairs) == json::parse(R"([["stopped",1]])"));
-        CHECK(json(string_keys) == json::parse(R"({"a":"stopped"})"));
-        CHECK(json(int_keys) == json::parse("[[1,2]]"));
-    }
-
-    SECTION("maps with non-unique keys are still stored as arrays of pairs")
-    {
-        const std::multimap<TaskState, int> mm = {{TS_STOPPED, 1}, {TS_STOPPED, 2}};
-        const std::unordered_multimap<TaskState, int, enum_hash> umm = {{TS_RUNNING, 3}, {TS_RUNNING, 3}};
-        CHECK(json(mm) == json::parse(R"([["stopped",1],["stopped",2]])"));
-        CHECK(json(umm) == json::parse(R"([["running",3],["running",3]])"));
-    }
-
-    SECTION("keys that do not serialize to strings")
-    {
-        const std::map<TaskState, int> null_key = {{TS_INVALID, 1}};
-        const std::map<digit, int> number_key = {{digit::zero, 1}};
-        json j = "unchanged";
-
-        // mapped to null
-        CHECK_THROWS_WITH_AS(j = null_key,
-                             "[json.exception.type_error.302] type must be string, but is null", json::type_error&);
-
-        // mapped to a number
-        CHECK_THROWS_WITH_AS(j = number_key,
-                             "[json.exception.type_error.302] type must be string, but is number", json::type_error&);
-
-#ifndef SKIP_TESTS_FOR_ENUM_SERIALIZATION
-        // enum without NLOHMANN_JSON_SERIALIZE_ENUM
-        const std::map<plain, int> plain_key = {{plain::zero, 1}};
-        CHECK_THROWS_WITH_AS(j = plain_key,
-                             "[json.exception.type_error.302] type must be string, but is number", json::type_error&);
-#endif
-
-        CHECK(j == "unchanged");
-    }
-
-    SECTION("keys that serialize to the same string")
-    {
-        const std::map<color, int> m = {{color::red, 1}, {color::blue, 2}};
-        json j = "unchanged";
-
-        // color::blue is not mapped and falls back to "red"
-        CHECK_THROWS_WITH_AS(j = m,
-                             "[json.exception.type_error.318] duplicate object key 'red'", json::type_error&);
-
-        CHECK(j == "unchanged");
-    }
-}
-
-DOCTEST_CLANG_SUPPRESS_WARNING_POP

tests/src/unit-enum_keyed_maps_default.cpp

@@ -1,141 +0,0 @@
-//     __ _____ _____ _____
-//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
-// |  |  |__   |  |  | | | |  version 3.12.0
-// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
-//
-// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
-// SPDX-License-Identifier: MIT
-
-#include "doctest_compatibility.h"
-
-// This file tests maps with enum keys with the default setting of
-// JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS (or whatever a -D flag sets it to).
-// unit-enum_keyed_maps.cpp tests JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS=1.
-// These tests are not part of unit-conversions.cpp, because that object file
-// is already too big for the MinGW linker of some compilers.
-
-#include <nlohmann/json.hpp>
-using nlohmann::json;
-
-#include <cstddef>
-#include <functional>
-#include <map>
-#include <string>
-#include <unordered_map>
-
-// NLOHMANN_JSON_SERIALIZE_ENUM uses a static std::pair
-DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
-DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors")
-
-enum class cards {kreuz, pik, herz, karo};
-
-// NOLINTNEXTLINE(misc-use-internal-linkage,misc-const-correctness,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM(cards,
-{
-    {cards::kreuz, "kreuz"},
-    {cards::pik, "pik"},
-    {cards::herz, "herz"},
-    {cards::karo, "karo"}
-})
-
-enum TaskState // NOLINT(cert-int09-c,readability-enum-initial-value,cppcoreguidelines-use-enum-class)
-{
-    TS_STOPPED,
-    TS_RUNNING,
-    TS_COMPLETED,
-    TS_INVALID = -1,
-};
-
-// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM(TaskState,
-{
-    {TS_INVALID, nullptr},
-    {TS_STOPPED, "stopped"},
-    {TS_RUNNING, "running"},
-    {TS_COMPLETED, "completed"},
-})
-
-enum class strict_cards {kreuz, pik, herz, karo, andere}; // andere not included in mapping
-
-// NOLINTNEXTLINE(misc-use-internal-linkage,misc-const-correctness,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
-NLOHMANN_JSON_SERIALIZE_ENUM_STRICT(strict_cards,
-{
-    {strict_cards::kreuz, "kreuz"},
-    {strict_cards::pik, "pik"},
-    {strict_cards::herz, "herz"},
-    {strict_cards::karo, "karo"}
-})
-
-namespace
-{
-// std::hash is only required for enums since C++14
-struct enum_hash
-{
-    template<typename T>
-    std::size_t operator()(T t) const noexcept
-    {
-        return static_cast<std::size_t>(t);
-    }
-};
-} // namespace
-
-// see unit-enum_keyed_maps.cpp for JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS=1
-TEST_CASE("maps with enum keys")
-{
-    using task_map = std::map<TaskState, std::string>;
-    using task_umap = std::unordered_map<TaskState, std::string, enum_hash>;
-    using task_gmap = std::map<TaskState, std::string, std::greater<TaskState>>;
-    using nested_map = std::map<cards, std::map<TaskState, int>>;
-    using strict_map = std::map<strict_cards, int>;
-    using int_map = std::map<int, int>;
-    using int_umap = std::unordered_map<int, int>;
-
-    const task_map m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
-
-#if !JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
-    SECTION("stored as array of pairs")
-    {
-        CHECK(json(m) == json::parse(R"([["stopped","aa"],["completed","bb"]])"));
-        CHECK(json(task_umap {{TS_RUNNING, "cc"}}) == json::parse(R"([["running","cc"]])"));
-    }
-#endif
-
-    SECTION("read from array of pairs")
-    {
-        CHECK(json::parse(R"([["stopped","aa"],["completed","bb"]])").get<task_map>() == m);
-    }
-
-    SECTION("read from object (#4378)")
-    {
-        const json j = json::parse(R"({"stopped":"aa","completed":"bb"})");
-        CHECK(j.get<task_map>() == m);
-        CHECK(j.get<task_umap>() == task_umap(m.begin(), m.end()));
-        CHECK(j.get<task_gmap>() == task_gmap(m.begin(), m.end()));
-        CHECK(json::parse(R"({"kreuz":{"stopped":1}})").get<nested_map>() == nested_map {{cards::kreuz, {{TS_STOPPED, 1}}}});
-        CHECK(nlohmann::ordered_json::parse(R"({"stopped":"aa","completed":"bb"})").get<task_map>() == m);
-
-        // object keys go through the enum's from_json
-        strict_map sm;
-        CHECK_THROWS_WITH_AS(json::parse(R"({"what?":1})").get_to(sm),
-                             "[json.exception.out_of_range.410] enum value out of range for strict_cards: \"what?\"", json::out_of_range&);
-    }
-
-    SECTION("objects are only read for enum keys")
-    {
-        int_map im;
-        int_umap ium;
-        CHECK_THROWS_WITH_AS(json::parse(R"({"1":2})").get_to(im),
-                             "[json.exception.type_error.302] type must be array, but is object", json::type_error&);
-        CHECK_THROWS_WITH_AS(json::parse(R"({"1":2})").get_to(ium),
-                             "[json.exception.type_error.302] type must be array, but is object", json::type_error&);
-    }
-
-    SECTION("other types are rejected")
-    {
-        task_map tm;
-        CHECK_THROWS_WITH_AS(json("stopped").get_to(tm),
-                             "[json.exception.type_error.302] type must be array, but is string", json::type_error&);
-    }
-}
-
-DOCTEST_CLANG_SUPPRESS_WARNING_POP

tests/src/unit-regression2.cpp

@@ -766,6 +766,15 @@ TEST_CASE("regression tests 2")
         CHECK(j == k);
     }
 
+    SECTION("issue #4552 - UTF-8 invalid characters are not always ignored when dumping with error_handler_t::ignore")
+    {
+        json node;
+        node["test"] = "test\334\005";
+        CHECK(node.dump(-1, ' ', false, json::error_handler_t::ignore) == "{\"test\":\"test\\u0005\"}");
+        CHECK(node.dump(-1, ' ', false, json::error_handler_t::keep) == "{\"test\":\"test\334\\u0005\"}");
+        CHECK(node.dump(-1, ' ', true, json::error_handler_t::keep) == "{\"test\":\"test\334\\u0005\"}");
+    }
+
 }
 
 TEST_CASE("regression test - parser callback must not lose a duplicate key's prior value")

tests/src/unit-serialization.cpp

@@ -92,6 +92,8 @@ TEST_CASE("serialization")
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"äü\"");
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"ä\xEF\xBF\xBDü\"");
             CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"\\u00e4\\ufffd\\u00fc\"");
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"ä\xA9ü\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"\\u00e4\xA9\\u00fc\"");
         }
 
         SECTION("invalid character (regression guard for shared UTF-8 decoder, see #5529)")
@@ -114,6 +116,8 @@ TEST_CASE("serialization")
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123\"");
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\"");
             CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd\"");
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"123\xC2\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"123\xC2\"");
         }
 
         SECTION("unexpected character")
@@ -126,6 +130,39 @@ TEST_CASE("serialization")
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123456\"");
             CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\x34\x35\x36\"");
             CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd456\"");
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"123\xF1\xB0\x34\x35\x36\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"123\xF1\xB0\x34\x35\x36\"");
+        }
+
+        SECTION("keep: valid characters are still escaped")
+        {
+            // an invalid byte followed by characters that must be escaped
+            const json j = "\xC2\"\\\n\xFF\x05";
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == "\"\xC2\\\"\\\\\\n\xFF\\u0005\"");
+            CHECK(j.dump(-1, ' ', true, json::error_handler_t::keep) == "\"\xC2\\\"\\\\\\n\xFF\\u0005\"");
+        }
+
+        SECTION("keep: truncated multibyte sequences")
+        {
+            CHECK(json("\xF0\x9F\x98").dump(-1, ' ', false, json::error_handler_t::keep) == "\"\xF0\x9F\x98\"");
+            CHECK(json("\xF0\x9F\x98").dump(-1, ' ', true, json::error_handler_t::keep) == "\"\xF0\x9F\x98\"");
+            CHECK(json("\xF0\x9F\x98" "a").dump(-1, ' ', false, json::error_handler_t::keep) == "\"\xF0\x9F\x98" "a\"");
+            CHECK(json("\xF0\x9F\x98" "a").dump(-1, ' ', true, json::error_handler_t::keep) == "\"\xF0\x9F\x98" "a\"");
+        }
+
+        SECTION("keep: long string with many invalid bytes")
+        {
+            // exceeds the internal string buffer several times
+            std::string input;
+            std::string expected = "\"";
+            for (int i = 0; i < 2000; ++i)
+            {
+                input += "\xFF\xE2\x82\n\xC3\xA4";
+                expected += "\xFF\xE2\x82\\n\xC3\xA4";
+            }
+            expected += "\"";
+            const json j = input;
+            CHECK(j.dump(-1, ' ', false, json::error_handler_t::keep) == expected);
         }
 
         SECTION("U+FFFD Substitution of Maximal Subparts")

tests/src/unit-unicode2.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tests/src/unit-unicode3.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tests/src/unit-unicode4.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tests/src/unit-unicode5.cpp

@@ -14,6 +14,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <algorithm>
 #include <fstream>
 #include <sstream>
 #include <iostream>
@@ -75,8 +76,11 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     static std::string s_replaced2;
     static std::string s_replaced_ascii;
     static std::string s_replaced2_ascii;
+    static std::string s_kept;
+    static std::string s_kept2;
+    static std::string s_kept_ascii;
 
-    // dumping with ignore/replace must not throw in any case
+    // dumping with ignore/replace/keep must not throw in any case
     s_ignored = j.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored2 = j2.dump(-1, ' ', false, json::error_handler_t::ignore);
     s_ignored_ascii = j.dump(-1, ' ', true, json::error_handler_t::ignore);
@@ -85,6 +89,9 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     s_replaced2 = j2.dump(-1, ' ', false, json::error_handler_t::replace);
     s_replaced_ascii = j.dump(-1, ' ', true, json::error_handler_t::replace);
     s_replaced2_ascii = j2.dump(-1, ' ', true, json::error_handler_t::replace);
+    s_kept = j.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept2 = j2.dump(-1, ' ', false, json::error_handler_t::keep);
+    s_kept_ascii = j.dump(-1, ' ', true, json::error_handler_t::keep);
 
     if (success_expected)
     {
@@ -94,6 +101,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
         // all dumps should agree on the string
         CHECK(s_strict == s_ignored);
         CHECK(s_strict == s_replaced);
+        CHECK(s_strict == s_kept);
     }
     else
     {
@@ -105,6 +113,20 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
 
         // check that replace string contains a replacement character
         CHECK(s_replaced.find("\xEF\xBF\xBD") != std::string::npos);
+
+        // ignore drops the invalid bytes, keep copies them
+        CHECK(s_ignored != s_kept);
+        CHECK(s_ignored_ascii != s_kept_ascii);
+
+        // unless a byte needs escaping, keep copies the input unchanged
+        const bool needs_escaping = std::any_of(json_string.begin(), json_string.end(), [](char c)
+        {
+            return static_cast<unsigned char>(c) < 0x20 || c == '"' || c == '\\';
+        });
+        if (!needs_escaping)
+        {
+            CHECK(s_kept == "\"" + json_string + "\"");
+        }
     }
 
     // check that prefix and suffix are preserved
@@ -116,6 +138,8 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
     CHECK(s_replaced2.substr(s_replaced2.size() - 4, 3) == "xyz");
     CHECK(s_replaced2_ascii.substr(1, 3) == "abc");
     CHECK(s_replaced2_ascii.substr(s_replaced2_ascii.size() - 4, 3) == "xyz");
+    CHECK(s_kept2.substr(1, 3) == "abc");
+    CHECK(s_kept2.substr(s_kept2.size() - 4, 3) == "xyz");
 }
 
 void check_utf8string(bool success_expected, int byte1, int byte2, int byte3, int byte4);

tools/generate_natvis/generate_natvis.py

@@ -20,7 +20,7 @@ if __name__ == '__main__':
 
     namespaces = ['nlohmann']
     abi_prefix = 'json_abi'
-    abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics', '_psp', '_snul', '_ekmo']
+    abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics', '_psp', '_snul']
     version = '_v' + args.version.replace('.', '_')
     inline_namespaces = []