Adds error_handler_t::keep (invalid UTF-8 sequences are left unchanged) as
a fourth error_handler_t value, and threads an error_handler parameter
through the binary writers and readers:
- to_cbor/to_ubjson/to_bjdata/to_bson gain a trailing error_handler
parameter (default strict, matching their existing type_error.316
behavior); keep writes a string value or object key's bytes as is
instead of throwing, and replace/ignore sanitize it exactly like
dump() would, including for the BSON length prefix. to_msgpack and
to_bon8 are unchanged.
- from_cbor/from_msgpack/from_ubjson/from_bjdata/from_bson gain a
trailing error_handler parameter (default keep, i.e. the lenient
behavior every binary reader had in 3.12.0 and still has after
#5741); strict checks every string value and object key and raises
parse_error.113 for ill-formed UTF-8, honoring allow_exceptions;
replace/ignore sanitize it like dump() would. from_bon8 is
unchanged, since UTF-8 lead bytes are structural there.
dump()'s own keep support writes ill-formed bytes as is, even with
ensure_ascii, while still \u-escaping well-formed characters around
them as usual.
The UTF-8 validity check (is_valid_utf8) and the replace/ignore
sanitizing logic (sanitize_utf8) now live in string_utils.hpp, shared
by the serializer and the binary reader/writer; error_handler_t itself
moved to its own header (detail/output/error_handler.hpp) so that
string_utils.hpp does not need to depend on serializer.hpp.
See #5529 and #5741.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
