amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-03 13:10:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Accept ill-formed UTF-8 in all binary readers again

Browse Source 
RFC 8949 and the MessagePack/BSON/UBJSON/BJData specs leave UTF-8
well-formedness checking up to the decoder, so following #5529 the
binary readers are lenient by default again, as in release 3.12.0
(the reader-side check was added by #5185/#5531, not in any release);
reader-side validation becomes opt-in in a follow-up PR. The writers
stay strict and throw type_error.316 for ill-formed UTF-8. BON8 is
unchanged, since UTF-8 lead bytes are structural there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-10-01 07:59:14 +02:00
parent f3e30fa5b7
commit 92c729f657
12 changed files with 366 additions and 301 deletions
Download Patch File Download Diff File Expand all files Collapse all files
include/nlohmann/detail/input/binary_reader.hpp
+7 -26
View File
@@ -4031,32 +4031,13 @@ class binary_reader
const NumberType len,
string_t& result)
{
// get_bytes() appends to result, and CBOR indefinite-length strings
// collect all their chunks in the same result; validating only the
// newly read bytes keeps the check linear in the input size
const std::size_t old_size = result.size();
if (JSON_HEDLEY_UNLIKELY(!get_bytes(format, len, "string", result)))
{
return false;
}
// RFC 8949 (CBOR) §3.1 and the BSON/UBJSON specifications require
// text strings to be valid UTF-8; reject anything else right here so
// malformed input is caught at decode time instead of only surfacing
// later as a type_error.316 when the value is dumped (which would
// defeat allow_exceptions=false / strict discarding). The MessagePack
// specification explicitly allows a str object to contain an invalid
// byte sequence and expects deserializers to hand back the original
// bytes, so msgpack strings (and map keys, which go through this
// function as well) are exempt.
if (format != input_format_t::msgpack && JSON_HEDLEY_UNLIKELY(!is_valid_utf8(result, old_size)))
{
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(113, chars_read,
exception_message(format, "invalid string: ill-formed UTF-8 byte", "string"), nullptr));
}
return true;
// Strings are taken as is: none of CBOR (RFC 8949 §3.1 leaves the
// choice to the decoder), MessagePack (whose spec explicitly allows
// a str object to contain an invalid byte sequence), UBJSON, BJData,
// or BSON requires a decoder to reject ill-formed UTF-8. The bytes
// are kept unchanged; dump() and the binary writers are the ones
// that check them and report type_error.316 if they are not valid.
return get_bytes(format, len, "string", result);
}
/*!