Validate UTF-8 in CBOR/MessagePack/BSON/UBJSON/BJData text strings on develop

PR #5531 fixed the UTF-8-validation gap described in #5529, but it was
merged onto the still-unmerged bson-sizes branch rather than develop, so
develop was left with the original bug for all affected formats. A
follow-up comment on #5529 reproduced this on develop and additionally
found that UBJSON (and, by the same code path, BJData) has the identical
gap, undocumented.

Port the same fix directly onto develop: extract the UTF-8 DFA decoder
out of serializer<>::decode() into a shared detail::decode()/is_valid_utf8()
in string_utils.hpp, and call it from binary_reader::get_string() - the
single choke point shared by all five binary readers - so malformed text
strings are rejected at decode time (parse_error.113) instead of only
failing later on dump() (type_error.316). Byte/binary payloads are
unaffected. Add matching decode-time tests for CBOR, MessagePack, BSON,
UBJSON, and BJData, and document the new behavior on all five binary
format pages (the two UBJSON/BJData pages didn't get this note in #5531).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017sdieJCn6BHxzRMaXP49sP
This commit is contained in:
Claude
2026-09-16 05:20:00 +00:00
parent c72f37a40d
commit 5e5a087cb3
15 changed files with 448 additions and 180 deletions
@@ -32,6 +32,7 @@
#include <nlohmann/detail/meta/is_sax.hpp>
#include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_utils.hpp>
#include <nlohmann/detail/value_t.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -3304,7 +3305,24 @@ class binary_reader
const NumberType len,
string_t& result)
{
return get_bytes(format, len, "string", result);
if (JSON_HEDLEY_UNLIKELY(!get_bytes(format, len, "string", result)))
{
return false;
}
// RFC 8949 (CBOR) §3.1 and the MessagePack/BSON/UBJSON specifications
// all require text strings to be valid UTF-8; reject anything else
// right here so malformed input is caught at decode time instead of
// only surfacing later as a type_error.316 when the value is dumped
// (which would defeat allow_exceptions=false / strict discarding).
if (JSON_HEDLEY_UNLIKELY(!is_valid_utf8(result)))
{
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(113, chars_read,
exception_message(format, "invalid string: ill-formed UTF-8 byte", "string"), nullptr));
}
return true;
}
/*!