amorfo77/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-23 14:28:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

1.5: use utf8_shape in get_bon8_string, add exhaustive UTF-8 validator test (#5891)

Browse Source 
get_bon8_string takes the valid range of the second byte from utf8_shape()
(0xF5..0xF7 stay errors, unget behavior unchanged). validate_one_utf8 keeps its
if-chain (hot bulk path); the new test checks decode, validate_one_utf8,
utf8_shape, json::accept and the BON8 reader against each other for every lead
and second byte.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-23 10:30:29 +02:00
1 parent df98274f6c
commit 643acf23e2
2 files changed
+88 -17
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

include/nlohmann/detail/input/binary_reader.hpp
+5 -17
View File
@@ -3770,23 +3770,11 @@ class binary_reader
// the valid range of the second byte excludes overlong forms,
// surrogates, and code points above U+10FFFF
// (RFC 3629, section 4)
int continuation_bytes = 0;
bool valid_second = true;
if (byte <= 0xDF)
{
continuation_bytes = 1;
}
else if (byte <= 0xEF)
{
continuation_bytes = 2;
valid_second = (byte != 0xE0 || second >= 0xA0) && (byte != 0xED || second <= 0x9F);
}
else
{
continuation_bytes = 3;
valid_second = byte <= 0xF4 && (byte != 0xF0 || second >= 0x90) && (byte != 0xF4 || second <= 0x8F);
}
// (RFC 3629, section 4); length 0 flags the lead bytes
// 0xF5..0xF7, which BON8 can read but UTF-8 cannot encode
const auto shape = utf8_shape(static_cast<std::uint8_t>(byte));
const int continuation_bytes = (byte <= 0xDF) ? 1 : ((byte <= 0xEF) ? 2 : 3);
const bool valid_second = shape.length != 0 && second >= shape.second_lo && second <= shape.second_hi;
if (JSON_HEDLEY_UNLIKELY(!valid_second))
{