amorfo77/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-24 06:48:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

1.5: keep get_bon8_string's own UTF-8 ranges (#5891)

Browse Source 
Taking the second-byte range from utf8_shape() made BON8 decoding 2-3% slower
(FromBinaryShape/scalars, FromBinaryBuffer, alignment-pinned builds), so the byte
path keeps its hand-written ranges. The exhaustive validator test still checks
that it accepts the same sequences as the other validators.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-23 12:18:56 +02:00
1 parent fce5a54722
commit fc347b5779
2 files changed
+19 -7
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

include/nlohmann/detail/input/binary_reader.hpp
+17 -5
View File
@@ -3810,11 +3810,23 @@ class binary_reader
// the valid range of the second byte excludes overlong forms, // the valid range of the second byte excludes overlong forms,
// surrogates, and code points above U+10FFFF // surrogates, and code points above U+10FFFF
// (RFC 3629, section 4); length 0 flags the lead bytes // (RFC 3629, section 4)
// 0xF5..0xF7, which BON8 can read but UTF-8 cannot encode int continuation_bytes = 0;
const auto shape = utf8_shape(static_cast<std::uint8_t>(byte)); bool valid_second = true;
const int continuation_bytes = (byte <= 0xDF) ? 1 : ((byte <= 0xEF) ? 2 : 3); if (byte <= 0xDF)
const bool valid_second = shape.length != 0 && second >= shape.second_lo && second <= shape.second_hi; {
continuation_bytes = 1;
}
else if (byte <= 0xEF)
{
continuation_bytes = 2;
valid_second = (byte != 0xE0 || second >= 0xA0) && (byte != 0xED || second <= 0x9F);
}
else
{
continuation_bytes = 3;
valid_second = byte <= 0xF4 && (byte != 0xF0 || second >= 0x90) && (byte != 0xF4 || second <= 0x8F);
}
if (JSON_HEDLEY_UNLIKELY(!valid_second)) if (JSON_HEDLEY_UNLIKELY(!valid_second))
{ {
include/nlohmann/detail/string_utils.hpp
+2 -2
View File
@@ -210,8 +210,8 @@ places, which differ in speed, diagnostics, and how they read the input:
bulk string scan, the bulk path of the BON8 reader, and the BON8 writer. bulk string scan, the bulk path of the BON8 reader, and the BON8 writer.
They must accept exactly what the lexer's switch accepts. They must accept exactly what the lexer's switch accepts.
- the byte path of binary_reader::get_bon8_string(): BON8 input without bulk - the byte path of binary_reader::get_bon8_string(): BON8 input without bulk
access, and the bytes the bulk path leaves to it. It takes the valid range access, and the bytes the bulk path leaves to it. It keeps its own ranges:
of the second byte from utf8_shape(). taking them from utf8_shape() made BON8 decoding 2-3% slower (#5891).
- check_utf8() in detail/view/edit.hpp: strings stored in an editable - check_utf8() in detail/view/edit.hpp: strings stored in an editable
json_document, with the index of the offending byte. It also uses json_document, with the index of the offending byte. It also uses
utf8_shape(). utf8_shape().