mirror of
https://github.com/nlohmann/json.git
synced 2026-10-31 09:55:42 +00:00
Niels Lohmann
22d2013b13
With error_handler_t::strict (and replace/ignore), every string a binary reader reads is checked for valid UTF-8, and every string a binary writer writes. The readers used the byte-at-a-time decoder; the writers skipped ASCII eight bytes at a time. - New valid_utf8_string() decides strings of up to 16 ASCII bytes, most keys and many values, with one or two word tests, and checks longer strings 16 bytes at a time with the NEON/SSSE3 UTF-8 check the lexer uses (vector_utf8_valid_blocks: the same lookup, but quotes and control characters are ordinary characters). Other targets and JSON_NO_SIMD use valid_utf8_prefix() as before. - The readers check strings with it (for one-byte character types) in an out-of-line function, so that reading without a check does not pay for its code. - The writers' check_utf8() uses it, and the CBOR and MessagePack writers write a checked string of up to max_text_in_reservation characters in one reservation, as they write unchecked ones. Signed-off-by: Niels Lohmann <mail@nlohmann.me>