mirror of
https://github.com/nlohmann/json.git
synced 2026-10-15 18:57:13 +00:00
Niels Lohmann
dcb96e82fb
* Escape characters below 0x60 from a table straight into the write buffer When the bulk scanner in dump_escaped_impl() stops at a control character, a quote or a backslash, that byte used to go through the UTF-8 decoder and a switch that wrote the escape byte by byte into string_buffer, which was then copied into write_buffer at the next run. A byte below 0x60 at a character boundary is a complete code point, so it now skips the decoder: its escape sequence comes from a 96-entry table of 8-byte rows (sequence padded with zeros, length in the last byte), and all 8 bytes are copied straight into write_buffer with one room check. The check sits inside the existing character-boundary block, so it runs once per scanner stop and not once per byte; testing every byte cost about 6% on twitter.json with ensure_ascii. The switch in the decoder's accept branch lost its cases for characters below 0x60, which can no longer reach it. Output is byte-identical. The string-buffer boundary tests now use \u-escaped non-ASCII characters, since simple escapes no longer reach string_buffer, and a new test case checks every ASCII character at every position against a reference escaper. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Avoid raw string literals with backslashes in CHECK macros MSVC's traditional preprocessor mis-lexes raw string literals inside macro arguments and reports illegal escape sequences (C2017, C3850). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Sweep invalid UTF-8 at every position through dump() Compare dump() of strings with ill-formed UTF-8 (stray continuation bytes, invalid lead bytes, overlong encodings, surrogates, code points above U+10FFFF, and sequences cut short by ASCII, a quote, a backslash, a lead byte, or the end of the string) with an independent reference serializer, for every error handler and with ensure_ascii on and off. The pattern is put at every position of short strings to cross the strides of the bulk scanners, and of longer ones to cross the end of the 512-byte string buffer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>