amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-09-03 15:05:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Do not scan for a copyable run that cannot exist

Browse Source 
Under ensure_ascii, dump_escaped() calls find_ascii_copyable_run() at every
character boundary. When the text is dense non-ASCII - CJK, where every byte
is >= 0x80 - the scanner stops on its first byte and returns zero, so its SWAR
block runs once per character and buys nothing, on top of the escaping that
still has to happen afterwards.

A run can only be non-empty when the first byte is one the scanner may copy,
so test that single byte before calling it. Runs that do exist are found
exactly as before, so the bulk-copy win is unchanged; only the calls that were
always going to return zero are skipped.

Output is unchanged: the dump digest over canada/citm/twitter, in compact,
pretty and ensure_ascii form, matches develop byte for byte.

  dump(ensure_ascii=true)   develop    before     after
  CJK text                   3.54ms    4.25ms    3.36ms
  CJK, no ASCII at all       3.09ms    4.02ms    3.02ms
  Latin-1-ish text           4.39ms    3.04ms    2.93ms
  plain ASCII                3.92ms    0.80ms    0.79ms

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-09-03 07:42:11 +02:00
parent 49e12fabce
commit 46d2b68d3d
2 changed files with 28 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
include/nlohmann/detail/output/serializer.hpp
+14 -3
View File
@@ -844,9 +844,20 @@ class serializer
if (state == UTF8_ACCEPT)
{
const auto* const data = reinterpret_cast<const unsigned char*>(s.data());
const std::size_t run = EnsureAscii
? find_ascii_copyable_run(data + i, s.size() - i)
: string_bulk_run(data + i, s.size() - i);
// A run can only be non-empty when the very first byte is one
// the scanner may copy, so test that single byte before paying
// for the scan. Without it, text whose characters all have to be
// escaped - CJK under ensure_ascii, where every byte is >= 0x80 -
// runs the scanner once per character only to be told zero.
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(data + i, s.size() - i);
}
else if (is_ascii_copyable(data[i]))
{
run = find_ascii_copyable_run(data + i, s.size() - i);
}
if (run != 0)
{
// emit any bytes still pending in string_buffer first to