* Add SWAR bulk fast path to string serialization (dump_escaped)
When ensure_ascii is false, dump_escaped previously ran every byte of
every string and object key through the UTF-8 DFA decoder, even for the
common case of ordinary text with nothing to escape. This mirrors the
per-byte cost the parser had before the contiguous fast paths.
At a character boundary, bulk-copy the longest run of bytes that need no
escaping using string_bulk_run() - the same SWAR scanner and UTF-8 bulk
validator the lexer's contiguous path uses - and only fall back to the
byte-at-a-time DFA loop for the first byte that needs individual handling
(a quote, backslash, control character, or ill-formed/truncated UTF-8).
Because every "hard" or invalid byte is still processed by the unchanged
byte path, escaping output and error handling (including strict-mode
error 316 position and message) are byte-identical to before.
The ensure_ascii=true path is unchanged: it must escape non-ASCII and
0x7F, which string_bulk_run does not stop on, so a separate predicate
would be needed for it.
Verified byte-for-byte identical dump output against the pre-change
implementation across ~20k randomized byte strings plus curated edge
cases (all escapes, control chars, valid multibyte, surrogates,
overlong, truncated sequences) for both ensure_ascii settings and all
three error handlers, in C++11/17/20 at -O2/-O3.
Throughput (g++ -O3, ensure_ascii=false, vs pre-change):
long ASCII strings 4.2x
twitter-like objects 2.3x
dense CJK 1.4x (further headroom with JSON_USE_SIMDUTF)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Buffer serializer output and add ensure_ascii string fast path
Two further serialization speedups on top of the ensure_ascii=false bulk
copy, both reusing the SWAR primitives in detail/input/string_scan.hpp.
1. Internal write buffer (devirtualization). Every structural character
('{', '"', ',', ...) previously went straight to the output adapter
through a virtual call. Route all writes through put_char/put_chars
into a 1 KiB buffer that flushes in bulk; the public dump() flushes
once the top-level value is done (the recursive worker is split out as
dump_internal). Runs larger than the buffer are written straight
through, so large payloads are not copied twice. This is the dominant
cost for object/array-heavy values.
2. ensure_ascii fast path. dump_escaped previously ran the UTF-8 DFA over
every byte when escaping non-ASCII. Add find_ascii_copyable_run() (a
SWAR scan stopping at '"', '\\', < 0x20, 0x7F, and >= 0x80) so runs of
printable ASCII are bulk-copied, with the byte path handling each
escape/non-ASCII byte exactly as before.
Behavior is unchanged: dump output is byte-for-byte identical to the
previous implementation across ~20k randomized byte strings plus curated
edge cases (all escapes, control chars, 0x7F, valid multibyte,
surrogates, overlong, truncated), for object/array/pretty output, both
ensure_ascii settings, and all three error handlers, in C++11/17/20 at
-O2/-O3. New unit tests cover the buffer flush boundaries, the escape and
0x7F handling, multibyte under both settings, and invalid-UTF-8 handling.
Throughput (g++ -O3, vs the ensure_ascii=false-only baseline):
long ASCII, ensure_ascii=0 4.2x
long ASCII, ensure_ascii=1 4.1x
twitter-like objects 2.7x
dense CJK 1.8x
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Flush serializer buffer in dump_escaped unit test
test-convenience failed (macOS finished first; the failure is
platform-independent) because check_escaped() calls the internal
serializer::dump_escaped() directly and then reads the output stream.
Since dump_escaped() now writes into the serializer's internal write
buffer, the bytes were still buffered and the stream was empty.
Expose flush() under JSON_PRIVATE_UNLESS_TESTED (same visibility as
dump_escaped) and flush in check_escaped() before inspecting the output.
Per-string flushing inside dump_escaped() was rejected on purpose: it
would defeat the buffering that makes object/array-heavy dumps faster.
Library behavior is unchanged (flush()'s body is identical; only its
access label moved).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid deep recursion in serialization write-buffer test
The "many small structural writes exceed the write buffer" subcase built
a 1100-deep nested array and dumped it to force >1024 consecutive
single-character writes through put_char (exercising the write buffer's
flush-when-full branch). dump() recurses per nesting level, so on MSVC
debug builds (smaller default stack, larger frames) this overflowed the
stack and crashed test-serialization; Linux/macOS have enough headroom to
hide it.
Replace the nesting with a flat array of 500 empty strings. Each element
emits '"', '"', ',' via put_char, so the dump is a long run of
single-character writes (1501 bytes > the 1024-byte buffer) at nesting
depth two, hitting the same flush branch without deep recursion. Library
code is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the write-buffer helpers and write indentation directly
Follow-up to @gregmarr's review: put_chars() was doing four unrelated jobs, so
give the two that can be made safe their own entry points.
- put_literal(): takes the literal by reference and deduces the length from the
array bound, so the 27 hand-counted lengths at the call sites can no longer
drift from the literals they describe. A literal is checked at compile time to
fit the buffer, so this path needs no write-through branch.
- put_buffer(): takes the fixed-size buffer itself rather than a bare pointer,
so the length can be checked against the buffer's own bound.
- put_indent(): memsets the indentation into the write buffer, filling and
flushing it as needed. This removes indent_string entirely, and with it both
bugs of #5186: the indentation string was grown by doubling, which is not
enough when indent_step more than doubles it (a heap over-read - dump(2000)
read 2000 bytes out of a 1024-byte string), and the grown part was filled with
a space instead of the configured indent_char. next_indent() keeps that PR's
assertion against the unsigned indentation accumulation wrapping on deep
nesting.
put_chars() keeps the two cases that are genuinely a pointer and a count: the
run-length copies out of the string being escaped, and to_chars() output.
Tests cover an indent_step wider than the write buffer, a non-space indentation
character past the old growth point, and nesting whose accumulated indentation
spans several buffer-fulls. All three fail against develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fill the indentation buffer once instead of once per flush
@gregmarr's point on the fill-and-flush loop: flushing does not disturb what
the write buffer holds, so an indentation spanning several buffer-fulls only
has to be written into the buffer once and can then be handed to the adapter
as many times as needed. The loop re-filled it every time, doing work it
already knew was there.
put_indent() now fills the room left in the buffer, and if anything remains,
flushes, fills the buffer once, and re-flushes that same content. It also
returns early for a zero-width indentation, which is what the closing brace of
every outermost value asks for.
Measured over a dump(), counting memset calls and bytes inside put_indent:
indent before after
4 1 call / 4 B 1 call / 4 B
2000 2 calls / 2000 B 2 calls / 2046 B
100000 98 calls / 100000 B 2 calls / 2046 B
The wide case is now constant work rather than proportional to the indentation
width; ordinary widths are unchanged. Tests extended to cover several whole
buffer-fulls and an exact multiple of the buffer size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Tighten the write-buffer helpers after review
More of @gregmarr's review on the put_* split:
- Reattach the put_chars() doc comment, which the new helpers had been
inserted in front of, leaving it describing put_indent().
- Compute the literal length once in put_literal() instead of spelling N - 1
at each use.
- Add put_string(str, start, end), which keeps the pointer arithmetic and the
bounds assertions inside the function instead of at the call site. With
dump_float()'s to_chars() output moved onto put_buffer() as well, put_chars()
now has no callers outside put_string()/put_buffer(): nothing passes a bare
pointer and a count any more.
- Carry the indentation as std::size_t rather than unsigned int. It is a size,
it is compared and combined with buffer sizes throughout, and the casts in
put_indent() disappear. next_indent() keeps its assertion, which is far
harder to trip on a 64-bit size_t but still reachable where that is 32 bits.
No output change: pretty and compact dumps, binary values included, are
byte-identical to develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence avoid-c-arrays on put_literal's array reference
clang-tidy flags the reference-to-array parameter under
cppcoreguidelines/hicpp/modernize-avoid-c-arrays, and the CI treats warnings as
errors. Binding to the array is the whole point here - it is what lets the
length be deduced from the literal instead of hand-written at the call site - so
suppress it the same way from_json(), to_json() and get_to() already suppress it
for their own T (&arr)[N] parameters.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bound the descent of dump()
Serializing a container serializes its elements, so dump() descended into one
call per nesting level. A value nested deeply enough exhausted the call stack
and terminated the process with a segmentation fault - no exception, nothing
the caller could catch. Parsing such a value works, as the parser is
iterative, and so does destroying one, as #1436 made destruction iterative.
Bound how far the descent goes rather than take the call stack away from it.
The first 128 levels are written by exactly the code that always wrote them,
and only below that does dump_iteratively write out what is left, keeping the
containers it has entered on an explicit stack. Serializing can therefore no
longer exhaust the stack, however deeply a value is nested, while a value
nested less deeply than the bound pays only for one comparison per container.
Writing every value that way instead measured between 2% and 20% slower - 20%
on object-heavy documents - which is why the descent is kept for all but the
values that cannot afford it. The bound costs nothing measurable: between
-1.4% and +1.2% across compact and pretty output of number, integer, string,
object-heavy, wide-object and deeply nested documents.
The output is unchanged for every value. Both ways of writing a container
emit the separator in front of every element but the first, rather than
after every element but the last, which puts exactly one between each pair
and none at the end.
This fixes#5387 for dump(). The copy constructor is fixed in #5389.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fold ensure_ascii into the escaper and write bytes without dump_integer
Two hot spots that the write buffer and the bulk scanner left behind.
dump_escaped took ensure_ascii as a runtime flag and tested it inside the
loop, once per character run, although it cannot change while a string is
written. It is now a template parameter, dispatched once per string, which
folds the choice of scanner and lets each of the two be inlined into a loop
of its own. This is the hottest loop in the serializer: it runs over every
string and every object key.
A binary value's bytes went through dump_integer, which counts digits and
does 64-bit arithmetic for a number that is always in [0, 255]. dump_byte
writes the three digits it takes at most straight into the write buffer
instead. Any byte type that is not a plain unsigned byte is still left to
dump_integer, whose representation of it may differ.
Measured against the previous commit (medians of 9 interleaved runs, clang
-O3): binary values -33.8%, dense CJK with ensure_ascii -20.6%, key-heavy
objects -17.8%, deeply nested pretty output -17.9%, dense CJK without
ensure_ascii -11.8%, object-heavy documents -9.3% compact and -9.5% pretty,
a small value dumped in a loop -21.4%, wide objects -2.3%. Arrays of plain
ASCII strings measured 3.5% to 4.2% slower, the one shape that loses; number
and integer arrays are unchanged.
Also tried and dropped: leaving the write and string buffers uninitialized
rather than zeroing 1.5 KB per dump() call. It is worth -30% on small values,
but two nearly identical string workloads moved 18% apart in opposite
directions, so the measurements did not support it.
The output is unchanged for every value: the differential now also covers
every one of the 256 byte values, alone and together, in both binary layouts.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write a byte without walking a pointer over the buffer
clang-tidy's misc-const-correctness reads the pointer dump_byte advanced over
the write buffer as one whose pointee could be const. Index the buffer
instead, which says the same thing without a raw pointer at all.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Parenthesize the reserve arithmetic in the deep-nesting test
clang-tidy's readability-math-missing-parentheses wants the multiplication
spelled out in reserve(6 * depth + 1), and CI treats its warnings as errors.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not scan for a copyable run that cannot exist
Under ensure_ascii, dump_escaped() calls find_ascii_copyable_run() at every
character boundary. When the text is dense non-ASCII - CJK, where every byte
is >= 0x80 - the scanner stops on its first byte and returns zero, so its SWAR
block runs once per character and buys nothing, on top of the escaping that
still has to happen afterwards.
A run can only be non-empty when the first byte is one the scanner may copy,
so test that single byte before calling it. Runs that do exist are found
exactly as before, so the bulk-copy win is unchanged; only the calls that were
always going to return zero are skipped.
Output is unchanged: the dump digest over canada/citm/twitter, in compact,
pretty and ensure_ascii form, matches develop byte for byte.
dump(ensure_ascii=true) develop before after
CJK text 3.54ms 4.25ms 3.36ms
CJK, no ASCII at all 3.09ms 4.02ms 3.02ms
Latin-1-ish text 4.39ms 3.04ms 2.93ms
plain ASCII 3.92ms 0.80ms 0.79ms
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review of the write-buffer helpers
Three points from @gregmarr's review:
put_chars() is gone. It was the only entry point taking a bare pointer and
a count, and it existed only so put_string() and put_buffer() had something
to delegate to. Its body now lives in put_string(), and put_buffer() is
put_string(buffer, 0, length) - std::array already carries data() and
size(), so it satisfies the same interface a string does. Nothing appends
characters without a bound any more.
dump_escaped()'s documentation block was duplicated. The dispatcher was
inserted between the original comment and the function it described, and
the comment was copied rather than split. The worker now has its own short
comment saying why ensure_ascii is a template parameter.
The local in dump_byte() is deliberate, and is now documented as such:
writing through write_buffer[] is a char write, which may alias any object,
so with write_buffer_pos updated in place the compiler must reload and
store it around every digit. Measured on a dump of a 4 MiB binary value,
18.0 ms without the local against 7.4 ms with it.
Output is unchanged: byte-identical dumps across 77 files in compact,
pretty, ensure_ascii, pretty+ascii, indent 600 and tab-indent form.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: drop unneeded backslash-escapes and duplicate scan loop
'"' does not need escaping in a char literal, unlike in a string literal.
find_ascii_copyable_run() also duplicated the byte-at-a-time search that
already exists as the loop's own scalar tail; break into it instead of
re-deriving the offset in a second, near-identical loop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move pretty_print, ensure_ascii and indent_step into the serializer
None of these change over the life of a serializer, unlike current_indent
and depth, which do change on every recursive call. They are now captured
once in the constructor - matching indent_char and error_handler - instead
of being threaded through dump(), dump_internal(), dump_iteratively(),
dump_value() and dump_escaped() on every call.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop the serializer from holding onto std::localeconv()'s pointer
loc was only ever read twice, immediately, to seed thousands_sep and
decimal_point; nothing else in the class used it. A local in the
constructor body serves the same purpose without keeping the pointer
around for the serializer's lifetime.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep thousands_sep/decimal_point const via a small locale_chars struct
const members can't be assigned in a constructor body, so seeding them
from std::localeconv() meant either dropping const or holding onto the
lconv* for longer than needed. A sub-object computes both from the
pointer in its own constructor and is itself initialized in serializer's
mem-initializer-list, so the two chars stay const, std::localeconv() is
still called exactly once, and nothing outlives the constructor.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Replace snprintf with a branch-free writer for \uXXXX escapes
dump_escaped called std::snprintf(..., "\u%04x", ...) once per escaped
code point in the string serialization hot path. snprintf re-parses
the format string and pulls in locale/printf machinery on every call,
which is far heavier than the fixed 6-/12-byte output warrants. This
is hot for any string containing control characters, and for all
non-ASCII text when ensure_ascii is set.
Replace it with write_u_escape, a small helper that writes the escape
directly into string_buffer via a nibble-to-hex lookup table, mirroring
the existing hand-rolled dump_integer fast path in the same file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy avoid-c-arrays warning in write_u_escape
Use a const char* rather than a char[] lookup table, matching the
existing hex_bytes helper in the same file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* ♻️ adjust write_u_escape signature
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels LohmannandClaude Opus 4.8
