From ff80ed329564c51b8b7e1203db2b114ce7815b42 Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Fri, 11 Sep 2026 08:22:49 +0200 Subject: [PATCH] Speed up dump(), and keep it from overflowing the stack (#5285) * Add SWAR bulk fast path to string serialization (dump_escaped) When ensure_ascii is false, dump_escaped previously ran every byte of every string and object key through the UTF-8 DFA decoder, even for the common case of ordinary text with nothing to escape. This mirrors the per-byte cost the parser had before the contiguous fast paths. At a character boundary, bulk-copy the longest run of bytes that need no escaping using string_bulk_run() - the same SWAR scanner and UTF-8 bulk validator the lexer's contiguous path uses - and only fall back to the byte-at-a-time DFA loop for the first byte that needs individual handling (a quote, backslash, control character, or ill-formed/truncated UTF-8). Because every "hard" or invalid byte is still processed by the unchanged byte path, escaping output and error handling (including strict-mode error 316 position and message) are byte-identical to before. The ensure_ascii=true path is unchanged: it must escape non-ASCII and 0x7F, which string_bulk_run does not stop on, so a separate predicate would be needed for it. Verified byte-for-byte identical dump output against the pre-change implementation across ~20k randomized byte strings plus curated edge cases (all escapes, control chars, valid multibyte, surrogates, overlong, truncated sequences) for both ensure_ascii settings and all three error handlers, in C++11/17/20 at -O2/-O3. Throughput (g++ -O3, ensure_ascii=false, vs pre-change): long ASCII strings 4.2x twitter-like objects 2.3x dense CJK 1.4x (further headroom with JSON_USE_SIMDUTF) Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann * Buffer serializer output and add ensure_ascii string fast path Two further serialization speedups on top of the ensure_ascii=false bulk copy, both reusing the SWAR primitives in detail/input/string_scan.hpp. 1. Internal write buffer (devirtualization). Every structural character ('{', '"', ',', ...) previously went straight to the output adapter through a virtual call. Route all writes through put_char/put_chars into a 1 KiB buffer that flushes in bulk; the public dump() flushes once the top-level value is done (the recursive worker is split out as dump_internal). Runs larger than the buffer are written straight through, so large payloads are not copied twice. This is the dominant cost for object/array-heavy values. 2. ensure_ascii fast path. dump_escaped previously ran the UTF-8 DFA over every byte when escaping non-ASCII. Add find_ascii_copyable_run() (a SWAR scan stopping at '"', '\\', < 0x20, 0x7F, and >= 0x80) so runs of printable ASCII are bulk-copied, with the byte path handling each escape/non-ASCII byte exactly as before. Behavior is unchanged: dump output is byte-for-byte identical to the previous implementation across ~20k randomized byte strings plus curated edge cases (all escapes, control chars, 0x7F, valid multibyte, surrogates, overlong, truncated), for object/array/pretty output, both ensure_ascii settings, and all three error handlers, in C++11/17/20 at -O2/-O3. New unit tests cover the buffer flush boundaries, the escape and 0x7F handling, multibyte under both settings, and invalid-UTF-8 handling. Throughput (g++ -O3, vs the ensure_ascii=false-only baseline): long ASCII, ensure_ascii=0 4.2x long ASCII, ensure_ascii=1 4.1x twitter-like objects 2.7x dense CJK 1.8x Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann * Flush serializer buffer in dump_escaped unit test test-convenience failed (macOS finished first; the failure is platform-independent) because check_escaped() calls the internal serializer::dump_escaped() directly and then reads the output stream. Since dump_escaped() now writes into the serializer's internal write buffer, the bytes were still buffered and the stream was empty. Expose flush() under JSON_PRIVATE_UNLESS_TESTED (same visibility as dump_escaped) and flush in check_escaped() before inspecting the output. Per-string flushing inside dump_escaped() was rejected on purpose: it would defeat the buffering that makes object/array-heavy dumps faster. Library behavior is unchanged (flush()'s body is identical; only its access label moved). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann * Avoid deep recursion in serialization write-buffer test The "many small structural writes exceed the write buffer" subcase built a 1100-deep nested array and dumped it to force >1024 consecutive single-character writes through put_char (exercising the write buffer's flush-when-full branch). dump() recurses per nesting level, so on MSVC debug builds (smaller default stack, larger frames) this overflowed the stack and crashed test-serialization; Linux/macOS have enough headroom to hide it. Replace the nesting with a flat array of 500 empty strings. Each element emits '"', '"', ',' via put_char, so the dump is a long run of single-character writes (1501 bytes > the 1024-byte buffer) at nesting depth two, hitting the same flush branch without deep recursion. Library code is unchanged. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG Signed-off-by: Niels Lohmann * Split the write-buffer helpers and write indentation directly Follow-up to @gregmarr's review: put_chars() was doing four unrelated jobs, so give the two that can be made safe their own entry points. - put_literal(): takes the literal by reference and deduces the length from the array bound, so the 27 hand-counted lengths at the call sites can no longer drift from the literals they describe. A literal is checked at compile time to fit the buffer, so this path needs no write-through branch. - put_buffer(): takes the fixed-size buffer itself rather than a bare pointer, so the length can be checked against the buffer's own bound. - put_indent(): memsets the indentation into the write buffer, filling and flushing it as needed. This removes indent_string entirely, and with it both bugs of #5186: the indentation string was grown by doubling, which is not enough when indent_step more than doubles it (a heap over-read - dump(2000) read 2000 bytes out of a 1024-byte string), and the grown part was filled with a space instead of the configured indent_char. next_indent() keeps that PR's assertion against the unsigned indentation accumulation wrapping on deep nesting. put_chars() keeps the two cases that are genuinely a pointer and a count: the run-length copies out of the string being escaped, and to_chars() output. Tests cover an indent_step wider than the write buffer, a non-space indentation character past the old growth point, and nesting whose accumulated indentation spans several buffer-fulls. All three fail against develop. Signed-off-by: Niels Lohmann * Fill the indentation buffer once instead of once per flush @gregmarr's point on the fill-and-flush loop: flushing does not disturb what the write buffer holds, so an indentation spanning several buffer-fulls only has to be written into the buffer once and can then be handed to the adapter as many times as needed. The loop re-filled it every time, doing work it already knew was there. put_indent() now fills the room left in the buffer, and if anything remains, flushes, fills the buffer once, and re-flushes that same content. It also returns early for a zero-width indentation, which is what the closing brace of every outermost value asks for. Measured over a dump(), counting memset calls and bytes inside put_indent: indent before after 4 1 call / 4 B 1 call / 4 B 2000 2 calls / 2000 B 2 calls / 2046 B 100000 98 calls / 100000 B 2 calls / 2046 B The wide case is now constant work rather than proportional to the indentation width; ordinary widths are unchanged. Tests extended to cover several whole buffer-fulls and an exact multiple of the buffer size. Signed-off-by: Niels Lohmann * Tighten the write-buffer helpers after review More of @gregmarr's review on the put_* split: - Reattach the put_chars() doc comment, which the new helpers had been inserted in front of, leaving it describing put_indent(). - Compute the literal length once in put_literal() instead of spelling N - 1 at each use. - Add put_string(str, start, end), which keeps the pointer arithmetic and the bounds assertions inside the function instead of at the call site. With dump_float()'s to_chars() output moved onto put_buffer() as well, put_chars() now has no callers outside put_string()/put_buffer(): nothing passes a bare pointer and a count any more. - Carry the indentation as std::size_t rather than unsigned int. It is a size, it is compared and combined with buffer sizes throughout, and the casts in put_indent() disappear. next_indent() keeps its assertion, which is far harder to trip on a 64-bit size_t but still reachable where that is 32 bits. No output change: pretty and compact dumps, binary values included, are byte-identical to develop. Signed-off-by: Niels Lohmann * Silence avoid-c-arrays on put_literal's array reference clang-tidy flags the reference-to-array parameter under cppcoreguidelines/hicpp/modernize-avoid-c-arrays, and the CI treats warnings as errors. Binding to the array is the whole point here - it is what lets the length be deduced from the literal instead of hand-written at the call site - so suppress it the same way from_json(), to_json() and get_to() already suppress it for their own T (&arr)[N] parameters. Signed-off-by: Niels Lohmann * Bound the descent of dump() Serializing a container serializes its elements, so dump() descended into one call per nesting level. A value nested deeply enough exhausted the call stack and terminated the process with a segmentation fault - no exception, nothing the caller could catch. Parsing such a value works, as the parser is iterative, and so does destroying one, as #1436 made destruction iterative. Bound how far the descent goes rather than take the call stack away from it. The first 128 levels are written by exactly the code that always wrote them, and only below that does dump_iteratively write out what is left, keeping the containers it has entered on an explicit stack. Serializing can therefore no longer exhaust the stack, however deeply a value is nested, while a value nested less deeply than the bound pays only for one comparison per container. Writing every value that way instead measured between 2% and 20% slower - 20% on object-heavy documents - which is why the descent is kept for all but the values that cannot afford it. The bound costs nothing measurable: between -1.4% and +1.2% across compact and pretty output of number, integer, string, object-heavy, wide-object and deeply nested documents. The output is unchanged for every value. Both ways of writing a container emit the separator in front of every element but the first, rather than after every element but the last, which puts exactly one between each pair and none at the end. This fixes #5387 for dump(). The copy constructor is fixed in #5389. Signed-off-by: Niels Lohmann * Fold ensure_ascii into the escaper and write bytes without dump_integer Two hot spots that the write buffer and the bulk scanner left behind. dump_escaped took ensure_ascii as a runtime flag and tested it inside the loop, once per character run, although it cannot change while a string is written. It is now a template parameter, dispatched once per string, which folds the choice of scanner and lets each of the two be inlined into a loop of its own. This is the hottest loop in the serializer: it runs over every string and every object key. A binary value's bytes went through dump_integer, which counts digits and does 64-bit arithmetic for a number that is always in [0, 255]. dump_byte writes the three digits it takes at most straight into the write buffer instead. Any byte type that is not a plain unsigned byte is still left to dump_integer, whose representation of it may differ. Measured against the previous commit (medians of 9 interleaved runs, clang -O3): binary values -33.8%, dense CJK with ensure_ascii -20.6%, key-heavy objects -17.8%, deeply nested pretty output -17.9%, dense CJK without ensure_ascii -11.8%, object-heavy documents -9.3% compact and -9.5% pretty, a small value dumped in a loop -21.4%, wide objects -2.3%. Arrays of plain ASCII strings measured 3.5% to 4.2% slower, the one shape that loses; number and integer arrays are unchanged. Also tried and dropped: leaving the write and string buffers uninitialized rather than zeroing 1.5 KB per dump() call. It is worth -30% on small values, but two nearly identical string workloads moved 18% apart in opposite directions, so the measurements did not support it. The output is unchanged for every value: the differential now also covers every one of the 256 byte values, alone and together, in both binary layouts. Signed-off-by: Niels Lohmann * Write a byte without walking a pointer over the buffer clang-tidy's misc-const-correctness reads the pointer dump_byte advanced over the write buffer as one whose pointee could be const. Index the buffer instead, which says the same thing without a raw pointer at all. Signed-off-by: Niels Lohmann * Parenthesize the reserve arithmetic in the deep-nesting test clang-tidy's readability-math-missing-parentheses wants the multiplication spelled out in reserve(6 * depth + 1), and CI treats its warnings as errors. Signed-off-by: Niels Lohmann * Do not scan for a copyable run that cannot exist Under ensure_ascii, dump_escaped() calls find_ascii_copyable_run() at every character boundary. When the text is dense non-ASCII - CJK, where every byte is >= 0x80 - the scanner stops on its first byte and returns zero, so its SWAR block runs once per character and buys nothing, on top of the escaping that still has to happen afterwards. A run can only be non-empty when the first byte is one the scanner may copy, so test that single byte before calling it. Runs that do exist are found exactly as before, so the bulk-copy win is unchanged; only the calls that were always going to return zero are skipped. Output is unchanged: the dump digest over canada/citm/twitter, in compact, pretty and ensure_ascii form, matches develop byte for byte. dump(ensure_ascii=true) develop before after CJK text 3.54ms 4.25ms 3.36ms CJK, no ASCII at all 3.09ms 4.02ms 3.02ms Latin-1-ish text 4.39ms 3.04ms 2.93ms plain ASCII 3.92ms 0.80ms 0.79ms Signed-off-by: Niels Lohmann * Address review of the write-buffer helpers Three points from @gregmarr's review: put_chars() is gone. It was the only entry point taking a bare pointer and a count, and it existed only so put_string() and put_buffer() had something to delegate to. Its body now lives in put_string(), and put_buffer() is put_string(buffer, 0, length) - std::array already carries data() and size(), so it satisfies the same interface a string does. Nothing appends characters without a bound any more. dump_escaped()'s documentation block was duplicated. The dispatcher was inserted between the original comment and the function it described, and the comment was copied rather than split. The worker now has its own short comment saying why ensure_ascii is a template parameter. The local in dump_byte() is deliberate, and is now documented as such: writing through write_buffer[] is a char write, which may alias any object, so with write_buffer_pos updated in place the compiler must reload and store it around every digit. Measured on a dump of a 4 MiB binary value, 18.0 ms without the local against 7.4 ms with it. Output is unchanged: byte-identical dumps across 77 files in compact, pretty, ensure_ascii, pretty+ascii, indent 600 and tab-indent form. Signed-off-by: Niels Lohmann * Address review: drop unneeded backslash-escapes and duplicate scan loop '"' does not need escaping in a char literal, unlike in a string literal. find_ascii_copyable_run() also duplicated the byte-at-a-time search that already exists as the loop's own scalar tail; break into it instead of re-deriving the offset in a second, near-identical loop. Signed-off-by: Niels Lohmann * Move pretty_print, ensure_ascii and indent_step into the serializer None of these change over the life of a serializer, unlike current_indent and depth, which do change on every recursive call. They are now captured once in the constructor - matching indent_char and error_handler - instead of being threaded through dump(), dump_internal(), dump_iteratively(), dump_value() and dump_escaped() on every call. Signed-off-by: Niels Lohmann * Stop the serializer from holding onto std::localeconv()'s pointer loc was only ever read twice, immediately, to seed thousands_sep and decimal_point; nothing else in the class used it. A local in the constructor body serves the same purpose without keeping the pointer around for the serializer's lifetime. Signed-off-by: Niels Lohmann * Keep thousands_sep/decimal_point const via a small locale_chars struct const members can't be assigned in a constructor body, so seeding them from std::localeconv() meant either dropping const or holding onto the lconv* for longer than needed. A sub-object computes both from the pointer in its own constructor and is itself initialized in serializer's mem-initializer-list, so the two chars stay const, std::localeconv() is still called exactly once, and nothing outlives the constructor. Signed-off-by: Niels Lohmann --------- Signed-off-by: Niels Lohmann Co-authored-by: Claude Opus 4.8 --- include/nlohmann/detail/input/string_scan.hpp | 46 + include/nlohmann/detail/output/serializer.hpp | 967 +++++++++++++--- include/nlohmann/json.hpp | 14 +- single_include/nlohmann/json.hpp | 1028 ++++++++++++++--- tests/src/unit-convenience.cpp | 5 +- tests/src/unit-serialization.cpp | 229 ++++ 6 files changed, 2009 insertions(+), 280 deletions(-) diff --git a/include/nlohmann/detail/input/string_scan.hpp b/include/nlohmann/detail/input/string_scan.hpp index dc5b07a54..6af0e6c5d 100644 --- a/include/nlohmann/detail/input/string_scan.hpp +++ b/include/nlohmann/detail/input/string_scan.hpp @@ -93,6 +93,52 @@ inline std::size_t find_string_special(const unsigned char* data, std::size_t n) return n; } +// classify a byte as one the serializer must NOT copy verbatim when +// ensure_ascii is requested: the closing quote, an escape, a control character +// (< 0x20), DEL (0x7F), or any non-ASCII byte (>= 0x80). Everything else - +// printable ASCII except '"' and '\\' - is emitted unchanged. Note this differs +// from is_string_special() only in that 0x7F is also a stop (it is escaped as +// \u007f under ensure_ascii). +inline bool is_ascii_copyable(unsigned char c) noexcept +{ + return c >= 0x20u && c < 0x7Fu && c != '"' && c != '\\'; +} + +// return the index of the first byte in [data, data+n) that is NOT +// is_ascii_copyable(), or n if every byte can be copied verbatim; scans 8 bytes +// at a time. Used by the serializer's ensure_ascii fast path. +inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_t n) noexcept +{ + constexpr std::uint64_t ones = 0x0101010101010101ull; + constexpr std::uint64_t high = 0x8080808080808080ull; + std::size_t i = 0; + for (; i + 8 <= n; i += 8) + { + std::uint64_t v = 0; + std::memcpy(&v, data + i, sizeof(v)); + const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22) + const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C) + const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL (0x7F) + const std::uint64_t stop = ((q - ones) & ~q & high) // == '"' + | ((b - ones) & ~b & high) // == '\\' + | ((d - ones) & ~d & high) // == 0x7F + | ((v - 0x2020202020202020ull) & ~v & high) // < 0x20 + | (v & high); // >= 0x80 + if (stop != 0) + { + break; + } + } + for (; i < n; ++i) + { + if (!is_ascii_copyable(data[i])) + { + return i; + } + } + return n; +} + // Validate one UTF-8 sequence at the front of [data, data+avail). Returns its // length (2..4) only when the bytes form a *well-formed* sequence using exactly // the same ranges as scan_string()'s per-byte switch, so the bulk path accepts diff --git a/include/nlohmann/detail/output/serializer.hpp b/include/nlohmann/detail/output/serializer.hpp index 857fc2445..3dd9162df 100644 --- a/include/nlohmann/detail/output/serializer.hpp +++ b/include/nlohmann/detail/output/serializer.hpp @@ -9,20 +9,23 @@ #pragma once -#include // reverse, remove, fill, find, none_of +#include // reverse, remove, fill, find, none_of, min #include // array #include // localeconv, lconv #include // labs, isfinite, isnan, signbit #include // size_t, ptrdiff_t #include // uint8_t #include // snprintf +#include // memcpy, memset #include // numeric_limits #include // string, char_traits #include // is_same #include // move +#include // vector #include #include +#include #include #include #include @@ -61,16 +64,29 @@ class serializer /*! @param[in] s output stream to serialize to @param[in] ichar indentation character to use + @param[in] pretty_print_ whether the output shall be pretty-printed + @param[in] ensure_ascii_ If @a ensure_ascii_ is true, all non-ASCII + characters in the output are escaped with `\uXXXX` sequences, and the + result consists of ASCII characters only. + @param[in] indent_step_ the indent level @param[in] error_handler_ how to react on decoding errors + + None of @a pretty_print_, @a ensure_ascii_ and @a indent_step_ change over + the life of the serializer, so they are captured once here instead of + being threaded through every call to @ref dump, @ref dump_internal and + @ref dump_iteratively. */ serializer(output_adapter_t s, const char ichar, + const bool pretty_print_ = false, + const bool ensure_ascii_ = false, + const std::size_t indent_step_ = 0, error_handler_t error_handler_ = error_handler_t::strict) : o(std::move(s)) - , loc(std::localeconv()) - , thousands_sep(loc->thousands_sep == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->thousands_sep))) - , decimal_point(loc->decimal_point == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->decimal_point))) + , locale(std::localeconv()) , indent_char(ichar) - , indent_string(512, indent_char) + , pretty_print(pretty_print_) + , ensure_ascii(ensure_ascii_) + , indent_step(indent_step_) , error_handler(error_handler_) {} @@ -86,8 +102,8 @@ class serializer This function is called by the public member function dump and organizes the serialization internally. The indentation level is propagated as - additional parameter. In case of arrays and objects, the function is - called recursively. + additional parameter. Arrays and objects are serialized without recursion, + however deeply they are nested. - strings and object keys are escaped using `escape_string()` - integer numbers are converted implicitly via `operator<<` @@ -96,89 +112,109 @@ class serializer byte array @param[in] val value to serialize - @param[in] pretty_print whether the output shall be pretty-printed - @param[in] ensure_ascii If @a ensure_ascii is true, all non-ASCII characters - in the output are escaped with `\uXXXX` sequences, and the result consists - of ASCII characters only. - @param[in] indent_step the indent level @param[in] current_indent the current indent level (only used internally) */ void dump(const BasicJsonType& val, - const bool pretty_print, - const bool ensure_ascii, - const unsigned int indent_step, - const unsigned int current_indent = 0) + const std::size_t current_indent = 0) + { + dump_internal(val, current_indent); + flush(); + } + + JSON_PRIVATE_UNLESS_TESTED: + /*! + @brief worker for @ref dump + + Identical in behavior to the historical @ref dump, but writes into the + serializer's internal @ref write_buffer instead of issuing a virtual call + per token. The public @ref dump wraps this and flushes the buffer once the + top-level value has been serialized. + + Serializing a container descends into its elements, so a value nested deeply + enough used to exhaust the call stack and terminate the process with no + exception to catch. The descent is bounded here: once @ref dump_depth_limit + levels have been entered, @ref dump_iteratively writes out what is left + without the call stack. A value nested less deeply than that - all but a + vanishing minority - is written by exactly the code that always wrote it. + + @sa https://github.com/nlohmann/json/issues/5387 + */ + void dump_internal(const BasicJsonType& val, + const std::size_t current_indent = 0, + const std::size_t depth = 0) { switch (val.m_data.m_type) { case value_t::object: { + if (JSON_HEDLEY_UNLIKELY(depth >= dump_depth_limit())) + { + dump_iteratively(val, current_indent); + return; + } + if (val.m_data.m_value.object->empty()) { - o->write_characters("{}", 2); + put_literal("{}"); return; } if (pretty_print) { - o->write_characters("{\n", 2); + put_literal("{\n"); // variable to hold indentation for recursive calls - const auto new_indent = current_indent + indent_step; - if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) - { - indent_string.resize(indent_string.size() * 2, ' '); - } + const auto new_indent = next_indent(current_indent, indent_step); // first n-1 elements auto i = val.m_data.m_value.object->cbegin(); for (std::size_t cnt = 0; cnt < val.m_data.m_value.object->size() - 1; ++cnt, ++i) { - o->write_characters(indent_string.c_str(), new_indent); - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\": ", 3); - dump(i->second, true, ensure_ascii, indent_step, new_indent); - o->write_characters(",\n", 2); + put_indent(new_indent); + put_char('"'); + dump_escaped(i->first); + put_literal("\": "); + dump_internal(i->second, new_indent, depth + 1); + put_literal(",\n"); } // last element JSON_ASSERT(i != val.m_data.m_value.object->cend()); JSON_ASSERT(std::next(i) == val.m_data.m_value.object->cend()); - o->write_characters(indent_string.c_str(), new_indent); - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\": ", 3); - dump(i->second, true, ensure_ascii, indent_step, new_indent); + put_indent(new_indent); + put_char('"'); + dump_escaped(i->first); + put_literal("\": "); + dump_internal(i->second, new_indent, depth + 1); - o->write_character('\n'); - o->write_characters(indent_string.c_str(), current_indent); - o->write_character('}'); + put_char('\n'); + put_indent(current_indent); + put_char('}'); } else { - o->write_character('{'); + put_char('{'); // first n-1 elements auto i = val.m_data.m_value.object->cbegin(); for (std::size_t cnt = 0; cnt < val.m_data.m_value.object->size() - 1; ++cnt, ++i) { - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\":", 2); - dump(i->second, false, ensure_ascii, indent_step, current_indent); - o->write_character(','); + put_char('"'); + dump_escaped(i->first); + put_literal("\":"); + dump_internal(i->second, current_indent, depth + 1); + put_char(','); } // last element JSON_ASSERT(i != val.m_data.m_value.object->cend()); JSON_ASSERT(std::next(i) == val.m_data.m_value.object->cend()); - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\":", 2); - dump(i->second, false, ensure_ascii, indent_step, current_indent); + put_char('"'); + dump_escaped(i->first); + put_literal("\":"); + dump_internal(i->second, current_indent, depth + 1); - o->write_character('}'); + put_char('}'); } return; @@ -186,58 +222,60 @@ class serializer case value_t::array: { + if (JSON_HEDLEY_UNLIKELY(depth >= dump_depth_limit())) + { + dump_iteratively(val, current_indent); + return; + } + if (val.m_data.m_value.array->empty()) { - o->write_characters("[]", 2); + put_literal("[]"); return; } if (pretty_print) { - o->write_characters("[\n", 2); + put_literal("[\n"); // variable to hold indentation for recursive calls - const auto new_indent = current_indent + indent_step; - if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) - { - indent_string.resize(indent_string.size() * 2, ' '); - } + const auto new_indent = next_indent(current_indent, indent_step); // first n-1 elements for (auto i = val.m_data.m_value.array->cbegin(); i != val.m_data.m_value.array->cend() - 1; ++i) { - o->write_characters(indent_string.c_str(), new_indent); - dump(*i, true, ensure_ascii, indent_step, new_indent); - o->write_characters(",\n", 2); + put_indent(new_indent); + dump_internal(*i, new_indent, depth + 1); + put_literal(",\n"); } // last element JSON_ASSERT(!val.m_data.m_value.array->empty()); - o->write_characters(indent_string.c_str(), new_indent); - dump(val.m_data.m_value.array->back(), true, ensure_ascii, indent_step, new_indent); + put_indent(new_indent); + dump_internal(val.m_data.m_value.array->back(), new_indent, depth + 1); - o->write_character('\n'); - o->write_characters(indent_string.c_str(), current_indent); - o->write_character(']'); + put_char('\n'); + put_indent(current_indent); + put_char(']'); } else { - o->write_character('['); + put_char('['); // first n-1 elements for (auto i = val.m_data.m_value.array->cbegin(); i != val.m_data.m_value.array->cend() - 1; ++i) { - dump(*i, false, ensure_ascii, indent_step, current_indent); - o->write_character(','); + dump_internal(*i, current_indent, depth + 1); + put_char(','); } // last element JSON_ASSERT(!val.m_data.m_value.array->empty()); - dump(val.m_data.m_value.array->back(), false, ensure_ascii, indent_step, current_indent); + dump_internal(val.m_data.m_value.array->back(), current_indent, depth + 1); - o->write_character(']'); + put_char(']'); } return; @@ -245,9 +283,9 @@ class serializer case value_t::string: { - o->write_character('\"'); - dump_escaped(*val.m_data.m_value.string, ensure_ascii); - o->write_character('\"'); + put_char('"'); + dump_escaped(*val.m_data.m_value.string); + put_char('"'); return; } @@ -255,70 +293,66 @@ class serializer { if (pretty_print) { - o->write_characters("{\n", 2); + put_literal("{\n"); // variable to hold indentation for recursive calls - const auto new_indent = current_indent + indent_step; - if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) - { - indent_string.resize(indent_string.size() * 2, ' '); - } + const auto new_indent = next_indent(current_indent, indent_step); - o->write_characters(indent_string.c_str(), new_indent); + put_indent(new_indent); - o->write_characters("\"bytes\": [", 10); + put_literal("\"bytes\": ["); if (!val.m_data.m_value.binary->empty()) { for (auto i = val.m_data.m_value.binary->cbegin(); i != val.m_data.m_value.binary->cend() - 1; ++i) { - dump_integer(*i); - o->write_characters(", ", 2); + dump_byte(*i); + put_literal(", "); } - dump_integer(val.m_data.m_value.binary->back()); + dump_byte(val.m_data.m_value.binary->back()); } - o->write_characters("],\n", 3); - o->write_characters(indent_string.c_str(), new_indent); + put_literal("],\n"); + put_indent(new_indent); - o->write_characters("\"subtype\": ", 11); + put_literal("\"subtype\": "); if (val.m_data.m_value.binary->has_subtype()) { dump_integer(val.m_data.m_value.binary->subtype()); } else { - o->write_characters("null", 4); + put_literal("null"); } - o->write_character('\n'); - o->write_characters(indent_string.c_str(), current_indent); - o->write_character('}'); + put_char('\n'); + put_indent(current_indent); + put_char('}'); } else { - o->write_characters("{\"bytes\":[", 10); + put_literal("{\"bytes\":["); if (!val.m_data.m_value.binary->empty()) { for (auto i = val.m_data.m_value.binary->cbegin(); i != val.m_data.m_value.binary->cend() - 1; ++i) { - dump_integer(*i); - o->write_character(','); + dump_byte(*i); + put_char(','); } - dump_integer(val.m_data.m_value.binary->back()); + dump_byte(val.m_data.m_value.binary->back()); } - o->write_characters("],\"subtype\":", 12); + put_literal("],\"subtype\":"); if (val.m_data.m_value.binary->has_subtype()) { dump_integer(val.m_data.m_value.binary->subtype()); - o->write_character('}'); + put_char('}'); } else { - o->write_characters("null}", 5); + put_literal("null}"); } } return; @@ -328,11 +362,11 @@ class serializer { if (val.m_data.m_value.boolean) { - o->write_characters("true", 4); + put_literal("true"); } else { - o->write_characters("false", 5); + put_literal("false"); } return; } @@ -357,13 +391,13 @@ class serializer case value_t::discarded: { - o->write_characters("", 11); + put_literal(""); return; } case value_t::null: { - o->write_characters("null", 4); + put_literal("null"); return; } @@ -372,6 +406,367 @@ class serializer } } + private: + /// the number of levels @ref dump_internal descends into before it hands + /// over to @ref dump_iteratively + static constexpr std::size_t dump_depth_limit() + { + return 128; + } + + /*! + @brief write out @a val and everything below it without the call stack + + Emits the same bytes as @ref dump_internal, keeping the containers it has + entered on an explicit stack instead of descending into them. Only reached + for values nested deeper than @ref dump_depth_limit, which is why it is not + written for speed: walking every value this way measured up to 20% slower on + object-heavy documents than letting the compiler drive the descent. + */ + void dump_iteratively(const BasicJsonType& val, + const std::size_t current_indent = 0) + { + // Scalars, empty containers and binary values are written by dump_value + // alone, so nothing is allocated for them: only a container with + // elements is ever pushed. + std::vector stack; + + dump_value(val, current_indent, stack); + + while (!stack.empty()) + { + dump_frame& frame = stack.back(); + + if (frame.value->m_data.m_type == value_t::object) + { + const auto* object = frame.value->m_data.m_value.object; + + if (frame.object_it == object->cend()) + { + if (pretty_print) + { + put_char('\n'); + put_indent(frame.current_indent); + } + + put_char('}'); + stack.pop_back(); + continue; + } + + // the separator goes in front of every element but the first, + // which puts exactly one between each pair and none at the end + if (frame.object_it != object->cbegin()) + { + if (pretty_print) + { + put_literal(",\n"); + } + else + { + put_char(','); + } + } + + if (pretty_print) + { + put_indent(frame.child_indent); + } + + put_char('"'); + dump_escaped(frame.object_it->first); + + if (pretty_print) + { + put_literal("\": "); + } + else + { + put_literal("\":"); + } + + const BasicJsonType& element = frame.object_it->second; + ++frame.object_it; + + // read everything needed from the frame before this: entering a + // container pushes another one and can move them all + const std::size_t element_indent = frame.child_indent; + dump_value(element, element_indent, stack); + } + else + { + const auto* array = frame.value->m_data.m_value.array; + + if (frame.array_it == array->cend()) + { + if (pretty_print) + { + put_char('\n'); + put_indent(frame.current_indent); + } + + put_char(']'); + stack.pop_back(); + continue; + } + + if (frame.array_it != array->cbegin()) + { + if (pretty_print) + { + put_literal(",\n"); + } + else + { + put_char(','); + } + } + + if (pretty_print) + { + put_indent(frame.child_indent); + } + + const BasicJsonType& element = *frame.array_it; + ++frame.array_it; + + // see above + const std::size_t element_indent = frame.child_indent; + dump_value(element, element_indent, stack); + } + } + } + + private: + /// @brief a container that has been opened but not closed yet + struct dump_frame + { + dump_frame(const BasicJsonType* value_, const std::size_t current_indent_, + const std::size_t child_indent_) noexcept + : value(value_) + , current_indent(current_indent_) + , child_indent(child_indent_) + {} + + /// the object or array being serialized + const BasicJsonType* value; + /// the element to serialize next; which of the two is live follows from + /// the type of @a value. They are kept side by side rather than in a + /// union, which would need its special members written out by hand, see + /// detail/iterators/internal_iterator.hpp + typename BasicJsonType::object_t::const_iterator object_it{}; + typename BasicJsonType::array_t::const_iterator array_it{}; + /// the indentation of the container itself, used by its closing bracket + std::size_t current_indent; + /// the indentation of the container's elements + std::size_t child_indent; + }; + + /*! + @brief serialize the value @a val, but not the elements of a container + + An object or array with elements is opened and pushed onto @a stack for + @ref dump_internal to walk; everything else - including a binary value, + which looks like an object but has no elements to descend into - is written + out here in full. + */ + void dump_value(const BasicJsonType& val, + const std::size_t current_indent, + std::vector& stack) + { + switch (val.m_data.m_type) + { + case value_t::object: + { + if (val.m_data.m_value.object->empty()) + { + put_literal("{}"); + return; + } + + std::size_t child_indent = current_indent; + + if (pretty_print) + { + put_literal("{\n"); + child_indent = next_indent(current_indent, indent_step); + } + else + { + put_char('{'); + } + + stack.emplace_back(&val, current_indent, child_indent); + stack.back().object_it = val.m_data.m_value.object->cbegin(); + return; + } + + case value_t::array: + { + if (val.m_data.m_value.array->empty()) + { + put_literal("[]"); + return; + } + + std::size_t child_indent = current_indent; + + if (pretty_print) + { + put_literal("[\n"); + child_indent = next_indent(current_indent, indent_step); + } + else + { + put_char('['); + } + + stack.emplace_back(&val, current_indent, child_indent); + stack.back().array_it = val.m_data.m_value.array->cbegin(); + return; + } + + case value_t::string: + { + put_char('"'); + dump_escaped(*val.m_data.m_value.string); + put_char('"'); + return; + } + + case value_t::binary: + { + if (pretty_print) + { + put_literal("{\n"); + + // variable to hold indentation for the bytes + const auto new_indent = next_indent(current_indent, indent_step); + + put_indent(new_indent); + + put_literal("\"bytes\": ["); + + if (!val.m_data.m_value.binary->empty()) + { + for (auto i = val.m_data.m_value.binary->cbegin(); + i != val.m_data.m_value.binary->cend() - 1; ++i) + { + dump_byte(*i); + put_literal(", "); + } + dump_byte(val.m_data.m_value.binary->back()); + } + + put_literal("],\n"); + put_indent(new_indent); + + put_literal("\"subtype\": "); + if (val.m_data.m_value.binary->has_subtype()) + { + dump_integer(val.m_data.m_value.binary->subtype()); + } + else + { + put_literal("null"); + } + put_char('\n'); + put_indent(current_indent); + put_char('}'); + } + else + { + put_literal("{\"bytes\":["); + + if (!val.m_data.m_value.binary->empty()) + { + for (auto i = val.m_data.m_value.binary->cbegin(); + i != val.m_data.m_value.binary->cend() - 1; ++i) + { + dump_byte(*i); + put_char(','); + } + dump_byte(val.m_data.m_value.binary->back()); + } + + put_literal("],\"subtype\":"); + if (val.m_data.m_value.binary->has_subtype()) + { + dump_integer(val.m_data.m_value.binary->subtype()); + put_char('}'); + } + else + { + put_literal("null}"); + } + } + return; + } + + case value_t::boolean: + { + if (val.m_data.m_value.boolean) + { + put_literal("true"); + } + else + { + put_literal("false"); + } + return; + } + + case value_t::number_integer: + { + dump_integer(val.m_data.m_value.number_integer); + return; + } + + case value_t::number_unsigned: + { + dump_integer(val.m_data.m_value.number_unsigned); + return; + } + + case value_t::number_float: + { + dump_float(val.m_data.m_value.number_float); + return; + } + + case value_t::discarded: + { + put_literal(""); + return; + } + + case value_t::null: + { + put_literal("null"); + return; + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE + } + } + + + /*! + @brief the indentation level to use for the children of the current value + + A very large @a indent_step can wrap the unsigned accumulation on deep + nesting, which would silently truncate the indentation. Far harder to reach + now that the accumulator is a std::size_t, but still reachable where that is + 32 bits wide. + */ + static std::size_t next_indent(const std::size_t current_indent, const std::size_t indent_step) + { + const std::size_t new_indent = current_indent + indent_step; + JSON_ASSERT(new_indent >= current_indent); + return new_indent; + } + JSON_PRIVATE_UNLESS_TESTED: /*! @brief dump escaped string @@ -382,12 +777,32 @@ class serializer representation. The escaped string is written to output stream @a o. @param[in] s the string to escape - @param[in] ensure_ascii whether to escape non-ASCII characters with - \uXXXX sequences @complexity Linear in the length of string @a s. */ - void dump_escaped(const string_t& s, const bool ensure_ascii) + void dump_escaped(const string_t& s) + { + // dispatch once here rather than test the flag inside the loop: it does + // not change while a string is written, and folding it lets each of the + // two scanners be inlined into a loop of its own + if (ensure_ascii) + { + dump_escaped_impl(s); + } + else + { + dump_escaped_impl(s); + } + } + + /*! + @brief worker for @ref dump_escaped + + @a ensure_ascii is a template parameter here so that the branch on it is + resolved once, outside the loop; see @ref dump_escaped. + */ + template + void dump_escaped_impl(const string_t& s) { std::uint32_t codepoint{}; std::uint8_t state = UTF8_ACCEPT; @@ -399,6 +814,56 @@ class serializer for (std::size_t i = 0; i < s.size(); ++i) { + // Fast path: at a character boundary (state == UTF8_ACCEPT), + // bulk-copy the longest run of bytes that need no escaping using a + // SWAR scanner shared with the lexer's contiguous path. The scanner + // stops exactly at the first byte dump_escaped would handle + // individually, so that byte is left to the byte-at-a-time path + // below, keeping escaping output and error diagnostics unchanged. + // + // - EnsureAscii == false: string_bulk_run() copies ordinary bytes + // and complete well-formed UTF-8, stopping at a quote, backslash, + // control character (< 0x20), or ill-formed/truncated sequence. + // - EnsureAscii == true: only printable ASCII may be copied + // verbatim; find_ascii_copyable_run() additionally stops at 0x7F + // and every non-ASCII byte (>= 0x80), which must be \u-escaped. + if (state == UTF8_ACCEPT) + { + const auto* const data = reinterpret_cast(s.data()); + // A run can only be non-empty when the very first byte is one + // the scanner may copy, so test that single byte before paying + // for the scan. Without it, text whose characters all have to be + // escaped - CJK under ensure_ascii, where every byte is >= 0x80 - + // runs the scanner once per character only to be told zero. + std::size_t run = 0; + if (!EnsureAscii) + { + run = string_bulk_run(data + i, s.size() - i); + } + else if (is_ascii_copyable(data[i])) + { + run = find_ascii_copyable_run(data + i, s.size() - i); + } + if (run != 0) + { + // emit any bytes still pending in string_buffer first to + // preserve output order, then write the run directly + if (bytes != 0) + { + put_buffer(string_buffer, bytes); + bytes = 0; + } + put_string(s, i, i + run); + bytes_after_last_accept = 0; + undumped_chars = 0; + i += run; + if (i >= s.size()) + { + break; + } + } + } + const auto byte = static_cast(s[i]); switch (decode(state, codepoint, byte)) @@ -445,7 +910,7 @@ class serializer case 0x22: // quotation mark { string_buffer[bytes++] = '\\'; - string_buffer[bytes++] = '\"'; + string_buffer[bytes++] = '"'; break; } @@ -459,8 +924,8 @@ class serializer default: { // escape control characters (0x00..0x1F) or, if - // ensure_ascii parameter is used, non-ASCII characters - if ((codepoint <= 0x1F) || (ensure_ascii && (codepoint >= 0x7F))) + // EnsureAscii parameter is used, non-ASCII characters + if ((codepoint <= 0x1F) || (EnsureAscii && (codepoint >= 0x7F))) { if (codepoint <= 0xFFFF) { @@ -487,7 +952,7 @@ class serializer // written ("\uxxxx\uxxxx\0") for one code point if (string_buffer.size() - bytes < 13) { - o->write_characters(string_buffer.data(), bytes); + put_buffer(string_buffer, bytes); bytes = 0; } @@ -525,7 +990,7 @@ class serializer if (error_handler == error_handler_t::replace) { // add a replacement character - if (ensure_ascii) + if (EnsureAscii) { string_buffer[bytes++] = '\\'; string_buffer[bytes++] = 'u'; @@ -546,7 +1011,7 @@ class serializer // written ("\uxxxx\uxxxx\0") for one code point if (string_buffer.size() - bytes < 13) { - o->write_characters(string_buffer.data(), bytes); + put_buffer(string_buffer, bytes); bytes = 0; } @@ -568,7 +1033,7 @@ class serializer default: // decode found yet incomplete multibyte code point { - if (!ensure_ascii) + if (!EnsureAscii) { // code point will not be escaped - copy byte to buffer string_buffer[bytes++] = s[i]; @@ -585,7 +1050,7 @@ class serializer // write buffer if (bytes > 0) { - o->write_characters(string_buffer.data(), bytes); + put_buffer(string_buffer, bytes); } } else @@ -601,22 +1066,22 @@ class serializer case error_handler_t::ignore: { // write all accepted bytes - o->write_characters(string_buffer.data(), bytes_after_last_accept); + put_buffer(string_buffer, bytes_after_last_accept); break; } case error_handler_t::replace: { // write all accepted bytes - o->write_characters(string_buffer.data(), bytes_after_last_accept); + put_buffer(string_buffer, bytes_after_last_accept); // add a replacement character - if (ensure_ascii) + if (EnsureAscii) { - o->write_characters("\\ufffd", 6); + put_literal("\\ufffd"); } else { - o->write_characters("\xEF\xBF\xBD", 3); + put_literal("\xEF\xBF\xBD"); } break; } @@ -627,6 +1092,160 @@ class serializer } } + private: + /*! + @brief append a single character to the write buffer + + Structural characters ('{', '"', ',', ...) previously went straight to the + output adapter, one virtual call each. Buffering them and flushing in bulk + turns those many indirect calls into a single memcpy plus an occasional + flush, which dominates the cost of serializing object/array-heavy values. + */ + void put_char(char c) + { + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos == write_buffer.size())) + { + flush(); + } + write_buffer[write_buffer_pos++] = c; + } + + /*! + @brief append @a indent indentation characters to the write buffer + + Writes the indentation straight into the buffer instead of copying it out of + a pre-grown indentation string, so no auxiliary string has to be sized, + resized, or kept in sync with the deepest nesting level reached. + + An indentation wider than the buffer is emitted by filling the buffer with + the indentation character once and flushing that same content repeatedly: + flushing does not disturb what the buffer holds, so re-filling it between + flushes would be redundant work. + */ + void put_indent(std::size_t indent) + { + // closing braces at the outermost level ask for no indentation at all + if (indent == 0) + { + return; + } + + const std::size_t capacity = write_buffer.size(); + + // fill whatever room is left in the buffer; this is the whole job + // whenever the indentation is narrower than the buffer, which is the + // case for every sane indent_step + const std::size_t head = (std::min)(indent, capacity - write_buffer_pos); + std::memset(write_buffer.data() + write_buffer_pos, indent_char, head); + write_buffer_pos += head; + indent -= head; + + if (JSON_HEDLEY_LIKELY(indent == 0)) + { + return; + } + + // the buffer is full and the remainder spans whole buffer-fulls: flush + // what is pending, then fill the buffer with the indentation character + // exactly once and hand the same bytes to the adapter as often as needed + flush(); + std::memset(write_buffer.data(), indent_char, capacity); + + while (indent >= capacity) + { + write_buffer_pos = capacity; + flush(); + indent -= capacity; + } + + // the buffer still holds indentation characters throughout, so the tail + // only has to be claimed, not written again + write_buffer_pos = indent; + } + + /*! + @brief append a string literal to the write buffer + + The length comes from the array bound rather than a hand-written count, so + it cannot drift out of sync with the literal. A literal always fits into the + buffer (checked at compile time), so unlike @ref put_string this needs no + write-through path for oversized runs. + */ + template + void put_literal(const char (&s)[N]) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) + { + static_assert(N >= 2, "put_literal expects a non-empty string literal"); + // the array bound counts the terminating NUL, which is not written + constexpr std::size_t length = N - 1; + static_assert(length < write_buffer_size, "string literal must fit into the write buffer"); + + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + length > write_buffer.size())) + { + flush(); + } + std::memcpy(write_buffer.data() + write_buffer_pos, s, length); + write_buffer_pos += length; + } + + /*! + @brief append the characters of @a str in [@a start, @a end) + + The only way to append a run of characters: @a str carries its own bound, + so the range can be checked against it, which a bare pointer plus a count + could not do. Runs that do not fit the buffer are written straight through + the output adapter (after flushing what is pending), so large string and + number payloads are not copied an extra time. + */ + template + void put_string(const StringType& str, std::size_t start, std::size_t end) + { + JSON_ASSERT(start <= end); + JSON_ASSERT(end <= str.size()); + + const char* const s = str.data() + start; + const std::size_t length = end - start; + + if (JSON_HEDLEY_UNLIKELY(length >= write_buffer.size())) + { + flush(); + o->write_characters(s, length); + return; + } + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + length > write_buffer.size())) + { + flush(); + } + std::memcpy(write_buffer.data() + write_buffer_pos, s, length); + write_buffer_pos += length; + } + + /*! + @brief append the first @a length characters of a fixed-size buffer + */ + template + void put_buffer(const std::array& buffer, std::size_t length) + { + put_string(buffer, 0, length); + } + + JSON_PRIVATE_UNLESS_TESTED: + /*! + @brief flush the write buffer to the output adapter + + Writing zero characters is a well-defined no-op for every output adapter, so + the buffered length is passed through unconditionally (no empty-guard branch + to leave uncovered). + + @note dump_escaped() and dump_integer()/dump_float() write into the internal + write buffer; callers that invoke them directly (rather than through the + public dump()) must call flush() before inspecting the output. + */ + void flush() + { + o->write_characters(write_buffer.data(), write_buffer_pos); + write_buffer_pos = 0; + } + private: /*! @brief count digits @@ -715,6 +1334,62 @@ class serializer return false; } + /*! + @brief write the decimal representation of the byte @a value + + A binary value's bytes are always in [0, 255], so writing one needs neither + the digit counting nor the 64-bit arithmetic that @ref dump_integer does for + an arbitrary number, and the three digits it takes at most are written + straight into the write buffer. + + Any byte type that is not a plain unsigned byte is left to @ref dump_integer, + whose representation of it may differ. + */ + template + void dump_byte(const ByteType value) + { + dump_byte(value, std::integral_constant < bool, + std::is_unsigned::value && sizeof(ByteType) == 1 + && !std::is_same::value > {}); + } + + template + void dump_byte(const ByteType value, std::false_type /*is_plain_byte*/) + { + dump_integer(value); + } + + template + void dump_byte(const ByteType value, std::true_type /*is_plain_byte*/) + { + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + 3 > write_buffer.size())) + { + flush(); + } + + const auto byte = static_cast(value); + // Accumulate the offset in a local and store it back once. Writing + // through write_buffer[] is a char write, which may alias any object, + // so with the member updated in place the compiler has to reload and + // store it around every digit - measured 2.4x slower on a dump of a + // multi-megabyte binary value. + std::size_t pos = write_buffer_pos; + + if (byte >= 100) + { + write_buffer[pos++] = static_cast('0' + (byte / 100)); + write_buffer[pos++] = static_cast('0' + ((byte / 10) % 10)); + } + else if (byte >= 10) + { + write_buffer[pos++] = static_cast('0' + (byte / 10)); + } + + write_buffer[pos++] = static_cast('0' + (byte % 10)); + + write_buffer_pos = pos; + } + /*! @brief dump an integer @@ -751,7 +1426,7 @@ class serializer // special case for "0" if (x == 0) { - o->write_character('0'); + put_char('0'); return; } @@ -804,7 +1479,7 @@ class serializer *(--buffer_ptr) = static_cast('0' + abs_value); } - o->write_characters(number_buffer.data(), n_chars); + put_buffer(number_buffer, n_chars); } /*! @@ -820,7 +1495,7 @@ class serializer // NaN / inf if (!std::isfinite(x)) { - o->write_characters("null", 4); + put_literal("null"); return; } @@ -841,7 +1516,7 @@ class serializer auto* begin = number_buffer.data(); auto* end = ::nlohmann::detail::to_chars(begin, begin + number_buffer.size(), x); - o->write_characters(begin, static_cast(end - begin)); + put_buffer(number_buffer, static_cast(end - begin)); } JSON_HEDLEY_NON_NULL(1) @@ -872,27 +1547,27 @@ class serializer JSON_ASSERT(static_cast(len) < number_buffer.size()); // erase thousands separators - if (thousands_sep != '\0') + if (locale.thousands_sep != '\0') { // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::remove returns an iterator, see https://github.com/nlohmann/json/issues/3081 - const auto end = std::remove(number_buffer.begin(), number_buffer.begin() + len, thousands_sep); + const auto end = std::remove(number_buffer.begin(), number_buffer.begin() + len, locale.thousands_sep); std::fill(end, number_buffer.end(), '\0'); JSON_ASSERT((end - number_buffer.begin()) <= len); len = (end - number_buffer.begin()); } // convert decimal point to '.' - if (decimal_point != '\0' && decimal_point != '.') + if (locale.decimal_point != '\0' && locale.decimal_point != '.') { // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::find returns an iterator, see https://github.com/nlohmann/json/issues/3081 - const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), decimal_point); + const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), locale.decimal_point); if (dec_pos != number_buffer.end()) { *dec_pos = '.'; } } - o->write_characters(number_buffer.data(), static_cast(len)); + put_buffer(number_buffer, static_cast(len)); // determine if we need to append ".0" const bool value_is_int_like = @@ -904,7 +1579,7 @@ class serializer if (value_is_int_like) { - o->write_characters(".0", 2); + put_literal(".0"); } } @@ -991,29 +1666,53 @@ class serializer } private: + /// the locale's thousand separator and decimal point characters + struct locale_chars + { + explicit locale_chars(const std::lconv* loc) noexcept + : thousands_sep(loc->thousands_sep == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->thousands_sep))) + , decimal_point(loc->decimal_point == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->decimal_point))) + {} + + const char thousands_sep; + const char decimal_point; + }; + /// the output of the serializer output_adapter_t o = nullptr; /// a (hopefully) large enough character buffer std::array number_buffer{{}}; - /// the locale - const std::lconv* loc = nullptr; - /// the locale's thousand separator character - const char thousands_sep = '\0'; - /// the locale's decimal point character - const char decimal_point = '\0'; + /// computed once from std::localeconv() at construction; @ref + /// locale_chars keeps std::localeconv()'s pointer from having to be held + /// past the constructor, while still letting these stay const + const locale_chars locale; /// string buffer std::array string_buffer{{}}; /// the indentation character const char indent_char; - /// the indentation string - string_t indent_string; + + /// whether to pretty-print the output + const bool pretty_print; + + /// whether to escape non-ASCII characters with \uXXXX sequences + const bool ensure_ascii; + + /// the indent level + const std::size_t indent_step; /// error_handler how to react on decoding errors const error_handler_t error_handler; + + /// buffer collecting output before it is flushed to the output adapter, so + /// that the many small structural writes become few bulk writes + static constexpr std::size_t write_buffer_size = 1024; + std::array write_buffer{{}}; + /// number of valid bytes currently held in @ref write_buffer + std::size_t write_buffer_pos = 0; }; } // namespace detail diff --git a/include/nlohmann/json.hpp b/include/nlohmann/json.hpp index 9bbd98f15..7243c3c44 100644 --- a/include/nlohmann/json.hpp +++ b/include/nlohmann/json.hpp @@ -1343,15 +1343,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec const error_handler_t error_handler = error_handler_t::strict) const { string_t result; - serializer s(detail::output_adapter(result), indent_char, error_handler); if (indent >= 0) { - s.dump(*this, true, ensure_ascii, static_cast(indent)); + serializer s(detail::output_adapter(result), indent_char, + true, ensure_ascii, static_cast(indent), error_handler); + s.dump(*this); } else { - s.dump(*this, false, ensure_ascii, 0); + serializer s(detail::output_adapter(result), indent_char, + false, ensure_ascii, 0, error_handler); + s.dump(*this); } return result; @@ -4080,8 +4083,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec o.width(0); // do the actual serialization - serializer s(detail::output_adapter(o), o.fill()); - s.dump(j, pretty_print, false, static_cast(indentation)); + serializer s(detail::output_adapter(o), o.fill(), + pretty_print, false, static_cast(indentation)); + s.dump(j); return o; } diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index 34db800b7..8b1f4506e 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -8327,6 +8327,52 @@ inline std::size_t find_string_special(const unsigned char* data, std::size_t n) return n; } +// classify a byte as one the serializer must NOT copy verbatim when +// ensure_ascii is requested: the closing quote, an escape, a control character +// (< 0x20), DEL (0x7F), or any non-ASCII byte (>= 0x80). Everything else - +// printable ASCII except '"' and '\\' - is emitted unchanged. Note this differs +// from is_string_special() only in that 0x7F is also a stop (it is escaped as +// \u007f under ensure_ascii). +inline bool is_ascii_copyable(unsigned char c) noexcept +{ + return c >= 0x20u && c < 0x7Fu && c != '"' && c != '\\'; +} + +// return the index of the first byte in [data, data+n) that is NOT +// is_ascii_copyable(), or n if every byte can be copied verbatim; scans 8 bytes +// at a time. Used by the serializer's ensure_ascii fast path. +inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_t n) noexcept +{ + constexpr std::uint64_t ones = 0x0101010101010101ull; + constexpr std::uint64_t high = 0x8080808080808080ull; + std::size_t i = 0; + for (; i + 8 <= n; i += 8) + { + std::uint64_t v = 0; + std::memcpy(&v, data + i, sizeof(v)); + const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22) + const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C) + const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL (0x7F) + const std::uint64_t stop = ((q - ones) & ~q & high) // == '"' + | ((b - ones) & ~b & high) // == '\\' + | ((d - ones) & ~d & high) // == 0x7F + | ((v - 0x2020202020202020ull) & ~v & high) // < 0x20 + | (v & high); // >= 0x80 + if (stop != 0) + { + break; + } + } + for (; i < n; ++i) + { + if (!is_ascii_copyable(data[i])) + { + return i; + } + } + return n; +} + // Validate one UTF-8 sequence at the front of [data, data+avail). Returns its // length (2..4) only when the bytes form a *well-formed* sequence using exactly // the same ranges as scan_string()'s per-byte switch, so the bulk path accepts @@ -20183,17 +20229,19 @@ NLOHMANN_JSON_NAMESPACE_END -#include // reverse, remove, fill, find, none_of +#include // reverse, remove, fill, find, none_of, min #include // array #include // localeconv, lconv #include // labs, isfinite, isnan, signbit #include // size_t, ptrdiff_t #include // uint8_t #include // snprintf +#include // memcpy, memset #include // numeric_limits #include // string, char_traits #include // is_same #include // move +#include // vector // #include // __ _____ _____ _____ @@ -21318,6 +21366,8 @@ NLOHMANN_JSON_NAMESPACE_END // #include +// #include + // #include // #include @@ -21362,16 +21412,29 @@ class serializer /*! @param[in] s output stream to serialize to @param[in] ichar indentation character to use + @param[in] pretty_print_ whether the output shall be pretty-printed + @param[in] ensure_ascii_ If @a ensure_ascii_ is true, all non-ASCII + characters in the output are escaped with `\uXXXX` sequences, and the + result consists of ASCII characters only. + @param[in] indent_step_ the indent level @param[in] error_handler_ how to react on decoding errors + + None of @a pretty_print_, @a ensure_ascii_ and @a indent_step_ change over + the life of the serializer, so they are captured once here instead of + being threaded through every call to @ref dump, @ref dump_internal and + @ref dump_iteratively. */ serializer(output_adapter_t s, const char ichar, + const bool pretty_print_ = false, + const bool ensure_ascii_ = false, + const std::size_t indent_step_ = 0, error_handler_t error_handler_ = error_handler_t::strict) : o(std::move(s)) - , loc(std::localeconv()) - , thousands_sep(loc->thousands_sep == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->thousands_sep))) - , decimal_point(loc->decimal_point == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->decimal_point))) + , locale(std::localeconv()) , indent_char(ichar) - , indent_string(512, indent_char) + , pretty_print(pretty_print_) + , ensure_ascii(ensure_ascii_) + , indent_step(indent_step_) , error_handler(error_handler_) {} @@ -21387,8 +21450,8 @@ class serializer This function is called by the public member function dump and organizes the serialization internally. The indentation level is propagated as - additional parameter. In case of arrays and objects, the function is - called recursively. + additional parameter. Arrays and objects are serialized without recursion, + however deeply they are nested. - strings and object keys are escaped using `escape_string()` - integer numbers are converted implicitly via `operator<<` @@ -21397,89 +21460,109 @@ class serializer byte array @param[in] val value to serialize - @param[in] pretty_print whether the output shall be pretty-printed - @param[in] ensure_ascii If @a ensure_ascii is true, all non-ASCII characters - in the output are escaped with `\uXXXX` sequences, and the result consists - of ASCII characters only. - @param[in] indent_step the indent level @param[in] current_indent the current indent level (only used internally) */ void dump(const BasicJsonType& val, - const bool pretty_print, - const bool ensure_ascii, - const unsigned int indent_step, - const unsigned int current_indent = 0) + const std::size_t current_indent = 0) + { + dump_internal(val, current_indent); + flush(); + } + + JSON_PRIVATE_UNLESS_TESTED: + /*! + @brief worker for @ref dump + + Identical in behavior to the historical @ref dump, but writes into the + serializer's internal @ref write_buffer instead of issuing a virtual call + per token. The public @ref dump wraps this and flushes the buffer once the + top-level value has been serialized. + + Serializing a container descends into its elements, so a value nested deeply + enough used to exhaust the call stack and terminate the process with no + exception to catch. The descent is bounded here: once @ref dump_depth_limit + levels have been entered, @ref dump_iteratively writes out what is left + without the call stack. A value nested less deeply than that - all but a + vanishing minority - is written by exactly the code that always wrote it. + + @sa https://github.com/nlohmann/json/issues/5387 + */ + void dump_internal(const BasicJsonType& val, + const std::size_t current_indent = 0, + const std::size_t depth = 0) { switch (val.m_data.m_type) { case value_t::object: { + if (JSON_HEDLEY_UNLIKELY(depth >= dump_depth_limit())) + { + dump_iteratively(val, current_indent); + return; + } + if (val.m_data.m_value.object->empty()) { - o->write_characters("{}", 2); + put_literal("{}"); return; } if (pretty_print) { - o->write_characters("{\n", 2); + put_literal("{\n"); // variable to hold indentation for recursive calls - const auto new_indent = current_indent + indent_step; - if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) - { - indent_string.resize(indent_string.size() * 2, ' '); - } + const auto new_indent = next_indent(current_indent, indent_step); // first n-1 elements auto i = val.m_data.m_value.object->cbegin(); for (std::size_t cnt = 0; cnt < val.m_data.m_value.object->size() - 1; ++cnt, ++i) { - o->write_characters(indent_string.c_str(), new_indent); - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\": ", 3); - dump(i->second, true, ensure_ascii, indent_step, new_indent); - o->write_characters(",\n", 2); + put_indent(new_indent); + put_char('"'); + dump_escaped(i->first); + put_literal("\": "); + dump_internal(i->second, new_indent, depth + 1); + put_literal(",\n"); } // last element JSON_ASSERT(i != val.m_data.m_value.object->cend()); JSON_ASSERT(std::next(i) == val.m_data.m_value.object->cend()); - o->write_characters(indent_string.c_str(), new_indent); - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\": ", 3); - dump(i->second, true, ensure_ascii, indent_step, new_indent); + put_indent(new_indent); + put_char('"'); + dump_escaped(i->first); + put_literal("\": "); + dump_internal(i->second, new_indent, depth + 1); - o->write_character('\n'); - o->write_characters(indent_string.c_str(), current_indent); - o->write_character('}'); + put_char('\n'); + put_indent(current_indent); + put_char('}'); } else { - o->write_character('{'); + put_char('{'); // first n-1 elements auto i = val.m_data.m_value.object->cbegin(); for (std::size_t cnt = 0; cnt < val.m_data.m_value.object->size() - 1; ++cnt, ++i) { - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\":", 2); - dump(i->second, false, ensure_ascii, indent_step, current_indent); - o->write_character(','); + put_char('"'); + dump_escaped(i->first); + put_literal("\":"); + dump_internal(i->second, current_indent, depth + 1); + put_char(','); } // last element JSON_ASSERT(i != val.m_data.m_value.object->cend()); JSON_ASSERT(std::next(i) == val.m_data.m_value.object->cend()); - o->write_character('\"'); - dump_escaped(i->first, ensure_ascii); - o->write_characters("\":", 2); - dump(i->second, false, ensure_ascii, indent_step, current_indent); + put_char('"'); + dump_escaped(i->first); + put_literal("\":"); + dump_internal(i->second, current_indent, depth + 1); - o->write_character('}'); + put_char('}'); } return; @@ -21487,58 +21570,60 @@ class serializer case value_t::array: { + if (JSON_HEDLEY_UNLIKELY(depth >= dump_depth_limit())) + { + dump_iteratively(val, current_indent); + return; + } + if (val.m_data.m_value.array->empty()) { - o->write_characters("[]", 2); + put_literal("[]"); return; } if (pretty_print) { - o->write_characters("[\n", 2); + put_literal("[\n"); // variable to hold indentation for recursive calls - const auto new_indent = current_indent + indent_step; - if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) - { - indent_string.resize(indent_string.size() * 2, ' '); - } + const auto new_indent = next_indent(current_indent, indent_step); // first n-1 elements for (auto i = val.m_data.m_value.array->cbegin(); i != val.m_data.m_value.array->cend() - 1; ++i) { - o->write_characters(indent_string.c_str(), new_indent); - dump(*i, true, ensure_ascii, indent_step, new_indent); - o->write_characters(",\n", 2); + put_indent(new_indent); + dump_internal(*i, new_indent, depth + 1); + put_literal(",\n"); } // last element JSON_ASSERT(!val.m_data.m_value.array->empty()); - o->write_characters(indent_string.c_str(), new_indent); - dump(val.m_data.m_value.array->back(), true, ensure_ascii, indent_step, new_indent); + put_indent(new_indent); + dump_internal(val.m_data.m_value.array->back(), new_indent, depth + 1); - o->write_character('\n'); - o->write_characters(indent_string.c_str(), current_indent); - o->write_character(']'); + put_char('\n'); + put_indent(current_indent); + put_char(']'); } else { - o->write_character('['); + put_char('['); // first n-1 elements for (auto i = val.m_data.m_value.array->cbegin(); i != val.m_data.m_value.array->cend() - 1; ++i) { - dump(*i, false, ensure_ascii, indent_step, current_indent); - o->write_character(','); + dump_internal(*i, current_indent, depth + 1); + put_char(','); } // last element JSON_ASSERT(!val.m_data.m_value.array->empty()); - dump(val.m_data.m_value.array->back(), false, ensure_ascii, indent_step, current_indent); + dump_internal(val.m_data.m_value.array->back(), current_indent, depth + 1); - o->write_character(']'); + put_char(']'); } return; @@ -21546,9 +21631,9 @@ class serializer case value_t::string: { - o->write_character('\"'); - dump_escaped(*val.m_data.m_value.string, ensure_ascii); - o->write_character('\"'); + put_char('"'); + dump_escaped(*val.m_data.m_value.string); + put_char('"'); return; } @@ -21556,70 +21641,66 @@ class serializer { if (pretty_print) { - o->write_characters("{\n", 2); + put_literal("{\n"); // variable to hold indentation for recursive calls - const auto new_indent = current_indent + indent_step; - if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) - { - indent_string.resize(indent_string.size() * 2, ' '); - } + const auto new_indent = next_indent(current_indent, indent_step); - o->write_characters(indent_string.c_str(), new_indent); + put_indent(new_indent); - o->write_characters("\"bytes\": [", 10); + put_literal("\"bytes\": ["); if (!val.m_data.m_value.binary->empty()) { for (auto i = val.m_data.m_value.binary->cbegin(); i != val.m_data.m_value.binary->cend() - 1; ++i) { - dump_integer(*i); - o->write_characters(", ", 2); + dump_byte(*i); + put_literal(", "); } - dump_integer(val.m_data.m_value.binary->back()); + dump_byte(val.m_data.m_value.binary->back()); } - o->write_characters("],\n", 3); - o->write_characters(indent_string.c_str(), new_indent); + put_literal("],\n"); + put_indent(new_indent); - o->write_characters("\"subtype\": ", 11); + put_literal("\"subtype\": "); if (val.m_data.m_value.binary->has_subtype()) { dump_integer(val.m_data.m_value.binary->subtype()); } else { - o->write_characters("null", 4); + put_literal("null"); } - o->write_character('\n'); - o->write_characters(indent_string.c_str(), current_indent); - o->write_character('}'); + put_char('\n'); + put_indent(current_indent); + put_char('}'); } else { - o->write_characters("{\"bytes\":[", 10); + put_literal("{\"bytes\":["); if (!val.m_data.m_value.binary->empty()) { for (auto i = val.m_data.m_value.binary->cbegin(); i != val.m_data.m_value.binary->cend() - 1; ++i) { - dump_integer(*i); - o->write_character(','); + dump_byte(*i); + put_char(','); } - dump_integer(val.m_data.m_value.binary->back()); + dump_byte(val.m_data.m_value.binary->back()); } - o->write_characters("],\"subtype\":", 12); + put_literal("],\"subtype\":"); if (val.m_data.m_value.binary->has_subtype()) { dump_integer(val.m_data.m_value.binary->subtype()); - o->write_character('}'); + put_char('}'); } else { - o->write_characters("null}", 5); + put_literal("null}"); } } return; @@ -21629,11 +21710,11 @@ class serializer { if (val.m_data.m_value.boolean) { - o->write_characters("true", 4); + put_literal("true"); } else { - o->write_characters("false", 5); + put_literal("false"); } return; } @@ -21658,13 +21739,13 @@ class serializer case value_t::discarded: { - o->write_characters("", 11); + put_literal(""); return; } case value_t::null: { - o->write_characters("null", 4); + put_literal("null"); return; } @@ -21673,6 +21754,367 @@ class serializer } } + private: + /// the number of levels @ref dump_internal descends into before it hands + /// over to @ref dump_iteratively + static constexpr std::size_t dump_depth_limit() + { + return 128; + } + + /*! + @brief write out @a val and everything below it without the call stack + + Emits the same bytes as @ref dump_internal, keeping the containers it has + entered on an explicit stack instead of descending into them. Only reached + for values nested deeper than @ref dump_depth_limit, which is why it is not + written for speed: walking every value this way measured up to 20% slower on + object-heavy documents than letting the compiler drive the descent. + */ + void dump_iteratively(const BasicJsonType& val, + const std::size_t current_indent = 0) + { + // Scalars, empty containers and binary values are written by dump_value + // alone, so nothing is allocated for them: only a container with + // elements is ever pushed. + std::vector stack; + + dump_value(val, current_indent, stack); + + while (!stack.empty()) + { + dump_frame& frame = stack.back(); + + if (frame.value->m_data.m_type == value_t::object) + { + const auto* object = frame.value->m_data.m_value.object; + + if (frame.object_it == object->cend()) + { + if (pretty_print) + { + put_char('\n'); + put_indent(frame.current_indent); + } + + put_char('}'); + stack.pop_back(); + continue; + } + + // the separator goes in front of every element but the first, + // which puts exactly one between each pair and none at the end + if (frame.object_it != object->cbegin()) + { + if (pretty_print) + { + put_literal(",\n"); + } + else + { + put_char(','); + } + } + + if (pretty_print) + { + put_indent(frame.child_indent); + } + + put_char('"'); + dump_escaped(frame.object_it->first); + + if (pretty_print) + { + put_literal("\": "); + } + else + { + put_literal("\":"); + } + + const BasicJsonType& element = frame.object_it->second; + ++frame.object_it; + + // read everything needed from the frame before this: entering a + // container pushes another one and can move them all + const std::size_t element_indent = frame.child_indent; + dump_value(element, element_indent, stack); + } + else + { + const auto* array = frame.value->m_data.m_value.array; + + if (frame.array_it == array->cend()) + { + if (pretty_print) + { + put_char('\n'); + put_indent(frame.current_indent); + } + + put_char(']'); + stack.pop_back(); + continue; + } + + if (frame.array_it != array->cbegin()) + { + if (pretty_print) + { + put_literal(",\n"); + } + else + { + put_char(','); + } + } + + if (pretty_print) + { + put_indent(frame.child_indent); + } + + const BasicJsonType& element = *frame.array_it; + ++frame.array_it; + + // see above + const std::size_t element_indent = frame.child_indent; + dump_value(element, element_indent, stack); + } + } + } + + private: + /// @brief a container that has been opened but not closed yet + struct dump_frame + { + dump_frame(const BasicJsonType* value_, const std::size_t current_indent_, + const std::size_t child_indent_) noexcept + : value(value_) + , current_indent(current_indent_) + , child_indent(child_indent_) + {} + + /// the object or array being serialized + const BasicJsonType* value; + /// the element to serialize next; which of the two is live follows from + /// the type of @a value. They are kept side by side rather than in a + /// union, which would need its special members written out by hand, see + /// detail/iterators/internal_iterator.hpp + typename BasicJsonType::object_t::const_iterator object_it{}; + typename BasicJsonType::array_t::const_iterator array_it{}; + /// the indentation of the container itself, used by its closing bracket + std::size_t current_indent; + /// the indentation of the container's elements + std::size_t child_indent; + }; + + /*! + @brief serialize the value @a val, but not the elements of a container + + An object or array with elements is opened and pushed onto @a stack for + @ref dump_internal to walk; everything else - including a binary value, + which looks like an object but has no elements to descend into - is written + out here in full. + */ + void dump_value(const BasicJsonType& val, + const std::size_t current_indent, + std::vector& stack) + { + switch (val.m_data.m_type) + { + case value_t::object: + { + if (val.m_data.m_value.object->empty()) + { + put_literal("{}"); + return; + } + + std::size_t child_indent = current_indent; + + if (pretty_print) + { + put_literal("{\n"); + child_indent = next_indent(current_indent, indent_step); + } + else + { + put_char('{'); + } + + stack.emplace_back(&val, current_indent, child_indent); + stack.back().object_it = val.m_data.m_value.object->cbegin(); + return; + } + + case value_t::array: + { + if (val.m_data.m_value.array->empty()) + { + put_literal("[]"); + return; + } + + std::size_t child_indent = current_indent; + + if (pretty_print) + { + put_literal("[\n"); + child_indent = next_indent(current_indent, indent_step); + } + else + { + put_char('['); + } + + stack.emplace_back(&val, current_indent, child_indent); + stack.back().array_it = val.m_data.m_value.array->cbegin(); + return; + } + + case value_t::string: + { + put_char('"'); + dump_escaped(*val.m_data.m_value.string); + put_char('"'); + return; + } + + case value_t::binary: + { + if (pretty_print) + { + put_literal("{\n"); + + // variable to hold indentation for the bytes + const auto new_indent = next_indent(current_indent, indent_step); + + put_indent(new_indent); + + put_literal("\"bytes\": ["); + + if (!val.m_data.m_value.binary->empty()) + { + for (auto i = val.m_data.m_value.binary->cbegin(); + i != val.m_data.m_value.binary->cend() - 1; ++i) + { + dump_byte(*i); + put_literal(", "); + } + dump_byte(val.m_data.m_value.binary->back()); + } + + put_literal("],\n"); + put_indent(new_indent); + + put_literal("\"subtype\": "); + if (val.m_data.m_value.binary->has_subtype()) + { + dump_integer(val.m_data.m_value.binary->subtype()); + } + else + { + put_literal("null"); + } + put_char('\n'); + put_indent(current_indent); + put_char('}'); + } + else + { + put_literal("{\"bytes\":["); + + if (!val.m_data.m_value.binary->empty()) + { + for (auto i = val.m_data.m_value.binary->cbegin(); + i != val.m_data.m_value.binary->cend() - 1; ++i) + { + dump_byte(*i); + put_char(','); + } + dump_byte(val.m_data.m_value.binary->back()); + } + + put_literal("],\"subtype\":"); + if (val.m_data.m_value.binary->has_subtype()) + { + dump_integer(val.m_data.m_value.binary->subtype()); + put_char('}'); + } + else + { + put_literal("null}"); + } + } + return; + } + + case value_t::boolean: + { + if (val.m_data.m_value.boolean) + { + put_literal("true"); + } + else + { + put_literal("false"); + } + return; + } + + case value_t::number_integer: + { + dump_integer(val.m_data.m_value.number_integer); + return; + } + + case value_t::number_unsigned: + { + dump_integer(val.m_data.m_value.number_unsigned); + return; + } + + case value_t::number_float: + { + dump_float(val.m_data.m_value.number_float); + return; + } + + case value_t::discarded: + { + put_literal(""); + return; + } + + case value_t::null: + { + put_literal("null"); + return; + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE + } + } + + + /*! + @brief the indentation level to use for the children of the current value + + A very large @a indent_step can wrap the unsigned accumulation on deep + nesting, which would silently truncate the indentation. Far harder to reach + now that the accumulator is a std::size_t, but still reachable where that is + 32 bits wide. + */ + static std::size_t next_indent(const std::size_t current_indent, const std::size_t indent_step) + { + const std::size_t new_indent = current_indent + indent_step; + JSON_ASSERT(new_indent >= current_indent); + return new_indent; + } + JSON_PRIVATE_UNLESS_TESTED: /*! @brief dump escaped string @@ -21683,12 +22125,32 @@ class serializer representation. The escaped string is written to output stream @a o. @param[in] s the string to escape - @param[in] ensure_ascii whether to escape non-ASCII characters with - \uXXXX sequences @complexity Linear in the length of string @a s. */ - void dump_escaped(const string_t& s, const bool ensure_ascii) + void dump_escaped(const string_t& s) + { + // dispatch once here rather than test the flag inside the loop: it does + // not change while a string is written, and folding it lets each of the + // two scanners be inlined into a loop of its own + if (ensure_ascii) + { + dump_escaped_impl(s); + } + else + { + dump_escaped_impl(s); + } + } + + /*! + @brief worker for @ref dump_escaped + + @a ensure_ascii is a template parameter here so that the branch on it is + resolved once, outside the loop; see @ref dump_escaped. + */ + template + void dump_escaped_impl(const string_t& s) { std::uint32_t codepoint{}; std::uint8_t state = UTF8_ACCEPT; @@ -21700,6 +22162,56 @@ class serializer for (std::size_t i = 0; i < s.size(); ++i) { + // Fast path: at a character boundary (state == UTF8_ACCEPT), + // bulk-copy the longest run of bytes that need no escaping using a + // SWAR scanner shared with the lexer's contiguous path. The scanner + // stops exactly at the first byte dump_escaped would handle + // individually, so that byte is left to the byte-at-a-time path + // below, keeping escaping output and error diagnostics unchanged. + // + // - EnsureAscii == false: string_bulk_run() copies ordinary bytes + // and complete well-formed UTF-8, stopping at a quote, backslash, + // control character (< 0x20), or ill-formed/truncated sequence. + // - EnsureAscii == true: only printable ASCII may be copied + // verbatim; find_ascii_copyable_run() additionally stops at 0x7F + // and every non-ASCII byte (>= 0x80), which must be \u-escaped. + if (state == UTF8_ACCEPT) + { + const auto* const data = reinterpret_cast(s.data()); + // A run can only be non-empty when the very first byte is one + // the scanner may copy, so test that single byte before paying + // for the scan. Without it, text whose characters all have to be + // escaped - CJK under ensure_ascii, where every byte is >= 0x80 - + // runs the scanner once per character only to be told zero. + std::size_t run = 0; + if (!EnsureAscii) + { + run = string_bulk_run(data + i, s.size() - i); + } + else if (is_ascii_copyable(data[i])) + { + run = find_ascii_copyable_run(data + i, s.size() - i); + } + if (run != 0) + { + // emit any bytes still pending in string_buffer first to + // preserve output order, then write the run directly + if (bytes != 0) + { + put_buffer(string_buffer, bytes); + bytes = 0; + } + put_string(s, i, i + run); + bytes_after_last_accept = 0; + undumped_chars = 0; + i += run; + if (i >= s.size()) + { + break; + } + } + } + const auto byte = static_cast(s[i]); switch (decode(state, codepoint, byte)) @@ -21746,7 +22258,7 @@ class serializer case 0x22: // quotation mark { string_buffer[bytes++] = '\\'; - string_buffer[bytes++] = '\"'; + string_buffer[bytes++] = '"'; break; } @@ -21760,8 +22272,8 @@ class serializer default: { // escape control characters (0x00..0x1F) or, if - // ensure_ascii parameter is used, non-ASCII characters - if ((codepoint <= 0x1F) || (ensure_ascii && (codepoint >= 0x7F))) + // EnsureAscii parameter is used, non-ASCII characters + if ((codepoint <= 0x1F) || (EnsureAscii && (codepoint >= 0x7F))) { if (codepoint <= 0xFFFF) { @@ -21788,7 +22300,7 @@ class serializer // written ("\uxxxx\uxxxx\0") for one code point if (string_buffer.size() - bytes < 13) { - o->write_characters(string_buffer.data(), bytes); + put_buffer(string_buffer, bytes); bytes = 0; } @@ -21826,7 +22338,7 @@ class serializer if (error_handler == error_handler_t::replace) { // add a replacement character - if (ensure_ascii) + if (EnsureAscii) { string_buffer[bytes++] = '\\'; string_buffer[bytes++] = 'u'; @@ -21847,7 +22359,7 @@ class serializer // written ("\uxxxx\uxxxx\0") for one code point if (string_buffer.size() - bytes < 13) { - o->write_characters(string_buffer.data(), bytes); + put_buffer(string_buffer, bytes); bytes = 0; } @@ -21869,7 +22381,7 @@ class serializer default: // decode found yet incomplete multibyte code point { - if (!ensure_ascii) + if (!EnsureAscii) { // code point will not be escaped - copy byte to buffer string_buffer[bytes++] = s[i]; @@ -21886,7 +22398,7 @@ class serializer // write buffer if (bytes > 0) { - o->write_characters(string_buffer.data(), bytes); + put_buffer(string_buffer, bytes); } } else @@ -21902,22 +22414,22 @@ class serializer case error_handler_t::ignore: { // write all accepted bytes - o->write_characters(string_buffer.data(), bytes_after_last_accept); + put_buffer(string_buffer, bytes_after_last_accept); break; } case error_handler_t::replace: { // write all accepted bytes - o->write_characters(string_buffer.data(), bytes_after_last_accept); + put_buffer(string_buffer, bytes_after_last_accept); // add a replacement character - if (ensure_ascii) + if (EnsureAscii) { - o->write_characters("\\ufffd", 6); + put_literal("\\ufffd"); } else { - o->write_characters("\xEF\xBF\xBD", 3); + put_literal("\xEF\xBF\xBD"); } break; } @@ -21928,6 +22440,160 @@ class serializer } } + private: + /*! + @brief append a single character to the write buffer + + Structural characters ('{', '"', ',', ...) previously went straight to the + output adapter, one virtual call each. Buffering them and flushing in bulk + turns those many indirect calls into a single memcpy plus an occasional + flush, which dominates the cost of serializing object/array-heavy values. + */ + void put_char(char c) + { + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos == write_buffer.size())) + { + flush(); + } + write_buffer[write_buffer_pos++] = c; + } + + /*! + @brief append @a indent indentation characters to the write buffer + + Writes the indentation straight into the buffer instead of copying it out of + a pre-grown indentation string, so no auxiliary string has to be sized, + resized, or kept in sync with the deepest nesting level reached. + + An indentation wider than the buffer is emitted by filling the buffer with + the indentation character once and flushing that same content repeatedly: + flushing does not disturb what the buffer holds, so re-filling it between + flushes would be redundant work. + */ + void put_indent(std::size_t indent) + { + // closing braces at the outermost level ask for no indentation at all + if (indent == 0) + { + return; + } + + const std::size_t capacity = write_buffer.size(); + + // fill whatever room is left in the buffer; this is the whole job + // whenever the indentation is narrower than the buffer, which is the + // case for every sane indent_step + const std::size_t head = (std::min)(indent, capacity - write_buffer_pos); + std::memset(write_buffer.data() + write_buffer_pos, indent_char, head); + write_buffer_pos += head; + indent -= head; + + if (JSON_HEDLEY_LIKELY(indent == 0)) + { + return; + } + + // the buffer is full and the remainder spans whole buffer-fulls: flush + // what is pending, then fill the buffer with the indentation character + // exactly once and hand the same bytes to the adapter as often as needed + flush(); + std::memset(write_buffer.data(), indent_char, capacity); + + while (indent >= capacity) + { + write_buffer_pos = capacity; + flush(); + indent -= capacity; + } + + // the buffer still holds indentation characters throughout, so the tail + // only has to be claimed, not written again + write_buffer_pos = indent; + } + + /*! + @brief append a string literal to the write buffer + + The length comes from the array bound rather than a hand-written count, so + it cannot drift out of sync with the literal. A literal always fits into the + buffer (checked at compile time), so unlike @ref put_string this needs no + write-through path for oversized runs. + */ + template + void put_literal(const char (&s)[N]) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) + { + static_assert(N >= 2, "put_literal expects a non-empty string literal"); + // the array bound counts the terminating NUL, which is not written + constexpr std::size_t length = N - 1; + static_assert(length < write_buffer_size, "string literal must fit into the write buffer"); + + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + length > write_buffer.size())) + { + flush(); + } + std::memcpy(write_buffer.data() + write_buffer_pos, s, length); + write_buffer_pos += length; + } + + /*! + @brief append the characters of @a str in [@a start, @a end) + + The only way to append a run of characters: @a str carries its own bound, + so the range can be checked against it, which a bare pointer plus a count + could not do. Runs that do not fit the buffer are written straight through + the output adapter (after flushing what is pending), so large string and + number payloads are not copied an extra time. + */ + template + void put_string(const StringType& str, std::size_t start, std::size_t end) + { + JSON_ASSERT(start <= end); + JSON_ASSERT(end <= str.size()); + + const char* const s = str.data() + start; + const std::size_t length = end - start; + + if (JSON_HEDLEY_UNLIKELY(length >= write_buffer.size())) + { + flush(); + o->write_characters(s, length); + return; + } + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + length > write_buffer.size())) + { + flush(); + } + std::memcpy(write_buffer.data() + write_buffer_pos, s, length); + write_buffer_pos += length; + } + + /*! + @brief append the first @a length characters of a fixed-size buffer + */ + template + void put_buffer(const std::array& buffer, std::size_t length) + { + put_string(buffer, 0, length); + } + + JSON_PRIVATE_UNLESS_TESTED: + /*! + @brief flush the write buffer to the output adapter + + Writing zero characters is a well-defined no-op for every output adapter, so + the buffered length is passed through unconditionally (no empty-guard branch + to leave uncovered). + + @note dump_escaped() and dump_integer()/dump_float() write into the internal + write buffer; callers that invoke them directly (rather than through the + public dump()) must call flush() before inspecting the output. + */ + void flush() + { + o->write_characters(write_buffer.data(), write_buffer_pos); + write_buffer_pos = 0; + } + private: /*! @brief count digits @@ -22016,6 +22682,62 @@ class serializer return false; } + /*! + @brief write the decimal representation of the byte @a value + + A binary value's bytes are always in [0, 255], so writing one needs neither + the digit counting nor the 64-bit arithmetic that @ref dump_integer does for + an arbitrary number, and the three digits it takes at most are written + straight into the write buffer. + + Any byte type that is not a plain unsigned byte is left to @ref dump_integer, + whose representation of it may differ. + */ + template + void dump_byte(const ByteType value) + { + dump_byte(value, std::integral_constant < bool, + std::is_unsigned::value && sizeof(ByteType) == 1 + && !std::is_same::value > {}); + } + + template + void dump_byte(const ByteType value, std::false_type /*is_plain_byte*/) + { + dump_integer(value); + } + + template + void dump_byte(const ByteType value, std::true_type /*is_plain_byte*/) + { + if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + 3 > write_buffer.size())) + { + flush(); + } + + const auto byte = static_cast(value); + // Accumulate the offset in a local and store it back once. Writing + // through write_buffer[] is a char write, which may alias any object, + // so with the member updated in place the compiler has to reload and + // store it around every digit - measured 2.4x slower on a dump of a + // multi-megabyte binary value. + std::size_t pos = write_buffer_pos; + + if (byte >= 100) + { + write_buffer[pos++] = static_cast('0' + (byte / 100)); + write_buffer[pos++] = static_cast('0' + ((byte / 10) % 10)); + } + else if (byte >= 10) + { + write_buffer[pos++] = static_cast('0' + (byte / 10)); + } + + write_buffer[pos++] = static_cast('0' + (byte % 10)); + + write_buffer_pos = pos; + } + /*! @brief dump an integer @@ -22052,7 +22774,7 @@ class serializer // special case for "0" if (x == 0) { - o->write_character('0'); + put_char('0'); return; } @@ -22105,7 +22827,7 @@ class serializer *(--buffer_ptr) = static_cast('0' + abs_value); } - o->write_characters(number_buffer.data(), n_chars); + put_buffer(number_buffer, n_chars); } /*! @@ -22121,7 +22843,7 @@ class serializer // NaN / inf if (!std::isfinite(x)) { - o->write_characters("null", 4); + put_literal("null"); return; } @@ -22142,7 +22864,7 @@ class serializer auto* begin = number_buffer.data(); auto* end = ::nlohmann::detail::to_chars(begin, begin + number_buffer.size(), x); - o->write_characters(begin, static_cast(end - begin)); + put_buffer(number_buffer, static_cast(end - begin)); } JSON_HEDLEY_NON_NULL(1) @@ -22173,27 +22895,27 @@ class serializer JSON_ASSERT(static_cast(len) < number_buffer.size()); // erase thousands separators - if (thousands_sep != '\0') + if (locale.thousands_sep != '\0') { // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::remove returns an iterator, see https://github.com/nlohmann/json/issues/3081 - const auto end = std::remove(number_buffer.begin(), number_buffer.begin() + len, thousands_sep); + const auto end = std::remove(number_buffer.begin(), number_buffer.begin() + len, locale.thousands_sep); std::fill(end, number_buffer.end(), '\0'); JSON_ASSERT((end - number_buffer.begin()) <= len); len = (end - number_buffer.begin()); } // convert decimal point to '.' - if (decimal_point != '\0' && decimal_point != '.') + if (locale.decimal_point != '\0' && locale.decimal_point != '.') { // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::find returns an iterator, see https://github.com/nlohmann/json/issues/3081 - const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), decimal_point); + const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), locale.decimal_point); if (dec_pos != number_buffer.end()) { *dec_pos = '.'; } } - o->write_characters(number_buffer.data(), static_cast(len)); + put_buffer(number_buffer, static_cast(len)); // determine if we need to append ".0" const bool value_is_int_like = @@ -22205,7 +22927,7 @@ class serializer if (value_is_int_like) { - o->write_characters(".0", 2); + put_literal(".0"); } } @@ -22292,29 +23014,53 @@ class serializer } private: + /// the locale's thousand separator and decimal point characters + struct locale_chars + { + explicit locale_chars(const std::lconv* loc) noexcept + : thousands_sep(loc->thousands_sep == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->thousands_sep))) + , decimal_point(loc->decimal_point == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->decimal_point))) + {} + + const char thousands_sep; + const char decimal_point; + }; + /// the output of the serializer output_adapter_t o = nullptr; /// a (hopefully) large enough character buffer std::array number_buffer{{}}; - /// the locale - const std::lconv* loc = nullptr; - /// the locale's thousand separator character - const char thousands_sep = '\0'; - /// the locale's decimal point character - const char decimal_point = '\0'; + /// computed once from std::localeconv() at construction; @ref + /// locale_chars keeps std::localeconv()'s pointer from having to be held + /// past the constructor, while still letting these stay const + const locale_chars locale; /// string buffer std::array string_buffer{{}}; /// the indentation character const char indent_char; - /// the indentation string - string_t indent_string; + + /// whether to pretty-print the output + const bool pretty_print; + + /// whether to escape non-ASCII characters with \uXXXX sequences + const bool ensure_ascii; + + /// the indent level + const std::size_t indent_step; /// error_handler how to react on decoding errors const error_handler_t error_handler; + + /// buffer collecting output before it is flushed to the output adapter, so + /// that the many small structural writes become few bulk writes + static constexpr std::size_t write_buffer_size = 1024; + std::array write_buffer{{}}; + /// number of valid bytes currently held in @ref write_buffer + std::size_t write_buffer_pos = 0; }; } // namespace detail @@ -23992,15 +24738,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec const error_handler_t error_handler = error_handler_t::strict) const { string_t result; - serializer s(detail::output_adapter(result), indent_char, error_handler); if (indent >= 0) { - s.dump(*this, true, ensure_ascii, static_cast(indent)); + serializer s(detail::output_adapter(result), indent_char, + true, ensure_ascii, static_cast(indent), error_handler); + s.dump(*this); } else { - s.dump(*this, false, ensure_ascii, 0); + serializer s(detail::output_adapter(result), indent_char, + false, ensure_ascii, 0, error_handler); + s.dump(*this); } return result; @@ -26729,8 +27478,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec o.width(0); // do the actual serialization - serializer s(detail::output_adapter(o), o.fill()); - s.dump(j, pretty_print, false, static_cast(indentation)); + serializer s(detail::output_adapter(o), o.fill(), + pretty_print, false, static_cast(indentation)); + s.dump(j); return o; } diff --git a/tests/src/unit-convenience.cpp b/tests/src/unit-convenience.cpp index 037a4e589..266497867 100644 --- a/tests/src/unit-convenience.cpp +++ b/tests/src/unit-convenience.cpp @@ -98,8 +98,9 @@ void check_escaped(const char* original, const char* escaped = "", bool ensure_a void check_escaped(const char* original, const char* escaped, const bool ensure_ascii) { std::stringstream ss; - json::serializer s(nlohmann::detail::output_adapter(ss), ' '); - s.dump_escaped(original, ensure_ascii); + json::serializer s(nlohmann::detail::output_adapter(ss), ' ', false, ensure_ascii); + s.dump_escaped(original); + s.flush(); // dump_escaped writes into the serializer's internal buffer CHECK(ss.str() == escaped); } } // namespace diff --git a/tests/src/unit-serialization.cpp b/tests/src/unit-serialization.cpp index caf720671..eddf59f2c 100644 --- a/tests/src/unit-serialization.cpp +++ b/tests/src/unit-serialization.cpp @@ -387,3 +387,232 @@ TEST_CASE("dump for basic_json with long double number_float_t") check_same(100.0L, 100.0); } } + +TEST_CASE("serialization of strings (bulk fast path)") +{ + // These cases exercise the SWAR bulk-copy fast path in dump_escaped and the + // internal write buffer: long runs, escapes interrupting runs, 0x7F/DEL, + // multibyte UTF-8 under both ensure_ascii settings, and payloads larger than + // the write buffer. + + SECTION("long unescaped ASCII exceeds the write buffer") + { + const std::string big(3000, 'a'); + const json j = big; + CHECK(j.dump() == '"' + big + '"'); + CHECK(j.dump(-1, ' ', true) == '"' + big + '"'); + // round-trips + CHECK(json::parse(j.dump()) == j); + } + + SECTION("runs interrupted by escapes") + { + const json j = std::string(500, 'x') + "\n\"\\" + std::string(500, 'y'); + const std::string out = j.dump(); + CHECK(out == '"' + std::string(500, 'x') + "\\n\\\"\\\\" + std::string(500, 'y') + '"'); + CHECK(json::parse(out) == j); + } + + SECTION("DEL (0x7F) depends on ensure_ascii") + { + const json j = std::string("a\x7f" "b"); + CHECK(j.dump(-1, ' ', false) == "\"a\x7f" "b\""); // copied verbatim + CHECK(j.dump(-1, ' ', true) == "\"a\\u007fb\""); // escaped + } + + SECTION("multibyte UTF-8 under both ensure_ascii settings") + { + const json j = std::string("A\xc3\xa9\xe4\xbd\xa0\xf0\x9f\x98\x80Z"); // A é 你 😀 Z + // not escaping non-ASCII: bytes are copied through the bulk validator + CHECK(j.dump(-1, ' ', false) == "\"A\xc3\xa9\xe4\xbd\xa0\xf0\x9f\x98\x80Z\""); + // ensure_ascii: escaped (with a surrogate pair for the emoji) + CHECK(j.dump(-1, ' ', true) == "\"A\\u00e9\\u4f60\\ud83d\\ude00Z\""); + CHECK(json::parse(j.dump(-1, ' ', true)) == j); + } + + SECTION("many small structural writes exceed the write buffer") + { + json arr = json::array(); + for (int i = 0; i < 2000; ++i) + { + arr.push_back(i); + } + const std::string out = arr.dump(); + CHECK(out.front() == '['); + CHECK(out.back() == ']'); + CHECK(json::parse(out) == arr); + + json obj = json::object(); + for (int i = 0; i < 500; ++i) + { + obj["key" + std::to_string(i)] = i; + } + CHECK(json::parse(obj.dump()) == obj); + CHECK(json::parse(obj.dump(2)) == obj); + + // an array of many empty strings emits a long run of single-character + // writes ('"', '"', ',') at shallow nesting depth, so the write buffer + // fills and flushes mid-run without the deep recursion that would + // overflow the stack on some debug builds + json many_empty = json::array(); + for (int i = 0; i < 500; ++i) + { + many_empty.push_back(""); + } + const std::string out2 = many_empty.dump(); + CHECK(out2.size() > 1024); // spans multiple write-buffer flushes + CHECK(out2.front() == '['); + CHECK(out2.back() == ']'); + CHECK(json::parse(out2) == many_empty); + } + + SECTION("invalid UTF-8 handling is unaffected by the fast path") + { + const json j = std::string("valid\xff" "more"); + CHECK_THROWS_WITH_AS(j.dump(), "[json.exception.type_error.316] invalid UTF-8 byte at index 5: 0xFF", json::type_error&); + CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"valid\xef\xbf\xbd" "more\""); + CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"valid\\ufffdmore\""); + CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"validmore\""); + } +} + +TEST_CASE("indentation is written straight into the write buffer") +{ + // put_indent() memsets the indentation into the write buffer instead of + // copying it out of a pre-grown indentation string. These cases cover an + // indentation wider than the buffer, a non-space indentation character, and + // nesting deep enough that the accumulated indentation spans several + // buffer-fulls - the situations the old grow-a-string approach got wrong. + + SECTION("indent_step wider than the write buffer") + { + const json j = {{"a", 1}}; + // 2000 > the 1024-byte write buffer, and > the 512 the indentation + // string used to start at + CHECK(j.dump(2000) == "{\n" + std::string(2000, ' ') + "\"a\": 1\n}"); + // several whole buffer-fulls, so the buffer is refilled once and then + // flushed repeatedly + CHECK(j.dump(5000) == "{\n" + std::string(5000, ' ') + "\"a\": 1\n}"); + CHECK(j.dump(5000, '\t') == "{\n" + std::string(5000, '\t') + "\"a\": 1\n}"); + // an exact multiple of the buffer size + CHECK(j.dump(4096) == "{\n" + std::string(4096, ' ') + "\"a\": 1\n}"); + } + + SECTION("a non-space indentation character is used throughout") + { + const json j = {{"a", 1}}; + // 600 is past the point where the indentation used to be grown, which + // is where a hard-coded space would have shown up + CHECK(j.dump(600, '\t') == "{\n" + std::string(600, '\t') + "\"a\": 1\n}"); + CHECK(j.dump(3, '.') == "{\n...\"a\": 1\n}"); + } + + SECTION("accumulated indentation spans several buffer-fulls") + { + // five levels deep at 400 per level: the innermost value is indented by + // 2000 characters, reached in steps that each straddle the buffer end + json j = json::array({1}); + for (int i = 0; i < 4; ++i) + { + j = json::array({j}); + } + + const std::string out = j.dump(400); + CHECK(out.find(std::string("\n") + std::string(2000, ' ') + "1\n") != std::string::npos); + CHECK(json::parse(out) == j); + } + + SECTION("indentation is unchanged for ordinary widths") + { + const json j = {{"a", {1, 2}}, {"b", nullptr}}; + CHECK(j.dump(2) == "{\n \"a\": [\n 1,\n 2\n ],\n \"b\": null\n}"); + CHECK(j.dump(0) == "{\n\"a\": [\n1,\n2\n],\n\"b\": null\n}"); + } +} + +TEST_CASE("serialization of deeply nested values") +{ + // dump() descends into a bounded number of levels and writes out whatever + // is nested deeper than that without the call stack; see + // https://github.com/nlohmann/json/issues/5387 + + SECTION("nested deeper than the call stack could follow") + { + // parsing is iterative, so building these costs little + const std::size_t depth = 100000; + + const std::string array_text = std::string(depth, '[') + '0' + std::string(depth, ']'); + CHECK(json::parse(array_text).dump() == array_text); + + std::string object_text; + object_text.reserve((6 * depth) + 1); + for (std::size_t i = 0; i < depth; ++i) + { + object_text += "{\"a\":"; + } + object_text += '1'; + object_text.append(depth, '}'); + CHECK(json::parse(object_text).dump() == object_text); + } + + SECTION("depths around the bound of the descent") + { + // Cover every depth around the bound, so that the two ways of writing a + // value are known to meet cleanly - wherever the bound is set. + for (std::size_t d = 1; d <= 300; ++d) + { + CAPTURE(d); + + const std::string array_text = std::string(d, '[') + '7' + std::string(d, ']'); + CHECK(json::parse(array_text).dump() == array_text); + + std::string object_text; + for (std::size_t i = 0; i < d; ++i) + { + object_text += "{\"k\":"; + } + object_text += '7'; + object_text.append(d, '}'); + CHECK(json::parse(object_text).dump() == object_text); + } + } + + SECTION("pretty-printing across the bound") + { + for (std::size_t d = 120; d <= 140; ++d) + { + CAPTURE(d); + + const json j = json::parse(std::string(d, '[') + '7' + std::string(d, ']')); + + std::string expected; + for (std::size_t i = 0; i < d; ++i) + { + expected += std::string(2 * i, ' ') + "[\n"; + } + expected += std::string(2 * d, ' ') + '7'; + for (std::size_t i = d; i > 0; --i) + { + expected += '\n' + std::string(2 * (i - 1), ' ') + ']'; + } + + CHECK(j.dump(2) == expected); + } + } + + SECTION("an empty container below the bound") + { + // an empty container is written out in full and never descended into, + // so it must not gain a newline when it is reached iteratively + for (std::size_t d = 125; d <= 135; ++d) + { + CAPTURE(d); + + const std::string compact = std::string(d, '[') + "[]" + std::string(d, ']'); + CHECK(json::parse(compact).dump() == compact); + + const std::string with_object = std::string(d, '[') + "{}" + std::string(d, ']'); + CHECK(json::parse(with_object).dump() == with_object); + } + } +}