Write short plain strings and keys in one go in dump()

Most strings in real documents are object keys and short values. For
those, dump_escaped() scanned the bytes, called memcpy with a length
known only at run time, and the surrounding quote, colon and space were
three more bounds-checked writes.

A string of up to 16 printable ASCII characters (no quote, backslash or
DEL) is now read with two overlapping loads, checked with one SWAR test
per word, and written together with its quotes and the colon with two
overlapping stores. Anything else, and a write buffer without room, takes
the general path unchanged. Little-endian targets only; others keep the
general path.

dump_scalar() held the code of every scalar type and saved and restored
all callee-saved registers on every call. It only dispatches now, and the
string, integer, float and binary writers are functions of their own, so
each pays only for what it uses (the string path keeps the check for
long strings in front of any attempt, which keeps documents of long
strings neutral).

Apple M1, Apple clang, retired instructions per dump(), on top of the
printable-ASCII table of #5878:
  citm_catalog  7.54M -> 5.96M (-21 %)   twitter 4.87M -> 4.40M (-10 %)
  jeopardy      603M -> 501M   (-17 %)   logs     8.65M -> 8.17M (-5 %)
  a 139-byte object 3497 -> 2997 (-14 %)   signed_ints -4 %, canada -3 %
Linux x86-64 (callgrind), the whole series against develop, gcc 13 / clang 18:
  citm_catalog -16 % / -19 % (compact), twitter -22 % / -24 %.
Documents of long strings (paths, latin and CJK text) are within 1 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-22 09:28:11 +02:00
1 parent 2dc1e54856
commit 5bbc7a1098
4 files changed
+569 -176

No files matched your search

+51
View File
@@ -1580,3 +1580,54 @@ TEST_CASE("invalid UTF-8 at every position is handled like the reference")
}
}
}
TEST_CASE("short strings are written like the reference, as values and as keys")
{
// strings of up to 16 characters that need no escaping are written as a
// whole (together with the quotes and, for a key, the colon); everything
// else, including a string that is merely short, takes the general path.
// Every length around the edges of the size classes of the fast path is
// combined with each kind of byte that must send a string to the general
// path, at each position, and compared with the reference for values and,
// via the value, for keys.
const std::vector<std::string> specials =
{
"\"", "\\", "\n", "\x1f", "\x7f", "\xc3\xa9", "\xe2\x82\xac", "\xf0\x9f\x98\x80", "\xff", "\x80", "\xc3"
};
for (std::size_t length = 0; length <= 20; ++length)
{
const std::string plain(length, 'k');
check_dump_against_reference(plain, false);
for (const auto& special : specials)
{
for (std::size_t pos = 0; pos <= length; ++pos)
{
std::string s = plain;
s.insert(pos, special);
check_dump_against_reference(s, false);
for (const bool ensure_ascii :
{
false, true
})
{
// a key and a value are written alike, in compact and
// in pretty-printed form
const json value = s;
const json object = json::object({{s, 1}});
for (const auto error_handler :
{
json::error_handler_t::replace, json::error_handler_t::keep
})
{
const std::string quoted = value.dump(-1, ' ', ensure_ascii, error_handler);
CHECK(object.dump(-1, ' ', ensure_ascii, error_handler) == "{" + quoted + ":1}");
CHECK(object.dump(2, ' ', ensure_ascii, error_handler) == "{\n " + quoted + ": 1\n}");
}
}
}
}
}
}