Skip the indentation after a newline 8 bytes at a time

Pretty-printed JSON indents every line with a run of spaces, which
skip_whitespace() read one character at a time. For contiguous input
(bulk_scan) the lexer now skips the spaces after a newline 8 bytes at a
time directly in the input buffer, including the leading spaces of the
last, partial word. This bypasses get(), which is safe because
bulk_scan implies the lazy position from the previous commit: nothing
is counted per character.

Compared to lazy positions alone, pretty-printed input (twitter,
citm_catalog with indent 2 and 4) parses up to 13% faster and accept()
gets up to 30% faster (GCC 13 and clang 18 on x86-64, Apple clang 21 on
arm64). Minified input is unchanged within noise.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-13 11:24:24 +02:00
1 parent 1d4dfc3af0
commit da4ce8b8a3
6 files changed
+227 -25

No files matched your search

+43
View File
@@ -17,6 +17,7 @@
#include <utility> // move
#include <vector> // vector
#include <nlohmann/detail/bit_ops.hpp>
#include <nlohmann/detail/input/input_adapters.hpp>
#include <nlohmann/detail/input/number_parse.hpp>
#include <nlohmann/detail/input/position_t.hpp>
@@ -2079,11 +2080,53 @@ scan_number_done:
// into one that reloads the pointer from memory every character
do
{
if (current == '\n')
{
skip_indentation(std::integral_constant<bool, bulk_scan> {});
}
get_ignoring_pending_unget();
}
while (current_is_whitespace());
}
/*!
@brief contiguous input: skip the space indentation that follows a newline
8 bytes at a time
Pretty-printed JSON indents every line with a run of spaces, which the loop
in skip_whitespace() would otherwise read one character at a time. The
spaces are skipped 8 bytes at a time; the last 7 bytes of the input and any
other whitespace are left to that loop. Skipping bypasses get(), which is
fine because bulk_scan implies lazy_position: the position is derived from
the adapter, not counted per character.
*/
void skip_indentation(std::true_type /*bulk*/)
{
JSON_ASSERT(!next_unget);
const std::size_t remaining = ia.bulk_remaining();
if (remaining < 8)
{
return;
}
const char* const data = reinterpret_cast<const char*>(ia.bulk_data());
std::size_t pos = 0;
for (; pos + 8 <= remaining; pos += 8)
{
// little-endian, so the first byte is the lowest
const std::uint64_t non_space = read_eight_bytes(data + pos) ^ 0x2020202020202020ull;
if (non_space != 0)
{
// skip the spaces before the first other byte of this word
pos += static_cast<std::size_t>(count_trailing_zeros(non_space)) / 8;
break;
}
}
ia.bulk_skip(pos);
}
/// streaming input: no bulk fast path
void skip_indentation(std::false_type /*bulk*/) const noexcept {}
token_type scan()
{
// initially, skip the BOM