json_view: dump canada.json faster than yyjson (batched float decisions)

Dump of float tokens of 16 or 17 digits without an exponent (SIMD targets,
little-endian), in batches of up to 256 floats ahead of the walk, one loop
per step:
- the digits (16 bytes at a time) and the token's last four digits;
- locate_token(): Eisel-Lemire's product gives the double x, and x's position
  relative to the token is exact in a 64-bit fixed-point number (56 bits after
  the point of the token's last digit), with the radius of its interval;
- token_digits_1/2(), one loop per kind (g): the multiple of 10^g nearest to x
  if it is in the interval, else the multiple of 10^(g - 1) nearest to x, the
  digits that dump() (Zmij) writes. Exact ties between two candidates (like
  754570942.33984373, whose double is ...33984375) are left to the full
  conversion;
- the output is the token with its last four digits replaced; a rounding that
  carries further goes to the writer of Zmij's results.
The batch storage is on the heap (allocated by the first conversion), and a
pair of floats like [x, y] in a read-only document is written without a
frame. Reading: get<double>() converts such tokens with a lean Eisel-Lemire
(coordinate_float()).

Output is byte-identical to the previous writer: 10^9 generated tokens in one
array and 3 * 10^8 in arrays of 1 to 9 elements on Apple M1, 2.5 * 10^8 and
10^8 on x86-64 (all exponents, subnormals, 15-18 digits, exact binary
fractions and ties, near-short decimals); get<double>() equals json::parse
on 5 * 10^7 values.

compare.py --download, Apple M1 Max, clang 21, 30 rounds, two runs (and two
with aligned code: -falign-loops=64 -mllvm -align-all-functions=6); json_view,
and the factor of yyjson 0.13.0 over it:
- canada dump      3636 -> 1623 us (0.47x -> 1.06x; aligned 1.05x)
- canada traverse  2287 -> 1817 us (0.85x -> 1.07x; aligned 0.99x)
- canada patch     4628 -> 2735 us (0.80x -> 1.36x)
- canada update    4763 -> 2911 us (0.78x -> 1.28x)
Linux x86-64 (Haswell VPS, GCC 13, one run): canada dump 0.53x -> 0.72x,
traverse 0.89x -> 0.95x, patch 1.01x -> 1.17x, update 0.96x -> 1.13x.

Open: canada traverse in the aligned build (0.99x) and canada dump on x86-64
(0.72x) are still behind yyjson. Canada parse and select are about 2 % slower
in bench_view in both builds, with identical parser code (likely layout; the
corpus parse row is not slower). In the plain build only, rpc, status, and
twitter dump are 2-4 % slower (0-6 % faster in the aligned build).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-25 07:23:43 +01:00
1 parent d38a04bd3e
commit e7d245db7e
4 files changed
+721 -266

No files matched your search

+9 -6
View File
@@ -151,10 +151,12 @@ TEST_CASE("json_view dump")
}
// tokens of 16 and 17 digits without an exponent (like GeoJSON
// coordinates), which dump() converts in batches: random digits, and
// digits near a shorter decimal (...99999997, ...00000003), whose
// rounding carries; in pairs and among other values, so that the
// batches are interrupted; the last token near the end of the coordinates
// coordinates), which dump() converts in batches: doubles exactly
// halfway between two candidates for the shortest digits, random
// digits, and digits near a shorter decimal (...99999997,
// ...00000003), whose rounding carries; in pairs and among other
// values, so that the batches are interrupted; the last token near the
// end of the text
{
std::mt19937_64 tokens(5627); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
const auto draw = [&tokens](std::size_t n)
@@ -162,7 +164,8 @@ TEST_CASE("json_view dump")
const std::uint64_t r = tokens() % n;
return static_cast<std::size_t>(r);
};
std::string coordinates = "[";
std::string coordinates = "[754570942.33984373,257769445.56835938,858106901.94921875,97944964.876953125,"
"[0.5,1e-7],[1.5,-0.0,2.5],[1E2,12.5,0.1,3.25,4.5,5.75,6.125,7.0,8.5],[],[1.5],[1,2.5],[2.5,\"x\"],[[1.5],2.5],";
for (int i = 0; i < 20000; ++i)
{
const std::size_t length = 16 + draw(2);
@@ -191,7 +194,7 @@ TEST_CASE("json_view dump")
{
const ordered_json& e = j[i++];
CHECK(v.dump() == e.dump());
if (v.is_array())
if (v.is_array() && v.size() != 0 && v[0].is_number())
{
CHECK(v[0].get<double>() == e[0].get<double>());
}