mirror of
https://github.com/nlohmann/json.git
synced 2026-11-04 11:49:22 +00:00
docs: make the comparison of the 3.13.0 post fairer
Three reviews of "How it compares" (a maintainer audit of the harness, a
methodology review, a reader's red team) led to a full rerun on the Mac and
x86-64 with simdjson 5.1.1 and new ways in the harness:
- in-place parses with the copy made before the timer ("(copy untimed)",
never chosen as a library's best) and json_document::parse_copy();
- RapidJSON through StringStream, and with default precision in validate,
which reads no number;
- BENCH_ONLY takes several prefixes.
The post now gives the copy's real cost (11-14 us for twitter.json on the
Mac, not 30), what yyjson gains without it, the run-to-run noise, the LTO
check, the single-file losses that were missing, and qualifies the headline
claim. Every number in the section and Appendix E is recomputed; the
NUL-byte remark moved out of the body.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
14 files changed
+15257
-11745
No files matched your search
@@ -43,6 +43,10 @@ files use the select and edit of their originals.
|
||||
The input is a `const std::string` holding the file, with at least 64 bytes of spare capacity (`reserve`), so libraries
|
||||
that need padding (simdjson) can use it without a copy. A variant that modifies the input (in-situ parsing: yyjson
|
||||
`YYJSON_READ_INSITU`, RapidJSON `ParseInsitu`, simd-json) copies it inside the timed call; its name says `(copy)`.
|
||||
The C++ in-situ variants also run as `(copy untimed)`: the copy is made before the timer starts and freed after it
|
||||
stops, as for a program that reads a file into a buffer it owns and no longer needs. They are reported but never
|
||||
chosen as a library's best. `json_document`'s `parse_copy` is the opposite case: a document that owns a copy of its
|
||||
text.
|
||||
Each timed call starts from the text and frees everything it built, except variants named `(reused)` or
|
||||
`(reused allocator)`, which keep a parser, document or allocator across calls (simdjson's reused parsers,
|
||||
`json_document::read()`, Boost.JSON's reset `parser` and `basic_parser`, yyjson's dynamic allocator, RapidJSON's
|
||||
@@ -107,11 +111,11 @@ Level on the Mac in brackets where it is not `full` (`P` = partial, `U` = no UTF
|
||||
|
||||
| library | validate | sum | select | edit | minify | write |
|
||||
|---|---|---|---|---|---|---|
|
||||
| nlohmann json_document | `accept`, `parse`, `read (reused)` | `parse`, `read (reused)` | `parse`, `structs (from_json)`, `read (reused)` | `editable`, `editable (source numbers)` (`json_editable_document`: set/erase, `dump` with `number_format::source`) | `parse + dump`, `parse + dump (source numbers)`, `read + dump (reused)` | `dump`, `dump (source numbers)` |
|
||||
| nlohmann json_document | `accept`, `parse`, `read (reused)` | `parse`, `parse_copy`, `read (reused)` | `parse`, `parse_copy`, `structs (from_json)`, `read (reused)` | `editable`, `editable (source numbers)` (`json_editable_document`: set/erase, `dump` with `number_format::source`) | `parse + dump`, `parse_copy + dump`, `parse + dump (source numbers)`, `read + dump (reused)` | `dump`, `dump (source numbers)` |
|
||||
| nlohmann json | `accept`, `parse` | `parse`, `sax` | `parse`, `structs (from_json)` | `parse`, `ordered_json` | `parse + dump` | `dump` |
|
||||
| yyjson | `read`, `insitu (copy)`, `read (raw numbers)`, `read (reused allocator)` | `read`, `insitu (copy)`, `read (reused allocator)` | `read`, `insitu (copy)`, `read (raw numbers)`, `read (reused allocator)` | `mut copy`, `insitu (copy) + mut copy` (`yyjson_doc_mut_copy`) | `read + write`, `insitu (copy) + write`, `read (raw numbers) + write`, `read + write (reused allocator)` | `write` |
|
||||
| yyjson | `read`, `insitu (copy)`, `insitu (copy untimed)`, `read (raw numbers)`, `read (reused allocator)` | `read`, `insitu (copy)`, `insitu (copy untimed)`, `read (reused allocator)` | `read`, `insitu (copy)`, `insitu (copy untimed)`, `read (raw numbers)`, `read (reused allocator)` | `mut copy`, `insitu (copy) + mut copy` (`yyjson_doc_mut_copy`), `insitu (copy untimed) + mut copy` | `read + write`, `insitu (copy) + write`, `insitu (copy untimed) + write`, `read (raw numbers) + write`, `read + write (reused allocator)` | `write` |
|
||||
| simdjson | `DOM`, `DOM (reused)` | `DOM`, `DOM (reused)`, `On-Demand`, `On-Demand (reused)` | `DOM`, `DOM (reused)`, `On-Demand` [P], `On-Demand (reused)` [P], `On-Demand structs` [P], `On-Demand (key selectors)` [P] | n/a (no mutable document) | `DOM + to_string`, `DOM (reused) + to_string`, `minify` [P] | `to_string` |
|
||||
| RapidJSON | `SAX` (null handler), `Document`, `insitu (copy)`, `SAX (raw numbers)`, `Document (reused allocator)` | `Document`, `SAX`, `insitu (copy)`, `Document (reused allocator)` | `Document`, `insitu (copy)`, `Document (raw numbers)`, `Document (reused allocator)` | `Document`, `insitu (copy)` | `Document + Writer`, `SAX to Writer`, `SAX to Writer (raw numbers)`, `Document + Writer (reused allocator)` | `Writer` |
|
||||
| RapidJSON | `SAX` (null handler), `Document`, `insitu (copy)`, `insitu (copy untimed)`, `SAX (string stream)`, `SAX (default precision)`, `SAX (string stream, default precision)`, `SAX (raw numbers)`, `Document (reused allocator)` | `Document`, `SAX`, `insitu (copy)`, `insitu (copy untimed)`, `SAX (string stream)`, `Document (reused allocator)` | `Document`, `insitu (copy)`, `insitu (copy untimed)`, `Document (string stream)`, `Document (raw numbers)`, `Document (reused allocator)` | `Document`, `insitu (copy)`, `insitu (copy untimed)`, `Document (string stream)` | `Document + Writer`, `SAX to Writer`, `SAX to Writer (string stream)`, `SAX to Writer (raw numbers)`, `Document + Writer (reused allocator)` | `Writer` |
|
||||
| RapidJSON, `(no encoding check)` | the same [U] | the same [U] | the same [U] | the same [U] | the same [U] | |
|
||||
| sonic-cpp | `Document` [U] | `Document` [U] | `Document` [U] | `Document` [U] | `Document + Serialize` [U] | `Serialize` |
|
||||
| Glaze | `validate_json`, `generic` [P], `generic (double)` [P], `generic (strict)`, `generic (double, strict)` | `generic` [P], `generic (double)` [P], `generic (strict)`, `generic (double, strict)`, `lazy_json` [P], `lazy_json (string_view)` (x) | `structs` [P], `structs (validate_skipped)`, `generic` [P], `generic (double)` [P], `generic (strict)`, `generic (double, strict)`, `lazy_json` [P], `lazy_json (string_view)` (x) | `generic` [P], `generic (double)` (x), `generic (strict)`, `generic (double, strict)` (x) | `generic` [P], `generic (double)` (x), `generic (strict)`, `generic (double, strict)` (x), `minify_json` [P] | `generic`, `generic (double)` (x), `generic (strict)`, `generic (double, strict)` (x) |
|
||||
@@ -140,7 +144,10 @@ described struct, so their members are not counted. `key selectors` are simdjson
|
||||
`DeserializeSeed` that sums without building a value; `SAX` and `sax` are the libraries' event interfaces with a
|
||||
summing (or, for `validate`, an empty) handler. RapidJSON parses with `kParseFullPrecisionFlag` (correctly rounded
|
||||
floats; without it, sums and written floats are wrong) and Boost.JSON with `number_precision::precise`, as every other
|
||||
library rounds correctly by default.
|
||||
library rounds correctly by default; only `validate`, which reads no number, also has RapidJSON's `(default
|
||||
precision)`. RapidJSON's `Document::Parse(data, size)` and our `SAX` read through `EncodedInputStream<MemoryStream>`;
|
||||
`(string stream)` reads the NUL-terminated string through `StringStream`, as `Document::Parse(const char*)` does,
|
||||
where RapidJSON has a SIMD string scanner (used only without `kParseValidateEncodingFlag`).
|
||||
|
||||
Variants named `(raw numbers)` or `(numbers: none)` use a library's mode that validates numbers without converting
|
||||
them (yyjson `YYJSON_READ_NUMBER_AS_RAW`, RapidJSON `kParseNumbersAsStringsFlag`, Boost.JSON
|
||||
@@ -159,7 +166,7 @@ control characters, e.g. U+0001, unescaped, which is not JSON; `gsoc-2018.json`
|
||||
|---|---|---|
|
||||
| `json_document`, `json` | this checkout | |
|
||||
| yyjson | 0.13.0 | |
|
||||
| simdjson | 5.0.8 | |
|
||||
| simdjson | 5.1.1 | the single-header files of the tag |
|
||||
| sonic-cpp | 1.0.2 | |
|
||||
| RapidJSON | master 24b5e7a8b27f (2025-02-05) | |
|
||||
| Glaze | 9.1.0 | `generic_u64` and `generic` (owning DOMs, member order kept) |
|
||||
@@ -173,7 +180,8 @@ SHA-256 of the downloads:
|
||||
30edf9e47c417f653546c39295fdb66ba5f89fb37eb3780122c6ee66605e5837 sonic-cpp v1.0.2.tar.gz
|
||||
2d2601a82d2d3b7e143a3c8d43ef616671391034bc46891a9816b79cf2d3e7a8 rapidjson 24b5e7a8b27f42fa16b96fc70aade9106cf7102f.tar.gz
|
||||
34e0f62a2bc11ab20d601e8ca1cc2b2079503aa45119a19133d89d19b94a0fae yyjson 0.13.0.tar.gz
|
||||
356cc3a2d6a3d0d45bf5eba813e58ef4f2f6d9cc2a25d8aada2563bc13755789 simdjson-v5.0.8.tar.gz
|
||||
f3e5ee5f39051960b5ce65762ee4b6919caf270675eb52240ebd92f21a5bcefd simdjson v5.1.1 singleheader/simdjson.h
|
||||
3ea5f501cef6c2e16e28095b566feeb56021ca473aaf50aad32102f45d734195 simdjson v5.1.1 singleheader/simdjson.cpp
|
||||
c4a3b310ddd2472416e091067166b0713be97c63f38c212c484ada022fd296ce boost_1_92_0.tar.gz
|
||||
```
|
||||
|
||||
@@ -189,13 +197,13 @@ at most the given number).
|
||||
|
||||
All libraries except yyjson and simdjson are used header-only. With the downloads unpacked into `DL`
|
||||
(`compare.py --download` fetches yyjson and Boost into its build directory, and simdjson 4.6.11, so take simdjson
|
||||
5.0.8 from its release archive):
|
||||
5.1.1's `singleheader` files from its tag):
|
||||
|
||||
```sh
|
||||
cc -O3 -DNDEBUG -c $DL/yyjson-0.13.0/src/yyjson.c -o yyjson.o
|
||||
c++ -std=c++17 -O3 -DNDEBUG -c $DL/simdjson-5.0.8/singleheader/simdjson.cpp -o simdjson.o
|
||||
c++ -std=c++17 -O3 -DNDEBUG -c $DL/simdjson-5.1.1/singleheader/simdjson.cpp -o simdjson.o
|
||||
c++ -std=c++23 -O3 -DNDEBUG -DRAPIDJSON_NEON \
|
||||
-I ../../../../include -I $DL/yyjson-0.13.0/src -I $DL/simdjson-5.0.8/singleheader -I $DL/boost_1_92_0 \
|
||||
-I ../../../../include -I $DL/yyjson-0.13.0/src -I $DL/simdjson-5.1.1/singleheader -I $DL/boost_1_92_0 \
|
||||
-I $DL/glaze-9.1.0/include -I $DL/rapidjson/include -I $DL/sonic/include \
|
||||
bench_all.cpp yyjson.o simdjson.o -o bench_all
|
||||
./bench_all <json_test_data> 30 > mac-cpp-1.log
|
||||
|
||||
@@ -21,6 +21,7 @@
|
||||
// Usage: bench_all <json_test_data dir> [rounds] [document] [use case]
|
||||
// bench_all --files <dir> [rounds] [document] [use case] (every *.json file of dir: validate, sum, minify, write)
|
||||
// bench_all --conformance <dir> (every validate variant on every *.json file of dir)
|
||||
// BENCH_ONLY=<prefix>[,<prefix>...] times only the variants whose name starts with one of them (e.g. "nlohmann ")
|
||||
#include <nlohmann/json_view.hpp>
|
||||
|
||||
#include <boost/describe.hpp>
|
||||
@@ -357,6 +358,79 @@ struct yy_insitu
|
||||
}
|
||||
};
|
||||
|
||||
// "(copy untimed)": an in-situ way whose input copy is made before the timer starts and freed after it stops, as for
|
||||
// a program that reads a file into a buffer it owns and no longer needs; never chosen as a library's best. Outside
|
||||
// the timing loop (the checks and probes), take() copies the input itself.
|
||||
struct input_copies
|
||||
{
|
||||
std::vector<char*> bufs;
|
||||
std::size_t next = 0;
|
||||
void make(const std::string& s, int n)
|
||||
{
|
||||
clear();
|
||||
for (int i = 0; i < n; ++i)
|
||||
{
|
||||
char* b = static_cast<char*>(std::malloc(s.size() + YYJSON_PADDING_SIZE));
|
||||
std::memcpy(b, s.data(), s.size());
|
||||
std::memset(b + s.size(), 0, YYJSON_PADDING_SIZE);
|
||||
bufs.push_back(b);
|
||||
}
|
||||
}
|
||||
void clear()
|
||||
{
|
||||
for (char* b : bufs)
|
||||
{
|
||||
std::free(b);
|
||||
}
|
||||
bufs.clear();
|
||||
next = 0;
|
||||
}
|
||||
};
|
||||
static input_copies g_copies;
|
||||
|
||||
struct taken_copy
|
||||
{
|
||||
char* buf = nullptr;
|
||||
bool owned = false;
|
||||
explicit taken_copy(const std::string& s)
|
||||
{
|
||||
if (g_copies.next < g_copies.bufs.size())
|
||||
{
|
||||
buf = g_copies.bufs[g_copies.next++];
|
||||
return;
|
||||
}
|
||||
owned = true;
|
||||
buf = static_cast<char*>(std::malloc(s.size() + YYJSON_PADDING_SIZE));
|
||||
std::memcpy(buf, s.data(), s.size());
|
||||
std::memset(buf + s.size(), 0, YYJSON_PADDING_SIZE);
|
||||
}
|
||||
taken_copy(const taken_copy&) = delete;
|
||||
taken_copy& operator=(const taken_copy&) = delete;
|
||||
~taken_copy()
|
||||
{
|
||||
if (owned)
|
||||
{
|
||||
std::free(buf);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// yyjson in situ on a copy made before the timer
|
||||
struct yy_insitu_untimed
|
||||
{
|
||||
taken_copy c;
|
||||
yyjson_doc* doc = nullptr;
|
||||
explicit yy_insitu_untimed(const std::string& s)
|
||||
: c(s), doc(yyjson_read_opts(c.buf, s.size(), YYJSON_READ_INSITU, nullptr, nullptr))
|
||||
{}
|
||||
yy_insitu_untimed(const yy_insitu_untimed&) = delete;
|
||||
yy_insitu_untimed& operator=(const yy_insitu_untimed&) = delete;
|
||||
~yy_insitu_untimed()
|
||||
{
|
||||
yyjson_doc_free(doc);
|
||||
}
|
||||
};
|
||||
|
||||
// ---- simdjson DOM and On-Demand
|
||||
// after the type switch, the unchecked accessors (value_unsafe()) instead of the throwing conversions (int64_t(e),
|
||||
// as in the 3.13.0 post; measured the same)
|
||||
@@ -579,6 +653,16 @@ static bool rj_read(const std::string& s, Handler& h)
|
||||
return !reader.Parse<Flags>(is, h).IsError();
|
||||
}
|
||||
|
||||
// RapidJSON's Reader on the NUL-terminated string, as Document::Parse(const char*) does it: StringStream has
|
||||
// RapidJSON's SIMD string scanner, which skips the encoding check
|
||||
template<unsigned Flags, class Handler>
|
||||
static bool rj_read_cstr(const std::string& s, Handler& h)
|
||||
{
|
||||
rapidjson::StringStream is(s.c_str());
|
||||
rapidjson::Reader reader;
|
||||
return !reader.Parse<Flags>(is, h).IsError();
|
||||
}
|
||||
|
||||
// RapidJSON's Writer writes a raw number (kParseNumbersAsStringsFlag) as a string: write its text instead
|
||||
struct rj_raw_writer : rapidjson::Writer<rapidjson::StringBuffer>
|
||||
{
|
||||
@@ -2706,6 +2790,7 @@ struct variant
|
||||
std::string name; // "<library>: <variant>"
|
||||
std::function<bool(const std::string&, result&)> fn;
|
||||
bool any_order = false; // sum: visits objects in key order, so its sum may differ in the last bits
|
||||
bool untimed_copy = false; // takes its input copy from g_copies, made before the timer starts
|
||||
};
|
||||
|
||||
struct context
|
||||
@@ -2732,6 +2817,7 @@ static std::vector<variant> validate_variants(context& c)
|
||||
{"nlohmann json: parse", [](const std::string & s, result & r) { json j = json::parse(s); r.acc = static_cast<double>(j.size()); return true; }},
|
||||
{"yyjson: read", [](const std::string & s, result&) { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); const bool ok = d != nullptr; yyjson_doc_free(d); return ok; }},
|
||||
{"yyjson: insitu (copy)", [](const std::string & s, result&) { yy_insitu d(s); return d.doc != nullptr; }},
|
||||
{"yyjson: insitu (copy untimed)", [](const std::string & s, result&) { yy_insitu_untimed d(s); return d.doc != nullptr; }, false, true},
|
||||
{"yyjson: read (raw numbers)", [](const std::string & s, result&) { yyjson_doc* d = yyjson_read(s.data(), s.size(), yy_raw); const bool ok = d != nullptr; yyjson_doc_free(d); return ok; }},
|
||||
{"yyjson: read (reused allocator)", [&c](const std::string & s, result&) { yyjson_doc* d = yyjson_read_opts(const_cast<char*>(s.data()), s.size(), 0, c.yy.a, nullptr); const bool ok = d != nullptr; yyjson_doc_free(d); return ok; }},
|
||||
{"simdjson: DOM", [](const std::string & s, result&) { simdjson::dom::parser p; return !p.parse(s).error(); }},
|
||||
@@ -2758,11 +2844,13 @@ static std::vector<variant> sum_variants(context& c)
|
||||
return
|
||||
{
|
||||
{"nlohmann json_document: parse", [](const std::string & s, result & r) { auto d = json_document::parse(s); walk(d.root(), r.st); return true; }},
|
||||
{"nlohmann json_document: parse_copy", [](const std::string & s, result & r) { auto d = json_document::parse_copy(s); walk(d.root(), r.st); return true; }},
|
||||
{"nlohmann json_document: read (reused)", [&c](const std::string & s, result & r) { c.reused.read(s); walk(c.reused.root(), r.st); return true; }},
|
||||
{"nlohmann json: parse", [](const std::string & s, result & r) { json j = json::parse(s); walk(j, r.st); return true; }, true},
|
||||
{"nlohmann json: sax", [](const std::string & s, result & r) { json_sum_sax h{&r.st}; return json::sax_parse(s, &h); }},
|
||||
{"yyjson: read", [](const std::string & s, result & r) { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); if (d == nullptr) { return false; } walk(yyjson_doc_get_root(d), r.st); yyjson_doc_free(d); return true; }},
|
||||
{"yyjson: insitu (copy)", [](const std::string & s, result & r) { yy_insitu d(s); if (d.doc == nullptr) { return false; } walk(yyjson_doc_get_root(d.doc), r.st); return true; }},
|
||||
{"yyjson: insitu (copy untimed)", [](const std::string & s, result & r) { yy_insitu_untimed d(s); if (d.doc == nullptr) { return false; } walk(yyjson_doc_get_root(d.doc), r.st); return true; }, false, true},
|
||||
{"yyjson: read (reused allocator)", [&c](const std::string & s, result & r) { yyjson_doc* d = yyjson_read_opts(const_cast<char*>(s.data()), s.size(), 0, c.yy.a, nullptr); if (d == nullptr) { return false; } walk(yyjson_doc_get_root(d), r.st); yyjson_doc_free(d); return true; }},
|
||||
{"simdjson: DOM", [](const std::string & s, result & r) { simdjson::dom::parser p; auto e = p.parse(s); if (e.error()) { return false; } walk(e.value_unsafe(), r.st); return true; }},
|
||||
{"simdjson: DOM (reused)", [&c](const std::string & s, result & r) { auto e = c.sj.parse(s); if (e.error()) { return false; } walk(e.value_unsafe(), r.st); return true; }},
|
||||
@@ -2790,12 +2878,14 @@ static std::vector<variant> select_variants(context& c)
|
||||
return
|
||||
{
|
||||
{"nlohmann json_document: parse", [n](const std::string & s, result & r) { auto d = json_document::parse(s); r.acc = pick<acc_view>(n, d.root()); return true; }},
|
||||
{"nlohmann json_document: parse_copy", [n](const std::string & s, result & r) { auto d = json_document::parse_copy(s); r.acc = pick<acc_view>(n, d.root()); return true; }},
|
||||
{"nlohmann json_document: read (reused)", [&c, n](const std::string & s, result & r) { c.reused.read(s); r.acc = pick<acc_view>(n, c.reused.root()); return true; }},
|
||||
{"nlohmann json: parse", [n](const std::string & s, result & r) { json j = json::parse(s); r.acc = pick<acc_json>(n, &j); return true; }},
|
||||
{"nlohmann json_document: structs (from_json)", [n](const std::string & s, result & r) { auto d = json_document::parse(s); r.acc = pick_nlohmann_structs(n, d.root()); return true; }},
|
||||
{"nlohmann json: structs (from_json)", [n](const std::string & s, result & r) { r.acc = pick_nlohmann_structs(n, json::parse(s)); return true; }},
|
||||
{"yyjson: read", [n](const std::string & s, result & r) { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); if (d == nullptr) { return false; } r.acc = pick<acc_yyjson>(n, yyjson_doc_get_root(d)); yyjson_doc_free(d); return true; }},
|
||||
{"yyjson: insitu (copy)", [n](const std::string & s, result & r) { yy_insitu d(s); if (d.doc == nullptr) { return false; } r.acc = pick<acc_yyjson>(n, yyjson_doc_get_root(d.doc)); return true; }},
|
||||
{"yyjson: insitu (copy untimed)", [n](const std::string & s, result & r) { yy_insitu_untimed d(s); if (d.doc == nullptr) { return false; } r.acc = pick<acc_yyjson>(n, yyjson_doc_get_root(d.doc)); return true; }, false, true},
|
||||
{"yyjson: read (raw numbers)", [n](const std::string & s, result & r) { yyjson_doc* d = yyjson_read(s.data(), s.size(), yy_raw); if (d == nullptr) { return false; } r.acc = pick<acc_yyjson_raw>(n, yyjson_doc_get_root(d)); yyjson_doc_free(d); return true; }},
|
||||
{"yyjson: read (reused allocator)", [&c, n](const std::string & s, result & r) { yyjson_doc* d = yyjson_read_opts(const_cast<char*>(s.data()), s.size(), 0, c.yy.a, nullptr); if (d == nullptr) { return false; } r.acc = pick<acc_yyjson>(n, yyjson_doc_get_root(d)); yyjson_doc_free(d); return true; }},
|
||||
{"simdjson: DOM", [n](const std::string & s, result & r) { simdjson::dom::parser p; auto e = p.parse(s); if (e.error()) { return false; } r.acc = pick<acc_simdjson>(n, e.value_unsafe()); return true; }},
|
||||
@@ -2833,6 +2923,7 @@ static std::vector<variant> edit_variants(context& c)
|
||||
{"nlohmann json: ordered_json", [n](const std::string & s, result & r) { ordered_json j = ordered_json::parse(s); edit(n, j); return emit(r, j.dump()); }},
|
||||
{"yyjson: mut copy", [n](const std::string & s, result & r) { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); const bool ok = edit_yyjson(n, d, r); yyjson_doc_free(d); return ok; }},
|
||||
{"yyjson: insitu (copy) + mut copy", [n](const std::string & s, result & r) { yy_insitu d(s); return edit_yyjson(n, d.doc, r); }},
|
||||
{"yyjson: insitu (copy untimed) + mut copy", [n](const std::string & s, result & r) { yy_insitu_untimed d(s); return edit_yyjson(n, d.doc, r); }, false, true},
|
||||
{"sonic-cpp: Document", [n](const std::string & s, result & r) { return edit_sonic(n, s, r); }},
|
||||
#if !BENCH_NO_GLAZE
|
||||
{"Glaze: generic", [n](const std::string & s, result & r) { return edit_glaze(n, s, r); }},
|
||||
@@ -2849,11 +2940,13 @@ static std::vector<variant> minify_variants(context& c)
|
||||
return
|
||||
{
|
||||
{"nlohmann json_document: parse + dump", [](const std::string & s, result & r) { auto d = json_document::parse(s); return emit(r, d.root().dump()); }},
|
||||
{"nlohmann json_document: parse_copy + dump", [](const std::string & s, result & r) { auto d = json_document::parse_copy(s); return emit(r, d.root().dump()); }},
|
||||
{"nlohmann json_document: parse + dump (source numbers)", [](const std::string & s, result & r) { auto d = json_document::parse(s); return emit(r, d.root().dump(-1, ' ', false, json_view::number_format::source)); }},
|
||||
{"nlohmann json_document: read + dump (reused)", [&c](const std::string & s, result & r) { c.reused.read(s); return emit(r, c.reused.root().dump()); }},
|
||||
{"nlohmann json: parse + dump", [](const std::string & s, result & r) { json j = json::parse(s); return emit(r, j.dump()); }},
|
||||
{"yyjson: read + write", [](const std::string & s, result & r) { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); if (d == nullptr) { return false; } std::size_t n = 0; char* o = yyjson_write(d, 0, &n); const bool ok = o != nullptr && emit(r, o, n); std::free(o); yyjson_doc_free(d); return ok; }},
|
||||
{"yyjson: insitu (copy) + write", [](const std::string & s, result & r) { yy_insitu d(s); if (d.doc == nullptr) { return false; } std::size_t n = 0; char* o = yyjson_write(d.doc, 0, &n); const bool ok = o != nullptr && emit(r, o, n); std::free(o); return ok; }},
|
||||
{"yyjson: insitu (copy untimed) + write", [](const std::string & s, result & r) { yy_insitu_untimed d(s); if (d.doc == nullptr) { return false; } std::size_t n = 0; char* o = yyjson_write(d.doc, 0, &n); const bool ok = o != nullptr && emit(r, o, n); std::free(o); return ok; }, false, true},
|
||||
// the raw number text is written verbatim
|
||||
{"yyjson: read (raw numbers) + write", [](const std::string & s, result & r) { yyjson_doc* d = yyjson_read(s.data(), s.size(), yy_raw); if (d == nullptr) { return false; } std::size_t n = 0; char* o = yyjson_write(d, 0, &n); const bool ok = o != nullptr && emit(r, o, n); std::free(o); yyjson_doc_free(d); return ok; }},
|
||||
{"yyjson: read + write (reused allocator)", [&c](const std::string & s, result & r) { yyjson_alc* a = c.yy.a; yyjson_doc* d = yyjson_read_opts(const_cast<char*>(s.data()), s.size(), 0, a, nullptr); if (d == nullptr) { return false; } std::size_t n = 0; char* o = yyjson_write_opts(d, 0, a, &n, nullptr); const bool ok = o != nullptr && emit(r, o, n); if (o != nullptr) { a->free(a->ctx, o); } yyjson_doc_free(d); return ok; }},
|
||||
@@ -2887,9 +2980,14 @@ static void add_rapidjson(std::vector<variant>& vs, const std::string& uc, const
|
||||
rj_reuse* const rp = &rr;
|
||||
if (uc == "validate")
|
||||
{
|
||||
// validate reads no number, so the default (not correctly rounded) precision is allowed here
|
||||
vs.push_back({std::string("RapidJSON: SAX") + sfx, [](const std::string & s, result&) { rapidjson::BaseReaderHandler<> h; return rj_read<F>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: Document") + sfx, [](const std::string & s, result&) { rapidjson::Document d; d.Parse<F>(s.data(), s.size()); return !d.HasParseError(); }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy)") + sfx, [](const std::string & s, result&) { std::string b = s; rapidjson::Document d; d.ParseInsitu<F>(&b[0]); return !d.HasParseError(); }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy untimed)") + sfx, [](const std::string & s, result&) { taken_copy b(s); rapidjson::Document d; d.ParseInsitu<F>(b.buf); return !d.HasParseError(); }, false, true});
|
||||
vs.push_back({std::string("RapidJSON: SAX (string stream)") + sfx, [](const std::string & s, result&) { rapidjson::BaseReaderHandler<> h; return rj_read_cstr<F>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: SAX (default precision)") + sfx, [](const std::string & s, result&) { rapidjson::BaseReaderHandler<> h; return rj_read<F & ~rapidjson::kParseFullPrecisionFlag>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: SAX (string stream, default precision)") + sfx, [](const std::string & s, result&) { rapidjson::BaseReaderHandler<> h; return rj_read_cstr<F & ~rapidjson::kParseFullPrecisionFlag>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: SAX (raw numbers)") + sfx, [](const std::string & s, result&) { rapidjson::BaseReaderHandler<> h; return rj_read<F | rj_raw>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: Document (reused allocator)") + sfx, [rp](const std::string & s, result&) { return rp->parse<F>(s, [](const rapidjson::Value&) { return true; }); }});
|
||||
}
|
||||
@@ -2898,12 +2996,16 @@ static void add_rapidjson(std::vector<variant>& vs, const std::string& uc, const
|
||||
vs.push_back({std::string("RapidJSON: Document") + sfx, [](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F>(s.data(), s.size()); if (d.HasParseError()) { return false; } walk(d, r.st); return true; }});
|
||||
vs.push_back({std::string("RapidJSON: SAX") + sfx, [](const std::string & s, result & r) { rj_sum h(&r.st); return rj_read<F>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy)") + sfx, [](const std::string & s, result & r) { std::string b = s; rapidjson::Document d; d.ParseInsitu<F>(&b[0]); if (d.HasParseError()) { return false; } walk(d, r.st); return true; }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy untimed)") + sfx, [](const std::string & s, result & r) { taken_copy b(s); rapidjson::Document d; d.ParseInsitu<F>(b.buf); if (d.HasParseError()) { return false; } walk(d, r.st); return true; }, false, true});
|
||||
vs.push_back({std::string("RapidJSON: SAX (string stream)") + sfx, [](const std::string & s, result & r) { rj_sum h(&r.st); return rj_read_cstr<F>(s, h); }});
|
||||
vs.push_back({std::string("RapidJSON: Document (reused allocator)") + sfx, [rp](const std::string & s, result & r) { return rp->parse<F>(s, [&r](const rapidjson::Value & d) { walk(d, r.st); return true; }); }});
|
||||
}
|
||||
else if (uc == "select")
|
||||
{
|
||||
vs.push_back({std::string("RapidJSON: Document") + sfx, [n](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F>(s.data(), s.size()); if (d.HasParseError()) { return false; } r.acc = pick<acc_rapidjson>(n, &d); return true; }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy)") + sfx, [n](const std::string & s, result & r) { std::string b = s; rapidjson::Document d; d.ParseInsitu<F>(&b[0]); if (d.HasParseError()) { return false; } r.acc = pick<acc_rapidjson>(n, &d); return true; }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy untimed)") + sfx, [n](const std::string & s, result & r) { taken_copy b(s); rapidjson::Document d; d.ParseInsitu<F>(b.buf); if (d.HasParseError()) { return false; } r.acc = pick<acc_rapidjson>(n, &d); return true; }, false, true});
|
||||
vs.push_back({std::string("RapidJSON: Document (string stream)") + sfx, [n](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F>(s.c_str()); if (d.HasParseError()) { return false; } r.acc = pick<acc_rapidjson>(n, &d); return true; }});
|
||||
vs.push_back({std::string("RapidJSON: Document (raw numbers)") + sfx, [n](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F | rj_raw>(s.data(), s.size()); if (d.HasParseError()) { return false; } r.acc = pick<acc_rapidjson_raw>(n, &d); return true; }});
|
||||
vs.push_back({std::string("RapidJSON: Document (reused allocator)") + sfx, [n, rp](const std::string & s, result & r) { return rp->parse<F>(s, [&](const rapidjson::Value & d) { r.acc = pick<acc_rapidjson>(n, &d); return true; }); }});
|
||||
}
|
||||
@@ -2911,11 +3013,14 @@ static void add_rapidjson(std::vector<variant>& vs, const std::string& uc, const
|
||||
{
|
||||
vs.push_back({std::string("RapidJSON: Document") + sfx, [n](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F>(s.data(), s.size()); return !d.HasParseError() && edit_rapidjson(n, d, r); }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy)") + sfx, [n](const std::string & s, result & r) { std::string b = s; rapidjson::Document d; d.ParseInsitu<F>(&b[0]); return !d.HasParseError() && edit_rapidjson(n, d, r); }});
|
||||
vs.push_back({std::string("RapidJSON: insitu (copy untimed)") + sfx, [n](const std::string & s, result & r) { taken_copy b(s); rapidjson::Document d; d.ParseInsitu<F>(b.buf); return !d.HasParseError() && edit_rapidjson(n, d, r); }, false, true});
|
||||
vs.push_back({std::string("RapidJSON: Document (string stream)") + sfx, [n](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F>(s.c_str()); return !d.HasParseError() && edit_rapidjson(n, d, r); }});
|
||||
}
|
||||
else if (uc == "minify")
|
||||
{
|
||||
vs.push_back({std::string("RapidJSON: Document + Writer") + sfx, [](const std::string & s, result & r) { rapidjson::Document d; d.Parse<F>(s.data(), s.size()); if (d.HasParseError()) { return false; } rapidjson::StringBuffer sb; rapidjson::Writer<rapidjson::StringBuffer> w(sb); return d.Accept(w) && emit(r, sb.GetString(), sb.GetSize()); }});
|
||||
vs.push_back({std::string("RapidJSON: SAX to Writer") + sfx, [](const std::string & s, result & r) { rapidjson::StringBuffer sb; rapidjson::Writer<rapidjson::StringBuffer> w(sb); return rj_read<F>(s, w) && emit(r, sb.GetString(), sb.GetSize()); }});
|
||||
vs.push_back({std::string("RapidJSON: SAX to Writer (string stream)") + sfx, [](const std::string & s, result & r) { rapidjson::StringBuffer sb; rapidjson::Writer<rapidjson::StringBuffer> w(sb); return rj_read_cstr<F>(s, w) && emit(r, sb.GetString(), sb.GetSize()); }});
|
||||
vs.push_back({std::string("RapidJSON: SAX to Writer (raw numbers)") + sfx, [](const std::string & s, result & r) { rapidjson::StringBuffer sb; rj_raw_writer w(sb); return rj_read<F | rapidjson::kParseNumbersAsStringsFlag>(s, w) && emit(r, sb.GetString(), sb.GetSize()); }});
|
||||
vs.push_back({std::string("RapidJSON: Document + Writer (reused allocator)") + sfx, [rp](const std::string & s, result & r) { return rp->parse<F>(s, [&r](const rapidjson::Value & d) { rapidjson::StringBuffer sb; rapidjson::Writer<rapidjson::StringBuffer> w(sb); return d.Accept(w) && emit(r, sb.GetString(), sb.GetSize()); }); }});
|
||||
}
|
||||
@@ -3286,6 +3391,22 @@ int main(int argc, char** argv)
|
||||
add_rapidjson<rj_flags>(all, uc, dc.base, "", rj);
|
||||
add_rapidjson<rj_fast>(all, uc, dc.base, " (no encoding check)", rj);
|
||||
}
|
||||
// BENCH_ONLY=<prefix>[,<prefix>...]: only the variants whose name starts with one of them (A/B runs of two
|
||||
// nlohmann headers, or a few libraries)
|
||||
if (const char* const only = std::getenv("BENCH_ONLY"))
|
||||
{
|
||||
std::vector<std::string> prefixes;
|
||||
std::string rest = only;
|
||||
for (std::size_t comma; (comma = rest.find(',')) != std::string::npos; rest.erase(0, comma + 1))
|
||||
{
|
||||
prefixes.push_back(rest.substr(0, comma));
|
||||
}
|
||||
prefixes.push_back(rest);
|
||||
all.erase(std::remove_if(all.begin(), all.end(), [&](const variant& v)
|
||||
{
|
||||
return std::none_of(prefixes.begin(), prefixes.end(), [&](const std::string& p) { return v.name.compare(0, p.size(), p) == 0; });
|
||||
}), all.end());
|
||||
}
|
||||
|
||||
// the reference result, from json_document (sum, select) or json (edit, minify, write)
|
||||
stats ref_st;
|
||||
@@ -3401,8 +3522,16 @@ int main(int argc, char** argv)
|
||||
for (const std::size_t k : order)
|
||||
{
|
||||
result res;
|
||||
if (vs[k].untimed_copy)
|
||||
{
|
||||
g_copies.make(s, 1);
|
||||
}
|
||||
run(vs[k], s, res); // untimed: the previous variant's leftovers are cleaned up here
|
||||
g_sink = res.st.num + res.acc;
|
||||
if (vs[k].untimed_copy)
|
||||
{
|
||||
g_copies.make(s, dc.batch);
|
||||
}
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
for (int b = 0; b < dc.batch; ++b)
|
||||
{
|
||||
@@ -3410,6 +3539,7 @@ int main(int argc, char** argv)
|
||||
g_sink = run(vs[k], s, rt) ? rt.st.num + rt.acc : -1;
|
||||
}
|
||||
best[k] = std::min(best[k], std::chrono::duration<double, std::nano>(std::chrono::steady_clock::now() - t0).count() / dc.batch);
|
||||
g_copies.clear();
|
||||
}
|
||||
}
|
||||
std::printf("%-13s %-9s", dc.name.c_str(), uc.c_str());
|
||||
|
||||
@@ -17,8 +17,9 @@ error, or crashes on any probe), or "n/a" (write: no input to validate; counts a
|
||||
worst it got for any file. Prints, for every use case and file,
|
||||
(a) per library the best "full" variant and, if faster, the best variant at any level, with their ratios to the
|
||||
best "full" nlohmann json_document variant (variants named "(reused)" or "(reused allocator)" keep a parser,
|
||||
document or allocator across calls and are never chosen as a library's best), and in a "reused:" line the
|
||||
library's best reused variant in the same way,
|
||||
document or allocator across calls and are never chosen as a library's best; neither are "(copy untimed)"
|
||||
variants, which copy their input before the timer starts), and in a "reused:" line the library's best reused
|
||||
variant in the same way,
|
||||
(b) all variants with their levels and times,
|
||||
(c) the variants below "full" with what the probes found, and the variants excluded for wrong results.
|
||||
--json writes {use case: {file: {library: {"best": {"variant", "us", "level"} or null, "best_any": {...},
|
||||
@@ -54,6 +55,10 @@ def is_reused(name):
|
||||
return re.search(r"\(reused[ )]", name) is not None
|
||||
|
||||
|
||||
def is_untimed_copy(name):
|
||||
return "(copy untimed)" in name
|
||||
|
||||
|
||||
def worse(a, b):
|
||||
return a if a is not None and (b is None or RANK[a] >= RANK[b]) else b
|
||||
|
||||
@@ -159,7 +164,7 @@ def main():
|
||||
return {"variant": variant(x[0]), "us": x[1], "level": levels[(uc, x[0])]} if x else None
|
||||
|
||||
for lib, vs in libs.items():
|
||||
b, a = pick({n: t for n, t in vs.items() if not is_reused(n)})
|
||||
b, a = pick({n: t for n, t in vs.items() if not is_reused(n) and not is_untimed_copy(n)})
|
||||
rb, ra = pick({n: t for n, t in vs.items() if is_reused(n)})
|
||||
entry[lib] = {
|
||||
"best": described(b),
|
||||
|
||||
Reference in new issue
Block a user