mirror of
https://github.com/nlohmann/json.git
synced 2026-11-04 11:49:22 +00:00
Niels Lohmann
ea848cd2e8
A post for the documentation (Home > Blog) on how 3.13.0 became faster: eight themes with the pull requests behind them, the ideas taken from other libraries, what did not work, how the fast paths are tested, and the compile-time cost. Every number comes from a run on 2026-10-27 (3.12.0 against develop plus the open non-draft PRs, Apple M1 Max) or from the linked PR. - docs/mkdocs/docs/home/blog/how_3_13_0_got_faster.md: the post. - docs/mkdocs/scripts/figures/perf_blog.py: draws its 12 figures (overview, eight timeline panels, the SWAR test, the cliffs, json_view throughput) as inline SVGs in the nj-charts style; data/perf_timeline.csv holds the timeline. - tests/benchmarks/history: history.cpp and history.py time one fixed program against the header of every commit that changed it, interleaved, and write one CSV row per commit with its PR. The timeline of the post comes from it. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
44 lines
2.5 KiB
Markdown
44 lines
2.5 KiB
Markdown
# Performance history
|
|
|
|
The benchmarks in [`tests/benchmarks`](../README.md) compare two versions. The program here answers a different
|
|
question: *which commit* made something faster or slower. `history.cpp` is one fixed benchmark that uses only API
|
|
3.12.0 already has. `history.py` compiles it against the `single_include` of every first-parent commit that changed
|
|
`json.hpp` or `json_view.hpp`, runs all of them interleaved, and writes one CSV row per commit with its pull request.
|
|
The timeline in the post [How 3.13.0 got faster](https://json.nlohmann.me/home/blog/how_3_13_0_got_faster/) comes from
|
|
it. It is not built by CMake or run by CI.
|
|
|
|
## Workloads
|
|
|
|
| name | what it does |
|
|
|---|---|
|
|
| `parse_twitter`, `parse_citm`, `parse_canada` | `json::parse()` of a `std::string` (`citm_catalog.json` as distributed is indented by four spaces) |
|
|
| `parse_citm_compact` | the same with a minified copy of `citm_catalog.json` |
|
|
| `parse_twitter_ifstream` | `json::parse()` of an `std::ifstream`, opening the file included |
|
|
| `accept_twitter` | `json::accept()` |
|
|
| `view_parse_*` | `json_document::parse()`; only where the header has `json_view.hpp` |
|
|
| `dump_twitter`, `dump_citm`, `dump_canada`, `dump4_citm` | `dump()` and `dump(4)` |
|
|
| `from_cbor_twitter`, `to_cbor_twitter` | read and write CBOR |
|
|
| `copy_destroy_citm` | copy a parsed document and destroy the copy |
|
|
| `equal_citm` | copy a parsed document and compare it with `==` |
|
|
|
|
Each workload runs in batches of at least 20 ms; the program prints the median of 9 batches in microseconds per call.
|
|
|
|
## Running it
|
|
|
|
```sh
|
|
mkdir -p ~/history/data
|
|
cp <json_test_data>/nativejson-benchmark/{twitter,citm_catalog,canada}.json ~/history/data/
|
|
python3 history.py build --repo ../../.. --work ~/history # about 300 builds, a few minutes in parallel
|
|
caffeinate -i python3 history.py run --work ~/history --rounds 4 # about 1 h 45 min on an M1 Max (about 4 s per binary and round)
|
|
python3 history.py summarize --work ~/history > history.csv
|
|
```
|
|
|
|
`--extra DIR` adds one more header directory, such as an amalgamation of `develop` with open pull requests merged
|
|
in. `run` waits whenever the 1-minute load average is 4 or more.
|
|
|
|
## Reading the results
|
|
|
|
A single commit can move a workload by a few percent through code layout alone: the functions it did not touch
|
|
move to other addresses. Read a change as a step only if it is larger than that and stays. To check one, time the
|
|
two commits against each other with more rounds, or compare instruction counts.
|