Update Google Benchmark to 1.10.0 (#5823)

* Update Google Benchmark to 1.10.0

Bump the pinned release and its archive checksum. Reword the comment and
README: the version is fixed for reproducible builds and updated like any
other dependency; results stay comparable because both sides of a
comparison are built with the same version.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not name the Google Benchmark version in the README

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann authored and GitHub committed 2026-10-14 17:39:16 +02:00
1 parent 91a7a94cc6
commit 1c92a3c5de
2 files changed
+15 -6

No files matched your search

+11 -3
View File
@@ -27,8 +27,11 @@ The input files are those of [nativejson-benchmark](https://github.com/miloyip/n
- Network access on the first configure: CMake downloads Google Benchmark and the
[test data](https://github.com/nlohmann/json_test_data) into the build directory. To reuse a download of the test
data, pass `-DJSON_TestDataDirectory=<build directory>/test_files`.
- Google Benchmark is pinned to a release (1.9.5), so that results from different days stay comparable. To update it,
change `JSON_GOOGLE_BENCHMARK_VERSION` and the archive's `URL_HASH` in `CMakeLists.txt` together.
- Google Benchmark is fetched as a fixed release with a checksum, so that every build uses the same
code and a download cannot change under you. It is updated like any other dependency when a new release comes out:
change `JSON_GOOGLE_BENCHMARK_VERSION` and the archive's `URL_HASH` in `CMakeLists.txt` together. Results are kept
comparable by building both sides of a comparison with the same version, not by keeping that version forever (see
[Comparing two versions](#comparing-two-versions)).
- The benchmarks include `single_include/nlohmann/json.hpp`, so run `make amalgamate` after changing anything in
`include/`.
@@ -110,13 +113,18 @@ Python, unpinned versions work as well. In its output:
- `OVERALL_GEOMEAN` summarizes all benchmarks;
- `-a` shows only the aggregates, not every repetition.
Both builds use the same Google Benchmark, the one `CMakeLists.txt` names, so only the header differs. For the same
reason, do not compare against result files from an earlier checkout: a different Google Benchmark, compiler or
machine state can shift the numbers as much as the change under test. Rerun the baseline instead.
The header you compare with must support everything the benchmarks use. The current benchmarks build against 3.12.0.
Only benchmarks present in both result files are compared, so for older releases, either filter the benchmarks or
build that release's own `tests/benchmarks` against its own header.
## Getting stable numbers
- Build and run both versions on the same machine, one right after the other.
- Build and run both versions on the same machine with the same compiler and Google Benchmark, one right after the
other.
- Keep the machine otherwise idle: no builds, no browser, and a laptop plugged in.
- On Linux, set the CPU frequency governor to `performance`, e.g. `sudo cpupower frequency-set --governor performance`.
Google Benchmark prints a warning when frequency scaling is enabled. Pinning the process to a core