amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-09-29 11:10:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Pin Google Benchmark to release 1.9.5

Browse Source 
The benchmarks fetched Google Benchmark's main branch, so two builds on
different days could measure with different library code, and CMake 3.30
and later warn that the single-argument FetchContent_Populate() is
deprecated. Fetch the 1.9.5 release archive, verified by its SHA-256,
with FetchContent_MakeAvailable() instead. That needs CMake 3.14; Google
Benchmark itself already needed 3.13.

Its -Werror is switched off, so a newer compiler's new warnings cannot
break the pinned release, and its install rules are no longer added.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-09-24 09:48:18 +02:00
parent 67ae3fd85b
commit b87a618151
2 changed files with 23 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/benchmarks/CMakeLists.txt
+15 -14
View File
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.11...3.14) cmake_minimum_required(VERSION 3.14)
project(JSON_Benchmarks LANGUAGES CXX) project(JSON_Benchmarks LANGUAGES CXX)
# set compiler flags # set compiler flags
@@ -6,21 +6,22 @@ if((CMAKE_CXX_COMPILER_ID MATCHES GNU) OR (CMAKE_CXX_COMPILER_ID MATCHES Clang))
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -flto -DNDEBUG -O3") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -flto -DNDEBUG -O3")
endif() endif()
# configure Google Benchmarks # configure Google Benchmark; a fixed release, so that results stay comparable
set(JSON_GOOGLE_BENCHMARK_VERSION 1.9.5)
include(FetchContent) include(FetchContent)
FetchContent_Declare(
benchmark
GIT_REPOSITORY https://github.com/google/benchmark.git
GIT_TAG origin/main
GIT_SHALLOW TRUE
)
FetchContent_GetProperties(benchmark) # only the library is needed; -Werror would break the pinned release as soon as
if(NOT benchmark_POPULATED) # a newer compiler adds a warning
FetchContent_Populate(benchmark) set(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_TESTING OFF CACHE INTERNAL "" FORCE) set(BENCHMARK_ENABLE_INSTALL OFF CACHE BOOL "" FORCE)
add_subdirectory(${benchmark_SOURCE_DIR} ${benchmark_BINARY_DIR}) set(BENCHMARK_ENABLE_WERROR OFF CACHE BOOL "" FORCE)
endif()
FetchContent_Declare(benchmark
URL https://github.com/google/benchmark/archive/refs/tags/v${JSON_GOOGLE_BENCHMARK_VERSION}.tar.gz
URL_HASH SHA256=9631341c82bac4a288bef951f8b26b41f69021794184ece969f8473977eaa340
DOWNLOAD_EXTRACT_TIMESTAMP TRUE
)
FetchContent_MakeAvailable(benchmark)
# download test data # download test data
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../cmake ${CMAKE_MODULE_PATH}) set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../cmake ${CMAKE_MODULE_PATH})
tests/benchmarks/README.md
+8 -6
View File
@@ -23,10 +23,12 @@ The input files are those of [nativejson-benchmark](https://github.com/miloyip/n
## Requirements ## Requirements
- CMake 3.11 or later, a C++11 compiler, and Ninja for the `make` target. - CMake 3.14 or later, a C++11 compiler, and Ninja for the `make` target.
- Network access on the first configure: CMake fetches Google Benchmark and downloads the - Network access on the first configure: CMake downloads Google Benchmark and the
[test data](https://github.com/nlohmann/json_test_data) into the build directory. To reuse a download, pass [test data](https://github.com/nlohmann/json_test_data) into the build directory. To reuse a download of the test
`-DJSON_TestDataDirectory=<build directory>/test_files`. data, pass `-DJSON_TestDataDirectory=<build directory>/test_files`.
- Google Benchmark is pinned to a release (1.9.5), so that results from different days stay comparable. To update it,
change `JSON_GOOGLE_BENCHMARK_VERSION` and the archive's `URL_HASH` in `CMakeLists.txt` together.
- The benchmarks include `single_include/nlohmann/json.hpp`, so run `make amalgamate` after changing anything in - The benchmarks include `single_include/nlohmann/json.hpp`, so run `make amalgamate` after changing anything in
`include/`. `include/`.
@@ -99,8 +101,8 @@ build-venv/bin/pip install numpy scipy
build-venv/bin/python build-current/_deps/benchmark-src/tools/compare.py -a benchmarks build-baseline/results.json build-current/results.json build-venv/bin/python build-current/_deps/benchmark-src/tools/compare.py -a benchmarks build-baseline/results.json build-current/results.json
``` ```
The tool's own `tools/requirements.txt` pins the newest NumPy and SciPy, which may need a newer Python than yours; The tool's own `tools/requirements.txt` pins NumPy and SciPy versions that need Python 3.11 or later; with an older
unpinned versions work as well. In its output: Python, unpinned versions work as well. In its output:
- the `Time` and `CPU` columns are relative changes: `-0.35` means 35% faster, `+0.10` means 10% slower; - the `Time` and `CPU` columns are relative changes: `-0.35` means 35% faster, `+0.10` means 10% slower;
- `_pvalue` lines report a Mann-Whitney U test of whether the two versions differ. It needs at least 9 - `_pvalue` lines report a Mann-Whitney U test of whether the two versions differ. It needs at least 9