Compare commits

..
Author SHA1 Message Date
Niels Lohmann 023ab67ecc Document options to reduce compile times
Add an integration page that collects the ways to reduce compile times
with measurements: json_fwd.hpp in headers, JSON_NO_AUTOMATIC_UDLS,
explicit instantiation with extern template, modules, and precompiled
headers, and notes that JSON_NO_IO and JSON_USE_GLOBAL_UDLS have no
measurable effect.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:08:22 +02:00
13 changed files with 176 additions and 142 deletions
@@ -62,6 +62,7 @@ By default, `#!cpp JSON_NO_AUTOMATIC_UDLS` is not defined, and `<nlohmann/json.h
- [`operator""_json`](../operator_literal_json.md)
- [`operator""_json_pointer`](../operator_literal_json_pointer.md)
- [`JSON_USE_GLOBAL_UDLS`](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [Compile times](../../integration/compile_times.md) - options to reduce compile times
## Version history
-6
View File
@@ -18,10 +18,6 @@ Deserializes an input stream to a JSON value.
the stream `i`
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes in `j`.
## Exceptions
- Throws [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101) in case of an unexpected token, or if
@@ -129,5 +125,3 @@ being read.
the stream; planned to become the default in version 4.0.0.
- Fixed a null pointer dereference for an `std::istream` without a stream buffer (now throws `parse_error.101`), and a
crash (`std::terminate`) when `i` has `eofbit` in its exception mask, in version 3.13.0.
- Changed to the strong exception safety guarantee in version 3.13.0: `j` is no longer left with a partially parsed
value if parsing throws.
@@ -389,7 +389,6 @@ using array_t = ArrayType<basic_json, AllocatorType<basic_json>>;
| Functionality | Additional requirement |
|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`diff`](../../api/basic_json/diff.md), [`items`](../../api/basic_json/items.md), [`std::hash`](../../api/basic_json/std_hash.md) | conversion of a `#!cpp std::size_t` to `StringType`: either assignability from the result of `#!cpp std::to_string`, or an ADL overload `#!cpp void int_to_string(StringType&, std::size_t)` |
| [`operator/(std::size_t)`](../../api/json_pointer/operator_slash.md) | the same conversion of a `#!cpp std::size_t` to `StringType` as `diff`, `items`, and `std::hash` above |
| [`std::hash<basic_json>`](../../api/basic_json/std_hash.md) | additionally a specialization of `#!cpp std::hash<StringType>` |
| [`to_bson`](../../api/basic_json/to_bson.md) | `find(value_type)` and `npos` |
| [`parse`](../../api/basic_json/parse.md) from a `string_t` | the input adapters must accept it; otherwise pass a character range |
@@ -0,0 +1,149 @@
# Compile times
The library is header-only and makes heavy use of templates, so every translation unit that includes
`<nlohmann/json.hpp>` pays for parsing the header and instantiating what it uses. This page lists the options to reduce
that cost, ordered by how much they typically save.
!!! info "Measurements"
The numbers below are medians of nine runs compiling a single translation unit with `-std=c++17 -c` against the
single-header version, with Apple clang and GCC 16 on macOS (Apple silicon). They show the order of magnitude to
expect; measure your own code before and after a change.
## Include `json_fwd.hpp` in headers
Header files that only need to *name* the `json` type — for function declarations, members held by pointer or
reference, or friend declarations — can include `<nlohmann/json_fwd.hpp>` instead of `<nlohmann/json.hpp>`. It only
forward-declares `basic_json`, `json`, `ordered_json`, `json_pointer`, and `adl_serializer`. The translation units that
actually use the values then include `<nlohmann/json.hpp>`.
```cpp title="person.hpp"
#pragma once
#include <nlohmann/json_fwd.hpp>
struct person;
void to_json(nlohmann::json& j, const person& p);
void from_json(const nlohmann::json& j, person& p);
```
```cpp title="person.cpp"
#include "person.hpp"
#include <nlohmann/json.hpp>
void to_json(nlohmann::json& j, const person& p) { /* ... */ }
void from_json(const nlohmann::json& j, person& p) { /* ... */ }
```
| Compiler | `json.hpp` (`-O0`) | `json_fwd.hpp` (`-O0`) | Change |
|-------------|-------------------:|-----------------------:|-------:|
| Apple clang | 704 ms | 329 ms | −53% |
| GCC 16 | 779 ms | 242 ms | −69% |
This is the most effective option, because it avoids the full header in every translation unit that includes
*your* headers.
## Opt out of the automatic user-defined string literals
The user-defined string literals [`operator""_json`](../api/operator_literal_json.md) and
[`operator""_json_pointer`](../api/operator_literal_json_pointer.md) are ordinary inline functions whose bodies call the
parser. As `<nlohmann/json.hpp>` includes them by default, every translation unit instantiates the parser, even if it
never parses anything itself.
Define [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) for the whole project and include
`<nlohmann/json_literals.hpp>` only in the files that use the literals:
```cmake
target_compile_definitions(my_target PRIVATE JSON_NO_AUTOMATIC_UDLS)
```
```cpp
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp> // only where "..."_json is used
```
The saving applies to translation units that do not parse JSON, for example ones that define types and their
conversions or only pass `json` values around:
| Compiler | Translation unit | Default (`-O0` / `-O2`) | `JSON_NO_AUTOMATIC_UDLS` (`-O0` / `-O2`) | Change |
|-------------|------------------|------------------------:|-----------------------------------------:|------------:|
| Apple clang | model | 776 ms / 846 ms | 629 ms / 692 ms | −19% / −18% |
| GCC 16 | model | 1022 ms / 1120 ms | 882 ms / 965 ms | −14% / −14% |
| Apple clang | parsing | 992 ms / 1815 ms | 1006 ms / 1823 ms | +1% / 0% |
| GCC 16 | parsing | 2018 ms / 3420 ms | 1990 ms / 3454 ms | −1% / +1% |
Translation units that include only the header save up to a third. Translation units that parse anyway instantiate
the parser regardless and see no difference.
## Instantiate `basic_json` once
Each translation unit instantiates the member functions of `nlohmann::json` it uses. An explicit instantiation
declaration tells the compiler that the non-template members are instantiated elsewhere, so it can skip them:
```cpp title="json_instance.hpp"
#pragma once
#include <nlohmann/json.hpp>
extern template class nlohmann::basic_json<>;
```
```cpp title="json_instance.cpp"
#include "json_instance.hpp"
template class nlohmann::basic_json<>;
```
Include `json_instance.hpp` instead of `<nlohmann/json.hpp>` and compile and link `json_instance.cpp` once.
| Compiler | Translation unit | Default (`-O0` / `-O2`) | `extern template` (`-O0` / `-O2`) | Change |
|-------------|---------------------|------------------------:|----------------------------------:|------------:|
| Apple clang | parsing | 992 ms / 1815 ms | 953 ms / 1625 ms | −4% / −10% |
| GCC 16 | parsing | 2018 ms / 3420 ms | 1522 ms / 2728 ms | −25% / −20% |
| Apple clang | `json_instance.cpp` | — | 2166 ms / 4660 ms | — |
| GCC 16 | `json_instance.cpp` | — | 5085 ms / 10616 ms | — |
Notes:
- The saving grows with the number of translation units that use `json`, while the instantiation translation unit is
compiled only once (and is rarely recompiled, as it does not depend on your code).
- Member function templates (such as `get<T>()`, `parse(InputType&&)`, or `value(key, default)`) are not covered by
the explicit instantiation and are still instantiated where they are used.
- The declaration covers exactly `nlohmann::json`. Add the same lines for `nlohmann::ordered_json`
(`nlohmann::basic_json<nlohmann::ordered_map>`) or your own `basic_json` specializations if you use them.
## Use C++20 modules
With a toolchain that supports named modules, `import nlohmann.json;` compiles the library once into a module and
avoids parsing the header in every translation unit. See [Modules](../features/modules.md) for requirements and known
issues. Module support is experimental and currently depends heavily on the compiler version.
## Use precompiled headers
Build systems can precompile `<nlohmann/json.hpp>` together with other stable headers, for example with CMake's
[`target_precompile_headers`](https://cmake.org/cmake/help/latest/command/target_precompile_headers.html):
```cmake
target_precompile_headers(my_target PRIVATE <nlohmann/json.hpp>)
```
This removes the cost of parsing the header, but not of instantiating templates in each translation unit, so it
combines well with the options above.
## Options without effect on compile times
Some configuration macros change what the library declares, but do not measurably change compile times:
| Macro | Apple clang, model (`-O0` / `-O2`) | GCC 16, model (`-O0` / `-O2`) |
|------------------------------------------------------------------------|-----------------------------------:|------------------------------:|
| default | 776 ms / 846 ms | 1022 ms / 1120 ms |
| [`JSON_NO_IO`](../api/macros/json_no_io.md) | 764 ms / 836 ms | 1022 ms / 1117 ms |
| [`JSON_USE_GLOBAL_UDLS`](../api/macros/json_use_global_udls.md)`=0` | 763 ms / 852 ms | 1019 ms / 1106 ms |
`JSON_USE_GLOBAL_UDLS` only controls *where* the literals are declared; to avoid their cost, use
`JSON_NO_AUTOMATIC_UDLS` instead.
## See also
- [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) - do not include the user-defined string
literals automatically
- [Modules](../features/modules.md) - C++20 module support
- [Header only](index.md) - including the library
+1 -1
View File
@@ -15,7 +15,7 @@ Clang).
You can further use file
[`single_include/nlohmann/json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_fwd.hpp)
for forward declarations, and file
for forward declarations (see [Compile times](compile_times.md)), and file
[`single_include/nlohmann/json_literals.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_literals.hpp)
for the user-defined string literals if you define
[`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md).
+1
View File
@@ -106,6 +106,7 @@ nav:
- integration/cmake.md
- integration/package_managers.md
- integration/pkg-config.md
- integration/compile_times.md
- API Documentation:
- basic_json:
- 'Overview': api/basic_json/index.md
+11 -22
View File
@@ -26,7 +26,6 @@
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_escape.hpp>
#include <nlohmann/detail/string_utils.hpp>
#include <nlohmann/detail/value_t.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -117,7 +116,7 @@ class json_pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/
json_pointer& operator/=(std::size_t array_idx)
{
return *this /= detail::to_string<string_t>(array_idx);
return *this /= std::to_string(array_idx);
}
/// @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer
@@ -679,29 +678,19 @@ class json_pointer
return nullptr;
}
// tokens that array_index() rejects with parse_error.106/109
// are passed on to it; all other tokens that it would reject
// with out_of_range.404/410 are detected here, so that this
// also works without exceptions
if (JSON_HEDLEY_UNLIKELY(reference_token.size() > 1 && !(reference_token[0] >= '1' && reference_token[0] <= '9')))
// may throw parse_error.106/109 for a malformed index; an
// index that is syntactically valid but cannot be
// represented (out_of_range.404/410) is treated like an
// out-of-range index below
typename BasicJsonType::size_type idx{};
JSON_TRY
{
static_cast<void>(array_index<BasicJsonType>(reference_token)); // throws parse_error.106/109
idx = array_index<BasicJsonType>(reference_token);
}
if (JSON_HEDLEY_UNLIKELY(reference_token.empty() || !std::all_of(reference_token.begin(), reference_token.end(), [](const char c)
{
return c >= '0' && c <= '9';
})))
JSON_INTERNAL_CATCH (detail::out_of_range&)
{
return nullptr;
}
errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.data(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE || magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{
return nullptr;
}
const auto idx = static_cast<typename BasicJsonType::size_type>(magnitude);
if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
{
@@ -763,7 +752,7 @@ class json_pointer
// would throw out_of_range.404 -- contains() must not throw (see #5395)
return false;
}
if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !('0' <= reference_token[0] && reference_token[0] <= '9')))
if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !("0" <= reference_token && reference_token <= "9")))
{
// invalid char
return false;
@@ -791,7 +780,7 @@ class json_pointer
// not throw (see #5395), so such a reference token is treated as "not found"
errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.data(), &p_end, 10); // NOLINT(runtime/int)
const unsigned long long magnitude = std::strtoull(reference_token.c_str(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX
|| magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{
+1 -4
View File
@@ -5092,10 +5092,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/operator_gtgt/
friend std::istream& operator>>(std::istream& i, basic_json& j)
{
// parse into a temporary so that j is left unchanged if parsing fails
basic_json result;
parser(detail::input_adapter(i)).parse(false, result);
j = std::move(result);
parser(detail::input_adapter(i)).parse(false, j);
return i;
}
#endif // JSON_NO_IO
+12 -27
View File
@@ -19636,8 +19636,6 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/string_escape.hpp>
// #include <nlohmann/detail/string_utils.hpp>
// #include <nlohmann/detail/value_t.hpp>
@@ -19729,7 +19727,7 @@ class json_pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/
json_pointer& operator/=(std::size_t array_idx)
{
return *this /= detail::to_string<string_t>(array_idx);
return *this /= std::to_string(array_idx);
}
/// @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer
@@ -20291,29 +20289,19 @@ class json_pointer
return nullptr;
}
// tokens that array_index() rejects with parse_error.106/109
// are passed on to it; all other tokens that it would reject
// with out_of_range.404/410 are detected here, so that this
// also works without exceptions
if (JSON_HEDLEY_UNLIKELY(reference_token.size() > 1 && !(reference_token[0] >= '1' && reference_token[0] <= '9')))
// may throw parse_error.106/109 for a malformed index; an
// index that is syntactically valid but cannot be
// represented (out_of_range.404/410) is treated like an
// out-of-range index below
typename BasicJsonType::size_type idx{};
JSON_TRY
{
static_cast<void>(array_index<BasicJsonType>(reference_token)); // throws parse_error.106/109
idx = array_index<BasicJsonType>(reference_token);
}
if (JSON_HEDLEY_UNLIKELY(reference_token.empty() || !std::all_of(reference_token.begin(), reference_token.end(), [](const char c)
{
return c >= '0' && c <= '9';
})))
JSON_INTERNAL_CATCH (detail::out_of_range&)
{
return nullptr;
}
errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.data(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE || magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{
return nullptr;
}
const auto idx = static_cast<typename BasicJsonType::size_type>(magnitude);
if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
{
@@ -20375,7 +20363,7 @@ class json_pointer
// would throw out_of_range.404 -- contains() must not throw (see #5395)
return false;
}
if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !('0' <= reference_token[0] && reference_token[0] <= '9')))
if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !("0" <= reference_token && reference_token <= "9")))
{
// invalid char
return false;
@@ -20403,7 +20391,7 @@ class json_pointer
// not throw (see #5395), so such a reference token is treated as "not found"
errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.data(), &p_end, 10); // NOLINT(runtime/int)
const unsigned long long magnitude = std::strtoull(reference_token.c_str(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX
|| magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{
@@ -32029,10 +32017,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/operator_gtgt/
friend std::istream& operator>>(std::istream& i, basic_json& j)
{
// parse into a temporary so that j is left unchanged if parsing fails
basic_json result;
parser(detail::input_adapter(i)).parse(false, result);
j = std::move(result);
parser(detail::input_adapter(i)).parse(false, j);
return i;
}
#endif // JSON_NO_IO
-35
View File
@@ -352,41 +352,6 @@ TEST_CASE("alternative string type")
CHECK(j2.flatten().unflatten() == j2);
}
SECTION("contains(json_pointer)")
{
// contains(json_pointer) must compile and work with a string_t that has
// no c_str() and no comparison with const char* (see #5666)
auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})");
// present: object key and array indices
CHECK(j.contains(alt_json::json_pointer("/foo")));
CHECK(j.contains(alt_json::json_pointer("/foo/0")));
CHECK(j.contains(alt_json::json_pointer("/foo/1")));
// missing: absent object key and out-of-range array index
CHECK_FALSE(j.contains(alt_json::json_pointer("/bar")));
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/2")));
// "-" always fails the range check
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/-")));
// an array index must not have a leading zero
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/01")));
// a reference token that is not a number
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/bar")));
}
SECTION("operator/(std::size_t)")
{
// json_pointer::operator/=(std::size_t) must compile without string_t
// being constructible from std::string (see #5666)
auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})");
CHECK(j.at(alt_json::json_pointer("/foo") / std::size_t(0)) == j["foo"][0]);
CHECK(j.at(alt_json::json_pointer("/foo") / std::size_t(1)) == j["foo"][1]);
}
SECTION("patch")
{
alt_json const patch1 = alt_json::parse(R"([{ "op": "add", "path": "/a/b", "value": [ "foo", "bar" ] }])");
-16
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <map>
#include <unordered_map>
#include <sstream>
TEST_CASE("Better diagnostics")
{
@@ -493,21 +492,6 @@ TEST_CASE("Regression tests for extended diagnostics")
CHECK(copy == j);
}
}
SECTION("Regression test for issue #5652 - operator>> leaves a partial value in its target on a parse error")
{
json j = "old value";
std::istringstream is("[1, x");
CHECK_THROWS_WITH_AS(is >> j, "[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - invalid literal; last read: '1, x'", json::parse_error);
// j must be left unchanged, as json::parse() guarantees for its result
CHECK(j == "old value");
// copying j must not trigger assert_invariant(): a failed parse must
// not leave array/object elements without a parent pointer
json const copy = j; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(copy == j);
}
}
TEST_CASE("Better diagnostics past the descent bound of update() and merge_patch()")
-15
View File
@@ -47,21 +47,6 @@ TEST_CASE("Tests with disabled exceptions")
delete sax_no_exception::error_string; // NOLINT(cppcoreguidelines-owning-memory)
}
SECTION("issue #5672 - value(json_pointer, default) must not abort for array tokens that are not a valid index")
{
const json j = {1, 2, 3};
// a syntactically valid index that is out of range for this array
CHECK(j.value("/7"_json_pointer, 42) == 42);
// a reference token that is not a number at all
CHECK(j.value("/1a"_json_pointer, 42) == 42);
// the empty reference token (JSON pointer "/")
CHECK(j.value("/"_json_pointer, 42) == 42);
// an index whose magnitude does not fit into size_type
CHECK(j.value("/99999999999999999999999"_json_pointer, 42) == 42);
CHECK(j.value("/18446744073709551615"_json_pointer, 42) == 42);
}
SECTION("growing an ordered_json object")
{
auto j = nlohmann::ordered_json::object();
-15
View File
@@ -516,21 +516,6 @@ TEST_CASE_TEMPLATE("element access 2", Json, nlohmann::json, nlohmann::ordered_j
CHECK(j_array.value("/-"_json_pointer, 42) == 42);
CHECK(j_array_const.value("/-"_json_pointer, 42) == 42);
// Test an index with a non-digit after a valid leading digit; this is
// out_of_range (not parse_error) and must not throw (see #5672)
CHECK(j_array.value("/1a"_json_pointer, 42) == 42);
CHECK(j_array_const.value("/1a"_json_pointer, 42) == 42);
// Test the empty reference token (JSON pointer "/"); see #5672
CHECK(j_array.value("/"_json_pointer, 42) == 42);
CHECK(j_array_const.value("/"_json_pointer, 42) == 42);
// Test an index whose magnitude does not fit into size_type (see #5672)
CHECK(j_array.value("/99999999999999999999999"_json_pointer, 42) == 42);
CHECK(j_array_const.value("/99999999999999999999999"_json_pointer, 42) == 42);
CHECK(j_array.value("/18446744073709551615"_json_pointer, 42) == 42);
CHECK(j_array_const.value("/18446744073709551615"_json_pointer, 42) == 42);
#if !defined(JSON_NOEXCEPTION)
// Test malformed index (non-numeric) throws parse_error
CHECK_THROWS_WITH_AS(j_array.value("/foo"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'foo' is not a number", typename Json::parse_error&);