Compare commits

..

5 Commits

Author SHA1 Message Date
Niels Lohmann c6f532d29f Fix flaky ci_nvhpc job: pin nvc++ target to generic baseline (-tp=px)
The ci_nvhpc job crashes intermittently with nvc++ 25.5's llc failing during
instruction selection on std::ldexp<unsigned>, a defect in LLVM that manifests
when nvc++ auto-detects certain CPU targets (particularly -tp znver4) from the
GitHub Actions runner's hardware.

Pinning to -tp=px (generic x86-64 baseline, SSE2-only) removes this variability
and is robust to future nvc++/LLVM updates. Empirical analysis of 19+ recent
ci_nvhpc runs shows the crash occurs specifically with znver4 runners; non-Zen4
runs pass cleanly, and the fix trades a bit of potential codegen performance
for reliability and reproducibility.

Closes: nlohmann/json#136 (todo tracking)
Co-Authored-By: Claude Code <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-09 11:57:17 +02:00
Niels Lohmann fe0299545a 📡 Document cross-basic_json conversion limitation (#3425) (#5249)
When converting objects or strings between different basic_json specializations,
the target's object_t::key_type or string_t must be directly constructible from
the source's corresponding type. If this requirement is not met, the conversion
silently falls back to the array-conversion path, producing incorrect results.

This documents the limitation and provides references to issue #3425, which tracks
this behavior. The comment in unit-alt-string.cpp is clarified to reference the
known limitation with a link to the issue, and suggests the parse() workaround.

Fixes #3425 (documentation; full fix deferred pending type-trait redesign)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-09 06:44:51 +02:00
Niels Lohmann 366f3d26e5 Replace snprintf with a branch-free writer for \uXXXX escapes (#5235)
* Replace snprintf with a branch-free writer for \uXXXX escapes

dump_escaped called std::snprintf(..., "\u%04x", ...) once per escaped
code point in the string serialization hot path. snprintf re-parses
the format string and pulls in locale/printf machinery on every call,
which is far heavier than the fixed 6-/12-byte output warrants. This
is hot for any string containing control characters, and for all
non-ASCII text when ensure_ascii is set.

Replace it with write_u_escape, a small helper that writes the escape
directly into string_buffer via a nibble-to-hex lookup table, mirroring
the existing hand-rolled dump_integer fast path in the same file.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy avoid-c-arrays warning in write_u_escape

Use a const char* rather than a char[] lookup table, matching the
existing hex_bytes helper in the same file.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* ♻️ adjust write_u_escape signature

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-08 20:25:13 +02:00
Niels Lohmann 7c9208bfb3 📡 make documentation more LLM friendly (#5244)
Implement the scoped agent-readiness subset for json.nlohmann.me:
- Add the mkdocs-llmstxt plugin to generate llms.txt from the nav
  (full_output/llms-full.txt deliberately omitted to avoid dumping
  500+ API reference pages into one giant file).
- Add a permissive robots.txt with a Sitemap reference.
- Add a build hook (hooks/copy_markdown_source.py) that copies each
  page's Markdown source into the built site as a `<path>.md` sibling
  of its HTML output, so agents/tools can fetch raw Markdown directly.

sitemap.xml was already emitted by default and needed no change.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-08 20:18:24 +02:00
Niels Lohmann bb60941f0e 👪 fix security findings (#5245)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-08 16:40:58 +02:00
17 changed files with 206 additions and 61 deletions
+12
View File
@@ -4,6 +4,8 @@ updates:
directory: /
schedule:
interval: daily
cooldown:
default-days: 7
groups:
codeql-action:
patterns:
@@ -13,23 +15,33 @@ updates:
directory: /docs/mkdocs
schedule:
interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip
directory: /tools/astyle
schedule:
interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip
directory: /tools/generate_natvis
schedule:
interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip
directory: /tools/serve_header
schedule:
interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip
directory: /cmake/requirements
schedule:
interval: daily
cooldown:
default-days: 7
+1 -1
View File
@@ -49,7 +49,7 @@ jobs:
# SEMGREP_APP_TOKEN is still passed through so registry auth works if a
# token is ever added.
- name: Install Semgrep
run: python3 -m pip install --user semgrep
run: python3 -m pip install --user semgrep==1.168.0
# `semgrep scan --sarif` always exits 0 even with findings; continue-on-error
# is a safety net so the SARIF upload still runs if the scan itself errors.
+6 -1
View File
@@ -720,6 +720,11 @@ add_custom_target(ci_icpx
# to zero and does not honor NaN ordering; -Kieee restores strict IEEE 754 behavior
# (needed for the dtoa/grisu and NaN-comparison code paths).
#
# -tp=px pins the target processor to the generic x86-64 baseline (SSE2-only) to avoid
# a nvc++ 25.5 / LLVM issue: when nvc++ auto-detects -tp from the runner's CPU (e.g. -tp znver4),
# certain attribute combinations trigger an llc instruction-selection crash on std::ldexp<unsigned>.
# Pinning to px removes this variability and is robust to future llc/nvc++ updates.
#
# The following tests are excluded as they trigger known nvc++ 25.5 defects (not
# library bugs); see https://github.com/nlohmann/json for tracking. Only the
# affected language-standard variants are excluded so coverage is otherwise kept:
@@ -733,7 +738,7 @@ add_custom_target(ci_nvhpc
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DCMAKE_C_COMPILER=nvc -DCMAKE_CXX_COMPILER=nvc++
-DCMAKE_CXX_FLAGS=-Kieee
-DCMAKE_CXX_FLAGS="-Kieee;-tp=px"
-DJSON_BuildTests=ON -DJSON_FastTests=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_nvhpc
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_nvhpc
+12 -1
View File
@@ -82,7 +82,13 @@ basic_json(basic_json&& other) noexcept;
4. This is a constructor for existing `basic_json` types. It does not hijack copy/move constructors, since the parameter
has different template arguments than the current ones.
The constructor tries to convert the internal `m_value` of the parameter.
The constructor tries to convert the internal `m_value` of the parameter. Each member value (object, array, string,
etc.) is serialized via the corresponding `to_json()` overload. For objects and strings, the conversion requires
that the *target* `basic_json` type's `object_t::key_type` (or `string_t`) be directly constructible from the
*source* type's corresponding member type via `is_constructible`. If this requirement is not met, the conversion
does not fail to compile; instead, it silently falls back to the array-conversion path, which represents objects
as arrays of `[key, value]` pairs and strings as arrays of character codes. This is a known limitation tracked in
[issue #3425](https://github.com/nlohmann/json/issues/3425).
5. Creates a JSON value of type array or object from the passed initializer list `init`. In case `type_deduction` is
`#!cpp true` (default), the type of the JSON value to be created is deducted from the initializer list `init`
@@ -146,6 +152,11 @@ basic_json(basic_json&& other) noexcept;
- `BasicJsonType` is a `basic_json` type.
- `BasicJsonType` has different template arguments than `basic_json_t`.
**Note:** For cross-`basic_json` conversions to produce correct results, the target `basic_json`'s
`object_t::key_type` and `string_t` must be directly constructible from the source `basic_json`'s
corresponding types. See the description of overload (4) above for details on what happens when
this requirement is not met.
`U`:
: `uncvref_t<CompatibleType>`
@@ -93,6 +93,15 @@ alphabetical order as `std::map` with `std::less` is used by default. Please not
[RFC 8259](https://tools.ietf.org/html/rfc8259), because any order implements the specified "unordered" nature of JSON
objects.
#### Cross-`basic_json` conversion requirements
When converting an object from one `basic_json` specialization to another via the
[converting constructor](basic_json.md#overload-4), the target `object_t`'s `key_type` must be
directly constructible from the source `basic_json`'s `string_t` type (or more generally, from the
source object's key type). If this requirement is not met, the conversion does not fail; instead,
the object is silently converted as an array of key-value pairs, which is incorrect. See
[issue #3425](https://github.com/nlohmann/json/issues/3425) for details and an example.
## Examples
??? example
@@ -45,6 +45,15 @@ This implementation is interoperable as it does compare strings code unit by cod
String values are stored as pointers in a `basic_json` type. That is, for any access to string values, a pointer of type
`string_t*` must be dereferenced.
#### Cross-`basic_json` conversion requirements
When converting a string value from one `basic_json` specialization to another via the
[converting constructor](basic_json.md#overload-4), the target `string_t` must be directly
constructible from the source `basic_json`'s `string_t` type. If this requirement is not met, the
conversion does not fail; instead, the string is silently converted as an array of character codes,
which is incorrect. See [issue #3425](https://github.com/nlohmann/json/issues/3425) for details
and an example.
## Examples
??? example
-18
View File
@@ -66,24 +66,6 @@ which forces the explicit `get` form and can catch unintended conversions at com
floating-point value as an integer truncates it, and narrowing conversions may overflow. See
[number conversion](types/number_handling.md#number-conversion) for details and how to guard against it.
!!! warning "std::optional direct construction from JSON null throws"
Constructing or assigning `std::optional<T>` directly from a JSON value does not correctly produce
`std::nullopt` for a JSON `null`:
```cpp
json j_null;
std::optional<std::string> opt = j_null; // ❌ throws type_error 302
```
This is due to C++ language rules: `std::optional<T>` has its own converting constructor that is chosen over
`basic_json::operator T()` when both are viable. Use `get<std::optional<T>>()` or `get_to()` instead:
```cpp
auto opt = j_null.get<std::optional<std::string>>(); // ✅ std::nullopt
j_null.get_to(opt); // ✅ std::nullopt
```
## Putting values in
The reverse direction works the same way: assigning or constructing a `json` from a C++ value converts it to JSON.
+4
View File
@@ -0,0 +1,4 @@
User-agent: *
Allow: /
Sitemap: https://json.nlohmann.me/sitemap.xml
+25
View File
@@ -0,0 +1,25 @@
"""Copy each documentation page's Markdown source into the built site."""
# Creates a `<path>.md` sibling of each HTML output (for example,
# `features/comments/` becomes `features/comments.md`) so agents and tools can
# fetch the raw Markdown directly instead of parsing rendered HTML.
import os
import shutil
_pages = []
def on_files(files, config):
global _pages
_pages = [f for f in files if f.is_documentation_page()]
return files
def on_post_build(config):
site_dir = config["site_dir"]
for file in _pages:
url = file.url.rstrip("/")
target = os.path.join(site_dir, (url or "index") + ".md")
os.makedirs(os.path.dirname(target), exist_ok=True)
shutil.copyfile(file.abs_src_path, target)
+30
View File
@@ -367,6 +367,9 @@ markdown_extensions:
auto_append:
- ../includes/glossary.md
hooks:
- hooks/copy_markdown_source.py
plugins:
- search:
separator: '[\s\-\.]'
@@ -389,6 +392,33 @@ plugins:
- https://nlohmann.github.io/json/*
- mailto:*
- privacy
- llmstxt:
markdown_description: >
JSON for Modern C++ is a C++11 header-only library implementing a JSON
value type with an STL-like API, JSON Pointer/Patch, CBOR/MessagePack/
BSON/UBJSON/BJData binary format support, and a SAX-style parser interface.
sections:
Home:
- index.md
- home/*.md
Features:
- features/*.md
- features/binary_formats/*.md
- features/element_access/*.md
- features/parsing/*.md
- features/types/*.md
Integration:
- integration/*.md
API Documentation:
- api/*.md
- api/basic_json/*.md
- api/adl_serializer/*.md
- api/byte_container_with_subtype/*.md
- api/json_pointer/*.md
- api/json_sax/*.md
- api/macros/*.md
Community:
- community/*.md
extra_css:
- css/custom.css
+1
View File
@@ -7,5 +7,6 @@ mkdocs-material-extensions==1.3.1 # extensions
mkdocs-minify-plugin==0.8.0 # plugin "minify"
mkdocs-redirects==1.2.3 # plugin "redirects"
mkdocs-htmlproofer-plugin==1.5.0 # plugin "htmlproofer"
mkdocs-llmstxt==0.5.0 # plugin "llmstxt"
PyYAML==6.0.3 # linter
+29 -9
View File
@@ -465,18 +465,12 @@ class serializer
{
if (codepoint <= 0xFFFF)
{
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 7, "\\u%04x",
static_cast<std::uint16_t>(codepoint)));
bytes += 6;
write_u_escape(bytes, static_cast<std::uint16_t>(codepoint));
}
else
{
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 13, "\\u%04x\\u%04x",
static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)),
static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu))));
bytes += 12;
write_u_escape(bytes, static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)));
write_u_escape(bytes, static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu)));
}
}
else
@@ -683,6 +677,32 @@ class serializer
return result;
}
/*!
* @brief write a lowercase "\uXXXX" escape sequence into @a string_buffer
*
* Branch-free replacement for `snprintf(buf, 7, "\\u%04x", codeunit)` in the
* string escaping hot path. It writes exactly six characters ('\\', 'u' and
* four hex digits) at position @a pos of @a string_buffer via a nibble
* lookup table, avoiding the format-string parsing and locale machinery of
* `snprintf`. Advances @a pos by the number of bytes written (6).
*
* @param[in] pos position in @a string_buffer to write at; there must
* be at least 6 bytes of headroom
* @param[in] codeunit 16-bit value to encode
*/
void write_u_escape(std::size_t& pos, std::uint16_t codeunit) noexcept
{
JSON_ASSERT(string_buffer.size() - pos >= 6);
constexpr const char* nibble_to_hex = "0123456789abcdef";
string_buffer[pos + 0] = '\\';
string_buffer[pos + 1] = 'u';
string_buffer[pos + 2] = nibble_to_hex[(codeunit >> 12u) & 0x0Fu];
string_buffer[pos + 3] = nibble_to_hex[(codeunit >> 8u) & 0x0Fu];
string_buffer[pos + 4] = nibble_to_hex[(codeunit >> 4u) & 0x0Fu];
string_buffer[pos + 5] = nibble_to_hex[codeunit & 0x0Fu];
pos += 6;
}
// templates to avoid warnings about useless casts
template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0>
bool is_negative_number(NumberType x)
+29 -9
View File
@@ -19962,18 +19962,12 @@ class serializer
{
if (codepoint <= 0xFFFF)
{
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 7, "\\u%04x",
static_cast<std::uint16_t>(codepoint)));
bytes += 6;
write_u_escape(bytes, static_cast<std::uint16_t>(codepoint));
}
else
{
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 13, "\\u%04x\\u%04x",
static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)),
static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu))));
bytes += 12;
write_u_escape(bytes, static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)));
write_u_escape(bytes, static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu)));
}
}
else
@@ -20180,6 +20174,32 @@ class serializer
return result;
}
/*!
* @brief write a lowercase "\uXXXX" escape sequence into @a string_buffer
*
* Branch-free replacement for `snprintf(buf, 7, "\\u%04x", codeunit)` in the
* string escaping hot path. It writes exactly six characters ('\\', 'u' and
* four hex digits) at position @a pos of @a string_buffer via a nibble
* lookup table, avoiding the format-string parsing and locale machinery of
* `snprintf`. Advances @a pos by the number of bytes written (6).
*
* @param[in] pos position in @a string_buffer to write at; there must
* be at least 6 bytes of headroom
* @param[in] codeunit 16-bit value to encode
*/
void write_u_escape(std::size_t& pos, std::uint16_t codeunit) noexcept
{
JSON_ASSERT(string_buffer.size() - pos >= 6);
constexpr const char* nibble_to_hex = "0123456789abcdef";
string_buffer[pos + 0] = '\\';
string_buffer[pos + 1] = 'u';
string_buffer[pos + 2] = nibble_to_hex[(codeunit >> 12u) & 0x0Fu];
string_buffer[pos + 3] = nibble_to_hex[(codeunit >> 8u) & 0x0Fu];
string_buffer[pos + 4] = nibble_to_hex[(codeunit >> 4u) & 0x0Fu];
string_buffer[pos + 5] = nibble_to_hex[codeunit & 0x0Fu];
pos += 6;
}
// templates to avoid warnings about useless casts
template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0>
bool is_negative_number(NumberType x)
+3
View File
@@ -232,6 +232,9 @@ TEST_CASE("algorithms")
// only the first four elements are expected to be sorted, the rest are
// unspecified by the standard
const json expected({nullptr, false, true, 3});
// std::equal below only bounds-checks the first range; assert the
// second range is at least as long to rule out an over-read (CWE-126)
CHECK(std::distance(begin(expected), end(expected)) >= 4);
CHECK(std::equal(j.begin(), j.begin() + 4, begin(expected)));
}
}
+6 -8
View File
@@ -322,14 +322,12 @@ TEST_CASE("alternative string type")
SECTION("JSON pointer")
{
// conversion from json to alt_json fails to compile (see #3425);
// attempted fix(*) produces: [[['b','a','r'],['b','a','z']]] (with each char being an integer)
// (*) disable implicit conversion for json_refs of any basic_json type
// alt_json j = R"(
// {
// "foo": ["bar", "baz"]
// }
// )"_json;
// Direct conversion from a json literal to alt_json is not supported due to issue #3425:
// alt_json's string_t (alt_string) is not directly constructible from std::string, so the
// cross-basic_json conversion falls back to the array-conversion path, incorrectly representing
// objects as arrays of [key, value] pairs and strings as arrays of character codes.
// See https://github.com/nlohmann/json/issues/3425 for details.
// Workaround: use alt_json::parse() instead of implicit conversion.
auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})");
CHECK(j.at(alt_json::json_pointer("/foo/0")) == j["foo"][0]);
+26
View File
@@ -168,6 +168,32 @@ TEST_CASE("convenience functions")
CHECK_THROWS_WITH_AS(check_escaped("\xC2"), "[json.exception.type_error.316] incomplete UTF-8 string; last byte: 0xC2", json::type_error&);
}
SECTION("string escape with ensure_ascii")
{
// control characters are escaped regardless of ensure_ascii
check_escaped("\x01", "\\u0001", true);
check_escaped("\x1f", "\\u001f", true);
// non-ASCII code points in the Basic Multilingual Plane are emitted as
// a single lowercase \uXXXX escape (exercises every nibble position)
check_escaped("\xC2\x80", "\\u0080", true); // U+0080
check_escaped("\xC3\xBF", "\\u00ff", true); // U+00FF (ÿ)
check_escaped("\xDF\xBF", "\\u07ff", true); // U+07FF
check_escaped("\xE4\xBD\xA0", "\\u4f60", true); // U+4F60 (你)
check_escaped("\xEA\xAF\x8D", "\\uabcd", true); // U+ABCD
check_escaped("\xEF\xBF\xBD", "\\ufffd", true); // U+FFFD (replacement char, all-f nibbles)
// code points outside the BMP are emitted as a UTF-16 surrogate pair
// of two lowercase \uXXXX escapes
check_escaped("\xF0\x90\x80\x80", "\\ud800\\udc00", true); // U+10000 (lowest astral)
check_escaped("\xF0\x9F\x98\x80", "\\ud83d\\ude00", true); // U+1F600 (😀)
check_escaped("\xF4\x8F\xBF\xBF", "\\udbff\\udfff", true); // U+10FFFF (highest code point)
// with ensure_ascii disabled, non-ASCII input is passed through verbatim
check_escaped("\xE4\xBD\xA0", "\xE4\xBD\xA0", false);
check_escaped("\xF0\x9F\x98\x80", "\xF0\x9F\x98\x80", false);
}
SECTION("string concat")
{
using nlohmann::detail::concat;
+4 -14
View File
@@ -1761,27 +1761,16 @@ TEST_CASE("std::filesystem::path")
}
#endif
#if !JSON_USE_IMPLICIT_CONVERSIONS
TEST_CASE("std::optional")
{
SECTION("null")
{
const json j_null;
const std::optional<std::string> opt_null;
json j_null;
std::optional<std::string> opt_null;
CHECK(json(opt_null) == j_null);
CHECK(j_null.get<std::optional<std::string>>() == std::nullopt);
// Constructing std::optional<T> directly from JSON null throws because
// std::optional's own converting constructor is chosen over basic_json's
// operator T(). This is a language-level limitation (std::optional<T> is
// constructible from T, and T is constructible from basic_json via the
// operator); there is no SFINAE path that distinguishes "call from inside
// std::optional's constructor" from "direct call". Use get<std::optional<T>>()
// or get_to() instead for correct null handling. See #4864 and #5246.
CHECK_THROWS_WITH_AS(std::optional<std::string>(j_null),
"[json.exception.type_error.302] type must be string, but is null", json::type_error&);
CHECK_THROWS_WITH_AS(std::optional<int>(j_null),
"[json.exception.type_error.302] type must be number, but is null", json::type_error&);
}
SECTION("string")
@@ -1830,6 +1819,7 @@ TEST_CASE("std::optional")
}
}
#endif
#endif
#ifdef JSON_HAS_CPP_17
#undef JSON_HAS_CPP_17