Compare commits

...

6 Commits

Author SHA1 Message Date
Niels Lohmann fdba4991ab 🚶 fix format
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-09 23:09:49 +02:00
Niels Lohmann 68c87ad9de Fix std::hash contract violation for numeric types
Fixes #5256: json(42) == json(42u) is true, but their hashes differed,
violating the std::hash contract. This also applied to float comparisons:
json(42) == json(42.0) is true, but they hashed differently.

Solution: Normalize numeric type hashing to ensure equal values hash equal.
- Signed/unsigned integers: normalize unsigned to signed via static_cast,
  matching the existing operator== behavior (lines 3711-3717 in json.hpp)
- Integer/float bridging: for values exactly representable as the float type,
  hash via the float form to collide correctly with float values
- All numeric types share a single type tag to ensure hash collision

The fix is rigorous for the reported issue (int/uint, any magnitude) with zero
gaps. For int/float comparisons, there's a documented edge case at extreme
magnitudes due to float precision limits, mirroring limitations already
present in operator==.

Changes:
- include/nlohmann/detail/hash.hpp: core fix with new
  is_exactly_representable_as_float helper
- tests/src/unit-hash.cpp: update expected hash counts (21 -> 19 distinct),
  add explicit std::hash contract verification
- docs/mkdocs/docs/api/basic_json/std_hash.md: update description
- docs/mkdocs/docs/examples/std_hash.cpp/.output: show the fix in action
- single_include/nlohmann/json.hpp: regenerated via amalgamate

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-09 21:12:31 +02:00
Niels Lohmann fe0299545a 📡 Document cross-basic_json conversion limitation (#3425) (#5249)
When converting objects or strings between different basic_json specializations,
the target's object_t::key_type or string_t must be directly constructible from
the source's corresponding type. If this requirement is not met, the conversion
silently falls back to the array-conversion path, producing incorrect results.

This documents the limitation and provides references to issue #3425, which tracks
this behavior. The comment in unit-alt-string.cpp is clarified to reference the
known limitation with a link to the issue, and suggests the parse() workaround.

Fixes #3425 (documentation; full fix deferred pending type-trait redesign)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-09 06:44:51 +02:00
Niels Lohmann 366f3d26e5 Replace snprintf with a branch-free writer for \uXXXX escapes (#5235)
* Replace snprintf with a branch-free writer for \uXXXX escapes

dump_escaped called std::snprintf(..., "\u%04x", ...) once per escaped
code point in the string serialization hot path. snprintf re-parses
the format string and pulls in locale/printf machinery on every call,
which is far heavier than the fixed 6-/12-byte output warrants. This
is hot for any string containing control characters, and for all
non-ASCII text when ensure_ascii is set.

Replace it with write_u_escape, a small helper that writes the escape
directly into string_buffer via a nibble-to-hex lookup table, mirroring
the existing hand-rolled dump_integer fast path in the same file.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy avoid-c-arrays warning in write_u_escape

Use a const char* rather than a char[] lookup table, matching the
existing hex_bytes helper in the same file.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* ♻️ adjust write_u_escape signature

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-08 20:25:13 +02:00
Niels Lohmann 7c9208bfb3 📡 make documentation more LLM friendly (#5244)
Implement the scoped agent-readiness subset for json.nlohmann.me:
- Add the mkdocs-llmstxt plugin to generate llms.txt from the nav
  (full_output/llms-full.txt deliberately omitted to avoid dumping
  500+ API reference pages into one giant file).
- Add a permissive robots.txt with a Sitemap reference.
- Add a build hook (hooks/copy_markdown_source.py) that copies each
  page's Markdown source into the built site as a `<path>.md` sibling
  of its HTML output, so agents/tools can fetch raw Markdown directly.

sitemap.xml was already emitted by default and needed no change.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-08 20:18:24 +02:00
Niels Lohmann bb60941f0e 👪 fix security findings (#5245)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-08 16:40:58 +02:00
19 changed files with 414 additions and 53 deletions
+12
View File
@@ -4,6 +4,8 @@ updates:
directory: / directory: /
schedule: schedule:
interval: daily interval: daily
cooldown:
default-days: 7
groups: groups:
codeql-action: codeql-action:
patterns: patterns:
@@ -13,23 +15,33 @@ updates:
directory: /docs/mkdocs directory: /docs/mkdocs
schedule: schedule:
interval: daily interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip - package-ecosystem: pip
directory: /tools/astyle directory: /tools/astyle
schedule: schedule:
interval: daily interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip - package-ecosystem: pip
directory: /tools/generate_natvis directory: /tools/generate_natvis
schedule: schedule:
interval: daily interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip - package-ecosystem: pip
directory: /tools/serve_header directory: /tools/serve_header
schedule: schedule:
interval: daily interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip - package-ecosystem: pip
directory: /cmake/requirements directory: /cmake/requirements
schedule: schedule:
interval: daily interval: daily
cooldown:
default-days: 7
+1 -1
View File
@@ -49,7 +49,7 @@ jobs:
# SEMGREP_APP_TOKEN is still passed through so registry auth works if a # SEMGREP_APP_TOKEN is still passed through so registry auth works if a
# token is ever added. # token is ever added.
- name: Install Semgrep - name: Install Semgrep
run: python3 -m pip install --user semgrep run: python3 -m pip install --user semgrep==1.168.0
# `semgrep scan --sarif` always exits 0 even with findings; continue-on-error # `semgrep scan --sarif` always exits 0 even with findings; continue-on-error
# is a safety net so the SARIF upload still runs if the scan itself errors. # is a safety net so the SARIF upload still runs if the scan itself errors.
+12 -1
View File
@@ -82,7 +82,13 @@ basic_json(basic_json&& other) noexcept;
4. This is a constructor for existing `basic_json` types. It does not hijack copy/move constructors, since the parameter 4. This is a constructor for existing `basic_json` types. It does not hijack copy/move constructors, since the parameter
has different template arguments than the current ones. has different template arguments than the current ones.
The constructor tries to convert the internal `m_value` of the parameter. The constructor tries to convert the internal `m_value` of the parameter. Each member value (object, array, string,
etc.) is serialized via the corresponding `to_json()` overload. For objects and strings, the conversion requires
that the *target* `basic_json` type's `object_t::key_type` (or `string_t`) be directly constructible from the
*source* type's corresponding member type via `is_constructible`. If this requirement is not met, the conversion
does not fail to compile; instead, it silently falls back to the array-conversion path, which represents objects
as arrays of `[key, value]` pairs and strings as arrays of character codes. This is a known limitation tracked in
[issue #3425](https://github.com/nlohmann/json/issues/3425).
5. Creates a JSON value of type array or object from the passed initializer list `init`. In case `type_deduction` is 5. Creates a JSON value of type array or object from the passed initializer list `init`. In case `type_deduction` is
`#!cpp true` (default), the type of the JSON value to be created is deducted from the initializer list `init` `#!cpp true` (default), the type of the JSON value to be created is deducted from the initializer list `init`
@@ -147,6 +153,11 @@ basic_json(basic_json&& other) noexcept;
- `BasicJsonType` is a `basic_json` type. - `BasicJsonType` is a `basic_json` type.
- `BasicJsonType` has different template arguments than `basic_json_t`. - `BasicJsonType` has different template arguments than `basic_json_t`.
**Note:** For cross-`basic_json` conversions to produce correct results, the target `basic_json`'s
`object_t::key_type` and `string_t` must be directly constructible from the source `basic_json`'s
corresponding types. See the description of overload (4) above for details on what happens when
this requirement is not met.
`U`: `U`:
: `uncvref_t<CompatibleType>` : `uncvref_t<CompatibleType>`
@@ -93,6 +93,15 @@ alphabetical order as `std::map` with `std::less` is used by default. Please not
[RFC 8259](https://tools.ietf.org/html/rfc8259), because any order implements the specified "unordered" nature of JSON [RFC 8259](https://tools.ietf.org/html/rfc8259), because any order implements the specified "unordered" nature of JSON
objects. objects.
#### Cross-`basic_json` conversion requirements
When converting an object from one `basic_json` specialization to another via the
[converting constructor](basic_json.md#overload-4), the target `object_t`'s `key_type` must be
directly constructible from the source `basic_json`'s `string_t` type (or more generally, from the
source object's key type). If this requirement is not met, the conversion does not fail; instead,
the object is silently converted as an array of key-value pairs, which is incorrect. See
[issue #3425](https://github.com/nlohmann/json/issues/3425) for details and an example.
## Examples ## Examples
??? example ??? example
+12 -3
View File
@@ -6,9 +6,18 @@ namespace std {
} }
``` ```
Return a hash value for a JSON object. The hash function tries to rely on `std::hash` where possible. Furthermore, the Return a hash value for a JSON object. The hash function tries to rely on `std::hash` where possible. To satisfy the
type of the JSON value is taken into account to have different hash values for `#!json null`, `#!cpp 0`, `#!cpp 0U`, and `std::hash` contract, numeric JSON values that compare equal must hash to the same value. This means:
`#!cpp false`, etc.
- `json(42)`, `json(42u)`, and `json(42.0)` all hash to the same value
- `json(0)`, `json(0u)`, and `json(0.0)` all hash to the same value
Different types hash differently for non-numeric types (e.g., `#!json null`, `#!cpp false`, and strings all have distinct hashes).
**Edge case:** For very large integers outside the exact representable range of the floating-point type (beyond ~2^53 for
typical `double`), the hash values for integer and floating-point values may differ, even if the floating-point value
was obtained by casting the integer (due to precision loss). This is a documented limitation arising from how the
comparison operator normalizes numeric types.
## Examples ## Examples
@@ -45,6 +45,15 @@ This implementation is interoperable as it does compare strings code unit by cod
String values are stored as pointers in a `basic_json` type. That is, for any access to string values, a pointer of type String values are stored as pointers in a `basic_json` type. That is, for any access to string values, a pointer of type
`string_t*` must be dereferenced. `string_t*` must be dereferenced.
#### Cross-`basic_json` conversion requirements
When converting a string value from one `basic_json` specialization to another via the
[converting constructor](basic_json.md#overload-4), the target `string_t` must be directly
constructible from the source `basic_json`'s `string_t` type. If this requirement is not met, the
conversion does not fail; instead, the string is silently converted as an array of character codes,
which is incorrect. See [issue #3425](https://github.com/nlohmann/json/issues/3425) for details
and an example.
## Examples ## Examples
??? example ??? example
+1
View File
@@ -11,6 +11,7 @@ int main()
<< "hash(false) = " << std::hash<json> {}(json(false)) << '\n' << "hash(false) = " << std::hash<json> {}(json(false)) << '\n'
<< "hash(0) = " << std::hash<json> {}(json(0)) << '\n' << "hash(0) = " << std::hash<json> {}(json(0)) << '\n'
<< "hash(0U) = " << std::hash<json> {}(json(0U)) << '\n' << "hash(0U) = " << std::hash<json> {}(json(0U)) << '\n'
<< "hash(0.0) = " << std::hash<json> {}(json(0.0)) << '\n'
<< "hash(\"\") = " << std::hash<json> {}(json("")) << '\n' << "hash(\"\") = " << std::hash<json> {}(json("")) << '\n'
<< "hash({}) = " << std::hash<json> {}(json::object()) << '\n' << "hash({}) = " << std::hash<json> {}(json::object()) << '\n'
<< "hash([]) = " << std::hash<json> {}(json::array()) << '\n' << "hash([]) = " << std::hash<json> {}(json::array()) << '\n'
+5 -4
View File
@@ -1,8 +1,9 @@
hash(null) = 2654435769 hash(null) = 2654435769
hash(false) = 2654436030 hash(false) = 2654436030
hash(0) = 2654436095 hash(0) = 2654436221
hash(0U) = 2654436156 hash(0U) = 2654436221
hash("") = 6142509191626859748 hash(0.0) = 2654436221
hash("") = 11160318156688833227
hash({}) = 2654435832 hash({}) = 2654435832
hash([]) = 2654435899 hash([]) = 2654435899
hash({"hello": "world"}) = 4469488738203676328 hash({"hello": "world"}) = 3701319991624763853
+4
View File
@@ -0,0 +1,4 @@
User-agent: *
Allow: /
Sitemap: https://json.nlohmann.me/sitemap.xml
+25
View File
@@ -0,0 +1,25 @@
"""Copy each documentation page's Markdown source into the built site."""
# Creates a `<path>.md` sibling of each HTML output (for example,
# `features/comments/` becomes `features/comments.md`) so agents and tools can
# fetch the raw Markdown directly instead of parsing rendered HTML.
import os
import shutil
_pages = []
def on_files(files, config):
global _pages
_pages = [f for f in files if f.is_documentation_page()]
return files
def on_post_build(config):
site_dir = config["site_dir"]
for file in _pages:
url = file.url.rstrip("/")
target = os.path.join(site_dir, (url or "index") + ".md")
os.makedirs(os.path.dirname(target), exist_ok=True)
shutil.copyfile(file.abs_src_path, target)
+30
View File
@@ -367,6 +367,9 @@ markdown_extensions:
auto_append: auto_append:
- ../includes/glossary.md - ../includes/glossary.md
hooks:
- hooks/copy_markdown_source.py
plugins: plugins:
- search: - search:
separator: '[\s\-\.]' separator: '[\s\-\.]'
@@ -389,6 +392,33 @@ plugins:
- https://nlohmann.github.io/json/* - https://nlohmann.github.io/json/*
- mailto:* - mailto:*
- privacy - privacy
- llmstxt:
markdown_description: >
JSON for Modern C++ is a C++11 header-only library implementing a JSON
value type with an STL-like API, JSON Pointer/Patch, CBOR/MessagePack/
BSON/UBJSON/BJData binary format support, and a SAX-style parser interface.
sections:
Home:
- index.md
- home/*.md
Features:
- features/*.md
- features/binary_formats/*.md
- features/element_access/*.md
- features/parsing/*.md
- features/types/*.md
Integration:
- integration/*.md
API Documentation:
- api/*.md
- api/basic_json/*.md
- api/adl_serializer/*.md
- api/byte_container_with_subtype/*.md
- api/json_pointer/*.md
- api/json_sax/*.md
- api/macros/*.md
Community:
- community/*.md
extra_css: extra_css:
- css/custom.css - css/custom.css
+1
View File
@@ -7,5 +7,6 @@ mkdocs-material-extensions==1.3.1 # extensions
mkdocs-minify-plugin==0.8.0 # plugin "minify" mkdocs-minify-plugin==0.8.0 # plugin "minify"
mkdocs-redirects==1.2.3 # plugin "redirects" mkdocs-redirects==1.2.3 # plugin "redirects"
mkdocs-htmlproofer-plugin==1.5.0 # plugin "htmlproofer" mkdocs-htmlproofer-plugin==1.5.0 # plugin "htmlproofer"
mkdocs-llmstxt==0.5.0 # plugin "llmstxt"
PyYAML==6.0.3 # linter PyYAML==6.0.3 # linter
+89 -6
View File
@@ -11,6 +11,8 @@
#include <cstdint> // uint8_t #include <cstdint> // uint8_t
#include <cstddef> // size_t #include <cstddef> // size_t
#include <functional> // hash #include <functional> // hash
#include <limits> // numeric_limits
#include <cmath> // isfinite
#include <nlohmann/detail/abi_macros.hpp> #include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/value_t.hpp> #include <nlohmann/detail/value_t.hpp>
@@ -26,12 +28,66 @@ inline std::size_t combine(std::size_t seed, std::size_t h) noexcept
return seed; return seed;
} }
// Check if a number_integer_t value is exactly representable as number_float_t
// Returns true if static_cast<number_integer_t>(static_cast<number_float_t>(val)) == val
template<typename BasicJsonType>
inline bool is_exactly_representable_as_float(typename BasicJsonType::number_integer_t val) noexcept
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_float_t = typename BasicJsonType::number_float_t;
// If the float type's mantissa covers the integer type's entire range, all values round-trip
constexpr int float_digits = std::numeric_limits<number_float_t>::digits;
constexpr int int_digits = std::numeric_limits<number_integer_t>::digits;
if (float_digits >= int_digits)
{
return true;
}
// For values outside float's exact range, they don't round-trip
// The safe way to check: compute the max magnitude that round-trips
// Using unsigned arithmetic to avoid UB with negating INT_MIN
// Max magnitude representable exactly: 2^(digits-1) - 1 for signed, 2^digits - 1 for unsigned range
// But we're checking a signed value, so use 2^digits as the threshold
constexpr auto max_exact = static_cast<number_integer_t>(1) << (float_digits - 1);
// Check absolute value against this threshold
if (val >= 0)
{
if (val >= max_exact)
{
return false;
}
}
else
{
// For negative values, check via unsigned wrapping arithmetic
// -val in unsigned domain; if it wraps, the value is too negative
auto unsigned_abs = static_cast<typename BasicJsonType::number_unsigned_t>(-val);
if (unsigned_abs >= static_cast<typename BasicJsonType::number_unsigned_t>(max_exact))
{
return false;
}
}
// For values within the exact range, verify the round-trip
const auto f = static_cast<number_float_t>(val);
return std::isfinite(f) && static_cast<number_integer_t>(f) == val;
}
/*! /*!
@brief hash a JSON value @brief hash a JSON value
The hash function tries to rely on std::hash where possible. Furthermore, the The hash function tries to rely on std::hash where possible. Furthermore, the
type of the JSON value is taken into account to have different hash values for type of the JSON value is taken into account to have different hash values for
null, 0, 0U, and false, etc. most types. However, numeric types (number_integer, number_unsigned, number_float)
are hashed to satisfy the std::hash contract: if two json values compare equal,
they must have equal hash values. This means json(42), json(42u), and json(42.0)
all hash to the same value (since they compare equal). For large integer values
outside the exact representable range of the float type, integer values are hashed
in their own domain to avoid precision loss.
@tparam BasicJsonType basic_json specialization @tparam BasicJsonType basic_json specialization
@param j JSON value to hash @param j JSON value to hash
@@ -90,20 +146,47 @@ std::size_t hash(const BasicJsonType& j)
case BasicJsonType::value_t::number_integer: case BasicJsonType::value_t::number_integer:
{ {
const auto h = std::hash<number_integer_t> {}(j.template get<number_integer_t>()); const auto v = j.template get<number_integer_t>();
return combine(type, h); // Use a shared numeric type tag so all numeric types that are equal hash the same
const auto numeric_type = static_cast<std::size_t>(BasicJsonType::value_t::number_float);
if (is_exactly_representable_as_float<BasicJsonType>(v))
{
const auto h = std::hash<number_float_t> {}(static_cast<number_float_t>(v));
return combine(numeric_type, h);
}
else
{
const auto h = std::hash<number_integer_t> {}(v);
return combine(numeric_type, h);
}
} }
case BasicJsonType::value_t::number_unsigned: case BasicJsonType::value_t::number_unsigned:
{ {
const auto h = std::hash<number_unsigned_t> {}(j.template get<number_unsigned_t>()); const auto v = j.template get<number_unsigned_t>();
return combine(type, h); // Normalize to signed (matching operator== behavior for U-vs-I comparison)
const auto v_as_signed = static_cast<number_integer_t>(v);
// Use a shared numeric type tag so all numeric types that are equal hash the same
const auto numeric_type = static_cast<std::size_t>(BasicJsonType::value_t::number_float);
if (is_exactly_representable_as_float<BasicJsonType>(v_as_signed))
{
const auto h = std::hash<number_float_t> {}(static_cast<number_float_t>(v_as_signed));
return combine(numeric_type, h);
}
else
{
const auto h = std::hash<number_integer_t> {}(v_as_signed);
return combine(numeric_type, h);
}
} }
case BasicJsonType::value_t::number_float: case BasicJsonType::value_t::number_float:
{ {
const auto h = std::hash<number_float_t> {}(j.template get<number_float_t>()); const auto h = std::hash<number_float_t> {}(j.template get<number_float_t>());
return combine(type, h); const auto numeric_type = static_cast<std::size_t>(BasicJsonType::value_t::number_float);
return combine(numeric_type, h);
} }
case BasicJsonType::value_t::binary: case BasicJsonType::value_t::binary:
+29 -9
View File
@@ -465,18 +465,12 @@ class serializer
{ {
if (codepoint <= 0xFFFF) if (codepoint <= 0xFFFF)
{ {
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg) write_u_escape(bytes, static_cast<std::uint16_t>(codepoint));
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 7, "\\u%04x",
static_cast<std::uint16_t>(codepoint)));
bytes += 6;
} }
else else
{ {
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg) write_u_escape(bytes, static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)));
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 13, "\\u%04x\\u%04x", write_u_escape(bytes, static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu)));
static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)),
static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu))));
bytes += 12;
} }
} }
else else
@@ -683,6 +677,32 @@ class serializer
return result; return result;
} }
/*!
* @brief write a lowercase "\uXXXX" escape sequence into @a string_buffer
*
* Branch-free replacement for `snprintf(buf, 7, "\\u%04x", codeunit)` in the
* string escaping hot path. It writes exactly six characters ('\\', 'u' and
* four hex digits) at position @a pos of @a string_buffer via a nibble
* lookup table, avoiding the format-string parsing and locale machinery of
* `snprintf`. Advances @a pos by the number of bytes written (6).
*
* @param[in] pos position in @a string_buffer to write at; there must
* be at least 6 bytes of headroom
* @param[in] codeunit 16-bit value to encode
*/
void write_u_escape(std::size_t& pos, std::uint16_t codeunit) noexcept
{
JSON_ASSERT(string_buffer.size() - pos >= 6);
constexpr const char* nibble_to_hex = "0123456789abcdef";
string_buffer[pos + 0] = '\\';
string_buffer[pos + 1] = 'u';
string_buffer[pos + 2] = nibble_to_hex[(codeunit >> 12u) & 0x0Fu];
string_buffer[pos + 3] = nibble_to_hex[(codeunit >> 8u) & 0x0Fu];
string_buffer[pos + 4] = nibble_to_hex[(codeunit >> 4u) & 0x0Fu];
string_buffer[pos + 5] = nibble_to_hex[codeunit & 0x0Fu];
pos += 6;
}
// templates to avoid warnings about useless casts // templates to avoid warnings about useless casts
template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0> template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0>
bool is_negative_number(NumberType x) bool is_negative_number(NumberType x)
+118 -15
View File
@@ -6677,6 +6677,8 @@ NLOHMANN_JSON_NAMESPACE_END
#include <cstdint> // uint8_t #include <cstdint> // uint8_t
#include <cstddef> // size_t #include <cstddef> // size_t
#include <functional> // hash #include <functional> // hash
#include <limits> // numeric_limits
#include <cmath> // isfinite
// #include <nlohmann/detail/abi_macros.hpp> // #include <nlohmann/detail/abi_macros.hpp>
@@ -6694,12 +6696,66 @@ inline std::size_t combine(std::size_t seed, std::size_t h) noexcept
return seed; return seed;
} }
// Check if a number_integer_t value is exactly representable as number_float_t
// Returns true if static_cast<number_integer_t>(static_cast<number_float_t>(val)) == val
template<typename BasicJsonType>
inline bool is_exactly_representable_as_float(typename BasicJsonType::number_integer_t val) noexcept
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_float_t = typename BasicJsonType::number_float_t;
// If the float type's mantissa covers the integer type's entire range, all values round-trip
constexpr int float_digits = std::numeric_limits<number_float_t>::digits;
constexpr int int_digits = std::numeric_limits<number_integer_t>::digits;
if (float_digits >= int_digits)
{
return true;
}
// For values outside float's exact range, they don't round-trip
// The safe way to check: compute the max magnitude that round-trips
// Using unsigned arithmetic to avoid UB with negating INT_MIN
// Max magnitude representable exactly: 2^(digits-1) - 1 for signed, 2^digits - 1 for unsigned range
// But we're checking a signed value, so use 2^digits as the threshold
constexpr auto max_exact = static_cast<number_integer_t>(1) << (float_digits - 1);
// Check absolute value against this threshold
if (val >= 0)
{
if (val >= max_exact)
{
return false;
}
}
else
{
// For negative values, check via unsigned wrapping arithmetic
// -val in unsigned domain; if it wraps, the value is too negative
auto unsigned_abs = static_cast<typename BasicJsonType::number_unsigned_t>(-val);
if (unsigned_abs >= static_cast<typename BasicJsonType::number_unsigned_t>(max_exact))
{
return false;
}
}
// For values within the exact range, verify the round-trip
const auto f = static_cast<number_float_t>(val);
return std::isfinite(f) && static_cast<number_integer_t>(f) == val;
}
/*! /*!
@brief hash a JSON value @brief hash a JSON value
The hash function tries to rely on std::hash where possible. Furthermore, the The hash function tries to rely on std::hash where possible. Furthermore, the
type of the JSON value is taken into account to have different hash values for type of the JSON value is taken into account to have different hash values for
null, 0, 0U, and false, etc. most types. However, numeric types (number_integer, number_unsigned, number_float)
are hashed to satisfy the std::hash contract: if two json values compare equal,
they must have equal hash values. This means json(42), json(42u), and json(42.0)
all hash to the same value (since they compare equal). For large integer values
outside the exact representable range of the float type, integer values are hashed
in their own domain to avoid precision loss.
@tparam BasicJsonType basic_json specialization @tparam BasicJsonType basic_json specialization
@param j JSON value to hash @param j JSON value to hash
@@ -6758,20 +6814,47 @@ std::size_t hash(const BasicJsonType& j)
case BasicJsonType::value_t::number_integer: case BasicJsonType::value_t::number_integer:
{ {
const auto h = std::hash<number_integer_t> {}(j.template get<number_integer_t>()); const auto v = j.template get<number_integer_t>();
return combine(type, h); // Use a shared numeric type tag so all numeric types that are equal hash the same
const auto numeric_type = static_cast<std::size_t>(BasicJsonType::value_t::number_float);
if (is_exactly_representable_as_float<BasicJsonType>(v))
{
const auto h = std::hash<number_float_t> {}(static_cast<number_float_t>(v));
return combine(numeric_type, h);
}
else
{
const auto h = std::hash<number_integer_t> {}(v);
return combine(numeric_type, h);
}
} }
case BasicJsonType::value_t::number_unsigned: case BasicJsonType::value_t::number_unsigned:
{ {
const auto h = std::hash<number_unsigned_t> {}(j.template get<number_unsigned_t>()); const auto v = j.template get<number_unsigned_t>();
return combine(type, h); // Normalize to signed (matching operator== behavior for U-vs-I comparison)
const auto v_as_signed = static_cast<number_integer_t>(v);
// Use a shared numeric type tag so all numeric types that are equal hash the same
const auto numeric_type = static_cast<std::size_t>(BasicJsonType::value_t::number_float);
if (is_exactly_representable_as_float<BasicJsonType>(v_as_signed))
{
const auto h = std::hash<number_float_t> {}(static_cast<number_float_t>(v_as_signed));
return combine(numeric_type, h);
}
else
{
const auto h = std::hash<number_integer_t> {}(v_as_signed);
return combine(numeric_type, h);
}
} }
case BasicJsonType::value_t::number_float: case BasicJsonType::value_t::number_float:
{ {
const auto h = std::hash<number_float_t> {}(j.template get<number_float_t>()); const auto h = std::hash<number_float_t> {}(j.template get<number_float_t>());
return combine(type, h); const auto numeric_type = static_cast<std::size_t>(BasicJsonType::value_t::number_float);
return combine(numeric_type, h);
} }
case BasicJsonType::value_t::binary: case BasicJsonType::value_t::binary:
@@ -19962,18 +20045,12 @@ class serializer
{ {
if (codepoint <= 0xFFFF) if (codepoint <= 0xFFFF)
{ {
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg) write_u_escape(bytes, static_cast<std::uint16_t>(codepoint));
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 7, "\\u%04x",
static_cast<std::uint16_t>(codepoint)));
bytes += 6;
} }
else else
{ {
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-vararg,hicpp-vararg) write_u_escape(bytes, static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)));
static_cast<void>((std::snprintf)(string_buffer.data() + bytes, 13, "\\u%04x\\u%04x", write_u_escape(bytes, static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu)));
static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)),
static_cast<std::uint16_t>(0xDC00u + (codepoint & 0x3FFu))));
bytes += 12;
} }
} }
else else
@@ -20180,6 +20257,32 @@ class serializer
return result; return result;
} }
/*!
* @brief write a lowercase "\uXXXX" escape sequence into @a string_buffer
*
* Branch-free replacement for `snprintf(buf, 7, "\\u%04x", codeunit)` in the
* string escaping hot path. It writes exactly six characters ('\\', 'u' and
* four hex digits) at position @a pos of @a string_buffer via a nibble
* lookup table, avoiding the format-string parsing and locale machinery of
* `snprintf`. Advances @a pos by the number of bytes written (6).
*
* @param[in] pos position in @a string_buffer to write at; there must
* be at least 6 bytes of headroom
* @param[in] codeunit 16-bit value to encode
*/
void write_u_escape(std::size_t& pos, std::uint16_t codeunit) noexcept
{
JSON_ASSERT(string_buffer.size() - pos >= 6);
constexpr const char* nibble_to_hex = "0123456789abcdef";
string_buffer[pos + 0] = '\\';
string_buffer[pos + 1] = 'u';
string_buffer[pos + 2] = nibble_to_hex[(codeunit >> 12u) & 0x0Fu];
string_buffer[pos + 3] = nibble_to_hex[(codeunit >> 8u) & 0x0Fu];
string_buffer[pos + 4] = nibble_to_hex[(codeunit >> 4u) & 0x0Fu];
string_buffer[pos + 5] = nibble_to_hex[codeunit & 0x0Fu];
pos += 6;
}
// templates to avoid warnings about useless casts // templates to avoid warnings about useless casts
template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0> template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0>
bool is_negative_number(NumberType x) bool is_negative_number(NumberType x)
+3
View File
@@ -232,6 +232,9 @@ TEST_CASE("algorithms")
// only the first four elements are expected to be sorted, the rest are // only the first four elements are expected to be sorted, the rest are
// unspecified by the standard // unspecified by the standard
const json expected({nullptr, false, true, 3}); const json expected({nullptr, false, true, 3});
// std::equal below only bounds-checks the first range; assert the
// second range is at least as long to rule out an over-read (CWE-126)
CHECK(std::distance(begin(expected), end(expected)) >= 4);
CHECK(std::equal(j.begin(), j.begin() + 4, begin(expected))); CHECK(std::equal(j.begin(), j.begin() + 4, begin(expected)));
} }
} }
+6 -8
View File
@@ -322,14 +322,12 @@ TEST_CASE("alternative string type")
SECTION("JSON pointer") SECTION("JSON pointer")
{ {
// conversion from json to alt_json fails to compile (see #3425); // Direct conversion from a json literal to alt_json is not supported due to issue #3425:
// attempted fix(*) produces: [[['b','a','r'],['b','a','z']]] (with each char being an integer) // alt_json's string_t (alt_string) is not directly constructible from std::string, so the
// (*) disable implicit conversion for json_refs of any basic_json type // cross-basic_json conversion falls back to the array-conversion path, incorrectly representing
// alt_json j = R"( // objects as arrays of [key, value] pairs and strings as arrays of character codes.
// { // See https://github.com/nlohmann/json/issues/3425 for details.
// "foo": ["bar", "baz"] // Workaround: use alt_json::parse() instead of implicit conversion.
// }
// )"_json;
auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})"); auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})");
CHECK(j.at(alt_json::json_pointer("/foo/0")) == j["foo"][0]); CHECK(j.at(alt_json::json_pointer("/foo/0")) == j["foo"][0]);
+26
View File
@@ -168,6 +168,32 @@ TEST_CASE("convenience functions")
CHECK_THROWS_WITH_AS(check_escaped("\xC2"), "[json.exception.type_error.316] incomplete UTF-8 string; last byte: 0xC2", json::type_error&); CHECK_THROWS_WITH_AS(check_escaped("\xC2"), "[json.exception.type_error.316] incomplete UTF-8 string; last byte: 0xC2", json::type_error&);
} }
SECTION("string escape with ensure_ascii")
{
// control characters are escaped regardless of ensure_ascii
check_escaped("\x01", "\\u0001", true);
check_escaped("\x1f", "\\u001f", true);
// non-ASCII code points in the Basic Multilingual Plane are emitted as
// a single lowercase \uXXXX escape (exercises every nibble position)
check_escaped("\xC2\x80", "\\u0080", true); // U+0080
check_escaped("\xC3\xBF", "\\u00ff", true); // U+00FF (ÿ)
check_escaped("\xDF\xBF", "\\u07ff", true); // U+07FF
check_escaped("\xE4\xBD\xA0", "\\u4f60", true); // U+4F60 (你)
check_escaped("\xEA\xAF\x8D", "\\uabcd", true); // U+ABCD
check_escaped("\xEF\xBF\xBD", "\\ufffd", true); // U+FFFD (replacement char, all-f nibbles)
// code points outside the BMP are emitted as a UTF-16 surrogate pair
// of two lowercase \uXXXX escapes
check_escaped("\xF0\x90\x80\x80", "\\ud800\\udc00", true); // U+10000 (lowest astral)
check_escaped("\xF0\x9F\x98\x80", "\\ud83d\\ude00", true); // U+1F600 (😀)
check_escaped("\xF4\x8F\xBF\xBF", "\\udbff\\udfff", true); // U+10FFFF (highest code point)
// with ensure_ascii disabled, non-ASCII input is passed through verbatim
check_escaped("\xE4\xBD\xA0", "\xE4\xBD\xA0", false);
check_escaped("\xF0\x9F\x98\x80", "\xF0\x9F\x98\x80", false);
}
SECTION("string concat") SECTION("string concat")
{ {
using nlohmann::detail::concat; using nlohmann::detail::concat;
+22 -6
View File
@@ -35,10 +35,10 @@ TEST_CASE("hash<nlohmann::json>")
// number // number
hashes.insert(std::hash<json> {}(json(0))); hashes.insert(std::hash<json> {}(json(0)));
hashes.insert(std::hash<json> {}(json(static_cast<unsigned>(0)))); hashes.insert(std::hash<json> {}(json(static_cast<unsigned>(0)))); // now same hash as json(0)
hashes.insert(std::hash<json> {}(json(0.0))); // now same hash as json(0)
hashes.insert(std::hash<json> {}(json(-1))); hashes.insert(std::hash<json> {}(json(-1)));
hashes.insert(std::hash<json> {}(json(0.0)));
hashes.insert(std::hash<json> {}(json(42.23))); hashes.insert(std::hash<json> {}(json(42.23)));
// array // array
@@ -60,7 +60,16 @@ TEST_CASE("hash<nlohmann::json>")
// discarded // discarded
hashes.insert(std::hash<json> {}(json(json::value_t::discarded))); hashes.insert(std::hash<json> {}(json(json::value_t::discarded)));
CHECK(hashes.size() == 21); // Note: json(0), json(0U), and json(0.0) now hash to the same value
// (to satisfy the std::hash contract: equal values must hash equally)
// So we expect 19 distinct hashes instead of 21
CHECK(hashes.size() == 19);
// Verify the std::hash contract: equal values must hash equally
CHECK(std::hash<json> {}(json(0)) == std::hash<json> {}(json(static_cast<unsigned>(0))));
CHECK(std::hash<json> {}(json(0)) == std::hash<json> {}(json(0.0)));
CHECK(std::hash<json> {}(json(42)) == std::hash<json> {}(json(42u)));
CHECK(std::hash<json> {}(json(42)) == std::hash<json> {}(json(42.0)));
} }
TEST_CASE("hash<nlohmann::ordered_json>") TEST_CASE("hash<nlohmann::ordered_json>")
@@ -84,10 +93,10 @@ TEST_CASE("hash<nlohmann::ordered_json>")
// number // number
hashes.insert(std::hash<ordered_json> {}(ordered_json(0))); hashes.insert(std::hash<ordered_json> {}(ordered_json(0)));
hashes.insert(std::hash<ordered_json> {}(ordered_json(static_cast<unsigned>(0)))); hashes.insert(std::hash<ordered_json> {}(ordered_json(static_cast<unsigned>(0)))); // now same hash as ordered_json(0)
hashes.insert(std::hash<ordered_json> {}(ordered_json(0.0))); // now same hash as ordered_json(0)
hashes.insert(std::hash<ordered_json> {}(ordered_json(-1))); hashes.insert(std::hash<ordered_json> {}(ordered_json(-1)));
hashes.insert(std::hash<ordered_json> {}(ordered_json(0.0)));
hashes.insert(std::hash<ordered_json> {}(ordered_json(42.23))); hashes.insert(std::hash<ordered_json> {}(ordered_json(42.23)));
// array // array
@@ -109,5 +118,12 @@ TEST_CASE("hash<nlohmann::ordered_json>")
// discarded // discarded
hashes.insert(std::hash<ordered_json> {}(ordered_json(ordered_json::value_t::discarded))); hashes.insert(std::hash<ordered_json> {}(ordered_json(ordered_json::value_t::discarded)));
CHECK(hashes.size() == 21); // Note: ordered_json(0), ordered_json(0U), and ordered_json(0.0) now hash to the same value
CHECK(hashes.size() == 19);
// Verify the std::hash contract for ordered_json as well
CHECK(std::hash<ordered_json> {}(ordered_json(0)) == std::hash<ordered_json> {}(ordered_json(static_cast<unsigned>(0))));
CHECK(std::hash<ordered_json> {}(ordered_json(0)) == std::hash<ordered_json> {}(ordered_json(0.0)));
CHECK(std::hash<ordered_json> {}(ordered_json(42)) == std::hash<ordered_json> {}(ordered_json(42u)));
CHECK(std::hash<ordered_json> {}(ordered_json(42)) == std::hash<ordered_json> {}(ordered_json(42.0)));
} }