Compare commits

..
Author SHA1 Message Date
Niels Lohmann b8d53c0888 Derive the JSON_HEDLEY_* leak-check test from hedley.hpp at build time
tests/src/unit-no-macro-leak.cpp previously hardcoded a static list of
~151 #ifdef/#error checks, one per JSON_HEDLEY_* macro name known at the
time it was written. That list would silently go stale the next time
`make update_hedley` pulls in a vendor update that adds, removes, or
renames a macro, since nothing would force it to be regenerated.

Add cmake/scripts/gen_hedley_undef_check.cmake, which derives the full
list of JSON_HEDLEY_* macro names directly from
include/nlohmann/thirdparty/hedley/hedley.hpp:

- tests/CMakeLists.txt uses it (MODE=checks) to (re)generate
  hedley_undef_checks.inc at configure and build time, and wires the
  generating custom target as a dependency of the test-no-macro-leak_cpp*
  targets so it can never build against a stale copy. unit-no-macro-leak.cpp
  now just #include-s the generated file inside its TEST_CASE instead of
  carrying the checks itself.
- The Makefile's `update_hedley` target now delegates hedley_undef.hpp
  generation to the same script (MODE=undef, new `update_hedley_undef`
  target), so the vendored header, the generated #undef list, and the
  generated test checks are all derived from the same extraction logic and
  cannot drift apart.

This mirrors the approach taken independently in #5415 for the same
issue (#5408), credited there to a self-regenerating mechanism that
"can never drift again" -- ported into this branch instead of the
static list originally proposed here.

Verified with a local CMake configure + build + ctest, both against
include/ (JSON_MultipleHeaders=ON) and against the amalgamated
single_include/nlohmann/json.hpp (JSON_MultipleHeaders=OFF), and by
temporarily deleting a #undef line from hedley_undef.hpp to confirm the
generated test actually fails on a real leak.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-05 21:38:21 +02:00
Niels Lohmann 06a4ce1306 Undefine the four JSON_HEDLEY_* macros that leak after including json.hpp
include/nlohmann/detail/macro_unscope.hpp includes hedley_undef.hpp to
#undef every JSON_HEDLEY_* macro so none of them leak into the including
translation unit. Four macros were missing from that list and therefore
stayed defined after #include <nlohmann/json.hpp>:

- JSON_HEDLEY_PRAGMA
- JSON_HEDLEY_PREDICT_TRUE
- JSON_HEDLEY_PREDICT_FALSE
- JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE

hedley_undef.hpp is generated (via `make update_hedley`) by grepping
hedley.hpp for its own internal `#undef JSON_HEDLEY_X` redefinition
guards. JSON_HEDLEY_PRAGMA/PREDICT_TRUE/PREDICT_FALSE have no such guard
in upstream Hedley, so they were never picked up. The guard for
JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE also has an upstream typo
(`JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE`), so hedley_undef.hpp
was undefining the wrong (never-defined) name.

Fixes:
- include/nlohmann/thirdparty/hedley/hedley_undef.hpp: corrected the
  DECLSPEC_ATTRIBUTE typo and added the three missing #undef lines,
  keeping the file's alphabetical ordering.
- Makefile (update_hedley target): changed hedley_undef.hpp generation to
  extract macro names directly from every `#define JSON_HEDLEY_...` in
  hedley.hpp instead of from existing `#undef` guards, so a future
  `make update_hedley` run undefines every macro Hedley actually defines,
  even ones without a pre-existing redefinition guard. This was not run
  in this PR (it would also pull in an unrelated upstream Hedley sync);
  hedley_undef.hpp was hand-patched instead and single_include was
  regenerated with `make amalgamate`.
- tests/src/unit-no-macro-leak.cpp: new regression test (picked up
  automatically by tests/CMakeLists.txt's existing unit-*.cpp glob) that
  includes json.hpp and then #ifdef/#error-checks every JSON_HEDLEY_*
  macro name, so any future leak of any of the 151 vendored macros fails
  the build, not just the four fixed here.

Fixes #5408.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-05 20:49:15 +02:00
10 changed files with 242 additions and 423 deletions
+18 -3
View File
@@ -1,4 +1,4 @@
.PHONY: pretty clean ChangeLog.md release
.PHONY: pretty clean ChangeLog.md release update_hedley update_hedley_undef
##########################################################################
# configuration
@@ -41,6 +41,8 @@ all:
@echo "fuzz_testing_ubjson - prepare fuzz testing of the UBJSON parser"
@echo "pretty - beautify code with Artistic Style"
@echo "run_benchmarks - build and run benchmarks"
@echo "update_hedley - download Hedley and regenerate hedley.hpp / hedley_undef.hpp"
@echo "update_hedley_undef - rebuild hedley_undef.hpp from the JSON_HEDLEY_* #define names in hedley.hpp"
##########################################################################
@@ -241,11 +243,24 @@ update_hedley:
rm -f include/nlohmann/thirdparty/hedley/hedley.hpp include/nlohmann/thirdparty/hedley/hedley_undef.hpp
curl https://raw.githubusercontent.com/nemequ/hedley/master/hedley.h -o include/nlohmann/thirdparty/hedley/hedley.hpp
$(SED) -i 's/HEDLEY_/JSON_HEDLEY_/g' include/nlohmann/thirdparty/hedley/hedley.hpp
grep "[[:blank:]]*#[[:blank:]]*undef" include/nlohmann/thirdparty/hedley/hedley.hpp | grep -v "__" | sort | uniq | $(SED) 's/ //g' | $(SED) 's/undef/undef /g' > include/nlohmann/thirdparty/hedley/hedley_undef.hpp
$(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley.hpp
$(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley_undef.hpp
$(MAKE) update_hedley_undef
$(MAKE) amalgamate
# Rebuild hedley_undef.hpp from every JSON_HEDLEY_* name that hedley.hpp
# #defines. Hedley does not #undef all of its public macros internally (see
# #5408), so grepping those #undef lines misses names such as
# JSON_HEDLEY_PRAGMA. cmake/scripts/gen_hedley_undef_check.cmake is the
# single source of truth for this extraction (tests/CMakeLists.txt uses the
# same script, in MODE=checks, to generate the matching leak-check test), so
# the vendored header, the generated #undef list, and the regression test
# cannot drift apart.
update_hedley_undef:
cmake -DHEDLEY_HPP=include/nlohmann/thirdparty/hedley/hedley.hpp \
-DOUTPUT=include/nlohmann/thirdparty/hedley/hedley_undef.hpp \
-DMODE=undef \
-P cmake/scripts/gen_hedley_undef_check.cmake
##########################################################################
# serve_header.py
##########################################################################
+105
View File
@@ -0,0 +1,105 @@
# Shared extractor for the JSON_HEDLEY_* macro names defined in hedley.hpp.
#
# Every macro that hedley.hpp #defines must be #undef-ed again once json.hpp
# has been fully processed (see include/nlohmann/detail/macro_unscope.hpp
# and https://github.com/nlohmann/json/issues/5408). Deriving the macro list
# straight from hedley.hpp here -- instead of hand-maintaining it in two
# places -- means hedley_undef.hpp and the regression test that checks for
# leaked macros can never drift apart, even after a future `make
# update_hedley` pulls in new macros from upstream Hedley.
#
# MODE=undef (default): write hedley_undef.hpp (SPDX header, #pragma once,
# one #undef per macro name) -- used by `make update_hedley_undef`
# MODE=checks: write one #ifdef/FAIL_CHECK/#endif per macro name,
# meant to be #include-d inside a TEST_CASE -- used by
# tests/CMakeLists.txt to (re)generate the include for
# tests/src/unit-no-macro-leak.cpp
#
# Required variables:
# HEDLEY_HPP path to include/nlohmann/thirdparty/hedley/hedley.hpp
# OUTPUT path of the file to (over)write
# Optional:
# MODE "undef" (default) or "checks"
if(NOT DEFINED HEDLEY_HPP OR NOT DEFINED OUTPUT)
message(FATAL_ERROR "HEDLEY_HPP and OUTPUT must be set")
endif()
if(NOT EXISTS "${HEDLEY_HPP}")
message(FATAL_ERROR "Hedley header not found: ${HEDLEY_HPP}")
endif()
if(NOT DEFINED MODE)
set(MODE undef)
endif()
if(NOT MODE STREQUAL "undef" AND NOT MODE STREQUAL "checks")
message(FATAL_ERROR "MODE must be undef or checks, got: ${MODE}")
endif()
# Line-anchored, like `grep -oE "^[[:blank:]]*#[[:blank:]]*define[[:blank:]]+JSON_HEDLEY_[A-Za-z0-9_]+"`.
# Unanchored matching would also pick up JSON_HEDLEY_* mentions inside
# comments or string literals elsewhere in the file, which must not turn
# into #undef lines.
file(STRINGS "${HEDLEY_HPP}" hedley_lines)
set(macro_names)
foreach(line IN LISTS hedley_lines)
if("${line}" MATCHES "^[ \t]*#[ \t]*define[ \t]+(JSON_HEDLEY_[A-Za-z0-9_]+)")
list(APPEND macro_names "${CMAKE_MATCH_1}")
endif()
endforeach()
if(NOT macro_names)
message(FATAL_ERROR "No JSON_HEDLEY_* macros found in ${HEDLEY_HPP}")
endif()
list(REMOVE_DUPLICATES macro_names)
# Lexicographic, locale-independent (ASCII-only names) -- matches `LC_ALL=C sort`.
list(SORT macro_names COMPARE STRING)
list(LENGTH macro_names macro_count)
set(generated "")
if(MODE STREQUAL "undef")
# Same banner `make update_hedley_undef` would stamp by hand, so the
# recipe is self-contained and its output is byte-stable across reruns.
string(APPEND generated "// __ _____ _____ _____\n")
string(APPEND generated "// __| | __| | | | JSON for Modern C++\n")
string(APPEND generated "// | | |__ | | | | | | version 3.12.0\n")
string(APPEND generated "// |_____|_____|_____|_|___| https://github.com/nlohmann/json\n")
string(APPEND generated "//\n")
string(APPEND generated "// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>\n")
string(APPEND generated "// SPDX-License-Identifier: MIT\n")
string(APPEND generated "\n")
string(APPEND generated "#pragma once\n")
string(APPEND generated "\n")
foreach(name IN LISTS macro_names)
string(APPEND generated "#undef ${name}\n")
endforeach()
else()
string(APPEND generated "// This file is generated by cmake/scripts/gen_hedley_undef_check.cmake\n")
string(APPEND generated "// from include/nlohmann/thirdparty/hedley/hedley.hpp. Do not edit it by\n")
string(APPEND generated "// hand -- it is regenerated on every build. ${macro_count} macros checked.\n\n")
foreach(name IN LISTS macro_names)
string(APPEND generated "#ifdef ${name}\n")
string(APPEND generated " FAIL_CHECK(\"${name} leaked after including nlohmann/json.hpp\");\n")
string(APPEND generated "#endif\n")
endforeach()
endif()
get_filename_component(output_dir "${OUTPUT}" DIRECTORY)
if(output_dir)
file(MAKE_DIRECTORY "${output_dir}")
endif()
# Avoid rewriting the file (and busting downstream incremental rebuilds)
# when the content has not actually changed.
set(write_output TRUE)
if(EXISTS "${OUTPUT}")
file(READ "${OUTPUT}" existing_content)
if(existing_content STREQUAL generated)
set(write_output FALSE)
endif()
endif()
if(write_output)
file(WRITE "${OUTPUT}" "${generated}")
endif()
+5 -128
View File
@@ -149,11 +149,10 @@ class lexer : public lexer_base<BasicJsonType>
public:
using token_type = typename lexer_base<BasicJsonType>::token_type;
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false) noexcept
: ia(std::move(adapter))
, ignore_comments(ignore_comments_)
, decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
, discard_number_values(discard_number_values_)
{}
// deleted because of pointer members
@@ -1280,58 +1279,6 @@ scan_number_done:
// we are done scanning a number)
unget();
// If the caller does not need the converted value (only whether the
// input is syntactically valid; see json_sax_acceptor/accept()), an
// unsigned/integer token can be reported without calling
// strtoull()/strtoll() at all, *provided* we can already tell from
// the digit count alone that the conversion cannot overflow 64 bits.
// Such tokens are always finite and are accepted unconditionally by
// the parser regardless of their actual value (parser::sax_parse_internal()
// never checks finiteness for value_unsigned/value_integer), so the
// classification below is all that is needed.
//
// A decimal number with up to 18 digits is always representable in
// both std::uint64_t and std::int64_t (18 nines is ~1e18, well below
// both UINT64_MAX ~1.8e19 and INT64_MAX ~9.2e18), so strtoull()/strtoll()
// could not have set errno to ERANGE for it. Numbers with more digits
// (rare in practice) fall through to the exact code below, unchanged,
// so their handling -- including reclassification to value_float when
// the value overflows 64 bits, and rejection when it is not even
// finite as a double -- is bit-for-bit identical to before this
// optimization.
//
// Note this reasons about std::uint64_t/std::int64_t, not about
// number_unsigned_t/number_integer_t (BasicJsonType's own, possibly
// narrower, template parameters -- e.g. std::uint32_t). That is fine
// *only* because discard_number_values is exclusively set by
// accept() (see json.hpp), and accept() always parses through the
// library's own json_sax_acceptor -- never a user-supplied SAX
// consumer -- whose number_unsigned()/number_integer()/number_float()
// callbacks unconditionally discard their argument and return true.
// So for every caller that can reach this branch, neither the token
// classification below nor the eventual (possibly narrowed, and on
// this fast path left stale/unset) value_unsigned/value_integer is
// ever consulted -- an unsigned/integer token is accepted outright,
// and even a >18-digit token that this fast path deliberately falls
// through for is, once reclassified to value_float, still finite
// (and thus accepted) for any digit count that fits in number_unsigned_t
// or number_integer_t regardless of that type's width. If this
// function is ever taught to run with discard_number_values true for
// a caller that *does* read the converted value, this reasoning (and
// the fast path below) would need to be revisited.
if (discard_number_values)
{
constexpr std::size_t safe_digit_count = 18;
if (number_type == token_type::value_unsigned && token_buffer.size() <= safe_digit_count)
{
return token_type::value_unsigned;
}
if (number_type == token_type::value_integer && token_buffer.size() - 1 <= safe_digit_count)
{
return token_type::value_integer;
}
}
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
errno = 0;
@@ -1446,7 +1393,8 @@ scan_number_done:
*/
char_int_type get()
{
advance_position();
++position.chars_read_total;
++position.chars_read_current_line;
if (next_unget)
{
@@ -1458,23 +1406,6 @@ scan_number_done:
current = ia.get_character();
}
return track_after_read();
}
/// shared head of get() / get_ignoring_pending_unget(): bump the
/// per-character position counters (line-count-on-'\n' bookkeeping is
/// handled afterwards, in track_after_read(), once `current` is known)
void advance_position() noexcept
{
++position.chars_read_total;
++position.chars_read_current_line;
}
/// shared tail of get() / get_ignoring_pending_unget(): capture the
/// character for error messages (if needed) and update line/column
/// bookkeeping for the character now in `current`
char_int_type track_after_read()
{
// seekable adapters reconstruct the token lazily on error (see
// get_token_string), so the eager per-character copy is skipped
capture_char(std::integral_constant<bool, lazy_token_string> {});
@@ -1488,29 +1419,6 @@ scan_number_done:
return current;
}
/*!
@brief like get(), but for call sites that can prove no unget() is pending
get() has to check the `next_unget` flag on every call, because a
previous token may have ended with unget() (e.g. scan_number() always
ungets the character that terminated the number, so the next call to
scan() can see it again). skip_whitespace() reads that first,
possibly-ungotten character via a plain get(), but every further
character it reads is guaranteed to be a fresh read: nothing between
those calls invokes unget(). This variant skips the (otherwise always
false) next_unget branch for those calls; it is not a general
replacement for get().
*/
char_int_type get_ignoring_pending_unget()
{
JSON_ASSERT(!next_unget);
advance_position();
current = ia.get_character();
return track_after_read();
}
/// seekable adapter: nothing to capture, the token is rebuilt on error
void capture_char(std::true_type /*lazy*/) const noexcept {}
@@ -1704,37 +1612,13 @@ scan_number_done:
return true;
}
/// whether `current` is one of the four JSON whitespace characters
bool current_is_whitespace() const noexcept
{
return current == ' ' || current == '\t' || current == '\n' || current == '\r';
}
void skip_whitespace()
{
// the first character may be a pending unget() left over from the
// previous token (see get_ignoring_pending_unget()); every
// subsequent character read by this loop is guaranteed fresh, since
// nothing below calls unget()
get();
if (!current_is_whitespace())
{
return;
}
// this is written as an if-guarded do-while (rather than a plain
// while loop) because that shape is what lets both GCC and Clang
// keep the input adapter's read pointer in a register across
// iterations; the equivalent while-loop measurably defeated that
// optimization in testing, turning long whitespace runs (e.g. the
// indentation of pretty-printed JSON) from a register-only loop
// into one that reloads the pointer from memory every character
do
{
get_ignoring_pending_unget();
get();
}
while (current_is_whitespace());
while (current == ' ' || current == '\t' || current == '\n' || current == '\r');
}
token_type scan()
@@ -1870,13 +1754,6 @@ scan_number_done:
const char_int_type decimal_point_char = '.';
/// the position of the decimal point in the input
std::size_t decimal_point_position = std::string::npos;
/// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
/// token classification and never looks at the converted numeric value;
/// when set, scan_number() may skip strtoull()/strtoll() for
/// value_unsigned/value_integer tokens whose digit count guarantees they
/// fit into 64 bits (see scan_number())
const bool discard_number_values = false;
};
} // namespace detail
+2 -3
View File
@@ -72,10 +72,9 @@ class parser
parser_callback_t<BasicJsonType> cb = nullptr,
const bool allow_exceptions_ = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas_ = false,
const bool discard_number_values_ = false)
const bool ignore_trailing_commas_ = false)
: callback(std::move(cb))
, m_lexer(std::move(adapter), ignore_comments, discard_number_values_)
, m_lexer(std::move(adapter), ignore_comments)
, allow_exceptions(allow_exceptions_)
, ignore_trailing_commas(ignore_trailing_commas_)
{
+5 -6
View File
@@ -164,12 +164,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
detail::parser_callback_t<basic_json>cb = nullptr,
const bool allow_exceptions = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas = false,
const bool discard_number_values = false
const bool ignore_trailing_commas = false
)
{
return ::nlohmann::detail::parser<basic_json, InputAdapterType>(std::move(adapter),
std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas, discard_number_values);
std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
}
private:
@@ -4134,7 +4133,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
}
/// @brief check if the input is valid JSON (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -4145,7 +4144,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
}
JSON_HEDLEY_WARN_UNUSED_RESULT
@@ -4154,7 +4153,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
}
/// @brief generate SAX events
+4 -1
View File
@@ -17,7 +17,7 @@
#undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_BUILTIN
#undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_EXTENSION
#undef JSON_HEDLEY_CLANG_HAS_FEATURE
#undef JSON_HEDLEY_CLANG_HAS_WARNING
@@ -108,7 +108,10 @@
#undef JSON_HEDLEY_PELLES_VERSION_CHECK
#undef JSON_HEDLEY_PGI_VERSION
#undef JSON_HEDLEY_PGI_VERSION_CHECK
#undef JSON_HEDLEY_PRAGMA
#undef JSON_HEDLEY_PREDICT
#undef JSON_HEDLEY_PREDICT_FALSE
#undef JSON_HEDLEY_PREDICT_TRUE
#undef JSON_HEDLEY_PRINTF_FORMAT
#undef JSON_HEDLEY_PRIVATE
#undef JSON_HEDLEY_PUBLIC
+16 -138
View File
@@ -7932,11 +7932,10 @@ class lexer : public lexer_base<BasicJsonType>
public:
using token_type = typename lexer_base<BasicJsonType>::token_type;
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false) noexcept
: ia(std::move(adapter))
, ignore_comments(ignore_comments_)
, decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
, discard_number_values(discard_number_values_)
{}
// deleted because of pointer members
@@ -9063,58 +9062,6 @@ scan_number_done:
// we are done scanning a number)
unget();
// If the caller does not need the converted value (only whether the
// input is syntactically valid; see json_sax_acceptor/accept()), an
// unsigned/integer token can be reported without calling
// strtoull()/strtoll() at all, *provided* we can already tell from
// the digit count alone that the conversion cannot overflow 64 bits.
// Such tokens are always finite and are accepted unconditionally by
// the parser regardless of their actual value (parser::sax_parse_internal()
// never checks finiteness for value_unsigned/value_integer), so the
// classification below is all that is needed.
//
// A decimal number with up to 18 digits is always representable in
// both std::uint64_t and std::int64_t (18 nines is ~1e18, well below
// both UINT64_MAX ~1.8e19 and INT64_MAX ~9.2e18), so strtoull()/strtoll()
// could not have set errno to ERANGE for it. Numbers with more digits
// (rare in practice) fall through to the exact code below, unchanged,
// so their handling -- including reclassification to value_float when
// the value overflows 64 bits, and rejection when it is not even
// finite as a double -- is bit-for-bit identical to before this
// optimization.
//
// Note this reasons about std::uint64_t/std::int64_t, not about
// number_unsigned_t/number_integer_t (BasicJsonType's own, possibly
// narrower, template parameters -- e.g. std::uint32_t). That is fine
// *only* because discard_number_values is exclusively set by
// accept() (see json.hpp), and accept() always parses through the
// library's own json_sax_acceptor -- never a user-supplied SAX
// consumer -- whose number_unsigned()/number_integer()/number_float()
// callbacks unconditionally discard their argument and return true.
// So for every caller that can reach this branch, neither the token
// classification below nor the eventual (possibly narrowed, and on
// this fast path left stale/unset) value_unsigned/value_integer is
// ever consulted -- an unsigned/integer token is accepted outright,
// and even a >18-digit token that this fast path deliberately falls
// through for is, once reclassified to value_float, still finite
// (and thus accepted) for any digit count that fits in number_unsigned_t
// or number_integer_t regardless of that type's width. If this
// function is ever taught to run with discard_number_values true for
// a caller that *does* read the converted value, this reasoning (and
// the fast path below) would need to be revisited.
if (discard_number_values)
{
constexpr std::size_t safe_digit_count = 18;
if (number_type == token_type::value_unsigned && token_buffer.size() <= safe_digit_count)
{
return token_type::value_unsigned;
}
if (number_type == token_type::value_integer && token_buffer.size() - 1 <= safe_digit_count)
{
return token_type::value_integer;
}
}
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
errno = 0;
@@ -9229,7 +9176,8 @@ scan_number_done:
*/
char_int_type get()
{
advance_position();
++position.chars_read_total;
++position.chars_read_current_line;
if (next_unget)
{
@@ -9241,23 +9189,6 @@ scan_number_done:
current = ia.get_character();
}
return track_after_read();
}
/// shared head of get() / get_ignoring_pending_unget(): bump the
/// per-character position counters (line-count-on-'\n' bookkeeping is
/// handled afterwards, in track_after_read(), once `current` is known)
void advance_position() noexcept
{
++position.chars_read_total;
++position.chars_read_current_line;
}
/// shared tail of get() / get_ignoring_pending_unget(): capture the
/// character for error messages (if needed) and update line/column
/// bookkeeping for the character now in `current`
char_int_type track_after_read()
{
// seekable adapters reconstruct the token lazily on error (see
// get_token_string), so the eager per-character copy is skipped
capture_char(std::integral_constant<bool, lazy_token_string> {});
@@ -9271,29 +9202,6 @@ scan_number_done:
return current;
}
/*!
@brief like get(), but for call sites that can prove no unget() is pending
get() has to check the `next_unget` flag on every call, because a
previous token may have ended with unget() (e.g. scan_number() always
ungets the character that terminated the number, so the next call to
scan() can see it again). skip_whitespace() reads that first,
possibly-ungotten character via a plain get(), but every further
character it reads is guaranteed to be a fresh read: nothing between
those calls invokes unget(). This variant skips the (otherwise always
false) next_unget branch for those calls; it is not a general
replacement for get().
*/
char_int_type get_ignoring_pending_unget()
{
JSON_ASSERT(!next_unget);
advance_position();
current = ia.get_character();
return track_after_read();
}
/// seekable adapter: nothing to capture, the token is rebuilt on error
void capture_char(std::true_type /*lazy*/) const noexcept {}
@@ -9487,37 +9395,13 @@ scan_number_done:
return true;
}
/// whether `current` is one of the four JSON whitespace characters
bool current_is_whitespace() const noexcept
{
return current == ' ' || current == '\t' || current == '\n' || current == '\r';
}
void skip_whitespace()
{
// the first character may be a pending unget() left over from the
// previous token (see get_ignoring_pending_unget()); every
// subsequent character read by this loop is guaranteed fresh, since
// nothing below calls unget()
get();
if (!current_is_whitespace())
{
return;
}
// this is written as an if-guarded do-while (rather than a plain
// while loop) because that shape is what lets both GCC and Clang
// keep the input adapter's read pointer in a register across
// iterations; the equivalent while-loop measurably defeated that
// optimization in testing, turning long whitespace runs (e.g. the
// indentation of pretty-printed JSON) from a register-only loop
// into one that reloads the pointer from memory every character
do
{
get_ignoring_pending_unget();
get();
}
while (current_is_whitespace());
while (current == ' ' || current == '\t' || current == '\n' || current == '\r');
}
token_type scan()
@@ -9653,13 +9537,6 @@ scan_number_done:
const char_int_type decimal_point_char = '.';
/// the position of the decimal point in the input
std::size_t decimal_point_position = std::string::npos;
/// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
/// token classification and never looks at the converted numeric value;
/// when set, scan_number() may skip strtoull()/strtoll() for
/// value_unsigned/value_integer tokens whose digit count guarantees they
/// fit into 64 bits (see scan_number())
const bool discard_number_values = false;
};
} // namespace detail
@@ -14166,10 +14043,9 @@ class parser
parser_callback_t<BasicJsonType> cb = nullptr,
const bool allow_exceptions_ = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas_ = false,
const bool discard_number_values_ = false)
const bool ignore_trailing_commas_ = false)
: callback(std::move(cb))
, m_lexer(std::move(adapter), ignore_comments, discard_number_values_)
, m_lexer(std::move(adapter), ignore_comments)
, allow_exceptions(allow_exceptions_)
, ignore_trailing_commas(ignore_trailing_commas_)
{
@@ -21716,12 +21592,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
detail::parser_callback_t<basic_json>cb = nullptr,
const bool allow_exceptions = true,
const bool ignore_comments = false,
const bool ignore_trailing_commas = false,
const bool discard_number_values = false
const bool ignore_trailing_commas = false
)
{
return ::nlohmann::detail::parser<basic_json, InputAdapterType>(std::move(adapter),
std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas, discard_number_values);
std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
}
private:
@@ -25686,7 +25561,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
}
/// @brief check if the input is valid JSON (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -25697,7 +25572,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
}
JSON_HEDLEY_WARN_UNUSED_RESULT
@@ -25706,7 +25581,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
}
/// @brief generate SAX events
@@ -27090,7 +26965,7 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
#undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_BUILTIN
#undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_EXTENSION
#undef JSON_HEDLEY_CLANG_HAS_FEATURE
#undef JSON_HEDLEY_CLANG_HAS_WARNING
@@ -27181,7 +27056,10 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
#undef JSON_HEDLEY_PELLES_VERSION_CHECK
#undef JSON_HEDLEY_PGI_VERSION
#undef JSON_HEDLEY_PGI_VERSION_CHECK
#undef JSON_HEDLEY_PRAGMA
#undef JSON_HEDLEY_PREDICT
#undef JSON_HEDLEY_PREDICT_FALSE
#undef JSON_HEDLEY_PREDICT_TRUE
#undef JSON_HEDLEY_PRINTF_FORMAT
#undef JSON_HEDLEY_PRIVATE
#undef JSON_HEDLEY_PUBLIC
+53
View File
@@ -125,6 +125,51 @@ json_test_set_test_options(test-unicode4 TEST_PROPERTIES TIMEOUT 3000)
# add unit tests
#############################################################################
# Generate the leak checks for every JSON_HEDLEY_* macro defined in
# hedley.hpp; tests/src/unit-no-macro-leak.cpp #include-s the result after
# nlohmann/json.hpp (see issue #5408). Using the shared
# cmake/scripts/gen_hedley_undef_check.cmake script (also used by `make
# update_hedley_undef`) instead of a hand-maintained list of macro names
# means this test can never go stale after a future `make update_hedley`.
set(hedley_hpp "${PROJECT_SOURCE_DIR}/include/nlohmann/thirdparty/hedley/hedley.hpp")
set(hedley_undef_check_script "${PROJECT_SOURCE_DIR}/cmake/scripts/gen_hedley_undef_check.cmake")
set(hedley_undef_checks "${PROJECT_BINARY_DIR}/include/hedley_undef_checks.inc")
# Reconfigure whenever the vendored header or the generator script changes,
# so a `cmake --build` after `make update_hedley` does not silently keep a
# stale generated file around.
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${hedley_hpp}"
"${hedley_undef_check_script}")
# Generate once at configure time, so the very first build (before any
# custom-command build step has run) already has an up-to-date file.
execute_process(
COMMAND ${CMAKE_COMMAND}
"-DHEDLEY_HPP=${hedley_hpp}"
"-DOUTPUT=${hedley_undef_checks}"
-DMODE=checks
-P "${hedley_undef_check_script}"
RESULT_VARIABLE hedley_undef_check_result
)
if(NOT hedley_undef_check_result EQUAL 0)
message(FATAL_ERROR "Failed to generate ${hedley_undef_checks}")
endif()
# Also (re)generate as a build step, so an incremental build after editing
# hedley.hpp without a full reconfigure still picks up the change.
add_custom_command(
OUTPUT "${hedley_undef_checks}"
COMMAND ${CMAKE_COMMAND}
"-DHEDLEY_HPP=${hedley_hpp}"
"-DOUTPUT=${hedley_undef_checks}"
-DMODE=checks
-P "${hedley_undef_check_script}"
DEPENDS "${hedley_hpp}" "${hedley_undef_check_script}"
COMMENT "Generating Hedley undef leak checks"
VERBATIM)
add_custom_target(generate_hedley_undef_checks DEPENDS "${hedley_undef_checks}")
if("${JSON_TestStandards}" STREQUAL "")
set(test_cxx_standards 11 14 17 20 23)
unset(test_force)
@@ -163,6 +208,14 @@ foreach(file ${files})
json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force})
endforeach()
# tests/src/unit-no-macro-leak.cpp #include-s the generated leak-check file,
# so its test targets must be built after generate_hedley_undef_checks.
foreach(cxx_standard ${test_cxx_standards})
if(TARGET test-no-macro-leak_cpp${cxx_standard})
add_dependencies(test-no-macro-leak_cpp${cxx_standard} generate_hedley_undef_checks)
endif()
endforeach()
if(json_32bit_test_only)
# Skip all other tests in this file
return()
-144
View File
@@ -930,94 +930,6 @@ TEST_CASE("parser class")
CHECK(accept_helper("+1") == false);
CHECK(accept_helper("+0") == false);
}
SECTION("issue #5411 - skip conversion when accept() does not need the numeric value")
{
// lexer::scan_number() may skip strtoull()/strtoll() for
// value_unsigned/value_integer tokens when the caller (e.g.
// json::accept()) does not need the converted value, as long
// as the digit count alone guarantees no 64-bit overflow (see
// the "safe_digit_count" fast path in scan_number()). This
// differential test checks that json::accept() (which enables
// the fast path) and json::parse() (which never does) always
// agree, over a corpus that exercises both the fast path
// (<=18 digits) and the untouched, exact fallback path (>=19
// digits) -- including reclassification of huge digit-only
// integers to a (possibly non-finite) floating-point value.
const std::vector<std::pair<std::string, bool>> cases =
{
// normal small/large integers, both signs
{"0", true}, {"1", true}, {"-1", true}, {"42", true}, {"-42", true},
{"123456789", true}, {"-123456789", true},
// digit-count boundary around the 18-digit safe cutoff (both signs)
{std::string(17, '9'), true},
{std::string(18, '9'), true},
{std::string(19, '9'), true},
{std::string(20, '9'), true},
{"-" + std::string(17, '9'), true},
{"-" + std::string(18, '9'), true},
{"-" + std::string(19, '9'), true},
{"-" + std::string(20, '9'), true},
// 64-bit boundaries
{"9223372036854775807", true}, // INT64_MAX
{"-9223372036854775808", true}, // INT64_MIN
{"18446744073709551615", true}, // UINT64_MAX
{"18446744073709551616", true}, // UINT64_MAX + 1 (overflows uint64_t, finite double)
// the 28-digit example from the issue: overflows uint64_t
// but is finite as a double, so the scanner reclassifies
// it to value_float and it is accepted
{"9999999999999999999999999999", true},
// huge digit-only integers that overflow even a double -> rejected
{std::string(309, '9'), false},
{std::string(400, '9'), false},
{"1" + std::string(400, '0'), false},
// 1e999 / 1e400 style overflow -> rejected
{"1e999", false},
{"1e400", false},
{"-1e999", false},
{"1E999", false},
// values straddling DBL_MAX
{"1.7976931348623157e308", true}, // <= DBL_MAX, finite
{"1.7976931348623159e308", false}, // > DBL_MAX, overflows to inf
// a mix of other valid/invalid numeric syntax
{"3.14159", true},
{"-0.0", true},
{"1.0e10", true},
{"01", false},
{"-", false},
{"1.", false},
{"1e", false},
{"+1", false},
};
for (const auto& c : cases)
{
const std::string& number = c.first;
const bool expected = c.second;
CAPTURE(number)
CAPTURE(expected)
// accept() takes the fast path (skips conversion when possible)
CHECK(json::accept(number) == expected);
// parse() always performs the full conversion; it must agree
json j;
CHECK_NOTHROW(json::parser(nlohmann::detail::input_adapter(number), nullptr, false).parse(true, j));
CHECK(!j.is_discarded() == expected);
// wrap in an array so get_token() is exercised beyond the
// very first (constructor-time) scan as well
const std::string wrapped = "[" + number + "," + number + "]";
CHECK(json::accept(wrapped) == expected);
}
}
}
}
@@ -1482,62 +1394,6 @@ TEST_CASE("parser class")
CHECK(accept_helper("\"\\uD80C\\uFFFF\"") == false);
}
SECTION("issue #5412 - whitespace skipping bookkeeping (compact vs. pretty-printed)")
{
// lexer::skip_whitespace() reads its first character with get() (to
// honor a possibly pending unget() from the previous token) and every
// further whitespace character with get_ignoring_pending_unget() (a
// get() variant that skips the then-always-false next_unget check).
// This must not change the reported byte offset, line, or column of
// a syntax error, even when a long run of whitespace containing
// multiple newlines is skipped beforehand (as with pretty-printed
// input). The expected values below were captured from the
// unmodified do-while(get()) loop, so any regression that miscounts
// characters or newlines while skipping whitespace changes them.
const auto check_error = [](const std::string & input, std::size_t expected_byte,
const std::string & expected_what)
{
CAPTURE(input)
try
{
json _ = json::parse(input);
FAIL_CHECK("expected a parse_error, but parsing succeeded");
}
catch (const json::parse_error& e)
{
CHECK(e.byte == expected_byte);
CHECK(std::string(e.what()) == expected_what);
}
};
// a nested document, serialized both compactly and pretty-printed
// (dump(4)), each truncated right before the final closing '}' so
// that the parser hits EOF after skipping all of the (in the
// pretty-printed case, substantial) indentation whitespace
const json doc =
{
{"a", 1},
{"b", json::array({true, false, nullptr, "x"})},
{"c", json::object({{"d", 3.14}, {"e", json::array({1, 2, 3})}})}
};
const std::string compact = doc.dump();
const std::string pretty = doc.dump(4);
check_error(compact.substr(0, compact.size() - 1), 60,
"[json.exception.parse_error.101] parse error at line 1, column 60: syntax error while parsing object - unexpected end of input; expected '}'");
check_error(pretty.substr(0, pretty.size() - 1), 193,
"[json.exception.parse_error.101] parse error at line 17, column 1: syntax error while parsing object - unexpected end of input; expected '}'");
// an invalid token appearing after several indented, multi-line
// whitespace runs vs. the same document without any of that
// whitespace
check_error("{\n \"a\": 1,\n \"b\": [\n true,\n false\n ],\n \"c\": @\n}", 70,
"[json.exception.parse_error.101] parse error at line 7, column 10: syntax error while parsing value - invalid literal; last read: '\"c\": @'");
check_error("{\"a\":1,\"b\":[true,false],\"c\":@}", 29,
"[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing value - invalid literal; last read: '\"c\":@'");
}
SECTION("tests found by mutate++")
{
// test case to make sure no comma precedes the first key
+34
View File
@@ -0,0 +1,34 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// This file makes sure that none of the internal JSON_HEDLEY_* macros (vendored
// from https://nemequ.github.io/hedley/, see
// include/nlohmann/thirdparty/hedley/hedley.hpp) leak into the including
// translation unit. include/nlohmann/detail/macro_unscope.hpp is supposed to
// #undef every JSON_HEDLEY_* macro (via hedley_undef.hpp) once json.hpp has
// been fully processed. See https://github.com/nlohmann/json/issues/5408,
// where JSON_HEDLEY_PRAGMA, JSON_HEDLEY_PREDICT_TRUE, JSON_HEDLEY_PREDICT_FALSE,
// and JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE escaped this cleanup because
// hedley_undef.hpp had no matching #undef for them.
//
// hedley_undef_checks.inc (included below) is generated at CMake configure/
// build time by cmake/scripts/gen_hedley_undef_check.cmake, which derives the
// full list of JSON_HEDLEY_* macro names directly from hedley.hpp. That way
// this test covers every macro Hedley actually defines -- not a hardcoded
// snapshot that would silently go stale the next time `make update_hedley`
// runs -- and can never drift from the vendored header.
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
TEST_CASE("JSON_HEDLEY macros do not leak after including json.hpp")
{
#include "hedley_undef_checks.inc"
CHECK(true); // keep an assertion when nothing leaked
}