This commit extends the C++20 ranges support (iterator+sentinel pairs) to the
binary format deserializers from_cbor, from_msgpack, from_ubjson, from_bjdata,
and from_bson, matching what was already done for parse(), accept(), and
sax_parse().
Changes:
- Add istreambuf_sentinel helper to test_utils.hpp for EOF detection in tests
- Add 5 new test cases that read binary files directly via
std::istreambuf_iterator<char> + sentinel, without pre-buffering
- Update documentation for all 5 from_* functions to document overload (3)
with SentinelType parameter
- All tests pass; verified against existing test suite data
- Fix potential buffer over-read warning in heterogeneous iterator test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix container input_adapter SFINAE for lvalue-only ADL begin/end (#111)
The container overload of json::parse(c) / accept(c) / sax_parse(c, ...)
silently dropped from overload resolution for user types whose ADL
begin(T&) / end(T&) accepted only non-const lvalue references
(a legitimate pattern matching std::begin semantics). This was because
the detection code used std::declval<ContainerType>() which synthesized
an rvalue, and the rvalue failed to bind to lvalue-only ADL functions.
Fix by making both the outer input_adapter(ContainerType&&) and the
factory's create(ContainerType&&) forwarding references, preserving the
caller's value category and constness via reference collapsing. This
ensures detection (std::declval) and actual use (std::forward) always
match without needing decay/remove_reference.
- Rewrite input_adapters.hpp container overload with forwarding refs
- Add regression tests for lvalue-only non-const ADL begin/end
- Add regression test for rvalue containers (no breakage)
- Update API docs (parse, accept, sax_parse, from_*) to clarify
that begin/end must match std::begin/std::end semantics
- Add version history notes for 3.13.0
- Regenerate amalgamation
Second-order effect: binary_reader.hpp's internal call to
input_adapter(number_vector) now deduces iterator vs const_iterator
based on the lvalue; functionally harmless (iterator_input_adapter is
iterator-type-agnostic), verified via unit-ubjson/unit-bjdata tests.
Closes remaining limitation from #4354 / PR #5218 (todo 106).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid strlen() in test container to fix Codacy CWE-126 flag
Suppressing the strlen()-based CWE-126 warning with NOLINT/nosec
comments only silenced clang-tidy and the standalone Flawfinder
Action; Codacy's own analysis (which also flags this pattern and
doesn't honor those suppression comments) still reported it as a new
issue, plus flagged the near-duplicate begin/end pair as cloned code.
Store the buffer's size explicitly in MyContainerNonConstADL instead
of computing it via strlen() in end(), which removes the flagged
pattern outright and also de-duplicates the struct from the existing
MyContainer's char*-based begin/end pair.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid trailing return type to satisfy clang-tidy fuchsia-trailing-return
The forwarding-reference input_adapter(ContainerType&&) entry point was
written with an auto/trailing-decltype return type, but this project's
ci_clang_tidy job enables the fuchsia-trailing-return check as an
error, which rejects it. The return type only depends on the template
parameter ContainerType, not on the runtime parameter, so it can be
written as an ordinary leading return type instead - no functional
change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid C-style array in test to satisfy clang-tidy avoid-c-arrays
clang-tidy's cppcoreguidelines/hicpp/modernize-avoid-c-arrays checks
flagged the char raw_data[] declaration used to reproduce the
lvalue-only non-const ADL begin/end scenario. Use std::string instead
and take a mutable pointer via &raw_data[0], which is the standard
way to get a non-const char* into a string's buffer under C++11
(std::string::data() only returns non-const in C++17 and later).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix stale Clang -Weverything suppression comments; eliminate -Wno-missing-noreturn
cmake/clang_flags.cmake claimed -Wno-unsafe-buffer-usage was needed only
for Doctest and that -Wno-missing-noreturn had "no way to silence...
otherwise" (PR #4871, which never actually attempted a source fix).
Neither held up under investigation (todo 130):
- -Wno-unsafe-buffer-usage is pervasive (208 distinct sites across 19
files measured with clang trunk in silkeh/clang:dev), spanning the
library's own low-level numeric/buffer code (to_chars, serializer,
lexer, binary reader/writer, input adapters, json_pointer) as well as
vendored Doctest itself (96 of the 208 sites). A source-level fix is
not feasible at this scale; the comment now says so instead of
blaming Doctest alone.
- -Wno-missing-noreturn had exactly two real trigger sites, both
genuinely and unconditionally non-returning: a test-only throwing
allocator (tests/src/unit-allocator.cpp) and, previously undiscovered,
wide_string_input_adapter::get_elements<T>() in
include/nlohmann/detail/input/input_adapters.hpp. Verified this isn't
a wider pattern by checking all 160 JSON_THROW call sites in the
library for functions whose entire body is an unconditional throw.
Annotated both ([[noreturn]] in the test file, since JSON_HEDLEY_NO_RETURN
is #undef'd by the time test code runs; JSON_HEDLEY_NO_RETURN in the
library file, its first real use anywhere in the codebase) and
dropped the suppression entirely.
single_include/nlohmann/json.hpp regenerated via `make amalgamate`;
`make check-amalgamation` passes.
Verified in Docker (silkeh/clang:dev, matching the ci_static_analysis_clang
CI job): baseline builds clean, and the full 194-target test suite builds
with zero warnings under the corrected CLANG_CXXFLAGS (-Wno-missing-noreturn
no longer in the list). Also sanity-compiled and ran unit-allocator.cpp and
unit-wstring.cpp on host Apple Clang to confirm behavior is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C4702 warning caused by JSON_HEDLEY_NO_RETURN on get_elements()
PR #5250 annotated wide_string_input_adapter::get_elements<T>() with
JSON_HEDLEY_NO_RETURN (it unconditionally throws). On MSVC this expands to
__declspec(noreturn), and MSVC correctly determined that the code following
its call in binary_reader.hpp is unreachable for that instantiation, firing
C4702 under /W4 /WX in the msvc, msvc-vs2026, and msvc-arm64 Debug jobs.
Clang doesn't flag this case, so the Docker verification for #5250 (which
only checked Clang -Weverything) didn't catch it.
This is the same warning class already tolerated for Release builds since
PR #5216, where MSVC's optimizer independently found the same dead code
after /Od was removed. Extend that existing /wd4702 suppression to Debug
builds too, instead of reverting the noreturn annotation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
`lexer::get()` copied every scanned character into `token_string` on the
whole successful-parse hot path, yet that buffer is consumed only by
`get_token_string()` when rendering the "last read" fragment of a parse
error. On well-formed input the per-byte copy (plus the `unget()` pop)
is pure overhead that is always discarded.
For seekable input adapters - random-access, single-byte iterators such
as those backing `std::string`, `const char*`, and `std::vector<char>` -
the offending token is now reconstructed on demand from the input when
an error is reported, using a saved start offset, and the eager copy is
skipped. Streaming adapters (file, istream, wide-string, and user-defined
adapters) keep the eager copy; the strategy is chosen at compile time via
`input_adapter_supports_seek`, so adapters without the capability are
unaffected.
Error messages are byte-for-byte identical across all adapters, verified
by a new parity regression test. Microbenchmark (4 MB mixed JSON, parsed
from a std::string): ~149 -> ~160 MB/s, about +8%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(cbor): reject negative ints overflowing int64
CBOR encodes negative integers as "-1 - n" where n is uint64_t. When
n > INT64_MAX, casting to int64_t caused undefined behavior and silent
data corruption. Large negative values were incorrectly parsed as
positive integers (e.g., -9223372036854775809 became 9223372036854775807).
Add bounds check for to reject values that exceed int64_t
representable range, returning parse_error instead of silently
corrupting data.
Added regression test cases to verify.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* chore: clarify tests
Add test for "n=0" case (result=-1) to cover the smallest magnitude
boundary. Update comments to explain CBOR 0x3B encoding and why
"result=0" is not possible. Clarify that n is an unsigned integer
in the formula "result = -1 - n" to help understanding the tests.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* fix(cbor): extend overflow checks for other types
Extend negative integer overflow detection to all CBOR negative
integer cases (0x38, 0x39, 0x3A) for consistency with the existing
0x3B check.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
---------
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Adds pre-multiplication overflow detection to catch cases where dimension
products would exceed size_t max. The previous check only detected when
overflow resulted in exactly 0 or SIZE_MAX, missing other cases.
Retains the original post-multiplication check for backward compatibility.
Adds tests verifying overflow detection with dimensions (2^32+1)×(2^32),
which previously overflowed silently to 2^32.
This prevents custom SAX handlers from receiving incorrect array sizes
that could lead to buffer overflows.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* Add implementation to retrieve start and end positions of json during parse
* Add more unit tests and add start/stop parsing for arrays
* Add raw value for all types
* Add more tests and fix compiler warning
* Amalgamate
* Fix CLang GCC warnings
* Fix error in build
* Style using astyle 3.1
* Fix whitespace changes
* revert
* more whitespace reverts
* Address PR comments
* Fix failing issues
* More whitespace reverts
* Address remaining PR comments
* Address comments
* Switch to using custom base class instead of default basic_json
* Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures
* Address decltype comments
* Diagnostic positions macro (#4)
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Fix missed include deletion
* Add docs and address other PR comments (#5)
* Add docs and address other PR comments
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address new PR comments and fix CI tests for documentation
* Update documentation based on feedback (#6)
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address std::size_t and other comments
* Fix new CI issues
* Fix lcov
* Improve lcov case with update to handle_diagnostic_positions call for discarded values
* Fix indentation of LCOV_EXCL_STOP comments
* fix amalgamation astyle issue
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* multibyte binary reader
* wide_string_input_adapter fallback to get_character
Update input_adapters.hpp
* Update json.hpp
* Add from msgpack test
* Test for broken msgpack with stream, address some warnings
* Reading binary number from wchar as an error, address warnings
* Not casting float to int, it violates strict aliasing rule
* fix: integer parsed as float when EINTR set in errno
* chore: make amalgamate
* chore: make pretty
---------
Co-authored-by: Stuart Gorman <Stuart.Gorman@kallipr.com>
* Possible fix for #4485
Throw's an exception when i is nullptr,
also added a testcase for this scenario though most likely in the wrong test file.cpp
* quick cleanup
* Fix compile issues
* moved tests around, changed exceptions, removed a possibly unneeded include
* add back include <memory> for testing something
* Ninja doesn't like not having a \n, at end of file, adding it back
* update input_adapter file to deal with empty/null file ptr.
* ran make pretty
* added test for inputadapter
* ran make amalgamate
* Update tests/src/unit-deserialization.cpp
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Update tests/src/unit-deserialization.cpp
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Update input adapters.hpp with new includes
* fix unabigious use of _, (there was a double declare)
* did the amalagamate
* rm duplicate includes
* make amalgamate again
* reorder
* amalgamate
* moved it above
* amalgamate
---------
Co-authored-by: Jordan <jordan-hoang@users.noreply.github.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Add versioned inline namespace
Add a versioned inline namespace to prevent ABI issues when linking code
using multiple library versions.
* Add namespace macros
* Encode ABI information in inline namespace
Add _diag suffix to inline namespace if JSON_DIAGNOSTICS is enabled, and
_ldvcmp suffix if JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON is enabled.
* Move ABI-affecting macros into abi_macros.hpp
* Move std_fs namespace definition into std_fs.hpp
* Remove std_fs namespace from unit test
* Format more files in tests directory
* Add unit tests
* Update documentation
* Fix GDB pretty printer
* fixup! Add namespace macros
* Derive ABI prefix from NLOHMANN_JSON_VERSION_*
Niels Lohmann
Ville Vesilehto
David Kilzer
chirsz
Michael Valladolid
Nebojša Cvetković
Sushrut Shringarputale
Tianyi Chen
Stuart Gorman
jh96
Colby Haskell
Colby Haskell
Mathieu Westphal
Raphael Grimm
Dirk Stolle
Florian Albrechtskirchinger