Flawfinder flags the bare identifier `read` as the POSIX read() syscall
(CWE-120/CWE-20), producing several false-positive code-scanning alerts.
Renaming the local variable removes them; the byte-copy itself is
unchanged and remains bounds-checked (get_elements caps the copy to the
bytes available). This is a pure rename with identical semantics.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- get_bytes(): replace `len -= static_cast<NumberType>(wanted)` with an
explicit `len = static_cast<NumberType>(len - ...)`. The compound
assignment promoted the operands to int and narrowed back, which GCC
rejects under -Werror=arith-conversion when NumberType is narrower than
int (e.g. UBJSON's int16 length). This also fixes the CodeQL build.
- get_bytes(): add JSON_ASSERT(read == wanted) documenting that
get_elements() never returns more than requested (per review).
- iterator_input_adapter: de-duplicate the iterator_is_contiguous
definition with the #if inside the initializer (per review).
- Add a comment explaining why the memcpy source is &*current (needed for
non-pointer contiguous iterators), and drop the redundant parentheses.
- Run astyle on the new unit test so the amalgamation/format check passes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The binary reader read CBOR/MessagePack/BSON/UBJSON strings and byte
arrays one byte at a time via get()/push_back(), and the iterator input
adapter's get_elements() fallback was itself a per-byte loop, so even
contiguous inputs never benefited from a block copy.
Two changes:
1. iterator_input_adapter::get_elements() gains a contiguous fast path
that copies the whole requested range with std::memcpy. Contiguity is
detected via std::is_pointer (all standards) and, in C++20,
std::contiguous_iterator (so std::vector/std::string iterators also
qualify). Non-contiguous iterators keep the element-by-element loop.
2. get_string()/get_binary() now share get_bytes(), which reads into the
result in bounded chunks through get_elements() instead of byte by
byte. Capping the chunk size preserves the deliberate "do not
reserve(len) for an untrusted length" DoS protection while turning the
inner loop into block copies. The min(chunk_size, len) computation is
width-safe so narrow length types (e.g. MessagePack ext-8's uint8_t)
cannot truncate chunk_size to zero.
Microbenchmark (2 MiB string + 2 MiB blob + 2000x1 KiB strings, Apple
clang, -O2):
C++20 from_cbor(vector) 20.1 ms -> 1.0 ms (~20x)
from_cbor(pointer) 18.9 ms -> 1.0 ms (~19x)
C++17 from_cbor(pointer) 18.7 ms -> 1.0 ms (~19x, memcpy)
from_cbor(vector) 18.7 ms -> 4.0 ms (~4.6x, tight loop)
Behavior is unchanged: truncated input still throws parse_error.110 at
the same byte offset, and all binary-format unit tests pass.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(cbor): reject negative ints overflowing int64
CBOR encodes negative integers as "-1 - n" where n is uint64_t. When
n > INT64_MAX, casting to int64_t caused undefined behavior and silent
data corruption. Large negative values were incorrectly parsed as
positive integers (e.g., -9223372036854775809 became 9223372036854775807).
Add bounds check for to reject values that exceed int64_t
representable range, returning parse_error instead of silently
corrupting data.
Added regression test cases to verify.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* chore: clarify tests
Add test for "n=0" case (result=-1) to cover the smallest magnitude
boundary. Update comments to explain CBOR 0x3B encoding and why
"result=0" is not possible. Clarify that n is an unsigned integer
in the formula "result = -1 - n" to help understanding the tests.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* fix(cbor): extend overflow checks for other types
Extend negative integer overflow detection to all CBOR negative
integer cases (0x38, 0x39, 0x3A) for consistency with the existing
0x3B check.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
---------
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Adds pre-multiplication overflow detection to catch cases where dimension
products would exceed size_t max. The previous check only detected when
overflow resulted in exactly 0 or SIZE_MAX, missing other cases.
Retains the original post-multiplication check for backward compatibility.
Adds tests verifying overflow detection with dimensions (2^32+1)×(2^32),
which previously overflowed silently to 2^32.
This prevents custom SAX handlers from receiving incorrect array sizes
that could lead to buffer overflows.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* Add implementation to retrieve start and end positions of json during parse
* Add more unit tests and add start/stop parsing for arrays
* Add raw value for all types
* Add more tests and fix compiler warning
* Amalgamate
* Fix CLang GCC warnings
* Fix error in build
* Style using astyle 3.1
* Fix whitespace changes
* revert
* more whitespace reverts
* Address PR comments
* Fix failing issues
* More whitespace reverts
* Address remaining PR comments
* Address comments
* Switch to using custom base class instead of default basic_json
* Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures
* Address decltype comments
* Diagnostic positions macro (#4)
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Fix missed include deletion
* Add docs and address other PR comments (#5)
* Add docs and address other PR comments
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address new PR comments and fix CI tests for documentation
* Update documentation based on feedback (#6)
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address std::size_t and other comments
* Fix new CI issues
* Fix lcov
* Improve lcov case with update to handle_diagnostic_positions call for discarded values
* Fix indentation of LCOV_EXCL_STOP comments
* fix amalgamation astyle issue
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* multibyte binary reader
* wide_string_input_adapter fallback to get_character
Update input_adapters.hpp
* Update json.hpp
* Add from msgpack test
* Test for broken msgpack with stream, address some warnings
* Reading binary number from wchar as an error, address warnings
* Not casting float to int, it violates strict aliasing rule
* Add versioned inline namespace
Add a versioned inline namespace to prevent ABI issues when linking code
using multiple library versions.
* Add namespace macros
* Encode ABI information in inline namespace
Add _diag suffix to inline namespace if JSON_DIAGNOSTICS is enabled, and
_ldvcmp suffix if JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON is enabled.
* Move ABI-affecting macros into abi_macros.hpp
* Move std_fs namespace definition into std_fs.hpp
* Remove std_fs namespace from unit test
* Format more files in tests directory
* Add unit tests
* Update documentation
* Fix GDB pretty printer
* fixup! Add namespace macros
* Derive ABI prefix from NLOHMANN_JSON_VERSION_*
* BJData dimension length can not be string_t::npos, fix#3541
* handle error messages on 32bit machine
* add explanation to why size can not be string_t::npos
* add test cases to 32bit unit test
Co-authored-by: Florian Albrechtskirchinger <falbrechtskirchinger@gmail.com>
* Fix ndarray dimension signness, fix ndarray length overflow, close#3519
* detect size overflow in ubjson and bjdata
* force reformatting
* Fix MSVC compiler warning
* Add value_in_range_of trait
* Use value_in_range_of trait
* Correct 408 parse_errors to out_of_range
* Add 32bit unit test
The test can be enabled by setting JSON_32bitTest=ON.
* Exclude unreachable lines from coverage
Certain lines are unreachable in 64bit builds.
Co-authored-by: Qianqian Fang <fangqq@gmail.com>
* Discard optimized containers with negative counts in UBJSON/BJData (#3491,#3492,#3490)
* fix msvc error
* update unit tests for negative sized containers
* use a loop to test 0 ndarray dimension
* throw an error when count is negative, merge CHECK_THROW_AS and _WITH with _WITH_AS
* change bjdata ndarray flag to detect negative size, fix https://github.com/nlohmann/json/issues/3475
* fix CI error
* fix CI on 32bit windows
* remove platform specific out_of_range error messages
* Incorporate suggestions from @nlohmann and @falbrechtskirchinger
* fix CI errors
* add coverage
* fix sax event order
* fix coverage
* support UBJSON-derived Binary JData (BJData) format
* fix Codacy warning
* partially fix VS compilation errors
* fix additional VS errors
* fix more VS compilation errors
* fix additional warnings and errors for clang and msvc
* add more tests to cover the new bjdata types
* add tests for optimized ndarray, improve coverage, fix clang/gcc warnings
* gcc warn useless conversion but msvc gives an error
* fix ci_test errors
* complete test coverage, fix ci_test errors
* add half precision error test
* fix No newline at end of file error by clang
* simplify endian condition, format unit-bjdata
* remove broken test due to alloc limit
* full coverage, I hope
* move bjdata new markers from default to the same level as ubjson markers
* fix ci errors, add tests for new bjdata switch structure
* make is_bjdata const after using initializer list
* remove the unwanted assert
* move is_bjdata to an optional param to write_ubjson
* pass use_bjdata via output adapter
* revert order to avoid msvc 2015 unreferenced formal param error
* update BJData Spect V1 Draft-2 URL after spec release
* amalgamate code
* code polishing following @gregmarr's feedback
* make use_bjdata a non-default parameter
* fix ci error, remove unwanted param comment
* encode and decode bjdata ndarray in jdata annotations, enable roundtrip tests
* partially fix ci errors, add tests to improve coverage
* polish patch to remove ci errors
* fix a ndarray dim vector condition
* fix clang tidy error
* add sax test cases for ndarray
* add additional sax event tests
* adjust sax event numbering
* fix sax tests
* ndarray can only be used with array containers, discard if used in object
* complete test coverage
* disable [{SHTFNZ in optimized type due to security risks in #2793 and hampered readability
* fix ci error
* move OutputIsLittleEndian from tparam to param to replace use_bjdata
* fix ci clang gcc error
* fix ci static analysis error
* update json_test_data to 3.1.0, enable file-based bjdata unit tests
* fix stack overflow error on msvc 2019 and 2022
* use https link, update sax_parse_error after rebase
* make input_format const and use initializer
* return bool for write_bjdata_ndarray
* test write_bjdata_ndarray return value as boolean
* fix ci error
* Make exception context optional
Change exception context parameter to pointer and replace context with
nullptr where appropriate.
* Support escaping other string types
* Add string concatenation function
Add variadic concat() function for concatenating char *, char, and
string types.
* Replace string concatenations using + with concat()
* Template json_pointer on string type
Change json_pointer from being templated on basic_json to being
templated on string type.
* Add unit test for #3388Closes#3388.
* Fix regression test for #2958
* Add backwards compatibility with json_pointer<basic_json>
* Update json_pointer docs
* Allow comparing different json_pointers
* Update version numbers
Niels LohmannandClaude Opus 4.8
Ville Vesilehto
Michael Valladolid
Nebojša Cvetković
Sushrut ShringarputaleandSush Shringarputale
Tianyi Chen
Colby Haskell
Dirk Stolle
Qianqian FangandFlorian Albrechtskirchinger
Niels Lohmann
Alex Beregszaszi
Ferry Huberts