get_ubjson_size_type() takes an inside_ndarray parameter saying whether it is
being called for an ndarray's dimension vector, where another ndarray is not
allowed. It then seeded the flag it passes down to get_ubjson_size_value()
with `false` rather than with that parameter, and only consulted
inside_ndarray afterwards, on the '$' branch.
So on the '#' branch nothing stopped the descent: every "#[" pair of an input
like "[" followed by "#[#[#[..." opened another dimension vector, several
native stack frames deeper each time, and the recursion was only reported on
the way back out. 100,000 pairs crash the process. This is #5104 again, in a
path that has nothing to do with containers.
Seed the flag with inside_ndarray, which is what get_ubjson_size_value()
documents it wants: "for input, `true` means already inside an ndarray vector
or ndarray dimension is not allowed". The nested '[' is then refused where it
is read, so the length of the chain no longer matters.
Both post-checks gain `&& !inside_ndarray`, because an ndarray was found
*here* only if the flag flipped -- get_ubjson_size_value() only ever returns
`true` when its initial value was `false`, as its documentation says. With
that, the "ndarray can not be recursive" branch is unreachable: a recursive
ndarray is now caught one level earlier, and reported as "ndarray dimensional
vector is not allowed" like every other nested dimension vector.
Three existing expectations move accordingly (vR2, vR4, vR6). All three now
fail earlier, and all three now report the same error that vR1, vR5 and vH
already reported for the same shape, which is the more consistent outcome.
Everything else is unchanged: valid 1D and 2D ndarrays, optimized containers
and plain arrays produce identical results, and unit-ubjson is untouched.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_cbor_string() and get_cbor_binary() handled the indefinite-length forms
(0x7F and 0x5F) by calling themselves once per chunk. Each chunk therefore
cost a native stack frame, and since a chunk may itself be an indefinite-
length string, an input of repeated 0x7F bytes reached one frame per input
byte: 200,000 of them crash the process with SIGSEGV before a single byte is
rejected. This is the same defect as #5104, in a path the container-level
work does not touch.
Count the open levels instead of recursing through them. That is enough here
because every chunk is appended to the same result -- get_bytes() writes at
result.size() -- so there is no per-level state to keep. The temporary chunk
string and its copy into the result go away with the recursion.
The definite-length cases move to get_cbor_string_chunk() and
get_cbor_binary_chunk() unchanged, including their error messages, which
still name 0x7F and 0x5F because those are handled one level up.
Behaviour is unchanged. Comparing against develop over the interesting byte
sequences -- empty, single-chunk, nested, over-closed and truncated forms,
both strings and byte arrays, and an indefinite-length map key -- produces
identical values, error codes, messages and byte offsets. The 200,000-level
input now reports parse_error.110 at byte 200001 instead of crashing.
Note that nesting these is not valid CBOR: RFC 8949, Section 3.2.3 forbids
it. This does not change that either way -- it has always been accepted, and
rejecting it is a separate decision (#5317, #5325). Should it be rejected
later, that is now one condition on the level counter rather than a change to
the control flow.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CBOR tag handlers not recognizing tags 0-5 and 21-23
The tagged-item switch in binary_reader::parse_cbor_internal() only handled
head bytes 0xC6-0xD4 and 0xD8-0xDB. Bytes 0xC0-0xC5 (tags 0-5: date/time,
epoch, bignum, decimal, bigfloat) and 0xD5-0xD7 (tags 21-23: base64url,
base64, base16 conversion hints) fell through to the default case and were
reported as invalid bytes, even under cbor_tag_handler_t::ignore and ::store,
despite being valid CBOR major-type-6 tags per RFC 8949.
Add the missing case labels so the full 0xC0-0xDB range is handled
uniformly. Extend the "Tagged values" test in unit-cbor.cpp to cover
0xC0-0xD7, and update the CBOR docs to state the corrected tag range.
Fixes#5315
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
* Fix stale CBOR tag docs and add store-mode binary-payload test
The "Incomplete mapping" warning still listed tags 0-5 (date/time,
bignum, decimal fraction, bigfloat) and 21-23 (expected conversions)
as unsupported, even though they now parse correctly under
cbor_tag_handler_t::ignore/store, same as 0xC6..0xD4/0xD8..0xDB.
Remove those five bullets and cross-reference the "Tagged items"
warning below, matching the equivalent docs fix landed independently
in PR #5367.
Also add a cbor_tag_handler_t::store test that wraps a binary
payload (not just a string) for every byte in 0xC0..0xD7, confirming
these tags are unwrapped the same way as 0xC6..0xD4 rather than
mistaken for the 0xD8..0xDB binary-subtype marker syntax, per review
feedback on #5331.
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
---------
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
The block documenting get_char and tag_handler sat above
get_cbor_negative_integer(), which takes neither, so Doxygen attached it
there and parse_cbor_internal() was left undocumented.
Comment placement only.
Signed-off-by: Dmitry <45711841+darkdi@users.noreply.github.com>
The comment asked whether the no-op marker 'N' may be ignored when a
string is read. It may not: at that point the next byte must be a string
length type specification, and 'N' is not one. No-ops at positions where
a value may start are already consumed by the callers through
get_ignore_noop(), so nothing is lost by not skipping them here.
Replace the TODO with a comment stating that, and add regression tests
pinning both directions: a no-op is accepted at top level (also
repeated), before and after an array element, and before an object key,
between key and value, and before the closing brace of an object of
unknown size; it is rejected where a length type specification is
expected, i.e. after the 'S' marker of a string value and as the key
length of an object of known size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* reject CBOR array/map length equal to the indefinite-length marker
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
* reject CBOR lengths that do not fit in std::size_t via value_in_range_of
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
---------
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
This adds a std::isfinite check to the UBJSON floating-point parsing path, throwing out_of_range.406 on overflow. This makes the UBJSON parser's behavior consistent with the normal JSON parser. Fixes#5322.
Signed-off-by: AJ369ninja <abhishek.j@iitg.ac.in>
Co-authored-by: AJ369ninja <abhishek.j@iitg.ac.in>
* fix(cbor): reject negative ints overflowing int64
CBOR encodes negative integers as "-1 - n" where n is uint64_t. When
n > INT64_MAX, casting to int64_t caused undefined behavior and silent
data corruption. Large negative values were incorrectly parsed as
positive integers (e.g., -9223372036854775809 became 9223372036854775807).
Add bounds check for to reject values that exceed int64_t
representable range, returning parse_error instead of silently
corrupting data.
Added regression test cases to verify.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* chore: clarify tests
Add test for "n=0" case (result=-1) to cover the smallest magnitude
boundary. Update comments to explain CBOR 0x3B encoding and why
"result=0" is not possible. Clarify that n is an unsigned integer
in the formula "result = -1 - n" to help understanding the tests.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* fix(cbor): extend overflow checks for other types
Extend negative integer overflow detection to all CBOR negative
integer cases (0x38, 0x39, 0x3A) for consistency with the existing
0x3B check.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
---------
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Adds pre-multiplication overflow detection to catch cases where dimension
products would exceed size_t max. The previous check only detected when
overflow resulted in exactly 0 or SIZE_MAX, missing other cases.
Retains the original post-multiplication check for backward compatibility.
Adds tests verifying overflow detection with dimensions (2^32+1)×(2^32),
which previously overflowed silently to 2^32.
This prevents custom SAX handlers from receiving incorrect array sizes
that could lead to buffer overflows.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* Add implementation to retrieve start and end positions of json during parse
* Add more unit tests and add start/stop parsing for arrays
* Add raw value for all types
* Add more tests and fix compiler warning
* Amalgamate
* Fix CLang GCC warnings
* Fix error in build
* Style using astyle 3.1
* Fix whitespace changes
* revert
* more whitespace reverts
* Address PR comments
* Fix failing issues
* More whitespace reverts
* Address remaining PR comments
* Address comments
* Switch to using custom base class instead of default basic_json
* Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures
* Address decltype comments
* Diagnostic positions macro (#4)
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Fix missed include deletion
* Add docs and address other PR comments (#5)
* Add docs and address other PR comments
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address new PR comments and fix CI tests for documentation
* Update documentation based on feedback (#6)
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address std::size_t and other comments
* Fix new CI issues
* Fix lcov
* Improve lcov case with update to handle_diagnostic_positions call for discarded values
* Fix indentation of LCOV_EXCL_STOP comments
* fix amalgamation astyle issue
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* multibyte binary reader
* wide_string_input_adapter fallback to get_character
Update input_adapters.hpp
* Update json.hpp
* Add from msgpack test
* Test for broken msgpack with stream, address some warnings
* Reading binary number from wchar as an error, address warnings
* Not casting float to int, it violates strict aliasing rule
* Add versioned inline namespace
Add a versioned inline namespace to prevent ABI issues when linking code
using multiple library versions.
* Add namespace macros
* Encode ABI information in inline namespace
Add _diag suffix to inline namespace if JSON_DIAGNOSTICS is enabled, and
_ldvcmp suffix if JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON is enabled.
* Move ABI-affecting macros into abi_macros.hpp
* Move std_fs namespace definition into std_fs.hpp
* Remove std_fs namespace from unit test
* Format more files in tests directory
* Add unit tests
* Update documentation
* Fix GDB pretty printer
* fixup! Add namespace macros
* Derive ABI prefix from NLOHMANN_JSON_VERSION_*
* BJData dimension length can not be string_t::npos, fix#3541
* handle error messages on 32bit machine
* add explanation to why size can not be string_t::npos
* add test cases to 32bit unit test
Co-authored-by: Florian Albrechtskirchinger <falbrechtskirchinger@gmail.com>
* Fix ndarray dimension signness, fix ndarray length overflow, close#3519
* detect size overflow in ubjson and bjdata
* force reformatting
* Fix MSVC compiler warning
* Add value_in_range_of trait
* Use value_in_range_of trait
* Correct 408 parse_errors to out_of_range
* Add 32bit unit test
The test can be enabled by setting JSON_32bitTest=ON.
* Exclude unreachable lines from coverage
Certain lines are unreachable in 64bit builds.
Co-authored-by: Qianqian Fang <fangqq@gmail.com>
* Discard optimized containers with negative counts in UBJSON/BJData (#3491,#3492,#3490)
* fix msvc error
* update unit tests for negative sized containers
* use a loop to test 0 ndarray dimension
* throw an error when count is negative, merge CHECK_THROW_AS and _WITH with _WITH_AS
* change bjdata ndarray flag to detect negative size, fix https://github.com/nlohmann/json/issues/3475
* fix CI error
* fix CI on 32bit windows
* remove platform specific out_of_range error messages
* Incorporate suggestions from @nlohmann and @falbrechtskirchinger
* fix CI errors
* add coverage
* fix sax event order
* fix coverage
Niels Lohmann
Sahil_Kamate
Dmitry
tomatotomata
Angadi56
hum4nBeingandAJ369ninja
Yash Bavadiya
Patrick10199
Ville Vesilehto
Michael Valladolid
Nebojša Cvetković
Sushrut ShringarputaleandSush Shringarputale
Tianyi Chen
Colby Haskell
Dirk Stolle
Qianqian FangandFlorian Albrechtskirchinger