Compare commits

..
Author SHA1 Message Date
Niels Lohmann 1cba195509 Read UBJSON and BJData containers without recursing per nesting level
get_ubjson_array() and get_ubjson_object() read their elements by calling
back into the value reader, which called them again for a nested container,
so the native call stack grew with the nesting depth of the input. '[' alone
opens a container, so half a million of them crashes the process before the
input runs out (#5104). The optimized forms reach the same path through a
size or type annotation, and in plain UBJSON '[' and '{' are permitted as the
type of an optimized container, so "[$[#i\x01" repeated nests just as deeply
at six bytes a level.

Both readers now only open their container, and parse_ubjson_internal() loops:
it closes the containers that have ended, claims the next element of the
innermost one, reads its key when it is an object, and works out the marker
of the value to read next. That last part is where the formats differ, and
the loop follows what the four element loops used to do:

  - a sized, typed container gives its elements no marker of their own
  - a sized, untyped container reads one for each element
  - a container that ends at a marker has the byte already, from the test
    against ']' or '}'; for an object it is the first byte of the key

The ND-array wrapper and the 'B' binary shortcut stay as they are. Both read
a complete value rather than opening a container, and their elements are
always scalars: BJData does not permit '[' or '{' as an optimized type, which
is also why only plain UBJSON needed the type-marker case above.

A container of no-ops keeps its behaviour of holding no elements while still
announcing its declared size to the SAX parser, by opening it and then
setting its count to zero.

unit-ubjson and unit-bjdata pass unchanged, 1.39 million assertions between
them, and a behaviour comparison against the previous commit over every
container form -- sized, unsized, typed, untyped, empty, no-op, ND-array,
binary, and the forms nested inside one another -- gives identical values,
error codes, messages and byte offsets. 500,000 levels of each vector now
report a parse error instead of crashing, and a well-formed 100,000-level
value is read to completion.

The driver costs about 3 % on parsing 60,000 small objects and one array of a
million integers, for the reason given in the previous commit; reading the
frame once per element rather than per branch halved what it cost before.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 15:45:43 +02:00
Niels Lohmann 1df654d19d Read CBOR containers and tags without recursing per nesting level
get_cbor_array() and get_cbor_object() read their elements by calling back
into the value reader, which called them again for a nested container, and a
tag was handled by reading the tagged value the same way. All three cost
native stack, and all three cost a single byte to encode: 0x9F opens an
indefinite-length array, 0x81 a one-element array, and 0xC2 is a tag. Half a
million of any of them crashes the process before the input runs out (#5104).

Apply the shape the MessagePack reader already uses: the open containers live
on the heap stack, parse_cbor_value() reads a single value and only opens a
container rather than reading it to its end, and parse_cbor_internal() loops,
resuming the innermost container after each element.

Two things are specific to CBOR. An indefinite-length container ends at a
break marker rather than at a count, and testing for that marker consumes a
byte which is the first byte of the next element when it is not one; the
frame's count is npos for those, and the driver tracks whether the next value
starts at a fresh byte. And a tag is not a value of its own: instead of
reading the tagged value by recursing, the value reader reports that a tag was
read and the driver reads on, so a chain of tags costs no stack at all.

The switch that decodes a value is unchanged apart from the twelve container
cases and the two tag sites. Verified against the previous commit over
definite and indefinite arrays and maps, all four counted forms, empty
containers, nesting of the forms inside each other, truncated inputs, and all
three tag handlers: identical values, error codes, messages and byte offsets.
500,000 levels of each of the three vectors now report parse_error.110 instead
of crashing, and a well-formed 200,000-level value is read to completion.

On performance: the driver does per element what a counted loop used to do
per container, and CBOR pays for it more than MessagePack because the value
reader also has to be told whether to fetch a byte. Parsing 60,000 small
objects and one array of a million integers is 3 to 4 % slower than the
recursive reader, measured over five alternating runs. Against develop the
same two inputs are about 44 % faster, because the entry point no longer
copies the value it parsed; the earlier commit in this series is what pays
for that.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 15:45:33 +02:00
Niels Lohmann 90bd0f7fec Read MessagePack containers without recursing per nesting level
get_msgpack_array() and get_msgpack_object() read their elements by calling
back into parse_msgpack_internal(), which calls them again for a nested
container. The native call stack therefore grew with the nesting depth of the
input, and each level costs only one byte to encode: 0x91 is a one-element
array, so a few hundred thousand of them crash the process before any of the
input is rejected (#5104).

Keep the open containers on a heap stack instead, the way
parser::sax_parse_internal() has always done for JSON text. A frame records
how many elements are left and whether to close with end_object() or
end_array(); parse_msgpack_value() reads a single value and, for a container,
only opens it; and parse_msgpack_internal() loops, resuming the innermost
container after each element and closing it when its count runs out. Whether
the value that was begun is complete is answered by the stack being empty, so
no separate bookkeeping is needed.

The switch that decodes a value is untouched apart from the six container
cases, which now call enter_container() rather than a reader that loops. That
keeps this diff to the control flow and leaves the decoding of every other
type byte-identical.

enter_container() is the only place a binary reader emits start_object() or
start_array(), so a check that rejects a container can be added there once and
is guaranteed to run before the start event. The frame type and the stack are
shared, ready for the other three formats.

Verified against develop over empty, nested, counted (array 16/32, map 16/32)
and truncated inputs: identical values, error codes, messages and byte
offsets. 300,000 levels now report parse_error.110 instead of crashing, and a
well-formed 300,000-level value is read to completion through the SAX
interface, where develop crashes.

Reading such a value into a basic_json needs the return-by-move change as
well, without which the recursive copy constructor overflows on the way out;
that is the parent commit, and the test for the value path covers the two
together. Timing is unchanged: parsing 60,000 small objects and one array of
a million integers is within run-to-run noise of develop either way.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 14:08:26 +02:00
Niels Lohmann 3d89ac50a0 Split unit-regression2.cpp so the MinGW linker can relocate it
Linking test-regression2 with clang and MinGW fails with

    relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'

once the translation unit grows past a certain size: the code can no longer
reach the read-only data it references within the range of a 32-bit
relocation. The file is one of the largest in the test suite and had been
sitting just under that limit, so an unrelated change elsewhere in the
library is enough to tip it over. It is already the second such file --
unit-regression1.cpp was split for size before -- and windows.yml already
carries a workaround for the same limit hitting the debug sections of this
same target, where -g0 was enough because that relocation was against
`.debug_line'. This one is against `.rdata', which no compiler flag avoids.

Move the second half of the regression tests, and the helper types only they
use, into unit-regression3.cpp. The sections are independent -- every
statement in "regression tests 2" was already inside a SECTION -- so they
move unchanged, and the counts confirm nothing was lost: 168 assertions
before the split, 50 plus 118 after.

The result is that both files are comfortably smaller than the one that used
to link, measured with clang at -O1 for C++20:

                        read-only data        text     object
    before                      58,233   1,287,764  3,158,120
    unit-regression2.cpp        48,161   1,012,988  2,522,296
    unit-regression3.cpp        41,710     772,704  1,878,880

No CMake change is needed: tests/CMakeLists.txt globs src/unit-*.cpp, so the
new file is picked up and built for every standard like its siblings.

CONTRIBUTING.md pointed contributors at unit-regression2.cpp for new bug
tests; it now points at the smaller file and says why the two exist, so the
split does not quietly undo itself.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 14:06:58 +02:00
Niels Lohmann ee69490b28 Bound UBJSON optimized arrays of a valueless type
An element of type 'Z' (null), 'T' (true) or 'F' (false) is encoded by its
type marker alone, so an optimized UBJSON array of one of those has no
payload: reading an element consumes no input at all. Its declared count is
therefore the only thing that decides how much is allocated, and nothing
bounded it. "[$Z#l" and a four-byte count is nine bytes of input describing
two billion values; #2793 reports 35 GB and 150 seconds from ten bytes, and
OSS-Fuzz has an out-of-memory and a timeout report for the same shape.

Every other type costs at least one byte per element, so the end of the input
bounds it. 'N' (no-op) is already skipped rather than stored. Objects are not
affected either: each element is preceded by its key, which costs bytes. And
BJData already refuses these markers as an optimized type, so this is a plain
UBJSON matter.

Reject a count above 1,048,576 elements for those three types with
out_of_range.408, the code this reader already uses for a declared size it
will not honour. The check runs before the SAX start event, so no container
is opened and then abandoned.

Rejecting on the read side alone would break the guarantee that anything
to_ubjson() writes can be read back, and would trip the round-trip assertion
in fuzzer-parse_ubjson.cpp. So the writer falls back to the unoptimized
encoding, one byte per element, for arrays of these types above the same
limit. Its decision depends only on the array's size, which is identical for
a value and for anything parsed back from it, so the round trip is stable.

No existing test changes: the largest such count in the test suite is 65,793.
The excessive-size test that already used this shape still passes, now
rejected a little earlier than by the max_size() check it used to reach.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 07:50:29 +02:00
Niels Lohmann 3970ecccd1 Reject a nested BJData ndarray dimension vector where it is read
get_ubjson_size_type() takes an inside_ndarray parameter saying whether it is
being called for an ndarray's dimension vector, where another ndarray is not
allowed. It then seeded the flag it passes down to get_ubjson_size_value()
with `false` rather than with that parameter, and only consulted
inside_ndarray afterwards, on the '$' branch.

So on the '#' branch nothing stopped the descent: every "#[" pair of an input
like "[" followed by "#[#[#[..." opened another dimension vector, several
native stack frames deeper each time, and the recursion was only reported on
the way back out. 100,000 pairs crash the process. This is #5104 again, in a
path that has nothing to do with containers.

Seed the flag with inside_ndarray, which is what get_ubjson_size_value()
documents it wants: "for input, `true` means already inside an ndarray vector
or ndarray dimension is not allowed". The nested '[' is then refused where it
is read, so the length of the chain no longer matters.

Both post-checks gain `&& !inside_ndarray`, because an ndarray was found
*here* only if the flag flipped -- get_ubjson_size_value() only ever returns
`true` when its initial value was `false`, as its documentation says. With
that, the "ndarray can not be recursive" branch is unreachable: a recursive
ndarray is now caught one level earlier, and reported as "ndarray dimensional
vector is not allowed" like every other nested dimension vector.

Three existing expectations move accordingly (vR2, vR4, vR6). All three now
fail earlier, and all three now report the same error that vR1, vR5 and vH
already reported for the same shape, which is the more consistent outcome.
Everything else is unchanged: valid 1D and 2D ndarrays, optimized containers
and plain arrays produce identical results, and unit-ubjson is untouched.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 07:50:28 +02:00
Niels Lohmann 00e5d21041 Stop CBOR indefinite-length strings from recursing per chunk
get_cbor_string() and get_cbor_binary() handled the indefinite-length forms
(0x7F and 0x5F) by calling themselves once per chunk. Each chunk therefore
cost a native stack frame, and since a chunk may itself be an indefinite-
length string, an input of repeated 0x7F bytes reached one frame per input
byte: 200,000 of them crash the process with SIGSEGV before a single byte is
rejected. This is the same defect as #5104, in a path the container-level
work does not touch.

Count the open levels instead of recursing through them. That is enough here
because every chunk is appended to the same result -- get_bytes() writes at
result.size() -- so there is no per-level state to keep. The temporary chunk
string and its copy into the result go away with the recursion.

The definite-length cases move to get_cbor_string_chunk() and
get_cbor_binary_chunk() unchanged, including their error messages, which
still name 0x7F and 0x5F because those are handled one level up.

Behaviour is unchanged. Comparing against develop over the interesting byte
sequences -- empty, single-chunk, nested, over-closed and truncated forms,
both strings and byte arrays, and an indefinite-length map key -- produces
identical values, error codes, messages and byte offsets. The 200,000-level
input now reports parse_error.110 at byte 200001 instead of crashing.

Note that nesting these is not valid CBOR: RFC 8949, Section 3.2.3 forbids
it. This does not change that either way -- it has always been accepted, and
rejecting it is a separate decision (#5317, #5325). Should it be rejected
later, that is now one condition on the level counter rather than a change to
the control flow.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 07:50:27 +02:00
Niels Lohmann 755c547003 Return the parsed value by move from from_cbor() and friends
The binary entry points end with

    return res ? result : basic_json(value_t::discarded);

The condition operator's second operand is an lvalue, so this is not a case
where the return value can be elided or implicitly moved from: every
successful from_cbor(), from_msgpack(), from_ubjson(), from_bjdata() and
from_bson() call deep-copies the value it just parsed, and then destroys the
original.

The copy is not cheap, and it is not incidental: basic_json's copy
constructor walks the whole value. Parsing a 2 MB CBOR document with 60,000
objects, median of 25 runs, clang 17 -O3:

    from_cbor      26.99 ms  ->  14.65 ms
    from_msgpack   26.82 ms  ->  14.82 ms

Moving instead of copying is the entire change; the parsed value is not used
again after the return expression is evaluated.

There is a second reason to prefer the move. The copy constructor recurses
once per nesting level, so the copy is also a stack-overflow path on the
return side, on a value the reader has already accepted. That is currently
masked because the readers themselves recurse and overflow first (#5104), but
it has to be fixed for making them iterative to have any effect.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-07 07:50:09 +02:00
18 changed files with 2595 additions and 2118 deletions
+3 -1
View File
@@ -108,7 +108,9 @@ The tests are located in [`tests/src/unit-*.cpp`](https://github.com/nlohmann/js
are structured along the features of the library or the nature of the tests. Usually, it should be clear from the
context which existing file needs to be extended, and only very few cases require creating new test files.
When fixing a bug, edit `unit-regression2.cpp` and add a section referencing the fixed issue.
When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue.
`unit-regression2.cpp` holds the older tests; the two files exist because a single one grew large enough for the
MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing one of them.
#### Exceptions
@@ -88,8 +88,6 @@ Strong exception safety: if an exception occurs, the original value stays intact
do not belong to the same JSON value; example: `"iterators do not fit"`
- Throws [`invalid_iterator.211`](../../home/exceptions.md#jsonexceptioninvalid_iterator211) if `first` or `last`
are iterators into container for which insert is called; example: `"passed iterators may not belong to container"`
- Throws [`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) if `first` or `last`
do not point to an array; example: `"iterators first and last must point to arrays"`
4. The function can throw the following exceptions:
- Throws [`type_error.309`](../../home/exceptions.md#jsonexceptiontype_error309) if called on JSON values other than
arrays; example: `"cannot use insert() with string"`
@@ -69,6 +69,12 @@ The library uses the following mapping from JSON values types to UBJSON types ac
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
receiving side is immediately informed on the number of elements of the container.
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576, and
`to_ubjson` writes longer arrays of these types without the annotation, so any value it produces can be read back.
!!! info "Binary values"
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON
+9
View File
@@ -868,6 +868,12 @@ The size of an array or object in a [binary format](../features/binary_formats/i
the size following `#` for [UBJSON](../features/binary_formats/ubjson.md)/[BJData](../features/binary_formats/bjdata.md),
or the encoded length for [CBOR](../features/binary_formats/cbor.md).
The exception is also thrown for a [UBJSON](../features/binary_formats/ubjson.md) array of a type that is encoded by its
marker alone (`Z`, `T` or `F`) whose declared count exceeds 1,048,576. Such an array has no payload, so its count alone
decides how much memory is allocated, and a handful of bytes would otherwise describe billions of values.
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the size and type annotation,
so any value it produces can still be read back.
!!! failure "Example messages"
```
@@ -879,6 +885,9 @@ or the encoded length for [CBOR](../features/binary_formats/cbor.md).
```
[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive map size
```
```
[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive array size
```
### json.exception.out_of_range.409
File diff suppressed because it is too large Load Diff
+17 -96
View File
@@ -8,11 +8,10 @@
#pragma once
#include <algorithm> // find_if
#include <cstddef>
#include <string> // string
#include <type_traits> // enable_if_t
#include <utility> // move, pair
#include <utility> // move
#include <vector> // vector
#include <nlohmann/detail/exceptions.hpp>
@@ -250,7 +249,7 @@ class json_sax_dom_parser
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return parse_error(0, "", out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
JSON_THROW(out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
}
return true;
@@ -299,7 +298,7 @@ class json_sax_dom_parser
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return parse_error(0, "", out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
JSON_THROW(out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
}
return true;
@@ -569,7 +568,7 @@ class json_sax_dom_callback_parser
// check object limit
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return parse_error(0, "", out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
JSON_THROW(out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
}
}
return true;
@@ -586,17 +585,7 @@ class json_sax_dom_callback_parser
// add discarded value at the given key and store the reference for later
if (keep && ref_stack.back())
{
auto& obj = *ref_stack.back()->m_data.m_value.object;
const auto it = obj.find(val);
if (it != obj.end())
{
// this is a duplicate key (legal in JSON); remember its
// current value so it can be restored later if the new
// value is rejected by the callback, instead of being
// erased together with the discarded placeholder
duplicate_key_stash.emplace_back(&(it->second), it->second);
}
object_element = &(obj[val] = discarded);
object_element = &(ref_stack.back()->m_data.m_value.object->operator[](val) = discarded);
}
return true;
@@ -608,18 +597,13 @@ class json_sax_dom_callback_parser
{
if (!callback(static_cast<int>(ref_stack.size()) - 1, parse_event_t::object_end, *ref_stack.back()))
{
// discard object, unless this slot holds a duplicate key's
// previous value pending restoration, in which case that
// value is restored instead of being discarded
if (!resolve_duplicate_key_stash(ref_stack.back(), true))
{
*ref_stack.back() = discarded;
// discard object
*ref_stack.back() = discarded;
#if JSON_DIAGNOSTIC_POSITIONS
// Set start/end positions for discarded object.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
// Set start/end positions for discarded object.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
#endif
}
}
else
{
@@ -633,10 +617,6 @@ class json_sax_dom_callback_parser
#endif
ref_stack.back()->set_parents();
// this object is finally, definitively kept; drop any
// pending duplicate-key stash entry for its slot since it
// can no longer be restored
resolve_duplicate_key_stash(ref_stack.back(), false);
}
}
@@ -679,7 +659,7 @@ class json_sax_dom_callback_parser
// check array limit
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return parse_error(0, "", out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
JSON_THROW(out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
}
}
@@ -706,25 +686,16 @@ class json_sax_dom_callback_parser
#endif
ref_stack.back()->set_parents();
// this array is finally, definitively kept; drop any
// pending duplicate-key stash entry for its slot since it
// can no longer be restored
resolve_duplicate_key_stash(ref_stack.back(), false);
}
else
{
// discard array, unless this slot holds a duplicate key's
// previous value pending restoration, in which case that
// value is restored instead of being discarded
if (!resolve_duplicate_key_stash(ref_stack.back(), true))
{
*ref_stack.back() = discarded;
// discard array
*ref_stack.back() = discarded;
#if JSON_DIAGNOSTIC_POSITIONS
// Set start/end positions for discarded array.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
// Set start/end positions for discarded array.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
#endif
}
}
}
@@ -838,48 +809,14 @@ class json_sax_dom_callback_parser
}
#endif
/// if there is a pending duplicate-key stash entry for this exact slot,
/// remove it from the stash; if restore_value is true, the stashed
/// previous value is moved back into the slot first (use this when the
/// new value at that slot was rejected); otherwise the stash entry is
/// simply dropped (use this when the new value was accepted, so it
/// correctly supersedes the old one and no restore should ever happen
/// for this slot again)
/// @return whether a matching stash entry was found (and processed)
bool resolve_duplicate_key_stash(BasicJsonType* slot, bool restore_value)
{
const auto it = std::find_if(duplicate_key_stash.begin(), duplicate_key_stash.end(),
[slot](const std::pair<BasicJsonType*, BasicJsonType>& entry)
{
return entry.first == slot;
});
if (it == duplicate_key_stash.end())
{
return false;
}
if (restore_value)
{
*slot = std::move(it->second);
}
duplicate_key_stash.erase(it);
return true;
}
/// remove the discarded value the callback rejected from its parent,
/// unless it is a duplicate key's slot with a stashed previous value,
/// in which case that previous value is restored instead
void remove_discarded_value(BasicJsonType& parent)
/// remove the discarded value the callback rejected from its parent
static void remove_discarded_value(BasicJsonType& parent)
{
for (auto it = parent.begin(); it != parent.end(); ++it)
{
if (it->is_discarded())
{
if (!resolve_duplicate_key_stash(&(*it), true))
{
parent.erase(it);
}
parent.erase(it);
break;
}
}
@@ -977,16 +914,6 @@ class json_sax_dom_callback_parser
JSON_ASSERT(object_element);
*object_element = std::move(value);
if (!skip_callback)
{
// this scalar value finally, definitively replaces whatever was
// at this slot; drop any pending duplicate-key stash entry for
// it since it can no longer be restored (a container value at
// this slot is resolved later, in end_object()/end_array(),
// since skip_callback is true for the placeholder handling that
// happens here for those)
resolve_duplicate_key_stash(object_element, false);
}
return {true, object_element};
}
@@ -1000,12 +927,6 @@ class json_sax_dom_callback_parser
std::vector<bool> key_keep_stack {}; // NOLINT(readability-redundant-member-init)
/// helper to hold the reference for the next object element
BasicJsonType* object_element = nullptr;
/// stash of (slot pointer, previous value) for object members that
/// already existed when key() was called again for the same key
/// (duplicate keys); used to restore the previous value if the new
/// value is later rejected by the callback, instead of erasing the
/// member entirely
std::vector<std::pair<BasicJsonType*, BasicJsonType>> duplicate_key_stash {};
/// whether a syntax error occurred
bool errored = false;
/// callback function
@@ -826,7 +826,17 @@ class binary_writer
std::vector<CharType> bjdx = {'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z'}; // excluded markers in bjdata optimized type
if (same_prefix && !(use_bjdata && std::find(bjdx.begin(), bjdx.end(), first_prefix) != bjdx.end()))
// an optimized array of a valueless type carries no payload, so a
// reader has nothing but the declared count to bound the allocation
// by and refuses an excessive one. Write the unoptimized form for
// those, at one byte per element, so the result can be read back.
// Objects are not affected: every element is preceded by its key.
const bool valueless_type = (first_prefix == 'Z' || first_prefix == 'T' || first_prefix == 'F');
const bool excessive_valueless = valueless_type
&& j.m_data.m_value.array->size() > detail::max_valueless_container_size;
if (same_prefix && !excessive_valueless
&& !(use_bjdata && std::find(bjdx.begin(), bjdx.end(), first_prefix) != bjdx.end()))
{
prefix_required = false;
oa->write_character(to_char_type('$'));
+84 -155
View File
@@ -3425,12 +3425,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_THROW(invalid_iterator::create(211, "passed iterators may not belong to container", this));
}
// passed iterators must belong to arrays
if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_array()))
{
JSON_THROW(invalid_iterator::create(202, "iterators first and last must point to arrays", this));
}
// insert to array and return iterator
return insert_iterator(pos, first.m_it.array_iterator, last.m_it.array_iterator);
}
@@ -3579,7 +3573,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
using std::swap;
swap(*(m_data.m_value.array), other);
set_parents();
}
else
{
@@ -3596,7 +3589,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
using std::swap;
swap(*(m_data.m_value.object), other);
set_parents();
}
else
{
@@ -4481,8 +4473,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::cbor).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::cbor).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in CBOR format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -4498,8 +4493,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::cbor).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::cbor).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
template<typename T>
@@ -4524,8 +4522,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
auto ia = i.get();
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
// NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::cbor).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::cbor).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in MessagePack format
@@ -4539,8 +4540,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::msgpack).sax_parse(input_format_t::msgpack, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::msgpack).sax_parse(input_format_t::msgpack, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in MessagePack format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -4555,8 +4559,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::msgpack).sax_parse(input_format_t::msgpack, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::msgpack).sax_parse(input_format_t::msgpack, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
template<typename T>
@@ -4579,8 +4586,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
auto ia = i.get();
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
// NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::msgpack).sax_parse(input_format_t::msgpack, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::msgpack).sax_parse(input_format_t::msgpack, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in UBJSON format
@@ -4594,8 +4604,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::ubjson).sax_parse(input_format_t::ubjson, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::ubjson).sax_parse(input_format_t::ubjson, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in UBJSON format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -4610,8 +4623,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::ubjson).sax_parse(input_format_t::ubjson, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::ubjson).sax_parse(input_format_t::ubjson, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
template<typename T>
@@ -4634,8 +4650,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
auto ia = i.get();
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
// NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::ubjson).sax_parse(input_format_t::ubjson, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::ubjson).sax_parse(input_format_t::ubjson, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BJData format
@@ -4649,8 +4668,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::bjdata).sax_parse(input_format_t::bjdata, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bjdata).sax_parse(input_format_t::bjdata, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BJData format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -4665,8 +4687,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::bjdata).sax_parse(input_format_t::bjdata, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bjdata).sax_parse(input_format_t::bjdata, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BSON format
@@ -4680,8 +4705,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::bson).sax_parse(input_format_t::bson, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bson).sax_parse(input_format_t::bson, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BSON format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -4696,8 +4724,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::bson).sax_parse(input_format_t::bson, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bson).sax_parse(input_format_t::bson, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
template<typename T>
@@ -4720,8 +4751,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
auto ia = i.get();
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
// NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
const bool res = binary_reader<decltype(ia)>(std::move(ia), input_format_t::bson).sax_parse(input_format_t::bson, &sdp, strict); // cppcheck-suppress[accessMoved]
return res ? result : basic_json(value_t::discarded);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bson).sax_parse(input_format_t::bson, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @}
@@ -5165,139 +5199,34 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
case value_t::object:
{
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the recursive per-key diffs in the fast path below,
// to match source's original iteration order (as the
// original, pre-reordering-aware implementation did) instead
// of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
// first pass: traverse this object's elements
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
// escape the key name to be used in a JSON patch
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path below, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// patch ops for keys that were added (i.e., in target but not
// in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
// recursive call to compare object values at key it
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key recursive diff
// is correct and minimal, as before. common_keys_source_order
// is, by construction, the subsequence of source's keys
// that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it;
}
else
{
// found a key that is not in target -> remove it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
}
// append the "add" ops for brand-new keys collected above
// during the pass over target -- no second source.find()
// per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
else
{
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
// found a key that is not in o -> remove it
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
// second pass: traverse other object's elements
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
// found a key that is not in this -> add it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(
{
File diff suppressed because it is too large Load Diff
+18 -3
View File
@@ -3288,8 +3288,10 @@ TEST_CASE("BJData")
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR1), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR1, true, false).is_discarded());
// a dimension vector that opens another one is rejected where the
// nested '[' is read, rather than after it has been descended into
std::vector<uint8_t> const vR2 = {'[', '$', 'i', '#', '[', '#', '[', 'i', 1, ']', ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR2), "[json.exception.parse_error.113] parse error at byte 11: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x5D", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR2), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR2, true, false).is_discarded());
std::vector<uint8_t> const vR3 = {'[', '#', '[', 'i', '2', 'i', 2, ']'};
@@ -3297,7 +3299,7 @@ TEST_CASE("BJData")
CHECK(json::from_bjdata(vR3, true, false).is_discarded());
std::vector<uint8_t> const vR4 = {'[', '$', 'i', '#', '[', '$', 'i', '#', '[', 'i', 1, ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR4), "[json.exception.parse_error.110] parse error at byte 14: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR4), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR4, true, false).is_discarded());
std::vector<uint8_t> const vR5 = {'[', '$', 'i', '#', '[', '[', '[', ']', ']', ']'};
@@ -3305,12 +3307,25 @@ TEST_CASE("BJData")
CHECK(json::from_bjdata(vR5, true, false).is_discarded());
std::vector<uint8_t> const vR6 = {'[', '$', 'i', '#', '[', '$', 'i', '#', '[', 'i', '2', 'i', 2, ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR6), "[json.exception.parse_error.112] parse error at byte 14: syntax error while parsing BJData size: ndarray can not be recursive", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR6), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR6, true, false).is_discarded());
std::vector<uint8_t> const vH = {'[', 'H', '[', '#', '[', '$', 'i', '#', '[', 'i', '2', 'i', 2, ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vH), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vH, true, false).is_discarded());
// Every "#[" of this chain used to open another dimension vector
// and cost several stack frames before anything was rejected, so a
// long enough chain crashed the process (see #5104). The nested
// vector is refused where it is read, so the length is irrelevant.
std::vector<uint8_t> vRdeep = {'['};
for (std::size_t i = 0; i < 100000; ++i)
{
vRdeep.push_back('#');
vRdeep.push_back('[');
}
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vRdeep), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vRdeep, true, false).is_discarded());
}
SECTION("objects")
+139
View File
@@ -2035,6 +2035,145 @@ TEST_CASE("CBOR definite length equal to the indefinite-length sentinel")
}
}
TEST_CASE("CBOR nesting does not consume the call stack")
{
// Containers used to be read by calling back into the value reader once
// per element, and a tag by calling it for the tagged value, so the native
// call stack grew with the nesting depth of the input. Each of the three
// costs a single byte to encode -- 0x9F, 0x81 and 0xC2 -- so a payload of
// repeated bytes crashed the process (#5104). The containers are kept on a
// heap stack now, and a tag is read in a loop.
//
// Deeply nested values must not be compared, copied or dumped here: those
// operations are still recursive and would reintroduce the crash.
json _;
SECTION("indefinite-length containers")
{
const std::vector<uint8_t> input(500000, 0x9F);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), "[json.exception.parse_error.110] parse error at byte 500001: syntax error while parsing CBOR value: unexpected end of input", json::parse_error&);
CHECK(json::from_cbor(input, true, false).is_discarded());
}
SECTION("definite-length containers")
{
const std::vector<uint8_t> input(500000, 0x81);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), "[json.exception.parse_error.110] parse error at byte 500001: syntax error while parsing CBOR value: unexpected end of input", json::parse_error&);
CHECK(json::from_cbor(input, true, false).is_discarded());
}
SECTION("tags")
{
// a tag is not a value of its own, so a chain of them used to recurse
const std::vector<uint8_t> input(500000, 0xC2);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input, true, true, json::cbor_tag_handler_t::ignore), "[json.exception.parse_error.110] parse error at byte 500001: syntax error while parsing CBOR value: unexpected end of input", json::parse_error&);
CHECK(json::from_cbor(input, true, false, json::cbor_tag_handler_t::ignore).is_discarded());
}
SECTION("a well-formed deep value is read through the SAX interface")
{
std::vector<uint8_t> input(200000, 0x9F);
input.insert(input.end(), 200000, 0xFF);
SaxCountdown accept_all(1000000);
CHECK(json::sax_parse(input, &accept_all, json::input_format_t::cbor));
}
SECTION("a well-formed deep value is read into a value")
{
const std::size_t depth = 10000;
std::vector<uint8_t> input(depth, 0x81);
input.push_back(0x00);
json j = json::from_cbor(input);
std::size_t measured = 0;
const json* p = &j;
while (p->is_array() && !p->empty())
{
p = &p->front();
++measured;
}
CHECK(measured == depth);
CHECK(p->is_number());
}
SECTION("containers are still read the same way")
{
CHECK(json::from_cbor(std::vector<uint8_t>({0x80})) == json::array());
CHECK(json::from_cbor(std::vector<uint8_t>({0xA0})) == json::object());
CHECK(json::from_cbor(std::vector<uint8_t>({0x9F, 0xFF})) == json::array());
CHECK(json::from_cbor(std::vector<uint8_t>({0xBF, 0xFF})) == json::object());
CHECK(json::from_cbor(std::vector<uint8_t>({0x9F, 0x01, 0x02, 0xFF})) == json({1, 2}));
CHECK(json::from_cbor(std::vector<uint8_t>({0xBF, 0x61, 'a', 0x01, 0xFF})) == json({{"a", 1}}));
// definite and indefinite forms nested inside each other
CHECK(json::from_cbor(std::vector<uint8_t>({0x9F, 0x82, 0x01, 0x02, 0xA1, 0x61, 'k', 0xBF, 0xFF, 0xFF})) == json({{1, 2}, {{"k", json::object()}}}));
}
SECTION("tagged values are still read the same way")
{
const auto ignore = json::cbor_tag_handler_t::ignore;
CHECK(json::from_cbor(std::vector<uint8_t>({0xC2, 0x01}), true, true, ignore) == json(1));
// a chain of tags resolves to the value that follows it
CHECK(json::from_cbor(std::vector<uint8_t>({0xC2, 0xC2, 0xC2, 0x01}), true, true, ignore) == json(1));
// a tag inside a container, and one in front of a container
CHECK(json::from_cbor(std::vector<uint8_t>({0x82, 0xC2, 0x01, 0x02}), true, true, ignore) == json({1, 2}));
CHECK(json::from_cbor(std::vector<uint8_t>({0xC2, 0x82, 0x01, 0x02}), true, true, ignore) == json({1, 2}));
}
}
TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
{
// Reading an indefinite-length string or byte array used to call itself
// once per chunk, so a payload of repeated 0x7F (or 0x5F) bytes exhausted
// the call stack before any of the input was rejected. The open levels are
// counted now, and the levels below prove the reader still reads the same
// values and reports the same errors at the same byte offsets.
json _;
SECTION("many open levels are reported, not crashed on")
{
const std::vector<uint8_t> input(200000, 0x7F);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), "[json.exception.parse_error.110] parse error at byte 200001: syntax error while parsing CBOR string: unexpected end of input", json::parse_error&);
CHECK(json::from_cbor(input, true, false).is_discarded());
}
SECTION("many open levels are reported, not crashed on (binary)")
{
const std::vector<uint8_t> input(200000, 0x5F);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), "[json.exception.parse_error.110] parse error at byte 200001: syntax error while parsing CBOR binary: unexpected end of input", json::parse_error&);
CHECK(json::from_cbor(input, true, false).is_discarded());
}
SECTION("chunks are still concatenated")
{
CHECK(json::from_cbor(std::vector<uint8_t>({0x7F, 0xFF})) == json(""));
CHECK(json::from_cbor(std::vector<uint8_t>({0x7F, 0x61, 0x61, 0xFF})) == json("a"));
// nested indefinite-length strings are concatenated across levels
CHECK(json::from_cbor(std::vector<uint8_t>({0x7F, 0x7F, 0x61, 0x61, 0xFF, 0x61, 0x62, 0xFF})) == json("ab"));
CHECK(json::from_cbor(std::vector<uint8_t>({0x7F, 0x7F, 0x7F, 0x61, 0x7A, 0xFF, 0xFF, 0xFF})) == json("z"));
CHECK(json::from_cbor(std::vector<uint8_t>({0xA1, 0x7F, 0x61, 0x61, 0xFF, 0x01})) == json({{"a", 1}}));
}
SECTION("chunks are still concatenated (binary)")
{
CHECK(json::from_cbor(std::vector<uint8_t>({0x5F, 0x41, 0x61, 0xFF})) == json::binary({0x61}));
CHECK(json::from_cbor(std::vector<uint8_t>({0x5F, 0x5F, 0x41, 0x61, 0xFF, 0x41, 0x62, 0xFF})) == json::binary({0x61, 0x62}));
}
SECTION("a chunk that is not a string is still rejected")
{
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0x7F, 0x7F, 0x00})), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x00", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0x5F, 0x5F, 0x00})), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing CBOR binary: expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x00", json::parse_error&);
}
SECTION("a break marker outside an indefinite-length string is not a string")
{
// 0xFF only closes a string that was opened; on its own it is not one
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
}
}
TEST_CASE("CBOR roundtrips" * doctest::skip())
{
SECTION("input from flynn")
-31
View File
@@ -273,36 +273,5 @@ TEST_CASE("Regression tests for extended diagnostics")
CHECK(j1["numbers"]["two"] == 2);
CHECK(j1["string"] == "t");
}
SECTION("Regression test - swap(array_t&)/swap(object_t&) must update JSON_DIAGNOSTICS parent pointers")
{
// swap(array_t&)
{
json j = json::array();
json::array_t arr = {json::array({1})};
j.swap(arr);
// parent pointers of the moved-in elements must point into j, not
// into the now-defunct free-standing array_t
CHECK_THROWS_WITH_AS(j[0][0].get<std::string>(), "[json.exception.type_error.302] (/0/0) type must be string, but is number", json::type_error);
// must not trigger assert_invariant() in a debug/assert-enabled build
json const k = j;
CHECK(k == j);
}
// swap(object_t&)
{
json o = json::object();
json::object_t obj = {{"a", json::array({1})}};
o.swap(obj);
CHECK_THROWS_WITH_AS(o["a"][0].get<std::string>(), "[json.exception.type_error.302] (/a/0) type must be string, but is number", json::type_error);
// must not trigger assert_invariant() in a debug/assert-enabled build
json const p = o;
CHECK(p == o);
}
}
}
-14
View File
@@ -641,20 +641,6 @@ TEST_CASE("modifiers")
CHECK_THROWS_WITH_AS(j_array.insert(j_array.end(), j_other_array.begin(), j_other_array2.end()), "[json.exception.invalid_iterator.210] iterators do not fit",
json::invalid_iterator&);
}
SECTION("iterators not pointing into an array")
{
json j_object2 = {{"k", 1}, {"l", 2}};
json j_primitive = 5;
json j_null;
CHECK_THROWS_WITH_AS(j_array.insert(j_array.begin(), j_object2.begin(), j_object2.end()), "[json.exception.invalid_iterator.202] iterators first and last must point to arrays",
json::invalid_iterator&);
CHECK_THROWS_WITH_AS(j_array.insert(j_array.begin(), j_primitive.begin(), j_primitive.end()), "[json.exception.invalid_iterator.202] iterators first and last must point to arrays",
json::invalid_iterator&);
CHECK_THROWS_WITH_AS(j_array.insert(j_array.begin(), j_null.begin(), j_null.end()), "[json.exception.invalid_iterator.202] iterators first and last must point to arrays",
json::invalid_iterator&);
}
}
SECTION("range for object")
+61
View File
@@ -1598,6 +1598,67 @@ TEST_CASE("MessagePack")
}
// use this testcase outside [hide] to run it with Valgrind
TEST_CASE("MessagePack nesting does not consume the call stack")
{
// Reading a container used to call back into the value reader once per
// element, so the native call stack grew with the nesting depth of the
// input: one frame per byte for repeated 0x91 (a one-element array), which
// crashes the process long before the input is exhausted (#5104). The
// containers are kept on a heap stack now.
//
// Note that deeply nested values must not be compared, copied or dumped
// here: those operations are still recursive, and would reintroduce the
// very crash this checks for. Depth is measured by descending instead.
SECTION("an unterminated chain is reported, not crashed on")
{
json _;
const std::vector<uint8_t> input(300000, 0x91);
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(input), "[json.exception.parse_error.110] parse error at byte 300001: syntax error while parsing MessagePack value: unexpected end of input", json::parse_error&);
CHECK(json::from_msgpack(input, true, false).is_discarded());
}
SECTION("a well-formed deep value is read through the SAX interface")
{
std::vector<uint8_t> input(300000, 0x91);
input.push_back(0x01); // innermost value
SaxCountdown accept_all(600001);
CHECK(json::sax_parse(input, &accept_all, json::input_format_t::msgpack));
}
SECTION("a well-formed deep value is read into a value")
{
const std::size_t depth = 10000;
std::vector<uint8_t> input(depth, 0x91);
input.push_back(0x01);
json j = json::from_msgpack(input);
std::size_t measured = 0;
const json* p = &j;
while (p->is_array() && !p->empty())
{
p = &p->front();
++measured;
}
CHECK(measured == depth);
CHECK(p->is_number());
}
SECTION("containers are still read the same way")
{
CHECK(json::from_msgpack(std::vector<uint8_t>({0x90})) == json::array());
CHECK(json::from_msgpack(std::vector<uint8_t>({0x80})) == json::object());
CHECK(json::from_msgpack(std::vector<uint8_t>({0x92, 0x90, 0x80})) == json({json::array(), json::object()}));
CHECK(json::from_msgpack(std::vector<uint8_t>({0x91, 0x91, 0x91, 0x90})) == json({{{json::array()}}}));
CHECK(json::from_msgpack(std::vector<uint8_t>({0x81, 0xA1, 'a', 0x81, 0xA1, 'b', 0x92, 0x01, 0x02})) == json({{"a", {{"b", {1, 2}}}}}));
// array 16 and map 32, i.e. the counted forms
CHECK(json::from_msgpack(std::vector<uint8_t>({0xDC, 0x00, 0x02, 0x01, 0x02})) == json({1, 2}));
CHECK(json::from_msgpack(std::vector<uint8_t>({0xDF, 0x00, 0x00, 0x00, 0x01, 0xA1, 'k', 0xC3})) == json({{"k", true}}));
}
}
TEST_CASE("single MessagePack roundtrip")
{
SECTION("sample.json")
-81
View File
@@ -81,84 +81,3 @@ TEST_CASE("regression test for issue #3732 - iteration_proxy_value<iter_impl<ord
};
static_cast<void>(fn);
}
TEST_CASE("regression test - diff() must account for ordered_json member order")
{
SECTION("pure reorder, no value changes")
{
ordered_json a = {{"a", 1}, {"b", 2}};
ordered_json b = {{"b", 2}, {"a", 1}};
CHECK(a != b); // order-sensitive equality
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("new key must land at the front")
{
ordered_json c = {{"b", 2}};
ordered_json e = {{"a", 1}, {"b", 2}};
CHECK(c.patch(ordered_json::diff(c, e)) == e);
}
SECTION("reorder plus a value change on one of the reordered keys")
{
ordered_json a = {{"a", 1}, {"b", 2}};
ordered_json b = {{"b", 20}, {"a", 1}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("reorder plus a deleted key")
{
ordered_json a = {{"a", 1}, {"b", 2}, {"c", 3}};
ordered_json b = {{"b", 2}, {"a", 1}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("reorder plus a nested value that itself needs a recursive diff")
{
ordered_json a = {{"a", {{"x", 1}, {"y", 2}}}, {"b", 2}};
ordered_json b = {{"b", 2}, {"a", {{"x", 1}, {"y", 99}}}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("three or more keys shuffled into a different order")
{
ordered_json a = {{"a", 1}, {"b", 2}, {"c", 3}, {"d", 4}};
ordered_json b = {{"d", 4}, {"b", 2}, {"a", 1}, {"c", 3}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("matching order still produces a minimal patch (fast path unaffected)")
{
ordered_json a = {{"a", 1}, {"b", 2}, {"c", 3}};
ordered_json b = {{"a", 1}, {"b", 20}, {"c", 3}};
auto p = ordered_json::diff(a, b);
// only the changed value should be touched, not a wholesale remove+add
CHECK(p.size() == 1);
CHECK(p[0]["op"] == "replace");
CHECK(p[0]["path"] == "/b");
CHECK(a.patch(p) == b);
}
SECTION("plain json (std::map-backed) is unaffected by same-key-different-insertion-order")
{
json a;
a["b"] = 2;
a["a"] = 1;
json b;
b["a"] = 1;
b["b"] = 2;
// std::map iteration is always sorted by key, so a == b regardless of
// insertion order, and diff() must still produce the same minimal
// (empty) result as before this fix
CHECK(a == b);
auto p = json::diff(a, b);
CHECK(p.empty());
CHECK(a.patch(p) == b);
}
}
-910
View File
@@ -239,209 +239,6 @@ class my_allocator : public std::allocator<T>
};
};
/////////////////////////////////////////////////////////////////////
// for #3077
/////////////////////////////////////////////////////////////////////
class FooAlloc
{};
class Foo
{
public:
explicit Foo(const FooAlloc& /* unused */ = FooAlloc()) {}
bool value = false;
};
class FooBar
{
public:
Foo foo{}; // NOLINT(readability-redundant-member-init)
};
inline void from_json(const nlohmann::json& j, FooBar& fb) // NOLINT(misc-use-internal-linkage)
{
j.at("value").get_to(fb.foo.value);
}
/////////////////////////////////////////////////////////////////////
// for #3171
/////////////////////////////////////////////////////////////////////
struct for_3171_base // NOLINT(cppcoreguidelines-special-member-functions)
{
for_3171_base(const std::string& /*unused*/ = {}) {}
virtual ~for_3171_base();
for_3171_base(const for_3171_base& other) // NOLINT(hicpp-use-equals-default,modernize-use-equals-default)
: str(other.str)
{}
for_3171_base& operator=(const for_3171_base& other)
{
if (this != &other)
{
str = other.str;
}
return *this;
}
for_3171_base(for_3171_base&& other) noexcept
: str(std::move(other.str))
{}
for_3171_base& operator=(for_3171_base&& other) noexcept
{
if (this != &other)
{
str = std::move(other.str);
}
return *this;
}
virtual void _from_json(const json& j)
{
j.at("str").get_to(str);
}
std::string str{}; // NOLINT(readability-redundant-member-init)
};
for_3171_base::~for_3171_base() = default;
struct for_3171_derived : public for_3171_base
{
for_3171_derived() = default;
~for_3171_derived() override;
explicit for_3171_derived(const std::string& /*unused*/) { }
for_3171_derived(const for_3171_derived& other) // NOLINT(hicpp-use-equals-default,modernize-use-equals-default)
: for_3171_base(other)
{}
for_3171_derived& operator=(const for_3171_derived& other)
{
if (this != &other)
{
for_3171_base::operator=(other); // Call base class assignment operator
}
return *this;
}
for_3171_derived(for_3171_derived&& other) noexcept
: for_3171_base(std::move(other))
{}
for_3171_derived& operator=(for_3171_derived&& other) noexcept
{
if (this != &other)
{
for_3171_base::operator=(std::move(other)); // Call base class move assignment operator
}
return *this;
}
};
for_3171_derived::~for_3171_derived() = default;
inline void from_json(const json& j, for_3171_base& tb) // NOLINT(misc-use-internal-linkage)
{
tb._from_json(j);
}
/////////////////////////////////////////////////////////////////////
// for #3312
/////////////////////////////////////////////////////////////////////
#ifdef JSON_HAS_CPP_20
struct for_3312
{
std::string name;
};
inline void from_json(const json& j, for_3312& obj) // NOLINT(misc-use-internal-linkage)
{
j.at("name").get_to(obj.name);
}
#endif
/////////////////////////////////////////////////////////////////////
// for #3204
/////////////////////////////////////////////////////////////////////
struct for_3204_foo
{
for_3204_foo() = default;
explicit for_3204_foo(std::string /*unused*/) {} // NOLINT(performance-unnecessary-value-param)
};
struct for_3204_bar
{
enum constructed_from_t // NOLINT(cppcoreguidelines-use-enum-class)
{
constructed_from_none = 0,
constructed_from_foo = 1,
constructed_from_json = 2
};
explicit for_3204_bar(std::function<void(for_3204_foo)> /*unused*/) noexcept // NOLINT(performance-unnecessary-value-param)
: constructed_from(constructed_from_foo) {}
explicit for_3204_bar(std::function<void(json)> /*unused*/) noexcept // NOLINT(performance-unnecessary-value-param)
: constructed_from(constructed_from_json) {}
constructed_from_t constructed_from = constructed_from_none;
};
/////////////////////////////////////////////////////////////////////
// for #3333
/////////////////////////////////////////////////////////////////////
struct for_3333 final
{
for_3333(int x_ = 0, int y_ = 0) : x(x_), y(y_) {}
template <class T>
for_3333(const T& /*unused*/)
{
CHECK(false);
}
int x = 0;
int y = 0;
};
template <>
inline for_3333::for_3333(const json& j)
: for_3333(j.value("x", 0), j.value("y", 0))
{}
/////////////////////////////////////////////////////////////////////
// for #3810
/////////////////////////////////////////////////////////////////////
struct Example_3810
{
int bla{};
Example_3810() = default;
};
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(Example_3810, bla) // NOLINT(misc-use-internal-linkage)
/////////////////////////////////////////////////////////////////////
// for #4740
/////////////////////////////////////////////////////////////////////
#ifdef JSON_HAS_CPP_17
struct Example_4740
{
std::optional<std::string> host = std::nullopt;
std::optional<int> port = std::nullopt;
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT(Example_4740, host, port)
};
#endif
TEST_CASE("regression tests 2")
{
SECTION("issue #1001 - Fix memory leak during parser callback")
@@ -959,713 +756,6 @@ TEST_CASE("regression tests 2")
CHECK(j == k);
}
#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
// JSON_HAS_CPP_17 (do not remove; see note at top of file)
SECTION("issue #3070 - Version 3.10.3 breaks backward-compatibility with 3.10.2 ")
{
nlohmann::detail::std_fs::path text_path("/tmp/text.txt");
const json j(text_path);
const auto j_path = j.get<nlohmann::detail::std_fs::path>();
CHECK(j_path == text_path);
#if DOCTEST_CLANG || DOCTEST_GCC >= DOCTEST_COMPILER(8, 4, 0)
// only known to work on Clang and GCC >=8.4
CHECK_THROWS_WITH_AS(nlohmann::detail::std_fs::path(json(1)), "[json.exception.type_error.302] type must be string, but is number", json::type_error);
#endif
}
#endif
SECTION("issue #3077 - explicit constructor with default does not compile")
{
json j;
j[0]["value"] = true;
std::vector<FooBar> foo;
j.get_to(foo);
}
SECTION("issue #3108 - ordered_json doesn't support range based erase")
{
ordered_json j = {1, 2, 2, 4};
auto last = std::unique(j.begin(), j.end());
j.erase(last, j.end());
CHECK(j.dump() == "[1,2,4]");
j.erase(std::remove_if(j.begin(), j.end(), [](const ordered_json & val)
{
return val == 2;
}), j.end());
CHECK(j.dump() == "[1,4]");
}
SECTION("issue #3343 - json and ordered_json are not interchangeable")
{
json::object_t jobj({ { "product", "one" } });
ordered_json::object_t ojobj({{"product", "one"}});
auto jit = jobj.begin();
auto ojit = ojobj.begin();
CHECK(jit->first == ojit->first);
CHECK(jit->second.get<std::string>() == ojit->second.get<std::string>());
}
SECTION("issue #3171 - if class is_constructible from std::string wrong from_json overload is being selected, compilation failed")
{
const json j{{ "str", "value"}};
// failed with: error: no match for operator= (operand types are for_3171_derived and const nlohmann::basic_json<>::string_t
// {aka const std::__cxx11::basic_string<char>})
// s = *j.template get_ptr<const typename BasicJsonType::string_t*>();
auto td = j.get<for_3171_derived>();
CHECK(td.str == "value");
}
#ifdef JSON_HAS_CPP_20
SECTION("issue #3312 - Parse to custom class from unordered_json breaks on G++11.2.0 with C++20")
{
// see test for #3171
const ordered_json j = {{"name", "class"}};
for_3312 obj{};
j.get_to(obj);
CHECK(obj.name == "class");
}
#endif
#if defined(JSON_HAS_CPP_17) && JSON_USE_IMPLICIT_CONVERSIONS
SECTION("issue #3428 - Error occurred when converting nlohmann::json to std::any")
{
const json j;
const std::any a1 = j;
std::any&& a2 = j;
CHECK(a1.type() == typeid(j));
CHECK(a2.type() == typeid(j));
}
#endif
SECTION("issue #3204 - ambiguous regression")
{
const for_3204_bar bar_from_foo([](for_3204_foo) noexcept {}); // NOLINT(performance-unnecessary-value-param)
const for_3204_bar bar_from_json([](json) noexcept {}); // NOLINT(performance-unnecessary-value-param)
CHECK(bar_from_foo.constructed_from == for_3204_bar::constructed_from_foo);
CHECK(bar_from_json.constructed_from == for_3204_bar::constructed_from_json);
}
SECTION("issue #3333 - Ambiguous conversion from nlohmann::basic_json<> to custom class")
{
const json j
{
{"x", 1},
{"y", 2}
};
const for_3333 p = j;
CHECK(p.x == 1);
CHECK(p.y == 2);
}
SECTION("issue #3810 - ordered_json doesn't support construction from C array of custom type")
{
Example_3810 states[45]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
// fix "not used" warning
states[0].bla = 1;
const auto* const expected = R"([{"bla":1},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0}])";
// This works:
nlohmann::json j;
j["test"] = states;
CHECK(j["test"].dump() == expected);
// This doesn't compile:
nlohmann::ordered_json oj;
oj["test"] = states;
CHECK(oj["test"].dump() == expected);
}
#ifdef JSON_HAS_CPP_17
SECTION("issue #4740 - build issue with std::optional")
{
const auto t1 = Example_4740();
const auto j1 = nlohmann::json(t1);
CHECK(j1.dump() == "{\"host\":null,\"port\":null}");
const auto t2 = j1.get<Example_4740>();
CHECK(!t2.host.has_value());
CHECK(!t2.port.has_value());
// improve coverage
auto t3 = Example_4740();
t3.port = 80;
t3.host = "example.com";
const auto j2 = nlohmann::json(t3);
CHECK(j2.dump() == "{\"host\":\"example.com\",\"port\":80}");
const auto t4 = j2.get<Example_4740>();
CHECK(t4.host.has_value());
CHECK(t4.port.has_value());
}
#endif
#if !defined(_MSVC_LANG)
// MSVC returns garbage on invalid enum values, so this test is excluded
// there.
SECTION("issue #4762 - json exception 302 with unhelpful explanation : type must be number, but is number")
{
// In #4762, the main issue was that a json object with an invalid type
// returned "number" as type_name(), because this was the default case.
// This test makes sure we now return "invalid" instead.
json j;
j.m_data.m_type = static_cast<json::value_t>(100); // NOLINT(clang-analyzer-optin.core.EnumCastOutOfRange)
CHECK(j.type_name() == "invalid");
}
#endif
#ifdef JSON_HAS_CPP_17
SECTION("issue #4804: from_cbor incompatible with std::vector<std::byte> as binary_t")
{
const std::vector<std::uint8_t> data = {0x80};
const auto decoded = json_4804::from_cbor(data);
CHECK((decoded == json_4804::array()));
}
SECTION("discussion #4209 - custom BinaryType direct assignment and round-tripping")
{
// Test that assigning a custom BinaryType directly creates a binary value, not an array
const std::vector<std::byte> original{std::byte{1}, std::byte{2}, std::byte{3}};
const json_4804 j = original;
CHECK(j.is_binary());
CHECK(!j.is_array());
// Test round-tripping: extracting the binary value back as the custom container type
const auto extracted = j.get<std::vector<std::byte>>();
CHECK(extracted == original);
// Test that the default json alias behavior is unchanged: std::vector<uint8_t> -> array
const json default_json = std::vector<std::uint8_t> {1, 2, 3};
CHECK(default_json.is_array());
CHECK(!default_json.is_binary());
}
SECTION("discussion #4209 - custom BinaryType extraction from parsed array")
{
// Test that extracting a custom BinaryType from a parsed JSON array still works
// (not just from a binary-typed node)
const auto j = json_4804::parse("[1,2,3]");
CHECK(j.is_array());
CHECK(!j.is_binary());
// Extracting as custom BinaryType should work from arrays
const auto extracted = j.get<std::vector<std::byte>>();
CHECK(extracted.size() == 3);
CHECK(extracted[0] == std::byte{1});
CHECK(extracted[1] == std::byte{2});
CHECK(extracted[2] == std::byte{3});
}
SECTION("issue #5046 - implicit conversion of return json to std::optional no longer implicit")
{
const json jval{};
auto GetValue = [](const json & valRoot) -> std::optional<json>
{
if (valRoot.contains("default"))
{
return valRoot.at("default");
}
return std::nullopt;
};
auto result = GetValue(jval);
CHECK(!result.has_value());
}
#endif
#if JSON_HAS_RANGES == 1
SECTION("issue #4440 - assert when using std::views::filter and GCC 10")
{
auto noOpFilter = std::views::filter([](auto&&) noexcept
{
return true;
});
json j = {1, 2, 3};
auto filtered = j | noOpFilter;
CHECK(*filtered.begin() == 1);
}
#endif
#if JSON_HAS_RANGES && !defined(__MINGW32__)
SECTION("issue #4916 - constructing array from C++20 ranges view does not work")
{
std::vector<int> nums{1, 2, 37, 42, 21};
auto filteredNums = nums | std::views::filter([](int i)
{
return i > 10;
});
json const j(filteredNums);
CHECK(j.type() == json::value_t::array);
CHECK(j == json({37, 42, 21}));
}
#endif
// owning_view is not available in libstdc++ < 12
#if JSON_HAS_RANGES && !defined(__MINGW32__) && !(defined(__GLIBCXX__) && _GLIBCXX_RELEASE < 12)
SECTION("issue #4916 - constructing array from prvalue C++20 ranges view (owning_view)")
{
json const j(std::vector<int> {1, 2, 37, 42, 21} | std::views::filter([](int i)
{
return i > 10;
}));
CHECK(j.type() == json::value_t::array);
CHECK(j == json({37, 42, 21}));
}
#endif
#if JSON_HAS_RANGES && !defined(__MINGW32__)
SECTION("issue #4916 - constructing array from C++20 transform view (prvalue elements)")
{
std::vector<int> nums{1, 2, 3};
auto t = nums | std::views::transform([](int i) noexcept
{
return i * 2;
});
json const j(t);
CHECK(j.type() == json::value_t::array);
CHECK(j == json({2, 4, 6}));
}
#endif
}
TEST_CASE_TEMPLATE("issue #4798 - nlohmann::json::to_msgpack() encode float NaN as double", T, double, float) // NOLINT(readability-math-missing-parentheses, bugprone-throwing-static-initialization)
{
// With issue #4798, we encode NaN, infinity, and -infinity as float instead
// of double to allow for smaller encodings.
const json jx = std::numeric_limits<T>::quiet_NaN();
const json jy = std::numeric_limits<T>::infinity();
const json jz = -std::numeric_limits<T>::infinity();
/////////////////////////////////////////////////////////////////////////
// MessagePack
/////////////////////////////////////////////////////////////////////////
// expected MessagePack values
const std::vector<std::uint8_t> msgpack_x = {{0xCA, 0x7F, 0xC0, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_y = {{0xCA, 0x7F, 0x80, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_z = {{0xCA, 0xFF, 0x80, 0x00, 0x00}};
CHECK(json::to_msgpack(jx) == msgpack_x);
CHECK(json::to_msgpack(jy) == msgpack_y);
CHECK(json::to_msgpack(jz) == msgpack_z);
CHECK(std::isnan(json::from_msgpack(msgpack_x).get<T>()));
CHECK(json::from_msgpack(msgpack_y).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_msgpack(msgpack_z).get<T>() == -std::numeric_limits<T>::infinity());
// Make sure the other MessagePakc encodings for NaN, infinity, and
// -infinity are still supported.
const std::vector<std::uint8_t> msgpack_x_2 = {{0xCB, 0x7F, 0xF8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_y_2 = {{0xCB, 0x7F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_z_2 = {{0xCB, 0xFF, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
CHECK(std::isnan(json::from_msgpack(msgpack_x_2).get<T>()));
CHECK(json::from_msgpack(msgpack_y_2).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_msgpack(msgpack_z_2).get<T>() == -std::numeric_limits<T>::infinity());
/////////////////////////////////////////////////////////////////////////
// CBOR
/////////////////////////////////////////////////////////////////////////
// expected CBOR values
const std::vector<std::uint8_t> cbor_x = {{0xF9, 0x7E, 0x00}};
const std::vector<std::uint8_t> cbor_y = {{0xF9, 0x7C, 0x00}};
const std::vector<std::uint8_t> cbor_z = {{0xF9, 0xfC, 0x00}};
CHECK(json::to_cbor(jx) == cbor_x);
CHECK(json::to_cbor(jy) == cbor_y);
CHECK(json::to_cbor(jz) == cbor_z);
CHECK(std::isnan(json::from_cbor(cbor_x).get<T>()));
CHECK(json::from_cbor(cbor_y).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_cbor(cbor_z).get<T>() == -std::numeric_limits<T>::infinity());
// Make sure the other CBOR encodings for NaN, infinity, and -infinity are
// still supported.
const std::vector<std::uint8_t> cbor_x_2 = {{0xFA, 0x7F, 0xC0, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_y_2 = {{0xFA, 0x7F, 0x80, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_z_2 = {{0xFA, 0xFF, 0x80, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_x_3 = {{0xFB, 0x7F, 0xF8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_y_3 = {{0xFB, 0x7F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_z_3 = {{0xFB, 0xFF, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
CHECK(std::isnan(json::from_cbor(cbor_x_2).get<T>()));
CHECK(json::from_cbor(cbor_y_2).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_cbor(cbor_z_2).get<T>() == -std::numeric_limits<T>::infinity());
CHECK(std::isnan(json::from_cbor(cbor_x_3).get<T>()));
CHECK(json::from_cbor(cbor_y_3).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_cbor(cbor_z_3).get<T>() == -std::numeric_limits<T>::infinity());
}
TEST_CASE("regression test #5074 - portable workaround for single-element brace init")
{
json const j_obj = {{"key", "value"}};
json const j = json::array({j_obj});
CHECK(j.is_array());
CHECK(j.size() == 1);
CHECK(j[0] == j_obj);
}
#if defined(JSON_BRACE_INIT_COPY_SEMANTICS) && (JSON_BRACE_INIT_COPY_SEMANTICS == 1)
TEST_CASE("regression test #5074 - single-element brace init with JSON_BRACE_INIT_COPY_SEMANTICS")
{
// with JSON_BRACE_INIT_COPY_SEMANTICS: single-element brace init copies/moves
json const j_obj = {{"key", "value"}, {"num", 42}};
json const j_arr = {1, 2, 3};
// object: brace init copies instead of wrapping
json const j1{j_obj};
CHECK(j1.is_object());
CHECK(j1 == j_obj);
// array: brace init copies instead of wrapping
json const j2{j_arr};
CHECK(j2.is_array());
CHECK(j2.size() == 3);
CHECK(j2 == j_arr);
// primitives still work as initializer lists
json const j3{true};
CHECK(j3.is_boolean());
json const j4{42};
CHECK(j4.is_number_integer());
}
#endif
struct Example_5122
{
float b = 2;
nlohmann::ordered_map<std::string, std::string> c{}; // NOLINT(readability-redundant-member-init): needed for GCC -Weffc++
int a = 1;
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT(Example_5122, b, c, a)
};
TEST_CASE("regression test #5122 - from_json into types holding nlohmann::ordered_map")
{
Example_5122 src;
src.c.emplace("first", "1");
src.c.emplace("second", "2");
ordered_json const j = src;
Example_5122 const dst = j.get<Example_5122>();
CHECK(dst.b == src.b);
CHECK(dst.a == src.a);
REQUIRE(dst.c.size() == src.c.size());
auto src_it = src.c.begin();
auto dst_it = dst.c.begin();
for (; src_it != src.c.end(); ++src_it, ++dst_it)
{
CHECK(dst_it->first == src_it->first);
CHECK(dst_it->second == src_it->second);
}
}
// -Wself-assign-overloaded was introduced in Clang 7. Gate the pragma on
// __has_warning so older Clang versions do not error with "unknown warning
// group". The __has_warning check has to stay inside the __clang__ branch
// because GCC does not provide it and would tokenize-error on the argument.
#if defined(__clang__) && defined(__has_warning)
#if __has_warning("-Wself-assign-overloaded")
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
DOCTEST_CLANG_SUPPRESS_WARNING("-Wself-assign-overloaded")
#endif
#endif
TEST_CASE("regression test #5122 - nlohmann::ordered_map copy-assignment is self-assignment safe")
{
nlohmann::ordered_map<std::string, std::string> m;
m.emplace("first", "1");
m.emplace("second", "2");
// Insertion order is preserved by ordered_map, so we can check it directly.
m = m;
REQUIRE(m.size() == 2);
auto it = m.begin();
CHECK(it->first == "first");
CHECK(it->second == "1");
++it;
CHECK(it->first == "second");
CHECK(it->second == "2");
}
#if defined(__clang__) && defined(__has_warning)
#if __has_warning("-Wself-assign-overloaded")
DOCTEST_CLANG_SUPPRESS_WARNING_POP
#endif
#endif
TEST_CASE("regression test #5122 - nlohmann::ordered_map move-assignment transfers contents")
{
nlohmann::ordered_map<std::string, std::string> src;
src.emplace("first", "1");
src.emplace("second", "2");
nlohmann::ordered_map<std::string, std::string> dst;
dst.emplace("stale", "x");
dst = std::move(src);
REQUIRE(dst.size() == 2);
auto it = dst.begin();
CHECK(it->first == "first");
CHECK(it->second == "1");
++it;
CHECK(it->first == "second");
CHECK(it->second == "2");
// Re-assigning into the moved-from object must leave it in a usable state.
src = nlohmann::ordered_map<std::string, std::string> {};
src.emplace("after-move", "3");
REQUIRE(src.size() == 1);
CHECK(src.begin()->first == "after-move");
}
// Stand-in for a third-party library (e.g., Eigen as of 3.4, which added
// STL-compatible begin()/end() to its vector types), living in its own
// namespace with its own to_json overload for its vector type.
namespace issue_4320_eigen
{
// "array-compatible" from the library's point of view (it has begin()/end()),
// but for which this (fake) third-party namespace provides its own to_json.
struct vector3
{
double v[3]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays,cppcoreguidelines-use-default-member-init,modernize-use-default-member-init)
vector3(double x, double y, double z) : v{x, y, z} {} // NOLINT(hicpp-member-init,cppcoreguidelines-pro-type-member-init)
double x() const
{
return v[0];
}
double y() const
{
return v[1];
}
double z() const
{
return v[2];
}
double* begin()
{
return v;
}
double* end()
{
return v + 3;
}
const double* begin() const
{
return v;
}
const double* end() const
{
return v + 3;
}
};
inline void to_json(json& j, const vector3& v) // NOLINT(misc-use-internal-linkage)
{
j = {{"x", v.x()}, {"y", v.y()}, {"z", v.z()}};
}
} // namespace issue_4320_eigen
// The user's own namespace, using the (fake) Eigen type as an implementation
// detail behind a payload type that has nothing to do with vectors/arrays.
namespace issue_4320
{
// Publicly derives from issue_4320_eigen::vector3 but does *not* define its
// own to_json - it is only ever used as a temporary to reach the base
// class's to_json via ADL.
struct vector3_wrapper : issue_4320_eigen::vector3
{
using issue_4320_eigen::vector3::vector3;
};
struct payload
{
double x, y, z;
};
inline vector3_wrapper to_eigen(const payload& p) // NOLINT(misc-use-internal-linkage)
{
return {p.x, p.y, p.z};
}
inline void to_json(json& j, const payload& p) // NOLINT(misc-use-internal-linkage)
{
// Unqualified call, passing a *derived* vector3_wrapper: relies on ADL
// finding issue_4320_eigen::to_json(json&, const vector3&) through the
// vector3 base class, via a derived-to-base conversion. Must NOT resolve
// to the library's own generic array-compatible to_json (an exact-match
// template for vector3_wrapper, since it also has begin()/end()), which
// would serialize this as [x, y, z] instead of {"x":x, "y":y, "z":z}.
to_json(j, to_eigen(p));
}
} // namespace issue_4320
TEST_CASE("issue #4320 - custom base class must not leak nlohmann::detail into ADL")
{
// Before the fix, basic_json unconditionally derived from a type living in
// nlohmann::detail (json_default_base), which made nlohmann::detail an
// associated namespace of every basic_json for ADL purposes. That leaked
// the library's internal generic-array to_json overload into unqualified
// to_json() calls made from user code, silently bypassing user-defined
// to_json overloads reached via a derived-to-base conversion.
const issue_4320::payload p{1.0, 2.0, 3.0};
json j;
to_json(j, p);
CHECK(j == json({{"x", 1.0}, {"y", 2.0}, {"z", 3.0}}));
}
TEST_CASE("issue #5338 - truncated CBOR tagged binary subtype is rejected")
{
const std::vector<std::vector<std::uint8_t>> truncated_tags =
{
{0xD8},
{0xD9, 0x00},
{0xDA, 0x00, 0x00, 0x00},
{0xDB, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}
};
for (const auto& data : truncated_tags)
{
CAPTURE(data);
for (const auto tag_handler :
{
json::cbor_tag_handler_t::ignore, json::cbor_tag_handler_t::store
})
{
CAPTURE(tag_handler);
const auto result = json::from_cbor(data, true, false, tag_handler);
CHECK(result.is_discarded());
}
}
}
TEST_CASE("issue #5402 - update(merge_objects=true) overwrites a primitive with an object")
{
json t = {{"k", 1}};
t.update(json{{"k", {{"x", 2}}}}, true);
CHECK(t == json({{"k", {{"x", 2}}}}));
json mixed = {{"keep", {{"a", 1}}}, {"replace", 1}};
mixed.update(json{{"keep", {{"b", 2}}}, {"replace", {{"x", 2}}}}, true);
CHECK(mixed == json({{"keep", {{"a", 1}, {"b", 2}}}, {"replace", {{"x", 2}}}}));
}
TEST_CASE("regression test - parser callback must not lose a duplicate key's prior value")
{
// a callback that rejects only the scalar value 2
const json::parser_callback_t drop_value_2 = [](int /*depth*/, json::parse_event_t ev, json & v) noexcept
{
return !(ev == json::parse_event_t::value && v == 2);
};
SECTION("duplicate key, second (scalar) value rejected - prior value is restored")
{
const json j = json::parse(R"({"a":1,"a":2})", drop_value_2);
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key, second value is an object rejected at object_end - prior value is restored")
{
const json j = json::parse(R"({"a":1,"a":{"x":2}})",
[](int depth, json::parse_event_t ev, json& /*parsed*/) noexcept
{
return !(ev == json::parse_event_t::object_end && depth == 1);
});
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key, second value is an array rejected at array_end - prior value is restored")
{
const json j = json::parse(R"({"a":1,"a":[9,9]})",
[](int depth, json::parse_event_t ev, json& /*parsed*/) noexcept
{
return !(ev == json::parse_event_t::array_end && depth == 1);
});
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key, second value accepted (scalar) - last value wins")
{
const json j = json::parse(R"({"a":1,"a":2})", [](int, json::parse_event_t, json&) noexcept
{
return true;
});
CHECK(j.dump() == "{\"a\":2}");
}
SECTION("duplicate key, second value accepted (object) - last value wins")
{
const json j = json::parse(R"({"a":1,"a":{"x":2}})", [](int, json::parse_event_t, json&) noexcept
{
return true;
});
CHECK(j.dump() == "{\"a\":{\"x\":2}}");
}
SECTION("brand new (non-duplicate) key, value rejected - member is fully absent")
{
const json j = json::parse(R"({"a":1,"b":2})", drop_value_2);
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key nested two levels deep")
{
const json j = json::parse(R"({"outer":{"a":1,"a":2}})", drop_value_2);
CHECK(j.dump() == "{\"outer\":{\"a\":1}}");
}
SECTION("three occurrences of the same key - middle rejected, last accepted")
{
const json j = json::parse(R"({"k":1,"k":2,"k":3})", drop_value_2);
CHECK(j.dump() == "{\"k\":3}");
}
}
TEST_CASE("regression test - excessive binary container size honors allow_exceptions=false")
{
// CBOR array with declared length 2^63
const std::vector<std::uint8_t> cbor = {0x9b, 0x80, 0, 0, 0, 0, 0, 0, 0};
// CBOR map with declared length 2^63
const std::vector<std::uint8_t> cbor_m = {0xbb, 0x80, 0, 0, 0, 0, 0, 0, 0};
// UBJSON array with declared length 2^63-1
const std::vector<std::uint8_t> ubj = {'[', '#', 'L', 0x7f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
// BJData array with declared length 2^63-1 (little endian)
const std::vector<std::uint8_t> bjd = {'[', '#', 'L', 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f};
// allow_exceptions=false must report failure instead of throwing/aborting
CHECK(json::from_cbor(cbor, true, false).is_discarded());
CHECK(json::from_cbor(cbor_m, true, false).is_discarded());
CHECK(json::from_ubjson(ubj, true, false).is_discarded());
CHECK(json::from_bjdata(bjd, true, false).is_discarded());
// allow_exceptions=true (the default) must still throw exactly as before.
// The exact message text is not checked here: on platforms where
// std::size_t is 32-bit, the CBOR reader's own length-narrowing check
// (get_cbor_container_size(), unrelated to this fix) intercepts a
// declared length of 2^63 before it ever reaches the check this test
// targets, with different (but equally valid, and already correct)
// wording -- see unit-cbor.cpp for coverage of that message.
json _;
CHECK_THROWS_AS(_ = json::from_cbor(cbor), json::out_of_range);
// regression guard: a genuinely truncated CBOR input must remain discarded
CHECK(json::from_cbor(std::vector<std::uint8_t> {0x9b, 0, 0, 0, 0, 0, 0, 0, 0x02}, true, false).is_discarded());
}
DOCTEST_CLANG_SUPPRESS_WARNING_POP
+899
View File
@@ -0,0 +1,899 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// cmake/test.cmake selects the C++ standard versions with which to build a
// unit test based on the presence of JSON_HAS_CPP_<VERSION> macros.
// When using macros that are only defined for particular versions of the standard
// (e.g., JSON_HAS_FILESYSTEM for C++17 and up), please mention the corresponding
// version macro in a comment close by, like this:
// JSON_HAS_CPP_<VERSION> (do not remove; see note at top of file)
#include "doctest_compatibility.h"
// for some reason including this after the json header leads to linker errors with VS 2017...
#include <locale>
#define JSON_TESTS_PRIVATE
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
#ifdef JSON_TEST_NO_GLOBAL_UDLS
using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
#endif
#include <cstdio>
#include <list>
#include <type_traits>
#include <utility>
#ifdef JSON_HAS_CPP_17
#include <any>
#include <variant>
#endif
#ifdef JSON_HAS_CPP_17
#if __has_include(<optional>)
#include <optional>
#elif __has_include(<experimental/optional>)
#endif
/////////////////////////////////////////////////////////////////////
// for #4804
/////////////////////////////////////////////////////////////////////
using json_4804 = nlohmann::basic_json<std::map, // ObjectType
std::vector, // ArrayType
std::string, // StringType
bool, // BooleanType
std::int64_t, // NumberIntegerType
std::uint64_t, // NumberUnsignedType
double, // NumberFloatType
std::allocator, // AllocatorType
nlohmann::adl_serializer, // JSONSerializer
std::vector<std::byte>, // BinaryType
void // CustomBaseClass
>;
#endif
#ifdef JSON_HAS_CPP_20
#if __has_include(<span>)
#include <span>
#endif
#endif
/////////////////////////////////////////////////////////////////////
// for #4825 - explicitly instantiating basic_json must compile; this
// forces instantiation of binary_writer::write_bjdata_ndarray, whose
// static_cast<string_t> was ambiguous under explicit instantiation on
// C++17. Merely compiling this translation unit is the regression test.
/////////////////////////////////////////////////////////////////////
template class nlohmann::basic_json<>;
/////////////////////////////////////////////////////////////////////
// for #4440
/////////////////////////////////////////////////////////////////////
#if JSON_HAS_RANGES == 1
#include <ranges>
#endif
// NLOHMANN_JSON_SERIALIZE_ENUM uses a static std::pair
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors")
/////////////////////////////////////////////////////////////////////
// for #3077
/////////////////////////////////////////////////////////////////////
class FooAlloc
{};
class Foo
{
public:
explicit Foo(const FooAlloc& /* unused */ = FooAlloc()) {}
bool value = false;
};
class FooBar
{
public:
Foo foo{}; // NOLINT(readability-redundant-member-init)
};
inline void from_json(const nlohmann::json& j, FooBar& fb) // NOLINT(misc-use-internal-linkage)
{
j.at("value").get_to(fb.foo.value);
}
/////////////////////////////////////////////////////////////////////
// for #3171
/////////////////////////////////////////////////////////////////////
struct for_3171_base // NOLINT(cppcoreguidelines-special-member-functions)
{
for_3171_base(const std::string& /*unused*/ = {}) {}
virtual ~for_3171_base();
for_3171_base(const for_3171_base& other) // NOLINT(hicpp-use-equals-default,modernize-use-equals-default)
: str(other.str)
{}
for_3171_base& operator=(const for_3171_base& other)
{
if (this != &other)
{
str = other.str;
}
return *this;
}
for_3171_base(for_3171_base&& other) noexcept
: str(std::move(other.str))
{}
for_3171_base& operator=(for_3171_base&& other) noexcept
{
if (this != &other)
{
str = std::move(other.str);
}
return *this;
}
virtual void _from_json(const json& j)
{
j.at("str").get_to(str);
}
std::string str{}; // NOLINT(readability-redundant-member-init)
};
for_3171_base::~for_3171_base() = default;
struct for_3171_derived : public for_3171_base
{
for_3171_derived() = default;
~for_3171_derived() override;
explicit for_3171_derived(const std::string& /*unused*/) { }
for_3171_derived(const for_3171_derived& other) // NOLINT(hicpp-use-equals-default,modernize-use-equals-default)
: for_3171_base(other)
{}
for_3171_derived& operator=(const for_3171_derived& other)
{
if (this != &other)
{
for_3171_base::operator=(other); // Call base class assignment operator
}
return *this;
}
for_3171_derived(for_3171_derived&& other) noexcept
: for_3171_base(std::move(other))
{}
for_3171_derived& operator=(for_3171_derived&& other) noexcept
{
if (this != &other)
{
for_3171_base::operator=(std::move(other)); // Call base class move assignment operator
}
return *this;
}
};
for_3171_derived::~for_3171_derived() = default;
inline void from_json(const json& j, for_3171_base& tb) // NOLINT(misc-use-internal-linkage)
{
tb._from_json(j);
}
/////////////////////////////////////////////////////////////////////
// for #3312
/////////////////////////////////////////////////////////////////////
#ifdef JSON_HAS_CPP_20
struct for_3312
{
std::string name;
};
inline void from_json(const json& j, for_3312& obj) // NOLINT(misc-use-internal-linkage)
{
j.at("name").get_to(obj.name);
}
#endif
/////////////////////////////////////////////////////////////////////
// for #3204
/////////////////////////////////////////////////////////////////////
struct for_3204_foo
{
for_3204_foo() = default;
explicit for_3204_foo(std::string /*unused*/) {} // NOLINT(performance-unnecessary-value-param)
};
struct for_3204_bar
{
enum constructed_from_t // NOLINT(cppcoreguidelines-use-enum-class)
{
constructed_from_none = 0,
constructed_from_foo = 1,
constructed_from_json = 2
};
explicit for_3204_bar(std::function<void(for_3204_foo)> /*unused*/) noexcept // NOLINT(performance-unnecessary-value-param)
: constructed_from(constructed_from_foo) {}
explicit for_3204_bar(std::function<void(json)> /*unused*/) noexcept // NOLINT(performance-unnecessary-value-param)
: constructed_from(constructed_from_json) {}
constructed_from_t constructed_from = constructed_from_none;
};
/////////////////////////////////////////////////////////////////////
// for #3333
/////////////////////////////////////////////////////////////////////
struct for_3333 final
{
for_3333(int x_ = 0, int y_ = 0) : x(x_), y(y_) {}
template <class T>
for_3333(const T& /*unused*/)
{
CHECK(false);
}
int x = 0;
int y = 0;
};
template <>
inline for_3333::for_3333(const json& j)
: for_3333(j.value("x", 0), j.value("y", 0))
{}
/////////////////////////////////////////////////////////////////////
// for #3810
/////////////////////////////////////////////////////////////////////
struct Example_3810
{
int bla{};
Example_3810() = default;
};
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(Example_3810, bla) // NOLINT(misc-use-internal-linkage)
/////////////////////////////////////////////////////////////////////
// for #4740
/////////////////////////////////////////////////////////////////////
#ifdef JSON_HAS_CPP_17
struct Example_4740
{
std::optional<std::string> host = std::nullopt;
std::optional<int> port = std::nullopt;
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT(Example_4740, host, port)
};
#endif
TEST_CASE("regression tests 3")
{
#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
// JSON_HAS_CPP_17 (do not remove; see note at top of file)
SECTION("issue #3070 - Version 3.10.3 breaks backward-compatibility with 3.10.2 ")
{
nlohmann::detail::std_fs::path text_path("/tmp/text.txt");
const json j(text_path);
const auto j_path = j.get<nlohmann::detail::std_fs::path>();
CHECK(j_path == text_path);
#if DOCTEST_CLANG || DOCTEST_GCC >= DOCTEST_COMPILER(8, 4, 0)
// only known to work on Clang and GCC >=8.4
CHECK_THROWS_WITH_AS(nlohmann::detail::std_fs::path(json(1)), "[json.exception.type_error.302] type must be string, but is number", json::type_error);
#endif
}
#endif
SECTION("issue #3077 - explicit constructor with default does not compile")
{
json j;
j[0]["value"] = true;
std::vector<FooBar> foo;
j.get_to(foo);
}
SECTION("issue #3108 - ordered_json doesn't support range based erase")
{
ordered_json j = {1, 2, 2, 4};
auto last = std::unique(j.begin(), j.end());
j.erase(last, j.end());
CHECK(j.dump() == "[1,2,4]");
j.erase(std::remove_if(j.begin(), j.end(), [](const ordered_json & val)
{
return val == 2;
}), j.end());
CHECK(j.dump() == "[1,4]");
}
SECTION("issue #3343 - json and ordered_json are not interchangeable")
{
json::object_t jobj({ { "product", "one" } });
ordered_json::object_t ojobj({{"product", "one"}});
auto jit = jobj.begin();
auto ojit = ojobj.begin();
CHECK(jit->first == ojit->first);
CHECK(jit->second.get<std::string>() == ojit->second.get<std::string>());
}
SECTION("issue #3171 - if class is_constructible from std::string wrong from_json overload is being selected, compilation failed")
{
const json j{{ "str", "value"}};
// failed with: error: no match for operator= (operand types are for_3171_derived and const nlohmann::basic_json<>::string_t
// {aka const std::__cxx11::basic_string<char>})
// s = *j.template get_ptr<const typename BasicJsonType::string_t*>();
auto td = j.get<for_3171_derived>();
CHECK(td.str == "value");
}
#ifdef JSON_HAS_CPP_20
SECTION("issue #3312 - Parse to custom class from unordered_json breaks on G++11.2.0 with C++20")
{
// see test for #3171
const ordered_json j = {{"name", "class"}};
for_3312 obj{};
j.get_to(obj);
CHECK(obj.name == "class");
}
#endif
#if defined(JSON_HAS_CPP_17) && JSON_USE_IMPLICIT_CONVERSIONS
SECTION("issue #3428 - Error occurred when converting nlohmann::json to std::any")
{
const json j;
const std::any a1 = j;
std::any&& a2 = j;
CHECK(a1.type() == typeid(j));
CHECK(a2.type() == typeid(j));
}
#endif
SECTION("issue #3204 - ambiguous regression")
{
const for_3204_bar bar_from_foo([](for_3204_foo) noexcept {}); // NOLINT(performance-unnecessary-value-param)
const for_3204_bar bar_from_json([](json) noexcept {}); // NOLINT(performance-unnecessary-value-param)
CHECK(bar_from_foo.constructed_from == for_3204_bar::constructed_from_foo);
CHECK(bar_from_json.constructed_from == for_3204_bar::constructed_from_json);
}
SECTION("issue #3333 - Ambiguous conversion from nlohmann::basic_json<> to custom class")
{
const json j
{
{"x", 1},
{"y", 2}
};
const for_3333 p = j;
CHECK(p.x == 1);
CHECK(p.y == 2);
}
SECTION("issue #3810 - ordered_json doesn't support construction from C array of custom type")
{
Example_3810 states[45]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
// fix "not used" warning
states[0].bla = 1;
const auto* const expected = R"([{"bla":1},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0},{"bla":0}])";
// This works:
nlohmann::json j;
j["test"] = states;
CHECK(j["test"].dump() == expected);
// This doesn't compile:
nlohmann::ordered_json oj;
oj["test"] = states;
CHECK(oj["test"].dump() == expected);
}
#ifdef JSON_HAS_CPP_17
SECTION("issue #4740 - build issue with std::optional")
{
const auto t1 = Example_4740();
const auto j1 = nlohmann::json(t1);
CHECK(j1.dump() == "{\"host\":null,\"port\":null}");
const auto t2 = j1.get<Example_4740>();
CHECK(!t2.host.has_value());
CHECK(!t2.port.has_value());
// improve coverage
auto t3 = Example_4740();
t3.port = 80;
t3.host = "example.com";
const auto j2 = nlohmann::json(t3);
CHECK(j2.dump() == "{\"host\":\"example.com\",\"port\":80}");
const auto t4 = j2.get<Example_4740>();
CHECK(t4.host.has_value());
CHECK(t4.port.has_value());
}
#endif
#if !defined(_MSVC_LANG)
// MSVC returns garbage on invalid enum values, so this test is excluded
// there.
SECTION("issue #4762 - json exception 302 with unhelpful explanation : type must be number, but is number")
{
// In #4762, the main issue was that a json object with an invalid type
// returned "number" as type_name(), because this was the default case.
// This test makes sure we now return "invalid" instead.
json j;
j.m_data.m_type = static_cast<json::value_t>(100); // NOLINT(clang-analyzer-optin.core.EnumCastOutOfRange)
CHECK(j.type_name() == "invalid");
}
#endif
#ifdef JSON_HAS_CPP_17
SECTION("issue #4804: from_cbor incompatible with std::vector<std::byte> as binary_t")
{
const std::vector<std::uint8_t> data = {0x80};
const auto decoded = json_4804::from_cbor(data);
CHECK((decoded == json_4804::array()));
}
SECTION("discussion #4209 - custom BinaryType direct assignment and round-tripping")
{
// Test that assigning a custom BinaryType directly creates a binary value, not an array
const std::vector<std::byte> original{std::byte{1}, std::byte{2}, std::byte{3}};
const json_4804 j = original;
CHECK(j.is_binary());
CHECK(!j.is_array());
// Test round-tripping: extracting the binary value back as the custom container type
const auto extracted = j.get<std::vector<std::byte>>();
CHECK(extracted == original);
// Test that the default json alias behavior is unchanged: std::vector<uint8_t> -> array
const json default_json = std::vector<std::uint8_t> {1, 2, 3};
CHECK(default_json.is_array());
CHECK(!default_json.is_binary());
}
SECTION("discussion #4209 - custom BinaryType extraction from parsed array")
{
// Test that extracting a custom BinaryType from a parsed JSON array still works
// (not just from a binary-typed node)
const auto j = json_4804::parse("[1,2,3]");
CHECK(j.is_array());
CHECK(!j.is_binary());
// Extracting as custom BinaryType should work from arrays
const auto extracted = j.get<std::vector<std::byte>>();
CHECK(extracted.size() == 3);
CHECK(extracted[0] == std::byte{1});
CHECK(extracted[1] == std::byte{2});
CHECK(extracted[2] == std::byte{3});
}
SECTION("issue #5046 - implicit conversion of return json to std::optional no longer implicit")
{
const json jval{};
auto GetValue = [](const json & valRoot) -> std::optional<json>
{
if (valRoot.contains("default"))
{
return valRoot.at("default");
}
return std::nullopt;
};
auto result = GetValue(jval);
CHECK(!result.has_value());
}
#endif
#if JSON_HAS_RANGES == 1
SECTION("issue #4440 - assert when using std::views::filter and GCC 10")
{
auto noOpFilter = std::views::filter([](auto&&) noexcept
{
return true;
});
json j = {1, 2, 3};
auto filtered = j | noOpFilter;
CHECK(*filtered.begin() == 1);
}
#endif
#if JSON_HAS_RANGES && !defined(__MINGW32__)
SECTION("issue #4916 - constructing array from C++20 ranges view does not work")
{
std::vector<int> nums{1, 2, 37, 42, 21};
auto filteredNums = nums | std::views::filter([](int i)
{
return i > 10;
});
json const j(filteredNums);
CHECK(j.type() == json::value_t::array);
CHECK(j == json({37, 42, 21}));
}
#endif
// owning_view is not available in libstdc++ < 12
#if JSON_HAS_RANGES && !defined(__MINGW32__) && !(defined(__GLIBCXX__) && _GLIBCXX_RELEASE < 12)
SECTION("issue #4916 - constructing array from prvalue C++20 ranges view (owning_view)")
{
json const j(std::vector<int> {1, 2, 37, 42, 21} | std::views::filter([](int i)
{
return i > 10;
}));
CHECK(j.type() == json::value_t::array);
CHECK(j == json({37, 42, 21}));
}
#endif
#if JSON_HAS_RANGES && !defined(__MINGW32__)
SECTION("issue #4916 - constructing array from C++20 transform view (prvalue elements)")
{
std::vector<int> nums{1, 2, 3};
auto t = nums | std::views::transform([](int i) noexcept
{
return i * 2;
});
json const j(t);
CHECK(j.type() == json::value_t::array);
CHECK(j == json({2, 4, 6}));
}
#endif
}
TEST_CASE_TEMPLATE("issue #4798 - nlohmann::json::to_msgpack() encode float NaN as double", T, double, float) // NOLINT(readability-math-missing-parentheses, bugprone-throwing-static-initialization)
{
// With issue #4798, we encode NaN, infinity, and -infinity as float instead
// of double to allow for smaller encodings.
const json jx = std::numeric_limits<T>::quiet_NaN();
const json jy = std::numeric_limits<T>::infinity();
const json jz = -std::numeric_limits<T>::infinity();
/////////////////////////////////////////////////////////////////////////
// MessagePack
/////////////////////////////////////////////////////////////////////////
// expected MessagePack values
const std::vector<std::uint8_t> msgpack_x = {{0xCA, 0x7F, 0xC0, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_y = {{0xCA, 0x7F, 0x80, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_z = {{0xCA, 0xFF, 0x80, 0x00, 0x00}};
CHECK(json::to_msgpack(jx) == msgpack_x);
CHECK(json::to_msgpack(jy) == msgpack_y);
CHECK(json::to_msgpack(jz) == msgpack_z);
CHECK(std::isnan(json::from_msgpack(msgpack_x).get<T>()));
CHECK(json::from_msgpack(msgpack_y).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_msgpack(msgpack_z).get<T>() == -std::numeric_limits<T>::infinity());
// Make sure the other MessagePakc encodings for NaN, infinity, and
// -infinity are still supported.
const std::vector<std::uint8_t> msgpack_x_2 = {{0xCB, 0x7F, 0xF8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_y_2 = {{0xCB, 0x7F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> msgpack_z_2 = {{0xCB, 0xFF, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
CHECK(std::isnan(json::from_msgpack(msgpack_x_2).get<T>()));
CHECK(json::from_msgpack(msgpack_y_2).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_msgpack(msgpack_z_2).get<T>() == -std::numeric_limits<T>::infinity());
/////////////////////////////////////////////////////////////////////////
// CBOR
/////////////////////////////////////////////////////////////////////////
// expected CBOR values
const std::vector<std::uint8_t> cbor_x = {{0xF9, 0x7E, 0x00}};
const std::vector<std::uint8_t> cbor_y = {{0xF9, 0x7C, 0x00}};
const std::vector<std::uint8_t> cbor_z = {{0xF9, 0xfC, 0x00}};
CHECK(json::to_cbor(jx) == cbor_x);
CHECK(json::to_cbor(jy) == cbor_y);
CHECK(json::to_cbor(jz) == cbor_z);
CHECK(std::isnan(json::from_cbor(cbor_x).get<T>()));
CHECK(json::from_cbor(cbor_y).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_cbor(cbor_z).get<T>() == -std::numeric_limits<T>::infinity());
// Make sure the other CBOR encodings for NaN, infinity, and -infinity are
// still supported.
const std::vector<std::uint8_t> cbor_x_2 = {{0xFA, 0x7F, 0xC0, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_y_2 = {{0xFA, 0x7F, 0x80, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_z_2 = {{0xFA, 0xFF, 0x80, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_x_3 = {{0xFB, 0x7F, 0xF8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_y_3 = {{0xFB, 0x7F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
const std::vector<std::uint8_t> cbor_z_3 = {{0xFB, 0xFF, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}};
CHECK(std::isnan(json::from_cbor(cbor_x_2).get<T>()));
CHECK(json::from_cbor(cbor_y_2).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_cbor(cbor_z_2).get<T>() == -std::numeric_limits<T>::infinity());
CHECK(std::isnan(json::from_cbor(cbor_x_3).get<T>()));
CHECK(json::from_cbor(cbor_y_3).get<T>() == std::numeric_limits<T>::infinity());
CHECK(json::from_cbor(cbor_z_3).get<T>() == -std::numeric_limits<T>::infinity());
}
TEST_CASE("regression test #5074 - portable workaround for single-element brace init")
{
json const j_obj = {{"key", "value"}};
json const j = json::array({j_obj});
CHECK(j.is_array());
CHECK(j.size() == 1);
CHECK(j[0] == j_obj);
}
#if defined(JSON_BRACE_INIT_COPY_SEMANTICS) && (JSON_BRACE_INIT_COPY_SEMANTICS == 1)
TEST_CASE("regression test #5074 - single-element brace init with JSON_BRACE_INIT_COPY_SEMANTICS")
{
// with JSON_BRACE_INIT_COPY_SEMANTICS: single-element brace init copies/moves
json const j_obj = {{"key", "value"}, {"num", 42}};
json const j_arr = {1, 2, 3};
// object: brace init copies instead of wrapping
json const j1{j_obj};
CHECK(j1.is_object());
CHECK(j1 == j_obj);
// array: brace init copies instead of wrapping
json const j2{j_arr};
CHECK(j2.is_array());
CHECK(j2.size() == 3);
CHECK(j2 == j_arr);
// primitives still work as initializer lists
json const j3{true};
CHECK(j3.is_boolean());
json const j4{42};
CHECK(j4.is_number_integer());
}
#endif
struct Example_5122
{
float b = 2;
nlohmann::ordered_map<std::string, std::string> c{}; // NOLINT(readability-redundant-member-init): needed for GCC -Weffc++
int a = 1;
NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT(Example_5122, b, c, a)
};
TEST_CASE("regression test #5122 - from_json into types holding nlohmann::ordered_map")
{
Example_5122 src;
src.c.emplace("first", "1");
src.c.emplace("second", "2");
ordered_json const j = src;
Example_5122 const dst = j.get<Example_5122>();
CHECK(dst.b == src.b);
CHECK(dst.a == src.a);
REQUIRE(dst.c.size() == src.c.size());
auto src_it = src.c.begin();
auto dst_it = dst.c.begin();
for (; src_it != src.c.end(); ++src_it, ++dst_it)
{
CHECK(dst_it->first == src_it->first);
CHECK(dst_it->second == src_it->second);
}
}
// -Wself-assign-overloaded was introduced in Clang 7. Gate the pragma on
// __has_warning so older Clang versions do not error with "unknown warning
// group". The __has_warning check has to stay inside the __clang__ branch
// because GCC does not provide it and would tokenize-error on the argument.
#if defined(__clang__) && defined(__has_warning)
#if __has_warning("-Wself-assign-overloaded")
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
DOCTEST_CLANG_SUPPRESS_WARNING("-Wself-assign-overloaded")
#endif
#endif
TEST_CASE("regression test #5122 - nlohmann::ordered_map copy-assignment is self-assignment safe")
{
nlohmann::ordered_map<std::string, std::string> m;
m.emplace("first", "1");
m.emplace("second", "2");
// Insertion order is preserved by ordered_map, so we can check it directly.
m = m;
REQUIRE(m.size() == 2);
auto it = m.begin();
CHECK(it->first == "first");
CHECK(it->second == "1");
++it;
CHECK(it->first == "second");
CHECK(it->second == "2");
}
#if defined(__clang__) && defined(__has_warning)
#if __has_warning("-Wself-assign-overloaded")
DOCTEST_CLANG_SUPPRESS_WARNING_POP
#endif
#endif
TEST_CASE("regression test #5122 - nlohmann::ordered_map move-assignment transfers contents")
{
nlohmann::ordered_map<std::string, std::string> src;
src.emplace("first", "1");
src.emplace("second", "2");
nlohmann::ordered_map<std::string, std::string> dst;
dst.emplace("stale", "x");
dst = std::move(src);
REQUIRE(dst.size() == 2);
auto it = dst.begin();
CHECK(it->first == "first");
CHECK(it->second == "1");
++it;
CHECK(it->first == "second");
CHECK(it->second == "2");
// Re-assigning into the moved-from object must leave it in a usable state.
src = nlohmann::ordered_map<std::string, std::string> {};
src.emplace("after-move", "3");
REQUIRE(src.size() == 1);
CHECK(src.begin()->first == "after-move");
}
// Stand-in for a third-party library (e.g., Eigen as of 3.4, which added
// STL-compatible begin()/end() to its vector types), living in its own
// namespace with its own to_json overload for its vector type.
namespace issue_4320_eigen
{
// "array-compatible" from the library's point of view (it has begin()/end()),
// but for which this (fake) third-party namespace provides its own to_json.
struct vector3
{
double v[3]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays,cppcoreguidelines-use-default-member-init,modernize-use-default-member-init)
vector3(double x, double y, double z) : v{x, y, z} {} // NOLINT(hicpp-member-init,cppcoreguidelines-pro-type-member-init)
double x() const
{
return v[0];
}
double y() const
{
return v[1];
}
double z() const
{
return v[2];
}
double* begin()
{
return v;
}
double* end()
{
return v + 3;
}
const double* begin() const
{
return v;
}
const double* end() const
{
return v + 3;
}
};
inline void to_json(json& j, const vector3& v) // NOLINT(misc-use-internal-linkage)
{
j = {{"x", v.x()}, {"y", v.y()}, {"z", v.z()}};
}
} // namespace issue_4320_eigen
// The user's own namespace, using the (fake) Eigen type as an implementation
// detail behind a payload type that has nothing to do with vectors/arrays.
namespace issue_4320
{
// Publicly derives from issue_4320_eigen::vector3 but does *not* define its
// own to_json - it is only ever used as a temporary to reach the base
// class's to_json via ADL.
struct vector3_wrapper : issue_4320_eigen::vector3
{
using issue_4320_eigen::vector3::vector3;
};
struct payload
{
double x, y, z;
};
inline vector3_wrapper to_eigen(const payload& p) // NOLINT(misc-use-internal-linkage)
{
return {p.x, p.y, p.z};
}
inline void to_json(json& j, const payload& p) // NOLINT(misc-use-internal-linkage)
{
// Unqualified call, passing a *derived* vector3_wrapper: relies on ADL
// finding issue_4320_eigen::to_json(json&, const vector3&) through the
// vector3 base class, via a derived-to-base conversion. Must NOT resolve
// to the library's own generic array-compatible to_json (an exact-match
// template for vector3_wrapper, since it also has begin()/end()), which
// would serialize this as [x, y, z] instead of {"x":x, "y":y, "z":z}.
to_json(j, to_eigen(p));
}
} // namespace issue_4320
TEST_CASE("issue #4320 - custom base class must not leak nlohmann::detail into ADL")
{
// Before the fix, basic_json unconditionally derived from a type living in
// nlohmann::detail (json_default_base), which made nlohmann::detail an
// associated namespace of every basic_json for ADL purposes. That leaked
// the library's internal generic-array to_json overload into unqualified
// to_json() calls made from user code, silently bypassing user-defined
// to_json overloads reached via a derived-to-base conversion.
const issue_4320::payload p{1.0, 2.0, 3.0};
json j;
to_json(j, p);
CHECK(j == json({{"x", 1.0}, {"y", 2.0}, {"z", 3.0}}));
}
TEST_CASE("issue #5338 - truncated CBOR tagged binary subtype is rejected")
{
const std::vector<std::vector<std::uint8_t>> truncated_tags =
{
{0xD8},
{0xD9, 0x00},
{0xDA, 0x00, 0x00, 0x00},
{0xDB, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}
};
for (const auto& data : truncated_tags)
{
CAPTURE(data);
for (const auto tag_handler :
{
json::cbor_tag_handler_t::ignore, json::cbor_tag_handler_t::store
})
{
CAPTURE(tag_handler);
const auto result = json::from_cbor(data, true, false, tag_handler);
CHECK(result.is_discarded());
}
}
}
TEST_CASE("issue #5402 - update(merge_objects=true) overwrites a primitive with an object")
{
json t = {{"k", 1}};
t.update(json{{"k", {{"x", 2}}}}, true);
CHECK(t == json({{"k", {{"x", 2}}}}));
json mixed = {{"keep", {{"a", 1}}}, {"replace", 1}};
mixed.update(json{{"keep", {{"b", 2}}}, {"replace", {{"x", 2}}}}, true);
CHECK(mixed == json({{"keep", {{"a", 1}, {"b", 2}}}, {"replace", {{"x", 2}}}}));
}
DOCTEST_CLANG_SUPPRESS_WARNING_POP
+166
View File
@@ -2149,6 +2149,172 @@ TEST_CASE("UBJSON")
}
}
TEST_CASE("UBJSON nesting does not consume the call stack")
{
// Containers used to be read by calling back into the value reader once
// per element, so the native call stack grew with the nesting depth of the
// input. '[' alone opens a container, so a payload of repeated '[' crashed
// the process (#5104), as did the optimized forms, which reach the same
// path through a type or size annotation. The containers are kept on a
// heap stack now.
//
// Deeply nested values must not be compared, copied or dumped here: those
// operations are still recursive and would reintroduce the crash.
json _;
SECTION("containers that end at a marker")
{
const std::vector<uint8_t> input(500000, '[');
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.parse_error.110] parse error at byte 500001: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
CHECK(json::from_ubjson(input, true, false).is_discarded());
}
SECTION("containers with a size")
{
std::vector<uint8_t> input;
for (std::size_t i = 0; i < 100000; ++i)
{
input.push_back('[');
input.push_back('#');
input.push_back('i');
input.push_back(1);
}
CHECK_THROWS_AS(_ = json::from_ubjson(input), json::parse_error&);
CHECK(json::from_ubjson(input, true, false).is_discarded());
}
SECTION("containers with a type and a size")
{
// '[' is a permitted optimized type in UBJSON, so each element of such
// a container is itself a container, read without a marker of its own
std::vector<uint8_t> input;
for (std::size_t i = 0; i < 100000; ++i)
{
const std::vector<uint8_t> level = {'[', '$', '[', '#', 'i', 1};
input.insert(input.end(), level.begin(), level.end());
}
CHECK_THROWS_AS(_ = json::from_ubjson(input), json::parse_error&);
CHECK(json::from_ubjson(input, true, false).is_discarded());
}
SECTION("a well-formed deep value is read through the SAX interface")
{
std::vector<uint8_t> input(100000, '[');
input.insert(input.end(), 100000, ']');
SaxCountdown accept_all(1000000);
CHECK(json::sax_parse(input, &accept_all, json::input_format_t::ubjson));
}
SECTION("a well-formed deep value is read into a value")
{
const std::size_t depth = 10000;
std::vector<uint8_t> input(depth, '[');
input.insert(input.end(), depth, ']');
json j = json::from_ubjson(input);
std::size_t measured = 0;
const json* p = &j;
while (p->is_array() && !p->empty())
{
p = &p->front();
++measured;
}
// the innermost array is empty, so the descent stops one level short
CHECK(measured == depth - 1);
}
SECTION("containers are still read the same way")
{
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', ']'})) == json::array());
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', '}'})) == json::object());
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '#', 'i', 0})) == json::array());
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', '#', 'i', 0})) == json::object());
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'i', '#', 'i', 2, 1, 2})) == json({1, 2}));
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '#', 'i', 2, 'i', 1, 'i', 2})) == json({1, 2}));
CHECK(json::from_ubjson(std::vector<uint8_t>({'{', '$', 'i', '#', 'i', 1, 'i', 1, 'a', 1})) == json({{"a", 1}}));
// a no-op is not a value, so a container of them holds none
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'N', '#', 'i', 2})) == json::array());
// sized and unsized forms nested inside one another
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '[', '#', 'i', 2, 'i', 1, 'i', 2, ']'})) == json({{1, 2}}));
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '#', 'i', 1, '[', 'i', 1, ']'})) == json({{1}}));
// an optimized container of containers
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', '[', '#', 'i', 2, 'i', 1, ']', 'i', 2, ']'})) == json({{1}, {2}}));
}
SECTION("BJData containers are still read the same way")
{
// the ND-array wrapper and the binary shortcut are complete values,
// not containers the reader descends into
CHECK(json::from_bjdata(std::vector<uint8_t>({'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5, 6})) ==
json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}}));
CHECK(json::from_bjdata(std::vector<uint8_t>({'[', '$', 'i', '#', 'i', 2, 1, 2})) == json({1, 2}));
CHECK(json::from_bjdata(std::vector<uint8_t>({'[', '[', 'i', 1, ']', ']'})) == json({{1}}));
}
}
TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
{
// An element of type 'Z', 'T' or 'F' is encoded by its marker alone, so an
// optimized array of one of those has no payload and the declared count is
// the only thing deciding how much is allocated. Ten bytes used to produce
// billions of values (#2793); every other type costs at least one byte per
// element and is bounded by the end of the input.
json _;
SECTION("an excessive count is rejected")
{
// 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value
for (const auto marker :
{'Z', 'T', 'F'
})
{
const std::vector<uint8_t> input = {'[', '$', static_cast<uint8_t>(marker), '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF};
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive array size", json::out_of_range&);
CHECK(json::from_ubjson(input, true, false).is_discarded());
}
}
SECTION("ordinary counts are unaffected")
{
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'Z', '#', 'i', 3})) == json({nullptr, nullptr, nullptr}));
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'T', '#', 'i', 2})) == json({true, true}));
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'F', '#', 'i', 2})) == json({false, false}));
// 'N' is a no-op rather than a value, and still yields an empty array
CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'N', '#', 'i', 2})) == json::array());
}
SECTION("a type with a payload is unaffected")
{
// A count past the limit is not rejected for 'U', which costs a byte
// per element and is bounded by the end of the input instead. The
// count is kept just past the limit rather than made huge, because a
// count that also exceeds the array's max_size() is reported as
// out_of_range before the input runs out, and max_size() depends on
// the width of std::size_t.
const std::vector<uint8_t> input = {'[', '$', 'U', '#', 'l', 0x00, 0x10, 0x00, 0x01};
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing UBJSON number: unexpected end of input", json::parse_error&);
CHECK(json::from_ubjson(input, true, false).is_discarded());
}
SECTION("the writer stays within what the reader accepts")
{
// below the limit the optimized form is used and is tiny; above it the
// writer falls back so that the result can still be read back
json const at_limit(1048576, nullptr);
const auto v_at_limit = json::to_ubjson(at_limit, true, true);
CHECK(v_at_limit.size() == 9);
CHECK(v_at_limit.at(1) == '$');
CHECK(json::from_ubjson(v_at_limit) == at_limit);
json const above_limit(1048577, nullptr);
const auto v_above_limit = json::to_ubjson(above_limit, true, true);
CHECK(v_above_limit.at(1) != '$');
CHECK(json::from_ubjson(v_above_limit) == above_limit);
}
}
TEST_CASE("Universal Binary JSON Specification Examples 1")
{
SECTION("Null Value")