An embedded document (record type 0x03) or array (0x04) was read by calling
back into the document reader, which read its element list, which called the
element reader again for the next embedded one. The native call stack
therefore grew with the nesting depth of the input, and about seven bytes buy
a level, so a document of a few hundred kilobytes crashes the process
(#5104). This is the last of the four binary formats to still do that.
Apply the same shape as the other three: open_bson_document() reads the size
prefix and opens the document, parse_bson_element_internal() calls it for both
record types instead of recursing, and parse_bson_internal() loops over the
element list of whichever document is innermost, closing it when its
terminator is reached and resuming the one below.
check_bson_document_size() is unchanged, and so is when it runs: a document is
still measured from the byte before its size prefix to the byte after its
terminator, and still reported before the end event. The frame carries those
two values, which is what a per-document check needs once the reads are
interleaved rather than nested. Nothing else about the element reader changes.
unit-bson passes unchanged. Round trips through to_bson of nested objects,
arrays, arrays of objects and mixed nesting are identical to the previous
commit, as are the errors for a truncated document, an unsupported record
type, a negative size and a size that does not match, including their byte
offsets. A 30,000-level document built by to_bson is now read to completion
where it used to crash.
Note for sequencing: #5185 changes parse_bson_internal(), the element list and
the array reader, which are the functions this commit restructures. It should
land first; this commit then keeps its checks and moves them onto the loop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_ubjson_array() and get_ubjson_object() read their elements by calling
back into the value reader, which called them again for a nested container,
so the native call stack grew with the nesting depth of the input. '[' alone
opens a container, so half a million of them crashes the process before the
input runs out (#5104). The optimized forms reach the same path through a
size or type annotation, and in plain UBJSON '[' and '{' are permitted as the
type of an optimized container, so "[$[#i\x01" repeated nests just as deeply
at six bytes a level.
Both readers now only open their container, and parse_ubjson_internal() loops:
it closes the containers that have ended, claims the next element of the
innermost one, reads its key when it is an object, and works out the marker
of the value to read next. That last part is where the formats differ, and
the loop follows what the four element loops used to do:
- a sized, typed container gives its elements no marker of their own
- a sized, untyped container reads one for each element
- a container that ends at a marker has the byte already, from the test
against ']' or '}'; for an object it is the first byte of the key
The ND-array wrapper and the 'B' binary shortcut stay as they are. Both read
a complete value rather than opening a container, and their elements are
always scalars: BJData does not permit '[' or '{' as an optimized type, which
is also why only plain UBJSON needed the type-marker case above.
A container of no-ops keeps its behaviour of holding no elements while still
announcing its declared size to the SAX parser, by opening it and then
setting its count to zero.
unit-ubjson and unit-bjdata pass unchanged, 1.39 million assertions between
them, and a behaviour comparison against the previous commit over every
container form -- sized, unsized, typed, untyped, empty, no-op, ND-array,
binary, and the forms nested inside one another -- gives identical values,
error codes, messages and byte offsets. 500,000 levels of each vector now
report a parse error instead of crashing, and a well-formed 100,000-level
value is read to completion.
The driver costs about 3 % on parsing 60,000 small objects and one array of a
million integers, for the reason given in the previous commit; reading the
frame once per element rather than per branch halved what it cost before.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_cbor_array() and get_cbor_object() read their elements by calling back
into the value reader, which called them again for a nested container, and a
tag was handled by reading the tagged value the same way. All three cost
native stack, and all three cost a single byte to encode: 0x9F opens an
indefinite-length array, 0x81 a one-element array, and 0xC2 is a tag. Half a
million of any of them crashes the process before the input runs out (#5104).
Apply the shape the MessagePack reader already uses: the open containers live
on the heap stack, parse_cbor_value() reads a single value and only opens a
container rather than reading it to its end, and parse_cbor_internal() loops,
resuming the innermost container after each element.
Two things are specific to CBOR. An indefinite-length container ends at a
break marker rather than at a count, and testing for that marker consumes a
byte which is the first byte of the next element when it is not one; the
frame's count is npos for those, and the driver tracks whether the next value
starts at a fresh byte. And a tag is not a value of its own: instead of
reading the tagged value by recursing, the value reader reports that a tag was
read and the driver reads on, so a chain of tags costs no stack at all.
The switch that decodes a value is unchanged apart from the twelve container
cases and the two tag sites. Verified against the previous commit over
definite and indefinite arrays and maps, all four counted forms, empty
containers, nesting of the forms inside each other, truncated inputs, and all
three tag handlers: identical values, error codes, messages and byte offsets.
500,000 levels of each of the three vectors now report parse_error.110 instead
of crashing, and a well-formed 200,000-level value is read to completion.
On performance: the driver does per element what a counted loop used to do
per container, and CBOR pays for it more than MessagePack because the value
reader also has to be told whether to fetch a byte. Parsing 60,000 small
objects and one array of a million integers is 3 to 4 % slower than the
recursive reader, measured over five alternating runs. Against develop the
same two inputs are about 44 % faster, because the entry point no longer
copies the value it parsed; the earlier commit in this series is what pays
for that.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_msgpack_array() and get_msgpack_object() read their elements by calling
back into parse_msgpack_internal(), which calls them again for a nested
container. The native call stack therefore grew with the nesting depth of the
input, and each level costs only one byte to encode: 0x91 is a one-element
array, so a few hundred thousand of them crash the process before any of the
input is rejected (#5104).
Keep the open containers on a heap stack instead, the way
parser::sax_parse_internal() has always done for JSON text. A frame records
how many elements are left and whether to close with end_object() or
end_array(); parse_msgpack_value() reads a single value and, for a container,
only opens it; and parse_msgpack_internal() loops, resuming the innermost
container after each element and closing it when its count runs out. Whether
the value that was begun is complete is answered by the stack being empty, so
no separate bookkeeping is needed.
The switch that decodes a value is untouched apart from the six container
cases, which now call enter_container() rather than a reader that loops. That
keeps this diff to the control flow and leaves the decoding of every other
type byte-identical.
enter_container() is the only place a binary reader emits start_object() or
start_array(), so a check that rejects a container can be added there once and
is guaranteed to run before the start event. The frame type and the stack are
shared, ready for the other three formats.
Verified against develop over empty, nested, counted (array 16/32, map 16/32)
and truncated inputs: identical values, error codes, messages and byte
offsets. 300,000 levels now report parse_error.110 instead of crashing, and a
well-formed 300,000-level value is read to completion through the SAX
interface, where develop crashes.
Reading such a value into a basic_json needs the return-by-move change as
well, without which the recursive copy constructor overflows on the way out;
that is the parent commit, and the test for the value path covers the two
together. Timing is unchanged: parsing 60,000 small objects and one array of
a million integers is within run-to-run noise of develop either way.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
An element of type 'Z' (null), 'T' (true) or 'F' (false) is encoded by its
type marker alone, so an optimized UBJSON array of one of those has no
payload: reading an element consumes no input at all. Its declared count is
therefore the only thing that decides how much is allocated, and nothing
bounded it. "[$Z#l" and a four-byte count is nine bytes of input describing
two billion values; #2793 reports 35 GB and 150 seconds from ten bytes, and
OSS-Fuzz has an out-of-memory and a timeout report for the same shape.
Every other type costs at least one byte per element, so the end of the input
bounds it. 'N' (no-op) is already skipped rather than stored. Objects are not
affected either: each element is preceded by its key, which costs bytes. And
BJData already refuses these markers as an optimized type, so this is a plain
UBJSON matter.
Reject a count above 1,048,576 elements for those three types with
out_of_range.408, the code this reader already uses for a declared size it
will not honour. The check runs before the SAX start event, so no container
is opened and then abandoned.
Rejecting on the read side alone would break the guarantee that anything
to_ubjson() writes can be read back, and would trip the round-trip assertion
in fuzzer-parse_ubjson.cpp. So the writer falls back to the unoptimized
encoding, one byte per element, for arrays of these types above the same
limit. Its decision depends only on the array's size, which is identical for
a value and for anything parsed back from it, so the round trip is stable.
No existing test changes: the largest such count in the test suite is 65,793.
The excessive-size test that already used this shape still passes, now
rejected a little earlier than by the max_size() check it used to reach.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_ubjson_size_type() takes an inside_ndarray parameter saying whether it is
being called for an ndarray's dimension vector, where another ndarray is not
allowed. It then seeded the flag it passes down to get_ubjson_size_value()
with `false` rather than with that parameter, and only consulted
inside_ndarray afterwards, on the '$' branch.
So on the '#' branch nothing stopped the descent: every "#[" pair of an input
like "[" followed by "#[#[#[..." opened another dimension vector, several
native stack frames deeper each time, and the recursion was only reported on
the way back out. 100,000 pairs crash the process. This is #5104 again, in a
path that has nothing to do with containers.
Seed the flag with inside_ndarray, which is what get_ubjson_size_value()
documents it wants: "for input, `true` means already inside an ndarray vector
or ndarray dimension is not allowed". The nested '[' is then refused where it
is read, so the length of the chain no longer matters.
Both post-checks gain `&& !inside_ndarray`, because an ndarray was found
*here* only if the flag flipped -- get_ubjson_size_value() only ever returns
`true` when its initial value was `false`, as its documentation says. With
that, the "ndarray can not be recursive" branch is unreachable: a recursive
ndarray is now caught one level earlier, and reported as "ndarray dimensional
vector is not allowed" like every other nested dimension vector.
Three existing expectations move accordingly (vR2, vR4, vR6). All three now
fail earlier, and all three now report the same error that vR1, vR5 and vH
already reported for the same shape, which is the more consistent outcome.
Everything else is unchanged: valid 1D and 2D ndarrays, optimized containers
and plain arrays produce identical results, and unit-ubjson is untouched.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_cbor_string() and get_cbor_binary() handled the indefinite-length forms
(0x7F and 0x5F) by calling themselves once per chunk. Each chunk therefore
cost a native stack frame, and since a chunk may itself be an indefinite-
length string, an input of repeated 0x7F bytes reached one frame per input
byte: 200,000 of them crash the process with SIGSEGV before a single byte is
rejected. This is the same defect as #5104, in a path the container-level
work does not touch.
Count the open levels instead of recursing through them. That is enough here
because every chunk is appended to the same result -- get_bytes() writes at
result.size() -- so there is no per-level state to keep. The temporary chunk
string and its copy into the result go away with the recursion.
The definite-length cases move to get_cbor_string_chunk() and
get_cbor_binary_chunk() unchanged, including their error messages, which
still name 0x7F and 0x5F because those are handled one level up.
Behaviour is unchanged. Comparing against develop over the interesting byte
sequences -- empty, single-chunk, nested, over-closed and truncated forms,
both strings and byte arrays, and an indefinite-length map key -- produces
identical values, error codes, messages and byte offsets. The 200,000-level
input now reports parse_error.110 at byte 200001 instead of crashing.
Note that nesting these is not valid CBOR: RFC 8949, Section 3.2.3 forbids
it. This does not change that either way -- it has always been accepted, and
rejecting it is a separate decision (#5317, #5325). Should it be rejected
later, that is now one condition on the level counter rather than a change to
the control flow.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The binary entry points end with
return res ? result : basic_json(value_t::discarded);
The condition operator's second operand is an lvalue, so this is not a case
where the return value can be elided or implicitly moved from: every
successful from_cbor(), from_msgpack(), from_ubjson(), from_bjdata() and
from_bson() call deep-copies the value it just parsed, and then destroys the
original.
The copy is not cheap, and it is not incidental: basic_json's copy
constructor walks the whole value. Parsing a 2 MB CBOR document with 60,000
objects, median of 25 runs, clang 17 -O3:
from_cbor 26.99 ms -> 14.65 ms
from_msgpack 26.82 ms -> 14.82 ms
Moving instead of copying is the entire change; the parsed value is not used
again after the return expression is evaluated.
There is a second reason to prefer the move. The copy constructor recurses
once per nesting level, so the copy is also a stack-overflow path on the
return side, on a value the reader has already accepted. That is currently
masked because the readers themselves recurse and overflow first (#5104), but
it has to be fixed for making them iterative to have any effect.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_bjdata() emitting unparsable output when _ArraySize_ is not an array
write_bjdata_ndarray() never checked that _ArraySize_ is an array. The shape
is written verbatim as the header length, so a null shape emitted 'Z' and an
object shape emitted '{' after the '#', neither of which from_bjdata()
accepts, and the round-trip guarantee in the BJData docs was broken.
Both slipped through the existing validation: for null, empty() is true so
the element count starts at 0 and the per-dimension loop never runs, and for
an object the loop walks its values, which can satisfy the non-negative
integer check. When _ArrayData_ then matched that count, the writer took the
ndarray path.
Require the shape to be an array, so anything else falls back to a plain
object encoding that round-trips, as the fallback rule in the docs already
specifies.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
* Document that _ArraySize_ must be an array in the ndarray requirements
The list at bjdata.md is the exhaustive set of conditions for the ndarray
encoding, but it only implied this one through 'every entry of'.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
---------
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
* docs: add 89 customers and sort all sections
Extend the customers page from 136 to 225 entries. New entries were
found by searching vendor open-source notices and by inspecting
dependency manifests in public repositories.
Every added entry was verified to link either to a page that credits
the library, or to an open source repository where its use is directly
visible (a vendored copy, a build manifest, or an include). Candidates
whose only evidence was a transitive dependency (via ICU or
KDDockWidgets), packaging metadata, or an unused vendored file were
dropped rather than listed.
Also sort every section alphabetically ignoring case, and keep the
Peregrine lunar lander first under "Space Exploration".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: add 41 customers and rebuild the overview image
The list was assembled by searching for names one at a time, which is why it
had plateaued. These additions come from enumerating instead: a public code
search returning every repository that references the library, and the reverse
build-dependencies of the Debian and Ubuntu source indices.
Every addition cites a first-party call site rather than a repository root,
and every evidence URL was checked to resolve. Several prominent candidates
were rejected on exactly that test:
- Node.js reaches the library only through deps/icu-small, the inherited
attribution ICU carries into everything that ships it.
- Visual Studio Code's only hits are copies of the library's own headers
used as sample C++ in the Copilot extension's test fixtures.
- OSS-Fuzz fuzzes the library rather than calling it; projects/json/ is the
library's own OSS-Fuzz integration.
- libuv matched only its AUTHORS file, which lists a contributor by name.
- simdjson and LLVM matched only benchmarks and a mangled-symbol test.
Two entries were already present under a different name and are merged rather
than duplicated: PrestoDB, and DB Browser for SQLite under its repository name.
GitHub CodeQL's citation moves from the repository root to shared/cpp/Diagnostics.h,
which includes the header directly.
The image is rebuilt with 236 logos over 17 rows. It is smaller than the one it
replaces, 1.02 MB against 1.35 MB, despite carrying 100 more marks.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The mixed number arms of JSON_IMPLEMENT_OPERATOR cast the integer to
number_float_t before comparing. Past the float's mantissa that cast is lossy:
2^63-2 and 2^63-1 both round to 2^63, so each compares equal to that float
while differing from each other. Equality is therefore intransitive and the
ordering is not a strict weak ordering, which makes std::sort over such values,
or using them as keys in std::set or std::map, undefined behavior.
Compare the two exactly instead. The integer's range is a power of two the
float represents exactly, so a float outside it is ordered by magnitude alone;
inside it, truncating the float is exact, and the integer parts and then any
fractional part decide. The helper hands back a pair whose comparison with the
original operator reproduces that ordering, which keeps every operator's return
type as it was, including partial_ordering for the spaceship.
A NaN operand is returned in both members, so NaN stays false for the
relational operators and unordered for <=>. Values a float represents exactly
still compare equal, so json(1) == json(1.0) is unchanged.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
When merge_objects is true, recurse only if the existing value is an
object. Otherwise overwrite, matching the documented "all other values
are overwritten as usual" behavior.
Fixes#5402
Signed-off-by: elix3r <157088510+22elix3r@users.noreply.github.com>
A std::pair or std::tuple whose every element is itself a two-element array
with a string first element (e.g. std::pair<std::string, int>) serializes to
a JSON object instead of a JSON array, because to_json builds the value with a
brace initializer and the initializer-list object-detection rule fires. The
resulting object cannot be read back into the original type and collapses
duplicate keys. Document this quirk in the conversions guide, together with the
unaffected cases and the idiom to force an array.
Claude-Session: https://claude.ai/code/session_016cwQq8WQRFzQcGQtbJTtJg
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
* Throw other_error.502 when UBJSON use_type is set without use_size
Fixes#5321
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Scope UBJSON use_type check to container branches and expand tests
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Re-amalgamate single_include/json.hpp
The previous commit updated the split headers but the amalgamated
file didn't go back through astyle before I committed it, so CI's
amalgamation check caught formatting drift in json_fwd.hpp and a
few noexcept clauses in basic_json, plus one doc example. None of
it touches the UBJSON logic. Applied the patch CI generated to
bring single_include back in sync.
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
---------
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Reformat parser_callback_t example with astyle
The file uses "json & /*parsed*/" in three lambda parameter lists, which
astyle rewrites to "json& /*parsed*/" per --align-reference=type. The
drift went unnoticed because CI never format-checked the documentation
examples; "make pretty" does cover them.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Format-check the documentation examples in CI
The examples live in docs/mkdocs/docs/examples, but both format checks
still referenced the long-gone docs/examples path:
- check_amalgamation.yml passed it to find, which printed an error for
the missing path and carried on, so astyle only ever saw include and
tests. The step still exited 0.
- ci.cmake globbed it into INDENT_FILES, and a GLOB_RECURSE over a
missing directory silently yields nothing, so the ci_test_amalgamation
target skipped the examples too.
Either way the 231 example files have never been format-checked. Point
both at the real path, and guard the workflow with an explicit directory
check so a future rename fails the job instead of quietly shrinking the
file list again.
Also drop the dead docs/examples/** path filter from
publish_documentation.yml; docs/mkdocs/** already covers the examples.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CBOR tag handlers not recognizing tags 0-5 and 21-23
The tagged-item switch in binary_reader::parse_cbor_internal() only handled
head bytes 0xC6-0xD4 and 0xD8-0xDB. Bytes 0xC0-0xC5 (tags 0-5: date/time,
epoch, bignum, decimal, bigfloat) and 0xD5-0xD7 (tags 21-23: base64url,
base64, base16 conversion hints) fell through to the default case and were
reported as invalid bytes, even under cbor_tag_handler_t::ignore and ::store,
despite being valid CBOR major-type-6 tags per RFC 8949.
Add the missing case labels so the full 0xC0-0xDB range is handled
uniformly. Extend the "Tagged values" test in unit-cbor.cpp to cover
0xC0-0xD7, and update the CBOR docs to state the corrected tag range.
Fixes#5315
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
* Fix stale CBOR tag docs and add store-mode binary-payload test
The "Incomplete mapping" warning still listed tags 0-5 (date/time,
bignum, decimal fraction, bigfloat) and 21-23 (expected conversions)
as unsupported, even though they now parse correctly under
cbor_tag_handler_t::ignore/store, same as 0xC6..0xD4/0xD8..0xDB.
Remove those five bullets and cross-reference the "Tagged items"
warning below, matching the equivalent docs fix landed independently
in PR #5367.
Also add a cbor_tag_handler_t::store test that wraps a binary
payload (not just a string) for every byte in 0xC0..0xD7, confirming
these tags are unwrapped the same way as 0xC6..0xD4 rather than
mistaken for the 0xD8..0xDB binary-subtype marker syntax, per review
feedback on #5331.
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
---------
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
The block documenting get_char and tag_handler sat above
get_cbor_negative_integer(), which takes neither, so Doxygen attached it
there and parse_cbor_internal() was left undocumented.
Comment placement only.
Signed-off-by: Dmitry <45711841+darkdi@users.noreply.github.com>
The diagnostic position of a string value was derived by subtracting the
parsed value's length from the end position. Escape sequences make the
source token longer than the value it parses to, so the reported start
position landed inside the string, one byte off per escape sequence:
input: {"a":"\n\n\n\n\n\n"}
start_pos() == 11, so the reported range covered n\n\n\n"
instead of the documented "\n\n\n\n\n\n"
This contradicts the documented behavior of start_pos(), which is the
position of the opening quote, and it also corrupted the "(bytes N-M)"
part of JSON_DIAGNOSTICS exception messages. Strings with multi-byte
UTF-8 but no escapes were unaffected, which is why this went unnoticed.
Record the offset of the token in the lexer when it starts scanning and
use that, instead of reconstructing it from the parsed value. Booleans,
null and numbers already reported correct positions and are unchanged.
The new lexer member and accessor are compiled only when
JSON_DIAGNOSTIC_POSITIONS is enabled, which is already part of the ABI
tag, so the default build is unaffected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The comment asked whether the no-op marker 'N' may be ignored when a
string is read. It may not: at that point the next byte must be a string
length type specification, and 'N' is not one. No-ops at positions where
a value may start are already consumed by the callers through
get_ignore_noop(), so nothing is lost by not skipping them here.
Replace the TODO with a comment stating that, and add regression tests
pinning both directions: a no-op is accepted at top level (also
repeated), before and after an array element, and before an object key,
between key and value, and before the closing brace of an object of
unknown size; it is rejected where a length type specification is
expected, i.e. after the 'S' marker of a string value and as the key
length of an object of known size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not write BJData ndarrays whose size overflows std::size_t
write_bjdata_ndarray() multiplied the _ArraySize_ dimensions into a
std::size_t without checking for overflow. A product that wraps around
to a value that happens to match the size of _ArrayData_ passed the
length check, and the writer emitted an ndarray header announcing an
element count that cannot be represented:
{"_ArrayType_":"uint8","_ArraySize_":[9223372036854775808,2],"_ArrayData_":[]}
was encoded as 5b 24 55 23 5b 4d 00 00 00 00 00 00 00 80 69 02 5d, an
ndarray of 2^64 elements followed by no data. Reading that back throws
out_of_range.408 ("excessive ndarray size caused overflow"), so to_bjdata
produced output that from_bjdata rejects. This is reachable by parsing
untrusted JSON and re-encoding it as BJData.
Mirror the overflow check the binary reader already performs, and also
reject a single dimension that does not fit into std::size_t, which the
previous cast silently truncated where std::size_t is narrower than 64
bits. Such objects now fall back to a plain object encoding, which is
what the surrounding type and length validation already does for
annotations it cannot represent, and they round-trip unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document when to_bjdata converts a JData annotation to an ND-array
The BJData page described the 1-D vector case as the only situation in
which an object carrying _ArrayType_/_ArraySize_/_ArrayData_ is not
written as a compact ND-array. The writer has always had several other
fallbacks -- an unknown _ArrayType_, a dimension that is not a
non-negative integer, an _ArrayData_ whose length does not match the
product of the dimensions, and elements that are not numbers of the
annotated kind -- all of which cause the value to be serialized as a
regular JSON object instead.
Spell out the conditions, including the size-overflow check added in the
preceding commit, so the documented behavior matches the implementation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Linking test-regression2_cpp20 intermittently fails with
unit-regression2.cpp.obj:(.debug_info+0x16): relocation truncated to
fit: IMAGE_REL_AMD64_SECREL against `.debug_line'
The failure moves between matrix entries from run to run, and the same
commit can pass and fail on consecutive runs, so it is the size of the
debug sections rather than any one Clang version.
The jobs only build and run the tests, so override CMAKE_CXX_FLAGS_DEBUG
to drop the default -g. Everything else about the Debug build is
unchanged: no optimization flag is added and NDEBUG stays undefined, so
JSON_ASSERT remains active.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conversions whose element count is fixed by the destination C++ type --
`std::pair`, `std::tuple`, `std::array<T, N>`, C arrays, and
`std::map`/`std::unordered_map` with a non-string key -- read exactly the
elements they need via `at` and never compare the JSON array's size to
that number. Excess elements are silently discarded, while a shortfall
throws `out_of_range.401` rather than a `type_error`. Neither direction
was documented in `conversions.md`, `get.md`, or `from_json.md`.
The existing warning covered only `std::array` and stated that a too-short
JSON array leaves the remaining elements default-constructed with no
exception thrown; that is not what happens. Generalize it to all
fixed-size destinations and correct the shortfall direction.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document the complexity of ordered_map operations
ordered_map stores its elements in a std::vector in insertion order and
has no lookup index, so emplace, operator[], at, find, count, erase, and
insert are all linear scans. The documentation stated no complexity for
any operation, neither in ordered_map.md nor in ordered_json.md.
Add a per-operation complexity table and note the consequence: building
or parsing an ordered_json object of n keys is O(n^2). Measured with
-O2 -DNDEBUG for parsing a flat object of n keys, ordered_json is 5x
slower than json at n=2000 and 54x slower at n=16000, with the timings
quadrupling per doubling of n. Cross-reference the table from
ordered_json.md and from the object order page, which recommends
ordered_json without mentioning the cost.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: move the Complexity section after Member functions
scripts/check_structure.py enforces a fixed section order for pages under
docs/mkdocs/docs/api, in which Complexity comes after Member functions.
The section had been placed right after Iterator invalidation, which made
ci_test_build_documentation fail with structure/section_order.
No content change beyond the move; the table columns are realigned to the
narrower content.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document exceptions newly thrown by the binary-format hardening
A round of binary-format input validation (#5274, #5284, #5287, #5332)
added new failure modes without updating exceptions.md, and left two
descriptions factually narrower than the code:
- parse_error.110 said "CBOR or MessagePack"; BSON and UBJSON also
throw it. Generalized, and added the BSON EOF example (#5332).
- parse_error.112: added the BSON document-size mismatch example
(#5287).
- parse_error.113 said "while parsing a map key", but its own existing
UBJSON char example already contradicted that. Broadened to cover
invalid length specifications, and added the negative-string-length
example (#5284).
- out_of_range.408 said "of an UBJSON array or object"; CBOR now throws
it too (#5274). Generalized and added both CBOR examples.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Correct which types may be referenced in a tuple extraction
The note added in #5271 said a referenced type must be one the library
stores "or an arithmetic type it can convert to/from". The parenthetical
is wrong: is_compatible_reference_type requires an exact match against
the stored types, so std::tuple<int&> is rejected by static_assert even
though int converts fine as a value. Only the value case is permissive.
Spell out the eight admissible types, give the int& counter-example, and
separate the reference restriction from by-value conversion.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Follow-up to #5342, which fixed the parser callback leaving a discarded
member behind when an array or a value under an object key was rejected.
The documentation of parser_callback_t only stated that discarded values
in structured types are skipped, without saying that this covers object
parents and that the key is removed along with the value, so there was no
way to tell the fixed behavior from the buggy one.
Spell out the discarding rules, add an example that exercises the cases
the fix repaired, and correct the return value description: a discarded
top-level value is replaced by null, not by "an empty discarded object".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: use HTTPS for the astyle and cppcheck links in README
Both links were still `http://`. `astyle.sourceforge.net` serves HTTPS
directly; `cppcheck.sourceforge.net` redirects to
`https://cppcheck.sourceforge.io`, which is also the URL already used in
`docs/mkdocs/docs/community/quality_assurance.md`, so the redirect is
skipped here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CI: suppress -Wc2y-extensions for Clang
Clang 22.1 (now shipped by silkeh/clang:latest) diagnoses __COUNTER__ as a
C2y extension, and does so in C++ mode as well. Under -Weverything -Werror
this breaks every ci_test_clang_cxx* / ci_test_clang_libcxx_cxx* target,
independently of the code under test.
The library itself does not use __COUNTER__; all diagnostics originate in
vendored Doctest (DOCTEST_ANONYMOUS, used by TEST_CASE and SECTION).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document RFC 8259 / JSONTestSuite compliance and parse() vs operator>> strictness
The compliance story lived only in tests/src/unit-testsuites.cpp, so
drive-by comparisons kept claiming the library "does not fully pass
JSONTestSuite". Make it discoverable:
- README: add a "Standards compliance" note stating that both nst
JSONTestSuite revisions run in CI, that all mandatory y_/n_ cases pass
through the strict parse() entry point, and listing the deliberate
implementation-defined i_ choices (unbounded nesting, silent BOM
stripping, noncharacters forwarded, strict rejection of invalid UTF-8
and lone surrogates, out_of_range.406 on numeric overflow).
- features/parsing: add a "Strictness and trailing data" section
documenting that parse() is strict and rejects trailing data while
operator>> follows relaxed iostream semantics (parses one value and
leaves the stream positioned after it) -- the single place a naive
test yields a "non-compliant" result.
Documentation only; no parser behavior change. Closes#5290.
Signed-off-by: manon <youdie006@users.noreply.github.com>
* docs: correct test-data vendoring and parse()/operator>> claims per review
- README: the JSONTestSuite data is downloaded from nlohmann/json_test_data at
configure time, not vendored/committed; say so.
- README: only the updated suite runs y_ and n_ cases through strict parse();
the original suite's y_ cases go through operator>>. Narrow the claim.
- parsing/index.md and operator_gtgt.md: note that operator>> consumes a number's
terminating byte, so concatenated numbers must be whitespace-separated (1 2
works, 1true does not); structural and literal values are unaffected.
Signed-off-by: manon <youdie006@users.noreply.github.com>
---------
Signed-off-by: manon <youdie006@users.noreply.github.com>
Co-authored-by: manon <youdie006@users.noreply.github.com>
Niels Lohmann
Qatadaha Bin Matloob
Avionic Harshit
dependabot[bot]
elix3r
whn
Krishnanand G
Sahil_Kamate
ljcjclljc
Dmitry
Angadi56
Petr Bělohlávek
KBSandmanon