Both come from instantiating basic_json with containers other than the
default ones, and neither shows up with the Clang-Tidy version available
outside CI:
- insert(const_iterator, basic_json&&) forwards its by-value iterator to
the const-reference overload. performance-unnecessary-value-param asks
for the copy to be a move; it only fires for an iterator that is not
trivially copyable, as std::deque's is not. The NOLINT on the function
does not cover it, because the finding is reported where the parameter
is used rather than where it is declared. Move it, which is what the
check asks for and is a (very small) improvement in its own right.
- cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed
the inherited key_compare member type. An enum class would not do, since
it declares a type of that name and the probe would find it again; a
member function declaration hides the name just as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
iter_impl declared its defaulted move operations noexcept. The exception
specification a defaulted function gets implicitly follows from its members,
here internal_iterator, which holds the object and array iterators. libstdc++
gives std::deque's iterator a user-provided copy constructor without noexcept
before version 11, so the implicit specification is noexcept(false) and does
not match the declared one. That deletes the function -- and with g++ 4.8,
which predates CWG 1778, it is an error outright:
error: function 'iter_impl<basic_json<std::map, std::deque> >::iter_impl(
iter_impl&&)' defaulted on its first declaration with an
exception-specification that differs from the implicit declaration
So std::deque, which this branch documents as a usable array type, could not
be used with an older standard library. Leaving the specification to be
computed cannot mismatch; iteration_proxy_value already spells out the same
condition next door.
The default configuration is unaffected: json::iterator, json::const_iterator
and ordered_json::iterator stay nothrow move constructible and move
assignable, which the test now checks so it cannot regress unnoticed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The two sections added to unit-regression2.cpp brought a third full
basic_json instantiation into a translation unit that was already large.
With Clang on MinGW that pushed the object over the reach of a 32-bit
relocation and test-regression2_cpp20.exe failed to link:
relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'
unit-regression2.cpp is restored to exactly what it was before, and the
coverage moves to unit-custom-binary-type.cpp, next to the object and array
type tests it belongs with. The signed value type is now also covered in
C++11, where std::byte is not available.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception
message, which a parsed value has and an in-memory one does not, so the two
message checks failed in that configuration. Build the array in memory
instead of parsing it; the test is about at(size_type) not needing
array_t::at(), and the byte range is beside the point.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Building the token from data() and size() kept an embedded null byte in the
key, and since what() hands out a C string, that truncated the whole message
rather than just the key: to_bson() on a key containing U+0000 reported
"[json.exception.out_of_range.409] (/en" instead of the full explanation.
This broke test-bson under JSON_DIAGNOSTICS.
Constructing from data() alone stops at the first null byte, which is what
c_str() did before, so the message is unchanged -- without requiring
string_t to provide c_str().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
object_t is probed for key_compare inside the definition of basic_json, so
it is instantiated while basic_json is still incomplete. Whether a hash map
survives that depends on the standard library: libstdc++ 9 needs the size of
the mapped type to instantiate std::unordered_map's node type and rejects
the adapter, which broke the GCC 9 builds.
The test now derives its no-key_compare object type from std::map -- which
does cope -- and shadows the inherited key_compare member type with an
entity that is not a type, so the library's probe finds none, exactly as for
a hash map. The unflatten() order-independence checks in unit-json_pointer
already cover the behaviour that the unordered object type was there for.
The limitation is documented for std::unordered_map.
Also address two Clang-Tidy findings the earlier commits introduced:
erase_from_object() declares its iterator with auto, and at(size_type) checks
the type first and then falls through to the return instead of throwing from
an else branch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
MSVC rejects char(0xFF) with C4310 (cast truncates constant value),
which the Windows workflow treats as an error. The character literals
carry the same byte values without a narrowing cast.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Drop c_str(), back(), find(str, pos), replace(), and substr() from the
StringType requirements and at(size_type) from the ArrayType ones, and
note the string assignment the JSON pointer code performs. Streaming a
json_pointer no longer needs assignability from a std::string.
Add the non-null-terminated data() to the list of violations that are not
diagnosed at compile time -- it was described in the StringType section
but missing from the summary at the top -- and correct the QString row,
which no longer fails for the c_str() it lacks.
JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one
went away with array_t::at(). Describe what the library actually catches.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Several members were required only because of how the library happened to
be written, not because the functionality needs them. Dropping them widens
the set of usable string and array types, and one of them was also a
performance problem.
string_t:
- c_str() is gone. Every call site already knew the length and passed it
along, so data() is enough. The one place that did not, the diagnostics
path in exceptions.hpp, now builds the token from data() and size(),
which also stops it from truncating keys that contain a null byte.
- back() is gone; the serializer indexes the last character instead.
- find(str, pos), replace(), and substr() are gone. escape() and
unescape() rebuilt the string with one replace() per escaped character,
which moves the tail every time: escaping a string of n characters that
all need escaping cost O(n^2). Both now scan with find_first_of() -- a
member the pointer parser already required -- and append whole runs, so
the common case is one search and one copy. Escaping 64000 tildes drops
from 717 ms to 20 ms; a string with nothing to escape gets faster too
(8.4 ms to 5.8 ms), because the scan is still a single memchr per pass.
json_pointer::split() takes its reference tokens with the
(const char*, size_type) constructor rather than substr().
- json_pointer::to_string() accumulates with concat<string_t> instead of
letting concat default to std::string and converting afterwards, so
streaming a json_pointer no longer requires string_t to be assignable
from a std::string.
array_t:
- at(size_type) is gone. basic_json::at(size_type) checked the index by
calling array_t::at() and translating std::out_of_range, which also
required the array type to throw that exact exception. It now compares
against size() and uses operator[]. The thrown exception, its message,
and the behaviour under JSON_NOEXCEPTION are unchanged.
The BSON writer wrote the terminating null byte out of the string's own
buffer (size() + 1). It now writes the byte itself, so string_t::data()
need not be null-terminated for to_bson().
The tests pin the reduced API: alt_string loses the five dropped members
and gains coverage of the escaping paths, and a std::vector whose at() is
hidden is used as an ArrayType.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Every claim on the page was re-checked by compiling and running it, including
the rows that say a type cannot be used, which were checked to fail for the
documented reason and not merely to fail. Twenty-four claims were wrong.
The most consequential: the incomplete-type constraint applies to ObjectType
only. object_t is instantiated inside the class definition, because it is
probed for key_compare; array_t is only named there and is not instantiated
until basic_json is complete. So eastl::vector, QList and QVector are not
excluded by incomplete types at all -- they simply have no max_size() -- and
absl::InlinedVector is excluded for a subtler reason of its own.
Further corrections: ObjectType does not need erase(key), which has a fallback,
but does need at(key) for UBJSON output; only == and < are used, or == and <=>
under C++20, not all six; the documented adapter does not fit ankerl or
robin_hood. ArrayType needs no initializer-list insert, and value_type, the
(count, value) constructor and swappability are per-function, not always.
BinaryType needs a range insert for CBOR indefinite-length byte strings and
does not need push_back. StringType needs append(const StringType&)
unconditionally, and does not need operator!= or operator== against const
char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is
needed by diff, items and std::hash rather than by JSON Pointer or flatten.
BooleanType must be implicitly convertible from bool, and JSONSerializer's
second parameter need not carry a default.
std::pmr::string was wrong in the other direction this time: a moved-in string
does keep its memory resource, and later growth allocates from it. Only copies
land on the default resource.
Five requirement violations are not caught at compile time rather than the two
the page claimed; they are now listed together up front. Split every
compatibility table into what works and what does not, as the reasons in the
second half are the useful part.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The StringType section carried two 'Compatible types' tables and two copies of
the reference-implementation tip. The second table was a stale copy from before
the binary format string fixes and still listed std::pmr::string and
std::basic_string with a custom allocator as unusable, contradicting the
corrected table a few lines above it, and it dragged along the old explanation
that blamed int_to_string.
Drop the stale copy and put the surviving table before the notes, so the
'see below' in the std::pmr::string row points forwards.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Listing std::pmr::string as fully supported was an overclaim: it was only ever
checked with the default memory resource, which is not what PMR is for.
basic_json cannot be given an allocator or a memory resource, so a pmr string
inside a value always allocates from std::pmr::get_default_resource(), and
assigning an arena-backed string into a value silently drops its resource,
because polymorphic_allocator does not propagate on copy construction. Passing
polymorphic_allocator as AllocatorType does not compile either. Only the
process-global set_default_resource() redirects these allocations.
Say so, and separate the row from std::basic_string with a custom stateless
allocator, which is unaffected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
nlohmann::fifo_map works through the adapter that has always been documented
for it, and preserves the insertion order. Restore its mention in the object
order page, which was dropped together with the tsl::ordered_map one: unlike
ordered_map it keeps a lookup index, so it is the insertion-ordered option
without the quadratic cost.
gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter
through an alias that drops the allocator, whose value type it disagrees on.
gtl::btree_map does not, for the same reason as the other btree containers.
None of the Qt containers can be used, each for its own reason: QMap has no
value_type, QHash iterators yield the mapped value rather than a pair, QList
has no max_size(), QByteArray spells empty() as isEmpty(), and QString is
UTF-16.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Folly works, with the caveat that its headers need C++20: folly::fbstring as
StringType, folly::fbvector and folly::small_vector as ArrayType,
folly::fbvector<std::uint8_t> as BinaryType, and folly::F14NodeMap as
ObjectType through the usual argument-order adapter. folly::F14FastMap is the
exception and requires a complete value type.
For ArrayType, boost::container::devector, boost::container::static_vector
(within its fixed capacity), and std::pmr::vector work as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and
node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType
through the same adapter as Abseil's and Boost's hash maps, which only has to
restore the template argument order.
phmap::btree_map and robin_hood::unordered_node_map do not: like the other
btree containers they require a complete value type.
Note that none of these hash maps defines key_compare, so every one of them
depends on object_comparator_t falling back to default_object_comparator_t.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Three places built a std::string and handed it to something expecting a
string_t: the UBJSON high-precision number reader, which every binary reader
instantiates, and the BSON writer's array element size calculation and write.
That silently required string_t to be implicitly convertible from std::string,
which std::string itself and types with a string_view conversion satisfy, but
many string types do not.
Construct the string_t explicitly from the data and size, which the
requirements already cover. This makes boost::container::string, eastl::string,
std::pmr::string, and std::basic_string with a custom allocator work as
StringType, none of which could previously be used with any binary format.
Add binary format coverage to the alt_string test, which had none, including a
UBJSON high-precision number -- the case that goes through the reader path.
BSON stays uncovered there: it additionally needs string_t::find(value_type),
which alt_string does not provide.
Also record which containers from Boost, Abseil, and EASTL work for each
template parameter, and correct two claims: std::pmr::string is usable after
this change, and tsl::ordered_map is not usable at all, because its iterators
expose the mapped value as const while basic_json modifies it in place.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Two requirements forced users of otherwise suitable containers to write a
wrapper, and neither was load-bearing.
array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and
operator[](size_type), but set_parent() only looks at the value under
JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it
through array_capacity(), which reports unknown_size() when diagnostics are off
or when the array type has no capacity() at all, and treat an unknown capacity
as "the elements may have moved" so the parent pointers are refreshed
conservatively. std::deque now works as ArrayType, in both builds, and
capacity() is no longer named at all in a default build. Since the capacity is
now only meaningful for array insertions, it moves out of set_parent() into
set_parent_after_array_insert().
basic_json::erase(iterator) assigned the object's erase() return value, which
requires the container to return the following iterator. Abseil's hash maps
return void to avoid computing a successor the caller may not need. Detect that
and compute the successor before erasing; containers that return an iterator,
including the vector-backed ordered_map where a precomputed successor would be
wrong, keep the existing path.
Together these leave an Abseil hash map needing only an alias that restores the
template argument order, and no adapter at all for std::deque.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Checked against Abseil release 20250127.0 with the same workload as the other
entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON
round-trip, flatten, hash), with and without JSON_DIAGNOSTICS.
absl::flat_hash_map and absl::node_hash_map work as ObjectType through an
adapter that restores the template argument order and makes erase(iterator)
return the following iterator, which Abseil's returns as void. The page now
carries that adapter, and notes that absl::flat_hash_map does not keep
references to the mapped values valid across insertions while
absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS
already refreshes the parent pointers conservatively for them.
absl::btree_map and absl::InlinedVector cannot be used at all: object_t and
array_t are formed while basic_json is still incomplete, and both inspect
their value type at class scope. std::map and std::vector are required by the
standard to tolerate this, third-party containers generally are not, so the
page states the constraint on its own rather than only per container.
absl::InlinedVector does work as BinaryType, where it is instantiated with a
complete type. absl::FixedArray and absl::Cord are not usable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
unflatten() decided between array and object by looking at the first reference
token it happened to see for a node: it started an array only when that token
was 0. With a sorted object type the token 0 always arrives first, so the
result was correct by accident; with an object type whose iteration order is
unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the
keys "0", "1", and "2" instead of an array.
Collect the pointer prefixes that have a reference token 0 among their children
before building the result, and let get_and_create() consult that set. The
outcome is now independent of the iteration order and matches, for every input,
what a sorted object type produced before: a value is restored as an array if
and only if one of its keys is 0. Iterating the flattened object in a different
order would have been simpler, but it would have changed the key order of the
result for insertion-ordered object types.
The serializer, std::hash, and the UBJSON writer converted the elements of a
binary value to an integer implicitly, which does not compile for a BinaryType
whose value type is std::byte, and which made dump() write the bytes of a
signed value type as negative numbers. Convert to std::uint8_t explicitly in
all three places, so every byte type dumps as 0..255. The default
std::vector<std::uint8_t> configuration is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Follow up on the template parameter requirements page: state, for every
template parameter, which concrete types work and where they stop working.
Each entry was verified by compiling and running a common workload (DOM
access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that
instantiation.
Findings worth calling out:
- ObjectType no longer needs a key_compare member type, so the std::unordered_map
adapter only has to restore the template argument order. A hash-ordered
ObjectType works everywhere except unflatten(), which reconstructs an array
only when it meets the reference token 0 before the other indices.
- ArrayType: std::deque works when wrapped to add capacity(); std::list does not.
- StringType: std::pmr::string and std::basic_string with a custom allocator
compile for the DOM, dump, and parse, but not for the binary readers, flatten,
or diff, because the library assigns std::string values to string_t and
int_to_string cannot be overloaded for a type in namespace std.
- NumberFloatType: long double works for dump and parse but not for the binary
formats, which have no encoding for it.
- BinaryType: std::vector<std::byte> supports assignment, get, and the binary
formats, but neither dump nor std::hash<basic_json>.
Also record the object_comparator_t fix in its version history.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
detail::actual_object_comparator selected between object_t::key_compare and
default_object_comparator_t with std::conditional. Both type arguments of
std::conditional are named eagerly, so object_t::key_compare had to exist
regardless of the condition, and the has_key_compare guard added in 3.11.0
never took effect: any ObjectType without a key_compare member type failed to
compile while instantiating basic_json itself.
Use detected_or_t instead, which resolves through a SFINAE partial
specialization and only names object_t::key_compare when it exists. The
selected type is unchanged for every object type that compiled before, so
object_comparator_t -- a public member type -- keeps its meaning and ABI.
has_key_compare had no other users and is removed.
Add a regression test using an adapter around std::unordered_map, which has no
key_compare; it fails to compile without this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The requirements that basic_json places on its eleven template parameters
were only implied by how the library uses the resulting object_t, array_t,
string_t, etc. Consumers had to discover them by trial and error.
Add "Template Parameter Requirements" collecting them, split into what is
always required and what is only required when a particular part of the API
is instantiated. Notable findings that were previously undocumented:
- ObjectType must provide a key_compare member type (actual_object_comparator
names object_t::key_compare in both arms of a std::conditional), and its
third template parameter is used as a comparator, so std::unordered_map
cannot be used without a wrapper.
- ArrayType must provide capacity() -- push_back(), emplace_back(),
operator+=(), and operator[](size_type) call it unconditionally -- and
needs random-access iterators, so std::deque and std::list do not work.
- StringType needs contiguous, null-terminated data(), a one-byte value_type,
and either assignability from std::to_string or an ADL int_to_string().
- NumberFloatType must be float, double, or long double for parsing and
serialization; the integer types must satisfy std::is_integral.
- AllocatorType must be stateless, support incomplete types, and use plain
pointers.
- BooleanType and the number types are union members and must be trivial.
Link the new page from the basic_json overview, the types feature page, and
the individual type alias pages, and correct the container examples given for
ObjectType (std::unordered_map) and ArrayType (std::list), which do not work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When merge_objects is true, recurse only if the existing value is an
object. Otherwise overwrite, matching the documented "all other values
are overwritten as usual" behavior.
Fixes#5402
Signed-off-by: elix3r <157088510+22elix3r@users.noreply.github.com>
A std::pair or std::tuple whose every element is itself a two-element array
with a string first element (e.g. std::pair<std::string, int>) serializes to
a JSON object instead of a JSON array, because to_json builds the value with a
brace initializer and the initializer-list object-detection rule fires. The
resulting object cannot be read back into the original type and collapses
duplicate keys. Document this quirk in the conversions guide, together with the
unaffected cases and the idiom to force an array.
Claude-Session: https://claude.ai/code/session_016cwQq8WQRFzQcGQtbJTtJg
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
* Throw other_error.502 when UBJSON use_type is set without use_size
Fixes#5321
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Scope UBJSON use_type check to container branches and expand tests
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Re-amalgamate single_include/json.hpp
The previous commit updated the split headers but the amalgamated
file didn't go back through astyle before I committed it, so CI's
amalgamation check caught formatting drift in json_fwd.hpp and a
few noexcept clauses in basic_json, plus one doc example. None of
it touches the UBJSON logic. Applied the patch CI generated to
bring single_include back in sync.
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
---------
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Reformat parser_callback_t example with astyle
The file uses "json & /*parsed*/" in three lambda parameter lists, which
astyle rewrites to "json& /*parsed*/" per --align-reference=type. The
drift went unnoticed because CI never format-checked the documentation
examples; "make pretty" does cover them.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Format-check the documentation examples in CI
The examples live in docs/mkdocs/docs/examples, but both format checks
still referenced the long-gone docs/examples path:
- check_amalgamation.yml passed it to find, which printed an error for
the missing path and carried on, so astyle only ever saw include and
tests. The step still exited 0.
- ci.cmake globbed it into INDENT_FILES, and a GLOB_RECURSE over a
missing directory silently yields nothing, so the ci_test_amalgamation
target skipped the examples too.
Either way the 231 example files have never been format-checked. Point
both at the real path, and guard the workflow with an explicit directory
check so a future rename fails the job instead of quietly shrinking the
file list again.
Also drop the dead docs/examples/** path filter from
publish_documentation.yml; docs/mkdocs/** already covers the examples.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CBOR tag handlers not recognizing tags 0-5 and 21-23
The tagged-item switch in binary_reader::parse_cbor_internal() only handled
head bytes 0xC6-0xD4 and 0xD8-0xDB. Bytes 0xC0-0xC5 (tags 0-5: date/time,
epoch, bignum, decimal, bigfloat) and 0xD5-0xD7 (tags 21-23: base64url,
base64, base16 conversion hints) fell through to the default case and were
reported as invalid bytes, even under cbor_tag_handler_t::ignore and ::store,
despite being valid CBOR major-type-6 tags per RFC 8949.
Add the missing case labels so the full 0xC0-0xDB range is handled
uniformly. Extend the "Tagged values" test in unit-cbor.cpp to cover
0xC0-0xD7, and update the CBOR docs to state the corrected tag range.
Fixes#5315
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
* Fix stale CBOR tag docs and add store-mode binary-payload test
The "Incomplete mapping" warning still listed tags 0-5 (date/time,
bignum, decimal fraction, bigfloat) and 21-23 (expected conversions)
as unsupported, even though they now parse correctly under
cbor_tag_handler_t::ignore/store, same as 0xC6..0xD4/0xD8..0xDB.
Remove those five bullets and cross-reference the "Tagged items"
warning below, matching the equivalent docs fix landed independently
in PR #5367.
Also add a cbor_tag_handler_t::store test that wraps a binary
payload (not just a string) for every byte in 0xC0..0xD7, confirming
these tags are unwrapped the same way as 0xC6..0xD4 rather than
mistaken for the 0xD8..0xDB binary-subtype marker syntax, per review
feedback on #5331.
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
---------
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
The block documenting get_char and tag_handler sat above
get_cbor_negative_integer(), which takes neither, so Doxygen attached it
there and parse_cbor_internal() was left undocumented.
Comment placement only.
Signed-off-by: Dmitry <45711841+darkdi@users.noreply.github.com>
The diagnostic position of a string value was derived by subtracting the
parsed value's length from the end position. Escape sequences make the
source token longer than the value it parses to, so the reported start
position landed inside the string, one byte off per escape sequence:
input: {"a":"\n\n\n\n\n\n"}
start_pos() == 11, so the reported range covered n\n\n\n"
instead of the documented "\n\n\n\n\n\n"
This contradicts the documented behavior of start_pos(), which is the
position of the opening quote, and it also corrupted the "(bytes N-M)"
part of JSON_DIAGNOSTICS exception messages. Strings with multi-byte
UTF-8 but no escapes were unaffected, which is why this went unnoticed.
Record the offset of the token in the lexer when it starts scanning and
use that, instead of reconstructing it from the parsed value. Booleans,
null and numbers already reported correct positions and are unchanged.
The new lexer member and accessor are compiled only when
JSON_DIAGNOSTIC_POSITIONS is enabled, which is already part of the ABI
tag, so the default build is unaffected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The comment asked whether the no-op marker 'N' may be ignored when a
string is read. It may not: at that point the next byte must be a string
length type specification, and 'N' is not one. No-ops at positions where
a value may start are already consumed by the callers through
get_ignore_noop(), so nothing is lost by not skipping them here.
Replace the TODO with a comment stating that, and add regression tests
pinning both directions: a no-op is accepted at top level (also
repeated), before and after an array element, and before an object key,
between key and value, and before the closing brace of an object of
unknown size; it is rejected where a length type specification is
expected, i.e. after the 'S' marker of a string value and as the key
length of an object of known size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not write BJData ndarrays whose size overflows std::size_t
write_bjdata_ndarray() multiplied the _ArraySize_ dimensions into a
std::size_t without checking for overflow. A product that wraps around
to a value that happens to match the size of _ArrayData_ passed the
length check, and the writer emitted an ndarray header announcing an
element count that cannot be represented:
{"_ArrayType_":"uint8","_ArraySize_":[9223372036854775808,2],"_ArrayData_":[]}
was encoded as 5b 24 55 23 5b 4d 00 00 00 00 00 00 00 80 69 02 5d, an
ndarray of 2^64 elements followed by no data. Reading that back throws
out_of_range.408 ("excessive ndarray size caused overflow"), so to_bjdata
produced output that from_bjdata rejects. This is reachable by parsing
untrusted JSON and re-encoding it as BJData.
Mirror the overflow check the binary reader already performs, and also
reject a single dimension that does not fit into std::size_t, which the
previous cast silently truncated where std::size_t is narrower than 64
bits. Such objects now fall back to a plain object encoding, which is
what the surrounding type and length validation already does for
annotations it cannot represent, and they round-trip unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document when to_bjdata converts a JData annotation to an ND-array
The BJData page described the 1-D vector case as the only situation in
which an object carrying _ArrayType_/_ArraySize_/_ArrayData_ is not
written as a compact ND-array. The writer has always had several other
fallbacks -- an unknown _ArrayType_, a dimension that is not a
non-negative integer, an _ArrayData_ whose length does not match the
product of the dimensions, and elements that are not numbers of the
annotated kind -- all of which cause the value to be serialized as a
regular JSON object instead.
Spell out the conditions, including the size-overflow check added in the
preceding commit, so the documented behavior matches the implementation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Linking test-regression2_cpp20 intermittently fails with
unit-regression2.cpp.obj:(.debug_info+0x16): relocation truncated to
fit: IMAGE_REL_AMD64_SECREL against `.debug_line'
The failure moves between matrix entries from run to run, and the same
commit can pass and fail on consecutive runs, so it is the size of the
debug sections rather than any one Clang version.
The jobs only build and run the tests, so override CMAKE_CXX_FLAGS_DEBUG
to drop the default -g. Everything else about the Debug build is
unchanged: no optimization flag is added and NDEBUG stays undefined, so
JSON_ASSERT remains active.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conversions whose element count is fixed by the destination C++ type --
`std::pair`, `std::tuple`, `std::array<T, N>`, C arrays, and
`std::map`/`std::unordered_map` with a non-string key -- read exactly the
elements they need via `at` and never compare the JSON array's size to
that number. Excess elements are silently discarded, while a shortfall
throws `out_of_range.401` rather than a `type_error`. Neither direction
was documented in `conversions.md`, `get.md`, or `from_json.md`.
The existing warning covered only `std::array` and stated that a too-short
JSON array leaves the remaining elements default-constructed with no
exception thrown; that is not what happens. Generalize it to all
fixed-size destinations and correct the shortfall direction.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document the complexity of ordered_map operations
ordered_map stores its elements in a std::vector in insertion order and
has no lookup index, so emplace, operator[], at, find, count, erase, and
insert are all linear scans. The documentation stated no complexity for
any operation, neither in ordered_map.md nor in ordered_json.md.
Add a per-operation complexity table and note the consequence: building
or parsing an ordered_json object of n keys is O(n^2). Measured with
-O2 -DNDEBUG for parsing a flat object of n keys, ordered_json is 5x
slower than json at n=2000 and 54x slower at n=16000, with the timings
quadrupling per doubling of n. Cross-reference the table from
ordered_json.md and from the object order page, which recommends
ordered_json without mentioning the cost.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: move the Complexity section after Member functions
scripts/check_structure.py enforces a fixed section order for pages under
docs/mkdocs/docs/api, in which Complexity comes after Member functions.
The section had been placed right after Iterator invalidation, which made
ci_test_build_documentation fail with structure/section_order.
No content change beyond the move; the table columns are realigned to the
narrower content.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document exceptions newly thrown by the binary-format hardening
A round of binary-format input validation (#5274, #5284, #5287, #5332)
added new failure modes without updating exceptions.md, and left two
descriptions factually narrower than the code:
- parse_error.110 said "CBOR or MessagePack"; BSON and UBJSON also
throw it. Generalized, and added the BSON EOF example (#5332).
- parse_error.112: added the BSON document-size mismatch example
(#5287).
- parse_error.113 said "while parsing a map key", but its own existing
UBJSON char example already contradicted that. Broadened to cover
invalid length specifications, and added the negative-string-length
example (#5284).
- out_of_range.408 said "of an UBJSON array or object"; CBOR now throws
it too (#5274). Generalized and added both CBOR examples.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Correct which types may be referenced in a tuple extraction
The note added in #5271 said a referenced type must be one the library
stores "or an arithmetic type it can convert to/from". The parenthetical
is wrong: is_compatible_reference_type requires an exact match against
the stored types, so std::tuple<int&> is rejected by static_assert even
though int converts fine as a value. Only the value case is permissive.
Spell out the eight admissible types, give the int& counter-example, and
separate the reference restriction from by-value conversion.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
elix3randGitHub
whnandGitHub
Krishnanand GandGitHub
Sahil_KamateandGitHub
ljcjclljcandGitHub
DmitryandGitHub
Angadi56andGitHub
Petr BělohlávekandGitHub