- json_pointer::array_index() dispatches on array_index_status with a
switch, matching the other parse_array_index() caller
- drop `inline` from the function templates this PR adds or moves in
from_json.hpp
- reword a comment that described the change rather than the code
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The merge commit kept develop's single_include/nlohmann/json.hpp because
make amalgamate saw it as up to date.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conflicts:
- include/nlohmann/detail/conversions/from_json.hpp: kept the shared from_json_pair_array_to_map() body; develop's #5681 element-path fix (&p instead of &j) is already applied there
- include/nlohmann/detail/json_pointer.hpp: kept the parse_array_index()-based contains(); it already returns false for an empty token (#5614) and only uses documented StringType members (#5692)
- include/nlohmann/detail/macro_scope.hpp: combined develop's #5698 NLOHMANN_JSON_SERIALIZE_ENUM_STRICT message with the ::nlohmann::detail::templated_json_throw qualification
- single_include/nlohmann/json.hpp: regenerated with make amalgamate
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
operator>> parsed directly into its basic_json& target, so a parse
error left the target holding whatever was parsed before the error
instead of its previous value. With JSON_DIAGNOSTICS=1, that partial
value also violated the class invariant, because the parent pointers
of an array or object's elements are only set when the container is
closed, which a failed parse never reaches; copying such a value then
aborted in assert_invariant().
Fix it the way basic_json::parse() already handles this: parse into a
temporary and move it into the target only once parsing succeeds, so
the target is left unchanged if an exception is thrown.
Fixes#5652.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
contains(const json_pointer&) and operator/=(std::size_t) (and hence
operator/(std::size_t)) used string_t operations that the StringType
template parameter documentation explicitly does not require:
comparing string_t with a const char* literal, c_str(), and
constructibility from std::string. This made both functions fail to
compile for a conforming custom StringType, even though the
documentation's own reference StringType satisfies the requirements.
Fix contains() to compare individual chars ('0'..'9') instead of
comparing string_t with const char* literals, and to call data()
(documented to be null-terminated) instead of c_str(). Fix
operator/=(std::size_t) to build the array-index token via the
existing detail::to_string<StringType> helper (ADL int_to_string() or
assignment from std::to_string()) instead of via std::to_string()
directly, matching how diff(), items(), and std::hash already convert
a std::size_t to a StringType.
Add regression tests to tests/src/unit-alt-string.cpp: contains() for
present/missing keys and indices, "-", a leading zero, and a
non-numeric token on an array, plus json_pointer::operator/(std::size_t).
Fixes#5666.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add JSON_NO_UDLS to leave out the user-defined string literals
The bodies of operator""_json and operator""_json_pointer call the
parser, so every translation unit including the library instantiates it,
even if it never parses anything. Defining JSON_NO_UDLS leaves the
literals out entirely, which saves 15-35% compile time for such
translation units (#5294). Nothing changes if the macro is not defined.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mention JSON_NO_UDLS in the list of exported module symbols
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the user-defined string literals to <nlohmann/json_literals.hpp>
Following the review in #5294, the literals now live in their own header
instead of being removed entirely: <nlohmann/json.hpp> includes it at the
end unless JSON_NO_AUTOMATIC_UDLS (renamed from JSON_NO_UDLS) is defined,
so a project can opt out globally and include the header only where the
literals are used.
The header only uses public and standard macros, because the library's
internal macros are undefined at the end of json.hpp and the amalgamation
inlines macro_scope.hpp only once. For the same reason, the library no
longer defines and undefines JSON_USE_GLOBAL_UDLS, so a user's definition
is still visible to the header. The single-header copy is identical to
the multi-header one, as it only includes <nlohmann/json.hpp>. The module
always exports the literals.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: include cycle, GCC 4.8 literal operator spacing, and global UDLs off in the JSON_NO_AUTOMATIC_UDLS test
- Suppress clang-tidy misc-header-include-cycle on the intentional mutual
include of json.hpp and json_literals.hpp.
- Use operator"" _json with a space for GCC 4.8 in the test's detection
aliases, as the header does.
- Only test the global literal operators when JSON_USE_GLOBAL_UDLS is on.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare the literal operators through a local macro
The GCC 4.8 spacing condition was repeated for both operator definitions
and the global using-declarations. NLOHMANN_JSON_LITERAL_OPERATOR(suffix)
now selects operator""##suffix or operator"" suffix in one place and is
undefined at the end of json_literals.hpp.
Suggested by gregmarr in review.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When copying a value nested deeper than 128 levels, and an allocation
fails while an inner array or object is being copied, the partially
built copy ended up with an element typed array/object but holding a
null pointer. That element was already a fully constructed member of
its parent's container, so destroying the parent during stack
unwinding dereferenced the null pointer (release builds) or failed
assert_invariant() (debug builds), instead of letting std::bad_alloc
reach the caller.
copy_iteratively() set a pending worklist element's type right after
popping it, before the next loop iteration created its container in
copy_array_level()/copy_object_level(). Move that type assignment into
those two functions, right after the container is successfully
created, and drop the premature one in copy_iteratively(), so a
half-built element stays a null value - as copy_shallow()'s comment
already promised - until it can safely hold one.
Add a regression test to tests/src/unit-allocator.cpp that copies a
value nested 130 levels deep (both arrays and objects, with a
std::map- and an ordered_map-backed object_t) and fails every
allocation of the copy in turn: each attempt must throw std::bad_alloc
without crashing, and the source must stay unchanged.
Fixes#5640.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
For an object type whose comparator treats unequal keys as equivalent
(for example a std::map with a case-insensitive comparator),
compare_iteratively() looked up a mismatched left key in the right
object with find(), which uses the object's own comparator, and
accepted whatever entry it found without checking that the keys are
actually equal. A case-insensitive comparator then found "KEY" for
"key", so two objects nested past the recursion bound (or at every
depth with JSON_NO_THREAD_LOCAL) could compare equal even though the
object type's own operator== - and basic_json itself, below the bound
- consider them different.
Accept the found entry only if its key equals (not just compares
equivalent to) the looked-up key.
Fixes#5655.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_json(first, last) treated value_t::binary like the structured
types (array, object) in the range check, so it always copied the
whole binary value regardless of the iterators, even for an empty
range such as (b.end(), b.end()). The other primitive types (number,
boolean, string) already reject such a range with
invalid_iterator.204, and erase(first, last) already does the same
for binary values, so this made the constructor inconsistent with
both. Move case value_t::binary into the group of checked primitive
types.
Also update the two matching passages in basic_json.md that describe
overload 7, and add a version-history note.
Fixes#5670.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When a parser callback rejects an object's or array's start event,
json_sax_dom_callback_parser kept calling it for everything inside
that container anyway: nested keys, values, and the start/end events
of containers below it. This contradicts parser_callback_t's own
documentation, which promises that discarding a container at its
start event also hides its content from the callback.
The same code path also kept a full copy of every key inside such a
discarded container in key_stack until the whole parse finished,
because the early return for values that are not stored skipped the
matching pop. Filtering out a large subtree is the main reason to use
a callback, so this made peak memory during the parse scale with the
size of the very subtree the callback was trying to skip.
Fix start_object(), start_array(), and key() so that a container
whose own start event was discarded, or that is nested inside one, is
never handed to the callback, and no longer pushes onto the key
stacks. A container whose start event was accepted but whose key was
rejected still gets its content reported, as documented ("the
callback is still called for the associated value, but its return
value has no further effect"); only its own bookkeeping is skipped
since it will not be stored.
Fixes#5643.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
j.contains(0), j.find(0), and j.count(0) used to compile: the literal 0 is
a null pointer constant, so it converts to a null const char*, and the
overloads taking const typename object_t::key_type& accepted it by
constructing a std::string from that null pointer, which is undefined
behavior (a crash with both libc++ and libstdc++). value(0, default_value)
had the same problem in C++11, where the object comparator is not
transparent.
Add deleted overloads for integral arguments to contains(), find()
(const and non-const), count(), and value() so that these calls are
compile errors in every supported language mode instead of crashing.
Calls with string, string_view, json_pointer, and size-typed element
access (at(), operator[](), erase()) are unaffected.
Fixes#5657.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
to_bson() rejected a binary value's subtype above 255 (out_of_range.415)
in write_bson_binary(), which only has the binary_t, not the basic_json
value that holds it, so the exception was created with no JSON_DIAGNOSTICS
context even though the equivalent to_msgpack() check names the value's
path. The check also ran after the document size, all preceding elements,
and this element's header and length had already reached the output
adapter, so a caller-provided std::vector or std::string ended up holding
a truncated document.
calc_bson_sizes() already walks every value before anything is written,
to size embedded documents and arrays and to reject invalid keys
(out_of_range.409) up front. The subtype check now runs there instead,
in calc_bson_binary_size(), which is given the basic_json value so the
exception can use it as context. The now-redundant check in
write_bson_binary() is removed, since calc_bson_sizes() always throws
first if any binary value in the document has an oversized subtype.
Fixes#5675.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With JSON_STRICT_NUL_HANDLING defined to 1, parsing a wide, UTF-16,
UTF-32, or (C++20) UTF-8 string literal (e.g. json::parse(L"[1]"))
failed with parse_error.101 at the terminating NUL of the literal,
and accept() returned false. The array overload of input_adapter()
only dropped the compiler-added trailing '\0' for arrays of char,
so for wchar_t, char16_t, char32_t, and char8_t arrays that
terminator was passed to the parser as data, which the macro then
rejected.
Broaden the trimming to every character type that a string literal
can use (char, wchar_t, char16_t, char32_t, and, since C++20,
char8_t). Arrays of any other element type (unsigned char,
std::uint8_t, ...), as used for CBOR/MessagePack, are unaffected: a
trailing zero byte there is still read as genuine data.
Fixes#5658.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Parsing from a std::istream crashed in two unusual but valid stream
states, both in input_stream_adapter:
- With eofbit in the stream's exceptions() mask, get_character() sets
eofbit via is->clear(), which throws std::ios_base::failure. While
that exception unwinds, ~input_stream_adapter() called clear() again
to reset eofbit, which is still set and still in the exception mask,
so it throws a second time out of the (implicitly noexcept)
destructor and std::terminate() is called. The destructor now only
calls clear() if a bit other than eofbit remains set, so the first
exception can propagate normally.
- For an std::istream without a stream buffer (rdbuf() == nullptr,
e.g. std::istream(nullptr)), the constructor stored the null
pointer without checking it, and get_character() dereferenced it.
input_adapter(std::istream&) now throws parse_error.101 for such a
stream, the same as it already does for a null FILE* or char*.
Added regression tests to unit-deserialization.cpp and, for the
JSON_PRECISE_STREAM_POSITION variant of get_character(), to
unit-precise-stream-position.cpp; both crashed before this fix.
Documented the two exceptions in parse.md and operator_gtgt.md.
Fixes#5646.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
from_json built its out_of_range.410 message with "..." + j.dump(). If the
unmatched value is (or contains) a string with invalid UTF-8, that dump()
itself throws type_error.316, so the caller got type_error.316 instead of
the documented out_of_range.410; such strings can reach get<Enum>()
unvalidated, e.g. from from_cbor()/from_msgpack(). With a custom string_t,
j.dump() returns that type, and "const char*" + string_t does not compile
unless the type happens to provide operator+, so the macro failed to
compile for such types.
Build the message with detail::concat(), which appends any type exposing
data()/size() and always yields a std::string, and dump with
error_handler_t::replace so building the message itself cannot throw.
Added regression tests: an invalid-UTF-8 case in the existing strict-enum
test in unit-conversions.cpp, and a strict-enum use with alt_string (the
custom string_t from unit-alt-string.cpp) to cover the compile failure.
Fixes#5667.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_json::swap() (and the friend swap() and the pre-C++20 std::swap
overload that forward to it) only exchanged m_data.m_type/m_data.m_value,
leaving each value's json_base_class_t subobject in place. This is
inconsistent with the copy and move constructors and copy assignment,
which all carry the base class along with the value, so after
a.swap(b) any metadata stored in a CustomBaseClass ended up attached to
the wrong value. Algorithms that mix swap() with moves, such as
std::sort, scrambled the metadata across the whole container.
Fix the member swap() to also exchange the json_base_class_t subobject
and extend the noexcept specifications of swap() and the friend swap()
accordingly.
Fixes#5653.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_msgpack() reading the inactive number union member
basic_json stores number_integer and number_unsigned in a union, and
number_unsigned_t only has to be at least as wide as number_integer_t
(with the default types, both are 64-bit and have the same
representation). When number_integer_t is narrower, write_msgpack()
read the wrong union member in two places:
- The number_unsigned case wrote number_integer's bits instead of
number_unsigned's, silently writing the wrong value whenever it
did not fit in number_integer_t.
- The number_integer case (non-negative branch) picked the encoded
width by comparing number_unsigned's bits, which is undefined
behavior, though the value written was still number_integer's, so
at worst a too-wide encoding was chosen.
Read the active member in both cases, like the other binary writers
(CBOR, UBJSON, BJData, BSON, BON8) already do.
Fixes#5644.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cast number_integer to number_unsigned_t only once in to_msgpack()
Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Copy values before inserting an initializer list into an array
insert(pos, {...}) inserted wrong values when the initializer list
contained const references to elements of the array being inserted
into. json_ref stores only a pointer for a const lvalue, so the
initializer_list_t range passed straight to the array's range insert
aliased the array's own storage; std::vector::insert(pos, first, last)
may move or shift elements before copying from that range, so the
source elements were already stale by the time they were read
(different wrong results on libc++ and libstdc++).
Copy the referenced values into a temporary array_t first, then move
that temporary into place, so the source range never aliases the
array being modified.
Fixes#5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the reserve_array helper in the initializer_list insert fix
The previous commit called array_t::reserve() directly on the
temporary buffer used to copy an ilist's values before inserting.
std::deque, a documented ArrayType (tests/src/unit-custom-array-type.cpp),
has no reserve(), so insert(pos, initializer_list) no longer compiled
for it. Use the existing detail::reserve_array() SFINAE helper (already
used by the SAX DOM parser) instead, which leaves array types without
reserve() untouched.
Added a regression check that deque_json::insert(pos, {...}) compiles
and handles the aliasing case from #5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Take diff()'s fast path unless the object type reorders members
For every object type except an insertion-ordered one like ordered_map,
diff() no longer produced a member-by-member patch when target had a key
that sorts before a key the two objects share: it fell through to the
slow path, which removes every member of source and re-adds every member
of target, instead of just adding the new key.
#5465 added an order check to require the fast path to also reproduce
target's member order, needed because ordered_map's patch()-driven "add"
appends a new member at the end. The check compared the common keys'
order between source and target and also required that every added key
come after every common key in target's order ("new_keys_form_suffix").
The comment above it argued this check is always true for std::map, and
that reasoning is correct for the order of the common keys themselves,
but not for new_keys_form_suffix: a std::map iterates in sorted key
order, so a new key that sorts before an existing common key is
enumerated between common keys, making new_keys_form_suffix false even
though std::map's own key order does not need reordering at all - it
places every member itself, regardless of insertion history, so a
member-by-member diff already reproduces target's iteration order.
Only require the order check for an object type that keeps insertion
order, using the same detail::is_ordered_map trait the library already
uses to recognize such an object type in set_parent(). Every other
object type - std::map in key order, a hash map in an order its
operator== ignores - always takes the fast path.
Added a regression test to unit-json_patch.cpp: the issue's example now
yields a single "add" op for json, while ordered_json still takes the
slow path to reproduce target's member order.
Fixes#5639.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Only track the target key order in diff() for insertion-ordered objects
common_keys_target_order and new_keys_form_suffix are only read when object_t keeps its members in insertion order; skip building them otherwise. Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
For idx == SIZE_MAX, the non-const array operator[] computed the new size as
idx + 1, which wraps to 0. resize(0) then emptied the array, and the
subsequent operator[](idx) on the now-empty vector wrote one element before
its buffer. Every other too-large index (e.g. SIZE_MAX - 1) already went
through resize(), which throws std::length_error and leaves the array
unchanged; SIZE_MAX was the one value for which the overflow bypassed that
safety net.
Add a guard that throws std::length_error before computing idx + 1 when idx
is the largest representable size_type value, so the array is left
unchanged, matching the exception vector::resize() already throws for
smaller (but still too large) indices.
Fixes#5647.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Let to_json(std::optional<T>) propagate exceptions from T's to_json
The overload was marked noexcept even though its body assigns *opt to
the JSON value, which calls T's to_json (or allocates for std::string,
std::vector, or json). Any exception from there -- a user-defined
to_json reporting an error, or std::bad_alloc -- called std::terminate()
instead of propagating. The noexcept also made basic_json's converting
constructor noexcept(true) for std::optional<T>, so json j = opt; could
not report the error either.
Fixes#5642.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: make to_json(std::optional<T>) conditionally noexcept
GCC's -Wnoexcept (an error in ci_test_gcc) fired at to_json_fn's
noexcept(noexcept(to_json(j, val))): after dropping the unconditional
noexcept, to_json(std::optional<int>) had no exception specification
although GCC could prove its body cannot throw. It also made
json(std::optional<int>) lose its noexcept.
Declare the overload noexcept exactly when assigning the contained
value to the JSON value is (std::is_nothrow_assignable<BasicJsonType&,
const T&>), which is what the body does. std::optional<int> is noexcept
again; a T whose to_json may throw still propagates the exception.
Static assertions in the test check both cases.
The test's throwing to_json triggered -Wmissing-prototypes and
-Wmissing-noreturn (clang) and -Wmissing-declarations and
-Wsuggest-attribute=noreturn (GCC). Move the type and its to_json into
an anonymous namespace, mark the function [[noreturn]], and compile
them only without JSON_NOEXCEPTION, like the test that uses them.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
value(KeyType&&, default) is constrained on is_comparable_with_object_key,
which passes KeyType as a reference. is_comparable's dispatch on
is_json_pointer_of<A, B> only matches a json_pointer as a plain type or a
plain reference, so a const-qualified reference (as produced when KeyType
is deduced from a json_pointer argument) fell through to
is_comparable_no_json_pointer, which instantiates the deprecated
json_pointer/string comparison operators. This made ordered_json's
transparent comparator (and any transparent comparator on a custom string
type) warn under -Wdeprecated-declarations when calling
value(json_pointer, default), even though no such comparison is ever
performed. at() was already fixed for this in #5289, which does not use
is_comparable_with_object_key.
Strip references and cv-qualifiers with uncvref_t before the
is_json_pointer_of dispatch, so any reference-to-json_pointer is
recognized regardless of qualifiers.
Fixes#5664.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When converting a JSON array to std::map or std::unordered_map with a
non-string key, each element must itself be a [key, value] array. If an
element is not an array, from_json() threw type_error 302 with the outer
array's value (&j) as the exception context, so with JSON_DIAGNOSTICS
enabled the message pointed at the whole array instead of the offending
element (e.g. "(/outer/m)" instead of "(/outer/m/2)"), even though the
message text already described the element's type.
Both from_json() overloads now pass the element (&p) as the context, so
the reported JSON Pointer matches the type named in the message, the
same way std::vector<std::vector<T>> and similar conversions already do.
Fixes#5668.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clear() to also reset the subtype of a binary value
clear() on a binary value cleared the bytes but left the subtype
untouched, so the result was not equal to a default-constructed
binary value even though the documentation says clear() has the same
effect as *this = basic_json(type()). The fix calls
byte_container_with_subtype::clear_subtype() alongside the existing
clear() call.
Extended the "filled binary" clear() test in unit-modifiers.cpp with
a case that uses a subtype, since the existing cases only covered
binary values without one.
Fixes#5669.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the table alignment in clear.md
Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_arithmetic_value() rejects boolean_t, so the default from_json()
for enums failed to compile for an enum whose underlying type is
bool (e.g. enum class Flag : bool { off, on }), even though the
matching to_json() serializes such enums as an unsigned number.
Read the underlying value through number_unsigned_t in that case,
matching what to_json() writes, then cast back to the underlying
type before constructing the enum.
Fixes#5671.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Copying and comparing stopped their descent at
basic_json::nesting_depth_limit(), while serializing, hashing and
merging used detail::recursion_depth_limit(). Both were 128, but
nothing kept them equal. The thread-local count now tests against
detail::recursion_depth_limit() as well, and a static_assert keeps
the limit small enough for the byte that holds the count.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not throw in contains() for an empty array reference token
json_pointer::contains() rejected malformed array indices, but an empty
reference token (e.g. "/a/" where "a" is an array, or "/" on an array)
passed every check and reached array_index(), which throws
out_of_range.404. contains() must not throw (cf. #5395), so it now
returns false for an empty token. at() still throws out_of_range.404.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: bind j_nested_const by reference in the json_pointer test
clang-tidy (ci_clang_tidy) flagged the new test with
performance-unnecessary-copy-initialization: the local copy
j_nested_const of j_nested is never modified. Bind it as a const
reference instead; it still exercises the const overloads of at() and
contains().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move instead of deep-copy ordered_json values when an object grows
ordered_map keeps its elements in a std::vector<std::pair<const Key, T>>.
With a std::string key, that pair is not nothrow move constructible (the
const key has to be copied), so std::vector copies every element when it
reallocates. For ordered_json, this deep-copies every member value an
object already holds, including whole nested subtrees, on each growth
step.
Grow the storage in ordered_map instead, copying the keys and moving the
values. This happens in two phases, so the strong exception guarantee is
kept without try/catch. The first phase may throw, but only touches a
temporary buffer: it copies the keys, value-initializes the values, and
constructs the new element. The second phase moves the values (noexcept)
and swaps the buffers. Because the new element is constructed before any
value is moved, arguments that refer to elements of the container stay
valid, as with std::vector. Types that cannot take this path keep the
std::vector behavior.
Parsing into ordered_json (ParseStringOrdered, Apple M1 Max, clang -O3):
twitter 3.20 -> 1.70 ms, citm_catalog 7.73 -> 3.67 ms, jeopardy 219 ->
177 ms, canada unchanged. The number of allocations for twitter and
citm_catalog drops by two thirds.
Also add ParseStringOrdered rows to the benchmarks, and document the
growth behavior and the exception safety of ordered_map.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: skip std::pair noexcept assumptions on EDG-based compilers
ci_icpc and ci_nvhpc failed to compile unit-ordered_map.cpp: the static
assertion that std::pair<const std::string, ordered_json> is not nothrow
move-constructible fails there. The EDG front end (Intel icpc 2021.10,
NVIDIA nvc++ 25.5) considers the defaulted move constructor of
std::pair<const Key, T> noexcept even if copying Key can throw. With these
compilers, std::vector already moves such elements itself when it grows,
and ordered_map correctly leaves growing to it.
The same misjudgement makes std::vector call std::terminate when a key copy
throws during growth, so the exception-safety test with throwing_key would
abort on these compilers as well.
Skip the static assertion and the exception-safety section when __EDG__ is
defined. Verified with icpc 2021.10 (-std=gnu++11) and nvc++ 25.5 (C++11 and
C++17) on Compiler Explorer: unit-ordered_map and unit-disabled_exceptions
build and pass.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Convert floats with Eisel-Lemire when std::from_chars is unavailable
Float tokens that Clinger's fast path cannot convert (e.g. the 17-digit
coordinates of canada.json) went to strtod unless std::from_chars was
available. It is not used in C++11/14, and not with libc++, which does not
define __cpp_lib_to_chars. The Eisel-Lemire algorithm (after fast_float's
compute_float) now converts them with integer arithmetic, correctly rounded
for any token with at most 19 significant digits. Longer tokens are
truncated; the result is used if w and w + 1 round alike, else strtod
decides as before. Overflow still yields infinity (out_of_range.406).
The table of powers of five (fast_float's) lives in pow5_table.hpp; a unit
test recomputes every entry with big-integer arithmetic. Further tests:
known values generated with Python (whose float() is correctly rounded),
200,000 round trips through to_chars, and the 128-bit multiplication and
leading-zero count against big-integer references (both with and without a
128-bit type). Checked against strtod on 6.5 million tokens, among them
60,000 exact halfway cases: no difference.
json::parse on canada.json: -8.6% (C++11), -7.6% (C++17, Apple clang);
other files unchanged. Compile time of a TU including json.hpp: +0.7%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: unused parse result and find() == npos in the Eisel-Lemire tests
GCC (-Werror=unused-result) rejected CHECK_THROWS_WITH_AS(json::parse(...))
because parse() is [[nodiscard]]; assign the result to a dummy json as the
other tests do. clang-tidy flagged longer.find('.') == npos with
abseil-string-find-str-contains; store the position in a variable first.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
lexer::convert_number() converted float tokens with std::from_chars (when
available), Clinger's fast path, and the locale-aware strtod fallback, all
as lexer members. They are now free functions in number_parse.hpp:
- convert_float_fast(): std::from_chars, then Clinger's fast path, skipped
when the mantissa has too many significant digits
- convert_float_locale_aware(): strtof/strtod/strtold with the decimal point
of the current locale, retried when the locale changed (#5198)
so that other code converting JSON number tokens gets the same values. No
change in behavior; the lexer no longer includes <clocale> and <cstdlib>.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A header that builds on json.hpp (such as the planned json_view.hpp) must
not inline json.hpp: its single-header version would contain a second copy
of the library, and that copy would change with every library change.
The optional config key "external" lists include paths that are kept as
#include directives. Only the first directive per path is kept; repeated
ones are commented out, as the tool already does for inlined headers.
config_json_view.json uses it for json_view.hpp; the existing configs do
not set it, and json.hpp and json_fwd.hpp regenerate byte-identically.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
array_index(), contains() and get_checked_or_null() each re-implemented
the RFC 6901 array-index rules and the size_type range check: array_index()
does the canonical parse and throws; contains() (which must not throw,
#5395) re-validates every digit by hand and runs its own strtoull/ERANGE
check before calling array_index() anyway, parsing every array token
twice; get_checked_or_null() wraps array_index() in JSON_TRY/
JSON_INTERNAL_CATCH (detail::out_of_range&) to turn an unrepresentable
index into "not found".
Add a single private, noexcept parse_array_index(s, idx) returning an
array_index_status (ok / leading_zero / not_a_number / unresolved /
exceeds_size_type). array_index() becomes a thin wrapper mapping each
status to the existing parse_error.106/109 or out_of_range.404/410;
contains() and get_checked_or_null() switch on the status directly. This
removes contains()'s digit-validation loop and its second strtoull call,
and get_checked_or_null()'s JSON_TRY/JSON_INTERNAL_CATCH.
Bugfix as a consequence: get_checked_or_null()'s JSON_TRY/
JSON_INTERNAL_CATCH was dead code under JSON_NOEXCEPTION (JSON_TRY
expands to "if(true)" and the catch to "if(false)", so JSON_THROW's
std::abort() ran unconditionally), meaning value() and contains() would
abort instead of returning the default/false for an out-of-range-sized
or oversized array index when exceptions are disabled (#5672). Switching
on parse_array_index()'s return value instead of relying on an actual
throw/catch fixes this: get_checked_or_null() now returns nullptr for
array_index_status::unresolved/exceeds_size_type in every build
configuration, and still calls JSON_THROW (aborting under
JSON_NOEXCEPTION, as before) only for a malformed index
(leading_zero/not_a_number), matching its documented @throw list.
All existing error ids, messages and diagnostic paths are unchanged; a
few reference tokens that used to fail contains()'s manual per-character
validation (e.g. "1a") now fail via array_index_status::unresolved
instead, with no observable difference since contains() only returns
bool.
Adds regression tests to unit-element_access2.cpp's "access on array
type" section covering value() with an index that exceeds size_type and
one with a trailing non-digit, both of which must yield the default
value rather than abort/throw.
Public API: no change.
Overlaps #5700, #5614 and #5692, which touch the contains() and
get_checked_or_null() array hunks; this change replaces those hunks with
calls into the new shared parser, so a rebase will need to re-apply
their token-handling changes (e.g. the empty-token case) on top of the
switch statements here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5708 item 4
Several from_json() overload pairs in from_json.hpp were copies of each
other, so a fix has to be applied twice (as #5681 already does):
- from_json(..., std::map&) and from_json(..., std::unordered_map&) for
non-string keys had identical 16-line bodies: array check, m.clear(),
pair check loop, m.emplace(...). Route both through a new
from_json_pair_array_to_map(j, m) helper.
- The from_json_array_impl priority_tag<1> and priority_tag<0> fallbacks
ran the same std::transform/std::inserter loop, differing only in
ret.reserve(j.size()). Merge them into one body and, modeled on the
existing from_json_object_reserve, add a from_json_array_reserve pair
so the reserve() call is only made for ConstructibleArrayType that
support it.
Error ids (type_error.302), messages, diagnostic paths ((at(0)/at(1))
and behavior for types with/without reserve() are unchanged; only the
duplication is removed.
Public API: no change.
Overlaps #5681, which changes the "&j" to "&p" line in both map bodies;
the shared helper here should make that a one-line change instead of two
on rebase.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5708 item 5
The std::ranges view conversion (excluded on MinGW because of its
incomplete C++20 ranges support, #4916) was gated by the same
#if JSON_HAS_RANGES && !defined(__MINGW32__) condition at seven
independent sites in to_json.hpp and type_traits.hpp, with the MinGW
rationale duplicated in two of them and missing from the rest. Since the
sites come in matching pairs (one enables is_compatible_range_view and a
view-based overload, the other adds the exclusion to the
plain-array-type overload), a drift between any pair would produce an
ambiguous or missing overload on exactly one platform.
Add JSON_HAS_RANGE_VIEW_CONVERSION next to JSON_HAS_RANGES in
macro_scope.hpp, combining both conditions with the #4916 reasoning in
one place, #undef it in macro_unscope.hpp, and use it at all seven
sites. This does not fold the MinGW check into JSON_HAS_RANGES itself:
JSON_HAS_RANGES is user-overridable and also gates the
enable_borrowed_range specialization in iteration_proxy.hpp, which is
not excluded on MinGW.
No behavior or public API change: JSON_HAS_RANGE_VIEW_CONVERSION expands
to exactly the condition that was previously written out at each site.
Overlaps #5585, #5600 and #3575, which touch the same to_json.hpp and
type_traits.hpp lines; the change here is a mechanical
search-and-replace of the guard condition and should rebase cleanly.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5708 item 11
templated_json_throw() was defined in macro_scope.hpp, which is included
outside NLOHMANN_JSON_NAMESPACE_BEGIN, so the helper leaked into the
global namespace as ::templated_json_throw with no ABI tag. Unqualified
lookup in NLOHMANN_JSON_SERIALIZE_ENUM_STRICT could then bind to a
same-named function declared in the user's own namespace instead, which
fails to compile with Clang ("does not name a template").
Move the helper next to the exception classes in exceptions.hpp, inside
nlohmann::detail, and call it qualified as
::nlohmann::detail::templated_json_throw<...>(...) from both macro
expansion sites. Rewrite the doc comment to give the real reason for the
helper (JSON_THROW may expand to code that discards its argument, e.g.
when exceptions are disabled) and fix the "supress" typo.
templated_json_throw was never released (added by #5151 after v3.12.0),
so it can be moved freely.
Adds a regression test that expands NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
inside a namespace declaring its own templated_json_throw.
Public API: no change (::templated_json_throw was an unreleased,
unintentional global-namespace leak with no callers relying on its
location).
Overlaps #5698, which rewrites the same two macro call lines; the
overlapping hunks are small and should be trivial to reconcile on
rebase.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5708 item 2
from_json() for C arrays had four hand-unrolled overloads (rank 1-4,
added incrementally in #4262), each with its own nested loops. to_json()
already handles any rank recursively, so a rank-5+ C array could be
serialized but not read back with get_to()/get<>().
Replace the four overloads with one from_json() SFINAE-constrained on
get<remove_all_extents<T>::type>() existing, forwarding to a pair of
mutually recursive from_json_c_array_element() helpers: one assigns a
non-array element via get<T>(), the other loops over a array element and
recurses one dimension at a time. Each dimension still goes through at(),
so type_error.304/out_of_range.401 stay unchanged; ranks 1-4 keep their
existing behavior and semantics.
Adds rank-5 round-trip and mismatched-shape tests to unit-conversions.cpp.
Public API: additive only (rank 5+ C arrays become readable).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5708 item 1
The JSON_HAS_RANGES feature-detection block had its libc++ comment
sitting above the clang+libstdc++ branch it does not describe,
leaving the libc++ branch uncommented and the clang+libstdc++ branch
without its own rationale. Move each comment to sit under its own
branch, and give the clang+libstdc++ branch (added in issue 5161) its
own one-line reason referencing that issue instead of reusing the
libc++ branch's comment. Also fix a duplicated-word typo ("in large
in large cpp files") in from_json.hpp, drop two unanswered 2017
design questions left as comments in type_traits.hpp and
from_json.hpp that no longer reflect open questions, and correct
NLOHMANN_JSON_SERIALIZE_ENUM_STRICT's @since tag from 3.12.0 to
3.13.0, the release it was actually introduced in.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The static_assert meant to check the LegacyBidirectionalIterator
named requirement had a first clause comparing
std::bidirectional_iterator_tag to itself, which is always true and
checks nothing; only array_t::iterator was actually being checked,
despite the message claiming object iterators were checked too.
Drop the tautological clause, reword the message to describe what
is actually checked, and note that object_t may use a forward-only
iterator as long as reverse iteration and operator-- are unused.
The check is intentionally not extended to object_t::iterator, since
that would reject object types with forward-only iterators that
compile and work correctly today.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The const and non-const get_checked() overloads had byte-identical
50-line bodies, differing only in the signature. The remaining
template deduces a const-qualified BasicJsonType for const callers,
so at(), the out_of_range::create() calls and the bounds check all
still work.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
is_ordered_map re-detected capacity() with a C++03 sizeof/vararg
trick right after has_capacity did the same detection through
is_detected. For ordered_map, the old trick took the address of
std::vector::capacity, which [namespace.std]/6 makes unspecified.
Reuse has_capacity instead, which removes the unspecified-behavior
pointer-to-std-member and two NOLINT suppressions.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Besides detail::result_of_begin/end, which is_range and iterator_t use,
the macro defined a namespace detail2 with a tag type, a catch-all
overload and would_call_std_begin/end, plus would_call_std_begin/end
structs directly in namespace nlohmann. Nothing has used them since
they were added in #3020.
Reduce the macro to its detail part. Without the trailing struct the
';' after the two invocations would be an empty declaration that
-Wextra-semi flags, so drop it. macro_scope.hpp included
meta/detected.hpp only for this macro; all users of detected.hpp
include it (or type_traits.hpp) themselves, so remove the include.
Behavior and ABI are unchanged. The undocumented, untested and unused
names nlohmann::would_call_std_begin, nlohmann::would_call_std_end and
namespace nlohmann::detail2 are no longer declared.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
meta/logic.hpp added a second set of type-level boolean helpers
(cxpr_and, cxpr_or, cxpr_not, ...) next to the existing conjunction
and negation in type_traits.hpp. It was used only by one static_assert
in from_json_tuple_impl, two of its templates were never used, and it
was the only header without the license banner and relied on
transitive includes for <type_traits>.
Add the missing disjunction next to conjunction and negation, use the
three in the static_assert, and delete logic.hpp together with its
BUILD.bazel entry. same_sign now uses disjunction as well, which
resolves the 2022 TODO waiting for such a trait.
The static_assert accepts and rejects the same types as before. Only
names in nlohmann::detail change; behavior, public API and ABI are
unchanged.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
detail::is_sax had no user: the parser and the binary reader only use
is_sax_static_asserts, so is_sax was a second, unchecked copy of the
SAX event list. is_sax_static_asserts asserted boolean(bool) twice in
a row, and detail::is_detected_convertible was never used anywhere.
Remove all three and include <cstddef> for size_t instead of <cstdint>.
Only names in nlohmann::detail are removed; behavior, public API and ABI
are unchanged. The diagnostics for an incomplete SAX handler are the
same, apart from the duplicated boolean() message.
Part of #5708
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5597 was merged before all of its CI jobs had run, and two of them fail
on develop now, and so on every pull request:
- ci_clang_tidy: cert-err33-c for the two std::setlocale(LC_NUMERIC, "C")
calls whose result was discarded. Check the result, like the other
resets in the file.
- ci_test_standards_gcc (20) with GCC 16: -Wnoexcept for the two parser
callbacks, which cannot throw but were not declared noexcept.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Look up the locale decimal point at conversion time, not lexer construction
The lexer read localeconv()->decimal_point once in its constructor and wrote
that character into token_buffer in place of '.'. The strtod fallback then
used the locale current at conversion time, so an LC_NUMERIC change in
between (parser callback, SAX handler, another thread) truncated the value
in release builds and fired the endptr assertion in debug builds.
token_buffer now always holds '.'. Only the strtof/strtod/strtold fallback
depends on the locale: it looks up the decimal point right before the call,
restores '.' afterwards, and repeats the conversion if the locale changed in
between. As a side effect, std::from_chars and Clinger's fast path now also
apply under locales whose decimal point is not '.'.
Fixes#5198
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop the strtod retry loop when the decimal point is unchanged
convert_float_locale_aware() repeated the conversion until strtod
consumed the whole token, assuming an early stop can only mean a locale
change. Under a locale whose decimal point is not a single character
(e.g. the two-byte U+066B of ar_EG.UTF-8, ar_SA.UTF-8, or fa_IR.UTF-8,
all available on macOS), the in-place substitution can never succeed,
so parsing any float that reaches the strtod fallback (for example
3.14159265358979323846 at C++11) hung forever. Before this branch, the
same input was truncated.
Retry only if the decimal point changed since the previous attempt;
otherwise keep the value strtod parsed so far, as before. Add a test
that parses such numbers under a multi-byte decimal point locale; it
hangs without this change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Weffc++ errors in the #5198 locale test
GCC's -Weffc++ (an error in ci_test_gcc and ci_test_standards_gcc)
rejected LocaleSwitchingSax: it has a pointer data member but does not
declare its copy operations, and its vectors are not initialized in the
member initializer list. Store the locale name as a std::string and give
the vectors brace initializers, like SaxEventLogger in
unit-deserialization.cpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the key type when rejecting non-string CBOR/MessagePack map keys
CBOR and MessagePack allow map keys of any type, but JSON object keys
are always strings, so such maps are rejected. The error so far was the
one for a malformed string (e.g. "expected length specification
(0xA0-0xBF, 0xD9-0xDB); last byte: 0xC0" for a nil key), which does not
tell the user what went wrong. Report the type of the key instead:
syntax error while parsing MessagePack object key: only string keys
are supported, but found nil; last byte: 0xC0
The exception id (parse_error.113) and type are unchanged. Malformed
string keys and a missing key keep their previous messages. Document
the restriction on the CBOR and MessagePack pages.
Refs #2766, #3381
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Point the MessagePack key note to the spec's profile section
The note linked to "Serialization: type to format conversion", which says nothing about key types. Restricting map keys to strings is only mentioned in the "Profile" section (under "Future discussion") as an example of a JSON-compatible profile, so link there and describe it as such instead of as a permission.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The "Size above uint32" tests for arrays and objects fake a container
size of 2^32 and expect to_msgpack() to throw out_of_range.412. But
to_msgpack(j) first reserves binary_reserve_hint(j) bytes, which is
size + 1 for arrays and 2 * size + 1 for objects, i.e. 4 or 8 GiB.
Linux and macOS overcommit, so the reservation succeeds; on Windows it
throws std::bad_alloc before the size check is reached (seen with
msvc-vs2026 Debug x64 on the object test).
Write into a caller-owned vector instead, so nothing is reserved.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5559 added a 224-character line to cbor_tag_handler_t.md; the
documentation style check allows at most 160.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The line added in #5559 exceeded the 160-character limit enforced by
docs/mkdocs/scripts/check_structure.py, breaking the documentation build.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The tests added by #5515 fail two ways on develop:
- clang-tidy reports the size() overrides of huge_string and huge_binary
(readability-convert-member-functions-to-static) and the non-const
test value (misc-const-correctness); mark them like the #5584 types
- clang with libstdc++ 10 cannot compile the file for C++17: the
std::filesystem::path conversion considered for huge_string, a class
derived from std::string, is ambiguous. Guard it with
JSON_TEST_BEYOND_UINT32_STRING, which #5584 introduced for the same
reason, and define that macro before both test blocks.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
