* Add basic_json::as_base_class and document name conflicts with custom base classes
Members of basic_json hide members of a custom base class with the same
name, and future releases may add members that hide ones accessible
today. Document this in json_base_class_t and add as_base_class() to
reach hidden members without spelling out the cast.
Also make json_base_class_t a public member type. It was documented
since 3.12.0, but declared private, so users could not name it.
Supersedes #3899.
Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add as_base_class to the docset search index
New public members get an entry in docs/docset/docSet.sql (as done for
to_bon8/from_bon8 in #2998). Without it, the Dash/Zeal docset built from
the documentation cannot find basic_json::as_base_class.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence clang-tidy for the hidden type_name() in the base class test
ci_clang_tidy failed with readability-convert-member-functions-to-static
on base_class_with_hidden_members::type_name(). It must stay a
non-static member: the test shows that it is hidden by the non-static
basic_json::type_name() and reachable through as_base_class().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com>
* Remove unused private aliases from basic_json
The private aliases primitive_iterator_t, internal_iterator and
output_adapter_t are not used anywhere: iter_impl, binary_writer and
the tests refer to the detail:: names directly. As the aliases are
private, no user or derived class can depend on them. The
internal_iterator.hpp include stays because iter_impl needs it.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix meta()'s dead, syntactically invalid HP aCC branch
The HP aCC branch of basic_json::meta() was missing a semicolon and
has therefore never compiled; adding only a semicolon would also make
it throw type_error.305, since it assigned a plain string to
result["compiler"] and then indexed into it like the other branches
do into an object. Make the branch consistent with the others by
assigning an object with "family" and "version" keys, narrow the
condition to __HP_aCC (a C compiler cannot build this header-only
library), and fix meta.md, which documented the old (impossible)
plain-string behavior.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop the noexcept null constructor delegating to a throwing one
basic_json(std::nullptr_t) delegated to basic_json(value_t), whose
underlying json_value(value_t) constructor allocates for other types
and can therefore throw, which is why the noexcept had a
NOLINT(bugprone-exception-escape). The delegated-to constructor also
called assert_invariant() a second time. The default member
initializers of data already produce the same null state (a
value-initialized, i.e. zeroed, union with object == nullptr), so the
delegation and its NOLINT can simply be dropped.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove four cppcheck accessForwarded suppressions in the move constructor
basic_json(basic_json&&) built its base subobject with
std::forward<json_base_class_t>(other), so cppcheck saw the whole of
other as forwarded and flagged every subsequent access to it as
accessForwarded, three of them still marked "TODO check". Only the
base subobject is actually moved from; cast explicitly to the base
type instead, the way ordered_map already does, so cppcheck can tell
the two are unrelated. Behavior is unchanged: for a non-reference T,
std::forward<T>(x) is defined as static_cast<T&&>(x).
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove stale cppcheck suppressions and name the local parser in parse()
Running the pinned cppcheck (ci_cppcheck's invocation) without
--inline-suppr across all configurations reports no syntaxError, no
ignoredReturnValue and no assertWithSideEffect, so the corresponding
suppressions in json_fwd.hpp, string_concat.hpp and
assert_invariant() no longer match anything (json_fwd.hpp's is kept,
since downstream users who run an older cppcheck against it could
still hit the warning it once silenced).
The three basic_json::parse() overloads still trigger a false-positive
accessMoved/accessForwarded because they build a temporary parser and
call .parse() on it in the same expression; giving that parser a name
makes the warning go away without changing behavior, and removes the
last of the inline suppressions on these functions.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the string/binary cleanup in the two erase() overloads
erase(pos) and erase(first, last) each carried a byte-identical
14-line block that destroys and deallocates a string or binary
value before resetting the type to null. That reimplements the
string/binary cases of json_value::destroy(), so any future change to
how those values are freed would have to be made in three places
instead of one. Both overloads now just call destroy() and reset the
union; for the other primitive types (boolean, numbers) destroy() is
a no-op, so behavior is unchanged.
Also fix erase(first, last)'s error-path branch hint, which used
JSON_HEDLEY_LIKELY where erase(pos), the iterator-range constructor,
and every other error path in the class use JSON_HEDLEY_UNLIKELY.
This only affects code layout, not semantics.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix stale and copy-pasted comments in basic_json
Several comments no longer match the code: the class invariant and
assert_invariant()'s doc still named the members m_value/m_type
(now m_data.m_value/m_data.m_type) and did not mention the binary
invariant that assert_invariant() already checks; the json_value note
and the get<PointerType>() @tparam list omitted binary_t even though
binary is a variable-length, pointer-stored type like the others; the
key-based value() overload's brief said "via JSON Pointer", which is
the other overload; and swap(binary_t&)/swap(binary_t::container_type&)
both carried "swap only works for strings", copied from swap(string_t&).
Comment-only change; behavior, the public API and the ABI are
unchanged. The private get_impl() doxygen and the emplace() comments
that border #5585's hunk are intentionally left alone.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the 16 copied from_cbor/msgpack/ubjson/bjdata/bon8/bson bodies
Each of the 16 binary deserialization overloads (from_cbor,
from_msgpack, from_ubjson, from_bjdata, from_bon8, from_bson, each in
an InputType&& and an iterator/sentinel version, plus the deprecated
span overloads of from_cbor/from_msgpack/from_ubjson/from_bson) had
the same body, differing only in the input_format_t value. Every copy
built a temporary binary_reader from std::move(ia) and called
sax_parse on it in the same expression, which also produced a
false-positive cppcheck accessMoved on all 16 lines and needed a
NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg) on
the four span overloads.
Add a private from_binary_impl() helper that builds the reader as a
named local instead, and make each of the 16 overloads a one-line
forward to it. All public signatures, default arguments,
JSON_HEDLEY_WARN_UNUSED_RESULT and JSON_HEDLEY_DEPRECATED_FOR
attributes are unchanged, tag_handler keeps defaulting to
cbor_tag_handler_t::error for the non-CBOR formats (matching
binary_reader::sax_parse's own default), and the helper is placed in
the existing private section before the binary section banner rather
than between the from_* overloads, so from_binary_impl() itself does
not collide with #5688's insertion point. Collapsing the
from_bjdata/from_bon8 bodies into one-line forwards does rewrite the
"return result; }" context lines that #5688 inserts its two
deprecated overloads after, so that PR will need a small manual
rebase (reinserting its overloads after the new one-line bodies)
rather than applying cleanly.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make insert(pos, basic_json&&) move its argument instead of copying it
insert(const_iterator pos, basic_json&& val) delegated to
insert(pos, val), but val is a named rvalue reference, so inside the
function it is an lvalue: the call always resolved to
insert(const_iterator, const basic_json&) and deep-copied the value.
This has been the case since the overload was introduced, in every
release. push_back(basic_json&&), by contrast, already moves.
Give the rvalue overload its own body with the same two checks
(type_error.309, invalid_iterator.202), then move the argument into a
local before inserting it. Moving into a local first, rather than
inserting std::move(val) directly, keeps this safe even when val
aliases an element of the same array (e.g.
arr.insert(arr.begin(), std::move(arr[1]))), since
std::vector::insert(pos, T&&) is not guaranteed to handle an argument
that aliases one of its own elements.
This is a deliberate, small behavior change: the moved-from argument
now ends up null afterwards, the same as after push_back(&&), instead
of keeping its old value unchanged. No signature changes, so the
public API and ABI are unaffected. Add unit-modifiers coverage for
the moved-from state and for self-aliasing insertion, both with and
without reallocation of the underlying array.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate object key lookup and checked at() access
at()/find()/count()/contains()/const operator[]/erase_internal() each
repeated the raw object lookup (m_value.object->find(key)), and the
six at() overloads additionally repeated the type_error.304 check and
out_of_range.401/403 throw. Route them all through two new private
helpers, object_lookup()/object_at() (plus array_at() for the index
overloads of at()), templated on the constness of the receiver so one
body serves both the const and non-const overload. count() is left
untouched, since it already goes through object_t::count() rather
than a second find().
The at(KeyType&&) overloads used to forward the same key twice: once
into object->find() and again, on the not-found path, into the
string_t() conversion for the exception message. object_at() now
forwards it only into the lookup and reuses the (unmoved) key for the
message. clang-tidy 22 (Docker silkeh/clang:22) still flags that reuse
under bugprone-use-after-move/hicpp-invalid-access-moved even with the
single forward, since it cannot see that object_t::find() (a plain
std::map or ordered_map) never actually moves from its argument; add a
NOLINTNEXTLINE with that reasoning rather than avoid the pattern.
No signature, exception id/message, or set_parent() behavior changes.
Overlaps #5689, #5705, #5606, #5687 and #5585, which touch the same
hunks; whichever of this commit and those PRs lands second will need
a small rebase.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the lookup/default/throw body of value()
The six non-deprecated value() overloads each held a full copy of the
same body: the four key-based overloads looked up the key and either
returned the found element converted to the requested type or the
default value (throwing type_error.306 if this is not an object), and
the two json_pointer overloads did the same via
ptr.get_checked_or_null(), throwing type_error.306 unless
is_structured(). Replace the duplicated bodies with two private
helpers, value_member() and value_pointee(), that return a
const basic_json* (null when not found) and do the type check/throw
once each. Every value() overload now just picks between the found
pointer's get<T>() and the default.
Same signatures, template parameters, SFINAE conditions, exception id,
message and this context on every overload.
Overlaps #5689 (routes find() through lookup_key()) and #5705 (adds a
deleted integral-key value() next to these overloads); whichever of
this commit and those PRs lands second will need a small rebase.
#5724 item 4
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the null-to-container conversion into convert_null_to()
Nine sites wrote out the same "turn a null value into an empty array
or object" logic with two different idioms: operator[](size_type),
operator[](key_type), operator[](KeyType&&) and update() set m_type
then assigned m_value.array/object directly via create<T>(), while the
three push_back() overloads, emplace_back() and emplace() set m_type
then assigned m_value = value_t::array/object (going through
json_value's converting constructor and a temporary). Both idioms end
up calling create<T>() and produce the same state, just via a
different path; both also share a latent exception-safety bug, since
m_type is written before the (possibly throwing) allocation, so a
throwing allocator leaves m_type == array/object with a null pointer
behind it, violating the class invariant and crashing on the next
access to, or destruction of, the value.
Add a private convert_null_to(value_t) helper and call it from all
nine sites. Unlike the idioms it replaces, it allocates the container
first and only then writes m_type, so a throwing allocation leaves the
value as a valid null instead of a mistyped, half-constructed one;
verified with a throwing allocator (see unit-allocator.cpp's
bad_allocator) that j["x"] = ... on a null j now stays null, and no
longer trips assert_invariant()/crashes, when create<object_t>()
throws. Same allocator usage and assert_invariant() call as before,
otherwise.
Overlaps #5585, which reorders these same nine blocks for exception
safety; whichever of this commit and that PR lands second will need a
small rebase.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the linear key search in ordered_map
emplace, at, erase(key), count and find each repeated the same
"for (auto it = begin(); it != end(); ++it) if (m_compare(it->first,
key)) ..." loop (15 copies across their key_type and transparent
KeyType&& overloads), and both erase(key) overloads additionally
repeated the exception-sensitive in-place reconstruction (destroy,
placement-new, pop_back) used to remove an element while keeping the
const Key non-movable.
Add two private helpers: find_impl(Self&, KeyType&&), a static member
template that runs the search once for either constness of the
receiver, and erase_at(iterator), which keeps the existing
pop_back-based reconstruction instead of switching to erase()/resize()
(which would add a DefaultInsertable requirement). Route find, at,
count, emplace, insert(const value_type&) and both erase(key)
overloads through them.
Same signatures, is_usable_as_key_type constraints and exception
messages/types.
Overlaps #5609 and #5685, which both rewrite emplace (#5609 also
touches insert and adds private members at the end of the class);
whichever of this commit and those PRs lands second will need a
rebase.
#5724 item 6
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-enable bugprone-use-after-move/hicpp-invalid-access-moved
These two checks (and portability-template-virtual-member-function)
were disabled in #4489 (November 2024) "only removed to get the CI
going". portability-template-virtual-member-function is a separate,
still-open cleanup (#5725 item 3 on its own branch) and stays
disabled here; this commit only re-enables the move/forward checks
and cleans up what they flag on this branch.
The move constructor (json.hpp) already casts to the base type
instead of forwarding the whole object (#5724 item 9), so it no
longer trips either check. The at(KeyType&&) double-forward this
check used to flag was reduced to a single forward with the
now-unforwarded reuse annotated by a NOLINTNEXTLINE in #5724 item 3's
object_at() helper (clang-tidy 22 still flags that reuse even after a
single forward; see that commit's message). What is left here:
- from_json_inplace_array_impl(), from_json_tuple_impl_base() and the
std::pair overload of from_json_tuple_impl() forwarded j into every
j.at(...) call in a pack expansion or a pair of calls. at() has no
ref-qualified overloads, so the forward was a no-op; call j.at(...)
directly.
- container_input_adapter_factory::create() forwards container twice
on purpose, into begin() and end(), so both see the same value
category and produce matching iterator types. Annotate it with
NOLINTNEXTLINE and a comment instead of changing it.
- unit-class_parser.cpp's "move constructor resets the moved-from
value to npos" test still pointed at the pre-static_cast move
constructor by line number and mentioned the cppcheck-suppress
annotation that #5724 item 9 already removed; update the comment.
No behavior change anywhere in include/. Verified with clang-tidy 22.1.8
(Docker silkeh/clang:22, --platform linux/amd64) against a TU including
json.hpp with the repo's .clang-tidy: bugprone-use-after-move and
hicpp-invalid-access-moved report nothing unsuppressed.
Overlaps #5737 (open PR for the rest of #5725 item 3: the
from_json.hpp/input_adapters.hpp cleanup above, and
portability-template-virtual-member-function), which currently keeps
both checks disabled pending this move-constructor change; whichever
of this commit and that PR lands second will need a small rebase of
.clang-tidy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make convert_null_to() take the container type as a template argument
Passing array_t or object_t instead of a value_t makes an invalid target
a compile error instead of a runtime assertion, and removes the branch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make cross-string-type basic_json conversion explicit without implicit conversions
The converting constructor from another basic_json specialization was
always implicit, so a value with a different string_t (std::wstring, a
string with a custom allocator, ...) silently converted into a temporary,
e.g. when passed to a function taking const nlohmann::json&. Such
conversions do not produce correct values (#3425), and
JSON_USE_IMPLICIT_CONVERSIONS=0 did not catch them.
When JSON_USE_IMPLICIT_CONVERSIONS is 0, the constructor is now explicit
if the string types differ. Specializations sharing a string type (json
and ordered_json, different serializers or object maps) stay implicitly
convertible, so the NLOHMANN_DEFINE_TYPE_* macros keep working with
nested json members. get<BasicJsonType>() constructs explicitly and
works in both modes.
Fixes#2649.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Construct explicitly in get_to() and to_json(std::optional)
With JSON_USE_IMPLICIT_CONVERSIONS=0 the conversion from a basic_json
with a different string type is now explicit, but two library paths
still assigned such a value implicitly and failed to compile inside the
library:
- get_to() with a basic_json target (the #2175 overload) did
`v = *this`, so json(42).get_to(alt_json&) broke although
get<alt_json>() works.
- to_json(BasicJsonType&, const std::optional<T>&) is constrained on
std::is_constructible (which accepts the explicit constructor) but did
`j = *opt`, so converting a std::optional<alt_json> into a json broke.
Both now construct the value explicitly, as get_impl() already does.
Also replace static_cast<bool>(JSON_USE_IMPLICIT_CONVERSIONS) with a
comparison: clang-tidy's modernize-use-bool-literals rejected the cast
of the integer literal the macro expands to, failing ci_clang_tidy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Since #4958, a key type implicitly convertible to std::string_view was
accepted by is_usable_as_basic_json_key_type without checking that the
object's comparator can actually compare object_t::key_type with that
key type. The key was then forwarded unchanged to the underlying map,
so const operator[], at, find, count, contains, erase and value failed
to compile (a hard error inside <map>) for a key convertible only to
std::string_view, and value() rejected such keys outright. For keys
convertible to both std::string and std::string_view, the KeyType&&
templates now won overload resolution over the object_t::key_type
overloads and then failed the same way, a regression from 3.12.0. Only
the non-const operator[] worked, because it uses emplace(), which
constructs a std::string from the key explicitly. ordered_json was not
affected, since ordered_map checks comparability itself.
Add a trait, is_string_view_convertible_key_type, that recognizes a key
type that is convertible to std::string_view but not directly
comparable with the object's key type, provided std::string_view itself
is comparable with it. at(), operator[], find(), count(), contains(),
erase() and value() now route such keys through a new lookup_key()
helper that converts them to std::string_view before they reach the
object, matching how the object's transparent comparator already
supports std::string_view lookups.
Fixes#5663.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Accept lvalues in ordered_map::emplace's value parameter
ordered_map::emplace(key, value) took the mapped value only by T&&, an
rvalue reference rather than a forwarding reference, so
ordered_json::emplace("a", value) failed to compile whenever value was
an lvalue or a const lvalue, even though the same call compiles for
json (whose object_t is std::map, with a variadic emplace). Turn the
value parameter into a separately-deduced forwarding reference,
constrained with std::is_constructible so the overloads still only
accept something convertible to the mapped type. std::map-compatible
semantics are unchanged: emplace still does nothing if the key already
exists.
Open PR #5609 also touches ordered_map.hpp (moving values on vector
growth); this change only touches the two emplace() overloads and
should not conflict.
Fixes#5673.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid astyle's padding in ordered_map::emplace's template headers
Use detail::conjunction instead of && and drop the redundant V&& in detail::is_constructible, so astyle keeps the usual template formatting. Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The const operator[] overloads are unchecked by design, and a missing key
or index is undefined behavior. The key overload guards this with a
runtime assertion, but the index overload did not, although the element
access documentation says an assertion fires in both cases. The const
JSON pointer overload inherits both through json_pointer::get_unchecked(),
so a pointer to a missing array index read out of bounds even in debug
builds, and its documentation promised out_of_range.404 for any pointer
that cannot be resolved.
Add JSON_ASSERT(idx < size()) to const operator[](size_type), which also
covers the index leg of the const JSON pointer overload. Document the
undefined behavior for the const JSON pointer overload in operator[].md
and in the runtime assertions page. Release builds are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The comparison operators taking a scalar (==, !=, <, <=, >, >=, and
C++20's <=>) convert the scalar to a basic_json and compare, but were
unconditionally noexcept. When that conversion throws, the program
called std::terminate instead of propagating the exception, e.g. when
comparing a json with a string literal under memory pressure
(std::bad_alloc) or with an enum value not mapped by
NLOHMANN_JSON_SERIALIZE_ENUM_STRICT (out_of_range.410). clang-tidy
22.1 reports the latter as bugprone-exception-escape.
Declare the 16 scalar overloads
noexcept(std::is_nothrow_constructible<basic_json, ScalarType>::value):
they stay noexcept for numbers, Booleans, nullptr, and plain enums,
and are noexcept(false) for strings and enums whose to_json may throw.
The comparisons of two basic_json values are unchanged.
Restore the strict-enum comparisons removed from unit-conversions.cpp
in the previous PR, check that comparing an unmapped strict enum now
throws, and pin the new exception specifications in unit-noexcept.cpp.
Document the exception safety of overload (2) on all seven operator
pages. Ran make amalgamate.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conflicts in See also lists (docs/mkdocs/docs/api/basic_json/operator_ne.md), where develop (#5638) and this branch both edited: kept develop's entries and added this branch's basic_json_view links.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conflicts in See also lists (docs/mkdocs/docs/api/basic_json/begin.md,docs/mkdocs/docs/api/basic_json/cbegin.md,docs/mkdocs/docs/api/basic_json/cend.md,docs/mkdocs/docs/api/basic_json/end.md,docs/mkdocs/docs/api/basic_json/type_name.md), where develop (#5638) and this branch both edited: kept develop's entries and added this branch's basic_json_view links.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conflicts in the See also lists of nine basic_json pages, is_discarded.md, and features/index.md, where develop (#5638) and this branch both added entries: kept both. Ran make amalgamate.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conflicts:
- number_parse.hpp: kept this branch's float parser, which replaces the
Eisel-Lemire code that develop's side changed (#5750 made its digit
counter unsigned; this parser has no such counter, and it compiles
cleanly with GCC's -Wstrict-overflow=5).
- number_handling.md, template_parameters.md: kept this branch's
description of the conversion and added develop's "Before version
3.13.0" sentence.
Ran make amalgamate.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Review and extend the documentation, and check it in CI
A review of all documentation pages found factual errors, dead links,
missing cross-references, and gaps in examples. This fixes them and adds
checks so the same problems are caught automatically.
Fixes:
- wrong signatures and version histories (operator!= C++20 member,
binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...)
- stale descriptions (number parsing since #5283, UBJSON table, SAX
example that no longer compiled, tsl::ordered_map advice)
- dead internal and external links; repology.org badges (the domain is
suspended) replaced by badges that query the registries directly
- deprecation notes link the migration guide; the guide itself fixed
Additions:
- "See also" sections, cross-references, 25 runnable examples, 12
Mermaid diagrams, new API pages for json_pointer::operator<=> and
byte_container_with_subtype::operator==/!=
- landing page, guides for untrusted input and performance
- "unreleased" badge after versions newer than the latest release
Checks:
- strict documentation build (broken links/anchors fail it); CI and
the publish workflow fetch the full history the build needs
- weekly external link check, Mermaid syntax check in CI
- check_structure.py: example titles, heading levels, alt texts,
header links, docset index coverage; its unused-example check works
again
- all examples produce the same output on every platform
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the customer links that could not be fixed
A dead link on the customers page is still the evidence of where the
use of the library was documented. Keep the original URLs of the entries
without a working replacement (Marne, Cisco Webex Desk Camera, Philips
Hue, CyberArk) and exclude exactly these URLs from the link check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Correct the duplicate-key recipe's claim about SAX positions
The SAX interface's key() receives no position either; only parse_error()
does. Also note that the recipe does not report the path to the repeated
key (see discussion #5085).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Say the library is available as a single header and mention json_fwd.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Correct documentation errors found while hunting for bugs
- patch/patch_inplace: list the JSON pointer errors parse_error.106-109
and out_of_range.402/404, and quote the actual parse_error.105 message.
- unflatten: list parse_error.106/107/108 and out_of_range.404.
- to_bson: list out_of_range.415 (binary subtype above 255) and note
that 412 and 415 are new in 3.13.0.
- to_string: state that string_t must be convertible to std::string, also
in the StringType requirements table.
- JSON Lines: a `while (input >> j)` loop also throws after the last value
for concatenated JSON values; show a loop that works for both.
- BON8: a string gets 0xFF only if nothing follows it in the message; a
string at the end of an array or object is ended by 0xFE.
- custom_string_type.hpp: add operator+=(char), which the "Always
required" list asks for (json_pointer::to_string, flatten, unflatten,
and diff did not compile), and an ADL int_to_string for diff and items.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cache the release headers with functools.lru_cache
Codacy (Pylint) flagged the mutable default argument that header() used
as its cache. functools.lru_cache keeps the same memoization without it.
The script's output is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conflicted only in tests/src/unit-class_lexer.cpp, where develop's #5737
lint fix (CAPTURE(x); -> CAPTURE(x)) collided with this PR's rewrite of
the Eisel-Lemire float tests; kept the PR's new tests and applied the
lint-fixed CAPTURE style. single_include regenerated via make amalgamate.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix math formulas rendering as raw TeX in the published docs
The privacy plugin self-hosts MathJax 2.7.0 but drops its
?config=TeX-MML-AM_CHTML query string, so the rehosted script loads no
input jax and the 10 formulas across 6 pages render as raw TeX to
readers. Remove pymdownx.arithmatex and the MathJax extra_javascript
entry, and rewrite the formulas in plain HTML (<sup>, <i>) instead.
This also drops a nine-year-old third-party script from every page.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix publish_documentation triggers and persist unneeded git credentials
publish_documentation.yml only triggered on docs/mkdocs/** pushes, but
the site also embeds .github/CODE_OF_CONDUCT.md, CONTRIBUTING.md,
SECURITY.md, cmake/{clang,gcc}_flags.cmake, .clang-tidy,
tools/astyle/.astylerc and tests/fmt_formatter/project/main.cpp via
pymdownx.snippets, so changes to those files never republished the
site. Extend the path filter to cover them, and switch runs-on from
the long-pinned ubuntu-22.04 to ubuntu-latest to match
ci_test_documentation.
Also add persist-credentials: false to the checkouts in ci_icpx and
ci_nvhpc (ubuntu.yml) and msvc-vs2026/msvc-arm64 (windows.yml), none
of which pushes with git, matching every other checkout in these
workflows.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix example build: broken debug echo, deprecations hidden for everything
docs/Makefile's debug echo used a space instead of a comma in
$(call cxx_standard ...), so it always printed an empty standard.
Every example was also compiled with -Wno-deprecated-declarations,
which would silently hide an accidental deprecated-API call in any of
them. Factor the duplicated compile flags into EXAMPLE_CPPFLAGS/
EXAMPLE_WARNFLAGS, build with -Werror=deprecated-declarations by
default, and only allow the three examples that intentionally
document deprecated API (the DEPRECATED_EXAMPLES list) to suppress it.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix check_structure.py NOLINT parsing and an off-by-one report line
`line.strip("<!-- NOLINT")` followed by `.strip(" -->")` strips any of
the characters in those sets from both ends, not a literal prefix; it
only happened to work for "Examples". A NOLINT'd section name
starting with N, O, L, I or T (e.g. "Notes", "Template parameters",
"Iterator invalidation", "Literals") was silently mangled, so the
suppression did not apply and the checker could report a spurious
missing/misordered section. Parse the comment with a regex instead.
The same fragile strip() pattern was used for heading text; replace it
with a plain prefix slice. Also fix the admonition_title report, which
used the 0-based line index while every other report in the file uses
lineno+1.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix docset README's non-existent make target and stale fallback URL
The README told readers to run `make nlohmann_json.docset`, but the
Makefile's targets are `all`, `JSON_for_Modern_C++.docset` and
`install_docset_zeal`; the documented command has failed with "No
rule to make target" since #2967 (2021). Point the README at the real
target and folder name. Info.plist's DashDocSetFallbackURL also still
pointed at the old nlohmann.github.io/json/ URL instead of the
canonical https://json.nlohmann.me/ from mkdocs.yml's site_url.
Leave list_missing_pages/list_removed_paths alone: they may become
redundant once #5638's check_docset() lands, which is a follow-up.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove two leftover Doxygen-era .link files from the examples directory
parse__iterator_pair.link and parse__pointers.link each held only a
Wandbox "online" permalink from the old Doxygen docs. #3071 deleted
every other .link file in 2021; these two came in through a parallel
PR (#3100) and were never referenced by any page, script or config.
Part of #5718
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MacPorts CMake example to include its own snippet files
The MacPorts "Example: CMake" block included
integration/homebrew/example.cpp and
integration/homebrew/CMakeLists.txt instead of the MacPorts files
right next to it, a copy-paste slip from the Homebrew section. Nothing
referenced integration/macports/CMakeLists.txt as a result. The page
rendered correctly only because the homebrew, macports and
vcpkg/CMakeLists.txt snippets are byte-identical, so a future edit to
the MacPorts files would not have shown up on the page.
Part of #5718 item 6
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not persist git credentials in publish_documentation's checkout
The checkout step in publish_documentation.yml left the default
persist-credentials: true, so GITHUB_TOKEN stayed writable in
.git/config for the rest of the job (zizmor's artipacked finding). The
Deploy documentation step authenticates through its own github_token
input to peaceiris/actions-gh-pages and does not push with the
checked-out credentials, so persist-credentials: false is safe here,
matching every other checkout in the workflow set.
Overlaps #5638, which edits this same checkout step (adds
fetch-depth: 0); expect a rebase conflict there.
Part of #5718 item 2
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Replace list_missing_pages/list_removed_paths with a comm(1)-based diff
The docset Makefile's list_missing_pages ran one sqlite3 query per
mkdocs page, and list_removed_paths nested a loop over all mkdocs pages
inside a loop over all docset index paths (O(n*m) shell iteration).
Issue #5718 item 5 suggested removing or reducing these targets once
#5638's check_docset() lands, but that PR is still open and covers only
API pages and macros, not the full page set these targets check.
Replace the loops with two sorted path lists (DOCSET_PAGE_PATHS from
mkdocs' markdown sources, DOCSET_INDEX_PATHS from the built docset
index) compared with a single comm(1) call each, verified to produce
output identical to the old loops against the current docSet.dsidx.
The sed expression used '#' as its delimiter, which GNU Make reads as
a comment character even inside a variable assignment, truncating the
line and orphaning the closing paren of $(shell ...) ("unterminated
call to function 'shell': missing ')'"). Use '@' as the delimiter
instead.
Part of #5718 item 5
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for operator== and operator!= of basic_json_view, linked
both ways with the basic_json pages
- the feature page and the class overview list the comparisons
- the examples show when the view helps: detecting a changed document
without building json values
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for dump, number_format, and operator<< of basic_json_view,
linked both ways with the basic_json pages
- the feature page describes document order and number_format::source
- the examples show when the view helps: forwarding part of a message
and writing numbers exactly as they were read
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for get, get_to, get_string, number_token, and value of
basic_json_view; JSON pointer overloads of operator[], at, and
contains; links both ways with the basic_json pages
- the feature page describes which conversions copy nothing
- the examples show when the view helps: strings without copies, numbers
exactly as written, and paths into a large text
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for operator[], at, front, back, find, contains, count,
begin, end, cbegin, cend, items, and type_name of basic_json_view,
linked both ways with the basic_json pages
- the feature page and size() describe document order and duplicate
keys
- the examples show when the view helps: reading a few fields of a large
text, probing optional members, and members in source order
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- API pages for basic_json_document and basic_json_view, one per member,
and for the four aliases, each with an example
- features/json_view.md: the problem the view solves, ownership and
lifetime, what matches basic_json::parse() and what differs, and when
to choose json, ordered_json, SAX, or the view
- the examples show why one would use the view, not only how: borrowed
vs. owned input, reading a few fields and materializing one subtree,
reusing a document across many messages
- registered in the mkdocs navigation, llms.txt, the docset, the
exceptions page (out_of_range.416), architecture.md, the integration
page, and the README; the yyjson credit is added to the README and
license.md
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION to fix std::tuple conversions
basic_json can be constructed from std::tuple<json&>, which it turns into
a one-element array. Because of this, std::tuple picks its converting
constructor that converts the whole source tuple instead of the
element-wise one. As a result, std::tuple<const json&> built from
std::forward_as_tuple(j) binds to a temporary (a compile error with libc++,
a dangling reference with other standard libraries), and std::tuple<json>
built the same way holds [j] instead of a copy of j.
The new opt-in macro JSON_DISABLE_TUPLE_REFERENCE_CONVERSION (CMake option
JSON_DisableTupleReferenceConversion) removes the conversion from a
one-element tuple holding a reference to the same basic_json type, so
std::tuple converts element-wise. It is off by default, so existing
behavior is unchanged. It does not change any function body and therefore
is not part of the ABI tag.
Fixes#2226
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Convert one-element tuples to arrays on every compiler
to_json for std::tuple assigns a braced list, j = { std::get<Idx>(t)... }.
With a single element that is itself a basic_json, Apple clang 15 and 16
treat j = {x} as a copy of x, so std::tuple<json>{true} became true
instead of [true]. The macOS jobs (Xcode 15.1, 16.1) failed the new
checks in unit-disable-tuple-reference-conversion and unit-regression2.
The one-element overload that already handles
JSON_BRACE_INIT_COPY_SEMANTICS builds the array (or object, for a
[string, value] element) explicitly, the same way the initializer-list
constructor does. Use it unconditionally. The output is unchanged on
compilers that already wrapped the element.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip json reference tuple tests on clang < 4 and GCC < 5
ci_test_compilers_gcc_old (4.8) and ci_test_compilers_clang (3.4) could
not compile the new tuple tests. Creating a std::tuple of basic_json
references, e.g. std::forward_as_tuple(j), makes these compilers
instantiate basic_json's conversion operator for libstdc++'s internal
tuple bases, which fails hard. This happens with and without
JSON_DISABLE_TUPLE_REFERENCE_CONVERSION, so it is a limitation of these
compilers, not of the new option.
Tested with the CI images: clang 3.4 to 3.9 and GCC 4.8 and 4.9 fail,
clang 4, 5, and 6 and GCC 5 and 6 compile all cases. Skip only the
checks that create such tuples; the is_constructible checks still run.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
float, double, and long double where it is IEEE-754 binary64 (MSVC, Apple
arm64) are now converted by the library itself, correctly rounded and
independent of the locale and of the C and C++ libraries:
- The token is split into sign, significand w (at most 19 digits), and
decimal exponent q, using the positions of the decimal point and the
exponent that the scanners already recorded, so no character is
classified again.
- Clinger's fast path where w and 10^|q| are exact.
- Eisel-Lemire otherwise, now templated for binary32 and binary64.
- For tokens with more than 19 digits whose w and w + 1 round differently,
an exact big-integer comparison with the midpoint between the two
candidates (the digit comparison of fast_float, simplified).
This replaces the separate token walks of Clinger's fast path and of
Eisel-Lemire, the significant-digit gate that avoided the former, and, for
float and double, std::from_chars and the locale-aware strtod. std::from_chars
and strtold remain only for other long double formats (x87, binary128,
double-double) and for types that are not IEEE-754. Values are bit-identical
to before wherever the previous conversion was correctly rounded; tokens
converted in a locale with a multi-byte decimal point are now also exact.
Overflow still gives out_of_range.406, underflow a signed zero.
convert_float() is the entry point for other parsers of JSON text: it
converts like the lexer, without allocation for binary32/binary64.
Tests: exact-bit tests for double and float (ties, subnormal and overflow
boundaries, huge exponents, more digits than any midpoint), Eisel-Lemire for
binary32, the round trips of 200,000 doubles and 100,000 floats without
declines, 508 generated hard cases with the expected bits of both formats
(float_hard_cases.hpp) through the converter and both scanners, and
JSON-level overflow/underflow checks for double and float. The locale tests
now check the values in a locale with a multi-byte decimal point.
Docs: the statements that parsing uses strtod/strtof/strtold; the fast_float
credit now names the digit comparison.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_json(first, last) treated value_t::binary like the structured
types (array, object) in the range check, so it always copied the
whole binary value regardless of the iterators, even for an empty
range such as (b.end(), b.end()). The other primitive types (number,
boolean, string) already reject such a range with
invalid_iterator.204, and erase(first, last) already does the same
for binary values, so this made the constructor inconsistent with
both. Move case value_t::binary into the group of checked primitive
types.
Also update the two matching passages in basic_json.md that describe
overload 7, and add a version-history note.
Fixes#5670.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When a parser callback rejects an object's or array's start event,
json_sax_dom_callback_parser kept calling it for everything inside
that container anyway: nested keys, values, and the start/end events
of containers below it. This contradicts parser_callback_t's own
documentation, which promises that discarding a container at its
start event also hides its content from the callback.
The same code path also kept a full copy of every key inside such a
discarded container in key_stack until the whole parse finished,
because the early return for values that are not stored skipped the
matching pop. Filtering out a large subtree is the main reason to use
a callback, so this made peak memory during the parse scale with the
size of the very subtree the callback was trying to skip.
Fix start_object(), start_array(), and key() so that a container
whose own start event was discarded, or that is nested inside one, is
never handed to the callback, and no longer pushes onto the key
stacks. A container whose start event was accepted but whose key was
rejected still gets its content reported, as documented ("the
callback is still called for the associated value, but its return
value has no further effect"); only its own bookkeeping is skipped
since it will not be stored.
Fixes#5643.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
j.contains(0), j.find(0), and j.count(0) used to compile: the literal 0 is
a null pointer constant, so it converts to a null const char*, and the
overloads taking const typename object_t::key_type& accepted it by
constructing a std::string from that null pointer, which is undefined
behavior (a crash with both libc++ and libstdc++). value(0, default_value)
had the same problem in C++11, where the object comparator is not
transparent.
Add deleted overloads for integral arguments to contains(), find()
(const and non-const), count(), and value() so that these calls are
compile errors in every supported language mode instead of crashing.
Calls with string, string_view, json_pointer, and size-typed element
access (at(), operator[](), erase()) are unaffected.
Fixes#5657.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
to_bson() rejected a binary value's subtype above 255 (out_of_range.415)
in write_bson_binary(), which only has the binary_t, not the basic_json
value that holds it, so the exception was created with no JSON_DIAGNOSTICS
context even though the equivalent to_msgpack() check names the value's
path. The check also ran after the document size, all preceding elements,
and this element's header and length had already reached the output
adapter, so a caller-provided std::vector or std::string ended up holding
a truncated document.
calc_bson_sizes() already walks every value before anything is written,
to size embedded documents and arrays and to reject invalid keys
(out_of_range.409) up front. The subtype check now runs there instead,
in calc_bson_binary_size(), which is given the basic_json value so the
exception can use it as context. The now-redundant check in
write_bson_binary() is removed, since calc_bson_sizes() always throws
first if any binary value in the document has an oversized subtype.
Fixes#5675.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Parsing from a std::istream crashed in two unusual but valid stream
states, both in input_stream_adapter:
- With eofbit in the stream's exceptions() mask, get_character() sets
eofbit via is->clear(), which throws std::ios_base::failure. While
that exception unwinds, ~input_stream_adapter() called clear() again
to reset eofbit, which is still set and still in the exception mask,
so it throws a second time out of the (implicitly noexcept)
destructor and std::terminate() is called. The destructor now only
calls clear() if a bit other than eofbit remains set, so the first
exception can propagate normally.
- For an std::istream without a stream buffer (rdbuf() == nullptr,
e.g. std::istream(nullptr)), the constructor stored the null
pointer without checking it, and get_character() dereferenced it.
input_adapter(std::istream&) now throws parse_error.101 for such a
stream, the same as it already does for a null FILE* or char*.
Added regression tests to unit-deserialization.cpp and, for the
JSON_PRECISE_STREAM_POSITION variant of get_character(), to
unit-precise-stream-position.cpp; both crashed before this fix.
Documented the two exceptions in parse.md and operator_gtgt.md.
Fixes#5646.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
basic_json::swap() (and the friend swap() and the pre-C++20 std::swap
overload that forward to it) only exchanged m_data.m_type/m_data.m_value,
leaving each value's json_base_class_t subobject in place. This is
inconsistent with the copy and move constructors and copy assignment,
which all carry the base class along with the value, so after
a.swap(b) any metadata stored in a CustomBaseClass ended up attached to
the wrong value. Algorithms that mix swap() with moves, such as
std::sort, scrambled the metadata across the whole container.
Fix the member swap() to also exchange the json_base_class_t subobject
and extend the noexcept specifications of swap() and the friend swap()
accordingly.
Fixes#5653.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_msgpack() reading the inactive number union member
basic_json stores number_integer and number_unsigned in a union, and
number_unsigned_t only has to be at least as wide as number_integer_t
(with the default types, both are 64-bit and have the same
representation). When number_integer_t is narrower, write_msgpack()
read the wrong union member in two places:
- The number_unsigned case wrote number_integer's bits instead of
number_unsigned's, silently writing the wrong value whenever it
did not fit in number_integer_t.
- The number_integer case (non-negative branch) picked the encoded
width by comparing number_unsigned's bits, which is undefined
behavior, though the value written was still number_integer's, so
at worst a too-wide encoding was chosen.
Read the active member in both cases, like the other binary writers
(CBOR, UBJSON, BJData, BSON, BON8) already do.
Fixes#5644.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cast number_integer to number_unsigned_t only once in to_msgpack()
Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Copy values before inserting an initializer list into an array
insert(pos, {...}) inserted wrong values when the initializer list
contained const references to elements of the array being inserted
into. json_ref stores only a pointer for a const lvalue, so the
initializer_list_t range passed straight to the array's range insert
aliased the array's own storage; std::vector::insert(pos, first, last)
may move or shift elements before copying from that range, so the
source elements were already stale by the time they were read
(different wrong results on libc++ and libstdc++).
Copy the referenced values into a temporary array_t first, then move
that temporary into place, so the source range never aliases the
array being modified.
Fixes#5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the reserve_array helper in the initializer_list insert fix
The previous commit called array_t::reserve() directly on the
temporary buffer used to copy an ilist's values before inserting.
std::deque, a documented ArrayType (tests/src/unit-custom-array-type.cpp),
has no reserve(), so insert(pos, initializer_list) no longer compiled
for it. Use the existing detail::reserve_array() SFINAE helper (already
used by the SAX DOM parser) instead, which leaves array types without
reserve() untouched.
Added a regression check that deque_json::insert(pos, {...}) compiles
and handles the aliasing case from #5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
For idx == SIZE_MAX, the non-const array operator[] computed the new size as
idx + 1, which wraps to 0. resize(0) then emptied the array, and the
subsequent operator[](idx) on the now-empty vector wrote one element before
its buffer. Every other too-large index (e.g. SIZE_MAX - 1) already went
through resize(), which throws std::length_error and leaves the array
unchanged; SIZE_MAX was the one value for which the overflow bypassed that
safety net.
Add a guard that throws std::length_error before computing idx + 1 when idx
is the largest representable size_type value, so the array is left
unchanged, matching the exception vector::resize() already throws for
smaller (but still too large) indices.
Fixes#5647.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clear() to also reset the subtype of a binary value
clear() on a binary value cleared the bytes but left the subtype
untouched, so the result was not equal to a default-constructed
binary value even though the documentation says clear() has the same
effect as *this = basic_json(type()). The fix calls
byte_container_with_subtype::clear_subtype() alongside the existing
clear() call.
Extended the "filled binary" clear() test in unit-modifiers.cpp with
a case that uses a subtype, since the existing cases only covered
binary values without one.
Fixes#5669.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the table alignment in clear.md
Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the key type when rejecting non-string CBOR/MessagePack map keys
CBOR and MessagePack allow map keys of any type, but JSON object keys
are always strings, so such maps are rejected. The error so far was the
one for a malformed string (e.g. "expected length specification
(0xA0-0xBF, 0xD9-0xDB); last byte: 0xC0" for a nil key), which does not
tell the user what went wrong. Report the type of the key instead:
syntax error while parsing MessagePack object key: only string keys
are supported, but found nil; last byte: 0xC0
The exception id (parse_error.113) and type are unchanged. Malformed
string keys and a missing key keep their previous messages. Document
the restriction on the CBOR and MessagePack pages.
Refs #2766, #3381
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Point the MessagePack key note to the spec's profile section
The note linked to "Serialization: type to format conversion", which says nothing about key types. Restricting map keys to strings is only mentioned in the "Profile" section (under "Future discussion") as an example of a JSON-compatible profile, so link there and describe it as such instead of as a permission.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The line added in #5559 exceeded the 160-character limit enforced by
docs/mkdocs/scripts/check_structure.py, breaking the documentation build.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add BON8 support
Add to_bon8/from_bon8 and input_format_t::bon8 for BON8, a binary format
that uses the byte values that cannot begin a UTF-8 character as type
markers, so strings need no length prefix. It is the most compact of the
supported binary formats on the benchmark files.
The reader is non-recursive like the other binary readers. A string ends
at the first byte that cannot continue it, so the reader hands the one or
two bytes it reads past a string back to the value that follows. The
writer produces the canonical representation of the specification, except
for NFC normalization; its output is identical to that of the reference
implementation (HikoGUI) on all files of the test data.
The round-trip tests need the .bon8 files of json_test_data 3.2.0.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review comments
- Reuse detail::validate_one_utf8 to check strings in to_bon8; the error
now names the first byte of the invalid sequence.
- Document that to_bon8 leaves bytes in the output adapter on an
exception, and that string_open is only an output of write_bon8_marker.
- Explain why the pushback buffer of the BON8 reader cannot overflow.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Select the BON8 float prefix by type
get_bon8_float_prefix only depends on the type of its argument, so make
the type a template parameter instead of passing an unused value.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Rename a test variable that Flawfinder mistakes for read()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the BON8 CI failures
- compare the float in write_bon8_float with number_float_t constants,
so GCC does not warn about a float-to-double conversion
- mark check_bon8_utf8's context as used when exceptions are disabled
- choose the compact float prefix in a helper rather than with nested
conditional operators (clang-tidy)
- use auto for the cast in the BON8 integer reader (clang-tidy)
- write the int32 minimum test values as long long literals (MSVC C4146)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read BON8 strings in bulk from contiguous input
- copy the valid UTF-8 of a string in one step when the input is
contiguous (twitter.json is read in 1.68 instead of 2.52 ms,
jeopardy.json in 196 instead of 297 ms, close to CBOR and MessagePack)
- share the new valid_utf8_prefix() with the writer's UTF-8 check, which
now skips ASCII 8 bytes at a time
- let the fuzzer check that contiguous and stream input give the same
value or error, and test both paths in the unit tests
- clarify that a second 0xFF after a string is an empty string
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Link the BON8 functions from the other binary format pages
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the bulk scan flag after the input, not BON8
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read BSON keys in bulk from contiguous input
BSON keys (and array indices) are C-style strings, which were read byte
by byte. For contiguous input they are now read up to their \x00-byte in
one step, using the same bulk_scan flag as BON8 strings: twitter.json is
read in 1.46 instead of 2.01 ms, citm_catalog.json in 2.93 instead of
3.33 ms, jeopardy.json in 182 instead of 207 ms. canada.json, whose keys
are almost all one-digit array indices, takes 2 % longer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the BON8 CI failures of the bulk-read tests
- skip the contiguous-versus-stream tests of BON8 strings and BSON keys
when exceptions are disabled: they catch the parse errors of invalid
input, and without exceptions the library aborts instead
- use static_cast for the int64 test value (google-readability-casting)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the explicit basic_json instantiation into its own test file
Linking test-regression3_cpp20 with clang and MinGW failed with
"relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'",
as test-regression2 did before #5511. The explicit instantiation of
basic_json<> for #4825 compiles every member function, including the
BON8 reader and writer, into that object, and it was already close to
the limit (2,226,104 bytes on develop, 2,234,960 with BON8; clang -O1,
C++20).
Give the instantiation a file of its own: unit-regression3 is now
1,594,736 bytes and unit-explicit_instantiation 1,095,064. The new file
mentions JSON_HAS_CPP_17 and JSON_HAS_CPP_20 so it keeps being built
for the C++17 standard the regression was about.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Convert the bytes of the BON8 test strings explicitly
The str() helper constructed a std::string from a byte range, which
converts each unsigned char implicitly; -fsanitize=integer reports that
for bytes of 0x80 and above (ci_test_clang_sanitizer).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
