from_json(ArithmeticType&) now reads numbers through get_number_value() and
handles a boolean on a cold path, which puts the three number types first in
the per-element type dispatch.
get<std::array<T, N>>() always took the identity_tag overload, a pack
expansion of N calls j.at(i).get<T>() that clang unrolls into straight-line
code (about 28,000 instructions for N = 1000), so its speed depended on
inlining decisions in that body. For trivial element types, which cannot tell
default construction plus assignment from construction in place, it now fills
a std::array in a loop. That loop, shared with get_to(), checks the size once
and reads the inner array directly; a short array still converts the elements
it has and then throws the out_of_range error at() would throw.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Taking the second-byte range from utf8_shape() made BON8 decoding 2-3% slower
(FromBinaryShape/scalars, FromBinaryBuffer, alignment-pinned builds), so the byte
path keeps its hand-written ranges. The exhaustive validator test still checks
that it accepts the same sequences as the other validators.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Moving parse_array_index out of json_pointer stopped clang from inlining it
into the pointer resolution (JsonPointerAt/citm_catalog +17%,
JsonPointerAt/twitter +8%); JSON_HEDLEY_ALWAYS_INLINE restores the baseline.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Calling get_number_value() from from_json(ArithmeticType&) made
GetArray<std::array<int, 1000>> 2.4x slower (it changes the inlining of the
std::array conversion), so that overload keeps its own switch, with a comment.
get_arithmetic_value() still uses the shared helper.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Move json_pointer::parse_array_index() and its status enum to namespace detail
(json_pointer keeps a forwarding member), and let the json_view pointer
resolution use it instead of its own copy of the RFC 6901 index grammar. Both
accepted the same tokens with the same outcome. Also update the comment on
detail::decode(), which listed four of the five UTF-8 validators and now points
to the test that keeps them in sync.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
sax_parse keeps its if (result) guard and return value. Also adds the
clang-tidy NOLINT for the 2.3.6 message table.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Holds errored, allow_exceptions, parse_error() and is_errored(). The scalar
handlers stay in each class (parse hot path).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_bon8_string takes the valid range of the second byte from utf8_shape()
(0xF5..0xF7 stay errors, unget behavior unchanged). validate_one_utf8 keeps its
if-chain (hot bulk path); the new test checks decode, validate_one_utf8,
utf8_shape, json::accept and the BON8 reader against each other for every lead
and second byte.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- detail::exception_id at every numeric id; throw_* helpers take it
- hex_byte, hex4_lower, utf8_shape replace local hex/table code
- throw_array_index_out_of_range/_past_end, throw_key_not_found
- has_native_float_format replaces the hand-written IEEE check
No change of messages, ids or output.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The three number cases of get_arithmetic_value and the arithmetic
from_json share one helper (boolean stays accepted only by the latter);
the array and object reserve overload pairs become one.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Object and array stay the first branches; everything else is the
primitive case. operator== and operator< share the different-container
check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The leading-slash and invalid '~' escape checks are shared helpers; the
reference-token walks test object and array first and treat everything
else as primitive.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The four create() bodies build their message through one protected
static helper; the diagnostics parent walk handles array/object and
ignores all other types.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
1.1: build ascii_copyable() at compile time from is_ascii_copyable and
test find_ascii_copyable_run against it for every byte. 1.2/2.5.4: use
detail::hex4_lower in both \uXXXX writers. 2.1.6: factor the repeated
flush/emit blocks in dump_escaped_impl into always-inline helpers.
2.1.7: comment why the escape copy does not use put_buffer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The CBOR/MessagePack/UBJSON/BSON/BJData/BON8 literals in throw_on_discarded and
throw_integer_out_of_range calls now come from detail::input_format_name; the
texts are identical, and UBJSON vs BJData still follows use_bjdata.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_cbor_key and write_msgpack_key were the same SFINAE pair apart from the
string/scalar writers; both now delegate to write_object_key<Walk>. UBJSON key
and to_format_length stay as they are.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The 13 oa.write_characters(reinterpret_cast<const CharType*>(x.data()), n)
sites now call write_bytes(const void*, size_t); copy_characters sites and the
buffer-based writes are untouched.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_ubjson_start_array/_object are now write_ubjson_start_container<IsArray>,
keeping the array-only begin()+1 scan and excessive_valueless rule.
ubjson_frame derives from binary_container_frame instead of duplicating it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_cbor_iterative and write_msgpack_iterative (and their *_value_or_push
helpers) were identical but for the head, scalar, and key writers. They are now
one template over a small policy struct, so everything inlines. BON8 keeps its
own walker (open string run, closing marker), noted in a comment.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Array and object prefixes differed only in the fix marker and the 16/32-bit
markers; both now delegate to write_msgpack_container_prefix. Output is
byte-identical.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_bson_entry_header and write_bson_string re-derived the error-handler
policy differently. Both now use sanitize_utf8_for_bson_write, which treats
strict like keep (the size pass already validated) and sanitizes replace/ignore
into the bytes the size pass measured. No context plumbing is needed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Add detail::is_primitive(value_t), detail::hex4_lower(), detail::utf8_shape(),
detail::input_format_name(), detail::empty_input_message(), the exception ids
116, 319, 320 and 416 to detail::exception_id, and make
detail::is_ascii_copyable() constexpr. Later commits switch the duplicated
code over to them.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5876 added JSON_LITTLE_ENDIAN_TARGET and JSON_BIG_ENDIAN_TARGET to
macro_scope.hpp. They must stay macros because they select code in #if,
so suppress the clang-tidy check as simd_scan.hpp already does.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Flatten into a std::map by inserting next to the previous entry
flatten() added every path with operator[], so each insertion into the
std::map of the result searched the tree from the root and compared long
shared path prefixes, which took half the time on canada.json. Paths come
out in nearly sorted order, so they are now inserted with emplace_hint
next to the entry added last, for object types that have it. ordered_map
keeps its append without lookup, and other object types keep
operator[].
Path segments are also appended in place, without a temporary string
for each escaped key or array index.
flatten() of canada.json, citm_catalog.json and twitter.json is 1.4x to
1.9x as fast; ordered_json is unchanged (1.05x). The output is
unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Unflatten without splitting keys or walking each path from the root
unflatten() split every key into a vector of reference tokens, stored the
pointer prefixes in a tree to decide which values become arrays, and then
looked up every token of every key from the root again.
Keys are now checked by a scan that raises the same parse_error.107/108 as
split(), skipping the part a key shares with the previous one, which was
checked already. The prefixes that denote arrays are kept in a small hash
set of key slices; the array/object decision is unchanged. The walk keeps
the values along the previous key's path, so only the part of a key that
differs from the previous one is looked up. Keys of a flattened object are
usually sorted, so that part is short.
unflatten() of canada.json, citm_catalog.json and twitter.json is 1.9x to
2.3x as fast; the output and the exceptions are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Apply plain JSON Patch operations without building a json_pointer
patch() built a json_pointer for every operation, which splits the path
into a vector of strings, and for "add" and "remove" a second one for
the parent. That took 20% to 45% of the time not spent copying the
document.
"add", "remove", "replace" and "test" with a path that has no escapes and
denotes existing values (the parent, for "add"), as diff() creates them,
are now applied by resolving the path in place, reusing one string for
the reference tokens. Every other operation, and every operation that
would fail, takes the existing code, so the results, the exceptions and
the state left after a failing operation are unchanged.
patch() of the diffs of canada.json, citm_catalog.json and twitter.json
is 1.02x to 1.10x as fast; the rest is the copy of the document.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build diff operations member by member
diff() created every operation from an initializer list, so each
{"op", "replace"} pair became a temporary array with two heap strings
before it was turned into an object member; an operation took about 25
allocations. The operations are now built as objects whose members are
assigned one by one. Paths of members and elements are also built in
place, without a temporary string for each escaped key or array index.
diff() of canada.json, citm_catalog.json and twitter.json, with every
eighth scalar changed, is 1.3x to 1.6x as fast; the output is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Diff objects with the same keys in lockstep and keep one path buffer
diff() looked up every key of the source object in the target object,
and built a new string for the path of every value it descended into.
Objects of the same size usually have the same keys in the same order,
so they are now walked together as long as that holds, without a
lookup; objects that differ fall back to the lookup from the first key
that does not match. The path is one buffer that is extended before
descending and shortened again afterwards. Objects whose members are
equal but in a different order (possible with ordered_json) still get
no operations.
Together with the previous commit, diff() of canada.json,
citm_catalog.json and twitter.json is 1.5x to 2.0x as fast; ordered_json
1.65x to 1.85x. The output is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use std::array for the index buffer in append_index()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare the partner iterator in the loop initializer in diff_object_keys()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix GCC and clang-tidy warnings in the JSON pointer speedups
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use standard library helpers where available, with C++11 fallbacks
Add detail helpers that call the standard library function when the
library provides it (checked with the __cpp_lib_* feature-test macros, so
no new ABI-tag macros) and a C++11 implementation otherwise:
- detail::clamp (std::clamp): by value, so that static constexpr members
need no out-of-line definition before C++17; used in json_view's dump()
buffer sizing, where step() can now be a static constexpr member.
- detail::bit_cast (std::bit_cast): replaces dtoa_impl::reinterpret_bits
and the memcpy bit copies in the number parser, to_chars, the json_view
serializer, and binary_writer::to_char_type.
- detail::byteswap (std::byteswap): one implementation instead of the
binary writer's own byte_swap() overloads and the bswap chain in
to_chars' store_msb_first().
- count_leading_zeros/count_trailing_zeros: two 32-bit _BitScanReverse/
_BitScanForward calls for MSVC x86/ARM32 instead of the loop, and
std::countl_zero/countr_zero for compilers without builtins.
- conjunction, disjunction, negation, bool_constant, and uncvref_t map to
their std counterparts in C++17/C++20.
Builtins and intrinsics stay ahead of the <bit> functions, as MSVC's
std::countl_zero checks for LZCNT at run time unless AVX2 is enabled.
The generated code is unchanged with Clang (C++11/20/23, arm64 and
x86-64) and GCC 16 for parsing, dumping, and the binary formats. The
binary reader keeps its own byte_swap() and the writer its memcpy calls:
the smaller code from detail::byteswap/std::bit_cast there made GCC 16
inline get_number()/write_number() differently, which slowed reading by
10-18 % and writing by about 1 %.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the target byte order once (JSON_LITTLE_ENDIAN_TARGET, JSON_BIG_ENDIAN_TARGET)
to_chars.hpp and bit_ops.hpp repeated the long __BYTE_ORDER__/_MSC_VER
conditions; macro_scope.hpp now defines them once (undefined again in
macro_unscope.hpp).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Explain why the serializer growth bound is not a clamp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With ensure_ascii, the view's dump wrote every string through the escaping
path, which decodes UTF-8 byte by byte, even strings of the source that are
plain ASCII. A string of the source without escape sequences has no quote,
backslash, or control character, so ensure_ascii leaves it unchanged
exactly when it has no byte >= 0x7F. The dump now checks that eight bytes
at a time (one add and one or per word) and copies such strings like the
dump without ensure_ascii.
The compact walk (dump_compact) now also serves ensure_ascii, through a
separate out-of-line instantiation, so the walk without ensure_ascii keeps
its speed.
Parsing is unchanged. A flag set by the parser would let the dump skip the
check and was about twice as fast for compact output, but it costs every
parse of string-heavy input 1-2% with GCC.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
dump() now checks strings of up to 32 bytes against a table of the bytes
that can be written verbatim, four bytes per step, and copies them in one
go if all of them can. That is cheaper than setting up the vector scanner,
whose setup costs more than such a string takes to scan; SQLite's
jsonAppendString() writes strings with the same kind of table loop. Most
object keys and many values are this short (citm_catalog averages 8 bytes
per string, jeopardy 14).
GCC 16 and 13 stopped inlining put_indent() and put_literal() into the
pretty-printing loops once dump_escaped() had grown, which made indented
output up to 13% slower; both are now always inlined.
The new test puts every ASCII character and a 2-byte character at every
position of strings of 1 to 40 bytes and compares the output with the
reference implementation of the existing UTF-8 tests.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
json_sax_dom_parser::key() copied the key from the lexer's buffer into the
object. The lexer starts every token with an empty buffer (reset()), so the
key can be moved instead, which saves an allocation and a copy for every key
that does not fit into the string's inline buffer. json_sax_dom_parser
already moved string values the same way. ordered_map has no rvalue
operator[], so ordered_json still copies keys.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Match literals without a stack array and keep parser states in bytes
scan_after_whitespace() built the expected text of true, false, and null in
local std::array objects. With -fstack-protector-strong (the default of
Ubuntu's GCC, among others), a local array gives the function a stack canary
check, which then ran on every token. scan_literal() now takes the text
after the first character as a string literal, with its length as a template
parameter; for contiguous input it compares the rest at once and falls back
to the byte loop on a mismatch, so errors are reported at the same
character. It is out of line, which keeps the token dispatch small.
The parser kept the nesting of arrays and objects in a std::vector<bool>,
whose push_back() GCC calls out of line; it is a std::vector<std::uint8_t>
now.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the literal fast path test without exceptions
The test case compares the error messages of both lexer paths, so it does
not compile with JSON_NOEXCEPTION (the caught exception does not exist).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Allow the string literal parameter of scan_literal() for clang-tidy
The parameter takes the literal by array reference so that its length is
a template parameter; this is the same exemption the C array overloads of
to_json() and from_json() have.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann