Compare commits

..
Author SHA1 Message Date
Niels Lohmann 7b2a1b2de5 Docs: fill documentation gaps found in user questions
Add pages on class-hierarchy/smart-pointer/variant/chrono recipes, working
without exceptions, and large, streamed and partial input. Extend the FAQ
(23 entries, Known bugs first), parse error documentation (empty-input
checklist, line/column, diagnostic positions), element access, conversions,
creating values, ADL troubleshooting, memory, and build/distribution notes.
Add 14 examples with generated output. Correct the statement that parse errors
cannot be recovered from with -fno-exceptions.

The topics come from recurring questions on Stack Overflow, Reddit, Hacker
News, YouTube, and the project's GitHub discussions and issues.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 19:13:49 +02:00
Niels Lohmann 01139dc192 Silence an Infer false positive in json_view materialize (#5905)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 17:10:31 +02:00
Niels Lohmann 9c50bdd140 Make the DOM parser cheaper per container and per member (#5882)
* Reserve the parser's nesting stack once

The stack of open arrays and objects was a std::vector<std::uint8_t> that
allocated at the first push_back and then reallocated at 1, 2, 4, 8 bytes,
so a document nested four levels deep made four allocations before it
parsed a value. A document that starts with a container now reserves 32
levels in one allocation; deeper documents grow as before. The loop that
reads the tokens is the code it was, so large documents are not affected
(a stack object with its own storage, and a reserve at every container,
both cost the token loop instructions or cycles).

Fixed cost of parsing a 124-byte object (macOS instructions): accept
-7.6 %, DOM -2.5 %; no change for a single number.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build arrays of the DOM parser in one exact-size allocation

Reading an array of unknown length appended its elements to a std::vector
one at a time, so a small array reallocated (and moved its elements) at
sizes 1, 2, 4, ... The DOM SAX parser now collects the elements of arrays
whose length is unknown on one stack shared by all open arrays and moves
them into an array of exactly the right size when the closing bracket
arrives. An array that grows past 16 elements switches to appending to its
own vector, so large arrays pay no second copy. Binary formats, which
announce the length, and builds with JSON_DIAGNOSTICS or
JSON_DIAGNOSTIC_POSITIONS (whose values need their final place from the
start) keep storing into the array directly.

The stacks of open containers are reserved on first use instead of growing
from one element.

The binary readers get a DOM parser of their own, json_sax_dom_binary_parser:
it is the plain parser that stores every value directly in its container
(the code the readers used before). Their containers announce their size, so
the bookkeeping of the collecting parser (frame entries, the destination of a
finished container) was pure overhead for them: with the collecting parser
FromBinaryShape nested was 3-12 % and containers up to 5 % more instructions
(GCC) than before. With the plain parser Linux Ir of FromBinaryShape and
FromBinaryBuffer is within 0.8 % of develop (Clang: identical).

canada.json has 56 000 two-element arrays that took three allocations each
and now take two. DOM parse Ir on Linux against develop (GCC / Clang): canada
-7.0 % / -7.7 %, citm -0.8 % / -1.6 %, 124-byte object -5.4 % / -5.5 %;
accept and SAX are untouched.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Reuse one value for the key events of the callback parser

parse() with a callback built a new string value for every key it passed to
the callback (a heap allocation and release per key), and copied the key a
second time into the object. The callback parser now keeps one string value
for key events, which it assigns, and moves the key into the object.

Callback parse, Linux Ir (GCC / Clang): twitter -13.3 % / -11.7 %, citm
-9.5 % / -7.6 %. (Earlier macOS measurement: twitter -19.5 % instructions.)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remember where the DOM parser stores the next value

The DOM SAX parser looked up the innermost open container for every value:
the end of its frame stack, then the object or array pointer in the frame.
Compilers have to reload all of that after each value is stored, since a
store of a value may alias the frame. The parser now keeps the innermost
container's kind and, for an array that stores directly, the array itself
in members, updated when a container opens or closes.

Flat arrays of numbers are the case that mattered: parsing floats.json
(1 000 000 doubles) into a DOM took 2 % more cycles than before the
previous commit on macOS and now takes 0.4 % fewer than the code before it
(signed_ints.json -0.8 %). Linux Ir is unchanged (canada -6.1 % GCC /
-6.9 % Clang, citm -0.3 % / -0.8 %, twitter +0.2 % / 0.0 %).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Look up the keys of big ordered_json objects in a hash table

ordered_map keeps its elements in a vector and searches it linearly for
every key it adds, so parsing an object with n keys into ordered_json took
O(n^2) key comparisons: 33 billion instructions for one object with 30 000
keys, where json needs 245 million. From 64 keys on, the DOM parser now
keeps a hash table of the keys of the open object (hashes and positions
only; the elements stay in the vector) and finds duplicate keys there. The
result is the same, including the position of a key that occurs twice.
Smaller objects, std::map based objects and key types without std::hash
are untouched.

ordered_json parse, macOS instructions: object with 1 000 keys 64.8 M ->
14.6 M, 10 000 keys 5 104 M -> 38.7 M, 30 000 keys 33 358 M -> 89 M.
Linux Ir of an object with 2 000 keys: 43.0 M -> 1.9 M (GCC), 37.9 M ->
1.8 M (Clang).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Trim the work the DOM parser does per container and per member

Opening and closing an array or object cost the DOM parser more than before
the exact-size array change, which showed on documents of many small
objects (jeopardy.json: +1.4 % Ir with GCC). This takes it back:

- the stack of open containers keeps its first eight entries in the parser
  object instead of in a vector, so it needs no capacity check beyond the
  push itself and no allocation for typical documents;
- a container is created in the place it ends up in, instead of as a
  temporary that is then moved there;
- a new object member, which is null, is constructed over the null instead
  of being swapped with it (only with the default base class of basic_json);
- the key index that big ordered_json objects use is only a part of the
  stack entries of objects that can have one;
- an array that outgrows the stack reserves twice what it has so far, as a
  vector that grew by itself would have.

DOM parse Ir of the series up to here against develop, Linux GCC / Clang:
canada -6.6 % / -7.3 %, citm +1.2 % / +0.2 %, jeopardy +0.4 % / -0.1 %,
124-byte object -6.5 % / -6.2 %. (twitter measures +6.8 % / +6.6 % at this
point in steady state, almost all of it in glibc's malloc/consolidate for the
new allocation sizes; with the last commit of the series it is -0.1 % / 0.0 %.)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test parsing containers of the sizes where the DOM parser changes storage

The DOM parser collects the elements of an array on a stack (up to 16) and
the members of an ordered_json object (up to 32) before it moves them into
the container, and uses a hash table for keys from 64 members on. Parse
arrays and objects of the sizes around these limits, nested into each other,
with repeated keys, and as random trees that are built next to their text,
and compare json and ordered_json against the value built through the API.

The tests pass on the code from before these changes as well. The random
trees are what finds the mistakes that are easy to make here: while the
collecting change was being written, a collected array that outgrew the
stack at the moment its parent had just reached the limit left its elements
in the parent, and only these tests (142 failed assertions) showed it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not convert floats that cannot overflow in accept()

accept() only has to find out whether a float token is finite: json_sax_acceptor
ignores the value, and the parser rejects only a float that overflows. A
float whose digits fit into 19 digits and whose decimal exponent is at most
max_exponent10 - 19 is below 10^max_exponent10 and cannot overflow, so the
lexer, told by accept() that the value is not needed, skips the conversion
for those. A token that may overflow, with more digits, or in another float
format is converted as before, so 1e309 and its neighbours are rejected as
they were (the new test compares accept() with parse() on tokens around the
limit).

json::accept(): canada.json Ir -8.6 % (GCC) / -10.9 % (Clang), 124-byte
object -3.2 %; cycles on the M1 (release benchmarks, LTO): canada -7.8 %,
floats.json -9.0 %. The check tests the accept() flag first, so the parser
that needs the value pays one load and branch: SaxParse canada +0.35 % Ir
(GCC and Clang), DOM canada +0.4 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Collect the members of ordered_json objects on a stack until they are known

An ordered_json object keeps its members in a vector and appends every new
member to it, so reading an object of unknown size reallocated the vector at
1, 2, 4, 8, ... members, and ordered_map copies all keys on every
reallocation. The DOM SAX parser now collects the members of such objects on
one stack shared by all open objects, as it does the elements of arrays, and
moves them into a vector of exactly the right size when the closing brace
arrives. An object that gets past 32 members switches to appending to its
own vector (and, from 64, to the key index of the earlier commit). The
members are the same, in the same order, and a key that occurs twice keeps
the place of its first occurrence. std::map based objects, objects whose
size the input announces (binary formats), and builds with JSON_DIAGNOSTICS
or JSON_DIAGNOSTIC_POSITIONS are untouched.

What only ordered objects need (the collected members, the key indexes) is
a member of the parser only for them, and the code that stores a value tries
the array that takes its elements directly first. An array that outgrows
the stack now reserves 32 elements, the capacity a vector that grew by
itself has at that point; with the larger reserve before, parsing
floats.json (1 000 000 numbers) was 2 % slower on Apple M1 than before this
series.

ordered_json parse, Linux Ir against develop (GCC / Clang): citm -11.6 % /
-11.7 %, twitter -8.4 % / -8.4 %, jeopardy (12 000 objects of 7 keys)
-10.5 % / -8.0 %, canada -3.9 % / -5.2 %, 124-byte object -20.9 % / -16.5 %.
macOS time (M1, earlier base): citm -19 %, twitter -13 %, jeopardy -13 %,
canada -14 %, 124-byte object -26 %. json parse: steady-state twitter -0.1 %
(see the previous commits), citm -1.2 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename the key index's match callback so Flawfinder doesn't read it as std::equal

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Bind the parser test's strings by reference in the range-for loop

GCC and Clang 14 (-Wrange-loop-construct) reported the copy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Collect container values only in basic_json's own parse functions

A json_sax_dom_parser that a user constructs (as in the sax_no_exception
example) must hold the values read before a parse error, as on develop:
its value is used after the error. The values of containers of unknown
size are now collected on a stack only with the new template parameter
Collect (default false), which json::parse() and the json_view
materialization set; without it, every value is stored in place as it
arrives. A test checks the values kept after an error for json and
ordered_json.

Also fixes the clang-tidy, cpplint and Infer findings: the frame stack
keeps its first frames in a std::array (left uninitialized) and the
others in a std::unique_ptr, auto for create<>(), NOLINT for member
initializers -Weffc++ needs, and an @infer-ignore for the nullptr that
place_container() does not use.

json::parse() and ordered_json::parse() instructions are unchanged
(Apple clang; twitter, citm, canada, jeopardy: -0.03 % to +0.25 %).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Clarify the DOM parser's collection stacks and rename reenter_enclosing_container()

Address review comments: explain why a closed container erases only the top
of the shared stack, set the payload before the type, rename the helper that
makes the enclosing container current again, and declare the parser's state
stack right before reserving it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Export json_sax_dom_parser from the C++20 module for MSVC

MSVC needs the detail classes that basic_json befriends to be visible in
the module (#3970, as json_sax_dom_callback_parser); with its new
template parameter, json_sax_dom_parser failed in ci_module_cpp20
(C2039, not a member of nlohmann::detail).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the overflow-limit numbers of the parser test in a std::vector

clang-tidy (cppcoreguidelines-avoid-c-arrays) rejected the C array.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the clang-tidy findings of the ordered_json parser tests

The container sizes are a std::array, the expected text uses raw string
literals, the generator's arithmetic is in parentheses, auto for the
result of make<>(), and single characters are appended as characters.
clang-tidy now reports nothing on the lines the branch adds to
unit-ordered_json.cpp and unit-class_parser.cpp (apart from checks CI's
clang-tidy does not have).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 07:41:21 +02:00
Niels Lohmann dd7686a269 json_view: faster reading, indexing and C-stream parsing (#5888)
* Read C streams into a json_document in blocks

json_document::parse(FILE*) (and read(), accept(), parse_copy()) went through the
library's file input adapter and appended the text character by character
with fgetc. Read it with fread in blocks instead, like the std::istream
overload already does.

Parsing citm_catalog.json (1.7 MB) from a FILE*:
  Apple M1: 80.6 M -> 8.4 M instructions, 4.8 ms -> 0.55 ms (parse of the same text from a std::string: 8.7 M)
  Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 47.0 M -> 13.7 M / 45.8 M -> 14.2 M
The parsed documents and the error cases are identical (checked over the test corpus
and 30 mutants of each file). A null FILE* is handled as before (by the library's adapter).

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Index arrays without subtrees directly

operator[](idx), at(idx) and back() skipped the elements of an array one
after the other, whatever they were. In an array none of whose elements is
an array or object (numbers, strings, booleans, null) the elements are
adjacent nodes, which the array node's own sizes tell: its subtree has
exactly one node more than it has elements. Such an element is now the
node idx after the array node; any other array is walked as before.

Reading r[i] for every i of an array of 3000 integers:
  Apple M1: 28.8 M instructions, 3.1 ms -> 0.1 M instructions, 4 us
  Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 84.1 M -> 3.1 M / 75.0 M -> 3.0 M
Arrays that contain arrays are unchanged.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Skip the bounds checks of number and string tokens in NUL-terminated input

For a NUL-terminated input (std::string, C string, string literal), the
parser reads the terminator instead of comparing p with the end, via
cursor::cur(). Number and string scanning still tested `p != e` before each
look at the next byte (after the integer digits for '.', for 'e', after
the exponent sign and digits, and after a string for its closing quote),
although the terminator can never be one of those characters. They now use
cur(), which is the plain load for NUL-terminated input and the bounds check
for ptr+len input, as before.

Parsing, Apple M1 instructions per parse: twitter 3.126 M -> 3.089 M (-1.2 %),
citm_catalog 8.70 M -> 8.58 M (-1.3 %), jeopardy 306.1 M -> 302.5 M (-1.2 %), signed
integers 281 M -> 266 M (-5.4 %), canada and floats -0.1 % / -0.9 %.
Linux x86-64 Ir, GCC 13 / Clang 18: twitter -0.5 % / -0.0 %, signed integers +0.2 % / -0.7 %.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the index text of items() on arrays lazy

basic_json_view::items() on an array formatted every element's index with
std::to_string when the item was made, whether or not the loop asks for the
key. The item now keeps the position and makes the text on the first key()
call. Objects are not affected.

Iterating over the items of an array of 3000 numbers and reading only the values:
  Apple M1: 25.5 us -> 4.6 us (0.45 M -> ~0 net instructions)
  Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 14.9 M -> 4.1 M / 15.3 M -> 4.6 M
Reading key() costs what it did (24.5 us).

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the default of value() with a C string only if the member is missing

basic_json_view::value(key, "text") and value(pointer, "text") built a
std::string from the default first and passed it on, so a call allocated
for the default even when the member exists, and copied it again when it
does not. They now look the member up and make the result directly: from the
member if there is one, else from the C string. Behavior, including the
exceptions, is the same.

Two calls (member found, member missing) with a 40 character default, Apple M1:
  1.6 K -> 0.7 K instructions
  Linux x86-64 Ir (500 rounds), GCC 13 / Clang 18: 2.46 M -> 2.22 M / 2.40 M -> 2.15 M

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Materialize scalars directly and keep the stack of small subtrees local

basic_json_view::materialize() (and get<BasicJsonType>()) always set up the
SAX handler that parse() uses and a std::vector for the open containers,
sized from the document's deepest nesting. For a scalar, which is most of what
is materialized one value at a time, neither is needed: the value is made
directly (the same constructors the handler ends up calling). For arrays and
objects the stack of open containers is now a local array for the first 16
levels, and the vector is only used for deeper documents.

Materializing five scalars of a small document, Apple M1: 3.6 K -> 0.9 K instructions
  Linux x86-64 Ir (500 rounds), GCC 13 / Clang 18: 2.96 M -> 2.37 M / 2.90 M -> 2.29 M

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Copy the text between escapes while scanning it

A string that contains an escape is decoded by builder::slow_string(). After
each escape it scanned the next run of plain bytes (8 or 16 byte-by-byte
steps, then a vector loop), and then copied that run to the decode arena in a
second pass. The run is now copied 16 bytes at a time while it is scanned
(one vector load, a store of the same 16 bytes, and the mask of the bytes
that end the run), and the write position lives in a local variable, not in
the builder, since a store of a character may alias the builder's members and
made the compiler reload it after every escape. A run that ends at a quote
needs no scan, and non-ASCII text goes through the UTF-8 check as before;
the portable code is unchanged.

Parsing, Apple M1 instructions: jeopardy.json 304.0 M -> 293.8 M (-3.4 %), 3000 strings with
escapes 1.72 M -> 1.54 M (-10 %), twitter -0.6 %.
Linux x86-64 Ir, GCC 13 / Clang 18: escaped strings -7.6 % / -11.4 %, twitter 0.0 % / 0.0 %.
Documents and error messages are identical on 900 k random strings of escapes, UTF-8 and control
characters (also under ASan), and on mutants of the test corpus.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Hash the last bytes of a key of a large object without a loop

key_hash() mixed a key eight bytes at a time and then packed the remaining
0-7 bytes one by one, in a loop. The last word of a key of eight or more
bytes is now read as the last eight bytes of the key (overlapping the word
before), and a key of 2-7 bytes as two overlapping words of four or two
bytes. A product only moves bits upwards, and the slot is taken from the low
bits: the high bits of the last word are folded down before the
multiplication, since keys that differ only in their last characters ("user_17",
"user_18") would otherwise share slots. The hash is not stored, so its values
may change.

Looking up every key of an object of 1000 random keys, Apple M1 instructions per 1000 lookups:
  keys of 5 bytes 175 K -> 93 K, hex strings of 8 bytes 177 K -> 168 K, keys of 12 bytes 196 K -> 186 K
Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 5-byte keys -4.7 % / -8.8 %, 12-byte keys -3.3 % / -7.4 %.
Building the tables while parsing such objects is cheaper as well (5-byte keys: 13.3 us -> 10.1 us).

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare short keys of a large object without a call of memcmp

find_indexed() compared the key of a member the table points to with
memcmp, a library call for any length. Up to 16 bytes, the bytes are now
compared as two overlapping words (the first and the last 8, 4, or 2 bytes),
as find_member() already does for the objects without a table; longer keys
still use memcmp.

Looking up every key of an object of 1000 random keys, Apple M1 instructions per 1000 lookups
(with the previous commit): keys of 5 bytes 93 K -> 72 K, hex strings of 8 bytes 168 K -> 148 K,
keys of 12 bytes 186 K -> 170 K, keys of 24 bytes 198 K -> 193 K (unchanged, memcmp).
Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 5-byte keys -7.2 % / -7.3 %, 12-byte keys -5.9 % / -6.6 %.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Read an unedited editable document like a read-only one

Views of a json_editable_document followed edits at every step: the
lookup, the element access, back(), dump() and materialize() were built
for nodes that can be moved or linked, though no edit changes the index until
the first one (every edit creates the edit state before it touches a node).
Until then these now take the read-only code: one check of the document's
edit state per call, not per member or per node. After the first edit
nothing changes. Views that were made earlier keep working, as the check is
made when they are used.

Unedited editable document, Apple M1: dump() of twitter.json 1.40 M -> 1.22 M instructions (-12.6 %);
lookups unchanged. Linux x86-64 Ir, GCC 13 / Clang 18: dump -1.7 % / -1.0 %, lookups -0.1 % / 0.0 %.
Walking with iterators is unchanged, and so is dump() after an edit.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Format the numbered test keys of json_view's index with a stream

The keys were made with snprintf() and a format from a list, which
Flawfinder reports as an error (CWE-134, a format string that is not a
literal). A stream now formats them; the 30000 keys of the test are the
same as before.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Put the key number of the json_view index test in parentheses

astyle formatted i * step after << as a pointer declaration (i* step),
which failed the amalgamation check.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the compiler and lint findings of json_view's read side and tests

- find() and contains() dispatch on Editable for the member lookup; for
  documents that cannot be edited both branches were the same call
  (GCC -Wduplicated-branches)
- json_view.hpp includes <cstdio> for the FILE overload (cpplint)
- the arena check macro of the builder is on one line (cpplint read it
  as an if body without braces)
- tests: no nested conditional (also MSVC C4018), no useless cast,
  parentheses around mixed operators, raw string literals, += instead
  of temporary concatenations, and NOLINT for the fixed seeds and a
  find() != npos, as elsewhere in the tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Grow the decode arena through a function and store its position back on every return

slow_string() kept its write position in locals and grew the arena through
the macro NLOHMANN_VIEW_ENSURE. The macro is now the always-inline member
arena_ensure() (a lambda is not inlined at every call site by clang, which
then keeps the position in memory: +11 % instructions on escape-heavy
strings), and a guard stores the position back into m_arena_write on every
return instead of by hand on the successful one. Instruction counts are
unchanged (clang) or 1 % lower (GCC).

Also word the complexity of back() as that of at().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 07:41:06 +02:00
Niels Lohmann e9c4fe3cdf json_view: indented and ensure_ascii dumps through the fast writer (#5889)
* Write the indented output of json_view's dump() with the fast writer

The compact writer of dump() (cursor in a register, fixed-size copies of
tokens) served dump(-1) only; every dump(indent), and everything with
ensure_ascii that was indented, went through a generic walk that calls put()
for each token and put_repeated() (a memset call) for each line break. The
fast writer now writes indented output, too, with or without ensure_ascii, and
the generic walk is gone. A line break is a copy of a 64-byte ruler (a line
break and 63 indentation characters) when the line is at most 64 bytes, and a
store plus memset beyond that. The depth is counted only in the indented
instantiations (counting it in the compact one cost an editable document 5 %
with Clang). The indented walks are out of line (dump_pretty()): inlined
into dump() next to the compact ones, they made compact dumps of integer
arrays 7 % slower.

A decoded string (one with escape sequences) of a damaged image loaded with
image_check::bounds can contain invalid UTF-8. As before, the compact dump
writes such bytes unchanged, and the indented dump throws type_error.316; its
runs are found by the lexer's validating scan (a scalar validation of every
sequence made dump(2) of decoded CJK text twice as slow as before).

dump(2) instructions per call, macOS arm64 (before -> after):

               twitter         citm            canada          jeopardy(4k)
  json         3.75M -> 1.85M  8.28M -> 3.93M  67.1M -> 56.0M  9.56M -> 4.81M
  editable     3.82M -> 1.84M  8.48M -> 4.02M  68.1M -> 56.2M  9.66M -> 4.75M

Linux instructions (GCC 13 / Clang 18), dump(2) and dump(2, ' ', true):

  twitter      -25 % / -30 %      -11 % / -11 %
  citm         -29 % / -28 %      -24 % / -24 %
  jeopardy(4k) -25 % / -32 %      -20 % / -24 %
  CJK text with \u escapes  -6 % / -11 %   -2 % / -2 %

The compact dump is unchanged (within +0.4 % on Linux, within noise on the Mac).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Scan the strings of json_view's dump() with ensure_ascii 16 bytes at a time

With ensure_ascii, a string of the source that has a byte >= 0x7F was written
by a generic path: put() per piece and a validate-and-decode step for every
character beyond ASCII. It now goes through a writer with the cursor in a
local variable. The part of the string that has nothing to escape is found
16 bytes at a time (SSE2 or NEON; the existing 8-byte scan without vector
support) and copied; strings that are ASCII throughout are still decided by
the 8-byte check of dump() with ensure_ascii (a 16-byte scan for them was
2-5 % slower for short strings). The two-byte sequences (lead C2..DF) and
the three-byte sequences whose continuation bytes may have any value (lead
E1..EF except ED) are decoded with a few compares; E0, ED, four-byte
sequences, and damaged input go through validate_one_utf8() as before, so
damaged images still throw type_error.316.

dump(-1, ' ', true) instructions per call, macOS arm64 (develop -> now);
the compact dump without ensure_ascii and dumps of ASCII text are unchanged:

  twitter 5.42M -> 3.57M    citm 3.28M -> 3.24M     jeopardy(4k) 5.78M -> 5.65M
  CJK text, raw 66.5M -> 35.3M    escaped \u 32.7M -> 17.0M
  Latin text with accents 11.5M -> 9.4M    ASCII text 4.40M -> 4.40M

Linux instructions (GCC 13 / Clang 18), dump(-1, ' ', true):

  twitter -9 % / -33 %    CJK text -14 % / -47 %    Latin text -8 % / -18 %
  citm -0.2 % / -0.4 %    ASCII text -0.1 % / -0.3 %

The compact dump without ensure_ascii is unchanged (canada 0.0 %).

The unit test covers every character of the Basic Multilingual Plane, and a
character that needs an escape at each position of strings of 0..40 bytes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Find the end of the plain run of a decoded string 16 bytes at a time with SSE2

The strings of a document that had escape sequences are written in runs
without a quote, backslash, or control character; the compact dump found the
end of a run 8 bytes at a time with a SWAR test. With SSE2 (x86-64) it now
compares 16 bytes per step. NEON keeps the SWAR code: there the mask of a
step costs more than it saves on the short runs that are typical here.

Instructions of dump(), Linux (GCC 13 / Clang 18):

  twitter.json                  1.99M -> 1.88M (-5.2 %) / 1.97M -> 1.92M (-3.0 %)
  CJK text with \u escapes      5.98M -> 4.52M (-24.5 %) / 5.30M -> 4.25M (-19.9 %)
  ASCII text with a few escapes 4.28M -> 4.09M (-4.4 %) / 4.24M -> 4.11M (-3.2 %)
  jeopardy (4,000 questions)    5.08M -> 4.78M (-5.8 %) / 4.97M -> 4.80M (-3.4 %)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename depth_counter's open() and close() to enter() and leave()

Flawfinder took them for the file functions open() and close().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the clang-tidy findings in json_view's dump and its tests

- the SSE2 loop of the string scan is within NOLINTBEGIN/END
  (portability-simd-intrinsics), as in to_chars.hpp
- depth_counter<false>'s leave() and get() keep the interface of
  depth_counter<true> instead of being static
- write_ascii() compares the room left as a size_t variable
- the test's texts are a std::array, and a JSON fragment with escapes
  is a raw string literal

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move dump_ascii() and dump_pretty() next to dump()

Address a review comment: the two dispatchers that dump() calls now follow
it directly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 07:38:17 +02:00
dependabot[bot] 1e9222f955 Bump dompurify from 3.4.16 to 3.4.17 in /docs/mkdocs/scripts/mermaid (#5904)
Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.16 to 3.4.17.
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](https://github.com/cure53/DOMPurify/compare/3.4.16...3.4.17)

---
updated-dependencies:
- dependency-name: dompurify
  dependency-version: 3.4.17
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-24 07:37:42 +02:00
Niels Lohmann ffcd392be0 Fix Semgrep findings in Python tooling (#5899)
- generate_docset.py: use SHA-256 instead of SHA-1 for the cache file
  name digest of downloaded images.
- generate_docset.py: suppress dynamic-urllib-use-detected; the URL
  scheme is restricted to http(s) right before the call.
- generate_natvis.py: suppress direct-use-of-jinja2; the template
  renders a local XML file with autoescape enabled.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-23 07:11:17 +02:00
89 changed files with 6560 additions and 2185 deletions

No files matched your search

+1 -8
View File
@@ -29,13 +29,6 @@ updates:
- dependency-name: mermaid
update-types: ["version-update:semver-major"]
- package-ecosystem: pip
directory: /.github/requirements
schedule:
interval: daily
cooldown:
default-days: 7
- package-ecosystem: pip
directory: /tools/astyle
schedule:
@@ -43,7 +36,7 @@ updates:
cooldown:
default-days: 7
ignore:
# astyle is deliberately pinned (see tools/astyle/requirements.in): newer versions
# astyle is deliberately pinned (see tools/astyle/requirements.txt): newer versions
# reformat the code, and the pinned version defines the formatting CI enforces
- dependency-name: astyle
-1
View File
@@ -1 +0,0 @@
meson==1.12.1
-10
View File
@@ -1,10 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.14
# by the following command:
#
# pip-compile --generate-hashes --no-emit-index-url --output-file=.github/requirements/meson.txt --strip-extras .github/requirements/meson.in
#
meson==1.12.1 \
--hash=sha256:930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02 \
--hash=sha256:ab0a6ca09f8ef70c564c8241fb5a23957886a0b53fb58412b5e07eaf07dba743
# via -r .github/requirements/meson.in
-1
View File
@@ -1 +0,0 @@
semgrep==1.182.1
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -67,10 +67,10 @@ jobs:
- name: Install astyle
run: |
python3 -mvenv venv
venv/bin/pip3 install --require-hashes -r $MAIN_DIR/tools/astyle/requirements.txt
venv/bin/pip3 install -r $MAIN_DIR/tools/astyle/requirements.txt
- name: Install generate_natvis dependencies
run: pip3 install --require-hashes -r $MAIN_DIR/tools/generate_natvis/requirements.txt
run: pip3 install -r $NATVIS_TOOL_DIR/requirements.txt
- name: Regenerate the tools/macro_builder tables in macro_scope.hpp
run: |
-9
View File
@@ -31,12 +31,3 @@ jobs:
persist-credentials: false
- name: 'Dependency Review'
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
with:
# Semgrep pins mcp==1.29.0 and opentelemetry-api~=1.37.0 exactly
# (.github/requirements/semgrep.txt), so these cannot be upgraded
# until Semgrep lifts its pins. They do not apply to our use: CI
# only runs `semgrep scan` on the checkout and never starts the MCP
# server (GHSA-84m7-p3x7-pcfv, StreamableHTTP sessions) or parses
# incoming trace context (GHSA-xcq7-p3pf-p88h, tracestate parsing).
# Remove the entries once a Semgrep release allows the fixed versions.
allow-ghsas: GHSA-84m7-p3x7-pcfv, GHSA-xcq7-p3pf-p88h
+1 -1
View File
@@ -53,7 +53,7 @@ jobs:
# SEMGREP_APP_TOKEN is still passed through so registry auth works if a
# token is ever added.
- name: Install Semgrep
run: python3 -m pip install --user --require-hashes -r .github/requirements/semgrep.txt
run: python3 -m pip install --user semgrep==1.168.0
# `semgrep scan --sarif` always exits 0 even with findings; continue-on-error
# is a safety net so the SARIF upload still runs if the scan itself errors.
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
- name: Check that Meson and pkg-config offer the CMake options
run: make check_build_options
- name: Install Meson
run: pip install --require-hashes -r .github/requirements/meson.txt
run: pip install meson
- name: Install with Meson
run: |
meson setup build-meson --prefix=${{ github.workspace }}/install
+2 -1
View File
@@ -341,13 +341,14 @@ def download(url, docs) -> str:
raise ValueError(f'not an http(s) URL: {url}')
req = urllib.request.Request(u.geturl(), headers={'User-Agent': USER_AGENT})
# (the scheme is checked above)
# nosemgrep: python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected
with urllib.request.urlopen(req, timeout=20) as r: # nosec B310
data = r.read()
ctype = r.headers.get_content_type()
path = urllib.parse.unquote(u.path).lstrip('/')
ext = os.path.splitext(path)[1]
if u.query or not ext or path.endswith('/'):
digest = hashlib.sha1(url.encode(), usedforsecurity=False).hexdigest()[:12]
digest = hashlib.sha256(url.encode()).hexdigest()[:12]
path = os.path.join(os.path.dirname(path), digest + CONTENT_TYPE_EXT.get(ctype, ext or '.bin'))
rel = os.path.normpath(os.path.join('assets', 'external', u.hostname, path))
out = os.path.join(docs, rel)
+2 -1
View File
@@ -80,7 +80,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
index (unlike `BasicJsonType`'s array, which is random-access). Constant if no element of the array is an array
or object, since the elements are then adjacent in the index.
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level (as 1.) or the index into the array (as 2.).
+2 -1
View File
@@ -28,7 +28,8 @@ This exception does not carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.
Linear in the size of the array or object: unlike [`front()`](front.md), which only ever looks at the first
element, `back()` has to walk every element to find where they end, since elements are not a fixed size in the
index.
index. Constant if no element of the array is an array or object, since the elements are then adjacent
in the index.
## Notes
@@ -86,7 +86,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
index (unlike `BasicJsonType`'s array, which is random-access). Constant if no element of the array is an array
or object, since the elements are then adjacent in the index.
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level (as 1.) or the index into the array (as 2.).
@@ -0,0 +1,42 @@
#include <iostream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
json j = json::parse(R"({"name": "Mary", "age": 42})");
// look up the key, then test the type: neither step throws
for (const char* key : {"name", "age", "city"})
{
const auto it = j.find(key);
if (it == j.end())
{
std::cout << key << ": missing\n";
}
else if (it->is_string())
{
std::cout << key << ": string \"" << it->get_ref<const std::string&>() << "\"\n";
}
else
{
std::cout << key << ": not a string, but " << it->type_name() << '\n';
}
}
// get_ptr returns a null pointer if the value has another type
const std::string* name = j["name"].get_ptr<const std::string*>();
const std::string* age = j["age"].get_ptr<const std::string*>();
std::cout << "get_ptr<string*>: name = " << (name != nullptr) << ", age = " << (age != nullptr) << '\n';
// value() is NOT exception-free: it throws if the key exists with a different type
try
{
std::cout << j.value("name", 0) << '\n';
}
catch (const json::type_error& e)
{
std::cout << e.what() << '\n';
}
}
@@ -0,0 +1,5 @@
name: string "Mary"
age: not a string, but number
city: missing
get_ptr<string*>: name = 1, age = 0
[json.exception.type_error.302] type must be number, but is string
@@ -0,0 +1,46 @@
#include <iostream>
#include <optional>
#include <string>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
struct person
{
std::string name;
int age;
};
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(person, name, age)
// Compile this function with exceptions enabled and call it from code that does not use them.
std::optional<person> to_person(const json& j)
{
try
{
return j.get<person>();
}
catch (const json::exception&)
{
return std::nullopt;
}
}
int main()
{
const json good = {{"name", "Mary"}, {"age", 42}};
const json bad = {{"name", "Mary"}, {"age", "forty-two"}};
const json incomplete = {{"name", "Mary"}};
for (const json& j : {good, bad, incomplete})
{
if (const auto p = to_person(j))
{
std::cout << "person: " << p->name << ", " << p->age << '\n';
}
else
{
std::cout << "not a person: " << j << '\n';
}
}
}
@@ -0,0 +1,3 @@
person: Mary, 42
not a person: {"age":"forty-two","name":"Mary"}
not a person: {"name":"Mary"}
@@ -0,0 +1,28 @@
#include <cstdint>
#include <iostream>
#include <vector>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
// None of the calls below throws, so this example also works with -fno-exceptions.
int main()
{
const std::string invalid = R"({"name": "Mary", "age": )";
// parse() with allow_exceptions = false returns a discarded value
json j = json::parse(invalid, nullptr, false);
std::cout << std::boolalpha << "parse: discarded = " << j.is_discarded() << '\n';
// accept() only tells whether the input is valid JSON
std::cout << "accept: " << json::accept(invalid) << '\n';
// the binary formats have the same parameter
const std::vector<std::uint8_t> truncated_cbor = {0x83, 0x01};
json c = json::from_cbor(truncated_cbor, true, false);
std::cout << "from_cbor: discarded = " << c.is_discarded() << '\n';
// valid input is parsed as usual
json valid = json::parse(R"({"name": "Mary"})", nullptr, false);
std::cout << "valid input: " << valid << '\n';
}
@@ -0,0 +1,4 @@
parse: discarded = true
accept: false
from_cbor: discarded = true
valid input: {"name":"Mary"}
@@ -0,0 +1,42 @@
#include <iostream>
#include <utility>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
// line and column (both starting at 1) of a byte index in a text
std::pair<std::size_t, std::size_t> line_column(const std::string& input, std::size_t byte)
{
std::size_t line = 1;
std::size_t column = 1;
for (std::size_t i = 0; i + 1 < byte && i < input.size(); ++i)
{
if (input[i] == '\n')
{
++line;
column = 1;
}
else
{
++column;
}
}
return {line, column};
}
int main()
{
const std::string input = "{\n \"a\": 1,\n \"b\": ]\n}";
try
{
const json j = json::parse(input);
std::cout << j << '\n';
}
catch (const json::parse_error& e)
{
const auto position = line_column(input, e.byte);
std::cout << e.what() << '\n';
std::cout << "byte " << e.byte << " = line " << position.first << ", column " << position.second << '\n';
}
}
@@ -0,0 +1,2 @@
[json.exception.parse_error.101] parse error at line 3, column 8: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal
byte 20 = line 3, column 8
@@ -0,0 +1,46 @@
#define JSON_DIAGNOSTIC_POSITIONS 1
#include <iostream>
#include <utility>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
// line and column (both starting at 1) of a byte index in a text
std::pair<std::size_t, std::size_t> line_column(const std::string& input, std::size_t byte)
{
std::size_t line = 1;
std::size_t column = 1;
for (std::size_t i = 0; i + 1 < byte && i < input.size(); ++i)
{
if (input[i] == '\n')
{
++line;
column = 1;
}
else
{
++column;
}
}
return {line, column};
}
int main()
{
const std::string input = "{\n \"name\": \"Mary\",\n \"age\": \"forty-two\"\n}";
const json j = json::parse(input); // the input is valid JSON ...
try
{
const int age = j["age"]; // ... but the value has the wrong type
std::cout << age << '\n';
}
catch (const json::exception& e)
{
std::cout << e.what() << '\n';
// start_pos() is a 0-based index into the input; the helper expects a 1-based byte index
const auto position = line_column(input, j["age"].start_pos() + 1);
std::cout << "the value starts at line " << position.first << ", column " << position.second << '\n';
}
}
@@ -0,0 +1,2 @@
[json.exception.type_error.302] (bytes 29-40) type must be number, but is string
the value starts at line 3, column 10
@@ -0,0 +1,46 @@
#include <iostream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
std::string buffer;
// call for every chunk; returns true and sets `value` once the buffer holds one complete JSON value
bool push(const std::string& chunk, json& value)
{
buffer += chunk;
try
{
value = json::parse(buffer);
buffer.clear();
return true;
}
catch (const json::parse_error& e)
{
if (e.byte == buffer.size() + 1)
{
return false; // the input ends prematurely: wait for more data
}
throw; // a real syntax error
}
}
int main()
{
json value;
for (const std::string chunk : {"{\"na", "me\": \"Ma", "ry\", \"l", "ist\": [1, 2", ", 3]}"})
{
std::cout << "chunk " << chunk << " -> " << (push(chunk, value) ? "complete" : "incomplete") << '\n';
}
std::cout << value << '\n';
// a syntax error in the middle is not mistaken for missing data
try
{
push("[1, ]", value);
}
catch (const json::parse_error& e)
{
std::cout << e.what() << '\n';
}
}
@@ -0,0 +1,7 @@
chunk {"na -> incomplete
chunk me": "Ma -> incomplete
chunk ry", "l -> incomplete
chunk ist": [1, 2 -> incomplete
chunk , 3]} -> complete
{"list":[1,2,3],"name":"Mary"}
[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal
@@ -0,0 +1,23 @@
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
// a CBOR value followed by other data
const std::vector<std::uint8_t> cbor = json::to_cbor(json{1, 2, 3});
std::string data(cbor.begin(), cbor.end());
data += "TRAILING";
std::istringstream input(data);
json j = json::from_cbor(input, false); // strict = false: stop after the first value
std::cout << "value: " << j << '\n';
std::cout << "bytes used: " << input.tellg() << '\n';
std::string rest;
input >> rest;
std::cout << "remaining input: " << rest << '\n';
}
@@ -0,0 +1,3 @@
value: [1,2,3]
bytes used: 4
remaining input: TRAILING
@@ -0,0 +1,18 @@
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
std::istringstream input(R"({"a":1} [2,3] 4 "s")");
// check for the end of the stream before each read, because a read at the end throws
json j;
while (input >> std::ws && input.peek() != std::char_traits<char>::eof())
{
input >> j;
std::cout << j << '\n';
}
}
@@ -0,0 +1,4 @@
{"a":1}
[2,3]
4
"s"
@@ -0,0 +1,27 @@
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
void handle(const json& record)
{
std::cout << "handling " << record << '\n';
}
int main()
{
std::istringstream file(R"([{"id": 1}, {"id": 2}, {"id": 3}])");
json result = json::parse(file, [](int depth, json::parse_event_t event, json & record)
{
if (depth == 1 && event == json::parse_event_t::object_end)
{
handle(record);
return false; // discard the record
}
return true;
});
std::cout << "result: " << result << '\n';
}
@@ -0,0 +1,4 @@
handling {"id":1}
handling {"id":2}
handling {"id":3}
result: []
@@ -0,0 +1,81 @@
#include <iostream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
struct find_id : nlohmann::json_sax<json>
{
bool next_is_id = false;
json::number_unsigned_t id = 0;
bool key(std::string& name) override
{
next_is_id = (name == "id");
return true;
}
bool number_unsigned(json::number_unsigned_t value) override
{
if (next_is_id)
{
id = value;
return false; // found: stop parsing
}
return true;
}
bool null() override
{
return true;
}
bool boolean(bool /*val*/) override
{
return true;
}
bool number_integer(json::number_integer_t /*val*/) override
{
return true;
}
bool number_float(json::number_float_t /*val*/, const std::string& /*s*/) override
{
return true;
}
bool string(std::string& /*val*/) override
{
return true;
}
bool binary(json::binary_t& /*val*/) override
{
return true;
}
bool start_object(std::size_t /*elements*/) override
{
return true;
}
bool end_object() override
{
return true;
}
bool start_array(std::size_t /*elements*/) override
{
return true;
}
bool end_array() override
{
return true;
}
bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const json::exception& /*ex*/) override
{
return false;
}
};
int main()
{
find_id handler;
const bool completed = json::sax_parse(R"({"x": [1, 2, 3], "id": 42, "rest": [4, 5, 6]})", &handler);
std::cout << std::boolalpha;
std::cout << "parse completed: " << completed << '\n';
std::cout << "id: " << handler.id << '\n';
}
@@ -0,0 +1,2 @@
parse completed: false
id: 42
@@ -0,0 +1,56 @@
#include <chrono>
#include <iostream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
namespace nlohmann
{
template <typename Rep, typename Period>
struct adl_serializer<std::chrono::duration<Rep, Period>>
{
static void to_json(json& j, const std::chrono::duration<Rep, Period>& d)
{
j = d.count();
}
static void from_json(const json& j, std::chrono::duration<Rep, Period>& d)
{
d = std::chrono::duration<Rep, Period>(j.get<Rep>());
}
};
template <typename Duration>
struct adl_serializer<std::chrono::time_point<std::chrono::system_clock, Duration>>
{
static void to_json(json& j, const std::chrono::time_point<std::chrono::system_clock, Duration>& t)
{
j = t.time_since_epoch();
}
static void from_json(const json& j, std::chrono::time_point<std::chrono::system_clock, Duration>& t)
{
t = std::chrono::time_point<std::chrono::system_clock, Duration>(j.get<Duration>());
}
};
} // namespace nlohmann
int main()
{
using namespace std::chrono;
std::cout << std::boolalpha;
// the unit is not part of the JSON value
const json timeout = milliseconds(1500);
std::cout << "duration: " << timeout << '\n';
std::cout << "as milliseconds: " << timeout.get<milliseconds>().count() << '\n';
// a system_clock time point is the time since the Unix epoch
const time_point<system_clock, milliseconds> t(milliseconds(1700000000123));
const json stamp = t;
std::cout << "time point: " << stamp << '\n';
const auto back = stamp.get<time_point<system_clock, milliseconds>>();
std::cout << "round trip: " << (back == t) << '\n';
}
@@ -0,0 +1,4 @@
duration: 1500
as milliseconds: 1500
time point: 1700000000123
round trip: true
@@ -0,0 +1,99 @@
#include <iostream>
#include <memory>
#include <stdexcept>
#include <vector>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
struct shape
{
virtual ~shape() = default;
virtual json serialize() const = 0;
};
struct circle : shape
{
double radius;
explicit circle(double r) : radius(r) {}
json serialize() const override
{
return {{"type", "circle"}, {"radius", radius}};
}
};
struct rectangle : shape
{
double width;
double height;
rectangle(double w, double h) : width(w), height(h) {}
json serialize() const override
{
return {{"type", "rectangle"}, {"width", width}, {"height", height}};
}
};
namespace nlohmann
{
template <>
struct adl_serializer<std::unique_ptr<shape>>
{
static void to_json(json& j, const std::unique_ptr<shape>& p)
{
j = p ? p->serialize() : json(nullptr);
}
static void from_json(const json& j, std::unique_ptr<shape>& p)
{
if (j.is_null())
{
p.reset();
return;
}
const auto& type = j.at("type").get_ref<const std::string&>();
if (type == "circle")
{
p = std::make_unique<circle>(j.at("radius").get<double>());
}
else if (type == "rectangle")
{
p = std::make_unique<rectangle>(j.at("width").get<double>(), j.at("height").get<double>());
}
else
{
throw std::invalid_argument("unknown shape type: " + type);
}
}
};
} // namespace nlohmann
int main()
{
std::vector<std::unique_ptr<shape>> shapes;
shapes.push_back(std::make_unique<circle>(1.5));
shapes.push_back(std::make_unique<rectangle>(2, 3));
const json j = shapes;
std::cout << j << '\n';
// reading creates objects of the derived types
const auto back = j.get<std::vector<std::unique_ptr<shape>>>();
std::cout << std::boolalpha;
std::cout << "first is a circle: " << (dynamic_cast<circle*>(back[0].get()) != nullptr) << '\n';
std::cout << "second is a rectangle: " << (dynamic_cast<rectangle*>(back[1].get()) != nullptr) << '\n';
// an unknown type tag is reported
try
{
json{{"type", "star"}}.get<std::unique_ptr<shape>>();
}
catch (const std::invalid_argument& e)
{
std::cout << e.what() << '\n';
}
}
@@ -0,0 +1,4 @@
[{"radius":1.5,"type":"circle"},{"height":3.0,"type":"rectangle","width":2.0}]
first is a circle: true
second is a rectangle: true
unknown shape type: star
@@ -0,0 +1,90 @@
#include <iostream>
#include <memory>
#include <vector>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
namespace nlohmann
{
template <typename T>
struct adl_serializer<std::shared_ptr<T>>
{
static void to_json(json& j, const std::shared_ptr<T>& p)
{
if (p)
{
j = *p;
}
else
{
j = nullptr;
}
}
static void from_json(const json& j, std::shared_ptr<T>& p)
{
if (j.is_null())
{
p = nullptr;
}
else
{
p = std::make_shared<T>(j.get<typename std::remove_const<T>::type>());
}
}
};
template <typename T>
struct adl_serializer<std::unique_ptr<T>>
{
static void to_json(json& j, const std::unique_ptr<T>& p)
{
if (p)
{
j = *p;
}
else
{
j = nullptr;
}
}
static void from_json(const json& j, std::unique_ptr<T>& p)
{
if (j.is_null())
{
p = nullptr;
}
else
{
p = std::make_unique<T>(j.get<T>());
}
}
};
} // namespace nlohmann
struct point
{
int x;
int y;
};
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(point, x, y)
int main()
{
// an empty pointer is written as null
const std::vector<std::shared_ptr<point>> points = {std::make_shared<point>(point{1, 2}), nullptr};
const json j = points;
std::cout << j << '\n';
// reading allocates new objects
const auto back = j.get<std::vector<std::shared_ptr<point>>>();
std::cout << back[0]->x << ", " << back[0]->y << ", " << (back[1] == nullptr) << '\n';
// pointers to const and unique_ptr work as well
const auto const_ptr = j[0].get<std::shared_ptr<const point>>();
const auto unique = j[0].get<std::unique_ptr<point>>();
std::cout << const_ptr->y << ", " << unique->x << '\n';
}
@@ -0,0 +1,3 @@
[{"x":1,"y":2},null]
1, 2, 1
2, 1
@@ -0,0 +1,63 @@
#include <iostream>
#include <stdexcept>
#include <variant>
#include <vector>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
namespace geo
{
struct circ
{
double radius;
};
struct rect
{
double width;
double height;
};
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(circ, radius)
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(rect, width, height)
using figure = std::variant<circ, rect>;
void to_json(json& j, const figure& f)
{
std::visit([&j](const auto & alternative)
{
j = alternative;
}, f);
j["type"] = std::holds_alternative<circ>(f) ? "circ" : "rect";
}
void from_json(const json& j, figure& f)
{
const auto& type = j.at("type").get_ref<const std::string&>();
if (type == "circ")
{
f = j.get<circ>();
}
else if (type == "rect")
{
f = j.get<rect>();
}
else
{
throw std::invalid_argument("unknown figure type: " + type);
}
}
} // namespace geo
int main()
{
const std::vector<geo::figure> figures = {geo::circ{2}, geo::rect{3, 4}};
const json j = figures;
std::cout << j << '\n';
const auto back = j.get<std::vector<geo::figure>>();
std::cout << "alternatives read back: " << back[0].index() << ", " << back[1].index() << '\n';
}
@@ -0,0 +1,2 @@
[{"radius":2.0,"type":"circ"},{"height":4.0,"type":"rect","width":3.0}]
alternatives read back: 0, 1
@@ -409,6 +409,32 @@ namespace nlohmann {
}
```
## Why isn't my `to_json` or `from_json` found?
The library calls `to_json` and `from_json` without qualification, so the compiler finds them by
[argument-dependent lookup](https://en.cppreference.com/w/cpp/language/adl) in the namespace of the type, or in the
namespaces of its template arguments. The usual reasons for a function that is "not found" are:
- **The type is an alias of a standard type.** After `#!cpp using Hash = std::array<std::uint8_t, 6>;` the type
*is* `std::array<std::uint8_t, 6>`, which lives in namespace `std`. A `to_json(json&, const Hash&)` in your namespace
is never selected. Either give the type its own identity, for example `#!cpp struct Hash : std::array<std::uint8_t, 6> {};`
(the function is then found in `Hash`'s namespace), or specialize
[`adl_serializer<Hash>`](#how-do-i-convert-third-party-types).
- **The function is in the wrong namespace.** It must be in the namespace of the type (a global type needs a global
function), not in the namespace of the code that calls the conversion. Specializations of `adl_serializer` belong in
namespace `nlohmann`.
- **It is declared too late.** A conversion can only use what has been declared when it is instantiated. If
`#!cpp json j = std::vector<third_party::string>{...};` appears before the `adl_serializer` specialization, the
compiler reports an error such as "explicit specialization of `adl_serializer<...>` after instantiation". Put the
specialization in a small header that every file includes **before** it uses the type with `json`.
- **It is a template.** For `template <class T> struct box`, write `template <class T> void to_json(json&, const box<T>&)`,
not an overload for one `T`.
- **It worked by accident.** Some versions found `to_json` overloads in `nlohmann::detail` through argument-dependent
lookup of the base class of `basic_json`. Do not rely on this; it is not part of the interface.
For smart pointers, class hierarchies, `std::variant`, and `std::chrono`, see
[Recipes for common types](type_recipes.md).
## Why can't I convert to/from `std::any`?
`std::any` is intentionally excluded from `get<T>()`/generic conversion support, so `get<std::any>()` and
@@ -505,6 +531,7 @@ struct bad_serializer
## See also
- [Recipes for common types](type_recipes.md) - smart pointers, class hierarchies, `std::variant`, and `std::chrono`
- [Converting values](conversions.md) - the general overview of `get`/`get_to` and implicit conversions
- [Specializing enum conversion](enum_conversion.md) - map enums to JSON strings instead of integers
- [Supported macros](macros.md) - reference for `NLOHMANN_DEFINE_TYPE_*` and related macros
+3 -1
View File
@@ -7,7 +7,9 @@ detect bugs in client code.
## Switch off runtime assertions
Runtime assertions can be switched off by defining the preprocessor macro `NDEBUG` (see the
[documentation of assert](https://en.cppreference.com/w/cpp/error/assert)) which is the default for release builds.
[documentation of assert](https://en.cppreference.com/w/cpp/error/assert)) which is the default for release builds. Without `NDEBUG`, the assertions are compiled in and contain the name of the source
file (`__FILE__`); see [notes for builds and distribution](../integration/index.md#notes-for-builds-and-distribution)
if that path must not end up in your binaries.
## Change assertion behavior
+23
View File
@@ -112,6 +112,29 @@ Implicit conversions are convenient but can be surprising (for example, in overl
be disabled by defining [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) to `#!cpp 0`,
which forces the explicit `get` form and can catch unintended conversions at compile time.
!!! warning "`auto` copies the JSON value; it does not convert it"
The type of `#!cpp j["name"]` is `json`, not the C++ type of the value inside. `#!cpp auto` therefore deduces
`json`:
```cpp
json j = {{"name", "Mary"}, {"age", 42}};
auto name = j["name"]; // json: a copy of the value; prints as "Mary" (with quotes)
std::string s1 = j["name"]; // std::string: implicit conversion
auto s2 = j["name"].get<std::string>(); // std::string: explicit conversion
auto& age = j["age"]; // json&: a reference, no copy
age = 43; // changes j
```
This is also why `#!cpp std::cout << j["name"]` prints the quotes (see
[getting a string without quotes](#getting-values-out)), and why a `json` value cannot be passed to `printf`: the
argument is not an `int` or a `char*`, and a compiler either rejects passing it through `...` (Clang does) or
compiles it and prints garbage. Convert first: `#!cpp std::printf("%d", j["age"].get<int>());`.
Prefer `get<T>()` over `auto` when you want the C++ value, and `auto&` or `const auto&` when you want to avoid a
copy of a large value.
!!! warning "Conversions do not range-check numbers"
Just like C++ itself, the `get` family performs numeric conversions without range checks — retrieving a
@@ -59,6 +59,25 @@ with a string as the first element is treated as an object, everything else as a
[FAQ](../home/faq.md#brace-initialization-yields-arrays) for details and the opt-in
[`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) macro.
### What does this produce?
The following table shows how some common spellings are interpreted. When in doubt, use `#!cpp json::array()`,
`#!cpp json::object()`, or the `#!cpp json(value)` form with parentheses.
| code | result | note |
|-------------------------------------------------|------------------------------|----------------------------------------------|
| `#!cpp json j = {};` / `#!cpp json j{};` | `#!json null` | use `json::object()` or `json::array()` |
| `#!cpp json j = json::object();` | `#!json {}` | |
| `#!cpp json j = json::array();` | `#!json []` | |
| `#!cpp json j(true);` | `#!json true` | parentheses create the value itself |
| `#!cpp json j = {true};` | `#!json [true]` | braces create a one-element array (`#!cpp json j{true};` depends on the compiler, see the [FAQ](../home/faq.md#brace-initialization-yields-arrays)) |
| `#!cpp json j = {{"a", 1}};` | `#!json {"a":1}` | all elements are `[string, value]` pairs: object |
| `#!cpp json j = {{"a", 1}, {"b"}};` | `#!json [["a",1],["b"]]` | one element is not a pair: array |
| `#!cpp json j = {{"a", 1}, 2};` | `#!json [["a",1],2]` | a plain value among pairs: array |
| `#!cpp json j = {{"a", 1, 2}};` | `#!json [["a",1,2]]` | a three-element list is not a pair: array |
| `#!cpp json j = "{\"a\":1}";` | `#!json "{\"a\":1}"` | a string **value**; it is not parsed |
| `#!cpp json j = json::parse("{\"a\":1}");` | `#!json {"a":1}` | parse text with `parse` or `_json` |
## Building incrementally
A value can also be built up piece by piece. Accessing a non-existing object key or array index with
@@ -21,3 +21,42 @@ flowchart TD
A --> F{"just testing first?"}
F -->|"yes"| G["contains() / find()"]
```
## Does this key exist?
Use [`contains`](../../api/basic_json/contains.md) to test for a key; it does not modify the value and does not throw:
```cpp
json j = {{"name", "Mary"}, {"address", {{"city", "Berlin"}}}};
j.contains("name"); // true
j.contains("age"); // false
j.contains("city"); // false: contains() only looks at the top level
j["address"].contains("city"); // true
j.contains(json::json_pointer("/address/city")); // true: a JSON pointer tests a nested path in one call
```
If the value is needed afterwards, [`find`](../../api/basic_json/find.md) tests and accesses with a single lookup:
```cpp
if (auto it = j.find("name"); it != j.end())
{
std::cout << *it << '\n';
}
```
`contains` also returns `true` for a key whose value is `null`; use `j.contains("k") && !j["k"].is_null()` if a
`null` should count as missing. To read a value with a fallback instead, see [`value`](default_value.md).
!!! warning "`operator[]` inserts missing keys"
Like `#!cpp std::map::operator[]`, the non-const [`operator[]`](unchecked_access.md) adds a `null` value if the key
is missing. Testing a key with it therefore *changes the object*:
```cpp
json j = {{"foo", 1}};
bool absent = j["bar"].is_null(); // true, but j is now {"bar": null, "foo": 1}
```
Use `contains`/`find` to test, and [`at`](checked_access.md) to access a value that must exist (it throws an
[`out_of_range`](../../home/exceptions.md#out-of-range) exception otherwise).
+113
View File
@@ -0,0 +1,113 @@
# Working Without Exceptions
By default, the library reports errors by throwing exceptions (see [Exceptions](../home/exceptions.md)). This page
collects what to use when exceptions are unwanted or unavailable, for example in embedded systems, game engines, or
code compiled with `-fno-exceptions`.
## Two different questions
1. **Can my code avoid *catching* exceptions?** Yes. For parsing and for accessing values there are functions that
report failure through a return value. They work in a normal build and need no `#!cpp try`/`#!cpp catch`.
2. **Can the library be *compiled* without exceptions?** Yes, with `-fno-exceptions` or by defining
[`JSON_NOEXCEPTION`](../api/macros/json_noexception.md). Both have the same effect: every `#!cpp throw` in the
library becomes a call to `std::abort()`. The functions listed below never reach a `#!cpp throw` for the failures
they report, so they keep working in this configuration. Anything else that fails aborts the program.
## Parsing
| function | on invalid input |
|---------------------------------------------------------------------------------------------|-----------------------------------------------------|
| `#!cpp json::parse(input, nullptr, false)` | returns a *discarded* value; test with `is_discarded()` |
| [`accept`](../api/basic_json/accept.md) | returns `#!cpp false` |
| `#!cpp json::from_cbor(input, true, false)` (and the other `from_*` functions) | returns a *discarded* value |
| [`sax_parse`](../api/basic_json/sax_parse.md) with your own handler | returns `#!cpp false` after calling `parse_error` |
??? example "Example: parse and validate without exceptions"
```cpp
--8<-- "examples/no_exceptions_parse.cpp"
```
Output:
```
--8<-- "examples/no_exceptions_parse.output"
```
None of the calls in this example throws, so it also compiles and runs with `-fno-exceptions`.
The discarded value carries no message. If you need the position or the reason, use the SAX interface; see
[Parsing and exceptions](parsing/parse_exceptions.md).
## Accessing values
The access functions are not equally safe. Only some of them never throw:
| expression | if the key is missing or the type is wrong |
|-------------------------------------|-------------------------------------------------------------------------------|
| `#!cpp j.contains("k")` | returns `#!cpp false` for a missing key; never throws |
| `#!cpp j.find("k")` | returns `end()` for a missing key; never throws |
| `#!cpp j.is_string()`, ... | the `is_*` functions test the type; never throw |
| `#!cpp j.get_ptr<const std::string*>()` | returns `#!cpp nullptr` for a different type; never throws |
| `#!cpp j.at("k")` | throws `out_of_range.403` for a missing key |
| `#!cpp j.get<int>()` | throws `type_error.302` for a value that is not a number |
| `#!cpp j.value("k", 0)` | returns the default for a missing key, but **throws `type_error.302` if `k` exists with another type** |
| `#!cpp j["k"]` (non-const) | inserts `null` for a missing key |
| `#!cpp j["k"]` (const) | undefined behavior for a missing key |
So reading a value that may be absent or of the wrong type without exceptions takes two steps: look up, then test the
type.
??? example "Example: look up a key and test its type"
```cpp
--8<-- "examples/no_exceptions_access.cpp"
```
Output:
```
--8<-- "examples/no_exceptions_access.output"
```
The last lines of the example show why `value` is not on the safe list: it returns the default only for a *missing*
key.
For numbers, test with `is_number()` (or `is_number_integer()`, `is_number_unsigned()`, `is_number_float()`) before
calling `get<int>()`, and see [number handling](types/number_handling.md) for range and precision. Whether a value
fits into the target type is not checked by `get`.
## Converting to your own types
`from_json` functions (including the ones generated by the
[`NLOHMANN_DEFINE_TYPE_*` macros](arbitrary_types.md#simplify-your-life-with-macros)) report problems by throwing.
There is no variant that returns an error code. Without exceptions you have two options:
- Validate the JSON value yourself, as shown above, and fill your struct field by field.
- Convert in a function that is compiled **with** exceptions enabled and catches them at the boundary, returning
`std::optional` or an error code to the rest of your code:
??? example "Example: convert to a struct and report failure as std::optional"
```cpp
--8<-- "examples/no_exceptions_convert.c++17.cpp"
```
Output:
```
--8<-- "examples/no_exceptions_convert.c++17.output"
```
## Overriding what happens instead of an exception
With exceptions off, you can replace the `abort()` by your own code, for example to log the message first. See
[Switch off exceptions](../home/exceptions.md#switch-off-exceptions) and
[`JSON_THROW_USER`](../api/macros/json_throw_user.md). The handler must not return normally.
## See also
- [Exceptions](../home/exceptions.md) - all exceptions and their ids
- [Parsing and exceptions](parsing/parse_exceptions.md) - the parsing alternatives in detail
- [Parsing untrusted input](parsing/untrusted_input.md)
- [`JSON_NOEXCEPTION`](../api/macros/json_noexception.md)
+13 -2
View File
@@ -33,7 +33,9 @@ json data = json::parse(f);
```
The input must be encoded in UTF-8; other encodings are not supported. A single input may contain only one JSON value.
Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format.
Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format; input
that arrives in chunks or contains several values is covered in
[Large, streamed, and partial input](large_and_partial_input.md).
By default, the library rejects comments and trailing commas. Both can be enabled with parameters of the `parse`
function — see [comments](../comments.md) and [trailing commas](../trailing_commas.md).
@@ -85,7 +87,7 @@ again without changing any result.
## SAX vs. DOM parsing
The library offers two parsing models:
The library offers two parsing models, and a third, read-only alternative:
- **DOM parsing** (the default): the complete input is read and stored as an in-memory `basic_json` value that can be
traversed and modified freely. This is what [`parse`](../../api/basic_json/parse.md) does, and it is the right choice
@@ -95,6 +97,13 @@ The library offers two parsing models:
inputs or when you only need to extract parts of the input. See the [SAX interface](sax_interface.md) for details and
[`sax_parse`](../../api/basic_json/sax_parse.md) for the API.
- **Document and views** (`#!cpp <nlohmann/json_view.hpp>`, added in version 3.13.0): [`json_document::parse`](../../api/basic_json_document/parse.md)
reads the same JSON grammar but builds a flat, read-only index of the values instead of a `basic_json` tree. Strings and
floating-point numbers are read from the text on demand, and only the parts you
[`materialize()`](../../api/basic_json_view/materialize.md) become `basic_json` values. This avoids most of the work when you
need only a few values from a large or frequently parsed text; see [Zero-copy JSON views](../json_view.md). It is read-only, so use
`parse` when you need to modify the value.
You can influence a DOM parse without switching to the SAX interface by passing a
[parser callback](parser_callbacks.md), which is called during parsing and can, for example, discard parts of the input.
@@ -111,7 +120,9 @@ options.
- [`accept`](../../api/basic_json/accept.md) - check if the input is valid JSON
- [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events
- [JSON Lines](json_lines.md) - parse newline-delimited JSON
- [Large, streamed, and partial input](large_and_partial_input.md) - chunked, concatenated, and very large input
- [parser callbacks](parser_callbacks.md) - influence the parsing by a callback function
- [SAX interface](sax_interface.md) - implement a custom SAX handler
- [Zero-copy JSON views](../json_view.md) - parse into a read-only document and read it through views
- [parsing and exceptions](parse_exceptions.md) - control error handling
- [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources
@@ -0,0 +1,154 @@
# Large, Streamed, and Partial Input
[`parse`](../../api/basic_json/parse.md) reads exactly one complete JSON value and builds it in memory. It cannot be
paused and resumed, and it does not hand back a half-parsed value when the input ends early. This page shows what to do
when the input does not fit that model: it arrives in pieces, it contains several values, or it is too large to hold
as a `json` value.
```mermaid
flowchart TD
A["my input is..."] --> B{"several JSON values in one stream?"}
B -->|"yes"| C["read them one at a time"]
A --> D{"arriving in chunks (socket, pipe)?"}
D -->|"yes"| E["frame the messages, or buffer until complete"]
A --> F{"too large to keep in memory?"}
F -->|"yes"| G["discard parts with a callback, or use SAX"]
A --> H{"only a small part is needed?"}
H -->|"yes"| I["stop early with SAX, or read through a view"]
```
## Several values in one stream
For newline-delimited input, read line by line and parse each line, as described in [JSON Lines](json_lines.md). For
values that are simply concatenated, [`operator>>`](../../api/operator_gtgt.md) reads one value and leaves the stream
positioned after it. Check for the end of the stream before each read, because a read at the end throws:
??? example "Example: read concatenated values"
```cpp
--8<-- "examples/partial_input_concatenated.cpp"
```
Output:
```
--8<-- "examples/partial_input_concatenated.output"
```
The same works for binary formats. [`from_cbor`](../../api/basic_json/from_cbor.md) and its relatives accept
`#!cpp strict = false` to stop after the first value and leave the rest of the stream untouched; `tellg()` then reports
how many bytes the value took:
??? example "Example: read one CBOR value and find out how many bytes it used"
```cpp
--8<-- "examples/partial_input_cbor.cpp"
```
Output:
```
--8<-- "examples/partial_input_cbor.output"
```
## Input arriving in chunks
A socket, pipe, or HTTP body hands over data in pieces of arbitrary size, and a piece can end in the middle of a
number, a string, or an object. The parser does not keep state between calls, so there are two ways to deal with this.
**Frame the messages.** The most robust approach is a protocol that says where a value ends: one value per line
([JSON Lines](json_lines.md)), a length prefix, or a delimiter. Collect the bytes of one message and parse them when
the frame is complete. Prefer this whenever you control the protocol.
**Buffer until the value is complete.** If the data has no framing, append every chunk to a buffer and try to parse
it. A parse error at the very end of the buffer means that the input ended prematurely, while an error anywhere else is
a real syntax error. The member `byte` of the [`parse_error`](../../home/exceptions.md#jsonexceptionparse_error101)
exception is one past the buffer size in the first case:
??? example "Example: buffer chunks until the value is complete"
```cpp
--8<-- "examples/partial_input_buffering.cpp"
```
Output:
```
--8<-- "examples/partial_input_buffering.output"
```
The example feeds a document in five chunks. The first four are reported as incomplete, and a syntax error is reported as
such, not as missing data.
!!! warning "Limits of buffering"
- The buffer must contain exactly one value. Data that belongs to the next value is a syntax error. Use framing if
values follow each other without a separator.
- A number at the end of the buffer is a complete value, even if more digits are still to come (`12` may be the
start of `123`). Buffering cannot detect this; use framing for top-level numbers.
- Every attempt parses the buffer from the beginning, so the total work grows quadratically with the size of the
value. Limit the size of the buffer for input you do not trust (see [untrusted input](untrusted_input.md)).
## Input too large for memory
The input does not have to become one huge `json` value. The two tools below process it piece by piece.
### Handle and discard elements with a callback
A [parser callback](parser_callbacks.md) is called for every finished value. If it returns `#!cpp false`, the value is
discarded and its memory is released at once. For a file that consists of one huge array (or one huge object) of
records, handle each record at depth 1 and discard it:
??? example "Example: handle and discard each record of a large array"
```cpp
--8<-- "examples/partial_input_discard_records.cpp"
```
Output:
```
--8<-- "examples/partial_input_discard_records.output"
```
Only one record is in memory at any time. The same recipe, with more detail, is in the
[parser callbacks](parser_callbacks.md#recipe-streaming-a-large-homogeneous-array) page. To skip entire members you do not
need, discard them when their key is read; see the [example](parser_callbacks.md) for `parse_event_t::key`.
### Extract a few values and stop with SAX
With the [SAX interface](sax_interface.md), no value is built at all, and **returning `#!cpp false` from any handler
function stops the parser**. [`sax_parse`](../../api/basic_json/sax_parse.md) then returns `#!cpp false`, and the rest
of the input is not read. The example finds the first `id` in a document without parsing the remainder:
??? example "Example: stop parsing as soon as a value is found"
```cpp
--8<-- "examples/partial_input_sax_stop.cpp"
```
Output:
```
--8<-- "examples/partial_input_sax_stop.output"
```
Note that `sax_parse` returns `#!cpp false` both when a handler stopped the parse and when the input is invalid, so
record the reason in the handler (here: `id`) if you need to tell the two apart.
### Read parts of a large document with a view
If the whole text is available and you only need to read some of it, a [`json_document`](../json_view.md) from
`<nlohmann/json_view.hpp>` (added in version 3.13.0) avoids the `basic_json` tree altogether: it keeps the text, builds a flat index of the values,
and gives read-only [views](../../api/basic_json_view/index.md) of the parts you look at. Only what you
[`materialize()`](../../api/basic_json_view/materialize.md) becomes a `basic_json` value. See
[Zero-copy JSON views](../json_view.md) for details and for the lifetime rules.
## See also
- [Parsing](index.md) - overview of the parsing functions
- [JSON Lines](json_lines.md) - newline-delimited JSON
- [Parser callbacks](parser_callbacks.md) - discard values while parsing
- [SAX interface](sax_interface.md) - event-based parsing
- [Zero-copy JSON views](../json_view.md) - read parts of a large document without building a tree
- [Untrusted input](untrusted_input.md) - limits for input you do not control
@@ -21,6 +21,10 @@ catch (json::parse_error& ex)
In case exceptions are undesired or not supported by the environment, there are different ways to proceed:
!!! tip
[Working without exceptions](../no_exceptions.md) also covers accessing values and converting to your own types.
## Switch off exceptions
The [`parse()`](../../api/basic_json/parse.md) function accepts a `#!cpp bool` parameter `allow_exceptions` which
@@ -81,3 +85,61 @@ The return value indicates whether the parsing should continue, so the function
```
--8<-- "examples/sax_no_exception.output"
```
## Where did the error occur?
The message of a [`parse_error`](../../home/exceptions.md#jsonexceptionparse_error101) for JSON text contains the line
and column of the error:
```
[json.exception.parse_error.101] parse error at line 3, column 8: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal
```
The exception object itself only stores the **byte index** of the last read character in the member `byte` (the first
byte of the input has index 1). The same index is passed to `parse_error` in the [SAX interface](sax_interface.md). If
you need the line and column as numbers, for example to show them in an editor, compute them from the input:
??? example "Example: compute line and column of a parse error"
```cpp
--8<-- "examples/parse_error_position.cpp"
```
Output:
```
--8<-- "examples/parse_error_position.output"
```
Columns count bytes, so a line with multi-byte UTF-8 characters before the error may differ from the column your editor
shows.
### Where is the value that caused a later error?
A `parse_error` is raised while the text is read. Many mistakes show up later: the text is valid JSON, but a value has
the wrong type when you convert it (`type_error.302`), or a key is missing (`out_of_range.403`). The exception then says
what is wrong, but not where in the input the value was. Define
[`JSON_DIAGNOSTIC_POSITIONS`](../../api/macros/json_diagnostic_positions.md) before including the library to add the
byte range of the offending value to such messages (since version 3.12.0), and to query the range of any value with
[`start_pos()`](../../api/basic_json/start_pos.md) and [`end_pos()`](../../api/basic_json/end_pos.md):
??? example "Example: locate a value with diagnostic positions"
```cpp
--8<-- "examples/parse_error_position_diagnostic.cpp"
```
Output:
```
--8<-- "examples/parse_error_position_diagnostic.output"
```
Positions are only available for values created by [`parse`](../../api/basic_json/parse.md), they are not updated when a
value is changed, and the macro makes every value 16 bytes larger on 64-bit platforms, so use it where the cost is
acceptable, for example for configuration files. Note that `start_pos()` is a 0-based index into the input, whereas
`byte` of a `parse_error` starts at 1.
For binary formats such as CBOR, there are no lines; `byte` is the position in the byte sequence. To find out how many
bytes a successfully parsed value used, see
[Several values in one stream](large_and_partial_input.md#several-values-in-one-stream).
@@ -26,9 +26,11 @@ several alternatives; see [Parsing and exceptions](parse_exceptions.md) for the
[example](parse_exceptions.md#user-defined-sax-interface) that overrides it to print instead of throw.
If exceptions are unavailable entirely (`-fno-exceptions`, or [`JSON_NOEXCEPTION`](../../api/macros/json_noexception.md)
defined), every `#!cpp throw` in the library becomes a call to `std::abort()` -- there is no way to recover from a
parse error of untrusted input in that configuration; see
[Switch off exceptions](../../home/exceptions.md#switch-off-exceptions) for the details and for overriding this with
defined), every `#!cpp throw` in the library becomes a call to `std::abort()`. The alternatives above do not throw, so
`parse()` with `allow_exceptions` set to `#!cpp false` (and likewise `accept()` and the `from_*` functions with
`allow_exceptions` set to `#!cpp false`) can still reject invalid input in that configuration; a plain `parse()` call
aborts on invalid input. See [Working without exceptions](../no_exceptions.md) and
[Switch off exceptions](../../home/exceptions.md#switch-off-exceptions) for the details and for overriding the abort with
`JSON_THROW_USER`.
## Nesting depth
+26 -1
View File
@@ -122,7 +122,9 @@ an adapter, since their template argument order does not match what `basic_json`
[linear in its size](../api/basic_json/basic_json.md#complexity) for the copy constructor but
[constant](../api/basic_json/basic_json.md#complexity) for the move constructor. The same applies to assigning a
large `#!cpp std::string`, `#!cpp std::vector`, or other container into a value: pass it as `#!cpp std::move(x)`
rather than `x` whenever `x` is no longer needed afterwards.
rather than `x` whenever `x` is no longer needed afterwards. Since version 3.13.0, converting an rvalue container
(`#!cpp json j = std::move(vec);`) moves the elements instead of copying them; the source is then left in a valid but
unspecified state.
- **Access without copying.** [`get<T>()`](../api/basic_json/get.md) returns a copy of the stored value converted to
`T`. When a reference or pointer to the value already stored inside the `basic_json` is enough,
[`get_ref()`](../api/basic_json/get_ref.md) and [`get_ptr()`](../api/basic_json/get_ptr.md) access it directly:
@@ -151,6 +153,29 @@ an adapter, since their template argument order does not match what `basic_json`
["reserving array capacity"](element_access/unchecked_access.md#performance-reserving-array-capacity) for the
one-line recipe.
## Memory
- **A value is small, its content is on the heap.** A `basic_json` is 16 bytes on a 64-bit platform (a type tag and a
union). Strings, arrays, objects, and binary values are allocated on the heap and owned by the value. Destroying the
value frees everything it owns.
- **Copies are deep.** After `#!cpp json b = a;`, changing `b` does not change `a`, and both own their memory. Copying a
large value therefore doubles its memory; see [Avoiding copies](#avoiding-copies). `#!cpp auto x = j["key"];` is a
copy, `#!cpp auto& x = j["key"];` is not.
- **A moved-from value is `null`.** After `#!cpp json c = std::move(a);` the value `a` is `#!json null` and owns nothing.
- **Reading does not require a tree.** If you only read the data, parse it into a [`json_document`](json_view.md)
(`#!cpp <nlohmann/json_view.hpp>`, added in version 3.13.0) instead of a `basic_json`. It stores 16 bytes per value, does not copy floating-point
numbers or strings without escape sequences out of the text, and creates no `basic_json` value except for the parts
you [`materialize()`](../api/basic_json_view/materialize.md). A document that borrows the text needs the text to
outlive it; see [Zero-copy JSON views](json_view.md).
- **Containers keep their capacity.** `clear()` and `erase()` on an array remove the elements, but the underlying
`#!cpp std::vector` keeps its allocated capacity, like `#!cpp std::vector::clear` does. To give the memory back,
assign a new value (`#!cpp j = json::array();`) or destroy the value.
- **Freed memory does not always go back to the operating system.** The library returns memory to the allocator
(`#!cpp std::allocator` by default, or the [one you configured](types/template_parameters.md)). Whether the process
size shrinks then depends on the allocator and the platform, as for any C++ program. A process that parses a large
document and drops it is not leaking if its resident size stays high; run it under a tool such as
[Valgrind](https://valgrind.org) or the sanitizers to tell a leak from retained memory.
## Serialization
[`dump()`](../api/basic_json/dump.md) with the default `#!cpp indent = -1` selects "the most compact representation"
+122
View File
@@ -0,0 +1,122 @@
# Recipes for Common Types
The library converts the standard containers, strings, numbers, tuples, `std::optional`, and `std::filesystem::path`
(the last two since version 3.12.0, C++17) out of the box, and your own types once you write conversion functions or use
the macros (see [Arbitrary type conversions](arbitrary_types.md)). It deliberately does **not** convert smart pointers,
`std::variant`, `std::chrono` types, or class hierarchies, because there is more than one reasonable JSON
representation for each of them. This page shows a representation for each that you can copy and adapt.
All recipes use a specialization of [`adl_serializer`](../api/adl_serializer/index.md), which you may provide for types
you do not own, or free `to_json`/`from_json` functions in the namespace of the type. Place them where they are
visible before any code that converts the type; see
[Why isn't my `to_json` found?](arbitrary_types.md#why-isnt-my-to_json-or-from_json-found).
## Smart pointers
A `std::shared_ptr<T>` or `std::unique_ptr<T>` is written as the value it points to, or as `#!json null` for an empty
pointer, and read back by allocating a new `T`:
??? example "Example: serialize shared_ptr and unique_ptr"
```cpp
--8<-- "examples/type_recipes_smart_pointers.c++14.cpp"
```
Output:
```
--8<-- "examples/type_recipes_smart_pointers.c++14.output"
```
With this, containers of smart pointers, pointers to `const`, and members of these types in your own structs convert as
usual. A pointer to a base class cannot be filled this way, because `T` would have
to be created as a derived type; see the next section.
Reading creates a new object. Two `shared_ptr`s that pointed to the same object are written twice and read back as two
separate objects.
## Class hierarchies
!!! tip "Derived classes whose type is known at compile time"
If you only need to serialize the data members of a derived class, and the code always knows the concrete type, you do
not need a type tag. The [`NLOHMANN_DEFINE_DERIVED_TYPE_*` macros](arbitrary_types.md#simplify-your-life-with-macros)
(since version 3.12.0) add the members of the derived class to those of its base class. This section is for the case
where the concrete type is only known at run time, such as a `std::unique_ptr<shape>` that holds a circle or a
rectangle.
JSON has no notion of types, so a value in a hierarchy needs a **type tag** that says which derived class to create.
Write the tag in the object, and select the class from it when reading:
??? example "Example: serialize a class hierarchy with a type tag"
```cpp
--8<-- "examples/type_recipes_class_hierarchy.c++14.cpp"
```
Output:
```
--8<-- "examples/type_recipes_class_hierarchy.c++14.output"
```
A `#!cpp std::vector<std::unique_ptr<shape>>` converts to an array of tagged objects and back, creating a `circle`
and a `rectangle`. The same works for `std::shared_ptr<shape>`. Remember the virtual destructor: without it, deleting a
derived object through the base pointer is undefined behavior.
## `std::variant`
A variant has the same problem: the alternative must be recorded. If the alternatives are your own types, write a type
tag next to their members, exactly as for a hierarchy:
??? example "Example: serialize a std::variant with a type tag"
```cpp
--8<-- "examples/type_recipes_variant.c++17.cpp"
```
Output:
```
--8<-- "examples/type_recipes_variant.c++17.output"
```
In the example, the functions are found
because `geo` is the namespace of the template arguments of `#!cpp std::variant<circ, rect>`; for a variant of only
standard types (for example `#!cpp std::variant<int, std::string>`) there is no such namespace, and you need an
`adl_serializer` specialization or a wrapper type of your own.
If the alternatives are JSON-native (numbers, strings, ...), a variant such as `#!cpp std::variant<int, std::string>`
can instead be written as the plain value, and read by testing the JSON type (`is_number_integer()`, `is_string()`)
to choose the alternative.
## `std::chrono`
A duration is written as its count and read back with the same representation and period. The **unit is not part of
the JSON value**, so document it, for example in the member name (`timeout_ms`):
??? example "Example: serialize durations and system_clock time points"
```cpp
--8<-- "examples/type_recipes_chrono.cpp"
```
Output:
```
--8<-- "examples/type_recipes_chrono.output"
```
A `system_clock::time_point` with millisecond precision is written as the number of milliseconds since the Unix epoch.
The time point serializer is limited to `system_clock`,
because the epoch of other clocks (for example `steady_clock`) has no meaning outside the running program.
If you prefer a text representation such as ISO 8601 (`2023-11-14T22:13:20Z`), write the string with
`#!cpp std::format("{:%FT%TZ}", t)` and read it with `std::chrono::parse` (both C++20), or with a date library, in the
same two functions.
## See also
- [Arbitrary type conversions](arbitrary_types.md) - the conversion mechanism and macros
- [Why isn't my `to_json` found?](arbitrary_types.md#why-isnt-my-to_json-or-from_json-found)
- [Converting values](conversions.md) - `get`, implicit conversions, and `std::optional`
+30
View File
@@ -39,6 +39,8 @@ classDiagram
Exceptions are used widely within the library. They can, however, be switched off with either using the compiler flag `-fno-exceptions` or by defining the symbol [`JSON_NOEXCEPTION`](../api/macros/json_noexception.md). In this case, exceptions are replaced by `abort()` calls. You can further control this behavior by defining `JSON_THROW_USER` (overriding `#!cpp throw`), `JSON_TRY_USER` (overriding `#!cpp try`), and `JSON_CATCH_USER` (overriding `#!cpp catch`).
See [Working without exceptions](../features/no_exceptions.md) for the functions that report errors without throwing.
Note that [`JSON_THROW_USER`](../api/macros/json_throw_user.md) should leave the current scope (e.g., by throwing or aborting), as continuing after it may yield undefined behavior.
??? example "Example: switch off exceptions and log errors before aborting"
@@ -192,6 +194,34 @@ This error indicates a syntax error while deserializing a JSON text. The error m
- Make sure the input is correctly read. Try to write the input to standard output to check if, for instance, the input file was successfully opened.
- Paste the input to a JSON validator like <https://jsonlint.com> or a tool like [jq](https://jqlang.github.io/jq/).
??? info "Parsing a file fails at line 1, column 1 (or \"unexpected end of input\")"
The messages `attempting to parse an empty input` (since version 3.11.3) and `unexpected end of input` (also the message of
earlier versions for an empty input) at the very start of the input mean
that the parser did not see a single character. The library only reads from the stream it is given: it does not
open files and cannot tell a missing file from an empty one. The usual causes are:
- **The file was not opened.** A relative path is resolved against the *working directory of the running program*,
not against the source file or the project. IDEs and build tools often start the program in a build directory
(for example `build/` or `cmake-build-debug/`), so `"example.json"` may need to be `"../example.json"`. Use an
absolute path to rule this out.
- **The file is empty**, or a previous run truncated it (for example when it was opened for writing).
- **The stream was already read.** A stream whose content has been consumed (by printing it with
`std::cout << file.rdbuf()`, or by an earlier `accept`/`parse`/`>>`) is at its end. Parse it once, or rewind it
with `seekg(0)` after clearing the state with `clear()`.
Check the stream before parsing it:
```cpp
std::ifstream f("example.json");
if (!f.is_open())
{
std::cerr << "cannot open example.json\n";
return 1;
}
json data = json::parse(f);
```
### json.exception.parse_error.102
JSON uses the `\uxxxx` format to describe Unicode characters. Code points above 0xFFFF are split into two `\uxxxx` entries ("surrogate pairs"). This error indicates that the surrogate pair is incomplete or contains an invalid code point.
+234 -1
View File
@@ -58,6 +58,208 @@ json j{obj}; // -> {"key":"value"} (copy, not array)
Without the macro (default behavior), `json j{obj}` creates `[{"key":"value"}]`. This opt-in macro fixes issue #5074 while preserving backwards compatibility for existing code.
## Getting started
### Which file do I need?
!!! question
Do I have to clone the repository or build the library?
No. The library is header-only: the single file [`json.hpp`](https://github.com/nlohmann/json/releases) from the
releases page is all you need, including the `_json` literals. Put it in a directory `nlohmann` on your include path.
Companion headers (`json_fwd.hpp`, `json_view.hpp`, `json_literals.hpp`) are only needed if you use them. See
[Integration](../integration/index.md#header-only).
### Using CMake or a package manager
!!! question
How do I add the library with CMake, Conan, vcpkg, Homebrew, or another tool?
See [CMake](../integration/cmake.md) for `find_package`, `add_subdirectory`, and `FetchContent`, and
[Package managers](../integration/package_managers.md) for the list of supported package managers.
### The IDE cannot open the header
!!! question
The compiler finds `nlohmann/json.hpp`, but my IDE reports "cannot open source file". Why?
The compiler's include path is set correctly, but the IDE's is not. Add the same directory to the include paths of the
IDE (or of its language server); the build itself is not affected.
### Compile times
!!! question
Including the library slows down my build. What can I do?
Include [`json_fwd.hpp`](../integration/compile_times.md#include-json_fwdhpp-in-headers) in headers and `json.hpp` only
in the source files that need it, instantiate `basic_json` once, and consider precompiled headers or
[C++20 modules](../features/modules.md). See [Compile times](../integration/compile_times.md).
## Parsing
### Parsing a file fails with "unexpected end of input"
!!! question
`json::parse(file)` throws `parse_error.101` at line 1, column 1 for a file that is valid JSON. Why?
The parser did not see any character: the file was most likely not opened. A relative path is resolved against the
working directory of the running program, which is often the build directory of your IDE, not the source directory.
Check `is_open()` and try an absolute path. Other causes are an empty file and a stream that was already read. See
[`parse_error.101`](exceptions.md#jsonexceptionparse_error101).
### Input that arrives in chunks
!!! question
My data comes from a socket in pieces. How do I parse it?
The parser cannot be resumed. Frame the messages (one value per line, a length prefix) and parse each complete frame, or
buffer the chunks until the value is complete; a `parse_error` at the end of the buffer means the input ended
prematurely. See [Large, streamed, and partial input](../features/parsing/large_and_partial_input.md#input-arriving-in-chunks).
### Several values in one stream
!!! question
How do I read concatenated JSON values or one value per line?
Read newline-delimited input line by line ([JSON Lines](../features/parsing/json_lines.md)). For concatenated values, use
`operator>>` and check for the end of the stream before each read, as shown in
[Several values in one stream](../features/parsing/large_and_partial_input.md#several-values-in-one-stream).
### Files that are too large for memory
!!! question
How do I process a huge JSON file?
Handle and discard the records of a large array with a [parser callback](../features/parsing/parser_callbacks.md), stop
early with the [SAX interface](../features/parsing/sax_interface.md), or read parts of the text through a
[`json_document`](../features/json_view.md) (added in version 3.13.0). See
[Input too large for memory](../features/parsing/large_and_partial_input.md#input-too-large-for-memory).
### Position of a parse error
!!! question
How do I get the line and column of a parse error?
The message contains them. The exception stores the byte index in `byte`, from which you can compute both; see
[Where did the error occur?](../features/parsing/parse_exceptions.md#where-did-the-error-occur). To locate a value that
causes a *later* error, such as a `type_error`, use
[`JSON_DIAGNOSTIC_POSITIONS`](../features/parsing/parse_exceptions.md#where-is-the-value-that-caused-a-later-error).
### Comments and trailing commas
!!! question
Can the parser read JSON with comments or trailing commas?
Yes, when asked to: see [Comments](../features/comments.md) and [Trailing commas](../features/trailing_commas.md). Both
are rejected by default.
## Accessing and creating values
### Reading a value into a variable
!!! question
How do I get a C++ value out of a `json` value?
Use [`get<T>()`](../api/basic_json/get.md), for example `#!cpp int age = j["age"].get<int>();`, or an implicit conversion.
`#!cpp auto x = j["age"];` is **not** a conversion: it copies the `json` value. See
[Converting values](../features/conversions.md).
### Strings are printed with quotes
!!! question
`std::cout << j["name"]` prints `"Mary"` with quotes. How do I get the string?
The value is a `json`, and printing it produces JSON text. Convert it first: `#!cpp j["name"].get<std::string>()`. See
[Getting a string without quotes](../features/conversions.md#getting-values-out).
### Testing whether a key exists
!!! question
How do I check if a key exists, without changing the object?
Use `contains` or `find`. Do not test with the non-const `operator[]`: like `std::map::operator[]`, it inserts a `null`
value for a missing key. See [Does this key exist?](../features/element_access/index.md#does-this-key-exist).
### Reading a value with a fallback
!!! question
How do I read a value that may be missing?
Use [`value`](../features/element_access/default_value.md), which returns a default for a missing key. It still throws if
the key exists with a different type; see [Working without exceptions](../features/no_exceptions.md).
### Iterating and nested values
!!! question
How do I loop over an array or object, or reach a nested value?
See [Iterators](../features/iterators.md) and [`items()`](../api/basic_json/items.md) for loops, and
[JSON Pointer](../features/json_pointer.md) to address or test a nested value by path.
### Unexpected arrays when creating values
!!! question
Why does `json j{x}` or `{{"a", 1}, {"b"}}` produce an array instead of what I expected?
Braces create arrays and objects based on their content, and single-element braces create a one-element array. See
[Brace initialization yields arrays](#brace-initialization-yields-arrays) and the table in
[Creating values](../features/creating_values.md#what-does-this-produce).
### Keeping the order of object keys
!!! question
The keys of my object come out sorted. How do I keep the insertion order?
Use `ordered_json`, and call `ordered_json::parse` when reading. See [Object order](../features/object_order.md).
## Converting your own types
### Structs, enums, and optional fields
!!! question
How do I convert my own types to and from JSON?
Write `to_json`/`from_json` functions or use the `NLOHMANN_DEFINE_TYPE_*` macros; see
[Arbitrary type conversions](../features/arbitrary_types.md). Enums can be mapped to strings
([Enum conversion](../features/enum_conversion.md)), and an empty `std::optional` can be omitted
([Omitting a field](../features/conversions.md#omitting-a-field-when-serializing-stdoptional)).
### Smart pointers, variants, chrono, and class hierarchies
!!! question
Why can't I convert `std::shared_ptr`, `std::variant`, `std::chrono` types, or a pointer to a base class?
There is more than one sensible JSON representation for them, so the library leaves the choice to you.
[Recipes for common types](../features/type_recipes.md) shows one for each.
### My `to_json` is not found
!!! question
I wrote `to_json`/`from_json`, but the compiler does not find them. Why?
Typical causes are a type alias of a standard type, a function in the wrong namespace, or a declaration that comes too
late. See [Why isn't my `to_json` found?](../features/arbitrary_types.md#why-isnt-my-to_json-or-from_json-found).
## Limitations
### Relaxed parsing
@@ -202,6 +404,36 @@ Not directly, but the companion project [json-schema-validator](https://github.c
builds JSON Schema (draft 7; draft 4 in its older, now-superseded 1.x releases) validation on top of this library
and is a common recommendation for this use case.
### Memory use
!!! question
Does the library leak memory? My process stays large after I free a `json` value.
Destroying a value frees everything it owns. Freed memory is returned to the allocator, which does not necessarily give
it back to the operating system, so a high resident size alone is not a leak. Copies are deep, and `clear()` keeps the
capacity of an array. See [Memory](../features/performance.md#memory), and use Valgrind or the sanitizers to tell a leak
from retained memory.
### Reproducible builds
!!! question
The header path ends up in my binary. How do I get path-independent builds?
The runtime assertions embed `__FILE__`. Define `NDEBUG` (the default for release builds) or map the path prefix; see
[Notes for builds and distribution](../integration/index.md#notes-for-builds-and-distribution).
### Slow debug builds
!!! question
Parsing a large file is very slow in my debug build. Can I compile the library with optimizations?
The library is header-only, so there is no separate source file. Keep the code that uses `json` in its own source file and
compile that file with optimizations; see
[Notes for builds and distribution](../integration/index.md#notes-for-builds-and-distribution).
## Exceptions
### Parsing without exceptions
@@ -210,7 +442,8 @@ and is a common recommendation for this use case.
Is it possible to indicate a parse error without throwing an exception?
Yes, see [Parsing and exceptions](../features/parsing/parse_exceptions.md).
Yes, see [Parsing and exceptions](../features/parsing/parse_exceptions.md) and
[Working without exceptions](../features/no_exceptions.md).
### Key name in exceptions
+4
View File
@@ -29,6 +29,10 @@ int main()
j["name"] = "Niels";
j["list"] = {1, 0, 2};
// read a value into a C++ variable and use it
double pi = j["pi"].get<double>();
std::cout << "2 * pi = " << 2 * pi << '\n';
// serialize with an indent of 4 spaces
std::cout << j.dump(4) << '\n';
}
+33
View File
@@ -43,6 +43,21 @@ using json = nlohmann::json;
to the files you want to process JSON and set the necessary switches to enable C++11 (e.g., `-std=c++11` for GCC and
Clang).
!!! tip "Which file do I download?"
For a project without a package manager, the library itself, including the `_json` literals, is **one file**:
[`json.hpp`](https://github.com/nlohmann/json/releases/latest/download/json.hpp) from the
[releases page](https://github.com/nlohmann/json/releases). You do not need to clone the repository, build
anything, or link a library. Put the file in a directory named `nlohmann` that is on your include path (for
example `third_party/nlohmann/json.hpp` with `-I third_party`), so that `#include <nlohmann/json.hpp>` works.
Download the companion headers listed below only if you use them: `json_view.hpp` for the
[zero-copy views](../features/json_view.md), `json_fwd.hpp` for forward declarations, and `json_literals.hpp` if
you define `JSON_NO_AUTOMATIC_UDLS`. Keep them in the same `nlohmann` directory.
Do not copy only parts of the repository's `include/` directory: the headers in it include each other and must be
kept together. If your IDE reports "cannot open source file" while the compiler succeeds, the compiler's include
path is set correctly but the IDE's is not; add the same directory to the IDE's include paths.
You can further use file
[`single_include/nlohmann/json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_fwd.hpp)
for forward declarations (see [Compile times](compile_times.md)), and file
@@ -54,3 +69,21 @@ For the read-only, non-owning [`basic_json_document`](../api/basic_json_document
types, additionally include
[`single_include/nlohmann/json_view.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_view.hpp);
see [Zero-copy JSON views](../features/json_view.md).
## Notes for builds and distribution
**Reproducible and path-independent builds.** The runtime assertions use `assert`, which embeds the name of the source
file (`__FILE__`) in the binary. Release builds do not contain them, because they define `NDEBUG`. If you compile with
assertions enabled and need binaries that do not depend on the build directory, either define `NDEBUG` or map the
prefix of the path with the compiler option `-ffile-prefix-map=<old>=<new>` (GCC 8 and later, Clang 10 and later). See
[Runtime assertions](../features/assertions.md).
**There is no separate source file to compile.** The library is header-only, so there is no `.cpp` that could be
built with different options than the code that includes it. It is also not available as a precompiled library. To
compile the library with optimizations in an otherwise unoptimized build (for example to parse large files in a debug
build, which is slow because of the unoptimized code and the checks of the debug standard library), keep the code that
uses `json` in a separate source file and set the options for that file only. [Compile times](compile_times.md)
shows how to [instantiate `basic_json` once](compile_times.md#instantiate-basic_json-once).
**Debugger visualization.** The Natvis file works with the Visual Studio debugger, including the `cppvsdbg` engine in
Visual Studio Code, but not with debuggers that wrap LLDB, such as `codelldb`; see [Debugging](../home/debugging.md).
+3
View File
@@ -60,6 +60,7 @@ nav:
- Features:
- features/index.md
- features/arbitrary_types.md
- features/type_recipes.md
- Binary Formats:
- features/binary_formats/index.md
- features/binary_formats/bjdata.md
@@ -84,11 +85,13 @@ nav:
- features/modifying_values.md
- features/modules.md
- 'nlohmann Namespace': features/namespace.md
- features/no_exceptions.md
- features/object_order.md
- features/performance.md
- Parsing:
- features/parsing/index.md
- features/parsing/json_lines.md
- features/parsing/large_and_partial_input.md
- features/parsing/parse_exceptions.md
- features/parsing/parser_callbacks.md
- features/parsing/sax_interface.md
+8 -9
View File
@@ -1241,29 +1241,28 @@
}
},
"node_modules/katex": {
"version": "0.18.11",
"resolved": "https://registry.npmjs.org/katex/-/katex-0.18.11.tgz",
"integrity": "sha512-yvyz/2VMKqoho/35Hat5w37wIEdWJ5ElVMidTAlWrxr5uW4aBGsEpveTmKh18/zw08v/7lGswCPon4OhqmqrTA==",
"deprecated": "Accidentally published with breaking changes. Use 0.19.0 instead.",
"version": "0.16.47",
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.47.tgz",
"integrity": "sha512-Eeo8Ys1doU1z+x8AZsPpQu+p/QcZBI5PeOo7QGQdy2x2m0MU/hYagBbGOmXwr5KVbEfVuWv9LpnQWeehogurjg==",
"funding": [
"https://opencollective.com/katex",
"https://github.com/sponsors/katex"
],
"license": "MIT",
"dependencies": {
"commander": "^15.0.0"
"commander": "^8.3.0"
},
"bin": {
"katex": "cli.js"
}
},
"node_modules/katex/node_modules/commander": {
"version": "15.0.0",
"resolved": "https://registry.npmjs.org/commander/-/commander-15.0.0.tgz",
"integrity": "sha512-z67u4ZhzCL/Tydu1lJARtEZYWbWaN7oYLHbsuzocr6y4N6WZAagG3RQ4FW61V1/0+jImpj293XfrcYnd1qxtPg==",
"version": "8.3.0",
"resolved": "https://registry.npmjs.org/commander/-/commander-8.3.0.tgz",
"integrity": "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==",
"license": "MIT",
"engines": {
"node": ">=22.12.0"
"node": ">= 12"
}
},
"node_modules/khroma": {
-3
View File
@@ -6,8 +6,5 @@
"dependencies": {
"jsdom": "30.1.3",
"mermaid": "11.17.2"
},
"overrides": {
"katex": "0.18.11"
}
}
@@ -86,7 +86,7 @@ JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 2
/*!
@brief deserialization of BJData, BON8, BSON, CBOR, MessagePack, and UBJSON values
*/
template<typename BasicJsonType, typename InputAdapterType, typename SAX = json_sax_dom_parser<BasicJsonType, InputAdapterType>>
template<typename BasicJsonType, typename InputAdapterType, typename SAX = json_sax_dom_binary_parser<BasicJsonType, InputAdapterType>>
class binary_reader
{
using number_integer_t = typename BasicJsonType::number_integer_t;
File diff suppressed because it is too large. Load diff
+10
View File
@@ -14,6 +14,7 @@
#include <cstdio> // snprintf
#include <cstring> // memcmp
#include <initializer_list> // initializer_list
#include <limits> // numeric_limits
#include <string> // char_traits, string
#include <utility> // move
#include <vector> // vector
@@ -1816,6 +1817,15 @@ scan_number_done:
/// significand (see scan_number_bulk_contiguous())
bool convert_accumulated_float(const float_significand& s, std::true_type /*native*/) noexcept
{
// accept() only needs to know that the value is finite (see the parser):
// w < 10^19, so a value below 10^max_exponent10 cannot overflow, and a
// value that underflows is a finite number as well. Only a token that
// may overflow has to be converted to find out.
if (JSON_HEDLEY_UNLIKELY(discard_number_values && s.exponent + 19 <= std::numeric_limits<number_float_t>::max_exponent10))
{
value_float = 0;
return true;
}
value_float = decimal_to_float<number_float_t>(s);
return true;
}
+12 -5
View File
@@ -56,8 +56,8 @@ using parser_callback_t =
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
@@ -131,7 +131,7 @@ class parser
}
else
{
json_sax_dom_parser<BasicJsonType, InputAdapterType> sdp(result, allow_exceptions, &m_lexer);
json_sax_dom_parser<BasicJsonType, InputAdapterType, true> sdp(result, allow_exceptions, &m_lexer);
// in case of an error, return a discarded value
if (!parse_dom(sdp, strict))
@@ -243,12 +243,19 @@ class parser
JSON_HEDLEY_NON_NULL(2)
bool sax_parse_internal(SAX* sax)
{
// value to avoid a goto (see comment where set to true)
bool skip_to_state_evaluation = false;
// stack to remember the hierarchy of structured values we are parsing
// true = array; false = object (bytes, not std::vector<bool>, whose
// push_back() is a call with GCC)
std::vector<std::uint8_t> states;
// value to avoid a goto (see comment where set to true)
bool skip_to_state_evaluation = false;
// a document that starts with a container pushes onto the stack at
// once: allocate it in one step instead of growing it from 1 byte
if (last_token == token_type::begin_object || last_token == token_type::begin_array)
{
states.reserve(32);
}
while (true)
{
+163 -47
View File
@@ -275,11 +275,34 @@ class builder
0, 0, '\b', 0, 0, 0, '\f', 0, 0, 0, 0, 0, 0, 0, '\n', 0, 0, 0, '\r', 0, '\t', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
}
};
// The write position is kept in locals: the stores of characters into
// the arena may alias the members of the builder, which would then be
// reloaded after every one. It is stored back where the arena grows,
// and by `write_back` on every return.
const unsigned char* const end = m_end;
const std::size_t start = arena_used();
arena_run(s, static_cast<std::size_t>(p - s));
char* w = m_arena_write;
const char* w_end = m_arena_end;
const arena_write_back write_back(*this, w);
{
const auto n = static_cast<std::size_t>(p - s);
if (n <= 16 && end - s >= 16)
{
arena_ensure(w, w_end, 16);
std::memcpy(w, s, 16);
}
else if (n != 0)
{
// (an empty run leaves an empty arena unallocated, and memcpy()
// must not be passed its null pointer even for zero bytes)
arena_ensure(w, w_end, n);
std::memcpy(w, s, n);
}
w += n;
}
for (;;)
{
if (p == m_end)
if (p == end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -287,7 +310,9 @@ class builder
if (c == '"')
{
++p;
return decoded{p, start, arena_used() - start};
// (from w: `write_back` stores it only after the return value
// is made; the arena is allocated, since an escape was written)
return decoded{p, start, static_cast<std::size_t>(w - m_doc.arena.data()) - start};
}
if (c != '\\')
{
@@ -297,7 +322,7 @@ class builder
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == m_end)
if (p == end)
{
return failed(error_code::string_missing_quote, p);
}
@@ -315,7 +340,7 @@ class builder
{
return failed(error_code::string_surrogate_low, p);
}
if (m_end - p < 2 || p[0] != '\\' || p[1] != 'u')
if (end - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
@@ -331,26 +356,106 @@ class builder
}
cp = 0x10000 + ((cp - 0xD800) << 10) + (lo - 0xDC00);
}
arena_ensure(4);
m_arena_write = put_utf8(m_arena_write, cp);
arena_ensure(w, w_end, 4);
w = put_utf8(w, cp);
}
else if (NLOHMANN_VIEW_LIKELY(d < 128 && simple_escape[d] != 0))
{
arena_ensure(1);
*m_arena_write++ = simple_escape[d];
arena_ensure(w, w_end, 1);
*w++ = simple_escape[d];
}
else
{
--p;
return failed(error_code::string_escape, p);
}
if (p != m_end && *p == '\\')
#if NLOHMANN_VIEW_NEON
if (p != end && *p == '\\')
{
continue; // consecutive escapes ("\u00e4\u00f6"): no run in between
}
// The first 16 bytes of the run are checked one by one, in predicted branches, and copied
// as one block: where the runs between escapes are short, the vector step below would put
// its longer dependency chain (load, compare, mask, count, add) on p for each of them,
// and be slower than these steps (the scanner of the keys works the same way)
{
if (end - p >= 16)
{
const std::uint8_t* const plain = string_plain();
std::size_t steps = 16;
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { steps = (i); goto run_found; }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
#undef NLOHMANN_VIEW_STEP
run_found:
arena_ensure(w, w_end, 16);
std::memcpy(w, p, 16); // (only the first `steps` bytes are kept)
w += steps;
p += steps;
if (steps < 16)
{
if (*p >= 0x80)
{
// non-ASCII: the UTF-8 check, and the rest of the run
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
arena_ensure(w, w_end, n);
std::memcpy(w, r, n);
w += n;
}
continue;
}
}
}
#else
if (p != end && (*p == '\\' || *p == '"'))
{
continue; // consecutive escapes ("\u00e4\u00f6"), or the end of the string: no run in between
}
#endif
#if NLOHMANN_VIEW_VECTOR
// the plain bytes up to the next quote, backslash, or control
// character are copied as they are scanned, 16 bytes at a time
for (;;)
{
if (NLOHMANN_VIEW_UNLIKELY(end - p < 16))
{
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
arena_ensure(w, w_end, n);
std::memcpy(w, r, n);
w += n;
break;
}
arena_ensure(w, w_end, 16);
const unsigned k = copy_plain_run16(p, w);
w += k;
p += k;
if (k == 16)
{
continue;
}
if (NLOHMANN_VIEW_UNLIKELY(*p >= 0x80))
{
// non-ASCII: the UTF-8 check, and the rest of the run
const unsigned char* const r = p;
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
arena_ensure(w, w_end, n);
std::memcpy(w, r, n);
w += n;
}
break;
}
#else
const unsigned char* const r = p;
p = scan_string_run(p, m_end);
arena_run(r, static_cast<std::size_t>(p - r));
p = scan_string_run(p, end);
const auto n = static_cast<std::size_t>(p - r);
arena_ensure(w, w_end, n);
std::memcpy(w, r, n);
w += n;
#endif
}
}
@@ -392,19 +497,49 @@ class builder
char* m_arena_write = nullptr;
char* m_arena_end = nullptr;
/// room for n more bytes at w, the write position kept in a local, with
/// w_end the end of the arena. (A lambda instead is not always inlined,
/// and then w would be kept in memory, as m_arena_write is.)
NLOHMANN_VIEW_ALWAYS_INLINE void arena_ensure(char*& w, const char*& w_end, std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(w_end - w) < n))
{
m_arena_write = w;
arena_grow(n);
w = m_arena_write;
w_end = m_arena_end;
}
}
/// stores a write position kept in a local back into m_arena_write when
/// the function that keeps it returns, on every path
class arena_write_back
{
public:
arena_write_back(builder& b, char*& w) noexcept
: m_builder(b)
, m_w(w)
{}
~arena_write_back()
{
m_builder.m_arena_write = m_w;
}
arena_write_back(const arena_write_back&) = delete;
arena_write_back& operator=(const arena_write_back&) = delete;
arena_write_back(arena_write_back&&) = delete;
arena_write_back& operator=(arena_write_back&&) = delete;
private:
builder& m_builder;
char*& m_w;
};
NLOHMANN_VIEW_ALWAYS_INLINE std::size_t arena_used() const noexcept
{
return m_arena_write != nullptr ? static_cast<std::size_t>(m_arena_write - m_doc.arena.data()) : 0;
}
NLOHMANN_VIEW_ALWAYS_INLINE void arena_ensure(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_arena_end - m_arena_write) < n))
{
arena_grow(n);
}
}
NLOHMANN_VIEW_NOINLINE void arena_grow(std::size_t n)
{
const std::size_t used = arena_used();
@@ -413,25 +548,6 @@ class builder
m_arena_end = m_doc.arena.data() + m_doc.arena.size();
}
/// append the run [r, r + n) to the arena; short runs as one fixed-size
/// 16-byte move when both sides have the room (no library call)
NLOHMANN_VIEW_ALWAYS_INLINE void arena_run(const unsigned char* r, std::size_t n)
{
if (n <= 16 && m_end - r >= 16)
{
arena_ensure(16);
std::memcpy(m_arena_write, r, 16);
}
else if (n != 0)
{
// (an empty run leaves an empty arena unallocated, and memcpy()
// must not be passed its null pointer even for zero bytes)
arena_ensure(n);
std::memcpy(m_arena_write, r, n);
}
m_arena_write += n;
}
/// UTF-8 encoding of cp at w (room for 4 bytes); returns the position after
/// it. (A returned pointer stays in a register; writing through a
/// reference to m_arena_write would store and reload it around every byte.)
@@ -954,11 +1070,11 @@ indent_done:
++p;
}
const unsigned char* const int_start = p;
if (p != e && *p == '0')
if (cur() == '0')
{
++p;
}
else if (NLOHMANN_VIEW_LIKELY(p != e && *p >= '1' && *p <= '9'))
else if (NLOHMANN_VIEW_LIKELY(cur() >= '1' && cur() <= '9'))
{
p = skip_digits(p + 1, e);
}
@@ -969,7 +1085,7 @@ indent_done:
const auto int_digits = static_cast<std::size_t>(p - int_start);
std::size_t frac_digits = 0;
bool is_float = false;
if (p != e && *p == '.')
if (cur() == '.')
{
++p;
const unsigned char* const f0 = p;
@@ -982,20 +1098,20 @@ indent_done:
is_float = true;
}
std::int64_t exponent = 0;
if (p != e && (*p | 0x20) == 'e')
if ((cur() | 0x20) == 'e')
{
++p;
bool exp_negative = false;
if (p != e && (*p == '+' || *p == '-'))
if (cur() == '+' || cur() == '-')
{
exp_negative = *p == '-';
++p;
}
if (NLOHMANN_VIEW_UNLIKELY(p == e || !is_digit(*p)))
if (NLOHMANN_VIEW_UNLIKELY(!is_digit(cur())))
{
return fail(error_code::number_after_exponent);
}
while (p != e && is_digit(*p))
while (is_digit(cur()))
{
if (exponent < 100000)
{
@@ -1057,7 +1173,7 @@ indent_done:
++p; // opening quote
const unsigned char* const s = p;
p = scan_string_run<Value>(p, e);
if (NLOHMANN_VIEW_LIKELY(p != e && *p == '"'))
if (NLOHMANN_VIEW_LIKELY(cur() == '"'))
{
push_node(value_t::string, 0, 0, static_cast<std::size_t>(s - b), static_cast<std::uint64_t>(p - s));
++p;
+34
View File
@@ -10,6 +10,9 @@
#include <algorithm> // max
#include <cstddef> // size_t
#ifndef JSON_NO_IO
#include <cstdio> // FILE, fread
#endif
#include <istream> // istream, streambuf
#include <string> // basic_string, char_traits, string
#include <type_traits> // decay, integral_constant, is_array, is_const, is_integral, is_lvalue_reference, is_pointer, is_same, remove_cv, remove_extent, remove_pointer, remove_reference
@@ -154,6 +157,37 @@ inline void append_stream(std::string& buf, std::istream& is)
is.clear(is.rdstate() | std::ios::eofbit);
}
/// a pointer to a C stream (never with JSON_NO_IO, which leaves out <cstdio>)
template<typename T>
struct is_c_stream
#ifndef JSON_NO_IO
: std::is_same<T, std::FILE*>
#else
: std::false_type
#endif
{};
#ifndef JSON_NO_IO
/// append the rest of a C stream: in blocks with fread, not character by
/// character (the library's file adapter reads with fgetc);
/// f is not null
inline void append_file(std::string& buf, std::FILE* f)
{
for (;;)
{
const std::size_t used = buf.size();
const std::size_t room = (std::max)(buf.capacity() - used, std::size_t{4096});
buf.resize(used + room);
const std::size_t got = std::fread(&buf[used], 1, room, f);
buf.resize(used + got);
if (got < room)
{
break; // end of file, or an error (as fgetc, which returns EOF for both)
}
}
}
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+10 -8
View File
@@ -147,20 +147,21 @@ class view_item
view_item(const iterator& it, std::size_t index)
: m_it(it)
{
if (!it.is_object_iterator())
{
m_index = std::to_string(index);
}
}
, m_position(index)
{}
/// the member key, or the element index for arrays
/// the member key, or the element index for arrays (as text made on the
/// first call, so that iterating over the values pays nothing for it)
string_view_t key() const
{
if (m_it.is_object_iterator())
{
return m_it.key();
}
if (m_index.empty())
{
m_index = std::to_string(m_position);
}
return string_view_t(m_index.data(), m_index.size());
}
@@ -183,7 +184,8 @@ class view_item
private:
iterator m_it;
std::string m_index{}; // NOLINT(readability-redundant-member-init)
std::size_t m_position; ///< the index of an array element
mutable std::string m_index{}; ///< the text of m_position, once key() was called // NOLINT(readability-redundant-member-init)
};
/// the range returned by basic_json_view::items()
+8
View File
@@ -166,6 +166,10 @@ const node* element_at(const document_data& d, const node* array, std::size_t id
{
return e + idx; // a growable block: one link per element
}
if (array->next == array->len + 1u && (!nav::editable() || (array->flags & node_flags::moved) == 0))
{
return e + idx; // no element has a subtree: the elements are the nodes after the array
}
for (std::size_t i = 0; i < idx; ++i)
{
e = document_data::after(e);
@@ -181,6 +185,10 @@ const node* last_child(const document_data& d, const node* container) noexcept
const std::size_t value_offset = container->kind == static_cast<std::uint8_t>(value_t::object) ? 1 : 0;
const node* const end = navigation<Editable>::end(d, container);
const node* last = navigation<Editable>::first(d, container);
if (value_offset == 0 && container->next == container->len + 1u && (!navigation<Editable>::editable() || (container->flags & node_flags::moved) == 0))
{
return last + (container->len - 1u); // an array without subtrees: the elements are the nodes after it
}
for (const node* c = document_data::after(last + value_offset); c != end; c = document_data::after(c + value_offset))
{
last = c;
+47 -7
View File
@@ -8,7 +8,7 @@
#pragma once
#include <algorithm> // max
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t
#include <string> // string
@@ -40,12 +40,13 @@ template<typename BasicJsonType, bool Editable>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>>;
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>, true>;
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
// their end, and whether they are objects; sized once from the
// document's deepest nesting (for a subtree an upper bound)
// their end, and whether they are objects; for the first 16 levels on the
// stack, else sized once from the document's deepest nesting (for a
// subtree an upper bound)
struct frame
{
const node* pos;
@@ -53,10 +54,43 @@ BasicJsonType materialize(const document_data& d, const node* n)
bool object;
};
// a scalar needs neither the handler nor the stack of open containers
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
return BasicJsonType(string_t(d.str(*n), n->len));
case value_t::number_integer:
return BasicJsonType(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(*n))));
case value_t::number_unsigned:
return BasicJsonType(static_cast<typename BasicJsonType::number_unsigned_t>(integer_bits(*n)));
case value_t::number_float:
return BasicJsonType(float_value<typename BasicJsonType::number_float_t>(d, *n));
case value_t::boolean:
return BasicJsonType((n->flags & node_flags::is_true) != 0);
case value_t::null:
return BasicJsonType(nullptr);
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
break;
}
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
std::vector<frame> open((std::max)(d.max_depth, std::size_t{1}));
// (shallow subtrees on the stack: a call for a small value then allocates only for the value)
std::array<frame, 16> shallow; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled as containers open
std::vector<frame> deep{};
frame* open = shallow.data();
std::size_t capacity = shallow.size();
if (d.max_depth > capacity)
{
deep.resize(d.max_depth);
open = deep.data();
capacity = deep.size();
}
std::size_t depth = 0;
for (;;)
{
@@ -76,9 +110,13 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
if (NLOHMANN_VIEW_UNLIKELY(depth == open.size()))
if (NLOHMANN_VIEW_UNLIKELY(depth == capacity))
{
open.resize(depth * 2); // (deeper than the recorded nesting, e.g. after edits)
// (deeper than the recorded nesting, e.g. after edits)
deep.assign(open, open + depth);
deep.resize(depth * 2);
open = deep.data();
capacity = deep.size();
}
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
break;
@@ -105,6 +143,8 @@ BasicJsonType materialize(const document_data& d, const node* n)
case value_t::binary:
case value_t::discarded:
default:
// false positive: the parser's frame stack points into its own storage (never null)
// @infer-ignore NULLPTR_DEREFERENCE
sax.null();
break;
}
+101 -11
View File
@@ -47,27 +47,96 @@ constexpr std::size_t index_max_displacement = 64;
static_assert(document_data::index_min_members >= 2 * index_max_displacement,
"a table must be much larger than the probe window of a lookup");
/// hash of a key: its bytes, eight at a time, in a fixed byte order
/// hash of a key: its bytes, eight at a time (in a fixed byte order), where the
/// last word may overlap the one before (so that no loop over single bytes is
/// needed); a key of fewer than eight bytes is read as two overlapping words of
/// four or two bytes (in the native byte order: the hash is not stored, and the
/// same on both sides of a lookup)
inline std::uint64_t key_hash(const char* s, std::size_t n) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15u * (n + 1);
const auto* p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
while (n >= 8)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
n -= 8;
}
std::uint64_t w = 0;
for (std::size_t i = 0; i < n; ++i)
if (n >= 8)
{
w |= static_cast<std::uint64_t>(p[i]) << (8u * i);
const unsigned char* const last = p + n - 8;
while (p < last)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
}
w = read_eight_bytes(last);
}
h = (h ^ w) * 0x94D049BB133111EBu;
else if (n >= 4)
{
std::uint32_t head = 0;
std::uint32_t tail = 0;
std::memcpy(&head, p, 4);
std::memcpy(&tail, p + n - 4, 4);
w = static_cast<std::uint64_t>(head) | (static_cast<std::uint64_t>(tail) << 32u);
}
else if (n >= 2)
{
std::uint16_t head = 0;
std::uint16_t tail = 0;
std::memcpy(&head, p, 2);
std::memcpy(&tail, p + n - 2, 2);
w = static_cast<std::uint64_t>(head) | (static_cast<std::uint64_t>(tail) << 16u);
}
else if (n == 1)
{
w = p[0];
}
// (a product only carries bits upwards, and the slot is taken from the low
// bits: the high bits of the last word are folded down before the multiplication)
h = (h ^ w ^ (w >> 29u)) * 0x94D049BB133111EBu;
return h ^ (h >> 31u);
}
/// whether the n <= 16 bytes at a and b are equal, as two overlapping words
/// (the first and the last 8, 4, or 2 bytes) and without a call
NLOHMANN_VIEW_ALWAYS_INLINE bool same_short_bytes(const unsigned char* a, const unsigned char* b, std::size_t n) noexcept
{
if (n >= 8)
{
std::uint64_t x0 = 0;
std::uint64_t x1 = 0;
std::uint64_t y0 = 0;
std::uint64_t y1 = 0;
std::memcpy(&x0, a, 8);
std::memcpy(&x1, a + n - 8, 8);
std::memcpy(&y0, b, 8);
std::memcpy(&y1, b + n - 8, 8);
return x0 == y0 && x1 == y1;
}
if (n >= 4)
{
std::uint32_t x0 = 0;
std::uint32_t x1 = 0;
std::uint32_t y0 = 0;
std::uint32_t y1 = 0;
std::memcpy(&x0, a, 4);
std::memcpy(&x1, a + n - 4, 4);
std::memcpy(&y0, b, 4);
std::memcpy(&y1, b + n - 4, 4);
return x0 == y0 && x1 == y1;
}
if (n >= 2)
{
std::uint16_t x0 = 0;
std::uint16_t x1 = 0;
std::uint16_t y0 = 0;
std::uint16_t y1 = 0;
std::memcpy(&x0, a, 2);
std::memcpy(&x1, a + n - 2, 2);
std::memcpy(&y0, b, 2);
std::memcpy(&y1, b + n - 2, 2);
return x0 == y0 && x1 == y1;
}
return n == 0 || *a == *b;
}
/// build the table of a large object
inline void build_object_index(document_data& d, node* obj)
{
@@ -132,6 +201,27 @@ inline const node* find_indexed(const document_data& d, const node* obj, const c
const document_data::object_index& ix = d.indexes[obj->extra - 1u];
const std::uint32_t* const table = d.index_slots.data() + ix.start;
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, n)) & ix.mask;
const auto* const k_bytes = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
// (the loop for longer keys comes first, as the code is laid out; it is not the more common case)
if (NLOHMANN_VIEW_UNLIKELY(n <= 16))
{
// (up to 16 bytes, the bytes are compared as two words, without a call)
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
if (s == 0)
{
return nullptr;
}
const node* const k = obj + s;
if (k->len == n && same_short_bytes(reinterpret_cast<const unsigned char*>(d.str(*k)), k_bytes, n)) // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
{
return k;
}
slot = (slot + 1) & ix.mask;
}
return nullptr;
}
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
+373 -256
View File
@@ -23,6 +23,7 @@
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/simd.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -80,13 +81,6 @@ class output_buffer
put(s, N - 1);
}
void put_repeated(char c, std::size_t n)
{
reserve(n);
std::memset(m_pos, c, n);
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
@@ -143,9 +137,27 @@ class output_buffer
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
std::size_t i = 0;
#if NLOHMANN_VIEW_SSE2
// (16 bytes per step; with NEON, the mask of a step costs more than it saves on the
// short runs that are typical here)
// NOLINTBEGIN(portability-simd-intrinsics)
for (; i + 16 <= n; i += 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
// (a byte is below 0x20 if it is its own minimum with 0x1F)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (mask != 0)
{
return i + static_cast<std::size_t>(count_trailing_zeros(mask));
}
}
// NOLINTEND(portability-simd-intrinsics)
#endif
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
@@ -196,6 +208,70 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
return true;
}
#if NLOHMANN_VIEW_VECTOR
/// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
/// backslash, a control character, DEL, or not ASCII; a signed compare with
/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
/// byte (x86) or one nibble per byte (NEON)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
#endif
}
/// the width in bits of a byte in the mask of ascii_stop_mask()
constexpr unsigned ascii_stop_bits() noexcept
{
return NLOHMANN_VIEW_NEON ? 4u : 1u;
}
#endif
/// As find_ascii_copyable_run(), 16 bytes per step. end is the end of the
/// buffer that holds the n bytes at p (a short run is read as 16 bytes if
/// they are in the buffer).
inline std::size_t ascii_copyable_run(const unsigned char* p, std::size_t n, const unsigned char* end) noexcept
{
#if NLOHMANN_VIEW_VECTOR
if (n >= 16)
{
std::size_t i = 0;
for (; i + 16 <= n; i += 16)
{
const std::uint64_t mask = ascii_stop_mask(p + i);
if (mask != 0)
{
return i + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits());
}
}
if (i == n)
{
return n;
}
// the last 16 bytes: those before i are known to be copyable
const std::uint64_t mask = ascii_stop_mask(p + n - 16);
return mask != 0 ? n - 16 + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits()) : n;
}
if (end - p >= 16)
{
// (the bytes after the run are masked out)
const std::uint64_t mask = ascii_stop_mask(p) & ((std::uint64_t{1} << (n * ascii_stop_bits())) - 1u);
return mask != 0 ? static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits() : n;
}
#else
static_cast<void>(end);
#endif
return find_ascii_copyable_run(p, n);
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
@@ -275,6 +351,39 @@ NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexc
return w + N - 1;
}
/// the depth of the innermost open container, counted only for indented output
template<bool Counted>
struct depth_counter
{
std::size_t value = 0;
void enter() noexcept
{
++value;
}
std::size_t leave() noexcept
{
return --value;
}
std::size_t get() const noexcept
{
return value;
}
};
template<>
struct depth_counter<false>
{
void enter() noexcept {}
std::size_t leave() noexcept // NOLINT(readability-convert-member-functions-to-static): same interface as depth_counter<true>
{
return 0;
}
std::size_t get() const noexcept // NOLINT(readability-convert-member-functions-to-static): same interface as depth_counter<true>
{
return 0;
}
};
/// how the view's dump() writes a value
struct dump_style
{
@@ -311,139 +420,95 @@ class view_serializer
void dump(const node* root)
{
if (!m_style.pretty)
if (m_style.ensure_ascii)
{
if (m_style.ensure_ascii)
{
dump_compact_ascii(root);
}
else if (m_style.source_numbers)
{
dump_compact<true, false>(root);
}
else
{
dump_compact<false, false>(root);
}
return;
dump_ascii(root);
}
struct frame
else if (m_style.pretty)
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
bool first; ///< nothing written yet
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const node* n = root;
for (;;)
dump_pretty(root);
}
else if (m_style.source_numbers)
{
// write the value at n
if (is_container(*n))
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
if (n->len == 0)
{
m_out.put(object ? "{}" : "[]", 2);
}
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
}
}
else
{
write_scalar(*n);
}
// go to the next value: close finished containers, then separate
for (;;)
{
if (stack.empty())
{
m_out.finish();
return;
}
if (stack.back().pos == stack.back().end)
{
const bool object = stack.back().object;
stack.pop_back();
newline(stack.size());
m_out.put(object ? '}' : ']');
continue;
}
frame& f = stack.back();
if (!f.first)
{
m_out.put(',');
}
f.first = false;
newline(stack.size());
if (f.object)
{
write_string(*f.pos);
if (m_style.pretty)
{
m_out.put(": ");
}
else
{
m_out.put(':');
}
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
}
else
{
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
}
break;
}
dump_text<true, false, false>(root);
}
else
{
dump_text<false, false, false>(root);
}
}
private:
/// dump_text() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_ascii(const node* root)
{
if (m_style.pretty)
{
if (m_style.source_numbers)
{
dump_text<true, true, true>(root);
}
else
{
dump_text<false, true, true>(root);
}
}
else if (m_style.source_numbers)
{
dump_text<true, false, true>(root);
}
else
{
dump_text<false, false, true>(root);
}
}
/// dump_text() with indentation, out of line: the compact walk is faster
/// when it is not merged with the indented ones into dump()
NLOHMANN_VIEW_NOINLINE void dump_pretty(const node* root)
{
if (m_style.source_numbers)
{
dump_text<true, true, false>(root);
}
else
{
dump_text<false, true, false>(root);
}
}
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
/// are copied. If validate is set, the string must be valid UTF-8: the
/// runs are found by the lexer's scan, which validates the sequences, and
/// invalid UTF-8 throws what basic_json::dump() throws. (The compact output
/// writes the bytes of a damaged image unchanged, and scans without
/// validating them.)
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n, bool validate)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
const std::size_t run = validate ? string_bulk_run(s + i, n.len - i) : plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
if (validate && s[i] >= 0x80)
{
invalid_utf8(s, n.len); // (the scan stopped at ill-formed UTF-8)
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// dump_compact() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_compact_ascii(const node* root)
{
if (m_style.source_numbers)
{
dump_compact<true, true>(root);
}
else
{
dump_compact<false, true>(root);
}
}
/// whether dump_compact() copies a string or key from the source: one of
/// whether dump_text() copies a string or key from the source: one of
/// the source, and with ensure_ascii one without bytes >= 0x7F (an overload
/// each, so that the walk without ensure_ascii is compiled exactly as
/// before: with GCC, a condition that only folds away later changed its
@@ -458,13 +523,155 @@ class view_serializer
return (n.flags & node_flags::storage) == 0 && is_ascii_without_delete(src + n.off, n.len);
}
/// a string with ensure_ascii (dump_compact()), out of line
NLOHMANN_VIEW_NOINLINE void write_ascii_string(const node& n)
/*!
@brief a string with ensure_ascii: the characters that are not printable ASCII as \\uXXXX
(a character beyond U+FFFF as a surrogate pair); strings of a document
are valid UTF-8, except in a damaged image loaded with image_check::bounds,
for which this throws what basic_json::dump() throws for the string
*/
NLOHMANN_VIEW_NOINLINE void write_ascii(const node& n)
{
write_string(n);
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const std::size_t size = n.len;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for k bytes
const auto room = [&](std::size_t k)
{
const auto left = static_cast<std::size_t>(lim - w);
if (NLOHMANN_VIEW_UNLIKELY(left < k))
{
m_out.set_cursor(w);
m_out.reserve(k);
w = m_out.cursor();
lim = m_out.limit();
}
};
static constexpr const char* hex = "0123456789abcdef";
// \\uXXXX
const auto u_escape = [&](std::uint32_t u)
{
w[0] = '\\';
w[1] = 'u';
w[2] = hex[(u >> 12u) & 0xFu];
w[3] = hex[(u >> 8u) & 0xFu];
w[4] = hex[(u >> 4u) & 0xFu];
w[5] = hex[u & 0xFu];
w += 6;
};
room(1);
*w++ = '"';
std::size_t i = 0;
// (a string of the source starts with a run that has nothing to escape, 16 bytes per step)
const std::size_t prefix = (n.flags & node_flags::storage) == 0 ? ascii_copyable_run(s, size, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)) : 0; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (prefix != 0)
{
room(prefix);
std::memcpy(w, s, prefix);
w += prefix;
i = prefix;
}
while (i < size)
{
// (a character that needs an escape sequence is not worth a call)
const std::size_t run = is_ascii_copyable(s[i]) ? find_ascii_copyable_run(s + i, size - i) : 0;
if (run != 0)
{
room(run);
std::memcpy(w, s + i, run);
w += run;
i += run;
continue;
}
room(12);
if (s[i] >= 0x80)
{
const unsigned char lead = s[i];
// (the two-byte sequences of Latin text and the three-byte sequences of most other
// text, with continuation bytes of any value: the ranges that are restricted (E0,
// ED, and the sequences of four bytes) go through validate_one_utf8())
if (lead >= 0xE1 && lead <= 0xEF && lead != 0xED && size - i >= 3 && (s[i + 1] & 0xC0u) == 0x80 && (s[i + 2] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x0Fu) << 12u) | ((s[i + 1] & 0x3Fu) << 6u) | (s[i + 2] & 0x3Fu));
i += 3;
continue;
}
if (lead >= 0xC2 && lead <= 0xDF && size - i >= 2 && (s[i + 1] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x1Fu) << 6u) | (s[i + 1] & 0x3Fu));
i += 2;
continue;
}
const std::size_t len = validate_one_utf8(s + i, size - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
m_out.set_cursor(w);
invalid_utf8(s, size);
return; // LCOV_EXCL_LINE (invalid_utf8 throws)
}
std::uint32_t codepoint = s[i] & (0xFFu >> (len + 1));
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
i += len;
if (codepoint <= 0xFFFF)
{
u_escape(codepoint);
}
else
{
u_escape(0xD7C0u + (codepoint >> 10u));
u_escape(0xDC00u + (codepoint & 0x3FFu));
}
continue;
}
// a quote, a backslash, or a control character (or DEL)
const unsigned char c = s[i++];
char escape = 0;
switch (c)
{
case 0x08:
escape = 'b';
break;
case 0x09:
escape = 't';
break;
case 0x0A:
escape = 'n';
break;
case 0x0C:
escape = 'f';
break;
case 0x0D:
escape = 'r';
break;
case 0x22:
escape = '"';
break;
case 0x5C:
escape = '\\';
break;
default:
break;
}
if (escape != 0)
{
w[0] = '\\';
w[1] = escape;
w += 2;
}
else
{
u_escape(c);
}
}
room(1);
*w++ = '"';
m_out.set_cursor(w);
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// the copies of dump_text() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
@@ -481,7 +688,7 @@ class view_serializer
}
/*!
@brief the compact output (the default dump(), or with ensure_ascii)
@brief the output, compact (the default dump()) or indented, with or without ensure_ascii
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
@@ -489,11 +696,11 @@ class view_serializer
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot. With ensure_ascii (Ascii), a
string of the source is copied only if it has no byte >= 0x7F (see
is_ascii_without_delete()); any other string is written by write_string().
string of the source is copied only if it has nothing to escape (see
copied()); any other string is written by write_ascii().
*/
template<bool SourceNumbers, bool Ascii>
void dump_compact(const node* root)
template<bool SourceNumbers, bool Pretty, bool Ascii>
void dump_text(const node* root)
{
struct frame
{
@@ -508,6 +715,16 @@ class view_serializer
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// (indented output) a line break and 63 indentation characters, from
// which the line break of a level of up to 63 characters is a copy
std::array<char, 64> ruler; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled below if used
if (Pretty)
{
ruler.fill(m_style.indent_char);
ruler[0] = '\n';
}
const std::size_t indent = m_style.indent;
depth_counter<Pretty> level; // depth of the innermost open container (indented output)
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
@@ -539,6 +756,22 @@ class view_serializer
}
w += n;
};
// (indented output) the line break and the indentation of a level
const auto newline = [&](std::size_t depth)
{
const std::size_t count = 1 + (depth * indent);
room(count);
if (count <= ruler.size())
{
std::memcpy(w, ruler.data(), ruler.size());
}
else
{
*w = '\n';
std::memset(w + 1, m_style.indent_char, count - 1);
}
w += count;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
@@ -546,11 +779,11 @@ class view_serializer
m_out.set_cursor(w);
if (enabled(Ascii))
{
write_ascii_string(n);
write_ascii(n);
}
else
{
write_decoded(n);
write_decoded(n, Pretty);
}
w = m_out.cursor();
lim = m_out.limit();
@@ -651,6 +884,7 @@ class view_serializer
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
level.enter();
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
@@ -693,6 +927,10 @@ class view_serializer
{
break;
}
if (Pretty)
{
newline(level.leave());
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
@@ -701,13 +939,17 @@ class view_serializer
room(1);
*w++ = ',';
}
if (Pretty)
{
newline(level.get());
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if (copied(key, src, std::integral_constant<bool, Ascii>()))
{
room(key.len + 3);
room(key.len + (Pretty ? 4 : 3));
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
@@ -717,9 +959,13 @@ class view_serializer
else
{
escaped(key);
room(1);
room(Pretty ? 2 : 1);
*w++ = ':';
}
if (Pretty)
{
*w++ = ' ';
}
if (Editable)
{
n = nav::value(at + 1);
@@ -738,69 +984,6 @@ class view_serializer
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
{
m_out.put('\n');
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
}
}
void write_scalar(const node& n)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
m_out.put("null");
break;
case value_t::boolean:
if ((n.flags & node_flags::is_true) != 0)
{
m_out.put("true");
}
else
{
m_out.put("false");
}
break;
case value_t::string:
write_string(n);
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const char* const token = m_doc.str(n);
const std::uint32_t len = number_length(n);
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
{
m_out.put('0'); // parse() reads -0 as the integer 0
}
else
{
m_out.put(token, len);
}
break;
}
case value_t::number_float:
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
{
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
}
else
{
write_float_node(n);
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
case value_t::array: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
}
/// a float node as dump() writes it
void write_float_node(const node& n)
{
@@ -911,72 +1094,6 @@ class view_serializer
m_out.put(s.data(), s.size());
}
void write_string(const node& n)
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::storage) == 0 && (!m_style.ensure_ascii || is_ascii_without_delete(s, n.len)))
{
// a string of the source without escape sequences has nothing to
// escape (with ensure_ascii, if it is printable ASCII)
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
{
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
else
{
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
m_out.put('"');
}
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(s + i, n - i);
}
else if (is_ascii_copyable(s[i]))
{
run = find_ascii_copyable_run(s + i, n - i);
}
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0x80)
{
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
invalid_utf8(s, n);
return;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
}
write_codepoint<EnsureAscii>(codepoint, s + i, len);
i += len;
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
+28
View File
@@ -116,6 +116,34 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned
}
return p;
}
/*!
@brief copy 16 bytes from p to w, and find the first quote, backslash,
control character, or non-ASCII byte among them
The bytes are copied whether or not they are all plain (the caller has room
for 16 bytes and keeps only the plain ones).
@return the offset of that byte, or 16 if there is none
*/
NLOHMANN_VIEW_ALWAYS_INLINE unsigned copy_plain_run16(const unsigned char* p, char* w) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
vst1q_u8(reinterpret_cast<std::uint8_t*>(w), in); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits) >> 2u) : 16u;
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
_mm_storeu_si128(static_cast<__m128i*>(static_cast<void*>(w)), in); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
return bits != 0 ? static_cast<unsigned>(count_trailing_zeros(bits)) : 16u;
#endif
}
#endif
#if NLOHMANN_VIEW_SSSE3_DISPATCH
+5 -3
View File
@@ -159,9 +159,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader;
template<typename BasicJsonType, typename InputAdapterType>
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
friend class ::nlohmann::detail::json_sax_dom_parser;
template<typename BasicJsonType, typename InputAdapterType>
friend class ::nlohmann::detail::json_sax_dom_binary_parser;
template<typename BasicJsonType, typename InputAdapterType>
friend class ::nlohmann::detail::json_sax_dom_callback_parser;
#if JSON_DIAGNOSTIC_POSITIONS
friend struct ::nlohmann::detail::diagnostic_positions;
@@ -6288,7 +6290,7 @@ public:
from_ubjson/from_bjdata/from_bon8/from_bson overloads
Building the @ref binary_reader as a named local, rather than as a
temporary that @ref detail::json_sax_dom_parser::parse is called on in
temporary that @ref detail::json_sax_dom_binary_parser::parse is called on in
the same expression, avoids a false-positive cppcheck accessMoved
warning in each of the 16 callers.
*/
@@ -6299,7 +6301,7 @@ public:
const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error)
{
basic_json result;
detail::json_sax_dom_parser<basic_json, InputAdapterType> sdp(result, allow_exceptions);
detail::json_sax_dom_binary_parser<basic_json, InputAdapterType> sdp(result, allow_exceptions);
binary_reader<InputAdapterType> reader(std::move(ia), format, error_handler);
if (!reader.sax_parse(&sdp, strict, tag_handler))
{
+84 -10
View File
@@ -30,6 +30,7 @@
#include <array> // array
#include <cstddef> // nullptr_t, size_t // IWYU pragma: keep
#include <cstdint> // uint8_t, uint32_t
#include <cstdio> // FILE
#include <cstring> // memcpy, strlen
#include <istream> // istream // IWYU pragma: keep
#include <iterator> // distance, input_iterator_tag, iterator_traits
@@ -295,7 +296,7 @@ class basic_json_view
}
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
return idx < m_node->len ? element(idx) : basic_json_view();
}
/// any other integer type (int, unsigned, long, std::int64_t, ...; a
@@ -358,7 +359,7 @@ class basic_json_view
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
return element(idx);
}
/// any other integer type, see operator[]; negative values are out of
@@ -392,7 +393,13 @@ class basic_json_view
string_t value(string_view_t key, const char* default_value) const
{
return value(key, string_t(default_value));
if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
{
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
// (the default is made only if there is no such member)
const basic_json_view r = lookup(key);
return r.is_discarded() ? string_t(default_value) : r.template get<string_t>();
}
/// the value a JSON pointer refers to converted to T, or the default
@@ -411,7 +418,12 @@ class basic_json_view
string_t value(const json_pointer& ptr, const char* default_value) const
{
return value(ptr, string_t(default_value));
if (NLOHMANN_VIEW_UNLIKELY(!is_structured()))
{
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
const basic_json_view r = detail::view::resolve_pointer(*this, detail::json_pointer_access::reference_tokens(ptr), detail::view::pointer_mode::value);
return r.is_discarded() ? string_t(default_value) : r.template get<string_t>();
}
/// the first element or member value; a primitive value itself. Throws
@@ -433,7 +445,8 @@ class basic_json_view
{
if (is_structured() && m_node->len != 0)
{
return basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
return unedited() ? basic_json_view(m_doc, detail::view::last_child<false>(*m_doc, m_node) + (is_object() ? 1 : 0))
: basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
}
return front();
}
@@ -450,7 +463,7 @@ class basic_json_view
{
return end();
}
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
const node* const k = find_key(key);
return k != nullptr ? iterator(m_doc, k, true) : end();
}
@@ -467,7 +480,7 @@ class basic_json_view
/// whether this is an object with a member with this key
bool contains(string_view_t key) const
{
return is_object() && detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr;
return is_object() && find_key(key) != nullptr;
}
bool contains(const char* key) const
@@ -627,7 +640,14 @@ class basic_json_view
// the compact writer keeps 64 bytes of slack, so that it does not grow
// the buffer just before the end
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 160;
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
if (unedited())
{
detail::view::view_serializer<BasicJsonType, false>(*m_doc, out, estimate, style).dump(m_node);
}
else
{
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
}
return out;
}
@@ -707,7 +727,7 @@ class basic_json_view
{
return BasicJsonType(value_t::discarded);
}
return detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
return unedited() ? detail::view::materialize<BasicJsonType, false>(*m_doc, m_node) : detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
}
/// byte offset of this value in the source text (for strings: of the
@@ -785,10 +805,50 @@ class basic_json_view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
{
if (unedited())
{
const node* const k = detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, k + 1) : basic_json_view();
}
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, navigation::value(k + 1)) : basic_json_view();
}
/// whether the document has not been edited yet: until the first edit, its index is walked like that of a
/// read-only document (nothing is moved or linked), which is cheaper
NLOHMANN_VIEW_ALWAYS_INLINE bool unedited() const noexcept
{
return !navigation::editable() || m_doc->edits == nullptr;
}
/// the member of this object with @a key (the last one if the key occurs more than once), or nullptr
NLOHMANN_VIEW_ALWAYS_INLINE const node* find_key(string_view_t key) const
{
return find_key(key, std::integral_constant<bool, Editable> {});
}
NLOHMANN_VIEW_ALWAYS_INLINE const node* find_key(string_view_t key, std::true_type /*editable*/) const
{
// (an editable document without edits is looked up as one that cannot be edited)
return unedited() ? detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size())
: detail::view::find_member<true>(*m_doc, m_node, key.data(), key.size());
}
NLOHMANN_VIEW_ALWAYS_INLINE const node* find_key(string_view_t key, std::false_type /*editable*/) const
{
return detail::view::find_member<false>(*m_doc, m_node, key.data(), key.size());
}
/// the element of an array at an index below its size
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view element(size_type idx) const
{
if (unedited())
{
return basic_json_view(m_doc, detail::view::element_at<false>(*m_doc, m_node, idx));
}
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
// --- get() dispatch ---
bool get_impl(detail::view::value_tag<bool> /*unused*/, detail::priority_tag<2> /*unused*/) const
@@ -1528,7 +1588,8 @@ class basic_json_document
// NOLINTBEGIN(readability-avoid-nested-conditional-operator): a constant expression of C++11
append_input_impl(buf, std::forward<T>(input), std::integral_constant < int, detail::is_contiguous_byte_container<D>::value ? 0
: std::is_base_of<std::istream, D>::value ? 1
: detail::view::is_adapter_input<D>::value ? 2 : 3 > {});
: detail::view::is_adapter_input<D>::value ? 2
: detail::view::is_c_stream<D>::value ? 4 : 3 > {});
// NOLINTEND(readability-avoid-nested-conditional-operator)
}
@@ -1550,6 +1611,19 @@ class basic_json_document
detail::view::append_adapter(buf, std::move(input.ia));
}
#ifndef JSON_NO_IO
static void append_input_impl(std::string& buf, std::FILE* file, std::integral_constant<int, 4> /*C stream*/)
{
if (file == nullptr)
{
// (as before: the library's adapter decides what a null stream yields)
detail::view::append_adapter(buf, detail::input_adapter(file));
return;
}
detail::view::append_file(buf, file);
}
#endif
template<typename T>
static void append_input_impl(std::string& buf, T&& input, std::integral_constant<int, 3> /*other*/)
{
+1141 -72
View File
File diff suppressed because it is too large. Load diff
+849 -339
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -60,6 +60,7 @@ inline namespace json_literals
namespace detail
{
using NLOHMANN_JSON_NAMESPACE::detail::json_sax_dom_callback_parser;
using NLOHMANN_JSON_NAMESPACE::detail::json_sax_dom_parser;
using NLOHMANN_JSON_NAMESPACE::detail::unknown_size;
} // namespace detail
+78
View File
@@ -2952,3 +2952,81 @@ TEST_CASE("diagnostic positions: value lifetime, input adapters, and SAX")
}
}
#endif
TEST_CASE("accept() agrees with parse() on floats around the overflow limit")
{
// accept() does not convert a float that cannot overflow; the tokens
// around the limit (and beyond it) must still be rejected exactly as
// parse() rejects them
const std::vector<const char*> numbers =
{
"1e0", "123.456e-5", "1e-400", "-1e-400", "1e307", "1e308", "1e309", "-1e309",
"10e307", "100e307", "0.1e309", "0.1e310", "1E+308", "1E+309",
"1.7976931348623157e308", "1.7976931348623159e308", "1.8e308", "-1.7976931348623157e308",
"12345678901234567e291", "12345678901234567e292", "123456789012345678e290", "123456789012345678e291",
"1234567890123456789e289", "1234567890123456789e290", "9999999999999999999e289", "9999999999999999999e290",
"0.00000000000000000001e328", "0.00000000000000000001e329"
};
for (const char* number : numbers)
{
for (const std::string& text :
{
std::string(number), "[" + std::string(number) + "]", "{\"a\":" + std::string(number) + "}"
})
{
INFO(text);
const bool parses = !json::parse(text, nullptr, false).is_discarded();
CHECK(json::accept(text) == parses);
}
}
}
namespace
{
// a DOM parser that reports parse errors without throwing, as in the sax_no_exception example
template<typename BasicJsonType>
class quiet_dom_parser : public nlohmann::detail::json_sax_dom_parser<BasicJsonType, nlohmann::detail::string_input_adapter_type>
{
public:
explicit quiet_dom_parser(BasicJsonType& j)
: nlohmann::detail::json_sax_dom_parser<BasicJsonType, nlohmann::detail::string_input_adapter_type>(j, false)
{}
bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const nlohmann::detail::exception& /*unused*/)
{
return false;
}
};
template<typename BasicJsonType>
std::string read_before_error(const std::string& text)
{
BasicJsonType j;
quiet_dom_parser<BasicJsonType> sax(j);
CHECK(!BasicJsonType::sax_parse(text, &sax));
return j.dump();
}
} // namespace
TEST_CASE("a json_sax_dom_parser keeps the values read before a parse error")
{
// a json_sax_dom_parser that a user constructs stores every value in place, so a
// handler that does not throw on a parse error sees what was read before it
const std::vector<std::pair<std::string, std::string>> cases =
{
{"[1,2,3,]", "[1,2,3]"},
{"[[1,[2,3],4],[5,", "[[1,[2,3],4],[5]]"},
{R"({"a":1,"b":[1,2,{"c":)", R"({"a":1,"b":[1,2,{"c":null}]})"},
{R"({"a":1,"a":)", R"({"a":1})"},
{"[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,]", "[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19]"},
{R"({"k0":0,"k1":1,"k2":{"x":[true,)", R"({"k0":0,"k1":1,"k2":{"x":[true]}})"},
{R"([{},[],{"a":[]},x)", R"([{},[],{"a":[]}])"},
};
for (const auto& c : cases)
{
CAPTURE(c.first)
CHECK(read_before_error<json>(c.first) == c.second);
CHECK(read_before_error<nlohmann::ordered_json>(c.first) == c.second);
}
}
+127
View File
@@ -185,6 +185,33 @@ TEST_CASE("json_view")
// duplicate keys: the last value, at the position of the first key
CHECK(materialized<json_document>(R"({"a":1,"b":2,"a":3})") == json::parse(R"({"a":1,"b":2,"a":3})"));
CHECK(materialized<ordered_json_document>(R"({"a":1,"b":2,"a":3})").dump() == R"({"a":3,"b":2})");
// scalars (made without the handler), and nesting around the depth up to which the stack of open containers is local
for (const char* scalar :
{"null", "true", "false", "0", "-7", "18446744073709551615", "1.5e300", "-0.0", "\"\"", "\"short\"", "\"a string that is longer than fifteen bytes\"", R"("\u00e4\n")"
})
{
CAPTURE(scalar)
CHECK(materialized<json_document>(scalar) == json::parse(scalar));
const std::string in_array = std::string("[1,") + scalar + "]";
const json_document d = json_document::parse(in_array);
CHECK(d.root()[1].materialize() == json::parse(scalar));
CHECK(d.root()[1].get<json>() == json::parse(scalar));
}
for (const std::size_t levels :
{
std::size_t{1}, std::size_t{15}, std::size_t{16}, std::size_t{17}, std::size_t{32}, std::size_t{33}
})
{
CAPTURE(levels)
const std::string nested = std::string(levels, '[') + R"({"k":[1,{"j":2}]})" + std::string(levels, ']');
CHECK(materialized<json_document>(nested) == json::parse(nested));
std::string wide = R"({"a":)";
wide += nested;
wide += R"(,"b":)";
wide += nested;
wide += "}";
CHECK(materialized<json_document>(wide) == json::parse(wide));
}
// very deep nesting (iterative, as parse())
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
// (one check per step, so that a failure shows which step it is)
@@ -410,6 +437,31 @@ TEST_CASE("json_view")
std::istringstream rvalue_source("[1]");
CHECK(json_document::parse_copy(std::move(rvalue_source)).owns_source());
// a C stream is read in blocks as well (more than one block, and a
// text whose length is a multiple of the block size)
for (const std::size_t extra :
{
std::size_t{0}, std::size_t{1}
})
{
std::FILE* long_file = std::tmpfile(); // NOLINT(cppcoreguidelines-owning-memory)
if (long_file != nullptr)
{
std::string padded = text;
padded.append(((4096 - (padded.size() % 4096)) % 4096) + extra, ' ');
CHECK(std::fwrite(padded.data(), 1, padded.size(), long_file) == padded.size());
CHECK(std::fseek(long_file, 0, SEEK_SET) == 0);
const json_document from_long_file = json_document::parse(long_file);
CHECK(from_long_file.source() == padded);
CHECK(from_long_file.root().materialize() == json::parse(text));
CHECK(std::fclose(long_file) == 0); // NOLINT(cppcoreguidelines-owning-memory)
}
}
{
json_document from_null;
CHECK_THROWS_AS(from_null = json_document::parse(static_cast<std::FILE*>(nullptr)), json::parse_error&);
}
// a stream without a buffer is an empty input
std::istream no_buffer(nullptr);
CHECK(json_document::parse(no_buffer, false).is_discarded());
@@ -1006,6 +1058,49 @@ TEST_CASE("json_view element access and iteration")
#endif
}
SECTION("arrays whose elements have no subtrees: index and back()")
{
// (the elements are adjacent in the index; an array, an object, or an
// empty container anywhere makes the lookup skip element by element)
const std::vector<std::string> texts =
{
"[]", "[1]", "[1,2,3]", "[1,\"two\",3.5,true,null,-7,18446744073709551615]", "[[]]", "[{}]", "[1,[2],3]",
"[1,2,{\"a\":3}]", "[[1,2],3,4]", "[1,2,3,[4,5,6,7,8],9,10,11,12]", "[{\"a\":[1]},2]", R"(["a\u00e4","b"])"
};
for (const std::string& text : texts)
{
CAPTURE(text)
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json expected = json::parse(text);
for (std::size_t i = 0; i < expected.size(); ++i)
{
CHECK(v[i].materialize() == expected[i]);
CHECK(v.at(i).materialize() == expected[i]);
}
CHECK(v[expected.size()].is_discarded());
if (!expected.empty())
{
CHECK(v.back().materialize() == expected.back());
}
}
// a long array (the direct lookup is not limited to short ones), also with a container at its end
std::string text = "[0";
for (int i = 1; i < 1000; ++i)
{
text += "," + std::to_string(i);
}
const json_document scalars = json_document::parse(text + "]");
CHECK(scalars.root()[999].get<int>() == 999);
CHECK(scalars.root()[500].get<int>() == 500);
CHECK(scalars.root().at(1).get<int>() == 1);
CHECK(scalars.root().back().get<int>() == 999);
const json_document container_last = json_document::parse(text + ",[1000]]");
CHECK(container_last.root()[999].get<int>() == 999);
CHECK(container_last.root()[1000][0].get<int>() == 1000);
CHECK(container_last.root().back()[0].get<int>() == 1000);
}
SECTION("iterators")
{
const json_document d = json_document::parse(R"({"x":[1,{"y":2}],"z":null})");
@@ -1045,6 +1140,28 @@ TEST_CASE("json_view element access and iteration")
CHECK((*first++).key() == "0");
CHECK((*first).key() == "1");
CHECK(++first == items.end());
// the keys of array elements are their indexes, whether or not they are asked for
std::string numbers = "[0";
for (int i = 1; i < 1234; ++i)
{
numbers += "," + std::to_string(i);
}
const json_document array = json_document::parse(numbers + "]");
std::size_t position = 0;
for (const auto& item : array.root().items())
{
CHECK(item.value().get<std::size_t>() == position);
if (position % 3 != 0) // (not for every item: values alone do not make the key)
{
CHECK(item.key() == std::to_string(position));
CHECK(item.key().data() == item.key().data()); // (the same text every time)
const json_view::item copy = item;
CHECK(copy.key() == std::to_string(position));
}
++position;
}
CHECK(position == 1234);
#ifdef JSON_HAS_CPP_17
std::string pairs;
for (const auto [key, value] : d.root().items())
@@ -1383,7 +1500,17 @@ TEST_CASE("json_view values")
CHECK(exception_of([&] { static_cast<void>(v.value("s", 0)); }) == without_path(exception_of([&] { static_cast<void>(j.value("s", 0)); })));
CHECK(exception_of([&] { static_cast<void>(v["n"].value("x", 0)); }) == without_path(exception_of([&] { static_cast<void>(j["n"].value("x", 0)); })));
CHECK(exception_of([&] { static_cast<void>(v["n"].value(json::json_pointer("/x"), 0)); }) == without_path(exception_of([&] { static_cast<void>(j["n"].value(json::json_pointer("/x"), 0)); })));
// string defaults: the member is not a string, or the value is neither an object nor an array
CHECK(exception_of([&] { static_cast<void>(v.value("n", "default")); }) == without_path(exception_of([&] { static_cast<void>(j.value("n", "default")); })));
CHECK(exception_of([&] { static_cast<void>(v["n"].value("x", "default")); }) == without_path(exception_of([&] { static_cast<void>(j["n"].value("x", "default")); })));
CHECK(exception_of([&] { static_cast<void>(v.value(json::json_pointer("/n"), "default")); }) == without_path(exception_of([&] { static_cast<void>(j.value(json::json_pointer("/n"), "default")); })));
CHECK(exception_of([&] { static_cast<void>(v["s"].value(json::json_pointer("/x"), "default")); }) == without_path(exception_of([&] { static_cast<void>(j["s"].value(json::json_pointer("/x"), "default")); })));
#endif
// (defaults and members longer than the short string optimization)
CHECK(v.value("s", "a default longer than fifteen characters") == "text");
CHECK(v.value("missing", "a default longer than fifteen characters") == "a default longer than fifteen characters");
CHECK(v.value(json::json_pointer("/s"), "a default longer than fifteen characters") == "text");
CHECK(v.value(json::json_pointer("/zz"), "a default longer than fifteen characters") == "a default longer than fifteen characters");
}
}
+52
View File
@@ -523,6 +523,58 @@ TEST_CASE("json_view builder: strings across vector blocks")
}
}
TEST_CASE("json_view builder: escapes between runs of text")
{
// The text between escapes is copied 16 bytes at a time where there are
// that many bytes left, and with the UTF-8 check where there is
// non-ASCII text. Runs of every length from 0 to 40 between valid and
// invalid escapes, UTF-8 sequences, and control characters, in strings
// that are cut at all places near the end of the text.
const std::array<const char*, 9> valid = {{"\\n", "\\\"", "\\\\", "\\/", "\\t", "\\u00e4", "\\u20ac", "\\ud83d\\ude00", "\\u0000"}};
const std::array<const char*, 9> invalid = {{"\\x", "\\ud800", "\\udc00", "\\u12G4", "\\u12", "\\ud83dx", "\\ud83d\\u0041", "\\", "\x01"}};
const std::array<const char*, 6> utf8 = {{"\xc3\xa4", "\xe2\x82\xac", "\xf0\x9f\x98\x80", "\xc3", "\xff", "\x80"}};
std::mt19937 rng(2026); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
for (std::size_t run = 0; run <= 40; ++run)
{
for (int i = 0; i < 60; ++i)
{
std::string content = std::string(rng() % 20, 'a');
const auto pieces = 1 + (rng() % 6);
for (unsigned k = 0; k < pieces; ++k)
{
content += valid[rng() % valid.size()];
content += std::string(k == 0 ? run : rng() % 41, static_cast<char>('b' + (rng() % 20)));
switch (rng() % 8)
{
case 0:
content += utf8[rng() % 3];
break;
case 1:
content += utf8[rng() % utf8.size()];
break;
case 2:
content += invalid[rng() % invalid.size()];
break;
default:
break;
}
}
check_string(content);
check_string(content.substr(0, content.size() - (rng() % 4)));
}
}
// the text before the first escape, and runs and escapes at the end of the text
for (std::size_t before = 0; before <= 40; ++before)
{
for (const char* escape : valid)
{
check_string(std::string(before, 'a') + escape);
check_string(std::string(before, 'a') + escape + std::string(before % 19, 'z'));
check_string(std::string(before, 'a') + escape + escape + std::string(before % 19, 'z'));
}
}
}
TEST_CASE("json_view node integer bits")
{
using nlohmann::detail::view::integer_bits;
+275
View File
@@ -21,6 +21,7 @@ using nlohmann::ordered_json_view;
using json_view_test::generator;
using json_view_test::has_duplicate_keys;
#include <algorithm>
#include <array>
#include <cmath>
#include <cstddef>
@@ -29,6 +30,7 @@ using json_view_test::has_duplicate_keys;
#include <iomanip>
#include <map>
#include <random>
#include <set>
#include <sstream>
#include <string>
#include <vector>
@@ -186,6 +188,133 @@ TEST_CASE("json_view dump")
CHECK(deep_document.root().dump() == deep);
}
SECTION("ensure_ascii: a character that needs an escape at each position of a string")
{
// string lengths around the width of the vector steps, in a string value, a key, and a value near the end of the text
for (const char* const special :
{
"\xC3\xA9", "\xE4\xB8\xAD", "\xF0\x9F\x98\x80", "\x7F", "\\n", "\\u0001", "\\\"", "\\\\"
})
{
for (std::size_t length = 0; length <= 40; ++length)
{
for (std::size_t position = 0; position <= length; ++position)
{
std::string content(length, 'a');
content.insert(position, special);
const std::array<std::string, 3> texts =
{
{
"\"" + content + "\"",
"[\"" + content + "\"]",
"{\"" + content + "\":1}",
}
};
for (const std::string& text : texts)
{
CAPTURE(text)
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
}
}
}
}
}
SECTION("ensure_ascii: every character of the Basic Multilingual Plane and some beyond")
{
// (the two- and three-byte sequences, including the ranges that need a check: E0, ED, EF BF BF)
std::string text = "[";
std::size_t in_chunk = 0;
const auto append = [&](std::uint32_t cp)
{
if (cp < 0x80)
{
if (cp < 0x20 || cp == '"' || cp == '\\')
{
return;
}
text += static_cast<char>(cp);
}
else if (cp < 0x800)
{
text += static_cast<char>(0xC0 | (cp >> 6));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else if (cp < 0x10000)
{
text += static_cast<char>(0xE0 | (cp >> 12));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else
{
text += static_cast<char>(0xF0 | (cp >> 18));
text += static_cast<char>(0x80 | ((cp >> 12) & 0x3F));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
};
for (std::uint32_t cp = 0x20; cp < 0x10400; ++cp)
{
if (cp >= 0xD800 && cp < 0xE000)
{
continue;
}
if (in_chunk == 0)
{
text += text.size() > 1 ? ",\"" : "\"";
}
append(cp);
if (++in_chunk == 37)
{
text += "\"";
in_chunk = 0;
}
}
text += in_chunk != 0 ? "\"]" : "]";
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root().dump(2) == j.dump(2));
}
SECTION("deep nesting, indented")
{
// indentations of up to and beyond the width of the writer's fixed-size copy
for (const int depth :
{
1, 5, 20, 70, 300
})
{
std::string text;
for (int i = 0; i < depth; ++i)
{
text += i % 2 == 0 ? "[1," : R"({"k":2,"\u00e9":)";
}
text += "[]";
for (int i = depth - 1; i >= 0; --i)
{
text += i % 2 == 0 ? "]" : "}";
}
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
for (const int indent :
{
0, 1, 2, 3, 8
})
{
CHECK(d.root().dump(indent) == j.dump(indent));
CHECK(d.root().dump(indent, '\t') == j.dump(indent, '\t'));
}
}
}
SECTION("the output buffer of a small value is small")
{
// an escaped key after the value: its node lies in the arena, so the
@@ -598,6 +727,152 @@ TEST_CASE("json_view large objects")
CHECK(without_collisions.memory_usage() >= with_collisions.memory_usage() + (table_size * sizeof(std::uint32_t)));
}
SECTION("keys that differ in their last bytes are spread over the table")
{
// The hash must spread the usual families of keys (numbered names, hex
// strings, dates, short names): a key farther than index_max_displacement
// from its home slot would leave the object without a table. The keys
// are placed with linear probing, as the table does.
const auto max_displacement = [](const std::vector<std::string>& keys)
{
std::size_t slots = 16;
while (slots < 2 * keys.size())
{
slots *= 2;
}
std::vector<bool> used(slots, false);
std::size_t farthest = 0;
for (const std::string& key : keys)
{
std::size_t i = nlohmann::detail::conditional_static_cast<std::size_t>(nlohmann::detail::view::key_hash(key.data(), key.size())) & (slots - 1);
std::size_t distance = 0;
while (used[i])
{
i = (i + 1) & (slots - 1);
++distance;
}
used[i] = true;
farthest = (std::max)(farthest, distance);
}
return farthest;
};
// the keys of a printf-like pattern with one %d, %0Nd or %0Nx, formatted with a stream
// (a format string that is not a literal is a Flawfinder error)
const auto numbered = [](const char* format, int count, int step)
{
const std::string pattern(format);
const std::size_t percent = pattern.find('%');
std::size_t conversion = percent + 1;
std::size_t width = 0;
while (pattern[conversion] >= '0' && pattern[conversion] <= '9')
{
width = (width * 10) + static_cast<std::size_t>(pattern[conversion] - '0');
++conversion;
}
const bool hex = pattern[conversion] == 'x';
std::vector<std::string> keys;
for (int i = 0; i < count; ++i)
{
std::ostringstream key;
key << pattern.substr(0, percent) << std::setfill('0') << std::setw(static_cast<int>(width))
<< (hex ? std::hex : std::dec) << (i * step) << pattern.substr(conversion + 1);
keys.push_back(key.str());
}
return keys;
};
for (const char* format :
{"key%d", "%d", "item_%04d", "a%05d", "%03dxyz", "id%d", "user_%d", "%d_suffix", "0x%04x", "A%07d", "12345678%d", "1234567%d", "123456789%d",
"configuration_parameter_value_%04d_extra", "2026-01-%02dT00:00:00Z"
})
{
CAPTURE(format)
CHECK(max_displacement(numbered(format, std::string(format).find("%02d") != std::string::npos ? 60 : 1000, 1)) < nlohmann::detail::view::index_max_displacement); // NOLINT(abseil-string-find-str-contains)
CHECK(max_displacement(numbered(format, 600, 7)) < nlohmann::detail::view::index_max_displacement);
}
std::mt19937_64 rng(7); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
for (const std::size_t length :
{
std::size_t{1}, std::size_t{2}, std::size_t{3}, std::size_t{5}, std::size_t{7}, std::size_t{8}, std::size_t{9}, std::size_t{12}, std::size_t{16}, std::size_t{17}, std::size_t{33}
})
{
CAPTURE(length)
std::set<std::string> distinct;
// (there are 26 and 676 keys of one and two letters)
std::size_t wanted = 700;
if (length == 1)
{
wanted = 26;
}
else if (length == 2)
{
wanted = 500;
}
while (distinct.size() < wanted)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
distinct.insert(key);
}
CHECK(max_displacement(std::vector<std::string>(distinct.begin(), distinct.end())) < nlohmann::detail::view::index_max_displacement);
}
}
SECTION("keys of every length that differ in one byte")
{
// In a large object, a key is found by comparing its bytes with those of the member the
// table points to: up to 16 bytes as two overlapping words, longer ones with memcmp.
// Keys of every length from 0 to 40, and keys that differ from them in one byte.
std::mt19937_64 rng(2026); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
std::vector<std::string> keys;
std::set<std::string> distinct;
for (std::size_t length = 0; length <= 40; ++length)
{
for (int copy = 0; copy < 5; ++copy)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
if (distinct.insert(key).second)
{
keys.push_back(key);
}
}
}
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i);
}
text += "}";
REQUIRE(keys.size() >= 128); // (a table is built for 128 members)
const json_document d = json_document::parse(text);
const json_view v = d.root();
for (std::size_t i = 0; i < keys.size(); ++i)
{
CAPTURE(keys[i])
CHECK(v[keys[i]].get<std::size_t>() == i);
CHECK(v.at(keys[i]).get<std::size_t>() == i);
CHECK(v.contains(keys[i]));
for (std::size_t at = 0; at < keys[i].size(); ++at)
{
std::string other = keys[i];
other[at] = other[at] == 'z' ? 'y' : 'z';
CAPTURE(other)
CHECK(v.contains(other) == (distinct.count(other) != 0));
}
CHECK(v.contains(keys[i] + "a") == (distinct.count(keys[i] + "a") != 0));
if (!keys[i].empty())
{
CHECK(v.contains(keys[i].substr(1)) == (distinct.count(keys[i].substr(1)) != 0));
}
}
}
SECTION("shrink_to_fit releases the tables' spare capacity")
{
const auto make_text = [](int objects, int members)
+109
View File
@@ -661,6 +661,115 @@ TEST_CASE("json_view edits: replacing arrays and objects by scalars")
CHECK(d.root().materialize() == expected);
}
TEST_CASE("json_view edits: arrays whose elements have no subtrees")
{
// such an array is indexed directly while it is in its parsed layout; edits
// that replace elements by scalars keep it, others move it
json_editable_document d = json_editable_document::parse("[10,20,30,40]");
json expected = json::array({10, 20, 30, 40});
const auto check = [&]
{
CHECK(d.root().size() == expected.size());
for (std::size_t i = 0; i < expected.size(); ++i)
{
CHECK(d.root()[i].materialize() == expected[i]);
CHECK(d.root().at(i).materialize() == expected[i]);
}
CHECK(d.root()[expected.size()].is_discarded());
CHECK(d.root().back().materialize() == expected.back());
CHECK(d.root().dump() == expected.dump());
};
check();
d.set(d.root()[1], "x");
expected[1] = "x";
check();
d.set(d.root()[3], 2.5);
expected[3] = 2.5;
check();
d.set(d.root()[2], json::array({1, 2}));
expected[2] = json::array({1, 2});
check();
d.push_back(d.root(), 50);
expected.push_back(50);
check();
d.insert(d.root(), 0, true);
expected.insert(expected.begin(), true);
check();
d.erase(d.root(), 1);
expected.erase(expected.begin() + 1);
check();
d.set(d.root()[2], nullptr);
expected[2] = nullptr;
check();
}
TEST_CASE("json_view edits: materializing values nested deeper than the document was parsed")
{
// the stack of open containers of materialize() is sized from the nesting of the parsed text;
// a value added by an edit can be deeper than that (and than the stack that is local to the call)
json_editable_document d = json_editable_document::parse("[1,[2]]");
json expected = json::array({1, json::array({2})});
json nested = 3;
for (int level = 0; level < 40; ++level)
{
nested = json::array({nested, json{{"k", level}}});
}
d.push_back(d.root(), nested);
expected.push_back(nested);
CHECK(d.root().materialize() == expected);
CHECK(d.root()[2].materialize() == nested);
d.set(d.root()[1][0], nested);
expected[1][0] = nested;
CHECK(d.root().materialize() == expected);
CHECK(d.root().get<json>() == expected);
}
TEST_CASE("json_view edits: an editable document before and after its first edit")
{
// until the first edit, an editable document is read like a read-only one; views made before the edit
// keep working after it
const std::string text = R"({"a":[1,2,{"x":"y"},4],"b":{"c":true,"d":null},"e":"text","f":[]})";
const json_document read_only = json_document::parse(text);
json_editable_document d = json_editable_document::parse(text);
const auto root = d.root();
const auto same_as_read_only = [&]
{
CHECK(root.dump() == read_only.root().dump());
CHECK(root.dump(2) == read_only.root().dump(2));
CHECK(root.materialize() == read_only.root().materialize());
CHECK(root["a"][2]["x"].get_string() == "y");
CHECK(root["a"].at(3).get<int>() == 4);
CHECK(root["a"].back().get<int>() == 4);
CHECK(root.contains("b"));
CHECK(!root.contains("zz"));
CHECK(root.find("e")->get_string() == "text");
CHECK(root.find("zz") == root.end());
CHECK(root["b"]["c"].get<bool>());
CHECK(root.value("e", "none") == "text");
std::size_t count = 0;
for (const auto m : root["a"])
{
CHECK(m.materialize() == read_only.root()["a"][count].materialize());
++count;
}
CHECK(count == 4);
};
same_as_read_only();
// an edit that changes a scalar in place, then one that moves an array
d.set(root["e"], "other");
CHECK(root["e"].get_string() == "other");
CHECK(root["a"][2]["x"].get_string() == "y");
CHECK(root["a"].back().get<int>() == 4);
d.push_back(root["f"], 7);
CHECK(root["f"][0].get<int>() == 7);
CHECK(root["f"].back().get<int>() == 7);
CHECK(root.dump() == R"({"a":[1,2,{"x":"y"},4],"b":{"c":true,"d":null},"e":"other","f":[7]})");
d.set(root["a"], 2, json::array({1, 2}));
CHECK(root["a"][2][1].get<int>() == 2);
CHECK(root["a"].dump() == "[1,2,[1,2],4]");
CHECK(root.materialize() == json::parse(R"({"a":[1,2,[1,2],4],"b":{"c":true,"d":null},"e":"other","f":[7]})"));
}
namespace
{
// reads of "a" in an object that repeats it: the first member wins, however the
+20
View File
@@ -682,6 +682,26 @@ TEST_CASE("json_view images: ownership")
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view images: errors")
{
SECTION("invalid UTF-8 of a decoded string: the compact dump writes it, the others throw")
{
const json_document d = json_document::parse(R"({"k\n":"\u00e9x"})");
std::vector<std::uint8_t> image = d.save();
const std::size_t arena = text_at(image) + header_size_field(image, 16) + 1;
image[arena + 3] = 0x28; // the key is "k\n"; the value starts at 3: 0xC3 0x28 is invalid
const json_document loaded = json_document::load(image, image_check::bounds);
CHECK(loaded.root().dump().size() == 13);
for (const int indent :
{
0, 2
})
{
CHECK_THROWS_AS(loaded.root().dump(indent), json::type_error);
CHECK_THROWS_AS(loaded.root().dump(indent, ' ', true), json::type_error);
}
CHECK_THROWS_AS(loaded.root().dump(-1, ' ', true), json::type_error);
CHECK_THROWS_WITH_AS(loaded.root().dump(2), "[json.exception.type_error.316] invalid UTF-8 byte at index 1: 0x28", json::type_error&);
}
SECTION("a literal as the root: dump() after loading")
{
for (const char* text :
+278
View File
@@ -12,6 +12,11 @@
using nlohmann::json;
using nlohmann::ordered_json;
#include <array>
#include <cstddef>
#include <cstdint>
#include <string>
TEST_CASE("ordered_json")
{
json j;
@@ -237,3 +242,276 @@ TEST_CASE("regression test for issue #5673 - ordered_json::emplace with a non-rv
CHECK(oj.dump() == "{\"a\":1}");
}
}
namespace
{
// the sizes around which the DOM parser changes how it stores the elements of
// an array (16) and the members of an object in a vector (32, and 64 for the
// key index)
const std::array<std::size_t, 17> container_sizes = {{0, 1, 2, 3, 15, 16, 17, 18, 31, 32, 33, 34, 63, 64, 65, 66, 200}};
template<typename Json>
std::string array_text(const std::size_t n, const std::size_t depth)
{
std::string text = "[";
for (std::size_t i = 0; i < n; ++i)
{
text += (i == 0 ? "" : ",");
// alternate scalars, empty containers and nested containers
switch (i % 4)
{
case 0:
text += std::to_string(i);
break;
case 1:
text += "\"s" + std::to_string(i) + "\"";
break;
case 2:
text += depth == 0 ? "[]" : array_text<Json>(n / 2, depth - 1);
break;
default:
text += depth == 0 ? "{}" : "{\"a\":" + array_text<Json>(n / 3, depth - 1) + ",\"b\":null}";
break;
}
}
return text + "]";
}
template<typename Json>
std::string object_text(const std::size_t n, const std::size_t depth, const bool duplicates)
{
std::string text = "{";
for (std::size_t i = 0; i < n; ++i)
{
text += (i == 0 ? "" : ",");
text += "\"key" + std::to_string(i) + "\":";
switch (i % 4)
{
case 0:
text += std::to_string(i);
break;
case 1:
text += depth == 0 ? "[]" : array_text<Json>(n / 2, depth - 1);
break;
case 2:
text += depth == 0 ? "{}" : object_text<Json>(n / 2, depth - 1, duplicates);
break;
default:
text += "\"value\"";
break;
}
}
if (duplicates)
{
// keys that are there already, the first and the last one, with other values
text += n == 0 ? "" : ",";
text += R"("key0":[1,2],"key)";
text += std::to_string(n - 1);
text += R"(":{"again":true})";
}
return text + "}";
}
template<typename Json>
void check_same_dump(const std::string& text)
{
// the text is what dump() writes for the parsed value, and the value
// survives a second round trip: arrays keep their elements in order, an
// ordered_json keeps its members in order of appearance
const Json parsed = Json::parse(text);
const Json again = Json::parse(parsed.dump());
CHECK(parsed == again);
CHECK(parsed.dump() == again.dump());
CHECK(Json::accept(text));
}
} // namespace
TEST_CASE("parsing containers of sizes around the limits of the DOM parser")
{
SECTION("arrays")
{
for (const std::size_t n : container_sizes)
{
const std::string text = array_text<json>(n, 2);
INFO("size ", n);
const json parsed = json::parse(text);
CHECK(parsed.is_array());
CHECK(parsed.size() == n);
for (std::size_t i = 0; i < n; i += 4)
{
CHECK(parsed[i] == i);
}
check_same_dump<json>(text);
check_same_dump<ordered_json>(text);
CHECK(ordered_json::parse(text).dump() == parsed.dump());
}
}
SECTION("objects")
{
for (const bool duplicates :
{
false, true
})
{
for (const std::size_t n : container_sizes)
{
const std::string text = object_text<json>(n, 2, duplicates);
INFO("size ", n, " duplicates ", duplicates);
const json parsed = json::parse(text);
const ordered_json ordered = ordered_json::parse(text);
CHECK(parsed.is_object());
CHECK(ordered.is_object());
CHECK(parsed.size() == ordered.size());
// the same members, in the order of their first appearance
std::size_t i = 0;
for (auto it = ordered.begin(); it != ordered.end(); ++it, ++i)
{
if (i < n)
{
CHECK(it.key() == "key" + std::to_string(i));
}
CHECK(parsed.at(it.key()) == json(it.value()));
}
if (duplicates && n > 1)
{
// the later value replaced the earlier one in its place
CHECK(ordered.at("key0") == ordered_json::parse("[1,2]"));
CHECK(ordered.at("key" + std::to_string(n - 1)) == ordered_json::parse("{\"again\":true}"));
CHECK(ordered.begin().key() == "key0");
CHECK(ordered.size() == n);
}
check_same_dump<json>(text);
check_same_dump<ordered_json>(text);
}
}
}
SECTION("objects with many different and repeated keys")
{
// 300 members of which a third repeat an earlier key
std::string text = "{";
for (std::size_t i = 0; i < 300; ++i)
{
text += (i == 0 ? "" : ",");
text += "\"k" + std::to_string(i % 200) + "\":" + std::to_string(i);
}
text += '}';
const ordered_json ordered = ordered_json::parse(text);
const json parsed = json::parse(text);
CHECK(ordered.size() == 200);
CHECK(parsed.size() == 200);
std::size_t i = 0;
for (auto it = ordered.begin(); it != ordered.end(); ++it, ++i)
{
CHECK(it.key() == "k" + std::to_string(i));
// the last value written
CHECK(it.value() == (i < 100 ? i + 200 : i));
CHECK(parsed.at(it.key()) == json(it.value()));
}
}
}
namespace
{
// a deterministic source of structure for parsing tests: builds a JSON text
// and, at the same time, the value it has to parse to
class tree_generator
{
public:
explicit tree_generator(std::uint32_t seed) : state(seed) {}
template<typename Json>
Json make(std::string& text, const std::size_t depth)
{
switch (next(depth == 0 ? 5 : 9))
{
case 0:
text += "null";
return Json(nullptr);
case 1:
text += "true";
return Json(true);
case 2:
{
const int n = static_cast<int>(next(2000)) - 1000;
text += std::to_string(n);
return Json(n);
}
case 3:
text += "\"text\"";
return Json("text");
case 4:
text += "2.5";
return Json(2.5);
case 5:
case 6:
case 7:
{
Json array = Json::array();
text += '[';
const std::size_t n = size();
for (std::size_t i = 0; i < n; ++i)
{
text += (i == 0 ? "" : ",");
array.push_back(make<Json>(text, depth - 1));
}
text += ']';
return array;
}
default:
{
Json object = Json::object();
text += '{';
const std::size_t n = size();
for (std::size_t i = 0; i < n; ++i)
{
const std::string key = "k" + std::to_string(i);
text += (i == 0 ? "\"" : ",\"") + key + "\":";
object[key] = make<Json>(text, depth - 1);
}
text += '}';
return object;
}
}
}
private:
std::uint32_t next(const std::uint32_t bound)
{
state = (state * 1664525U) + 1013904223U;
return (state >> 8) % bound;
}
std::size_t size()
{
return container_sizes[next(static_cast<std::uint32_t>(container_sizes.size()))];
}
std::uint32_t state;
};
} // namespace
TEST_CASE("parsing random trees of arrays and objects")
{
for (std::uint32_t seed = 1; seed <= 150; ++seed)
{
INFO("seed ", seed);
std::string text;
std::string ordered_text;
const json expected = tree_generator(seed).make<json>(text, 3);
const auto ordered_expected = tree_generator(seed).make<ordered_json>(ordered_text, 3);
CHECK(text == ordered_text);
CHECK(json::parse(text) == expected);
CHECK(json::accept(text));
const ordered_json ordered = ordered_json::parse(text);
CHECK(ordered == ordered_expected);
CHECK(ordered.dump() == ordered_expected.dump());
CHECK(json::parse(text).dump() == expected.dump());
}
}
-4
View File
@@ -1,4 +0,0 @@
# astyle is deliberately pinned: newer versions reformat the code, and this version
# defines the formatting that `make amalgamate` produces and check_amalgamation.yml enforces.
# Dependabot is told to ignore it in .github/dependabot.yml.
astyle==3.4.13
+4 -62
View File
@@ -1,62 +1,4 @@
#
# This file is autogenerated by pip-compile with Python 3.14
# by the following command:
#
# pip-compile --generate-hashes --no-emit-index-url --output-file=tools/astyle/requirements.txt --strip-extras tools/astyle/requirements.in
#
astyle==3.4.13 \
--hash=sha256:0456c5bb1bed18b30f7d9e9c7d6499684a62cc808949173d52bf6def655f48b9 \
--hash=sha256:061b320ea1d265ac01babdd82b056c7b55913c726c770cff26191af78884c5ab \
--hash=sha256:0ad050135504994bee50233db59bedf95f1343977118e27f495eaf2f401ecfe5 \
--hash=sha256:162cb29abc5ddd5d03ab48b15fd8603baf04c1ba5913e1ef475e09818744216b \
--hash=sha256:181d7b6da8eef2cc4dc757dc3381f0d1f2932d6ceb2f6ce5bea432de698e9a8f \
--hash=sha256:191abde742ff5ae788a489722a669f7bc0b18cefc1020b617bc0e84361b71a58 \
--hash=sha256:1bf59aa3006546d795443d1ff69682f989fe95e4c71850c67f8251eab183fee3 \
--hash=sha256:31c86eb8aae517b9cf1debd325ea5f6ac653d52787d5c5c9d56197afc9bc97e3 \
--hash=sha256:35b979a62d1499b2daf241517ddb0aee3ab4b82dc82aa3f8604a2db1e9cc4e66 \
--hash=sha256:36a9f14f88c6bc3c4240f977a3f3fd3d0eac84373bc10004258712e6b3cb08f2 \
--hash=sha256:46ad33bf795e68f4b18bf947cc7276e26579ab6df89574617c7797b0ed2cf473 \
--hash=sha256:4803a16330b67b98c994b87b984af00d7dcaff0d4af69d17ab0fd705141725fe \
--hash=sha256:481b6e195dfa56380bb19cd134dc698cd7f5732d7d5cd781c13b5c15112b8fc2 \
--hash=sha256:48f0218622e807360d50d9d419e527d8e4c0f7f49c51c6fac3d4c6f343a0573e \
--hash=sha256:4a3179783910ea386125d88a16e86cc6cfee41c513d0d8e692bdabd17577ce93 \
--hash=sha256:508d5debe7874a5dd1b491bbae0c48ebc94aa3736545718208bb33c648962e62 \
--hash=sha256:55b4335f20b717f6ec57a237d29246e3c7196bd014e509f2aa376aca36ea9967 \
--hash=sha256:56714bb2e2a0fe3911474b66e563aac508d97a9f48d5e55c8e182b1675db0e05 \
--hash=sha256:599ab574cd53b7105877d5e1b6ab0c933b771e4b2343b0bc54aa06af64653449 \
--hash=sha256:5b183422a1edbce971758b2b4a2b180127d9e8c46119183d410c77ec1d42a516 \
--hash=sha256:5c6de6933f7818b4380b309704c45da9330d906bbe479bb76023261e63545900 \
--hash=sha256:5f7efd14d9aaee69a4a33e60722a8a09d0b7cb834aed698ce0793cecec4e7452 \
--hash=sha256:66ecd8a2a778bc78bdb2df4f5f371f100c6f180ae859e7d414f374eb2636b531 \
--hash=sha256:6fa1c231c82a0ea6ccfa8e19276dec4753d9e084229d606f9522260fe37b955f \
--hash=sha256:76d53ee9e07f8ad32050363c8402923095020856c44015968b4dd9ea5b7a18a1 \
--hash=sha256:81f0173912da9cf687cf2ed1361026fd957fcfd5b9673803c50265db63427d7f \
--hash=sha256:81f4345adbbb463a2d5ce314a1ca689a0aa05a6240c86fc0225e92474e3f44b4 \
--hash=sha256:840e1d0436a1a3793ebe14c95fefe797f77bd78d6194a7c0bcdde705cb583403 \
--hash=sha256:8d6a69f9b316b3941cd43f16eb109f738bb171963408b5f135521ef70ad364f1 \
--hash=sha256:8e2879d770e3fbf53ff62728f4d62a6edcacd1ed3d62b36c012706d778048340 \
--hash=sha256:900e442ed002e5e05c31628fac817db62f6d28e1cc2426040c69a10d39f92651 \
--hash=sha256:920c8516f420da68765ced84692e550d0f34932663219c48db9b0c79544002d3 \
--hash=sha256:9417573994a9d264f873cd6122f9e0ea7044d628d9a8b9e84dbbc85b0e7c2880 \
--hash=sha256:95eaf11aadd770bccd833c8ffe222ea6055fe5c2c4b3974cb43f3366913b62a4 \
--hash=sha256:9b7471f4a5f174604cdfdf4f9091ba319edb52471e07672ea71745157d121fe7 \
--hash=sha256:9e016b099f809c8d4894fa6d36ef925973f52ebc50136306b1b4b9ce65094063 \
--hash=sha256:9e04dbd51f564316a4879448258d9b9e68ade9b062351ea6fc7356d0f42471a0 \
--hash=sha256:a002a3a1ac187ba1b441f678aca16b4ecea47a1f3d2462c3ec9c59e71e6083b7 \
--hash=sha256:a177039c38f6f6a43f46bcd3571756be385c0c47462e6a06eb5950397d0bdcbe \
--hash=sha256:a8963c261b980a0c12fc1523f86eab3edc67940c114ea544b21d2b44e9196d23 \
--hash=sha256:a97f8ef2a054ec067e3af65bc5b145c51b6c2a5f2efe997a72a9d958c3ab6d1a \
--hash=sha256:aa8ba72c1349b7e45d1ce01d69daecfac680713624d27d83ebf2a9d1b4984f65 \
--hash=sha256:abba627fb930a5398414ba398169f52e3865855501b5ee5430bb53d0974074c6 \
--hash=sha256:ac8db98b392b609f36cd7d6e7f18a247376723b4397578a6f450a445a9dacb3d \
--hash=sha256:c9d4b1a0a7fbc35b96273695143c791df9c1a4dcab0815ae6d9db8483314430f \
--hash=sha256:cf247a3bca29f43398af4e20c7c922b7f5130a22f306f068ad038b94e72854d6 \
--hash=sha256:d3da892b9ec2932a2cb49941aae79447f6baafe5023ee31680c9cabcc471b451 \
--hash=sha256:db8c4555b66067aba253c2f32dc4ff6ee92882dd3e95a00ebb3bbf145efe067d \
--hash=sha256:dde61d5acb6008ebefcecc37625b7f2776df2f368f92d47714e5f50276347c9f \
--hash=sha256:dede5398c41b3a00ad0a9f77c57c05444640f8b28500b9bb4624ab6f2d0a90b7 \
--hash=sha256:e3cc45784dbac7ade37967e24cbd6b5b89517f3df8fbed276dffffdcf289f04d \
--hash=sha256:eb4d38f4a6be78cd28ce44397a1244c08af15c1a00293c40634c2104b44c943c \
--hash=sha256:f9e6cd9ee60a8c42eab85cb57e0c82b1c947261b71f5091bbb9f61e864c64e07 \
--hash=sha256:fb1ddc816b483c83de15db98847a5d2dd3ba9df9f4a937ad47aabadeeaeed58c
# via -r tools/astyle/requirements.in
# astyle is deliberately pinned: newer versions reformat the code, and this version
# defines the formatting that `make amalgamate` produces and check_amalgamation.yml enforces.
# Dependabot is told to ignore it in .github/dependabot.yml.
astyle==3.4.13
+3
View File
@@ -139,9 +139,12 @@ if __name__ == '__main__':
namespaces += [f'{namespaces[0]}::{ns}' for ns in inline_namespaces]
# The template renders a local XML file, not a web response, and autoescape is enabled.
# nosemgrep: python.flask.security.xss.audit.direct-use-of-jinja2.direct-use-of-jinja2
env = jinja2.Environment(loader=jinja2.FileSystemLoader(searchpath=sys.path[0]), autoescape=True, trim_blocks=True,
lstrip_blocks=True, keep_trailing_newline=True)
template = env.get_template('nlohmann_json.natvis.j2')
# nosemgrep: python.flask.security.xss.audit.direct-use-of-jinja2.direct-use-of-jinja2
natvis = template.render(namespaces=namespaces)
output = args.output or os.path.join(args.repo_root, 'nlohmann_json.natvis')
-1
View File
@@ -1 +0,0 @@
jinja2==3.1.6
+1 -163
View File
@@ -1,163 +1 @@
#
# This file is autogenerated by pip-compile with Python 3.14
# by the following command:
#
# pip-compile --generate-hashes --no-emit-index-url --output-file=tools/generate_natvis/requirements.txt --strip-extras tools/generate_natvis/requirements.in
#
jinja2==3.1.6 \
--hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \
--hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67
# via -r tools/generate_natvis/requirements.in
markupsafe==3.0.4 \
--hash=sha256:007e1ffd9bf65bb6ee96df7b258fc632a4868dd5566037986c64781f35a36e98 \
--hash=sha256:02fa4acbc6a3fc5c693c34d4dd8c1130b7fe99cc915181b0ddd6f72aeb296002 \
--hash=sha256:03470d1a8268e692ecf79ecd565593e59d44219377a7ead61f1f1b94c1f7ff6b \
--hash=sha256:04e7902ba80ee4bac1d50a549606527a1dcf0476cd81403db41099d3b60ec653 \
--hash=sha256:051417f74bcaaefa316276e0ff723f541616ca51043d070da00249d9bddd3e3c \
--hash=sha256:05295589e619b9bed252a86b532b8e27350abc372d18ba89b59375325e91ec1e \
--hash=sha256:06de8ef6331f6e822c28d577dc8bf43fe398800477c49498f38fc38b67ff33fc \
--hash=sha256:0764a13d34cae40db7bbf3a09b7e9b491bf4603e20b263a7a9d6b8e324975d0a \
--hash=sha256:077293e425f28ec737dbcad442a71752e28f8ae27cde3d68acd1fb212091cd92 \
--hash=sha256:0930db9bdc62d22944e10b066448bb65dc9abe9112880c7cab8da54db4284d5f \
--hash=sha256:0cee7cb0f9a1b6892ea482237d9403b3d1b4603aee057d0ff01f0fac2d019a97 \
--hash=sha256:0d9c47709875fdb321452056622e930c52afbc07a7d780762fbb8b4d91ce6fa4 \
--hash=sha256:11935df9bf455ed0c04eb87bcd720f02b1fe5e02128a9430f23aed6f93336fc7 \
--hash=sha256:12a606a492de952afcb43b59a14aaaaad120e708d3663dd0fdf2d738d427a691 \
--hash=sha256:14bd2d845d62ab678eaf81da89d7b621b51756c72346745c1a594c09d49207a2 \
--hash=sha256:15ba9e28640feef770374b116a6f019c21f52404aeabe516aa7f800587b98cfc \
--hash=sha256:18a801868a884f216e784d7d14db2a4077143ce7610440aee2ce8f734e7cfcde \
--hash=sha256:1c0df495a977d10460a94941799c72d5b5ab03d3858d949b55b5a66c8f371c99 \
--hash=sha256:1caa2fa5a6184fb233153b35f654e6687bd555476f6170f29d8ee9be1a8b0af9 \
--hash=sha256:1e1451fab512d1bcc3dc26988ec1edb0b82c2db909132872cd9356070a6b63df \
--hash=sha256:1f1f9477e174582b0a1b583d60b66e1f2cf5d3fe12cee985e4aedf44766600e5 \
--hash=sha256:2628d3a8cb648ecebb3c5d6b0a1052d400e4d8b7ac0fb786be8d285b50040d17 \
--hash=sha256:26e9867520db70d37f7fb421a7f0d8adb40171011fb84ce869afa1a83370dfa8 \
--hash=sha256:2a6ef68ae94aed8721934072b27a3b654ea2100b97e4ab864cf1489c90926fbc \
--hash=sha256:2b2b1e18af909b448bb3cf9e3433366f7a8726271fc214e8b10e0f62a78c724b \
--hash=sha256:2cb3dd71fc6be918ad4264346a8ed69485f9b7ed7bf35495d8e22807cd6b8bea \
--hash=sha256:2d1b7d9308288661f56672b1b157d75fc536714d3638487bbea17b6318a78248 \
--hash=sha256:2dad610540cb2e6272855c178f08ae9a1c7ac258a7fb71660553a5f104b42741 \
--hash=sha256:2e5a7cd7fdd14fcb1ae5d7d8bf23d24fbd1daefd1fbca2580132e1ea75f098b5 \
--hash=sha256:2e9ad7dd851bf45fab9f75cbff4cb493fee9979e8d8c7c9c3ee119022518edd6 \
--hash=sha256:340cbb1957ba99929cbf19a75626d36ba1ae21d1730b287d1cf7f824a20c4fc7 \
--hash=sha256:34bdde374c5932765d7dc685c4a1d191a3207852d67e8e0a9eb6ea85156181f1 \
--hash=sha256:353bd63081912ab8cfa6a0c7d185934cdf8426f04c618bba6bc4b394f2069b67 \
--hash=sha256:387d8cd30e69b3f0a72877b9ae717033396404e19095b17fe89753a981fda44f \
--hash=sha256:3882fb412298575bae3b9c46868251f15cc69307359f87bb1b382e53d6e5a2c9 \
--hash=sha256:38fc55594dab834470b6733dead2ee9e3f657fb0608c769dcafa0ba5ab52f45c \
--hash=sha256:396ec4e65cc889f69786b3b89478b471cee5a3bcf468b9d9bb03e1a30fb291fc \
--hash=sha256:39dbacefc411633db5b4378b066a9aca70a3d7e2922c9e578d825f844026eeba \
--hash=sha256:3a93d9616ddecfb393727a0041a562cf0b15a244e20f2bd25efc7949be4c4f17 \
--hash=sha256:3d23795802fc8bd72534836d64489bbf0f67c088959091bdb22e10735a5107bf \
--hash=sha256:434139499bb20b502ed3baa1f169e618f924a97e7a777fea1a49446d80106cf6 \
--hash=sha256:436e3ffc6310d3c41878c601db29098102fe5d8a467c49da4a4125254e0980f2 \
--hash=sha256:489505b03f692c3f376394e49194fa7a7f9e8558d6e293a7056a0032b0c38163 \
--hash=sha256:4a540e2d3192792fc84eced57bef37851ccb2b41f73291bb17408eea77bcd278 \
--hash=sha256:4a7cdc2a420ca01058182da4253329764d4bfa055564d1eced90e6ba1e8b1d3d \
--hash=sha256:4bced6e2a6dba6a28f7dd3c6ce14df1b2dd495923f16ea484cad03decd463b2b \
--hash=sha256:4cf3468d5ec187ffffcaca8e61929a37448f215dafc1386a12c750a72fe53634 \
--hash=sha256:4e2c4809c14559aa7ef426f27fb35afbb38104c349a903bf8f3600456764bb38 \
--hash=sha256:4ed644d75aa94a2baf7ec3a96eaa160ea58c742eb9d27c6506053c5c40fc84ed \
--hash=sha256:4f6e0852a0283b1b1fd776eeb7b766a5f440b3e2bd31ab51af3b400585f3965c \
--hash=sha256:5066b244f576f91afc8ee3ba029a89f99d39c79b1853fe9d39bea9f0afbec148 \
--hash=sha256:5086f9975abb1ab531ee6afca1761e4b59a19b446f3f6522ed776963228cfe5a \
--hash=sha256:50b5bedc9ed8a94fc8857a42ef4f84a81ea88f8d4f05dc8705fb23ee6d8dcca7 \
--hash=sha256:52704c5d36eb6dda8866493decd61111fff86244c9b1ad225ca01b9e91e5970f \
--hash=sha256:55ffd6ce583d97dc71dc92e930324c8c0d25aea7e3ade6ae54ef77cedb096811 \
--hash=sha256:569d65055d367e3dcdf30c3f41119467b73d9ee9faf332bdf40402644f5ac08e \
--hash=sha256:57f9947a7e57a081c1e3e0a2dd0d2dcf290a4531450e6f611e30084c222a7295 \
--hash=sha256:5989cb26b2e1efc6a42216a9f6b5ee495ce5ace2e5b352a9af489976b32d1ee2 \
--hash=sha256:5c22873ad1f0532ba40fa1727f3c0fc1bbbaab6d373d4cbe3f0dc74b2e2521c7 \
--hash=sha256:5e8b3d0b18fd623afa12ecb2ce8d8becef69f9b5440c6330c7972200e0bb84b0 \
--hash=sha256:61631e08084be9e21a8967ec3139c7616ed7c5e9368e05c86d1b39562c8a57b6 \
--hash=sha256:64511c54db4e4987aef4c41923235927428729e8174c5dba488429be70a998ed \
--hash=sha256:6669c1bf34080161ce49c589cc512ef24d4c704ac9d2b2d3667f519c60418378 \
--hash=sha256:672d207103e6b16ca098611b0f9efad6bc00afd47c03d6ef62186495ca677dc0 \
--hash=sha256:6768d67d1bce64270e0fdc2e69309d68b9b18ae56ddf6c711d168e9d051c2cac \
--hash=sha256:6a45c3d514f2436064db00d7fc8778d888f0236ebfed649b53d13a59e69ad51b \
--hash=sha256:6bd9e1788e15bfcf6a9082de42e30387e7b85d211ab21e57a939bb8cfaaf8d96 \
--hash=sha256:6d2a9efe686f9de00d0d1ea32a4a5a86d558a2277501bd78d964214eab625e59 \
--hash=sha256:6da83a088f8ef93b2d483a8232a4dbf4d69d3d8496b568a03c56becac43e1808 \
--hash=sha256:7018d4af1cd272e847aa5917983ab5e83e4f6579f9dbfecd4a79c0ca80b144c2 \
--hash=sha256:71f88e749ea29f67f21f3b36433c1dc54c7729ed2a6d9e2da2e0d9e0d7b224eb \
--hash=sha256:737c9c3981998eba27f11786f84fddcbabc74068b72a4a1f454ea02094b57b65 \
--hash=sha256:73e77980c7207854f00fc4e71fb1626868d5740ab4012623d55c7a99ad122a72 \
--hash=sha256:799c39bdf5e2f1292fedd3009f7b3c9e760f10b2420cb9638d56920840ff6db8 \
--hash=sha256:7a83aa6e4805df46fed18e989d3d16f86ef60cb50bbc8d9ce3a6be89165fbf6e \
--hash=sha256:7d3391b2188d18737cb2fa147028b1096236eaa7e156446c650a489fa2cadc91 \
--hash=sha256:7e1636da3d8dfc220b6dd10264db5f2b165e4888c4518594898fbe381049af8a \
--hash=sha256:805c8b84534fa10891890f0e4be39f3a99e94615d93e8836bf9fa1fdca2feeb2 \
--hash=sha256:811d02d5122171c1941357efd8f9bf4ffe907b7f0a1a4e729a880e4be3f46e3e \
--hash=sha256:8138eb83940ec7299024d92d4dee45f601b9e6c5ffde9d25f4e35e326203c707 \
--hash=sha256:83b3944fea42a8400edf92fd1770fb8d0d4f7de651353bd2d8525a92dba69a21 \
--hash=sha256:849dd2bb0e5e4ab2b71c7191726a4a8d5aa8a610daa584728cbee0b710ddc4ef \
--hash=sha256:8698d70a8081ee8c090dbb394768b5789a1da8b131b5499f89d071dd3cfaf6be \
--hash=sha256:8781a792a070cf2bd1b86d3aa943894115faaba6e88122a7bf32d62072742453 \
--hash=sha256:88d59b473bfb03259722600839af9bbd7fa13a2eb514beefeedb95997882f69a \
--hash=sha256:8909c2f1c6dd65e054ac4b573a91c8384d1492281e55d82d159d653f7a13adf6 \
--hash=sha256:8965520ac587c94a4ac48b729be3d8b8de00af39699b17585dfb599babe77977 \
--hash=sha256:8b5d563170ff8ba3181caa967c99a3c804d1dedb702c7cb93a6a7c32247da978 \
--hash=sha256:8e124f974786f831d6043728e38296969d3579db8896fe004682f5758e613581 \
--hash=sha256:8f0fac8b13d14bb06c68195f849371924ae53dd7b1c00fed24650f704383b692 \
--hash=sha256:9240187afb63d2f9ddc3e032c670356fe941f6e20662ea168a5dc3f1f317e1b3 \
--hash=sha256:925f929d6b59a8b3f8b8c6ac363cd0af7eecc81efb3071770b3c6717c450a369 \
--hash=sha256:9348cbb300d224fe3b89793262cb093504d4ae927004468463f745188a193e4a \
--hash=sha256:9388003072b95f2f1e3fd908604194d653ba21330d811961a78b7da1a77e9e36 \
--hash=sha256:9438a2648b2195980cb2dd8e53ed7b8df91319e2d0b70ae61a9e1d1bc8d3bec9 \
--hash=sha256:94e4c421742086aeee4c32a506eec8859d7634aad943f7e6aacf70f813478768 \
--hash=sha256:94f5407f7bc64fa6463906b896f9904beeeb7dd8dc116ee8e9056c8714ff9916 \
--hash=sha256:971a3bbb75d97ae4e2e8f7d4834236f86f85f0c85e04ab2e191db1123b04f80b \
--hash=sha256:9e227f3dbe6bde7491cf0a9965d00b88c6b1a4a95d11480ddf88bb96d397c19f \
--hash=sha256:9e25feb9e330b63edb0278a0acdf85e50d0cb0fbf49c3084abbe4e24ae195346 \
--hash=sha256:9f098115c247e11d138ab83a28fa0323c77015007ea2df73ba5fd714dfefd67c \
--hash=sha256:a18f38cafc329bac5e3c2b96c765b4c96d3d103421ed22ab7988c1e3fce27464 \
--hash=sha256:a4bbd2d87dd233b9fc5812160c3d0ffbe42edc22a26ce0469f58479ede633fe9 \
--hash=sha256:a5fcffb37e602b0b3c1638a97746b9b96125caa9bcf6fa41d337a9261de231ee \
--hash=sha256:a8e9f292fcda89b324f2f5c91d13f1424a153e40fc2756f38ee23b15835ff300 \
--hash=sha256:a9f54054101545a9a9cccefddf54316aa6e4491611fcbef9e91b3b6bebec04f6 \
--hash=sha256:aa2c838cc024642cc04c6854232f32b43e5e22833dd11119c1766c7873b8370d \
--hash=sha256:ac0c7c9f1609b0c4c114feb1d7a3409564c7fb77e360bed9e97e5d25dfeaf868 \
--hash=sha256:add96447a86d205ab616665d53b2950ee81083757f56e6ea833c8b2917646b46 \
--hash=sha256:ae9dcb8fbe244cb82f8a6458b455b927a03685e383d9bacf1ea5ce180b96dc97 \
--hash=sha256:b4a635a0487774f841cb1fb62e907e7195cc95bc761e053184b8acc3ceb20733 \
--hash=sha256:b4d12837e0203bbace818ff4a7461afdcd78bcd782351cea148139180d7bcffe \
--hash=sha256:b61687d0828e72bf5cda24a2690188f37170bd31c9359ac97e4e66569f120a16 \
--hash=sha256:b807e598953730f82e4eae3bd30f6a122cf6b31c398c6b504c0e04c13c170429 \
--hash=sha256:b8cd1f918b26fd7b1832ece557cc18f2d8747309ff8b3f0ef9d4250c5ad67a39 \
--hash=sha256:b91cc9d336957239ff200f30097e6fea2dc6d6fb3c81e853eaa09eac904fd894 \
--hash=sha256:bd3ce56ae2cbae3ba82b683bc425cd7e48d2ed8b10f3e818186b6f5646d9271c \
--hash=sha256:be6cb0c799abb0e2ba3e618e6d28ddddf7e485f6c2ce938dfa237daf3905072c \
--hash=sha256:befb4158af32106b9a93db8d6d1d1cbbd418c0d5aca0cabb7b1780abf0c89169 \
--hash=sha256:bf053da3c97a4bc5ecfbb218cdd2983febd91c617be8367d139882aa11e490aa \
--hash=sha256:c02e8f18bdedba082cef725942ac823b9b60656db07f7e265cb31618dfd00d77 \
--hash=sha256:c1bc67752d5f21013cfe430df4062441714eab79f65a6a05e01505957e9c35fe \
--hash=sha256:c61750fadcd119d0825bcb7d7d675dd264dcc89cc05292aab5be68ebdbb374ad \
--hash=sha256:c90d5b3d4e944e065a301d741b3c1d784f6bd1f503aa68b4967e32b2ba313d85 \
--hash=sha256:c9a7f43c0b202b334cc9184af09bb8f21d3a209e038efaf106936fb69e6b026e \
--hash=sha256:cb96e6e088d6cf71c1ea977510948320234824cf226e32f6f6e044f7a9c82b34 \
--hash=sha256:cf63c214fe879a65e69a386f915e36104fc84254ab141240f8854602d8e0be2a \
--hash=sha256:d1aca03ede943eb80ab3d63bb082c84b7aab85ea83bd0fd0c200260945fb49d9 \
--hash=sha256:d2e56fd3b00222722abfb3f5f0759ddbae4b90811b5ad4343c64030ad1bde70c \
--hash=sha256:d5f93ebbeb8032d47e349328ec8662d973d9b05a70b3c35df1f91fe419b84749 \
--hash=sha256:d882a373d8093c2941e01291b7ced96e9cbe4781da9a7751ca7e6c70385e5214 \
--hash=sha256:d920abdfa61279ba1a2ef9484aab07bf03331f8c08a10120fa332353d06e6932 \
--hash=sha256:da2af0d7aebfc2074080d72efa6ab8317c62481ef1f896f65d9999c1c01f4494 \
--hash=sha256:dd8ea6ebee7aedbf7c749fa80521d9ccf1ba473e0d1e14805caafbaad281c889 \
--hash=sha256:de8b364c423ef0a4bad9069657d617f9a5d2b2062457a89b1fa16ee199c399c1 \
--hash=sha256:df1ae86ff54725a01fa1a0510b914ca53a161b7050be74f6204e24aded5971d0 \
--hash=sha256:dff05cb7016dff1e9fd68f4122c127b65dfc59de5306cfb7ad92f956f230bee2 \
--hash=sha256:e1a622f13970d81f95d0c72f9dc090dce9085fccfa4c9f2174377ee32bd15786 \
--hash=sha256:e49fb0d1ce92cfa0cb198cc5b1b11cdf9d0638658e2a2db2687e39db7c87fc78 \
--hash=sha256:e5c802729725bd07e2bc3ab7b76dc7e0bbfc53129d8f1eb1c002c24cf774717e \
--hash=sha256:e841068dc0be4cb6dfb5c890eb88cbdcff2f4a332393c7ec94e8e618bd32c1a8 \
--hash=sha256:e916035e3e9930cbdfdd10abf48861340221857f45509565898e012263f7b289 \
--hash=sha256:eba154571c16e032112afac0dc2dfe9e63c2ceb7aedd07bb7eecf2ce26d4dd4c \
--hash=sha256:f03460ff076f70ab595bb45a0205ccea1971443575b6920c52e755dec2b3fbfe \
--hash=sha256:f0ec3b750b59375eab5b0fb2b9254810c00a3375be6d789899f1055a1d556237 \
--hash=sha256:f291bcf42ae98eb5107edb162c3c998b4a89648fd8e99ed4cbd12705292788cd \
--hash=sha256:f61efe1d2fe0de16158a5fe1d1cf3c14bdb6aecd54d8938fd26512c525c1f624 \
--hash=sha256:f68edfc67aabac33708941f26f22a7b8e9f81429bc0cf249fcf7d66b23af8d19 \
--hash=sha256:fa95848c929b6a75f6848d3c9793e59db365ee436776e57db835cdbfa79ba977 \
--hash=sha256:fd9f8797427910198f95bced71ddfed61130d7e349213bfb8466c9c99e2c46a8 \
--hash=sha256:fdb4ca07ab75ffadab4a8b135ad59cdbb3156b99310f3d565370da74a15d6bd3
# via jinja2
jinja2==3.1.6