Compare commits

..
Author SHA1 Message Date
Niels Lohmann 405b8dcb13 BSON reader: recover from the document size checks made on reading the size
Since #5930, a size below 5, a size that exceeds the enclosing document, and
a size too small for the document's content are reported before the document
is read. When recovering, such a document is now read up to its terminator,
within the enclosing document, and its size is not reported a second time.
BJData now checks an ndarray's type before the object that would hold it is
begun, so that test expects null.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 13:47:31 +01:00
Niels Lohmann 02d2fa8605 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 13:47:30 +01:00
Niels Lohmann 52e9f098eb Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-25 18:01:19 +01:00
Niels Lohmann a6c5d970f3 Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-25 06:24:39 +01:00
Niels Lohmann d9c0fda829 State the frame stack invariants of json_sax_dom_parser as assertions
clang-tidy's static analyzer analyzes the forwarding SAX handlers of the
error recovery tests (RecoveringDomParser, BasicRecoveringParser) as entry
points, with the state of their json_sax_dom_parser member unknown. It then
assumes a frame stack of capacity 0 that grow() cannot enlarge
(clang-analyzer-security.ArrayBound) and a collected container without an
enclosing frame (clang-analyzer-core.NonNullParamChecker). Neither can
happen: the stack starts with local_capacity frames, and only a container
inside another one is collected. Assert both, which also lets the analyzer
discard these paths.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 14:39:04 +02:00
Niels Lohmann a763c9eb83 Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-24 07:47:46 +02:00
Niels Lohmann 5c2aba70f9 Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-23 06:35:52 +02:00
Niels Lohmann 510ee0ab4f Let the json_view error replay handler return from parse_error()
sax_parse() recovers from an error if parse_error() returns true. For a
handler whose parse_error() is [[noreturn]], MSVC reports that code as
unreachable (C4702), which fails the /WX module build. throwing_sax now
throws under a flag, as json_sax_dom_parser does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-21 18:45:37 +02:00
Niels Lohmann 5da0e9181f Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-21 12:49:18 +02:00
Niels Lohmann 0aff24ca37 Restart the token string of token-tracking adapters after an error
Since #5828, a stream adapter keeps the current token itself
(mark_token_start() / copy_token()), so lazy_token_string no longer
implies a seekable adapter. restart_token_string_impl(std::true_type)
still called get_consumed_count(), which only seekable adapters have, and
failed to compile for std::istream input. Dispatch on adapter_token as
note_token_start() does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-21 11:59:41 +02:00
Niels Lohmann c25aad3ca5 Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-21 11:59:40 +02:00
Niels Lohmann 6a0cf2ba2f Recover by error kind instead of by error message
The lexer records an error_kind with each error_message, and
recover_token() dispatches on it instead of comparing message prefixes.
add_codepoint() now uses encode_utf8(), and the surrogate pair
formula is shared by scan_string() and recover_low_surrogate().
The stubs for AllowRecovery == false select via negation, which astyle
leaves alone.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-21 09:06:28 +02:00
Niels Lohmann 24de5b2f5c Count lines incrementally when a recovering parse reports many errors
With seekable input, the lexer derives line and column on demand (#5809)
by counting the newlines in all input consumed so far. A SAX parser that
recovers from errors asks for the position once per error, so input with
an error at every character took quadratic time: "{" followed by 100000
commas took 85 s at -O0, and the parser tests timed out under Valgrind
(1500 s, was 69 s on develop).

get_position() now remembers what it counted and only scans the input
consumed since the previous call; count_consumed_lines() takes the start
of the range. The 100000 commas now take 0.17 s, unit-class_parser 0.9 s
instead of 86 s. Over 69284 error messages from random multi-line inputs,
the messages are identical to those before the change.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-17 20:10:10 +02:00
Niels Lohmann fb729028ba Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-17 08:35:02 +02:00
Niels Lohmann c9fdfc39db Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 22:46:35 +02:00
Niels Lohmann 870bdec01f Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 20:56:37 +02:00
Niels Lohmann effa51d528 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 10:47:22 +02:00
Niels Lohmann 545477ffa8 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-11 08:33:14 +02:00
Niels Lohmann aeba637593 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-10 00:08:31 +02:00
Niels Lohmann 466e56d92d Move the MessagePack SAX tests into unit-sax_parse.cpp
With develop's latest changes, the MinGW Debug object of unit-msgpack.cpp
has 33134 COFF sections on its own and 33411 with this PR, more than the
32767 that the MinGW linker of the "clang (20.1.8)" job can relocate. Its
sax_parse calls, all with utils::SaxCountdown, instantiate a parser and
binary reader of their own; moving those tests into unit-sax_parse.cpp
brings the object down to 32271 sections (unit-sax_parse: 20678).

The moved tests are unchanged: the "SAX aborts" sections, the user-defined
SAX consumer of #5405, the deep value read through the SAX interface (#5104),
and "MessagePack SAX parsing stops at every event".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-09 23:35:38 +02:00
Niels Lohmann b7b75aac8a Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-09 23:30:03 +02:00
Niels Lohmann 516b9b516a Fix the Visual Studio 2015 build of the recovery code and its tests
- parser::report_error() calls only sax->parse_error(). For a SAX parser
  whose parse_error() is static, as in unit-disabled_exceptions.cpp,
  Visual Studio 2015 then reports the parameter as unreferenced (C4100),
  which /WX turns into an error.
- The variadic concatenated() helper in unit-sax_parse.cpp made Visual
  Studio 2015 fail with an internal compiler error (C1001). It now takes
  an initializer list of byte vectors.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-09 23:14:53 +02:00
Niels Lohmann 1d2dea7769 Do not force inlining of the recovery stubs with MSVC
f8ff0942b marked the stubs that stand in for error recovery with
JSON_HEDLEY_ALWAYS_INLINE, which is __forceinline with MSVC. Visual
Studio 2015 and 2017 do not inline them and warn (C4714), which /WX
turns into an error on AppVeyor. The new internal macro
JSON_INTERNAL_ALWAYS_INLINE forces inlining everywhere else (including
clang-cl), and leaves the decision to MSVC. The MinGW objects keep their
smaller section counts, as GCC and clang still inline the stubs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-09 00:52:35 +02:00
Niels Lohmann f8ff0942b2 Keep the MinGW test objects below the linker's section limit
The MinGW clang job links with a GNU ld that cannot relocate an object
with more than 32767 sections. unit-regression2, unit-regression3 and
unit-msgpack went over it:

- At -O0, the parser and binary reader for parse(), accept() and
  from_*() still emitted the stubs that stand in for recovering, and
  the parser referenced its recovery helpers from code that never runs.
  The stubs are now always inlined, and the parser reaches its helpers
  through the same allow_recovery tag as the stubs, so that this code
  is not even instantiated there. This removes about 300 sections from
  unit-msgpack.
- Every sax_parse call instantiates the parser and binary reader that
  recover from errors. The tests that call it move from
  unit-regression2.cpp (#3989) and unit-regression3.cpp (#5676) into
  unit-sax_parse.cpp, like unit-explicit_instantiation.cpp got a file
  of its own (#5511), and CONTRIBUTING.md points there.

Section counts of the MinGW Debug objects (clang 19, -g0), develop / PR:
unit-msgpack 32281 / 32558, unit-regression2 31272 / 31278,
unit-regression3 32392 / 27462, unit-sax_parse - / 19836.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-08 10:36:16 +02:00
Niels Lohmann d1e6439187 Use the file-scope float_json alias in the #3989 test
The local alias that the out-of-range test declared shadows the identical
file-scope one, which GCC's -Wshadow=global turns into an error.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 22:56:50 +02:00
Niels Lohmann dcf7e2222c Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 19:24:12 +02:00
Niels Lohmann d99932fb3f Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 09:04:47 +02:00
Niels Lohmann 1d2cb49ff3 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 08:54:17 +02:00
Niels Lohmann a2766df787 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 07:42:58 +02:00
Niels Lohmann 2ea5bed0e3 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-06 08:39:31 +02:00
Niels Lohmann 632f5369fe Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-06 07:43:24 +02:00
Niels Lohmann c2619bf460 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-04 18:03:20 +02:00
Niels Lohmann 351aeb7440 Merge remote-tracking branch 'origin/develop' into HEAD
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-04 17:19:32 +02:00
Niels Lohmann 7b0af0072e Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	include/nlohmann/detail/input/binary_reader.hpp
#	include/nlohmann/detail/string_utils.hpp
#	single_include/nlohmann/json.hpp
2026-10-04 12:22:05 +02:00
Niels Lohmann 650886d126 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	tests/src/unit-alt-string.cpp
2026-10-04 11:50:33 +02:00
Niels Lohmann a759afc99f Fix CI: no semicolon after CAPTURE() in the #3989 tests
#5737 enabled -Wextra-semi-stmt for the tests and removed the
semicolons after CAPTURE(), which expands to a complete statement; the
#3989 tests added twelve more.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-03 09:11:00 +02:00
Niels Lohmann 1143da4faa Fix CI: keep the binary readers' recovery code out of from_*()
The MinGW job ("clang (20.1.8)", Debug) failed to link test-msgpack_cpp17
with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against
`.rdata'". Its GNU ld (MinGW 12.2.0) cannot link an object file with
more than 32767 sections. Reproduced with clang 20.1.8 and binutils
2.40: unit-msgpack.cpp's object has 31958 sections on develop and had
33210 with #3989; binutils 2.44 links both.

The readers of from_*() never recover, but unoptimized builds still
emitted the recovery code their dead branches referenced: about 20
helpers per reader instantiation, plus the lexer's recovery functions
through the UBJSON high-precision number repair. Now:

- Errors are reported with report_error() only. The helpers that act on
  the SAX parser's answer (repair_requested(), resync(), value_failed(),
  close_open_containers(), and the repairs of high-precision numbers and
  unsupported BSON elements) have one trivial variant for readers that do
  not recover, so the recovery code is not referenced there.
- Object keys that are not strings and BSON elements whose end is lost
  are marked at the error site and skipped by resync() in the reading
  loops; report_repairable_error(), report_error_repairable_if(),
  report_bson_element_error(), and bon8_error_repairable_if() are gone.
- is_cbor_item_head() is a free function, and the BSON element type is
  formatted as on develop again.

The object now has 32638 sections and links. Behavior is unchanged
(regression tests, binary format suites, and fuzzing), and an optimized
program that only calls from_*() is now 256 bytes smaller than on
develop.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-02 19:48:27 +02:00
Niels Lohmann 8de151f928 Fix CI: no backslash sequence in the lexer's doc comments
clang's -Wdocumentation-unknown-command reads "\u" in a doc comment as
an unknown command, even in backticks. Two comments of the recovery code
now say "Unicode escape" instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-02 19:08:41 +02:00
Niels Lohmann ab52c98f71 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	docs/mkdocs/docs/api/json_sax/parse_error.md
#	docs/mkdocs/docs/features/parsing/index.md
2026-10-02 11:38:09 +02:00
Niels Lohmann 5e93415d91 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	include/nlohmann/detail/input/lexer.hpp
#	include/nlohmann/detail/input/parser.hpp
#	single_include/nlohmann/json.hpp
#	tests/src/fuzzer-parse_json.cpp
2026-10-01 07:36:30 +02:00
Niels Lohmann 7e8e8e219b Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	include/nlohmann/detail/input/binary_reader.hpp
#	include/nlohmann/json.hpp
#	single_include/nlohmann/json.hpp
2026-09-30 22:53:03 +02:00
Niels Lohmann 5f1727cef2 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	tests/src/unit-alt-string.cpp
2026-09-30 20:11:28 +02:00
Niels Lohmann c16dd7e4f5 Fix CI: do not hide base class methods in the #3989 test SAX parsers
clang-tidy's bugprone-derived-method-shadowing-base-method rejected the
test SAX parsers that derive from json_sax_dom_parser (or the test's
SaxEventLogger) and redefine their non-virtual event functions. The
recovering DOM parsers of unit-class_parser.cpp and unit-regression2.cpp
now hold a json_sax_dom_parser and forward to it, SaxEventLogger gets a
flag to recover from errors instead of a derived class, and the one
function unit-alt-string.cpp redefines is marked as intended.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 04:50:11 +02:00
Niels Lohmann 792853d725 Fix CI: clang-tidy and JSON_NOEXCEPTION in the #3989 changes
ci_clang_tidy flagged nested conditional operators in the BON8 skip
code and in remove_incomplete_utf8_sequence()
(readability-avoid-nested-conditional-operator), and two branches with
the same body in recover_string() (bugprone-branch-clone). Use if
chains instead of the nested conditionals and merge the two branches
into one condition; the short-circuit order is unchanged.

ci_test_noexceptions aborted in the #3989 regression test: it compares
the first recovered error with the message of the exception that
from_cbor() and friends throw, and under JSON_NOEXCEPTION that call
aborts instead of throwing. Guard binary_error_message() and its uses
with #if !defined(JSON_NOEXCEPTION), as other tests in the suite do.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 16:42:57 +02:00
Niels Lohmann 4bdf1b7e74 Fix CI: no global std::string in the #3989 regression test
The U+FFFD constant was a global std::string, which the clang CI build
rejects with -Wglobal-constructors. It is now a function.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 09:27:46 +02:00
Niels Lohmann b1e9d98e41 Merge remote-tracking branch 'origin/claude/fix-issue-3989-db7e45' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 03:31:07 +02:00
Niels Lohmann f855d257df Fix CI: keep raw strings with backslashes out of test macros for MSVC
MSVC stringizes the arguments of doctest's CHECK() so that a raw string
literal becomes an ordinary one, and then reported the "\q" in the new
alt_string recovery test as warning C4129, an error with /WX. The input
is now a variable. The same pattern with "\u0000" in the parser's
recovery test is replaced by a JSON value built from a std::string.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 03:30:39 +02:00
Niels Lohmann 3e683e9c04 Merge branch 'develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-28 22:22:25 +02:00
Niels Lohmann d1d84ed9af Fix Flawfinder: format the BSON element type without snprintf
Moving the report of an unsupported BSON element type into
skip_unsupported_bson_element() moved its snprintf() call, which
Flawfinder then reported as a new CWE-134 finding. The two hexadecimal
digits are now computed directly; the message is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-28 20:36:28 +02:00
Niels Lohmann de8529f99b Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	include/nlohmann/detail/input/binary_reader.hpp
#	single_include/nlohmann/json.hpp
2026-09-28 17:53:51 +02:00
Niels Lohmann 677794f076 Fix CI: recover numbers with the string operations every string_t has
recover_number() used back(), pop_back(), front(), and
find_first_of(const char*), which the minimal alt_string of
unit-alt-string.cpp does not provide. Since the binary readers recover
UBJSON/BJData high-precision numbers with it, from_ubjson() instantiated
it, too, and the test no longer compiled. It now uses only size(),
operator[], and resize(), and unit-alt-string.cpp recovers from errors
in JSON text, UBJSON, and CBOR.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-28 12:17:11 +02:00
Niels Lohmann 437a95cfdb Repair complete items in binary formats when parse_error() returns true (#3989)
When the SAX parser asks to recover, the binary readers now repair an
item whose end is known and read on after it, as RFC 8949, Section 5.3
describes for CBOR:

- CBOR: tags are ignored, and simple values other than false, true, and
  null become null (RFC 8949, Section 6.1); a negative integer below the
  range of number_integer_t becomes the nearest floating-point number.
- Strings that are not valid UTF-8 get U+FFFD for each ill-formed
  sequence, as in JSON text; so does a UBJSON/BJData char above 0x7F.
- UBJSON/BJData high-precision numbers keep their longest valid
  beginning (via the lexer's recover_token()), or become infinity.
- Members whose key is not a string are skipped (CBOR, MessagePack,
  BON8), like members without a key in JSON text.
- BSON elements of types the library does not read (ObjectId, datetime,
  decimal128, ...) become null; a string without its terminator and a
  document whose size does not match are kept.

Where the end of an item is unknown, reading stops as before, except
that BSON skips to the end of the document, whose size it knows.

The value read before such an error is now completed by the reader from
its container stack, as the JSON parser does, instead of by a proxy SAX
parser, which is removed. Like the parser, binary_reader gets an
AllowRecovery template parameter, so that from_*() compile without the
new code.

Tests: a table of repairs, numbers out of range, errors that stop, and
a sweep over changed and removed bytes of eight encodings that checks
balanced events and that the first error is the one from_*() reports.
All fuzzers now run a recovering checker; the binary ones also check
that it reports an error exactly when from_*() fails.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 21:40:59 +02:00
Niels Lohmann c8735246d0 Merge remote-tracking branch 'origin/develop' into claude/fix-issue-3989-db7e45
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 21:02:38 +02:00
Niels Lohmann 9adb510a0d Recover from parse errors when parse_error() returns true (#3989)
The return value of json_sax::parse_error() was documented both as "must
return false" and as "whether the parsing should continue", and the code
just passed it on. For JSON text, parsing stopped anyway, but sax_parse()
could report success for invalid input. The binary readers read on after
the error, looping forever on a CBOR indefinite-length array without its
end.

Now false stops parsing, and true recovers from the error:

- JSON text is repaired with the smallest local edit (insert a missing
  ',' or ':', remove a stray token, keep the readable part of a broken
  string or number, null for a value that cannot be read, close the
  innermost container at a wrong closing bracket and all of them at the
  end of the input), and parsing continues. The SAX events stay balanced,
  every key is followed by exactly one value, and each token is reported
  at most once.
- The binary formats cannot resynchronize, so they stop, but complete
  the value read so far.

sax_parse() returns false after any error. parse(), accept(), and the
from_*() functions never recover and compile to the same code as before.

Supersedes #4522.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 20:16:42 +02:00
170 changed files with 8574 additions and 4530 deletions

No files matched your search

+2
View File
@@ -111,6 +111,8 @@ context which existing file needs to be extended, and only very few cases requir
When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue. When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue.
`unit-regression2.cpp` holds the older tests; the two files exist because a single one grew large enough for the `unit-regression2.cpp` holds the older tests; the two files exist because a single one grew large enough for the
MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing the larger one. MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing the larger one.
Tests that call `sax_parse` go into `unit-sax_parse.cpp` rather than into a large test file: every call instantiates
the parser and the binary reader that recover from errors, which grows a test file considerably.
#### Exceptions #### Exceptions
+2 -2
View File
@@ -495,7 +495,7 @@ bool key(string_t& val);
bool parse_error(std::size_t position, const std::string& last_token, const detail::exception& ex); bool parse_error(std::size_t position, const std::string& last_token, const detail::exception& ex);
``` ```
The return value of each function determines whether parsing should proceed. The return value of each function determines whether parsing should proceed. For `parse_error`, returning `true` [recovers from the error](https://json.nlohmann.me/features/parsing/error_recovery/): the parser repairs the input and continues.
To implement your own SAX handler, proceed as follows: To implement your own SAX handler, proceed as follows:
@@ -503,7 +503,7 @@ To implement your own SAX handler, proceed as follows:
2. Create an object of your SAX interface class, e.g. `my_sax`. 2. Create an object of your SAX interface class, e.g. `my_sax`.
3. Call `bool json::sax_parse(input, &my_sax)`; where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface. 3. Call `bool json::sax_parse(input, &my_sax)`; where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface.
Note the `sax_parse` function only returns a `bool` indicating the result of the last executed SAX event. It does not return a `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error -- it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp). Note the `sax_parse` function only returns a `bool` indicating whether the input was parsed without errors and no SAX event returned `false`. It does not return a `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error -- it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp).
### STL-like access ### STL-like access
+3 -3
View File
@@ -348,20 +348,20 @@ add_custom_target(ci_test_no_thread_local
add_custom_target(ci_test_coverage add_custom_target(ci_test_coverage
COMMAND CXX=g++ ${CMAKE_COMMAND} COMMAND CXX=g++ ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja -DCMAKE_CXX_FLAGS="--coverage;-fprofile-arcs;-ftest-coverage;-fvar-tracking-assignments" -DCMAKE_BUILD_TYPE=Debug -GNinja -DCMAKE_CXX_FLAGS="--coverage;-fprofile-arcs;-ftest-coverage"
-DJSON_BuildTests=ON -DJSON_BuildTests=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_coverage -S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_coverage
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_coverage COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_coverage
COMMAND cd ${PROJECT_BINARY_DIR}/build_coverage && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure COMMAND cd ${PROJECT_BINARY_DIR}/build_coverage && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
COMMAND CXX=g++ ${CMAKE_COMMAND} COMMAND CXX=g++ ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja -DCMAKE_CXX_FLAGS="-m32;--coverage;-fprofile-arcs;-ftest-coverage;-fvar-tracking-assignments" -DCMAKE_BUILD_TYPE=Debug -GNinja -DCMAKE_CXX_FLAGS="-m32;--coverage;-fprofile-arcs;-ftest-coverage"
-DJSON_BuildTests=ON -DJSON_32bitTest=ONLY -DJSON_BuildTests=ON -DJSON_32bitTest=ONLY
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_coverage32 -S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_coverage32
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_coverage32 COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_coverage32
COMMAND cd ${PROJECT_BINARY_DIR}/build_coverage32 && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure COMMAND cd ${PROJECT_BINARY_DIR}/build_coverage32 && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
COMMAND ${LCOV_TOOL} --directory . --capture --output-file json.info --rc branch_coverage=1 --ignore-errors mismatch --ignore-errors unused --ignore-errors inconsistent COMMAND ${LCOV_TOOL} --directory . --capture --output-file json.info --rc branch_coverage=1 --rc geninfo_unexecuted_blocks=1 --ignore-errors mismatch --ignore-errors unused --ignore-errors inconsistent
COMMAND ${LCOV_TOOL} -e json.info ${SRC_FILES} --output-file json.info.filtered --rc branch_coverage=1 --ignore-errors unused --ignore-errors inconsistent COMMAND ${LCOV_TOOL} -e json.info ${SRC_FILES} --output-file json.info.filtered --rc branch_coverage=1 --ignore-errors unused --ignore-errors inconsistent
COMMAND ${CMAKE_SOURCE_DIR}/tests/thirdparty/imapdl/filterbr.py json.info.filtered > json.info.filtered.noexcept COMMAND ${CMAKE_SOURCE_DIR}/tests/thirdparty/imapdl/filterbr.py json.info.filtered > json.info.filtered.noexcept
COMMAND ${GENHTML_TOOL} --title "JSON for Modern C++" --legend --demangle-cpp --output-directory html --show-details --branch-coverage --ignore-errors inconsistent json.info.filtered.noexcept COMMAND ${GENHTML_TOOL} --title "JSON for Modern C++" --legend --demangle-cpp --output-directory html --show-details --branch-coverage --ignore-errors inconsistent json.info.filtered.noexcept
+1 -3
View File
@@ -178,10 +178,8 @@ function(_json_test_add_test test_name file main cxx_standard)
COMMAND ${memcheck_command} $<TARGET_FILE:${test_target}> ${DOCTEST_TEST_FILTER} COMMAND ${memcheck_command} $<TARGET_FILE:${test_target}> ${DOCTEST_TEST_FILTER}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
) )
# under Valgrind the slowest tests take close to 25 minutes, so ctest's
# default timeout of 1500 seconds leaves no headroom
set_tests_properties(${test_target}_valgrind PROPERTIES set_tests_properties(${test_target}_valgrind PROPERTIES
LABELS "valgrind" FIXTURES_REQUIRED TEST_DATA TIMEOUT 3600 LABELS "valgrind" FIXTURES_REQUIRED TEST_DATA
) )
endif() endif()
endfunction() endfunction()
-10
View File
@@ -120,16 +120,6 @@ overload (3).
See [Number conversion](../../features/types/number_handling.md#number-conversion) See [Number conversion](../../features/types/number_handling.md#number-conversion)
for more information. for more information.
!!! note "Booleans as numbers"
A JSON boolean converts to `#!cpp 0` or `#!cpp 1` for every arithmetic type **except** [`number_integer_t`](number_integer_t.md),
[`number_unsigned_t`](number_unsigned_t.md) and [`number_float_t`](number_float_t.md) (by default `#!cpp std::int64_t`,
`#!cpp std::uint64_t`, and `#!cpp double`), for which `#!cpp get<T>()` throws
[`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) ("type must be number, but is boolean"). So
`#!cpp json(true).get<int>()` is `#!cpp 1`, while `#!cpp json(true).get<std::int64_t>()` and
`#!cpp json(true).get<double>()` throw. This is the behavior of all 3.x versions;
[`basic_json_view::get<T>()`](../basic_json_view/get.md) behaves the same.
!!! note "`std::optional` conversions" !!! note "`std::optional` conversions"
Prior to version 3.13.0, `#!cpp get<std::optional<T>>()` (and other conversions to `std::optional<T>`) failed to Prior to version 3.13.0, `#!cpp get<std::optional<T>>()` (and other conversions to `std::optional<T>`) failed to
+2 -2
View File
@@ -50,7 +50,7 @@ Besides `basic_json`, the library offers a second way to hold a parsed JSON text
[`basic_json_document`](../basic_json_document/index.md), which owns or borrows the text and holds a flat index of the [`basic_json_document`](../basic_json_document/index.md), which owns or borrows the text and holds a flat index of the
values in it, and a [`basic_json_view`](../basic_json_view/index.md), a read-only handle to one value of such a document. values in it, and a [`basic_json_view`](../basic_json_view/index.md), a read-only handle to one value of such a document.
Both are defined in the header `<nlohmann/json_view.hpp>`; see Both are defined in the header `<nlohmann/json_view.hpp>`; see
[Zero-copy JSON views](../../features/json_view/index.md) for the concept and the trade-offs. [Zero-copy JSON views](../../features/json_view.md) for the concept and the trade-offs.
- [**json_document**](../json_document.md) / [**ordered_json_document**](../ordered_json_document.md) - read-only - [**json_document**](../json_document.md) / [**ordered_json_document**](../ordered_json_document.md) - read-only
documents of the `basic_json` specialization they belong to documents of the `basic_json` specialization they belong to
@@ -363,7 +363,7 @@ Access to the JSON value
specification of the JSON data interchange format specification of the JSON data interchange format
- [`basic_json_document`](../basic_json_document/index.md) - / [`basic_json_view`](../basic_json_view/index.md) - parse a JSON - [`basic_json_document`](../basic_json_document/index.md) - / [`basic_json_view`](../basic_json_view/index.md) - parse a JSON
text into a flat index and read it through a view text into a flat index and read it through a view
- [Zero-copy JSON views](../../features/json_view/index.md) - when to use a document and a view instead of a `basic_json` - [Zero-copy JSON views](../../features/json_view.md) - when to use a document and a view instead of a `basic_json`
- [Types](../../features/types/index.md) - how JSON values are stored and which types exist - [Types](../../features/types/index.md) - how JSON values are stored and which types exist
- [Converting values](../../features/conversions.md) - get values out of a JSON value and put values in - [Converting values](../../features/conversions.md) - get values out of a JSON value and put values in
- [Creating JSON values](../../features/creating_values.md) - all the ways to build a JSON value - [Creating JSON values](../../features/creating_values.md) - all the ways to build a JSON value
+1 -6
View File
@@ -65,12 +65,7 @@ the same index, which is now a different value.
## Exception safety ## Exception safety
Overload (1) gives the strong guarantee: if an exception is thrown, there are no changes in the JSON value. Strong exception safety: if an exception occurs, the original value stays intact.
Overloads (2) to (5) give the basic guarantee: if an exception is thrown while the elements or members are copied
(for example, `std::bad_alloc`), the JSON value may be partially modified, but it stays a valid array or object.
The exceptions listed below are thrown before the JSON value is changed.
## Exceptions ## Exceptions
+1 -1
View File
@@ -124,7 +124,7 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
[`basic_json_document::parse`](../basic_json_document/parse.md) accepts the same inputs and the same options, but [`basic_json_document::parse`](../basic_json_document/parse.md) accepts the same inputs and the same options, but
keeps the result as a flat index of the parsed values instead of a tree of `basic_json` values. It is faster and uses keeps the result as a flat index of the parsed values instead of a tree of `basic_json` values. It is faster and uses
less memory when only parts of the document are read, and it also accepts a pointer to a null-terminated wide string, less memory when only parts of the document are read, and it also accepts a pointer to a null-terminated wide string,
which `parse()` does not. See [Zero-copy JSON views](../../features/json_view/index.md). which `parse()` does not. See [Zero-copy JSON views](../../features/json_view.md).
!!! tip "Performance" !!! tip "Performance"
+4 -1
View File
@@ -104,7 +104,9 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
## Return value ## Return value
return value of the last processed SAX event `#!cpp true` if the input was parsed without errors and no SAX event returned `#!cpp false`; `#!cpp false` otherwise.
In particular, the result is `#!cpp false` for input with errors, even if the SAX parser recovered from all of them
(see [error recovery](../../features/parsing/error_recovery.md)).
## Exception safety ## Exception safety
@@ -163,6 +165,7 @@ use the same file with stdio functions, but it must not wait for another thread
- Added `ignore_trailing_commas` in version 3.13.0. - Added `ignore_trailing_commas` in version 3.13.0.
- Added `tag_handler` in version 3.13.0. - Added `tag_handler` in version 3.13.0.
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0. - Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
- Recovering from parse errors (see [`parse_error`](../json_sax/parse_error.md)) added in version 3.13.0.
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0. - Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
- `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave a `#!cpp std::istream` positioned right - `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave a `#!cpp std::istream` positioned right
after the parsed value when `strict` is `#!cpp false`. after the parsed value when `strict` is `#!cpp false`.
@@ -77,7 +77,7 @@ compile; see [`parse`](parse.md#notes).
- [`parse`](parse.md) - deserialize from a compatible input - [`parse`](parse.md) - deserialize from a compatible input
- [`BasicJsonType::accept`](../basic_json/accept.md) - the corresponding function of `basic_json` - [`BasicJsonType::accept`](../basic_json/accept.md) - the corresponding function of `basic_json`
- [Zero-copy JSON views](../../features/json_view/differences.md#what-is-the-same) - how parsing a document differs from, and matches, basic_json::parse - [Zero-copy JSON views](../../features/json_view.md#what-is-the-same-as-parse) - how parsing a document differs from, and matches, basic_json::parse
## Version history ## Version history
@@ -52,7 +52,7 @@ Constant.
- [`parse`](parse.md) - deserialize from a compatible input - [`parse`](parse.md) - deserialize from a compatible input
- [`is_discarded`](is_discarded.md) - return whether the last parse failed - [`is_discarded`](is_discarded.md) - return whether the last parse failed
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - who owns the source text and for how long - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - who owns the source text and for how long
## Version history ## Version history
@@ -123,7 +123,7 @@ by an erase, not only those at or after the removed element, since the container
- [`root`](root.md) - the view of the root value, the starting point of overload 3 - [`root`](root.md) - the view of the root value, the starting point of overload 3
- [`BasicJsonType::erase`](../basic_json/erase.md) - the corresponding function of `basic_json` - [`BasicJsonType::erase`](../basic_json/erase.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload - [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Zero-copy JSON views](../../features/json_view/editing.md) - what an edit guarantees and costs - [Zero-copy JSON views](../../features/json_view.md#editing-a-document) - what an edit guarantees and costs
## Version history ## Version history
@@ -109,7 +109,7 @@ points apply to every edit:
anything `BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...). anything `BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
- [`dump()`](../basic_json_view/dump.md) writes an edited document with members in document order, new members at - [`dump()`](../basic_json_view/dump.md) writes an edited document with members in document order, new members at
the end, and, with [`number_format::source`](../basic_json_view/number_format.md), keeps the spelling of every the end, and, with [`number_format::source`](../basic_json_view/number_format.md), keeps the spelling of every
number that was not itself edited — see [Editing a document](../../features/json_view/editing.md) for number that was not itself edited — see [Editing a document](../../features/json_view.md#editing-a-document) for
why this matters. why this matters.
- [`read()`](read.md) discards all edits, [`shrink_to_fit()`](shrink_to_fit.md) does not move the node index once - [`read()`](read.md) discards all edits, [`shrink_to_fit()`](shrink_to_fit.md) does not move the node index once
there are edits, and [`memory_usage()`](memory_usage.md) includes the memory edits use. there are edits, and [`memory_usage()`](memory_usage.md) includes the memory edits use.
@@ -118,7 +118,7 @@ points apply to every edit:
## See also ## See also
- [Zero-copy JSON views](../../features/json_view/index.md) - guide to documents and views - [Zero-copy JSON views](../../features/json_view.md) - guide to documents and views
- [`basic_json_view`](../basic_json_view/index.md) - the non-owning handle to a value in a document - [`basic_json_view`](../basic_json_view/index.md) - the non-owning handle to a value in a document
- [`json_document`](../json_document.md) - and [`json_editable_document`](../json_editable_document.md) - the default specializations - [`json_document`](../json_document.md) - and [`json_editable_document`](../json_editable_document.md) - the default specializations
- [`materialize`](../basic_json_view/materialize.md) - build a `basic_json` value from a view - [`materialize`](../basic_json_view/materialize.md) - build a `basic_json` value from a view
@@ -108,7 +108,7 @@ an edit in general.
- [`set`](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to - [`set`](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [`BasicJsonType::insert`](../basic_json/insert.md) - the corresponding function of `basic_json` - [`BasicJsonType::insert`](../basic_json/insert.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload - [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Zero-copy JSON views](../../features/json_view/editing.md) - what an edit guarantees and costs - [Zero-copy JSON views](../../features/json_view.md#editing-a-document) - what an edit guarantees and costs
## Version history ## Version history
@@ -43,7 +43,7 @@ When the document is discarded, [`root()`](root.md) returns a discarded view (it
- [`parse`](parse.md) - deserialize from a compatible input - [`parse`](parse.md) - deserialize from a compatible input
- [`basic_json_view::is_discarded`](../basic_json_view/is_discarded.md) - return whether a view is invalid - [`basic_json_view::is_discarded`](../basic_json_view/is_discarded.md) - return whether a view is invalid
- [Zero-copy JSON views](../../features/json_view/differences.md#what-is-the-same) - how a discarded document arises - [Zero-copy JSON views](../../features/json_view.md#what-is-the-same-as-parse) - how a discarded document arises
## Version history ## Version history
@@ -24,7 +24,7 @@ static basic_json_document load(std::vector<std::uint8_t>&& image,
In every overload, the node index is copied into storage the document itself owns — so that it is properly aligned, In every overload, the node index is copied into storage the document itself owns — so that it is properly aligned,
and, for an [editable](index.md#edits) document, can be edited — while the text and the decoded strings stay in and, for an [editable](index.md#edits) document, can be edited — while the text and the decoded strings stay in
`image`. The hash indexes [large objects](../../features/json_view/index.md) use for lookup are rebuilt, exactly as after `image`. The hash indexes [large objects](../../features/json_view.md) use for lookup are rebuilt, exactly as after
parsing. parsing.
## Parameters ## Parameters
@@ -172,7 +172,7 @@ returned document, and any [view](../basic_json_view/index.md) taken from it, is
- [`save`](save.md) - write the document as an image - [`save`](save.md) - write the document as an image
- [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text - [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text
- [`parse`](parse.md) - deserialize from JSON text instead of an image - [`parse`](parse.md) - deserialize from JSON text instead of an image
- [Images](../../features/json_view/images.md) - why and when to use images - [Images](../../features/json_view.md#images) - why and when to use images
## Version history ## Version history
@@ -44,7 +44,7 @@ to observe the effect of [`shrink_to_fit()`](shrink_to_fit.md).
- [`node_count`](node_count.md) - the number of index entries - [`node_count`](node_count.md) - the number of index entries
- [`shrink_to_fit`](shrink_to_fit.md) - release unused index capacity - [`shrink_to_fit`](shrink_to_fit.md) - release unused index capacity
- [Zero-copy JSON views](../../features/json_view/index.md#the-idea) - the flat node index and its memory - [Zero-copy JSON views](../../features/json_view.md#the-idea) - the flat node index and its memory
## Version history ## Version history
@@ -43,7 +43,7 @@ strings and, for an owned document, the text).
- [`memory_usage`](memory_usage.md) - the number of bytes held by the document - [`memory_usage`](memory_usage.md) - the number of bytes held by the document
- [`shrink_to_fit`](shrink_to_fit.md) - release unused index capacity - [`shrink_to_fit`](shrink_to_fit.md) - release unused index capacity
- [Zero-copy JSON views](../../features/json_view/index.md#the-idea) - the flat node index and its memory - [Zero-copy JSON views](../../features/json_view.md#the-idea) - the flat node index and its memory
## Version history ## Version history
@@ -43,7 +43,7 @@ document (`#!cpp owns_source() == false`) is only valid while the buffer it was
- [`parse`](parse.md) - deserialize from a compatible input - [`parse`](parse.md) - deserialize from a compatible input
- [`parse_copy`](parse_copy.md) - deserialize a copy of a compatible input, always owned - [`parse_copy`](parse_copy.md) - deserialize a copy of a compatible input, always owned
- [`source`](source.md) - the parsed text - [`source`](source.md) - the parsed text
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - who owns the source text and for how long - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - who owns the source text and for how long
## Version history ## Version history
@@ -112,7 +112,7 @@ told apart from other class-type iterators, so their range is read into an owned
range (e.g. of a `#!cpp std::list<char>`). range (e.g. of a `#!cpp std::list<char>`).
See [`owns_source`](owns_source.md) to check which happened after a call, and the See [`owns_source`](owns_source.md) to check which happened after a call, and the
[feature page](../../features/json_view/index.md) for the reasoning. [feature page](../../features/json_view.md) for the reasoning.
**Numbers.** As for [`BasicJsonType::parse()`](../basic_json/parse.md), an integer literal too large for the 64-bit **Numbers.** As for [`BasicJsonType::parse()`](../basic_json/parse.md), an integer literal too large for the 64-bit
integer type becomes a floating-point value. integer type becomes a floating-point value.
@@ -74,7 +74,7 @@ compile; see [`parse`](parse.md#notes).
- [`parse`](parse.md) - deserialize from a compatible input, borrowing it where possible - [`parse`](parse.md) - deserialize from a compatible input, borrowing it where possible
- [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text - [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - borrowing or copying the source text - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - borrowing or copying the source text
## Version history ## Version history
@@ -96,7 +96,7 @@ Like [`set`](set.md) on a member or an element, `push_back` never moves an exist
- [`root`](root.md) - the view of the root value - [`root`](root.md) - the view of the root value
- [`BasicJsonType::push_back`](../basic_json/push_back.md) - the corresponding function of `basic_json` - [`BasicJsonType::push_back`](../basic_json/push_back.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload - [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Zero-copy JSON views](../../features/json_view/editing.md) - what an edit guarantees and costs - [Zero-copy JSON views](../../features/json_view.md#editing-a-document) - what an edit guarantees and costs
## Version history ## Version history
@@ -81,7 +81,7 @@ compile; see [`parse`](parse.md#notes).
- [`parse`](parse.md) - deserialize from a compatible input - [`parse`](parse.md) - deserialize from a compatible input
- [`root`](root.md) - the view of the root value - [`root`](root.md) - the view of the root value
- [`load`](load.md) - read a document from an image instead of parsing JSON text - [`load`](load.md) - read a document from an image instead of parsing JSON text
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - reusing a document and what it invalidates - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - reusing a document and what it invalidates
## Version history ## Version history
@@ -40,7 +40,7 @@ auto v = doc.root();
returned view is valid under the same conditions as any other view of the document — see returned view is valid under the same conditions as any other view of the document — see
[Object inspection](../basic_json_view/index.md) — in particular, it is invalidated by the next [Object inspection](../basic_json_view/index.md) — in particular, it is invalidated by the next
[`read()`](read.md) or [`shrink_to_fit()`](shrink_to_fit.md) on this document. A view can dangle, like a [`read()`](read.md) or [`shrink_to_fit()`](shrink_to_fit.md) on this document. A view can dangle, like a
`#!cpp std::string_view`; see [Ownership and lifetime](../../features/json_view/lifetime.md). `#!cpp std::string_view`; see [Ownership and lifetime](../../features/json_view.md#ownership-and-lifetime).
## Examples ## Examples
@@ -91,7 +91,7 @@ big-endian reader could not interpret correctly.
- [`load`](load.md) - read an image written by `save()` - [`load`](load.md) - read an image written by `save()`
- [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text - [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document to JSON text instead of an image - [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document to JSON text instead of an image
- [Images](../../features/json_view/images.md) - why and when to use images - [Images](../../features/json_view.md#images) - why and when to use images
## Version history ## Version history
@@ -195,7 +195,7 @@ one-node scalar.
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document, keeping an untouched number's - [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document, keeping an untouched number's
spelling with `#!cpp number_format::source` spelling with `#!cpp number_format::source`
- [Edits](index.md#edits) - what an edit guarantees, for every overload - [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Editing a document](../../features/json_view/editing.md) - why editable documents keep the source - [Editing a document](../../features/json_view.md#editing-a-document) - why editable documents keep the source
text's order and number spelling text's order and number spelling
## Version history ## Version history
@@ -56,7 +56,7 @@ possible would be an internal optimization only, not a guarantee to rely on.
- [`node_count`](node_count.md) - the number of index entries - [`node_count`](node_count.md) - the number of index entries
- [`memory_usage`](memory_usage.md) - the number of bytes held by the document - [`memory_usage`](memory_usage.md) - the number of bytes held by the document
- [`root`](root.md) - the view of the root value - [`root`](root.md) - the view of the root value
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - what releasing memory invalidates - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - what releasing memory invalidates
## Version history ## Version history
@@ -42,7 +42,7 @@ is; see [`owns_source`](owns_source.md).
- [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text - [`owns_source`](owns_source.md) - return whether the document holds its own copy of the text
- [`source_offset`](../basic_json_view/source_offset.md) - byte offset of a value in the source text - [`source_offset`](../basic_json_view/source_offset.md) - byte offset of a value in the source text
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - who owns the source text and for how long - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - who owns the source text and for how long
## Version history ## Version history
+1 -1
View File
@@ -92,7 +92,7 @@ Pointer path of the value within the document, and with
If the source text repeats a key, this resolves to the *first* member with it, not to the last one that If the source text repeats a key, this resolves to the *first* member with it, not to the last one that
[`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of [`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of
`operator[]` and [Duplicate keys](../../features/json_view/differences.md#duplicate-keys) for the reasons and for how to get `operator[]` and [Duplicate keys](../../features/json_view.md#duplicate-keys) for the reasons and for how to get
the last value. the last value.
Unlike [`operator[]`](operator[].md), which returns a [discarded](is_discarded.md) view for a missing key or an Unlike [`operator[]`](operator[].md), which returns a [discarded](is_discarded.md) view for a missing key or an
@@ -58,7 +58,7 @@ placeholder for "no value yet" and later be assigned a real view.
- [`is_discarded`](is_discarded.md) - return whether the view is invalid - [`is_discarded`](is_discarded.md) - return whether the view is invalid
- [`root`](../basic_json_document/root.md) - the view of a document's root value - [`root`](../basic_json_document/root.md) - the view of a document's root value
- [`view_mode`](../view_mode.md) - whether a document and its views can be edited - [`view_mode`](../view_mode.md) - whether a document and its views can be edited
- [Zero-copy JSON views](../../features/json_view/lifetime.md) - how long a view stays valid - [Zero-copy JSON views](../../features/json_view.md#ownership-and-lifetime) - how long a view stays valid
## Version history ## Version history
@@ -47,7 +47,7 @@ No-throw guarantee: this function never throws exceptions.
If the source text repeats a key, this resolves to the *first* member with it, not to the last one that If the source text repeats a key, this resolves to the *first* member with it, not to the last one that
[`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of [`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of
`operator[]` and [Duplicate keys](../../features/json_view/differences.md#duplicate-keys) for the reasons and for how to get `operator[]` and [Duplicate keys](../../features/json_view.md#duplicate-keys) for the reasons and for how to get
the last value. the last value.
Overload 1 always returns `#!cpp false` when the value is not an object — including a [discarded](is_discarded.md) Overload 1 always returns `#!cpp false` when the value is not an object — including a [discarded](is_discarded.md)
+2 -2
View File
@@ -70,7 +70,7 @@ except that `#!cpp -0` is written as `#!cpp 0`, the way [`BasicJsonType::parse()
`#!cpp number_format::source` copies every number exactly as written in the source text instead, with no exception `#!cpp number_format::source` copies every number exactly as written in the source text instead, with no exception
for `#!cpp -0` — `#!cpp 1.50`, `#!cpp 1E2`, `#!cpp -0.0`, `#!cpp -0`, or all digits of an integer literal with more for `#!cpp -0` — `#!cpp 1.50`, `#!cpp 1E2`, `#!cpp -0.0`, `#!cpp -0`, or all digits of an integer literal with more
digits than any number type holds (such a literal is itself classified as a float, see digits than any number type holds (such a literal is itself classified as a float, see
[What is different](../../features/json_view/differences.md)) — something `BasicJsonType` cannot do, since [What is different](../../features/json_view.md#what-is-different)) — something `BasicJsonType` cannot do, since
parsing already reduces every number to its parsed value. parsing already reduces every number to its parsed value.
## Examples ## Examples
@@ -98,7 +98,7 @@ parsing already reduces every number to its parsed value.
- [`operator<<`](operator_ltlt.md) - serialize this value to a stream - [`operator<<`](operator_ltlt.md) - serialize this value to a stream
- [`materialize`](materialize.md) - build a `BasicJsonType` value, e.g. to use `BasicJsonType::dump`'s `error_handler` - [`materialize`](materialize.md) - build a `BasicJsonType` value, e.g. to use `BasicJsonType::dump`'s `error_handler`
- [`BasicJsonType::dump`](../basic_json/dump.md) - the corresponding function of `basic_json` - [`BasicJsonType::dump`](../basic_json/dump.md) - the corresponding function of `basic_json`
- [Zero-copy JSON views](../../features/json_view/reading.md#writing-a-view-back) - writing a view back without building a tree - [Zero-copy JSON views](../../features/json_view.md#writing-a-view-back) - writing a view back without building a tree
## Version history ## Version history
+1 -1
View File
@@ -37,7 +37,7 @@ constant time on average.
If the source text repeats a key, this resolves to the *first* member with it, not to the last one that If the source text repeats a key, this resolves to the *first* member with it, not to the last one that
[`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of [`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of
`operator[]` and [Duplicate keys](../../features/json_view/differences.md#duplicate-keys) for the reasons and for how to get `operator[]` and [Duplicate keys](../../features/json_view.md#duplicate-keys) for the reasons and for how to get
the last value. the last value.
Unlike [`BasicJsonType::find`](../basic_json/find.md), which always returns `#!cpp end()` for a non-object type, this Unlike [`BasicJsonType::find`](../basic_json/find.md), which always returns `#!cpp end()` for a non-object type, this
+4 -5
View File
@@ -11,8 +11,7 @@ For the types below, the conversion works directly on the flat index — no `Bas
- `#!cpp bool` - `#!cpp bool`
- arithmetic types other than `#!cpp bool` (for a boolean, `#!cpp 0`/`#!cpp 1`, exactly as for - arithmetic types other than `#!cpp bool` (for a boolean, `#!cpp 0`/`#!cpp 1`, exactly as for
[`BasicJsonType::get<T>()`](../basic_json/get.md#notes), except for `number_integer_t`, `number_unsigned_t` and [`BasicJsonType::get<T>()`](../basic_json/get.md))
`number_float_t`, which throw `type_error.302` for a boolean, as `BasicJsonType::get<T>()` does)
- `#!cpp std::nullptr_t` - `#!cpp std::nullptr_t`
- `#!cpp std::basic_string<char, Traits, Alloc>` (including `string_t`) — a copy of the string - `#!cpp std::basic_string<char, Traits, Alloc>` (including `string_t`) — a copy of the string
- [`string_view_t`](index.md#member-types) — **no copy**: the returned view points into the document's - [`string_view_t`](index.md#member-types) — **no copy**: the returned view points into the document's
@@ -39,14 +38,14 @@ as [`BasicJsonType::parse()`](../basic_json/parse.md) does), as well as the type
accept views, e.g. with [`is_json_view`](../is_json_view.md). A `from_json()` that also accepts types that are no JSON accept views, e.g. with [`is_json_view`](../is_json_view.md). A `from_json()` that also accepts types that are no JSON
values at all, such as the unconstrained `#!cpp template<typename BasicJsonType> void from_json(const BasicJsonType&, values at all, such as the unconstrained `#!cpp template<typename BasicJsonType> void from_json(const BasicJsonType&,
T&)` of [arbitrary type conversions](../../features/arbitrary_types.md), is not called with a view: it may use T&)` of [arbitrary type conversions](../../features/arbitrary_types.md), is not called with a view: it may use
`BasicJsonType` functions a view does not have. See [Writing generic code](../../features/json_view/generic_code.md). `BasicJsonType` functions a view does not have. See [Writing generic code](../../features/json_view.md#writing-generic-code).
!!! warning "Duplicate keys" !!! warning "Duplicate keys"
Such a `from_json()` looks members up in the view, where a repeated key finds the **first** member, while Such a `from_json()` looks members up in the view, where a repeated key finds the **first** member, while
`#!cpp materialize().get<T>()` sees the **last** one, as [`BasicJsonType::parse()`](../basic_json/parse.md) `#!cpp materialize().get<T>()` sees the **last** one, as [`BasicJsonType::parse()`](../basic_json/parse.md)
keeps it: for `#!cpp {"x":1,"x":2}`, `#!cpp get<point>()` reads `#!cpp x == 1`. See keeps it: for `#!cpp {"x":1,"x":2}`, `#!cpp get<point>()` reads `#!cpp x == 1`. See
[Duplicate keys](../../features/json_view/differences.md#duplicate-keys). [Duplicate keys](../../features/json_view.md#duplicate-keys).
Every other `T` — user types whose `from_json()` is written for `BasicJsonType` only, types without a default Every other `T` — user types whose `from_json()` is written for `BasicJsonType` only, types without a default
constructor, `#!cpp std::tuple`s holding references, ... — is converted by `#!cpp materialize().get<T>()`: the subtree is built into a real `BasicJsonType` constructor, `#!cpp std::tuple`s holding references, ... — is converted by `#!cpp materialize().get<T>()`: the subtree is built into a real `BasicJsonType`
@@ -152,7 +151,7 @@ instead: its path is relative to the root of the view's subtree, because `materi
- [`number_token`](number_token.md) - a number's token text, without a copy - [`number_token`](number_token.md) - a number's token text, without a copy
- [`materialize`](materialize.md) - build the `BasicJsonType` value of this subtree - [`materialize`](materialize.md) - build the `BasicJsonType` value of this subtree
- [`BasicJsonType::get`](../basic_json/get.md) - the corresponding function of `basic_json` - [`BasicJsonType::get`](../basic_json/get.md) - the corresponding function of `basic_json`
- [Zero-copy JSON views](../../features/json_view/reading.md#getting-values-out-without-copying) - which conversions avoid building a basic_json value - [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - which conversions avoid building a basic_json value
## Version history ## Version history
@@ -70,7 +70,7 @@ since floating-point values and unescaped strings are read from the source lazil
- [`get`](get.md) - convert the value to a given type (`#!cpp get<string_view_t>()` is equivalent to this function) - [`get`](get.md) - convert the value to a given type (`#!cpp get<string_view_t>()` is equivalent to this function)
- [`number_token`](number_token.md) - a number's token text, without a copy - [`number_token`](number_token.md) - a number's token text, without a copy
- [`BasicJsonType::get_ref`](../basic_json/get_ref.md) - the closest counterpart of `basic_json` - [`BasicJsonType::get_ref`](../basic_json/get_ref.md) - the closest counterpart of `basic_json`
- [Zero-copy JSON views](../../features/json_view/reading.md#getting-values-out-without-copying) - borrowing strings instead of copying them - [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - borrowing strings instead of copying them
## Version history ## Version history
@@ -75,7 +75,7 @@ them with those of a default-constructed `T`.
- [`get`](get.md) - convert the value to a given type - [`get`](get.md) - convert the value to a given type
- [`BasicJsonType::get_to`](../basic_json/get_to.md) - the corresponding function of `basic_json` - [`BasicJsonType::get_to`](../basic_json/get_to.md) - the corresponding function of `basic_json`
- [Zero-copy JSON views](../../features/json_view/reading.md#getting-values-out-without-copying) - which conversions avoid building a basic_json value - [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - which conversions avoid building a basic_json value
## Version history ## Version history
@@ -147,7 +147,7 @@ still refers to it — including ones taken before the change — reads the new
## See also ## See also
- [Zero-copy JSON views](../../features/json_view/index.md) - guide to documents and views - [Zero-copy JSON views](../../features/json_view.md) - guide to documents and views
- [`basic_json_document`](../basic_json_document/index.md) - the document that owns the parsed data - [`basic_json_document`](../basic_json_document/index.md) - the document that owns the parsed data
- [`json_view`](../json_view.md) - and [`json_editable_view`](../json_editable_view.md) - the default specializations - [`json_view`](../json_view.md) - and [`json_editable_view`](../json_editable_view.md) - the default specializations
- [`materialize`](materialize.md) - build a `basic_json` value from a view - [`materialize`](materialize.md) - build a `basic_json` value from a view
@@ -54,7 +54,7 @@ A `basic_json_view` is not convertible to `#!cpp bool`: such a conversion would
- [`(constructor)`](basic_json_view.md) - the default constructor creates a discarded view - [`(constructor)`](basic_json_view.md) - the default constructor creates a discarded view
- [`basic_json_document::is_discarded`](../basic_json_document/is_discarded.md) - return whether the last parse failed - [`basic_json_document::is_discarded`](../basic_json_document/is_discarded.md) - return whether the last parse failed
- [`BasicJsonType::is_discarded`](../basic_json/is_discarded.md) - the corresponding function of `basic_json` - [`BasicJsonType::is_discarded`](../basic_json/is_discarded.md) - the corresponding function of `basic_json`
- [Zero-copy JSON views](../../features/json_view/differences.md) - how views differ from basic_json - [Zero-copy JSON views](../../features/json_view.md#what-is-different) - how views differ from basic_json
## Version history ## Version history
@@ -78,7 +78,7 @@ never caches the result.
- [`root`](../basic_json_document/root.md) - the view of a document's root value - [`root`](../basic_json_document/root.md) - the view of a document's root value
- [`BasicJsonType::parse`](../basic_json/parse.md) - build a `BasicJsonType` value directly from a JSON text - [`BasicJsonType::parse`](../basic_json/parse.md) - build a `BasicJsonType` value directly from a JSON text
- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values - [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values
- [Zero-copy JSON views](../../features/json_view/reading.md#getting-values-out-without-copying) - when to build a basic_json value from a view - [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - when to build a basic_json value from a view
## Version history ## Version history
@@ -45,7 +45,7 @@ source
- [`number_token`](number_token.md) - get a single number's token text without dumping the whole value - [`number_token`](number_token.md) - get a single number's token text without dumping the whole value
- [`BasicJsonType::error_handler_t`](../basic_json/error_handler_t.md) - the analogous enumeration for - [`BasicJsonType::error_handler_t`](../basic_json/error_handler_t.md) - the analogous enumeration for
`BasicJsonType::dump`'s decoding-error behavior `BasicJsonType::dump`'s decoding-error behavior
- [Zero-copy JSON views](../../features/json_view/reading.md#writing-a-view-back) - writing numbers as they appeared in the source - [Zero-copy JSON views](../../features/json_view.md#writing-a-view-back) - writing numbers as they appeared in the source
## Version history ## Version history
@@ -68,7 +68,7 @@ buffer — and is valid exactly as long as that text is, see the [validity rules
- [`BasicJsonType::number_integer_t`](../basic_json/number_integer_t.md), - [`BasicJsonType::number_integer_t`](../basic_json/number_integer_t.md),
[`number_unsigned_t`](../basic_json/number_unsigned_t.md), [`number_float_t`](../basic_json/number_float_t.md) - the [`number_unsigned_t`](../basic_json/number_unsigned_t.md), [`number_float_t`](../basic_json/number_float_t.md) - the
number types `#!cpp get<T>()` and `materialize()` convert into number types `#!cpp get<T>()` and `materialize()` convert into
- [Zero-copy JSON views](../../features/json_view/reading.md#getting-values-out-without-copying) - reading a number exactly as written - [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - reading a number exactly as written
## Version history ## Version history
@@ -120,7 +120,7 @@ document.
[`materialize()`](materialize.md) or iterate the members with [`items()`](items.md) and keep the last match. [`materialize()`](materialize.md) or iterate the members with [`items()`](items.md) and keep the last match.
The key hash index of a larger object (128 members or more) leads to the first member of a key as well. The key hash index of a larger object (128 members or more) leads to the first member of a key as well.
[`begin()`](begin.md)/[`end()`](end.md) and [`items()`](items.md) iterate over *all* members, including [`begin()`](begin.md)/[`end()`](end.md) and [`items()`](items.md) iterate over *all* members, including
duplicates, in document order. See [Duplicate keys](../../features/json_view/differences.md#duplicate-keys) and duplicates, in document order. See [Duplicate keys](../../features/json_view.md#duplicate-keys) and
[`size()`](size.md#notes). [`size()`](size.md#notes).
!!! info "JSON pointer resolution" !!! info "JSON pointer resolution"
@@ -19,7 +19,7 @@ Neither overload builds a `BasicJsonType` value for a view to do the comparison
compare by value across their types (`#!cpp 1 == 1.0`), and an object compares by its members, with duplicate keys compare by value across their types (`#!cpp 1 == 1.0`), and an object compares by its members, with duplicate keys
resolved exactly as `parse()` resolves them — the last value, at the position of the first occurrence of the key. resolved exactly as `parse()` resolves them — the last value, at the position of the first occurrence of the key.
This is not what [`operator[]`](operator[].md) and the other lookups return for a duplicate key: they find the *first* This is not what [`operator[]`](operator[].md) and the other lookups return for a duplicate key: they find the *first*
member (see [Duplicate keys](../../features/json_view/differences.md#duplicate-keys)). member (see [Duplicate keys](../../features/json_view.md#duplicate-keys)).
## Parameters ## Parameters
@@ -68,7 +68,7 @@ Linear, as [`dump`](dump.md#complexity).
- [`dump`](dump.md) - serialize to a JSON-formatted string - [`dump`](dump.md) - serialize to a JSON-formatted string
- [`operator<<(std::ostream&)`](../operator_ltlt.md) - the corresponding operator for `basic_json` - [`operator<<(std::ostream&)`](../operator_ltlt.md) - the corresponding operator for `basic_json`
- [`JSON_NO_IO`](../macros/json_no_io.md) - switch off functions relying on certain C++ I/O headers - [`JSON_NO_IO`](../macros/json_no_io.md) - switch off functions relying on certain C++ I/O headers
- [Zero-copy JSON views](../../features/json_view/reading.md#writing-a-view-back) - writing a view back without building a tree - [Zero-copy JSON views](../../features/json_view.md#writing-a-view-back) - writing a view back without building a tree
## Version history ## Version history
@@ -49,7 +49,7 @@ string, whose token length is not the same as its decoded [`size()`](size.md).
- [`source`](../basic_json_document/source.md) - the parsed text - [`source`](../basic_json_document/source.md) - the parsed text
- [`is_string`](is_string.md) - return whether the value is a string - [`is_string`](is_string.md) - return whether the value is a string
- [Zero-copy JSON views](../../features/json_view/images.md) - images and the source text - [Zero-copy JSON views](../../features/json_view.md#images) - images and the source text
## Version history ## Version history
@@ -84,7 +84,7 @@ Pointer path of the value within the document, and with
If the source text repeats a key, this resolves to the *first* member with it, not to the last one that If the source text repeats a key, this resolves to the *first* member with it, not to the last one that
[`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of [`materialize()`](materialize.md) and `parse()` keep. See the [Notes on duplicate keys](operator[].md#notes) of
`operator[]` and [Duplicate keys](../../features/json_view/differences.md#duplicate-keys) for the reasons and for how to get `operator[]` and [Duplicate keys](../../features/json_view.md#duplicate-keys) for the reasons and for how to get
the last value. the last value.
!!! info "Differences to `at` and `operator[]`" !!! info "Differences to `at` and `operator[]`"
+1 -1
View File
@@ -49,7 +49,7 @@ view class.
- [basic_json_view](basic_json_view/index.md) - a handle to one value of a document - [basic_json_view](basic_json_view/index.md) - a handle to one value of a document
- [get](basic_json_view/get.md) - which conversions read the view directly - [get](basic_json_view/get.md) - which conversions read the view directly
- [view_mode](view_mode.md) - read-only or editable - [view_mode](view_mode.md) - read-only or editable
- [Writing generic code](../features/json_view/generic_code.md) - functions and `from_json()` for views - [Writing generic code](../features/json_view.md#writing-generic-code) - functions and `from_json()` for views
## Version history ## Version history
+1 -1
View File
@@ -29,7 +29,7 @@ specialization.
- [`json_view`](json_view.md) - a view of a value of a `json_document` - [`json_view`](json_view.md) - a view of a value of a `json_document`
- [`ordered_json_document`](ordered_json_document.md) - the corresponding document for `ordered_json` - [`ordered_json_document`](ordered_json_document.md) - the corresponding document for `ordered_json`
- [Zero-copy JSON views](../features/json_view/index.md) - how to parse, inspect and edit a document without building a tree - [Zero-copy JSON views](../features/json_view.md) - how to parse, inspect and edit a document without building a tree
## Version history ## Version history
@@ -10,7 +10,7 @@ This type is an **editable** [`basic_json_document`](basic_json_document/index.m
[`json`](json.md) specialization: in addition to everything [`json_document`](json_document.md) offers, [`json`](json.md) specialization: in addition to everything [`json_document`](json_document.md) offers,
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) change values after [`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) change values after
parsing, without ever rewriting the source text — see [Edits](basic_json_document/index.md#edits) and parsing, without ever rewriting the source text — see [Edits](basic_json_document/index.md#edits) and
[Editing a document](../features/json_view/editing.md). [Editing a document](../features/json_view.md#editing-a-document).
## Examples ## Examples
+1 -1
View File
@@ -35,7 +35,7 @@ still referring to it seeing the change, see [Edits](basic_json_document/index.m
- [`json_view`](json_view.md) - the corresponding read-only view - [`json_view`](json_view.md) - the corresponding read-only view
- [`ordered_json_editable_view`](ordered_json_editable_view.md) - the corresponding view for - [`ordered_json_editable_view`](ordered_json_editable_view.md) - the corresponding view for
`ordered_json_editable_document` `ordered_json_editable_document`
- [Zero-copy JSON views](../features/json_view/editing.md) - editing a document in place and writing it back - [Zero-copy JSON views](../features/json_view.md#editing-a-document) - editing a document in place and writing it back
## Version history ## Version history
+2 -1
View File
@@ -7,7 +7,8 @@ struct json_sax;
This class describes the SAX interface used by [sax_parse](../basic_json/sax_parse.md). Each function is called in This class describes the SAX interface used by [sax_parse](../basic_json/sax_parse.md). Each function is called in
different situations while the input is parsed. The boolean return value informs the parser whether to continue different situations while the input is parsed. The boolean return value informs the parser whether to continue
processing the input. processing the input; for [`parse_error`](parse_error.md), it decides whether to
[recover from the error](../../features/parsing/error_recovery.md).
For instance, parsing the JSON text `{"a": [1, true]}` triggers the following callbacks, in order: For instance, parsing the JSON text `{"a": [1, true]}` triggers the following callbacks, in order:
+26 -2
View File
@@ -21,11 +21,18 @@ A parse error occurred.
## Return value ## Return value
Whether parsing should proceed (**must return `#!cpp false`**). Whether to recover from the error:
- `#!cpp false` stops parsing.
- `#!cpp true` recovers from the error: the error is repaired and parsing continues. If that is not possible, which
happens in the binary formats when the end of the item with the error is unknown, the value read so far is completed
and parsing stops. See [error recovery](../../features/parsing/error_recovery.md) for how errors are repaired.
Either way, [`sax_parse`](../basic_json/sax_parse.md) returns `#!cpp false`.
## Examples ## Examples
??? example ??? example "Example: (1) the SAX interface"
The example below shows how the SAX interface is used. The example below shows how the SAX interface is used.
@@ -39,13 +46,30 @@ Whether parsing should proceed (**must return `#!cpp false`**).
--8<-- "examples/sax_parse.output" --8<-- "examples/sax_parse.output"
``` ```
??? example "Example: (2) recovering from errors"
The example below shows how a SAX parser recovers from errors.
```cpp
--8<-- "examples/sax_parse__error_recovery.cpp"
```
Output:
```
--8<-- "examples/sax_parse__error_recovery.output"
```
## See also ## See also
- [`sax_parse`](../basic_json/sax_parse.md) - SAX parser - [`sax_parse`](../basic_json/sax_parse.md) - SAX parser
- [Parsing and Exceptions](../../features/parsing/parse_exceptions.md) - the article on handling parse errors without - [Parsing and Exceptions](../../features/parsing/parse_exceptions.md) - the article on handling parse errors without
exceptions exceptions
- [Error Recovery](../../features/parsing/error_recovery.md) - the article on recovering from parse errors
- [SAX Interface](../../features/parsing/sax_interface.md) - guide to event-based parsing - [SAX Interface](../../features/parsing/sax_interface.md) - guide to event-based parsing
## Version history ## Version history
- Added in version 3.2.0. - Added in version 3.2.0.
- Returning `#!cpp true` recovers from the error since version 3.13.0; before, parsing stopped, but the result of
[`sax_parse`](../basic_json/sax_parse.md) could be wrong.
+1 -1
View File
@@ -28,7 +28,7 @@ This type is a [`basic_json_view`](basic_json_view/index.md) of a value of a [`j
- [`json_document`](json_document.md) - the document type this view refers into - [`json_document`](json_document.md) - the document type this view refers into
- [`ordered_json_view`](ordered_json_view.md) - the corresponding view for `ordered_json_document` - [`ordered_json_view`](ordered_json_view.md) - the corresponding view for `ordered_json_document`
- [Zero-copy JSON views](../features/json_view/index.md) - how to parse, inspect and edit a document without building a tree - [Zero-copy JSON views](../features/json_view.md) - how to parse, inspect and edit a document without building a tree
## Version history ## Version history
@@ -35,9 +35,6 @@ Note that enabling this macro increases the size of every JSON value by two `std
overhead to parsing, copying JSON value objects, and the generation of error messages for exceptions. It also causes overhead to parsing, copying JSON value objects, and the generation of error messages for exceptions. It also causes
these values to be reported in those error messages. these values to be reported in those error messages.
If [`JSON_DIAGNOSTICS`](json_diagnostics.md) is also set, the message contains the path and the byte range, such as
`(/1) (bytes 4-8)`. An exception about the root value has no path and only the range, such as `(bytes 0-4)`.
## Default definition ## Default definition
The default value is `0` (position diagnostics are switched off). The default value is `0` (position diagnostics are switched off).
+1 -1
View File
@@ -80,7 +80,7 @@ By default, `#!cpp JSON_NO_SIMD` is not defined, and vector instructions are use
- [Parsing: input sources and performance](../../features/parsing/index.md#input-sources-and-performance) - how input - [Parsing: input sources and performance](../../features/parsing/index.md#input-sources-and-performance) - how input
sources affect parsing performance sources affect parsing performance
- [Serialization](../../features/serialization.md) - pretty-printing, UTF-8 handling, and number output - [Serialization](../../features/serialization.md) - pretty-printing, UTF-8 handling, and number output
- [Zero-copy JSON views](../../features/json_view/index.md) - the zero-copy view, whose parser the macro also affects - [Zero-copy JSON views](../../features/json_view.md) - the zero-copy view, whose parser the macro also affects
## Version history ## Version history
@@ -32,7 +32,7 @@ of object keys, instead of sorting them like [`json_document`](json_document.md)
- [`ordered_json_view`](ordered_json_view.md) - a view of a value of an `ordered_json_document` - [`ordered_json_view`](ordered_json_view.md) - a view of a value of an `ordered_json_document`
- [`json_document`](json_document.md) - the corresponding document for the default `json` specialization - [`json_document`](json_document.md) - the corresponding document for the default `json` specialization
- [Object Order](../features/object_order.md) - preserving the insertion order of object keys - [Object Order](../features/object_order.md) - preserving the insertion order of object keys
- [Zero-copy JSON views](../features/json_view/index.md) - how to parse, inspect and edit a document without building a tree - [Zero-copy JSON views](../features/json_view.md) - how to parse, inspect and edit a document without building a tree
## Version history ## Version history
@@ -41,7 +41,7 @@ members [`set`](basic_json_document/set.md) added — instead of sorting them li
specialization specialization
- [Object Order](../features/object_order.md) - preserving the insertion order of object keys - [Object Order](../features/object_order.md) - preserving the insertion order of object keys
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees - [Edits](basic_json_document/index.md#edits) - what an edit guarantees
- [Zero-copy JSON views](../features/json_view/editing.md) - editing a document in place and writing it back - [Zero-copy JSON views](../features/json_view.md#editing-a-document) - editing a document in place and writing it back
## Version history ## Version history
@@ -34,7 +34,7 @@ This type is a [`basic_json_view`](basic_json_view/index.md) of a value of an
- [`ordered_json_editable_document`](ordered_json_editable_document.md) - the document type this view refers into - [`ordered_json_editable_document`](ordered_json_editable_document.md) - the document type this view refers into
- [`ordered_json_view`](ordered_json_view.md) - the corresponding read-only view - [`ordered_json_view`](ordered_json_view.md) - the corresponding read-only view
- [`json_editable_view`](json_editable_view.md) - the corresponding view for `json_editable_document` - [`json_editable_view`](json_editable_view.md) - the corresponding view for `json_editable_document`
- [Zero-copy JSON views](../features/json_view/editing.md) - editing a document in place and writing it back - [Zero-copy JSON views](../features/json_view.md#editing-a-document) - editing a document in place and writing it back
## Version history ## Version history
+1 -1
View File
@@ -29,7 +29,7 @@ This type is a [`basic_json_view`](basic_json_view/index.md) of a value of an
- [`ordered_json_document`](ordered_json_document.md) - the document type this view refers into - [`ordered_json_document`](ordered_json_document.md) - the document type this view refers into
- [`json_view`](json_view.md) - the corresponding view for `json_document` - [`json_view`](json_view.md) - the corresponding view for `json_document`
- [Zero-copy JSON views](../features/json_view/index.md) - how to parse, inspect and edit a document without building a tree - [Zero-copy JSON views](../features/json_view.md) - how to parse, inspect and edit a document without building a tree
## Version history ## Version history
+1 -1
View File
@@ -82,7 +82,7 @@ nor a [`json_editable_view`](json_editable_view.md). Views of the two modes comp
- [basic_json_view](basic_json_view/index.md) - a handle to one value of a document - [basic_json_view](basic_json_view/index.md) - a handle to one value of a document
- [json_editable_document](json_editable_document.md) - an editable document of `json` - [json_editable_document](json_editable_document.md) - an editable document of `json`
- [(constructor)](basic_json_view/basic_json_view.md) - the conversion from a read-only to an editable view - [(constructor)](basic_json_view/basic_json_view.md) - the conversion from a read-only to an editable view
- [Writing generic code](../features/json_view/generic_code.md) - functions that accept views - [Writing generic code](../features/json_view.md#writing-generic-code) - functions that accept views
## Version history ## Version history
+1 -1
View File
@@ -1,4 +1,4 @@
/* figures: jsonview (features/json_view/index.md; SVGs from scripts/figures/json_view_layout.py, stepper in js/fig-jsonview.js) */ /* figures: jsonview (features/json_view.md; SVGs from scripts/figures/json_view_layout.py, stepper in js/fig-jsonview.js) */
/* one colour per concept, the same in every figure: structure (object/array, views) blue, keys navy, strings green, /* one colour per concept, the same in every figure: structure (object/array, views) blue, keys navy, strings green,
numbers orange, decoded strings purple, owned edit storage teal, dangling/invalid red */ numbers orange, decoded strings purple, owned edit storage teal, dangling/invalid red */
@@ -0,0 +1,43 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
// a SAX parser that creates a JSON value like json::parse does, but that
// recovers from parse errors instead of stopping at the first one
class recovering_parser : public nlohmann::detail::json_sax_dom_parser<json>
{
public:
explicit recovering_parser(json& result)
: nlohmann::detail::json_sax_dom_parser<json>(result, false)
{}
bool parse_error(std::size_t position,
const std::string& /*last_token*/,
const json::exception& ex)
{
std::cout << "byte " << position << ": " << ex.what() << '\n';
// repair the input and continue
return true;
}
};
int main()
{
// JSON text with several mistakes that ends too early
const std::string text = R"({
"name": "Hello World",
"tags": ["a" "b",],
"valid": tru,
"size": 1.,
"nested": {"x": 1)";
json result;
recovering_parser sax(result);
const bool valid = json::sax_parse(text, &sax);
std::cout << "\nvalid JSON: " << std::boolalpha << valid << '\n'
<< std::setw(4) << result << std::endl;
}
@@ -0,0 +1,19 @@
byte 49: [json.exception.parse_error.101] parse error at line 3, column 20: syntax error while parsing array - unexpected string literal; expected ']'
byte 51: [json.exception.parse_error.101] parse error at line 3, column 22: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal
byte 70: [json.exception.parse_error.101] parse error at line 4, column 17: syntax error while parsing value - invalid literal; last read: '"valid": tru,'
byte 86: [json.exception.parse_error.101] parse error at line 5, column 15: syntax error while parsing value - invalid number; expected digit after '.'; last read: '1.,'
byte 109: [json.exception.parse_error.101] parse error at line 6, column 22: syntax error while parsing object - unexpected end of input; expected '}'
valid JSON: false
{
"name": "Hello World",
"nested": {
"x": 1
},
"size": 1,
"tags": [
"a",
"b"
],
"valid": null
}
+2 -2
View File
@@ -48,7 +48,7 @@ which avoids repeating it:
[`number_token()`](../api/basic_json_view/number_token.md) returns a number exactly as it was written. Every other [`number_token()`](../api/basic_json_view/number_token.md) returns a number exactly as it was written. Every other
type is obtained from a subtree that is built on demand with type is obtained from a subtree that is built on demand with
[`materialize()`](../api/basic_json_view/materialize.md). See [`materialize()`](../api/basic_json_view/materialize.md). See
[Zero-copy JSON views](json_view/reading.md#getting-values-out-without-copying). [Zero-copy JSON views](json_view.md#getting-values-out-without-copying).
The library already knows how to convert to and from the scalar types and the STL containers (such as The library already knows how to convert to and from the scalar types and the STL containers (such as
`#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::array`, `#!cpp std::optional`, and many more). Converting a JSON `#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::array`, `#!cpp std::optional`, and many more). Converting a JSON
@@ -269,7 +269,7 @@ The conversions above are built in for standard types. To make the same syntax w
- [`get_ref`](../api/basic_json/get_ref.md) - / [`get_ptr`](../api/basic_json/get_ptr.md) - access the stored value without copying - [`get_ref`](../api/basic_json/get_ref.md) - / [`get_ptr`](../api/basic_json/get_ptr.md) - access the stored value without copying
- [Arbitrary types conversions](arbitrary_types.md) - support your own types - [Arbitrary types conversions](arbitrary_types.md) - support your own types
- [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) - toggle implicit conversions - [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) - toggle implicit conversions
- [Zero-copy JSON views](json_view/index.md) - get values out of a document or a view - [Zero-copy JSON views](json_view.md) - get values out of a document or a view
- [`adl_serializer`](../api/adl_serializer/index.md) - the default serializer behind the conversions - [`adl_serializer`](../api/adl_serializer/index.md) - the default serializer behind the conversions
- [`type`](../api/basic_json/type.md) - query the type of a value - [`type`](../api/basic_json/type.md) - query the type of a value
- [`JSON_DISABLE_TUPLE_REFERENCE_CONVERSION`](../api/macros/json_disable_tuple_reference_conversion.md) - toggle conversion to tuples of references - [`JSON_DISABLE_TUPLE_REFERENCE_CONVERSION`](../api/macros/json_disable_tuple_reference_conversion.md) - toggle conversion to tuples of references
+2 -2
View File
@@ -3,7 +3,7 @@
There are several ways to create a JSON value in memory. This page gives an overview; to read a value from JSON text There are several ways to create a JSON value in memory. This page gives an overview; to read a value from JSON text
instead, see [parsing](parsing/index.md). A JSON text can also be parsed into a instead, see [parsing](parsing/index.md). A JSON text can also be parsed into a
[document](../api/basic_json_document/index.md) and read through a [view](../api/basic_json_view/index.md) instead of [document](../api/basic_json_document/index.md) and read through a [view](../api/basic_json_view/index.md) instead of
into a value; see [Zero-copy JSON views](json_view/index.md). into a value; see [Zero-copy JSON views](json_view.md).
## From C++ values ## From C++ values
@@ -122,7 +122,7 @@ string `#!json "42"`.
- [`operator""_json`](../api/operator_literal_json.md) - the `_json` literal - [`operator""_json`](../api/operator_literal_json.md) - the `_json` literal
- [Converting values](conversions.md) - which C++ types can be used - [Converting values](conversions.md) - which C++ types can be used
- [Parsing](parsing/index.md) - create a value from JSON text - [Parsing](parsing/index.md) - create a value from JSON text
- [Zero-copy JSON views](json_view/index.md) - create a document and a view from JSON text instead of a value - [Zero-copy JSON views](json_view.md) - create a document and a view from JSON text instead of a value
- [`basic_json`](../api/basic_json/index.md) - the class template and its member functions - [`basic_json`](../api/basic_json/index.md) - the class template and its member functions
- [Types](types/index.md) - the JSON value types - [Types](types/index.md) - the JSON value types
- [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) - change how a single braced value is interpreted - [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) - change how a single braced value is interpreted
+3 -3
View File
@@ -13,10 +13,10 @@ C++ types, and finally serialize it again.
[SAX interface](parsing/sax_interface.md), [error handling](parsing/parse_exceptions.md), [SAX interface](parsing/sax_interface.md), [error handling](parsing/parse_exceptions.md),
[large and partial input](parsing/large_and_partial_input.md), and [large and partial input](parsing/large_and_partial_input.md), and
[parsing untrusted input](parsing/untrusted_input.md). [parsing untrusted input](parsing/untrusted_input.md).
- [Zero-copy JSON views](json_view/index.md) — read a JSON text through a flat index instead of building a `json` tree; - [Zero-copy JSON views](json_view.md) — read a JSON text through a flat index instead of building a `json` tree;
strings and numbers stay in the input and are only decoded when needed. strings and numbers stay in the input and are only decoded when needed.
- [Editable documents](json_view/editing.md) — modify a parsed document without building a `json` tree. - [Editable documents](json_view.md#editing-a-document) — modify a parsed document without building a `json` tree.
- [Images](json_view/images.md) — save a parsed document and load it again without parsing it. - [Images](json_view.md#images) — save a parsed document and load it again without parsing it.
- [Comments](comments.md) and [trailing commas](trailing_commas.md) — opt-in relaxations of the JSON grammar. - [Comments](comments.md) and [trailing commas](trailing_commas.md) — opt-in relaxations of the JSON grammar.
## Accessing and modifying values ## Accessing and modifying values
+589
View File
@@ -0,0 +1,589 @@
# Zero-copy JSON views
`#!cpp <nlohmann/json_view.hpp>` adds a read-only, non-owning way to look at a parsed JSON text, as an alternative to
building a [`basic_json`](../api/basic_json/index.md) tree with [`parse()`](../api/basic_json/parse.md).
## The problem
[`basic_json::parse()`](../api/basic_json/parse.md) builds a tree of `basic_json` values: one allocation for every
array and object, and every string copied into its own `std::string`. That is the right trade-off when the program
goes on to read and write the value freely, but it does more work than necessary when only a small part of a large
JSON text is actually needed, or when the same text is parsed over and over (many small messages, for instance) and
most of the resulting tree is thrown away almost immediately.
## The idea
[`basic_json_document::parse()`](../api/basic_json_document/parse.md) parses the same JSON grammar, with the same
options, but instead of a tree it builds a flat index of the values it found: one
[16-byte entry](../home/architecture.md#node-index-of-json-views) per value (and one per object key), in document
order. Integers are converted while parsing and stored in their entry. Floating-point numbers and strings are
referenced in place in the source text: a float is converted when it is read, and only strings that contain escape
sequences are decoded, into one shared buffer owned by the document.
[`basic_json_view`](../api/basic_json_view/index.md) is a small, trivially copyable handle (two pointers) into that
index. It gives you the read-only, type-inspection part of the `basic_json` interface —
[`type()`](../api/basic_json_view/type.md) and the `is_*()` predicates,
[`size()`](../api/basic_json_view/size.md)/[`empty()`](../api/basic_json_view/empty.md) — as well as element access
([`operator[]`](../api/basic_json_view/operator%5B%5D.md), [`at`](../api/basic_json_view/at.md),
[`front`](../api/basic_json_view/front.md)/[`back`](../api/basic_json_view/back.md)), lookup
([`find`](../api/basic_json_view/find.md), [`contains`](../api/basic_json_view/contains.md),
[`count`](../api/basic_json_view/count.md)), and iteration
([`begin`](../api/basic_json_view/begin.md)/[`end`](../api/basic_json_view/end.md),
[`items`](../api/basic_json_view/items.md)) — without ever allocating a `basic_json` value. When you do need an
actual `basic_json` value for a subtree, [`materialize()`](../api/basic_json_view/materialize.md) builds exactly the
one [`parse()`](../api/basic_json/parse.md) would have produced for it.
The same small text, parsed both ways:
<div class="nj-fig-wrap" markdown>
--8<-- "images/json_view_layout.svg"
</div>
<p class="nj-caption" markdown>Top: the tree of separate heap allocations that `basic_json::parse()` builds. Bottom: `json_document::parse()` keeps the source text and builds one flat index of 16-byte nodes that point back into it, plus a buffer of decoded strings.</p>
## How to use it
Include `<nlohmann/json_view.hpp>` in addition to (or instead of) `<nlohmann/json.hpp>`. Parse into a
[`json_document`](../api/json_document.md), inspect its [`root()`](../api/basic_json_document/root.md), and
`materialize()` when you need a real value. `materialize()` works on any view, not just the root, so you can turn just
the subtree you need into a value:
??? example "Example: parse a document, inspect its root, and materialize it"
```cpp
--8<-- "examples/json_document.cpp"
```
Output:
```json
--8<-- "examples/json_document.output"
```
[`ordered_json_document`](../api/ordered_json_document.md) is the equivalent for
[`ordered_json`](../api/ordered_json.md), just as [`ordered_json`](../api/ordered_json.md) is to
[`json`](../api/json.md).
## Ownership and lifetime
A document either **borrows** the text it was parsed from, or **owns** its own copy of it; call
[`owns_source()`](../api/basic_json_document/owns_source.md) to find out which happened.
[`parse()`](../api/basic_json_document/parse.md) decides this from the value category and type of its argument (an
lvalue `#!cpp std::string` is borrowed; an rvalue `#!cpp std::string` is moved in, owned without a copy; a stream is
read into an owned buffer; and so on — see [`parse`'s Notes](../api/basic_json_document/parse.md#notes) for the full
table). [`parse_copy()`](../api/basic_json_document/parse_copy.md) always owns a copy, regardless of the input.
!!! warning "A borrowed document depends on your buffer"
If a document borrows its text, that text **must outlive the document** (and every view taken from it) and must
stay **unmodified** for as long as the document exists, since floating-point values and unescaped strings are
read from it lazily. Reading or writing through a view after the underlying buffer is gone is undefined behavior,
exactly as it would be for a dangling `#!cpp std::string_view`.
A view is valid only while all of the following hold:
- the document is alive,
- the document has not been re-parsed since the view was taken (with [`read()`](../api/basic_json_document/read.md)
or [`parse()`](../api/basic_json_document/parse.md) into it),
- the document has not had [`shrink_to_fit()`](../api/basic_json_document/shrink_to_fit.md) called on it since, and
- if the document borrows its source text, that text is still alive and unmodified.
Moving the `basic_json_document<>` itself is fine and does **not** invalidate its views: the index is a separate heap
allocation that keeps its address across the move. Take a fresh view from [`root()`](../api/basic_json_document/root.md)
whenever any of the other conditions above was not met.
<div class="nj-fig-wrap" markdown>
--8<-- "images/json_view_lifetime.svg"
</div>
<p class="nj-caption" markdown>Four timelines of a document and its views, each bar showing when an object is alive or valid and when it dangles. Destroying the borrowed text, re-parsing, or shrinking the document invalidates the views taken before; moving the document does not.</p>
A view can dangle, exactly like a `#!cpp std::string_view`: nothing stops you from using it after its document is
gone. This is deliberate. A view is two pointers and trivially copyable; making it safe (a `#!cpp std::shared_ptr` in
the document and a `#!cpp std::weak_ptr` in each view) would make it larger, add reference counting to every copy,
and a `lock()` to every access — exactly the costs the view exists to avoid. What is caught is the obvious case:
because a view dies with its document, [`root()`](../api/basic_json_document/root.md) is not callable on a temporary
document: `#!cpp auto v = json_document::parse(text).root();` does not compile. Give the document a name first.
??? example "Example: borrowed and owned documents, and when views become invalid"
```cpp
--8<-- "examples/json_view_ownership.cpp"
```
Output:
```json
--8<-- "examples/json_view_ownership.output"
```
## What is the same as `parse()`
- **Accept/reject.** [`accept()`](../api/basic_json_document/accept.md) and
[`parse()`](../api/basic_json_document/parse.md) accept and reject exactly the same inputs as
[`basic_json::accept()`](../api/basic_json/accept.md)/[`basic_json::parse()`](../api/basic_json/parse.md), with the
same `ignore_comments` and `ignore_trailing_commas` options.
- **Errors.** A failing parse throws the same exception — the same id, message, and position — because on failure
the library's own parser is run on the same bytes to produce the diagnostic. Valid input never pays for this, and
the replay builds no values: it costs about a lexing pass up to the error.
`#!cpp allow_exceptions == false` skips the replay and gives a
[discarded](../api/basic_json_document/is_discarded.md) document instead of throwing, just as it gives a discarded
value for `#!cpp basic_json::parse()`.
- **Number classification.** An integer literal that does not fit into the 64-bit integer type becomes a
floating-point value, exactly as it does for `#!cpp basic_json::parse()`.
- **Macros.** [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) and
[`JSON_NOEXCEPTION`](../api/macros/json_noexception.md)/[`JSON_THROW_USER`](../api/macros/json_throw_user.md)
behave the same way they do for `<nlohmann/json.hpp>`. These are the macros that change what the parser accepts or
how it reports errors. Two more macros add context to the exceptions of views, as they do for those of
`#!cpp basic_json`:
- With [`JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md), an exception names the JSON Pointer path of the
value within the document.
- With [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md), an exception names the bytes of
the value in the source text.
A view has neither parent pointers nor stored positions, so both are found from the document's index when the
exception is thrown. This takes time linear in the size of the document. One exception has neither:
`key()` of an iterator over an array or a primitive value
([`invalid_iterator.207`](../home/exceptions.md#jsonexceptioninvalid_iterator207)), because a view's iterator does
not know the value it iterates over.
The other macros only affect the values `materialize()` produces, through `BasicJsonType`.
## What is different
- **Only 64-bit integers.** `basic_json_document<BasicJsonType>` requires `BasicJsonType::number_integer_t` and
`number_unsigned_t` to both be 64 bits wide; this is a compile-time `#!cpp static_assert`.
- **A 4 GiB input limit.** An input of 4294967280 bytes (4 GiB minus 16 bytes) or more throws
[`out_of_range.416`](../home/exceptions.md#jsonexceptionout_of_range416), a limit
`#!cpp basic_json::parse()` does not have. It keeps an index entry at 16 bytes: offsets and counts are 32 bits
wide. The limit is 16 bytes below 2^32, because the parser reads in 16-byte steps, and a position one step past the
end must still fit.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
is read to its end into a buffer the document owns, and parsing starts after that.
- **Fewer source positions after edits.** With
[`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
[`materialize()`](../api/basic_json_view/materialize.md) sets the same positions as `#!cpp basic_json::parse()`.
In an [editable document](../api/json_editable_document.md), a value written by an edit has none, and neither has
an array or object whose elements changed or whose last element has none; a `basic_json` value keeps its positions
when it is changed. See [`materialize()`](../api/basic_json_view/materialize.md#notes).
- **Objects iterate in document order.** [`begin()`](../api/basic_json_view/begin.md)/
[`end()`](../api/basic_json_view/end.md) and [`items()`](../api/basic_json_view/items.md) visit an object's
members in the order they appear in the source text. `basic_json`'s default `object_t` is a `std::map`, which
sorts by key, so iterating a [`materialize()`](../api/basic_json_view/materialize.md)d value can print members in
a different order than iterating the view they came from.
- **Chained access is safe.** [`operator[]`](../api/basic_json_view/operator%5B%5D.md) with a missing key, an index
out of range, or an unresolvable JSON pointer returns a [discarded](../api/basic_json_view/is_discarded.md) view, and
`operator[]` on a discarded view returns a discarded view without throwing: `#!cpp v["a"]["b"][0]` can be tested
once at the end. Type errors on values that exist (a key on an array, an index on an object) still throw, and
[`at`](../api/basic_json_view/at.md) throws for every missing value.
- **Duplicate keys: lookups find the first member, `parse()` keeps the last.** See [Duplicate keys](#duplicate-keys)
below.
- **Ordering comparisons are not provided** by `basic_json_view` — there is no `#!cpp operator<`.
[`operator==`](../api/basic_json_view/operator_eq.md) and [`operator!=`](../api/basic_json_view/operator_ne.md) are
provided, though: two views, or a view and a `BasicJsonType` value, compare equal exactly when
[`materialize()`](../api/basic_json_view/materialize.md) or [`parse()`](../api/basic_json/parse.md) would produce
equal values for them, without ever building a tree to do it. For ordering, too,
[`materialize()`](../api/basic_json_view/materialize.md) is the way to get a value you can compare.
## Duplicate keys
!!! warning "A lookup in a view and in the parsed value can give different answers"
If an object in the source text repeats a key, [`operator[]`](../api/basic_json_view/operator%5B%5D.md),
[`at`](../api/basic_json_view/at.md), [`find`](../api/basic_json_view/find.md),
[`value`](../api/basic_json_view/value.md), [`contains`](../api/basic_json_view/contains.md), and JSON pointer
resolution all return the **first** member with that key. `basic_json::parse()` instead keeps the **last** value
of a repeated key, so for `#!cpp {"a":1,"a":2}`, `#!cpp view["a"]` is `#!cpp 1` while `#!cpp parse(text)["a"]` is
`#!cpp 2`. [`count`](../api/basic_json_view/count.md) counts every member with the key, so it is `#!cpp 2` here.
[`begin()`](../api/basic_json_view/begin.md)/[`end()`](../api/basic_json_view/end.md) and
[`items()`](../api/basic_json_view/items.md) visit *every* occurrence, in document order, and
[`size()`](../api/basic_json_view/size.md) counts all of them, so `#!cpp view.size()` can be larger than
`#!cpp view.materialize().size()`.
**Why the first?** A lookup can stop as soon as it finds a match. Returning the last member would force every lookup
to scan all members of the object, even when the key is found at the very first one: this made lookups in small
objects 1.6 to 3.4 times slower. The hash index of the keys of objects with 128 members or more leads to the first member of a
key as well. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also return the first
member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a receiver that sees
duplicates is unpredictable, so neither choice is wrong.
The same holds for [`get<T>()`](../api/basic_json_view/get.md) of a type whose `from_json()` reads the view, such as
the types of the `NLOHMANN_DEFINE_TYPE_*` macros: their members are looked up, so `#!cpp get<point>()` of
`#!cpp {"x":1,"x":2}` reads `#!cpp x == 1`, while `#!cpp materialize().get<point>()` reads `#!cpp 2`.
**What stays the same as `parse()`?** [`materialize()`](../api/basic_json_view/materialize.md) and
[`get<std::map<...>>()`](../api/basic_json_view/get.md) replay every member in order, so they keep the *last* value
exactly like `#!cpp basic_json::parse()` (at the position of the first occurrence of the key, for an
[`ordered_json`](../api/ordered_json.md)).
**How do I get the value `parse()` would give?** Either call `#!cpp view.materialize()` and look the key up in the
result, or iterate the members and keep the last match:
```cpp
// the last member with the key "a", as parse() would keep it
json_view last;
for (const auto item : view.items())
{
if (item.key() == "a")
{
last = item.value();
}
}
```
If you cannot trust the source text to have unique keys, check [`size()`](../api/basic_json_view/size.md) against the
number of distinct keys, or reject duplicates when iterating.
## Getting values out without copying
[`get<T>()`](../api/basic_json_view/get.md) converts most `T` directly from the flat index, without ever building a
`basic_json` value for the conversion: `#!cpp bool`, arithmetic types, `#!cpp std::nullptr_t`,
`#!cpp std::string`/other `#!cpp std::basic_string`s (copied once), `basic_json_view` itself, the standard containers
and other types the library converts (`#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::list`, `#!cpp std::array`,
`#!cpp std::pair`, `#!cpp std::tuple`, `#!cpp std::optional`, ...), and the types of the `NLOHMANN_DEFINE_TYPE_*` and
`NLOHMANN_JSON_SERIALIZE_ENUM` macros. Types whose `from_json()` is written for `basic_json` only go through
[`materialize()`](../api/basic_json_view/materialize.md)`.get<T>()` instead: the subtree is built into a real
`basic_json` value first, exactly as [`parse()`](../api/basic_json/parse.md) would, and converted from there. See
[Writing generic code](#writing-generic-code) for how to write a `from_json()` that reads views.
Two conversions never copy at all:
- [`get_string()`](../api/basic_json_view/get_string.md) (equivalently, `#!cpp get<string_view_t>()`) returns a
string as a `string_view_t` pointing into the document's [`source()`](../api/basic_json_document/source.md) text —
or, for a string that contains escape sequences, into the document's own buffer of decoded strings — instead of
allocating a new `#!cpp std::string`.
- [`number_token()`](../api/basic_json_view/number_token.md) returns a number exactly as it was written in the
source, e.g. `#!cpp "1.50"`, `#!cpp "1E2"`, or an integer with more digits than any number type holds, instead of
rounding it into a `#!cpp double`/`#!cpp int64_t` the way `#!cpp get<T>()` (and
[`basic_json::parse()`](../api/basic_json/parse.md)) would.
Both results are only valid as long as the view — and, for a string with no escapes, the borrowed source text — is.
### `string_view` members
A struct whose member is a `#!cpp std::string_view` (C++17) can be decoded from a view with the
`NLOHMANN_DEFINE_TYPE_*` macros, and the member is not copied either: for a string without escape sequences it points
into the document's source text, and for a string with escape sequences into the document's buffer of decoded strings.
The member is valid as long as the document is alive and, for a document that
[borrows its source text](#ownership-and-lifetime) (as [`parse()`](../api/basic_json_document/parse.md) does for
an lvalue `#!cpp std::string`, a `#!cpp std::string_view`, or a C string), as long as that text is.
```cpp
struct record
{
std::string_view name;
int count;
NLOHMANN_DEFINE_TYPE_INTRUSIVE(record, name, count)
};
const std::string text = R"({"name": "widget", "count": 3})";
const json_document doc = json_document::parse(text); // borrows text
const record r = doc.root().get<record>(); // r.name points into text, valid while text and doc are
```
!!! warning "Decoding into a `string_view` from a `json` value dangles at once"
The same struct also compiles with `#!cpp json::parse(text).get<record>()`, but there `name` points into the
temporary `json` value, which is destroyed at the end of the statement. Use a named
[`json_document`](../api/json_document.md), as above, to decode such a struct.
## Writing generic code
A view's type names its `BasicJsonType` and its [`view_mode`](../api/view_mode.md):
[`json_view`](../api/json_view.md) is `#!cpp basic_json_view<json, view_mode::read_only>`, and
[`json_editable_view`](../api/json_editable_view.md) is `#!cpp basic_json_view<json, view_mode::editable>`. These are
different types. The only conversion between them turns a view of a read-only document into a view of an editable
document of the same `BasicJsonType`, so a function written for one shortcut accepts few other views:
```cpp
void print_names(json_view v); // accepts neither an ordered_json_view nor a json_editable_view
void print_names(json_editable_view v); // also accepts a json_view, but no ordered_json_view
```
This is the same pitfall as a `to_json`/`from_json` written for `json` instead of any `BasicJsonType`, which then does
not work for `ordered_json`. The shortcuts are for declaring documents and views; a function that takes a view names
the views it accepts with a template:
```cpp
// views of editable documents only
template<typename BasicJsonType>
void f(basic_json_view<BasicJsonType, view_mode::editable> v);
// views of read-only documents only
template<typename BasicJsonType>
void f(basic_json_view<BasicJsonType, view_mode::read_only> v);
// views of either document type
template<typename BasicJsonType, view_mode Mode>
void f(basic_json_view<BasicJsonType, Mode> v);
```
Views are passed by value: they are two pointers, cheap to copy, like a `#!cpp std::string_view`. A `const&` works as
well, but a non-`const` reference does not bind to the views that [`root()`](../api/basic_json_document/root.md),
[`operator[]`](../api/basic_json_view/operator%5B%5D.md), and [`at()`](../api/basic_json_view/at.md) return, so
`#!cpp f(doc.root())` would not compile.
There is no conversion from an editable view to a read-only one, by design: the mode decides how a view walks the
index. A read-only view reads a container in place, while a view of an editable document follows the links that edits
leave behind, so a read-only view of an edited document would read stale values. The other direction is safe: a
read-only document has no edits, so a view of it [converts](../api/basic_json_view/basic_json_view.md) to an
editable view, which reads it like an editable document before its first edit. A template deduces the mode without
conversions, so the template forms above still accept only the modes they name. Views of both modes compare with
[`operator==`](../api/basic_json_view/operator_eq.md).
[`is_json_view<T>`](../api/is_json_view.md) tells views from other types, for constraints that cannot be written as a
`basic_json_view<...>` parameter.
### A `from_json()` that reads views
[`get<T>()`](../api/basic_json_view/get.md) on a view calls `T`'s `from_json()` with the view itself if that
`from_json()` accepts views, and with [`materialize()`](../api/basic_json_view/materialize.md) otherwise. The
`from_json()` functions of the [`NLOHMANN_DEFINE_TYPE_*`](../api/macros/nlohmann_define_type_intrusive.md),
[`NLOHMANN_DEFINE_DERIVED_TYPE_*`](../api/macros/nlohmann_define_derived_type.md), and
[`NLOHMANN_JSON_SERIALIZE_ENUM`](../api/macros/nlohmann_json_serialize_enum.md) macros, and those of the library
for the standard types, accept views, so these types need nothing else. For decoding the statuses of
twitter.json into structs, reading the view is about 5 times faster than `#!cpp materialize().get<T>()`.
A hand-written `from_json()` reads views if it is constrained to `json` values and views:
```cpp
template < typename J, typename std::enable_if <
nlohmann::is_json_view<J>::value || std::is_same<J, nlohmann::json>::value, int >::type = 0 >
void from_json(const J& j, person& p)
{
p.name = j.at("name").template get<std::string>();
p.age = j.at("age").template get<int>();
}
```
Its body may only use what views and `basic_json` have in common: the type checks, `at()`, `operator[]`, `contains()`,
`value()`, `size()`, iteration and `items()`, and `get<T>()`/`get_to()`. An unconstrained
`#!cpp template<typename BasicJsonType> void from_json(const BasicJsonType& j, person& p)`, as in
[arbitrary type conversions](arbitrary_types.md), is never called with a view, as it might use `basic_json` functions
a view does not have; such a type is converted through `materialize()`. A `from_json()` for `json` only is
converted through `materialize()` as well.
One thing differs when a `from_json()` reads the view: members looked up by name (`at()`, `value()`, ... and thus
the macros) find the **first** member with a repeated key, see [Duplicate keys](#duplicate-keys);
`#!cpp materialize().get<T>()` would see the last one. Containers built from an object, such as `#!cpp std::map`,
keep the **last** value either way, as [`parse()`](../api/basic_json/parse.md) does.
`to_json()` is unaffected: it writes `basic_json` values, never views.
## Writing a view back
[`dump()`](../api/basic_json_view/dump.md) serializes a view directly from the flat index, without ever building a
`basic_json` value. An object's members are written in document order, not sorted by key, and *every* occurrence of a
repeated key is written, not only the last one — the same two ways [iteration](#what-is-different) already differs
from a [`materialize()`](../api/basic_json_view/materialize.md)d value, see [Duplicate keys](#duplicate-keys).
`#!cpp materialize().dump()` gives a different result in both respects for a `json_view`.
By default, numbers are written the way [`basic_json::dump()`](../api/basic_json/dump.md) would.
[`number_format::source`](../api/basic_json_view/number_format.md) instead copies every number exactly as it was
written in the source text — a price like `#!cpp 19.90`, a long order or account ID with more digits than any number
type holds, or a high-precision coordinate — something `basic_json` cannot do at all, since parsing already reduces
a number to its parsed `#!cpp double`/`#!cpp int64_t` value.
[`operator<<`](../api/basic_json_view/operator_ltlt.md) writes a view to a stream the way `basic_json`'s does, using
the stream's `width`/`fill` for indentation.
## Editing a document
Everything above is read-only: a `json_document`/`json_view` lets you look at a parsed text without copying it, but
not change it. [`basic_json_document<BasicJsonType, view_mode::editable>`](../api/basic_json_document/index.md) — more conveniently
spelled [`json_editable_document`](../api/json_editable_document.md) or
[`ordered_json_editable_document`](../api/ordered_json_editable_document.md) — also lets you
[`set`](../api/basic_json_document/set.md) a value, [`push_back`](../api/basic_json_document/push_back.md) onto or
[`insert`](../api/basic_json_document/insert.md) into an array, and [`erase`](../api/basic_json_document/erase.md)
an object member or an array element, still without ever building a `basic_json` tree for parts you do not touch.
The [mode](../api/view_mode.md) defaults to `#!cpp view_mode::read_only`, so `json_document`/`ordered_json_document` are unaffected — they carry
none of the bookkeeping edits need, and calling `set`/`push_back`/`insert`/`erase` on one is a compile error, not a
runtime one.
### Why: editing without reformatting
The `#!cpp 19.90` price from [above](#writing-a-view-back) is exactly the kind of value that makes editing a `json`
or `ordered_json` value in place lossy. Say you parse a configuration file, patch one field, and write it back:
- **`json`** re-sorts every key on the way in (`object_t` is a `#!cpp std::map`) and rewrites every number to its
shortest round-trip form on the way out — a one-field patch turns into a diff that reorders the whole file and
rewrites `#!cpp 19.90` to `#!cpp 19.9`.
- **`ordered_json`** keeps the key order, but still rewrites every number the same way: parsing has already reduced
it to a `#!cpp double`/`#!cpp int64_t`, and there is no way back to how it was spelled in the source text.
The patch of the example below (set `"host"`, add `"retries"`) written back by each of them:
<div class="nj-jsonview-cmp" role="group" aria-label="The same patch written back by json, ordered_json, and json_editable_document">
<div><p class="nj-jsonview-cmp-h"><code>json</code></p><p class="nj-jsonview-cmp-v">keys sorted, numbers rewritten</p><pre>{<span class="nj-jsonview-bad">"host"</span>:"db2",
"name":"cache",
"port":6379,
"price":<span class="nj-jsonview-bad">19.9</span>,
"retries":3}</pre></div>
<div><p class="nj-jsonview-cmp-h"><code>ordered_json</code></p><p class="nj-jsonview-cmp-v">key order kept, numbers rewritten</p><pre>{"name":"cache",
"host":"db2",
"port":6379,
"price":<span class="nj-jsonview-bad">19.9</span>,
"retries":3}</pre></div>
<div class="nj-jsonview-cmp--best"><p class="nj-jsonview-cmp-h"><code>json_editable_document</code></p><p class="nj-jsonview-cmp-v">order and spelling kept</p><pre>{"name":"cache",
"host":"db2",
"port":6379,
"price":<span class="nj-jsonview-ok">19.90</span>,
"retries":3}</pre></div>
</div>
An editable document keeps both. [`dump()`](../api/basic_json_view/dump.md) of an edited document writes members in
document order — a member [`set`](../api/basic_json_document/set.md) added goes at the end, exactly where it was
inserted, and an [`erase`](../api/basic_json_document/erase.md)d member simply leaves a gap: everything around it
keeps its place — and [`number_format::source`](../api/basic_json_view/number_format.md) keeps the exact spelling
of every number an edit did not itself touch; a number an edit *did* touch is written the way
[`BasicJsonType::dump()`](../api/basic_json/dump.md) would write it, since there is no source spelling for a brand
new value.
??? example "Example: patch a configuration, keeping member order and number spellings"
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
### What stays valid, and what an edit costs
The source text itself is **never written**, and the parsed index never moves — a value keeps the node it was
parsed into for as long as it is not itself replaced. So every [view](../api/basic_json_view/index.md) taken before
an edit, including a previously obtained [`root()`](../api/basic_json_document/root.md), stays valid and, if it
still refers to the edited value, sees the edit; a view of a value a later edit drops or replaces just keeps showing
what it last held. New values go to storage the document allocates and owns on demand. The one thing an edit does
invalidate is the **iterators** taken over an edited array or object: the first time an element is added to it,
inserted into it, erased from it, or replaced by a value that spans a different number of nodes, its elements move from the parsed, fixed layout to a growable block of links
so that [`push_back`](../api/basic_json_document/push_back.md) can later grow it in amortized constant time —
existing elements are not touched, but an iterator that was walking the old layout no longer matches. A string
obtained with
[`get_string()`](../api/basic_json_view/get_string.md) is unaffected either way and stays valid across further
edits. See [`basic_json_document`'s Edits](../api/basic_json_document/index.md#edits) for the details, and
[`set`'s Exception safety](../api/basic_json_document/set.md#exception-safety) for what an edit guarantees if it
throws (the *basic* guarantee, not the strong one `dump()` and the read-only functions provide). How edits are kept in
the index is described in the [architecture overview](../home/architecture.md#node-index-of-json-views).
Step through the edits of a small document (the node fields follow the [architecture overview](../home/architecture.md#node-index-of-json-views)):
<div class="nj-jsonview-steps" data-nj-jsonview-steps>
<figure class="nj-jsonview-step" data-label="After parsing">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_0.svg"
</div>
<figcaption>Step 1 of 5. After parsing: nine nodes in the parsed index, no owned storage yet.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="set an existing member">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_1.svg"
</div>
<figcaption>Step 2 of 5. <code>doc.set(doc.root(), "port", 6380)</code> overwrites the node of the integer in place; no member is added or removed, so the object keeps its parsed layout.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="erase an array element">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_2.svg"
</div>
<figcaption>Step 3 of 5. <code>doc.erase(doc.root()["tags"], 0)</code>: the array is edited for the first time, so its elements move to a block of links in owned storage. The node of <code>"eu"</code> stays in the index, but nothing refers to it any more; iterators over the array taken before are invalid.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="push_back onto the array">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_3.svg"
</div>
<figcaption>Step 4 of 5. <code>doc.push_back(doc.root()["tags"], "new")</code>: the new string gets its own node, its text goes to the buffer of edited text, and a link to it is appended to the block of the array.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="set a new member">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_4.svg"
</div>
<figcaption>Step 5 of 5. <code>doc.set(doc.root(), "retries", 3)</code>: a new member, so the object moves to a block too: copies of its keys, links to the values it already had, then the new key and a link to the new integer.</figcaption>
</figure>
</div>
<p class="nj-caption" markdown>The node index and the owned storage of the document after each of five edits. Use *Back*, *Next*, or the numbered buttons to switch between the steps; without JavaScript all steps are shown one below the other.</p>
## Images
[`save()`](../api/basic_json_document/save.md) writes a document as an *image*: a byte buffer that
[`load()`](../api/basic_json_document/load.md) reads back into a document without parsing — no lexing, no building
the node index, nothing but copying the nodes and pointing the text and the decoded strings at the image. Where
[`parse_copy()`](../api/basic_json_document/parse_copy.md) still has to scan the whole input,
[`load()`](../api/basic_json_document/load.md) turns that scan into a copy of the node index alone.
**Why.** A document that is parsed once and then read many times — a configuration loaded at startup, a template
rendered on every request, a large reference dataset a worker process needs in memory — pays for parsing once but
can amortize [`save()`](../api/basic_json_document/save.md)'s cost across every later load. That makes images useful
for a cache: save a document the first time it is parsed (to a file, a shared-memory segment, an in-process cache),
and [`load()`](../api/basic_json_document/load.md) it on every later use instead of parsing the source text again.
They are just as useful for handing a parsed document to another process (or a forked worker) running the same build
of the library, since [`load()`](../api/basic_json_document/load.md) turns the transfer into a copy of the node index
plus pointers into the received bytes, not a re-parse.
**Choosing a check.** [`load()`](../api/basic_json_document/load.md) takes an
[`image_check`](../api/basic_json_document/load.md#image_check) that trades validation against speed:
`image_check::full` (the default) checks everything the parser itself guarantees, so a checked image is exactly as
safe to read and serialize as a freshly parsed document — the right choice whenever the image did not come straight
from this process's own [`save()`](../api/basic_json_document/save.md), such as a file or a network peer.
`image_check::bounds` only checks structure and bounds — cheaper, since it skips scanning the text and the decoded
strings — and fits a cache the process trusts, one it wrote and reads back itself. `image_check::none` skips
validation entirely, for an image trusted as much as the process's own memory. See
[`load()`'s Notes](../api/basic_json_document/load.md#notes) for exactly what each level does and does not guarantee.
??? example "Example: cache a parsed configuration as an image"
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable. It is little-endian only (on big-endian targets, `save()` and `load()` do not exist), and tied to the
library build that wrote it — use it to cache a document or to hand one to another process running the *same*
build, not as a long-term storage format; keep the original JSON text if a saved document needs to be readable by
a future library version.
The idea of a document you can read without parsing comes from zero-copy formats such as
[FlatBuffers](https://github.com/google/flatbuffers) and [YaFF](https://github.com/yandex/yaff); the check
[`load()`](../api/basic_json_document/load.md) runs follows the idea of FlatBuffers' Verifier. No code is taken from
either.
## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) | [`json_editable_document`](../api/json_editable_document.md) / [`json_editable_view`](../api/json_editable_view.md) |
|---|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document | same as `json_document`; edits go to storage the document owns |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only | [`set`](../api/basic_json_document/set.md)/[`push_back`](../api/basic_json_document/push_back.md)/[`insert`](../api/basic_json_document/insert.md)/[`erase`](../api/basic_json_document/erase.md) edit in place; the source text is never rewritten |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use | the same, plus [`dump()`](../api/basic_json_view/dump.md) of an edited document that keeps the member order and, with [`number_format::source`](../api/basic_json_view/number_format.md), the spelling of every untouched number |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives | a document you read, patch a few fields of, and write back — a configuration file, for instance — where the rest of it should come back exactly as it was |
| **Caching/reload** | not applicable — re-parse, or roll your own serialization | not applicable | [`save()`](../api/basic_json_document/save.md)/[`load()`](../api/basic_json_document/load.md): cache the parsed index as an image and reload it without parsing | same, saving the document's current — possibly edited — state |
<p class="nj-table-caption" markdown>Each column is one way to work with JSON in this library, and each row compares one property of them.</p>
## See also
- [`basic_json_document`](../api/basic_json_document/index.md) - the document that owns the parsed index (parse, access, images, edits)
- [`basic_json_view`](../api/basic_json_view/index.md) - the non-owning handle to a value in a document
- [`json_document`](../api/json_document.md) - and [`ordered_json_document`](../api/ordered_json_document.md) - the read-only documents
- [`json_editable_document`](../api/json_editable_document.md) - and [`ordered_json_editable_document`](../api/ordered_json_editable_document.md) - the documents that can be edited in place
- [Architecture](../home/architecture.md#node-index-of-json-views) - the layout of the node index
- [SAX interface](parsing/sax_interface.md) - event-based parsing without building a tree
- [Performance](performance.md) - benchmarks and tuning of parsing and serialization
- [Large, Streamed, and Partial Input](parsing/large_and_partial_input.md) - reading only the part of a text you need
## Version history
- Added in version 3.13.0.
@@ -1,219 +0,0 @@
# Differences from `basic_json`
A [`basic_json_view`](../../api/basic_json_view/index.md) offers the read-only part of the
[`basic_json`](../../api/basic_json/index.md) interface, and in most respects it behaves exactly like the value
[`basic_json::parse()`](../../api/basic_json/parse.md) would have built from the same text. This page lists where it
does not: what a document parses differently, what a view's functions do differently, and which parts of the
`basic_json` interface a view does not have.
## Summary
| | `basic_json` value from `parse()` | view of a `json_document` | to get the `basic_json` behavior |
|---|---|---|---|
| [Integer types](#parsing) | any `number_integer_t`/`number_unsigned_t` | 64-bit only (compile-time check) | — |
| [Input size](#parsing) | unlimited | less than 4 GiB | — |
| [Parser callbacks](#parsing) | `parser_callback_t` | none | filter while iterating |
| [`operator[]` on a missing key or index](#element-access) | inserts (non-`const`) or undefined behavior (`const`) | returns a discarded view | — |
| [Object member order](#object-member-order) | sorted by key (`json`), insertion order (`ordered_json`) | document order | `materialize()` |
| [Repeated keys](#duplicate-keys) | the last value is kept | lookups find the first member; iteration, `size()`, and `dump()` see all of them | `materialize()` |
| [Conversions](#conversions) | `get<T>()`, implicit conversions, `get_ptr()`, `get_ref()` | `get<T>()`, `get_string()`, `number_token()` | `materialize()` |
| [Number spelling](#serialization) | lost when parsing | kept: `number_token()`, `number_format::source` | — |
| [Comparison](#comparison) | `==`, `!=`, `<`, `<=>`, ... | `==` and `!=` only | `materialize()` |
| [Modification](#what-a-view-does-not-have) | member functions of the value | only through an [editable document](editing.md) | `materialize()` |
| [Iterators](#what-a-view-does-not-have) | bidirectional, reverse iterators | forward only | `materialize()` |
<p class="nj-table-caption" markdown>Each row is one difference; the last column names the way to get `basic_json`'s behavior from a view, where there is one.</p>
## Parsing
### What is the same
- **Accept/reject.** [`accept()`](../../api/basic_json_document/accept.md) and
[`parse()`](../../api/basic_json_document/parse.md) accept and reject exactly the same inputs as
[`basic_json::accept()`](../../api/basic_json/accept.md)/[`basic_json::parse()`](../../api/basic_json/parse.md),
with the same `ignore_comments` and `ignore_trailing_commas` options.
- **Errors.** A failing parse throws the same exception — the same id, message, and position — because on failure
the library's own parser is run on the same bytes to produce the diagnostic. Valid input never pays for this, and
the replay builds no values: it costs about a lexing pass up to the error.
`#!cpp allow_exceptions == false` skips the replay and gives a
[discarded](../../api/basic_json_document/is_discarded.md) document instead of throwing, just as it gives a
discarded value for `#!cpp basic_json::parse()`.
- **Number classification.** An integer literal that does not fit into the 64-bit integer type becomes a
floating-point value, exactly as it does for `#!cpp basic_json::parse()`.
- **Macros.** [`JSON_STRICT_NUL_HANDLING`](../../api/macros/json_strict_nul_handling.md) and
[`JSON_NOEXCEPTION`](../../api/macros/json_noexception.md)/[`JSON_THROW_USER`](../../api/macros/json_throw_user.md)
behave the same way they do for `<nlohmann/json.hpp>`. These are the macros that change what the parser accepts or
how it reports errors. The other macros only affect the values `materialize()` produces, through `BasicJsonType`.
Two of them also add context to the exceptions of views, see [Exception context](#exception-context).
### What is different
- **Only 64-bit integers.** `basic_json_document<BasicJsonType>` requires `BasicJsonType::number_integer_t` and
`number_unsigned_t` to both be 64 bits wide; this is a compile-time `#!cpp static_assert`.
- **A 4 GiB input limit.** An input of 4294967280 bytes (4 GiB minus 16 bytes) or more throws
[`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416), a limit
`#!cpp basic_json::parse()` does not have. It keeps an index entry at 16 bytes: offsets and counts are 32 bits
wide. The limit is 16 bytes below 2^32, because the parser reads in 16-byte steps, and a position one step past the
end must still fit.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
is read to its end into a buffer the document owns, and parsing starts after that.
- **No parser callbacks.** [`parse()`](../../api/basic_json_document/parse.md) takes no
[`parser_callback_t`](../../api/basic_json/parser_callback_t.md): a document always indexes the whole text. Skip
the values you do not need when reading the view instead.
- **JSON text only.** A document is parsed from JSON text; there is no counterpart of
[`from_cbor()`](../../api/basic_json/from_cbor.md) and the other binary formats, and
[`is_binary()`](../../api/basic_json_view/is_binary.md) is always `#!cpp false`.
## Element access
**Chained access is safe.** [`operator[]`](../../api/basic_json_view/operator%5B%5D.md) with a missing key, an index
out of range, or an unresolvable JSON pointer returns a [discarded](../../api/basic_json_view/is_discarded.md) view,
and `operator[]` on a discarded view returns a discarded view without throwing: `#!cpp v["a"]["b"][0]` can be tested
once at the end. Type errors on values that exist (a key on an array, an index on an object) still throw, and
[`at`](../../api/basic_json_view/at.md) throws for every missing value.
`basic_json`'s non-`const` `operator[]` instead inserts a missing key, and its `const` `operator[]` has undefined
behavior for one.
## Objects
### Object member order
[`begin()`](../../api/basic_json_view/begin.md)/[`end()`](../../api/basic_json_view/end.md) and
[`items()`](../../api/basic_json_view/items.md) visit an object's members in the order they appear in the source text.
`basic_json`'s default `object_t` is a `std::map`, which sorts by key, so iterating a
[`materialize()`](../../api/basic_json_view/materialize.md)d value can print members in a different order than
iterating the view they came from. [`dump()`](../../api/basic_json_view/dump.md) of a view writes members in document
order as well.
### Duplicate keys
!!! warning "A lookup in a view and in the parsed value can give different answers"
If an object in the source text repeats a key, [`operator[]`](../../api/basic_json_view/operator%5B%5D.md),
[`at`](../../api/basic_json_view/at.md), [`find`](../../api/basic_json_view/find.md),
[`value`](../../api/basic_json_view/value.md), [`contains`](../../api/basic_json_view/contains.md), and JSON
pointer resolution all return the **first** member with that key. `basic_json::parse()` instead keeps the **last**
value of a repeated key, so for `#!cpp {"a":1,"a":2}`, `#!cpp view["a"]` is `#!cpp 1` while
`#!cpp parse(text)["a"]` is `#!cpp 2`. [`count`](../../api/basic_json_view/count.md) counts every member with the
key, so it is `#!cpp 2` here.
[`begin()`](../../api/basic_json_view/begin.md)/[`end()`](../../api/basic_json_view/end.md) and
[`items()`](../../api/basic_json_view/items.md) visit *every* occurrence, in document order, and
[`size()`](../../api/basic_json_view/size.md) counts all of them, so `#!cpp view.size()` can be larger than
`#!cpp view.materialize().size()`. [`dump()`](../../api/basic_json_view/dump.md) writes every occurrence, too, so
`#!cpp materialize().dump()` gives a different result for such an object.
**Why the first?** A lookup can stop as soon as it finds a match. Returning the last member would force every lookup
to scan all members of the object, even when the key is found at the very first one: this made lookups in small
objects 1.6 to 3.4 times slower. The hash index of the keys of objects with 128 members or more leads to the first
member of a key as well. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also return
the first member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a receiver
that sees duplicates is unpredictable, so neither choice is wrong.
The same holds for [`get<T>()`](../../api/basic_json_view/get.md) of a type whose `from_json()` reads the view, such
as the types of the `NLOHMANN_DEFINE_TYPE_*` macros: their members are looked up, so `#!cpp get<point>()` of
`#!cpp {"x":1,"x":2}` reads `#!cpp x == 1`, while `#!cpp materialize().get<point>()` reads `#!cpp 2`.
**What stays the same as `parse()`?** [`materialize()`](../../api/basic_json_view/materialize.md) and
[`get<std::map<...>>()`](../../api/basic_json_view/get.md) (and the other containers built from an object) replay
every member in order, so they keep the *last* value exactly like `#!cpp basic_json::parse()` (at the position of the
first occurrence of the key, for an [`ordered_json`](../../api/ordered_json.md)).
**How do I get the value `parse()` would give?** Either call `#!cpp view.materialize()` and look the key up in the
result, or iterate the members and keep the last match:
```cpp
// the last member with the key "a", as parse() would keep it
json_view last;
for (const auto item : view.items())
{
if (item.key() == "a")
{
last = item.value();
}
}
```
If you cannot trust the source text to have unique keys, check [`size()`](../../api/basic_json_view/size.md) against
the number of distinct keys, or reject duplicates when iterating.
## Conversions
- **No implicit conversions.** A view has no `#!cpp operator ValueType()`, no `#!cpp get_ptr()`, and no
`#!cpp get_ref()`; use [`get<T>()`](../../api/basic_json_view/get.md), or
[`get_string()`](../../api/basic_json_view/get_string.md) for a string without a copy.
- **`from_json()` overloads.** [`get<T>()`](../../api/basic_json_view/get.md) calls `T`'s `from_json()` with the view
only if that `from_json()` accepts views, and with [`materialize()`](../../api/basic_json_view/materialize.md)
otherwise; see [A `from_json()` that reads views](generic_code.md#a-from_json-that-reads-views). Both give the same
result except for [repeated keys](#duplicate-keys).
- **`string_view` members.** A struct with a `#!cpp std::string_view` member decoded from a view points into the
document, see [`string_view` members](reading.md#string_view-members). Decoding the same struct from a `json` value
compiles, too, but dangles at once if the value is a temporary.
## Serialization
[`dump()`](../../api/basic_json_view/dump.md) writes numbers the way
[`basic_json::dump()`](../../api/basic_json/dump.md) would by default, but writes object members in
[document order](#object-member-order) and writes [every occurrence of a repeated key](#duplicate-keys).
A view can also do what a `basic_json` value cannot: [`number_token()`](../../api/basic_json_view/number_token.md)
and [`number_format::source`](../../api/basic_json_view/number_format.md) give every number exactly as it was spelled
in the source text, while parsing into `basic_json` already reduces a number to its `#!cpp double`/`#!cpp int64_t`
value. See [Writing a view back](reading.md#writing-a-view-back).
## Comparison
**Ordering comparisons are not provided** by `basic_json_view` — there is no `#!cpp operator<`.
[`operator==`](../../api/basic_json_view/operator_eq.md) and [`operator!=`](../../api/basic_json_view/operator_ne.md)
are provided, though: two views, or a view and a `BasicJsonType` value, compare equal exactly when
[`materialize()`](../../api/basic_json_view/materialize.md) or [`parse()`](../../api/basic_json/parse.md) would produce
equal values for them, without ever building a tree to do it. For ordering,
[`materialize()`](../../api/basic_json_view/materialize.md) is the way to get a value you can compare.
## Exception context
With [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md), an exception of a view names the JSON Pointer path of
the value within the document, and with
[`JSON_DIAGNOSTIC_POSITIONS`](../../api/macros/json_diagnostic_positions.md), it names the bytes of the value in the
source text, as they do for the exceptions of `#!cpp basic_json`.
A view has neither parent pointers nor stored positions, so both are found from the document's index when the
exception is thrown. This takes time linear in the size of the document. One exception has neither: `key()` of an
iterator over an array or a primitive value
([`invalid_iterator.207`](../../home/exceptions.md#jsonexceptioninvalid_iterator207)), because a view's iterator does
not know the value it iterates over.
**Fewer source positions after edits.** With
[`JSON_DIAGNOSTIC_POSITIONS`](../../api/macros/json_diagnostic_positions.md) enabled,
[`materialize()`](../../api/basic_json_view/materialize.md) sets the same positions as `#!cpp basic_json::parse()`. In
an [editable document](editing.md), a value written by an edit has none, and neither has an array or object whose
elements changed or whose last element has none; a `basic_json` value keeps its positions when it is changed. See
[`materialize()`](../../api/basic_json_view/materialize.md#notes).
## What a view does not have
A view is a read-only handle. These parts of the `basic_json` interface have no counterpart on a view;
[`materialize()`](../../api/basic_json_view/materialize.md) the subtree to use them:
- **Modifiers** such as `push_back`, `insert`, `erase`, `update`, or `operator=`. A view never changes a value
itself; an [editable document](editing.md) changes the values its views refer to through
[`set`](../../api/basic_json_document/set.md), [`push_back`](../../api/basic_json_document/push_back.md),
[`insert`](../../api/basic_json_document/insert.md), and [`erase`](../../api/basic_json_document/erase.md).
- **Reverse iteration.** A view's iterator is a forward iterator; there is no `rbegin()`/`rend()`.
- **Ordering comparisons**, see [Comparison](#comparison).
- **JSON Patch and friends**: [`patch()`](../../api/basic_json/patch.md),
[`merge_patch()`](../../api/basic_json/merge_patch.md), [`diff()`](../../api/basic_json/diff.md),
[`flatten()`](../../api/basic_json/flatten.md).
- **Binary formats**: [`to_cbor()`](../../api/basic_json/to_cbor.md) and the other writers, and binary values
([`get_binary()`](../../api/basic_json/get_binary.md)).
## See also
- [Zero-copy JSON views](index.md) - overview
- [`basic_json_view`](../../api/basic_json_view/index.md) - the full interface of a view
- [`materialize()`](../../api/basic_json_view/materialize.md) - build the `basic_json` value of a view
## Version history
- Added in version 3.13.0.
@@ -1,133 +0,0 @@
# Editing a document
A [`json_document`](../../api/json_document.md)/[`json_view`](../../api/json_view.md) lets you look at a parsed text
without copying it, but not change it.
[`basic_json_document<BasicJsonType, view_mode::editable>`](../../api/basic_json_document/index.md) — more
conveniently spelled [`json_editable_document`](../../api/json_editable_document.md) or
[`ordered_json_editable_document`](../../api/ordered_json_editable_document.md) — also lets you
[`set`](../../api/basic_json_document/set.md) a value, [`push_back`](../../api/basic_json_document/push_back.md) onto
or [`insert`](../../api/basic_json_document/insert.md) into an array, and
[`erase`](../../api/basic_json_document/erase.md) an object member or an array element, still without ever building a
`basic_json` tree for parts you do not touch.
The [mode](../../api/view_mode.md) defaults to `#!cpp view_mode::read_only`, so
`json_document`/`ordered_json_document` are unaffected — they carry none of the bookkeeping edits need, and calling
`set`/`push_back`/`insert`/`erase` on one is a compile error, not a runtime one.
## Why: editing without reformatting
A price like `#!cpp 19.90` is exactly the kind of value that makes editing a `json` or `ordered_json` value in place
lossy. Say you parse a configuration file, patch one field, and write it back:
- **`json`** re-sorts every key on the way in (`object_t` is a `#!cpp std::map`) and rewrites every number to its
shortest round-trip form on the way out — a one-field patch turns into a diff that reorders the whole file and
rewrites `#!cpp 19.90` to `#!cpp 19.9`.
- **`ordered_json`** keeps the key order, but still rewrites every number the same way: parsing has already reduced
it to a `#!cpp double`/`#!cpp int64_t`, and there is no way back to how it was spelled in the source text.
The patch of the example below (set `"host"`, add `"retries"`) written back by each of them:
<div class="nj-jsonview-cmp" role="group" aria-label="The same patch written back by json, ordered_json, and json_editable_document">
<div><p class="nj-jsonview-cmp-h"><code>json</code></p><p class="nj-jsonview-cmp-v">keys sorted, numbers rewritten</p><pre>{<span class="nj-jsonview-bad">"host"</span>:"db2",
"name":"cache",
"port":6379,
"price":<span class="nj-jsonview-bad">19.9</span>,
"retries":3}</pre></div>
<div><p class="nj-jsonview-cmp-h"><code>ordered_json</code></p><p class="nj-jsonview-cmp-v">key order kept, numbers rewritten</p><pre>{"name":"cache",
"host":"db2",
"port":6379,
"price":<span class="nj-jsonview-bad">19.9</span>,
"retries":3}</pre></div>
<div class="nj-jsonview-cmp--best"><p class="nj-jsonview-cmp-h"><code>json_editable_document</code></p><p class="nj-jsonview-cmp-v">order and spelling kept</p><pre>{"name":"cache",
"host":"db2",
"port":6379,
"price":<span class="nj-jsonview-ok">19.90</span>,
"retries":3}</pre></div>
</div>
An editable document keeps both. [`dump()`](../../api/basic_json_view/dump.md) of an edited document writes members in
document order — a member [`set`](../../api/basic_json_document/set.md) added goes at the end, exactly where it was
inserted, and an [`erase`](../../api/basic_json_document/erase.md)d member simply leaves a gap: everything around it
keeps its place — and [`number_format::source`](../../api/basic_json_view/number_format.md) keeps the exact spelling
of every number an edit did not itself touch; a number an edit *did* touch is written the way
[`BasicJsonType::dump()`](../../api/basic_json/dump.md) would write it, since there is no source spelling for a brand
new value. See [Writing a view back](reading.md#writing-a-view-back).
??? example "Example: patch a configuration, keeping member order and number spellings"
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## What stays valid, and what an edit costs
The source text itself is **never written**, and the parsed index never moves — a value keeps the node it was
parsed into for as long as it is not itself replaced. So every [view](../../api/basic_json_view/index.md) taken before
an edit, including a previously obtained [`root()`](../../api/basic_json_document/root.md), stays valid and, if it
still refers to the edited value, sees the edit; a view of a value a later edit drops or replaces just keeps showing
what it last held. New values go to storage the document allocates and owns on demand. The one thing an edit does
invalidate is the **iterators** taken over an edited array or object: the first time an element is added to it,
inserted into it, erased from it, or replaced by a value that spans a different number of nodes, its elements move
from the parsed, fixed layout to a growable block of links so that
[`push_back`](../../api/basic_json_document/push_back.md) can later grow it in amortized constant time — existing
elements are not touched, but an iterator that was walking the old layout no longer matches. A string obtained with
[`get_string()`](../../api/basic_json_view/get_string.md) is unaffected either way and stays valid across further
edits. See [`basic_json_document`'s Edits](../../api/basic_json_document/index.md#edits) for the details, and
[`set`'s Exception safety](../../api/basic_json_document/set.md#exception-safety) for what an edit guarantees if it
throws (the *basic* guarantee, not the strong one `dump()` and the read-only functions provide). How edits are kept in
the index is described in the [architecture overview](../../home/architecture.md#node-index-of-json-views).
Step through the edits of a small document (the node fields follow the
[architecture overview](../../home/architecture.md#node-index-of-json-views)):
<div class="nj-jsonview-steps" data-nj-jsonview-steps>
<figure class="nj-jsonview-step" data-label="After parsing">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_0.svg"
</div>
<figcaption>Step 1 of 5. After parsing: nine nodes in the parsed index, no owned storage yet.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="set an existing member">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_1.svg"
</div>
<figcaption>Step 2 of 5. <code>doc.set(doc.root(), "port", 6380)</code> overwrites the node of the integer in place; no member is added or removed, so the object keeps its parsed layout.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="erase an array element">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_2.svg"
</div>
<figcaption>Step 3 of 5. <code>doc.erase(doc.root()["tags"], 0)</code>: the array is edited for the first time, so its elements move to a block of links in owned storage. The node of <code>"eu"</code> stays in the index, but nothing refers to it any more; iterators over the array taken before are invalid.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="push_back onto the array">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_3.svg"
</div>
<figcaption>Step 4 of 5. <code>doc.push_back(doc.root()["tags"], "new")</code>: the new string gets its own node, its text goes to the buffer of edited text, and a link to it is appended to the block of the array.</figcaption>
</figure>
<figure class="nj-jsonview-step" data-label="set a new member">
<div class="nj-jsonview-scroll">
--8<-- "images/json_view_edit_4.svg"
</div>
<figcaption>Step 5 of 5. <code>doc.set(doc.root(), "retries", 3)</code>: a new member, so the object moves to a block too: copies of its keys, links to the values it already had, then the new key and a link to the new integer.</figcaption>
</figure>
</div>
<p class="nj-caption" markdown>The node index and the owned storage of the document after each of five edits. Use *Back*, *Next*, or the numbered buttons to switch between the steps; without JavaScript all steps are shown one below the other.</p>
## See also
- [Zero-copy JSON views](index.md) - overview
- [`json_editable_document`](../../api/json_editable_document.md) - and [`ordered_json_editable_document`](../../api/ordered_json_editable_document.md) - the documents that can be edited in place
- [`basic_json_document`'s Edits](../../api/basic_json_document/index.md#edits) - the rules of edits in detail
- [Differences from `basic_json`](differences.md#exception-context) - source positions after edits
## Version history
- Added in version 3.13.0.
@@ -1,95 +0,0 @@
# Writing generic code
## View types
A view's type names its `BasicJsonType` and its [`view_mode`](../../api/view_mode.md):
[`json_view`](../../api/json_view.md) is `#!cpp basic_json_view<json, view_mode::read_only>`, and
[`json_editable_view`](../../api/json_editable_view.md) is `#!cpp basic_json_view<json, view_mode::editable>`. These
are different types. The only conversion between them turns a view of a read-only document into a view of an
editable document of the same `BasicJsonType`, so a function written for one shortcut accepts few other views:
```cpp
void print_names(json_view v); // accepts neither an ordered_json_view nor a json_editable_view
void print_names(json_editable_view v); // also accepts a json_view, but no ordered_json_view
```
This is the same pitfall as a `to_json`/`from_json` written for `json` instead of any `BasicJsonType`, which then does
not work for `ordered_json`. The shortcuts are for declaring documents and views; a function that takes a view names
the views it accepts with a template:
```cpp
// views of editable documents only
template<typename BasicJsonType>
void f(basic_json_view<BasicJsonType, view_mode::editable> v);
// views of read-only documents only
template<typename BasicJsonType>
void f(basic_json_view<BasicJsonType, view_mode::read_only> v);
// views of either document type
template<typename BasicJsonType, view_mode Mode>
void f(basic_json_view<BasicJsonType, Mode> v);
```
Views are passed by value: they are two pointers, cheap to copy, like a `#!cpp std::string_view`. A `const&` works as
well, but a non-`const` reference does not bind to the views that
[`root()`](../../api/basic_json_document/root.md), [`operator[]`](../../api/basic_json_view/operator%5B%5D.md), and
[`at()`](../../api/basic_json_view/at.md) return, so `#!cpp f(doc.root())` would not compile.
There is no conversion from an editable view to a read-only one, by design: the mode decides how a view walks the
index. A read-only view reads a container in place, while a view of an editable document follows the links that edits
leave behind, so a read-only view of an edited document would read stale values. The other direction is safe: a
read-only document has no edits, so a view of it [converts](../../api/basic_json_view/basic_json_view.md) to an
editable view, which reads it like an editable document before its first edit. A template deduces the mode without
conversions, so the template forms above still accept only the modes they name. Views of both modes compare with
[`operator==`](../../api/basic_json_view/operator_eq.md).
[`is_json_view<T>`](../../api/is_json_view.md) tells views from other types, for constraints that cannot be written as
a `basic_json_view<...>` parameter.
## A `from_json()` that reads views
[`get<T>()`](../../api/basic_json_view/get.md) on a view calls `T`'s `from_json()` with the view itself if that
`from_json()` accepts views, and with [`materialize()`](../../api/basic_json_view/materialize.md) otherwise. The
`from_json()` functions of the [`NLOHMANN_DEFINE_TYPE_*`](../../api/macros/nlohmann_define_type_intrusive.md),
[`NLOHMANN_DEFINE_DERIVED_TYPE_*`](../../api/macros/nlohmann_define_derived_type.md), and
[`NLOHMANN_JSON_SERIALIZE_ENUM`](../../api/macros/nlohmann_json_serialize_enum.md) macros, and those of the library
for the standard types, accept views, so these types need nothing else. For decoding the statuses of
twitter.json into structs, reading the view is about 5 times faster than `#!cpp materialize().get<T>()`.
A hand-written `from_json()` reads views if it is constrained to `json` values and views:
```cpp
template < typename J, typename std::enable_if <
nlohmann::is_json_view<J>::value || std::is_same<J, nlohmann::json>::value, int >::type = 0 >
void from_json(const J& j, person& p)
{
p.name = j.at("name").template get<std::string>();
p.age = j.at("age").template get<int>();
}
```
Its body may only use what views and `basic_json` have in common: the type checks, `at()`, `operator[]`,
`contains()`, `value()`, `size()`, iteration and `items()`, and `get<T>()`/`get_to()`. An unconstrained
`#!cpp template<typename BasicJsonType> void from_json(const BasicJsonType& j, person& p)`, as in
[arbitrary type conversions](../arbitrary_types.md), is never called with a view, as it might use `basic_json`
functions a view does not have; such a type is converted through `materialize()`. A `from_json()` for `json` only is
converted through `materialize()` as well.
One thing differs when a `from_json()` reads the view: members looked up by name (`at()`, `value()`, ... and thus
the macros) find the **first** member with a repeated key, see [Duplicate keys](differences.md#duplicate-keys);
`#!cpp materialize().get<T>()` would see the last one. Containers built from an object, such as `#!cpp std::map`,
keep the **last** value either way, as [`parse()`](../../api/basic_json/parse.md) does.
`to_json()` is unaffected: it writes `basic_json` values, never views.
## See also
- [Zero-copy JSON views](index.md) - overview
- [`is_json_view`](../../api/is_json_view.md) - type trait to detect views
- [`view_mode`](../../api/view_mode.md) - read-only and editable views
- [Reading values](reading.md) - what `get<T>()` converts without materializing
## Version history
- Added in version 3.13.0.
@@ -1,70 +0,0 @@
# Images
[`save()`](../../api/basic_json_document/save.md) writes a document as an *image*: a byte buffer that
[`load()`](../../api/basic_json_document/load.md) reads back into a document without parsing — no lexing, no building
the node index, nothing but copying the nodes and pointing the text and the decoded strings at the image. Where
[`parse_copy()`](../../api/basic_json_document/parse_copy.md) still has to scan the whole input,
[`load()`](../../api/basic_json_document/load.md) turns that scan into a copy of the node index alone.
## Why
A document that is parsed once and then read many times — a configuration loaded at startup, a template rendered on
every request, a large reference dataset a worker process needs in memory — pays for parsing once but can amortize
[`save()`](../../api/basic_json_document/save.md)'s cost across every later load. That makes images useful for a
cache: save a document the first time it is parsed (to a file, a shared-memory segment, an in-process cache), and
[`load()`](../../api/basic_json_document/load.md) it on every later use instead of parsing the source text again.
They are just as useful for handing a parsed document to another process (or a forked worker) running the same build
of the library, since [`load()`](../../api/basic_json_document/load.md) turns the transfer into a copy of the node
index plus pointers into the received bytes, not a re-parse.
## Choosing a check
[`load()`](../../api/basic_json_document/load.md) takes an
[`image_check`](../../api/basic_json_document/load.md#image_check) that trades validation against speed:
- `image_check::full` (the default) checks everything the parser itself guarantees, so a checked image is exactly as
safe to read and serialize as a freshly parsed document — the right choice whenever the image did not come straight
from this process's own [`save()`](../../api/basic_json_document/save.md), such as a file or a network peer.
- `image_check::bounds` only checks structure and bounds — cheaper, since it skips scanning the text and the decoded
strings — and fits a cache the process trusts, one it wrote and reads back itself.
- `image_check::none` skips validation entirely, for an image trusted as much as the process's own memory.
See [`load()`'s Notes](../../api/basic_json_document/load.md#notes) for exactly what each level does and does not
guarantee.
??? example "Example: cache a parsed configuration as an image"
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable. It is little-endian only (on big-endian targets, `save()` and `load()` do not exist), and tied to the
library build that wrote it — use it to cache a document or to hand one to another process running the *same*
build, not as a long-term storage format; keep the original JSON text if a saved document needs to be readable by
a future library version.
## Credits
The idea of a document you can read without parsing comes from zero-copy formats such as
[FlatBuffers](https://github.com/google/flatbuffers) and [YaFF](https://github.com/yandex/yaff); the check
[`load()`](../../api/basic_json_document/load.md) runs follows the idea of FlatBuffers' Verifier. No code is taken from
either.
## See also
- [Zero-copy JSON views](index.md) - overview
- [`save()`](../../api/basic_json_document/save.md) - write a document as an image
- [`load()`](../../api/basic_json_document/load.md) - read an image back into a document
## Version history
- Added in version 3.13.0.
@@ -1,106 +0,0 @@
# Zero-copy JSON views
`#!cpp <nlohmann/json_view.hpp>` adds a read-only, non-owning way to look at a parsed JSON text, as an alternative to
building a [`basic_json`](../../api/basic_json/index.md) tree with [`parse()`](../../api/basic_json/parse.md).
## The problem
[`basic_json::parse()`](../../api/basic_json/parse.md) builds a tree of `basic_json` values: one allocation for every
array and object, and every string copied into its own `std::string`. That is the right trade-off when the program
goes on to read and write the value freely, but it does more work than necessary when only a small part of a large
JSON text is actually needed, or when the same text is parsed over and over (many small messages, for instance) and
most of the resulting tree is thrown away almost immediately.
## The idea
[`basic_json_document::parse()`](../../api/basic_json_document/parse.md) parses the same JSON grammar, with the same
options, but instead of a tree it builds a flat index of the values it found: one
[16-byte entry](../../home/architecture.md#node-index-of-json-views) per value (and one per object key), in document
order. Integers are converted while parsing and stored in their entry. Floating-point numbers and strings are
referenced in place in the source text: a float is converted when it is read, and only strings that contain escape
sequences are decoded, into one shared buffer owned by the document.
[`basic_json_view`](../../api/basic_json_view/index.md) is a small, trivially copyable handle (two pointers) into that
index. It gives you the read-only, type-inspection part of the `basic_json` interface —
[`type()`](../../api/basic_json_view/type.md) and the `is_*()` predicates,
[`size()`](../../api/basic_json_view/size.md)/[`empty()`](../../api/basic_json_view/empty.md) — as well as element
access ([`operator[]`](../../api/basic_json_view/operator%5B%5D.md), [`at`](../../api/basic_json_view/at.md),
[`front`](../../api/basic_json_view/front.md)/[`back`](../../api/basic_json_view/back.md)), lookup
([`find`](../../api/basic_json_view/find.md), [`contains`](../../api/basic_json_view/contains.md),
[`count`](../../api/basic_json_view/count.md)), and iteration
([`begin`](../../api/basic_json_view/begin.md)/[`end`](../../api/basic_json_view/end.md),
[`items`](../../api/basic_json_view/items.md)) — without ever allocating a `basic_json` value. When you do need an
actual `basic_json` value for a subtree, [`materialize()`](../../api/basic_json_view/materialize.md) builds exactly the
one [`parse()`](../../api/basic_json/parse.md) would have produced for it.
The same small text, parsed both ways:
<div class="nj-fig-wrap" markdown>
--8<-- "images/json_view_layout.svg"
</div>
<p class="nj-caption" markdown>Top: the tree of separate heap allocations that `basic_json::parse()` builds. Bottom: `json_document::parse()` keeps the source text and builds one flat index of 16-byte nodes that point back into it, plus a buffer of decoded strings.</p>
## How to use it
Include `<nlohmann/json_view.hpp>` in addition to (or instead of) `<nlohmann/json.hpp>`. Parse into a
[`json_document`](../../api/json_document.md), inspect its [`root()`](../../api/basic_json_document/root.md), and
`materialize()` when you need a real value. `materialize()` works on any view, not just the root, so you can turn just
the subtree you need into a value:
??? example "Example: parse a document, inspect its root, and materialize it"
```cpp
--8<-- "examples/json_document.cpp"
```
Output:
```json
--8<-- "examples/json_document.output"
```
[`ordered_json_document`](../../api/ordered_json_document.md) is the equivalent for
[`ordered_json`](../../api/ordered_json.md), just as [`ordered_json`](../../api/ordered_json.md) is to
[`json`](../../api/json.md).
A view behaves like a `basic_json` value in most respects, but not in all: lookups of a repeated key, the order in
which an object's members are visited, and what a missing key does to chained `operator[]` are the most visible
differences. [Differences from `basic_json`](differences.md) lists all of them.
## Where to go next
- [Ownership and lifetime](lifetime.md) - when a document borrows its source text, and how long a view stays valid
- [Differences from `basic_json`](differences.md) - everything a view does differently from a parsed `basic_json` value
- [Reading values](reading.md) - converting with `get<T>()`, reading strings and numbers without copying, and writing a
view back with `dump()`
- [Writing generic code](generic_code.md) - functions and `from_json()` overloads that accept views
- [Editing a document](editing.md) - patching a document in place, keeping its member order and number spellings
- [Images](images.md) - saving a parsed document and loading it back without parsing
## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
| | [`json`](../../api/json.md) / [`ordered_json`](../../api/ordered_json.md) | [SAX interface](../parsing/sax_interface.md) | [`json_document`](../../api/json_document.md) / [`json_view`](../../api/json_view.md) | [`json_editable_document`](../../api/json_editable_document.md) / [`json_editable_view`](../../api/json_editable_view.md) |
|---|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document | same as `json_document`; edits go to storage the document owns |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only | [`set`](../../api/basic_json_document/set.md)/[`push_back`](../../api/basic_json_document/push_back.md)/[`insert`](../../api/basic_json_document/insert.md)/[`erase`](../../api/basic_json_document/erase.md) edit in place; the source text is never rewritten |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use | the same, plus [`dump()`](../../api/basic_json_view/dump.md) of an edited document that keeps the member order and, with [`number_format::source`](../../api/basic_json_view/number_format.md), the spelling of every untouched number |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives | a document you read, patch a few fields of, and write back — a configuration file, for instance — where the rest of it should come back exactly as it was |
| **Caching/reload** | not applicable — re-parse, or roll your own serialization | not applicable | [`save()`](../../api/basic_json_document/save.md)/[`load()`](../../api/basic_json_document/load.md): cache the parsed index as an [image](images.md) and reload it without parsing | same, saving the document's current — possibly edited — state |
<p class="nj-table-caption" markdown>Each column is one way to work with JSON in this library, and each row compares one property of them.</p>
## See also
- [`basic_json_document`](../../api/basic_json_document/index.md) - the document that owns the parsed index (parse, access, images, edits)
- [`basic_json_view`](../../api/basic_json_view/index.md) - the non-owning handle to a value in a document
- [`json_document`](../../api/json_document.md) - and [`ordered_json_document`](../../api/ordered_json_document.md) - the read-only documents
- [`json_editable_document`](../../api/json_editable_document.md) - and [`ordered_json_editable_document`](../../api/ordered_json_editable_document.md) - the documents that can be edited in place
- [Architecture](../../home/architecture.md#node-index-of-json-views) - the layout of the node index
- [SAX interface](../parsing/sax_interface.md) - event-based parsing without building a tree
- [Performance](../performance.md) - benchmarks and tuning of parsing and serialization
- [Large, Streamed, and Partial Input](../parsing/large_and_partial_input.md) - reading only the part of a text you need
## Version history
- Added in version 3.13.0.
@@ -1,73 +0,0 @@
# Ownership and lifetime
A document either **borrows** the text it was parsed from, or **owns** its own copy of it; call
[`owns_source()`](../../api/basic_json_document/owns_source.md) to find out which happened.
[`parse()`](../../api/basic_json_document/parse.md) decides this from the value category and type of its argument (an
lvalue `#!cpp std::string` is borrowed; an rvalue `#!cpp std::string` is moved in, owned without a copy; a stream is
read into an owned buffer; and so on — see [`parse`'s Notes](../../api/basic_json_document/parse.md#notes) for the
full table). [`parse_copy()`](../../api/basic_json_document/parse_copy.md) always owns a copy, regardless of the input.
!!! warning "A borrowed document depends on your buffer"
If a document borrows its text, that text **must outlive the document** (and every view taken from it) and must
stay **unmodified** for as long as the document exists, since floating-point values and unescaped strings are
read from it lazily. Reading or writing through a view after the underlying buffer is gone is undefined behavior,
exactly as it would be for a dangling `#!cpp std::string_view`.
## When a view is valid
A view is valid only while all of the following hold:
- the document is alive,
- the document has not been re-parsed since the view was taken (with
[`read()`](../../api/basic_json_document/read.md) or [`parse()`](../../api/basic_json_document/parse.md) into it),
- the document has not had [`shrink_to_fit()`](../../api/basic_json_document/shrink_to_fit.md) called on it since, and
- if the document borrows its source text, that text is still alive and unmodified.
Moving the `basic_json_document<>` itself is fine and does **not** invalidate its views: the index is a separate heap
allocation that keeps its address across the move. Take a fresh view from
[`root()`](../../api/basic_json_document/root.md) whenever any of the other conditions above was not met. Edits of an
[editable document](editing.md) keep views valid, too; see [What stays valid](editing.md#what-stays-valid-and-what-an-edit-costs).
<div class="nj-fig-wrap" markdown>
--8<-- "images/json_view_lifetime.svg"
</div>
<p class="nj-caption" markdown>Four timelines of a document and its views, each bar showing when an object is alive or valid and when it dangles. Destroying the borrowed text, re-parsing, or shrinking the document invalidates the views taken before; moving the document does not.</p>
## Why views can dangle
A view can dangle, exactly like a `#!cpp std::string_view`: nothing stops you from using it after its document is
gone. This is deliberate. A view is two pointers and trivially copyable; making it safe (a `#!cpp std::shared_ptr` in
the document and a `#!cpp std::weak_ptr` in each view) would make it larger, add reference counting to every copy,
and a `lock()` to every access — exactly the costs the view exists to avoid. What is caught is the obvious case:
because a view dies with its document, [`root()`](../../api/basic_json_document/root.md) is not callable on a
temporary document: `#!cpp auto v = json_document::parse(text).root();` does not compile. Give the document a name
first.
The same holds for what a view hands out without copying: a string from
[`get_string()`](../../api/basic_json_view/get_string.md), a token from
[`number_token()`](../../api/basic_json_view/number_token.md), and [`string_view` members](reading.md#string_view-members)
of decoded structs are valid only as long as the document and, if it borrows, its source text.
??? example "Example: borrowed and owned documents, and when views become invalid"
```cpp
--8<-- "examples/json_view_ownership.cpp"
```
Output:
```json
--8<-- "examples/json_view_ownership.output"
```
## See also
- [Zero-copy JSON views](index.md) - overview
- [`basic_json_document`](../../api/basic_json_document/index.md) - the document that owns the parsed index
- [`owns_source()`](../../api/basic_json_document/owns_source.md) - whether a document borrows its source text
## Version history
- Added in version 3.13.0.
@@ -1,87 +0,0 @@
# Reading values
## Getting values out without copying
[`get<T>()`](../../api/basic_json_view/get.md) converts most `T` directly from the flat index, without ever building a
`basic_json` value for the conversion: `#!cpp bool`, arithmetic types, `#!cpp std::nullptr_t`,
`#!cpp std::string`/other `#!cpp std::basic_string`s (copied once), `basic_json_view` itself, the standard containers
and other types the library converts (`#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::list`, `#!cpp std::array`,
`#!cpp std::pair`, `#!cpp std::tuple`, `#!cpp std::optional`, ...), and the types of the `NLOHMANN_DEFINE_TYPE_*` and
`NLOHMANN_JSON_SERIALIZE_ENUM` macros. Types whose `from_json()` is written for `basic_json` only go through
[`materialize()`](../../api/basic_json_view/materialize.md)`.get<T>()` instead: the subtree is built into a real
`basic_json` value first, exactly as [`parse()`](../../api/basic_json/parse.md) would, and converted from there. See
[A `from_json()` that reads views](generic_code.md#a-from_json-that-reads-views) for how to write a `from_json()` that
reads views, and [Duplicate keys](differences.md#duplicate-keys) for the one case in which the two ways give different
results.
Two conversions never copy at all:
- [`get_string()`](../../api/basic_json_view/get_string.md) (equivalently, `#!cpp get<string_view_t>()`) returns a
string as a `string_view_t` pointing into the document's [`source()`](../../api/basic_json_document/source.md) text
— or, for a string that contains escape sequences, into the document's own buffer of decoded strings — instead of
allocating a new `#!cpp std::string`.
- [`number_token()`](../../api/basic_json_view/number_token.md) returns a number exactly as it was written in the
source, e.g. `#!cpp "1.50"`, `#!cpp "1E2"`, or an integer with more digits than any number type holds, instead of
rounding it into a `#!cpp double`/`#!cpp int64_t` the way `#!cpp get<T>()` (and
[`basic_json::parse()`](../../api/basic_json/parse.md)) would.
Both results are only valid as long as the view — and, for a string with no escapes, the borrowed source text — is.
### `string_view` members
A struct whose member is a `#!cpp std::string_view` (C++17) can be decoded from a view with the
`NLOHMANN_DEFINE_TYPE_*` macros, and the member is not copied either: for a string without escape sequences it points
into the document's source text, and for a string with escape sequences into the document's buffer of decoded
strings. The member is valid as long as the document is alive and, for a document that
[borrows its source text](lifetime.md) (as [`parse()`](../../api/basic_json_document/parse.md) does for an lvalue
`#!cpp std::string`, a `#!cpp std::string_view`, or a C string), as long as that text is.
```cpp
struct record
{
std::string_view name;
int count;
NLOHMANN_DEFINE_TYPE_INTRUSIVE(record, name, count)
};
const std::string text = R"({"name": "widget", "count": 3})";
const json_document doc = json_document::parse(text); // borrows text
const record r = doc.root().get<record>(); // r.name points into text, valid while text and doc are
```
!!! warning "Decoding into a `string_view` from a `json` value dangles at once"
The same struct also compiles with `#!cpp json::parse(text).get<record>()`, but there `name` points into the
temporary `json` value, which is destroyed at the end of the statement. Use a named
[`json_document`](../../api/json_document.md), as above, to decode such a struct.
## Writing a view back
[`dump()`](../../api/basic_json_view/dump.md) serializes a view directly from the flat index, without ever building a
`basic_json` value. An object's members are written in document order, not sorted by key, and *every* occurrence of a
repeated key is written, not only the last one — the same two ways
[iteration differs](differences.md#object-member-order) from a
[`materialize()`](../../api/basic_json_view/materialize.md)d value, see
[Duplicate keys](differences.md#duplicate-keys). `#!cpp materialize().dump()` gives a different result in both
respects for a `json_view`.
By default, numbers are written the way [`basic_json::dump()`](../../api/basic_json/dump.md) would.
[`number_format::source`](../../api/basic_json_view/number_format.md) instead copies every number exactly as it was
written in the source text — a price like `#!cpp 19.90`, a long order or account ID with more digits than any number
type holds, or a high-precision coordinate — something `basic_json` cannot do at all, since parsing already reduces
a number to its parsed `#!cpp double`/`#!cpp int64_t` value. Together with an [editable document](editing.md), this
lets you patch a file without reformatting the rest of it.
[`operator<<`](../../api/basic_json_view/operator_ltlt.md) writes a view to a stream the way `basic_json`'s does, using
the stream's `width`/`fill` for indentation.
## See also
- [Zero-copy JSON views](index.md) - overview
- [`get`](../../api/basic_json_view/get.md) - convert a view
- [`dump`](../../api/basic_json_view/dump.md) - serialize a view
- [Writing generic code](generic_code.md) - functions and `from_json()` overloads that accept views
## Version history
- Added in version 3.13.0.
@@ -88,7 +88,7 @@ a.erase(1); // [1,3,4] (erase by index)
## Editing a document ## Editing a document
The operations above work on a `basic_json` value. A JSON text that was parsed into a The operations above work on a `basic_json` value. A JSON text that was parsed into a
[document](json_view/editing.md) can also be changed in place, without building a `basic_json` tree for the [document](json_view.md#editing-a-document) can also be changed in place, without building a `basic_json` tree for the
parts that are not touched: [`set`](../api/basic_json_document/set.md) replaces a value, adds or replaces an object parts that are not touched: [`set`](../api/basic_json_document/set.md) replaces a value, adds or replaces an object
member, or replaces an array element (also through a JSON pointer), member, or replaces an array element (also through a JSON pointer),
[`push_back`](../api/basic_json_document/push_back.md) appends to an array, [`push_back`](../api/basic_json_document/push_back.md) appends to an array,
@@ -105,7 +105,7 @@ editable document types, such as [`json_editable_document`](../api/json_editable
- [`merge_patch`](../api/basic_json/merge_patch.md) - apply an RFC 7396 merge patch - [`merge_patch`](../api/basic_json/merge_patch.md) - apply an RFC 7396 merge patch
- [`erase`](../api/basic_json/erase.md) - / [`clear`](../api/basic_json/clear.md) - remove elements - [`erase`](../api/basic_json/erase.md) - / [`clear`](../api/basic_json/clear.md) - remove elements
- [JSON Patch and Diff](json_patch.md) - and [JSON Merge Patch](merge_patch.md) - structured modifications - [JSON Patch and Diff](json_patch.md) - and [JSON Merge Patch](merge_patch.md) - structured modifications
- [Editing a document](json_view/editing.md) - change a parsed document in place - [Editing a document](json_view.md#editing-a-document) - change a parsed document in place
- [`insert`](../api/basic_json/insert.md) - insert elements at an iterator position or merge a range - [`insert`](../api/basic_json/insert.md) - insert elements at an iterator position or merge a range
- [`operator+=`](../api/basic_json/operator+=.md) - shorthand for `push_back` - [`operator+=`](../api/basic_json/operator+=.md) - shorthand for `push_back`
- [Element Access](element_access/index.md) - read elements - [Element Access](element_access/index.md) - read elements
@@ -0,0 +1,122 @@
# Error Recovery
By default, parsing stops at the first error. With the [SAX interface](sax_interface.md), you can instead ask the
parser to *recover*: to repair the error and continue, so that you get as much as possible out of malformed input, for
instance a file that was cut off, JSON edited by hand, or the output of a language model.
## Recovering from errors
The SAX parser's [`parse_error`](../../api/json_sax/parse_error.md) function is called for every error. Its return value
decides what happens next:
- `#!cpp false` stops parsing. This is what the SAX parsers of the library do, so [`parse`](../../api/basic_json/parse.md)
and [`accept`](../../api/basic_json/accept.md) never recover.
- `#!cpp true` repairs the error and continues parsing.
When recovering, the SAX parser still receives well-formed events: every `start_object` or `start_array` is followed by
the matching `end_object` or `end_array`, and every `key` is followed by exactly one value. A SAX parser that creates a
JSON value, such as the one in the example below, therefore gets a complete value. Parsing always ends, and
[`sax_parse`](../../api/basic_json/sax_parse.md) returns `#!cpp false` for input that is not valid JSON, even if every
error was repaired. Each token is reported at most once, and the SAX parser can stop at any error by returning
`#!cpp false`.
!!! example
The example below derives a SAX parser from the library's parser for `json` values (`json_sax_dom_parser`),
and recovers from all errors.
```cpp
--8<-- "examples/sax_parse__error_recovery.cpp"
```
Output:
```
--8<-- "examples/sax_parse__error_recovery.output"
```
## How errors are repaired
Each error is repaired with the smallest local edit: a missing separator is inserted, a stray token is removed, what can
be read of a broken string or number is kept, and a value that cannot be read at all becomes `#!json null`.
| Mistake | Repair | Example | Result |
|---------------------------|--------------------------------------------------------------------------------|------------------------------------------|----------------------------|
| missing `,` or `:` | inserted | `#!json [1 2]`, `#!json {"a" 1}` | `[1,2]`, `{"a":1}` |
| missing value | `#!json null` for an object key or between commas in an array | `#!json {"a":}`, `#!json [1,,2]` | `{"a":null}`, `[1,null,2]` |
| trailing comma | removed | `#!json [1,2,]` | `[1,2]` |
| broken string | invalid escapes and bytes are replaced (see below); a line break ends the string | `#!json ["a\qb"]` | `["aqb"]` |
| broken number | the longest valid beginning is kept | `#!json [1., 2e+]` | `[1,2]` |
| unreadable value | `#!json null` | `#!json [1, NaN, tru]` | `[1,null,null]` |
| number too large | passed as infinity, together with its text | `#!json [1e999]` | infinity (see below) |
| stray `:` | removed | `#!json ["a":1]` | `["a",1]` |
| member without a key | skipped up to the next `,` or `}` | `#!json {1:2, "b":3}` | `{"b":3}` |
| wrong closing bracket | closes the innermost array or object | `#!json {"a":[1,2}, "b":3}` | `{"a":[1,2],"b":3}` |
| input ends too early | all open arrays and objects are closed | `#!json {"a":[1,2` | `{"a":[1,2]}` |
| text before the value | skipped | `#!json )]}'{"a":1}` | `{"a":1}` |
In a string, an unknown escape like `\q` stands for the escaped character (`q`), as in JavaScript. An invalid `\u`
escape, a lone surrogate, and ill-formed UTF-8 are each replaced by U+FFFD (REPLACEMENT CHARACTER), and control
characters are kept. A string without its closing quote ends at the next line break or at the end of the input.
The input after the top-level value is not repaired: as without recovery, it is reported as an error, and parsing stops.
## Binary formats
The binary formats ([BJData](../binary_formats/bjdata.md), [BON8](../binary_formats/bon8.md),
[BSON](../binary_formats/bson.md), [CBOR](../binary_formats/cbor.md), [MessagePack](../binary_formats/messagepack.md),
and [UBJSON](../binary_formats/ubjson.md)) have no delimiters to find the next value by. So what can be repaired depends
on whether the end of the item with the error is known, a distinction that
[RFC 8949, Section 5.3](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3) makes for CBOR, too.
If the item is complete, but cannot be passed on as it is, it is replaced, and parsing continues after it:
| Mistake | Formats | Repair |
|---------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------|
| tag, if `tag_handler` is `cbor_tag_handler_t::error` (the default) | CBOR | ignored |
| simple value other than `false`, `true`, and `null`, like undefined | CBOR | `#!json null` |
| number too large for a custom `number_float_t`, like `float` | all | infinity |
| character (`C`) that is not ASCII | BJData, UBJSON | U+FFFD |
| invalid high-precision number (`H`) | BJData, UBJSON | the longest valid beginning is kept, as for JSON text, or `#!json null` |
| high-precision number too large | BJData, UBJSON | passed as infinity, together with its text |
| object key that is not a string | BON8, CBOR, MessagePack | the member is skipped |
| element of a type the library does not read, like ObjectId or date | BSON | `#!json null` |
| string without its terminator | BSON | kept |
| document whose size does not match its content | BSON | kept |
CBOR tags and simple values are repaired as [RFC 8949, Section 6.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-6.1)
suggests for converting CBOR to JSON. By default, [`sax_parse`](../../api/basic_json/sax_parse.md) reports every tag as
an error; when recovering, tags are then ignored like with
[`cbor_tag_handler_t::ignore`](../../api/basic_json/cbor_tag_handler_t.md). Strings that are not valid UTF-8 are no
error: like [`from_cbor`](../../api/basic_json/from_cbor.md) and the other functions by default, `sax_parse` passes
them on as they are.
After any other error, the end of the item is unknown: the input ended, a byte is not a valid type marker, or a size
cannot be right. Parsing then stops, and the value read so far is completed: a key that waits for its value gets
`#!json null`, and all open arrays and objects are closed. This keeps everything before the error of an input that was
cut off. The exception is BSON, which stores the size of every document: an element whose end is unknown gets
`#!json null`, the rest of its document is skipped, and parsing continues after the document.
## Limitations
- A repair is a guess. For example, `#!json {"a" "b": 1}` could be meant as `#!json {"a": "b"}` or as
`#!json {"a": null, "b": 1}`; it is repaired to the former. Treat recovered values as a best effort, and check the
reported errors.
- A closing bracket always closes the innermost array or object. If a bracket is missing rather than wrong, the
repair differs from the intention: `#!json {"a": {"b": [1, 2}, "c": 3}` is repaired to
`#!json {"a": {"b": [1, 2], "c": 3}}`, although `#!json {"a": {"b": [1, 2]}, "c": 3}` may have been meant.
- Keys without quotes, and strings in single quotes, are not supported; such members are skipped.
- In the binary formats, a member that is skipped because its key is not a string is lost, and so are the elements of a
BSON document after one whose end is unknown.
- A number that is too large for `number_float_t` is passed as positive or negative infinity. The SAX parser's
`number_float` also gets the number's text, but a JSON value cannot store it, and
[`dump`](../../api/basic_json/dump.md) serializes infinity as `#!json null`.
- When parsing is not strict (see [`sax_parse`](../../api/basic_json/sax_parse.md)), a repair may read parts of the
input after the value, for instance of the next value in a stream of concatenated values.
## See also
- [SAX interface](sax_interface.md) - implement a custom SAX handler
- [`parse_error`](../../api/json_sax/parse_error.md) - the SAX event for parse errors
- [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events
- [parsing and exceptions](parse_exceptions.md) - control error handling
+5 -4
View File
@@ -110,7 +110,7 @@ The library offers two parsing models, and a third, read-only alternative:
reads the same JSON grammar but builds a flat, read-only index of the values instead of a `basic_json` tree. Strings and reads the same JSON grammar but builds a flat, read-only index of the values instead of a `basic_json` tree. Strings and
floating-point numbers are read from the text on demand, and only the parts you floating-point numbers are read from the text on demand, and only the parts you
[`materialize()`](../../api/basic_json_view/materialize.md) become `basic_json` values. This avoids most of the work when you [`materialize()`](../../api/basic_json_view/materialize.md) become `basic_json` values. This avoids most of the work when you
need only a few values from a large or frequently parsed text; see [Zero-copy JSON views](../json_view/index.md). It is read-only, so use need only a few values from a large or frequently parsed text; see [Zero-copy JSON views](../json_view.md). It is read-only, so use
`parse` when you need to modify the value. `parse` when you need to modify the value.
You can influence a DOM parse without switching to the SAX interface by passing a You can influence a DOM parse without switching to the SAX interface by passing a
@@ -134,14 +134,14 @@ json_document doc = json_document::parse(R"({"happy": true, "pi": 3.141})");
std::cout << doc.root()["happy"].get<bool>() << '\n'; std::cout << doc.root()["happy"].get<bool>() << '\n';
``` ```
See [Zero-copy JSON views](../json_view/index.md) for the concept and the trade-offs. See [Zero-copy JSON views](../json_view.md) for the concept and the trade-offs.
## Exceptions ## Exceptions
When the input is not valid JSON, the `parse` function throws an exception by default. If exceptions are undesired or When the input is not valid JSON, the `parse` function throws an exception by default. If exceptions are undesired or
unavailable, the parser can instead return a discarded value, or [`accept`](../../api/basic_json/accept.md) can be used unavailable, the parser can instead return a discarded value, or [`accept`](../../api/basic_json/accept.md) can be used
to only check whether an input is valid JSON. See [parsing and exceptions](parse_exceptions.md) for the available to only check whether an input is valid JSON. See [parsing and exceptions](parse_exceptions.md) for the available
options. options. To get as much as possible out of malformed input, a SAX parser can [recover from errors](error_recovery.md).
## See also ## See also
@@ -153,8 +153,9 @@ options.
- [Large, streamed, and partial input](large_and_partial_input.md) - chunked, concatenated, and very large input - [Large, streamed, and partial input](large_and_partial_input.md) - chunked, concatenated, and very large input
- [parser callbacks](parser_callbacks.md) - influence the parsing by a callback function - [parser callbacks](parser_callbacks.md) - influence the parsing by a callback function
- [SAX interface](sax_interface.md) - implement a custom SAX handler - [SAX interface](sax_interface.md) - implement a custom SAX handler
- [Zero-copy JSON views](../json_view/index.md) - parse into a read-only document and read it through views - [Zero-copy JSON views](../json_view.md) - parse into a read-only document and read it through views
- [parsing and exceptions](parse_exceptions.md) - control error handling - [parsing and exceptions](parse_exceptions.md) - control error handling
- [error recovery](error_recovery.md) - get as much as possible out of malformed input
- [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources - [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources
- [`operator>>`](../../api/operator_gtgt.md) - deserialize from stream - [`operator>>`](../../api/operator_gtgt.md) - deserialize from stream
- [`json_sax`](../../api/json_sax/index.md) - documentation of the SAX interface - [`json_sax`](../../api/json_sax/index.md) - documentation of the SAX interface
@@ -156,11 +156,11 @@ record the reason in the handler (here: `id`) if you need to tell the two apart.
### Read parts of a large document with a view ### Read parts of a large document with a view
If the whole text is available and you only need to read some of it, a [`json_document`](../json_view/index.md) from If the whole text is available and you only need to read some of it, a [`json_document`](../json_view.md) from
`<nlohmann/json_view.hpp>` (added in version 3.13.0) avoids the `basic_json` tree altogether: it keeps the text, builds a flat index of the values, `<nlohmann/json_view.hpp>` (added in version 3.13.0) avoids the `basic_json` tree altogether: it keeps the text, builds a flat index of the values,
and gives read-only [views](../../api/basic_json_view/index.md) of the parts you look at. Only what you and gives read-only [views](../../api/basic_json_view/index.md) of the parts you look at. Only what you
[`materialize()`](../../api/basic_json_view/materialize.md) becomes a `basic_json` value. See [`materialize()`](../../api/basic_json_view/materialize.md) becomes a `basic_json` value. See
[Zero-copy JSON views](../json_view/index.md) for details and for the lifetime rules. [Zero-copy JSON views](../json_view.md) for details and for the lifetime rules.
## See also ## See also
@@ -168,7 +168,7 @@ and gives read-only [views](../../api/basic_json_view/index.md) of the parts you
- [JSON Lines](json_lines.md) - newline-delimited JSON - [JSON Lines](json_lines.md) - newline-delimited JSON
- [Parser callbacks](parser_callbacks.md) - discard values while parsing - [Parser callbacks](parser_callbacks.md) - discard values while parsing
- [SAX interface](sax_interface.md) - event-based parsing - [SAX interface](sax_interface.md) - event-based parsing
- [Zero-copy JSON views](../json_view/index.md) - read parts of a large document without building a tree - [Zero-copy JSON views](../json_view.md) - read parts of a large document without building a tree
- [Untrusted input](untrusted_input.md) - limits for input you do not control - [Untrusted input](untrusted_input.md) - limits for input you do not control
- [`operator>>`](../../api/operator_gtgt.md) - deserialize from stream - [`operator>>`](../../api/operator_gtgt.md) - deserialize from stream
- [`parse`](../../api/basic_json/parse.md) - deserialize from a compatible input - [`parse`](../../api/basic_json/parse.md) - deserialize from a compatible input
@@ -68,7 +68,8 @@ bool parse_error(std::size_t position,
const json::exception& ex); const json::exception& ex);
``` ```
The return value indicates whether the parsing should continue, so the function should usually return `#!cpp false`. The return value decides whether to stop parsing (`#!cpp false`) or to repair the error and continue
(`#!cpp true`); see [error recovery](error_recovery.md) for the latter.
??? example "Example: report parse errors without exceptions" ??? example "Example: report parse errors without exceptions"
@@ -62,7 +62,8 @@ bool key(string_t& val);
bool parse_error(std::size_t position, const std::string& last_token, const json::exception& ex); bool parse_error(std::size_t position, const std::string& last_token, const json::exception& ex);
``` ```
The return value of each function determines whether parsing should proceed. The return value of each function determines whether parsing should proceed. For `parse_error`, returning
`#!cpp true` [recovers from the error](error_recovery.md).
Object keys are passed to `key` decoded and validated: escape sequences are already resolved, so different spellings of Object keys are passed to `key` decoded and validated: escape sequences are already resolved, so different spellings of
the same name yield the same string. See [`key`](../../api/json_sax/key.md#notes) for the exact guarantees. the same name yield the same string. See [`key`](../../api/json_sax/key.md#notes) for the exact guarantees.
@@ -101,7 +102,7 @@ To implement your own SAX handler, proceed as follows:
2. Create an object of your SAX interface class, e.g. `my_sax`. 2. Create an object of your SAX interface class, e.g. `my_sax`.
3. Call `#!cpp bool json::sax_parse(input, &my_sax);` where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface. 3. Call `#!cpp bool json::sax_parse(input, &my_sax);` where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface.
Note the `sax_parse` function only returns a `#!cpp bool` indicating the result of the last executed SAX event. It does not return `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error - it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file `json_sax.hpp`. Note the `sax_parse` function only returns a `#!cpp bool` indicating whether the input was parsed without errors and no SAX event returned `#!cpp false`. It does not return `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error - it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file `json_sax.hpp`.
## See also ## See also
+5 -5
View File
@@ -71,12 +71,12 @@ a DOM, three alternatives avoid building it:
discard finished elements as soon as they are handled, so memory usage stays bounded by one element (plus the discard finished elements as soon as they are handled, so memory usage stays bounded by one element (plus the
unparsed remainder of the input) instead of the whole document — see the [recipe for streaming a large homogeneous unparsed remainder of the input) instead of the whole document — see the [recipe for streaming a large homogeneous
array](parsing/parser_callbacks.md#recipe-streaming-a-large-homogeneous-array). array](parsing/parser_callbacks.md#recipe-streaming-a-large-homogeneous-array).
- Parse into a [`json_document`](json_view/index.md) (`#!cpp <nlohmann/json_view.hpp>`) instead of a `basic_json`. It keeps - Parse into a [`json_document`](json_view.md) (`#!cpp <nlohmann/json_view.hpp>`) instead of a `basic_json`. It keeps
the input text and builds a flat index of 16 bytes per value; strings and floating-point numbers are not copied, but the input text and builds a flat index of 16 bytes per value; strings and floating-point numbers are not copied, but
read from the text when needed. Read-only [views](../api/basic_json_view/index.md) give the familiar element access, and only the read from the text when needed. Read-only [views](../api/basic_json_view/index.md) give the familiar element access, and only the
parts you [`materialize()`](../api/basic_json_view/materialize.md) become `basic_json` values. A document that parts you [`materialize()`](../api/basic_json_view/materialize.md) become `basic_json` values. A document that
borrows the text instead of owning a copy needs the text to outlive it; see borrows the text instead of owning a copy needs the text to outlive it; see
[choosing between `json`, the SAX interface, and `json_view`](json_view/index.md#choosing-between-json-ordered_json-the-sax-interface-and-json_view). [choosing between `json`, the SAX interface, and `json_view`](json_view.md#choosing-between-json-ordered_json-the-sax-interface-and-json_view).
If the data is naturally record-oriented, consider [JSON Lines](parsing/json_lines.md) instead of one large JSON If the data is naturally record-oriented, consider [JSON Lines](parsing/json_lines.md) instead of one large JSON
document: reading and parsing it line by line with `#!cpp std::getline` means only one line's value is ever in memory document: reading and parsing it line by line with `#!cpp std::getline` means only one line's value is ever in memory
@@ -180,11 +180,11 @@ an adapter, since their template argument order does not match what `basic_json`
large value therefore doubles its memory; see [Avoiding copies](#avoiding-copies). `#!cpp auto x = j["key"];` is a large value therefore doubles its memory; see [Avoiding copies](#avoiding-copies). `#!cpp auto x = j["key"];` is a
copy, `#!cpp auto& x = j["key"];` is not. copy, `#!cpp auto& x = j["key"];` is not.
- **A moved-from value is `null`.** After `#!cpp json c = std::move(a);` the value `a` is `#!json null` and owns nothing. - **A moved-from value is `null`.** After `#!cpp json c = std::move(a);` the value `a` is `#!json null` and owns nothing.
- **Reading does not require a tree.** If you only read the data, parse it into a [`json_document`](json_view/index.md) - **Reading does not require a tree.** If you only read the data, parse it into a [`json_document`](json_view.md)
(`#!cpp <nlohmann/json_view.hpp>`, added in version 3.13.0) instead of a `basic_json`. It stores 16 bytes per value, does not copy floating-point (`#!cpp <nlohmann/json_view.hpp>`, added in version 3.13.0) instead of a `basic_json`. It stores 16 bytes per value, does not copy floating-point
numbers or strings without escape sequences out of the text, and creates no `basic_json` value except for the parts numbers or strings without escape sequences out of the text, and creates no `basic_json` value except for the parts
you [`materialize()`](../api/basic_json_view/materialize.md). A document that borrows the text needs the text to you [`materialize()`](../api/basic_json_view/materialize.md). A document that borrows the text needs the text to
outlive it; see [Zero-copy JSON views](json_view/index.md). outlive it; see [Zero-copy JSON views](json_view.md).
- **Containers keep their capacity.** `clear()` and `erase()` on an array remove the elements, but the underlying - **Containers keep their capacity.** `clear()` and `erase()` on an array remove the elements, but the underlying
`#!cpp std::vector` keeps its allocated capacity, like `#!cpp std::vector::clear` does. To give the memory back, `#!cpp std::vector` keeps its allocated capacity, like `#!cpp std::vector::clear` does. To give the memory back,
assign a new value (`#!cpp j = json::array();`) or destroy the value. assign a new value (`#!cpp j = json::array();`) or destroy the value.
@@ -265,7 +265,7 @@ those headers are then never processed by the compiler at all.
- [Architecture](../home/architecture.md) - how input adapters, the lexer, and the serializer fit together - [Architecture](../home/architecture.md) - how input adapters, the lexer, and the serializer fit together
- [Parsing](parsing/index.md) - the available parsing functions and inputs - [Parsing](parsing/index.md) - the available parsing functions and inputs
- [SAX interface](parsing/sax_interface.md) - parse without building a DOM - [SAX interface](parsing/sax_interface.md) - parse without building a DOM
- [Zero-copy JSON views](json_view/index.md) - parse into a flat index of the text and read it without building a DOM - [Zero-copy JSON views](json_view.md) - parse into a flat index of the text and read it without building a DOM
- [Binary formats](binary_formats/index.md) - compact alternatives to JSON text - [Binary formats](binary_formats/index.md) - compact alternatives to JSON text
- [Object Order](object_order.md) - `json` vs. `ordered_json` and other `ObjectType` choices - [Object Order](object_order.md) - `json` vs. `ordered_json` and other `ObjectType` choices
- [Template Parameter Requirements](types/template_parameters.md) - custom container and allocator types - [Template Parameter Requirements](types/template_parameters.md) - custom container and allocator types
@@ -406,7 +406,6 @@ using array_t = ArrayType<basic_json, AllocatorType<basic_json>>;
| `#!cpp operator<<(std::ostream&, const json_pointer&)` | streamability to `#!cpp std::ostream` | | `#!cpp operator<<(std::ostream&, const json_pointer&)` | streamability to `#!cpp std::ostream` |
| [`to_string`](../../api/basic_json/to_string.md) | conversion of `StringType` to `#!cpp std::string` (the function returns a `#!cpp std::string`) | | [`to_string`](../../api/basic_json/to_string.md) | conversion of `StringType` to `#!cpp std::string` (the function returns a `#!cpp std::string`) |
| exception messages | `data()` and `size()`, or `begin()` and `end()` | | exception messages | `data()` and `size()`, or `begin()` and `end()` |
| [`basic_json_view::dump`](../../api/basic_json_view/dump.md) | `reserve(n)` and `resize(n)` (one argument); `capacity()` is **not** required |
### Compatible types ### Compatible types
+3 -3
View File
@@ -56,7 +56,7 @@ The public headers are in [`include/nlohmann`](https://github.com/nlohmann/json/
- [`json_view.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json_view.hpp) is a separate, - [`json_view.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json_view.hpp) is a separate,
optional header that defines [`basic_json_document`](../api/basic_json_document/index.md) and optional header that defines [`basic_json_document`](../api/basic_json_document/index.md) and
[`basic_json_view`](../api/basic_json_view/index.md), a flat-index, read-only, non-owning way to look at a parsed [`basic_json_view`](../api/basic_json_view/index.md), a flat-index, read-only, non-owning way to look at a parsed
JSON text; see [Zero-copy JSON views](../features/json_view/index.md). It builds on `json.hpp` internals (it requires the JSON text; see [Zero-copy JSON views](../features/json_view.md). It builds on `json.hpp` internals (it requires the
same library version) and has its own `detail/view/` subdirectory. same library version) and has its own `detail/view/` subdirectory.
Everything else lives in [`detail/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail) and namespace `nlohmann::detail`, which is not part of the public API. Paths Everything else lives in [`detail/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail) and namespace `nlohmann::detail`, which is not part of the public API. Paths
@@ -175,7 +175,7 @@ maintains the invariant that the pointer matching `m_type` is never null; `asser
## Node index of JSON views ## Node index of JSON views
A [`basic_json_document`](../api/basic_json_document/index.md) (see [Zero-copy JSON views](../features/json_view/index.md)) A [`basic_json_document`](../api/basic_json_document/index.md) (see [Zero-copy JSON views](../features/json_view.md))
does not build a tree of values. Its parser does not build a tree of values. Its parser
([`detail/view/builder.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/builder.hpp)) ([`detail/view/builder.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/builder.hpp))
writes a flat array of 16-byte nodes, one per value and one per object key, in document order. A writes a flat array of 16-byte nodes, one per value and one per object key, in document order. A
@@ -466,7 +466,7 @@ release. Besides the components above, it contains:
- [`adl_serializer`](../api/adl_serializer/index.md) - customization point for type conversions - [`adl_serializer`](../api/adl_serializer/index.md) - customization point for type conversions
- [`json_sax`](../api/json_sax/index.md) - interface of the SAX parser events - [`json_sax`](../api/json_sax/index.md) - interface of the SAX parser events
- [SAX Interface](../features/parsing/sax_interface.md) - how to consume parser events without building a DOM - [SAX Interface](../features/parsing/sax_interface.md) - how to consume parser events without building a DOM
- [Zero-copy JSON views](../features/json_view/index.md) - the flat node index that backs views and documents - [Zero-copy JSON views](../features/json_view.md) - the flat node index that backs views and documents
- [Template Parameter Requirements](../features/types/template_parameters.md) - the template parameters of basic_json - [Template Parameter Requirements](../features/types/template_parameters.md) - the template parameters of basic_json
- [Serialization](../features/serialization.md) - the output side of the library - [Serialization](../features/serialization.md) - the output side of the library
- [`nlohmann` Namespace](../features/namespace.md) - how the library namespace is structured - [`nlohmann` Namespace](../features/namespace.md) - how the library namespace is structured
+7 -7
View File
@@ -438,7 +438,7 @@ A UBJSON or BJData high-precision number could not be parsed.
### json.exception.parse_error.116 ### json.exception.parse_error.116
[`basic_json_document::load()`](../api/basic_json_document/load.md) rejected an [`basic_json_document::load()`](../api/basic_json_document/load.md) rejected an
[image](../features/json_view/images.md): either the bytes are not one [`save()`](../api/basic_json_document/save.md) [image](../features/json_view.md#images): either the bytes are not one [`save()`](../api/basic_json_document/save.md)
could have written (too short, an unknown magic number or format version, or sizes that do not fit the buffer), or could have written (too short, an unknown magic number or format version, or sizes that do not fit the buffer), or
they are, but fail the requested [`image_check`](../api/basic_json_document/load.md#image_check). they are, but fail the requested [`image_check`](../api/basic_json_document/load.md#image_check).
@@ -450,7 +450,7 @@ they are, but fail the requested [`image_check`](../api/basic_json_document/load
!!! note !!! note
This exception was added in version 3.13.0, together with [images](../features/json_view/images.md). This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
## Iterator errors ## Iterator errors
@@ -889,12 +889,12 @@ from JSON text.
!!! note !!! note
This exception was added in version 3.13.0, together with editable [`json_document`s](../features/json_view/index.md). This exception was added in version 3.13.0, together with editable [`json_document`s](../features/json_view.md).
### json.exception.type_error.320 ### json.exception.type_error.320
[`basic_json_document::save()`](../api/basic_json_document/save.md) cannot write an [`basic_json_document::save()`](../api/basic_json_document/save.md) cannot write an
[image](../features/json_view/images.md) of a [discarded](../api/basic_json_document/is_discarded.md) document. [image](../features/json_view.md#images) of a [discarded](../api/basic_json_document/is_discarded.md) document.
!!! failure "Example messages" !!! failure "Example messages"
@@ -904,7 +904,7 @@ from JSON text.
!!! note !!! note
This exception was added in version 3.13.0, together with [images](../features/json_view/images.md). This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
### json.exception.type_error.321 ### json.exception.type_error.321
@@ -1192,7 +1192,7 @@ so they do not support an input of 4294967280 bytes (4 GiB minus 16 bytes) or mo
storage: [`set`](../api/basic_json_document/set.md) and [`push_back`](../api/basic_json_document/push_back.md) throw storage: [`set`](../api/basic_json_document/set.md) and [`push_back`](../api/basic_json_document/push_back.md) throw
this exception once the strings and number tokens written by edits reach 4 GiB in total, or once more than this exception once the strings and number tokens written by edits reach 4 GiB in total, or once more than
4294967295 arrays/objects have had an element set or appended to them. The same limit applies to an 4294967295 arrays/objects have had an element set or appended to them. The same limit applies to an
[image](../features/json_view/images.md): [`save()`](../api/basic_json_document/save.md) throws it if the node [image](../features/json_view.md#images): [`save()`](../api/basic_json_document/save.md) throws it if the node
count, the text, or the decoded strings it would write would individually reach 4 GiB. count, the text, or the decoded strings it would write would individually reach 4 GiB.
!!! failure "Example messages" !!! failure "Example messages"
@@ -1209,7 +1209,7 @@ count, the text, or the decoded strings it would write would individually reach
!!! note !!! note
This exception was added in version 3.13.0, together with [`<nlohmann/json_view.hpp>`](../features/json_view/index.md). This exception was added in version 3.13.0, together with [`<nlohmann/json_view.hpp>`](../features/json_view.md).
[`basic_json::parse()`](../api/basic_json/parse.md) has no such limit. [`basic_json::parse()`](../api/basic_json/parse.md) has no such limit.
## Further exceptions ## Further exceptions
+1 -1
View File
@@ -140,7 +140,7 @@ Read newline-delimited input line by line ([JSON Lines](../features/parsing/json
Handle and discard the records of a large array with a [parser callback](../features/parsing/parser_callbacks.md), stop Handle and discard the records of a large array with a [parser callback](../features/parsing/parser_callbacks.md), stop
early with the [SAX interface](../features/parsing/sax_interface.md), or read parts of the text through a early with the [SAX interface](../features/parsing/sax_interface.md), or read parts of the text through a
[`json_document`](../features/json_view/index.md) (added in version 3.13.0). See [`json_document`](../features/json_view.md) (added in version 3.13.0). See
[Input too large for memory](../features/parsing/large_and_partial_input.md#input-too-large-for-memory). [Input too large for memory](../features/parsing/large_and_partial_input.md#input-too-large-for-memory).
### Position of a parse error ### Position of a parse error
+2 -2
View File
@@ -58,7 +58,7 @@ Clang).
anything, or link a library. Put the file in a directory named `nlohmann` that is on your include path (for anything, or link a library. Put the file in a directory named `nlohmann` that is on your include path (for
example `third_party/nlohmann/json.hpp` with `-I third_party`), so that `#include <nlohmann/json.hpp>` works. example `third_party/nlohmann/json.hpp` with `-I third_party`), so that `#include <nlohmann/json.hpp>` works.
Download the companion headers listed below only if you use them: `json_view.hpp` for the Download the companion headers listed below only if you use them: `json_view.hpp` for the
[zero-copy views](../features/json_view/index.md), `json_fwd.hpp` for forward declarations, and `json_literals.hpp` if [zero-copy views](../features/json_view.md), `json_fwd.hpp` for forward declarations, and `json_literals.hpp` if
you define `JSON_NO_AUTOMATIC_UDLS`. Keep them in the same `nlohmann` directory. you define `JSON_NO_AUTOMATIC_UDLS`. Keep them in the same `nlohmann` directory.
Do not copy only parts of the repository's `include/` directory: the headers in it include each other and must be Do not copy only parts of the repository's `include/` directory: the headers in it include each other and must be
@@ -75,7 +75,7 @@ for the user-defined string literals if you define
For the read-only, non-owning [`basic_json_document`](../api/basic_json_document/index.md)/[`basic_json_view`](../api/basic_json_view/index.md) For the read-only, non-owning [`basic_json_document`](../api/basic_json_document/index.md)/[`basic_json_view`](../api/basic_json_view/index.md)
types, additionally include types, additionally include
[`single_include/nlohmann/json_view.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_view.hpp); [`single_include/nlohmann/json_view.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_view.hpp);
see [Zero-copy JSON views](../features/json_view/index.md). see [Zero-copy JSON views](../features/json_view.md).
## Notes for builds and distribution ## Notes for builds and distribution
+1 -1
View File
@@ -1,4 +1,4 @@
/* figures: jsonview -- stepper for the edit frames on features/json_view/index.md. /* figures: jsonview -- stepper for the edit frames on features/json_view.md.
Without JS the frames (.nj-jsonview-step) show one below the other; with JS one frame at a time, with Without JS the frames (.nj-jsonview-step) show one below the other; with JS one frame at a time, with
previous/next and one button per step. Never advances on its own. */ previous/next and one button per step. Never advances on its own. */
(function () { (function () {
+2 -8
View File
@@ -142,6 +142,7 @@ nav:
- features/performance.md - features/performance.md
- Parsing: - Parsing:
- features/parsing/index.md - features/parsing/index.md
- features/parsing/error_recovery.md
- features/parsing/json_lines.md - features/parsing/json_lines.md
- features/parsing/large_and_partial_input.md - features/parsing/large_and_partial_input.md
- features/parsing/parse_exceptions.md - features/parsing/parse_exceptions.md
@@ -157,14 +158,7 @@ nav:
- features/types/index.md - features/types/index.md
- features/types/number_handling.md - features/types/number_handling.md
- features/types/template_parameters.md - features/types/template_parameters.md
- Zero-copy views: - features/json_view.md
- features/json_view/index.md
- features/json_view/lifetime.md
- features/json_view/differences.md
- features/json_view/reading.md
- features/json_view/generic_code.md
- features/json_view/editing.md
- features/json_view/images.md
- Integration: - Integration:
- integration/index.md - integration/index.md
- integration/migration_guide.md - integration/migration_guide.md
+2 -2
View File
@@ -1,8 +1,8 @@
wheel==0.48.0 wheel==0.48.0
mkdocs==1.6.1 # documentation framework mkdocs==1.6.1 # documentation framework
mkdocs-git-revision-date-localized-plugin==1.6.2 # plugin "git-revision-date-localized" mkdocs-git-revision-date-localized-plugin==1.6.0 # plugin "git-revision-date-localized"
mkdocs-material==9.8.0 # theme for mkdocs mkdocs-material==9.7.7 # theme for mkdocs
mkdocs-material-extensions==1.3.1 # extensions mkdocs-material-extensions==1.3.1 # extensions
mkdocs-minify-plugin==0.8.0 # plugin "minify" mkdocs-minify-plugin==0.8.0 # plugin "minify"
mkdocs-redirects==1.2.3 # plugin "redirects" mkdocs-redirects==1.2.3 # plugin "redirects"
@@ -91,7 +91,7 @@ const typename BasicJsonType::binary_t& binary_value(const BasicJsonType& j)
} }
template<typename View, enable_if_t<is_json_view<View>::value, int> = 0> template<typename View, enable_if_t<is_json_view<View>::value, int> = 0>
typename View::basic_json_type::binary_t binary_value(const View& /*unused*/) // LCOV_EXCL_LINE typename View::basic_json_type::binary_t binary_value(const View& /*unused*/)
{ {
return {}; // LCOV_EXCL_LINE return {}; // LCOV_EXCL_LINE
} }
+10 -12
View File
@@ -198,19 +198,18 @@ class exception : public std::exception
// all other parent types (primitives) contribute no token // all other parent types (primitives) contribute no token
} }
// the root value has no path, but still has a byte range if (tokens.empty())
std::string prefix;
if (!tokens.empty())
{ {
const auto str = std::accumulate(tokens.rbegin(), tokens.rend(), std::string{}, return "";
[](const std::string & a, const std::string & b)
{
return concat(a, '/', detail::escape(b));
});
prefix = concat('(', str, ") ");
} }
return concat(prefix, get_byte_positions(leaf_element)); auto str = std::accumulate(tokens.rbegin(), tokens.rend(), std::string{},
[](const std::string & a, const std::string & b)
{
return concat(a, '/', detail::escape(b));
});
return concat('(', str, ") ", get_byte_positions(leaf_element));
#else #else
return get_byte_positions(leaf_element); return get_byte_positions(leaf_element);
#endif #endif
@@ -223,8 +222,7 @@ class exception : public std::exception
template<typename BasicJsonType> template<typename BasicJsonType>
static std::string get_byte_positions(const BasicJsonType* leaf_element) static std::string get_byte_positions(const BasicJsonType* leaf_element)
{ {
// a value-initialized iterator has no value (invalid_iterator.212) if ((leaf_element->start_pos() != std::string::npos) && (leaf_element->end_pos() != std::string::npos))
if (leaf_element != nullptr && leaf_element->start_pos() != std::string::npos && leaf_element->end_pos() != std::string::npos)
{ {
return concat("(bytes ", std::to_string(leaf_element->start_pos()), "-", std::to_string(leaf_element->end_pos()), ") "); return concat("(bytes ", std::to_string(leaf_element->start_pos()), "-", std::to_string(leaf_element->end_pos()), ") ");
} }
File diff suppressed because it is too large. Load diff
@@ -545,9 +545,12 @@ class input_stream_adapter
std::char_traits<char>::int_type get_character_slow() std::char_traits<char>::int_type get_character_slow()
{ {
#if !JSON_DISABLE_STREAM_BULK_SCAN #if !JSON_DISABLE_STREAM_BULK_SCAN
// the get area is exhausted (get_character() handles the other case), // sbumpc() refills the get area if it is exhausted
// so sbumpc() refills it const bool refill = streambuf_access::get_ptr(sb) == streambuf_access::get_end(sb);
save_token_segment(); if (JSON_HEDLEY_UNLIKELY(refill))
{
save_token_segment();
}
#endif #endif
auto res = sb->sbumpc(); auto res = sb->sbumpc();
@@ -560,13 +563,17 @@ class input_stream_adapter
is->clear(is->rdstate() | std::ios::eofbit); is->clear(is->rdstate() | std::ios::eofbit);
} }
#if !JSON_DISABLE_STREAM_BULK_SCAN #if !JSON_DISABLE_STREAM_BULK_SCAN
else else if (JSON_HEDLEY_UNLIKELY(refill))
{ {
// the consumed character is at gptr() - 1, unless the streambuf // the consumed character is at gptr() - 1, unless the streambuf
// has no get area and returned it from uflow() // has no get area and returned it from uflow()
const char* const next = streambuf_access::get_ptr(sb); const char* const next = streambuf_access::get_ptr(sb);
after_refill(res, next != nullptr && next != streambuf_access::get_begin(sb)); after_refill(res, next != nullptr && next != streambuf_access::get_begin(sb));
} }
else
{
current_in_get_area = true;
}
#endif #endif
return res; return res;
} }
@@ -789,17 +796,22 @@ class iterator_input_adapter
return get_consumed_count() + eof_reads; return get_consumed_count() + eof_reads;
} }
// number of newlines in the consumed range [0, last_index); @a line_start // number of newlines in the consumed range [first_index, last_index); if
// is set to the index after the last of them (0 if there is none); only // there is one, @a line_start is set to the index after the last of them,
// valid when supports_seek // otherwise it is left unchanged; only valid when supports_seek
std::size_t count_consumed_lines(std::size_t last_index, std::size_t& line_start) const std::size_t count_consumed_lines(std::size_t first_index, std::size_t last_index, std::size_t& line_start) const
{ {
const auto to = std::next(begin, static_cast<typename std::iterator_traits<IteratorType>::difference_type>(last_index)); using difference_type = typename std::iterator_traits<IteratorType>::difference_type;
const auto from = std::next(begin, static_cast<difference_type>(first_index));
const auto to = std::next(from, static_cast<difference_type>(last_index - first_index));
const auto newline = char_traits<char_type>::to_char_type('\n'); const auto newline = char_traits<char_type>::to_char_type('\n');
using reverse_it = std::reverse_iterator<IteratorType>; using reverse_it = std::reverse_iterator<IteratorType>;
const auto last_newline = std::find(reverse_it(to), reverse_it(begin), newline); const auto last_newline = std::find(reverse_it(to), reverse_it(from), newline);
line_start = static_cast<std::size_t>(std::distance(begin, last_newline.base())); if (last_newline.base() != from)
return static_cast<std::size_t>(std::count(begin, last_newline.base(), newline)); {
line_start = static_cast<std::size_t>(std::distance(begin, last_newline.base()));
}
return static_cast<std::size_t>(std::count(from, last_newline.base(), newline));
} }
// append the already-consumed characters in the half-open range // append the already-consumed characters in the half-open range
+16 -28
View File
@@ -138,7 +138,9 @@ struct json_sax
@param[in] position the position in the input where the error occurs @param[in] position the position in the input where the error occurs
@param[in] last_token the last read token @param[in] last_token the last read token
@param[in] ex an exception object describing the error @param[in] ex an exception object describing the error
@return whether parsing should proceed (must return false) @return whether to recover from the error: false stops parsing; true
repairs the error and continues, or, if that is not possible,
stops after completing the value read so far
*/ */
virtual bool parse_error(std::size_t position, virtual bool parse_error(std::size_t position,
const std::string& last_token, const std::string& last_token,
@@ -485,6 +487,9 @@ stored in place as it arrives, so after a parse error the value holds what was
read up to it. read up to it.
@tparam BasicJsonType the JSON type @tparam BasicJsonType the JSON type
@tparam InputAdapterType the input adapter of the lexer that can be passed to
the constructor to record diagnostic positions; it
does not matter if no lexer is passed
@tparam Collect whether to collect the values of containers of unknown size on @tparam Collect whether to collect the values of containers of unknown size on
a stack (only for a caller that does not use the value after a a stack (only for a caller that does not use the value after a
parse error) parse error)
@@ -493,7 +498,7 @@ read up to it.
it, so any type with its three functions works (as it, so any type with its three functions works (as
materialize() of json_view uses it) materialize() of json_view uses it)
*/ */
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false, typename LexerType = lexer<BasicJsonType, InputAdapterType>> template<typename BasicJsonType, typename InputAdapterType = string_input_adapter_type, bool Collect = false, typename LexerType = lexer<BasicJsonType, InputAdapterType>>
class json_sax_dom_parser : public json_sax_dom_error_state class json_sax_dom_parser : public json_sax_dom_error_state
{ {
public: public:
@@ -799,6 +804,8 @@ class json_sax_dom_parser : public json_sax_dom_error_state
JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NEVER_INLINE
void grow() void grow()
{ {
// the stack starts with local_capacity frames and only grows
JSON_ASSERT(capacity != 0);
const std::size_t new_capacity = capacity * 2; const std::size_t new_capacity = capacity * 2;
std::unique_ptr<frame[]> new_heap(new frame[new_capacity]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) std::unique_ptr<frame[]> new_heap(new frame[new_capacity]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
for (std::size_t i = 0; i < count; ++i) for (std::size_t i = 0; i < count; ++i)
@@ -999,7 +1006,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
ordered_data.members[f.ordered.dest_member].second = std::move(value); ordered_data.members[f.ordered.dest_member].second = std::move(value);
} }
static void place_member(const frame& /*f*/, BasicJsonType&& /*value*/, std::false_type /*indexed*/) noexcept {} // LCOV_EXCL_LINE static void place_member(const frame& /*f*/, BasicJsonType&& /*value*/, std::false_type /*indexed*/) noexcept {}
/// store the members of a new object directly in it /// store the members of a new object directly in it
bool start_stored_object(std::size_t len) bool start_stored_object(std::size_t len)
@@ -1303,7 +1310,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
{ {
// a new member is null and holds nothing: construct the value // a new member is null and holds nothing: construct the value
// over it instead of destroying and swapping // over it instead of destroying and swapping
construct_member(member, std::forward<Value>(v), std::is_nothrow_constructible<BasicJsonType, Value> {}); ::new (static_cast<void*>(&member)) BasicJsonType(std::forward<Value>(v));
} }
else else
{ {
@@ -1312,26 +1319,6 @@ class json_sax_dom_parser : public json_sax_dom_error_state
} }
} }
/// constructing the value cannot throw (for example a number or a boolean): construct it in place
template<typename Value>
JSON_HEDLEY_ALWAYS_INLINE
static void construct_member(BasicJsonType& member, Value&& v, std::true_type /*nothrow*/)
{
::new (static_cast<void*>(&member)) BasicJsonType(std::forward<Value>(v));
}
/// constructing the value can throw (for example a container or a string), and a
/// constructor that was interrupted must not have touched the member (#5985):
/// create the value first, then move it over the member (the move constructor is
/// noexcept)
template<typename Value>
JSON_HEDLEY_ALWAYS_INLINE
static void construct_member(BasicJsonType& member, Value&& v, std::false_type /*may throw*/)
{
BasicJsonType value(std::forward<Value>(v));
::new (static_cast<void*>(&member)) BasicJsonType(std::move(value));
}
/// a custom base class may care about the lifetime of the value /// a custom base class may care about the lifetime of the value
template<typename Value> template<typename Value>
static void set_member(BasicJsonType& member, Value&& v, std::false_type /*custom base*/) static void set_member(BasicJsonType& member, Value&& v, std::false_type /*custom base*/)
@@ -1634,7 +1621,7 @@ class json_sax_dom_binary_parser : public json_sax_dom_error_state
lexer_t* m_lexer_ref = nullptr; lexer_t* m_lexer_ref = nullptr;
}; };
template<typename BasicJsonType, typename InputAdapterType> template<typename BasicJsonType, typename InputAdapterType = string_input_adapter_type>
class json_sax_dom_callback_parser : public json_sax_dom_error_state class json_sax_dom_callback_parser : public json_sax_dom_error_state
{ {
public: public:
@@ -1757,7 +1744,7 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
key_stack.push_back(val); key_stack.push_back(val);
// add discarded value at the given key and store the reference for later // add discarded value at the given key and store the reference for later
if (keep) if (keep && ref_stack.back())
{ {
auto& obj = *ref_stack.back()->m_data.m_value.object; auto& obj = *ref_stack.back()->m_data.m_value.object;
// a single lookup: insert the placeholder, or find the existing member // a single lookup: insert the placeholder, or find the existing member
@@ -1814,7 +1801,7 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
const string_t object_key = pop_container(); const string_t object_key = pop_container();
if (!ref_stack.empty() && ref_stack.back()) if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_structured())
{ {
// remove discarded value // remove discarded value
remove_discarded_value(*ref_stack.back(), object_key); remove_discarded_value(*ref_stack.back(), object_key);
@@ -1980,7 +1967,8 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
*/ */
string_t current_key() const string_t current_key() const
{ {
if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_object()) if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_object()
&& !key_stack.empty())
{ {
return key_stack.back(); return key_stack.back();
} }
+681 -37
View File
@@ -10,7 +10,7 @@
#include <array> // array #include <array> // array
#include <cstddef> // size_t #include <cstddef> // size_t
#include <cstdint> // uint32_t #include <cstdint> // uint8_t, uint32_t
#include <cstdio> // snprintf #include <cstdio> // snprintf
#include <cstring> // memcmp #include <cstring> // memcmp
#include <initializer_list> // initializer_list #include <initializer_list> // initializer_list
@@ -300,16 +300,14 @@ class lexer : public lexer_base<BasicJsonType>
/// calls would (including the position and token bookkeeping of a /// calls would (including the position and token bookkeeping of a
/// streaming adapter, see bulk_consume) and leaves @a current holding the last of /// streaming adapter, see bulk_consume) and leaves @a current holding the last of
/// the 4 digits, just as the last such get() would; the codepoint is /// the 4 digits, just as the last such get() would; the codepoint is
/// written to @a out. Makes no state change and returns false for fewer /// written to @a out. Makes no state change and returns false for a
/// than 4 remaining bytes or any of the 4 bytes not being a hex digit, so /// pending unget, fewer than 4 remaining bytes, or any of the 4 bytes not
/// the caller falls back unchanged to the per-character loop, which then /// being a hex digit, so the caller falls back unchanged to the
/// reports the same diagnostic (stopping at the first invalid digit) as /// per-character loop, which then reports the same diagnostic (stopping
/// before this optimization. /// at the first invalid digit) as before this optimization.
bool get_codepoint_bulk(std::true_type /*bulk*/, int& out) bool get_codepoint_bulk(std::true_type /*bulk*/, int& out)
{ {
// no unget is pending inside a string (see get_after_string_run()) if (next_unget || ia.bulk_remaining() < 4)
JSON_ASSERT(!next_unget);
if (ia.bulk_remaining() < 4)
{ {
return false; return false;
} }
@@ -387,6 +385,20 @@ class lexer : public lexer_base<BasicJsonType>
return codepoint; return codepoint;
} }
/// the code point of the surrogate pair @a high (U+D800..U+DBFF) and @a low (U+DC00..U+DFFF)
static constexpr int combine_surrogates(const int high, const int low) noexcept
{
return static_cast<int>(
// high surrogate occupies the most significant 22 bits
(static_cast<unsigned int>(high) << 10u)
// low surrogate occupies the least significant 15 bits
+ static_cast<unsigned int>(low)
// there is still the 0xD800, 0xDC00, and 0x10000 noise
// in the result, so we have to subtract with:
// (0xD800 << 10) + DC00 - 0x10000 = 0x35FDC00
- 0x35FDC00u);
}
/*! /*!
@brief check if the next byte(s) are inside a given range @brief check if the next byte(s) are inside a given range
@@ -420,6 +432,7 @@ class lexer : public lexer_base<BasicJsonType>
else else
{ {
error_message = "invalid string: ill-formed UTF-8 byte"; error_message = "invalid string: ill-formed UTF-8 byte";
error_cause = error_kind::string_ill_formed_utf8;
return false; return false;
} }
} }
@@ -439,10 +452,11 @@ class lexer : public lexer_base<BasicJsonType>
template<bool Keep> template<bool Keep>
char_int_type get_after_string_run(std::true_type /*bulk*/, std::integral_constant<bool, Keep> keep) char_int_type get_after_string_run(std::true_type /*bulk*/, std::integral_constant<bool, Keep> keep)
{ {
// scan_string() is entered from scan_after_whitespace(), whose current // a pending unget must be consumed through the normal path first
// was read by skip_whitespace() (its first get() clears a pending if (next_unget)
// unget), and nothing in scan_string() calls unget() {
JSON_ASSERT(!next_unget); return get();
}
const std::size_t remaining = ia.bulk_remaining(); const std::size_t remaining = ia.bulk_remaining();
if (remaining == 0) if (remaining == 0)
{ {
@@ -542,6 +556,7 @@ class lexer : public lexer_base<BasicJsonType>
case char_traits<char_type>::eof(): case char_traits<char_type>::eof():
{ {
error_message = "invalid string: missing closing quote"; error_message = "invalid string: missing closing quote";
error_cause = error_kind::string;
return token_type::parse_error; return token_type::parse_error;
} }
@@ -598,6 +613,7 @@ class lexer : public lexer_base<BasicJsonType>
if (JSON_HEDLEY_UNLIKELY(codepoint1 == -1)) if (JSON_HEDLEY_UNLIKELY(codepoint1 == -1))
{ {
error_message = "invalid string: '\\u' must be followed by 4 hex digits"; error_message = "invalid string: '\\u' must be followed by 4 hex digits";
error_cause = error_kind::string_invalid_escape;
return token_type::parse_error; return token_type::parse_error;
} }
@@ -605,13 +621,23 @@ class lexer : public lexer_base<BasicJsonType>
if (0xD800 <= codepoint1 && codepoint1 <= 0xDBFF) if (0xD800 <= codepoint1 && codepoint1 <= 0xDBFF)
{ {
// expect next \uxxxx entry // expect next \uxxxx entry
if (JSON_HEDLEY_LIKELY(get() == '\\' && get() == 'u')) if (JSON_HEDLEY_LIKELY(get() == '\\'))
{ {
if (JSON_HEDLEY_UNLIKELY(get() != 'u'))
{
// current is the character escaped by the backslash
error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF";
error_cause = error_kind::string_invalid_escape;
string_error_resume = resume_kind::escaped_character;
return token_type::parse_error;
}
const int codepoint2 = get_codepoint(); const int codepoint2 = get_codepoint();
if (JSON_HEDLEY_UNLIKELY(codepoint2 == -1)) if (JSON_HEDLEY_UNLIKELY(codepoint2 == -1))
{ {
error_message = "invalid string: '\\u' must be followed by 4 hex digits"; error_message = "invalid string: '\\u' must be followed by 4 hex digits";
error_cause = error_kind::string_invalid_escape;
return token_type::parse_error; return token_type::parse_error;
} }
@@ -619,25 +645,23 @@ class lexer : public lexer_base<BasicJsonType>
if (JSON_HEDLEY_LIKELY(0xDC00 <= codepoint2 && codepoint2 <= 0xDFFF)) if (JSON_HEDLEY_LIKELY(0xDC00 <= codepoint2 && codepoint2 <= 0xDFFF))
{ {
// overwrite codepoint // overwrite codepoint
codepoint = static_cast<int>( codepoint = combine_surrogates(codepoint1, codepoint2);
// high surrogate occupies the most significant 22 bits
(static_cast<unsigned int>(codepoint1) << 10u)
// low surrogate occupies the least significant 15 bits
+ static_cast<unsigned int>(codepoint2)
// there is still the 0xD800, 0xDC00, and 0x10000 noise
// in the result, so we have to subtract with:
// (0xD800 << 10) + DC00 - 0x10000 = 0x35FDC00
- 0x35FDC00u);
} }
else else
{ {
// the second escape was read completely and is a
// code point of its own
error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF"; error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF";
error_cause = error_kind::string_invalid_escape;
string_error_resume = resume_kind::after_escape;
string_error_codepoint = codepoint2;
return token_type::parse_error; return token_type::parse_error;
} }
} }
else else
{ {
error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF"; error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF";
error_cause = error_kind::string_invalid_escape;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -645,7 +669,10 @@ class lexer : public lexer_base<BasicJsonType>
{ {
if (JSON_HEDLEY_UNLIKELY(0xDC00 <= codepoint1 && codepoint1 <= 0xDFFF)) if (JSON_HEDLEY_UNLIKELY(0xDC00 <= codepoint1 && codepoint1 <= 0xDFFF))
{ {
// the escape was read completely
error_message = "invalid string: surrogate U+DC00..U+DFFF must follow U+D800..U+DBFF"; error_message = "invalid string: surrogate U+DC00..U+DFFF must follow U+D800..U+DBFF";
error_cause = error_kind::string_invalid_escape;
string_error_resume = resume_kind::after_escape;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -662,6 +689,7 @@ class lexer : public lexer_base<BasicJsonType>
// other characters after escape // other characters after escape
default: default:
error_message = "invalid string: forbidden character after backslash"; error_message = "invalid string: forbidden character after backslash";
error_cause = error_kind::string;
return token_type::parse_error; return token_type::parse_error;
} }
@@ -738,6 +766,7 @@ class lexer : public lexer_base<BasicJsonType>
} }
}; };
error_message = control_character_messages[static_cast<std::size_t>(current)]; error_message = control_character_messages[static_cast<std::size_t>(current)];
error_cause = error_kind::string;
return token_type::parse_error; return token_type::parse_error;
} }
@@ -960,6 +989,7 @@ class lexer : public lexer_base<BasicJsonType>
default: default:
{ {
error_message = "invalid string: ill-formed UTF-8 byte"; error_message = "invalid string: ill-formed UTF-8 byte";
error_cause = error_kind::string_ill_formed_utf8;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -1015,6 +1045,7 @@ class lexer : public lexer_base<BasicJsonType>
case char_traits<char_type>::eof(): case char_traits<char_type>::eof():
{ {
error_message = "invalid comment; missing closing '*/'"; error_message = "invalid comment; missing closing '*/'";
error_cause = error_kind::unclosed_comment;
return false; return false;
} }
@@ -1045,6 +1076,7 @@ class lexer : public lexer_base<BasicJsonType>
default: default:
{ {
error_message = "invalid comment; expecting '/' or '*' after '/'"; error_message = "invalid comment; expecting '/' or '*' after '/'";
error_cause = error_kind::other;
return false; return false;
} }
} }
@@ -1170,6 +1202,7 @@ scan_number_minus:
default: default:
{ {
error_message = "invalid number; expected digit after '-'"; error_message = "invalid number; expected digit after '-'";
error_cause = error_kind::number;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -1256,6 +1289,7 @@ scan_number_decimal1:
default: default:
{ {
error_message = "invalid number; expected digit after '.'"; error_message = "invalid number; expected digit after '.'";
error_cause = error_kind::number;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -1324,6 +1358,7 @@ scan_number_exponent:
{ {
error_message = error_message =
"invalid number; expected '+', '-', or digit after exponent"; "invalid number; expected '+', '-', or digit after exponent";
error_cause = error_kind::number;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -1350,6 +1385,7 @@ scan_number_sign:
default: default:
{ {
error_message = "invalid number; expected digit after exponent sign"; error_message = "invalid number; expected digit after exponent sign";
error_cause = error_kind::number;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -1411,7 +1447,7 @@ scan_number_done:
return token_type::value_unsigned; return token_type::value_unsigned;
} }
} }
else // value_integer (the callers handle value_float themselves) else if (number_type == token_type::value_integer)
{ {
if (parse_integer_signed(first, last, value_integer)) if (parse_integer_signed(first, last, value_integer))
{ {
@@ -1517,9 +1553,11 @@ scan_number_done:
*/ */
token_type scan_number_bulk_contiguous() token_type scan_number_bulk_contiguous()
{ {
// called right after scan_after_whitespace() read current (see // a pending unget offsets the buffer position from current; fall back
// get_after_string_run()), so no unget is pending if (next_unget)
JSON_ASSERT(!next_unget); {
return token_type::uninitialized;
}
const std::size_t rem = ia.bulk_remaining(); const std::size_t rem = ia.bulk_remaining();
if (rem == 0) if (rem == 0)
{ {
@@ -1540,7 +1578,11 @@ scan_number_done:
if (data[0] == '-') if (data[0] == '-')
{ {
number_type = token_type::value_integer; number_type = token_type::value_integer;
i = 1; // avail >= 2: rem == 0 returned above i = 1;
if (i >= avail)
{
return token_type::uninitialized;
}
} }
const std::size_t integer_begin = i; const std::size_t integer_begin = i;
if (data[i] == '0') if (data[i] == '0')
@@ -1771,6 +1813,7 @@ scan_number_done:
if (JSON_HEDLEY_UNLIKELY(char_traits<char_type>::to_char_type(get()) != static_cast<char_type>(rest[i]))) if (JSON_HEDLEY_UNLIKELY(char_traits<char_type>::to_char_type(get()) != static_cast<char_type>(rest[i])))
{ {
error_message = "invalid literal"; error_message = "invalid literal";
error_cause = error_kind::other;
return token_type::parse_error; return token_type::parse_error;
} }
} }
@@ -1782,10 +1825,7 @@ scan_number_done:
/// the error at the right character /// the error at the right character
JSON_HEDLEY_ALWAYS_INLINE bool scan_literal_bulk(const char* rest, const std::size_t length, std::true_type /*bulk*/) JSON_HEDLEY_ALWAYS_INLINE bool scan_literal_bulk(const char* rest, const std::size_t length, std::true_type /*bulk*/)
{ {
// called right after scan_after_whitespace() read current (see if (next_unget || ia.bulk_remaining() < length
// get_after_string_run()), so no unget is pending
JSON_ASSERT(!next_unget);
if (ia.bulk_remaining() < length
|| std::memcmp(ia.bulk_data(), rest, length) != 0) || std::memcmp(ia.bulk_data(), rest, length) != 0)
{ {
return false; return false;
@@ -2192,17 +2232,25 @@ scan_number_done:
return position.chars_read_total; return position.chars_read_total;
} }
/// seekable adapter: count the lines in the consumed input; O(input read /// seekable adapter: count the lines in the consumed input. Only the
/// so far), but only called on errors /// input consumed since the previous call is scanned, so a parse that
/// reports many errors (a SAX parser recovering from them) stays linear
/// in the input instead of rescanning it for every error.
position_t get_position_impl(std::true_type /*lazy*/) const position_t get_position_impl(std::true_type /*lazy*/) const
{ {
position_t result; position_t result;
result.chars_read_total = get_chars_read_total(); result.chars_read_total = get_chars_read_total();
// reads at EOF count as characters but are not part of the input // reads at EOF count as characters but are not part of the input
const std::size_t consumed = (std::min)(result.chars_read_total, ia.get_consumed_count()); const std::size_t consumed = (std::min)(result.chars_read_total, ia.get_consumed_count());
std::size_t line_start = 0; if (JSON_HEDLEY_UNLIKELY(consumed < position_cache.consumed))
result.lines_read = ia.count_consumed_lines(consumed, line_start); {
result.chars_read_current_line = result.chars_read_total - line_start; // the consumed input never shrinks; recount from the start if it did
position_cache = {};
}
position_cache.lines += ia.count_consumed_lines(position_cache.consumed, consumed, position_cache.line_start);
position_cache.consumed = consumed;
result.lines_read = position_cache.lines;
result.chars_read_current_line = result.chars_read_total - position_cache.line_start;
return result; return result;
} }
@@ -2386,6 +2434,7 @@ scan_number_done:
if (get_chars_read_total() == 0 && !skip_bom()) if (get_chars_read_total() == 0 && !skip_bom())
{ {
error_message = "invalid BOM; must be 0xEF 0xBB 0xBF if given"; error_message = "invalid BOM; must be 0xEF 0xBB 0xBF if given";
error_cause = error_kind::other;
return token_type::parse_error; return token_type::parse_error;
} }
@@ -2495,10 +2544,583 @@ scan_number_done:
// error // error
default: default:
error_message = "invalid literal"; error_message = "invalid literal";
error_cause = error_kind::other;
return token_type::parse_error; return token_type::parse_error;
} }
} }
public:
/////////////////////
// error recovery
/////////////////////
/*!
@brief make the best of the token that scan() rejected
Called by the parser after scan() returned token_type::parse_error and the
SAX parser asked to recover from the error (see #3989). Keeps what can be
read of the token and skips the rest:
- A string keeps its characters. An unknown escape stands for the escaped
character itself (as in JavaScript), an invalid Unicode escape and ill-formed
UTF-8 become U+FFFD, and a control character is kept. A line break or the
end of the input ends a string that lacks its closing quote.
- A number keeps its longest valid prefix, e.g. `1` for `1.` or `1e+`.
- A block comment that is not closed runs to the end of the input.
- Anything else is skipped.
The rest of an invalid token is skipped up to the next delimiter
(whitespace, a structural character, or a quote). A delimiter that the
invalid token consumed is returned to the input, so that the next scan()
reads it.
@return token_type::value_string or a number token type if a string or a
number could be read, token_type::end_of_input for a block comment
that is not closed, token_type::uninitialized otherwise
*/
token_type recover_token()
{
const resume_kind resume = string_error_resume;
const int codepoint = string_error_codepoint;
string_error_resume = resume_kind::character;
string_error_codepoint = -1;
const error_kind cause = error_cause;
error_cause = error_kind::other;
switch (cause)
{
case error_kind::string:
case error_kind::string_invalid_escape:
case error_kind::string_ill_formed_utf8:
return recover_string(cause, resume, codepoint);
case error_kind::number:
return recover_number();
case error_kind::unclosed_comment:
// the comment runs to the end of the input
return token_type::end_of_input;
case error_kind::other:
default:
skip_to_delimiter();
return token_type::uninitialized;
}
}
/*!
@brief return the token that scan() read last to the input, so that the
next scan() reads it again
Called by the parser when recovering from an error. The token must be a
single character (',', ':', '[', ']', '{', or '}') or the end of the
input, and scan() must have read it last.
*/
void unget_token()
{
JSON_ASSERT(!next_unget);
unget();
}
/*!
@brief let the token string for the next error begin at the current character
The token string of an error reaches back to the beginning of the last
string or number. After an error, the parser calls this function so that
the next error does not report (and, with many errors, copy) everything
read since then.
*/
void restart_token_string()
{
restart_token_string_impl(std::integral_constant<bool, lazy_token_string> {});
}
private:
/// how recover_string() continues after the error scan_string() reported
enum class resume_kind : std::uint8_t
{
/// current is the next character of the string (or the end of input)
character,
/// current is the character escaped by the preceding backslash
escaped_character,
/// current is the last character of a complete escape
after_escape
};
/// what kind of token scan() rejected, which decides how recover_token() repairs it
enum class error_kind : std::uint8_t
{
/// anything not listed below; skipped up to the next delimiter
other,
/// a string
string,
/// a string with an invalid Unicode escape, which becomes U+FFFD
string_invalid_escape,
/// a string with ill-formed UTF-8, which becomes U+FFFD
string_ill_formed_utf8,
/// a number
number,
/// a block comment that is not closed
unclosed_comment
};
/// whether current ends an invalid token (see recover_token())
bool current_is_delimiter() const noexcept
{
switch (current)
{
case ' ':
case '\t':
case '\n':
case '\r':
case '[':
case ']':
case '{':
case '}':
case ',':
case ':':
case '\"':
#if !JSON_STRICT_NUL_HANDLING
case '\0':
#endif
case char_traits<char_type>::eof():
return true;
case '/':
return ignore_comments;
default:
return false;
}
}
/// skip the rest of an invalid token and return its delimiter to the input
void skip_to_delimiter()
{
while (!current_is_delimiter())
{
get();
}
if (current != char_traits<char_type>::eof())
{
unget();
}
}
/// append U+FFFD REPLACEMENT CHARACTER to token_buffer
void add_replacement_character()
{
add_codepoint(0xFFFD);
}
/// append the UTF-8 encoding of @a codepoint (not a surrogate) to token_buffer
void add_codepoint(const int codepoint)
{
JSON_ASSERT(0x00 <= codepoint && codepoint <= 0x10FFFF);
encode_utf8(static_cast<std::uint32_t>(codepoint), [this](std::uint32_t byte)
{
add(static_cast<char_int_type>(byte));
});
}
/// append a code point read from a Unicode escape; a surrogate becomes U+FFFD
void add_escaped_codepoint(const int codepoint)
{
if (0xD800 <= codepoint && codepoint <= 0xDFFF)
{
add_replacement_character();
}
else
{
add_codepoint(codepoint);
}
}
/*!
@brief remove an incomplete UTF-8 sequence from the end of token_buffer
next_byte_in_range() adds the bytes of a sequence as it checks them, so
when it rejects a byte, the beginning of the sequence is already in
token_buffer, which otherwise holds only complete sequences.
@return whether an incomplete sequence was removed
*/
bool remove_incomplete_utf8_sequence()
{
std::size_t lead = token_buffer.size();
std::size_t continuation_bytes = 0;
while (lead > 0 && continuation_bytes < 3
&& (static_cast<unsigned char>(token_buffer[lead - 1]) & 0xC0u) == 0x80u)
{
--lead;
++continuation_bytes;
}
if (lead == 0)
{
return false;
}
const auto lead_byte = static_cast<unsigned char>(token_buffer[lead - 1]);
std::size_t expected = 0;
if (lead_byte >= 0xF0)
{
expected = 3;
}
else if (lead_byte >= 0xE0)
{
expected = 2;
}
else if (lead_byte >= 0xC0)
{
expected = 1;
}
if (continuation_bytes >= expected)
{
return false;
}
token_buffer.resize(lead - 1);
return true;
}
/*!
@brief read the UTF-8 sequence that begins with current, which is not ASCII
@return whether the next character must be read; false if current still
needs to be handled, because it does not belong to the sequence
*/
bool recover_utf8_sequence()
{
// the number of continuation bytes and the range of the first one;
// see the ranges in scan_string()
std::size_t count = 0;
char_int_type low = 0x80;
char_int_type high = 0xBF;
if (current >= 0xC2 && current <= 0xDF)
{
count = 1;
}
else if (current >= 0xE0 && current <= 0xEF)
{
count = 2;
low = (current == 0xE0) ? 0xA0 : 0x80;
high = (current == 0xED) ? 0x9F : 0xBF;
}
else if (current >= 0xF0 && current <= 0xF4)
{
count = 3;
low = (current == 0xF0) ? 0x90 : 0x80;
high = (current == 0xF4) ? 0x8F : 0xBF;
}
else
{
// an ill-formed byte
add_replacement_character();
return true;
}
const std::size_t start = token_buffer.size();
add(current);
for (std::size_t i = 0; i < count; ++i)
{
get();
if (current < low || current > high)
{
token_buffer.resize(start);
add_replacement_character();
return false;
}
add(current);
low = 0x80;
high = 0xBF;
}
return true;
}
/*!
@brief read the low surrogate that must follow the high surrogate @a high
@return whether the next character must be read; false if current still
needs to be handled
*/
bool recover_low_surrogate(int high)
{
while (true)
{
if (get() != '\\')
{
add_replacement_character();
return false;
}
if (get() != 'u')
{
add_replacement_character();
// not 'u', so this does not come back here
return recover_escape();
}
const int low = get_codepoint();
if (low == -1)
{
add_replacement_character();
return false;
}
if (0xDC00 <= low && low <= 0xDFFF)
{
add_codepoint(combine_surrogates(high, low));
return true;
}
// high has no low surrogate
add_replacement_character();
if (low < 0xD800 || low > 0xDBFF)
{
add_codepoint(low);
return true;
}
// another high surrogate
high = low;
}
}
/*!
@brief read the escape whose backslash was read; current is the escaped character
@return whether the next character must be read; false if current still
needs to be handled
*/
bool recover_escape()
{
switch (current)
{
case '\"':
add('\"');
return true;
case '\\':
add('\\');
return true;
case '/':
add('/');
return true;
case 'b':
add('\b');
return true;
case 'f':
add('\f');
return true;
case 'n':
add('\n');
return true;
case 'r':
add('\r');
return true;
case 't':
add('\t');
return true;
case 'u':
{
const int codepoint = get_codepoint();
if (codepoint == -1)
{
add_replacement_character();
return false;
}
if (0xD800 <= codepoint && codepoint <= 0xDBFF)
{
return recover_low_surrogate(codepoint);
}
add_escaped_codepoint(codepoint);
return true;
}
// an unknown escape stands for the escaped character
default:
return false;
}
}
/*!
@brief read the rest of a string after scan_string() rejected it
token_buffer holds what scan_string() read before the error. See
recover_token() for how errors are repaired.
@param[in] cause the kind of error scan_string() reported
@param[in] resume how to continue, see resume_kind
@param[in] codepoint for a high surrogate followed by an escape of another
code point: that code point; -1 otherwise
*/
token_type recover_string(const error_kind cause, const resume_kind resume, const int codepoint)
{
// whether the next character must be read before it can be handled
bool fetch = false;
if (cause == error_kind::string_invalid_escape
|| (cause == error_kind::string_ill_formed_utf8 && remove_incomplete_utf8_sequence()))
{
add_replacement_character();
}
switch (resume)
{
case resume_kind::escaped_character:
fetch = recover_escape();
break;
case resume_kind::after_escape:
if (0xD800 <= codepoint && codepoint <= 0xDBFF)
{
fetch = recover_low_surrogate(codepoint);
}
else
{
if (codepoint != -1)
{
add_escaped_codepoint(codepoint);
}
fetch = true;
}
break;
case resume_kind::character:
default:
break;
}
while (true)
{
if (fetch)
{
get();
}
fetch = true;
switch (current)
{
case '\"':
// a line break or the end of the input ends a string that
// lacks its closing quote
case '\n':
case '\r':
case char_traits<char_type>::eof():
return token_type::value_string;
#if !JSON_STRICT_NUL_HANDLING
case '\0':
// the end of the input, see scan()
unget();
return token_type::value_string;
#endif
case '\\':
get();
fetch = recover_escape();
break;
default:
if (current < 0x80)
{
// including control characters
add(current);
}
else
{
fetch = recover_utf8_sequence();
}
break;
}
}
}
/*!
@brief keep the longest valid prefix of a number that scan_number() rejected
token_buffer holds the characters scan_number() accepted before the error,
so the prefix ends at its last digit.
*/
token_type recover_number()
{
// only size(), operator[], and resize() are used, which every string
// type the library supports provides
std::size_t length = token_buffer.size();
while (length != 0 && (token_buffer[length - 1] < '0' || token_buffer[length - 1] > '9'))
{
--length;
}
token_buffer.resize(length);
if (length == 0)
{
skip_to_delimiter();
return token_type::uninitialized;
}
if (decimal_point_position >= length)
{
decimal_point_position = std::string::npos;
}
std::size_t exponent = std::string::npos;
for (std::size_t i = 0; i < length; ++i)
{
if (token_buffer[i] == 'e' || token_buffer[i] == 'E')
{
exponent = i;
break;
}
}
const std::size_t mantissa_end = (exponent == std::string::npos) ? length : exponent;
token_type number_type = token_type::value_unsigned;
if (decimal_point_position != std::string::npos || exponent != std::string::npos)
{
number_type = token_type::value_float;
}
else if (token_buffer[0] == '-')
{
number_type = token_type::value_integer;
}
const token_type result = convert_number(number_type, mantissa_end);
skip_to_delimiter();
return result;
}
/// seekable or token-tracking adapter: the token string begins at current
void restart_token_string_impl(std::true_type /*lazy*/)
{
restart_token_string_lazy(std::integral_constant<bool, adapter_token> {});
}
/// seekable adapter: the token string begins at current, which was consumed
void restart_token_string_lazy(std::false_type /*adapter_token*/) noexcept
{
const std::size_t consumed = ia.get_consumed_count();
token_string_start = (consumed > 0 && current != char_traits<char_type>::eof()) ? consumed - 1 : consumed;
}
/// token-tracking adapter: the token begins with current, or with the next
/// character if current was put back or is the end of input
void restart_token_string_lazy(std::true_type /*adapter_token*/)
{
if (!next_unget && current != char_traits<char_type>::eof())
{
ia.mark_token_start(current);
}
else
{
ia.begin_token();
}
}
/// streaming adapter: the token string begins at current; a character
/// that was put back is copied again when it is read again
void restart_token_string_impl(std::false_type /*lazy*/)
{
token_string.clear();
if (!next_unget && current != char_traits<char_type>::eof())
{
token_string.push_back(char_traits<char_type>::to_char_type(current));
}
}
/// input adapter /// input adapter
InputAdapterType ia; InputAdapterType ia;
@@ -2523,6 +3145,19 @@ scan_number_done:
/// that unget() can restore the column instead of leaving it at 0 /// that unget() can restore the column instead of leaving it at 0
std::size_t chars_read_before_newline = 0; std::size_t chars_read_before_newline = 0;
/// seekable adapter: what get_position() counted last, so that the next
/// call only scans the input consumed since
struct position_cache_t
{
/// consumed characters scanned so far
std::size_t consumed = 0;
/// newlines among them
std::size_t lines = 0;
/// index after the last of these newlines
std::size_t line_start = 0;
};
mutable position_cache_t position_cache {};
/// raw input token string for error messages; only populated for streaming /// raw input token string for error messages; only populated for streaming
/// adapters (seekable adapters reconstruct it lazily via token_string_start) /// adapters (seekable adapters reconstruct it lazily via token_string_start)
std::vector<char_type> token_string {}; std::vector<char_type> token_string {};
@@ -2543,6 +3178,15 @@ scan_number_done:
/// a description of occurred lexer errors /// a description of occurred lexer errors
const char* error_message = ""; const char* error_message = "";
/// what kind of token scan() rejected; set together with error_message
error_kind error_cause = error_kind::other;
/// how recover_token() continues a string that scan_string() rejected;
/// set only on the error paths that need more than error_message
resume_kind string_error_resume = resume_kind::character;
/// the code point of the second escape when a high surrogate is followed
/// by an escape that is not a low surrogate; -1 otherwise
int string_error_codepoint = -1;
// number values // number values
number_integer_t value_integer = 0; number_integer_t value_integer = 0;
number_unsigned_t value_unsigned = 0; number_unsigned_t value_unsigned = 0;
+753 -32
View File
@@ -152,25 +152,72 @@ class parser
bool accept(const bool strict = true) bool accept(const bool strict = true)
{ {
json_sax_acceptor<BasicJsonType> sax_acceptor; json_sax_acceptor<BasicJsonType> sax_acceptor;
return sax_parse(&sax_acceptor, strict); return sax_parse_impl<false>(&sax_acceptor, strict);
} }
/*!
@brief public SAX interface
If the SAX parser's parse_error() returns true, the parser recovers from
the error: it repairs the input and continues (see #3989).
@param[in] sax the SAX parser
@param[in] strict whether to expect the last token to be EOF
@return whether the input was parsed without errors and no SAX event
returned false
*/
template<typename SAX> template<typename SAX>
JSON_HEDLEY_NON_NULL(2) JSON_HEDLEY_NON_NULL(2)
bool sax_parse(SAX* sax, const bool strict = true) bool sax_parse(SAX* sax, const bool strict = true)
{ {
(void)detail::is_sax_static_asserts<SAX, BasicJsonType> {}; return sax_parse_impl<true>(sax, strict);
const bool result = sax_parse_internal(sax);
if (result)
{
return finish_input(*sax, strict);
}
return false;
} }
private: private:
/// what sax_parse_internal() does after an object key was expected
enum class next_step : std::uint8_t
{
/// stop parsing
stop,
/// parse a value that begins with last_token
parse_value,
/// evaluate the state of the innermost container, which reads
/// last_token again
evaluate_state
};
template<bool AllowRecovery, typename SAX>
JSON_HEDLEY_NON_NULL(2)
bool sax_parse_impl(SAX* sax, const bool strict)
{
(void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
const bool result = sax_parse_internal<AllowRecovery>(sax);
if (result)
{
if (strict)
{
// strict mode: next byte must be EOF; after recovering from an
// error, the end of the input may already have been read
if (last_token != token_type::end_of_input && get_token() != token_type::end_of_input)
{
// the value is complete, so there is nothing to recover
static_cast<void>(report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr),
std::integral_constant<bool, AllowRecovery> {}));
return false;
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
}
return result && !error_reported;
}
/// the text of the current float token for @a sax (see sax_ignores_number_text) /// the text of the current float token for @a sax (see sax_ignores_number_text)
template<typename SAX> template<typename SAX>
const string_t& number_float_text(SAX* /*sax*/) const string_t& number_float_text(SAX* /*sax*/)
@@ -238,17 +285,26 @@ class parser
template<typename DomSax> template<typename DomSax>
bool parse_dom(DomSax& sdp, const bool strict) bool parse_dom(DomSax& sdp, const bool strict)
{ {
sax_parse_internal(&sdp); sax_parse_internal<false>(&sdp);
static_cast<void>(finish_input(sdp, strict)); static_cast<void>(finish_input(sdp, strict));
return !sdp.is_errored(); return !sdp.is_errored();
} }
template<typename SAX> /*!
@brief parse a JSON value and pass it to a SAX parser
@tparam AllowRecovery whether to recover from an error if the SAX parser's
parse_error() returns true; false for the SAX parsers
of parse() and accept(), which never do, so that no
code for recovering is generated for them
*/
template<bool AllowRecovery, typename SAX>
JSON_HEDLEY_NON_NULL(2) JSON_HEDLEY_NON_NULL(2)
bool sax_parse_internal(SAX* sax) bool sax_parse_internal(SAX* sax)
{ {
const std::integral_constant<bool, AllowRecovery> allow_recovery{};
// value to avoid a goto (see comment where set to true) // value to avoid a goto (see comment where set to true)
bool skip_to_state_evaluation = false; bool skip_to_state_evaluation = false;
@@ -287,10 +343,18 @@ class parser
break; break;
} }
// parse key // remember we are now inside an object
states.push_back(0);
// parse key (the steps of parse_key(), which are
// repeated here and below for speed)
if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string)) if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
{ {
return syntax_error(*sax, exception_message(token_type::value_string, "object key")); if (!continue_after(key_error(sax, allow_recovery, false), skip_to_state_evaluation))
{
return false;
}
continue;
} }
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token()))) if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{ {
@@ -300,12 +364,13 @@ class parser
// parse separator (:) // parse separator (:)
if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator))) if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator)))
{ {
return syntax_error(*sax, exception_message(token_type::name_separator, "object separator")); if (!continue_after(key_error(sax, allow_recovery, true), skip_to_state_evaluation))
{
return false;
}
continue;
} }
// remember we are now inside an object
states.push_back(0);
// parse values // parse values
get_token(); get_token();
continue; continue;
@@ -341,9 +406,11 @@ class parser
if (JSON_HEDLEY_UNLIKELY(!std::isfinite(res))) if (JSON_HEDLEY_UNLIKELY(!std::isfinite(res)))
{ {
return sax->parse_error(m_lexer.get_position(), if (!overflow_error(sax, res, allow_recovery))
m_lexer.get_token_string(), {
out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr)); return false;
}
break;
} }
if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, number_float_text(sax)))) if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, number_float_text(sax))))
@@ -411,16 +478,63 @@ class parser
case token_type::parse_error: case token_type::parse_error:
{ {
// using "uninitialized" to avoid an "expected" message // using "uninitialized" to avoid an "expected" message
return syntax_error(*sax, exception_message(token_type::uninitialized, "value")); if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::uninitialized, "value"), nullptr), allow_recovery))
{
return false;
}
// recover: keep what can be read of the token
recover_token(allow_recovery);
if (last_token != token_type::uninitialized)
{
// a string or a number
continue;
}
if (states.empty())
{
// look for the value after the garbage
if (!skip_to_value(allow_recovery))
{
return false;
}
continue;
}
// nothing could be read
if (JSON_HEDLEY_UNLIKELY(!sax->null()))
{
return false;
}
break;
} }
case token_type::end_of_input: case token_type::end_of_input:
{ {
if (JSON_HEDLEY_UNLIKELY(m_lexer.get_chars_read_total() == 1)) if (JSON_HEDLEY_UNLIKELY(m_lexer.get_chars_read_total() == 1))
{ {
return syntax_error(*sax, empty_input_message()); // there is nothing to recover
static_cast<void>(report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(),
empty_input_message(), nullptr), allow_recovery));
return false;
} }
return syntax_error(*sax, exception_message(token_type::literal_or_value, "value")); if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::literal_or_value, "value"), nullptr), allow_recovery))
{
return false;
}
// recover: the input ends where a value is missing
if (states.empty())
{
// there is no value
return false;
}
if (!recover_missing_value(sax, states, allow_recovery))
{
return false;
}
// the state evaluation reads the token again
unget_token(allow_recovery);
skip_to_state_evaluation = true;
continue;
} }
case token_type::uninitialized: case token_type::uninitialized:
case token_type::end_array: case token_type::end_array:
@@ -430,7 +544,35 @@ class parser
case token_type::literal_or_value: case token_type::literal_or_value:
default: // the last token was unexpected default: // the last token was unexpected
{ {
return syntax_error(*sax, exception_message(token_type::literal_or_value, "value")); if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::literal_or_value, "value"), nullptr), allow_recovery))
{
return false;
}
// recover
if (states.empty())
{
// look for the value after the garbage
if (!skip_to_value(allow_recovery))
{
return false;
}
continue;
}
if (last_token == token_type::name_separator)
{
// a stray ':'; the value may follow
get_token();
continue;
}
if (!recover_missing_value(sax, states, allow_recovery))
{
return false;
}
// the state evaluation reads the token again
unget_token(allow_recovery);
skip_to_state_evaluation = true;
continue;
} }
} }
} }
@@ -481,7 +623,30 @@ class parser
continue; continue;
} }
return syntax_error(*sax, exception_message(token_type::end_array, "array")); if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::end_array, "array"), nullptr), allow_recovery))
{
return false;
}
// recover
if (last_token == token_type::end_of_input)
{
// the input ends inside the array
return close_containers(sax, states, allow_recovery);
}
if (last_token == token_type::end_object)
{
// a wrong closing bracket closes the innermost container
if (JSON_HEDLEY_UNLIKELY(!sax->end_array()))
{
return false;
}
states.pop_back();
skip_to_state_evaluation = true;
}
// otherwise, a missing ',' (or a stray ':', which value
// parsing drops): the next value begins here
continue;
} }
// states.back() is false -> object // states.back() is false -> object
@@ -498,9 +663,12 @@ class parser
// parse key // parse key
if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string)) if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
{ {
return syntax_error(*sax, exception_message(token_type::value_string, "object key")); if (!continue_after(key_error(sax, allow_recovery, false), skip_to_state_evaluation))
{
return false;
}
continue;
} }
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token()))) if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{ {
return false; return false;
@@ -509,7 +677,11 @@ class parser
// parse separator (:) // parse separator (:)
if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator))) if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator)))
{ {
return syntax_error(*sax, exception_message(token_type::name_separator, "object separator")); if (!continue_after(key_error(sax, allow_recovery, true), skip_to_state_evaluation))
{
return false;
}
continue;
} }
// parse values // parse values
@@ -537,10 +709,546 @@ class parser
continue; continue;
} }
return syntax_error(*sax, exception_message(token_type::end_object, "object")); if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::end_object, "object"), nullptr), allow_recovery))
{
return false;
}
// recover
if (last_token == token_type::end_of_input)
{
// the input ends inside the object
return close_containers(sax, states, allow_recovery);
}
if (last_token == token_type::end_array)
{
// a wrong closing bracket closes the innermost container
if (JSON_HEDLEY_UNLIKELY(!sax->end_object()))
{
return false;
}
states.pop_back();
skip_to_state_evaluation = true;
continue;
}
if (!continue_after(recover_member(sax, allow_recovery), skip_to_state_evaluation))
{
return false;
}
} }
} }
/*!
@brief continue sax_parse_internal() after a recovery
@return whether to continue parsing
*/
bool continue_after(const next_step step, bool& skip_to_state_evaluation)
{
if (step == next_step::evaluate_state)
{
// the state evaluation reads the token again
m_lexer.unget_token();
skip_to_state_evaluation = true;
}
return step != next_step::stop;
}
/// the parser for parse() and accept() never recovers: stop parsing
JSON_INTERNAL_ALWAYS_INLINE
static std::false_type continue_after(std::false_type /*step*/, bool& /*skip_to_state_evaluation*/) noexcept
{
return {};
}
/*!
@brief parse an object key and the name separator (:) after it
last_token is the token where the key is expected. sax_parse_internal()
repeats these steps rather than calling this function, which is used
when recovering from an error.
@return next_step::parse_value if the value follows, with last_token its
first token; next_step::evaluate_state if the object's state is
to be evaluated after recovering from an error; next_step::stop
to stop parsing
*/
template<typename SAX>
next_step parse_key(SAX* sax)
{
const std::true_type allow_recovery{};
if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
{
return key_error(sax, allow_recovery, false);
}
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{
return next_step::stop;
}
// parse separator (:)
if (JSON_HEDLEY_UNLIKELY(get_token() != token_type::name_separator))
{
return key_error(sax, allow_recovery, true);
}
// the value begins with the next token
get_token();
return next_step::parse_value;
}
/*!
@brief report a number that is too large for number_float_t, and recover
from the error by passing the value on; the SAX parser gets the
number's text as well
This is a separate function, as reading other numbers is measurably
slower if the error is handled where they are read.
@param[in] sax the SAX parser
@param[in] value the value that is not finite
@return whether to continue parsing
*/
template<typename SAX>
bool overflow_error(SAX* sax, const number_float_t value, std::true_type allow_recovery)
{
if (!report_error(sax, out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr), allow_recovery))
{
return false;
}
return sax->number_float(value, number_float_text(sax));
}
/// @copydoc overflow_error
template<typename SAX>
JSON_INTERNAL_ALWAYS_INLINE
std::false_type overflow_error(SAX* sax, const number_float_t /*value*/, std::false_type allow_recovery)
{
return report_error(sax, out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr), allow_recovery);
}
/*!
@brief report a missing key, or a missing name separator (:) after the
key; the parser for parse() and accept() never recovers
@param[in] key_read whether the key was read, so that the name separator
is missing
@return std::false_type, see report_error()
*/
template<typename SAX>
JSON_INTERNAL_ALWAYS_INLINE
std::false_type key_error(SAX* sax, std::false_type allow_recovery, const bool key_read)
{
return report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), key_read
? exception_message(token_type::name_separator, "object separator")
: exception_message(token_type::value_string, "object key"), nullptr), allow_recovery);
}
/*!
@brief report a missing key, or a missing name separator (:) after the
key, and recover from it
@param[in] key_read whether the key was read, so that the name separator
is missing
*/
template<typename SAX>
next_step key_error(SAX* sax, std::true_type allow_recovery, const bool key_read)
{
if (!key_read)
{
if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::value_string, "object key"), nullptr), allow_recovery))
{
return next_step::stop;
}
return recover_key(sax);
}
if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::name_separator, "object separator"), nullptr), allow_recovery))
{
return next_step::stop;
}
return recover_name_separator(sax);
}
/////////////////////
// error recovery
/////////////////////
/*
The functions below repair an error after the SAX parser's parse_error()
returned true (see #3989). Each mistake is repaired by the smallest local
edit: a missing ',' or ':' is inserted, a stray token is removed, what can
be read of an invalid string or number is kept (see
lexer::recover_token()), a missing value becomes null, a wrong closing
bracket closes the innermost container, and the end of the input closes
all of them. The events stay balanced, and every key() is followed by
exactly one value.
A repair hands a token to the state evaluation, by returning it to the
lexer (lexer::unget_token()) so that the state evaluation reads it again,
only if it is ',', ']', '}', or the end of the input. The state evaluation
hands a token to value or key parsing only if it is none of them, so a
token is never handed back and forth. Every other step reads a token or
closes a container, so parsing always ends.
*/
/*!
@brief report an error to the SAX parser; the parser for parse() and
accept() never recovers
@return std::false_type rather than false: its value is known where the
function is called even if the call is not inlined, so the code
for recovering is not generated
*/
template<typename SAX, typename Exception>
JSON_INTERNAL_ALWAYS_INLINE
std::false_type report_error(SAX* sax, const Exception& ex, std::false_type /*allow_recovery*/)
{
static_cast<void>(sax); // MSVC 2015 does not count calling a static parse_error() as using it
error_reported = true;
static_cast<void>(sax->parse_error(m_lexer.get_position(), m_lexer.get_token_string(), ex));
return {};
}
/*!
@brief report an error to the SAX parser
@return whether to recover from the error
*/
template<typename SAX, typename Exception>
bool report_error(SAX* sax, const Exception& ex, std::true_type /*allow_recovery*/)
{
static_cast<void>(sax); // MSVC 2015 does not count calling a static parse_error() as using it
const std::size_t position = m_lexer.get_position().chars_read_total;
if (error_reported && position == last_error_position && last_token == last_error_token)
{
// a repair handed on the token of the error it repaired; the
// token was reported already, and the SAX parser asked to recover
return true;
}
error_reported = true;
last_error_position = position;
last_error_token = last_token;
if (!sax->parse_error(m_lexer.get_position(), m_lexer.get_token_string(), ex))
{
return false;
}
// the token string of the next error begins here
m_lexer.restart_token_string();
return true;
}
/*!
@brief keep what can be read of the token that the lexer rejected
The error was reported for the rejected token, so it is not reported again
for the token it is repaired to (see lexer::recover_token()).
*/
token_type recover_token()
{
last_token = m_lexer.recover_token();
last_error_position = m_lexer.get_position().chars_read_total;
last_error_token = last_token;
return last_token;
}
// The functions below are called from sax_parse_internal() after an error
// was reported. The parser for parse() and accept() stopped there, so for
// it they are stubs: the code for recovering is then not even referenced,
// and unoptimized builds do not emit it either.
/// @copydoc recover_token()
JSON_INTERNAL_ALWAYS_INLINE
token_type recover_token(std::true_type /*allow_recovery*/)
{
return recover_token();
}
/// @copydoc recover_token()
JSON_INTERNAL_ALWAYS_INLINE
static std::false_type recover_token(std::false_type /*allow_recovery*/) noexcept
{
return {};
}
/// return the token that was read last to the lexer (see lexer::unget_token())
JSON_INTERNAL_ALWAYS_INLINE
void unget_token(std::true_type /*allow_recovery*/)
{
m_lexer.unget_token();
}
/// @copydoc unget_token(std::true_type)
JSON_INTERNAL_ALWAYS_INLINE
static void unget_token(std::false_type /*allow_recovery*/) noexcept {}
/// @copydoc skip_to_value
JSON_INTERNAL_ALWAYS_INLINE
static std::false_type skip_to_value(std::false_type /*allow_recovery*/) noexcept
{
return {};
}
/// @copydoc recover_missing_value
template<typename SAX>
JSON_INTERNAL_ALWAYS_INLINE
static std::false_type recover_missing_value(SAX* /*sax*/, const std::vector<std::uint8_t>& /*states*/, std::false_type /*allow_recovery*/) noexcept
{
return {};
}
/// pass the end events of all open containers
template<typename SAX>
bool close_containers(SAX* sax, std::vector<std::uint8_t>& states, std::true_type /*allow_recovery*/)
{
while (!states.empty())
{
const bool is_array = states.back() != 0;
states.pop_back();
if (JSON_HEDLEY_UNLIKELY(is_array ? !sax->end_array() : !sax->end_object()))
{
return false;
}
}
return true;
}
/// @copydoc close_containers
template<typename SAX>
JSON_INTERNAL_ALWAYS_INLINE
static std::false_type close_containers(SAX* /*sax*/, std::vector<std::uint8_t>& /*states*/, std::false_type /*allow_recovery*/) noexcept
{
return {};
}
/*!
@brief read tokens until one begins a value, skipping everything before
the top-level value
@return whether a value begins with last_token
*/
bool skip_to_value(std::true_type /*allow_recovery*/)
{
while (true)
{
switch (get_token())
{
case token_type::begin_array:
case token_type::begin_object:
case token_type::literal_false:
case token_type::literal_null:
case token_type::literal_true:
case token_type::value_float:
case token_type::value_integer:
case token_type::value_string:
case token_type::value_unsigned:
return true;
case token_type::end_of_input:
return false;
case token_type::parse_error:
recover_token();
if (last_token != token_type::uninitialized)
{
return true;
}
break;
case token_type::uninitialized:
case token_type::end_array:
case token_type::end_object:
case token_type::name_separator:
case token_type::value_separator:
case token_type::literal_or_value:
default:
break;
}
}
}
/*!
@brief skip the rest of an object member that cannot be read
Reads tokens, beginning with last_token, until a ',', '}', or ']' that is
not inside a container that begins in the skipped tokens, or the end of
the input.
*/
void skip_member()
{
std::size_t depth = 0;
while (true)
{
switch (last_token)
{
case token_type::begin_array:
case token_type::begin_object:
++depth;
break;
case token_type::end_array:
case token_type::end_object:
if (depth == 0)
{
return;
}
--depth;
break;
case token_type::value_separator:
if (depth == 0)
{
return;
}
break;
case token_type::end_of_input:
return;
case token_type::parse_error:
recover_token();
break;
case token_type::uninitialized:
case token_type::literal_true:
case token_type::literal_false:
case token_type::literal_null:
case token_type::value_string:
case token_type::value_unsigned:
case token_type::value_integer:
case token_type::value_float:
case token_type::name_separator:
case token_type::literal_or_value:
default:
break;
}
get_token();
}
}
/*!
@brief pass a value where it is missing
last_token is ',', ']', '}', or the end of the input, where a value was
expected. In an object, the key gets null; in an array, a ',' where a
value is missing stands for null (as in JavaScript), while an array that
ends there just ends.
*/
template<typename SAX>
bool recover_missing_value(SAX* sax, const std::vector<std::uint8_t>& states, std::true_type /*allow_recovery*/)
{
JSON_ASSERT(!states.empty());
if (states.back() == 0 || last_token == token_type::value_separator)
{
return sax->null();
}
return true;
}
/// recover from a missing key; last_token is where it was expected
template<typename SAX>
next_step recover_key(SAX* sax)
{
switch (last_token)
{
case token_type::value_separator:
case token_type::end_object:
case token_type::end_array:
case token_type::end_of_input:
// no member: the object's state handles the token
return next_step::evaluate_state;
case token_type::parse_error:
recover_token();
if (last_token == token_type::value_string)
{
// a key that could be repaired
return parse_key(sax);
}
skip_member();
return next_step::evaluate_state;
case token_type::uninitialized:
case token_type::literal_true:
case token_type::literal_false:
case token_type::literal_null:
case token_type::value_string:
case token_type::value_unsigned:
case token_type::value_integer:
case token_type::value_float:
case token_type::begin_array:
case token_type::begin_object:
case token_type::name_separator:
case token_type::literal_or_value:
default:
// a member without a key
skip_member();
return next_step::evaluate_state;
}
}
/// recover from a missing name separator (:) after the key; last_token
/// is where it was expected
template<typename SAX>
next_step recover_name_separator(SAX* sax)
{
switch (last_token)
{
case token_type::value_separator:
case token_type::end_object:
case token_type::end_array:
case token_type::end_of_input:
// the value is missing as well
return sax->null() ? next_step::evaluate_state : next_step::stop;
case token_type::uninitialized:
case token_type::literal_true:
case token_type::literal_false:
case token_type::literal_null:
case token_type::value_string:
case token_type::value_unsigned:
case token_type::value_integer:
case token_type::value_float:
case token_type::begin_array:
case token_type::begin_object:
case token_type::name_separator:
case token_type::parse_error:
case token_type::literal_or_value:
default:
// a missing ':'; the value begins here
return next_step::parse_value;
}
}
/// recover from a token after an object member that is neither ',' nor
/// '}' (nor ']' or the end of the input, which the caller handles)
template<typename SAX>
next_step recover_member(SAX* sax, std::true_type /*allow_recovery*/)
{
if (last_token == token_type::parse_error)
{
recover_token();
}
if (last_token == token_type::value_string)
{
// a missing ','; the next key begins here
return parse_key(sax);
}
skip_member();
return next_step::evaluate_state;
}
/// the parser for parse() and accept() never recovers (and does not come
/// here, as report_error() returned false)
template<typename SAX>
JSON_INTERNAL_ALWAYS_INLINE
std::false_type recover_member(SAX* /*sax*/, std::false_type /*allow_recovery*/) const noexcept
{
return {};
}
/// get next token from lexer /// get next token from lexer
token_type get_token() token_type get_token()
{ {
@@ -573,7 +1281,14 @@ class parser
std::string exception_message(const token_type expected, const std::string& context) std::string exception_message(const token_type expected, const std::string& context)
{ {
std::string error_msg = concat("syntax error while parsing ", context, " - "); std::string error_msg = "syntax error ";
if (!context.empty())
{
error_msg += concat("while parsing ", context, ' ');
}
error_msg += "- ";
if (last_token == token_type::parse_error) if (last_token == token_type::parse_error)
{ {
@@ -591,7 +1306,7 @@ class parser
} }
return error_msg; return error_msg;
} // LCOV_EXCL_LINE (exception cleanup only) }
private: private:
/// callback function /// callback function
@@ -604,6 +1319,12 @@ class parser
const bool allow_exceptions = true; const bool allow_exceptions = true;
/// whether trailing commas in objects and arrays should be ignored (true) or signaled as errors (false) /// whether trailing commas in objects and arrays should be ignored (true) or signaled as errors (false)
const bool ignore_trailing_commas = false; const bool ignore_trailing_commas = false;
/// whether an error was reported to the SAX parser
bool error_reported = false;
/// the position of the last reported error
std::size_t last_error_position = 0;
/// the token of the last reported error
token_type last_error_token = token_type::uninitialized;
}; };
} // namespace detail } // namespace detail
@@ -108,9 +108,6 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{ {
JSON_ASSERT(m_object != nullptr); JSON_ASSERT(m_object != nullptr);
// m_it is already value-initialized, so this is not needed for
// correctness; it is kept because GCC generates faster loops with it
// (iterating a parsed document is 0.5-2.7 % slower without it, #6005)
if (m_object->is_object()) if (m_object->is_object())
{ {
m_it.object_iterator = typename object_t::iterator(); m_it.object_iterator = typename object_t::iterator();
+4 -4
View File
@@ -304,7 +304,7 @@ class json_pointer
result.reference_tokens.insert(result.reference_tokens.end(), lhs.reference_tokens.begin(), lhs.reference_tokens.end()); result.reference_tokens.insert(result.reference_tokens.end(), lhs.reference_tokens.begin(), lhs.reference_tokens.end());
result.reference_tokens.insert(result.reference_tokens.end(), rhs.reference_tokens.begin(), rhs.reference_tokens.end()); result.reference_tokens.insert(result.reference_tokens.end(), rhs.reference_tokens.begin(), rhs.reference_tokens.end());
return result; return result;
} // LCOV_EXCL_LINE (exception cleanup only) }
/// @brief create a new JSON pointer by appending the unescaped token at the end of the JSON pointer /// @brief create a new JSON pointer by appending the unescaped token at the end of the JSON pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slash/ /// @sa https://json.nlohmann.me/api/json_pointer/operator_slash/
@@ -315,7 +315,7 @@ class json_pointer
result.reference_tokens.insert(result.reference_tokens.end(), lhs.reference_tokens.begin(), lhs.reference_tokens.end()); result.reference_tokens.insert(result.reference_tokens.end(), lhs.reference_tokens.begin(), lhs.reference_tokens.end());
result.reference_tokens.push_back(std::move(token)); result.reference_tokens.push_back(std::move(token));
return result; return result;
} // LCOV_EXCL_LINE (exception cleanup only) }
/// @brief create a new JSON pointer by appending the array-index-token at the end of the JSON pointer /// @brief create a new JSON pointer by appending the array-index-token at the end of the JSON pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slash/ /// @sa https://json.nlohmann.me/api/json_pointer/operator_slash/
@@ -2180,7 +2180,7 @@ class json_pointer
{ {
// create an entry in the array // create an entry in the array
current = &current->operator[](array_index<BasicJsonType>(token)); current = &current->operator[](array_index<BasicJsonType>(token));
break; // LCOV_EXCL_LINE (needs an array index with 10 or more digits: more than 1e9 elements) break;
} }
/* /*
@@ -2217,7 +2217,7 @@ class json_pointer
json_pointer<string_t> result; json_pointer<string_t> result;
result.reference_tokens = reference_tokens; result.reference_tokens = reference_tokens;
return result; return result;
} // LCOV_EXCL_LINE (exception cleanup only) }
json_pointer<string_t> convert()&& json_pointer<string_t> convert()&&
{ {
+11
View File
@@ -215,6 +215,17 @@
#define JSON_NO_UNIQUE_ADDRESS #define JSON_NO_UNIQUE_ADDRESS
#endif #endif
// Inlines small functions even in unoptimized builds, so that they are not
// emitted. The parsers for parse(), accept(), and from_*() use it for the
// functions that stand in for the code recovering from errors (see #3989).
// MSVC is left to decide, as it warns (C4714) where it does not inline a
// __forceinline function.
#if defined(_MSC_VER) && !defined(__clang__)
#define JSON_INTERNAL_ALWAYS_INLINE
#else
#define JSON_INTERNAL_ALWAYS_INLINE JSON_HEDLEY_ALWAYS_INLINE
#endif
// Clang targeting MinGW does not survive the thread_local storage the copy // Clang targeting MinGW does not survive the thread_local storage the copy
// constructor uses to bound its descent: every test that copies a value // constructor uses to bound its descent: every test that copies a value
// segfaults with clang 11.0.1 and clang 18.1.8, while the same tests pass with // segfaults with clang 11.0.1 and clang 18.1.8, while the same tests pass with
@@ -19,6 +19,7 @@
#undef NLOHMANN_CAN_CALL_STD_FUNC_IMPL #undef NLOHMANN_CAN_CALL_STD_FUNC_IMPL
#undef JSON_INLINE_VARIABLE #undef JSON_INLINE_VARIABLE
#undef JSON_NO_UNIQUE_ADDRESS #undef JSON_NO_UNIQUE_ADDRESS
#undef JSON_INTERNAL_ALWAYS_INLINE
#undef JSON_DISABLE_ENUM_SERIALIZATION #undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION #undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
#undef JSON_UNLOCKED_FILE_IO #undef JSON_UNLOCKED_FILE_IO
Loaded 100 of 170 files, more files were not shown because too many files have changed in this diff. Show more