- Name std::hash only once the object type has emplace_new_key: VS 2015
static-asserts on std::hash of a type without one (alt_string) instead of
reporting a substitution failure (C2338).
- Replace the std::copy calls on frames by loops: VS 2015 reports raw
pointers handed to std::copy as C4996, an error under /WX.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: byte layout sections, glossary, and QA page updates
- Move "Byte layout" of each binary format to its own section before
"Serialization" and describe the idea of the format.
- Glossary: add LLDB and other terms used throughout the documentation.
- quality_assurance: correct Clang Static Analyzer and CMake entries; add
CodeQL, Semgrep, Flawfinder, CIFuzz, OpenSSF Scorecard, REUSE, Meson/SwiftPM,
and reproducibility checks.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: drop unused length colour from CBOR and MessagePack byte layout legends
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document clang-tidy exclusions and drop unneeded ones
Re-enable every excluded check in a run of clang-tidy 22 over all tests
and remove the seven that never fire (cert-err58-cpp,
clang-analyzer-webkit.NoUncountedMemberChecker,
cppcoreguidelines-virtual-class-destructor, google-readability-function-size,
hicpp-function-size, hicpp-no-assembler, readability-function-size).
Document the reason for every remaining exclusion at the top of the file.
Exclude in advance the checks that only exist in clang-tidy 23 and are
noise here (bugprone-signed-bitwise, *-explicit-constructor,
readability-trailing-comma, modernize-use-string-view), plus
modernize-use-integer-sign-comparison, which needs C++20 std::cmp_less
and flags deliberate casts.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in documentation examples and replace redundant aliases
Apply clang-tidy fixes to the documentation examples:
- performance-avoid-endl: std::endl -> '\n'
- readability-qualified-auto: auto -> auto* / const auto*
- performance-for-range-copy: copy -> const reference
- google-readability-casting: C-style cast -> static_cast
Replace redundant aliases (readability-redundant-qualified-alias) such as
"using json = nlohmann::json;" with "using nlohmann::json;" in the
documentation, README, examples, tests, and benchmarks. Aliases at class
scope and aliases of other types (with_base_class_t etc.) are kept.
No changes to the library or its public API. All changed examples were
compiled and run; their output matches the stored .output files (except
max_size and meta, which depend on the platform and compiler).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Format doc examples with the pinned astyle
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Docs: fill documentation gaps found in user questions
Add pages on class-hierarchy/smart-pointer/variant/chrono recipes, working
without exceptions, and large, streamed and partial input. Extend the FAQ
(23 entries, Known bugs first), parse error documentation (empty-input
checklist, line/column, diagnostic positions), element access, conversions,
creating values, ADL troubleshooting, memory, and build/distribution notes.
Add 14 examples with generated output. Correct the statement that parse errors
cannot be recovered from with -fno-exceptions.
The topics come from recurring questions on Stack Overflow, Reddit, Hacker
News, YouTube, and the project's GitHub discussions and issues.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: declare range-for lists in variables to keep astyle quiet
astyle reformats a braced init list placed directly in a range-for
header; declare the list first so check_amalgamation passes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: cross-reference json_view and json_document for 3.13.0
Document the new zero-copy document/view API across the docs: add a
"Documents and views" section to the basic_json API hub linking to all
specializations, point parsing/conversion/modifying pages to the view-based
alternatives, add document/view sections to feature pages, update the
landing page to mention json_view.hpp/json_literals.hpp, and cross-link
debugging/exceptions.
Also document wide-string inputs (and ill-formed UTF-16/32 rejection),
add parse/accept examples for wide strings, fix related doc gaps, and
align features/parsing and FAQ with the added support.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: fix article before std::istream in FAQ
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: clarify document inputs, decoding, get<T>() and editing
Address review comments: say which inputs are borrowed and that a wide
string pointer is an extra input, which values are decoded during parsing,
that get<T>() reads only built-in types directly, all forms of set() and
that the document's insert() is array-only, and how to store wide strings.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: integers are converted while parsing a document
Also say "They accept every input" in basic_json_document::parse.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* from_json: number-first dispatch for arithmetic types, loop for std::array (#5891)
from_json(ArithmeticType&) now reads numbers through get_number_value() and
handles a boolean on a cold path, which puts the three number types first in
the per-element type dispatch.
get<std::array<T, N>>() always took the identity_tag overload, a pack
expansion of N calls j.at(i).get<T>() that clang unrolls into straight-line
code (about 28,000 instructions for N = 1000), so its speed depended on
inlining decisions in that body. For trivial element types, which cannot tell
default construction plus assignment from construction in place, it now fills
a std::array in a loop. That loop, shared with get_to(), checks the size once
and reads the inner array directly; a short array still converts the elements
it has and then throws the out_of_range error at() would throw.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the get_to result check without exceptions
Without exceptions, the get_to calls in CHECK_THROWS_WITH_AS do not run,
so the array keeps its initial values (ci_test_noexceptions).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make is_trivial_element a struct so astyle does not indent it
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Expect the element path in the std::array type-error tests with JSON_DIAGNOSTICS
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Shorten from_json_inplace_array so astyle aligns its parameters
The opening parenthesis of the previous name sat in column 41, beyond
astyle's default --max-continuation-indent of 40, so the continuation
lines fell back to a plain double indent.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the compact template<...> style for the new from_json overloads
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add shared detail helpers for the #5891 cleanup
Add detail::is_primitive(value_t), detail::hex4_lower(), detail::utf8_shape(),
detail::input_format_name(), detail::empty_input_message(), the exception ids
116, 319, 320 and 416 to detail::exception_id, and make
detail::is_ascii_copyable() constexpr. Later commits switch the duplicated
code over to them.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.4: BSON writers share one write-pass UTF-8 sanitizer (#5891)
write_bson_entry_header and write_bson_string re-derived the error-handler
policy differently. Both now use sanitize_utf8_for_bson_write, which treats
strict like keep (the size pass already validated) and sanitizes replace/ignore
into the bytes the size pass measured. No context plumbing is needed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.1.1: one MessagePack container prefix helper (#5891)
Array and object prefixes differed only in the fix marker and the 16/32-bit
markers; both now delegate to write_msgpack_container_prefix. Output is
byte-identical.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.1.2: one iterative walker for CBOR and MessagePack (#5891)
write_cbor_iterative and write_msgpack_iterative (and their *_value_or_push
helpers) were identical but for the head, scalar, and key writers. They are now
one template over a small policy struct, so everything inlines. BON8 keeps its
own walker (open string run, closing marker), noted in a comment.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.1.3, 2.1.8: one UBJSON container start helper, shared frame base (#5891)
write_ubjson_start_array/_object are now write_ubjson_start_container<IsArray>,
keeping the array-only begin()+1 scan and excessive_valueless rule.
ubjson_frame derives from binary_container_frame instead of duplicating it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.1.4: write_bytes helper for the byte-string write sites (#5891)
The 13 oa.write_characters(reinterpret_cast<const CharType*>(x.data()), n)
sites now call write_bytes(const void*, size_t); copy_characters sites and the
buffer-based writes are untouched.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.1.5: shared object-key writer for CBOR and MessagePack (#5891)
write_cbor_key and write_msgpack_key were the same SFINAE pair apart from the
string/scalar writers; both now delegate to write_object_key<Walk>. UBJSON key
and to_format_length stay as they are.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.10: writer-side format names from input_format_name (#5891)
The CBOR/MessagePack/UBJSON/BSON/BJData/BON8 literals in throw_on_discarded and
throw_integer_out_of_range calls now come from detail::input_format_name; the
texts are identical, and UBJSON vs BJData still follows use_bjdata.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* serializer: constexpr ascii table, hex4_lower, dedupe helpers (#5891)
1.1: build ascii_copyable() at compile time from is_ascii_copyable and
test find_ascii_copyable_run against it for every byte. 1.2/2.5.4: use
detail::hex4_lower in both \uXXXX writers. 2.1.6: factor the repeated
flush/emit blocks in dump_escaped_impl into always-inline helpers.
2.1.7: comment why the escape copy does not use put_buffer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.6: use string_t-aware to_string/concat in meta() (#5891)
Replaces the four std::to_string sites; rendered text unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.7: use detail::is_primitive in json.hpp switches (#5891)
has_no_children, set_parents, max_size and the two diff walkers.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.1: factor insert()/iterator throw helpers (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.2: share swap() overloads via swap_impl (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.4: factor primitive reset and range check in erase()/range ctor (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.6: share push_back array bodies via push_back_array_impl (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.7: route parse/accept/sax_parse through *_impl helpers (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.8: share the vector-returning to_* bodies via to_binary_vector (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.2.10: merge destroy_string/destroy_binary into destroy_pointer (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.4.1/1.7: exceptions: share what_impl, flatten diagnostics switch (#5891)
The four create() bodies build their message through one protected
static helper; the diagnostics parent walk handles array/object and
ignores all other types.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.4.2/2.4.3/1.7: json_pointer: share slash and escape checks, drop primitive case lists (#5891)
The leading-slash and invalid '~' escape checks are shared helpers; the
reference-token walks test object and array first and treat everything
else as primitive.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.7/2.4.9: iter_impl, iteration_proxy: drop primitive case lists, share compare preamble (#5891)
Object and array stay the first branches; everything else is the
primitive case. operator== and operator< share the different-container
check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.4.5/2.4.6: from_json: share get_number_value and reserve_if_possible (#5891)
The three number cases of get_arithmetic_value and the arithmetic
from_json share one helper (boolean stays accepted only by the latter);
the array and object reserve overload pairs become one.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.4.8: ordered_map: share erase_impl between the key erase overloads (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.7: json_pointer: flatten walk drops the primitive case list (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.2/1.3/1.5/2.5.1/2.5.2/2.5.5: share helpers in the view code (#5891)
- detail::exception_id at every numeric id; throw_* helpers take it
- hex_byte, hex4_lower, utf8_shape replace local hex/table code
- throw_array_index_out_of_range/_past_end, throw_key_not_found
- has_native_float_format replaces the hand-written IEEE check
No change of messages, ids or output.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.5: use utf8_shape in get_bon8_string, add exhaustive UTF-8 validator test (#5891)
get_bon8_string takes the valid range of the second byte from utf8_shape()
(0xF5..0xF7 stay errors, unget behavior unchanged). validate_one_utf8 keeps its
if-chain (hot bulk path); the new test checks decode, validate_one_utf8,
utf8_shape, json::accept and the BON8 reader against each other for every lead
and second byte.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.1: get_cbor_sized_bytes for the CBOR chunk getters (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.2: get_string_with_length for the plain-order string sites (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.3, 2.3.4, 2.3.10: unsupported_key_type, empty_input_message, input_format_name in the readers (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.7, 2.3.9: read_object_key and leave_container_and_check for CBOR/MessagePack/BON8 (#5891)
UBJSON keys (different getter signature) and its container tail stay as is.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.5: json_sax_dom_error_state base for the DOM SAX parsers (#5891)
Holds errored, allow_exceptions, parse_error() and is_errored(). The scalar
handlers stay in each class (parse hot path).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.6: table of control-character messages in the lexer (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.3.8: finish_input helper for the strict-EOF block in the parser (#5891)
sax_parse keeps its if (result) guard and return value. Also adds the
clang-tidy NOLINT for the 2.3.6 message table.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.5.7: share parse_array_index between json_pointer and json_view (#5891)
Move json_pointer::parse_array_index() and its status enum to namespace detail
(json_pointer keeps a forwarding member), and let the json_view pointer
resolution use it instead of its own copy of the RFC 6901 index grammar. Both
accepted the same tokens with the same outcome. Also update the comment on
detail::decode(), which listed four of the five UTF-8 validators and now points
to the test that keeps them in sync.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.4.5: keep the inlined number cases in from_json(ArithmeticType&) (#5891)
Calling get_number_value() from from_json(ArithmeticType&) made
GetArray<std::array<int, 1000>> 2.4x slower (it changes the inlining of the
std::array conversion), so that overload keeps its own switch, with a comment.
get_arithmetic_value() still uses the shared helper.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 2.5.7: force-inline detail::parse_array_index (#5891)
Moving parse_array_index out of json_pointer stopped clang from inlining it
into the pointer resolution (JsonPointerAt/citm_catalog +17%,
JsonPointerAt/twitter +8%); JSON_HEDLEY_ALWAYS_INLINE restores the baseline.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 1.5: keep get_bon8_string's own UTF-8 ranges (#5891)
Taking the second-byte range from utf8_shape() made BON8 decoding 2-3% slower
(FromBinaryShape/scalars, FromBinaryBuffer, alignment-pinned builds), so the byte
path keeps its hand-written ranges. The exhaustive validator test still checks
that it accepts the same sequences as the other validators.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate (#5891)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures: constexpr, NRVO, cppcheck, IWYU (#5891)
- json_sax_dom_error_state::is_errored() cannot be constexpr: the class
is not a literal type (GCC 4.8, GCC -Wpedantic, Clang 3.5).
- to_binary_vector: destroy the vector writer before returning, as the
output_vector_sink sets the size of the vector in its destructor.
Without NRVO (MSVC Debug in C++14 mode), the vector was moved out
before it was resized.
- parser::sax_parse: return false instead of result after the early exit
(cppcheck identicalConditionAfterEarlyExit).
- json.hpp: export detail/exceptions.hpp for IWYU, so json_view.hpp can
name exception_id without including a detail header.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: throw helpers take input_format_t, simpler type checks
- throw_on_discarded() and throw_integer_out_of_range() take an
input_format_t and call input_format_name() themselves
- has_no_children(), set_parents() and max_size() check array, then
object, instead of calling is_primitive() first
- parse_impl() chains like accept_impl(); sax_parse_impl() uses if/else
- std::array for the control character messages; negation<> in
write_object_key(); clarify the is_primitive() comment
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* json_view: use exception_id in the value() overloads
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* json_sax: assert the invariant of place_container for the static analyzer
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- Make the iterator range illustrations legible in dark mode (prefers-color-scheme
rules inside the SVGs).
- Add a "Byte layout" section with a color-coded byte table to each binary format
page (BJData, BON8, BSON, CBOR, MessagePack, UBJSON), and highlight the smallest
size per file in the comparison table.
- Add "See also" sections to the overview pages that lacked one.
- Split the release notes into one page per release, grouped by minor version on
the overview page, with an "API changes" section per release generated from the
API pages' version histories (hooks/api_changes.py).
- Fact-check the release notes and version histories against the release tags and
correct wrong or misleading entries (e.g., json_serializer 2.1.0, merge_patch
3.1.0, items, count, operator[], to_bjdata, std::string_view key support).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
clang-tidy's static analyzer analyzes the forwarding SAX handlers of the
error recovery tests (RecoveringDomParser, BasicRecoveringParser) as entry
points, with the state of their json_sax_dom_parser member unknown. It then
assumes a frame stack of capacity 0 that grow() cannot enlarge
(clang-analyzer-security.ArrayBound) and a collected container without an
enclosing frame (clang-analyzer-core.NonNullParamChecker). Neither can
happen: the stack starts with local_capacity frames, and only a container
inside another one is collected. Assert both, which also lets the analyzer
discard these paths.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
sax_parse() recovers from an error if parse_error() returns true. For a
handler whose parse_error() is [[noreturn]], MSVC reports that code as
unreachable (C4702), which fails the /WX module build. throwing_sax now
throws under a flag, as json_sax_dom_parser does.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Since #5828, a stream adapter keeps the current token itself
(mark_token_start() / copy_token()), so lazy_token_string no longer
implies a seekable adapter. restart_token_string_impl(std::true_type)
still called get_consumed_count(), which only seekable adapters have, and
failed to compile for std::istream input. Dispatch on adapter_token as
note_token_start() does.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The lexer records an error_kind with each error_message, and
recover_token() dispatches on it instead of comparing message prefixes.
add_codepoint() now uses encode_utf8(), and the surrogate pair
formula is shared by scan_string() and recover_low_surrogate().
The stubs for AllowRecovery == false select via negation, which astyle
leaves alone.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With seekable input, the lexer derives line and column on demand (#5809)
by counting the newlines in all input consumed so far. A SAX parser that
recovers from errors asks for the position once per error, so input with
an error at every character took quadratic time: "{" followed by 100000
commas took 85 s at -O0, and the parser tests timed out under Valgrind
(1500 s, was 69 s on develop).
get_position() now remembers what it counted and only scans the input
consumed since the previous call; count_consumed_lines() takes the start
of the range. The 100000 commas now take 0.17 s, unit-class_parser 0.9 s
instead of 86 s. Over 69284 error messages from random multi-line inputs,
the messages are identical to those before the change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With develop's latest changes, the MinGW Debug object of unit-msgpack.cpp
has 33134 COFF sections on its own and 33411 with this PR, more than the
32767 that the MinGW linker of the "clang (20.1.8)" job can relocate. Its
sax_parse calls, all with utils::SaxCountdown, instantiate a parser and
binary reader of their own; moving those tests into unit-sax_parse.cpp
brings the object down to 32271 sections (unit-sax_parse: 20678).
The moved tests are unchanged: the "SAX aborts" sections, the user-defined
SAX consumer of #5405, the deep value read through the SAX interface (#5104),
and "MessagePack SAX parsing stops at every event".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- parser::report_error() calls only sax->parse_error(). For a SAX parser
whose parse_error() is static, as in unit-disabled_exceptions.cpp,
Visual Studio 2015 then reports the parameter as unreferenced (C4100),
which /WX turns into an error.
- The variadic concatenated() helper in unit-sax_parse.cpp made Visual
Studio 2015 fail with an internal compiler error (C1001). It now takes
an initializer list of byte vectors.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
f8ff0942b marked the stubs that stand in for error recovery with
JSON_HEDLEY_ALWAYS_INLINE, which is __forceinline with MSVC. Visual
Studio 2015 and 2017 do not inline them and warn (C4714), which /WX
turns into an error on AppVeyor. The new internal macro
JSON_INTERNAL_ALWAYS_INLINE forces inlining everywhere else (including
clang-cl), and leaves the decision to MSVC. The MinGW objects keep their
smaller section counts, as GCC and clang still inline the stubs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The MinGW clang job links with a GNU ld that cannot relocate an object
with more than 32767 sections. unit-regression2, unit-regression3 and
unit-msgpack went over it:
- At -O0, the parser and binary reader for parse(), accept() and
from_*() still emitted the stubs that stand in for recovering, and
the parser referenced its recovery helpers from code that never runs.
The stubs are now always inlined, and the parser reaches its helpers
through the same allow_recovery tag as the stubs, so that this code
is not even instantiated there. This removes about 300 sections from
unit-msgpack.
- Every sax_parse call instantiates the parser and binary reader that
recover from errors. The tests that call it move from
unit-regression2.cpp (#3989) and unit-regression3.cpp (#5676) into
unit-sax_parse.cpp, like unit-explicit_instantiation.cpp got a file
of its own (#5511), and CONTRIBUTING.md points there.
Section counts of the MinGW Debug objects (clang 19, -g0), develop / PR:
unit-msgpack 32281 / 32558, unit-regression2 31272 / 31278,
unit-regression3 32392 / 27462, unit-sax_parse - / 19836.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The local alias that the out-of-range test declared shadows the identical
file-scope one, which GCC's -Wshadow=global turns into an error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5737 enabled -Wextra-semi-stmt for the tests and removed the
semicolons after CAPTURE(), which expands to a complete statement; the
#3989 tests added twelve more.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The MinGW job ("clang (20.1.8)", Debug) failed to link test-msgpack_cpp17
with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against
`.rdata'". Its GNU ld (MinGW 12.2.0) cannot link an object file with
more than 32767 sections. Reproduced with clang 20.1.8 and binutils
2.40: unit-msgpack.cpp's object has 31958 sections on develop and had
33210 with #3989; binutils 2.44 links both.
The readers of from_*() never recover, but unoptimized builds still
emitted the recovery code their dead branches referenced: about 20
helpers per reader instantiation, plus the lexer's recovery functions
through the UBJSON high-precision number repair. Now:
- Errors are reported with report_error() only. The helpers that act on
the SAX parser's answer (repair_requested(), resync(), value_failed(),
close_open_containers(), and the repairs of high-precision numbers and
unsupported BSON elements) have one trivial variant for readers that do
not recover, so the recovery code is not referenced there.
- Object keys that are not strings and BSON elements whose end is lost
are marked at the error site and skipped by resync() in the reading
loops; report_repairable_error(), report_error_repairable_if(),
report_bson_element_error(), and bon8_error_repairable_if() are gone.
- is_cbor_item_head() is a free function, and the BSON element type is
formatted as on develop again.
The object now has 32638 sections and links. Behavior is unchanged
(regression tests, binary format suites, and fuzzing), and an optimized
program that only calls from_*() is now 256 bytes smaller than on
develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
clang's -Wdocumentation-unknown-command reads "\u" in a doc comment as
an unknown command, even in backticks. Two comments of the recovery code
now say "Unicode escape" instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
clang-tidy's bugprone-derived-method-shadowing-base-method rejected the
test SAX parsers that derive from json_sax_dom_parser (or the test's
SaxEventLogger) and redefine their non-virtual event functions. The
recovering DOM parsers of unit-class_parser.cpp and unit-regression2.cpp
now hold a json_sax_dom_parser and forward to it, SaxEventLogger gets a
flag to recover from errors instead of a derived class, and the one
function unit-alt-string.cpp redefines is marked as intended.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
ci_clang_tidy flagged nested conditional operators in the BON8 skip
code and in remove_incomplete_utf8_sequence()
(readability-avoid-nested-conditional-operator), and two branches with
the same body in recover_string() (bugprone-branch-clone). Use if
chains instead of the nested conditionals and merge the two branches
into one condition; the short-circuit order is unchanged.
ci_test_noexceptions aborted in the #3989 regression test: it compares
the first recovered error with the message of the exception that
from_cbor() and friends throw, and under JSON_NOEXCEPTION that call
aborts instead of throwing. Guard binary_error_message() and its uses
with #if !defined(JSON_NOEXCEPTION), as other tests in the suite do.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The U+FFFD constant was a global std::string, which the clang CI build
rejects with -Wglobal-constructors. It is now a function.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
MSVC stringizes the arguments of doctest's CHECK() so that a raw string
literal becomes an ordinary one, and then reported the "\q" in the new
alt_string recovery test as warning C4129, an error with /WX. The input
is now a variable. The same pattern with "\u0000" in the parser's
recovery test is replaced by a JSON value built from a std::string.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Moving the report of an unsupported BSON element type into
skip_unsupported_bson_element() moved its snprintf() call, which
Flawfinder then reported as a new CWE-134 finding. The two hexadecimal
digits are now computed directly; the message is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
recover_number() used back(), pop_back(), front(), and
find_first_of(const char*), which the minimal alt_string of
unit-alt-string.cpp does not provide. Since the binary readers recover
UBJSON/BJData high-precision numbers with it, from_ubjson() instantiated
it, too, and the test no longer compiled. It now uses only size(),
operator[], and resize(), and unit-alt-string.cpp recovers from errors
in JSON text, UBJSON, and CBOR.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When the SAX parser asks to recover, the binary readers now repair an
item whose end is known and read on after it, as RFC 8949, Section 5.3
describes for CBOR:
- CBOR: tags are ignored, and simple values other than false, true, and
null become null (RFC 8949, Section 6.1); a negative integer below the
range of number_integer_t becomes the nearest floating-point number.
- Strings that are not valid UTF-8 get U+FFFD for each ill-formed
sequence, as in JSON text; so does a UBJSON/BJData char above 0x7F.
- UBJSON/BJData high-precision numbers keep their longest valid
beginning (via the lexer's recover_token()), or become infinity.
- Members whose key is not a string are skipped (CBOR, MessagePack,
BON8), like members without a key in JSON text.
- BSON elements of types the library does not read (ObjectId, datetime,
decimal128, ...) become null; a string without its terminator and a
document whose size does not match are kept.
Where the end of an item is unknown, reading stops as before, except
that BSON skips to the end of the document, whose size it knows.
The value read before such an error is now completed by the reader from
its container stack, as the JSON parser does, instead of by a proxy SAX
parser, which is removed. Like the parser, binary_reader gets an
AllowRecovery template parameter, so that from_*() compile without the
new code.
Tests: a table of repairs, numbers out of range, errors that stop, and
a sweep over changed and removed bytes of eight encodings that checks
balanced events and that the first error is the one from_*() reports.
All fuzzers now run a recovering checker; the binary ones also check
that it reports an error exactly when from_*() fails.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The return value of json_sax::parse_error() was documented both as "must
return false" and as "whether the parsing should continue", and the code
just passed it on. For JSON text, parsing stopped anyway, but sax_parse()
could report success for invalid input. The binary readers read on after
the error, looping forever on a CBOR indefinite-length array without its
end.
Now false stops parsing, and true recovers from the error:
- JSON text is repaired with the smallest local edit (insert a missing
',' or ':', remove a stray token, keep the readable part of a broken
string or number, null for a value that cannot be read, close the
innermost container at a wrong closing bracket and all of them at the
end of the input), and parsing continues. The SAX events stay balanced,
every key is followed by exactly one value, and each token is reported
at most once.
- The binary formats cannot resynchronize, so they stop, but complete
the value read so far.
sax_parse() returns false after any error. parse(), accept(), and the
from_*() functions never recover and compile to the same code as before.
Supersedes #4522.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 20:16:42 +02:00
584 changed files with 14461 additions and 5240 deletions
The return value of each function determines whether parsing should proceed.
The return value of each function determines whether parsing should proceed. For `parse_error`, returning `true` [recovers from the error](https://json.nlohmann.me/features/parsing/error_recovery/): the parser repairs the input and continues.
To implement your own SAX handler, proceed as follows:
@@ -503,7 +503,7 @@ To implement your own SAX handler, proceed as follows:
2. Create an object of your SAX interface class, e.g. `my_sax`.
3. Call `bool json::sax_parse(input, &my_sax)`; where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface.
Note the `sax_parse` function only returns a `bool` indicating the result of the last executed SAX event. It does not return a `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error -- it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp).
Note the `sax_parse` function only returns a `bool` indicating whether the input was parsed without errors and no SAX event returned `false`. It does not return a `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error -- it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp).
### STL-like access
@@ -842,7 +842,7 @@ assert(p == p2);
To make this work with one of your types, you only need to provide two functions:
```cpp
usingjson=nlohmann::json;
usingnlohmann::json;
namespacens{
voidto_json(json&j,constperson&p){
@@ -1257,7 +1257,7 @@ See the page [quality assurance](https://json.nlohmann.me/community/quality_assu
#include<nlohmann/json.hpp>
// for convenience
usingjson=nlohmann::json;
usingnlohmann::json;
```
to the files you want to process JSON and set the necessary switches to enable C++11 (e.g., `-std=c++11` for GCC and Clang).
1. Added in version 1.0.0. Changed parameter `default_value` type from `const ValueType&` to `ValueType&&` in version
3.11.0. Deleted overload for integral key types added in version 3.13.0 to reject such calls at compile time
instead of causing undefined behavior at runtime.
2. Added in version 3.11.0. Made `ValueType` the first template parameter in version 3.11.2. Fixed in version 3.13.0
to consistently accept `std::string_view`-convertible keys, as already supported by
[`operator[]`](operator[].md), [`at`](at.md), [`find`](find.md), and other lookup functions.
3. Added in version 2.0.2. Extended to work with arrays in version 3.13.0, including fixing an issue where resolving
`ptr` through an array unexpectedly threw `out_of_range` instead of returning the resolved element (or
`default_value`, as documented). Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109)
2. Added in version 3.11.0. Made `ValueType` the first template parameter in version 3.11.2. Support for keys
convertible to `std::string_view` added in version 3.13.0, consistently across the lookup functions.
3. Added in version 2.0.2. Extended to work with arrays in version 3.13.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109)
instead of returning `default_value` for a one-character array index that is not a digit (e.g., `/x`) in version
A parsed JSON text, held as a flat index of its values
([16 bytes per value](../../home/architecture.md#node-index-of-json-views)) instead of a tree of `BasicJsonType` values.
Strings and numbers stay in the source text; only strings that contain escapes are decoded, into one buffer owned by
the document. [`basic_json_view`](../basic_json_view/index.md) is a read-only handle to one value of a
Strings and floating-point numbers stay in the source text; integers are converted while parsing, and strings that
contain escapes are decoded into one buffer owned by the document. [`basic_json_view`](../basic_json_view/index.md) is a read-only handle to one value of a
`basic_json_document`; [`materialize()`](../basic_json_view/materialize.md) turns a subtree into the
`BasicJsonType` value that [`BasicJsonType::parse()`](../basic_json/parse.md) would have produced for it.
@@ -115,6 +115,13 @@ points apply to every edit:
[`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is
`#!cpp static_cast<std::size_t>(-1)`, the same value it reports for a decoded string.
## See also
- [Zero-copy JSON views](../../features/json_view.md) - guide to documents and views
- [**basic_json_view**](../basic_json_view/index.md) - the non-owning handle to a value in a document
- [**json_document**](../json_document.md) and [**json_editable_document**](../json_editable_document.md) - the default specializations
- [**materialize**](../basic_json_view/materialize.md) - build a `basic_json` value from a view
Loaded 100 of 584 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Niels Lohmann