After the digits are converted while scanning, copying the token text into
token_buffer (an out-of-line std::string::append) was about 10% of parsing
canada.json. The only reader is the string argument of
json_sax::number_float(), and the library's own handlers
(json_sax_dom_parser, json_sax_dom_callback_parser, json_sax_acceptor)
ignore it.
For contiguous input, the fast path now keeps a pointer to the token in the
input, and get_string() copies it on first use. The parser passes an empty
string to the handlers marked by the new internal trait
sax_ignores_number_text instead of calling get_string(), so json::parse()
and json::accept() never copy float text. User handlers, and
JSON_DIAGNOSTIC_POSITIONS (which reads get_string().size()), still see the
exact text. Keys and strings, which never have pending text, use the
unchecked get_string_token(). Input read through a window into a stream
(std::istream, FILE*) copies the text right away, because the next read may
refill the window.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Accumulate number digits while scanning them
The contiguous number fast path validated a token byte by byte, copied it
into token_buffer, and then converted it in out-of-line functions that
scanned the digits a second time: parse_float_native() called
parse_float_significand(), which returned its result through memory and
called std::any_of (out of line with GCC) twice per number, and
parse_integer_signed() divided by ten on every digit for its overflow check.
scan_number_bulk_contiguous() now accumulates the significand while it
validates the digits, eight at a time where at least eight bytes remain
(is_eight_digits() and parse_eight_digits(), after fast_float). Floats with
at most 19 digits go straight to decimal_to_float(); integers with at most
18 digits are stored without an overflow check (a narrower number type
still rejects values that do not round-trip). Longer tokens take the
existing path. has_nonzero_digit() is a plain loop and
parse_float_significand() is always inlined, which helps that path.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the digit accumulation checks without exceptions
The section compares error messages from both lexer paths and parses
out-of-range numbers, so it does not compile (the caught exception does not
exist) and would abort with JSON_NOEXCEPTION.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Share the significant-digit loop of the integer part and the fraction
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Since #5828, a stream adapter keeps the current token itself
(mark_token_start() / copy_token()), so lazy_token_string no longer
implies a seekable adapter. restart_token_string_impl(std::true_type)
still called get_consumed_count(), which only seekable adapters have, and
failed to compile for std::istream input. Dispatch on adapter_token as
note_token_start() does.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With glibc and optimization, <stdio.h> turns getc_unlocked into an extern
inline function that reads the stdio buffer (fp->_IO_read_ptr) directly.
glibc, which is not instrumented, fills that buffer, so MemorySanitizer
treats the bytes as uninitialized and reports a false positive when the
parsers read from a FILE* (e.g., the fuzz harness FILE* leg, OSS-Fuzz
575915982). std::fgetc is an opaque call whose result MemorySanitizer treats
as initialized.
Select JSON_UNLOCKED_FILE_IO 0 when compiled with MemorySanitizer. This is a
regression from #5828 and not part of any release.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The lexer records an error_kind with each error_message, and
recover_token() dispatches on it instead of comparing message prefixes.
add_codepoint() now uses encode_utf8(), and the surrogate pair
formula is shared by scan_string() and recover_low_surrogate().
The stubs for AllowRecovery == false select via negation, which astyle
leaves alone.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make parsing from std::istream and FILE* fast
Squashed onto develop cd6b428ef (CI fixes from #5860). Original commits:
- Let the lexer's bulk fast paths run on a window of the input
- Scan the get area of an input stream in bulk
- Lock a FILE once per parse instead of once per character
- Add benchmarks for parsing from a FILE* and for many values from a stream
- Avoid implicit unsigned char to char conversion in stream tests
- Make file_input_adapter::unlock() safe to call on a moved-from adapter
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add ABI tags for JSON_DISABLE_STREAM_BULK_SCAN and JSON_DISABLE_UNLOCKED_FILE_IO
Both macros select between two definitions of an input adapter, so
translation units compiled with different settings violated the ODR when
linked together. Like the other ABI-affecting macros, they now append a
tag to the inline namespace: _dsbs and _dufio. Their defaults move to
abi_macros.hpp so that json_fwd.hpp sees them, and the natvis file is
regenerated for the new namespaces.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the exception-only checks of the stream input tests without exceptions
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With seekable input, the lexer derives line and column on demand (#5809)
by counting the newlines in all input consumed so far. A SAX parser that
recovers from errors asks for the position once per error, so input with
an error at every character took quadratic time: "{" followed by 100000
commas took 85 s at -O0, and the parser tests timed out under Valgrind
(1500 s, was 69 s on develop).
get_position() now remembers what it counted and only scans the input
consumed since the previous call; count_consumed_lines() takes the start
of the range. The 100000 commas now take 0.17 s, unit-class_parser 0.9 s
instead of 86 s. Over 69284 error messages from random multi-line inputs,
the messages are identical to those before the change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
These PRs were merged while their CI runs were cancelled, so the warnings
went unnoticed. All of them fail the -Werror builds:
- json_pointer.hpp (#5836): size_t += int under GCC -Wsign-conversion
- binary_reader.hpp (#5839): char_int_type passed as char under GCC
-Wconversion
- json.hpp (#5838): @throw without a description under clang
-Wdocumentation
- string_escape.hpp (#5836): escape_into() was inserted between escape()
and its documentation, so the @return was attached to a void function
- unit-diagnostics.cpp (#5832): clang -Wnrvo in a lambda without an
explicit return type
- unit-to_json-rvalue.cpp (#5837): GCC -Wnoexcept (C++20) and clang
-Wunneeded-member-function on the test type
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Cppcheck 2.20 rejects invocations of unknown function-like macros in #if
conditions instead of evaluating them as 0. default_defines.hpp expands
the Hedley macros to the compiler's feature-test operators (__has_attribute
etc.), so define them as 0 in a generated include file.
It also reports a false bufferAccessOutOfBounds in the float overload of
float_from_bits, sizing the parameter as the double overload's; use the
size of the source instead (equal by the static_assert).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Allow the same type for number_integer_t and number_unsigned_t
basic_json did not compile with NumberUnsignedType == NumberIntegerType
because overloads on both types collided. Such a basic_json now stores
every integer as number_integer: the colliding overloads take a
placeholder type, unsigned C++ integers are converted to number_integer_t,
and value_t::number_unsigned maps to number_integer.
Also fixes two problems this configuration would hit: the unsigned
integer parser accepted values that wrap around to negative numbers, and
dump() overflowed when computing the absolute value of the minimum.
A repeated NumberFloatType gets a static_assert.
Refs #2573
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use json::with_integers_t in the test and the docs
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* ✅ skip patch test on GCC 5 to avoid internal compiler error
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: name the unsigned overload type through template parameters
basic_json used number_unsigned_overload_t<basic_json> while the class was
still incomplete. MSVC rejected it as a recursive alias declaration and GCC
4.9, 5 and 7 crashed with an internal compiler error. basic_json now uses
number_unsigned_overload_type<NumberIntegerType, NumberUnsignedType>, which
also makes the GCC 5 workaround in the patch test unnecessary.
Infer reported a null dereference for the number_unsigned case in
from_json, which cannot happen for a basic_json whose number_unsigned_t is
number_integer_t; assert it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: conjunction/negation trait, astyle-friendly alias, docs table order
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write doubles with the shortest digits (Żmij), digits in registers
Write doubles with the conversion of Zmij by Victor Zverovich (MIT),
ported to C++11 (detail/conversions/zmij.hpp). It finds the
shortest decimal that reads back as the same double, and the
closest one if there are several. Grisu2, used until now, is fast
but not always shortest: it sometimes writes a 17th digit where 16
suffice, or a last digit that is not the closest. The layout is
unchanged (1.5, 100.0, 1e+100, -0.0); float keeps Grisu2.
Digits are converted eight at a time with the BCD conversion of
Xiang JunBo, as in Zmij, and written with one byte swap per eight
digits and fixed-size moves instead of per-digit loops. Leading and
trailing zeros are counted from those bytes. to_chars() uses a
local buffer when the caller's is shorter than the 41 bytes this
may write. The powers of ten come from the number-parsing table,
adjusted where it holds values rounded up, and extended with Zmij's
compressed tables beyond 10^308.
write_shortest() converts its 16 digits in one vector register
(SSE2 on x86-64, NEON on 64-bit Arm, both baseline) and inserts the
decimal point inside the register, avoiding a store-forwarding
stall that cost about 25% of the time to write a double. dump()
writes floats and integers straight into the serializer's write
buffer instead of copying them from a member buffer, and small
integers eight digits at a time. read_eight_bytes() and
parse_eight_digits() are marked always-inline, which GCC had been
calling out of line in the number-parsing loops.
Of one million random doubles, about 0.14% are now written with
different digits, always to a value that still reads back as the
same double.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI findings in the Zmij writer and its tests
- bit_ops.hpp: include macro_scope.hpp (JSON_HEDLEY_ALWAYS_INLINE); fixes IWYU
- to_chars.hpp: C4100 for the unused parameter in release builds, clang-tidy
sign comparison, cpplint runtime/int, GCC -Wstrict-overflow (unsigned abs)
- unit-to_chars.cpp: parse with the library instead of strtod (MinGW's strtod
rounds some 16/17 digit inputs wrongly); no floating-point std::to_chars
with icpc
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Select the Zmij conversion by a trait, not by the double overload
The non-template overloads for double asserted binary64 doubles wherever json.hpp was included, so the library no longer compiled where double is not IEEE 754 binary64 (AVR, -fshort-double). A trait now picks Zmij for any binary64 type, including a long double of that format (MSVC, Apple Arm), and Grisu2 for the others. Remove the unused write_short_decimal, powers_of_ten_16, zmij::decimal, zmij::to_decimal, and shortest_digits(double).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Credit Zmij's authors in to_chars.hpp, the README, and the license page
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence MSVC C4127 for a platform-constant check in the to_chars test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up the lexer: own float parser, string scan, and \u table
Give the library its own correctly rounded float converter for
binary32 and binary64 (IEEE 754), and speed up the lexer's string
and escape scanning.
The converter splits a number token into sign, significand, and
decimal exponent, then tries Clinger's fast path, then a templated
Eisel-Lemire step, and falls back to an exact big-integer digit
comparison for tokens with more than 19 significant digits whose two
candidate values round differently. This replaces std::from_chars
and strtod/strtof for both formats, so parsed values no longer
depend on the C/C++ library or the current locale. The strtold
fallback kept for other long double formats (x87, binary128) now
also copies a multi-byte decimal point correctly, fixing #5660.
eisel_lemire() and decimal_to_float() are always inlined so callers
keep the whole conversion in their hot loop.
The string-scanning kernels in string_scan.hpp find a stop byte with
the trailing-zero count of the SWAR mask instead of a byte loop, and
scalar_string_bulk_run() validates a run of multi-byte UTF-8
sequences one after another instead of re-searching after each one.
get_codepoint() decodes a contiguous \uXXXX escape with one table
lookup per byte instead of four range-checked get() calls; the
streaming path and all error positions are unchanged.
Adds 508 generated hard float-parsing cases with expected binary32
and binary64 bits, and kernel-comparison tests for the string scans
and the escape table against byte-by-byte references.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI warnings in own float parser
- pow5_table.hpp: pow5_128_largest_power was unused in this branch's
own code (GCC -Werror=unused-const-variable); tie it to the table
size with a static_assert instead of removing it, since a later
branch in the stack (json-view/23-zmij) uses it.
- number_parse.hpp: rename the local variable `copy` to `buffer` to
satisfy cpplint's build/include_what_you_use check.
- unit-class_lexer.cpp: extend the NOLINT list on the seeded mt19937
with bugprone-random-generator-seed, and parenthesize
`8 * sizeof(Bits) - 1` for clang-tidy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Load string scan words with memcpy and use MSVC bit-scan intrinsics
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reword float conversion notes for the string type and number_float_t docs
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy 22 findings in unit-class_lexer.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark the unsigned long of the MSVC bit scans for cpplint
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Drop the position updates in the bulk \u path, which #5809 made unused
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: std::any_of in has_nonzero_digit, comment punctuation
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use data() instead of c_str() in the multi-byte decimal point path
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Pretty-printed JSON indents every line with a run of spaces, which
skip_whitespace() read one character at a time. For contiguous input
(bulk_scan) the lexer now skips the spaces after a newline 8 bytes at a
time directly in the input buffer, including the leading spaces of the
last, partial word. This bypasses get(), which is safe because
bulk_scan implies the lazy position from the previous commit: nothing
is counted per character.
Compared to lazy positions alone, pretty-printed input (twitter,
citm_catalog with indent 2 and 4) parses up to 13% faster and accept()
gets up to 30% faster (GCC 13 and clang 18 on x86-64, Apple clang 21 on
arm64). Minified input is unchanged within noise.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Derive the lexer position lazily for seekable input
The lexer counted the read position on every character: two increments
and a newline check per get(), all on members kept in memory. For
seekable input (pointers, strings, other random-access byte ranges) the
position can be derived instead: chars_read_total is the adapter's read
count (including reads at EOF) minus a pending unget, and line and
column are recomputed from the consumed input only when get_position()
is called, i.e. on parse errors. JSON_DIAGNOSTIC_POSITIONS uses the new
O(1) get_chars_read_total(). Stream, file, wide-string and user-defined
adapters keep the eager counters.
Parsing pretty-printed input gets 8-12% faster and accept() 12-27%
(canada, citm_catalog, twitter; clang 21 arm64 and GCC 13 x86-64).
Error messages, byte offsets, SAX positions and diagnostic positions are
unchanged (differential comparison against develop on 548k inputs).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the std::vector<std::byte> binary type tests out of unit-regression3
The #4804/#4209 tests instantiated a second basic_json specialization
(with_binary_t<std::vector<std::byte>>), including its parser and CBOR
reader. With clang 18, the C++20 object of unit-regression3.cpp had
32656 COFF sections on develop; the lazy position helpers pushed it to
32975, past what the MinGW linker of the Windows clang jobs handles
("relocation truncated to fit: IMAGE_REL_AMD64_REL32 against .rdata").
unit-custom-binary-type.cpp already instantiates the same type, so the
tests move there; unit-regression3.cpp drops to 27616 sections.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Report the parse errors of the position test via SAX
The test caught json::parse_error, so it aborted in the
ci_test_noexceptions build, where JSON_NOEXCEPTION turns the throw into
std::abort(). sax_parse() hands the error position and the exception
object to parse_error() without throwing, so the test now records them
there and runs in every configuration.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use and extend the detail helpers to remove duplicated code
Library:
- binary_reader: format bytes with hex_byte() instead of snprintf
- add throw_type_must_be() for the 20 copies of type_error.302
- binary_reader: add last_byte_error()/unexpected_byte() for the 32
"parse error at the last read byte" sites (replaces bon8_error)
- json_sax: add check_container_size() for out_of_range.408 and
diagnostic_positions::set_container_start/_end()
- json_pointer: add throw_no_parent() (405) and throw_unresolved() (404)
Tests:
- unit-class_parser uses the shared utils::SaxCountdown
- move SaxEventLogger (and its ExitAfter* variants) from unit-class_parser
and unit-deserialization into the new tests/src/test_sax.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Factor out more repeated error paths and test boilerplate
Library:
- add throw_cannot_use_with() for the 34 copies of type_error.304-312
"cannot use X with Y"
- iter_impl: add throw_cannot_get_value() (invalid_iterator.214)
- parser: add syntax_error() for the 12 parse_error.101 sites
- ordered_map: share the four at() bodies via at_impl()
- json_sax_dom_callback_parser: add pop_container() for end_object()
and end_array()
- binary_reader: build the two UBJSON/BJData length-type messages with
concat() and last_byte_error()
Tests:
- move same_value(), the NDEBUG guard, and step 0 (parse without
exceptions) of the seven fuzzer drivers into tests/src/fuzzer_common.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the exception ids and address review comments
Add detail::exception_id, a scoped enum with one named enumerator per
documented exception id, and use it for every id in the library. The
create() functions get an overload for it; the int overloads stay for
user code.
Following the review of #5783: add binary_reader::invalid_byte() and
length_type_error(), basic_json::throw_subscript_wrong_type(), move the
fuzzer includes and the using-declaration into fuzzer_common.hpp, and
rename test_sax.hpp to sax_event_loggers.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence -Wweak-vtables for the SAX event loggers
The loggers moved from anonymous namespaces in the test files into
sax_event_loggers.hpp, so clang now warns that their vtables are
emitted in every translation unit.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence MSVC 2015 C4100 in parser::syntax_error for static SAX::parse_error
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Inline invalid_byte() into its call sites
The wrapper only fixed the message string of unexpected_byte(), which is
the same kind of per-argument helper that was declined for
throw_type_must_be(). Call unexpected_byte("invalid byte", ...) directly.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- parser::report_error() calls only sax->parse_error(). For a SAX parser
whose parse_error() is static, as in unit-disabled_exceptions.cpp,
Visual Studio 2015 then reports the parameter as unreferenced (C4100),
which /WX turns into an error.
- The variadic concatenated() helper in unit-sax_parse.cpp made Visual
Studio 2015 fail with an internal compiler error (C1001). It now takes
an initializer list of byte vectors.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
f8ff0942b marked the stubs that stand in for error recovery with
JSON_HEDLEY_ALWAYS_INLINE, which is __forceinline with MSVC. Visual
Studio 2015 and 2017 do not inline them and warn (C4714), which /WX
turns into an error on AppVeyor. The new internal macro
JSON_INTERNAL_ALWAYS_INLINE forces inlining everywhere else (including
clang-cl), and leaves the decision to MSVC. The MinGW objects keep their
smaller section counts, as GCC and clang still inline the stubs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The MinGW clang job links with a GNU ld that cannot relocate an object
with more than 32767 sections. unit-regression2, unit-regression3 and
unit-msgpack went over it:
- At -O0, the parser and binary reader for parse(), accept() and
from_*() still emitted the stubs that stand in for recovering, and
the parser referenced its recovery helpers from code that never runs.
The stubs are now always inlined, and the parser reaches its helpers
through the same allow_recovery tag as the stubs, so that this code
is not even instantiated there. This removes about 300 sections from
unit-msgpack.
- Every sax_parse call instantiates the parser and binary reader that
recover from errors. The tests that call it move from
unit-regression2.cpp (#3989) and unit-regression3.cpp (#5676) into
unit-sax_parse.cpp, like unit-explicit_instantiation.cpp got a file
of its own (#5511), and CONTRIBUTING.md points there.
Section counts of the MinGW Debug objects (clang 19, -g0), develop / PR:
unit-msgpack 32281 / 32558, unit-regression2 31272 / 31278,
unit-regression3 32392 / 27462, unit-sax_parse - / 19836.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check for the expected separator before the lexer's token switch
After a key the parser expects ':', after a value usually ','. Test for
that character first instead of going through scan()'s switch, which
compiles to an indirect jump. Any other character takes the old path,
so tokens and error messages are unchanged.
Parsing 6.3% faster with GCC 15.2 and 2.7% with Clang 22.1 (geomean of
the ParseString, ParseFile and ParseIndented benchmarks).
Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
* Improvement: address PR comments
Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
* fix: address comments
Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
* Fix clang-tidy bugprone-signed-char-misuse in scan_expecting
Convert the expected separator through unsigned char before storing it as
char_int_type. The generated code is unchanged.
Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
* Use raw string literals in the separator comment tests
Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
---------
Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
Co-authored-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
* Fix CI on develop after #5600, #5607, and #5755
- binary_reader: cast the result of -1 - number back to number_integer_t,
because a number_integer_t narrower than int is promoted to int, which
GCC's -Warith-conversion rejects (ci_test_gcc, ci_test_standards_gcc)
- JSON_DELETE_DEPRECATED_FUNCTIONS: declare the deleted stream operators
as function templates at namespace scope, because GCC < 5 rejects deleted
friend functions ("can't initialize friend function") and Clang 7-9 report
a redefinition when a class template has a deleted friend function
- docs: give the examples of JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS "Example:"
titles and add the page to the docset (style_check)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Ignore libstdc++'s <format> sign change in the sanitizer job
libstdc++ 14's <format> initializes a size_t parameter with -1 (GCC bug
119429), so every std::format call fails ci_test_clang_sanitizer under
-fsanitize=integer (test-std-format_cpp20). Exclude only the
implicit-integer-sign-change check and only that header via
-fsanitize-ignorelist.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the AppVeyor (MSVC 2015-2019) build
- binary_reader: emit_signed/emit_unsigned pass integers that do not fit
the number types to emit_float as long double, because MSVC's <cmath>
has no integer overloads of std::isfinite (C2668 'fpclassify'), from #5607
- scalar comparisons: the friend operators take the JSON type for their
noexcept from their parameter, because MSVC 2015/2017 take basic_json as
the class template there (C3203) and MSVC 2019 16.0 does not see member
types or template parameters, from #5751
- unit-conversions2: skip the !is_nothrow_constructible static_assert for
std::optional on MSVC 2017, which evaluates the conditional noexcept as
true (C2607), from #5754
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-amalgamate
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid MSVC 2015's C4800 for enums with underlying type bool
MSVC 2015 warns about any conversion to bool (C4800), even with an
explicit cast, so the enum conversions from #5754 (#5671) failed the
AppVeyor build with /WX. Convert to bool by comparing with zero via the
new detail::bool_aware_static_cast, and keep doctest from printing the
enum in the test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the #5650 allocator test on MSVC 2015 debug builds
MSVC 2015's debug STL constructs the containers' debug proxies through
the allocator in noexcept constructors, so countdown_allocator's failing
construction crashes test-allocator (SIGSEGV) instead of throwing
std::bad_alloc. Use the guard #5585 uses for the same reason.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip deleted-function detection checks on MSVC 2015
MSVC 2015 does not treat selecting a deleted function in decltype as a
substitution failure, so the detection traits in
unit-delete_deprecated_functions (#5755) and the integral-key checks in
unit-element_access2 (#5657) report deleted overloads as callable there.
Calling them still fails to compile. Skip those checks for
_MSC_VER < 1910, and use the stream operators for real in the runtime
section, so MSVC 2015 still compiles them with the macro set.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the Visual Studio 2017 AppVeyor jobs in two
The VS 2017 jobs hit AppVeyor's 60-minute limit per job while still
compiling the tests (77 of about 108 test targets after 58 minutes).
They pass /std:c++17 for everything anyway, so build only the C++17
variant of each test (JSON_TestStandards=17), and split the unit test
files across two jobs each with the new JSON_TestShard=<index>/<count>
option, which keeps every <count>-th test file starting at <index>.
The extra variants of single test files are built in shard 0 only.
CMAKE_OPTIONS is no longer quoted in appveyor.yml, so that it can hold
more than one option.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix std::terminate when converting std::optional with MSVC 2017
MSVC 2017 evaluates std::is_nothrow_assignable<json&, const T&> as true
even if T's to_json throws, so to_json(json&, const std::optional<T>&)
was noexcept there and the exception from #5642's test called
std::terminate instead of propagating. Make that conversion never
noexcept on MSVC 2017; all other compilers keep the exact condition.
The static_asserts on the condition are skipped for MSVC 2017; the
runtime check that the exception propagates still runs there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Handle numbers that do not fit narrow number types in the binary readers
With custom number types narrower than the values in a binary document,
for example basic_json<..., std::int32_t, std::uint32_t, float>, every
binary reader (CBOR, MessagePack, UBJSON, BJData, BSON, BON8) passed the
decoded number to the SAX interface with an implicit conversion: the
integer 5000000000 silently became 705032704, and a finite double such as
1e300 became infinity. The lexer handles the same values in JSON text: an
integer that fits neither integer type is stored as number_float_t, and a
finite number that overflows number_float_t is rejected with
out_of_range.406.
Pass every number read from binary input through three helpers that
apply the lexer's rules:
- emit_signed(): number_integer_t, else number_unsigned_t for a
non-negative value, else number_float_t
- emit_unsigned(): number_unsigned_t, else number_float_t
- emit_float(): out_of_range.406 if a finite value overflows
number_float_t; infinity and NaN are passed on
For consistency, a CBOR negative integer below the range of
number_integer_t is now stored as number_float_t, like a too small
integer in JSON text, instead of being rejected with parse_error.112.
With the default number types, this is the only change in behavior.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC and clang 3.5 in the narrow number type test
MSVC types 3000000000 and 5000000000 as unsigned long, so
json(-3000000000) triggered C4146 (unary minus on an unsigned type),
which /WX turns into an error. Use LL literals, as elsewhere in the
tests.
clang 3.5 cannot convert the lambdas in the braced initializer of the
format table to function pointers. Use named functions instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check integer-to-float fallbacks for overflow in the binary readers
emit_signed, emit_unsigned, and the CBOR negative integer fallback now
pass their number_float_t fallback through emit_float, so a value that
overflows number_float_t is rejected with out_of_range.406 like a
floating-point value, instead of silently becoming infinity. This only
matters for a number_float_t that cannot represent 2^64, such as a
half-precision type. The CBOR value -1 - n is computed as long double so
that emit_float sees a finite value.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the input_format member instead of passing the format to binary_reader helpers
The helpers (get_number, get_to, get_string, get_binary, get_bytes,
emit_signed, emit_unsigned, emit_float, unexpect_eof, exception_message)
are members of binary_reader, which already stores the format it was
constructed with, so the parameter was redundant.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the serializer conversion for objects whose keys cannot be converted
#5591 added a test converting nlohmann::json into a basic_json whose
string type cannot be constructed from std::string. That instantiates
convert_iteratively(), whose members.emplace_back(next.key(), ...) needs
exactly that key conversion, and broke the build of unit-alt-string.
Dispatch on the key's constructibility and leave such conversions to the
serializers, as the levels above the nesting bound already do (#3425).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the remaining CI failures on develop
- unit-wstring: with a 16-bit wchar_t (Windows), a lone surrogate is
reported as the ill-formed byte 0xFF since #5704; the std::wstring
expectations still had the previous <U+0000>.
- ci_single_binaries: json_literals.hpp (#5610) and json.hpp include each
other on purpose, and IWYU, not following the cycle, asks to replace
json.hpp with json_fwd.hpp. Report its findings without failing the
build, as already done for json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the library warnings and noexcept specifications from the merged PRs
- binary_reader: rename the error_handler constructor parameter, which
shadowed the member (-Wshadow, -Wshadow-field-in-constructor; #5746)
- basic_json(copy_construct_tag, ...): declare it noexcept when copying
the base class is (GCC 16 -Wnoexcept; #5690)
- the scalar-on-left legacy comparison operators: noexcept only when
converting the scalar is, like their member counterparts (#5682, #5751)
- compare_leaves: use std::is_eq/is_lt/is_gt instead of comparing a
std::partial_ordering with 0 (-Wzero-as-null-pointer-constant; #5686)
- serializer: silence MSVC C4127 for the EnsureAscii template parameter
(#5741, #5746)
- clang-tidy: return the sanitized reference in binary_writer, take the
key of ordered_map::find_impl by const reference (#5727), and mark the
switches over parse_array_index (#5728)
- ordered_map: keep <memory> for std::allocator (IWYU)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split unit-conversions.cpp so MinGW can link it
clang 18 with the MinGW linker failed to link test-conversions_cpp17
("relocation truncated to fit: IMAGE_REL_AMD64_REL32"). As windows.yml
recommends, keep the objects small by splitting the test file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the tests added by the merged PRs for all CI configurations
- discard the results of dump() and from_*() in CHECK_THROWS with
utils::ignore_return_value (GCC -Werror=unused-result)
- give unit-bson's huge_string_t a default constructor (MSVC C2512,
GCC 5, clang 3.5)
- unit-disabled_exceptions: use the literals namespace when the global
UDLs are off (ci_test_noglobaludls; #5700)
- unit-binary_utf8_strict: expect the JSON pointer prefix with
JSON_DIAGNOSTICS (#5741)
- skip the tests that rely on exceptions under JSON_NOEXCEPTION
(#5678, #5732)
- clang-tidy and clang -Werror: static test data, CAPTURE(...);,
const-correctness, use-after-move alias, unused conversion operator,
a missing <iterator> include
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Title the macro examples and add JSON_STRICT_BINARY_UTF8 to the docset
The documentation style check requires "Example: ..." titles on pages with several examples (#5741, #5591) and a docset entry for every macro page.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate BUILD.bazel and nlohmann_json.natvis
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5746 added detail/output/error_handler.hpp and #5741 the json_abi_sbu8 ABI tag.
* Install libidn11 for the CMake 3.5.0 binary in ci_cmake_flags
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5733 moved ci_cmake_options from ubuntu:focal to ubuntu:24.04, which no longer ships libidn.so.11; the CMake 3.5.0 release binary links against it, so every ci_cmake_flags run has failed since. Install focal's libidn11 package for that matrix entry only.
* Suppress Infer's false STACK_VARIABLE_ADDRESS_ESCAPE in get_impl
get_impl() returns its local by value. A test added by the merged PRs instantiates it with a type Infer misreads, so ci_infer reported the 2021 code for the first time.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The wide-string input adapter (used for std::u16string, std::u32string,
std::wstring, and iterators over 2- or 4-byte character types) passed
some malformed code units on to the lexer as values that are neither a
byte (0x00..0xFF) nor char_traits<char>::eof(). As a result:
- A lone UTF-16 surrogate inside true/false/null was accepted if its low
byte matched the expected letter, or ended the input silently if it
was the last unit.
- A high surrogate followed by a unit that is not its low surrogate
swallowed that unit; if the swallowed unit was the newline ending a
// comment, the comment silently extended over the next line.
- Where wint_t is a signed int (macOS, the BSDs), a negative wchar_t
collided with char_traits<char>::eof() (ending the input early) or was
truncated to its low byte, depending on its value.
The UTF-32 helper now converts the code unit to std::uint32_t before the
range checks, so a negative unit reaches the same "emit 0xFF" branch
already used for code points above U+10FFFF. The UTF-16 helper now
peeks at the next unit before consuming it, and emits 0xFF instead of
the raw surrogate when no valid pair is found, matching how ill-formed
UTF-8 bytes are rejected elsewhere in the lexer.
Fixes#5645.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove unused private aliases from basic_json
The private aliases primitive_iterator_t, internal_iterator and
output_adapter_t are not used anywhere: iter_impl, binary_writer and
the tests refer to the detail:: names directly. As the aliases are
private, no user or derived class can depend on them. The
internal_iterator.hpp include stays because iter_impl needs it.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix meta()'s dead, syntactically invalid HP aCC branch
The HP aCC branch of basic_json::meta() was missing a semicolon and
has therefore never compiled; adding only a semicolon would also make
it throw type_error.305, since it assigned a plain string to
result["compiler"] and then indexed into it like the other branches
do into an object. Make the branch consistent with the others by
assigning an object with "family" and "version" keys, narrow the
condition to __HP_aCC (a C compiler cannot build this header-only
library), and fix meta.md, which documented the old (impossible)
plain-string behavior.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop the noexcept null constructor delegating to a throwing one
basic_json(std::nullptr_t) delegated to basic_json(value_t), whose
underlying json_value(value_t) constructor allocates for other types
and can therefore throw, which is why the noexcept had a
NOLINT(bugprone-exception-escape). The delegated-to constructor also
called assert_invariant() a second time. The default member
initializers of data already produce the same null state (a
value-initialized, i.e. zeroed, union with object == nullptr), so the
delegation and its NOLINT can simply be dropped.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove four cppcheck accessForwarded suppressions in the move constructor
basic_json(basic_json&&) built its base subobject with
std::forward<json_base_class_t>(other), so cppcheck saw the whole of
other as forwarded and flagged every subsequent access to it as
accessForwarded, three of them still marked "TODO check". Only the
base subobject is actually moved from; cast explicitly to the base
type instead, the way ordered_map already does, so cppcheck can tell
the two are unrelated. Behavior is unchanged: for a non-reference T,
std::forward<T>(x) is defined as static_cast<T&&>(x).
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove stale cppcheck suppressions and name the local parser in parse()
Running the pinned cppcheck (ci_cppcheck's invocation) without
--inline-suppr across all configurations reports no syntaxError, no
ignoredReturnValue and no assertWithSideEffect, so the corresponding
suppressions in json_fwd.hpp, string_concat.hpp and
assert_invariant() no longer match anything (json_fwd.hpp's is kept,
since downstream users who run an older cppcheck against it could
still hit the warning it once silenced).
The three basic_json::parse() overloads still trigger a false-positive
accessMoved/accessForwarded because they build a temporary parser and
call .parse() on it in the same expression; giving that parser a name
makes the warning go away without changing behavior, and removes the
last of the inline suppressions on these functions.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the string/binary cleanup in the two erase() overloads
erase(pos) and erase(first, last) each carried a byte-identical
14-line block that destroys and deallocates a string or binary
value before resetting the type to null. That reimplements the
string/binary cases of json_value::destroy(), so any future change to
how those values are freed would have to be made in three places
instead of one. Both overloads now just call destroy() and reset the
union; for the other primitive types (boolean, numbers) destroy() is
a no-op, so behavior is unchanged.
Also fix erase(first, last)'s error-path branch hint, which used
JSON_HEDLEY_LIKELY where erase(pos), the iterator-range constructor,
and every other error path in the class use JSON_HEDLEY_UNLIKELY.
This only affects code layout, not semantics.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix stale and copy-pasted comments in basic_json
Several comments no longer match the code: the class invariant and
assert_invariant()'s doc still named the members m_value/m_type
(now m_data.m_value/m_data.m_type) and did not mention the binary
invariant that assert_invariant() already checks; the json_value note
and the get<PointerType>() @tparam list omitted binary_t even though
binary is a variable-length, pointer-stored type like the others; the
key-based value() overload's brief said "via JSON Pointer", which is
the other overload; and swap(binary_t&)/swap(binary_t::container_type&)
both carried "swap only works for strings", copied from swap(string_t&).
Comment-only change; behavior, the public API and the ABI are
unchanged. The private get_impl() doxygen and the emplace() comments
that border #5585's hunk are intentionally left alone.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the 16 copied from_cbor/msgpack/ubjson/bjdata/bon8/bson bodies
Each of the 16 binary deserialization overloads (from_cbor,
from_msgpack, from_ubjson, from_bjdata, from_bon8, from_bson, each in
an InputType&& and an iterator/sentinel version, plus the deprecated
span overloads of from_cbor/from_msgpack/from_ubjson/from_bson) had
the same body, differing only in the input_format_t value. Every copy
built a temporary binary_reader from std::move(ia) and called
sax_parse on it in the same expression, which also produced a
false-positive cppcheck accessMoved on all 16 lines and needed a
NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg) on
the four span overloads.
Add a private from_binary_impl() helper that builds the reader as a
named local instead, and make each of the 16 overloads a one-line
forward to it. All public signatures, default arguments,
JSON_HEDLEY_WARN_UNUSED_RESULT and JSON_HEDLEY_DEPRECATED_FOR
attributes are unchanged, tag_handler keeps defaulting to
cbor_tag_handler_t::error for the non-CBOR formats (matching
binary_reader::sax_parse's own default), and the helper is placed in
the existing private section before the binary section banner rather
than between the from_* overloads, so from_binary_impl() itself does
not collide with #5688's insertion point. Collapsing the
from_bjdata/from_bon8 bodies into one-line forwards does rewrite the
"return result; }" context lines that #5688 inserts its two
deprecated overloads after, so that PR will need a small manual
rebase (reinserting its overloads after the new one-line bodies)
rather than applying cleanly.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make insert(pos, basic_json&&) move its argument instead of copying it
insert(const_iterator pos, basic_json&& val) delegated to
insert(pos, val), but val is a named rvalue reference, so inside the
function it is an lvalue: the call always resolved to
insert(const_iterator, const basic_json&) and deep-copied the value.
This has been the case since the overload was introduced, in every
release. push_back(basic_json&&), by contrast, already moves.
Give the rvalue overload its own body with the same two checks
(type_error.309, invalid_iterator.202), then move the argument into a
local before inserting it. Moving into a local first, rather than
inserting std::move(val) directly, keeps this safe even when val
aliases an element of the same array (e.g.
arr.insert(arr.begin(), std::move(arr[1]))), since
std::vector::insert(pos, T&&) is not guaranteed to handle an argument
that aliases one of its own elements.
This is a deliberate, small behavior change: the moved-from argument
now ends up null afterwards, the same as after push_back(&&), instead
of keeping its old value unchanged. No signature changes, so the
public API and ABI are unaffected. Add unit-modifiers coverage for
the moved-from state and for self-aliasing insertion, both with and
without reallocation of the underlying array.
Part of #5724
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate object key lookup and checked at() access
at()/find()/count()/contains()/const operator[]/erase_internal() each
repeated the raw object lookup (m_value.object->find(key)), and the
six at() overloads additionally repeated the type_error.304 check and
out_of_range.401/403 throw. Route them all through two new private
helpers, object_lookup()/object_at() (plus array_at() for the index
overloads of at()), templated on the constness of the receiver so one
body serves both the const and non-const overload. count() is left
untouched, since it already goes through object_t::count() rather
than a second find().
The at(KeyType&&) overloads used to forward the same key twice: once
into object->find() and again, on the not-found path, into the
string_t() conversion for the exception message. object_at() now
forwards it only into the lookup and reuses the (unmoved) key for the
message. clang-tidy 22 (Docker silkeh/clang:22) still flags that reuse
under bugprone-use-after-move/hicpp-invalid-access-moved even with the
single forward, since it cannot see that object_t::find() (a plain
std::map or ordered_map) never actually moves from its argument; add a
NOLINTNEXTLINE with that reasoning rather than avoid the pattern.
No signature, exception id/message, or set_parent() behavior changes.
Overlaps #5689, #5705, #5606, #5687 and #5585, which touch the same
hunks; whichever of this commit and those PRs lands second will need
a small rebase.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the lookup/default/throw body of value()
The six non-deprecated value() overloads each held a full copy of the
same body: the four key-based overloads looked up the key and either
returned the found element converted to the requested type or the
default value (throwing type_error.306 if this is not an object), and
the two json_pointer overloads did the same via
ptr.get_checked_or_null(), throwing type_error.306 unless
is_structured(). Replace the duplicated bodies with two private
helpers, value_member() and value_pointee(), that return a
const basic_json* (null when not found) and do the type check/throw
once each. Every value() overload now just picks between the found
pointer's get<T>() and the default.
Same signatures, template parameters, SFINAE conditions, exception id,
message and this context on every overload.
Overlaps #5689 (routes find() through lookup_key()) and #5705 (adds a
deleted integral-key value() next to these overloads); whichever of
this commit and those PRs lands second will need a small rebase.
#5724 item 4
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the null-to-container conversion into convert_null_to()
Nine sites wrote out the same "turn a null value into an empty array
or object" logic with two different idioms: operator[](size_type),
operator[](key_type), operator[](KeyType&&) and update() set m_type
then assigned m_value.array/object directly via create<T>(), while the
three push_back() overloads, emplace_back() and emplace() set m_type
then assigned m_value = value_t::array/object (going through
json_value's converting constructor and a temporary). Both idioms end
up calling create<T>() and produce the same state, just via a
different path; both also share a latent exception-safety bug, since
m_type is written before the (possibly throwing) allocation, so a
throwing allocator leaves m_type == array/object with a null pointer
behind it, violating the class invariant and crashing on the next
access to, or destruction of, the value.
Add a private convert_null_to(value_t) helper and call it from all
nine sites. Unlike the idioms it replaces, it allocates the container
first and only then writes m_type, so a throwing allocation leaves the
value as a valid null instead of a mistyped, half-constructed one;
verified with a throwing allocator (see unit-allocator.cpp's
bad_allocator) that j["x"] = ... on a null j now stays null, and no
longer trips assert_invariant()/crashes, when create<object_t>()
throws. Same allocator usage and assert_invariant() call as before,
otherwise.
Overlaps #5585, which reorders these same nine blocks for exception
safety; whichever of this commit and that PR lands second will need a
small rebase.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Deduplicate the linear key search in ordered_map
emplace, at, erase(key), count and find each repeated the same
"for (auto it = begin(); it != end(); ++it) if (m_compare(it->first,
key)) ..." loop (15 copies across their key_type and transparent
KeyType&& overloads), and both erase(key) overloads additionally
repeated the exception-sensitive in-place reconstruction (destroy,
placement-new, pop_back) used to remove an element while keeping the
const Key non-movable.
Add two private helpers: find_impl(Self&, KeyType&&), a static member
template that runs the search once for either constness of the
receiver, and erase_at(iterator), which keeps the existing
pop_back-based reconstruction instead of switching to erase()/resize()
(which would add a DefaultInsertable requirement). Route find, at,
count, emplace, insert(const value_type&) and both erase(key)
overloads through them.
Same signatures, is_usable_as_key_type constraints and exception
messages/types.
Overlaps #5609 and #5685, which both rewrite emplace (#5609 also
touches insert and adds private members at the end of the class);
whichever of this commit and those PRs lands second will need a
rebase.
#5724 item 6
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-enable bugprone-use-after-move/hicpp-invalid-access-moved
These two checks (and portability-template-virtual-member-function)
were disabled in #4489 (November 2024) "only removed to get the CI
going". portability-template-virtual-member-function is a separate,
still-open cleanup (#5725 item 3 on its own branch) and stays
disabled here; this commit only re-enables the move/forward checks
and cleans up what they flag on this branch.
The move constructor (json.hpp) already casts to the base type
instead of forwarding the whole object (#5724 item 9), so it no
longer trips either check. The at(KeyType&&) double-forward this
check used to flag was reduced to a single forward with the
now-unforwarded reuse annotated by a NOLINTNEXTLINE in #5724 item 3's
object_at() helper (clang-tidy 22 still flags that reuse even after a
single forward; see that commit's message). What is left here:
- from_json_inplace_array_impl(), from_json_tuple_impl_base() and the
std::pair overload of from_json_tuple_impl() forwarded j into every
j.at(...) call in a pack expansion or a pair of calls. at() has no
ref-qualified overloads, so the forward was a no-op; call j.at(...)
directly.
- container_input_adapter_factory::create() forwards container twice
on purpose, into begin() and end(), so both see the same value
category and produce matching iterator types. Annotate it with
NOLINTNEXTLINE and a comment instead of changing it.
- unit-class_parser.cpp's "move constructor resets the moved-from
value to npos" test still pointed at the pre-static_cast move
constructor by line number and mentioned the cppcheck-suppress
annotation that #5724 item 9 already removed; update the comment.
No behavior change anywhere in include/. Verified with clang-tidy 22.1.8
(Docker silkeh/clang:22, --platform linux/amd64) against a TU including
json.hpp with the repo's .clang-tidy: bugprone-use-after-move and
hicpp-invalid-access-moved report nothing unsuppressed.
Overlaps #5737 (open PR for the rest of #5725 item 3: the
from_json.hpp/input_adapters.hpp cleanup above, and
portability-template-virtual-member-function), which currently keeps
both checks disabled pending this move-constructor change; whichever
of this commit and that PR lands second will need a small rebase of
.clang-tidy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make convert_null_to() take the container type as a template argument
Passing array_t or object_t instead of a value_t makes an invalid target
a compile error instead of a runtime assertion, and removes the branch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
With the default NUL handling (JSON_STRICT_NUL_HANDLING not set), a NUL
byte in the input is treated as the real end of input everywhere -
except when it immediately ends a `//` comment: scan_comment() matched
'\0' as a comment terminator like '\n', so the NUL was consumed as
part of the comment and scan() never saw it as end of input; the next
get() then kept reading past it. Multi-line comments and
JSON_STRICT_NUL_HANDLING=1 were unaffected, since there the NUL is
just part of the comment text.
Fix scan_comment() to leave the NUL unconsumed (unget()) instead of
returning it as part of the comment, so the following scan() reports
it as end of input, exactly as for a NUL anywhere else.
Fixes#5659.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The MinGW job ("clang (20.1.8)", Debug) failed to link test-msgpack_cpp17
with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against
`.rdata'". Its GNU ld (MinGW 12.2.0) cannot link an object file with
more than 32767 sections. Reproduced with clang 20.1.8 and binutils
2.40: unit-msgpack.cpp's object has 31958 sections on develop and had
33210 with #3989; binutils 2.44 links both.
The readers of from_*() never recover, but unoptimized builds still
emitted the recovery code their dead branches referenced: about 20
helpers per reader instantiation, plus the lexer's recovery functions
through the UBJSON high-precision number repair. Now:
- Errors are reported with report_error() only. The helpers that act on
the SAX parser's answer (repair_requested(), resync(), value_failed(),
close_open_containers(), and the repairs of high-precision numbers and
unsupported BSON elements) have one trivial variant for readers that do
not recover, so the recovery code is not referenced there.
- Object keys that are not strings and BSON elements whose end is lost
are marked at the error site and skipped by resync() in the reading
loops; report_repairable_error(), report_error_repairable_if(),
report_bson_element_error(), and bon8_error_repairable_if() are gone.
- is_cbor_item_head() is a free function, and the BSON element type is
formatted as on develop again.
The object now has 32638 sections and links. Behavior is unchanged
(regression tests, binary format suites, and fuzzing), and an optimized
program that only calls from_*() is now 256 bytes smaller than on
develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
clang's -Wdocumentation-unknown-command reads "\u" in a doc comment as
an unknown command, even in backticks. Two comments of the recovery code
now say "Unicode escape" instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Review and extend the documentation, and check it in CI
A review of all documentation pages found factual errors, dead links,
missing cross-references, and gaps in examples. This fixes them and adds
checks so the same problems are caught automatically.
Fixes:
- wrong signatures and version histories (operator!= C++20 member,
binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...)
- stale descriptions (number parsing since #5283, UBJSON table, SAX
example that no longer compiled, tsl::ordered_map advice)
- dead internal and external links; repology.org badges (the domain is
suspended) replaced by badges that query the registries directly
- deprecation notes link the migration guide; the guide itself fixed
Additions:
- "See also" sections, cross-references, 25 runnable examples, 12
Mermaid diagrams, new API pages for json_pointer::operator<=> and
byte_container_with_subtype::operator==/!=
- landing page, guides for untrusted input and performance
- "unreleased" badge after versions newer than the latest release
Checks:
- strict documentation build (broken links/anchors fail it); CI and
the publish workflow fetch the full history the build needs
- weekly external link check, Mermaid syntax check in CI
- check_structure.py: example titles, heading levels, alt texts,
header links, docset index coverage; its unused-example check works
again
- all examples produce the same output on every platform
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the customer links that could not be fixed
A dead link on the customers page is still the evidence of where the
use of the library was documented. Keep the original URLs of the entries
without a working replacement (Marne, Cisco Webex Desk Camera, Philips
Hue, CyberArk) and exclude exactly these URLs from the link check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Correct the duplicate-key recipe's claim about SAX positions
The SAX interface's key() receives no position either; only parse_error()
does. Also note that the recipe does not report the path to the repeated
key (see discussion #5085).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Say the library is available as a single header and mention json_fwd.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Correct documentation errors found while hunting for bugs
- patch/patch_inplace: list the JSON pointer errors parse_error.106-109
and out_of_range.402/404, and quote the actual parse_error.105 message.
- unflatten: list parse_error.106/107/108 and out_of_range.404.
- to_bson: list out_of_range.415 (binary subtype above 255) and note
that 412 and 415 are new in 3.13.0.
- to_string: state that string_t must be convertible to std::string, also
in the StringType requirements table.
- JSON Lines: a `while (input >> j)` loop also throws after the last value
for concatenated JSON values; show a loop that works for both.
- BON8: a string gets 0xFF only if nothing follows it in the message; a
string at the end of an array or object is ended by 0xFE.
- custom_string_type.hpp: add operator+=(char), which the "Always
required" list asks for (json_pointer::to_string, flatten, unflatten,
and diff did not compile), and an ADL int_to_string for diff and items.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cache the release headers with functools.lru_cache
Codacy (Pylint) flagged the mutable default argument that header() used
as its cache. functools.lru_cache keeps the same memoization without it.
The script's output is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI configuration broken by recent merges and tool updates
- gcc_flags.cmake: drop -Wexperimental-fmv-target, which GCC 16 accepts
only on aarch64; amd64 rejects it, so every GCC job failed while
checking the compiler.
- ci_get_cmake: add VERBATIM so the checksum pipeline is passed to the
shell intact (the unescaped `$'` broke the generated Makefile and
build.ninja, failing ci_cmake_flags and ci_module_cpp20); match the
SHA-256 entry case-insensitively, as CMake 3.5.0 lists the archive as
"Linux-x86_64"; and unpack with --strip-components, as that archive's
top-level directory is spelled "Linux" too.
- ci_single_binaries: compile json.hpp's TU without IWYU's --error, as
the comment above the gate already intends.
- tests: restore -Wno-deprecated-declarations for all non-MSVC compilers
(#5737 kept it for GCC only), as several tests call deprecated
functions on purpose; include thirdparty/fifo_map as SYSTEM.
- .clang-tidy: set misc-use-internal-linkage.AnalyzeTypes to false;
clang-tidy 22.1 extended the check to classes and enums and flagged
100 test helper types.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix library warnings and a JSON_DIAGNOSTICS parent bug
- binary_reader: pass integers to sax->number_integer() through
conditional_static_cast<number_integer_t>, making the existing
narrowing for a narrow number_integer_t explicit (MSVC C4244 and GCC
-Wconversion/-Warith-conversion with the int16_t test from #5694);
mark two Infer DEAD_STORE false positives with @infer-ignore.
- to_json: set the parents of an array built from a C++20 range view
after all elements are in place; a reallocating push_back moved the
earlier elements and left their parent pointers stale, failing the
JSON_DIAGNOSTICS invariant assertion.
- json.hpp: suppress MSVC C4127 for the new is_ordered_map check in
diff(), like the three existing ones; spell out std::formatter::parse's
return and iterator types for clang-tidy 22.1.
- number_parse: make the Eisel-Lemire digit counter unsigned
(GCC -Wstrict-overflow).
- string_utils: take encode_utf8's callable by const reference
(cppcoreguidelines-missing-std-forward) and drop a \u from its doc
comment (-Wdocumentation-unknown-command).
- ordered_map: include <memory> for std::allocator (cpplint).
Ran make amalgamate.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix tests failing in CI on develop
- unit-allocator: skip the #5640 test under MSVC STL iterator debugging,
where containers allocate a debug proxy in noexcept move constructors
and a failing allocation terminates; move a decrement out of an if
condition (bugprone-inc-dec-in-conditions).
- unit-conversions: expect the "(/0)" path with JSON_DIAGNOSTICS;
compare strict enums via get<>() rather than through the noexcept
operator==(ScalarType, json), which bugprone-exception-escape flags.
- unit-alt-string: suppress -Wexit-time-destructors for the strict enum
macro and misc-use-internal-linkage for its enum.
- unit-bjdata: call the static lookup functions through the type and
pass unsigned char (-Wsign-conversion on amd64).
- Mark Infer false positives with @infer-ignore in unit-diagnostics,
unit-pointer_access, unit-udt, and unit-conversions.
- Smaller clang-tidy 22.1 findings in unit-class_parser,
unit-constructor2, unit-custom-base-class, unit-locale-cpp, and
unit-noexcept.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
fhgffy
Michiel van SlobbeandMichiel van Slobbe