Compare commits

..
Author SHA1 Message Date
Niels Lohmann 99716a5ade Keep multimaps with enum keys as arrays of pairs
With JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS, is_enum_keyed_map also matched
std::multimap and std::unordered_multimap. Storing them as objects throws
type_error.318 as soon as a key occurs twice, which is the normal case for
a multimap, so such values could no longer be serialized at all once the
macro was enabled, although they are stored losslessly as arrays of
[key, value] pairs without it.

Exclude maps with non-unique keys from is_enum_keyed_map. They are
detected by insert(value_type) returning an iterator rather than a
pair<iterator, bool>. Map-like types without such an insert() are still
treated as before.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 18:24:48 +02:00
Niels LohmannandMuhammad Amir bin Mohamad Ghazaly d914e3a027 Store maps with enum keys as objects (opt-in)
Maps with enum keys, such as std::map<E, T>, are stored as arrays of
[key, value] pairs, because enums are not convertible to the string type
of object keys - even if NLOHMANN_JSON_SERIALIZE_ENUM maps them to
strings (#4378).

The new JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS macro stores them as objects
instead, converting each key with the enum's to_json. It applies to any
map-like type with enum keys (std::map with any comparator,
std::unordered_map, ...). A key that does not convert to a string throws
type_error.302, and two keys converting to the same string throw the new
type_error.318, rather than losing an entry. The macro changes the output
of inline functions, so it is part of the ABI tag (_ekmo).

Reading needs no macro: std::map and std::unordered_map with enum keys
are now also read from objects, converting each key with the enum's
from_json. That input was rejected before, and arrays of pairs are still
read, so data written either way can be read.

This supersedes #4531, which first proposed storing these maps as
objects.

Co-authored-by: Muhammad Amir bin Mohamad Ghazaly <amirghaz@umich.edu>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 17:48:58 +02:00
Niels Lohmann 1e101ecac1 Add BON8 support (#2998)
* Add BON8 support

Add to_bon8/from_bon8 and input_format_t::bon8 for BON8, a binary format
that uses the byte values that cannot begin a UTF-8 character as type
markers, so strings need no length prefix. It is the most compact of the
supported binary formats on the benchmark files.

The reader is non-recursive like the other binary readers. A string ends
at the first byte that cannot continue it, so the reader hands the one or
two bytes it reads past a string back to the value that follows. The
writer produces the canonical representation of the specification, except
for NFC normalization; its output is identical to that of the reference
implementation (HikoGUI) on all files of the test data.

The round-trip tests need the .bon8 files of json_test_data 3.2.0.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review comments

- Reuse detail::validate_one_utf8 to check strings in to_bon8; the error
  now names the first byte of the invalid sequence.
- Document that to_bon8 leaves bytes in the output adapter on an
  exception, and that string_open is only an output of write_bon8_marker.
- Explain why the pushback buffer of the BON8 reader cannot overflow.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Select the BON8 float prefix by type

get_bon8_float_prefix only depends on the type of its argument, so make
the type a template parameter instead of passing an unused value.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename a test variable that Flawfinder mistakes for read()

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the BON8 CI failures

- compare the float in write_bon8_float with number_float_t constants,
  so GCC does not warn about a float-to-double conversion
- mark check_bon8_utf8's context as used when exceptions are disabled
- choose the compact float prefix in a helper rather than with nested
  conditional operators (clang-tidy)
- use auto for the cast in the BON8 integer reader (clang-tidy)
- write the int32 minimum test values as long long literals (MSVC C4146)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Amalgamate

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Read BON8 strings in bulk from contiguous input

- copy the valid UTF-8 of a string in one step when the input is
  contiguous (twitter.json is read in 1.68 instead of 2.52 ms,
  jeopardy.json in 196 instead of 297 ms, close to CBOR and MessagePack)
- share the new valid_utf8_prefix() with the writer's UTF-8 check, which
  now skips ASCII 8 bytes at a time
- let the fuzzer check that contiguous and stream input give the same
  value or error, and test both paths in the unit tests
- clarify that a second 0xFF after a string is an empty string

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Link the BON8 functions from the other binary format pages

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Name the bulk scan flag after the input, not BON8

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Read BSON keys in bulk from contiguous input

BSON keys (and array indices) are C-style strings, which were read byte
by byte. For contiguous input they are now read up to their \x00-byte in
one step, using the same bulk_scan flag as BON8 strings: twitter.json is
read in 1.46 instead of 2.01 ms, citm_catalog.json in 2.93 instead of
3.33 ms, jeopardy.json in 182 instead of 207 ms. canada.json, whose keys
are almost all one-digit array indices, takes 2 % longer.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the BON8 CI failures of the bulk-read tests

- skip the contiguous-versus-stream tests of BON8 strings and BSON keys
  when exceptions are disabled: they catch the parse errors of invalid
  input, and without exceptions the library aborts instead
- use static_cast for the int64 test value (google-readability-casting)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the explicit basic_json instantiation into its own test file

Linking test-regression3_cpp20 with clang and MinGW failed with
"relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'",
as test-regression2 did before #5511. The explicit instantiation of
basic_json<> for #4825 compiles every member function, including the
BON8 reader and writer, into that object, and it was already close to
the limit (2,226,104 bytes on develop, 2,234,960 with BON8; clang -O1,
C++20).

Give the instantiation a file of its own: unit-regression3 is now
1,594,736 bytes and unit-explicit_instantiation 1,095,064. The new file
mentions JSON_HAS_CPP_17 and JSON_HAS_CPP_20 so it keeps being built
for the C++17 standard the regression was about.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Convert the bytes of the BON8 test strings explicitly

The str() helper constructed a std::string from a byte range, which
converts each unsigned char implicitly; -fsanitize=integer reports that
for bytes of 0x80 and above (ci_test_clang_sanitizer).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 16:56:21 +02:00
Niels Lohmann f682cd2ef1 Skip the #5515 MessagePack size tests on 32-bit platforms (#5590)
The tests fake a container size of UINT32_MAX + 1, which does not fit
into a 32-bit std::size_t: MSVC rejects the truncation (C4305/C4309
with /WX), and clang-cl wraps the size to 0 so nothing throws. Guard
them with SIZE_MAX > UINT32_MAX like the tests from #5584.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 15:58:59 +02:00
Kartikey Negi 6bd106893a Fix CBOR tag handling in cbor_tag_handler_t::store for non-binary items (#5559)
When using cbor_tag_handler_t::store, tags 0xD8-0xDB previously assumed
that the tagged item was a byte string, unconditionally attempting to
parse binary data and failing on valid CBOR documents containing tags
applied to integers, strings, arrays, or objects (such as self-describe
tag 55799).

Check whether the tagged data item is a byte string (0x40-0x5B or 0x5F).
If it is a byte string, store the subtype on the binary value as before.
Otherwise, iteratively process the tagged value in the driver loop using
item_read so that chained tags do not consume native stack space.

Part of #5316.

Signed-off-by: ReturnKartikey <kartikeynegi2000.work@gmail.com>
2026-09-27 14:28:55 +02:00
bucketbase26 98e00d22e5 Cut test suite runtime in binary roundtrips and integer sweeps (#5519)
* Cut test suite runtime in binary roundtrips and integer sweeps

The Linux CI jobs pass --no-skip, so skip() does not help there.
Parse each corpus file once in the binary roundtrip loops instead of
four times. Sample the 16-bit integer ranges with stride 7 (still hits
every low byte) and always keep the endpoints.

Also drop the 5M-node parse test to 500k, which still covers the
non-recursive destructor, and move jeopardy.json into its own skipped
test so the cheaper binary-format size checks actually run.

See #5418.

Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>

* Drop useless int32_t casts in the sampled integer loops

ci_test_gcc compiles with -Werror=useless-cast. On that compiler
int32_t is int, so static_cast<int32_t> of the loop bound is an
error. The bounds are already int, and the sampled values do not
change.

Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>

* Revert unit-binary_formats.cpp to develop and fix comment

Revert tests/src/unit-binary_formats.cpp to its develop state.
The test-case split made valgrind jobs slower instead of faster,
because the cheaper corpus files (canada/twitter/citm/sample)
now ran under valgrind where they never did before.

Fix the next_integer_sample comment: the function has no 'first'
parameter, so describe what the function actually does.

Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>

---------

Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>
2026-09-27 14:28:38 +02:00
Niels Lohmann f7972970a4 Throw instead of writing MessagePack lengths beyond UINT32_MAX (#5584)
* Throw instead of writing MessagePack lengths beyond UINT32_MAX

MessagePack stores the length of a string, binary value, array, or
object in at most 32 bits. For a larger value, to_msgpack wrote no length
at all, so the output could not be read back. It now throws
out_of_range.412, which BSON already uses for its 32-bit length fields.

The check lives in one function, so each length is written by an
if/else chain that ends in a plain else, without a condition that can
never be false. It is tested with string and binary types that report a
size beyond UINT32_MAX without allocating it, like the BSON tests do.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the CI failures of the MessagePack length check

- mark to_msgpack_length's value as used when exceptions are disabled
  (-Wunused-parameter, misc-unused-parameters)
- put "Exception safety" before "Exceptions" in to_msgpack.md, as the
  documentation style check requires
- create the test's string value from its type: constructing it from a
  beyond_uint32_string_t considers the std::filesystem::path conversion,
  which libstdc++ 10 reports as ambiguous for a class derived from
  std::string (clang 13)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Skip the MessagePack string length test for clang with libstdc++ 10

C++17 builds consider the std::filesystem::path conversion for the
string type, and with clang and libstdc++ 10 that conversion is
ambiguous for a class derived from std::string. Creating the value from
its type did not avoid it, since any basic_json with that string type
instantiates the check. The binary and ext cases are still tested there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the MessagePack string test type and its alias in one block

astyle indented the alias oddly when it had an #ifdef of its own after
the binary alias; declare it right after the string type, in the same
block.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 14:28:16 +02:00
Niels Lohmann 6178982b8d Compare unordered objects by key below the nesting bound (#5582)
* Compare unordered objects by key below the nesting bound

Values nested deeper than the nesting bound are compared without the
call stack, walking both objects entry by entry. Two equal objects of a
type that enumerates its entries in no fixed order - std::unordered_map,
say - can be walked in different orders, so they compared unequal, and
a deep copy compared unequal to its original. std::unordered_map's own
operator== does not depend on the order, which is what applies above the
bound.

Where the keys differ, equality now finds the entry by its key instead.
An ordering, and ordered_map, whose operator== compares its entries in
sequence, still decide by the key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test unordered object equality without std::unordered_map

basic_json<std::unordered_map> instantiates std::pair<const string,
basic_json> while basic_json is still incomplete. The standard does not
require std::unordered_map to support that, and libstdc++ 6 to 9 as well
as the EDG front ends of icpc and nvc++ reject it, which broke the build
of unit-comparison on those CI jobs.

The test now uses an object type derived from std::map (which, as the
default object type, works everywhere) whose comparator orders keys
ascending or descending as chosen at construction, and whose operator==
does not depend on the order of the entries - the property of
std::unordered_map the test is about.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare the test object type's entries with std::all_of

clang-tidy (readability-use-anyofallof) asked for std::all_of instead of
the loop in unordered_object_t's operator==. The entry type is spelled
out, as C++11 needs typename for base_type::value_type and C++20
reports it as redundant.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 14:21:58 +02:00
Niels Lohmann 85f8b21e1c Add tests for uncovered code paths (#5581)
* Add tests for uncovered code paths

Cover code the test suite did not reach, found from the Coveralls report
of develop and a local coverage run of HEAD:

- dump() of every kind of value below the bound of the recursive descent
  (pretty-printed objects, binary values, discarded values, scalars), and
  flushes of the escape and write buffers mid-string and mid-binary
- the iterative comparison: objects with different keys, containers that
  are a prefix of each other, and elements that cannot be ordered, each
  both at the top level and below the nesting bound
- SAX handlers that stop at any event, including the end of a nested
  container, in the BSON, CBOR, MessagePack, UBJSON and BJData readers
- from_bson/cbor/msgpack/ubjson/bjdata returning a discarded value
  through the iterator and pointer overloads
- JSON Patch, diff, merge_patch and update(..., true) on ordered_json
- smaller gaps: get_allocator(), to_ubjson/to_bjdata into a string,
  value() with an unresolvable JSON pointer, integer/float comparison
  below the integer range and with negative fractions, conversion to a
  custom binary type, std::formatter::parse on a spec without '}',
  unescape() of a lone '~', and the callback parser's start_array()

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Cover more paths that were thought unreachable

- parse_float_fast() declining malformed or inexact input, called
  directly since the lexer only passes well-formed numbers to it
- a UTF-16 high surrogate followed by a unit above the low surrogates
- self-assignment of a const_iterator
- a truncated CBOR string read through non-contiguous iterators
- serializing a long double under the de_DE locale, which undoes the
  locale's decimal point and thousands separator
- values read from a binary format carrying no diagnostic positions,
  with and without a parser callback

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the CI failures of the new coverage tests

- declare the self-assignment reference const (misc-const-correctness)
- expect the (/path) prefix that JSON_DIAGNOSTICS adds to the messages
  of the failing ordered_json patch operations

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect the byte range JSON_DIAGNOSTIC_POSITIONS adds to the patch errors

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the expected dump of the nested-object test with +=

clang-tidy (performance-inefficient-string-concatenation) reported the
chain of operator+ calls that assembled the expected indented output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare the BJData and UBJSON test outputs byte by byte

Building a std::string from the byte vector converts each byte
implicitly, which -fsanitize=integer reports for bytes of 0x80 and
above (ci_test_clang_sanitizer).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 14:19:22 +02:00
Niels Lohmann 4fa95d9810 Remove unreachable branches from the binary writer (#5583)
Coverage reported conditions in the binary writer that can never be
false, and marked the code behind them with LCOV_EXCL. Remove them
instead of excluding them:

- CBOR writes the length of a string, binary value, array, or object
  exactly like an unsigned integer, only with another major type. One
  function, write_cbor_head(), now writes both, so the integer tests
  cover every width and the four excluded 64-bit length branches are
  gone.
- A last `else if` whose condition holds for every remaining value
  (an unsigned value at most UINT64_MAX, a signed one in the range of
  int64_t) is now a plain `else`.
- Whether a signed integer fits into an int64 for UBJSON and BJData is
  decided by its type at compile time. Only an integer type wider than
  64 bits gets a range check and the high-precision fallback.
- The private get_impl(boolean_t*) was never called.

The UBJSON type prefix 'H' of an optimized container of unsigned
integers beyond the range of int64 was reachable although excluded; it
is tested now.

The output is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 14:18:46 +02:00
Dadi Reddy Sai Praneeth Reddy fe4a544c7e handled when size exceed uint32 (#5515)
* handled when size exceed uint32

Signed-off-by: dsp0redy <saipraneethreddy.dadireddy@gmail.com>

* addressed review comments

Signed-off-by: dsp0redy <saipraneethreddy.dadireddy@gmail.com>

* updated unit test

Signed-off-by: dsp0redy <saipraneethreddy.dadireddy@gmail.com>

* added amalgamation patch

Signed-off-by: dsp0redy <saipraneethreddy.dadireddy@gmail.com>

---------

Signed-off-by: dsp0redy <saipraneethreddy.dadireddy@gmail.com>
2026-09-27 14:17:32 +02:00
dependabot[bot] f422b753cc Bump the codeql-action group across 1 directory with 4 updates (#5576)
Bumps the codeql-action group with 4 updates in the / directory: [github/codeql-action/init](https://github.com/github/codeql-action), [github/codeql-action/autobuild](https://github.com/github/codeql-action), [github/codeql-action/analyze](https://github.com/github/codeql-action) and [github/codeql-action/upload-sarif](https://github.com/github/codeql-action).


Updates `github/codeql-action/init` from 4.38.0 to 4.38.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b96794f015dfd88f77b49b1c93e0fa7110f94c63...1c5b675653bb5c22dbe9b12b556ec555138e09fd)

Updates `github/codeql-action/autobuild` from 4.38.0 to 4.38.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b96794f015dfd88f77b49b1c93e0fa7110f94c63...1c5b675653bb5c22dbe9b12b556ec555138e09fd)

Updates `github/codeql-action/analyze` from 4.38.0 to 4.38.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b96794f015dfd88f77b49b1c93e0fa7110f94c63...1c5b675653bb5c22dbe9b12b556ec555138e09fd)

Updates `github/codeql-action/upload-sarif` from 4.38.0 to 4.38.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b96794f015dfd88f77b49b1c93e0fa7110f94c63...1c5b675653bb5c22dbe9b12b556ec555138e09fd)

---
updated-dependencies:
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.38.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/autobuild
  dependency-version: 4.38.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/init
  dependency-version: 4.38.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/upload-sarif
  dependency-version: 4.38.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 21:57:28 +02:00
Niels Lohmann 95e9a5931c Write BSON in linear time, without recursing per nesting level (#5553)
* Write BSON in linear time, without recursing per nesting level

to_bson() had two problems with nested values:

- It recursed once per nesting level, so a value nested deeply enough -
  100,000 levels on an 8 MiB stack - exhausted the call stack and
  terminated the process, although parse() accepts such values without
  complaint.
- BSON prefixes every document and array with its length. The writer
  computed that length by walking the entire value below it, again for
  every nested document it wrote, which made serializing O(size x depth).
  A 200-level document took 30 ms instead of 1.

Both passes are now iterative, and each length is computed exactly once:

- calc_bson_sizes() computes the length of every document and array in
  one pass, each from the lengths of its entries, into a table ordered
  the way they are written.
- write_bson_document() then writes the document, taking each length from
  the table.

Everything observable is unchanged, as a differential test against
develop confirms byte for byte:

- The same bytes are written.
- A key containing U+0000 still throws out_of_range.409 for the same
  first key, with the same diagnostics path, before anything is written.
- A document too large for BSON still throws out_of_range.412 before
  anything is written.
- A binary subtype above 255 still throws out_of_range.415 after the
  same partial output.

Only the enclosing objects and arrays are kept on a stack, so a flat
document allocates nothing for it. Measured against develop (clang -O3,
median of 201 runs): flat objects unchanged, flat arrays 37% faster (the
array length was computed twice), a nested 3,000-object document 2x
faster, a 200-level document 33x faster.

to_bson.md documented the quadratic complexity since #5334; it is linear
again.

Fixes #5392 for BSON, and #5308.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not require a default-constructible string_t in the BSON writer

GCC 4.9 and MSVC rejected the test's huge_string_t, which has no default
constructor; develop never default-constructed string_t here either.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Let the BSON index-name helper only fill its output parameter

It returned a reference to the string it filled, so callers held a second
name for index_name. Addresses review feedback.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 21:56:39 +02:00
Niels Lohmann 1e44262091 Make JSON_STRICT_NUL_HANDLING part of the ABI tag (#5560)
* Make JSON_STRICT_NUL_HANDLING part of the ABI tag

JSON_STRICT_NUL_HANDLING (#5534) changes the bodies of inline functions:
the lexer's handling of '\0' and input_adapter() for char arrays. So
translation units compiled with and without it define the same functions
differently, an ODR violation - the case the ABI tag exists for, as with
JSON_BRACE_INIT_COPY_SEMANTICS (_bics). It now appends _snul to the inline
namespace. The macro is new in 3.13.0, so no existing namespace changes.

Its default moves to abi_macros.hpp, and it is only #undef'd without
JSON_TEST_KEEP_MACROS, as for the other ABI macros. The ABI config tests,
the namespace docs, the macro's docs and the Natvis file cover the new tag.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Amalgamate

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 21:54:11 +02:00
Niels Lohmann c60a0bc336 Allocate the deep copy's key scratch space with the provided allocator (#5573)
* Allocate the deep copy's key scratch space with the provided allocator

The iterative deep copy builds each object's keys in a temporary vector of
key/value pairs before handing them to the object's range constructor. That
vector holds basic_json values, so like the values themselves it now uses
AllocatorType instead of std::allocator.

Also document that AllocatorType covers the JSON values, while most
temporary storage still uses std::allocator.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Count allocate_at_least in the scratch-counting test allocator

From C++23 on, libc++'s containers allocate through allocate_at_least when
the allocator has one. The test allocator inherited it from std::allocator,
so the scratch allocations were not counted and the test failed on Xcode.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 21:53:49 +02:00
Niels LohmannandClaude Sonnet 5 d19f7f5dce Fix BSON conformance issue (#5185)
* 🐛 fix BSON conformance issue

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* 🐛 fix BSON conformance issue

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* 🐛 reject ill-formed UTF-8 in CBOR/MessagePack/BSON text strings at decode time (#5531)

from_cbor()/from_msgpack()/from_bson() copied the raw bytes of a decoded
text string into the resulting json value without any UTF-8 validation,
even though RFC 8949 §3.1 (CBOR) and the MessagePack/BSON specifications
all require text strings to be valid UTF-8. Malformed input only failed
later, if the value was dump()'d, with a type_error.316 - so the
allow_exceptions=false pattern used specifically to get a discarded
sentinel instead of an exception did not discard this category of
malformed input, unlike every other kind of malformed binary input this
library rejects at decode time (see #5529).

Fix this at the single choke point shared by BSON/CBOR/MessagePack/UBJSON
string reads, binary_reader::get_string(): validate the bytes with the
UTF-8 DFA right after they are read, and report failures the same way as
every other binary_reader error (parse_error.113), so allow_exceptions
and strict discarding behave consistently. get_binary()/binary blob reads
are untouched and still accept arbitrary bytes, since only text strings
are required to be UTF-8.

There were two independent implementations of a UTF-8 validator: the
lexer's streaming scanner, and the serializer's Hoehrmann DFA used by
dump_escaped_impl(). Rather than write a third, the serializer's decode()
function, its utf8d table and the UTF8_ACCEPT/UTF8_REJECT constants are
extracted into detail/string_utils.hpp (a low-level header already
included before both detail/input/ and detail/output/), alongside a new
is_valid_utf8() helper built on the same decode() step. serializer.hpp's
dump_escaped_impl() now calls the shared decode(), so there is exactly
one UTF-8 validator in the codebase; dump()'s exact type_error.316
messages and byte-index reporting are unchanged (see the added
regression-guard test in unit-serialization.cpp).

Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* ⚡ validate only newly read bytes of binary-format strings

get_string() validated the whole result after each call, but get_bytes()
appends to it and CBOR indefinite-length strings collect all chunks in
the same result, so every chunk re-validated everything read before it.
An input of many small chunks took quadratic time (80000 one-byte chunks,
160 KB of input, took about 7 seconds). Only the newly read bytes are
validated now, which also matches RFC 8949's requirement that every
chunk is valid UTF-8 on its own.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-25 20:45:28 +02:00
Niels Lohmann 632a5812a8 Support zero-member types in NLOHMANN_DEFINE_TYPE_* macros (#4041) (#5272)
* Support zero-member types in NLOHMANN_DEFINE_TYPE_* macros (#4041)

NLOHMANN_DEFINE_TYPE_INTRUSIVE(Type) and its 11 sibling macros produced
broken code for types with no members to serialize. Invoking a variadic
macro so __VA_ARGS__ is empty is only standard-conforming since C++20,
so a plain __VA_OPT__ fix (as tried in #5142) breaks every pre-C++20
build under -pedantic. Instead, make all 12 macros purely variadic and
dispatch on argument count using a sentinel-padded extension of the
existing NLOHMANN_JSON_GET_MACRO idiom, giving full C++11-C++26 support
with no feature-test gate.

Verified against real GCC 16 and Clang at -std=c++11/14/17/20 with
-pedantic -Werror -Wvariadic-macros: zero regressions in the existing
unit-udt_macro.cpp suite plus 12 new zero-member test cases.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI failures in zero-member NLOHMANN_DEFINE_TYPE_* macros

Three issues surfaced on PR #5272's real CI that weren't caught by
local testing against a narrower flag set:

- GCC -Werror=noexcept: the four truly-empty from_json bodies (plain
  INTRUSIVE/NON_INTRUSIVE, with and without _WITH_DEFAULT) provably
  never throw but weren't declared noexcept; mark them noexcept
  explicitly. to_json and the derived-type from_json overloads are
  left alone since they genuinely can throw (object assignment /
  delegating to the base class's from_json).
- clang-tidy bugprone-macro-parentheses: false positive on the same
  8 zero-member bodies (Type/BaseType used purely as declarator
  types); suppressed with NOLINTNEXTLINE comments in the same style
  already used elsewhere in this file (see NLOHMANN_JSON_SERIALIZE_ENUM).
- MSVC's traditional preprocessor doesn't fully expand
  NLOHMANN_JSON_CAT(prefix, NLOHMANN_JSON_TYPE_TAG(...))(...) in one
  pass, which broke a pre-existing one-member usage in
  unit-regression2.cpp with syntax errors. Wrap all 12 public
  dispatcher macros in an extra outer NLOHMANN_JSON_EXPAND(...),
  matching the pattern NLOHMANN_JSON_PASTE already uses for the same
  MSVC quirk.

Re-verified against real GCC 16 and Clang at -std=c++11/14/17/20 with
-pedantic -Werror -Wvariadic-macros -Wnoexcept, including the exact
files that failed in CI (unit-udt_macro.cpp, unit-regression2.cpp),
against both the modular headers and the re-amalgamated single header.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy misc-const-correctness in unit-udt_macro.cpp

The four zero-member ONLY_SERIALIZE test objects are only ever read
(via to_json), never mutated, so mark them const per clang-tidy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix derived-type macro dispatch capping members at 62 instead of 63

NLOHMANN_JSON_GET_MACRO resolves 64 positional arguments, with NAME at
position 65. NLOHMANN_JSON_TYPE_TAG dispatches on Type plus the member
list, so it resolves correctly up to the 63 members NLOHMANN_JSON_PASTE
supports. NLOHMANN_JSON_DERIVED_TYPE_TAG dispatched on the two-token
Type,BaseType prefix plus the member list, running out one slot early:
at 63 members, position 65 landed on the last member name instead of a
sentinel and NLOHMANN_JSON_CAT built an undefined identifier such as
NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_m63, with the compiler
reporting "unknown type name 'm1'" once per member and nothing pointing
at an argument-count limit.

That silently reduced all six NLOHMANN_DEFINE_DERIVED_TYPE_* macros from
63 members to 62, contradicting the "up to 63 members" contract in
docs/mkdocs/docs/api/macros/nlohmann_define_derived_type.md.

Drop the leading Type and defer to NLOHMANN_JSON_TYPE_TAG so the tag is
computed from BaseType plus the member list, which fits the available
slots. The zero-own-member derived bodies are therefore selected by tag
1 rather than 2, and the sentinel table for the derived tag is no longer
needed.

Add a regression test at the documented maximum for both the plain and
the derived macros; it fails to compile against the previous dispatch.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Name the zero-member macro bodies by intent, not argument count

The dispatch tag was the literal token 1 or N, pasted onto a macro prefix
to select the zero-member or member-carrying body. For the derived-type
macros that reads wrong: their tag is computed after dropping the leading
Type, so the zero-member body was named _1 while taking two parameters
(Type, BaseType).

Emit EMPTY and MEMBERS instead. The mechanism is unchanged -- the tag is
still a token pasted onto the prefix by NLOHMANN_JSON_CAT -- but the body
names now say what they are rather than encoding an argument count that
only lines up for half of the macros.

Collapse the four duplicated zero-member bodies while here: with no
members there is nothing to default, so each _WITH_DEFAULT_EMPTY body was
a byte-for-byte copy of its plain counterpart. They are now one-line
aliases, leaving a single definition of what an empty object serializes
to per intrusive/non-intrusive and base/derived combination.

No functional change: for both zero-member and member-carrying types the
preprocessed to_json/from_json output is token-for-token identical, and
the arity limits are unchanged (63 members, base and derived).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document zero-member support in the macro API reference

docs/mkdocs/docs/features/arbitrary_types.md already gained a note, but
the three api/macros pages are where the parameter contract is actually
specified and they still described member as a non-empty list.

State that the list may be empty on each page, and add a note showing
what the zero-member case generates: an empty JSON object for the plain
macros, and base-type-only serialization for the derived ones. Both notes
record that the WITH_NAMES variants do not support this.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep user macros named EMPTY or MEMBERS out of the member-count dispatch

The dispatch produced the bare token EMPTY or MEMBERS and pasted it onto
the macro prefix afterwards. In between, the token was rescanned, so a
user macro with either name replaced it: with `#define MEMBERS x` in
scope, even NLOHMANN_DEFINE_TYPE_INTRUSIVE(A, member) -- which compiled
before -- expanded to garbage, and `#define EMPTY` broke the zero-member
form.

Paste the suffix onto the prefix directly in the GET_MACRO slot table
instead. Operands of ## are not macro-expanded, so the selected body name
is formed before any user macro can interfere. NLOHMANN_JSON_TYPE_TAG and
NLOHMANN_JSON_DERIVED_TYPE_TAG become NLOHMANN_JSON_TYPE_BODY and
NLOHMANN_JSON_DERIVED_TYPE_BODY, taking the prefix as their first
argument; NLOHMANN_JSON_CAT is no longer needed. The body macro names are
unchanged, and so is the generated code.

Add a regression test that defines EMPTY and MEMBERS around plain and
derived types, with and without members.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test for EMPTY and MEMBERS so -Wunused-macros accepts them

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 20:44:30 +02:00
Niels Lohmann ec4bdc398a Document the benchmarks, make them build and compare versions again, and pin Google Benchmark (#5556)
* Document the benchmarks, and make them build and compare versions again

The benchmark project hasn't configured since #4793: download_test_data.cmake
compiles cmake/detect_libcpp_version.cpp relative to CMAKE_SOURCE_DIR, which
is tests/benchmarks when that is the top-level project, so try_run fails and
so does `make run_benchmarks`. The path is now relative to the module itself,
which is the same file for the main build.

The Dump benchmark discarded dump()'s result, which is [[nodiscard]] by now;
it warned, and left the optimizer free to shorten the loop. The result is
now kept with benchmark::DoNotOptimize.

A new cache variable, JSON_BENCHMARK_INCLUDE_DIR, names the directory holding
the nlohmann/json.hpp to benchmark (single_include by default, as before),
so the same benchmarks can be built against two versions and compared.

tests/benchmarks/README.md documents what is measured, how to build and run
the benchmarks, how to read the output, and how to compare two versions with
Google Benchmark's compare.py; it recommends doing so by hand before a
release rather than in CI.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Point ci_benchmarks at tests/benchmarks

The target has configured ${PROJECT_SOURCE_DIR}/benchmarks since it was
added in #2561, but the benchmarks live in tests/benchmarks.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Pin Google Benchmark to release 1.9.5

The benchmarks fetched Google Benchmark's main branch, so two builds on
different days could measure with different library code, and CMake 3.30
and later warn that the single-argument FetchContent_Populate() is
deprecated. Fetch the 1.9.5 release archive, verified by its SHA-256,
with FetchContent_MakeAvailable() instead. That needs CMake 3.14; Google
Benchmark itself already needed 3.13.

Its -Werror is switched off, so a newer compiler's new warnings cannot
break the pinned release, and its install rules are no longer added.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 20:43:34 +02:00
Niels Lohmann a9ab2a62ba Cancel superseded runs of the remaining workflows (#5579)
Ubuntu, Windows, macOS, and CodeQL already cancel an older run of the same
workflow on the same ref. Check amalgamation, CIFuzz, Dependency Review,
Flawfinder, Semgrep, Scorecard, and the labeler did not, so every push
to a pull request left their earlier runs going. Give them the same
concurrency group. The labeler runs on pull_request_target, where
github.ref is the base branch, so it groups by pull request number.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 20:40:28 +02:00
Niels Lohmann ad2c14b985 Document response times, supported versions, access, secrets, and dependency policies (#5580)
Answer the OpenSSF Best Practices criteria that asked for policies the
project follows but had not written down:

- SECURITY.md: a first response within 14 days, publishing an advisory
  with credit once a fix is released, and that only the latest release
  receives security fixes.
- Governance: who has access to the project's resources, how write or
  admin access is granted, and how CI secrets are stored and rotated.
- Quality assurance: how dependencies of the build, test, and
  documentation tooling are pinned, scanned, and kept free of known
  vulnerabilities.

Also update the assurance case, since comparison no longer recurses per
nesting level (#5390), and point the best practices badge and links to
bestpractices.dev under the program's current name.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 20:34:58 +02:00
Niels Lohmann e1310ad43c Fix CI: resolve clang-tidy findings in the stream position tests (#5578)
#5344 added two lines to unit-deserialization.cpp that clang-tidy
reports: modernize-return-braced-init-list for the remaining() helper
and readability-isolate-declaration for "json j1, j2, j3;".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 20:19:43 +02:00
Alexander LaninandNiels Lohmann 465407f3ce Improve error message for const fields (#2818)
* Improve error message for const fields

* Reject const arguments to get_to() with a clear message

Reword the static_assert, add it to the C array overload of get_to() as well,
and document that v must not be const.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 18:02:38 +02:00
Niels Lohmann 02dd3e67f2 Fix stack overflow and exponential runtime when comparing nested values (#5390)
* Compare values without recursing, and without comparing them twice

Comparing two values compared their containers, which compare their elements,
which brought the comparison back once per nesting level. Two values nested
deeply enough exhausted the call stack and terminated the process with a
segmentation fault - the same bug as #5387, in the last operation that still
had it.

Worse, an ordered comparison took exponentially long in the nesting depth
before C++20. std::vector's operator< is a lexicographical comparison, which
asks whether an element is less than its counterpart and then whether the
counterpart is less than it - two full comparisons of everything below that
element, at every level. Comparing two equal values nested 30 levels deep,
which is nothing unusual, took 3.8 seconds; 40 levels would have taken an
hour, and nothing about the value has to be pathological to get there. C++20
is unaffected: std::lexicographical_compare_three_way asks once.

Compare a value that is nested too deeply to descend into on an explicit
stack instead, in a single pass that yields less, equal, greater or unordered
at once. Equality and the three-way comparison descend as they always did for
the first 128 levels, which nothing measurable costs them; an ordered
comparison no longer descends at all, which is what takes the exponent out of
it. Objects and arrays that are not nested deeply are otherwise compared
exactly as before.

The results are unchanged for every pair of values: 68121 comparisons of a
corpus that covers NaN, discarded values, mixed number types, binary values,
empty containers and both object types are identical to develop, in C++11,
C++17 and C++20, with and without thread_local storage and legacy discarded
comparison. Reproducing that meant reproducing two subtleties: a lexicographic
comparison steps over a pair it cannot order, where a three-way comparison
stops at it, and an object compares its keys with < where its entries are
ordered but with == where they are only checked for equality - not with the
object's own comparator, which for nlohmann::ordered_map tells equality.

Equality needs no ordering, so it no longer asks for any: a key or string type
that can only be compared for equality still works.

Measured (medians of 7 interleaved runs, clang -O3, C++11): comparing two
equal values nested 30 levels deep 3778 ms -> 0.002 ms; ordering flat objects
-33.6%; ordering flat arrays of numbers +27.3%, the one shape that pays for
the single pass; equality unchanged throughout.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Describe comparison in the no-thread-local docs and CI target

Comparing two values now bounds its descent with a thread_local counter
just as copying does, so the JSON_NO_THREAD_LOCAL page, the macro
overview and the ci_test_no_thread_local target cover both rather than
copying alone.

Also record what switching the macro on costs a comparison: on the
benchmark documents, comparing two equal values takes 10% to 90% longer.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Take the descent flag as an argument rather than testing it

MSVC reports the test of a constant as C4127 ("conditional expression is
constant"), which the Windows builds treat as an error: may_descend is
false for operator<, so the operand short-circuits the whole condition.

Passing it to compare_descent_exhausted() puts the test where the value
is an ordinary parameter, and leaves the call sites with no condition of
their own.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Note the comparison fallback in the no-thread-local documentation

The macro page describes what the library defines JSON_NO_THREAD_LOCAL for
by itself in terms of copying alone; comparing falls back the same way.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Parenthesise the reserve() computation in the comparison test

clang-tidy reports the mixed * and + as readability-math-missing-
parentheses, as it does for the identical line in the copy test.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Use the shared descent bookkeeping rather than a second set

Comparing kept a thread_local count, a limit and a guard of its own beside
the ones copying already had, all three the same thing under a different
name. They are gone; the shared count, limit and guard do the work.

The guard grows a second constructor here, because the comparison
operators are written as a macro and a macro cannot use the preprocessor:
it cannot look the count up behind an #ifdef the way copy_structured does,
so the guard looks it up for it. nesting_depth_exhausted() arrives for the
same reason - whether an operator descends at all is a constant at every
call site, and testing it there is what MSVC reports as C4127.

Also say in compare_leaves what happens to a pair that is an array on one
side and an object on the other, since the answer is not obvious from the
code: an operator only descends into two values of the same type, so such
a pair is told apart by its types alone - unequal, and ordered the way the
types are - exactly as it is above the bound.

And record what the explicit stack costs: the comparison operators are
noexcept and the container comparison this replaces allocated nothing, so
running out of memory here ends the process instead of throwing. It takes
a value nested past the bound and an exhausted heap to reach, and the same
comparison used to exhaust the call stack, but it is a new way to fail.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Amalgamate

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 17:58:59 +02:00
Niels Lohmann abbe52d6de Add JSON_PRECISE_STREAM_POSITION to leave the character that terminates a number in the stream (#5344)
* docs: qualify the operator>> stream positioning guarantee

operator>>'s notes state that it leaves the stream positioned right
after the parsed value, so that concatenated JSON values can be read
back to back. That does not hold when the value is a number: a number
is only terminated by the character that follows it, and the lexer's
unget() is simulated (it rewinds only the lexer's own bookkeeping),
so that character stays consumed from the stream.

Document the actual behaviour: the guarantee holds for all value types
except numbers, which must be followed by whitespace. Also qualify the
cross-reference on the JSON Lines page, which repeated the unqualified
claim.

Documentation only; the behaviour itself is tracked in #5340.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* fix: restore the character that terminates a number (#5340)

operator>> is documented to leave the stream positioned right after the
parsed value, so that concatenated JSON values can be read back to back.
That did not hold for numbers: a number is only terminated by the
character following it, and lexer::scan_number() reads that character
and calls unget() -- which is simulated and rewinds only the lexer's own
bookkeeping. input_stream_adapter consumes via sbumpc() with no matching
sungetc(), so the terminating character stayed consumed and the next
extraction started one byte too late ('1true' left the stream at 'rue').

Propagating unget() to the adapter directly does not work: next_unget
makes the following get() replay the cached character, so the terminator
would be delivered twice. Instead, restore the still-pending character
once at the end of a non-strict parse, where the input is handed back to
the caller:

- input_stream_adapter gains unget_character() (sungetc()) and advertises
  it via supports_unget, detected the same way as supports_seek.
- lexer::restore_pending_unget() turns a pending simulated unget of a
  real (non-EOF) character into a real one and clears next_unget so the
  character is not also replayed. It is a no-op for adapters that cannot
  unget, and reports failure when sungetc() fails, in which case the
  input is left as it was before.
- parser calls it on the three non-strict paths, i.e. for operator>> and
  sax_parse(strict = false).

Strict parse()/accept() are unaffected: they require the input to end
after the value, so the character is consumed by the end-of-input check
anyway. Parse error messages and reported positions are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* tests: fix CI failures in the #5340 test helpers

Four CI failures, all in the new test code:

- GCC (-Werror=useless-cast): drop the `json(...)` wrapper around
  `json::parse(...)`, which already returns a `json`.
- GCC (-Werror=unused-result): assign the discarded `json::parse()`
  result to a dummy, the idiom used elsewhere in the test suite, and
  catch `json::parse_error&` for consistency.
- clang-tidy (google-default-arguments): remove the default argument
  from the `pbackfail()` override; `sungetc()` supplies the base
  declaration's default.
- MSVC (bad allocation): `no_putback_streambuf::underflow()` set a
  one-character get area without advancing `m_pos`, so an implementation
  whose `istream::get` peeks before it bumps re-read the same character
  forever. Keep no get area at all: `underflow()` peeks, `uflow()`
  consumes, and `sungetc()` still always lands in `pbackfail()`, which
  is what the test needs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* fix: leave the character that terminates a number in the input

Read the character following a number without consuming it, instead of
consuming it and putting it back. input_stream_adapter now peeks with
sgetc() and only steps over the character when the next one is requested
or when the adapter is destroyed, so releasing it cannot fail - no
putback position is required from the streambuf.

Suggested by gregmarr in #5344.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: match the version history wording to the peek-based fix

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: drop the whitespace-separator caveat from the parsing pages

The caveat added in #5343 describes the behavior this branch fixes: a
number no longer consumes the character that terminates it, so
concatenated values need no separator.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* refactor: split the strict and non-strict paths in parser

Folding the release_lookahead() call into the existing strict check left
the "in strict mode" comment on an else-if branch, and made the strict
condition in sax_parse() redundant with the branch it followed.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Put the stream position fix behind JSON_PRECISE_STREAM_POSITION

Leaving the character that terminates a number in the stream is observable:
reading "1,2,3" with repeated operator>> works today only because the comma
after each number is swallowed, and std::getline after a number skips the
line break. Both break with the fix, so make it opt-in for 3.x, as suggested
by @gregmarr in the review.

- JSON_PRECISE_STREAM_POSITION (default 0) selects the peek-based
  input_stream_adapter. Without it, the adapter is the consuming one from
  develop and has no supports_lookahead, so lexer::release_lookahead() and
  the parser's calls to it compile to nothing.
- The macro changes input_stream_adapter's layout and member functions, so
  it gets the ABI tag _psp, after _bics. The ABI config tests, the natvis
  generator, and nlohmann_json.natvis (regenerated) know the tag.
- The tests for the fix move to unit-precise-stream-position.cpp, which
  defines the macro itself and runs in every build, and gain the two cases
  above. unit-deserialization.cpp pins the default behavior instead.
- The docs describe the default behavior again and point to the new macro
  page; version history says "added in 3.13.0, planned default in 4.0.0".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 17:56:18 +02:00
Niels Lohmann 98278dc3f6 Fix CI: disable MSVC warning C5285 for the vendored doctest (#5577)
The windows-11-arm runner now ships MSVC 19.51, which reports doctest's
forward declaration of std::tuple as C5285 ("cannot declare a
specialization for 'std::tuple'"). With /WX this breaks the msvc-arm64
job on develop and on every open pull request. Disable the warning for
the test targets, like the other MSVC warnings already disabled there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 17:54:40 +02:00
Jeremy Nimmer 6c8ea0a6d1 Remove Bazel alwayslink=True (#5376)
This should have no effect for header only libraries as mentioned.

It was previously removed in e509007d but then accidentally added
again in 26cfec34.

Signed-off-by: Jeremy Nimmer <jeremy.nimmer@tri.global>
2026-09-25 08:46:19 +02:00
Niels Lohmann 01b53c8c15 Keep JSON_DIAGNOSTICS parent pointers of ordered_json members after erase() and update() (#5552)
* Keep JSON_DIAGNOSTICS parent pointers of ordered_json members after erase() and update()

ordered_json stores its members in a vector, and two operations moved
members without restoring their parent pointers afterwards:

- ordered_map::erase() re-constructs every member after the erased one in
  place. The basic_json move constructor leaves m_parent at nullptr, and
  none of the object branches of basic_json::erase() (by key, iterator, or
  iterator range) called set_parents(). This also affected merge_patch()
  with a null member and patch() with a remove operation.
- update() only set the parent pointer of the inserted member. Adding a key
  can reallocate the vector, which copies all other members and leaves
  their m_parent at nullptr. The set_parents() call added for #4813 only
  repaired this for the nested object of a merge, not for the target.

The next assert_invariant() on such an object (for instance, when copying
it) aborted, and diagnostic messages lost the path prefix above the moved
member. std::map-based json was not affected, because its nodes do not
move.

Erasing from an ordered_map object now calls set_parents(), and update()
uses set_parent(), which already refreshes all members for vector-based
objects. This makes the #4813 workaround redundant.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Account for JSON_DIAGNOSTIC_POSITIONS in the ordered_json parent-pointer test

The merge_patch() case parses its input, so with JSON_DIAGNOSTIC_POSITIONS
the exception message also carries the byte range of the parsed value.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Silence clang-tidy for the intentional copy in the ordered_json parent-pointer test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep parent pointers when update() merges past its descent bound

The iterative path of update() only set the parent pointer of the member
it inserted, like the recursive one did before. It now uses set_parent()
too, so ordered_json members that move when a nested object grows keep
their parents, and the set_parents() calls that patched this up after
each nested merge are gone.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:29:36 +02:00
Niels Lohmann cc472af13f Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests (#5569)
* Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests

The strongest correctness checks for the UBJSON and BJData writers lived
only in the OSS-Fuzz drivers: anything from_ubjson()/from_bjdata()
returns must serialize with every option combination, parse back, and
re-serialize stably. Those checks only run at OSS-Fuzz, so regressions
surfaced days later as external reports - the same BJData assert pair
was reported five times over three years, and #5494's harness change
was followed by OSS-Fuzz 563659413 within a day.

Add "UBJSON round-trip invariants" and "BJData round-trip invariants"
test cases that run the drivers' checks on a fixed, deterministic corpus
(tests/src/round_trip_corpus.hpp): integer and float boundaries,
non-finite numbers, strings, binary values, optimized containers, deep
nesting, the JData annotated-array matrix, and seeded random containers.
They also check two properties the drivers do not: the first round trip
preserves the value, and re-serializing reproduces the exact bytes. For
BJData both exclude values containing a binary value, which is read back
as an array of integers unless it was written as a Draft 3 optimized
binary array; this carve-out is now documented in bjdata.md. Run against
the headers before #5542, the BJData test fails, including on the shape
from OSS-Fuzz 563659413.

Also document how OSS-Fuzz reports are handled (reference them as
"OSS-Fuzz: <id>", turn the reproducer into a unit test, keep drivers and
unit tests in sync) in tests/fuzzing.md, and link it from the PR
template and the quality assurance page.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add the OSS-Fuzz reproducers for 474400817 and 474480402 as unit tests

Following the convention added to tests/fuzzing.md, the reproducers of
the two BJData fuzzer asserts tracked since January are now unit tests:

- 474400817 (assert(false)): an empty object _ArraySize_ was written as
  the ND-array header length, which from_bjdata() could not read back.
  Fixed by #5455.

- 474480402 (to_bjdata(j2, false, false) == vec2): a one-byte Draft 3
  binary array is written in Draft 2 mode as a uint8 array and then
  re-serialized with the int8 marker. This is the documented exception to
  byte stability, not a library bug; OSS-Fuzz closed it after #5494
  relaxed the harness to value stability. The test pins the exact bytes
  so the exception stays deliberate.

The 563659413 reproducer is already a unit test (#5542). A comment also
ties the existing UBJSON excessive-count test to the timeout OSS-Fuzz
reported for that shape (testcase 6347769435193344).

OSS-Fuzz: 474400817
OSS-Fuzz: 474480402

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix GCC -Weffc++ and -Wuseless-cast warnings in the round-trip corpus

Initialize the atoms in the member initialization list, and drop the cast of
the generator's result, which already is std::size_t on 64-bit Linux.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:29:02 +02:00
Niels Lohmann 80bf54a5a2 Add a security assurance case to the documentation (#5572)
Describe the threat model, the trust boundaries, the secure-design
argument, and how common weaknesses are countered, with links to the
quality assurance page as evidence.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:28:34 +02:00
Niels Lohmann aada27405d Add a roadmap page to the documentation (#5571)
Describe what the project will and will not do over the next year,
and point to issue #3453 for the open question of a 4.0 release.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:28:16 +02:00
Niels Lohmann 3901b223e5 Complete the architecture documentation page (#5570)
* Complete the architecture documentation page

Replace the placeholder bullets and TODOs with a description of the
component pipeline (with a diagram), the source layout, the template
parameters, the value storage (now in struct data), the input and
output adapters, and the SAX interface.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Link sources and basic_json, document full input adapter interface

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Align the default column of the template parameter table

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:28:00 +02:00
Niels Lohmann f290b36ad2 Fix CI: use VS 2026 on windows-11-arm and use raw string literals in tests (#5575)
The windows-11-arm runner image moved to windows-11-vs2026-arm64, which no
longer ships Visual Studio 2022, so the msvc-arm64 job failed at configure
time. Use the "Visual Studio 18 2026" generator like the msvc2026 job.

clang-tidy's modernize-raw-string-literal check flagged two string literals
in the nesting tests added by #5546 and #5547.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:23:53 +02:00
Niels Lohmann 4daca40d7b Merge deeply nested objects without recursing per nesting level (#5547)
* Merge deeply nested objects without recursing per nesting level

merge_patch() and update(j, true) merged a nested object by calling
themselves on it, once per nesting level. A value nested deeply enough -
50,000 levels of objects on an 8 MiB stack - exhausted the call stack
and terminated the process, although parse() accepts such values without
complaint.

Bound the descent the same way dump() does. The recursion now carries
the nesting level, and once merge_depth_limit() (128) levels have been
entered, update_members_iteratively() and merge_patch_iteratively()
finish the merge on an explicit stack. They still merge a nested object
completely before the next member, and in the same order, so the results,
including the parents JSON_DIAGNOSTICS reports paths from, are unchanged.
Values nested less deeply than the bound run the same code as before, so
the common case does not pay for the stack: merging only on it cost
10-14% in a first version.

The public signatures are unchanged. The recursive worker behind
merge_patch() has its own name rather than being a private overload, so
that &basic_json::merge_patch stays unambiguous.

Tests check every depth up to 300 against recursive reference
implementations of both operations, check the diagnostic paths past the
bound, and merge objects nested 100,000 levels deep.

Fixes #5545 for update(j, true), and #5393 for merge_patch().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Use the shared recursion limit in update() and merge_patch()

merge_depth_limit() is gone in favor of detail::recursion_depth_limit().
The two identical function-local frame structs become one member struct,
merge_frame, with a constructor, so both loops emplace_back() their
frames. merge_patch_iteratively() copies the frame it works on out of the
stack and changes it only through stack.back().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the update()/merge_patch() diagnostics test values instead of parsing them

Parsed values carry byte positions under JSON_DIAGNOSTIC_POSITIONS, which
the expected messages do not include.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 17:12:03 +02:00
Niels Lohmann 7c90ec2323 Hash deeply nested values without recursing per nesting level (#5546)
* Hash deeply nested values without recursing per nesting level

std::hash<basic_json> hashed an array or object by hashing each element,
which called detail::hash again once per nesting level. A value nested
deeply enough - 50,000 levels of objects on an 8 MiB stack - exhausted
the call stack and terminated the process. parse() accepts such values
without complaint, since the parser is iterative, and a parsed value is
hashed wherever it is used as a key in an unordered container.

Bound the descent the same way dump() does: detail::hash takes the
nesting level, and once hash_depth_limit() (128) levels have been entered,
hash_iteratively() hashes what is left on an explicit stack. It combines
the seeds in exactly the same order, so hash values are unchanged. A value
nested less deeply than the bound is hashed by the same code as before,
without allocating, and is as fast as before.

Tests check that every depth up to twice the bound hashes exactly like
the recursive definition of the hash, and that values nested 100,000
levels deep hash without crashing.

Fixes #5545 for std::hash.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Declare hash_frame's constructor noexcept

GCC's -Wnoexcept (an error in CI) flags the emplace_back() into the
hash stack under C++26: the constructor cannot throw, since cbegin() is
noexcept, but it did not say so. dump_frame's constructor is noexcept
for the same reason.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Share one recursion depth limit, and copy the hash frame out of the stack

dump() and hash() each defined their own limit on how many nesting levels
they recurse into, and the operations still to come would have added more,
free to diverge over time. They now all use detail::recursion_depth_limit(),
in a header of its own; serializer::dump_depth_limit() and
hash_depth_limit() are gone.

hash_iteratively() now copies the frame it works on out of the stack and
changes the frame only through stack.back(), so nothing can refer into
the stack after entering an element has grown it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Parenthesize multiplications in the hash test for clang-tidy

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 17:12:02 +02:00
146 changed files with 17609 additions and 1227 deletions
+1
View File
@@ -2,6 +2,7 @@
- [ ] The changes are described in detail, both the what and why.
- [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
- [ ] If applicable, a fixed [OSS-Fuzz](https://issues.oss-fuzz.com) issue is referenced as `OSS-Fuzz: <id>` (see [fuzz testing](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports)).
- [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
- [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
- [ ] The source code is amalgamated by running `make amalgamate`.
+14 -3
View File
@@ -9,12 +9,23 @@ identified a security vulnerability in this repository, please use the GitHub Se
Until it is published, this draft security advisory will only be visible to the maintainers of this project. Other
users and teams may be added once the advisory is created.
We will send a response indicating the next steps in handling your report. After the initial reply to your report, we
will keep you informed of the progress towards a fix and full announcement and may ask for additional information or
guidance.
We will send a first response within 14 days, indicating the next steps in handling your report. After the initial
reply to your report, we will keep you informed of the progress towards a fix and full announcement and may ask for
additional information or guidance.
For vulnerabilities in third-party dependencies or modules, please report them directly to the respective maintainers.
## Disclosure and credit
Once a fix is released, we publish the security advisory and list the fixed vulnerability in the release notes. We
credit the reporter in both, unless they ask not to be named.
## Supported versions
Security fixes are made on the `develop` branch and shipped with the next release. Only the latest release receives
security fixes; they are not backported to older releases. A release stops receiving security fixes when the next
release is published, so please update to the latest release to get them.
## Unofficial packages
This project does not publish an official npm package. The npm package
+4 -4
View File
@@ -37,13 +37,13 @@ labels:
files:
- "include/nlohmann/detail/input/binary_reader\\.hpp"
- "include/nlohmann/detail/output/binary_writer\\.hpp"
- "tests/src/unit-(bson|cbor|msgpack|ubjson|bjdata|binary_formats)"
- "tests/src/fuzzer-parse_(bson|cbor|msgpack|ubjson|bjdata)"
- "tests/src/unit-(bson|cbor|msgpack|ubjson|bjdata|bon8|binary_formats)"
- "tests/src/fuzzer-parse_(bson|cbor|msgpack|ubjson|bjdata|bon8)"
- "docs/mkdocs/docs/features/binary_formats/"
- "docs/mkdocs/docs/(api/basic_json|examples)/(to|from)_(bson|cbor|msgpack|ubjson|bjdata)"
- "docs/mkdocs/docs/(api/basic_json|examples)/(to|from)_(bson|cbor|msgpack|ubjson|bjdata|bon8)"
- label: "aspect: binary formats"
title: "(?i)(bson|cbor|msgpack|messagepack|ubjson|bjdata|binary format)"
title: "(?i)(bson|cbor|msgpack|messagepack|ubjson|bjdata|bon8|binary format)"
- label: "python"
files:
+4
View File
@@ -3,6 +3,10 @@ name: "Check amalgamation"
on:
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
+4
View File
@@ -1,6 +1,10 @@
name: CIFuzz
on: [pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
+3 -3
View File
@@ -38,14 +38,14 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/init@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
with:
languages: c-cpp
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/autobuild@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/analyze@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
+4
View File
@@ -9,6 +9,10 @@
name: 'Dependency Review'
on: [pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
+5 -1
View File
@@ -5,6 +5,10 @@
name: flawfinder
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
@@ -43,6 +47,6 @@ jobs:
output: 'flawfinder_results.sarif'
- name: Upload analysis results to GitHub Security tab
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/upload-sarif@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
with:
sarif_file: ${{github.workspace}}/flawfinder_results.sarif
+6
View File
@@ -4,6 +4,12 @@ on:
pull_request_target:
types: [opened, synchronize]
# pull_request_target runs on the base branch, so github.ref would put all pull
# requests into one group; group by pull request number instead
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
+5 -1
View File
@@ -14,6 +14,10 @@ on:
push:
branches: ["develop"]
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
@@ -76,6 +80,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/upload-sarif@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
with:
sarif_file: results.sarif
+5 -1
View File
@@ -19,6 +19,10 @@ on:
schedule:
- cron: '23 2 * * 4'
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
@@ -61,7 +65,7 @@ jobs:
# Upload SARIF file generated in previous step
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/upload-sarif@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
with:
sarif_file: semgrep.sarif
if: always()
+2 -2
View File
@@ -124,11 +124,11 @@ jobs:
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Run CMake (Release)
run: cmake -S . -B build -G "Visual Studio 17 2022" -A ARM64 -DJSON_BuildTests=On -DCMAKE_CXX_FLAGS="/W4 /WX"
run: cmake -S . -B build -G "Visual Studio 18 2026" -A ARM64 -DJSON_BuildTests=On -DCMAKE_CXX_FLAGS="/W4 /WX"
if: matrix.build_type == 'Release'
shell: pwsh
- name: Run CMake (Debug)
run: cmake -S . -B build -G "Visual Studio 17 2022" -A ARM64 -DJSON_BuildTests=On -DJSON_FastTests=ON -DCMAKE_CXX_FLAGS="/W4 /WX"
run: cmake -S . -B build -G "Visual Studio 18 2026" -A ARM64 -DJSON_BuildTests=On -DJSON_FastTests=ON -DCMAKE_CXX_FLAGS="/W4 /WX"
if: matrix.build_type == 'Debug'
shell: pwsh
- name: Build
-1
View File
@@ -71,7 +71,6 @@ cc_library(
],
includes = ["include"],
visibility = ["//visibility:public"],
alwayslink = True,
)
cc_library(
+9
View File
@@ -36,6 +36,7 @@ all:
@echo "clean - remove built files"
@echo "doctest - compile example files and check their output"
@echo "fuzz_testing - prepare fuzz testing of the JSON parser"
@echo "fuzz_testing_bon8 - prepare fuzz testing of the BON8 parser"
@echo "fuzz_testing_bson - prepare fuzz testing of the BSON parser"
@echo "fuzz_testing_cbor - prepare fuzz testing of the CBOR parser"
@echo "fuzz_testing_msgpack - prepare fuzz testing of the MessagePack parser"
@@ -71,6 +72,14 @@ fuzz_testing:
find tests/data/json_tests -size -5k -name *json | xargs -I{} cp "{}" fuzz-testing/testcases
@echo "Execute: afl-fuzz -i fuzz-testing/testcases -o fuzz-testing/out fuzz-testing/fuzzer"
fuzz_testing_bon8:
rm -fr fuzz-testing
mkdir -p fuzz-testing fuzz-testing/testcases fuzz-testing/out
$(MAKE) parse_bon8_fuzzer -C tests CXX=afl-clang++
mv tests/parse_bon8_fuzzer fuzz-testing/fuzzer
find tests/data -size -5k -name *.bon8 | xargs -I{} cp "{}" fuzz-testing/testcases
@echo "Execute: afl-fuzz -i fuzz-testing/testcases -o fuzz-testing/out fuzz-testing/fuzzer"
fuzz_testing_bson:
rm -fr fuzz-testing
mkdir -p fuzz-testing fuzz-testing/testcases fuzz-testing/out
+14 -6
View File
@@ -17,7 +17,7 @@
[![GitHub Downloads](https://img.shields.io/github/downloads/nlohmann/json/total)](https://github.com/nlohmann/json/releases)
[![GitHub Issues](https://img.shields.io/github/issues/nlohmann/json.svg)](https://github.com/nlohmann/json/issues)
[![Average time to resolve an issue](https://isitmaintained.com/badge/resolution/nlohmann/json.svg)](https://isitmaintained.com/project/nlohmann/json "Average time to resolve an issue")
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/289/badge)](https://bestpractices.coreinfrastructure.org/projects/289)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/289/badge)](https://www.bestpractices.dev/projects/289)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/nlohmann/json/badge)](https://scorecard.dev/viewer/?uri=github.com/nlohmann/json)
[![Backup Status](https://app.cloudback.it/badge/nlohmann/json)](https://cloudback.it)
[![GitHub Sponsors](https://img.shields.io/badge/GitHub-Sponsors-ff69b4)](https://github.com/sponsors/nlohmann)
@@ -40,7 +40,7 @@
- [Implicit conversions](#implicit-conversions)
- [Conversions to/from arbitrary types](#arbitrary-types-conversions)
- [Specializing enum conversion](#specializing-enum-conversion)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)](#binary-formats-bson-cbor-messagepack-ubjson-and-bjdata)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, BJData, and BON8)](#binary-formats-bson-cbor-messagepack-ubjson-bjdata-and-bon8)
- [Customers](#customers)
- [Ecosystem](#ecosystem)
- [Supported compilers](#supported-compilers)
@@ -63,7 +63,7 @@ There are myriads of [JSON](https://json.org) libraries out there, and each may
- **Trivial integration**. Our whole code consists of a single header file [`json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp). That's it. No library, no subproject, no dependencies, no complex build system. The class is written in vanilla C++11. All in all, everything should require no adjustment of your compiler flags or project settings. The library is also included in all popular [package managers](https://json.nlohmann.me/integration/package_managers/).
- **Serious testing**. Our code is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/tests/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](https://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests against all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [Core Infrastructure Initiative (CII) best practices](https://bestpractices.coreinfrastructure.org/projects/289). See the [quality assurance](https://json.nlohmann.me/community/quality_assurance) overview documentation.
- **Serious testing**. Our code is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/tests/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](https://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests against all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [OpenSSF Best Practices](https://www.bestpractices.dev/projects/289). See the [quality assurance](https://json.nlohmann.me/community/quality_assurance) overview documentation.
Other aspects were not so important to us:
@@ -128,7 +128,7 @@ There is also a [**docset**](https://github.com/Kapeli/Dash-User-Contributions/t
- **JSON Pointer functions**: [flatten](https://json.nlohmann.me/api/basic_json/flatten), [unflatten](https://json.nlohmann.me/api/basic_json/unflatten)
- **JSON Patch functions**: [patch](https://json.nlohmann.me/api/basic_json/patch), [patch_inplace](https://json.nlohmann.me/api/basic_json/patch_inplace), [diff](https://json.nlohmann.me/api/basic_json/diff), [merge_patch](https://json.nlohmann.me/api/basic_json/merge_patch)
- **Static functions**: [meta](https://json.nlohmann.me/api/basic_json/meta), [get_allocator](https://json.nlohmann.me/api/basic_json/get_allocator)
- **Binary formats**: [from_bjdata](https://json.nlohmann.me/api/basic_json/from_bjdata), [from_bson](https://json.nlohmann.me/api/basic_json/from_bson), [from_cbor](https://json.nlohmann.me/api/basic_json/from_cbor), [from_msgpack](https://json.nlohmann.me/api/basic_json/from_msgpack), [from_ubjson](https://json.nlohmann.me/api/basic_json/from_ubjson), [to_bjdata](https://json.nlohmann.me/api/basic_json/to_bjdata), [to_bson](https://json.nlohmann.me/api/basic_json/to_bson), [to_cbor](https://json.nlohmann.me/api/basic_json/to_cbor), [to_msgpack](https://json.nlohmann.me/api/basic_json/to_msgpack), [to_ubjson](https://json.nlohmann.me/api/basic_json/to_ubjson)
- **Binary formats**: [from_bjdata](https://json.nlohmann.me/api/basic_json/from_bjdata), [from_bon8](https://json.nlohmann.me/api/basic_json/from_bon8), [from_bson](https://json.nlohmann.me/api/basic_json/from_bson), [from_cbor](https://json.nlohmann.me/api/basic_json/from_cbor), [from_msgpack](https://json.nlohmann.me/api/basic_json/from_msgpack), [from_ubjson](https://json.nlohmann.me/api/basic_json/from_ubjson), [to_bjdata](https://json.nlohmann.me/api/basic_json/to_bjdata), [to_bon8](https://json.nlohmann.me/api/basic_json/to_bon8), [to_bson](https://json.nlohmann.me/api/basic_json/to_bson), [to_cbor](https://json.nlohmann.me/api/basic_json/to_cbor), [to_msgpack](https://json.nlohmann.me/api/basic_json/to_msgpack), [to_ubjson](https://json.nlohmann.me/api/basic_json/to_ubjson)
- **Non-member functions**: [operator<<](https://json.nlohmann.me/api/operator_ltlt/), [operator>>](https://json.nlohmann.me/api/operator_gtgt/), [to_string](https://json.nlohmann.me/api/basic_json/to_string)
- **Literals**: [operator""_json](https://json.nlohmann.me/api/operator_literal_json)
- **Helper classes**: [std::hash&lt;basic_json&gt;](https://json.nlohmann.me/api/basic_json/std_hash), [std::swap&lt;basic_json&gt;](https://json.nlohmann.me/api/basic_json/std_swap)
@@ -1110,9 +1110,9 @@ Other Important points:
- When using `get<ENUM_TYPE>()`, undefined JSON values will default to the first pair specified in your map. Select this default pair carefully. If you desire an exception in this circumstance use `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT()` which behaves identically except for throwing an exception on unrecognized values.
- If an enum or JSON value is specified more than once in your map, the first matching occurrence from the top of the map will be returned when converting to or from JSON.
### Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)
### Binary formats (BSON, CBOR, MessagePack, UBJSON, BJData, and BON8)
Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [BSON](https://bsonspec.org) (Binary JSON), [CBOR](https://cbor.io) (Concise Binary Object Representation), [MessagePack](https://msgpack.org), [UBJSON](https://ubjson.org) (Universal Binary JSON Specification) and [BJData](https://neurojson.org/bjdata) (Binary JData) to efficiently encode JSON values to byte vectors and to decode such vectors.
Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [BSON](https://bsonspec.org) (Binary JSON), [CBOR](https://cbor.io) (Concise Binary Object Representation), [MessagePack](https://msgpack.org), [UBJSON](https://ubjson.org) (Universal Binary JSON Specification), [BJData](https://neurojson.org/bjdata) (Binary JData), and [BON8](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md) (Binary Object Notation 8) to efficiently encode JSON values to byte vectors and to decode such vectors.
```cpp
// create a JSON value
@@ -1149,6 +1149,14 @@ std::vector<std::uint8_t> v_ubjson = json::to_ubjson(j);
// roundtrip
json j_from_ubjson = json::from_ubjson(v_ubjson);
// serialize to BON8
std::vector<std::uint8_t> v_bon8 = json::to_bon8(j);
// 0x88, 0x63, 0x6F, 0x6D, 0x70, 0x61, 0x63, 0x74, 0xF9, 0x73, 0x63, 0x68, 0x65, 0x6D, 0x61, 0x90
// roundtrip
json j_from_bon8 = json::from_bon8(v_bon8);
```
The library also supports binary types from BSON, CBOR (byte strings), and MessagePack (bin, ext, fixext). They are stored by default as `std::vector<std::uint8_t>` to be processed outside the library.
+6 -6
View File
@@ -300,10 +300,10 @@ add_custom_target(ci_test_skiplibraryversioncheck
# Disable thread-local storage.
###############################################################################
# Without thread-local storage, the copy constructor cannot bound its descent
# and copies every object and array without the call stack. That path is
# otherwise only reached by values nested deeper than the bound, so this target
# is what runs the whole test suite through it.
# Without thread-local storage, copying and comparing cannot bound their
# descent and handle every object and array without the call stack. Those paths
# are otherwise only reached by values nested deeper than the bound, so this
# target is what runs the whole test suite through them.
add_custom_target(ci_test_no_thread_local
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
@@ -542,7 +542,7 @@ add_custom_target(ci_infer
add_custom_target(ci_offline_testdata
COMMAND mkdir -p ${PROJECT_BINARY_DIR}/build_offline_testdata/test_data
COMMAND cd ${PROJECT_BINARY_DIR}/build_offline_testdata/test_data && ${GIT_TOOL} clone -c advice.detachedHead=false --branch v3.1.0 https://github.com/nlohmann/json_test_data.git --quiet --depth 1
COMMAND cd ${PROJECT_BINARY_DIR}/build_offline_testdata/test_data && ${GIT_TOOL} clone -c advice.detachedHead=false --branch v3.2.0 https://github.com/nlohmann/json_test_data.git --quiet --depth 1
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_FastTests=ON -DJSON_TestDataDirectory=${PROJECT_BINARY_DIR}/build_offline_testdata/test_data/json_test_data
@@ -619,7 +619,7 @@ add_custom_target(ci_single_binaries
add_custom_target(ci_benchmarks
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Release -GNinja
-S${PROJECT_SOURCE_DIR}/benchmarks -B${PROJECT_BINARY_DIR}/build_benchmarks
-S${PROJECT_SOURCE_DIR}/tests/benchmarks -B${PROJECT_BINARY_DIR}/build_benchmarks
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_benchmarks --target json_benchmarks
COMMAND cd ${PROJECT_BINARY_DIR}/build_benchmarks && ./json_benchmarks
COMMENT "Run benchmarks"
+2 -2
View File
@@ -1,5 +1,5 @@
set(JSON_TEST_DATA_URL https://github.com/nlohmann/json_test_data)
set(JSON_TEST_DATA_VERSION 3.1.0)
set(JSON_TEST_DATA_VERSION 3.2.0)
include(ExternalProject)
@@ -77,7 +77,7 @@ if(CMAKE_CROSSCOMPILING)
endif()
if(NOT DEFINED LIBCPP_VERSION_OUTPUT_CACHED)
try_run(RUN_RESULT_VAR COMPILE_RESULT_VAR
"${CMAKE_BINARY_DIR}" SOURCES "${CMAKE_SOURCE_DIR}/cmake/detect_libcpp_version.cpp"
"${CMAKE_BINARY_DIR}" SOURCES "${CMAKE_CURRENT_LIST_DIR}/detect_libcpp_version.cpp"
RUN_OUTPUT_VARIABLE LIBCPP_VERSION_OUTPUT
COMPILE_OUTPUT_VARIABLE LIBCPP_VERSION_COMPILE_OUTPUT
)
-1
View File
@@ -42,7 +42,6 @@ string(APPEND CONTENT [=[
],
includes = ["include"],
visibility = ["//visibility:public"],
alwayslink = True,
)
cc_library(
+3
View File
@@ -48,6 +48,7 @@ INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_bjdata', 'Fu
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_bson', 'Function', 'api/basic_json/from_bson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_cbor', 'Function', 'api/basic_json/from_cbor/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_msgpack', 'Function', 'api/basic_json/from_msgpack/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_bon8', 'Function', 'api/basic_json/from_bon8/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_ubjson', 'Function', 'api/basic_json/from_ubjson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::front', 'Method', 'api/basic_json/front/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::get', 'Method', 'api/basic_json/get/index.html');
@@ -121,6 +122,7 @@ INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_bjdata', 'Func
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_bson', 'Function', 'api/basic_json/to_bson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_cbor', 'Function', 'api/basic_json/to_cbor/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_msgpack', 'Function', 'api/basic_json/to_msgpack/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_bon8', 'Function', 'api/basic_json/to_bon8/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_string', 'Method', 'api/basic_json/to_string/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_ubjson', 'Function', 'api/basic_json/to_ubjson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::value', 'Method', 'api/basic_json/value/index.html');
@@ -171,6 +173,7 @@ INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: BJData', 'Gui
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: BSON', 'Guide', 'features/binary_formats/bson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: CBOR', 'Guide', 'features/binary_formats/cbor/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: MessagePack', 'Guide', 'features/binary_formats/messagepack/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: BON8', 'Guide', 'features/binary_formats/bon8/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: UBJSON', 'Guide', 'features/binary_formats/ubjson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Values', 'Guide', 'features/binary_values/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Comments', 'Guide', 'features/comments/index.html');
@@ -18,7 +18,7 @@ ignore
: ignore tags
store
: store tagged values as binary container with subtype (for bytes 0xd8..0xdb)
: store tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored.
## Examples
@@ -104,6 +104,7 @@ Linear in the size of the input.
- [from_msgpack](from_msgpack.md) create a JSON value from an input in MessagePack format
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -0,0 +1,108 @@
# <small>nlohmann::basic_json::</small>from_bon8
```cpp
// (1)
template<typename InputType>
static basic_json from_bon8(InputType&& i,
const bool strict = true,
const bool allow_exceptions = true);
// (2)
template<typename IteratorType, typename SentinelType = IteratorType>
static basic_json from_bon8(IteratorType first, SentinelType last,
const bool strict = true,
const bool allow_exceptions = true);
```
Deserializes a given input to a JSON value using the BON8 (Binary Object Notation 8) serialization format.
1. Reads from a compatible input.
2. Reads from an iterator range, or an iterator and a sentinel of a different type (C++20 ranges support).
The exact mapping and its limitations are described on a [dedicated page](../../features/binary_formats/bon8.md).
## Template parameters
`InputType`
: A compatible input, for instance:
- an `std::istream` object
- a `FILE` pointer
- a C-style array of characters
- a pointer to a null-terminated string of single byte characters
- a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
(as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
`IteratorType`
: a compatible iterator type
`SentinelType`
: defaults to `IteratorType`; may be a different type comparable to `IteratorType` via `operator!=`, for instance.
- a custom sentinel type for C++20 ranges
- `std::default_sentinel_t`, when `IteratorType` is `std::counted_iterator`
## Parameters
`i` (in)
: an input in BON8 format convertible to an input adapter
`first` (in)
: iterator to the start of the input
`last` (in)
: iterator to the end of the input, or a sentinel value that compares equal to the end iterator with `operator!=`
`strict` (in)
: whether to expect the input to be consumed until EOF (`#!cpp true` by default)
`allow_exceptions` (in)
: whether to throw exceptions in case of a parse error (optional, `#!cpp true` by default)
## Return value
deserialized JSON value; in case of a parse error and `allow_exceptions` set to `#!cpp false`, the return value will be
`value_t::discarded`. The latter can be checked with [`is_discarded`](is_discarded.md).
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
## Exceptions
- Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or
the end of the file was not reached when `strict` was set to true
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs, for instance
an invalid byte, a string that is not valid UTF-8, or an object key that is not a string
## Complexity
Linear in the size of the input.
## Examples
??? example
The example shows the deserialization of a byte vector in BON8 format to a JSON value.
```cpp
--8<-- "examples/from_bon8.cpp"
```
Output:
```json
--8<-- "examples/from_bon8.output"
```
## See also
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
- [from_cbor](from_cbor.md) create a JSON value from an input in CBOR format
- [from_msgpack](from_msgpack.md) create a JSON value from an input in MessagePack format
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
## Version history
- Added in version 3.13.0.
@@ -104,6 +104,7 @@ Linear in the size of the input.
- [from_msgpack](from_msgpack.md) for the related MessagePack format
- [from_ubjson](from_ubjson.md) for the related UBJSON format
- [from_bjdata](from_bjdata.md) for the related BJData format
- [from_bon8](from_bon8.md) for the related BON8 format
## Version history
@@ -110,6 +110,7 @@ Linear in the size of the input.
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -103,6 +103,7 @@ Linear in the size of the input.
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -104,6 +104,7 @@ Linear in the size of the input.
- [from_msgpack](from_msgpack.md) create a JSON value from an input in MessagePack format
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -21,6 +21,10 @@ This overload is chosen if:
- `ValueType` is not `basic_json`,
- `json_serializer<ValueType>` has a `from_json()` method of the form `void from_json(const basic_json&, ValueType&)`
`v` must not be `const`. Passing a `const` object is a compile-time error. For types such as arithmetic types, enums,
and C arrays, the error is a `static_assert` that names the problem. For other types, the overload is not viable, and
the compiler reports that no matching `get_to` was found.
## Template parameters
`ValueType`
@@ -67,3 +71,4 @@ Depends on the `json_serializer<ValueType>::from_json()` implementation.
## Version history
- Since version 3.3.0.
- Added a `static_assert` with a clear message for `const` arguments in version 3.13.0.
+2
View File
@@ -290,11 +290,13 @@ Access to the JSON value
### Binary formats
- [**from_bjdata**](from_bjdata.md) (_static_) - create a JSON value from an input in BJData format
- [**from_bon8**](from_bon8.md) (_static_) - create a JSON value from an input in BON8 format
- [**from_bson**](from_bson.md) (_static_) - create a JSON value from an input in BSON format
- [**from_cbor**](from_cbor.md) (_static_) - create a JSON value from an input in CBOR format
- [**from_msgpack**](from_msgpack.md) (_static_) - create a JSON value from an input in MessagePack format
- [**from_ubjson**](from_ubjson.md) (_static_) - create a JSON value from an input in UBJSON format
- [**to_bjdata**](to_bjdata.md) (_static_) - create a BJData serialization of a given JSON value
- [**to_bon8**](to_bon8.md) (_static_) - create a BON8 serialization of a given JSON value
- [**to_bson**](to_bson.md) (_static_) - create a BSON serialization of a given JSON value
- [**to_cbor**](to_cbor.md) (_static_) - create a CBOR serialization of a given JSON value
- [**to_msgpack**](to_msgpack.md) (_static_) - create a MessagePack serialization of a given JSON value
@@ -7,7 +7,8 @@ enum class input_format_t {
msgpack,
ubjson,
bson,
bjdata
bjdata,
bon8
};
```
@@ -31,6 +32,9 @@ bson
bjdata
: BJData (Binary JData)
bon8
: BON8 (Binary Object Notation 8)
## Examples
??? example
@@ -5,7 +5,7 @@ class parse_error : public exception;
```
The library throws this exception when a parse error occurs. Parse errors can occur during the deserialization of
JSON text, BSON, CBOR, MessagePack, UBJSON, as well as when using JSON Patch.
JSON text, BJData, BON8, BSON, CBOR, MessagePack, UBJSON, as well as when using JSON Patch.
Member `byte` holds the byte index of the last read character in the input file (see note below).
+7 -2
View File
@@ -65,11 +65,14 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
: SAX event listener (must not be null)
`format` (in)
: the format to parse (JSON, CBOR, MessagePack, or UBJSON) (optional, `input_format_t::json` by default), see
: the format to parse (JSON, BJData, BON8, BSON, CBOR, MessagePack, or UBJSON) (optional, `input_format_t::json` by
default), see
[`input_format_t`](input_format_t.md) for more information
`strict` (in)
: whether the input has to be consumed completely (optional, `#!cpp true` by default)
: whether the input has to be consumed completely (optional, `#!cpp true` by default); when `#!cpp false` and the
input is a `#!cpp std::istream`, the character that terminates a number is consumed unless
[`JSON_PRECISE_STREAM_POSITION`](../macros/json_precise_stream_position.md) is defined to `1`; see [`operator>>`](../operator_gtgt.md#notes)
`ignore_comments` (in)
: whether comments should be ignored and treated like whitespace (`#!cpp true`) or yield a parse error
@@ -136,6 +139,8 @@ A UTF-8 byte order mark is silently ignored.
- Added `ignore_trailing_commas` in version 3.13.0.
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
- `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave a `#!cpp std::istream` positioned right
after the parsed value when `strict` is `#!cpp false`.
!!! warning "Deprecation"
@@ -84,6 +84,7 @@ Linear in the size of the JSON value `j`.
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
@@ -0,0 +1,76 @@
# <small>nlohmann::basic_json::</small>to_bon8
```cpp
// (1)
static std::vector<std::uint8_t> to_bon8(const basic_json& j);
// (2)
static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o);
static void to_bon8(const basic_json& j, detail::output_adapter<char> o);
```
Serializes a given JSON value `j` to a byte vector using the BON8 (Binary Object Notation 8) serialization format. BON8
is a compact binary serialization format that stores strings as UTF-8 without a length prefix.
1. Returns a byte vector containing the BON8 serialization.
2. Writes the BON8 serialization to an output adapter.
The exact mapping and its limitations are described on a [dedicated page](../../features/binary_formats/bon8.md).
## Parameters
`j` (in)
: JSON value to serialize
`o` (in)
: output adapter to write serialization to
## Return value
1. BON8 serialization as a byte vector
2. (none)
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes in the JSON value `j`, which is never modified.
With (2), the bytes written before the exception remain in the output adapter.
## Exceptions
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an unsigned integer
above 9223372036854775807, which BON8 cannot represent
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if `j` contains a string that is not
valid UTF-8
## Complexity
Linear in the size of the JSON value `j`.
## Examples
??? example
The example shows the serialization of a JSON value to a byte vector in BON8 format.
```cpp
--8<-- "examples/to_bon8.cpp"
```
Output:
```json
--8<-- "examples/to_bon8.output"
```
## See also
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
- [to_cbor](to_cbor.md) create a CBOR serialization of a JSON value
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
## Version history
- Added in version 3.13.0.
+4 -3
View File
@@ -46,9 +46,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
## Complexity
Proportional to the size of the JSON value `j` multiplied by its maximum nesting
depth, `O(n × d)`. BSON length prefixes are computed recursively before nested
values are written.
Linear in the size of the JSON value `j`. The length prefixes of all nested documents and arrays are computed in one
pass before anything is written.
## Examples
@@ -73,7 +72,9 @@ values are written.
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
- Added in version 3.4.0.
- Linear in the size of `j`, and no longer limited by the call stack for deeply nested values, since version 3.13.0.
@@ -62,6 +62,7 @@ Linear in the size of the JSON value `j`.
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
@@ -34,6 +34,15 @@ The exact mapping and its limitations are described on a [dedicated page](../../
Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
## Exceptions
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a string, binary
value, array, or object exceeds 4294967295, the maximum MessagePack can store; example:
`"MessagePack length 4294967296 exceeds maximum of 4294967295"`
- Throws [`out_of_range.415`](../../home/exceptions.md#jsonexceptionout_of_range415) if the subtype of a binary value
exceeds 255, the maximum of the MessagePack ext type; example:
`"subtype 70000 is too large for the MessagePack ext type (max 255)"`
## Complexity
Linear in the size of the JSON value `j`.
@@ -61,7 +70,9 @@ Linear in the size of the JSON value `j`.
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
- Added in version 2.0.9.
- Throws `out_of_range.412` and `out_of_range.415` since version 3.13.0.
@@ -77,6 +77,7 @@ Linear in the size of the JSON value `j`.
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
+4
View File
@@ -16,6 +16,8 @@ header. See also the [macro overview page](../../features/macros.md).
## Parsing
- [**JSON_PRECISE_STREAM_POSITION**](json_precise_stream_position.md) - opt in to leaving an input stream positioned
right after a parsed number
- [**JSON_STRICT_NUL_HANDLING**](json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input instead of
treating it as end of input
@@ -51,6 +53,8 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_BRACE_INIT_COPY_SEMANTICS**](json_brace_init_copy_semantics.md) - opt in to copy/move semantics for single-element brace initialization
- [**JSON_DISABLE_ENUM_SERIALIZATION**](json_disable_enum_serialization.md) - switch off default serialization/deserialization functions for enums
- [**JSON_USE_IMPLICIT_CONVERSIONS**](json_use_implicit_conversions.md) - control implicit conversions
- [**JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS**](json_use_objects_for_enum_keyed_maps.md) - opt in to storing maps with enum
keys as objects
## Comparison behavior
@@ -7,16 +7,16 @@
When defined, the library does not use `#!cpp thread_local` storage. This is relevant for the few environments whose
toolchain does not support it.
The copy constructor copies the first levels of a value by copying the containers, which copy their elements, and
completes whatever is nested deeper than that without the call stack, so that copying a value cannot exhaust the stack
however deeply it is nested. It counts the levels it has descended into in a `#!cpp thread_local` variable, as a counter
shared between threads would be raced.
Copying a value and comparing two values both descend into the first levels by letting the containers copy or compare
themselves, and finish whatever is nested deeper than that without the call stack, so that neither can exhaust the stack
however deeply the values are nested. Each counts the levels it has descended into in a `#!cpp thread_local` variable, as
a counter shared between threads would be raced.
Without that counter, no descent can be bounded safely, so objects and arrays are copied without the call stack right
away. Copying keeps working exactly as it does otherwise - the same values come out, and deeply nested values are copied
just as safely - but copying is slower, because the containers no longer copy themselves. Copying the benchmark
documents takes 9% (`canada.json`) to 34% (`twitter.json`) longer; values built mostly from objects are affected the
most.
Without those counters, no descent can be bounded safely, so objects and arrays are copied and compared without the call
stack right away. Both keep working exactly as they do otherwise - the same values come out, the same comparisons hold,
and deeply nested values are handled just as safely - but both are slower, because the containers no longer copy or
compare themselves. Copying the benchmark documents takes 9% (`canada.json`) to 34% (`twitter.json`) longer, and
comparing two equal ones 10% (`citm_catalog.json`) to 90% (`canada.json`) longer.
## Default definition
@@ -28,6 +28,7 @@ By default, `#!cpp JSON_NO_THREAD_LOCAL` is not defined.
The library defines it by itself for Clang targeting MinGW, which does not survive the `#!cpp thread_local` storage:
copying a value segfaults there, with both old and current Clang versions, while GCC targeting MinGW is unaffected.
Copying and comparing fall back to working without the call stack there, as they do whenever the macro is defined.
## Examples
@@ -0,0 +1,131 @@
# JSON_PRECISE_STREAM_POSITION
```cpp
#define JSON_PRECISE_STREAM_POSITION /* value */
```
When defined to `1`, [`operator>>`](../operator_gtgt.md) and [`sax_parse`](../basic_json/sax_parse.md) with
`strict = false` leave a `#!cpp std::istream` positioned right after the parsed value for every value type. By default,
the character that terminates a number is consumed as well.
The macro only affects reading from a `#!cpp std::istream` when the rest of the stream is not required to be consumed.
[`parse`](../basic_json/parse.md), [`accept`](../basic_json/accept.md), and all other inputs (strings, iterators,
containers, `#!cpp FILE*`) are never affected.
## Default definition
The default value is `0` (disabled — existing behavior is preserved).
```cpp
#define JSON_PRECISE_STREAM_POSITION 0
```
## Notes
!!! note "Background"
A number is the only JSON value whose end can be detected solely by reading the character that follows it. By
default, that character is consumed and not put back, so the stream is left one byte too far after a number, and
only after a number:
```cpp
std::istringstream input("1true");
json j;
input >> j; // j == 1, but the stream now starts at "rue"
```
With this macro, the character is only looked at and left in the stream, so the stream starts at `true`. This
does not require the stream buffer to support putting a character back.
This was not changed unconditionally, because code can depend on the consumed character, even unknowingly (see
[#5340](https://github.com/nlohmann/json/issues/5340)). Both of the following work by default only because the
character after each number is swallowed, and behave differently with this macro:
```cpp
std::istringstream input("1,2,3");
json j1, j2, j3;
input >> j1 >> j2 >> j3; // default: 1, 2, 3
// with the macro: throws parse_error.101 at the ','
```
```cpp
std::istringstream input("42\nfoo");
json j;
std::string line;
input >> j;
std::getline(input, line); // default: "foo"
// with the macro: "" (like after reading an int with >>)
```
In both cases, the behavior with the macro is what you already get today when the value is not a number: `"a","b"`
fails at the `,`, and `std::getline` after `{}` returns an empty string. This macro offers an opt-in path to
the consistent behavior ahead of version 4.0.0, where it is planned to become the default.
!!! warning "Opt-in only"
This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no
effect.
!!! note "ABI compatibility"
The value of this macro is encoded in the [namespace](../../features/namespace.md) (tag `_psp`), resulting in
distinct symbol names. Translation units compiled with and without it can therefore be linked into the same program
without One Definition Rule (ODR) violations, but they cannot exchange instances of library types.
!!! tip "Workaround without the macro"
Separate the values in the stream with whitespace. The character consumed after a number is then the separator,
and whitespace before the next value is skipped anyway.
## Examples
??? example "Default behavior (macro not defined)"
Without the macro, the character after a number is consumed:
```cpp
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
std::istringstream input("1true");
json j1, j2;
input >> j1; // j1 == 1
input >> j2; // throws parse_error.101: the stream now starts at "rue"
}
```
??? example "Opt-in precise stream position (macro defined to 1)"
With the macro, the stream is positioned right after the number:
```cpp
#define JSON_PRECISE_STREAM_POSITION 1
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
std::istringstream input("1true");
json j1, j2;
input >> j1; // j1 == 1
input >> j2; // j2 == true
}
```
## See also
- [**operator>>**](../operator_gtgt.md) - deserialize from stream
- [**sax_parse**](../basic_json/sax_parse.md) - generate SAX events
## Version history
- Added in version 3.13.0.
- Planned to become the default (with the macro removed) in version 4.0.0.
@@ -11,9 +11,9 @@ The macro only affects the JSON text parser ([`parse`](../basic_json/parse.md),
[`sax_parse`](../basic_json/sax_parse.md), and [`operator>>`](../operator_gtgt.md)). There are three cases where a NUL
byte is still not rejected:
- The binary formats ([`from_bjdata`](../basic_json/from_bjdata.md), [`from_bson`](../basic_json/from_bson.md),
[`from_cbor`](../basic_json/from_cbor.md), [`from_msgpack`](../basic_json/from_msgpack.md),
[`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data.
- The binary formats ([`from_bjdata`](../basic_json/from_bjdata.md), [`from_bon8`](../basic_json/from_bon8.md),
[`from_bson`](../basic_json/from_bson.md), [`from_cbor`](../basic_json/from_cbor.md),
[`from_msgpack`](../basic_json/from_msgpack.md), [`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data.
- A bare `const char*` pointer has no length of its own, so its length is still determined with `strlen()`. The first
NUL byte therefore still marks the end of the input, and nothing after it is read.
- One trailing `'\0'` at the end of a `char` array (e.g., a string literal) is trimmed; see the warning below.
@@ -65,6 +65,12 @@ The default value is `0` (disabled — existing behavior is preserved).
for CBOR or MessagePack, are never affected by this trimming; their full extent - including a genuine trailing
`0x00` - is always preserved, in both states of this macro.
!!! note "ABI compatibility"
The value of this macro is encoded in the [namespace](../../features/namespace.md) (tag `_snul`), resulting in
distinct symbol names. Translation units compiled with and without it can therefore be linked into the same program
without One Definition Rule (ODR) violations, but they cannot exchange instances of library types.
!!! tip "Workaround without the macro"
To reject a NUL byte without enabling this macro, trim your input yourself before calling `parse()`:
@@ -0,0 +1,139 @@
# JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
```cpp
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS /* value */
```
When defined to `1`, maps whose keys are enums (such as `std::map<E, T>` or `std::unordered_map<E, T>`) are stored as
JSON objects, using the enum's own conversion for the keys. By default, they are stored as arrays of `[key, value]`
pairs.
## Default definition
The default value is `0` (disabled — existing behavior is preserved).
```cpp
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
```
## Notes
!!! note "Background"
JSON object keys are strings, so a map is only stored as an object if its keys can be converted to a string type.
Enums are not, even if [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md) maps them to strings, so a
map with enum keys becomes an array of `[key, value]` pairs:
```json
[["stopped", "aa"], ["completed", "bb"]]
```
With this macro, the same map becomes an object
(see [#4378](https://github.com/nlohmann/json/issues/4378)):
```json
{"completed": "bb", "stopped": "aa"}
```
!!! note "Maps with non-unique keys"
Maps that allow duplicate keys, such as `std::multimap<E, T>` or `std::unordered_multimap<E, T>`, are not affected
by the macro and are still stored as arrays of `[key, value]` pairs, as an object cannot hold duplicate keys.
!!! note "Reading"
Reading is not affected by the macro: a map with enum keys can always be read from both an array of pairs and an
object. For the latter, each key is converted to the enum with its `from_json` function, e.g., the one defined by
[`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md). Data written without the macro can therefore
still be read after enabling it.
!!! warning "Keys must serialize to distinct strings"
Each key is converted with the enum's `to_json` function. If a key is not converted to a string (for instance, an
enum without [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md), which is stored as an integer, or an
enumerator mapped to `nullptr`), [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) is thrown.
If two keys are converted to the same string (for instance, because
[`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md) maps an unlisted enumerator to the first entry),
[`type_error.318`](../../home/exceptions.md#jsonexceptiontype_error318) is thrown. In both cases, the target value
is not changed.
!!! warning "Opt-in only"
This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no effect.
!!! note "ABI compatibility"
The value of this macro is encoded in the [namespace](../../features/namespace.md) (tag `_ekmo`), resulting in
distinct symbol names. Translation units compiled with and without it can therefore be linked into the same program
without One Definition Rule (ODR) violations, but they cannot exchange instances of library types.
## Examples
??? example "Default behavior (macro not defined)"
Without the macro, a map with enum keys is stored as an array of pairs:
```cpp
#include <map>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
enum TaskState { TS_STOPPED, TS_RUNNING, TS_COMPLETED };
NLOHMANN_JSON_SERIALIZE_ENUM(TaskState, {
{TS_STOPPED, "stopped"},
{TS_RUNNING, "running"},
{TS_COMPLETED, "completed"},
})
int main()
{
std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
json j = m;
// j is [["stopped","aa"],["completed","bb"]]
}
```
??? example "Objects for enum-keyed maps (macro defined to 1)"
With the macro, the same map is stored as an object:
```cpp
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 1
#include <map>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
enum TaskState { TS_STOPPED, TS_RUNNING, TS_COMPLETED };
NLOHMANN_JSON_SERIALIZE_ENUM(TaskState, {
{TS_STOPPED, "stopped"},
{TS_RUNNING, "running"},
{TS_COMPLETED, "completed"},
})
int main()
{
std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
json j = m;
// j is {"completed":"bb","stopped":"aa"}
auto m2 = j.get<std::map<TaskState, std::string>>();
// m2 == m
}
```
## See also
- [Specializing enum conversion](../../features/enum_conversion.md)
- [**NLOHMANN_JSON_SERIALIZE_ENUM**](nlohmann_json_serialize_enum.md) - serialize/deserialize an enum
- [**NLOHMANN_JSON_SERIALIZE_ENUM_STRICT**](nlohmann_json_serialize_enum_strict.md) - serialize/deserialize an enum with
exceptions
## Version history
- Added in version 3.13.0.
@@ -57,7 +57,8 @@ Summary:
: name of the base type (class, struct) `type` is derived from
`member` (in)
: name of the member variable to serialize/deserialize; up to 63 members can be given as a comma-separated list
: name of the member variable to serialize/deserialize; up to 63 members can be given as a comma-separated
list, which may also be empty
## Default definition
@@ -127,6 +128,20 @@ void to_json(BasicJsonType& j, const B& b) {
- Macros 4, 5, and 6 have the same prerequisites of [NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE](nlohmann_define_type_non_intrusive.md).
- Serialization/deserialization of base types must be defined.
!!! info "Derived types without own members"
The member list may be empty. The macro then generates a `to_json`/`from_json` pair that only delegates to
the base type, so `type` serializes exactly like `base_type`:
```cpp
struct derived : base
{
NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE(derived, base)
};
```
The `WITH_NAMES` variants do not support this.
!!! warning "Implementation limits"
See Implementation limits for [NLOHMANN_DEFINE_TYPE_INTRUSIVE](nlohmann_define_type_intrusive.md) and
@@ -33,7 +33,8 @@ Summary:
: name of the type (class, struct) to serialize/deserialize
`member` (in)
: name of the member variable to serialize/deserialize; up to 63 members can be given as a comma-separated list
: name of the member variable to serialize/deserialize; up to 63 members can be given as a comma-separated
list, which may also be empty
## Default definition
@@ -58,6 +59,20 @@ See the examples below for the concrete generated code.
[GetNonDefNonCopy]: ../../features/arbitrary_types.md#how-can-i-use-get-for-non-default-constructiblenon-copyable-types
!!! info "Types without members"
The member list may be empty. The macro then generates a `to_json` that produces an empty JSON object
`#!json {}`, and a `from_json` that reads no members:
```cpp
struct marker
{
NLOHMANN_DEFINE_TYPE_INTRUSIVE(marker)
};
```
The `WITH_NAMES` variants do not support this.
!!! warning "Implementation limits"
- The current implementation is limited to at most 63 member variables. If you want to serialize/deserialize types
@@ -33,7 +33,8 @@ Summary:
: name of the type (class, struct) to serialize/deserialize
`member` (in)
: name of the (public) member variable to serialize/deserialize; up to 63 members can be given as a comma-separated list
: name of the (public) member variable to serialize/deserialize; up to 63 members can be given as a
comma-separated list, which may also be empty
## Default definition
@@ -59,6 +60,18 @@ See the examples below for the concrete generated code.
[GetNonDefNonCopy]: ../../features/arbitrary_types.md#how-can-i-use-get-for-non-default-constructiblenon-copyable-types
!!! info "Types without members"
The member list may be empty. The macro then generates a `to_json` that produces an empty JSON object
`#!json {}`, and a `from_json` that reads no members:
```cpp
struct marker {};
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(marker)
```
The `WITH_NAMES` variants do not support this.
!!! warning "Implementation limits"
- The current implementation is limited to at most 63 member variables. If you want to serialize/deserialize types
@@ -41,6 +41,9 @@ inline void from_json(const BasicJsonType& j, type& e);
conversion. Select this default pair carefully. See example 1 below.
- If an enum or JSON value is specified in multiple conversions, the first matching conversion from the top of the
list will be returned when converting to or from JSON. See example 2 below.
- Maps with enum keys (e.g., `std::map<ENUM_TYPE, T>`) are stored as arrays of `[key, value]` pairs by default.
Define [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md) to store them as objects
with the converted keys. Such maps can be read from both forms.
## Examples
@@ -80,6 +83,7 @@ inline void from_json(const BasicJsonType& j, type& e);
- [Specializing enum conversion](../../features/enum_conversion.md)
- [`NLOHMANN_JSON_SERIALIZE_ENUM_STRICT`](./nlohmann_json_serialize_enum_strict.md)
- [`JSON_DISABLE_ENUM_SERIALIZATION`](json_disable_enum_serialization.md)
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md)
## Version history
@@ -44,6 +44,9 @@ inline void from_json(const BasicJsonType& j, type& e);
`"enum value out of range for <type>"`.
- If an enum or JSON value is specified in multiple conversions, the first matching conversion from the top of the
list will be returned when converting to or from JSON. See example 2 below.
- Maps with enum keys (e.g., `std::map<ENUM_TYPE, T>`) are stored as arrays of `[key, value]` pairs by default.
Define [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md) to store them as objects
with the converted keys. Such maps can be read from both forms.
## Examples
@@ -99,6 +102,7 @@ inline void from_json(const BasicJsonType& j, type& e);
- [Specializing enum conversion](../../features/enum_conversion.md)
- [`NLOHMANN_JSON_SERIALIZE_ENUM`](./nlohmann_json_serialize_enum.md)
- [`JSON_DISABLE_ENUM_SERIALIZATION`](json_disable_enum_serialization.md)
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md)
## Version history
+6 -1
View File
@@ -67,7 +67,9 @@ input >> j2; // parses the next value
Only numbers are affected. Values ending in a self-delimiting character do not read past themselves, so
`truefalse`, `[1][2]`, `{"a":1}{"b":2}`, and `"a""b"` can be read back to back without a separator.
This is tracked in [#5340](https://github.com/nlohmann/json/issues/5340).
Define [`JSON_PRECISE_STREAM_POSITION`](macros/json_precise_stream_position.md) to `1` to leave the terminating character in the stream
instead, so that the stream is positioned right after the value for every value type and no separator is
needed. This is tracked in [#5340](https://github.com/nlohmann/json/issues/5340).
Note that reading concatenated values does **not** work for [JSON Lines](../features/parsing/json_lines.md)
(newline-delimited JSON) input -- see that page for why and for the recommended alternative.
@@ -107,9 +109,12 @@ being read.
- [parse](basic_json/parse.md) - deserialize from a compatible input
- [`JSON_STRICT_NUL_HANDLING`](macros/json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input
instead of treating it as end of input
- [`JSON_PRECISE_STREAM_POSITION`](macros/json_precise_stream_position.md) - opt in to leaving the stream positioned right after a number
## Version history
- Added in version 1.0.0.
- `JSON_STRICT_NUL_HANDLING` added in version 3.13.0 to optionally reject a NUL byte in the input instead of treating
it as end of input; planned to become the default in version 4.0.0.
- `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave the character that terminates a number in
the stream; planned to become the default in version 4.0.0.
@@ -0,0 +1,70 @@
# Assurance case
This page argues why the library meets its security requirements. It describes the threats the library faces, where the
trust boundaries lie, and how the library's design and the [quality assurance](quality_assurance.md) counter these
threats. To report a vulnerability, see the [security policy](security_policy.md).
## Threat model
The library parses, stores, and serializes JSON values in memory. It does not open network connections, does not open
files (it only reads from streams or `std::FILE*` handles that the caller has already opened), does not read environment
variables, and does not implement cryptography or handle credentials.
The primary threat is therefore **untrusted input**: JSON text or binary data (BJData, BSON, CBOR, MessagePack, UBJSON)
that an attacker controls, passed to [`parse`](../api/basic_json/parse.md), [`accept`](../api/basic_json/accept.md),
[`sax_parse`](../api/basic_json/sax_parse.md), or one of the `from_*` functions such as
[`from_cbor`](../api/basic_json/from_cbor.md). Such input may try to
- make the library read or write out of bounds (malformed lengths, truncated input, invalid UTF-8),
- trigger undefined behavior (integer overflow in sizes or numbers, invalid casts),
- exhaust memory (huge announced sizes), or
- exhaust the call stack (deeply nested arrays and objects).
## Trust boundaries
- **Untrusted:** all serialized input read by the parser, the SAX interface, and the binary readers. The library must
handle every possible input by either producing a value or throwing a [`parse_error`](../home/exceptions.md#parse-errors)
(or returning `false` when exceptions are disabled for the call).
- **Trusted:** the C++ code that calls the library. Calling a function with violated preconditions, for instance
accessing an array with [`operator[]`](../api/basic_json/operator%5B%5D.md) out of range, is a programming error and
not a security boundary. Such preconditions are checked with [runtime assertions](../features/assertions.md) in debug
builds; functions such as [`at`](../api/basic_json/at.md) offer checked access with exceptions.
## Secure design
- **Strict parsing.** The parser accepts exactly the JSON grammar of [RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259).
Extensions such as [comments](../features/comments.md) and [trailing commas](../features/trailing_commas.md) must be
enabled explicitly. Invalid UTF-8 is rejected.
- **Errors are reported, not ignored.** Malformed input results in a [`parse_error`](../home/exceptions.md#parse-errors)
with the byte position of the error. Binary readers do not trust announced sizes: strings and binary values grow
only as bytes are actually read, arrays reserve at most a fixed number of elements up front, and sizes that no
container can hold are rejected.
- **Memory is owned by values.** Each `basic_json` value owns its content, and there is no manual memory management in
user code. The destructor does not recurse, so destroying a deeply nested value does not exhaust the stack.
- **Bounded recursion.** The JSON parser and the binary readers keep their state in explicit stacks instead of
recursing per nesting level. Operations that walk a value, such as [`dump`](../api/basic_json/dump.md), copying,
comparison, hashing, and [`merge_patch`](../api/basic_json/merge_patch.md), recurse only up to a fixed depth and
continue with an explicit stack below it. Some operations, such as [`diff`](../api/basic_json/diff.md),
[`flatten`](../api/basic_json/flatten.md), and the binary writers, still recurse once per nesting level; work on them
is in progress. Applications that process untrusted input can limit its nesting depth with a
[parser callback](../features/parsing/parser_callbacks.md).
- **Invariants are checked.** The class invariant (for instance, that the pointer for the stored type is never null) is
checked with runtime assertions throughout the test suite.
## Common weaknesses
The following table maps the relevant classes of the [Common Weakness Enumeration](https://cwe.mitre.org) to the
measures that counter them. The measures are described in detail in [Quality assurance](quality_assurance.md).
| Weakness | Countermeasures |
|---------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|
| Out-of-bounds read/write ([CWE-125](https://cwe.mitre.org/data/definitions/125.html), [CWE-787](https://cwe.mitre.org/data/definitions/787.html)) | bounds checks on all reads from the input; AddressSanitizer and Valgrind on the test suite; OSS-Fuzz |
| Integer overflow ([CWE-190](https://cwe.mitre.org/data/definitions/190.html)) | UndefinedBehaviorSanitizer with integer overflow detection; Clang-Tidy; Cppcheck |
| Use after free, double free ([CWE-416](https://cwe.mitre.org/data/definitions/416.html), [CWE-415](https://cwe.mitre.org/data/definitions/415.html)) | ownership of all memory by values; AddressSanitizer and Valgrind; Clang Static Analyzer |
| Memory leaks ([CWE-401](https://cwe.mitre.org/data/definitions/401.html)) | Valgrind (Memcheck) on the test suite |
| Uncontrolled recursion ([CWE-674](https://cwe.mitre.org/data/definitions/674.html)) | iterative parser, binary readers, and destructor; bounded recursion in value operations; tests with deeply nested inputs |
| Uncontrolled resource consumption ([CWE-400](https://cwe.mitre.org/data/definitions/400.html)) | allocations based on announced sizes are capped; OSS-Fuzz with memory limits |
| Undefined behavior in general ([CWE-758](https://cwe.mitre.org/data/definitions/758.html)) | UndefinedBehaviorSanitizer; runtime assertions; Clang-Tidy, Cppcheck, Clang Static Analyzer, Infer |
In addition, every line of the library is covered by the unit tests, and all parsers are fuzz-tested around the clock
by [OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json).
+25
View File
@@ -91,6 +91,31 @@ activities include (but are not limited to):
Users who continue to engage with the project and its community will often find themselves becoming more and more
involved. Such users may then go on to become contributors, as described above.
## Access to project resources
The project's resources are the [GitHub repository](https://github.com/nlohmann/json) with its settings, CI workflows
and secrets, and the documentation at [json.nlohmann.me](https://json.nlohmann.me), which is built and deployed from
the repository. Currently, the project lead is the only person with write or admin access to them.
### Granting access
Write or admin access is only granted by the project lead, and only to a contributor whose track record in the project
the project lead has reviewed first. The role is assigned manually and is the lowest one that is needed for the task.
Access is removed when it is no longer needed. GitHub requires two-factor authentication for everyone who can modify the
repository.
### Secrets
The CI workflows mostly use the token that GitHub creates for each workflow run. It is read-only by default, and each
workflow requests only the additional permissions it needs. The few other credentials, such as the token for
[Semgrep](https://semgrep.dev), are stored as encrypted GitHub Actions secrets:
- Only people with admin access can create, change, or delete them. Their values cannot be read back, not even by
admins.
- They are not passed to workflows that run for pull requests from forks.
- They must never be committed to the repository or printed in logs.
- They are rotated whenever someone with admin access leaves the project, and immediately if a leak is suspected.
## Support
All participants in the community are encouraged to provide support for new users within the project management
+2
View File
@@ -5,4 +5,6 @@
- [Contribution Guidelines](contribution_guidelines.md) - guidelines how to contribute to this project
- [Governance](governance.md) - the governance model of this project
- [Quality Assurance](quality_assurance.md) - how the quality of this project is assured
- [Roadmap](roadmap.md) - what the project will and will not do
- [Security Policy](security_policy.md) - the security policy of the project
- [Assurance Case](assurance_case.md) - why the library meets its security requirements
@@ -164,6 +164,9 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
- [x] The parser is tested against extensive correctness suites for JSON compliance.
- [x] In addition, the library is continuously fuzz-tested at [OSS-Fuzz](https://google.github.io/oss-fuzz/) where the
library is checked against billions of inputs.
- [x] Every crash reported by OSS-Fuzz is fixed together with a unit test that reproduces it, and the fix references
the OSS-Fuzz issue. The round-trip checks of the fuzzer drivers are also part of the unit tests. See the
[fuzz testing documentation](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports).
## Static analysis
@@ -197,6 +200,25 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
- [x] The test suite is executed with [Sanitizers](https://github.com/google/sanitizers) (address sanitizer, undefined
behavior sanitizer, integer overflow detection, nullability violations).
## Dependencies
!!! success "Requirement: No vulnerable dependencies"
The library has no dependencies besides the C++ standard library. The tools used to build, test, and document it
are kept free of known vulnerabilities.
- [x] GitHub Actions are pinned to a commit hash, and the Python packages used by the documentation and the tools are
pinned to exact versions.
- [x] [Dependabot](https://docs.github.com/en/code-security/dependabot) checks these dependencies daily and proposes
updates as pull requests.
- [x] Every pull request is checked with the
[dependency review action](https://github.com/actions/dependency-review-action). A pull request that adds a
dependency with a known vulnerability of any severity fails this check and is not merged.
- [x] Vulnerability alerts for dependencies are fixed or dismissed with a documented reason before the next release.
No release is made while such an alert is open.
- [x] Third-party code included in the repository for testing, such as [doctest](https://github.com/doctest/doctest),
is updated manually.
## Style check
!!! success "Requirement: Common code style"
+43
View File
@@ -0,0 +1,43 @@
# Roadmap
This page describes what the project intends to do, and what it does not intend to do, over the next year. Concrete
work items are tracked in the [GitHub milestones](https://github.com/nlohmann/json/milestones) and the
[issue tracker](https://github.com/nlohmann/json/issues).
## What the project will do
- **Keep the C++11 baseline.** The library will continue to compile with every
[supported C++11 compiler](https://github.com/nlohmann/json/blob/develop/README.md#supported-compilers). Features of
later standards are only used when they are guarded by the `JSON_HAS_CPP_*` macros.
- **Stay conformant to JSON.** The parser and serializer follow [RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259).
Extensions such as [comments](../features/comments.md) or [trailing commas](../features/trailing_commas.md) remain
opt-in.
- **Keep the 3.x public API stable.** Releases follow [semantic versioning](https://semver.org). Changes that would
break existing code are only added behind a feature macro, so users can opt in and test their code before a next
major release.
- **Support a broad range of compilers and platforms.** The [CI](quality_assurance.md) keeps testing old and new
versions of GCC, Clang, MSVC, and other compilers on Linux, macOS, and Windows.
- **Keep the quality assurance up.** Every change keeps the test coverage at 100%, passes the static and dynamic
analysis, and is fuzz-tested by OSS-Fuzz, see [Quality assurance](quality_assurance.md).
- **Harden the library against hostile input.** Handling deeply nested values without exhausting the call stack is
ongoing work.
- **Fix bugs and security issues** reported through the issue tracker and the [security policy](security_policy.md).
## What the project will not do
- **Break the public API of version 3.x.** See the
[contribution guidelines](https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#break-the-public-api)
for what counts as a breaking change.
- **Require a newer C++ standard than C++11.**
- **Break JSON conformance** or enable non-standard extensions by default.
- **Add dependencies** or require a build step. The library remains header-only, and the single header
`json.hpp` remains a complete distribution.
- **Trade simplicity for speed or memory efficiency.** Performance improvements are welcome, but the library is not
meant to compete with the fastest JSON libraries, see [Design goals](../home/design_goals.md).
## Version 4.0
There is no decision yet on whether or when a version 4.0 with breaking changes will be released. Proposals that need
a major version, for instance stricter type conversions, are collected in issue
[#3453](https://github.com/nlohmann/json/issues/3453). Until then, such changes are only added as opt-in behavior
behind feature macros.
+21
View File
@@ -0,0 +1,21 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
// create byte vector
std::vector<std::uint8_t> v = {0x89, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x63, 0x74,
0xf9, 0x66, 0x6f, 0x72, 0x6d, 0x61, 0x74, 0xff,
0x42, 0x4f, 0x4e, 0x38, 0xff, 0x73, 0x63, 0x68,
0x65, 0x6d, 0x61, 0x90
};
// deserialize it with BON8
json j = json::from_bon8(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
}
@@ -0,0 +1,5 @@
{
"compact": true,
"format": "BON8",
"schema": 0
}
+22
View File
@@ -0,0 +1,22 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "format": "BON8", "schema": 0})"_json;
// serialize it to BON8
std::vector<std::uint8_t> v = json::to_bon8(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
}
std::cout << std::endl;
}
+1
View File
@@ -0,0 +1 @@
0x89 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0xf9 0x66 0x6f 0x72 0x6d 0x61 0x74 0xff 0x42 0x4f 0x4e 0x38 0xff 0x73 0x63 0x68 0x65 0x6d 0x61 0x90
@@ -215,6 +215,24 @@ For _derived_ classes and structs, use the following macros
nlohmann::ordered_json j = p; // keys appear in declaration order: name, address, age
```
!!! note "Zero-member types"
All 12 `NLOHMANN_DEFINE_TYPE_*`/`NLOHMANN_DEFINE_DERIVED_TYPE_*` macros (excluding the `WITH_NAMES` variants)
also accept types with no member variables to serialize, producing/accepting an empty JSON object `{}`
(or, for the derived-type macros, just the base class's own JSON representation):
```cpp
namespace ns {
struct marker {
bool operator==(const marker&) const { return true; }
NLOHMANN_DEFINE_TYPE_INTRUSIVE(marker)
};
}
ns::marker m{};
nlohmann::json j = m; // {}
```
!!! note "No macro for non-default-constructible types"
There is currently no `NLOHMANN_DEFINE_TYPE_*`-style macro for types that are not
@@ -208,6 +208,16 @@ The library maps BJData types to JSON value types as follows:
The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
!!! info "Round trips"
A value returned by [`from_bjdata`](../../api/basic_json/from_bjdata.md) can be serialized with
[`to_bjdata`](../../api/basic_json/to_bjdata.md) using any combination of options and parsed back into an equal
value, and serializing that value again with the same options produces the same bytes. The exception is binary
values: they are only written as an optimized binary array (`[$B`) if Draft 3 is enabled and both `use_size` and
`use_type` are set. Otherwise, they are written as arrays of integers and parsed back as such (see the notes on
binary values above), and serializing such an array again may choose different, but equally valid, type markers.
The bytes can then differ, but parsing them again yields the same value.
??? example
```cpp
@@ -0,0 +1,159 @@
# BON8
BON8 (Binary Object Notation 8) is a compact binary serialization format for JSON values. It uses the byte values that
cannot begin a UTF-8 character as type markers, so strings are stored as plain UTF-8 without a length prefix: a string
ends at the first byte that cannot continue it. Integers from -10 to 39, `true`, `false`, `null`, and the floating-point
values -1.0, 0.0, and 1.0 take a single byte, and arrays and objects with up to four elements need no terminator.
!!! abstract "References"
- [BON8 specification](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md)
- [Reference implementation](https://github.com/hikoworks/hikogui/blob/main/src/hikogui/codec/BON8.hpp) in HikoGUI
## Serialization
The library uses the following mapping from JSON values types to BON8 types according to the BON8 specification:
| JSON value type | value/range | BON8 type | first byte |
|-----------------|----------------------------------------------|-------------------------------|------------|
| null | `null` | null | 0xFA |
| boolean | `true` | true | 0xF9 |
| boolean | `false` | false | 0xF8 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x8D |
| number_integer | -2147483648..-33818507 | int32 | 0x8C |
| number_integer | -33818506..-264075 | 4-byte negative integer | 0xF0..0xF7 |
| number_integer | -264074..-1931 | 3-byte negative integer | 0xE0..0xEF |
| number_integer | -1930..-11 | 2-byte negative integer | 0xC2..0xDF |
| number_integer | -10..-1 | 1-byte negative integer | 0xB8..0xC1 |
| number_integer | 0..39 | 1-byte positive integer | 0x90..0xB7 |
| number_integer | 40..3879 | 2-byte positive integer | 0xC2..0xDF |
| number_integer | 3880..528167 | 3-byte positive integer | 0xE0..0xEF |
| number_integer | 528168..67637031 | 4-byte positive integer | 0xF0..0xF7 |
| number_integer | 67637032..2147483647 | int32 | 0x8C |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x8D |
| number_unsigned | 0..39 | 1-byte positive integer | 0x90..0xB7 |
| number_unsigned | 40..3879 | 2-byte positive integer | 0xC2..0xDF |
| number_unsigned | 3880..528167 | 3-byte positive integer | 0xE0..0xEF |
| number_unsigned | 528168..67637031 | 4-byte positive integer | 0xF0..0xF7 |
| number_unsigned | 67637032..2147483647 | int32 | 0x8C |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x8D |
| number_float | `-1.0` | -1.0 | 0xFB |
| number_float | `0.0` | 0.0 | 0xFC |
| number_float | `1.0` | 1.0 | 0xFD |
| number_float | *any other value representable by a float* | binary32 | 0x8E |
| number_float | *any value NOT representable by a float* | binary64 | 0x8F |
| string | *empty* | end of string | 0xFF |
| string | *non-empty* | UTF-8 string | 0x00..0x7F, 0xC2..0xF4 |
| array | *size*: 0..4 | array with count | 0x80..0x84 |
| array | *size*: 5 or more | array (terminated by 0xFE) | 0x85 |
| object | *size*: 0..4 | object with count | 0x86..0x8A |
| object | *size*: 5 or more | object (terminated by 0xFE) | 0x8B |
| binary | *size*: 0..4 | array with count | 0x80..0x84 |
| binary | *size*: 5 or more | array (terminated by 0xFE) | 0x85 |
An integer that takes 2 to 4 bytes starts with a UTF-8 lead byte (0xC2..0xF7) that is followed by a byte that cannot
continue a UTF-8 character: 0x00..0x7F for positive and 0xC0..0xFF for negative integers. A string is terminated by
0xFF only if it is empty, if another string follows it, or if it is the last value of the message; otherwise, the first
byte of the next value ends it.
!!! success "Complete mapping"
Except for the values listed below, any JSON value can be converted to a BON8 value.
Any BON8 output created by `to_bon8` can be successfully parsed by `from_bon8`.
!!! warning "Unsupported values"
The following values can **not** be converted to a BON8 value:
- unsigned integers above 9223372036854775807, because BON8 has no unsigned 64-bit integer type
([out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407))
- strings that are not valid UTF-8, because the end of a string is determined from its encoding
([type_error.316](../../home/exceptions.md#jsonexceptiontype_error316))
!!! info "NaN/infinity handling"
`-0.0`, `Infinity`, and `-Infinity` are serialized as binary32 (type 0x8E, 5 bytes total). `NaN` is serialized as
the binary32 value 0x7F800001 that the specification recommends. This is in contrast to the
[dump](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
!!! warning "Binary values"
BON8 has no binary type. Binary values are serialized as arrays of integers (0..255), so they are read back as
arrays. The subtype is not serialized.
!!! info "Canonical representation"
The output follows the specification's canonical representation rules: every value uses the shortest encoding,
floating-point numbers use binary32 whenever that loses no precision, and object keys are sorted by their UTF-8
code units. There are two exceptions:
- Strings are not normalized to Unicode Normalization Form C (NFC).
- Object keys are written in the order of the object type, which is sorted for `json`, but not for
[`ordered_json`](../../api/ordered_json.md).
??? example
```cpp
--8<-- "examples/to_bon8.cpp"
```
Output:
```c
--8<-- "examples/to_bon8.output"
```
## Deserialization
The library maps BON8 types to JSON value types as follows:
| BON8 type | JSON value type | first byte |
|-------------------------------|-----------------|------------------------|
| UTF-8 string | string | 0x00..0x7F |
| array with count | array | 0x80..0x84 |
| array (terminated by 0xFE) | array | 0x85 |
| object with count | object | 0x86..0x8A |
| object (terminated by 0xFE) | object | 0x8B |
| int32 | number_unsigned or number_integer | 0x8C |
| int64 | number_unsigned or number_integer | 0x8D |
| binary32 | number_float | 0x8E |
| binary64 | number_float | 0x8F |
| 1-byte positive integer | number_unsigned | 0x90..0xB7 |
| 1-byte negative integer | number_integer | 0xB8..0xC1 |
| UTF-8 string | string | 0xC2..0xF4, followed by 0x80..0xBF |
| 2- to 4-byte positive integer | number_unsigned | 0xC2..0xF7, followed by 0x00..0x7F |
| 2- to 4-byte negative integer | number_integer | 0xC2..0xF7, followed by 0xC0..0xFF |
| false | `false` | 0xF8 |
| true | `true` | 0xF9 |
| null | `null` | 0xFA |
| -1.0 | number_float | 0xFB |
| 0.0 | number_float | 0xFC |
| 1.0 | number_float | 0xFD |
| empty string | string | 0xFF |
Non-negative integers are read as number_unsigned, negative integers as number_integer.
!!! info
Values that do not use the canonical representation, such as integers with a longer encoding than necessary,
arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a
string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty string.
Strings must be valid UTF-8, and the last string of a message must be terminated by 0xFF.
!!! info
Any BON8 output created by `to_bon8` can be successfully parsed by `from_bon8`.
??? example
```cpp
--8<-- "examples/from_bon8.cpp"
```
Output:
```json
--8<-- "examples/from_bon8.output"
```
@@ -109,6 +109,15 @@ The library maps BSON record types to JSON value types as follows:
If BSON input must be validated for strict specification compliance, validate it separately before passing it to
`from_bson()`.
!!! warning "UTF-8 validation of string values"
The BSON specification requires `string` values (type `0x02`) to be valid UTF-8. This library validates the
bytes of every such string at decode time and rejects ill-formed UTF-8 with a
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions`
set to `false`, a discarded value), rather than only failing later when the resulting value is dumped. Element
(key) names and `binary` values (type `0x05`) are unaffected and are never validated, since they are read
byte-by-byte as a C string, or are not required to hold text, respectively.
??? example
```cpp
@@ -176,9 +176,19 @@ The library maps CBOR types to JSON value types as follows:
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected.
!!! warning "UTF-8 validation of text strings"
[RFC 8949, Section 3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1) requires CBOR text strings
(major type 3) to be valid UTF-8. This library validates the bytes of every text string (object keys included) at
decode time and rejects ill-formed UTF-8 with a
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
`allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting value is
dumped. Byte strings (major type 2) are unaffected and are never validated, since they are not required to hold
text.
!!! warning "Tagged items"
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. They can be stored by passing `cbor_tag_handler_t::store` to function `from_cbor`. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
??? example
@@ -4,6 +4,7 @@ Though JSON is a ubiquitous data format, it is not a very compact format suitabl
a network. Hence, the library supports
- [BJData](bjdata.md) (Binary JData),
- [BON8](bon8.md) (Binary Object Notation 8),
- [BSON](bson.md) (Binary JSON),
- [CBOR](cbor.md) (Concise Binary Object Representation),
- [MessagePack](messagepack.md), and
@@ -18,6 +19,7 @@ to efficiently encode JSON values to byte vectors and to decode such vectors.
| Format | Serialization | Deserialization |
|-------------|-----------------------------------------------|----------------------------------------------|
| BJData | complete | complete |
| BON8 | incomplete: no unsigned integers above int64 | complete |
| BSON | incomplete: top-level value must be an object | incomplete, but all JSON types are supported |
| CBOR | complete | incomplete, but all JSON types are supported |
| MessagePack | complete | complete |
@@ -28,6 +30,7 @@ to efficiently encode JSON values to byte vectors and to decode such vectors.
| Format | Binary values | Binary subtypes |
|-------------|---------------|-----------------|
| BJData | not supported | not supported |
| BON8 | not supported | not supported |
| BSON | supported | supported |
| CBOR | supported | supported |
| MessagePack | supported | supported |
@@ -42,6 +45,7 @@ See [binary values](../binary_values.md) for more information.
| BJData | 53.2 % | 91.1 % | 78.1 % | 96.6 % |
| BJData (size) | 58.6 % | 92.1 % | 86.7 % | 97.4 % |
| BJData (size+type) | 58.6 % | 92.1 % | 86.5 % | 97.4 % |
| BON8 | 50.5 % | 83.8 % | 63.5 % | 87.5 % |
| BSON | 85.8 % | 95.2 % | 95.8 % | 106.7 % |
| CBOR | 50.5 % | 86.3 % | 68.4 % | 88.0 % |
| MessagePack | 50.5 % | 86.0 % | 68.5 % | 87.9 % |
@@ -65,6 +65,8 @@ specification:
- arrays with more than 4294967295 elements
- objects with more than 4294967295 elements
Serializing such a value throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412).
!!! info "NaN/infinity handling"
`NaN`, `Infinity`, and `-Infinity` are serialized as a MessagePack float 32 (type 0xCA, 5 bytes total),
@@ -136,6 +138,14 @@ The library maps MessagePack types to JSON value types as follows:
Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
!!! warning "UTF-8 validation of string values"
The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8.
This library validates the bytes of every such string (object keys included) at decode time and rejects
ill-formed UTF-8 with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or,
with `allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting
value is dumped. `bin`/`ext`/`fixext` values are unaffected and are never validated, since they are not required
to hold text.
??? example
@@ -187,6 +187,41 @@ as an array of uint8 values. The library implements this translation.
}
```
### BON8
[BON8](binary_formats/bon8.md) neither supports binary values nor subtypes. The library serializes binary values as an
array of integers.
??? example
Code:
```cpp
// create a binary value of subtype 42 (will be ignored in BON8)
json j;
j["binary"] = json::binary({0xCA, 0xFE, 0xBA, 0xBE}, 42);
// convert to BON8
auto v = json::to_bon8(j);
```
`v` is a `std::vector<std::uint8_t>` with the following 16 elements:
```c
0x87 // object with 1 member
0x62 0x69 0x6E 0x61 0x72 0x79 // "binary"
0x84 // array with 4 elements
0xC3 0x22 0xC3 0x56 0xC3 0x12 0xC3 0x16 // content (each byte as a 2-byte integer)
```
Note that the subtype is lost, and deserializing `v` would yield the following value:
```json
{
"binary": [202, 254, 186, 190]
}
```
### BSON
[BSON](binary_formats/bson.md) supports binary values and subtypes. If a subtype is given, it is used and added as an
@@ -43,6 +43,23 @@ json jPi = 3.14;
assert(jPi.get<TaskState>() == TS_INVALID );
```
## Maps with enum keys
By default, maps with enum keys, such as `std::map<TaskState, std::string>`, are stored as arrays of `[key, value]`
pairs, because JSON object keys must be strings. Define
[`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) before including the
library to store them as objects, with the keys converted by the enum's `to_json()` function:
```cpp
std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
json j = m;
// default: [["stopped","aa"],["completed","bb"]]
// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS: {"completed":"bb","stopped":"aa"}
```
Either form can be read back, with or without the macro.
## Notes
Just as in [Arbitrary Type Conversions](arbitrary_types.md) above,
+2 -2
View File
@@ -35,8 +35,8 @@ C++ types, and finally serialize it again.
- [Serialization](serialization.md) — turn a value back into JSON text with [`dump`](../api/basic_json/dump.md),
including pretty-printing and handling of non-ASCII and invalid UTF-8.
- [Binary formats](binary_formats/index.md) — encode values more compactly as
[BJData](binary_formats/bjdata.md), [BSON](binary_formats/bson.md), [CBOR](binary_formats/cbor.md),
[MessagePack](binary_formats/messagepack.md), or [UBJSON](binary_formats/ubjson.md).
[BJData](binary_formats/bjdata.md), [BON8](binary_formats/bon8.md), [BSON](binary_formats/bson.md),
[CBOR](binary_formats/cbor.md), [MessagePack](binary_formats/messagepack.md), or [UBJSON](binary_formats/ubjson.md).
- [Binary values](binary_values.md) — store and exchange raw byte sequences.
## How values are stored and configured
+19 -2
View File
@@ -93,11 +93,21 @@ See [full documentation of `JSON_NO_IO`](../api/macros/json_no_io.md).
## `JSON_NO_THREAD_LOCAL`
When defined, the library does not use `#!cpp thread_local` storage. Copying a value then always avoids the call stack
rather than descending into a bounded number of levels first, which is slower but yields the same values.
When defined, the library does not use `#!cpp thread_local` storage. Copying a value and comparing two values then
always avoid the call stack rather than descending into a bounded number of levels first, which is slower but yields the
same values and the same comparisons.
See [full documentation of `JSON_NO_THREAD_LOCAL`](../api/macros/json_no_thread_local.md).
## `JSON_PRECISE_STREAM_POSITION`
When defined to `1`, [`operator>>`](../api/operator_gtgt.md) and non-strict
[`sax_parse`](../api/basic_json/sax_parse.md) leave an input stream positioned right after the parsed value, instead of
also consuming the character that terminates a number. The default value is `0`, which preserves the existing behavior;
this is planned to become the default in version 4.0.0.
See [full documentation of `JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md).
## `JSON_SKIP_LIBRARY_VERSION_CHECK`
When defined, the library will not create a compiler warning when a different version of the library was already
@@ -157,6 +167,13 @@ behavior is deprecated and switched off (`0`) by default.
See [full documentation of `JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md).
## `JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`
When defined to `1`, maps with enum keys (e.g., `std::map<E, T>`) are stored as objects, using the enum's conversion for
the keys, instead of arrays of `[key, value]` pairs. It is switched off (`0`) by default.
See [full documentation of `JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md).
## `JSON_USE_SIMDUTF`
When defined, UTF-8 validation of JSON strings read from contiguous byte input is delegated to the
+4
View File
@@ -18,6 +18,10 @@ The complete default namespace name is derived as follows:
- [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) defined non-zero appends `_dp`.
- [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) defined non-zero appends
`_bics`.
- [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) defined non-zero appends `_psp`.
- [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) defined non-zero appends `_snul`.
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) defined non-zero
appends `_ekmo`.
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
below.
+2 -1
View File
@@ -41,7 +41,8 @@ document followed by trailing bytes" is accepted rather than rejected. If you ar
reject any input that is not exactly one JSON document, prefer `parse`.
When using `operator>>` to read several concatenated values this way, a value that is a number must be followed by
whitespace, because `operator>>` consumes the character that terminates a number — see the
whitespace, because `operator>>` consumes the character that terminates a number, unless
[`JSON_PRECISE_STREAM_POSITION`](../../api/macros/json_precise_stream_position.md) is defined to `1` — see the
[`operator>>` notes](../../api/operator_gtgt.md#notes) for details and examples.
## SAX vs. DOM parsing
+1 -1
View File
@@ -117,7 +117,7 @@ For the [{fmt}](https://github.com/fmtlib/fmt) library, the library ships a
## Serializing to other formats
Besides JSON text, a value can also be serialized to the more compact [binary formats](binary_formats/index.md)
(BJData, BSON, CBOR, MessagePack, UBJSON).
(BJData, BON8, BSON, CBOR, MessagePack, UBJSON).
## See also
@@ -547,7 +547,7 @@ Grisu2 algorithm, which produces the shortest representation that round-trips. O
### Required for the binary formats
`NumberFloatType` must be `#!cpp float` or `#!cpp double`. The writers for
[CBOR, MessagePack, UBJSON, BJData, and BSON](../binary_formats/index.md) map a floating-point value onto an IEEE 754
[CBOR, MessagePack, UBJSON, BJData, BON8, and BSON](../binary_formats/index.md) map a floating-point value onto an IEEE 754
binary32 or binary64 field and have no encoding for `#!cpp long double`.
### Compatible types
@@ -562,7 +562,11 @@ binary32 or binary64 field and have no encoding for `#!cpp long double`.
## `AllocatorType`
`AllocatorType` is instantiated with **one** argument, for each of `object_t`, `array_t`, `string_t`, `binary_t`,
`basic_json`, and `#!cpp std::pair<const StringType, basic_json>`.
`basic_json`, `#!cpp std::pair<const StringType, basic_json>`, and `#!cpp std::pair<StringType, basic_json>`.
`AllocatorType` is not the only allocator a `basic_json` uses. It allocates the JSON values themselves, but most
temporary storage is allocated with `#!cpp std::allocator`. This includes the parser's stacks and the stacks that
process deeply nested values without recursion.
### Always required
+177 -35
View File
@@ -1,34 +1,125 @@
# Architecture
!!! info
This page is still under construction. Its goal is to provide a high-level overview of the library's architecture.
This should help new contributors to get an idea of the used concepts and where to make changes.
This page gives a high-level overview of the library's architecture. It should help new contributors to get an idea of
the used concepts and where to make changes.
## Overview
The main structure is class [nlohmann::basic_json](../api/basic_json/index.md).
The library is built around a single class template, [`nlohmann::basic_json`](../api/basic_json/index.md). A
`basic_json` value is a node in a tree of JSON values. All other components either create such a tree from an input
(parsing), write a tree to an output (serialization), or give access to it (iterators, JSON Pointer, conversions).
- public API
- container interface
- iterators
```mermaid
flowchart LR
input[/"input<br>(string, stream,<br>iterator range, file)"/]
ia["input adapter"]
lexer["lexer"]
parser["parser"]
breader["binary_reader"]
sax["SAX interface"]
value[("basic_json<br>value tree")]
serializer["serializer"]
bwriter["binary_writer"]
oa["output adapter"]
output[/"output<br>(string, stream,<br>vector)"/]
## Template specializations
input --> ia
ia --> lexer --> parser --> sax
ia --> breader --> sax
sax --> value
value --> serializer --> oa
value --> bwriter --> oa
oa --> output
```
- describe template parameters of `basic_json`
- [`json`](../api/json.md)
- [`ordered_json`](../api/ordered_json.md) via [`ordered_map`](../api/ordered_map.md)
- **JSON text** is read by an [input adapter](#input-adapters), tokenized by the lexer, and turned into SAX events by
the parser.
- **Binary formats** (BJData, BSON, CBOR, MessagePack, UBJSON) are read by an input adapter and turned into the same SAX
events by the `binary_reader`.
- A [SAX consumer](#sax-interface) receives the events. The one used by [`parse`](../api/basic_json/parse.md) builds a
`basic_json` value tree.
- The `serializer` (JSON text) or the `binary_writer` (binary formats) writes a value tree to an
[output adapter](#output-adapters).
## Source layout
The public headers are in [`include/nlohmann`](https://github.com/nlohmann/json/tree/develop/include/nlohmann):
- [`json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json.hpp) defines class [`basic_json`](../api/basic_json/index.md).
- [`json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json_fwd.hpp) contains forward declarations.
- [`adl_serializer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/adl_serializer.hpp), [`byte_container_with_subtype.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/byte_container_with_subtype.hpp), and [`ordered_map.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/ordered_map.hpp) define
[`adl_serializer`](../api/adl_serializer/index.md),
[`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md), and
[`ordered_map`](../api/ordered_map.md).
Everything else lives in [`detail/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail) and namespace `nlohmann::detail`, which is not part of the public API. Paths
below are relative to `include/nlohmann`.
| Component | Location |
|-----------|----------|
| Value type enumeration | [`detail/value_t.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/value_t.hpp) |
| Input adapters | [`detail/input/input_adapters.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/input_adapters.hpp) |
| Lexer | [`detail/input/lexer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/lexer.hpp), [`detail/input/number_parse.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/number_parse.hpp), [`detail/input/string_scan.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/string_scan.hpp) |
| Parser | [`detail/input/parser.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/parser.hpp) |
| SAX interface and DOM builders | [`detail/input/json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp) |
| Binary format readers | [`detail/input/binary_reader.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/binary_reader.hpp) |
| JSON serializer | [`detail/output/serializer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/serializer.hpp), [`detail/conversions/to_chars.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/to_chars.hpp) |
| Binary format writers | [`detail/output/binary_writer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/binary_writer.hpp) |
| Output adapters | [`detail/output/output_adapters.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/output_adapters.hpp) |
| Iterators | [`detail/iterators/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail/iterators) |
| Conversions from/to arbitrary types | [`detail/conversions/from_json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/from_json.hpp), [`detail/conversions/to_json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/to_json.hpp) |
| JSON Pointer | [`detail/json_pointer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/json_pointer.hpp) |
| Exceptions | [`detail/exceptions.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/exceptions.hpp) |
| Type traits and C++ feature backports | [`detail/meta/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail/meta) |
| Macros | [`detail/macro_scope.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/macro_scope.hpp), [`detail/macro_unscope.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/macro_unscope.hpp), [`detail/abi_macros.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/abi_macros.hpp) |
The single-header version [`single_include/nlohmann/json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp)
is generated from these files with `make amalgamate` and must not be edited by hand.
## Template parameters
[`basic_json`](../api/basic_json/index.md) is parameterized by the types it uses to store values and to convert from and to other types:
| Template parameter | Default | Used for |
|----------------------|-----------------------------|-------------------------------------------------------------------|
| `ObjectType` | `std::map` | objects, see [`object_t`](../api/basic_json/object_t.md) |
| `ArrayType` | `std::vector` | arrays, see [`array_t`](../api/basic_json/array_t.md) |
| `StringType` | `std::string` | strings and object keys, see [`string_t`](../api/basic_json/string_t.md) |
| `BooleanType` | `bool` | Booleans, see [`boolean_t`](../api/basic_json/boolean_t.md) |
| `NumberIntegerType` | `std::int64_t` | signed integers, see [`number_integer_t`](../api/basic_json/number_integer_t.md) |
| `NumberUnsignedType` | `std::uint64_t` | unsigned integers, see [`number_unsigned_t`](../api/basic_json/number_unsigned_t.md) |
| `NumberFloatType` | `double` | floating-point numbers, see [`number_float_t`](../api/basic_json/number_float_t.md) |
| `AllocatorType` | `std::allocator` | allocating objects, arrays, strings, and binary values |
| `JSONSerializer` | `adl_serializer` | conversions from/to other types, see [`adl_serializer`](../api/adl_serializer/index.md) |
| `BinaryType` | `std::vector<std::uint8_t>` | binary values, see [`binary_t`](../api/basic_json/binary_t.md) |
| `CustomBaseClass` | `void` | an optional base class, see [`json_base_class_t`](../api/basic_json/json_base_class_t.md) |
The library provides two specializations:
- [`json`](../api/json.md) uses all default template arguments.
- [`ordered_json`](../api/ordered_json.md) uses [`ordered_map`](../api/ordered_map.md) as `ObjectType` to keep the
insertion order of object keys.
The requirements on the template arguments are listed in
[Template Parameter Requirements](../features/types/template_parameters.md).
## Value storage
Values are stored as a tagged union of [value_t](../api/basic_json/value_t.md) and json_value.
Each [`basic_json`](../api/basic_json/index.md) value stores its content as a tagged union: an enumeration [`value_t`](../api/basic_json/value_t.md)
names the type of the value, and a union `json_value` holds the value itself. Both are members of the nested struct
`data`, which is the only data member `m_data` of `basic_json`:
```cpp
/// the type of the current element
value_t m_type = value_t::null;
struct data
{
/// the type of the current element
value_t m_type = value_t::null;
/// the value of the current element
json_value m_value = {};
/// the value of the current element
json_value m_value = {};
};
data m_data = {};
```
with
@@ -68,42 +159,83 @@ union json_value {
};
```
## Parsing inputs (deserialization)
Objects, arrays, strings, and binary values are allocated on the heap with `AllocatorType`, and the union only stores a
pointer to them. This keeps a `basic_json` value small: one pointer-sized union and one byte for the type. The class
maintains the invariant that the pointer matching `m_type` is never null; `assert_invariant()` checks it with
[runtime assertions](../features/assertions.md).
Input is read via **input adapters** that abstract a source with a common interface:
## Input adapters
Input is read via **input adapters** that abstract a source. Every input adapter provides this interface:
```cpp
/// read a single character
std::char_traits<char>::int_type get_character() noexcept;
/// the type of the characters in the input
using char_type = ...;
/// read multiple characters to a destination buffer and
/// returns the number of characters successfully read
/// read a single character; returns std::char_traits<char_type>::eof() at the end of the input
typename std::char_traits<char_type>::int_type get_character();
/// read up to count * sizeof(T) bytes into dest and return the number of bytes read
/// (used by the binary readers)
template<class T>
std::size_t get_elements(T* dest, std::size_t count = 1);
```
List examples of input adapters.
The lexer detects two optional extensions at compile time. Only `iterator_input_adapter` provides them, and only for
random-access input of single-byte characters:
## SAX Interface
- `supports_seek`, `get_consumed_count()`, and `copy_consumed_range()` let the lexer reconstruct already consumed input
for error messages instead of copying every character it reads.
- `supports_bulk_scan`, `bulk_data()`, `bulk_remaining()`, and `bulk_skip()` let the lexer scan strings directly in
contiguous memory, several bytes at a time.
TODO
The function `input_adapter` picks the right adapter for the argument passed to `parse`, `accept`, `sax_parse`, or the
`from_*` functions:
## Writing outputs (serialization)
- `iterator_input_adapter` reads from an iterator range, which also covers strings, containers, and pointers.
- `wide_string_input_adapter` reads from ranges of `wchar_t`, `char16_t`, or `char32_t` and converts them to UTF-8.
It cannot be used for binary formats; its `get_elements()` throws.
- `input_stream_adapter` reads from a `std::istream`.
- `file_input_adapter` reads from a `std::FILE*`.
## SAX interface
The parser does not build values itself. It reports what it reads as events to a [SAX](../features/parsing/sax_interface.md)
consumer, which implements the interface [`json_sax`](../api/json_sax/index.md): `null`, `boolean`, `number_integer`,
`number_unsigned`, `number_float`, `string`, `binary`, `start_object`, `key`, `end_object`, `start_array`, `end_array`,
and `parse_error`.
The library comes with two consumers in `detail/input/json_sax.hpp`:
- `json_sax_dom_parser` builds a [`basic_json`](../api/basic_json/index.md) value tree. [`parse`](../api/basic_json/parse.md) uses it.
- `json_sax_dom_callback_parser` does the same, but calls a [parser callback](../features/parsing/parser_callbacks.md)
for each event, which can skip values. `parse` uses it when a callback is given.
The `binary_reader` emits the same events for binary formats, so [`sax_parse`](../api/basic_json/sax_parse.md) works
with a user-defined consumer for JSON and for all binary formats alike.
## Output adapters
Output is written via **output adapters**:
```cpp
template<typename T>
void write_character(CharType c);
template<typename CharType>
void write_characters(const CharType* s, std::size_t length);
```
List examples of output adapters.
The `serializer` (used by [`dump`](../api/basic_json/dump.md) and [`operator<<`](../api/operator_ltlt.md)) and the
`binary_writer` (used by the `to_*` functions) write to one of these adapters:
- `output_vector_adapter` appends to a `std::vector`.
- `output_stream_adapter` writes to a `std::ostream`.
- `output_string_adapter` appends to a string.
## Value conversion
Values are converted from and to other types with the `JSONSerializer` template parameter. The default,
[`adl_serializer`](../api/adl_serializer/index.md), calls the free functions
```cpp
template<class T>
void to_json(basic_json& j, const T& t);
@@ -112,13 +244,23 @@ template<class T>
void from_json(const basic_json& j, T& t);
```
found by argument-dependent lookup. The library defines them for standard types in `detail/conversions`; users add them
for their own types, see [Arbitrary Type Conversions](../features/arbitrary_types.md). The
[serialization macros](../features/macros.md) generate these functions.
## Additional features
- JSON Pointers
- Binary formats
- Custom base class
- Conversion macros
- [JSON Pointer](../features/json_pointer.md) (class `json_pointer`) addresses values inside a tree. It is also the
basis of [JSON Patch](../features/json_patch.md).
- [Binary formats](../features/binary_formats/index.md) are read by `binary_reader` and written by `binary_writer`.
- A [custom base class](../api/basic_json/json_base_class_t.md) can add members to every [`basic_json`](../api/basic_json/index.md) value.
- [Serialization macros](../features/macros.md) generate `to_json` and `from_json` functions for user-defined types.
## Details namespace
- C++ feature backports
Namespace `nlohmann::detail` contains all implementation details. It is not part of the public API and may change in any
release. Besides the components above, it contains:
- type traits to detect the capabilities of user-defined types (`detail/meta/type_traits.hpp`),
- backports of C++14/17 features to C++11 (`detail/meta/cpp_future.hpp`), and
- helpers such as `string_concat` and `string_escape`.
+1 -1
View File
@@ -6,7 +6,7 @@ There are myriads of [JSON](https://json.org) libraries out there, and each may
- **Trivial integration**. Our whole code consists of a single header file [`json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp). That's it. No library, no subproject, no dependencies, no complex build system. The class is written in vanilla C++11. All in all, everything should require no adjustment of your compiler flags or project settings.
- **Serious testing**. Our class is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/tests/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](http://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests against all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [Core Infrastructure Initiative (CII) best practices](https://bestpractices.coreinfrastructure.org/projects/289).
- **Serious testing**. Our class is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/tests/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](http://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests against all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [OpenSSF Best Practices](https://www.bestpractices.dev/projects/289).
Other aspects were not so important to us:
+31 -5
View File
@@ -340,7 +340,8 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
### json.exception.parse_error.113
A string could not be read from a [binary format](../features/binary_formats/index.md): either a value that is not a
string was read where one was required (for instance as a map key), or the string's length specification is invalid.
string was read where one was required (for instance as a map key), the string's length specification is invalid, or
the string's bytes are not valid UTF-8.
!!! failure "Example messages"
@@ -356,6 +357,9 @@ string was read where one was required (for instance as a map key), or the strin
```
[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData string: string length must not be negative
```
```
[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing CBOR string: invalid string: ill-formed UTF-8 byte
```
### json.exception.parse_error.114
@@ -585,6 +589,9 @@ During implicit or explicit value conversion, the JSON type must be compatible w
[json.exception.type_error.302] type must be string, but is object
```
This exception is also thrown with [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md)
if a key of a map with enum keys is not converted to a string, for instance, because the enum is stored as an integer.
### json.exception.type_error.303
To retrieve a reference to a value stored in a `basic_json` object with `get_ref`, the type of the reference must match the value type. For instance, for a JSON array, the `ReferenceType` must be `array_t &`.
@@ -771,6 +778,19 @@ The dynamic type of the object cannot be represented in the requested serializat
Encapsulate the JSON value in an object. That is, instead of serializing `#!json true`, serialize `#!json {"value": true}`
### json.exception.type_error.318
With [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md), a map with enum
keys is stored as an object. This exception is thrown if two of its keys are converted to the same string, so one of the
entries would be lost. This happens, for instance, if [`NLOHMANN_JSON_SERIALIZE_ENUM`](../api/macros/nlohmann_json_serialize_enum.md)
does not list an enumerator and it is therefore converted like the first listed one.
!!! failure "Example message"
```
[json.exception.type_error.318] duplicate object key 'red'
```
## Out of range
This exception is thrown in case a library function is called on an input parameter that exceeds the expected range, for instance, in the case of array indices or nonexisting object keys.
@@ -928,19 +948,25 @@ A JSON Patch `add` operation cannot be applied because the target location's par
### json.exception.out_of_range.412
BSON stores the length of documents, arrays, strings, and binary values in a signed 32-bit integer. This exception is thrown when a value is too large to be described by such a length field.
BSON stores the length of documents, arrays, strings, and binary values in a signed 32-bit integer, and MessagePack
stores the length of strings, binary values, arrays, and objects in at most an unsigned 32-bit integer. This exception
is thrown when a value is too large to be described by such a length field.
!!! failure "Example message"
!!! failure "Example messages"
```
BSON length 2147483661 exceeds maximum of 2147483647
```
```
MessagePack length 4294967296 exceeds maximum of 4294967295
```
!!! note
This exception was added in version 3.13.0. Before that, the length was silently truncated, and
This exception was added in version 3.13.0. Before that, the BSON length was silently truncated, and
[`to_bson`](../api/basic_json/to_bson.md) produced documents with negative length prefixes that
[`from_bson`](../api/basic_json/from_bson.md) rejected.
[`from_bson`](../api/basic_json/from_bson.md) rejected; [`to_msgpack`](../api/basic_json/to_msgpack.md) wrote such
a value without any length, producing output that could not be read back.
### json.exception.out_of_range.413
+1
View File
@@ -5,6 +5,7 @@
*[ASCII]: American Standard Code for Information Interchange
*[BDFL]: Benevolent Dictator for Life
*[BJData]: Binary JData
*[BON8]: Binary Object Notation 8
*[BSON]: Binary JSON
*[CBOR]: Concise Binary Object Representation
*[CC0]: Creative Commons Zero
+8 -1
View File
@@ -63,6 +63,7 @@ nav:
- Binary Formats:
- features/binary_formats/index.md
- features/binary_formats/bjdata.md
- features/binary_formats/bon8.md
- features/binary_formats/bson.md
- features/binary_formats/cbor.md
- features/binary_formats/messagepack.md
@@ -142,6 +143,7 @@ nav:
- 'flatten': api/basic_json/flatten.md
- 'format_as': api/basic_json/format_as.md
- 'from_bjdata': api/basic_json/from_bjdata.md
- 'from_bon8': api/basic_json/from_bon8.md
- 'from_bson': api/basic_json/from_bson.md
- 'from_cbor': api/basic_json/from_cbor.md
- 'from_msgpack': api/basic_json/from_msgpack.md
@@ -213,6 +215,7 @@ nav:
- 'swap': api/basic_json/swap.md
- 'std::swap&lt;basic_json&gt;': api/basic_json/std_swap.md
- 'to_bjdata': api/basic_json/to_bjdata.md
- 'to_bon8': api/basic_json/to_bon8.md
- 'to_bson': api/basic_json/to_bson.md
- 'to_cbor': api/basic_json/to_cbor.md
- 'to_msgpack': api/basic_json/to_msgpack.md
@@ -293,12 +296,14 @@ nav:
- 'JSON_NOEXCEPTION': api/macros/json_noexception.md
- 'JSON_NO_IO': api/macros/json_no_io.md
- 'JSON_NO_THREAD_LOCAL': api/macros/json_no_thread_local.md
- 'JSON_PRECISE_STREAM_POSITION': api/macros/json_precise_stream_position.md
- 'JSON_SKIP_LIBRARY_VERSION_CHECK': api/macros/json_skip_library_version_check.md
- 'JSON_SKIP_UNSUPPORTED_COMPILER_CHECK': api/macros/json_skip_unsupported_compiler_check.md
- 'JSON_STRICT_NUL_HANDLING': api/macros/json_strict_nul_handling.md
- 'JSON_USE_GLOBAL_UDLS': api/macros/json_use_global_udls.md
- 'JSON_USE_IMPLICIT_CONVERSIONS': api/macros/json_use_implicit_conversions.md
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
- 'JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS': api/macros/json_use_objects_for_enum_keyed_maps.md
- 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
- 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
- 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md
@@ -317,7 +322,9 @@ nav:
- community/contribution_guidelines.md
- community/quality_assurance.md
- community/governance.md
- community/roadmap.md
- community/security_policy.md
- community/assurance_case.md
# Extras
extra:
@@ -409,7 +416,7 @@ plugins:
markdown_description: >
JSON for Modern C++ is a C++11 header-only library implementing a JSON
value type with an STL-like API, JSON Pointer/Patch, CBOR/MessagePack/
BSON/UBJSON/BJData binary format support, and a SAX-style parser interface.
BSON/UBJSON/BJData/BON8 binary format support, and a SAX-style parser interface.
sections:
Home:
- index.md
+37 -4
View File
@@ -38,6 +38,18 @@
#define JSON_BRACE_INIT_COPY_SEMANTICS 0
#endif
#ifndef JSON_PRECISE_STREAM_POSITION
#define JSON_PRECISE_STREAM_POSITION 0
#endif
#ifndef JSON_STRICT_NUL_HANDLING
#define JSON_STRICT_NUL_HANDLING 0
#endif
#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
#endif
#if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else
@@ -62,21 +74,42 @@
#define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS
#endif
#if JSON_PRECISE_STREAM_POSITION
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION _psp
#else
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION
#endif
#if JSON_STRICT_NUL_HANDLING
#define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING _snul
#else
#define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
#else
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d) json_abi ## a ## b ## c ## d
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \
NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS)
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
@@ -550,11 +550,40 @@ auto from_json(BasicJsonType&& j, TupleRelated&& t)
return from_json_tuple_impl(std::forward<BasicJsonType>(j), std::forward<TupleRelated>(t), priority_tag<3> {});
}
// read a map with enum keys from an object, using the enum's own from_json for
// the keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); this is the form written
// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
template<typename BasicJsonType, typename Map>
inline bool from_json_enum_keyed_object(const BasicJsonType& j, Map& m, std::true_type /*key is enum*/)
{
if (!j.is_object())
{
return false;
}
m.clear();
for (const auto& p : *j.template get_ptr<const typename BasicJsonType::object_t*>())
{
m.emplace(BasicJsonType(p.first).template get<typename Map::key_type>(), p.second.template get<typename Map::mapped_type>());
}
return true;
}
template<typename BasicJsonType, typename Map>
inline bool from_json_enum_keyed_object(const BasicJsonType& /*j*/, Map& /*m*/, std::false_type /*key is enum*/)
{
return false;
}
template < typename BasicJsonType, typename Key, typename Value, typename Compare, typename Allocator,
typename = enable_if_t < !std::is_constructible <
typename BasicJsonType::string_t, Key >::value >>
inline void from_json(const BasicJsonType& j, std::map<Key, Value, Compare, Allocator>& m)
{
// NOLINTNEXTLINE(modernize-type-traits) we use C++11
if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
{
return;
}
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
@@ -575,6 +604,11 @@ template < typename BasicJsonType, typename Key, typename Value, typename Hash,
typename BasicJsonType::string_t, Key >::value >>
inline void from_json(const BasicJsonType& j, std::unordered_map<Key, Value, Hash, KeyEqual, Allocator>& m)
{
// NOLINTNEXTLINE(modernize-type-traits) we use C++11
if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
{
return;
}
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
@@ -23,6 +23,7 @@
#include <valarray> // valarray
#include <vector> // vector
#include <nlohmann/detail/exceptions.hpp>
#include <nlohmann/detail/iterators/iteration_proxy.hpp>
#include <nlohmann/detail/meta/cpp_future.hpp>
#include <nlohmann/detail/meta/std_fs.hpp>
@@ -381,6 +382,9 @@ template < typename BasicJsonType, typename CompatibleArrayType,
!is_basic_json<CompatibleArrayType>::value
#if JSON_HAS_RANGES && !defined(__MINGW32__)
&& !is_compatible_range_view<CompatibleArrayType>::value
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
&& !is_enum_keyed_map<CompatibleArrayType>::value
#endif
,
int > = 0 >
@@ -435,6 +439,33 @@ inline void to_json(BasicJsonType& j, const CompatibleObjectType& obj)
external_constructor<value_t::object>::construct(j, obj);
}
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
// store a map with enum keys as an object, using the enum's own to_json for the
// keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); without the macro, such maps
// are stored as arrays of [key, value] pairs
template < typename BasicJsonType, typename EnumKeyedMap,
enable_if_t < is_enum_keyed_map<EnumKeyedMap>::value&& !is_basic_json<EnumKeyedMap>::value, int > = 0 >
inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
{
typename BasicJsonType::object_t obj;
for (const auto& p : map)
{
BasicJsonType key = p.first;
if (JSON_HEDLEY_UNLIKELY(!key.is_string()))
{
JSON_THROW(type_error::create(302, concat("type must be string, but is ", key.type_name()), &key));
}
auto& key_string = *key.template get_ptr<typename BasicJsonType::string_t*>();
if (JSON_HEDLEY_UNLIKELY(!obj.emplace(key_string, BasicJsonType(p.second)).second))
{
JSON_THROW(type_error::create(318, concat("duplicate object key '", key_string, "'"), &key));
}
}
external_constructor<value_t::object>::construct(j, std::move(obj));
}
#endif
template<typename BasicJsonType>
inline void to_json(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
{
+672 -10
View File
@@ -28,10 +28,12 @@
#include <nlohmann/detail/input/input_adapters.hpp>
#include <nlohmann/detail/input/json_sax.hpp>
#include <nlohmann/detail/input/lexer.hpp>
#include <nlohmann/detail/input/string_scan.hpp>
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/is_sax.hpp>
#include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_utils.hpp>
#include <nlohmann/detail/value_t.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -43,7 +45,7 @@ enum class cbor_tag_handler_t
{
error, ///< throw a parse_error exception in case of a tag
ignore, ///< ignore tags
store ///< store tags as binary type
store ///< store tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored
};
/*!
@@ -83,7 +85,7 @@ JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 2
///////////////////
/*!
@brief deserialization of CBOR, MessagePack, and UBJSON values
@brief deserialization of BJData, BON8, BSON, CBOR, MessagePack, and UBJSON values
*/
template<typename BasicJsonType, typename InputAdapterType, typename SAX = json_sax_dom_parser<BasicJsonType, InputAdapterType>>
class binary_reader
@@ -97,6 +99,11 @@ class binary_reader
using char_type = typename InputAdapterType::char_type;
using char_int_type = typename char_traits<char_type>::int_type;
/// whether the input is a contiguous block of bytes that can be inspected
/// and consumed in bulk (as in the lexer); used by @ref get_bon8_string_bulk
static constexpr bool bulk_scan =
input_adapter_supports_bulk_scan<InputAdapterType>(is_detected<detect_supports_bulk_scan, InputAdapterType> {});
public:
/*!
@brief create a binary reader
@@ -131,6 +138,7 @@ class binary_reader
{
sax = sax_;
container_stack.clear();
bon8_pushback_size = 0;
bool result = false;
switch (format)
@@ -152,6 +160,10 @@ class binary_reader
result = parse_ubjson_internal();
break;
case input_format_t::bon8:
result = parse_bon8_internal();
break;
case input_format_t::json: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
@@ -164,6 +176,11 @@ class binary_reader
{
get_ignore_noop();
}
else if (input_format == input_format_t::bon8)
{
// a string that ends a container hands back the byte after it
get_bon8();
}
else
{
get();
@@ -395,6 +412,11 @@ class binary_reader
*/
bool get_bson_cstr(string_t& result)
{
if (get_bson_cstr_bulk(result, std::integral_constant<bool, bulk_scan> {}))
{
return true;
}
auto out = std::back_inserter(result);
while (true)
{
@@ -411,6 +433,46 @@ class binary_reader
}
}
/*!
@brief read a C-style string from contiguous input in one step
@param[in,out] result the string to append to
@return whether the string was read; if the input has no \x00-byte, nothing
is read, and @ref get_bson_cstr reports the end of the input
*/
bool get_bson_cstr_bulk(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining == 0)
{
return false;
}
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
// a plain loop rather than std::memchr: most keys are short (array
// indices are keys, too), and the call would cost more than it saves
std::size_t length = 0;
while (length < remaining && data[length] != 0x00)
{
++length;
}
if (length == remaining)
{
return false;
}
result.append(reinterpret_cast<const typename string_t::value_type*>(data), length);
// consume the string and its \x00-byte, as the byte-wise path does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
/// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, std::false_type /*bulk*/) const noexcept
{
return false;
}
/*!
@brief Parses a zero-terminated string of length @a len from the BSON
input.
@@ -432,7 +494,21 @@ class binary_reader
exception_message(input_format_t::bson, concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr));
}
return get_string(input_format_t::bson, len - static_cast<NumberType>(1), result) && get() != char_traits<char_type>::eof();
if (JSON_HEDLEY_UNLIKELY(!get_string(input_format_t::bson, len - static_cast<NumberType>(1), result)))
{
return false;
}
if (JSON_HEDLEY_UNLIKELY(get() != 0x00))
{
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(input_format_t::bson,
"BSON string is not null-terminated",
"string"), nullptr));
}
return true;
}
/*!
@@ -550,8 +626,6 @@ class binary_reader
}
}
//////////
// CBOR //
//////////
@@ -579,14 +653,18 @@ class binary_reader
input (true) or whether the last read character should
be considered instead (false)
@param[in] tag_handler how CBOR tags should be treated
@param[out] tag_pending whether a tag was parsed and its value follows
@param[out] item_read whether the tagged value's initial byte is already in current
@return whether a valid CBOR value was passed to the SAX parser
*/
bool parse_cbor_value(const bool get_char,
const cbor_tag_handler_t tag_handler,
bool& tag_pending)
bool& tag_pending,
bool& item_read)
{
tag_pending = false;
item_read = false;
switch (get_char ? get() : current)
{
@@ -1008,7 +1086,17 @@ class binary_reader
}
}
get();
return get_cbor_binary(b) && sax->binary(b);
// a byte string (the heads accepted by get_cbor_binary) keeps the tag as subtype
if ((current >= 0x40 && current <= 0x5B) || current == 0x5F)
{
return get_cbor_binary(b) && sax->binary(b);
}
// not a byte string: the tagged value, whose first byte
// was just read, is read by the caller like for ignore
tag_pending = true;
item_read = true;
return true;
}
default: // LCOV_EXCL_LINE
@@ -1490,13 +1578,14 @@ class binary_reader
// a tag is not a value of its own: read on until the tagged value
bool tag_pending = false;
bool item_read = false;
do
{
if (JSON_HEDLEY_UNLIKELY(!parse_cbor_value(fetch, tag_handler, tag_pending)))
if (JSON_HEDLEY_UNLIKELY(!parse_cbor_value(fetch, tag_handler, tag_pending, item_read)))
{
return false;
}
fetch = true;
fetch = !item_read;
}
while (tag_pending);
@@ -3171,6 +3260,549 @@ class binary_reader
}
}
//////////
// BON8 //
//////////
/*!
@brief get the next byte of a BON8 value
A BON8 string has no length prefix and no mandatory terminator: it ends at
the first byte that cannot continue it, which is already the first byte (or,
for an integer that begins with a UTF-8 lead byte, the first two bytes) of
whatever follows. The string reader hands those bytes back with
@ref unget_bon8, and every BON8 read goes through this function so that
they are seen again.
@return character read from the input
*/
char_int_type get_bon8()
{
if (bon8_pushback_size != 0)
{
++chars_read;
return current = bon8_pushback[--bon8_pushback_size];
}
return get();
}
/*!
@brief hand a byte back so that the next @ref get_bon8 returns it again
@param[in] c the byte to hand back; bytes handed back are returned in
reverse order
*/
void unget_bon8(const char_int_type c)
{
// At most two bytes are ever handed back: a byte is only handed back
// right after it was read with get_bon8(), and the only place that
// hands back two bytes (a lead byte and the byte after it) read both
// of them in a row, which emptied the buffer first. This is an
// invariant of the reader rather than a property of the input, so
// an assertion suffices (the fuzzers are built with assertions).
JSON_ASSERT(bon8_pushback_size < bon8_pushback.size());
bon8_pushback[bon8_pushback_size++] = c;
--chars_read;
}
/*!
@param[in] c a byte
@return whether @a c is a UTF-8 continuation byte (0x80..0xBF)
*/
static constexpr bool is_bon8_continuation(const char_int_type c) noexcept
{
return 0x80 <= c && c <= 0xBF;
}
/*!
@brief report a parse error at the last read byte
@param[in] detail a detailed error message
@param[in] context further context information
@return false
*/
bool bon8_error(const std::string& detail, const char* context)
{
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr));
}
/*!
@brief read a BON8 value and everything nested inside it
Reads values until the one that was begun here is complete, resuming the
enclosing container after each element, so that the nesting depth of the
input costs heap rather than native stack (see #5104).
@return whether reading the value succeeded
*/
bool parse_bon8_internal()
{
// the key currently being read; hoisted out of the loop so that its
// capacity is reused across elements and across nesting levels
string_t key;
while (true)
{
if (!container_stack.empty())
{
// a copy, not a reference: it must stay valid across the
// pop_back() below, which destroys the container_stack element
// it would otherwise alias
const container_frame top = container_stack.back();
bool at_end = false;
if (top.remaining != npos)
{
// counted container (0x80..0x84, 0x86..0x8A): it ends once
// its elements have been read
at_end = (top.remaining == 0);
if (!at_end)
{
// claim the element about to be read
--container_stack.back().remaining;
}
}
else
{
// container 0x85 or 0x8B: it ends at an end-of-container
// marker (0xFE); any other byte begins the next element
at_end = (get_bon8() == 0xFE);
if (!at_end)
{
unget_bon8(current);
}
}
if (at_end)
{
container_stack.pop_back();
if (JSON_HEDLEY_UNLIKELY(top.is_object ? !sax->end_object() : !sax->end_array()))
{
return false;
}
// the value begun here is complete once its container is
if (container_stack.empty())
{
return true;
}
continue;
}
if (top.is_object)
{
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bon8_key(key) || !sax->key(key)))
{
return false;
}
}
}
if (JSON_HEDLEY_UNLIKELY(!parse_bon8_value()))
{
return false;
}
// a value that opened a container left it on the stack; one that
// did not, and that was not inside a container, was the whole value
if (container_stack.empty())
{
return true;
}
}
}
/*!
@brief read one BON8 value
Reads a single value and passes it to the SAX parser. A value that begins
a container is not read to its end: the container is opened with
@ref enter_container and its elements are read by
@ref parse_bon8_internal, so that nesting does not consume native stack.
@return whether reading the value succeeded
*/
bool parse_bon8_value()
{
const auto byte = get_bon8();
if (byte == char_traits<char_type>::eof())
{
return unexpect_eof(input_format_t::bon8, "value");
}
// string: ASCII character
if (byte <= 0x7F)
{
string_t s;
unget_bon8(byte);
return get_bon8_string(s) && sax->string(s);
}
// array with 0..4 elements
if (byte <= 0x84)
{
return enter_array(static_cast<std::size_t>(byte - 0x80));
}
// array terminated by 0xFE
if (byte == 0x85)
{
return enter_array(npos);
}
// object with 0..4 members
if (byte <= 0x8A)
{
return enter_object(static_cast<std::size_t>(byte - 0x86));
}
switch (byte)
{
case 0x8B: // object terminated by 0xFE
return enter_object(npos);
case 0x8C: // int32
{
std::int32_t number{};
return get_number(input_format_t::bon8, number) && emit_bon8_integer(number);
}
case 0x8D: // int64
{
std::int64_t number{};
return get_number(input_format_t::bon8, number) && emit_bon8_integer(number);
}
case 0x8E: // binary32
{
float number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0x8F: // binary64
{
double number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xF8:
return sax->boolean(false);
case 0xF9:
return sax->boolean(true);
case 0xFA:
return sax->null();
case 0xFB:
return sax->number_float(static_cast<number_float_t>(-1.0), "");
case 0xFC:
return sax->number_float(static_cast<number_float_t>(0.0), "");
case 0xFD:
return sax->number_float(static_cast<number_float_t>(1.0), "");
case 0xFF: // empty string
{
string_t s;
return sax->string(s);
}
default:
break;
}
// integer 0..39
if (byte <= 0xB7)
{
return sax->number_unsigned(static_cast<number_unsigned_t>(byte - 0x90));
}
// integer -1..-10
if (byte <= 0xC1)
{
return sax->number_integer(-1 - static_cast<number_integer_t>(byte - 0xB8));
}
// 0xC2..0xF7: a UTF-8 lead byte begins a string if a continuation
// byte follows and an integer otherwise
if (byte <= 0xF7)
{
const auto second = get_bon8();
if (is_bon8_continuation(second))
{
string_t s;
unget_bon8(second);
unget_bon8(byte);
return get_bon8_string(s) && sax->string(s);
}
return get_bon8_integer(byte, second);
}
// 0xFE: end of container where a value is expected
return bon8_error("invalid byte", "value");
}
/*!
@brief pass an integer to the SAX parser
Non-negative integers are passed as unsigned, negative integers as signed
numbers, like the other binary formats do.
@param[in] number the integer
@return whether the SAX parser accepted the value
*/
bool emit_bon8_integer(const std::int64_t number)
{
if (number >= 0)
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return sax->number_integer(static_cast<number_integer_t>(number));
}
/*!
@brief read an integer encoded in 2..4 bytes
The first byte is a UTF-8 lead byte (0xC2..0xF7) that is followed by a
byte that is not a continuation byte: 0x00..0x7F for positive and
0xC0..0xFF for negative integers. The lead byte's low bits and the second
byte's low 7 (positive) or 6 (negative) bits are the most significant bits
of the value; 3- and 4-byte integers add one or two full bytes. Each range
starts where the shorter one ends, so no value has two encodings of the
same length.
@param[in] lead the first byte (0xC2..0xF7)
@param[in] second the second byte
@return whether reading the integer succeeded
*/
bool get_bon8_integer(const char_int_type lead, const char_int_type second)
{
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number")))
{
return false;
}
const bool negative = second >= 0xC0;
auto value = static_cast<std::int64_t>(negative ? (second & 0x3F) : second);
std::int64_t offset = 0;
int extra_bytes = 0;
if (lead <= 0xDF)
{
value |= static_cast<std::int64_t>(lead - 0xC2) << (negative ? 6 : 7);
offset = negative ? 11 : 40;
}
else if (lead <= 0xEF)
{
value |= static_cast<std::int64_t>(lead & 0x0F) << (negative ? 6 : 7);
offset = negative ? 1931 : 3880;
extra_bytes = 1;
}
else
{
value |= static_cast<std::int64_t>(lead & 0x07) << (negative ? 6 : 7);
offset = negative ? 264075 : 528168;
extra_bytes = 2;
}
for (int i = 0; i < extra_bytes; ++i)
{
if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits<char_type>::eof()))
{
return unexpect_eof(input_format_t::bon8, "number");
}
value = (value << 8) | static_cast<std::int64_t>(current);
}
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
}
/*!
@brief read an object key
A key must be a string, so its first byte must be an ASCII character, a
UTF-8 lead byte followed by a continuation byte, or 0xFF (empty string).
@param[out] result the key
@return whether reading the key succeeded
*/
bool get_bon8_key(string_t& result)
{
const auto byte = get_bon8();
if (byte == char_traits<char_type>::eof())
{
return unexpect_eof(input_format_t::bon8, "key");
}
if (byte == 0xFF)
{
return true;
}
if (byte <= 0x7F)
{
unget_bon8(byte);
return get_bon8_string(result);
}
if (0xC2 <= byte && byte <= 0xF7)
{
const auto second = get_bon8();
unget_bon8(second);
if (is_bon8_continuation(second))
{
unget_bon8(byte);
return get_bon8_string(result);
}
// an integer: report its first byte rather than the one after it
current = byte;
}
return bon8_error("expected a string; last byte", "key");
}
/*!
@brief append the run of valid UTF-8 at the read position to a string
For contiguous input, the ASCII characters and complete well-formed UTF-8
sequences at the read position are appended to @a result in one step. The
byte that stops the run (an end-of-string marker, the first byte of the
next value, or an ill-formed byte) is left for @ref get_bon8_string, so
that strings end and errors are reported exactly as without this step.
@param[in,out] result the string to append to
*/
void get_bon8_string_bulk(string_t& result, std::true_type /*bulk*/)
{
// bytes handed back must be read through get_bon8() first
if (bon8_pushback_size != 0)
{
return;
}
const std::size_t remaining = ia.bulk_remaining();
if (remaining == 0)
{
return;
}
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = valid_utf8_prefix(data, remaining);
if (length != 0)
{
result.append(reinterpret_cast<const typename string_t::value_type*>(data), length);
ia.bulk_skip(length);
chars_read += length;
}
}
/// input that is not contiguous: strings are read byte by byte
void get_bon8_string_bulk(string_t& /*result*/, std::false_type /*bulk*/) const noexcept {}
/*!
@brief read a string
Reads UTF-8 characters until an end-of-string marker (0xFF), which is
consumed, or a byte that cannot continue the string, which is handed back
to be read as the start of the next value. The string must be valid UTF-8,
and it must not end at the end of the input: the last string of a message
is always terminated by 0xFF.
@param[out] result the string
@return whether reading the string succeeded
*/
bool get_bon8_string(string_t& result)
{
while (true)
{
get_bon8_string_bulk(result, std::integral_constant<bool, bulk_scan> {});
const auto byte = get_bon8();
if (byte == char_traits<char_type>::eof())
{
return unexpect_eof(input_format_t::bon8, "string");
}
// end of string
if (byte == 0xFF)
{
return true;
}
// ASCII character
if (byte <= 0x7F)
{
result.push_back(static_cast<typename string_t::value_type>(byte));
continue;
}
// a byte that cannot begin a character ends the string and begins
// the next value
if (byte < 0xC2 || byte > 0xF7)
{
unget_bon8(byte);
return true;
}
// a lead byte ends the string if no continuation byte follows: it
// is then the first byte of an integer
const auto second = get_bon8();
if (!is_bon8_continuation(second))
{
unget_bon8(second);
unget_bon8(byte);
return true;
}
// the valid range of the second byte excludes overlong forms,
// surrogates, and code points above U+10FFFF
// (RFC 3629, section 4)
int continuation_bytes = 0;
bool valid_second = true;
if (byte <= 0xDF)
{
continuation_bytes = 1;
}
else if (byte <= 0xEF)
{
continuation_bytes = 2;
valid_second = (byte != 0xE0 || second >= 0xA0) && (byte != 0xED || second <= 0x9F);
}
else
{
continuation_bytes = 3;
valid_second = byte <= 0xF4 && (byte != 0xF0 || second >= 0x90) && (byte != 0xF4 || second <= 0x8F);
}
if (JSON_HEDLEY_UNLIKELY(!valid_second))
{
return bon8_error("invalid UTF-8 byte", "string");
}
result.push_back(static_cast<typename string_t::value_type>(byte));
result.push_back(static_cast<typename string_t::value_type>(second));
for (int i = 1; i < continuation_bytes; ++i)
{
if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits<char_type>::eof()))
{
return unexpect_eof(input_format_t::bon8, "string");
}
if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current)))
{
return bon8_error("invalid UTF-8 byte", "string");
}
result.push_back(static_cast<typename string_t::value_type>(current));
}
}
}
///////////////////////
// Utility functions //
///////////////////////
@@ -3304,7 +3936,28 @@ class binary_reader
const NumberType len,
string_t& result)
{
return get_bytes(format, len, "string", result);
// get_bytes() appends to result, and CBOR indefinite-length strings
// collect all their chunks in the same result; validating only the
// newly read bytes keeps the check linear in the input size
const std::size_t old_size = result.size();
if (JSON_HEDLEY_UNLIKELY(!get_bytes(format, len, "string", result)))
{
return false;
}
// RFC 8949 (CBOR) §3.1 and the MessagePack/BSON/UBJSON specifications
// all require text strings to be valid UTF-8; reject anything else
// right here so malformed input is caught at decode time instead of
// only surfacing later as a type_error.316 when the value is dumped
// (which would defeat allow_exceptions=false / strict discarding).
if (JSON_HEDLEY_UNLIKELY(!is_valid_utf8(result, old_size)))
{
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(113, chars_read,
exception_message(format, "invalid string: ill-formed UTF-8 byte", "string"), nullptr));
}
return true;
}
/*!
@@ -3448,6 +4101,10 @@ class binary_reader
error_msg += "BJData";
break;
case input_format_t::bon8:
error_msg += "BON8";
break;
case input_format_t::json: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
@@ -3480,6 +4137,11 @@ class binary_reader
/// the containers that have been opened and not closed yet; see @ref container_frame
std::vector<container_frame> container_stack{};
/// BON8: bytes read past the end of a string, returned again by @ref get_bon8
std::array<char_int_type, 2> bon8_pushback{{}};
/// BON8: number of bytes in @ref bon8_pushback
std::size_t bon8_pushback_size = 0;
// excluded markers in bjdata optimized type
#define JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_ \
make_array<char_int_type>('F', 'H', 'N', 'S', 'T', 'Z', '[', '{')
@@ -34,7 +34,7 @@ namespace detail
{
/// the supported input formats
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata };
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata, bon8 };
////////////////////
// input adapters //
@@ -101,6 +101,11 @@ class input_stream_adapter
// maintain ifstream flags, except eof
if (is != nullptr)
{
#if JSON_PRECISE_STREAM_POSITION
// consume the character last returned by get_character() unless it
// was given back with release_lookahead()
commit_lookahead();
#endif
is->clear(is->rdstate() & std::ios::eofbit);
}
}
@@ -114,6 +119,58 @@ class input_stream_adapter
input_stream_adapter& operator=(input_stream_adapter&) = delete;
input_stream_adapter& operator=(input_stream_adapter&&) = delete;
#if JSON_PRECISE_STREAM_POSITION
input_stream_adapter(input_stream_adapter&& rhs) noexcept
: is(rhs.is), sb(rhs.sb), lookahead(rhs.lookahead)
{
rhs.is = nullptr;
rhs.sb = nullptr;
rhs.lookahead = false;
}
// Whether the character last returned by get_character() can be given back
// to the input with release_lookahead().
static constexpr bool supports_lookahead = true;
// std::istream/std::streambuf use std::char_traits<char>::to_int_type, to
// ensure that std::char_traits<char>::eof() and the character 0xFF do not
// end up as the same value, e.g., 0xFFFFFFFF.
//
// The character is peeked rather than consumed: it is only stepped over
// once the next character is requested, or when the adapter is destroyed.
// Until then, release_lookahead() can leave it in the input.
std::char_traits<char>::int_type get_character()
{
if (lookahead)
{
// step over the character returned by the previous call
sb->sbumpc();
}
auto res = sb->sgetc();
// set eof manually, as we don't use the istream interface.
if (JSON_HEDLEY_UNLIKELY(res == std::char_traits<char>::eof()))
{
// there is nothing to step over next time
lookahead = false;
is->clear(is->rdstate() | std::ios::eofbit);
}
else
{
lookahead = true;
}
return res;
}
// Leave the character last returned by get_character() in the input, so
// that the next read from the stream - by this adapter or by the caller
// once parsing is done - sees it again. Unlike putting a consumed
// character back, this cannot fail.
void release_lookahead() noexcept
{
lookahead = false;
}
#else
input_stream_adapter(input_stream_adapter&& rhs) noexcept
: is(rhs.is), sb(rhs.sb)
{
@@ -124,6 +181,9 @@ class input_stream_adapter
// std::istream/std::streambuf use std::char_traits<char>::to_int_type, to
// ensure that std::char_traits<char>::eof() and the character 0xFF do not
// end up as the same value, e.g., 0xFFFFFFFF.
//
// The character is consumed, so the character that terminates a number
// stays consumed after parsing; see JSON_PRECISE_STREAM_POSITION.
std::char_traits<char>::int_type get_character()
{
auto res = sb->sbumpc();
@@ -134,10 +194,14 @@ class input_stream_adapter
}
return res;
}
#endif
template<class T>
std::size_t get_elements(T* dest, std::size_t count = 1)
{
#if JSON_PRECISE_STREAM_POSITION
commit_lookahead();
#endif
auto res = static_cast<std::size_t>(sb->sgetn(reinterpret_cast<char*>(dest), static_cast<std::streamsize>(count * sizeof(T))));
if (JSON_HEDLEY_UNLIKELY(res < count * sizeof(T)))
{
@@ -147,9 +211,27 @@ class input_stream_adapter
}
private:
#if JSON_PRECISE_STREAM_POSITION
// Step over the character last returned by get_character(). The character
// has already been peeked successfully, so for every streambuf with a get
// area this is a pointer increment that cannot fail.
void commit_lookahead()
{
if (lookahead)
{
lookahead = false;
sb->sbumpc();
}
}
#endif
/// the associated input stream
std::istream* is = nullptr;
std::streambuf* sb = nullptr;
#if JSON_PRECISE_STREAM_POSITION
/// whether get_character() peeked a character that is not consumed yet
bool lookahead = false;
#endif
};
#endif // JSON_NO_IO
+65
View File
@@ -127,6 +127,25 @@ constexpr bool input_adapter_supports_seek(std::false_type /*detected*/)
return false;
}
// Detect whether an input adapter reads with one character of lookahead that
// can be left in the input (see input_stream_adapter::supports_lookahead,
// which is only defined with JSON_PRECISE_STREAM_POSITION), detected like
// supports_seek above.
template<typename InputAdapterType>
using detect_supports_lookahead = decltype(InputAdapterType::supports_lookahead);
template<typename InputAdapterType>
constexpr bool input_adapter_supports_lookahead(std::true_type /*detected*/)
{
return InputAdapterType::supports_lookahead;
}
template<typename InputAdapterType>
constexpr bool input_adapter_supports_lookahead(std::false_type /*detected*/)
{
return false;
}
// Detect whether an input adapter exposes a contiguous byte block that the
// lexer can scan directly (see iterator_input_adapter::supports_bulk_scan).
// Adapters without the flag - file, stream, wide-string, user-defined - fall
@@ -167,6 +186,12 @@ class lexer : public lexer_base<BasicJsonType>
static constexpr bool lazy_token_string =
input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {});
/// whether a simulated unget can be passed on to the input adapter, which
/// then leaves the character in the input; see
/// input_adapter_supports_lookahead
static constexpr bool can_release_lookahead =
input_adapter_supports_lookahead<InputAdapterType>(is_detected<detect_supports_lookahead, InputAdapterType> {});
/// whether string scanning may bulk-consume runs of ordinary characters
/// directly from a contiguous input buffer (SWAR fast path). This requires
/// the token to be reconstructible lazily (lazy_token_string), so bypassing
@@ -1898,6 +1923,21 @@ scan_number_done:
uncapture_char(std::integral_constant<bool, lazy_token_string> {});
}
/// adapter without lookahead: nothing to do (see release_lookahead)
void release_lookahead_impl(std::false_type /*can_release*/) const noexcept {}
/// adapter with lookahead: leave the character in the input instead
void release_lookahead_impl(std::true_type /*can_release*/)
{
if (next_unget)
{
// the character is read from the input again rather than replayed
// from current, so the adapter must not step over it
next_unget = false;
ia.release_lookahead();
}
}
/// seekable adapter: nothing was captured, so nothing to undo
void uncapture_char(std::true_type /*lazy*/) const noexcept {}
@@ -1961,6 +2001,31 @@ scan_number_done:
return position;
}
/*!
@brief pass a pending simulated unget on to the input
unget() only rewinds the lexer's own bookkeeping, so the character that
terminated the last token (e.g. the character after a number) would still
be stepped over when the input adapter is done. Callers that hand the
input back to the user afterwards - operator>> and non-strict sax_parse -
call this once when scanning is done, so that the input is positioned
right after the value.
Adapters without lookahead (see input_adapter_supports_lookahead) are not
handed back to the user, so this is a no-op for them. Without
JSON_PRECISE_STREAM_POSITION, no adapter has lookahead, so this is always a
no-op and the terminating character stays consumed.
Scanning may continue after this call: @a next_unget is cleared, and the
character is read from the input again instead of being replayed from
@a current. A pending unget of EOF needs no special case, because reaching
EOF leaves no lookahead to release.
*/
void release_lookahead()
{
release_lookahead_impl(std::integral_constant<bool, can_release_lookahead> {});
}
#if JSON_DIAGNOSTIC_POSITIONS
/// return the offset of the first character of the last read token; unlike
/// the token's parsed value, this accounts for escape sequences
+45 -16
View File
@@ -100,13 +100,22 @@ class parser
json_sax_dom_callback_parser<BasicJsonType, InputAdapterType> sdp(result, callback, allow_exceptions, &m_lexer);
sax_parse_internal(&sdp);
// in strict mode, input must be completely read
if (strict && (get_token() != token_type::end_of_input))
if (strict)
{
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(),
exception_message(token_type::end_of_input, "value"), nullptr));
// in strict mode, input must be completely read
if (get_token() != token_type::end_of_input)
{
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(),
exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
// in case of an error, return a discarded value
@@ -128,12 +137,20 @@ class parser
json_sax_dom_parser<BasicJsonType, InputAdapterType> sdp(result, allow_exceptions, &m_lexer);
sax_parse_internal(&sdp);
// in strict mode, input must be completely read
if (strict && (get_token() != token_type::end_of_input))
if (strict)
{
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
// in strict mode, input must be completely read
if (get_token() != token_type::end_of_input)
{
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// see above
m_lexer.release_lookahead();
}
// in case of an error, return a discarded value
@@ -166,12 +183,24 @@ class parser
(void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
const bool result = sax_parse_internal(sax);
// strict mode: next byte must be EOF
if (result && strict && (get_token() != token_type::end_of_input))
if (result)
{
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
if (strict)
{
// strict mode: next byte must be EOF
if (get_token() != token_type::end_of_input)
{
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
}
return result;
@@ -201,6 +201,43 @@ inline std::size_t validate_one_utf8(const unsigned char* data, std::size_t avai
return 0; // invalid, incomplete, or must be diagnosed by the byte path
}
// Return the length of the longest prefix of [data, data+n) that consists of
// ASCII characters and complete well-formed UTF-8 sequences; n if all of it is
// valid UTF-8. Unlike scalar_string_bulk_run(), quotes, escapes, and control
// characters are ordinary characters here. ASCII is skipped 8 bytes at a time.
inline std::size_t valid_utf8_prefix(const unsigned char* data, std::size_t n) noexcept
{
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t pos = 0;
while (pos < n)
{
if (pos + 8 <= n)
{
std::uint64_t word = 0;
std::memcpy(&word, data + pos, sizeof(word));
if ((word & high) == 0)
{
pos += 8;
continue;
}
}
if (data[pos] < 0x80u)
{
++pos;
continue;
}
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
{
break; // ill-formed or truncated
}
pos += seq;
}
return pos;
}
// Scalar (C++11) computation of the bulk run length: the number of leading
// bytes in [data, data+n) that are ordinary ASCII or complete well-formed UTF-8
// sequences, stopping before the first byte that needs individual handling (the
+119 -16
View File
@@ -596,16 +596,62 @@ void templated_json_throw(ExceptionType exception)
#define NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(...) template<typename BasicJsonType, nlohmann::detail::enable_if_t<nlohmann::detail::is_basic_json<BasicJsonType>::value, int> = 0> __VA_ARGS__
// Helpers used to dispatch the NLOHMANN_DEFINE_TYPE_*/NLOHMANN_DEFINE_DERIVED_TYPE_*
// macros below between a zero-member and a one-or-more-member implementation
// (issue #4041, e.g. NLOHMANN_DEFINE_TYPE_INTRUSIVE(Type) with no further
// arguments). NLOHMANN_JSON_TYPE_BODY(Prefix, ...) expands to the macro name
// Prefix##EMPTY when __VA_ARGS__ is a single argument (Type alone) and to
// Prefix##MEMBERS for two or more (Type, member...). It reuses the existing
// 64-slot NLOHMANN_JSON_GET_MACRO dispatch with one extra trailing sentinel
// token appended so its own trailing "..." is never left completely empty at
// the lowest supported argument count -- invoking a variadic macro so that
// "..." matches nothing is only granted unconditionally by the standard since
// C++20, and pre-C++20 compilers may reject it under -pedantic regardless of
// what the macro body does.
//
// The EMPTY/MEMBERS suffixes are pasted onto Prefix right in the slot table:
// operands of ## are not macro-expanded, so the dispatch keeps working even if
// user code defines macros named EMPTY or MEMBERS. Producing the bare suffix
// first and pasting it later would let such a macro replace it.
//
// NLOHMANN_JSON_DERIVED_TYPE_BODY(Prefix, ...) answers the same question for
// the derived-type macros, whose fixed prefix is Type,BaseType. It drops the
// leading Type and defers to NLOHMANN_JSON_TYPE_BODY rather than shifting the
// slot table by one: NLOHMANN_JSON_GET_MACRO only resolves 64 positional
// arguments, so dispatching on Type,BaseType,member... directly would run out
// one slot early and cap the derived-type macros at 62 members instead of the
// 63 that NLOHMANN_JSON_PASTE supports.
#define NLOHMANN_JSON_TYPE_BODY(Prefix, ...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_GET_MACRO(__VA_ARGS__, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, \
Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## MEMBERS, Prefix ## EMPTY, \
NLOHMANN_JSON_TYPE_BODY_SENTINEL))
#define NLOHMANN_JSON_DERIVED_TYPE_BODY_(Prefix, Type, ...) NLOHMANN_JSON_TYPE_BODY(Prefix, __VA_ARGS__)
#define NLOHMANN_JSON_DERIVED_TYPE_BODY(Prefix, ...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY_(Prefix, __VA_ARGS__))
/*!
@brief macro
@def NLOHMANN_DEFINE_TYPE_INTRUSIVE
@since version 3.9.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_type_intrusive/
*/
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE(Type, ...) \
#define NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_MEMBERS(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_EMPTY(Type) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type&) { nlohmann_json_j = BasicJsonType::object(); }) \
/* NOLINTNEXTLINE(bugprone-macro-parentheses) Type is used as a declarator type, not in an expression */ \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType&, Type&) noexcept { })
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_TYPE_BODY(NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_NAMES(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_NAME, __VA_ARGS__)) })
@@ -616,10 +662,15 @@ void templated_json_throw(ExceptionType exception)
@since version 3.11.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_type_intrusive/
*/
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT(Type, ...) \
#define NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT_MEMBERS(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT, __VA_ARGS__)) })
// identical to NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_EMPTY: with no members there is nothing to default
#define NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT_EMPTY(Type) NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_EMPTY(Type)
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_TYPE_BODY(NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT_WITH_NAMES(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT_WITH_NAME, __VA_ARGS__)) })
@@ -630,9 +681,14 @@ void templated_json_throw(ExceptionType exception)
@since version 3.11.3
@sa https://json.nlohmann.me/api/macros/nlohmann_define_type_intrusive/
*/
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE(Type, ...) \
#define NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE_MEMBERS(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE_EMPTY(Type) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type&) { nlohmann_json_j = BasicJsonType::object(); })
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_TYPE_BODY(NLOHMANN_JSON_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) })
@@ -642,10 +698,17 @@ void templated_json_throw(ExceptionType exception)
@since version 3.9.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_type_non_intrusive/
*/
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(Type, ...) \
#define NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_MEMBERS(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_EMPTY(Type) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type&) { nlohmann_json_j = BasicJsonType::object(); }) \
/* NOLINTNEXTLINE(bugprone-macro-parentheses) Type is used as a declarator type, not in an expression */ \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType&, Type&) noexcept { })
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_TYPE_BODY(NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_NAMES(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_NAME, __VA_ARGS__)) })
@@ -656,10 +719,15 @@ void templated_json_throw(ExceptionType exception)
@since version 3.11.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_type_non_intrusive/
*/
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT(Type, ...) \
#define NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT_MEMBERS(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT, __VA_ARGS__)) })
// identical to NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_EMPTY: with no members there is nothing to default
#define NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT_EMPTY(Type) NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_EMPTY(Type)
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_TYPE_BODY(NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT_WITH_NAMES(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT_WITH_NAME, __VA_ARGS__)) })
@@ -670,9 +738,14 @@ void templated_json_throw(ExceptionType exception)
@since version 3.11.3
@sa https://json.nlohmann.me/api/macros/nlohmann_define_type_non_intrusive/
*/
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE(Type, ...) \
#define NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_MEMBERS(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_EMPTY(Type) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type&) { nlohmann_json_j = BasicJsonType::object(); })
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_TYPE_BODY(NLOHMANN_JSON_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES(Type, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) })
@@ -682,10 +755,17 @@ void templated_json_throw(ExceptionType exception)
@since version 3.12.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_derived_type/
*/
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE(Type, BaseType, ...) \
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_MEMBERS(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_EMPTY(Type, BaseType) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); }) \
/* NOLINTNEXTLINE(bugprone-macro-parentheses) Type/BaseType are used as declarator types, not in expressions */ \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); })
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY(NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_NAMES(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_NAME, __VA_ARGS__)) })
@@ -696,10 +776,15 @@ void templated_json_throw(ExceptionType exception)
@since version 3.12.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_derived_type/
*/
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT(Type, BaseType, ...) \
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT_MEMBERS(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType&>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT, __VA_ARGS__)) })
// identical to NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_EMPTY: with no members there is nothing to default
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT_EMPTY(Type, BaseType) NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_EMPTY(Type, BaseType)
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY(NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT_WITH_NAMES(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType&>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT_WITH_NAME, __VA_ARGS__)) })
@@ -710,9 +795,14 @@ void templated_json_throw(ExceptionType exception)
@since version 3.12.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_derived_type/
*/
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE(Type, BaseType, ...) \
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE_MEMBERS(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE_EMPTY(Type, BaseType) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); })
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY(NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(friend void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) })
@@ -723,10 +813,17 @@ void templated_json_throw(ExceptionType exception)
@since version 3.12.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_derived_type/
*/
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE(Type, BaseType, ...) \
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_MEMBERS(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_EMPTY(Type, BaseType) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); }) \
/* NOLINTNEXTLINE(bugprone-macro-parentheses) Type/BaseType are used as declarator types, not in expressions */ \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); })
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY(NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_NAMES(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_NAME, __VA_ARGS__)) })
@@ -737,10 +834,15 @@ void templated_json_throw(ExceptionType exception)
@since version 3.12.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_derived_type/
*/
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT(Type, BaseType, ...) \
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT_MEMBERS(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT, __VA_ARGS__)) })
// identical to NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_EMPTY: with no members there is nothing to default
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT_EMPTY(Type, BaseType) NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_EMPTY(Type, BaseType)
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY(NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT_WITH_NAMES(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) }) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void from_json(const BasicJsonType& nlohmann_json_j, Type& nlohmann_json_t) { nlohmann::from_json(nlohmann_json_j, static_cast<BaseType&>(nlohmann_json_t)); const Type nlohmann_json_default_obj{}; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_FROM_WITH_DEFAULT_WITH_NAME, __VA_ARGS__)) })
@@ -751,9 +853,14 @@ void templated_json_throw(ExceptionType exception)
@since version 3.12.0
@sa https://json.nlohmann.me/api/macros/nlohmann_define_derived_type/
*/
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE(Type, BaseType, ...) \
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_MEMBERS(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) })
#define NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_EMPTY(Type, BaseType) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); })
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DERIVED_TYPE_BODY(NLOHMANN_JSON_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_, __VA_ARGS__)(__VA_ARGS__))
#define NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE_WITH_NAMES(Type, BaseType, ...) \
NLOHMANN_JSON_BASIC_TYPE_TEMPLATE(void to_json(BasicJsonType& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann::to_json(nlohmann_json_j, static_cast<const BaseType &>(nlohmann_json_t)); NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_DOUBLE_PASTE(NLOHMANN_JSON_TO_WITH_NAME, __VA_ARGS__)) })
@@ -812,7 +919,3 @@ void templated_json_throw(ExceptionType exception)
#ifndef JSON_USE_GLOBAL_UDLS
#define JSON_USE_GLOBAL_UDLS 1
#endif
#ifndef JSON_STRICT_NUL_HANDLING
#define JSON_STRICT_NUL_HANDLING 0
#endif
+3 -1
View File
@@ -26,7 +26,6 @@
#undef JSON_NO_UNIQUE_ADDRESS
#undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_USE_GLOBAL_UDLS
#undef JSON_STRICT_NUL_HANDLING
#ifndef JSON_TEST_KEEP_MACROS
#undef JSON_CATCH
@@ -45,6 +44,9 @@
#undef JSON_HAS_STATIC_RTTI
#undef JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
#undef JSON_BRACE_INIT_COPY_SEMANTICS
#undef JSON_PRECISE_STREAM_POSITION
#undef JSON_STRICT_NUL_HANDLING
#undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#include <nlohmann/thirdparty/hedley/hedley_undef.hpp>
@@ -400,6 +400,30 @@ template<typename BasicJsonType, typename CompatibleObjectType>
struct is_compatible_object_type
: is_compatible_object_type_impl<BasicJsonType, CompatibleObjectType> {};
template<typename T>
using insert_result_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()));
template<typename T>
using insert_result_second_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()).second);
// a map-like type (std::map, std::unordered_map, ...) whose keys are enums; see
// JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
template<typename T, typename = void>
struct is_enum_keyed_map : std::false_type {};
template<typename T>
struct is_enum_keyed_map <
T, enable_if_t < is_detected<mapped_type_t, T>::value&&
is_detected<key_type_t, T>::value >>
{
// maps with non-unique keys (std::multimap, std::unordered_multimap, ...)
// are excluded, because an object cannot hold duplicate keys; they are
// detected by insert() returning an iterator instead of a pair<iterator, bool>
// NOLINTNEXTLINE(modernize-type-traits) we use C++11
static constexpr bool value = std::is_enum<typename T::key_type>::value &&
!(is_detected<insert_result_t, T>::value && !is_detected<insert_result_second_t, T>::value);
};
template<typename BasicJsonType, typename ConstructibleObjectType,
typename = void>
struct is_constructible_object_type_impl : std::false_type {};
File diff suppressed because it is too large Load Diff
+1 -58
View File
@@ -32,6 +32,7 @@
#include <nlohmann/detail/output/output_adapters.hpp>
#include <nlohmann/detail/recursion_depth_limit.hpp>
#include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_utils.hpp>
#include <nlohmann/detail/value_t.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -58,8 +59,6 @@ class serializer
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using binary_char_t = typename BasicJsonType::binary_t::value_type;
static constexpr std::uint8_t UTF8_ACCEPT = 0;
static constexpr std::uint8_t UTF8_REJECT = 1;
public:
/*!
@@ -1592,62 +1591,6 @@ class serializer
}
}
/*!
@brief check whether a string is UTF-8 encoded
The function checks each byte of a string whether it is UTF-8 encoded. The
result of the check is stored in the @a state parameter. The function must
be called initially with state 0 (accept). State 1 means the string must
be rejected, because the current byte is not allowed. If the string is
completely processed, but the state is non-zero, the string ended
prematurely; that is, the last byte indicated more bytes should have
followed.
@param[in,out] state the state of the decoding
@param[in,out] codep codepoint (valid only if resulting state is UTF8_ACCEPT)
@param[in] byte next byte to decode
@return new state
@note The function has been edited: a std::array is used.
@copyright Copyright (c) 2008-2009 Bjoern Hoehrmann <bjoern@hoehrmann.de>
@sa http://bjoern.hoehrmann.de/utf-8/decoder/dfa/
*/
static std::uint8_t decode(std::uint8_t& state, std::uint32_t& codep, const std::uint8_t byte) noexcept
{
static const std::array<std::uint8_t, 400> utf8d =
{
{
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 00..1F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 20..3F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 40..5F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 60..7F
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, // 80..9F
7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, // A0..BF
8, 8, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, // C0..DF
0xA, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x4, 0x3, 0x3, // E0..EF
0xB, 0x6, 0x6, 0x6, 0x5, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, // F0..FF
0x0, 0x1, 0x2, 0x3, 0x5, 0x8, 0x7, 0x1, 0x1, 0x1, 0x4, 0x6, 0x1, 0x1, 0x1, 0x1, // s0..s0
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, // s1..s2
1, 2, 1, 1, 1, 1, 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, // s3..s4
1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1, 3, 1, 1, 1, 1, 1, 1, // s5..s6
1, 3, 1, 1, 1, 1, 1, 3, 1, 3, 1, 1, 1, 1, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 // s7..s8
}
};
JSON_ASSERT(static_cast<std::size_t>(byte) < utf8d.size());
const std::uint8_t type = utf8d[byte];
codep = (state != UTF8_ACCEPT)
? (byte & 0x3fu) | (codep << 6u)
: (0xFFu >> type) & (byte);
const std::size_t index = 256u + (static_cast<size_t>(state) * 16u) + static_cast<size_t>(type);
JSON_ASSERT(index < utf8d.size());
state = utf8d[index];
return state;
}
/*
* Overload to make the compiler happy while it is instantiating
* dump_integer for number_unsigned_t.
+100
View File
@@ -8,10 +8,13 @@
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <string> // string, to_string
#include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -33,5 +36,102 @@ StringType to_string(std::size_t value)
return result;
}
///////////////////
// UTF-8 decoding //
///////////////////
// UTF-8 decoder states used by decode() below
static constexpr std::uint8_t UTF8_ACCEPT = 0;
static constexpr std::uint8_t UTF8_REJECT = 1;
/*!
@brief process a byte of a UTF-8 sequence
This is a single-byte step of a "shift-based" UTF-8 decoder originally
written by Björn Hoehrmann. See
http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.
This decoder is the single source of truth for UTF-8 validation in this
library: it is used both by the serializer (to escape and, in strict mode,
reject ill-formed UTF-8 when dumping a string) and by the binary readers
(to reject ill-formed UTF-8 in CBOR/MessagePack/BSON/UBJSON text strings at
decode time; see @ref is_valid_utf8 below).
@param[in,out] state the current decoder state
@param[in,out] codep codepoint (valid only if resulting state is UTF8_ACCEPT)
@param[in] byte next byte to decode
@return new state
@note Original source: http://bjoern.hoehrmann.de/utf-8/decoder/dfa/
@sa http://bjoern.hoehrmann.de/utf-8/decoder/dfa/
*/
inline std::uint8_t decode(std::uint8_t& state, std::uint32_t& codep, const std::uint8_t byte) noexcept
{
static const std::array<std::uint8_t, 400> utf8d =
{
{
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 00..1F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 20..3F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 40..5F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 60..7F
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, // 80..9F
7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, // A0..BF
8, 8, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, // C0..DF
0xA, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x4, 0x3, 0x3, // E0..EF
0xB, 0x6, 0x6, 0x6, 0x5, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, // F0..FF
0x0, 0x1, 0x2, 0x3, 0x5, 0x8, 0x7, 0x1, 0x1, 0x1, 0x4, 0x6, 0x1, 0x1, 0x1, 0x1, // s0..s0
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, // s1..s2
1, 2, 1, 1, 1, 1, 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, // s3..s4
1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1, 3, 1, 1, 1, 1, 1, 1, // s5..s6
1, 3, 1, 1, 1, 1, 1, 3, 1, 3, 1, 1, 1, 1, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 // s7..s8
}
};
JSON_ASSERT(static_cast<std::size_t>(byte) < utf8d.size());
const std::uint8_t type = utf8d[byte];
codep = (state != UTF8_ACCEPT)
? (byte & 0x3fu) | (codep << 6u)
: (0xFFu >> type) & (byte);
const std::size_t index = 256u + (static_cast<std::size_t>(state) * 16u) + static_cast<std::size_t>(type);
JSON_ASSERT(index < utf8d.size());
state = utf8d[index];
return state;
}
/*!
@brief check whether a string consists solely of valid UTF-8
Used by the CBOR/MessagePack/BSON/UBJSON binary readers to reject text
strings that are not valid UTF-8 at decode time (RFC 8949 §3.1 and the
MessagePack/BSON specifications all require text strings to be UTF-8), so
that malformed input is caught immediately instead of only surfacing later
as a type_error.316 when the resulting value is dumped.
@param[in] s the string to check
@param[in] first index of the first byte to check; the bytes before it are
assumed to have been validated already and to end on a
code point boundary
@return whether @a s (from index @a first on) is valid UTF-8
*/
template<typename StringType>
inline bool is_valid_utf8(const StringType& s, const std::size_t first = 0) noexcept
{
std::uint8_t state = UTF8_ACCEPT;
std::uint32_t codepoint = 0;
for (std::size_t i = first; i < s.size(); ++i)
{
decode(state, codepoint, static_cast<std::uint8_t>(s[i]));
if (state == UTF8_REJECT)
{
return false;
}
}
return state == UTF8_ACCEPT;
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+410 -34
View File
@@ -924,6 +924,31 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
#endif
/*!
@brief whether a descent must stop here and finish without the call stack
@a may_descend says whether the operator descends at all; it is a constant
at every call site, and is passed rather than tested by the caller so that
the test does not become a constant condition there, which MSVC reports as
C4127.
The comparison operators use this rather than @ref nesting_depth_guard::okay,
because they are written as a macro and a macro cannot use the preprocessor
the way the guard's constructor does; @ref copy_structured, which can, asks
the guard instead and never calls this.
*/
static bool nesting_depth_exhausted(bool may_descend = true) noexcept
{
#ifdef JSON_NO_THREAD_LOCAL
// without a count of its own per thread, a descent cannot be bounded
// without racing another one, so none is made
static_cast<void>(may_descend);
return true;
#else
return !may_descend || nesting_depth() >= nesting_depth_limit();
#endif
}
/*!
@brief counts one level of a bounded descent for as long as it runs, and
reports whether the descent was still within the limit when it began
@@ -978,7 +1003,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>;
/// scratch space to build the key skeleton of an object copy in one go
using copy_scratch_t = std::vector<std::pair<typename object_t::key_type, basic_json>>;
using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
/// @brief copy everything of @a src into @a dst but its type and value
static void copy_metadata(const basic_json& src, basic_json& dst)
@@ -1243,6 +1269,289 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
/// the result of comparing two values, including values that cannot be
/// ordered at all, such as a discarded value or a NaN
enum class compare_result { less, equal, greater, unordered };
#if JSON_HAS_THREE_WAY_COMPARISON
/// @brief the ordering that @a result stands for
static std::partial_ordering to_partial_ordering(compare_result result) noexcept // *NOPAD*
{
switch (result)
{
case compare_result::less:
return std::partial_ordering::less;
case compare_result::greater:
return std::partial_ordering::greater;
case compare_result::equal:
return std::partial_ordering::equivalent;
case compare_result::unordered:
default:
return std::partial_ordering::unordered;
}
}
#endif
/*!
@brief compare two values that are not both an array or both an object
Such a pair is compared by the operators themselves, which cannot descend
into it and therefore cannot recurse.
That holds for a pair whose types differ as much as for a pair of leaves: an
array and an object are told apart by their types alone, because an operator
only ever descends into two values of the same type. So `==` reports them as
unequal without looking inside either, and an ordering falls back to the
order of the types - an object sorts before an array - exactly as it does
for a value that is not nested deeply enough to get here.
*/
template<bool Ordered>
static compare_result compare_leaves(const_reference lhs, const_reference rhs) noexcept
{
if (lhs == rhs)
{
return compare_result::equal;
}
return order_leaves(lhs, rhs, std::integral_constant<bool, Ordered> {});
}
/*!
@brief compare two object keys
An object compares its entries as pairs of a key and a value, so its keys
are compared exactly as std::pair compares them: with < where the objects
are being ordered, and with == where they are only checked for equality.
Note that this is not the object's own comparator, which for a vector-backed
object type such as nlohmann::ordered_map tells equality rather than order.
*/
static compare_result compare_keys(const typename object_t::key_type& lhs,
const typename object_t::key_type& rhs,
std::true_type /*ordered*/)
{
if (lhs < rhs)
{
return compare_result::less;
}
if (rhs < lhs)
{
return compare_result::greater;
}
return compare_result::equal;
}
/// @brief check two object keys for equality
static compare_result compare_keys(const typename object_t::key_type& lhs,
const typename object_t::key_type& rhs,
std::false_type /*ordered*/)
{
return lhs == rhs ? compare_result::equal : compare_result::unordered;
}
/// @brief tell apart two values that are not equal
/// @note only instantiated where the values are being ordered, as a key or
/// string type is not required to be ordered to be compared for equality
static compare_result order_leaves(const_reference lhs, const_reference rhs, std::true_type /*ordered*/) noexcept
{
if (lhs < rhs)
{
return compare_result::less;
}
if (rhs < lhs)
{
return compare_result::greater;
}
return compare_result::unordered;
}
/// @brief report two values as not equal without ordering them
static compare_result order_leaves(const_reference /*lhs*/, const_reference /*rhs*/, std::false_type /*ordered*/) noexcept
{
return compare_result::unordered;
}
/*!
@brief compare @a lhs and @a rhs without descending into them
Reached once a comparison has descended @ref nesting_depth_limit levels, so
that comparing values cannot exhaust the call stack however deeply they are
nested. The two values are walked in lockstep on an explicit stack and
compared lexicographically, element by element in the order the containers
enumerate them - which is how the container types this library ships compare
themselves: a std::map enumerates its entries in key order, and
nlohmann::ordered_map in insertion order. An object type that enumerates its
entries in an unspecified order, such as std::unordered_map, compares them
pairwise instead; the difference could only ever show below the bound.
Note that the stack this walks with is allocated, while the comparison
operators are noexcept and the container comparison this replaces allocated
nothing. Failing that allocation therefore ends the process rather than
throwing. It only arises for values nested past the bound, and only when
memory has run out - where the same comparison used to exhaust the call
stack instead - but it is a way to fail that the operators did not have.
*/
template<bool Ordered>
static compare_result compare_iteratively(const_reference lhs, const_reference rhs,
const bool unordered_compares_equal) noexcept
{
/// a pair of containers being compared in lockstep
struct frame
{
const basic_json* lhs_value{nullptr};
const basic_json* rhs_value{nullptr};
typename array_t::const_iterator lhs_array_it{};
typename array_t::const_iterator rhs_array_it{};
typename object_t::const_iterator lhs_object_it{};
typename object_t::const_iterator rhs_object_it{};
};
std::vector<frame> stack;
const basic_json* left = &lhs;
const basic_json* right = &rhs;
for (;;)
{
const auto type = left->m_data.m_type;
if (type == right->m_data.m_type && (type == value_t::array || type == value_t::object))
{
// descend: the elements decide, and are compared further down
stack.emplace_back();
frame& pushed = stack.back();
pushed.lhs_value = left;
pushed.rhs_value = right;
if (type == value_t::array)
{
pushed.lhs_array_it = left->m_data.m_value.array->cbegin();
pushed.rhs_array_it = right->m_data.m_value.array->cbegin();
}
else
{
pushed.lhs_object_it = left->m_data.m_value.object->cbegin();
pushed.rhs_object_it = right->m_data.m_value.object->cbegin();
}
}
else
{
const compare_result result = compare_leaves<Ordered>(*left, *right);
// Values that cannot be ordered - a NaN, say - end an ordered
// comparison for std::lexicographical_compare_three_way, but
// std::lexicographical_compare treats them as equivalent and
// carries on with the next element. Both are reproduced here,
// so that a value nested too deeply to descend into compares
// exactly as one that is not.
if (result != compare_result::equal &&
!(unordered_compares_equal && result == compare_result::unordered))
{
return result;
}
}
// walk back up past the containers that are exhausted, then take the
// next pair of elements from the innermost one that is not
for (;;)
{
if (stack.empty())
{
return compare_result::equal;
}
frame& current = stack.back();
const bool is_object = current.lhs_value->m_data.m_type == value_t::object;
const bool lhs_done = is_object
? current.lhs_object_it == current.lhs_value->m_data.m_value.object->cend()
: current.lhs_array_it == current.lhs_value->m_data.m_value.array->cend();
const bool rhs_done = is_object
? current.rhs_object_it == current.rhs_value->m_data.m_value.object->cend()
: current.rhs_array_it == current.rhs_value->m_data.m_value.array->cend();
if (lhs_done || rhs_done)
{
// whichever ran out first holds the smaller container; if
// both did, they are equal and the container above decides
if (lhs_done != rhs_done)
{
return lhs_done ? compare_result::less : compare_result::greater;
}
stack.pop_back();
continue;
}
if (is_object)
{
// an entry is a key and a value, and the key decides first
const compare_result key_result =
compare_keys(current.lhs_object_it->first, current.rhs_object_it->first,
std::integral_constant<bool, Ordered> {});
left = &(current.lhs_object_it->second);
right = &(current.rhs_object_it->second);
if (key_result != compare_result::equal)
{
// An object type without a fixed order of its entries -
// std::unordered_map, say - may enumerate two equal
// objects differently, and its operator== does not care.
// Equality then finds the entry by its key; an ordering,
// or an object type that compares its entries in
// sequence (ordered_map), is decided by the key itself.
const auto* rhs_object = current.rhs_value->m_data.m_value.object;
const auto found = (!Ordered && !detail::is_ordered_map<object_t>::value)
? rhs_object->find(current.lhs_object_it->first)
: rhs_object->cend();
if (found == rhs_object->cend())
{
return key_result;
}
right = &(found->second);
}
++current.lhs_object_it;
++current.rhs_object_it;
}
else
{
left = &(*current.lhs_array_it);
right = &(*current.rhs_array_it);
++current.lhs_array_it;
++current.rhs_array_it;
}
break;
}
}
}
/// @brief restore the parent pointers after erasing from an object
/// ordered_json keeps its members in a vector, and erasing a member
/// re-constructs every member after it in place, which resets their
/// parent pointers
void set_parents_after_object_erase()
{
#if JSON_DIAGNOSTICS
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning(push )
#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
#endif
if (detail::is_ordered_map<object_t>::value)
{
set_parents();
}
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning( pop )
#endif
#endif
}
public:
//////////////////////////
// JSON parser callback //
@@ -1863,17 +2172,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// value access //
//////////////////
/// get a boolean (explicit)
boolean_t get_impl(boolean_t* /*unused*/) const
{
if (JSON_HEDLEY_LIKELY(is_boolean()))
{
return m_data.m_value.boolean;
}
JSON_THROW(type_error::create(302, detail::concat("type must be boolean, but is ", type_name()), this));
}
/// get a pointer to the value (object)
object_t* get_impl_ptr(object_t* /*unused*/) noexcept
{
@@ -2260,6 +2558,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
ValueType & get_to(ValueType& v) const noexcept(noexcept(
JSONSerializer<ValueType>::from_json(std::declval<const basic_json_t&>(), v)))
{
static_assert(!std::is_const<ValueType>::value, "get_to() cannot deserialize into a const value");
JSONSerializer<ValueType>::from_json(*this, v);
return v;
}
@@ -2285,6 +2584,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
noexcept(noexcept(JSONSerializer<Array>::from_json(
std::declval<const basic_json_t&>(), v)))
{
static_assert(!std::is_const<T>::value, "get_to() cannot deserialize into a const value");
JSONSerializer<Array>::from_json(*this, v);
return v;
}
@@ -2931,6 +3231,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
case value_t::object:
{
result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator);
set_parents_after_object_erase();
break;
}
@@ -3003,6 +3304,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator,
last.m_it.object_iterator);
set_parents_after_object_erase();
break;
}
@@ -3033,7 +3335,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
}
return m_data.m_value.object->erase(std::forward<KeyType>(key));
const auto erased = m_data.m_value.object->erase(std::forward<KeyType>(key));
set_parents_after_object_erase();
return erased;
}
template < typename KeyType, detail::enable_if_t <
@@ -3050,6 +3354,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (it != m_data.m_value.object->end())
{
m_data.m_value.object->erase(it);
set_parents_after_object_erase();
return 1;
}
return 0;
@@ -3961,16 +4266,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (it2 != m_data.m_value.object->end() && it2->second.is_object())
{
it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1);
#if JSON_DIAGNOSTICS
it2->second.set_parents();
#endif
continue;
}
}
m_data.m_value.object->operator[](it.key()) = it.value();
#if JSON_DIAGNOSTICS
m_data.m_value.object->operator[](it.key()).m_parent = this;
#endif
// set_parent() also repairs the other members, which ordered_json
// relocates when adding a key makes its vector grow
set_parent(m_data.m_value.object->operator[](it.key()) = it.value());
}
}
@@ -3999,9 +4300,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
// a nested object is merged: continue with its parent
#if JSON_DIAGNOSTICS
target->set_parents();
#endif
target = stack.back().target;
first = stack.back().position;
last = stack.back().last;
@@ -4023,10 +4321,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
continue;
}
}
target->m_data.m_value.object->operator[](first.key()) = first.value();
#if JSON_DIAGNOSTICS
target->m_data.m_value.object->operator[](first.key()).m_parent = target;
#endif
// set_parent() also repairs the other members, which ordered_json
// relocates when adding a key makes its vector grow
target->set_parent(target->m_data.m_value.object->operator[](first.key()) = first.value());
++first;
}
}
@@ -4165,7 +4462,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// because any negative signed value is smaller than any unsigned value.
// Otherwise, the non-negative signed value is cast to unsigned before the
// comparison to avoid wraparound.
#define JSON_IMPLEMENT_OPERATOR(op, null_result, unordered_result, default_result) \
#define JSON_IMPLEMENT_OPERATOR(op, null_result, unordered_result, default_result, deep_result, may_descend) \
const auto lhs_type = lhs.type(); \
const auto rhs_type = rhs.type(); \
\
@@ -4174,11 +4471,25 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
switch (lhs_type) \
{ \
case value_t::array: \
{ \
if (JSON_HEDLEY_UNLIKELY(nesting_depth_exhausted(may_descend))) \
{ \
return (deep_result); \
} \
const nesting_depth_guard guard; \
return (*lhs.m_data.m_value.array) op (*rhs.m_data.m_value.array); \
\
} \
\
case value_t::object: \
{ \
if (JSON_HEDLEY_UNLIKELY(nesting_depth_exhausted(may_descend))) \
{ \
return (deep_result); \
} \
const nesting_depth_guard guard; \
return (*lhs.m_data.m_value.object) op (*rhs.m_data.m_value.object); \
\
} \
\
case value_t::null: \
return (null_result); \
\
@@ -4278,7 +4589,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal")
#endif
const_reference lhs = *this;
JSON_IMPLEMENT_OPERATOR( ==, true, false, false)
JSON_IMPLEMENT_OPERATOR( ==, true, false, false,
compare_iteratively<false>(lhs, rhs, false) == compare_result::equal, true)
#ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP
#endif
@@ -4303,7 +4615,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_IMPLEMENT_OPERATOR(<=>, // *NOPAD*
std::partial_ordering::equivalent,
std::partial_ordering::unordered,
lhs_type <=> rhs_type) // *NOPAD*
lhs_type <=> rhs_type, // *NOPAD*
to_partial_ordering(compare_iteratively<true>(lhs, rhs, false)), true)
}
/// @brief comparison: 3-way
@@ -4370,7 +4683,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_HEDLEY_DIAGNOSTIC_PUSH
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal")
#endif
JSON_IMPLEMENT_OPERATOR( ==, true, false, false)
JSON_IMPLEMENT_OPERATOR( ==, true, false, false,
compare_iteratively<false>(lhs, rhs, false) == compare_result::equal, true)
#ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP
#endif
@@ -4426,7 +4740,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// default_result is used if we cannot compare values. In that case,
// we compare types. Note we have to call the operator explicitly,
// because MSVC has problems otherwise.
JSON_IMPLEMENT_OPERATOR( <, false, false, operator<(lhs_type, rhs_type))
JSON_IMPLEMENT_OPERATOR( <, false, false, operator<(lhs_type, rhs_type),
compare_iteratively<true>(lhs, rhs, true) == compare_result::less, false)
}
/// @brief comparison: less than
@@ -4975,6 +5290,30 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
binary_writer<char>(o).write_bson(j);
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static std::vector<std::uint8_t> to_bon8(const basic_json& j)
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_bon8(j);
return result;
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o)
{
binary_writer<std::uint8_t>(o).write_bon8(j);
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static void to_bon8(const basic_json& j, detail::output_adapter<char> o)
{
binary_writer<char>(o).write_bon8(j);
}
/// @brief create a JSON value from an input in CBOR format
/// @sa https://json.nlohmann.me/api/basic_json/from_cbor/
template<typename InputType>
@@ -5208,6 +5547,43 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
return result;
}
/// @brief create a JSON value from an input in BON8 format
/// @sa https://json.nlohmann.me/api/basic_json/from_bon8/
template<typename InputType>
JSON_HEDLEY_WARN_UNUSED_RESULT
static basic_json from_bon8(InputType&& i,
const bool strict = true,
const bool allow_exceptions = true)
{
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bon8).sax_parse(input_format_t::bon8, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BON8 format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
/// @sa https://json.nlohmann.me/api/basic_json/from_bon8/
template<typename IteratorType, typename SentinelType = IteratorType,
detail::enable_if_t<detail::can_compare_ne<IteratorType, SentinelType>::value, int> = 0>
JSON_HEDLEY_WARN_UNUSED_RESULT
static basic_json from_bon8(IteratorType first, SentinelType last,
const bool strict = true,
const bool allow_exceptions = true)
{
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bon8).sax_parse(input_format_t::bon8, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BSON format
/// @sa https://json.nlohmann.me/api/basic_json/from_bson/
template<typename InputType>
+6720
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+37 -4
View File
@@ -56,6 +56,18 @@
#define JSON_BRACE_INIT_COPY_SEMANTICS 0
#endif
#ifndef JSON_PRECISE_STREAM_POSITION
#define JSON_PRECISE_STREAM_POSITION 0
#endif
#ifndef JSON_STRICT_NUL_HANDLING
#define JSON_STRICT_NUL_HANDLING 0
#endif
#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
#endif
#if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else
@@ -80,21 +92,42 @@
#define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS
#endif
#if JSON_PRECISE_STREAM_POSITION
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION _psp
#else
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION
#endif
#if JSON_STRICT_NUL_HANDLING
#define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING _snul
#else
#define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
#else
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d) json_abi ## a ## b ## c ## d
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \
NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS)
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
+5 -2
View File
@@ -80,7 +80,10 @@ target_compile_options(test_main PUBLIC
# std::allocator_traits<...>::construct for the custom-base-class
# test's map type past VS2015's limit. The name is only used for
# debug info, so truncation does not affect the build.
$<$<CXX_COMPILER_ID:MSVC>:/W4;/wd4566;/wd4996;/wd4702;/wd4503>
# Disable warning C5285: cannot declare a specialization for 'std::tuple'; MSVC 19.51
# reports the forward declarations of standard library
# templates in the vendored doctest.h
$<$<CXX_COMPILER_ID:MSVC>:/W4;/wd4566;/wd4996;/wd4702;/wd4503;/wd5285>
# https://github.com/nlohmann/json/issues/1114
$<$<CXX_COMPILER_ID:MSVC>:/bigobj> $<$<BOOL:${MINGW}>:-Wa,-mbig-obj>
@@ -109,7 +112,7 @@ endif()
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# avoid stack overflow, see https://github.com/nlohmann/json/issues/2955
json_test_set_test_options("test-cbor;test-msgpack;test-ubjson;test-bjdata;test-binary_formats" LINK_OPTIONS /STACK:4000000)
json_test_set_test_options("test-bon8;test-cbor;test-msgpack;test-ubjson;test-bjdata;test-binary_formats" LINK_OPTIONS /STACK:4000000)
endif()
# disable exceptions for test-disabled_exceptions
+4 -1
View File
@@ -10,7 +10,7 @@ CXXFLAGS += -std=c++11
CPPFLAGS += -I ../single_include
FUZZER_ENGINE = src/fuzzer-driver_afl.cpp
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer
fuzzers: $(FUZZERS)
parse_afl_fuzzer:
@@ -30,3 +30,6 @@ parse_ubjson_fuzzer:
parse_bjdata_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bjdata.cpp -o $@
parse_bon8_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bon8.cpp -o $@
+12
View File
@@ -36,6 +36,18 @@ TEST_CASE("default namespace")
expected += "_bics";
#endif
#if JSON_PRECISE_STREAM_POSITION
expected += "_psp";
#endif
#if JSON_STRICT_NUL_HANDLING
expected += "_snul";
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
expected += "_ekmo";
#endif
expected += "_v" STRINGIZE(NLOHMANN_JSON_VERSION_MAJOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_MINOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_PATCH) "::basic_json";
+12
View File
@@ -37,6 +37,18 @@ TEST_CASE("default namespace without version component")
expected += "_bics";
#endif
#if JSON_PRECISE_STREAM_POSITION
expected += "_psp";
#endif
#if JSON_STRICT_NUL_HANDLING
expected += "_snul";
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
expected += "_ekmo";
#endif
expected += "::basic_json";
// fallback for Clang
+21 -15
View File
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.11...3.14)
cmake_minimum_required(VERSION 3.14)
project(JSON_Benchmarks LANGUAGES CXX)
# set compiler flags
@@ -6,29 +6,35 @@ if((CMAKE_CXX_COMPILER_ID MATCHES GNU) OR (CMAKE_CXX_COMPILER_ID MATCHES Clang))
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -flto -DNDEBUG -O3")
endif()
# configure Google Benchmarks
# configure Google Benchmark; a fixed release, so that results stay comparable
set(JSON_GOOGLE_BENCHMARK_VERSION 1.9.5)
include(FetchContent)
FetchContent_Declare(
benchmark
GIT_REPOSITORY https://github.com/google/benchmark.git
GIT_TAG origin/main
GIT_SHALLOW TRUE
)
FetchContent_GetProperties(benchmark)
if(NOT benchmark_POPULATED)
FetchContent_Populate(benchmark)
set(BENCHMARK_ENABLE_TESTING OFF CACHE INTERNAL "" FORCE)
add_subdirectory(${benchmark_SOURCE_DIR} ${benchmark_BINARY_DIR})
endif()
# only the library is needed; -Werror would break the pinned release as soon as
# a newer compiler adds a warning
set(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_INSTALL OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_WERROR OFF CACHE BOOL "" FORCE)
FetchContent_Declare(benchmark
URL https://github.com/google/benchmark/archive/refs/tags/v${JSON_GOOGLE_BENCHMARK_VERSION}.tar.gz
URL_HASH SHA256=9631341c82bac4a288bef951f8b26b41f69021794184ece969f8473977eaa340
DOWNLOAD_EXTRACT_TIMESTAMP TRUE
)
FetchContent_MakeAvailable(benchmark)
# download test data
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../cmake ${CMAKE_MODULE_PATH})
include(download_test_data)
# the header to benchmark; point this at a directory holding another version's
# nlohmann/json.hpp to compare versions (see README.md)
set(JSON_BENCHMARK_INCLUDE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../single_include" CACHE PATH
"directory containing the nlohmann/json.hpp to benchmark")
# benchmark binary
add_executable(json_benchmarks src/benchmarks.cpp)
target_compile_features(json_benchmarks PRIVATE cxx_std_11)
target_link_libraries(json_benchmarks benchmark ${CMAKE_THREAD_LIBS_INIT})
add_dependencies(json_benchmarks download_test_data)
target_include_directories(json_benchmarks PRIVATE ${CMAKE_SOURCE_DIR}/../../single_include ${CMAKE_BINARY_DIR}/include)
target_include_directories(json_benchmarks PRIVATE ${JSON_BENCHMARK_INCLUDE_DIR} ${CMAKE_BINARY_DIR}/include)

Some files were not shown because too many files have changed in this diff Show More