Compare commits

..
Author SHA1 Message Date
Niels Lohmann f386f367df Merge branch 'develop' into copy-scratch-allocator
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:24:19 +02:00
Niels Lohmann abf827a5bd Count allocate_at_least in the scratch-counting test allocator
From C++23 on, libc++'s containers allocate through allocate_at_least when
the allocator has one. The test allocator inherited it from std::allocator,
so the scratch allocations were not counted and the test failed on Xcode.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:20:13 +02:00
Niels Lohmann a335a81ba7 Allocate the deep copy's key scratch space with the provided allocator
The iterative deep copy builds each object's keys in a temporary vector of
key/value pairs before handing them to the object's range constructor. That
vector holds basic_json values, so like the values themselves it now uses
AllocatorType instead of std::allocator.

Also document that AllocatorType covers the JSON values, while most
temporary storage still uses std::allocator.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 22:10:07 +02:00
38 changed files with 199 additions and 2717 deletions
-1
View File
@@ -2,7 +2,6 @@
- [ ] The changes are described in detail, both the what and why. - [ ] The changes are described in detail, both the what and why.
- [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced. - [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
- [ ] If applicable, a fixed [OSS-Fuzz](https://issues.oss-fuzz.com) issue is referenced as `OSS-Fuzz: <id>` (see [fuzz testing](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports)).
- [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code. - [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
- [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated. - [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
- [ ] The source code is amalgamated by running `make amalgamate`. - [ ] The source code is amalgamated by running `make amalgamate`.
+1 -5
View File
@@ -69,9 +69,7 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
[`input_format_t`](input_format_t.md) for more information [`input_format_t`](input_format_t.md) for more information
`strict` (in) `strict` (in)
: whether the input has to be consumed completely (optional, `#!cpp true` by default); when `#!cpp false` and the : whether the input has to be consumed completely (optional, `#!cpp true` by default)
input is a `#!cpp std::istream`, the character that terminates a number is consumed unless
[`JSON_PRECISE_STREAM_POSITION`](../macros/json_precise_stream_position.md) is defined to `1`; see [`operator>>`](../operator_gtgt.md#notes)
`ignore_comments` (in) `ignore_comments` (in)
: whether comments should be ignored and treated like whitespace (`#!cpp true`) or yield a parse error : whether comments should be ignored and treated like whitespace (`#!cpp true`) or yield a parse error
@@ -138,8 +136,6 @@ A UTF-8 byte order mark is silently ignored.
- Added `ignore_trailing_commas` in version 3.13.0. - Added `ignore_trailing_commas` in version 3.13.0.
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0. - Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0. - Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
- `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave a `#!cpp std::istream` positioned right
after the parsed value when `strict` is `#!cpp false`.
!!! warning "Deprecation" !!! warning "Deprecation"
-2
View File
@@ -16,8 +16,6 @@ header. See also the [macro overview page](../../features/macros.md).
## Parsing ## Parsing
- [**JSON_PRECISE_STREAM_POSITION**](json_precise_stream_position.md) - opt in to leaving an input stream positioned
right after a parsed number
- [**JSON_STRICT_NUL_HANDLING**](json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input instead of - [**JSON_STRICT_NUL_HANDLING**](json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input instead of
treating it as end of input treating it as end of input
@@ -1,131 +0,0 @@
# JSON_PRECISE_STREAM_POSITION
```cpp
#define JSON_PRECISE_STREAM_POSITION /* value */
```
When defined to `1`, [`operator>>`](../operator_gtgt.md) and [`sax_parse`](../basic_json/sax_parse.md) with
`strict = false` leave a `#!cpp std::istream` positioned right after the parsed value for every value type. By default,
the character that terminates a number is consumed as well.
The macro only affects reading from a `#!cpp std::istream` when the rest of the stream is not required to be consumed.
[`parse`](../basic_json/parse.md), [`accept`](../basic_json/accept.md), and all other inputs (strings, iterators,
containers, `#!cpp FILE*`) are never affected.
## Default definition
The default value is `0` (disabled — existing behavior is preserved).
```cpp
#define JSON_PRECISE_STREAM_POSITION 0
```
## Notes
!!! note "Background"
A number is the only JSON value whose end can be detected solely by reading the character that follows it. By
default, that character is consumed and not put back, so the stream is left one byte too far after a number, and
only after a number:
```cpp
std::istringstream input("1true");
json j;
input >> j; // j == 1, but the stream now starts at "rue"
```
With this macro, the character is only looked at and left in the stream, so the stream starts at `true`. This
does not require the stream buffer to support putting a character back.
This was not changed unconditionally, because code can depend on the consumed character, even unknowingly (see
[#5340](https://github.com/nlohmann/json/issues/5340)). Both of the following work by default only because the
character after each number is swallowed, and behave differently with this macro:
```cpp
std::istringstream input("1,2,3");
json j1, j2, j3;
input >> j1 >> j2 >> j3; // default: 1, 2, 3
// with the macro: throws parse_error.101 at the ','
```
```cpp
std::istringstream input("42\nfoo");
json j;
std::string line;
input >> j;
std::getline(input, line); // default: "foo"
// with the macro: "" (like after reading an int with >>)
```
In both cases, the behavior with the macro is what you already get today when the value is not a number: `"a","b"`
fails at the `,`, and `std::getline` after `{}` returns an empty string. This macro offers an opt-in path to
the consistent behavior ahead of version 4.0.0, where it is planned to become the default.
!!! warning "Opt-in only"
This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no
effect.
!!! note "ABI compatibility"
The value of this macro is encoded in the [namespace](../../features/namespace.md) (tag `_psp`), resulting in
distinct symbol names. Translation units compiled with and without it can therefore be linked into the same program
without One Definition Rule (ODR) violations, but they cannot exchange instances of library types.
!!! tip "Workaround without the macro"
Separate the values in the stream with whitespace. The character consumed after a number is then the separator,
and whitespace before the next value is skipped anyway.
## Examples
??? example "Default behavior (macro not defined)"
Without the macro, the character after a number is consumed:
```cpp
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
std::istringstream input("1true");
json j1, j2;
input >> j1; // j1 == 1
input >> j2; // throws parse_error.101: the stream now starts at "rue"
}
```
??? example "Opt-in precise stream position (macro defined to 1)"
With the macro, the stream is positioned right after the number:
```cpp
#define JSON_PRECISE_STREAM_POSITION 1
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
std::istringstream input("1true");
json j1, j2;
input >> j1; // j1 == 1
input >> j2; // j2 == true
}
```
## See also
- [**operator>>**](../operator_gtgt.md) - deserialize from stream
- [**sax_parse**](../basic_json/sax_parse.md) - generate SAX events
## Version history
- Added in version 3.13.0.
- Planned to become the default (with the macro removed) in version 4.0.0.
+1 -6
View File
@@ -67,9 +67,7 @@ input >> j2; // parses the next value
Only numbers are affected. Values ending in a self-delimiting character do not read past themselves, so Only numbers are affected. Values ending in a self-delimiting character do not read past themselves, so
`truefalse`, `[1][2]`, `{"a":1}{"b":2}`, and `"a""b"` can be read back to back without a separator. `truefalse`, `[1][2]`, `{"a":1}{"b":2}`, and `"a""b"` can be read back to back without a separator.
Define [`JSON_PRECISE_STREAM_POSITION`](macros/json_precise_stream_position.md) to `1` to leave the terminating character in the stream This is tracked in [#5340](https://github.com/nlohmann/json/issues/5340).
instead, so that the stream is positioned right after the value for every value type and no separator is
needed. This is tracked in [#5340](https://github.com/nlohmann/json/issues/5340).
Note that reading concatenated values does **not** work for [JSON Lines](../features/parsing/json_lines.md) Note that reading concatenated values does **not** work for [JSON Lines](../features/parsing/json_lines.md)
(newline-delimited JSON) input -- see that page for why and for the recommended alternative. (newline-delimited JSON) input -- see that page for why and for the recommended alternative.
@@ -109,12 +107,9 @@ being read.
- [parse](basic_json/parse.md) - deserialize from a compatible input - [parse](basic_json/parse.md) - deserialize from a compatible input
- [`JSON_STRICT_NUL_HANDLING`](macros/json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input - [`JSON_STRICT_NUL_HANDLING`](macros/json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input
instead of treating it as end of input instead of treating it as end of input
- [`JSON_PRECISE_STREAM_POSITION`](macros/json_precise_stream_position.md) - opt in to leaving the stream positioned right after a number
## Version history ## Version history
- Added in version 1.0.0. - Added in version 1.0.0.
- `JSON_STRICT_NUL_HANDLING` added in version 3.13.0 to optionally reject a NUL byte in the input instead of treating - `JSON_STRICT_NUL_HANDLING` added in version 3.13.0 to optionally reject a NUL byte in the input instead of treating
it as end of input; planned to become the default in version 4.0.0. it as end of input; planned to become the default in version 4.0.0.
- `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave the character that terminates a number in
the stream; planned to become the default in version 4.0.0.
@@ -1,70 +0,0 @@
# Assurance case
This page argues why the library meets its security requirements. It describes the threats the library faces, where the
trust boundaries lie, and how the library's design and the [quality assurance](quality_assurance.md) counter these
threats. To report a vulnerability, see the [security policy](security_policy.md).
## Threat model
The library parses, stores, and serializes JSON values in memory. It does not open network connections, does not open
files (it only reads from streams or `std::FILE*` handles that the caller has already opened), does not read environment
variables, and does not implement cryptography or handle credentials.
The primary threat is therefore **untrusted input**: JSON text or binary data (BJData, BSON, CBOR, MessagePack, UBJSON)
that an attacker controls, passed to [`parse`](../api/basic_json/parse.md), [`accept`](../api/basic_json/accept.md),
[`sax_parse`](../api/basic_json/sax_parse.md), or one of the `from_*` functions such as
[`from_cbor`](../api/basic_json/from_cbor.md). Such input may try to
- make the library read or write out of bounds (malformed lengths, truncated input, invalid UTF-8),
- trigger undefined behavior (integer overflow in sizes or numbers, invalid casts),
- exhaust memory (huge announced sizes), or
- exhaust the call stack (deeply nested arrays and objects).
## Trust boundaries
- **Untrusted:** all serialized input read by the parser, the SAX interface, and the binary readers. The library must
handle every possible input by either producing a value or throwing a [`parse_error`](../home/exceptions.md#parse-errors)
(or returning `false` when exceptions are disabled for the call).
- **Trusted:** the C++ code that calls the library. Calling a function with violated preconditions, for instance
accessing an array with [`operator[]`](../api/basic_json/operator%5B%5D.md) out of range, is a programming error and
not a security boundary. Such preconditions are checked with [runtime assertions](../features/assertions.md) in debug
builds; functions such as [`at`](../api/basic_json/at.md) offer checked access with exceptions.
## Secure design
- **Strict parsing.** The parser accepts exactly the JSON grammar of [RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259).
Extensions such as [comments](../features/comments.md) and [trailing commas](../features/trailing_commas.md) must be
enabled explicitly. Invalid UTF-8 is rejected.
- **Errors are reported, not ignored.** Malformed input results in a [`parse_error`](../home/exceptions.md#parse-errors)
with the byte position of the error. Binary readers do not trust announced sizes: strings and binary values grow
only as bytes are actually read, arrays reserve at most a fixed number of elements up front, and sizes that no
container can hold are rejected.
- **Memory is owned by values.** Each `basic_json` value owns its content, and there is no manual memory management in
user code. The destructor does not recurse, so destroying a deeply nested value does not exhaust the stack.
- **Bounded recursion.** The JSON parser and the binary readers keep their state in explicit stacks instead of
recursing per nesting level. Operations that walk a value, such as [`dump`](../api/basic_json/dump.md), copying,
hashing, and [`merge_patch`](../api/basic_json/merge_patch.md), recurse only up to a fixed depth and continue with an
explicit stack below it. Some operations, such as comparison, [`diff`](../api/basic_json/diff.md),
[`flatten`](../api/basic_json/flatten.md), and the binary writers, still recurse once per nesting level; work on them
is in progress. Applications that process untrusted input can limit its nesting depth with a
[parser callback](../features/parsing/parser_callbacks.md).
- **Invariants are checked.** The class invariant (for instance, that the pointer for the stored type is never null) is
checked with runtime assertions throughout the test suite.
## Common weaknesses
The following table maps the relevant classes of the [Common Weakness Enumeration](https://cwe.mitre.org) to the
measures that counter them. The measures are described in detail in [Quality assurance](quality_assurance.md).
| Weakness | Countermeasures |
|---------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|
| Out-of-bounds read/write ([CWE-125](https://cwe.mitre.org/data/definitions/125.html), [CWE-787](https://cwe.mitre.org/data/definitions/787.html)) | bounds checks on all reads from the input; AddressSanitizer and Valgrind on the test suite; OSS-Fuzz |
| Integer overflow ([CWE-190](https://cwe.mitre.org/data/definitions/190.html)) | UndefinedBehaviorSanitizer with integer overflow detection; Clang-Tidy; Cppcheck |
| Use after free, double free ([CWE-416](https://cwe.mitre.org/data/definitions/416.html), [CWE-415](https://cwe.mitre.org/data/definitions/415.html)) | ownership of all memory by values; AddressSanitizer and Valgrind; Clang Static Analyzer |
| Memory leaks ([CWE-401](https://cwe.mitre.org/data/definitions/401.html)) | Valgrind (Memcheck) on the test suite |
| Uncontrolled recursion ([CWE-674](https://cwe.mitre.org/data/definitions/674.html)) | iterative parser, binary readers, and destructor; bounded recursion in value operations; tests with deeply nested inputs |
| Uncontrolled resource consumption ([CWE-400](https://cwe.mitre.org/data/definitions/400.html)) | allocations based on announced sizes are capped; OSS-Fuzz with memory limits |
| Undefined behavior in general ([CWE-758](https://cwe.mitre.org/data/definitions/758.html)) | UndefinedBehaviorSanitizer; runtime assertions; Clang-Tidy, Cppcheck, Clang Static Analyzer, Infer |
In addition, every line of the library is covered by the unit tests, and all parsers are fuzz-tested around the clock
by [OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json).
-2
View File
@@ -5,6 +5,4 @@
- [Contribution Guidelines](contribution_guidelines.md) - guidelines how to contribute to this project - [Contribution Guidelines](contribution_guidelines.md) - guidelines how to contribute to this project
- [Governance](governance.md) - the governance model of this project - [Governance](governance.md) - the governance model of this project
- [Quality Assurance](quality_assurance.md) - how the quality of this project is assured - [Quality Assurance](quality_assurance.md) - how the quality of this project is assured
- [Roadmap](roadmap.md) - what the project will and will not do
- [Security Policy](security_policy.md) - the security policy of the project - [Security Policy](security_policy.md) - the security policy of the project
- [Assurance Case](assurance_case.md) - why the library meets its security requirements
@@ -164,9 +164,6 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
- [x] The parser is tested against extensive correctness suites for JSON compliance. - [x] The parser is tested against extensive correctness suites for JSON compliance.
- [x] In addition, the library is continuously fuzz-tested at [OSS-Fuzz](https://google.github.io/oss-fuzz/) where the - [x] In addition, the library is continuously fuzz-tested at [OSS-Fuzz](https://google.github.io/oss-fuzz/) where the
library is checked against billions of inputs. library is checked against billions of inputs.
- [x] Every crash reported by OSS-Fuzz is fixed together with a unit test that reproduces it, and the fix references
the OSS-Fuzz issue. The round-trip checks of the fuzzer drivers are also part of the unit tests. See the
[fuzz testing documentation](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports).
## Static analysis ## Static analysis
-43
View File
@@ -1,43 +0,0 @@
# Roadmap
This page describes what the project intends to do, and what it does not intend to do, over the next year. Concrete
work items are tracked in the [GitHub milestones](https://github.com/nlohmann/json/milestones) and the
[issue tracker](https://github.com/nlohmann/json/issues).
## What the project will do
- **Keep the C++11 baseline.** The library will continue to compile with every
[supported C++11 compiler](https://github.com/nlohmann/json/blob/develop/README.md#supported-compilers). Features of
later standards are only used when they are guarded by the `JSON_HAS_CPP_*` macros.
- **Stay conformant to JSON.** The parser and serializer follow [RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259).
Extensions such as [comments](../features/comments.md) or [trailing commas](../features/trailing_commas.md) remain
opt-in.
- **Keep the 3.x public API stable.** Releases follow [semantic versioning](https://semver.org). Changes that would
break existing code are only added behind a feature macro, so users can opt in and test their code before a next
major release.
- **Support a broad range of compilers and platforms.** The [CI](quality_assurance.md) keeps testing old and new
versions of GCC, Clang, MSVC, and other compilers on Linux, macOS, and Windows.
- **Keep the quality assurance up.** Every change keeps the test coverage at 100%, passes the static and dynamic
analysis, and is fuzz-tested by OSS-Fuzz, see [Quality assurance](quality_assurance.md).
- **Harden the library against hostile input.** Handling deeply nested values without exhausting the call stack is
ongoing work.
- **Fix bugs and security issues** reported through the issue tracker and the [security policy](security_policy.md).
## What the project will not do
- **Break the public API of version 3.x.** See the
[contribution guidelines](https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#break-the-public-api)
for what counts as a breaking change.
- **Require a newer C++ standard than C++11.**
- **Break JSON conformance** or enable non-standard extensions by default.
- **Add dependencies** or require a build step. The library remains header-only, and the single header
`json.hpp` remains a complete distribution.
- **Trade simplicity for speed or memory efficiency.** Performance improvements are welcome, but the library is not
meant to compete with the fastest JSON libraries, see [Design goals](../home/design_goals.md).
## Version 4.0
There is no decision yet on whether or when a version 4.0 with breaking changes will be released. Proposals that need
a major version, for instance stricter type conversions, are collected in issue
[#3453](https://github.com/nlohmann/json/issues/3453). Until then, such changes are only added as opt-in behavior
behind feature macros.
@@ -208,16 +208,6 @@ The library maps BJData types to JSON value types as follows:
The mapping is **complete** in the sense that any BJData value can be converted to a JSON value. The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
!!! info "Round trips"
A value returned by [`from_bjdata`](../../api/basic_json/from_bjdata.md) can be serialized with
[`to_bjdata`](../../api/basic_json/to_bjdata.md) using any combination of options and parsed back into an equal
value, and serializing that value again with the same options produces the same bytes. The exception is binary
values: they are only written as an optimized binary array (`[$B`) if Draft 3 is enabled and both `use_size` and
`use_type` are set. Otherwise, they are written as arrays of integers and parsed back as such (see the notes on
binary values above), and serializing such an array again may choose different, but equally valid, type markers.
The bytes can then differ, but parsing them again yields the same value.
??? example ??? example
```cpp ```cpp
-9
View File
@@ -98,15 +98,6 @@ rather than descending into a bounded number of levels first, which is slower bu
See [full documentation of `JSON_NO_THREAD_LOCAL`](../api/macros/json_no_thread_local.md). See [full documentation of `JSON_NO_THREAD_LOCAL`](../api/macros/json_no_thread_local.md).
## `JSON_PRECISE_STREAM_POSITION`
When defined to `1`, [`operator>>`](../api/operator_gtgt.md) and non-strict
[`sax_parse`](../api/basic_json/sax_parse.md) leave an input stream positioned right after the parsed value, instead of
also consuming the character that terminates a number. The default value is `0`, which preserves the existing behavior;
this is planned to become the default in version 4.0.0.
See [full documentation of `JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md).
## `JSON_SKIP_LIBRARY_VERSION_CHECK` ## `JSON_SKIP_LIBRARY_VERSION_CHECK`
When defined, the library will not create a compiler warning when a different version of the library was already When defined, the library will not create a compiler warning when a different version of the library was already
-1
View File
@@ -18,7 +18,6 @@ The complete default namespace name is derived as follows:
- [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) defined non-zero appends `_dp`. - [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) defined non-zero appends `_dp`.
- [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) defined non-zero appends - [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) defined non-zero appends
`_bics`. `_bics`.
- [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) defined non-zero appends `_psp`.
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by - The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component) underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
below. below.
+1 -2
View File
@@ -41,8 +41,7 @@ document followed by trailing bytes" is accepted rather than rejected. If you ar
reject any input that is not exactly one JSON document, prefer `parse`. reject any input that is not exactly one JSON document, prefer `parse`.
When using `operator>>` to read several concatenated values this way, a value that is a number must be followed by When using `operator>>` to read several concatenated values this way, a value that is a number must be followed by
whitespace, because `operator>>` consumes the character that terminates a number, unless whitespace, because `operator>>` consumes the character that terminates a number — see the
[`JSON_PRECISE_STREAM_POSITION`](../../api/macros/json_precise_stream_position.md) is defined to `1` — see the
[`operator>>` notes](../../api/operator_gtgt.md#notes) for details and examples. [`operator>>` notes](../../api/operator_gtgt.md#notes) for details and examples.
## SAX vs. DOM parsing ## SAX vs. DOM parsing
@@ -562,7 +562,11 @@ binary32 or binary64 field and have no encoding for `#!cpp long double`.
## `AllocatorType` ## `AllocatorType`
`AllocatorType` is instantiated with **one** argument, for each of `object_t`, `array_t`, `string_t`, `binary_t`, `AllocatorType` is instantiated with **one** argument, for each of `object_t`, `array_t`, `string_t`, `binary_t`,
`basic_json`, and `#!cpp std::pair<const StringType, basic_json>`. `basic_json`, `#!cpp std::pair<const StringType, basic_json>`, and `#!cpp std::pair<StringType, basic_json>`.
`AllocatorType` is not the only allocator a `basic_json` uses. It allocates the JSON values themselves, but most
temporary storage is allocated with `#!cpp std::allocator`. This includes the parser's stacks and the stacks that
process deeply nested values without recursion.
### Always required ### Always required
+35 -177
View File
@@ -1,125 +1,34 @@
# Architecture # Architecture
This page gives a high-level overview of the library's architecture. It should help new contributors to get an idea of !!! info
the used concepts and where to make changes.
This page is still under construction. Its goal is to provide a high-level overview of the library's architecture.
This should help new contributors to get an idea of the used concepts and where to make changes.
## Overview ## Overview
The library is built around a single class template, [`nlohmann::basic_json`](../api/basic_json/index.md). A The main structure is class [nlohmann::basic_json](../api/basic_json/index.md).
`basic_json` value is a node in a tree of JSON values. All other components either create such a tree from an input
(parsing), write a tree to an output (serialization), or give access to it (iterators, JSON Pointer, conversions).
```mermaid - public API
flowchart LR - container interface
input[/"input<br>(string, stream,<br>iterator range, file)"/] - iterators
ia["input adapter"]
lexer["lexer"]
parser["parser"]
breader["binary_reader"]
sax["SAX interface"]
value[("basic_json<br>value tree")]
serializer["serializer"]
bwriter["binary_writer"]
oa["output adapter"]
output[/"output<br>(string, stream,<br>vector)"/]
input --> ia ## Template specializations
ia --> lexer --> parser --> sax
ia --> breader --> sax
sax --> value
value --> serializer --> oa
value --> bwriter --> oa
oa --> output
```
- **JSON text** is read by an [input adapter](#input-adapters), tokenized by the lexer, and turned into SAX events by - describe template parameters of `basic_json`
the parser. - [`json`](../api/json.md)
- **Binary formats** (BJData, BSON, CBOR, MessagePack, UBJSON) are read by an input adapter and turned into the same SAX - [`ordered_json`](../api/ordered_json.md) via [`ordered_map`](../api/ordered_map.md)
events by the `binary_reader`.
- A [SAX consumer](#sax-interface) receives the events. The one used by [`parse`](../api/basic_json/parse.md) builds a
`basic_json` value tree.
- The `serializer` (JSON text) or the `binary_writer` (binary formats) writes a value tree to an
[output adapter](#output-adapters).
## Source layout
The public headers are in [`include/nlohmann`](https://github.com/nlohmann/json/tree/develop/include/nlohmann):
- [`json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json.hpp) defines class [`basic_json`](../api/basic_json/index.md).
- [`json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json_fwd.hpp) contains forward declarations.
- [`adl_serializer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/adl_serializer.hpp), [`byte_container_with_subtype.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/byte_container_with_subtype.hpp), and [`ordered_map.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/ordered_map.hpp) define
[`adl_serializer`](../api/adl_serializer/index.md),
[`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md), and
[`ordered_map`](../api/ordered_map.md).
Everything else lives in [`detail/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail) and namespace `nlohmann::detail`, which is not part of the public API. Paths
below are relative to `include/nlohmann`.
| Component | Location |
|-----------|----------|
| Value type enumeration | [`detail/value_t.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/value_t.hpp) |
| Input adapters | [`detail/input/input_adapters.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/input_adapters.hpp) |
| Lexer | [`detail/input/lexer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/lexer.hpp), [`detail/input/number_parse.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/number_parse.hpp), [`detail/input/string_scan.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/string_scan.hpp) |
| Parser | [`detail/input/parser.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/parser.hpp) |
| SAX interface and DOM builders | [`detail/input/json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp) |
| Binary format readers | [`detail/input/binary_reader.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/binary_reader.hpp) |
| JSON serializer | [`detail/output/serializer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/serializer.hpp), [`detail/conversions/to_chars.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/to_chars.hpp) |
| Binary format writers | [`detail/output/binary_writer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/binary_writer.hpp) |
| Output adapters | [`detail/output/output_adapters.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/output_adapters.hpp) |
| Iterators | [`detail/iterators/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail/iterators) |
| Conversions from/to arbitrary types | [`detail/conversions/from_json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/from_json.hpp), [`detail/conversions/to_json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/to_json.hpp) |
| JSON Pointer | [`detail/json_pointer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/json_pointer.hpp) |
| Exceptions | [`detail/exceptions.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/exceptions.hpp) |
| Type traits and C++ feature backports | [`detail/meta/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail/meta) |
| Macros | [`detail/macro_scope.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/macro_scope.hpp), [`detail/macro_unscope.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/macro_unscope.hpp), [`detail/abi_macros.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/abi_macros.hpp) |
The single-header version [`single_include/nlohmann/json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp)
is generated from these files with `make amalgamate` and must not be edited by hand.
## Template parameters
[`basic_json`](../api/basic_json/index.md) is parameterized by the types it uses to store values and to convert from and to other types:
| Template parameter | Default | Used for |
|----------------------|-----------------------------|-------------------------------------------------------------------|
| `ObjectType` | `std::map` | objects, see [`object_t`](../api/basic_json/object_t.md) |
| `ArrayType` | `std::vector` | arrays, see [`array_t`](../api/basic_json/array_t.md) |
| `StringType` | `std::string` | strings and object keys, see [`string_t`](../api/basic_json/string_t.md) |
| `BooleanType` | `bool` | Booleans, see [`boolean_t`](../api/basic_json/boolean_t.md) |
| `NumberIntegerType` | `std::int64_t` | signed integers, see [`number_integer_t`](../api/basic_json/number_integer_t.md) |
| `NumberUnsignedType` | `std::uint64_t` | unsigned integers, see [`number_unsigned_t`](../api/basic_json/number_unsigned_t.md) |
| `NumberFloatType` | `double` | floating-point numbers, see [`number_float_t`](../api/basic_json/number_float_t.md) |
| `AllocatorType` | `std::allocator` | allocating objects, arrays, strings, and binary values |
| `JSONSerializer` | `adl_serializer` | conversions from/to other types, see [`adl_serializer`](../api/adl_serializer/index.md) |
| `BinaryType` | `std::vector<std::uint8_t>` | binary values, see [`binary_t`](../api/basic_json/binary_t.md) |
| `CustomBaseClass` | `void` | an optional base class, see [`json_base_class_t`](../api/basic_json/json_base_class_t.md) |
The library provides two specializations:
- [`json`](../api/json.md) uses all default template arguments.
- [`ordered_json`](../api/ordered_json.md) uses [`ordered_map`](../api/ordered_map.md) as `ObjectType` to keep the
insertion order of object keys.
The requirements on the template arguments are listed in
[Template Parameter Requirements](../features/types/template_parameters.md).
## Value storage ## Value storage
Each [`basic_json`](../api/basic_json/index.md) value stores its content as a tagged union: an enumeration [`value_t`](../api/basic_json/value_t.md) Values are stored as a tagged union of [value_t](../api/basic_json/value_t.md) and json_value.
names the type of the value, and a union `json_value` holds the value itself. Both are members of the nested struct
`data`, which is the only data member `m_data` of `basic_json`:
```cpp ```cpp
struct data /// the type of the current element
{ value_t m_type = value_t::null;
/// the type of the current element
value_t m_type = value_t::null;
/// the value of the current element /// the value of the current element
json_value m_value = {}; json_value m_value = {};
};
data m_data = {};
``` ```
with with
@@ -159,83 +68,42 @@ union json_value {
}; };
``` ```
Objects, arrays, strings, and binary values are allocated on the heap with `AllocatorType`, and the union only stores a ## Parsing inputs (deserialization)
pointer to them. This keeps a `basic_json` value small: one pointer-sized union and one byte for the type. The class
maintains the invariant that the pointer matching `m_type` is never null; `assert_invariant()` checks it with
[runtime assertions](../features/assertions.md).
## Input adapters Input is read via **input adapters** that abstract a source with a common interface:
Input is read via **input adapters** that abstract a source. Every input adapter provides this interface:
```cpp ```cpp
/// the type of the characters in the input /// read a single character
using char_type = ...; std::char_traits<char>::int_type get_character() noexcept;
/// read a single character; returns std::char_traits<char_type>::eof() at the end of the input /// read multiple characters to a destination buffer and
typename std::char_traits<char_type>::int_type get_character(); /// returns the number of characters successfully read
/// read up to count * sizeof(T) bytes into dest and return the number of bytes read
/// (used by the binary readers)
template<class T> template<class T>
std::size_t get_elements(T* dest, std::size_t count = 1); std::size_t get_elements(T* dest, std::size_t count = 1);
``` ```
The lexer detects two optional extensions at compile time. Only `iterator_input_adapter` provides them, and only for List examples of input adapters.
random-access input of single-byte characters:
- `supports_seek`, `get_consumed_count()`, and `copy_consumed_range()` let the lexer reconstruct already consumed input ## SAX Interface
for error messages instead of copying every character it reads.
- `supports_bulk_scan`, `bulk_data()`, `bulk_remaining()`, and `bulk_skip()` let the lexer scan strings directly in
contiguous memory, several bytes at a time.
The function `input_adapter` picks the right adapter for the argument passed to `parse`, `accept`, `sax_parse`, or the TODO
`from_*` functions:
- `iterator_input_adapter` reads from an iterator range, which also covers strings, containers, and pointers. ## Writing outputs (serialization)
- `wide_string_input_adapter` reads from ranges of `wchar_t`, `char16_t`, or `char32_t` and converts them to UTF-8.
It cannot be used for binary formats; its `get_elements()` throws.
- `input_stream_adapter` reads from a `std::istream`.
- `file_input_adapter` reads from a `std::FILE*`.
## SAX interface
The parser does not build values itself. It reports what it reads as events to a [SAX](../features/parsing/sax_interface.md)
consumer, which implements the interface [`json_sax`](../api/json_sax/index.md): `null`, `boolean`, `number_integer`,
`number_unsigned`, `number_float`, `string`, `binary`, `start_object`, `key`, `end_object`, `start_array`, `end_array`,
and `parse_error`.
The library comes with two consumers in `detail/input/json_sax.hpp`:
- `json_sax_dom_parser` builds a [`basic_json`](../api/basic_json/index.md) value tree. [`parse`](../api/basic_json/parse.md) uses it.
- `json_sax_dom_callback_parser` does the same, but calls a [parser callback](../features/parsing/parser_callbacks.md)
for each event, which can skip values. `parse` uses it when a callback is given.
The `binary_reader` emits the same events for binary formats, so [`sax_parse`](../api/basic_json/sax_parse.md) works
with a user-defined consumer for JSON and for all binary formats alike.
## Output adapters
Output is written via **output adapters**: Output is written via **output adapters**:
```cpp ```cpp
template<typename T>
void write_character(CharType c); void write_character(CharType c);
template<typename CharType>
void write_characters(const CharType* s, std::size_t length); void write_characters(const CharType* s, std::size_t length);
``` ```
The `serializer` (used by [`dump`](../api/basic_json/dump.md) and [`operator<<`](../api/operator_ltlt.md)) and the List examples of output adapters.
`binary_writer` (used by the `to_*` functions) write to one of these adapters:
- `output_vector_adapter` appends to a `std::vector`.
- `output_stream_adapter` writes to a `std::ostream`.
- `output_string_adapter` appends to a string.
## Value conversion ## Value conversion
Values are converted from and to other types with the `JSONSerializer` template parameter. The default,
[`adl_serializer`](../api/adl_serializer/index.md), calls the free functions
```cpp ```cpp
template<class T> template<class T>
void to_json(basic_json& j, const T& t); void to_json(basic_json& j, const T& t);
@@ -244,23 +112,13 @@ template<class T>
void from_json(const basic_json& j, T& t); void from_json(const basic_json& j, T& t);
``` ```
found by argument-dependent lookup. The library defines them for standard types in `detail/conversions`; users add them
for their own types, see [Arbitrary Type Conversions](../features/arbitrary_types.md). The
[serialization macros](../features/macros.md) generate these functions.
## Additional features ## Additional features
- [JSON Pointer](../features/json_pointer.md) (class `json_pointer`) addresses values inside a tree. It is also the - JSON Pointers
basis of [JSON Patch](../features/json_patch.md). - Binary formats
- [Binary formats](../features/binary_formats/index.md) are read by `binary_reader` and written by `binary_writer`. - Custom base class
- A [custom base class](../api/basic_json/json_base_class_t.md) can add members to every [`basic_json`](../api/basic_json/index.md) value. - Conversion macros
- [Serialization macros](../features/macros.md) generate `to_json` and `from_json` functions for user-defined types.
## Details namespace ## Details namespace
Namespace `nlohmann::detail` contains all implementation details. It is not part of the public API and may change in any - C++ feature backports
release. Besides the components above, it contains:
- type traits to detect the capabilities of user-defined types (`detail/meta/type_traits.hpp`),
- backports of C++14/17 features to C++11 (`detail/meta/cpp_future.hpp`), and
- helpers such as `string_concat` and `string_escape`.
-3
View File
@@ -293,7 +293,6 @@ nav:
- 'JSON_NOEXCEPTION': api/macros/json_noexception.md - 'JSON_NOEXCEPTION': api/macros/json_noexception.md
- 'JSON_NO_IO': api/macros/json_no_io.md - 'JSON_NO_IO': api/macros/json_no_io.md
- 'JSON_NO_THREAD_LOCAL': api/macros/json_no_thread_local.md - 'JSON_NO_THREAD_LOCAL': api/macros/json_no_thread_local.md
- 'JSON_PRECISE_STREAM_POSITION': api/macros/json_precise_stream_position.md
- 'JSON_SKIP_LIBRARY_VERSION_CHECK': api/macros/json_skip_library_version_check.md - 'JSON_SKIP_LIBRARY_VERSION_CHECK': api/macros/json_skip_library_version_check.md
- 'JSON_SKIP_UNSUPPORTED_COMPILER_CHECK': api/macros/json_skip_unsupported_compiler_check.md - 'JSON_SKIP_UNSUPPORTED_COMPILER_CHECK': api/macros/json_skip_unsupported_compiler_check.md
- 'JSON_STRICT_NUL_HANDLING': api/macros/json_strict_nul_handling.md - 'JSON_STRICT_NUL_HANDLING': api/macros/json_strict_nul_handling.md
@@ -318,9 +317,7 @@ nav:
- community/contribution_guidelines.md - community/contribution_guidelines.md
- community/quality_assurance.md - community/quality_assurance.md
- community/governance.md - community/governance.md
- community/roadmap.md
- community/security_policy.md - community/security_policy.md
- community/assurance_case.md
# Extras # Extras
extra: extra:
+4 -15
View File
@@ -38,10 +38,6 @@
#define JSON_BRACE_INIT_COPY_SEMANTICS 0 #define JSON_BRACE_INIT_COPY_SEMANTICS 0
#endif #endif
#ifndef JSON_PRECISE_STREAM_POSITION
#define JSON_PRECISE_STREAM_POSITION 0
#endif
#if JSON_DIAGNOSTICS #if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag #define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else #else
@@ -66,28 +62,21 @@
#define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS #define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS
#endif #endif
#if JSON_PRECISE_STREAM_POSITION
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION _psp
#else
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION #ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0 #define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif #endif
// Construct the namespace ABI tags component // Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e) json_abi ## a ## b ## c ## d ## e #define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d) json_abi ## a ## b ## c ## d
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e) \ #define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e) NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d)
#define NLOHMANN_JSON_ABI_TAGS \ #define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \ NLOHMANN_JSON_ABI_TAGS_CONCAT( \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \ NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \
NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \ NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \ NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \ NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS)
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION)
// Construct the namespace version component // Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \ #define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
@@ -101,11 +101,6 @@ class input_stream_adapter
// maintain ifstream flags, except eof // maintain ifstream flags, except eof
if (is != nullptr) if (is != nullptr)
{ {
#if JSON_PRECISE_STREAM_POSITION
// consume the character last returned by get_character() unless it
// was given back with release_lookahead()
commit_lookahead();
#endif
is->clear(is->rdstate() & std::ios::eofbit); is->clear(is->rdstate() & std::ios::eofbit);
} }
} }
@@ -119,58 +114,6 @@ class input_stream_adapter
input_stream_adapter& operator=(input_stream_adapter&) = delete; input_stream_adapter& operator=(input_stream_adapter&) = delete;
input_stream_adapter& operator=(input_stream_adapter&&) = delete; input_stream_adapter& operator=(input_stream_adapter&&) = delete;
#if JSON_PRECISE_STREAM_POSITION
input_stream_adapter(input_stream_adapter&& rhs) noexcept
: is(rhs.is), sb(rhs.sb), lookahead(rhs.lookahead)
{
rhs.is = nullptr;
rhs.sb = nullptr;
rhs.lookahead = false;
}
// Whether the character last returned by get_character() can be given back
// to the input with release_lookahead().
static constexpr bool supports_lookahead = true;
// std::istream/std::streambuf use std::char_traits<char>::to_int_type, to
// ensure that std::char_traits<char>::eof() and the character 0xFF do not
// end up as the same value, e.g., 0xFFFFFFFF.
//
// The character is peeked rather than consumed: it is only stepped over
// once the next character is requested, or when the adapter is destroyed.
// Until then, release_lookahead() can leave it in the input.
std::char_traits<char>::int_type get_character()
{
if (lookahead)
{
// step over the character returned by the previous call
sb->sbumpc();
}
auto res = sb->sgetc();
// set eof manually, as we don't use the istream interface.
if (JSON_HEDLEY_UNLIKELY(res == std::char_traits<char>::eof()))
{
// there is nothing to step over next time
lookahead = false;
is->clear(is->rdstate() | std::ios::eofbit);
}
else
{
lookahead = true;
}
return res;
}
// Leave the character last returned by get_character() in the input, so
// that the next read from the stream - by this adapter or by the caller
// once parsing is done - sees it again. Unlike putting a consumed
// character back, this cannot fail.
void release_lookahead() noexcept
{
lookahead = false;
}
#else
input_stream_adapter(input_stream_adapter&& rhs) noexcept input_stream_adapter(input_stream_adapter&& rhs) noexcept
: is(rhs.is), sb(rhs.sb) : is(rhs.is), sb(rhs.sb)
{ {
@@ -181,9 +124,6 @@ class input_stream_adapter
// std::istream/std::streambuf use std::char_traits<char>::to_int_type, to // std::istream/std::streambuf use std::char_traits<char>::to_int_type, to
// ensure that std::char_traits<char>::eof() and the character 0xFF do not // ensure that std::char_traits<char>::eof() and the character 0xFF do not
// end up as the same value, e.g., 0xFFFFFFFF. // end up as the same value, e.g., 0xFFFFFFFF.
//
// The character is consumed, so the character that terminates a number
// stays consumed after parsing; see JSON_PRECISE_STREAM_POSITION.
std::char_traits<char>::int_type get_character() std::char_traits<char>::int_type get_character()
{ {
auto res = sb->sbumpc(); auto res = sb->sbumpc();
@@ -194,14 +134,10 @@ class input_stream_adapter
} }
return res; return res;
} }
#endif
template<class T> template<class T>
std::size_t get_elements(T* dest, std::size_t count = 1) std::size_t get_elements(T* dest, std::size_t count = 1)
{ {
#if JSON_PRECISE_STREAM_POSITION
commit_lookahead();
#endif
auto res = static_cast<std::size_t>(sb->sgetn(reinterpret_cast<char*>(dest), static_cast<std::streamsize>(count * sizeof(T)))); auto res = static_cast<std::size_t>(sb->sgetn(reinterpret_cast<char*>(dest), static_cast<std::streamsize>(count * sizeof(T))));
if (JSON_HEDLEY_UNLIKELY(res < count * sizeof(T))) if (JSON_HEDLEY_UNLIKELY(res < count * sizeof(T)))
{ {
@@ -211,27 +147,9 @@ class input_stream_adapter
} }
private: private:
#if JSON_PRECISE_STREAM_POSITION
// Step over the character last returned by get_character(). The character
// has already been peeked successfully, so for every streambuf with a get
// area this is a pointer increment that cannot fail.
void commit_lookahead()
{
if (lookahead)
{
lookahead = false;
sb->sbumpc();
}
}
#endif
/// the associated input stream /// the associated input stream
std::istream* is = nullptr; std::istream* is = nullptr;
std::streambuf* sb = nullptr; std::streambuf* sb = nullptr;
#if JSON_PRECISE_STREAM_POSITION
/// whether get_character() peeked a character that is not consumed yet
bool lookahead = false;
#endif
}; };
#endif // JSON_NO_IO #endif // JSON_NO_IO
-65
View File
@@ -127,25 +127,6 @@ constexpr bool input_adapter_supports_seek(std::false_type /*detected*/)
return false; return false;
} }
// Detect whether an input adapter reads with one character of lookahead that
// can be left in the input (see input_stream_adapter::supports_lookahead,
// which is only defined with JSON_PRECISE_STREAM_POSITION), detected like
// supports_seek above.
template<typename InputAdapterType>
using detect_supports_lookahead = decltype(InputAdapterType::supports_lookahead);
template<typename InputAdapterType>
constexpr bool input_adapter_supports_lookahead(std::true_type /*detected*/)
{
return InputAdapterType::supports_lookahead;
}
template<typename InputAdapterType>
constexpr bool input_adapter_supports_lookahead(std::false_type /*detected*/)
{
return false;
}
// Detect whether an input adapter exposes a contiguous byte block that the // Detect whether an input adapter exposes a contiguous byte block that the
// lexer can scan directly (see iterator_input_adapter::supports_bulk_scan). // lexer can scan directly (see iterator_input_adapter::supports_bulk_scan).
// Adapters without the flag - file, stream, wide-string, user-defined - fall // Adapters without the flag - file, stream, wide-string, user-defined - fall
@@ -186,12 +167,6 @@ class lexer : public lexer_base<BasicJsonType>
static constexpr bool lazy_token_string = static constexpr bool lazy_token_string =
input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {}); input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {});
/// whether a simulated unget can be passed on to the input adapter, which
/// then leaves the character in the input; see
/// input_adapter_supports_lookahead
static constexpr bool can_release_lookahead =
input_adapter_supports_lookahead<InputAdapterType>(is_detected<detect_supports_lookahead, InputAdapterType> {});
/// whether string scanning may bulk-consume runs of ordinary characters /// whether string scanning may bulk-consume runs of ordinary characters
/// directly from a contiguous input buffer (SWAR fast path). This requires /// directly from a contiguous input buffer (SWAR fast path). This requires
/// the token to be reconstructible lazily (lazy_token_string), so bypassing /// the token to be reconstructible lazily (lazy_token_string), so bypassing
@@ -1923,21 +1898,6 @@ scan_number_done:
uncapture_char(std::integral_constant<bool, lazy_token_string> {}); uncapture_char(std::integral_constant<bool, lazy_token_string> {});
} }
/// adapter without lookahead: nothing to do (see release_lookahead)
void release_lookahead_impl(std::false_type /*can_release*/) const noexcept {}
/// adapter with lookahead: leave the character in the input instead
void release_lookahead_impl(std::true_type /*can_release*/)
{
if (next_unget)
{
// the character is read from the input again rather than replayed
// from current, so the adapter must not step over it
next_unget = false;
ia.release_lookahead();
}
}
/// seekable adapter: nothing was captured, so nothing to undo /// seekable adapter: nothing was captured, so nothing to undo
void uncapture_char(std::true_type /*lazy*/) const noexcept {} void uncapture_char(std::true_type /*lazy*/) const noexcept {}
@@ -2001,31 +1961,6 @@ scan_number_done:
return position; return position;
} }
/*!
@brief pass a pending simulated unget on to the input
unget() only rewinds the lexer's own bookkeeping, so the character that
terminated the last token (e.g. the character after a number) would still
be stepped over when the input adapter is done. Callers that hand the
input back to the user afterwards - operator>> and non-strict sax_parse -
call this once when scanning is done, so that the input is positioned
right after the value.
Adapters without lookahead (see input_adapter_supports_lookahead) are not
handed back to the user, so this is a no-op for them. Without
JSON_PRECISE_STREAM_POSITION, no adapter has lookahead, so this is always a
no-op and the terminating character stays consumed.
Scanning may continue after this call: @a next_unget is cleared, and the
character is read from the input again instead of being replayed from
@a current. A pending unget of EOF needs no special case, because reaching
EOF leaves no lookahead to release.
*/
void release_lookahead()
{
release_lookahead_impl(std::integral_constant<bool, can_release_lookahead> {});
}
#if JSON_DIAGNOSTIC_POSITIONS #if JSON_DIAGNOSTIC_POSITIONS
/// return the offset of the first character of the last read token; unlike /// return the offset of the first character of the last read token; unlike
/// the token's parsed value, this accounts for escape sequences /// the token's parsed value, this accounts for escape sequences
+16 -45
View File
@@ -100,22 +100,13 @@ class parser
json_sax_dom_callback_parser<BasicJsonType, InputAdapterType> sdp(result, callback, allow_exceptions, &m_lexer); json_sax_dom_callback_parser<BasicJsonType, InputAdapterType> sdp(result, callback, allow_exceptions, &m_lexer);
sax_parse_internal(&sdp); sax_parse_internal(&sdp);
if (strict) // in strict mode, input must be completely read
if (strict && (get_token() != token_type::end_of_input))
{ {
// in strict mode, input must be completely read sdp.parse_error(m_lexer.get_position(),
if (get_token() != token_type::end_of_input) m_lexer.get_token_string(),
{ parse_error::create(101, m_lexer.get_position(),
sdp.parse_error(m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(),
exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
} }
// in case of an error, return a discarded value // in case of an error, return a discarded value
@@ -137,20 +128,12 @@ class parser
json_sax_dom_parser<BasicJsonType, InputAdapterType> sdp(result, allow_exceptions, &m_lexer); json_sax_dom_parser<BasicJsonType, InputAdapterType> sdp(result, allow_exceptions, &m_lexer);
sax_parse_internal(&sdp); sax_parse_internal(&sdp);
if (strict) // in strict mode, input must be completely read
if (strict && (get_token() != token_type::end_of_input))
{ {
// in strict mode, input must be completely read sdp.parse_error(m_lexer.get_position(),
if (get_token() != token_type::end_of_input) m_lexer.get_token_string(),
{ parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// see above
m_lexer.release_lookahead();
} }
// in case of an error, return a discarded value // in case of an error, return a discarded value
@@ -183,24 +166,12 @@ class parser
(void)detail::is_sax_static_asserts<SAX, BasicJsonType> {}; (void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
const bool result = sax_parse_internal(sax); const bool result = sax_parse_internal(sax);
if (result) // strict mode: next byte must be EOF
if (result && strict && (get_token() != token_type::end_of_input))
{ {
if (strict) return sax->parse_error(m_lexer.get_position(),
{ m_lexer.get_token_string(),
// strict mode: next byte must be EOF parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
if (get_token() != token_type::end_of_input)
{
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
} }
return result; return result;
@@ -45,7 +45,6 @@
#undef JSON_HAS_STATIC_RTTI #undef JSON_HAS_STATIC_RTTI
#undef JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON #undef JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
#undef JSON_BRACE_INIT_COPY_SEMANTICS #undef JSON_BRACE_INIT_COPY_SEMANTICS
#undef JSON_PRECISE_STREAM_POSITION
#endif #endif
#include <nlohmann/thirdparty/hedley/hedley_undef.hpp> #include <nlohmann/thirdparty/hedley/hedley_undef.hpp>
+17 -34
View File
@@ -978,7 +978,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>; using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>;
/// scratch space to build the key skeleton of an object copy in one go /// scratch space to build the key skeleton of an object copy in one go
using copy_scratch_t = std::vector<std::pair<typename object_t::key_type, basic_json>>; using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
/// @brief copy everything of @a src into @a dst but its type and value /// @brief copy everything of @a src into @a dst but its type and value
static void copy_metadata(const basic_json& src, basic_json& dst) static void copy_metadata(const basic_json& src, basic_json& dst)
@@ -1243,27 +1244,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
} }
/// @brief restore the parent pointers after erasing from an object
/// ordered_json keeps its members in a vector, and erasing a member
/// re-constructs every member after it in place, which resets their
/// parent pointers
void set_parents_after_object_erase()
{
#if JSON_DIAGNOSTICS
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning(push )
#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
#endif
if (detail::is_ordered_map<object_t>::value)
{
set_parents();
}
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning( pop )
#endif
#endif
}
public: public:
////////////////////////// //////////////////////////
// JSON parser callback // // JSON parser callback //
@@ -2952,7 +2932,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
case value_t::object: case value_t::object:
{ {
result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator); result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator);
set_parents_after_object_erase();
break; break;
} }
@@ -3025,7 +3004,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator, result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator,
last.m_it.object_iterator); last.m_it.object_iterator);
set_parents_after_object_erase();
break; break;
} }
@@ -3056,9 +3034,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this)); JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
} }
const auto erased = m_data.m_value.object->erase(std::forward<KeyType>(key)); return m_data.m_value.object->erase(std::forward<KeyType>(key));
set_parents_after_object_erase();
return erased;
} }
template < typename KeyType, detail::enable_if_t < template < typename KeyType, detail::enable_if_t <
@@ -3075,7 +3051,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (it != m_data.m_value.object->end()) if (it != m_data.m_value.object->end())
{ {
m_data.m_value.object->erase(it); m_data.m_value.object->erase(it);
set_parents_after_object_erase();
return 1; return 1;
} }
return 0; return 0;
@@ -3987,12 +3962,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (it2 != m_data.m_value.object->end() && it2->second.is_object()) if (it2 != m_data.m_value.object->end() && it2->second.is_object())
{ {
it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1); it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1);
#if JSON_DIAGNOSTICS
it2->second.set_parents();
#endif
continue; continue;
} }
} }
// set_parent() also repairs the other members, which ordered_json m_data.m_value.object->operator[](it.key()) = it.value();
// relocates when adding a key makes its vector grow #if JSON_DIAGNOSTICS
set_parent(m_data.m_value.object->operator[](it.key()) = it.value()); m_data.m_value.object->operator[](it.key()).m_parent = this;
#endif
} }
} }
@@ -4021,6 +4000,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
} }
// a nested object is merged: continue with its parent // a nested object is merged: continue with its parent
#if JSON_DIAGNOSTICS
target->set_parents();
#endif
target = stack.back().target; target = stack.back().target;
first = stack.back().position; first = stack.back().position;
last = stack.back().last; last = stack.back().last;
@@ -4042,9 +4024,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
continue; continue;
} }
} }
// set_parent() also repairs the other members, which ordered_json target->m_data.m_value.object->operator[](first.key()) = first.value();
// relocates when adding a key makes its vector grow #if JSON_DIAGNOSTICS
target->set_parent(target->m_data.m_value.object->operator[](first.key()) = first.value()); target->m_data.m_value.object->operator[](first.key()).m_parent = target;
#endif
++first; ++first;
} }
} }
File diff suppressed because it is too large Load Diff
+37 -242
View File
@@ -95,10 +95,6 @@
#define JSON_BRACE_INIT_COPY_SEMANTICS 0 #define JSON_BRACE_INIT_COPY_SEMANTICS 0
#endif #endif
#ifndef JSON_PRECISE_STREAM_POSITION
#define JSON_PRECISE_STREAM_POSITION 0
#endif
#if JSON_DIAGNOSTICS #if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag #define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else #else
@@ -123,28 +119,21 @@
#define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS #define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS
#endif #endif
#if JSON_PRECISE_STREAM_POSITION
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION _psp
#else
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION #ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0 #define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif #endif
// Construct the namespace ABI tags component // Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e) json_abi ## a ## b ## c ## d ## e #define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d) json_abi ## a ## b ## c ## d
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e) \ #define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e) NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d)
#define NLOHMANN_JSON_ABI_TAGS \ #define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \ NLOHMANN_JSON_ABI_TAGS_CONCAT( \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \ NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \
NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \ NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \ NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \ NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS)
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION)
// Construct the namespace version component // Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \ #define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
@@ -7430,11 +7419,6 @@ class input_stream_adapter
// maintain ifstream flags, except eof // maintain ifstream flags, except eof
if (is != nullptr) if (is != nullptr)
{ {
#if JSON_PRECISE_STREAM_POSITION
// consume the character last returned by get_character() unless it
// was given back with release_lookahead()
commit_lookahead();
#endif
is->clear(is->rdstate() & std::ios::eofbit); is->clear(is->rdstate() & std::ios::eofbit);
} }
} }
@@ -7448,58 +7432,6 @@ class input_stream_adapter
input_stream_adapter& operator=(input_stream_adapter&) = delete; input_stream_adapter& operator=(input_stream_adapter&) = delete;
input_stream_adapter& operator=(input_stream_adapter&&) = delete; input_stream_adapter& operator=(input_stream_adapter&&) = delete;
#if JSON_PRECISE_STREAM_POSITION
input_stream_adapter(input_stream_adapter&& rhs) noexcept
: is(rhs.is), sb(rhs.sb), lookahead(rhs.lookahead)
{
rhs.is = nullptr;
rhs.sb = nullptr;
rhs.lookahead = false;
}
// Whether the character last returned by get_character() can be given back
// to the input with release_lookahead().
static constexpr bool supports_lookahead = true;
// std::istream/std::streambuf use std::char_traits<char>::to_int_type, to
// ensure that std::char_traits<char>::eof() and the character 0xFF do not
// end up as the same value, e.g., 0xFFFFFFFF.
//
// The character is peeked rather than consumed: it is only stepped over
// once the next character is requested, or when the adapter is destroyed.
// Until then, release_lookahead() can leave it in the input.
std::char_traits<char>::int_type get_character()
{
if (lookahead)
{
// step over the character returned by the previous call
sb->sbumpc();
}
auto res = sb->sgetc();
// set eof manually, as we don't use the istream interface.
if (JSON_HEDLEY_UNLIKELY(res == std::char_traits<char>::eof()))
{
// there is nothing to step over next time
lookahead = false;
is->clear(is->rdstate() | std::ios::eofbit);
}
else
{
lookahead = true;
}
return res;
}
// Leave the character last returned by get_character() in the input, so
// that the next read from the stream - by this adapter or by the caller
// once parsing is done - sees it again. Unlike putting a consumed
// character back, this cannot fail.
void release_lookahead() noexcept
{
lookahead = false;
}
#else
input_stream_adapter(input_stream_adapter&& rhs) noexcept input_stream_adapter(input_stream_adapter&& rhs) noexcept
: is(rhs.is), sb(rhs.sb) : is(rhs.is), sb(rhs.sb)
{ {
@@ -7510,9 +7442,6 @@ class input_stream_adapter
// std::istream/std::streambuf use std::char_traits<char>::to_int_type, to // std::istream/std::streambuf use std::char_traits<char>::to_int_type, to
// ensure that std::char_traits<char>::eof() and the character 0xFF do not // ensure that std::char_traits<char>::eof() and the character 0xFF do not
// end up as the same value, e.g., 0xFFFFFFFF. // end up as the same value, e.g., 0xFFFFFFFF.
//
// The character is consumed, so the character that terminates a number
// stays consumed after parsing; see JSON_PRECISE_STREAM_POSITION.
std::char_traits<char>::int_type get_character() std::char_traits<char>::int_type get_character()
{ {
auto res = sb->sbumpc(); auto res = sb->sbumpc();
@@ -7523,14 +7452,10 @@ class input_stream_adapter
} }
return res; return res;
} }
#endif
template<class T> template<class T>
std::size_t get_elements(T* dest, std::size_t count = 1) std::size_t get_elements(T* dest, std::size_t count = 1)
{ {
#if JSON_PRECISE_STREAM_POSITION
commit_lookahead();
#endif
auto res = static_cast<std::size_t>(sb->sgetn(reinterpret_cast<char*>(dest), static_cast<std::streamsize>(count * sizeof(T)))); auto res = static_cast<std::size_t>(sb->sgetn(reinterpret_cast<char*>(dest), static_cast<std::streamsize>(count * sizeof(T))));
if (JSON_HEDLEY_UNLIKELY(res < count * sizeof(T))) if (JSON_HEDLEY_UNLIKELY(res < count * sizeof(T)))
{ {
@@ -7540,27 +7465,9 @@ class input_stream_adapter
} }
private: private:
#if JSON_PRECISE_STREAM_POSITION
// Step over the character last returned by get_character(). The character
// has already been peeked successfully, so for every streambuf with a get
// area this is a pointer increment that cannot fail.
void commit_lookahead()
{
if (lookahead)
{
lookahead = false;
sb->sbumpc();
}
}
#endif
/// the associated input stream /// the associated input stream
std::istream* is = nullptr; std::istream* is = nullptr;
std::streambuf* sb = nullptr; std::streambuf* sb = nullptr;
#if JSON_PRECISE_STREAM_POSITION
/// whether get_character() peeked a character that is not consumed yet
bool lookahead = false;
#endif
}; };
#endif // JSON_NO_IO #endif // JSON_NO_IO
@@ -8972,25 +8879,6 @@ constexpr bool input_adapter_supports_seek(std::false_type /*detected*/)
return false; return false;
} }
// Detect whether an input adapter reads with one character of lookahead that
// can be left in the input (see input_stream_adapter::supports_lookahead,
// which is only defined with JSON_PRECISE_STREAM_POSITION), detected like
// supports_seek above.
template<typename InputAdapterType>
using detect_supports_lookahead = decltype(InputAdapterType::supports_lookahead);
template<typename InputAdapterType>
constexpr bool input_adapter_supports_lookahead(std::true_type /*detected*/)
{
return InputAdapterType::supports_lookahead;
}
template<typename InputAdapterType>
constexpr bool input_adapter_supports_lookahead(std::false_type /*detected*/)
{
return false;
}
// Detect whether an input adapter exposes a contiguous byte block that the // Detect whether an input adapter exposes a contiguous byte block that the
// lexer can scan directly (see iterator_input_adapter::supports_bulk_scan). // lexer can scan directly (see iterator_input_adapter::supports_bulk_scan).
// Adapters without the flag - file, stream, wide-string, user-defined - fall // Adapters without the flag - file, stream, wide-string, user-defined - fall
@@ -9031,12 +8919,6 @@ class lexer : public lexer_base<BasicJsonType>
static constexpr bool lazy_token_string = static constexpr bool lazy_token_string =
input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {}); input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {});
/// whether a simulated unget can be passed on to the input adapter, which
/// then leaves the character in the input; see
/// input_adapter_supports_lookahead
static constexpr bool can_release_lookahead =
input_adapter_supports_lookahead<InputAdapterType>(is_detected<detect_supports_lookahead, InputAdapterType> {});
/// whether string scanning may bulk-consume runs of ordinary characters /// whether string scanning may bulk-consume runs of ordinary characters
/// directly from a contiguous input buffer (SWAR fast path). This requires /// directly from a contiguous input buffer (SWAR fast path). This requires
/// the token to be reconstructible lazily (lazy_token_string), so bypassing /// the token to be reconstructible lazily (lazy_token_string), so bypassing
@@ -10768,21 +10650,6 @@ scan_number_done:
uncapture_char(std::integral_constant<bool, lazy_token_string> {}); uncapture_char(std::integral_constant<bool, lazy_token_string> {});
} }
/// adapter without lookahead: nothing to do (see release_lookahead)
void release_lookahead_impl(std::false_type /*can_release*/) const noexcept {}
/// adapter with lookahead: leave the character in the input instead
void release_lookahead_impl(std::true_type /*can_release*/)
{
if (next_unget)
{
// the character is read from the input again rather than replayed
// from current, so the adapter must not step over it
next_unget = false;
ia.release_lookahead();
}
}
/// seekable adapter: nothing was captured, so nothing to undo /// seekable adapter: nothing was captured, so nothing to undo
void uncapture_char(std::true_type /*lazy*/) const noexcept {} void uncapture_char(std::true_type /*lazy*/) const noexcept {}
@@ -10846,31 +10713,6 @@ scan_number_done:
return position; return position;
} }
/*!
@brief pass a pending simulated unget on to the input
unget() only rewinds the lexer's own bookkeeping, so the character that
terminated the last token (e.g. the character after a number) would still
be stepped over when the input adapter is done. Callers that hand the
input back to the user afterwards - operator>> and non-strict sax_parse -
call this once when scanning is done, so that the input is positioned
right after the value.
Adapters without lookahead (see input_adapter_supports_lookahead) are not
handed back to the user, so this is a no-op for them. Without
JSON_PRECISE_STREAM_POSITION, no adapter has lookahead, so this is always a
no-op and the terminating character stays consumed.
Scanning may continue after this call: @a next_unget is cleared, and the
character is read from the input again instead of being replayed from
@a current. A pending unget of EOF needs no special case, because reaching
EOF leaves no lookahead to release.
*/
void release_lookahead()
{
release_lookahead_impl(std::integral_constant<bool, can_release_lookahead> {});
}
#if JSON_DIAGNOSTIC_POSITIONS #if JSON_DIAGNOSTIC_POSITIONS
/// return the offset of the first character of the last read token; unlike /// return the offset of the first character of the last read token; unlike
/// the token's parsed value, this accounts for escape sequences /// the token's parsed value, this accounts for escape sequences
@@ -16118,22 +15960,13 @@ class parser
json_sax_dom_callback_parser<BasicJsonType, InputAdapterType> sdp(result, callback, allow_exceptions, &m_lexer); json_sax_dom_callback_parser<BasicJsonType, InputAdapterType> sdp(result, callback, allow_exceptions, &m_lexer);
sax_parse_internal(&sdp); sax_parse_internal(&sdp);
if (strict) // in strict mode, input must be completely read
if (strict && (get_token() != token_type::end_of_input))
{ {
// in strict mode, input must be completely read sdp.parse_error(m_lexer.get_position(),
if (get_token() != token_type::end_of_input) m_lexer.get_token_string(),
{ parse_error::create(101, m_lexer.get_position(),
sdp.parse_error(m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(),
exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
} }
// in case of an error, return a discarded value // in case of an error, return a discarded value
@@ -16155,20 +15988,12 @@ class parser
json_sax_dom_parser<BasicJsonType, InputAdapterType> sdp(result, allow_exceptions, &m_lexer); json_sax_dom_parser<BasicJsonType, InputAdapterType> sdp(result, allow_exceptions, &m_lexer);
sax_parse_internal(&sdp); sax_parse_internal(&sdp);
if (strict) // in strict mode, input must be completely read
if (strict && (get_token() != token_type::end_of_input))
{ {
// in strict mode, input must be completely read sdp.parse_error(m_lexer.get_position(),
if (get_token() != token_type::end_of_input) m_lexer.get_token_string(),
{ parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// see above
m_lexer.release_lookahead();
} }
// in case of an error, return a discarded value // in case of an error, return a discarded value
@@ -16201,24 +16026,12 @@ class parser
(void)detail::is_sax_static_asserts<SAX, BasicJsonType> {}; (void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
const bool result = sax_parse_internal(sax); const bool result = sax_parse_internal(sax);
if (result) // strict mode: next byte must be EOF
if (result && strict && (get_token() != token_type::end_of_input))
{ {
if (strict) return sax->parse_error(m_lexer.get_position(),
{ m_lexer.get_token_string(),
// strict mode: next byte must be EOF parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
if (get_token() != token_type::end_of_input)
{
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
} }
return result; return result;
@@ -25623,7 +25436,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>; using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>;
/// scratch space to build the key skeleton of an object copy in one go /// scratch space to build the key skeleton of an object copy in one go
using copy_scratch_t = std::vector<std::pair<typename object_t::key_type, basic_json>>; using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
/// @brief copy everything of @a src into @a dst but its type and value /// @brief copy everything of @a src into @a dst but its type and value
static void copy_metadata(const basic_json& src, basic_json& dst) static void copy_metadata(const basic_json& src, basic_json& dst)
@@ -25888,27 +25702,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
} }
/// @brief restore the parent pointers after erasing from an object
/// ordered_json keeps its members in a vector, and erasing a member
/// re-constructs every member after it in place, which resets their
/// parent pointers
void set_parents_after_object_erase()
{
#if JSON_DIAGNOSTICS
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning(push )
#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
#endif
if (detail::is_ordered_map<object_t>::value)
{
set_parents();
}
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning( pop )
#endif
#endif
}
public: public:
////////////////////////// //////////////////////////
// JSON parser callback // // JSON parser callback //
@@ -27597,7 +27390,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
case value_t::object: case value_t::object:
{ {
result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator); result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator);
set_parents_after_object_erase();
break; break;
} }
@@ -27670,7 +27462,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator, result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator,
last.m_it.object_iterator); last.m_it.object_iterator);
set_parents_after_object_erase();
break; break;
} }
@@ -27701,9 +27492,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this)); JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
} }
const auto erased = m_data.m_value.object->erase(std::forward<KeyType>(key)); return m_data.m_value.object->erase(std::forward<KeyType>(key));
set_parents_after_object_erase();
return erased;
} }
template < typename KeyType, detail::enable_if_t < template < typename KeyType, detail::enable_if_t <
@@ -27720,7 +27509,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (it != m_data.m_value.object->end()) if (it != m_data.m_value.object->end())
{ {
m_data.m_value.object->erase(it); m_data.m_value.object->erase(it);
set_parents_after_object_erase();
return 1; return 1;
} }
return 0; return 0;
@@ -28632,12 +28420,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (it2 != m_data.m_value.object->end() && it2->second.is_object()) if (it2 != m_data.m_value.object->end() && it2->second.is_object())
{ {
it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1); it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1);
#if JSON_DIAGNOSTICS
it2->second.set_parents();
#endif
continue; continue;
} }
} }
// set_parent() also repairs the other members, which ordered_json m_data.m_value.object->operator[](it.key()) = it.value();
// relocates when adding a key makes its vector grow #if JSON_DIAGNOSTICS
set_parent(m_data.m_value.object->operator[](it.key()) = it.value()); m_data.m_value.object->operator[](it.key()).m_parent = this;
#endif
} }
} }
@@ -28666,6 +28458,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
} }
// a nested object is merged: continue with its parent // a nested object is merged: continue with its parent
#if JSON_DIAGNOSTICS
target->set_parents();
#endif
target = stack.back().target; target = stack.back().target;
first = stack.back().position; first = stack.back().position;
last = stack.back().last; last = stack.back().last;
@@ -28687,9 +28482,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
continue; continue;
} }
} }
// set_parent() also repairs the other members, which ordered_json target->m_data.m_value.object->operator[](first.key()) = first.value();
// relocates when adding a key makes its vector grow #if JSON_DIAGNOSTICS
target->set_parent(target->m_data.m_value.object->operator[](first.key()) = first.value()); target->m_data.m_value.object->operator[](first.key()).m_parent = target;
#endif
++first; ++first;
} }
} }
@@ -30952,7 +30748,6 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
#undef JSON_HAS_STATIC_RTTI #undef JSON_HAS_STATIC_RTTI
#undef JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON #undef JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
#undef JSON_BRACE_INIT_COPY_SEMANTICS #undef JSON_BRACE_INIT_COPY_SEMANTICS
#undef JSON_PRECISE_STREAM_POSITION
#endif #endif
// #include <nlohmann/thirdparty/hedley/hedley_undef.hpp> // #include <nlohmann/thirdparty/hedley/hedley_undef.hpp>
+4 -15
View File
@@ -56,10 +56,6 @@
#define JSON_BRACE_INIT_COPY_SEMANTICS 0 #define JSON_BRACE_INIT_COPY_SEMANTICS 0
#endif #endif
#ifndef JSON_PRECISE_STREAM_POSITION
#define JSON_PRECISE_STREAM_POSITION 0
#endif
#if JSON_DIAGNOSTICS #if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag #define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else #else
@@ -84,28 +80,21 @@
#define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS #define NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS
#endif #endif
#if JSON_PRECISE_STREAM_POSITION
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION _psp
#else
#define NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION #ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0 #define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif #endif
// Construct the namespace ABI tags component // Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e) json_abi ## a ## b ## c ## d ## e #define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d) json_abi ## a ## b ## c ## d
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e) \ #define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e) NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d)
#define NLOHMANN_JSON_ABI_TAGS \ #define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \ NLOHMANN_JSON_ABI_TAGS_CONCAT( \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \ NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \
NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \ NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \ NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \ NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS)
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION)
// Construct the namespace version component // Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \ #define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
-4
View File
@@ -36,10 +36,6 @@ TEST_CASE("default namespace")
expected += "_bics"; expected += "_bics";
#endif #endif
#if JSON_PRECISE_STREAM_POSITION
expected += "_psp";
#endif
expected += "_v" STRINGIZE(NLOHMANN_JSON_VERSION_MAJOR); expected += "_v" STRINGIZE(NLOHMANN_JSON_VERSION_MAJOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_MINOR); expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_MINOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_PATCH) "::basic_json"; expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_PATCH) "::basic_json";
-4
View File
@@ -37,10 +37,6 @@ TEST_CASE("default namespace without version component")
expected += "_bics"; expected += "_bics";
#endif #endif
#if JSON_PRECISE_STREAM_POSITION
expected += "_psp";
#endif
expected += "::basic_json"; expected += "::basic_json";
// fallback for Clang // fallback for Clang
-23
View File
@@ -79,26 +79,3 @@ the same `fuzzers` target as above and also relies on the `FUZZER_ENGINE` variab
[build script](https://github.com/google/oss-fuzz/blob/master/projects/json/build.sh) for more information. [build script](https://github.com/google/oss-fuzz/blob/master/projects/json/build.sh) for more information.
In case the build at OSS-Fuzz fails, an issue will be created automatically. In case the build at OSS-Fuzz fails, an issue will be created automatically.
### Handling OSS-Fuzz reports
OSS-Fuzz files the crashes it finds in its own [issue tracker](https://issues.oss-fuzz.com), not on GitHub. So that
each report can be traced to the change that fixed it, and each fix to the report it answers, fixes follow these
conventions:
- **Reference the OSS-Fuzz issue in the pull request**, next to any GitHub issue it closes, as `OSS-Fuzz: <id>` (for
example, `OSS-Fuzz: 563659413`), and in the commit message. The ID alone does not disclose the crash. If the report
was triaged into a GitHub issue, link the OSS-Fuzz issue there too.
- **Turn the reproducer into a unit test.** Download the testcase from the OSS-Fuzz report, reduce it if possible, and
add it as a regression test to the unit test of the affected format (e.g., `tests/src/unit-bjdata.cpp`), with a
comment naming the OSS-Fuzz issue. This way the input is checked by every CI run rather than only by OSS-Fuzz, and
it stays covered even if OSS-Fuzz later closes the report as not reproducible.
- **Keep the fuzzer drivers and the unit tests in sync.** The round-trip checks of the UBJSON and BJData drivers are
also run on a fixed corpus in the unit tests (see `tests/src/round_trip_corpus.hpp` and the "round-trip invariants"
test cases), so a regression shows up in CI first. When a driver's checks change, change the unit tests with them.
- **Record in the report whether the bug shipped.** OSS-Fuzz asks whether a crash was a short-lived regression or
affects a released version; answer it when the fix is merged, as it decides whether the fix needs a release note or
a security advisory (see the [security policy](../.github/SECURITY.md)).
After the fix is merged, OSS-Fuzz re-runs the reproducer on its next build and marks the report as verified and
closed. If it does not, the fix is incomplete.
-3
View File
@@ -42,9 +42,6 @@ dump() serializes any non-finite double the same deterministic way (as JSON
`null`, since JSON itself cannot represent NaN/Infinity), so comparing `null`, since JSON itself cannot represent NaN/Infinity), so comparing
dumps is stable under exactly the same values that break operator==. dumps is stable under exactly the same values that break operator==.
The unit tests run the same checks on a fixed corpus (see the "BJData round-trip
invariants" test case), so keep both in sync.
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers. drivers.
*/ */
-3
View File
@@ -21,9 +21,6 @@ array data, it performs the following steps:
- j4 = from_ubjson(vec3) - j4 = from_ubjson(vec3)
- assert(j1 == j4) - assert(j1 == j4)
The unit tests run the same checks on a fixed corpus (see the "UBJSON round-trip
invariants" test case), so keep both in sync.
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers. drivers.
*/ */
-213
View File
@@ -1,213 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cmath> // nan
#include <cstddef> // size_t
#include <cstdint> // int32_t, int64_t, uint32_t, uint64_t
#include <limits> // numeric_limits
#include <random> // mt19937
#include <string> // string, to_string
#include <utility> // move
#include <vector> // vector
#include <nlohmann/json.hpp>
// Values for the round-trip property tests of the UBJSON and BJData writers.
//
// The fuzzer drivers (tests/src/fuzzer-parse_ubjson.cpp and
// fuzzer-parse_bjdata.cpp) check that anything the library parses can be
// serialized, parsed back, and serialized again without loss. Those checks
// only run at OSS-Fuzz, so a regression used to surface days later as an
// external report. The unit tests run the same checks on this corpus in CI.
//
// The corpus is deterministic: std::mt19937's output sequence is fixed by
// the standard, and it is used directly rather than through a distribution
// (whose results are implementation-defined).
namespace utils
{
class round_trip_corpus
{
public:
using json = nlohmann::json;
static std::vector<json> values()
{
round_trip_corpus corpus;
return corpus.build();
}
// whether a value contains a binary value, which a BJData or UBJSON round
// trip may turn into an array of integers
static bool contains_binary(const json& j)
{
if (j.is_binary())
{
return true;
}
if (j.is_structured())
{
for (const auto& element : j)
{
if (contains_binary(element))
{
return true;
}
}
}
return false;
}
private:
std::vector<json> atoms;
// a fixed seed is the point: the corpus must be the same in every run
std::mt19937 generator{42}; // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
round_trip_corpus()
: atoms
{
nullptr, true, false,
// integers at the boundaries of every UBJSON/BJData integer type
0, 1, -1, 127, 128, 255, 256, -128, -129,
32767, 32768, 65535, 65536, -32768, -32769,
(std::numeric_limits<std::int32_t>::min)(), (std::numeric_limits<std::int32_t>::max)(),
(std::numeric_limits<std::uint32_t>::max)(),
(std::numeric_limits<std::int64_t>::min)(), (std::numeric_limits<std::int64_t>::max)(),
static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()) + 1u,
(std::numeric_limits<std::uint64_t>::max)(),
// floating-point numbers, including non-finite ones
0.0, -0.0, 1.5, -2.25, 3.4e38, (std::numeric_limits<double>::max)(),
std::nan(""), std::numeric_limits<double>::infinity(), -std::numeric_limits<double>::infinity(),
// strings, including a non-ASCII one and one longer than 255 bytes
"", "a", "\xC3\xA4", std::string(300, 'x'),
// binary values with and without subtype
json::binary({}), json::binary({1, 2, 255}), json::binary({0x80, 0x7F}, 42), json::binary({1}, 0)
}
{}
std::vector<json> build()
{
std::vector<json> result = atoms;
// each atom inside containers, including homogeneous ones that the
// writers encode as optimized (typed) containers
result.emplace_back(json::array());
result.emplace_back(json::object());
for (const auto& atom : atoms)
{
result.push_back(json::array({atom}));
result.push_back(json::array({atom, atom, atom}));
result.push_back(json::array({json::array({atom})}));
result.push_back(json::object({{"key", atom}}));
}
result.push_back(json::array({1, 1.5}));
result.push_back(json::array({-1, 255}));
result.push_back(json::array({"a", "b"}));
// deep, but well below any recursion or depth limit
json nested_array = 1;
json nested_object = 1;
for (int i = 0; i < 300; ++i)
{
nested_array = json::array({nested_array});
nested_object = json::object({{"key", nested_object}});
}
result.push_back(nested_array);
result.push_back(nested_object);
add_annotated_arrays(result);
add_random_values(result);
return result;
}
// objects in the JData annotated array format, which the BJData writer
// encodes as ND-arrays when the annotation describes a packed array, and
// as plain objects otherwise (see #5398, #5399, #5403, #5404, and #5542)
static void add_annotated_arrays(std::vector<json>& result)
{
const std::vector<json> types =
{
"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64",
"single", "double", "char", "byte", "bool", "unknown", 5, nullptr
};
const std::vector<json> sizes =
{
json::array(), {3}, {1, 3}, {3, 1}, {2, 3}, {2, 0}, {0, 2}, {2, 2, 2}, {-1, 2}, {2, 1.5},
"3", 3, nullptr, json::binary({})
};
const std::vector<json> data =
{
nullptr, 5, "s", json::object({{"a", 1}}), json::array(),
{1, 2, 3}, {1, 2, 3, 4, 5, 6}, {1, 2, 3, 4, 5, 6, 7, 8},
{1.5, 2.5, 3.5, 4.5, 5.5, 6.5}, {300, -300, 70000, -70000, 1, 2},
{"a", "b", "c", "d", "e", "f"}, {json::array({1, 2, 3}), json::array({4, 5, 6})}
};
for (const auto& type : types)
{
for (const auto& size : sizes)
{
for (const auto& d : data)
{
result.push_back({{"_ArrayType_", type}, {"_ArraySize_", size}, {"_ArrayData_", d}});
}
}
}
// incomplete annotations and annotations with an extra key
result.push_back({{"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
result.push_back({{"_ArrayType_", "uint8"}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
result.push_back({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}});
result.push_back({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"extra", 1}});
}
// random containers of atoms, both homogeneous and mixed
void add_random_values(std::vector<json>& result)
{
for (int i = 0; i < 1000; ++i)
{
result.push_back(random_value(0));
}
}
std::size_t random_below(std::size_t bound)
{
return generator() % bound;
}
json random_value(int depth)
{
const auto kind = random_below(10);
if (depth > 3 || kind < 5)
{
return atoms[random_below(atoms.size())];
}
json result = kind < 8 ? json::array() : json::object();
const auto count = random_below(5);
const bool homogeneous = random_below(2) == 0;
const json fixed = atoms[random_below(atoms.size())];
for (std::size_t i = 0; i < count; ++i)
{
json element = homogeneous ? fixed : random_value(depth + 1);
if (result.is_array())
{
result.push_back(std::move(element));
}
else
{
result[std::to_string(i)] = std::move(element);
}
}
return result;
}
};
} // namespace utils
+76
View File
@@ -270,6 +270,82 @@ TEST_CASE("controlled bad_alloc")
} }
} }
namespace
{
// counts the allocations of pairs with a non-const first member: the object
// types store std::pair<const Key, T>, so only the scratch space of the
// iterative deep copy allocates std::pair<Key, T>
std::size_t scratch_pair_allocations = 0;
template<class T>
struct is_scratch_pair : std::false_type {};
template<class K, class V>
struct is_scratch_pair<std::pair<K, V>> : std::integral_constant < bool, !std::is_const<K>::value > {};
template<class T>
struct scratch_counting_allocator : std::allocator<T>
{
using std::allocator<T>::allocator;
T* allocate(std::size_t n)
{
if (is_scratch_pair<T>::value)
{
++scratch_pair_allocations;
}
return std::allocator<T>::allocate(n);
}
#ifdef __cpp_lib_allocate_at_least
// std::allocator<T>::allocate_at_least would bypass the counting, and
// libc++'s containers prefer it over allocate from C++23 on
auto allocate_at_least(std::size_t n)
{
if (is_scratch_pair<T>::value)
{
++scratch_pair_allocations;
}
return std::allocator<T>::allocate_at_least(n);
}
#endif
template <class U>
struct rebind
{
using other = scratch_counting_allocator<U>;
};
};
} // namespace
TEST_CASE("deep copy uses the provided allocator")
{
using counting_json = nlohmann::basic_json<std::map,
std::vector,
std::string,
bool,
std::int64_t,
std::uint64_t,
double,
scratch_counting_allocator>;
// deeper than the 128 levels the copy constructor descends into, so the
// innermost objects are copied by the iterative deep copy
counting_json j = 1;
for (std::size_t i = 0; i < 300; ++i)
{
counting_json wrapper = counting_json::object();
wrapper["a"] = std::move(j);
j = std::move(wrapper);
}
scratch_pair_allocations = 0;
// NOLINTNEXTLINE(performance-unnecessary-copy-initialization): the copy is what is tested
const counting_json copy(j);
CHECK(scratch_pair_allocations > 0);
CHECK(copy == j);
}
namespace namespace
{ {
template<class T> template<class T>
-103
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <fstream> #include <fstream>
#include <set> #include <set>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "round_trip_corpus.hpp"
#include "test_utils.hpp" #include "test_utils.hpp"
namespace namespace
@@ -2868,21 +2867,6 @@ TEST_CASE("BJData")
const auto out_num = json::to_bjdata(j_num); const auto out_num = json::to_bjdata(j_num);
CHECK(out_num.at(0) == '{'); CHECK(out_num.at(0) == '{');
CHECK(json::from_bjdata(out_num) == j_num); CHECK(json::from_bjdata(out_num) == j_num);
// OSS-Fuzz issue 474400817: an empty object _ArraySize_ was
// written as the ND-array header length, which from_bjdata()
// could not read back
const std::vector<uint8_t> input =
{
'[', '{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '{', '}', '}', ']'
};
const json j1 = json::from_bjdata(input);
CHECK(j1 == json::parse(R"([{"_ArrayType_":"int16","_ArraySize_":{},"_ArrayData_":null}])"));
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(json::to_bjdata(j1, false, false)));
CHECK(j2 == j1);
} }
SECTION("ndarray with out-of-range _ArrayData_ elements stays as object") SECTION("ndarray with out-of-range _ArrayData_ elements stays as object")
@@ -4289,93 +4273,6 @@ TEST_CASE("BJData use_type requires use_size")
} }
} }
TEST_CASE("BJData round-trip invariants")
{
// This checks what the parse_bjdata_fuzzer driver checks (see
// tests/src/fuzzer-parse_bjdata.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_bjdata() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// yields a value-equal result.
//
// Beyond the driver, this also checks that j2 equals j1 and that
// serializing j2 reproduces the exact bytes, both except for values that
// contain a binary value: a binary value is only written as a binary
// value with Draft 3's optimized binary array, and otherwise read back as
// an array of integers, for which the writer may choose different (but
// equally valid) type markers when it is serialized again (see #5494).
//
// Values are compared with dump() rather than operator==, because a NaN
// never compares equal to itself.
struct options
{
bool use_size;
bool use_type;
json::bjdata_version_t version;
};
const std::vector<options> all_options =
{
{false, false, json::bjdata_version_t::draft2},
{true, false, json::bjdata_version_t::draft2},
{true, true, json::bjdata_version_t::draft2},
{false, false, json::bjdata_version_t::draft3},
{true, false, json::bjdata_version_t::draft3},
{true, true, json::bjdata_version_t::draft3},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_bjdata() returns it
for (const auto& initial : all_options)
{
const json j1 = json::from_bjdata(json::to_bjdata(j0, initial.use_size, initial.use_type, initial.version));
const bool has_binary = utils::round_trip_corpus::contains_binary(j1);
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type
<< ", draft3 = " << (o.version == json::bjdata_version_t::draft3));
const std::vector<std::uint8_t> vec = json::to_bjdata(j1, o.use_size, o.use_type, o.version);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_bjdata(vec));
const std::vector<std::uint8_t> vec2 = json::to_bjdata(j2, o.use_size, o.use_type, o.version);
CHECK(json::from_bjdata(vec2).dump() == j2.dump());
if (!has_binary)
{
CHECK(j2.dump() == j1.dump());
CHECK(vec2 == vec);
}
}
}
}
}
TEST_CASE("BJData round trip of a binary value is value-stable, not byte-stable")
{
// OSS-Fuzz issue 474480402: a Draft 3 optimized binary array is read as a
// binary value, which to_bjdata() writes in the default Draft 2 mode as a
// plain array of uint8 numbers. That is read back as an array of numbers,
// for which the writer then picks the smallest type marker, int8 ('i'),
// so re-serializing changes the bytes, but not the value. This is the
// exception described in the "Round trips" note of the BJData
// documentation, and why the fuzzer checks value stability (see #5494).
const std::vector<uint8_t> input = {'[', '$', 'B', '#', 'U', 1, 0x20};
const json j1 = json::from_bjdata(input);
CHECK(j1 == json::binary({0x20}));
const std::vector<uint8_t> vec = json::to_bjdata(j1, false, false);
CHECK(vec == std::vector<uint8_t>({'[', 'U', 0x20, ']'}));
const json j2 = json::from_bjdata(vec);
CHECK(j2 == json::array({0x20}));
const std::vector<uint8_t> vec2 = json::to_bjdata(j2, false, false);
CHECK(vec2 == std::vector<uint8_t>({'[', 'i', 0x20, ']'}));
CHECK(json::from_bjdata(vec2) == j2);
}
TEST_CASE("BJData roundtrips" * doctest::skip()) TEST_CASE("BJData roundtrips" * doctest::skip())
{ {
SECTION("input from self-generated BJData files") SECTION("input from self-generated BJData files")
-52
View File
@@ -25,7 +25,6 @@ using nlohmann::json;
#include <iostream> #include <iostream>
#include <iterator> #include <iterator>
#include <sstream> #include <sstream>
#include <string>
#include <valarray> #include <valarray>
#if defined(_WIN32) #if defined(_WIN32)
@@ -1234,57 +1233,6 @@ TEST_CASE("deserialization")
} }
} }
SECTION("stream position after extraction without JSON_PRECISE_STREAM_POSITION (#5340)")
{
// By default, the character that terminates a number is consumed, so
// the stream is left one byte too far after a number (and only after a
// number). JSON_PRECISE_STREAM_POSITION changes this; see
// unit-precise-stream-position.cpp. These checks pin the default.
const auto remaining = [](std::istream & is)
{
return std::string(std::istreambuf_iterator<char>(is), std::istreambuf_iterator<char>());
};
SECTION("the character after a number is consumed")
{
std::istringstream ss("1true");
json j;
ss >> j;
CHECK(j == 1);
CHECK(remaining(ss) == "rue");
}
SECTION("the character after other values is not consumed")
{
std::istringstream ss("[1]true");
json j;
ss >> j;
CHECK(j == json::parse("[1]"));
CHECK(remaining(ss) == "true");
}
SECTION("comma-separated numbers can be read one by one")
{
std::istringstream ss("1,2,3");
json j1, j2, j3;
ss >> j1 >> j2 >> j3;
CHECK(j1 == 1);
CHECK(j2 == 2);
CHECK(j3 == 3);
}
SECTION("std::getline after a number skips the line break")
{
std::istringstream ss("42\nfoo");
json j;
std::string line;
ss >> j;
std::getline(ss, line);
CHECK(j == 42);
CHECK(line == "foo");
}
}
// build with C++20 // build with C++20
// JSON_HAS_CPP_20 // JSON_HAS_CPP_20
#if defined(__cpp_char8_t) #if defined(__cpp_char8_t)
-100
View File
@@ -361,106 +361,6 @@ TEST_CASE("Regression tests for extended diagnostics")
CHECK(p == o); CHECK(p == o);
} }
} }
SECTION("Regression test - erase() and update() must keep JSON_DIAGNOSTICS parent pointers of ordered_json members")
{
// ordered_json keeps its members in a vector: erasing a member
// re-constructs all members after it in place, and adding a key may
// reallocate the vector; both reset the parent pointers of the members
// that were moved
using nlohmann::ordered_json;
const auto check_parents = [](const ordered_json & j)
{
// const access, so operator[] cannot repair the parent pointers
CHECK_THROWS_WITH_AS(j["z"]["x"].at(0), "[json.exception.type_error.304] (/z/x) cannot use at() with number", ordered_json::type_error);
// must not trigger assert_invariant() in a debug/assert-enabled build
ordered_json const copy = j; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(copy == j);
};
// erase(key)
{
ordered_json j = {{"a", 1}, {"z", {{"x", 1}}}};
CHECK(j.erase("a") == 1);
check_parents(j);
}
// erase(iterator)
{
ordered_json j = {{"a", 1}, {"z", {{"x", 1}}}};
j.erase(j.begin());
check_parents(j);
}
// erase(iterator, iterator)
{
ordered_json j = {{"a", 1}, {"b", 2}, {"z", {{"x", 1}}}};
j.erase(j.begin(), j.find("z"));
check_parents(j);
}
// patch() removes via erase(iterator)
{
ordered_json j = {{"a", 1}, {"z", {{"x", 1}}}};
j.patch_inplace(ordered_json::parse(R"([{"op": "remove", "path": "/a"}])"));
check_parents(j);
}
// update(j)
{
ordered_json j = {{"z", {{"x", 1}}}};
j.update({{"a", 1}, {"b", 2}});
check_parents(j);
}
// update(j, true), the outer and the nested vector both grow
{
ordered_json j = {{"z", {{"x", 1}}}};
j.update({{"z", {{"y", 2}}}, {"a", 1}}, true);
check_parents(j);
}
// update(j, true) around its descent bound, where the nested vectors
// grow while the objects are merged without recursing
for (const std::size_t depth :
{
nlohmann::detail::recursion_depth_limit() - 1, nlohmann::detail::recursion_depth_limit(), nlohmann::detail::recursion_depth_limit() + 2
})
{
ordered_json j = {{"z", {{"x", 1}}}};
ordered_json patch = {{"a", 1}, {"b", 2}, {"c", {{"d", 3}}}};
for (std::size_t i = 0; i < depth; ++i)
{
j = ordered_json{{"k", 0}, {"n", std::move(j)}};
patch = ordered_json{{"n", std::move(patch)}, {"l", 1}, {"m", 2}};
}
j.update(patch, true);
// must not trigger assert_invariant() on any level in a
// debug/assert-enabled build
ordered_json const copy = j; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(copy == j);
}
// merge_patch() inserts "c" and removes "d" at /a/c, then inserts "e"
// at /a, which copies /a/c
{
auto j = ordered_json::parse(R"({"a": {"c": {"d": {}}}})");
j.merge_patch(ordered_json::parse(R"({"a": {"c": {"c": "s", "d": null}, "e": "s"}})"));
CHECK(j.dump() == R"({"a":{"c":{"c":"s"},"e":"s"}})");
auto const& constJ = j;
#if JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(constJ["a"]["c"]["c"].at(0), "[json.exception.type_error.304] (/a/c/c) (bytes 18-21) cannot use at() with string", ordered_json::type_error);
#else
CHECK_THROWS_WITH_AS(constJ["a"]["c"]["c"].at(0), "[json.exception.type_error.304] (/a/c/c) cannot use at() with string", ordered_json::type_error);
#endif
ordered_json const copy = j;
CHECK(copy == j);
}
}
} }
TEST_CASE("Better diagnostics past the descent bound of update() and merge_patch()") TEST_CASE("Better diagnostics past the descent bound of update() and merge_patch()")
-237
View File
@@ -1,237 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
// This file tests the opt-in JSON_PRECISE_STREAM_POSITION, so it defines the
// macro itself rather than relying on a -D flag, and runs in every build. The
// default behavior is pinned in unit-deserialization.cpp.
#ifdef JSON_PRECISE_STREAM_POSITION
#undef JSON_PRECISE_STREAM_POSITION
#endif
#define JSON_PRECISE_STREAM_POSITION 1
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cstddef>
#include <sstream>
#include <streambuf>
#include <string>
#include <utility>
#include <vector>
#define STRINGIZE_EX(x) #x
#define STRINGIZE(x) STRINGIZE_EX(x)
namespace
{
// A streambuf that keeps no get area at all and refuses every putback: with an
// empty get area, sungetc() always ends up in pbackfail(). Used to check that
// the character terminating a number is left in the input without relying on
// the streambuf being able to put a consumed character back.
class no_putback_streambuf : public std::streambuf
{
public:
explicit no_putback_streambuf(std::string s) : m_data(std::move(s)) {}
protected:
// peek at the next character without consuming it
int_type underflow() override
{
if (m_pos >= m_data.size())
{
return traits_type::eof();
}
return traits_type::to_int_type(m_data[m_pos]);
}
// consume the next character
int_type uflow() override
{
if (m_pos >= m_data.size())
{
return traits_type::eof();
}
return traits_type::to_int_type(m_data[m_pos++]);
}
int_type pbackfail(int_type /*c*/) override
{
return traits_type::eof();
}
private:
std::string m_data;
std::size_t m_pos = 0;
};
// read the characters that are left in a stream
std::string remaining(std::istream& is)
{
std::string result;
char c = 0;
while (is.get(c))
{
result += c;
}
return result;
}
} // namespace
TEST_CASE("JSON_PRECISE_STREAM_POSITION")
{
SECTION("the macro is part of the ABI tag")
{
const std::string ns = STRINGIZE(NLOHMANN_JSON_NAMESPACE);
// other tags may come before it, e.g. json_abi_diag_psp
CHECK(ns.find("_psp") != std::string::npos);
}
SECTION("a number does not consume the character that terminates it")
{
// a number is only terminated by the character following it; that
// character must be given back so the stream is positioned right
// after the value
const std::vector<std::pair<std::string, std::string>> tests =
{
{"1true", "true"},
{"1[2]", "[2]"},
{"1{}", "{}"},
{R"(1"a")", R"("a")"},
{"1 true", " true"},
{"12,", ","},
{"-0.5e3x", "x"},
{"1null", "null"}
};
for (const auto& test : tests)
{
CAPTURE(test.first);
std::istringstream ss(test.first);
json j;
ss >> j;
CHECK(j == json::parse(test.first.substr(0, test.first.size() - test.second.size())));
CHECK(remaining(ss) == test.second);
}
}
SECTION("values that are self-delimiting are unaffected")
{
const std::vector<std::pair<std::string, std::string>> tests =
{
{"truefalse", "false"},
{"[1][2]", "[2]"},
{R"({"a":1}{"b":2})", R"({"b":2})"},
{R"("a""b")", R"("b")"},
{"null null", " null"}
};
for (const auto& test : tests)
{
CAPTURE(test.first);
std::istringstream ss(test.first);
json j;
ss >> j;
CHECK(remaining(ss) == test.second);
}
}
SECTION("a number at the end of the input leaves nothing behind")
{
for (const std::string s :
{"1", "12", "-3.5e2", " 7 "
})
{
CAPTURE(s);
std::istringstream ss(s);
json j;
ss >> j;
CHECK(remaining(ss).find_first_not_of(" \t\n\r") == std::string::npos);
}
}
SECTION("repeated extraction of concatenated values")
{
std::istringstream ss(R"(1true[2]3"x"{"a":4}5)");
const std::vector<json> expected =
{
json(1), json(true), json::parse("[2]"), json(3),
json("x"), json::parse(R"({"a":4})"), json(5)
};
for (const auto& e : expected)
{
json j;
ss >> j;
CHECK(j == e);
}
}
SECTION("differences to the default behavior")
{
// both of these work by accident without the macro, because the
// character after a number is swallowed; see unit-deserialization.cpp
SECTION("a separator after a number is not skipped")
{
std::istringstream ss("1,2");
json j;
ss >> j;
CHECK(j == 1);
CHECK_THROWS_AS(ss >> j, json::parse_error&);
}
SECTION("std::getline after a number sees the line break")
{
std::istringstream ss("42\nfoo");
json j;
std::string line;
ss >> j;
std::getline(ss, line);
CHECK(j == 42);
CHECK(line.empty());
std::getline(ss, line);
CHECK(line == "foo");
}
}
SECTION("sax_parse with strict == false")
{
std::istringstream ss("1true");
json j;
nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::input_stream_adapter> sdp(j, true);
CHECK(json::sax_parse(ss, &sdp, nlohmann::detail::input_format_t::json, false));
CHECK(j == 1);
CHECK(remaining(ss) == "true");
}
SECTION("strict parsing still rejects trailing data")
{
std::istringstream ss("1true");
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(ss),
"[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - unexpected true literal; expected end of input", json::parse_error&);
std::istringstream ss2("1true");
CHECK_FALSE(json::accept(ss2));
}
SECTION("a streambuf that cannot put back is not needed")
{
// the terminating character is never consumed, so no putback
// position is required
no_putback_streambuf buf("1true");
std::istream is(&buf);
json j;
is >> j;
CHECK(j == json(1));
CHECK(remaining(is) == "true");
}
}
+1 -49
View File
@@ -15,7 +15,6 @@ using nlohmann::json;
#include <fstream> #include <fstream>
#include <set> #include <set>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "round_trip_corpus.hpp"
#include "test_utils.hpp" #include "test_utils.hpp"
namespace namespace
@@ -2266,9 +2265,7 @@ TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
SECTION("an excessive count is rejected") SECTION("an excessive count is rejected")
{ {
// 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value; // 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value
// OSS-Fuzz reported this shape as a parse_ubjson_fuzzer timeout
// (testcase 6347769435193344, no issue filed)
for (const auto marker : for (const auto marker :
{'Z', 'T', 'F' {'Z', 'T', 'F'
}) })
@@ -2820,51 +2817,6 @@ TEST_CASE("UBJSON use_type requires use_size")
} }
} }
TEST_CASE("UBJSON round-trip invariants")
{
// This checks what the parse_ubjson_fuzzer driver checks (see
// tests/src/fuzzer-parse_ubjson.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_ubjson() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// reproduces the exact bytes. Beyond the driver, this also checks that j2
// equals j1. Values are compared with dump() rather than operator==,
// because a NaN never compares equal to itself.
struct options
{
bool use_size;
bool use_type;
};
const std::vector<options> all_options =
{
{false, false},
{true, false},
{true, true},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_ubjson() returns it; this
// has no binary values, as UBJSON writes them as arrays of integers
for (const auto& initial : all_options)
{
const json j1 = json::from_ubjson(json::to_ubjson(j0, initial.use_size, initial.use_type));
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type);
const std::vector<std::uint8_t> vec = json::to_ubjson(j1, o.use_size, o.use_type);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_ubjson(vec));
CHECK(j2.dump() == j1.dump());
CHECK(json::to_ubjson(j2, o.use_size, o.use_type) == vec);
}
}
}
}
TEST_CASE("UBJSON roundtrips" * doctest::skip()) TEST_CASE("UBJSON roundtrips" * doctest::skip())
{ {
SECTION("input from self-generated UBJSON files") SECTION("input from self-generated UBJSON files")
+1 -1
View File
@@ -20,7 +20,7 @@ if __name__ == '__main__':
namespaces = ['nlohmann'] namespaces = ['nlohmann']
abi_prefix = 'json_abi' abi_prefix = 'json_abi'
abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics', '_psp'] abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics']
version = '_v' + args.version.replace('.', '_') version = '_v' + args.version.replace('.', '_')
inline_namespaces = [] inline_namespaces = []