Merge branch 'develop' into copy-scratch-allocator

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,7 +2,6 @@
 
 - [ ] The changes are described in detail, both the what and why.
 - [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
-- [ ] If applicable, a fixed [OSS-Fuzz](https://issues.oss-fuzz.com) issue is referenced as `OSS-Fuzz: <id>` (see [fuzz testing](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports)).
 - [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
 - [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
 - [ ] The source code is amalgamated by running `make amalgamate`.

BUILD.bazel

@@ -71,6 +71,7 @@ cc_library(
     ],
     includes = ["include"],
     visibility = ["//visibility:public"],
+    alwayslink = True,
 )
 
 cc_library(

cmake/scripts/gen_bazel_build_file.cmake

@@ -42,6 +42,7 @@ string(APPEND CONTENT [=[
     ],
     includes = ["include"],
     visibility = ["//visibility:public"],
+    alwayslink = True,
 )
 
 cc_library(

docs/mkdocs/docs/community/assurance_case.md

@@ -1,70 +0,0 @@
-# Assurance case
-
-This page argues why the library meets its security requirements. It describes the threats the library faces, where the
-trust boundaries lie, and how the library's design and the [quality assurance](quality_assurance.md) counter these
-threats. To report a vulnerability, see the [security policy](security_policy.md).
-
-## Threat model
-
-The library parses, stores, and serializes JSON values in memory. It does not open network connections, does not open
-files (it only reads from streams or `std::FILE*` handles that the caller has already opened), does not read environment
-variables, and does not implement cryptography or handle credentials.
-
-The primary threat is therefore **untrusted input**: JSON text or binary data (BJData, BSON, CBOR, MessagePack, UBJSON)
-that an attacker controls, passed to [`parse`](../api/basic_json/parse.md), [`accept`](../api/basic_json/accept.md),
-[`sax_parse`](../api/basic_json/sax_parse.md), or one of the `from_*` functions such as
-[`from_cbor`](../api/basic_json/from_cbor.md). Such input may try to
-
-- make the library read or write out of bounds (malformed lengths, truncated input, invalid UTF-8),
-- trigger undefined behavior (integer overflow in sizes or numbers, invalid casts),
-- exhaust memory (huge announced sizes), or
-- exhaust the call stack (deeply nested arrays and objects).
-
-## Trust boundaries
-
-- **Untrusted:** all serialized input read by the parser, the SAX interface, and the binary readers. The library must
-  handle every possible input by either producing a value or throwing a [`parse_error`](../home/exceptions.md#parse-errors)
-  (or returning `false` when exceptions are disabled for the call).
-- **Trusted:** the C++ code that calls the library. Calling a function with violated preconditions, for instance
-  accessing an array with [`operator[]`](../api/basic_json/operator%5B%5D.md) out of range, is a programming error and
-  not a security boundary. Such preconditions are checked with [runtime assertions](../features/assertions.md) in debug
-  builds; functions such as [`at`](../api/basic_json/at.md) offer checked access with exceptions.
-
-## Secure design
-
-- **Strict parsing.** The parser accepts exactly the JSON grammar of [RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259).
-  Extensions such as [comments](../features/comments.md) and [trailing commas](../features/trailing_commas.md) must be
-  enabled explicitly. Invalid UTF-8 is rejected.
-- **Errors are reported, not ignored.** Malformed input results in a [`parse_error`](../home/exceptions.md#parse-errors)
-  with the byte position of the error. Binary readers do not trust announced sizes: strings and binary values grow
-  only as bytes are actually read, arrays reserve at most a fixed number of elements up front, and sizes that no
-  container can hold are rejected.
-- **Memory is owned by values.** Each `basic_json` value owns its content, and there is no manual memory management in
-  user code. The destructor does not recurse, so destroying a deeply nested value does not exhaust the stack.
-- **Bounded recursion.** The JSON parser and the binary readers keep their state in explicit stacks instead of
-  recursing per nesting level. Operations that walk a value, such as [`dump`](../api/basic_json/dump.md), copying,
-  hashing, and [`merge_patch`](../api/basic_json/merge_patch.md), recurse only up to a fixed depth and continue with an
-  explicit stack below it. Some operations, such as comparison, [`diff`](../api/basic_json/diff.md),
-  [`flatten`](../api/basic_json/flatten.md), and the binary writers, still recurse once per nesting level; work on them
-  is in progress. Applications that process untrusted input can limit its nesting depth with a
-  [parser callback](../features/parsing/parser_callbacks.md).
-- **Invariants are checked.** The class invariant (for instance, that the pointer for the stored type is never null) is
-  checked with runtime assertions throughout the test suite.
-
-## Common weaknesses
-
-The following table maps the relevant classes of the [Common Weakness Enumeration](https://cwe.mitre.org) to the
-measures that counter them. The measures are described in detail in [Quality assurance](quality_assurance.md).
-
-| Weakness                                                                  | Countermeasures                                                                                  |
-|---------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|
-| Out-of-bounds read/write ([CWE-125](https://cwe.mitre.org/data/definitions/125.html), [CWE-787](https://cwe.mitre.org/data/definitions/787.html)) | bounds checks on all reads from the input; AddressSanitizer and Valgrind on the test suite; OSS-Fuzz      |
-| Integer overflow ([CWE-190](https://cwe.mitre.org/data/definitions/190.html)) | UndefinedBehaviorSanitizer with integer overflow detection; Clang-Tidy; Cppcheck                 |
-| Use after free, double free ([CWE-416](https://cwe.mitre.org/data/definitions/416.html), [CWE-415](https://cwe.mitre.org/data/definitions/415.html)) | ownership of all memory by values; AddressSanitizer and Valgrind; Clang Static Analyzer          |
-| Memory leaks ([CWE-401](https://cwe.mitre.org/data/definitions/401.html)) | Valgrind (Memcheck) on the test suite                                                            |
-| Uncontrolled recursion ([CWE-674](https://cwe.mitre.org/data/definitions/674.html)) | iterative parser, binary readers, and destructor; bounded recursion in value operations; tests with deeply nested inputs |
-| Uncontrolled resource consumption ([CWE-400](https://cwe.mitre.org/data/definitions/400.html)) | allocations based on announced sizes are capped; OSS-Fuzz with memory limits                     |
-| Undefined behavior in general ([CWE-758](https://cwe.mitre.org/data/definitions/758.html)) | UndefinedBehaviorSanitizer; runtime assertions; Clang-Tidy, Cppcheck, Clang Static Analyzer, Infer |
-
-In addition, every line of the library is covered by the unit tests, and all parsers are fuzz-tested around the clock
-by [OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json).

docs/mkdocs/docs/community/index.md

@@ -5,6 +5,4 @@
 - [Contribution Guidelines](contribution_guidelines.md) - guidelines how to contribute to this project
 - [Governance](governance.md) - the governance model of this project
 - [Quality Assurance](quality_assurance.md) - how the quality of this project is assured
-- [Roadmap](roadmap.md) - what the project will and will not do
 - [Security Policy](security_policy.md) - the security policy of the project
-- [Assurance Case](assurance_case.md) - why the library meets its security requirements

docs/mkdocs/docs/community/quality_assurance.md

@@ -164,9 +164,6 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
 - [x] The parser is tested against extensive correctness suites for JSON compliance.
 - [x] In addition, the library is continuously fuzz-tested at [OSS-Fuzz](https://google.github.io/oss-fuzz/) where the
   library is checked against billions of inputs.
-- [x] Every crash reported by OSS-Fuzz is fixed together with a unit test that reproduces it, and the fix references
-  the OSS-Fuzz issue. The round-trip checks of the fuzzer drivers are also part of the unit tests. See the
-  [fuzz testing documentation](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports).
 
 ## Static analysis
 

docs/mkdocs/docs/community/roadmap.md

@@ -1,43 +0,0 @@
-# Roadmap
-
-This page describes what the project intends to do, and what it does not intend to do, over the next year. Concrete
-work items are tracked in the [GitHub milestones](https://github.com/nlohmann/json/milestones) and the
-[issue tracker](https://github.com/nlohmann/json/issues).
-
-## What the project will do
-
-- **Keep the C++11 baseline.** The library will continue to compile with every
-  [supported C++11 compiler](https://github.com/nlohmann/json/blob/develop/README.md#supported-compilers). Features of
-  later standards are only used when they are guarded by the `JSON_HAS_CPP_*` macros.
-- **Stay conformant to JSON.** The parser and serializer follow [RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259).
-  Extensions such as [comments](../features/comments.md) or [trailing commas](../features/trailing_commas.md) remain
-  opt-in.
-- **Keep the 3.x public API stable.** Releases follow [semantic versioning](https://semver.org). Changes that would
-  break existing code are only added behind a feature macro, so users can opt in and test their code before a next
-  major release.
-- **Support a broad range of compilers and platforms.** The [CI](quality_assurance.md) keeps testing old and new
-  versions of GCC, Clang, MSVC, and other compilers on Linux, macOS, and Windows.
-- **Keep the quality assurance up.** Every change keeps the test coverage at 100%, passes the static and dynamic
-  analysis, and is fuzz-tested by OSS-Fuzz, see [Quality assurance](quality_assurance.md).
-- **Harden the library against hostile input.** Handling deeply nested values without exhausting the call stack is
-  ongoing work.
-- **Fix bugs and security issues** reported through the issue tracker and the [security policy](security_policy.md).
-
-## What the project will not do
-
-- **Break the public API of version 3.x.** See the
-  [contribution guidelines](https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#break-the-public-api)
-  for what counts as a breaking change.
-- **Require a newer C++ standard than C++11.**
-- **Break JSON conformance** or enable non-standard extensions by default.
-- **Add dependencies** or require a build step. The library remains header-only, and the single header
-  `json.hpp` remains a complete distribution.
-- **Trade simplicity for speed or memory efficiency.** Performance improvements are welcome, but the library is not
-  meant to compete with the fastest JSON libraries, see [Design goals](../home/design_goals.md).
-
-## Version 4.0
-
-There is no decision yet on whether or when a version 4.0 with breaking changes will be released. Proposals that need
-a major version, for instance stricter type conversions, are collected in issue
-[#3453](https://github.com/nlohmann/json/issues/3453). Until then, such changes are only added as opt-in behavior
-behind feature macros.

docs/mkdocs/docs/features/binary_formats/bjdata.md

@@ -208,16 +208,6 @@ The library maps BJData types to JSON value types as follows:
 
     The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
 
-!!! info "Round trips"
-
-    A value returned by [`from_bjdata`](../../api/basic_json/from_bjdata.md) can be serialized with
-    [`to_bjdata`](../../api/basic_json/to_bjdata.md) using any combination of options and parsed back into an equal
-    value, and serializing that value again with the same options produces the same bytes. The exception is binary
-    values: they are only written as an optimized binary array (`[$B`) if Draft 3 is enabled and both `use_size` and
-    `use_type` are set. Otherwise, they are written as arrays of integers and parsed back as such (see the notes on
-    binary values above), and serializing such an array again may choose different, but equally valid, type markers.
-    The bytes can then differ, but parsing them again yields the same value.
-
 ??? example
 
     ```cpp

docs/mkdocs/docs/features/types/template_parameters.md

@@ -562,7 +562,11 @@ binary32 or binary64 field and have no encoding for `#!cpp long double`.
 ## `AllocatorType`
 
 `AllocatorType` is instantiated with **one** argument, for each of `object_t`, `array_t`, `string_t`, `binary_t`,
-`basic_json`, and `#!cpp std::pair<const StringType, basic_json>`.
+`basic_json`, `#!cpp std::pair<const StringType, basic_json>`, and `#!cpp std::pair<StringType, basic_json>`.
+
+`AllocatorType` is not the only allocator a `basic_json` uses. It allocates the JSON values themselves, but most
+temporary storage is allocated with `#!cpp std::allocator`. This includes the parser's stacks and the stacks that
+process deeply nested values without recursion.
 
 ### Always required
 

docs/mkdocs/docs/home/architecture.md

@@ -1,125 +1,34 @@
 # Architecture
 
-This page gives a high-level overview of the library's architecture. It should help new contributors to get an idea of
+!!! info
-the used concepts and where to make changes.
+
+    This page is still under construction. Its goal is to provide a high-level overview of the library's architecture.
+    This should help new contributors to get an idea of the used concepts and where to make changes.
 
 ## Overview
 
-The library is built around a single class template, [`nlohmann::basic_json`](../api/basic_json/index.md). A
+The main structure is class [nlohmann::basic_json](../api/basic_json/index.md).
-`basic_json` value is a node in a tree of JSON values. All other components either create such a tree from an input
-(parsing), write a tree to an output (serialization), or give access to it (iterators, JSON Pointer, conversions).
 
-```mermaid
+- public API
-flowchart LR
+- container interface
-    input[/"input<br>(string, stream,<br>iterator range, file)"/]
+- iterators
-    ia["input adapter"]
-    lexer["lexer"]
-    parser["parser"]
-    breader["binary_reader"]
-    sax["SAX interface"]
-    value[("basic_json<br>value tree")]
-    serializer["serializer"]
-    bwriter["binary_writer"]
-    oa["output adapter"]
-    output[/"output<br>(string, stream,<br>vector)"/]
 
-    input --> ia
+## Template specializations
-    ia --> lexer --> parser --> sax
-    ia --> breader --> sax
-    sax --> value
-    value --> serializer --> oa
-    value --> bwriter --> oa
-    oa --> output
-```
 
-- **JSON text** is read by an [input adapter](#input-adapters), tokenized by the lexer, and turned into SAX events by
+- describe template parameters of `basic_json`
-  the parser.
+- [`json`](../api/json.md)
-- **Binary formats** (BJData, BSON, CBOR, MessagePack, UBJSON) are read by an input adapter and turned into the same SAX
+- [`ordered_json`](../api/ordered_json.md) via [`ordered_map`](../api/ordered_map.md)
-  events by the `binary_reader`.
-- A [SAX consumer](#sax-interface) receives the events. The one used by [`parse`](../api/basic_json/parse.md) builds a
-  `basic_json` value tree.
-- The `serializer` (JSON text) or the `binary_writer` (binary formats) writes a value tree to an
-  [output adapter](#output-adapters).
-
-## Source layout
-
-The public headers are in [`include/nlohmann`](https://github.com/nlohmann/json/tree/develop/include/nlohmann):
-
-- [`json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json.hpp) defines class [`basic_json`](../api/basic_json/index.md).
-- [`json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json_fwd.hpp) contains forward declarations.
-- [`adl_serializer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/adl_serializer.hpp), [`byte_container_with_subtype.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/byte_container_with_subtype.hpp), and [`ordered_map.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/ordered_map.hpp) define
-  [`adl_serializer`](../api/adl_serializer/index.md),
-  [`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md), and
-  [`ordered_map`](../api/ordered_map.md).
-
-Everything else lives in [`detail/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail) and namespace `nlohmann::detail`, which is not part of the public API. Paths
-below are relative to `include/nlohmann`.
-
-| Component | Location |
-|-----------|----------|
-| Value type enumeration | [`detail/value_t.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/value_t.hpp) |
-| Input adapters | [`detail/input/input_adapters.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/input_adapters.hpp) |
-| Lexer | [`detail/input/lexer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/lexer.hpp), [`detail/input/number_parse.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/number_parse.hpp), [`detail/input/string_scan.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/string_scan.hpp) |
-| Parser | [`detail/input/parser.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/parser.hpp) |
-| SAX interface and DOM builders | [`detail/input/json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp) |
-| Binary format readers | [`detail/input/binary_reader.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/binary_reader.hpp) |
-| JSON serializer | [`detail/output/serializer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/serializer.hpp), [`detail/conversions/to_chars.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/to_chars.hpp) |
-| Binary format writers | [`detail/output/binary_writer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/binary_writer.hpp) |
-| Output adapters | [`detail/output/output_adapters.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/output/output_adapters.hpp) |
-| Iterators | [`detail/iterators/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail/iterators) |
-| Conversions from/to arbitrary types | [`detail/conversions/from_json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/from_json.hpp), [`detail/conversions/to_json.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/conversions/to_json.hpp) |
-| JSON Pointer | [`detail/json_pointer.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/json_pointer.hpp) |
-| Exceptions | [`detail/exceptions.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/exceptions.hpp) |
-| Type traits and C++ feature backports | [`detail/meta/`](https://github.com/nlohmann/json/tree/develop/include/nlohmann/detail/meta) |
-| Macros | [`detail/macro_scope.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/macro_scope.hpp), [`detail/macro_unscope.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/macro_unscope.hpp), [`detail/abi_macros.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/abi_macros.hpp) |
-
-The single-header version [`single_include/nlohmann/json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp)
-is generated from these files with `make amalgamate` and must not be edited by hand.
-
-## Template parameters
-
-[`basic_json`](../api/basic_json/index.md) is parameterized by the types it uses to store values and to convert from and to other types:
-
-| Template parameter   | Default                     | Used for                                                          |
-|----------------------|-----------------------------|-------------------------------------------------------------------|
-| `ObjectType`         | `std::map`                  | objects, see [`object_t`](../api/basic_json/object_t.md)          |
-| `ArrayType`          | `std::vector`               | arrays, see [`array_t`](../api/basic_json/array_t.md)             |
-| `StringType`         | `std::string`               | strings and object keys, see [`string_t`](../api/basic_json/string_t.md) |
-| `BooleanType`        | `bool`                      | Booleans, see [`boolean_t`](../api/basic_json/boolean_t.md)       |
-| `NumberIntegerType`  | `std::int64_t`              | signed integers, see [`number_integer_t`](../api/basic_json/number_integer_t.md) |
-| `NumberUnsignedType` | `std::uint64_t`             | unsigned integers, see [`number_unsigned_t`](../api/basic_json/number_unsigned_t.md) |
-| `NumberFloatType`    | `double`                    | floating-point numbers, see [`number_float_t`](../api/basic_json/number_float_t.md) |
-| `AllocatorType`      | `std::allocator`            | allocating objects, arrays, strings, and binary values            |
-| `JSONSerializer`     | `adl_serializer`            | conversions from/to other types, see [`adl_serializer`](../api/adl_serializer/index.md) |
-| `BinaryType`         | `std::vector<std::uint8_t>` | binary values, see [`binary_t`](../api/basic_json/binary_t.md)  |
-| `CustomBaseClass`    | `void`                      | an optional base class, see [`json_base_class_t`](../api/basic_json/json_base_class_t.md) |
-
-The library provides two specializations:
-
-- [`json`](../api/json.md) uses all default template arguments.
-- [`ordered_json`](../api/ordered_json.md) uses [`ordered_map`](../api/ordered_map.md) as `ObjectType` to keep the
-  insertion order of object keys.
-
-The requirements on the template arguments are listed in
-[Template Parameter Requirements](../features/types/template_parameters.md).
 
 ## Value storage
 
-Each [`basic_json`](../api/basic_json/index.md) value stores its content as a tagged union: an enumeration [`value_t`](../api/basic_json/value_t.md)
+Values are stored as a tagged union of [value_t](../api/basic_json/value_t.md) and json_value.
-names the type of the value, and a union `json_value` holds the value itself. Both are members of the nested struct
-`data`, which is the only data member `m_data` of `basic_json`:
 
 ```cpp
-struct data
+/// the type of the current element
-{
+value_t m_type = value_t::null;
-    /// the type of the current element
-    value_t m_type = value_t::null;
 
-    /// the value of the current element
+/// the value of the current element
-    json_value m_value = {};
+json_value m_value = {};
-};
-
-data m_data = {};
 ```
 
 with
@@ -159,83 +68,42 @@ union json_value {
 };
 ```
 
-Objects, arrays, strings, and binary values are allocated on the heap with `AllocatorType`, and the union only stores a
+## Parsing inputs (deserialization)
-pointer to them. This keeps a `basic_json` value small: one pointer-sized union and one byte for the type. The class
-maintains the invariant that the pointer matching `m_type` is never null; `assert_invariant()` checks it with
-[runtime assertions](../features/assertions.md).
 
-## Input adapters
+Input is read via **input adapters** that abstract a source with a common interface:
-
-Input is read via **input adapters** that abstract a source. Every input adapter provides this interface:
 
 ```cpp
-/// the type of the characters in the input
+/// read a single character
-using char_type = ...;
+std::char_traits<char>::int_type get_character() noexcept;
 
-/// read a single character; returns std::char_traits<char_type>::eof() at the end of the input
+/// read multiple characters to a destination buffer and
-typename std::char_traits<char_type>::int_type get_character();
+/// returns the number of characters successfully read
-
-/// read up to count * sizeof(T) bytes into dest and return the number of bytes read
-/// (used by the binary readers)
 template<class T>
 std::size_t get_elements(T* dest, std::size_t count = 1);
 ```
 
-The lexer detects two optional extensions at compile time. Only `iterator_input_adapter` provides them, and only for
+List examples of input adapters.
-random-access input of single-byte characters:
 
-- `supports_seek`, `get_consumed_count()`, and `copy_consumed_range()` let the lexer reconstruct already consumed input
+## SAX Interface
-  for error messages instead of copying every character it reads.
-- `supports_bulk_scan`, `bulk_data()`, `bulk_remaining()`, and `bulk_skip()` let the lexer scan strings directly in
-  contiguous memory, several bytes at a time.
 
-The function `input_adapter` picks the right adapter for the argument passed to `parse`, `accept`, `sax_parse`, or the
+TODO
-`from_*` functions:
 
-- `iterator_input_adapter` reads from an iterator range, which also covers strings, containers, and pointers.
+## Writing outputs (serialization)
-- `wide_string_input_adapter` reads from ranges of `wchar_t`, `char16_t`, or `char32_t` and converts them to UTF-8.
-  It cannot be used for binary formats; its `get_elements()` throws.
-- `input_stream_adapter` reads from a `std::istream`.
-- `file_input_adapter` reads from a `std::FILE*`.
-
-## SAX interface
-
-The parser does not build values itself. It reports what it reads as events to a [SAX](../features/parsing/sax_interface.md)
-consumer, which implements the interface [`json_sax`](../api/json_sax/index.md): `null`, `boolean`, `number_integer`,
-`number_unsigned`, `number_float`, `string`, `binary`, `start_object`, `key`, `end_object`, `start_array`, `end_array`,
-and `parse_error`.
-
-The library comes with two consumers in `detail/input/json_sax.hpp`:
-
-- `json_sax_dom_parser` builds a [`basic_json`](../api/basic_json/index.md) value tree. [`parse`](../api/basic_json/parse.md) uses it.
-- `json_sax_dom_callback_parser` does the same, but calls a [parser callback](../features/parsing/parser_callbacks.md)
-  for each event, which can skip values. `parse` uses it when a callback is given.
-
-The `binary_reader` emits the same events for binary formats, so [`sax_parse`](../api/basic_json/sax_parse.md) works
-with a user-defined consumer for JSON and for all binary formats alike.
-
-## Output adapters
 
 Output is written via **output adapters**:
 
 ```cpp
+template<typename T>
 void write_character(CharType c);
 
+template<typename CharType>
 void write_characters(const CharType* s, std::size_t length);
 ```
 
-The `serializer` (used by [`dump`](../api/basic_json/dump.md) and [`operator<<`](../api/operator_ltlt.md)) and the
+List examples of output adapters.
-`binary_writer` (used by the `to_*` functions) write to one of these adapters:
-
-- `output_vector_adapter` appends to a `std::vector`.
-- `output_stream_adapter` writes to a `std::ostream`.
-- `output_string_adapter` appends to a string.
 
 ## Value conversion
 
-Values are converted from and to other types with the `JSONSerializer` template parameter. The default,
-[`adl_serializer`](../api/adl_serializer/index.md), calls the free functions
-
 ```cpp
 template<class T>
 void to_json(basic_json& j, const T& t);
@@ -244,23 +112,13 @@ template<class T>
 void from_json(const basic_json& j, T& t);
 ```
 
-found by argument-dependent lookup. The library defines them for standard types in `detail/conversions`; users add them
-for their own types, see [Arbitrary Type Conversions](../features/arbitrary_types.md). The
-[serialization macros](../features/macros.md) generate these functions.
-
 ## Additional features
 
-- [JSON Pointer](../features/json_pointer.md) (class `json_pointer`) addresses values inside a tree. It is also the
+- JSON Pointers
-  basis of [JSON Patch](../features/json_patch.md).
+- Binary formats
-- [Binary formats](../features/binary_formats/index.md) are read by `binary_reader` and written by `binary_writer`.
+- Custom base class
-- A [custom base class](../api/basic_json/json_base_class_t.md) can add members to every [`basic_json`](../api/basic_json/index.md) value.
+- Conversion macros
-- [Serialization macros](../features/macros.md) generate `to_json` and `from_json` functions for user-defined types.
 
 ## Details namespace
 
-Namespace `nlohmann::detail` contains all implementation details. It is not part of the public API and may change in any
+- C++ feature backports
-release. Besides the components above, it contains:
-
-- type traits to detect the capabilities of user-defined types (`detail/meta/type_traits.hpp`),
-- backports of C++14/17 features to C++11 (`detail/meta/cpp_future.hpp`), and
-- helpers such as `string_concat` and `string_escape`.

docs/mkdocs/mkdocs.yml

@@ -317,9 +317,7 @@ nav:
       - community/contribution_guidelines.md
       - community/quality_assurance.md
       - community/governance.md
-      - community/roadmap.md
       - community/security_policy.md
-      - community/assurance_case.md
 
 # Extras
 extra:

include/nlohmann/json.hpp

@@ -978,7 +978,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>;
 
     /// scratch space to build the key skeleton of an object copy in one go
-    using copy_scratch_t = std::vector<std::pair<typename object_t::key_type, basic_json>>;
+    using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
+    using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
 
     /// @brief copy everything of @a src into @a dst but its type and value
     static void copy_metadata(const basic_json& src, basic_json& dst)
@@ -1243,27 +1244,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     }
 
 
-    /// @brief restore the parent pointers after erasing from an object
-    /// ordered_json keeps its members in a vector, and erasing a member
-    /// re-constructs every member after it in place, which resets their
-    /// parent pointers
-    void set_parents_after_object_erase()
-    {
-#if JSON_DIAGNOSTICS
-#ifdef JSON_HEDLEY_MSVC_VERSION
-#pragma warning(push )
-#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
-#endif
-        if (detail::is_ordered_map<object_t>::value)
-        {
-            set_parents();
-        }
-#ifdef JSON_HEDLEY_MSVC_VERSION
-#pragma warning( pop )
-#endif
-#endif
-    }
-
   public:
     //////////////////////////
     // JSON parser callback //
@@ -2952,7 +2932,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             case value_t::object:
             {
                 result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator);
-                set_parents_after_object_erase();
                 break;
             }
 
@@ -3025,7 +3004,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             {
                 result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator,
                                               last.m_it.object_iterator);
-                set_parents_after_object_erase();
                 break;
             }
 
@@ -3056,9 +3034,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
         }
 
-        const auto erased = m_data.m_value.object->erase(std::forward<KeyType>(key));
+        return m_data.m_value.object->erase(std::forward<KeyType>(key));
-        set_parents_after_object_erase();
-        return erased;
     }
 
     template < typename KeyType, detail::enable_if_t <
@@ -3075,7 +3051,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         if (it != m_data.m_value.object->end())
         {
             m_data.m_value.object->erase(it);
-            set_parents_after_object_erase();
             return 1;
         }
         return 0;
@@ -3987,12 +3962,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 if (it2 != m_data.m_value.object->end() && it2->second.is_object())
                 {
                     it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1);
+#if JSON_DIAGNOSTICS
+                    it2->second.set_parents();
+#endif
                     continue;
                 }
             }
-            // set_parent() also repairs the other members, which ordered_json
+            m_data.m_value.object->operator[](it.key()) = it.value();
-            // relocates when adding a key makes its vector grow
+#if JSON_DIAGNOSTICS
-            set_parent(m_data.m_value.object->operator[](it.key()) = it.value());
+            m_data.m_value.object->operator[](it.key()).m_parent = this;
+#endif
         }
     }
 
@@ -4021,6 +4000,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 }
 
                 // a nested object is merged: continue with its parent
+#if JSON_DIAGNOSTICS
+                target->set_parents();
+#endif
                 target = stack.back().target;
                 first = stack.back().position;
                 last = stack.back().last;
@@ -4042,9 +4024,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                     continue;
                 }
             }
-            // set_parent() also repairs the other members, which ordered_json
+            target->m_data.m_value.object->operator[](first.key()) = first.value();
-            // relocates when adding a key makes its vector grow
+#if JSON_DIAGNOSTICS
-            target->set_parent(target->m_data.m_value.object->operator[](first.key()) = first.value());
+            target->m_data.m_value.object->operator[](first.key()).m_parent = target;
+#endif
             ++first;
         }
     }

single_include/nlohmann/json.hpp

@@ -25436,7 +25436,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     using copy_worklist_t = std::vector<std::pair<const basic_json*, basic_json*>>;
 
     /// scratch space to build the key skeleton of an object copy in one go
-    using copy_scratch_t = std::vector<std::pair<typename object_t::key_type, basic_json>>;
+    using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
+    using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
 
     /// @brief copy everything of @a src into @a dst but its type and value
     static void copy_metadata(const basic_json& src, basic_json& dst)
@@ -25701,27 +25702,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     }
 
 
-    /// @brief restore the parent pointers after erasing from an object
-    /// ordered_json keeps its members in a vector, and erasing a member
-    /// re-constructs every member after it in place, which resets their
-    /// parent pointers
-    void set_parents_after_object_erase()
-    {
-#if JSON_DIAGNOSTICS
-#ifdef JSON_HEDLEY_MSVC_VERSION
-#pragma warning(push )
-#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
-#endif
-        if (detail::is_ordered_map<object_t>::value)
-        {
-            set_parents();
-        }
-#ifdef JSON_HEDLEY_MSVC_VERSION
-#pragma warning( pop )
-#endif
-#endif
-    }
-
   public:
     //////////////////////////
     // JSON parser callback //
@@ -27410,7 +27390,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             case value_t::object:
             {
                 result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator);
-                set_parents_after_object_erase();
                 break;
             }
 
@@ -27483,7 +27462,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             {
                 result.m_it.object_iterator = m_data.m_value.object->erase(first.m_it.object_iterator,
                                               last.m_it.object_iterator);
-                set_parents_after_object_erase();
                 break;
             }
 
@@ -27514,9 +27492,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
         }
 
-        const auto erased = m_data.m_value.object->erase(std::forward<KeyType>(key));
+        return m_data.m_value.object->erase(std::forward<KeyType>(key));
-        set_parents_after_object_erase();
-        return erased;
     }
 
     template < typename KeyType, detail::enable_if_t <
@@ -27533,7 +27509,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         if (it != m_data.m_value.object->end())
         {
             m_data.m_value.object->erase(it);
-            set_parents_after_object_erase();
             return 1;
         }
         return 0;
@@ -28445,12 +28420,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 if (it2 != m_data.m_value.object->end() && it2->second.is_object())
                 {
                     it2->second.update_members(it.value().cbegin(), it.value().cend(), true, depth + 1);
+#if JSON_DIAGNOSTICS
+                    it2->second.set_parents();
+#endif
                     continue;
                 }
             }
-            // set_parent() also repairs the other members, which ordered_json
+            m_data.m_value.object->operator[](it.key()) = it.value();
-            // relocates when adding a key makes its vector grow
+#if JSON_DIAGNOSTICS
-            set_parent(m_data.m_value.object->operator[](it.key()) = it.value());
+            m_data.m_value.object->operator[](it.key()).m_parent = this;
+#endif
         }
     }
 
@@ -28479,6 +28458,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 }
 
                 // a nested object is merged: continue with its parent
+#if JSON_DIAGNOSTICS
+                target->set_parents();
+#endif
                 target = stack.back().target;
                 first = stack.back().position;
                 last = stack.back().last;
@@ -28500,9 +28482,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                     continue;
                 }
             }
-            // set_parent() also repairs the other members, which ordered_json
+            target->m_data.m_value.object->operator[](first.key()) = first.value();
-            // relocates when adding a key makes its vector grow
+#if JSON_DIAGNOSTICS
-            target->set_parent(target->m_data.m_value.object->operator[](first.key()) = first.value());
+            target->m_data.m_value.object->operator[](first.key()).m_parent = target;
+#endif
             ++first;
         }
     }

tests/fuzzing.md

@@ -79,26 +79,3 @@ the same `fuzzers` target as above and also relies on the `FUZZER_ENGINE` variab
 [build script](https://github.com/google/oss-fuzz/blob/master/projects/json/build.sh) for more information.
 
 In case the build at OSS-Fuzz fails, an issue will be created automatically.
-
-### Handling OSS-Fuzz reports
-
-OSS-Fuzz files the crashes it finds in its own [issue tracker](https://issues.oss-fuzz.com), not on GitHub. So that
-each report can be traced to the change that fixed it, and each fix to the report it answers, fixes follow these
-conventions:
-
-- **Reference the OSS-Fuzz issue in the pull request**, next to any GitHub issue it closes, as `OSS-Fuzz: <id>` (for
-  example, `OSS-Fuzz: 563659413`), and in the commit message. The ID alone does not disclose the crash. If the report
-  was triaged into a GitHub issue, link the OSS-Fuzz issue there too.
-- **Turn the reproducer into a unit test.** Download the testcase from the OSS-Fuzz report, reduce it if possible, and
-  add it as a regression test to the unit test of the affected format (e.g., `tests/src/unit-bjdata.cpp`), with a
-  comment naming the OSS-Fuzz issue. This way the input is checked by every CI run rather than only by OSS-Fuzz, and
-  it stays covered even if OSS-Fuzz later closes the report as not reproducible.
-- **Keep the fuzzer drivers and the unit tests in sync.** The round-trip checks of the UBJSON and BJData drivers are
-  also run on a fixed corpus in the unit tests (see `tests/src/round_trip_corpus.hpp` and the "round-trip invariants"
-  test cases), so a regression shows up in CI first. When a driver's checks change, change the unit tests with them.
-- **Record in the report whether the bug shipped.** OSS-Fuzz asks whether a crash was a short-lived regression or
-  affects a released version; answer it when the fix is merged, as it decides whether the fix needs a release note or
-  a security advisory (see the [security policy](../.github/SECURITY.md)).
-
-After the fix is merged, OSS-Fuzz re-runs the reproducer on its next build and marks the report as verified and
-closed. If it does not, the fix is incomplete.

tests/src/fuzzer-parse_bjdata.cpp

@@ -42,9 +42,6 @@ dump() serializes any non-finite double the same deterministic way (as JSON
 `null`, since JSON itself cannot represent NaN/Infinity), so comparing
 dumps is stable under exactly the same values that break operator==.
 
-The unit tests run the same checks on a fixed corpus (see the "BJData round-trip
-invariants" test case), so keep both in sync.
-
 The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
 drivers.
 */

tests/src/fuzzer-parse_ubjson.cpp

@@ -21,9 +21,6 @@ array data, it performs the following steps:
 - j4 = from_ubjson(vec3)
 - assert(j1 == j4)
 
-The unit tests run the same checks on a fixed corpus (see the "UBJSON round-trip
-invariants" test case), so keep both in sync.
-
 The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
 drivers.
 */

tests/src/round_trip_corpus.hpp

@@ -1,213 +0,0 @@
-//     __ _____ _____ _____
-//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
-// |  |  |__   |  |  | | | |  version 3.12.0
-// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
-//
-// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
-// SPDX-License-Identifier: MIT
-
-#pragma once
-
-#include <cmath> // nan
-#include <cstddef> // size_t
-#include <cstdint> // int32_t, int64_t, uint32_t, uint64_t
-#include <limits> // numeric_limits
-#include <random> // mt19937
-#include <string> // string, to_string
-#include <utility> // move
-#include <vector> // vector
-
-#include <nlohmann/json.hpp>
-
-// Values for the round-trip property tests of the UBJSON and BJData writers.
-//
-// The fuzzer drivers (tests/src/fuzzer-parse_ubjson.cpp and
-// fuzzer-parse_bjdata.cpp) check that anything the library parses can be
-// serialized, parsed back, and serialized again without loss. Those checks
-// only run at OSS-Fuzz, so a regression used to surface days later as an
-// external report. The unit tests run the same checks on this corpus in CI.
-//
-// The corpus is deterministic: std::mt19937's output sequence is fixed by
-// the standard, and it is used directly rather than through a distribution
-// (whose results are implementation-defined).
-namespace utils
-{
-
-class round_trip_corpus
-{
-  public:
-    using json = nlohmann::json;
-
-    static std::vector<json> values()
-    {
-        round_trip_corpus corpus;
-        return corpus.build();
-    }
-
-    // whether a value contains a binary value, which a BJData or UBJSON round
-    // trip may turn into an array of integers
-    static bool contains_binary(const json& j)
-    {
-        if (j.is_binary())
-        {
-            return true;
-        }
-        if (j.is_structured())
-        {
-            for (const auto& element : j)
-            {
-                if (contains_binary(element))
-                {
-                    return true;
-                }
-            }
-        }
-        return false;
-    }
-
-  private:
-    std::vector<json> atoms;
-    // a fixed seed is the point: the corpus must be the same in every run
-    std::mt19937 generator{42}; // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
-
-    round_trip_corpus()
-        : atoms
-    {
-        nullptr, true, false,
-        // integers at the boundaries of every UBJSON/BJData integer type
-        0, 1, -1, 127, 128, 255, 256, -128, -129,
-        32767, 32768, 65535, 65536, -32768, -32769,
-        (std::numeric_limits<std::int32_t>::min)(), (std::numeric_limits<std::int32_t>::max)(),
-        (std::numeric_limits<std::uint32_t>::max)(),
-        (std::numeric_limits<std::int64_t>::min)(), (std::numeric_limits<std::int64_t>::max)(),
-        static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()) + 1u,
-        (std::numeric_limits<std::uint64_t>::max)(),
-        // floating-point numbers, including non-finite ones
-        0.0, -0.0, 1.5, -2.25, 3.4e38, (std::numeric_limits<double>::max)(),
-        std::nan(""), std::numeric_limits<double>::infinity(), -std::numeric_limits<double>::infinity(),
-        // strings, including a non-ASCII one and one longer than 255 bytes
-        "", "a", "\xC3\xA4", std::string(300, 'x'),
-        // binary values with and without subtype
-        json::binary({}), json::binary({1, 2, 255}), json::binary({0x80, 0x7F}, 42), json::binary({1}, 0)
-    }
-    {}
-
-    std::vector<json> build()
-    {
-        std::vector<json> result = atoms;
-
-        // each atom inside containers, including homogeneous ones that the
-        // writers encode as optimized (typed) containers
-        result.emplace_back(json::array());
-        result.emplace_back(json::object());
-        for (const auto& atom : atoms)
-        {
-            result.push_back(json::array({atom}));
-            result.push_back(json::array({atom, atom, atom}));
-            result.push_back(json::array({json::array({atom})}));
-            result.push_back(json::object({{"key", atom}}));
-        }
-        result.push_back(json::array({1, 1.5}));
-        result.push_back(json::array({-1, 255}));
-        result.push_back(json::array({"a", "b"}));
-
-        // deep, but well below any recursion or depth limit
-        json nested_array = 1;
-        json nested_object = 1;
-        for (int i = 0; i < 300; ++i)
-        {
-            nested_array = json::array({nested_array});
-            nested_object = json::object({{"key", nested_object}});
-        }
-        result.push_back(nested_array);
-        result.push_back(nested_object);
-
-        add_annotated_arrays(result);
-        add_random_values(result);
-        return result;
-    }
-
-    // objects in the JData annotated array format, which the BJData writer
-    // encodes as ND-arrays when the annotation describes a packed array, and
-    // as plain objects otherwise (see #5398, #5399, #5403, #5404, and #5542)
-    static void add_annotated_arrays(std::vector<json>& result)
-    {
-        const std::vector<json> types =
-        {
-            "uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64",
-            "single", "double", "char", "byte", "bool", "unknown", 5, nullptr
-        };
-        const std::vector<json> sizes =
-        {
-            json::array(), {3}, {1, 3}, {3, 1}, {2, 3}, {2, 0}, {0, 2}, {2, 2, 2}, {-1, 2}, {2, 1.5},
-            "3", 3, nullptr, json::binary({})
-        };
-        const std::vector<json> data =
-        {
-            nullptr, 5, "s", json::object({{"a", 1}}), json::array(),
-            {1, 2, 3}, {1, 2, 3, 4, 5, 6}, {1, 2, 3, 4, 5, 6, 7, 8},
-            {1.5, 2.5, 3.5, 4.5, 5.5, 6.5}, {300, -300, 70000, -70000, 1, 2},
-            {"a", "b", "c", "d", "e", "f"}, {json::array({1, 2, 3}), json::array({4, 5, 6})}
-        };
-
-        for (const auto& type : types)
-        {
-            for (const auto& size : sizes)
-            {
-                for (const auto& d : data)
-                {
-                    result.push_back({{"_ArrayType_", type}, {"_ArraySize_", size}, {"_ArrayData_", d}});
-                }
-            }
-        }
-
-        // incomplete annotations and annotations with an extra key
-        result.push_back({{"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
-        result.push_back({{"_ArrayType_", "uint8"}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
-        result.push_back({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}});
-        result.push_back({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"extra", 1}});
-    }
-
-    // random containers of atoms, both homogeneous and mixed
-    void add_random_values(std::vector<json>& result)
-    {
-        for (int i = 0; i < 1000; ++i)
-        {
-            result.push_back(random_value(0));
-        }
-    }
-
-    std::size_t random_below(std::size_t bound)
-    {
-        return generator() % bound;
-    }
-
-    json random_value(int depth)
-    {
-        const auto kind = random_below(10);
-        if (depth > 3 || kind < 5)
-        {
-            return atoms[random_below(atoms.size())];
-        }
-
-        json result = kind < 8 ? json::array() : json::object();
-        const auto count = random_below(5);
-        const bool homogeneous = random_below(2) == 0;
-        const json fixed = atoms[random_below(atoms.size())];
-        for (std::size_t i = 0; i < count; ++i)
-        {
-            json element = homogeneous ? fixed : random_value(depth + 1);
-            if (result.is_array())
-            {
-                result.push_back(std::move(element));
-            }
-            else
-            {
-                result[std::to_string(i)] = std::move(element);
-            }
-        }
-        return result;
-    }
-};
-
-} // namespace utils

tests/src/unit-allocator.cpp

@@ -270,6 +270,82 @@ TEST_CASE("controlled bad_alloc")
     }
 }
 
+namespace
+{
+// counts the allocations of pairs with a non-const first member: the object
+// types store std::pair<const Key, T>, so only the scratch space of the
+// iterative deep copy allocates std::pair<Key, T>
+std::size_t scratch_pair_allocations = 0;
+
+template<class T>
+struct is_scratch_pair : std::false_type {};
+
+template<class K, class V>
+struct is_scratch_pair<std::pair<K, V>> : std::integral_constant < bool, !std::is_const<K>::value > {};
+
+template<class T>
+struct scratch_counting_allocator : std::allocator<T>
+{
+    using std::allocator<T>::allocator;
+
+    T* allocate(std::size_t n)
+    {
+        if (is_scratch_pair<T>::value)
+        {
+            ++scratch_pair_allocations;
+        }
+        return std::allocator<T>::allocate(n);
+    }
+
+#ifdef __cpp_lib_allocate_at_least
+    // std::allocator<T>::allocate_at_least would bypass the counting, and
+    // libc++'s containers prefer it over allocate from C++23 on
+    auto allocate_at_least(std::size_t n)
+    {
+        if (is_scratch_pair<T>::value)
+        {
+            ++scratch_pair_allocations;
+        }
+        return std::allocator<T>::allocate_at_least(n);
+    }
+#endif
+
+    template <class U>
+    struct rebind
+    {
+        using other = scratch_counting_allocator<U>;
+    };
+};
+} // namespace
+
+TEST_CASE("deep copy uses the provided allocator")
+{
+    using counting_json = nlohmann::basic_json<std::map,
+          std::vector,
+          std::string,
+          bool,
+          std::int64_t,
+          std::uint64_t,
+          double,
+          scratch_counting_allocator>;
+
+    // deeper than the 128 levels the copy constructor descends into, so the
+    // innermost objects are copied by the iterative deep copy
+    counting_json j = 1;
+    for (std::size_t i = 0; i < 300; ++i)
+    {
+        counting_json wrapper = counting_json::object();
+        wrapper["a"] = std::move(j);
+        j = std::move(wrapper);
+    }
+
+    scratch_pair_allocations = 0;
+    // NOLINTNEXTLINE(performance-unnecessary-copy-initialization): the copy is what is tested
+    const counting_json copy(j);
+    CHECK(scratch_pair_allocations > 0);
+    CHECK(copy == j);
+}
+
 namespace
 {
 template<class T>

tests/src/unit-bjdata.cpp

@@ -19,7 +19,6 @@ using nlohmann::json;
 #include <fstream>
 #include <set>
 #include "make_test_data_available.hpp"
-#include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 
 namespace
@@ -2868,21 +2867,6 @@ TEST_CASE("BJData")
                 const auto out_num = json::to_bjdata(j_num);
                 CHECK(out_num.at(0) == '{');
                 CHECK(json::from_bjdata(out_num) == j_num);
-
-                // OSS-Fuzz issue 474400817: an empty object _ArraySize_ was
-                // written as the ND-array header length, which from_bjdata()
-                // could not read back
-                const std::vector<uint8_t> input =
-                {
-                    '[', '{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
-                    'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
-                    'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '{', '}', '}', ']'
-                };
-                const json j1 = json::from_bjdata(input);
-                CHECK(j1 == json::parse(R"([{"_ArrayType_":"int16","_ArraySize_":{},"_ArrayData_":null}])"));
-                json j2;
-                CHECK_NOTHROW(j2 = json::from_bjdata(json::to_bjdata(j1, false, false)));
-                CHECK(j2 == j1);
             }
 
             SECTION("ndarray with out-of-range _ArrayData_ elements stays as object")
@@ -4289,93 +4273,6 @@ TEST_CASE("BJData use_type requires use_size")
     }
 }
 
-TEST_CASE("BJData round-trip invariants")
-{
-    // This checks what the parse_bjdata_fuzzer driver checks (see
-    // tests/src/fuzzer-parse_bjdata.cpp), so that a regression shows up in CI
-    // rather than as an OSS-Fuzz report: every value from_bjdata() returns
-    // (j1) can be serialized with any combination of options, the result can
-    // be parsed back (j2), and serializing j2 again with the same options
-    // yields a value-equal result.
-    //
-    // Beyond the driver, this also checks that j2 equals j1 and that
-    // serializing j2 reproduces the exact bytes, both except for values that
-    // contain a binary value: a binary value is only written as a binary
-    // value with Draft 3's optimized binary array, and otherwise read back as
-    // an array of integers, for which the writer may choose different (but
-    // equally valid) type markers when it is serialized again (see #5494).
-    //
-    // Values are compared with dump() rather than operator==, because a NaN
-    // never compares equal to itself.
-    struct options
-    {
-        bool use_size;
-        bool use_type;
-        json::bjdata_version_t version;
-    };
-    const std::vector<options> all_options =
-    {
-        {false, false, json::bjdata_version_t::draft2},
-        {true, false, json::bjdata_version_t::draft2},
-        {true, true, json::bjdata_version_t::draft2},
-        {false, false, json::bjdata_version_t::draft3},
-        {true, false, json::bjdata_version_t::draft3},
-        {true, true, json::bjdata_version_t::draft3},
-    };
-
-    for (const auto& j0 : utils::round_trip_corpus::values())
-    {
-        // turn the corpus value into a value as from_bjdata() returns it
-        for (const auto& initial : all_options)
-        {
-            const json j1 = json::from_bjdata(json::to_bjdata(j0, initial.use_size, initial.use_type, initial.version));
-            const bool has_binary = utils::round_trip_corpus::contains_binary(j1);
-
-            for (const auto& o : all_options)
-            {
-                INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type
-                     << ", draft3 = " << (o.version == json::bjdata_version_t::draft3));
-
-                const std::vector<std::uint8_t> vec = json::to_bjdata(j1, o.use_size, o.use_type, o.version);
-                json j2;
-                // anything the library writes must be parsable by the library
-                REQUIRE_NOTHROW(j2 = json::from_bjdata(vec));
-                const std::vector<std::uint8_t> vec2 = json::to_bjdata(j2, o.use_size, o.use_type, o.version);
-                CHECK(json::from_bjdata(vec2).dump() == j2.dump());
-
-                if (!has_binary)
-                {
-                    CHECK(j2.dump() == j1.dump());
-                    CHECK(vec2 == vec);
-                }
-            }
-        }
-    }
-}
-
-TEST_CASE("BJData round trip of a binary value is value-stable, not byte-stable")
-{
-    // OSS-Fuzz issue 474480402: a Draft 3 optimized binary array is read as a
-    // binary value, which to_bjdata() writes in the default Draft 2 mode as a
-    // plain array of uint8 numbers. That is read back as an array of numbers,
-    // for which the writer then picks the smallest type marker, int8 ('i'),
-    // so re-serializing changes the bytes, but not the value. This is the
-    // exception described in the "Round trips" note of the BJData
-    // documentation, and why the fuzzer checks value stability (see #5494).
-    const std::vector<uint8_t> input = {'[', '$', 'B', '#', 'U', 1, 0x20};
-    const json j1 = json::from_bjdata(input);
-    CHECK(j1 == json::binary({0x20}));
-
-    const std::vector<uint8_t> vec = json::to_bjdata(j1, false, false);
-    CHECK(vec == std::vector<uint8_t>({'[', 'U', 0x20, ']'}));
-    const json j2 = json::from_bjdata(vec);
-    CHECK(j2 == json::array({0x20}));
-
-    const std::vector<uint8_t> vec2 = json::to_bjdata(j2, false, false);
-    CHECK(vec2 == std::vector<uint8_t>({'[', 'i', 0x20, ']'}));
-    CHECK(json::from_bjdata(vec2) == j2);
-}
-
 TEST_CASE("BJData roundtrips" * doctest::skip())
 {
     SECTION("input from self-generated BJData files")

tests/src/unit-diagnostics.cpp

@@ -361,106 +361,6 @@ TEST_CASE("Regression tests for extended diagnostics")
             CHECK(p == o);
         }
     }
-
-    SECTION("Regression test - erase() and update() must keep JSON_DIAGNOSTICS parent pointers of ordered_json members")
-    {
-        // ordered_json keeps its members in a vector: erasing a member
-        // re-constructs all members after it in place, and adding a key may
-        // reallocate the vector; both reset the parent pointers of the members
-        // that were moved
-        using nlohmann::ordered_json;
-
-        const auto check_parents = [](const ordered_json & j)
-        {
-            // const access, so operator[] cannot repair the parent pointers
-            CHECK_THROWS_WITH_AS(j["z"]["x"].at(0), "[json.exception.type_error.304] (/z/x) cannot use at() with number", ordered_json::type_error);
-
-            // must not trigger assert_invariant() in a debug/assert-enabled build
-            ordered_json const copy = j; // NOLINT(performance-unnecessary-copy-initialization)
-            CHECK(copy == j);
-        };
-
-        // erase(key)
-        {
-            ordered_json j = {{"a", 1}, {"z", {{"x", 1}}}};
-            CHECK(j.erase("a") == 1);
-            check_parents(j);
-        }
-
-        // erase(iterator)
-        {
-            ordered_json j = {{"a", 1}, {"z", {{"x", 1}}}};
-            j.erase(j.begin());
-            check_parents(j);
-        }
-
-        // erase(iterator, iterator)
-        {
-            ordered_json j = {{"a", 1}, {"b", 2}, {"z", {{"x", 1}}}};
-            j.erase(j.begin(), j.find("z"));
-            check_parents(j);
-        }
-
-        // patch() removes via erase(iterator)
-        {
-            ordered_json j = {{"a", 1}, {"z", {{"x", 1}}}};
-            j.patch_inplace(ordered_json::parse(R"([{"op": "remove", "path": "/a"}])"));
-            check_parents(j);
-        }
-
-        // update(j)
-        {
-            ordered_json j = {{"z", {{"x", 1}}}};
-            j.update({{"a", 1}, {"b", 2}});
-            check_parents(j);
-        }
-
-        // update(j, true), the outer and the nested vector both grow
-        {
-            ordered_json j = {{"z", {{"x", 1}}}};
-            j.update({{"z", {{"y", 2}}}, {"a", 1}}, true);
-            check_parents(j);
-        }
-
-        // update(j, true) around its descent bound, where the nested vectors
-        // grow while the objects are merged without recursing
-        for (const std::size_t depth :
-                {
-                    nlohmann::detail::recursion_depth_limit() - 1, nlohmann::detail::recursion_depth_limit(), nlohmann::detail::recursion_depth_limit() + 2
-                })
-        {
-            ordered_json j = {{"z", {{"x", 1}}}};
-            ordered_json patch = {{"a", 1}, {"b", 2}, {"c", {{"d", 3}}}};
-            for (std::size_t i = 0; i < depth; ++i)
-            {
-                j = ordered_json{{"k", 0}, {"n", std::move(j)}};
-                patch = ordered_json{{"n", std::move(patch)}, {"l", 1}, {"m", 2}};
-            }
-            j.update(patch, true);
-
-            // must not trigger assert_invariant() on any level in a
-            // debug/assert-enabled build
-            ordered_json const copy = j; // NOLINT(performance-unnecessary-copy-initialization)
-            CHECK(copy == j);
-        }
-
-        // merge_patch() inserts "c" and removes "d" at /a/c, then inserts "e"
-        // at /a, which copies /a/c
-        {
-            auto j = ordered_json::parse(R"({"a": {"c": {"d": {}}}})");
-            j.merge_patch(ordered_json::parse(R"({"a": {"c": {"c": "s", "d": null}, "e": "s"}})"));
-            CHECK(j.dump() == R"({"a":{"c":{"c":"s"},"e":"s"}})");
-
-            auto const& constJ = j;
-#if JSON_DIAGNOSTIC_POSITIONS
-            CHECK_THROWS_WITH_AS(constJ["a"]["c"]["c"].at(0), "[json.exception.type_error.304] (/a/c/c) (bytes 18-21) cannot use at() with string", ordered_json::type_error);
-#else
-            CHECK_THROWS_WITH_AS(constJ["a"]["c"]["c"].at(0), "[json.exception.type_error.304] (/a/c/c) cannot use at() with string", ordered_json::type_error);
-#endif
-            ordered_json const copy = j;
-            CHECK(copy == j);
-        }
-    }
 }
 
 TEST_CASE("Better diagnostics past the descent bound of update() and merge_patch()")

tests/src/unit-ubjson.cpp

@@ -15,7 +15,6 @@ using nlohmann::json;
 #include <fstream>
 #include <set>
 #include "make_test_data_available.hpp"
-#include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 
 namespace
@@ -2266,9 +2265,7 @@ TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
 
     SECTION("an excessive count is rejected")
     {
-        // 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value;
+        // 'l' is a big-endian int32: 0x7FFFFFFF elements, about 34 GB of value
-        // OSS-Fuzz reported this shape as a parse_ubjson_fuzzer timeout
-        // (testcase 6347769435193344, no issue filed)
         for (const auto marker :
                 {'Z', 'T', 'F'
                 })
@@ -2820,51 +2817,6 @@ TEST_CASE("UBJSON use_type requires use_size")
     }
 }
 
-TEST_CASE("UBJSON round-trip invariants")
-{
-    // This checks what the parse_ubjson_fuzzer driver checks (see
-    // tests/src/fuzzer-parse_ubjson.cpp), so that a regression shows up in CI
-    // rather than as an OSS-Fuzz report: every value from_ubjson() returns
-    // (j1) can be serialized with any combination of options, the result can
-    // be parsed back (j2), and serializing j2 again with the same options
-    // reproduces the exact bytes. Beyond the driver, this also checks that j2
-    // equals j1. Values are compared with dump() rather than operator==,
-    // because a NaN never compares equal to itself.
-    struct options
-    {
-        bool use_size;
-        bool use_type;
-    };
-    const std::vector<options> all_options =
-    {
-        {false, false},
-        {true, false},
-        {true, true},
-    };
-
-    for (const auto& j0 : utils::round_trip_corpus::values())
-    {
-        // turn the corpus value into a value as from_ubjson() returns it; this
-        // has no binary values, as UBJSON writes them as arrays of integers
-        for (const auto& initial : all_options)
-        {
-            const json j1 = json::from_ubjson(json::to_ubjson(j0, initial.use_size, initial.use_type));
-
-            for (const auto& o : all_options)
-            {
-                INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type);
-
-                const std::vector<std::uint8_t> vec = json::to_ubjson(j1, o.use_size, o.use_type);
-                json j2;
-                // anything the library writes must be parsable by the library
-                REQUIRE_NOTHROW(j2 = json::from_ubjson(vec));
-                CHECK(j2.dump() == j1.dump());
-                CHECK(json::to_ubjson(j2, o.use_size, o.use_type) == vec);
-            }
-        }
-    }
-}
-
 TEST_CASE("UBJSON roundtrips" * doctest::skip())
 {
     SECTION("input from self-generated UBJSON files")