Compare commits

..
Author SHA1 Message Date
Niels Lohmann 90078597c7 Test the remaining to_json overloads with a failing allocation
The test only reached 5 of the 14 external_constructor overloads that
were reordered: reverting the fix in any of the other 9 (const string_t&,
compatible strings, binary_t&&, array_t const&/&&, std::valarray, ranges
views, object_t const&/&&) left it green. Each of them is now called with
a failing allocation, and reverting any single hunk of the fix makes the
test fail under ASan.

binary_t&& is only reached by to_json for a binary container type other
than std::vector<std::uint8_t>, so the test calls the constructor
directly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 18:29:37 +02:00
Niels Lohmann fa28fff7d6 Skip the vector<bool> failed-allocation check for VS 2015 with iterator debugging
AppVeyor's VS 2015 Debug build terminated in "converting into an
existing value": to_json for std::vector<bool> default-constructs an
array_t first, and VS 2015's std::vector default constructor is noexcept
but, with iterator debugging, constructs its proxy through the test's
allocator, which throws.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 13:30:57 +02:00
Niels Lohmann e9c84befa1 Keep the created pointer rather than an uninitialized json_value
clang-tidy reported the json_value unions declared in the string, array,
and object constructors of to_json.hpp as uninitialized
(cppcoreguidelines-pro-type-member-init). Assign the created pointer to
the member directly once the old value is destroyed.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-26 10:07:22 +02:00
Niels Lohmann 8a26f2dc8f Skip the failed-allocation test when exceptions are disabled
The no-exceptions CI job runs doctest with --no-throw, which skips every
CHECK_THROWS_AS. The test then left next_construct_fails set, and the
next allocation outside a check threw std::bad_alloc.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-26 07:41:42 +02:00
Niels Lohmann 2c108d0b56 Create a value before giving it its type
Several functions set the type of a value before creating the string,
array, object, or binary value it stands for. When that creation threw -
std::bad_alloc from the allocator, say - the value was left behind with
the new type but nothing behind it:

- json::binary() returned no value, but its destructor asserted that a
  binary value has a binary array, aborting debug builds.
- operator[], push_back, emplace_back, emplace, and update turned a null
  value into an array or object that did not exist; any later access
  dereferenced a null pointer.
- The to_json conversions destroyed the old value first. A failed
  creation of the new one left a pointer to the destroyed old value,
  which was then used and freed again (heap-use-after-free).

The value is now created first and the type set after it, and to_json
destroys the old value only once the new one exists, so a failed
allocation leaves the value unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 22:19:39 +02:00
100 changed files with 985 additions and 10343 deletions
+4 -4
View File
@@ -37,13 +37,13 @@ labels:
files:
- "include/nlohmann/detail/input/binary_reader\\.hpp"
- "include/nlohmann/detail/output/binary_writer\\.hpp"
- "tests/src/unit-(bson|cbor|msgpack|ubjson|bjdata|bon8|binary_formats)"
- "tests/src/fuzzer-parse_(bson|cbor|msgpack|ubjson|bjdata|bon8)"
- "tests/src/unit-(bson|cbor|msgpack|ubjson|bjdata|binary_formats)"
- "tests/src/fuzzer-parse_(bson|cbor|msgpack|ubjson|bjdata)"
- "docs/mkdocs/docs/features/binary_formats/"
- "docs/mkdocs/docs/(api/basic_json|examples)/(to|from)_(bson|cbor|msgpack|ubjson|bjdata|bon8)"
- "docs/mkdocs/docs/(api/basic_json|examples)/(to|from)_(bson|cbor|msgpack|ubjson|bjdata)"
- label: "aspect: binary formats"
title: "(?i)(bson|cbor|msgpack|messagepack|ubjson|bjdata|bon8|binary format)"
title: "(?i)(bson|cbor|msgpack|messagepack|ubjson|bjdata|binary format)"
- label: "python"
files:
-9
View File
@@ -36,7 +36,6 @@ all:
@echo "clean - remove built files"
@echo "doctest - compile example files and check their output"
@echo "fuzz_testing - prepare fuzz testing of the JSON parser"
@echo "fuzz_testing_bon8 - prepare fuzz testing of the BON8 parser"
@echo "fuzz_testing_bson - prepare fuzz testing of the BSON parser"
@echo "fuzz_testing_cbor - prepare fuzz testing of the CBOR parser"
@echo "fuzz_testing_msgpack - prepare fuzz testing of the MessagePack parser"
@@ -72,14 +71,6 @@ fuzz_testing:
find tests/data/json_tests -size -5k -name *json | xargs -I{} cp "{}" fuzz-testing/testcases
@echo "Execute: afl-fuzz -i fuzz-testing/testcases -o fuzz-testing/out fuzz-testing/fuzzer"
fuzz_testing_bon8:
rm -fr fuzz-testing
mkdir -p fuzz-testing fuzz-testing/testcases fuzz-testing/out
$(MAKE) parse_bon8_fuzzer -C tests CXX=afl-clang++
mv tests/parse_bon8_fuzzer fuzz-testing/fuzzer
find tests/data -size -5k -name *.bon8 | xargs -I{} cp "{}" fuzz-testing/testcases
@echo "Execute: afl-fuzz -i fuzz-testing/testcases -o fuzz-testing/out fuzz-testing/fuzzer"
fuzz_testing_bson:
rm -fr fuzz-testing
mkdir -p fuzz-testing fuzz-testing/testcases fuzz-testing/out
+4 -12
View File
@@ -40,7 +40,7 @@
- [Implicit conversions](#implicit-conversions)
- [Conversions to/from arbitrary types](#arbitrary-types-conversions)
- [Specializing enum conversion](#specializing-enum-conversion)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, BJData, and BON8)](#binary-formats-bson-cbor-messagepack-ubjson-bjdata-and-bon8)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)](#binary-formats-bson-cbor-messagepack-ubjson-and-bjdata)
- [Customers](#customers)
- [Ecosystem](#ecosystem)
- [Supported compilers](#supported-compilers)
@@ -128,7 +128,7 @@ There is also a [**docset**](https://github.com/Kapeli/Dash-User-Contributions/t
- **JSON Pointer functions**: [flatten](https://json.nlohmann.me/api/basic_json/flatten), [unflatten](https://json.nlohmann.me/api/basic_json/unflatten)
- **JSON Patch functions**: [patch](https://json.nlohmann.me/api/basic_json/patch), [patch_inplace](https://json.nlohmann.me/api/basic_json/patch_inplace), [diff](https://json.nlohmann.me/api/basic_json/diff), [merge_patch](https://json.nlohmann.me/api/basic_json/merge_patch)
- **Static functions**: [meta](https://json.nlohmann.me/api/basic_json/meta), [get_allocator](https://json.nlohmann.me/api/basic_json/get_allocator)
- **Binary formats**: [from_bjdata](https://json.nlohmann.me/api/basic_json/from_bjdata), [from_bon8](https://json.nlohmann.me/api/basic_json/from_bon8), [from_bson](https://json.nlohmann.me/api/basic_json/from_bson), [from_cbor](https://json.nlohmann.me/api/basic_json/from_cbor), [from_msgpack](https://json.nlohmann.me/api/basic_json/from_msgpack), [from_ubjson](https://json.nlohmann.me/api/basic_json/from_ubjson), [to_bjdata](https://json.nlohmann.me/api/basic_json/to_bjdata), [to_bon8](https://json.nlohmann.me/api/basic_json/to_bon8), [to_bson](https://json.nlohmann.me/api/basic_json/to_bson), [to_cbor](https://json.nlohmann.me/api/basic_json/to_cbor), [to_msgpack](https://json.nlohmann.me/api/basic_json/to_msgpack), [to_ubjson](https://json.nlohmann.me/api/basic_json/to_ubjson)
- **Binary formats**: [from_bjdata](https://json.nlohmann.me/api/basic_json/from_bjdata), [from_bson](https://json.nlohmann.me/api/basic_json/from_bson), [from_cbor](https://json.nlohmann.me/api/basic_json/from_cbor), [from_msgpack](https://json.nlohmann.me/api/basic_json/from_msgpack), [from_ubjson](https://json.nlohmann.me/api/basic_json/from_ubjson), [to_bjdata](https://json.nlohmann.me/api/basic_json/to_bjdata), [to_bson](https://json.nlohmann.me/api/basic_json/to_bson), [to_cbor](https://json.nlohmann.me/api/basic_json/to_cbor), [to_msgpack](https://json.nlohmann.me/api/basic_json/to_msgpack), [to_ubjson](https://json.nlohmann.me/api/basic_json/to_ubjson)
- **Non-member functions**: [operator<<](https://json.nlohmann.me/api/operator_ltlt/), [operator>>](https://json.nlohmann.me/api/operator_gtgt/), [to_string](https://json.nlohmann.me/api/basic_json/to_string)
- **Literals**: [operator""_json](https://json.nlohmann.me/api/operator_literal_json)
- **Helper classes**: [std::hash&lt;basic_json&gt;](https://json.nlohmann.me/api/basic_json/std_hash), [std::swap&lt;basic_json&gt;](https://json.nlohmann.me/api/basic_json/std_swap)
@@ -1110,9 +1110,9 @@ Other Important points:
- When using `get<ENUM_TYPE>()`, undefined JSON values will default to the first pair specified in your map. Select this default pair carefully. If you desire an exception in this circumstance use `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT()` which behaves identically except for throwing an exception on unrecognized values.
- If an enum or JSON value is specified more than once in your map, the first matching occurrence from the top of the map will be returned when converting to or from JSON.
### Binary formats (BSON, CBOR, MessagePack, UBJSON, BJData, and BON8)
### Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)
Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [BSON](https://bsonspec.org) (Binary JSON), [CBOR](https://cbor.io) (Concise Binary Object Representation), [MessagePack](https://msgpack.org), [UBJSON](https://ubjson.org) (Universal Binary JSON Specification), [BJData](https://neurojson.org/bjdata) (Binary JData), and [BON8](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md) (Binary Object Notation 8) to efficiently encode JSON values to byte vectors and to decode such vectors.
Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [BSON](https://bsonspec.org) (Binary JSON), [CBOR](https://cbor.io) (Concise Binary Object Representation), [MessagePack](https://msgpack.org), [UBJSON](https://ubjson.org) (Universal Binary JSON Specification) and [BJData](https://neurojson.org/bjdata) (Binary JData) to efficiently encode JSON values to byte vectors and to decode such vectors.
```cpp
// create a JSON value
@@ -1149,14 +1149,6 @@ std::vector<std::uint8_t> v_ubjson = json::to_ubjson(j);
// roundtrip
json j_from_ubjson = json::from_ubjson(v_ubjson);
// serialize to BON8
std::vector<std::uint8_t> v_bon8 = json::to_bon8(j);
// 0x88, 0x63, 0x6F, 0x6D, 0x70, 0x61, 0x63, 0x74, 0xF9, 0x73, 0x63, 0x68, 0x65, 0x6D, 0x61, 0x90
// roundtrip
json j_from_bon8 = json::from_bon8(v_bon8);
```
The library also supports binary types from BSON, CBOR (byte strings), and MessagePack (bin, ext, fixext). They are stored by default as `std::vector<std::uint8_t>` to be processed outside the library.
+1 -1
View File
@@ -542,7 +542,7 @@ add_custom_target(ci_infer
add_custom_target(ci_offline_testdata
COMMAND mkdir -p ${PROJECT_BINARY_DIR}/build_offline_testdata/test_data
COMMAND cd ${PROJECT_BINARY_DIR}/build_offline_testdata/test_data && ${GIT_TOOL} clone -c advice.detachedHead=false --branch v3.2.0 https://github.com/nlohmann/json_test_data.git --quiet --depth 1
COMMAND cd ${PROJECT_BINARY_DIR}/build_offline_testdata/test_data && ${GIT_TOOL} clone -c advice.detachedHead=false --branch v3.1.0 https://github.com/nlohmann/json_test_data.git --quiet --depth 1
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_FastTests=ON -DJSON_TestDataDirectory=${PROJECT_BINARY_DIR}/build_offline_testdata/test_data/json_test_data
+1 -1
View File
@@ -1,5 +1,5 @@
set(JSON_TEST_DATA_URL https://github.com/nlohmann/json_test_data)
set(JSON_TEST_DATA_VERSION 3.2.0)
set(JSON_TEST_DATA_VERSION 3.1.0)
include(ExternalProject)
-3
View File
@@ -48,7 +48,6 @@ INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_bjdata', 'Fu
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_bson', 'Function', 'api/basic_json/from_bson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_cbor', 'Function', 'api/basic_json/from_cbor/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_msgpack', 'Function', 'api/basic_json/from_msgpack/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_bon8', 'Function', 'api/basic_json/from_bon8/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::from_ubjson', 'Function', 'api/basic_json/from_ubjson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::front', 'Method', 'api/basic_json/front/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::get', 'Method', 'api/basic_json/get/index.html');
@@ -122,7 +121,6 @@ INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_bjdata', 'Func
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_bson', 'Function', 'api/basic_json/to_bson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_cbor', 'Function', 'api/basic_json/to_cbor/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_msgpack', 'Function', 'api/basic_json/to_msgpack/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_bon8', 'Function', 'api/basic_json/to_bon8/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_string', 'Method', 'api/basic_json/to_string/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::to_ubjson', 'Function', 'api/basic_json/to_ubjson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::value', 'Method', 'api/basic_json/value/index.html');
@@ -173,7 +171,6 @@ INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: BJData', 'Gui
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: BSON', 'Guide', 'features/binary_formats/bson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: CBOR', 'Guide', 'features/binary_formats/cbor/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: MessagePack', 'Guide', 'features/binary_formats/messagepack/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: BON8', 'Guide', 'features/binary_formats/bon8/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Formats: UBJSON', 'Guide', 'features/binary_formats/ubjson/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Binary Values', 'Guide', 'features/binary_values/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('Comments', 'Guide', 'features/comments/index.html');
@@ -18,7 +18,7 @@ ignore
: ignore tags
store
: store tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored.
: store tagged values as binary container with subtype (for bytes 0xd8..0xdb)
## Examples
@@ -104,7 +104,6 @@ Linear in the size of the input.
- [from_msgpack](from_msgpack.md) create a JSON value from an input in MessagePack format
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -1,108 +0,0 @@
# <small>nlohmann::basic_json::</small>from_bon8
```cpp
// (1)
template<typename InputType>
static basic_json from_bon8(InputType&& i,
const bool strict = true,
const bool allow_exceptions = true);
// (2)
template<typename IteratorType, typename SentinelType = IteratorType>
static basic_json from_bon8(IteratorType first, SentinelType last,
const bool strict = true,
const bool allow_exceptions = true);
```
Deserializes a given input to a JSON value using the BON8 (Binary Object Notation 8) serialization format.
1. Reads from a compatible input.
2. Reads from an iterator range, or an iterator and a sentinel of a different type (C++20 ranges support).
The exact mapping and its limitations are described on a [dedicated page](../../features/binary_formats/bon8.md).
## Template parameters
`InputType`
: A compatible input, for instance:
- an `std::istream` object
- a `FILE` pointer
- a C-style array of characters
- a pointer to a null-terminated string of single byte characters
- a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
(as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
`IteratorType`
: a compatible iterator type
`SentinelType`
: defaults to `IteratorType`; may be a different type comparable to `IteratorType` via `operator!=`, for instance.
- a custom sentinel type for C++20 ranges
- `std::default_sentinel_t`, when `IteratorType` is `std::counted_iterator`
## Parameters
`i` (in)
: an input in BON8 format convertible to an input adapter
`first` (in)
: iterator to the start of the input
`last` (in)
: iterator to the end of the input, or a sentinel value that compares equal to the end iterator with `operator!=`
`strict` (in)
: whether to expect the input to be consumed until EOF (`#!cpp true` by default)
`allow_exceptions` (in)
: whether to throw exceptions in case of a parse error (optional, `#!cpp true` by default)
## Return value
deserialized JSON value; in case of a parse error and `allow_exceptions` set to `#!cpp false`, the return value will be
`value_t::discarded`. The latter can be checked with [`is_discarded`](is_discarded.md).
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
## Exceptions
- Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or
the end of the file was not reached when `strict` was set to true
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs, for instance
an invalid byte, a string that is not valid UTF-8, or an object key that is not a string
## Complexity
Linear in the size of the input.
## Examples
??? example
The example shows the deserialization of a byte vector in BON8 format to a JSON value.
```cpp
--8<-- "examples/from_bon8.cpp"
```
Output:
```json
--8<-- "examples/from_bon8.output"
```
## See also
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
- [from_cbor](from_cbor.md) create a JSON value from an input in CBOR format
- [from_msgpack](from_msgpack.md) create a JSON value from an input in MessagePack format
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
## Version history
- Added in version 3.13.0.
@@ -104,7 +104,6 @@ Linear in the size of the input.
- [from_msgpack](from_msgpack.md) for the related MessagePack format
- [from_ubjson](from_ubjson.md) for the related UBJSON format
- [from_bjdata](from_bjdata.md) for the related BJData format
- [from_bon8](from_bon8.md) for the related BON8 format
## Version history
@@ -110,7 +110,6 @@ Linear in the size of the input.
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -103,7 +103,6 @@ Linear in the size of the input.
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_ubjson](from_ubjson.md) create a JSON value from an input in UBJSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
@@ -104,7 +104,6 @@ Linear in the size of the input.
- [from_msgpack](from_msgpack.md) create a JSON value from an input in MessagePack format
- [from_bson](from_bson.md) create a JSON value from an input in BSON format
- [from_bjdata](from_bjdata.md) create a JSON value from an input in BJData format
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
## Version history
-2
View File
@@ -290,13 +290,11 @@ Access to the JSON value
### Binary formats
- [**from_bjdata**](from_bjdata.md) (_static_) - create a JSON value from an input in BJData format
- [**from_bon8**](from_bon8.md) (_static_) - create a JSON value from an input in BON8 format
- [**from_bson**](from_bson.md) (_static_) - create a JSON value from an input in BSON format
- [**from_cbor**](from_cbor.md) (_static_) - create a JSON value from an input in CBOR format
- [**from_msgpack**](from_msgpack.md) (_static_) - create a JSON value from an input in MessagePack format
- [**from_ubjson**](from_ubjson.md) (_static_) - create a JSON value from an input in UBJSON format
- [**to_bjdata**](to_bjdata.md) (_static_) - create a BJData serialization of a given JSON value
- [**to_bon8**](to_bon8.md) (_static_) - create a BON8 serialization of a given JSON value
- [**to_bson**](to_bson.md) (_static_) - create a BSON serialization of a given JSON value
- [**to_cbor**](to_cbor.md) (_static_) - create a CBOR serialization of a given JSON value
- [**to_msgpack**](to_msgpack.md) (_static_) - create a MessagePack serialization of a given JSON value
@@ -7,8 +7,7 @@ enum class input_format_t {
msgpack,
ubjson,
bson,
bjdata,
bon8
bjdata
};
```
@@ -32,9 +31,6 @@ bson
bjdata
: BJData (Binary JData)
bon8
: BON8 (Binary Object Notation 8)
## Examples
??? example
@@ -5,7 +5,7 @@ class parse_error : public exception;
```
The library throws this exception when a parse error occurs. Parse errors can occur during the deserialization of
JSON text, BJData, BON8, BSON, CBOR, MessagePack, UBJSON, as well as when using JSON Patch.
JSON text, BSON, CBOR, MessagePack, UBJSON, as well as when using JSON Patch.
Member `byte` holds the byte index of the last read character in the input file (see note below).
+1 -2
View File
@@ -65,8 +65,7 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
: SAX event listener (must not be null)
`format` (in)
: the format to parse (JSON, BJData, BON8, BSON, CBOR, MessagePack, or UBJSON) (optional, `input_format_t::json` by
default), see
: the format to parse (JSON, CBOR, MessagePack, or UBJSON) (optional, `input_format_t::json` by default), see
[`input_format_t`](input_format_t.md) for more information
`strict` (in)
@@ -84,7 +84,6 @@ Linear in the size of the JSON value `j`.
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
@@ -1,76 +0,0 @@
# <small>nlohmann::basic_json::</small>to_bon8
```cpp
// (1)
static std::vector<std::uint8_t> to_bon8(const basic_json& j);
// (2)
static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o);
static void to_bon8(const basic_json& j, detail::output_adapter<char> o);
```
Serializes a given JSON value `j` to a byte vector using the BON8 (Binary Object Notation 8) serialization format. BON8
is a compact binary serialization format that stores strings as UTF-8 without a length prefix.
1. Returns a byte vector containing the BON8 serialization.
2. Writes the BON8 serialization to an output adapter.
The exact mapping and its limitations are described on a [dedicated page](../../features/binary_formats/bon8.md).
## Parameters
`j` (in)
: JSON value to serialize
`o` (in)
: output adapter to write serialization to
## Return value
1. BON8 serialization as a byte vector
2. (none)
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes in the JSON value `j`, which is never modified.
With (2), the bytes written before the exception remain in the output adapter.
## Exceptions
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an unsigned integer
above 9223372036854775807, which BON8 cannot represent
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if `j` contains a string that is not
valid UTF-8
## Complexity
Linear in the size of the JSON value `j`.
## Examples
??? example
The example shows the serialization of a JSON value to a byte vector in BON8 format.
```cpp
--8<-- "examples/to_bon8.cpp"
```
Output:
```json
--8<-- "examples/to_bon8.output"
```
## See also
- [from_bon8](from_bon8.md) create a JSON value from an input in BON8 format
- [to_cbor](to_cbor.md) create a CBOR serialization of a JSON value
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
## Version history
- Added in version 3.13.0.
@@ -72,7 +72,6 @@ pass before anything is written.
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
@@ -62,7 +62,6 @@ Linear in the size of the JSON value `j`.
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
@@ -34,15 +34,6 @@ The exact mapping and its limitations are described on a [dedicated page](../../
Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
## Exceptions
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a string, binary
value, array, or object exceeds 4294967295, the maximum MessagePack can store; example:
`"MessagePack length 4294967296 exceeds maximum of 4294967295"`
- Throws [`out_of_range.415`](../../home/exceptions.md#jsonexceptionout_of_range415) if the subtype of a binary value
exceeds 255, the maximum of the MessagePack ext type; example:
`"subtype 70000 is too large for the MessagePack ext type (max 255)"`
## Complexity
Linear in the size of the JSON value `j`.
@@ -70,9 +61,7 @@ Linear in the size of the JSON value `j`.
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_ubjson](to_ubjson.md) create a UBJSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
- Added in version 2.0.9.
- Throws `out_of_range.412` and `out_of_range.415` since version 3.13.0.
@@ -77,7 +77,6 @@ Linear in the size of the JSON value `j`.
- [to_msgpack](to_msgpack.md) create a MessagePack serialization of a JSON value
- [to_bson](to_bson.md) create a BSON serialization of a JSON value
- [to_bjdata](to_bjdata.md) create a BJData serialization of a JSON value
- [to_bon8](to_bon8.md) create a BON8 serialization of a JSON value
## Version history
-2
View File
@@ -53,8 +53,6 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_BRACE_INIT_COPY_SEMANTICS**](json_brace_init_copy_semantics.md) - opt in to copy/move semantics for single-element brace initialization
- [**JSON_DISABLE_ENUM_SERIALIZATION**](json_disable_enum_serialization.md) - switch off default serialization/deserialization functions for enums
- [**JSON_USE_IMPLICIT_CONVERSIONS**](json_use_implicit_conversions.md) - control implicit conversions
- [**JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS**](json_use_objects_for_enum_keyed_maps.md) - opt in to storing maps with enum
keys as objects
## Comparison behavior
@@ -11,9 +11,9 @@ The macro only affects the JSON text parser ([`parse`](../basic_json/parse.md),
[`sax_parse`](../basic_json/sax_parse.md), and [`operator>>`](../operator_gtgt.md)). There are three cases where a NUL
byte is still not rejected:
- The binary formats ([`from_bjdata`](../basic_json/from_bjdata.md), [`from_bon8`](../basic_json/from_bon8.md),
[`from_bson`](../basic_json/from_bson.md), [`from_cbor`](../basic_json/from_cbor.md),
[`from_msgpack`](../basic_json/from_msgpack.md), [`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data.
- The binary formats ([`from_bjdata`](../basic_json/from_bjdata.md), [`from_bson`](../basic_json/from_bson.md),
[`from_cbor`](../basic_json/from_cbor.md), [`from_msgpack`](../basic_json/from_msgpack.md),
[`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data.
- A bare `const char*` pointer has no length of its own, so its length is still determined with `strlen()`. The first
NUL byte therefore still marks the end of the input, and nothing after it is read.
- One trailing `'\0'` at the end of a `char` array (e.g., a string literal) is trimmed; see the warning below.
@@ -1,139 +0,0 @@
# JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
```cpp
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS /* value */
```
When defined to `1`, maps whose keys are enums (such as `std::map<E, T>` or `std::unordered_map<E, T>`) are stored as
JSON objects, using the enum's own conversion for the keys. By default, they are stored as arrays of `[key, value]`
pairs.
## Default definition
The default value is `0` (disabled — existing behavior is preserved).
```cpp
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
```
## Notes
!!! note "Background"
JSON object keys are strings, so a map is only stored as an object if its keys can be converted to a string type.
Enums are not, even if [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md) maps them to strings, so a
map with enum keys becomes an array of `[key, value]` pairs:
```json
[["stopped", "aa"], ["completed", "bb"]]
```
With this macro, the same map becomes an object
(see [#4378](https://github.com/nlohmann/json/issues/4378)):
```json
{"completed": "bb", "stopped": "aa"}
```
!!! note "Maps with non-unique keys"
Maps that allow duplicate keys, such as `std::multimap<E, T>` or `std::unordered_multimap<E, T>`, are not affected
by the macro and are still stored as arrays of `[key, value]` pairs, as an object cannot hold duplicate keys.
!!! note "Reading"
Reading is not affected by the macro: a map with enum keys can always be read from both an array of pairs and an
object. For the latter, each key is converted to the enum with its `from_json` function, e.g., the one defined by
[`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md). Data written without the macro can therefore
still be read after enabling it.
!!! warning "Keys must serialize to distinct strings"
Each key is converted with the enum's `to_json` function. If a key is not converted to a string (for instance, an
enum without [`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md), which is stored as an integer, or an
enumerator mapped to `nullptr`), [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) is thrown.
If two keys are converted to the same string (for instance, because
[`NLOHMANN_JSON_SERIALIZE_ENUM`](nlohmann_json_serialize_enum.md) maps an unlisted enumerator to the first entry),
[`type_error.318`](../../home/exceptions.md#jsonexceptiontype_error318) is thrown. In both cases, the target value
is not changed.
!!! warning "Opt-in only"
This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no effect.
!!! note "ABI compatibility"
The value of this macro is encoded in the [namespace](../../features/namespace.md) (tag `_ekmo`), resulting in
distinct symbol names. Translation units compiled with and without it can therefore be linked into the same program
without One Definition Rule (ODR) violations, but they cannot exchange instances of library types.
## Examples
??? example "Default behavior (macro not defined)"
Without the macro, a map with enum keys is stored as an array of pairs:
```cpp
#include <map>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
enum TaskState { TS_STOPPED, TS_RUNNING, TS_COMPLETED };
NLOHMANN_JSON_SERIALIZE_ENUM(TaskState, {
{TS_STOPPED, "stopped"},
{TS_RUNNING, "running"},
{TS_COMPLETED, "completed"},
})
int main()
{
std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
json j = m;
// j is [["stopped","aa"],["completed","bb"]]
}
```
??? example "Objects for enum-keyed maps (macro defined to 1)"
With the macro, the same map is stored as an object:
```cpp
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 1
#include <map>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
enum TaskState { TS_STOPPED, TS_RUNNING, TS_COMPLETED };
NLOHMANN_JSON_SERIALIZE_ENUM(TaskState, {
{TS_STOPPED, "stopped"},
{TS_RUNNING, "running"},
{TS_COMPLETED, "completed"},
})
int main()
{
std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
json j = m;
// j is {"completed":"bb","stopped":"aa"}
auto m2 = j.get<std::map<TaskState, std::string>>();
// m2 == m
}
```
## See also
- [Specializing enum conversion](../../features/enum_conversion.md)
- [**NLOHMANN_JSON_SERIALIZE_ENUM**](nlohmann_json_serialize_enum.md) - serialize/deserialize an enum
- [**NLOHMANN_JSON_SERIALIZE_ENUM_STRICT**](nlohmann_json_serialize_enum_strict.md) - serialize/deserialize an enum with
exceptions
## Version history
- Added in version 3.13.0.
@@ -41,9 +41,6 @@ inline void from_json(const BasicJsonType& j, type& e);
conversion. Select this default pair carefully. See example 1 below.
- If an enum or JSON value is specified in multiple conversions, the first matching conversion from the top of the
list will be returned when converting to or from JSON. See example 2 below.
- Maps with enum keys (e.g., `std::map<ENUM_TYPE, T>`) are stored as arrays of `[key, value]` pairs by default.
Define [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md) to store them as objects
with the converted keys. Such maps can be read from both forms.
## Examples
@@ -83,7 +80,6 @@ inline void from_json(const BasicJsonType& j, type& e);
- [Specializing enum conversion](../../features/enum_conversion.md)
- [`NLOHMANN_JSON_SERIALIZE_ENUM_STRICT`](./nlohmann_json_serialize_enum_strict.md)
- [`JSON_DISABLE_ENUM_SERIALIZATION`](json_disable_enum_serialization.md)
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md)
## Version history
@@ -44,9 +44,6 @@ inline void from_json(const BasicJsonType& j, type& e);
`"enum value out of range for <type>"`.
- If an enum or JSON value is specified in multiple conversions, the first matching conversion from the top of the
list will be returned when converting to or from JSON. See example 2 below.
- Maps with enum keys (e.g., `std::map<ENUM_TYPE, T>`) are stored as arrays of `[key, value]` pairs by default.
Define [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md) to store them as objects
with the converted keys. Such maps can be read from both forms.
## Examples
@@ -102,7 +99,6 @@ inline void from_json(const BasicJsonType& j, type& e);
- [Specializing enum conversion](../../features/enum_conversion.md)
- [`NLOHMANN_JSON_SERIALIZE_ENUM`](./nlohmann_json_serialize_enum.md)
- [`JSON_DISABLE_ENUM_SERIALIZATION`](json_disable_enum_serialization.md)
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](json_use_objects_for_enum_keyed_maps.md)
## Version history
-21
View File
@@ -1,21 +0,0 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
// create byte vector
std::vector<std::uint8_t> v = {0x89, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x63, 0x74,
0xf9, 0x66, 0x6f, 0x72, 0x6d, 0x61, 0x74, 0xff,
0x42, 0x4f, 0x4e, 0x38, 0xff, 0x73, 0x63, 0x68,
0x65, 0x6d, 0x61, 0x90
};
// deserialize it with BON8
json j = json::from_bon8(v);
// print the deserialized JSON value
std::cout << std::setw(2) << j << std::endl;
}
@@ -1,5 +0,0 @@
{
"compact": true,
"format": "BON8",
"schema": 0
}
-22
View File
@@ -1,22 +0,0 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using namespace nlohmann::literals;
int main()
{
// create a JSON value
json j = R"({"compact": true, "format": "BON8", "schema": 0})"_json;
// serialize it to BON8
std::vector<std::uint8_t> v = json::to_bon8(j);
// print the vector content
for (auto& byte : v)
{
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)byte << " ";
}
std::cout << std::endl;
}
-1
View File
@@ -1 +0,0 @@
0x89 0x63 0x6f 0x6d 0x70 0x61 0x63 0x74 0xf9 0x66 0x6f 0x72 0x6d 0x61 0x74 0xff 0x42 0x4f 0x4e 0x38 0xff 0x73 0x63 0x68 0x65 0x6d 0x61 0x90
@@ -1,159 +0,0 @@
# BON8
BON8 (Binary Object Notation 8) is a compact binary serialization format for JSON values. It uses the byte values that
cannot begin a UTF-8 character as type markers, so strings are stored as plain UTF-8 without a length prefix: a string
ends at the first byte that cannot continue it. Integers from -10 to 39, `true`, `false`, `null`, and the floating-point
values -1.0, 0.0, and 1.0 take a single byte, and arrays and objects with up to four elements need no terminator.
!!! abstract "References"
- [BON8 specification](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md)
- [Reference implementation](https://github.com/hikoworks/hikogui/blob/main/src/hikogui/codec/BON8.hpp) in HikoGUI
## Serialization
The library uses the following mapping from JSON values types to BON8 types according to the BON8 specification:
| JSON value type | value/range | BON8 type | first byte |
|-----------------|----------------------------------------------|-------------------------------|------------|
| null | `null` | null | 0xFA |
| boolean | `true` | true | 0xF9 |
| boolean | `false` | false | 0xF8 |
| number_integer | -9223372036854775808..-2147483649 | int64 | 0x8D |
| number_integer | -2147483648..-33818507 | int32 | 0x8C |
| number_integer | -33818506..-264075 | 4-byte negative integer | 0xF0..0xF7 |
| number_integer | -264074..-1931 | 3-byte negative integer | 0xE0..0xEF |
| number_integer | -1930..-11 | 2-byte negative integer | 0xC2..0xDF |
| number_integer | -10..-1 | 1-byte negative integer | 0xB8..0xC1 |
| number_integer | 0..39 | 1-byte positive integer | 0x90..0xB7 |
| number_integer | 40..3879 | 2-byte positive integer | 0xC2..0xDF |
| number_integer | 3880..528167 | 3-byte positive integer | 0xE0..0xEF |
| number_integer | 528168..67637031 | 4-byte positive integer | 0xF0..0xF7 |
| number_integer | 67637032..2147483647 | int32 | 0x8C |
| number_integer | 2147483648..9223372036854775807 | int64 | 0x8D |
| number_unsigned | 0..39 | 1-byte positive integer | 0x90..0xB7 |
| number_unsigned | 40..3879 | 2-byte positive integer | 0xC2..0xDF |
| number_unsigned | 3880..528167 | 3-byte positive integer | 0xE0..0xEF |
| number_unsigned | 528168..67637031 | 4-byte positive integer | 0xF0..0xF7 |
| number_unsigned | 67637032..2147483647 | int32 | 0x8C |
| number_unsigned | 2147483648..9223372036854775807 | int64 | 0x8D |
| number_float | `-1.0` | -1.0 | 0xFB |
| number_float | `0.0` | 0.0 | 0xFC |
| number_float | `1.0` | 1.0 | 0xFD |
| number_float | *any other value representable by a float* | binary32 | 0x8E |
| number_float | *any value NOT representable by a float* | binary64 | 0x8F |
| string | *empty* | end of string | 0xFF |
| string | *non-empty* | UTF-8 string | 0x00..0x7F, 0xC2..0xF4 |
| array | *size*: 0..4 | array with count | 0x80..0x84 |
| array | *size*: 5 or more | array (terminated by 0xFE) | 0x85 |
| object | *size*: 0..4 | object with count | 0x86..0x8A |
| object | *size*: 5 or more | object (terminated by 0xFE) | 0x8B |
| binary | *size*: 0..4 | array with count | 0x80..0x84 |
| binary | *size*: 5 or more | array (terminated by 0xFE) | 0x85 |
An integer that takes 2 to 4 bytes starts with a UTF-8 lead byte (0xC2..0xF7) that is followed by a byte that cannot
continue a UTF-8 character: 0x00..0x7F for positive and 0xC0..0xFF for negative integers. A string is terminated by
0xFF only if it is empty, if another string follows it, or if it is the last value of the message; otherwise, the first
byte of the next value ends it.
!!! success "Complete mapping"
Except for the values listed below, any JSON value can be converted to a BON8 value.
Any BON8 output created by `to_bon8` can be successfully parsed by `from_bon8`.
!!! warning "Unsupported values"
The following values can **not** be converted to a BON8 value:
- unsigned integers above 9223372036854775807, because BON8 has no unsigned 64-bit integer type
([out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407))
- strings that are not valid UTF-8, because the end of a string is determined from its encoding
([type_error.316](../../home/exceptions.md#jsonexceptiontype_error316))
!!! info "NaN/infinity handling"
`-0.0`, `Infinity`, and `-Infinity` are serialized as binary32 (type 0x8E, 5 bytes total). `NaN` is serialized as
the binary32 value 0x7F800001 that the specification recommends. This is in contrast to the
[dump](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
!!! warning "Binary values"
BON8 has no binary type. Binary values are serialized as arrays of integers (0..255), so they are read back as
arrays. The subtype is not serialized.
!!! info "Canonical representation"
The output follows the specification's canonical representation rules: every value uses the shortest encoding,
floating-point numbers use binary32 whenever that loses no precision, and object keys are sorted by their UTF-8
code units. There are two exceptions:
- Strings are not normalized to Unicode Normalization Form C (NFC).
- Object keys are written in the order of the object type, which is sorted for `json`, but not for
[`ordered_json`](../../api/ordered_json.md).
??? example
```cpp
--8<-- "examples/to_bon8.cpp"
```
Output:
```c
--8<-- "examples/to_bon8.output"
```
## Deserialization
The library maps BON8 types to JSON value types as follows:
| BON8 type | JSON value type | first byte |
|-------------------------------|-----------------|------------------------|
| UTF-8 string | string | 0x00..0x7F |
| array with count | array | 0x80..0x84 |
| array (terminated by 0xFE) | array | 0x85 |
| object with count | object | 0x86..0x8A |
| object (terminated by 0xFE) | object | 0x8B |
| int32 | number_unsigned or number_integer | 0x8C |
| int64 | number_unsigned or number_integer | 0x8D |
| binary32 | number_float | 0x8E |
| binary64 | number_float | 0x8F |
| 1-byte positive integer | number_unsigned | 0x90..0xB7 |
| 1-byte negative integer | number_integer | 0xB8..0xC1 |
| UTF-8 string | string | 0xC2..0xF4, followed by 0x80..0xBF |
| 2- to 4-byte positive integer | number_unsigned | 0xC2..0xF7, followed by 0x00..0x7F |
| 2- to 4-byte negative integer | number_integer | 0xC2..0xF7, followed by 0xC0..0xFF |
| false | `false` | 0xF8 |
| true | `true` | 0xF9 |
| null | `null` | 0xFA |
| -1.0 | number_float | 0xFB |
| 0.0 | number_float | 0xFC |
| 1.0 | number_float | 0xFD |
| empty string | string | 0xFF |
Non-negative integers are read as number_unsigned, negative integers as number_integer.
!!! info
Values that do not use the canonical representation, such as integers with a longer encoding than necessary,
arrays and objects with up to four elements that are terminated by 0xFE, unsorted object keys, or a 0xFF after a
string that would also end without it, are accepted. A second 0xFF is not a terminator but an empty string.
Strings must be valid UTF-8, and the last string of a message must be terminated by 0xFF.
!!! info
Any BON8 output created by `to_bon8` can be successfully parsed by `from_bon8`.
??? example
```cpp
--8<-- "examples/from_bon8.cpp"
```
Output:
```json
--8<-- "examples/from_bon8.output"
```
@@ -188,7 +188,7 @@ The library maps CBOR types to JSON value types as follows:
!!! warning "Tagged items"
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. They can be stored by passing `cbor_tag_handler_t::store` to function `from_cbor`. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
??? example
@@ -4,7 +4,6 @@ Though JSON is a ubiquitous data format, it is not a very compact format suitabl
a network. Hence, the library supports
- [BJData](bjdata.md) (Binary JData),
- [BON8](bon8.md) (Binary Object Notation 8),
- [BSON](bson.md) (Binary JSON),
- [CBOR](cbor.md) (Concise Binary Object Representation),
- [MessagePack](messagepack.md), and
@@ -19,7 +18,6 @@ to efficiently encode JSON values to byte vectors and to decode such vectors.
| Format | Serialization | Deserialization |
|-------------|-----------------------------------------------|----------------------------------------------|
| BJData | complete | complete |
| BON8 | incomplete: no unsigned integers above int64 | complete |
| BSON | incomplete: top-level value must be an object | incomplete, but all JSON types are supported |
| CBOR | complete | incomplete, but all JSON types are supported |
| MessagePack | complete | complete |
@@ -30,7 +28,6 @@ to efficiently encode JSON values to byte vectors and to decode such vectors.
| Format | Binary values | Binary subtypes |
|-------------|---------------|-----------------|
| BJData | not supported | not supported |
| BON8 | not supported | not supported |
| BSON | supported | supported |
| CBOR | supported | supported |
| MessagePack | supported | supported |
@@ -45,7 +42,6 @@ See [binary values](../binary_values.md) for more information.
| BJData | 53.2 % | 91.1 % | 78.1 % | 96.6 % |
| BJData (size) | 58.6 % | 92.1 % | 86.7 % | 97.4 % |
| BJData (size+type) | 58.6 % | 92.1 % | 86.5 % | 97.4 % |
| BON8 | 50.5 % | 83.8 % | 63.5 % | 87.5 % |
| BSON | 85.8 % | 95.2 % | 95.8 % | 106.7 % |
| CBOR | 50.5 % | 86.3 % | 68.4 % | 88.0 % |
| MessagePack | 50.5 % | 86.0 % | 68.5 % | 87.9 % |
@@ -65,8 +65,6 @@ specification:
- arrays with more than 4294967295 elements
- objects with more than 4294967295 elements
Serializing such a value throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412).
!!! info "NaN/infinity handling"
`NaN`, `Infinity`, and `-Infinity` are serialized as a MessagePack float 32 (type 0xCA, 5 bytes total),
@@ -187,41 +187,6 @@ as an array of uint8 values. The library implements this translation.
}
```
### BON8
[BON8](binary_formats/bon8.md) neither supports binary values nor subtypes. The library serializes binary values as an
array of integers.
??? example
Code:
```cpp
// create a binary value of subtype 42 (will be ignored in BON8)
json j;
j["binary"] = json::binary({0xCA, 0xFE, 0xBA, 0xBE}, 42);
// convert to BON8
auto v = json::to_bon8(j);
```
`v` is a `std::vector<std::uint8_t>` with the following 16 elements:
```c
0x87 // object with 1 member
0x62 0x69 0x6E 0x61 0x72 0x79 // "binary"
0x84 // array with 4 elements
0xC3 0x22 0xC3 0x56 0xC3 0x12 0xC3 0x16 // content (each byte as a 2-byte integer)
```
Note that the subtype is lost, and deserializing `v` would yield the following value:
```json
{
"binary": [202, 254, 186, 190]
}
```
### BSON
[BSON](binary_formats/bson.md) supports binary values and subtypes. If a subtype is given, it is used and added as an
@@ -43,23 +43,6 @@ json jPi = 3.14;
assert(jPi.get<TaskState>() == TS_INVALID );
```
## Maps with enum keys
By default, maps with enum keys, such as `std::map<TaskState, std::string>`, are stored as arrays of `[key, value]`
pairs, because JSON object keys must be strings. Define
[`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) before including the
library to store them as objects, with the keys converted by the enum's `to_json()` function:
```cpp
std::map<TaskState, std::string> m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
json j = m;
// default: [["stopped","aa"],["completed","bb"]]
// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS: {"completed":"bb","stopped":"aa"}
```
Either form can be read back, with or without the macro.
## Notes
Just as in [Arbitrary Type Conversions](arbitrary_types.md) above,
+2 -2
View File
@@ -35,8 +35,8 @@ C++ types, and finally serialize it again.
- [Serialization](serialization.md) — turn a value back into JSON text with [`dump`](../api/basic_json/dump.md),
including pretty-printing and handling of non-ASCII and invalid UTF-8.
- [Binary formats](binary_formats/index.md) — encode values more compactly as
[BJData](binary_formats/bjdata.md), [BON8](binary_formats/bon8.md), [BSON](binary_formats/bson.md),
[CBOR](binary_formats/cbor.md), [MessagePack](binary_formats/messagepack.md), or [UBJSON](binary_formats/ubjson.md).
[BJData](binary_formats/bjdata.md), [BSON](binary_formats/bson.md), [CBOR](binary_formats/cbor.md),
[MessagePack](binary_formats/messagepack.md), or [UBJSON](binary_formats/ubjson.md).
- [Binary values](binary_values.md) — store and exchange raw byte sequences.
## How values are stored and configured
-7
View File
@@ -167,13 +167,6 @@ behavior is deprecated and switched off (`0`) by default.
See [full documentation of `JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md).
## `JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`
When defined to `1`, maps with enum keys (e.g., `std::map<E, T>`) are stored as objects, using the enum's conversion for
the keys, instead of arrays of `[key, value]` pairs. It is switched off (`0`) by default.
See [full documentation of `JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md).
## `JSON_USE_SIMDUTF`
When defined, UTF-8 validation of JSON strings read from contiguous byte input is delegated to the
-2
View File
@@ -20,8 +20,6 @@ The complete default namespace name is derived as follows:
`_bics`.
- [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) defined non-zero appends `_psp`.
- [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) defined non-zero appends `_snul`.
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) defined non-zero
appends `_ekmo`.
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
below.
+1 -1
View File
@@ -117,7 +117,7 @@ For the [{fmt}](https://github.com/fmtlib/fmt) library, the library ships a
## Serializing to other formats
Besides JSON text, a value can also be serialized to the more compact [binary formats](binary_formats/index.md)
(BJData, BON8, BSON, CBOR, MessagePack, UBJSON).
(BJData, BSON, CBOR, MessagePack, UBJSON).
## See also
@@ -547,7 +547,7 @@ Grisu2 algorithm, which produces the shortest representation that round-trips. O
### Required for the binary formats
`NumberFloatType` must be `#!cpp float` or `#!cpp double`. The writers for
[CBOR, MessagePack, UBJSON, BJData, BON8, and BSON](../binary_formats/index.md) map a floating-point value onto an IEEE 754
[CBOR, MessagePack, UBJSON, BJData, and BSON](../binary_formats/index.md) map a floating-point value onto an IEEE 754
binary32 or binary64 field and have no encoding for `#!cpp long double`.
### Compatible types
+4 -26
View File
@@ -589,9 +589,6 @@ During implicit or explicit value conversion, the JSON type must be compatible w
[json.exception.type_error.302] type must be string, but is object
```
This exception is also thrown with [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md)
if a key of a map with enum keys is not converted to a string, for instance, because the enum is stored as an integer.
### json.exception.type_error.303
To retrieve a reference to a value stored in a `basic_json` object with `get_ref`, the type of the reference must match the value type. For instance, for a JSON array, the `ReferenceType` must be `array_t &`.
@@ -778,19 +775,6 @@ The dynamic type of the object cannot be represented in the requested serializat
Encapsulate the JSON value in an object. That is, instead of serializing `#!json true`, serialize `#!json {"value": true}`
### json.exception.type_error.318
With [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md), a map with enum
keys is stored as an object. This exception is thrown if two of its keys are converted to the same string, so one of the
entries would be lost. This happens, for instance, if [`NLOHMANN_JSON_SERIALIZE_ENUM`](../api/macros/nlohmann_json_serialize_enum.md)
does not list an enumerator and it is therefore converted like the first listed one.
!!! failure "Example message"
```
[json.exception.type_error.318] duplicate object key 'red'
```
## Out of range
This exception is thrown in case a library function is called on an input parameter that exceeds the expected range, for instance, in the case of array indices or nonexisting object keys.
@@ -948,25 +932,19 @@ A JSON Patch `add` operation cannot be applied because the target location's par
### json.exception.out_of_range.412
BSON stores the length of documents, arrays, strings, and binary values in a signed 32-bit integer, and MessagePack
stores the length of strings, binary values, arrays, and objects in at most an unsigned 32-bit integer. This exception
is thrown when a value is too large to be described by such a length field.
BSON stores the length of documents, arrays, strings, and binary values in a signed 32-bit integer. This exception is thrown when a value is too large to be described by such a length field.
!!! failure "Example messages"
!!! failure "Example message"
```
BSON length 2147483661 exceeds maximum of 2147483647
```
```
MessagePack length 4294967296 exceeds maximum of 4294967295
```
!!! note
This exception was added in version 3.13.0. Before that, the BSON length was silently truncated, and
This exception was added in version 3.13.0. Before that, the length was silently truncated, and
[`to_bson`](../api/basic_json/to_bson.md) produced documents with negative length prefixes that
[`from_bson`](../api/basic_json/from_bson.md) rejected; [`to_msgpack`](../api/basic_json/to_msgpack.md) wrote such
a value without any length, producing output that could not be read back.
[`from_bson`](../api/basic_json/from_bson.md) rejected.
### json.exception.out_of_range.413
-1
View File
@@ -5,7 +5,6 @@
*[ASCII]: American Standard Code for Information Interchange
*[BDFL]: Benevolent Dictator for Life
*[BJData]: Binary JData
*[BON8]: Binary Object Notation 8
*[BSON]: Binary JSON
*[CBOR]: Concise Binary Object Representation
*[CC0]: Creative Commons Zero
+1 -5
View File
@@ -63,7 +63,6 @@ nav:
- Binary Formats:
- features/binary_formats/index.md
- features/binary_formats/bjdata.md
- features/binary_formats/bon8.md
- features/binary_formats/bson.md
- features/binary_formats/cbor.md
- features/binary_formats/messagepack.md
@@ -143,7 +142,6 @@ nav:
- 'flatten': api/basic_json/flatten.md
- 'format_as': api/basic_json/format_as.md
- 'from_bjdata': api/basic_json/from_bjdata.md
- 'from_bon8': api/basic_json/from_bon8.md
- 'from_bson': api/basic_json/from_bson.md
- 'from_cbor': api/basic_json/from_cbor.md
- 'from_msgpack': api/basic_json/from_msgpack.md
@@ -215,7 +213,6 @@ nav:
- 'swap': api/basic_json/swap.md
- 'std::swap&lt;basic_json&gt;': api/basic_json/std_swap.md
- 'to_bjdata': api/basic_json/to_bjdata.md
- 'to_bon8': api/basic_json/to_bon8.md
- 'to_bson': api/basic_json/to_bson.md
- 'to_cbor': api/basic_json/to_cbor.md
- 'to_msgpack': api/basic_json/to_msgpack.md
@@ -303,7 +300,6 @@ nav:
- 'JSON_USE_GLOBAL_UDLS': api/macros/json_use_global_udls.md
- 'JSON_USE_IMPLICIT_CONVERSIONS': api/macros/json_use_implicit_conversions.md
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
- 'JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS': api/macros/json_use_objects_for_enum_keyed_maps.md
- 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
- 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
- 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md
@@ -416,7 +412,7 @@ plugins:
markdown_description: >
JSON for Modern C++ is a C++11 header-only library implementing a JSON
value type with an STL-like API, JSON Pointer/Patch, CBOR/MessagePack/
BSON/UBJSON/BJData/BON8 binary format support, and a SAX-style parser interface.
BSON/UBJSON/BJData binary format support, and a SAX-style parser interface.
sections:
Home:
- index.md
+4 -15
View File
@@ -46,10 +46,6 @@
#define JSON_STRICT_NUL_HANDLING 0
#endif
#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
#endif
#if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else
@@ -86,20 +82,14 @@
#define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
#else
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f) json_abi ## a ## b ## c ## d ## e ## f
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -108,8 +98,7 @@
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
@@ -550,40 +550,11 @@ auto from_json(BasicJsonType&& j, TupleRelated&& t)
return from_json_tuple_impl(std::forward<BasicJsonType>(j), std::forward<TupleRelated>(t), priority_tag<3> {});
}
// read a map with enum keys from an object, using the enum's own from_json for
// the keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); this is the form written
// with JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
template<typename BasicJsonType, typename Map>
inline bool from_json_enum_keyed_object(const BasicJsonType& j, Map& m, std::true_type /*key is enum*/)
{
if (!j.is_object())
{
return false;
}
m.clear();
for (const auto& p : *j.template get_ptr<const typename BasicJsonType::object_t*>())
{
m.emplace(BasicJsonType(p.first).template get<typename Map::key_type>(), p.second.template get<typename Map::mapped_type>());
}
return true;
}
template<typename BasicJsonType, typename Map>
inline bool from_json_enum_keyed_object(const BasicJsonType& /*j*/, Map& /*m*/, std::false_type /*key is enum*/)
{
return false;
}
template < typename BasicJsonType, typename Key, typename Value, typename Compare, typename Allocator,
typename = enable_if_t < !std::is_constructible <
typename BasicJsonType::string_t, Key >::value >>
inline void from_json(const BasicJsonType& j, std::map<Key, Value, Compare, Allocator>& m)
{
// NOLINTNEXTLINE(modernize-type-traits) we use C++11
if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
{
return;
}
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
@@ -604,11 +575,6 @@ template < typename BasicJsonType, typename Key, typename Value, typename Hash,
typename BasicJsonType::string_t, Key >::value >>
inline void from_json(const BasicJsonType& j, std::unordered_map<Key, Value, Hash, KeyEqual, Allocator>& m)
{
// NOLINTNEXTLINE(modernize-type-traits) we use C++11
if (from_json_enum_keyed_object(j, m, std::is_enum<Key> {}))
{
return;
}
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
+45 -56
View File
@@ -23,7 +23,6 @@
#include <valarray> // valarray
#include <vector> // vector
#include <nlohmann/detail/exceptions.hpp>
#include <nlohmann/detail/iterators/iteration_proxy.hpp>
#include <nlohmann/detail/meta/cpp_future.hpp>
#include <nlohmann/detail/meta/std_fs.hpp>
@@ -43,6 +42,10 @@ namespace detail
* j.m_data.m_value.destroy(j.m_data.m_type) to avoid a memory leak in case j contains an
* allocated value (e.g., a string). See bug issue
* https://github.com/nlohmann/json/issues/2865 for more information.
*
* A value that has to be allocated is created before the old one is destroyed:
* were it the other way around, an exception while creating the new value would
* leave j with the type of the new value, but the pointer to the destroyed old one.
*/
template<value_t> struct external_constructor;
@@ -66,18 +69,20 @@ struct external_constructor<value_t::string>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, const typename BasicJsonType::string_t& s)
{
const typename BasicJsonType::json_value value(s);
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::string;
j.m_data.m_value = s;
j.m_data.m_value = value;
j.assert_invariant();
}
template<typename BasicJsonType>
static void construct(BasicJsonType& j, typename BasicJsonType::string_t&& s)
{
const typename BasicJsonType::json_value value(std::move(s));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::string;
j.m_data.m_value = std::move(s);
j.m_data.m_value = value;
j.assert_invariant();
}
@@ -86,9 +91,10 @@ struct external_constructor<value_t::string>
int > = 0 >
static void construct(BasicJsonType& j, const CompatibleStringType& str)
{
auto* created = j.template create<typename BasicJsonType::string_t>(str);
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::string;
j.m_data.m_value.string = j.template create<typename BasicJsonType::string_t>(str);
j.m_data.m_value.string = created;
j.assert_invariant();
}
};
@@ -99,18 +105,20 @@ struct external_constructor<value_t::binary>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, const typename BasicJsonType::binary_t& b)
{
const typename BasicJsonType::json_value value(b);
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::binary;
j.m_data.m_value = typename BasicJsonType::binary_t(b);
j.m_data.m_value = value;
j.assert_invariant();
}
template<typename BasicJsonType>
static void construct(BasicJsonType& j, typename BasicJsonType::binary_t&& b)
{
const typename BasicJsonType::json_value value(std::move(b));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::binary;
j.m_data.m_value = typename BasicJsonType::binary_t(std::move(b));
j.m_data.m_value = value;
j.assert_invariant();
}
};
@@ -160,9 +168,10 @@ struct external_constructor<value_t::array>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, const typename BasicJsonType::array_t& arr)
{
const typename BasicJsonType::json_value value(arr);
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = arr;
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
@@ -170,9 +179,10 @@ struct external_constructor<value_t::array>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, typename BasicJsonType::array_t&& arr)
{
const typename BasicJsonType::json_value value(std::move(arr));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = std::move(arr);
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
@@ -188,9 +198,10 @@ struct external_constructor<value_t::array>
using std::begin;
using std::end;
auto* created = j.template create<typename BasicJsonType::array_t>(begin(arr), end(arr));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value.array = j.template create<typename BasicJsonType::array_t>(begin(arr), end(arr));
j.m_data.m_value.array = created;
j.set_parents();
j.assert_invariant();
}
@@ -198,15 +209,17 @@ struct external_constructor<value_t::array>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, const std::vector<bool>& arr)
{
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = value_t::array;
j.m_data.m_value.array->reserve(arr.size());
typename BasicJsonType::array_t elements;
elements.reserve(arr.size());
for (const bool x : arr)
{
j.m_data.m_value.array->push_back(x);
j.set_parent(j.m_data.m_value.array->back());
elements.push_back(x);
}
const typename BasicJsonType::json_value value(std::move(elements));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
@@ -214,11 +227,12 @@ struct external_constructor<value_t::array>
enable_if_t<std::is_convertible<T, BasicJsonType>::value, int> = 0>
static void construct(BasicJsonType& j, const std::valarray<T>& arr)
{
typename BasicJsonType::array_t elements(arr.size());
std::copy(std::begin(arr), std::end(arr), elements.begin());
const typename BasicJsonType::json_value value(std::move(elements));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = value_t::array;
j.m_data.m_value.array->resize(arr.size());
std::copy(std::begin(arr), std::end(arr), j.m_data.m_value.array->begin());
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
@@ -230,14 +244,16 @@ struct external_constructor<value_t::array>
enable_if_t<is_compatible_range_view<std::remove_cvref_t<CompatibleArrayType>>::value, int> = 0>
static void construct(BasicJsonType& j, CompatibleArrayType && arr)
{
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = value_t::array;
typename BasicJsonType::array_t elements;
for (auto&& x : std::forward<CompatibleArrayType>(arr))
{
j.m_data.m_value.array->push_back(x);
j.set_parent(j.m_data.m_value.array->back());
elements.push_back(x);
}
const typename BasicJsonType::json_value value(std::move(elements));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::array;
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
#endif
@@ -249,9 +265,10 @@ struct external_constructor<value_t::object>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, const typename BasicJsonType::object_t& obj)
{
const typename BasicJsonType::json_value value(obj);
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::object;
j.m_data.m_value = obj;
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
@@ -259,9 +276,10 @@ struct external_constructor<value_t::object>
template<typename BasicJsonType>
static void construct(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
{
const typename BasicJsonType::json_value value(std::move(obj));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::object;
j.m_data.m_value = std::move(obj);
j.m_data.m_value = value;
j.set_parents();
j.assert_invariant();
}
@@ -273,9 +291,10 @@ struct external_constructor<value_t::object>
using std::begin;
using std::end;
auto* created = j.template create<typename BasicJsonType::object_t>(begin(obj), end(obj));
j.m_data.m_value.destroy(j.m_data.m_type);
j.m_data.m_type = value_t::object;
j.m_data.m_value.object = j.template create<typename BasicJsonType::object_t>(begin(obj), end(obj));
j.m_data.m_value.object = created;
j.set_parents();
j.assert_invariant();
}
@@ -382,9 +401,6 @@ template < typename BasicJsonType, typename CompatibleArrayType,
!is_basic_json<CompatibleArrayType>::value
#if JSON_HAS_RANGES && !defined(__MINGW32__)
&& !is_compatible_range_view<CompatibleArrayType>::value
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
&& !is_enum_keyed_map<CompatibleArrayType>::value
#endif
,
int > = 0 >
@@ -439,33 +455,6 @@ inline void to_json(BasicJsonType& j, const CompatibleObjectType& obj)
external_constructor<value_t::object>::construct(j, obj);
}
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
// store a map with enum keys as an object, using the enum's own to_json for the
// keys (e.g., from NLOHMANN_JSON_SERIALIZE_ENUM); without the macro, such maps
// are stored as arrays of [key, value] pairs
template < typename BasicJsonType, typename EnumKeyedMap,
enable_if_t < is_enum_keyed_map<EnumKeyedMap>::value&& !is_basic_json<EnumKeyedMap>::value, int > = 0 >
inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
{
typename BasicJsonType::object_t obj;
for (const auto& p : map)
{
BasicJsonType key = p.first;
if (JSON_HEDLEY_UNLIKELY(!key.is_string()))
{
JSON_THROW(type_error::create(302, concat("type must be string, but is ", key.type_name()), &key));
}
auto& key_string = *key.template get_ptr<typename BasicJsonType::string_t*>();
if (JSON_HEDLEY_UNLIKELY(!obj.emplace(key_string, BasicJsonType(p.second)).second))
{
JSON_THROW(type_error::create(318, concat("duplicate object key '", key_string, "'"), &key));
}
}
external_constructor<value_t::object>::construct(j, std::move(obj));
}
#endif
template<typename BasicJsonType>
inline void to_json(BasicJsonType& j, typename BasicJsonType::object_t&& obj)
{
+6 -634
View File
@@ -28,7 +28,6 @@
#include <nlohmann/detail/input/input_adapters.hpp>
#include <nlohmann/detail/input/json_sax.hpp>
#include <nlohmann/detail/input/lexer.hpp>
#include <nlohmann/detail/input/string_scan.hpp>
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/is_sax.hpp>
#include <nlohmann/detail/meta/type_traits.hpp>
@@ -45,7 +44,7 @@ enum class cbor_tag_handler_t
{
error, ///< throw a parse_error exception in case of a tag
ignore, ///< ignore tags
store ///< store tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored
store ///< store tags as binary type
};
/*!
@@ -85,7 +84,7 @@ JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 2
///////////////////
/*!
@brief deserialization of BJData, BON8, BSON, CBOR, MessagePack, and UBJSON values
@brief deserialization of CBOR, MessagePack, and UBJSON values
*/
template<typename BasicJsonType, typename InputAdapterType, typename SAX = json_sax_dom_parser<BasicJsonType, InputAdapterType>>
class binary_reader
@@ -99,11 +98,6 @@ class binary_reader
using char_type = typename InputAdapterType::char_type;
using char_int_type = typename char_traits<char_type>::int_type;
/// whether the input is a contiguous block of bytes that can be inspected
/// and consumed in bulk (as in the lexer); used by @ref get_bon8_string_bulk
static constexpr bool bulk_scan =
input_adapter_supports_bulk_scan<InputAdapterType>(is_detected<detect_supports_bulk_scan, InputAdapterType> {});
public:
/*!
@brief create a binary reader
@@ -138,7 +132,6 @@ class binary_reader
{
sax = sax_;
container_stack.clear();
bon8_pushback_size = 0;
bool result = false;
switch (format)
@@ -160,10 +153,6 @@ class binary_reader
result = parse_ubjson_internal();
break;
case input_format_t::bon8:
result = parse_bon8_internal();
break;
case input_format_t::json: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
@@ -176,11 +165,6 @@ class binary_reader
{
get_ignore_noop();
}
else if (input_format == input_format_t::bon8)
{
// a string that ends a container hands back the byte after it
get_bon8();
}
else
{
get();
@@ -412,11 +396,6 @@ class binary_reader
*/
bool get_bson_cstr(string_t& result)
{
if (get_bson_cstr_bulk(result, std::integral_constant<bool, bulk_scan> {}))
{
return true;
}
auto out = std::back_inserter(result);
while (true)
{
@@ -433,46 +412,6 @@ class binary_reader
}
}
/*!
@brief read a C-style string from contiguous input in one step
@param[in,out] result the string to append to
@return whether the string was read; if the input has no \x00-byte, nothing
is read, and @ref get_bson_cstr reports the end of the input
*/
bool get_bson_cstr_bulk(string_t& result, std::true_type /*bulk*/)
{
const std::size_t remaining = ia.bulk_remaining();
if (remaining == 0)
{
return false;
}
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
// a plain loop rather than std::memchr: most keys are short (array
// indices are keys, too), and the call would cost more than it saves
std::size_t length = 0;
while (length < remaining && data[length] != 0x00)
{
++length;
}
if (length == remaining)
{
return false;
}
result.append(reinterpret_cast<const typename string_t::value_type*>(data), length);
// consume the string and its \x00-byte, as the byte-wise path does
ia.bulk_skip(length + 1);
chars_read += length + 1;
current = 0x00;
return true;
}
/// input that is not contiguous: C-style strings are read byte by byte
bool get_bson_cstr_bulk(string_t& /*result*/, std::false_type /*bulk*/) const noexcept
{
return false;
}
/*!
@brief Parses a zero-terminated string of length @a len from the BSON
input.
@@ -653,18 +592,14 @@ class binary_reader
input (true) or whether the last read character should
be considered instead (false)
@param[in] tag_handler how CBOR tags should be treated
@param[out] tag_pending whether a tag was parsed and its value follows
@param[out] item_read whether the tagged value's initial byte is already in current
@return whether a valid CBOR value was passed to the SAX parser
*/
bool parse_cbor_value(const bool get_char,
const cbor_tag_handler_t tag_handler,
bool& tag_pending,
bool& item_read)
bool& tag_pending)
{
tag_pending = false;
item_read = false;
switch (get_char ? get() : current)
{
@@ -1086,17 +1021,7 @@ class binary_reader
}
}
get();
// a byte string (the heads accepted by get_cbor_binary) keeps the tag as subtype
if ((current >= 0x40 && current <= 0x5B) || current == 0x5F)
{
return get_cbor_binary(b) && sax->binary(b);
}
// not a byte string: the tagged value, whose first byte
// was just read, is read by the caller like for ignore
tag_pending = true;
item_read = true;
return true;
return get_cbor_binary(b) && sax->binary(b);
}
default: // LCOV_EXCL_LINE
@@ -1578,14 +1503,13 @@ class binary_reader
// a tag is not a value of its own: read on until the tagged value
bool tag_pending = false;
bool item_read = false;
do
{
if (JSON_HEDLEY_UNLIKELY(!parse_cbor_value(fetch, tag_handler, tag_pending, item_read)))
if (JSON_HEDLEY_UNLIKELY(!parse_cbor_value(fetch, tag_handler, tag_pending)))
{
return false;
}
fetch = !item_read;
fetch = true;
}
while (tag_pending);
@@ -3260,549 +3184,6 @@ class binary_reader
}
}
//////////
// BON8 //
//////////
/*!
@brief get the next byte of a BON8 value
A BON8 string has no length prefix and no mandatory terminator: it ends at
the first byte that cannot continue it, which is already the first byte (or,
for an integer that begins with a UTF-8 lead byte, the first two bytes) of
whatever follows. The string reader hands those bytes back with
@ref unget_bon8, and every BON8 read goes through this function so that
they are seen again.
@return character read from the input
*/
char_int_type get_bon8()
{
if (bon8_pushback_size != 0)
{
++chars_read;
return current = bon8_pushback[--bon8_pushback_size];
}
return get();
}
/*!
@brief hand a byte back so that the next @ref get_bon8 returns it again
@param[in] c the byte to hand back; bytes handed back are returned in
reverse order
*/
void unget_bon8(const char_int_type c)
{
// At most two bytes are ever handed back: a byte is only handed back
// right after it was read with get_bon8(), and the only place that
// hands back two bytes (a lead byte and the byte after it) read both
// of them in a row, which emptied the buffer first. This is an
// invariant of the reader rather than a property of the input, so
// an assertion suffices (the fuzzers are built with assertions).
JSON_ASSERT(bon8_pushback_size < bon8_pushback.size());
bon8_pushback[bon8_pushback_size++] = c;
--chars_read;
}
/*!
@param[in] c a byte
@return whether @a c is a UTF-8 continuation byte (0x80..0xBF)
*/
static constexpr bool is_bon8_continuation(const char_int_type c) noexcept
{
return 0x80 <= c && c <= 0xBF;
}
/*!
@brief report a parse error at the last read byte
@param[in] detail a detailed error message
@param[in] context further context information
@return false
*/
bool bon8_error(const std::string& detail, const char* context)
{
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr));
}
/*!
@brief read a BON8 value and everything nested inside it
Reads values until the one that was begun here is complete, resuming the
enclosing container after each element, so that the nesting depth of the
input costs heap rather than native stack (see #5104).
@return whether reading the value succeeded
*/
bool parse_bon8_internal()
{
// the key currently being read; hoisted out of the loop so that its
// capacity is reused across elements and across nesting levels
string_t key;
while (true)
{
if (!container_stack.empty())
{
// a copy, not a reference: it must stay valid across the
// pop_back() below, which destroys the container_stack element
// it would otherwise alias
const container_frame top = container_stack.back();
bool at_end = false;
if (top.remaining != npos)
{
// counted container (0x80..0x84, 0x86..0x8A): it ends once
// its elements have been read
at_end = (top.remaining == 0);
if (!at_end)
{
// claim the element about to be read
--container_stack.back().remaining;
}
}
else
{
// container 0x85 or 0x8B: it ends at an end-of-container
// marker (0xFE); any other byte begins the next element
at_end = (get_bon8() == 0xFE);
if (!at_end)
{
unget_bon8(current);
}
}
if (at_end)
{
container_stack.pop_back();
if (JSON_HEDLEY_UNLIKELY(top.is_object ? !sax->end_object() : !sax->end_array()))
{
return false;
}
// the value begun here is complete once its container is
if (container_stack.empty())
{
return true;
}
continue;
}
if (top.is_object)
{
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bon8_key(key) || !sax->key(key)))
{
return false;
}
}
}
if (JSON_HEDLEY_UNLIKELY(!parse_bon8_value()))
{
return false;
}
// a value that opened a container left it on the stack; one that
// did not, and that was not inside a container, was the whole value
if (container_stack.empty())
{
return true;
}
}
}
/*!
@brief read one BON8 value
Reads a single value and passes it to the SAX parser. A value that begins
a container is not read to its end: the container is opened with
@ref enter_container and its elements are read by
@ref parse_bon8_internal, so that nesting does not consume native stack.
@return whether reading the value succeeded
*/
bool parse_bon8_value()
{
const auto byte = get_bon8();
if (byte == char_traits<char_type>::eof())
{
return unexpect_eof(input_format_t::bon8, "value");
}
// string: ASCII character
if (byte <= 0x7F)
{
string_t s;
unget_bon8(byte);
return get_bon8_string(s) && sax->string(s);
}
// array with 0..4 elements
if (byte <= 0x84)
{
return enter_array(static_cast<std::size_t>(byte - 0x80));
}
// array terminated by 0xFE
if (byte == 0x85)
{
return enter_array(npos);
}
// object with 0..4 members
if (byte <= 0x8A)
{
return enter_object(static_cast<std::size_t>(byte - 0x86));
}
switch (byte)
{
case 0x8B: // object terminated by 0xFE
return enter_object(npos);
case 0x8C: // int32
{
std::int32_t number{};
return get_number(input_format_t::bon8, number) && emit_bon8_integer(number);
}
case 0x8D: // int64
{
std::int64_t number{};
return get_number(input_format_t::bon8, number) && emit_bon8_integer(number);
}
case 0x8E: // binary32
{
float number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0x8F: // binary64
{
double number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xF8:
return sax->boolean(false);
case 0xF9:
return sax->boolean(true);
case 0xFA:
return sax->null();
case 0xFB:
return sax->number_float(static_cast<number_float_t>(-1.0), "");
case 0xFC:
return sax->number_float(static_cast<number_float_t>(0.0), "");
case 0xFD:
return sax->number_float(static_cast<number_float_t>(1.0), "");
case 0xFF: // empty string
{
string_t s;
return sax->string(s);
}
default:
break;
}
// integer 0..39
if (byte <= 0xB7)
{
return sax->number_unsigned(static_cast<number_unsigned_t>(byte - 0x90));
}
// integer -1..-10
if (byte <= 0xC1)
{
return sax->number_integer(-1 - static_cast<number_integer_t>(byte - 0xB8));
}
// 0xC2..0xF7: a UTF-8 lead byte begins a string if a continuation
// byte follows and an integer otherwise
if (byte <= 0xF7)
{
const auto second = get_bon8();
if (is_bon8_continuation(second))
{
string_t s;
unget_bon8(second);
unget_bon8(byte);
return get_bon8_string(s) && sax->string(s);
}
return get_bon8_integer(byte, second);
}
// 0xFE: end of container where a value is expected
return bon8_error("invalid byte", "value");
}
/*!
@brief pass an integer to the SAX parser
Non-negative integers are passed as unsigned, negative integers as signed
numbers, like the other binary formats do.
@param[in] number the integer
@return whether the SAX parser accepted the value
*/
bool emit_bon8_integer(const std::int64_t number)
{
if (number >= 0)
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return sax->number_integer(static_cast<number_integer_t>(number));
}
/*!
@brief read an integer encoded in 2..4 bytes
The first byte is a UTF-8 lead byte (0xC2..0xF7) that is followed by a
byte that is not a continuation byte: 0x00..0x7F for positive and
0xC0..0xFF for negative integers. The lead byte's low bits and the second
byte's low 7 (positive) or 6 (negative) bits are the most significant bits
of the value; 3- and 4-byte integers add one or two full bytes. Each range
starts where the shorter one ends, so no value has two encodings of the
same length.
@param[in] lead the first byte (0xC2..0xF7)
@param[in] second the second byte
@return whether reading the integer succeeded
*/
bool get_bon8_integer(const char_int_type lead, const char_int_type second)
{
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number")))
{
return false;
}
const bool negative = second >= 0xC0;
auto value = static_cast<std::int64_t>(negative ? (second & 0x3F) : second);
std::int64_t offset = 0;
int extra_bytes = 0;
if (lead <= 0xDF)
{
value |= static_cast<std::int64_t>(lead - 0xC2) << (negative ? 6 : 7);
offset = negative ? 11 : 40;
}
else if (lead <= 0xEF)
{
value |= static_cast<std::int64_t>(lead & 0x0F) << (negative ? 6 : 7);
offset = negative ? 1931 : 3880;
extra_bytes = 1;
}
else
{
value |= static_cast<std::int64_t>(lead & 0x07) << (negative ? 6 : 7);
offset = negative ? 264075 : 528168;
extra_bytes = 2;
}
for (int i = 0; i < extra_bytes; ++i)
{
if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits<char_type>::eof()))
{
return unexpect_eof(input_format_t::bon8, "number");
}
value = (value << 8) | static_cast<std::int64_t>(current);
}
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
}
/*!
@brief read an object key
A key must be a string, so its first byte must be an ASCII character, a
UTF-8 lead byte followed by a continuation byte, or 0xFF (empty string).
@param[out] result the key
@return whether reading the key succeeded
*/
bool get_bon8_key(string_t& result)
{
const auto byte = get_bon8();
if (byte == char_traits<char_type>::eof())
{
return unexpect_eof(input_format_t::bon8, "key");
}
if (byte == 0xFF)
{
return true;
}
if (byte <= 0x7F)
{
unget_bon8(byte);
return get_bon8_string(result);
}
if (0xC2 <= byte && byte <= 0xF7)
{
const auto second = get_bon8();
unget_bon8(second);
if (is_bon8_continuation(second))
{
unget_bon8(byte);
return get_bon8_string(result);
}
// an integer: report its first byte rather than the one after it
current = byte;
}
return bon8_error("expected a string; last byte", "key");
}
/*!
@brief append the run of valid UTF-8 at the read position to a string
For contiguous input, the ASCII characters and complete well-formed UTF-8
sequences at the read position are appended to @a result in one step. The
byte that stops the run (an end-of-string marker, the first byte of the
next value, or an ill-formed byte) is left for @ref get_bon8_string, so
that strings end and errors are reported exactly as without this step.
@param[in,out] result the string to append to
*/
void get_bon8_string_bulk(string_t& result, std::true_type /*bulk*/)
{
// bytes handed back must be read through get_bon8() first
if (bon8_pushback_size != 0)
{
return;
}
const std::size_t remaining = ia.bulk_remaining();
if (remaining == 0)
{
return;
}
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t length = valid_utf8_prefix(data, remaining);
if (length != 0)
{
result.append(reinterpret_cast<const typename string_t::value_type*>(data), length);
ia.bulk_skip(length);
chars_read += length;
}
}
/// input that is not contiguous: strings are read byte by byte
void get_bon8_string_bulk(string_t& /*result*/, std::false_type /*bulk*/) const noexcept {}
/*!
@brief read a string
Reads UTF-8 characters until an end-of-string marker (0xFF), which is
consumed, or a byte that cannot continue the string, which is handed back
to be read as the start of the next value. The string must be valid UTF-8,
and it must not end at the end of the input: the last string of a message
is always terminated by 0xFF.
@param[out] result the string
@return whether reading the string succeeded
*/
bool get_bon8_string(string_t& result)
{
while (true)
{
get_bon8_string_bulk(result, std::integral_constant<bool, bulk_scan> {});
const auto byte = get_bon8();
if (byte == char_traits<char_type>::eof())
{
return unexpect_eof(input_format_t::bon8, "string");
}
// end of string
if (byte == 0xFF)
{
return true;
}
// ASCII character
if (byte <= 0x7F)
{
result.push_back(static_cast<typename string_t::value_type>(byte));
continue;
}
// a byte that cannot begin a character ends the string and begins
// the next value
if (byte < 0xC2 || byte > 0xF7)
{
unget_bon8(byte);
return true;
}
// a lead byte ends the string if no continuation byte follows: it
// is then the first byte of an integer
const auto second = get_bon8();
if (!is_bon8_continuation(second))
{
unget_bon8(second);
unget_bon8(byte);
return true;
}
// the valid range of the second byte excludes overlong forms,
// surrogates, and code points above U+10FFFF
// (RFC 3629, section 4)
int continuation_bytes = 0;
bool valid_second = true;
if (byte <= 0xDF)
{
continuation_bytes = 1;
}
else if (byte <= 0xEF)
{
continuation_bytes = 2;
valid_second = (byte != 0xE0 || second >= 0xA0) && (byte != 0xED || second <= 0x9F);
}
else
{
continuation_bytes = 3;
valid_second = byte <= 0xF4 && (byte != 0xF0 || second >= 0x90) && (byte != 0xF4 || second <= 0x8F);
}
if (JSON_HEDLEY_UNLIKELY(!valid_second))
{
return bon8_error("invalid UTF-8 byte", "string");
}
result.push_back(static_cast<typename string_t::value_type>(byte));
result.push_back(static_cast<typename string_t::value_type>(second));
for (int i = 1; i < continuation_bytes; ++i)
{
if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits<char_type>::eof()))
{
return unexpect_eof(input_format_t::bon8, "string");
}
if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current)))
{
return bon8_error("invalid UTF-8 byte", "string");
}
result.push_back(static_cast<typename string_t::value_type>(current));
}
}
}
///////////////////////
// Utility functions //
///////////////////////
@@ -4101,10 +3482,6 @@ class binary_reader
error_msg += "BJData";
break;
case input_format_t::bon8:
error_msg += "BON8";
break;
case input_format_t::json: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
@@ -4137,11 +3514,6 @@ class binary_reader
/// the containers that have been opened and not closed yet; see @ref container_frame
std::vector<container_frame> container_stack{};
/// BON8: bytes read past the end of a string, returned again by @ref get_bon8
std::array<char_int_type, 2> bon8_pushback{{}};
/// BON8: number of bytes in @ref bon8_pushback
std::size_t bon8_pushback_size = 0;
// excluded markers in bjdata optimized type
#define JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_ \
make_array<char_int_type>('F', 'H', 'N', 'S', 'T', 'Z', '[', '{')
@@ -34,7 +34,7 @@ namespace detail
{
/// the supported input formats
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata, bon8 };
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata };
////////////////////
// input adapters //
@@ -201,43 +201,6 @@ inline std::size_t validate_one_utf8(const unsigned char* data, std::size_t avai
return 0; // invalid, incomplete, or must be diagnosed by the byte path
}
// Return the length of the longest prefix of [data, data+n) that consists of
// ASCII characters and complete well-formed UTF-8 sequences; n if all of it is
// valid UTF-8. Unlike scalar_string_bulk_run(), quotes, escapes, and control
// characters are ordinary characters here. ASCII is skipped 8 bytes at a time.
inline std::size_t valid_utf8_prefix(const unsigned char* data, std::size_t n) noexcept
{
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t pos = 0;
while (pos < n)
{
if (pos + 8 <= n)
{
std::uint64_t word = 0;
std::memcpy(&word, data + pos, sizeof(word));
if ((word & high) == 0)
{
pos += 8;
continue;
}
}
if (data[pos] < 0x80u)
{
++pos;
continue;
}
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
{
break; // ill-formed or truncated
}
pos += seq;
}
return pos;
}
// Scalar (C++11) computation of the bulk run length: the number of leading
// bytes in [data, data+n) that are ordinary ASCII or complete well-formed UTF-8
// sequences, stopping before the first byte that needs individual handling (the
@@ -46,7 +46,6 @@
#undef JSON_BRACE_INIT_COPY_SEMANTICS
#undef JSON_PRECISE_STREAM_POSITION
#undef JSON_STRICT_NUL_HANDLING
#undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#include <nlohmann/thirdparty/hedley/hedley_undef.hpp>
@@ -400,30 +400,6 @@ template<typename BasicJsonType, typename CompatibleObjectType>
struct is_compatible_object_type
: is_compatible_object_type_impl<BasicJsonType, CompatibleObjectType> {};
template<typename T>
using insert_result_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()));
template<typename T>
using insert_result_second_t = decltype(std::declval<T&>().insert(std::declval<const value_type_t<T>&>()).second);
// a map-like type (std::map, std::unordered_map, ...) whose keys are enums; see
// JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
template<typename T, typename = void>
struct is_enum_keyed_map : std::false_type {};
template<typename T>
struct is_enum_keyed_map <
T, enable_if_t < is_detected<mapped_type_t, T>::value&&
is_detected<key_type_t, T>::value >>
{
// maps with non-unique keys (std::multimap, std::unordered_multimap, ...)
// are excluded, because an object cannot hold duplicate keys; they are
// detected by insert() returning an iterator instead of a pair<iterator, bool>
// NOLINTNEXTLINE(modernize-type-traits) we use C++11
static constexpr bool value = std::is_enum<typename T::key_type>::value &&
!(is_detected<insert_result_t, T>::value && !is_detected<insert_result_second_t, T>::value);
};
template<typename BasicJsonType, typename ConstructibleObjectType,
typename = void>
struct is_constructible_object_type_impl : std::false_type {};
+234 -492
View File
@@ -25,7 +25,6 @@
#endif
#include <nlohmann/detail/input/binary_reader.hpp>
#include <nlohmann/detail/input/string_scan.hpp>
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/output/output_adapters.hpp>
#include <nlohmann/detail/string_concat.hpp>
@@ -77,7 +76,7 @@ std::size_t binary_reserve_hint(const BasicJsonType& j)
}
/*!
@brief serialization to BJData, BON8, BSON, CBOR, MessagePack, and UBJSON values
@brief serialization to CBOR and MessagePack values
*/
template<typename BasicJsonType, typename CharType, typename OutputSinkType = output_adapter_sink<CharType>>
class binary_writer
@@ -169,20 +168,92 @@ class binary_writer
if (j.m_data.m_value.number_integer >= 0)
{
// CBOR does not differentiate between positive signed
// integers and unsigned integers
write_cbor_head(0x00, static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
// integers and unsigned integers. Therefore, we used the
// code from the value_t::number_unsigned case here.
if (j.m_data.m_value.number_integer <= 0x17)
{
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0x18));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0x19));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0x1A));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_integer));
}
else
{
oa.write_character(to_char_type(0x1B));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
}
}
else
{
// a negative integer n is encoded as -1 - n
write_cbor_head(0x20, static_cast<std::uint64_t>(-1 - j.m_data.m_value.number_integer));
// The conversions below encode the sign in the first
// byte, and the value is converted to a positive number.
const auto positive_number = -1 - j.m_data.m_value.number_integer;
if (j.m_data.m_value.number_integer >= -24)
{
write_number(static_cast<std::uint8_t>(0x20 + positive_number));
}
else if (positive_number <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0x38));
write_number(static_cast<std::uint8_t>(positive_number));
}
else if (positive_number <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0x39));
write_number(static_cast<std::uint16_t>(positive_number));
}
else if (positive_number <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0x3A));
write_number(static_cast<std::uint32_t>(positive_number));
}
else
{
oa.write_character(to_char_type(0x3B));
write_number(static_cast<std::uint64_t>(positive_number));
}
}
break;
}
case value_t::number_unsigned:
{
write_cbor_head(0x00, j.m_data.m_value.number_unsigned);
if (j.m_data.m_value.number_unsigned <= 0x17)
{
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0x18));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0x19));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0x1A));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_unsigned));
}
else
{
oa.write_character(to_char_type(0x1B));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
}
break;
}
@@ -212,7 +283,33 @@ class binary_writer
case value_t::string:
{
// step 1: write control byte and the string length
write_cbor_head(0x60, j.m_data.m_value.string->size());
const auto N = j.m_data.m_value.string->size();
if (N <= 0x17)
{
write_number(static_cast<std::uint8_t>(0x60 + N));
}
else if (N <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0x78));
write_number(static_cast<std::uint8_t>(N));
}
else if (N <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0x79));
write_number(static_cast<std::uint16_t>(N));
}
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0x7A));
write_number(static_cast<std::uint32_t>(N));
}
// LCOV_EXCL_START
else if (N <= (std::numeric_limits<std::uint64_t>::max)())
{
oa.write_character(to_char_type(0x7B));
write_number(static_cast<std::uint64_t>(N));
}
// LCOV_EXCL_STOP
// step 2: write the string
oa.write_characters(
@@ -224,7 +321,33 @@ class binary_writer
case value_t::array:
{
// step 1: write control byte and the array size
write_cbor_head(0x80, j.m_data.m_value.array->size());
const auto N = j.m_data.m_value.array->size();
if (N <= 0x17)
{
write_number(static_cast<std::uint8_t>(0x80 + N));
}
else if (N <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0x98));
write_number(static_cast<std::uint8_t>(N));
}
else if (N <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0x99));
write_number(static_cast<std::uint16_t>(N));
}
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0x9A));
write_number(static_cast<std::uint32_t>(N));
}
// LCOV_EXCL_START
else if (N <= (std::numeric_limits<std::uint64_t>::max)())
{
oa.write_character(to_char_type(0x9B));
write_number(static_cast<std::uint64_t>(N));
}
// LCOV_EXCL_STOP
// step 2: write each element
for (const auto& el : *j.m_data.m_value.array)
@@ -253,7 +376,7 @@ class binary_writer
write_number(static_cast<std::uint8_t>(0xda));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.binary->subtype()));
}
else
else if (j.m_data.m_value.binary->subtype() <= (std::numeric_limits<std::uint64_t>::max)())
{
write_number(static_cast<std::uint8_t>(0xdb));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.binary->subtype()));
@@ -262,7 +385,32 @@ class binary_writer
// step 1: write control byte and the binary array size
const auto N = j.m_data.m_value.binary->size();
write_cbor_head(0x40, N);
if (N <= 0x17)
{
write_number(static_cast<std::uint8_t>(0x40 + N));
}
else if (N <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0x58));
write_number(static_cast<std::uint8_t>(N));
}
else if (N <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0x59));
write_number(static_cast<std::uint16_t>(N));
}
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0x5A));
write_number(static_cast<std::uint32_t>(N));
}
// LCOV_EXCL_START
else if (N <= (std::numeric_limits<std::uint64_t>::max)())
{
oa.write_character(to_char_type(0x5B));
write_number(static_cast<std::uint64_t>(N));
}
// LCOV_EXCL_STOP
// step 2: write each element
oa.write_characters(
@@ -275,7 +423,33 @@ class binary_writer
case value_t::object:
{
// step 1: write control byte and the object size
write_cbor_head(0xA0, j.m_data.m_value.object->size());
const auto N = j.m_data.m_value.object->size();
if (N <= 0x17)
{
write_number(static_cast<std::uint8_t>(0xA0 + N));
}
else if (N <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(0xB8));
write_number(static_cast<std::uint8_t>(N));
}
else if (N <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(0xB9));
write_number(static_cast<std::uint16_t>(N));
}
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(0xBA));
write_number(static_cast<std::uint32_t>(N));
}
// LCOV_EXCL_START
else if (N <= (std::numeric_limits<std::uint64_t>::max)())
{
oa.write_character(to_char_type(0xBB));
write_number(static_cast<std::uint64_t>(N));
}
// LCOV_EXCL_STOP
// step 2: write each element
for (const auto& el : *j.m_data.m_value.object)
@@ -292,23 +466,6 @@ class binary_writer
}
}
/*!
@brief check that @a length fits into the 32 bits that MessagePack stores
the length of a string, binary value, array, or object in
@return the length as an unsigned 32-bit integer
@throw out_of_range.412 if @a length exceeds the range of std::uint32_t
*/
static std::uint32_t to_msgpack_length(const std::size_t length, const BasicJsonType& j)
{
if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::uint32_t>(length)))
{
JSON_THROW(out_of_range::create(412, concat("MessagePack length ", std::to_string(length), " exceeds maximum of ", std::to_string((std::numeric_limits<std::uint32_t>::max)())), &j));
}
static_cast<void>(j);
return static_cast<std::uint32_t>(length);
}
/*!
@param[in] j JSON value to serialize
*/
@@ -360,7 +517,7 @@ class binary_writer
oa.write_character(to_char_type(0xCE));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_integer));
}
else
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint64_t>::max)())
{
// uint 64
oa.write_character(to_char_type(0xCF));
@@ -395,7 +552,8 @@ class binary_writer
oa.write_character(to_char_type(0xD2));
write_number(static_cast<std::int32_t>(j.m_data.m_value.number_integer));
}
else
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int64_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int64_t>::max)())
{
// int 64
oa.write_character(to_char_type(0xD3));
@@ -430,7 +588,7 @@ class binary_writer
oa.write_character(to_char_type(0xCE));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_integer));
}
else
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint64_t>::max)())
{
// uint 64
oa.write_character(to_char_type(0xCF));
@@ -448,7 +606,7 @@ class binary_writer
case value_t::string:
{
// step 1: write control byte and the string length
const auto N = to_msgpack_length(j.m_data.m_value.string->size(), j);
const auto N = j.m_data.m_value.string->size();
if (N <= 31)
{
// fixstr
@@ -466,7 +624,7 @@ class binary_writer
oa.write_character(to_char_type(0xDA));
write_number(static_cast<std::uint16_t>(N));
}
else
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
// str 32
oa.write_character(to_char_type(0xDB));
@@ -483,7 +641,7 @@ class binary_writer
case value_t::array:
{
// step 1: write control byte and the array size
const auto N = to_msgpack_length(j.m_data.m_value.array->size(), j);
const auto N = j.m_data.m_value.array->size();
if (N <= 15)
{
// fixarray
@@ -495,7 +653,7 @@ class binary_writer
oa.write_character(to_char_type(0xDC));
write_number(static_cast<std::uint16_t>(N));
}
else
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
// array 32
oa.write_character(to_char_type(0xDD));
@@ -517,7 +675,7 @@ class binary_writer
const bool use_ext = j.m_data.m_value.binary->has_subtype();
// step 1: write control byte and the byte string length
const auto N = to_msgpack_length(j.m_data.m_value.binary->size(), j);
const auto N = j.m_data.m_value.binary->size();
if (N <= (std::numeric_limits<std::uint8_t>::max)())
{
std::uint8_t output_type{};
@@ -569,7 +727,7 @@ class binary_writer
oa.write_character(to_char_type(output_type));
write_number(static_cast<std::uint16_t>(N));
}
else
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
const std::uint8_t output_type = use_ext
? 0xC9 // ext 32
@@ -601,7 +759,7 @@ class binary_writer
case value_t::object:
{
// step 1: write control byte and the object size
const auto N = to_msgpack_length(j.m_data.m_value.object->size(), j);
const auto N = j.m_data.m_value.object->size();
if (N <= 15)
{
// fixmap
@@ -613,7 +771,7 @@ class binary_writer
oa.write_character(to_char_type(0xDE));
write_number(static_cast<std::uint16_t>(N));
}
else
else if (N <= (std::numeric_limits<std::uint32_t>::max)())
{
// map 32
oa.write_character(to_char_type(0xDF));
@@ -874,21 +1032,6 @@ class binary_writer
}
}
/*!
@param[in] j JSON value to serialize
*/
void write_bon8(const BasicJsonType& j)
{
bool string_open = false;
write_bon8_value(j, string_open);
// the last string of a message must be terminated
if (string_open)
{
oa.write_character(to_char_type(0xFF));
}
}
private:
//////////
// BSON //
@@ -1383,46 +1526,6 @@ class binary_writer
// CBOR //
//////////
/*!
@brief write the head of a CBOR data item
The head is the major type in the upper three bits of the first byte and
an argument - an unsigned integer, the length of a string, the number of
elements of a container - in the shortest of its encodings: in the lower
five bits of the first byte itself if it is at most 23, otherwise in the
1, 2, 4, or 8 bytes that follow (RFC 8949, section 3).
@param[in] major_type the major type, shifted into the upper three bits
@param[in] argument the argument of the data item
*/
void write_cbor_head(const std::uint8_t major_type, const std::uint64_t argument)
{
if (argument <= 0x17)
{
write_number(static_cast<std::uint8_t>(major_type + argument));
}
else if (argument <= (std::numeric_limits<std::uint8_t>::max)())
{
oa.write_character(to_char_type(static_cast<std::uint8_t>(major_type + 0x18)));
write_number(static_cast<std::uint8_t>(argument));
}
else if (argument <= (std::numeric_limits<std::uint16_t>::max)())
{
oa.write_character(to_char_type(static_cast<std::uint8_t>(major_type + 0x19)));
write_number(static_cast<std::uint16_t>(argument));
}
else if (argument <= (std::numeric_limits<std::uint32_t>::max)())
{
oa.write_character(to_char_type(static_cast<std::uint8_t>(major_type + 0x1A)));
write_number(static_cast<std::uint32_t>(argument));
}
else
{
oa.write_character(to_char_type(static_cast<std::uint8_t>(major_type + 0x1B)));
write_number(argument);
}
}
static constexpr CharType get_cbor_float_prefix(float /*unused*/)
{
return to_char_type(0xFA); // Single-Precision Float
@@ -1447,28 +1550,6 @@ class binary_writer
return to_char_type(0xCB); // float 64
}
/// @return the BON8 type marker for binary32 (float) or binary64 (double)
template<typename FloatType>
static constexpr CharType get_bon8_float_prefix()
{
return to_char_type(std::is_same<FloatType, float>::value ? 0x8E : 0x8F);
}
/// @return the type marker for a FloatType value in @a format (CBOR, MessagePack, or BON8)
template<typename FloatType>
static CharType get_compact_float_prefix(const detail::input_format_t format)
{
if (format == detail::input_format_t::cbor)
{
return get_cbor_float_prefix(FloatType{});
}
if (format == detail::input_format_t::bon8)
{
return get_bon8_float_prefix<FloatType>();
}
return get_msgpack_float_prefix(FloatType{});
}
////////////
// UBJSON //
////////////
@@ -1550,7 +1631,7 @@ class binary_writer
}
write_number(static_cast<std::int64_t>(n), use_bjdata);
}
else if (use_bjdata)
else if (use_bjdata && n <= (std::numeric_limits<uint64_t>::max)())
{
if (add_prefix)
{
@@ -1630,59 +1711,30 @@ class binary_writer
}
write_number(static_cast<uint32_t>(n), use_bjdata);
}
else if ((std::numeric_limits<std::int64_t>::min)() <= n && n <= (std::numeric_limits<std::int64_t>::max)())
{
if (add_prefix)
{
oa.write_character(to_char_type('L')); // int64
}
write_number(static_cast<std::int64_t>(n), use_bjdata);
}
// LCOV_EXCL_START
else
{
// every value of an integer type of at most 64 bits fits into an
// int64; only a wider type needs a range check
write_ubjson_int64_or_high_precision(n, add_prefix, use_bjdata,
std::integral_constant < bool, std::numeric_limits<NumberType>::digits <= std::numeric_limits<std::int64_t>::digits > {});
if (add_prefix)
{
oa.write_character(to_char_type('H')); // high-precision number
}
const auto number = BasicJsonType(n).dump();
write_number_with_ubjson_prefix(number.size(), true, use_bjdata);
for (std::size_t i = 0; i < number.size(); ++i)
{
oa.write_character(to_char_type(static_cast<std::uint8_t>(number[i])));
}
}
}
template<typename NumberType>
void write_ubjson_int64_or_high_precision(const NumberType n, const bool add_prefix, const bool use_bjdata, std::true_type /*fits_int64*/)
{
if (add_prefix)
{
oa.write_character(to_char_type('L')); // int64
}
write_number(static_cast<std::int64_t>(n), use_bjdata);
}
template<typename NumberType>
void write_ubjson_int64_or_high_precision(const NumberType n, const bool add_prefix, const bool use_bjdata, std::false_type /*fits_int64*/)
{
if ((std::numeric_limits<std::int64_t>::min)() <= n && n <= (std::numeric_limits<std::int64_t>::max)())
{
write_ubjson_int64_or_high_precision(n, add_prefix, use_bjdata, std::true_type {});
return;
}
if (add_prefix)
{
oa.write_character(to_char_type('H')); // high-precision number
}
const auto number = BasicJsonType(n).dump();
write_number_with_ubjson_prefix(number.size(), true, use_bjdata);
for (std::size_t i = 0; i < number.size(); ++i)
{
oa.write_character(to_char_type(static_cast<std::uint8_t>(number[i])));
}
}
template<typename NumberType>
static constexpr CharType ubjson_int64_or_high_precision_prefix(const NumberType /*n*/, std::true_type /*fits_int64*/) noexcept
{
return 'L';
}
template<typename NumberType>
static CharType ubjson_int64_or_high_precision_prefix(const NumberType n, std::false_type /*fits_int64*/) noexcept
{
// anything outside of the range of an int64 is treated as a
// high-precision number
return ((std::numeric_limits<std::int64_t>::min)() <= n && n <= (std::numeric_limits<std::int64_t>::max)()) ? 'L' : 'H';
// LCOV_EXCL_STOP
}
/*!
@@ -1724,10 +1776,12 @@ class binary_writer
{
return 'm';
}
// every value of an integer type of at most 64 bits fits into
// an int64; only a wider type needs a range check
return ubjson_int64_or_high_precision_prefix(j.m_data.m_value.number_integer,
std::integral_constant < bool, std::numeric_limits<typename BasicJsonType::number_integer_t>::digits <= std::numeric_limits<std::int64_t>::digits > {});
if ((std::numeric_limits<std::int64_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::int64_t>::max)())
{
return 'L';
}
// anything else is treated as a high-precision number
return 'H'; // LCOV_EXCL_LINE
}
case value_t::number_unsigned:
@@ -1760,12 +1814,12 @@ class binary_writer
{
return 'L';
}
if (use_bjdata)
if (use_bjdata && j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint64_t>::max)())
{
return 'M';
}
// anything else is treated as a high-precision number
return 'H';
return 'H'; // LCOV_EXCL_LINE
}
case value_t::number_float:
@@ -2088,322 +2142,6 @@ class binary_writer
return false;
}
//////////
// BON8 //
//////////
/*!
@brief write a BON8 value
A string is written without length or terminator: it ends at the first
byte that cannot continue it, which is the first byte of any non-string
value and of the end-of-container marker 0xFE. It only needs an explicit
end-of-string marker (0xFF) when it is empty, when another string follows,
or when it is the last thing in the message.
@param[in] j JSON value to serialize
@param[in,out] string_open whether the output ends with a non-empty
string that has not been terminated with 0xFF
*/
void write_bon8_value(const BasicJsonType& j, bool& string_open)
{
switch (j.type())
{
case value_t::null:
{
write_bon8_marker(0xFA, string_open);
break;
}
case value_t::boolean:
{
write_bon8_marker(j.m_data.m_value.boolean ? 0xF9 : 0xF8, string_open);
break;
}
case value_t::number_unsigned:
{
if (j.m_data.m_value.number_unsigned > static_cast<typename BasicJsonType::number_unsigned_t>((std::numeric_limits<std::int64_t>::max)()))
{
JSON_THROW(out_of_range::create(407, concat("integer number ", std::to_string(j.m_data.m_value.number_unsigned), " cannot be represented by BON8 as it does not fit int64"), &j));
}
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_unsigned));
string_open = false;
break;
}
case value_t::number_integer:
{
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
string_open = false;
break;
}
case value_t::number_float:
{
write_bon8_float(j.m_data.m_value.number_float);
string_open = false;
break;
}
case value_t::string:
{
write_bon8_string(*j.m_data.m_value.string, string_open, j);
break;
}
case value_t::array:
{
const auto N = j.m_data.m_value.array->size();
// 0x80..0x84: array with 0..4 elements; 0x85: array ended by 0xFE
write_bon8_marker(static_cast<std::uint8_t>(N <= 4 ? 0x80 + N : 0x85), string_open);
for (const auto& el : *j.m_data.m_value.array)
{
write_bon8_value(el, string_open);
}
if (N > 4)
{
write_bon8_marker(0xFE, string_open);
}
break;
}
case value_t::object:
{
const auto N = j.m_data.m_value.object->size();
// 0x86..0x8A: object with 0..4 members; 0x8B: object ended by 0xFE
write_bon8_marker(static_cast<std::uint8_t>(N <= 4 ? 0x86 + N : 0x8B), string_open);
for (const auto& el : *j.m_data.m_value.object)
{
write_bon8_string(el.first, string_open, j);
write_bon8_value(el.second, string_open);
}
if (N > 4)
{
write_bon8_marker(0xFE, string_open);
}
break;
}
case value_t::binary:
{
// BON8 has no binary type: write the bytes as an array of
// integers, like UBJSON and BJData do
const auto N = j.m_data.m_value.binary->size();
write_bon8_marker(static_cast<std::uint8_t>(N <= 4 ? 0x80 + N : 0x85), string_open);
for (std::size_t i = 0; i < N; ++i)
{
// the cast is needed for binary types whose value type
// is not an integer (e.g., std::byte)
write_bon8_integer(static_cast<std::uint8_t>(j.m_data.m_value.binary->data()[i]));
}
if (N > 4)
{
write_bon8_marker(0xFE, string_open);
}
break;
}
case value_t::discarded:
default:
break;
}
}
/*!
@brief write a single byte that is not part of a string
@param[in] marker the byte to write
@param[out] string_open set to false, because the output no longer ends
with a string; see @ref write_bon8_value
*/
void write_bon8_marker(const std::uint8_t marker, bool& string_open)
{
oa.write_character(to_char_type(marker));
string_open = false;
}
/*!
@brief write a string
@param[in] s the string to write
@param[in,out] string_open see @ref write_bon8_value
@param[in] context the value the string belongs to (for diagnostics)
@throw type_error.316 if @a s is not valid UTF-8, because the end of a
string is determined from its encoding
*/
void write_bon8_string(const string_t& s, bool& string_open, const BasicJsonType& context)
{
check_bon8_utf8(s, context);
// a string that follows another string terminates it
if (string_open)
{
oa.write_character(to_char_type(0xFF));
}
if (s.empty())
{
// the empty string is just the end-of-string marker
oa.write_character(to_char_type(0xFF));
string_open = false;
}
else
{
oa.write_characters(reinterpret_cast<const CharType*>(s.data()), s.size());
string_open = true;
}
}
/*!
@brief check that a string is valid UTF-8 (RFC 3629)
@param[in] s the string to check
@param[in] context the value the string belongs to (for diagnostics)
@throw type_error.316 if @a s is not valid UTF-8; the message names the
first byte of the first invalid or incomplete sequence
*/
static void check_bon8_utf8(const string_t& s, const BasicJsonType& context)
{
static_cast<void>(context); // only used when exceptions are enabled
const auto* data = reinterpret_cast<const unsigned char*>(s.data());
const std::size_t valid = valid_utf8_prefix(data, s.size());
if (JSON_HEDLEY_UNLIKELY(valid != s.size()))
{
JSON_THROW(type_error::create(316, concat("invalid UTF-8 byte at index ", std::to_string(valid), ": 0x", hex_byte(data[valid])), &context));
}
}
/// @return a byte as two uppercase hexadecimal digits
static std::string hex_byte(const std::uint8_t byte)
{
std::string result = "00";
constexpr const char* nibble_to_hex = "0123456789ABCDEF";
result[0] = nibble_to_hex[byte / 16];
result[1] = nibble_to_hex[byte % 16];
return result;
}
/*!
@brief write an integer in the shortest encoding
Integers from -10 to 39 take one byte. Up to -33818506 and 67637031, an
integer takes 2 to 4 bytes that begin with a UTF-8 lead byte (0xC2..0xF7)
followed by a byte that is not a continuation byte: 0x00..0x7F for
positive and 0xC0..0xFF for negative integers. Each range starts where the
shorter one ends. Larger integers are written as int32 (0x8C) or int64
(0x8D) in big-endian byte order.
@param[in] value the integer to write
*/
void write_bon8_integer(std::int64_t value)
{
if (value < (std::numeric_limits<std::int32_t>::min)() || value > (std::numeric_limits<std::int32_t>::max)())
{
oa.write_character(to_char_type(0x8D));
write_number(value);
}
else if (value < -33818506 || value > 67637031)
{
oa.write_character(to_char_type(0x8C));
write_number(static_cast<std::int32_t>(value));
}
else if (value <= -264075)
{
value = -(value + 264075);
write_bon8_bytes(0xF0 + ((value >> 22) & 0x07), 0xC0 + ((value >> 16) & 0x3F), value >> 8, value);
}
else if (value <= -1931)
{
value = -(value + 1931);
write_bon8_bytes(0xE0 + ((value >> 14) & 0x0F), 0xC0 + ((value >> 8) & 0x3F), value);
}
else if (value <= -11)
{
value = -(value + 11);
write_bon8_bytes(0xC2 + ((value >> 6) & 0x1F), 0xC0 + (value & 0x3F));
}
else if (value <= -1)
{
write_bon8_bytes(0xB8 - (value + 1));
}
else if (value <= 39)
{
write_bon8_bytes(0x90 + value);
}
else if (value <= 3879)
{
value -= 40;
write_bon8_bytes(0xC2 + ((value >> 7) & 0x1F), value & 0x7F);
}
else if (value <= 528167)
{
value -= 3880;
write_bon8_bytes(0xE0 + ((value >> 15) & 0x0F), (value >> 8) & 0x7F, value);
}
else
{
value -= 528168;
write_bon8_bytes(0xF0 + ((value >> 23) & 0x07), (value >> 16) & 0x7F, value >> 8, value);
}
}
/// write the low byte of each argument
template<typename... Bytes>
void write_bon8_bytes(const Bytes... bytes)
{
const std::array<CharType, sizeof...(Bytes)> buffer{{to_char_type(static_cast<std::uint8_t>(bytes & 0xFF))...}};
oa.write_characters(buffer.data(), buffer.size());
}
/*!
@brief write a floating-point number
-1.0, +0.0, and 1.0 take one byte. Other numbers are written as binary32
(0x8E) if that loses no precision, and as binary64 (0x8F) otherwise; -0.0,
infinities, and NaN are always written as binary32, NaN as 0x7F800001.
@param[in] n the number to write
*/
void write_bon8_float(const number_float_t n)
{
#ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_PUSH
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal")
#endif
if (n == static_cast<number_float_t>(-1))
{
oa.write_character(to_char_type(0xFB));
}
else if (n == static_cast<number_float_t>(0) && !std::signbit(n))
{
oa.write_character(to_char_type(0xFC));
}
else if (n == static_cast<number_float_t>(1))
{
oa.write_character(to_char_type(0xFD));
}
else if (std::isnan(n))
{
write_bon8_bytes(0x8E, 0x7F, 0x80, 0x00, 0x01);
}
else
{
write_compact_float(n, detail::input_format_t::bon8);
}
#ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP
#endif
}
///////////////////////
// Utility functions //
///////////////////////
@@ -2538,12 +2276,16 @@ class binary_writer
static_cast<double>(n) <= static_cast<double>((std::numeric_limits<float>::max)()) &&
static_cast<double>(static_cast<float>(n)) == static_cast<double>(n))))
{
oa.write_character(get_compact_float_prefix<float>(format));
oa.write_character(format == detail::input_format_t::cbor
? get_cbor_float_prefix(static_cast<float>(n))
: get_msgpack_float_prefix(static_cast<float>(n)));
write_number(static_cast<float>(n));
}
else
{
oa.write_character(get_compact_float_prefix<number_float_t>(format));
oa.write_character(format == detail::input_format_t::cbor
? get_cbor_float_prefix(n)
: get_msgpack_float_prefix(n));
write_number(n);
}
#ifdef __GNUC__
+29 -94
View File
@@ -1492,28 +1492,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
compare_keys(current.lhs_object_it->first, current.rhs_object_it->first,
std::integral_constant<bool, Ordered> {});
left = &(current.lhs_object_it->second);
right = &(current.rhs_object_it->second);
if (key_result != compare_result::equal)
{
// An object type without a fixed order of its entries -
// std::unordered_map, say - may enumerate two equal
// objects differently, and its operator== does not care.
// Equality then finds the entry by its key; an ordering,
// or an object type that compares its entries in
// sequence (ordered_map), is decided by the key itself.
const auto* rhs_object = current.rhs_value->m_data.m_value.object;
const auto found = (!Ordered && !detail::is_ordered_map<object_t>::value)
? rhs_object->find(current.lhs_object_it->first)
: rhs_object->cend();
if (found == rhs_object->cend())
{
return key_result;
}
right = &(found->second);
return key_result;
}
left = &(current.lhs_object_it->second);
right = &(current.rhs_object_it->second);
++current.lhs_object_it;
++current.rhs_object_it;
}
@@ -1702,8 +1687,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (is_an_object)
{
// the initializer list is a list of pairs -> create an object
m_data.m_type = value_t::object;
m_data.m_value = value_t::object;
m_data.m_type = value_t::object;
for (auto& element_ref : init)
{
@@ -1725,8 +1710,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
#endif
// the initializer list describes an array -> create an array
m_data.m_type = value_t::array;
m_data.m_value.array = create<array_t>(init.begin(), init.end());
m_data.m_type = value_t::array;
}
set_parents();
@@ -1739,8 +1724,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static basic_json binary(const typename binary_t::container_type& init)
{
auto res = basic_json();
res.m_data.m_type = value_t::binary;
res.m_data.m_value = init;
res.m_data.m_type = value_t::binary;
return res;
}
@@ -1750,8 +1735,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static basic_json binary(const typename binary_t::container_type& init, typename binary_t::subtype_type subtype)
{
auto res = basic_json();
res.m_data.m_type = value_t::binary;
res.m_data.m_value = binary_t(init, subtype);
res.m_data.m_type = value_t::binary;
return res;
}
@@ -1761,8 +1746,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static basic_json binary(typename binary_t::container_type&& init)
{
auto res = basic_json();
res.m_data.m_type = value_t::binary;
res.m_data.m_value = std::move(init);
res.m_data.m_type = value_t::binary;
return res;
}
@@ -1772,8 +1757,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static basic_json binary(typename binary_t::container_type&& init, typename binary_t::subtype_type subtype)
{
auto res = basic_json();
res.m_data.m_type = value_t::binary;
res.m_data.m_value = binary_t(std::move(init), subtype);
res.m_data.m_type = value_t::binary;
return res;
}
@@ -2172,6 +2157,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// value access //
//////////////////
/// get a boolean (explicit)
boolean_t get_impl(boolean_t* /*unused*/) const
{
if (JSON_HEDLEY_LIKELY(is_boolean()))
{
return m_data.m_value.boolean;
}
JSON_THROW(type_error::create(302, detail::concat("type must be boolean, but is ", type_name()), this));
}
/// get a pointer to the value (object)
object_t* get_impl_ptr(object_t* /*unused*/) noexcept
{
@@ -2819,8 +2815,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// implicitly convert a null value to an empty array
if (is_null())
{
m_data.m_type = value_t::array;
m_data.m_value.array = create<array_t>();
m_data.m_type = value_t::array;
assert_invariant();
}
@@ -2879,8 +2875,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// implicitly convert a null value to an empty object
if (is_null())
{
m_data.m_type = value_t::object;
m_data.m_value.object = create<object_t>();
m_data.m_type = value_t::object;
assert_invariant();
}
@@ -2932,8 +2928,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// implicitly convert a null value to an empty object
if (is_null())
{
m_data.m_type = value_t::object;
m_data.m_value.object = create<object_t>();
m_data.m_type = value_t::object;
assert_invariant();
}
@@ -3868,8 +3864,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// transform a null object into an array
if (is_null())
{
m_data.m_type = value_t::array;
m_data.m_value = value_t::array;
m_data.m_type = value_t::array;
assert_invariant();
}
@@ -3901,8 +3897,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// transform a null object into an array
if (is_null())
{
m_data.m_type = value_t::array;
m_data.m_value = value_t::array;
m_data.m_type = value_t::array;
assert_invariant();
}
@@ -3933,8 +3929,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// transform a null object into an object
if (is_null())
{
m_data.m_type = value_t::object;
m_data.m_value = value_t::object;
m_data.m_type = value_t::object;
assert_invariant();
}
@@ -3989,8 +3985,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// transform a null object into an array
if (is_null())
{
m_data.m_type = value_t::array;
m_data.m_value = value_t::array;
m_data.m_type = value_t::array;
assert_invariant();
}
@@ -4014,8 +4010,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// transform a null object into an object
if (is_null())
{
m_data.m_type = value_t::object;
m_data.m_value = value_t::object;
m_data.m_type = value_t::object;
assert_invariant();
}
@@ -4196,8 +4192,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// implicitly convert a null value to an empty object
if (is_null())
{
m_data.m_type = value_t::object;
m_data.m_value.object = create<object_t>();
m_data.m_type = value_t::object;
assert_invariant();
}
@@ -5290,30 +5286,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
binary_writer<char>(o).write_bson(j);
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static std::vector<std::uint8_t> to_bon8(const basic_json& j)
{
std::vector<std::uint8_t> result;
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_bon8(j);
return result;
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o)
{
binary_writer<std::uint8_t>(o).write_bon8(j);
}
/// @brief create a BON8 serialization of a given JSON value
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
static void to_bon8(const basic_json& j, detail::output_adapter<char> o)
{
binary_writer<char>(o).write_bon8(j);
}
/// @brief create a JSON value from an input in CBOR format
/// @sa https://json.nlohmann.me/api/basic_json/from_cbor/
template<typename InputType>
@@ -5547,43 +5519,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
return result;
}
/// @brief create a JSON value from an input in BON8 format
/// @sa https://json.nlohmann.me/api/basic_json/from_bon8/
template<typename InputType>
JSON_HEDLEY_WARN_UNUSED_RESULT
static basic_json from_bon8(InputType&& i,
const bool strict = true,
const bool allow_exceptions = true)
{
basic_json result;
auto ia = detail::input_adapter(std::forward<InputType>(i));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bon8).sax_parse(input_format_t::bon8, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BON8 format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
/// @sa https://json.nlohmann.me/api/basic_json/from_bon8/
template<typename IteratorType, typename SentinelType = IteratorType,
detail::enable_if_t<detail::can_compare_ne<IteratorType, SentinelType>::value, int> = 0>
JSON_HEDLEY_WARN_UNUSED_RESULT
static basic_json from_bon8(IteratorType first, SentinelType last,
const bool strict = true,
const bool allow_exceptions = true)
{
basic_json result;
auto ia = detail::input_adapter(std::move(first), std::move(last));
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bon8).sax_parse(input_format_t::bon8, &sdp, strict)) // cppcheck-suppress[accessMoved]
{
result = value_t::discarded;
}
return result;
}
/// @brief create a JSON value from an input in BSON format
/// @sa https://json.nlohmann.me/api/basic_json/from_bson/
template<typename InputType>
-3840
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+4 -15
View File
@@ -64,10 +64,6 @@
#define JSON_STRICT_NUL_HANDLING 0
#endif
#ifndef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 0
#endif
#if JSON_DIAGNOSTICS
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
#else
@@ -104,20 +100,14 @@
#define NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS _ekmo
#else
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g) json_abi ## a ## b ## c ## d ## e ## f ## g
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f) json_abi ## a ## b ## c ## d ## e ## f
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -126,8 +116,7 @@
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS, \
NLOHMANN_JSON_ABI_TAG_BRACE_INIT_COPY_SEMANTICS, \
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
+1 -1
View File
@@ -112,7 +112,7 @@ endif()
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# avoid stack overflow, see https://github.com/nlohmann/json/issues/2955
json_test_set_test_options("test-bon8;test-cbor;test-msgpack;test-ubjson;test-bjdata;test-binary_formats" LINK_OPTIONS /STACK:4000000)
json_test_set_test_options("test-cbor;test-msgpack;test-ubjson;test-bjdata;test-binary_formats" LINK_OPTIONS /STACK:4000000)
endif()
# disable exceptions for test-disabled_exceptions
+1 -4
View File
@@ -10,7 +10,7 @@ CXXFLAGS += -std=c++11
CPPFLAGS += -I ../single_include
FUZZER_ENGINE = src/fuzzer-driver_afl.cpp
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer
fuzzers: $(FUZZERS)
parse_afl_fuzzer:
@@ -30,6 +30,3 @@ parse_ubjson_fuzzer:
parse_bjdata_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bjdata.cpp -o $@
parse_bon8_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bon8.cpp -o $@
-4
View File
@@ -44,10 +44,6 @@ TEST_CASE("default namespace")
expected += "_snul";
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
expected += "_ekmo";
#endif
expected += "_v" STRINGIZE(NLOHMANN_JSON_VERSION_MAJOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_MINOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_PATCH) "::basic_json";
-4
View File
@@ -45,10 +45,6 @@ TEST_CASE("default namespace without version component")
expected += "_snul";
#endif
#if JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
expected += "_ekmo";
#endif
expected += "::basic_json";
// fallback for Clang
+1 -17
View File
@@ -274,8 +274,7 @@ enum class binary_format
ubjson_optimized,
bjdata,
bjdata_optimized,
bson,
bon8
bson
};
static std::vector<std::uint8_t> to_binary(const json& j, const binary_format format)
@@ -294,8 +293,6 @@ static std::vector<std::uint8_t> to_binary(const json& j, const binary_format fo
return json::to_bjdata(j);
case binary_format::bjdata_optimized:
return json::to_bjdata(j, true, true);
case binary_format::bon8:
return json::to_bon8(j);
case binary_format::bson:
default:
return json::to_bson(j);
@@ -316,8 +313,6 @@ static json from_binary(const std::vector<std::uint8_t>& bytes, const binary_for
case binary_format::bjdata:
case binary_format::bjdata_optimized:
return json::from_bjdata(bytes);
case binary_format::bon8:
return json::from_bon8(bytes);
case binary_format::bson:
default:
return json::from_bson(bytes);
@@ -338,8 +333,6 @@ static json from_binary(std::FILE* file, const binary_format format)
case binary_format::bjdata:
case binary_format::bjdata_optimized:
return json::from_bjdata(file);
case binary_format::bon8:
return json::from_bon8(file);
case binary_format::bson:
default:
return json::from_bson(file);
@@ -414,10 +407,6 @@ BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata / canada, TEST_DATA_DIRECTORY "/nativ
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata_optimized / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bjdata_optimized);
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata_optimized / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bjdata_optimized);
BENCHMARK_CAPTURE(FromBinaryBuffer, bon8 / jeopardy, TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json", binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryBuffer, bon8 / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryBuffer, bon8 / citm_catalog, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryBuffer, bon8 / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bon8);
// BSON requires an object at the top level, so the array-rooted test files
// (jeopardy and the regression files) cannot be captured here
BENCHMARK_CAPTURE(FromBinaryBuffer, bson / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bson);
@@ -461,8 +450,6 @@ BENCHMARK_CAPTURE(FromBinaryFile, cbor / twitter, TEST_DATA_DIRECTORY "/nativejs
BENCHMARK_CAPTURE(FromBinaryFile, ubjson / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryFile, ubjson / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryFile, bjdata / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryFile, bon8 / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryFile, bon8 / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryFile, bson / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bson);
//////////////////////////////////////////////////////////////////////////////
@@ -543,21 +530,18 @@ BENCHMARK_CAPTURE(FromBinaryShape, nested / msgpack, make_nested, binary_format:
BENCHMARK_CAPTURE(FromBinaryShape, nested / ubjson, make_nested, binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryShape, nested / bjdata, make_nested, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, nested / bson, make_nested, binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryShape, nested / bon8, make_nested, binary_format::bon8);
BENCHMARK_CAPTURE(FromBinaryShape, containers / cbor, make_containers, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, containers / msgpack, make_containers, binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryShape, containers / ubjson, make_containers, binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryShape, containers / ubjson_optimized, make_containers, binary_format::ubjson_optimized);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bjdata, make_containers, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bson, make_containers, binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bon8, make_containers, binary_format::bon8);
// BSON names every array element, so a large array measures key generation
// rather than scalar decoding and is left out here
BENCHMARK_CAPTURE(FromBinaryShape, scalars / cbor, make_scalars, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / msgpack, make_scalars, binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / ubjson, make_scalars, binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / bjdata, make_scalars, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / bon8, make_scalars, binary_format::bon8);
/*!
@brief parse an indefinite-length CBOR string
+3 -3
View File
@@ -1,6 +1,6 @@
# Fuzz testing
Each parser of the library (JSON, BJData, BON8, BSON, CBOR, MessagePack, and UBJSON) can be fuzz tested. Currently,
Each parser of the library (JSON, BJData, BSON, CBOR, MessagePack, and UBJSON) can be fuzz tested. Currently,
[libFuzzer](https://llvm.org/docs/LibFuzzer.html) and [afl++](https://github.com/AFLplusplus/AFLplusplus) are supported.
## Corpus creation
@@ -10,11 +10,11 @@ directory with some simple input files that cover several features of the parser
for mutations.
```shell
TEST_DATA_VERSION=3.2.0
TEST_DATA_VERSION=3.1.0
wget https://github.com/nlohmann/json_test_data/archive/refs/tags/v$TEST_DATA_VERSION.zip
unzip v$TEST_DATA_VERSION.zip
rm v$TEST_DATA_VERSION.zip
for FORMAT in json bjdata bon8 bson cbor msgpack ubjson
for FORMAT in json bjdata bson cbor msgpack ubjson
do
rm -fr corpus_$FORMAT
mkdir corpus_$FORMAT
-103
View File
@@ -1,103 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
/*
This file implements a parser test suitable for fuzz testing. Given a byte
array data, it performs the following steps:
- j1 = from_bon8(data)
- vec = to_bon8(j1)
- j2 = from_bon8(vec)
- assert(j1 == j2)
It also checks that reading the data from a stream, which reads strings byte by
byte, gives the same value or error as reading it from contiguous memory, which
copies strings in bulk.
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include <cassert>
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
namespace
{
// the serialization of the value read from @a input, or the error message
template<typename InputType>
std::string read_bon8(InputType&& input)
{
try
{
const auto vec = json::to_bon8(json::from_bon8(std::forward<InputType>(input)));
return {vec.begin(), vec.end()};
}
catch (const json::exception& e)
{
return e.what();
}
}
} // namespace
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
// contiguous and stream input must be read alike
{
std::istringstream stream(std::string(reinterpret_cast<const char*>(data), size));
assert(read_bon8(std::vector<uint8_t>(data, data + size)) == read_bon8(stream));
}
try
{
// step 1: parse input
std::vector<uint8_t> const vec1(data, data + size);
json const j1 = json::from_bon8(vec1);
try
{
// step 2: round trip
std::vector<uint8_t> const vec2 = json::to_bon8(j1);
// parse serialization
json const j2 = json::from_bon8(vec2);
// serializations must match
assert(json::to_bon8(j2) == vec2);
}
catch (const json::parse_error&)
{
// parsing a BON8 serialization must not fail
assert(false);
}
}
catch (const json::parse_error&)
{
// parse errors are ok, because input may be random bytes
}
catch (const json::type_error&)
{
// type errors can occur during parsing, too
}
catch (const json::out_of_range&)
{
// out of range errors may happen if provided sizes are excessive
}
// return 0 - non-zero return values are reserved for future use
return 0;
}
-18
View File
@@ -9,7 +9,6 @@
#pragma once
#include <cstdint> // uint8_t
#include <cstddef> // size_t
#include <fstream> // ifstream, istreambuf_iterator, ios
#include <vector> // vector
@@ -25,23 +24,6 @@ namespace utils
template<typename T>
inline void ignore_return_value(T&& /*unused*/) noexcept {}
// Advance i toward last (inclusive) by stride, always visiting last.
// stride 7 is coprime to 256, so every low-byte residue is still hit.
template<typename T>
T next_integer_sample(T i, T last, T stride)
{
if (i >= last)
{
return static_cast<T>(last + 1);
}
if (stride > 0 && i > static_cast<T>(last - stride))
{
return last;
}
const T n = static_cast<T>(i + stride);
return n < last ? n : last;
}
inline std::vector<std::uint8_t> read_binary_file(const std::string& filename)
{
std::ifstream file(filename, std::ios::binary);
+175 -6
View File
@@ -12,6 +12,11 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <valarray>
#if JSON_HAS_RANGES
#include <ranges>
#endif
namespace
{
// special test case to check if memory is leaked if constructor throws
@@ -37,12 +42,6 @@ struct bad_allocator : std::allocator<T>
};
} // namespace
TEST_CASE("get_allocator")
{
const auto alloc = nlohmann::json::get_allocator();
CHECK(alloc == std::allocator<nlohmann::json>());
}
TEST_CASE("bad_alloc")
{
SECTION("bad_alloc")
@@ -394,3 +393,173 @@ TEST_CASE("bad my_allocator::construct")
j["test"].push_back("should not leak");
}
}
// the no-exceptions CI job skips every CHECK_THROWS_AS, which would leave
// next_construct_fails set for the next allocation outside a check
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("a failed allocation leaves the value unchanged")
{
// create JSON type using the throwing allocator
using my_json = nlohmann::basic_json<std::map,
std::vector,
std::string,
bool,
std::int64_t,
std::uint64_t,
double,
my_allocator>;
// Each of these creates a string, array, object, or binary value. The
// value must be created before the type is changed: otherwise, a failed
// creation left a value of the new type without anything behind it (an
// assertion in its destructor, a null pointer everywhere else) or, when
// an old value was destroyed first, with a pointer to that destroyed one.
SECTION("creating a binary value")
{
const std::vector<std::uint8_t> bytes = {1, 2, 3};
my_json _;
next_construct_fails = true;
CHECK_THROWS_AS(_ = my_json::binary(bytes), std::bad_alloc&);
next_construct_fails = true;
CHECK_THROWS_AS(_ = my_json::binary(bytes, 42), std::bad_alloc&);
next_construct_fails = true;
CHECK_THROWS_AS(_ = my_json::binary(std::vector<std::uint8_t>(bytes)), std::bad_alloc&);
next_construct_fails = true;
CHECK_THROWS_AS(_ = my_json::binary(std::vector<std::uint8_t>(bytes), 42), std::bad_alloc&);
next_construct_fails = false;
}
SECTION("turning a null value into an array or object")
{
my_json j;
next_construct_fails = true;
CHECK_THROWS_AS(j[0], std::bad_alloc&);
CHECK(j.is_null());
next_construct_fails = true;
CHECK_THROWS_AS(j["key"], std::bad_alloc&);
CHECK(j.is_null());
#ifdef JSON_HAS_CPP_17
next_construct_fails = true;
CHECK_THROWS_AS(j[std::string_view("key")], std::bad_alloc&);
CHECK(j.is_null());
#endif
next_construct_fails = true;
CHECK_THROWS_AS(j.push_back(my_json(1)), std::bad_alloc&);
CHECK(j.is_null());
const my_json one = 1;
next_construct_fails = true;
CHECK_THROWS_AS(j.push_back(one), std::bad_alloc&);
CHECK(j.is_null());
next_construct_fails = true;
CHECK_THROWS_AS(j.push_back(my_json::object_t::value_type("key", 1)), std::bad_alloc&);
CHECK(j.is_null());
next_construct_fails = true;
CHECK_THROWS_AS(j.emplace_back(1), std::bad_alloc&);
CHECK(j.is_null());
next_construct_fails = true;
CHECK_THROWS_AS(j.emplace("key", 1), std::bad_alloc&);
CHECK(j.is_null());
const my_json object = {{"key", 1}};
next_construct_fails = true;
CHECK_THROWS_AS(j.update(object), std::bad_alloc&);
CHECK(j.is_null());
next_construct_fails = false;
}
SECTION("converting into an existing value")
{
// to_json replaces the value it is given; the old one must survive a
// failed creation of the new one
my_json j = "old";
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::string("new")), std::bad_alloc&);
CHECK(j == "old");
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::vector<int> {1, 2}), std::bad_alloc&);
CHECK(j == "old");
// with iterator debugging, the default constructor of VS 2015's
// std::vector is noexcept but constructs a proxy with the allocator,
// which terminates when this allocator throws
#if !(defined(_MSC_VER) && _MSC_VER < 1910 && defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::vector<bool> {true, false}), std::bad_alloc&);
CHECK(j == "old");
#endif
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::map<std::string, int> {{"a", 1}}), std::bad_alloc&);
CHECK(j == "old");
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, my_json::binary_t({1, 2})), std::bad_alloc&);
CHECK(j == "old");
// the overloads for lvalues of the value types, for the value types
// themselves, and for the remaining compatible types
const std::string string = "new";
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, string), std::bad_alloc&);
CHECK(j == "old");
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, "new"), std::bad_alloc&);
CHECK(j == "old");
// to_json only moves a binary value that it converted from another
// container type, which my_json's std::vector<std::uint8_t> is not
using binary_constructor = nlohmann::detail::external_constructor<nlohmann::detail::value_t::binary>;
next_construct_fails = true;
CHECK_THROWS_AS(binary_constructor::construct(j, my_json::binary_t({1, 2})), std::bad_alloc&);
CHECK(j == "old");
my_json::array_t array = {1, 2};
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, array), std::bad_alloc&);
CHECK(j == "old");
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::move(array)), std::bad_alloc&);
CHECK(j == "old");
my_json::object_t object = {{"a", 1}};
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, object), std::bad_alloc&);
CHECK(j == "old");
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::move(object)), std::bad_alloc&);
CHECK(j == "old");
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, std::valarray<int> {1, 2}), std::bad_alloc&);
CHECK(j == "old");
#if JSON_HAS_RANGES && !defined(__MINGW32__)
const std::vector<int> numbers = {1, 2};
next_construct_fails = true;
CHECK_THROWS_AS(nlohmann::to_json(j, numbers | std::views::filter([](int /*unused*/)
{
return true;
})), std::bad_alloc&);
CHECK(j == "old");
#endif
next_construct_fails = false;
nlohmann::to_json(j, std::vector<int> {1, 2});
CHECK(j == my_json({1, 2}));
}
}
#endif
-1
View File
@@ -185,7 +185,6 @@ TEST_CASE("alternative string type")
CHECK(alt_json::from_cbor(alt_json::to_cbor(doc)) == doc);
CHECK(alt_json::from_msgpack(alt_json::to_msgpack(doc)) == doc);
CHECK(alt_json::from_bon8(alt_json::to_bon8(doc)) == doc);
// BSON is not covered: it additionally needs string_t::find(value_type),
// which alt_string does not provide
CHECK(alt_json::from_ubjson(alt_json::to_ubjson(doc)) == doc);
-15
View File
@@ -25,7 +25,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
const auto bjdata_1_size = json::to_bjdata(j).size();
const auto bjdata_2_size = json::to_bjdata(j, true).size();
const auto bjdata_3_size = json::to_bjdata(j, true, true).size();
const auto bon8_size = json::to_bon8(j).size();
const auto bson_size = json::to_bson(j).size();
const auto cbor_size = json::to_cbor(j).size();
const auto msgpack_size = json::to_msgpack(j).size();
@@ -37,7 +36,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK(bjdata_1_size == 1112030);
CHECK(bjdata_2_size == 1224148);
CHECK(bjdata_3_size == 1224148);
CHECK(bon8_size == 1055792);
CHECK(bson_size == 1794522);
CHECK(cbor_size == 1055552);
CHECK(msgpack_size == 1056145);
@@ -49,7 +47,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.509));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.849));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.497));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.526));
@@ -67,7 +64,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
const auto bjdata_1_size = json::to_bjdata(j).size();
const auto bjdata_2_size = json::to_bjdata(j, true).size();
const auto bjdata_3_size = json::to_bjdata(j, true, true).size();
const auto bon8_size = json::to_bon8(j).size();
const auto bson_size = json::to_bson(j).size();
const auto cbor_size = json::to_cbor(j).size();
const auto msgpack_size = json::to_msgpack(j).size();
@@ -79,7 +75,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK(bjdata_1_size == 425342);
CHECK(bjdata_2_size == 429970);
CHECK(bjdata_3_size == 429970);
CHECK(bon8_size == 391396);
CHECK(bson_size == 444568);
CHECK(cbor_size == 402814);
CHECK(msgpack_size == 401510);
@@ -91,7 +86,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(91.097));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(92.089));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(92.089));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(83.828));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(95.215));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(86.273));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(85.993));
@@ -109,7 +103,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
const auto bjdata_1_size = json::to_bjdata(j).size();
const auto bjdata_2_size = json::to_bjdata(j, true).size();
const auto bjdata_3_size = json::to_bjdata(j, true, true).size();
const auto bon8_size = json::to_bon8(j).size();
const auto bson_size = json::to_bson(j).size();
const auto cbor_size = json::to_cbor(j).size();
const auto msgpack_size = json::to_msgpack(j).size();
@@ -121,7 +114,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK(bjdata_1_size == 390781);
CHECK(bjdata_2_size == 433557);
CHECK(bjdata_3_size == 432964);
CHECK(bon8_size == 317879);
CHECK(bson_size == 479430);
CHECK(cbor_size == 342373);
CHECK(msgpack_size == 342473);
@@ -133,7 +125,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(78.109));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(86.659));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(86.541));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(63.538));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(95.828));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(68.433));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(68.453));
@@ -151,7 +142,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
const auto bjdata_1_size = json::to_bjdata(j).size();
const auto bjdata_2_size = json::to_bjdata(j, true).size();
const auto bjdata_3_size = json::to_bjdata(j, true, true).size();
const auto bon8_size = json::to_bon8(j).size();
const auto bson_size = json::to_bson({{"", j}}).size(); // wrap array in object for BSON
const auto cbor_size = json::to_cbor(j).size();
const auto msgpack_size = json::to_msgpack(j).size();
@@ -163,7 +153,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK(bjdata_1_size == 50710965);
CHECK(bjdata_2_size == 51144830);
CHECK(bjdata_3_size == 51144830);
CHECK(bon8_size == 45942080);
CHECK(bson_size == 56008520);
CHECK(cbor_size == 46187320);
CHECK(msgpack_size == 46158575);
@@ -175,7 +164,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(96.576));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(97.402));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(97.402));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(87.494));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(106.665));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(87.961));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(87.906));
@@ -193,7 +181,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
const auto bjdata_1_size = json::to_bjdata(j).size();
const auto bjdata_2_size = json::to_bjdata(j, true).size();
const auto bjdata_3_size = json::to_bjdata(j, true, true).size();
const auto bon8_size = json::to_bon8(j).size();
// BSON cannot process the file as it contains code point U+0000
const auto cbor_size = json::to_cbor(j).size();
const auto msgpack_size = json::to_msgpack(j).size();
@@ -205,7 +192,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK(bjdata_1_size == 148695);
CHECK(bjdata_2_size == 150569);
CHECK(bjdata_3_size == 150569);
CHECK(bon8_size == 144477);
CHECK(cbor_size == 147095);
CHECK(msgpack_size == 147017);
CHECK(ubjson_1_size == 148695);
@@ -216,7 +202,6 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(88.153));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(89.264));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(89.264));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(85.653));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(87.205));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(87.158));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(88.153));
-18
View File
@@ -12,7 +12,6 @@
using nlohmann::json;
#include <cstdint>
#include <limits>
#include <string>
#include <vector>
@@ -50,12 +49,6 @@ std::vector<json> test_values()
};
}
// BON8 has no integers above the int64 range, so to_bon8() rejects them
bool bon8_representable(const json& j)
{
return !j.is_number_unsigned() || j.get<std::uint64_t>() <= static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)());
}
// values to_bson() accepts: the document must be an object
std::vector<json> bson_values()
{
@@ -99,13 +92,6 @@ TEST_CASE("binary writer output sinks")
json::to_msgpack(j, msgpack);
CHECK(json::to_msgpack(j) == msgpack);
if (bon8_representable(j))
{
std::vector<std::uint8_t> bon8;
json::to_bon8(j, bon8);
CHECK(json::to_bon8(j) == bon8);
}
for (const bool use_size :
{
false, true
@@ -186,10 +172,6 @@ TEST_CASE("binary_reserve_hint never over-reserves")
CHECK(hint <= json::to_ubjson(j).size());
CHECK(hint <= json::to_ubjson(j, true, true).size());
CHECK(hint <= json::to_bjdata(j).size());
if (bon8_representable(j))
{
CHECK(hint <= json::to_bon8(j).size());
}
}
for (const auto& j : bson_values())
+25 -120
View File
@@ -418,7 +418,7 @@ TEST_CASE("BJData")
SECTION("-32768..-129 (int16)")
{
for (int32_t i = -32768; i <= -129; i = utils::next_integer_sample(i, -129, 7))
for (int32_t i = -32768; i <= -129; ++i)
{
CAPTURE(i)
@@ -578,7 +578,7 @@ TEST_CASE("BJData")
SECTION("256..32767 (int16)")
{
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
for (size_t i = 256; i <= 32767; ++i)
{
CAPTURE(i)
@@ -911,7 +911,7 @@ TEST_CASE("BJData")
SECTION("256..32767 (int16)")
{
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
for (size_t i = 256; i <= 32767; ++i)
{
CAPTURE(i)
@@ -3763,49 +3763,6 @@ TEST_CASE("BJData")
}
}
TEST_CASE("BJData input that cannot be read is discarded by every overload")
{
std::vector<std::uint8_t> input = json::to_bjdata(json({{"a", {1, 2}}}));
input.pop_back();
json _;
CHECK_THROWS_AS(_ = json::from_bjdata(input.begin(), input.end()), json::parse_error&);
CHECK(json::from_bjdata(input, true, false).is_discarded());
CHECK(json::from_bjdata(input.begin(), input.end(), true, false).is_discarded());
}
TEST_CASE("BJData SAX parsing stops at every event")
{
// Containers are opened and closed by the loop that reads them; a SAX
// handler that rejects any event - including the end of a nested
// container - must stop the parse right there.
const auto count_events = [](const std::vector<std::uint8_t>& input)
{
int events = 0;
while (true)
{
SaxCountdown scp(events);
if (json::sax_parse(input, &scp, json::input_format_t::bjdata))
{
return events;
}
++events;
REQUIRE(events < 1000);
}
};
// 20 events: every container kind closes inside another one
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
CHECK(count_events(json::to_bjdata(j)) == 20);
CHECK(count_events(json::to_bjdata(j, true)) == 20);
CHECK(count_events(json::to_bjdata(j, true, true)) == 20);
// an ND-array is announced as an annotated object: start_object, then
// _ArrayType_, _ArraySize_ and _ArrayData_ with its elements
const json ndarray = json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})");
CHECK(count_events(json::to_bjdata(ndarray, true, true)) == 16);
}
TEST_CASE("issue #5405 - array reserve for definite-length BJData arrays")
{
#if !defined(JSON_NOEXCEPTION)
@@ -4290,65 +4247,6 @@ TEST_CASE("all BJData first bytes")
}
#endif
TEST_CASE("BJData and UBJSON can be written to a string")
{
const std::vector<json> values =
{
{{"a", {1, 2.5, "x", nullptr}}, {"b", json::binary({1, 2})}},
// an annotated ND-array, and objects that only look like one
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": 1, "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": 4, "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, -2], "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": 1})"),
};
// compared byte by byte: building a std::string from the bytes would
// convert them implicitly, which -fsanitize=integer reports for bytes of
// 0x80 and above
const auto same_bytes = [](const std::vector<std::uint8_t>& bytes, const std::string & text)
{
return bytes.size() == text.size() && std::equal(bytes.begin(), bytes.end(), text.begin(), [](std::uint8_t byte, char c)
{
return byte == static_cast<std::uint8_t>(c);
});
};
for (const auto& j : values)
{
CAPTURE(j.dump());
for (const bool use_size :
{
false, true
})
{
for (const bool use_type :
{
false, true
})
{
if (use_type && !use_size)
{
continue;
}
CAPTURE(use_size);
CAPTURE(use_type);
const auto bjdata = json::to_bjdata(j, use_size, use_type);
std::string bjdata_string;
json::to_bjdata(j, bjdata_string, use_size, use_type);
CHECK(same_bytes(bjdata, bjdata_string));
const auto ubjson = json::to_ubjson(j, use_size, use_type);
std::string ubjson_string;
json::to_ubjson(j, ubjson_string, use_size, use_type);
CHECK(same_bytes(ubjson, ubjson_string));
}
}
}
}
TEST_CASE("BJData use_type requires use_size")
{
SECTION("non-empty object throws other_error.502")
@@ -4367,17 +4265,6 @@ TEST_CASE("BJData use_type requires use_size")
json::other_error&);
}
SECTION("non-empty binary value throws other_error.502")
{
const json j = json::binary({1, 2, 3});
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
"[json.exception.other_error.502] use_type requires use_size = true",
json::other_error&);
CHECK_THROWS_WITH_AS(json::to_ubjson(j, false, true),
"[json.exception.other_error.502] use_type requires use_size = true",
json::other_error&);
}
SECTION("scalars do not throw with use_type=true, use_count=false")
{
CHECK_NOTHROW(json::to_bjdata(42, false, true));
@@ -4541,27 +4428,45 @@ TEST_CASE("BJData roundtrips" * doctest::skip())
{
CAPTURE(filename)
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
auto packed = utils::read_binary_file(filename + ".bjdata");
{
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse BJData file
auto packed = utils::read_binary_file(filename + ".bjdata");
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(packed));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": std::ifstream");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse BJData file
std::ifstream f_bjdata(filename + ".bjdata", std::ios::binary);
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(f_bjdata));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": output to output adapters");
// parse JSON file
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
// parse BJData file
auto packed = utils::read_binary_file(filename + ".bjdata");
{
INFO_WITH_TEMP(filename + ": output adapters: std::vector<uint8_t>");
std::vector<uint8_t> vec;
File diff suppressed because it is too large Load Diff
-77
View File
@@ -1066,45 +1066,6 @@ TEST_CASE("Incomplete BSON Input")
}
}
// the test catches the exceptions of invalid input
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("BSON keys from contiguous and stream input")
{
// contiguous input reads a key up to its \x00-byte in one step, a stream
// reads it byte by byte; both must give the same value or error for the
// complete document and for every truncation of it
const json j = {{"", true}, {"k", {1, 2, 3}}, {std::string(40, 'x'), {{"nested key", "value"}}}};
const std::vector<std::uint8_t> bson = json::to_bson(j);
CHECK(json::from_bson(bson) == j);
for (std::size_t length = 0; length <= bson.size(); ++length)
{
CAPTURE(length)
const std::vector<std::uint8_t> input(bson.begin(), bson.begin() + static_cast<std::ptrdiff_t>(length));
std::string from_vector;
std::string from_stream;
try
{
from_vector = json::from_bson(input).dump();
}
catch (const json::parse_error& e)
{
from_vector = e.what();
}
try
{
std::istringstream stream(std::string(input.begin(), input.end()));
from_stream = json::from_bson(stream).dump();
}
catch (const json::parse_error& e)
{
from_stream = e.what();
}
CHECK(from_vector == from_stream);
}
}
#endif
TEST_CASE("Negative size of binary value")
{
// invalid BSON: the size of the binary value is -1
@@ -1283,44 +1244,6 @@ TEST_CASE("BSON nesting does not consume the call stack")
}
}
TEST_CASE("BSON input that cannot be read is discarded by every overload")
{
std::vector<std::uint8_t> input = json::to_bson(json({{"a", {1, 2}}}));
input.pop_back();
json _;
CHECK_THROWS_AS(_ = json::from_bson(input.begin(), input.end()), json::parse_error&);
CHECK(json::from_bson(input, true, false).is_discarded());
CHECK(json::from_bson(input.begin(), input.end(), true, false).is_discarded());
CHECK(json::from_bson(input.data(), input.size(), true, false).is_discarded());
CHECK(json::from_bson({input.data(), input.size()}, true, false).is_discarded());
}
TEST_CASE("BSON SAX parsing stops at every event")
{
// Containers are opened and closed by the loop that reads them; a SAX
// handler that rejects any event - including the end of a nested
// container - must stop the parse right there.
const auto count_events = [](const std::vector<std::uint8_t>& input)
{
int events = 0;
while (true)
{
SaxCountdown scp(events);
if (json::sax_parse(input, &scp, json::input_format_t::bson))
{
return events;
}
++events;
REQUIRE(events < 1000);
}
};
// 20 events: every container kind closes inside another one
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
CHECK(count_events(json::to_bson(j)) == 20);
}
TEST_CASE("BSON numerical data")
{
SECTION("number")
+34 -140
View File
@@ -15,7 +15,6 @@ using nlohmann::json;
#include <sstream>
#include <iomanip>
#include <limits>
#include <list>
#include <set>
#include "make_test_data_available.hpp"
#include "test_utils.hpp"
@@ -291,7 +290,7 @@ TEST_CASE("CBOR")
SECTION("-65536..-257")
{
for (int32_t i = -65536; i <= -257; i = utils::next_integer_sample(i, -257, 7))
for (int32_t i = -65536; i <= -257; ++i)
{
CAPTURE(i)
@@ -479,7 +478,7 @@ TEST_CASE("CBOR")
SECTION("256..65535")
{
for (size_t i = 256; i <= 65535; i = utils::next_integer_sample(i, static_cast<size_t>(65535), static_cast<size_t>(7)))
for (size_t i = 256; i <= 65535; ++i)
{
CAPTURE(i)
@@ -614,7 +613,7 @@ TEST_CASE("CBOR")
SECTION("-32768..-129 (int 16)")
{
for (int16_t i = -32768; i <= static_cast<std::int16_t>(-129); i = utils::next_integer_sample(i, static_cast<int16_t>(-129), static_cast<int16_t>(7)))
for (int16_t i = -32768; i <= static_cast<std::int16_t>(-129); ++i)
{
CAPTURE(i)
@@ -719,7 +718,7 @@ TEST_CASE("CBOR")
SECTION("256..65535 (two-byte uint16_t)")
{
for (size_t i = 256; i <= 65535; i = utils::next_integer_sample(i, static_cast<size_t>(65535), static_cast<size_t>(7)))
for (size_t i = 256; i <= 65535; ++i)
{
CAPTURE(i)
@@ -2124,20 +2123,6 @@ TEST_CASE("CBOR nesting does not consume the call stack")
CHECK(json::from_cbor(input, true, false, json::cbor_tag_handler_t::ignore).is_discarded());
}
SECTION("stored tags")
{
// a tag over something other than a byte string is read like for
// ignore, so a chain of them must not recurse either (#5316)
std::vector<uint8_t> input;
for (std::size_t i = 0; i < 500000; ++i)
{
input.push_back(0xD8);
input.push_back(0x18);
}
input.push_back(0x01);
CHECK(json::from_cbor(input, true, true, json::cbor_tag_handler_t::store) == 1);
}
SECTION("a well-formed deep value is read through the SAX interface")
{
std::vector<uint8_t> input(200000, 0x9F);
@@ -2190,52 +2175,6 @@ TEST_CASE("CBOR nesting does not consume the call stack")
}
}
TEST_CASE("CBOR input that cannot be read is discarded by every overload")
{
std::vector<std::uint8_t> input = json::to_cbor(json({{"a", {1, 2}}}));
input.pop_back();
json _;
CHECK_THROWS_AS(_ = json::from_cbor(input.begin(), input.end()), json::parse_error&);
CHECK(json::from_cbor(input, true, false).is_discarded());
CHECK(json::from_cbor(input.begin(), input.end(), true, false).is_discarded());
CHECK(json::from_cbor(input.data(), input.size(), true, false).is_discarded());
CHECK(json::from_cbor({input.data(), input.size()}, true, false).is_discarded());
// a string that ends early, read through iterators that are not
// contiguous and have to be copied from one element at a time
const std::list<std::uint8_t> truncated_string = {0x63, 'a', 'b'};
CHECK(json::from_cbor(truncated_string.begin(), truncated_string.end(), true, false).is_discarded());
const std::list<std::uint8_t> complete_string = {0x63, 'a', 'b', 'c'};
CHECK(json::from_cbor(complete_string.begin(), complete_string.end()) == "abc");
}
TEST_CASE("CBOR SAX parsing stops at every event")
{
// Containers are opened and closed by the loop that reads them; a SAX
// handler that rejects any event - including the end of a nested
// container - must stop the parse right there.
const auto count_events = [](const std::vector<std::uint8_t>& input)
{
int events = 0;
while (true)
{
SaxCountdown scp(events);
if (json::sax_parse(input, &scp, json::input_format_t::cbor))
{
return events;
}
++events;
REQUIRE(events < 1000);
}
};
// 20 events: every container kind closes inside another one
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
CHECK(count_events(json::to_cbor(j)) == 20);
CHECK(count_events(std::vector<std::uint8_t>({0xBF, 0x61, 'a', 0x9F, 0x01, 0xFF, 0xFF})) == 6);
}
TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
{
// Reading an indefinite-length string or byte array used to call itself
@@ -2543,34 +2482,60 @@ TEST_CASE("CBOR roundtrips" * doctest::skip())
{
CAPTURE(filename)
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
const auto packed = utils::read_binary_file(filename + ".cbor");
{
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse CBOR file
const auto packed = utils::read_binary_file(filename + ".cbor");
json j2;
CHECK_NOTHROW(j2 = json::from_cbor(packed));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": std::ifstream");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse CBOR file
std::ifstream f_cbor(filename + ".cbor", std::ios::binary);
json j2;
CHECK_NOTHROW(j2 = json::from_cbor(f_cbor));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": uint8_t* and size");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse CBOR file
const auto packed = utils::read_binary_file(filename + ".cbor");
json j2;
CHECK_NOTHROW(j2 = json::from_cbor({packed.data(), packed.size()}));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": output to output adapters");
// parse JSON file
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
// parse CBOR file
const auto packed = utils::read_binary_file(filename + ".cbor");
if (exclude_packed.count(filename) == 0u)
{
{
@@ -3068,77 +3033,6 @@ TEST_CASE("Tagged values")
CHECK_THROWS_AS(_ = json::from_cbor(v_tagged, true, true, json::cbor_tag_handler_t::error), json::parse_error);
CHECK_THROWS_AS(_ = json::from_cbor(v_tagged, true, true, json::cbor_tag_handler_t::ignore), json::parse_error);
}
SECTION("issue #5316 - cbor_tag_handler_t::store on non-binary tagged items")
{
// 55799({"a": 1}) -- CBOR self-describe magic followed by a map
const std::vector<std::uint8_t> v_map{0xD9, 0xD9, 0xF7, 0xA1, 0x61, 0x61, 0x01};
CHECK(json::from_cbor(v_map, true, true, json::cbor_tag_handler_t::ignore) == json({{"a", 1}}));
CHECK(json::from_cbor(v_map, true, true, json::cbor_tag_handler_t::store) == json({{"a", 1}}));
// Tag 24 over unsigned integer 5
const std::vector<std::uint8_t> v_int{0xD8, 0x18, 0x05};
CHECK(json::from_cbor(v_int, true, true, json::cbor_tag_handler_t::ignore) == 5);
CHECK(json::from_cbor(v_int, true, true, json::cbor_tag_handler_t::store) == 5);
// Tag 24 over text string "foo"
const std::vector<std::uint8_t> v_str{0xD8, 0x18, 0x63, 'f', 'o', 'o'};
CHECK(json::from_cbor(v_str, true, true, json::cbor_tag_handler_t::ignore) == "foo");
CHECK(json::from_cbor(v_str, true, true, json::cbor_tag_handler_t::store) == "foo");
// Tag 24 over array [1, 2]
const std::vector<std::uint8_t> v_arr{0xD8, 0x18, 0x82, 0x01, 0x02};
CHECK(json::from_cbor(v_arr, true, true, json::cbor_tag_handler_t::ignore) == json({1, 2}));
CHECK(json::from_cbor(v_arr, true, true, json::cbor_tag_handler_t::store) == json({1, 2}));
// Tag 24 over boolean true
const std::vector<std::uint8_t> v_bool{0xD8, 0x18, 0xF5};
CHECK(json::from_cbor(v_bool, true, true, json::cbor_tag_handler_t::ignore) == true);
CHECK(json::from_cbor(v_bool, true, true, json::cbor_tag_handler_t::store) == true);
// Tag 24 over null
const std::vector<std::uint8_t> v_null{0xD8, 0x18, 0xF6};
CHECK(json::from_cbor(v_null, true, true, json::cbor_tag_handler_t::ignore) == nullptr);
CHECK(json::from_cbor(v_null, true, true, json::cbor_tag_handler_t::store) == nullptr);
// Nested tags: tag 55799 over tag 24 over integer 42
const std::vector<std::uint8_t> v_nested{0xD9, 0xD9, 0xF7, 0xD8, 0x18, 0x18, 0x2A};
CHECK(json::from_cbor(v_nested, true, true, json::cbor_tag_handler_t::ignore) == 42);
CHECK(json::from_cbor(v_nested, true, true, json::cbor_tag_handler_t::store) == 42);
// Tag 24 over byte string continues to store subtype as before
const std::vector<std::uint8_t> v_bin{0xD8, 0x18, 0x42, 0xCA, 0xFE};
auto j_bin_store = json::from_cbor(v_bin, true, true, json::cbor_tag_handler_t::store);
CHECK(j_bin_store.is_binary());
CHECK(j_bin_store.get_binary().has_subtype());
CHECK(j_bin_store.get_binary().subtype() == 24);
CHECK(j_bin_store.get_binary() == json::binary({0xCA, 0xFE}, 24).get_binary());
// Tagged values inside a container under store: [24(1), 25(h'0001')]
const std::vector<std::uint8_t> v_container{0x82, 0xD8, 0x18, 0x01, 0xD8, 0x19, 0x42, 0x00, 0x01};
auto j_container_store = json::from_cbor(v_container, true, true, json::cbor_tag_handler_t::store);
CHECK(j_container_store.is_array());
CHECK(j_container_store.size() == 2);
CHECK(j_container_store[0] == 1);
CHECK(j_container_store[1].is_binary());
CHECK(j_container_store[1].get_binary().has_subtype());
CHECK(j_container_store[1].get_binary().subtype() == 25);
CHECK(j_container_store[1].get_binary() == json::binary({0x00, 0x01}, 25).get_binary());
// Tagged values as object values under store: {"a": 55799(1), "b": 24(h'01')}
const std::vector<std::uint8_t> v_object{0xA2, 0x61, 'a', 0xD9, 0xD9, 0xF7, 0x01, 0x61, 'b', 0xD8, 0x18, 0x41, 0x01};
CHECK(json::from_cbor(v_object, true, true, json::cbor_tag_handler_t::store) == json({{"a", 1}, {"b", json::binary({0x01}, 24)}}));
// two tags in a row before a byte string: the inner tag is stored
// (this uses item_read and then the byte-string path)
const std::vector<std::uint8_t> v_nested_byte_string{0xD8, 0x18, 0xD8, 0x19, 0x42, 0x00, 0x01};
CHECK(json::from_cbor(v_nested_byte_string, true, true, json::cbor_tag_handler_t::store) == json::binary({0x00, 0x01}, 25));
// errors after a stored tag are now the same as with ignore
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<std::uint8_t> {0xD8, 0x18}, true, true, json::cbor_tag_handler_t::store), "[json.exception.parse_error.110] parse error at byte 3: syntax error while parsing CBOR value: unexpected end of input", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<std::uint8_t> {0xD8, 0x18, 0x1C}, true, true, json::cbor_tag_handler_t::store), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing CBOR value: invalid byte: 0x1C", json::parse_error&);
}
}
SECTION("negative integer overflow")
-7
View File
@@ -43,13 +43,6 @@ TEST_CASE("const_iterator class")
json::const_iterator const it(&j);
json::const_iterator it2(&j);
it2 = it;
// assigning an iterator to itself leaves it unchanged
json const a = {1, 2, 3};
json::const_iterator it3 = a.cbegin() + 1;
const json::const_iterator& same = it3;
it3 = same;
CHECK(*it3 == 2);
}
SECTION("copy constructor from non-const iterator")
-43
View File
@@ -12,7 +12,6 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cfloat> // FLT_EVAL_METHOD
#include <cstdlib> // strtod
#include <sstream> // stringstream
#include <string> // string
@@ -658,45 +657,3 @@ TEST_CASE("lexer string fast path")
}
}
}
TEST_CASE("parse_float_fast declines what it cannot convert exactly")
{
// The lexer only hands well-formed numbers to parse_float_fast, so the
// malformed ones below can only be passed to it directly. Declining is
// always safe: the caller then falls back to a slower, exact conversion.
const auto fast = [](const std::string & s, double & out)
{
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), '.', out);
};
double out = 0;
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
// without true double precision, the fast path declines everything
CHECK_FALSE(fast("1.5", out));
#else
CHECK(fast("1.5", out));
CHECK(out == 1.5);
CHECK(fast("+2.5e1", out));
CHECK(out == 25.0);
CHECK(fast("-25E-1", out));
CHECK(out == -2.5);
CHECK(fast("1e", out));
CHECK(out == 1.0);
#endif
// not a number
CHECK_FALSE(fast("", out));
CHECK_FALSE(fast("-", out));
CHECK_FALSE(fast(".", out));
CHECK_FALSE(fast("1.2.3", out));
CHECK_FALSE(fast("1x", out));
CHECK_FALSE(fast("1e+", out));
CHECK_FALSE(fast("1e1x", out));
// numbers that are not represented exactly on the fast path
CHECK_FALSE(fast("12345678901234567890", out));
CHECK_FALSE(fast("1e10000", out));
CHECK_FALSE(fast("9007199254740993", out));
CHECK_FALSE(fast("1e23", out));
CHECK_FALSE(fast("1e-23", out));
}
+1 -5
View File
@@ -2761,7 +2761,7 @@ TEST_CASE("diagnostic positions: value lifetime, input adapters, and SAX")
SECTION("binary formats have no text positions")
{
// binary formats (BJData, BON8, BSON, CBOR, MessagePack, UBJSON) are
// binary formats (CBOR, MessagePack, UBJSON, BSON, BJData) are
// parsed via detail::binary_reader, which never sets
// start_position/end_position on the values it produces (they
// have no notion of a text offset), so every value's position
@@ -2778,10 +2778,6 @@ TEST_CASE("diagnostic positions: value lifetime, input adapters, and SAX")
CHECK(from_msgpack.start_pos() == std::string::npos);
CHECK(from_msgpack.end_pos() == std::string::npos);
const json from_bon8 = json::from_bon8(json::to_bon8(src));
CHECK(from_bon8.start_pos() == std::string::npos);
CHECK(from_bon8.end_pos() == std::string::npos);
const json from_ubjson = json::from_ubjson(json::to_ubjson(src));
CHECK(from_ubjson.start_pos() == std::string::npos);
CHECK(from_ubjson.end_pos() == std::string::npos);
-217
View File
@@ -15,13 +15,7 @@
#include "doctest_compatibility.h"
#include <algorithm>
#include <cstdint>
#include <map>
#include <string>
#include <utility>
#include <vector>
#define JSON_TESTS_PRIVATE
#include <nlohmann/json.hpp>
@@ -365,15 +359,6 @@ TEST_CASE("lexicographical comparison operators")
CHECK(json(1) < json(1.5));
CHECK(json(1.5) < json(2));
CHECK(json(2) > json(1.5));
CHECK(json(-1) > json(-1.5));
CHECK(json(-1.5) < json(-1));
CHECK(json(-2) < json(-1.5));
// a float below the range of the integer type
CHECK(json(0) > json(-1e30));
CHECK(json(-1e30) < json(0));
CHECK(json(0u) > json(-0.5));
CHECK(json(-0.5) < json(0u));
// a NaN operand stays unordered against either integer kind
CHECK_FALSE(json(1) == json(nan));
@@ -750,205 +735,3 @@ TEST_CASE("regression #3868 - heterogeneous comparisons compile under C++20 (P24
}
}
#endif
namespace
{
// orders keys ascending or descending, as chosen when a map is created
template<class Key>
class directed_less
{
public:
directed_less() = default;
explicit directed_less(const bool descending) noexcept
: m_descending(descending)
{}
bool operator()(const Key& lhs, const Key& rhs) const
{
return m_descending ? rhs < lhs : lhs < rhs;
}
private:
bool m_descending = false;
};
// An object type that, like std::unordered_map, enumerates its entries in no
// fixed order - ascending or descending by key, depending on how the map was
// created - and whose operator== does not depend on that order.
// std::unordered_map itself cannot be used here: the standard does not
// require it to accept an incomplete mapped type such as basic_json, and
// libstdc++ 6 to 9 as well as the EDG front ends of icpc and nvc++ reject
// basic_json<std::unordered_map>. std::map, the default object type, works
// with all supported compilers.
template<class Key, class Value, class /*Compare*/, class Allocator>
struct unordered_object_t : std::map<Key, Value, directed_less<Key>, Allocator>
{
using base_type = std::map<Key, Value, directed_less<Key>, Allocator>;
using base_type::base_type;
friend bool operator==(const unordered_object_t& lhs, const unordered_object_t& rhs)
{
return lhs.size() == rhs.size() && std::all_of(lhs.begin(), lhs.end(), [&rhs](const std::pair<const Key, Value>& entry)
{
const auto it = rhs.find(entry.first);
return it != rhs.end() && it->second == entry.second;
});
}
friend bool operator!=(const unordered_object_t& lhs, const unordered_object_t& rhs)
{
return !(lhs == rhs);
}
};
using unordered_json = nlohmann::basic_json<unordered_object_t>;
// the entries "0" to "9", enumerated in ascending or in descending order
unordered_json make_unordered_object(const bool descending)
{
unordered_json j = unordered_json::object_t(directed_less<std::string>(descending));
for (int i = 0; i < 10; ++i)
{
j[std::to_string(i)] = i;
}
return j;
}
template<typename Json>
Json nest(Json j, const std::size_t depth)
{
for (std::size_t i = 0; i < depth; ++i)
{
Json outer = Json::object();
outer["x"] = std::move(j);
j = std::move(outer);
}
return j;
}
} // namespace
TEST_CASE("equality of objects whose entries have no fixed order")
{
// Values nested deeper than a bound are compared without the call stack,
// entry by entry. That must agree with the object type's own operator==,
// which for unordered_object_t (as for std::unordered_map) does not
// depend on the order of the entries, and for ordered_map does.
REQUIRE(make_unordered_object(true).begin().key() == "9");
REQUIRE(make_unordered_object(false).begin().key() == "0");
for (const std::size_t depth : std::vector<std::size_t> {0, 200})
{
CAPTURE(depth);
const unordered_json descending = nest(make_unordered_object(true), depth);
const unordered_json ascending = nest(make_unordered_object(false), depth);
CHECK(descending == ascending);
CHECK_FALSE(descending != ascending);
// a copy is equal to its original
const unordered_json copy = descending; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(copy == descending);
// a different value, a different key, or another entry still count
unordered_json other_value = make_unordered_object(true);
other_value["5"] = 42;
CHECK_FALSE(nest(other_value, depth) == ascending);
unordered_json other_key = make_unordered_object(true);
other_key.erase("5");
other_key["50"] = 5;
CHECK_FALSE(nest(other_key, depth) == ascending);
unordered_json more_entries = make_unordered_object(true);
more_entries["10"] = 10;
CHECK_FALSE(nest(more_entries, depth) == ascending);
CHECK_FALSE(ascending == nest(more_entries, depth));
// ordered_json compares its entries in sequence
const nlohmann::ordered_json ab = nest(nlohmann::ordered_json({{"a", 1}, {"b", 2}}), depth);
const nlohmann::ordered_json ba = nest(nlohmann::ordered_json({{"b", 2}, {"a", 1}}), depth);
CHECK_FALSE(ab == ba);
CHECK(ab != ba);
}
}
TEST_CASE("containers are compared element by element")
{
// Containers nested deeper than a bound are compared without the call
// stack, by code of their own; every relation is checked both at the top
// level and below that bound.
const auto deep = [](const json & j, const std::size_t depth)
{
json result = j;
for (std::size_t i = 0; i < depth; ++i)
{
result = json::array({std::move(result)});
}
return result;
};
for (const std::size_t depth : std::vector<std::size_t> {0, 200})
{
CAPTURE(depth);
// objects with different keys
{
const json a = deep({{"a", 1}}, depth);
const json b = deep({{"b", 1}}, depth);
CHECK_FALSE(a == b);
CHECK(a != b);
CHECK(a < b);
CHECK(b > a);
CHECK_FALSE(b < a);
#if JSON_HAS_THREE_WAY_COMPARISON
// JSON_HAS_CPP_20 (do not remove; see note at top of file)
CHECK((a <=> b) == std::partial_ordering::less); // *NOPAD*
CHECK((b <=> a) == std::partial_ordering::greater); // *NOPAD*
CHECK((a <=> a) == std::partial_ordering::equivalent); // *NOPAD*
#endif
}
// a container that is a prefix of the other one
{
// the one that runs out of elements first is the smaller one
const json shorter = deep({1}, depth);
const json longer = deep({1, 2}, depth);
CHECK(shorter < longer);
CHECK(longer > shorter);
CHECK_FALSE(longer < shorter);
CHECK_FALSE(shorter == longer);
const json smaller_object = deep({{"a", 1}}, depth);
const json larger_object = deep({{"a", 1}, {"b", 2}}, depth);
CHECK(smaller_object < larger_object);
CHECK(larger_object > smaller_object);
CHECK_FALSE(smaller_object == larger_object);
#if JSON_HAS_THREE_WAY_COMPARISON
// JSON_HAS_CPP_20 (do not remove; see note at top of file)
CHECK((shorter <=> longer) == std::partial_ordering::less); // *NOPAD*
CHECK((longer <=> shorter) == std::partial_ordering::greater); // *NOPAD*
#endif
}
// elements that cannot be ordered
{
const double nan = std::numeric_limits<double>::quiet_NaN();
const json lhs = deep({nan, 1}, depth);
const json rhs = deep({nan, 2}, depth);
CHECK_FALSE(lhs == lhs);
CHECK_FALSE(rhs < lhs);
#if JSON_HAS_THREE_WAY_COMPARISON
// JSON_HAS_CPP_20 (do not remove; see note at top of file)
// operator<=> stops there, as std::lexicographical_compare_three_way
// does, and operator< is derived from it
CHECK((lhs <=> rhs) == std::partial_ordering::unordered); // *NOPAD*
CHECK_FALSE(lhs < rhs);
#else
// operator< skips a pair of elements that cannot be ordered, as
// std::lexicographical_compare does, and the next pair decides
CHECK(lhs < rhs);
#endif
}
}
}
-73
View File
@@ -26,7 +26,6 @@ using nlohmann::json;
#include <deque>
#include <forward_list>
#include <functional>
#include <list>
#include <set>
#include <unordered_map>
@@ -1766,78 +1765,6 @@ TEST_CASE("Strict JSON to enum mapping")
}
}
namespace
{
// std::hash is only required for enums since C++14
struct enum_hash
{
template<typename T>
std::size_t operator()(T t) const noexcept
{
return static_cast<std::size_t>(t);
}
};
} // namespace
// see unit-enum_keyed_maps.cpp for JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS=1
TEST_CASE("maps with enum keys")
{
using task_map = std::map<TaskState, std::string>;
using task_umap = std::unordered_map<TaskState, std::string, enum_hash>;
using task_gmap = std::map<TaskState, std::string, std::greater<TaskState>>;
using nested_map = std::map<cards, std::map<TaskState, int>>;
using strict_map = std::map<strict_cards, int>;
using int_map = std::map<int, int>;
using int_umap = std::unordered_map<int, int>;
const task_map m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
#if !JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
SECTION("stored as array of pairs")
{
CHECK(json(m) == json::parse(R"([["stopped","aa"],["completed","bb"]])"));
CHECK(json(task_umap {{TS_RUNNING, "cc"}}) == json::parse(R"([["running","cc"]])"));
}
#endif
SECTION("read from array of pairs")
{
CHECK(json::parse(R"([["stopped","aa"],["completed","bb"]])").get<task_map>() == m);
}
SECTION("read from object (#4378)")
{
const json j = json::parse(R"({"stopped":"aa","completed":"bb"})");
CHECK(j.get<task_map>() == m);
CHECK(j.get<task_umap>() == task_umap(m.begin(), m.end()));
CHECK(j.get<task_gmap>() == task_gmap(m.begin(), m.end()));
CHECK(json::parse(R"({"kreuz":{"stopped":1}})").get<nested_map>() == nested_map {{cards::kreuz, {{TS_STOPPED, 1}}}});
CHECK(nlohmann::ordered_json::parse(R"({"stopped":"aa","completed":"bb"})").get<task_map>() == m);
// object keys go through the enum's from_json
strict_map sm;
CHECK_THROWS_WITH_AS(json::parse(R"({"what?":1})").get_to(sm),
"[json.exception.out_of_range.410] enum value out of range for strict_cards: \"what?\"", json::out_of_range&);
}
SECTION("objects are only read for enum keys")
{
int_map im;
int_umap ium;
CHECK_THROWS_WITH_AS(json::parse(R"({"1":2})").get_to(im),
"[json.exception.type_error.302] type must be array, but is object", json::type_error&);
CHECK_THROWS_WITH_AS(json::parse(R"({"1":2})").get_to(ium),
"[json.exception.type_error.302] type must be array, but is object", json::type_error&);
}
SECTION("other types are rejected")
{
task_map tm;
CHECK_THROWS_WITH_AS(json("stopped").get_to(tm),
"[json.exception.type_error.302] type must be array, but is string", json::type_error&);
}
}
#ifdef JSON_HAS_CPP_17
#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
-12
View File
@@ -49,16 +49,6 @@ TEST_CASE("binary type whose value type is not std::uint8_t")
CHECK(char_binary_json::binary({}).dump() == R"({"bytes":[],"subtype":null})");
}
SECTION("a value is converted to the binary type if it is binary or an array")
{
const std::vector<char> chars{'\0', '\x01', '\x7F'};
CHECK(char_binary_json::binary(chars).get<std::vector<char>>() == chars);
CHECK(char_binary_json({0, 1, 127}).get<std::vector<char>>() == chars);
CHECK_THROWS_WITH_AS(char_binary_json(1).get<std::vector<char>>(),
"[json.exception.type_error.302] type must be binary or array, but is number",
char_binary_json::type_error&);
}
SECTION("the default binary type is unchanged")
{
CHECK(nlohmann::json::binary({0, 1, 255}, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})");
@@ -84,8 +74,6 @@ TEST_CASE("binary type whose value type is not std::uint8_t")
// UBJSON has no binary type, so binary values are written as an array
CHECK(byte_binary_json::from_ubjson(byte_binary_json::to_ubjson(j)) == byte_binary_json({0, 1, 255}));
// the same holds for BON8
CHECK(byte_binary_json::from_bon8(byte_binary_json::to_bon8(j)) == byte_binary_json({0, 1, 255}));
}
#endif
}
-1
View File
@@ -297,7 +297,6 @@ TEST_CASE("object type without key_compare")
const auto j = no_key_compare_json::parse(R"({"a":[1,2,3],"b":"x"})");
CHECK(no_key_compare_json::from_cbor(no_key_compare_json::to_cbor(j)) == j);
CHECK(no_key_compare_json::from_msgpack(no_key_compare_json::to_msgpack(j)) == j);
CHECK(no_key_compare_json::from_bon8(no_key_compare_json::to_bon8(j)) == j);
}
SECTION("flatten and unflatten")
-35
View File
@@ -156,38 +156,3 @@ TEST_CASE("Better diagnostics with positions")
#endif
}
}
TEST_CASE("values read from a binary format have no positions")
{
// only the JSON lexer knows where a value started and ended
const json source = {{"a", {1, "x", json::binary({1})}}, {"b", {{"c", true}}}, {"d", nullptr}, {"e", 1.5}};
const std::vector<std::uint8_t> cbor = json::to_cbor(source);
const auto check_no_positions = [](const json & j)
{
CHECK(j.start_pos() == std::string::npos);
CHECK(j.end_pos() == std::string::npos);
CHECK(j.at("a").start_pos() == std::string::npos);
CHECK(j.at("a").at(1).end_pos() == std::string::npos);
CHECK(j.at("b").at("c").start_pos() == std::string::npos);
};
SECTION("DOM parser")
{
const json j = json::from_cbor(cbor);
CHECK(j == source);
check_no_positions(j);
}
SECTION("DOM parser with a callback")
{
json j;
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(cbor))> sdp(j, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
{
return true;
});
CHECK(json::sax_parse(cbor, &sdp, json::input_format_t::cbor));
CHECK(j == source);
check_no_positions(j);
}
}
-10
View File
@@ -1517,16 +1517,6 @@ TEST_CASE_TEMPLATE("element access 2 (throwing tests)", Json, nlohmann::json, nl
CHECK(j.value("/not/existing"_json_pointer, Json({{"foo", "bar"}})) == Json({{"foo", "bar"}}));
CHECK(j.value("/not/existing"_json_pointer, Json({10, 100})) == Json({10, 100}));
// an array index that is out of range, too large to be
// represented, or "-", and a token below a scalar
CHECK(j.value("/array/3"_json_pointer, 2) == 2);
CHECK(j.value("/array/-"_json_pointer, 2) == 2);
CHECK(j.value("/array/99999999999999999999999999"_json_pointer, 2) == 2);
CHECK(j.value("/integer/0"_json_pointer, 2) == 2);
CHECK(j.value("/string/x"_json_pointer, 2) == 2);
CHECK(j.value("/null/x"_json_pointer, 2) == 2);
CHECK(j.value("/array/0"_json_pointer, 2) == 1);
CHECK(j_const.value("/not/existing"_json_pointer, 2) == 2);
CHECK(j_const.value("/not/existing"_json_pointer, 2u) == 2u);
CHECK(j_const.value("/not/existing"_json_pointer, false) == false);
-246
View File
@@ -1,246 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
// skip tests if JSON_DisableEnumSerialization=ON (#4384)
#if defined(JSON_DISABLE_ENUM_SERIALIZATION) && (JSON_DISABLE_ENUM_SERIALIZATION == 1)
#define SKIP_TESTS_FOR_ENUM_SERIALIZATION
#endif
// This file tests the opt-in JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS, so it defines
// the macro itself rather than relying on a -D flag, and runs in every build.
// The default behavior is tested in unit-conversions.cpp.
#ifdef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#define JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS 1
#include <nlohmann/json.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
#include <cstddef>
#include <functional>
#include <map>
#include <string>
#include <unordered_map>
#include <utility>
#include <vector>
#define STRINGIZE_EX(x) #x
#define STRINGIZE(x) STRINGIZE_EX(x)
// NLOHMANN_JSON_SERIALIZE_ENUM uses a static std::pair
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors")
namespace
{
// std::hash is only required for enums since C++14
struct enum_hash
{
template<typename T>
std::size_t operator()(T t) const noexcept
{
return static_cast<std::size_t>(t);
}
};
} // namespace
// the example from #4378
enum TaskState // NOLINT(cert-int09-c,readability-enum-initial-value,cppcoreguidelines-use-enum-class)
{
TS_STOPPED,
TS_RUNNING,
TS_COMPLETED,
TS_INVALID = -1,
};
// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
NLOHMANN_JSON_SERIALIZE_ENUM(TaskState,
{
{TS_INVALID, nullptr},
{TS_STOPPED, "stopped"},
{TS_RUNNING, "running"},
{TS_COMPLETED, "completed"},
})
enum class color {red, green, blue}; // blue is not mapped and falls back to "red"
// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
NLOHMANN_JSON_SERIALIZE_ENUM(color,
{
{color::red, "red"},
{color::green, "green"},
})
enum class strict_color {red, green, blue}; // blue is not mapped
// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
NLOHMANN_JSON_SERIALIZE_ENUM_STRICT(strict_color,
{
{strict_color::red, "red"},
{strict_color::green, "green"},
})
enum class digit {zero, one};
// NOLINTNEXTLINE(misc-const-correctness,misc-use-internal-linkage,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays) - false positive
NLOHMANN_JSON_SERIALIZE_ENUM(digit,
{
{digit::zero, 0},
{digit::one, 1},
})
#ifndef SKIP_TESTS_FOR_ENUM_SERIALIZATION
enum class plain {zero, one}; // serialized as integer
#endif
TEST_CASE("JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS")
{
SECTION("the macro is part of the ABI tag")
{
const std::string ns = STRINGIZE(NLOHMANN_JSON_NAMESPACE);
CHECK(ns.find("_ekmo") != std::string::npos);
}
SECTION("std::map (#4378)")
{
using task_map = std::map<TaskState, std::string>;
const task_map m = {{TS_STOPPED, "aa"}, {TS_COMPLETED, "bb"}};
const json j = m;
CHECK(j == json::parse(R"({"stopped":"aa","completed":"bb"})"));
CHECK(j.get<task_map>() == m);
json j2;
j2["x"] = m;
CHECK(j2.dump() == R"({"x":{"completed":"bb","stopped":"aa"}})");
}
SECTION("std::map with custom comparator")
{
using task_map = std::map<TaskState, int, std::greater<TaskState>>;
const task_map m = {{TS_STOPPED, 1}, {TS_RUNNING, 2}};
const json j = m;
CHECK(j == json::parse(R"({"stopped":1,"running":2})"));
CHECK(j.get<task_map>() == m);
}
SECTION("std::unordered_map")
{
using task_map = std::unordered_map<TaskState, int, enum_hash>;
const task_map m = {{TS_STOPPED, 1}, {TS_RUNNING, 2}};
const json j = m;
CHECK(j == json::parse(R"({"stopped":1,"running":2})"));
CHECK(j.get<task_map>() == m);
}
SECTION("nested maps")
{
using nested_map = std::map<color, std::map<TaskState, int>>;
const nested_map m = {{color::green, {{TS_RUNNING, 1}}}, {color::red, {}}};
const json j = m;
CHECK(j == json::parse(R"({"green":{"running":1},"red":{}})"));
CHECK(j.get<nested_map>() == m);
}
SECTION("ordered_json keeps the order of the map")
{
using task_map = std::map<TaskState, int>;
const task_map m = {{TS_STOPPED, 1}, {TS_RUNNING, 2}, {TS_COMPLETED, 3}};
const ordered_json j = m;
CHECK(j.dump() == R"({"stopped":1,"running":2,"completed":3})");
CHECK(j.get<task_map>() == m);
}
SECTION("empty map")
{
const json j = std::map<TaskState, int>();
CHECK(j.is_object());
CHECK(j.empty());
}
SECTION("NLOHMANN_JSON_SERIALIZE_ENUM_STRICT")
{
using color_map = std::map<strict_color, int>;
const color_map m = {{strict_color::red, 1}, {strict_color::green, 2}};
const json j = m;
CHECK(j == json::parse(R"({"red":1,"green":2})"));
CHECK(j.get<color_map>() == m);
const color_map unmapped = {{strict_color::blue, 1}};
json _;
CHECK_THROWS_WITH_AS(_ = unmapped,
"[json.exception.out_of_range.410] enum value out of range for strict_color", json::out_of_range&);
}
SECTION("arrays of [key, value] pairs are still read")
{
using task_map = std::map<TaskState, int>;
const task_map m = {{TS_STOPPED, 1}};
CHECK(json::parse(R"([["stopped",1]])").get<task_map>() == m);
}
SECTION("other containers are not affected")
{
const std::vector<std::pair<TaskState, int>> pairs = {{TS_STOPPED, 1}};
const std::map<std::string, TaskState> string_keys = {{"a", TS_STOPPED}};
const std::map<int, int> int_keys = {{1, 2}};
CHECK(json(pairs) == json::parse(R"([["stopped",1]])"));
CHECK(json(string_keys) == json::parse(R"({"a":"stopped"})"));
CHECK(json(int_keys) == json::parse("[[1,2]]"));
}
SECTION("maps with non-unique keys are still stored as arrays of pairs")
{
const std::multimap<TaskState, int> mm = {{TS_STOPPED, 1}, {TS_STOPPED, 2}};
const std::unordered_multimap<TaskState, int, enum_hash> umm = {{TS_RUNNING, 3}, {TS_RUNNING, 3}};
CHECK(json(mm) == json::parse(R"([["stopped",1],["stopped",2]])"));
CHECK(json(umm) == json::parse(R"([["running",3],["running",3]])"));
}
SECTION("keys that do not serialize to strings")
{
const std::map<TaskState, int> null_key = {{TS_INVALID, 1}};
const std::map<digit, int> number_key = {{digit::zero, 1}};
json j = "unchanged";
// mapped to null
CHECK_THROWS_WITH_AS(j = null_key,
"[json.exception.type_error.302] type must be string, but is null", json::type_error&);
// mapped to a number
CHECK_THROWS_WITH_AS(j = number_key,
"[json.exception.type_error.302] type must be string, but is number", json::type_error&);
#ifndef SKIP_TESTS_FOR_ENUM_SERIALIZATION
// enum without NLOHMANN_JSON_SERIALIZE_ENUM
const std::map<plain, int> plain_key = {{plain::zero, 1}};
CHECK_THROWS_WITH_AS(j = plain_key,
"[json.exception.type_error.302] type must be string, but is number", json::type_error&);
#endif
CHECK(j == "unchanged");
}
SECTION("keys that serialize to the same string")
{
const std::map<color, int> m = {{color::red, 1}, {color::blue, 2}};
json j = "unchanged";
// color::blue is not mapped and falls back to "red"
CHECK_THROWS_WITH_AS(j = m,
"[json.exception.type_error.318] duplicate object key 'red'", json::type_error&);
CHECK(j == "unchanged");
}
}
DOCTEST_CLANG_SUPPRESS_WARNING_POP
-37
View File
@@ -1,37 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// cmake/test.cmake selects the C++ standard versions with which to build a
// unit test based on the presence of JSON_HAS_CPP_<VERSION> macros.
// The regression below only showed on C++17, so build this file for every
// standard like the other regression tests:
// JSON_HAS_CPP_17 JSON_HAS_CPP_20 (do not remove; see note at top of file)
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using json = nlohmann::json;
/////////////////////////////////////////////////////////////////////
// for #4825 - explicitly instantiating basic_json must compile; this
// forces instantiation of binary_writer::write_bjdata_ndarray, whose
// static_cast<string_t> was ambiguous under explicit instantiation on
// C++17. Merely compiling this translation unit is the regression test.
//
// The instantiation compiles every member function, so it has a file of its
// own: in unit-regression3.cpp it made the object too large for the MinGW
// linker to relocate (see #5511).
/////////////////////////////////////////////////////////////////////
template class nlohmann::basic_json<>;
TEST_CASE("explicit instantiation of basic_json (#4825)")
{
const json j = {1, "two", 3.0};
CHECK(j.size() == 3);
CHECK(json::from_bjdata(json::to_bjdata(j)) == j);
}
-95
View File
@@ -1751,98 +1751,3 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
CHECK(source.patch(patch) == target);
}
}
TEST_CASE("JSON patch - every operation on ordered_json")
{
using nlohmann::ordered_json;
const ordered_json doc = {{"foo", "bar"}, {"arr", {1, 2, 3}}, {"obj", {{"a", 1}}}};
SECTION("successful operations")
{
const ordered_json patch = ordered_json::parse(R"([
{"op": "add", "path": "/obj/b", "value": 2},
{"op": "add", "path": "/arr/1", "value": 9},
{"op": "add", "path": "/arr/-", "value": 4},
{"op": "remove", "path": "/arr/0"},
{"op": "remove", "path": "/obj/a"},
{"op": "replace", "path": "/foo", "value": "baz"},
{"op": "move", "from": "/foo", "path": "/moved"},
{"op": "copy", "from": "/obj", "path": "/copied"},
{"op": "test", "path": "/copied/b", "value": 2}
])");
const ordered_json expected = ordered_json::parse(R"({
"arr": [9, 2, 3, 4], "obj": {"b": 2}, "moved": "baz", "copied": {"b": 2}
})");
CHECK(doc.patch(patch) == expected);
// adding to the root replaces the document
CHECK(doc.patch(ordered_json::parse(R"([{"op": "add", "path": "", "value": [1]}])")) == ordered_json({1}));
}
SECTION("failing operations")
{
ordered_json _;
#if JSON_DIAGNOSTICS
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "add", "path": "/arr/4", "value": 1}])")),
"[json.exception.out_of_range.401] (/arr) array index 4 is out of range", ordered_json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "add", "path": "/arr/4", "value": 1}])")),
"[json.exception.out_of_range.401] array index 4 is out of range", ordered_json::out_of_range&);
#endif
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "add", "path": "/nope/x", "value": 1}])")),
"[json.exception.out_of_range.403] key 'nope' not found", ordered_json::out_of_range&);
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "remove", "path": "/obj/nope"}])")),
"[json.exception.out_of_range.403] key 'nope' not found", ordered_json::out_of_range&);
#if JSON_DIAGNOSTICS
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "remove", "path": "/arr/3"}])")),
"[json.exception.out_of_range.401] (/arr) array index 3 is out of range", ordered_json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "remove", "path": "/arr/3"}])")),
"[json.exception.out_of_range.401] array index 3 is out of range", ordered_json::out_of_range&);
#endif
#if JSON_DIAGNOSTICS
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "test", "path": "/foo", "value": "qux"}])")),
"[json.exception.other_error.501] (/0) unsuccessful: {\"op\":\"test\",\"path\":\"/foo\",\"value\":\"qux\"}", ordered_json::other_error&);
#elif JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "test", "path": "/foo", "value": "qux"}])")),
"[json.exception.other_error.501] (bytes 1-47) unsuccessful: {\"op\":\"test\",\"path\":\"/foo\",\"value\":\"qux\"}", ordered_json::other_error&);
#else
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "test", "path": "/foo", "value": "qux"}])")),
"[json.exception.other_error.501] unsuccessful: {\"op\":\"test\",\"path\":\"/foo\",\"value\":\"qux\"}", ordered_json::other_error&);
#endif
#if JSON_DIAGNOSTICS
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "add", "path": "/foo"}])")),
"[json.exception.parse_error.105] parse error: (/0) operation 'add' must have member 'value'", ordered_json::parse_error&);
#elif JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "add", "path": "/foo"}])")),
"[json.exception.parse_error.105] parse error: (bytes 1-30) operation 'add' must have member 'value'", ordered_json::parse_error&);
#else
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "add", "path": "/foo"}])")),
"[json.exception.parse_error.105] parse error: operation 'add' must have member 'value'", ordered_json::parse_error&);
#endif
CHECK_THROWS_WITH_AS(_ = doc.patch(ordered_json::parse(R"([{"op": "move", "from": "/obj", "path": "/obj/a/b"}])")),
"[json.exception.out_of_range.414] cannot move value: 'from' path '/obj' is a proper prefix of 'path' '/obj/a/b'", ordered_json::out_of_range&);
}
SECTION("diff reproduces the target")
{
const ordered_json source = {{"a", 1}, {"b", 2}, {"c", {{"x", 1}}}, {"l", {1, 2, 3}}};
const std::vector<ordered_json> targets =
{
// a key removed, a key added, a nested change, a shorter array
{{"a", 1}, {"c", {{"x", 2}}}, {"l", {1}}, {"d", 4}},
// the same keys in another order
{{"c", {{"x", 1}}}, {"a", 1}, {"b", 2}, {"l", {1, 2, 3}}},
// new keys ahead of the common ones
{{"new", true}, {"a", 1}, {"b", 3}, {"c", {{"x", 1}}}, {"l", {1, 2, 3}}},
};
for (const auto& target : targets)
{
CAPTURE(target.dump());
CHECK(source.patch(ordered_json::diff(source, target)) == target);
}
}
}
-13
View File
@@ -872,16 +872,3 @@ TEST_CASE("JSON pointers")
}
#endif
}
TEST_CASE("unescaping keeps a '~' that does not start an escape sequence")
{
// the parser of a JSON pointer rejects such reference tokens before it
// unescapes them, so this is only reachable by calling unescape directly
std::string s = "a~2b~";
nlohmann::detail::unescape(s);
CHECK(s == "a~2b~");
s = "~0~1~";
nlohmann::detail::unescape(s);
CHECK(s == "~/~");
}
+1 -1
View File
@@ -18,7 +18,7 @@ TEST_CASE("tests on very large JSONs")
{
SECTION("issue #1419 - Segmentation fault (stack overflow) due to unbounded recursion")
{
const auto depth = 500000;
const auto depth = 5000000;
std::string s(static_cast<std::size_t>(2 * depth), '[');
std::fill(s.begin() + depth, s.end(), ']');
-11
View File
@@ -158,17 +158,6 @@ TEST_CASE("locale-dependent test (LC_NUMERIC=de_DE)")
json::sax_parse("12.34", &sax);
CHECK(sax.float_string_copy == "12.34");
}
SECTION("serializing a long double")
{
// a floating-point type that is not a float or a double is written
// with snprintf, whose locale-specific decimal point and thousands
// separator are undone afterwards
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
CHECK(long_double_json(12345.5L).dump() == "12345.5");
CHECK(long_double_json(1.0L).dump() == "1.0");
CHECK(long_double_json(-0.25L).dump() == "-0.25");
}
}
else
{
-29
View File
@@ -345,32 +345,3 @@ TEST_CASE("JSON Merge Patch on deeply nested values")
CHECK(p->at("x") == 1);
}
}
TEST_CASE("JSON Merge Patch and update on ordered_json")
{
using nlohmann::ordered_json;
SECTION("merge_patch")
{
ordered_json target = ordered_json::parse(R"({"a": {"b": 1, "c": 2}, "d": 3, "e": [1]})");
target.merge_patch(ordered_json::parse(R"({"a": {"b": null, "f": 4}, "d": {"x": {"y": null}}, "e": null, "g": {"h": 5}})"));
CHECK(target == ordered_json::parse(R"({"a": {"c": 2, "f": 4}, "d": {"x": {}}, "g": {"h": 5}})"));
// a patch that is not an object replaces the target
target.merge_patch(ordered_json({1, 2}));
CHECK(target == ordered_json({1, 2}));
// an object patch turns a target that is not an object into one
target.merge_patch(ordered_json::parse(R"({"k": {"l": null}})"));
CHECK(target == ordered_json::parse(R"({"k": {}})"));
}
SECTION("update with merge_objects")
{
ordered_json target = ordered_json::parse(R"({"a": {"b": 1, "c": {"d": 2}}, "e": 3})");
target.update(ordered_json::parse(R"({"a": {"c": {"x": 1}, "f": 4}, "e": {"y": 5}, "g": 6})"), true);
CHECK(target == ordered_json::parse(R"({"a": {"b": 1, "c": {"d": 2, "x": 1}, "f": 4}, "e": {"y": 5}, "g": 6})"));
target.update(ordered_json::parse(R"({"a": 1})"), false);
CHECK(target == ordered_json::parse(R"({"a": 1, "e": {"y": 5}, "g": 6})"));
}
}
+33 -292
View File
@@ -14,7 +14,6 @@ using nlohmann::json;
using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
#endif
#include <cstdint> // SIZE_MAX, UINT32_MAX
#include <fstream>
#include <sstream>
#include <iomanip>
@@ -256,7 +255,7 @@ TEST_CASE("MessagePack")
SECTION("256..65535 (int 16)")
{
for (size_t i = 256; i <= 65535; i = utils::next_integer_sample(i, static_cast<size_t>(65535), static_cast<size_t>(7)))
for (size_t i = 256; i <= 65535; ++i)
{
CAPTURE(i)
@@ -441,7 +440,7 @@ TEST_CASE("MessagePack")
SECTION("-32768..-129 (int 16)")
{
for (int16_t i = -32768; i <= static_cast<std::int16_t>(-129); i = utils::next_integer_sample(i, static_cast<int16_t>(-129), static_cast<int16_t>(7)))
for (int16_t i = -32768; i <= static_cast<std::int16_t>(-129); ++i)
{
CAPTURE(i)
@@ -647,7 +646,7 @@ TEST_CASE("MessagePack")
SECTION("256..65535 (uint 16)")
{
for (size_t i = 256; i <= 65535; i = utils::next_integer_sample(i, static_cast<size_t>(65535), static_cast<size_t>(7)))
for (size_t i = 256; i <= 65535; ++i)
{
CAPTURE(i)
@@ -1781,44 +1780,6 @@ TEST_CASE("MessagePack nesting does not consume the call stack")
}
}
TEST_CASE("MessagePack input that cannot be read is discarded by every overload")
{
std::vector<std::uint8_t> input = json::to_msgpack(json({{"a", {1, 2}}}));
input.pop_back();
json _;
CHECK_THROWS_AS(_ = json::from_msgpack(input.begin(), input.end()), json::parse_error&);
CHECK(json::from_msgpack(input, true, false).is_discarded());
CHECK(json::from_msgpack(input.begin(), input.end(), true, false).is_discarded());
CHECK(json::from_msgpack(input.data(), input.size(), true, false).is_discarded());
CHECK(json::from_msgpack({input.data(), input.size()}, true, false).is_discarded());
}
TEST_CASE("MessagePack SAX parsing stops at every event")
{
// Containers are opened and closed by the loop that reads them; a SAX
// handler that rejects any event - including the end of a nested
// container - must stop the parse right there.
const auto count_events = [](const std::vector<std::uint8_t>& input)
{
int events = 0;
while (true)
{
SaxCountdown scp(events);
if (json::sax_parse(input, &scp, json::input_format_t::msgpack))
{
return events;
}
++events;
REQUIRE(events < 1000);
}
};
// 20 events: every container kind closes inside another one
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
CHECK(count_events(json::to_msgpack(j)) == 20);
}
TEST_CASE("single MessagePack roundtrip")
{
SECTION("sample.json")
@@ -2043,34 +2004,60 @@ TEST_CASE("MessagePack roundtrips" * doctest::skip())
{
CAPTURE(filename)
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
auto packed = utils::read_binary_file(filename + ".msgpack");
{
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse MessagePack file
auto packed = utils::read_binary_file(filename + ".msgpack");
json j2;
CHECK_NOTHROW(j2 = json::from_msgpack(packed));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": std::ifstream");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse MessagePack file
std::ifstream f_msgpack(filename + ".msgpack", std::ios::binary);
json j2;
CHECK_NOTHROW(j2 = json::from_msgpack(f_msgpack));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": uint8_t* and size");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse MessagePack file
auto packed = utils::read_binary_file(filename + ".msgpack");
json j2;
CHECK_NOTHROW(j2 = json::from_msgpack({packed.data(), packed.size()}));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": output to output adapters");
// parse JSON file
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
// parse MessagePack file
auto packed = utils::read_binary_file(filename + ".msgpack");
if (exclude_packed.count(filename) == 0u)
{
{
@@ -2163,249 +2150,3 @@ TEST_CASE("MessagePack with std::byte")
}
}
#endif
// the fake sizes below do not fit into a 32-bit std::size_t
#if SIZE_MAX > UINT32_MAX
template<typename T, typename A = std::allocator<T>>
struct huge_array : std::vector<T, A>
{
using base = std::vector<T, A>;
using base::base;
bool fake_size = false;
std::size_t size() const noexcept
{
if (fake_size)
{
return (std::numeric_limits<std::uint32_t>::max)() + 1ULL;
}
return base::size();
}
};
using huge_array_json = nlohmann::basic_json <
std::map, huge_array, std::string, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer,
std::vector<std::uint8_t>, void >;
TEST_CASE("MessagePack Size above uint32 for array")
{
huge_array_json j = huge_array_json::array();
j.push_back(1);
j.push_back(2);
j.push_back(3);
auto& array = j.get_ref<huge_array_json::array_t&>();
array.fake_size = true;
CHECK_THROWS_WITH_AS(
huge_array_json::to_msgpack(j),
"[json.exception.out_of_range.412] MessagePack length 4294967296 exceeds maximum of 4294967295",
json::out_of_range&);
array.fake_size = false;
}
template<typename K, typename V,
typename C = std::less<K>,
typename A = std::allocator<std::pair<const K, V>>>
struct huge_map : std::map<K, V, C, A>
{
using base = std::map<K, V, C, A>;
using base::base;
bool fake_size = false;
std::size_t size() const noexcept
{
if (fake_size)
{
return static_cast<std::size_t>(UINT32_MAX) + 1ULL;
}
return base::size();
}
};
using huge_object_json = nlohmann::basic_json <
huge_map,
std::vector,
std::string,
bool,
std::int64_t,
std::uint64_t,
double,
std::allocator,
nlohmann::adl_serializer,
std::vector<std::uint8_t>,
void >;
TEST_CASE("MessagePack Size above uint32 for object")
{
huge_object_json j = huge_object_json::object();
j["one"] = 1;
j["two"] = 2;
auto& object = j.get_ref<huge_object_json::object_t&>();
object.fake_size = true;
CHECK_THROWS_WITH_AS(
huge_object_json::to_msgpack(j),
"[json.exception.out_of_range.412] MessagePack length 4294967296 exceeds maximum of 4294967295",
json::out_of_range&);
object.fake_size = false;
}
struct huge_string : std::string
{
using std::string::string;
std::size_t size() const noexcept
{
return static_cast<std::size_t>(UINT32_MAX) + 1ULL;
}
};
using huge_string_json = nlohmann::basic_json <
std::map,
std::vector,
huge_string,
bool,
std::int64_t,
std::uint64_t,
double,
std::allocator,
nlohmann::adl_serializer,
std::vector<std::uint8_t>,
void >;
TEST_CASE("MessagePack Size above uint32 for string")
{
huge_string_json j = "hello";
CHECK_THROWS_WITH_AS(
huge_string_json::to_msgpack(j),
"[json.exception.out_of_range.412] MessagePack length 4294967296 exceeds maximum of 4294967295",
json::out_of_range&);
}
struct huge_binary : std::vector<std::uint8_t>
{
using std::vector<std::uint8_t>::vector;
std::size_t size() const noexcept
{
return static_cast<std::size_t>(UINT32_MAX) + 1ULL;
}
};
using huge_binary_json = nlohmann::basic_json <
std::map,
std::vector,
std::string,
bool,
std::int64_t,
std::uint64_t,
double,
std::allocator,
nlohmann::adl_serializer,
huge_binary,
void >;
TEST_CASE("MessagePack Size above uint32 for binary")
{
huge_binary_json j = huge_binary_json::binary(huge_binary{});
j.get_binary().push_back(0x01);
j.get_binary().push_back(0x02);
CHECK_THROWS_WITH_AS(
huge_binary_json::to_msgpack(j),
"[json.exception.out_of_range.412] MessagePack length 4294967296 exceeds maximum of 4294967295",
json::out_of_range&);
}
#endif
namespace
{
// types that report a size beyond UINT32_MAX without allocating that much
// memory, so the MessagePack length limit can be tested cheaply; see the
// similar types in unit-bson.cpp
std::size_t beyond_uint32_size()
{
return static_cast<std::size_t>((std::numeric_limits<std::uint32_t>::max)()) + 1;
}
class beyond_uint32_binary_t : public std::vector<std::uint8_t>
{
public:
using std::vector<std::uint8_t>::vector;
size_type size() const noexcept // NOLINT(readability-convert-member-functions-to-static)
{
return beyond_uint32_size();
}
};
// with clang and libstdc++ 10, the std::filesystem::path conversion that
// C++17 builds consider for every string type is ambiguous for a class
// derived from std::string, so the string case is not tested there
#if !(defined(__clang__) && defined(_GLIBCXX_RELEASE) && _GLIBCXX_RELEASE < 11)
#define JSON_TEST_BEYOND_UINT32_STRING 1
#endif
#ifdef JSON_TEST_BEYOND_UINT32_STRING
class beyond_uint32_string_t : public std::string
{
public:
using std::string::string;
size_type size() const noexcept // NOLINT(readability-convert-member-functions-to-static)
{
return beyond_uint32_size();
}
};
using beyond_uint32_string_json = nlohmann::basic_json <
std::map, std::vector, beyond_uint32_string_t, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, std::vector<std::uint8_t>, void >;
#endif
using beyond_uint32_binary_json = nlohmann::basic_json <
std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, beyond_uint32_binary_t, void >;
} // namespace
TEST_CASE("MessagePack lengths beyond UINT32_MAX cannot be serialized")
{
// MessagePack stores the length of a string, binary value, array, or
// object in at most 32 bits; a larger one used to be written without any
// length at all
#if SIZE_MAX > UINT32_MAX
{
const char* const expected = "[json.exception.out_of_range.412] MessagePack length 4294967296 exceeds maximum of 4294967295";
const beyond_uint32_binary_json binary = beyond_uint32_binary_json::binary(beyond_uint32_binary_t{});
CHECK_THROWS_WITH_AS(beyond_uint32_binary_json::to_msgpack(binary), expected, beyond_uint32_binary_json::out_of_range&);
const beyond_uint32_binary_json ext = beyond_uint32_binary_json::binary(beyond_uint32_binary_t{}, 42);
CHECK_THROWS_WITH_AS(beyond_uint32_binary_json::to_msgpack(ext), expected, beyond_uint32_binary_json::out_of_range&);
#ifdef JSON_TEST_BEYOND_UINT32_STRING
// created from its type rather than from a beyond_uint32_string_t:
// that would consider the std::filesystem::path conversion, which
// libstdc++ 10 cannot decide for a class derived from std::string
const beyond_uint32_string_json string(beyond_uint32_string_json::value_t::string);
CHECK_THROWS_WITH_AS(beyond_uint32_string_json::to_msgpack(string), expected, beyond_uint32_string_json::out_of_range&);
#endif
}
#endif
}
-16
View File
@@ -326,15 +326,6 @@ TEST_CASE("ordered_json across binary formats")
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
SECTION("BON8")
{
const auto bytes = ordered_json::to_bon8(original);
const auto restored = ordered_json::from_bon8(bytes);
CHECK(restored == original);
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
}
TEST_CASE("alt_json (custom string_t) across binary formats")
@@ -362,13 +353,6 @@ TEST_CASE("alt_json (custom string_t) across binary formats")
CHECK(restored == original);
}
SECTION("BON8")
{
const auto bytes = alt_json::to_bon8(original);
const auto restored = alt_json::from_bon8(bytes);
CHECK(restored == original);
}
SECTION("BSON")
{
const auto bytes = alt_json::to_bson(original);
-1
View File
@@ -332,7 +332,6 @@ TEST_CASE("regression tests 2")
CHECK(float_json::from_cbor(float_json::to_cbor(j)) == j);
CHECK(float_json::from_msgpack(float_json::to_msgpack(j)) == j);
CHECK(float_json::from_ubjson(float_json::to_ubjson(j)) == j);
CHECK(float_json::from_bon8(float_json::to_bon8(j)) == j);
float_json j2 = {1000.0, 2000.0, 3000.0};
CHECK(float_json::from_ubjson(float_json::to_ubjson(j2, true, true)) == j2);
+7 -2
View File
@@ -74,7 +74,13 @@ using ordered_json = nlohmann::ordered_json;
#endif
#endif
// the explicit instantiation for #4825 is in unit-explicit_instantiation.cpp
/////////////////////////////////////////////////////////////////////
// for #4825 - explicitly instantiating basic_json must compile; this
// forces instantiation of binary_writer::write_bjdata_ndarray, whose
// static_cast<string_t> was ambiguous under explicit instantiation on
// C++17. Merely compiling this translation unit is the regression test.
/////////////////////////////////////////////////////////////////////
template class nlohmann::basic_json<>;
/////////////////////////////////////////////////////////////////////
// for #4440
@@ -888,7 +894,6 @@ TEST_CASE("regression test #5476 - array type without reserve()")
// the binary formats pass a definite length to start_array()
CHECK(deque_json::from_cbor(deque_json::to_cbor(j)) == j);
CHECK(deque_json::from_msgpack(deque_json::to_msgpack(j)) == j);
CHECK(deque_json::from_bon8(deque_json::to_bon8(j)) == j);
// parse() instantiates the callback parser as well, which reserves too
const auto with_callback = deque_json::parse(R"([1,2,3])", [](int /*depth*/, deque_json::parse_event_t /*event*/, deque_json& /*parsed*/) noexcept
-160
View File
@@ -639,163 +639,3 @@ TEST_CASE("serialization of deeply nested values")
}
}
}
namespace
{
// wraps @a inner into @a depth single-element arrays
json wrap_in_arrays(const json& inner, const std::size_t depth)
{
json j = inner;
for (std::size_t i = 0; i < depth; ++i)
{
j = json::array({std::move(j)});
}
return j;
}
// what wrap_in_arrays(inner, depth).dump(2) is expected to be: the arrays
// around inner.dump(2), with inner's own lines indented by the depth
std::string expected_pretty_in_arrays(const json& inner, const std::size_t depth)
{
std::string expected;
for (std::size_t i = 0; i < depth; ++i)
{
expected += std::string(2 * i, ' ') + "[\n";
}
const std::string indent(2 * depth, ' ');
expected += indent;
for (const char c : inner.dump(2))
{
expected += c;
if (c == '\n')
{
expected += indent;
}
}
for (std::size_t i = depth; i > 0; --i)
{
expected += '\n' + std::string(2 * (i - 1), ' ') + ']';
}
return expected;
}
} // namespace
TEST_CASE("serialization of every kind of value below the bound of the descent")
{
// Values nested deeper than the bound are written without the call stack,
// by code of their own; each kind of value must come out the same there as
// it does at the top level, compact and pretty-printed.
std::vector<json> values =
{
json::parse(R"({"a": 1, "b": [1, 2, {"c": "x"}], "d": {}, "e": []})"),
json::parse(R"([1, [2, 3], {"k": null}, "s"])"),
json::object(),
json::array(),
json::binary({1, 2, 3}, 42),
json::binary({1, 2, 3}),
json::binary({}, 7),
json::binary({}),
"a string with \"escapes\"\n",
true,
false,
-42,
42u,
1.5,
nullptr,
json(json::value_t::discarded),
};
// a pretty-printed object whose members are themselves deep
values.push_back({{"x", wrap_in_arrays(1, 5)}, {"y", {{"z", 2}}}});
for (const std::size_t depth : std::vector<std::size_t> {1, 200})
{
CAPTURE(depth);
for (const auto& inner : values)
{
CAPTURE(inner.dump());
const json j = wrap_in_arrays(inner, depth);
CHECK(j.dump() == std::string(depth, '[') + inner.dump() + std::string(depth, ']'));
CHECK(j.dump(2) == expected_pretty_in_arrays(inner, depth));
}
}
SECTION("pretty-printed objects across the bound")
{
for (std::size_t d = 120; d <= 140; ++d)
{
CAPTURE(d);
// built from the inside out: {"k": <level below>, "n": <level>}
json j = 7;
std::string expected = "7";
for (std::size_t i = d; i > 0; --i)
{
j = json({{"k", std::move(j)}, {"n", i}});
const std::string indent(2 * i, ' ');
const std::string outer_indent(2 * (i - 1), ' ');
std::string next = "{\n";
next += indent;
next += "\"k\": ";
next += expected;
next += ",\n";
next += indent;
next += "\"n\": ";
next += std::to_string(i);
next += '\n';
next += outer_indent;
next += '}';
expected = std::move(next);
}
CHECK(j.dump(2) == expected);
CHECK(json::parse(j.dump(2)) == j);
CHECK(json::parse(j.dump()) == j);
}
}
}
TEST_CASE("serializer buffers are flushed mid-string and mid-binary")
{
SECTION("a long run of escaped characters")
{
// each character is escaped on its own, so the escape buffer fills up
const json newlines = std::string(600, '\n');
std::string expected = "\"";
for (int i = 0; i < 600; ++i)
{
expected += "\\n";
}
expected += '"';
CHECK(newlines.dump() == expected);
// every character is \u-escaped under ensure_ascii
std::string umlauts;
std::string escaped_umlauts = "\"";
for (int i = 0; i < 300; ++i)
{
umlauts += "\xC3\xA4";
escaped_umlauts += "\\u00e4";
}
escaped_umlauts += '"';
CHECK(json(umlauts).dump(-1, ' ', true) == escaped_umlauts);
}
SECTION("a large binary value")
{
std::vector<std::uint8_t> bytes(3000);
std::string expected_bytes;
std::string expected_pretty_bytes;
for (std::size_t i = 0; i < bytes.size(); ++i)
{
bytes[i] = static_cast<std::uint8_t>(i % 256);
expected_bytes += (i == 0 ? "" : ",") + std::to_string(i % 256);
expected_pretty_bytes += (i == 0 ? "" : ", ") + std::to_string(i % 256);
}
const json j = json::binary(bytes);
CHECK(j.dump() == "{\"bytes\":[" + expected_bytes + "],\"subtype\":null}");
CHECK(j.dump(2) == "{\n \"bytes\": [" + expected_pretty_bytes + "],\n \"subtype\": null\n}");
}
}
-23
View File
@@ -102,29 +102,6 @@ TEST_CASE("std::formatter<nlohmann::json>")
CHECK_THROWS_AS(std::vformat("{:{}}", std::make_format_args(j, dynamic_width)), std::format_error); // dynamic width
}
SECTION("a format spec may run to the end of the parse context")
{
// std::format always hands parse() a range that still holds the closing
// '}', but a parse context may also end right after the spec
const auto parse = [](const char* spec)
{
std::format_parse_context ctx(spec);
std::formatter<json> f;
CHECK(f.parse(ctx) == ctx.end());
return f;
};
CHECK(parse("").indent == -1);
CHECK(parse(">").indent == -1);
CHECK(parse("#").indent == 4);
CHECK(parse("3").indent == 3);
CHECK(parse("#12").indent == 12);
const auto f = parse(".>");
CHECK(f.indent == -1);
CHECK(f.indent_char == '.');
}
SECTION("std::format_to writes through an arbitrary output iterator")
{
const json j = {{"foo", 1}, {"bar", {1, 2, 3}}};
+33 -85
View File
@@ -265,7 +265,7 @@ TEST_CASE("UBJSON")
SECTION("-32768..-129 (int16)")
{
for (int32_t i = -32768; i <= -129; i = utils::next_integer_sample(i, -129, 7))
for (int32_t i = -32768; i <= -129; ++i)
{
CAPTURE(i)
@@ -425,7 +425,7 @@ TEST_CASE("UBJSON")
SECTION("256..32767 (int16)")
{
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
for (size_t i = 256; i <= 32767; ++i)
{
CAPTURE(i)
@@ -631,7 +631,7 @@ TEST_CASE("UBJSON")
SECTION("256..32767 (int16)")
{
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
for (size_t i = 256; i <= 32767; ++i)
{
CAPTURE(i)
@@ -1640,29 +1640,6 @@ TEST_CASE("UBJSON")
});
CHECK_THROWS_AS(_ = json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson), json::out_of_range&);
}
SECTION("array with a known size, read with a callback")
{
// a sized array announces its length to start_array()
std::vector<uint8_t> const v_ubjson = {'[', '#', 'i', 2, 'i', 1, 'i', 2};
json j;
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp(j, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
{
return true;
});
CHECK(json::sax_parse(v_ubjson, &scp, json::input_format_t::ubjson));
CHECK(j == json({1, 2}));
// the readers reject a size this large before they announce
// it, so it can only reach start_array() directly (the largest
// value stands for an unknown size and is never checked)
json k;
nlohmann::detail::json_sax_dom_callback_parser<json, decltype(nlohmann::detail::input_adapter(v_ubjson))> scp2(k, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
{
return true;
});
CHECK_THROWS_AS(scp2.start_array((std::numeric_limits<std::size_t>::max)() - 1), json::out_of_range&);
}
}
}
@@ -2278,46 +2255,6 @@ TEST_CASE("UBJSON nesting does not consume the call stack")
}
}
TEST_CASE("UBJSON input that cannot be read is discarded by every overload")
{
std::vector<std::uint8_t> input = json::to_ubjson(json({{"a", {1, 2}}}));
input.pop_back();
json _;
CHECK_THROWS_AS(_ = json::from_ubjson(input.begin(), input.end()), json::parse_error&);
CHECK(json::from_ubjson(input, true, false).is_discarded());
CHECK(json::from_ubjson(input.begin(), input.end(), true, false).is_discarded());
CHECK(json::from_ubjson(input.data(), input.size(), true, false).is_discarded());
CHECK(json::from_ubjson({input.data(), input.size()}, true, false).is_discarded());
}
TEST_CASE("UBJSON SAX parsing stops at every event")
{
// Containers are opened and closed by the loop that reads them; a SAX
// handler that rejects any event - including the end of a nested
// container - must stop the parse right there.
const auto count_events = [](const std::vector<std::uint8_t>& input)
{
int events = 0;
while (true)
{
SaxCountdown scp(events);
if (json::sax_parse(input, &scp, json::input_format_t::ubjson))
{
return events;
}
++events;
REQUIRE(events < 1000);
}
};
// 20 events: every container kind closes inside another one
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
CHECK(count_events(json::to_ubjson(j)) == 20);
CHECK(count_events(json::to_ubjson(j, true)) == 20);
CHECK(count_events(json::to_ubjson(j, true, true)) == 20);
}
TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
{
// An element of type 'Z', 'T' or 'F' is encoded by its marker alone, so an
@@ -2980,34 +2917,60 @@ TEST_CASE("UBJSON roundtrips" * doctest::skip())
{
CAPTURE(filename)
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
auto const packed = utils::read_binary_file(filename + ".ubjson");
{
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
// parse JSON file
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
// parse UBJSON file
auto const packed = utils::read_binary_file(filename + ".ubjson");
json j2;
CHECK_NOTHROW(j2 = json::from_ubjson(packed));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": std::ifstream");
// parse JSON file
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
// parse UBJSON file
std::ifstream f_ubjson(filename + ".ubjson", std::ios::binary);
json j2;
CHECK_NOTHROW(j2 = json::from_ubjson(f_ubjson));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": uint8_t* and size");
// parse JSON file
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
// parse UBJSON file
auto const packed = utils::read_binary_file(filename + ".ubjson");
json j2;
CHECK_NOTHROW(j2 = json::from_ubjson({packed.data(), packed.size()}));
// compare parsed JSON values
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": output to output adapters");
// parse JSON file
std::ifstream f_json(filename);
json const j1 = json::parse(f_json);
// parse UBJSON file
auto const packed = utils::read_binary_file(filename + ".ubjson");
{
INFO_WITH_TEMP(filename + ": output adapters: std::vector<uint8_t>");
std::vector<uint8_t> vec;
@@ -3018,18 +2981,3 @@ TEST_CASE("UBJSON roundtrips" * doctest::skip())
}
}
}
TEST_CASE("UBJSON optimized array of unsigned integers beyond int64")
{
// UBJSON has no unsigned 64-bit type, so such values are written as
// high-precision numbers - also as the type of an optimized container
const json j = {18446744073709551615ULL, 9223372036854775808ULL};
const std::vector<std::uint8_t> expected =
{
'[', '$', 'H', '#', 'i', 2,
'i', 20, '1', '8', '4', '4', '6', '7', '4', '4', '0', '7', '3', '7', '0', '9', '5', '5', '1', '6', '1', '5',
'i', 19, '9', '2', '2', '3', '3', '7', '2', '0', '3', '6', '8', '5', '4', '7', '7', '5', '8', '0', '8'
};
CHECK(json::to_ubjson(j, true, true) == expected);
CHECK(json::from_ubjson(expected) == j);
}
-4
View File
@@ -70,8 +70,6 @@ TEST_CASE("wide strings")
CHECK_THROWS_WITH_AS(_ = json::parse(std::wstring{L'"', static_cast<wchar_t>(0xDC00), L'"'}), error_low_surrogate, json::parse_error&);
// a high surrogate followed by a non-low-surrogate unit is invalid
CHECK_THROWS_WITH_AS(_ = json::parse(std::wstring{L'"', static_cast<wchar_t>(0xD800), L'a', L'"'}), error_high_surrogate, json::parse_error&);
// ... also when the unit is above the low surrogates
CHECK_THROWS_WITH_AS(_ = json::parse(std::wstring{L'"', static_cast<wchar_t>(0xD800), static_cast<wchar_t>(0xE000), L'"'}), error_high_surrogate, json::parse_error&);
// a lone low surrogate must not swallow the following unit: pairing
// it with any second unit would produce valid UTF-8, so the error
// has to report an ill-formed byte at the surrogate's own position
@@ -101,8 +99,6 @@ TEST_CASE("wide strings")
CHECK_THROWS_WITH_AS(_ = json::parse(std::u16string{u'"', 0xDC00, u'"'}), "[json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '\"<U+0000>'", json::parse_error&);
// a high surrogate followed by a non-low-surrogate unit is invalid
CHECK_THROWS_WITH_AS(_ = json::parse(std::u16string{u'"', 0xD800, u'a', u'"'}), "[json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '\"<U+0000>'", json::parse_error&);
// ... also when the unit is above the low surrogates
CHECK_THROWS_WITH_AS(_ = json::parse(std::u16string{u'"', 0xD800, 0xE000, u'"'}), "[json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '\"<U+0000>'", json::parse_error&);
// a lone low surrogate must not swallow the following unit: pairing
// it with any second unit would produce valid UTF-8, so the error
// has to report an ill-formed byte at the surrogate's own position
+1 -1
View File
@@ -20,7 +20,7 @@ if __name__ == '__main__':
namespaces = ['nlohmann']
abi_prefix = 'json_abi'
abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics', '_psp', '_snul', '_ekmo']
abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics', '_psp', '_snul']
version = '_v' + args.version.replace('.', '_')
inline_namespaces = []