Compare commits

..
Author SHA1 Message Date
Niels Lohmann 8502d73729 Merge branch 'develop' into claude/3453-breaking-change-flags-c937b4
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 20:58:00 +02:00
Niels Lohmann 3d18d2b7c5 Wrap overlong line in cbor_tag_handler_t documentation
The line added in #5559 exceeds the 160-character limit enforced by the
documentation style check.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 16:55:18 +02:00
Niels Lohmann fb3d9d9b8f List the deprecated functions removed in 4.0 in the roadmap
The roadmap lists what will be removed; the migration guide keeps the
examples for how to replace each item. Also mention the deprecated
(ptr, len) overloads of the from_* functions in the migration guide.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 16:51:39 +02:00
Niels Lohmann a164f926c5 Document the macros that preview version 4.0 in the roadmap
List the macros that guard breaking changes planned to become the
default in version 4.0.0, and explain that 4.0.0 will be the sum of
these opt-in flags, which can be tried on the 3.x release train.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 16:39:52 +02:00
12 changed files with 186 additions and 404 deletions
@@ -55,10 +55,6 @@ This implementation does exactly follow this approach, as it uses double precisi
smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
and be serialized to `null`.
During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
#### Storage
Floating-point number values are stored directly inside a `basic_json` type.
@@ -47,9 +47,8 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va
When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
or [`number_float_t`](number_float_t.md).
range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).
[RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
> Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
@@ -48,9 +48,8 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default
When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
[`number_float_t`](number_float_t.md).
when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).
[RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
> Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
@@ -115,3 +115,4 @@ The default value is `0` (disabled — existing behavior is preserved).
## Version history
- Added in version 3.13.0.
- Planned to become the default (with the macro removed) in version 4.0.0.
+62 -5
View File
@@ -14,7 +14,7 @@ work items are tracked in the [GitHub milestones](https://github.com/nlohmann/js
opt-in.
- **Keep the 3.x public API stable.** Releases follow [semantic versioning](https://semver.org). Changes that would
break existing code are only added behind a feature macro, so users can opt in and test their code before a next
major release.
major release, see [Version 4.0](#version-40).
- **Support a broad range of compilers and platforms.** The [CI](quality_assurance.md) keeps testing old and new
versions of GCC, Clang, MSVC, and other compilers on Linux, macOS, and Windows.
- **Keep the quality assurance up.** Every change keeps the test coverage at 100%, passes the static and dynamic
@@ -37,7 +37,64 @@ work items are tracked in the [GitHub milestones](https://github.com/nlohmann/js
## Version 4.0
There is no decision yet on whether or when a version 4.0 with breaking changes will be released. Proposals that need
a major version, for instance stricter type conversions, are collected in issue
[#3453](https://github.com/nlohmann/json/issues/3453). Until then, such changes are only added as opt-in behavior
behind feature macros.
There is no release date for version 4.0 yet. Proposals that need a major version, for instance stricter type
conversions, are collected in issue [#3453](https://github.com/nlohmann/json/issues/3453).
!!! note "Not final"
The plan for version 4.0 described below is not final and may still change: macros may be added to or removed from
the list, and planned defaults may be revised. Any such change will be documented on this page.
### Trying out 4.0 today
Version 4.0 will not be developed on a separate branch. Instead, every breaking change is first added to a 3.x release
behind a macro whose default keeps the 3.x behavior. Version 4.0 then switches the defaults and removes the macros.
Version 4.0 is therefore the sum of these macros: you can try it on the 3.x release train today by defining each macro
to its 4.0 value and fixing what no longer compiles or behaves differently. Once your code works with all of them, it
is ready for version 4.0.
The following macros guard changes that are planned to become the default in version 4.0:
| Macro | 3.x default | 4.0 behavior | CMake option | Added |
|------------------------------------------------------------------------------------------------------------------|-------------|-------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|--------|
| [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) | `1` | `0`: no implicit conversions from `basic_json` to other types; use [`get`](../api/basic_json/get.md) instead | [`JSON_ImplicitConversions`](../integration/cmake.md#json_implicitconversions) | 3.9.0 |
| [`JSON_USE_GLOBAL_UDLS`](../api/macros/json_use_global_udls.md) | `1` | `0`: the string literals `_json` and `_json_pointer` are only available in namespace `nlohmann::literals` | [`JSON_GlobalUDLs`](../integration/cmake.md#json_globaludls) | 3.11.0 |
| [`JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md) | `0` | removed: the deprecated legacy comparison of discarded values can no longer be enabled | [`JSON_LegacyDiscardedValueComparison`](../integration/cmake.md#json_legacydiscardedvaluecomparison) | 3.11.0 |
| [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) | `0` | `1`: single-element brace initialization such as `#!cpp json j{obj};` copies the element instead of creating an array | – | 3.13.0 |
| [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) | `0` | `1`: reading from a stream does not consume the character after a number | – | 3.13.0 |
| [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) | `0` | `1`: a NUL byte in the input is a parse error instead of the end of input | [`JSON_StrictNulHandling`](../integration/cmake.md#json_strictnulhandling) | 3.13.0 |
For example, the following makes a 3.x release behave like version 4.0 with respect to these changes:
```cpp
#define JSON_USE_IMPLICIT_CONVERSIONS 0
#define JSON_USE_GLOBAL_UDLS 0
#define JSON_BRACE_INIT_COPY_SEMANTICS 1
#define JSON_PRECISE_STREAM_POSITION 1
#define JSON_STRICT_NUL_HANDLING 1
#include <nlohmann/json.hpp>
```
The macros must be defined before the library header is included; setting them once in the build system is the easiest
way to achieve this.
### Removal of deprecated functions
Version 4.0 will remove all deprecated functions. Compiling with deprecation warnings enabled shows which of them your
code still uses. The [migration guide](../integration/migration_guide.md#replace-deprecated-functions) shows how to
replace each of them.
| Deprecated | Since | Migration |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|----------------------------------------------------------------------------------|
| `#!cpp operator<<(basic_json&, std::istream&)` | 3.0.0 | [Parsing](../integration/migration_guide.md#parsing) |
| `#!cpp operator>>(const basic_json&, std::ostream&)` | 3.0.0 | [Miscellaneous functions](../integration/migration_guide.md#miscellaneous-functions) |
| `iterator_wrapper` | 3.1.0 | [Miscellaneous functions](../integration/migration_guide.md#miscellaneous-functions) |
| [`parse`](../api/basic_json/parse.md), [`accept`](../api/basic_json/accept.md), and [`sax_parse`](../api/basic_json/sax_parse.md) with an initializer list `{ptr, len}` or `{first, last}` | 3.8.0 | [Parsing](../integration/migration_guide.md#parsing) |
| [`from_bson`](../api/basic_json/from_bson.md), [`from_cbor`](../api/basic_json/from_cbor.md), [`from_msgpack`](../api/basic_json/from_msgpack.md), and [`from_ubjson`](../api/basic_json/from_ubjson.md) with `(ptr, len)` or an initializer list | 3.8.0 | [Parsing](../integration/migration_guide.md#parsing) |
| [`json_pointer::operator string_t`](../api/json_pointer/operator_string_t.md) | 3.11.0 | [JSON Pointers](../integration/migration_guide.md#json-pointers) |
| [`json_pointer`](../api/json_pointer/index.md) with a `basic_json` type as template argument, and the overloads of `value`, `contains`, `operator[]`, and `at` accepting such a pointer | 3.11.0 | [JSON Pointers](../integration/migration_guide.md#json-pointers) |
| Comparing a [`json_pointer`](../api/json_pointer/index.md) with a string via [`operator==`](../api/json_pointer/operator_eq.md) or [`operator!=`](../api/json_pointer/operator_ne.md) | 3.11.2 | [JSON Pointers](../integration/migration_guide.md#json-pointers) |
The deprecated legacy comparison of discarded values is controlled by a macro and therefore listed in the table above.
New breaking changes will follow the same path: they are added to these tables when they land in a 3.x release.
@@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows:
!!! warning "Negative integer overflow"
CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
stored as `-1.8446744073709552e+19`.
result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
silently.
!!! warning "Object keys"
+5 -7
View File
@@ -331,6 +331,9 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
[json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
```
```
[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
```
```
[json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
```
@@ -844,18 +847,13 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme
### json.exception.out_of_range.406
A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
double-precision number when `number_float_t` is `#!cpp float`.
A parsed number could not be stored as without changing it to NaN or INF.
!!! failure "Example messages"
!!! failure "Example message"
```
number overflow parsing '10E1000'
```
```
[json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
```
### json.exception.out_of_range.407
@@ -1,10 +1,13 @@
# Migration Guide
This page collects some guidelines on how to future-proof your code for future versions of this library.
The [roadmap](../community/roadmap.md#version-40) lists what will change in version 4.0, including the macros that let
you try its behavior with a 3.x release; this page describes how to adjust your code.
## Replace deprecated functions
The following functions have been deprecated and will be removed in the next major version (i.e., 4.0.0). All
The following functions have been deprecated and will be removed in the next major version (i.e., 4.0.0), see the
[roadmap](../community/roadmap.md#removal-of-deprecated-functions) for an overview. All
deprecations are annotated with
[`HEDLEY_DEPRECATED_FOR`](https://nemequ.github.io/hedley/api-reference.html#HEDLEY_DEPRECATED_FOR) to report which
function to use instead.
@@ -34,8 +37,9 @@ function to use instead.
[`accept`](../api/basic_json/accept.md), [`sax_parse`](../api/basic_json/sax_parse.md),
[`from_cbor`](../api/basic_json/from_cbor.md), [`from_msgpack`](../api/basic_json/from_msgpack.md),
[`from_ubjson`](../api/basic_json/from_ubjson.md), and [`from_bson`](../api/basic_json/from_bson.md) via initializer
lists is deprecated since 3.8.0. Instead, pass two iterators; for instance, call `from_cbor(ptr, ptr+len)` instead of
`from_cbor({ptr, len})`.
lists is deprecated since 3.8.0. The same holds for passing a pointer and a length as two arguments to the `from_*`
functions. Instead, pass two iterators; for instance, call `from_cbor(ptr, ptr+len)` instead of
`from_cbor({ptr, len})` or `from_cbor(ptr, len)`.
=== "Deprecated"
+45 -122
View File
@@ -559,7 +559,7 @@ class binary_reader
case 0x01: // double
{
double number{};
return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0x02: // string
@@ -600,19 +600,19 @@ class binary_reader
case 0x10: // int32
{
std::int32_t value{};
return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(value);
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
}
case 0x12: // int64
{
std::int64_t value{};
return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(value);
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
}
case 0x11: // uint64
{
std::uint64_t value{};
return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(value);
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
}
default: // anything else is not supported (yet)
@@ -638,17 +638,14 @@ class binary_reader
{
return false;
}
// the value is -1 - number, which fits into number_integer_t
// whenever number does
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
if (number > max_val)
{
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(112, chars_read,
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
}
// like the lexer does for JSON text, store a value too small for
// number_integer_t as number_float_t
return sax->number_float(static_cast<number_float_t>(-1) - static_cast<number_float_t>(number), "");
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
}
/*!
@@ -705,25 +702,25 @@ class binary_reader
case 0x18: // Unsigned integer (one-byte uint8_t follows)
{
std::uint8_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
case 0x19: // Unsigned integer (two-byte uint16_t follows)
{
std::uint16_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
{
std::uint32_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
{
std::uint64_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
// Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -1168,13 +1165,13 @@ class binary_reader
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
{
float number{};
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
{
double number{};
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
default: // anything else (0xFF is handled inside the other types)
@@ -1864,61 +1861,61 @@ class binary_reader
case 0xCA: // float 32
{
float number{};
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xCB: // float 64
{
double number{};
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xCC: // uint 8
{
std::uint8_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xCD: // uint 16
{
std::uint16_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xCE: // uint 32
{
std::uint32_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xCF: // uint 64
{
std::uint64_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xD0: // int 8
{
std::int8_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xD1: // int 16
{
std::int16_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xD2: // int 32
{
std::int32_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xD3: // int 64
{
std::int64_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xDC: // array 16
@@ -2759,7 +2756,7 @@ class binary_reader
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i)))
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
{
return false;
}
@@ -2891,37 +2888,37 @@ class binary_reader
break;
}
std::uint8_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'U':
{
std::uint8_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'i':
{
std::int8_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'I':
{
std::int16_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'l':
{
std::int32_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'L':
{
std::int64_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'u':
@@ -2931,7 +2928,7 @@ class binary_reader
break;
}
std::uint16_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'm':
@@ -2941,7 +2938,7 @@ class binary_reader
break;
}
std::uint32_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'M':
@@ -2951,7 +2948,7 @@ class binary_reader
break;
}
std::uint64_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'h':
@@ -3009,13 +3006,13 @@ class binary_reader
case 'd':
{
float number{};
return get_number(input_format, number) && emit_float(input_format, number);
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 'D':
{
double number{};
return get_number(input_format, number) && emit_float(input_format, number);
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 'H':
@@ -3482,13 +3479,13 @@ class binary_reader
case 0x8E: // binary32
{
float number{};
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0x8F: // binary64
{
double number{};
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xF8:
@@ -3554,9 +3551,7 @@ class binary_reader
@brief pass an integer to the SAX parser
Non-negative integers are passed as unsigned, negative integers as signed
numbers, like the other binary formats do. A value that does not fit the
number type is passed as described for @ref emit_unsigned and
@ref emit_signed.
numbers, like the other binary formats do.
@param[in] number the integer
@return whether the SAX parser accepted the value
@@ -3565,9 +3560,9 @@ class binary_reader
{
if (number >= 0)
{
return emit_unsigned(static_cast<std::uint64_t>(number));
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return emit_signed(number);
return sax->number_integer(static_cast<number_integer_t>(number));
}
/*!
@@ -3624,7 +3619,8 @@ class binary_reader
value = (value << 8) | static_cast<std::int64_t>(current);
}
return emit_bon8_integer(negative ? -(value + offset) : value + offset);
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
}
/*!
@@ -3921,79 +3917,6 @@ class binary_reader
return true;
}
/*!
@brief pass a signed integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_integer_t is passed as number_unsigned_t if it is non-negative and
fits there, and as number_float_t otherwise. With the default number
types, every integer the binary formats can encode fits, so this only
matters for narrower custom number types.
@tparam NumberType a signed integer type
@param[in] number the integer
@return whether the SAX parser accepted the value
*/
template<typename NumberType>
bool emit_signed(const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
{
return sax->number_integer(static_cast<number_integer_t>(number));
}
if (value_in_range_of<number_unsigned_t>(number))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return sax->number_float(static_cast<number_float_t>(number), "");
}
/*!
@brief pass an unsigned integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_unsigned_t is passed as number_float_t.
@tparam NumberType an unsigned integer type
@param[in] number the integer
@return whether the SAX parser accepted the value
*/
template<typename NumberType>
bool emit_unsigned(const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return sax->number_float(static_cast<number_float_t>(number), "");
}
/*!
@brief pass a floating-point number read from the input to the SAX parser
Like the lexer does for JSON text, a finite value that overflows
number_float_t is rejected instead of silently becoming infinity. Infinity
and NaN in the input are passed on unchanged.
@tparam NumberType a floating-point type
@param[in] format the current format (for diagnostics)
@param[in] number the number
@return whether the SAX parser accepted the value
@throw out_of_range.406 if a finite @a number overflows number_float_t
*/
template<typename NumberType>
bool emit_float(const input_format_t format, const NumberType number)
{
const auto result = static_cast<number_float_t>(number);
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
{
return sax->parse_error(chars_read, get_token_string(),
out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
}
return sax->number_float(result, "");
}
/*!
@brief create a string by reading characters from the input
+45 -122
View File
@@ -13294,7 +13294,7 @@ class binary_reader
case 0x01: // double
{
double number{};
return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0x02: // string
@@ -13335,19 +13335,19 @@ class binary_reader
case 0x10: // int32
{
std::int32_t value{};
return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(value);
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
}
case 0x12: // int64
{
std::int64_t value{};
return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(value);
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
}
case 0x11: // uint64
{
std::uint64_t value{};
return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(value);
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
}
default: // anything else is not supported (yet)
@@ -13373,17 +13373,14 @@ class binary_reader
{
return false;
}
// the value is -1 - number, which fits into number_integer_t
// whenever number does
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
if (number > max_val)
{
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(112, chars_read,
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
}
// like the lexer does for JSON text, store a value too small for
// number_integer_t as number_float_t
return sax->number_float(static_cast<number_float_t>(-1) - static_cast<number_float_t>(number), "");
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
}
/*!
@@ -13440,25 +13437,25 @@ class binary_reader
case 0x18: // Unsigned integer (one-byte uint8_t follows)
{
std::uint8_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
case 0x19: // Unsigned integer (two-byte uint16_t follows)
{
std::uint16_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
{
std::uint32_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
{
std::uint64_t number{};
return get_number(input_format_t::cbor, number) && emit_unsigned(number);
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
}
// Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -13903,13 +13900,13 @@ class binary_reader
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
{
float number{};
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
{
double number{};
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
default: // anything else (0xFF is handled inside the other types)
@@ -14599,61 +14596,61 @@ class binary_reader
case 0xCA: // float 32
{
float number{};
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xCB: // float 64
{
double number{};
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xCC: // uint 8
{
std::uint8_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xCD: // uint 16
{
std::uint16_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xCE: // uint 32
{
std::uint32_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xCF: // uint 64
{
std::uint64_t number{};
return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
}
case 0xD0: // int 8
{
std::int8_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xD1: // int 16
{
std::int16_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xD2: // int 32
{
std::int32_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xD3: // int 64
{
std::int64_t number{};
return get_number(input_format_t::msgpack, number) && emit_signed(number);
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
}
case 0xDC: // array 16
@@ -15494,7 +15491,7 @@ class binary_reader
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i)))
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
{
return false;
}
@@ -15626,37 +15623,37 @@ class binary_reader
break;
}
std::uint8_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'U':
{
std::uint8_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'i':
{
std::int8_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'I':
{
std::int16_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'l':
{
std::int32_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'L':
{
std::int64_t number{};
return get_number(input_format, number) && emit_signed(number);
return get_number(input_format, number) && sax->number_integer(number);
}
case 'u':
@@ -15666,7 +15663,7 @@ class binary_reader
break;
}
std::uint16_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'm':
@@ -15676,7 +15673,7 @@ class binary_reader
break;
}
std::uint32_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'M':
@@ -15686,7 +15683,7 @@ class binary_reader
break;
}
std::uint64_t number{};
return get_number(input_format, number) && emit_unsigned(number);
return get_number(input_format, number) && sax->number_unsigned(number);
}
case 'h':
@@ -15744,13 +15741,13 @@ class binary_reader
case 'd':
{
float number{};
return get_number(input_format, number) && emit_float(input_format, number);
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 'D':
{
double number{};
return get_number(input_format, number) && emit_float(input_format, number);
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 'H':
@@ -16217,13 +16214,13 @@ class binary_reader
case 0x8E: // binary32
{
float number{};
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0x8F: // binary64
{
double number{};
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
}
case 0xF8:
@@ -16289,9 +16286,7 @@ class binary_reader
@brief pass an integer to the SAX parser
Non-negative integers are passed as unsigned, negative integers as signed
numbers, like the other binary formats do. A value that does not fit the
number type is passed as described for @ref emit_unsigned and
@ref emit_signed.
numbers, like the other binary formats do.
@param[in] number the integer
@return whether the SAX parser accepted the value
@@ -16300,9 +16295,9 @@ class binary_reader
{
if (number >= 0)
{
return emit_unsigned(static_cast<std::uint64_t>(number));
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return emit_signed(number);
return sax->number_integer(static_cast<number_integer_t>(number));
}
/*!
@@ -16359,7 +16354,8 @@ class binary_reader
value = (value << 8) | static_cast<std::int64_t>(current);
}
return emit_bon8_integer(negative ? -(value + offset) : value + offset);
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
}
/*!
@@ -16656,79 +16652,6 @@ class binary_reader
return true;
}
/*!
@brief pass a signed integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_integer_t is passed as number_unsigned_t if it is non-negative and
fits there, and as number_float_t otherwise. With the default number
types, every integer the binary formats can encode fits, so this only
matters for narrower custom number types.
@tparam NumberType a signed integer type
@param[in] number the integer
@return whether the SAX parser accepted the value
*/
template<typename NumberType>
bool emit_signed(const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
{
return sax->number_integer(static_cast<number_integer_t>(number));
}
if (value_in_range_of<number_unsigned_t>(number))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return sax->number_float(static_cast<number_float_t>(number), "");
}
/*!
@brief pass an unsigned integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_unsigned_t is passed as number_float_t.
@tparam NumberType an unsigned integer type
@param[in] number the integer
@return whether the SAX parser accepted the value
*/
template<typename NumberType>
bool emit_unsigned(const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return sax->number_float(static_cast<number_float_t>(number), "");
}
/*!
@brief pass a floating-point number read from the input to the SAX parser
Like the lexer does for JSON text, a finite value that overflows
number_float_t is rejected instead of silently becoming infinity. Infinity
and NaN in the input are passed on unchanged.
@tparam NumberType a floating-point type
@param[in] format the current format (for diagnostics)
@param[in] number the number
@return whether the SAX parser accepted the value
@throw out_of_range.406 if a finite @a number overflows number_float_t
*/
template<typename NumberType>
bool emit_float(const input_format_t format, const NumberType number)
{
const auto result = static_cast<number_float_t>(number);
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
{
return sax->parse_error(chars_read, get_token_string(),
out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
}
return sax->number_float(result, "");
}
/*!
@brief create a string by reading characters from the input
-116
View File
@@ -11,12 +11,7 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cmath>
#include <fstream>
#include <limits>
#include <map>
#include <string>
#include <vector>
#include "make_test_data_available.hpp"
TEST_CASE("Binary Formats" * doctest::skip())
@@ -229,114 +224,3 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
}
}
TEST_CASE("Binary formats with narrow number types")
{
// Numbers that do not fit the number types are handled like the lexer
// handles them in JSON text: an integer that fits neither integer type is
// stored as a floating-point number, and a finite floating-point number
// that overflows number_float_t is rejected with out_of_range.406.
using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
using bytes = std::vector<std::uint8_t>;
struct binary_format
{
const char* name;
bytes (*encode)(const json&);
narrow_json (*decode)(const bytes&, bool);
};
const std::vector<binary_format> formats =
{
{
"CBOR", [](const json & j) { return json::to_cbor(j); },
[](const bytes & v, bool allow_exceptions)
{
return narrow_json::from_cbor(v, true, allow_exceptions);
}
},
{
"MessagePack", [](const json & j) { return json::to_msgpack(j); },
[](const bytes & v, bool allow_exceptions)
{
return narrow_json::from_msgpack(v, true, allow_exceptions);
}
},
{
"UBJSON", [](const json & j) { return json::to_ubjson(j); },
[](const bytes & v, bool allow_exceptions)
{
return narrow_json::from_ubjson(v, true, allow_exceptions);
}
},
{
"BJData", [](const json & j) { return json::to_bjdata(j); },
[](const bytes & v, bool allow_exceptions)
{
return narrow_json::from_bjdata(v, true, allow_exceptions);
}
},
{
// BSON can only store numbers as object members
"BSON", [](const json & j) { return json::to_bson(json{{"a", j}}); },
[](const bytes & v, bool allow_exceptions)
{
const auto result = narrow_json::from_bson(v, true, allow_exceptions);
return result.is_discarded() ? result : result.at("a");
}
},
{
"BON8", [](const json & j) { return json::to_bon8(j); },
[](const bytes & v, bool allow_exceptions)
{
return narrow_json::from_bon8(v, true, allow_exceptions);
}
},
};
for (const auto& format : formats)
{
const std::string name = format.name;
INFO("format := ", name);
const auto roundtrip = [&format](const json & j)
{
return format.decode(format.encode(j), true);
};
// integers that fit keep their type
CHECK(roundtrip(json(-5)).is_number_integer());
CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
// integers that fit neither integer type are stored as float
CHECK(roundtrip(json(5000000000u)).is_number_float());
CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
{
CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
}
CHECK(roundtrip(json(-3000000000)).is_number_float());
CHECK(roundtrip(json(-3000000000)).get<float>() == -3000000000.0f);
CHECK(roundtrip(json(-5000000000)).is_number_float());
CHECK(roundtrip(json(-5000000000)).get<float>() == -5000000000.0f);
// floating-point numbers that fit
CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
std::numeric_limits<double>::infinity());
CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
// infinity and NaN are passed on
CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
// finite floating-point numbers that overflow number_float_t are rejected
const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
+ " value: number overflow";
CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
}
}
+14 -16
View File
@@ -3146,8 +3146,7 @@ TEST_CASE("Tagged values")
// CBOR encodes negative integers as: result = -1 - n
// For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
// the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
// When n > INT64_MAX, the result exceeds int64_t range and is stored
// as a floating-point number, as the lexer does for JSON text.
// When n > INT64_MAX, the result exceeds int64_t range and is rejected.
SECTION("n = 0 is valid (result = -1)")
{
@@ -3168,34 +3167,33 @@ TEST_CASE("Tagged values")
CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
}
SECTION("n = INT64_MAX + 1 is stored as float")
SECTION("n = INT64_MAX + 1 is rejected (overflow)")
{
// n = INT64_MAX + 1 (0x8000000000000000)
// result = -1 - n = -9223372036854775809, which exceeds int64_t range;
// the nearest double is -9223372036854775808.0
// result = -1 - n = -9223372036854775809, which exceeds int64_t range
const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
const auto result = json::from_cbor(input);
CHECK(result.is_number_float());
CHECK(result.get<double>() == -9223372036854775808.0);
CHECK(result == json::parse("-9223372036854775809"));
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
"[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
json::parse_error);
}
SECTION("n = UINT64_MAX is stored as float")
SECTION("n = UINT64_MAX is rejected (overflow)")
{
// n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
// result = -1 - n = -18446744073709551616, which exceeds int64_t range
const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
const auto result = json::from_cbor(input);
CHECK(result.is_number_float());
CHECK(result.get<double>() == -18446744073709551616.0);
CHECK(result == json::parse("-18446744073709551616"));
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
"[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
json::parse_error);
}
SECTION("overflow with allow_exceptions=false is not an error")
SECTION("overflow with allow_exceptions=false returns discarded")
{
const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
const auto result = json::from_cbor(input, true, false);
CHECK(result.is_number_float());
CHECK(result.is_discarded());
}
}