mirror of
https://github.com/nlohmann/json.git
synced 2026-10-10 16:37:14 +00:00
Fix silent wrong results for custom number types
Two bugs for custom number types that compile on develop (found while analysing #3578): - compare_integer_with_float() took the signedness of the integer type from std::is_signed, which is false for class types such as absl::int128 or boost::multiprecision::cpp_int. Any float below zero then compared less than every integer, e.g. json(int128(-5)) < json(-2.5) was false. The signedness now comes from std::numeric_limits, like the digits used for the range bound. - With number_integer_t/number_unsigned_t wider than 64 bits (e.g. __int128), CBOR, MessagePack, and BSON silently truncated integers beyond 64 bits (to_cbor of 2^100 read back as 0), BJData truncated unsigned ones with the 'M' marker and could encode truncated ND-array elements and dimensions, and BON8 did not compile (std::to_string is ambiguous for __int128). The writers now throw out_of_range.407 when an integer does not fit the format's range ([-2^64, 2^64-1] for CBOR, [-2^63, 2^64-1] for MessagePack, int64/uint64 for BSON, int64 for BON8); BJData writes such unsigned values as high-precision numbers, as it already did for signed ones and UBJSON does for both, and falls back to a plain object for the ND-array. The checks use std::numeric_limits digits and compile away when the number types are at most 64 bits wide, so the default types pay nothing (the CBOR, MessagePack, and BSON writers compile to identical code). The new unit-custom-number-types.cpp tests the comparison with a small class-type integer and the writers with __int128 where the standard library supports it as an integral type. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
10 files changed
+680
-47
No files matched your search
@@ -123,4 +123,6 @@ Linear in the size of the JSON value `j`.
|
||||
that is not valid UTF-8 unchanged, as before; `strict` (the default if
|
||||
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
|
||||
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
|
||||
array or object was silently skipped, producing invalid BJData.
|
||||
array or object was silently skipped, producing invalid BJData.
|
||||
- Writes unsigned integers wider than 64 bits as high-precision numbers since version 3.13.0; previously, they were
|
||||
silently truncated to 64 bits.
|
||||
@@ -37,8 +37,9 @@ With (2), the bytes written before the exception remain in the output adapter.
|
||||
|
||||
## Exceptions
|
||||
|
||||
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an unsigned integer
|
||||
above 9223372036854775807, which BON8 cannot represent
|
||||
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer outside
|
||||
the range of int64 (an unsigned integer above 9223372036854775807, or, with a number type wider than 64 bits, any
|
||||
integer beyond int64), which BON8 cannot represent
|
||||
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if `j` contains a string that is not
|
||||
valid UTF-8
|
||||
|
||||
|
||||
@@ -47,6 +47,10 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
|
||||
- Throws [`type_error.317`](../../home/exceptions.md#jsonexceptiontype_error317) if the top-level type of the JSON value
|
||||
is not an object; example: `"to serialize to BSON, top-level type must be object, but is string"`
|
||||
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains a signed integer
|
||||
outside the range of int64 or an unsigned integer outside the range of uint64, which is only possible with a number
|
||||
type wider than 64 bits; example:
|
||||
`"integer number 9223372036854775808 cannot be represented by BSON as it does not fit int64"`
|
||||
- Throws [`out_of_range.409`](../../home/exceptions.md#jsonexceptionout_of_range409) if a key in the JSON object contains
|
||||
a null byte (code point U+0000); example: `"BSON key cannot contain code point U+0000 (at byte 2)"`
|
||||
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a document, array,
|
||||
@@ -119,3 +123,5 @@ pass before anything is written.
|
||||
that is not valid UTF-8 unchanged, as before; `strict` (the default if
|
||||
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316` before anything
|
||||
is written.
|
||||
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
|
||||
number type wider than 64 bits were silently truncated.
|
||||
@@ -46,6 +46,9 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
|
||||
## Exceptions
|
||||
|
||||
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer
|
||||
outside [-2^64, 2^64-1], which is only possible with a number type wider than 64 bits; example:
|
||||
`"integer number 18446744073709551616 cannot be represented by CBOR as it does not fit [-2^64, 2^64-1]"`
|
||||
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
|
||||
not valid UTF-8 and `error_handler` is `strict` (the default only if
|
||||
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
|
||||
@@ -90,3 +93,5 @@ Linear in the size of the JSON value `j`.
|
||||
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
|
||||
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
|
||||
array or object was silently skipped, producing invalid CBOR.
|
||||
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
|
||||
number type wider than 64 bits were silently truncated.
|
||||
@@ -46,6 +46,9 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
|
||||
## Exceptions
|
||||
|
||||
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer
|
||||
outside [-2^63, 2^64-1], which is only possible with a number type wider than 64 bits; example:
|
||||
`"integer number 18446744073709551616 cannot be represented by MessagePack as it does not fit [-2^63, 2^64-1]"`
|
||||
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a string, binary
|
||||
value, array, or object exceeds 4294967295, the maximum MessagePack can store; example:
|
||||
`"MessagePack length 4294967296 exceeds maximum of 4294967295"`
|
||||
@@ -112,3 +115,5 @@ Linear in the size of the JSON value `j`.
|
||||
`number_unsigned_t`.
|
||||
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
|
||||
array or object was silently skipped, producing invalid MessagePack.
|
||||
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
|
||||
number type wider than 64 bits were silently truncated.
|
||||
@@ -906,14 +906,34 @@ double-precision number when `number_float_t` is `#!cpp float`.
|
||||
|
||||
### json.exception.out_of_range.407
|
||||
|
||||
This exception previously indicated that the UBJSON and BSON binary formats did not support integer numbers greater than
|
||||
9223372036854775807 due to limitations in the implemented mapping. However, these limitations have since been resolved,
|
||||
and this exception no longer occurs.
|
||||
An integer number cannot be represented by the binary format it is serialized to:
|
||||
|
||||
!!! success "Exception cannot occur any more"
|
||||
- [BON8](../features/binary_formats/bon8.md) only stores integers that fit into int64.
|
||||
- [CBOR](../features/binary_formats/cbor.md), [MessagePack](../features/binary_formats/msgpack.md), and
|
||||
[BSON](../features/binary_formats/bson.md) store integers in at most 64 bits. With the default number types, every
|
||||
integer fits, but a [`number_integer_t`](../api/basic_json/number_integer_t.md) or
|
||||
[`number_unsigned_t`](../api/basic_json/number_unsigned_t.md) wider than 64 bits (e.g., `__int128`) can hold values
|
||||
outside the range of the format: [-2^64, 2^64-1] for CBOR, [-2^63, 2^64-1] for MessagePack, and the range of int64
|
||||
(signed integers) or uint64 (unsigned integers) for BSON.
|
||||
|
||||
- Since version 3.9.0, integer numbers beyond int64 are serialized as high-precision UBJSON numbers.
|
||||
- Since version 3.12.0, integer numbers beyond int64 are serialized as uint64 BSON numbers.
|
||||
[UBJSON](../features/binary_formats/ubjson.md) and [BJData](../features/binary_formats/bjdata.md) never throw this
|
||||
exception, because they serialize integers beyond 64 bits as high-precision numbers.
|
||||
|
||||
!!! failure "Example messages"
|
||||
|
||||
```
|
||||
integer number 9223372036854775808 cannot be represented by BON8 as it does not fit int64
|
||||
```
|
||||
```
|
||||
integer number 1267650600228229401496703205376 cannot be represented by CBOR as it does not fit [-2^64, 2^64-1]
|
||||
```
|
||||
|
||||
!!! note
|
||||
|
||||
Before version 3.13.0, CBOR, MessagePack, and BSON silently truncated integers wider than 64 bits, and BJData
|
||||
truncated unsigned integers wider than 64 bits. This exception was previously thrown by UBJSON and BSON for
|
||||
integers greater than 9223372036854775807; since version 3.9.0, such integers are serialized as high-precision
|
||||
UBJSON numbers, and since version 3.12.0 as uint64 BSON numbers.
|
||||
|
||||
### json.exception.out_of_range.408
|
||||
|
||||
|
||||
Reference in new issue
Block a user