Fix silent wrong results for custom number types

Two bugs for custom number types that compile on develop (found while
analysing #3578):

- compare_integer_with_float() took the signedness of the integer type
  from std::is_signed, which is false for class types such as
  absl::int128 or boost::multiprecision::cpp_int. Any float below zero
  then compared less than every integer, e.g. json(int128(-5)) <
  json(-2.5) was false. The signedness now comes from
  std::numeric_limits, like the digits used for the range bound.

- With number_integer_t/number_unsigned_t wider than 64 bits (e.g.
  __int128), CBOR, MessagePack, and BSON silently truncated integers
  beyond 64 bits (to_cbor of 2^100 read back as 0), BJData truncated
  unsigned ones with the 'M' marker and could encode truncated ND-array
  elements and dimensions, and BON8 did not compile (std::to_string is
  ambiguous for __int128). The writers now throw out_of_range.407 when
  an integer does not fit the format's range ([-2^64, 2^64-1] for CBOR,
  [-2^63, 2^64-1] for MessagePack, int64/uint64 for BSON, int64 for
  BON8); BJData writes such unsigned values as high-precision numbers,
  as it already did for signed ones and UBJSON does for both, and falls
  back to a plain object for the ND-array. The checks use
  std::numeric_limits digits and compile away when the number types are
  at most 64 bits wide, so the default types pay nothing (the CBOR,
  MessagePack, and BSON writers compile to identical code).

The new unit-custom-number-types.cpp tests the comparison with a small
class-type integer and the writers with __int128 where the standard
library supports it as an integral type.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-10 16:55:26 +02:00
1 parent f8b47ff6f6
commit 4001fe68b9
10 files changed
+680 -47

No files matched your search

+3 -1
View File
@@ -123,4 +123,6 @@ Linear in the size of the JSON value `j`.
that is not valid UTF-8 unchanged, as before; `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid BJData.
array or object was silently skipped, producing invalid BJData.
- Writes unsigned integers wider than 64 bits as high-precision numbers since version 3.13.0; previously, they were
silently truncated to 64 bits.
+3 -2
View File
@@ -37,8 +37,9 @@ With (2), the bytes written before the exception remain in the output adapter.
## Exceptions
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an unsigned integer
above 9223372036854775807, which BON8 cannot represent
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer outside
the range of int64 (an unsigned integer above 9223372036854775807, or, with a number type wider than 64 bits, any
integer beyond int64), which BON8 cannot represent
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if `j` contains a string that is not
valid UTF-8
@@ -47,6 +47,10 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
- Throws [`type_error.317`](../../home/exceptions.md#jsonexceptiontype_error317) if the top-level type of the JSON value
is not an object; example: `"to serialize to BSON, top-level type must be object, but is string"`
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains a signed integer
outside the range of int64 or an unsigned integer outside the range of uint64, which is only possible with a number
type wider than 64 bits; example:
`"integer number 9223372036854775808 cannot be represented by BSON as it does not fit int64"`
- Throws [`out_of_range.409`](../../home/exceptions.md#jsonexceptionout_of_range409) if a key in the JSON object contains
a null byte (code point U+0000); example: `"BSON key cannot contain code point U+0000 (at byte 2)"`
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a document, array,
@@ -119,3 +123,5 @@ pass before anything is written.
that is not valid UTF-8 unchanged, as before; `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316` before anything
is written.
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
number type wider than 64 bits were silently truncated.
@@ -46,6 +46,9 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
## Exceptions
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer
outside [-2^64, 2^64-1], which is only possible with a number type wider than 64 bits; example:
`"integer number 18446744073709551616 cannot be represented by CBOR as it does not fit [-2^64, 2^64-1]"`
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
not valid UTF-8 and `error_handler` is `strict` (the default only if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
@@ -90,3 +93,5 @@ Linear in the size of the JSON value `j`.
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid CBOR.
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
number type wider than 64 bits were silently truncated.
@@ -46,6 +46,9 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
## Exceptions
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer
outside [-2^63, 2^64-1], which is only possible with a number type wider than 64 bits; example:
`"integer number 18446744073709551616 cannot be represented by MessagePack as it does not fit [-2^63, 2^64-1]"`
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a string, binary
value, array, or object exceeds 4294967295, the maximum MessagePack can store; example:
`"MessagePack length 4294967296 exceeds maximum of 4294967295"`
@@ -112,3 +115,5 @@ Linear in the size of the JSON value `j`.
`number_unsigned_t`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid MessagePack.
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
number type wider than 64 bits were silently truncated.
+26 -6
View File
@@ -906,14 +906,34 @@ double-precision number when `number_float_t` is `#!cpp float`.
### json.exception.out_of_range.407
This exception previously indicated that the UBJSON and BSON binary formats did not support integer numbers greater than
9223372036854775807 due to limitations in the implemented mapping. However, these limitations have since been resolved,
and this exception no longer occurs.
An integer number cannot be represented by the binary format it is serialized to:
!!! success "Exception cannot occur any more"
- [BON8](../features/binary_formats/bon8.md) only stores integers that fit into int64.
- [CBOR](../features/binary_formats/cbor.md), [MessagePack](../features/binary_formats/msgpack.md), and
[BSON](../features/binary_formats/bson.md) store integers in at most 64 bits. With the default number types, every
integer fits, but a [`number_integer_t`](../api/basic_json/number_integer_t.md) or
[`number_unsigned_t`](../api/basic_json/number_unsigned_t.md) wider than 64 bits (e.g., `__int128`) can hold values
outside the range of the format: [-2^64, 2^64-1] for CBOR, [-2^63, 2^64-1] for MessagePack, and the range of int64
(signed integers) or uint64 (unsigned integers) for BSON.
- Since version 3.9.0, integer numbers beyond int64 are serialized as high-precision UBJSON numbers.
- Since version 3.12.0, integer numbers beyond int64 are serialized as uint64 BSON numbers.
[UBJSON](../features/binary_formats/ubjson.md) and [BJData](../features/binary_formats/bjdata.md) never throw this
exception, because they serialize integers beyond 64 bits as high-precision numbers.
!!! failure "Example messages"
```
integer number 9223372036854775808 cannot be represented by BON8 as it does not fit int64
```
```
integer number 1267650600228229401496703205376 cannot be represented by CBOR as it does not fit [-2^64, 2^64-1]
```
!!! note
Before version 3.13.0, CBOR, MessagePack, and BSON silently truncated integers wider than 64 bits, and BJData
truncated unsigned integers wider than 64 bits. This exception was previously thrown by UBJSON and BSON for
integers greater than 9223372036854775807; since version 3.9.0, such integers are serialized as high-precision
UBJSON numbers, and since version 3.12.0 as uint64 BSON numbers.
### json.exception.out_of_range.408