Share the IEEE half-precision decoder between CBOR and BJData

binary_reader had two ~45-line copies of the IEEE 754 half-precision decoder: CBOR's case 0xF9 and BJData's case 'h'. Once formatting is normalised, the two blocks were identical except for the byte order used to assemble the 16-bit half (CBOR is big endian, BJData is little endian). Any future change to half-float decoding had to be made and kept in sync in both places. Add one get_half_float(format, little_endian) helper that does the two get()/unexpect_eof() reads, assembles the half in the requested byte order, decodes it per RFC 8949 Appendix D, and calls sax->number_float. Both cases now just call it with their byte order; the BJData case keeps its bjdata-only guard. Behavior, the public API and the ABI are unchanged. Verified with a scratch probe comparing the old and new decoders bit-for-bit (NaN by isnan()) over all 65536 wire byte pairs, in both formats, and by running unit-cbor and unit-bjdata (offline, against the stubbed test_data.hpp). Part of #5711 Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Remove dead get_char parameters in binary_reader
2026-09-30 11:40:30 +00:00 · 2026-09-30 10:19:57 +02:00 · 2026-09-30 10:19:56 +02:00 · 2026-09-30 10:19:54 +02:00 · 2026-09-30 09:37:28 +02:00
11 changed files with 681 additions and 1135 deletions
@@ -55,10 +55,6 @@ This implementation does exactly follow this approach, as it uses double precisi
 smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
 and be serialized to `null`.

-During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
-`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
-example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
-
 #### Storage

 Floating-point number values are stored directly inside a `basic_json` type.
@@ -47,9 +47,8 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va

 When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
 the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
-range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
-formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
-or [`number_float_t`](number_float_t.md).
+range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
+will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).

 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
@@ -48,9 +48,8 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default

 When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
 the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
-when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
-integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
-[`number_float_t`](number_float_t.md).
+when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
+as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).

 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
@@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows:
 !!! warning "Negative integer overflow"

    CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
-    result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
-    a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
-    stored as `-1.8446744073709552e+19`.
+    result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
+    [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
+    silently.

 !!! warning "Object keys"

@@ -331,6 +331,9 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
    [json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
    ```
    ```
+    [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
+    ```
+    ```
    [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
    ```

@@ -851,18 +854,13 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme

 ### json.exception.out_of_range.406

-A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
-finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
-double-precision number when `number_float_t` is `#!cpp float`.
+A parsed number could not be stored as without changing it to NaN or INF.

-!!! failure "Example messages"
+!!! failure "Example message"

    ```
    number overflow parsing '10E1000'
    ```
-    ```
-    [json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
-    ```

 ### json.exception.out_of_range.407

@@ -148,7 +148,7 @@ class binary_reader
                break;

            case input_format_t::cbor:
-                result = parse_cbor_internal(true, tag_handler);
+                result = parse_cbor_internal(tag_handler);
                break;

            case input_format_t::msgpack:
@@ -559,7 +559,7 @@ class binary_reader
            case 0x01: // double
            {
                double number{};
-                return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
+                return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 0x02: // string
@@ -600,19 +600,19 @@ class binary_reader
            case 0x10: // int32
            {
                std::int32_t value{};
-                return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
+                return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
            }

            case 0x12: // int64
            {
                std::int64_t value{};
-                return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
+                return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
            }

            case 0x11: // uint64
            {
                std::uint64_t value{};
-                return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(input_format_t::bson, value);
+                return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
            }

            default: // anything else is not supported (yet)
@@ -638,19 +638,14 @@ class binary_reader
        {
            return false;
        }
-
-        // the value is -1 - number, which fits into number_integer_t
-        // whenever number does
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
+        const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
+        if (number > max_val)
        {
-            return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
+            return sax->parse_error(chars_read, get_token_string(),
+                                    parse_error::create(112, chars_read,
+                                            exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
        }
-
-        // like the lexer does for JSON text, store a value too small for
-        // number_integer_t as number_float_t; compute it as long double so
-        // that emit_float sees a finite value and can detect an overflow of
-        // number_float_t
-        return emit_float(input_format_t::cbor, static_cast<long double>(-1) - static_cast<long double>(number));
+        return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
    }

    /*!
@@ -707,25 +702,25 @@ class binary_reader
            case 0x18: // Unsigned integer (one-byte uint8_t follows)
            {
                std::uint8_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
            }

            case 0x19: // Unsigned integer (two-byte uint16_t follows)
            {
                std::uint16_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
            }

            case 0x1A: // Unsigned integer (four-byte uint32_t follows)
            {
                std::uint32_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
            }

            case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
            {
                std::uint64_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
            }

            // Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -1120,63 +1115,18 @@ class binary_reader
                return sax->null();

            case 0xF9: // Half-Precision Float (two-byte IEEE 754)
-            {
-                const auto byte1_raw = get();
-                if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "number")))
-                {
-                    return false;
-                }
-                const auto byte2_raw = get();
-                if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "number")))
-                {
-                    return false;
-                }
-
-                const auto byte1 = static_cast<unsigned char>(byte1_raw);
-                const auto byte2 = static_cast<unsigned char>(byte2_raw);
-
-                // Code from RFC 8949, Appendix D, Figure 3:
-                // As half-precision floating-point numbers were only added
-                // to IEEE 754 in 2008, today's programming platforms often
-                // still only have limited support for them. It is very
-                // easy to include at least decoding support for them even
-                // without such support. An example of a small decoder for
-                // half-precision floating-point numbers in the C language
-                // is shown in Fig. 3.
-                const auto half = static_cast<unsigned int>((byte1 << 8u) + byte2);
-                const double val = [&half]
-                {
-                    const int exp = (half >> 10u) & 0x1Fu;
-                    const unsigned int mant = half & 0x3FFu;
-                    JSON_ASSERT(exp <= 31);
-                    JSON_ASSERT(mant <= 1023);
-                    switch (exp)
-                    {
-                        case 0:
-                            return std::ldexp(mant, -24);
-                        case 31:
-                            return (mant == 0)
-                            ? std::numeric_limits<double>::infinity()
-                            : std::numeric_limits<double>::quiet_NaN();
-                        default:
-                            return std::ldexp(mant + 1024, exp - 25);
-                    }
-                }();
-                return sax->number_float((half & 0x8000u) != 0
-                                         ? static_cast<number_float_t>(-val)
-                                         : static_cast<number_float_t>(val), "");
-            }
+                return get_half_float(input_format_t::cbor, false);

            case 0xFA: // Single-Precision Float (four-byte IEEE 754)
            {
                float number{};
-                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
            {
                double number{};
-                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            default: // anything else (0xFF is handled inside the other types)
@@ -1576,19 +1526,15 @@ class binary_reader
    enclosing container after each element, so that the nesting depth of the
    input costs heap rather than native stack (see #5104).

-    @param[in] get_char  whether a new character should be retrieved from the
-                         input (true) or whether the last read character
-                         @a current should be considered instead
    @param[in] tag_handler how CBOR tags should be treated

    @return whether reading the value succeeded
    */
-    bool parse_cbor_internal(const bool get_char,
-                             const cbor_tag_handler_t tag_handler)
+    bool parse_cbor_internal(const cbor_tag_handler_t tag_handler)
    {
        // whether the next value starts at a fresh byte or at the one already
        // read into `current`
-        bool fetch = get_char;
+        bool fetch = true;

        // the key currently being read; hoisted out of the loop so that its
        // capacity is reused across elements and across nesting levels
@@ -1681,9 +1627,6 @@ class binary_reader
    // MsgPack //
    /////////////

-    /*!
-    @return whether a valid MessagePack value was passed to the SAX parser
-    */
    /*!
    @brief read one MessagePack value

@@ -1940,61 +1883,61 @@ class binary_reader
            case 0xCA: // float 32
            {
                float number{};
-                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 0xCB: // float 64
            {
                double number{};
-                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 0xCC: // uint 8
            {
                std::uint8_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
            }

            case 0xCD: // uint 16
            {
                std::uint16_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
            }

            case 0xCE: // uint 32
            {
                std::uint32_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
            }

            case 0xCF: // uint 64
            {
                std::uint64_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
            }

            case 0xD0: // int 8
            {
                std::int8_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
            }

            case 0xD1: // int 16
            {
                std::int16_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
            }

            case 0xD2: // int 32
            {
                std::int32_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
            }

            case 0xD3: // int 64
            {
                std::int64_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
            }

            case 0xDC: // array 16
@@ -2428,20 +2371,16 @@ class binary_reader
    ////////////

    /*!
-    @param[in] get_char  whether a new character should be retrieved from the
-                         input (true, default) or whether the last read
-                         character should be considered instead
-
    @return whether a valid UBJSON value was passed to the SAX parser
    */
-    bool parse_ubjson_internal(const bool get_char = true)
+    bool parse_ubjson_internal()
    {
        // the key currently being read; hoisted out of the loop so that its
        // capacity is reused across elements and across nesting levels
        string_t key;

        // the type marker of the value to read next
-        char_int_type prefix = get_char ? get_ignore_noop() : current;
+        char_int_type prefix = get_ignore_noop();

        while (true)
        {
@@ -2927,7 +2866,7 @@ class binary_reader
                        {
                            return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
                        }
-                        if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(input_format, i)))
+                        if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
                        {
                            return false;
                        }
@@ -3059,37 +2998,37 @@ class binary_reader
                    break;
                }
                std::uint8_t number{};
-                return get_number(input_format, number) && emit_unsigned(input_format, number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
            }

            case 'U':
            {
                std::uint8_t number{};
-                return get_number(input_format, number) && emit_unsigned(input_format, number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
            }

            case 'i':
            {
                std::int8_t number{};
-                return get_number(input_format, number) && emit_signed(input_format, number);
+                return get_number(input_format, number) && sax->number_integer(number);
            }

            case 'I':
            {
                std::int16_t number{};
-                return get_number(input_format, number) && emit_signed(input_format, number);
+                return get_number(input_format, number) && sax->number_integer(number);
            }

            case 'l':
            {
                std::int32_t number{};
-                return get_number(input_format, number) && emit_signed(input_format, number);
+                return get_number(input_format, number) && sax->number_integer(number);
            }

            case 'L':
            {
                std::int64_t number{};
-                return get_number(input_format, number) && emit_signed(input_format, number);
+                return get_number(input_format, number) && sax->number_integer(number);
            }

            case 'u':
@@ -3099,7 +3038,7 @@ class binary_reader
                    break;
                }
                std::uint16_t number{};
-                return get_number(input_format, number) && emit_unsigned(input_format, number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
            }

            case 'm':
@@ -3109,7 +3048,7 @@ class binary_reader
                    break;
                }
                std::uint32_t number{};
-                return get_number(input_format, number) && emit_unsigned(input_format, number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
            }

            case 'M':
@@ -3119,7 +3058,7 @@ class binary_reader
                    break;
                }
                std::uint64_t number{};
-                return get_number(input_format, number) && emit_unsigned(input_format, number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
            }

            case 'h':
@@ -3128,62 +3067,19 @@ class binary_reader
                {
                    break;
                }
-                const auto byte1_raw = get();
-                if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number")))
-                {
-                    return false;
-                }
-                const auto byte2_raw = get();
-                if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number")))
-                {
-                    return false;
-                }
-
-                const auto byte1 = static_cast<unsigned char>(byte1_raw);
-                const auto byte2 = static_cast<unsigned char>(byte2_raw);
-
-                // Code from RFC 8949, Appendix D, Figure 3:
-                // As half-precision floating-point numbers were only added
-                // to IEEE 754 in 2008, today's programming platforms often
-                // still only have limited support for them. It is very
-                // easy to include at least decoding support for them even
-                // without such support. An example of a small decoder for
-                // half-precision floating-point numbers in the C language
-                // is shown in Fig. 3.
-                const auto half = static_cast<unsigned int>((byte2 << 8u) + byte1);
-                const double val = [&half]
-                {
-                    const int exp = (half >> 10u) & 0x1Fu;
-                    const unsigned int mant = half & 0x3FFu;
-                    JSON_ASSERT(exp <= 31);
-                    JSON_ASSERT(mant <= 1023);
-                    switch (exp)
-                    {
-                        case 0:
-                            return std::ldexp(mant, -24);
-                        case 31:
-                            return (mant == 0)
-                            ? std::numeric_limits<double>::infinity()
-                            : std::numeric_limits<double>::quiet_NaN();
-                        default:
-                            return std::ldexp(mant + 1024, exp - 25);
-                    }
-                }();
-                return sax->number_float((half & 0x8000u) != 0
-                                         ? static_cast<number_float_t>(-val)
-                                         : static_cast<number_float_t>(val), "");
+                return get_half_float(input_format, true);
            }

            case 'd':
            {
                float number{};
-                return get_number(input_format, number) && emit_float(input_format, number);
+                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 'D':
            {
                double number{};
-                return get_number(input_format, number) && emit_float(input_format, number);
+                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 'H':
@@ -3650,13 +3546,13 @@ class binary_reader
            case 0x8E: // binary32
            {
                float number{};
-                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
+                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 0x8F: // binary64
            {
                double number{};
-                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
+                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
            }

            case 0xF8:
@@ -3722,9 +3618,7 @@ class binary_reader
    @brief pass an integer to the SAX parser

    Non-negative integers are passed as unsigned, negative integers as signed
-    numbers, like the other binary formats do. A value that does not fit the
-    number type is passed as described for @ref emit_unsigned and
-    @ref emit_signed.
+    numbers, like the other binary formats do.

    @param[in] number  the integer
    @return whether the SAX parser accepted the value
@@ -3733,9 +3627,9 @@ class binary_reader
    {
        if (number >= 0)
        {
-            return emit_unsigned(input_format_t::bon8, static_cast<std::uint64_t>(number));
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
        }
-        return emit_signed(input_format_t::bon8, number);
+        return sax->number_integer(static_cast<number_integer_t>(number));
    }

    /*!
@@ -3792,7 +3686,8 @@ class binary_reader
            value = (value << 8) | static_cast<std::int64_t>(current);
        }

-        return emit_bon8_integer(negative ? -(value + offset) : value + offset);
+        return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
+               : sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
    }

    /*!
@@ -4090,85 +3985,65 @@ class binary_reader
    }

    /*!
-    @brief pass a signed integer read from the input to the SAX parser
+    @brief read and decode an IEEE 754 half-precision (16-bit) float

-    Like the lexer does for JSON text, a value that does not fit into
-    number_integer_t is passed as number_unsigned_t if it is non-negative and
-    fits there, and as number_float_t otherwise. With the default number
-    types, every integer the binary formats can encode fits, so this only
-    matters for narrower custom number types.
+    Used by CBOR (big endian) and BJData (little endian); the two formats
+    only differ in the byte order of the two bytes that make up the half.

-    @tparam NumberType a signed integer type
-    @param[in] format  the current format (for diagnostics)
-    @param[in] number  the integer
-    @return whether the SAX parser accepted the value
+    @param[in] format       the current format (for diagnostics)
+    @param[in] little_endian whether the two bytes are little endian (BJData)
+                             or big endian (CBOR)

-    @throw out_of_range.406 if @a number overflows number_float_t (see
-           @ref emit_float)
+    @return whether reading and decoding succeeded
    */
-    template<typename NumberType>
-    bool emit_signed(const input_format_t format, const NumberType number)
+    bool get_half_float(const input_format_t format, const bool little_endian)
    {
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
+        const auto byte1_raw = get();
+        if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number")))
        {
-            return sax->number_integer(static_cast<number_integer_t>(number));
+            return false;
        }
-        if (value_in_range_of<number_unsigned_t>(number))
+        const auto byte2_raw = get();
+        if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number")))
        {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+            return false;
        }
-        return emit_float(format, number);
-    }

-    /*!
-    @brief pass an unsigned integer read from the input to the SAX parser
+        const auto byte1 = static_cast<unsigned char>(byte1_raw);
+        const auto byte2 = static_cast<unsigned char>(byte2_raw);

-    Like the lexer does for JSON text, a value that does not fit into
-    number_unsigned_t is passed as number_float_t.
-
-    @tparam NumberType an unsigned integer type
-    @param[in] format  the current format (for diagnostics)
-    @param[in] number  the integer
-    @return whether the SAX parser accepted the value
-
-    @throw out_of_range.406 if @a number overflows number_float_t (see
-           @ref emit_float)
-    */
-    template<typename NumberType>
-    bool emit_unsigned(const input_format_t format, const NumberType number)
-    {
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
+        // Code from RFC 8949, Appendix D, Figure 3:
+        // As half-precision floating-point numbers were only added
+        // to IEEE 754 in 2008, today's programming platforms often
+        // still only have limited support for them. It is very
+        // easy to include at least decoding support for them even
+        // without such support. An example of a small decoder for
+        // half-precision floating-point numbers in the C language
+        // is shown in Fig. 3.
+        const auto half = little_endian
+                          ? static_cast<unsigned int>((byte2 << 8u) + byte1)
+                          : static_cast<unsigned int>((byte1 << 8u) + byte2);
+        const double val = [&half]
        {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
-        }
-        return emit_float(format, number);
-    }
-
-    /*!
-    @brief pass a floating-point number read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a finite value that overflows
-    number_float_t is rejected instead of silently becoming infinity. Infinity
-    and NaN in the input are passed on unchanged. Integers only overflow if
-    number_float_t cannot represent 2^64, e.g., a half-precision type.
-
-    @tparam NumberType a floating-point or integer type
-    @param[in] format  the current format (for diagnostics)
-    @param[in] number  the number
-    @return whether the SAX parser accepted the value
-
-    @throw out_of_range.406 if a finite @a number overflows number_float_t
-    */
-    template<typename NumberType>
-    bool emit_float(const input_format_t format, const NumberType number)
-    {
-        const auto result = static_cast<number_float_t>(number);
-        if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
-        {
-            return sax->parse_error(chars_read, get_token_string(),
-                                    out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
-        }
-        return sax->number_float(result, "");
+            const int exp = (half >> 10u) & 0x1Fu;
+            const unsigned int mant = half & 0x3FFu;
+            JSON_ASSERT(exp <= 31);
+            JSON_ASSERT(mant <= 1023);
+            switch (exp)
+            {
+                case 0:
+                    return std::ldexp(mant, -24);
+                case 31:
+                    return (mant == 0)
+                    ? std::numeric_limits<double>::infinity()
+                    : std::numeric_limits<double>::quiet_NaN();
+                default:
+                    return std::ldexp(mant + 1024, exp - 25);
+            }
+        }();
+        return sax->number_float((half & 0x8000u) != 0
+                                 ? static_cast<number_float_t>(-val)
+                                 : static_cast<number_float_t>(val), "");
    }

    /*!
@@ -115,7 +115,7 @@ class binary_writer

    /*!
    @param[in] j  JSON value to serialize
-    @pre       j.type() == value_t::object
+    @throw type_error.317 if @a j is not an object
    */
    void write_bson(const BasicJsonType& j)
    {
@@ -238,6 +238,13 @@ class binary_writer
            {
                if (j.m_data.m_value.binary->has_subtype())
                {
+                    // The subtype is always written as a tag with a 0xD8..0xDB
+                    // head, never in the one-byte form 0xC0..0xD7 that CBOR
+                    // allows for tags 0..23 (so this is not write_cbor_head).
+                    // binary_reader with cbor_tag_handler_t::store only turns
+                    // 0xD8..0xDB into a subtype and ignores the one-byte tags,
+                    // so the shorter form would lose subtypes 0..23 on a round
+                    // trip.
                    if (j.m_data.m_value.binary->subtype() <= (std::numeric_limits<std::uint8_t>::max)())
                    {
                        write_number(static_cast<std::uint8_t>(0xd8));
@@ -1487,202 +1494,122 @@ class binary_writer
        write_number(n, use_bjdata);
    }

-    // UBJSON: write number (unsigned integer)
+    // UBJSON: write number (integer)
    template<typename NumberType, typename std::enable_if<
-                 std::is_unsigned<NumberType>::value, int>::type = 0>
+                 std::is_integral<NumberType>::value, int>::type = 0>
    void write_number_with_ubjson_prefix(const NumberType n,
                                         const bool add_prefix,
                                         const bool use_bjdata)
    {
-        if (n <= static_cast<std::uint64_t>((std::numeric_limits<std::int8_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('i'));  // int8
-            }
-            write_number(static_cast<std::uint8_t>(n), use_bjdata);
-        }
-        else if (n <= (std::numeric_limits<std::uint8_t>::max)())
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('U'));  // uint8
-            }
-            write_number(static_cast<std::uint8_t>(n), use_bjdata);
-        }
-        else if (n <= static_cast<std::uint64_t>((std::numeric_limits<std::int16_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('I'));  // int16
-            }
-            write_number(static_cast<std::int16_t>(n), use_bjdata);
-        }
-        else if (use_bjdata && n <= static_cast<uint64_t>((std::numeric_limits<uint16_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('u'));  // uint16 - bjdata only
-            }
-            write_number(static_cast<std::uint16_t>(n), use_bjdata);
-        }
-        else if (n <= static_cast<std::uint64_t>((std::numeric_limits<std::int32_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('l'));  // int32
-            }
-            write_number(static_cast<std::int32_t>(n), use_bjdata);
-        }
-        else if (use_bjdata && n <= static_cast<uint64_t>((std::numeric_limits<uint32_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('m'));  // uint32 - bjdata only
-            }
-            write_number(static_cast<std::uint32_t>(n), use_bjdata);
-        }
-        else if (n <= static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('L'));  // int64
-            }
-            write_number(static_cast<std::int64_t>(n), use_bjdata);
-        }
-        else if (use_bjdata)
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('M'));  // uint64 - bjdata only
-            }
-            write_number(static_cast<std::uint64_t>(n), use_bjdata);
-        }
-        else
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('H'));  // high-precision number
-            }
-
-            const auto number = BasicJsonType(n).dump();
-            write_number_with_ubjson_prefix(number.size(), true, use_bjdata);
-            for (std::size_t i = 0; i < number.size(); ++i)
-            {
-                oa.write_character(to_char_type(static_cast<std::uint8_t>(number[i])));
-            }
-        }
-    }
-
-    // UBJSON: write number (signed integer)
-    template < typename NumberType, typename std::enable_if <
-                   std::is_signed<NumberType>::value&&
-                   !std::is_floating_point<NumberType>::value, int >::type = 0 >
-    void write_number_with_ubjson_prefix(const NumberType n,
-                                         const bool add_prefix,
-                                         const bool use_bjdata)
-    {
-        if ((std::numeric_limits<std::int8_t>::min)() <= n && n <= (std::numeric_limits<std::int8_t>::max)())
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('i'));  // int8
-            }
-            write_number(static_cast<std::int8_t>(n), use_bjdata);
-        }
-        else if (static_cast<std::int64_t>((std::numeric_limits<std::uint8_t>::min)()) <= n && n <= static_cast<std::int64_t>((std::numeric_limits<std::uint8_t>::max)()))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('U'));  // uint8
-            }
-            write_number(static_cast<std::uint8_t>(n), use_bjdata);
-        }
-        else if ((std::numeric_limits<std::int16_t>::min)() <= n && n <= (std::numeric_limits<std::int16_t>::max)())
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('I'));  // int16
-            }
-            write_number(static_cast<std::int16_t>(n), use_bjdata);
-        }
-        else if (use_bjdata && (static_cast<std::int64_t>((std::numeric_limits<std::uint16_t>::min)()) <= n && n <= static_cast<std::int64_t>((std::numeric_limits<std::uint16_t>::max)())))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('u'));  // uint16 - bjdata only
-            }
-            write_number(static_cast<uint16_t>(n), use_bjdata);
-        }
-        else if ((std::numeric_limits<std::int32_t>::min)() <= n && n <= (std::numeric_limits<std::int32_t>::max)())
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('l'));  // int32
-            }
-            write_number(static_cast<std::int32_t>(n), use_bjdata);
-        }
-        else if (use_bjdata && (static_cast<std::int64_t>((std::numeric_limits<std::uint32_t>::min)()) <= n && n <= static_cast<std::int64_t>((std::numeric_limits<std::uint32_t>::max)())))
-        {
-            if (add_prefix)
-            {
-                oa.write_character(to_char_type('m'));  // uint32 - bjdata only
-            }
-            write_number(static_cast<uint32_t>(n), use_bjdata);
-        }
-        else
-        {
-            // every value of an integer type of at most 64 bits fits into an
-            // int64; only a wider type needs a range check
-            write_ubjson_int64_or_high_precision(n, add_prefix, use_bjdata,
-                                                 std::integral_constant < bool, std::numeric_limits<NumberType>::digits <= std::numeric_limits<std::int64_t>::digits > {});
-        }
-    }
-
-    template<typename NumberType>
-    void write_ubjson_int64_or_high_precision(const NumberType n, const bool add_prefix, const bool use_bjdata, std::true_type /*fits_int64*/)
-    {
+        const CharType prefix = ubjson_integer_prefix(n, use_bjdata);
        if (add_prefix)
        {
-            oa.write_character(to_char_type('L'));  // int64
+            oa.write_character(prefix);
        }
-        write_number(static_cast<std::int64_t>(n), use_bjdata);
+        write_ubjson_integer_payload(prefix, n, use_bjdata);
    }

+    /*!
+    @brief determine the UBJSON/BJData type marker of an integer
+
+    This is the only place that picks the marker of an integer: both
+    write_number_with_ubjson_prefix() and ubjson_prefix() use it. An optimized
+    container announces the marker of its first value after `$` and then
+    writes every value without a marker, so the two must never disagree.
+
+    @param[in] n           the integer
+    @param[in] use_bjdata  whether the BJData-only markers `u`, `m`, and `M`
+                           may be used
+
+    @return the first marker of `i`, `U`, `I`, `u` (BJData), `l`, `m` (BJData),
+            `L`, `M` (BJData, unsigned types only), and `H` (high-precision
+            number) whose range contains @a n
+    */
    template<typename NumberType>
-    void write_ubjson_int64_or_high_precision(const NumberType n, const bool add_prefix, const bool use_bjdata, std::false_type /*fits_int64*/)
+    static CharType ubjson_integer_prefix(const NumberType n, const bool use_bjdata) noexcept
    {
-        if ((std::numeric_limits<std::int64_t>::min)() <= n && n <= (std::numeric_limits<std::int64_t>::max)())
+        if (value_in_range_of<std::int8_t>(n))
        {
-            write_ubjson_int64_or_high_precision(n, add_prefix, use_bjdata, std::true_type {});
-            return;
+            return 'i';
        }
-
-        if (add_prefix)
+        if (value_in_range_of<std::uint8_t>(n))
        {
-            oa.write_character(to_char_type('H'));  // high-precision number
+            return 'U';
        }
-
-        const auto number = BasicJsonType(n).dump();
-        write_number_with_ubjson_prefix(number.size(), true, use_bjdata);
-        for (std::size_t i = 0; i < number.size(); ++i)
+        if (value_in_range_of<std::int16_t>(n))
        {
-            oa.write_character(to_char_type(static_cast<std::uint8_t>(number[i])));
+            return 'I';
        }
+        if (use_bjdata && value_in_range_of<std::uint16_t>(n))
+        {
+            return 'u';
+        }
+        if (value_in_range_of<std::int32_t>(n))
+        {
+            return 'l';
+        }
+        if (use_bjdata && value_in_range_of<std::uint32_t>(n))
+        {
+            return 'm';
+        }
+        if (value_in_range_of<std::int64_t>(n))
+        {
+            return 'L';
+        }
+        if (use_bjdata && std::is_unsigned<NumberType>::value)
+        {
+            return 'M';
+        }
+        // anything else is treated as a high-precision number
+        return 'H';
    }

+    /*!
+    @brief write the value of an integer for the marker chosen by
+           ubjson_integer_prefix()
+    */
    template<typename NumberType>
-    static constexpr CharType ubjson_int64_or_high_precision_prefix(const NumberType /*n*/, std::true_type /*fits_int64*/) noexcept
+    void write_ubjson_integer_payload(const CharType prefix, const NumberType n, const bool use_bjdata)
    {
-        return 'L';
-    }
-
-    template<typename NumberType>
-    static CharType ubjson_int64_or_high_precision_prefix(const NumberType n, std::false_type /*fits_int64*/) noexcept
-    {
-        // anything outside of the range of an int64 is treated as a
-        // high-precision number
-        return ((std::numeric_limits<std::int64_t>::min)() <= n && n <= (std::numeric_limits<std::int64_t>::max)()) ? 'L' : 'H';
+        switch (prefix)
+        {
+            case 'i':
+                write_number(static_cast<std::int8_t>(n), use_bjdata);
+                break;
+            case 'U':
+                write_number(static_cast<std::uint8_t>(n), use_bjdata);
+                break;
+            case 'I':
+                write_number(static_cast<std::int16_t>(n), use_bjdata);
+                break;
+            case 'u':
+                write_number(static_cast<std::uint16_t>(n), use_bjdata);
+                break;
+            case 'l':
+                write_number(static_cast<std::int32_t>(n), use_bjdata);
+                break;
+            case 'm':
+                write_number(static_cast<std::uint32_t>(n), use_bjdata);
+                break;
+            case 'L':
+                write_number(static_cast<std::int64_t>(n), use_bjdata);
+                break;
+            case 'M':
+                write_number(static_cast<std::uint64_t>(n), use_bjdata);
+                break;
+            default:
+            {
+                // high-precision number: the decimal digits as a string
+                JSON_ASSERT(prefix == 'H');
+                const auto number = BasicJsonType(n).dump();
+                write_number_with_ubjson_prefix(number.size(), true, use_bjdata);
+                for (std::size_t i = 0; i < number.size(); ++i)
+                {
+                    oa.write_character(to_char_type(static_cast<std::uint8_t>(number[i])));
+                }
+                break;
+            }
+        }
    }

    /*!
@@ -1699,74 +1626,10 @@ class binary_writer
                return j.m_data.m_value.boolean ? 'T' : 'F';

            case value_t::number_integer:
-            {
-                if ((std::numeric_limits<std::int8_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
-                {
-                    return 'i';
-                }
-                if ((std::numeric_limits<std::uint8_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::uint8_t>::max)())
-                {
-                    return 'U';
-                }
-                if ((std::numeric_limits<std::int16_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
-                {
-                    return 'I';
-                }
-                if (use_bjdata && ((std::numeric_limits<std::uint16_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::uint16_t>::max)()))
-                {
-                    return 'u';
-                }
-                if ((std::numeric_limits<std::int32_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
-                {
-                    return 'l';
-                }
-                if (use_bjdata && ((std::numeric_limits<std::uint32_t>::min)() <= j.m_data.m_value.number_integer && j.m_data.m_value.number_integer <= (std::numeric_limits<std::uint32_t>::max)()))
-                {
-                    return 'm';
-                }
-                // every value of an integer type of at most 64 bits fits into
-                // an int64; only a wider type needs a range check
-                return ubjson_int64_or_high_precision_prefix(j.m_data.m_value.number_integer,
-                        std::integral_constant < bool, std::numeric_limits<typename BasicJsonType::number_integer_t>::digits <= std::numeric_limits<std::int64_t>::digits > {});
-            }
+                return ubjson_integer_prefix(j.m_data.m_value.number_integer, use_bjdata);

            case value_t::number_unsigned:
-            {
-                if (j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::int8_t>::max)()))
-                {
-                    return 'i';
-                }
-                if (j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::uint8_t>::max)()))
-                {
-                    return 'U';
-                }
-                if (j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::int16_t>::max)()))
-                {
-                    return 'I';
-                }
-                if (use_bjdata && j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::uint16_t>::max)()))
-                {
-                    return 'u';
-                }
-                if (j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::int32_t>::max)()))
-                {
-                    return 'l';
-                }
-                if (use_bjdata && j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::uint32_t>::max)()))
-                {
-                    return 'm';
-                }
-                if (j.m_data.m_value.number_unsigned <= static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()))
-                {
-                    return 'L';
-                }
-                if (use_bjdata)
-                {
-                    return 'M';
-                }
-                // anything else is treated as a high-precision number
-                return 'H';
-            }
+                return ubjson_integer_prefix(j.m_data.m_value.number_unsigned, use_bjdata);

            case value_t::number_float:
                return get_ubjson_float_prefix(j.m_data.m_value.number_float);
@@ -2408,19 +2271,6 @@ class binary_writer
    // Utility functions //
    ///////////////////////

-    /*
-    @brief write a number to output input
-    @param[in] n number of type @a NumberType
-    @param[in] OutputIsLittleEndian Set to true if output data is
-                                 required to be little endian
-    @tparam NumberType the type of the number
-
-    @note This function needs to respect the system's endianness, because bytes
-          in CBOR, MessagePack, and UBJSON are stored in network order (big
-          endian) and therefore need reordering on little endian systems.
-          On the other hand, BSON and BJData use little endian and should reorder
-          on big endian systems.
-    */
    // single-instruction byte swaps (compilers lower these to bswap/rev/movbe);
    // used to emit big-endian numbers without a per-byte std::reverse loop
    static std::uint16_t byte_swap(std::uint16_t x) noexcept
@@ -2502,6 +2352,19 @@ class binary_writer
        std::reverse(a.begin(), a.end());
    }

+    /*!
+    @brief write a number to the output
+    @param[in] n number of type @a NumberType
+    @param[in] OutputIsLittleEndian Set to true if output data is
+                                 required to be little endian
+    @tparam NumberType the type of the number
+
+    @note This function needs to respect the system's endianness, because bytes
+          in CBOR, MessagePack, UBJSON, and BON8 are stored in network order
+          (big endian) and therefore need reordering on little endian systems.
+          On the other hand, BSON and BJData use little endian and should
+          reorder on big endian systems.
+    */
    template<typename NumberType>
    void write_number(const NumberType n, const bool OutputIsLittleEndian = false)
    {
@@ -2552,7 +2415,7 @@ class binary_writer
    }

  public:
-    // The following to_char_type functions are implement the conversion
+    // The following to_char_type functions implement the conversion
    // between uint8_t and CharType. In case CharType is not unsigned,
    // such a conversion is required to allow values greater than 128.
    // See <https://github.com/nlohmann/json/issues/1286> for a discussion.
@@ -11,12 +11,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;

-#include <cmath>
 #include <fstream>
-#include <limits>
-#include <map>
-#include <string>
-#include <vector>
 #include "make_test_data_available.hpp"

 TEST_CASE("Binary Formats" * doctest::skip())
@@ -229,139 +224,3 @@ TEST_CASE("Binary Formats" * doctest::skip())
        CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
    }
 }
-
-namespace
-{
-// the binary formats as function pointers for "Binary formats with narrow number types";
-// named functions rather than lambdas, because clang 3.5 cannot convert a lambda
-// to a function pointer in the braced initializer of the format table
-using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
-using bytes = std::vector<std::uint8_t>;
-
-bytes encode_cbor(const json& j)
-{
-    return json::to_cbor(j);
-}
-narrow_json decode_cbor(const bytes& v, bool allow_exceptions)
-{
-    return narrow_json::from_cbor(v, true, allow_exceptions);
-}
-
-bytes encode_msgpack(const json& j)
-{
-    return json::to_msgpack(j);
-}
-narrow_json decode_msgpack(const bytes& v, bool allow_exceptions)
-{
-    return narrow_json::from_msgpack(v, true, allow_exceptions);
-}
-
-bytes encode_ubjson(const json& j)
-{
-    return json::to_ubjson(j);
-}
-narrow_json decode_ubjson(const bytes& v, bool allow_exceptions)
-{
-    return narrow_json::from_ubjson(v, true, allow_exceptions);
-}
-
-bytes encode_bjdata(const json& j)
-{
-    return json::to_bjdata(j);
-}
-narrow_json decode_bjdata(const bytes& v, bool allow_exceptions)
-{
-    return narrow_json::from_bjdata(v, true, allow_exceptions);
-}
-
-// BSON can only store numbers as object members
-bytes encode_bson(const json& j)
-{
-    return json::to_bson(json{{"a", j}});
-}
-narrow_json decode_bson(const bytes& v, bool allow_exceptions)
-{
-    const auto result = narrow_json::from_bson(v, true, allow_exceptions);
-    return result.is_discarded() ? result : result.at("a");
-}
-
-bytes encode_bon8(const json& j)
-{
-    return json::to_bon8(j);
-}
-narrow_json decode_bon8(const bytes& v, bool allow_exceptions)
-{
-    return narrow_json::from_bon8(v, true, allow_exceptions);
-}
-
-} // namespace
-
-TEST_CASE("Binary formats with narrow number types")
-{
-    // Numbers that do not fit the number types are handled like the lexer
-    // handles them in JSON text: an integer that fits neither integer type is
-    // stored as a floating-point number, and a finite floating-point number
-    // that overflows number_float_t is rejected with out_of_range.406.
-    struct binary_format
-    {
-        const char* name;
-        bytes (*encode)(const json&);
-        narrow_json (*decode)(const bytes&, bool);
-    };
-
-    const std::vector<binary_format> formats =
-    {
-        {"CBOR", encode_cbor, decode_cbor},
-        {"MessagePack", encode_msgpack, decode_msgpack},
-        {"UBJSON", encode_ubjson, decode_ubjson},
-        {"BJData", encode_bjdata, decode_bjdata},
-        {"BSON", encode_bson, decode_bson},
-        {"BON8", encode_bon8, decode_bon8},
-    };
-
-    for (const auto& format : formats)
-    {
-        const std::string name = format.name;
-        INFO("format := ", name);
-        const auto roundtrip = [&format](const json & j)
-        {
-            return format.decode(format.encode(j), true);
-        };
-
-        // integers that fit keep their type
-        CHECK(roundtrip(json(-5)).is_number_integer());
-        CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
-        CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
-        CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
-
-        // integers that fit neither integer type are stored as float
-        CHECK(roundtrip(json(5000000000u)).is_number_float());
-        CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
-        if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
-        {
-            CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
-            CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
-        }
-        CHECK(roundtrip(json(-3000000000LL)).is_number_float());
-        CHECK(roundtrip(json(-3000000000LL)).get<float>() == -3000000000.0f);
-        CHECK(roundtrip(json(-5000000000LL)).is_number_float());
-        CHECK(roundtrip(json(-5000000000LL)).get<float>() == -5000000000.0f);
-
-        // floating-point numbers that fit
-        CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
-        const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
-                                    std::numeric_limits<double>::infinity());
-        CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
-
-        // infinity and NaN are passed on
-        CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
-        CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
-
-        // finite floating-point numbers that overflow number_float_t are rejected
-        const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
-                                    + " value: number overflow";
-        CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
-        CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
-        CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
-    }
-}
@@ -3187,8 +3187,7 @@ TEST_CASE("Tagged values")
        // CBOR encodes negative integers as: result = -1 - n
        // For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
        // the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
-        // When n > INT64_MAX, the result exceeds int64_t range and is stored
-        // as a floating-point number, as the lexer does for JSON text.
+        // When n > INT64_MAX, the result exceeds int64_t range and is rejected.

        SECTION("n = 0 is valid (result = -1)")
        {
@@ -3209,34 +3208,33 @@ TEST_CASE("Tagged values")
            CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
        }

-        SECTION("n = INT64_MAX + 1 is stored as float")
+        SECTION("n = INT64_MAX + 1 is rejected (overflow)")
        {
            // n = INT64_MAX + 1 (0x8000000000000000)
-            // result = -1 - n = -9223372036854775809, which exceeds int64_t range;
-            // the nearest double is -9223372036854775808.0
+            // result = -1 - n = -9223372036854775809, which exceeds int64_t range
            const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
-            const auto result = json::from_cbor(input);
-            CHECK(result.is_number_float());
-            CHECK(result.get<double>() == -9223372036854775808.0);
-            CHECK(result == json::parse("-9223372036854775809"));
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
+                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
+                                 json::parse_error);
        }

-        SECTION("n = UINT64_MAX is stored as float")
+        SECTION("n = UINT64_MAX is rejected (overflow)")
        {
            // n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
            // result = -1 - n = -18446744073709551616, which exceeds int64_t range
            const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
-            const auto result = json::from_cbor(input);
-            CHECK(result.is_number_float());
-            CHECK(result.get<double>() == -18446744073709551616.0);
-            CHECK(result == json::parse("-18446744073709551616"));
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
+                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
+                                 json::parse_error);
        }

-        SECTION("overflow with allow_exceptions=false is not an error")
+        SECTION("overflow with allow_exceptions=false returns discarded")
        {
            const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
            const auto result = json::from_cbor(input, true, false);
-            CHECK(result.is_number_float());
+            CHECK(result.is_discarded());
        }
    }

@@ -3033,3 +3033,224 @@ TEST_CASE("UBJSON optimized array of unsigned integers beyond int64")
    CHECK(json::to_ubjson(j, true, true) == expected);
    CHECK(json::from_ubjson(expected) == j);
 }
+
+namespace
+{
+// the bytes that follow the marker of an integer: the value in the width of
+// the marker (big endian for UBJSON, little endian for BJData), or, for a
+// high-precision number, the length and the decimal digits
+std::vector<std::uint8_t> integer_payload(const char marker, const json& value, const bool little_endian)
+{
+    std::size_t width = 0;
+    switch (marker)
+    {
+        case 'i':
+        case 'U':
+            width = 1;
+            break;
+        case 'I':
+        case 'u':
+            width = 2;
+            break;
+        case 'l':
+        case 'm':
+            width = 4;
+            break;
+        case 'L':
+        case 'M':
+            width = 8;
+            break;
+        default:
+        {
+            const std::string digits = value.dump();
+            std::vector<std::uint8_t> result = {'i', static_cast<std::uint8_t>(digits.size())};
+            for (const char c : digits)
+            {
+                result.push_back(static_cast<std::uint8_t>(c));
+            }
+            return result;
+        }
+    }
+
+    const std::uint64_t bits = value.is_number_unsigned()
+                               ? value.get<std::uint64_t>()
+                               : static_cast<std::uint64_t>(value.get<std::int64_t>());
+    std::vector<std::uint8_t> result(width);
+    for (std::size_t i = 0; i < width; ++i)
+    {
+        result[little_endian ? i : width - 1 - i] = static_cast<std::uint8_t>(bits >> (8 * i));
+    }
+    return result;
+}
+
+json i64(const std::int64_t v)
+{
+    return v;
+}
+
+json u64(const std::uint64_t v)
+{
+    return v;
+}
+} // namespace
+
+TEST_CASE("UBJSON and BJData integer markers at every range edge")
+{
+    // An optimized container announces the marker of its values after `$` and
+    // then writes every value without a marker, so the marker the writer
+    // announces and the width it writes must match for every value. This
+    // checks both for the values around each edge of the integer types, as
+    // scalars and as the values of optimized arrays and objects.
+    struct integer_case
+    {
+        json value;
+        char ubjson; // expected UBJSON marker
+        char bjdata; // expected BJData marker
+    };
+
+    const std::int64_t int64_min = (std::numeric_limits<std::int64_t>::min)();
+    const std::int64_t int64_max = (std::numeric_limits<std::int64_t>::max)();
+    const std::uint64_t uint64_max = (std::numeric_limits<std::uint64_t>::max)();
+
+    const std::vector<integer_case> cases =
+    {
+        // int8
+        {i64(-129), 'I', 'I'},
+        {i64(-128), 'i', 'i'},
+        {i64(-127), 'i', 'i'},
+        {i64(-1), 'i', 'i'},
+        {i64(0), 'i', 'i'},
+        {u64(0), 'i', 'i'},
+        {i64(126), 'i', 'i'},
+        {i64(127), 'i', 'i'},
+        {u64(127), 'i', 'i'},
+        {i64(128), 'U', 'U'},
+        {u64(128), 'U', 'U'},
+        // uint8
+        {i64(254), 'U', 'U'},
+        {i64(255), 'U', 'U'},
+        {u64(255), 'U', 'U'},
+        {i64(256), 'I', 'I'},
+        {u64(256), 'I', 'I'},
+        // int16
+        {i64(-32769), 'l', 'l'},
+        {i64(-32768), 'I', 'I'},
+        {i64(-32767), 'I', 'I'},
+        {i64(32766), 'I', 'I'},
+        {i64(32767), 'I', 'I'},
+        {u64(32767), 'I', 'I'},
+        {i64(32768), 'l', 'u'},
+        {u64(32768), 'l', 'u'},
+        // uint16 (BJData only)
+        {i64(65534), 'l', 'u'},
+        {i64(65535), 'l', 'u'},
+        {u64(65535), 'l', 'u'},
+        {i64(65536), 'l', 'l'},
+        {u64(65536), 'l', 'l'},
+        // int32
+        {i64(-2147483649LL), 'L', 'L'},
+        {i64(-2147483648LL), 'l', 'l'},
+        {i64(-2147483647LL), 'l', 'l'},
+        {i64(2147483646LL), 'l', 'l'},
+        {i64(2147483647LL), 'l', 'l'},
+        {u64(2147483647ULL), 'l', 'l'},
+        {i64(2147483648LL), 'L', 'm'},
+        {u64(2147483648ULL), 'L', 'm'},
+        // uint32 (BJData only)
+        {i64(4294967294LL), 'L', 'm'},
+        {i64(4294967295LL), 'L', 'm'},
+        {u64(4294967295ULL), 'L', 'm'},
+        {i64(4294967296LL), 'L', 'L'},
+        {u64(4294967296ULL), 'L', 'L'},
+        // int64
+        {i64(int64_min), 'L', 'L'},
+        {i64(int64_min + 1), 'L', 'L'},
+        {i64(int64_max - 1), 'L', 'L'},
+        {i64(int64_max), 'L', 'L'},
+        {u64(static_cast<std::uint64_t>(int64_max)), 'L', 'L'},
+        // uint64 (BJData only; UBJSON writes a high-precision number)
+        {u64(static_cast<std::uint64_t>(int64_max) + 1), 'H', 'M'},
+        {u64(uint64_max - 1), 'H', 'M'},
+        {u64(uint64_max), 'H', 'M'},
+    };
+
+    for (const auto& c : cases)
+    {
+        for (const bool bjdata :
+                {
+                    false, true
+                })
+        {
+            const char marker = bjdata ? c.bjdata : c.ubjson;
+            const std::vector<std::uint8_t> payload = integer_payload(marker, c.value, bjdata);
+            const auto to_binary = [bjdata](const json & j, const bool use_size, const bool use_type)
+            {
+                return bjdata ? json::to_bjdata(j, use_size, use_type) : json::to_ubjson(j, use_size, use_type);
+            };
+            const auto from_binary = [bjdata](const std::vector<std::uint8_t>& v)
+            {
+                return bjdata ? json::from_bjdata(v) : json::from_ubjson(v);
+            };
+            INFO("value = " << c.value.dump() << (c.value.is_number_unsigned() ? " (unsigned)" : "") << ", format = " << (bjdata ? "BJData" : "UBJSON"));
+
+            // scalar
+            std::vector<std::uint8_t> expected = {static_cast<std::uint8_t>(marker)};
+            expected.insert(expected.end(), payload.begin(), payload.end());
+            for (const bool use_size :
+                    {
+                        false, true
+                    })
+            {
+                CHECK(to_binary(c.value, use_size, false) == expected);
+            }
+            CHECK(from_binary(expected) == c.value);
+
+            const json arr = {c.value, c.value, c.value};
+
+            // array without count or type: every value has its marker
+            expected = {'['};
+            for (int i = 0; i < 3; ++i)
+            {
+                expected.push_back(static_cast<std::uint8_t>(marker));
+                expected.insert(expected.end(), payload.begin(), payload.end());
+            }
+            expected.push_back(']');
+            CHECK(to_binary(arr, false, false) == expected);
+            CHECK(from_binary(expected) == arr);
+
+            // array with count: every value has its marker
+            expected = {'[', '#', 'i', 3};
+            for (int i = 0; i < 3; ++i)
+            {
+                expected.push_back(static_cast<std::uint8_t>(marker));
+                expected.insert(expected.end(), payload.begin(), payload.end());
+            }
+            CHECK(to_binary(arr, true, false) == expected);
+            CHECK(from_binary(expected) == arr);
+
+            // array with type and count: the marker once, then the payloads
+            expected = {'[', '$', static_cast<std::uint8_t>(marker), '#', 'i', 3};
+            for (int i = 0; i < 3; ++i)
+            {
+                expected.insert(expected.end(), payload.begin(), payload.end());
+            }
+            CHECK(to_binary(arr, true, true) == expected);
+            CHECK(from_binary(expected) == arr);
+
+            // object with type and count: the marker once, then key and payload
+            const json obj = {{"a", c.value}, {"b", c.value}};
+            expected = {'{', '$', static_cast<std::uint8_t>(marker), '#', 'i', 2};
+            for (const char key :
+                    {'a', 'b'
+                    })
+            {
+                expected.push_back('i');
+                expected.push_back(1);
+                expected.push_back(static_cast<std::uint8_t>(key));
+                expected.insert(expected.end(), payload.begin(), payload.end());
+            }
+            CHECK(to_binary(obj, true, true) == expected);
+            CHECK(from_binary(expected) == obj);
+        }
+    }
+}
Author	SHA1	Message	Date
Niels Lohmann	e072480872	Share the IEEE half-precision decoder between CBOR and BJData binary_reader had two ~45-line copies of the IEEE 754 half-precision decoder: CBOR's case 0xF9 and BJData's case 'h'. Once formatting is normalised, the two blocks were identical except for the byte order used to assemble the 16-bit half (CBOR is big endian, BJData is little endian). Any future change to half-float decoding had to be made and kept in sync in both places. Add one get_half_float(format, little_endian) helper that does the two get()/unexpect_eof() reads, assembles the half in the requested byte order, decodes it per RFC 8949 Appendix D, and calls sax->number_float. Both cases now just call it with their byte order; the BJData case keeps its bjdata-only guard. Behavior, the public API and the ABI are unchanged. Verified with a scratch probe comparing the old and new decoders bit-for-bit (NaN by isnan()) over all 65536 wire byte pairs, in both formats, and by running unit-cbor and unit-bjdata (offline, against the stubbed test_data.hpp). Part of #5711 Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-09-30 10:19:57 +02:00
Niels Lohmann	a3a94bb7eb	Remove dead get_char parameters in binary_reader The non-recursive rewrite of the binary readers (#5505, #5506, #5507) left parse_cbor_internal()'s and parse_ubjson_internal()'s get_char parameters dead: parse_cbor_internal() has one caller and it always passes true, and parse_ubjson_internal() has one caller and it always uses the true default. Both parameters, and the @param docs describing the "reuse the last character" mode they used to select, no longer correspond to anything. Drop both parameters, initialise fetch/prefix unconditionally, and update the two call sites in sax_parse(). parse_cbor_value()'s and get_ubjson_string()'s own get_char parameters are unrelated and are left alone; both still have a false caller. Also delete a stray `@return whether a valid MessagePack value was passed to the SAX parser` doxygen block that sits directly above parse_msgpack_value()'s real doc comment, a leftover of the same rewrite. Behavior, the public API and the ABI are unchanged; these are private members of detail::binary_reader. Verified by compiling with -Wunused-parameter and running unit-cbor, unit-ubjson, unit-bjdata and unit-msgpack (offline, against the stubbed test_data.hpp). Part of #5711 Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-09-30 10:19:56 +02:00
Niels Lohmann	0ffe9ab4a8	Merge the duplicated UBJSON/BJData integer marker ladders write_number_with_ubjson_prefix() (unsigned and signed overloads) and ubjson_prefix() (number_integer and number_unsigned cases) each picked the UBJSON/BJData integer marker (i, U, I, u, l, m, L, M, H) with their own independent if/else ladder, and the values beyond 64 bits were handled by a second, tag-dispatched pair of ladders. An optimized container announces the marker of its first element via ubjson_prefix() and then writes every element through write_number_with_ubjson_prefix(), so the two had to be kept in lockstep by hand across four call sites. Replace all of that with one ubjson_integer_prefix() built on value_in_range_of<T>, and one write_ubjson_integer_payload() that writes the value (or, for 'H', the decimal digits) for a given marker. write_number_with_ubjson_prefix() and ubjson_prefix() keep their signatures and now just call these two helpers. Behavior, the public API and the ABI are unchanged. Verified with a new regression test covering scalars and $-optimized arrays/objects at every int8/uint8/int16/uint16/int32/uint32/int64/uint64 boundary for to_ubjson/to_bjdata (both use_size/use_type settings), and by diffing to_ubjson/to_bjdata output before and after over the json_test_data corpus (bit-identical). Part of #5710 Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-09-30 10:19:54 +02:00
Niels Lohmann	e158b080bd	Fix stale and missing comments in binary_writer The doc block of write_number() ended up above the byte_swap() helpers added in #5286, about 80 lines from the function. It was also a plain comment that Doxygen skips, said "write a number to output input", and left BON8 out of the big-endian formats. Move it back onto write_number() as a /*! block and fix the text. write_bson() documented "@pre j.type() == value_t::object", but it throws type_error.317 for every other type, and to_bson() relies on that. Document the exception instead. Explain why the CBOR binary subtype is always written with a 0xD8..0xDB head and never in the one-byte tag form: binary_reader with cbor_tag_handler_t::store only keeps those heads as a subtype, so switching to write_cbor_head() would break round trips for subtypes 0..23. Also fix the grammar of the to_char_type comment. Comments only; no change in behavior, API or ABI. Part of #5710 Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-09-30 09:37:28 +02:00