diff --git a/docs/mkdocs/docs/api/basic_json/number_float_t.md b/docs/mkdocs/docs/api/basic_json/number_float_t.md index 8419a392d..282bb6123 100644 --- a/docs/mkdocs/docs/api/basic_json/number_float_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_float_t.md @@ -55,6 +55,10 @@ This implementation does exactly follow this approach, as it uses double precisi smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally and be serialized to `null`. +During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into +`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for +example a double-precision number in a binary format when `number_float_t` is `#!cpp float`. + ### Storage Floating-point number values are stored directly inside a `basic_json` type. diff --git a/docs/mkdocs/docs/api/basic_json/number_integer_t.md b/docs/mkdocs/docs/api/basic_json/number_integer_t.md index d77c6a24d..48d2873ed 100644 --- a/docs/mkdocs/docs/api/basic_json/number_integer_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_integer_t.md @@ -47,8 +47,9 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of -range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers -will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md). +range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary +formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) +or [`number_float_t`](number_float_t.md). [RFC 8259](https://tools.ietf.org/html/rfc8259) further states: > Note that when such software is used, numbers that are integers and are in the range [-253+1, 253-1] are diff --git a/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md b/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md index 774fda638..81cc00f77 100644 --- a/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md @@ -48,8 +48,9 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow -when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored -as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md). +when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small +integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or +[`number_float_t`](number_float_t.md). [RFC 8259](https://tools.ietf.org/html/rfc8259) further states: > Note that when such software is used, numbers that are integers and are in the range [-253+1, 253-1] are diff --git a/docs/mkdocs/docs/features/binary_formats/cbor.md b/docs/mkdocs/docs/features/binary_formats/cbor.md index 7b5be4631..1b226cd15 100644 --- a/docs/mkdocs/docs/features/binary_formats/cbor.md +++ b/docs/mkdocs/docs/features/binary_formats/cbor.md @@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows: !!! warning "Negative integer overflow" CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the - result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a - [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing - silently. + result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like + a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is + stored as `-1.8446744073709552e+19`. !!! warning "Object keys" diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index e7eb8fd03..a4706dfdd 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -331,9 +331,6 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde [json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1 ``` ``` - [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow - ``` - ``` [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5) ``` @@ -863,13 +860,18 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme ### json.exception.out_of_range.406 -A parsed number could not be stored as without changing it to NaN or INF. +A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a +finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a +double-precision number when `number_float_t` is `#!cpp float`. -!!! failure "Example message" +!!! failure "Example messages" ``` number overflow parsing '10E1000' ``` + ``` + [json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow + ``` ### json.exception.out_of_range.407 diff --git a/include/nlohmann/detail/input/binary_reader.hpp b/include/nlohmann/detail/input/binary_reader.hpp index d6b45909a..369e274fb 100644 --- a/include/nlohmann/detail/input/binary_reader.hpp +++ b/include/nlohmann/detail/input/binary_reader.hpp @@ -197,7 +197,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(current != char_traits::eof())) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(110, chars_read, - exception_message(input_format, concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); + exception_message(concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); } } @@ -320,7 +320,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast(document_size) != chars_read - document_start)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); + exception_message(concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); } return true; } @@ -335,7 +335,7 @@ class binary_reader // check_bson_document_size() measures the document from here const std::size_t document_start = chars_read; std::int32_t document_size{}; - if (!get_number(input_format_t::bson, document_size)) + if (!get_number(document_size)) { return false; } @@ -400,7 +400,7 @@ class binary_reader continue; } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "element list"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("element list"))) { return false; } @@ -444,7 +444,7 @@ class binary_reader while (true) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "cstring"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("cstring"))) { return false; } @@ -514,10 +514,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); + exception_message(concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!get_string(input_format_t::bson, len - static_cast(1), result))) + if (JSON_HEDLEY_UNLIKELY(!get_string(len - static_cast(1), result))) { return false; } @@ -526,8 +526,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, - "BSON string is not null-terminated", + exception_message("BSON string is not null-terminated", "string"), nullptr)); } @@ -550,18 +549,18 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); + exception_message(concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); } // All BSON binary values have a subtype std::uint8_t subtype{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::bson, subtype))) + if (JSON_HEDLEY_UNLIKELY(!get_number(subtype))) { return false; } result.set_subtype(subtype); - return get_binary(input_format_t::bson, len, result); + return get_binary(len, result); } /*! @@ -582,14 +581,14 @@ class binary_reader case 0x01: // double { double number{}; - return get_number(input_format_t::bson, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x02: // string { std::int32_t len{}; string_t value; - return get_number(input_format_t::bson, len) && get_bson_string(len, value) && sax->string(value); + return get_number(len) && get_bson_string(len, value) && sax->string(value); } case 0x03: // object @@ -606,13 +605,13 @@ class binary_reader { std::int32_t len{}; binary_t value; - return get_number(input_format_t::bson, len) && get_bson_binary(len, value) && sax->binary(value); + return get_number(len) && get_bson_binary(len, value) && sax->binary(value); } case 0x08: // boolean { std::uint8_t value{}; - return get_number(input_format_t::bson, value) && sax->boolean(value != 0); + return get_number(value) && sax->boolean(value != 0); } case 0x0A: // null @@ -623,19 +622,19 @@ class binary_reader case 0x10: // int32 { std::int32_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x12: // int64 { std::int64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x11: // uint64 { std::uint64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_unsigned(value); + return get_number(value) && emit_unsigned(value); } default: // anything else is not supported (yet) @@ -657,18 +656,23 @@ class binary_reader bool get_cbor_negative_integer() { NumberType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } - const auto max_val = static_cast((std::numeric_limits::max)()); - if (number > max_val) + + // the value is -1 - number, which fits into number_integer_t + // whenever number does + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) { - return sax->parse_error(chars_read, get_token_string(), - parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr)); + return sax->number_integer(static_cast(-1) - static_cast(number)); } - return sax->number_integer(conditional_static_cast(static_cast(-1) - static_cast(number))); + + // like the lexer does for JSON text, store a value too small for + // number_integer_t as number_float_t; compute it as long double so + // that emit_float sees a finite value and can detect an overflow of + // number_float_t + return emit_float(static_cast(-1) - static_cast(number)); } /*! @@ -693,7 +697,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::cbor, "value"); + return unexpect_eof("value"); // Integer 0x00..0x17 (0..23) case 0x00: @@ -725,25 +729,25 @@ class binary_reader case 0x18: // Unsigned integer (one-byte uint8_t follows) { std::uint8_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x19: // Unsigned integer (two-byte uint16_t follows) { std::uint16_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1A: // Unsigned integer (four-byte uint32_t follows) { std::uint32_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1B: // Unsigned integer (eight-byte uint64_t follows) { std::uint64_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } // Negative integer -1-0x00..-1-0x17 (-1..-24) @@ -964,7 +968,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } // ignore and store: the tag value is already in the head, so @@ -986,7 +990,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } case cbor_tag_handler_t::ignore: @@ -1044,25 +1048,25 @@ class binary_reader return sax->null(); case 0xF9: // Half-Precision Float (two-byte IEEE 754) - return get_half_float(input_format_t::cbor, false); + return get_half_float(false); case 0xFA: // Single-Precision Float (four-byte IEEE 754) { float number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xFB: // Double-Precision Float (eight-byte IEEE 754) { double number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } default: // anything else (0xFF is handled inside the other types) { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -1111,38 +1115,38 @@ class binary_reader case 0x76: case 0x77: { - return get_string(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_string(static_cast(current) & 0x1Fu, result); } case 0x78: // UTF-8 string (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x79: // UTF-8 string (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7A: // UTF-8 string (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -1170,7 +1174,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -1279,7 +1283,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -1326,42 +1330,42 @@ class binary_reader case 0x56: case 0x57: { - return get_binary(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_binary(static_cast(current) & 0x1Fu, result); } case 0x58: // Binary data (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x59: // Binary data (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5A: // Binary data (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5B: // Binary data (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); + exception_message(concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); } } } @@ -1385,7 +1389,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "binary"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("binary"))) { return false; } @@ -1443,7 +1447,7 @@ class binary_reader case 0x18: // 1 byte { std::uint8_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1454,7 +1458,7 @@ class binary_reader case 0x19: // 2 bytes { std::uint16_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1465,7 +1469,7 @@ class binary_reader case 0x1A: // 4 bytes { std::uint32_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1476,7 +1480,7 @@ class binary_reader case 0x1B: // 8 bytes { std::uint64_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1509,7 +1513,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(len) || len == detail::unknown_size())) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format_t::cbor, concat("excessive ", context, " size"), "size"), nullptr)); + exception_message(concat("excessive ", context, " size"), "size"), nullptr)); } result = conditional_static_cast(len); return true; @@ -1638,7 +1642,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::msgpack, "value"); + return unexpect_eof("value"); // positive fixint case 0x00: @@ -1878,85 +1882,85 @@ class binary_reader case 0xCA: // float 32 { float number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCB: // float 64 { double number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCC: // uint 8 { std::uint8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCD: // uint 16 { std::uint16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCE: // uint 32 { std::uint32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCF: // uint 64 { std::uint64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xD0: // int 8 { std::int8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD1: // int 16 { std::int16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD2: // int 32 { std::int32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD3: // int 64 { std::int64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xDC: // array 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(static_cast(len)); + return get_number(len) && enter_array(static_cast(len)); } case 0xDD: // array 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(conditional_static_cast(len)); + return get_number(len) && enter_array(conditional_static_cast(len)); } case 0xDE: // map 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(static_cast(len)); + return get_number(len) && enter_object(static_cast(len)); } case 0xDF: // map 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(conditional_static_cast(len)); + return get_number(len) && enter_object(conditional_static_cast(len)); } // negative fixint @@ -1998,7 +2002,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::msgpack, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -2015,7 +2019,7 @@ class binary_reader */ bool get_msgpack_string(string_t& result, const char* context = "string") { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::msgpack, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -2056,32 +2060,32 @@ class binary_reader case 0xBE: case 0xBF: { - return get_string(input_format_t::msgpack, static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); + return get_string(static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); } case 0xD9: // str 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDA: // str 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDB: // str 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -2175,7 +2179,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -2202,31 +2206,31 @@ class binary_reader case 0xC4: // bin 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC5: // bin 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC6: // bin 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC7: // ext 8 { std::uint8_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -2234,9 +2238,9 @@ class binary_reader { std::uint16_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -2244,49 +2248,49 @@ class binary_reader { std::uint32_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } case 0xD4: // fixext 1 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 1, result) && + return get_number(subtype) && + get_binary(1, result) && assign_and_return_true(subtype); } case 0xD5: // fixext 2 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 2, result) && + return get_number(subtype) && + get_binary(2, result) && assign_and_return_true(subtype); } case 0xD6: // fixext 4 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 4, result) && + return get_number(subtype) && + get_binary(4, result) && assign_and_return_true(subtype); } case 0xD7: // fixext 8 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 8, result) && + return get_number(subtype) && + get_binary(8, result) && assign_and_return_true(subtype); } case 0xD8: // fixext 16 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 16, result) && + return get_number(subtype) && + get_binary(16, result) && assign_and_return_true(subtype); } @@ -2485,7 +2489,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(len < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "string length must not be negative", "string"), nullptr)); + exception_message("string length must not be negative", "string"), nullptr)); } return true; } @@ -2515,7 +2519,7 @@ class binary_reader get(); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } @@ -2525,31 +2529,31 @@ class binary_reader case 'U': { std::uint8_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'i': { std::int8_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'I': { std::int16_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'l': { std::int32_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'L': { std::int64_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'u': @@ -2559,7 +2563,7 @@ class binary_reader break; } std::uint16_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'm': @@ -2569,7 +2573,7 @@ class binary_reader break; } std::uint32_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'M': @@ -2579,7 +2583,7 @@ class binary_reader break; } std::uint64_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: @@ -2596,7 +2600,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "string"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "string"), nullptr)); } /*! @@ -2675,19 +2679,19 @@ class binary_reader bool get_ubjson_signed_count(std::size_t& result) { SignedType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (JSON_HEDLEY_UNLIKELY(number < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr)); + exception_message("count in an optimized container must be positive", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(number))) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = static_cast(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char return true; @@ -2716,7 +2720,7 @@ class binary_reader case 'U': { std::uint8_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -2743,7 +2747,7 @@ class binary_reader break; } std::uint16_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -2758,7 +2762,7 @@ class binary_reader break; } std::uint32_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -2773,14 +2777,14 @@ class binary_reader break; } std::uint64_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (!value_in_range_of(number)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = detail::conditional_static_cast(number); return true; @@ -2794,7 +2798,7 @@ class binary_reader } if (is_ndarray) // ndarray dimensional vector can only contain integers and cannot embed another array { - return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message(input_format, "ndarray dimensional vector is not allowed", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message("ndarray dimensional vector is not allowed", "size"), nullptr)); } std::vector dim; if (JSON_HEDLEY_UNLIKELY(!get_ubjson_ndarray_size(dim))) @@ -2834,7 +2838,7 @@ class binary_reader // or SIZE_MAX. if (JSON_HEDLEY_UNLIKELY(result > (std::numeric_limits::max)() / i)) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } result *= i; // the pre-check above already rules out result becoming 0 @@ -2843,9 +2847,9 @@ class binary_reader // unknown-size container (see get_ubjson_size_type()) if (result == npos) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast(i)))) + if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i))) { return false; } @@ -2871,7 +2875,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "size"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "size"), nullptr)); } /*! @@ -2906,10 +2910,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); + exception_message(concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "type"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type"))) { return false; } @@ -2917,13 +2921,13 @@ class binary_reader get_ignore_noop(); if (JSON_HEDLEY_UNLIKELY(current != '#')) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); + exception_message(concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); } const bool is_error = get_ubjson_size_value(result.first, is_ndarray); @@ -2943,7 +2947,7 @@ class binary_reader if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format, "ndarray requires both type and size", "size"), nullptr)); + exception_message("ndarray requires both type and size", "size"), nullptr)); } return is_error; } @@ -2960,7 +2964,7 @@ class binary_reader switch (prefix) { case char_traits::eof(): // EOF - return unexpect_eof(input_format, "value"); + return unexpect_eof("value"); case 'T': // true return sax->boolean(true); @@ -2977,37 +2981,37 @@ class binary_reader break; } std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'U': { std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'i': { std::int8_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'I': { std::int16_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'l': { std::int32_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'L': { std::int64_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'u': @@ -3017,7 +3021,7 @@ class binary_reader break; } std::uint16_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'm': @@ -3027,7 +3031,7 @@ class binary_reader break; } std::uint32_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'M': @@ -3037,7 +3041,7 @@ class binary_reader break; } std::uint64_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'h': @@ -3046,19 +3050,19 @@ class binary_reader { break; } - return get_half_float(input_format, true); + return get_half_float(true); } case 'd': { float number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'D': { double number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'H': @@ -3069,7 +3073,7 @@ class binary_reader case 'C': // char { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "char"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("char"))) { return false; } @@ -3077,7 +3081,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); + exception_message(concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); } string_t s(1, static_cast(current)); return sax->string(s); @@ -3099,7 +3103,7 @@ class binary_reader break; } auto last_token = get_token_string(); - return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format, "invalid byte: 0x" + last_token, "value"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message("invalid byte: 0x" + last_token, "value"), nullptr)); } /*! @@ -3125,7 +3129,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "invalid byte: 0x" + last_token, "type"), nullptr)); + exception_message("invalid byte: 0x" + last_token, "type"), nullptr)); } string_t type = type_name; // sax->string() takes a reference @@ -3160,7 +3164,7 @@ class binary_reader if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B') { binary_t result; - return get_binary(input_format, size_and_type.first, result) && sax->binary(result); + return get_binary(size_and_type.first, result) && sax->binary(result); } if (size_and_type.first != npos) @@ -3173,7 +3177,7 @@ class binary_reader && size_and_type.first > max_valueless_container_size)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "excessive array size", "size"), nullptr)); + exception_message("excessive array size", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second))) @@ -3210,7 +3214,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "BJData object does not support ND-array size in optimized format", "object"), nullptr)); + exception_message("BJData object does not support ND-array size in optimized format", "object"), nullptr)); } if (size_and_type.first != npos) @@ -3240,7 +3244,7 @@ class binary_reader for (std::size_t i = 0; i < size; ++i) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -3259,7 +3263,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input)) { return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } switch (result_number) @@ -3299,7 +3303,7 @@ class binary_reader case token_type::literal_or_value: default: return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } } @@ -3368,7 +3372,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr)); + exception_message(concat(detail, ": 0x", last_token), context), nullptr)); } /*! @@ -3472,7 +3476,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "value"); + return unexpect_eof("value"); } // string: ASCII character @@ -3509,25 +3513,25 @@ class binary_reader case 0x8C: // int32 { std::int32_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8D: // int64 { std::int64_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8E: // binary32 { float number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x8F: // binary64 { double number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xF8: @@ -3593,7 +3597,9 @@ class binary_reader @brief pass an integer to the SAX parser Non-negative integers are passed as unsigned, negative integers as signed - numbers, like the other binary formats do. + numbers, like the other binary formats do. A value that does not fit the + number type is passed as described for @ref emit_unsigned and + @ref emit_signed. @param[in] number the integer @return whether the SAX parser accepted the value @@ -3602,9 +3608,9 @@ class binary_reader { if (number >= 0) { - return sax->number_unsigned(static_cast(number)); + return emit_unsigned(static_cast(number)); } - return sax->number_integer(static_cast(number)); + return emit_signed(number); } /*! @@ -3624,7 +3630,7 @@ class binary_reader */ bool get_bon8_integer(const char_int_type lead, const char_int_type second) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -3656,13 +3662,12 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "number"); + return unexpect_eof("number"); } value = (value << 8) | static_cast(current); } - return negative ? sax->number_integer(static_cast(-(value + offset))) - : sax->number_unsigned(static_cast(value + offset)); + return emit_bon8_integer(negative ? -(value + offset) : value + offset); } /*! @@ -3680,7 +3685,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } if (byte == 0xFF) @@ -3700,7 +3705,7 @@ class binary_reader if (second == char_traits::eof()) { // the input ends inside a character or an integer - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } unget_bon8(second); if (is_bon8_continuation(second)) @@ -3773,7 +3778,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } // end of string @@ -3804,7 +3809,7 @@ class binary_reader { // the input ends inside a character or an integer: either // way, the message is incomplete - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (!is_bon8_continuation(second)) { @@ -3845,7 +3850,7 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current))) { @@ -3884,7 +3889,7 @@ class binary_reader @return bool, whether the read was successful */ template - bool get_to(T& dest, const input_format_t format, const char* context) + bool get_to(T& dest, const char* context) { // false positive: new_chars_read is read on the next lines // @infer-ignore DEAD_STORE @@ -3894,7 +3899,7 @@ class binary_reader { // in case of failure, advance position by 1 to report the failing location ++chars_read; - sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); return false; } return true; @@ -3945,7 +3950,6 @@ class binary_reader @brief read a number from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[out] result number of type @a NumberType @return whether conversion completed @@ -3957,42 +3961,119 @@ class binary_reader should reorder on big endian systems. */ template - bool get_number(const input_format_t format, NumberType& result) + bool get_number(NumberType& result) { // read in the original format - if (JSON_HEDLEY_UNLIKELY(!get_to(result, format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!get_to(result, "number"))) { return false; } - if (is_little_endian != (InputIsLittleEndian || format == input_format_t::bjdata)) + if (is_little_endian != (InputIsLittleEndian || input_format == input_format_t::bjdata)) { byte_swap(result); } return true; } + /*! + @brief pass a signed integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_integer_t is passed as number_unsigned_t if it is non-negative and + fits there, and as number_float_t otherwise. With the default number + types, every integer the binary formats can encode fits, so this only + matters for narrower custom number types. + + @tparam NumberType a signed integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_signed(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_integer(static_cast(number)); + } + if (value_in_range_of(number)) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass an unsigned integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_unsigned_t is passed as number_float_t. + + @tparam NumberType an unsigned integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_unsigned(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass a floating-point number read from the input to the SAX parser + + Like the lexer does for JSON text, a finite value that overflows + number_float_t is rejected instead of silently becoming infinity. Infinity + and NaN in the input are passed on unchanged. Integers only overflow if + number_float_t cannot represent 2^64, e.g., a half-precision type. + + @tparam NumberType a floating-point or integer type + @param[in] number the number + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if a finite @a number overflows number_float_t + */ + template + bool emit_float(const NumberType number) + { + const auto result = static_cast(number); + if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result))) + { + return sax->parse_error(chars_read, get_token_string(), + out_of_range::create(406, exception_message("number overflow", "value"), nullptr)); + } + return sax->number_float(result, ""); + } + /*! @brief read and decode an IEEE 754 half-precision (16-bit) float Used by CBOR (big endian) and BJData (little endian); the two formats only differ in the byte order of the two bytes that make up the half. - - @param[in] format the current format (for diagnostics) @param[in] little_endian whether the two bytes are little endian (BJData) or big endian (CBOR) @return whether reading and decoding succeeded */ - bool get_half_float(const input_format_t format, const bool little_endian) + bool get_half_float(const bool little_endian) { const auto byte1_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } const auto byte2_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -4038,7 +4119,6 @@ class binary_reader @brief create a string by reading characters from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of characters to read @param[out] result string created by reading @a len bytes @@ -4049,8 +4129,7 @@ class binary_reader the input before we run out of string memory. */ template - bool get_string(const input_format_t format, - const NumberType len, + bool get_string(const NumberType len, string_t& result) { // Strings are taken as is by default: none of CBOR (RFC 8949 ยง3.1 @@ -4062,7 +4141,7 @@ class binary_reader // @ref error_handler, applied once the whole string (all chunks of // an indefinite-length CBOR string included) has been assembled, by // @ref check_string_utf8 at the call site. - return get_bytes(format, len, "string", result); + return get_bytes(len, "string", result); } /*! @@ -4093,7 +4172,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, "invalid string: ill-formed UTF-8 byte", context), nullptr)); + exception_message("invalid string: ill-formed UTF-8 byte", context), nullptr)); } result = sanitize_utf8(result, error_handler); @@ -4104,7 +4183,6 @@ class binary_reader @brief create a byte array by reading bytes from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[out] result byte array created by reading @a len bytes @@ -4115,11 +4193,10 @@ class binary_reader the input before we run out of memory. */ template - bool get_binary(const input_format_t format, - const NumberType len, + bool get_binary(const NumberType len, binary_t& result) { - return get_bytes(format, len, "binary", result); + return get_bytes(len, "binary", result); } /*! @@ -4127,7 +4204,6 @@ class binary_reader @tparam NumberType the type of the length @tparam ContainerType the destination container (string_t or binary_t) - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[in] context further context information (for diagnostics) @param[out] result container the bytes are appended to @@ -4142,8 +4218,7 @@ class binary_reader detects a premature end of input. */ template - bool get_bytes(const input_format_t format, - NumberType len, + bool get_bytes(NumberType len, const char* context, ContainerType& result) { @@ -4174,7 +4249,7 @@ class binary_reader result.resize(old_size + bytes_read); ++chars_read; current = char_traits::eof(); - return unexpect_eof(format, context); + return unexpect_eof(context); } // a full chunk was read; get_elements() never returns more than requested JSON_ASSERT(bytes_read == wanted); @@ -4184,17 +4259,16 @@ class binary_reader } /*! - @param[in] format the current format (for diagnostics) @param[in] context further context information (for diagnostics) @return whether the last read character is not EOF */ - JSON_HEDLEY_NON_NULL(3) - bool unexpect_eof(const input_format_t format, const char* context) const + JSON_HEDLEY_NON_NULL(2) + bool unexpect_eof(const char* context) const { if (JSON_HEDLEY_UNLIKELY(current == char_traits::eof())) { return sax->parse_error(chars_read, "", - parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); } return true; } @@ -4210,18 +4284,16 @@ class binary_reader } /*! - @param[in] format the current format @param[in] detail a detailed error message @param[in] context further context information @return a message string to use in the parse_error exceptions */ - std::string exception_message(const input_format_t format, - const std::string& detail, + std::string exception_message(const std::string& detail, const std::string& context) const { std::string error_msg = "syntax error while parsing "; - switch (format) + switch (input_format) { case input_format_t::cbor: error_msg += "CBOR"; diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index 75684866e..bea2305a2 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -13783,7 +13783,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(current != char_traits::eof())) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(110, chars_read, - exception_message(input_format, concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); + exception_message(concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); } } @@ -13906,7 +13906,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast(document_size) != chars_read - document_start)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); + exception_message(concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); } return true; } @@ -13921,7 +13921,7 @@ class binary_reader // check_bson_document_size() measures the document from here const std::size_t document_start = chars_read; std::int32_t document_size{}; - if (!get_number(input_format_t::bson, document_size)) + if (!get_number(document_size)) { return false; } @@ -13986,7 +13986,7 @@ class binary_reader continue; } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "element list"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("element list"))) { return false; } @@ -14030,7 +14030,7 @@ class binary_reader while (true) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "cstring"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("cstring"))) { return false; } @@ -14100,10 +14100,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); + exception_message(concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!get_string(input_format_t::bson, len - static_cast(1), result))) + if (JSON_HEDLEY_UNLIKELY(!get_string(len - static_cast(1), result))) { return false; } @@ -14112,8 +14112,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, - "BSON string is not null-terminated", + exception_message("BSON string is not null-terminated", "string"), nullptr)); } @@ -14136,18 +14135,18 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); + exception_message(concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); } // All BSON binary values have a subtype std::uint8_t subtype{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::bson, subtype))) + if (JSON_HEDLEY_UNLIKELY(!get_number(subtype))) { return false; } result.set_subtype(subtype); - return get_binary(input_format_t::bson, len, result); + return get_binary(len, result); } /*! @@ -14168,14 +14167,14 @@ class binary_reader case 0x01: // double { double number{}; - return get_number(input_format_t::bson, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x02: // string { std::int32_t len{}; string_t value; - return get_number(input_format_t::bson, len) && get_bson_string(len, value) && sax->string(value); + return get_number(len) && get_bson_string(len, value) && sax->string(value); } case 0x03: // object @@ -14192,13 +14191,13 @@ class binary_reader { std::int32_t len{}; binary_t value; - return get_number(input_format_t::bson, len) && get_bson_binary(len, value) && sax->binary(value); + return get_number(len) && get_bson_binary(len, value) && sax->binary(value); } case 0x08: // boolean { std::uint8_t value{}; - return get_number(input_format_t::bson, value) && sax->boolean(value != 0); + return get_number(value) && sax->boolean(value != 0); } case 0x0A: // null @@ -14209,19 +14208,19 @@ class binary_reader case 0x10: // int32 { std::int32_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x12: // int64 { std::int64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x11: // uint64 { std::uint64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_unsigned(value); + return get_number(value) && emit_unsigned(value); } default: // anything else is not supported (yet) @@ -14243,18 +14242,23 @@ class binary_reader bool get_cbor_negative_integer() { NumberType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } - const auto max_val = static_cast((std::numeric_limits::max)()); - if (number > max_val) + + // the value is -1 - number, which fits into number_integer_t + // whenever number does + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) { - return sax->parse_error(chars_read, get_token_string(), - parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr)); + return sax->number_integer(static_cast(-1) - static_cast(number)); } - return sax->number_integer(conditional_static_cast(static_cast(-1) - static_cast(number))); + + // like the lexer does for JSON text, store a value too small for + // number_integer_t as number_float_t; compute it as long double so + // that emit_float sees a finite value and can detect an overflow of + // number_float_t + return emit_float(static_cast(-1) - static_cast(number)); } /*! @@ -14279,7 +14283,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::cbor, "value"); + return unexpect_eof("value"); // Integer 0x00..0x17 (0..23) case 0x00: @@ -14311,25 +14315,25 @@ class binary_reader case 0x18: // Unsigned integer (one-byte uint8_t follows) { std::uint8_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x19: // Unsigned integer (two-byte uint16_t follows) { std::uint16_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1A: // Unsigned integer (four-byte uint32_t follows) { std::uint32_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1B: // Unsigned integer (eight-byte uint64_t follows) { std::uint64_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } // Negative integer -1-0x00..-1-0x17 (-1..-24) @@ -14550,7 +14554,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } // ignore and store: the tag value is already in the head, so @@ -14572,7 +14576,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } case cbor_tag_handler_t::ignore: @@ -14630,25 +14634,25 @@ class binary_reader return sax->null(); case 0xF9: // Half-Precision Float (two-byte IEEE 754) - return get_half_float(input_format_t::cbor, false); + return get_half_float(false); case 0xFA: // Single-Precision Float (four-byte IEEE 754) { float number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xFB: // Double-Precision Float (eight-byte IEEE 754) { double number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } default: // anything else (0xFF is handled inside the other types) { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -14697,38 +14701,38 @@ class binary_reader case 0x76: case 0x77: { - return get_string(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_string(static_cast(current) & 0x1Fu, result); } case 0x78: // UTF-8 string (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x79: // UTF-8 string (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7A: // UTF-8 string (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -14756,7 +14760,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -14865,7 +14869,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -14912,42 +14916,42 @@ class binary_reader case 0x56: case 0x57: { - return get_binary(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_binary(static_cast(current) & 0x1Fu, result); } case 0x58: // Binary data (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x59: // Binary data (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5A: // Binary data (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5B: // Binary data (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); + exception_message(concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); } } } @@ -14971,7 +14975,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "binary"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("binary"))) { return false; } @@ -15029,7 +15033,7 @@ class binary_reader case 0x18: // 1 byte { std::uint8_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15040,7 +15044,7 @@ class binary_reader case 0x19: // 2 bytes { std::uint16_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15051,7 +15055,7 @@ class binary_reader case 0x1A: // 4 bytes { std::uint32_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15062,7 +15066,7 @@ class binary_reader case 0x1B: // 8 bytes { std::uint64_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15095,7 +15099,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(len) || len == detail::unknown_size())) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format_t::cbor, concat("excessive ", context, " size"), "size"), nullptr)); + exception_message(concat("excessive ", context, " size"), "size"), nullptr)); } result = conditional_static_cast(len); return true; @@ -15224,7 +15228,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::msgpack, "value"); + return unexpect_eof("value"); // positive fixint case 0x00: @@ -15464,85 +15468,85 @@ class binary_reader case 0xCA: // float 32 { float number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCB: // float 64 { double number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCC: // uint 8 { std::uint8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCD: // uint 16 { std::uint16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCE: // uint 32 { std::uint32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCF: // uint 64 { std::uint64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xD0: // int 8 { std::int8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD1: // int 16 { std::int16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD2: // int 32 { std::int32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD3: // int 64 { std::int64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xDC: // array 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(static_cast(len)); + return get_number(len) && enter_array(static_cast(len)); } case 0xDD: // array 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(conditional_static_cast(len)); + return get_number(len) && enter_array(conditional_static_cast(len)); } case 0xDE: // map 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(static_cast(len)); + return get_number(len) && enter_object(static_cast(len)); } case 0xDF: // map 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(conditional_static_cast(len)); + return get_number(len) && enter_object(conditional_static_cast(len)); } // negative fixint @@ -15584,7 +15588,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::msgpack, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -15601,7 +15605,7 @@ class binary_reader */ bool get_msgpack_string(string_t& result, const char* context = "string") { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::msgpack, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -15642,32 +15646,32 @@ class binary_reader case 0xBE: case 0xBF: { - return get_string(input_format_t::msgpack, static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); + return get_string(static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); } case 0xD9: // str 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDA: // str 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDB: // str 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -15761,7 +15765,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -15788,31 +15792,31 @@ class binary_reader case 0xC4: // bin 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC5: // bin 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC6: // bin 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC7: // ext 8 { std::uint8_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -15820,9 +15824,9 @@ class binary_reader { std::uint16_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -15830,49 +15834,49 @@ class binary_reader { std::uint32_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } case 0xD4: // fixext 1 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 1, result) && + return get_number(subtype) && + get_binary(1, result) && assign_and_return_true(subtype); } case 0xD5: // fixext 2 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 2, result) && + return get_number(subtype) && + get_binary(2, result) && assign_and_return_true(subtype); } case 0xD6: // fixext 4 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 4, result) && + return get_number(subtype) && + get_binary(4, result) && assign_and_return_true(subtype); } case 0xD7: // fixext 8 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 8, result) && + return get_number(subtype) && + get_binary(8, result) && assign_and_return_true(subtype); } case 0xD8: // fixext 16 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 16, result) && + return get_number(subtype) && + get_binary(16, result) && assign_and_return_true(subtype); } @@ -16071,7 +16075,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(len < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "string length must not be negative", "string"), nullptr)); + exception_message("string length must not be negative", "string"), nullptr)); } return true; } @@ -16101,7 +16105,7 @@ class binary_reader get(); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } @@ -16111,31 +16115,31 @@ class binary_reader case 'U': { std::uint8_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'i': { std::int8_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'I': { std::int16_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'l': { std::int32_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'L': { std::int64_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'u': @@ -16145,7 +16149,7 @@ class binary_reader break; } std::uint16_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'm': @@ -16155,7 +16159,7 @@ class binary_reader break; } std::uint32_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'M': @@ -16165,7 +16169,7 @@ class binary_reader break; } std::uint64_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: @@ -16182,7 +16186,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "string"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "string"), nullptr)); } /*! @@ -16261,19 +16265,19 @@ class binary_reader bool get_ubjson_signed_count(std::size_t& result) { SignedType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (JSON_HEDLEY_UNLIKELY(number < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr)); + exception_message("count in an optimized container must be positive", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(number))) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = static_cast(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char return true; @@ -16302,7 +16306,7 @@ class binary_reader case 'U': { std::uint8_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -16329,7 +16333,7 @@ class binary_reader break; } std::uint16_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -16344,7 +16348,7 @@ class binary_reader break; } std::uint32_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -16359,14 +16363,14 @@ class binary_reader break; } std::uint64_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (!value_in_range_of(number)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = detail::conditional_static_cast(number); return true; @@ -16380,7 +16384,7 @@ class binary_reader } if (is_ndarray) // ndarray dimensional vector can only contain integers and cannot embed another array { - return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message(input_format, "ndarray dimensional vector is not allowed", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message("ndarray dimensional vector is not allowed", "size"), nullptr)); } std::vector dim; if (JSON_HEDLEY_UNLIKELY(!get_ubjson_ndarray_size(dim))) @@ -16420,7 +16424,7 @@ class binary_reader // or SIZE_MAX. if (JSON_HEDLEY_UNLIKELY(result > (std::numeric_limits::max)() / i)) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } result *= i; // the pre-check above already rules out result becoming 0 @@ -16429,9 +16433,9 @@ class binary_reader // unknown-size container (see get_ubjson_size_type()) if (result == npos) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast(i)))) + if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i))) { return false; } @@ -16457,7 +16461,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "size"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "size"), nullptr)); } /*! @@ -16492,10 +16496,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); + exception_message(concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "type"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type"))) { return false; } @@ -16503,13 +16507,13 @@ class binary_reader get_ignore_noop(); if (JSON_HEDLEY_UNLIKELY(current != '#')) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); + exception_message(concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); } const bool is_error = get_ubjson_size_value(result.first, is_ndarray); @@ -16529,7 +16533,7 @@ class binary_reader if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format, "ndarray requires both type and size", "size"), nullptr)); + exception_message("ndarray requires both type and size", "size"), nullptr)); } return is_error; } @@ -16546,7 +16550,7 @@ class binary_reader switch (prefix) { case char_traits::eof(): // EOF - return unexpect_eof(input_format, "value"); + return unexpect_eof("value"); case 'T': // true return sax->boolean(true); @@ -16563,37 +16567,37 @@ class binary_reader break; } std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'U': { std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'i': { std::int8_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'I': { std::int16_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'l': { std::int32_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'L': { std::int64_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'u': @@ -16603,7 +16607,7 @@ class binary_reader break; } std::uint16_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'm': @@ -16613,7 +16617,7 @@ class binary_reader break; } std::uint32_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'M': @@ -16623,7 +16627,7 @@ class binary_reader break; } std::uint64_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'h': @@ -16632,19 +16636,19 @@ class binary_reader { break; } - return get_half_float(input_format, true); + return get_half_float(true); } case 'd': { float number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'D': { double number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'H': @@ -16655,7 +16659,7 @@ class binary_reader case 'C': // char { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "char"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("char"))) { return false; } @@ -16663,7 +16667,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); + exception_message(concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); } string_t s(1, static_cast(current)); return sax->string(s); @@ -16685,7 +16689,7 @@ class binary_reader break; } auto last_token = get_token_string(); - return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format, "invalid byte: 0x" + last_token, "value"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message("invalid byte: 0x" + last_token, "value"), nullptr)); } /*! @@ -16711,7 +16715,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "invalid byte: 0x" + last_token, "type"), nullptr)); + exception_message("invalid byte: 0x" + last_token, "type"), nullptr)); } string_t type = type_name; // sax->string() takes a reference @@ -16746,7 +16750,7 @@ class binary_reader if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B') { binary_t result; - return get_binary(input_format, size_and_type.first, result) && sax->binary(result); + return get_binary(size_and_type.first, result) && sax->binary(result); } if (size_and_type.first != npos) @@ -16759,7 +16763,7 @@ class binary_reader && size_and_type.first > max_valueless_container_size)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "excessive array size", "size"), nullptr)); + exception_message("excessive array size", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second))) @@ -16796,7 +16800,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "BJData object does not support ND-array size in optimized format", "object"), nullptr)); + exception_message("BJData object does not support ND-array size in optimized format", "object"), nullptr)); } if (size_and_type.first != npos) @@ -16826,7 +16830,7 @@ class binary_reader for (std::size_t i = 0; i < size; ++i) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -16845,7 +16849,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input)) { return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } switch (result_number) @@ -16885,7 +16889,7 @@ class binary_reader case token_type::literal_or_value: default: return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } } @@ -16954,7 +16958,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr)); + exception_message(concat(detail, ": 0x", last_token), context), nullptr)); } /*! @@ -17058,7 +17062,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "value"); + return unexpect_eof("value"); } // string: ASCII character @@ -17095,25 +17099,25 @@ class binary_reader case 0x8C: // int32 { std::int32_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8D: // int64 { std::int64_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8E: // binary32 { float number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x8F: // binary64 { double number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xF8: @@ -17179,7 +17183,9 @@ class binary_reader @brief pass an integer to the SAX parser Non-negative integers are passed as unsigned, negative integers as signed - numbers, like the other binary formats do. + numbers, like the other binary formats do. A value that does not fit the + number type is passed as described for @ref emit_unsigned and + @ref emit_signed. @param[in] number the integer @return whether the SAX parser accepted the value @@ -17188,9 +17194,9 @@ class binary_reader { if (number >= 0) { - return sax->number_unsigned(static_cast(number)); + return emit_unsigned(static_cast(number)); } - return sax->number_integer(static_cast(number)); + return emit_signed(number); } /*! @@ -17210,7 +17216,7 @@ class binary_reader */ bool get_bon8_integer(const char_int_type lead, const char_int_type second) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -17242,13 +17248,12 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "number"); + return unexpect_eof("number"); } value = (value << 8) | static_cast(current); } - return negative ? sax->number_integer(static_cast(-(value + offset))) - : sax->number_unsigned(static_cast(value + offset)); + return emit_bon8_integer(negative ? -(value + offset) : value + offset); } /*! @@ -17266,7 +17271,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } if (byte == 0xFF) @@ -17286,7 +17291,7 @@ class binary_reader if (second == char_traits::eof()) { // the input ends inside a character or an integer - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } unget_bon8(second); if (is_bon8_continuation(second)) @@ -17359,7 +17364,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } // end of string @@ -17390,7 +17395,7 @@ class binary_reader { // the input ends inside a character or an integer: either // way, the message is incomplete - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (!is_bon8_continuation(second)) { @@ -17431,7 +17436,7 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current))) { @@ -17470,7 +17475,7 @@ class binary_reader @return bool, whether the read was successful */ template - bool get_to(T& dest, const input_format_t format, const char* context) + bool get_to(T& dest, const char* context) { // false positive: new_chars_read is read on the next lines // @infer-ignore DEAD_STORE @@ -17480,7 +17485,7 @@ class binary_reader { // in case of failure, advance position by 1 to report the failing location ++chars_read; - sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); return false; } return true; @@ -17531,7 +17536,6 @@ class binary_reader @brief read a number from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[out] result number of type @a NumberType @return whether conversion completed @@ -17543,42 +17547,119 @@ class binary_reader should reorder on big endian systems. */ template - bool get_number(const input_format_t format, NumberType& result) + bool get_number(NumberType& result) { // read in the original format - if (JSON_HEDLEY_UNLIKELY(!get_to(result, format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!get_to(result, "number"))) { return false; } - if (is_little_endian != (InputIsLittleEndian || format == input_format_t::bjdata)) + if (is_little_endian != (InputIsLittleEndian || input_format == input_format_t::bjdata)) { byte_swap(result); } return true; } + /*! + @brief pass a signed integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_integer_t is passed as number_unsigned_t if it is non-negative and + fits there, and as number_float_t otherwise. With the default number + types, every integer the binary formats can encode fits, so this only + matters for narrower custom number types. + + @tparam NumberType a signed integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_signed(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_integer(static_cast(number)); + } + if (value_in_range_of(number)) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass an unsigned integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_unsigned_t is passed as number_float_t. + + @tparam NumberType an unsigned integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_unsigned(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass a floating-point number read from the input to the SAX parser + + Like the lexer does for JSON text, a finite value that overflows + number_float_t is rejected instead of silently becoming infinity. Infinity + and NaN in the input are passed on unchanged. Integers only overflow if + number_float_t cannot represent 2^64, e.g., a half-precision type. + + @tparam NumberType a floating-point or integer type + @param[in] number the number + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if a finite @a number overflows number_float_t + */ + template + bool emit_float(const NumberType number) + { + const auto result = static_cast(number); + if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result))) + { + return sax->parse_error(chars_read, get_token_string(), + out_of_range::create(406, exception_message("number overflow", "value"), nullptr)); + } + return sax->number_float(result, ""); + } + /*! @brief read and decode an IEEE 754 half-precision (16-bit) float Used by CBOR (big endian) and BJData (little endian); the two formats only differ in the byte order of the two bytes that make up the half. - - @param[in] format the current format (for diagnostics) @param[in] little_endian whether the two bytes are little endian (BJData) or big endian (CBOR) @return whether reading and decoding succeeded */ - bool get_half_float(const input_format_t format, const bool little_endian) + bool get_half_float(const bool little_endian) { const auto byte1_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } const auto byte2_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -17624,7 +17705,6 @@ class binary_reader @brief create a string by reading characters from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of characters to read @param[out] result string created by reading @a len bytes @@ -17635,8 +17715,7 @@ class binary_reader the input before we run out of string memory. */ template - bool get_string(const input_format_t format, - const NumberType len, + bool get_string(const NumberType len, string_t& result) { // Strings are taken as is by default: none of CBOR (RFC 8949 ยง3.1 @@ -17648,7 +17727,7 @@ class binary_reader // @ref error_handler, applied once the whole string (all chunks of // an indefinite-length CBOR string included) has been assembled, by // @ref check_string_utf8 at the call site. - return get_bytes(format, len, "string", result); + return get_bytes(len, "string", result); } /*! @@ -17679,7 +17758,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, "invalid string: ill-formed UTF-8 byte", context), nullptr)); + exception_message("invalid string: ill-formed UTF-8 byte", context), nullptr)); } result = sanitize_utf8(result, error_handler); @@ -17690,7 +17769,6 @@ class binary_reader @brief create a byte array by reading bytes from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[out] result byte array created by reading @a len bytes @@ -17701,11 +17779,10 @@ class binary_reader the input before we run out of memory. */ template - bool get_binary(const input_format_t format, - const NumberType len, + bool get_binary(const NumberType len, binary_t& result) { - return get_bytes(format, len, "binary", result); + return get_bytes(len, "binary", result); } /*! @@ -17713,7 +17790,6 @@ class binary_reader @tparam NumberType the type of the length @tparam ContainerType the destination container (string_t or binary_t) - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[in] context further context information (for diagnostics) @param[out] result container the bytes are appended to @@ -17728,8 +17804,7 @@ class binary_reader detects a premature end of input. */ template - bool get_bytes(const input_format_t format, - NumberType len, + bool get_bytes(NumberType len, const char* context, ContainerType& result) { @@ -17760,7 +17835,7 @@ class binary_reader result.resize(old_size + bytes_read); ++chars_read; current = char_traits::eof(); - return unexpect_eof(format, context); + return unexpect_eof(context); } // a full chunk was read; get_elements() never returns more than requested JSON_ASSERT(bytes_read == wanted); @@ -17770,17 +17845,16 @@ class binary_reader } /*! - @param[in] format the current format (for diagnostics) @param[in] context further context information (for diagnostics) @return whether the last read character is not EOF */ - JSON_HEDLEY_NON_NULL(3) - bool unexpect_eof(const input_format_t format, const char* context) const + JSON_HEDLEY_NON_NULL(2) + bool unexpect_eof(const char* context) const { if (JSON_HEDLEY_UNLIKELY(current == char_traits::eof())) { return sax->parse_error(chars_read, "", - parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); } return true; } @@ -17796,18 +17870,16 @@ class binary_reader } /*! - @param[in] format the current format @param[in] detail a detailed error message @param[in] context further context information @return a message string to use in the parse_error exceptions */ - std::string exception_message(const input_format_t format, - const std::string& detail, + std::string exception_message(const std::string& detail, const std::string& context) const { std::string error_msg = "syntax error while parsing "; - switch (format) + switch (input_format) { case input_format_t::cbor: error_msg += "CBOR"; diff --git a/tests/src/unit-binary_formats.cpp b/tests/src/unit-binary_formats.cpp index ed6d89911..846a092fb 100644 --- a/tests/src/unit-binary_formats.cpp +++ b/tests/src/unit-binary_formats.cpp @@ -11,7 +11,12 @@ #include using nlohmann::json; +#include #include +#include +#include +#include +#include #include "make_test_data_available.hpp" TEST_CASE("Binary Formats" * doctest::skip()) @@ -224,3 +229,139 @@ TEST_CASE("Binary Formats" * doctest::skip()) CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450)); } } + +namespace +{ +// the binary formats as function pointers for "Binary formats with narrow number types"; +// named functions rather than lambdas, because clang 3.5 cannot convert a lambda +// to a function pointer in the braced initializer of the format table +using narrow_json = nlohmann::basic_json; +using bytes = std::vector; + +bytes encode_cbor(const json& j) +{ + return json::to_cbor(j); +} +narrow_json decode_cbor(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_cbor(v, true, allow_exceptions); +} + +bytes encode_msgpack(const json& j) +{ + return json::to_msgpack(j); +} +narrow_json decode_msgpack(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_msgpack(v, true, allow_exceptions); +} + +bytes encode_ubjson(const json& j) +{ + return json::to_ubjson(j); +} +narrow_json decode_ubjson(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_ubjson(v, true, allow_exceptions); +} + +bytes encode_bjdata(const json& j) +{ + return json::to_bjdata(j); +} +narrow_json decode_bjdata(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_bjdata(v, true, allow_exceptions); +} + +// BSON can only store numbers as object members +bytes encode_bson(const json& j) +{ + return json::to_bson(json{{"a", j}}); +} +narrow_json decode_bson(const bytes& v, bool allow_exceptions) +{ + const auto result = narrow_json::from_bson(v, true, allow_exceptions); + return result.is_discarded() ? result : result.at("a"); +} + +bytes encode_bon8(const json& j) +{ + return json::to_bon8(j); +} +narrow_json decode_bon8(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_bon8(v, true, allow_exceptions); +} + +} // namespace + +TEST_CASE("Binary formats with narrow number types") +{ + // Numbers that do not fit the number types are handled like the lexer + // handles them in JSON text: an integer that fits neither integer type is + // stored as a floating-point number, and a finite floating-point number + // that overflows number_float_t is rejected with out_of_range.406. + struct binary_format + { + const char* name; + bytes (*encode)(const json&); + narrow_json (*decode)(const bytes&, bool); + }; + + const std::vector formats = + { + {"CBOR", encode_cbor, decode_cbor}, + {"MessagePack", encode_msgpack, decode_msgpack}, + {"UBJSON", encode_ubjson, decode_ubjson}, + {"BJData", encode_bjdata, decode_bjdata}, + {"BSON", encode_bson, decode_bson}, + {"BON8", encode_bon8, decode_bon8}, + }; + + for (const auto& format : formats) + { + const std::string name = format.name; + INFO("format := ", name); + const auto roundtrip = [&format](const json & j) + { + return format.decode(format.encode(j), true); + }; + + // integers that fit keep their type + CHECK(roundtrip(json(-5)).is_number_integer()); + CHECK(roundtrip(json(-5)).get() == -5); + CHECK(roundtrip(json(3000000000u)).is_number_unsigned()); + CHECK(roundtrip(json(3000000000u)).get() == 3000000000u); + + // integers that fit neither integer type are stored as float + CHECK(roundtrip(json(5000000000u)).is_number_float()); + CHECK(roundtrip(json(5000000000u)).get() == 5000000000.0f); + if (name != "BON8") // BON8 cannot encode integers above INT64_MAX + { + CHECK(roundtrip(json(10000000000000000000u)).is_number_float()); + CHECK(roundtrip(json(10000000000000000000u)).get() == 10000000000000000000.0f); + } + CHECK(roundtrip(json(-3000000000LL)).is_number_float()); + CHECK(roundtrip(json(-3000000000LL)).get() == -3000000000.0f); + CHECK(roundtrip(json(-5000000000LL)).is_number_float()); + CHECK(roundtrip(json(-5000000000LL)).get() == -5000000000.0f); + + // floating-point numbers that fit + CHECK(roundtrip(json(1.5)).get() == 1.5f); + const auto just_above_max = std::nextafter(static_cast((std::numeric_limits::max)()), + std::numeric_limits::infinity()); + CHECK(roundtrip(json(just_above_max)).get() == (std::numeric_limits::max)()); + + // infinity and NaN are passed on + CHECK(std::isinf(roundtrip(json(std::numeric_limits::infinity())).get())); + CHECK(std::isnan(roundtrip(json(std::numeric_limits::quiet_NaN())).get())); + + // finite floating-point numbers that overflow number_float_t are rejected + const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name + + " value: number overflow"; + CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&); + CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&); + CHECK(format.decode(format.encode(json(1e300)), false).is_discarded()); + } +} diff --git a/tests/src/unit-cbor.cpp b/tests/src/unit-cbor.cpp index f3f4301e5..546e256b1 100644 --- a/tests/src/unit-cbor.cpp +++ b/tests/src/unit-cbor.cpp @@ -3185,7 +3185,8 @@ TEST_CASE("Tagged values") // CBOR encodes negative integers as: result = -1 - n // For type 0x3B, n is an 8-byte uint64_t. Valid range for n with // the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1]. - // When n > INT64_MAX, the result exceeds int64_t range and is rejected. + // When n > INT64_MAX, the result exceeds int64_t range and is stored + // as a floating-point number, as the lexer does for JSON text. SECTION("n = 0 is valid (result = -1)") { @@ -3206,33 +3207,34 @@ TEST_CASE("Tagged values") CHECK(result.get() == (std::numeric_limits::min)()); } - SECTION("n = INT64_MAX + 1 is rejected (overflow)") + SECTION("n = INT64_MAX + 1 is stored as float") { // n = INT64_MAX + 1 (0x8000000000000000) - // result = -1 - n = -9223372036854775809, which exceeds int64_t range + // result = -1 - n = -9223372036854775809, which exceeds int64_t range; + // the nearest double is -9223372036854775808.0 const std::vector input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; - json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), - "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow", - json::parse_error); + const auto result = json::from_cbor(input); + CHECK(result.is_number_float()); + CHECK(result.get() == -9223372036854775808.0); + CHECK(result == json::parse("-9223372036854775809")); } - SECTION("n = UINT64_MAX is rejected (overflow)") + SECTION("n = UINT64_MAX is stored as float") { // n = UINT64_MAX (0xFFFFFFFFFFFFFFFF) // result = -1 - n = -18446744073709551616, which exceeds int64_t range const std::vector input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; - json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), - "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow", - json::parse_error); + const auto result = json::from_cbor(input); + CHECK(result.is_number_float()); + CHECK(result.get() == -18446744073709551616.0); + CHECK(result == json::parse("-18446744073709551616")); } - SECTION("overflow with allow_exceptions=false returns discarded") + SECTION("overflow with allow_exceptions=false is not an error") { const std::vector input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; const auto result = json::from_cbor(input, true, false); - CHECK(result.is_discarded()); + CHECK(result.is_number_float()); } }