mirror of
https://github.com/nlohmann/json.git
synced 2026-09-29 11:10:30 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Niels Lohmann
|
93fe62fc0b |
@@ -80,8 +80,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
the end of the file was not reached when `strict` was set to true
|
||||
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from CBOR were
|
||||
used in the given input or if the input is not valid CBOR
|
||||
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other
|
||||
types are not supported, as JSON object keys are always strings) or a string is malformed
|
||||
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key,
|
||||
but not found
|
||||
|
||||
## Complexity
|
||||
|
||||
|
||||
@@ -73,8 +73,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
the end of the file was not reached when `strict` was set to true
|
||||
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from
|
||||
MessagePack were used in the given input or if the input is not valid MessagePack
|
||||
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other
|
||||
types are not supported, as JSON object keys are always strings) or a string is malformed
|
||||
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key,
|
||||
but not found
|
||||
|
||||
## Complexity
|
||||
|
||||
|
||||
@@ -55,10 +55,6 @@ This implementation does exactly follow this approach, as it uses double precisi
|
||||
smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
|
||||
and be serialized to `null`.
|
||||
|
||||
During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
|
||||
`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
|
||||
example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
|
||||
|
||||
#### Storage
|
||||
|
||||
Floating-point number values are stored directly inside a `basic_json` type.
|
||||
|
||||
@@ -47,9 +47,8 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va
|
||||
|
||||
When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
|
||||
the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
|
||||
range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
|
||||
formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
|
||||
or [`number_float_t`](number_float_t.md).
|
||||
range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
|
||||
will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).
|
||||
|
||||
[RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
|
||||
> Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
|
||||
|
||||
@@ -48,9 +48,8 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default
|
||||
|
||||
When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
|
||||
the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
|
||||
when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
|
||||
integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
|
||||
[`number_float_t`](number_float_t.md).
|
||||
when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
|
||||
as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).
|
||||
|
||||
[RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
|
||||
> Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
|
||||
|
||||
@@ -86,6 +86,9 @@ Strong exception safety: if an exception occurs, the original value stays intact
|
||||
- Throws [`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) if an array index in the passed
|
||||
JSON pointer `ptr` exceeds the range of `size_type` (e.g., on 32-bit platforms).
|
||||
|
||||
For the **const** version, an object key or array index in `ptr` that does not exist is not reported by an
|
||||
exception, but is undefined behavior (see the notes below). Use [`at`](at.md) for checked access.
|
||||
|
||||
## Complexity
|
||||
|
||||
1. Constant if `idx` is in the range of the array. Otherwise, linear in `idx - size()`.
|
||||
@@ -100,9 +103,12 @@ Strong exception safety: if an exception occurs, the original value stays intact
|
||||
The following cases apply to the **const** overloads; the non-const overloads instead insert the missing element
|
||||
(see the notes below).
|
||||
|
||||
1. If the element at index `idx` does not exist, the behavior is undefined.
|
||||
1. If the element at index `idx` does not exist, the behavior is undefined and is **guarded by a
|
||||
[runtime assertion](../../features/assertions.md)**!
|
||||
2. If the element with key `key` does not exist, the behavior is undefined and is **guarded by a
|
||||
[runtime assertion](../../features/assertions.md)**!
|
||||
3. If the JSON pointer `ptr` refers to an object key or an array index that does not exist, the behavior is
|
||||
undefined and is **guarded by a [runtime assertion](../../features/assertions.md)**!
|
||||
|
||||
1. The non-const version may add values: If `idx` is beyond the range of the array (i.e., `idx >= size()`), then the
|
||||
array is silently filled up with `#!json null` values to make `idx` a valid reference to the last stored element. In
|
||||
@@ -257,9 +263,11 @@ Strong exception safety: if an exception occurs, the original value stays intact
|
||||
|
||||
## Version history
|
||||
|
||||
1. Added in version 1.0.0.
|
||||
1. Added in version 1.0.0. A missing index in the const version is guarded by a runtime assertion since version
|
||||
3.13.0.
|
||||
2. Added in version 1.0.0. Added overloads for `T* key` in version 1.1.0. Removed overloads for `T* key` (replaced by 3)
|
||||
in version 3.11.0.
|
||||
3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as
|
||||
already supported by [`at`](at.md), [`value`](value.md), [`find`](find.md), and other lookup functions.
|
||||
4. Added in version 2.0.0.
|
||||
4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
|
||||
version 3.13.0.
|
||||
|
||||
@@ -16,14 +16,15 @@ before including the `json.hpp` header.
|
||||
|
||||
## Function with runtime assertions
|
||||
|
||||
### Unchecked object access to a const value
|
||||
### Unchecked access to a const value
|
||||
|
||||
Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for objects. Whereas a missing
|
||||
key is added in the case of non-const objects, accessing a const object with a missing key is undefined behavior (think
|
||||
of a dereferenced null pointer) and yields a runtime assertion.
|
||||
Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for arrays and objects. Whereas
|
||||
a missing element is added in the case of non-const values, accessing a const value with a missing object key or an
|
||||
invalid array index is undefined behavior (think of a dereferenced null pointer) and yields a runtime assertion. This
|
||||
also applies to a [JSON pointer](json_pointer.md) that refers to a missing key or an invalid index.
|
||||
|
||||
If you are not sure whether an element in an object exists, use checked access with the
|
||||
[`at` function](../api/basic_json/at.md) or call the [`contains` function](../api/basic_json/contains.md) before.
|
||||
If you are not sure whether an element exists, use checked access with the [`at` function](../api/basic_json/at.md)
|
||||
or call the [`contains` function](../api/basic_json/contains.md) before.
|
||||
|
||||
See also the documentation on [element access](element_access/index.md).
|
||||
|
||||
@@ -46,7 +47,30 @@ See also the documentation on [element access](element_access/index.md).
|
||||
Output:
|
||||
|
||||
```
|
||||
Assertion failed: (m_value.object->find(key) != m_value.object->end()), function operator[], file json.hpp, line 2144.
|
||||
Assertion failed: (it != m_data.m_value.object->end()), function operator[], file json.hpp, line 28795.
|
||||
```
|
||||
|
||||
??? example "Example 2: Invalid array index in a JSON pointer"
|
||||
|
||||
The following code will trigger an assertion at runtime:
|
||||
|
||||
```cpp
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
using json = nlohmann::json;
|
||||
using namespace nlohmann::literals;
|
||||
|
||||
int main()
|
||||
{
|
||||
const json j = {{"array", {1, 2, 3}}};
|
||||
auto v = j["/array/5"_json_pointer];
|
||||
}
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
Assertion failed: (idx < m_data.m_value.array->size()), function operator[], file json.hpp, line 28758.
|
||||
```
|
||||
|
||||
### Constructing from an uninitialized iterator range
|
||||
|
||||
@@ -168,26 +168,13 @@ The library maps CBOR types to JSON value types as follows:
|
||||
!!! warning "Negative integer overflow"
|
||||
|
||||
CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
|
||||
result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
|
||||
a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
|
||||
stored as `-1.8446744073709552e+19`.
|
||||
result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
|
||||
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
|
||||
silently.
|
||||
|
||||
!!! warning "Object keys"
|
||||
|
||||
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps
|
||||
with keys other than text strings (major type 3) are rejected with a
|
||||
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set
|
||||
to `false`, a discarded value) naming the type of the key that was found, for instance:
|
||||
|
||||
```
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
|
||||
```
|
||||
|
||||
This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
|
||||
on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR
|
||||
maps with integer keys, such as COSE ([RFC 9052](https://www.rfc-editor.org/rfc/rfc9052.html)) or CWT
|
||||
([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a
|
||||
general-purpose CBOR library instead.
|
||||
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected.
|
||||
|
||||
!!! warning "UTF-8 validation of text strings"
|
||||
|
||||
|
||||
@@ -138,21 +138,6 @@ The library maps MessagePack types to JSON value types as follows:
|
||||
|
||||
Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
|
||||
|
||||
!!! warning "Object keys"
|
||||
|
||||
MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the
|
||||
JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the
|
||||
MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are
|
||||
rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
|
||||
`allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance:
|
||||
|
||||
```
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
|
||||
```
|
||||
|
||||
This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
|
||||
on. Such input needs a general-purpose MessagePack library instead.
|
||||
|
||||
!!! warning "UTF-8 validation of string values"
|
||||
|
||||
The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8.
|
||||
|
||||
@@ -331,6 +331,9 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
|
||||
[json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
|
||||
```
|
||||
```
|
||||
[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
|
||||
```
|
||||
```
|
||||
[json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
|
||||
```
|
||||
|
||||
@@ -340,20 +343,13 @@ A string could not be read from a [binary format](../features/binary_formats/ind
|
||||
string was read where one was required (for instance as a map key), the string's length specification is invalid, or
|
||||
the string's bytes are not valid UTF-8.
|
||||
|
||||
CBOR and MessagePack allow map keys of any type, but JSON object keys are always strings. Maps with keys of any other
|
||||
type (for instance integers or `null`) are therefore not supported; see the notes on
|
||||
[CBOR](../features/binary_formats/cbor.md) and [MessagePack](../features/binary_formats/messagepack.md).
|
||||
|
||||
!!! failure "Example messages"
|
||||
|
||||
```
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF
|
||||
```
|
||||
```
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
|
||||
```
|
||||
```
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF
|
||||
```
|
||||
```
|
||||
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82
|
||||
@@ -851,18 +847,13 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme
|
||||
|
||||
### json.exception.out_of_range.406
|
||||
|
||||
A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
|
||||
finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
|
||||
double-precision number when `number_float_t` is `#!cpp float`.
|
||||
A parsed number could not be stored as without changing it to NaN or INF.
|
||||
|
||||
!!! failure "Example messages"
|
||||
!!! failure "Example message"
|
||||
|
||||
```
|
||||
number overflow parsing '10E1000'
|
||||
```
|
||||
```
|
||||
[json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
|
||||
```
|
||||
|
||||
### json.exception.out_of_range.407
|
||||
|
||||
|
||||
@@ -559,7 +559,7 @@ class binary_reader
|
||||
case 0x01: // double
|
||||
{
|
||||
double number{};
|
||||
return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
|
||||
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0x02: // string
|
||||
@@ -600,19 +600,19 @@ class binary_reader
|
||||
case 0x10: // int32
|
||||
{
|
||||
std::int32_t value{};
|
||||
return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
|
||||
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
|
||||
}
|
||||
|
||||
case 0x12: // int64
|
||||
{
|
||||
std::int64_t value{};
|
||||
return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
|
||||
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
|
||||
}
|
||||
|
||||
case 0x11: // uint64
|
||||
{
|
||||
std::uint64_t value{};
|
||||
return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(input_format_t::bson, value);
|
||||
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
|
||||
}
|
||||
|
||||
default: // anything else is not supported (yet)
|
||||
@@ -638,19 +638,14 @@ class binary_reader
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// the value is -1 - number, which fits into number_integer_t
|
||||
// whenever number does
|
||||
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
|
||||
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
|
||||
if (number > max_val)
|
||||
{
|
||||
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
|
||||
return sax->parse_error(chars_read, get_token_string(),
|
||||
parse_error::create(112, chars_read,
|
||||
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
|
||||
}
|
||||
|
||||
// like the lexer does for JSON text, store a value too small for
|
||||
// number_integer_t as number_float_t; compute it as long double so
|
||||
// that emit_float sees a finite value and can detect an overflow of
|
||||
// number_float_t
|
||||
return emit_float(input_format_t::cbor, static_cast<long double>(-1) - static_cast<long double>(number));
|
||||
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
|
||||
}
|
||||
|
||||
/*!
|
||||
@@ -707,25 +702,25 @@ class binary_reader
|
||||
case 0x18: // Unsigned integer (one-byte uint8_t follows)
|
||||
{
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0x19: // Unsigned integer (two-byte uint16_t follows)
|
||||
{
|
||||
std::uint16_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
|
||||
{
|
||||
std::uint32_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
|
||||
{
|
||||
std::uint64_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
// Negative integer -1-0x00..-1-0x17 (-1..-24)
|
||||
@@ -1170,13 +1165,13 @@ class binary_reader
|
||||
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
default: // anything else (0xFF is handled inside the other types)
|
||||
@@ -1329,80 +1324,6 @@ class binary_reader
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a CBOR object key
|
||||
|
||||
RFC 8949 allows any data item as a map key, but only strings have a
|
||||
counterpart in JSON. A key of any other type is rejected with a message
|
||||
naming that type, rather than the one @ref get_cbor_string gives for a
|
||||
malformed string.
|
||||
|
||||
@param[out] result created key
|
||||
|
||||
@return whether key creation completed
|
||||
*/
|
||||
bool get_cbor_object_key(string_t& result)
|
||||
{
|
||||
// EOF and major type 3 (text string) are left to get_cbor_string
|
||||
if (current == char_traits<char_type>::eof() || (static_cast<unsigned int>(current) & 0xE0u) == 0x60u)
|
||||
{
|
||||
return get_cbor_string(result);
|
||||
}
|
||||
|
||||
const char* found = nullptr;
|
||||
switch (static_cast<unsigned int>(current) >> 5u)
|
||||
{
|
||||
case 0:
|
||||
found = "an unsigned integer";
|
||||
break;
|
||||
case 1:
|
||||
found = "a negative integer";
|
||||
break;
|
||||
case 2:
|
||||
found = "a byte string";
|
||||
break;
|
||||
case 4:
|
||||
found = "an array";
|
||||
break;
|
||||
case 5:
|
||||
found = "a map";
|
||||
break;
|
||||
case 6:
|
||||
found = "a tag";
|
||||
break;
|
||||
default: // major type 7
|
||||
switch (current)
|
||||
{
|
||||
case 0xF4:
|
||||
case 0xF5:
|
||||
found = "a boolean";
|
||||
break;
|
||||
case 0xF6:
|
||||
found = "null";
|
||||
break;
|
||||
case 0xF7:
|
||||
found = "undefined";
|
||||
break;
|
||||
case 0xF9:
|
||||
case 0xFA:
|
||||
case 0xFB:
|
||||
found = "a floating-point number";
|
||||
break;
|
||||
case 0xFF:
|
||||
found = "a break stop code";
|
||||
break;
|
||||
default:
|
||||
found = "a simple value";
|
||||
break;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
auto last_token = get_token_string();
|
||||
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
||||
exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a definite-length CBOR byte array
|
||||
|
||||
@@ -1647,7 +1568,7 @@ class binary_reader
|
||||
if (top.is_object)
|
||||
{
|
||||
key.clear();
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -1940,61 +1861,61 @@ class binary_reader
|
||||
case 0xCA: // float 32
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xCB: // float 64
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xCC: // uint 8
|
||||
{
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xCD: // uint 16
|
||||
{
|
||||
std::uint16_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xCE: // uint 32
|
||||
{
|
||||
std::uint32_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xCF: // uint 64
|
||||
{
|
||||
std::uint64_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xD0: // int 8
|
||||
{
|
||||
std::int8_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xD1: // int 16
|
||||
{
|
||||
std::int16_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xD2: // int 32
|
||||
{
|
||||
std::int32_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xD3: // int 64
|
||||
{
|
||||
std::int64_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xDC: // array 16
|
||||
@@ -2148,98 +2069,6 @@ class binary_reader
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a MessagePack object key
|
||||
|
||||
The MessagePack specification allows any type as a map key, but only
|
||||
strings have a counterpart in JSON. A key of any other type is rejected
|
||||
with a message naming that type, rather than the one @ref
|
||||
get_msgpack_string gives for a malformed string.
|
||||
|
||||
@param[out] result created key
|
||||
|
||||
@return whether key creation completed
|
||||
*/
|
||||
bool get_msgpack_object_key(string_t& result)
|
||||
{
|
||||
const char* found = nullptr;
|
||||
switch (current)
|
||||
{
|
||||
case 0xC0:
|
||||
found = "nil";
|
||||
break;
|
||||
case 0xC2:
|
||||
case 0xC3:
|
||||
found = "a boolean";
|
||||
break;
|
||||
case 0xCA:
|
||||
case 0xCB:
|
||||
found = "a float";
|
||||
break;
|
||||
case 0xC4:
|
||||
case 0xC5:
|
||||
case 0xC6:
|
||||
found = "a bin";
|
||||
break;
|
||||
case 0xC7:
|
||||
case 0xC8:
|
||||
case 0xC9:
|
||||
case 0xD4:
|
||||
case 0xD5:
|
||||
case 0xD6:
|
||||
case 0xD7:
|
||||
case 0xD8:
|
||||
found = "an ext";
|
||||
break;
|
||||
case 0xCC:
|
||||
case 0xCD:
|
||||
case 0xCE:
|
||||
case 0xCF:
|
||||
case 0xD0:
|
||||
case 0xD1:
|
||||
case 0xD2:
|
||||
case 0xD3:
|
||||
found = "an integer";
|
||||
break;
|
||||
case 0xDC:
|
||||
case 0xDD:
|
||||
found = "an array";
|
||||
break;
|
||||
case 0xDE:
|
||||
case 0xDF:
|
||||
found = "a map";
|
||||
break;
|
||||
default:
|
||||
// fixint, fixmap, and fixarray; strings, EOF, and the unused
|
||||
// byte 0xC1 are left to get_msgpack_string
|
||||
if (current == char_traits<char_type>::eof())
|
||||
{
|
||||
return get_msgpack_string(result);
|
||||
}
|
||||
if (current <= 0x7F || current >= 0xE0)
|
||||
{
|
||||
found = "an integer";
|
||||
}
|
||||
else if (current <= 0x8F)
|
||||
{
|
||||
found = "a map";
|
||||
}
|
||||
else if (current <= 0x9F)
|
||||
{
|
||||
found = "an array";
|
||||
}
|
||||
else
|
||||
{
|
||||
return get_msgpack_string(result);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
auto last_token = get_token_string();
|
||||
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
||||
exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a MessagePack byte array
|
||||
|
||||
@@ -2402,7 +2231,7 @@ class binary_reader
|
||||
{
|
||||
get();
|
||||
key.clear();
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -2927,7 +2756,7 @@ class binary_reader
|
||||
{
|
||||
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(input_format, i)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -3059,37 +2888,37 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'U':
|
||||
{
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'i':
|
||||
{
|
||||
std::int8_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'I':
|
||||
{
|
||||
std::int16_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'l':
|
||||
{
|
||||
std::int32_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'L':
|
||||
{
|
||||
std::int64_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'u':
|
||||
@@ -3099,7 +2928,7 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint16_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'm':
|
||||
@@ -3109,7 +2938,7 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint32_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'M':
|
||||
@@ -3119,7 +2948,7 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint64_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'h':
|
||||
@@ -3177,13 +3006,13 @@ class binary_reader
|
||||
case 'd':
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format, number) && emit_float(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 'D':
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format, number) && emit_float(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 'H':
|
||||
@@ -3650,13 +3479,13 @@ class binary_reader
|
||||
case 0x8E: // binary32
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
|
||||
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0x8F: // binary64
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
|
||||
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xF8:
|
||||
@@ -3722,9 +3551,7 @@ class binary_reader
|
||||
@brief pass an integer to the SAX parser
|
||||
|
||||
Non-negative integers are passed as unsigned, negative integers as signed
|
||||
numbers, like the other binary formats do. A value that does not fit the
|
||||
number type is passed as described for @ref emit_unsigned and
|
||||
@ref emit_signed.
|
||||
numbers, like the other binary formats do.
|
||||
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
@@ -3733,9 +3560,9 @@ class binary_reader
|
||||
{
|
||||
if (number >= 0)
|
||||
{
|
||||
return emit_unsigned(input_format_t::bon8, static_cast<std::uint64_t>(number));
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return emit_signed(input_format_t::bon8, number);
|
||||
return sax->number_integer(static_cast<number_integer_t>(number));
|
||||
}
|
||||
|
||||
/*!
|
||||
@@ -3792,7 +3619,8 @@ class binary_reader
|
||||
value = (value << 8) | static_cast<std::int64_t>(current);
|
||||
}
|
||||
|
||||
return emit_bon8_integer(negative ? -(value + offset) : value + offset);
|
||||
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
|
||||
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
|
||||
}
|
||||
|
||||
/*!
|
||||
@@ -4089,88 +3917,6 @@ class binary_reader
|
||||
return true;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass a signed integer read from the input to the SAX parser
|
||||
|
||||
Like the lexer does for JSON text, a value that does not fit into
|
||||
number_integer_t is passed as number_unsigned_t if it is non-negative and
|
||||
fits there, and as number_float_t otherwise. With the default number
|
||||
types, every integer the binary formats can encode fits, so this only
|
||||
matters for narrower custom number types.
|
||||
|
||||
@tparam NumberType a signed integer type
|
||||
@param[in] format the current format (for diagnostics)
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
|
||||
@throw out_of_range.406 if @a number overflows number_float_t (see
|
||||
@ref emit_float)
|
||||
*/
|
||||
template<typename NumberType>
|
||||
bool emit_signed(const input_format_t format, const NumberType number)
|
||||
{
|
||||
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
|
||||
{
|
||||
return sax->number_integer(static_cast<number_integer_t>(number));
|
||||
}
|
||||
if (value_in_range_of<number_unsigned_t>(number))
|
||||
{
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return emit_float(format, number);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass an unsigned integer read from the input to the SAX parser
|
||||
|
||||
Like the lexer does for JSON text, a value that does not fit into
|
||||
number_unsigned_t is passed as number_float_t.
|
||||
|
||||
@tparam NumberType an unsigned integer type
|
||||
@param[in] format the current format (for diagnostics)
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
|
||||
@throw out_of_range.406 if @a number overflows number_float_t (see
|
||||
@ref emit_float)
|
||||
*/
|
||||
template<typename NumberType>
|
||||
bool emit_unsigned(const input_format_t format, const NumberType number)
|
||||
{
|
||||
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
|
||||
{
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return emit_float(format, number);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass a floating-point number read from the input to the SAX parser
|
||||
|
||||
Like the lexer does for JSON text, a finite value that overflows
|
||||
number_float_t is rejected instead of silently becoming infinity. Infinity
|
||||
and NaN in the input are passed on unchanged. Integers only overflow if
|
||||
number_float_t cannot represent 2^64, e.g., a half-precision type.
|
||||
|
||||
@tparam NumberType a floating-point or integer type
|
||||
@param[in] format the current format (for diagnostics)
|
||||
@param[in] number the number
|
||||
@return whether the SAX parser accepted the value
|
||||
|
||||
@throw out_of_range.406 if a finite @a number overflows number_float_t
|
||||
*/
|
||||
template<typename NumberType>
|
||||
bool emit_float(const input_format_t format, const NumberType number)
|
||||
{
|
||||
const auto result = static_cast<number_float_t>(number);
|
||||
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
|
||||
{
|
||||
return sax->parse_error(chars_read, get_token_string(),
|
||||
out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
|
||||
}
|
||||
return sax->number_float(result, "");
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief create a string by reading characters from the input
|
||||
|
||||
|
||||
@@ -206,6 +206,7 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
|
||||
: ia(std::move(adapter))
|
||||
, ignore_comments(ignore_comments_)
|
||||
, decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
|
||||
, discard_number_values(discard_number_values_)
|
||||
{}
|
||||
|
||||
@@ -221,7 +222,8 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
// locales
|
||||
/////////////////////
|
||||
|
||||
/// return the decimal point of the current locale
|
||||
/// return the locale-dependent decimal point
|
||||
JSON_HEDLEY_PURE
|
||||
static char get_decimal_point() noexcept
|
||||
{
|
||||
const auto* loc = localeconv();
|
||||
@@ -1090,10 +1092,9 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
token_type::value_float if number could be successfully scanned,
|
||||
token_type::parse_error otherwise
|
||||
|
||||
@note The scanner is independent of the current locale: token_buffer
|
||||
always holds `.`. Only the std::strtod fallback of convert_number()
|
||||
depends on the locale, and it looks up the decimal point right
|
||||
before converting (see convert_float_locale_aware()).
|
||||
@note The scanner is independent of the current locale. Internally, the
|
||||
locale's decimal point is used instead of `.` to work with the
|
||||
locale-dependent converters.
|
||||
*/
|
||||
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
|
||||
{
|
||||
@@ -1182,7 +1183,7 @@ scan_number_zero:
|
||||
{
|
||||
case '.':
|
||||
{
|
||||
add(current);
|
||||
add(decimal_point_char);
|
||||
decimal_point_position = token_buffer.size() - 1;
|
||||
goto scan_number_decimal1;
|
||||
}
|
||||
@@ -1219,7 +1220,7 @@ scan_number_any1:
|
||||
|
||||
case '.':
|
||||
{
|
||||
add(current);
|
||||
add(decimal_point_char);
|
||||
decimal_point_position = token_buffer.size() - 1;
|
||||
goto scan_number_decimal1;
|
||||
}
|
||||
@@ -1461,9 +1462,9 @@ scan_number_done:
|
||||
|
||||
// Only a number below 1 can carry further insignificant zeros, and only
|
||||
// while the count stays at the limit does removing them change the
|
||||
// answer - so this loop is skipped for all but a few tokens. The
|
||||
// fraction is located through decimal_point_position rather than by
|
||||
// searching '.'.
|
||||
// answer - so this loop is skipped for all but a few tokens. Note
|
||||
// token_buffer holds the locale's decimal point, so the fraction is
|
||||
// located through decimal_point_position rather than by searching '.'.
|
||||
if (lead_zero != 0)
|
||||
{
|
||||
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
|
||||
@@ -1481,8 +1482,8 @@ scan_number_done:
|
||||
@brief convert the number text in token_buffer to its value and token type
|
||||
|
||||
The digit sequence in token_buffer has already been validated (by the
|
||||
scan_number() state machine or by the contiguous fast path) and holds '.'
|
||||
as decimal point, independent of the locale. Integers are parsed first and fall
|
||||
scan_number() state machine or by the contiguous fast path) and holds the
|
||||
locale decimal point in place of '.'. Integers are parsed first and fall
|
||||
back to floating point on overflow. This is shared so both scanners produce
|
||||
identical results.
|
||||
|
||||
@@ -1562,7 +1563,7 @@ scan_number_done:
|
||||
// integer conversion above overflowed. Prefer std::from_chars
|
||||
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
|
||||
// otherwise the exact Clinger fast path (double only); otherwise the
|
||||
// locale-aware strtof/strtod/strtold.
|
||||
// locale-aware strtof/strtod.
|
||||
if (parse_float_from_chars(num_begin, num_end, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
@@ -1571,75 +1572,26 @@ scan_number_done:
|
||||
// extra pass over the token's bytes, which otherwise shows up on
|
||||
// high-precision inputs such as canada.json
|
||||
if (mantissa_fits_clinger(mantissa_end)
|
||||
&& parse_float_fast(num_begin, num_end, value_float))
|
||||
&& parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
convert_float_locale_aware();
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof(value_float, token_buffer.data(), &endptr);
|
||||
|
||||
// we checked the number format before
|
||||
JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
|
||||
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert the float in token_buffer with strtof/strtod/strtold
|
||||
|
||||
These functions expect the decimal point of the *current* locale, so it is
|
||||
looked up right before the conversion instead of once when the lexer is
|
||||
constructed: a locale change in between (by a parser callback, a SAX
|
||||
handler, or another thread) must not truncate the value (#5198). The
|
||||
token has been validated before, so if the conversion stops early and the
|
||||
decimal point changed in the meantime, the locale changed between the
|
||||
lookup and the call, and the conversion is repeated with the new decimal
|
||||
point. If the decimal point did not change, a retry cannot succeed: the
|
||||
locale's decimal point is not a single character (e.g., the two-byte
|
||||
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
|
||||
The value strtod parsed up to that point is kept, as before this change.
|
||||
|
||||
Note that changing the locale in another thread *while* strtod runs is
|
||||
undefined behavior of the C library, which this function cannot prevent.
|
||||
*/
|
||||
void convert_float_locale_aware()
|
||||
{
|
||||
const bool has_dot = decimal_point_position != std::string::npos;
|
||||
char decimal_point = get_decimal_point();
|
||||
for (;;)
|
||||
{
|
||||
const bool substitute = has_dot && decimal_point != '.';
|
||||
if (substitute)
|
||||
{
|
||||
token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
|
||||
}
|
||||
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof(value_float, token_buffer.data(), &endptr);
|
||||
|
||||
if (substitute)
|
||||
{
|
||||
// get_string() hands the token to the SAX interface with '.'
|
||||
token_buffer[decimal_point_position] = '.';
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// retry only if the locale changed; otherwise, this would loop forever
|
||||
const char current_decimal_point = get_decimal_point();
|
||||
if (current_decimal_point == decimal_point)
|
||||
{
|
||||
return;
|
||||
}
|
||||
decimal_point = current_decimal_point;
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief contiguous fast path for scanning a number
|
||||
|
||||
Parses the whole number token straight from the input buffer, avoiding the
|
||||
per-character get()/add() of scan_number(). On success it fills token_buffer
|
||||
(as scan_number() does) and
|
||||
(with the locale decimal point substituted, as scan_number() does) and
|
||||
returns the token type. On anything it does not fully recognize as a
|
||||
well-formed number it makes no state change and returns
|
||||
token_type::uninitialized, so the caller falls back to scan_number(), which
|
||||
@@ -1755,11 +1707,16 @@ scan_number_done:
|
||||
}
|
||||
#endif
|
||||
|
||||
// materialize the token exactly as scan_number() would. reset() already
|
||||
// cleared token_buffer, so append() fills it (assign() is avoided
|
||||
// because custom string_t types need not provide it)
|
||||
// materialize the token exactly as scan_number() would, substituting the
|
||||
// locale decimal point so convert_number()'s strtof fallback stays valid.
|
||||
// reset() already cleared token_buffer, so append() fills it (assign() is
|
||||
// avoided because custom string_t types need not provide it)
|
||||
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
|
||||
decimal_point_position = dot_index;
|
||||
if (dot_index != std::string::npos)
|
||||
{
|
||||
token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
|
||||
decimal_point_position = dot_index;
|
||||
}
|
||||
|
||||
ia.bulk_skip(len - 1);
|
||||
position.chars_read_total += (len - 1);
|
||||
@@ -2026,7 +1983,11 @@ scan_number_done:
|
||||
/// return current string value (implicitly resets the token; useful only once)
|
||||
string_t& get_string()
|
||||
{
|
||||
// a number token holds '.' regardless of the locale (#4084)
|
||||
// translate decimal points from locale back to '.' (#4084)
|
||||
if (decimal_point_char != '.' && decimal_point_position != std::string::npos)
|
||||
{
|
||||
token_buffer[decimal_point_position] = '.';
|
||||
}
|
||||
return token_buffer;
|
||||
}
|
||||
|
||||
@@ -2322,7 +2283,9 @@ scan_number_done:
|
||||
number_unsigned_t value_unsigned = 0;
|
||||
number_float_t value_float = 0;
|
||||
|
||||
/// the position of the decimal point in token_buffer
|
||||
/// the decimal point
|
||||
const char_int_type decimal_point_char = '.';
|
||||
/// the position of the decimal point in the input
|
||||
std::size_t decimal_point_position = std::string::npos;
|
||||
|
||||
/// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
|
||||
|
||||
@@ -118,12 +118,14 @@ std::strtod. The parser only activates for number_float_t == double; float and
|
||||
long double keep the std::strtof/std::strtold paths (see the templated overload
|
||||
below).
|
||||
|
||||
@param[in] first pointer to the first character of the number
|
||||
@param[in] last pointer past the last character
|
||||
@param[out] out the parsed value on success
|
||||
@param[in] first pointer to the first character of the number
|
||||
@param[in] last pointer past the last character
|
||||
@param[in] decimal_point the (locale-dependent) decimal point character
|
||||
@param[out] out the parsed value on success
|
||||
@return true if the value was parsed exactly; false to fall back to strtod
|
||||
*/
|
||||
inline bool parse_float_fast(const char* first, const char* last, double& out) noexcept
|
||||
template<typename DecimalPointType>
|
||||
bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
|
||||
{
|
||||
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
|
||||
// Clinger's fast path is only exact when double operations are evaluated in
|
||||
@@ -134,6 +136,7 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
|
||||
// std::from_chars / std::strtod path.
|
||||
static_cast<void>(first);
|
||||
static_cast<void>(last);
|
||||
static_cast<void>(decimal_point);
|
||||
static_cast<void>(out);
|
||||
return false;
|
||||
#else
|
||||
@@ -172,7 +175,7 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
|
||||
++num_digits;
|
||||
fractional_digits += static_cast<int>(seen_dot);
|
||||
}
|
||||
else if (c == '.')
|
||||
else if (static_cast<DecimalPointType>(c) == decimal_point)
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(seen_dot))
|
||||
{
|
||||
@@ -257,8 +260,8 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
|
||||
}
|
||||
|
||||
/// fast float path is only exact for `double`; decline for float/long double
|
||||
template<typename FloatType>
|
||||
bool parse_float_fast(const char* /*first*/, const char* /*last*/, FloatType& /*out*/) noexcept
|
||||
template<typename DecimalPointType, typename FloatType>
|
||||
bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -270,7 +273,9 @@ std::from_chars is locale-independent, correctly rounded, and - via the
|
||||
Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
|
||||
over the whole value range (not just the Clinger subset). It is used only when
|
||||
__cpp_lib_to_chars indicates full floating-point support and only when it
|
||||
consumes the entire token ([first, last)). An under-/overflow (result_out_of_range) also declines, so
|
||||
consumes the entire token ([first, last)); a partial parse means the buffer
|
||||
uses a non-'.' locale decimal point, in which case the caller falls back to the
|
||||
locale-aware path. An under-/overflow (result_out_of_range) also declines, so
|
||||
the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
|
||||
expects (side-stepping the P4168 divergence between implementations).
|
||||
|
||||
|
||||
@@ -535,6 +535,10 @@ class json_pointer
|
||||
@return const reference to the JSON value pointed to by the JSON
|
||||
pointer
|
||||
|
||||
@pre Every object key and array index the pointer refers to exists.
|
||||
Like the const operator[] for keys and indices, a missing one is
|
||||
undefined behavior, guarded by a runtime assertion.
|
||||
|
||||
@throw parse_error.106 if an array index begins with '0'
|
||||
@throw parse_error.109 if an array index was not a number
|
||||
@throw out_of_range.402 if the array index '-' is used
|
||||
@@ -549,7 +553,8 @@ class json_pointer
|
||||
{
|
||||
case detail::value_t::object:
|
||||
{
|
||||
// use unchecked object access
|
||||
// use unchecked object access; the const operator[]
|
||||
// asserts that the key exists
|
||||
ptr = &ptr->operator[](reference_token);
|
||||
break;
|
||||
}
|
||||
@@ -562,7 +567,8 @@ class json_pointer
|
||||
JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
|
||||
}
|
||||
|
||||
// use unchecked array access
|
||||
// use unchecked array access; the const operator[]
|
||||
// asserts that the index exists
|
||||
ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -2866,6 +2866,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
|
||||
// const operator[] only works for arrays
|
||||
if (JSON_HEDLEY_LIKELY(is_array()))
|
||||
{
|
||||
JSON_ASSERT(idx < m_data.m_value.array->size());
|
||||
return m_data.m_value.array->operator[](idx);
|
||||
}
|
||||
|
||||
|
||||
+108
-387
@@ -8605,12 +8605,14 @@ std::strtod. The parser only activates for number_float_t == double; float and
|
||||
long double keep the std::strtof/std::strtold paths (see the templated overload
|
||||
below).
|
||||
|
||||
@param[in] first pointer to the first character of the number
|
||||
@param[in] last pointer past the last character
|
||||
@param[out] out the parsed value on success
|
||||
@param[in] first pointer to the first character of the number
|
||||
@param[in] last pointer past the last character
|
||||
@param[in] decimal_point the (locale-dependent) decimal point character
|
||||
@param[out] out the parsed value on success
|
||||
@return true if the value was parsed exactly; false to fall back to strtod
|
||||
*/
|
||||
inline bool parse_float_fast(const char* first, const char* last, double& out) noexcept
|
||||
template<typename DecimalPointType>
|
||||
bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
|
||||
{
|
||||
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
|
||||
// Clinger's fast path is only exact when double operations are evaluated in
|
||||
@@ -8621,6 +8623,7 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
|
||||
// std::from_chars / std::strtod path.
|
||||
static_cast<void>(first);
|
||||
static_cast<void>(last);
|
||||
static_cast<void>(decimal_point);
|
||||
static_cast<void>(out);
|
||||
return false;
|
||||
#else
|
||||
@@ -8659,7 +8662,7 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
|
||||
++num_digits;
|
||||
fractional_digits += static_cast<int>(seen_dot);
|
||||
}
|
||||
else if (c == '.')
|
||||
else if (static_cast<DecimalPointType>(c) == decimal_point)
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(seen_dot))
|
||||
{
|
||||
@@ -8744,8 +8747,8 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
|
||||
}
|
||||
|
||||
/// fast float path is only exact for `double`; decline for float/long double
|
||||
template<typename FloatType>
|
||||
bool parse_float_fast(const char* /*first*/, const char* /*last*/, FloatType& /*out*/) noexcept
|
||||
template<typename DecimalPointType, typename FloatType>
|
||||
bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -8757,7 +8760,9 @@ std::from_chars is locale-independent, correctly rounded, and - via the
|
||||
Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
|
||||
over the whole value range (not just the Clinger subset). It is used only when
|
||||
__cpp_lib_to_chars indicates full floating-point support and only when it
|
||||
consumes the entire token ([first, last)). An under-/overflow (result_out_of_range) also declines, so
|
||||
consumes the entire token ([first, last)); a partial parse means the buffer
|
||||
uses a non-'.' locale decimal point, in which case the caller falls back to the
|
||||
locale-aware path. An under-/overflow (result_out_of_range) also declines, so
|
||||
the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
|
||||
expects (side-stepping the P4168 divergence between implementations).
|
||||
|
||||
@@ -9298,6 +9303,7 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
|
||||
: ia(std::move(adapter))
|
||||
, ignore_comments(ignore_comments_)
|
||||
, decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
|
||||
, discard_number_values(discard_number_values_)
|
||||
{}
|
||||
|
||||
@@ -9313,7 +9319,8 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
// locales
|
||||
/////////////////////
|
||||
|
||||
/// return the decimal point of the current locale
|
||||
/// return the locale-dependent decimal point
|
||||
JSON_HEDLEY_PURE
|
||||
static char get_decimal_point() noexcept
|
||||
{
|
||||
const auto* loc = localeconv();
|
||||
@@ -10182,10 +10189,9 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
token_type::value_float if number could be successfully scanned,
|
||||
token_type::parse_error otherwise
|
||||
|
||||
@note The scanner is independent of the current locale: token_buffer
|
||||
always holds `.`. Only the std::strtod fallback of convert_number()
|
||||
depends on the locale, and it looks up the decimal point right
|
||||
before converting (see convert_float_locale_aware()).
|
||||
@note The scanner is independent of the current locale. Internally, the
|
||||
locale's decimal point is used instead of `.` to work with the
|
||||
locale-dependent converters.
|
||||
*/
|
||||
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
|
||||
{
|
||||
@@ -10274,7 +10280,7 @@ scan_number_zero:
|
||||
{
|
||||
case '.':
|
||||
{
|
||||
add(current);
|
||||
add(decimal_point_char);
|
||||
decimal_point_position = token_buffer.size() - 1;
|
||||
goto scan_number_decimal1;
|
||||
}
|
||||
@@ -10311,7 +10317,7 @@ scan_number_any1:
|
||||
|
||||
case '.':
|
||||
{
|
||||
add(current);
|
||||
add(decimal_point_char);
|
||||
decimal_point_position = token_buffer.size() - 1;
|
||||
goto scan_number_decimal1;
|
||||
}
|
||||
@@ -10553,9 +10559,9 @@ scan_number_done:
|
||||
|
||||
// Only a number below 1 can carry further insignificant zeros, and only
|
||||
// while the count stays at the limit does removing them change the
|
||||
// answer - so this loop is skipped for all but a few tokens. The
|
||||
// fraction is located through decimal_point_position rather than by
|
||||
// searching '.'.
|
||||
// answer - so this loop is skipped for all but a few tokens. Note
|
||||
// token_buffer holds the locale's decimal point, so the fraction is
|
||||
// located through decimal_point_position rather than by searching '.'.
|
||||
if (lead_zero != 0)
|
||||
{
|
||||
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
|
||||
@@ -10573,8 +10579,8 @@ scan_number_done:
|
||||
@brief convert the number text in token_buffer to its value and token type
|
||||
|
||||
The digit sequence in token_buffer has already been validated (by the
|
||||
scan_number() state machine or by the contiguous fast path) and holds '.'
|
||||
as decimal point, independent of the locale. Integers are parsed first and fall
|
||||
scan_number() state machine or by the contiguous fast path) and holds the
|
||||
locale decimal point in place of '.'. Integers are parsed first and fall
|
||||
back to floating point on overflow. This is shared so both scanners produce
|
||||
identical results.
|
||||
|
||||
@@ -10654,7 +10660,7 @@ scan_number_done:
|
||||
// integer conversion above overflowed. Prefer std::from_chars
|
||||
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
|
||||
// otherwise the exact Clinger fast path (double only); otherwise the
|
||||
// locale-aware strtof/strtod/strtold.
|
||||
// locale-aware strtof/strtod.
|
||||
if (parse_float_from_chars(num_begin, num_end, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
@@ -10663,75 +10669,26 @@ scan_number_done:
|
||||
// extra pass over the token's bytes, which otherwise shows up on
|
||||
// high-precision inputs such as canada.json
|
||||
if (mantissa_fits_clinger(mantissa_end)
|
||||
&& parse_float_fast(num_begin, num_end, value_float))
|
||||
&& parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
convert_float_locale_aware();
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof(value_float, token_buffer.data(), &endptr);
|
||||
|
||||
// we checked the number format before
|
||||
JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
|
||||
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert the float in token_buffer with strtof/strtod/strtold
|
||||
|
||||
These functions expect the decimal point of the *current* locale, so it is
|
||||
looked up right before the conversion instead of once when the lexer is
|
||||
constructed: a locale change in between (by a parser callback, a SAX
|
||||
handler, or another thread) must not truncate the value (#5198). The
|
||||
token has been validated before, so if the conversion stops early and the
|
||||
decimal point changed in the meantime, the locale changed between the
|
||||
lookup and the call, and the conversion is repeated with the new decimal
|
||||
point. If the decimal point did not change, a retry cannot succeed: the
|
||||
locale's decimal point is not a single character (e.g., the two-byte
|
||||
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
|
||||
The value strtod parsed up to that point is kept, as before this change.
|
||||
|
||||
Note that changing the locale in another thread *while* strtod runs is
|
||||
undefined behavior of the C library, which this function cannot prevent.
|
||||
*/
|
||||
void convert_float_locale_aware()
|
||||
{
|
||||
const bool has_dot = decimal_point_position != std::string::npos;
|
||||
char decimal_point = get_decimal_point();
|
||||
for (;;)
|
||||
{
|
||||
const bool substitute = has_dot && decimal_point != '.';
|
||||
if (substitute)
|
||||
{
|
||||
token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
|
||||
}
|
||||
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof(value_float, token_buffer.data(), &endptr);
|
||||
|
||||
if (substitute)
|
||||
{
|
||||
// get_string() hands the token to the SAX interface with '.'
|
||||
token_buffer[decimal_point_position] = '.';
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// retry only if the locale changed; otherwise, this would loop forever
|
||||
const char current_decimal_point = get_decimal_point();
|
||||
if (current_decimal_point == decimal_point)
|
||||
{
|
||||
return;
|
||||
}
|
||||
decimal_point = current_decimal_point;
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief contiguous fast path for scanning a number
|
||||
|
||||
Parses the whole number token straight from the input buffer, avoiding the
|
||||
per-character get()/add() of scan_number(). On success it fills token_buffer
|
||||
(as scan_number() does) and
|
||||
(with the locale decimal point substituted, as scan_number() does) and
|
||||
returns the token type. On anything it does not fully recognize as a
|
||||
well-formed number it makes no state change and returns
|
||||
token_type::uninitialized, so the caller falls back to scan_number(), which
|
||||
@@ -10847,11 +10804,16 @@ scan_number_done:
|
||||
}
|
||||
#endif
|
||||
|
||||
// materialize the token exactly as scan_number() would. reset() already
|
||||
// cleared token_buffer, so append() fills it (assign() is avoided
|
||||
// because custom string_t types need not provide it)
|
||||
// materialize the token exactly as scan_number() would, substituting the
|
||||
// locale decimal point so convert_number()'s strtof fallback stays valid.
|
||||
// reset() already cleared token_buffer, so append() fills it (assign() is
|
||||
// avoided because custom string_t types need not provide it)
|
||||
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
|
||||
decimal_point_position = dot_index;
|
||||
if (dot_index != std::string::npos)
|
||||
{
|
||||
token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
|
||||
decimal_point_position = dot_index;
|
||||
}
|
||||
|
||||
ia.bulk_skip(len - 1);
|
||||
position.chars_read_total += (len - 1);
|
||||
@@ -11118,7 +11080,11 @@ scan_number_done:
|
||||
/// return current string value (implicitly resets the token; useful only once)
|
||||
string_t& get_string()
|
||||
{
|
||||
// a number token holds '.' regardless of the locale (#4084)
|
||||
// translate decimal points from locale back to '.' (#4084)
|
||||
if (decimal_point_char != '.' && decimal_point_position != std::string::npos)
|
||||
{
|
||||
token_buffer[decimal_point_position] = '.';
|
||||
}
|
||||
return token_buffer;
|
||||
}
|
||||
|
||||
@@ -11414,7 +11380,9 @@ scan_number_done:
|
||||
number_unsigned_t value_unsigned = 0;
|
||||
number_float_t value_float = 0;
|
||||
|
||||
/// the position of the decimal point in token_buffer
|
||||
/// the decimal point
|
||||
const char_int_type decimal_point_char = '.';
|
||||
/// the position of the decimal point in the input
|
||||
std::size_t decimal_point_position = std::string::npos;
|
||||
|
||||
/// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
|
||||
@@ -13326,7 +13294,7 @@ class binary_reader
|
||||
case 0x01: // double
|
||||
{
|
||||
double number{};
|
||||
return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
|
||||
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0x02: // string
|
||||
@@ -13367,19 +13335,19 @@ class binary_reader
|
||||
case 0x10: // int32
|
||||
{
|
||||
std::int32_t value{};
|
||||
return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
|
||||
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
|
||||
}
|
||||
|
||||
case 0x12: // int64
|
||||
{
|
||||
std::int64_t value{};
|
||||
return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
|
||||
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
|
||||
}
|
||||
|
||||
case 0x11: // uint64
|
||||
{
|
||||
std::uint64_t value{};
|
||||
return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(input_format_t::bson, value);
|
||||
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
|
||||
}
|
||||
|
||||
default: // anything else is not supported (yet)
|
||||
@@ -13405,19 +13373,14 @@ class binary_reader
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// the value is -1 - number, which fits into number_integer_t
|
||||
// whenever number does
|
||||
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
|
||||
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
|
||||
if (number > max_val)
|
||||
{
|
||||
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
|
||||
return sax->parse_error(chars_read, get_token_string(),
|
||||
parse_error::create(112, chars_read,
|
||||
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
|
||||
}
|
||||
|
||||
// like the lexer does for JSON text, store a value too small for
|
||||
// number_integer_t as number_float_t; compute it as long double so
|
||||
// that emit_float sees a finite value and can detect an overflow of
|
||||
// number_float_t
|
||||
return emit_float(input_format_t::cbor, static_cast<long double>(-1) - static_cast<long double>(number));
|
||||
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
|
||||
}
|
||||
|
||||
/*!
|
||||
@@ -13474,25 +13437,25 @@ class binary_reader
|
||||
case 0x18: // Unsigned integer (one-byte uint8_t follows)
|
||||
{
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0x19: // Unsigned integer (two-byte uint16_t follows)
|
||||
{
|
||||
std::uint16_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
|
||||
{
|
||||
std::uint32_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
|
||||
{
|
||||
std::uint64_t number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
// Negative integer -1-0x00..-1-0x17 (-1..-24)
|
||||
@@ -13937,13 +13900,13 @@ class binary_reader
|
||||
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
|
||||
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
default: // anything else (0xFF is handled inside the other types)
|
||||
@@ -14096,80 +14059,6 @@ class binary_reader
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a CBOR object key
|
||||
|
||||
RFC 8949 allows any data item as a map key, but only strings have a
|
||||
counterpart in JSON. A key of any other type is rejected with a message
|
||||
naming that type, rather than the one @ref get_cbor_string gives for a
|
||||
malformed string.
|
||||
|
||||
@param[out] result created key
|
||||
|
||||
@return whether key creation completed
|
||||
*/
|
||||
bool get_cbor_object_key(string_t& result)
|
||||
{
|
||||
// EOF and major type 3 (text string) are left to get_cbor_string
|
||||
if (current == char_traits<char_type>::eof() || (static_cast<unsigned int>(current) & 0xE0u) == 0x60u)
|
||||
{
|
||||
return get_cbor_string(result);
|
||||
}
|
||||
|
||||
const char* found = nullptr;
|
||||
switch (static_cast<unsigned int>(current) >> 5u)
|
||||
{
|
||||
case 0:
|
||||
found = "an unsigned integer";
|
||||
break;
|
||||
case 1:
|
||||
found = "a negative integer";
|
||||
break;
|
||||
case 2:
|
||||
found = "a byte string";
|
||||
break;
|
||||
case 4:
|
||||
found = "an array";
|
||||
break;
|
||||
case 5:
|
||||
found = "a map";
|
||||
break;
|
||||
case 6:
|
||||
found = "a tag";
|
||||
break;
|
||||
default: // major type 7
|
||||
switch (current)
|
||||
{
|
||||
case 0xF4:
|
||||
case 0xF5:
|
||||
found = "a boolean";
|
||||
break;
|
||||
case 0xF6:
|
||||
found = "null";
|
||||
break;
|
||||
case 0xF7:
|
||||
found = "undefined";
|
||||
break;
|
||||
case 0xF9:
|
||||
case 0xFA:
|
||||
case 0xFB:
|
||||
found = "a floating-point number";
|
||||
break;
|
||||
case 0xFF:
|
||||
found = "a break stop code";
|
||||
break;
|
||||
default:
|
||||
found = "a simple value";
|
||||
break;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
auto last_token = get_token_string();
|
||||
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
||||
exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a definite-length CBOR byte array
|
||||
|
||||
@@ -14414,7 +14303,7 @@ class binary_reader
|
||||
if (top.is_object)
|
||||
{
|
||||
key.clear();
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -14707,61 +14596,61 @@ class binary_reader
|
||||
case 0xCA: // float 32
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xCB: // float 64
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xCC: // uint 8
|
||||
{
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xCD: // uint 16
|
||||
{
|
||||
std::uint16_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xCE: // uint 32
|
||||
{
|
||||
std::uint32_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xCF: // uint 64
|
||||
{
|
||||
std::uint64_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 0xD0: // int 8
|
||||
{
|
||||
std::int8_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xD1: // int 16
|
||||
{
|
||||
std::int16_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xD2: // int 32
|
||||
{
|
||||
std::int32_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xD3: // int 64
|
||||
{
|
||||
std::int64_t number{};
|
||||
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
|
||||
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 0xDC: // array 16
|
||||
@@ -14915,98 +14804,6 @@ class binary_reader
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a MessagePack object key
|
||||
|
||||
The MessagePack specification allows any type as a map key, but only
|
||||
strings have a counterpart in JSON. A key of any other type is rejected
|
||||
with a message naming that type, rather than the one @ref
|
||||
get_msgpack_string gives for a malformed string.
|
||||
|
||||
@param[out] result created key
|
||||
|
||||
@return whether key creation completed
|
||||
*/
|
||||
bool get_msgpack_object_key(string_t& result)
|
||||
{
|
||||
const char* found = nullptr;
|
||||
switch (current)
|
||||
{
|
||||
case 0xC0:
|
||||
found = "nil";
|
||||
break;
|
||||
case 0xC2:
|
||||
case 0xC3:
|
||||
found = "a boolean";
|
||||
break;
|
||||
case 0xCA:
|
||||
case 0xCB:
|
||||
found = "a float";
|
||||
break;
|
||||
case 0xC4:
|
||||
case 0xC5:
|
||||
case 0xC6:
|
||||
found = "a bin";
|
||||
break;
|
||||
case 0xC7:
|
||||
case 0xC8:
|
||||
case 0xC9:
|
||||
case 0xD4:
|
||||
case 0xD5:
|
||||
case 0xD6:
|
||||
case 0xD7:
|
||||
case 0xD8:
|
||||
found = "an ext";
|
||||
break;
|
||||
case 0xCC:
|
||||
case 0xCD:
|
||||
case 0xCE:
|
||||
case 0xCF:
|
||||
case 0xD0:
|
||||
case 0xD1:
|
||||
case 0xD2:
|
||||
case 0xD3:
|
||||
found = "an integer";
|
||||
break;
|
||||
case 0xDC:
|
||||
case 0xDD:
|
||||
found = "an array";
|
||||
break;
|
||||
case 0xDE:
|
||||
case 0xDF:
|
||||
found = "a map";
|
||||
break;
|
||||
default:
|
||||
// fixint, fixmap, and fixarray; strings, EOF, and the unused
|
||||
// byte 0xC1 are left to get_msgpack_string
|
||||
if (current == char_traits<char_type>::eof())
|
||||
{
|
||||
return get_msgpack_string(result);
|
||||
}
|
||||
if (current <= 0x7F || current >= 0xE0)
|
||||
{
|
||||
found = "an integer";
|
||||
}
|
||||
else if (current <= 0x8F)
|
||||
{
|
||||
found = "a map";
|
||||
}
|
||||
else if (current <= 0x9F)
|
||||
{
|
||||
found = "an array";
|
||||
}
|
||||
else
|
||||
{
|
||||
return get_msgpack_string(result);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
auto last_token = get_token_string();
|
||||
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
||||
exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief reads a MessagePack byte array
|
||||
|
||||
@@ -15169,7 +14966,7 @@ class binary_reader
|
||||
{
|
||||
get();
|
||||
key.clear();
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -15694,7 +15491,7 @@ class binary_reader
|
||||
{
|
||||
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(input_format, i)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -15826,37 +15623,37 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'U':
|
||||
{
|
||||
std::uint8_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'i':
|
||||
{
|
||||
std::int8_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'I':
|
||||
{
|
||||
std::int16_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'l':
|
||||
{
|
||||
std::int32_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'L':
|
||||
{
|
||||
std::int64_t number{};
|
||||
return get_number(input_format, number) && emit_signed(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_integer(number);
|
||||
}
|
||||
|
||||
case 'u':
|
||||
@@ -15866,7 +15663,7 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint16_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'm':
|
||||
@@ -15876,7 +15673,7 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint32_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'M':
|
||||
@@ -15886,7 +15683,7 @@ class binary_reader
|
||||
break;
|
||||
}
|
||||
std::uint64_t number{};
|
||||
return get_number(input_format, number) && emit_unsigned(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_unsigned(number);
|
||||
}
|
||||
|
||||
case 'h':
|
||||
@@ -15944,13 +15741,13 @@ class binary_reader
|
||||
case 'd':
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format, number) && emit_float(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 'D':
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format, number) && emit_float(input_format, number);
|
||||
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 'H':
|
||||
@@ -16417,13 +16214,13 @@ class binary_reader
|
||||
case 0x8E: // binary32
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
|
||||
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0x8F: // binary64
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
|
||||
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xF8:
|
||||
@@ -16489,9 +16286,7 @@ class binary_reader
|
||||
@brief pass an integer to the SAX parser
|
||||
|
||||
Non-negative integers are passed as unsigned, negative integers as signed
|
||||
numbers, like the other binary formats do. A value that does not fit the
|
||||
number type is passed as described for @ref emit_unsigned and
|
||||
@ref emit_signed.
|
||||
numbers, like the other binary formats do.
|
||||
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
@@ -16500,9 +16295,9 @@ class binary_reader
|
||||
{
|
||||
if (number >= 0)
|
||||
{
|
||||
return emit_unsigned(input_format_t::bon8, static_cast<std::uint64_t>(number));
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return emit_signed(input_format_t::bon8, number);
|
||||
return sax->number_integer(static_cast<number_integer_t>(number));
|
||||
}
|
||||
|
||||
/*!
|
||||
@@ -16559,7 +16354,8 @@ class binary_reader
|
||||
value = (value << 8) | static_cast<std::int64_t>(current);
|
||||
}
|
||||
|
||||
return emit_bon8_integer(negative ? -(value + offset) : value + offset);
|
||||
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
|
||||
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
|
||||
}
|
||||
|
||||
/*!
|
||||
@@ -16856,88 +16652,6 @@ class binary_reader
|
||||
return true;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass a signed integer read from the input to the SAX parser
|
||||
|
||||
Like the lexer does for JSON text, a value that does not fit into
|
||||
number_integer_t is passed as number_unsigned_t if it is non-negative and
|
||||
fits there, and as number_float_t otherwise. With the default number
|
||||
types, every integer the binary formats can encode fits, so this only
|
||||
matters for narrower custom number types.
|
||||
|
||||
@tparam NumberType a signed integer type
|
||||
@param[in] format the current format (for diagnostics)
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
|
||||
@throw out_of_range.406 if @a number overflows number_float_t (see
|
||||
@ref emit_float)
|
||||
*/
|
||||
template<typename NumberType>
|
||||
bool emit_signed(const input_format_t format, const NumberType number)
|
||||
{
|
||||
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
|
||||
{
|
||||
return sax->number_integer(static_cast<number_integer_t>(number));
|
||||
}
|
||||
if (value_in_range_of<number_unsigned_t>(number))
|
||||
{
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return emit_float(format, number);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass an unsigned integer read from the input to the SAX parser
|
||||
|
||||
Like the lexer does for JSON text, a value that does not fit into
|
||||
number_unsigned_t is passed as number_float_t.
|
||||
|
||||
@tparam NumberType an unsigned integer type
|
||||
@param[in] format the current format (for diagnostics)
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
|
||||
@throw out_of_range.406 if @a number overflows number_float_t (see
|
||||
@ref emit_float)
|
||||
*/
|
||||
template<typename NumberType>
|
||||
bool emit_unsigned(const input_format_t format, const NumberType number)
|
||||
{
|
||||
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
|
||||
{
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return emit_float(format, number);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass a floating-point number read from the input to the SAX parser
|
||||
|
||||
Like the lexer does for JSON text, a finite value that overflows
|
||||
number_float_t is rejected instead of silently becoming infinity. Infinity
|
||||
and NaN in the input are passed on unchanged. Integers only overflow if
|
||||
number_float_t cannot represent 2^64, e.g., a half-precision type.
|
||||
|
||||
@tparam NumberType a floating-point or integer type
|
||||
@param[in] format the current format (for diagnostics)
|
||||
@param[in] number the number
|
||||
@return whether the SAX parser accepted the value
|
||||
|
||||
@throw out_of_range.406 if a finite @a number overflows number_float_t
|
||||
*/
|
||||
template<typename NumberType>
|
||||
bool emit_float(const input_format_t format, const NumberType number)
|
||||
{
|
||||
const auto result = static_cast<number_float_t>(number);
|
||||
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
|
||||
{
|
||||
return sax->parse_error(chars_read, get_token_string(),
|
||||
out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
|
||||
}
|
||||
return sax->number_float(result, "");
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief create a string by reading characters from the input
|
||||
|
||||
@@ -19465,6 +19179,10 @@ class json_pointer
|
||||
@return const reference to the JSON value pointed to by the JSON
|
||||
pointer
|
||||
|
||||
@pre Every object key and array index the pointer refers to exists.
|
||||
Like the const operator[] for keys and indices, a missing one is
|
||||
undefined behavior, guarded by a runtime assertion.
|
||||
|
||||
@throw parse_error.106 if an array index begins with '0'
|
||||
@throw parse_error.109 if an array index was not a number
|
||||
@throw out_of_range.402 if the array index '-' is used
|
||||
@@ -19479,7 +19197,8 @@ class json_pointer
|
||||
{
|
||||
case detail::value_t::object:
|
||||
{
|
||||
// use unchecked object access
|
||||
// use unchecked object access; the const operator[]
|
||||
// asserts that the key exists
|
||||
ptr = &ptr->operator[](reference_token);
|
||||
break;
|
||||
}
|
||||
@@ -19492,7 +19211,8 @@ class json_pointer
|
||||
JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
|
||||
}
|
||||
|
||||
// use unchecked array access
|
||||
// use unchecked array access; the const operator[]
|
||||
// asserts that the index exists
|
||||
ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
|
||||
break;
|
||||
}
|
||||
@@ -29035,6 +28755,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
|
||||
// const operator[] only works for arrays
|
||||
if (JSON_HEDLEY_LIKELY(is_array()))
|
||||
{
|
||||
JSON_ASSERT(idx < m_data.m_value.array->size());
|
||||
return m_data.m_value.array->operator[](idx);
|
||||
}
|
||||
|
||||
|
||||
@@ -11,12 +11,7 @@
|
||||
#include <nlohmann/json.hpp>
|
||||
using nlohmann::json;
|
||||
|
||||
#include <cmath>
|
||||
#include <fstream>
|
||||
#include <limits>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include "make_test_data_available.hpp"
|
||||
|
||||
TEST_CASE("Binary Formats" * doctest::skip())
|
||||
@@ -229,139 +224,3 @@ TEST_CASE("Binary Formats" * doctest::skip())
|
||||
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
|
||||
}
|
||||
}
|
||||
|
||||
namespace
|
||||
{
|
||||
// the binary formats as function pointers for "Binary formats with narrow number types";
|
||||
// named functions rather than lambdas, because clang 3.5 cannot convert a lambda
|
||||
// to a function pointer in the braced initializer of the format table
|
||||
using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
|
||||
using bytes = std::vector<std::uint8_t>;
|
||||
|
||||
bytes encode_cbor(const json& j)
|
||||
{
|
||||
return json::to_cbor(j);
|
||||
}
|
||||
narrow_json decode_cbor(const bytes& v, bool allow_exceptions)
|
||||
{
|
||||
return narrow_json::from_cbor(v, true, allow_exceptions);
|
||||
}
|
||||
|
||||
bytes encode_msgpack(const json& j)
|
||||
{
|
||||
return json::to_msgpack(j);
|
||||
}
|
||||
narrow_json decode_msgpack(const bytes& v, bool allow_exceptions)
|
||||
{
|
||||
return narrow_json::from_msgpack(v, true, allow_exceptions);
|
||||
}
|
||||
|
||||
bytes encode_ubjson(const json& j)
|
||||
{
|
||||
return json::to_ubjson(j);
|
||||
}
|
||||
narrow_json decode_ubjson(const bytes& v, bool allow_exceptions)
|
||||
{
|
||||
return narrow_json::from_ubjson(v, true, allow_exceptions);
|
||||
}
|
||||
|
||||
bytes encode_bjdata(const json& j)
|
||||
{
|
||||
return json::to_bjdata(j);
|
||||
}
|
||||
narrow_json decode_bjdata(const bytes& v, bool allow_exceptions)
|
||||
{
|
||||
return narrow_json::from_bjdata(v, true, allow_exceptions);
|
||||
}
|
||||
|
||||
// BSON can only store numbers as object members
|
||||
bytes encode_bson(const json& j)
|
||||
{
|
||||
return json::to_bson(json{{"a", j}});
|
||||
}
|
||||
narrow_json decode_bson(const bytes& v, bool allow_exceptions)
|
||||
{
|
||||
const auto result = narrow_json::from_bson(v, true, allow_exceptions);
|
||||
return result.is_discarded() ? result : result.at("a");
|
||||
}
|
||||
|
||||
bytes encode_bon8(const json& j)
|
||||
{
|
||||
return json::to_bon8(j);
|
||||
}
|
||||
narrow_json decode_bon8(const bytes& v, bool allow_exceptions)
|
||||
{
|
||||
return narrow_json::from_bon8(v, true, allow_exceptions);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST_CASE("Binary formats with narrow number types")
|
||||
{
|
||||
// Numbers that do not fit the number types are handled like the lexer
|
||||
// handles them in JSON text: an integer that fits neither integer type is
|
||||
// stored as a floating-point number, and a finite floating-point number
|
||||
// that overflows number_float_t is rejected with out_of_range.406.
|
||||
struct binary_format
|
||||
{
|
||||
const char* name;
|
||||
bytes (*encode)(const json&);
|
||||
narrow_json (*decode)(const bytes&, bool);
|
||||
};
|
||||
|
||||
const std::vector<binary_format> formats =
|
||||
{
|
||||
{"CBOR", encode_cbor, decode_cbor},
|
||||
{"MessagePack", encode_msgpack, decode_msgpack},
|
||||
{"UBJSON", encode_ubjson, decode_ubjson},
|
||||
{"BJData", encode_bjdata, decode_bjdata},
|
||||
{"BSON", encode_bson, decode_bson},
|
||||
{"BON8", encode_bon8, decode_bon8},
|
||||
};
|
||||
|
||||
for (const auto& format : formats)
|
||||
{
|
||||
const std::string name = format.name;
|
||||
INFO("format := ", name);
|
||||
const auto roundtrip = [&format](const json & j)
|
||||
{
|
||||
return format.decode(format.encode(j), true);
|
||||
};
|
||||
|
||||
// integers that fit keep their type
|
||||
CHECK(roundtrip(json(-5)).is_number_integer());
|
||||
CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
|
||||
CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
|
||||
CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
|
||||
|
||||
// integers that fit neither integer type are stored as float
|
||||
CHECK(roundtrip(json(5000000000u)).is_number_float());
|
||||
CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
|
||||
if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
|
||||
{
|
||||
CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
|
||||
CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
|
||||
}
|
||||
CHECK(roundtrip(json(-3000000000LL)).is_number_float());
|
||||
CHECK(roundtrip(json(-3000000000LL)).get<float>() == -3000000000.0f);
|
||||
CHECK(roundtrip(json(-5000000000LL)).is_number_float());
|
||||
CHECK(roundtrip(json(-5000000000LL)).get<float>() == -5000000000.0f);
|
||||
|
||||
// floating-point numbers that fit
|
||||
CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
|
||||
const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
|
||||
std::numeric_limits<double>::infinity());
|
||||
CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
|
||||
|
||||
// infinity and NaN are passed on
|
||||
CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
|
||||
CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
|
||||
|
||||
// finite floating-point numbers that overflow number_float_t are rejected
|
||||
const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
|
||||
+ " value: number overflow";
|
||||
CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
|
||||
CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
|
||||
CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
|
||||
}
|
||||
}
|
||||
|
||||
+16
-59
@@ -1830,51 +1830,10 @@ TEST_CASE("CBOR")
|
||||
SECTION("invalid string in map")
|
||||
{
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
|
||||
CHECK(json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01}), true, false).is_discarded());
|
||||
}
|
||||
|
||||
SECTION("non-string key (see #2766 and #3381)")
|
||||
{
|
||||
// only text strings map to JSON object keys; any other key is
|
||||
// rejected with a message naming its type
|
||||
const std::vector<std::pair<std::vector<std::uint8_t>, std::string>> cases =
|
||||
{
|
||||
{{0xA1, 0x01, 0x01}, "an unsigned integer; last byte: 0x01"},
|
||||
{{0xA1, 0x20, 0x01}, "a negative integer; last byte: 0x20"},
|
||||
{{0xA1, 0x41, 0x61, 0x01}, "a byte string; last byte: 0x41"},
|
||||
{{0xA1, 0x80, 0x01}, "an array; last byte: 0x80"},
|
||||
{{0xA1, 0xA0, 0x01}, "a map; last byte: 0xA0"},
|
||||
{{0xA1, 0xC0, 0x61, 0x61, 0x01}, "a tag; last byte: 0xC0"},
|
||||
{{0xA1, 0xF4, 0x01}, "a boolean; last byte: 0xF4"},
|
||||
{{0xA1, 0xF5, 0x01}, "a boolean; last byte: 0xF5"},
|
||||
{{0xA1, 0xF6, 0x01}, "null; last byte: 0xF6"},
|
||||
{{0xA1, 0xF7, 0x01}, "undefined; last byte: 0xF7"},
|
||||
{{0xA1, 0xF9, 0x3C, 0x00, 0x01}, "a floating-point number; last byte: 0xF9"},
|
||||
{{0xA1, 0xFA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFA"},
|
||||
{{0xA1, 0xFB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFB"},
|
||||
{{0xA1, 0xE0, 0x01}, "a simple value; last byte: 0xE0"},
|
||||
{{0xA1, 0xF8, 0x20, 0x01}, "a simple value; last byte: 0xF8"},
|
||||
// indefinite-length map
|
||||
{{0xBF, 0x01, 0x01, 0xFF}, "an unsigned integer; last byte: 0x01"},
|
||||
};
|
||||
|
||||
for (const auto& c : cases)
|
||||
{
|
||||
CAPTURE(c.first)
|
||||
const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found " + c.second;
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(c.first), expected.c_str(), json::parse_error&);
|
||||
CHECK(json::from_cbor(c.first, true, false).is_discarded());
|
||||
}
|
||||
|
||||
// a key of major type 3 with a reserved length is still reported as
|
||||
// a malformed string, and a missing key as the end of input
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing CBOR string: unexpected end of input", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0x7C, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C", json::parse_error&);
|
||||
}
|
||||
|
||||
SECTION("invalid UTF-8 in string (see #5529)")
|
||||
{
|
||||
// a two-character text string (major type 3) whose bytes are not
|
||||
@@ -2325,7 +2284,7 @@ TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
|
||||
SECTION("a break marker outside an indefinite-length string is not a string")
|
||||
{
|
||||
// 0xFF only closes a string that was opened; on its own it is not one
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3187,8 +3146,7 @@ TEST_CASE("Tagged values")
|
||||
// CBOR encodes negative integers as: result = -1 - n
|
||||
// For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
|
||||
// the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
|
||||
// When n > INT64_MAX, the result exceeds int64_t range and is stored
|
||||
// as a floating-point number, as the lexer does for JSON text.
|
||||
// When n > INT64_MAX, the result exceeds int64_t range and is rejected.
|
||||
|
||||
SECTION("n = 0 is valid (result = -1)")
|
||||
{
|
||||
@@ -3209,34 +3167,33 @@ TEST_CASE("Tagged values")
|
||||
CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
|
||||
}
|
||||
|
||||
SECTION("n = INT64_MAX + 1 is stored as float")
|
||||
SECTION("n = INT64_MAX + 1 is rejected (overflow)")
|
||||
{
|
||||
// n = INT64_MAX + 1 (0x8000000000000000)
|
||||
// result = -1 - n = -9223372036854775809, which exceeds int64_t range;
|
||||
// the nearest double is -9223372036854775808.0
|
||||
// result = -1 - n = -9223372036854775809, which exceeds int64_t range
|
||||
const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
|
||||
const auto result = json::from_cbor(input);
|
||||
CHECK(result.is_number_float());
|
||||
CHECK(result.get<double>() == -9223372036854775808.0);
|
||||
CHECK(result == json::parse("-9223372036854775809"));
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
|
||||
"[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
|
||||
json::parse_error);
|
||||
}
|
||||
|
||||
SECTION("n = UINT64_MAX is stored as float")
|
||||
SECTION("n = UINT64_MAX is rejected (overflow)")
|
||||
{
|
||||
// n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
|
||||
// result = -1 - n = -18446744073709551616, which exceeds int64_t range
|
||||
const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
|
||||
const auto result = json::from_cbor(input);
|
||||
CHECK(result.is_number_float());
|
||||
CHECK(result.get<double>() == -18446744073709551616.0);
|
||||
CHECK(result == json::parse("-18446744073709551616"));
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
|
||||
"[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
|
||||
json::parse_error);
|
||||
}
|
||||
|
||||
SECTION("overflow with allow_exceptions=false is not an error")
|
||||
SECTION("overflow with allow_exceptions=false returns discarded")
|
||||
{
|
||||
const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
|
||||
const auto result = json::from_cbor(input, true, false);
|
||||
CHECK(result.is_number_float());
|
||||
CHECK(result.is_discarded());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -666,7 +666,7 @@ TEST_CASE("parse_float_fast declines what it cannot convert exactly")
|
||||
// always safe: the caller then falls back to a slower, exact conversion.
|
||||
const auto fast = [](const std::string & s, double & out)
|
||||
{
|
||||
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
|
||||
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), '.', out);
|
||||
};
|
||||
double out = 0;
|
||||
|
||||
|
||||
@@ -12,12 +12,7 @@
|
||||
#include <nlohmann/json.hpp>
|
||||
using nlohmann::json;
|
||||
|
||||
#include <array>
|
||||
#include <clocale>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
struct ParserImpl final: public nlohmann::json_sax<json>
|
||||
{
|
||||
@@ -180,208 +175,3 @@ TEST_CASE("locale-dependent test (LC_NUMERIC=de_DE)")
|
||||
MESSAGE("locale de_DE is not usable");
|
||||
}
|
||||
}
|
||||
|
||||
namespace
|
||||
{
|
||||
// records the numbers of a flat array and switches LC_NUMERIC to the given
|
||||
// locale once the array opens - after the lexer was constructed, but before
|
||||
// any number in the array is lexed
|
||||
struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
|
||||
{
|
||||
explicit LocaleSwitchingSax(const char* switch_to)
|
||||
: locale_after_open(switch_to)
|
||||
{}
|
||||
|
||||
bool null() override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool boolean(bool /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool number_integer(json::number_integer_t /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool number_unsigned(json::number_unsigned_t /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool number_float(json::number_float_t val, const json::string_t& s) override
|
||||
{
|
||||
values.push_back(val);
|
||||
strings.push_back(s);
|
||||
return true;
|
||||
}
|
||||
bool string(json::string_t& /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool binary(json::binary_t& /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool start_object(std::size_t /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool key(json::string_t& /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool end_object() override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool start_array(std::size_t /*val*/) override
|
||||
{
|
||||
switched = std::setlocale(LC_NUMERIC, locale_after_open.c_str()) != nullptr;
|
||||
return true;
|
||||
}
|
||||
bool end_array() override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool parse_error(std::size_t /*val*/, const std::string& /*val*/, const nlohmann::detail::exception& /*val*/) override
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string locale_after_open;
|
||||
bool switched = false;
|
||||
std::vector<json::number_float_t> values {}; // NOLINT(readability-redundant-member-init)
|
||||
std::vector<json::string_t> strings {}; // NOLINT(readability-redundant-member-init)
|
||||
};
|
||||
} // namespace
|
||||
|
||||
TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
|
||||
{
|
||||
// The numbers are chosen so that the conversion also takes the strtod
|
||||
// fallback, which honors the locale that is current at conversion time:
|
||||
// too many significant digits for Clinger's fast path, an underflow that
|
||||
// std::from_chars rejects, and a plain value.
|
||||
const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
|
||||
std::string text = "[";
|
||||
for (const auto& n : numbers)
|
||||
{
|
||||
text += (text.size() == 1 ? "" : ",") + n;
|
||||
}
|
||||
text += "]";
|
||||
|
||||
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
|
||||
|
||||
// reference values, parsed without a locale switch
|
||||
REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
|
||||
const json expected = json::parse(text);
|
||||
const long_double_json expected_ld = long_double_json::parse(text);
|
||||
|
||||
const std::array<std::pair<const char*, const char*>, 2> transitions =
|
||||
{
|
||||
{
|
||||
{"C", "de_DE"},
|
||||
{"de_DE", "C"}
|
||||
}
|
||||
};
|
||||
|
||||
for (const auto& transition : transitions)
|
||||
{
|
||||
CAPTURE(transition.first);
|
||||
CAPTURE(transition.second);
|
||||
|
||||
if (std::setlocale(LC_NUMERIC, transition.first) == nullptr)
|
||||
{
|
||||
MESSAGE("locale is not usable");
|
||||
continue;
|
||||
}
|
||||
|
||||
// SAX parsing
|
||||
{
|
||||
LocaleSwitchingSax sax(transition.second);
|
||||
CHECK(json::sax_parse(text, &sax));
|
||||
if (sax.switched)
|
||||
{
|
||||
CHECK(sax.values == expected.get<std::vector<json::number_float_t>>());
|
||||
CHECK(sax.strings == numbers);
|
||||
}
|
||||
}
|
||||
|
||||
// DOM parsing with a callback
|
||||
{
|
||||
bool switched = false;
|
||||
const auto cb = [&](int /*depth*/, json::parse_event_t event, json& /*parsed*/) noexcept
|
||||
{
|
||||
if (event == json::parse_event_t::array_start)
|
||||
{
|
||||
switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
|
||||
}
|
||||
return true;
|
||||
};
|
||||
const json j = json::parse(text, cb);
|
||||
if (switched)
|
||||
{
|
||||
CHECK(j == expected);
|
||||
}
|
||||
}
|
||||
|
||||
// a long double goes through std::strtold unless std::from_chars supports it
|
||||
{
|
||||
bool switched = false;
|
||||
const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/) noexcept
|
||||
{
|
||||
if (event == long_double_json::parse_event_t::array_start)
|
||||
{
|
||||
switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
|
||||
}
|
||||
return true;
|
||||
};
|
||||
const long_double_json j = long_double_json::parse(text, cb);
|
||||
if (switched)
|
||||
{
|
||||
CHECK(j == expected_ld);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
CHECK(std::setlocale(LC_NUMERIC, "C") != nullptr);
|
||||
}
|
||||
|
||||
TEST_CASE("locale with a multi-byte decimal point")
|
||||
{
|
||||
// Some locales use a decimal point that is not a single character, e.g.
|
||||
// U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
|
||||
// substituted in place for '.', so the strtod fallback stops early. The
|
||||
// conversion must still terminate rather than retry forever.
|
||||
const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
|
||||
bool tested = false;
|
||||
for (const char* name : names)
|
||||
{
|
||||
if (std::setlocale(LC_NUMERIC, name) == nullptr)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
const std::string decimal_point = std::localeconv()->decimal_point;
|
||||
if (decimal_point.size() < 2)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
CAPTURE(name);
|
||||
tested = true;
|
||||
|
||||
// too many significant digits for Clinger's fast path, and an underflow
|
||||
// that std::from_chars rejects: both reach the strtod fallback
|
||||
json j;
|
||||
CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
|
||||
CHECK(j.is_array());
|
||||
CHECK(json::accept("3.14159265358979323846"));
|
||||
|
||||
// a value the locale-independent paths convert is not affected
|
||||
CHECK(json::parse("12.5") == 12.5);
|
||||
}
|
||||
if (!tested)
|
||||
{
|
||||
MESSAGE("no locale with a multi-byte decimal point is usable");
|
||||
}
|
||||
|
||||
CHECK(std::setlocale(LC_NUMERIC, "C") != nullptr);
|
||||
}
|
||||
|
||||
@@ -1551,69 +1551,10 @@ TEST_CASE("MessagePack")
|
||||
SECTION("invalid string in map")
|
||||
{
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found an integer; last byte: 0xFF", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF", json::parse_error&);
|
||||
CHECK(json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01}), true, false).is_discarded());
|
||||
}
|
||||
|
||||
SECTION("non-string key (see #3381)")
|
||||
{
|
||||
// only strings map to JSON object keys; any other key is rejected
|
||||
// with a message naming its type
|
||||
const std::vector<std::pair<std::vector<std::uint8_t>, std::string>> cases =
|
||||
{
|
||||
{{0x81, 0xC0, 0x01}, "nil; last byte: 0xC0"},
|
||||
{{0x81, 0xC2, 0x01}, "a boolean; last byte: 0xC2"},
|
||||
{{0x81, 0xC3, 0x01}, "a boolean; last byte: 0xC3"},
|
||||
{{0x81, 0xCA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a float; last byte: 0xCA"},
|
||||
{{0x81, 0xCB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a float; last byte: 0xCB"},
|
||||
{{0x81, 0xC4, 0x00, 0x01}, "a bin; last byte: 0xC4"},
|
||||
{{0x81, 0xC5, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC5"},
|
||||
{{0x81, 0xC6, 0x00, 0x00, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC6"},
|
||||
{{0x81, 0xC7, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC7"},
|
||||
{{0x81, 0xC8, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC8"},
|
||||
{{0x81, 0xC9, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC9"},
|
||||
{{0x81, 0xD4, 0x01, 0x00, 0x01}, "an ext; last byte: 0xD4"},
|
||||
{{0x81, 0xD5, 0x01, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD5"},
|
||||
{{0x81, 0xD6, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD6"},
|
||||
{{0x81, 0xD7, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD7"},
|
||||
{{0x81, 0xD8, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD8"},
|
||||
{{0x81, 0xCC, 0x01, 0x01}, "an integer; last byte: 0xCC"},
|
||||
{{0x81, 0xCD, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCD"},
|
||||
{{0x81, 0xCE, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCE"},
|
||||
{{0x81, 0xCF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCF"},
|
||||
{{0x81, 0xD0, 0x01, 0x01}, "an integer; last byte: 0xD0"},
|
||||
{{0x81, 0xD1, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD1"},
|
||||
{{0x81, 0xD2, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD2"},
|
||||
{{0x81, 0xD3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD3"},
|
||||
{{0x81, 0x00, 0x01}, "an integer; last byte: 0x00"},
|
||||
{{0x81, 0x7F, 0x01}, "an integer; last byte: 0x7F"},
|
||||
{{0x81, 0xE0, 0x01}, "an integer; last byte: 0xE0"},
|
||||
{{0x81, 0x80, 0x01}, "a map; last byte: 0x80"},
|
||||
{{0x81, 0x8F, 0x01}, "a map; last byte: 0x8F"},
|
||||
{{0x81, 0xDE, 0x00, 0x00, 0x01}, "a map; last byte: 0xDE"},
|
||||
{{0x81, 0xDF, 0x00, 0x00, 0x00, 0x00, 0x01}, "a map; last byte: 0xDF"},
|
||||
{{0x81, 0x90, 0x01}, "an array; last byte: 0x90"},
|
||||
{{0x81, 0x9F, 0x01}, "an array; last byte: 0x9F"},
|
||||
{{0x81, 0xDC, 0x00, 0x00, 0x01}, "an array; last byte: 0xDC"},
|
||||
{{0x81, 0xDD, 0x00, 0x00, 0x00, 0x00, 0x01}, "an array; last byte: 0xDD"},
|
||||
};
|
||||
|
||||
for (const auto& c : cases)
|
||||
{
|
||||
CAPTURE(c.first)
|
||||
const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found " + c.second;
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(c.first), expected.c_str(), json::parse_error&);
|
||||
CHECK(json::from_msgpack(c.first, true, false).is_discarded());
|
||||
}
|
||||
|
||||
json _;
|
||||
// the unused byte 0xC1 is still reported as a malformed string
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xC1, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xC1", json::parse_error&);
|
||||
// a missing key is still reported as the end of input
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing MessagePack string: unexpected end of input", json::parse_error&);
|
||||
}
|
||||
|
||||
SECTION("invalid UTF-8 in string (see #5529)")
|
||||
{
|
||||
// a fixstr of length 2 (0xA0 | 2) whose bytes are not valid UTF-8
|
||||
|
||||
@@ -1018,7 +1018,7 @@ TEST_CASE("regression tests 1")
|
||||
};
|
||||
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an array; last byte: 0x98", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x98", json::parse_error&);
|
||||
|
||||
// related test case: nonempty UTF-8 string (indefinite length)
|
||||
std::vector<uint8_t> const vec1 {0x7f, 0x61, 0x61};
|
||||
@@ -1065,7 +1065,7 @@ TEST_CASE("regression tests 1")
|
||||
};
|
||||
|
||||
json _;
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&);
|
||||
|
||||
// related test case: double-precision
|
||||
std::vector<uint8_t> const vec2
|
||||
@@ -1077,7 +1077,7 @@ TEST_CASE("regression tests 1")
|
||||
0x96, 0x96, 0xb4, 0xb4, 0xfa, 0x94, 0x94, 0x61,
|
||||
0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0xfb
|
||||
};
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&);
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&);
|
||||
}
|
||||
|
||||
SECTION("issue #452 - Heap-buffer-overflow (OSS-Fuzz issue 585)")
|
||||
|
||||
Reference in New Issue
Block a user