mirror of
https://github.com/nlohmann/json.git
synced 2026-09-29 11:10:30 +00:00
Add BON8 support (#2998)
* Add BON8 support Add to_bon8/from_bon8 and input_format_t::bon8 for BON8, a binary format that uses the byte values that cannot begin a UTF-8 character as type markers, so strings need no length prefix. It is the most compact of the supported binary formats on the benchmark files. The reader is non-recursive like the other binary readers. A string ends at the first byte that cannot continue it, so the reader hands the one or two bytes it reads past a string back to the value that follows. The writer produces the canonical representation of the specification, except for NFC normalization; its output is identical to that of the reference implementation (HikoGUI) on all files of the test data. The round-trip tests need the .bon8 files of json_test_data 3.2.0. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review comments - Reuse detail::validate_one_utf8 to check strings in to_bon8; the error now names the first byte of the invalid sequence. - Document that to_bon8 leaves bytes in the output adapter on an exception, and that string_open is only an output of write_bon8_marker. - Explain why the pushback buffer of the BON8 reader cannot overflow. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Select the BON8 float prefix by type get_bon8_float_prefix only depends on the type of its argument, so make the type a template parameter instead of passing an unused value. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename a test variable that Flawfinder mistakes for read() Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the BON8 CI failures - compare the float in write_bon8_float with number_float_t constants, so GCC does not warn about a float-to-double conversion - mark check_bon8_utf8's context as used when exceptions are disabled - choose the compact float prefix in a helper rather than with nested conditional operators (clang-tidy) - use auto for the cast in the BON8 integer reader (clang-tidy) - write the int32 minimum test values as long long literals (MSVC C4146) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Amalgamate Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Read BON8 strings in bulk from contiguous input - copy the valid UTF-8 of a string in one step when the input is contiguous (twitter.json is read in 1.68 instead of 2.52 ms, jeopardy.json in 196 instead of 297 ms, close to CBOR and MessagePack) - share the new valid_utf8_prefix() with the writer's UTF-8 check, which now skips ASCII 8 bytes at a time - let the fuzzer check that contiguous and stream input give the same value or error, and test both paths in the unit tests - clarify that a second 0xFF after a string is an empty string Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Link the BON8 functions from the other binary format pages Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Name the bulk scan flag after the input, not BON8 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Read BSON keys in bulk from contiguous input BSON keys (and array indices) are C-style strings, which were read byte by byte. For contiguous input they are now read up to their \x00-byte in one step, using the same bulk_scan flag as BON8 strings: twitter.json is read in 1.46 instead of 2.01 ms, citm_catalog.json in 2.93 instead of 3.33 ms, jeopardy.json in 182 instead of 207 ms. canada.json, whose keys are almost all one-digit array indices, takes 2 % longer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the BON8 CI failures of the bulk-read tests - skip the contiguous-versus-stream tests of BON8 strings and BSON keys when exceptions are disabled: they catch the parse errors of invalid input, and without exceptions the library aborts instead - use static_cast for the int64 test value (google-readability-casting) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the explicit basic_json instantiation into its own test file Linking test-regression3_cpp20 with clang and MinGW failed with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'", as test-regression2 did before #5511. The explicit instantiation of basic_json<> for #4825 compiles every member function, including the BON8 reader and writer, into that object, and it was already close to the limit (2,226,104 bytes on develop, 2,234,960 with BON8; clang -O1, C++20). Give the instantiation a file of its own: unit-regression3 is now 1,594,736 bytes and unit-explicit_instantiation 1,095,064. The new file mentions JSON_HAS_CPP_17 and JSON_HAS_CPP_20 so it keeps being built for the C++17 standard the regression was about. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Convert the bytes of the BON8 test strings explicitly The str() helper constructed a std::string from a byte range, which converts each unsigned char implicitly; -fsanitize=integer reports that for bytes of 0x80 and above (ci_test_clang_sanitizer). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
@@ -28,6 +28,7 @@
|
||||
#include <nlohmann/detail/input/input_adapters.hpp>
|
||||
#include <nlohmann/detail/input/json_sax.hpp>
|
||||
#include <nlohmann/detail/input/lexer.hpp>
|
||||
#include <nlohmann/detail/input/string_scan.hpp>
|
||||
#include <nlohmann/detail/macro_scope.hpp>
|
||||
#include <nlohmann/detail/meta/is_sax.hpp>
|
||||
#include <nlohmann/detail/meta/type_traits.hpp>
|
||||
@@ -84,7 +85,7 @@ JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 2
|
||||
///////////////////
|
||||
|
||||
/*!
|
||||
@brief deserialization of CBOR, MessagePack, and UBJSON values
|
||||
@brief deserialization of BJData, BON8, BSON, CBOR, MessagePack, and UBJSON values
|
||||
*/
|
||||
template<typename BasicJsonType, typename InputAdapterType, typename SAX = json_sax_dom_parser<BasicJsonType, InputAdapterType>>
|
||||
class binary_reader
|
||||
@@ -98,6 +99,11 @@ class binary_reader
|
||||
using char_type = typename InputAdapterType::char_type;
|
||||
using char_int_type = typename char_traits<char_type>::int_type;
|
||||
|
||||
/// whether the input is a contiguous block of bytes that can be inspected
|
||||
/// and consumed in bulk (as in the lexer); used by @ref get_bon8_string_bulk
|
||||
static constexpr bool bulk_scan =
|
||||
input_adapter_supports_bulk_scan<InputAdapterType>(is_detected<detect_supports_bulk_scan, InputAdapterType> {});
|
||||
|
||||
public:
|
||||
/*!
|
||||
@brief create a binary reader
|
||||
@@ -132,6 +138,7 @@ class binary_reader
|
||||
{
|
||||
sax = sax_;
|
||||
container_stack.clear();
|
||||
bon8_pushback_size = 0;
|
||||
bool result = false;
|
||||
|
||||
switch (format)
|
||||
@@ -153,6 +160,10 @@ class binary_reader
|
||||
result = parse_ubjson_internal();
|
||||
break;
|
||||
|
||||
case input_format_t::bon8:
|
||||
result = parse_bon8_internal();
|
||||
break;
|
||||
|
||||
case input_format_t::json: // LCOV_EXCL_LINE
|
||||
default: // LCOV_EXCL_LINE
|
||||
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
|
||||
@@ -165,6 +176,11 @@ class binary_reader
|
||||
{
|
||||
get_ignore_noop();
|
||||
}
|
||||
else if (input_format == input_format_t::bon8)
|
||||
{
|
||||
// a string that ends a container hands back the byte after it
|
||||
get_bon8();
|
||||
}
|
||||
else
|
||||
{
|
||||
get();
|
||||
@@ -396,6 +412,11 @@ class binary_reader
|
||||
*/
|
||||
bool get_bson_cstr(string_t& result)
|
||||
{
|
||||
if (get_bson_cstr_bulk(result, std::integral_constant<bool, bulk_scan> {}))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
auto out = std::back_inserter(result);
|
||||
while (true)
|
||||
{
|
||||
@@ -412,6 +433,46 @@ class binary_reader
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief read a C-style string from contiguous input in one step
|
||||
|
||||
@param[in,out] result the string to append to
|
||||
@return whether the string was read; if the input has no \x00-byte, nothing
|
||||
is read, and @ref get_bson_cstr reports the end of the input
|
||||
*/
|
||||
bool get_bson_cstr_bulk(string_t& result, std::true_type /*bulk*/)
|
||||
{
|
||||
const std::size_t remaining = ia.bulk_remaining();
|
||||
if (remaining == 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
|
||||
// a plain loop rather than std::memchr: most keys are short (array
|
||||
// indices are keys, too), and the call would cost more than it saves
|
||||
std::size_t length = 0;
|
||||
while (length < remaining && data[length] != 0x00)
|
||||
{
|
||||
++length;
|
||||
}
|
||||
if (length == remaining)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
result.append(reinterpret_cast<const typename string_t::value_type*>(data), length);
|
||||
// consume the string and its \x00-byte, as the byte-wise path does
|
||||
ia.bulk_skip(length + 1);
|
||||
chars_read += length + 1;
|
||||
current = 0x00;
|
||||
return true;
|
||||
}
|
||||
|
||||
/// input that is not contiguous: C-style strings are read byte by byte
|
||||
bool get_bson_cstr_bulk(string_t& /*result*/, std::false_type /*bulk*/) const noexcept
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief Parses a zero-terminated string of length @a len from the BSON
|
||||
input.
|
||||
@@ -3199,6 +3260,549 @@ class binary_reader
|
||||
}
|
||||
}
|
||||
|
||||
//////////
|
||||
// BON8 //
|
||||
//////////
|
||||
|
||||
/*!
|
||||
@brief get the next byte of a BON8 value
|
||||
|
||||
A BON8 string has no length prefix and no mandatory terminator: it ends at
|
||||
the first byte that cannot continue it, which is already the first byte (or,
|
||||
for an integer that begins with a UTF-8 lead byte, the first two bytes) of
|
||||
whatever follows. The string reader hands those bytes back with
|
||||
@ref unget_bon8, and every BON8 read goes through this function so that
|
||||
they are seen again.
|
||||
|
||||
@return character read from the input
|
||||
*/
|
||||
char_int_type get_bon8()
|
||||
{
|
||||
if (bon8_pushback_size != 0)
|
||||
{
|
||||
++chars_read;
|
||||
return current = bon8_pushback[--bon8_pushback_size];
|
||||
}
|
||||
return get();
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief hand a byte back so that the next @ref get_bon8 returns it again
|
||||
|
||||
@param[in] c the byte to hand back; bytes handed back are returned in
|
||||
reverse order
|
||||
*/
|
||||
void unget_bon8(const char_int_type c)
|
||||
{
|
||||
// At most two bytes are ever handed back: a byte is only handed back
|
||||
// right after it was read with get_bon8(), and the only place that
|
||||
// hands back two bytes (a lead byte and the byte after it) read both
|
||||
// of them in a row, which emptied the buffer first. This is an
|
||||
// invariant of the reader rather than a property of the input, so
|
||||
// an assertion suffices (the fuzzers are built with assertions).
|
||||
JSON_ASSERT(bon8_pushback_size < bon8_pushback.size());
|
||||
bon8_pushback[bon8_pushback_size++] = c;
|
||||
--chars_read;
|
||||
}
|
||||
|
||||
/*!
|
||||
@param[in] c a byte
|
||||
@return whether @a c is a UTF-8 continuation byte (0x80..0xBF)
|
||||
*/
|
||||
static constexpr bool is_bon8_continuation(const char_int_type c) noexcept
|
||||
{
|
||||
return 0x80 <= c && c <= 0xBF;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief report a parse error at the last read byte
|
||||
|
||||
@param[in] detail a detailed error message
|
||||
@param[in] context further context information
|
||||
@return false
|
||||
*/
|
||||
bool bon8_error(const std::string& detail, const char* context)
|
||||
{
|
||||
auto last_token = get_token_string();
|
||||
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
||||
exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief read a BON8 value and everything nested inside it
|
||||
|
||||
Reads values until the one that was begun here is complete, resuming the
|
||||
enclosing container after each element, so that the nesting depth of the
|
||||
input costs heap rather than native stack (see #5104).
|
||||
|
||||
@return whether reading the value succeeded
|
||||
*/
|
||||
bool parse_bon8_internal()
|
||||
{
|
||||
// the key currently being read; hoisted out of the loop so that its
|
||||
// capacity is reused across elements and across nesting levels
|
||||
string_t key;
|
||||
|
||||
while (true)
|
||||
{
|
||||
if (!container_stack.empty())
|
||||
{
|
||||
// a copy, not a reference: it must stay valid across the
|
||||
// pop_back() below, which destroys the container_stack element
|
||||
// it would otherwise alias
|
||||
const container_frame top = container_stack.back();
|
||||
bool at_end = false;
|
||||
|
||||
if (top.remaining != npos)
|
||||
{
|
||||
// counted container (0x80..0x84, 0x86..0x8A): it ends once
|
||||
// its elements have been read
|
||||
at_end = (top.remaining == 0);
|
||||
if (!at_end)
|
||||
{
|
||||
// claim the element about to be read
|
||||
--container_stack.back().remaining;
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
// container 0x85 or 0x8B: it ends at an end-of-container
|
||||
// marker (0xFE); any other byte begins the next element
|
||||
at_end = (get_bon8() == 0xFE);
|
||||
if (!at_end)
|
||||
{
|
||||
unget_bon8(current);
|
||||
}
|
||||
}
|
||||
|
||||
if (at_end)
|
||||
{
|
||||
container_stack.pop_back();
|
||||
if (JSON_HEDLEY_UNLIKELY(top.is_object ? !sax->end_object() : !sax->end_array()))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
// the value begun here is complete once its container is
|
||||
if (container_stack.empty())
|
||||
{
|
||||
return true;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (top.is_object)
|
||||
{
|
||||
key.clear();
|
||||
if (JSON_HEDLEY_UNLIKELY(!get_bon8_key(key) || !sax->key(key)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_UNLIKELY(!parse_bon8_value()))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// a value that opened a container left it on the stack; one that
|
||||
// did not, and that was not inside a container, was the whole value
|
||||
if (container_stack.empty())
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief read one BON8 value
|
||||
|
||||
Reads a single value and passes it to the SAX parser. A value that begins
|
||||
a container is not read to its end: the container is opened with
|
||||
@ref enter_container and its elements are read by
|
||||
@ref parse_bon8_internal, so that nesting does not consume native stack.
|
||||
|
||||
@return whether reading the value succeeded
|
||||
*/
|
||||
bool parse_bon8_value()
|
||||
{
|
||||
const auto byte = get_bon8();
|
||||
|
||||
if (byte == char_traits<char_type>::eof())
|
||||
{
|
||||
return unexpect_eof(input_format_t::bon8, "value");
|
||||
}
|
||||
|
||||
// string: ASCII character
|
||||
if (byte <= 0x7F)
|
||||
{
|
||||
string_t s;
|
||||
unget_bon8(byte);
|
||||
return get_bon8_string(s) && sax->string(s);
|
||||
}
|
||||
|
||||
// array with 0..4 elements
|
||||
if (byte <= 0x84)
|
||||
{
|
||||
return enter_array(static_cast<std::size_t>(byte - 0x80));
|
||||
}
|
||||
|
||||
// array terminated by 0xFE
|
||||
if (byte == 0x85)
|
||||
{
|
||||
return enter_array(npos);
|
||||
}
|
||||
|
||||
// object with 0..4 members
|
||||
if (byte <= 0x8A)
|
||||
{
|
||||
return enter_object(static_cast<std::size_t>(byte - 0x86));
|
||||
}
|
||||
|
||||
switch (byte)
|
||||
{
|
||||
case 0x8B: // object terminated by 0xFE
|
||||
return enter_object(npos);
|
||||
|
||||
case 0x8C: // int32
|
||||
{
|
||||
std::int32_t number{};
|
||||
return get_number(input_format_t::bon8, number) && emit_bon8_integer(number);
|
||||
}
|
||||
|
||||
case 0x8D: // int64
|
||||
{
|
||||
std::int64_t number{};
|
||||
return get_number(input_format_t::bon8, number) && emit_bon8_integer(number);
|
||||
}
|
||||
|
||||
case 0x8E: // binary32
|
||||
{
|
||||
float number{};
|
||||
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0x8F: // binary64
|
||||
{
|
||||
double number{};
|
||||
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
||||
}
|
||||
|
||||
case 0xF8:
|
||||
return sax->boolean(false);
|
||||
|
||||
case 0xF9:
|
||||
return sax->boolean(true);
|
||||
|
||||
case 0xFA:
|
||||
return sax->null();
|
||||
|
||||
case 0xFB:
|
||||
return sax->number_float(static_cast<number_float_t>(-1.0), "");
|
||||
|
||||
case 0xFC:
|
||||
return sax->number_float(static_cast<number_float_t>(0.0), "");
|
||||
|
||||
case 0xFD:
|
||||
return sax->number_float(static_cast<number_float_t>(1.0), "");
|
||||
|
||||
case 0xFF: // empty string
|
||||
{
|
||||
string_t s;
|
||||
return sax->string(s);
|
||||
}
|
||||
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
// integer 0..39
|
||||
if (byte <= 0xB7)
|
||||
{
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(byte - 0x90));
|
||||
}
|
||||
|
||||
// integer -1..-10
|
||||
if (byte <= 0xC1)
|
||||
{
|
||||
return sax->number_integer(-1 - static_cast<number_integer_t>(byte - 0xB8));
|
||||
}
|
||||
|
||||
// 0xC2..0xF7: a UTF-8 lead byte begins a string if a continuation
|
||||
// byte follows and an integer otherwise
|
||||
if (byte <= 0xF7)
|
||||
{
|
||||
const auto second = get_bon8();
|
||||
if (is_bon8_continuation(second))
|
||||
{
|
||||
string_t s;
|
||||
unget_bon8(second);
|
||||
unget_bon8(byte);
|
||||
return get_bon8_string(s) && sax->string(s);
|
||||
}
|
||||
return get_bon8_integer(byte, second);
|
||||
}
|
||||
|
||||
// 0xFE: end of container where a value is expected
|
||||
return bon8_error("invalid byte", "value");
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief pass an integer to the SAX parser
|
||||
|
||||
Non-negative integers are passed as unsigned, negative integers as signed
|
||||
numbers, like the other binary formats do.
|
||||
|
||||
@param[in] number the integer
|
||||
@return whether the SAX parser accepted the value
|
||||
*/
|
||||
bool emit_bon8_integer(const std::int64_t number)
|
||||
{
|
||||
if (number >= 0)
|
||||
{
|
||||
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
|
||||
}
|
||||
return sax->number_integer(static_cast<number_integer_t>(number));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief read an integer encoded in 2..4 bytes
|
||||
|
||||
The first byte is a UTF-8 lead byte (0xC2..0xF7) that is followed by a
|
||||
byte that is not a continuation byte: 0x00..0x7F for positive and
|
||||
0xC0..0xFF for negative integers. The lead byte's low bits and the second
|
||||
byte's low 7 (positive) or 6 (negative) bits are the most significant bits
|
||||
of the value; 3- and 4-byte integers add one or two full bytes. Each range
|
||||
starts where the shorter one ends, so no value has two encodings of the
|
||||
same length.
|
||||
|
||||
@param[in] lead the first byte (0xC2..0xF7)
|
||||
@param[in] second the second byte
|
||||
@return whether reading the integer succeeded
|
||||
*/
|
||||
bool get_bon8_integer(const char_int_type lead, const char_int_type second)
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number")))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
const bool negative = second >= 0xC0;
|
||||
auto value = static_cast<std::int64_t>(negative ? (second & 0x3F) : second);
|
||||
std::int64_t offset = 0;
|
||||
int extra_bytes = 0;
|
||||
|
||||
if (lead <= 0xDF)
|
||||
{
|
||||
value |= static_cast<std::int64_t>(lead - 0xC2) << (negative ? 6 : 7);
|
||||
offset = negative ? 11 : 40;
|
||||
}
|
||||
else if (lead <= 0xEF)
|
||||
{
|
||||
value |= static_cast<std::int64_t>(lead & 0x0F) << (negative ? 6 : 7);
|
||||
offset = negative ? 1931 : 3880;
|
||||
extra_bytes = 1;
|
||||
}
|
||||
else
|
||||
{
|
||||
value |= static_cast<std::int64_t>(lead & 0x07) << (negative ? 6 : 7);
|
||||
offset = negative ? 264075 : 528168;
|
||||
extra_bytes = 2;
|
||||
}
|
||||
|
||||
for (int i = 0; i < extra_bytes; ++i)
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits<char_type>::eof()))
|
||||
{
|
||||
return unexpect_eof(input_format_t::bon8, "number");
|
||||
}
|
||||
value = (value << 8) | static_cast<std::int64_t>(current);
|
||||
}
|
||||
|
||||
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
|
||||
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief read an object key
|
||||
|
||||
A key must be a string, so its first byte must be an ASCII character, a
|
||||
UTF-8 lead byte followed by a continuation byte, or 0xFF (empty string).
|
||||
|
||||
@param[out] result the key
|
||||
@return whether reading the key succeeded
|
||||
*/
|
||||
bool get_bon8_key(string_t& result)
|
||||
{
|
||||
const auto byte = get_bon8();
|
||||
|
||||
if (byte == char_traits<char_type>::eof())
|
||||
{
|
||||
return unexpect_eof(input_format_t::bon8, "key");
|
||||
}
|
||||
|
||||
if (byte == 0xFF)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
if (byte <= 0x7F)
|
||||
{
|
||||
unget_bon8(byte);
|
||||
return get_bon8_string(result);
|
||||
}
|
||||
|
||||
if (0xC2 <= byte && byte <= 0xF7)
|
||||
{
|
||||
const auto second = get_bon8();
|
||||
unget_bon8(second);
|
||||
if (is_bon8_continuation(second))
|
||||
{
|
||||
unget_bon8(byte);
|
||||
return get_bon8_string(result);
|
||||
}
|
||||
// an integer: report its first byte rather than the one after it
|
||||
current = byte;
|
||||
}
|
||||
|
||||
return bon8_error("expected a string; last byte", "key");
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief append the run of valid UTF-8 at the read position to a string
|
||||
|
||||
For contiguous input, the ASCII characters and complete well-formed UTF-8
|
||||
sequences at the read position are appended to @a result in one step. The
|
||||
byte that stops the run (an end-of-string marker, the first byte of the
|
||||
next value, or an ill-formed byte) is left for @ref get_bon8_string, so
|
||||
that strings end and errors are reported exactly as without this step.
|
||||
|
||||
@param[in,out] result the string to append to
|
||||
*/
|
||||
void get_bon8_string_bulk(string_t& result, std::true_type /*bulk*/)
|
||||
{
|
||||
// bytes handed back must be read through get_bon8() first
|
||||
if (bon8_pushback_size != 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
const std::size_t remaining = ia.bulk_remaining();
|
||||
if (remaining == 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
|
||||
const std::size_t length = valid_utf8_prefix(data, remaining);
|
||||
if (length != 0)
|
||||
{
|
||||
result.append(reinterpret_cast<const typename string_t::value_type*>(data), length);
|
||||
ia.bulk_skip(length);
|
||||
chars_read += length;
|
||||
}
|
||||
}
|
||||
|
||||
/// input that is not contiguous: strings are read byte by byte
|
||||
void get_bon8_string_bulk(string_t& /*result*/, std::false_type /*bulk*/) const noexcept {}
|
||||
|
||||
/*!
|
||||
@brief read a string
|
||||
|
||||
Reads UTF-8 characters until an end-of-string marker (0xFF), which is
|
||||
consumed, or a byte that cannot continue the string, which is handed back
|
||||
to be read as the start of the next value. The string must be valid UTF-8,
|
||||
and it must not end at the end of the input: the last string of a message
|
||||
is always terminated by 0xFF.
|
||||
|
||||
@param[out] result the string
|
||||
@return whether reading the string succeeded
|
||||
*/
|
||||
bool get_bon8_string(string_t& result)
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
get_bon8_string_bulk(result, std::integral_constant<bool, bulk_scan> {});
|
||||
|
||||
const auto byte = get_bon8();
|
||||
|
||||
if (byte == char_traits<char_type>::eof())
|
||||
{
|
||||
return unexpect_eof(input_format_t::bon8, "string");
|
||||
}
|
||||
|
||||
// end of string
|
||||
if (byte == 0xFF)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// ASCII character
|
||||
if (byte <= 0x7F)
|
||||
{
|
||||
result.push_back(static_cast<typename string_t::value_type>(byte));
|
||||
continue;
|
||||
}
|
||||
|
||||
// a byte that cannot begin a character ends the string and begins
|
||||
// the next value
|
||||
if (byte < 0xC2 || byte > 0xF7)
|
||||
{
|
||||
unget_bon8(byte);
|
||||
return true;
|
||||
}
|
||||
|
||||
// a lead byte ends the string if no continuation byte follows: it
|
||||
// is then the first byte of an integer
|
||||
const auto second = get_bon8();
|
||||
if (!is_bon8_continuation(second))
|
||||
{
|
||||
unget_bon8(second);
|
||||
unget_bon8(byte);
|
||||
return true;
|
||||
}
|
||||
|
||||
// the valid range of the second byte excludes overlong forms,
|
||||
// surrogates, and code points above U+10FFFF
|
||||
// (RFC 3629, section 4)
|
||||
int continuation_bytes = 0;
|
||||
bool valid_second = true;
|
||||
if (byte <= 0xDF)
|
||||
{
|
||||
continuation_bytes = 1;
|
||||
}
|
||||
else if (byte <= 0xEF)
|
||||
{
|
||||
continuation_bytes = 2;
|
||||
valid_second = (byte != 0xE0 || second >= 0xA0) && (byte != 0xED || second <= 0x9F);
|
||||
}
|
||||
else
|
||||
{
|
||||
continuation_bytes = 3;
|
||||
valid_second = byte <= 0xF4 && (byte != 0xF0 || second >= 0x90) && (byte != 0xF4 || second <= 0x8F);
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_UNLIKELY(!valid_second))
|
||||
{
|
||||
return bon8_error("invalid UTF-8 byte", "string");
|
||||
}
|
||||
|
||||
result.push_back(static_cast<typename string_t::value_type>(byte));
|
||||
result.push_back(static_cast<typename string_t::value_type>(second));
|
||||
|
||||
for (int i = 1; i < continuation_bytes; ++i)
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits<char_type>::eof()))
|
||||
{
|
||||
return unexpect_eof(input_format_t::bon8, "string");
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current)))
|
||||
{
|
||||
return bon8_error("invalid UTF-8 byte", "string");
|
||||
}
|
||||
result.push_back(static_cast<typename string_t::value_type>(current));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
///////////////////////
|
||||
// Utility functions //
|
||||
///////////////////////
|
||||
@@ -3497,6 +4101,10 @@ class binary_reader
|
||||
error_msg += "BJData";
|
||||
break;
|
||||
|
||||
case input_format_t::bon8:
|
||||
error_msg += "BON8";
|
||||
break;
|
||||
|
||||
case input_format_t::json: // LCOV_EXCL_LINE
|
||||
default: // LCOV_EXCL_LINE
|
||||
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
|
||||
@@ -3529,6 +4137,11 @@ class binary_reader
|
||||
/// the containers that have been opened and not closed yet; see @ref container_frame
|
||||
std::vector<container_frame> container_stack{};
|
||||
|
||||
/// BON8: bytes read past the end of a string, returned again by @ref get_bon8
|
||||
std::array<char_int_type, 2> bon8_pushback{{}};
|
||||
/// BON8: number of bytes in @ref bon8_pushback
|
||||
std::size_t bon8_pushback_size = 0;
|
||||
|
||||
// excluded markers in bjdata optimized type
|
||||
#define JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_ \
|
||||
make_array<char_int_type>('F', 'H', 'N', 'S', 'T', 'Z', '[', '{')
|
||||
|
||||
@@ -34,7 +34,7 @@ namespace detail
|
||||
{
|
||||
|
||||
/// the supported input formats
|
||||
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata };
|
||||
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata, bon8 };
|
||||
|
||||
////////////////////
|
||||
// input adapters //
|
||||
|
||||
@@ -201,6 +201,43 @@ inline std::size_t validate_one_utf8(const unsigned char* data, std::size_t avai
|
||||
return 0; // invalid, incomplete, or must be diagnosed by the byte path
|
||||
}
|
||||
|
||||
// Return the length of the longest prefix of [data, data+n) that consists of
|
||||
// ASCII characters and complete well-formed UTF-8 sequences; n if all of it is
|
||||
// valid UTF-8. Unlike scalar_string_bulk_run(), quotes, escapes, and control
|
||||
// characters are ordinary characters here. ASCII is skipped 8 bytes at a time.
|
||||
inline std::size_t valid_utf8_prefix(const unsigned char* data, std::size_t n) noexcept
|
||||
{
|
||||
constexpr std::uint64_t high = 0x8080808080808080ull;
|
||||
std::size_t pos = 0;
|
||||
while (pos < n)
|
||||
{
|
||||
if (pos + 8 <= n)
|
||||
{
|
||||
std::uint64_t word = 0;
|
||||
std::memcpy(&word, data + pos, sizeof(word));
|
||||
if ((word & high) == 0)
|
||||
{
|
||||
pos += 8;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if (data[pos] < 0x80u)
|
||||
{
|
||||
++pos;
|
||||
continue;
|
||||
}
|
||||
|
||||
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
|
||||
if (seq == 0)
|
||||
{
|
||||
break; // ill-formed or truncated
|
||||
}
|
||||
pos += seq;
|
||||
}
|
||||
return pos;
|
||||
}
|
||||
|
||||
// Scalar (C++11) computation of the bulk run length: the number of leading
|
||||
// bytes in [data, data+n) that are ordinary ASCII or complete well-formed UTF-8
|
||||
// sequences, stopping before the first byte that needs individual handling (the
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
#endif
|
||||
|
||||
#include <nlohmann/detail/input/binary_reader.hpp>
|
||||
#include <nlohmann/detail/input/string_scan.hpp>
|
||||
#include <nlohmann/detail/macro_scope.hpp>
|
||||
#include <nlohmann/detail/output/output_adapters.hpp>
|
||||
#include <nlohmann/detail/string_concat.hpp>
|
||||
@@ -76,7 +77,7 @@ std::size_t binary_reserve_hint(const BasicJsonType& j)
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief serialization to CBOR and MessagePack values
|
||||
@brief serialization to BJData, BON8, BSON, CBOR, MessagePack, and UBJSON values
|
||||
*/
|
||||
template<typename BasicJsonType, typename CharType, typename OutputSinkType = output_adapter_sink<CharType>>
|
||||
class binary_writer
|
||||
@@ -873,6 +874,21 @@ class binary_writer
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@param[in] j JSON value to serialize
|
||||
*/
|
||||
void write_bon8(const BasicJsonType& j)
|
||||
{
|
||||
bool string_open = false;
|
||||
write_bon8_value(j, string_open);
|
||||
|
||||
// the last string of a message must be terminated
|
||||
if (string_open)
|
||||
{
|
||||
oa.write_character(to_char_type(0xFF));
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
//////////
|
||||
// BSON //
|
||||
@@ -1431,6 +1447,28 @@ class binary_writer
|
||||
return to_char_type(0xCB); // float 64
|
||||
}
|
||||
|
||||
/// @return the BON8 type marker for binary32 (float) or binary64 (double)
|
||||
template<typename FloatType>
|
||||
static constexpr CharType get_bon8_float_prefix()
|
||||
{
|
||||
return to_char_type(std::is_same<FloatType, float>::value ? 0x8E : 0x8F);
|
||||
}
|
||||
|
||||
/// @return the type marker for a FloatType value in @a format (CBOR, MessagePack, or BON8)
|
||||
template<typename FloatType>
|
||||
static CharType get_compact_float_prefix(const detail::input_format_t format)
|
||||
{
|
||||
if (format == detail::input_format_t::cbor)
|
||||
{
|
||||
return get_cbor_float_prefix(FloatType{});
|
||||
}
|
||||
if (format == detail::input_format_t::bon8)
|
||||
{
|
||||
return get_bon8_float_prefix<FloatType>();
|
||||
}
|
||||
return get_msgpack_float_prefix(FloatType{});
|
||||
}
|
||||
|
||||
////////////
|
||||
// UBJSON //
|
||||
////////////
|
||||
@@ -2050,6 +2088,322 @@ class binary_writer
|
||||
return false;
|
||||
}
|
||||
|
||||
//////////
|
||||
// BON8 //
|
||||
//////////
|
||||
|
||||
/*!
|
||||
@brief write a BON8 value
|
||||
|
||||
A string is written without length or terminator: it ends at the first
|
||||
byte that cannot continue it, which is the first byte of any non-string
|
||||
value and of the end-of-container marker 0xFE. It only needs an explicit
|
||||
end-of-string marker (0xFF) when it is empty, when another string follows,
|
||||
or when it is the last thing in the message.
|
||||
|
||||
@param[in] j JSON value to serialize
|
||||
@param[in,out] string_open whether the output ends with a non-empty
|
||||
string that has not been terminated with 0xFF
|
||||
*/
|
||||
void write_bon8_value(const BasicJsonType& j, bool& string_open)
|
||||
{
|
||||
switch (j.type())
|
||||
{
|
||||
case value_t::null:
|
||||
{
|
||||
write_bon8_marker(0xFA, string_open);
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::boolean:
|
||||
{
|
||||
write_bon8_marker(j.m_data.m_value.boolean ? 0xF9 : 0xF8, string_open);
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::number_unsigned:
|
||||
{
|
||||
if (j.m_data.m_value.number_unsigned > static_cast<typename BasicJsonType::number_unsigned_t>((std::numeric_limits<std::int64_t>::max)()))
|
||||
{
|
||||
JSON_THROW(out_of_range::create(407, concat("integer number ", std::to_string(j.m_data.m_value.number_unsigned), " cannot be represented by BON8 as it does not fit int64"), &j));
|
||||
}
|
||||
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_unsigned));
|
||||
string_open = false;
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::number_integer:
|
||||
{
|
||||
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
|
||||
string_open = false;
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::number_float:
|
||||
{
|
||||
write_bon8_float(j.m_data.m_value.number_float);
|
||||
string_open = false;
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::string:
|
||||
{
|
||||
write_bon8_string(*j.m_data.m_value.string, string_open, j);
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::array:
|
||||
{
|
||||
const auto N = j.m_data.m_value.array->size();
|
||||
// 0x80..0x84: array with 0..4 elements; 0x85: array ended by 0xFE
|
||||
write_bon8_marker(static_cast<std::uint8_t>(N <= 4 ? 0x80 + N : 0x85), string_open);
|
||||
|
||||
for (const auto& el : *j.m_data.m_value.array)
|
||||
{
|
||||
write_bon8_value(el, string_open);
|
||||
}
|
||||
|
||||
if (N > 4)
|
||||
{
|
||||
write_bon8_marker(0xFE, string_open);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::object:
|
||||
{
|
||||
const auto N = j.m_data.m_value.object->size();
|
||||
// 0x86..0x8A: object with 0..4 members; 0x8B: object ended by 0xFE
|
||||
write_bon8_marker(static_cast<std::uint8_t>(N <= 4 ? 0x86 + N : 0x8B), string_open);
|
||||
|
||||
for (const auto& el : *j.m_data.m_value.object)
|
||||
{
|
||||
write_bon8_string(el.first, string_open, j);
|
||||
write_bon8_value(el.second, string_open);
|
||||
}
|
||||
|
||||
if (N > 4)
|
||||
{
|
||||
write_bon8_marker(0xFE, string_open);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::binary:
|
||||
{
|
||||
// BON8 has no binary type: write the bytes as an array of
|
||||
// integers, like UBJSON and BJData do
|
||||
const auto N = j.m_data.m_value.binary->size();
|
||||
write_bon8_marker(static_cast<std::uint8_t>(N <= 4 ? 0x80 + N : 0x85), string_open);
|
||||
|
||||
for (std::size_t i = 0; i < N; ++i)
|
||||
{
|
||||
// the cast is needed for binary types whose value type
|
||||
// is not an integer (e.g., std::byte)
|
||||
write_bon8_integer(static_cast<std::uint8_t>(j.m_data.m_value.binary->data()[i]));
|
||||
}
|
||||
|
||||
if (N > 4)
|
||||
{
|
||||
write_bon8_marker(0xFE, string_open);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case value_t::discarded:
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief write a single byte that is not part of a string
|
||||
|
||||
@param[in] marker the byte to write
|
||||
@param[out] string_open set to false, because the output no longer ends
|
||||
with a string; see @ref write_bon8_value
|
||||
*/
|
||||
void write_bon8_marker(const std::uint8_t marker, bool& string_open)
|
||||
{
|
||||
oa.write_character(to_char_type(marker));
|
||||
string_open = false;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief write a string
|
||||
|
||||
@param[in] s the string to write
|
||||
@param[in,out] string_open see @ref write_bon8_value
|
||||
@param[in] context the value the string belongs to (for diagnostics)
|
||||
|
||||
@throw type_error.316 if @a s is not valid UTF-8, because the end of a
|
||||
string is determined from its encoding
|
||||
*/
|
||||
void write_bon8_string(const string_t& s, bool& string_open, const BasicJsonType& context)
|
||||
{
|
||||
check_bon8_utf8(s, context);
|
||||
|
||||
// a string that follows another string terminates it
|
||||
if (string_open)
|
||||
{
|
||||
oa.write_character(to_char_type(0xFF));
|
||||
}
|
||||
|
||||
if (s.empty())
|
||||
{
|
||||
// the empty string is just the end-of-string marker
|
||||
oa.write_character(to_char_type(0xFF));
|
||||
string_open = false;
|
||||
}
|
||||
else
|
||||
{
|
||||
oa.write_characters(reinterpret_cast<const CharType*>(s.data()), s.size());
|
||||
string_open = true;
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief check that a string is valid UTF-8 (RFC 3629)
|
||||
|
||||
@param[in] s the string to check
|
||||
@param[in] context the value the string belongs to (for diagnostics)
|
||||
|
||||
@throw type_error.316 if @a s is not valid UTF-8; the message names the
|
||||
first byte of the first invalid or incomplete sequence
|
||||
*/
|
||||
static void check_bon8_utf8(const string_t& s, const BasicJsonType& context)
|
||||
{
|
||||
static_cast<void>(context); // only used when exceptions are enabled
|
||||
const auto* data = reinterpret_cast<const unsigned char*>(s.data());
|
||||
const std::size_t valid = valid_utf8_prefix(data, s.size());
|
||||
if (JSON_HEDLEY_UNLIKELY(valid != s.size()))
|
||||
{
|
||||
JSON_THROW(type_error::create(316, concat("invalid UTF-8 byte at index ", std::to_string(valid), ": 0x", hex_byte(data[valid])), &context));
|
||||
}
|
||||
}
|
||||
|
||||
/// @return a byte as two uppercase hexadecimal digits
|
||||
static std::string hex_byte(const std::uint8_t byte)
|
||||
{
|
||||
std::string result = "00";
|
||||
constexpr const char* nibble_to_hex = "0123456789ABCDEF";
|
||||
result[0] = nibble_to_hex[byte / 16];
|
||||
result[1] = nibble_to_hex[byte % 16];
|
||||
return result;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief write an integer in the shortest encoding
|
||||
|
||||
Integers from -10 to 39 take one byte. Up to -33818506 and 67637031, an
|
||||
integer takes 2 to 4 bytes that begin with a UTF-8 lead byte (0xC2..0xF7)
|
||||
followed by a byte that is not a continuation byte: 0x00..0x7F for
|
||||
positive and 0xC0..0xFF for negative integers. Each range starts where the
|
||||
shorter one ends. Larger integers are written as int32 (0x8C) or int64
|
||||
(0x8D) in big-endian byte order.
|
||||
|
||||
@param[in] value the integer to write
|
||||
*/
|
||||
void write_bon8_integer(std::int64_t value)
|
||||
{
|
||||
if (value < (std::numeric_limits<std::int32_t>::min)() || value > (std::numeric_limits<std::int32_t>::max)())
|
||||
{
|
||||
oa.write_character(to_char_type(0x8D));
|
||||
write_number(value);
|
||||
}
|
||||
else if (value < -33818506 || value > 67637031)
|
||||
{
|
||||
oa.write_character(to_char_type(0x8C));
|
||||
write_number(static_cast<std::int32_t>(value));
|
||||
}
|
||||
else if (value <= -264075)
|
||||
{
|
||||
value = -(value + 264075);
|
||||
write_bon8_bytes(0xF0 + ((value >> 22) & 0x07), 0xC0 + ((value >> 16) & 0x3F), value >> 8, value);
|
||||
}
|
||||
else if (value <= -1931)
|
||||
{
|
||||
value = -(value + 1931);
|
||||
write_bon8_bytes(0xE0 + ((value >> 14) & 0x0F), 0xC0 + ((value >> 8) & 0x3F), value);
|
||||
}
|
||||
else if (value <= -11)
|
||||
{
|
||||
value = -(value + 11);
|
||||
write_bon8_bytes(0xC2 + ((value >> 6) & 0x1F), 0xC0 + (value & 0x3F));
|
||||
}
|
||||
else if (value <= -1)
|
||||
{
|
||||
write_bon8_bytes(0xB8 - (value + 1));
|
||||
}
|
||||
else if (value <= 39)
|
||||
{
|
||||
write_bon8_bytes(0x90 + value);
|
||||
}
|
||||
else if (value <= 3879)
|
||||
{
|
||||
value -= 40;
|
||||
write_bon8_bytes(0xC2 + ((value >> 7) & 0x1F), value & 0x7F);
|
||||
}
|
||||
else if (value <= 528167)
|
||||
{
|
||||
value -= 3880;
|
||||
write_bon8_bytes(0xE0 + ((value >> 15) & 0x0F), (value >> 8) & 0x7F, value);
|
||||
}
|
||||
else
|
||||
{
|
||||
value -= 528168;
|
||||
write_bon8_bytes(0xF0 + ((value >> 23) & 0x07), (value >> 16) & 0x7F, value >> 8, value);
|
||||
}
|
||||
}
|
||||
|
||||
/// write the low byte of each argument
|
||||
template<typename... Bytes>
|
||||
void write_bon8_bytes(const Bytes... bytes)
|
||||
{
|
||||
const std::array<CharType, sizeof...(Bytes)> buffer{{to_char_type(static_cast<std::uint8_t>(bytes & 0xFF))...}};
|
||||
oa.write_characters(buffer.data(), buffer.size());
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief write a floating-point number
|
||||
|
||||
-1.0, +0.0, and 1.0 take one byte. Other numbers are written as binary32
|
||||
(0x8E) if that loses no precision, and as binary64 (0x8F) otherwise; -0.0,
|
||||
infinities, and NaN are always written as binary32, NaN as 0x7F800001.
|
||||
|
||||
@param[in] n the number to write
|
||||
*/
|
||||
void write_bon8_float(const number_float_t n)
|
||||
{
|
||||
#ifdef __GNUC__
|
||||
JSON_HEDLEY_DIAGNOSTIC_PUSH
|
||||
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal")
|
||||
#endif
|
||||
if (n == static_cast<number_float_t>(-1))
|
||||
{
|
||||
oa.write_character(to_char_type(0xFB));
|
||||
}
|
||||
else if (n == static_cast<number_float_t>(0) && !std::signbit(n))
|
||||
{
|
||||
oa.write_character(to_char_type(0xFC));
|
||||
}
|
||||
else if (n == static_cast<number_float_t>(1))
|
||||
{
|
||||
oa.write_character(to_char_type(0xFD));
|
||||
}
|
||||
else if (std::isnan(n))
|
||||
{
|
||||
write_bon8_bytes(0x8E, 0x7F, 0x80, 0x00, 0x01);
|
||||
}
|
||||
else
|
||||
{
|
||||
write_compact_float(n, detail::input_format_t::bon8);
|
||||
}
|
||||
#ifdef __GNUC__
|
||||
JSON_HEDLEY_DIAGNOSTIC_POP
|
||||
#endif
|
||||
}
|
||||
|
||||
///////////////////////
|
||||
// Utility functions //
|
||||
///////////////////////
|
||||
@@ -2184,16 +2538,12 @@ class binary_writer
|
||||
static_cast<double>(n) <= static_cast<double>((std::numeric_limits<float>::max)()) &&
|
||||
static_cast<double>(static_cast<float>(n)) == static_cast<double>(n))))
|
||||
{
|
||||
oa.write_character(format == detail::input_format_t::cbor
|
||||
? get_cbor_float_prefix(static_cast<float>(n))
|
||||
: get_msgpack_float_prefix(static_cast<float>(n)));
|
||||
oa.write_character(get_compact_float_prefix<float>(format));
|
||||
write_number(static_cast<float>(n));
|
||||
}
|
||||
else
|
||||
{
|
||||
oa.write_character(format == detail::input_format_t::cbor
|
||||
? get_cbor_float_prefix(n)
|
||||
: get_msgpack_float_prefix(n));
|
||||
oa.write_character(get_compact_float_prefix<number_float_t>(format));
|
||||
write_number(n);
|
||||
}
|
||||
#ifdef __GNUC__
|
||||
|
||||
@@ -5290,6 +5290,30 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
|
||||
binary_writer<char>(o).write_bson(j);
|
||||
}
|
||||
|
||||
/// @brief create a BON8 serialization of a given JSON value
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
|
||||
static std::vector<std::uint8_t> to_bon8(const basic_json& j)
|
||||
{
|
||||
std::vector<std::uint8_t> result;
|
||||
result.reserve(detail::binary_reserve_hint(j));
|
||||
vector_writer(result).write_bon8(j);
|
||||
return result;
|
||||
}
|
||||
|
||||
/// @brief create a BON8 serialization of a given JSON value
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
|
||||
static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o)
|
||||
{
|
||||
binary_writer<std::uint8_t>(o).write_bon8(j);
|
||||
}
|
||||
|
||||
/// @brief create a BON8 serialization of a given JSON value
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
|
||||
static void to_bon8(const basic_json& j, detail::output_adapter<char> o)
|
||||
{
|
||||
binary_writer<char>(o).write_bon8(j);
|
||||
}
|
||||
|
||||
/// @brief create a JSON value from an input in CBOR format
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/from_cbor/
|
||||
template<typename InputType>
|
||||
@@ -5523,6 +5547,43 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
|
||||
return result;
|
||||
}
|
||||
|
||||
/// @brief create a JSON value from an input in BON8 format
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/from_bon8/
|
||||
template<typename InputType>
|
||||
JSON_HEDLEY_WARN_UNUSED_RESULT
|
||||
static basic_json from_bon8(InputType&& i,
|
||||
const bool strict = true,
|
||||
const bool allow_exceptions = true)
|
||||
{
|
||||
basic_json result;
|
||||
auto ia = detail::input_adapter(std::forward<InputType>(i));
|
||||
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
|
||||
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bon8).sax_parse(input_format_t::bon8, &sdp, strict)) // cppcheck-suppress[accessMoved]
|
||||
{
|
||||
result = value_t::discarded;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/// @brief create a JSON value from an input in BON8 format (iterator pair, or iterator+sentinel pair for C++20 ranges support)
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/from_bon8/
|
||||
template<typename IteratorType, typename SentinelType = IteratorType,
|
||||
detail::enable_if_t<detail::can_compare_ne<IteratorType, SentinelType>::value, int> = 0>
|
||||
JSON_HEDLEY_WARN_UNUSED_RESULT
|
||||
static basic_json from_bon8(IteratorType first, SentinelType last,
|
||||
const bool strict = true,
|
||||
const bool allow_exceptions = true)
|
||||
{
|
||||
basic_json result;
|
||||
auto ia = detail::input_adapter(std::move(first), std::move(last));
|
||||
detail::json_sax_dom_parser<basic_json, decltype(ia)> sdp(result, allow_exceptions);
|
||||
if (!binary_reader<decltype(ia)>(std::move(ia), input_format_t::bon8).sax_parse(input_format_t::bon8, &sdp, strict)) // cppcheck-suppress[accessMoved]
|
||||
{
|
||||
result = value_t::discarded;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/// @brief create a JSON value from an input in BSON format
|
||||
/// @sa https://json.nlohmann.me/api/basic_json/from_bson/
|
||||
template<typename InputType>
|
||||
|
||||
Reference in New Issue
Block a user