amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-02 12:40:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
e5a89d671f3627dbe2dac0dd481c01ea037c21e2
json/tests/src/unit-bjdata.cpp
T
Niels Lohmann e5a89d671f Fix lint debt: enum-macro NOLINTs, doctest as SYSTEM, no-op analyzer (#5737) 
* Drop stale LCOV_EXCL_LINE from the json_pointer out_of_range.410 throw

The comment said the size_type overflow check in array_index() is only
triggered on special platforms like 32-bit, and the throw was excluded
from coverage. On 64-bit platforms the check is true for SIZE_MAX
itself, and unit-json_pointer.cpp has asserted that case four times
since #5395, so the line is executed in the coverage job. Reword the
comment and remove the exclusion marker so the coverage report notices
if the tests stop reaching it.

Part of #5725

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Name all three C-array check aliases in the enum-macro NOLINTs

NLOHMANN_JSON_SERIALIZE_ENUM(_STRICT) suppressed the c-array warning
under modernize-avoid-c-arrays only, but clang-tidy emits the same
diagnostic under the aliases cppcoreguidelines-avoid-c-arrays and
hicpp-avoid-c-arrays too. Any user running those checks got a false
positive at every macro expansion, and our own tests needed a local
NOLINT at each call site to work around it.

Name all three aliases in the four macro comments instead, and drop
the now-redundant c-array names from the five test call-site NOLINTs.
Comment-only change; behavior, the public API, and the ABI do not
change. Ran make amalgamate.

Part of #5725

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Include doctest as a SYSTEM directory instead of disabling warnings for all tests

test_main added -Wno-deprecated and -Wno-float-equal as PUBLIC compile
options for every non-MSVC compiler, so they were applied to every
translation unit, library headers included, and silenced the CI
warnings meant to check the library's own -Wfloat-equal pragmas. The
only code that actually needed the suppression was the vendored
doctest.h, which was included as a normal (non-SYSTEM) directory.

Include thirdparty/doctest as SYSTEM for test_main, matching what
tests/abi/CMakeLists.txt already does, and drop the two suppressions
from both targets. Verified locally that unit-comparison,
unit-conversions and unit-constructor1 compile clean with
-Werror -Weverything and doctest as -isystem, and that CMake still
configures with JSON_BuildTests=ON.

Part of #5725

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remove the no-op ci_clang_analyze target

ci_clang_analyze configured the build with the real compiler and only
then wrapped ninja with scan-build. scan-build intercepts compiles by
overriding CC/CXX, but build.ninja already had the compiler path baked
in from the configure step, so every run bypassed the analyzer: CI
logs show "No bugs found" after a normal build, never an analysis.
The job also used Debian's frozen clang-tools-14 rather than the
image's own clang, and CLANG_ANALYZER_CHECKS still named three
valist.* checkers that current clang merged into security.VAList.

ci_clang_tidy already runs every clang-analyzer-* check (via
.clang-tidy's "Checks: '*'") with warnings as errors, so nothing is
lost by removing the dead job. Delete ci_clang_analyze,
CLANG_ANALYZER_CHECKS and the SCAN_BUILD_TOOL lookup from
cmake/ci.cmake, drop it from the ubuntu.yml ci_static_analysis_clang
matrix, and drop the now-unused clang-tools apt package (iwyu stays
for ci_single_binaries). Reword quality_assurance.md and
assurance_case.md, which described the dead job as a working control,
to say the Clang Static Analyzer checks run through clang-tidy.

Verified that `cmake -DJSON_CI=ON` still configures cleanly and that
ci_clang_analyze no longer appears in the generated build or in any
CMake/workflow file.

Part of #5725

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Re-enable portability-template-virtual-member-function; remove redundant forwards

.clang-tidy disabled three checks "to get the CI going" (#4489,
2024-11-13): portability-template-virtual-member-function,
bugprone-use-after-move and its alias hicpp-invalid-access-moved.

portability-template-virtual-member-function only flagged
output_stream_adapter::write_character/write_characters; annotate
both with NOLINT and re-enable the check.

bugprone-use-after-move flagged several double forwards that have no
effect at runtime:
- from_json.hpp calls std::forward<BasicJsonType>(j).at(Idx) inside
  pack expansions; at() has no ref-qualified overloads and always
  returns an lvalue reference, so the forward is a no-op. Replace with
  plain j.at(Idx) in all four places.
- the move constructor forwards the whole object to its base class
  and then reads other's members. That is item 9 of #5724 (together
  with its cppcheck suppressions) and is left to that change.
- input_adapters.hpp forwards the container twice on purpose, so the
  begin/end iterator types match adapter_type; annotate with NOLINT
  and a comment instead of changing behavior.

The check still flags the move constructor (see above) and two sites
in at(KeyType&&) (both overloads, json.hpp, in the throw's
string_t(std::forward<KeyType>(key)) after
find(std::forward<KeyType>(key))). Open PR #5689 rewrites that hunk,
so bugprone-use-after-move (and hicpp-invalid-access-moved)
stay disabled for now, with a comment explaining why; re-enable them
once #5689 and the #5724 move-constructor change have landed.

Also resolve the portability-avoid-pragma-once TODO: single_include
never has #pragma once (amalgamate.py strips it) and every supported
compiler accepts it in include/, so keep it disabled with an
explanatory comment instead of a TODO. Fix the stale "json.hpp,
around line 1265" comment in unit-class_parser.cpp, which now points
at the move constructor's actual line.

Behavior, the public API and the ABI do not change. Verified with
clang-tidy 22.1.8 that portability-template-virtual-member-function
now reports nothing, that bugprone-use-after-move/
hicpp-invalid-access-moved report only the known at(KeyType&&) and
move-constructor sites, and that unit-custom-base-class, unit-constructor1,
unit-conversions, unit-element_access2, unit-class_parser and
unit-diagnostic-positions (JSON_DIAGNOSTIC_POSITIONS=1) compile
under ASan/UBSan and pass with the same assertion counts as before.
Ran make amalgamate.

Part of #5725

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix stale and malformed NOLINT comments

json_sax.hpp named "-warnings-as-errors" in the NOLINT list on the two
JSON_ASSERT(false) lines; that is the suffix clang-tidy appends to a
diagnostic tag under WarningsAsErrors, not a check name, and every
other JSON_ASSERT(false) omits it.

unit-capacity.cpp carried 30 "// NOLINT(misc-const-correctness)"
comments on "json j = ...;" declarations that are all used with
non-const members afterwards, so the check has nothing to report
there.

unit-constructor2.cpp used a blanket "// NOLINT: access after move is
OK here" on a use-after-move that hides every check on the line;
naming bugprone-use-after-move and hicpp-invalid-access-moved keeps
the intent once those checks are re-enabled (#5724).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

#5725 item 10

* Remove stale .clang-tidy entries

-google-runtime-references disabled a check that neither clang-tidy
22.1.8 nor 23.1.2 lists under --list-checks -checks='*'; it was
removed upstream. The commented-out HeaderFilterRegex line has been
unused since the active HeaderFilterRegex was introduced in #2561
(2021).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

#5725 item 11

* Remove the GCC C++20 -Wignored-attributes pragma in json.hpp

The pragma (added in #5164) claimed to work around the C++ modules
redefinition errors of #5103, but #5103 is about hard errors (e.g.
"redefinition of std::__is_constant_evaluated()", conflicting
std::integral_constant) that ignoring a warning cannot suppress; they
are traced to GCC PR 124430 and reproduce with <map> or <string>
instead of json.hpp too. A GCC 16.2 -std=gnu++20 -fmodules build
following #5103's repro steps still fails with the pragma in place,
and a build of all test TUs with GCC_CXXFLAGS (which enable
-Wignored-attributes) and the pragma removed produces no such
warning. The block only hid a warning class from GCC C++20 users
while suggesting #5103 was handled.

Overlaps #5610, whose hunks touch the closing half of this pragma to
insert the json_literals.hpp include.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

#5725 item 9

* Fix stale doxygen comments hidden by the -Wdocumentation pragma

macro_scope.hpp ignores -Wdocumentation and -Wdocumentation-unknown-command
for the whole library, which also hides genuine documentation mistakes:

- detail::unescape() documented "@return unescaped string" but returns
  void and unescapes its argument in place; reworded to
  "@param[in,out] s string to unescape in place" and dropped the
  bogus @return.
- basic_json::get()'s copy-conversion overload wrote "converted to
  @tparam ValueType" inside @return, which Doxygen and Clang parse as
  a second, malformed @tparam; changed to "@a ValueType", matching the
  two other get() overloads a few lines above that already use it.

This narrows the gap the -Wdocumentation pragma needs to cover; fully
replacing the Doxygen-only commands it also hides (item 2c) is left
for after #5267.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

#5725 item 2

* Fix -Wextra-semi-stmt at its actual source, not assert()

clang_flags.cmake blamed the global -Wno-extra-semi-stmt on assert(),
but assert() expands to an expression under glibc and libc++ and does
not trigger this warning. unit-assert_macro.cpp overrides JSON_ASSERT
with "{if (!(x)) ++assert_counter; }", a bare block followed by a
semicolon at every JSON_ASSERT(...) call site in the library; that
was the actual source of 151 of the 208 -Wextra-semi-stmt sites found
in a Clang 22 -Weverything sweep of the test suite with the flag
removed. Switched to the standard do/while(false) macro idiom, which
does not expand to a statement-plus-semicolon, and corrected the
comment to name the remaining source instead: vendored Doctest's
CAPTURE(x) shim, which already ends in a semicolon.

Verified with clang++ -Wextra-semi-stmt (plus the file's other CI
ignores) that unit-assert_macro.cpp now compiles without any
-Wextra-semi-stmt diagnostic.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

#5725 item 8 (step 1 of 2; step 2 covers the CAPTURE() call sites)

* Drop the redundant semicolon from CAPTURE() call sites; remove -Wno-extra-semi-stmt

doctest_compatibility.h defines CAPTURE(x) as DOCTEST_CAPTURE(x); (with
a trailing semicolon baked into the macro), specifically so call sites
do not need to add one themselves; most of the ~267 call sites already
follow that convention. The remaining 64 call sites across 20 files
wrote "CAPTURE(x);" anyway, turning into a statement plus an empty
statement and triggering -Wextra-semi-stmt. Dropped the redundant
semicolon at each of those sites.

With item 6 having already made vendored Doctest a SYSTEM include, and
this the last known source of -Wextra-semi-stmt findings, removed the
flag from clang_flags.cmake entirely.

Verified with clang++ -Wextra-semi-stmt (plus the file's other CI
ignores) that all 20 touched files, plus a file with no CAPTURE() use
(unit-json_pointer.cpp), compile without any -Wextra-semi-stmt
diagnostic.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

#5725 item 8 (step 2 of 2)

* Switch ci_static_analysis_clang off the frozen LLVM 22 dev image

ubuntu.yml pinned the clang-tidy/clang-tidy-sanitizer/single-binaries
job to silkeh/clang:dev, a tag last pushed 2026-02-18 that reports
"clang version 22.0.0 (...+20251015...)", a pre-release snapshot from
before the LLVM 22 release; the maintainer now updates dev-unstable,
22, and latest instead. Switched to silkeh/clang:22, matching the
other clang jobs on :latest.

Verified with clang-tidy 22.1.8 (the image's actual version) against
this repository's .clang-tidy and library headers what the release
image newly reports compared to :dev:

- readability-redundant-typename fires at ~250 sites across the
  _cpp20-relevant conversion/to_chars headers; the library targets
  C++11 and keeps the typenames, so the check is disabled in
  .clang-tidy, matching how the file already handles checks that
  don't fit a C++11 codebase.
- misc-anonymous-namespace-in-header fires on the two anonymous
  namespaces in from_json.hpp and to_json.hpp; added the alias to
  their existing NOLINT (cert-dcl59-cpp, fuchsia-header-anon-namespaces,
  google-build-namespaces).
- bugprone-std-namespace-modification fires on every addition to
  namespace std: the std::hash, std::formatter and std::swap
  overloads in json.hpp, and the std::tuple_size/std::tuple_element
  specializations in iteration_proxy.hpp (this last file is not named
  in #5725's item 5, found by actually running clang-tidy 22.1.8
  against the current tree). All six are legal, deliberate additions
  to namespace std (explicit/partial specializations of std types, or
  the pre-C++20 std::swap overload); annotated each with the check
  name next to its existing cert-dcl58-cpp NOLINT.
- modernize-avoid-c-style-cast reported nothing new.

Also added clang++-22/21, clang-tidy-22/21, g++-16 and gcov-16 to the
find_program search lists in ci.cmake so a local "maximal warnings"
configure prefers the current toolchain version over an older one on
PATH.

#5725 item 5

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate cmake/gcc_flags.cmake for GCC 16.2.0

GCC_CXXFLAGS was generated for GCC 15.1.0, but ci_test_gcc and
ci_test_gcc_cxx{11..26} now run in gcc:latest, currently GCC 16.2.0,
so the "maximal warnings" job was missing warnings introduced since
15.1.0 while carrying entries GCC 16 treats as duplicates or no-ops.

Regenerated with https://github.com/nlohmann/gcc_flags (patched
locally to not crash on an option whose "-x c++ <opt> -" probe fails
before it reads stdin, e.g. -Wabi=; the tool otherwise raises
BrokenPipeError instead of recording the option as an error) run
against g++ 16.2.0 in the official gcc:16 Docker image, keeping the
documented -Wno-* exclusions and the same alphabetical placement
scheme as before.

Also added three GCC 16 warnings the generator cannot discover on its
own because it only probes value ranges/lists it finds in the -Q
option name itself, not in the enum choices --help=warnings documents
separately:

- -Wbidi-chars=any, -Wleading-whitespace=spaces: manually verified
  these compile cleanly with g++ 16.2.0.
- -Wstrict-flex-arrays: deliberately NOT added, unlike the other two.
  Without -fstrict-flex-arrays (which the library does not enable, as
  it would change codegen for flexible array members), GCC prints
  "'-Wstrict-flex-arrays' is ignored when '-fstrict-flex-arrays' is
  not present" on every translation unit, and under our -Werror that
  note itself aborts the build. This differs from the harmless
  no-op warnings already kept in the file (-Whsa, -Wsynth,
  -Wunreachable-code, -Wunsafe-loop-optimizations), which emit
  nothing; #5725 item 7 named -Wstrict-flex-arrays as one of the
  flags GCC 16 adds, but did not anticipate this failure mode.

Verified: compiled the library header and a representative set of
test translation units (including ones touched by items 1, 3, 8, 9,
10 of this issue) with the regenerated GCC_CXXFLAGS plus -Werror
under g++ 16.2.0 at -std=c++11 through -std=c++26, with zero warnings;
ran the full local test suite (129/129 passing, unrelated to this
compiler) as a regression check. CI must still confirm the actual
ci_test_gcc / ci_test_standards_gcc targets end to end, since this was
verified with direct g++ invocations rather than through the CMake/
CXXFLAGS environment-variable plumbing in ci.cmake.

#5725 item 7

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Avoid std::basic_string<CharType> for non-character output_adapter CharType

output_adapter<CharType, StringType> defaulted StringType to
std::basic_string<CharType>, and (with JSON_NO_IO undefined) always
declared a std::basic_ostream<CharType>&-taking constructor. For
CharType with no non-deprecated std::char_traits specialization (only
std::uint8_t is ever used this way, by the binary writers), simply
naming either type - as an unused default template argument, or as an
unused, never-called constructor's parameter type - instantiates
std::char_traits<CharType> merely to name it, which some standard
libraries mark deprecated: with the library-wide -Wdocumentation
pragma (item 2's other half, left for a later commit) temporarily
removed, an Apple clang 21 / libc++ TU calling json::to_cbor(j, vec)
with std::vector<std::uint8_t>& got one -Wdeprecated-declarations
warning per binary writer at the old output_adapters.hpp:193.

Replaced the eager std::basic_string<CharType> / std::basic_ostream
<CharType> defaults with a bool-tagged partial specialization (not
std::conditional, which requires naming both branches' types up
front regardless of which is selected, reproducing the same warning)
that only ever names std::basic_string<CharType> / std::basic_ostream
<CharType> when CharType is actually one of char, wchar_t, char16_t,
char32_t, or (with __cpp_lib_char8_t) char8_t. For any other
CharType, output_adapter's StringType and ostream-constructor
parameter fall back to two distinct empty placeholder types, kept
distinct so the two constructor overloads do not collide into a
single redeclaration.

Public API / behavior: passing a std::basic_string<std::uint8_t>& or
std::basic_ostream<std::uint8_t>& directly to a binary writer's
output_adapter now fails to compile instead of compiling with a
deprecation warning; this was neither documented nor tested. All
documented uses (std::vector<CharType>, std::basic_ostream<CharType>
and StringType for character CharType) are unaffected.

Verified with Apple clang 21 / libc++, with the two -Wdocumentation*
"ignored" pragma lines in macro_scope.hpp temporarily removed and
-std=c++11/c++20 plus the project's -Weverything flag set: calling
to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson/to_bon8 on a
std::vector<std::uint8_t> now produces no char_traits<unsigned char>
(or any other) deprecation warning, while the char-based string- and
ostream-adapter paths, and a to_cbor/from_cbor round trip, still
compile and run correctly; also verified with GCC 16.2.0. Ran the
full local test suite, including the binary-format unit tests
(unit-cbor, unit-msgpack, unit-ubjson, unit-bjdata, unit-bson,
unit-bon8, unit-binary_writer_sinks, unit-binary_formats,
unit-custom-binary-type): 129/129 passing.

#5725 item 2 (step a)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remove the library-wide -Wdocumentation pragma; fix what it hid

macro_scope.hpp / macro_unscope.hpp pushed and popped a Clang
diagnostic region over the entire library that ignored -Wdocumentation
and -Wdocumentation-unknown-command. Removed both pragmas and fixed
every finding a full -Wdocumentation (which implies
-Wdocumentation-unknown-command and -Wdocumentation-deprecated-sync)
build reports, so the library now compiles clean under Clang's
documentation checks without a blanket suppression. Overlaps #5267,
which is still open and edits a nearby doc block (json.hpp's
get()/get_impl() @return, already fixed in the item 2 step (b) commit
of this branch); this commit does not touch that block again.

Unknown Doxygen alias commands (Doxyfile removed in #3071, so these
were never rendered by anything) rewritten as plain prose, keeping the
same information:
- @requirement REQ-JSON-01 / REQ-JSON-02 (iter_impl.hpp,
  json_reverse_iterator.hpp): now "This class satisfies the following
  concept requirements (REQ-JSON-0N):".
- @liveexample{prose,example-id} (three sites in json.hpp): kept the
  prose, dropped the command wrapper and the trailing example-id
  (docs/mkdocs/docs/examples/*.cpp still exist and are used directly
  by the rendered docs, not through this in-header alias) and
  unescaped the "\," commas that were only needed for the old alias's
  comma-separated argument syntax.
- @complexity X (json.hpp x4, json_pointer.hpp x2, serializer.hpp x1):
  now "Complexity: X".

Backslash sequences Clang's comment lexer tried to parse as commands,
escaped to render as literal backslashes:
- lexer.hpp get_codepoint(): two `\u` occurrences.
- binary_reader.hpp get_bson_cstr() / get_bson_cstr_bulk(): two
  `\x00` occurrences.
- serializer.hpp: three `\uXXXX` occurrences (constructor @param,
  append_codepoint_to_string_buffer() @brief, and the ensure_ascii
  member comment).

One finding remained after all of the above: Clang reports
"declaration is marked with '@deprecated' command but does not have a
deprecation attribute" on the deprecated sax_parse(span_input_adapter&&, ...)
overload, even though JSON_HEDLEY_DEPRECATED_FOR does expand to
__attribute__((deprecated(...))) for Clang. Several isolated
reproductions of this exact declaration shape - doc comment,
template<>, two stacked __attribute__ macros, an overload set sharing
the name - did not reproduce the warning, so this looks like a
Clang comment/declaration-association quirk specific to this overload
inside the much larger basic_json class template, not an actual
documentation defect. Rather than keep the pragma library-wide for one
Clang false positive, added a tightly scoped
-Wdocumentation-deprecated-sync push/pop around just that overload.

Verified with Apple clang 21 and the project's actual -Weverything
flag set (cmake/clang_flags.cmake) on the full header at -std=c++11
and -std=c++20: zero -Wdocumentation* diagnostics. Also compiled
clean with GCC 16.2.0 (the pragmas are already __clang__-gated, so
this only confirms no unrelated breakage). Ran make check-amalgamation
and the full local test suite: 129/129 passing.

#5725 item 2 (step c)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Take the JSON value by const reference in the array and tuple from_json paths

Review feedback on #5737 (gregmarr): once the no-op std::forward calls
are gone, the forwarding references have no purpose. from_json_fn
passes the value as const BasicJsonType&, so these functions were only
ever instantiated with a const lvalue anyway.

The std::array, std::pair and std::tuple overloads of from_json and
their helpers now take const BasicJsonType& and pass j on unchanged.
Because the deduced BasicJsonType is now the plain type, tuple_type and
the static_assert name const BasicJsonType& explicitly, so the
reference checks are unchanged: get<std::tuple<const std::string&>>()
still works, and get<std::tuple<std::string&>>() still fails the same
static_assert. from_json_tuple_get_impl keeps its forwarding reference,
since tuple_type calls it through std::declval.

Behavior, the public API and the ABI do not change. unit-conversions,
unit-constructor1, unit-udt, unit-udt_macro, unit-regression1/2/3,
unit-deserialization, unit-noexcept, unit-items, unit-allocator,
unit-custom-object-type, unit-ordered_json2 and
unit-brace-init-copy-semantics pass at C++11, C++17 and C++20 with
unchanged assertion counts. Ran make amalgamate.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-01 07:37:47 +02:00

4457 lines
228 KiB
C++
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#define JSON_TESTS_PRIVATE
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <algorithm>
#include <climits>
#include <limits>
#include <iostream>
#include <fstream>
#include <set>
#include "make_test_data_available.hpp"
#include "round_trip_corpus.hpp"
#include "test_utils.hpp"
#include "sax_countdown.hpp"
using utils::SaxCountdown;
// trait_test_arg and the "value_in_range_of trait" TEST_CASE_TEMPLATE_DEFINE
// are shared with unit-32bit.cpp
#include "value_in_range_of_test.hpp"
// NOLINTNEXTLINE(bugprone-throwing-static-initialization)
TEST_CASE_TEMPLATE_INVOKE(value_in_range_of_test, \
trait_test_arg<std::int32_t, std::int32_t, true, true>, \
trait_test_arg<std::int32_t, std::uint32_t, true, false>, \
trait_test_arg<std::uint32_t, std::int32_t, false, true>, \
trait_test_arg<std::uint32_t, std::uint32_t, true, true>, \
trait_test_arg<std::int32_t, std::int64_t, false, false>, \
trait_test_arg<std::int32_t, std::uint64_t, true, false>, \
trait_test_arg<std::uint32_t, std::int64_t, false, false>, \
trait_test_arg<std::uint32_t, std::uint64_t, true, false>, \
trait_test_arg<std::int64_t, std::int32_t, true, true>, \
trait_test_arg<std::int64_t, std::uint32_t, true, true>, \
trait_test_arg<std::uint64_t, std::int32_t, false, true>, \
trait_test_arg<std::uint64_t, std::uint32_t, true, true>, \
trait_test_arg<std::int64_t, std::int64_t, true, true>, \
trait_test_arg<std::int64_t, std::uint64_t, true, false>, \
trait_test_arg<std::uint64_t, std::int64_t, false, true>, \
trait_test_arg<std::uint64_t, std::uint64_t, true, true>);
#if SIZE_MAX == 0xffffffff
TEST_CASE_TEMPLATE_INVOKE(value_in_range_of_test, \
trait_test_arg<std::size_t, std::int32_t, false, true>, \
trait_test_arg<std::size_t, std::uint32_t, true, true>, \
trait_test_arg<std::size_t, std::int64_t, false, false>, \
trait_test_arg<std::size_t, std::uint64_t, true, false>);
#else
// NOLINTNEXTLINE(bugprone-throwing-static-initialization)
TEST_CASE_TEMPLATE_INVOKE(value_in_range_of_test, \
trait_test_arg<std::size_t, std::int32_t, false, true>, \
trait_test_arg<std::size_t, std::uint32_t, true, true>, \
trait_test_arg<std::size_t, std::int64_t, false, true>, \
trait_test_arg<std::size_t, std::uint64_t, true, true>);
#endif
TEST_CASE("BJData")
{
SECTION("binary_reader BJData lookup tables")
{
std::vector<std::uint8_t> const data;
auto ia = nlohmann::detail::input_adapter(data);
// NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
nlohmann::detail::binary_reader<json, decltype(ia)> const br{std::move(ia), json::input_format_t::bjdata};
// the excluded optimized-type markers must match binary_writer's
// is_bjdata_excluded_type_marker(), which encodes the same 8 markers
for (const char marker :
{'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z'
})
{
CHECK(br.is_bjd_excluded_optimized_type(marker));
}
for (const char marker :
{'U', 'i', 'u', 'I', 'm', 'l', 'M', 'L', 'd', 'D', 'C', 'B', 'x'
})
{
CHECK(!br.is_bjd_excluded_optimized_type(marker));
}
// every dtype marker must round-trip to its ND-array type name
const std::vector<std::pair<char, std::string>> types
{
{'B', "byte"}, {'C', "char"}, {'D', "double"}, {'I', "int16"},
{'L', "int64"}, {'M', "uint64"}, {'U', "uint8"}, {'d', "single"},
{'i', "int8"}, {'l', "int32"}, {'m', "uint32"}, {'u', "uint16"}
};
for (const auto& type : types)
{
const char* name = br.bjd_type_name(type.first);
REQUIRE(name != nullptr);
CHECK(std::string(name) == type.second);
}
CHECK(br.bjd_type_name('x') == nullptr);
}
SECTION("individual values")
{
SECTION("discarded")
{
// discarded values are not serialized
json const j = json::value_t::discarded;
const auto result = json::to_bjdata(j);
CHECK(result.empty());
}
SECTION("null")
{
json const j = nullptr;
std::vector<uint8_t> const expected = {'Z'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("boolean")
{
SECTION("true")
{
json const j = true;
std::vector<uint8_t> const expected = {'T'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("false")
{
json const j = false;
std::vector<uint8_t> const expected = {'F'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("byte")
{
SECTION("0..255 (uint8)")
{
for (size_t i = 0; i <= 255; ++i)
{
CAPTURE(i)
// create JSON value with integer number (no byte type in JSON)
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create byte vector
std::vector<uint8_t> const value
{
static_cast<uint8_t>('B'),
static_cast<uint8_t>(i),
};
// compare value
CHECK(json::from_bjdata(value) == j);
}
}
}
SECTION("number")
{
SECTION("signed")
{
SECTION("-9223372036854775808..-2147483649 (int64)")
{
std::vector<int64_t> const numbers
{
(std::numeric_limits<int64_t>::min)(),
-1000000000000000000LL,
-100000000000000000LL,
-10000000000000000LL,
-1000000000000000LL,
-100000000000000LL,
-10000000000000LL,
-1000000000000LL,
-100000000000LL,
-10000000000LL,
-2147483649LL,
};
for (const auto i : numbers)
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('L'),
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
static_cast<uint8_t>((i >> 16) & 0xff),
static_cast<uint8_t>((i >> 24) & 0xff),
static_cast<uint8_t>((i >> 32) & 0xff),
static_cast<uint8_t>((i >> 40) & 0xff),
static_cast<uint8_t>((i >> 48) & 0xff),
static_cast<uint8_t>((i >> 56) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 9);
// check individual bytes
CHECK(result[0] == 'L');
int64_t const restored = (static_cast<int64_t>(result[8]) << 070) +
(static_cast<int64_t>(result[7]) << 060) +
(static_cast<int64_t>(result[6]) << 050) +
(static_cast<int64_t>(result[5]) << 040) +
(static_cast<int64_t>(result[4]) << 030) +
(static_cast<int64_t>(result[3]) << 020) +
(static_cast<int64_t>(result[2]) << 010) +
static_cast<int64_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("-2147483648..-32769 (int32)")
{
std::vector<int32_t> const numbers
{
-32769,
-100000,
-1000000,
-10000000,
-100000000,
-1000000000,
-2147483647 - 1, // https://stackoverflow.com/a/29356002/266378
};
for (const auto i : numbers)
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('l'),
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
static_cast<uint8_t>((i >> 16) & 0xff),
static_cast<uint8_t>((i >> 24) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 5);
// check individual bytes
CHECK(result[0] == 'l');
int32_t const restored = (static_cast<int32_t>(result[4]) << 030) +
(static_cast<int32_t>(result[3]) << 020) +
(static_cast<int32_t>(result[2]) << 010) +
static_cast<int32_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("-32768..-129 (int16)")
{
for (int32_t i = -32768; i <= -129; i = utils::next_integer_sample(i, -129, 7))
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('I'),
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 3);
// check individual bytes
CHECK(result[0] == 'I');
auto const restored = static_cast<int16_t>(((result[2] << 8) + result[1]));
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("-9263 (int16)")
{
json const j = -9263;
std::vector<uint8_t> const expected = {'I', 0xd1, 0xdb};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 3);
// check individual bytes
CHECK(result[0] == 'I');
auto const restored = static_cast<int16_t>(((result[2] << 8) + result[1]));
CHECK(restored == -9263);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("-128..-1 (int8)")
{
for (auto i = -128; i <= -1; ++i)
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
'i',
static_cast<uint8_t>(i),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 2);
// check individual bytes
CHECK(result[0] == 'i');
CHECK(static_cast<int8_t>(result[1]) == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("0..127 (int8)")
{
for (size_t i = 0; i <= 127; ++i)
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('i'),
static_cast<uint8_t>(i),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 2);
// check individual bytes
CHECK(result[0] == 'i');
CHECK(result[1] == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("128..255 (uint8)")
{
for (size_t i = 128; i <= 255; ++i)
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('U'),
static_cast<uint8_t>(i),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 2);
// check individual bytes
CHECK(result[0] == 'U');
CHECK(result[1] == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("256..32767 (int16)")
{
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('I'),
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 3);
// check individual bytes
CHECK(result[0] == 'I');
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("32768..65535 (uint16)")
{
for (const uint32_t i :
{
32768u, 55555u, 65535u
})
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
static_cast<uint8_t>('u'),
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 3);
// check individual bytes
CHECK(result[0] == 'u');
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("65536..2147483647 (int32)")
{
for (const uint32_t i :
{
65536u, 77777u, 2147483647u
})
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
'l',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
static_cast<uint8_t>((i >> 16) & 0xff),
static_cast<uint8_t>((i >> 24) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 5);
// check individual bytes
CHECK(result[0] == 'l');
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
(static_cast<uint32_t>(result[3]) << 020) +
(static_cast<uint32_t>(result[2]) << 010) +
static_cast<uint32_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("2147483648..4294967295 (uint32)")
{
for (const uint32_t i :
{
2147483648u, 3333333333u, 4294967295u
})
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
'm',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
static_cast<uint8_t>((i >> 16) & 0xff),
static_cast<uint8_t>((i >> 24) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 5);
// check individual bytes
CHECK(result[0] == 'm');
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
(static_cast<uint32_t>(result[3]) << 020) +
(static_cast<uint32_t>(result[2]) << 010) +
static_cast<uint32_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("4294967296..9223372036854775807 (int64)")
{
std::vector<uint64_t> const v = {4294967296LU, 9223372036854775807LU};
for (const uint64_t i : v)
{
CAPTURE(i)
// create JSON value with integer number
json j = -1;
j.get_ref<json::number_integer_t&>() = static_cast<json::number_integer_t>(i);
// check type
CHECK(j.is_number_integer());
// create expected byte vector
std::vector<uint8_t> const expected
{
'L',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 010) & 0xff),
static_cast<uint8_t>((i >> 020) & 0xff),
static_cast<uint8_t>((i >> 030) & 0xff),
static_cast<uint8_t>((i >> 040) & 0xff),
static_cast<uint8_t>((i >> 050) & 0xff),
static_cast<uint8_t>((i >> 060) & 0xff),
static_cast<uint8_t>((i >> 070) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 9);
// check individual bytes
CHECK(result[0] == 'L');
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
(static_cast<uint64_t>(result[7]) << 060) +
(static_cast<uint64_t>(result[6]) << 050) +
(static_cast<uint64_t>(result[5]) << 040) +
(static_cast<uint64_t>(result[4]) << 030) +
(static_cast<uint64_t>(result[3]) << 020) +
(static_cast<uint64_t>(result[2]) << 010) +
static_cast<uint64_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("9223372036854775808..18446744073709551615 (uint64)")
{
std::vector<uint64_t> const v = {9223372036854775808ull, 18446744073709551615ull};
for (const uint64_t i : v)
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'M',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 010) & 0xff),
static_cast<uint8_t>((i >> 020) & 0xff),
static_cast<uint8_t>((i >> 030) & 0xff),
static_cast<uint8_t>((i >> 040) & 0xff),
static_cast<uint8_t>((i >> 050) & 0xff),
static_cast<uint8_t>((i >> 060) & 0xff),
static_cast<uint8_t>((i >> 070) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 9);
// check individual bytes
CHECK(result[0] == 'M');
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
(static_cast<uint64_t>(result[7]) << 060) +
(static_cast<uint64_t>(result[6]) << 050) +
(static_cast<uint64_t>(result[5]) << 040) +
(static_cast<uint64_t>(result[4]) << 030) +
(static_cast<uint64_t>(result[3]) << 020) +
(static_cast<uint64_t>(result[2]) << 010) +
static_cast<uint64_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
}
SECTION("unsigned")
{
SECTION("0..127 (int8)")
{
for (size_t i = 0; i <= 127; ++i)
{
CAPTURE(i)
// create JSON value with unsigned integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected{'i', static_cast<uint8_t>(i)};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 2);
// check individual bytes
CHECK(result[0] == 'i');
auto const restored = static_cast<uint8_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("128..255 (uint8)")
{
for (size_t i = 128; i <= 255; ++i)
{
CAPTURE(i)
// create JSON value with unsigned integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected{'U', static_cast<uint8_t>(i)};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 2);
// check individual bytes
CHECK(result[0] == 'U');
auto const restored = static_cast<uint8_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("256..32767 (int16)")
{
for (size_t i = 256; i <= 32767; i = utils::next_integer_sample(i, static_cast<size_t>(32767), static_cast<size_t>(7)))
{
CAPTURE(i)
// create JSON value with unsigned integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'I',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 3);
// check individual bytes
CHECK(result[0] == 'I');
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("32768..65535 (uint16)")
{
for (const uint32_t i :
{
32768u, 55555u, 65535u
})
{
CAPTURE(i)
// create JSON value with unsigned integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'u',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 3);
// check individual bytes
CHECK(result[0] == 'u');
auto const restored = static_cast<uint16_t>((static_cast<uint8_t>(result[2]) * 256) + static_cast<uint8_t>(result[1]));
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("65536..2147483647 (int32)")
{
for (const uint32_t i :
{
65536u, 77777u, 2147483647u
})
{
CAPTURE(i)
// create JSON value with unsigned integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'l',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
static_cast<uint8_t>((i >> 16) & 0xff),
static_cast<uint8_t>((i >> 24) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 5);
// check individual bytes
CHECK(result[0] == 'l');
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
(static_cast<uint32_t>(result[3]) << 020) +
(static_cast<uint32_t>(result[2]) << 010) +
static_cast<uint32_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("2147483648..4294967295 (uint32)")
{
for (const uint32_t i :
{
2147483648u, 3333333333u, 4294967295u
})
{
CAPTURE(i)
// create JSON value with unsigned integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'm',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 8) & 0xff),
static_cast<uint8_t>((i >> 16) & 0xff),
static_cast<uint8_t>((i >> 24) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 5);
// check individual bytes
CHECK(result[0] == 'm');
uint32_t const restored = (static_cast<uint32_t>(result[4]) << 030) +
(static_cast<uint32_t>(result[3]) << 020) +
(static_cast<uint32_t>(result[2]) << 010) +
static_cast<uint32_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("4294967296..9223372036854775807 (int64)")
{
std::vector<uint64_t> const v = {4294967296ul, 9223372036854775807ul};
for (const uint64_t i : v)
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'L',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 010) & 0xff),
static_cast<uint8_t>((i >> 020) & 0xff),
static_cast<uint8_t>((i >> 030) & 0xff),
static_cast<uint8_t>((i >> 040) & 0xff),
static_cast<uint8_t>((i >> 050) & 0xff),
static_cast<uint8_t>((i >> 060) & 0xff),
static_cast<uint8_t>((i >> 070) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 9);
// check individual bytes
CHECK(result[0] == 'L');
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
(static_cast<uint64_t>(result[7]) << 060) +
(static_cast<uint64_t>(result[6]) << 050) +
(static_cast<uint64_t>(result[5]) << 040) +
(static_cast<uint64_t>(result[4]) << 030) +
(static_cast<uint64_t>(result[3]) << 020) +
(static_cast<uint64_t>(result[2]) << 010) +
static_cast<uint64_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("9223372036854775808..18446744073709551615 (uint64)")
{
std::vector<uint64_t> const v = {9223372036854775808ull, 18446744073709551615ull};
for (const uint64_t i : v)
{
CAPTURE(i)
// create JSON value with integer number
json const j = i;
// check type
CHECK(j.is_number_unsigned());
// create expected byte vector
std::vector<uint8_t> const expected
{
'M',
static_cast<uint8_t>(i & 0xff),
static_cast<uint8_t>((i >> 010) & 0xff),
static_cast<uint8_t>((i >> 020) & 0xff),
static_cast<uint8_t>((i >> 030) & 0xff),
static_cast<uint8_t>((i >> 040) & 0xff),
static_cast<uint8_t>((i >> 050) & 0xff),
static_cast<uint8_t>((i >> 060) & 0xff),
static_cast<uint8_t>((i >> 070) & 0xff),
};
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == 9);
// check individual bytes
CHECK(result[0] == 'M');
uint64_t const restored = (static_cast<uint64_t>(result[8]) << 070) +
(static_cast<uint64_t>(result[7]) << 060) +
(static_cast<uint64_t>(result[6]) << 050) +
(static_cast<uint64_t>(result[5]) << 040) +
(static_cast<uint64_t>(result[4]) << 030) +
(static_cast<uint64_t>(result[3]) << 020) +
(static_cast<uint64_t>(result[2]) << 010) +
static_cast<uint64_t>(result[1]);
CHECK(restored == i);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
}
SECTION("float64")
{
SECTION("3.1415925")
{
double v = 3.1415925;
json const j = v;
std::vector<uint8_t> const expected =
{
'D', 0xfc, 0xde, 0xa6, 0x3f, 0xfb, 0x21, 0x09, 0x40
};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result) == v);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("half-precision float")
{
SECTION("simple half floats")
{
CHECK(json::parse("0.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x00})));
CHECK(json::parse("-0.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x80})));
CHECK(json::parse("1.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x3c})));
CHECK(json::parse("1.5") == json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x3e})));
CHECK(json::parse("65504.0") == json::from_bjdata(std::vector<uint8_t>({'h', 0xff, 0x7b})));
}
SECTION("errors")
{
SECTION("no byte follows")
{
json _;
std::vector<uint8_t> const vec0 = {'h'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec0), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vec0, true, false).is_discarded());
}
SECTION("only one byte follows")
{
json _;
std::vector<uint8_t> const vec1 = {'h', 0x00};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec1), "[json.exception.parse_error.110] parse error at byte 3: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vec1, true, false).is_discarded());
}
}
}
SECTION("half-precision float (edge cases)")
{
SECTION("exp = 0b00000")
{
SECTION("0 (0 00000 0000000000)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x00}));
const json::number_float_t d{j};
CHECK(d == 0.0);
}
SECTION("-0 (1 00000 0000000000)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x80}));
const json::number_float_t d{j};
CHECK(d == -0.0);
}
SECTION("2**-24 (0 00000 0000000001)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x01, 0x00}));
const json::number_float_t d{j};
CHECK(d == std::pow(2.0, -24.0));
}
}
SECTION("exp = 0b11111")
{
SECTION("infinity (0 11111 0000000000)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x7c}));
const json::number_float_t d{j};
CHECK(d == std::numeric_limits<json::number_float_t>::infinity());
CHECK(j.dump() == "null");
}
SECTION("-infinity (1 11111 0000000000)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0xfc}));
const json::number_float_t d{j};
CHECK(d == -std::numeric_limits<json::number_float_t>::infinity());
CHECK(j.dump() == "null");
}
}
SECTION("other values from https://en.wikipedia.org/wiki/Half-precision_floating-point_format")
{
SECTION("1 (0 01111 0000000000)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x3c}));
const json::number_float_t d{j};
CHECK(d == 1);
}
SECTION("-2 (1 10000 0000000000)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0xc0}));
const json::number_float_t d{j};
CHECK(d == -2);
}
SECTION("65504 (0 11110 1111111111)")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0xff, 0x7b}));
const json::number_float_t d{j};
CHECK(d == 65504);
}
}
SECTION("infinity")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x7c}));
json::number_float_t const d{j};
CHECK_FALSE(std::isfinite(d));
CHECK(j.dump() == "null");
}
SECTION("NaN")
{
json const j = json::from_bjdata(std::vector<uint8_t>({'h', 0x00, 0x7e }));
json::number_float_t const d{j};
CHECK(std::isnan(d));
CHECK(j.dump() == "null");
}
}
SECTION("high-precision number")
{
SECTION("unsigned integer number")
{
std::vector<uint8_t> const vec = {'H', 'i', 0x14, '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0'};
const auto j = json::from_bjdata(vec);
CHECK(j.is_number_unsigned());
CHECK(j.dump() == "12345678901234567890");
}
SECTION("signed integer number")
{
std::vector<uint8_t> const vec = {'H', 'i', 0x13, '-', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8'};
const auto j = json::from_bjdata(vec);
CHECK(j.is_number_integer());
CHECK(j.dump() == "-123456789012345678");
}
SECTION("floating-point number")
{
std::vector<uint8_t> const vec = {'H', 'i', 0x16, '3', '.', '1', '4', '1', '5', '9', '2', '6', '5', '3', '5', '8', '9', '7', '9', '3', '2', '3', '8', '4', '6'};
const auto j = json::from_bjdata(vec);
CHECK(j.is_number_float());
CHECK(j.dump() == "3.141592653589793");
}
SECTION("errors")
{
// error while parsing length
std::vector<uint8_t> const vec0 = {'H', 'i'};
CHECK(json::from_bjdata(vec0, true, false).is_discarded());
// error while parsing string
std::vector<uint8_t> const vec1 = {'H', 'i', '1'};
CHECK(json::from_bjdata(vec1, true, false).is_discarded());
json _;
std::vector<uint8_t> const vec2 = {'H', 'i', 2, '1', 'A', '3'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec2), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text: 1A", json::parse_error);
std::vector<uint8_t> const vec3 = {'H', 'i', 2, '1', '.'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec3), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text: 1.", json::parse_error);
std::vector<uint8_t> const vec_overflow = {'H', 'i', 5, '1', 'e', '4', '0', '0'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec_overflow), "[json.exception.out_of_range.406] number overflow parsing '1e400'", json::out_of_range);
std::vector<uint8_t> const vec4 = {'H', 2, '1', '0'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec4), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x02", json::parse_error);
}
}
}
SECTION("string")
{
SECTION("N = 0..127")
{
for (size_t N = 0; N <= 127; ++N)
{
CAPTURE(N)
// create JSON value with string containing of N * 'x'
const auto s = std::string(N, 'x');
json const j = s;
// create expected byte vector
std::vector<uint8_t> expected;
expected.push_back('S');
expected.push_back('i');
expected.push_back(static_cast<uint8_t>(N));
for (size_t i = 0; i < N; ++i)
{
expected.push_back('x');
}
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == N + 3);
// check that no null byte is appended
if (N > 0)
{
CHECK(result.back() != '\x00');
}
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("N = 128..255")
{
for (size_t N = 128; N <= 255; ++N)
{
CAPTURE(N)
// create JSON value with string containing of N * 'x'
const auto s = std::string(N, 'x');
json const j = s;
// create expected byte vector
std::vector<uint8_t> expected;
expected.push_back('S');
expected.push_back('U');
expected.push_back(static_cast<uint8_t>(N));
for (size_t i = 0; i < N; ++i)
{
expected.push_back('x');
}
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == N + 3);
// check that no null byte is appended
CHECK(result.back() != '\x00');
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("N = 256..32767")
{
for (const size_t N :
{
256u, 999u, 1025u, 3333u, 2048u, 32767u
})
{
CAPTURE(N)
// create JSON value with string containing of N * 'x'
const auto s = std::string(N, 'x');
json const j = s;
// create expected byte vector (hack: create string first)
std::vector<uint8_t> expected(N, 'x');
// reverse order of commands, because we insert at begin()
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
expected.insert(expected.begin(), 'I');
expected.insert(expected.begin(), 'S');
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == N + 4);
// check that no null byte is appended
CHECK(result.back() != '\x00');
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("N = 32768..65535")
{
for (const size_t N :
{
32768u, 55555u, 65535u
})
{
CAPTURE(N)
// create JSON value with string containing of N * 'x'
const auto s = std::string(N, 'x');
json const j = s;
// create expected byte vector (hack: create string first)
std::vector<uint8_t> expected(N, 'x');
// reverse order of commands, because we insert at begin()
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
expected.insert(expected.begin(), 'u');
expected.insert(expected.begin(), 'S');
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == N + 4);
// check that no null byte is appended
CHECK(result.back() != '\x00');
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("N = 65536..2147483647")
{
for (const size_t N :
{
65536u, 77777u, 1048576u
})
{
CAPTURE(N)
// create JSON value with string containing of N * 'x'
const auto s = std::string(N, 'x');
json const j = s;
// create expected byte vector (hack: create string first)
std::vector<uint8_t> expected(N, 'x');
// reverse order of commands, because we insert at begin()
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 24) & 0xff));
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 16) & 0xff));
expected.insert(expected.begin(), static_cast<uint8_t>((N >> 8) & 0xff));
expected.insert(expected.begin(), static_cast<uint8_t>(N & 0xff));
expected.insert(expected.begin(), 'l');
expected.insert(expected.begin(), 'S');
// compare result + size
const auto result = json::to_bjdata(j);
CHECK(result == expected);
CHECK(result.size() == N + 6);
// check that no null byte is appended
CHECK(result.back() != '\x00');
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
}
SECTION("binary")
{
for (json::bjdata_version_t bjdata_version :
{
json::bjdata_version_t::draft2, json::bjdata_version_t::draft3
})
{
CAPTURE(bjdata_version)
const bool draft3 = (bjdata_version == json::bjdata_version_t::draft3);
SECTION("N = 0..127")
{
for (std::size_t N = 0; N <= 127; ++N)
{
CAPTURE(N)
// create JSON value with byte array containing of N * 'x'
const auto s = std::vector<std::uint8_t>(N, 'x');
json const j = json::binary(s);
// create expected byte vector
std::vector<std::uint8_t> expected;
expected.push_back(static_cast<std::uint8_t>('['));
if (draft3 || N != 0)
{
expected.push_back(static_cast<std::uint8_t>('$'));
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
}
expected.push_back(static_cast<std::uint8_t>('#'));
expected.push_back(static_cast<std::uint8_t>('i'));
expected.push_back(static_cast<std::uint8_t>(N));
for (size_t i = 0; i < N; ++i)
{
expected.push_back(0x78);
}
// compare result + size
const auto result = json::to_bjdata(j, true, true, bjdata_version);
CHECK(result == expected);
if (!draft3 && N == 0)
{
CHECK(result.size() == N + 4);
}
else
{
CHECK(result.size() == N + 6);
}
// check that no null byte is appended
if (N > 0)
{
CHECK(result.back() != '\x00');
}
if (draft3)
{
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
else
{
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
}
}
SECTION("N = 128..255")
{
for (std::size_t N = 128; N <= 255; ++N)
{
CAPTURE(N)
// create JSON value with byte array containing of N * 'x'
const auto s = std::vector<std::uint8_t>(N, 'x');
json const j = json::binary(s);
// create expected byte vector
std::vector<uint8_t> expected;
expected.push_back(static_cast<std::uint8_t>('['));
expected.push_back(static_cast<std::uint8_t>('$'));
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
expected.push_back(static_cast<std::uint8_t>('#'));
expected.push_back(static_cast<std::uint8_t>('U'));
expected.push_back(static_cast<std::uint8_t>(N));
for (size_t i = 0; i < N; ++i)
{
expected.push_back(0x78);
}
// compare result + size
const auto result = json::to_bjdata(j, true, true, bjdata_version);
CHECK(result == expected);
CHECK(result.size() == N + 6);
// check that no null byte is appended
CHECK(result.back() != '\x00');
if (draft3)
{
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
else
{
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
}
}
SECTION("N = 256..32767")
{
for (const std::size_t N :
{
256u, 999u, 1025u, 3333u, 2048u, 32767u
})
{
CAPTURE(N)
// create JSON value with byte array containing of N * 'x'
const auto s = std::vector<std::uint8_t>(N, 'x');
json const j = json::binary(s);
// create expected byte vector
std::vector<std::uint8_t> expected(N + 7, 'x');
expected[0] = '[';
expected[1] = '$';
expected[2] = draft3 ? 'B' : 'U';
expected[3] = '#';
expected[4] = 'I';
expected[5] = static_cast<std::uint8_t>(N & 0xFF);
expected[6] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
// compare result + size
const auto result = json::to_bjdata(j, true, true, bjdata_version);
CHECK(result == expected);
CHECK(result.size() == N + 7);
// check that no null byte is appended
CHECK(result.back() != '\x00');
if (draft3)
{
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
else
{
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
}
}
SECTION("N = 32768..65535")
{
for (const std::size_t N :
{
32768u, 55555u, 65535u
})
{
CAPTURE(N)
// create JSON value with byte array containing of N * 'x'
const auto s = std::vector<std::uint8_t>(N, 'x');
json const j = json::binary(s);
// create expected byte vector
std::vector<std::uint8_t> expected(N + 7, 'x');
expected[0] = '[';
expected[1] = '$';
expected[2] = draft3 ? 'B' : 'U';
expected[3] = '#';
expected[4] = 'u';
expected[5] = static_cast<std::uint8_t>(N & 0xFF);
expected[6] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
// compare result + size
const auto result = json::to_bjdata(j, true, true, bjdata_version);
CHECK(result == expected);
CHECK(result.size() == N + 7);
// check that no null byte is appended
CHECK(result.back() != '\x00');
if (draft3)
{
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
else
{
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
}
}
SECTION("N = 65536..2147483647")
{
for (const std::size_t N :
{
65536u, 77777u, 1048576u
})
{
CAPTURE(N)
// create JSON value with byte array containing of N * 'x'
const auto s = std::vector<std::uint8_t>(N, 'x');
json const j = json::binary(s);
// create expected byte vector
std::vector<std::uint8_t> expected(N + 9, 'x');
expected[0] = '[';
expected[1] = '$';
expected[2] = draft3 ? 'B' : 'U';
expected[3] = '#';
expected[4] = 'l';
expected[5] = static_cast<std::uint8_t>(N & 0xFF);
expected[6] = static_cast<std::uint8_t>((N >> 8) & 0xFF);
expected[7] = static_cast<std::uint8_t>((N >> 16) & 0xFF);
expected[8] = static_cast<std::uint8_t>((N >> 24) & 0xFF);
// compare result + size
const auto result = json::to_bjdata(j, true, true, bjdata_version);
CHECK(result == expected);
CHECK(result.size() == N + 9);
// check that no null byte is appended
CHECK(result.back() != '\x00');
if (draft3)
{
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
else
{
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
}
}
SECTION("Other Serializations")
{
const std::size_t N = 10;
const auto s = std::vector<std::uint8_t>(N, 'x');
json const j = json::binary(s);
SECTION("No Count No Type")
{
std::vector<uint8_t> expected;
expected.push_back(static_cast<std::uint8_t>('['));
for (std::size_t i = 0; i < N; ++i)
{
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
expected.push_back(static_cast<std::uint8_t>(0x78));
}
expected.push_back(static_cast<std::uint8_t>(']'));
// compare result + size
const auto result = json::to_bjdata(j, false, false, bjdata_version);
CHECK(result == expected);
CHECK(result.size() == N + 12);
// check that no null byte is appended
CHECK(result.back() != '\x00');
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
SECTION("Yes Count No Type")
{
std::vector<std::uint8_t> expected;
expected.push_back(static_cast<std::uint8_t>('['));
expected.push_back(static_cast<std::uint8_t>('#'));
expected.push_back(static_cast<std::uint8_t>('i'));
expected.push_back(static_cast<std::uint8_t>(N));
for (size_t i = 0; i < N; ++i)
{
expected.push_back(static_cast<std::uint8_t>(draft3 ? 'B' : 'U'));
expected.push_back(static_cast<std::uint8_t>(0x78));
}
// compare result + size
const auto result = json::to_bjdata(j, true, false, bjdata_version);
CHECK(result == expected);
CHECK(result.size() == N + 14);
// check that no null byte is appended
CHECK(result.back() != '\x00');
// roundtrip only works to an array of numbers
json j_out = s;
CHECK(json::from_bjdata(result) == j_out);
CHECK(json::from_bjdata(result, true, false) == j_out);
}
}
}
}
SECTION("array")
{
SECTION("empty")
{
SECTION("size=false type=false")
{
json const j = json::array();
std::vector<uint8_t> const expected = {'[', ']'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = json::array();
std::vector<uint8_t> const expected = {'[', '#', 'i', 0};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=true")
{
json const j = json::array();
std::vector<uint8_t> const expected = {'[', '#', 'i', 0};
const auto result = json::to_bjdata(j, true, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("[null]")
{
SECTION("size=false type=false")
{
json const j = {nullptr};
std::vector<uint8_t> const expected = {'[', 'Z', ']'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = {nullptr};
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, 'Z'};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=true")
{
json const j = {nullptr};
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, 'Z'};
const auto result = json::to_bjdata(j, true, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("[1,2,3,4,5]")
{
SECTION("size=false type=false")
{
json const j = json::parse("[1,2,3,4,5]");
std::vector<uint8_t> const expected = {'[', 'i', 1, 'i', 2, 'i', 3, 'i', 4, 'i', 5, ']'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = json::parse("[1,2,3,4,5]");
std::vector<uint8_t> const expected = {'[', '#', 'i', 5, 'i', 1, 'i', 2, 'i', 3, 'i', 4, 'i', 5};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=true")
{
json const j = json::parse("[1,2,3,4,5]");
std::vector<uint8_t> const expected = {'[', '$', 'i', '#', 'i', 5, 1, 2, 3, 4, 5};
const auto result = json::to_bjdata(j, true, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("[[[[]]]]")
{
SECTION("size=false type=false")
{
json const j = json::parse("[[[[]]]]");
std::vector<uint8_t> const expected = {'[', '[', '[', '[', ']', ']', ']', ']'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = json::parse("[[[[]]]]");
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 0};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=true")
{
json const j = json::parse("[[[[]]]]");
std::vector<uint8_t> const expected = {'[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 1, '[', '#', 'i', 0};
const auto result = json::to_bjdata(j, true, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("array with int16_t elements")
{
SECTION("size=false type=false")
{
json j(257, nullptr);
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
expected[0] = '['; // opening array
expected[258] = ']'; // closing array
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json j(257, nullptr);
std::vector<uint8_t> expected(j.size() + 5, 'Z'); // all null
expected[0] = '['; // opening array
expected[1] = '#'; // array size
expected[2] = 'I'; // int16
expected[3] = 0x01; // 0x0101, first byte
expected[4] = 0x01; // 0x0101, second byte
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("array with uint16_t elements")
{
SECTION("size=false type=false")
{
json j(32768, nullptr);
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
expected[0] = '['; // opening array
expected[32769] = ']'; // closing array
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json j(32768, nullptr);
std::vector<uint8_t> expected(j.size() + 5, 'Z'); // all null
expected[0] = '['; // opening array
expected[1] = '#'; // array size
expected[2] = 'u'; // int16
expected[3] = 0x00; // 0x0101, first byte
expected[4] = 0x80; // 0x0101, second byte
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("array with int32_t elements")
{
SECTION("size=false type=false")
{
json j(65793, nullptr);
std::vector<uint8_t> expected(j.size() + 2, 'Z'); // all null
expected[0] = '['; // opening array
expected[65794] = ']'; // closing array
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json j(65793, nullptr);
std::vector<uint8_t> expected(j.size() + 7, 'Z'); // all null
expected[0] = '['; // opening array
expected[1] = '#'; // array size
expected[2] = 'l'; // int32
expected[3] = 0x01; // 0x00010101, fourth byte
expected[4] = 0x01; // 0x00010101, third byte
expected[5] = 0x01; // 0x00010101, second byte
expected[6] = 0x00; // 0x00010101, first byte
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
}
SECTION("object")
{
SECTION("empty")
{
SECTION("size=false type=false")
{
json const j = json::object();
std::vector<uint8_t> const expected = {'{', '}'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = json::object();
std::vector<uint8_t> const expected = {'{', '#', 'i', 0};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=true")
{
json const j = json::object();
std::vector<uint8_t> const expected = {'{', '#', 'i', 0};
const auto result = json::to_bjdata(j, true, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("{\"\":null}")
{
SECTION("size=false type=false")
{
json const j = {{"", nullptr}};
std::vector<uint8_t> const expected = {'{', 'i', 0, 'Z', '}'};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = {{"", nullptr}};
std::vector<uint8_t> const expected = {'{', '#', 'i', 1, 'i', 0, 'Z'};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
SECTION("{\"a\": {\"b\": {\"c\": {}}}}")
{
SECTION("size=false type=false")
{
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
std::vector<uint8_t> const expected =
{
'{', 'i', 1, 'a', '{', 'i', 1, 'b', '{', 'i', 1, 'c', '{', '}', '}', '}', '}'
};
const auto result = json::to_bjdata(j);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=false")
{
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
std::vector<uint8_t> const expected =
{
'{', '#', 'i', 1, 'i', 1, 'a', '{', '#', 'i', 1, 'i', 1, 'b', '{', '#', 'i', 1, 'i', 1, 'c', '{', '#', 'i', 0
};
const auto result = json::to_bjdata(j, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
SECTION("size=true type=true ignore object type marker")
{
json const j = json::parse(R"({"a": {"b": {"c": {}}}})");
std::vector<uint8_t> const expected =
{
'{', '#', 'i', 1, 'i', 1, 'a', '{', '#', 'i', 1, 'i', 1, 'b', '{', '#', 'i', 1, 'i', 1, 'c', '{', '#', 'i', 0
};
const auto result = json::to_bjdata(j, true, true);
CHECK(result == expected);
// roundtrip
CHECK(json::from_bjdata(result) == j);
CHECK(json::from_bjdata(result, true, false) == j);
}
}
}
}
SECTION("errors")
{
SECTION("strict mode")
{
std::vector<uint8_t> const vec = {'Z', 'Z'};
SECTION("non-strict mode")
{
const auto result = json::from_bjdata(vec, false);
CHECK(result == json());
}
SECTION("strict mode")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec),
"[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData value: expected end of input; last byte: 0x5A", json::parse_error&);
}
}
}
SECTION("SAX aborts")
{
SECTION("start_array()")
{
std::vector<uint8_t> const v = {'[', 'T', 'F', ']'};
SaxCountdown scp(0);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("start_object()")
{
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
SaxCountdown scp(0);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("key() in object")
{
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
SaxCountdown scp(1);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("start_array(len)")
{
std::vector<uint8_t> const v = {'[', '#', 'i', '2', 'T', 'F'};
SaxCountdown scp(0);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("start_object(len)")
{
std::vector<uint8_t> const v = {'{', '#', 'i', '1', 3, 'f', 'o', 'o', 'F'};
SaxCountdown scp(0);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("key() in object with length")
{
std::vector<uint8_t> const v = {'{', 'i', 3, 'f', 'o', 'o', 'F', '}'};
SaxCountdown scp(1);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("start_array() in ndarray _ArraySize_")
{
std::vector<uint8_t> const v = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 2, 1, 1, 2};
SaxCountdown scp(2);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("number_integer() in ndarray _ArraySize_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 1, 1, 2};
SaxCountdown scp(3);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("key() in ndarray _ArrayType_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
SaxCountdown scp(6);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("string() in ndarray _ArrayType_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
SaxCountdown scp(7);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("key() in ndarray _ArrayData_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
SaxCountdown scp(8);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("string() in ndarray _ArrayData_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'U', '#', 'i', 2, 2, 2, 1, 2, 3, 4};
SaxCountdown scp(9);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("string() in ndarray _ArrayType_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 3, 2, 6, 5, 4, 3, 2, 1};
SaxCountdown scp(11);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
SECTION("start_array() in ndarray _ArrayData_")
{
std::vector<uint8_t> const v = {'[', '$', 'U', '#', '[', 'i', 2, 'i', 3, ']', 6, 5, 4, 3, 2, 1};
SaxCountdown scp(13);
CHECK_FALSE(json::sax_parse(v, &scp, json::input_format_t::bjdata));
}
}
SECTION("parsing values")
{
SECTION("strings")
{
// create a single-character string for all number types
std::vector<uint8_t> s_i = {'S', 'i', 1, 'a'};
std::vector<uint8_t> const s_U = {'S', 'U', 1, 'a'};
std::vector<uint8_t> const s_I = {'S', 'I', 1, 0, 'a'};
std::vector<uint8_t> const s_u = {'S', 'u', 1, 0, 'a'};
std::vector<uint8_t> const s_l = {'S', 'l', 1, 0, 0, 0, 'a'};
std::vector<uint8_t> const s_m = {'S', 'm', 1, 0, 0, 0, 'a'};
std::vector<uint8_t> const s_L = {'S', 'L', 1, 0, 0, 0, 0, 0, 0, 0, 'a'};
std::vector<uint8_t> const s_M = {'S', 'M', 1, 0, 0, 0, 0, 0, 0, 0, 'a'};
// check if string is parsed correctly to "a"
CHECK(json::from_bjdata(s_i) == "a");
CHECK(json::from_bjdata(s_U) == "a");
CHECK(json::from_bjdata(s_I) == "a");
CHECK(json::from_bjdata(s_u) == "a");
CHECK(json::from_bjdata(s_l) == "a");
CHECK(json::from_bjdata(s_m) == "a");
CHECK(json::from_bjdata(s_L) == "a");
CHECK(json::from_bjdata(s_M) == "a");
// roundtrip: output should be optimized
CHECK(json::to_bjdata(json::from_bjdata(s_i)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_U)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_I)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_u)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_l)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_m)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_L)) == s_i);
CHECK(json::to_bjdata(json::from_bjdata(s_M)) == s_i);
}
SECTION("number")
{
SECTION("float")
{
// float32
std::vector<uint8_t> const v_d = {'d', 0xd0, 0x0f, 0x49, 0x40};
CHECK(json::from_bjdata(v_d) == 3.14159f);
// float64
std::vector<uint8_t> const v_D = {'D', 0x6e, 0x86, 0x1b, 0xf0, 0xf9, 0x21, 0x09, 0x40};
CHECK(json::from_bjdata(v_D) == 3.14159);
// float32 is serialized as float64 as the library does not support float32
CHECK(json::to_bjdata(json::from_bjdata(v_d)) == json::to_bjdata(3.14159f));
}
}
SECTION("array")
{
SECTION("optimized version (length only)")
{
// create vector with two elements of the same type
std::vector<uint8_t> const v_TU = {'[', '#', 'U', 2, 'T', 'T'};
std::vector<uint8_t> const v_T = {'[', '#', 'i', 2, 'T', 'T'};
std::vector<uint8_t> const v_F = {'[', '#', 'i', 2, 'F', 'F'};
std::vector<uint8_t> const v_Z = {'[', '#', 'i', 2, 'Z', 'Z'};
std::vector<uint8_t> const v_i = {'[', '#', 'i', 2, 'i', 0x7F, 'i', 0x7F};
std::vector<uint8_t> const v_U = {'[', '#', 'i', 2, 'U', 0xFF, 'U', 0xFF};
std::vector<uint8_t> const v_I = {'[', '#', 'i', 2, 'I', 0xFF, 0x7F, 'I', 0xFF, 0x7F};
std::vector<uint8_t> const v_u = {'[', '#', 'i', 2, 'u', 0x0F, 0xA7, 'u', 0x0F, 0xA7};
std::vector<uint8_t> const v_l = {'[', '#', 'i', 2, 'l', 0xFF, 0xFF, 0xFF, 0x7F, 'l', 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_m = {'[', '#', 'i', 2, 'm', 0xFF, 0xC9, 0x9A, 0xBB, 'm', 0xFF, 0xC9, 0x9A, 0xBB};
std::vector<uint8_t> const v_L = {'[', '#', 'i', 2, 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_M = {'[', '#', 'i', 2, 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
std::vector<uint8_t> const v_D = {'[', '#', 'i', 2, 'D', 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 'D', 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
std::vector<uint8_t> const v_S = {'[', '#', 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
std::vector<uint8_t> const v_C = {'[', '#', 'i', 2, 'C', 'a', 'C', 'a'};
std::vector<uint8_t> const v_B = {'[', '#', 'i', 2, 'B', 0xFF, 'B', 0xFF};
// check if vector is parsed correctly
CHECK(json::from_bjdata(v_TU) == json({true, true}));
CHECK(json::from_bjdata(v_T) == json({true, true}));
CHECK(json::from_bjdata(v_F) == json({false, false}));
CHECK(json::from_bjdata(v_Z) == json({nullptr, nullptr}));
CHECK(json::from_bjdata(v_i) == json({127, 127}));
CHECK(json::from_bjdata(v_U) == json({255, 255}));
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
CHECK(json::from_bjdata(v_B) == json({255, 255}));
// roundtrip: output should be optimized
CHECK(json::to_bjdata(json::from_bjdata(v_T), true) == v_T);
CHECK(json::to_bjdata(json::from_bjdata(v_F), true) == v_F);
CHECK(json::to_bjdata(json::from_bjdata(v_Z), true) == v_Z);
CHECK(json::to_bjdata(json::from_bjdata(v_i), true) == v_i);
CHECK(json::to_bjdata(json::from_bjdata(v_U), true) == v_U);
CHECK(json::to_bjdata(json::from_bjdata(v_I), true) == v_I);
CHECK(json::to_bjdata(json::from_bjdata(v_u), true) == v_u);
CHECK(json::to_bjdata(json::from_bjdata(v_l), true) == v_l);
CHECK(json::to_bjdata(json::from_bjdata(v_m), true) == v_m);
CHECK(json::to_bjdata(json::from_bjdata(v_L), true) == v_L);
CHECK(json::to_bjdata(json::from_bjdata(v_M), true) == v_M);
CHECK(json::to_bjdata(json::from_bjdata(v_D), true) == v_D);
CHECK(json::to_bjdata(json::from_bjdata(v_S), true) == v_S);
CHECK(json::to_bjdata(json::from_bjdata(v_C), true) == v_S); // char is serialized to string
CHECK(json::to_bjdata(json::from_bjdata(v_B), true) == v_U); // byte is serialized to uint8
}
SECTION("optimized version (type and length)")
{
// create vector with two elements of the same type
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', 'i', 2, 0x7F, 0x7F};
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', 'i', 2, 0xFF, 0xFF};
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', 'i', 2, 0xFF, 0x7F, 0xFF, 0x7F};
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', 'i', 2, 0x0F, 0xA7, 0x0F, 0xA7};
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', 'i', 2, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', 'i', 2, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', 'i', 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', 'i', 2, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
std::vector<uint8_t> const v_S = {'[', '#', 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', 'i', 2, 'a', 'a'};
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', 'i', 2, 0xFF, 0xFF};
// check if vector is parsed correctly
CHECK(json::from_bjdata(v_i) == json({127, 127}));
CHECK(json::from_bjdata(v_U) == json({255, 255}));
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
// roundtrip: output should be optimized
std::vector<uint8_t> const v_empty = {'[', '#', 'i', 0};
CHECK(json::to_bjdata(json::from_bjdata(v_i), true, true) == v_i);
CHECK(json::to_bjdata(json::from_bjdata(v_U), true, true) == v_U);
CHECK(json::to_bjdata(json::from_bjdata(v_I), true, true) == v_I);
CHECK(json::to_bjdata(json::from_bjdata(v_u), true, true) == v_u);
CHECK(json::to_bjdata(json::from_bjdata(v_l), true, true) == v_l);
CHECK(json::to_bjdata(json::from_bjdata(v_m), true, true) == v_m);
CHECK(json::to_bjdata(json::from_bjdata(v_L), true, true) == v_L);
CHECK(json::to_bjdata(json::from_bjdata(v_M), true, true) == v_M);
CHECK(json::to_bjdata(json::from_bjdata(v_D), true, true) == v_D);
CHECK(json::to_bjdata(json::from_bjdata(v_S), true, true) == v_S);
CHECK(json::to_bjdata(json::from_bjdata(v_C), true, true) == v_S); // char is serialized to string
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft2) == v_U);
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft3) == v_B);
}
SECTION("optimized ndarray (type and vector-size as optimized 1D array)")
{
// create vector with two elements of the same type
std::vector<uint8_t> const v_0 = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 1, 0};
std::vector<uint8_t> const v_1 = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 1, 2, 0x7F, 0x7F};
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0x7F, 0x7F};
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF};
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0x7F, 0xFF, 0x7F};
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0x0F, 0xA7, 0x0F, 0xA7};
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
std::vector<uint8_t> const v_S = {'[', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 'a', 'a'};
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 1, 2, 0xFF, 0xFF};
// check if vector is parsed correctly
CHECK(json::from_bjdata(v_0) == json::array());
CHECK(json::from_bjdata(v_1) == json({127, 127}));
CHECK(json::from_bjdata(v_i) == json({127, 127}));
CHECK(json::from_bjdata(v_U) == json({255, 255}));
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
}
SECTION("optimized ndarray (type and vector-size ndarray with JData annotations)")
{
// create vector with 0, 1, 2 elements of the same type
std::vector<uint8_t> const v_e = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 1, 0xFE, 0xFF};
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04, 0x00, 0x05, 0x00, 0x06, 0x00};
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04, 0x00, 0x05, 0x00, 0x06, 0x00};
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00};
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00};
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
std::vector<uint8_t> const v_d = {'[', '$', 'd', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x00, 0x00, 0x80, 0x3F, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x40, 0x40, 0x00, 0x00, 0x80, 0x40, 0x00, 0x00, 0xA0, 0x40, 0x00, 0x00, 0xC0, 0x40};
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xF0, 0x3F, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x14, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x40};
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 'a', 'b', 'c', 'd', 'e', 'f'};
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
// check if vector is parsed correctly
CHECK(json::from_bjdata(v_e) == json({{"_ArrayData_", {254, 255}}, {"_ArraySize_", {2, 1}}, {"_ArrayType_", "uint8"}}));
CHECK(json::from_bjdata(v_U) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint8"}}));
CHECK(json::from_bjdata(v_i) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int8"}}));
CHECK(json::from_bjdata(v_i) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int8"}}));
CHECK(json::from_bjdata(v_u) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint16"}}));
CHECK(json::from_bjdata(v_I) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int16"}}));
CHECK(json::from_bjdata(v_m) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint32"}}));
CHECK(json::from_bjdata(v_l) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int32"}}));
CHECK(json::from_bjdata(v_M) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint64"}}));
CHECK(json::from_bjdata(v_L) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "int64"}}));
CHECK(json::from_bjdata(v_d) == json({{"_ArrayData_", {1.f, 2.f, 3.f, 4.f, 5.f, 6.f}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "single"}}));
CHECK(json::from_bjdata(v_D) == json({{"_ArrayData_", {1., 2., 3., 4., 5., 6.}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "double"}}));
CHECK(json::from_bjdata(v_C) == json({{"_ArrayData_", {'a', 'b', 'c', 'd', 'e', 'f'}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "char"}}));
CHECK(json::from_bjdata(v_B) == json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "byte"}}));
// roundtrip: output should be optimized
CHECK(json::to_bjdata(json::from_bjdata(v_e), true, true) == v_e);
CHECK(json::to_bjdata(json::from_bjdata(v_U), true, true) == v_U);
CHECK(json::to_bjdata(json::from_bjdata(v_i), true, true) == v_i);
CHECK(json::to_bjdata(json::from_bjdata(v_u), true, true) == v_u);
CHECK(json::to_bjdata(json::from_bjdata(v_I), true, true) == v_I);
CHECK(json::to_bjdata(json::from_bjdata(v_m), true, true) == v_m);
CHECK(json::to_bjdata(json::from_bjdata(v_l), true, true) == v_l);
CHECK(json::to_bjdata(json::from_bjdata(v_M), true, true) == v_M);
CHECK(json::to_bjdata(json::from_bjdata(v_L), true, true) == v_L);
CHECK(json::to_bjdata(json::from_bjdata(v_d), true, true) == v_d);
CHECK(json::to_bjdata(json::from_bjdata(v_D), true, true) == v_D);
CHECK(json::to_bjdata(json::from_bjdata(v_C), true, true) == v_C);
// v_B uses the Draft-3-only 'B' marker, so it round-trips only when
// Draft 3 is explicitly selected (see GitHub issue #5404); the
// default Draft 2 falls back to a plain object instead, covered by
// the "ndarray with _ArrayType_ "byte" is gated by the BJData draft
// version" section below
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft3) == v_B);
}
SECTION("ndarray with data not matching _ArrayType_ is written as an object")
{
// A JData-annotated object is only serialized as an ndarray when
// its _ArrayData_ elements are actually stored as the number kind
// named by _ArrayType_. Otherwise the writer would read the wrong
// union member (e.g. a std::string's heap pointer as a uint64) and
// emit it, so such an object falls back to a plain object encoding
// that still round-trips.
// string data declared as a uint64 array
json const j_str = json({{"_ArrayType_", "uint64"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {"pointer", "value"}}});
const auto out_str = json::to_bjdata(j_str);
CHECK(out_str.at(0) == '{');
CHECK(json::from_bjdata(out_str) == j_str);
// integer data declared as a double array
json const j_float = json({{"_ArrayType_", "double"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, 2}}});
const auto out_float = json::to_bjdata(j_float);
CHECK(out_float.at(0) == '{');
CHECK(json::from_bjdata(out_float) == j_float);
// a non-integer shape entry is likewise not treated as an ndarray
json const j_size = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {"x", 1}}, {"_ArrayData_", {1}}});
const auto out_size = json::to_bjdata(j_size);
CHECK(out_size.at(0) == '{');
CHECK(json::from_bjdata(out_size) == j_size);
// a negative shape entry is not a usable dimension either
json const j_neg = json::parse(R"({"_ArrayType_":"uint8","_ArraySize_":[-1,1],"_ArrayData_":[1]})");
const auto out_neg = json::to_bjdata(j_neg);
CHECK(out_neg.at(0) == '{');
CHECK(json::from_bjdata(out_neg) == j_neg);
}
SECTION("ndarray parsed from text is written as a typed array")
{
// json::parse stores a non-negative integer as number_unsigned while
// the C++ API stores an int literal as number_integer, so _ArrayType_
// names the wire type rather than the storage. Both storages have to
// produce the same typed array for every type.
// "byte" is checked separately below since it additionally requires
// BJData Draft 3 to be selected explicitly (see GitHub issue #5404).
for (const char* type :
{"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char"
})
{
CAPTURE(type)
const std::string text = std::string(R"({"_ArrayType_":")") + type +
R"(","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
const auto from_text = json::to_bjdata(json::parse(text));
CHECK(from_text.at(0) == '[');
CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", type}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}})));
}
{
const std::string text = R"({"_ArrayType_":"byte","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
const auto from_text = json::to_bjdata(json::parse(text), true, true, json::bjdata_version_t::draft3);
CHECK(from_text.at(0) == '[');
CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}}),
true, true, json::bjdata_version_t::draft3));
}
// negative values under a signed type behave the same way
const auto from_neg = json::to_bjdata(json::parse(R"({"_ArrayType_":"int32","_ArraySize_":[2,1],"_ArrayData_":[-5,7]})"));
CHECK(from_neg.at(0) == '[');
CHECK(from_neg == json::to_bjdata(json({{"_ArrayType_", "int32"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {-5, 7}}})));
// and so do the floating point types
const auto from_float = json::to_bjdata(json::parse(R"({"_ArrayType_":"double","_ArraySize_":[2,1],"_ArrayData_":[1.5,2.5]})"));
CHECK(from_float.at(0) == '[');
CHECK(from_float == json::to_bjdata(json({{"_ArrayType_", "double"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, 2.5}}})));
}
SECTION("optimized ndarray (type and vector-size as 1D array)")
{
// create vector with two elements of the same type
std::vector<uint8_t> const v_0 = {'[', '$', 'i', '#', '[', ']'};
std::vector<uint8_t> const v_E = {'[', '$', 'i', '#', '[', 'i', 2, 'i', 0, ']'};
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', 'i', 1, 'i', 2, ']', 0x7F, 0x7F};
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF};
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0x7F, 0xFF, 0x7F};
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', 'i', 1, 'i', 2, ']', 0x0F, 0xA7, 0x0F, 0xA7};
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', 'i', 1, 'i', 2, ']', 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
std::vector<uint8_t> const v_S = {'[', '#', '[', 'i', 1, 'i', 2, ']', 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', 'i', 1, 'i', 2, ']', 'a', 'a'};
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', 'i', 1, 'i', 2, ']', 0xFF, 0xFF};
std::vector<uint8_t> const v_R = {'[', '#', '[', 'i', 2, ']', 'i', 6, 'U', 7};
// check if vector is parsed correctly
CHECK(json::from_bjdata(v_0) == json::array());
CHECK(json::from_bjdata(v_E) == json::array());
CHECK(json::from_bjdata(v_i) == json({127, 127}));
CHECK(json::from_bjdata(v_U) == json({255, 255}));
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
CHECK(json::from_bjdata(v_R) == json({6, 7}));
}
SECTION("optimized ndarray (type and vector-size as size-optimized array)")
{
// create vector with two elements of the same type
std::vector<uint8_t> const v_i = {'[', '$', 'i', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0x7F, 0x7F};
std::vector<uint8_t> const v_U = {'[', '$', 'U', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF};
std::vector<uint8_t> const v_I = {'[', '$', 'I', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0x7F, 0xFF, 0x7F};
std::vector<uint8_t> const v_u = {'[', '$', 'u', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0x0F, 0xA7, 0x0F, 0xA7};
std::vector<uint8_t> const v_l = {'[', '$', 'l', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_m = {'[', '$', 'm', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0xFF, 0xC9, 0x9A, 0xBB};
std::vector<uint8_t> const v_L = {'[', '$', 'L', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x7F};
std::vector<uint8_t> const v_M = {'[', '$', 'M', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
std::vector<uint8_t> const v_D = {'[', '$', 'D', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40, 0x4a, 0xd8, 0x12, 0x4d, 0xfb, 0x21, 0x09, 0x40};
std::vector<uint8_t> const v_S = {'[', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 'S', 'i', 1, 'a', 'S', 'i', 1, 'a'};
std::vector<uint8_t> const v_C = {'[', '$', 'C', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 'a', 'a'};
std::vector<uint8_t> const v_B = {'[', '$', 'B', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2, 0xFF, 0xFF};
// check if vector is parsed correctly
CHECK(json::from_bjdata(v_i) == json({127, 127}));
CHECK(json::from_bjdata(v_U) == json({255, 255}));
CHECK(json::from_bjdata(v_I) == json({32767, 32767}));
CHECK(json::from_bjdata(v_u) == json({42767, 42767}));
CHECK(json::from_bjdata(v_l) == json({2147483647, 2147483647}));
CHECK(json::from_bjdata(v_m) == json({3147483647, 3147483647}));
CHECK(json::from_bjdata(v_L) == json({9223372036854775807, 9223372036854775807}));
CHECK(json::from_bjdata(v_M) == json({10223372036854775807ull, 10223372036854775807ull}));
CHECK(json::from_bjdata(v_D) == json({3.1415926, 3.1415926}));
CHECK(json::from_bjdata(v_S) == json({"a", "a"}));
CHECK(json::from_bjdata(v_C) == json({"a", "a"}));
CHECK(json::from_bjdata(v_B) == json::binary(std::vector<uint8_t>({static_cast<uint8_t>(255), static_cast<uint8_t>(255)})));
}
SECTION("invalid ndarray annotations remains as object")
{
// check if invalid ND array annotations stay as object
json j_type = json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "invalidtype"}});
json j_size = json({{"_ArrayData_", {1, 2, 3, 4, 5}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint8"}});
// roundtrip: output should stay as object
CHECK(json::from_bjdata(json::to_bjdata(j_type), true, true) == j_type);
CHECK(json::from_bjdata(json::to_bjdata(j_size), true, true) == j_size);
}
SECTION("ndarray whose _ArrayType_ is not a string stays as object")
{
// the type name is looked up as a string below the annotation
// check; a non-string _ArrayType_ cannot name a known dtype,
// so calling get<string_t>() on it would throw type_error.302
// instead of falling back like an unrecognized type name
// already does (see GitHub issue #5398)
json const j_number = json({{"_ArrayType_", 1}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_number = json::to_bjdata(j_number);
CHECK(out_number.at(0) == '{');
CHECK(json::from_bjdata(out_number) == j_number);
json const j_null = json({{"_ArrayType_", nullptr}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_null = json::to_bjdata(j_null);
CHECK(out_null.at(0) == '{');
CHECK(json::from_bjdata(out_null) == j_null);
json const j_bool = json({{"_ArrayType_", true}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_bool = json::to_bjdata(j_bool);
CHECK(out_bool.at(0) == '{');
CHECK(json::from_bjdata(out_bool) == j_bool);
json const j_array = json({{"_ArrayType_", {"uint8"}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_array = json::to_bjdata(j_array);
CHECK(out_array.at(0) == '{');
CHECK(json::from_bjdata(out_array) == j_array);
json const j_object = json({{"_ArrayType_", {{"a", 1}}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_object = json::to_bjdata(j_object);
CHECK(out_object.at(0) == '{');
CHECK(json::from_bjdata(out_object) == j_object);
}
SECTION("re-serializing a value containing a plain-array-of-bytes is value-stable but not byte-stable")
{
// OSS-Fuzz found this input (an array whose first element is a
// binary_t byte, followed by an object whose _ArrayType_ is
// not a string) while exercising the fix for #5398 above: once
// the fix stops to_bjdata() from throwing type_error.302 for
// the third element, serialization proceeds far enough to
// reach a pre-existing, unrelated round-trip quirk in how a
// single-byte binary_t value is re-encoded.
std::vector<std::uint8_t> const input
{
0x5b, 0x5b, 0x24, 0x42, 0x23, 0x5b, 0x69, 0x01, 0x5d, 0x5b, 0x5b, 0x5d, 0x7b, 0x55, 0x0b,
0x5f, 0x41, 0x72, 0x72, 0x61, 0x79, 0x44, 0x61, 0x74, 0x61, 0x5f, 0x54, 0x55, 0x0b, 0x5f,
0x41, 0x72, 0x72, 0x61, 0x79, 0x53, 0x69, 0x7a, 0x65, 0x5f, 0x5a, 0x55, 0x0b, 0x5f, 0x41,
0x72, 0x72, 0x61, 0x79, 0x54, 0x79, 0x70, 0x65, 0x5f, 0x54, 0x7d, 0x5d
};
json const j1 = json::from_bjdata(input);
// to_bjdata() must not throw (this is what #5398 fixes)
std::vector<std::uint8_t> vec2;
CHECK_NOTHROW(vec2 = json::to_bjdata(j1, false, false));
// parsing back a plain (non-optimized) array of bytes cannot
// recover that it used to be a binary_t: from_bjdata() has no
// way to distinguish "array of uint8 numbers" from "array of
// bytes" unless the compact "$U#" array header is used, so
// the binary_t collapses into a plain JSON array
json const j2 = json::from_bjdata(vec2);
CHECK(j1 != j2);
CHECK(j2 == json({{91}, json::array(), {{"_ArrayData_", true}, {"_ArraySize_", nullptr}, {"_ArrayType_", true}}}));
// re-serializing j2 no longer goes through the dedicated
// binary_t writer (which always uses the 'U' marker for raw
// bytes); the now-plain number 91 goes through the generic
// smallest-type writer instead, which - like the rest of the
// UBJSON/BJData writer, and unchanged by this fix - prefers
// the 'i' (int8) marker over 'U' (uint8) for values that fit
// both. Both markers are valid BJData and both decode back to
// 91, so this is not byte-for-byte identical to vec2, but it
// is value-stable: parsing it again reproduces j2 exactly.
std::vector<std::uint8_t> const vec3 = json::to_bjdata(j2, false, false);
CHECK(json::from_bjdata(vec3) == j2);
}
SECTION("ndarray whose dimensions overflow stays as object")
{
// the product of the dimensions wraps around std::size_t to 0
// and so matches the size of the empty _ArrayData_; writing this
// as an ndarray would announce an element count no reader can
// honor, so it has to stay a plain object
json j_overflow = json({{"_ArrayData_", json::array()}, {"_ArraySize_", {9223372036854775808ull, 2}}, {"_ArrayType_", "uint8"}});
CHECK(json::from_bjdata(json::to_bjdata(j_overflow), true, true) == j_overflow);
// a single dimension that does not fit into std::size_t is
// rejected for the same reason (only observable where
// std::size_t is narrower than 64 bit)
json j_huge = json({{"_ArrayData_", json::array()}, {"_ArraySize_", {18446744073709551615ull, 2}}, {"_ArrayType_", "uint8"}});
CHECK(json::from_bjdata(json::to_bjdata(j_huge), true, true) == j_huge);
// a well-formed ndarray is still encoded as one
json j_ok = json({{"_ArrayData_", {1, 2, 3, 4, 5, 6}}, {"_ArraySize_", {2, 3}}, {"_ArrayType_", "uint8"}});
CHECK(json::to_bjdata(j_ok) == std::vector<uint8_t>({'[', '$', 'U', '#', '[', 'i', 2, 'i', 3, ']', 1, 2, 3, 4, 5, 6}));
CHECK(json::from_bjdata(json::to_bjdata(j_ok), true, true) == j_ok);
}
SECTION("ndarray whose _ArraySize_ is not an array stays as object")
{
// the shape is written verbatim as the header length, so a
// value that is not an array cannot produce a valid one: null
// would emit 'Z' and an object '{', neither of which a reader
// accepts after '#'. Both have to stay plain objects.
json const j_null = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", nullptr}, {"_ArrayData_", json::array()}});
const auto out_null = json::to_bjdata(j_null);
CHECK(out_null.at(0) == '{');
CHECK(json::from_bjdata(out_null) == j_null);
// an object shape passes the per-entry check by iterating its
// values rather than dimensions, so it needs rejecting too
json const j_obj = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {{"a", 1}}}, {"_ArrayData_", {1}}});
const auto out_obj = json::to_bjdata(j_obj);
CHECK(out_obj.at(0) == '{');
CHECK(json::from_bjdata(out_obj) == j_obj);
// a scalar shape is not a dimension list either
json const j_num = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", 1}, {"_ArrayData_", {1}}});
const auto out_num = json::to_bjdata(j_num);
CHECK(out_num.at(0) == '{');
CHECK(json::from_bjdata(out_num) == j_num);
// OSS-Fuzz issue 474400817: an empty object _ArraySize_ was
// written as the ND-array header length, which from_bjdata()
// could not read back
const std::vector<uint8_t> input =
{
'[', '{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '{', '}', '}', ']'
};
const json j1 = json::from_bjdata(input);
CHECK(j1 == json::parse(R"([{"_ArrayType_":"int16","_ArraySize_":{},"_ArrayData_":null}])"));
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(json::to_bjdata(j1, false, false)));
CHECK(j2 == j1);
}
SECTION("ndarray with out-of-range _ArrayData_ elements stays as object")
{
// each element is cast to the (possibly narrower) C++ type
// named by _ArrayType_ before being written; a value that
// does not fit that type would silently wrap instead of
// being reported, so such an object falls back to a plain
// object encoding that still round-trips (see GitHub issue #5403)
// an unsigned element that does not fit uint8
json const j_uint8 = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, 256}}});
const auto out_uint8 = json::to_bjdata(j_uint8);
CHECK(out_uint8.at(0) == '{');
CHECK(json::from_bjdata(out_uint8) == j_uint8);
// a signed element that does not fit int8
json const j_int8 = json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, 200}}});
const auto out_int8 = json::to_bjdata(j_int8);
CHECK(out_int8.at(0) == '{');
CHECK(json::from_bjdata(out_int8) == j_int8);
// a negative element is likewise out of range for an
// unsigned _ArrayType_
json const j_uint16_neg = json({{"_ArrayType_", "uint16"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1, -1}}});
const auto out_uint16_neg = json::to_bjdata(j_uint16_neg);
CHECK(out_uint16_neg.at(0) == '{');
CHECK(json::from_bjdata(out_uint16_neg) == j_uint16_neg);
// a double element that overflows to infinity when narrowed
// to the "single" (float) precision named by _ArrayType_
json const j_single = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, 1e40}}});
const auto out_single = json::to_bjdata(j_single);
CHECK(out_single.at(0) == '{');
CHECK(json::from_bjdata(out_single) == j_single);
// in-range boundary values still use the compact ndarray encoding
json const j_uint8_ok = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {0, 255}}});
CHECK(json::to_bjdata(j_uint8_ok) == std::vector<uint8_t>({'[', '$', 'U', '#', '[', 'i', 2, 'i', 1, ']', 0, 255}));
json const j_int8_ok = json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {-128, 127}}});
CHECK(json::to_bjdata(j_int8_ok) == std::vector<uint8_t>({'[', '$', 'i', '#', '[', 'i', 2, 'i', 1, ']', 0x80, 0x7F}));
json const j_single_ok = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5, -1.5}}});
const auto out_single_ok = json::to_bjdata(j_single_ok);
CHECK(out_single_ok.at(0) == '[');
CHECK(json::from_bjdata(out_single_ok) == json({{"_ArrayType_", "single"}, {"_ArraySize_", {2, 1}}, {"_ArrayData_", {1.5f, -1.5f}}}));
}
SECTION("ndarray that would not be read back as an annotated object stays as object")
{
// the reader only restores an annotated object from an ND-array
// with at least two non-zero dimensions that is not a 1xN row
// vector; any other shape is read back as a plain array. Writing
// such an object as an ND-array would drop its annotation, so it
// falls back to a plain object encoding that round-trips.
for (const char* text :
{
R"({"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":[]})",
R"({"_ArrayType_":"int16","_ArraySize_":[2],"_ArrayData_":[1,2]})",
R"({"_ArrayType_":"int16","_ArraySize_":[1,2],"_ArrayData_":[1,2]})",
R"({"_ArrayType_":"int16","_ArraySize_":[0],"_ArrayData_":[]})",
R"({"_ArrayType_":"int16","_ArraySize_":[2,0],"_ArrayData_":[]})",
R"({"_ArrayType_":"int16","_ArraySize_":[0,2],"_ArrayData_":[]})"
})
{
CAPTURE(text)
const json j = json::parse(text);
for (const bool use_size :
{
false, true
})
{
const auto out = json::to_bjdata(j, use_size, use_size);
CHECK(out.at(0) == '{');
CHECK(json::from_bjdata(out) == j);
}
}
// a genuine ND-array still uses the compact encoding and round-trips
const json j_2d = json::parse(R"({"_ArrayType_":"int16","_ArraySize_":[2,1],"_ArrayData_":[1,2]})");
const auto out_2d = json::to_bjdata(j_2d);
CHECK(out_2d.at(0) == '[');
CHECK(json::from_bjdata(out_2d) == j_2d);
}
SECTION("ndarray with non-array _ArrayData_ stays as object")
{
// the elements are written from _ArrayData_ as a flat list, so it
// has to be an array: null has size 0, any other scalar has size 1,
// and iterating an object visits its values, so each of these could
// match the dimensions and be encoded as an unrelated ND-array
for (const char* text :
{
R"({"_ArrayType_":"int16","_ArraySize_":[2,1],"_ArrayData_":null})",
R"({"_ArrayType_":"int16","_ArraySize_":[2,1],"_ArrayData_":{"a":1,"b":2}})",
R"({"_ArrayType_":"int16","_ArraySize_":[1],"_ArrayData_":5})",
R"({"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null})"
})
{
CAPTURE(text)
const json j = json::parse(text);
const auto out = json::to_bjdata(j);
CHECK(out.at(0) == '{');
CHECK(json::from_bjdata(out) == j);
}
// OSS-Fuzz issue 563659413: an empty binary _ArraySize_ is written
// as a plain object and read back as an empty array, after which
// the object with a null _ArrayData_ was encoded as an empty
// ND-array and re-read as [], so a second round trip lost the value
const std::vector<uint8_t> input =
{
'{', 'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'D', 'a', 't', 'a', '_', 'Z',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'T', 'y', 'p', 'e', '_', 'S', 'i', 5, 'i', 'n', 't', '1', '6',
'U', 11, '_', 'A', 'r', 'r', 'a', 'y', 'S', 'i', 'z', 'e', '_', '[', '$', 'B', '#', '[', ']', '}'
};
const json j1 = json::from_bjdata(input);
const json j2 = json::from_bjdata(json::to_bjdata(j1, false, false));
CHECK(j2 == json::parse(R"({"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null})"));
CHECK(json::from_bjdata(json::to_bjdata(j2, false, false)) == j2);
}
SECTION("ndarray with _ArrayType_ \"byte\" is gated by the BJData draft version")
{
// the 'B' (byte) marker used by _ArrayType_ "byte" is only defined
// by BJData Draft 3; Draft 2 (the default) has no such marker, so
// emitting it unconditionally produced a stream that a Draft 2
// reader could not parse as intended (see GitHub issue #5404).
// Two dimensions are used so that a successfully written ndarray
// round-trips back into the annotated object (a single dimension
// is, by the BJData ndarray convention, read back as a plain
// binary value rather than the annotated object, same as every
// other single-dimension ndarray of a non-"byte" type is read
// back as a plain array instead of the annotated object).
json const j_byte = json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
// default (Draft 2): falls back to a plain object and round-trips
const auto out_draft2 = json::to_bjdata(j_byte);
CHECK(out_draft2.at(0) == '{');
CHECK(json::from_bjdata(out_draft2) == j_byte);
// explicit Draft 2: same as the default
const auto out_draft2_explicit = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft2);
CHECK(out_draft2_explicit.at(0) == '{');
CHECK(json::from_bjdata(out_draft2_explicit) == j_byte);
// Draft 3 explicitly selected: still uses the compact 'B' ndarray encoding
const auto out_draft3 = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft3);
CHECK(out_draft3 == std::vector<uint8_t>({'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5, 6}));
CHECK(json::from_bjdata(out_draft3) == j_byte);
}
}
}
SECTION("parse errors")
{
SECTION("empty byte vector")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(std::vector<uint8_t>()),
"[json.exception.parse_error.110] parse error at byte 1: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
}
SECTION("char")
{
SECTION("eof after C byte")
{
std::vector<uint8_t> const v = {'C'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData char: unexpected end of input", json::parse_error&);
}
SECTION("byte out of range")
{
std::vector<uint8_t> const v = {'C', 130};
json _;
CHECK_THROWS_WITH(_ = json::from_bjdata(v), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82");
}
}
SECTION("byte")
{
SECTION("parse bjdata markers in ubjson")
{
std::vector<uint8_t> const v = {'B', 1};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.112] parse error at byte 1: syntax error while parsing UBJSON value: invalid byte: 0x42", json::parse_error&);
}
}
SECTION("strings")
{
SECTION("eof after S byte")
{
std::vector<uint8_t> const v = {'S'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
}
SECTION("invalid byte")
{
std::vector<uint8_t> const v = {'S', '1', 'a'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData string: expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x31", json::parse_error&);
}
SECTION("negative length")
{
json _;
std::vector<uint8_t> const vi = {'S', 'i', 0xFF};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData string: string length must not be negative", json::parse_error&);
CHECK(json::from_bjdata(vi, true, false).is_discarded());
std::vector<uint8_t> const vl = {'S', 'l', 0xFF, 0xFF, 0xFF, 0xFF};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vl), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData string: string length must not be negative", json::parse_error&);
CHECK(json::from_bjdata(vl, true, false).is_discarded());
}
SECTION("parse bjdata markers in ubjson")
{
// create a single-character string for all number types
std::vector<uint8_t> const s_u = {'S', 'u', 1, 0, 'a'};
std::vector<uint8_t> const s_m = {'S', 'm', 1, 0, 0, 0, 'a'};
std::vector<uint8_t> const s_M = {'S', 'M', 1, 0, 0, 0, 0, 0, 0, 0, 'a'};
json _;
// check if string is parsed correctly to "a"
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(s_u), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x75", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(s_m), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x6D", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(s_M), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON string: expected length type specification (U, i, I, l, L); last byte: 0x4D", json::parse_error&);
}
}
SECTION("array")
{
SECTION("optimized array: no size following type")
{
std::vector<uint8_t> const v = {'[', '$', 'i', 2};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 4: syntax error while parsing BJData size: expected '#' after type information; last byte: 0x02", json::parse_error&);
}
SECTION("optimized array: negative size")
{
std::vector<uint8_t> const v1 = {'[', '#', 'i', 0xF1};
std::vector<uint8_t> const v2 = {'[', '$', 'I', '#', 'i', 0xF2};
std::vector<uint8_t> const v3 = {'[', '$', 'I', '#', '[', 'i', 0xF4, 'i', 0x02, ']'};
std::vector<uint8_t> const v4 = {'[', '$', 0xF6, '#', 'i', 0xF7};
std::vector<uint8_t> const v5 = {'[', '$', 'I', '#', '[', 'i', 0xF5, 'i', 0xF1, ']'};
std::vector<uint8_t> const v6 = {'[', '#', '[', 'i', 0xF3, 'i', 0x02, ']'};
std::vector<uint8_t> const vI = {'[', '#', 'I', 0x00, 0xF1};
std::vector<uint8_t> const vl = {'[', '#', 'l', 0x00, 0x00, 0x00, 0xF2};
std::vector<uint8_t> const vL = {'[', '#', 'L', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xF3};
std::vector<uint8_t> const vM = {'[', '$', 'M', '#', '[', 'I', 0x00, 0x20, 'M', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0xFF, ']'};
std::vector<uint8_t> const vMX = {'[', '$', 'U', '#', '[', 'M', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'U', 0x01, ']'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v1), "[json.exception.parse_error.113] parse error at byte 4: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(v1, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v2), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(v2, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v3), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(v3, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v4), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(v4, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v5), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(v5, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v6), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(v6, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vI), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(vI, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vl), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(vl, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.parse_error.113] parse error at byte 11: syntax error while parsing BJData size: count in an optimized container must be positive", json::parse_error&);
CHECK(json::from_bjdata(vL, true, false).is_discarded());
#if SIZE_MAX != 0xffffffff
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: excessive ndarray size caused overflow", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
#endif
CHECK(json::from_bjdata(vM, true, false).is_discarded());
#if SIZE_MAX != 0xffffffff
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vMX), "[json.exception.out_of_range.408] syntax error while parsing BJData size: excessive ndarray size caused overflow", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vMX), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
#endif
CHECK(json::from_bjdata(vMX, true, false).is_discarded());
}
SECTION("optimized array: integer value overflow")
{
#if SIZE_MAX == 0xffffffff
std::vector<uint8_t> const vL = {'[', '#', 'L', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7F};
std::vector<uint8_t> const vM = {'[', '$', 'M', '#', '[', 'I', 0x00, 0x20, 'M', 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0xFF, ']'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
CHECK(json::from_bjdata(vL, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
CHECK(json::from_bjdata(vM, true, false).is_discarded());
#endif
}
SECTION("overflow detection in dimension multiplication")
{
// Simple SAX handler just to monitor if overflow is detected
struct SimpleOverflowSaxHandler : public nlohmann::json_sax<json>
{
bool overflow_detected = false;
// Implement all required virtual methods with minimal implementation
bool null() override
{
return true;
}
bool boolean(bool /*val*/) override
{
return true;
}
bool number_integer(json::number_integer_t /*val*/) override
{
return true;
}
bool number_unsigned(json::number_unsigned_t /*val*/) override
{
return true;
}
bool number_float(json::number_float_t /*val*/, const std::string& /*s*/) override
{
return true;
}
bool string(std::string& /*val*/) override
{
return true;
}
bool binary(json::binary_t& /*val*/) override
{
return true;
}
bool start_object(std::size_t /*elements*/) override
{
return true;
}
bool key(std::string& /*val*/) override
{
return true;
}
bool end_object() override
{
return true;
}
bool start_array(std::size_t /*elements*/) override
{
return true;
}
bool end_array() override
{
return true;
}
// This is the only method we care about - detecting error 408
bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const json::exception& ex) override
{
if (ex.id == 408)
{
overflow_detected = true;
}
return false;
}
};
// Create BJData payload with overflow-causing dimensions (2^32+1) × (2^32)
const std::vector<uint8_t> bjdata_payload =
{
0x5B, // '[' start array
0x24, 0x55, // '$', 'U' (type uint8)
0x23, 0x5B, // '#', '[' (dimensions array)
0x4D, // 'M' (uint64)
0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, // 2^32 + 1 (4294967297) as little-endian
0x4D, // 'M' (uint64)
0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, // 2^32 (4294967296) as little-endian
0x5D // ']' end dimensions
// No data - we don't need it for this test, we just want to hit the overflow check
};
// Test with overflow dimensions using SAX parser
{
SimpleOverflowSaxHandler handler;
const auto result = json::sax_parse(bjdata_payload, &handler,
nlohmann::detail::input_format_t::bjdata, false);
// Should detect overflow
CHECK(handler.overflow_detected == true);
CHECK(result == false);
}
// Test with DOM parser (should throw)
{
json _;
CHECK_THROWS_AS(_ = json::from_bjdata(bjdata_payload), json::out_of_range);
}
// Test with normal dimensions
const std::vector<uint8_t> normal_payload =
{
0x5B, // '[' start array
0x24, 0x55, // '$', 'U' (type uint8)
0x23, 0x5B, // '#', '[' (dimensions array)
0x55, 0x02, // 'U', 2 (uint8)
0x55, 0x03, // 'U', 3 (uint8)
0x5D, // ']' end dimensions
// 6 data bytes for a 2×3 array (enough to avoid EOF but not entire array)
0x01, 0x02, 0x03, 0x04, 0x05, 0x06
};
// For normal dimensions, overflow should not be detected
{
SimpleOverflowSaxHandler handler;
const auto result = json::sax_parse(normal_payload, &handler,
nlohmann::detail::input_format_t::bjdata, false);
CHECK(handler.overflow_detected == false);
CHECK(result == true);
}
}
SECTION("do not accept NTFZ markers in ndarray optimized type (with count)")
{
json _;
std::vector<uint8_t> const v_N = {'[', '$', 'N', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
std::vector<uint8_t> const v_T = {'[', '$', 'T', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
std::vector<uint8_t> const v_F = {'[', '$', 'F', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
std::vector<uint8_t> const v_Z = {'[', '$', 'Z', '#', '[', '#', 'i', 2, 'i', 1, 'i', 2};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_N), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x4E is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_N, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_T), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x54 is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_T, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_F), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x46 is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_F, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_Z), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5A is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_Z, true, false).is_discarded());
}
SECTION("do not accept NTFZ markers in ndarray optimized type (without count)")
{
json _;
std::vector<uint8_t> const v_N = {'[', '$', 'N', '#', '[', 'i', 1, 'i', 2, ']'};
std::vector<uint8_t> const v_T = {'[', '$', 'T', '#', '[', 'i', 1, 'i', 2, ']'};
std::vector<uint8_t> const v_F = {'[', '$', 'F', '#', '[', 'i', 1, 'i', 2, ']'};
std::vector<uint8_t> const v_Z = {'[', '$', 'Z', '#', '[', 'i', 1, 'i', 2, ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_N), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x4E is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_N, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_T), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x54 is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_T, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_F), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x46 is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_F, true, false).is_discarded());
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v_Z), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5A is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v_Z, true, false).is_discarded());
}
}
SECTION("strings")
{
std::vector<uint8_t> const vS = {'S'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vS, true, false).is_discarded());
std::vector<uint8_t> const v = {'S', 'i', '2', 'a'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 5: syntax error while parsing BJData string: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v, true, false).is_discarded());
std::vector<uint8_t> const vC = {'C'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vC), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing BJData char: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vC, true, false).is_discarded());
}
SECTION("sizes")
{
std::vector<uint8_t> const vU = {'[', '#', 'U'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vU, true, false).is_discarded());
std::vector<uint8_t> const vi = {'[', '#', 'i'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vi, true, false).is_discarded());
std::vector<uint8_t> const vI = {'[', '#', 'I'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vI), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vI, true, false).is_discarded());
std::vector<uint8_t> const vu = {'[', '#', 'u'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vu), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vu, true, false).is_discarded());
std::vector<uint8_t> const vl = {'[', '#', 'l'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vl), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vl, true, false).is_discarded());
std::vector<uint8_t> const vm = {'[', '#', 'm'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vm), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vm, true, false).is_discarded());
std::vector<uint8_t> const vL = {'[', '#', 'L'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vL, true, false).is_discarded());
std::vector<uint8_t> const vM = {'[', '#', 'M'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vM, true, false).is_discarded());
std::vector<uint8_t> const v0 = {'[', '#', 'T', ']'};
CHECK_THROWS_WITH(_ = json::from_bjdata(v0), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x54");
CHECK(json::from_bjdata(v0, true, false).is_discarded());
std::vector<uint8_t> const vB = {'[', '#', 'B', ']'};
CHECK_THROWS_WITH(_ = json::from_bjdata(vB), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x42");
CHECK(json::from_bjdata(v0, true, false).is_discarded());
}
SECTION("parse bjdata markers as array size in ubjson")
{
json _;
std::vector<uint8_t> const vu = {'[', '#', 'u'};
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vu), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x75", json::parse_error&);
CHECK(json::from_ubjson(vu, true, false).is_discarded());
std::vector<uint8_t> const vm = {'[', '#', 'm'};
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vm), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x6D", json::parse_error&);
CHECK(json::from_ubjson(vm, true, false).is_discarded());
std::vector<uint8_t> const vM = {'[', '#', 'M'};
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vM), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x4D", json::parse_error&);
CHECK(json::from_ubjson(vM, true, false).is_discarded());
std::vector<uint8_t> const v0 = {'[', '#', '['};
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v0), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x5B", json::parse_error&);
CHECK(json::from_ubjson(v0, true, false).is_discarded());
}
SECTION("types")
{
std::vector<uint8_t> const v0 = {'[', '$'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v0), "[json.exception.parse_error.110] parse error at byte 3: syntax error while parsing BJData type: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v0, true, false).is_discarded());
std::vector<uint8_t> const vi = {'[', '$', '#'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vi, true, false).is_discarded());
std::vector<uint8_t> const vU = {'[', '$', 'U'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vU, true, false).is_discarded());
std::vector<uint8_t> const v1 = {'[', '$', '['};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v1), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5B is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v1, true, false).is_discarded());
}
SECTION("arrays")
{
std::vector<uint8_t> const vST = {'[', '$', 'i', '#', 'i', 2, 1};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vST, true, false).is_discarded());
std::vector<uint8_t> const vS = {'[', '#', 'i', 2, 'i', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vS, true, false).is_discarded());
std::vector<uint8_t> const v = {'[', 'i', 2, 'i', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 6: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v, true, false).is_discarded());
}
SECTION("ndarrays")
{
std::vector<uint8_t> const vST = {'[', '$', 'i', '#', '[', '$', 'i', '#'};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0xFF", json::parse_error&);
CHECK(json::from_bjdata(vST, true, false).is_discarded());
std::vector<uint8_t> const v = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 1, 2};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 13: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v, true, false).is_discarded());
std::vector<uint8_t> const vS0 = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS0), "[json.exception.parse_error.110] parse error at byte 12: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vS0, true, false).is_discarded());
std::vector<uint8_t> const vS = {'[', '$', 'i', '#', '[', '#', 'i', 2, 1, 2, 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x01", json::parse_error&);
CHECK(json::from_bjdata(vS, true, false).is_discarded());
std::vector<uint8_t> const vT = {'[', '$', 'i', '#', '[', 'i', 2, 'i'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vT), "[json.exception.parse_error.110] parse error at byte 9: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vT, true, false).is_discarded());
std::vector<uint8_t> const vT0 = {'[', '$', 'i', '#', '[', 'i'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vT0), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vT0, true, false).is_discarded());
std::vector<uint8_t> const vu = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'u', 1, 0};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vu), "[json.exception.parse_error.110] parse error at byte 12: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vu, true, false).is_discarded());
std::vector<uint8_t> const vm = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'm', 1, 0, 0, 0};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vm), "[json.exception.parse_error.110] parse error at byte 14: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vm, true, false).is_discarded());
std::vector<uint8_t> const vM = {'[', '$', 'i', '#', '[', '$', 'i', '#', 'M', 1, 0, 0, 0, 0, 0, 0, 0};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.parse_error.110] parse error at byte 18: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vM, true, false).is_discarded());
std::vector<uint8_t> const vU = {'[', '$', 'U', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 18: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vU, true, false).is_discarded());
std::vector<uint8_t> const vB = {'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vU), "[json.exception.parse_error.110] parse error at byte 18: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vU, true, false).is_discarded());
std::vector<uint8_t> const vT1 = {'[', '$', 'T', '#', '[', '$', 'i', '#', 'i', 2, 2, 3};
CHECK(json::from_bjdata(vT1, true, false).is_discarded());
std::vector<uint8_t> const vh = {'[', '$', 'h', '#', '[', '$', 'i', '#', 'i', 2, 2, 3};
CHECK(json::from_bjdata(vh, true, false).is_discarded());
std::vector<uint8_t> const vR = {'[', '$', 'i', '#', '[', 'i', 1, '[', ']', ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR), "[json.exception.parse_error.113] parse error at byte 8: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR, true, false).is_discarded());
std::vector<uint8_t> const vRo = {'[', '$', 'i', '#', '[', 'i', 0, '{', '}', ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vRo), "[json.exception.parse_error.113] parse error at byte 8: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x7B", json::parse_error&);
CHECK(json::from_bjdata(vRo, true, false).is_discarded());
std::vector<uint8_t> const vR1 = {'[', '$', 'i', '#', '[', '[', 'i', 1, ']', ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR1), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR1, true, false).is_discarded());
// a dimension vector that opens another one is rejected where the
// nested '[' is read, rather than after it has been descended into
std::vector<uint8_t> const vR2 = {'[', '$', 'i', '#', '[', '#', '[', 'i', 1, ']', ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR2), "[json.exception.parse_error.113] parse error at byte 7: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR2, true, false).is_discarded());
std::vector<uint8_t> const vR3 = {'[', '#', '[', 'i', '2', 'i', 2, ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR3), "[json.exception.parse_error.112] parse error at byte 8: syntax error while parsing BJData size: ndarray requires both type and size", json::parse_error&);
CHECK(json::from_bjdata(vR3, true, false).is_discarded());
std::vector<uint8_t> const vR4 = {'[', '$', 'i', '#', '[', '$', 'i', '#', '[', 'i', 1, ']', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR4), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR4, true, false).is_discarded());
std::vector<uint8_t> const vR5 = {'[', '$', 'i', '#', '[', '[', '[', ']', ']', ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR5), "[json.exception.parse_error.113] parse error at byte 6: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR5, true, false).is_discarded());
std::vector<uint8_t> const vR6 = {'[', '$', 'i', '#', '[', '$', 'i', '#', '[', 'i', '2', 'i', 2, ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vR6), "[json.exception.parse_error.113] parse error at byte 9: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vR6, true, false).is_discarded());
std::vector<uint8_t> const vH = {'[', 'H', '[', '#', '[', '$', 'i', '#', '[', 'i', '2', 'i', 2, ']'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vH), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vH, true, false).is_discarded());
// Every "#[" of this chain used to open another dimension vector
// and cost several stack frames before anything was rejected, so a
// long enough chain crashed the process (see #5104). The nested
// vector is refused where it is read, so the length is irrelevant.
std::vector<uint8_t> vRdeep = {'['};
for (std::size_t i = 0; i < 100000; ++i)
{
vRdeep.push_back('#');
vRdeep.push_back('[');
}
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vRdeep), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData size: ndarray dimensional vector is not allowed", json::parse_error&);
CHECK(json::from_bjdata(vRdeep, true, false).is_discarded());
}
SECTION("objects")
{
std::vector<uint8_t> const vST = {'{', '$', 'i', '#', 'i', 2, 'i', 1, 'a', 1};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST), "[json.exception.parse_error.110] parse error at byte 11: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vST, true, false).is_discarded());
std::vector<uint8_t> const vT = {'{', '$', 'i', 'i', 1, 'a', 1};
CHECK_THROWS_WITH(_ = json::from_bjdata(vT), "[json.exception.parse_error.112] parse error at byte 4: syntax error while parsing BJData size: expected '#' after type information; last byte: 0x69");
CHECK(json::from_bjdata(vT, true, false).is_discarded());
std::vector<uint8_t> const vS = {'{', '#', 'i', 2, 'i', 1, 'a', 'i', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vS), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vS, true, false).is_discarded());
std::vector<uint8_t> const v = {'{', 'i', 1, 'a', 'i', 1};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.110] parse error at byte 7: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v, true, false).is_discarded());
std::vector<uint8_t> const v2 = {'{', 'i', 1, 'a', 'i', 1, 'i'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v2), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v2, true, false).is_discarded());
std::vector<uint8_t> const v3 = {'{', 'i', 1, 'a'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v3), "[json.exception.parse_error.110] parse error at byte 5: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(v3, true, false).is_discarded());
std::vector<uint8_t> const vST1 = {'{', '$', 'd', '#', 'i', 2, 'i', 1, 'a'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST1), "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing BJData number: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vST1, true, false).is_discarded());
std::vector<uint8_t> const vST2 = {'{', '#', 'i', 2, 'i', 1, 'a'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vST2), "[json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
CHECK(json::from_bjdata(vST2, true, false).is_discarded());
std::vector<uint8_t> const vO = {'{', '#', '[', 'i', 2, 'i', 1, ']', 'i', 1, 'a', 'i', 1, 'i', 1, 'b', 'i', 2};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vO), "[json.exception.parse_error.112] parse error at byte 8: syntax error while parsing BJData size: ndarray requires both type and size", json::parse_error&);
CHECK(json::from_bjdata(vO, true, false).is_discarded());
std::vector<uint8_t> const vO2 = {'{', '$', 'i', '#', '[', 'i', 2, 'i', 1, ']', 'i', 1, 'a', 1, 'i', 1, 'b', 2};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vO2), "[json.exception.parse_error.112] parse error at byte 10: syntax error while parsing BJData object: BJData object does not support ND-array size in optimized format", json::parse_error&);
CHECK(json::from_bjdata(vO2, true, false).is_discarded());
}
}
SECTION("writing optimized values")
{
SECTION("integer")
{
SECTION("array of i")
{
json const j = {1, -1};
std::vector<uint8_t> const expected = {'[', '$', 'i', '#', 'i', 2, 1, 0xff};
CHECK(json::to_bjdata(j, true, true) == expected);
}
SECTION("array of U")
{
json const j = {200, 201};
std::vector<uint8_t> const expected = {'[', '$', 'U', '#', 'i', 2, 0xC8, 0xC9};
CHECK(json::to_bjdata(j, true, true) == expected);
}
SECTION("array of I")
{
json const j = {30000, -30000};
std::vector<uint8_t> const expected = {'[', '$', 'I', '#', 'i', 2, 0x30, 0x75, 0xd0, 0x8a};
CHECK(json::to_bjdata(j, true, true) == expected);
}
SECTION("array of u")
{
json const j = {50000, 50001};
std::vector<uint8_t> const expected = {'[', '$', 'u', '#', 'i', 2, 0x50, 0xC3, 0x51, 0xC3};
CHECK(json::to_bjdata(j, true, true) == expected);
}
SECTION("array of l")
{
json const j = {70000, -70000};
std::vector<uint8_t> const expected = {'[', '$', 'l', '#', 'i', 2, 0x70, 0x11, 0x01, 0x00, 0x90, 0xEE, 0xFE, 0xFF};
CHECK(json::to_bjdata(j, true, true) == expected);
}
SECTION("array of m")
{
json const j = {3147483647, 3147483648};
std::vector<uint8_t> const expected = {'[', '$', 'm', '#', 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0x00, 0xCA, 0x9A, 0xBB};
CHECK(json::to_bjdata(j, true, true) == expected);
}
SECTION("array of L")
{
json const j = {5000000000, -5000000000};
std::vector<uint8_t> const expected = {'[', '$', 'L', '#', 'i', 2, 0x00, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00, 0x00, 0x0E, 0xFA, 0xD5, 0xFE, 0xFF, 0xFF, 0xFF};
CHECK(json::to_bjdata(j, true, true) == expected);
}
}
SECTION("unsigned integer")
{
SECTION("array of i")
{
json const j = {1u, 2u};
std::vector<uint8_t> const expected = {'[', '$', 'i', '#', 'i', 2, 1, 2};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'i', 1, 'i', 2};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of U")
{
json const j = {200u, 201u};
std::vector<uint8_t> const expected = {'[', '$', 'U', '#', 'i', 2, 0xC8, 0xC9};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'U', 0xC8, 'U', 0xC9};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of I")
{
json const j = {30000u, 30001u};
std::vector<uint8_t> const expected = {'[', '$', 'I', '#', 'i', 2, 0x30, 0x75, 0x31, 0x75};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'I', 0x30, 0x75, 'I', 0x31, 0x75};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of u")
{
json const j = {50000u, 50001u};
std::vector<uint8_t> const expected = {'[', '$', 'u', '#', 'i', 2, 0x50, 0xC3, 0x51, 0xC3};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'u', 0x50, 0xC3, 'u', 0x51, 0xC3};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of l")
{
json const j = {70000u, 70001u};
std::vector<uint8_t> const expected = {'[', '$', 'l', '#', 'i', 2, 0x70, 0x11, 0x01, 0x00, 0x71, 0x11, 0x01, 0x00};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'l', 0x70, 0x11, 0x01, 0x00, 'l', 0x71, 0x11, 0x01, 0x00};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of m")
{
json const j = {3147483647u, 3147483648u};
std::vector<uint8_t> const expected = {'[', '$', 'm', '#', 'i', 2, 0xFF, 0xC9, 0x9A, 0xBB, 0x00, 0xCA, 0x9A, 0xBB};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'm', 0xFF, 0xC9, 0x9A, 0xBB, 'm', 0x00, 0xCA, 0x9A, 0xBB};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of L")
{
json const j = {5000000000u, 5000000001u};
std::vector<uint8_t> const expected = {'[', '$', 'L', '#', 'i', 2, 0x00, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00, 0x01, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'L', 0x00, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00, 'L', 0x01, 0xF2, 0x05, 0x2A, 0x01, 0x00, 0x00, 0x00};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
SECTION("array of M")
{
json const j = {10223372036854775807ull, 10223372036854775808ull};
std::vector<uint8_t> const expected = {'[', '$', 'M', '#', 'i', 2, 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 0x00, 0x00, 0x64, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
std::vector<uint8_t> const expected_size = {'[', '#', 'i', 2, 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D, 'M', 0x00, 0x00, 0x64, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D};
CHECK(json::to_bjdata(j, true, true) == expected);
CHECK(json::to_bjdata(j, true) == expected_size);
}
}
}
}
TEST_CASE("BJData input that cannot be read is discarded by every overload")
{
std::vector<std::uint8_t> input = json::to_bjdata(json({{"a", {1, 2}}}));
input.pop_back();
json _;
CHECK_THROWS_AS(_ = json::from_bjdata(input.begin(), input.end()), json::parse_error&);
CHECK(json::from_bjdata(input, true, false).is_discarded());
CHECK(json::from_bjdata(input.begin(), input.end(), true, false).is_discarded());
}
TEST_CASE("BJData SAX parsing stops at every event")
{
// Containers are opened and closed by the loop that reads them; a SAX
// handler that rejects any event - including the end of a nested
// container - must stop the parse right there.
const auto count_events = [](const std::vector<std::uint8_t>& input)
{
int events = 0;
while (true)
{
SaxCountdown scp(events);
if (json::sax_parse(input, &scp, json::input_format_t::bjdata))
{
return events;
}
++events;
REQUIRE(events < 1000);
}
};
// 20 events: every container kind closes inside another one
const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
CHECK(count_events(json::to_bjdata(j)) == 20);
CHECK(count_events(json::to_bjdata(j, true)) == 20);
CHECK(count_events(json::to_bjdata(j, true, true)) == 20);
// an ND-array is announced as an annotated object: start_object, then
// _ArrayType_, _ArraySize_ and _ArrayData_ with its elements
const json ndarray = json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})");
CHECK(count_events(json::to_bjdata(ndarray, true, true)) == 16);
}
TEST_CASE("issue #5405 - array reserve for definite-length BJData arrays")
{
#if !defined(JSON_NOEXCEPTION)
// this SECTION relies on catching a thrown exception to distinguish
// which of two acceptable, bounded rejections a hostile header took;
// under JSON_NOEXCEPTION, JSON_THROW never produces a catchable C++
// exception (it aborts instead), so this cannot be tested that way here
SECTION("a huge claimed length with no element data must not over-allocate")
{
// optimized form [$type#count: type 'i' (int8), count as a four-byte
// little-endian 'l' (int32) of 0x7FFFFFFF (2147483647), but no
// element data at all. max_size() for a std::vector is far larger
// than this count, so it does not reject the header outright; the
// (capped) reservation must not attempt to allocate space for
// billions of elements before the missing data is detected.
json _;
const std::vector<uint8_t> input = {'[', '$', 'i', '#', 'l', 0xFF, 0xFF, 0xFF, 0x7F};
// On a platform where std::vector<json>::max_size() is smaller than
// the claimed count (e.g. 32-bit, where max_size() is bounded by a
// 32-bit SIZE_MAX divided by sizeof(json)), the SAX consumer's own
// check rejects the header outright (out_of_range.408, with the
// claimed count in the message) instead of accepting it and only
// finding it short of data once the (capped) reservation looks for
// element bytes that were never provided (parse_error.110). Either
// is an acceptable, bounded rejection of the hostile header -- the
// property under test is that no path attempts to allocate space
// for billions of elements.
bool threw = false;
try
{
_ = json::from_bjdata(input);
}
catch (const json::parse_error& e)
{
threw = true;
CHECK(e.id == 110);
CHECK(std::string(e.what()) == "[json.exception.parse_error.110] parse error at byte 10: syntax error while parsing BJData number: unexpected end of input");
}
catch (const json::out_of_range& e)
{
threw = true;
CHECK(e.id == 408);
CHECK(std::string(e.what()).find("excessive array size") != std::string::npos);
}
CHECK(threw);
// json_sax_dom_parser::start_array()'s max_size() check (unlike the
// scanner's own parse_error path) throws unconditionally via
// JSON_THROW rather than going through sax->parse_error(), so it is
// not gated by allow_exceptions=false on a platform where this
// header hits that check (e.g. 32-bit, see above) -- allow either
// a discarded result or the same out_of_range it throws with
// exceptions enabled.
try
{
CHECK(json::from_bjdata(input, true, false).is_discarded());
}
catch (const json::out_of_range& e)
{
CHECK(e.id == 408);
}
}
#endif
SECTION("arrays of various sizes decode to the same value as before the reserve optimization")
{
for (const auto size :
{
std::size_t{0}, std::size_t{1}, std::size_t{5}, // small
std::size_t{16384}, // exactly at the reserve cap
std::size_t{20000} // above the reserve cap
})
{
CAPTURE(size)
json j = json::array();
for (std::size_t i = 0; i < size; ++i)
{
j.push_back(static_cast<int>(i % 1000));
}
// exercise both the plain and the optimized [$type#count encoding
const auto packed_plain = json::to_bjdata(j);
CHECK(json::from_bjdata(packed_plain) == j);
const auto packed_optimized = json::to_bjdata(j, true, true);
CHECK(json::from_bjdata(packed_optimized) == j);
}
}
SECTION("a user-defined SAX consumer is unaffected by the internal DOM reserve optimization")
{
// the reserve() call is local to json_sax_dom_parser / json_sax_dom_callback_parser;
// a custom SAX consumer that does not touch a DOM array sees identical events
json j = json::array();
for (int i = 0; i < 100; ++i)
{
j.push_back(i);
}
const auto packed = json::to_bjdata(j, true, true);
SaxCountdown scp(1000000); // large enough to never trigger an abort
CHECK(json::sax_parse(packed, &scp, json::input_format_t::bjdata));
}
}
TEST_CASE("Universal Binary JSON Specification Examples 1")
{
SECTION("Null Value")
{
json const j = {{"passcode", nullptr}};
std::vector<uint8_t> v = {'{', 'i', 8, 'p', 'a', 's', 's', 'c', 'o', 'd', 'e', 'Z', '}'};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("No-Op Value")
{
json const j = {"foo", "bar", "baz"};
std::vector<uint8_t> v = {'[', 'S', 'i', 3, 'f', 'o', 'o',
'S', 'i', 3, 'b', 'a', 'r',
'S', 'i', 3, 'b', 'a', 'z', ']'
};
std::vector<uint8_t> const v2 = {'[', 'S', 'i', 3, 'f', 'o', 'o', 'N',
'S', 'i', 3, 'b', 'a', 'r', 'N', 'N', 'N',
'S', 'i', 3, 'b', 'a', 'z', 'N', 'N', ']'
};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
CHECK(json::from_bjdata(v2) == j);
}
SECTION("Boolean Types")
{
json const j = {{"authorized", true}, {"verified", false}};
std::vector<uint8_t> v = {'{', 'i', 10, 'a', 'u', 't', 'h', 'o', 'r', 'i', 'z', 'e', 'd', 'T',
'i', 8, 'v', 'e', 'r', 'i', 'f', 'i', 'e', 'd', 'F', '}'
};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Numeric Types")
{
json j =
{
{"int8", 16},
{"uint8", 255},
{"int16", 32767},
{"uint16", 42767},
{"int32", 2147483647},
{"uint32", 3147483647},
{"int64", 9223372036854775807},
{"uint64", 10223372036854775807ull},
{"float64", 113243.7863123}
};
std::vector<uint8_t> v = {'{',
'i', 7, 'f', 'l', 'o', 'a', 't', '6', '4', 'D', 0xcf, 0x34, 0xbc, 0x94, 0xbc, 0xa5, 0xfb, 0x40,
'i', 5, 'i', 'n', 't', '1', '6', 'I', 0xff, 0x7f,
'i', 5, 'i', 'n', 't', '3', '2', 'l', 0xff, 0xff, 0xff, 0x7f,
'i', 5, 'i', 'n', 't', '6', '4', 'L', 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f,
'i', 4, 'i', 'n', 't', '8', 'i', 16,
'i', 6, 'u', 'i', 'n', 't', '1', '6', 'u', 0x0F, 0xA7,
'i', 6, 'u', 'i', 'n', 't', '3', '2', 'm', 0xFF, 0xC9, 0x9A, 0xBB,
'i', 6, 'u', 'i', 'n', 't', '6', '4', 'M', 0xFF, 0xFF, 0x63, 0xA7, 0xB3, 0xB6, 0xE0, 0x8D,
'i', 5, 'u', 'i', 'n', 't', '8', 'U', 0xff,
'}'
};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Char Type")
{
json const j = {{"rolecode", "a"}, {"delim", ";"}};
std::vector<uint8_t> const v = {'{', 'i', 5, 'd', 'e', 'l', 'i', 'm', 'C', ';', 'i', 8, 'r', 'o', 'l', 'e', 'c', 'o', 'd', 'e', 'C', 'a', '}'};
//CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Byte Type")
{
const auto s = std::vector<std::uint8_t>(
{
static_cast<std::uint8_t>(222),
static_cast<std::uint8_t>(173),
static_cast<std::uint8_t>(190),
static_cast<std::uint8_t>(239)
});
json const j = {{"binary", json::binary(s)}, {"val", 123}};
std::vector<uint8_t> const v = {'{', 'i', 6, 'b', 'i', 'n', 'a', 'r', 'y', '[', '$', 'B', '#', 'i', 4, 222, 173, 190, 239, 'i', 3, 'v', 'a', 'l', 'i', 123, '}'};
//CHECK(json::to_bjdata(j) == v); // 123 value gets encoded as uint8
CHECK(json::from_bjdata(v) == j);
}
SECTION("String Type")
{
SECTION("English")
{
json const j = "hello";
std::vector<uint8_t> v = {'S', 'i', 5, 'h', 'e', 'l', 'l', 'o'};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Russian")
{
json const j = "привет";
std::vector<uint8_t> v = {'S', 'i', 12, 0xD0, 0xBF, 0xD1, 0x80, 0xD0, 0xB8, 0xD0, 0xB2, 0xD0, 0xB5, 0xD1, 0x82};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Russian")
{
json const j = "مرحبا";
std::vector<uint8_t> v = {'S', 'i', 10, 0xD9, 0x85, 0xD8, 0xB1, 0xD8, 0xAD, 0xD8, 0xA8, 0xD8, 0xA7};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
}
SECTION("Array Type")
{
SECTION("size=false type=false")
{
// note the float has been replaced by a double
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
std::vector<uint8_t> v = {'[', 'Z', 'T', 'F', 'L', 0xE9, 0xCB, 0x0C, 0x1D, 0x01, 0x00, 0x00, 0x00, 'D', 0x4e, 0x62, 0x10, 0x58, 0x39, 0x24, 0x63, 0x40, 'S', 'i', 3, 'h', 'a', 'm', ']'};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("size=true type=false")
{
// note the float has been replaced by a double
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
std::vector<uint8_t> v = {'[', '#', 'i', 6, 'Z', 'T', 'F', 'L', 0xE9, 0xCB, 0x0C, 0x1D, 0x01, 0x00, 0x00, 0x00, 'D', 0x4e, 0x62, 0x10, 0x58, 0x39, 0x24, 0x63, 0x40, 'S', 'i', 3, 'h', 'a', 'm'};
CHECK(json::to_bjdata(j, true) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("size=true type=true")
{
// note the float has been replaced by a double
json const j = {nullptr, true, false, 4782345193, 153.132, "ham"};
std::vector<uint8_t> v = {'[', '#', 'i', 6, 'Z', 'T', 'F', 'L', 0xE9, 0xCB, 0x0C, 0x1D, 0x01, 0x00, 0x00, 0x00, 'D', 0x4e, 0x62, 0x10, 0x58, 0x39, 0x24, 0x63, 0x40, 'S', 'i', 3, 'h', 'a', 'm'};
CHECK(json::to_bjdata(j, true, true) == v);
CHECK(json::from_bjdata(v) == j);
}
}
SECTION("Object Type")
{
SECTION("size=false type=false")
{
json j =
{
{
"post", {
{"id", 1137},
{"author", "rkalla"},
{"timestamp", 1364482090592},
{"body", "I totally agree!"}
}
}
};
std::vector<uint8_t> v = {'{', 'i', 4, 'p', 'o', 's', 't', '{',
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
'i', 2, 'i', 'd', 'I', 0x71, 0x04,
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x60, 0x66, 0x78, 0xB1, 0x3D, 0x01, 0x00, 0x00,
'}', '}'
};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("size=true type=false")
{
json j =
{
{
"post", {
{"id", 1137},
{"author", "rkalla"},
{"timestamp", 1364482090592},
{"body", "I totally agree!"}
}
}
};
std::vector<uint8_t> v = {'{', '#', 'i', 1, 'i', 4, 'p', 'o', 's', 't', '{', '#', 'i', 4,
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
'i', 2, 'i', 'd', 'I', 0x71, 0x04,
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x60, 0x66, 0x78, 0xB1, 0x3D, 0x01, 0x00, 0x00,
};
CHECK(json::to_bjdata(j, true) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("size=true type=true")
{
json j =
{
{
"post", {
{"id", 1137},
{"author", "rkalla"},
{"timestamp", 1364482090592},
{"body", "I totally agree!"}
}
}
};
std::vector<uint8_t> v = {'{', '#', 'i', 1, 'i', 4, 'p', 'o', 's', 't', '{', '#', 'i', 4,
'i', 6, 'a', 'u', 't', 'h', 'o', 'r', 'S', 'i', 6, 'r', 'k', 'a', 'l', 'l', 'a',
'i', 4, 'b', 'o', 'd', 'y', 'S', 'i', 16, 'I', ' ', 't', 'o', 't', 'a', 'l', 'l', 'y', ' ', 'a', 'g', 'r', 'e', 'e', '!',
'i', 2, 'i', 'd', 'I', 0x71, 0x04,
'i', 9, 't', 'i', 'm', 'e', 's', 't', 'a', 'm', 'p', 'L', 0x60, 0x66, 0x78, 0xB1, 0x3D, 0x01, 0x00, 0x00,
};
CHECK(json::to_bjdata(j, true, true) == v);
CHECK(json::from_bjdata(v) == j);
}
}
SECTION("Optimized Format")
{
SECTION("Array Example")
{
SECTION("No Optimization")
{
// note the floats have been replaced by doubles
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
std::vector<uint8_t> v = {'[',
'D', 0xb8, 0x1e, 0x85, 0xeb, 0x51, 0xf8, 0x3d, 0x40,
'D', 0xe1, 0x7a, 0x14, 0xae, 0x47, 0x21, 0x3f, 0x40,
'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
'D', 0x81, 0x95, 0x43, 0x8b, 0x6c, 0xe7, 0x00, 0x40,
'D', 0x17, 0xd9, 0xce, 0xf7, 0x53, 0xe3, 0x37, 0x40,
']'
};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Optimized with count")
{
// note the floats have been replaced by doubles
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
std::vector<uint8_t> v = {'[', '#', 'i', 5,
'D', 0xb8, 0x1e, 0x85, 0xeb, 0x51, 0xf8, 0x3d, 0x40,
'D', 0xe1, 0x7a, 0x14, 0xae, 0x47, 0x21, 0x3f, 0x40,
'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
'D', 0x81, 0x95, 0x43, 0x8b, 0x6c, 0xe7, 0x00, 0x40,
'D', 0x17, 0xd9, 0xce, 0xf7, 0x53, 0xe3, 0x37, 0x40,
};
CHECK(json::to_bjdata(j, true) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Optimized with type & count")
{
// note the floats have been replaced by doubles
json const j = {29.97, 31.13, 67.0, 2.113, 23.888};
std::vector<uint8_t> v = {'[', '$', 'D', '#', 'i', 5,
0xb8, 0x1e, 0x85, 0xeb, 0x51, 0xf8, 0x3d, 0x40,
0xe1, 0x7a, 0x14, 0xae, 0x47, 0x21, 0x3f, 0x40,
0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
0x81, 0x95, 0x43, 0x8b, 0x6c, 0xe7, 0x00, 0x40,
0x17, 0xd9, 0xce, 0xf7, 0x53, 0xe3, 0x37, 0x40,
};
CHECK(json::to_bjdata(j, true, true) == v);
CHECK(json::from_bjdata(v) == j);
}
}
SECTION("Object Example")
{
SECTION("No Optimization")
{
// note the floats have been replaced by doubles
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
std::vector<uint8_t> v = {'{',
'i', 3, 'a', 'l', 't', 'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
'i', 3, 'l', 'a', 't', 'D', 0x60, 0xe5, 0xd0, 0x22, 0xdb, 0xf9, 0x3d, 0x40,
'i', 4, 'l', 'o', 'n', 'g', 'D', 0xa8, 0xc6, 0x4b, 0x37, 0x89, 0x21, 0x3f, 0x40,
'}'
};
CHECK(json::to_bjdata(j) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Optimized with count")
{
// note the floats have been replaced by doubles
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
std::vector<uint8_t> v = {'{', '#', 'i', 3,
'i', 3, 'a', 'l', 't', 'D', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
'i', 3, 'l', 'a', 't', 'D', 0x60, 0xe5, 0xd0, 0x22, 0xdb, 0xf9, 0x3d, 0x40,
'i', 4, 'l', 'o', 'n', 'g', 'D', 0xa8, 0xc6, 0x4b, 0x37, 0x89, 0x21, 0x3f, 0x40,
};
CHECK(json::to_bjdata(j, true) == v);
CHECK(json::from_bjdata(v) == j);
}
SECTION("Optimized with type & count")
{
// note the floats have been replaced by doubles
json const j = { {"lat", 29.976}, {"long", 31.131}, {"alt", 67.0} };
std::vector<uint8_t> v = {'{', '$', 'D', '#', 'i', 3,
'i', 3, 'a', 'l', 't', 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x50, 0x40,
'i', 3, 'l', 'a', 't', 0x60, 0xe5, 0xd0, 0x22, 0xdb, 0xf9, 0x3d, 0x40,
'i', 4, 'l', 'o', 'n', 'g', 0xa8, 0xc6, 0x4b, 0x37, 0x89, 0x21, 0x3f, 0x40,
};
CHECK(json::to_bjdata(j, true, true) == v);
CHECK(json::from_bjdata(v) == j);
}
}
SECTION("Special Cases (Null, No-Op and Boolean)")
{
SECTION("Array")
{
json _;
std::vector<uint8_t> const v = {'[', '$', 'N', '#', 'I', 0x00, 0x02};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x4E is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v, true, false).is_discarded());
}
SECTION("Object")
{
json _;
std::vector<uint8_t> const v = {'{', '$', 'Z', '#', 'i', 3, 'i', 4, 'n', 'a', 'm', 'e', 'i', 8, 'p', 'a', 's', 's', 'w', 'o', 'r', 'd', 'i', 5, 'e', 'm', 'a', 'i', 'l'};
CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x5A is not a permitted optimized array type", json::parse_error&);
CHECK(json::from_bjdata(v, true, false).is_discarded());
}
}
}
}
TEST_CASE("Parse BJData directly from a file using iterator and sentinel")
{
std::string const filename = TEST_DATA_DIRECTORY "/json_testsuite/sample.json.bjdata";
std::ifstream file(filename, std::ios::binary);
const std::istreambuf_iterator<char> first(file);
const json parsed = json::from_bjdata(first, utils::istreambuf_sentinel{});
CHECK((parsed.is_object() || parsed.is_array()));
}
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("all BJData first bytes")
{
// these bytes will fail immediately with exception parse_error.112
std::set<uint8_t> supported =
{
'T', 'F', 'Z', 'B', 'U', 'i', 'I', 'l', 'L', 'd', 'D', 'C', 'S', '[', '{', 'N', 'H', 'u', 'm', 'M', 'h'
};
for (auto i = 0; i < 256; ++i)
{
const auto byte = static_cast<uint8_t>(i);
CAPTURE(byte)
try
{
auto res = json::from_bjdata(std::vector<uint8_t>(1, byte));
}
catch (const json::parse_error& e)
{
// check that parse_error.112 is only thrown if the
// first byte is not in the supported set
INFO_WITH_TEMP(e.what());
if (supported.find(byte) == supported.end())
{
CHECK(e.id == 112);
}
else
{
CHECK(e.id != 112);
}
}
}
}
#endif
TEST_CASE("BJData and UBJSON can be written to a string")
{
const std::vector<json> values =
{
{{"a", {1, 2.5, "x", nullptr}}, {"b", json::binary({1, 2})}},
// an annotated ND-array, and objects that only look like one
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": 1, "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": 4, "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, -2], "_ArrayData_": [1, 2, 3, 4]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": [1, 2, 3]})"),
json::parse(R"({"_ArrayType_": "uint8", "_ArraySize_": [2, 2], "_ArrayData_": 1})"),
};
// compared byte by byte: building a std::string from the bytes would
// convert them implicitly, which -fsanitize=integer reports for bytes of
// 0x80 and above
const auto same_bytes = [](const std::vector<std::uint8_t>& bytes, const std::string & text)
{
return bytes.size() == text.size() && std::equal(bytes.begin(), bytes.end(), text.begin(), [](std::uint8_t byte, char c)
{
return byte == static_cast<std::uint8_t>(c);
});
};
for (const auto& j : values)
{
CAPTURE(j.dump())
for (const bool use_size :
{
false, true
})
{
for (const bool use_type :
{
false, true
})
{
if (use_type && !use_size)
{
continue;
}
CAPTURE(use_size)
CAPTURE(use_type)
const auto bjdata = json::to_bjdata(j, use_size, use_type);
std::string bjdata_string;
json::to_bjdata(j, bjdata_string, use_size, use_type);
CHECK(same_bytes(bjdata, bjdata_string));
const auto ubjson = json::to_ubjson(j, use_size, use_type);
std::string ubjson_string;
json::to_ubjson(j, ubjson_string, use_size, use_type);
CHECK(same_bytes(ubjson, ubjson_string));
}
}
}
}
TEST_CASE("BJData use_type requires use_size")
{
SECTION("non-empty object throws other_error.502")
{
const json j = {{"a", 1}, {"b", 2}};
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
"[json.exception.other_error.502] use_type requires use_size = true",
json::other_error&);
}
SECTION("non-empty array throws other_error.502")
{
const json j = {1, 2, 3};
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
"[json.exception.other_error.502] use_type requires use_size = true",
json::other_error&);
}
SECTION("non-empty binary value throws other_error.502")
{
const json j = json::binary({1, 2, 3});
CHECK_THROWS_WITH_AS(json::to_bjdata(j, false, true),
"[json.exception.other_error.502] use_type requires use_size = true",
json::other_error&);
CHECK_THROWS_WITH_AS(json::to_ubjson(j, false, true),
"[json.exception.other_error.502] use_type requires use_size = true",
json::other_error&);
}
SECTION("scalars do not throw with use_type=true, use_count=false")
{
CHECK_NOTHROW(json::to_bjdata(42, false, true));
CHECK_NOTHROW(json::to_bjdata(3.14, false, true));
CHECK_NOTHROW(json::to_bjdata("hello", false, true));
CHECK_NOTHROW(json::to_bjdata(true, false, true));
CHECK_NOTHROW(json::to_bjdata(nullptr, false, true));
}
SECTION("empty containers do not throw with use_type=true, use_count=false")
{
CHECK_NOTHROW(json::to_bjdata(json::array(), false, true));
CHECK_NOTHROW(json::to_bjdata(json::object(), false, true));
}
SECTION("valid combinations on non-empty containers")
{
const json j = {{"a", 1}, {"b", 2}};
CHECK_NOTHROW(json::to_bjdata(j, false, false));
CHECK_NOTHROW(json::to_bjdata(j, true, false));
CHECK_NOTHROW(json::to_bjdata(j, true, true));
}
}
TEST_CASE("BJData round-trip invariants")
{
// This checks what the parse_bjdata_fuzzer driver checks (see
// tests/src/fuzzer-parse_bjdata.cpp), so that a regression shows up in CI
// rather than as an OSS-Fuzz report: every value from_bjdata() returns
// (j1) can be serialized with any combination of options, the result can
// be parsed back (j2), and serializing j2 again with the same options
// yields a value-equal result.
//
// Beyond the driver, this also checks that j2 equals j1 and that
// serializing j2 reproduces the exact bytes, both except for values that
// contain a binary value: a binary value is only written as a binary
// value with Draft 3's optimized binary array, and otherwise read back as
// an array of integers, for which the writer may choose different (but
// equally valid) type markers when it is serialized again (see #5494).
//
// Values are compared with dump() rather than operator==, because a NaN
// never compares equal to itself.
struct options
{
bool use_size;
bool use_type;
json::bjdata_version_t version;
};
const std::vector<options> all_options =
{
{false, false, json::bjdata_version_t::draft2},
{true, false, json::bjdata_version_t::draft2},
{true, true, json::bjdata_version_t::draft2},
{false, false, json::bjdata_version_t::draft3},
{true, false, json::bjdata_version_t::draft3},
{true, true, json::bjdata_version_t::draft3},
};
for (const auto& j0 : utils::round_trip_corpus::values())
{
// turn the corpus value into a value as from_bjdata() returns it
for (const auto& initial : all_options)
{
const json j1 = json::from_bjdata(json::to_bjdata(j0, initial.use_size, initial.use_type, initial.version));
const bool has_binary = utils::round_trip_corpus::contains_binary(j1);
for (const auto& o : all_options)
{
INFO("j1 = " << j1.dump() << ", use_size = " << o.use_size << ", use_type = " << o.use_type
<< ", draft3 = " << (o.version == json::bjdata_version_t::draft3));
const std::vector<std::uint8_t> vec = json::to_bjdata(j1, o.use_size, o.use_type, o.version);
json j2;
// anything the library writes must be parsable by the library
REQUIRE_NOTHROW(j2 = json::from_bjdata(vec));
const std::vector<std::uint8_t> vec2 = json::to_bjdata(j2, o.use_size, o.use_type, o.version);
CHECK(json::from_bjdata(vec2).dump() == j2.dump());
if (!has_binary)
{
CHECK(j2.dump() == j1.dump());
CHECK(vec2 == vec);
}
}
}
}
}
TEST_CASE("BJData round trip of a binary value is value-stable, not byte-stable")
{
// OSS-Fuzz issue 474480402: a Draft 3 optimized binary array is read as a
// binary value, which to_bjdata() writes in the default Draft 2 mode as a
// plain array of uint8 numbers. That is read back as an array of numbers,
// for which the writer then picks the smallest type marker, int8 ('i'),
// so re-serializing changes the bytes, but not the value. This is the
// exception described in the "Round trips" note of the BJData
// documentation, and why the fuzzer checks value stability (see #5494).
const std::vector<uint8_t> input = {'[', '$', 'B', '#', 'U', 1, 0x20};
const json j1 = json::from_bjdata(input);
CHECK(j1 == json::binary({0x20}));
const std::vector<uint8_t> vec = json::to_bjdata(j1, false, false);
CHECK(vec == std::vector<uint8_t>({'[', 'U', 0x20, ']'}));
const json j2 = json::from_bjdata(vec);
CHECK(j2 == json::array({0x20}));
const std::vector<uint8_t> vec2 = json::to_bjdata(j2, false, false);
CHECK(vec2 == std::vector<uint8_t>({'[', 'i', 0x20, ']'}));
CHECK(json::from_bjdata(vec2) == j2);
}
TEST_CASE("BJData roundtrips" * doctest::skip())
{
SECTION("input from self-generated BJData files")
{
for (const std::string filename :
{
TEST_DATA_DIRECTORY "/json_nlohmann_tests/all_unicode.json",
TEST_DATA_DIRECTORY "/json.org/1.json",
TEST_DATA_DIRECTORY "/json.org/2.json",
TEST_DATA_DIRECTORY "/json.org/3.json",
TEST_DATA_DIRECTORY "/json.org/4.json",
TEST_DATA_DIRECTORY "/json.org/5.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip01.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip02.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip03.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip04.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip05.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip06.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip07.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip08.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip09.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip10.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip11.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip12.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip13.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip14.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip15.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip16.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip17.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip18.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip19.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip20.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip21.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip22.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip23.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip24.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip25.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip26.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip27.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip28.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip29.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip30.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip31.json",
TEST_DATA_DIRECTORY "/json_roundtrip/roundtrip32.json",
TEST_DATA_DIRECTORY "/json_testsuite/sample.json",
TEST_DATA_DIRECTORY "/json_tests/pass1.json",
TEST_DATA_DIRECTORY "/json_tests/pass2.json",
TEST_DATA_DIRECTORY "/json_tests/pass3.json"
})
{
CAPTURE(filename)
std::ifstream f_json(filename);
const json j1 = json::parse(f_json);
auto packed = utils::read_binary_file(filename + ".bjdata");
{
INFO_WITH_TEMP(filename + ": std::vector<uint8_t>");
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(packed));
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": std::ifstream");
std::ifstream f_bjdata(filename + ".bjdata", std::ios::binary);
json j2;
CHECK_NOTHROW(j2 = json::from_bjdata(f_bjdata));
CHECK(j1 == j2);
}
{
INFO_WITH_TEMP(filename + ": output to output adapters");
{
INFO_WITH_TEMP(filename + ": output adapters: std::vector<uint8_t>");
std::vector<uint8_t> vec;
json::to_bjdata(j1, vec);
CHECK(vec == packed);
}
}
}
}
}
Reference in New Issue View Git Blame Copy Permalink