mirror of
https://github.com/nlohmann/json.git
synced 2026-09-16 21:27:58 +00:00
Niels LohmannandClaude Opus 5
c72f37a40d
* docs: document the implicit requirements on basic_json's template parameters The requirements that basic_json places on its eleven template parameters were only implied by how the library uses the resulting object_t, array_t, string_t, etc. Consumers had to discover them by trial and error. Add "Template Parameter Requirements" collecting them, split into what is always required and what is only required when a particular part of the API is instantiated. Notable findings that were previously undocumented: - ObjectType must provide a key_compare member type (actual_object_comparator names object_t::key_compare in both arms of a std::conditional), and its third template parameter is used as a comparator, so std::unordered_map cannot be used without a wrapper. - ArrayType must provide capacity() -- push_back(), emplace_back(), operator+=(), and operator[](size_type) call it unconditionally -- and needs random-access iterators, so std::deque and std::list do not work. - StringType needs contiguous, null-terminated data(), a one-byte value_type, and either assignability from std::to_string or an ADL int_to_string(). - NumberFloatType must be float, double, or long double for parsing and serialization; the integer types must satisfy std::is_integral. - AllocatorType must be stateless, support incomplete types, and use plain pointers. - BooleanType and the number types are union members and must be trivial. Link the new page from the basic_json overview, the types feature page, and the individual type alias pages, and correct the container examples given for ObjectType (std::unordered_map) and ArrayType (std::list), which do not work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix object_comparator_t for object types without key_compare detail::actual_object_comparator selected between object_t::key_compare and default_object_comparator_t with std::conditional. Both type arguments of std::conditional are named eagerly, so object_t::key_compare had to exist regardless of the condition, and the has_key_compare guard added in 3.11.0 never took effect: any ObjectType without a key_compare member type failed to compile while instantiating basic_json itself. Use detected_or_t instead, which resolves through a SFINAE partial specialization and only names object_t::key_compare when it exists. The selected type is unchanged for every object type that compiled before, so object_comparator_t -- a public member type -- keeps its meaning and ABI. has_key_compare had no other users and is removed. Add a regression test using an adapter around std::unordered_map, which has no key_compare; it fails to compile without this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: list the types that are known to work for each template parameter Follow up on the template parameter requirements page: state, for every template parameter, which concrete types work and where they stop working. Each entry was verified by compiling and running a common workload (DOM access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that instantiation. Findings worth calling out: - ObjectType no longer needs a key_compare member type, so the std::unordered_map adapter only has to restore the template argument order. A hash-ordered ObjectType works everywhere except unflatten(), which reconstructs an array only when it meets the reference token 0 before the other indices. - ArrayType: std::deque works when wrapped to add capacity(); std::list does not. - StringType: std::pmr::string and std::basic_string with a custom allocator compile for the DOM, dump, and parse, but not for the binary readers, flatten, or diff, because the library assigns std::string values to string_t and int_to_string cannot be overloaded for a type in namespace std. - NumberFloatType: long double works for dump and parse but not for the binary formats, which have no encoding for it. - BinaryType: std::vector<std::byte> supports assignment, get, and the binary formats, but neither dump nor std::hash<basic_json>. Also record the object_comparator_t fix in its version history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix unflatten and binary dumping for non-default configurations unflatten() decided between array and object by looking at the first reference token it happened to see for a node: it started an array only when that token was 0. With a sorted object type the token 0 always arrives first, so the result was correct by accident; with an object type whose iteration order is unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the keys "0", "1", and "2" instead of an array. Collect the pointer prefixes that have a reference token 0 among their children before building the result, and let get_and_create() consult that set. The outcome is now independent of the iteration order and matches, for every input, what a sorted object type produced before: a value is restored as an array if and only if one of its keys is 0. Iterating the flattened object in a different order would have been simpler, but it would have changed the key order of the result for insertion-ordered object types. The serializer, std::hash, and the UBJSON writer converted the elements of a binary value to an integer implicitly, which does not compile for a BinaryType whose value type is std::byte, and which made dump() write the bytes of a signed value type as negative numbers. Convert to std::uint8_t explicitly in all three places, so every byte type dumps as 0..255. The default std::vector<std::uint8_t> configuration is unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: note which Abseil containers can be used as template arguments Checked against Abseil release 20250127.0 with the same workload as the other entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON round-trip, flatten, hash), with and without JSON_DIAGNOSTICS. absl::flat_hash_map and absl::node_hash_map work as ObjectType through an adapter that restores the template argument order and makes erase(iterator) return the following iterator, which Abseil's returns as void. The page now carries that adapter, and notes that absl::flat_hash_map does not keep references to the mapped values valid across insertions while absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS already refreshes the parent pointers conservatively for them. absl::btree_map and absl::InlinedVector cannot be used at all: object_t and array_t are formed while basic_json is still incomplete, and both inspect their value type at class scope. std::map and std::vector are required by the standard to tolerate this, third-party containers generally are not, so the page states the constraint on its own rather than only per container. absl::InlinedVector does work as BinaryType, where it is instantiated with a complete type. absl::FixedArray and absl::Cord are not usable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Relax the ArrayType and ObjectType requirements Two requirements forced users of otherwise suitable containers to write a wrapper, and neither was load-bearing. array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and operator[](size_type), but set_parent() only looks at the value under JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it through array_capacity(), which reports unknown_size() when diagnostics are off or when the array type has no capacity() at all, and treat an unknown capacity as "the elements may have moved" so the parent pointers are refreshed conservatively. std::deque now works as ArrayType, in both builds, and capacity() is no longer named at all in a default build. Since the capacity is now only meaningful for array insertions, it moves out of set_parent() into set_parent_after_array_insert(). basic_json::erase(iterator) assigned the object's erase() return value, which requires the container to return the following iterator. Abseil's hash maps return void to avoid computing a successor the caller may not need. Detect that and compute the successor before erasing; containers that return an iterator, including the vector-backed ordered_map where a precomputed successor would be wrong, keep the existing path. Together these leave an Abseil hash map needing only an alias that restores the template argument order, and no adapter at all for std::deque. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not require string_t to be convertible from std::string Three places built a std::string and handed it to something expecting a string_t: the UBJSON high-precision number reader, which every binary reader instantiates, and the BSON writer's array element size calculation and write. That silently required string_t to be implicitly convertible from std::string, which std::string itself and types with a string_view conversion satisfy, but many string types do not. Construct the string_t explicitly from the data and size, which the requirements already cover. This makes boost::container::string, eastl::string, std::pmr::string, and std::basic_string with a custom allocator work as StringType, none of which could previously be used with any binary format. Add binary format coverage to the alt_string test, which had none, including a UBJSON high-precision number -- the case that goes through the reader path. BSON stays uncovered there: it additionally needs string_t::find(value_type), which alt_string does not provide. Also record which containers from Boost, Abseil, and EASTL work for each template parameter, and correct two claims: std::pmr::string is usable after this change, and tsl::ordered_map is not usable at all, because its iterators expose the mapped value as const while basic_json modifies it in place. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: record compatibility for the common header-only hash maps ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType through the same adapter as Abseil's and Boost's hash maps, which only has to restore the template argument order. phmap::btree_map and robin_hood::unordered_node_map do not: like the other btree containers they require a complete value type. Note that none of these hash maps defines key_compare, so every one of them depends on object_comparator_t falling back to default_object_comparator_t. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: record Folly and the remaining vector replacements Folly works, with the caveat that its headers need C++20: folly::fbstring as StringType, folly::fbvector and folly::small_vector as ArrayType, folly::fbvector<std::uint8_t> as BinaryType, and folly::F14NodeMap as ObjectType through the usual argument-order adapter. folly::F14FastMap is the exception and requires a complete value type. For ArrayType, boost::container::devector, boost::container::static_vector (within its fixed capacity), and std::pmr::vector work as well. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: cover fifo_map, gtl, folly::sorted_vector_map, and Qt nlohmann::fifo_map works through the adapter that has always been documented for it, and preserves the insertion order. Restore its mention in the object order page, which was dropped together with the tsl::ordered_map one: unlike ordered_map it keeps a lookup index, so it is the insertion-ordered option without the quadratic cost. gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter through an alias that drops the allocator, whose value type it disagrees on. gtl::btree_map does not, for the same reason as the other btree containers. None of the Qt containers can be used, each for its own reason: QMap has no value_type, QHash iterators yield the mapped value rather than a pair, QList has no max_size(), QByteArray spells empty() as isEmpty(), and QString is UTF-16. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: qualify the std::pmr::string support claim Listing std::pmr::string as fully supported was an overclaim: it was only ever checked with the default memory resource, which is not what PMR is for. basic_json cannot be given an allocator or a memory resource, so a pmr string inside a value always allocates from std::pmr::get_default_resource(), and assigning an arena-backed string into a value silently drops its resource, because polymorphic_allocator does not propagate on copy construction. Passing polymorphic_allocator as AllocatorType does not compile either. Only the process-global set_default_resource() redirects these allocations. Say so, and separate the row from std::basic_string with a custom stateless allocator, which is unaffected. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: remove a duplicated StringType compatibility section The StringType section carried two 'Compatible types' tables and two copies of the reference-implementation tip. The second table was a stale copy from before the binary format string fixes and still listed std::pmr::string and std::basic_string with a custom allocator as unusable, contradicting the corrected table a few lines above it, and it dragged along the old explanation that blamed int_to_string. Drop the stale copy and put the surviving table before the notes, so the 'see below' in the std::pmr::string row points forwards. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: correct the template parameter requirements after independent verification Every claim on the page was re-checked by compiling and running it, including the rows that say a type cannot be used, which were checked to fail for the documented reason and not merely to fail. Twenty-four claims were wrong. The most consequential: the incomplete-type constraint applies to ObjectType only. object_t is instantiated inside the class definition, because it is probed for key_compare; array_t is only named there and is not instantiated until basic_json is complete. So eastl::vector, QList and QVector are not excluded by incomplete types at all -- they simply have no max_size() -- and absl::InlinedVector is excluded for a subtler reason of its own. Further corrections: ObjectType does not need erase(key), which has a fallback, but does need at(key) for UBJSON output; only == and < are used, or == and <=> under C++20, not all six; the documented adapter does not fit ankerl or robin_hood. ArrayType needs no initializer-list insert, and value_type, the (count, value) constructor and swappability are per-function, not always. BinaryType needs a range insert for CBOR indefinite-length byte strings and does not need push_back. StringType needs append(const StringType&) unconditionally, and does not need operator!= or operator== against const char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is needed by diff, items and std::hash rather than by JSON Pointer or flatten. BooleanType must be implicitly convertible from bool, and JSONSerializer's second parameter need not carry a default. std::pmr::string was wrong in the other direction this time: a moved-in string does keep its memory resource, and later growth allocates from it. Only copies land on the default resource. Five requirement violations are not caught at compile time rather than the two the page claimed; they are now listed together up front. Split every compatibility table into what works and what does not, as the reasons in the second half are the useful part. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Reduce the string_t and array_t members the library requires Several members were required only because of how the library happened to be written, not because the functionality needs them. Dropping them widens the set of usable string and array types, and one of them was also a performance problem. string_t: - c_str() is gone. Every call site already knew the length and passed it along, so data() is enough. The one place that did not, the diagnostics path in exceptions.hpp, now builds the token from data() and size(), which also stops it from truncating keys that contain a null byte. - back() is gone; the serializer indexes the last character instead. - find(str, pos), replace(), and substr() are gone. escape() and unescape() rebuilt the string with one replace() per escaped character, which moves the tail every time: escaping a string of n characters that all need escaping cost O(n^2). Both now scan with find_first_of() -- a member the pointer parser already required -- and append whole runs, so the common case is one search and one copy. Escaping 64000 tildes drops from 717 ms to 20 ms; a string with nothing to escape gets faster too (8.4 ms to 5.8 ms), because the scan is still a single memchr per pass. json_pointer::split() takes its reference tokens with the (const char*, size_type) constructor rather than substr(). - json_pointer::to_string() accumulates with concat<string_t> instead of letting concat default to std::string and converting afterwards, so streaming a json_pointer no longer requires string_t to be assignable from a std::string. array_t: - at(size_type) is gone. basic_json::at(size_type) checked the index by calling array_t::at() and translating std::out_of_range, which also required the array type to throw that exact exception. It now compares against size() and uses operator[]. The thrown exception, its message, and the behaviour under JSON_NOEXCEPTION are unchanged. The BSON writer wrote the terminating null byte out of the string's own buffer (size() + 1). It now writes the byte itself, so string_t::data() need not be null-terminated for to_bson(). The tests pin the reduced API: alt_string loses the five dropped members and gains coverage of the escaping paths, and a std::vector whose at() is hidden is used as an ArrayType. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: record the reduced string_t and array_t requirements Drop c_str(), back(), find(str, pos), replace(), and substr() from the StringType requirements and at(size_type) from the ArrayType ones, and note the string assignment the JSON pointer code performs. Streaming a json_pointer no longer needs assignability from a std::string. Add the non-null-terminated data() to the list of violations that are not diagnosed at compile time -- it was described in the StringType section but missing from the summary at the top -- and correct the QString row, which no longer fails for the c_str() it lacks. JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one went away with array_t::at(). Describe what the library actually catches. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use character literals for the signed BinaryType test MSVC rejects char(0xFF) with C4310 (cast truncates constant value), which the Windows workflow treats as an error. The character literals carry the same byte values without a narrowing cast. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not instantiate a hash map with an incomplete basic_json in the tests object_t is probed for key_compare inside the definition of basic_json, so it is instantiated while basic_json is still incomplete. Whether a hash map survives that depends on the standard library: libstdc++ 9 needs the size of the mapped type to instantiate std::unordered_map's node type and rejects the adapter, which broke the GCC 9 builds. The test now derives its no-key_compare object type from std::map -- which does cope -- and shadows the inherited key_compare member type with an entity that is not a type, so the library's probe finds none, exactly as for a hash map. The unflatten() order-independence checks in unit-json_pointer already cover the behaviour that the unordered object type was there for. The limitation is documented for std::unordered_map. Also address two Clang-Tidy findings the earlier commits introduced: erase_from_object() declares its iterator with auto, and at(size_type) checks the type first and then falls through to the return instead of throwing from an else branch. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep diagnostic key paths null-terminated Building the token from data() and size() kept an embedded null byte in the key, and since what() hands out a C string, that truncated the whole message rather than just the key: to_bson() on a key containing U+0000 reported "[json.exception.out_of_range.409] (/en" instead of the full explanation. This broke test-bson under JSON_DIAGNOSTICS. Constructing from data() alone stops at the first null byte, which is what c_str() did before, so the message is unchanged -- without requiring string_t to provide c_str(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not parse the value in the array-at() test JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception message, which a parsed value has and an in-memory one does not, so the two message checks failed in that configuration. Build the array in memory instead of parsing it; the test is about at(size_type) not needing array_t::at(), and the byte range is beside the point. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the custom BinaryType tests into their own translation unit The two sections added to unit-regression2.cpp brought a third full basic_json instantiation into a translation unit that was already large. With Clang on MinGW that pushed the object over the reach of a 32-bit relocation and test-regression2_cpp20.exe failed to link: relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata' unit-regression2.cpp is restored to exactly what it was before, and the coverage moves to unit-custom-binary-type.cpp, next to the object and array type tests it belongs with. The signed value type is now also covered in C++11, where std::byte is not available. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not require the container iterators to be nothrow move constructible iter_impl declared its defaulted move operations noexcept. The exception specification a defaulted function gets implicitly follows from its members, here internal_iterator, which holds the object and array iterators. libstdc++ gives std::deque's iterator a user-provided copy constructor without noexcept before version 11, so the implicit specification is noexcept(false) and does not match the declared one. That deletes the function -- and with g++ 4.8, which predates CWG 1778, it is an error outright: error: function 'iter_impl<basic_json<std::map, std::deque> >::iter_impl( iter_impl&&)' defaulted on its first declaration with an exception-specification that differs from the implicit declaration So std::deque, which this branch documents as a usable array type, could not be used with an older standard library. Leaving the specification to be computed cannot mismatch; iteration_proxy_value already spells out the same condition next door. The default configuration is unaffected: json::iterator, json::const_iterator and ordered_json::iterator stay nothrow move constructible and move assignable, which the test now checks so it cannot regress unnoticed. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address two Clang-Tidy findings the custom container tests exposed Both come from instantiating basic_json with containers other than the default ones, and neither shows up with the Clang-Tidy version available outside CI: - insert(const_iterator, basic_json&&) forwards its by-value iterator to the const-reference overload. performance-unnecessary-value-param asks for the copy to be a move; it only fires for an iterator that is not trivially copyable, as std::deque's is not. The NOLINT on the function does not cover it, because the finding is reported where the parameter is used rather than where it is declared. Move it, which is what the check asks for and is a (very small) improvement in its own right. - cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed the inherited key_compare member type. An enum class would not do, since it declares a type of that name and the probe would find it again; a member function declaration hides the name just as well. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Assert the iterators' exception specification relative to the container The test pinned that nlohmann::json's iterators stay nothrow movable after iter_impl's defaulted move operations lost their declared noexcept. That is not a property of the library, though: the exception specification is now computed from the container iterators, so it holds only for standard library implementations whose iterators are themselves nothrow movable. MSVC's checked iterators before VS2017 are not -- _Iterator_base12 registers the iterator with the container's debug proxy in a copy constructor that carries no noexcept -- so the assertions fail on a Visual Studio 2015 debug build, which is the one debug configuration in the AppVeyor matrix and has no counterpart in the GitHub Actions matrix. Assert what the change actually guarantees instead: the iterators are nothrow movable exactly when the object and array iterators they are built from are. That still pins the default configuration against a silent regression, and it is true whatever the standard library provides. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Detect a void-returning erase() through a named trait erase_from_object() distinguished its two overloads with a decltype of a member call written inline in a default template argument. Every other detection in the library goes through the detector machinery in detected.hpp instead -- has_erase_with_key_type is the same question about the same member function -- and the inline form is the one shape older compilers are least reliable about. Express it the same way: detect_erase_with_iterator plus is_detected_exact, both of which the library already relies on elsewhere. No behaviour changes. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Give the custom container types only the constructors the library uses The three container types in the new tests inherited every constructor of their base with using Base::Base. That asks for more than the test needs: the library builds an object or an array by default construction, by copy or move, and -- when converting between two basic_json types or from an initializer list -- from an iterator range. Declaring those directly makes the requirement visible in the test, and keeps object types out of a corner where a compiler has to declare std::map's whole constructor set for a derived class while basic_json is still incomplete. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Temporarily disable the new custom container tests AppVeyor is the only CI that builds MSVC 2015 and 2017, and it has now rejected three heads of this branch. Its build log is not reachable from where this is being worked on, so the verdict is a single bit and the cause has to be narrowed down by bisection. Everything else stays: the library changes, the reduced alt_string, and the unflatten() tests. If AppVeyor passes with these three translation units disabled, the cause is one of the six basic_json instantiations they add; if it fails, it is in the library. Either way this commit is reverted. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Guard the disabled tests with a macro rather than #if 0 Clang-Tidy's readability-avoid-unconditional-preprocessor-if rejects a literal #if 0. Use a macro that is never defined instead, which the check does not look at. Still temporary, and reverted together with the previous commit. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Re-enable the array and binary container tests AppVeyor passed with all three new translation units disabled, so the library changes, the reduced alt_string, and the unflatten() tests are fine on MSVC 2015 and 2017; the cause is one of the six basic_json instantiations the new tests add. Bring back two of the three. If AppVeyor passes again, the cause is in unit-custom-object-type.cpp, which is the one still disabled; if it fails, it is in one of these two and needs one more split. Still temporary. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diagnose two silently violated template parameter requirements Both were on the list of requirements that are not caught at compile time and corrupt values rather than failing, and both are a plain size comparison: - A BinaryType whose value_type is wider than one byte, which the readers and writers reinterpret as raw bytes anyway. - A NumberUnsignedType too narrow to hold the absolute value of every NumberIntegerType value, which makes basic_json(INT64_MIN).dump() yield -0 for std::int64_t with std::uint32_t. Neither static_assert rejects a configuration that worked before: both only fire where the result was already wrong. Also add the two comments the review asked for, in write_bson_string() and calc_bson_array_size(), matching the ones their counterparts already carry. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Align the template parameter tables and record what is now diagnosed Every table in the page is reformatted so each column is exactly as wide as its widest cell, which is what the review asked for in a dozen places: the separator rows that ran two dashes long, the stray spaces, and the columns padded well past their content. The row listing six containers that require a complete mapped type is split in two so that one cell no longer sets the width of the whole table. Content changes: NumberUnsignedType is described as any unsigned integer type at least as wide as NumberIntegerType rather than any unsigned integer type; the two requirements that are now static_asserts move out of the list of violations that are not caught at compile time; and the two places that require a non-const operator[] say why data() will not do (std::string has no non-const data() before C++17). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bisect the other way: only the object container tests The previous head touched only docs/, which AppVeyor's only_commits filter skips, so it produced no build and no status at all -- the pull request looked green without ever having been built on MSVC 2015 or 2017. Swap the guards instead of repeating that step: unit-custom-object-type.cpp is enabled and the array and binary translation units are disabled. AppVeyor already passed with all three disabled, so a failure here pins the cause on no_key_compare_json or void_erase_json, and a pass pins it on the array or binary file. Still temporary. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the two object types apart AppVeyor failed with only unit-custom-object-type.cpp enabled and passed with all three new translation units disabled, so the cause is one of the two object types in this file and not the array or binary ones. Guard out void_erase_map and leave no_key_compare_map, which separates the two constructs under suspicion: shadowing the inherited key_compare member type with an entity that is not a type, and hiding the inherited erase with a void-returning overload. A failure here points at the first, a pass at the second. Still temporary. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Build the no-key_compare object type by composition, not inheritance The "object type without key_compare" test failed on AppVeyor's MSVC 2017 jobs (/std:c++17): its no_key_compare_map derived publicly from std::map and shadowed the inherited key_compare type with a same-named member function, relying on ordinary member hiding to make key_compare unreachable as a type for the library's detection trait. MSVC 2017 does not honor that hiding for a typename-qualified lookup performed from outside the class and still resolves key_compare to the base's comparator type, so object_comparator_t incorrectly picked it up instead of falling back to default_object_comparator_t. Wrapping a std::map by composition instead removes the base class entirely, so there is no key_compare to find under any lookup rule, on any compiler. Also drops the now-unneeded JSON_BISECT_CUSTOM_CONTAINER_TESTS guard left over from narrowing this down: the void_erase_map test in the same file was never the cause and is re-enabled unconditionally. Verified locally with clang++ and g++ under C++17 and C++20. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Re-enable the array and binary custom-container tests unit-custom-array-type.cpp and unit-custom-binary-type.cpp were still guarded behind JSON_BISECT_CUSTOM_CONTAINER_TESTS from bisecting the AppVeyor failure fixed in7c39f3227, which was unrelated to either file. The macro was never defined, so none of these tests actually ran in CI. Verified locally with clang++ and g++ under C++17 and C++20 before removing the guards. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix indentation of custom_object_type per astyle The one-line function bodies in the composition-based no_key_compare_map (7c39f3227) do not match the project's Allman brace style, which the ci_test_amalgamation job enforces with astyle. Reformatted with the pinned astyle 3.4.13; no functional change. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * docs: add compiled reference implementations for the container template parameters Each of ObjectType, ArrayType, StringType, and BinaryType now links to a minimal, self-contained header (docs/mkdocs/docs/examples/custom_*_type.hpp) that wraps the corresponding standard container by composition and satisfies every "Always required" member listed on that page. Unlike the prose requirement lists, these are real code: each header has a companion .cpp that instantiates a basic_json specialization with it and is compiled and run by the existing ci_test_examples check (docs/Makefile's check_output_portable), so the reference implementations cannot silently drift from what the library actually requires. The .output files were generated with that same target. StringType's existing pointer to tests/src/unit-alt-string.cpp's alt_string is kept alongside the new header as a more thorough, battle-tested example. Verified locally: astyle (pinned 3.4.13, project .astylerc) on the new files; clang++/g++ under C++11/17/20 for each example against the amalgamated header; `make check_output_portable` in docs/; `mkdocs build --strict` and scripts/check_structure.py for the page itself. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Declare no_key_compare_map's accessors noexcept The GCC C++20 job builds with -Wnoexcept and -Werror, and the standard library takes noexcept(c.begin()) and noexcept(c.end()) in ranges_base.h and range_access.h. Forwarding to std::map without repeating its noexcept made those expressions false, which the warning reports as an error: error: noexcept-expression evaluates to 'false' because of a call to no_key_compare_map<...>::begin() [-Werror=noexcept] note: but ... does not throw; perhaps it should be declared 'noexcept' Give the accessors the exception specification of what they forward to. std::map declares begin, end, cbegin, cend, empty, size, max_size, and clear noexcept, so the wrapper does too. swap is left alone: std::map's is only conditionally noexcept, and nothing asks for it. void_erase_map is unaffected because it still derives from std::map and inherits accessors that already carry the specification. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Write out what the defaulted constructor of no_key_compare_map implied ci_test_gcc builds with -Weffc++, which asks for data to be initialized in a member initialization list; a defaulted default constructor does not do that: error: 'no_key_compare_map<...>::data' should be initialized in the member initialization list [-Werror=effc++] Writing the constructor out satisfies that but drops the exception specification the defaulted one carried, which -Wnoexcept then objects to where the standard library takes noexcept(construct(...)). Declare it the way the defaulted constructor was: noexcept when the wrapped map's default constructor is. This is the cost of composition -- inheritance carried std::map's exception specifications and initialization for free, and forwarding by hand has to restate them. Checked with the repository's own GCC warning set from cmake/gcc_flags.cmake, all 346 flags, at C++11, C++17 and C++20: no diagnostics for this file, nor for the two custom container translation units that were disabled while the MSVC failure was narrowed down and are built again now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mark no_key_compare_map::swap noexcept Clang-Tidy rejects a swap that is not: error: swap functions should be marked noexcept [cppcoreguidelines-noexcept-swap,performance-noexcept-swap] It was left unmarked on the grounds that std::map::swap is only conditionally noexcept, so an unconditional promise would be wrong for a comparator or allocator that can throw while swapping. Both concerns are met by taking the specification from the wrapped map rather than asserting one: noexcept(noexcept(data.swap(other.data))). Clang-Tidy accepts that, and no NOLINT is needed. Last in the series of specifications that inheritance used to supply and composition has to write out by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
3524 lines
124 KiB
C++
3524 lines
124 KiB
C++
// __ _____ _____ _____
|
|
// __| | __| | | | JSON for Modern C++
|
|
// | | |__ | | | | | | version 3.12.0
|
|
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
|
//
|
|
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
#pragma once
|
|
|
|
#include <algorithm> // generate_n
|
|
#include <array> // array
|
|
#include <cmath> // ldexp
|
|
#include <cstddef> // size_t
|
|
#include <cstdint> // uint8_t, uint16_t, uint32_t, uint64_t, uintmax_t
|
|
#include <cstdio> // snprintf
|
|
#include <cstring> // memcpy
|
|
#include <iterator> // back_inserter
|
|
#include <limits> // numeric_limits
|
|
#include <string> // char_traits, string
|
|
#include <utility> // make_pair, move
|
|
#include <vector> // vector
|
|
#ifdef __cpp_lib_byteswap
|
|
#include <bit> //byteswap
|
|
#endif
|
|
|
|
#include <nlohmann/detail/exceptions.hpp>
|
|
#include <nlohmann/detail/input/input_adapters.hpp>
|
|
#include <nlohmann/detail/input/json_sax.hpp>
|
|
#include <nlohmann/detail/input/lexer.hpp>
|
|
#include <nlohmann/detail/macro_scope.hpp>
|
|
#include <nlohmann/detail/meta/is_sax.hpp>
|
|
#include <nlohmann/detail/meta/type_traits.hpp>
|
|
#include <nlohmann/detail/string_concat.hpp>
|
|
#include <nlohmann/detail/value_t.hpp>
|
|
|
|
NLOHMANN_JSON_NAMESPACE_BEGIN
|
|
namespace detail
|
|
{
|
|
|
|
/// how to treat CBOR tags
|
|
enum class cbor_tag_handler_t
|
|
{
|
|
error, ///< throw a parse_error exception in case of a tag
|
|
ignore, ///< ignore tags
|
|
store ///< store tags as binary type
|
|
};
|
|
|
|
/*!
|
|
@brief determine system byte order
|
|
|
|
@return true if and only if system's byte order is little endian
|
|
|
|
@note from https://stackoverflow.com/a/1001328/266378
|
|
*/
|
|
inline bool little_endianness(int num = 1) noexcept
|
|
{
|
|
return *reinterpret_cast<char*>(&num) == 1;
|
|
}
|
|
|
|
/*!
|
|
@brief largest element count accepted for a UBJSON container of a valueless type
|
|
|
|
An element of type 'Z' (null), 'T' (true) or 'F' (false) is encoded by its
|
|
type marker alone, so an optimized container of one of those types has no
|
|
payload at all and its declared count is the only thing that decides how much
|
|
is allocated: `[$Z#L` followed by a large count turns some ten bytes of input
|
|
into that many values (see #2793, which reports 35 GB and 150 seconds). Every
|
|
other type costs at least one byte per element and is bounded by the end of
|
|
the input.
|
|
|
|
This is a sanity bound rather than a security boundary, and it is far above
|
|
any container met in practice. @ref binary_writer falls back to the
|
|
unoptimized encoding for longer containers, so that a value serialized by
|
|
this library can always be read back.
|
|
|
|
@sa https://github.com/nlohmann/json/issues/2793
|
|
*/
|
|
JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 20;
|
|
|
|
///////////////////
|
|
// binary reader //
|
|
///////////////////
|
|
|
|
/*!
|
|
@brief deserialization of CBOR, MessagePack, and UBJSON values
|
|
*/
|
|
template<typename BasicJsonType, typename InputAdapterType, typename SAX = json_sax_dom_parser<BasicJsonType, InputAdapterType>>
|
|
class binary_reader
|
|
{
|
|
using number_integer_t = typename BasicJsonType::number_integer_t;
|
|
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
|
|
using number_float_t = typename BasicJsonType::number_float_t;
|
|
using string_t = typename BasicJsonType::string_t;
|
|
using binary_t = typename BasicJsonType::binary_t;
|
|
using json_sax_t = SAX;
|
|
using char_type = typename InputAdapterType::char_type;
|
|
using char_int_type = typename char_traits<char_type>::int_type;
|
|
|
|
public:
|
|
/*!
|
|
@brief create a binary reader
|
|
|
|
@param[in] adapter input adapter to read from
|
|
*/
|
|
explicit binary_reader(InputAdapterType&& adapter, const input_format_t format = input_format_t::json) noexcept : ia(std::move(adapter)), input_format(format)
|
|
{
|
|
(void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
|
|
}
|
|
|
|
// make class move-only
|
|
binary_reader(const binary_reader&) = delete;
|
|
binary_reader(binary_reader&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor)
|
|
binary_reader& operator=(const binary_reader&) = delete;
|
|
binary_reader& operator=(binary_reader&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor)
|
|
~binary_reader() = default;
|
|
|
|
/*!
|
|
@param[in] format the binary format to parse
|
|
@param[in] sax_ a SAX event processor
|
|
@param[in] strict whether to expect the input to be consumed completed
|
|
@param[in] tag_handler how to treat CBOR tags
|
|
|
|
@return whether parsing was successful
|
|
*/
|
|
JSON_HEDLEY_NON_NULL(3)
|
|
bool sax_parse(const input_format_t format,
|
|
json_sax_t* sax_,
|
|
const bool strict = true,
|
|
const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error)
|
|
{
|
|
sax = sax_;
|
|
container_stack.clear();
|
|
bool result = false;
|
|
|
|
switch (format)
|
|
{
|
|
case input_format_t::bson:
|
|
result = parse_bson_internal();
|
|
break;
|
|
|
|
case input_format_t::cbor:
|
|
result = parse_cbor_internal(true, tag_handler);
|
|
break;
|
|
|
|
case input_format_t::msgpack:
|
|
result = parse_msgpack_internal();
|
|
break;
|
|
|
|
case input_format_t::ubjson:
|
|
case input_format_t::bjdata:
|
|
result = parse_ubjson_internal();
|
|
break;
|
|
|
|
case input_format_t::json: // LCOV_EXCL_LINE
|
|
default: // LCOV_EXCL_LINE
|
|
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
|
|
}
|
|
|
|
// strict mode: next byte must be EOF
|
|
if (result && strict)
|
|
{
|
|
if (input_format == input_format_t::ubjson || input_format == input_format_t::bjdata)
|
|
{
|
|
get_ignore_noop();
|
|
}
|
|
else
|
|
{
|
|
get();
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(current != char_traits<char_type>::eof()))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(110, chars_read,
|
|
exception_message(input_format, concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr));
|
|
}
|
|
}
|
|
|
|
return result;
|
|
}
|
|
|
|
private:
|
|
////////////////////////
|
|
// nested containers //
|
|
////////////////////////
|
|
|
|
/*!
|
|
@brief a container that has been opened and not closed yet
|
|
|
|
The binary readers do not call themselves once per nesting level. Like
|
|
@ref parser::sax_parse_internal, which does the same for JSON text, they
|
|
keep the containers they are inside of on a heap-allocated stack, so that
|
|
the native call stack does not grow with the nesting depth of the input
|
|
and a deeply nested value is bounded by memory rather than by the stack
|
|
(see #5104).
|
|
|
|
The members are ordered by decreasing alignment, which is the ordering that
|
|
keeps a struct from growing as members are added to it.
|
|
*/
|
|
struct container_frame
|
|
{
|
|
container_frame(const std::size_t remaining_, const bool is_object_,
|
|
const char_int_type type_marker_ = 0) noexcept
|
|
: remaining(remaining_), type_marker(type_marker_), is_object(is_object_) {}
|
|
|
|
/// number of elements that have not been read yet, or npos when the
|
|
/// container is not sized and ends at a marker instead
|
|
std::size_t remaining;
|
|
/// BSON: value of chars_read before this document's size prefix, which
|
|
/// check_bson_document_size() needs once the document has been read
|
|
std::size_t start_position = 0;
|
|
/// UBJSON/BJData: the type marker of an optimized container, so that
|
|
/// its elements are read without one of their own; 0 otherwise
|
|
char_int_type type_marker;
|
|
/// BSON: the size this document declares, in bytes
|
|
std::int32_t declared_size = 0;
|
|
/// whether to close this container with end_object() or end_array()
|
|
bool is_object;
|
|
};
|
|
|
|
/*!
|
|
@brief open a nested array or object
|
|
|
|
Emits the SAX start event and records the container. This is the only
|
|
place the binary readers start a container, so a check that rejects one
|
|
can be made here and is then guaranteed to run before the start event.
|
|
|
|
@param[in] is_object whether an object (true) or an array (false) begins
|
|
@param[in] len number of elements the container declares
|
|
|
|
@return whether the SAX parser accepted the start event
|
|
*/
|
|
bool enter_container(const bool is_object, const std::size_t len,
|
|
const char_int_type type_marker = 0)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(is_object ? !sax->start_object(len) : !sax->start_array(len)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
container_stack.emplace_back(len, is_object, type_marker);
|
|
return true;
|
|
}
|
|
|
|
/// @copydoc enter_container
|
|
bool enter_array(const std::size_t len, const char_int_type type_marker = 0)
|
|
{
|
|
return enter_container(/*is_object*/false, len, type_marker);
|
|
}
|
|
|
|
/// @copydoc enter_container
|
|
bool enter_object(const std::size_t len, const char_int_type type_marker = 0)
|
|
{
|
|
return enter_container(/*is_object*/true, len, type_marker);
|
|
}
|
|
|
|
//////////
|
|
// BSON //
|
|
//////////
|
|
|
|
/*!
|
|
@brief Validate a BSON document's declared size against the bytes read.
|
|
|
|
A BSON document starts with an int32 that counts its own total length in
|
|
bytes, including that prefix and the trailing 0x00. The reader is driven
|
|
by the terminator rather than the declared length, so without this check a
|
|
nested document could declare a length that disagrees with where its
|
|
terminator actually falls and quietly hand the bytes in between to the
|
|
enclosing document. A well-formed document is at least 5 bytes (the prefix
|
|
plus the terminator); the equality also rejects those impossible sizes,
|
|
since at least 5 bytes are always consumed.
|
|
|
|
@param[in] document_start value of chars_read before the size prefix
|
|
@param[in] document_size the declared document size
|
|
@return whether the declared size matches the number of bytes read
|
|
*/
|
|
bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::bson, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr));
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@brief Reads in a BSON-object and passes it to the SAX-parser.
|
|
@return whether a valid BSON-value was passed to the SAX parser
|
|
*/
|
|
bool open_bson_document(const bool is_object)
|
|
{
|
|
// recorded before the size prefix is read, because
|
|
// check_bson_document_size() measures the document from here
|
|
const std::size_t document_start = chars_read;
|
|
std::int32_t document_size{};
|
|
if (!get_number<std::int32_t, true>(input_format_t::bson, document_size))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!enter_container(is_object, detail::unknown_size())))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
container_frame& frame = container_stack.back();
|
|
frame.start_position = document_start;
|
|
frame.declared_size = document_size;
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@brief read a BSON document and everything nested inside it
|
|
|
|
Reads elements until the document that was begun here is complete,
|
|
resuming the enclosing document each time an embedded one ends, so that
|
|
the nesting depth of the input costs heap rather than native stack
|
|
(see #5104).
|
|
|
|
@return whether reading the document succeeded
|
|
*/
|
|
bool parse_bson_internal()
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!open_bson_document(/*is_object*/true)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
// the key currently being read; hoisted out of the loop so that its
|
|
// capacity is reused across elements and across nesting levels
|
|
string_t key;
|
|
|
|
while (true)
|
|
{
|
|
const auto element_type = get();
|
|
|
|
if (element_type == 0) // end of the innermost document
|
|
{
|
|
// a copy, not a reference: it must stay valid across the
|
|
// pop_back() below, which destroys the container_stack
|
|
// element it would otherwise alias
|
|
const container_frame top = container_stack.back();
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!check_bson_document_size(top.start_position, top.declared_size)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
container_stack.pop_back();
|
|
if (JSON_HEDLEY_UNLIKELY(top.is_object ? !sax->end_object() : !sax->end_array()))
|
|
{
|
|
return false;
|
|
}
|
|
// the document begun here is complete once it is not inside one
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
continue;
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "element list")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
const std::size_t element_type_parse_position = chars_read;
|
|
key.clear();
|
|
if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
// an array's elements are named "0", "1", ... in the wire format,
|
|
// and those names are not passed on
|
|
if (container_stack.back().is_object && !sax->key(key))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!parse_bson_element_internal(element_type, element_type_parse_position)))
|
|
{
|
|
return false;
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief Parses a C-style string from the BSON input.
|
|
@param[in,out] result A reference to the string variable where the read
|
|
string is to be stored.
|
|
@return `true` if the \x00-byte indicating the end of the string was
|
|
encountered before the EOF; false` indicates an unexpected EOF.
|
|
*/
|
|
bool get_bson_cstr(string_t& result)
|
|
{
|
|
auto out = std::back_inserter(result);
|
|
while (true)
|
|
{
|
|
get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "cstring")))
|
|
{
|
|
return false;
|
|
}
|
|
if (current == 0x00)
|
|
{
|
|
return true;
|
|
}
|
|
*out++ = static_cast<typename string_t::value_type>(current);
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief Parses a zero-terminated string of length @a len from the BSON
|
|
input.
|
|
@param[in] len The length (including the zero-byte at the end) of the
|
|
string to be read.
|
|
@param[in,out] result A reference to the string variable where the read
|
|
string is to be stored.
|
|
@tparam NumberType The type of the length @a len
|
|
@pre len >= 1
|
|
@return `true` if the string was successfully parsed
|
|
*/
|
|
template<typename NumberType>
|
|
bool get_bson_string(const NumberType len, string_t& result)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(len < 1))
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::bson, concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr));
|
|
}
|
|
|
|
return get_string(input_format_t::bson, len - static_cast<NumberType>(1), result) && get() != char_traits<char_type>::eof();
|
|
}
|
|
|
|
/*!
|
|
@brief Parses a byte array input of length @a len from the BSON input.
|
|
@param[in] len The length of the byte array to be read.
|
|
@param[in,out] result A reference to the binary variable where the read
|
|
array is to be stored.
|
|
@tparam NumberType The type of the length @a len
|
|
@pre len >= 0
|
|
@return `true` if the byte array was successfully parsed
|
|
*/
|
|
template<typename NumberType>
|
|
bool get_bson_binary(const NumberType len, binary_t& result)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(len < 0))
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::bson, concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr));
|
|
}
|
|
|
|
// All BSON binary values have a subtype
|
|
std::uint8_t subtype{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number<std::uint8_t>(input_format_t::bson, subtype)))
|
|
{
|
|
return false;
|
|
}
|
|
result.set_subtype(subtype);
|
|
|
|
return get_binary(input_format_t::bson, len, result);
|
|
}
|
|
|
|
/*!
|
|
@brief Read a BSON document element of the given @a element_type.
|
|
@param[in] element_type The BSON element type, c.f. http://bsonspec.org/spec.html
|
|
@param[in] element_type_parse_position The position in the input stream,
|
|
where the `element_type` was read.
|
|
@warning Not all BSON element types are supported yet. An unsupported
|
|
@a element_type will give rise to a parse_error.114:
|
|
Unsupported BSON record type 0x...
|
|
@return whether a valid BSON-object/array was passed to the SAX parser
|
|
*/
|
|
bool parse_bson_element_internal(const char_int_type element_type,
|
|
const std::size_t element_type_parse_position)
|
|
{
|
|
switch (element_type)
|
|
{
|
|
case 0x01: // double
|
|
{
|
|
double number{};
|
|
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
case 0x02: // string
|
|
{
|
|
std::int32_t len{};
|
|
string_t value;
|
|
return get_number<std::int32_t, true>(input_format_t::bson, len) && get_bson_string(len, value) && sax->string(value);
|
|
}
|
|
|
|
case 0x03: // object
|
|
{
|
|
return open_bson_document(/*is_object*/true);
|
|
}
|
|
|
|
case 0x04: // array
|
|
{
|
|
return open_bson_document(/*is_object*/false);
|
|
}
|
|
|
|
case 0x05: // binary
|
|
{
|
|
std::int32_t len{};
|
|
binary_t value;
|
|
return get_number<std::int32_t, true>(input_format_t::bson, len) && get_bson_binary(len, value) && sax->binary(value);
|
|
}
|
|
|
|
case 0x08: // boolean
|
|
{
|
|
std::uint8_t value{};
|
|
return get_number<std::uint8_t>(input_format_t::bson, value) && sax->boolean(value != 0);
|
|
}
|
|
|
|
case 0x0A: // null
|
|
{
|
|
return sax->null();
|
|
}
|
|
|
|
case 0x10: // int32
|
|
{
|
|
std::int32_t value{};
|
|
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
|
|
}
|
|
|
|
case 0x12: // int64
|
|
{
|
|
std::int64_t value{};
|
|
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
|
|
}
|
|
|
|
case 0x11: // uint64
|
|
{
|
|
std::uint64_t value{};
|
|
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
|
|
}
|
|
|
|
default: // anything else is not supported (yet)
|
|
{
|
|
std::array<char, 3> cr{{}};
|
|
static_cast<void>((std::snprintf)(cr.data(), cr.size(), "%.2hhX", static_cast<unsigned char>(element_type))); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
|
const std::string cr_str{cr.data()};
|
|
return sax->parse_error(element_type_parse_position, cr_str,
|
|
parse_error::create(114, element_type_parse_position, concat("Unsupported BSON record type 0x", cr_str), nullptr));
|
|
}
|
|
}
|
|
}
|
|
|
|
|
|
|
|
//////////
|
|
// CBOR //
|
|
//////////
|
|
|
|
template<typename NumberType>
|
|
bool get_cbor_negative_integer()
|
|
{
|
|
NumberType number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, number)))
|
|
{
|
|
return false;
|
|
}
|
|
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
|
|
if (number > max_val)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(),
|
|
parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
|
|
}
|
|
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
|
|
}
|
|
|
|
/*!
|
|
@param[in] get_char whether a new character should be retrieved from the
|
|
input (true) or whether the last read character should
|
|
be considered instead (false)
|
|
@param[in] tag_handler how CBOR tags should be treated
|
|
|
|
@return whether a valid CBOR value was passed to the SAX parser
|
|
*/
|
|
bool parse_cbor_value(const bool get_char,
|
|
const cbor_tag_handler_t tag_handler,
|
|
bool& tag_pending)
|
|
{
|
|
tag_pending = false;
|
|
|
|
switch (get_char ? get() : current)
|
|
{
|
|
// EOF
|
|
case char_traits<char_type>::eof():
|
|
return unexpect_eof(input_format_t::cbor, "value");
|
|
|
|
// Integer 0x00..0x17 (0..23)
|
|
case 0x00:
|
|
case 0x01:
|
|
case 0x02:
|
|
case 0x03:
|
|
case 0x04:
|
|
case 0x05:
|
|
case 0x06:
|
|
case 0x07:
|
|
case 0x08:
|
|
case 0x09:
|
|
case 0x0A:
|
|
case 0x0B:
|
|
case 0x0C:
|
|
case 0x0D:
|
|
case 0x0E:
|
|
case 0x0F:
|
|
case 0x10:
|
|
case 0x11:
|
|
case 0x12:
|
|
case 0x13:
|
|
case 0x14:
|
|
case 0x15:
|
|
case 0x16:
|
|
case 0x17:
|
|
return sax->number_unsigned(static_cast<number_unsigned_t>(current));
|
|
|
|
case 0x18: // Unsigned integer (one-byte uint8_t follows)
|
|
{
|
|
std::uint8_t number{};
|
|
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0x19: // Unsigned integer (two-byte uint16_t follows)
|
|
{
|
|
std::uint16_t number{};
|
|
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
|
|
{
|
|
std::uint32_t number{};
|
|
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
|
|
{
|
|
std::uint64_t number{};
|
|
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
// Negative integer -1-0x00..-1-0x17 (-1..-24)
|
|
case 0x20:
|
|
case 0x21:
|
|
case 0x22:
|
|
case 0x23:
|
|
case 0x24:
|
|
case 0x25:
|
|
case 0x26:
|
|
case 0x27:
|
|
case 0x28:
|
|
case 0x29:
|
|
case 0x2A:
|
|
case 0x2B:
|
|
case 0x2C:
|
|
case 0x2D:
|
|
case 0x2E:
|
|
case 0x2F:
|
|
case 0x30:
|
|
case 0x31:
|
|
case 0x32:
|
|
case 0x33:
|
|
case 0x34:
|
|
case 0x35:
|
|
case 0x36:
|
|
case 0x37:
|
|
return sax->number_integer(static_cast<std::int8_t>(0x20 - 1 - current));
|
|
|
|
case 0x38: // Negative integer (one-byte uint8_t follows)
|
|
return get_cbor_negative_integer<std::uint8_t>();
|
|
|
|
case 0x39: // Negative integer -1-n (two-byte uint16_t follows)
|
|
return get_cbor_negative_integer<std::uint16_t>();
|
|
|
|
case 0x3A: // Negative integer -1-n (four-byte uint32_t follows)
|
|
return get_cbor_negative_integer<std::uint32_t>();
|
|
|
|
case 0x3B: // Negative integer -1-n (eight-byte uint64_t follows)
|
|
return get_cbor_negative_integer<std::uint64_t>();
|
|
|
|
// Binary data (0x00..0x17 bytes follow)
|
|
case 0x40:
|
|
case 0x41:
|
|
case 0x42:
|
|
case 0x43:
|
|
case 0x44:
|
|
case 0x45:
|
|
case 0x46:
|
|
case 0x47:
|
|
case 0x48:
|
|
case 0x49:
|
|
case 0x4A:
|
|
case 0x4B:
|
|
case 0x4C:
|
|
case 0x4D:
|
|
case 0x4E:
|
|
case 0x4F:
|
|
case 0x50:
|
|
case 0x51:
|
|
case 0x52:
|
|
case 0x53:
|
|
case 0x54:
|
|
case 0x55:
|
|
case 0x56:
|
|
case 0x57:
|
|
case 0x58: // Binary data (one-byte uint8_t for n follows)
|
|
case 0x59: // Binary data (two-byte uint16_t for n follow)
|
|
case 0x5A: // Binary data (four-byte uint32_t for n follow)
|
|
case 0x5B: // Binary data (eight-byte uint64_t for n follow)
|
|
case 0x5F: // Binary data (indefinite length)
|
|
{
|
|
binary_t b;
|
|
return get_cbor_binary(b) && sax->binary(b);
|
|
}
|
|
|
|
// UTF-8 string (0x00..0x17 bytes follow)
|
|
case 0x60:
|
|
case 0x61:
|
|
case 0x62:
|
|
case 0x63:
|
|
case 0x64:
|
|
case 0x65:
|
|
case 0x66:
|
|
case 0x67:
|
|
case 0x68:
|
|
case 0x69:
|
|
case 0x6A:
|
|
case 0x6B:
|
|
case 0x6C:
|
|
case 0x6D:
|
|
case 0x6E:
|
|
case 0x6F:
|
|
case 0x70:
|
|
case 0x71:
|
|
case 0x72:
|
|
case 0x73:
|
|
case 0x74:
|
|
case 0x75:
|
|
case 0x76:
|
|
case 0x77:
|
|
case 0x78: // UTF-8 string (one-byte uint8_t for n follows)
|
|
case 0x79: // UTF-8 string (two-byte uint16_t for n follow)
|
|
case 0x7A: // UTF-8 string (four-byte uint32_t for n follow)
|
|
case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow)
|
|
case 0x7F: // UTF-8 string (indefinite length)
|
|
{
|
|
string_t s;
|
|
return get_cbor_string(s) && sax->string(s);
|
|
}
|
|
|
|
// array (0x00..0x17 data items follow)
|
|
case 0x80:
|
|
case 0x81:
|
|
case 0x82:
|
|
case 0x83:
|
|
case 0x84:
|
|
case 0x85:
|
|
case 0x86:
|
|
case 0x87:
|
|
case 0x88:
|
|
case 0x89:
|
|
case 0x8A:
|
|
case 0x8B:
|
|
case 0x8C:
|
|
case 0x8D:
|
|
case 0x8E:
|
|
case 0x8F:
|
|
case 0x90:
|
|
case 0x91:
|
|
case 0x92:
|
|
case 0x93:
|
|
case 0x94:
|
|
case 0x95:
|
|
case 0x96:
|
|
case 0x97:
|
|
return enter_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
|
|
|
|
case 0x98: // array (one-byte uint8_t for n follows)
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format_t::cbor, len) && enter_array(static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0x99: // array (two-byte uint16_t for n follow)
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::cbor, len) && enter_array(static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0x9A: // array (four-byte uint32_t for n follow)
|
|
{
|
|
std::uint32_t len{};
|
|
std::size_t size{};
|
|
return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "array") && enter_array(size);
|
|
}
|
|
|
|
case 0x9B: // array (eight-byte uint64_t for n follow)
|
|
{
|
|
std::uint64_t len{};
|
|
std::size_t size{};
|
|
return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "array") && enter_array(size);
|
|
}
|
|
|
|
case 0x9F: // array (indefinite length)
|
|
return enter_array(detail::unknown_size());
|
|
|
|
// map (0x00..0x17 pairs of data items follow)
|
|
case 0xA0:
|
|
case 0xA1:
|
|
case 0xA2:
|
|
case 0xA3:
|
|
case 0xA4:
|
|
case 0xA5:
|
|
case 0xA6:
|
|
case 0xA7:
|
|
case 0xA8:
|
|
case 0xA9:
|
|
case 0xAA:
|
|
case 0xAB:
|
|
case 0xAC:
|
|
case 0xAD:
|
|
case 0xAE:
|
|
case 0xAF:
|
|
case 0xB0:
|
|
case 0xB1:
|
|
case 0xB2:
|
|
case 0xB3:
|
|
case 0xB4:
|
|
case 0xB5:
|
|
case 0xB6:
|
|
case 0xB7:
|
|
return enter_object(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
|
|
|
|
case 0xB8: // map (one-byte uint8_t for n follows)
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format_t::cbor, len) && enter_object(static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0xB9: // map (two-byte uint16_t for n follow)
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::cbor, len) && enter_object(static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0xBA: // map (four-byte uint32_t for n follow)
|
|
{
|
|
std::uint32_t len{};
|
|
std::size_t size{};
|
|
return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "map") && enter_object(size);
|
|
}
|
|
|
|
case 0xBB: // map (eight-byte uint64_t for n follow)
|
|
{
|
|
std::uint64_t len{};
|
|
std::size_t size{};
|
|
return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "map") && enter_object(size);
|
|
}
|
|
|
|
case 0xBF: // map (indefinite length)
|
|
return enter_object(detail::unknown_size());
|
|
|
|
case 0xC0: // tagged item
|
|
case 0xC1:
|
|
case 0xC2:
|
|
case 0xC3:
|
|
case 0xC4:
|
|
case 0xC5:
|
|
case 0xC6:
|
|
case 0xC7:
|
|
case 0xC8:
|
|
case 0xC9:
|
|
case 0xCA:
|
|
case 0xCB:
|
|
case 0xCC:
|
|
case 0xCD:
|
|
case 0xCE:
|
|
case 0xCF:
|
|
case 0xD0:
|
|
case 0xD1:
|
|
case 0xD2:
|
|
case 0xD3:
|
|
case 0xD4:
|
|
case 0xD5:
|
|
case 0xD6:
|
|
case 0xD7:
|
|
case 0xD8: // tagged item (1 byte follows)
|
|
case 0xD9: // tagged item (2 bytes follow)
|
|
case 0xDA: // tagged item (4 bytes follow)
|
|
case 0xDB: // tagged item (8 bytes follow)
|
|
{
|
|
switch (tag_handler)
|
|
{
|
|
case cbor_tag_handler_t::error:
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr));
|
|
}
|
|
|
|
case cbor_tag_handler_t::ignore:
|
|
{
|
|
// ignore binary subtype
|
|
switch (current)
|
|
{
|
|
case 0xD8:
|
|
{
|
|
std::uint8_t subtype_to_ignore{};
|
|
if (!get_number(input_format_t::cbor, subtype_to_ignore))
|
|
{
|
|
return false;
|
|
}
|
|
break;
|
|
}
|
|
case 0xD9:
|
|
{
|
|
std::uint16_t subtype_to_ignore{};
|
|
if (!get_number(input_format_t::cbor, subtype_to_ignore))
|
|
{
|
|
return false;
|
|
}
|
|
break;
|
|
}
|
|
case 0xDA:
|
|
{
|
|
std::uint32_t subtype_to_ignore{};
|
|
if (!get_number(input_format_t::cbor, subtype_to_ignore))
|
|
{
|
|
return false;
|
|
}
|
|
break;
|
|
}
|
|
case 0xDB:
|
|
{
|
|
std::uint64_t subtype_to_ignore{};
|
|
if (!get_number(input_format_t::cbor, subtype_to_ignore))
|
|
{
|
|
return false;
|
|
}
|
|
break;
|
|
}
|
|
default:
|
|
break;
|
|
}
|
|
// the tagged value follows; it is read by the loop in
|
|
// parse_cbor_internal() rather than by recursing here
|
|
tag_pending = true;
|
|
return true;
|
|
}
|
|
|
|
case cbor_tag_handler_t::store:
|
|
{
|
|
binary_t b;
|
|
// use binary subtype and store in a binary container
|
|
switch (current)
|
|
{
|
|
case 0xD8:
|
|
{
|
|
std::uint8_t subtype{};
|
|
if (!get_number(input_format_t::cbor, subtype))
|
|
{
|
|
return false;
|
|
}
|
|
b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
|
|
break;
|
|
}
|
|
case 0xD9:
|
|
{
|
|
std::uint16_t subtype{};
|
|
if (!get_number(input_format_t::cbor, subtype))
|
|
{
|
|
return false;
|
|
}
|
|
b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
|
|
break;
|
|
}
|
|
case 0xDA:
|
|
{
|
|
std::uint32_t subtype{};
|
|
if (!get_number(input_format_t::cbor, subtype))
|
|
{
|
|
return false;
|
|
}
|
|
b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
|
|
break;
|
|
}
|
|
case 0xDB:
|
|
{
|
|
std::uint64_t subtype{};
|
|
if (!get_number(input_format_t::cbor, subtype))
|
|
{
|
|
return false;
|
|
}
|
|
b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
|
|
break;
|
|
}
|
|
default:
|
|
{
|
|
// as above, the tagged value is read by the caller
|
|
tag_pending = true;
|
|
return true;
|
|
}
|
|
}
|
|
get();
|
|
return get_cbor_binary(b) && sax->binary(b);
|
|
}
|
|
|
|
default: // LCOV_EXCL_LINE
|
|
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
|
|
return false; // LCOV_EXCL_LINE
|
|
}
|
|
}
|
|
|
|
case 0xF4: // false
|
|
return sax->boolean(false);
|
|
|
|
case 0xF5: // true
|
|
return sax->boolean(true);
|
|
|
|
case 0xF6: // null
|
|
return sax->null();
|
|
|
|
case 0xF9: // Half-Precision Float (two-byte IEEE 754)
|
|
{
|
|
const auto byte1_raw = get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "number")))
|
|
{
|
|
return false;
|
|
}
|
|
const auto byte2_raw = get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "number")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
const auto byte1 = static_cast<unsigned char>(byte1_raw);
|
|
const auto byte2 = static_cast<unsigned char>(byte2_raw);
|
|
|
|
// Code from RFC 8949, Appendix D, Figure 3:
|
|
// As half-precision floating-point numbers were only added
|
|
// to IEEE 754 in 2008, today's programming platforms often
|
|
// still only have limited support for them. It is very
|
|
// easy to include at least decoding support for them even
|
|
// without such support. An example of a small decoder for
|
|
// half-precision floating-point numbers in the C language
|
|
// is shown in Fig. 3.
|
|
const auto half = static_cast<unsigned int>((byte1 << 8u) + byte2);
|
|
const double val = [&half]
|
|
{
|
|
const int exp = (half >> 10u) & 0x1Fu;
|
|
const unsigned int mant = half & 0x3FFu;
|
|
JSON_ASSERT(exp <= 31);
|
|
JSON_ASSERT(mant <= 1023);
|
|
switch (exp)
|
|
{
|
|
case 0:
|
|
return std::ldexp(mant, -24);
|
|
case 31:
|
|
return (mant == 0)
|
|
? std::numeric_limits<double>::infinity()
|
|
: std::numeric_limits<double>::quiet_NaN();
|
|
default:
|
|
return std::ldexp(mant + 1024, exp - 25);
|
|
}
|
|
}();
|
|
return sax->number_float((half & 0x8000u) != 0
|
|
? static_cast<number_float_t>(-val)
|
|
: static_cast<number_float_t>(val), "");
|
|
}
|
|
|
|
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
|
|
{
|
|
float number{};
|
|
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
|
|
{
|
|
double number{};
|
|
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
default: // anything else (0xFF is handled inside the other types)
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr));
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reads a definite-length CBOR string
|
|
|
|
Reads everything @ref get_cbor_string accepts except the indefinite-length
|
|
form, which that function handles itself. The bytes are appended to @a
|
|
result, so consecutive chunks of an indefinite-length string can be read
|
|
into the same string.
|
|
|
|
@param[out] result string the bytes are appended to
|
|
|
|
@return whether string creation completed
|
|
|
|
@pre @a current is not EOF
|
|
*/
|
|
bool get_cbor_string_chunk(string_t& result)
|
|
{
|
|
switch (current)
|
|
{
|
|
// UTF-8 string (0x00..0x17 bytes follow)
|
|
case 0x60:
|
|
case 0x61:
|
|
case 0x62:
|
|
case 0x63:
|
|
case 0x64:
|
|
case 0x65:
|
|
case 0x66:
|
|
case 0x67:
|
|
case 0x68:
|
|
case 0x69:
|
|
case 0x6A:
|
|
case 0x6B:
|
|
case 0x6C:
|
|
case 0x6D:
|
|
case 0x6E:
|
|
case 0x6F:
|
|
case 0x70:
|
|
case 0x71:
|
|
case 0x72:
|
|
case 0x73:
|
|
case 0x74:
|
|
case 0x75:
|
|
case 0x76:
|
|
case 0x77:
|
|
{
|
|
return get_string(input_format_t::cbor, static_cast<unsigned int>(current) & 0x1Fu, result);
|
|
}
|
|
|
|
case 0x78: // UTF-8 string (one-byte uint8_t for n follows)
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
case 0x79: // UTF-8 string (two-byte uint16_t for n follow)
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
case 0x7A: // UTF-8 string (four-byte uint32_t for n follow)
|
|
{
|
|
std::uint32_t len{};
|
|
return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow)
|
|
{
|
|
std::uint64_t len{};
|
|
return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
default:
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
|
exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr));
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reads a CBOR string
|
|
|
|
This function first reads starting bytes to determine the expected
|
|
string length and then copies this number of bytes into a string.
|
|
Additionally, CBOR's strings with indefinite lengths are supported.
|
|
|
|
@param[out] result created string
|
|
|
|
@return whether string creation completed
|
|
*/
|
|
bool get_cbor_string(string_t& result)
|
|
{
|
|
// number of indefinite-length strings that have been opened and not
|
|
// closed yet. RFC 8949, Section 3.2.3 does not permit nesting them,
|
|
// but this reader has always accepted it, so the open levels are
|
|
// counted instead of recursed through, which overflowed the stack for
|
|
// an input of repeated 0x7F bytes (see #5104). Every chunk is appended
|
|
// to the same result, so no per-level state is needed.
|
|
std::size_t open = 0;
|
|
|
|
while (true)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "string")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (current == 0x7F) // UTF-8 string (indefinite length)
|
|
{
|
|
++open;
|
|
get();
|
|
continue;
|
|
}
|
|
|
|
// a break marker closes the innermost indefinite-length string;
|
|
// outside of one it is not a string and falls through to the error
|
|
if (open != 0 && current == 0xFF)
|
|
{
|
|
if (--open == 0)
|
|
{
|
|
return true;
|
|
}
|
|
get();
|
|
continue;
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!get_cbor_string_chunk(result)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (open == 0)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
get();
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reads a definite-length CBOR byte array
|
|
|
|
Reads everything @ref get_cbor_binary accepts except the indefinite-length
|
|
form, which that function handles itself. The bytes are appended to @a
|
|
result, so consecutive chunks of an indefinite-length byte array can be
|
|
read into the same byte array.
|
|
|
|
@param[out] result byte array the bytes are appended to
|
|
|
|
@return whether byte array creation completed
|
|
|
|
@pre @a current is not EOF
|
|
*/
|
|
bool get_cbor_binary_chunk(binary_t& result)
|
|
{
|
|
switch (current)
|
|
{
|
|
// Binary data (0x00..0x17 bytes follow)
|
|
case 0x40:
|
|
case 0x41:
|
|
case 0x42:
|
|
case 0x43:
|
|
case 0x44:
|
|
case 0x45:
|
|
case 0x46:
|
|
case 0x47:
|
|
case 0x48:
|
|
case 0x49:
|
|
case 0x4A:
|
|
case 0x4B:
|
|
case 0x4C:
|
|
case 0x4D:
|
|
case 0x4E:
|
|
case 0x4F:
|
|
case 0x50:
|
|
case 0x51:
|
|
case 0x52:
|
|
case 0x53:
|
|
case 0x54:
|
|
case 0x55:
|
|
case 0x56:
|
|
case 0x57:
|
|
{
|
|
return get_binary(input_format_t::cbor, static_cast<unsigned int>(current) & 0x1Fu, result);
|
|
}
|
|
|
|
case 0x58: // Binary data (one-byte uint8_t for n follows)
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format_t::cbor, len) &&
|
|
get_binary(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
case 0x59: // Binary data (two-byte uint16_t for n follow)
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::cbor, len) &&
|
|
get_binary(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
case 0x5A: // Binary data (four-byte uint32_t for n follow)
|
|
{
|
|
std::uint32_t len{};
|
|
return get_number(input_format_t::cbor, len) &&
|
|
get_binary(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
case 0x5B: // Binary data (eight-byte uint64_t for n follow)
|
|
{
|
|
std::uint64_t len{};
|
|
return get_number(input_format_t::cbor, len) &&
|
|
get_binary(input_format_t::cbor, len, result);
|
|
}
|
|
|
|
default:
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
|
exception_message(input_format_t::cbor, concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr));
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reads a CBOR byte array
|
|
|
|
This function first reads starting bytes to determine the expected
|
|
byte array length and then copies this number of bytes into the byte array.
|
|
Additionally, CBOR's byte arrays with indefinite lengths are supported.
|
|
|
|
@param[out] result created byte array
|
|
|
|
@return whether byte array creation completed
|
|
*/
|
|
bool get_cbor_binary(binary_t& result)
|
|
{
|
|
// the open indefinite-length byte arrays are counted rather than
|
|
// recursed through, for the reason given in @ref get_cbor_string
|
|
std::size_t open = 0;
|
|
|
|
while (true)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "binary")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (current == 0x5F) // Binary data (indefinite length)
|
|
{
|
|
++open;
|
|
get();
|
|
continue;
|
|
}
|
|
|
|
// a break marker closes the innermost indefinite-length byte
|
|
// array; outside of one it falls through to the error below
|
|
if (open != 0 && current == 0xFF)
|
|
{
|
|
if (--open == 0)
|
|
{
|
|
return true;
|
|
}
|
|
get();
|
|
continue;
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!get_cbor_binary_chunk(result)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (open == 0)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
get();
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief narrow a definite CBOR array/map length to std::size_t
|
|
|
|
A definite length is rejected if it does not fit in std::size_t or if it
|
|
equals detail::unknown_size(), which is reserved to mark an indefinite-
|
|
length container and would otherwise make the length read as indefinite.
|
|
Both cases exceed any container's max_size(), so no representable input
|
|
is affected.
|
|
|
|
@param[in] len the declared length
|
|
@param[out] result the length narrowed to std::size_t
|
|
@param[in] context "array" or "map", for the error message
|
|
@return whether the length is usable
|
|
*/
|
|
bool get_cbor_container_size(const std::uint64_t len, std::size_t& result, const char* context)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::size_t>(len) || len == detail::unknown_size()))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
|
|
exception_message(input_format_t::cbor, concat("excessive ", context, " size"), "size"), nullptr));
|
|
}
|
|
result = conditional_static_cast<std::size_t>(len);
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@brief read a CBOR value and everything nested inside it
|
|
|
|
Reads values until the one that was begun here is complete, resuming the
|
|
enclosing container after each element, so that the nesting depth of the
|
|
input costs heap rather than native stack (see #5104).
|
|
|
|
@param[in] get_char whether a new character should be retrieved from the
|
|
input (true) or whether the last read character
|
|
@a current should be considered instead
|
|
@param[in] tag_handler how CBOR tags should be treated
|
|
|
|
@return whether reading the value succeeded
|
|
*/
|
|
bool parse_cbor_internal(const bool get_char,
|
|
const cbor_tag_handler_t tag_handler)
|
|
{
|
|
// whether the next value starts at a fresh byte or at the one already
|
|
// read into `current`
|
|
bool fetch = get_char;
|
|
|
|
// the key currently being read; hoisted out of the loop so that its
|
|
// capacity is reused across elements and across nesting levels
|
|
string_t key;
|
|
|
|
while (true)
|
|
{
|
|
if (!container_stack.empty())
|
|
{
|
|
// a copy, not a reference: it must stay valid across the
|
|
// pop_back() below, which destroys the container_stack element
|
|
// it would otherwise alias
|
|
const container_frame top = container_stack.back();
|
|
bool at_end = false;
|
|
|
|
if (top.remaining != npos)
|
|
{
|
|
// definite length: the container ends once its elements
|
|
// have been read
|
|
at_end = (top.remaining == 0);
|
|
if (!at_end)
|
|
{
|
|
// claim the element about to be read
|
|
--container_stack.back().remaining;
|
|
if (top.is_object)
|
|
{
|
|
get();
|
|
}
|
|
}
|
|
fetch = true;
|
|
}
|
|
else
|
|
{
|
|
// indefinite length: the container ends at a break marker.
|
|
// Testing for it consumes a byte, which is the first byte
|
|
// of the next element when it is not one.
|
|
at_end = (get() == 0xFF);
|
|
fetch = top.is_object;
|
|
}
|
|
|
|
if (at_end)
|
|
{
|
|
container_stack.pop_back();
|
|
if (JSON_HEDLEY_UNLIKELY(top.is_object ? !sax->end_object() : !sax->end_array()))
|
|
{
|
|
return false;
|
|
}
|
|
// the value begun here is complete once its container is
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
continue;
|
|
}
|
|
|
|
if (top.is_object)
|
|
{
|
|
key.clear();
|
|
if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key)))
|
|
{
|
|
return false;
|
|
}
|
|
fetch = true;
|
|
}
|
|
}
|
|
|
|
// a tag is not a value of its own: read on until the tagged value
|
|
bool tag_pending = false;
|
|
do
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!parse_cbor_value(fetch, tag_handler, tag_pending)))
|
|
{
|
|
return false;
|
|
}
|
|
fetch = true;
|
|
}
|
|
while (tag_pending);
|
|
|
|
// a value that opened a container left it on the stack; one that
|
|
// did not, and that was not inside a container, was the whole value
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
}
|
|
}
|
|
|
|
/////////////
|
|
// MsgPack //
|
|
/////////////
|
|
|
|
/*!
|
|
@return whether a valid MessagePack value was passed to the SAX parser
|
|
*/
|
|
/*!
|
|
@brief read one MessagePack value
|
|
|
|
Reads a single value and passes it to the SAX parser. A value that begins
|
|
a container is not read to its end: the container is opened with
|
|
@ref enter_container and its elements are read by
|
|
@ref parse_msgpack_internal, so that nesting does not consume native stack.
|
|
|
|
@return whether reading the value succeeded
|
|
*/
|
|
bool parse_msgpack_value()
|
|
{
|
|
switch (get())
|
|
{
|
|
// EOF
|
|
case char_traits<char_type>::eof():
|
|
return unexpect_eof(input_format_t::msgpack, "value");
|
|
|
|
// positive fixint
|
|
case 0x00:
|
|
case 0x01:
|
|
case 0x02:
|
|
case 0x03:
|
|
case 0x04:
|
|
case 0x05:
|
|
case 0x06:
|
|
case 0x07:
|
|
case 0x08:
|
|
case 0x09:
|
|
case 0x0A:
|
|
case 0x0B:
|
|
case 0x0C:
|
|
case 0x0D:
|
|
case 0x0E:
|
|
case 0x0F:
|
|
case 0x10:
|
|
case 0x11:
|
|
case 0x12:
|
|
case 0x13:
|
|
case 0x14:
|
|
case 0x15:
|
|
case 0x16:
|
|
case 0x17:
|
|
case 0x18:
|
|
case 0x19:
|
|
case 0x1A:
|
|
case 0x1B:
|
|
case 0x1C:
|
|
case 0x1D:
|
|
case 0x1E:
|
|
case 0x1F:
|
|
case 0x20:
|
|
case 0x21:
|
|
case 0x22:
|
|
case 0x23:
|
|
case 0x24:
|
|
case 0x25:
|
|
case 0x26:
|
|
case 0x27:
|
|
case 0x28:
|
|
case 0x29:
|
|
case 0x2A:
|
|
case 0x2B:
|
|
case 0x2C:
|
|
case 0x2D:
|
|
case 0x2E:
|
|
case 0x2F:
|
|
case 0x30:
|
|
case 0x31:
|
|
case 0x32:
|
|
case 0x33:
|
|
case 0x34:
|
|
case 0x35:
|
|
case 0x36:
|
|
case 0x37:
|
|
case 0x38:
|
|
case 0x39:
|
|
case 0x3A:
|
|
case 0x3B:
|
|
case 0x3C:
|
|
case 0x3D:
|
|
case 0x3E:
|
|
case 0x3F:
|
|
case 0x40:
|
|
case 0x41:
|
|
case 0x42:
|
|
case 0x43:
|
|
case 0x44:
|
|
case 0x45:
|
|
case 0x46:
|
|
case 0x47:
|
|
case 0x48:
|
|
case 0x49:
|
|
case 0x4A:
|
|
case 0x4B:
|
|
case 0x4C:
|
|
case 0x4D:
|
|
case 0x4E:
|
|
case 0x4F:
|
|
case 0x50:
|
|
case 0x51:
|
|
case 0x52:
|
|
case 0x53:
|
|
case 0x54:
|
|
case 0x55:
|
|
case 0x56:
|
|
case 0x57:
|
|
case 0x58:
|
|
case 0x59:
|
|
case 0x5A:
|
|
case 0x5B:
|
|
case 0x5C:
|
|
case 0x5D:
|
|
case 0x5E:
|
|
case 0x5F:
|
|
case 0x60:
|
|
case 0x61:
|
|
case 0x62:
|
|
case 0x63:
|
|
case 0x64:
|
|
case 0x65:
|
|
case 0x66:
|
|
case 0x67:
|
|
case 0x68:
|
|
case 0x69:
|
|
case 0x6A:
|
|
case 0x6B:
|
|
case 0x6C:
|
|
case 0x6D:
|
|
case 0x6E:
|
|
case 0x6F:
|
|
case 0x70:
|
|
case 0x71:
|
|
case 0x72:
|
|
case 0x73:
|
|
case 0x74:
|
|
case 0x75:
|
|
case 0x76:
|
|
case 0x77:
|
|
case 0x78:
|
|
case 0x79:
|
|
case 0x7A:
|
|
case 0x7B:
|
|
case 0x7C:
|
|
case 0x7D:
|
|
case 0x7E:
|
|
case 0x7F:
|
|
return sax->number_unsigned(static_cast<number_unsigned_t>(current));
|
|
|
|
// fixmap
|
|
case 0x80:
|
|
case 0x81:
|
|
case 0x82:
|
|
case 0x83:
|
|
case 0x84:
|
|
case 0x85:
|
|
case 0x86:
|
|
case 0x87:
|
|
case 0x88:
|
|
case 0x89:
|
|
case 0x8A:
|
|
case 0x8B:
|
|
case 0x8C:
|
|
case 0x8D:
|
|
case 0x8E:
|
|
case 0x8F:
|
|
return enter_object(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x0Fu));
|
|
|
|
// fixarray
|
|
case 0x90:
|
|
case 0x91:
|
|
case 0x92:
|
|
case 0x93:
|
|
case 0x94:
|
|
case 0x95:
|
|
case 0x96:
|
|
case 0x97:
|
|
case 0x98:
|
|
case 0x99:
|
|
case 0x9A:
|
|
case 0x9B:
|
|
case 0x9C:
|
|
case 0x9D:
|
|
case 0x9E:
|
|
case 0x9F:
|
|
return enter_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x0Fu));
|
|
|
|
// fixstr
|
|
case 0xA0:
|
|
case 0xA1:
|
|
case 0xA2:
|
|
case 0xA3:
|
|
case 0xA4:
|
|
case 0xA5:
|
|
case 0xA6:
|
|
case 0xA7:
|
|
case 0xA8:
|
|
case 0xA9:
|
|
case 0xAA:
|
|
case 0xAB:
|
|
case 0xAC:
|
|
case 0xAD:
|
|
case 0xAE:
|
|
case 0xAF:
|
|
case 0xB0:
|
|
case 0xB1:
|
|
case 0xB2:
|
|
case 0xB3:
|
|
case 0xB4:
|
|
case 0xB5:
|
|
case 0xB6:
|
|
case 0xB7:
|
|
case 0xB8:
|
|
case 0xB9:
|
|
case 0xBA:
|
|
case 0xBB:
|
|
case 0xBC:
|
|
case 0xBD:
|
|
case 0xBE:
|
|
case 0xBF:
|
|
case 0xD9: // str 8
|
|
case 0xDA: // str 16
|
|
case 0xDB: // str 32
|
|
{
|
|
string_t s;
|
|
return get_msgpack_string(s) && sax->string(s);
|
|
}
|
|
|
|
case 0xC0: // nil
|
|
return sax->null();
|
|
|
|
case 0xC2: // false
|
|
return sax->boolean(false);
|
|
|
|
case 0xC3: // true
|
|
return sax->boolean(true);
|
|
|
|
case 0xC4: // bin 8
|
|
case 0xC5: // bin 16
|
|
case 0xC6: // bin 32
|
|
case 0xC7: // ext 8
|
|
case 0xC8: // ext 16
|
|
case 0xC9: // ext 32
|
|
case 0xD4: // fixext 1
|
|
case 0xD5: // fixext 2
|
|
case 0xD6: // fixext 4
|
|
case 0xD7: // fixext 8
|
|
case 0xD8: // fixext 16
|
|
{
|
|
binary_t b;
|
|
return get_msgpack_binary(b) && sax->binary(b);
|
|
}
|
|
|
|
case 0xCA: // float 32
|
|
{
|
|
float number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
case 0xCB: // float 64
|
|
{
|
|
double number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
case 0xCC: // uint 8
|
|
{
|
|
std::uint8_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0xCD: // uint 16
|
|
{
|
|
std::uint16_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0xCE: // uint 32
|
|
{
|
|
std::uint32_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0xCF: // uint 64
|
|
{
|
|
std::uint64_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 0xD0: // int 8
|
|
{
|
|
std::int8_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 0xD1: // int 16
|
|
{
|
|
std::int16_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 0xD2: // int 32
|
|
{
|
|
std::int32_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 0xD3: // int 64
|
|
{
|
|
std::int64_t number{};
|
|
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 0xDC: // array 16
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::msgpack, len) && enter_array(static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0xDD: // array 32
|
|
{
|
|
std::uint32_t len{};
|
|
return get_number(input_format_t::msgpack, len) && enter_array(conditional_static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0xDE: // map 16
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::msgpack, len) && enter_object(static_cast<std::size_t>(len));
|
|
}
|
|
|
|
case 0xDF: // map 32
|
|
{
|
|
std::uint32_t len{};
|
|
return get_number(input_format_t::msgpack, len) && enter_object(conditional_static_cast<std::size_t>(len));
|
|
}
|
|
|
|
// negative fixint
|
|
case 0xE0:
|
|
case 0xE1:
|
|
case 0xE2:
|
|
case 0xE3:
|
|
case 0xE4:
|
|
case 0xE5:
|
|
case 0xE6:
|
|
case 0xE7:
|
|
case 0xE8:
|
|
case 0xE9:
|
|
case 0xEA:
|
|
case 0xEB:
|
|
case 0xEC:
|
|
case 0xED:
|
|
case 0xEE:
|
|
case 0xEF:
|
|
case 0xF0:
|
|
case 0xF1:
|
|
case 0xF2:
|
|
case 0xF3:
|
|
case 0xF4:
|
|
case 0xF5:
|
|
case 0xF6:
|
|
case 0xF7:
|
|
case 0xF8:
|
|
case 0xF9:
|
|
case 0xFA:
|
|
case 0xFB:
|
|
case 0xFC:
|
|
case 0xFD:
|
|
case 0xFE:
|
|
case 0xFF:
|
|
return sax->number_integer(static_cast<std::int8_t>(current));
|
|
|
|
default: // anything else
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format_t::msgpack, concat("invalid byte: 0x", last_token), "value"), nullptr));
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reads a MessagePack string
|
|
|
|
This function first reads starting bytes to determine the expected
|
|
string length and then copies this number of bytes into a string.
|
|
|
|
@param[out] result created string
|
|
|
|
@return whether string creation completed
|
|
*/
|
|
bool get_msgpack_string(string_t& result)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::msgpack, "string")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
switch (current)
|
|
{
|
|
// fixstr
|
|
case 0xA0:
|
|
case 0xA1:
|
|
case 0xA2:
|
|
case 0xA3:
|
|
case 0xA4:
|
|
case 0xA5:
|
|
case 0xA6:
|
|
case 0xA7:
|
|
case 0xA8:
|
|
case 0xA9:
|
|
case 0xAA:
|
|
case 0xAB:
|
|
case 0xAC:
|
|
case 0xAD:
|
|
case 0xAE:
|
|
case 0xAF:
|
|
case 0xB0:
|
|
case 0xB1:
|
|
case 0xB2:
|
|
case 0xB3:
|
|
case 0xB4:
|
|
case 0xB5:
|
|
case 0xB6:
|
|
case 0xB7:
|
|
case 0xB8:
|
|
case 0xB9:
|
|
case 0xBA:
|
|
case 0xBB:
|
|
case 0xBC:
|
|
case 0xBD:
|
|
case 0xBE:
|
|
case 0xBF:
|
|
{
|
|
return get_string(input_format_t::msgpack, static_cast<unsigned int>(current) & 0x1Fu, result);
|
|
}
|
|
|
|
case 0xD9: // str 8
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result);
|
|
}
|
|
|
|
case 0xDA: // str 16
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result);
|
|
}
|
|
|
|
case 0xDB: // str 32
|
|
{
|
|
std::uint32_t len{};
|
|
return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result);
|
|
}
|
|
|
|
default:
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
|
exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr));
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reads a MessagePack byte array
|
|
|
|
This function first reads starting bytes to determine the expected
|
|
byte array length and then copies this number of bytes into a byte array.
|
|
|
|
@param[out] result created byte array
|
|
|
|
@return whether byte array creation completed
|
|
*/
|
|
bool get_msgpack_binary(binary_t& result)
|
|
{
|
|
// helper function to set the subtype
|
|
auto assign_and_return_true = [&result](std::int8_t subtype)
|
|
{
|
|
result.set_subtype(static_cast<std::uint8_t>(subtype));
|
|
return true;
|
|
};
|
|
|
|
switch (current)
|
|
{
|
|
case 0xC4: // bin 8
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format_t::msgpack, len) &&
|
|
get_binary(input_format_t::msgpack, len, result);
|
|
}
|
|
|
|
case 0xC5: // bin 16
|
|
{
|
|
std::uint16_t len{};
|
|
return get_number(input_format_t::msgpack, len) &&
|
|
get_binary(input_format_t::msgpack, len, result);
|
|
}
|
|
|
|
case 0xC6: // bin 32
|
|
{
|
|
std::uint32_t len{};
|
|
return get_number(input_format_t::msgpack, len) &&
|
|
get_binary(input_format_t::msgpack, len, result);
|
|
}
|
|
|
|
case 0xC7: // ext 8
|
|
{
|
|
std::uint8_t len{};
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, len) &&
|
|
get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, len, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xC8: // ext 16
|
|
{
|
|
std::uint16_t len{};
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, len) &&
|
|
get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, len, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xC9: // ext 32
|
|
{
|
|
std::uint32_t len{};
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, len) &&
|
|
get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, len, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xD4: // fixext 1
|
|
{
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, 1, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xD5: // fixext 2
|
|
{
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, 2, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xD6: // fixext 4
|
|
{
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, 4, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xD7: // fixext 8
|
|
{
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, 8, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
case 0xD8: // fixext 16
|
|
{
|
|
std::int8_t subtype{};
|
|
return get_number(input_format_t::msgpack, subtype) &&
|
|
get_binary(input_format_t::msgpack, 16, result) &&
|
|
assign_and_return_true(subtype);
|
|
}
|
|
|
|
default: // LCOV_EXCL_LINE
|
|
return false; // LCOV_EXCL_LINE
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief read a MessagePack value and everything nested inside it
|
|
|
|
Reads values until the one that was begun here is complete, resuming the
|
|
enclosing container each time an element ends, so that the nesting depth
|
|
of the input costs heap rather than native stack (see #5104).
|
|
|
|
@return whether reading the value succeeded
|
|
*/
|
|
bool parse_msgpack_internal()
|
|
{
|
|
// the key currently being read; hoisted out of the loop so that its
|
|
// capacity is reused across elements and across nesting levels
|
|
string_t key;
|
|
|
|
while (true)
|
|
{
|
|
if (!container_stack.empty())
|
|
{
|
|
// copied out before anything can push onto the stack and
|
|
// invalidate a reference into it
|
|
const bool is_object = container_stack.back().is_object;
|
|
|
|
if (container_stack.back().remaining == 0)
|
|
{
|
|
container_stack.pop_back();
|
|
if (JSON_HEDLEY_UNLIKELY(is_object ? !sax->end_object() : !sax->end_array()))
|
|
{
|
|
return false;
|
|
}
|
|
// the value begun here is complete once its container is
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
continue;
|
|
}
|
|
|
|
// claim the element about to be read
|
|
--container_stack.back().remaining;
|
|
|
|
if (is_object)
|
|
{
|
|
get();
|
|
key.clear();
|
|
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key)))
|
|
{
|
|
return false;
|
|
}
|
|
}
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!parse_msgpack_value()))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
// a value that opened a container left it on the stack; one that
|
|
// did not, and that was not inside a container, was the whole value
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
}
|
|
}
|
|
|
|
////////////
|
|
// UBJSON //
|
|
////////////
|
|
|
|
/*!
|
|
@param[in] get_char whether a new character should be retrieved from the
|
|
input (true, default) or whether the last read
|
|
character should be considered instead
|
|
|
|
@return whether a valid UBJSON value was passed to the SAX parser
|
|
*/
|
|
bool parse_ubjson_internal(const bool get_char = true)
|
|
{
|
|
// the key currently being read; hoisted out of the loop so that its
|
|
// capacity is reused across elements and across nesting levels
|
|
string_t key;
|
|
|
|
// the type marker of the value to read next
|
|
char_int_type prefix = get_char ? get_ignore_noop() : current;
|
|
|
|
while (true)
|
|
{
|
|
const std::size_t depth = container_stack.size();
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_value(prefix)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
// the value begun here is complete once it is not inside anything
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
|
|
// a value was completed rather than a container opened; a
|
|
// container that ends at a marker needs the next byte to test
|
|
if (container_stack.size() == depth && container_stack.back().remaining == npos)
|
|
{
|
|
get_ignore_noop();
|
|
}
|
|
|
|
// advance to the next element, closing the containers that ended.
|
|
// top is a copy, not a reference: it must stay valid across the
|
|
// pop_back() below, which destroys the container_stack element it
|
|
// would otherwise alias.
|
|
for (;;)
|
|
{
|
|
const container_frame top = container_stack.back();
|
|
|
|
if (top.remaining != npos)
|
|
{
|
|
if (top.remaining != 0)
|
|
{
|
|
--container_stack.back().remaining;
|
|
if (top.is_object)
|
|
{
|
|
key.clear();
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_string(key) || !sax->key(key)))
|
|
{
|
|
return false;
|
|
}
|
|
}
|
|
// an optimized container gives its elements no marker
|
|
prefix = (top.type_marker != 0) ? top.type_marker : get_ignore_noop();
|
|
break;
|
|
}
|
|
}
|
|
// the end marker is compared against a literal rather than
|
|
// against a conditional expression, because char_int_type is
|
|
// unsigned for some input adapters and MSVC then reports the
|
|
// comparison as a signed/unsigned mismatch
|
|
else if (top.is_object ? (current != '}') : (current != ']'))
|
|
{
|
|
// a container that ends at a marker is never optimized, so
|
|
// every element carries its own marker; for an object the
|
|
// byte tested above is the first byte of the key
|
|
if (top.is_object)
|
|
{
|
|
key.clear();
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_string(key, false) || !sax->key(key)))
|
|
{
|
|
return false;
|
|
}
|
|
prefix = get_ignore_noop();
|
|
}
|
|
else
|
|
{
|
|
prefix = current;
|
|
}
|
|
break;
|
|
}
|
|
|
|
container_stack.pop_back();
|
|
if (JSON_HEDLEY_UNLIKELY(top.is_object ? !sax->end_object() : !sax->end_array()))
|
|
{
|
|
return false;
|
|
}
|
|
if (container_stack.empty())
|
|
{
|
|
return true;
|
|
}
|
|
// the container that just ended was an element of the one
|
|
// below it, which may need the next byte for its own test
|
|
if (container_stack.back().remaining == npos)
|
|
{
|
|
get_ignore_noop();
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/*!
|
|
@brief reject a negative UBJSON/BJData string length
|
|
|
|
String and key lengths are written with signed integer markers (i, I, l,
|
|
L). A negative value is malformed; without this check get_string() would
|
|
silently treat it as an empty string and leave the following bytes to be
|
|
misread as the next value. This mirrors the non-negative check the
|
|
optimized-container count path already performs in get_ubjson_size_value.
|
|
|
|
@param[in] len the string length read from the input
|
|
@return whether the length is valid (non-negative)
|
|
*/
|
|
template<typename NumberType>
|
|
bool check_ubjson_string_length(const NumberType len)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(len < 0))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
|
|
exception_message(input_format, "string length must not be negative", "string"), nullptr));
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@brief reads a UBJSON string
|
|
|
|
This function is either called after reading the 'S' byte explicitly
|
|
indicating a string, or in case of an object key where the 'S' byte can be
|
|
left out.
|
|
|
|
@param[out] result created string
|
|
@param[in] get_char whether a new character should be retrieved from the
|
|
input (true, default) or whether the last read
|
|
character should be considered instead
|
|
|
|
@return whether string creation completed
|
|
*/
|
|
bool get_ubjson_string(string_t& result, const bool get_char = true)
|
|
{
|
|
if (get_char)
|
|
{
|
|
// no get_ignore_noop() here: the byte read next must be a string
|
|
// length type specification, and a no-op ('N') is not valid in
|
|
// that position. No-ops at positions where a value may appear are
|
|
// already consumed by the callers via get_ignore_noop().
|
|
get();
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
switch (current)
|
|
{
|
|
case 'U':
|
|
{
|
|
std::uint8_t len{};
|
|
return get_number(input_format, len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'i':
|
|
{
|
|
std::int8_t len{};
|
|
return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'I':
|
|
{
|
|
std::int16_t len{};
|
|
return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'l':
|
|
{
|
|
std::int32_t len{};
|
|
return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'L':
|
|
{
|
|
std::int64_t len{};
|
|
return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'u':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint16_t len{};
|
|
return get_number(input_format, len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'm':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint32_t len{};
|
|
return get_number(input_format, len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
case 'M':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint64_t len{};
|
|
return get_number(input_format, len) && get_string(input_format, len, result);
|
|
}
|
|
|
|
default:
|
|
break;
|
|
}
|
|
auto last_token = get_token_string();
|
|
std::string message;
|
|
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
message = "expected length type specification (U, i, I, l, L); last byte: 0x" + last_token;
|
|
}
|
|
else
|
|
{
|
|
message = "expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x" + last_token;
|
|
}
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "string"), nullptr));
|
|
}
|
|
|
|
/*!
|
|
@param[out] dim an integer vector storing the ND array dimensions
|
|
@return whether reading ND array size vector is successful
|
|
*/
|
|
bool get_ubjson_ndarray_size(std::vector<size_t>& dim)
|
|
{
|
|
std::pair<std::size_t, char_int_type> size_and_type;
|
|
size_t dimlen = 0;
|
|
bool no_ndarray = true;
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_type(size_and_type, no_ndarray)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (size_and_type.first != npos)
|
|
{
|
|
if (size_and_type.second != 0)
|
|
{
|
|
if (size_and_type.second != 'N')
|
|
{
|
|
for (std::size_t i = 0; i < size_and_type.first; ++i)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
|
|
{
|
|
return false;
|
|
}
|
|
dim.push_back(dimlen);
|
|
}
|
|
}
|
|
}
|
|
else
|
|
{
|
|
for (std::size_t i = 0; i < size_and_type.first; ++i)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray)))
|
|
{
|
|
return false;
|
|
}
|
|
dim.push_back(dimlen);
|
|
}
|
|
}
|
|
}
|
|
else
|
|
{
|
|
while (current != ']')
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, current)))
|
|
{
|
|
return false;
|
|
}
|
|
dim.push_back(dimlen);
|
|
get_ignore_noop();
|
|
}
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@param[out] result determined size
|
|
@param[in,out] is_ndarray for input, `true` means already inside an ndarray vector
|
|
or ndarray dimension is not allowed; `false` means ndarray
|
|
is allowed; for output, `true` means an ndarray is found;
|
|
is_ndarray can only return `true` when its initial value
|
|
is `false`
|
|
@param[in] prefix type marker if already read, otherwise set to 0
|
|
|
|
@return whether size determination completed
|
|
*/
|
|
bool get_ubjson_size_value(std::size_t& result, bool& is_ndarray, char_int_type prefix = 0)
|
|
{
|
|
if (prefix == 0)
|
|
{
|
|
prefix = get_ignore_noop();
|
|
}
|
|
|
|
switch (prefix)
|
|
{
|
|
case 'U':
|
|
{
|
|
std::uint8_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
result = static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case 'i':
|
|
{
|
|
std::int8_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
if (number < 0)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
|
|
exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr));
|
|
}
|
|
result = static_cast<std::size_t>(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char
|
|
return true;
|
|
}
|
|
|
|
case 'I':
|
|
{
|
|
std::int16_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
if (number < 0)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
|
|
exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr));
|
|
}
|
|
result = static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case 'l':
|
|
{
|
|
std::int32_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
if (number < 0)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
|
|
exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr));
|
|
}
|
|
result = static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case 'L':
|
|
{
|
|
std::int64_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
if (number < 0)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
|
|
exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr));
|
|
}
|
|
if (!value_in_range_of<std::size_t>(number))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
|
|
exception_message(input_format, "integer value overflow", "size"), nullptr));
|
|
}
|
|
result = static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case 'u':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint16_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
result = static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case 'm':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint32_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
result = conditional_static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case 'M':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint64_t number{};
|
|
if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number)))
|
|
{
|
|
return false;
|
|
}
|
|
if (!value_in_range_of<std::size_t>(number))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
|
|
exception_message(input_format, "integer value overflow", "size"), nullptr));
|
|
}
|
|
result = detail::conditional_static_cast<std::size_t>(number);
|
|
return true;
|
|
}
|
|
|
|
case '[':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
if (is_ndarray) // ndarray dimensional vector can only contain integers and cannot embed another array
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message(input_format, "ndarray dimensional vector is not allowed", "size"), nullptr));
|
|
}
|
|
std::vector<size_t> dim;
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_ndarray_size(dim)))
|
|
{
|
|
return false;
|
|
}
|
|
if (dim.size() == 1 || (dim.size() == 2 && dim.at(0) == 1)) // return normal array size if 1D row vector
|
|
{
|
|
result = dim.at(dim.size() - 1);
|
|
return true;
|
|
}
|
|
if (!dim.empty()) // if ndarray, convert to an object in JData annotated array format
|
|
{
|
|
for (auto i : dim) // test if any dimension in an ndarray is 0, if so, return a 1D empty container
|
|
{
|
|
if ( i == 0 )
|
|
{
|
|
result = 0;
|
|
return true;
|
|
}
|
|
}
|
|
|
|
string_t key = "_ArraySize_";
|
|
if (JSON_HEDLEY_UNLIKELY(!sax->start_object(3) || !sax->key(key) || !sax->start_array(dim.size())))
|
|
{
|
|
return false;
|
|
}
|
|
result = 1;
|
|
for (auto i : dim)
|
|
{
|
|
// Pre-multiplication overflow check: if i > 0 and result > SIZE_MAX/i, then result*i would overflow.
|
|
// This check must happen before multiplication since overflow detection after the fact is unreliable
|
|
// as modular arithmetic can produce any value, not just 0 or SIZE_MAX.
|
|
if (JSON_HEDLEY_UNLIKELY(i > 0 && result > (std::numeric_limits<std::size_t>::max)() / i))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
|
|
}
|
|
result *= i;
|
|
// Additional post-multiplication check to catch any edge cases the pre-check might miss
|
|
if (result == 0 || result == npos)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
|
|
}
|
|
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
|
|
{
|
|
return false;
|
|
}
|
|
}
|
|
is_ndarray = true;
|
|
return sax->end_array();
|
|
}
|
|
result = 0;
|
|
return true;
|
|
}
|
|
|
|
default:
|
|
break;
|
|
}
|
|
auto last_token = get_token_string();
|
|
std::string message;
|
|
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
message = "expected length type specification (U, i, I, l, L) after '#'; last byte: 0x" + last_token;
|
|
}
|
|
else
|
|
{
|
|
message = "expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x" + last_token;
|
|
}
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "size"), nullptr));
|
|
}
|
|
|
|
/*!
|
|
@brief determine the type and size for a container
|
|
|
|
In the optimized UBJSON format, a type and a size can be provided to allow
|
|
for a more compact representation.
|
|
|
|
@param[out] result pair of the size and the type
|
|
@param[in] inside_ndarray whether the parser is parsing an ND array dimensional vector
|
|
|
|
@return whether pair creation completed
|
|
*/
|
|
bool get_ubjson_size_type(std::pair<std::size_t, char_int_type>& result, bool inside_ndarray = false)
|
|
{
|
|
result.first = npos; // size
|
|
result.second = 0; // type
|
|
// seed the flag with the caller's context: inside an ndarray dimension
|
|
// vector another ndarray is not allowed, and get_ubjson_size_value()
|
|
// rejects it up front instead of reading it and reporting afterwards.
|
|
// Seeding it with `false` made every '#' of a "[#[#[..." chain descend
|
|
// another level, which overflowed the stack (see #5104).
|
|
bool is_ndarray = inside_ndarray;
|
|
|
|
get_ignore_noop();
|
|
|
|
if (current == '$')
|
|
{
|
|
result.second = get(); // must not ignore 'N', because 'N' maybe the type
|
|
if (input_format == input_format_t::bjdata
|
|
&& JSON_HEDLEY_UNLIKELY(std::binary_search(bjd_optimized_type_markers.begin(), bjd_optimized_type_markers.end(), result.second)))
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format, concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr));
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "type")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
get_ignore_noop();
|
|
if (JSON_HEDLEY_UNLIKELY(current != '#'))
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value")))
|
|
{
|
|
return false;
|
|
}
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format, concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr));
|
|
}
|
|
|
|
const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
|
|
// an ndarray was read here only if the flag flipped; when it was
|
|
// seeded true, get_ubjson_size_value() already rejected the nested
|
|
// dimension vector
|
|
if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
|
|
{
|
|
result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters
|
|
}
|
|
return is_error;
|
|
}
|
|
|
|
if (current == '#')
|
|
{
|
|
const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
|
|
if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read,
|
|
exception_message(input_format, "ndarray requires both type and size", "size"), nullptr));
|
|
}
|
|
return is_error;
|
|
}
|
|
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@param prefix the previously read or set type prefix
|
|
@return whether value creation completed
|
|
*/
|
|
bool get_ubjson_value(const char_int_type prefix)
|
|
{
|
|
switch (prefix)
|
|
{
|
|
case char_traits<char_type>::eof(): // EOF
|
|
return unexpect_eof(input_format, "value");
|
|
|
|
case 'T': // true
|
|
return sax->boolean(true);
|
|
case 'F': // false
|
|
return sax->boolean(false);
|
|
|
|
case 'Z': // null
|
|
return sax->null();
|
|
|
|
case 'B': // byte
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint8_t number{};
|
|
return get_number(input_format, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 'U':
|
|
{
|
|
std::uint8_t number{};
|
|
return get_number(input_format, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 'i':
|
|
{
|
|
std::int8_t number{};
|
|
return get_number(input_format, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 'I':
|
|
{
|
|
std::int16_t number{};
|
|
return get_number(input_format, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 'l':
|
|
{
|
|
std::int32_t number{};
|
|
return get_number(input_format, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 'L':
|
|
{
|
|
std::int64_t number{};
|
|
return get_number(input_format, number) && sax->number_integer(number);
|
|
}
|
|
|
|
case 'u':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint16_t number{};
|
|
return get_number(input_format, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 'm':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint32_t number{};
|
|
return get_number(input_format, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 'M':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
std::uint64_t number{};
|
|
return get_number(input_format, number) && sax->number_unsigned(number);
|
|
}
|
|
|
|
case 'h':
|
|
{
|
|
if (input_format != input_format_t::bjdata)
|
|
{
|
|
break;
|
|
}
|
|
const auto byte1_raw = get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number")))
|
|
{
|
|
return false;
|
|
}
|
|
const auto byte2_raw = get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number")))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
const auto byte1 = static_cast<unsigned char>(byte1_raw);
|
|
const auto byte2 = static_cast<unsigned char>(byte2_raw);
|
|
|
|
// Code from RFC 8949, Appendix D, Figure 3:
|
|
// As half-precision floating-point numbers were only added
|
|
// to IEEE 754 in 2008, today's programming platforms often
|
|
// still only have limited support for them. It is very
|
|
// easy to include at least decoding support for them even
|
|
// without such support. An example of a small decoder for
|
|
// half-precision floating-point numbers in the C language
|
|
// is shown in Fig. 3.
|
|
const auto half = static_cast<unsigned int>((byte2 << 8u) + byte1);
|
|
const double val = [&half]
|
|
{
|
|
const int exp = (half >> 10u) & 0x1Fu;
|
|
const unsigned int mant = half & 0x3FFu;
|
|
JSON_ASSERT(exp <= 31);
|
|
JSON_ASSERT(mant <= 1023);
|
|
switch (exp)
|
|
{
|
|
case 0:
|
|
return std::ldexp(mant, -24);
|
|
case 31:
|
|
return (mant == 0)
|
|
? std::numeric_limits<double>::infinity()
|
|
: std::numeric_limits<double>::quiet_NaN();
|
|
default:
|
|
return std::ldexp(mant + 1024, exp - 25);
|
|
}
|
|
}();
|
|
return sax->number_float((half & 0x8000u) != 0
|
|
? static_cast<number_float_t>(-val)
|
|
: static_cast<number_float_t>(val), "");
|
|
}
|
|
|
|
case 'd':
|
|
{
|
|
float number{};
|
|
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
case 'D':
|
|
{
|
|
double number{};
|
|
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
|
|
}
|
|
|
|
case 'H':
|
|
{
|
|
return get_ubjson_high_precision_number();
|
|
}
|
|
|
|
case 'C': // char
|
|
{
|
|
get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "char")))
|
|
{
|
|
return false;
|
|
}
|
|
if (JSON_HEDLEY_UNLIKELY(current > 127))
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
|
|
exception_message(input_format, concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr));
|
|
}
|
|
string_t s(1, static_cast<typename string_t::value_type>(current));
|
|
return sax->string(s);
|
|
}
|
|
|
|
case 'S': // string
|
|
{
|
|
string_t s;
|
|
return get_ubjson_string(s) && sax->string(s);
|
|
}
|
|
|
|
case '[': // array
|
|
return get_ubjson_array();
|
|
|
|
case '{': // object
|
|
return get_ubjson_object();
|
|
|
|
default: // anything else
|
|
break;
|
|
}
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format, "invalid byte: 0x" + last_token, "value"), nullptr));
|
|
}
|
|
|
|
/*!
|
|
@return whether array creation completed
|
|
*/
|
|
bool get_ubjson_array()
|
|
{
|
|
std::pair<std::size_t, char_int_type> size_and_type;
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_type(size_and_type)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
// if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata):
|
|
// {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]}
|
|
|
|
if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
|
|
{
|
|
size_and_type.second &= ~(static_cast<char_int_type>(1) << 8); // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker
|
|
auto it = std::lower_bound(bjd_types_map.begin(), bjd_types_map.end(), size_and_type.second, [](const bjd_type & p, char_int_type t)
|
|
{
|
|
return p.first < t;
|
|
});
|
|
string_t key = "_ArrayType_";
|
|
if (JSON_HEDLEY_UNLIKELY(it == bjd_types_map.end() || it->first != size_and_type.second))
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format, "invalid byte: 0x" + last_token, "type"), nullptr));
|
|
}
|
|
|
|
string_t type = it->second; // sax->string() takes a reference
|
|
if (JSON_HEDLEY_UNLIKELY(!sax->key(key) || !sax->string(type)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (size_and_type.second == 'C' || size_and_type.second == 'B')
|
|
{
|
|
size_and_type.second = 'U';
|
|
}
|
|
|
|
key = "_ArrayData_";
|
|
if (JSON_HEDLEY_UNLIKELY(!sax->key(key) || !sax->start_array(size_and_type.first) ))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
for (std::size_t i = 0; i < size_and_type.first; ++i)
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_value(size_and_type.second)))
|
|
{
|
|
return false;
|
|
}
|
|
}
|
|
|
|
return (sax->end_array() && sax->end_object());
|
|
}
|
|
|
|
// If BJData type marker is 'B' decode as binary
|
|
if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B')
|
|
{
|
|
binary_t result;
|
|
return get_binary(input_format, size_and_type.first, result) && sax->binary(result);
|
|
}
|
|
|
|
if (size_and_type.first != npos)
|
|
{
|
|
// reading an element of a valueless type consumes no input, so the
|
|
// declared count alone decides how much is allocated; the check is
|
|
// made before the start event so that no container is opened that
|
|
// is then abandoned. See @ref max_valueless_container_size.
|
|
if (JSON_HEDLEY_UNLIKELY((size_and_type.second == 'Z' || size_and_type.second == 'T' || size_and_type.second == 'F')
|
|
&& size_and_type.first > max_valueless_container_size))
|
|
{
|
|
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
|
|
exception_message(input_format, "excessive array size", "size"), nullptr));
|
|
}
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (size_and_type.second == 'N')
|
|
{
|
|
// a no-op is not a value, so a container of them holds none;
|
|
// the declared size has already been passed to the SAX parser
|
|
container_stack.back().remaining = 0;
|
|
}
|
|
|
|
return true;
|
|
}
|
|
|
|
return enter_array(detail::unknown_size());
|
|
}
|
|
|
|
/*!
|
|
@return whether object creation completed
|
|
*/
|
|
bool get_ubjson_object()
|
|
{
|
|
std::pair<std::size_t, char_int_type> size_and_type;
|
|
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_type(size_and_type)))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
// do not accept ND-array size in objects in BJData
|
|
if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
|
|
{
|
|
auto last_token = get_token_string();
|
|
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
|
|
exception_message(input_format, "BJData object does not support ND-array size in optimized format", "object"), nullptr));
|
|
}
|
|
|
|
if (size_and_type.first != npos)
|
|
{
|
|
return enter_object(size_and_type.first, size_and_type.second);
|
|
}
|
|
|
|
return enter_object(detail::unknown_size());
|
|
}
|
|
|
|
// Note, no reader for UBJSON binary types is implemented because they do
|
|
// not exist
|
|
|
|
bool get_ubjson_high_precision_number()
|
|
{
|
|
// get the size of the following number string
|
|
std::size_t size{};
|
|
bool no_ndarray = true;
|
|
auto res = get_ubjson_size_value(size, no_ndarray);
|
|
if (JSON_HEDLEY_UNLIKELY(!res))
|
|
{
|
|
return res;
|
|
}
|
|
|
|
// get number string
|
|
std::vector<char> number_vector;
|
|
for (std::size_t i = 0; i < size; ++i)
|
|
{
|
|
get();
|
|
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number")))
|
|
{
|
|
return false;
|
|
}
|
|
number_vector.push_back(static_cast<char>(current));
|
|
}
|
|
|
|
// parse number string
|
|
using ia_type = decltype(detail::input_adapter(number_vector));
|
|
auto number_lexer = detail::lexer<BasicJsonType, ia_type>(detail::input_adapter(number_vector), false);
|
|
const auto result_number = number_lexer.scan();
|
|
const auto number_string = number_lexer.get_token_string();
|
|
const auto result_remainder = number_lexer.scan();
|
|
|
|
using token_type = typename detail::lexer_base<BasicJsonType>::token_type;
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input))
|
|
{
|
|
return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read,
|
|
exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr));
|
|
}
|
|
|
|
switch (result_number)
|
|
{
|
|
case token_type::value_integer:
|
|
return sax->number_integer(number_lexer.get_number_integer());
|
|
case token_type::value_unsigned:
|
|
return sax->number_unsigned(number_lexer.get_number_unsigned());
|
|
case token_type::value_float:
|
|
{
|
|
const auto parsed_float = number_lexer.get_number_float();
|
|
if (JSON_HEDLEY_UNLIKELY(!std::isfinite(parsed_float)))
|
|
{
|
|
return sax->parse_error(
|
|
chars_read,
|
|
number_string,
|
|
out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr));
|
|
}
|
|
// number_string is a std::string, while the SAX interface takes a
|
|
// string_t; convert explicitly, as the two are only implicitly
|
|
// convertible for some string types
|
|
return sax->number_float(parsed_float, string_t(number_string.data(), number_string.size()));
|
|
}
|
|
case token_type::uninitialized:
|
|
case token_type::literal_true:
|
|
case token_type::literal_false:
|
|
case token_type::literal_null:
|
|
case token_type::value_string:
|
|
case token_type::begin_array:
|
|
case token_type::begin_object:
|
|
case token_type::end_array:
|
|
case token_type::end_object:
|
|
case token_type::name_separator:
|
|
case token_type::value_separator:
|
|
case token_type::parse_error:
|
|
case token_type::end_of_input:
|
|
case token_type::literal_or_value:
|
|
default:
|
|
return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read,
|
|
exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr));
|
|
}
|
|
}
|
|
|
|
///////////////////////
|
|
// Utility functions //
|
|
///////////////////////
|
|
|
|
/*!
|
|
@brief get next character from the input
|
|
|
|
This function provides the interface to the used input adapter. It does
|
|
not throw in case the input reached EOF, but returns a -'ve valued
|
|
`char_traits<char_type>::eof()` in that case.
|
|
|
|
@return character read from the input
|
|
*/
|
|
char_int_type get()
|
|
{
|
|
++chars_read;
|
|
return current = ia.get_character();
|
|
}
|
|
|
|
/*!
|
|
@brief get_to read into a primitive type
|
|
|
|
This function provides the interface to the used input adapter. It does
|
|
not throw in case the input reached EOF, but returns false instead
|
|
|
|
@return bool, whether the read was successful
|
|
*/
|
|
template<class T>
|
|
bool get_to(T& dest, const input_format_t format, const char* context)
|
|
{
|
|
auto new_chars_read = ia.get_elements(&dest);
|
|
chars_read += new_chars_read;
|
|
if (JSON_HEDLEY_UNLIKELY(new_chars_read < sizeof(T)))
|
|
{
|
|
// in case of failure, advance position by 1 to report the failing location
|
|
++chars_read;
|
|
sax->parse_error(chars_read, "<end of file>", parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr));
|
|
return false;
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@return character read from the input after ignoring all 'N' entries
|
|
*/
|
|
char_int_type get_ignore_noop()
|
|
{
|
|
do
|
|
{
|
|
get();
|
|
}
|
|
while (current == 'N');
|
|
|
|
return current;
|
|
}
|
|
|
|
template<class NumberType>
|
|
static void byte_swap(NumberType& number)
|
|
{
|
|
constexpr std::size_t sz = sizeof(number);
|
|
#ifdef __cpp_lib_byteswap
|
|
if constexpr (sz == 1)
|
|
{
|
|
return;
|
|
}
|
|
else if constexpr(std::is_integral_v<NumberType>)
|
|
{
|
|
number = std::byteswap(number);
|
|
return;
|
|
}
|
|
else
|
|
{
|
|
#endif
|
|
auto* ptr = reinterpret_cast<std::uint8_t*>(&number);
|
|
for (std::size_t i = 0; i < sz / 2; ++i)
|
|
{
|
|
std::swap(ptr[i], ptr[sz - i - 1]);
|
|
}
|
|
#ifdef __cpp_lib_byteswap
|
|
}
|
|
#endif
|
|
}
|
|
|
|
/*
|
|
@brief read a number from the input
|
|
|
|
@tparam NumberType the type of the number
|
|
@param[in] format the current format (for diagnostics)
|
|
@param[out] result number of type @a NumberType
|
|
|
|
@return whether conversion completed
|
|
|
|
@note This function needs to respect the system's endianness, because
|
|
bytes in CBOR, MessagePack, and UBJSON are stored in network order
|
|
(big endian) and therefore need reordering on little endian systems.
|
|
On the other hand, BSON and BJData use little endian and should reorder
|
|
on big endian systems.
|
|
*/
|
|
template<typename NumberType, bool InputIsLittleEndian = false>
|
|
bool get_number(const input_format_t format, NumberType& result)
|
|
{
|
|
// read in the original format
|
|
|
|
if (JSON_HEDLEY_UNLIKELY(!get_to(result, format, "number")))
|
|
{
|
|
return false;
|
|
}
|
|
if (is_little_endian != (InputIsLittleEndian || format == input_format_t::bjdata))
|
|
{
|
|
byte_swap(result);
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@brief create a string by reading characters from the input
|
|
|
|
@tparam NumberType the type of the number
|
|
@param[in] format the current format (for diagnostics)
|
|
@param[in] len number of characters to read
|
|
@param[out] result string created by reading @a len bytes
|
|
|
|
@return whether string creation completed
|
|
|
|
@note We can not reserve @a len bytes for the result, because @a len
|
|
may be too large. Usually, @ref unexpect_eof() detects the end of
|
|
the input before we run out of string memory.
|
|
*/
|
|
template<typename NumberType>
|
|
bool get_string(const input_format_t format,
|
|
const NumberType len,
|
|
string_t& result)
|
|
{
|
|
return get_bytes(format, len, "string", result);
|
|
}
|
|
|
|
/*!
|
|
@brief create a byte array by reading bytes from the input
|
|
|
|
@tparam NumberType the type of the number
|
|
@param[in] format the current format (for diagnostics)
|
|
@param[in] len number of bytes to read
|
|
@param[out] result byte array created by reading @a len bytes
|
|
|
|
@return whether byte array creation completed
|
|
|
|
@note We can not reserve @a len bytes for the result, because @a len
|
|
may be too large. Usually, @ref unexpect_eof() detects the end of
|
|
the input before we run out of memory.
|
|
*/
|
|
template<typename NumberType>
|
|
bool get_binary(const input_format_t format,
|
|
const NumberType len,
|
|
binary_t& result)
|
|
{
|
|
return get_bytes(format, len, "binary", result);
|
|
}
|
|
|
|
/*!
|
|
@brief read @a len bytes from the input into a string or byte container
|
|
|
|
@tparam NumberType the type of the length
|
|
@tparam ContainerType the destination container (string_t or binary_t)
|
|
@param[in] format the current format (for diagnostics)
|
|
@param[in] len number of bytes to read
|
|
@param[in] context further context information (for diagnostics)
|
|
@param[out] result container the bytes are appended to
|
|
|
|
@return whether reading completed
|
|
|
|
@note We cannot reserve @a len bytes for the result up front, because
|
|
@a len may be far larger than the actual input. Instead we read in
|
|
bounded chunks, so the peak allocation is capped regardless of the
|
|
claimed length while the per-byte loop is replaced by block copies
|
|
(a std::memcpy for contiguous inputs). @ref unexpect_eof() still
|
|
detects a premature end of input.
|
|
*/
|
|
template<typename NumberType, typename ContainerType>
|
|
bool get_bytes(const input_format_t format,
|
|
NumberType len,
|
|
const char* context,
|
|
ContainerType& result)
|
|
{
|
|
// upper bound on the number of bytes read (and allocated) per chunk
|
|
constexpr std::size_t chunk_size = 4096;
|
|
|
|
while (len > 0)
|
|
{
|
|
// number of bytes to read this iteration: min(chunk_size, len),
|
|
// computed without truncating chunk_size to a narrow NumberType
|
|
const std::size_t wanted = (static_cast<std::uintmax_t>(len) < static_cast<std::uintmax_t>(chunk_size))
|
|
? static_cast<std::size_t>(len)
|
|
: chunk_size;
|
|
const std::size_t old_size = result.size();
|
|
result.resize(old_size + wanted);
|
|
// resize() is required to make size() exactly old_size + wanted;
|
|
// that is the room get_elements() is allowed to write into
|
|
JSON_ASSERT(result.size() == old_size + wanted);
|
|
const std::size_t bytes_read = ia.get_elements(&result[old_size], wanted);
|
|
chars_read += bytes_read;
|
|
if (JSON_HEDLEY_UNLIKELY(bytes_read < wanted))
|
|
{
|
|
// premature end of input: shrink to what was actually read and
|
|
// report the failure at the first missing byte (same position
|
|
// accounting as get_to() for partial number reads)
|
|
result.resize(old_size + bytes_read);
|
|
++chars_read;
|
|
current = char_traits<char_type>::eof();
|
|
return unexpect_eof(format, context);
|
|
}
|
|
// a full chunk was read; get_elements() never returns more than requested
|
|
JSON_ASSERT(bytes_read == wanted);
|
|
len = static_cast<NumberType>(len - static_cast<NumberType>(wanted));
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@param[in] format the current format (for diagnostics)
|
|
@param[in] context further context information (for diagnostics)
|
|
@return whether the last read character is not EOF
|
|
*/
|
|
JSON_HEDLEY_NON_NULL(3)
|
|
bool unexpect_eof(const input_format_t format, const char* context) const
|
|
{
|
|
if (JSON_HEDLEY_UNLIKELY(current == char_traits<char_type>::eof()))
|
|
{
|
|
return sax->parse_error(chars_read, "<end of file>",
|
|
parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr));
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/*!
|
|
@return a string representation of the last read byte
|
|
*/
|
|
std::string get_token_string() const
|
|
{
|
|
std::array<char, 3> cr{{}};
|
|
static_cast<void>((std::snprintf)(cr.data(), cr.size(), "%.2hhX", static_cast<unsigned char>(current))); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
|
return std::string{cr.data()};
|
|
}
|
|
|
|
/*!
|
|
@param[in] format the current format
|
|
@param[in] detail a detailed error message
|
|
@param[in] context further context information
|
|
@return a message string to use in the parse_error exceptions
|
|
*/
|
|
std::string exception_message(const input_format_t format,
|
|
const std::string& detail,
|
|
const std::string& context) const
|
|
{
|
|
std::string error_msg = "syntax error while parsing ";
|
|
|
|
switch (format)
|
|
{
|
|
case input_format_t::cbor:
|
|
error_msg += "CBOR";
|
|
break;
|
|
|
|
case input_format_t::msgpack:
|
|
error_msg += "MessagePack";
|
|
break;
|
|
|
|
case input_format_t::ubjson:
|
|
error_msg += "UBJSON";
|
|
break;
|
|
|
|
case input_format_t::bson:
|
|
error_msg += "BSON";
|
|
break;
|
|
|
|
case input_format_t::bjdata:
|
|
error_msg += "BJData";
|
|
break;
|
|
|
|
case input_format_t::json: // LCOV_EXCL_LINE
|
|
default: // LCOV_EXCL_LINE
|
|
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
|
|
}
|
|
|
|
return concat(error_msg, ' ', context, ": ", detail);
|
|
}
|
|
|
|
private:
|
|
static JSON_INLINE_VARIABLE constexpr std::size_t npos = detail::unknown_size();
|
|
|
|
/// input adapter
|
|
InputAdapterType ia;
|
|
|
|
/// the current character
|
|
char_int_type current = char_traits<char_type>::eof();
|
|
|
|
/// the number of characters read
|
|
std::size_t chars_read = 0;
|
|
|
|
/// whether we can assume little endianness
|
|
const bool is_little_endian = little_endianness();
|
|
|
|
/// input format
|
|
const input_format_t input_format = input_format_t::json;
|
|
|
|
/// the SAX parser
|
|
json_sax_t* sax = nullptr;
|
|
|
|
/// the containers that have been opened and not closed yet; see @ref container_frame
|
|
std::vector<container_frame> container_stack{};
|
|
|
|
// excluded markers in bjdata optimized type
|
|
#define JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_ \
|
|
make_array<char_int_type>('F', 'H', 'N', 'S', 'T', 'Z', '[', '{')
|
|
|
|
#define JSON_BINARY_READER_MAKE_BJD_TYPES_MAP_ \
|
|
make_array<bjd_type>( \
|
|
bjd_type{'B', "byte"}, \
|
|
bjd_type{'C', "char"}, \
|
|
bjd_type{'D', "double"}, \
|
|
bjd_type{'I', "int16"}, \
|
|
bjd_type{'L', "int64"}, \
|
|
bjd_type{'M', "uint64"}, \
|
|
bjd_type{'U', "uint8"}, \
|
|
bjd_type{'d', "single"}, \
|
|
bjd_type{'i', "int8"}, \
|
|
bjd_type{'l', "int32"}, \
|
|
bjd_type{'m', "uint32"}, \
|
|
bjd_type{'u', "uint16"})
|
|
|
|
JSON_PRIVATE_UNLESS_TESTED:
|
|
// lookup tables
|
|
// NOLINTNEXTLINE(cppcoreguidelines-non-private-member-variables-in-classes)
|
|
const decltype(JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_) bjd_optimized_type_markers =
|
|
JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_;
|
|
|
|
using bjd_type = std::pair<char_int_type, string_t>;
|
|
// NOLINTNEXTLINE(cppcoreguidelines-non-private-member-variables-in-classes)
|
|
const decltype(JSON_BINARY_READER_MAKE_BJD_TYPES_MAP_) bjd_types_map =
|
|
JSON_BINARY_READER_MAKE_BJD_TYPES_MAP_;
|
|
|
|
#undef JSON_BINARY_READER_MAKE_BJD_OPTIMIZED_TYPE_MARKERS_
|
|
#undef JSON_BINARY_READER_MAKE_BJD_TYPES_MAP_
|
|
};
|
|
|
|
#ifndef JSON_HAS_CPP_17
|
|
template<typename BasicJsonType, typename InputAdapterType, typename SAX>
|
|
constexpr std::size_t binary_reader<BasicJsonType, InputAdapterType, SAX>::npos;
|
|
#endif
|
|
|
|
} // namespace detail
|
|
NLOHMANN_JSON_NAMESPACE_END
|