From c72f37a40d36220255ee9eb097e74ee8950a2681 Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Tue, 15 Sep 2026 20:46:37 +0200 Subject: [PATCH] Support custom object/array types and improve template parameter handling (#5443) * docs: document the implicit requirements on basic_json's template parameters The requirements that basic_json places on its eleven template parameters were only implied by how the library uses the resulting object_t, array_t, string_t, etc. Consumers had to discover them by trial and error. Add "Template Parameter Requirements" collecting them, split into what is always required and what is only required when a particular part of the API is instantiated. Notable findings that were previously undocumented: - ObjectType must provide a key_compare member type (actual_object_comparator names object_t::key_compare in both arms of a std::conditional), and its third template parameter is used as a comparator, so std::unordered_map cannot be used without a wrapper. - ArrayType must provide capacity() -- push_back(), emplace_back(), operator+=(), and operator[](size_type) call it unconditionally -- and needs random-access iterators, so std::deque and std::list do not work. - StringType needs contiguous, null-terminated data(), a one-byte value_type, and either assignability from std::to_string or an ADL int_to_string(). - NumberFloatType must be float, double, or long double for parsing and serialization; the integer types must satisfy std::is_integral. - AllocatorType must be stateless, support incomplete types, and use plain pointers. - BooleanType and the number types are union members and must be trivial. Link the new page from the basic_json overview, the types feature page, and the individual type alias pages, and correct the container examples given for ObjectType (std::unordered_map) and ArrayType (std::list), which do not work. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * Fix object_comparator_t for object types without key_compare detail::actual_object_comparator selected between object_t::key_compare and default_object_comparator_t with std::conditional. Both type arguments of std::conditional are named eagerly, so object_t::key_compare had to exist regardless of the condition, and the has_key_compare guard added in 3.11.0 never took effect: any ObjectType without a key_compare member type failed to compile while instantiating basic_json itself. Use detected_or_t instead, which resolves through a SFINAE partial specialization and only names object_t::key_compare when it exists. The selected type is unchanged for every object type that compiled before, so object_comparator_t -- a public member type -- keeps its meaning and ABI. has_key_compare had no other users and is removed. Add a regression test using an adapter around std::unordered_map, which has no key_compare; it fails to compile without this change. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * docs: list the types that are known to work for each template parameter Follow up on the template parameter requirements page: state, for every template parameter, which concrete types work and where they stop working. Each entry was verified by compiling and running a common workload (DOM access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that instantiation. Findings worth calling out: - ObjectType no longer needs a key_compare member type, so the std::unordered_map adapter only has to restore the template argument order. A hash-ordered ObjectType works everywhere except unflatten(), which reconstructs an array only when it meets the reference token 0 before the other indices. - ArrayType: std::deque works when wrapped to add capacity(); std::list does not. - StringType: std::pmr::string and std::basic_string with a custom allocator compile for the DOM, dump, and parse, but not for the binary readers, flatten, or diff, because the library assigns std::string values to string_t and int_to_string cannot be overloaded for a type in namespace std. - NumberFloatType: long double works for dump and parse but not for the binary formats, which have no encoding for it. - BinaryType: std::vector supports assignment, get, and the binary formats, but neither dump nor std::hash. Also record the object_comparator_t fix in its version history. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * Fix unflatten and binary dumping for non-default configurations unflatten() decided between array and object by looking at the first reference token it happened to see for a node: it started an array only when that token was 0. With a sorted object type the token 0 always arrives first, so the result was correct by accident; with an object type whose iteration order is unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the keys "0", "1", and "2" instead of an array. Collect the pointer prefixes that have a reference token 0 among their children before building the result, and let get_and_create() consult that set. The outcome is now independent of the iteration order and matches, for every input, what a sorted object type produced before: a value is restored as an array if and only if one of its keys is 0. Iterating the flattened object in a different order would have been simpler, but it would have changed the key order of the result for insertion-ordered object types. The serializer, std::hash, and the UBJSON writer converted the elements of a binary value to an integer implicitly, which does not compile for a BinaryType whose value type is std::byte, and which made dump() write the bytes of a signed value type as negative numbers. Convert to std::uint8_t explicitly in all three places, so every byte type dumps as 0..255. The default std::vector configuration is unaffected. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * docs: note which Abseil containers can be used as template arguments Checked against Abseil release 20250127.0 with the same workload as the other entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON round-trip, flatten, hash), with and without JSON_DIAGNOSTICS. absl::flat_hash_map and absl::node_hash_map work as ObjectType through an adapter that restores the template argument order and makes erase(iterator) return the following iterator, which Abseil's returns as void. The page now carries that adapter, and notes that absl::flat_hash_map does not keep references to the mapped values valid across insertions while absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS already refreshes the parent pointers conservatively for them. absl::btree_map and absl::InlinedVector cannot be used at all: object_t and array_t are formed while basic_json is still incomplete, and both inspect their value type at class scope. std::map and std::vector are required by the standard to tolerate this, third-party containers generally are not, so the page states the constraint on its own rather than only per container. absl::InlinedVector does work as BinaryType, where it is instantiated with a complete type. absl::FixedArray and absl::Cord are not usable. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * Relax the ArrayType and ObjectType requirements Two requirements forced users of otherwise suitable containers to write a wrapper, and neither was load-bearing. array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and operator[](size_type), but set_parent() only looks at the value under JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it through array_capacity(), which reports unknown_size() when diagnostics are off or when the array type has no capacity() at all, and treat an unknown capacity as "the elements may have moved" so the parent pointers are refreshed conservatively. std::deque now works as ArrayType, in both builds, and capacity() is no longer named at all in a default build. Since the capacity is now only meaningful for array insertions, it moves out of set_parent() into set_parent_after_array_insert(). basic_json::erase(iterator) assigned the object's erase() return value, which requires the container to return the following iterator. Abseil's hash maps return void to avoid computing a successor the caller may not need. Detect that and compute the successor before erasing; containers that return an iterator, including the vector-backed ordered_map where a precomputed successor would be wrong, keep the existing path. Together these leave an Abseil hash map needing only an alias that restores the template argument order, and no adapter at all for std::deque. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * Do not require string_t to be convertible from std::string Three places built a std::string and handed it to something expecting a string_t: the UBJSON high-precision number reader, which every binary reader instantiates, and the BSON writer's array element size calculation and write. That silently required string_t to be implicitly convertible from std::string, which std::string itself and types with a string_view conversion satisfy, but many string types do not. Construct the string_t explicitly from the data and size, which the requirements already cover. This makes boost::container::string, eastl::string, std::pmr::string, and std::basic_string with a custom allocator work as StringType, none of which could previously be used with any binary format. Add binary format coverage to the alt_string test, which had none, including a UBJSON high-precision number -- the case that goes through the reader path. BSON stays uncovered there: it additionally needs string_t::find(value_type), which alt_string does not provide. Also record which containers from Boost, Abseil, and EASTL work for each template parameter, and correct two claims: std::pmr::string is usable after this change, and tsl::ordered_map is not usable at all, because its iterators expose the mapped value as const while basic_json modifies it in place. Signed-off-by: Niels Lohmann * docs: record compatibility for the common header-only hash maps ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType through the same adapter as Abseil's and Boost's hash maps, which only has to restore the template argument order. phmap::btree_map and robin_hood::unordered_node_map do not: like the other btree containers they require a complete value type. Note that none of these hash maps defines key_compare, so every one of them depends on object_comparator_t falling back to default_object_comparator_t. Signed-off-by: Niels Lohmann * docs: record Folly and the remaining vector replacements Folly works, with the caveat that its headers need C++20: folly::fbstring as StringType, folly::fbvector and folly::small_vector as ArrayType, folly::fbvector as BinaryType, and folly::F14NodeMap as ObjectType through the usual argument-order adapter. folly::F14FastMap is the exception and requires a complete value type. For ArrayType, boost::container::devector, boost::container::static_vector (within its fixed capacity), and std::pmr::vector work as well. Signed-off-by: Niels Lohmann * docs: cover fifo_map, gtl, folly::sorted_vector_map, and Qt nlohmann::fifo_map works through the adapter that has always been documented for it, and preserves the insertion order. Restore its mention in the object order page, which was dropped together with the tsl::ordered_map one: unlike ordered_map it keeps a lookup index, so it is the insertion-ordered option without the quadratic cost. gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter through an alias that drops the allocator, whose value type it disagrees on. gtl::btree_map does not, for the same reason as the other btree containers. None of the Qt containers can be used, each for its own reason: QMap has no value_type, QHash iterators yield the mapped value rather than a pair, QList has no max_size(), QByteArray spells empty() as isEmpty(), and QString is UTF-16. Signed-off-by: Niels Lohmann * docs: qualify the std::pmr::string support claim Listing std::pmr::string as fully supported was an overclaim: it was only ever checked with the default memory resource, which is not what PMR is for. basic_json cannot be given an allocator or a memory resource, so a pmr string inside a value always allocates from std::pmr::get_default_resource(), and assigning an arena-backed string into a value silently drops its resource, because polymorphic_allocator does not propagate on copy construction. Passing polymorphic_allocator as AllocatorType does not compile either. Only the process-global set_default_resource() redirects these allocations. Say so, and separate the row from std::basic_string with a custom stateless allocator, which is unaffected. Signed-off-by: Niels Lohmann * docs: remove a duplicated StringType compatibility section The StringType section carried two 'Compatible types' tables and two copies of the reference-implementation tip. The second table was a stale copy from before the binary format string fixes and still listed std::pmr::string and std::basic_string with a custom allocator as unusable, contradicting the corrected table a few lines above it, and it dragged along the old explanation that blamed int_to_string. Drop the stale copy and put the surviving table before the notes, so the 'see below' in the std::pmr::string row points forwards. Signed-off-by: Niels Lohmann * docs: correct the template parameter requirements after independent verification Every claim on the page was re-checked by compiling and running it, including the rows that say a type cannot be used, which were checked to fail for the documented reason and not merely to fail. Twenty-four claims were wrong. The most consequential: the incomplete-type constraint applies to ObjectType only. object_t is instantiated inside the class definition, because it is probed for key_compare; array_t is only named there and is not instantiated until basic_json is complete. So eastl::vector, QList and QVector are not excluded by incomplete types at all -- they simply have no max_size() -- and absl::InlinedVector is excluded for a subtler reason of its own. Further corrections: ObjectType does not need erase(key), which has a fallback, but does need at(key) for UBJSON output; only == and < are used, or == and <=> under C++20, not all six; the documented adapter does not fit ankerl or robin_hood. ArrayType needs no initializer-list insert, and value_type, the (count, value) constructor and swappability are per-function, not always. BinaryType needs a range insert for CBOR indefinite-length byte strings and does not need push_back. StringType needs append(const StringType&) unconditionally, and does not need operator!= or operator== against const char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is needed by diff, items and std::hash rather than by JSON Pointer or flatten. BooleanType must be implicitly convertible from bool, and JSONSerializer's second parameter need not carry a default. std::pmr::string was wrong in the other direction this time: a moved-in string does keep its memory resource, and later growth allocates from it. Only copies land on the default resource. Five requirement violations are not caught at compile time rather than the two the page claimed; they are now listed together up front. Split every compatibility table into what works and what does not, as the reasons in the second half are the useful part. Signed-off-by: Niels Lohmann * Reduce the string_t and array_t members the library requires Several members were required only because of how the library happened to be written, not because the functionality needs them. Dropping them widens the set of usable string and array types, and one of them was also a performance problem. string_t: - c_str() is gone. Every call site already knew the length and passed it along, so data() is enough. The one place that did not, the diagnostics path in exceptions.hpp, now builds the token from data() and size(), which also stops it from truncating keys that contain a null byte. - back() is gone; the serializer indexes the last character instead. - find(str, pos), replace(), and substr() are gone. escape() and unescape() rebuilt the string with one replace() per escaped character, which moves the tail every time: escaping a string of n characters that all need escaping cost O(n^2). Both now scan with find_first_of() -- a member the pointer parser already required -- and append whole runs, so the common case is one search and one copy. Escaping 64000 tildes drops from 717 ms to 20 ms; a string with nothing to escape gets faster too (8.4 ms to 5.8 ms), because the scan is still a single memchr per pass. json_pointer::split() takes its reference tokens with the (const char*, size_type) constructor rather than substr(). - json_pointer::to_string() accumulates with concat instead of letting concat default to std::string and converting afterwards, so streaming a json_pointer no longer requires string_t to be assignable from a std::string. array_t: - at(size_type) is gone. basic_json::at(size_type) checked the index by calling array_t::at() and translating std::out_of_range, which also required the array type to throw that exact exception. It now compares against size() and uses operator[]. The thrown exception, its message, and the behaviour under JSON_NOEXCEPTION are unchanged. The BSON writer wrote the terminating null byte out of the string's own buffer (size() + 1). It now writes the byte itself, so string_t::data() need not be null-terminated for to_bson(). The tests pin the reduced API: alt_string loses the five dropped members and gains coverage of the escaping paths, and a std::vector whose at() is hidden is used as an ArrayType. Signed-off-by: Niels Lohmann * docs: record the reduced string_t and array_t requirements Drop c_str(), back(), find(str, pos), replace(), and substr() from the StringType requirements and at(size_type) from the ArrayType ones, and note the string assignment the JSON pointer code performs. Streaming a json_pointer no longer needs assignability from a std::string. Add the non-null-terminated data() to the list of violations that are not diagnosed at compile time -- it was described in the StringType section but missing from the summary at the top -- and correct the QString row, which no longer fails for the c_str() it lacks. JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one went away with array_t::at(). Describe what the library actually catches. Signed-off-by: Niels Lohmann * Use character literals for the signed BinaryType test MSVC rejects char(0xFF) with C4310 (cast truncates constant value), which the Windows workflow treats as an error. The character literals carry the same byte values without a narrowing cast. Signed-off-by: Niels Lohmann * Do not instantiate a hash map with an incomplete basic_json in the tests object_t is probed for key_compare inside the definition of basic_json, so it is instantiated while basic_json is still incomplete. Whether a hash map survives that depends on the standard library: libstdc++ 9 needs the size of the mapped type to instantiate std::unordered_map's node type and rejects the adapter, which broke the GCC 9 builds. The test now derives its no-key_compare object type from std::map -- which does cope -- and shadows the inherited key_compare member type with an entity that is not a type, so the library's probe finds none, exactly as for a hash map. The unflatten() order-independence checks in unit-json_pointer already cover the behaviour that the unordered object type was there for. The limitation is documented for std::unordered_map. Also address two Clang-Tidy findings the earlier commits introduced: erase_from_object() declares its iterator with auto, and at(size_type) checks the type first and then falls through to the return instead of throwing from an else branch. Signed-off-by: Niels Lohmann * Keep diagnostic key paths null-terminated Building the token from data() and size() kept an embedded null byte in the key, and since what() hands out a C string, that truncated the whole message rather than just the key: to_bson() on a key containing U+0000 reported "[json.exception.out_of_range.409] (/en" instead of the full explanation. This broke test-bson under JSON_DIAGNOSTICS. Constructing from data() alone stops at the first null byte, which is what c_str() did before, so the message is unchanged -- without requiring string_t to provide c_str(). Signed-off-by: Niels Lohmann * Do not parse the value in the array-at() test JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception message, which a parsed value has and an in-memory one does not, so the two message checks failed in that configuration. Build the array in memory instead of parsing it; the test is about at(size_type) not needing array_t::at(), and the byte range is beside the point. Signed-off-by: Niels Lohmann * Move the custom BinaryType tests into their own translation unit The two sections added to unit-regression2.cpp brought a third full basic_json instantiation into a translation unit that was already large. With Clang on MinGW that pushed the object over the reach of a 32-bit relocation and test-regression2_cpp20.exe failed to link: relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata' unit-regression2.cpp is restored to exactly what it was before, and the coverage moves to unit-custom-binary-type.cpp, next to the object and array type tests it belongs with. The signed value type is now also covered in C++11, where std::byte is not available. Signed-off-by: Niels Lohmann * Do not require the container iterators to be nothrow move constructible iter_impl declared its defaulted move operations noexcept. The exception specification a defaulted function gets implicitly follows from its members, here internal_iterator, which holds the object and array iterators. libstdc++ gives std::deque's iterator a user-provided copy constructor without noexcept before version 11, so the implicit specification is noexcept(false) and does not match the declared one. That deletes the function -- and with g++ 4.8, which predates CWG 1778, it is an error outright: error: function 'iter_impl >::iter_impl( iter_impl&&)' defaulted on its first declaration with an exception-specification that differs from the implicit declaration So std::deque, which this branch documents as a usable array type, could not be used with an older standard library. Leaving the specification to be computed cannot mismatch; iteration_proxy_value already spells out the same condition next door. The default configuration is unaffected: json::iterator, json::const_iterator and ordered_json::iterator stay nothrow move constructible and move assignable, which the test now checks so it cannot regress unnoticed. Signed-off-by: Niels Lohmann * Address two Clang-Tidy findings the custom container tests exposed Both come from instantiating basic_json with containers other than the default ones, and neither shows up with the Clang-Tidy version available outside CI: - insert(const_iterator, basic_json&&) forwards its by-value iterator to the const-reference overload. performance-unnecessary-value-param asks for the copy to be a move; it only fires for an iterator that is not trivially copyable, as std::deque's is not. The NOLINT on the function does not cover it, because the finding is reported where the parameter is used rather than where it is declared. Move it, which is what the check asks for and is a (very small) improvement in its own right. - cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed the inherited key_compare member type. An enum class would not do, since it declares a type of that name and the probe would find it again; a member function declaration hides the name just as well. Signed-off-by: Niels Lohmann * Assert the iterators' exception specification relative to the container The test pinned that nlohmann::json's iterators stay nothrow movable after iter_impl's defaulted move operations lost their declared noexcept. That is not a property of the library, though: the exception specification is now computed from the container iterators, so it holds only for standard library implementations whose iterators are themselves nothrow movable. MSVC's checked iterators before VS2017 are not -- _Iterator_base12 registers the iterator with the container's debug proxy in a copy constructor that carries no noexcept -- so the assertions fail on a Visual Studio 2015 debug build, which is the one debug configuration in the AppVeyor matrix and has no counterpart in the GitHub Actions matrix. Assert what the change actually guarantees instead: the iterators are nothrow movable exactly when the object and array iterators they are built from are. That still pins the default configuration against a silent regression, and it is true whatever the standard library provides. Signed-off-by: Niels Lohmann * Detect a void-returning erase() through a named trait erase_from_object() distinguished its two overloads with a decltype of a member call written inline in a default template argument. Every other detection in the library goes through the detector machinery in detected.hpp instead -- has_erase_with_key_type is the same question about the same member function -- and the inline form is the one shape older compilers are least reliable about. Express it the same way: detect_erase_with_iterator plus is_detected_exact, both of which the library already relies on elsewhere. No behaviour changes. Signed-off-by: Niels Lohmann * Give the custom container types only the constructors the library uses The three container types in the new tests inherited every constructor of their base with using Base::Base. That asks for more than the test needs: the library builds an object or an array by default construction, by copy or move, and -- when converting between two basic_json types or from an initializer list -- from an iterator range. Declaring those directly makes the requirement visible in the test, and keeps object types out of a corner where a compiler has to declare std::map's whole constructor set for a derived class while basic_json is still incomplete. Signed-off-by: Niels Lohmann * Temporarily disable the new custom container tests AppVeyor is the only CI that builds MSVC 2015 and 2017, and it has now rejected three heads of this branch. Its build log is not reachable from where this is being worked on, so the verdict is a single bit and the cause has to be narrowed down by bisection. Everything else stays: the library changes, the reduced alt_string, and the unflatten() tests. If AppVeyor passes with these three translation units disabled, the cause is one of the six basic_json instantiations they add; if it fails, it is in the library. Either way this commit is reverted. Signed-off-by: Niels Lohmann * Guard the disabled tests with a macro rather than #if 0 Clang-Tidy's readability-avoid-unconditional-preprocessor-if rejects a literal #if 0. Use a macro that is never defined instead, which the check does not look at. Still temporary, and reverted together with the previous commit. Signed-off-by: Niels Lohmann * Re-enable the array and binary container tests AppVeyor passed with all three new translation units disabled, so the library changes, the reduced alt_string, and the unflatten() tests are fine on MSVC 2015 and 2017; the cause is one of the six basic_json instantiations the new tests add. Bring back two of the three. If AppVeyor passes again, the cause is in unit-custom-object-type.cpp, which is the one still disabled; if it fails, it is in one of these two and needs one more split. Still temporary. Signed-off-by: Niels Lohmann * Diagnose two silently violated template parameter requirements Both were on the list of requirements that are not caught at compile time and corrupt values rather than failing, and both are a plain size comparison: - A BinaryType whose value_type is wider than one byte, which the readers and writers reinterpret as raw bytes anyway. - A NumberUnsignedType too narrow to hold the absolute value of every NumberIntegerType value, which makes basic_json(INT64_MIN).dump() yield -0 for std::int64_t with std::uint32_t. Neither static_assert rejects a configuration that worked before: both only fire where the result was already wrong. Also add the two comments the review asked for, in write_bson_string() and calc_bson_array_size(), matching the ones their counterparts already carry. Signed-off-by: Niels Lohmann * Align the template parameter tables and record what is now diagnosed Every table in the page is reformatted so each column is exactly as wide as its widest cell, which is what the review asked for in a dozen places: the separator rows that ran two dashes long, the stray spaces, and the columns padded well past their content. The row listing six containers that require a complete mapped type is split in two so that one cell no longer sets the width of the whole table. Content changes: NumberUnsignedType is described as any unsigned integer type at least as wide as NumberIntegerType rather than any unsigned integer type; the two requirements that are now static_asserts move out of the list of violations that are not caught at compile time; and the two places that require a non-const operator[] say why data() will not do (std::string has no non-const data() before C++17). Signed-off-by: Niels Lohmann * Bisect the other way: only the object container tests The previous head touched only docs/, which AppVeyor's only_commits filter skips, so it produced no build and no status at all -- the pull request looked green without ever having been built on MSVC 2015 or 2017. Swap the guards instead of repeating that step: unit-custom-object-type.cpp is enabled and the array and binary translation units are disabled. AppVeyor already passed with all three disabled, so a failure here pins the cause on no_key_compare_json or void_erase_json, and a pass pins it on the array or binary file. Still temporary. Signed-off-by: Niels Lohmann * Split the two object types apart AppVeyor failed with only unit-custom-object-type.cpp enabled and passed with all three new translation units disabled, so the cause is one of the two object types in this file and not the array or binary ones. Guard out void_erase_map and leave no_key_compare_map, which separates the two constructs under suspicion: shadowing the inherited key_compare member type with an entity that is not a type, and hiding the inherited erase with a void-returning overload. A failure here points at the first, a pass at the second. Still temporary. Signed-off-by: Niels Lohmann * Build the no-key_compare object type by composition, not inheritance The "object type without key_compare" test failed on AppVeyor's MSVC 2017 jobs (/std:c++17): its no_key_compare_map derived publicly from std::map and shadowed the inherited key_compare type with a same-named member function, relying on ordinary member hiding to make key_compare unreachable as a type for the library's detection trait. MSVC 2017 does not honor that hiding for a typename-qualified lookup performed from outside the class and still resolves key_compare to the base's comparator type, so object_comparator_t incorrectly picked it up instead of falling back to default_object_comparator_t. Wrapping a std::map by composition instead removes the base class entirely, so there is no key_compare to find under any lookup rule, on any compiler. Also drops the now-unneeded JSON_BISECT_CUSTOM_CONTAINER_TESTS guard left over from narrowing this down: the void_erase_map test in the same file was never the cause and is re-enabled unconditionally. Verified locally with clang++ and g++ under C++17 and C++20. Signed-off-by: Niels Lohmann * Re-enable the array and binary custom-container tests unit-custom-array-type.cpp and unit-custom-binary-type.cpp were still guarded behind JSON_BISECT_CUSTOM_CONTAINER_TESTS from bisecting the AppVeyor failure fixed in 7c39f3227, which was unrelated to either file. The macro was never defined, so none of these tests actually ran in CI. Verified locally with clang++ and g++ under C++17 and C++20 before removing the guards. Signed-off-by: Niels Lohmann * Fix indentation of custom_object_type per astyle The one-line function bodies in the composition-based no_key_compare_map (7c39f3227) do not match the project's Allman brace style, which the ci_test_amalgamation job enforces with astyle. Reformatted with the pinned astyle 3.4.13; no functional change. Signed-off-by: Niels Lohmann * docs: add compiled reference implementations for the container template parameters Each of ObjectType, ArrayType, StringType, and BinaryType now links to a minimal, self-contained header (docs/mkdocs/docs/examples/custom_*_type.hpp) that wraps the corresponding standard container by composition and satisfies every "Always required" member listed on that page. Unlike the prose requirement lists, these are real code: each header has a companion .cpp that instantiates a basic_json specialization with it and is compiled and run by the existing ci_test_examples check (docs/Makefile's check_output_portable), so the reference implementations cannot silently drift from what the library actually requires. The .output files were generated with that same target. StringType's existing pointer to tests/src/unit-alt-string.cpp's alt_string is kept alongside the new header as a more thorough, battle-tested example. Verified locally: astyle (pinned 3.4.13, project .astylerc) on the new files; clang++/g++ under C++11/17/20 for each example against the amalgamated header; `make check_output_portable` in docs/; `mkdocs build --strict` and scripts/check_structure.py for the page itself. Signed-off-by: Niels Lohmann * Declare no_key_compare_map's accessors noexcept The GCC C++20 job builds with -Wnoexcept and -Werror, and the standard library takes noexcept(c.begin()) and noexcept(c.end()) in ranges_base.h and range_access.h. Forwarding to std::map without repeating its noexcept made those expressions false, which the warning reports as an error: error: noexcept-expression evaluates to 'false' because of a call to no_key_compare_map<...>::begin() [-Werror=noexcept] note: but ... does not throw; perhaps it should be declared 'noexcept' Give the accessors the exception specification of what they forward to. std::map declares begin, end, cbegin, cend, empty, size, max_size, and clear noexcept, so the wrapper does too. swap is left alone: std::map's is only conditionally noexcept, and nothing asks for it. void_erase_map is unaffected because it still derives from std::map and inherits accessors that already carry the specification. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * Write out what the defaulted constructor of no_key_compare_map implied ci_test_gcc builds with -Weffc++, which asks for data to be initialized in a member initialization list; a defaulted default constructor does not do that: error: 'no_key_compare_map<...>::data' should be initialized in the member initialization list [-Werror=effc++] Writing the constructor out satisfies that but drops the exception specification the defaulted one carried, which -Wnoexcept then objects to where the standard library takes noexcept(construct(...)). Declare it the way the defaulted constructor was: noexcept when the wrapped map's default constructor is. This is the cost of composition -- inheritance carried std::map's exception specifications and initialization for free, and forwarding by hand has to restate them. Checked with the repository's own GCC warning set from cmake/gcc_flags.cmake, all 346 flags, at C++11, C++17 and C++20: no diagnostics for this file, nor for the two custom container translation units that were disabled while the MSVC failure was narrowed down and are built again now. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann * Mark no_key_compare_map::swap noexcept Clang-Tidy rejects a swap that is not: error: swap functions should be marked noexcept [cppcoreguidelines-noexcept-swap,performance-noexcept-swap] It was left unmarked on the grounds that std::map::swap is only conditionally noexcept, so an unconditional promise would be wrong for a comparator or allocator that can throw while swapping. Both concerns are met by taking the specification from the wrapped map rather than asserting one: noexcept(noexcept(data.swap(other.data))). Clang-Tidy accepts that, and no NOLINT is needed. Last in the series of specifications that inheritance used to supply and composition has to write out by hand. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E Signed-off-by: Niels Lohmann --------- Signed-off-by: Niels Lohmann Co-authored-by: Claude Opus 5 --- docs/mkdocs/docs/api/basic_json/array_t.md | 7 +- docs/mkdocs/docs/api/basic_json/binary_t.md | 12 +- docs/mkdocs/docs/api/basic_json/boolean_t.md | 8 + docs/mkdocs/docs/api/basic_json/index.md | 4 + .../docs/api/basic_json/json_base_class_t.md | 5 +- .../docs/api/basic_json/json_serializer.md | 6 + .../docs/api/basic_json/number_float_t.md | 10 + .../docs/api/basic_json/number_integer_t.md | 7 + .../docs/api/basic_json/number_unsigned_t.md | 8 + .../api/basic_json/object_comparator_t.md | 2 + docs/mkdocs/docs/api/basic_json/object_t.md | 7 +- docs/mkdocs/docs/api/basic_json/string_t.md | 7 + docs/mkdocs/docs/api/basic_json/unflatten.md | 10 +- .../mkdocs/docs/api/macros/json_throw_user.md | 8 +- .../docs/examples/custom_array_type.cpp | 19 + .../docs/examples/custom_array_type.hpp | 152 ++++ .../docs/examples/custom_array_type.output | 2 + .../docs/examples/custom_binary_type.cpp | 21 + .../docs/examples/custom_binary_type.hpp | 112 +++ .../docs/examples/custom_binary_type.output | 2 + .../docs/examples/custom_object_type.cpp | 26 + .../docs/examples/custom_object_type.hpp | 144 ++++ .../docs/examples/custom_object_type.output | 11 + .../docs/examples/custom_string_type.cpp | 20 + .../docs/examples/custom_string_type.hpp | 134 ++++ .../docs/examples/custom_string_type.output | 10 + docs/mkdocs/docs/features/object_order.md | 6 +- docs/mkdocs/docs/features/types/index.md | 7 +- .../features/types/template_parameters.md | 747 ++++++++++++++++++ docs/mkdocs/mkdocs.yml | 1 + include/nlohmann/detail/exceptions.hpp | 5 +- include/nlohmann/detail/hash.hpp | 4 +- .../nlohmann/detail/input/binary_reader.hpp | 5 +- .../nlohmann/detail/iterators/iter_impl.hpp | 9 +- include/nlohmann/detail/json_pointer.hpp | 55 +- include/nlohmann/detail/meta/type_traits.hpp | 29 +- .../nlohmann/detail/output/binary_writer.hpp | 38 +- include/nlohmann/detail/output/serializer.hpp | 26 +- include/nlohmann/detail/string_escape.hpp | 99 ++- include/nlohmann/json.hpp | 151 ++-- single_include/nlohmann/json.hpp | 421 +++++++--- tests/src/unit-alt-string.cpp | 72 +- tests/src/unit-custom-array-type.cpp | 150 ++++ tests/src/unit-custom-binary-type.cpp | 79 ++ tests/src/unit-custom-object-type.cpp | 323 ++++++++ tests/src/unit-json_pointer.cpp | 10 + 46 files changed, 2721 insertions(+), 270 deletions(-) create mode 100644 docs/mkdocs/docs/examples/custom_array_type.cpp create mode 100644 docs/mkdocs/docs/examples/custom_array_type.hpp create mode 100644 docs/mkdocs/docs/examples/custom_array_type.output create mode 100644 docs/mkdocs/docs/examples/custom_binary_type.cpp create mode 100644 docs/mkdocs/docs/examples/custom_binary_type.hpp create mode 100644 docs/mkdocs/docs/examples/custom_binary_type.output create mode 100644 docs/mkdocs/docs/examples/custom_object_type.cpp create mode 100644 docs/mkdocs/docs/examples/custom_object_type.hpp create mode 100644 docs/mkdocs/docs/examples/custom_object_type.output create mode 100644 docs/mkdocs/docs/examples/custom_string_type.cpp create mode 100644 docs/mkdocs/docs/examples/custom_string_type.hpp create mode 100644 docs/mkdocs/docs/examples/custom_string_type.output create mode 100644 docs/mkdocs/docs/features/types/template_parameters.md create mode 100644 tests/src/unit-custom-array-type.cpp create mode 100644 tests/src/unit-custom-binary-type.cpp create mode 100644 tests/src/unit-custom-object-type.cpp diff --git a/docs/mkdocs/docs/api/basic_json/array_t.md b/docs/mkdocs/docs/api/basic_json/array_t.md index dd2b901d5..b6e41a6ad 100644 --- a/docs/mkdocs/docs/api/basic_json/array_t.md +++ b/docs/mkdocs/docs/api/basic_json/array_t.md @@ -14,7 +14,11 @@ To store objects in C++, a type is defined by the template parameters explained ## Template parameters `ArrayType` -: container type to store arrays (e.g., `std::vector` or `std::list`) +: container type to store arrays. It must be a vector-like container: the library uses `operator[]`, `at()`, and + `resize()`, and requires random-access iterators. `#!cpp std::vector` and `#!cpp std::deque` qualify; + `#!cpp std::list` does not. See + [Template Parameter Requirements](../../features/types/template_parameters.md#arraytype) for the full list of + requirements. `AllocatorType` : the allocator to use for objects (e.g., `std::allocator`) @@ -66,3 +70,4 @@ Arrays are stored as pointers in a `basic_json` type. That is, for any access to ## Version history - Added in version 1.0.0. +- Made `capacity()` optional, so that array types such as `#!cpp std::deque` can be used, in version 3.13.0. diff --git a/docs/mkdocs/docs/api/basic_json/binary_t.md b/docs/mkdocs/docs/api/basic_json/binary_t.md index 64506b25f..36600748a 100644 --- a/docs/mkdocs/docs/api/basic_json/binary_t.md +++ b/docs/mkdocs/docs/api/basic_json/binary_t.md @@ -42,7 +42,9 @@ represent a byte array in modern C++. `value_type` must additionally be exactly one byte wide (e.g., `std::uint8_t`/`char`/`std::byte`): the binary serializers (CBOR, MessagePack, BSON, UBJSON) read and write the container's raw bytes via `reinterpret_cast`, which is only correct for byte-sized elements -- a container like - `#!cpp std::vector` will not work as `BinaryType`. + `#!cpp std::vector` will not work as `BinaryType`. The elements must be stored contiguously, and + the binary readers additionally require `resize()` and `operator[]`. See + [Template Parameter Requirements](../../features/types/template_parameters.md#binarytype) for the full list. ## Notes @@ -50,6 +52,11 @@ represent a byte array in modern C++. The default values for `BinaryType` is `#!cpp std::vector`. +#### Supported byte types + +`#!cpp std::vector`, `#!cpp std::vector`, and `#!cpp std::vector` are supported. +Regardless of which of them is configured, [`dump`](dump.md) writes the bytes as the numbers 0..255. + #### Custom BinaryType behavior When a custom `BinaryType` is configured (other than the default `#!cpp std::vector`), you can assign @@ -126,3 +133,6 @@ type `#!cpp binary_t*` must be dereferenced. ## Version history - Added in version 3.8.0. Changed the type of subtype to `std::uint64_t` in version 3.10.0. +- Fixed [`dump`](dump.md), [`std::hash`](std_hash.md), and [`to_ubjson`](to_ubjson.md) for byte types that are not + integers (e.g., `#!cpp std::byte`) in version 3.13.0. `dump` now writes the bytes of a signed byte type (e.g., + `#!cpp char`) as 0..255 rather than as negative numbers. diff --git a/docs/mkdocs/docs/api/basic_json/boolean_t.md b/docs/mkdocs/docs/api/basic_json/boolean_t.md index c30afefdc..bfb8c3426 100644 --- a/docs/mkdocs/docs/api/basic_json/boolean_t.md +++ b/docs/mkdocs/docs/api/basic_json/boolean_t.md @@ -11,6 +11,14 @@ literals `#!json true` and `#!json false`. To store boolean values in C++, a type is defined by the template parameter `BooleanType` which chooses the type to use. +## Template parameters + +`BooleanType` +: the type to store booleans. As it is stored directly inside a `basic_json` value (in a union), it must be a + trivially default-constructible, trivially copyable, and trivially destructible type that is convertible to and + from `#!cpp bool`. See + [Template Parameter Requirements](../../features/types/template_parameters.md#booleantype). + ## Notes #### Default type diff --git a/docs/mkdocs/docs/api/basic_json/index.md b/docs/mkdocs/docs/api/basic_json/index.md index bd33f31ac..c5556ea3b 100644 --- a/docs/mkdocs/docs/api/basic_json/index.md +++ b/docs/mkdocs/docs/api/basic_json/index.md @@ -35,6 +35,10 @@ class basic_json; | `BinaryType` | type for binary arrays | [`binary_t`](binary_t.md) | | `CustomBaseClass` | extension point for user code | [`json_base_class_t`](json_base_class_t.md) | +The library imposes a number of requirements on these types that are not expressed as C++ concepts, such as the +container operations `object_t` and `array_t` must provide, or the fact that `StringType` must be `char`-based. They +are collected in [Template Parameter Requirements](../../features/types/template_parameters.md). + ## Specializations - [**json**](../json.md) - default specialization diff --git a/docs/mkdocs/docs/api/basic_json/json_base_class_t.md b/docs/mkdocs/docs/api/basic_json/json_base_class_t.md index dfe5f1cb7..7add54098 100644 --- a/docs/mkdocs/docs/api/basic_json/json_base_class_t.md +++ b/docs/mkdocs/docs/api/basic_json/json_base_class_t.md @@ -21,8 +21,11 @@ The default value for `CustomBaseClass` is `void`. In this case, an #### Limitations -The type `CustomBaseClass` has to be a default-constructible class. +The type `CustomBaseClass` has to be a default-constructible, non-`final` class. `basic_json` only supports copy/move construction/assignment if `CustomBaseClass` does so as well. +A `CustomBaseClass` with non-static data members forfeits `basic_json`'s +[standard layout](https://en.cppreference.com/w/cpp/named_req/StandardLayoutType) guarantee. See +[Template Parameter Requirements](../../features/types/template_parameters.md#custombaseclass). ## Examples diff --git a/docs/mkdocs/docs/api/basic_json/json_serializer.md b/docs/mkdocs/docs/api/basic_json/json_serializer.md index 24a37735c..b92cf3d17 100644 --- a/docs/mkdocs/docs/api/basic_json/json_serializer.md +++ b/docs/mkdocs/docs/api/basic_json/json_serializer.md @@ -19,6 +19,12 @@ using json_serializer = JSONSerializer; The default values for `json_serializer` is [`adl_serializer`](../adl_serializer/index.md). +#### Requirements + +A custom serializer must provide `#!cpp static void to_json(basic_json&, T)` for every type it serializes, and either +`#!cpp static void from_json(const basic_json&, T&)` or `#!cpp static T from_json(const basic_json&)` for every type it +deserializes. See [Template Parameter Requirements](../../features/types/template_parameters.md#jsonserializer). + ## Examples ??? example diff --git a/docs/mkdocs/docs/api/basic_json/number_float_t.md b/docs/mkdocs/docs/api/basic_json/number_float_t.md index 3e8933da6..83c7011c5 100644 --- a/docs/mkdocs/docs/api/basic_json/number_float_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_float_t.md @@ -20,6 +20,16 @@ used. To store floating-point numbers in C++, a type is defined by the template parameter `NumberFloatType` which chooses the type to use. +## Template parameters + +`NumberFloatType` +: the type to store floating-point numbers. Parsing and serialization are implemented in terms of + `#!cpp std::strtof`/`#!cpp std::strtod`/`#!cpp std::strtold` and `#!cpp std::snprintf`, so the type must be + `#!cpp float`, `#!cpp double`, or `#!cpp long double`. The + [binary formats](../../features/binary_formats/index.md) additionally require `#!cpp float` or `#!cpp double`, + because they have no encoding for `#!cpp long double`. See + [Template Parameter Requirements](../../features/types/template_parameters.md#numberfloattype). + ## Notes #### Default type diff --git a/docs/mkdocs/docs/api/basic_json/number_integer_t.md b/docs/mkdocs/docs/api/basic_json/number_integer_t.md index 79cbdf8ca..9a2ffab7f 100644 --- a/docs/mkdocs/docs/api/basic_json/number_integer_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_integer_t.md @@ -20,6 +20,13 @@ used. To store integer numbers in C++, a type is defined by the template parameter `NumberIntegerType` which chooses the type to use. +## Template parameters + +`NumberIntegerType` +: the type to store signed integers. It must be a **signed integral** type (`#!cpp std::is_integral`) with a + `#!cpp std::numeric_limits` specialization, and it is stored directly inside a `basic_json` value. See + [Template Parameter Requirements](../../features/types/template_parameters.md#numberintegertype-and-numberunsignedtype). + ## Notes #### Default type diff --git a/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md b/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md index f1010f2a6..674f7711d 100644 --- a/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md @@ -20,6 +20,14 @@ used. To store unsigned integer numbers in C++, a type is defined by the template parameter `NumberUnsignedType` which chooses the type to use. +## Template parameters + +`NumberUnsignedType` +: the type to store unsigned integers. It must be an **unsigned integral** type (`#!cpp std::is_integral`) with a + `#!cpp std::numeric_limits` specialization, and it must be able to represent the absolute value of every + [`number_integer_t`](number_integer_t.md) value. See + [Template Parameter Requirements](../../features/types/template_parameters.md#numberintegertype-and-numberunsignedtype). + ## Notes #### Default type diff --git a/docs/mkdocs/docs/api/basic_json/object_comparator_t.md b/docs/mkdocs/docs/api/basic_json/object_comparator_t.md index d41b98229..bbda0a0e7 100644 --- a/docs/mkdocs/docs/api/basic_json/object_comparator_t.md +++ b/docs/mkdocs/docs/api/basic_json/object_comparator_t.md @@ -30,3 +30,5 @@ and [`default_object_comparator_t`](default_object_comparator_t.md) otherwise. - Added in version 3.0.0. - Changed to be conditionally defined as `#!cpp typename object_t::key_compare` or `default_object_comparator_t` in version 3.11.0. +- Fixed the fallback to `default_object_comparator_t`, which previously failed to compile for object types without a + `key_compare` member type, in version 3.13.0. diff --git a/docs/mkdocs/docs/api/basic_json/object_t.md b/docs/mkdocs/docs/api/basic_json/object_t.md index de41b86e4..6ce393a1d 100644 --- a/docs/mkdocs/docs/api/basic_json/object_t.md +++ b/docs/mkdocs/docs/api/basic_json/object_t.md @@ -18,7 +18,11 @@ To store objects in C++, a type is defined by the template parameters described ## Template parameters `ObjectType` -: the container to store objects (e.g., `std::map` or `std::unordered_map`) +: the container to store objects. Its template parameters must have the same order and meaning as those of + `std::map`; in particular, the third parameter is a comparator. `#!cpp std::unordered_map`, whose third parameter + is a hash function, therefore needs an adapter -- see + [Template Parameter Requirements](../../features/types/template_parameters.md#objecttype) for the full list of + requirements, an adapter example, and the containers that are known to work. `StringType` : the type of the keys or names (e.g., `std::string`). The comparison function `std::less` is used to @@ -122,3 +126,4 @@ the object is silently converted as an array of key-value pairs, which is incorr ## Version history - Added in version 1.0.0. +- Allowed object types whose `erase(iterator)` returns `#!cpp void` in version 3.13.0. diff --git a/docs/mkdocs/docs/api/basic_json/string_t.md b/docs/mkdocs/docs/api/basic_json/string_t.md index 97c586f28..e8be0fe0e 100644 --- a/docs/mkdocs/docs/api/basic_json/string_t.md +++ b/docs/mkdocs/docs/api/basic_json/string_t.md @@ -23,6 +23,11 @@ JSON class into byte-sized characters during deserialization. `StringType`. To work with wide-character data, convert it to/from UTF-8 at the boundary instead -- see the FAQ's [wide string handling](../../home/faq.md#wide-string-handling) section for a conversion recipe. + Beyond the character type, the library expects a substantial part of the `#!cpp std::string` interface (contiguous + null-terminated `data()`, `substr()`, `find()`, `append()`, ...). See + [Template Parameter Requirements](../../features/types/template_parameters.md#stringtype) for the full list and + for the string types that are known to work. + ## Notes #### Default type @@ -78,3 +83,5 @@ and an example. ## Version history - Added in version 1.0.0. +- Removed the requirement that `string_t` be implicitly convertible from `#!cpp std::string`, which the BSON writer and + the UBJSON reader relied on, in version 3.13.0. diff --git a/docs/mkdocs/docs/api/basic_json/unflatten.md b/docs/mkdocs/docs/api/basic_json/unflatten.md index 1af243b58..ac88cd73e 100644 --- a/docs/mkdocs/docs/api/basic_json/unflatten.md +++ b/docs/mkdocs/docs/api/basic_json/unflatten.md @@ -37,7 +37,14 @@ Linear in the size of the JSON value. ## Notes Empty objects and arrays are flattened by [`flatten()`](flatten.md) to `#!json null` values and cannot unflattened to -their original type. Apart from this example, for a JSON value `j`, the following is always true: +their original type. + +A flattened array and a flattened object whose keys are array indices are indistinguishable, because both are +described by the same JSON pointers. A value is therefore restored as an array if and only if one of its keys is the +reference token `0`, and as an object otherwise: `#!json {"2": 1}` is restored unchanged, whereas `#!json {"0": 1}` is +restored as `#!json [1]`. This decision does not depend on the order in which the flattened object is iterated. + +Apart from these two cases, for a JSON value `j`, the following is always true: `#!cpp j == j.flatten().unflatten()`. ## Examples @@ -63,3 +70,4 @@ their original type. Apart from this example, for a JSON value `j`, the followin ## Version history - Added in version 2.0.0. +- Made the array/object decision independent of the object's iteration order in version 3.13.0. diff --git a/docs/mkdocs/docs/api/macros/json_throw_user.md b/docs/mkdocs/docs/api/macros/json_throw_user.md index b02918cf8..7f4fb6a8d 100644 --- a/docs/mkdocs/docs/api/macros/json_throw_user.md +++ b/docs/mkdocs/docs/api/macros/json_throw_user.md @@ -12,9 +12,11 @@ Controls how exceptions are handled by the library. 1. This macro overrides [`#!cpp catch`](https://en.cppreference.com/w/cpp/language/try_catch) calls inside the library. - The argument is the type of the exception to catch. As of version 3.8.0, the library only catches `std::out_of_range` - exceptions internally to rethrow them as [`json::out_of_range`](../../home/exceptions.md#out-of-range) exceptions. - The macro is always followed by a scope. + The argument is the type of the exception to catch. The library uses it in a single place: to swallow any exception + escaping the parent-pointer check that [`JSON_DIAGNOSTICS`](json_diagnostics.md) adds to the class invariant. The + places where the library catches its own [`json::out_of_range`](../../home/exceptions.md#out-of-range) exceptions + use `JSON_INTERNAL_CATCH` instead, which `JSON_CATCH_USER` also overrides unless `JSON_INTERNAL_CATCH_USER` is + defined. The macro is always followed by a scope. 2. This macro overrides `#!cpp throw` calls inside the library. The argument is the exception to be thrown. Note that `JSON_THROW_USER` should leave the current scope (e.g., by throwing or aborting), as continuing after it may yield undefined behavior. diff --git a/docs/mkdocs/docs/examples/custom_array_type.cpp b/docs/mkdocs/docs/examples/custom_array_type.cpp new file mode 100644 index 000000000..63651cadf --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_array_type.cpp @@ -0,0 +1,19 @@ +#include +#include + +#include + +#include "custom_array_type.hpp" + +using custom_json = nlohmann::basic_json; + +int main() +{ + custom_json j = custom_json::array(); + j.push_back(1); + j.push_back(2); + j.push_back(3); + + std::cout << j.dump() << std::endl; + std::cout << std::boolalpha << (custom_json::parse(j.dump()) == j) << std::endl; +} diff --git a/docs/mkdocs/docs/examples/custom_array_type.hpp b/docs/mkdocs/docs/examples/custom_array_type.hpp new file mode 100644 index 000000000..750d52a81 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_array_type.hpp @@ -0,0 +1,152 @@ +#pragma once + +#include +#include +#include + +// A minimal, self-contained ArrayType built around a private std::vector. +// See https://json.nlohmann.me/features/types/template_parameters/#arraytype +template> +class custom_array_type +{ + using vector_t = std::vector; + vector_t data_; + + public: + using value_type = typename vector_t::value_type; + using size_type = typename vector_t::size_type; + using iterator = typename vector_t::iterator; + using const_iterator = typename vector_t::const_iterator; + + custom_array_type() = default; + custom_array_type(const custom_array_type&) = default; + custom_array_type(custom_array_type&&) = default; + custom_array_type& operator=(const custom_array_type&) = default; + custom_array_type& operator=(custom_array_type&&) = default; + + template + custom_array_type(InputIt first, InputIt last) : data_(first, last) {} + + custom_array_type(size_type count, const T& value) : data_(count, value) {} + + iterator begin() + { + return data_.begin(); + } + iterator end() + { + return data_.end(); + } + const_iterator begin() const + { + return data_.begin(); + } + const_iterator end() const + { + return data_.end(); + } + const_iterator cbegin() const + { + return data_.cbegin(); + } + const_iterator cend() const + { + return data_.cend(); + } + + bool empty() const + { + return data_.empty(); + } + size_type size() const + { + return data_.size(); + } + size_type max_size() const + { + return data_.max_size(); + } + void clear() + { + data_.clear(); + } + void resize(size_type n) + { + data_.resize(n); + } + + T& operator[](size_type pos) + { + return data_[pos]; + } + const T& operator[](size_type pos) const + { + return data_[pos]; + } + + T& back() + { + return data_.back(); + } + const T& back() const + { + return data_.back(); + } + + void push_back(const T& value) + { + data_.push_back(value); + } + void push_back(T&& value) + { + data_.push_back(std::move(value)); + } + + template + void emplace_back(Args&& ... args) + { + data_.emplace_back(std::forward(args)...); + } + + void pop_back() + { + data_.pop_back(); + } + + iterator insert(const_iterator pos, const T& value) + { + return data_.insert(pos, value); + } + iterator insert(const_iterator pos, size_type count, const T& value) + { + return data_.insert(pos, count, value); + } + template + iterator insert(const_iterator pos, InputIt first, InputIt last) + { + return data_.insert(pos, first, last); + } + + iterator erase(const_iterator pos) + { + return data_.erase(pos); + } + iterator erase(const_iterator first, const_iterator last) + { + return data_.erase(first, last); + } + + void swap(custom_array_type& other) + { + data_.swap(other.data_); + } + + friend bool operator==(const custom_array_type& lhs, const custom_array_type& rhs) + { + return lhs.data_ == rhs.data_; + } + friend bool operator<(const custom_array_type& lhs, const custom_array_type& rhs) + { + return lhs.data_ < rhs.data_; + } +}; diff --git a/docs/mkdocs/docs/examples/custom_array_type.output b/docs/mkdocs/docs/examples/custom_array_type.output new file mode 100644 index 000000000..f5ceaa555 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_array_type.output @@ -0,0 +1,2 @@ +[1,2,3] +true diff --git a/docs/mkdocs/docs/examples/custom_binary_type.cpp b/docs/mkdocs/docs/examples/custom_binary_type.cpp new file mode 100644 index 000000000..cea34ea38 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_binary_type.cpp @@ -0,0 +1,21 @@ +#include +#include +#include +#include +#include + +#include + +#include "custom_binary_type.hpp" + +using custom_json = nlohmann::basic_json; + +int main() +{ + const auto j = custom_json::binary({0x01, 0x02, 0x03}); + + std::cout << j.dump() << std::endl; + std::cout << std::boolalpha << (custom_json::from_cbor(custom_json::to_cbor(j)) == j) << std::endl; +} diff --git a/docs/mkdocs/docs/examples/custom_binary_type.hpp b/docs/mkdocs/docs/examples/custom_binary_type.hpp new file mode 100644 index 000000000..77b4ad49a --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_binary_type.hpp @@ -0,0 +1,112 @@ +#pragma once + +#include +#include +#include + +// A minimal, self-contained BinaryType built around a private std::vector. +// See https://json.nlohmann.me/features/types/template_parameters/#binarytype +class custom_binary_type +{ + using vector_t = std::vector; + vector_t data_; + + public: + using value_type = vector_t::value_type; + using size_type = vector_t::size_type; + using iterator = vector_t::iterator; + using const_iterator = vector_t::const_iterator; + + custom_binary_type() = default; + custom_binary_type(const custom_binary_type&) = default; + custom_binary_type(custom_binary_type&&) = default; + custom_binary_type& operator=(const custom_binary_type&) = default; + custom_binary_type& operator=(custom_binary_type&&) = default; + + template + custom_binary_type(InputIt first, InputIt last) : data_(first, last) {} + + // so basic_json::binary({0x01, 0x02}) can build one directly + custom_binary_type(std::initializer_list init) : data_(init) {} + + size_type size() const + { + return data_.size(); + } + bool empty() const + { + return data_.empty(); + } + void clear() + { + data_.clear(); + } + void resize(size_type n) + { + data_.resize(n); + } + + // read-only is enough: the writers only ever read from a binary value + const std::uint8_t* data() const + { + return data_.data(); + } + + std::uint8_t& operator[](size_type pos) + { + return data_[pos]; + } + std::uint8_t operator[](size_type pos) const + { + return data_[pos]; + } + + std::uint8_t& back() + { + return data_.back(); + } + std::uint8_t back() const + { + return data_.back(); + } + + iterator begin() + { + return data_.begin(); + } + iterator end() + { + return data_.end(); + } + const_iterator begin() const + { + return data_.begin(); + } + const_iterator end() const + { + return data_.end(); + } + const_iterator cbegin() const + { + return data_.cbegin(); + } + const_iterator cend() const + { + return data_.cend(); + } + + template + iterator insert(const_iterator pos, InputIt first, InputIt last) + { + return data_.insert(pos, first, last); + } + + friend bool operator==(const custom_binary_type& lhs, const custom_binary_type& rhs) + { + return lhs.data_ == rhs.data_; + } + friend bool operator<(const custom_binary_type& lhs, const custom_binary_type& rhs) + { + return lhs.data_ < rhs.data_; + } +}; diff --git a/docs/mkdocs/docs/examples/custom_binary_type.output b/docs/mkdocs/docs/examples/custom_binary_type.output new file mode 100644 index 000000000..b4814d6ed --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_binary_type.output @@ -0,0 +1,2 @@ +{"bytes":[1,2,3],"subtype":null} +true diff --git a/docs/mkdocs/docs/examples/custom_object_type.cpp b/docs/mkdocs/docs/examples/custom_object_type.cpp new file mode 100644 index 000000000..d4f99ccf0 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_object_type.cpp @@ -0,0 +1,26 @@ +#include +#include +#include + +#include + +#include "custom_object_type.hpp" + +using custom_json = nlohmann::basic_json; + +int main() +{ + custom_json j; + j["pi"] = 3.141; + j["happy"] = true; + j["list"] = {1, 2, 3}; + + std::cout << j.dump(2) << std::endl; + std::cout << std::boolalpha << (custom_json::parse(j.dump()) == j) << std::endl; + + // custom_object_type has no key_compare member, so object_comparator_t + // falls back to its default + std::cout << std::boolalpha + << std::is_same::value + << std::endl; +} diff --git a/docs/mkdocs/docs/examples/custom_object_type.hpp b/docs/mkdocs/docs/examples/custom_object_type.hpp new file mode 100644 index 000000000..da715ed4e --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_object_type.hpp @@ -0,0 +1,144 @@ +#pragma once + +#include +#include + +// A minimal, self-contained ObjectType built around a private std::map. +// key_compare is deliberately not exposed: when an ObjectType has no +// key_compare member, the library falls back to its own default comparator. +// See https://json.nlohmann.me/features/types/template_parameters/#objecttype +template +class custom_object_type +{ + using map_t = std::map; + map_t data_; + + public: + using key_type = typename map_t::key_type; + using mapped_type = typename map_t::mapped_type; + using value_type = typename map_t::value_type; + using size_type = typename map_t::size_type; + using iterator = typename map_t::iterator; + using const_iterator = typename map_t::const_iterator; + + custom_object_type() = default; + custom_object_type(const custom_object_type&) = default; + custom_object_type(custom_object_type&&) = default; + custom_object_type& operator=(const custom_object_type&) = default; + custom_object_type& operator=(custom_object_type&&) = default; + + template + custom_object_type(InputIt first, InputIt last) : data_(first, last) {} + + iterator begin() + { + return data_.begin(); + } + iterator end() + { + return data_.end(); + } + const_iterator begin() const + { + return data_.begin(); + } + const_iterator end() const + { + return data_.end(); + } + const_iterator cbegin() const + { + return data_.cbegin(); + } + const_iterator cend() const + { + return data_.cend(); + } + + bool empty() const + { + return data_.empty(); + } + size_type size() const + { + return data_.size(); + } + size_type max_size() const + { + return data_.max_size(); + } + void clear() + { + data_.clear(); + } + + iterator find(const key_type& key) + { + return data_.find(key); + } + const_iterator find(const key_type& key) const + { + return data_.find(key); + } + size_type count(const key_type& key) const + { + return data_.count(key); + } + + std::pair emplace(const key_type& key, const mapped_type& value) + { + return data_.emplace(key, value); + } + + std::pair insert(const value_type& value) + { + return data_.insert(value); + } + + template + void insert(InputIt first, InputIt last) + { + data_.insert(first, last); + } + + mapped_type& operator[](const key_type& key) + { + return data_[key]; + } + + mapped_type& at(const key_type& key) + { + return data_.at(key); + } + const mapped_type& at(const key_type& key) const + { + return data_.at(key); + } + + iterator erase(iterator pos) + { + return data_.erase(pos); + } + iterator erase(iterator first, iterator last) + { + return data_.erase(first, last); + } + size_type erase(const key_type& key) + { + return data_.erase(key); + } + + void swap(custom_object_type& other) + { + data_.swap(other.data_); + } + + friend bool operator==(const custom_object_type& lhs, const custom_object_type& rhs) + { + return lhs.data_ == rhs.data_; + } + friend bool operator<(const custom_object_type& lhs, const custom_object_type& rhs) + { + return lhs.data_ < rhs.data_; + } +}; diff --git a/docs/mkdocs/docs/examples/custom_object_type.output b/docs/mkdocs/docs/examples/custom_object_type.output new file mode 100644 index 000000000..48b3e9630 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_object_type.output @@ -0,0 +1,11 @@ +{ + "happy": true, + "list": [ + 1, + 2, + 3 + ], + "pi": 3.141 +} +true +true diff --git a/docs/mkdocs/docs/examples/custom_string_type.cpp b/docs/mkdocs/docs/examples/custom_string_type.cpp new file mode 100644 index 000000000..63b798fc6 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_string_type.cpp @@ -0,0 +1,20 @@ +#include +#include +#include + +#include + +#include "custom_string_type.hpp" + +using custom_json = nlohmann::basic_json; + +int main() +{ + custom_json j; + j["pi"] = 3.141; + j["happy"] = true; + j["list"] = {1, 2, 3}; + + std::cout << j.dump(2) << std::endl; + std::cout << std::boolalpha << (custom_json::parse(j.dump()) == j) << std::endl; +} diff --git a/docs/mkdocs/docs/examples/custom_string_type.hpp b/docs/mkdocs/docs/examples/custom_string_type.hpp new file mode 100644 index 000000000..ec48501fc --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_string_type.hpp @@ -0,0 +1,134 @@ +#pragma once + +#include +#include + +// A minimal, self-contained StringType built around a private std::string. +// Wraps rather than inherits, so it exposes exactly what the library needs +// and nothing more of std::string's interface. +// +// Covers the "Always required" members, the extras needed for the binary +// formats, and the extras needed for JSON Pointer / flatten / unflatten / +// diff. Extending it further (e.g. for std::hash or to_bson) is +// a matter of adding the extra members listed in the "Required for other +// functionality" table. +// +// See https://json.nlohmann.me/features/types/template_parameters/#stringtype +class custom_string_type +{ + std::string data_; + + public: + using value_type = char; + using size_type = std::string::size_type; + using iterator = std::string::iterator; + using const_iterator = std::string::const_iterator; + + static constexpr size_type npos = std::string::npos; + + custom_string_type() = default; + custom_string_type(const custom_string_type&) = default; + custom_string_type(custom_string_type&&) = default; + custom_string_type& operator=(const custom_string_type&) = default; + custom_string_type& operator=(custom_string_type&&) = default; + + // not explicit: the library relies on being able to hand it a string literal + custom_string_type(const char* s) : data_(s) {} + custom_string_type(const char* s, size_type count) : data_(s, count) {} + custom_string_type(size_type count, char ch) : data_(count, ch) {} + + size_type size() const + { + return data_.size(); + } + bool empty() const + { + return data_.empty(); + } + void clear() + { + data_.clear(); + } + void resize(size_type n) + { + data_.resize(n); + } + void resize(size_type n, char c) + { + data_.resize(n, c); + } + void reserve(size_type n) + { + data_.reserve(n); + } + + // must stay null-terminated -- the parser hands this to std::strtoull & + // friends; std::string::data() has guaranteed that since C++11 + const char* data() const + { + return data_.data(); + } + + void push_back(char c) + { + data_.push_back(c); + } + + char& operator[](size_type pos) + { + return data_[pos]; + } + char operator[](size_type pos) const + { + return data_[pos]; + } + + custom_string_type& append(const char* s, size_type count) + { + data_.append(s, count); + return *this; + } + custom_string_type& append(const custom_string_type& other) + { + data_.append(other.data_); + return *this; + } + + size_type find_first_of(char c, size_type pos = 0) const + { + return data_.find_first_of(c, pos); + } + + iterator begin() + { + return data_.begin(); + } + iterator end() + { + return data_.end(); + } + const_iterator begin() const + { + return data_.begin(); + } + const_iterator end() const + { + return data_.end(); + } + + friend bool operator==(const custom_string_type& lhs, const custom_string_type& rhs) + { + return lhs.data_ == rhs.data_; + } + friend bool operator<(const custom_string_type& lhs, const custom_string_type& rhs) + { + return lhs.data_ < rhs.data_; + } + + // not required by the library itself, but dump() returns a custom_string_type + // and this makes `std::cout << j.dump()` work as expected + friend std::ostream& operator<<(std::ostream& os, const custom_string_type& s) + { + return os << s.data_; + } +}; diff --git a/docs/mkdocs/docs/examples/custom_string_type.output b/docs/mkdocs/docs/examples/custom_string_type.output new file mode 100644 index 000000000..d792e2a72 --- /dev/null +++ b/docs/mkdocs/docs/examples/custom_string_type.output @@ -0,0 +1,10 @@ +{ + "happy": true, + "list": [ + 1, + 2, + 3 + ], + "pi": 3.141 +} +true diff --git a/docs/mkdocs/docs/features/object_order.md b/docs/mkdocs/docs/features/object_order.md index f62474efd..200913fd2 100644 --- a/docs/mkdocs/docs/features/object_order.md +++ b/docs/mkdocs/docs/features/object_order.md @@ -51,7 +51,11 @@ If you do want to preserve the **insertion order**, you can use the type [`nlohm --8<-- "examples/ordered_json.output" ``` -Alternatively, you can use a more sophisticated ordered map like [`tsl::ordered_map`](https://github.com/Tessil/ordered-map) ([integration](https://github.com/nlohmann/json/issues/546#issuecomment-304447518)) or [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)). +Alternatively, [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) also preserves the insertion order and, unlike [`ordered_map`](../api/ordered_map.md), keeps a lookup index, so it does not have the quadratic cost described below. It is used through a small adapter ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)). + +If the order does not matter and you only want faster lookup, `boost::unordered_flat_map`, `absl::flat_hash_map`, `absl::node_hash_map`, and several other hash maps work through an adapter that restores the template argument order `basic_json` expects; see [Template Parameter Requirements](types/template_parameters.md#objecttype). Note these are *unordered*, not insertion-ordered. + +[`tsl::ordered_map`](https://github.com/Tessil/ordered-map) cannot be used: its iterators expose the mapped value as `const`, while `basic_json` needs to modify it in place. The [`ordered_map`](../api/ordered_map.md) behind `nlohmann::ordered_json` is deliberately minimal and has no lookup index, so every key access is a linear scan and building an object of `n` keys costs O(n²). This is unnoticeable at diff --git a/docs/mkdocs/docs/features/types/index.md b/docs/mkdocs/docs/features/types/index.md index 5990ac708..e6078b825 100644 --- a/docs/mkdocs/docs/features/types/index.md +++ b/docs/mkdocs/docs/features/types/index.md @@ -79,7 +79,8 @@ template< class NumberFloatType = double, template class AllocatorType = std::allocator, template class JSONSerializer = adl_serializer, - class BinaryType = std::vector + class BinaryType = std::vector, + class CustomBaseClass = void > class basic_json; ``` @@ -106,6 +107,10 @@ using number_float_t = NumberFloatType; using binary_t = nlohmann::byte_container_with_subtype; ``` +Not every type can be passed for these template arguments: the library uses the resulting types in ways that imply a +number of requirements, for instance that `StringType` is `char`-based or that `ArrayType` is vector-like. These +requirements are collected in [Template Parameter Requirements](template_parameters.md). + ## Objects diff --git a/docs/mkdocs/docs/features/types/template_parameters.md b/docs/mkdocs/docs/features/types/template_parameters.md new file mode 100644 index 000000000..760972dff --- /dev/null +++ b/docs/mkdocs/docs/features/types/template_parameters.md @@ -0,0 +1,747 @@ +# Template Parameter Requirements + +Class [`basic_json`](../../api/basic_json/index.md) is configurable through eleven template parameters. The library +never formally states what a type passed for one of these parameters has to provide -- the requirements are implied by +the way the library uses the resulting [`object_t`](../../api/basic_json/object_t.md), +[`array_t`](../../api/basic_json/array_t.md), [`string_t`](../../api/basic_json/string_t.md), etc. This page collects +these requirements so they do not have to be discovered by trial and error. Each section lists the concrete types +that are known to work for that parameter and the ones that do not, checked against Boost 1.83, Abseil 20250127.0, +Folly, EASTL 3.21, `ankerl::unordered_dense`, `phmap`, `gtl`, `robin_hood`, `tsl::ordered_map`, and Qt 6. + +## How to read this page + +Requirements are split into two groups: + +- **Always required** -- needed to instantiate `basic_json` at all, or needed by functions that virtually every program + uses (construction, element access, [`dump`](../../api/basic_json/dump.md)). +- **Required for ...** -- only needed when a particular part of the API is instantiated. Member function templates are + only instantiated when they are used, so a type may be perfectly usable even though it does not satisfy these + requirements, as long as the corresponding functions are never called. + +!!! warning "Requirements are not checked" + + Three requirements are checked with a `#!cpp static_assert`: the array iterator category, the width of + [`BinaryType`](#binarytype)'s `value_type`, and [`NumberUnsignedType`](#numberintegertype-and-numberunsignedtype) + being at least as wide as [`NumberIntegerType`](#numberintegertype-and-numberunsignedtype). The rest are not + diagnosed with dedicated error messages, and violating most of them results in a compiler error somewhere inside + the library. Four violations are not caught at compile time at all: + + - A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers, + because the lexer hands the buffer to `#!cpp std::strtoull`/`#!cpp std::strtoll`/`#!cpp std::strtod`. + - A stateful [`AllocatorType`](#allocatortype) compiles and silently ignores its state: allocation, deallocation, + and [`get_allocator()`](../../api/basic_json/get_allocator.md) each use a different default-constructed instance. + - The two [cross-specialization conversions](#cross-specialization-conversions) below. These abort on an assertion + in a normal build, and only fail silently under `#!cpp NDEBUG`. + +## Overview + +| Template parameter | Default | Notable substitutes | +|-------------------------------------------------------------------|-----------------------------------|-----------------------------------------------------------------------| +| [`ObjectType`](#objecttype) | `std::map` | [`nlohmann::ordered_map`](../../api/ordered_map.md), Abseil hash maps | +| [`ArrayType`](#arraytype) | `std::vector` | `#!cpp std::deque` | +| [`StringType`](#stringtype) | `std::string` | `std::string`-like types over `char` | +| [`BooleanType`](#booleantype) | `bool` | none worth using | +| [`NumberIntegerType`](#numberintegertype-and-numberunsignedtype) | `std::int64_t` | any signed integer type | +| [`NumberUnsignedType`](#numberintegertype-and-numberunsignedtype) | `std::uint64_t` | any unsigned integer type at least as wide as `NumberIntegerType` | +| [`NumberFloatType`](#numberfloattype) | `double` | `float` (`long double`: no binary formats) | +| [`AllocatorType`](#allocatortype) | `std::allocator` | stateless allocators | +| [`JSONSerializer`](#jsonserializer) | `adl_serializer` | serializers with the same interface | +| [`BinaryType`](#binarytype) | `#!cpp std::vector` | `#!cpp std::vector` | +| [`CustomBaseClass`](#custombaseclass) | `void` | any default-constructible class | + +!!! warning "Third-party containers and incomplete types" + + `object_t` is instantiated inside the definition of `basic_json` -- it is probed for a `key_compare` member to + form [`object_comparator_t`](../../api/basic_json/object_comparator_t.md) -- i.e. while `basic_json` is still an + incomplete type. `#!cpp std::map` is required by the standard to support incomplete mapped types; most + third-party maps are not, and inspecting the mapped type at class scope (for instance with + `#!cpp std::is_trivially_move_assignable`) makes them unusable as `ObjectType`, no matter how their template + arguments are adapted. This rules out `absl::btree_map`, `phmap::btree_map`, `gtl::btree_map`, + `robin_hood::unordered_node_map`, `folly::F14FastMap`, and `eastl::hash_map`. + + `array_t` is only *named* in the class definition and is not instantiated until `basic_json` is complete, so an + `ArrayType` that inspects its value type at class scope is generally fine -- `boost::container::small_vector` and + `static_vector` both reject incomplete value types yet work here. `absl::InlinedVector` is the exception: the + `#!cpp std::is_trivially_move_assignable` it evaluates while instantiating itself re-enters the + library's own trait machinery mid-instantiation. + +!!! note "Folly requires C++20" + + Folly's headers use `#!cpp consteval` and `#!cpp std::type_identity`, so any `basic_json` specialization that + names a Folly type has to be compiled as C++20 or later, whatever the rest of the library supports. + +## `ObjectType` + +`ObjectType` is instantiated as + +```cpp +using object_t = ObjectType>>; // allocator_type +``` + +i.e., the template arguments follow the order and meaning of `std::map`. + +### Always required + +- The template must be usable with **four** type arguments in the order shown above. The third argument is a + **comparator**; containers that expect something else in this position (e.g., a hash function) need an alias template + or wrapper -- see [Notes](#notes). +- An optional member type `key_compare`. If it is present it becomes + [`object_comparator_t`](../../api/basic_json/object_comparator_t.md); otherwise + [`default_object_comparator_t`](../../api/basic_json/default_object_comparator_t.md) is used. +- Member types `key_type`, `mapped_type`, `value_type`, and `iterator`. +- `value_type` must behave like `#!cpp std::pair`; the library accesses `.first` and + `.second` on it. +- `iterator` must be default-constructible and satisfy + [LegacyBidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator). The type returned + by `cbegin()`/`cend()` must satisfy the same requirements. +- Constructors: default, copy, move, and from an iterator range `(first, last)`. +- Member functions `begin()`, `end()`, `cbegin()`, `cend()`, `empty()`, `size()`, `max_size()`, `clear()`, + `find(key)`, `count(key)`, `emplace(key, value)`, `insert(value_type)`, `insert(first, last)`, `operator[](key)`, + `erase(iterator)`, and `erase(first, last)`. `erase(iterator)` may return the following iterator or `#!cpp void`; + in the latter case the library computes the successor itself, before erasing. +- `erase(key)` is **optional**: if the container does not provide one, the library falls back to `find(key)` followed + by `erase(iterator)`. +- `at(key)` is required only by [`to_ubjson`](../../api/basic_json/to_ubjson.md) and + [`to_bjdata`](../../api/basic_json/to_bjdata.md), but every container tried here provides it. +- `emplace` and `insert(value_type)` must return `#!cpp std::pair` and must have **unique-key** + semantics; multimaps cannot be used. +- The type must be swappable (via `std::swap` or an ADL `swap`). +- The comparison operators `==` and `<`; `!=`, `<=`, `>`, and `>=` are derived from them. Where the library uses + three-way comparison (C++20), `==` and `<=>` are required **instead** -- the six two-way operators do not satisfy + it. They implement [`basic_json`'s comparison operators](../../api/basic_json/operator_eq.md). + +### Required for heterogeneous key lookup + +The overloads of [`at`](../../api/basic_json/at.md), [`operator[]`](../../api/basic_json/operator%5B%5D.md), +[`find`](../../api/basic_json/find.md), [`contains`](../../api/basic_json/contains.md), +[`count`](../../api/basic_json/count.md), [`erase`](../../api/basic_json/erase.md), and +[`value`](../../api/basic_json/value.md) that accept a key type other than `object_t::key_type` require + +- a **transparent** comparator, i.e. [`object_comparator_t`](../../api/basic_json/object_comparator_t.md) has a member + type `is_transparent` (this is why the default comparator is `#!cpp std::less<>` since C++14), and +- corresponding heterogeneous `find`, `count`, `erase`, and `operator[]` overloads on the container. + +### Notes + +#### `std::unordered_map` needs an adapter + +`#!cpp std::unordered_map` cannot be passed directly: its third template parameter is a hash function, but +`basic_json` passes a comparator in that position. An alias template or wrapper that restores the expected argument +order makes it usable: + +```cpp +template +struct unordered_map_object + : std::unordered_map, std::equal_to, Allocator> +{ + using base_t = std::unordered_map, std::equal_to, Allocator>; + using base_t::base_t; +}; + +using unordered_json = nlohmann::basic_json; +``` + +Whether `#!cpp std::unordered_map` can be instantiated at all depends on the standard library: `object_t` is formed +while `basic_json` is still incomplete (see the warning above), and libstdc++ 9 needs the size of the mapped type to +instantiate the hash map's node type, so the adapter does not compile there. Newer libstdc++ versions, and the hash +maps listed below, do not have that problem. + +The adapter above works verbatim for Abseil's, Boost's, `phmap`'s and `gtl`'s hash maps, which all place the hash +function third and take a `#!cpp std::pair` allocator fifth. Two need a different adapter: + +- `ankerl::unordered_dense` expects an allocator over `#!cpp std::pair` (non-const key), so the allocator has + to be rebound to that or dropped. +- `robin_hood`'s fifth parameter is the non-type `MaxLoadFactor100`, so its adapter must drop the allocator entirely. + +None of these hash maps defines `key_compare`, so all of them additionally rely on `object_comparator_t` falling back +to [`default_object_comparator_t`](../../api/basic_json/default_object_comparator_t.md); see +[`object_comparator_t`](../../api/basic_json/object_comparator_t.md). + +#### Abseil hash maps + +`absl::flat_hash_map` and `absl::node_hash_map` tolerate an incomplete value type, but they take a hash function as +their third template argument. The same adapter as for `#!cpp std::unordered_map` makes them usable: + +```cpp +template +struct flat_hash_object + : absl::flat_hash_map, std::equal_to, Allocator> +{ + using base_t = absl::flat_hash_map, std::equal_to, Allocator>; + using base_t::base_t; +}; + +using flat_hash_json = nlohmann::basic_json; +``` + +`absl::node_hash_map` keeps references to the mapped values valid across insertions; `absl::flat_hash_map` does not, +which makes it behave like [`ordered_json`](../../api/ordered_json.md) with respect to +[iterator invalidation](../../api/basic_json/index.md#iterator-invalidation). Both expose a `capacity()` member +function, so [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md) treats them conservatively and keeps the +parent pointers correct either way. + +#### Iteration order + +The library never relies on the container's iteration order for correctness; it does determine the order in which +object keys are serialized by [`dump`](../../api/basic_json/dump.md) and visited by +[`items`](../../api/basic_json/items.md). See [Object Order](../object_order.md). + +#### `capacity()` marks a container as insertion-ordered + +With [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md) enabled, the library detects insertion-ordered maps by +probing for a `capacity()` member function (`nlohmann::ordered_map` inherits it from `std::vector`) and refreshes all +parent pointers after every insertion. An `ObjectType` that happens to have a `capacity()` member is therefore treated +conservatively -- this is correct, but slower. + +#### Key order and duplicate keys + +The library does not sort or de-duplicate keys itself; the behavior described in +[`object_t`](../../api/basic_json/object_t.md) is entirely the behavior of the chosen container. + +!!! tip "Reference implementation" + + `docs/mkdocs/docs/examples/custom_object_type.hpp` wraps a private `#!cpp std::map` and satisfies every + requirement above. It does not define `key_compare`, so `object_comparator_t` falls back to + [`default_object_comparator_t`](../../api/basic_json/default_object_comparator_t.md) -- a good starting point for + a custom `ObjectType`. + + ```cpp + --8<-- "examples/custom_object_type.hpp" + ``` + +??? example "Compiling and using it" + + ```cpp + --8<-- "examples/custom_object_type.cpp" + ``` + + Output: + + ```json + --8<-- "examples/custom_object_type.output" + ``` + +### Compatible containers + +| Container | Notes | +|----------------------------------------------------------------------------------|-------------------------------------------------------------------------------| +| `#!cpp std::map` (default) | | +| [`nlohmann::ordered_map`](../../api/ordered_map.md) | used by [`ordered_json`](../../api/ordered_json.md); keeps insertion order | +| [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) | keeps insertion order; adapter puts `fifo_map_compare` in the comparator slot | +| `boost::container::map`, `boost::container::flat_map` | no adapter needed | +| `#!cpp std::unordered_map` | through the adapter above; not with libstdc++ 9, see the note | +| `boost::unordered_map`, `boost::unordered_flat_map`, `boost::unordered_node_map` | through the adapter above | +| `absl::flat_hash_map`, `absl::node_hash_map` | through the adapter above; `flat_hash_map` moves mapped values on rehash | +| `phmap::flat_hash_map`, `phmap::node_hash_map`, `gtl::flat_hash_map` | through the adapter above | +| `ankerl::unordered_dense::map` and `segmented_map` | adapter must rebind or drop the allocator | +| `robin_hood::unordered_flat_map` | adapter must drop the allocator | +| `folly::F14NodeMap` | through the adapter above; requires C++20, see the note above | +| `folly::sorted_vector_map` | alias must drop the allocator, whose value type it disagrees on | + +### Containers that cannot be used + +| Container | Reason | +|--------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------| +| `absl::btree_map`, `phmap::btree_map`, `gtl::btree_map` | require a complete mapped type | +| `robin_hood::unordered_node_map`, `folly::F14FastMap`, `eastl::hash_map` | require a complete mapped type | +| `eastl::map` | EASTL iterators do not work with `#!cpp std::iterator_traits` | +| `tsl::ordered_map` | its iterators expose the mapped value as `#!cpp const` | +| `QMap` | no `value_type` member type | +| `QHash` | its `value_type` is the mapped type rather than a key/value pair, and its iterators dereference to the mapped value | +| `#!cpp std::multimap`, `#!cpp std::unordered_multimap` | `emplace` does not return `#!cpp std::pair` | + +## `ArrayType` + +`ArrayType` is instantiated as + +```cpp +using array_t = ArrayType>; +``` + +### Always required + +- The template must be usable with **two** type arguments (value type and allocator). +- Member types `value_type` and `iterator`. +- Constructors: default, copy, and move; and from an iterator range `(first, last)`. +- Member functions `begin()`, `end()`, `cbegin()`, `cend()`, `empty()`, `size()`, `max_size()`, `clear()`, + `operator[](size_type)`, `back()`, `push_back()`, `emplace_back()`, `pop_back()`, `resize()`, + `insert()` (single element, count, and range), `erase(pos)`, and `erase(first, last)`. + `basic_json::insert(pos, initializer_list)` goes through the range overload, so no initializer-list `insert` is + needed. `at(size_type)` is **not** required: [`basic_json::at(size_type)`](../../api/basic_json/at.md) checks the + index itself and then uses `operator[]`. +- `iterator` must be default-constructible, and it as well as the type returned by `cbegin()`/`cend()` must satisfy + [LegacyRandomAccessIterator](https://en.cppreference.com/w/cpp/named_req/RandomAccessIterator). + A `#!cpp static_assert` only checks for + [LegacyBidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator), but + [`dump`](../../api/basic_json/dump.md) (`cend() - 1`), + [`erase(idx)`](../../api/basic_json/erase.md) (`begin() + idx`), and the random-access operations of + [`basic_json::iterator`](../../api/basic_json/begin.md) require random access. +- The comparison operators, as for [`ObjectType`](#objecttype): `==` and `<`, or `==` and `<=>` under C++20. + +### Required for individual functions + +- A member type `value_type`, for [`to_bson`](../../api/basic_json/to_bson.md) of an array. +- A constructor from `(count, value)`, for + [`basic_json(size_type, const basic_json&)`](../../api/basic_json/basic_json.md). +- Swappability, via `#!cpp std::swap` or an ADL `swap`, for [`swap(array_t&)`](../../api/basic_json/swap.md). + +!!! note "`capacity()` is optional" + + With [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md) enabled, the library reads `array_t::capacity()` + to find out whether adding an element reallocated the array and moved its elements, which would invalidate the + parent pointers. An array type without a `capacity()` member function is handled conservatively: the parent + pointers of all elements are refreshed after every insertion, which makes adding *n* elements cost O(*n*²). Only + diagnostics builds pay this; without them `capacity()` is never called. + +!!! tip "Reference implementation" + + `docs/mkdocs/docs/examples/custom_array_type.hpp` wraps a private `#!cpp std::vector` and satisfies every + requirement above -- a good starting point for a custom `ArrayType`. + + ```cpp + --8<-- "examples/custom_array_type.hpp" + ``` + +??? example "Compiling and using it" + + ```cpp + --8<-- "examples/custom_array_type.cpp" + ``` + + Output: + + ```json + --8<-- "examples/custom_array_type.output" + ``` + +### Compatible containers + +| Container | Notes | +|---------------------------------------------------------|-------------------------------------------------------------------------------------------| +| `#!cpp std::vector` (default) | | +| `#!cpp std::deque` | references survive appends, but not insertions elsewhere; see the `capacity()` note above | +| `#!cpp std::pmr::vector` | through an alias, as the allocator comes from `AllocatorType` instead | +| `boost::container::vector`, `deque`, `devector` | | +| `boost::container::stable_vector` | the only one tried that keeps references valid across *every* insertion | +| `boost::container::small_vector`, `folly::small_vector` | through an alias that fixes the inline capacity | +| `boost::container::static_vector` | through the same kind of alias, for arrays that stay within the fixed capacity | +| `folly::fbvector` | requires C++20, see the note above | + +### Containers that cannot be used + +| Container | Reason | +|-------------------------------------|-----------------------------------------------------------------------------------------------| +| `#!cpp std::list` | no `operator[]`, and no random-access iterators | +| `eastl::vector`, `QList`, `QVector` | no `max_size()`; they handle the incomplete value type fine | +| `absl::InlinedVector` | requires a complete value type, see the note above | +| `absl::FixedArray` | the size is fixed at construction, so `resize`, `push_back`, `insert` and `erase` are missing | + +## `StringType` + +`StringType` is used **both** for JSON string values and for the keys of JSON objects +(`string_t` and `object_t::key_type`). + +### Always required + +- A member type `value_type` that is one byte wide and `char`-compatible. The library stores and processes UTF-8 + encoded `char` data and hands `data()` to `#!cpp std::strtoull`/`#!cpp std::strtoll`. + `#!cpp std::wstring`, `#!cpp std::u16string`, and `#!cpp std::u32string` are **not** valid choices; see the FAQ on + [wide string handling](../../home/faq.md#wide-string-handling). +- Constructors: default, copy, move, from `#!cpp const char*` (which must not be `#!cpp explicit`), from + `#!cpp (const char*, size_type)`, and from `#!cpp (size_type, char)`; and copy or move assignment. +- Member functions `size()`, `clear()`, `resize(n, c)`, `data()`, `push_back(char)`, and `operator[]` + (const and non-const, returning references). `c_str()` and `back()` are **not** required. +- `data()` must return a pointer to a contiguous, **null-terminated** buffer -- the parser hands it to + `#!cpp std::strtoull`. A type whose `data()` is not null-terminated does not fail to compile; it silently + misparses numbers. +- `append(const char*, size_type)`, used by [`dump`](../../api/basic_json/dump.md), and `append(const StringType&)`, + used by the CBOR reader for indefinite-length strings. The library's internal string concatenation additionally has + to append a `#!cpp char` and a `#!cpp const char*`; for each it selects between `append(arg)`, `#!cpp operator+=`, + `append(first, last)`, and `append(data, size)`. +- The comparison operator `==` against another `StringType`, and `<` for use as a key of the chosen + [`ObjectType`](#objecttype) (with the default comparator, `#!cpp std::less<>` must be able to compare two + `StringType` values, and a `StringType` with the key types used for lookup). `!=` is never applied to a + `StringType`, and `==` against `#!cpp const char*` is resolved by the implicit `#!cpp const char*` constructor. + +### Required for the binary formats + +- `resize(n)`, used by the readers to make room for a block of bytes. +- Non-const `operator[]`, into which the readers `#!cpp std::memcpy` those bytes. A non-`#!cpp const` `data()` would + serve just as well, but `#!cpp std::string` has only had one since C++17, and the library still supports C++11. + +### Required for JSON Pointer, `flatten`, and `diff` + +- A static member `npos` and the member function `find_first_of(char, size_type)` -- together with `data()`, + `reserve(n)`, and `append(const char*, size_type)` they implement the escaping and unescaping of reference tokens + described in RFC 6901. Neither `find(const StringType&, size_type)`, nor `substr(pos, count)`, nor + `replace(pos, count, const StringType&)` is required. +- `empty()`. +- `begin()` and `end()` -- used by + [`operator[](const json_pointer&)`](../../api/basic_json/operator%5B%5D.md) to decide whether a reference token + denotes an array index. + +### Required for other functionality + +| Functionality | Additional requirement | +|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [`diff`](../../api/basic_json/diff.md), [`items`](../../api/basic_json/items.md), [`std::hash`](../../api/basic_json/std_hash.md) | conversion of a `#!cpp std::size_t` to `StringType`: either assignability from the result of `#!cpp std::to_string`, or an ADL overload `#!cpp void int_to_string(StringType&, std::size_t)` | +| [`std::hash`](../../api/basic_json/std_hash.md) | additionally a specialization of `#!cpp std::hash` | +| [`to_bson`](../../api/basic_json/to_bson.md) | `find(value_type)` and `npos` | +| [`parse`](../../api/basic_json/parse.md) from a `string_t` | the input adapters must accept it; otherwise pass a character range | +| `#!cpp operator<<(std::ostream&, const json_pointer&)` | streamability to `#!cpp std::ostream` | +| exception messages | `data()` and `size()`, or `begin()` and `end()` | + +### Compatible types + +| Type | Notes | +|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `#!cpp std::string` (default) | | +| `#!cpp std::basic_string` with a custom **stateless** allocator | | +| `#!cpp std::pmr::string` | see the warning below before relying on the memory resource | +| `boost::container::string` | needs a user-supplied `#!cpp std::hash` specialization (Boost provides `boost::hash` instead) | +| `folly::fbstring` | requires C++20, see the note above | +| `eastl::string` | needs a user-supplied `#!cpp std::hash` and an ADL `int_to_string` (it is not assignable from a `#!cpp std::string`); [`parse`](../../api/basic_json/parse.md) does not accept it directly -- pass a character range or a `#!cpp std::string` | +| a custom string class in a user-defined namespace | if the requirements above are met | + +### Types that cannot be used + +| Type | Reason | +|----------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------| +| `#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string` | the character type is not one byte wide | +| `#!cpp std::u8string` | one byte wide, but `#!cpp char8_t` is not `#!cpp char`-compatible | +| `absl::Cord` | no `value_type`, and the storage is not contiguous | +| `QString` | no `append(const char*, size_type)`; its `QChar` is also two bytes wide, though that is never diagnosed | + +!!! warning "A `std::pmr::string` mostly does not use the memory resource you choose" + + `basic_json` cannot be given an allocator or a memory resource. `AllocatorType` is default-constructed at every + allocation and has to be stateless (see [`AllocatorType`](#allocatortype)), and string values the library creates + are constructed with their own default allocator. So: + + - Every string the library itself produces -- from [`parse`](../../api/basic_json/parse.md), from + [`dump`](../../api/basic_json/dump.md), or by default construction -- allocates from + `#!cpp std::pmr::get_default_resource()`. + - **Copying** an arena-backed string into a value silently drops its memory resource: the copy lands on the + default resource, because `#!cpp std::pmr::polymorphic_allocator` does not propagate on copy construction. + Nothing warns about this. + - **Moving** one in does keep it, and later growth still allocates from that arena -- but it does not survive a + copy of the enclosing `basic_json`. + - Passing `#!cpp std::pmr::polymorphic_allocator` as `AllocatorType` does not work around any of this; it does + not compile. + + Apart from moving a string in, the only way to redirect these allocations is the process-global + `#!cpp std::pmr::set_default_resource()`. + +!!! tip "Reference implementation" + + `docs/mkdocs/docs/examples/custom_string_type.hpp` wraps a private `#!cpp std::string` and satisfies every + requirement above -- a good starting point for a custom `StringType`. The unit test + `tests/src/unit-alt-string.cpp` contains a more thorough variant, `alt_string`, exercised against a larger part + of the API. + + ```cpp + --8<-- "examples/custom_string_type.hpp" + ``` + +??? example "Compiling and using it" + + ```cpp + --8<-- "examples/custom_string_type.cpp" + ``` + + Output: + + ```json + --8<-- "examples/custom_string_type.output" + ``` + +## `BooleanType` + +`boolean_t` is stored **directly** inside `basic_json`, as a member of an anonymous union. + +### Always required + +- A literal type that is trivially default-constructible, trivially copyable, and trivially destructible; otherwise the + union's special member functions are deleted. +- **Implicitly** convertible from `#!cpp bool` -- an `#!cpp explicit` constructor is not enough, because the + `to_json` overload for a custom `BooleanType` is constrained on `#!cpp std::is_convertible` -- and contextually + convertible to `#!cpp bool` (here an `#!cpp explicit operator bool` is fine). +- Comparison operators `==`, `!=`, `<`, `<=`, `>`, `>=` (or `<=>`). +- Convertible from and to `#!cpp bool` through the serializer, because + [`get()`](../../api/basic_json/get.md) is used internally. + +There is little reason to use anything other than `#!cpp bool` here. + +### Compatible types + +`#!cpp bool` is the only usable choice. Another trivially copyable type that is implicitly convertible to and from +`#!cpp bool` -- `#!cpp std::uint8_t`, say -- does compile, and JSON booleans still round-trip, but the type then +serves as both `boolean_t` and an ordinary integer: `basic_json` can no longer be constructed or assigned from a +`#!cpp std::uint8_t` at all (the boolean and unsigned-integer `to_json` overloads become ambiguous), and +[`get()`](../../api/basic_json/get.md) on a number throws +[`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) instead of returning the value. + +## `NumberIntegerType` and `NumberUnsignedType` + +Both types are stored **directly** inside `basic_json`'s union. + +### Always required + +- `#!cpp std::is_integral` must be satisfied: `NumberIntegerType` must be a **signed** integer type, + `NumberUnsignedType` an **unsigned** integer type. Class types are not supported -- among others, the constructors + taking integer values are constrained on `#!cpp std::is_integral`. +- Trivially default-constructible, trivially copyable, and trivially destructible (union member). +- `#!cpp std::numeric_limits` must be specialized for both types. +- `NumberUnsignedType` must be able to represent the absolute value of every `NumberIntegerType` value; serialization + of negative numbers converts the value to `NumberUnsignedType`. A `#!cpp static_assert` requires it to be at least as + wide as `NumberIntegerType`, which is what that amounts to for the standard integer types. +- Both types must fit into the internal 64-character number buffer used by + [`dump`](../../api/basic_json/dump.md), which is the case for all standard integer types. +- [`std::hash`](../../api/basic_json/std_hash.md) additionally requires `#!cpp std::hash` specializations. + +### Notes + +The number types influence what the parser accepts: an integer literal that does not round-trip through the chosen type +is stored as [`number_float_t`](../../api/basic_json/number_float_t.md) instead. Choosing types narrower than 64 bits +therefore silently changes parse results rather than raising an error. See +[Number Handling](number_handling.md) for details. + +### Compatible types + +| Type pair | Support | +|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `#!cpp std::int64_t` / `#!cpp std::uint64_t` (default) | full | +| `#!cpp std::int32_t` / `#!cpp std::uint32_t`, `#!cpp long long` / `#!cpp unsigned long long` | full; narrower types change which literals the parser can represent | +| any other pair of standard signed/unsigned integer types | full | +| class types, enumerations | not usable; `#!cpp std::is_integral` must hold | +| `#!cpp bool`, or a type already used for another member of the union | not usable; `#!cpp std::is_integral` is in fact `#!cpp true`, but the `get_impl_ptr` overloads for `boolean_t`, `number_integer_t`, `number_unsigned_t` and `number_float_t` would collide | + +## `NumberFloatType` + +`number_float_t` is stored **directly** inside `basic_json`'s union. + +### Always required + +- Trivially default-constructible, trivially copyable, and trivially destructible (union member). +- `#!cpp std::numeric_limits` must be specialized; `max_digits10` is used to size the conversion. +- `#!cpp std::isfinite` must be applicable to the type. + +### Required for parsing and serialization + +`NumberFloatType` must be one of `#!cpp float`, `#!cpp double`, or `#!cpp long double`: + +- The [parser](../parsing/index.md) converts number literals with `#!cpp std::strtof`, `#!cpp std::strtod`, or + `#!cpp std::strtold`; the library provides overloads for exactly these three types. +- [`dump`](../../api/basic_json/dump.md) falls back to `#!cpp std::snprintf` with the `%g` and `%Lg` conversion + specifiers, for which the library likewise provides only `#!cpp double` and `#!cpp long double` overloads + (`#!cpp float` is promoted to `#!cpp double`). + +If `#!cpp std::numeric_limits` describes an IEEE 754 binary32 or binary64 number, `dump` uses the +Grisu2 algorithm, which produces the shortest representation that round-trips. Otherwise the `snprintf` fallback with +`max_digits10` digits is used. + +### Required for the binary formats + +`NumberFloatType` must be `#!cpp float` or `#!cpp double`. The writers for +[CBOR, MessagePack, UBJSON, BJData, and BSON](../binary_formats/index.md) map a floating-point value onto an IEEE 754 +binary32 or binary64 field and have no encoding for `#!cpp long double`. + +### Compatible types + +| Type | Support | +|--------------------------|-----------------------------------------------------------------------------------------------------------------------| +| `#!cpp double` (default) | full; short round-trip output through Grisu2 | +| `#!cpp float` | full; short round-trip output through Grisu2 | +| `#!cpp long double` | `dump` and `parse` only; the binary format writers do not compile, as they only handle IEEE 754 binary32 and binary64 | +| any other type | not usable | + +## `AllocatorType` + +`AllocatorType` is instantiated with **one** argument, for each of `object_t`, `array_t`, `string_t`, `binary_t`, +`basic_json`, and `#!cpp std::pair`. + +### Always required + +- The template must be usable with exactly one type argument. The library instantiates `AllocatorType` directly and + never uses `#!cpp std::allocator_traits<...>::rebind_alloc`. +- It must satisfy the [Allocator](https://en.cppreference.com/w/cpp/named_req/Allocator) named requirement so that + `#!cpp std::allocator_traits` can be used with it. +- It must be **default-constructible and stateless**. Objects are allocated with a default-constructed allocator and + deallocated with a *different* default-constructed allocator, and + [`get_allocator()`](../../api/basic_json/get_allocator.md) returns a default-constructed instance. Allocators + carrying state are not supported, so there is no way to tell a `basic_json` where to allocate from; see the note + under [`StringType`](#stringtype) for what that means in practice. A stateful allocator is **not diagnosed**: it + compiles and silently ignores the state. +- It must support **incomplete types**: `AllocatorType` is instantiated inside the definition of + `basic_json` itself. +- `#!cpp std::allocator_traits>::pointer` becomes + [`basic_json::pointer`](../../api/basic_json/index.md#container-types), and iterators are constructed from raw + `#!cpp basic_json*` values. The `pointer` type must therefore be a plain pointer; fancy pointers are not supported. + +### Compatible types + +| Type | Support | +|-------------------------------------------------------------------|----------------------------------------| +| `#!cpp std::allocator` (default) | full | +| a custom stateless allocator template | full | +| stateful allocators, e.g. `#!cpp std::pmr::polymorphic_allocator` | not usable; see the requirements above | + +## `JSONSerializer` + +`JSONSerializer` is instantiated as `JSONSerializer` and defaults to +[`adl_serializer`](../../api/adl_serializer/index.md). + +### Always required + +- The template must accept **two** type arguments. It does not have to give the second one a default -- `basic_json` + declares the parameter as `#!cpp template class JSONSerializer`, so uses such as + `#!cpp JSONSerializer` inside the library supply `#!cpp void` themselves. The second parameter exists so that + partial specializations can be constrained by SFINAE. +- For every type `T` that is converted **to** a JSON value, a static member function + `#!cpp static void to_json(basic_json&, T)` must exist. +- For every type `T` that is converted **from** a JSON value, either + `#!cpp static void from_json(const basic_json&, T&)` or `#!cpp static T from_json(const basic_json&)` must exist. + The latter form is required for types that are not default-constructible; see + [Arbitrary Types Conversions](../arbitrary_types.md). +- To support the [converting constructor](../../api/basic_json/basic_json.md) between different `basic_json` + specializations, `to_json` must be available for `boolean_t`, `number_integer_t`, `number_unsigned_t`, + `number_float_t`, `string_t`, `object_t`, `array_t`, and `binary_t` of the *source* specialization. + +### Compatible types + +| Type | Support | +|---------------------------------------------------------------------------|-------------------------------------------------------------------| +| [`nlohmann::adl_serializer`](../../api/adl_serializer/index.md) (default) | full | +| a class template deriving from `adl_serializer` | full; the usual way to change behavior while keeping the defaults | +| an unrelated template with the same interface | full, but it has to handle every type the library converts | + +## `BinaryType` + +`BinaryType` is not a JSON type; it is used for the byte strings of the +[binary formats](../binary_formats/index.md). It is wrapped as + +```cpp +using binary_t = nlohmann::byte_container_with_subtype; +``` + +### Always required + +- A non-`final` class type -- [`byte_container_with_subtype`](../../api/byte_container_with_subtype/index.md) derives + from it publicly. +- A member type `value_type` that is **exactly one byte** wide (e.g., `#!cpp std::uint8_t`, `#!cpp char`, or + `#!cpp std::byte`). Readers and writers reinterpret the container's storage as raw bytes, so a wider `value_type` is + rejected with a `#!cpp static_assert`. +- Contiguous storage: the binary readers `#!cpp std::memcpy` into `#!cpp &binary[n]`, the writers `reinterpret_cast` + `data()`. `#!cpp data() + n` would do for the readers too, but they share one helper with + [`StringType`](#stringtype), whose non-`#!cpp const` `data()` is C++17 and later only. +- Default-constructible, copy-constructible, and move-constructible. +- Member functions `size()`, `empty()`, `data()`, `resize()`, `operator[]`, `back()`, `begin()`, `end()`, `cbegin()`, + and `cend()` with random-access iterators, and `insert(pos, first, last)`, which the CBOR reader uses to join the + chunks of an indefinite-length byte string. `push_back()` is **not** required. +- Comparison operators: `==` is used by + [`byte_container_with_subtype`](../../api/byte_container_with_subtype/index.md), the relational operators by + [`basic_json`'s comparison operators](../../api/basic_json/operator_le.md). + +### Required for individual functions + +- `clear()`, for [`basic_json::clear()`](../../api/basic_json/clear.md). + +`max_size()`, `at()`, `reserve()`, `erase()`, `pop_back()`, and `emplace_back()` are **not** used at all. + +See [`binary_t`](../../api/basic_json/binary_t.md) for how a non-default `BinaryType` changes the meaning of assigning +such a container to a `basic_json` value. + +!!! tip "Reference implementation" + + `docs/mkdocs/docs/examples/custom_binary_type.hpp` wraps a private `#!cpp std::vector` and satisfies + every requirement above -- a good starting point for a custom `BinaryType`. + + ```cpp + --8<-- "examples/custom_binary_type.hpp" + ``` + +??? example "Compiling and using it" + + ```cpp + --8<-- "examples/custom_binary_type.cpp" + ``` + + Output: + + ```json + --8<-- "examples/custom_binary_type.output" + ``` + +### Compatible containers + +| Container | Notes | +|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------| +| `#!cpp std::vector` (default) | | +| `#!cpp std::vector`, `#!cpp std::vector` | `dump()` writes the bytes as 0..255 whichever is used | +| `boost::container::vector`, `boost::container::small_vector` | | +| `absl::InlinedVector` | usable here, unlike as an `ArrayType`, because the value type is complete | +| `eastl::vector` | usable here, unlike as an `ArrayType`, because `max_size()` is not needed | +| `folly::fbvector` | requires C++20, see the note above | + +### Containers that cannot be used + +| Container | Reason | +|------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `QByteArray` | no `empty()` (it spells that `isEmpty()`); its `insert` takes an index rather than an iterator; and it converts to `string_t`, which makes `to_json` ambiguous between a string and a binary value | +| `#!cpp std::string` | `binary_t::container_type` and `string_t` would be the same type, so the two [`swap`](../../api/basic_json/swap.md) overloads collide and `basic_json` cannot be instantiated at all | +| `#!cpp std::deque` | storage is not contiguous, so there is no `data()` | +| containers whose `value_type` is wider than one byte | see above -- accepted by the compiler, wrong at runtime | + +## `CustomBaseClass` + +`CustomBaseClass` is an extension point: unless it is `#!cpp void` (the default, which selects the empty +`nlohmann::json_default_base`), `basic_json` publicly derives from it. + +### Always required + +- A non-`final`, default-constructible class type. +- `basic_json` is copy-/move-constructible and copy-/move-assignable only if `CustomBaseClass` is. + +### Notes + +`basic_json` is documented to be a +[StandardLayoutType](https://en.cppreference.com/w/cpp/named_req/StandardLayoutType). Because `basic_json` has +non-static data members of its own, a `CustomBaseClass` with non-static data members forfeits this guarantee. + +Note the namespace of `CustomBaseClass` becomes an associated namespace of `basic_json` for the purpose of +argument-dependent lookup. + +See [`json_base_class_t`](../../api/basic_json/json_base_class_t.md) for an example. + +### Compatible types + +| Type | Support | +|----------------------------------------------|----------------------------------------------------------------------------| +| `#!cpp void` (default) | an empty base class is used; no effect on `basic_json` | +| any default-constructible, non-`final` class | full; see [`json_base_class_t`](../../api/basic_json/json_base_class_t.md) | + +## Cross-specialization conversions + +Converting a value from one `basic_json` specialization into another (see the +[converting constructor](../../api/basic_json/basic_json.md)) imposes two additional requirements that are not +diagnosed at compile time. With assertions enabled they abort on the `#!cpp JSON_ASSERT` at the end of the converting +constructor; under `#!cpp NDEBUG` they fail **silently** at runtime: + +- The target `string_t` must be directly constructible from the source `string_t`. Otherwise the string is converted to + an array of character codes. +- The target `object_t::key_type` must be directly constructible from the source object's key type. Otherwise the + object is converted to an array of key/value pairs. + +See [issue #3425](https://github.com/nlohmann/json/issues/3425), [`string_t`](../../api/basic_json/string_t.md), and +[`object_t`](../../api/basic_json/object_t.md). + +## See also + +- [Types](index.md) -- overview of how JSON values are stored +- [Number Handling](number_handling.md) -- how the number types affect parsing and serialization +- [Object Order](../object_order.md) -- using an insertion-ordered `ObjectType` +- [`basic_json`](../../api/basic_json/index.md) -- API documentation of the class template diff --git a/docs/mkdocs/mkdocs.yml b/docs/mkdocs/mkdocs.yml index 5c9f72fc3..d0f9cfdfd 100644 --- a/docs/mkdocs/mkdocs.yml +++ b/docs/mkdocs/mkdocs.yml @@ -98,6 +98,7 @@ nav: - Types: - features/types/index.md - features/types/number_handling.md + - features/types/template_parameters.md - Integration: - integration/index.md - integration/migration_guide.md diff --git a/include/nlohmann/detail/exceptions.hpp b/include/nlohmann/detail/exceptions.hpp index 3e5b45101..808a3b6bf 100644 --- a/include/nlohmann/detail/exceptions.hpp +++ b/include/nlohmann/detail/exceptions.hpp @@ -101,7 +101,10 @@ class exception : public std::exception { if (&element.second == current) { - tokens.emplace_back(element.first.c_str()); + // data() is null-terminated, so a key containing + // a null byte is cut short here rather than + // truncating the whole message at what() + tokens.emplace_back(element.first.data()); break; } } diff --git a/include/nlohmann/detail/hash.hpp b/include/nlohmann/detail/hash.hpp index 61b3469f1..be8063f89 100644 --- a/include/nlohmann/detail/hash.hpp +++ b/include/nlohmann/detail/hash.hpp @@ -114,7 +114,9 @@ std::size_t hash(const BasicJsonType& j) seed = combine(seed, static_cast(j.get_binary().subtype())); for (const auto byte : j.get_binary()) { - seed = combine(seed, std::hash {}(byte)); + // the cast is needed for binary types whose value type is not + // an integer (e.g., std::byte) + seed = combine(seed, std::hash {}(static_cast(byte))); } return seed; } diff --git a/include/nlohmann/detail/input/binary_reader.hpp b/include/nlohmann/detail/input/binary_reader.hpp index e0343fd0b..df46eea58 100644 --- a/include/nlohmann/detail/input/binary_reader.hpp +++ b/include/nlohmann/detail/input/binary_reader.hpp @@ -3146,7 +3146,10 @@ class binary_reader number_string, out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr)); } - return sax->number_float(parsed_float, std::move(number_string)); + // number_string is a std::string, while the SAX interface takes a + // string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + return sax->number_float(parsed_float, string_t(number_string.data(), number_string.size())); } case token_type::uninitialized: case token_type::literal_true: diff --git a/include/nlohmann/detail/iterators/iter_impl.hpp b/include/nlohmann/detail/iterators/iter_impl.hpp index 44448611a..22f3ffc39 100644 --- a/include/nlohmann/detail/iterators/iter_impl.hpp +++ b/include/nlohmann/detail/iterators/iter_impl.hpp @@ -88,8 +88,13 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci iter_impl() = default; ~iter_impl() = default; - iter_impl(iter_impl&&) noexcept = default; - iter_impl& operator=(iter_impl&&) noexcept = default; + // the exception specification is left to be computed rather than declared: + // an array or object type whose iterator is not nothrow move constructible + // (std::deque's is not before libstdc++ 11) would make a declared noexcept + // differ from the implicit one, which deletes the function -- and is an + // error outright with older compilers + iter_impl(iter_impl&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor,cppcoreguidelines-noexcept-move-operations) + iter_impl& operator=(iter_impl&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor,cppcoreguidelines-noexcept-move-operations) /*! @brief constructor for a given JSON instance diff --git a/include/nlohmann/detail/json_pointer.hpp b/include/nlohmann/detail/json_pointer.hpp index 247c5babb..1540a8d6f 100644 --- a/include/nlohmann/detail/json_pointer.hpp +++ b/include/nlohmann/detail/json_pointer.hpp @@ -17,6 +17,7 @@ #endif // JSON_NO_IO #include // max #include // accumulate +#include // set #include // string #include // move #include // vector @@ -71,7 +72,7 @@ class json_pointer string_t{}, [](const string_t& a, const string_t& b) { - return detail::concat(a, '/', detail::escape(b)); + return detail::concat(a, '/', detail::escape(b)); }); } @@ -265,7 +266,7 @@ class json_pointer JSON_THROW(detail::parse_error::create(109, 0, detail::concat("array index '", s, "' is not a number"), nullptr)); } - const char* p = s.c_str(); + const char* p = s.data(); char* p_end = nullptr; // NOLINT(misc-const-correctness) errno = 0; // strtoull doesn't reset errno const unsigned long long res = std::strtoull(p, &p_end, 10); // NOLINT(runtime/int) @@ -300,19 +301,35 @@ class json_pointer } private: + /*! + @brief the reference token sequences that denote arrays + + @ref unflatten collects the pointer prefixes that have a reference token 0 + among their children; @ref get_and_create creates arrays exactly below + those prefixes and objects everywhere else. Deciding this up front keeps + the result independent of the order in which the flattened object is + iterated, which is unspecified for some object types. + */ + using array_parents_t = std::set>; + /*! @brief create and return a reference to the pointed to value @complexity Linear in the number of reference tokens. + @throw parse_error.106 if an array index begins with '0' @throw parse_error.109 if array index is not a number @throw type_error.313 if value cannot be unflattened */ template - BasicJsonType& get_and_create(BasicJsonType& j) const + BasicJsonType& get_and_create(BasicJsonType& j, const array_parents_t& array_parents) const { auto* result = &j; + // the reference tokens that have been consumed so far; used to look up + // whether the value to be created below is an array or an object + std::vector prefix; + // in case no reference tokens exist, return a reference to the JSON value // j which will be overwritten by a primitive value for (const auto& reference_token : reference_tokens) @@ -321,10 +338,11 @@ class json_pointer { case detail::value_t::null: { - if (reference_token == "0") + if (array_parents.find(prefix) != array_parents.end()) { - // start a new array if the reference token is 0 - result = &result->operator[](0); + // some reference token below this position is 0, so the + // value is an array + result = &result->operator[](array_index(reference_token)); } else { @@ -364,6 +382,8 @@ class json_pointer default: JSON_THROW(detail::type_error::create(313, "invalid value to unflatten", &j)); } + + prefix.push_back(reference_token); } return *result; @@ -837,7 +857,8 @@ class json_pointer { // use the text between the beginning of the reference token // (start) and the last slash (slash). - auto reference_token = reference_string.substr(start, slash - start); + const auto count = (slash == string_t::npos ? reference_string.size() : slash) - start; + auto reference_token = string_t(reference_string.data() + start, count); // check reference tokens are properly escaped for (std::size_t pos = reference_token.find_first_of('~'); @@ -953,6 +974,24 @@ class json_pointer BasicJsonType result; + // collect the pointer prefixes that have a reference token 0 among + // their children; the values below them are arrays, all others are + // objects (see array_parents_t) + array_parents_t array_parents; + for (const auto& element : *value.m_data.m_value.object) + { + json_pointer ptr(element.first); + std::vector prefix; + for (auto& reference_token : ptr.reference_tokens) + { + if (reference_token == "0") + { + array_parents.insert(prefix); + } + prefix.push_back(std::move(reference_token)); + } + } + // iterate the JSON object values for (const auto& element : *value.m_data.m_value.object) { @@ -965,7 +1004,7 @@ class json_pointer // that if the JSON pointer is "" (i.e., points to the whole value), // function get_and_create returns a reference to the result itself. // An assignment will then create a primitive value. - json_pointer(element.first).get_and_create(result) = element.second; + json_pointer(element.first).get_and_create(result, array_parents) = element.second; } return result; diff --git a/include/nlohmann/detail/meta/type_traits.hpp b/include/nlohmann/detail/meta/type_traits.hpp index ebf6a2c26..6f8bf2a3d 100644 --- a/include/nlohmann/detail/meta/type_traits.hpp +++ b/include/nlohmann/detail/meta/type_traits.hpp @@ -172,17 +172,18 @@ struct has_to_json < BasicJsonType, T, enable_if_t < !is_basic_json::value >> template using detect_key_compare = typename T::key_compare; -template -struct has_key_compare : std::integral_constant::value> {}; - -// obtains the actual object key comparator +// obtains the actual object key comparator: object_t::key_compare if the +// object type defines it, and default_object_comparator_t otherwise +// +// note detected_or_t is used rather than std::conditional, because the latter +// names both of its type arguments eagerly; object_t::key_compare would then +// be a hard error for an object type that does not define it template struct actual_object_comparator { using object_t = typename BasicJsonType::object_t; using object_comparator_t = typename BasicJsonType::default_object_comparator_t; - using type = typename std::conditional < has_key_compare::value, - typename object_t::key_compare, object_comparator_t>::type; + using type = detected_or_t; }; template @@ -778,6 +779,22 @@ using has_erase_with_key_type = typename std::conditional < std::true_type, std::false_type >::type; +template +using detect_erase_with_iterator = decltype(std::declval().erase(std::declval())); + +// type trait to check if erase(iterator) returns void instead of the following +// iterator, as the object types that do not compute a successor the caller may +// not need do +template +using erase_returns_void = is_detected_exact; + +template +using detect_capacity = decltype(std::declval().capacity()); + +// type trait to check if a type has a capacity() member function +template +struct has_capacity : std::integral_constant::value> {}; + // a naive helper to check if a type is an ordered_map (exploits the fact that // ordered_map inherits capacity() from std::vector) template diff --git a/include/nlohmann/detail/output/binary_writer.hpp b/include/nlohmann/detail/output/binary_writer.hpp index 28290de3e..e9ccd23b5 100644 --- a/include/nlohmann/detail/output/binary_writer.hpp +++ b/include/nlohmann/detail/output/binary_writer.hpp @@ -261,7 +261,7 @@ class binary_writer // step 2: write the string oa->write_characters( - reinterpret_cast(j.m_data.m_value.string->c_str()), + reinterpret_cast(j.m_data.m_value.string->data()), j.m_data.m_value.string->size()); break; } @@ -581,7 +581,7 @@ class binary_writer // step 2: write the string oa->write_characters( - reinterpret_cast(j.m_data.m_value.string->c_str()), + reinterpret_cast(j.m_data.m_value.string->data()), j.m_data.m_value.string->size()); break; } @@ -798,7 +798,7 @@ class binary_writer } write_number_with_ubjson_prefix(j.m_data.m_value.string->size(), true, use_bjdata); oa->write_characters( - reinterpret_cast(j.m_data.m_value.string->c_str()), + reinterpret_cast(j.m_data.m_value.string->data()), j.m_data.m_value.string->size()); break; } @@ -897,7 +897,9 @@ class binary_writer for (size_t i = 0; i < j.m_data.m_value.binary->size(); ++i) { oa->write_character(to_char_type(bjdata_draft3 ? 'B' : 'U')); - oa->write_character(to_char_type(j.m_data.m_value.binary->data()[i])); + // the cast is needed for binary types whose value type + // is not an integer (e.g., std::byte) + oa->write_character(to_char_type(static_cast(j.m_data.m_value.binary->data()[i]))); } } @@ -958,7 +960,7 @@ class binary_writer { write_number_with_ubjson_prefix(el.first.size(), true, use_bjdata); oa->write_characters( - reinterpret_cast(el.first.c_str()), + reinterpret_cast(el.first.data()), el.first.size()); write_ubjson(el.second, use_count, use_type, prefix_required, use_bjdata, bjdata_version); } @@ -1021,8 +1023,11 @@ class binary_writer { oa->write_character(to_char_type(element_type)); oa->write_characters( - reinterpret_cast(name.c_str()), - name.size() + 1u); + reinterpret_cast(name.data()), + name.size()); + // the terminating null byte is written explicitly rather than taken + // from the buffer, so that string_t::data() need not be null-terminated + oa->write_character(to_char_type(0x00)); } /*! @@ -1063,8 +1068,11 @@ class binary_writer write_number(to_bson_length(value.size() + 1ul), true); oa->write_characters( - reinterpret_cast(value.c_str()), - value.size() + 1); + reinterpret_cast(value.data()), + value.size()); + // the terminating null byte is written explicitly rather than taken + // from the buffer, so that string_t::data() need not be null-terminated + oa->write_character(to_char_type(0x00)); } /*! @@ -1155,7 +1163,11 @@ class binary_writer const std::size_t embedded_document_size = std::accumulate(std::begin(value), std::end(value), static_cast(0), [&array_index](std::size_t result, const typename BasicJsonType::array_t::value_type & el) { - return result + calc_bson_element_size(std::to_string(array_index++), el); + // the index is built as a std::string, while calc_bson_element_size + // takes a string_t; convert explicitly, as the two are only + // implicitly convertible for some string types + const auto key = std::to_string(array_index++); + return result + calc_bson_element_size(string_t(key.data(), key.size()), el); }); return sizeof(std::int32_t) + embedded_document_size + 1ul; @@ -1182,7 +1194,11 @@ class binary_writer for (const auto& el : value) { - write_bson_element(std::to_string(array_index++), el); + // the index is built as a std::string, while write_bson_element takes + // a string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + const auto key = std::to_string(array_index++); + write_bson_element(string_t(key.data(), key.size()), el); } oa->write_character(to_char_type(0x00)); diff --git a/include/nlohmann/detail/output/serializer.hpp b/include/nlohmann/detail/output/serializer.hpp index 9560729ad..f9e7f7840 100644 --- a/include/nlohmann/detail/output/serializer.hpp +++ b/include/nlohmann/detail/output/serializer.hpp @@ -1061,7 +1061,7 @@ class serializer { case error_handler_t::strict: { - JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", hex_bytes(static_cast(s.back() | 0))), nullptr)); + JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", hex_bytes(static_cast(s[s.size() - 1] | 0))), nullptr)); } case error_handler_t::ignore: @@ -1322,6 +1322,19 @@ class serializer pos += 6; } + /*! + @brief convert a single element of a binary value to its byte value + + The elements of a binary value are dumped as the numbers 0..255, regardless + of the value type of the configured BinaryType: that type may be signed + (`char`), unsigned (`std::uint8_t`), or not an integer at all + (`std::byte`), none of which @ref dump_integer can handle uniformly. + */ + static std::uint8_t to_byte_value(binary_char_t x) noexcept + { + return static_cast(x); + } + // templates to avoid warnings about useless casts template ::value, int> = 0> bool is_negative_number(NumberType x) @@ -1343,8 +1356,10 @@ class serializer an arbitrary number, and the three digits it takes at most are written straight into the write buffer. - Any byte type that is not a plain unsigned byte is left to @ref dump_integer, - whose representation of it may differ. + Any byte type that is not a plain unsigned byte is converted to its + @ref to_byte_value "byte value" and left to @ref dump_integer, so a signed + or non-integral BinaryType::value_type (`char`, `std::byte`, ...) still + dumps as 0..255. */ template void dump_byte(const ByteType value) @@ -1357,7 +1372,7 @@ class serializer template void dump_byte(const ByteType value, std::false_type /*is_plain_byte*/) { - dump_integer(value); + dump_integer(to_byte_value(value)); } template @@ -1403,8 +1418,7 @@ class serializer template < typename NumberType, detail::enable_if_t < std::is_integral::value || std::is_same::value || - std::is_same::value || - std::is_same::value, + std::is_same::value, int > = 0 > void dump_integer(NumberType x) { diff --git a/include/nlohmann/detail/string_escape.hpp b/include/nlohmann/detail/string_escape.hpp index 7715dde7a..0d24a56bc 100644 --- a/include/nlohmann/detail/string_escape.hpp +++ b/include/nlohmann/detail/string_escape.hpp @@ -8,50 +8,56 @@ #pragma once +#include // size_t + #include NLOHMANN_JSON_NAMESPACE_BEGIN namespace detail { -/*! -@brief replace all occurrences of a substring by another string - -@param[in,out] s the string to manipulate; changed so that all - occurrences of @a f are replaced with @a t -@param[in] f the substring to replace with @a t -@param[in] t the string to replace @a f - -@pre The search string @a f must not be empty. **This precondition is -enforced with an assertion.** - -@since version 2.0.0 -*/ -template -inline void replace_substring(StringType& s, const StringType& f, - const StringType& t) -{ - JSON_ASSERT(!f.empty()); - for (auto pos = s.find(f); // find the first occurrence of f - pos != StringType::npos; // make sure f was found - s.replace(pos, f.size(), t), // replace with t, and - pos = s.find(f, pos + t.size())) // find the next occurrence of f - {} -} - /*! * @brief string escaping as described in RFC 6901 (Sect. 4) * @param[in] s string to escape * @return escaped string * * Note the order of escaping "~" to "~0" and "/" to "~1" is important. + * + * The string is rebuilt in a single pass, appending whole runs between the + * characters that need escaping. Scanning with find_first_of() keeps the + * common case -- nothing to escape -- as fast as a single search, while + * repeated replace() calls would move the tail of the string once per + * escaped character. */ template -inline StringType escape(StringType s) +inline StringType escape(const StringType& s) { - replace_substring(s, StringType{"~"}, StringType{"~0"}); - replace_substring(s, StringType{"/"}, StringType{"~1"}); - return s; + auto next_special = [&s](std::size_t from) + { + const auto tilde = s.find_first_of('~', from); + const auto slash = s.find_first_of('/', from); + return tilde < slash ? tilde : slash; // npos is the largest value + }; + + auto pos = next_special(0); + if (pos == StringType::npos) + { + return s; + } + + StringType result; + result.reserve(s.size() + 2); + + std::size_t run = 0; + while (pos != StringType::npos) + { + result.append(s.data() + run, pos - run); + result.append(s[pos] == '~' ? "~0" : "~1", 2); + run = pos + 1; + pos = next_special(run); + } + result.append(s.data() + run, s.size() - run); + return result; } /*! @@ -60,12 +66,43 @@ inline StringType escape(StringType s) * @return unescaped string * * Note the order of escaping "~1" to "/" and "~0" to "~" is important. + * + * Rebuilt in a single pass, see @ref escape. A "~" that is followed by + * neither "0" nor "1" is passed through unchanged; @ref json_pointer rejects + * such input before it gets here. */ template inline void unescape(StringType& s) { - replace_substring(s, StringType{"~1"}, StringType{"/"}); - replace_substring(s, StringType{"~0"}, StringType{"~"}); + auto pos = s.find_first_of('~', 0); + if (pos == StringType::npos) + { + return; + } + + StringType result; + result.reserve(s.size()); + + std::size_t run = 0; + while (pos != StringType::npos) + { + result.append(s.data() + run, pos - run); + + const auto next = pos + 1; + if (next < s.size() && (s[next] == '0' || s[next] == '1')) + { + result.append(s[next] == '0' ? "~" : "/", 1); + run = pos + 2; + } + else + { + result.append("~", 1); + run = pos + 1; + } + pos = s.find_first_of('~', run); + } + result.append(s.data() + run, s.size() - run); + s = result; } } // namespace detail diff --git a/include/nlohmann/json.hpp b/include/nlohmann/json.hpp index ca3cda17a..1aafbf78a 100644 --- a/include/nlohmann/json.hpp +++ b/include/nlohmann/json.hpp @@ -404,6 +404,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec /// @} + // Two template parameter requirements that would otherwise be silently + // violated: neither produces a diagnostic of its own, and both corrupt + // values rather than failing. + + static_assert(sizeof(typename BinaryType::value_type) == 1, + "BinaryType::value_type must be exactly one byte wide, " + "because the binary readers and writers reinterpret the container's storage as raw bytes"); + + static_assert(sizeof(NumberUnsignedType) >= sizeof(NumberIntegerType), + "NumberUnsignedType must be at least as wide as NumberIntegerType, " + "because it has to hold the absolute value of every NumberIntegerType value"); + private: /// helper for exception-safe object creation @@ -784,21 +796,76 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec return it; } - reference set_parent(reference j, std::size_t old_capacity = detail::unknown_size()) + /// @brief erase an element from the object and return the following one + /// Not every map returns an iterator from erase(iterator): some containers + /// (e.g., Abseil's hash maps) return void to avoid computing a successor + /// the caller may not need. Compute it before erasing for those. + template < typename It, detail::enable_if_t < + !detail::erase_returns_void::value, int > = 0 > + typename object_t::iterator erase_from_object(It pos) + { + return m_data.m_value.object->erase(pos); + } + + template < typename It, detail::enable_if_t < + detail::erase_returns_void::value, int > = 0 > + typename object_t::iterator erase_from_object(It pos) + { + auto next = std::next(pos); + m_data.m_value.object->erase(pos); + return next; + } + + /// @brief the capacity of the stored array, or unknown_size() + /// Only JSON_DIAGNOSTICS uses the value, to detect a reallocation that + /// would invalidate the parent pointers. Array types that do not have a + /// capacity() member function report unknown_size(), which is treated as + /// "the elements may have moved". +#if JSON_DIAGNOSTICS + template < typename A = array_t, detail::enable_if_t < detail::has_capacity::value, int > = 0 > + std::size_t array_capacity() const noexcept + { + return m_data.m_value.array->capacity(); + } + + template < typename A = array_t, detail::enable_if_t < !detail::has_capacity::value, int > = 0 > + std::size_t array_capacity() const noexcept + { + return detail::unknown_size(); + } +#else + static constexpr std::size_t array_capacity() noexcept + { + return detail::unknown_size(); + } +#endif + + /// @brief set the parent of a value that has just been added to an array + /// @param j the added value + /// @param old_capacity the value @ref array_capacity() returned before the + /// insertion + reference set_parent_after_array_insert(reference j, std::size_t old_capacity) { #if JSON_DIAGNOSTICS - if (old_capacity != detail::unknown_size()) + // see https://github.com/nlohmann/json/issues/2838 + JSON_ASSERT(type() == value_t::array); + if (JSON_HEDLEY_UNLIKELY(old_capacity == detail::unknown_size() + || array_capacity() != old_capacity)) { - // see https://github.com/nlohmann/json/issues/2838 - JSON_ASSERT(type() == value_t::array); - if (JSON_HEDLEY_UNLIKELY(m_data.m_value.array->capacity() != old_capacity)) - { - // capacity has changed: update all parents - set_parents(); - return j; - } + // the capacity has changed, or the array type does not let us tell: + // the elements may have moved, so update all parents + set_parents(); + return j; } +#else + static_cast(old_capacity); +#endif + return set_parent(j); + } + reference set_parent(reference j) + { +#if JSON_DIAGNOSTICS // ordered_json uses a vector internally, so pointers could have // been invalidated; see https://github.com/nlohmann/json/issues/2962 #ifdef JSON_HEDLEY_MSVC_VERSION @@ -817,7 +884,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec j.m_parent = this; #else static_cast(j); - static_cast(old_capacity); #endif return j; } @@ -2029,22 +2095,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec reference at(size_type idx) { // at only works for arrays - if (JSON_HEDLEY_LIKELY(is_array())) - { - JSON_TRY - { - return set_parent(m_data.m_value.array->at(idx)); - } - JSON_CATCH (std::out_of_range&) - { - // create a better exception explanation - JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); - } // cppcheck-suppress[missingReturn] - } - else + if (JSON_HEDLEY_UNLIKELY(!is_array())) { JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", type_name()), this)); } + + if (JSON_HEDLEY_UNLIKELY(idx >= m_data.m_value.array->size())) + { + JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); + } + + return set_parent((*m_data.m_value.array)[idx]); } /// @brief access specified array element with bounds checking @@ -2052,22 +2113,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec const_reference at(size_type idx) const { // at only works for arrays - if (JSON_HEDLEY_LIKELY(is_array())) - { - JSON_TRY - { - return m_data.m_value.array->at(idx); - } - JSON_CATCH (std::out_of_range&) - { - // create a better exception explanation - JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); - } // cppcheck-suppress[missingReturn] - } - else + if (JSON_HEDLEY_UNLIKELY(!is_array())) { JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", type_name()), this)); } + + if (JSON_HEDLEY_UNLIKELY(idx >= m_data.m_value.array->size())) + { + JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); + } + + return (*m_data.m_value.array)[idx]; } /// @brief access specified object element with bounds checking @@ -2167,12 +2223,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec #if JSON_DIAGNOSTICS // remember array size & capacity before resizing const auto old_size = m_data.m_value.array->size(); - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); #endif m_data.m_value.array->resize(idx + 1); #if JSON_DIAGNOSTICS - if (JSON_HEDLEY_UNLIKELY(m_data.m_value.array->capacity() != old_capacity)) + if (JSON_HEDLEY_UNLIKELY(old_capacity == detail::unknown_size() + || array_capacity() != old_capacity)) { // capacity has changed: update all parents set_parents(); @@ -2563,7 +2620,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec case value_t::object: { - result.m_it.object_iterator = m_data.m_value.object->erase(pos.m_it.object_iterator); + result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator); break; } @@ -3202,9 +3259,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec } // add the element to the array (move semantics) - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); m_data.m_value.array->push_back(std::move(val)); - set_parent(m_data.m_value.array->back(), old_capacity); + set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity); // if val is moved from, basic_json move constructor marks it null, so we do not call the destructor } @@ -3235,9 +3292,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec } // add the element to the array - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); m_data.m_value.array->push_back(val); - set_parent(m_data.m_value.array->back(), old_capacity); + set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity); } /// @brief add an object to an array @@ -3323,9 +3380,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec } // add the element to the array (perfect forwarding) - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); m_data.m_value.array->emplace_back(std::forward(args)...); - return set_parent(m_data.m_value.array->back(), old_capacity); + return set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity); } /// @brief add an object to an object if key does not exist @@ -3404,7 +3461,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec /// @sa https://json.nlohmann.me/api/basic_json/insert/ iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param) { - return insert(pos, val); + return insert(std::move(pos), val); } /// @brief inserts copies of element into array diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index d4972a35e..35443e141 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -3362,6 +3362,8 @@ NLOHMANN_JSON_NAMESPACE_END +#include // size_t + // #include @@ -3369,44 +3371,48 @@ NLOHMANN_JSON_NAMESPACE_BEGIN namespace detail { -/*! -@brief replace all occurrences of a substring by another string - -@param[in,out] s the string to manipulate; changed so that all - occurrences of @a f are replaced with @a t -@param[in] f the substring to replace with @a t -@param[in] t the string to replace @a f - -@pre The search string @a f must not be empty. **This precondition is -enforced with an assertion.** - -@since version 2.0.0 -*/ -template -inline void replace_substring(StringType& s, const StringType& f, - const StringType& t) -{ - JSON_ASSERT(!f.empty()); - for (auto pos = s.find(f); // find the first occurrence of f - pos != StringType::npos; // make sure f was found - s.replace(pos, f.size(), t), // replace with t, and - pos = s.find(f, pos + t.size())) // find the next occurrence of f - {} -} - /*! * @brief string escaping as described in RFC 6901 (Sect. 4) * @param[in] s string to escape * @return escaped string * * Note the order of escaping "~" to "~0" and "/" to "~1" is important. + * + * The string is rebuilt in a single pass, appending whole runs between the + * characters that need escaping. Scanning with find_first_of() keeps the + * common case -- nothing to escape -- as fast as a single search, while + * repeated replace() calls would move the tail of the string once per + * escaped character. */ template -inline StringType escape(StringType s) +inline StringType escape(const StringType& s) { - replace_substring(s, StringType{"~"}, StringType{"~0"}); - replace_substring(s, StringType{"/"}, StringType{"~1"}); - return s; + auto next_special = [&s](std::size_t from) + { + const auto tilde = s.find_first_of('~', from); + const auto slash = s.find_first_of('/', from); + return tilde < slash ? tilde : slash; // npos is the largest value + }; + + auto pos = next_special(0); + if (pos == StringType::npos) + { + return s; + } + + StringType result; + result.reserve(s.size() + 2); + + std::size_t run = 0; + while (pos != StringType::npos) + { + result.append(s.data() + run, pos - run); + result.append(s[pos] == '~' ? "~0" : "~1", 2); + run = pos + 1; + pos = next_special(run); + } + result.append(s.data() + run, s.size() - run); + return result; } /*! @@ -3415,12 +3421,43 @@ inline StringType escape(StringType s) * @return unescaped string * * Note the order of escaping "~1" to "/" and "~0" to "~" is important. + * + * Rebuilt in a single pass, see @ref escape. A "~" that is followed by + * neither "0" nor "1" is passed through unchanged; @ref json_pointer rejects + * such input before it gets here. */ template inline void unescape(StringType& s) { - replace_substring(s, StringType{"~1"}, StringType{"/"}); - replace_substring(s, StringType{"~0"}, StringType{"~"}); + auto pos = s.find_first_of('~', 0); + if (pos == StringType::npos) + { + return; + } + + StringType result; + result.reserve(s.size()); + + std::size_t run = 0; + while (pos != StringType::npos) + { + result.append(s.data() + run, pos - run); + + const auto next = pos + 1; + if (next < s.size() && (s[next] == '0' || s[next] == '1')) + { + result.append(s[next] == '0' ? "~" : "/", 1); + run = pos + 2; + } + else + { + result.append("~", 1); + run = pos + 1; + } + pos = s.find_first_of('~', run); + } + result.append(s.data() + run, s.size() - run); + s = result; } } // namespace detail @@ -4000,17 +4037,18 @@ struct has_to_json < BasicJsonType, T, enable_if_t < !is_basic_json::value >> template using detect_key_compare = typename T::key_compare; -template -struct has_key_compare : std::integral_constant::value> {}; - -// obtains the actual object key comparator +// obtains the actual object key comparator: object_t::key_compare if the +// object type defines it, and default_object_comparator_t otherwise +// +// note detected_or_t is used rather than std::conditional, because the latter +// names both of its type arguments eagerly; object_t::key_compare would then +// be a hard error for an object type that does not define it template struct actual_object_comparator { using object_t = typename BasicJsonType::object_t; using object_comparator_t = typename BasicJsonType::default_object_comparator_t; - using type = typename std::conditional < has_key_compare::value, - typename object_t::key_compare, object_comparator_t>::type; + using type = detected_or_t; }; template @@ -4606,6 +4644,22 @@ using has_erase_with_key_type = typename std::conditional < std::true_type, std::false_type >::type; +template +using detect_erase_with_iterator = decltype(std::declval().erase(std::declval())); + +// type trait to check if erase(iterator) returns void instead of the following +// iterator, as the object types that do not compute a successor the caller may +// not need do +template +using erase_returns_void = is_detected_exact; + +template +using detect_capacity = decltype(std::declval().capacity()); + +// type trait to check if a type has a capacity() member function +template +struct has_capacity : std::integral_constant::value> {}; + // a naive helper to check if a type is an ordered_map (exploits the fact that // ordered_map inherits capacity() from std::vector) template @@ -5013,7 +5067,10 @@ class exception : public std::exception { if (&element.second == current) { - tokens.emplace_back(element.first.c_str()); + // data() is null-terminated, so a key containing + // a null byte is cut short here rather than + // truncating the whole message at what() + tokens.emplace_back(element.first.data()); break; } } @@ -7032,7 +7089,9 @@ std::size_t hash(const BasicJsonType& j) seed = combine(seed, static_cast(j.get_binary().subtype())); for (const auto byte : j.get_binary()) { - seed = combine(seed, std::hash {}(byte)); + // the cast is needed for binary types whose value type is not + // an integer (e.g., std::byte) + seed = combine(seed, std::hash {}(static_cast(byte))); } return seed; } @@ -15131,7 +15190,10 @@ class binary_reader number_string, out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr)); } - return sax->number_float(parsed_float, std::move(number_string)); + // number_string is a std::string, while the SAX interface takes a + // string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + return sax->number_float(parsed_float, string_t(number_string.data(), number_string.size())); } case token_type::uninitialized: case token_type::literal_true: @@ -16325,8 +16387,13 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci iter_impl() = default; ~iter_impl() = default; - iter_impl(iter_impl&&) noexcept = default; - iter_impl& operator=(iter_impl&&) noexcept = default; + // the exception specification is left to be computed rather than declared: + // an array or object type whose iterator is not nothrow move constructible + // (std::deque's is not before libstdc++ 11) would make a declared noexcept + // differ from the implicit one, which deletes the function -- and is an + // error outright with older compilers + iter_impl(iter_impl&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor,cppcoreguidelines-noexcept-move-operations) + iter_impl& operator=(iter_impl&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor,cppcoreguidelines-noexcept-move-operations) /*! @brief constructor for a given JSON instance @@ -17207,6 +17274,7 @@ NLOHMANN_JSON_NAMESPACE_END #endif // JSON_NO_IO #include // max #include // accumulate +#include // set #include // string #include // move #include // vector @@ -17266,7 +17334,7 @@ class json_pointer string_t{}, [](const string_t& a, const string_t& b) { - return detail::concat(a, '/', detail::escape(b)); + return detail::concat(a, '/', detail::escape(b)); }); } @@ -17460,7 +17528,7 @@ class json_pointer JSON_THROW(detail::parse_error::create(109, 0, detail::concat("array index '", s, "' is not a number"), nullptr)); } - const char* p = s.c_str(); + const char* p = s.data(); char* p_end = nullptr; // NOLINT(misc-const-correctness) errno = 0; // strtoull doesn't reset errno const unsigned long long res = std::strtoull(p, &p_end, 10); // NOLINT(runtime/int) @@ -17495,19 +17563,35 @@ class json_pointer } private: + /*! + @brief the reference token sequences that denote arrays + + @ref unflatten collects the pointer prefixes that have a reference token 0 + among their children; @ref get_and_create creates arrays exactly below + those prefixes and objects everywhere else. Deciding this up front keeps + the result independent of the order in which the flattened object is + iterated, which is unspecified for some object types. + */ + using array_parents_t = std::set>; + /*! @brief create and return a reference to the pointed to value @complexity Linear in the number of reference tokens. + @throw parse_error.106 if an array index begins with '0' @throw parse_error.109 if array index is not a number @throw type_error.313 if value cannot be unflattened */ template - BasicJsonType& get_and_create(BasicJsonType& j) const + BasicJsonType& get_and_create(BasicJsonType& j, const array_parents_t& array_parents) const { auto* result = &j; + // the reference tokens that have been consumed so far; used to look up + // whether the value to be created below is an array or an object + std::vector prefix; + // in case no reference tokens exist, return a reference to the JSON value // j which will be overwritten by a primitive value for (const auto& reference_token : reference_tokens) @@ -17516,10 +17600,11 @@ class json_pointer { case detail::value_t::null: { - if (reference_token == "0") + if (array_parents.find(prefix) != array_parents.end()) { - // start a new array if the reference token is 0 - result = &result->operator[](0); + // some reference token below this position is 0, so the + // value is an array + result = &result->operator[](array_index(reference_token)); } else { @@ -17559,6 +17644,8 @@ class json_pointer default: JSON_THROW(detail::type_error::create(313, "invalid value to unflatten", &j)); } + + prefix.push_back(reference_token); } return *result; @@ -18032,7 +18119,8 @@ class json_pointer { // use the text between the beginning of the reference token // (start) and the last slash (slash). - auto reference_token = reference_string.substr(start, slash - start); + const auto count = (slash == string_t::npos ? reference_string.size() : slash) - start; + auto reference_token = string_t(reference_string.data() + start, count); // check reference tokens are properly escaped for (std::size_t pos = reference_token.find_first_of('~'); @@ -18148,6 +18236,24 @@ class json_pointer BasicJsonType result; + // collect the pointer prefixes that have a reference token 0 among + // their children; the values below them are arrays, all others are + // objects (see array_parents_t) + array_parents_t array_parents; + for (const auto& element : *value.m_data.m_value.object) + { + json_pointer ptr(element.first); + std::vector prefix; + for (auto& reference_token : ptr.reference_tokens) + { + if (reference_token == "0") + { + array_parents.insert(prefix); + } + prefix.push_back(std::move(reference_token)); + } + } + // iterate the JSON object values for (const auto& element : *value.m_data.m_value.object) { @@ -18160,7 +18266,7 @@ class json_pointer // that if the JSON pointer is "" (i.e., points to the whole value), // function get_and_create returns a reference to the result itself. // An assignment will then create a primitive value. - json_pointer(element.first).get_and_create(result) = element.second; + json_pointer(element.first).get_and_create(result, array_parents) = element.second; } return result; @@ -18832,7 +18938,7 @@ class binary_writer // step 2: write the string oa->write_characters( - reinterpret_cast(j.m_data.m_value.string->c_str()), + reinterpret_cast(j.m_data.m_value.string->data()), j.m_data.m_value.string->size()); break; } @@ -19152,7 +19258,7 @@ class binary_writer // step 2: write the string oa->write_characters( - reinterpret_cast(j.m_data.m_value.string->c_str()), + reinterpret_cast(j.m_data.m_value.string->data()), j.m_data.m_value.string->size()); break; } @@ -19369,7 +19475,7 @@ class binary_writer } write_number_with_ubjson_prefix(j.m_data.m_value.string->size(), true, use_bjdata); oa->write_characters( - reinterpret_cast(j.m_data.m_value.string->c_str()), + reinterpret_cast(j.m_data.m_value.string->data()), j.m_data.m_value.string->size()); break; } @@ -19468,7 +19574,9 @@ class binary_writer for (size_t i = 0; i < j.m_data.m_value.binary->size(); ++i) { oa->write_character(to_char_type(bjdata_draft3 ? 'B' : 'U')); - oa->write_character(to_char_type(j.m_data.m_value.binary->data()[i])); + // the cast is needed for binary types whose value type + // is not an integer (e.g., std::byte) + oa->write_character(to_char_type(static_cast(j.m_data.m_value.binary->data()[i]))); } } @@ -19529,7 +19637,7 @@ class binary_writer { write_number_with_ubjson_prefix(el.first.size(), true, use_bjdata); oa->write_characters( - reinterpret_cast(el.first.c_str()), + reinterpret_cast(el.first.data()), el.first.size()); write_ubjson(el.second, use_count, use_type, prefix_required, use_bjdata, bjdata_version); } @@ -19592,8 +19700,11 @@ class binary_writer { oa->write_character(to_char_type(element_type)); oa->write_characters( - reinterpret_cast(name.c_str()), - name.size() + 1u); + reinterpret_cast(name.data()), + name.size()); + // the terminating null byte is written explicitly rather than taken + // from the buffer, so that string_t::data() need not be null-terminated + oa->write_character(to_char_type(0x00)); } /*! @@ -19634,8 +19745,11 @@ class binary_writer write_number(to_bson_length(value.size() + 1ul), true); oa->write_characters( - reinterpret_cast(value.c_str()), - value.size() + 1); + reinterpret_cast(value.data()), + value.size()); + // the terminating null byte is written explicitly rather than taken + // from the buffer, so that string_t::data() need not be null-terminated + oa->write_character(to_char_type(0x00)); } /*! @@ -19726,7 +19840,11 @@ class binary_writer const std::size_t embedded_document_size = std::accumulate(std::begin(value), std::end(value), static_cast(0), [&array_index](std::size_t result, const typename BasicJsonType::array_t::value_type & el) { - return result + calc_bson_element_size(std::to_string(array_index++), el); + // the index is built as a std::string, while calc_bson_element_size + // takes a string_t; convert explicitly, as the two are only + // implicitly convertible for some string types + const auto key = std::to_string(array_index++); + return result + calc_bson_element_size(string_t(key.data(), key.size()), el); }); return sizeof(std::int32_t) + embedded_document_size + 1ul; @@ -19753,7 +19871,11 @@ class binary_writer for (const auto& el : value) { - write_bson_element(std::to_string(array_index++), el); + // the index is built as a std::string, while write_bson_element takes + // a string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + const auto key = std::to_string(array_index++); + write_bson_element(string_t(key.data(), key.size()), el); } oa->write_character(to_char_type(0x00)); @@ -22782,7 +22904,7 @@ class serializer { case error_handler_t::strict: { - JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", hex_bytes(static_cast(s.back() | 0))), nullptr)); + JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", hex_bytes(static_cast(s[s.size() - 1] | 0))), nullptr)); } case error_handler_t::ignore: @@ -23043,6 +23165,19 @@ class serializer pos += 6; } + /*! + @brief convert a single element of a binary value to its byte value + + The elements of a binary value are dumped as the numbers 0..255, regardless + of the value type of the configured BinaryType: that type may be signed + (`char`), unsigned (`std::uint8_t`), or not an integer at all + (`std::byte`), none of which @ref dump_integer can handle uniformly. + */ + static std::uint8_t to_byte_value(binary_char_t x) noexcept + { + return static_cast(x); + } + // templates to avoid warnings about useless casts template ::value, int> = 0> bool is_negative_number(NumberType x) @@ -23064,8 +23199,10 @@ class serializer an arbitrary number, and the three digits it takes at most are written straight into the write buffer. - Any byte type that is not a plain unsigned byte is left to @ref dump_integer, - whose representation of it may differ. + Any byte type that is not a plain unsigned byte is converted to its + @ref to_byte_value "byte value" and left to @ref dump_integer, so a signed + or non-integral BinaryType::value_type (`char`, `std::byte`, ...) still + dumps as 0..255. */ template void dump_byte(const ByteType value) @@ -23078,7 +23215,7 @@ class serializer template void dump_byte(const ByteType value, std::false_type /*is_plain_byte*/) { - dump_integer(value); + dump_integer(to_byte_value(value)); } template @@ -23124,8 +23261,7 @@ class serializer template < typename NumberType, detail::enable_if_t < std::is_integral::value || std::is_same::value || - std::is_same::value || - std::is_same::value, + std::is_same::value, int > = 0 > void dump_integer(NumberType x) { @@ -24173,6 +24309,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec /// @} + // Two template parameter requirements that would otherwise be silently + // violated: neither produces a diagnostic of its own, and both corrupt + // values rather than failing. + + static_assert(sizeof(typename BinaryType::value_type) == 1, + "BinaryType::value_type must be exactly one byte wide, " + "because the binary readers and writers reinterpret the container's storage as raw bytes"); + + static_assert(sizeof(NumberUnsignedType) >= sizeof(NumberIntegerType), + "NumberUnsignedType must be at least as wide as NumberIntegerType, " + "because it has to hold the absolute value of every NumberIntegerType value"); + private: /// helper for exception-safe object creation @@ -24553,21 +24701,76 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec return it; } - reference set_parent(reference j, std::size_t old_capacity = detail::unknown_size()) + /// @brief erase an element from the object and return the following one + /// Not every map returns an iterator from erase(iterator): some containers + /// (e.g., Abseil's hash maps) return void to avoid computing a successor + /// the caller may not need. Compute it before erasing for those. + template < typename It, detail::enable_if_t < + !detail::erase_returns_void::value, int > = 0 > + typename object_t::iterator erase_from_object(It pos) + { + return m_data.m_value.object->erase(pos); + } + + template < typename It, detail::enable_if_t < + detail::erase_returns_void::value, int > = 0 > + typename object_t::iterator erase_from_object(It pos) + { + auto next = std::next(pos); + m_data.m_value.object->erase(pos); + return next; + } + + /// @brief the capacity of the stored array, or unknown_size() + /// Only JSON_DIAGNOSTICS uses the value, to detect a reallocation that + /// would invalidate the parent pointers. Array types that do not have a + /// capacity() member function report unknown_size(), which is treated as + /// "the elements may have moved". +#if JSON_DIAGNOSTICS + template < typename A = array_t, detail::enable_if_t < detail::has_capacity::value, int > = 0 > + std::size_t array_capacity() const noexcept + { + return m_data.m_value.array->capacity(); + } + + template < typename A = array_t, detail::enable_if_t < !detail::has_capacity::value, int > = 0 > + std::size_t array_capacity() const noexcept + { + return detail::unknown_size(); + } +#else + static constexpr std::size_t array_capacity() noexcept + { + return detail::unknown_size(); + } +#endif + + /// @brief set the parent of a value that has just been added to an array + /// @param j the added value + /// @param old_capacity the value @ref array_capacity() returned before the + /// insertion + reference set_parent_after_array_insert(reference j, std::size_t old_capacity) { #if JSON_DIAGNOSTICS - if (old_capacity != detail::unknown_size()) + // see https://github.com/nlohmann/json/issues/2838 + JSON_ASSERT(type() == value_t::array); + if (JSON_HEDLEY_UNLIKELY(old_capacity == detail::unknown_size() + || array_capacity() != old_capacity)) { - // see https://github.com/nlohmann/json/issues/2838 - JSON_ASSERT(type() == value_t::array); - if (JSON_HEDLEY_UNLIKELY(m_data.m_value.array->capacity() != old_capacity)) - { - // capacity has changed: update all parents - set_parents(); - return j; - } + // the capacity has changed, or the array type does not let us tell: + // the elements may have moved, so update all parents + set_parents(); + return j; } +#else + static_cast(old_capacity); +#endif + return set_parent(j); + } + reference set_parent(reference j) + { +#if JSON_DIAGNOSTICS // ordered_json uses a vector internally, so pointers could have // been invalidated; see https://github.com/nlohmann/json/issues/2962 #ifdef JSON_HEDLEY_MSVC_VERSION @@ -24586,7 +24789,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec j.m_parent = this; #else static_cast(j); - static_cast(old_capacity); #endif return j; } @@ -25798,22 +26000,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec reference at(size_type idx) { // at only works for arrays - if (JSON_HEDLEY_LIKELY(is_array())) - { - JSON_TRY - { - return set_parent(m_data.m_value.array->at(idx)); - } - JSON_CATCH (std::out_of_range&) - { - // create a better exception explanation - JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); - } // cppcheck-suppress[missingReturn] - } - else + if (JSON_HEDLEY_UNLIKELY(!is_array())) { JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", type_name()), this)); } + + if (JSON_HEDLEY_UNLIKELY(idx >= m_data.m_value.array->size())) + { + JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); + } + + return set_parent((*m_data.m_value.array)[idx]); } /// @brief access specified array element with bounds checking @@ -25821,22 +26018,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec const_reference at(size_type idx) const { // at only works for arrays - if (JSON_HEDLEY_LIKELY(is_array())) - { - JSON_TRY - { - return m_data.m_value.array->at(idx); - } - JSON_CATCH (std::out_of_range&) - { - // create a better exception explanation - JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); - } // cppcheck-suppress[missingReturn] - } - else + if (JSON_HEDLEY_UNLIKELY(!is_array())) { JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", type_name()), this)); } + + if (JSON_HEDLEY_UNLIKELY(idx >= m_data.m_value.array->size())) + { + JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this)); + } + + return (*m_data.m_value.array)[idx]; } /// @brief access specified object element with bounds checking @@ -25936,12 +26128,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec #if JSON_DIAGNOSTICS // remember array size & capacity before resizing const auto old_size = m_data.m_value.array->size(); - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); #endif m_data.m_value.array->resize(idx + 1); #if JSON_DIAGNOSTICS - if (JSON_HEDLEY_UNLIKELY(m_data.m_value.array->capacity() != old_capacity)) + if (JSON_HEDLEY_UNLIKELY(old_capacity == detail::unknown_size() + || array_capacity() != old_capacity)) { // capacity has changed: update all parents set_parents(); @@ -26332,7 +26525,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec case value_t::object: { - result.m_it.object_iterator = m_data.m_value.object->erase(pos.m_it.object_iterator); + result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator); break; } @@ -26971,9 +27164,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec } // add the element to the array (move semantics) - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); m_data.m_value.array->push_back(std::move(val)); - set_parent(m_data.m_value.array->back(), old_capacity); + set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity); // if val is moved from, basic_json move constructor marks it null, so we do not call the destructor } @@ -27004,9 +27197,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec } // add the element to the array - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); m_data.m_value.array->push_back(val); - set_parent(m_data.m_value.array->back(), old_capacity); + set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity); } /// @brief add an object to an array @@ -27092,9 +27285,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec } // add the element to the array (perfect forwarding) - const auto old_capacity = m_data.m_value.array->capacity(); + const auto old_capacity = array_capacity(); m_data.m_value.array->emplace_back(std::forward(args)...); - return set_parent(m_data.m_value.array->back(), old_capacity); + return set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity); } /// @brief add an object to an object if key does not exist @@ -27173,7 +27366,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec /// @sa https://json.nlohmann.me/api/basic_json/insert/ iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param) { - return insert(pos, val); + return insert(std::move(pos), val); } /// @brief inserts copies of element into array diff --git a/tests/src/unit-alt-string.cpp b/tests/src/unit-alt-string.cpp index 46e062c6e..ec2ac146b 100644 --- a/tests/src/unit-alt-string.cpp +++ b/tests/src/unit-alt-string.cpp @@ -11,8 +11,10 @@ #include +#include #include #include +#include /* forward declarations */ class alt_string; @@ -22,6 +24,10 @@ void int_to_string(alt_string& target, std::size_t value); // NOLINT(misc-use-in /* * This is virtually a string class. * It covers std::string under the hood. + * + * It deliberately does not provide c_str(), back(), find(str, pos), replace(), + * or substr(): the library must not rely on them. Do not add members here + * without checking that the library actually needs them. */ class alt_string { @@ -106,11 +112,6 @@ class alt_string return str_impl < op.str_impl; } - const char* c_str() const - { - return str_impl.c_str(); - } - char& operator[](std::size_t index) { return str_impl[index]; @@ -121,16 +122,6 @@ class alt_string return str_impl[index]; } - char& back() - { - return str_impl.back(); - } - - const char& back() const - { - return str_impl.back(); - } - void clear() { str_impl.clear(); @@ -146,28 +137,11 @@ class alt_string return str_impl.empty(); } - std::size_t find(const alt_string& str, std::size_t pos = 0) const - { - return str_impl.find(str.str_impl, pos); - } - std::size_t find_first_of(char c, std::size_t pos = 0) const { return str_impl.find_first_of(c, pos); } - alt_string substr(std::size_t pos = 0, std::size_t count = npos) const - { - const std::string s = str_impl.substr(pos, count); - return {s.data(), s.size()}; - } - - alt_string& replace(std::size_t pos, std::size_t count, const alt_string& str) - { - str_impl.replace(pos, count, str.str_impl); - return *this; - } - void reserve( std::size_t new_cap = 0 ) { str_impl.reserve(new_cap); @@ -202,6 +176,31 @@ bool operator<(const char* op1, const alt_string& op2) noexcept TEST_CASE("alternative string type") { + SECTION("binary formats") + { + alt_json doc; + doc["pi"] = 3.141; + doc["happy"] = true; + doc["list"] = {1, 2, 3}; + + CHECK(alt_json::from_cbor(alt_json::to_cbor(doc)) == doc); + CHECK(alt_json::from_msgpack(alt_json::to_msgpack(doc)) == doc); + // BSON is not covered: it additionally needs string_t::find(value_type), + // which alt_string does not provide + CHECK(alt_json::from_ubjson(alt_json::to_ubjson(doc)) == doc); + + // a UBJSON high-precision number is parsed into a std::string that the + // reader has to hand to the SAX interface as an alt_string + const std::vector high_precision = + { + 'H', 'i', 0x16, '3', '.', '1', '4', '1', '5', '9', '2', '6', '5', '3', + '5', '8', '9', '7', '9', '3', '2', '3', '8', '4', '6' + }; + const auto number = alt_json::from_ubjson(high_precision); + CHECK(number.is_number_float()); + CHECK(number.get() == doctest::Approx(3.14159265358979323846)); + } + SECTION("dump") { { @@ -332,6 +331,15 @@ TEST_CASE("alternative string type") CHECK(j.at(alt_json::json_pointer("/foo/0")) == j["foo"][0]); CHECK(j.at(alt_json::json_pointer("/foo/1")) == j["foo"][1]); + + // RFC 6901 escaping works without string_t::find(str, pos), replace(), + // and substr() + auto j2 = alt_json::parse(R"({"a/b": 1, "m~n": 2, "~/~~//": 3})"); + CHECK(j2.at(alt_json::json_pointer("/a~1b")) == 1); + CHECK(j2.at(alt_json::json_pointer("/m~0n")) == 2); + CHECK(j2.at(alt_json::json_pointer("/~0~1~0~0~1~1")) == 3); + CHECK(alt_json::json_pointer("/~0~1~0~0~1~1").to_string() == alt_string("/~0~1~0~0~1~1")); + CHECK(j2.flatten().unflatten() == j2); } SECTION("patch") diff --git a/tests/src/unit-custom-array-type.cpp b/tests/src/unit-custom-array-type.cpp new file mode 100644 index 000000000..00606c6e0 --- /dev/null +++ b/tests/src/unit-custom-array-type.cpp @@ -0,0 +1,150 @@ +// __ _____ _____ _____ +// __| | __| | | | JSON for Modern C++ (supporting code) +// | | |__ | | | | | | version 3.12.0 +// |_____|_____|_____|_|___| https://github.com/nlohmann/json +// +// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann +// SPDX-License-Identifier: MIT + +#include "doctest_compatibility.h" + +#include + +#include +#include +#include +#include +#include +#include + +namespace +{ + +// std::deque has no capacity() member function, which the library only needs +// to detect a reallocation for JSON_DIAGNOSTICS +using deque_json = nlohmann::basic_json; + +// a std::vector whose at() is hidden: the library performs its own bounds +// check and must not fall back to the container's checked accessor +template> +class vector_without_at : public std::vector +{ + public: + vector_without_at() = default; + + // the array of an initializer list is built from a range + template + vector_without_at(InputIt first, InputIt last) : std::vector(first, last) {} + + void at() = delete; +}; + +using no_at_json = nlohmann::basic_json; + +} // namespace + +TEST_CASE("array type without capacity()") +{ + SECTION("the iterators take their exception specification from the container") + { + // basic_json's iterators move exactly as the container iterators do: + // their move operations are defaulted without a declared noexcept, + // because an array or object type whose iterator is not nothrow move + // constructible would otherwise have them deleted (std::deque's is not + // with libstdc++ before 11, and neither are MSVC's debug iterators) + CHECK(std::is_nothrow_move_constructible::value == + (std::is_nothrow_move_constructible::value + && std::is_nothrow_move_constructible::value)); + CHECK(std::is_nothrow_move_assignable::value == + (std::is_nothrow_move_assignable::value + && std::is_nothrow_move_assignable::value)); + CHECK(std::is_nothrow_move_constructible::value == + (std::is_nothrow_move_constructible::value + && std::is_nothrow_move_constructible::value)); + + // and they are movable at all, which is what dropping the declared + // noexcept buys for a std::deque array + CHECK(std::is_move_constructible::value); + CHECK(std::is_move_assignable::value); + } + + SECTION("adding elements") + { + deque_json j = deque_json::array(); + j.push_back(1); + j.push_back("two"); + j.emplace_back(3); + j += 4; + + CHECK(j.size() == 4); + CHECK(j == deque_json({1, "two", 3, 4})); + CHECK(j.back() == 4); + CHECK(j.front() == 1); + } + + SECTION("accessing and modifying elements") + { + auto j = deque_json::parse(R"([1,2,3])"); + + CHECK(j[1] == 2); + CHECK(j.at(2) == 3); + + // growing through operator[] fills up with null values + j[5] = 6; + CHECK(j.size() == 6); + CHECK(j[4].is_null()); + CHECK(j[5] == 6); + + j.erase(0); + CHECK(j == deque_json({2, 3, nullptr, nullptr, 6})); + + auto it = j.erase(j.begin()); + CHECK(*it == 3); + + j.insert(j.begin(), 1); + CHECK(j.front() == 1); + } + + SECTION("serialization and deserialization") + { + const auto j = deque_json::parse(R"({"a":[1,[2,3]],"b":[]})"); + CHECK(j.dump() == R"({"a":[1,[2,3]],"b":[]})"); + CHECK(deque_json::parse(j.dump()) == j); + CHECK(deque_json::from_cbor(deque_json::to_cbor(j)) == j); + + // empty containers are flattened to null and cannot be restored + const auto nested = deque_json::parse(R"({"a":[1,[2,3]]})"); + CHECK(nested.flatten().unflatten() == nested); + } + + SECTION("references stay valid while the array grows") + { + deque_json j = deque_json::array(); + j.push_back(1); + auto& first = j[0]; + for (int i = 0; i < 100; ++i) + { + j.push_back(i); + } + CHECK(&first == &j[0]); + CHECK(first == 1); + } +} + +TEST_CASE("array type without at()") +{ + // built in memory rather than parsed, so that the exception message does + // not gain a byte range with JSON_DIAGNOSTIC_POSITIONS + no_at_json j = {1, 2, 3}; + const auto& jc = j; + + CHECK(j.at(0) == 1); + CHECK(j.at(2) == 3); + CHECK(jc.at(2) == 3); + + CHECK_THROWS_WITH_AS(j.at(3), "[json.exception.out_of_range.401] array index 3 is out of range", no_at_json::out_of_range); + CHECK_THROWS_WITH_AS(jc.at(3), "[json.exception.out_of_range.401] array index 3 is out of range", no_at_json::out_of_range); + + CHECK(j.at(no_at_json::json_pointer("/1")) == 2); + CHECK_THROWS_AS(j.at(no_at_json::json_pointer("/3")), no_at_json::out_of_range); +} diff --git a/tests/src/unit-custom-binary-type.cpp b/tests/src/unit-custom-binary-type.cpp new file mode 100644 index 000000000..d357ec9a3 --- /dev/null +++ b/tests/src/unit-custom-binary-type.cpp @@ -0,0 +1,79 @@ +// __ _____ _____ _____ +// __| | __| | | | JSON for Modern C++ (supporting code) +// | | |__ | | | | | | version 3.12.0 +// |_____|_____|_____|_|___| https://github.com/nlohmann/json +// +// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann +// SPDX-License-Identifier: MIT + +#include "doctest_compatibility.h" + +#include + +#include +#include +#include +#include +#include +#include + +#ifdef JSON_HAS_CPP_17 + #include +#endif + +namespace +{ + +// a BinaryType whose value type is signed: the elements must still be +// processed as the numbers 0..255 +using char_binary_json = nlohmann::basic_json < + std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, + double, std::allocator, nlohmann::adl_serializer, std::vector, void >; + +#ifdef JSON_HAS_CPP_17 + // a BinaryType whose value type is not an integer type at all + using byte_binary_json = nlohmann::basic_json < + std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, + double, std::allocator, nlohmann::adl_serializer, std::vector, void >; +#endif + +} // namespace + +TEST_CASE("binary type whose value type is not std::uint8_t") +{ + SECTION("a signed value type does not dump negative numbers") + { + const std::vector chars{'\0', '\x01', '\xFF'}; + CHECK(char_binary_json::binary(chars).dump() == R"({"bytes":[0,1,255],"subtype":null})"); + CHECK(char_binary_json::binary(chars, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})"); + CHECK(char_binary_json::binary({}).dump() == R"({"bytes":[],"subtype":null})"); + } + + SECTION("the default binary type is unchanged") + { + CHECK(nlohmann::json::binary({0, 1, 255}, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})"); + } + +#ifdef JSON_HAS_CPP_17 + SECTION("dumping a value type that is not an integer") + { + const std::vector bytes{std::byte{0}, std::byte{1}, std::byte{0xFF}}; + CHECK(byte_binary_json::binary(bytes).dump() == R"({"bytes":[0,1,255],"subtype":null})"); + CHECK(byte_binary_json::binary(bytes, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})"); + CHECK(byte_binary_json::binary({}).dump() == R"({"bytes":[],"subtype":null})"); + } + + SECTION("hashing and the binary formats") + { + const std::vector bytes{std::byte{0}, std::byte{1}, std::byte{0xFF}}; + const auto j = byte_binary_json::binary(bytes); + + CHECK(std::hash {}(j) == std::hash {}(j)); + CHECK(byte_binary_json::from_cbor(byte_binary_json::to_cbor(j)) == j); + CHECK(byte_binary_json::from_msgpack(byte_binary_json::to_msgpack(j)) == j); + + // UBJSON has no binary type, so binary values are written as an array + CHECK(byte_binary_json::from_ubjson(byte_binary_json::to_ubjson(j)) == byte_binary_json({0, 1, 255})); + } +#endif +} diff --git a/tests/src/unit-custom-object-type.cpp b/tests/src/unit-custom-object-type.cpp new file mode 100644 index 000000000..cb2cb5ff3 --- /dev/null +++ b/tests/src/unit-custom-object-type.cpp @@ -0,0 +1,323 @@ +// __ _____ _____ _____ +// __| | __| | | | JSON for Modern C++ (supporting code) +// | | |__ | | | | | | version 3.12.0 +// |_____|_____|_____|_|___| https://github.com/nlohmann/json +// +// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann +// SPDX-License-Identifier: MIT + +#include "doctest_compatibility.h" + +#include + +#include +#include +#include +#include +#include +#include + + +namespace +{ + +// An ObjectType that does *not* define a key_compare member type, which is +// what every hash map looks like to the library. +// +// A hash map is deliberately not used here: object_t is probed for +// key_compare inside the definition of basic_json, that is, while basic_json +// is still an incomplete type, and whether a hash map can be instantiated +// with an incomplete mapped type depends on the standard library (libstdc++ 9 +// needs the size of the mapped type for its node type and rejects it). So the +// object type wraps a std::map instead of inheriting from it: an earlier +// version derived from std::map and shadowed the inherited key_compare type +// with a same-named member function, relying on ordinary member hiding to +// make key_compare unreachable as a type. MSVC 2017 (AppVeyor, /std:c++17) +// does not honor that hiding for a typename-qualified lookup performed from +// outside the class and still resolves key_compare to the base's comparator +// type, so the library's probe incorrectly found one. Composition sidesteps +// the question entirely: with no base class, there is no key_compare to find +// under any lookup rule. +template +class no_key_compare_map +{ + using map_t = std::map; + map_t data; + + public: + using key_type = typename map_t::key_type; + using mapped_type = typename map_t::mapped_type; + using value_type = typename map_t::value_type; + using size_type = typename map_t::size_type; + using allocator_type = typename map_t::allocator_type; + using iterator = typename map_t::iterator; + using const_iterator = typename map_t::const_iterator; + + // -Weffc++ asks for the member to be initialized in the member + // initialization list, which a defaulted constructor does not do; the + // exception specification a defaulted one would have carried has to be + // written out as well, or -Wnoexcept objects where the standard library + // takes noexcept(construct(...)) + no_key_compare_map() noexcept(std::is_nothrow_default_constructible::value) : data() {} + + // converting between two basic_json types builds the object from a range + template + no_key_compare_map(InputIt first, InputIt last) : data(first, last) {} + + iterator begin() noexcept + { + return data.begin(); + } + iterator end() noexcept + { + return data.end(); + } + const_iterator begin() const noexcept + { + return data.begin(); + } + const_iterator end() const noexcept + { + return data.end(); + } + const_iterator cbegin() const noexcept + { + return data.cbegin(); + } + const_iterator cend() const noexcept + { + return data.cend(); + } + + bool empty() const noexcept + { + return data.empty(); + } + size_type size() const noexcept + { + return data.size(); + } + size_type max_size() const noexcept + { + return data.max_size(); + } + void clear() noexcept + { + data.clear(); + } + + iterator find(const key_type& key) + { + return data.find(key); + } + const_iterator find(const key_type& key) const + { + return data.find(key); + } + size_type count(const key_type& key) const + { + return data.count(key); + } + + std::pair emplace(const key_type& key, const mapped_type& value) + { + return data.emplace(key, value); + } + + std::pair insert(const value_type& value) + { + return data.insert(value); + } + + template + void insert(InputIt first, InputIt last) + { + data.insert(first, last); + } + + mapped_type& operator[](const key_type& key) + { + return data[key]; + } + + mapped_type& at(const key_type& key) + { + return data.at(key); + } + const mapped_type& at(const key_type& key) const + { + return data.at(key); + } + + iterator erase(iterator pos) + { + return data.erase(pos); + } + iterator erase(iterator first, iterator last) + { + return data.erase(first, last); + } + size_type erase(const key_type& key) + { + return data.erase(key); + } + + void swap(no_key_compare_map& other) noexcept(noexcept(data.swap(other.data))) + { + data.swap(other.data); + } + + friend bool operator==(const no_key_compare_map& lhs, const no_key_compare_map& rhs) + { + return lhs.data == rhs.data; + } + friend bool operator<(const no_key_compare_map& lhs, const no_key_compare_map& rhs) + { + return lhs.data < rhs.data; + } +}; + +using no_key_compare_json = nlohmann::basic_json; + +// An ObjectType whose erase(iterator) returns void rather than the following +// iterator, as for instance Abseil's hash maps do +template +struct void_erase_map : std::map +{ + using base_t = std::map; + using iterator = typename base_t::iterator; + using base_t::erase; + + void erase(iterator pos) + { + base_t::erase(pos); + } +}; + +using void_erase_json = nlohmann::basic_json; + +} // namespace + +TEST_CASE("object type whose erase() returns void") +{ + SECTION("erasing every element through the returned iterator") + { + void_erase_json j; + for (int i = 0; i < 8; ++i) + { + j["k" + std::to_string(i)] = i; + } + + std::size_t erased = 0; + for (auto it = j.begin(); it != j.end(); ++erased) + { + it = j.erase(it); + } + CHECK(erased == 8); + CHECK(j.empty()); + } + + SECTION("erasing in the middle returns the following element") + { + void_erase_json j; + for (int i = 0; i < 4; ++i) + { + j["k" + std::to_string(i)] = i; + } + + auto it = j.begin(); + ++it; + const auto after = j.erase(it); + CHECK(j.size() == 3); + CHECK(after.key() == "k2"); + CHECK(after.value() == 2); + CHECK(!j.contains("k1")); + } + + SECTION("the other erase overloads are unaffected") + { + void_erase_json j; + j["a"] = 1; + j["b"] = 2; + j["c"] = 3; + + CHECK(j.erase("a") == 1); + CHECK(j.erase("nope") == 0); + j.erase(j.begin(), j.end()); + CHECK(j.empty()); + } +} + +TEST_CASE("object type without key_compare") +{ + SECTION("object_comparator_t falls back to default_object_comparator_t") + { + CHECK(std::is_same < no_key_compare_json::object_comparator_t, + no_key_compare_json::default_object_comparator_t >::value); + } + + SECTION("object types defining key_compare are unaffected") + { + CHECK(std::is_same::value); + CHECK(std::is_same::value); + } + + SECTION("creating and accessing values") + { + no_key_compare_json j; + j["one"] = 1; + j["two"] = "zwei"; + j["three"]["nested"] = true; + + CHECK(j.size() == 3); + CHECK(j.at("one") == 1); + CHECK(j["two"] == "zwei"); + CHECK(j["three"]["nested"] == true); + CHECK(j.contains("one")); + CHECK(!j.contains("four")); + CHECK(j.find("one") != j.end()); + CHECK(j.count("one") == 1); + CHECK(j.erase("one") == 1); + CHECK(j.size() == 2); + } + + SECTION("serialization and deserialization") + { + const auto j = no_key_compare_json::parse(R"({"a":[1,2,3],"b":{"c":null}})"); + CHECK(j["a"].size() == 3); + CHECK(j["a"][2] == 3); + CHECK(j["b"]["c"].is_null()); + CHECK(no_key_compare_json::parse(j.dump()) == j); + } + + SECTION("binary formats") + { + const auto j = no_key_compare_json::parse(R"({"a":[1,2,3],"b":"x"})"); + CHECK(no_key_compare_json::from_cbor(no_key_compare_json::to_cbor(j)) == j); + CHECK(no_key_compare_json::from_msgpack(no_key_compare_json::to_msgpack(j)) == j); + } + + SECTION("flatten and unflatten") + { + // "o" has a key that looks like an array index, so unflatten() must + // not turn it into an array + const auto j = no_key_compare_json::parse( + R"({"c":[1,2,3],"d":{"e":"s"},"n":[[0,1],[2]],"o":{"2":"x"}})"); + CHECK(j.flatten().unflatten() == j); + } + + SECTION("conversion to and from nlohmann::json") + { + const auto j = no_key_compare_json::parse(R"({"a":1,"b":[true,null]})"); + const nlohmann::json converted(j); + + CHECK(converted.is_object()); + CHECK(converted["a"] == 1); + CHECK(converted["b"][0] == true); + CHECK(converted["b"][1].is_null()); + CHECK(no_key_compare_json(converted) == j); + } +} + diff --git a/tests/src/unit-json_pointer.cpp b/tests/src/unit-json_pointer.cpp index 4082de45c..b01df2921 100644 --- a/tests/src/unit-json_pointer.cpp +++ b/tests/src/unit-json_pointer.cpp @@ -507,6 +507,16 @@ TEST_CASE("JSON pointers") // explicit roundtrip check CHECK(j.flatten().unflatten() == j); + // an object is only unflattened to an array if one of its keys is the + // reference token 0; this must not depend on which key is seen first + CHECK(json({{"/2", "x"}}).unflatten() == json({{"2", "x"}})); + CHECK(json({{"/10", "y"}, {"/2", "z"}}).unflatten() == json({{"10", "y"}, {"2", "z"}})); + CHECK(json({{"/0", 1}, {"/1", 2}}).unflatten() == json({1, 2})); + CHECK(json({{"/1", 2}, {"/0", 1}}).unflatten() == json({1, 2})); + CHECK(json({{"/0", 1}, {"/2", 3}}).unflatten() == json({1, nullptr, 3})); + CHECK(json({{"/a/1", 2}, {"/a/0", 1}}).unflatten() == json({{"a", {1, 2}}})); + CHECK(json({{"/a/1", 2}, {"/a/x", 1}}).unflatten() == json({{"a", {{"1", 2}, {"x", 1}}}})); + // roundtrip for primitive values json j_null; CHECK(j_null.flatten().unflatten() == j_null);