diff --git a/docs/mkdocs/docs/api/basic_json/string_t.md b/docs/mkdocs/docs/api/basic_json/string_t.md index a07c9ba62..e8be0fe0e 100644 --- a/docs/mkdocs/docs/api/basic_json/string_t.md +++ b/docs/mkdocs/docs/api/basic_json/string_t.md @@ -25,7 +25,8 @@ JSON class into byte-sized characters during deserialization. Beyond the character type, the library expects a substantial part of the `#!cpp std::string` interface (contiguous null-terminated `data()`, `substr()`, `find()`, `append()`, ...). See - [Template Parameter Requirements](../../features/types/template_parameters.md#stringtype) for the full list. + [Template Parameter Requirements](../../features/types/template_parameters.md#stringtype) for the full list and + for the string types that are known to work. ## Notes @@ -82,3 +83,5 @@ and an example. ## Version history - Added in version 1.0.0. +- Removed the requirement that `string_t` be implicitly convertible from `#!cpp std::string`, which the BSON writer and + the UBJSON reader relied on, in version 3.13.0. diff --git a/docs/mkdocs/docs/features/object_order.md b/docs/mkdocs/docs/features/object_order.md index f62474efd..8cbb6e902 100644 --- a/docs/mkdocs/docs/features/object_order.md +++ b/docs/mkdocs/docs/features/object_order.md @@ -51,7 +51,9 @@ If you do want to preserve the **insertion order**, you can use the type [`nlohm --8<-- "examples/ordered_json.output" ``` -Alternatively, you can use a more sophisticated ordered map like [`tsl::ordered_map`](https://github.com/Tessil/ordered-map) ([integration](https://github.com/nlohmann/json/issues/546#issuecomment-304447518)) or [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)). +Alternatively, you can use a more sophisticated map with a lookup index. `boost::unordered_flat_map`, `absl::flat_hash_map`, and `absl::node_hash_map` all work through a small adapter that restores the template argument order `basic_json` expects; see [Template Parameter Requirements](types/template_parameters.md#objecttype). Note these are *unordered*, not insertion-ordered. + +[`tsl::ordered_map`](https://github.com/Tessil/ordered-map) cannot be used: its iterators expose the mapped value as `const`, while `basic_json` needs to modify it in place. The [`ordered_map`](../api/ordered_map.md) behind `nlohmann::ordered_json` is deliberately minimal and has no lookup index, so every key access is a linear scan and building an object of `n` keys costs O(n²). This is unnoticeable at diff --git a/docs/mkdocs/docs/features/types/template_parameters.md b/docs/mkdocs/docs/features/types/template_parameters.md index 07d7c2eb8..bb68ad523 100644 --- a/docs/mkdocs/docs/features/types/template_parameters.md +++ b/docs/mkdocs/docs/features/types/template_parameters.md @@ -5,7 +5,7 @@ never formally states what a type passed for one of these parameters has to prov the way the library uses the resulting [`object_t`](../../api/basic_json/object_t.md), [`array_t`](../../api/basic_json/array_t.md), [`string_t`](../../api/basic_json/string_t.md), etc. This page collects these requirements so they do not have to be discovered by trial and error. Each section also lists the concrete -types that are known to work for that parameter; the Abseil entries were checked against release 20250127.0. +types that are known to work for that parameter, checked against Boost 1.83, Abseil 20250127.0, and EASTL 3.21. ## How to read this page @@ -46,7 +46,9 @@ Requirements are split into two groups: incomplete type. `#!cpp std::map` and `#!cpp std::vector` are required by the standard to support incomplete value types; most third-party containers are not, and inspecting the value type at class scope (for instance with `#!cpp std::is_trivially_move_assignable`) makes them unusable as `ObjectType` or `ArrayType`. This rules out - `absl::btree_map` and `absl::InlinedVector`, among others, no matter how their template arguments are adapted. + `absl::btree_map`, `absl::InlinedVector`, `eastl::vector`, and `eastl::hash_map`, among others, no matter how their + template arguments are adapted. Boost.Container is the notable exception: it documents support for incomplete + types, and all of its containers work here. ## `ObjectType` @@ -169,9 +171,12 @@ The library does not sort or de-duplicate keys itself; the behavior described in | `#!cpp std::map` (default) | full | | [`nlohmann::ordered_map`](../../api/ordered_map.md) | full; used by [`ordered_json`](../../api/ordered_json.md) | | `#!cpp std::unordered_map`, through the adapter shown above | full | -| [`tsl::ordered_map`](https://github.com/Tessil/ordered-map), [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) | through the same adapter pattern; see [Object Order](../object_order.md) | +| `boost::container::map`, `boost::container::flat_map` | full, with no adapter | +| `boost::unordered_map`, `boost::unordered_flat_map`, `boost::unordered_node_map`, through the adapter shown above | full | | `absl::flat_hash_map`, `absl::node_hash_map`, through the adapter shown above | full | -| `absl::btree_map` | not usable; requires a complete value type | +| `absl::btree_map`, `eastl::hash_map` | not usable; require a complete value type | +| `eastl::map` | not usable; EASTL iterators do not work with `#!cpp std::iterator_traits` | +| `tsl::ordered_map` | not usable; its iterators expose the mapped value as `#!cpp const` | | `#!cpp std::multimap`, `#!cpp std::unordered_multimap` | not usable; `emplace` does not return `#!cpp std::pair` | ## `ArrayType` @@ -214,7 +219,9 @@ using array_t = ArrayType>; | `#!cpp std::vector` (default) | full | | `#!cpp std::deque` | full; keeps references valid while the array grows, but see the note on `capacity()` above | | `#!cpp std::list` | not usable; no `operator[]` and no random-access iterators | -| `absl::InlinedVector` | not usable; requires a complete value type | +| `boost::container::vector`, `boost::container::deque`, `boost::container::stable_vector` | full; `stable_vector` keeps references valid across every insertion | +| `boost::container::small_vector` | full, through an alias that fixes the inline capacity | +| `absl::InlinedVector`, `eastl::vector` | not usable; require a complete value type | | `absl::FixedArray` | not usable; the size is fixed at construction | ## `StringType` @@ -258,6 +265,7 @@ using array_t = ArrayType>; | Functionality | Additional requirement | |-------------------------------------------------------------|---------------------------------------------------| | [`to_bson`](../../api/basic_json/to_bson.md) | `find(value_type)` and `npos` | +| [`parse`](../../api/basic_json/parse.md) from a `string_t` | the input adapters must accept it; otherwise pass a character range | | [`std::hash`](../../api/basic_json/std_hash.md) | a specialization of `#!cpp std::hash` | | exception messages | `data()` and `size()`, or `begin()` and `end()` | @@ -268,6 +276,23 @@ using array_t = ArrayType>; ### Compatible types +| Type | Support | +|-----------------------------------------------------------------------------|-----------------------------------------------------------------------------| +| `#!cpp std::string` (default) | full | +| `#!cpp std::pmr::string`, `#!cpp std::basic_string` with a custom allocator | full | +| `boost::container::string` | full, once a `#!cpp std::hash` specialization is supplied (Boost provides `boost::hash` instead) | +| `eastl::string` | full, except that [`parse`](../../api/basic_json/parse.md) does not accept it directly; pass a character range or a `#!cpp std::string` | +| a custom string class in a user-defined namespace | full, if the requirements above are met | +| `#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string` | not usable; the character type is not one byte wide | +| `absl::Cord` | not usable; no `value_type`, and the storage is not contiguous | + +!!! tip "Reference implementation" + + The unit test `tests/src/unit-alt-string.cpp` contains `alt_string`, a minimal string type that satisfies the + requirements needed for the tested subset of the API. It is a good starting point for a custom `StringType`. + +### Compatible types + | Type | Support | |-----------------------------------------------------------------------------|-----------------------------------------------------------------------------| | `#!cpp std::string` (default) | full | @@ -470,7 +495,7 @@ such a container to a `basic_json` value. | `#!cpp std::vector` (default) | full | | `#!cpp std::vector` | full | | `#!cpp std::vector` | full | -| `absl::InlinedVector` | full | +| `absl::InlinedVector`, `eastl::vector`, `boost::container::vector`, `boost::container::small_vector` | full | | `#!cpp std::string` | not usable; `binary_t::container_type` and `string_t` would be the same type, which makes the [`swap`](../../api/basic_json/swap.md) overloads ambiguous | | containers whose `value_type` is wider than one byte | not usable | diff --git a/include/nlohmann/detail/input/binary_reader.hpp b/include/nlohmann/detail/input/binary_reader.hpp index 557d7669c..139cdab46 100644 --- a/include/nlohmann/detail/input/binary_reader.hpp +++ b/include/nlohmann/detail/input/binary_reader.hpp @@ -2899,7 +2899,10 @@ class binary_reader number_string, out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr)); } - return sax->number_float(parsed_float, std::move(number_string)); + // number_string is a std::string, while the SAX interface takes a + // string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + return sax->number_float(parsed_float, string_t(number_string.data(), number_string.size())); } case token_type::uninitialized: case token_type::literal_true: diff --git a/include/nlohmann/detail/output/binary_writer.hpp b/include/nlohmann/detail/output/binary_writer.hpp index ea4a83020..03390215a 100644 --- a/include/nlohmann/detail/output/binary_writer.hpp +++ b/include/nlohmann/detail/output/binary_writer.hpp @@ -1147,7 +1147,8 @@ class binary_writer const std::size_t embedded_document_size = std::accumulate(std::begin(value), std::end(value), static_cast(0), [&array_index](std::size_t result, const typename BasicJsonType::array_t::value_type & el) { - return result + calc_bson_element_size(std::to_string(array_index++), el); + const auto key = std::to_string(array_index++); + return result + calc_bson_element_size(string_t(key.data(), key.size()), el); }); return sizeof(std::int32_t) + embedded_document_size + 1ul; @@ -1174,7 +1175,11 @@ class binary_writer for (const auto& el : value) { - write_bson_element(std::to_string(array_index++), el); + // the index is built as a std::string, while write_bson_element takes + // a string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + const auto key = std::to_string(array_index++); + write_bson_element(string_t(key.data(), key.size()), el); } oa->write_character(to_char_type(0x00)); diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index e56c0970f..d598babdc 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -13531,7 +13531,10 @@ class binary_reader number_string, out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr)); } - return sax->number_float(parsed_float, std::move(number_string)); + // number_string is a std::string, while the SAX interface takes a + // string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + return sax->number_float(parsed_float, string_t(number_string.data(), number_string.size())); } case token_type::uninitialized: case token_type::literal_true: @@ -18138,7 +18141,8 @@ class binary_writer const std::size_t embedded_document_size = std::accumulate(std::begin(value), std::end(value), static_cast(0), [&array_index](std::size_t result, const typename BasicJsonType::array_t::value_type & el) { - return result + calc_bson_element_size(std::to_string(array_index++), el); + const auto key = std::to_string(array_index++); + return result + calc_bson_element_size(string_t(key.data(), key.size()), el); }); return sizeof(std::int32_t) + embedded_document_size + 1ul; @@ -18165,7 +18169,11 @@ class binary_writer for (const auto& el : value) { - write_bson_element(std::to_string(array_index++), el); + // the index is built as a std::string, while write_bson_element takes + // a string_t; convert explicitly, as the two are only implicitly + // convertible for some string types + const auto key = std::to_string(array_index++); + write_bson_element(string_t(key.data(), key.size()), el); } oa->write_character(to_char_type(0x00)); diff --git a/tests/src/unit-alt-string.cpp b/tests/src/unit-alt-string.cpp index 46e062c6e..8b7043589 100644 --- a/tests/src/unit-alt-string.cpp +++ b/tests/src/unit-alt-string.cpp @@ -11,8 +11,10 @@ #include +#include #include #include +#include /* forward declarations */ class alt_string; @@ -202,6 +204,31 @@ bool operator<(const char* op1, const alt_string& op2) noexcept TEST_CASE("alternative string type") { + SECTION("binary formats") + { + alt_json doc; + doc["pi"] = 3.141; + doc["happy"] = true; + doc["list"] = {1, 2, 3}; + + CHECK(alt_json::from_cbor(alt_json::to_cbor(doc)) == doc); + CHECK(alt_json::from_msgpack(alt_json::to_msgpack(doc)) == doc); + // BSON is not covered: it additionally needs string_t::find(value_type), + // which alt_string does not provide + CHECK(alt_json::from_ubjson(alt_json::to_ubjson(doc)) == doc); + + // a UBJSON high-precision number is parsed into a std::string that the + // reader has to hand to the SAX interface as an alt_string + const std::vector high_precision = + { + 'H', 'i', 0x16, '3', '.', '1', '4', '1', '5', '9', '2', '6', '5', '3', + '5', '8', '9', '7', '9', '3', '2', '3', '8', '4', '6' + }; + const auto number = alt_json::from_ubjson(high_precision); + CHECK(number.is_number_float()); + CHECK(number.get() == doctest::Approx(3.14159265358979323846)); + } + SECTION("dump") { {