docs: add version-history notes for fixed start_pos and error positions

Two bugs of 3.12.0 are fixed on develop without a note on the pages.
With JSON_DIAGNOSTIC_POSITIONS, start_pos() of a string that contains
escape sequences was too large; the position is now that of the opening
quote. A parse error for an unexpected number that is followed by a line
feed reported column 0 (for "1 2" and a line feed, line 1, column 0); for
"1", line feed, "2", line feed it now reports line 2, column 1.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/dependabot.yml

@@ -67,3 +67,4 @@ updates:
       - dependency-name: mcp
       - dependency-name: boltons
       - dependency-name: "opentelemetry-*"
+      - dependency-name: importlib-metadata

cmake/requirements/requirements-semgrep.txt

@@ -391,9 +391,9 @@ httpx-sse==0.4.3 \
 idna==3.20 \
     --hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \
     --hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c
-importlib-metadata==10.1.0 \
-    --hash=sha256:82b04047fa09792c1a8cfeddfe08bec79e4b13c29c8469b0d3949860f61017e8 \
-    --hash=sha256:cf45ce66a929c64e994185a809c04840d0501db17e663810d993d0f4829265c7
+importlib-metadata==8.7.1 \
+    --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \
+    --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151
 jsonschema==4.25.1 \
     --hash=sha256:3fba0169e345c7175110351d456342c364814cfcf3b964ba4587f22915230a63 \
     --hash=sha256:e4a9655ce0da0c0b67a085847e00a3a51449e1157f4f75e9fb5aa545e122eb85

docs/mkdocs/docs/api/basic_json/basic_json.md

@@ -67,11 +67,7 @@ basic_json(basic_json&& other) noexcept;
     - **objects**: [`object_t`](object_t.md) and all kinds of compatible associative containers such as `std::map`,
      `std::unordered_map`, `std::multimap`, and `std::unordered_multimap` with a `key_type` compatible to `string_t`
      and a `value_type` from which a `basic_json` value can be constructed.
-    - **strings**: `string_t`, string literals, and all compatible string containers can be used. Since version 3.13.0,
-      this includes `#!cpp std::u8string` (C++20): its code units are stored as a JSON string and
-      [`get<std::u8string>()`](get.md) reads it back (in version 3.12.0, a `#!cpp std::u8string` was stored as an array
-      of numbers; `get<std::u8string>()` still accepts such an array). `#!cpp std::u8string_view` and `#!cpp char8_t`
-      arrays are not strings; they are stored as arrays of code units.
+    - **strings**: `string_t`, string literals, and all compatible string containers can be used.
 
     !!! note "Rvalue containers are moved from"
 

docs/mkdocs/docs/api/basic_json/get.md

@@ -130,12 +130,6 @@ overload (3).
     `#!cpp json(true).get<double>()` throw. This is the behavior of all 3.x versions;
     [`basic_json_view::get<T>()`](../basic_json_view/get.md) behaves the same.
 
-!!! note "`std::u8string` conversions"
-
-    Since version 3.13.0, `#!cpp get<std::u8string>()` (C++20) reads a JSON string; it also accepts an array of numbers,
-    the form in which version 3.12.0 stored a `#!cpp std::u8string`. Any other JSON type throws
-    [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) ("type must be string or array").
-
 !!! note "`std::optional` conversions"
 
     Prior to version 3.13.0, `#!cpp get<std::optional<T>>()` (and other conversions to `std::optional<T>`) failed to

docs/mkdocs/docs/api/basic_json/operator_ge.md

@@ -60,6 +60,14 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
+    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
+    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
+    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
+    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
+    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
+    same for all language standards. The same program can therefore order such values differently when it is compiled
+    with a different language standard.
+
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_gt.md

@@ -60,6 +60,14 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
+    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
+    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
+    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
+    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
+    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
+    same for all language standards. The same program can therefore order such values differently when it is compiled
+    with a different language standard.
+
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_le.md

@@ -61,6 +61,14 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
+    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
+    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
+    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
+    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
+    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
+    same for all language standards. The same program can therefore order such values differently when it is compiled
+    with a different language standard.
+
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_lt.md

@@ -71,6 +71,14 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
+    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
+    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
+    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
+    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
+    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
+    same for all language standards. The same program can therefore order such values differently when it is compiled
+    with a different language standard.
+
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_spaceship.md

@@ -55,6 +55,10 @@ Linear.
         1. Comparing a `NaN` with itself.
         2. Comparing a `NaN` with another `NaN`.
         3. Comparing a `NaN` and any other number.
+    - For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
+      another `NaN`) before any other difference, the result is `std::partial_ordering::unordered`. (Without C++20, the
+      comparison operators `<`, `<=`, `>`, and `>=` use the comparison of the containers, which treats such a pair as
+      equivalent; see [**operator<**](operator_lt.md).)
 
 ## Examples
 

docs/mkdocs/docs/api/basic_json/parse.md

@@ -130,9 +130,11 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
 
     Contiguous inputs (strings, string views, vectors, arrays, pointer ranges) are the fastest to parse. A
     `#!cpp std::istream` comes close: the parser scans the characters its stream buffer holds in bulk, without reading
-    ahead, so the stream is left exactly where it would be after reading one character at a time. A `#!cpp FILE*` is
-    read one character at a time, but the file is locked only once, so other threads that use the same `FILE` object
-    wait until parsing is done. See [input sources and performance](../../features/parsing/index.md#input-sources-and-performance),
+    ahead, so the stream is left exactly where it would be after reading one character at a time. With glibc
+    and the Apple C library, the characters the `#!cpp FILE` holds in its buffer are scanned in bulk like a stream's;
+    elsewhere a `#!cpp FILE*` is read one character at a time. Where the C library has unlocked stdio functions, the
+    file is locked only once, so other threads that use the same `FILE` object wait until parsing is done. See
+    [input sources and performance](../../features/parsing/index.md#input-sources-and-performance),
     [`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md), and
     [`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md).
 
@@ -311,6 +313,10 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
   it as end of input; planned to become the default in version 4.0.0.
 - Extended empty-input detection to also cover an `std::istream` without a stream buffer, and fixed a crash
   (`std::terminate`) when parsing from an `std::istream` with `eofbit` in its exception mask, in version 3.13.0.
+- Fixed the position that a [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) reports for an
+  unexpected number that is followed by a line feed in version 3.13.0: for the input `1`, line feed, `2`, line feed,
+  version 3.12.0 reported "line 2, column 0" and version 3.13.0 reports "line 2, column 1"; for `1 2` and a line feed,
+  the report was "line 1, column 0" and is now "line 1, column 3".
 - Parsing from an `std::istream` scans the characters in its stream buffer in bulk in version 3.13.0
   ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md) restores reading one character at a
   time).

docs/mkdocs/docs/api/basic_json/sax_parse.md

@@ -124,9 +124,10 @@ consumer `sax` has a super-linear complexity.
 
 A UTF-8 byte order mark is silently ignored.
 
-When reading from a `#!c FILE*`, the file is locked while it is parsed. The lock is recursive, so the SAX handler may
-use the same file with stdio functions, but it must not wait for another thread that uses the same `FILE` object; see
-[`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md).
+When reading from a `#!c FILE*`, where the C library has unlocked stdio functions (see
+[`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)), the file is locked while it is parsed.
+The lock is recursive, so the SAX handler may use the same file with stdio functions, but it must not wait for another
+thread that uses the same `FILE` object.
 
 ## Examples
 

docs/mkdocs/docs/api/basic_json/start_pos.md

@@ -67,3 +67,7 @@ Constant.
 ## Version history
 
 - Added in version 3.12.0.
+- Fixed in version 3.13.0: in version 3.12.0, the start position of a string that contains escape sequences (for
+  example `#!cpp "\n"` or `#!cpp "\u00e4"`) was too large, by the number of bytes the escape sequences add to the
+  source text. The position is now always that of the opening `"`. The "(bytes a-b)" range in the messages of
+  [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md) was wrong in the same way.

docs/mkdocs/docs/api/macros/json_diagnostic_positions.md

@@ -125,3 +125,5 @@ When the macro is not defined, the library will define it to its default value.
 ## Version history
 
 - Added in version 3.12.0.
+- Fixed in version 3.13.0: in version 3.12.0, [`start_pos()`](../basic_json/start_pos.md) of a string that contains
+  escape sequences was too large; it is now the position of the opening `"`.

docs/mkdocs/docs/features/binary_formats/bjdata.md

@@ -88,7 +88,6 @@ The library uses the following mapping from JSON values types to BJData types ac
 | number_integer  | 65536..2147483647                         | int32       | `l`                                                                     |
 | number_integer  | 2147483648..4294967295                    | uint32      | `m`                                                                     |
 | number_integer  | 4294967296..9223372036854775807           | int64       | `L`                                                                     |
-| number_integer  | 9223372036854775808..18446744073709551615 | uint64      | `M`                                                                     |
 | number_unsigned | 0..127                                    | int8        | `i`                                                                     |
 | number_unsigned | 128..255                                  | uint8       | `U`                                                                     |
 | number_unsigned | 256..32767                                | int16       | `I`                                                                     |

docs/mkdocs/docs/features/binary_formats/messagepack.md

@@ -61,7 +61,7 @@ specification:
 | number_integer  | 128..255                                 | uint 8           | 0xCC       |
 | number_integer  | 256..65535                               | uint 16          | 0xCD       |
 | number_integer  | 65536..4294967295                        | uint 32          | 0xCE       |
-| number_integer  | 4294967296..18446744073709551615         | uint 64          | 0xCF       |
+| number_integer  | 4294967296..9223372036854775807          | uint 64          | 0xCF       |
 | number_unsigned | 0..127                                   | positive fixint  | 0x00..0x7F |
 | number_unsigned | 128..255                                 | uint 8           | 0xCC       |
 | number_unsigned | 256..65535                               | uint 16          | 0xCD       |

docs/mkdocs/docs/features/parsing/index.md

@@ -77,7 +77,7 @@ them faster than others:
 |--------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
 | strings, string views, string literals, contiguous containers, pointer or iterator ranges over contiguous memory | the whole input is scanned in bulk: runs of string characters, numbers, and indentation are recognized several bytes at a time |
 | `#!cpp std::istream` (`#!cpp std::ifstream`, `#!cpp std::istringstream`, ...)            | the characters the stream buffer holds are scanned in bulk like a string; the stream buffer is asked for more when they are used up |
-| `#!cpp FILE*`                                                                              | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; the file is locked once for the whole parse |
+| `#!cpp FILE*`                                                                              | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; where the C library has unlocked stdio functions, the file is locked once for the whole parse |
 | other iterator ranges (e.g., `#!cpp std::list<char>`, `#!cpp std::istreambuf_iterator<char>`), wide strings | one character at a time                                                             |
 
 Input in one block of memory is read even faster with the default options (no

docs/mkdocs/docs/features/performance.md

@@ -27,8 +27,9 @@ A `#!cpp std::istream` (including `#!cpp std::ifstream`) gets the same bulk scan
 characters its stream buffer already holds (the *get area*) and asks the stream buffer for more only when they are used
 up. Nothing is read ahead, so the stream is left exactly where it would be after reading one character at a time, and
 [`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
-area is read one character at a time. A `#!cpp FILE*` is locked once per parse instead of once per character, and with
-glibc and the Apple C library, the characters in its buffer are scanned in bulk as a stream's get area is. See
+area is read one character at a time. Where the C library has unlocked stdio functions, a `#!cpp FILE*` is locked once
+per parse instead of once per character, and with glibc and the Apple C library, the characters in its buffer are
+scanned in bulk as a stream's get area is. See
 [input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. Parsing from a
 stream takes about 1.15 to 1.45 times as long as parsing an equivalent `#!cpp std::string`, which `parse()` reads with
 a parser of its own (see there).

docs/mkdocs/docs/features/types/template_parameters.md

@@ -425,7 +425,7 @@ using array_t = ArrayType<basic_json, AllocatorType<basic_json>>;
 | Type                                                                 | Reason                                                                                                  |
 |----------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
 | `#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string` | the character type is not one byte wide                                                                 |
-| `#!cpp std::u8string`                                                | one byte wide, but `#!cpp char8_t` is not `#!cpp char`-compatible (as a *value*, it converts to and from a JSON string) |
+| `#!cpp std::u8string`                                                | one byte wide, but `#!cpp char8_t` is not `#!cpp char`-compatible                                       |
 | `absl::Cord`                                                         | no `value_type`, and the storage is not contiguous                                                      |
 | `QString`                                                            | no `append(const char*, size_type)`; its `QChar` is also two bytes wide, though that is never diagnosed |
 

docs/mkdocs/docs/home/releases/unreleased.md

@@ -8,16 +8,6 @@ These changes are part of the `develop` branch, but not of a release yet.
   exception is thrown during the conversion, the source may be partially moved-from, so the strong exception guarantee
   no longer holds for the source of an rvalue conversion (the JSON value itself is unaffected). Lvalues and `const`
   objects are still copied. See [`basic_json`](../../api/basic_json/basic_json.md).
-- In C++20 (library support for `char8_t`, `__cpp_lib_char8_t`), a `#!cpp std::u8string` is a JSON **string**:
-  `json j = std::u8string(u8"abc");` stores `"abc"`, and `#!cpp j.get<std::u8string>()` reads it back. Version 3.12.0
-  stored a `#!cpp std::u8string` as an array of code units (`[97,98,99]`); on the 3.13.0 development branch it was
-  already stored as a string if `std::filesystem` is available, but could not be read back. Now the result no longer
-  depends on whether `std::filesystem::path` is supported, and `#!cpp get<std::u8string>()` accepts both a JSON string
-  and the array of numbers written by 3.12.0. **Consequence:** JSON documents produced from a `#!cpp std::u8string` by
-  3.12.0 (arrays) are still read, but code that checks `is_array()` on such a value, or that reads it as
-  `#!cpp std::vector<char8_t>` / `#!cpp std::vector<std::uint8_t>`, now sees a string; to keep the array form, convert
-  explicitly, e.g., `#!cpp json(std::vector<char8_t>(s.begin(), s.end()))`. `#!cpp std::u8string_view` and
-  `#!cpp char8_t` arrays are unchanged (arrays of code units). See [`basic_json`](../../api/basic_json/basic_json.md).
 
 ## API changes
 

include/nlohmann/detail/conversions/from_json.hpp

@@ -785,37 +785,6 @@ void from_json(const BasicJsonType& j, std::unordered_map<Key, Value, Hash, KeyE
     from_json_pair_array_to_map(j, m);
 }
 
-#if defined(__cpp_lib_char8_t)
-// a std::u8string is read from a JSON string; the array of code units written by
-// versions before 3.13.0 is still accepted
-template<typename BasicJsonType, typename Tr, typename Allocator,
-         enable_if_t<std::is_same<typename json_type_of_t<BasicJsonType>::string_t::value_type, char>::value, int> = 0>
-inline void from_json(const BasicJsonType& j, std::basic_string<char8_t, Tr, Allocator>& s)
-{
-    if (j.is_string())
-    {
-        const auto& str = string_value(j);
-        const char* const data = str.data();
-        const std::size_t size = str.size();
-        std::basic_string<char8_t, Tr, Allocator> ret(s.get_allocator());
-        ret.resize(size);
-        for (std::size_t i = 0; i < size; ++i)
-        {
-            ret[i] = static_cast<char8_t>(static_cast<unsigned char>(data[i]));
-        }
-        s = std::move(ret);
-    }
-    else if (j.is_array())
-    {
-        from_json_array_impl(j, s, priority_tag<3> {});
-    }
-    else
-    {
-        throw_type_must_be("string or array", j);
-    }
-}
-#endif
-
 #if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 
 // Workaround for MSVC 19.51 (and possibly later): in large cpp files, the compiler may fail to resolve with generic has_from_json (issue #4996)

include/nlohmann/detail/conversions/to_json.hpp

@@ -625,20 +625,16 @@ inline void to_json(BasicJsonType& j, const T& t)
     to_json_tuple_impl(j, t, make_index_sequence<std::tuple_size<T>::value> {});
 }
 
-// a std::u8string is a JSON string (it was an array of code units before 3.13.0);
-// defined outside the std::filesystem guard so that the result does not depend
-// on whether std::filesystem::path is supported
+#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 #if defined(__cpp_lib_char8_t)
 template<typename BasicJsonType, typename Tr, typename Allocator>
 inline void to_json(BasicJsonType& j, const std::basic_string<char8_t, Tr, Allocator>& s)
 {
-    using string_t = typename BasicJsonType::string_t;
-    // char may alias any object; the code units are copied unchanged
-    j = string_t(reinterpret_cast<const char*>(s.data()), s.size());
+    using OtherAllocator = typename std::allocator_traits<Allocator>::template rebind_alloc<char>;
+    j = std::basic_string<char, std::char_traits<char>, OtherAllocator>(s.begin(), s.end(), s.get_allocator());
 }
 #endif
 
-#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 // Workaround for MSVC 19.51 (and possibly later): in large cpp files, the compiler may fail to resolve with generic has_to_json (issue #4996)
 template<typename BasicJsonType>
 struct has_to_json<BasicJsonType, std_fs::path, void> : std::true_type {};

single_include/nlohmann/json.hpp

@@ -6928,37 +6928,6 @@ void from_json(const BasicJsonType& j, std::unordered_map<Key, Value, Hash, KeyE
     from_json_pair_array_to_map(j, m);
 }
 
-#if defined(__cpp_lib_char8_t)
-// a std::u8string is read from a JSON string; the array of code units written by
-// versions before 3.13.0 is still accepted
-template<typename BasicJsonType, typename Tr, typename Allocator,
-         enable_if_t<std::is_same<typename json_type_of_t<BasicJsonType>::string_t::value_type, char>::value, int> = 0>
-inline void from_json(const BasicJsonType& j, std::basic_string<char8_t, Tr, Allocator>& s)
-{
-    if (j.is_string())
-    {
-        const auto& str = string_value(j);
-        const char* const data = str.data();
-        const std::size_t size = str.size();
-        std::basic_string<char8_t, Tr, Allocator> ret(s.get_allocator());
-        ret.resize(size);
-        for (std::size_t i = 0; i < size; ++i)
-        {
-            ret[i] = static_cast<char8_t>(static_cast<unsigned char>(data[i]));
-        }
-        s = std::move(ret);
-    }
-    else if (j.is_array())
-    {
-        from_json_array_impl(j, s, priority_tag<3> {});
-    }
-    else
-    {
-        throw_type_must_be("string or array", j);
-    }
-}
-#endif
-
 #if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 
 // Workaround for MSVC 19.51 (and possibly later): in large cpp files, the compiler may fail to resolve with generic has_from_json (issue #4996)
@@ -8360,20 +8329,16 @@ inline void to_json(BasicJsonType& j, const T& t)
     to_json_tuple_impl(j, t, make_index_sequence<std::tuple_size<T>::value> {});
 }
 
-// a std::u8string is a JSON string (it was an array of code units before 3.13.0);
-// defined outside the std::filesystem guard so that the result does not depend
-// on whether std::filesystem::path is supported
+#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 #if defined(__cpp_lib_char8_t)
 template<typename BasicJsonType, typename Tr, typename Allocator>
 inline void to_json(BasicJsonType& j, const std::basic_string<char8_t, Tr, Allocator>& s)
 {
-    using string_t = typename BasicJsonType::string_t;
-    // char may alias any object; the code units are copied unchanged
-    j = string_t(reinterpret_cast<const char*>(s.data()), s.size());
+    using OtherAllocator = typename std::allocator_traits<Allocator>::template rebind_alloc<char>;
+    j = std::basic_string<char, std::char_traits<char>, OtherAllocator>(s.begin(), s.end(), s.get_allocator());
 }
 #endif
 
-#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 // Workaround for MSVC 19.51 (and possibly later): in large cpp files, the compiler may fail to resolve with generic has_to_json (issue #4996)
 template<typename BasicJsonType>
 struct has_to_json<BasicJsonType, std_fs::path, void> : std::true_type {};

tests/CMakeLists.txt

@@ -364,16 +364,6 @@ json_test_add_test_for(src/unit-class_parser.cpp
     MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
 )
 
-# test the conversions again without std::filesystem::path support: a
-# std::u8string must be a JSON string in this configuration, too
-json_test_set_test_options(test-conversions2_no_filesystem
-    COMPILE_DEFINITIONS JSON_HAS_FILESYSTEM=0 JSON_HAS_EXPERIMENTAL_FILESYSTEM=0
-)
-json_test_add_test_for(src/unit-conversions2.cpp
-    NAME test-conversions2_no_filesystem
-    MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
-)
-
 # test diagnostic positions again without regular diagnostics (JSON pointer paths)
 json_test_set_test_options(test-diagnostic-positions_only
     COMPILE_DEFINITIONS JSON_DIAGNOSTICS=0

tests/src/unit-alt-string.cpp

@@ -536,20 +536,4 @@ TEST_CASE("alternative string type with json_view")
     }
 }
 
-// a std::u8string is a JSON string, whatever the string type of the JSON value
-// JSON_HAS_CPP_20 JSON_HAS_CPP_23 (do not remove; see note at top of file)
-#if defined(__cpp_lib_char8_t)
-TEST_CASE("alternative string type and std::u8string")
-{
-    const std::u8string s = u8"abc";
-    const alt_json j = s;
-    CHECK(j.is_string());
-    CHECK(j.get<alt_string>() == alt_string("abc"));
-    CHECK(j.get<std::u8string>() == s);
-
-    // the array of code units written by versions before 3.13.0
-    CHECK(alt_json::array({0x61, 0x62, 0x63}).get<std::u8string>() == s);
-}
-#endif
-
 DOCTEST_CLANG_SUPPRESS_WARNING_POP

tests/src/unit-conversions2.cpp

@@ -27,12 +27,10 @@ using nlohmann::json;
 #include <deque>
 #include <forward_list>
 #include <list>
-#include <map>
 #include <set>
 #include <unordered_map>
 #include <unordered_set>
 #include <valarray>
-#include <vector>
 
 // NLOHMANN_JSON_SERIALIZE_ENUM uses a static std::pair
 DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
@@ -709,115 +707,39 @@ TEST_CASE("std::filesystem::path")
 }
 #endif
 
-// a std::u8string is a JSON string since 3.13.0 (it was an array of code units
-// before); independent of whether std::filesystem::path is supported, see the
-// test-conversions2_no_filesystem target
-// JSON_HAS_CPP_20 JSON_HAS_CPP_23 (do not remove; see note at top of file)
+// the ADL to_json overload for std::u8string only exists under the same guard
+// as std::filesystem::path support (it is otherwise only reached indirectly,
+// via std::filesystem::path::u8string()) -- mirror both #if conditions from
+// include/nlohmann/detail/conversions/to_json.hpp exactly
+#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
 #if defined(__cpp_lib_char8_t)
 TEST_CASE("std::u8string")
 {
-    // use \u universal-character-names (rather than raw \x byte escapes
-    // or literal non-ASCII source bytes) to compose the multi-byte UTF-8
-    // encoding -- MSVC treats \x escapes used that way inside a u8
-    // literal as a nonstandard extension (warning C5321), which some of
-    // our CI configs promote to an error; \u is portable and produces
-    // the exact same encoded bytes without depending on the source
-    // file's encoding
-    const std::u8string utf8 = u8"P\u011B\u0161ina";
-
     SECTION("ascii")
     {
         const std::u8string s = u8"Path";
         json const j = s;
 
-        CHECK(j.is_string());
         CHECK(j.template get<std::string>() == "Path");
     }
 
     SECTION("utf-8")
     {
-        json const j = utf8;
+        // use \u universal-character-names (rather than raw \x byte escapes
+        // or literal non-ASCII source bytes) to compose the multi-byte UTF-8
+        // encoding -- MSVC treats \x escapes used that way inside a u8
+        // literal as a nonstandard extension (warning C5321), which some of
+        // our CI configs promote to an error; \u is portable and produces
+        // the exact same encoded bytes without depending on the source
+        // file's encoding
+        const std::u8string s = u8"P\u011B\u0161ina";
+        json const j = s;
 
-        CHECK(j.is_string());
         CHECK(j.template get<std::string>() == "P\xc4\x9b\xc5\xa1ina");
     }
-
-    SECTION("round trip")
-    {
-        json const j = utf8;
-        CHECK(j.template get<std::u8string>() == utf8);
-
-        std::u8string back;
-        j.get_to(back);
-        CHECK(back == utf8);
-
-        // from a string
-        json const j_string = "P\xc4\x9b\xc5\xa1ina";
-        CHECK(j_string.template get<std::u8string>() == utf8);
-
-        // empty, and with an embedded NUL
-        CHECK(json(std::u8string()).template get<std::u8string>().empty());
-        const std::u8string with_nul(u8"a\0b", 3);
-        CHECK(json(with_nul).template get<std::u8string>() == with_nul);
-
-        // the lvalue and rvalue paths, and in containers
-        std::u8string lvalue = utf8;
-        CHECK(json(lvalue).template get<std::u8string>() == utf8);
-        CHECK(json(std::move(lvalue)).template get<std::u8string>() == utf8);
-        const std::vector<std::u8string> vec = {utf8, u8"x"};
-        json const j_vec = vec;
-        CHECK(j_vec.is_array());
-        CHECK(j_vec[0].is_string());
-        CHECK(j_vec.template get<std::vector<std::u8string>>() == vec);
-        const std::map<std::string, std::u8string> map = {{"k", utf8}};
-        CHECK(json(map).template get<std::map<std::string, std::u8string>>() == map);
-    }
-
-    SECTION("array of code units (written by versions before 3.13.0)")
-    {
-        json const j = json::array({0x50, 0xC4, 0x9B, 0xC5, 0xA1, 0x69, 0x6E, 0x61});
-        CHECK(j.template get<std::u8string>() == utf8);
-        CHECK(json::array().template get<std::u8string>().empty());
-        CHECK(json::parse("[97, 98]").template get<std::u8string>() == u8"ab");
-
-        std::u8string s = u8"overwritten";
-        j.get_to(s);
-        CHECK(s == utf8);
-    }
-
-    SECTION("other types")
-    {
-#if !defined(JSON_NOEXCEPTION)
-        CHECK_THROWS_WITH_AS(json(1).template get<std::u8string>(), "[json.exception.type_error.302] type must be string or array, but is number", json::type_error);
-        CHECK_THROWS_WITH_AS(json(nullptr).template get<std::u8string>(), "[json.exception.type_error.302] type must be string or array, but is null", json::type_error);
-        CHECK_THROWS_AS(json::object().template get<std::u8string>(), json::type_error);
-        // an element that is not a number
-        CHECK_THROWS_AS(json::array({"a"}).template get<std::u8string>(), json::type_error);
-#endif
-    }
-
-    SECTION("ordered_json")
-    {
-        nlohmann::ordered_json const j = utf8;
-        CHECK(j.is_string());
-        CHECK(j.template get<std::string>() == "P\xc4\x9b\xc5\xa1ina");
-        CHECK(j.template get<std::u8string>() == utf8);
-        CHECK(nlohmann::ordered_json::array({0x61, 0x62}).template get<std::u8string>() == u8"ab");
-    }
-
-    SECTION("std::u8string_view and char8_t arrays are unchanged (arrays of code units)")
-    {
-        const std::u8string_view view = utf8;
-        json const j_view = view;
-        CHECK(j_view.is_array());
-        CHECK(j_view.size() == 8);
-
-        const char8_t arr[] = u8"ab";
-        json const j_arr = arr;
-        CHECK(j_arr == json::array({0x61, 0x62, 0}));
-    }
 }
 #endif
+#endif
 
 #if !defined(JSON_NOEXCEPTION)
 namespace