Read a std::u8string from a JSON string and store it as one in every configuration (#6075)

* Read a std::u8string from a JSON string and store it as one in every configuration

In C++20, json(std::u8string) was stored as a JSON string only when
std::filesystem::path support was available (the to_json overload sat
inside the JSON_HAS_FILESYSTEM guard), and there was no matching
from_json, so get<std::u8string>() threw type_error.302 ("type must be
array, but is string"). Without std::filesystem, the same expression
still produced an array of code units (the 3.12.0 behavior).

Move the to_json overload out of the filesystem guard so that a
std::u8string is a string in every configuration, build the value
directly from the code units (this also works for a custom string_t
such as the tests' alt_string), and add a from_json overload that reads
a JSON string. The array of numbers written by 3.12.0 is still accepted;
other types throw type_error.302 ("type must be string or array").
std::u8string_view and char8_t arrays are unchanged.

The tests now run in C++20/23 (the file mentions JSON_HAS_CPP_20/23),
also without std::filesystem (test-conversions2_no_filesystem), and with
ordered_json and a custom string type. The new behavior is documented.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fill the std::u8string with resize and indexing instead of push_back

The push_back loop is not vectorized: reading a 1 kB string took about
2.2 us against 0.39 us with resize and direct element access (clang -O2).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Write the std::u8string from_json template header in the compact form

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Ignore libstdc++ char to char8_t sign change in the clang sanitizer job

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/basic_json.md

@@ -67,7 +67,11 @@ basic_json(basic_json&& other) noexcept;
     - **objects**: [`object_t`](object_t.md) and all kinds of compatible associative containers such as `std::map`,
      `std::unordered_map`, `std::multimap`, and `std::unordered_multimap` with a `key_type` compatible to `string_t`
      and a `value_type` from which a `basic_json` value can be constructed.
-    - **strings**: `string_t`, string literals, and all compatible string containers can be used.
+    - **strings**: `string_t`, string literals, and all compatible string containers can be used. Since version 3.13.0,
+      this includes `#!cpp std::u8string` (C++20): its code units are stored as a JSON string and
+      [`get<std::u8string>()`](get.md) reads it back (in version 3.12.0, a `#!cpp std::u8string` was stored as an array
+      of numbers; `get<std::u8string>()` still accepts such an array). `#!cpp std::u8string_view` and `#!cpp char8_t`
+      arrays are not strings; they are stored as arrays of code units.
 
     !!! note "Rvalue containers are moved from"
 

docs/mkdocs/docs/api/basic_json/get.md

@@ -130,6 +130,12 @@ overload (3).
     `#!cpp json(true).get<double>()` throw. This is the behavior of all 3.x versions;
     [`basic_json_view::get<T>()`](../basic_json_view/get.md) behaves the same.
 
+!!! note "`std::u8string` conversions"
+
+    Since version 3.13.0, `#!cpp get<std::u8string>()` (C++20) reads a JSON string; it also accepts an array of numbers,
+    the form in which version 3.12.0 stored a `#!cpp std::u8string`. Any other JSON type throws
+    [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) ("type must be string or array").
+
 !!! note "`std::optional` conversions"
 
     Prior to version 3.13.0, `#!cpp get<std::optional<T>>()` (and other conversions to `std::optional<T>`) failed to