Read BJData ND-arrays of float16, restrict ND-array char to 0..127, correct BJData docs (#5931)

* Read ND-arrays of half floats and enforce the char limit in ND-arrays

BJData N-dimensional arrays with element type 'h' (float16) failed to
read with a misleading parse_error.112, because the reader's dtype name
table lacked 'h'. It now maps to "half", the JData name. The writer still
does not pack "half"; such an object is written as a plain object and
reads back equal.

BJData limits 'C' (char) to 0..127 in Draft 2 and Draft 3, and the 1-D
reader enforces this, but ND-arrays with _ArrayType_ "char" wrote values
up to 255 as 'C' elements and read them back without a check. The writer
now only packs values in 0..127 (others fall back to a plain object that
round-trips), and the reader rejects 'C' elements above 127 with
parse_error.113 like the 1-D reader.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: correct and complete the BJData documentation

Verify the BJData chapter against the library and the Draft 2/3/4
specifications and fix what did not match:

- bjdata.md: correct the list of unused markers (N, not Z), the ND-array
  example (3x2, not 2x3), the conditions under which an object is packed
  as an ND-array (exactly three keys, row vectors, float-only elements for
  single/double, char 0..127, byte only in Draft 3), the binary table row
  and notes (which flag/draft combinations write [$B), the "$" type
  restriction (Draft 3, Z is excluded too), the H marker in the
  deserialization table, and the claim of a complete deserialization
  mapping. Document the supported drafts, that Draft 4 (SoA containers,
  extension type E) and column-major ND-arrays are rejected, ND-arrays of
  half floats, duplicate object keys, and link the Draft 3 specifications
  instead of master.
- Add an API page for basic_json::bjdata_version_t with an example, and
  link it from the nav, the API index and to_bjdata.
- Update binary_formats/index.md and binary_values.md, which still claimed
  that BJData has no binary type, and fix a wrong byte comment there.
- Document parse_error.115 and the out_of_range.408 cases for from_bjdata,
  the precise other_error.502 condition for to_bjdata, and reword the note
  on truncated unsigned integers in exceptions.md (only ND-array elements
  were truncated before 3.13.0).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate single_include and apply astyle

Result of make amalgamate (including make pretty) for the preceding changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the BJData documentation

The $ type restriction exists since Draft 2 (Draft 3 added B), $ requires a count, ND char above 127 is rejected on read, and the new API page is listed in alphabetical order in the nav.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: format tables, use the Oxford comma

Reformat the tables changed by this PR (BJData serialization and
deserialization mappings, completeness and binary-value tables) so all
rows are aligned, and add the Oxford comma to lists of three or more
items in the added text.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

include/nlohmann/detail/input/binary_reader.hpp

@@ -3277,6 +3277,8 @@ class binary_reader
             // get_ubjson_size_value() (the type marker is known before the dimension vector
             // that determines size_and_type.first is read, so it is emitted first there to
             // match the documented _ArrayType_, _ArraySize_, _ArrayData_ key order)
+            // 'C' elements are read as unsigned bytes, but BJData limits a char to 0..127
+            const bool nd_is_char = size_and_type.second == 'C';
             if (size_and_type.second == 'C' || size_and_type.second == 'B')
             {
                 size_and_type.second = 'U';
@@ -3290,6 +3292,24 @@ class binary_reader
 
             for (std::size_t i = 0; i < size_and_type.first; ++i)
             {
+                if (nd_is_char)
+                {
+                    get();
+                    if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("char")))
+                    {
+                        return false;
+                    }
+                    if (JSON_HEDLEY_UNLIKELY(current > 127))
+                    {
+                        return last_byte_error(exception_id::invalid_string_or_size, "byte after 'C' must be in range 0x00..0x7F", "char", "");
+                    }
+                    if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(current))))
+                    {
+                        return false;
+                    }
+                    continue;
+                }
+
                 if (JSON_HEDLEY_UNLIKELY(!get_ubjson_value(size_and_type.second)))
                 {
                     return false;
@@ -4655,6 +4675,8 @@ class binary_reader
                 return "uint8";
             case 'd':
                 return "single";
+            case 'h':
+                return "half";
             case 'i':
                 return "int8";
             case 'l':

include/nlohmann/detail/output/binary_writer.hpp

@@ -3848,7 +3848,7 @@ class binary_writer
     A C++11 `constexpr` function cannot contain a `switch`, so this is a plain
     comparison chain instead; it is only reached once per ND-array candidate
     object. Keep in sync with binary_reader's `bjd_type_name()`, which maps
-    the other way.
+    the other way (it also reads "half", which is never written).
     */
     static CharType bjdata_ndarray_type_marker(const string_t& name)
     {
@@ -4056,10 +4056,13 @@ class binary_writer
             switch (dtype)
             {
                 case 'U':
-                case 'C':
                 case 'B':
                     ok = write_bjdata_ndarray_element<std::uint8_t>(el, dry_run);
                     break;
+                case 'C':
+                    // BJData limits a char to 0..127 (the intersection of the int8 and uint8 ranges)
+                    ok = bjdata_ndarray_value_in_range<std::int8_t>(el) && bjdata_ndarray_value_in_range<std::uint8_t>(el);
+                    break;
                 case 'i':
                     ok = write_bjdata_ndarray_element<std::int8_t>(el, dry_run);
                     break;