Review and extend the documentation, and check it in CI (#5638)

* Review and extend the documentation, and check it in CI

A review of all documentation pages found factual errors, dead links,
missing cross-references, and gaps in examples. This fixes them and adds
checks so the same problems are caught automatically.

Fixes:
- wrong signatures and version histories (operator!= C++20 member,
  binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...)
- stale descriptions (number parsing since #5283, UBJSON table, SAX
  example that no longer compiled, tsl::ordered_map advice)
- dead internal and external links; repology.org badges (the domain is
  suspended) replaced by badges that query the registries directly
- deprecation notes link the migration guide; the guide itself fixed

Additions:
- "See also" sections, cross-references, 25 runnable examples, 12
  Mermaid diagrams, new API pages for json_pointer::operator<=> and
  byte_container_with_subtype::operator==/!=
- landing page, guides for untrusted input and performance
- "unreleased" badge after versions newer than the latest release

Checks:
- strict documentation build (broken links/anchors fail it); CI and
  the publish workflow fetch the full history the build needs
- weekly external link check, Mermaid syntax check in CI
- check_structure.py: example titles, heading levels, alt texts,
  header links, docset index coverage; its unused-example check works
  again
- all examples produce the same output on every platform

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the customer links that could not be fixed

A dead link on the customers page is still the evidence of where the
use of the library was documented. Keep the original URLs of the entries
without a working replacement (Marne, Cisco Webex Desk Camera, Philips
Hue, CyberArk) and exclude exactly these URLs from the link check.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Correct the duplicate-key recipe's claim about SAX positions

The SAX interface's key() receives no position either; only parse_error()
does. Also note that the recipe does not report the path to the repeated
key (see discussion #5085).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Say the library is available as a single header and mention json_fwd.hpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Correct documentation errors found while hunting for bugs

- patch/patch_inplace: list the JSON pointer errors parse_error.106-109
  and out_of_range.402/404, and quote the actual parse_error.105 message.
- unflatten: list parse_error.106/107/108 and out_of_range.404.
- to_bson: list out_of_range.415 (binary subtype above 255) and note
  that 412 and 415 are new in 3.13.0.
- to_string: state that string_t must be convertible to std::string, also
  in the StringType requirements table.
- JSON Lines: a `while (input >> j)` loop also throws after the last value
  for concatenated JSON values; show a loop that works for both.
- BON8: a string gets 0xFF only if nothing follows it in the message; a
  string at the end of an array or object is ended by 0xFE.
- custom_string_type.hpp: add operator+=(char), which the "Always
  required" list asks for (json_pointer::to_string, flatten, unflatten,
  and diff did not compile), and an ADL int_to_string for diff and items.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Cache the release headers with functools.lru_cache

Codacy (Pylint) flagged the mutable default argument that header() used
as its cache. functools.lru_cache keeps the same memoization without it.
The script's output is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/scripts/check_version_history.py

@@ -0,0 +1,101 @@
+#!/usr/bin/env python
+"""Check the "Added in version" entries of the macro pages against the git tags.
+
+For every macro documented in docs/api/macros, find the first release tag whose amalgamated header mentions the macro
+and compare it with the version the page's "Version history" names. A macro documented as added *before* it appears
+in any release, or documented with a released version although no release contains it, is reported as a problem. A
+macro that appears in the header *before* its documented version is only a note: many macros existed internally before
+they were documented for users. The check is heuristic and meant to be run by hand, not in CI.
+
+usage: python3 check_version_history.py   (from docs/mkdocs/docs, needs the git tags)
+"""
+
+import functools
+import glob
+import re
+# the script only runs git with fixed arguments and without a shell
+import subprocess  # nosec B404
+import sys
+
+HEADER_PATHS = ["single_include/nlohmann/json.hpp", "src/json.hpp"]  # older releases used src/json.hpp
+VERSION_RE = re.compile(r"[Aa]dded in (?:version )?(\d+)\.(\d+)\.(\d+)")
+NAMED_VERSION_RE = re.compile(r"[Aa]dded `([A-Z0-9_]+)` in (?:version )?(\d+)\.(\d+)\.(\d+)")
+
+
+def release_tags():
+    # fixed git command without a shell
+    tags = subprocess.run(["git", "tag", "-l", "v*"], capture_output=True, text=True, check=True).stdout.split()  # nosec B603, B607
+    versions = []
+    for tag in tags:
+        match = re.fullmatch(r"v(\d+)\.(\d+)\.(\d+)", tag)
+        if match:
+            versions.append((tuple(map(int, match.groups())), tag))
+    return sorted(versions)
+
+
+@functools.lru_cache(maxsize=None)
+def header(tag):
+    for path in HEADER_PATHS:
+        # fixed git command without a shell; the tag names come from "git tag"
+        result = subprocess.run(["git", "show", f"{tag}:{path}"], capture_output=True, text=True)  # nosec B603, B607
+        if result.returncode == 0:
+            return result.stdout
+    return ""
+
+
+def macros_and_versions(page):
+    with open(page, encoding="utf-8") as content:
+        text = content.read()
+    match = re.search(r"^# (.+)$", text, re.MULTILINE) or re.search(r"<h1>(.*?)</h1>", text, re.DOTALL)
+    title = re.sub(r"<[^>]+>|\s+", " ", match.group(1))
+    macros = [x.strip() for x in re.split(r"[,/]", title) if x.strip()]
+    history = text.split("## Version history", 1)[-1]
+    entries = re.split(r"\n(?=\s*(?:\d+\.|-)\s)", history)
+    specific = {}  # entries like "Added `JSON_HAS_CPP_23` in version 3.12.0."
+    general = []
+    for entry in entries:
+        named = NAMED_VERSION_RE.search(entry)
+        if named:
+            specific[named.group(1)] = tuple(map(int, named.groups()[1:]))
+            continue
+        match = VERSION_RE.search(entry)
+        if match:
+            general.append(tuple(map(int, match.groups())))
+    rest = [macro for macro in macros if macro not in specific]
+    if len(general) == len(rest):  # numbered history: one entry per macro, in title order
+        pairs = list(zip(rest, general))
+    else:
+        pairs = [(macro, general[0]) for macro in rest] if general else []
+    return pairs + sorted(specific.items())
+
+
+def main():
+    tags = release_tags()
+    latest = tags[-1][0]
+    problems = notes = 0
+    for page in sorted(glob.glob("api/macros/*.md")):
+        if page.endswith("index.md"):
+            continue
+        for macro, documented in macros_and_versions(page):
+            pattern = re.compile(rf"\b{re.escape(macro)}\b")
+            first = next((version for version, tag in tags if pattern.search(header(tag))), None)
+            fmt = ".".join
+            if first is None:
+                if documented <= latest:
+                    problems += 1
+                    print(f"{page}: {macro} is documented as added in {fmt(map(str, documented))}, "
+                          f"but no release up to {fmt(map(str, latest))} contains it")
+            elif documented < first:
+                problems += 1
+                print(f"{page}: {macro} is documented as added in {fmt(map(str, documented))}, "
+                      f"but first appears in {fmt(map(str, first))}")
+            elif documented > first:
+                notes += 1
+                print(f"{page}: note: {macro} is documented as added in {fmt(map(str, documented))}, "
+                      f"but is mentioned in the header since {fmt(map(str, first))}")
+    print(f"{problems} possible problem(s), {notes} note(s)")
+    return 1 if problems else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())