Parse JSON in one block of memory with a fused pointer-based parser (#6031)

* Parse JSON in one block of memory with a fused pointer-based parser

The parser and the lexer exchange one token at a time, and the lexer keeps
its state (the current character, a pending unget, where the token starts)
in members, which every character pays for: json::accept() of twitter.json
took 2.8x as long as json_document::accept(). detail::contiguous_parser
reads text in one block of memory through a local pointer, with the
grammar, the whitespace and the tokens in one loop (after yyjson's reader),
and drives a SAX handler: parse()'s DOM handler, accept()'s handler, or the
user's handler of sax_parse(). It is used with the default options (no
callback, no comments, no trailing commas; sax_parse() with strict = true).

Errors are reported by the library parser, so the exception, its message and
its position stay the same. contiguous_parser stops at the token that the
library parser would reject, before reporting it, so the events up to the
error are the same; then:
- parse() runs the library parser with a handler that only reports the
  error (json_sax_error_reporter), with the lexer of parse(), which is
  instantiated anyway;
- sax_parse() runs it with a handler that forwards only parse_error() to
  the user's handler (json_sax_error_forwarder), so no event arrives twice;
- accept() returns the verdict.
A handler that returns false stops the parse as before. An error at the end
of a large input costs parse() about 1.2x of what it cost before.

On an Apple M1 Max: accept() 1.6-2.0x as fast, parse() 10-16% faster,
ordered_json 11-16%, sax_parse() with a user's handler 1.36-1.46x, small
messages 12-22% (parse()) and 2.1-2.4x (accept()). On x86-64 (GCC 15,
Clang 21): accept() 1.6-1.9x, parse() 5-17%, ordered_json 12-19%,
sax_parse() 1.32-1.57x. A translation unit that parses a string compiles 3%
(-O0) and 9% (-O2) slower.

unit-contiguous_parser.cpp compares the contiguous path with the byte path
(std::list iterators) on corner cases, generated documents, their
truncations and mutations, deep nesting, and narrow number types: the same
values with the same number types and float bits, exception messages,
accept() verdicts and sax_parse() events.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: unused parameter without exceptions, cpplint, BUILD.bazel

- json_sax_error_reporter::parse_error: mark the exception unused, as
  JSON_THROW does not use it with JSON_NOEXCEPTION
- unit-contiguous_parser.cpp: compare the exceptions of both paths only
  when exceptions are enabled (invalid input aborted the test otherwise)
- silence cpplint's readability/braces false positives on the if
  statements inside the parser's macros
- add contiguous_parser.hpp to BUILD.bazel

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* accept(): skip digits without computing their value

accept() needs the value of a number only for a float whose exponent may
reach the largest value. skip_digits() finds the end of the digits without
the multiply-adds of accumulate_digits(), and once an 8-byte word fails the
digit test, the remaining loop needs no bounds check: a non-digit is in
that word. The value is computed afterwards for the rare floats that need
it.

accept() of canada.json: -16 % on Apple M1, -22 % with GCC 15 and -21 %
with Clang 21 on x86-64; files with many numbers -9 to -15 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: mark parser callback in unit-contiguous_parser noexcept

Fixes the -Werror=noexcept failure with GCC 26 (ci_test_standards_gcc (26)).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: sweep_allocator::construct() accepts pointers to const

Recent libc++ (emsdk latest) probes construct(const T*, const T&) in an
unevaluated context while checking for trivial copies; the static_cast to
void* in the noexcept specification then fails to compile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit 794bf70823c12a2a1cb2a8a95f34dfb6357746a8)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: guard sweep_allocator::construct() against const types instead of const_cast

The const_cast added in the previous commit is rejected by
cppcoreguidelines-pro-type-const-cast (clang-tidy). Make construct() not
viable for const U instead, which is what libc++ probes for.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit de099fa607001b710862fd1bbecef3609c54cbc4)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: astyle and clang-tidy for sweep_allocator::construct()

astyle 3.4.13 reformats the enable_if<!...> default template argument
(Check amalgamation). Delete the const overload instead, which makes the
libc++ probe for const objects fail in the SFINAE context just the same
and needs no <type_traits>.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Contiguous parser: review fixes (wording, macros, diagnostic positions)

- "default parser" instead of "library parser" in comments and tests
- the three comments in json.hpp say the same thing
- current() replaces the repeated `p != end ? *p : 0`
- the backslashes of each macro are aligned
- the docs say that JSON_DIAGNOSTIC_POSITIONS turns the fast path off

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

docs/mkdocs/docs/api/macros/json_diagnostic_positions.md

@@ -34,6 +34,9 @@ or quotes.
 Note that enabling this macro increases the size of every JSON value by two `std::size_t` fields and adds slight runtime
 overhead to parsing, copying JSON value objects, and the generation of error messages for exceptions. It also causes
 these values to be reported in those error messages.
+In addition, `parse()`, `accept()`, and `sax_parse()` no longer use the
+[faster path for input in one block of memory](../../features/parsing/index.md#input-sources-and-performance), so
+parsing a string or a buffer is slower than without the macro.
 
 If [`JSON_DIAGNOSTICS`](json_diagnostics.md) is also set, the message contains the path and the byte range, such as
 `(/1) (bytes 4-8)`. An exception about the root value has no path and only the range, such as `(bytes 0-4)`.

docs/mkdocs/docs/api/macros/json_disable_stream_bulk_scan.md

@@ -48,8 +48,8 @@ The default value is `0`: streams are scanned in bulk.
 !!! note "Performance"
 
     Parsing from a `std::ifstream` or `std::istringstream` is typically 1.5 to 2 times as fast as before, and reading
-    many small values from one stream with `operator>>` 1.4 to 4 times as fast. Parsing from a stream is then within
-    about 10 to 20 % of first reading the whole file into a `std::string` and parsing that.
+    many small values from one stream with `operator>>` 1.4 to 4 times as fast. Parsing from a stream then takes about
+    1.15 to 1.45 times as long as first reading the whole file into a `std::string` and parsing that.
 
     A stream buffer without a get area (one that only overrides `underflow()` and `uflow()` and hands out every
     character from there) offers nothing to scan in bulk, so it is read one character at a time as before.

docs/mkdocs/docs/features/files.md

@@ -32,11 +32,11 @@ json read_json_file(const std::string& path)
   column 1 instead of saying that the file is missing. A relative path is resolved against the working directory of the
   running program, which is often not the directory of the source file (see the
   [FAQ](../home/faq.md#parsing-a-file-fails-with-unexpected-end-of-input)).
-- **Don't read the file into a string first.** Parsing from a `#!cpp std::ifstream` is only about 10 to 20 % slower
-  than reading the whole file into a string with one `read()` and parsing that, and it needs no memory for the copy (see
-  [input sources and performance](parsing/index.md#input-sources-and-performance)). The common way to read a file with
-  `#!cpp std::istreambuf_iterator<char>` is so slow that it takes about twice as long as parsing from the stream. Read
-  the file into a string only if you need the text itself, for example for an error message. This advice is for
+- **Read the file into a string only if the memory is no concern.** Parsing from a `#!cpp std::ifstream` takes about
+  1.15 to 1.45 times as long as reading the whole file into a string with one `read()` and parsing that, but it needs no
+  memory for the copy (see [input sources and performance](parsing/index.md#input-sources-and-performance)). The common
+  way to read a file with `#!cpp std::istreambuf_iterator<char>` is so slow that it takes about twice as long as parsing
+  from the stream. This advice is for
   `#!cpp json::parse`. [`json_document`](json_view/index.md) reads a stream into a buffer that it owns anyway, so there is no
   copy to save: reading the file into a `#!cpp std::string` yourself and passing that as an lvalue, which the document
   borrows, was about 6 to 12 % faster than parsing from the stream, with the same single buffer.

docs/mkdocs/docs/features/parsing/index.md

@@ -80,8 +80,16 @@ them faster than others:
 | `#!cpp FILE*`                                                                              | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; the file is locked once for the whole parse |
 | other iterator ranges (e.g., `#!cpp std::list<char>`, `#!cpp std::istreambuf_iterator<char>`), wide strings | one character at a time                                                             |
 
-Parsing a file from a `#!cpp std::ifstream` is therefore only about 10 to 20 % slower than reading the whole file into a
-`#!cpp std::string` first and parsing that, and it needs no memory for a copy of the file. The stream is never read
+Input in one block of memory is read even faster with the default options (no
+[parser callback](../../api/basic_json/parser_callback_t.md), no comments, no trailing commas, and for `sax_parse()`
+`strict = true`): `parse()`, `accept()`, and `sax_parse()` then read the grammar and the tokens in one loop over the
+memory. Only after an error do `parse()` and `sax_parse()` read the input again, to report the error exactly as before.
+This faster path is not used when [`JSON_DIAGNOSTIC_POSITIONS`](../../api/macros/json_diagnostic_positions.md) is
+enabled, because the positions are recorded by the default parser: the input is then parsed as before, at the same speed
+as before.
+
+Parsing a file from a `#!cpp std::ifstream` therefore takes about 1.15 to 1.45 times as long as reading the whole file
+into a `#!cpp std::string` first and parsing that, but it needs no memory for a copy of the file. The stream is never read
 ahead: after parsing, it is positioned exactly as if every character had been read one at a time, so
 [`operator>>`](../../api/operator_gtgt.md) can still read several values from one stream (see also
 [JSON Lines](json_lines.md)).

docs/mkdocs/docs/features/performance.md

@@ -29,9 +29,9 @@ up. Nothing is read ahead, so the stream is left exactly where it would be after
 [`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
 area is read one character at a time. A `#!cpp FILE*` is locked once per parse instead of once per character, and with
 glibc and the Apple C library, the characters in its buffer are scanned in bulk as a stream's get area is. See
-[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. According to the
-description of [#5828](https://github.com/nlohmann/json/pull/5828), parsing from a stream is within about 10 to 20 % of
-parsing an equivalent `#!cpp std::string`.
+[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. Parsing from a
+stream takes about 1.15 to 1.45 times as long as parsing an equivalent `#!cpp std::string`, which `parse()` reads with
+a parser of its own (see there).
 
 Inputs that are still read one character at a time: iterator ranges that are not contiguous (e.g.
 `#!cpp std::list<char>` or `#!cpp std::istreambuf_iterator<char>`), wide strings, and user-defined adapters. An