Update performance docs for fast std::istream and FILE* parsing (#5875)

performance.md still claimed streams have no fast path (outdated since #5828).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann authored and GitHub committed 2026-10-21 12:25:10 +02:00
1 parent bd5227c2ec
commit ef1cdb59ac
1 file changed
+26 -19
+26 -19
View File
@@ -20,32 +20,39 @@ below apply on every supported standard. This adapter exposes two optimizations
- it can reconstruct already-consumed input on demand for error messages, instead of copying every character as it is - it can reconstruct already-consumed input on demand for error messages, instead of copying every character as it is
read, and read, and
- the lexer can scan ordinary string characters directly out of the buffer, several bytes at a time, rather than one - the lexer can scan ordinary string characters, numbers, and indentation directly out of the buffer, several bytes at
character (and one function call) at a time. a time, rather than one character (and one function call) at a time.
A `#!cpp std::istream` (including `#!cpp std::ifstream`) or `#!cpp FILE*`, by contrast, is read through A `#!cpp std::istream` (including `#!cpp std::ifstream`) gets the same bulk scanning: `input_stream_adapter` scans the
`input_stream_adapter` or `file_input_adapter`, which read one character (or one block, for binary formats) at a time characters its stream buffer already holds (the *get area*) and asks the stream buffer for more only when they are used
and expose neither optimization -- the lexer falls back to the same byte-at-a-time path it uses for any up. Nothing is read ahead, so the stream is left exactly where it would be after reading one character at a time, and
non-contiguous, general-purpose iterator range. An iterator pair over non-contiguous but random-access storage (e.g. [`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
`#!cpp std::deque<char>::iterator`) gets the first optimization but not the second, since the byte-scanning fast path area is read one character at a time. A `#!cpp FILE*` is still read one character at a time, but `file_input_adapter`
additionally requires contiguous storage. locks the file once per parse instead of once per character. See
[input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. According to the
description of [#5828](https://github.com/nlohmann/json/pull/5828), parsing from a stream is within about 10 to 20 % of
parsing an equivalent `#!cpp std::string`.
Practically: if the JSON text is already in memory, or small enough to read into memory, prefer passing a Inputs that are still read one character at a time: iterator ranges that are not contiguous (e.g.
`#!cpp std::string`, a `#!cpp std::vector<char>`, or a pointer range to `parse` over a `#!cpp std::istream`. For a `#!cpp std::list<char>` or `#!cpp std::istreambuf_iterator<char>`), wide strings, and user-defined adapters. An
file, that means reading it into a string first and then parsing the string, rather than passing a iterator pair over non-contiguous but random-access storage (e.g. `#!cpp std::deque<char>::iterator`) gets the first
`#!cpp std::ifstream` directly to `parse` -- the latter never benefits from either optimization: optimization but not the second, since the byte-scanning fast path requires contiguous storage.
Practically: a file can be passed directly to `parse`, which also avoids the memory for a copy of the file. Reading it
into a string first is no longer necessary for speed:
```cpp ```cpp
// gets the contiguous fast paths // fast: the stream buffer's get area is scanned in bulk
std::ifstream f("example.json"); std::ifstream f("example.json");
std::string contents((std::istreambuf_iterator<char>(f)), std::istreambuf_iterator<char>()); json j = json::parse(f);
json j = json::parse(contents);
// does not: input_stream_adapter has no fast path
std::ifstream f2("example.json");
json j2 = json::parse(f2);
``` ```
Two macros switch the stream optimizations off without changing any result:
[`JSON_DISABLE_STREAM_BULK_SCAN`](../api/macros/json_disable_stream_bulk_scan.md) for stream buffers that do not follow
the usual `#!cpp std::streambuf` contract, and
[`JSON_DISABLE_UNLOCKED_FILE_IO`](../api/macros/json_disable_unlocked_file_io.md) for programs in which another thread
must use the same `#!cpp FILE` object while it is parsed.
For contiguous input with many non-ASCII characters, [`JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md) can For contiguous input with many non-ASCII characters, [`JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md) can
additionally speed up UTF-8 validation by using the [simdutf](https://github.com/simdutf/simdutf) library instead of additionally speed up UTF-8 validation by using the [simdutf](https://github.com/simdutf/simdutf) library instead of
the built-in scalar validator; streaming inputs (files, `#!cpp std::istream`, wide strings, user-defined adapters) the built-in scalar validator; streaming inputs (files, `#!cpp std::istream`, wide strings, user-defined adapters)