Parsing from a std::istream read one character at a time through the streambuf, so none of the parser's fast paths applied: it was 1.8 to 2.5 times slower than reading the file into a string and parsing that. input_stream_adapter now exposes the streambuf's get area (gptr() to egptr()) to the lexer as a window. The protected get-area members are reached through member pointers formed in a class derived from std::streambuf, which is ordinary, standard-conforming member access. The lexer scans the window with the same bulk paths as a string, and asks the streambuf for the next character as before when the window is used up. Nothing is read ahead, copied, or put back, so the stream is left exactly where it was left before, with and without JSON_PRECISE_STREAM_POSITION, and a streambuf without a get area is still read one character at a time. The adapter also keeps the current token for error messages as a pointer into the get area, and copies it out only before the get area is refilled, instead of the lexer copying every character. On an Apple M1 Max (Apple Clang, -O3), parsing from an std::ifstream is 1.6 to 2.1 times as fast (twitter.json 4.08 -> 2.30 ms, citm_catalog.json 8.75 -> 4.08 ms), and reading 100,000 small values with operator>> 1.4 (objects) to 4.0 (integers) times as fast. With GCC 13 and libstdc++ on x86-64 Linux, std::ifstream is 1.5 to 2.2 times as fast. Parsing from a stream is now within 11 to 18 % of reading the file into a string first. JSON_DISABLE_STREAM_BULK_SCAN restores reading one character at a time, for streambufs that do not follow the usual get-area contract. The new tests compare every observable result (values, exception messages, stream state, remaining characters) for get areas of 1 to 4096 bytes against a streambuf without a get area, and run again with the macro and with JSON_PRECISE_STREAM_POSITION. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
5.8 KiB
nlohmann::operator>>(basic_json)
std::istream& operator>>(std::istream& i, basic_json& j);
Deserializes an input stream to a JSON value.
Parameters
i(in, out)- input stream to read a serialized JSON value from
j(in, out)- JSON value to write the deserialized input to
Return value
the stream i
Exception safety
Strong guarantee: if an exception is thrown, there are no changes in j.
Exceptions
- Throws
parse_error.101in case of an unexpected token, or ifihas no stream buffer (#!cpp i.rdbuf() == nullptr, for instance#!cpp std::istream(nullptr)). - If reading from
ireaches the end of the input andeofbitis part ofi'sexceptions()mask, thestd::ios_base::failurethrown byiitself propagates instead of aparse_error, the same as it would for the standard library's own extraction operators.
Complexity
Linear in the length of the input. The parser is a predictive LL(1) parser.
Notes
A UTF-8 byte order mark is silently ignored.
Invalid Unicode escapes and unpaired surrogates in the input are reported as
parse_error.101 with a detailed message.
operator>> parses exactly one JSON value, so it can be called repeatedly to read a sequence of concatenated JSON
values from the same stream:
json j1, j2;
input >> j1; // parses the first value
input >> j2; // parses the next value
!!! warning "A number must be followed by whitespace"
A number is only terminated by the character that follows it. That character is read from the stream to detect the
end of the number, and it is **not** put back. When a value that is a number is immediately followed by the next
value, the first character of that next value is lost:
```cpp
std::istringstream input("1true");
json j1, j2;
input >> j1; // j1 == 1
input >> j2; // throws parse_error.101: the stream now starts at "rue"
```
Separating the values with whitespace avoids this, because the character that is eaten is then the separator:
```cpp
std::istringstream input("1 true");
json j1, j2;
input >> j1; // j1 == 1
input >> j2; // j2 == true
```
Only numbers are affected. Values ending in a self-delimiting character do not read past themselves, so
`truefalse`, `[1][2]`, `{"a":1}{"b":2}`, and `"a""b"` can be read back to back without a separator.
Define [`JSON_PRECISE_STREAM_POSITION`](macros/json_precise_stream_position.md) to `1` to leave the terminating character in the stream
instead, so that the stream is positioned right after the value for every value type and no separator is
needed. This is tracked in [#5340](https://github.com/nlohmann/json/issues/5340).
Note that reading concatenated values does not work for JSON Lines (newline-delimited JSON) input -- see that page for why and for the recommended alternative.
By default, a '\0' (NUL) byte encountered while reading a value is treated as end of input, rather than as an
ordinary (and, outside of a string, invalid) byte; see the FAQ entry for
details and the JSON_STRICT_NUL_HANDLING macro to opt into rejecting it
instead. Because operator>> only parses a single value and does not require the rest of the stream to be consumed,
a NUL byte after a complete value has no effect on operator>> either way; it only matters while a value is still
being read.
!!! warning "Deprecation"
This function replaces function `#!cpp std::istream& operator<<(basic_json& j, std::istream& i)` which has
been deprecated in version 3.0.0. It will be removed in version 4.0.0. Please replace calls like `#!cpp j << i;`
with `#!cpp i >> j;`.
See the [migration guide](../integration/migration_guide.md#parsing) for how to update existing code.
Examples
??? example
The example below shows how a JSON value is constructed by reading a serialization from a stream.
```cpp
--8<-- "examples/operator_deserialize.cpp"
```
Output:
```json
--8<-- "examples/operator_deserialize.output"
```
See also
- accept - check if the input is valid JSON
- parse - deserialize from a compatible input
JSON_STRICT_NUL_HANDLING- opt in to rejecting a NUL byte in the input instead of treating it as end of inputJSON_PRECISE_STREAM_POSITION- opt in to leaving the stream positioned right after a numberJSON_DISABLE_STREAM_BULK_SCAN- read the stream one character at a time
Version history
- Added in version 1.0.0.
JSON_STRICT_NUL_HANDLINGadded in version 3.13.0 to optionally reject a NUL byte in the input instead of treating it as end of input; planned to become the default in version 4.0.0.JSON_PRECISE_STREAM_POSITIONadded in version 3.13.0 to optionally leave the character that terminates a number in the stream; planned to become the default in version 4.0.0.- Fixed a null pointer dereference for an
std::istreamwithout a stream buffer (now throwsparse_error.101), and a crash (std::terminate) whenihaseofbitin its exception mask, in version 3.13.0. - Changed to the strong exception safety guarantee in version 3.13.0:
jis no longer left with a partially parsed value if parsing throws. - Scans the characters in the stream buffer of
iin bulk in version 3.13.0; the position ofiafter the call is unchanged (JSON_DISABLE_STREAM_BULK_SCANrestores reading one character at a time).