mirror of
https://github.com/nlohmann/json.git
synced 2026-10-14 18:27:13 +00:00
Niels Lohmann
d5631dccbd
Parsing from a std::istream read one character at a time through the streambuf, so none of the parser's fast paths applied: it was 1.8 to 2.5 times slower than reading the file into a string and parsing that. input_stream_adapter now exposes the streambuf's get area (gptr() to egptr()) to the lexer as a window. The protected get-area members are reached through member pointers formed in a class derived from std::streambuf, which is ordinary, standard-conforming member access. The lexer scans the window with the same bulk paths as a string, and asks the streambuf for the next character as before when the window is used up. Nothing is read ahead, copied, or put back, so the stream is left exactly where it was left before, with and without JSON_PRECISE_STREAM_POSITION, and a streambuf without a get area is still read one character at a time. The adapter also keeps the current token for error messages as a pointer into the get area, and copies it out only before the get area is refilled, instead of the lexer copying every character. On an Apple M1 Max (Apple Clang, -O3), parsing from an std::ifstream is 1.6 to 2.1 times as fast (twitter.json 4.08 -> 2.30 ms, citm_catalog.json 8.75 -> 4.08 ms), and reading 100,000 small values with operator>> 1.4 (objects) to 4.0 (integers) times as fast. With GCC 13 and libstdc++ on x86-64 Linux, std::ifstream is 1.5 to 2.2 times as fast. Parsing from a stream is now within 11 to 18 % of reading the file into a string first. JSON_DISABLE_STREAM_BULK_SCAN restores reading one character at a time, for streambufs that do not follow the usual get-area contract. The new tests compare every observable result (values, exception messages, stream state, remaining characters) for get areas of 1 to 4096 bytes against a streambuf without a get area, and run again with the macro and with JSON_PRECISE_STREAM_POSITION. Signed-off-by: Niels Lohmann <mail@nlohmann.me>