mirror of
https://github.com/nlohmann/json.git
synced 2026-10-14 10:17:12 +00:00
Derive the lexer position lazily for seekable input (#5809)
* Derive the lexer position lazily for seekable input The lexer counted the read position on every character: two increments and a newline check per get(), all on members kept in memory. For seekable input (pointers, strings, other random-access byte ranges) the position can be derived instead: chars_read_total is the adapter's read count (including reads at EOF) minus a pending unget, and line and column are recomputed from the consumed input only when get_position() is called, i.e. on parse errors. JSON_DIAGNOSTIC_POSITIONS uses the new O(1) get_chars_read_total(). Stream, file, wide-string and user-defined adapters keep the eager counters. Parsing pretty-printed input gets 8-12% faster and accept() 12-27% (canada, citm_catalog, twitter; clang 21 arm64 and GCC 13 x86-64). Error messages, byte offsets, SAX positions and diagnostic positions are unchanged (differential comparison against develop on 548k inputs). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the std::vector<std::byte> binary type tests out of unit-regression3 The #4804/#4209 tests instantiated a second basic_json specialization (with_binary_t<std::vector<std::byte>>), including its parser and CBOR reader. With clang 18, the C++20 object of unit-regression3.cpp had 32656 COFF sections on develop; the lazy position helpers pushed it to 32975, past what the MinGW linker of the Windows clang jobs handles ("relocation truncated to fit: IMAGE_REL_AMD64_REL32 against .rdata"). unit-custom-binary-type.cpp already instantiates the same type, so the tests move there; unit-regression3.cpp drops to 27616 sections. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Report the parse errors of the position test via SAX The test caught json::parse_error, so it aborted in the ci_test_noexceptions build, where JSON_NOEXCEPTION turns the throw into std::abort(). sax_parse() hands the error position and the exception object to parse_error() without throwing, so the test now records them there and runs in every configuration. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
1 parent
40a62947d3
commit
f23830b65e
9 files changed
+380
-105
No files matched your search
@@ -370,7 +370,7 @@ class parser
|
||||
}
|
||||
case token_type::end_of_input:
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(m_lexer.get_position().chars_read_total == 1))
|
||||
if (JSON_HEDLEY_UNLIKELY(m_lexer.get_chars_read_total() == 1))
|
||||
{
|
||||
return syntax_error(*sax, "attempting to parse an empty input; check that your input string or stream contains the expected JSON");
|
||||
}
|
||||
@@ -520,8 +520,10 @@ class parser
|
||||
{
|
||||
// MSVC 2015 reports C4100 (unreferenced parameter) when SAX::parse_error is static
|
||||
static_cast<void>(sax);
|
||||
return sax.parse_error(m_lexer.get_position(), m_lexer.get_token_string(),
|
||||
parse_error::create(exception_id::syntax_error, m_lexer.get_position(), message, nullptr));
|
||||
// computing the position is O(input read so far) for seekable input
|
||||
const position_t position = m_lexer.get_position();
|
||||
return sax.parse_error(position, m_lexer.get_token_string(),
|
||||
parse_error::create(exception_id::syntax_error, position, message, nullptr));
|
||||
}
|
||||
|
||||
std::string exception_message(const token_type expected, const std::string& context)
|
||||
|
||||
Reference in new issue
Block a user