amorfo77/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-06 14:40:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

Speedup; check for the expected separator before the lexer's token switch (#5592)

Browse Source 
* Check for the expected separator before the lexer's token switch

After a key the parser expects ':', after a value usually ','. Test for
that character first instead of going through scan()'s switch, which
compiles to an indirect jump. Any other character takes the old path,
so tokens and error messages are unchanged.

Parsing 6.3% faster with GCC 15.2 and 2.7% with Clang 22.1 (geomean of
the ParseString, ParseFile and ParseIndented benchmarks).

Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>

* Improvement: address PR comments

Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>

* fix: address comments

Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>

* Fix clang-tidy bugprone-signed-char-misuse in scan_expecting

Convert the expected separator through unsigned char before storing it as
char_int_type. The generated code is unchanged.

Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>

* Use raw string literals in the separator comment tests

Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>

---------

Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
Co-authored-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com>
This commit is contained in:
Michiel van SlobbeandMichiel van Slobbe authored and GitHub committed 2026-10-06 08:36:40 +02:00
1 parent 0490778fc3
commit 5ecb704f6b
4 files changed
+140 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/src/unit-class_parser.cpp
+52
View File
@@ -2317,6 +2317,58 @@ TEST_CASE("parser class")
#endif
}
SECTION("comments before separators")
{
// The parser first checks for the expected ':' or ',' and only then
// falls back to the full token switch, which skips comments. A comment
// directly before a separator takes that fallback.
json _;
SECTION("ignored")
{
const std::vector<std::pair<std::string, json>> inputs =
{
{"{\"a\" /* c */ : 1}", {{"a", 1}}},
{"{\"a\" // c\n: 1}", {{"a", 1}}},
{R"({"a": 1, "b" /* c */ : 2})", {{"a", 1}, {"b", 2}}},
{R"({"a": 1 /* c */ , "b": 2})", {{"a", 1}, {"b", 2}}},
{"{\"a\": 1 // c\n, \"b\": 2}", {{"a", 1}, {"b", 2}}},
{"[1 /* c */ , 2]", {1, 2}},
{"[1 // c\n, 2]", {1, 2}},
{"{\"a\" /* c */ /* d */ : [1 // c\n , 2 /**/ ] /**/ , \"b\" : 3}", {{"a", {1, 2}}, {"b", 3}}}
};
for (const auto& input : inputs)
{
CAPTURE(input.first)
CHECK(json::parse(input.first, nullptr, true, true) == input.second);
CHECK(json::accept(input.first, true));
}
}
SECTION("ignored, with trailing commas")
{
CHECK(json::parse(std::string("[1 /* c */ , ]"), nullptr, true, true, true) == json({1}));
CHECK(json::parse(std::string("{\"a\": 1 /* c */ , }"), nullptr, true, true, true) == json({{"a", 1}}));
CHECK_THROWS_WITH_AS(_ = json::parse(std::string("[1 /* c */ , ]"), nullptr, true, true),
"[json.exception.parse_error.101] parse error at line 1, column 14: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal", json::parse_error);
CHECK_THROWS_WITH_AS(_ = json::parse(std::string("{\"a\": 1 /* c */ , }"), nullptr, true, true),
"[json.exception.parse_error.101] parse error at line 1, column 19: syntax error while parsing object key - unexpected '}'; expected string literal", json::parse_error);
}
SECTION("not ignored")
{
CHECK_THROWS_WITH_AS(_ = json::parse(std::string("{\"a\" /* c */ : 1}")),
"[json.exception.parse_error.101] parse error at line 1, column 6: syntax error while parsing object separator - invalid literal; last read: '\"a\" /'; expected ':'", json::parse_error);
CHECK_THROWS_WITH_AS(_ = json::parse(std::string("{\"a\": 1, \"b\" /* c */ : 2}")),
"[json.exception.parse_error.101] parse error at line 1, column 14: syntax error while parsing object separator - invalid literal; last read: '\"b\" /'; expected ':'", json::parse_error);
CHECK_THROWS_WITH_AS(_ = json::parse(std::string("{\"a\": 1 /* c */ , \"b\": 2}")),
"[json.exception.parse_error.101] parse error at line 1, column 9: syntax error while parsing object - invalid literal; last read: '1 /'; expected '}'", json::parse_error);
CHECK_THROWS_WITH_AS(_ = json::parse(std::string("[1 /* c */ , 2]")),
"[json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing array - invalid literal; last read: '1 /'; expected ']'", json::parse_error);
CHECK(!json::accept(std::string("[1 /* c */ , 2]")));
}
}
#if JSON_DIAGNOSTIC_POSITIONS
// Macro for all test cases for start_pos and end_pos
#define SETUP_TESTCASES() \