amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-03 13:10:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Scan the strings of json_view with NEON and SSE2

Browse Source 
Long runs of string bytes are scanned 16 at a time with NEON (AArch64, with
GCC and Clang) and SSE2 (x86-64): both belong to the baseline instruction
sets. A signed compare with 0x20 finds control characters and non-ASCII
bytes at once. Keys keep 16 table checks before the vector loop (their
lengths repeat from record to record, so the branches predict well);
string values have 8, as their lengths vary more.

Non-ASCII text is validated 16 bytes at a time with the "lookup4" check of
simdjson (J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One
Instruction Per Byte", 2021): with NEON, and on x86-64 with SSSE3 if
JSON_VIEW_USE_SSSE3 is defined (SSSE3 is not part of x86-64, and the code
must not depend on the flags of a translation unit). JSON_VIEW_NO_SIMD
selects the portable code. The vector code sits in
detail/view/simd.hpp; the same input is accepted either way.

json_document::parse, best of 7 runs in separate processes (M1 Max):
poet.json (CJK text) -72%, random.json -25%, twitter.json -22%,
gsoc-2018.json -20%, semanticscholar -19%, github_events -11%,
apache_builds -9.5%, canada/citm -5/-6%; lottie +4%, tree-pretty +2.5%.

Tests: every two-byte sequence and three- and four-byte sequences with
continuation bytes at the edges of their ranges, at every offset around
the vector blocks of keys and values, cut short, and long runs of text
with a damaged byte, against json::accept and json::parse. CMake builds
the parser tests again with JSON_VIEW_NO_SIMD, and on x86-64 with
JSON_VIEW_USE_SSSE3 and -mssse3; the macros are documented.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-09-29 14:03:06 +02:00
parent a491cc7663
commit ebfbbf29cb
15 changed files with 863 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/CMakeLists.txt
+20
View File
@@ -324,6 +324,26 @@ json_test_add_test_for(src/unit-diagnostic-positions.cpp
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
# the json_view parser again with the portable string scanning instead of
# NEON/SSE2, and on x86-64 with the SSSE3 UTF-8 check (JSON_VIEW_USE_SSSE3)
json_test_set_test_options(test-json_view_builder_portable
COMPILE_DEFINITIONS JSON_VIEW_NO_SIMD
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_portable
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64)$" AND NOT MSVC)
json_test_set_test_options(test-json_view_builder_ssse3
COMPILE_DEFINITIONS JSON_VIEW_USE_SSSE3
COMPILE_OPTIONS -mssse3
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_ssse3
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
endif()
# *DO NOT* use json_test_set_test_options() below this line
#############################################################################