Compare commits

..
Author SHA1 Message Date
Niels Lohmann 81bc423ead Add benchmarks for the binary readers
The benchmark suite covered parsing JSON text, dumping, and serializing to
CBOR, but only one binary read: FromMsgpack. Nothing measured from_cbor,
from_ubjson, from_bjdata or from_bson, so a change to binary_reader.hpp had
no baseline to be compared against.

Add read benchmarks for every format, in the two shapes that matter: from a
contiguous buffer, which is what most callers pass, and from a FILE*, which
is what FromMsgpack already measures and which compiles to different code.
FromMsgpack itself is left untouched so its numbers stay comparable across
releases. The input is derived at setup time by serializing a parsed test
file, because the test data repository ships JSON only.

The test files are wide and shallow, but the readers' cost is per container,
so add three value shapes they do not cover -- deeply nested containers, many
sibling containers, and one flat array of numbers -- plus an indefinite-length
CBOR string, a form the writer never emits and which therefore has to be
assembled by hand. UBJSON and BJData are also captured in their size- and
type-annotated form, which the readers handle in a separate code path.

BSON requires an object at the top level, so it cannot reuse the array-rooted
test files; it is captured on the object-rooted ones, and the shapes are
wrapped in an object so every format measures the same value. The setup marks
the benchmark as skipped rather than letting the exception escape if that
requirement is ever violated.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-06 16:37:22 +02:00
63 changed files with 759 additions and 6292 deletions
+2 -2
View File
@@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
@@ -34,7 +34,7 @@ jobs:
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-22.04 runs-on: ubuntu-22.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
@@ -19,7 +19,7 @@ jobs:
pull-requests: write pull-requests: write
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
security-events: write security-events: write
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
runs-on: ubuntu-22.04 runs-on: ubuntu-22.04
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+6 -6
View File
@@ -35,7 +35,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
@@ -60,7 +60,7 @@ jobs:
target: [ci_test_amalgamation, ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind] target: [ci_test_amalgamation, ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind]
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
@@ -100,7 +100,7 @@ jobs:
container: ubuntu:focal container: ubuntu:focal
strategy: strategy:
matrix: matrix:
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_simdutf] target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls]
steps: steps:
- name: Install build-essential - name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev
@@ -118,7 +118,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
@@ -369,7 +369,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
@@ -392,7 +392,7 @@ jobs:
target: [ci_test_examples, ci_test_build_documentation] target: [ci_test_examples, ci_test_build_documentation]
steps: steps:
- name: Harden Runner - name: Harden Runner
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with: with:
egress-policy: audit egress-policy: audit
+3 -18
View File
@@ -1,4 +1,4 @@
.PHONY: pretty clean ChangeLog.md release update_hedley update_hedley_undef .PHONY: pretty clean ChangeLog.md release
########################################################################## ##########################################################################
# configuration # configuration
@@ -41,8 +41,6 @@ all:
@echo "fuzz_testing_ubjson - prepare fuzz testing of the UBJSON parser" @echo "fuzz_testing_ubjson - prepare fuzz testing of the UBJSON parser"
@echo "pretty - beautify code with Artistic Style" @echo "pretty - beautify code with Artistic Style"
@echo "run_benchmarks - build and run benchmarks" @echo "run_benchmarks - build and run benchmarks"
@echo "update_hedley - download Hedley and regenerate hedley.hpp / hedley_undef.hpp"
@echo "update_hedley_undef - rebuild hedley_undef.hpp from the JSON_HEDLEY_* #define names in hedley.hpp"
########################################################################## ##########################################################################
@@ -243,24 +241,11 @@ update_hedley:
rm -f include/nlohmann/thirdparty/hedley/hedley.hpp include/nlohmann/thirdparty/hedley/hedley_undef.hpp rm -f include/nlohmann/thirdparty/hedley/hedley.hpp include/nlohmann/thirdparty/hedley/hedley_undef.hpp
curl https://raw.githubusercontent.com/nemequ/hedley/master/hedley.h -o include/nlohmann/thirdparty/hedley/hedley.hpp curl https://raw.githubusercontent.com/nemequ/hedley/master/hedley.h -o include/nlohmann/thirdparty/hedley/hedley.hpp
$(SED) -i 's/HEDLEY_/JSON_HEDLEY_/g' include/nlohmann/thirdparty/hedley/hedley.hpp $(SED) -i 's/HEDLEY_/JSON_HEDLEY_/g' include/nlohmann/thirdparty/hedley/hedley.hpp
grep "[[:blank:]]*#[[:blank:]]*undef" include/nlohmann/thirdparty/hedley/hedley.hpp | grep -v "__" | sort | uniq | $(SED) 's/ //g' | $(SED) 's/undef/undef /g' > include/nlohmann/thirdparty/hedley/hedley_undef.hpp
$(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley.hpp $(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley.hpp
$(MAKE) update_hedley_undef $(SED) -i '1s/^/#pragma once\n\n/' include/nlohmann/thirdparty/hedley/hedley_undef.hpp
$(MAKE) amalgamate $(MAKE) amalgamate
# Rebuild hedley_undef.hpp from every JSON_HEDLEY_* name that hedley.hpp
# #defines. Hedley does not #undef all of its public macros internally (see
# #5408), so grepping those #undef lines misses names such as
# JSON_HEDLEY_PRAGMA. cmake/scripts/gen_hedley_undef_check.cmake is the
# single source of truth for this extraction (tests/CMakeLists.txt uses the
# same script, in MODE=checks, to generate the matching leak-check test), so
# the vendored header, the generated #undef list, and the regression test
# cannot drift apart.
update_hedley_undef:
cmake -DHEDLEY_HPP=include/nlohmann/thirdparty/hedley/hedley.hpp \
-DOUTPUT=include/nlohmann/thirdparty/hedley/hedley_undef.hpp \
-DMODE=undef \
-P cmake/scripts/gen_hedley_undef_check.cmake
########################################################################## ##########################################################################
# serve_header.py # serve_header.py
########################################################################## ##########################################################################
-18
View File
@@ -212,24 +212,6 @@ add_custom_target(ci_test_legacycomparison
COMMENT "Compile and test with legacy discarded value comparison enabled" COMMENT "Compile and test with legacy discarded value comparison enabled"
) )
###############################################################################
# Validate UTF-8 with simdutf.
###############################################################################
add_custom_target(ci_test_simdutf
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_TestSimdutf=ON
# simdutf needs C++17, so the library falls back to its scalar validator
# below that: build the suite at C++11 to cover the fallback with the macro
# defined, and at C++17 to run every test against simdutf itself
"-DJSON_TestStandards=11\;17"
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_simdutf
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_simdutf
COMMAND cd ${PROJECT_BINARY_DIR}/build_simdutf && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
COMMENT "Compile and test with simdutf UTF-8 validation enabled"
)
############################################################################### ###############################################################################
# Enable brace-init copy semantics. # Enable brace-init copy semantics.
############################################################################### ###############################################################################
-112
View File
@@ -1,112 +0,0 @@
# Shared extractor for the JSON_HEDLEY_* macro names defined in hedley.hpp.
#
# Every macro that hedley.hpp #defines must be #undef-ed again once json.hpp
# has been fully processed (see include/nlohmann/detail/macro_unscope.hpp
# and https://github.com/nlohmann/json/issues/5408). Deriving the macro list
# straight from hedley.hpp here -- instead of hand-maintaining it in two
# places -- means hedley_undef.hpp and the regression test that checks for
# leaked macros can never drift apart, even after a future `make
# update_hedley` pulls in new macros from upstream Hedley.
#
# MODE=undef (default): write hedley_undef.hpp (SPDX header, #pragma once,
# one #undef per macro name) -- used by `make update_hedley_undef`
# MODE=checks: write one #ifdef/FAIL_CHECK/#endif per macro name,
# meant to be #include-d inside a TEST_CASE -- used by
# tests/CMakeLists.txt to (re)generate the include for
# tests/src/unit-no-macro-leak.cpp
#
# Required variables:
# HEDLEY_HPP path to include/nlohmann/thirdparty/hedley/hedley.hpp
# OUTPUT path of the file to (over)write
# Optional:
# MODE "undef" (default) or "checks"
if(NOT DEFINED HEDLEY_HPP OR NOT DEFINED OUTPUT)
message(FATAL_ERROR "HEDLEY_HPP and OUTPUT must be set")
endif()
if(NOT EXISTS "${HEDLEY_HPP}")
message(FATAL_ERROR "Hedley header not found: ${HEDLEY_HPP}")
endif()
if(NOT DEFINED MODE)
set(MODE undef)
endif()
if(NOT MODE STREQUAL "undef" AND NOT MODE STREQUAL "checks")
message(FATAL_ERROR "MODE must be undef or checks, got: ${MODE}")
endif()
# Line-anchored, like `grep -oE "^[[:blank:]]*#[[:blank:]]*define[[:blank:]]+JSON_HEDLEY_[A-Za-z0-9_]+"`.
# Unanchored matching would also pick up JSON_HEDLEY_* mentions inside
# comments or string literals elsewhere in the file, which must not turn
# into #undef lines.
file(STRINGS "${HEDLEY_HPP}" hedley_lines)
set(macro_names)
foreach(line IN LISTS hedley_lines)
if("${line}" MATCHES "^[ \t]*#[ \t]*define[ \t]+(JSON_HEDLEY_[A-Za-z0-9_]+)")
list(APPEND macro_names "${CMAKE_MATCH_1}")
endif()
endforeach()
if(NOT macro_names)
message(FATAL_ERROR "No JSON_HEDLEY_* macros found in ${HEDLEY_HPP}")
endif()
list(REMOVE_DUPLICATES macro_names)
# Lexicographic, locale-independent (ASCII-only names) -- matches `LC_ALL=C sort`.
list(SORT macro_names COMPARE STRING)
list(LENGTH macro_names macro_count)
set(generated "")
if(MODE STREQUAL "undef")
# Same banner `make update_hedley_undef` would stamp by hand, so the
# recipe is self-contained and its output is byte-stable across reruns.
# The embedded SPDX tags below are part of the *generated* file's
# content, not a REUSE header for this .cmake script itself (which is
# already covered by the blanket "Files: *" rule in .reuse/dep5) -- keep
# them wrapped in REUSE-IgnoreStart/End so `reuse lint` does not try to
# parse "MIT\n")" as this file's own SPDX-License-Identifier value.
# REUSE-IgnoreStart
string(APPEND generated "// __ _____ _____ _____\n")
string(APPEND generated "// __| | __| | | | JSON for Modern C++\n")
string(APPEND generated "// | | |__ | | | | | | version 3.12.0\n")
string(APPEND generated "// |_____|_____|_____|_|___| https://github.com/nlohmann/json\n")
string(APPEND generated "//\n")
string(APPEND generated "// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>\n")
string(APPEND generated "// SPDX-License-Identifier: MIT\n")
# REUSE-IgnoreEnd
string(APPEND generated "\n")
string(APPEND generated "#pragma once\n")
string(APPEND generated "\n")
foreach(name IN LISTS macro_names)
string(APPEND generated "#undef ${name}\n")
endforeach()
else()
string(APPEND generated "// This file is generated by cmake/scripts/gen_hedley_undef_check.cmake\n")
string(APPEND generated "// from include/nlohmann/thirdparty/hedley/hedley.hpp. Do not edit it by\n")
string(APPEND generated "// hand -- it is regenerated on every build. ${macro_count} macros checked.\n\n")
foreach(name IN LISTS macro_names)
string(APPEND generated "#ifdef ${name}\n")
string(APPEND generated " FAIL_CHECK(\"${name} leaked after including nlohmann/json.hpp\");\n")
string(APPEND generated "#endif\n")
endforeach()
endif()
get_filename_component(output_dir "${OUTPUT}" DIRECTORY)
if(output_dir)
file(MAKE_DIRECTORY "${output_dir}")
endif()
# Avoid rewriting the file (and busting downstream incremental rebuilds)
# when the content has not actually changed.
set(write_output TRUE)
if(EXISTS "${OUTPUT}")
file(READ "${OUTPUT}" existing_content)
if(existing_content STREQUAL generated)
set(write_output FALSE)
endif()
endif()
if(write_output)
file(WRITE "${OUTPUT}" "${generated}")
endif()
-8
View File
@@ -34,10 +34,6 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
("add", "remove", "move") ("add", "remove", "move")
- Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target - Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target
location has a parent that is neither an object nor an array. location has a parent that is neither an object nor an array.
- Throws [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) if a "remove" operation's target
location has a parent that is neither an object nor an array.
- Throws [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) if a "move" operation's "from"
location is a proper prefix of its "path" location.
- Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was - Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was
unsuccessful. unsuccessful.
@@ -79,7 +75,3 @@ is thrown. In any case, the original value is not changed: the patch is applied
- Added in version 2.0.0. - Added in version 2.0.0.
- Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's - Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's
target location has a non-object/non-array parent in version 3.13.0. target location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) and stopped silently ignoring a "remove" operation whose target
location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper
prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0.
@@ -30,10 +30,6 @@ No guarantees, value may be corrupted by an unsuccessful patch operation.
("add", "remove", "move") ("add", "remove", "move")
- Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target - Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target
location has a parent that is neither an object nor an array. location has a parent that is neither an object nor an array.
- Throws [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) if a "remove" operation's target
location has a parent that is neither an object nor an array.
- Throws [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) if a "move" operation's "from"
location is a proper prefix of its "path" location.
- Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was - Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was
unsuccessful. unsuccessful.
@@ -76,7 +72,3 @@ function throws an exception.
- Added in version 3.11.0. - Added in version 3.11.0.
- Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's - Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's
target location has a non-object/non-array parent in version 3.13.0. target location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) and stopped silently ignoring a "remove" operation whose target
location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper
prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0.
-1
View File
@@ -24,7 +24,6 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_NO_IO**](json_no_io.md) - switch off functions relying on certain C++ I/O headers - [**JSON_NO_IO**](json_no_io.md) - switch off functions relying on certain C++ I/O headers
- [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers - [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
- [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace - [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [**JSON_USE_SIMDUTF**](json_use_simdutf.md) - use the simdutf library to accelerate UTF-8 validation
## Library version ## Library version
@@ -1,71 +0,0 @@
# JSON_USE_SIMDUTF
```cpp
#define JSON_USE_SIMDUTF
```
When defined, the parser validates the UTF-8 content of JSON strings that come from a **contiguous byte input**
(`std::string`, `std::vector<char>`/`<std::uint8_t>`, string literals, `const char*` ranges, …) using the
[simdutf](https://github.com/simdutf/simdutf) library instead of the built-in scalar validator. On text with many
non-ASCII characters (e.g. CJK or emoji) this can validate several times faster.
This is an **opt-in external dependency**. The library itself remains header-only and its behavior is unchanged: the
same input is accepted or rejected either way, and every parse error is reported at the same position with the same
message (simdutf is only used to fast-path *valid* runs; anything it flags falls back to the scalar path so the exact
diagnostic is preserved). Streaming inputs (files, `std::istream`, wide strings, user-defined adapters) always use the
scalar path.
When `JSON_USE_SIMDUTF` is defined you must make the `simdutf.h` header available on the include path and link the
simdutf library. When it is not defined, no simdutf header is included and there is no dependency.
!!! note "Requires C++17"
simdutf requires C++17 and its header rejects older standards with an `#!cpp #error`. The backend is therefore only
compiled in from C++17 on. In C++11 and C++14 the macro has no effect and the scalar validator is used, which
accepts and rejects exactly the same input -- only throughput differs. Setting the macro project-wide is therefore
safe even when some translation units are built with an older standard.
!!! warning "Define consistently"
The macro selects between two definitions of the same inline validation function. It must therefore be defined
identically for **every** translation unit that includes the library; mixing translation units that define it with
ones that do not is an ODR violation. Prefer setting it as a compile definition on the target rather than with
`#!cpp #define` in individual source files.
## Default definition
By default, `#!cpp JSON_USE_SIMDUTF` is not defined and the portable C++11 scalar validator is used.
```cpp
#undef JSON_USE_SIMDUTF
```
## Examples
??? example
The code below enables the simdutf backend for UTF-8 validation.
```cpp
#define JSON_USE_SIMDUTF 1
#include <nlohmann/json.hpp>
...
```
The project must also link against simdutf, e.g. with CMake:
```cmake
target_compile_definitions(your_target PRIVATE JSON_USE_SIMDUTF)
target_link_libraries(your_target PRIVATE simdutf::simdutf)
```
!!! hint "Testing this configuration"
The unit tests can be built against the simdutf backend with the CMake option `JSON_TestSimdutf` (`OFF` by
default), which fetches simdutf and defines `JSON_USE_SIMDUTF` for every test target. The `ci_test_simdutf` target
runs the whole test suite in that configuration.
## Version history
- Added in version 3.13.0.
@@ -4,8 +4,8 @@
Hello, world! Hello, world!
1 2 3 4 5 1 2 3 4 5
string: "Hello, world!"
number: {"floating-point":17.23,"integer":42} number: {"floating-point":17.23,"integer":42}
null: null null: null
string: "Hello, world!"
boolean: true boolean: true
array: [1,2,3,4,5] array: [1,2,3,4,5]
+1 -1
View File
@@ -4,8 +4,8 @@
Hello, world! Hello, world!
1 2 3 4 5 1 2 3 4 5
string: "Hello, world!"
number: {"floating-point":17.23,"integer":42} number: {"floating-point":17.23,"integer":42}
null: null null: null
string: "Hello, world!"
boolean: true boolean: true
array: [1,2,3,4,5] array: [1,2,3,4,5]
@@ -4,9 +4,9 @@
Hello, world! Hello, world!
1 2 3 4 5 1 2 3 4 5
string: "Hello, world!"
number: {"floating-point":17.23,"integer":42} number: {"floating-point":17.23,"integer":42}
null: null null: null
string: "Hello, world!"
boolean: true boolean: true
array: [1,2,3,4,5] array: [1,2,3,4,5]
[json.exception.type_error.302] type must be boolean, but is string [json.exception.type_error.302] type must be boolean, but is string
-8
View File
@@ -137,14 +137,6 @@ behavior is deprecated and switched off (`0`) by default.
See [full documentation of `JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md). See [full documentation of `JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md).
## `JSON_USE_SIMDUTF`
When defined, UTF-8 validation of JSON strings read from contiguous byte input is delegated to the
[simdutf](https://github.com/simdutf/simdutf) library instead of the built-in scalar validator. This is an opt-in
external dependency and is not defined by default.
See [full documentation of `JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md).
## `NLOHMANN_DEFINE_TYPE_*(...)`, `NLOHMANN_DEFINE_DERIVED_TYPE_*(...)` ## `NLOHMANN_DEFINE_TYPE_*(...)`, `NLOHMANN_DEFINE_DERIVED_TYPE_*(...)`
The library defines 12 macros to simplify the serialization/deserialization of types. See the page on The library defines 12 macros to simplify the serialization/deserialization of types. See the page on
-28
View File
@@ -933,34 +933,6 @@ BSON stores the length of documents, arrays, strings, and binary values in a sig
[`to_bson`](../api/basic_json/to_bson.md) produced documents with negative length prefixes that [`to_bson`](../api/basic_json/to_bson.md) produced documents with negative length prefixes that
[`from_bson`](../api/basic_json/from_bson.md) rejected. [`from_bson`](../api/basic_json/from_bson.md) rejected.
### json.exception.out_of_range.413
A JSON Patch `remove` operation cannot be applied because the target location's parent is neither an object nor an array. Per [RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902), a `remove` target must reference a member of an existing object or an element of an existing array; a primitive value (string, number, boolean, etc.) or `null` has no members or elements to remove.
!!! failure "Example message"
```
cannot remove value: the JSON Patch 'remove' target's parent is of type number, but must be an object or array
```
!!! note
This exception was added in version 3.13.0. Before that, this situation was silently ignored (the `remove` operation had no effect).
### json.exception.out_of_range.414
A JSON Patch `move` operation's `"from"` location is a proper prefix of its `"path"` location. Per [RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902) (section 4.4), a location cannot be moved into one of its own children.
!!! failure "Example message"
```
cannot move value: 'from' path '/0' is a proper prefix of 'path' '/0/0'
```
!!! note
This exception was added in version 3.13.0. Before that, this situation could succeed with a corrupted result: for an array target, removing the "from" element before the "add" step shifted subsequent indices, so "path" silently re-resolved to a different element than intended.
## Further exceptions ## Further exceptions
This exception is thrown in case of errors that cannot be classified with the This exception is thrown in case of errors that cannot be classified with the
-1
View File
@@ -296,7 +296,6 @@ nav:
- 'JSON_USE_GLOBAL_UDLS': api/macros/json_use_global_udls.md - 'JSON_USE_GLOBAL_UDLS': api/macros/json_use_global_udls.md
- 'JSON_USE_IMPLICIT_CONVERSIONS': api/macros/json_use_implicit_conversions.md - 'JSON_USE_IMPLICIT_CONVERSIONS': api/macros/json_use_implicit_conversions.md
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md - 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
- 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
- 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md - 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
- 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md - 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md
- 'NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_non_intrusive.md - 'NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_non_intrusive.md
@@ -398,17 +398,6 @@ inline void from_json(const BasicJsonType& j, CompatibleArrayType& bin)
} }
} }
template<typename ConstructibleObjectType>
auto from_json_object_reserve(ConstructibleObjectType& obj, typename ConstructibleObjectType::size_type size, priority_tag<1> /*unused*/)
-> decltype(obj.reserve(size), void())
{
obj.reserve(size);
}
template<typename ConstructibleObjectType>
inline void from_json_object_reserve(ConstructibleObjectType& /*obj*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
{}
template<typename BasicJsonType, typename ConstructibleObjectType, template<typename BasicJsonType, typename ConstructibleObjectType,
enable_if_t<is_constructible_object_type<BasicJsonType, ConstructibleObjectType>::value, int> = 0> enable_if_t<is_constructible_object_type<BasicJsonType, ConstructibleObjectType>::value, int> = 0>
inline void from_json(const BasicJsonType& j, ConstructibleObjectType& obj) inline void from_json(const BasicJsonType& j, ConstructibleObjectType& obj)
@@ -420,7 +409,6 @@ inline void from_json(const BasicJsonType& j, ConstructibleObjectType& obj)
ConstructibleObjectType ret; ConstructibleObjectType ret;
const auto* inner_object = j.template get_ptr<const typename BasicJsonType::object_t*>(); const auto* inner_object = j.template get_ptr<const typename BasicJsonType::object_t*>();
from_json_object_reserve(ret, inner_object->size(), priority_tag<1> {});
for (const auto& p : *inner_object) for (const auto& p : *inner_object)
{ {
ret.emplace(p.first, p.second.template get<typename ConstructibleObjectType::mapped_type>()); ret.emplace(p.first, p.second.template get<typename ConstructibleObjectType::mapped_type>());
@@ -1075,8 +1075,8 @@ char* to_chars(char* first, const char* last, FloatType value)
} }
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_PUSH #pragma GCC diagnostic push
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal") #pragma GCC diagnostic ignored "-Wfloat-equal"
#endif #endif
if (value == 0) // +-0 if (value == 0) // +-0
{ {
@@ -1087,7 +1087,7 @@ char* to_chars(char* first, const char* last, FloatType value)
return first; return first;
} }
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP #pragma GCC diagnostic pop
#endif #endif
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10); JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
+3 -3
View File
@@ -33,8 +33,8 @@
// code stumbling over this. See https://github.com/nlohmann/json/issues/4087 // code stumbling over this. See https://github.com/nlohmann/json/issues/4087
// for a discussion. // for a discussion.
#if defined(__clang__) #if defined(__clang__)
JSON_HEDLEY_DIAGNOSTIC_PUSH #pragma clang diagnostic push
JSON_HEDLEY_PRAGMA(clang diagnostic ignored "-Wweak-vtables") #pragma clang diagnostic ignored "-Wweak-vtables"
#endif #endif
NLOHMANN_JSON_NAMESPACE_BEGIN NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -287,5 +287,5 @@ class other_error : public exception
NLOHMANN_JSON_NAMESPACE_END NLOHMANN_JSON_NAMESPACE_END
#if defined(__clang__) #if defined(__clang__)
JSON_HEDLEY_DIAGNOSTIC_POP #pragma clang diagnostic pop
#endif #endif
+21 -131
View File
@@ -155,31 +155,11 @@ class input_stream_adapter
// General-purpose iterator-based adapter. It might not be as fast as // General-purpose iterator-based adapter. It might not be as fast as
// theoretically possible for some containers, but it is extremely versatile. // theoretically possible for some containers, but it is extremely versatile.
// SentinelType defaults to IteratorType for backward compatibility, but may be // SentinelType defaults to IteratorType for backward compatibility, but may
// a different type, e.g. a C++20 sentinel such as std::default_sentinel_t when // be a different type (e.g., a C++20 sentinel or counted_iterator).
// IteratorType is a std::counted_iterator.
template<typename IteratorType, typename SentinelType = IteratorType> template<typename IteratorType, typename SentinelType = IteratorType>
class iterator_input_adapter class iterator_input_adapter
{ {
// Whether the number of elements between two positions can be computed in
// O(1): either the iterator and the sentinel have the same type (plain
// std::distance) or, in C++20, the sentinel is a sized sentinel for the
// iterator (std::ranges::distance), e.g. std::default_sentinel_t paired
// with std::counted_iterator.
//
// JSON_HAS_RANGES gates the C++20 branch: on standard libraries with an
// incomplete <ranges> (libstdc++ < 11, see #4440) evaluating
// std::contiguous_iterator on a std::counted_iterator is a hard error
// instead of yielding false, and these traits are instantiated for every
// adapter. Such toolchains fall back to the pointer-only test and simply
// use the byte-at-a-time scanner.
static constexpr bool sentinel_is_sized =
#if JSON_HAS_RANGES && defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20)
std::is_same<IteratorType, SentinelType>::value || std::sized_sentinel_for<SentinelType, IteratorType>;
#else
std::is_same<IteratorType, SentinelType>::value;
#endif
public: public:
using char_type = typename std::iterator_traits<IteratorType>::value_type; using char_type = typename std::iterator_traits<IteratorType>::value_type;
@@ -191,7 +171,7 @@ class iterator_input_adapter
// in wide_string_input_adapter, which does not expose this). // in wide_string_input_adapter, which does not expose this).
static constexpr bool supports_seek = static constexpr bool supports_seek =
std::is_same<typename std::iterator_traits<IteratorType>::iterator_category, std::random_access_iterator_tag>::value std::is_same<typename std::iterator_traits<IteratorType>::iterator_category, std::random_access_iterator_tag>::value
&& sentinel_is_sized && std::is_same<IteratorType, SentinelType>::value
&& sizeof(char_type) == 1; && sizeof(char_type) == 1;
iterator_input_adapter(IteratorType first, SentinelType last) iterator_input_adapter(IteratorType first, SentinelType last)
@@ -239,60 +219,30 @@ class iterator_input_adapter
private: private:
// whether IteratorType refers to a contiguous range and therefore supports // whether IteratorType refers to a contiguous range and therefore supports
// a std::memcpy fast path (pointers always do; in C++20 we can also detect // a std::memcpy fast path (pointers always do; in C++20 we can also detect
// library iterators such as those of std::vector and std::string). The // library iterators such as those of std::vector and std::string).
// available element count must also be computable in O(1), hence // Computing the available element count needs either same-type iterators
// sentinel_is_sized. // (plain std::distance) or, in C++20, a sized sentinel (std::ranges::distance),
static constexpr bool iterator_is_contiguous = sentinel_is_sized && // e.g. std::counted_iterator paired with std::default_sentinel_t.
#if JSON_HAS_RANGES && defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20) static constexpr bool iterator_is_contiguous =
(std::contiguous_iterator<IteratorType> || std::is_pointer<IteratorType>::value); #if defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20)
(std::is_same<IteratorType, SentinelType>::value || std::sized_sentinel_for<SentinelType, IteratorType>)
&& (std::contiguous_iterator<IteratorType> || std::is_pointer<IteratorType>::value);
#else #else
std::is_pointer<IteratorType>::value; std::is_same<IteratorType, SentinelType>::value && std::is_pointer<IteratorType>::value;
#endif #endif
// number of unread elements in [current, end)
std::size_t remaining_count() const
{
#if JSON_HAS_RANGES && defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20)
// std::ranges::distance also supports sized sentinels of a different
// type (e.g. std::counted_iterator + std::default_sentinel_t)
return static_cast<std::size_t>(std::ranges::distance(current, end));
#else
return static_cast<std::size_t>(std::distance(current, end));
#endif
}
public:
// Whether the remaining input is a single contiguous block of 1-byte
// elements that the lexer can inspect directly (used for the SWAR string
// fast path).
static constexpr bool supports_bulk_scan =
iterator_is_contiguous && sizeof(char_type) == 1;
// Pointer to the next unread element; only valid when bulk_remaining() > 0.
const char_type* bulk_data() const
{
return &*current;
}
// Number of unread elements available as one contiguous block.
std::size_t bulk_remaining() const
{
return remaining_count();
}
// Consume @a n elements previously inspected via bulk_data().
void bulk_skip(std::size_t n)
{
std::advance(current, static_cast<typename std::iterator_traits<IteratorType>::difference_type>(n));
}
private:
// contiguous fast path: bulk copy the remaining range with std::memcpy // contiguous fast path: bulk copy the remaining range with std::memcpy
template<class T> template<class T>
std::size_t get_elements_impl(T* dest, std::size_t count, std::true_type /*contiguous*/) std::size_t get_elements_impl(T* dest, std::size_t count, std::true_type /*contiguous*/)
{ {
const std::size_t wanted = count * sizeof(T); const std::size_t wanted = count * sizeof(T);
const std::size_t available = remaining_count() * sizeof(char_type); #if defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20)
// std::ranges::distance also supports sized sentinels of a different
// type (e.g. std::counted_iterator + std::default_sentinel_t)
const std::size_t available = static_cast<std::size_t>(std::ranges::distance(current, end)) * sizeof(char_type);
#else
const std::size_t available = static_cast<std::size_t>(std::distance(current, end)) * sizeof(char_type);
#endif
const std::size_t copied = (std::min)(wanted, available); const std::size_t copied = (std::min)(wanted, available);
if (JSON_HEDLEY_LIKELY(copied != 0)) if (JSON_HEDLEY_LIKELY(copied != 0))
{ {
@@ -620,46 +570,6 @@ typename iterator_input_adapter_factory<IteratorType, SentinelType>::adapter_typ
return factory_type::create(first, last); return factory_type::create(first, last);
} }
// The element type a container's data() points at, cv-qualifiers removed.
// Ill-formed - and therefore SFINAE-friendly - for types without data().
template<typename ContainerType>
using container_data_t = typename std::remove_cv<typename std::remove_pointer <
decltype(std::declval<const ContainerType&>().data()) >::type >::type;
// The container's own element type, cv-qualifiers removed. It is looked up on
// the bare type so it is also found when ContainerType is deduced as a
// reference by the forwarding-reference overload below.
template<typename ContainerType>
using container_value_t = typename std::remove_cv <
typename std::remove_cv<typename std::remove_reference<ContainerType>::type>::type::value_type >::type;
// Detect a container that stores its elements contiguously as single bytes
// (std::string, std::vector<char/unsigned char>, std::array<char, N>,
// std::string_view, ...). Such inputs are wrapped in a pointer-based adapter so
// they benefit from the contiguous fast paths (bulk string scanning, memcpy for
// binary formats) in every C++ standard - not only in C++20, where the standard
// library iterators model std::contiguous_iterator and are detected directly.
//
// data() and size() on their own would be duck typing: they say nothing about
// size() counting the units data() points at, and reading [data(), data() +
// size()) as bytes would be wrong for a type where it does not. Requiring the
// container's own value_type to be that same single-byte element ties the two
// together; every contiguous standard container satisfies it. Anything else
// keeps the iterator-based adapter, which is always correct - only slower.
template<typename ContainerType, typename = void>
struct is_contiguous_byte_container : std::false_type {};
template<typename ContainerType>
struct is_contiguous_byte_container < ContainerType, void_t <
container_data_t<ContainerType>,
container_value_t<ContainerType>,
decltype(std::declval<const ContainerType&>().size()) >>
: std::integral_constant < bool,
std::is_pointer<decltype(std::declval<const ContainerType&>().data())>::value&&
std::is_integral<container_data_t<ContainerType>>::value&&
sizeof(container_data_t<ContainerType>) == 1 &&
std::is_same<container_data_t<ContainerType>, container_value_t<ContainerType>>::value > {};
// Convenience shorthand from container to iterator // Convenience shorthand from container to iterator
// Enables ADL on begin(container) and end(container) // Enables ADL on begin(container) and end(container)
// Encloses the using declarations in namespace for not to leak them to outside scope // Encloses the using declarations in namespace for not to leak them to outside scope
@@ -687,32 +597,12 @@ struct container_input_adapter_factory< ContainerType,
} // namespace container_input_adapter_factory_impl } // namespace container_input_adapter_factory_impl
// General container path (iterator-based). Contiguous single-byte containers template<typename ContainerType>
// are excluded here and routed through the pointer-based overload below. typename container_input_adapter_factory_impl::container_input_adapter_factory<ContainerType>::adapter_type input_adapter(ContainerType&& container)
template < typename ContainerType,
enable_if_t < !is_contiguous_byte_container<ContainerType>::value, int > = 0 >
typename container_input_adapter_factory_impl::container_input_adapter_factory<ContainerType>::adapter_type input_adapter(ContainerType && container)
{ {
return container_input_adapter_factory_impl::container_input_adapter_factory<ContainerType>::create(std::forward<ContainerType>(container)); return container_input_adapter_factory_impl::container_input_adapter_factory<ContainerType>::create(std::forward<ContainerType>(container));
} }
// Contiguous single-byte containers (std::string, std::vector<char>, ...) are
// wrapped in a pointer-based adapter so the contiguous fast paths apply in every
// standard. The pointer keeps the container's own element type (const char* for
// std::string, const std::uint8_t* for std::vector<std::uint8_t>, ...), so the
// resulting char_type - and therefore the parsing behavior - is byte-for-byte
// identical to the iterator-based path; only the raw pointer additionally
// enables the bulk fast paths. The container outlives the adapter for the whole
// parse (temporaries live until the end of the full expression), exactly as the
// iterators it replaces did.
template < typename ContainerType,
enable_if_t < is_contiguous_byte_container<ContainerType>::value, int > = 0 >
auto input_adapter(const ContainerType& container)
-> decltype(input_adapter(container.data(), container.data() + container.size()))
{
return input_adapter(container.data(), container.data() + container.size());
}
// specialization for std::string // specialization for std::string
using string_input_adapter_type = decltype(input_adapter(std::declval<std::string>())); using string_input_adapter_type = decltype(input_adapter(std::declval<std::string>()));
+32 -500
View File
@@ -19,9 +19,7 @@
#include <vector> // vector #include <vector> // vector
#include <nlohmann/detail/input/input_adapters.hpp> #include <nlohmann/detail/input/input_adapters.hpp>
#include <nlohmann/detail/input/number_parse.hpp>
#include <nlohmann/detail/input/position_t.hpp> #include <nlohmann/detail/input/position_t.hpp>
#include <nlohmann/detail/input/string_scan.hpp>
#include <nlohmann/detail/macro_scope.hpp> #include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/type_traits.hpp> #include <nlohmann/detail/meta/type_traits.hpp>
@@ -127,25 +125,6 @@ constexpr bool input_adapter_supports_seek(std::false_type /*detected*/)
return false; return false;
} }
// Detect whether an input adapter exposes a contiguous byte block that the
// lexer can scan directly (see iterator_input_adapter::supports_bulk_scan).
// Adapters without the flag - file, stream, wide-string, user-defined - fall
// back to the character-at-a-time string scanner.
template<typename InputAdapterType>
using detect_supports_bulk_scan = decltype(InputAdapterType::supports_bulk_scan);
template<typename InputAdapterType>
constexpr bool input_adapter_supports_bulk_scan(std::true_type /*detected*/)
{
return InputAdapterType::supports_bulk_scan;
}
template<typename InputAdapterType>
constexpr bool input_adapter_supports_bulk_scan(std::false_type /*detected*/)
{
return false;
}
/*! /*!
@brief lexical analysis @brief lexical analysis
@@ -167,22 +146,13 @@ class lexer : public lexer_base<BasicJsonType>
static constexpr bool lazy_token_string = static constexpr bool lazy_token_string =
input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {}); input_adapter_supports_seek<InputAdapterType>(is_detected<detect_supports_seek, InputAdapterType> {});
/// whether string scanning may bulk-consume runs of ordinary characters
/// directly from a contiguous input buffer (SWAR fast path). This requires
/// the token to be reconstructible lazily (lazy_token_string), so bypassing
/// the per-character capture in get() cannot lose error diagnostics.
static constexpr bool bulk_scan =
lazy_token_string
&& input_adapter_supports_bulk_scan<InputAdapterType>(is_detected<detect_supports_bulk_scan, InputAdapterType> {});
public: public:
using token_type = typename lexer_base<BasicJsonType>::token_type; using token_type = typename lexer_base<BasicJsonType>::token_type;
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false) noexcept
: ia(std::move(adapter)) : ia(std::move(adapter))
, ignore_comments(ignore_comments_) , ignore_comments(ignore_comments_)
, decimal_point_char(static_cast<char_int_type>(get_decimal_point())) , decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
, discard_number_values(discard_number_values_)
{} {}
// deleted because of pointer members // deleted because of pointer members
@@ -295,40 +265,6 @@ class lexer : public lexer_base<BasicJsonType>
return true; return true;
} }
/// contiguous input: bulk-append the run of ordinary characters and complete
/// well-formed UTF-8 sequences starting at the current read position, leaving
/// the first byte that needs individual handling (the closing quote, an
/// escape, a control character, or an ill-formed UTF-8 byte) for get()
void scan_string_bulk(std::true_type /*bulk*/)
{
// a pending unget must be consumed through the normal path first
if (next_unget)
{
return;
}
const std::size_t remaining = ia.bulk_remaining();
if (remaining == 0)
{
return;
}
const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
const std::size_t pos = string_bulk_run(data, remaining);
if (pos == 0)
{
return;
}
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), pos);
ia.bulk_skip(pos);
// the run contains no newline (all bytes < 0x20 are treated as special),
// so only the flat character counters advance
position.chars_read_total += pos;
position.chars_read_current_line += pos;
}
/// streaming input: no bulk fast path
void scan_string_bulk(std::false_type /*bulk*/) const noexcept {}
/*! /*!
@brief scan a string literal @brief scan a string literal
@@ -354,10 +290,6 @@ class lexer : public lexer_base<BasicJsonType>
while (true) while (true)
{ {
// bulk-consume ordinary characters from contiguous input, then
// handle the next special byte through the switch below
scan_string_bulk(std::integral_constant<bool, bulk_scan> {});
// get the next character // get the next character
switch (get()) switch (get())
{ {
@@ -1076,12 +1008,6 @@ class lexer : public lexer_base<BasicJsonType>
// changed if minus sign, decimal point, or exponent is read // changed if minus sign, decimal point, or exponent is read
token_type number_type = token_type::value_unsigned; token_type number_type = token_type::value_unsigned;
// offset just past the last mantissa byte in token_buffer (i.e. the
// index of 'e'/'E', or the whole token when there is no exponent).
// convert_number() uses it to count significant digits; npos means
// "not seen an exponent yet" and is resolved at scan_number_done
std::size_t mantissa_end = std::string::npos;
// state (init): we just found out we need to scan a number // state (init): we just found out we need to scan a number
switch (current) switch (current)
{ {
@@ -1267,9 +1193,6 @@ scan_number_decimal2:
scan_number_exponent: scan_number_exponent:
// we just parsed an exponent // we just parsed an exponent
number_type = token_type::value_float; number_type = token_type::value_float;
// this label is reached only right after the 'e'/'E' was appended (from
// the zero, any1, and decimal2 states), so the mantissa ends before it
mantissa_end = token_buffer.size() - 1;
switch (get()) switch (get())
{ {
case '+': case '+':
@@ -1356,199 +1279,45 @@ scan_number_done:
// we are done scanning a number) // we are done scanning a number)
unget(); unget();
// no exponent was scanned: the mantissa spans the whole token char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
if (mantissa_end == std::string::npos) errno = 0;
{
mantissa_end = token_buffer.size();
}
return convert_number(number_type, mantissa_end); // try to parse integers first and fall back to floats
}
/*!
@brief convert an already-validated integer token to its value
The digit sequence in [first, last) has been validated by the caller, so a
dedicated parser can avoid the locale/errno overhead of std::strtoull.
@return the token type on success; token_type::uninitialized if @a
number_type is not an integer type or the value does not fit, in
which case the caller falls back to the floating-point conversion
(matching the previous std::strtoull/std::strtoll behavior)
*/
token_type convert_integer(token_type number_type, const char* first, const char* last)
{
if (number_type == token_type::value_unsigned) if (number_type == token_type::value_unsigned)
{ {
if (parse_integer_unsigned(first, last, value_unsigned)) const auto x = std::strtoull(token_buffer.data(), &endptr, 10);
// we checked the number format before
JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
if (errno != ERANGE)
{ {
return token_type::value_unsigned; value_unsigned = static_cast<number_unsigned_t>(x);
if (value_unsigned == x)
{
return token_type::value_unsigned;
}
} }
} }
else if (number_type == token_type::value_integer) else if (number_type == token_type::value_integer)
{ {
if (parse_integer_signed(first, last, value_integer)) const auto x = std::strtoll(token_buffer.data(), &endptr, 10);
// we checked the number format before
JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
if (errno != ERANGE)
{ {
return token_type::value_integer; value_integer = static_cast<number_integer_t>(x);
} if (value_integer == x)
} {
return token_type::value_integer;
return token_type::uninitialized; }
}
/*!
@brief check whether Clinger's fast path can still succeed for this token
parse_float_fast() needs a significand below 2^53. A mantissa with 17 or
more significant digits is at least 10^16 and therefore always exceeds it,
so calling the fast path would walk the token one extra time only to
decline before strtod has to run anyway.
Significant digits are the mantissa's digits from the first nonzero one on;
the sign, the decimal point, leading zeros, and the exponent do not count.
The answer is derived from indices - the digits are not scanned again - so
this stays off the hot path of the number scanners.
@param[in] mantissa_end offset just past the last mantissa byte in
token_buffer
@return false if parse_float_fast() is guaranteed to decline
*/
bool mantissa_fits_clinger(std::size_t mantissa_end) const
{
// 10^16 already exceeds 2^53, so 17 digits can never fit
constexpr std::size_t limit = 17;
const std::size_t neg = (!token_buffer.empty() && token_buffer[0] == '-') ? 1u : 0u;
const std::size_t has_dot = (decimal_point_position != std::string::npos) ? 1u : 0u;
// the JSON grammar restricts the integer part to "0" or [1-9][0-9]*, so
// a leading zero can only be a lone "0", which is not significant
const std::size_t lead_zero = (token_buffer[neg] == '0') ? 1u : 0u;
JSON_ASSERT(mantissa_end >= neg + has_dot + lead_zero);
std::size_t digits = mantissa_end - neg - has_dot - lead_zero;
if (JSON_HEDLEY_LIKELY(digits < limit))
{
return true;
}
// Only a number below 1 can carry further insignificant zeros, and only
// while the count stays at the limit does removing them change the
// answer - so this loop is skipped for all but a few tokens. Note
// token_buffer holds the locale's decimal point, so the fraction is
// located through decimal_point_position rather than by searching '.'.
if (lead_zero != 0)
{
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
for (std::size_t i = decimal_point_position + 1;
digits >= limit && i < mantissa_end && token_buffer[i] == '0'; ++i)
{
--digits;
}
}
return digits < limit;
}
/*!
@brief convert the number text in token_buffer to its value and token type
The digit sequence in token_buffer has already been validated (by the
scan_number() state machine or by the contiguous fast path) and holds the
locale decimal point in place of '.'. Integers are parsed first and fall
back to floating point on overflow. This is shared so both scanners produce
identical results.
@param[in] mantissa_end offset just past the last mantissa byte in
token_buffer (the index of 'e'/'E', or
token_buffer.size() when there is no exponent);
used to skip Clinger's fast path when it cannot
possibly succeed - see mantissa_fits_clinger()
*/
token_type convert_number(token_type number_type, std::size_t mantissa_end)
{
// If the caller does not need the converted value (only whether the
// input is syntactically valid; see json_sax_acceptor/accept()), an
// unsigned/integer token can be reported without calling
// strtoull()/strtoll() at all, *provided* we can already tell from
// the digit count alone that the conversion cannot overflow 64 bits.
// Such tokens are always finite and are accepted unconditionally by
// the parser regardless of their actual value (parser::sax_parse_internal()
// never checks finiteness for value_unsigned/value_integer), so the
// classification below is all that is needed.
//
// A decimal number with up to 18 digits is always representable in
// both std::uint64_t and std::int64_t (18 nines is ~1e18, well below
// both UINT64_MAX ~1.8e19 and INT64_MAX ~9.2e18), so strtoull()/strtoll()
// could not have set errno to ERANGE for it. Numbers with more digits
// (rare in practice) fall through to the exact code below, unchanged,
// so their handling -- including reclassification to value_float when
// the value overflows 64 bits, and rejection when it is not even
// finite as a double -- is bit-for-bit identical to before this
// optimization.
//
// Note this reasons about std::uint64_t/std::int64_t, not about
// number_unsigned_t/number_integer_t (BasicJsonType's own, possibly
// narrower, template parameters -- e.g. std::uint32_t). That is fine
// *only* because discard_number_values is exclusively set by
// accept() (see json.hpp), and accept() always parses through the
// library's own json_sax_acceptor -- never a user-supplied SAX
// consumer -- whose number_unsigned()/number_integer()/number_float()
// callbacks unconditionally discard their argument and return true.
// So for every caller that can reach this branch, neither the token
// classification below nor the eventual (possibly narrowed, and on
// this fast path left stale/unset) value_unsigned/value_integer is
// ever consulted -- an unsigned/integer token is accepted outright,
// and even a >18-digit token that this fast path deliberately falls
// through for is, once reclassified to value_float, still finite
// (and thus accepted) for any digit count that fits in number_unsigned_t
// or number_integer_t regardless of that type's width. If this
// function is ever taught to run with discard_number_values true for
// a caller that *does* read the converted value, this reasoning (and
// the fast path below) would need to be revisited.
if (discard_number_values)
{
constexpr std::size_t safe_digit_count = 18;
if (number_type == token_type::value_unsigned && token_buffer.size() <= safe_digit_count)
{
return token_type::value_unsigned;
}
if (number_type == token_type::value_integer && token_buffer.size() - 1 <= safe_digit_count)
{
return token_type::value_integer;
}
}
const char* const num_begin = token_buffer.data();
const char* const num_end = num_begin + token_buffer.size();
if (number_type != token_type::value_float)
{
const token_type integer_result = convert_integer(number_type, num_begin, num_end);
if (integer_result != token_type::uninitialized)
{
return integer_result;
} }
} }
// this code is reached if we parse a floating-point number or if an // this code is reached if we parse a floating-point number or if an
// integer conversion above overflowed. Prefer std::from_chars // integer conversion above failed
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
// otherwise the exact Clinger fast path (double only); otherwise the
// locale-aware strtof/strtod.
if (parse_float_from_chars(num_begin, num_end, value_float))
{
return token_type::value_float;
}
// Skipping a fast path that cannot succeed is lossless and saves a full
// extra pass over the token's bytes, which otherwise shows up on
// high-precision inputs such as canada.json
if (mantissa_fits_clinger(mantissa_end)
&& parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
{
return token_type::value_float;
}
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
strtof(value_float, token_buffer.data(), &endptr); strtof(value_float, token_buffer.data(), &endptr);
// we checked the number format before // we checked the number format before
@@ -1557,158 +1326,6 @@ scan_number_done:
return token_type::value_float; return token_type::value_float;
} }
/*!
@brief contiguous fast path for scanning a number
Parses the whole number token straight from the input buffer, avoiding the
per-character get()/add() of scan_number(). On success it fills token_buffer
(with the locale decimal point substituted, as scan_number() does) and
returns the token type. On anything it does not fully recognize as a
well-formed number it makes no state change and returns
token_type::uninitialized, so the caller falls back to scan_number(), which
then produces the exact diagnostic. @a current is the first digit or the
leading minus (already read); the remaining bytes are taken from the adapter.
*/
token_type scan_number_bulk_contiguous()
{
// a pending unget offsets the buffer position from current; fall back
if (next_unget)
{
return token_type::uninitialized;
}
const std::size_t rem = ia.bulk_remaining();
if (rem == 0)
{
// the first digit is the last input byte; let scan_number() finish
return token_type::uninitialized;
}
// the byte before the next unread one is current (contiguous input)
const char* const data = reinterpret_cast<const char*>(ia.bulk_data()) - 1;
const std::size_t avail = rem + 1;
// validate + classify the number extent (mirrors scan_number()'s grammar)
std::size_t i = 0;
std::size_t dot_index = std::string::npos;
token_type number_type = token_type::value_unsigned;
if (data[0] == '-')
{
number_type = token_type::value_integer;
i = 1;
if (i >= avail)
{
return token_type::uninitialized;
}
}
if (data[i] == '0')
{
++i;
}
else if (data[i] >= '1' && data[i] <= '9')
{
++i;
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
++i;
}
}
else
{
return token_type::uninitialized;
}
if (i < avail && data[i] == '.')
{
number_type = token_type::value_float;
dot_index = i;
++i;
if (i >= avail || !(data[i] >= '0' && data[i] <= '9'))
{
return token_type::uninitialized;
}
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
++i;
}
}
// the mantissa ends here, whether or not an exponent part follows
const std::size_t mantissa_end = i;
if (i < avail && (data[i] == 'e' || data[i] == 'E'))
{
number_type = token_type::value_float;
++i;
if (i < avail && (data[i] == '+' || data[i] == '-'))
{
++i;
}
if (i >= avail || !(data[i] >= '0' && data[i] <= '9'))
{
return token_type::uninitialized;
}
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
++i;
}
}
const std::size_t len = i;
// reset() records where this token starts (for diagnostics), so it has
// to run before the input position advances below
reset();
// An integer token needs no token_buffer: the SAX callbacks for
// number_integer/number_unsigned take only the value, and the overflow
// diagnostic rebuilds the text from the input. Convert straight from the
// input buffer and leave token_buffer empty. (JSON_DIAGNOSTIC_POSITIONS
// derives a number's start position from get_string().size(), so there
// the token still has to be materialized.)
#if !JSON_DIAGNOSTIC_POSITIONS
if (number_type != token_type::value_float)
{
const token_type integer_result = convert_integer(number_type, data, data + len);
if (JSON_HEDLEY_LIKELY(integer_result != token_type::uninitialized))
{
ia.bulk_skip(len - 1);
position.chars_read_total += (len - 1);
position.chars_read_current_line += (len - 1);
return integer_result;
}
// The value does not fit an integer, so this token converts as a
// float. Recording that here keeps convert_number() below from
// repeating the integer attempt that just failed.
number_type = token_type::value_float;
}
#endif
// materialize the token exactly as scan_number() would, substituting the
// locale decimal point so convert_number()'s strtof fallback stays valid.
// reset() already cleared token_buffer, so append() fills it (assign() is
// avoided because custom string_t types need not provide it)
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
if (dot_index != std::string::npos)
{
token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
decimal_point_position = dot_index;
}
ia.bulk_skip(len - 1);
position.chars_read_total += (len - 1);
position.chars_read_current_line += (len - 1);
return convert_number(number_type, mantissa_end);
}
/// contiguous input: try the number fast path, else the byte-path scanner
token_type scan_number_dispatch(std::true_type /*bulk*/)
{
const token_type t = scan_number_bulk_contiguous();
return (t != token_type::uninitialized) ? t : scan_number();
}
/// streaming input: always use the byte-path scanner
token_type scan_number_dispatch(std::false_type /*bulk*/)
{
return scan_number();
}
/*! /*!
@param[in] literal_text the literal text to expect @param[in] literal_text the literal text to expect
@param[in] length the length of the passed literal text @param[in] length the length of the passed literal text
@@ -1776,7 +1393,8 @@ scan_number_done:
*/ */
char_int_type get() char_int_type get()
{ {
advance_position(); ++position.chars_read_total;
++position.chars_read_current_line;
if (next_unget) if (next_unget)
{ {
@@ -1788,23 +1406,6 @@ scan_number_done:
current = ia.get_character(); current = ia.get_character();
} }
return track_after_read();
}
/// shared head of get() / get_ignoring_pending_unget(): bump the
/// per-character position counters (line-count-on-'\n' bookkeeping is
/// handled afterwards, in track_after_read(), once `current` is known)
void advance_position() noexcept
{
++position.chars_read_total;
++position.chars_read_current_line;
}
/// shared tail of get() / get_ignoring_pending_unget(): capture the
/// character for error messages (if needed) and update line/column
/// bookkeeping for the character now in `current`
char_int_type track_after_read()
{
// seekable adapters reconstruct the token lazily on error (see // seekable adapters reconstruct the token lazily on error (see
// get_token_string), so the eager per-character copy is skipped // get_token_string), so the eager per-character copy is skipped
capture_char(std::integral_constant<bool, lazy_token_string> {}); capture_char(std::integral_constant<bool, lazy_token_string> {});
@@ -1812,38 +1413,12 @@ scan_number_done:
if (current == '\n') if (current == '\n')
{ {
++position.lines_read; ++position.lines_read;
// remember the column the newline was read at: chars_read_current_line
// is about to be cleared, and a matching unget() cannot reconstruct it
chars_read_before_newline = position.chars_read_current_line;
position.chars_read_current_line = 0; position.chars_read_current_line = 0;
} }
return current; return current;
} }
/*!
@brief like get(), but for call sites that can prove no unget() is pending
get() has to check the `next_unget` flag on every call, because a
previous token may have ended with unget() (e.g. scan_number() always
ungets the character that terminated the number, so the next call to
scan() can see it again). skip_whitespace() reads that first,
possibly-ungotten character via a plain get(), but every further
character it reads is guaranteed to be a fresh read: nothing between
those calls invokes unget(). This variant skips the (otherwise always
false) next_unget branch for those calls; it is not a general
replacement for get().
*/
char_int_type get_ignoring_pending_unget()
{
JSON_ASSERT(!next_unget);
advance_position();
current = ia.get_character();
return track_after_read();
}
/// seekable adapter: nothing to capture, the token is rebuilt on error /// seekable adapter: nothing to capture, the token is rebuilt on error
void capture_char(std::true_type /*lazy*/) const noexcept {} void capture_char(std::true_type /*lazy*/) const noexcept {}
@@ -1871,20 +1446,12 @@ scan_number_done:
--position.chars_read_total; --position.chars_read_total;
// in case we "unget" a newline, we have to also decrement the lines_read // in case we "unget" a newline, we have to also decrement the lines_read
// and restore the column that get() cleared when it saw the newline;
// chars_read_current_line == 0 can only mean the last get() read one
if (position.chars_read_current_line == 0) if (position.chars_read_current_line == 0)
{ {
if (position.lines_read > 0) if (position.lines_read > 0)
{ {
--position.lines_read; --position.lines_read;
} }
// chars_read_before_newline counts the newline itself, which is the
// character being ungotten, hence the -1
position.chars_read_current_line = (chars_read_before_newline > 0)
? chars_read_before_newline - 1
: 0;
} }
else else
{ {
@@ -2045,37 +1612,13 @@ scan_number_done:
return true; return true;
} }
/// whether `current` is one of the four JSON whitespace characters
bool current_is_whitespace() const noexcept
{
return current == ' ' || current == '\t' || current == '\n' || current == '\r';
}
void skip_whitespace() void skip_whitespace()
{ {
// the first character may be a pending unget() left over from the
// previous token (see get_ignoring_pending_unget()); every
// subsequent character read by this loop is guaranteed fresh, since
// nothing below calls unget()
get();
if (!current_is_whitespace())
{
return;
}
// this is written as an if-guarded do-while (rather than a plain
// while loop) because that shape is what lets both GCC and Clang
// keep the input adapter's read pointer in a register across
// iterations; the equivalent while-loop measurably defeated that
// optimization in testing, turning long whitespace runs (e.g. the
// indentation of pretty-printed JSON) from a register-only loop
// into one that reloads the pointer from memory every character
do do
{ {
get_ignoring_pending_unget(); get();
} }
while (current_is_whitespace()); while (current == ' ' || current == '\t' || current == '\n' || current == '\r');
} }
token_type scan() token_type scan()
@@ -2151,7 +1694,7 @@ scan_number_done:
case '7': case '7':
case '8': case '8':
case '9': case '9':
return scan_number_dispatch(std::integral_constant<bool, bulk_scan> {}); return scan_number();
// end of input (the null byte is needed when parsing from // end of input (the null byte is needed when parsing from
// string literals) // string literals)
@@ -2182,10 +1725,6 @@ scan_number_done:
/// the start position of the current token /// the start position of the current token
position_t position {}; position_t position {};
/// the value chars_read_current_line had when the last newline was read, so
/// that unget() can restore the column instead of leaving it at 0
std::size_t chars_read_before_newline = 0;
/// raw input token string for error messages; only populated for streaming /// raw input token string for error messages; only populated for streaming
/// adapters (seekable adapters reconstruct it lazily via token_string_start) /// adapters (seekable adapters reconstruct it lazily via token_string_start)
std::vector<char_type> token_string {}; std::vector<char_type> token_string {};
@@ -2215,13 +1754,6 @@ scan_number_done:
const char_int_type decimal_point_char = '.'; const char_int_type decimal_point_char = '.';
/// the position of the decimal point in the input /// the position of the decimal point in the input
std::size_t decimal_point_position = std::string::npos; std::size_t decimal_point_position = std::string::npos;
/// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
/// token classification and never looks at the converted numeric value;
/// when set, scan_number() may skip strtoull()/strtoll() for
/// value_unsigned/value_integer tokens whose digit count guarantees they
/// fit into 64 bits (see scan_number())
const bool discard_number_values = false;
}; };
} // namespace detail } // namespace detail
@@ -1,302 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cfloat> // FLT_EVAL_METHOD
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint64_t
#include <limits> // numeric_limits
#include <nlohmann/detail/macro_scope.hpp>
// std::from_chars lives in <charconv>, but being in C++17 mode does not
// guarantee the header exists: GCC 7 sets __cplusplus to C++17 yet ships no
// <charconv> (added in GCC 8; floating-point support in GCC 11). Guard the
// include with __has_include so such toolchains fall back to the scalar path.
#if defined(JSON_HAS_CPP_17) && defined(__has_include)
#if __has_include(<charconv>)
#include <charconv> // from_chars (only used when __cpp_lib_to_chars is defined)
#include <system_error> // errc
#endif
#endif
// This file contains the value-conversion helpers used by the lexer to turn an
// already-validated number token into a value, without the locale/errno
// overhead of std::strtoull/std::strtod. They are free functions so the lexer
// stays focused on scanning; see lexer::convert_number().
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
/*!
@brief fast integer parser for an already-validated unsigned integer
The number scanner has already checked that [first, last) is a valid JSON
integer, so this only needs to accumulate the digits and detect overflow. This
avoids the locale/errno machinery of std::strtoull, which dominates
integer-heavy inputs.
@param[in] first pointer to the first character (a digit)
@param[in] last pointer past the last character
@param[out] value the parsed value on success
@return true if the value fit into @a NumberUnsignedType; false on overflow, in
which case the caller falls back to floating-point parsing (matching the
previous std::strtoull behavior)
*/
template<typename NumberUnsignedType>
bool parse_integer_unsigned(const char* first, const char* last, NumberUnsignedType& value) noexcept
{
// accumulate in the widest unsigned type used by the previous strtoull
// path so the overflow behavior is unchanged for custom number types
std::uint64_t x = 0;
constexpr std::uint64_t cutoff = (std::numeric_limits<std::uint64_t>::max)() / 10u;
constexpr std::uint64_t cutlim = (std::numeric_limits<std::uint64_t>::max)() % 10u;
for (const char* p = first; p != last; ++p)
{
const auto digit = static_cast<std::uint64_t>(static_cast<unsigned char>(*p) - static_cast<unsigned char>('0'));
if (JSON_HEDLEY_UNLIKELY(x > cutoff || (x == cutoff && digit > cutlim)))
{
return false;
}
x = (x * 10u) + digit;
}
value = static_cast<NumberUnsignedType>(x);
// reject values that do not round-trip into a narrower NumberUnsignedType
return static_cast<std::uint64_t>(value) == x;
}
/*!
@brief fast integer parser for an already-validated negative integer
@param[in] first pointer to the leading '-'
@param[in] last pointer past the last character
@param[out] value the parsed (negative) value on success
@return true on success; false on overflow (caller falls back to float)
*/
template<typename NumberIntegerType>
bool parse_integer_signed(const char* first, const char* last, NumberIntegerType& value) noexcept
{
// the state machine only reaches the signed path via a leading '-'
JSON_ASSERT(first != last && *first == '-');
std::uint64_t magnitude = 0;
// |INT64_MIN| == INT64_MAX + 1; this is the largest admissible magnitude
constexpr std::uint64_t limit = static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()) + 1u;
for (const char* p = first + 1; p != last; ++p)
{
const auto digit = static_cast<std::uint64_t>(static_cast<unsigned char>(*p) - static_cast<unsigned char>('0'));
if (JSON_HEDLEY_UNLIKELY(magnitude > (limit - digit) / 10u))
{
return false;
}
magnitude = (magnitude * 10u) + digit;
}
const std::int64_t x = (magnitude == limit)
? (std::numeric_limits<std::int64_t>::min)()
: -static_cast<std::int64_t>(magnitude);
value = static_cast<NumberIntegerType>(x);
// reject values that do not round-trip into a narrower NumberIntegerType
return static_cast<std::int64_t>(value) == x;
}
/*!
@brief exact fast path for parsing a `double` (Clinger's algorithm)
For the common case - at most 19 significant digits, a decimal exponent in
[-22, 22], and a significand below 2^53 - the value equals significand *
10^exp computed in IEEE-754 double arithmetic, which is exact under
round-to-nearest because both operands are exactly representable. This is the
same fast path used by fast_float/simdjson; the general cases are left to
std::strtod. The parser only activates for number_float_t == double; float and
long double keep the std::strtof/std::strtold paths (see the templated overload
below).
@param[in] first pointer to the first character of the number
@param[in] last pointer past the last character
@param[in] decimal_point the (locale-dependent) decimal point character
@param[out] out the parsed value on success
@return true if the value was parsed exactly; false to fall back to strtod
*/
template<typename DecimalPointType>
bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
{
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
// Clinger's fast path is only exact when double operations are evaluated in
// true double precision. On platforms that keep intermediates in extended
// precision (e.g. the x87 FPU on 32-bit x86, where FLT_EVAL_METHOD == 2) the
// single significand * 10^scale step is double-rounded and can be 1 ULP off,
// so decline and let the caller fall back to the correctly-rounded
// std::from_chars / std::strtod path.
static_cast<void>(first);
static_cast<void>(last);
static_cast<void>(decimal_point);
static_cast<void>(out);
return false;
#else
static const std::array<double, 23> powers_of_ten =
{
{
1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, 1e10, 1e11,
1e12, 1e13, 1e14, 1e15, 1e16, 1e17, 1e18, 1e19, 1e20, 1e21, 1e22
}
};
const char* p = first;
bool negative = false;
if (p != last && (*p == '-' || *p == '+'))
{
negative = (*p == '-');
++p;
}
std::uint64_t significand = 0;
int num_digits = 0;
int fractional_digits = 0;
bool seen_dot = false;
bool any_digit = false;
for (; p != last; ++p)
{
const char c = *p;
if (c >= '0' && c <= '9')
{
any_digit = true;
if (JSON_HEDLEY_UNLIKELY(num_digits >= 19))
{
return false; // significand may not fit into uint64_t
}
significand = (significand * 10u) + static_cast<std::uint64_t>(c - '0');
++num_digits;
fractional_digits += static_cast<int>(seen_dot);
}
else if (static_cast<DecimalPointType>(c) == decimal_point)
{
if (JSON_HEDLEY_UNLIKELY(seen_dot))
{
return false;
}
seen_dot = true;
}
else if (c == 'e' || c == 'E')
{
++p;
break;
}
else
{
return false;
}
}
if (JSON_HEDLEY_UNLIKELY(!any_digit))
{
return false;
}
int exponent = 0;
if (p != last) // an exponent part remains
{
bool exp_negative = false;
if (p != last && (*p == '-' || *p == '+'))
{
exp_negative = (*p == '-');
++p;
}
bool any_exp_digit = false;
for (; p != last; ++p)
{
if (JSON_HEDLEY_UNLIKELY(*p < '0' || *p > '9'))
{
return false;
}
exponent = (exponent * 10) + (*p - '0');
any_exp_digit = true;
if (JSON_HEDLEY_UNLIKELY(exponent > 9999))
{
return false;
}
}
if (JSON_HEDLEY_UNLIKELY(!any_exp_digit))
{
return false;
}
if (exp_negative)
{
exponent = -exponent;
}
}
const int scale = exponent - fractional_digits;
if (JSON_HEDLEY_UNLIKELY(significand >= (static_cast<std::uint64_t>(1) << 53)))
{
return false; // significand not exactly representable as double
}
auto result = static_cast<double>(significand);
if (scale >= 0)
{
if (JSON_HEDLEY_UNLIKELY(scale > 22))
{
return false;
}
result *= powers_of_ten[static_cast<std::size_t>(scale)];
}
else
{
if (JSON_HEDLEY_UNLIKELY(-scale > 22))
{
return false;
}
result /= powers_of_ten[static_cast<std::size_t>(-scale)];
}
out = negative ? -result : result;
return true;
#endif
}
/// fast float path is only exact for `double`; decline for float/long double
template<typename DecimalPointType, typename FloatType>
bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
{
return false;
}
/*!
@brief parse a float with std::from_chars (Eisel-Lemire) when available
std::from_chars is locale-independent, correctly rounded, and - via the
Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
over the whole value range (not just the Clinger subset). It is used only when
__cpp_lib_to_chars indicates full floating-point support and only when it
consumes the entire token ([first, last)); a partial parse means the buffer
uses a non-'.' locale decimal point, in which case the caller falls back to the
locale-aware path. An under-/overflow (result_out_of_range) also declines, so
the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
expects (side-stepping the P4168 divergence between implementations).
@return true if the value was parsed exactly and fully; false to fall back
*/
template<typename FloatType>
bool parse_float_from_chars(const char* first, const char* last, FloatType& out) noexcept
{
// JSON_HAS_CPP_17 must gate the use as well as the <charconv> include above:
// some standard libraries (e.g. libstdc++ 15) define __cpp_lib_to_chars even
// in C++14 mode, where <charconv> is not included.
#if defined(JSON_HAS_CPP_17) && defined(__cpp_lib_to_chars)
const auto result = std::from_chars(first, last, out);
return result.ec == std::errc() && result.ptr == last;
#else
static_cast<void>(first);
static_cast<void>(last);
static_cast<void>(out);
return false;
#endif
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+2 -3
View File
@@ -72,10 +72,9 @@ class parser
parser_callback_t<BasicJsonType> cb = nullptr, parser_callback_t<BasicJsonType> cb = nullptr,
const bool allow_exceptions_ = true, const bool allow_exceptions_ = true,
const bool ignore_comments = false, const bool ignore_comments = false,
const bool ignore_trailing_commas_ = false, const bool ignore_trailing_commas_ = false)
const bool discard_number_values_ = false)
: callback(std::move(cb)) : callback(std::move(cb))
, m_lexer(std::move(adapter), ignore_comments, discard_number_values_) , m_lexer(std::move(adapter), ignore_comments)
, allow_exceptions(allow_exceptions_) , allow_exceptions(allow_exceptions_)
, ignore_trailing_commas(ignore_trailing_commas_) , ignore_trailing_commas(ignore_trailing_commas_)
{ {
@@ -1,293 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstddef> // size_t
#include <cstdint> // uint64_t
#include <cstring> // memcpy
#include <nlohmann/detail/macro_scope.hpp>
// Optional SIMD backend for bulk UTF-8 validation. This is an opt-in external
// dependency: nlohmann/json itself stays header-only and the C++11 scalar
// validator below is always available; defining JSON_USE_SIMDUTF additionally
// requires the simdutf headers on the include path and linking the simdutf
// library. See string_bulk_run().
//
// simdutf.h itself requires C++17 - it rejects older standards with an #error -
// so the backend is only compiled in from C++17 on. Below that the macro has no
// effect and the scalar validator is used; it accepts and rejects exactly the
// same input, so only throughput differs. macro_scope.hpp is included above to
// have JSON_HAS_CPP_17 available for this test.
#if defined(JSON_USE_SIMDUTF) && defined(JSON_HAS_CPP_17)
#include <simdutf.h>
#endif
// This file contains the byte-level string-scanning helpers used by the lexer's
// contiguous fast path. They operate purely on raw bytes (no dependency on the
// lexer's template parameters) so they are free functions, keeping the lexer
// itself focused on the state machine; see lexer::scan_string_bulk().
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
// classify a single byte as needing individual string handling: the closing
// quote, an escape, a control character, or a non-ASCII (UTF-8)
// lead/continuation byte. Ordinary bytes (0x20..0x7F except '"' and '\\') are
// copied verbatim, which the bulk scanner does 8 bytes at a time.
inline bool is_string_special(unsigned char c) noexcept
{
return c == '\"' || c == '\\' || c < 0x20u || c >= 0x80u;
}
// SWAR helper: return a word whose high bit is set in every byte of @a v that
// is_string_special(); zero if the 8 bytes are all ordinary.
inline std::uint64_t swar_string_special(std::uint64_t v) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22)
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
const std::uint64_t has_quote = (q - ones) & ~q & high;
const std::uint64_t has_backslash = (b - ones) & ~b & high;
const std::uint64_t has_control = (v - 0x2020202020202020ull) & ~v & high; // < 0x20
const std::uint64_t has_non_ascii = v & high; // >= 0x80
return has_quote | has_backslash | has_control | has_non_ascii;
}
// return the index of the first is_string_special() byte in [data, data+n), or
// n if every byte is ordinary; scans 8 bytes at a time
inline std::size_t find_string_special(const unsigned char* data, std::size_t n) noexcept
{
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t word = 0;
std::memcpy(&word, data + i, sizeof(word));
if (swar_string_special(word) != 0)
{
// a special byte is in this word; locate it (endian-agnostic)
for (std::size_t j = 0; j < 8; ++j)
{
if (is_string_special(data[i + j]))
{
return i + j;
}
}
}
}
for (; i < n; ++i)
{
if (is_string_special(data[i]))
{
return i;
}
}
return n;
}
// classify a byte as one the serializer must NOT copy verbatim when
// ensure_ascii is requested: the closing quote, an escape, a control character
// (< 0x20), DEL (0x7F), or any non-ASCII byte (>= 0x80). Everything else -
// printable ASCII except '"' and '\\' - is emitted unchanged. Note this differs
// from is_string_special() only in that 0x7F is also a stop (it is escaped as
// \u007f under ensure_ascii).
inline bool is_ascii_copyable(unsigned char c) noexcept
{
return c >= 0x20u && c < 0x7Fu && c != '\"' && c != '\\';
}
// return the index of the first byte in [data, data+n) that is NOT
// is_ascii_copyable(), or n if every byte can be copied verbatim; scans 8 bytes
// at a time. Used by the serializer's ensure_ascii fast path.
inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_t n) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t v = 0;
std::memcpy(&v, data + i, sizeof(v));
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22)
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL (0x7F)
const std::uint64_t stop = ((q - ones) & ~q & high) // == '"'
| ((b - ones) & ~b & high) // == '\\'
| ((d - ones) & ~d & high) // == 0x7F
| ((v - 0x2020202020202020ull) & ~v & high) // < 0x20
| (v & high); // >= 0x80
if (stop != 0)
{
for (std::size_t j = 0; j < 8; ++j)
{
if (!is_ascii_copyable(data[i + j]))
{
return i + j;
}
}
}
}
for (; i < n; ++i)
{
if (!is_ascii_copyable(data[i]))
{
return i;
}
}
return n;
}
// Validate one UTF-8 sequence at the front of [data, data+avail). Returns its
// length (2..4) only when the bytes form a *well-formed* sequence using exactly
// the same ranges as scan_string()'s per-byte switch, so the bulk path accepts
// precisely what the byte path accepts. Returns 0 for anything that is invalid,
// incomplete, or that the byte path must diagnose (the caller then defers to
// that path, keeping error messages unchanged). Lead bytes < 0x80 are handled
// by the caller and never passed here.
inline std::size_t validate_one_utf8(const unsigned char* data, std::size_t avail) noexcept
{
const unsigned char c0 = data[0];
if (c0 >= 0xC2 && c0 <= 0xDF) // U+0080..U+07FF
{
if (avail >= 2 && data[1] >= 0x80 && data[1] <= 0xBF)
{
return 2;
}
}
else if (c0 == 0xE0) // U+0800..U+0FFF
{
if (avail >= 3 && data[1] >= 0xA0 && data[1] <= 0xBF && data[2] >= 0x80 && data[2] <= 0xBF)
{
return 3;
}
}
else if ((c0 >= 0xE1 && c0 <= 0xEC) || c0 == 0xEE || c0 == 0xEF) // U+1000..U+CFFF, U+E000..U+FFFF
{
if (avail >= 3 && data[1] >= 0x80 && data[1] <= 0xBF && data[2] >= 0x80 && data[2] <= 0xBF)
{
return 3;
}
}
else if (c0 == 0xED) // U+D000..U+D7FF (excludes surrogates)
{
if (avail >= 3 && data[1] >= 0x80 && data[1] <= 0x9F && data[2] >= 0x80 && data[2] <= 0xBF)
{
return 3;
}
}
else if (c0 == 0xF0) // U+10000..U+3FFFF
{
if (avail >= 4 && data[1] >= 0x90 && data[1] <= 0xBF && data[2] >= 0x80 && data[2] <= 0xBF && data[3] >= 0x80 && data[3] <= 0xBF)
{
return 4;
}
}
else if (c0 >= 0xF1 && c0 <= 0xF3) // U+40000..U+FFFFF
{
if (avail >= 4 && data[1] >= 0x80 && data[1] <= 0xBF && data[2] >= 0x80 && data[2] <= 0xBF && data[3] >= 0x80 && data[3] <= 0xBF)
{
return 4;
}
}
else if (c0 == 0xF4) // U+100000..U+10FFFF
{
if (avail >= 4 && data[1] >= 0x80 && data[1] <= 0x8F && data[2] >= 0x80 && data[2] <= 0xBF && data[3] >= 0x80 && data[3] <= 0xBF)
{
return 4;
}
}
return 0; // invalid, incomplete, or must be diagnosed by the byte path
}
// Scalar (C++11) computation of the bulk run length: the number of leading
// bytes in [data, data+n) that are ordinary ASCII or complete well-formed UTF-8
// sequences, stopping before the first byte that needs individual handling (the
// closing quote, an escape, a control character, or an ill-formed/truncated
// sequence). ASCII is skipped 8 bytes at a time.
inline std::size_t scalar_string_bulk_run(const unsigned char* data, std::size_t n) noexcept
{
std::size_t pos = 0;
while (pos < n)
{
pos += find_string_special(data + pos, n - pos);
if (pos >= n || data[pos] < 0x80u)
{
break; // end of buffer, or a quote/escape/control byte
}
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
{
break; // ill-formed or truncated: let the byte path diagnose it
}
pos += seq;
}
return pos;
}
#if defined(JSON_USE_SIMDUTF) && defined(JSON_HAS_CPP_17)
// Index of the first quote/escape/control byte in [data, data+n) (non-ASCII
// bytes are *not* stops here - the whole run is handed to simdutf), or n.
inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t n) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t v = 0;
std::memcpy(&v, data + i, sizeof(v));
const std::uint64_t q = v ^ 0x2222222222222222ull;
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull;
const std::uint64_t hit = ((q - ones) & ~q & high)
| ((b - ones) & ~b & high)
| ((v - 0x2020202020202020ull) & ~v & high);
if (hit != 0)
{
for (std::size_t j = 0; j < 8; ++j)
{
const unsigned char c = data[i + j];
if (c == '\"' || c == '\\' || c < 0x20u)
{
return i + j;
}
}
}
}
for (; i < n; ++i)
{
const unsigned char c = data[i];
if (c == '\"' || c == '\\' || c < 0x20u)
{
return i;
}
}
return n;
}
#endif
// Backend-dispatched bulk run length. With JSON_USE_SIMDUTF the run up to the
// next delimiter is validated in one shot by simdutf; on the rare failure the
// scalar helper recomputes the exact valid prefix so the byte path still
// produces the precise diagnostic. Without it, the pure scalar path is used.
inline std::size_t string_bulk_run(const unsigned char* data, std::size_t n) noexcept
{
#if defined(JSON_USE_SIMDUTF) && defined(JSON_HAS_CPP_17)
const std::size_t run = find_string_delimiter(data, n);
if (run != 0 && simdutf::validate_utf8(reinterpret_cast<const char*>(data), run))
{
return run;
}
#endif
return scalar_string_bulk_run(data, n);
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -18,7 +18,6 @@
#endif #endif
#include <nlohmann/detail/abi_macros.hpp> #include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/type_traits.hpp> #include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/detail/string_utils.hpp> #include <nlohmann/detail/string_utils.hpp>
#include <nlohmann/detail/value_t.hpp> #include <nlohmann/detail/value_t.hpp>
@@ -207,10 +206,10 @@ NLOHMANN_JSON_NAMESPACE_END
namespace std namespace std
{ {
// Fix: https://github.com/nlohmann/json/issues/1401
#if defined(__clang__) #if defined(__clang__)
JSON_HEDLEY_DIAGNOSTIC_PUSH // Fix: https://github.com/nlohmann/json/issues/1401
JSON_HEDLEY_PRAGMA(clang diagnostic ignored "-Wmismatched-tags") #pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wmismatched-tags"
#endif #endif
template<typename IteratorType> template<typename IteratorType>
class tuple_size<::nlohmann::detail::iteration_proxy_value<IteratorType>> // NOLINT(cert-dcl58-cpp) class tuple_size<::nlohmann::detail::iteration_proxy_value<IteratorType>> // NOLINT(cert-dcl58-cpp)
@@ -225,7 +224,7 @@ class tuple_element<N, ::nlohmann::detail::iteration_proxy_value<IteratorType >>
::nlohmann::detail::iteration_proxy_value<IteratorType >> ())); ::nlohmann::detail::iteration_proxy_value<IteratorType >> ()));
}; };
#if defined(__clang__) #if defined(__clang__)
JSON_HEDLEY_DIAGNOSTIC_POP #pragma clang diagnostic pop
#endif #endif
} // namespace std } // namespace std
-14
View File
@@ -748,20 +748,6 @@ class json_pointer
} }
} }
// the reference token consists only of digits at this point (cf. checks
// above); however, its numeric value might not be representable, in which
// case array_index() would throw out_of_range.404/410 -- contains() must
// not throw (see #5395), so such a reference token is treated as "not found"
errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.c_str(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX
|| magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{
// the array index cannot be represented as size_type
return false;
}
const auto idx = array_index<BasicJsonType>(reference_token); const auto idx = array_index<BasicJsonType>(reference_token);
if (idx >= ptr->size()) if (idx >= ptr->size())
{ {
@@ -1850,8 +1850,8 @@ class binary_writer
void write_compact_float(const number_float_t n, detail::input_format_t format) void write_compact_float(const number_float_t n, detail::input_format_t format)
{ {
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_PUSH #pragma GCC diagnostic push
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal") #pragma GCC diagnostic ignored "-Wfloat-equal"
#endif #endif
if (!std::isfinite(n) || ((static_cast<double>(n) >= static_cast<double>(std::numeric_limits<float>::lowest()) && if (!std::isfinite(n) || ((static_cast<double>(n) >= static_cast<double>(std::numeric_limits<float>::lowest()) &&
static_cast<double>(n) <= static_cast<double>((std::numeric_limits<float>::max)()) && static_cast<double>(n) <= static_cast<double>((std::numeric_limits<float>::max)()) &&
@@ -1870,7 +1870,7 @@ class binary_writer
write_number(n); write_number(n);
} }
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP #pragma GCC diagnostic pop
#endif #endif
} }
File diff suppressed because it is too large Load Diff
+12 -75
View File
@@ -164,12 +164,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
detail::parser_callback_t<basic_json>cb = nullptr, detail::parser_callback_t<basic_json>cb = nullptr,
const bool allow_exceptions = true, const bool allow_exceptions = true,
const bool ignore_comments = false, const bool ignore_comments = false,
const bool ignore_trailing_commas = false, const bool ignore_trailing_commas = false
const bool discard_number_values = false
) )
{ {
return ::nlohmann::detail::parser<basic_json, InputAdapterType>(std::move(adapter), return ::nlohmann::detail::parser<basic_json, InputAdapterType>(std::move(adapter),
std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas, discard_number_values); std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
} }
private: private:
@@ -1336,7 +1335,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief serialization /// @brief serialization
/// @sa https://json.nlohmann.me/api/basic_json/dump/ /// @sa https://json.nlohmann.me/api/basic_json/dump/
JSON_HEDLEY_WARN_UNUSED_RESULT
string_t dump(const int indent = -1, string_t dump(const int indent = -1,
const char indent_char = ' ', const char indent_char = ' ',
const bool ensure_ascii = false, const bool ensure_ascii = false,
@@ -1347,7 +1345,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (indent >= 0) if (indent >= 0)
{ {
s.dump(*this, true, ensure_ascii, static_cast<std::size_t>(indent)); s.dump(*this, true, ensure_ascii, static_cast<unsigned int>(indent));
} }
else else
{ {
@@ -1359,7 +1357,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return the type of the JSON value (explicit) /// @brief return the type of the JSON value (explicit)
/// @sa https://json.nlohmann.me/api/basic_json/type/ /// @sa https://json.nlohmann.me/api/basic_json/type/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr value_t type() const noexcept constexpr value_t type() const noexcept
{ {
return m_data.m_type; return m_data.m_type;
@@ -1367,7 +1364,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether type is primitive /// @brief return whether type is primitive
/// @sa https://json.nlohmann.me/api/basic_json/is_primitive/ /// @sa https://json.nlohmann.me/api/basic_json/is_primitive/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_primitive() const noexcept constexpr bool is_primitive() const noexcept
{ {
return is_null() || is_string() || is_boolean() || is_number() || is_binary(); return is_null() || is_string() || is_boolean() || is_number() || is_binary();
@@ -1375,7 +1371,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether type is structured /// @brief return whether type is structured
/// @sa https://json.nlohmann.me/api/basic_json/is_structured/ /// @sa https://json.nlohmann.me/api/basic_json/is_structured/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_structured() const noexcept constexpr bool is_structured() const noexcept
{ {
return is_array() || is_object(); return is_array() || is_object();
@@ -1383,7 +1378,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is null /// @brief return whether value is null
/// @sa https://json.nlohmann.me/api/basic_json/is_null/ /// @sa https://json.nlohmann.me/api/basic_json/is_null/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_null() const noexcept constexpr bool is_null() const noexcept
{ {
return m_data.m_type == value_t::null; return m_data.m_type == value_t::null;
@@ -1391,7 +1385,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is a boolean /// @brief return whether value is a boolean
/// @sa https://json.nlohmann.me/api/basic_json/is_boolean/ /// @sa https://json.nlohmann.me/api/basic_json/is_boolean/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_boolean() const noexcept constexpr bool is_boolean() const noexcept
{ {
return m_data.m_type == value_t::boolean; return m_data.m_type == value_t::boolean;
@@ -1399,7 +1392,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is a number /// @brief return whether value is a number
/// @sa https://json.nlohmann.me/api/basic_json/is_number/ /// @sa https://json.nlohmann.me/api/basic_json/is_number/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_number() const noexcept constexpr bool is_number() const noexcept
{ {
return is_number_integer() || is_number_float(); return is_number_integer() || is_number_float();
@@ -1407,7 +1399,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is an integer number /// @brief return whether value is an integer number
/// @sa https://json.nlohmann.me/api/basic_json/is_number_integer/ /// @sa https://json.nlohmann.me/api/basic_json/is_number_integer/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_number_integer() const noexcept constexpr bool is_number_integer() const noexcept
{ {
return m_data.m_type == value_t::number_integer || m_data.m_type == value_t::number_unsigned; return m_data.m_type == value_t::number_integer || m_data.m_type == value_t::number_unsigned;
@@ -1415,7 +1406,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is an unsigned integer number /// @brief return whether value is an unsigned integer number
/// @sa https://json.nlohmann.me/api/basic_json/is_number_unsigned/ /// @sa https://json.nlohmann.me/api/basic_json/is_number_unsigned/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_number_unsigned() const noexcept constexpr bool is_number_unsigned() const noexcept
{ {
return m_data.m_type == value_t::number_unsigned; return m_data.m_type == value_t::number_unsigned;
@@ -1423,7 +1413,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is a floating-point number /// @brief return whether value is a floating-point number
/// @sa https://json.nlohmann.me/api/basic_json/is_number_float/ /// @sa https://json.nlohmann.me/api/basic_json/is_number_float/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_number_float() const noexcept constexpr bool is_number_float() const noexcept
{ {
return m_data.m_type == value_t::number_float; return m_data.m_type == value_t::number_float;
@@ -1431,7 +1420,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is an object /// @brief return whether value is an object
/// @sa https://json.nlohmann.me/api/basic_json/is_object/ /// @sa https://json.nlohmann.me/api/basic_json/is_object/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_object() const noexcept constexpr bool is_object() const noexcept
{ {
return m_data.m_type == value_t::object; return m_data.m_type == value_t::object;
@@ -1439,7 +1427,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is an array /// @brief return whether value is an array
/// @sa https://json.nlohmann.me/api/basic_json/is_array/ /// @sa https://json.nlohmann.me/api/basic_json/is_array/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_array() const noexcept constexpr bool is_array() const noexcept
{ {
return m_data.m_type == value_t::array; return m_data.m_type == value_t::array;
@@ -1447,7 +1434,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is a string /// @brief return whether value is a string
/// @sa https://json.nlohmann.me/api/basic_json/is_string/ /// @sa https://json.nlohmann.me/api/basic_json/is_string/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_string() const noexcept constexpr bool is_string() const noexcept
{ {
return m_data.m_type == value_t::string; return m_data.m_type == value_t::string;
@@ -1455,7 +1441,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is a binary array /// @brief return whether value is a binary array
/// @sa https://json.nlohmann.me/api/basic_json/is_binary/ /// @sa https://json.nlohmann.me/api/basic_json/is_binary/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_binary() const noexcept constexpr bool is_binary() const noexcept
{ {
return m_data.m_type == value_t::binary; return m_data.m_type == value_t::binary;
@@ -1463,7 +1448,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return whether value is discarded /// @brief return whether value is discarded
/// @sa https://json.nlohmann.me/api/basic_json/is_discarded/ /// @sa https://json.nlohmann.me/api/basic_json/is_discarded/
JSON_HEDLEY_WARN_UNUSED_RESULT
constexpr bool is_discarded() const noexcept constexpr bool is_discarded() const noexcept
{ {
return m_data.m_type == value_t::discarded; return m_data.m_type == value_t::discarded;
@@ -2795,7 +2779,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief returns the number of occurrences of a key in a JSON object /// @brief returns the number of occurrences of a key in a JSON object
/// @sa https://json.nlohmann.me/api/basic_json/count/ /// @sa https://json.nlohmann.me/api/basic_json/count/
JSON_HEDLEY_WARN_UNUSED_RESULT
size_type count(const typename object_t::key_type& key) const size_type count(const typename object_t::key_type& key) const
{ {
// return 0 for all nonobject types // return 0 for all nonobject types
@@ -2806,7 +2789,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/count/ /// @sa https://json.nlohmann.me/api/basic_json/count/
template<class KeyType, detail::enable_if_t< template<class KeyType, detail::enable_if_t<
detail::is_usable_as_basic_json_key_type<basic_json_t, KeyType>::value, int> = 0> detail::is_usable_as_basic_json_key_type<basic_json_t, KeyType>::value, int> = 0>
JSON_HEDLEY_WARN_UNUSED_RESULT
size_type count(KeyType && key) const size_type count(KeyType && key) const
{ {
// return 0 for all nonobject types // return 0 for all nonobject types
@@ -2815,7 +2797,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief check the existence of an element in a JSON object /// @brief check the existence of an element in a JSON object
/// @sa https://json.nlohmann.me/api/basic_json/contains/ /// @sa https://json.nlohmann.me/api/basic_json/contains/
JSON_HEDLEY_WARN_UNUSED_RESULT
bool contains(const typename object_t::key_type& key) const bool contains(const typename object_t::key_type& key) const
{ {
return is_object() && m_data.m_value.object->find(key) != m_data.m_value.object->end(); return is_object() && m_data.m_value.object->find(key) != m_data.m_value.object->end();
@@ -2825,7 +2806,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/contains/ /// @sa https://json.nlohmann.me/api/basic_json/contains/
template<class KeyType, detail::enable_if_t< template<class KeyType, detail::enable_if_t<
detail::is_usable_as_basic_json_key_type<basic_json_t, KeyType>::value, int> = 0> detail::is_usable_as_basic_json_key_type<basic_json_t, KeyType>::value, int> = 0>
JSON_HEDLEY_WARN_UNUSED_RESULT
bool contains(KeyType && key) const bool contains(KeyType && key) const
{ {
return is_object() && m_data.m_value.object->find(std::forward<KeyType>(key)) != m_data.m_value.object->end(); return is_object() && m_data.m_value.object->find(std::forward<KeyType>(key)) != m_data.m_value.object->end();
@@ -2833,14 +2813,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief check the existence of an element in a JSON object given a JSON pointer /// @brief check the existence of an element in a JSON object given a JSON pointer
/// @sa https://json.nlohmann.me/api/basic_json/contains/ /// @sa https://json.nlohmann.me/api/basic_json/contains/
JSON_HEDLEY_WARN_UNUSED_RESULT
bool contains(const json_pointer& ptr) const bool contains(const json_pointer& ptr) const
{ {
return ptr.contains(this); return ptr.contains(this);
} }
template<typename BasicJsonType, detail::enable_if_t<detail::is_basic_json<BasicJsonType>::value, int> = 0> template<typename BasicJsonType, detail::enable_if_t<detail::is_basic_json<BasicJsonType>::value, int> = 0>
JSON_HEDLEY_WARN_UNUSED_RESULT
JSON_HEDLEY_DEPRECATED_FOR(3.11.0, basic_json::json_pointer or nlohmann::json_pointer<basic_json::string_t>) // NOLINT(readability/alt_tokens) JSON_HEDLEY_DEPRECATED_FOR(3.11.0, basic_json::json_pointer or nlohmann::json_pointer<basic_json::string_t>) // NOLINT(readability/alt_tokens)
bool contains(const typename ::nlohmann::json_pointer<BasicJsonType>& ptr) const bool contains(const typename ::nlohmann::json_pointer<BasicJsonType>& ptr) const
{ {
@@ -2996,7 +2974,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief checks whether the container is empty. /// @brief checks whether the container is empty.
/// @sa https://json.nlohmann.me/api/basic_json/empty/ /// @sa https://json.nlohmann.me/api/basic_json/empty/
JSON_HEDLEY_WARN_UNUSED_RESULT
bool empty() const noexcept bool empty() const noexcept
{ {
switch (m_data.m_type) switch (m_data.m_type)
@@ -3036,7 +3013,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief returns the number of elements /// @brief returns the number of elements
/// @sa https://json.nlohmann.me/api/basic_json/size/ /// @sa https://json.nlohmann.me/api/basic_json/size/
JSON_HEDLEY_WARN_UNUSED_RESULT
size_type size() const noexcept size_type size() const noexcept
{ {
switch (m_data.m_type) switch (m_data.m_type)
@@ -3076,7 +3052,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief returns the maximum possible number of elements /// @brief returns the maximum possible number of elements
/// @sa https://json.nlohmann.me/api/basic_json/max_size/ /// @sa https://json.nlohmann.me/api/basic_json/max_size/
JSON_HEDLEY_WARN_UNUSED_RESULT
size_type max_size() const noexcept size_type max_size() const noexcept
{ {
switch (m_data.m_type) switch (m_data.m_type)
@@ -3795,13 +3770,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
bool operator==(const_reference rhs) const noexcept bool operator==(const_reference rhs) const noexcept
{ {
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_PUSH #pragma GCC diagnostic push
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal") #pragma GCC diagnostic ignored "-Wfloat-equal"
#endif #endif
const_reference lhs = *this; const_reference lhs = *this;
JSON_IMPLEMENT_OPERATOR( ==, true, false, false) JSON_IMPLEMENT_OPERATOR( ==, true, false, false)
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP #pragma GCC diagnostic pop
#endif #endif
} }
@@ -3888,12 +3863,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend bool operator==(const_reference lhs, const_reference rhs) noexcept friend bool operator==(const_reference lhs, const_reference rhs) noexcept
{ {
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_PUSH #pragma GCC diagnostic push
JSON_HEDLEY_PRAGMA(GCC diagnostic ignored "-Wfloat-equal") #pragma GCC diagnostic ignored "-Wfloat-equal"
#endif #endif
JSON_IMPLEMENT_OPERATOR( ==, true, false, false) JSON_IMPLEMENT_OPERATOR( ==, true, false, false)
#ifdef __GNUC__ #ifdef __GNUC__
JSON_HEDLEY_DIAGNOSTIC_POP #pragma GCC diagnostic pop
#endif #endif
} }
@@ -4154,24 +4129,22 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief check if the input is valid JSON /// @brief check if the input is valid JSON
/// @sa https://json.nlohmann.me/api/basic_json/accept/ /// @sa https://json.nlohmann.me/api/basic_json/accept/
template<typename InputType> template<typename InputType>
JSON_HEDLEY_WARN_UNUSED_RESULT
static bool accept(InputType&& i, static bool accept(InputType&& i,
const bool ignore_comments = false, const bool ignore_comments = false,
const bool ignore_trailing_commas = false) const bool ignore_trailing_commas = false)
{ {
return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true); return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
} }
/// @brief check if the input is valid JSON (iterator pair, or iterator+sentinel pair for C++20 ranges support) /// @brief check if the input is valid JSON (iterator pair, or iterator+sentinel pair for C++20 ranges support)
/// @sa https://json.nlohmann.me/api/basic_json/accept/ /// @sa https://json.nlohmann.me/api/basic_json/accept/
template<typename IteratorType, typename SentinelType = IteratorType, template<typename IteratorType, typename SentinelType = IteratorType,
detail::enable_if_t<detail::can_compare_ne<IteratorType, SentinelType>::value, int> = 0> detail::enable_if_t<detail::can_compare_ne<IteratorType, SentinelType>::value, int> = 0>
JSON_HEDLEY_WARN_UNUSED_RESULT
static bool accept(IteratorType first, SentinelType last, static bool accept(IteratorType first, SentinelType last,
const bool ignore_comments = false, const bool ignore_comments = false,
const bool ignore_trailing_commas = false) const bool ignore_trailing_commas = false)
{ {
return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true); return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
} }
JSON_HEDLEY_WARN_UNUSED_RESULT JSON_HEDLEY_WARN_UNUSED_RESULT
@@ -4180,7 +4153,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool ignore_comments = false, const bool ignore_comments = false,
const bool ignore_trailing_commas = false) const bool ignore_trailing_commas = false)
{ {
return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true); return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas).accept(true);
} }
/// @brief generate SAX events /// @brief generate SAX events
@@ -4266,7 +4239,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @brief return the type as string /// @brief return the type as string
/// @sa https://json.nlohmann.me/api/basic_json/type_name/ /// @sa https://json.nlohmann.me/api/basic_json/type_name/
JSON_HEDLEY_WARN_UNUSED_RESULT
JSON_HEDLEY_RETURNS_NON_NULL JSON_HEDLEY_RETURNS_NON_NULL
const char* type_name() const noexcept const char* type_name() const noexcept
{ {
@@ -4967,36 +4939,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// note erase performs range check // note erase performs range check
parent.erase(json_pointer::template array_index<basic_json_t>(last_path)); parent.erase(json_pointer::template array_index<basic_json_t>(last_path));
} }
else
{
// the parent of a "remove" target must be an object or array
// (see #5396)
JSON_THROW(out_of_range::create(413, detail::concat("cannot remove value: the JSON Patch 'remove' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
}
};
// RFC 6902 (section 4.4) forbids "from" from being a proper prefix
// of "path" for a "move" operation: a location cannot be moved into
// one of its own children. Compares reference tokens (already
// unescaped by json_pointer's parser) rather than the raw pointer
// strings, since a token may itself contain an escaped '/' or '~'
// that would defeat a naive string-prefix comparison. "from" equal
// to "path" is *not* a proper prefix and must return false.
const auto is_proper_prefix = [](const json_pointer & from, const json_pointer & to)
{
const auto from_size = from.reference_tokens.size();
if (from_size >= to.reference_tokens.size())
{
return false;
}
for (std::size_t i = 0; i < from_size; ++i)
{
if (!(from.reference_tokens[i] == to.reference_tokens[i]))
{
return false;
}
}
return true;
}; };
// type check: top level value must be an array // type check: top level value must be an array
@@ -5074,11 +5016,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const auto from_path = get_value("move", "from", true).template get<string_t>(); const auto from_path = get_value("move", "from", true).template get<string_t>();
json_pointer from_ptr(from_path); json_pointer from_ptr(from_path);
if (JSON_HEDLEY_UNLIKELY(is_proper_prefix(from_ptr, ptr)))
{
JSON_THROW(out_of_range::create(414, detail::concat("cannot move value: 'from' path '", from_path, "' is a proper prefix of 'path' '", path, "'"), &result));
}
// the "from" location must exist - use at() // the "from" location must exist - use at()
basic_json const v = result.at(from_ptr); basic_json const v = result.at(from_ptr);
+1 -4
View File
@@ -17,7 +17,7 @@
#undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_BUILTIN #undef JSON_HEDLEY_CLANG_HAS_BUILTIN
#undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE #undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE
#undef JSON_HEDLEY_CLANG_HAS_EXTENSION #undef JSON_HEDLEY_CLANG_HAS_EXTENSION
#undef JSON_HEDLEY_CLANG_HAS_FEATURE #undef JSON_HEDLEY_CLANG_HAS_FEATURE
#undef JSON_HEDLEY_CLANG_HAS_WARNING #undef JSON_HEDLEY_CLANG_HAS_WARNING
@@ -108,10 +108,7 @@
#undef JSON_HEDLEY_PELLES_VERSION_CHECK #undef JSON_HEDLEY_PELLES_VERSION_CHECK
#undef JSON_HEDLEY_PGI_VERSION #undef JSON_HEDLEY_PGI_VERSION
#undef JSON_HEDLEY_PGI_VERSION_CHECK #undef JSON_HEDLEY_PGI_VERSION_CHECK
#undef JSON_HEDLEY_PRAGMA
#undef JSON_HEDLEY_PREDICT #undef JSON_HEDLEY_PREDICT
#undef JSON_HEDLEY_PREDICT_FALSE
#undef JSON_HEDLEY_PREDICT_TRUE
#undef JSON_HEDLEY_PRINTF_FORMAT #undef JSON_HEDLEY_PRINTF_FORMAT
#undef JSON_HEDLEY_PRIVATE #undef JSON_HEDLEY_PRIVATE
#undef JSON_HEDLEY_PUBLIC #undef JSON_HEDLEY_PUBLIC
File diff suppressed because it is too large Load Diff
-121
View File
@@ -2,9 +2,6 @@ cmake_minimum_required(VERSION 3.13...4.0)
option(JSON_Valgrind "Execute test suite with Valgrind." OFF) option(JSON_Valgrind "Execute test suite with Valgrind." OFF)
option(JSON_FastTests "Skip expensive/slow tests." OFF) option(JSON_FastTests "Skip expensive/slow tests." OFF)
option(JSON_TestSimdutf "Build the unit tests against the simdutf UTF-8 validation backend." OFF)
set(JSON_SIMDUTF_VERSION 9.1.0 CACHE STRING "The simdutf version used by JSON_TestSimdutf.")
set(JSON_32bitTest AUTO CACHE STRING "Enable the 32bit unit test (ON/OFF/AUTO/ONLY).") set(JSON_32bitTest AUTO CACHE STRING "Enable the 32bit unit test (ON/OFF/AUTO/ONLY).")
set(JSON_TestStandards "" CACHE STRING "The list of standards to test explicitly.") set(JSON_TestStandards "" CACHE STRING "The list of standards to test explicitly.")
@@ -128,51 +125,6 @@ json_test_set_test_options(test-unicode4 TEST_PROPERTIES TIMEOUT 3000)
# add unit tests # add unit tests
############################################################################# #############################################################################
# Generate the leak checks for every JSON_HEDLEY_* macro defined in
# hedley.hpp; tests/src/unit-no-macro-leak.cpp #include-s the result after
# nlohmann/json.hpp (see issue #5408). Using the shared
# cmake/scripts/gen_hedley_undef_check.cmake script (also used by `make
# update_hedley_undef`) instead of a hand-maintained list of macro names
# means this test can never go stale after a future `make update_hedley`.
set(hedley_hpp "${PROJECT_SOURCE_DIR}/include/nlohmann/thirdparty/hedley/hedley.hpp")
set(hedley_undef_check_script "${PROJECT_SOURCE_DIR}/cmake/scripts/gen_hedley_undef_check.cmake")
set(hedley_undef_checks "${PROJECT_BINARY_DIR}/include/hedley_undef_checks.inc")
# Reconfigure whenever the vendored header or the generator script changes,
# so a `cmake --build` after `make update_hedley` does not silently keep a
# stale generated file around.
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${hedley_hpp}"
"${hedley_undef_check_script}")
# Generate once at configure time, so the very first build (before any
# custom-command build step has run) already has an up-to-date file.
execute_process(
COMMAND ${CMAKE_COMMAND}
"-DHEDLEY_HPP=${hedley_hpp}"
"-DOUTPUT=${hedley_undef_checks}"
-DMODE=checks
-P "${hedley_undef_check_script}"
RESULT_VARIABLE hedley_undef_check_result
)
if(NOT hedley_undef_check_result EQUAL 0)
message(FATAL_ERROR "Failed to generate ${hedley_undef_checks}")
endif()
# Also (re)generate as a build step, so an incremental build after editing
# hedley.hpp without a full reconfigure still picks up the change.
add_custom_command(
OUTPUT "${hedley_undef_checks}"
COMMAND ${CMAKE_COMMAND}
"-DHEDLEY_HPP=${hedley_hpp}"
"-DOUTPUT=${hedley_undef_checks}"
-DMODE=checks
-P "${hedley_undef_check_script}"
DEPENDS "${hedley_hpp}" "${hedley_undef_check_script}"
COMMENT "Generating Hedley undef leak checks"
VERBATIM)
add_custom_target(generate_hedley_undef_checks DEPENDS "${hedley_undef_checks}")
if("${JSON_TestStandards}" STREQUAL "") if("${JSON_TestStandards}" STREQUAL "")
set(test_cxx_standards 11 14 17 20 23) set(test_cxx_standards 11 14 17 20 23)
unset(test_force) unset(test_force)
@@ -197,71 +149,6 @@ if(test_force)
endif() endif()
message(STATUS "${msg}") message(STATUS "${msg}")
#############################################################################
# optionally validate UTF-8 with simdutf (JSON_USE_SIMDUTF)
#############################################################################
# The simdutf backend is opt-in and not vendored, so it is fetched here rather
# than being a checked-in dependency. Everything below hangs off test_main,
# whose usage requirements every test target inherits; the library target and
# the installed CMake package are deliberately left untouched.
if (JSON_TestSimdutf)
# simdutf requires C++17, both to compile itself and to be reachable from
# the library, which keeps its scalar validator below that. Find a tested
# standard that satisfies it.
set(simdutf_standard "")
foreach(cxx_standard ${test_cxx_standards})
if(NOT cxx_standard LESS 17 AND compiler_supports_cpp_${cxx_standard})
set(simdutf_standard ${cxx_standard})
break()
endif()
endforeach()
if("${simdutf_standard}" STREQUAL "")
# Building simdutf would fail outright without a C++17 compiler, and
# even with one it would go unused if no C++17-or-later standard is
# tested. Say so and fall back to the scalar validator rather than
# failing the build.
if(NOT compiler_supports_cpp_17)
set(simdutf_reason "the compiler does not support C++17")
else()
set(simdutf_reason "no tested standard is C++17 or later (testing ${msg_standards})")
endif()
message(WARNING
"JSON_TestSimdutf is enabled, but ${simdutf_reason}. simdutf requires C++17, so it "
"is not fetched and JSON_USE_SIMDUTF is not defined: the tests run against the "
"built-in scalar UTF-8 validator instead. Set JSON_TestStandards to include 17 or "
"later, or build with a compiler that supports C++17.")
else()
if (CMAKE_VERSION VERSION_LESS 3.18)
message(FATAL_ERROR "JSON_TestSimdutf requires CMake 3.18 or later (simdutf's minimum).")
endif()
include(FetchContent)
# simdutf builds its tests and tools by default, and its tests pull
# further dependencies of their own; only the library is needed here
set(SIMDUTF_TESTS OFF CACHE BOOL "" FORCE)
set(SIMDUTF_TOOLS OFF CACHE BOOL "" FORCE)
set(SIMDUTF_BENCHMARKS OFF CACHE BOOL "" FORCE)
set(SIMDUTF_ICONV OFF CACHE BOOL "" FORCE)
FetchContent_Declare(simdutf
URL https://github.com/simdutf/simdutf/archive/refs/tags/v${JSON_SIMDUTF_VERSION}.tar.gz
DOWNLOAD_EXTRACT_TIMESTAMP TRUE
)
FetchContent_MakeAvailable(simdutf)
target_compile_definitions(test_main PUBLIC JSON_USE_SIMDUTF)
target_link_libraries(test_main PUBLIC simdutf::simdutf)
# simdutf.h requires C++17; below that the library keeps its scalar
# validator, so any C++11/14 test targets exercise the fallback and the
# C++17-and-later ones exercise simdutf. Both must agree.
message(STATUS "UTF-8 validation delegated to simdutf ${JSON_SIMDUTF_VERSION} for C++17 and later (JSON_USE_SIMDUTF)")
endif()
endif()
# *DO* use json_test_set_test_options() above this line # *DO* use json_test_set_test_options() above this line
json_test_should_build_32bit_test(json_32bit_test json_32bit_test_only "${JSON_32bitTest}") json_test_should_build_32bit_test(json_32bit_test json_32bit_test_only "${JSON_32bitTest}")
@@ -276,14 +163,6 @@ foreach(file ${files})
json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}) json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force})
endforeach() endforeach()
# tests/src/unit-no-macro-leak.cpp #include-s the generated leak-check file,
# so its test targets must be built after generate_hedley_undef_checks.
foreach(cxx_standard ${test_cxx_standards})
if(TARGET test-no-macro-leak_cpp${cxx_standard})
add_dependencies(test-no-macro-leak_cpp${cxx_standard} generate_hedley_undef_checks)
endif()
endforeach()
if(json_32bit_test_only) if(json_32bit_test_only)
# Skip all other tests in this file # Skip all other tests in this file
return() return()
+319
View File
@@ -214,4 +214,323 @@ static void BinaryToCbor(benchmark::State& state)
} }
BENCHMARK(BinaryToCbor)->RangeMultiplier(2)->Range(8, 8 << 12); BENCHMARK(BinaryToCbor)->RangeMultiplier(2)->Range(8, 8 << 12);
//////////////////////////////////////////////////////////////////////////////
// parse binary formats
//////////////////////////////////////////////////////////////////////////////
// Only MessagePack had a read benchmark (FromMsgpack above, left untouched so
// its numbers stay comparable across releases). The benchmarks below cover the
// other formats, and read from a contiguous buffer as well as from a FILE*:
// most callers pass a container, and the two adapters compile to different
// code. The test data repository ships JSON only, so the input for each is
// derived at setup time by serializing a parsed test file.
/// binary format to benchmark; the _optimized variants add UBJSON/BJData size
/// and type annotations, which the readers handle in a separate code path
enum class binary_format
{
cbor,
msgpack,
ubjson,
ubjson_optimized,
bjdata,
bjdata_optimized,
bson
};
static std::vector<std::uint8_t> to_binary(const json& j, const binary_format format)
{
switch (format)
{
case binary_format::cbor:
return json::to_cbor(j);
case binary_format::msgpack:
return json::to_msgpack(j);
case binary_format::ubjson:
return json::to_ubjson(j);
case binary_format::ubjson_optimized:
return json::to_ubjson(j, true, true);
case binary_format::bjdata:
return json::to_bjdata(j);
case binary_format::bjdata_optimized:
return json::to_bjdata(j, true, true);
case binary_format::bson:
default:
return json::to_bson(j);
}
}
static json from_binary(const std::vector<std::uint8_t>& bytes, const binary_format format)
{
switch (format)
{
case binary_format::cbor:
return json::from_cbor(bytes);
case binary_format::msgpack:
return json::from_msgpack(bytes);
case binary_format::ubjson:
case binary_format::ubjson_optimized:
return json::from_ubjson(bytes);
case binary_format::bjdata:
case binary_format::bjdata_optimized:
return json::from_bjdata(bytes);
case binary_format::bson:
default:
return json::from_bson(bytes);
}
}
static json from_binary(std::FILE* file, const binary_format format)
{
switch (format)
{
case binary_format::cbor:
return json::from_cbor(file);
case binary_format::msgpack:
return json::from_msgpack(file);
case binary_format::ubjson:
case binary_format::ubjson_optimized:
return json::from_ubjson(file);
case binary_format::bjdata:
case binary_format::bjdata_optimized:
return json::from_bjdata(file);
case binary_format::bson:
default:
return json::from_bson(file);
}
}
/*!
@brief serialize a parsed test file to @a format
Returns an empty vector and marks the benchmark as skipped if the file cannot
be represented in the format, rather than letting the exception escape: BSON
requires an object at the top level, and several test files are arrays.
*/
static std::vector<std::uint8_t> binary_input(benchmark::State& state, const char* filename, const binary_format format)
{
std::ifstream f(filename);
std::string const str((std::istreambuf_iterator<char>(f)), std::istreambuf_iterator<char>());
const json j = json::parse(str);
if (format == binary_format::bson && !j.is_object())
{
state.SkipWithError("BSON requires an object at the top level");
return {};
}
return to_binary(j, format);
}
static void FromBinaryBuffer(benchmark::State& state, const char* filename, const binary_format format)
{
const std::vector<std::uint8_t> bytes = binary_input(state, filename, format);
if (bytes.empty())
{
return;
}
for (auto _ : state)
{
// the value is destroyed outside the timed section, because destroying
// a large DOM is not what this benchmark measures
state.PauseTiming();
auto* j = new json();
state.ResumeTiming();
*j = from_binary(bytes, format);
state.PauseTiming();
delete j;
state.ResumeTiming();
}
state.SetBytesProcessed(state.iterations() * bytes.size());
}
BENCHMARK_CAPTURE(FromBinaryBuffer, cbor / jeopardy, TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryBuffer, cbor / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryBuffer, cbor / citm_catalog, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryBuffer, cbor / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryBuffer, cbor / floats, TEST_DATA_DIRECTORY "/regression/floats.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryBuffer, cbor / signed_ints, TEST_DATA_DIRECTORY "/regression/signed_ints.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryBuffer, msgpack / jeopardy, TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json", binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryBuffer, msgpack / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryBuffer, msgpack / citm_catalog, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryBuffer, msgpack / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryBuffer, ubjson / jeopardy, TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryBuffer, ubjson / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryBuffer, ubjson / citm_catalog, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryBuffer, ubjson / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryBuffer, ubjson_optimized / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::ubjson_optimized);
BENCHMARK_CAPTURE(FromBinaryBuffer, ubjson_optimized / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::ubjson_optimized);
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata_optimized / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bjdata_optimized);
BENCHMARK_CAPTURE(FromBinaryBuffer, bjdata_optimized / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bjdata_optimized);
// BSON requires an object at the top level, so the array-rooted test files
// (jeopardy and the regression files) cannot be captured here
BENCHMARK_CAPTURE(FromBinaryBuffer, bson / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryBuffer, bson / citm_catalog, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryBuffer, bson / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bson);
static void FromBinaryFile(benchmark::State& state, const char* filename, const binary_format format)
{
const std::vector<std::uint8_t> bytes = binary_input(state, filename, format);
if (bytes.empty())
{
return;
}
const char* tmp = "benchmark_input.bin";
std::ofstream o(tmp, std::ios::binary);
o.write(reinterpret_cast<const char*>(bytes.data()), static_cast<std::streamsize>(bytes.size()));
o.flush();
o.close();
for (auto _ : state)
{
state.PauseTiming();
auto* j = new json();
auto* file = std::fopen(tmp, "rb");
state.ResumeTiming();
*j = from_binary(file, format);
state.PauseTiming();
std::fclose(file);
delete j;
state.ResumeTiming();
}
state.SetBytesProcessed(state.iterations() * bytes.size());
}
BENCHMARK_CAPTURE(FromBinaryFile, cbor / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryFile, cbor / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryFile, ubjson / canada, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryFile, ubjson / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryFile, bjdata / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryFile, bson / twitter, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", binary_format::bson);
//////////////////////////////////////////////////////////////////////////////
// parse binary formats: value shapes
//////////////////////////////////////////////////////////////////////////////
// The test files above are wide and shallow, but the readers' cost is per
// container, so these cover the shapes that stress the container handling
// itself. Every shape is wrapped in an object so that BSON, which requires an
// object at the top level, measures the same value as the other formats.
/// deeply nested arrays: one container per level, no other work
static json make_nested()
{
json nested = json::array();
json* p = &nested;
for (std::size_t i = 1; i < 1000; ++i)
{
p->push_back(json::array());
p = &p->operator[](0);
}
json j = json::object();
j["data"] = std::move(nested);
return j;
}
/// many sibling containers: maximum container churn, minimum nesting
static json make_containers()
{
json data = json::array();
for (std::size_t i = 0; i < 100000; ++i)
{
data.push_back(json::array({1, 2}));
}
json j = json::object();
j["data"] = std::move(data);
return j;
}
/// one flat array of numbers: the scalar decoding path, which must not move
static json make_scalars()
{
json data = json::array();
for (std::size_t i = 0; i < 1000000; ++i)
{
data.push_back(i);
}
json j = json::object();
j["data"] = std::move(data);
return j;
}
static void FromBinaryShape(benchmark::State& state, json (*build)(), const binary_format format)
{
const std::vector<std::uint8_t> bytes = to_binary(build(), format);
for (auto _ : state)
{
state.PauseTiming();
auto* j = new json();
state.ResumeTiming();
*j = from_binary(bytes, format);
state.PauseTiming();
delete j;
state.ResumeTiming();
}
state.SetBytesProcessed(state.iterations() * bytes.size());
}
BENCHMARK_CAPTURE(FromBinaryShape, nested / cbor, make_nested, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, nested / msgpack, make_nested, binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryShape, nested / ubjson, make_nested, binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryShape, nested / bjdata, make_nested, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, nested / bson, make_nested, binary_format::bson);
BENCHMARK_CAPTURE(FromBinaryShape, containers / cbor, make_containers, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, containers / msgpack, make_containers, binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryShape, containers / ubjson, make_containers, binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryShape, containers / ubjson_optimized, make_containers, binary_format::ubjson_optimized);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bjdata, make_containers, binary_format::bjdata);
BENCHMARK_CAPTURE(FromBinaryShape, containers / bson, make_containers, binary_format::bson);
// BSON names every array element, so a large array measures key generation
// rather than scalar decoding and is left out here
BENCHMARK_CAPTURE(FromBinaryShape, scalars / cbor, make_scalars, binary_format::cbor);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / msgpack, make_scalars, binary_format::msgpack);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / ubjson, make_scalars, binary_format::ubjson);
BENCHMARK_CAPTURE(FromBinaryShape, scalars / bjdata, make_scalars, binary_format::bjdata);
/*!
@brief parse an indefinite-length CBOR string
The writer never emits this form, so the input is assembled by hand: 0x7F
opens the string, each chunk is a one-character string, and 0xFF closes it.
*/
static void FromCborChunkedString(benchmark::State& state, const std::size_t chunks)
{
std::vector<std::uint8_t> bytes;
bytes.reserve(2 * chunks + 2);
bytes.push_back(0x7F);
for (std::size_t i = 0; i < chunks; ++i)
{
bytes.push_back(0x61); // string of length 1
bytes.push_back(0x61); // 'a'
}
bytes.push_back(0xFF);
for (auto _ : state)
{
json j = json::from_cbor(bytes);
benchmark::DoNotOptimize(j);
}
state.SetBytesProcessed(state.iterations() * bytes.size());
}
BENCHMARK_CAPTURE(FromCborChunkedString, 10000 chunks, 10000);
BENCHMARK_MAIN(); BENCHMARK_MAIN();
-9
View File
@@ -15,15 +15,6 @@
namespace utils namespace utils
{ {
// Some tests intentionally discard the [[nodiscard]]/JSON_HEDLEY_WARN_UNUSED_RESULT
// return value of a call they only make to exercise its side effects (e.g. checking
// that it does not throw). A plain (void) cast on the call expression does not
// suppress GCC's warning for functions using the GNU __attribute__((warn_unused_result))
// form (as opposed to the C++17 [[nodiscard]] attribute) -- passing the value into an
// ordinary function call does.
template<typename T>
inline void ignore_return_value(T&& /*unused*/) noexcept {}
inline std::vector<std::uint8_t> read_binary_file(const std::string& filename) inline std::vector<std::uint8_t> read_binary_file(const std::string& filename)
{ {
std::ifstream file(filename, std::ios::binary); std::ifstream file(filename, std::ios::binary);
+3 -142
View File
@@ -38,54 +38,6 @@ class huge_binary_t : public std::vector<std::uint8_t>
using huge_binary_json = nlohmann::basic_json < using huge_binary_json = nlohmann::basic_json <
std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, huge_binary_t, void >; double, std::allocator, nlohmann::adl_serializer, huge_binary_t, void >;
// a string type that can be made to report a size beyond INT32_MAX without
// allocating that much memory, so BSON length overflow can be tested for
// strings and (embedded) documents as well, following the same idea as
// huge_binary_t.
//
// Unlike huge_binary_t (which is only ever used as the BSON *value* type),
// this type doubles as basic_json's StringType and is therefore also used
// for *object keys* (e.g. "s" or "nested" below). Only the designated test
// value is meant to lie about its size - if every huge_string_t (including
// keys) reported a huge size, the running totals computed while walking the
// BSON document (see calc_bson_object_size & friends in binary_writer.hpp)
// would need more than 32 bits, and on platforms where std::size_t is only
// 32 bits wide that arithmetic would silently wrap around, producing wrong
// (or even unguarded) lengths. The fake size is therefore opt-in via
// as_huge(), and plain strings - in particular object keys - keep reporting
// their real, small size.
class huge_string_t : public std::string
{
public:
using std::string::string;
huge_string_t(const std::string& s) : std::string(s) {} // NOLINT(google-explicit-constructor,hicpp-explicit-conversions)
// returns a copy of @a s whose size() pretends to be huge
static huge_string_t as_huge(const std::string& s)
{
huge_string_t result(s);
result.pretend_huge = true;
return result;
}
size_type size() const noexcept
{
if (pretend_huge)
{
// one byte more than the BSON length field can represent
return static_cast<size_type>((std::numeric_limits<std::int32_t>::max)()) + 1;
}
return std::string::size();
}
private:
bool pretend_huge = false;
};
using huge_string_json = nlohmann::basic_json <
std::map, std::vector, huge_string_t, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, std::vector<std::uint8_t>, void >;
} // namespace } // namespace
TEST_CASE("BSON") TEST_CASE("BSON")
@@ -153,36 +105,10 @@ TEST_CASE("BSON")
SECTION("lengths exceeding INT32_MAX cannot be serialized to BSON") SECTION("lengths exceeding INT32_MAX cannot be serialized to BSON")
{ {
// out_of_range.412 is thrown from a single shared helper huge_binary_json j;
// (to_bson_length) that guards the BSON length fields of binary j["b"] = huge_binary_json::binary(huge_binary_t{});
// values, strings, and (embedded) documents alike
SECTION("binary")
{
huge_binary_json j;
j["b"] = huge_binary_json::binary(huge_binary_t{});
CHECK_THROWS_WITH_AS(huge_binary_json::to_bson(j), "[json.exception.out_of_range.412] BSON length 2147483661 exceeds maximum of 2147483647", huge_binary_json::out_of_range&); CHECK_THROWS_WITH_AS(huge_binary_json::to_bson(j), "[json.exception.out_of_range.412] BSON length 2147483661 exceeds maximum of 2147483647", huge_binary_json::out_of_range&);
}
SECTION("string")
{
huge_string_json j;
j["s"] = huge_string_t::as_huge("value");
CHECK_THROWS_WITH_AS(huge_string_json::to_bson(j), "[json.exception.out_of_range.412] BSON length 2147483661 exceeds maximum of 2147483647", huge_string_json::out_of_range&);
}
SECTION("document")
{
// an oversized string nested one level deep makes the
// *embedded* document's own length exceed INT32_MAX as well
huge_string_json nested;
nested["s"] = huge_string_t::as_huge("value");
huge_string_json j;
j["nested"] = nested;
CHECK_THROWS_WITH_AS(huge_string_json::to_bson(j), "[json.exception.out_of_range.412] BSON length 2147483674 exceeds maximum of 2147483647", huge_string_json::out_of_range&);
}
} }
SECTION("string length must be at least 1") SECTION("string length must be at least 1")
@@ -267,23 +193,6 @@ TEST_CASE("BSON")
CHECK(json::from_bson(result, true, false) == j); CHECK(json::from_bson(result, true, false) == j);
} }
SECTION("non-empty object with bool from a non-0/1 byte (lenient parsing)")
{
// documented lenient behavior (see gh-5333): any non-zero byte
// is accepted as `true`, not just 0x01
std::vector<std::uint8_t> const input =
{
0x0D, 0x00, 0x00, 0x00, // size (little endian)
0x08, // entry: boolean
'e', 'n', 't', 'r', 'y', '\x00',
0x02, // value = 0x02 (neither 0x00 nor 0x01)
0x00 // end marker
};
const json expected = { { "entry", true } };
CHECK(json::from_bson(input) == expected);
}
SECTION("non-empty object with double") SECTION("non-empty object with double")
{ {
json const j = json const j =
@@ -590,29 +499,6 @@ TEST_CASE("BSON")
CHECK(json::from_bson(result, true, false) == j); CHECK(json::from_bson(result, true, false) == j);
} }
SECTION("array elements with non-conforming keys (lenient parsing)")
{
// documented lenient behavior (see gh-5333): BSON array element
// keys are not checked against the required decimal sequence
// "0", "1", "2", ... - elements are taken in encoded order
std::vector<std::uint8_t> const input =
{
0x26, 0x00, 0x00, 0x00, // size (little endian)
0x04, 'e', 'n', 't', 'r', 'y', '\x00', // entry: embedded array
0x1A, 0x00, 0x00, 0x00, // size (little endian)
0x10, '5', 0x00, 0x0A, 0x00, 0x00, 0x00, // key "5" (bogus) -> 10
0x10, 'x', 0x00, 0x14, 0x00, 0x00, 0x00, // key "x" (non-numeric) -> 20
0x10, '1', 0x00, 0x1E, 0x00, 0x00, 0x00, // key "1" (out of order) -> 30
0x00, // end marker (embedded array)
0x00 // end marker
};
const json expected = { { "entry", json::array({10, 20, 30}) } };
CHECK(json::from_bson(input) == expected);
}
SECTION("non-empty object with binary member") SECTION("non-empty object with binary member")
{ {
const size_t N = 10; const size_t N = 10;
@@ -708,31 +594,6 @@ TEST_CASE("BSON")
CHECK(json::from_bson(result, true, false) == j); CHECK(json::from_bson(result, true, false) == j);
} }
SECTION("binary member with subtype 0x02 (old binary) keeps its inner length prefix (lenient parsing)")
{
// documented lenient behavior (see gh-5333): the payload for
// binary subtype 0x02 ("old binary") is returned as-is,
// including its own inner 4-byte length prefix; it is not
// stripped or reinterpreted
std::vector<std::uint8_t> const input =
{
0x17, 0x00, 0x00, 0x00, // size (little endian)
0x05, 'e', 'n', 't', 'r', 'y', '\x00', // entry: binary
0x06, 0x00, 0x00, 0x00, // size of binary (little endian)
0x02, // "old binary" subtype
0x02, 0x00, 0x00, 0x00, // inner length prefix (part of the old-binary payload)
0x68, 0x69, // payload ('h', 'i')
0x00 // end marker
};
// the inner length prefix is part of the (unmodified) payload
const std::vector<std::uint8_t> expected_payload = {0x02, 0x00, 0x00, 0x00, 0x68, 0x69};
const json expected = { { "entry", json::binary(expected_payload, 0x02) } };
CHECK(json::from_bson(input) == expected);
}
SECTION("Some more complex document") SECTION("Some more complex document")
{ {
json const j = json const j =
-433
View File
@@ -12,11 +12,6 @@
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
using nlohmann::json; using nlohmann::json;
#include <cstdlib> // strtod
#include <sstream> // stringstream
#include <string> // string
#include <vector> // vector
namespace namespace
{ {
// shortcut to scan a string literal // shortcut to scan a string literal
@@ -229,431 +224,3 @@ TEST_CASE("lexer class")
CHECK((scan_string("/**//**//**/", true) == json::lexer::token_type::end_of_input)); CHECK((scan_string("/**//**//**/", true) == json::lexer::token_type::end_of_input));
} }
} }
TEST_CASE("lexer number fast path")
{
// The contiguous fast path (used for pointer/string input) must agree with
// the streaming byte path (used for std::istream) on token type, numeric
// value, and round-trip text for every well-formed number, and reject the
// same malformed numbers with the same message.
SECTION("contiguous vs streaming parity")
{
const std::vector<std::string> numbers =
{
"0", "-0", "1", "-1", "42", "-42", "10", "100", "1234567890",
"0.0", "-0.0", "3.14", "-3.14", "0.5", "-0.001", "123.456789",
"1e0", "1E0", "1e10", "1e-10", "1e+10", "1.5e3", "-2.5E-4",
"9223372036854775807", // INT64_MAX -> unsigned
"9223372036854775808", // INT64_MAX + 1 -> unsigned
"18446744073709551615", // UINT64_MAX -> unsigned
"18446744073709551616", // UINT64_MAX + 1 -> float
"-9223372036854775808", // INT64_MIN -> integer
"-9223372036854775809", // INT64_MIN - 1 -> float
"123456789012345678901234567890", // huge -> float
"0.30000000000000004", "2.2250738585072014e-308", "1e308",
// high-precision / wide-exponent values that exercise the
// std::from_chars (Eisel-Lemire) path beyond the Clinger subset
"1.7976931348623157e308", "1.2345678901234567e-250",
"9007199254740993", "5e-324", "1e-320"
};
for (const auto& n : numbers)
{
const std::string doc = "[" + n + "]";
// contiguous fast path
const json a = json::parse(doc);
// streaming byte path
std::stringstream ss(doc);
const json b = json::parse(ss);
CAPTURE(n);
CHECK(a == b);
CHECK(a.dump() == b.dump());
CHECK(a[0].type() == b[0].type());
}
}
SECTION("significant-digit gate for the Clinger fast path")
{
// Clinger's fast path needs a significand below 2^53, so it cannot
// succeed once the mantissa has 17 or more significant digits (the
// significand would be at least 10^16). The lexer skips the attempt
// there. That is only allowed to save work: every value must still come
// out bit-exactly, and both scanners must agree. In particular the gate
// must not fire for tokens whose leading zeros merely look like extra
// digits - "0.1234567890123456" has 16 significant digits, not 17.
const std::vector<std::string> numbers =
{
"1234567890123456", // 16 significant digits
"12345678901234567", // 17 -> attempt skipped
"123456789012345678", // 18 -> attempt skipped
"0.1234567890123456", // 16: the leading "0" is not significant
"0.12345678901234567", // 17
"0.00000000000000001", // 1, in a long token
"0.000000000000000012345678901234", // 14, in a long token
"-0.0000000000000000000001", // 1, negative
"1.0000000000000000", // 17: trailing zeros are significant here
"10000000000000000", // 17
"9007199254740992", // 2^53
"9007199254740993", // 2^53 + 1
"-65.613616999999977", // canada.json shape
"1.2345678901234567e-250", // 17 with an exponent
"1.234567890123456e-250", // 16 with an exponent
"1e10", "0.0", "-0.0", "0e0", "0.000123"
};
for (const auto& n : numbers)
{
CAPTURE(n);
const std::string doc = "[" + n + "]";
const json a = json::parse(doc); // contiguous fast path
std::stringstream ss(doc);
const json b = json::parse(ss); // streaming byte path
CHECK(a[0].type() == b[0].type());
CHECK(a == b);
if (a[0].is_number_float())
{
const double expected = std::strtod(n.c_str(), nullptr);
CHECK(a[0].get<double>() == expected);
CHECK(b[0].get<double>() == expected);
}
}
}
SECTION("token type classification")
{
CHECK((scan_string("0") == json::lexer::token_type::value_unsigned));
CHECK((scan_string("-1") == json::lexer::token_type::value_integer));
CHECK((scan_string("1.5") == json::lexer::token_type::value_float));
CHECK((scan_string("1e5") == json::lexer::token_type::value_float));
CHECK((scan_string("18446744073709551615") == json::lexer::token_type::value_unsigned));
CHECK((scan_string("18446744073709551616") == json::lexer::token_type::value_float));
CHECK((scan_string("-9223372036854775808") == json::lexer::token_type::value_integer));
CHECK((scan_string("-9223372036854775809") == json::lexer::token_type::value_float));
}
SECTION("malformed numbers are rejected identically")
{
for (const char* bad :
{"-", "1.", "1e", "1e+", "1.2e", "01", "-01", "1..2", "1.2.3"
})
{
CAPTURE(bad);
// the contiguous fast path must decline and let the byte path report
const std::string doc = std::string("[") + bad + "]";
CHECK_FALSE(json::accept(doc));
std::stringstream ss(doc);
CHECK_FALSE(json::accept(ss));
}
}
#if !defined(JSON_NOEXCEPTION)
// these sections parse invalid input, which aborts when exceptions are off
SECTION("exhaustive grammar parity with the streaming path")
{
// The JSON number grammar is encoded twice: once as the scan_number()
// state machine and once as the contiguous fast path. Enumerate every
// short string over the number alphabet and require the two encodings to
// agree exactly - on acceptance, on the reported error, and on the parsed
// value - so they cannot drift apart.
const std::string alphabet = "01.eE+-";
// full outcome of parsing @a doc, so a mismatch in type, value, or error
// message is caught, not just a mismatch in acceptance
const auto outcome = [](const std::string & doc, bool streaming) -> std::string
{
try
{
if (streaming)
{
std::stringstream ss(doc);
const json j = json::parse(ss);
return std::string(j[0].type_name()) + '|' + j.dump();
}
const json j = json::parse(doc);
return std::string(j[0].type_name()) + '|' + j.dump();
}
catch (const json::parse_error& e)
{
return {e.what()};
}
};
std::vector<std::string> mismatches;
std::vector<std::string> tokens{""};
for (std::size_t length = 1; length <= 4; ++length)
{
std::vector<std::string> next;
next.reserve(tokens.size() * alphabet.size());
for (const auto& prefix : tokens)
{
for (const char c : alphabet)
{
next.push_back(prefix + c);
}
}
tokens = next;
for (const auto& token : tokens)
{
const std::string doc = "[" + token + "]";
if (outcome(doc, false) != outcome(doc, true))
{
mismatches.push_back(doc);
}
}
}
// 7 + 49 + 343 + 2401 tokens
CHECK(tokens.size() == 2401);
CAPTURE(mismatches);
CHECK(mismatches.empty());
}
SECTION("error positions match the streaming path")
{
// Rejecting identically is not enough: the fast path must also report the
// error at the same position as the byte path. A number directly followed
// by a newline is the interesting case, because the byte path reaches the
// newline (which resets the column) and then ungets it.
// returns the parse_error message, or "" if the document parsed
const auto contiguous_error = [](const std::string & doc) -> std::string
{
try
{
const json j = json::parse(doc);
static_cast<void>(j);
}
catch (const json::parse_error& e)
{
return {e.what()};
}
return {};
};
const auto streaming_error = [](const std::string & doc) -> std::string
{
try
{
std::stringstream ss(doc);
const json j = json::parse(ss);
static_cast<void>(j);
}
catch (const json::parse_error& e)
{
return {e.what()};
}
return {};
};
for (const char* bad :
{"[01\n]", "[00\n]", "[-01\n]", "{1\n}", "[1\n2]", "[1.2.3\n]",
"[1 \n2]", "[\n1\n2]", "1\n2", "[01\r\n]", "[1e\n]", "[-\n]"
})
{
CAPTURE(bad);
const std::string doc = bad;
const std::string contiguous_what = contiguous_error(doc);
CHECK_FALSE(contiguous_what.empty());
CHECK(contiguous_what == streaming_error(doc));
}
// A number terminated by a newline must report the same position as the
// same number terminated by anything else: scan_number() reads the
// terminator and ungets it, so the reported column is the one reached
// after the number's last character - not the 0 that an unget() across
// the newline used to leave behind.
CHECK(contiguous_error("[01\n]") == contiguous_error("[01 ]"));
CHECK(contiguous_error("[01\n]") ==
"[json.exception.parse_error.101] parse error at line 1, column 3: "
"syntax error while parsing array - unexpected number literal; expected ']'");
// the same for a multi-character token, where the column of the last
// character (the '3' of "-2.5e3") differs from the column it starts at
CHECK(contiguous_error("null -2.5e3\nfalse") == contiguous_error("null -2.5e3 false"));
CHECK(contiguous_error("null -2.5e3\nfalse") ==
"[json.exception.parse_error.101] parse error at line 1, column 11: "
"syntax error while parsing value - unexpected number literal; expected end of input");
}
#endif
}
TEST_CASE("lexer string fast path")
{
// Build a byte string from explicit values: a hex escape in a string
// literal swallows every following hex digit, which makes sequences like
// "\xC3\xA9b" mean something other than they look like.
const auto bytes = [](std::initializer_list<int> values)
{
std::string result;
for (const int value : values)
{
result.push_back(static_cast<char>(value));
}
return result;
};
#if !defined(JSON_NOEXCEPTION)
// the full outcome of parsing @a doc: the parsed value, or the exact error
// message, so a mismatch in either is caught. Only usable with exceptions
// on: parsing invalid input aborts when they are off.
const auto outcome = [](const std::string & doc, bool streaming) -> std::string
{
try
{
if (streaming)
{
std::stringstream ss(doc);
const json j = json::parse(ss);
return j.dump();
}
const json j = json::parse(doc);
return j.dump();
}
// not just parse_error: if a bulk scanner ever let ill-formed UTF-8
// through, dump() would throw type_error.316, and that has to surface
// as a reported mismatch rather than as an uncaught exception
catch (const json::exception& e)
{
return {e.what()};
}
};
#endif
// once at the start of the string, once past the first 8-byte SWAR word, so
// the bulk scanner sees each case with and without a run behind it
const std::vector<std::size_t> offsets{0, 9};
#if !defined(JSON_NOEXCEPTION)
SECTION("exhaustive contiguous vs streaming parity")
{
// ordinary ASCII, both specials, a control byte, characters that make
// the preceding backslash a valid escape, a UTF-8 lead byte of each
// length, a continuation byte, and a byte that is never valid
const std::vector<std::string> alphabet =
{
"a", "\"", "\\", "n", "u", "0", bytes({0x01}),
bytes({0xC3}), bytes({0xA9}), bytes({0xE4}), bytes({0xF0}),
bytes({0x80}), bytes({0xFF})
};
std::vector<std::string> mismatches;
std::vector<std::string> tokens{""};
for (std::size_t length = 1; length <= 3; ++length)
{
std::vector<std::string> next;
next.reserve(tokens.size() * alphabet.size());
for (const auto& prefix : tokens)
{
for (const auto& symbol : alphabet)
{
next.push_back(prefix + symbol);
}
}
tokens = next;
for (const auto& token : tokens)
{
for (const std::size_t offset : offsets)
{
const std::string doc = "[\"" + std::string(offset, 'a') + token + "\"]";
if (outcome(doc, false) != outcome(doc, true))
{
mismatches.push_back(doc);
}
}
}
}
// 13 + 169 + 2197 tokens, each at two offsets
CHECK(tokens.size() == 2197);
CAPTURE(mismatches);
CHECK(mismatches.empty());
}
SECTION("special bytes at every offset of the SWAR stride")
{
// The bulk scanner consumes 8 bytes at a time and then a tail; place
// every kind of byte that ends a run at each offset across two words,
// so multibyte sequences also straddle the word boundary.
const std::vector<std::string> specials =
{
"\"", "\\", bytes({0x01}), bytes({0x1F}), bytes({0x7F}),
bytes({0xC3, 0xA9}), bytes({0xE4, 0xB8, 0xAD}), bytes({0xF0, 0x9F, 0x98, 0x80}),
bytes({0xFF}), bytes({0xC3}), bytes({0xE4, 0xB8})
};
std::vector<std::string> mismatches;
for (std::size_t offset = 0; offset <= 17; ++offset)
{
for (const auto& special : specials)
{
const std::string doc = "[\"" + std::string(offset, 'a') + special + "\"]";
if (outcome(doc, false) != outcome(doc, true))
{
mismatches.push_back(doc);
}
}
}
CAPTURE(mismatches);
CHECK(mismatches.empty());
}
#endif
// json::accept() never throws, so the ranges stay covered without exceptions
SECTION("UTF-8 ranges are accepted and rejected as documented")
{
// The bulk validator must accept exactly what the byte-at-a-time
// scanner accepts, so pin the boundaries of every range it recognizes.
// aggregate, only ever brace-initialized below; default member
// initializers would stop it being an aggregate in C++11
struct utf8_case // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init)
{
std::string sequence;
bool valid;
const char* description;
};
const std::vector<utf8_case> cases =
{
{bytes({0xC2, 0x80}), true, "U+0080, shortest two-byte"},
{bytes({0xDF, 0xBF}), true, "U+07FF, longest two-byte"},
{bytes({0xC1, 0xBF}), false, "overlong two-byte"},
{bytes({0xC2, 0x7F}), false, "two-byte with bad continuation"},
{bytes({0xE0, 0xA0, 0x80}), true, "U+0800, shortest three-byte"},
{bytes({0xE0, 0x9F, 0xBF}), false, "overlong three-byte"},
{bytes({0xED, 0x9F, 0xBF}), true, "U+D7FF, just below the surrogates"},
{bytes({0xED, 0xA0, 0x80}), false, "surrogate U+D800"},
{bytes({0xED, 0xBF, 0xBF}), false, "surrogate U+DFFF"},
{bytes({0xEE, 0x80, 0x80}), true, "U+E000, just above the surrogates"},
{bytes({0xEF, 0xBF, 0xBF}), true, "U+FFFF"},
{bytes({0xF0, 0x90, 0x80, 0x80}), true, "U+10000, shortest four-byte"},
{bytes({0xF0, 0x8F, 0xBF, 0xBF}), false, "overlong four-byte"},
{bytes({0xF4, 0x8F, 0xBF, 0xBF}), true, "U+10FFFF, highest code point"},
{bytes({0xF4, 0x90, 0x80, 0x80}), false, "above U+10FFFF"},
{bytes({0xF5, 0x80, 0x80, 0x80}), false, "lead byte out of range"},
{bytes({0x80}), false, "bare continuation byte"},
{bytes({0xFF}), false, "byte that never appears in UTF-8"},
{bytes({0xC3}), false, "truncated two-byte"},
{bytes({0xE4, 0xB8}), false, "truncated three-byte"},
{bytes({0xF0, 0x9F, 0x98}), false, "truncated four-byte"}
};
for (const auto& test_case : cases)
{
CAPTURE(test_case.description);
for (const std::size_t offset : offsets)
{
CAPTURE(offset);
const std::string doc = "[\"" + std::string(offset, 'a') + test_case.sequence + "\"]";
CHECK(json::accept(doc) == test_case.valid);
#if !defined(JSON_NOEXCEPTION)
CHECK(outcome(doc, false) == outcome(doc, true));
#endif
}
}
}
}
+1 -161
View File
@@ -23,8 +23,6 @@ using nlohmann::json;
#include <utility> #include <utility>
#include <vector> #include <vector>
#include "test_utils.hpp"
namespace namespace
{ {
class SaxEventLogger class SaxEventLogger
@@ -626,8 +624,7 @@ TEST_CASE("parser class")
SECTION("overflow") SECTION("overflow")
{ {
// overflows during parsing yield an exception // overflows during parsing yield an exception
// empty() is nodiscard; the exception is thrown by parser_helper() itself, before empty() would run CHECK_THROWS_WITH_AS(parser_helper("1.18973e+4932").empty(), "[json.exception.out_of_range.406] number overflow parsing '1.18973e+4932'", json::out_of_range&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(parser_helper("1.18973e+4932").empty()), "[json.exception.out_of_range.406] number overflow parsing '1.18973e+4932'", json::out_of_range&);
} }
SECTION("invalid numbers") SECTION("invalid numbers")
@@ -933,98 +930,6 @@ TEST_CASE("parser class")
CHECK(accept_helper("+1") == false); CHECK(accept_helper("+1") == false);
CHECK(accept_helper("+0") == false); CHECK(accept_helper("+0") == false);
} }
SECTION("issue #5411 - skip conversion when accept() does not need the numeric value")
{
// lexer::scan_number() may skip strtoull()/strtoll() for
// value_unsigned/value_integer tokens when the caller (e.g.
// json::accept()) does not need the converted value, as long
// as the digit count alone guarantees no 64-bit overflow (see
// the "safe_digit_count" fast path in scan_number()). This
// differential test checks that json::accept() (which enables
// the fast path) and json::parse() (which never does) always
// agree, over a corpus that exercises both the fast path
// (<=18 digits) and the untouched, exact fallback path (>=19
// digits) -- including reclassification of huge digit-only
// integers to a (possibly non-finite) floating-point value.
const std::vector<std::pair<std::string, bool>> cases =
{
// normal small/large integers, both signs
{"0", true}, {"1", true}, {"-1", true}, {"42", true}, {"-42", true},
{"123456789", true}, {"-123456789", true},
// digit-count boundary around the 18-digit safe cutoff (both signs)
{std::string(17, '9'), true},
{std::string(18, '9'), true},
{std::string(19, '9'), true},
{std::string(20, '9'), true},
{"-" + std::string(17, '9'), true},
{"-" + std::string(18, '9'), true},
{"-" + std::string(19, '9'), true},
{"-" + std::string(20, '9'), true},
// 64-bit boundaries
{"9223372036854775807", true}, // INT64_MAX
{"-9223372036854775808", true}, // INT64_MIN
{"18446744073709551615", true}, // UINT64_MAX
{"18446744073709551616", true}, // UINT64_MAX + 1 (overflows uint64_t, finite double)
// the 28-digit example from the issue: overflows uint64_t
// but is finite as a double, so the scanner reclassifies
// it to value_float and it is accepted
{"9999999999999999999999999999", true},
// huge digit-only integers that overflow even a double -> rejected
{std::string(309, '9'), false},
{std::string(400, '9'), false},
{"1" + std::string(400, '0'), false},
// 1e999 / 1e400 style overflow -> rejected
{"1e999", false},
{"1e400", false},
{"-1e999", false},
{"1E999", false},
// values straddling DBL_MAX
{"1.7976931348623157e308", true}, // <= DBL_MAX, finite
{"1.7976931348623159e308", false}, // > DBL_MAX, overflows to inf
// a mix of other valid/invalid numeric syntax
{"3.14159", true},
{"-0.0", true},
{"1.0e10", true},
{"01", false},
{"-", false},
{"1.", false},
{"1e", false},
{"+1", false},
};
for (const auto& c : cases)
{
const std::string& number = c.first;
const bool expected = c.second;
CAPTURE(number)
CAPTURE(expected)
// accept() takes the fast path (skips conversion when possible)
CHECK(json::accept(number) == expected);
// parse() always performs the full conversion; it must agree
json j;
CHECK_NOTHROW(json::parser(nlohmann::detail::input_adapter(number), nullptr, false).parse(true, j));
CHECK(!j.is_discarded() == expected);
// wrap in an array so get_token() is exercised beyond the
// very first (constructor-time) scan as well
std::string wrapped = "[";
wrapped += number;
wrapped += ",";
wrapped += number;
wrapped += "]";
CHECK(json::accept(wrapped) == expected);
}
}
} }
} }
@@ -1489,71 +1394,6 @@ TEST_CASE("parser class")
CHECK(accept_helper("\"\\uD80C\\uFFFF\"") == false); CHECK(accept_helper("\"\\uD80C\\uFFFF\"") == false);
} }
#if !defined(JSON_NOEXCEPTION)
SECTION("issue #5412 - whitespace skipping bookkeeping (compact vs. pretty-printed)")
{
// lexer::skip_whitespace() reads its first character with get() (to
// honor a possibly pending unget() from the previous token) and every
// further whitespace character with get_ignoring_pending_unget() (a
// get() variant that skips the then-always-false next_unget check).
// This must not change the reported byte offset, line, or column of
// a syntax error, even when a long run of whitespace containing
// multiple newlines is skipped beforehand (as with pretty-printed
// input). The expected values below were captured from the
// unmodified do-while(get()) loop, so any regression that miscounts
// characters or newlines while skipping whitespace changes them.
const auto check_error = [](const std::string & input, std::size_t expected_byte,
const std::string & expected_what)
{
CAPTURE(input)
try
{
json _ = json::parse(input);
FAIL_CHECK("expected a parse_error, but parsing succeeded");
}
catch (const json::parse_error& e)
{
CHECK(e.byte == expected_byte);
CHECK(std::string(e.what()) == expected_what);
}
};
// a nested document, serialized both compactly and pretty-printed
// (dump(4)), each truncated right before the final closing '}' so
// that the parser hits EOF after skipping all of the (in the
// pretty-printed case, substantial) indentation whitespace
const json doc =
{
{"a", 1},
{"b", json::array({true, false, nullptr, "x"})},
{"c", json::object({{"d", 3.14}, {"e", json::array({1, 2, 3})}})}
};
const std::string compact = doc.dump();
const std::string pretty = doc.dump(4);
check_error(compact.substr(0, compact.size() - 1), 60,
"[json.exception.parse_error.101] parse error at line 1, column 60: syntax error while parsing object - unexpected end of input; expected '}'");
check_error(pretty.substr(0, pretty.size() - 1), 193,
"[json.exception.parse_error.101] parse error at line 17, column 1: syntax error while parsing object - unexpected end of input; expected '}'");
// an invalid token appearing after several indented, multi-line
// whitespace runs vs. the same document without any of that
// whitespace
check_error(R"({
"a": 1,
"b": [
true,
false
],
"c": @
})", 70,
"[json.exception.parse_error.101] parse error at line 7, column 10: syntax error while parsing value - invalid literal; last read: '\"c\": @'");
check_error(R"({"a":1,"b":[true,false],"c":@})", 29,
"[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing value - invalid literal; last read: '\"c\":@'");
}
#endif
SECTION("tests found by mutate++") SECTION("tests found by mutate++")
{ {
// test case to make sure no comma precedes the first key // test case to make sure no comma precedes the first key
@@ -22,8 +22,6 @@ using nlohmann::json;
#include <valarray> #include <valarray>
#include "test_utils.hpp"
namespace namespace
{ {
class SaxEventLogger class SaxEventLogger
@@ -631,8 +629,7 @@ TEST_CASE("parser class")
SECTION("overflow") SECTION("overflow")
{ {
// overflows during parsing yield an exception // overflows during parsing yield an exception
// empty() is nodiscard; the exception is thrown by parser_helper() itself, before empty() would run CHECK_THROWS_WITH_AS(parser_helper("1.18973e+4932").empty(), "[json.exception.out_of_range.406] number overflow parsing '1.18973e+4932'", json::out_of_range&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(parser_helper("1.18973e+4932").empty()), "[json.exception.out_of_range.406] number overflow parsing '1.18973e+4932'", json::out_of_range&);
} }
SECTION("invalid numbers") SECTION("invalid numbers")
-1
View File
@@ -100,7 +100,6 @@ void check_escaped(const char* original, const char* escaped, const bool ensure_
std::stringstream ss; std::stringstream ss;
json::serializer s(nlohmann::detail::output_adapter<char>(ss), ' '); json::serializer s(nlohmann::detail::output_adapter<char>(ss), ' ');
s.dump_escaped(original, ensure_ascii); s.dump_escaped(original, ensure_ascii);
s.flush(); // dump_escaped writes into the serializer's internal buffer
CHECK(ss.str() == escaped); CHECK(ss.str() == escaped);
} }
} // namespace } // namespace
-31
View File
@@ -1389,37 +1389,6 @@ TEST_CASE("value conversion")
// CHECK(m5["one"] == "eins"); // CHECK(m5["one"] == "eins");
} }
SECTION("reserve is called on containers that support it (#5406)")
{
// build a larger object so that a missing/incorrect reserve()
// call would be more likely to corrupt or drop elements
json j_large;
for (int i = 0; i < 100; ++i)
{
j_large[std::to_string(i)] = i;
}
SECTION("std::unordered_map (supports reserve)")
{
const auto m = j_large.get<std::unordered_map<std::string, int>>();
CHECK(m.size() == 100);
for (int i = 0; i < 100; ++i)
{
CHECK(m.at(std::to_string(i)) == i);
}
}
SECTION("std::map (no reserve, fallback path)")
{
const auto m = j_large.get<std::map<std::string, int>>();
CHECK(m.size() == 100);
for (int i = 0; i < 100; ++i)
{
CHECK(m.at(std::to_string(i)) == i);
}
}
}
SECTION("std::multimap") SECTION("std::multimap")
{ {
j1.get<std::multimap<std::string, int>>(); j1.get<std::multimap<std::string, int>>();
-42
View File
@@ -1,42 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// This file contains the C++17-only part of unit-items.cpp (structured
// bindings support for json::items()). It is kept in a separate
// translation unit so the (much larger) unit-items.cpp does not need to
// be compiled a second time just for this one SECTION.
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#ifdef JSON_HAS_CPP_17
#include <map>
#include <string>
TEST_CASE("items()")
{
SECTION("object")
{
SECTION("structured bindings")
{
json j = { {"A", 1}, {"B", 2} };
std::map<std::string, int> m;
for (auto const&[key, value] : j.items())
{
m.emplace(key, value);
}
CHECK(j.get<decltype(m)>() == m);
}
}
}
#endif
+16
View File
@@ -862,6 +862,22 @@ TEST_CASE("items()")
CHECK(counter == 3); CHECK(counter == 3);
} }
#ifdef JSON_HAS_CPP_17
SECTION("structured bindings")
{
json j = { {"A", 1}, {"B", 2} };
std::map<std::string, int> m;
for (auto const&[key, value] : j.items())
{
m.emplace(key, value);
}
CHECK(j.get<decltype(m)>() == m);
}
#endif
} }
SECTION("const object") SECTION("const object")
-186
View File
@@ -1389,192 +1389,6 @@ TEST_CASE("JSON patch - add to a primitive parent (regression #4292)")
} }
} }
TEST_CASE("JSON patch - remove with primitive or null parent (regression #5396)")
{
// Regression test for https://github.com/nlohmann/json/issues/5396
//
// RFC 6902 (§4.2) requires the target location of a "remove" operation
// to exist. When the target's parent resolves to a primitive value or
// null, the operation must fail. Previously operation_remove silently
// did nothing in this case (neither the "is_object" nor the "is_array"
// branch matched, and there was no final "else"), so the patch appeared
// to succeed without changing the document. It now throws
// out_of_range.413.
SECTION("parent is a primitive (number)")
{
json const doc = {{"a", 1}};
json const patch = {{{"op", "remove"}, {"path", "/a/b"}}};
#if JSON_DIAGNOSTICS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] (/a) cannot remove value: the JSON Patch 'remove' target's parent is of type number, but must be an object or array", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] cannot remove value: the JSON Patch 'remove' target's parent is of type number, but must be an object or array", json::out_of_range&);
#endif
}
SECTION("parent is a primitive (string)")
{
json const doc = {{"foo", {{"bar", "a string"}}}};
json const patch = {{{"op", "remove"}, {"path", "/foo/bar/baz"}}};
#if JSON_DIAGNOSTICS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] (/foo/bar) cannot remove value: the JSON Patch 'remove' target's parent is of type string, but must be an object or array", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] cannot remove value: the JSON Patch 'remove' target's parent is of type string, but must be an object or array", json::out_of_range&);
#endif
}
SECTION("top-level document is null")
{
json const doc = nullptr;
json const patch = {{{"op", "remove"}, {"path", "/a"}}};
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] cannot remove value: the JSON Patch 'remove' target's parent is of type null, but must be an object or array", json::out_of_range&);
}
SECTION("legitimate removes still work")
{
// object member
json const doc1 = {{"a", 1}, {"b", 2}};
json const patch1 = {{{"op", "remove"}, {"path", "/a"}}};
CHECK(doc1.patch(patch1) == json({{"b", 2}}));
// array element
json const doc2 = R"([1, 2, 3])"_json;
json const patch2 = {{{"op", "remove"}, {"path", "/1"}}};
CHECK(doc2.patch(patch2) == R"([1, 3])"_json);
}
}
TEST_CASE("JSON patch - move where 'from' is a proper prefix of 'path' (regression #5397)")
{
// Regression test for https://github.com/nlohmann/json/issues/5397
//
// RFC 6902 (§4.4) forbids "from" from being a proper prefix of "path"
// for a "move" operation: "a location cannot be moved into one of its
// children." "move" is implemented as remove-then-add; for an object
// target this happened to throw anyway as a side effect of the "add"
// step re-resolving through the now-removed parent, but for an array
// target the removal shifted subsequent indices, so "path" silently
// re-resolved to a different element and the operation "succeeded"
// with a corrupted result. It now throws out_of_range.414 for both
// object and array targets.
SECTION("array target (from the issue)")
{
json const doc = R"([[1,2],[3]])"_json;
json const patch = {{{"op", "move"}, {"from", "/0"}, {"path", "/0/0"}}};
#if JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] (bytes 0-11) cannot move value: 'from' path '/0' is a proper prefix of 'path' '/0/0'", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] cannot move value: 'from' path '/0' is a proper prefix of 'path' '/0/0'", json::out_of_range&);
#endif
}
SECTION("object target")
{
json const doc = R"({"a": {"b": 1}})"_json;
json const patch = {{{"op", "move"}, {"from", "/a"}, {"path", "/a/b"}}};
#if JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] (bytes 0-15) cannot move value: 'from' path '/a' is a proper prefix of 'path' '/a/b'", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] cannot move value: 'from' path '/a' is a proper prefix of 'path' '/a/b'", json::out_of_range&);
#endif
}
SECTION("from == path is not a proper prefix and must not be rejected")
{
// "from" equal to "path" is a no-op move; it is not a *proper*
// prefix relationship, so this new check must not reject it.
json const doc = R"({"a": 1, "b": 2})"_json;
json const patch = {{{"op", "move"}, {"from", "/a"}, {"path", "/a"}}};
CHECK(doc.patch(patch) == doc);
}
SECTION("raw string prefix that is not a pointer-token prefix must be allowed")
{
// "/ab" is a string-prefix of "/abc/x" as raw text, but "ab" and
// "abc" are different reference tokens, so this is NOT a
// pointer-token prefix relationship and the move must succeed.
// This is the key case proving the check compares tokens, not
// raw pointer text (a naive std::string prefix/rfind check on
// the undecoded pointer would wrongly reject this).
json const doc = R"({"ab": 1, "abc": {"x": 2}})"_json;
json const patch = {{{"op", "move"}, {"from", "/ab"}, {"path", "/abc/x"}}};
json const result = R"({"abc": {"x": 1}})"_json;
CHECK(doc.patch(patch) == result);
}
SECTION("escaped reference tokens are compared unescaped")
{
// "from" is the single token "a/b" (escaped as "a~1b"); "path"
// addresses member "x" of that same value, so "from" is a
// proper (token-level) prefix of "path" and must be rejected.
json const doc = R"({"a/b": {"x": 1}})"_json;
json const patch = {{{"op", "move"}, {"from", "/a~1b"}, {"path", "/a~1b/x"}}};
#if JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] (bytes 0-17) cannot move value: 'from' path '/a~1b' is a proper prefix of 'path' '/a~1b/x'", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] cannot move value: 'from' path '/a~1b' is a proper prefix of 'path' '/a~1b/x'", json::out_of_range&);
#endif
}
SECTION("ordinary valid moves still work")
{
// unrelated top-level members
json const doc1 = R"({"a": 1, "b": 2})"_json;
json const patch1 = {{{"op", "move"}, {"from", "/a"}, {"path", "/c"}}};
CHECK(doc1.patch(patch1) == R"({"b": 2, "c": 1})"_json);
// sibling paths that share a textual prefix but are unrelated
json const doc2 = R"({"a": {"x": 1}, "b": {"y": 2}})"_json;
json const patch2 = {{{"op", "move"}, {"from", "/a/x"}, {"path", "/b/z"}}};
CHECK(doc2.patch(patch2) == R"({"a": {}, "b": {"y": 2, "z": 1}})"_json);
// "path" is a proper prefix of "from" (the reverse relationship,
// which RFC 6902 does not forbid)
json const doc3 = R"({"a": {"b": 1}})"_json;
json const patch3 = {{{"op", "move"}, {"from", "/a/b"}, {"path", "/a"}}};
CHECK(doc3.patch(patch3) == R"({"a": 1})"_json);
}
SECTION("root 'from' is a proper prefix of every non-root 'path'")
{
// the whole document is a proper prefix of any location inside it
json const doc = R"({"a": 1})"_json;
json const patch = {{{"op", "move"}, {"from", ""}, {"path", "/a"}}};
#if JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] (bytes 0-8) cannot move value: 'from' path '' is a proper prefix of 'path' '/a'", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] cannot move value: 'from' path '' is a proper prefix of 'path' '/a'", json::out_of_range&);
#endif
}
SECTION("root 'path' is never a proper prefix violation for a non-root 'from'")
{
// the reverse of the above: moving a non-root location to the root
// is the "path is a prefix of from" relationship, which RFC 6902
// permits (already covered generally above; this pins the root
// case specifically, since root is the one path with no reference
// tokens at all)
json const doc = R"({"a": {"b": 1}})"_json;
json const patch = {{{"op", "move"}, {"from", "/a"}, {"path", ""}}};
CHECK(doc.patch(patch) == R"({"b": 1})"_json);
}
SECTION("the array-append token '-' is an ordinary child token")
{
// "-" (append-to-array) addresses a location *inside* the array,
// so "from" pointing at the array is still a proper prefix of
// "path" ending in "-" and must be rejected like any other child.
json const doc = R"({"a": [1, 2]})"_json;
json const patch = {{{"op", "move"}, {"from", "/a"}, {"path", "/a/-"}}};
#if JSON_DIAGNOSTIC_POSITIONS
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] (bytes 0-13) cannot move value: 'from' path '/a' is a proper prefix of 'path' '/a/-'", json::out_of_range&);
#else
CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.414] cannot move value: 'from' path '/a' is a proper prefix of 'path' '/a/-'", json::out_of_range&);
#endif
}
}
TEST_CASE("JSON patch - diff emits array removals in descending index order") TEST_CASE("JSON patch - diff emits array removals in descending index order")
{ {
SECTION("array shrunk to empty") SECTION("array shrunk to empty")
-42
View File
@@ -319,44 +319,6 @@ TEST_CASE("JSON pointers")
CHECK_THROWS_WITH_AS(j[jp] = 1, throw_msg.c_str(), json::out_of_range&); CHECK_THROWS_WITH_AS(j[jp] = 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j_const[jp] == 1, throw_msg.c_str(), json::out_of_range&); CHECK_THROWS_WITH_AS(j_const[jp] == 1, throw_msg.c_str(), json::out_of_range&);
// #5395: contains() must not throw for a reference token that is a
// syntactically valid array index but numerically exceeds ULLONG_MAX
// (causing strtoull() to set errno to ERANGE) -- it should just report
// that the pointer does not resolve to an element
CHECK(!j.contains(jp));
CHECK(!j_const.contains(jp));
}
{
// #5395: same as above, but using the exact reproduction from the issue
json::json_pointer const jp("/99999999999999999999");
std::string const throw_msg = "[json.exception.out_of_range.404] unresolved reference token '99999999999999999999'";
CHECK_THROWS_WITH_AS(j[jp] = 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j_const[jp] == 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j.at(jp) = 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j_const.at(jp) == 1, throw_msg.c_str(), json::out_of_range&);
CHECK(!j.contains(jp));
CHECK(!j_const.contains(jp));
}
{
// #5395: a reference token that is numerically representable in
// unsigned long long but exceeds size_type's max (e.g. ULLONG_MAX
// itself on typical 64-bit platforms, where size_type's max equals
// ULLONG_MAX) must not make contains() throw either
json::json_pointer const jp("/18446744073709551615");
std::string const throw_msg = "[json.exception.out_of_range.410] array index 18446744073709551615 exceeds size_type";
CHECK_THROWS_WITH_AS(j[jp] = 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j_const[jp] == 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j.at(jp) = 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j_const.at(jp) == 1, throw_msg.c_str(), json::out_of_range&);
CHECK(!j.contains(jp));
CHECK(!j_const.contains(jp));
} }
// on some machines, the check below is not constant // on some machines, the check below is not constant
@@ -372,10 +334,6 @@ TEST_CASE("JSON pointers")
CHECK_THROWS_WITH_AS(j[jp] = 1, throw_msg.c_str(), json::out_of_range&); CHECK_THROWS_WITH_AS(j[jp] = 1, throw_msg.c_str(), json::out_of_range&);
CHECK_THROWS_WITH_AS(j_const[jp] == 1, throw_msg.c_str(), json::out_of_range&); CHECK_THROWS_WITH_AS(j_const[jp] == 1, throw_msg.c_str(), json::out_of_range&);
// #5395: contains() must not throw for a reference token exceeding size_type's max
CHECK(!j.contains(jp));
CHECK(!j_const.contains(jp));
} }
DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP
-34
View File
@@ -1,34 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// This file makes sure that none of the internal JSON_HEDLEY_* macros (vendored
// from https://nemequ.github.io/hedley/, see
// include/nlohmann/thirdparty/hedley/hedley.hpp) leak into the including
// translation unit. include/nlohmann/detail/macro_unscope.hpp is supposed to
// #undef every JSON_HEDLEY_* macro (via hedley_undef.hpp) once json.hpp has
// been fully processed. See https://github.com/nlohmann/json/issues/5408,
// where JSON_HEDLEY_PRAGMA, JSON_HEDLEY_PREDICT_TRUE, JSON_HEDLEY_PREDICT_FALSE,
// and JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE escaped this cleanup because
// hedley_undef.hpp had no matching #undef for them.
//
// hedley_undef_checks.inc (included below) is generated at CMake configure/
// build time by cmake/scripts/gen_hedley_undef_check.cmake, which derives the
// full list of JSON_HEDLEY_* macro names directly from hedley.hpp. That way
// this test covers every macro Hedley actually defines -- not a hardcoded
// snapshot that would silently go stale the next time `make update_hedley`
// runs -- and can never drift from the vendored header.
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
TEST_CASE("JSON_HEDLEY macros do not leak after including json.hpp")
{
#include "hedley_undef_checks.inc"
CHECK(true); // keep an assertion when nothing leaked
}
+5 -3
View File
@@ -29,7 +29,10 @@ using nlohmann::json;
#include <limits> #include <limits>
#include <cstdio> #include <cstdio>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp"
#ifdef JSON_HAS_CPP_17
#include <variant>
#endif
#include "fifo_map.hpp" #include "fifo_map.hpp"
@@ -1370,8 +1373,7 @@ TEST_CASE("regression tests 1")
std::array<uint8_t, 28> key1 = {{ 103, 92, 117, 48, 48, 48, 55, 92, 114, 215, 126, 214, 95, 92, 34, 174, 40, 71, 38, 174, 40, 71, 38, 223, 134, 247, 127, 0 }}; std::array<uint8_t, 28> key1 = {{ 103, 92, 117, 48, 48, 48, 55, 92, 114, 215, 126, 214, 95, 92, 34, 174, 40, 71, 38, 174, 40, 71, 38, 223, 134, 247, 127, 0 }};
std::string const key1_str(reinterpret_cast<char*>(key1.data())); std::string const key1_str(reinterpret_cast<char*>(key1.data()));
json const j = key1_str; json const j = key1_str;
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_WITH_AS(j.dump(), "[json.exception.type_error.316] invalid UTF-8 byte at index 10: 0x7E", json::type_error&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(j.dump()), "[json.exception.type_error.316] invalid UTF-8 byte at index 10: 0x7E", json::type_error&);
} }
#if JSON_USE_IMPLICIT_CONVERSIONS #if JSON_USE_IMPLICIT_CONVERSIONS
+6 -11
View File
@@ -31,8 +31,6 @@ using ordered_json = nlohmann::ordered_json;
#include <type_traits> #include <type_traits>
#include <utility> #include <utility>
#include "test_utils.hpp"
#ifdef JSON_HAS_CPP_17 #ifdef JSON_HAS_CPP_17
#include <any> #include <any>
#include <variant> #include <variant>
@@ -641,8 +639,7 @@ TEST_CASE("regression tests 2")
s += static_cast<char>(i); s += static_cast<char>(i);
} }
dump_test["1"] = s; dump_test["1"] = s;
// dump() is nodiscard; this only checks that dumping does not throw/crash dump_test.dump(-1, ' ', true, nlohmann::json::error_handler_t::replace);
utils::ignore_return_value(dump_test.dump(-1, ' ', true, nlohmann::json::error_handler_t::replace));
} }
} }
@@ -734,14 +731,12 @@ TEST_CASE("regression tests 2")
{ {
const std::array<unsigned char, 23> data = {{0x81, 0xA4, 0x64, 0x61, 0x74, 0x61, 0xC4, 0x0F, 0x33, 0x30, 0x30, 0x32, 0x33, 0x34, 0x30, 0x31, 0x30, 0x37, 0x30, 0x35, 0x30, 0x31, 0x30}}; const std::array<unsigned char, 23> data = {{0x81, 0xA4, 0x64, 0x61, 0x74, 0x61, 0xC4, 0x0F, 0x33, 0x30, 0x30, 0x32, 0x33, 0x34, 0x30, 0x31, 0x30, 0x37, 0x30, 0x35, 0x30, 0x31, 0x30}};
const json j = json::from_msgpack(data.data(), data.size()); const json j = json::from_msgpack(data.data(), data.size());
// dump() is nodiscard; this only checks that dumping does not throw
CHECK_NOTHROW( CHECK_NOTHROW(
utils::ignore_return_value( j.dump(4, // Indent
j.dump(4, // Indent ' ', // Indent char
' ', // Indent char false, // Ensure ascii
false, // Ensure ascii json::error_handler_t::strict // Error
json::error_handler_t::strict // Error ));
)));
} }
SECTION("PR #2181 - regression bug with lvalue") SECTION("PR #2181 - regression bug with lvalue")
+6 -240
View File
@@ -15,8 +15,6 @@ using nlohmann::json;
#include <sstream> #include <sstream>
#include <iomanip> #include <iomanip>
#include "test_utils.hpp"
TEST_CASE("serialization") TEST_CASE("serialization")
{ {
SECTION("operator<<") SECTION("operator<<")
@@ -86,9 +84,8 @@ TEST_CASE("serialization")
{ {
const json j = "ä\xA9ü"; const json j = "ä\xA9ü";
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_WITH_AS(j.dump(), "[json.exception.type_error.316] invalid UTF-8 byte at index 2: 0xA9", json::type_error&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(j.dump()), "[json.exception.type_error.316] invalid UTF-8 byte at index 2: 0xA9", json::type_error&); CHECK_THROWS_WITH_AS(j.dump(1, ' ', false, json::error_handler_t::strict), "[json.exception.type_error.316] invalid UTF-8 byte at index 2: 0xA9", json::type_error&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(j.dump(1, ' ', false, json::error_handler_t::strict)), "[json.exception.type_error.316] invalid UTF-8 byte at index 2: 0xA9", json::type_error&);
CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"äü\""); CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"äü\"");
CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"ä\xEF\xBF\xBDü\""); CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"ä\xEF\xBF\xBDü\"");
CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"\\u00e4\\ufffd\\u00fc\""); CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"\\u00e4\\ufffd\\u00fc\"");
@@ -98,9 +95,8 @@ TEST_CASE("serialization")
{ {
const json j = "123\xC2"; const json j = "123\xC2";
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_WITH_AS(j.dump(), "[json.exception.type_error.316] incomplete UTF-8 string; last byte: 0xC2", json::type_error&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(j.dump()), "[json.exception.type_error.316] incomplete UTF-8 string; last byte: 0xC2", json::type_error&); CHECK_THROWS_AS(j.dump(1, ' ', false, json::error_handler_t::strict), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump(1, ' ', false, json::error_handler_t::strict)), json::type_error&);
CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123\""); CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123\"");
CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\""); CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\"");
CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd\""); CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd\"");
@@ -110,9 +106,8 @@ TEST_CASE("serialization")
{ {
const json j = "123\xF1\xB0\x34\x35\x36"; const json j = "123\xF1\xB0\x34\x35\x36";
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_WITH_AS(j.dump(), "[json.exception.type_error.316] invalid UTF-8 byte at index 5: 0x34", json::type_error&);
CHECK_THROWS_WITH_AS(utils::ignore_return_value(j.dump()), "[json.exception.type_error.316] invalid UTF-8 byte at index 5: 0x34", json::type_error&); CHECK_THROWS_AS(j.dump(1, ' ', false, json::error_handler_t::strict), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump(1, ' ', false, json::error_handler_t::strict)), json::type_error&);
CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123456\""); CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"123456\"");
CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\x34\x35\x36\""); CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"123\xEF\xBF\xBD\x34\x35\x36\"");
CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd456\""); CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"123\\ufffd456\"");
@@ -387,232 +382,3 @@ TEST_CASE("dump for basic_json with long double number_float_t")
check_same(100.0L, 100.0); check_same(100.0L, 100.0);
} }
} }
TEST_CASE("serialization of strings (bulk fast path)")
{
// These cases exercise the SWAR bulk-copy fast path in dump_escaped and the
// internal write buffer: long runs, escapes interrupting runs, 0x7F/DEL,
// multibyte UTF-8 under both ensure_ascii settings, and payloads larger than
// the write buffer.
SECTION("long unescaped ASCII exceeds the write buffer")
{
const std::string big(3000, 'a');
const json j = big;
CHECK(j.dump() == '"' + big + '"');
CHECK(j.dump(-1, ' ', true) == '"' + big + '"');
// round-trips
CHECK(json::parse(j.dump()) == j);
}
SECTION("runs interrupted by escapes")
{
const json j = std::string(500, 'x') + "\n\"\\" + std::string(500, 'y');
const std::string out = j.dump();
CHECK(out == '"' + std::string(500, 'x') + "\\n\\\"\\\\" + std::string(500, 'y') + '"');
CHECK(json::parse(out) == j);
}
SECTION("DEL (0x7F) depends on ensure_ascii")
{
const json j = std::string("a\x7f" "b");
CHECK(j.dump(-1, ' ', false) == "\"a\x7f" "b\""); // copied verbatim
CHECK(j.dump(-1, ' ', true) == "\"a\\u007fb\""); // escaped
}
SECTION("multibyte UTF-8 under both ensure_ascii settings")
{
const json j = std::string("A\xc3\xa9\xe4\xbd\xa0\xf0\x9f\x98\x80Z"); // A é 你 😀 Z
// not escaping non-ASCII: bytes are copied through the bulk validator
CHECK(j.dump(-1, ' ', false) == "\"A\xc3\xa9\xe4\xbd\xa0\xf0\x9f\x98\x80Z\"");
// ensure_ascii: escaped (with a surrogate pair for the emoji)
CHECK(j.dump(-1, ' ', true) == "\"A\\u00e9\\u4f60\\ud83d\\ude00Z\"");
CHECK(json::parse(j.dump(-1, ' ', true)) == j);
}
SECTION("many small structural writes exceed the write buffer")
{
json arr = json::array();
for (int i = 0; i < 2000; ++i)
{
arr.push_back(i);
}
const std::string out = arr.dump();
CHECK(out.front() == '[');
CHECK(out.back() == ']');
CHECK(json::parse(out) == arr);
json obj = json::object();
for (int i = 0; i < 500; ++i)
{
obj["key" + std::to_string(i)] = i;
}
CHECK(json::parse(obj.dump()) == obj);
CHECK(json::parse(obj.dump(2)) == obj);
// an array of many empty strings emits a long run of single-character
// writes ('"', '"', ',') at shallow nesting depth, so the write buffer
// fills and flushes mid-run without the deep recursion that would
// overflow the stack on some debug builds
json many_empty = json::array();
for (int i = 0; i < 500; ++i)
{
many_empty.push_back("");
}
const std::string out2 = many_empty.dump();
CHECK(out2.size() > 1024); // spans multiple write-buffer flushes
CHECK(out2.front() == '[');
CHECK(out2.back() == ']');
CHECK(json::parse(out2) == many_empty);
}
SECTION("invalid UTF-8 handling is unaffected by the fast path")
{
const json j = std::string("valid\xff" "more");
CHECK_THROWS_WITH_AS(j.dump(), "[json.exception.type_error.316] invalid UTF-8 byte at index 5: 0xFF", json::type_error&);
CHECK(j.dump(-1, ' ', false, json::error_handler_t::replace) == "\"valid\xef\xbf\xbd" "more\"");
CHECK(j.dump(-1, ' ', true, json::error_handler_t::replace) == "\"valid\\ufffdmore\"");
CHECK(j.dump(-1, ' ', false, json::error_handler_t::ignore) == "\"validmore\"");
}
}
TEST_CASE("indentation is written straight into the write buffer")
{
// put_indent() memsets the indentation into the write buffer instead of
// copying it out of a pre-grown indentation string. These cases cover an
// indentation wider than the buffer, a non-space indentation character, and
// nesting deep enough that the accumulated indentation spans several
// buffer-fulls - the situations the old grow-a-string approach got wrong.
SECTION("indent_step wider than the write buffer")
{
const json j = {{"a", 1}};
// 2000 > the 1024-byte write buffer, and > the 512 the indentation
// string used to start at
CHECK(j.dump(2000) == "{\n" + std::string(2000, ' ') + "\"a\": 1\n}");
// several whole buffer-fulls, so the buffer is refilled once and then
// flushed repeatedly
CHECK(j.dump(5000) == "{\n" + std::string(5000, ' ') + "\"a\": 1\n}");
CHECK(j.dump(5000, '\t') == "{\n" + std::string(5000, '\t') + "\"a\": 1\n}");
// an exact multiple of the buffer size
CHECK(j.dump(4096) == "{\n" + std::string(4096, ' ') + "\"a\": 1\n}");
}
SECTION("a non-space indentation character is used throughout")
{
const json j = {{"a", 1}};
// 600 is past the point where the indentation used to be grown, which
// is where a hard-coded space would have shown up
CHECK(j.dump(600, '\t') == "{\n" + std::string(600, '\t') + "\"a\": 1\n}");
CHECK(j.dump(3, '.') == "{\n...\"a\": 1\n}");
}
SECTION("accumulated indentation spans several buffer-fulls")
{
// five levels deep at 400 per level: the innermost value is indented by
// 2000 characters, reached in steps that each straddle the buffer end
json j = json::array({1});
for (int i = 0; i < 4; ++i)
{
j = json::array({j});
}
const std::string out = j.dump(400);
CHECK(out.find(std::string("\n") + std::string(2000, ' ') + "1\n") != std::string::npos);
CHECK(json::parse(out) == j);
}
SECTION("indentation is unchanged for ordinary widths")
{
const json j = {{"a", {1, 2}}, {"b", nullptr}};
CHECK(j.dump(2) == "{\n \"a\": [\n 1,\n 2\n ],\n \"b\": null\n}");
CHECK(j.dump(0) == "{\n\"a\": [\n1,\n2\n],\n\"b\": null\n}");
}
}
TEST_CASE("serialization of deeply nested values")
{
// dump() descends into a bounded number of levels and writes out whatever
// is nested deeper than that without the call stack; see
// https://github.com/nlohmann/json/issues/5387
SECTION("nested deeper than the call stack could follow")
{
// parsing is iterative, so building these costs little
const std::size_t depth = 100000;
const std::string array_text = std::string(depth, '[') + '0' + std::string(depth, ']');
CHECK(json::parse(array_text).dump() == array_text);
std::string object_text;
object_text.reserve((6 * depth) + 1);
for (std::size_t i = 0; i < depth; ++i)
{
object_text += "{\"a\":";
}
object_text += '1';
object_text.append(depth, '}');
CHECK(json::parse(object_text).dump() == object_text);
}
SECTION("depths around the bound of the descent")
{
// Cover every depth around the bound, so that the two ways of writing a
// value are known to meet cleanly - wherever the bound is set.
for (std::size_t d = 1; d <= 300; ++d)
{
CAPTURE(d);
const std::string array_text = std::string(d, '[') + '7' + std::string(d, ']');
CHECK(json::parse(array_text).dump() == array_text);
std::string object_text;
for (std::size_t i = 0; i < d; ++i)
{
object_text += "{\"k\":";
}
object_text += '7';
object_text.append(d, '}');
CHECK(json::parse(object_text).dump() == object_text);
}
}
SECTION("pretty-printing across the bound")
{
for (std::size_t d = 120; d <= 140; ++d)
{
CAPTURE(d);
const json j = json::parse(std::string(d, '[') + '7' + std::string(d, ']'));
std::string expected;
for (std::size_t i = 0; i < d; ++i)
{
expected += std::string(2 * i, ' ') + "[\n";
}
expected += std::string(2 * d, ' ') + '7';
for (std::size_t i = d; i > 0; --i)
{
expected += '\n' + std::string(2 * (i - 1), ' ') + ']';
}
CHECK(j.dump(2) == expected);
}
}
SECTION("an empty container below the bound")
{
// an empty container is written out in full and never descended into,
// so it must not gain a newline when it is reached iteratively
for (std::size_t d = 125; d <= 135; ++d)
{
CAPTURE(d);
const std::string compact = std::string(d, '[') + "[]" + std::string(d, ']');
CHECK(json::parse(compact).dump() == compact);
const std::string with_object = std::string(d, '[') + "{}" + std::string(d, ']');
CHECK(json::parse(with_object).dump() == with_object);
}
}
}
+2 -5
View File
@@ -17,7 +17,6 @@ using nlohmann::json;
#include <sstream> #include <sstream>
#include <iomanip> #include <iomanip>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp"
TEST_CASE("Unicode (1/5)" * doctest::skip()) TEST_CASE("Unicode (1/5)" * doctest::skip())
{ {
@@ -241,8 +240,7 @@ void roundtrip(bool success_expected, const std::string& s)
if (success_expected) if (success_expected)
{ {
// serialization succeeds // serialization succeeds
// dump() is nodiscard; this only checks that dumping does not throw CHECK_NOTHROW(j.dump());
CHECK_NOTHROW(utils::ignore_return_value(j.dump()));
// exclude parse test for U+0000 // exclude parse test for U+0000
if (s[0] != '\0') if (s[0] != '\0')
@@ -261,8 +259,7 @@ void roundtrip(bool success_expected, const std::string& s)
else else
{ {
// serialization fails // serialization fails
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_AS(j.dump(), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump()), json::type_error&);
// parsing JSON text fails // parsing JSON text fails
CHECK_THROWS_AS(_ = json::parse(ps), json::parse_error&); CHECK_THROWS_AS(_ = json::parse(ps), json::parse_error&);
+1 -3
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <iostream> #include <iostream>
#include <iomanip> #include <iomanip>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp"
// this test suite uses static variables with non-trivial destructors // this test suite uses static variables with non-trivial destructors
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
@@ -98,8 +97,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
else else
{ {
// strict mode must throw if success is not expected // strict mode must throw if success is not expected
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_AS(j.dump(), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump()), json::type_error&);
// ignore and replace must create different dumps // ignore and replace must create different dumps
CHECK(s_ignored != s_replaced); CHECK(s_ignored != s_replaced);
+1 -3
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <iostream> #include <iostream>
#include <iomanip> #include <iomanip>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp"
// this test suite uses static variables with non-trivial destructors // this test suite uses static variables with non-trivial destructors
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
@@ -98,8 +97,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
else else
{ {
// strict mode must throw if success is not expected // strict mode must throw if success is not expected
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_AS(j.dump(), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump()), json::type_error&);
// ignore and replace must create different dumps // ignore and replace must create different dumps
CHECK(s_ignored != s_replaced); CHECK(s_ignored != s_replaced);
+1 -3
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <iostream> #include <iostream>
#include <iomanip> #include <iomanip>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp"
// this test suite uses static variables with non-trivial destructors // this test suite uses static variables with non-trivial destructors
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
@@ -98,8 +97,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
else else
{ {
// strict mode must throw if success is not expected // strict mode must throw if success is not expected
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_AS(j.dump(), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump()), json::type_error&);
// ignore and replace must create different dumps // ignore and replace must create different dumps
CHECK(s_ignored != s_replaced); CHECK(s_ignored != s_replaced);
+1 -3
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <iostream> #include <iostream>
#include <iomanip> #include <iomanip>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp"
// this test suite uses static variables with non-trivial destructors // this test suite uses static variables with non-trivial destructors
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
@@ -98,8 +97,7 @@ void check_utf8dump(bool success_expected, int byte1, int byte2 = -1, int byte3
else else
{ {
// strict mode must throw if success is not expected // strict mode must throw if success is not expected
// dump() is nodiscard; the exception is thrown by dump() itself before it would return CHECK_THROWS_AS(j.dump(), json::type_error&);
CHECK_THROWS_AS(utils::ignore_return_value(j.dump()), json::type_error&);
// ignore and replace must create different dumps // ignore and replace must create different dumps
CHECK(s_ignored != s_replaced); CHECK(s_ignored != s_replaced);
-239
View File
@@ -18,12 +18,7 @@
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
using nlohmann::json; using nlohmann::json;
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint8_t
#include <list> #include <list>
#include <string> // string
#include <vector> // vector
#if defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20) #if defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20)
#include <iterator> #include <iterator>
@@ -217,66 +212,6 @@ TEST_CASE("Parse with heterogeneous iterator and sentinel types")
CHECK(j2.at(0) == 1); CHECK(j2.at(0) == 1);
} }
// A type whose data() hands out raw bytes but whose size() counts something
// else - here fixed-size records. Reading [data(), data() + size()) as bytes
// would silently truncate the input, so data() and size() alone must not be
// taken as evidence of contiguous byte storage.
struct record_buffer
{
using value_type = std::array<char, 4>;
std::string bytes;
const char* data() const noexcept
{
return bytes.data();
}
std::size_t size() const noexcept
{
return bytes.size() / sizeof(value_type);
}
const char* begin() const noexcept
{
return bytes.data();
}
const char* end() const noexcept
{
return bytes.data() + bytes.size();
}
};
TEST_CASE("Contiguous byte containers take the pointer adapter")
{
// Containers with contiguous single-byte storage are routed through the
// pointer-based adapter so the bulk fast paths apply in every standard, not
// only in C++20 where the library iterators model std::contiguous_iterator.
CHECK(nlohmann::detail::is_contiguous_byte_container<std::string>::value);
CHECK(nlohmann::detail::is_contiguous_byte_container<std::vector<char>>::value);
CHECK(nlohmann::detail::is_contiguous_byte_container<std::vector<std::uint8_t>>::value);
CHECK(nlohmann::detail::is_contiguous_byte_container<std::array<char, 4>>::value);
// input_adapter() takes its container by forwarding reference, so the trait
// is also asked about reference types
CHECK(nlohmann::detail::is_contiguous_byte_container<std::string&>::value);
CHECK(nlohmann::detail::is_contiguous_byte_container<const std::string&>::value);
// everything else keeps the iterator-based adapter
CHECK_FALSE(nlohmann::detail::is_contiguous_byte_container<std::list<char>>::value);
CHECK_FALSE(nlohmann::detail::is_contiguous_byte_container<std::vector<int>>::value);
CHECK_FALSE(nlohmann::detail::is_contiguous_byte_container<const char*>::value);
// including a type that has data() and size() but whose size() does not
// count the units data() points at: its value_type says so
CHECK_FALSE(nlohmann::detail::is_contiguous_byte_container<record_buffer>::value);
// and such a container still parses through its iterators, in full - taking
// it for a byte container would stop after data() + size() bytes
const record_buffer buffer{"[1,2,3,4,5]"};
CHECK(buffer.data() == buffer.bytes.data());
CHECK(buffer.size() * sizeof(record_buffer::value_type) < buffer.bytes.size());
CHECK(json::parse(buffer) == json({1, 2, 3, 4, 5}));
}
#if defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20) #if defined(__cpp_lib_concepts) && defined(JSON_HAS_CPP_20)
// JSON_HAS_CPP_20 (do not remove; see note at top of file) // JSON_HAS_CPP_20 (do not remove; see note at top of file)
TEST_CASE("Parse with std::counted_iterator and std::default_sentinel_t") TEST_CASE("Parse with std::counted_iterator and std::default_sentinel_t")
@@ -293,180 +228,6 @@ TEST_CASE("Parse with std::counted_iterator and std::default_sentinel_t")
const std::counted_iterator<iterator_type> first2(json_str.begin(), len); const std::counted_iterator<iterator_type> first2(json_str.begin(), len);
CHECK(json::accept(first2, std::default_sentinel)); CHECK(json::accept(first2, std::default_sentinel));
} }
TEST_CASE("std::counted_iterator reaches the contiguous fast paths")
{
// A sized sentinel makes the remaining element count computable in O(1), so
// std::counted_iterator over a contiguous iterator must reach the same bulk
// string/number scanners as a plain pointer - not just the byte-at-a-time
// fallback (see #5268 for the equivalent memcpy fast path).
#if JSON_HAS_RANGES
// JSON_HAS_RANGES is 0 on standard libraries with an incomplete <ranges>
// (libstdc++ < 11, libc++ < 16), where the adapter deliberately falls back
// to the byte-at-a-time scanner; everything below still has to work there.
using adapter_type = nlohmann::detail::iterator_input_adapter<std::counted_iterator<const char*>, std::default_sentinel_t>;
CHECK(adapter_type::supports_bulk_scan);
CHECK(adapter_type::supports_seek);
#endif
// exercise every fast path: long ASCII run, multibyte UTF-8, escapes, and
// integer/floating-point numbers
const std::string json_str =
R"({"ascii":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",)"
"\"utf8\":\"\xe4\xb8\xad\xe6\x96\x87\xf0\x9f\x98\x80\xc3\xa9\","
R"("escaped":"aéb\n\\","ints":[0,-1,18446744073709551615,-9223372036854775808],)"
R"("floats":[1.5,-2.25e3,0.30000000000000004]})";
const auto len = static_cast<std::iter_difference_t<const char*>>(json_str.size());
const std::counted_iterator<const char*> first(json_str.data(), len);
const json j = json::parse(first, std::default_sentinel);
// parsing through the pointer adapter must give exactly the same result
CHECK(j == json::parse(json_str));
#if !defined(JSON_NOEXCEPTION)
// Diagnostics that quote the offending token are reconstructed from the
// already-consumed input (supports_seek), a path a sized sentinel only
// reaches now; check a few that include the "last read" text. Parsing
// invalid input aborts when exceptions are off, hence the guard.
// Raw strings and explicit bytes: an escaped literal and two literals
// written next to each other both read as mistakes to static analysis.
const auto byte = [](int value)
{
return std::string(1, static_cast<char>(value));
};
const std::vector<std::string> diagnostic_docs =
{
"1\nx",
"truX",
"[tru]",
R"("abc)",
R"(["\ud834"])",
R"(["a)" + byte(0x01) + R"(b"])",
R"([")" + byte(0xC3) + byte(0x28) + R"("])",
"[1e]",
R"(["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaX)"
};
for (const auto& text : diagnostic_docs)
{
CAPTURE(text);
const std::counted_iterator<const char*> it(text.data(), static_cast<std::iter_difference_t<const char*>>(text.size()));
std::string counted_message;
std::string string_message;
try
{
const json counted_result = json::parse(it, std::default_sentinel);
static_cast<void>(counted_result);
}
catch (const json::parse_error& e)
{
counted_message = e.what();
}
try
{
const json string_result = json::parse(text);
static_cast<void>(string_result);
}
catch (const json::parse_error& e)
{
string_message = e.what();
}
CHECK_FALSE(counted_message.empty());
CHECK(counted_message == string_message);
}
// and errors must still be reported identically
const std::string bad = "[01\n]";
const std::counted_iterator<const char*> bad_first(bad.data(), static_cast<std::iter_difference_t<const char*>>(bad.size()));
std::string counted_what;
std::string string_what;
try
{
const json counted_result = json::parse(bad_first, std::default_sentinel);
static_cast<void>(counted_result);
}
catch (const json::parse_error& e)
{
counted_what = e.what();
}
try
{
const json string_result = json::parse(bad);
static_cast<void>(string_result);
}
catch (const json::parse_error& e)
{
string_what = e.what();
}
CHECK_FALSE(counted_what.empty());
CHECK(counted_what == string_what);
#endif
}
#if !defined(JSON_NOEXCEPTION)
// several cases below are truncated on purpose, and parsing invalid input
// aborts when exceptions are off
TEST_CASE("std::counted_iterator bulk scanning stops at the counted end")
{
// The count, not the size of the underlying buffer, is the end of the
// input: the bulk scanners must never look at the bytes behind it, even
// though they are readable. Each case is compared against parsing the
// equivalent prefix as a std::string.
const auto via_counted = [](const std::string & buf, std::size_t n) -> std::string
{
const std::counted_iterator<const char*> first(buf.data(), static_cast<std::iter_difference_t<const char*>>(n));
try
{
const json j = json::parse(first, std::default_sentinel);
return "OK|" + j.dump();
}
catch (const json::parse_error& e)
{
return {e.what()};
}
};
const auto via_prefix = [](const std::string & buf, std::size_t n) -> std::string
{
try
{
const json j = json::parse(buf.substr(0, n));
return "OK|" + j.dump();
}
catch (const json::parse_error& e)
{
return {e.what()};
}
};
struct testcase // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init)
{
const char* buffer;
std::size_t count;
};
const std::vector<testcase> cases =
{
{"[\"abc\"]____TRAILING____", 7}, // exact fit, tail hidden
{"[\"abcdefghijklmnop\"]____", 8}, // cut inside a string
{"[\"abc\"]____", 6}, // cut just before the closing quote
{"[12345]xxxxx", 4}, // cut inside a number
{"[123]999999", 5}, // number ends exactly at the count
{"[\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"]", 12}, // closing quote only behind the count
{"[\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"]", 19}, // cut inside an 8-byte SWAR stride
{"[\"\xe4\xb8\xad\xe6\x96\x87\"]", 5}, // cut inside a UTF-8 sequence
{"[\"\xe4\xb8\xad\xe6\x96\x87\"]____", 10}, // complete UTF-8, tail hidden
{"[1.25e3]TRAILINGDIGITS999", 7}, // number token reaches the count
};
for (const auto& tc : cases)
{
CAPTURE(tc.buffer);
CAPTURE(tc.count);
const std::string buffer = tc.buffer;
CHECK(via_counted(buffer, tc.count) == via_prefix(buffer, tc.count));
}
}
#endif
#endif #endif
} // namespace } // namespace