Compare commits

..
Author SHA1 Message Date
Niels Lohmann 90f85d6d75 Merge branch 'develop' into claude/iterative-diff
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 20:58:06 +02:00
Niels Lohmann 4aaeb01ea4 Merge remote-tracking branch 'origin/develop' into claude/iterative-diff
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 17:16:01 +02:00
Niels Lohmann 2038838eea Copy the diff frame's members instead of holding a reference to it
The loop in diff_iteratively held a reference to the top frame, which
enter() invalidates when it pushes and the end of the loop invalidates
when it pops. Nothing used it afterwards, but a later change could. As in
the other iterative walks, the members the loop reads are now copied out
as constants and the ones it advances are changed through stack.back().
The frame as a whole is not copied: it holds the common keys and the
"add" operations of an object.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 14:00:04 +02:00
Niels Lohmann aa5084d713 Keep diff()'s recursive levels small and its result elided
diff_recursively built every patch operation in place from initializer
lists. Unoptimized builds give each of those temporaries its own stack
slot, so every level of the bounded descent cost kilobytes of stack
(about 6 KB with clang -O0), and the 128 recursive levels overflowed the
1 MB stack of MSVC Debug in the "deeply nested values" test. The
operations and the key comparison of two objects are now built by
separate functions, which diff_iteratively shares, and both diff
functions append to one result instead of returning a patch per level
that the caller copies. With clang -O0, diffing values nested 300 levels
deep now peaks at about 190 KB of stack instead of 880 KB.

Since diff() now owns the only returned value, clang's -Wnrvo no longer
reports the returns of diff_recursively, which alternated between the
local patch and diff_iteratively's result.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-26 07:49:42 +02:00
Niels Lohmann 33c4dfdc18 Note that the diff frame reference is invalidated by pop_back() too
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-26 07:46:43 +02:00
Niels Lohmann b41e43fffc Bound diff()'s descent with a depth count instead of scanning the source
Now that operator== no longer recurses (#5390), diff() can keep its per-level
equality shortcut all the way down. It diffs recursively for the first
detail::recursion_depth_limit() levels, as merge_patch() does, and hands
anything deeper to diff_iteratively(). The nesting_exceeds() scan, which
cost about 30% on equal documents, is gone, and diff() is on par with
develop again.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 22:23:57 +02:00
Niels Lohmann 958e0a906b Merge remote-tracking branch 'origin/develop' into claude/iterative-diff 2026-09-25 22:19:57 +02:00
Niels Lohmann 49f038b86a Merge branch 'develop' into claude/iterative-diff
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 20:22:45 +02:00
Niels Lohmann 722c2bb561 Merge branch 'develop' into claude/iterative-diff
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 18:06:10 +02:00
Niels Lohmann f41296276c Mark the diff frame's value-initialized members for clang-tidy
The braces are kept for GCC's -Weffc++, as in json_sax.hpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 18:06:10 +02:00
Niels Lohmann 481b8d17fa Merge branch 'develop' into claude/iterative-diff
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:24:38 +02:00
Niels Lohmann 484f644b86 Diff fewer nesting depths so the test does not time out under Valgrind
Checking every depth up to 300 made test-json_patch exceed the 1500 s ctest
timeout in ci_test_valgrind. Check the depths up to 16, those around the
recursion limit of 128, and 300 instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 08:21:38 +02:00
Niels Lohmann 08e30eca78 Use the shared recursion limit in diff()
diff_depth_limit() is gone in favor of detail::recursion_depth_limit().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 17:12:07 +02:00
Niels Lohmann 582223eb8a Make diff_frame a member struct that declares its special members
GCC's -Weffc++ (an error in CI) asks a class with pointer members, a
user constructor and a non-trivial destructor to declare its copy
constructor and copy assignment; diff_frame's vector and basic_json
members make its destructor non-trivial. Declare all five as defaulted,
which also satisfies clang-tidy's special-member-functions check. Leave
their exception specifications implicit: GCC 4.8 rejects an explicit
one that differs from the implicit one, as it does for flatten_task in
#5517.

The converting constructor cannot throw, and is now declared noexcept
for GCC's -Wnoexcept, which flags the emplace_back() under C++26
otherwise. The struct also moves from diff_iteratively() into the class,
like dump_frame in the serializer.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 17:12:07 +02:00
Niels Lohmann c14208a8e0 Diff deeply nested values without recursing per nesting level
diff() descended into both values once per nesting level, and compared
them with operator== on every level on the way, which recurses as well.
Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted
the call stack and terminated the process, although parse() accepts
them without complaint. On such a chain the per-level comparisons and
path strings also made diff() quadratic in time and memory.

Both the recursion and operator== only descend as far as the source is
nested. So diff() first checks, recursing at most diff_depth_limit()
(128) levels, whether the source is nested more deeply than that. If not
- all but a vanishing minority of values - the recursive algorithm
diffs it exactly as before, now as diff_recursively(). Otherwise
diff_iteratively() walks the two values on an explicit stack, emitting
the same operations in the same order. It does not compare arrays and
objects with operator== up front (equal ones yield no operations
anyway), keeps the path in one buffer instead of a new string per
level, and hands every subtree that is not nested too deeply back to
diff_recursively(), so equal parts are still skipped quickly.

The check costs one pass over the source. On a 3,000-object document
that is about 30% of diffing two equal values (which is just an
operator== call), about 10% of diffing values that differ in a few
places, and noise when arrays change length. Once operator== no longer
recurses (#5390), the check can go.

Tests check that the patch reproduces the target at every depth up to
300, for json and ordered_json, including reordered members. They also
check the exact operation for a difference deep inside, and diff values
nested 100,000 levels deep.

Fixes #5393 for diff().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 17:12:07 +02:00
37 changed files with 1225 additions and 1839 deletions
-1
View File
@@ -67,7 +67,6 @@ jobs:
python3 $TOOL_DIR/amalgamate.py -c $TOOL_DIR/config_json.json -s .
python3 $TOOL_DIR/amalgamate.py -c $TOOL_DIR/config_json_fwd.json -s .
cp include/nlohmann/json_literals.hpp $INCLUDE_DIR/json_literals.hpp
# the header list of the Bazel "json" target must match the files in include/
cmake -P cmake/scripts/gen_bazel_build_file.cmake
-1
View File
@@ -65,7 +65,6 @@ cc_library(
"include/nlohmann/detail/value_t.hpp",
"include/nlohmann/json.hpp",
"include/nlohmann/json_fwd.hpp",
"include/nlohmann/json_literals.hpp",
"include/nlohmann/ordered_map.hpp",
"include/nlohmann/thirdparty/hedley/hedley.hpp",
"include/nlohmann/thirdparty/hedley/hedley_undef.hpp",
+5 -16
View File
@@ -21,8 +21,6 @@ TESTS_SRCS=$(shell find tests -type f \( -name '*.hpp' -o -name '*.cpp' -o -name
# the single headers (amalgamated from the source files)
AMALGAMATED_FILE=single_include/nlohmann/json.hpp
AMALGAMATED_FWD_FILE=single_include/nlohmann/json_fwd.hpp
# json_literals.hpp only includes <nlohmann/json.hpp>, so it is copied verbatim
AMALGAMATED_LITERALS_FILE=single_include/nlohmann/json_literals.hpp
##########################################################################
@@ -31,7 +29,7 @@ AMALGAMATED_LITERALS_FILE=single_include/nlohmann/json_literals.hpp
# main target
all:
@echo "amalgamate - amalgamate files single_include/nlohmann/json{,_fwd,_literals}.hpp from the include/nlohmann sources"
@echo "amalgamate - amalgamate files single_include/nlohmann/json{,_fwd}.hpp from the include/nlohmann sources"
@echo "BUILD.bazel - regenerate the Bazel BUILD file from the include/nlohmann sources"
@echo "ChangeLog.md - generate ChangeLog file"
@echo "check-amalgamation - check whether sources have been amalgamated and BUILD.bazel is up to date"
@@ -156,14 +154,14 @@ install_astyle:
# call the Artistic Style pretty printer on all source files
pretty: install_astyle
$(ASTYLE) --project=tools/astyle/.astylerc $(SRCS) $(TESTS_SRCS) $(AMALGAMATED_FILE) $(AMALGAMATED_FWD_FILE) $(AMALGAMATED_LITERALS_FILE) docs/mkdocs/docs/examples/*.cpp
$(ASTYLE) --project=tools/astyle/.astylerc $(SRCS) $(TESTS_SRCS) $(AMALGAMATED_FILE) $(AMALGAMATED_FWD_FILE) docs/mkdocs/docs/examples/*.cpp
# call the Clang-Format on all source files
pretty_format:
for FILE in $(SRCS) $(TESTS_SRCS) $(AMALGAMATED_FILE) docs/mkdocs/docs/examples/*.cpp; do echo $$FILE; clang-format -i $$FILE; done
# create single header files and pretty print
amalgamate: $(AMALGAMATED_FILE) $(AMALGAMATED_FWD_FILE) $(AMALGAMATED_LITERALS_FILE)
amalgamate: $(AMALGAMATED_FILE) $(AMALGAMATED_FWD_FILE)
$(MAKE) pretty
# call the amalgamation tool for json.hpp
@@ -174,23 +172,16 @@ $(AMALGAMATED_FILE): $(SRCS)
$(AMALGAMATED_FWD_FILE): $(SRCS)
tools/amalgamate/amalgamate.py -c tools/amalgamate/config_json_fwd.json -s . --verbose=yes
# copy json_literals.hpp
$(AMALGAMATED_LITERALS_FILE): include/nlohmann/json_literals.hpp
cp include/nlohmann/json_literals.hpp $(AMALGAMATED_LITERALS_FILE)
# check if file single_include/nlohmann/json.hpp has been amalgamated from the nlohmann sources
# Note: this target is called by Travis
check-amalgamation:
@mv $(AMALGAMATED_FILE) $(AMALGAMATED_FILE)~
@mv $(AMALGAMATED_FWD_FILE) $(AMALGAMATED_FWD_FILE)~
@mv $(AMALGAMATED_LITERALS_FILE) $(AMALGAMATED_LITERALS_FILE)~
@$(MAKE) amalgamate
@diff $(AMALGAMATED_FILE) $(AMALGAMATED_FILE)~ || (echo "===================================================================\n Amalgamation required! Please read the contribution guidelines\n in file .github/CONTRIBUTING.md.\n===================================================================" ; mv $(AMALGAMATED_FILE)~ $(AMALGAMATED_FILE) ; false)
@diff $(AMALGAMATED_FWD_FILE) $(AMALGAMATED_FWD_FILE)~ || (echo "===================================================================\n Amalgamation required! Please read the contribution guidelines\n in file .github/CONTRIBUTING.md.\n===================================================================" ; mv $(AMALGAMATED_FWD_FILE)~ $(AMALGAMATED_FWD_FILE) ; false)
@diff $(AMALGAMATED_LITERALS_FILE) $(AMALGAMATED_LITERALS_FILE)~ || (echo "===================================================================\n Amalgamation required! Please read the contribution guidelines\n in file .github/CONTRIBUTING.md.\n===================================================================" ; mv $(AMALGAMATED_LITERALS_FILE)~ $(AMALGAMATED_LITERALS_FILE) ; false)
@mv $(AMALGAMATED_FILE)~ $(AMALGAMATED_FILE)
@mv $(AMALGAMATED_FWD_FILE)~ $(AMALGAMATED_FWD_FILE)
@mv $(AMALGAMATED_LITERALS_FILE)~ $(AMALGAMATED_LITERALS_FILE)
@mv BUILD.bazel BUILD.bazel~
@$(MAKE) BUILD.bazel
@diff BUILD.bazel BUILD.bazel~ || (echo "===================================================================\n BUILD.bazel is out of date! Please run 'make BUILD.bazel'.\n===================================================================" ; mv BUILD.bazel~ BUILD.bazel ; false)
@@ -231,7 +222,7 @@ json.tar.xz:
# We use `-X` to make the resulting ZIP file reproducible, see
# <https://content.pivotal.io/blog/barriers-to-deterministic-reproducible-zip-files>.
include.zip: BUILD.bazel
zip -9 --recurse-paths -X include.zip $(SRCS) $(AMALGAMATED_FILE) $(AMALGAMATED_FWD_FILE) $(AMALGAMATED_LITERALS_FILE) BUILD.bazel MODULE.bazel meson.build LICENSE.MIT
zip -9 --recurse-paths -X include.zip $(SRCS) $(AMALGAMATED_FILE) $(AMALGAMATED_FWD_FILE) BUILD.bazel MODULE.bazel meson.build LICENSE.MIT
# Create the files for a release and add signatures and hashes.
release: include.zip json.tar.xz
@@ -240,12 +231,10 @@ release: include.zip json.tar.xz
gpg --armor --detach-sig include.zip
gpg --armor --detach-sig $(AMALGAMATED_FILE)
gpg --armor --detach-sig $(AMALGAMATED_FWD_FILE)
gpg --armor --detach-sig $(AMALGAMATED_LITERALS_FILE)
gpg --armor --detach-sig json.tar.xz
cp $(AMALGAMATED_FILE) release_files
cp $(AMALGAMATED_FWD_FILE) release_files
cp $(AMALGAMATED_LITERALS_FILE) release_files
mv $(AMALGAMATED_FILE).asc $(AMALGAMATED_FWD_FILE).asc $(AMALGAMATED_LITERALS_FILE).asc json.tar.xz json.tar.xz.asc include.zip include.zip.asc release_files
mv $(AMALGAMATED_FILE).asc $(AMALGAMATED_FWD_FILE).asc json.tar.xz json.tar.xz.asc include.zip include.zip.asc release_files
cd release_files ; shasum -a 256 json.hpp include.zip json.tar.xz > hashes.txt
+1 -4
View File
@@ -373,10 +373,9 @@ file(GLOB_RECURSE INDENT_FILES
set(include_dir ${PROJECT_SOURCE_DIR}/single_include/nlohmann)
set(tool_dir ${PROJECT_SOURCE_DIR}/tools/amalgamate)
add_custom_target(ci_test_amalgamation
COMMAND rm -fr ${include_dir}/json.hpp~ ${include_dir}/json_fwd.hpp~ ${include_dir}/json_literals.hpp~
COMMAND rm -fr ${include_dir}/json.hpp~ ${include_dir}/json_fwd.hpp~
COMMAND cp ${include_dir}/json.hpp ${include_dir}/json.hpp~
COMMAND cp ${include_dir}/json_fwd.hpp ${include_dir}/json_fwd.hpp~
COMMAND cp ${include_dir}/json_literals.hpp ${include_dir}/json_literals.hpp~
COMMAND ${Python3_EXECUTABLE} -mvenv venv_astyle
COMMAND venv_astyle/bin/pip3 --quiet install -r ${CMAKE_SOURCE_DIR}/tools/astyle/requirements.txt
@@ -384,12 +383,10 @@ add_custom_target(ci_test_amalgamation
COMMAND ${Python3_EXECUTABLE} ${tool_dir}/amalgamate.py -c ${tool_dir}/config_json.json -s .
COMMAND ${Python3_EXECUTABLE} ${tool_dir}/amalgamate.py -c ${tool_dir}/config_json_fwd.json -s .
COMMAND cp ${PROJECT_SOURCE_DIR}/include/nlohmann/json_literals.hpp ${include_dir}/json_literals.hpp
COMMAND venv_astyle/bin/astyle --project=tools/astyle/.astylerc --suffix=none ${include_dir}/json.hpp ${include_dir}/json_fwd.hpp
COMMAND diff ${include_dir}/json.hpp~ ${include_dir}/json.hpp
COMMAND diff ${include_dir}/json_fwd.hpp~ ${include_dir}/json_fwd.hpp
COMMAND diff ${include_dir}/json_literals.hpp~ ${include_dir}/json_literals.hpp
COMMAND venv_astyle/bin/astyle --project=tools/astyle/.astylerc --suffix=orig ${INDENT_FILES}
COMMAND for FILE in `find . -name '*.orig'`\; do false \; done
+2 -2
View File
@@ -80,8 +80,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
the end of the file was not reached when `strict` was set to true
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from CBOR were
used in the given input or if the input is not valid CBOR
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other
types are not supported, as JSON object keys are always strings) or a string is malformed
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key,
but not found
## Complexity
@@ -73,8 +73,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
the end of the file was not reached when `strict` was set to true
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from
MessagePack were used in the given input or if the input is not valid MessagePack
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other
types are not supported, as JSON object keys are always strings) or a string is malformed
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key,
but not found
## Complexity
-1
View File
@@ -28,7 +28,6 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_HAS_RANGES**](json_has_ranges.md) - control `std::ranges` support
- [**JSON_HAS_STD_FORMAT**](json_has_std_format.md) - control `std::format`/`std::formatter` support
- [**JSON_HAS_THREE_WAY_COMPARISON**](json_has_three_way_comparison.md) - control 3-way comparison support
- [**JSON_NO_AUTOMATIC_UDLS**](json_no_automatic_udls.md) - do not include the user-defined string literals (UDLs) automatically
- [**JSON_NO_IO**](json_no_io.md) - switch off functions relying on certain C++ I/O headers
- [**JSON_NO_THREAD_LOCAL**](json_no_thread_local.md) - switch off the use of `thread_local` storage
- [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
@@ -1,69 +0,0 @@
# JSON_NO_AUTOMATIC_UDLS
```cpp
#define JSON_NO_AUTOMATIC_UDLS
```
When defined, `<nlohmann/json.hpp>` does not include `<nlohmann/json_literals.hpp>`, so the user-defined string
literals [`operator""_json`](../operator_literal_json.md) and
[`operator""_json_pointer`](../operator_literal_json_pointer.md) are not declared. Include
`<nlohmann/json_literals.hpp>` in the files that use them.
The literals are ordinary inline functions whose bodies call the parser, so every translation unit that includes them
instantiates the parser — even if it never parses anything itself. Defining `JSON_NO_AUTOMATIC_UDLS` for a whole project
avoids this cost in translation units that do not parse (e.g., ones that only define types and conversions or pass
`json` values around) and reduces their compile time.
## Default definition
By default, `#!cpp JSON_NO_AUTOMATIC_UDLS` is not defined, and `<nlohmann/json.hpp>` includes
`<nlohmann/json_literals.hpp>`.
```cpp
#undef JSON_NO_AUTOMATIC_UDLS
```
## Notes
!!! info "Header `<nlohmann/json_literals.hpp>`"
The header includes `<nlohmann/json.hpp>` itself and places the literals according to
[`JSON_USE_GLOBAL_UDLS`](json_use_global_udls.md). It is part of the multi-header sources (`include/nlohmann`)
and of the single-header sources (`single_include/nlohmann`), next to `json.hpp`.
!!! info "C++ modules"
The `nlohmann.json` [module](../../features/modules.md) always exports the literals, regardless of this macro.
## Examples
??? example
The code below includes the library without the literals and adds them in a single translation unit.
```cpp
// compiled with -DJSON_NO_AUTOMATIC_UDLS for the whole project
#include <nlohmann/json.hpp>
// this file uses the literals, so it includes them explicitly
#include <nlohmann/json_literals.hpp>
int main()
{
auto j = R"({"foo": 42})"_json;
return j.at("/foo"_json_pointer) == 42 ? 0 : 1;
}
```
Without the include of `<nlohmann/json_literals.hpp>`, the code would fail to compile.
## See also
- [`operator""_json`](../operator_literal_json.md)
- [`operator""_json_pointer`](../operator_literal_json_pointer.md)
- [`JSON_USE_GLOBAL_UDLS`](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [Compile times](../../integration/compile_times.md) - options to reduce compile times
## Version history
- Added in version 3.13.0.
@@ -15,7 +15,7 @@ The default value is `1`.
#define JSON_USE_GLOBAL_UDLS 1
```
When the macro is not defined, the library behaves as if it were defined to its default value.
When the macro is not defined, the library will define it to its default value.
## Notes
@@ -32,11 +32,6 @@ When the macro is not defined, the library behaves as if it were defined to its
[`JSON_GlobalUDLs`](../../integration/cmake.md#json_globaludls) (`ON` by default) which defines
`JSON_USE_GLOBAL_UDLS` accordingly.
!!! info "Leaving out the literals"
If [`JSON_NO_AUTOMATIC_UDLS`](json_no_automatic_udls.md) is defined, the literals are only declared where
`<nlohmann/json_literals.hpp>` is included; this macro then applies to that header.
## Examples
??? example "Example 1: Default behavior"
@@ -97,7 +92,6 @@ When the macro is not defined, the library behaves as if it were defined to its
- [`operator""_json`](../operator_literal_json.md)
- [`operator""_json_pointer`](../operator_literal_json_pointer.md)
- [`JSON_NO_AUTOMATIC_UDLS`](json_no_automatic_udls.md) - do not include the user-defined string literals automatically
- [:simple-cmake: JSON_GlobalUDLs](../../integration/cmake.md#json_globaludls) - CMake option to control the macro
## Version history
@@ -18,9 +18,7 @@ using namespace nlohmann;
```
This is suggested to ease migration to the next major version release of the library. See
[`JSON_USE_GLOBAL_UDLS`](macros/json_use_global_udls.md#notes) for details. The operator is declared in header
`<nlohmann/json_literals.hpp>`, which `<nlohmann/json.hpp>` includes unless
[`JSON_NO_AUTOMATIC_UDLS`](macros/json_no_automatic_udls.md) is defined.
[`JSON_USE_GLOBAL_UDLS`](macros/json_use_global_udls.md#notes) for details.
## Parameters
@@ -61,8 +59,6 @@ Linear.
## See also
- [Creating JSON values](../features/creating_values.md) - the article on creating JSON values
- [JSON_NO_AUTOMATIC_UDLS](macros/json_no_automatic_udls.md) - do not include the user-defined string literals
automatically
## Version history
@@ -17,9 +17,7 @@ using namespace nlohmann::literals::json_literals;
using namespace nlohmann;
```
This is suggested to ease migration to the next major version release of the library. See
[`JSON_USE_GLOBAL_UDLS`](macros/json_use_global_udls.md#notes) for details. The operator is declared in header
`<nlohmann/json_literals.hpp>`, which `<nlohmann/json.hpp>` includes unless
[`JSON_NO_AUTOMATIC_UDLS`](macros/json_no_automatic_udls.md) is defined.
[`JSON_USE_GLOBAL_UDLS`](macros/json_use_global_udls.md#notes) for details.
## Parameters
@@ -60,8 +58,6 @@ Linear.
## See also
- [json_pointer](json_pointer/index.md) - type to represent JSON Pointers
- [JSON_NO_AUTOMATIC_UDLS](macros/json_no_automatic_udls.md) - do not include the user-defined string literals
automatically
## Version history
@@ -174,20 +174,7 @@ The library maps CBOR types to JSON value types as follows:
!!! warning "Object keys"
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps
with keys other than text strings (major type 3) are rejected with a
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set
to `false`, a discarded value) naming the type of the key that was found, for instance:
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
```
This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR
maps with integer keys, such as COSE ([RFC 9052](https://www.rfc-editor.org/rfc/rfc9052.html)) or CWT
([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a
general-purpose CBOR library instead.
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected.
!!! warning "UTF-8 validation of text strings"
@@ -138,21 +138,6 @@ The library maps MessagePack types to JSON value types as follows:
Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
!!! warning "Object keys"
MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the
JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the
MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are
rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
`allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance:
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
```
This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
on. Such input needs a general-purpose MessagePack library instead.
!!! warning "UTF-8 validation of string values"
The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8.
-9
View File
@@ -83,15 +83,6 @@ When defined, default parse and serialize functions for enums are excluded and h
See [full documentation of `JSON_DISABLE_ENUM_SERIALIZATION`](../api/macros/json_disable_enum_serialization.md).
## `JSON_NO_AUTOMATIC_UDLS`
When defined, `<nlohmann/json.hpp>` does not include `<nlohmann/json_literals.hpp>` with the user-defined string literals
`operator""_json` and `operator""_json_pointer`. This reduces the compile time of translation units that do not use
them, because the literals instantiate the parser in every translation unit that includes them. Include
`<nlohmann/json_literals.hpp>` where the literals are needed.
See [full documentation of `JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md).
## `JSON_NO_IO`
When defined, headers `<cstdio>`, `<ios>`, `<iosfwd>`, `<istream>`, and `<ostream>` are not included and parse functions
-3
View File
@@ -40,9 +40,6 @@ Only the following symbols are exported from `nlohmann.json`:
- `nlohmann::literals::json_literals::operator""_json`
- `nlohmann::literals::json_literals::operator""_json_pointer`
The module always exports the two user-defined string literals, even if
[`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) is defined when building it.
Additionally, the following `nlohmann::detail` symbols are exported, solely to work around an MSVC compilation issue
([#3970](https://github.com/nlohmann/json/issues/3970)). They are implementation details, not part of the public API,
and should not be used directly:
+2 -9
View File
@@ -343,20 +343,13 @@ A string could not be read from a [binary format](../features/binary_formats/ind
string was read where one was required (for instance as a map key), the string's length specification is invalid, or
the string's bytes are not valid UTF-8.
CBOR and MessagePack allow map keys of any type, but JSON object keys are always strings. Maps with keys of any other
type (for instance integers or `null`) are therefore not supported; see the notes on
[CBOR](../features/binary_formats/cbor.md) and [MessagePack](../features/binary_formats/messagepack.md).
!!! failure "Example messages"
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF
```
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
```
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF
```
```
[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82
@@ -1,149 +0,0 @@
# Compile times
The library is header-only and makes heavy use of templates, so every translation unit that includes
`<nlohmann/json.hpp>` pays for parsing the header and instantiating what it uses. This page lists the options to reduce
that cost, ordered by how much they typically save.
!!! info "Measurements"
The numbers below are medians of nine runs compiling a single translation unit with `-std=c++17 -c` against the
single-header version, with Apple clang and GCC 16 on macOS (Apple silicon). They show the order of magnitude to
expect; measure your own code before and after a change.
## Include `json_fwd.hpp` in headers
Header files that only need to *name* the `json` type — for function declarations, members held by pointer or
reference, or friend declarations — can include `<nlohmann/json_fwd.hpp>` instead of `<nlohmann/json.hpp>`. It only
forward-declares `basic_json`, `json`, `ordered_json`, `json_pointer`, and `adl_serializer`. The translation units that
actually use the values then include `<nlohmann/json.hpp>`.
```cpp title="person.hpp"
#pragma once
#include <nlohmann/json_fwd.hpp>
struct person;
void to_json(nlohmann::json& j, const person& p);
void from_json(const nlohmann::json& j, person& p);
```
```cpp title="person.cpp"
#include "person.hpp"
#include <nlohmann/json.hpp>
void to_json(nlohmann::json& j, const person& p) { /* ... */ }
void from_json(const nlohmann::json& j, person& p) { /* ... */ }
```
| Compiler | `json.hpp` (`-O0`) | `json_fwd.hpp` (`-O0`) | Change |
|-------------|-------------------:|-----------------------:|-------:|
| Apple clang | 704 ms | 329 ms | −53% |
| GCC 16 | 779 ms | 242 ms | −69% |
This is the most effective option, because it avoids the full header in every translation unit that includes
*your* headers.
## Opt out of the automatic user-defined string literals
The user-defined string literals [`operator""_json`](../api/operator_literal_json.md) and
[`operator""_json_pointer`](../api/operator_literal_json_pointer.md) are ordinary inline functions whose bodies call the
parser. As `<nlohmann/json.hpp>` includes them by default, every translation unit instantiates the parser, even if it
never parses anything itself.
Define [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) for the whole project and include
`<nlohmann/json_literals.hpp>` only in the files that use the literals:
```cmake
target_compile_definitions(my_target PRIVATE JSON_NO_AUTOMATIC_UDLS)
```
```cpp
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp> // only where "..."_json is used
```
The saving applies to translation units that do not parse JSON, for example ones that define types and their
conversions or only pass `json` values around:
| Compiler | Translation unit | Default (`-O0` / `-O2`) | `JSON_NO_AUTOMATIC_UDLS` (`-O0` / `-O2`) | Change |
|-------------|------------------|------------------------:|-----------------------------------------:|------------:|
| Apple clang | model | 776 ms / 846 ms | 629 ms / 692 ms | −19% / −18% |
| GCC 16 | model | 1022 ms / 1120 ms | 882 ms / 965 ms | −14% / −14% |
| Apple clang | parsing | 992 ms / 1815 ms | 1006 ms / 1823 ms | +1% / 0% |
| GCC 16 | parsing | 2018 ms / 3420 ms | 1990 ms / 3454 ms | −1% / +1% |
Translation units that include only the header save up to a third. Translation units that parse anyway instantiate
the parser regardless and see no difference.
## Instantiate `basic_json` once
Each translation unit instantiates the member functions of `nlohmann::json` it uses. An explicit instantiation
declaration tells the compiler that the non-template members are instantiated elsewhere, so it can skip them:
```cpp title="json_instance.hpp"
#pragma once
#include <nlohmann/json.hpp>
extern template class nlohmann::basic_json<>;
```
```cpp title="json_instance.cpp"
#include "json_instance.hpp"
template class nlohmann::basic_json<>;
```
Include `json_instance.hpp` instead of `<nlohmann/json.hpp>` and compile and link `json_instance.cpp` once.
| Compiler | Translation unit | Default (`-O0` / `-O2`) | `extern template` (`-O0` / `-O2`) | Change |
|-------------|---------------------|------------------------:|----------------------------------:|------------:|
| Apple clang | parsing | 992 ms / 1815 ms | 953 ms / 1625 ms | −4% / −10% |
| GCC 16 | parsing | 2018 ms / 3420 ms | 1522 ms / 2728 ms | −25% / −20% |
| Apple clang | `json_instance.cpp` | — | 2166 ms / 4660 ms | — |
| GCC 16 | `json_instance.cpp` | — | 5085 ms / 10616 ms | — |
Notes:
- The saving grows with the number of translation units that use `json`, while the instantiation translation unit is
compiled only once (and is rarely recompiled, as it does not depend on your code).
- Member function templates (such as `get<T>()`, `parse(InputType&&)`, or `value(key, default)`) are not covered by
the explicit instantiation and are still instantiated where they are used.
- The declaration covers exactly `nlohmann::json`. Add the same lines for `nlohmann::ordered_json`
(`nlohmann::basic_json<nlohmann::ordered_map>`) or your own `basic_json` specializations if you use them.
## Use C++20 modules
With a toolchain that supports named modules, `import nlohmann.json;` compiles the library once into a module and
avoids parsing the header in every translation unit. See [Modules](../features/modules.md) for requirements and known
issues. Module support is experimental and currently depends heavily on the compiler version.
## Use precompiled headers
Build systems can precompile `<nlohmann/json.hpp>` together with other stable headers, for example with CMake's
[`target_precompile_headers`](https://cmake.org/cmake/help/latest/command/target_precompile_headers.html):
```cmake
target_precompile_headers(my_target PRIVATE <nlohmann/json.hpp>)
```
This removes the cost of parsing the header, but not of instantiating templates in each translation unit, so it
combines well with the options above.
## Options without effect on compile times
Some configuration macros change what the library declares, but do not measurably change compile times:
| Macro | Apple clang, model (`-O0` / `-O2`) | GCC 16, model (`-O0` / `-O2`) |
|------------------------------------------------------------------------|-----------------------------------:|------------------------------:|
| default | 776 ms / 846 ms | 1022 ms / 1120 ms |
| [`JSON_NO_IO`](../api/macros/json_no_io.md) | 764 ms / 836 ms | 1022 ms / 1117 ms |
| [`JSON_USE_GLOBAL_UDLS`](../api/macros/json_use_global_udls.md)`=0` | 763 ms / 852 ms | 1019 ms / 1106 ms |
`JSON_USE_GLOBAL_UDLS` only controls *where* the literals are declared; to avoid their cost, use
`JSON_NO_AUTOMATIC_UDLS` instead.
## See also
- [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) - do not include the user-defined string
literals automatically
- [Modules](../features/modules.md) - C++20 module support
- [Header only](index.md) - including the library
+1 -4
View File
@@ -15,7 +15,4 @@ Clang).
You can further use file
[`single_include/nlohmann/json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_fwd.hpp)
for forward declarations (see [Compile times](compile_times.md)), and file
[`single_include/nlohmann/json_literals.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_literals.hpp)
for the user-defined string literals if you define
[`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md).
for forward declarations.
-2
View File
@@ -106,7 +106,6 @@ nav:
- integration/cmake.md
- integration/package_managers.md
- integration/pkg-config.md
- integration/compile_times.md
- API Documentation:
- basic_json:
- 'Overview': api/basic_json/index.md
@@ -294,7 +293,6 @@ nav:
- 'JSON_HAS_STATIC_RTTI': api/macros/json_has_static_rtti.md
- 'JSON_HAS_STD_FORMAT': api/macros/json_has_std_format.md
- 'JSON_HAS_THREE_WAY_COMPARISON': api/macros/json_has_three_way_comparison.md
- 'JSON_NO_AUTOMATIC_UDLS': api/macros/json_no_automatic_udls.md
- 'JSON_NOEXCEPTION': api/macros/json_noexception.md
- 'JSON_NO_IO': api/macros/json_no_io.md
- 'JSON_NO_THREAD_LOCAL': api/macros/json_no_thread_local.md
+2 -168
View File
@@ -1324,80 +1324,6 @@ class binary_reader
}
}
/*!
@brief reads a CBOR object key
RFC 8949 allows any data item as a map key, but only strings have a
counterpart in JSON. A key of any other type is rejected with a message
naming that type, rather than the one @ref get_cbor_string gives for a
malformed string.
@param[out] result created key
@return whether key creation completed
*/
bool get_cbor_object_key(string_t& result)
{
// EOF and major type 3 (text string) are left to get_cbor_string
if (current == char_traits<char_type>::eof() || (static_cast<unsigned int>(current) & 0xE0u) == 0x60u)
{
return get_cbor_string(result);
}
const char* found = nullptr;
switch (static_cast<unsigned int>(current) >> 5u)
{
case 0:
found = "an unsigned integer";
break;
case 1:
found = "a negative integer";
break;
case 2:
found = "a byte string";
break;
case 4:
found = "an array";
break;
case 5:
found = "a map";
break;
case 6:
found = "a tag";
break;
default: // major type 7
switch (current)
{
case 0xF4:
case 0xF5:
found = "a boolean";
break;
case 0xF6:
found = "null";
break;
case 0xF7:
found = "undefined";
break;
case 0xF9:
case 0xFA:
case 0xFB:
found = "a floating-point number";
break;
case 0xFF:
found = "a break stop code";
break;
default:
found = "a simple value";
break;
}
break;
}
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
}
/*!
@brief reads a definite-length CBOR byte array
@@ -1642,7 +1568,7 @@ class binary_reader
if (top.is_object)
{
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key)))
if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key)))
{
return false;
}
@@ -2143,98 +2069,6 @@ class binary_reader
}
}
/*!
@brief reads a MessagePack object key
The MessagePack specification allows any type as a map key, but only
strings have a counterpart in JSON. A key of any other type is rejected
with a message naming that type, rather than the one @ref
get_msgpack_string gives for a malformed string.
@param[out] result created key
@return whether key creation completed
*/
bool get_msgpack_object_key(string_t& result)
{
const char* found = nullptr;
switch (current)
{
case 0xC0:
found = "nil";
break;
case 0xC2:
case 0xC3:
found = "a boolean";
break;
case 0xCA:
case 0xCB:
found = "a float";
break;
case 0xC4:
case 0xC5:
case 0xC6:
found = "a bin";
break;
case 0xC7:
case 0xC8:
case 0xC9:
case 0xD4:
case 0xD5:
case 0xD6:
case 0xD7:
case 0xD8:
found = "an ext";
break;
case 0xCC:
case 0xCD:
case 0xCE:
case 0xCF:
case 0xD0:
case 0xD1:
case 0xD2:
case 0xD3:
found = "an integer";
break;
case 0xDC:
case 0xDD:
found = "an array";
break;
case 0xDE:
case 0xDF:
found = "a map";
break;
default:
// fixint, fixmap, and fixarray; strings, EOF, and the unused
// byte 0xC1 are left to get_msgpack_string
if (current == char_traits<char_type>::eof())
{
return get_msgpack_string(result);
}
if (current <= 0x7F || current >= 0xE0)
{
found = "an integer";
}
else if (current <= 0x8F)
{
found = "a map";
}
else if (current <= 0x9F)
{
found = "an array";
}
else
{
return get_msgpack_string(result);
}
break;
}
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
}
/*!
@brief reads a MessagePack byte array
@@ -2397,7 +2231,7 @@ class binary_reader
{
get();
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key)))
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key)))
{
return false;
}
+39 -76
View File
@@ -206,6 +206,7 @@ class lexer : public lexer_base<BasicJsonType>
explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
: ia(std::move(adapter))
, ignore_comments(ignore_comments_)
, decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
, discard_number_values(discard_number_values_)
{}
@@ -221,7 +222,8 @@ class lexer : public lexer_base<BasicJsonType>
// locales
/////////////////////
/// return the decimal point of the current locale
/// return the locale-dependent decimal point
JSON_HEDLEY_PURE
static char get_decimal_point() noexcept
{
const auto* loc = localeconv();
@@ -1090,10 +1092,9 @@ class lexer : public lexer_base<BasicJsonType>
token_type::value_float if number could be successfully scanned,
token_type::parse_error otherwise
@note The scanner is independent of the current locale: token_buffer
always holds `.`. Only the std::strtod fallback of convert_number()
depends on the locale, and it looks up the decimal point right
before converting (see convert_float_locale_aware()).
@note The scanner is independent of the current locale. Internally, the
locale's decimal point is used instead of `.` to work with the
locale-dependent converters.
*/
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
{
@@ -1182,7 +1183,7 @@ scan_number_zero:
{
case '.':
{
add(current);
add(decimal_point_char);
decimal_point_position = token_buffer.size() - 1;
goto scan_number_decimal1;
}
@@ -1219,7 +1220,7 @@ scan_number_any1:
case '.':
{
add(current);
add(decimal_point_char);
decimal_point_position = token_buffer.size() - 1;
goto scan_number_decimal1;
}
@@ -1461,9 +1462,9 @@ scan_number_done:
// Only a number below 1 can carry further insignificant zeros, and only
// while the count stays at the limit does removing them change the
// answer - so this loop is skipped for all but a few tokens. The
// fraction is located through decimal_point_position rather than by
// searching '.'.
// answer - so this loop is skipped for all but a few tokens. Note
// token_buffer holds the locale's decimal point, so the fraction is
// located through decimal_point_position rather than by searching '.'.
if (lead_zero != 0)
{
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
@@ -1481,8 +1482,8 @@ scan_number_done:
@brief convert the number text in token_buffer to its value and token type
The digit sequence in token_buffer has already been validated (by the
scan_number() state machine or by the contiguous fast path) and holds '.'
as decimal point, independent of the locale. Integers are parsed first and fall
scan_number() state machine or by the contiguous fast path) and holds the
locale decimal point in place of '.'. Integers are parsed first and fall
back to floating point on overflow. This is shared so both scanners produce
identical results.
@@ -1562,7 +1563,7 @@ scan_number_done:
// integer conversion above overflowed. Prefer std::from_chars
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
// otherwise the exact Clinger fast path (double only); otherwise the
// locale-aware strtof/strtod/strtold.
// locale-aware strtof/strtod.
if (parse_float_from_chars(num_begin, num_end, value_float))
{
return token_type::value_float;
@@ -1571,75 +1572,26 @@ scan_number_done:
// extra pass over the token's bytes, which otherwise shows up on
// high-precision inputs such as canada.json
if (mantissa_fits_clinger(mantissa_end)
&& parse_float_fast(num_begin, num_end, value_float))
&& parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
{
return token_type::value_float;
}
convert_float_locale_aware();
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
strtof(value_float, token_buffer.data(), &endptr);
// we checked the number format before
JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
return token_type::value_float;
}
/*!
@brief convert the float in token_buffer with strtof/strtod/strtold
These functions expect the decimal point of the *current* locale, so it is
looked up right before the conversion instead of once when the lexer is
constructed: a locale change in between (by a parser callback, a SAX
handler, or another thread) must not truncate the value (#5198). The
token has been validated before, so if the conversion stops early and the
decimal point changed in the meantime, the locale changed between the
lookup and the call, and the conversion is repeated with the new decimal
point. If the decimal point did not change, a retry cannot succeed: the
locale's decimal point is not a single character (e.g., the two-byte
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
The value strtod parsed up to that point is kept, as before this change.
Note that changing the locale in another thread *while* strtod runs is
undefined behavior of the C library, which this function cannot prevent.
*/
void convert_float_locale_aware()
{
const bool has_dot = decimal_point_position != std::string::npos;
char decimal_point = get_decimal_point();
for (;;)
{
const bool substitute = has_dot && decimal_point != '.';
if (substitute)
{
token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
}
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
strtof(value_float, token_buffer.data(), &endptr);
if (substitute)
{
// get_string() hands the token to the SAX interface with '.'
token_buffer[decimal_point_position] = '.';
}
if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
{
return;
}
// retry only if the locale changed; otherwise, this would loop forever
const char current_decimal_point = get_decimal_point();
if (current_decimal_point == decimal_point)
{
return;
}
decimal_point = current_decimal_point;
}
}
/*!
@brief contiguous fast path for scanning a number
Parses the whole number token straight from the input buffer, avoiding the
per-character get()/add() of scan_number(). On success it fills token_buffer
(as scan_number() does) and
(with the locale decimal point substituted, as scan_number() does) and
returns the token type. On anything it does not fully recognize as a
well-formed number it makes no state change and returns
token_type::uninitialized, so the caller falls back to scan_number(), which
@@ -1755,11 +1707,16 @@ scan_number_done:
}
#endif
// materialize the token exactly as scan_number() would. reset() already
// cleared token_buffer, so append() fills it (assign() is avoided
// because custom string_t types need not provide it)
// materialize the token exactly as scan_number() would, substituting the
// locale decimal point so convert_number()'s strtof fallback stays valid.
// reset() already cleared token_buffer, so append() fills it (assign() is
// avoided because custom string_t types need not provide it)
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
decimal_point_position = dot_index;
if (dot_index != std::string::npos)
{
token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
decimal_point_position = dot_index;
}
ia.bulk_skip(len - 1);
position.chars_read_total += (len - 1);
@@ -2026,7 +1983,11 @@ scan_number_done:
/// return current string value (implicitly resets the token; useful only once)
string_t& get_string()
{
// a number token holds '.' regardless of the locale (#4084)
// translate decimal points from locale back to '.' (#4084)
if (decimal_point_char != '.' && decimal_point_position != std::string::npos)
{
token_buffer[decimal_point_position] = '.';
}
return token_buffer;
}
@@ -2322,7 +2283,9 @@ scan_number_done:
number_unsigned_t value_unsigned = 0;
number_float_t value_float = 0;
/// the position of the decimal point in token_buffer
/// the decimal point
const char_int_type decimal_point_char = '.';
/// the position of the decimal point in the input
std::size_t decimal_point_position = std::string::npos;
/// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
+13 -8
View File
@@ -118,12 +118,14 @@ std::strtod. The parser only activates for number_float_t == double; float and
long double keep the std::strtof/std::strtold paths (see the templated overload
below).
@param[in] first pointer to the first character of the number
@param[in] last pointer past the last character
@param[out] out the parsed value on success
@param[in] first pointer to the first character of the number
@param[in] last pointer past the last character
@param[in] decimal_point the (locale-dependent) decimal point character
@param[out] out the parsed value on success
@return true if the value was parsed exactly; false to fall back to strtod
*/
inline bool parse_float_fast(const char* first, const char* last, double& out) noexcept
template<typename DecimalPointType>
bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
{
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
// Clinger's fast path is only exact when double operations are evaluated in
@@ -134,6 +136,7 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
// std::from_chars / std::strtod path.
static_cast<void>(first);
static_cast<void>(last);
static_cast<void>(decimal_point);
static_cast<void>(out);
return false;
#else
@@ -172,7 +175,7 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
++num_digits;
fractional_digits += static_cast<int>(seen_dot);
}
else if (c == '.')
else if (static_cast<DecimalPointType>(c) == decimal_point)
{
if (JSON_HEDLEY_UNLIKELY(seen_dot))
{
@@ -257,8 +260,8 @@ inline bool parse_float_fast(const char* first, const char* last, double& out) n
}
/// fast float path is only exact for `double`; decline for float/long double
template<typename FloatType>
bool parse_float_fast(const char* /*first*/, const char* /*last*/, FloatType& /*out*/) noexcept
template<typename DecimalPointType, typename FloatType>
bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
{
return false;
}
@@ -270,7 +273,9 @@ std::from_chars is locale-independent, correctly rounded, and - via the
Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
over the whole value range (not just the Clinger subset). It is used only when
__cpp_lib_to_chars indicates full floating-point support and only when it
consumes the entire token ([first, last)). An under-/overflow (result_out_of_range) also declines, so
consumes the entire token ([first, last)); a partial parse means the buffer
uses a non-'.' locale decimal point, in which case the caller falls back to the
locale-aware path. An under-/overflow (result_out_of_range) also declines, so
the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
expects (side-stepping the P4168 divergence between implementations).
+4
View File
@@ -915,3 +915,7 @@ void templated_json_throw(ExceptionType exception)
#ifndef JSON_DISABLE_ENUM_SERIALIZATION
#define JSON_DISABLE_ENUM_SERIALIZATION 0
#endif
#ifndef JSON_USE_GLOBAL_UDLS
#define JSON_USE_GLOBAL_UDLS 1
#endif
@@ -25,6 +25,7 @@
#undef JSON_INLINE_VARIABLE
#undef JSON_NO_UNIQUE_ADDRESS
#undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_USE_GLOBAL_UDLS
#ifndef JSON_TEST_KEEP_MACROS
#undef JSON_CATCH
+465 -175
View File
@@ -6061,21 +6061,240 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
// the patch
basic_json result(value_t::array);
diff_recursively(result, source, target, path, 0);
return result;
}
// if the values are the same, return an empty patch
private:
/// @brief two arrays or two objects @ref diff_iteratively is diffing
struct diff_frame
{
diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
: source(source_), target(target_), path_length(path_length_)
{}
// declared for GCC's -Weffc++, which asks for them in a class with
// pointer members and a non-trivial destructor; the exception
// specifications are left implicit, as GCC 4.8 rejects explicit ones
// that differ from them
diff_frame(const diff_frame&) = default;
diff_frame(diff_frame&&) = default;
diff_frame& operator=(const diff_frame&) = default;
diff_frame& operator=(diff_frame&&) = default;
~diff_frame() = default;
/// the values being diffed, both arrays or both objects
const basic_json* source;
const basic_json* target;
/// the length of their path in `current_path`
std::size_t path_length;
/// arrays: the next index to diff
std::size_t index = 0;
/// objects: the next member of source to look at
const_iterator member{}; // NOLINT(readability-redundant-member-init)
/// objects: the keys common to both, in source's order
std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
/// objects: the next entry of common_keys
std::size_t next_common = 0;
/// objects: the "add" operations for keys only target has
basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
};
// The operations of a diff are built by the functions below rather than
// where they are needed: building one takes several temporaries, and
// unoptimized builds give each temporary a stack slot of its own in the
// function it appears in. In diff_recursively, which is on the call stack
// once per nesting level, that made every level cost kilobytes of stack.
/// @brief append a "replace" operation for @a path with @a value to @a result
static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", value}
});
}
/// @brief append a "remove" operation for @a path to @a result
static void diff_remove(basic_json& result, const string_t& path)
{
result.push_back(object(
{
{"op", "remove"}, {"path", path}
}));
}
/// @brief append an "add" operation for @a path with @a value to @a result
static void diff_add(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "add"}, {"path", path}, {"value", value}
});
}
/// @brief append the "remove" operations for the elements of array
/// @a source from @a index on, and the "add" operations for the
/// elements of array @a target from source's size on, to @a result
static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t index)
{
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > index; --j)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
}
// add other remaining elements
for (std::size_t i = source.size(); i < target.size(); ++i)
{
diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
}
}
/*!
@brief compare the keys of objects @a source and @a target
If the keys both objects have are in the same order in both, and the keys
only @a target has come after them, stores the keys common to both in
source's order in @a common_keys, stores the "add" operations for the keys
only @a target has in @a added_ops, and returns true: the caller then diffs
the objects member by member. Otherwise, appends operations that remove
every member of @a source and add every member of @a target to @a result,
and returns false.
*/
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, std::vector<typename object_t::key_type>& common_keys,
basic_json& added_ops)
{
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the per-key diffs in the caller's fast path, to match
// source's original iteration order (as the original,
// pre-reordering-aware implementation did) instead of
// grouping all removes before all per-key diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// The patch ops for keys that were added (i.e., in target but not
// in source) are built here so the fast path can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key diff is correct
// and minimal, as before
common_keys = std::move(common_keys_source_order);
return true;
}
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
return false;
}
/*!
@brief @ref diff, for values at nesting level @a depth, appending the
operations to @a result
Diffing two arrays or objects calls this function again, once per nesting
level, so values nested deeply enough used to exhaust the call stack and
terminate the process. The descent is bounded here: once @ref
detail::recursion_depth_limit levels have been entered, @ref
diff_iteratively diffs what is left without the call stack.
*/
static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t depth)
{
// if the values are the same, there is nothing to do
if (source == target)
{
return result;
return;
}
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
diff_iteratively(result, source, target, path);
return;
}
if (source.type() != target.type())
{
// different types: replace value
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
return result;
diff_replace(result, path, target);
return;
}
switch (source.type())
@@ -6087,185 +6306,50 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
while (i < source.size() && i < target.size())
{
// recursive call to compare array values at index i
auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1);
++i;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > i; --j)
{
result.push_back(object(
{
{"op", "remove"},
{"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
}));
}
i = source.size();
// add other remaining elements
while (i < target.size())
{
result.push_back(
{
{"op", "add"},
{"path", detail::concat<string_t>(path, "/-")},
{"value", target[i]}
});
++i;
}
diff_array_tails(result, source, target, path, i);
break;
}
case value_t::object:
{
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the recursive per-key diffs in the fast path below,
// to match source's original iteration order (as the
// original, pre-reordering-aware implementation did) instead
// of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path below, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// patch ops for keys that were added (i.e., in target but not
// in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
if (diff_object_keys(result, source, target, path, common_keys, added_ops))
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key recursive diff
// is correct and minimal, as before. common_keys_source_order
// is, by construction, the subsequence of source's keys
// that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
// fast path: common_keys is, by construction, the
// subsequence of source's keys that are common to both
// objects, in source's iteration order -- so it can be
// walked in lockstep with `source` using a cheap key
// comparison instead of another lookup. Deleted keys
// (those source keys not in common_keys) are interleaved
// here too, in source's original order, to match the
// historical (pre-reordering-aware) output order.
auto common_it = common_keys.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
if (common_it != common_keys.cend() && it.key() == *common_it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1);
++common_it;
}
else
{
// found a key that is not in target -> remove it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
}
}
// append the "add" ops for brand-new keys collected above
// during the pass over target -- no second source.find()
// per target key needed
// append the "add" ops for brand-new keys collected by
// diff_object_keys -- no second source.find() per target
// key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
else
{
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
}
break;
}
@@ -6280,16 +6364,170 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
default:
{
// both primitive types: replace value
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
diff_replace(result, path, target);
break;
}
}
return result;
}
/*!
@brief @ref diff without the call stack, appending the operations to
@a result
Produces the same operations as @ref diff_recursively. Only reached for
values nested more deeply than @ref detail::recursion_depth_limit.
*/
static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path)
{
// The arrays and objects being diffed are kept on an explicit stack,
// and every pair of elements is still diffed completely before the
// next one, so the operations come out in the same order as in
// diff_recursively. The path of the values being diffed is kept in
// one buffer that grows and shrinks with the stack, rather than in a
// new string per level.
std::vector<diff_frame> stack;
string_t current_path = path;
// diff `s` against `t`, whose path is current_path: primitives,
// values of different types, and objects whose members were reordered
// are handled right away; arrays and other objects get a frame
const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
{
// if the values are the same, there is nothing to do. Arrays and
// objects are not compared up front: comparing them visits
// everything below them, so doing that at every level would take
// quadratic time in the nesting depth - equal ones yield no
// operations anyway.
if ((!s.is_structured() || !t.is_structured()) && s == t)
{
return;
}
if (s.type() != t.type())
{
// different types: replace value
diff_replace(result, current_path, t);
return;
}
switch (s.type())
{
case value_t::array:
{
stack.emplace_back(&s, &t, current_path.size());
return;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
{
// fast path: the frame walks source in lockstep with
// common_keys, as diff_recursively does, and appends
// added_ops once all members are done
stack.emplace_back(&s, &t, current_path.size());
stack.back().member = s.cbegin();
stack.back().common_keys = std::move(common_keys);
stack.back().added_ops = std::move(added_ops);
}
return;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// both primitive types: replace value
diff_replace(result, current_path, t);
return;
}
}
};
enter(source, target);
while (!stack.empty())
{
// the frame is copied out member by member and changed through
// stack.back(): enter() may push a frame and the end of the loop
// pops it, either of which would invalidate a reference to it
const basic_json* const s = stack.back().source;
const basic_json* const t = stack.back().target;
const std::size_t path_length = stack.back().path_length;
const std::size_t depth = stack.size();
if (s->is_array())
{
const auto& source_array = *s->m_data.m_value.array;
const auto& target_array = *t->m_data.m_value.array;
// first pass: traverse common elements
const std::size_t i = stack.back().index;
if (i < source_array.size() && i < target_array.size())
{
++stack.back().index;
detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
enter(source_array[i], target_array[i]);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
continue;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, *s, *t, current_path, i);
}
else
{
const const_iterator it = stack.back().member;
if (it != s->cend())
{
++stack.back().member;
const std::size_t next_common = stack.back().next_common;
if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
{
++stack.back().next_common;
const basic_json& target_value = (*t)[it.key()];
detail::concat_into(current_path, '/', detail::escape(it.key()));
enter(it.value(), target_value);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
}
continue;
}
// append the "add" ops for brand-new keys collected when the
// object was entered
result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
}
// this array or object is done: continue with the one it is in
stack.pop_back();
if (!stack.empty())
{
current_path.resize(stack.back().path_length);
}
}
}
public:
/// @}
////////////////////////////////
@@ -6423,6 +6661,55 @@ std::string format_as(const NLOHMANN_BASIC_JSON_TPL& j)
return j.dump();
}
inline namespace literals
{
inline namespace json_literals
{
/// @brief user-defined string literal for JSON values
/// @sa https://json.nlohmann.me/api/basic_json/operator_literal_json/
JSON_HEDLEY_NON_NULL(1)
#if !defined(JSON_HEDLEY_GCC_VERSION) || JSON_HEDLEY_GCC_VERSION_CHECK(4,9,0)
inline nlohmann::json operator""_json(const char* s, std::size_t n)
#else
// GCC 4.8 requires a space between "" and suffix
inline nlohmann::json operator"" _json(const char* s, std::size_t n)
#endif
{
return nlohmann::json::parse(s, s + n);
}
#if defined(__cpp_char8_t)
JSON_HEDLEY_NON_NULL(1)
inline nlohmann::json operator""_json(const char8_t* s, std::size_t n)
{
return nlohmann::json::parse(reinterpret_cast<const char*>(s),
reinterpret_cast<const char*>(s) + n);
}
#endif
/// @brief user-defined string literal for JSON pointer
/// @sa https://json.nlohmann.me/api/basic_json/operator_literal_json_pointer/
JSON_HEDLEY_NON_NULL(1)
#if !defined(JSON_HEDLEY_GCC_VERSION) || JSON_HEDLEY_GCC_VERSION_CHECK(4,9,0)
inline nlohmann::json::json_pointer operator""_json_pointer(const char* s, std::size_t n)
#else
// GCC 4.8 requires a space between "" and suffix
inline nlohmann::json::json_pointer operator"" _json_pointer(const char* s, std::size_t n)
#endif
{
return nlohmann::json::json_pointer(std::string(s, n));
}
#if defined(__cpp_char8_t)
inline nlohmann::json::json_pointer operator""_json_pointer(const char8_t* s, std::size_t n)
{
return nlohmann::json::json_pointer(std::string(reinterpret_cast<const char*>(s), n));
}
#endif
} // namespace json_literals
} // namespace literals
NLOHMANN_JSON_NAMESPACE_END
///////////////////////
@@ -6557,6 +6844,17 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
} // namespace std
#if JSON_USE_GLOBAL_UDLS
#if !defined(JSON_HEDLEY_GCC_VERSION) || JSON_HEDLEY_GCC_VERSION_CHECK(4,9,0)
using nlohmann::literals::json_literals::operator""_json; // NOLINT(misc-unused-using-decls,google-global-names-in-headers)
using nlohmann::literals::json_literals::operator""_json_pointer; //NOLINT(misc-unused-using-decls,google-global-names-in-headers)
#else
// GCC 4.8 requires a space between "" and suffix
using nlohmann::literals::json_literals::operator"" _json; // NOLINT(misc-unused-using-decls,google-global-names-in-headers)
using nlohmann::literals::json_literals::operator"" _json_pointer; //NOLINT(misc-unused-using-decls,google-global-names-in-headers)
#endif
#endif
#include <nlohmann/detail/macro_unscope.hpp>
// End of GCC diagnostic pragmas for C++ modules support
@@ -6564,12 +6862,4 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
#pragma GCC diagnostic pop
#endif
// The user-defined string literals are in a separate header, because their
// bodies instantiate the parser in every translation unit that includes them.
// Define JSON_NO_AUTOMATIC_UDLS to include <nlohmann/json_literals.hpp> only
// where needed.
#ifndef JSON_NO_AUTOMATIC_UDLS
#include <nlohmann/json_literals.hpp>
#endif
#endif // INCLUDE_NLOHMANN_JSON_HPP_
-83
View File
@@ -1,83 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#ifndef INCLUDE_NLOHMANN_JSON_LITERALS_HPP_
#define INCLUDE_NLOHMANN_JSON_LITERALS_HPP_
#include <cstddef> // size_t
#include <string> // string
#include <nlohmann/json.hpp>
// This header is included at the end of <nlohmann/json.hpp> unless
// JSON_NO_AUTOMATIC_UDLS is defined, and can be included on its own after that.
// Either way, the library's internal macros are no longer defined here (and the
// amalgamation inlines macro_scope.hpp only once), so only standard and public
// macros may be used below.
NLOHMANN_JSON_NAMESPACE_BEGIN
inline namespace literals
{
inline namespace json_literals
{
/// @brief user-defined string literal for JSON values
/// @sa https://json.nlohmann.me/api/operator_literal_json/
#if !defined(__GNUC__) || defined(__clang__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)
inline nlohmann::json operator""_json(const char* s, std::size_t n)
#else
// GCC 4.8 requires a space between "" and suffix
inline nlohmann::json operator"" _json(const char* s, std::size_t n)
#endif
{
return nlohmann::json::parse(s, s + n);
}
#if defined(__cpp_char8_t)
inline nlohmann::json operator""_json(const char8_t* s, std::size_t n)
{
return nlohmann::json::parse(reinterpret_cast<const char*>(s),
reinterpret_cast<const char*>(s) + n);
}
#endif
/// @brief user-defined string literal for JSON pointer
/// @sa https://json.nlohmann.me/api/operator_literal_json_pointer/
#if !defined(__GNUC__) || defined(__clang__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)
inline nlohmann::json::json_pointer operator""_json_pointer(const char* s, std::size_t n)
#else
// GCC 4.8 requires a space between "" and suffix
inline nlohmann::json::json_pointer operator"" _json_pointer(const char* s, std::size_t n)
#endif
{
return nlohmann::json::json_pointer(std::string(s, n));
}
#if defined(__cpp_char8_t)
inline nlohmann::json::json_pointer operator""_json_pointer(const char8_t* s, std::size_t n)
{
return nlohmann::json::json_pointer(std::string(reinterpret_cast<const char*>(s), n));
}
#endif
} // namespace json_literals
} // namespace literals
NLOHMANN_JSON_NAMESPACE_END
#if !defined(JSON_USE_GLOBAL_UDLS) || JSON_USE_GLOBAL_UDLS
#if !defined(__GNUC__) || defined(__clang__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)
using nlohmann::literals::json_literals::operator""_json; // NOLINT(misc-unused-using-decls,google-global-names-in-headers)
using nlohmann::literals::json_literals::operator""_json_pointer; //NOLINT(misc-unused-using-decls,google-global-names-in-headers)
#else
// GCC 4.8 requires a space between "" and suffix
using nlohmann::literals::json_literals::operator"" _json; // NOLINT(misc-unused-using-decls,google-global-names-in-headers)
using nlohmann::literals::json_literals::operator"" _json_pointer; //NOLINT(misc-unused-using-decls,google-global-names-in-headers)
#endif
#endif
#endif // INCLUDE_NLOHMANN_JSON_LITERALS_HPP_
-1
View File
@@ -15,7 +15,6 @@ nlohmann_json_multiple_headers = declare_dependency(
if not meson.is_subproject()
install_headers('single_include/nlohmann/json.hpp', subdir: 'nlohmann')
install_headers('single_include/nlohmann/json_fwd.hpp', subdir: 'nlohmann')
install_headers('single_include/nlohmann/json_literals.hpp', subdir: 'nlohmann')
pkgc = import('pkgconfig')
pkgc.generate(name: 'nlohmann_json',
File diff suppressed because it is too large Load Diff
-83
View File
@@ -1,83 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#ifndef INCLUDE_NLOHMANN_JSON_LITERALS_HPP_
#define INCLUDE_NLOHMANN_JSON_LITERALS_HPP_
#include <cstddef> // size_t
#include <string> // string
#include <nlohmann/json.hpp>
// This header is included at the end of <nlohmann/json.hpp> unless
// JSON_NO_AUTOMATIC_UDLS is defined, and can be included on its own after that.
// Either way, the library's internal macros are no longer defined here (and the
// amalgamation inlines macro_scope.hpp only once), so only standard and public
// macros may be used below.
NLOHMANN_JSON_NAMESPACE_BEGIN
inline namespace literals
{
inline namespace json_literals
{
/// @brief user-defined string literal for JSON values
/// @sa https://json.nlohmann.me/api/operator_literal_json/
#if !defined(__GNUC__) || defined(__clang__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)
inline nlohmann::json operator""_json(const char* s, std::size_t n)
#else
// GCC 4.8 requires a space between "" and suffix
inline nlohmann::json operator"" _json(const char* s, std::size_t n)
#endif
{
return nlohmann::json::parse(s, s + n);
}
#if defined(__cpp_char8_t)
inline nlohmann::json operator""_json(const char8_t* s, std::size_t n)
{
return nlohmann::json::parse(reinterpret_cast<const char*>(s),
reinterpret_cast<const char*>(s) + n);
}
#endif
/// @brief user-defined string literal for JSON pointer
/// @sa https://json.nlohmann.me/api/operator_literal_json_pointer/
#if !defined(__GNUC__) || defined(__clang__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)
inline nlohmann::json::json_pointer operator""_json_pointer(const char* s, std::size_t n)
#else
// GCC 4.8 requires a space between "" and suffix
inline nlohmann::json::json_pointer operator"" _json_pointer(const char* s, std::size_t n)
#endif
{
return nlohmann::json::json_pointer(std::string(s, n));
}
#if defined(__cpp_char8_t)
inline nlohmann::json::json_pointer operator""_json_pointer(const char8_t* s, std::size_t n)
{
return nlohmann::json::json_pointer(std::string(reinterpret_cast<const char*>(s), n));
}
#endif
} // namespace json_literals
} // namespace literals
NLOHMANN_JSON_NAMESPACE_END
#if !defined(JSON_USE_GLOBAL_UDLS) || JSON_USE_GLOBAL_UDLS
#if !defined(__GNUC__) || defined(__clang__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)
using nlohmann::literals::json_literals::operator""_json; // NOLINT(misc-unused-using-decls,google-global-names-in-headers)
using nlohmann::literals::json_literals::operator""_json_pointer; //NOLINT(misc-unused-using-decls,google-global-names-in-headers)
#else
// GCC 4.8 requires a space between "" and suffix
using nlohmann::literals::json_literals::operator"" _json; // NOLINT(misc-unused-using-decls,google-global-names-in-headers)
using nlohmann::literals::json_literals::operator"" _json_pointer; //NOLINT(misc-unused-using-decls,google-global-names-in-headers)
#endif
#endif
#endif // INCLUDE_NLOHMANN_JSON_LITERALS_HPP_
-1
View File
@@ -18,7 +18,6 @@ module;
// See: https://github.com/nlohmann/json/issues/5103
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp>
export module nlohmann.json;
+2 -43
View File
@@ -1830,51 +1830,10 @@ TEST_CASE("CBOR")
SECTION("invalid string in map")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
CHECK(json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01}), true, false).is_discarded());
}
SECTION("non-string key (see #2766 and #3381)")
{
// only text strings map to JSON object keys; any other key is
// rejected with a message naming its type
const std::vector<std::pair<std::vector<std::uint8_t>, std::string>> cases =
{
{{0xA1, 0x01, 0x01}, "an unsigned integer; last byte: 0x01"},
{{0xA1, 0x20, 0x01}, "a negative integer; last byte: 0x20"},
{{0xA1, 0x41, 0x61, 0x01}, "a byte string; last byte: 0x41"},
{{0xA1, 0x80, 0x01}, "an array; last byte: 0x80"},
{{0xA1, 0xA0, 0x01}, "a map; last byte: 0xA0"},
{{0xA1, 0xC0, 0x61, 0x61, 0x01}, "a tag; last byte: 0xC0"},
{{0xA1, 0xF4, 0x01}, "a boolean; last byte: 0xF4"},
{{0xA1, 0xF5, 0x01}, "a boolean; last byte: 0xF5"},
{{0xA1, 0xF6, 0x01}, "null; last byte: 0xF6"},
{{0xA1, 0xF7, 0x01}, "undefined; last byte: 0xF7"},
{{0xA1, 0xF9, 0x3C, 0x00, 0x01}, "a floating-point number; last byte: 0xF9"},
{{0xA1, 0xFA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFA"},
{{0xA1, 0xFB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFB"},
{{0xA1, 0xE0, 0x01}, "a simple value; last byte: 0xE0"},
{{0xA1, 0xF8, 0x20, 0x01}, "a simple value; last byte: 0xF8"},
// indefinite-length map
{{0xBF, 0x01, 0x01, 0xFF}, "an unsigned integer; last byte: 0x01"},
};
for (const auto& c : cases)
{
CAPTURE(c.first)
const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found " + c.second;
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(c.first), expected.c_str(), json::parse_error&);
CHECK(json::from_cbor(c.first, true, false).is_discarded());
}
// a key of major type 3 with a reserved length is still reported as
// a malformed string, and a missing key as the end of input
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing CBOR string: unexpected end of input", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0x7C, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C", json::parse_error&);
}
SECTION("invalid UTF-8 in string (see #5529)")
{
// a two-character text string (major type 3) whose bytes are not
@@ -2325,7 +2284,7 @@ TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
SECTION("a break marker outside an indefinite-length string is not a string")
{
// 0xFF only closes a string that was opened; on its own it is not one
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
}
}
+1 -1
View File
@@ -666,7 +666,7 @@ TEST_CASE("parse_float_fast declines what it cannot convert exactly")
// always safe: the caller then falls back to a slower, exact conversion.
const auto fast = [](const std::string & s, double & out)
{
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), '.', out);
};
double out = 0;
+153
View File
@@ -15,8 +15,65 @@ using nlohmann::json;
#endif
#include <fstream>
#include <string>
#include <vector>
#include "make_test_data_available.hpp"
namespace
{
// alternating objects and arrays nested `depth` levels deep, with members that
// depend on `variant` at some levels, so diffing two variants yields
// operations on many levels: replacing the innermost value, adding, removing,
// and (for ordered_json) reordering members, and changing array lengths
template<typename BasicJsonType>
BasicJsonType nested(const std::size_t depth, const int variant)
{
BasicJsonType value = variant;
for (std::size_t i = 0; i < depth; ++i)
{
if (i % 2 == 0)
{
BasicJsonType object = BasicJsonType::object();
if ((i + static_cast<std::size_t>(variant)) % 7 == 0)
{
object["x"] = i;
}
if (variant == 2 && i % 11 == 0)
{
object["z"] = "z";
}
object["a"] = std::move(value);
if (variant == 1 && i % 5 == 0)
{
object["y"] = 1;
}
value = std::move(object);
}
else
{
BasicJsonType array = BasicJsonType::array({std::move(value)});
if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
{
array.push_back(i);
}
value = std::move(array);
}
}
return value;
}
// a path of `depth` reference tokens, as nested() nests its values
std::string nested_path(const std::size_t depth)
{
std::string path;
for (std::size_t i = depth; i > 0; --i)
{
path += (i - 1) % 2 == 0 ? "/a" : "/0";
}
return path;
}
} // namespace
TEST_CASE("JSON patch")
{
SECTION("examples from RFC 6902")
@@ -1752,6 +1809,102 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
}
}
TEST_CASE("JSON patch: diff of deeply nested values")
{
SECTION("the diff reproduces the target at every depth")
{
// depths on either side of the nesting depth up to which diff()
// recurses (detail::recursion_depth_limit(), 128); not every depth up
// to 300, as the test would then time out under Valgrind
std::vector<std::size_t> depths;
for (std::size_t depth = 0; depth <= 16; ++depth)
{
depths.push_back(depth);
}
for (std::size_t depth = 120; depth <= 136; ++depth)
{
depths.push_back(depth);
}
depths.push_back(300);
for (const auto depth : depths)
{
CAPTURE(depth);
for (int from = 0; from < 3; ++from)
{
for (int to = 0; to < 3; ++to)
{
CAPTURE(from);
CAPTURE(to);
const auto source = nested<json>(depth, from);
const auto target = nested<json>(depth, to);
const auto patch = json::diff(source, target);
CHECK(source.patch(patch) == target);
CHECK(patch.empty() == (from == to));
const auto ordered_source = nested<nlohmann::ordered_json>(depth, from);
const auto ordered_target = nested<nlohmann::ordered_json>(depth, to);
CHECK(ordered_source.patch(nlohmann::ordered_json::diff(ordered_source, ordered_target)) == ordered_target);
}
}
}
}
SECTION("a difference only in the innermost value is one replace operation")
{
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth);
json source = 1;
json target = 2;
for (std::size_t i = 0; i < depth; ++i)
{
source = i % 2 == 0 ? json::object({{"a", std::move(source)}}) : json::array({std::move(source)});
target = i % 2 == 0 ? json::object({{"a", std::move(target)}}) : json::array({std::move(target)});
}
CHECK(json::diff(source, target, "/root") == json::array({{{"op", "replace"}, {"path", "/root" + nested_path(depth)}, {"value", 2}}}));
}
}
SECTION("values nested too deeply for the call stack (#5393)")
{
// diff() used to recurse once per nesting level, and compared the
// values with operator== on every level. The values are only
// parsed and diffed, never copied or compared, since those recurse
// too.
const std::size_t depth = 100000;
for (const bool objects :
{
false, true
})
{
CAPTURE(objects);
std::string source_text;
std::string target_text;
std::string equal_text;
std::string path;
for (std::size_t i = 0; i < depth; ++i)
{
source_text += objects ? "{\"a\":" : "[";
path += objects ? "/a" : "/0";
}
target_text = source_text + "2";
equal_text = source_text + "1";
source_text += "1";
const std::string closing(depth, objects ? '}' : ']');
const auto source = json::parse(source_text + closing);
const auto patch = json::diff(source, json::parse(target_text + closing));
REQUIRE(patch.size() == 1);
CHECK(patch[0]["op"] == "replace");
CHECK(patch[0]["path"] == path);
CHECK(patch[0]["value"] == 2);
CHECK(json::diff(source, json::parse(equal_text + closing)).empty());
}
}
}
TEST_CASE("JSON patch - every operation on ordered_json")
{
using nlohmann::ordered_json;
-210
View File
@@ -12,12 +12,7 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <array>
#include <clocale>
#include <map>
#include <string>
#include <utility>
#include <vector>
struct ParserImpl final: public nlohmann::json_sax<json>
{
@@ -180,208 +175,3 @@ TEST_CASE("locale-dependent test (LC_NUMERIC=de_DE)")
MESSAGE("locale de_DE is not usable");
}
}
namespace
{
// records the numbers of a flat array and switches LC_NUMERIC to the given
// locale once the array opens - after the lexer was constructed, but before
// any number in the array is lexed
struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
{
explicit LocaleSwitchingSax(const char* switch_to)
: locale_after_open(switch_to)
{}
bool null() override
{
return true;
}
bool boolean(bool /*val*/) override
{
return true;
}
bool number_integer(json::number_integer_t /*val*/) override
{
return true;
}
bool number_unsigned(json::number_unsigned_t /*val*/) override
{
return true;
}
bool number_float(json::number_float_t val, const json::string_t& s) override
{
values.push_back(val);
strings.push_back(s);
return true;
}
bool string(json::string_t& /*val*/) override
{
return true;
}
bool binary(json::binary_t& /*val*/) override
{
return true;
}
bool start_object(std::size_t /*val*/) override
{
return true;
}
bool key(json::string_t& /*val*/) override
{
return true;
}
bool end_object() override
{
return true;
}
bool start_array(std::size_t /*val*/) override
{
switched = std::setlocale(LC_NUMERIC, locale_after_open.c_str()) != nullptr;
return true;
}
bool end_array() override
{
return true;
}
bool parse_error(std::size_t /*val*/, const std::string& /*val*/, const nlohmann::detail::exception& /*val*/) override
{
return false;
}
std::string locale_after_open;
bool switched = false;
std::vector<json::number_float_t> values {}; // NOLINT(readability-redundant-member-init)
std::vector<json::string_t> strings {}; // NOLINT(readability-redundant-member-init)
};
} // namespace
TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
{
// The numbers are chosen so that the conversion also takes the strtod
// fallback, which honors the locale that is current at conversion time:
// too many significant digits for Clinger's fast path, an underflow that
// std::from_chars rejects, and a plain value.
const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
std::string text = "[";
for (const auto& n : numbers)
{
text += (text.size() == 1 ? "" : ",") + n;
}
text += "]";
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
// reference values, parsed without a locale switch
REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
const json expected = json::parse(text);
const long_double_json expected_ld = long_double_json::parse(text);
const std::array<std::pair<const char*, const char*>, 2> transitions =
{
{
{"C", "de_DE"},
{"de_DE", "C"}
}
};
for (const auto& transition : transitions)
{
CAPTURE(transition.first);
CAPTURE(transition.second);
if (std::setlocale(LC_NUMERIC, transition.first) == nullptr)
{
MESSAGE("locale is not usable");
continue;
}
// SAX parsing
{
LocaleSwitchingSax sax(transition.second);
CHECK(json::sax_parse(text, &sax));
if (sax.switched)
{
CHECK(sax.values == expected.get<std::vector<json::number_float_t>>());
CHECK(sax.strings == numbers);
}
}
// DOM parsing with a callback
{
bool switched = false;
const auto cb = [&](int /*depth*/, json::parse_event_t event, json& /*parsed*/) noexcept
{
if (event == json::parse_event_t::array_start)
{
switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
}
return true;
};
const json j = json::parse(text, cb);
if (switched)
{
CHECK(j == expected);
}
}
// a long double goes through std::strtold unless std::from_chars supports it
{
bool switched = false;
const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/) noexcept
{
if (event == long_double_json::parse_event_t::array_start)
{
switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
}
return true;
};
const long_double_json j = long_double_json::parse(text, cb);
if (switched)
{
CHECK(j == expected_ld);
}
}
}
CHECK(std::setlocale(LC_NUMERIC, "C") != nullptr);
}
TEST_CASE("locale with a multi-byte decimal point")
{
// Some locales use a decimal point that is not a single character, e.g.
// U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
// substituted in place for '.', so the strtod fallback stops early. The
// conversion must still terminate rather than retry forever.
const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
bool tested = false;
for (const char* name : names)
{
if (std::setlocale(LC_NUMERIC, name) == nullptr)
{
continue;
}
const std::string decimal_point = std::localeconv()->decimal_point;
if (decimal_point.size() < 2)
{
continue;
}
CAPTURE(name);
tested = true;
// too many significant digits for Clinger's fast path, and an underflow
// that std::from_chars rejects: both reach the strtod fallback
json j;
CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
CHECK(j.is_array());
CHECK(json::accept("3.14159265358979323846"));
// a value the locale-independent paths convert is not affected
CHECK(json::parse("12.5") == 12.5);
}
if (!tested)
{
MESSAGE("no locale with a multi-byte decimal point is usable");
}
CHECK(std::setlocale(LC_NUMERIC, "C") != nullptr);
}
+1 -60
View File
@@ -1551,69 +1551,10 @@ TEST_CASE("MessagePack")
SECTION("invalid string in map")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found an integer; last byte: 0xFF", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF", json::parse_error&);
CHECK(json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01}), true, false).is_discarded());
}
SECTION("non-string key (see #3381)")
{
// only strings map to JSON object keys; any other key is rejected
// with a message naming its type
const std::vector<std::pair<std::vector<std::uint8_t>, std::string>> cases =
{
{{0x81, 0xC0, 0x01}, "nil; last byte: 0xC0"},
{{0x81, 0xC2, 0x01}, "a boolean; last byte: 0xC2"},
{{0x81, 0xC3, 0x01}, "a boolean; last byte: 0xC3"},
{{0x81, 0xCA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a float; last byte: 0xCA"},
{{0x81, 0xCB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a float; last byte: 0xCB"},
{{0x81, 0xC4, 0x00, 0x01}, "a bin; last byte: 0xC4"},
{{0x81, 0xC5, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC5"},
{{0x81, 0xC6, 0x00, 0x00, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC6"},
{{0x81, 0xC7, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC7"},
{{0x81, 0xC8, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC8"},
{{0x81, 0xC9, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC9"},
{{0x81, 0xD4, 0x01, 0x00, 0x01}, "an ext; last byte: 0xD4"},
{{0x81, 0xD5, 0x01, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD5"},
{{0x81, 0xD6, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD6"},
{{0x81, 0xD7, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD7"},
{{0x81, 0xD8, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD8"},
{{0x81, 0xCC, 0x01, 0x01}, "an integer; last byte: 0xCC"},
{{0x81, 0xCD, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCD"},
{{0x81, 0xCE, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCE"},
{{0x81, 0xCF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCF"},
{{0x81, 0xD0, 0x01, 0x01}, "an integer; last byte: 0xD0"},
{{0x81, 0xD1, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD1"},
{{0x81, 0xD2, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD2"},
{{0x81, 0xD3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD3"},
{{0x81, 0x00, 0x01}, "an integer; last byte: 0x00"},
{{0x81, 0x7F, 0x01}, "an integer; last byte: 0x7F"},
{{0x81, 0xE0, 0x01}, "an integer; last byte: 0xE0"},
{{0x81, 0x80, 0x01}, "a map; last byte: 0x80"},
{{0x81, 0x8F, 0x01}, "a map; last byte: 0x8F"},
{{0x81, 0xDE, 0x00, 0x00, 0x01}, "a map; last byte: 0xDE"},
{{0x81, 0xDF, 0x00, 0x00, 0x00, 0x00, 0x01}, "a map; last byte: 0xDF"},
{{0x81, 0x90, 0x01}, "an array; last byte: 0x90"},
{{0x81, 0x9F, 0x01}, "an array; last byte: 0x9F"},
{{0x81, 0xDC, 0x00, 0x00, 0x01}, "an array; last byte: 0xDC"},
{{0x81, 0xDD, 0x00, 0x00, 0x00, 0x00, 0x01}, "an array; last byte: 0xDD"},
};
for (const auto& c : cases)
{
CAPTURE(c.first)
const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found " + c.second;
json _;
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(c.first), expected.c_str(), json::parse_error&);
CHECK(json::from_msgpack(c.first, true, false).is_discarded());
}
json _;
// the unused byte 0xC1 is still reported as a malformed string
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xC1, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xC1", json::parse_error&);
// a missing key is still reported as the end of input
CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing MessagePack string: unexpected end of input", json::parse_error&);
}
SECTION("invalid UTF-8 in string (see #5529)")
{
// a fixstr of length 2 (0xA0 | 2) whose bytes are not valid UTF-8
-97
View File
@@ -1,97 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// This translation unit checks JSON_NO_AUTOMATIC_UDLS, which keeps
// <nlohmann/json.hpp> from including <nlohmann/json_literals.hpp> and thereby
// leaves out the user-defined string literals operator""_json and
// operator""_json_pointer (see #5294), and that including
// <nlohmann/json_literals.hpp> afterwards brings them back.
#define JSON_NO_AUTOMATIC_UDLS 1
#include "doctest_compatibility.h"
#include <cstddef>
#include <utility>
#include <nlohmann/json.hpp>
using json = nlohmann::json;
// An argument type whose associated namespace is the library namespace, so
// argument-dependent lookup of a literal operator called by its function name
// also searches the inline namespaces nlohmann::literals::json_literals.
NLOHMANN_JSON_NAMESPACE_BEGIN
struct no_automatic_udls_probe
{
operator const char* () const // NOLINT(google-explicit-constructor,hicpp-explicit-conversions)
{
return "";
}
};
NLOHMANN_JSON_NAMESPACE_END
namespace
{
// The calls below use a dependent argument, so a literal operator that is not
// declared at all is a substitution failure rather than a hard error: lookup is
// deferred to the point of instantiation, where it considers the declarations
// visible from here (the global using-declarations of JSON_USE_GLOBAL_UDLS) plus
// argument-dependent lookup (the literals in the library namespace).
template<typename T>
using json_udl_t = decltype(operator""_json(std::declval<T>(), std::size_t()));
template<typename T>
using json_pointer_udl_t = decltype(operator""_json_pointer(std::declval<T>(), std::size_t()));
template<typename T>
using has_json_udl = nlohmann::detail::is_detected<json_udl_t, T>;
template<typename T>
using has_json_pointer_udl = nlohmann::detail::is_detected<json_pointer_udl_t, T>;
} // namespace
TEST_CASE("JSON_NO_AUTOMATIC_UDLS")
{
SECTION("literals are not declared")
{
// global namespace (JSON_USE_GLOBAL_UDLS defaults to 1)
CHECK_FALSE(has_json_udl<const char*>::value);
CHECK_FALSE(has_json_pointer_udl<const char*>::value);
// nlohmann::literals::json_literals
CHECK_FALSE(has_json_udl<nlohmann::no_automatic_udls_probe>::value);
CHECK_FALSE(has_json_pointer_udl<nlohmann::no_automatic_udls_probe>::value);
}
SECTION("the rest of the library keeps working")
{
const json j = json::parse(R"({"foo": {"bar": 42}})");
CHECK(j.dump() == R"({"foo":{"bar":42}})");
const json::json_pointer ptr("/foo/bar");
CHECK(j.at(ptr) == 42);
CHECK(j.contains(ptr));
}
}
// the literals can still be added where they are needed
#include <nlohmann/json_literals.hpp>
TEST_CASE("JSON_NO_AUTOMATIC_UDLS with <nlohmann/json_literals.hpp>")
{
SECTION("global namespace")
{
CHECK("[1,2]"_json == json({1, 2}));
CHECK("/a/0"_json_pointer == json::json_pointer("/a/0"));
}
SECTION("nlohmann::literals::json_literals")
{
using namespace nlohmann::literals::json_literals; // NOLINT(google-build-using-namespace)
CHECK(R"({"a":[42]})"_json.at("/a/0"_json_pointer) == 42);
}
}
+3 -3
View File
@@ -1018,7 +1018,7 @@ TEST_CASE("regression tests 1")
};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an array; last byte: 0x98", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x98", json::parse_error&);
// related test case: nonempty UTF-8 string (indefinite length)
std::vector<uint8_t> const vec1 {0x7f, 0x61, 0x61};
@@ -1065,7 +1065,7 @@ TEST_CASE("regression tests 1")
};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&);
// related test case: double-precision
std::vector<uint8_t> const vec2
@@ -1077,7 +1077,7 @@ TEST_CASE("regression tests 1")
0x96, 0x96, 0xb4, 0xb4, 0xfa, 0x94, 0x94, 0x61,
0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0xfb
};
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&);
}
SECTION("issue #452 - Heap-buffer-overflow (OSS-Fuzz issue 585)")