Compare commits

..
Author SHA1 Message Date
Niels Lohmann 8617d9c94c Fix test lints that are red on develop (effc++, clang-tidy) (#5946)
- unit-bson: initialize the SAX recorder's member (-Weffc++)
- unit-json_patch: build the reference document independently instead of
  copying it (performance-unnecessary-copy-initialization)
- unit-json_view_builder: use node_kind::link instead of an out-of-range
  enum cast (clang-analyzer-optin.core.EnumCastOutOfRange)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 17:25:18 +01:00
Niels Lohmann eb8f675bc9 docs: explain signed zero in the diff() round-trip exceptions (#5944)
Spell out that numbers comparing equal produce no operation even if one is
zero and the other negative zero, in diff.md and features/json_patch.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 17:16:08 +01:00
Niels Lohmann 78a0687147 Strong exception guarantee for operator[](json_pointer) (#5940)
* Keep the value unchanged when operator[] with a JSON pointer throws

The non-const operator[](json_pointer) is documented with strong exception
safety, but get_unchecked created intermediate values while walking the
pointer and kept them when a later reference token failed, e.g.
json j; j["/a/99999999999999999999999"_json_pointer] threw 404 and left
{"a":[]}; padding an existing array or appending with "-" behaved alike,
and so did std::length_error/std::bad_alloc from a huge index.

get_unchecked now only walks existing values up to the last reference
token. The last token's key insertion or array growth is strongly
exception safe by itself. Any other change goes through
get_unchecked_create, which records the one change it makes to an existing
value (null converted, key inserted, array grown) in a RAII guard and
reverts it if a later step throws; everything else it creates lies below
that change. The guard needs no try/catch, so JSON_NOEXCEPTION builds are
unaffected. Successful calls create the same values as before.

Docs: state the guarantee in the exceptions list and version history of
operator[], fix the complexity of the pointer overloads of operator[] and
at, and warn that padding arrays up to an untrusted index allocates memory
proportional to the index (operator[].md, untrusted_input.md).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the strongly exception-safe pointer operator[] faster than before

The previous commit made get_unchecked strongly exception safe but slower
with GCC 16 when it creates values (up to +6 % instructions for appending
with "-"). This restructures it so that no benchmark row is slower than
before the strong guarantee, with GCC 16, Clang 22 and Apple Clang.

- get_unchecked walks existing values in one loop that inserts nothing.
  The last reference token is resolved in that walk: "-" appends with
  emplace_back(), an index grows the array with operator[], and a missing
  key is inserted with emplace_hint() at the position the walk's
  lower_bound() found, so the key is searched once.
- With std::less<string_t> or std::less<> as comparator, the walk checks
  the key at the lower bound for equality instead of equivalence, which is
  the same for these comparators and cheaper.
- get_unchecked_create (not inlined) handles all other changes. Its guard
  swaps a null value with a fresh null value instead of copying it and
  reverts the change in a non-inlined function, so a successful call pays
  one compare for it. As every null value converted afterwards is a fresh
  one, it is converted in place instead of assigning a new array or object.
- Keys are added to new objects with emplace(), which also replaces the
  basic_json::operator[] call for ordered_json.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix rollback of pointer operator[] for unusual object and string types

Follow-up to the strong exception guarantee of the non-const
operator[](json_pointer):

- get_unchecked_create assumed that emplace() always inserts the key that
  the walk did not find. For object types whose lookup and emplace()
  disagree (e.g. a copied fifo_map), nothing was inserted, the rollback
  erased the existing member, and the walk continued with a value of the
  wrong type (assertion, type_error.305, or a crash). It now checks the
  result of emplace(), arms the rollback only if a key was inserted, and
  otherwise continues the walk with the member found, as basic_json's
  operator[] does: such pointers throw out_of_range.404 and leave the value
  unchanged, as before the strong guarantee.
- The rollback erased the inserted key by key, which converts string_t to
  object_t::key_type and may allocate inside a noexcept function
  (std::terminate on bad_alloc with custom key types). It now erases by the
  iterator emplace() returned, and shrinks a grown array with erase() at
  the end instead of resize(). The documentation states the remaining
  requirement for custom template parameters.
- Creating values no longer calls string_t::begin(), so the non-const
  operator[](json_pointer) compiles with string types that lack it (as the
  alternative string type of the tests).
- Correct the @throw lists of get_unchecked and get_unchecked_create.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review comments on operator[](json_pointer) rollback

- undo() shrinks the array with resize() instead of erase()
- comments say what "will be" done in get_unchecked_create
- tests: avoid a clang-tidy performance-unnecessary-copy-initialization
  hit (unit-alt-string) and the word "access" in section names, which
  Flawfinder flags as race/access

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Let the pointer rollback guard call undo() unconditionally

The check in the destructor and JSON_HEDLEY_NEVER_INLINE on undo() made no
measurable difference: instruction counts of j[ptr] are identical with
GCC 13 -O3 and within 0.4 % with clang, timings are within noise.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 17:15:55 +01:00
28 changed files with 1199 additions and 1021 deletions

No files matched your search

+2 -1
View File
@@ -90,7 +90,8 @@ Strong exception safety: if an exception occurs, the original value stays intact
1. Constant.
2. Logarithmic in the size of the container.
3. Logarithmic in the size of the container.
4. Logarithmic in the size of the container.
4. Linear in the number of reference tokens of `ptr`; for each of them, logarithmic in the size of the object, or
constant for an array.
## Notes
+4 -3
View File
@@ -19,9 +19,10 @@ The equality holds up to [`operator==`](operator_eq.md), so it has the following
- A value that is not equal to itself cannot round-trip. This applies to `NaN` (also nested in an array or object) and
to [discarded](is_discarded.md) values: `diff` emits a `replace` operation for them. A `NaN` in the patch is
serialized as `#!json null`, so a patch that was dumped and parsed again would set `#!json null`.
- Numbers that compare equal produce no operation, even if their types or signs differ: `diff(1, 1.0)`,
`diff(1u, 1)` (unsigned and signed integer) and `diff(0.0, -0.0)` all return `#!json []`. The patched value is then
the `source` and keeps its number type.
- Numbers that compare equal produce no operation, even if their types differ or if one is zero and the other is
negative zero: `diff(1, 1.0)` (integer and floating-point), `diff(1u, 1)` (unsigned and signed integer) and
`diff(0.0, -0.0)` (positive and negative zero) all return `#!json []`. The patched value is then the `source` and
keeps its number type and sign.
## Parameters
+20 -2
View File
@@ -92,12 +92,20 @@ Strong exception safety: if an exception occurs, the original value stays intact
For the **const** version, an object key or array index in `ptr` that does not exist is not reported by an
exception, but is undefined behavior (see the notes below). Use [`at`](at.md) for checked access.
If the **non-const** version throws, the value is left unchanged: values it created for earlier reference tokens
of `ptr` are removed again. This also holds for exceptions such as `#!cpp std::bad_alloc` or
`#!cpp std::length_error` thrown while padding an array up to a very large index. With custom
[template parameters](../../features/types/template_parameters.md), this requires that `erase()` of the object and
array types and the move operations of the custom base class do not throw.
## Complexity
1. Constant if `idx` is in the range of the array. Otherwise, linear in `idx - size()`.
2. Logarithmic in the size of the container.
3. Logarithmic in the size of the container.
4. Logarithmic in the size of the container.
4. Linear in the number of reference tokens of `ptr`; for each of them, logarithmic in the size of the object, or
constant for an array. When the non-const version pads an array with `#!json null` values, it is additionally
linear in `idx - size()` for that array.
## Notes
@@ -145,6 +153,14 @@ Strong exception safety: if an exception occurs, the original value stays intact
specified by the JSON Pointer RFC; it is this library's own, intentional disambiguation rule. See also
[JSON Pointer](../../features/json_pointer.md).
!!! warning "Large array indices"
As the array is filled up with `#!json null` values to the given index, the memory used grows with the index
and not with the size of the pointer: on a 64-bit platform, the index `10000000` allocates about 160 MB.
The library does not limit the index, so check the array indices of a JSON pointer from untrusted input
before using it with the non-const version. See
[Parsing untrusted input](../../features/parsing/untrusted_input.md#json-pointers-and-flattened-keys).
!!! warning "Deprecation"
Overload (4) also accepts a [`json_pointer`](../json_pointer/index.md) whose template argument is a `basic_json`
@@ -296,4 +312,6 @@ Strong exception safety: if an exception occurs, the original value stays intact
4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
version 3.13.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is not
a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.
a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. Fixed in version 3.13.0 to leave the value
unchanged if the non-const version throws (strong exception safety, as documented); before, values created for earlier
reference tokens of `ptr` were kept.
@@ -32,20 +32,9 @@ uses, so the result matches it exactly — including, for an object, that a repe
last value. The replay is iterative, so it is not limited by the call stack the way a naive recursive conversion
would be; the JSON nesting depth is limited only by available memory, as for `BasicJsonType::parse()` itself.
With [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values get the source
positions [`BasicJsonType::parse()`](../basic_json/parse.md) would give them, also for the subtree of a view (the
positions are those in the document's whole source text), and exceptions about them name the same bytes. For values of
an [editable document](../json_editable_document.md), positions are all-or-nothing, and [`start_pos()`](../basic_json/start_pos.md)
and [`end_pos()`](../basic_json/end_pos.md) return `std::string::npos` for
- a value written by an edit (a copy of an array or object of the document is also new; copies of its scalars keep
their positions),
- an array or object whose elements were changed (its elements keep theirs), and
- an array or object whose last element has no positions (its end is found from that element).
A `basic_json` value, in contrast, keeps its positions when it is changed. Documents [loaded](../basic_json_document/load.md)
from an image have the positions of the saved document, except that its strings with escapes have none if it was saved
without `JSON_DIAGNOSTIC_POSITIONS`.
Unlike parsing with [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values
produced by `materialize()` do not carry source positions. The index has the start offset of every value, but not the
end of arrays, objects, and escaped strings, which the positions also need.
Calling `materialize()` on the same view repeatedly builds a new, independent `BasicJsonType` value each time; it
never caches the result.
@@ -73,7 +62,8 @@ never caches the result.
- [`root`](../basic_json_document/root.md) - the view of a document's root value
- [`BasicJsonType::parse`](../basic_json/parse.md) - build a `BasicJsonType` value directly from a JSON text
- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values
- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values (not
produced by `materialize()`)
- [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - when to build a basic_json value from a view
## Version history
@@ -113,7 +113,6 @@ When the macro is not defined, the library will define it to its default value.
- [:simple-cmake: JSON_Diagnostic_Positions](../../integration/cmake.md#json_diagnostic_positions) - CMake option to control the macro
- [`JSON_DIAGNOSTICS`](json_diagnostics.md) - macro to control extended diagnostics
- [`materialize()`](../basic_json_view/materialize.md) - values of a JSON view, with the same positions
- [Parsing and Exceptions](../../features/parsing/parse_exceptions.md) - handling parse errors and locating them in the input
## Version history
+3 -2
View File
@@ -78,8 +78,9 @@ flowchart LR
- `NaN` and [discarded](../api/basic_json/is_discarded.md) values are not equal to themselves, so the invariant
cannot hold for them. `diff` emits a `replace` operation for such a value, and a `NaN` in a patch is serialized
as `#!json null`.
- Numbers that compare equal produce no operation even if their types differ: `diff(1, 1.0)`, `diff(1u, 1)`, and
`diff(0.0, -0.0)` return `#!json []`, so the patched value keeps the number type of *source*.
- Numbers that compare equal produce no operation even if their types differ or if one is zero and the other is
negative zero: `diff(1, 1.0)`, `diff(1u, 1)`, and `diff(0.0, -0.0)` return `#!json []`, so the patched value
keeps the number type and sign of *source*.
- A [binary value](binary_values.md) in a patch is serialized as an object
(`#!json {"bytes":[...],"subtype":null}`), so a patch that was dumped and parsed again no longer holds a binary
value.
+3 -5
View File
@@ -149,12 +149,10 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
end must still fit.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
is read to its end into a buffer the document owns, and parsing starts after that.
- **Fewer source positions after edits.** With
- **No source positions on `materialize()`.** Even with
[`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
[`materialize()`](../api/basic_json_view/materialize.md) sets the same positions as `#!cpp basic_json::parse()`.
In an [editable document](../api/json_editable_document.md), a value written by an edit has none, and neither has
an array or object whose elements changed or whose last element has none; a `basic_json` value keeps its positions
when it is changed. See [`materialize()`](../api/basic_json_view/materialize.md#notes).
[`materialize()`](../api/basic_json_view/materialize.md) does not set them. The index has the start offset of every
value, but not the end of arrays, objects, and escaped strings, which the positions also need.
- **Objects iterate in document order.** [`begin()`](../api/basic_json_view/begin.md)/
[`end()`](../api/basic_json_view/end.md) and [`items()`](../api/basic_json_view/items.md) visit an object's
members in the order they appear in the source text. `basic_json`'s default `object_t` is a `std::map`, which
@@ -130,6 +130,17 @@ parameters of [`parse()`](../../api/basic_json/parse.md) or [`accept()`](../../a
enable either for input whose conformance you cannot otherwise control, since interoperability with strictly
conforming JSON consumers is exactly what the default rejects.
## JSON Pointers and flattened keys
A [JSON Pointer](../json_pointer.md) can come from untrusted input as well, for instance as a path in a request or as a
key of a flattened object. The non-const [`operator[]`](../../api/basic_json/operator%5B%5D.md) with a JSON Pointer and
[`unflatten`](../../api/basic_json/unflatten.md) create missing values, and when an array index lies beyond the end of
an array, they fill the array with `#!json null` values up to that index. The memory used therefore grows with the
index, not with the size of the pointer: on a 64-bit platform, the index `10000000` allocates about 160 MB, the index
`100000000` about 1.6 GB. The library does not limit the index. For pointers from untrusted input, check the array
indices (or the pointer as a whole) before using them, or use [`at`](../../api/basic_json/at.md) or
[`contains`](../../api/basic_json/contains.md), which never add values.
## Checklist
- Wrap parsing in a `#!cpp try`/`#!cpp catch` block, or use `allow_exceptions=false`/`accept()` if your environment
@@ -145,6 +156,8 @@ conforming JSON consumers is exactly what the default rejects.
`JSON_STRICT_NUL_HANDLING`.
- Decide whether duplicate object keys should be an error for your application, and add a callback if so.
- Do not enable `ignore_comments` or `ignore_trailing_commas` for input that must be strictly conforming JSON.
- Check the array indices of untrusted JSON Pointers and flattened keys before passing them to the non-const
`operator[]` or `unflatten()`, which fill arrays with `#!json null` values up to the index.
For the broader design rationale and how it is tested (fuzzing, sanitizers, static analysis), see the
[assurance case](../../community/assurance_case.md) and [quality assurance](../../community/quality_assurance.md). To
+2 -7
View File
@@ -203,7 +203,7 @@ packet-beta
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their key hash index (1-based), or 0; otherwise 0 |
| 4-7 | `off` | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
| 8-11 | `len` | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself; strings with escapes: the position of the opening quote in the source plus one, or 0 (only with [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md)) |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself |
<p class="nj-table-caption" markdown>Fields of a node, by their byte offset within the 16-byte node. Integers use bytes 8-15 differently, see below.</p>
@@ -218,11 +218,6 @@ packet-beta
step from element to element this way and skip whole subtrees in constant time.
- **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
- **Source positions** ([`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md)) are derived by
[`materialize()`](../api/basic_json_view/materialize.md) and cost nothing while parsing: a scalar's range follows
from `off` and its length, a string with escapes is scanned from its opening quote (`next`) to the closing one,
and the end of an array or object is found by scanning from the end of its last element over whitespace, comments,
and a trailing comma. `materialize()` hands these positions to the SAX builder of `parse()` in place of its lexer.
- **Large objects** (128 members or more) get a hash index for their keys after parsing
([`detail/view/object_index.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/object_index.hpp)):
an open-addressing table whose slots hold the distance from the object's node to a key's node, so that a lookup does
@@ -282,7 +277,7 @@ Images ([`save`](../api/basic_json_document/save.md) and [`load`](../api/basic_j
the nodes as they are: a 64-byte header (the magic bytes `NJVI`, a format version, the sizes, and reserved bytes that
must be zero), the nodes, the text, and the decoded strings. An edited document is first written in document order, as
the parser would have written it (without links), and the numbers of the hash indexes are cleared, since `load`
rebuilds the indexes. The flag *new* is kept, as these values have no position in the text. So a change of the node layout is a change of the image format: it must raise `image_version`,
rebuilds the indexes. So a change of the node layout is a change of the image format: it must raise `image_version`,
and `load` then rejects images of other versions (`parse_error.116`) instead of misreading them.
## Input adapters
+3 -22
View File
@@ -230,10 +230,7 @@ struct diagnostic_positions
{
// Lexer has read the first character of the container, so
// subtract 1 from the position to get the correct start position.
// (npos: a value without a source position, see materialize() of
// json_view)
const std::size_t read = lexer->get_chars_read_total();
v.start_position = read == std::string::npos ? read : read - 1;
v.start_position = lexer->get_chars_read_total() - 1;
}
}
@@ -248,12 +245,6 @@ struct diagnostic_positions
{
// Lexer's position is past the closing brace or bracket, so set that as the end position.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a range has both ends or none (exception::diagnostics()
// prints only complete ones)
v.start_position = v.end_position;
}
}
}
@@ -270,12 +261,6 @@ struct diagnostic_positions
// The start position will be set below based on the length of the string representation
// of the value.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a value without a source position
v.start_position = v.end_position;
return;
}
switch (v.type())
{
@@ -488,12 +473,8 @@ read up to it.
@tparam Collect whether to collect the values of containers of unknown size on
a stack (only for a caller that does not use the value after a
parse error)
@tparam LexerType the type of the lexer that reports the source positions
(JSON_DIAGNOSTIC_POSITIONS); only diagnostic_positions uses
it, so any type with its three functions works (as
materialize() of json_view uses it)
*/
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false, typename LexerType = lexer<BasicJsonType, InputAdapterType>>
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false>
class json_sax_dom_parser : public json_sax_dom_error_state
{
public:
@@ -502,7 +483,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using binary_t = typename BasicJsonType::binary_t;
using lexer_t = LexerType;
using lexer_t = lexer<BasicJsonType, InputAdapterType>;
/*!
@param[in,out] r reference to a JSON value that is manipulated while
+2 -2
View File
@@ -56,8 +56,8 @@ using parser_callback_t =
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect, LexerType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
+485 -30
View File
@@ -8,7 +8,7 @@
#pragma once
#include <algorithm> // all_of, copy
#include <algorithm> // copy
#include <array> // array
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
@@ -16,9 +16,11 @@
#include <cstdint> // uint64_t
#include <cstdlib> // strtoull
#include <cstring> // memchr, memcmp, memcpy
#include <functional> // less
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
#include <iterator> // next
#include <limits> // max
#include <string> // string
#include <utility> // move
@@ -28,6 +30,7 @@
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/cpp_future.hpp>
#include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/detail/meta/void_t.hpp>
#include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_escape.hpp>
#include <nlohmann/detail/string_utils.hpp>
@@ -134,6 +137,26 @@ JSON_HEDLEY_ALWAYS_INLINE array_index_status parse_array_index(const StringType&
return array_index_status::ok;
}
/// whether two keys of type @a StringType are equivalent under @a Compare
/// exactly if they are equal: std::less<StringType> and std::less<>
template<typename Compare, typename StringType>
struct is_less_comparator : std::false_type {};
template<typename StringType>
struct is_less_comparator<std::less<StringType>, StringType> : std::true_type {};
template<typename StringType>
struct is_less_comparator<std::less<void>, StringType> : std::true_type {};
/// whether the objects of type @a ObjectType find a key of type @a StringType
/// exactly if it equals a stored key
template<typename ObjectType, typename StringType, typename = void>
struct finds_equal_keys : std::false_type {};
template<typename ObjectType, typename StringType>
struct finds_equal_keys<ObjectType, StringType, void_t<typename ObjectType::key_compare, typename ObjectType::key_type>>
: conjunction<is_less_comparator<typename ObjectType::key_compare, StringType>, std::is_same<typename ObjectType::key_type, StringType>> {};
} // namespace detail
/// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
@@ -589,6 +612,225 @@ class json_pointer
return object_member_impl(object, key, detail::priority_tag<1> {});
}
/// @brief append a null value to @a array, like operator[] with the size of @a array
template<typename BasicJsonType>
static BasicJsonType& array_append(BasicJsonType& array)
{
auto& elements = *array.m_data.m_value.array;
const auto old_capacity = array.container_capacity();
elements.emplace_back();
return array.set_parent_after_insert(elements.back(), old_capacity);
}
/// @brief add @a key, which is missing, to @a object
template<typename BasicJsonType>
static auto object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace(key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto it = object.m_data.m_value.object->emplace(key, nullptr).first;
return object.set_parent_after_insert(it->second, old_capacity);
}
template<typename BasicJsonType>
static BasicJsonType& object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object.operator[](key);
}
template<typename BasicJsonType>
static BasicJsonType& object_add(BasicJsonType& object, const string_t& key)
{
return object_add_impl(object, key, detail::priority_tag<1> {});
}
/// @brief search @a key in @a members: the result is where the key is or,
/// for objects that can insert at a position (std::map), where it belongs
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, members.lower_bound(key)->first), members.lower_bound(key))
{
return members.lower_bound(key);
}
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<0> /*unused*/)
-> decltype(members.find(key))
{
return members.find(key);
}
template<typename ObjectType>
static auto object_search(ObjectType& members, const string_t& key)
-> decltype(object_search_impl(members, key, detail::priority_tag<1> {}))
{
return object_search_impl(members, key, detail::priority_tag<1> {});
}
/// @brief whether @a it, the result of @ref object_search for @a key, points to @a key
///
/// With std::less, the key at the lower bound is equivalent to @a key
/// exactly if it is equal to it, and comparing for equality is cheaper.
template<typename ObjectType, typename Iterator,
detail::enable_if_t<detail::finds_equal_keys<ObjectType, string_t>::value, int> = 0>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<2> /*unused*/)
-> decltype(members.lower_bound(key), it->first == key)
{
return it != members.end() && it->first == key;
}
template<typename ObjectType, typename Iterator>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, it->first), members.lower_bound(key), bool())
{
return it != members.end() && !members.key_comp()(key, it->first);
}
template<typename ObjectType, typename Iterator>
static bool object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& /*key*/, detail::priority_tag<0> /*unused*/)
{
return it != members.end();
}
template<typename ObjectType, typename Iterator>
static bool object_search_found(ObjectType& members, const Iterator& it, const string_t& key)
{
return object_search_found_impl(members, it, key, detail::priority_tag<2> {});
}
/// @brief insert @a key, which is missing, into @a object at @a it, the
/// result of @ref object_search for @a key
template<typename BasicJsonType, typename Iterator>
static auto object_insert_impl(BasicJsonType& object, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace_hint(it, key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto pos = object.m_data.m_value.object->emplace_hint(it, key, nullptr);
return object.set_parent_after_insert(pos->second, old_capacity);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert_impl(BasicJsonType& object, const Iterator& /*it*/, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object_add(object, key);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert(BasicJsonType& object, const Iterator& it, const string_t& key)
{
return object_insert_impl(object, it, key, detail::priority_tag<1> {});
}
/*!
@brief undoes the change @ref get_unchecked_create made to an existing value
@ref get_unchecked walks existing values without changing them. At the
first reference token that does not select an existing value, @ref
get_unchecked_create makes exactly one change to an existing value: it
converts a null value, inserts a key into an object, or grows an array.
Everything it creates afterwards lies below that change. If a later step
throws, the destructor of this guard reverts that one change, which
removes everything created after it, so the value is left as it was
(strong exception safety). The guard needs no try/catch and therefore also
compiles with JSON_NOEXCEPTION.
Reverting the change must not throw, as it happens in a destructor. It
erases the inserted member by its iterator (no key is converted or
compared, nothing is allocated), erases the added elements at the end of
the array, or swaps the null value back. This holds for std::map,
std::vector, ordered_map, and every object_t, array_t, and
json_base_class_t whose erase() and move operations do not throw.
Between recording and reverting, nothing is inserted into or erased from
the object or array, so the iterator stays valid.
*/
template<typename BasicJsonType>
class get_unchecked_rollback
{
public:
using object_iterator = typename BasicJsonType::object_t::iterator;
/// record that @a member was inserted into the object @a value
get_unchecked_rollback(BasicJsonType* value, const object_iterator& member)
: m_value(value)
, m_member(member)
, m_change(change::key_inserted)
{}
/// record the change that the next reference token makes to the null value or array @a value
explicit get_unchecked_rollback(BasicJsonType* value)
: m_value(value)
{
if (value->is_null())
{
// keep the null value (with its base class and positions) and
// continue with a fresh null value in its place, which is what
// assigning an array or object to it would have left there
m_saved_null.swap(*value);
m_change = change::null_converted;
}
else
{
JSON_ASSERT(value->is_array());
m_old_size = value->m_data.m_value.array->size();
m_change = change::array_grown;
}
}
get_unchecked_rollback(const get_unchecked_rollback&) = delete;
get_unchecked_rollback(get_unchecked_rollback&&) = delete;
get_unchecked_rollback& operator=(const get_unchecked_rollback&) = delete;
get_unchecked_rollback& operator=(get_unchecked_rollback&&) = delete;
~get_unchecked_rollback()
{
undo();
}
/// keep the change
void commit() noexcept
{
m_change = change::none;
}
private:
void undo() noexcept
{
switch (m_change)
{
case change::null_converted:
// the values created since go away with m_saved_null
m_value->swap(m_saved_null);
break;
case change::key_inserted:
m_value->m_data.m_value.object->erase(m_member);
break;
case change::array_grown:
{
auto& elements = *m_value->m_data.m_value.array;
elements.resize(m_old_size);
break;
}
case change::none:
default:
break;
}
}
enum class change
{
none,
null_converted,
key_inserted,
array_grown
};
BasicJsonType* m_value;
object_iterator m_member {};
typename BasicJsonType::size_type m_old_size = 0;
change m_change = change::none;
BasicJsonType m_saved_null {};
};
JSON_PRIVATE_UNLESS_TESTED:
json_pointer top() const
{
@@ -972,47 +1214,245 @@ class json_pointer
Complexity: Linear in the length of the JSON pointer.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved
Exception safety: Strong guarantee. This function walks existing values
until the first reference token that does not select one. If that is the
last reference token, inserting the key into the object or growing the
array is strongly exception safe by itself; any other change is made by
@ref get_unchecked_create, which undoes it if a later step throws.
@throw parse_error.106 if an array index of an existing array begins with '0'
@throw parse_error.109 if an array index of an existing array was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved, or if an
array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
BasicJsonType& get_unchecked(BasicJsonType* ptr) const
{
for (const auto& reference_token : reference_tokens)
const auto* token = reference_tokens.data();
const auto* const end = token + reference_tokens.size();
if (token == end)
{
return *ptr;
}
if (ptr->is_null())
{
// the null value is converted to an array or an object
return get_unchecked_create(ptr, token, end);
}
// walk existing values; the walk itself inserts nothing, which keeps
// the loop fast (insertions are done below or in get_unchecked_create)
typename BasicJsonType::object_t::iterator it;
do
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
// the key is inserted below or in get_unchecked_create
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
if (token + 1 == end)
{
// explicitly treat "-" as index beyond the end
return array_append(*ptr);
}
// an element will be appended in get_unchecked_create
break;
}
if (token + 1 == end)
{
// convert array index to number; unchecked access
return ptr->operator[](array_index<BasicJsonType>(*token));
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
// the array will be padded in get_unchecked_create
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
// the null value will be converted in get_unchecked_create
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
while (++token != end);
if (token == end)
{
return *ptr;
}
if (token + 1 == end && ptr->is_object())
{
// insert the missing key where the walk found it belongs
return object_insert(*ptr, it, *token);
}
return get_unchecked_create(ptr, token, end);
}
/*!
@brief create the values for the reference tokens from @a first on
Continues @ref get_unchecked at the first reference token that does not
select an existing value. That token changes @a ptr, an existing value (see
@ref get_unchecked_rollback); all later tokens only change values created
here.
If the emplace() of the object type finds a member that the lookup in @ref
get_unchecked did not find (the two disagree, e.g. for fifo_map), nothing
is inserted, and the walk continues with that member as in @ref
get_unchecked, which is what basic_json::operator[] does.
@param[in] ptr the value the reference token @a first is applied to
@param[in] first the first reference token to resolve (an element of
@ref reference_tokens)
@param[in] end the end of @ref reference_tokens
@throw parse_error.106 if an array index of an existing array begins with
'0' (only after emplace() found a member, see above)
@throw parse_error.109 if an array index of an existing array was not a
number (only after emplace() found a member)
@throw out_of_range.404 if an array index is too large for strtoull, or (only
after emplace() found a member) if the JSON pointer
can not be resolved
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
JSON_HEDLEY_NEVER_INLINE static BasicJsonType& get_unchecked_create(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
const auto* token = first;
while (ptr->is_object())
{
const auto old_capacity = ptr->container_capacity();
const auto inserted = ptr->m_data.m_value.object->emplace(*token, nullptr);
if (JSON_HEDLEY_LIKELY(inserted.second))
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, inserted.first);
BasicJsonType& result = create_values(&ptr->set_parent_after_insert(inserted.first->second, old_capacity), token + 1, end);
rollback.commit();
return result;
}
// nothing was inserted: walk existing values from the member
// emplace() found on, like get_unchecked
ptr = &inserted.first->second;
while (++token != end)
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
const auto it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
break;
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
if (token == end)
{
return *ptr;
}
}
get_unchecked_rollback<BasicJsonType> rollback(ptr);
BasicJsonType& result = create_values(ptr, token, end);
rollback.commit();
return result;
}
/*!
@brief the loop of @ref get_unchecked_create
@a ptr is a null value or an array. Every value created here is a fresh
one, so every object a key is added to is empty.
@throw out_of_range.404 if an array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
static BasicJsonType& create_values(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
for (const auto* token = first; token != end; ++token)
{
const auto& reference_token = *token;
// convert null values to arrays or objects before continuing
if (ptr->is_null())
{
// check if the reference token is a valid array index, that is
// a nonempty sequence of digits without a leading '0'
// (cf. RFC 6901, Sect. 4); tokens that could never be a valid
// array index (such as "01" or "") are treated as object keys
const bool nums = !reference_token.empty()
&& (reference_token.size() == 1 || reference_token[0] != '0')
&& std::all_of(reference_token.begin(), reference_token.end(),
[](const unsigned char x)
// change value to an array for array indices or "-" or to an
// object otherwise; every null value here is a fresh one (see
// get_unchecked_rollback), so converting it in place leaves
// the same value as assigning an array or object to it
if (is_array_index_token(reference_token) || is_dash(reference_token))
{
return std::isdigit(x);
});
// change value to an array for array indices or "-" or to object otherwise
*ptr = (nums || reference_token == "-")
? detail::value_t::array
: detail::value_t::object;
ptr->template convert_null_to<typename BasicJsonType::array_t>();
}
else
{
ptr->template convert_null_to<typename BasicJsonType::object_t>();
}
}
if (ptr->is_object())
{
// use unchecked object access
ptr = &object_member(*ptr, reference_token);
// the object is new and empty
ptr = &object_add(*ptr, reference_token);
}
else if (ptr->is_array())
else
{
if (reference_token == "-")
// only the first value is not created here, and it is a null
// value or an array (see get_unchecked_create)
JSON_ASSERT(ptr->is_array());
if (is_dash(reference_token))
{
// explicitly treat "-" as index beyond the end
ptr = &ptr->operator[](ptr->m_data.m_value.array->size());
ptr = &array_append(*ptr);
}
else
{
@@ -1020,16 +1460,31 @@ class json_pointer
ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
}
}
else
{
// primitive value with a reference token left
throw_unresolved(reference_token, ptr);
}
}
return *ptr;
}
/// whether @a s is a valid array index, that is a nonempty sequence of
/// digits without a leading '0' (cf. RFC 6901, Sect. 4); tokens that could
/// never be a valid array index (such as "01" or "") are object keys
static bool is_array_index_token(const string_t& s)
{
const std::size_t size = s.size();
if (size == 0 || (size > 1 && s[0] == '0'))
{
return false;
}
for (std::size_t i = 0; i < size; ++i)
{
if (std::isdigit(static_cast<unsigned char>(s[i])) == 0)
{
return false;
}
}
return true;
}
/*!
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
-6
View File
@@ -1193,13 +1193,7 @@ indent_done:
return false;
}
p = r.p;
#if JSON_DIAGNOSTIC_POSITIONS
// the decoded string is in the arena: next keeps the source offset
// of the opening quote plus one (0: unknown) for materialize()
push_node(node_kind::string, node_flags::escaped, 0, r.start, static_cast<std::uint64_t>(r.len) | (static_cast<std::uint64_t>(s - b) << 32));
#else
push_node(node_kind::string, node_flags::escaped, 0, r.start, r.len);
#endif
return true;
}
};
+12 -18
View File
@@ -80,8 +80,7 @@ constexpr std::uint64_t image_limit = 0xFFFFFFF0u;
/// Copy the current structure of an edited document into nodes in document
/// order, as the parser would have written them. Text written by edits is
/// appended to text_tail (number tokens) and arena_tail (strings); floats that
/// are not finite become null, as dump() writes them. Nodes keep the flag
/// is_new: they have no source position (JSON_DIAGNOSTIC_POSITIONS).
/// are not finite become null, as dump() writes them.
inline void compact_nodes(const document_data& d, std::size_t arena_size, std::vector<node>& out, std::string& text_tail, std::string& arena_tail)
{
struct frame
@@ -104,18 +103,16 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
arena_tail.append(d.str(s), s.len);
r.flags = node_flags::escaped;
}
r.flags = static_cast<std::uint8_t>(r.flags | (s.flags & node_flags::is_new));
return r;
};
const auto copy_node = [&](const node * v)
{
node r = *v;
const auto is_new = static_cast<std::uint8_t>(v->flags & node_flags::is_new);
switch (static_cast<value_t>(v->kind))
{
case value_t::object:
case value_t::array:
r.flags = is_new;
r.flags = 0;
r.extra = 0;
r.off = (v->flags & (node_flags::moved | node_flags::is_new)) != 0 ? 0 : v->off;
r.len = 0; // counted below
@@ -132,7 +129,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(d.str(*v), number_length(*v));
}
r.flags = is_new;
r.flags = 0;
break;
case value_t::number_float:
if ((v->flags & node_flags::storage) == node_flags::edited)
@@ -142,23 +139,22 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
{
r = node{}; // nan and infinity: null, as dump() writes them
r.kind = node_kind::null;
r.flags = is_new;
break;
}
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(t, v->len);
r.extra = 0xFFFFu; // the digit layout is not recorded
}
r.flags = is_new;
r.flags = 0;
break;
case value_t::boolean:
r.flags = static_cast<std::uint8_t>(v->flags & (node_flags::is_true | node_flags::is_new));
r.flags = static_cast<std::uint8_t>(v->flags & node_flags::is_true);
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
r.flags = is_new;
r.flags = 0;
break;
}
out.push_back(r);
@@ -519,15 +515,13 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
std::vector<float_range> floats;
const auto check_string = [&](const node & n) -> bool
{
if ((n.flags & ~(node_flags::escaped | node_flags::is_new)) != 0 || n.extra != 0)
if ((n.flags & ~node_flags::escaped) != 0 || n.extra != 0)
{
return false;
}
const bool decoded = (n.flags & node_flags::escaped) != 0;
const std::size_t limit = decoded ? arena_size : text_size;
// (next: the source offset of a decoded string's opening quote plus
// one, or 0; see node)
if (n.off > limit || n.len > limit - n.off || n.next > text_size)
if (n.off > limit || n.len > limit - n.off)
{
return false;
}
@@ -599,13 +593,13 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
{
case value_t::null:
// (the offset of a literal is read to size the output of dump())
if ((n.flags & ~node_flags::is_new) != 0 || n.extra != 0 || n.off > text_size)
if (n.flags != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::boolean:
if ((n.flags & ~(node_flags::is_true | node_flags::is_new)) != 0 || n.extra != 0 || n.off > text_size)
if ((n.flags & ~node_flags::is_true) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
@@ -619,7 +613,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if ((n.flags & ~node_flags::is_new) != 0 || !number_in_bounds(n))
if (n.flags != 0 || !number_in_bounds(n))
{
return false;
}
@@ -641,7 +635,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
case value_t::object:
{
const std::size_t limit = stack.empty() ? count : stack.back().end;
if ((n.flags & ~node_flags::is_new) != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
if (n.flags != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
{
return false;
}
+4 -225
View File
@@ -26,176 +26,6 @@ namespace detail
namespace view
{
#if JSON_DIAGNOSTIC_POSITIONS
/*!
@brief the source positions of the values materialize() replays
Stands in for the lexer of json_sax_dom_parser, which reads the positions of
the values it stores from these three functions (diagnostic_positions). npos
is a value without a source position.
*/
struct position_source
{
/// the token, of which diagnostic_positions only reads the length
struct token
{
std::size_t length;
std::size_t size() const noexcept
{
return length;
}
};
std::size_t start = std::string::npos; ///< the first byte of the next value
std::size_t end = std::string::npos; ///< one past the next value (start_object/start_array: one past the bracket)
std::size_t get_chars_read_total() const noexcept
{
return end;
}
std::size_t get_token_start_position() const noexcept
{
return start;
}
token get_string() const noexcept
{
return token{end - start};
}
/// the range [first, last), or none if either is npos
void set(std::size_t first, std::size_t last) noexcept
{
const bool known = first != std::string::npos && last != std::string::npos;
start = known ? first : std::string::npos;
end = known ? last : std::string::npos;
}
/// one past the closing quote of the string whose opening quote is at
/// src[quote], or npos (the scan is bounded by size, as images can be
/// loaded without a check)
static std::size_t string_end(const char* src, std::size_t size, std::size_t quote) noexcept
{
for (std::size_t i = quote + 1; i < size; ++i)
{
if (src[i] == '\\')
{
++i;
}
else if (src[i] == '"')
{
return i + 1;
}
}
return std::string::npos;
}
/// one past the closing bracket of the array or object whose last element
/// ends at src[pos] (or whose opening bracket is at src[pos - 1]), or npos;
/// in between, there can only be whitespace, comments, and a trailing comma
/// (the source was validated when it was parsed)
static std::size_t container_end(const char* src, std::size_t size, std::size_t pos) noexcept
{
while (pos < size)
{
switch (src[pos])
{
case ' ':
case '\t':
case '\n':
case '\r':
case ',':
++pos;
break;
case ']':
case '}':
return pos + 1;
case '/':
if (pos + 1 < size && src[pos + 1] == '/')
{
pos += 2;
while (pos < size && src[pos] != '\n' && src[pos] != '\r')
{
++pos;
}
break;
}
if (pos + 1 < size && src[pos + 1] == '*')
{
pos += 2;
while (pos < size && !(src[pos] == '*' && pos + 1 < size && src[pos + 1] == '/'))
{
++pos;
}
pos += 2;
break;
}
return std::string::npos;
default:
return std::string::npos;
}
}
return std::string::npos;
}
/// the range of the scalar n of document d
void set_scalar(const document_data& d, const node& n) noexcept
{
const std::size_t size = d.size;
const std::size_t off = n.off;
std::size_t first = std::string::npos;
std::size_t last = std::string::npos;
if ((n.flags & node_flags::is_new) == 0)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::string:
if ((n.flags & node_flags::storage) == 0)
{
// off is the first byte after the opening quote
if (off != 0)
{
first = off - 1;
last = off + std::size_t{n.len} + 1;
}
}
else if ((n.flags & node_flags::storage) == node_flags::escaped && n.next != 0)
{
// next is the offset of the opening quote plus one
first = n.next - 1;
last = string_end(d.base[0], size, first);
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if ((n.flags & node_flags::storage) == 0)
{
first = off;
last = off + number_length(n);
}
break;
case value_t::boolean:
first = off;
last = off + ((n.flags & node_flags::is_true) != 0 ? 4u : 5u);
break;
case value_t::null:
first = off;
last = off + 4u;
break;
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
break;
}
}
// (a crafted image can point past the text)
set(first, last != std::string::npos && last <= size ? last : std::string::npos);
}
};
#endif
/*!
@brief the basic_json value of the subtree at n
@@ -203,22 +33,14 @@ The subtree is replayed into the SAX handler that parse() uses to build its
values, so the result is the value parse() would produce: duplicate keys keep
the last value, and with JSON_DIAGNOSTICS the parent pointers are set. It is
iterative, so the nesting depth is limited by memory only, as for parse().
With JSON_DIAGNOSTIC_POSITIONS, the values get their source positions as with
parse(): position_source stands in for the lexer. Values written by edits have
none (npos), and neither has an array or object whose elements changed, or
whose last element has none (the end of its closing bracket is found by
scanning from the end of the last element).
Without a lexer the handler records no source positions
(JSON_DIAGNOSTIC_POSITIONS).
*/
template<typename BasicJsonType, bool Editable>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
#if JSON_DIAGNOSTIC_POSITIONS
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>, true, position_source>;
#else
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>, true>;
#endif
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
@@ -230,15 +52,9 @@ BasicJsonType materialize(const document_data& d, const node* n)
const node* pos;
const node* end;
bool object;
#if JSON_DIAGNOSTIC_POSITIONS
std::size_t start; ///< the opening bracket (npos: unknown)
std::size_t last_end; ///< the end of the last element read (npos: unknown)
#endif
};
#if !JSON_DIAGNOSTIC_POSITIONS
// a scalar needs neither the handler nor the stack of open containers
// (with positions, it goes through the handler, which sets them)
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
@@ -260,16 +76,10 @@ BasicJsonType materialize(const document_data& d, const node* n)
default:
break;
}
#endif
BasicJsonType result;
#if JSON_DIAGNOSTIC_POSITIONS
position_source positions;
sax_t sax(result, true, &positions);
#else
sax_t sax(result, true);
#endif
const string_t no_token {};
const string_t no_token{};
// (shallow subtrees on the stack: a call for a small value then allocates only for the value)
std::array<frame, 16> shallow; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled as containers open
std::vector<frame> deep{};
@@ -284,16 +94,6 @@ BasicJsonType materialize(const document_data& d, const node* n)
std::size_t depth = 0;
for (;;)
{
#if JSON_DIAGNOSTIC_POSITIONS
if (!is_container(*n))
{
positions.set_scalar(d, *n);
if (depth != 0)
{
open[depth - 1].last_end = positions.end;
}
}
#endif
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
@@ -302,12 +102,6 @@ BasicJsonType materialize(const document_data& d, const node* n)
case value_t::array:
{
const bool object = n->kind == node_kind::object;
#if JSON_DIAGNOSTIC_POSITIONS
// (moved containers are also new)
const std::size_t start = (n->flags & node_flags::is_new) == 0 && n->off < d.size ? std::size_t{n->off} :
std::string::npos;
positions.set(start, start + 1);
#endif
if (object)
{
sax.start_object(n->len);
@@ -324,14 +118,8 @@ BasicJsonType materialize(const document_data& d, const node* n)
open = deep.data();
capacity = deep.size();
}
#if JSON_DIAGNOSTIC_POSITIONS
// (until an element is read, the container's own bracket is the last thing read)
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object, start, positions.end};
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
break;
#else
open[depth++] = frame {nav::first(d, n), nav::end(d, n), object};
break;
#endif
}
case value_t::string:
{
@@ -371,15 +159,6 @@ BasicJsonType materialize(const document_data& d, const node* n)
frame& f = open[depth - 1];
if (f.pos == f.end)
{
#if JSON_DIAGNOSTIC_POSITIONS
const std::size_t end = f.start == std::string::npos || f.last_end == std::string::npos ? std::string::npos : position_source::container_end(d.base[0], d.size, f.last_end);
// (an end of npos clears the start, see diagnostic_positions)
positions.set(end == std::string::npos ? end : end - 1, end);
if (depth > 1)
{
open[depth - 2].last_end = end;
}
#endif
if (f.object)
{
sax.end_object();
+1 -1
View File
@@ -69,7 +69,7 @@ struct node
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the object's key hash index (1-based, 0 = none); otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if escaped/edited; number of the element sequence if moved
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence); escaped string: source offset of the opening quote plus one (JSON_DIAGNOSTIC_POSITIONS; 0: unknown)
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");
+1 -1
View File
@@ -159,7 +159,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader;
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
friend class ::nlohmann::detail::json_sax_dom_parser;
template<typename BasicJsonType, typename InputAdapterType>
friend class ::nlohmann::detail::json_sax_dom_binary_parser;
+492 -55
View File
@@ -15629,10 +15629,7 @@ struct diagnostic_positions
{
// Lexer has read the first character of the container, so
// subtract 1 from the position to get the correct start position.
// (npos: a value without a source position, see materialize() of
// json_view)
const std::size_t read = lexer->get_chars_read_total();
v.start_position = read == std::string::npos ? read : read - 1;
v.start_position = lexer->get_chars_read_total() - 1;
}
}
@@ -15647,12 +15644,6 @@ struct diagnostic_positions
{
// Lexer's position is past the closing brace or bracket, so set that as the end position.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a range has both ends or none (exception::diagnostics()
// prints only complete ones)
v.start_position = v.end_position;
}
}
}
@@ -15669,12 +15660,6 @@ struct diagnostic_positions
// The start position will be set below based on the length of the string representation
// of the value.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a value without a source position
v.start_position = v.end_position;
return;
}
switch (v.type())
{
@@ -15887,12 +15872,8 @@ read up to it.
@tparam Collect whether to collect the values of containers of unknown size on
a stack (only for a caller that does not use the value after a
parse error)
@tparam LexerType the type of the lexer that reports the source positions
(JSON_DIAGNOSTIC_POSITIONS); only diagnostic_positions uses
it, so any type with its three functions works (as
materialize() of json_view uses it)
*/
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false, typename LexerType = lexer<BasicJsonType, InputAdapterType>>
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false>
class json_sax_dom_parser : public json_sax_dom_error_state
{
public:
@@ -15901,7 +15882,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using binary_t = typename BasicJsonType::binary_t;
using lexer_t = LexerType;
using lexer_t = lexer<BasicJsonType, InputAdapterType>;
/*!
@param[in,out] r reference to a JSON value that is manipulated while
@@ -22534,8 +22515,8 @@ using parser_callback_t =
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect, LexerType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
@@ -24083,7 +24064,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <algorithm> // all_of, copy
#include <algorithm> // copy
#include <array> // array
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
@@ -24091,9 +24072,11 @@ NLOHMANN_JSON_NAMESPACE_END
#include <cstdint> // uint64_t
#include <cstdlib> // strtoull
#include <cstring> // memchr, memcmp, memcpy
#include <functional> // less
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
#include <iterator> // next
#include <limits> // max
#include <string> // string
#include <utility> // move
@@ -24107,6 +24090,8 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/meta/type_traits.hpp>
// #include <nlohmann/detail/meta/void_t.hpp>
// #include <nlohmann/detail/string_concat.hpp>
// #include <nlohmann/detail/string_escape.hpp>
@@ -24217,6 +24202,26 @@ JSON_HEDLEY_ALWAYS_INLINE array_index_status parse_array_index(const StringType&
return array_index_status::ok;
}
/// whether two keys of type @a StringType are equivalent under @a Compare
/// exactly if they are equal: std::less<StringType> and std::less<>
template<typename Compare, typename StringType>
struct is_less_comparator : std::false_type {};
template<typename StringType>
struct is_less_comparator<std::less<StringType>, StringType> : std::true_type {};
template<typename StringType>
struct is_less_comparator<std::less<void>, StringType> : std::true_type {};
/// whether the objects of type @a ObjectType find a key of type @a StringType
/// exactly if it equals a stored key
template<typename ObjectType, typename StringType, typename = void>
struct finds_equal_keys : std::false_type {};
template<typename ObjectType, typename StringType>
struct finds_equal_keys<ObjectType, StringType, void_t<typename ObjectType::key_compare, typename ObjectType::key_type>>
: conjunction<is_less_comparator<typename ObjectType::key_compare, StringType>, std::is_same<typename ObjectType::key_type, StringType>> {};
} // namespace detail
/// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
@@ -24672,6 +24677,225 @@ class json_pointer
return object_member_impl(object, key, detail::priority_tag<1> {});
}
/// @brief append a null value to @a array, like operator[] with the size of @a array
template<typename BasicJsonType>
static BasicJsonType& array_append(BasicJsonType& array)
{
auto& elements = *array.m_data.m_value.array;
const auto old_capacity = array.container_capacity();
elements.emplace_back();
return array.set_parent_after_insert(elements.back(), old_capacity);
}
/// @brief add @a key, which is missing, to @a object
template<typename BasicJsonType>
static auto object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace(key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto it = object.m_data.m_value.object->emplace(key, nullptr).first;
return object.set_parent_after_insert(it->second, old_capacity);
}
template<typename BasicJsonType>
static BasicJsonType& object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object.operator[](key);
}
template<typename BasicJsonType>
static BasicJsonType& object_add(BasicJsonType& object, const string_t& key)
{
return object_add_impl(object, key, detail::priority_tag<1> {});
}
/// @brief search @a key in @a members: the result is where the key is or,
/// for objects that can insert at a position (std::map), where it belongs
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, members.lower_bound(key)->first), members.lower_bound(key))
{
return members.lower_bound(key);
}
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<0> /*unused*/)
-> decltype(members.find(key))
{
return members.find(key);
}
template<typename ObjectType>
static auto object_search(ObjectType& members, const string_t& key)
-> decltype(object_search_impl(members, key, detail::priority_tag<1> {}))
{
return object_search_impl(members, key, detail::priority_tag<1> {});
}
/// @brief whether @a it, the result of @ref object_search for @a key, points to @a key
///
/// With std::less, the key at the lower bound is equivalent to @a key
/// exactly if it is equal to it, and comparing for equality is cheaper.
template<typename ObjectType, typename Iterator,
detail::enable_if_t<detail::finds_equal_keys<ObjectType, string_t>::value, int> = 0>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<2> /*unused*/)
-> decltype(members.lower_bound(key), it->first == key)
{
return it != members.end() && it->first == key;
}
template<typename ObjectType, typename Iterator>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, it->first), members.lower_bound(key), bool())
{
return it != members.end() && !members.key_comp()(key, it->first);
}
template<typename ObjectType, typename Iterator>
static bool object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& /*key*/, detail::priority_tag<0> /*unused*/)
{
return it != members.end();
}
template<typename ObjectType, typename Iterator>
static bool object_search_found(ObjectType& members, const Iterator& it, const string_t& key)
{
return object_search_found_impl(members, it, key, detail::priority_tag<2> {});
}
/// @brief insert @a key, which is missing, into @a object at @a it, the
/// result of @ref object_search for @a key
template<typename BasicJsonType, typename Iterator>
static auto object_insert_impl(BasicJsonType& object, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace_hint(it, key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto pos = object.m_data.m_value.object->emplace_hint(it, key, nullptr);
return object.set_parent_after_insert(pos->second, old_capacity);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert_impl(BasicJsonType& object, const Iterator& /*it*/, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object_add(object, key);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert(BasicJsonType& object, const Iterator& it, const string_t& key)
{
return object_insert_impl(object, it, key, detail::priority_tag<1> {});
}
/*!
@brief undoes the change @ref get_unchecked_create made to an existing value
@ref get_unchecked walks existing values without changing them. At the
first reference token that does not select an existing value, @ref
get_unchecked_create makes exactly one change to an existing value: it
converts a null value, inserts a key into an object, or grows an array.
Everything it creates afterwards lies below that change. If a later step
throws, the destructor of this guard reverts that one change, which
removes everything created after it, so the value is left as it was
(strong exception safety). The guard needs no try/catch and therefore also
compiles with JSON_NOEXCEPTION.
Reverting the change must not throw, as it happens in a destructor. It
erases the inserted member by its iterator (no key is converted or
compared, nothing is allocated), erases the added elements at the end of
the array, or swaps the null value back. This holds for std::map,
std::vector, ordered_map, and every object_t, array_t, and
json_base_class_t whose erase() and move operations do not throw.
Between recording and reverting, nothing is inserted into or erased from
the object or array, so the iterator stays valid.
*/
template<typename BasicJsonType>
class get_unchecked_rollback
{
public:
using object_iterator = typename BasicJsonType::object_t::iterator;
/// record that @a member was inserted into the object @a value
get_unchecked_rollback(BasicJsonType* value, const object_iterator& member)
: m_value(value)
, m_member(member)
, m_change(change::key_inserted)
{}
/// record the change that the next reference token makes to the null value or array @a value
explicit get_unchecked_rollback(BasicJsonType* value)
: m_value(value)
{
if (value->is_null())
{
// keep the null value (with its base class and positions) and
// continue with a fresh null value in its place, which is what
// assigning an array or object to it would have left there
m_saved_null.swap(*value);
m_change = change::null_converted;
}
else
{
JSON_ASSERT(value->is_array());
m_old_size = value->m_data.m_value.array->size();
m_change = change::array_grown;
}
}
get_unchecked_rollback(const get_unchecked_rollback&) = delete;
get_unchecked_rollback(get_unchecked_rollback&&) = delete;
get_unchecked_rollback& operator=(const get_unchecked_rollback&) = delete;
get_unchecked_rollback& operator=(get_unchecked_rollback&&) = delete;
~get_unchecked_rollback()
{
undo();
}
/// keep the change
void commit() noexcept
{
m_change = change::none;
}
private:
void undo() noexcept
{
switch (m_change)
{
case change::null_converted:
// the values created since go away with m_saved_null
m_value->swap(m_saved_null);
break;
case change::key_inserted:
m_value->m_data.m_value.object->erase(m_member);
break;
case change::array_grown:
{
auto& elements = *m_value->m_data.m_value.array;
elements.resize(m_old_size);
break;
}
case change::none:
default:
break;
}
}
enum class change
{
none,
null_converted,
key_inserted,
array_grown
};
BasicJsonType* m_value;
object_iterator m_member {};
typename BasicJsonType::size_type m_old_size = 0;
change m_change = change::none;
BasicJsonType m_saved_null {};
};
JSON_PRIVATE_UNLESS_TESTED:
json_pointer top() const
{
@@ -25055,47 +25279,245 @@ class json_pointer
Complexity: Linear in the length of the JSON pointer.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved
Exception safety: Strong guarantee. This function walks existing values
until the first reference token that does not select one. If that is the
last reference token, inserting the key into the object or growing the
array is strongly exception safe by itself; any other change is made by
@ref get_unchecked_create, which undoes it if a later step throws.
@throw parse_error.106 if an array index of an existing array begins with '0'
@throw parse_error.109 if an array index of an existing array was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved, or if an
array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
BasicJsonType& get_unchecked(BasicJsonType* ptr) const
{
for (const auto& reference_token : reference_tokens)
const auto* token = reference_tokens.data();
const auto* const end = token + reference_tokens.size();
if (token == end)
{
return *ptr;
}
if (ptr->is_null())
{
// the null value is converted to an array or an object
return get_unchecked_create(ptr, token, end);
}
// walk existing values; the walk itself inserts nothing, which keeps
// the loop fast (insertions are done below or in get_unchecked_create)
typename BasicJsonType::object_t::iterator it;
do
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
// the key is inserted below or in get_unchecked_create
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
if (token + 1 == end)
{
// explicitly treat "-" as index beyond the end
return array_append(*ptr);
}
// an element will be appended in get_unchecked_create
break;
}
if (token + 1 == end)
{
// convert array index to number; unchecked access
return ptr->operator[](array_index<BasicJsonType>(*token));
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
// the array will be padded in get_unchecked_create
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
// the null value will be converted in get_unchecked_create
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
while (++token != end);
if (token == end)
{
return *ptr;
}
if (token + 1 == end && ptr->is_object())
{
// insert the missing key where the walk found it belongs
return object_insert(*ptr, it, *token);
}
return get_unchecked_create(ptr, token, end);
}
/*!
@brief create the values for the reference tokens from @a first on
Continues @ref get_unchecked at the first reference token that does not
select an existing value. That token changes @a ptr, an existing value (see
@ref get_unchecked_rollback); all later tokens only change values created
here.
If the emplace() of the object type finds a member that the lookup in @ref
get_unchecked did not find (the two disagree, e.g. for fifo_map), nothing
is inserted, and the walk continues with that member as in @ref
get_unchecked, which is what basic_json::operator[] does.
@param[in] ptr the value the reference token @a first is applied to
@param[in] first the first reference token to resolve (an element of
@ref reference_tokens)
@param[in] end the end of @ref reference_tokens
@throw parse_error.106 if an array index of an existing array begins with
'0' (only after emplace() found a member, see above)
@throw parse_error.109 if an array index of an existing array was not a
number (only after emplace() found a member)
@throw out_of_range.404 if an array index is too large for strtoull, or (only
after emplace() found a member) if the JSON pointer
can not be resolved
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
JSON_HEDLEY_NEVER_INLINE static BasicJsonType& get_unchecked_create(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
const auto* token = first;
while (ptr->is_object())
{
const auto old_capacity = ptr->container_capacity();
const auto inserted = ptr->m_data.m_value.object->emplace(*token, nullptr);
if (JSON_HEDLEY_LIKELY(inserted.second))
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, inserted.first);
BasicJsonType& result = create_values(&ptr->set_parent_after_insert(inserted.first->second, old_capacity), token + 1, end);
rollback.commit();
return result;
}
// nothing was inserted: walk existing values from the member
// emplace() found on, like get_unchecked
ptr = &inserted.first->second;
while (++token != end)
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
const auto it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
break;
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
if (token == end)
{
return *ptr;
}
}
get_unchecked_rollback<BasicJsonType> rollback(ptr);
BasicJsonType& result = create_values(ptr, token, end);
rollback.commit();
return result;
}
/*!
@brief the loop of @ref get_unchecked_create
@a ptr is a null value or an array. Every value created here is a fresh
one, so every object a key is added to is empty.
@throw out_of_range.404 if an array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
static BasicJsonType& create_values(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
for (const auto* token = first; token != end; ++token)
{
const auto& reference_token = *token;
// convert null values to arrays or objects before continuing
if (ptr->is_null())
{
// check if the reference token is a valid array index, that is
// a nonempty sequence of digits without a leading '0'
// (cf. RFC 6901, Sect. 4); tokens that could never be a valid
// array index (such as "01" or "") are treated as object keys
const bool nums = !reference_token.empty()
&& (reference_token.size() == 1 || reference_token[0] != '0')
&& std::all_of(reference_token.begin(), reference_token.end(),
[](const unsigned char x)
// change value to an array for array indices or "-" or to an
// object otherwise; every null value here is a fresh one (see
// get_unchecked_rollback), so converting it in place leaves
// the same value as assigning an array or object to it
if (is_array_index_token(reference_token) || is_dash(reference_token))
{
return std::isdigit(x);
});
// change value to an array for array indices or "-" or to object otherwise
*ptr = (nums || reference_token == "-")
? detail::value_t::array
: detail::value_t::object;
ptr->template convert_null_to<typename BasicJsonType::array_t>();
}
else
{
ptr->template convert_null_to<typename BasicJsonType::object_t>();
}
}
if (ptr->is_object())
{
// use unchecked object access
ptr = &object_member(*ptr, reference_token);
// the object is new and empty
ptr = &object_add(*ptr, reference_token);
}
else if (ptr->is_array())
else
{
if (reference_token == "-")
// only the first value is not created here, and it is a null
// value or an array (see get_unchecked_create)
JSON_ASSERT(ptr->is_array());
if (is_dash(reference_token))
{
// explicitly treat "-" as index beyond the end
ptr = &ptr->operator[](ptr->m_data.m_value.array->size());
ptr = &array_append(*ptr);
}
else
{
@@ -25103,16 +25525,31 @@ class json_pointer
ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
}
}
else
{
// primitive value with a reference token left
throw_unresolved(reference_token, ptr);
}
}
return *ptr;
}
/// whether @a s is a valid array index, that is a nonempty sequence of
/// digits without a leading '0' (cf. RFC 6901, Sect. 4); tokens that could
/// never be a valid array index (such as "01" or "") are object keys
static bool is_array_index_token(const string_t& s)
{
const std::size_t size = s.size();
if (size == 0 || (size > 1 && s[0] == '0'))
{
return false;
}
for (std::size_t i = 0; i < size; ++i)
{
if (std::isdigit(static_cast<unsigned char>(s[i])) == 0)
{
return false;
}
}
return true;
}
/*!
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@@ -36473,7 +36910,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader;
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
friend class ::nlohmann::detail::json_sax_dom_parser;
template<typename BasicJsonType, typename InputAdapterType>
friend class ::nlohmann::detail::json_sax_dom_binary_parser;
+17 -250
View File
@@ -269,7 +269,7 @@ struct node
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the object's key hash index (1-based, 0 = none); otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if escaped/edited; number of the element sequence if moved
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence); escaped string: source offset of the opening quote plus one (JSON_DIAGNOSTIC_POSITIONS; 0: unknown)
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");
@@ -2402,13 +2402,7 @@ indent_done:
return false;
}
p = r.p;
#if JSON_DIAGNOSTIC_POSITIONS
// the decoded string is in the arena: next keeps the source offset
// of the opening quote plus one (0: unknown) for materialize()
push_node(node_kind::string, node_flags::escaped, 0, r.start, static_cast<std::uint64_t>(r.len) | (static_cast<std::uint64_t>(s - b) << 32));
#else
push_node(node_kind::string, node_flags::escaped, 0, r.start, r.len);
#endif
return true;
}
};
@@ -4949,8 +4943,7 @@ constexpr std::uint64_t image_limit = 0xFFFFFFF0u;
/// Copy the current structure of an edited document into nodes in document
/// order, as the parser would have written them. Text written by edits is
/// appended to text_tail (number tokens) and arena_tail (strings); floats that
/// are not finite become null, as dump() writes them. Nodes keep the flag
/// is_new: they have no source position (JSON_DIAGNOSTIC_POSITIONS).
/// are not finite become null, as dump() writes them.
inline void compact_nodes(const document_data& d, std::size_t arena_size, std::vector<node>& out, std::string& text_tail, std::string& arena_tail)
{
struct frame
@@ -4973,18 +4966,16 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
arena_tail.append(d.str(s), s.len);
r.flags = node_flags::escaped;
}
r.flags = static_cast<std::uint8_t>(r.flags | (s.flags & node_flags::is_new));
return r;
};
const auto copy_node = [&](const node * v)
{
node r = *v;
const auto is_new = static_cast<std::uint8_t>(v->flags & node_flags::is_new);
switch (static_cast<value_t>(v->kind))
{
case value_t::object:
case value_t::array:
r.flags = is_new;
r.flags = 0;
r.extra = 0;
r.off = (v->flags & (node_flags::moved | node_flags::is_new)) != 0 ? 0 : v->off;
r.len = 0; // counted below
@@ -5001,7 +4992,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(d.str(*v), number_length(*v));
}
r.flags = is_new;
r.flags = 0;
break;
case value_t::number_float:
if ((v->flags & node_flags::storage) == node_flags::edited)
@@ -5011,23 +5002,22 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
{
r = node{}; // nan and infinity: null, as dump() writes them
r.kind = node_kind::null;
r.flags = is_new;
break;
}
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(t, v->len);
r.extra = 0xFFFFu; // the digit layout is not recorded
}
r.flags = is_new;
r.flags = 0;
break;
case value_t::boolean:
r.flags = static_cast<std::uint8_t>(v->flags & (node_flags::is_true | node_flags::is_new));
r.flags = static_cast<std::uint8_t>(v->flags & node_flags::is_true);
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
r.flags = is_new;
r.flags = 0;
break;
}
out.push_back(r);
@@ -5388,15 +5378,13 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
std::vector<float_range> floats;
const auto check_string = [&](const node & n) -> bool
{
if ((n.flags & ~(node_flags::escaped | node_flags::is_new)) != 0 || n.extra != 0)
if ((n.flags & ~node_flags::escaped) != 0 || n.extra != 0)
{
return false;
}
const bool decoded = (n.flags & node_flags::escaped) != 0;
const std::size_t limit = decoded ? arena_size : text_size;
// (next: the source offset of a decoded string's opening quote plus
// one, or 0; see node)
if (n.off > limit || n.len > limit - n.off || n.next > text_size)
if (n.off > limit || n.len > limit - n.off)
{
return false;
}
@@ -5468,13 +5456,13 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
{
case value_t::null:
// (the offset of a literal is read to size the output of dump())
if ((n.flags & ~node_flags::is_new) != 0 || n.extra != 0 || n.off > text_size)
if (n.flags != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::boolean:
if ((n.flags & ~(node_flags::is_true | node_flags::is_new)) != 0 || n.extra != 0 || n.off > text_size)
if ((n.flags & ~node_flags::is_true) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
@@ -5488,7 +5476,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if ((n.flags & ~node_flags::is_new) != 0 || !number_in_bounds(n))
if (n.flags != 0 || !number_in_bounds(n))
{
return false;
}
@@ -5510,7 +5498,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
case value_t::object:
{
const std::size_t limit = stack.empty() ? count : stack.back().end;
if ((n.flags & ~node_flags::is_new) != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
if (n.flags != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
{
return false;
}
@@ -6120,176 +6108,6 @@ namespace detail
namespace view
{
#if JSON_DIAGNOSTIC_POSITIONS
/*!
@brief the source positions of the values materialize() replays
Stands in for the lexer of json_sax_dom_parser, which reads the positions of
the values it stores from these three functions (diagnostic_positions). npos
is a value without a source position.
*/
struct position_source
{
/// the token, of which diagnostic_positions only reads the length
struct token
{
std::size_t length;
std::size_t size() const noexcept
{
return length;
}
};
std::size_t start = std::string::npos; ///< the first byte of the next value
std::size_t end = std::string::npos; ///< one past the next value (start_object/start_array: one past the bracket)
std::size_t get_chars_read_total() const noexcept
{
return end;
}
std::size_t get_token_start_position() const noexcept
{
return start;
}
token get_string() const noexcept
{
return token{end - start};
}
/// the range [first, last), or none if either is npos
void set(std::size_t first, std::size_t last) noexcept
{
const bool known = first != std::string::npos && last != std::string::npos;
start = known ? first : std::string::npos;
end = known ? last : std::string::npos;
}
/// one past the closing quote of the string whose opening quote is at
/// src[quote], or npos (the scan is bounded by size, as images can be
/// loaded without a check)
static std::size_t string_end(const char* src, std::size_t size, std::size_t quote) noexcept
{
for (std::size_t i = quote + 1; i < size; ++i)
{
if (src[i] == '\\')
{
++i;
}
else if (src[i] == '"')
{
return i + 1;
}
}
return std::string::npos;
}
/// one past the closing bracket of the array or object whose last element
/// ends at src[pos] (or whose opening bracket is at src[pos - 1]), or npos;
/// in between, there can only be whitespace, comments, and a trailing comma
/// (the source was validated when it was parsed)
static std::size_t container_end(const char* src, std::size_t size, std::size_t pos) noexcept
{
while (pos < size)
{
switch (src[pos])
{
case ' ':
case '\t':
case '\n':
case '\r':
case ',':
++pos;
break;
case ']':
case '}':
return pos + 1;
case '/':
if (pos + 1 < size && src[pos + 1] == '/')
{
pos += 2;
while (pos < size && src[pos] != '\n' && src[pos] != '\r')
{
++pos;
}
break;
}
if (pos + 1 < size && src[pos + 1] == '*')
{
pos += 2;
while (pos < size && !(src[pos] == '*' && pos + 1 < size && src[pos + 1] == '/'))
{
++pos;
}
pos += 2;
break;
}
return std::string::npos;
default:
return std::string::npos;
}
}
return std::string::npos;
}
/// the range of the scalar n of document d
void set_scalar(const document_data& d, const node& n) noexcept
{
const std::size_t size = d.size;
const std::size_t off = n.off;
std::size_t first = std::string::npos;
std::size_t last = std::string::npos;
if ((n.flags & node_flags::is_new) == 0)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::string:
if ((n.flags & node_flags::storage) == 0)
{
// off is the first byte after the opening quote
if (off != 0)
{
first = off - 1;
last = off + std::size_t{n.len} + 1;
}
}
else if ((n.flags & node_flags::storage) == node_flags::escaped && n.next != 0)
{
// next is the offset of the opening quote plus one
first = n.next - 1;
last = string_end(d.base[0], size, first);
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if ((n.flags & node_flags::storage) == 0)
{
first = off;
last = off + number_length(n);
}
break;
case value_t::boolean:
first = off;
last = off + ((n.flags & node_flags::is_true) != 0 ? 4u : 5u);
break;
case value_t::null:
first = off;
last = off + 4u;
break;
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
break;
}
}
// (a crafted image can point past the text)
set(first, last != std::string::npos && last <= size ? last : std::string::npos);
}
};
#endif
/*!
@brief the basic_json value of the subtree at n
@@ -6297,22 +6115,14 @@ The subtree is replayed into the SAX handler that parse() uses to build its
values, so the result is the value parse() would produce: duplicate keys keep
the last value, and with JSON_DIAGNOSTICS the parent pointers are set. It is
iterative, so the nesting depth is limited by memory only, as for parse().
With JSON_DIAGNOSTIC_POSITIONS, the values get their source positions as with
parse(): position_source stands in for the lexer. Values written by edits have
none (npos), and neither has an array or object whose elements changed, or
whose last element has none (the end of its closing bracket is found by
scanning from the end of the last element).
Without a lexer the handler records no source positions
(JSON_DIAGNOSTIC_POSITIONS).
*/
template<typename BasicJsonType, bool Editable>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
#if JSON_DIAGNOSTIC_POSITIONS
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>, true, position_source>;
#else
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>, true>;
#endif
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
@@ -6324,15 +6134,9 @@ BasicJsonType materialize(const document_data& d, const node* n)
const node* pos;
const node* end;
bool object;
#if JSON_DIAGNOSTIC_POSITIONS
std::size_t start; ///< the opening bracket (npos: unknown)
std::size_t last_end; ///< the end of the last element read (npos: unknown)
#endif
};
#if !JSON_DIAGNOSTIC_POSITIONS
// a scalar needs neither the handler nor the stack of open containers
// (with positions, it goes through the handler, which sets them)
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
@@ -6354,16 +6158,10 @@ BasicJsonType materialize(const document_data& d, const node* n)
default:
break;
}
#endif
BasicJsonType result;
#if JSON_DIAGNOSTIC_POSITIONS
position_source positions;
sax_t sax(result, true, &positions);
#else
sax_t sax(result, true);
#endif
const string_t no_token {};
const string_t no_token{};
// (shallow subtrees on the stack: a call for a small value then allocates only for the value)
std::array<frame, 16> shallow; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled as containers open
std::vector<frame> deep{};
@@ -6378,16 +6176,6 @@ BasicJsonType materialize(const document_data& d, const node* n)
std::size_t depth = 0;
for (;;)
{
#if JSON_DIAGNOSTIC_POSITIONS
if (!is_container(*n))
{
positions.set_scalar(d, *n);
if (depth != 0)
{
open[depth - 1].last_end = positions.end;
}
}
#endif
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
@@ -6396,12 +6184,6 @@ BasicJsonType materialize(const document_data& d, const node* n)
case value_t::array:
{
const bool object = n->kind == node_kind::object;
#if JSON_DIAGNOSTIC_POSITIONS
// (moved containers are also new)
const std::size_t start = (n->flags & node_flags::is_new) == 0 && n->off < d.size ? std::size_t{n->off} :
std::string::npos;
positions.set(start, start + 1);
#endif
if (object)
{
sax.start_object(n->len);
@@ -6418,14 +6200,8 @@ BasicJsonType materialize(const document_data& d, const node* n)
open = deep.data();
capacity = deep.size();
}
#if JSON_DIAGNOSTIC_POSITIONS
// (until an element is read, the container's own bracket is the last thing read)
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object, start, positions.end};
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
break;
#else
open[depth++] = frame {nav::first(d, n), nav::end(d, n), object};
break;
#endif
}
case value_t::string:
{
@@ -6465,15 +6241,6 @@ BasicJsonType materialize(const document_data& d, const node* n)
frame& f = open[depth - 1];
if (f.pos == f.end)
{
#if JSON_DIAGNOSTIC_POSITIONS
const std::size_t end = f.start == std::string::npos || f.last_end == std::string::npos ? std::string::npos : position_source::container_end(d.base[0], d.size, f.last_end);
// (an end of npos clears the start, see diagnostic_positions)
positions.set(end == std::string::npos ? end : end - 1, end);
if (depth > 1)
{
open[depth - 2].last_end = end;
}
#endif
if (f.object)
{
sax.end_object();
+17
View File
@@ -373,6 +373,23 @@ TEST_CASE("alternative string type")
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/bar")));
}
SECTION("non-const operator[](json_pointer)")
{
// creating values must compile without string_t::begin()
alt_json j = alt_json::parse(R"({"a":{"b":[1]},"c":null})");
const alt_json before = j;
j[alt_json::json_pointer("/a/b/3/x/-/y")] = 1; // pad, append, create
j[alt_json::json_pointer("/a/z")] = 2; // insert a key
j[alt_json::json_pointer("/c/0/q")] = 3; // convert null
j[alt_json::json_pointer("/a/b/-")] = 4; // append
CHECK(j.dump() == R"({"a":{"b":[1,null,null,{"x":[{"y":1}]},4],"z":2},"c":[{"q":3}]})");
// a failing access leaves the value unchanged
j = before;
CHECK_THROWS_AS(j[alt_json::json_pointer("/a/n/99999999999999999999999")], alt_json::out_of_range&);
CHECK(j == before);
}
SECTION("operator/(std::size_t)")
{
// json_pointer::operator/=(std::size_t) must compile without string_t
+1 -1
View File
@@ -1311,7 +1311,7 @@ TEST_CASE("BSON declared document size bounds the content")
// records the events that a SAX parser receives
struct recorder : nlohmann::json_sax<json>
{
std::string events;
std::string events {};
bool null() override
{
+8 -3
View File
@@ -2178,8 +2178,13 @@ TEST_CASE("JSON patch - behavior documented in the API reference")
SECTION("patch() has the strong guarantee")
{
const json doc = R"({"a": [1, 2], "o": {"k": "v"}})"_json;
const json before = doc;
// independent values, so that the comparison below checks the untouched document
const auto make_doc = []
{
return R"({"a": [1, 2], "o": {"k": "v"}})"_json;
};
const json doc = make_doc();
const json before = make_doc();
const std::vector<json> failing =
{
@@ -2202,7 +2207,7 @@ TEST_CASE("JSON patch - behavior documented in the API reference")
CHECK_THROWS_AS(doc.patch(patch), json::exception&);
CHECK(doc == before);
json copy = doc;
json copy = make_doc();
CHECK_THROWS_AS(copy.patch(patch), json::exception&);
CHECK(copy == before);
}
+72
View File
@@ -29,9 +29,64 @@ using nlohmann::json;
#include <map>
#include <random>
#include <sstream>
#include <stdexcept>
#include <string>
#include <vector>
namespace
{
// the non-const operator[] must throw for pointer and leave the value parsed
// from init unchanged (strong exception safety)
template<typename Json>
void check_strong_guarantee(const char* init, const std::string& pointer, const char* message)
{
CAPTURE(init)
CAPTURE(pointer)
Json j = Json::parse(init);
const Json before = Json::parse(init);
const typename Json::json_pointer ptr(pointer);
CHECK_THROWS_WITH_AS(j[ptr], message, typename Json::exception&);
CHECK(j == before);
}
// as above, for an index that fits into size_type, but not into an array
template<typename Json>
void check_strong_guarantee_length_error(const char* init, const std::string& pointer)
{
CAPTURE(init)
CAPTURE(pointer)
Json j = Json::parse(init);
const Json before = Json::parse(init);
const typename Json::json_pointer ptr(pointer);
CHECK_THROWS_AS(j[ptr], std::length_error&);
CHECK(j == before);
}
template<typename Json>
void check_strong_guarantee_cases()
{
const char* const unresolved = "[json.exception.out_of_range.404] unresolved reference token '99999999999999999999999'";
// the first change converts a null value
check_strong_guarantee<Json>("null", "/99999999999999999999999", unresolved);
check_strong_guarantee<Json>("null", "/a/99999999999999999999999", unresolved);
check_strong_guarantee<Json>(R"({"a":null})", "/a/b/-/99999999999999999999999", unresolved);
// the first change inserts a key into an object
check_strong_guarantee<Json>("{}", "/a/99999999999999999999999", unresolved);
check_strong_guarantee<Json>(R"({"a":{"b":1}})", "/a/c/d/99999999999999999999999", unresolved);
// the first change pads or appends to an array
check_strong_guarantee<Json>(R"({"a":[1]})", "/a/5/99999999999999999999999", unresolved);
check_strong_guarantee<Json>(R"({"a":[1]})", "/a/-/x/1/99999999999999999999999", unresolved);
// an index that does not fit into size_type
check_strong_guarantee<Json>("null", "/x/18446744073709551615",
"[json.exception.out_of_range.410] array index 18446744073709551615 exceeds size_type");
// an index that fits into size_type, but not into an array
const auto too_large = std::to_string((std::numeric_limits<typename Json::size_type>::max)() - 1);
check_strong_guarantee_length_error<Json>("null", "/" + too_large);
check_strong_guarantee_length_error<Json>(R"({"a":[[1]]})", "/a/0/3/" + too_large);
}
} // namespace
TEST_CASE("JSON pointers")
{
SECTION("errors")
@@ -557,6 +612,23 @@ TEST_CASE("JSON pointers")
CHECK_THROWS_WITH_AS(j.at("/01"_json_pointer),
"[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", json::parse_error&);
}
SECTION("a failing operator[] leaves the value unchanged")
{
check_strong_guarantee_cases<json>();
check_strong_guarantee_cases<nlohmann::ordered_json>();
}
SECTION("a successful operator[] still creates the same values")
{
json j = {{"a", {1}}};
j["/a/3/x/-/y"_json_pointer] = 1;
CHECK(j == json::parse(R"({"a":[1,null,null,{"x":[{"y":1}]}]})"));
j["/a/-"_json_pointer] = 2;
j["/a/0"_json_pointer] = 3;
j["/b"_json_pointer] = 4;
CHECK(j == json::parse(R"({"a":[3,null,null,{"x":[{"y":1}]},2],"b":4})"));
}
}
SECTION("flatten")
+9 -23
View File
@@ -749,20 +749,6 @@ std::string exception_of(F f)
}
return "";
}
// the exception a call on a basic_json value throws, without the byte range of
// the value (JSON_DIAGNOSTIC_POSITIONS), which the exceptions of views lack
template<typename F>
std::string json_exception_of(F f)
{
std::string what = exception_of(f);
const std::size_t at = what.find("] (bytes ");
if (at != std::string::npos)
{
what.erase(at + 2, what.find(") ", at) - at);
}
return what;
}
#endif
// compares a view with the ordered_json value materialize() gives for it:
@@ -971,24 +957,24 @@ TEST_CASE("json_view element access and iteration")
#if !defined(JSON_NOEXCEPTION)
if (!j.is_object())
{
CHECK(exception_of([&] { static_cast<void>(v["a"]); }) == json_exception_of([&] { static_cast<void>(j["a"]); }));
CHECK(exception_of([&] { static_cast<void>(v["a"]); }) == exception_of([&] { static_cast<void>(j["a"]); }));
}
if (!j.is_array())
{
CHECK(exception_of([&] { static_cast<void>(v[0]); }) == json_exception_of([&] { static_cast<void>(j[0]); }));
CHECK(exception_of([&] { static_cast<void>(v[0]); }) == exception_of([&] { static_cast<void>(j[0]); }));
}
CHECK(exception_of([&] { static_cast<void>(v.at("a")); }) == json_exception_of([&] { static_cast<void>(j.at("a")); }));
CHECK(exception_of([&] { static_cast<void>(v.at("missing")); }) == json_exception_of([&] { static_cast<void>(j.at("missing")); }));
CHECK(exception_of([&] { static_cast<void>(v.at(0)); }) == json_exception_of([&] { static_cast<void>(j.at(0)); }));
CHECK(exception_of([&] { static_cast<void>(v.at(5)); }) == json_exception_of([&] { static_cast<void>(j.at(5)); }));
CHECK(exception_of([&] { static_cast<void>(v.at("a")); }) == exception_of([&] { static_cast<void>(j.at("a")); }));
CHECK(exception_of([&] { static_cast<void>(v.at("missing")); }) == exception_of([&] { static_cast<void>(j.at("missing")); }));
CHECK(exception_of([&] { static_cast<void>(v.at(0)); }) == exception_of([&] { static_cast<void>(j.at(0)); }));
CHECK(exception_of([&] { static_cast<void>(v.at(5)); }) == exception_of([&] { static_cast<void>(j.at(5)); }));
if (!(j.is_object() && j.empty())) // (key() of an end iterator)
{
CHECK(exception_of([&] { static_cast<void>(v.begin().key()); }) == json_exception_of([&] { static_cast<void>(j.begin().key()); }));
CHECK(exception_of([&] { static_cast<void>(v.begin().key()); }) == exception_of([&] { static_cast<void>(j.begin().key()); }));
}
if (!j.empty() || j.is_null())
{
CHECK(exception_of([&] { static_cast<void>(v.front()); }) == json_exception_of([&] { static_cast<void>(j.front()); }));
CHECK(exception_of([&] { static_cast<void>(v.back()); }) == json_exception_of([&] { static_cast<void>(j.back()); }));
CHECK(exception_of([&] { static_cast<void>(v.front()); }) == exception_of([&] { static_cast<void>(j.front()); }));
CHECK(exception_of([&] { static_cast<void>(v.back()); }) == exception_of([&] { static_cast<void>(j.back()); }));
}
#endif
CHECK(v.contains("a") == j.contains("a"));
+2 -2
View File
@@ -592,7 +592,7 @@ TEST_CASE("json_view node integer bits")
})
{
CAPTURE(v)
n.kind = static_cast<nlohmann::detail::view::node_kind>(0x5A);
n.kind = nlohmann::detail::view::node_kind::link;
n.flags = 0xA5;
n.extra = 0x1234;
n.off = 0x89ABCDEFu;
@@ -601,7 +601,7 @@ TEST_CASE("json_view node integer bits")
CHECK(n.len == static_cast<std::uint32_t>(v));
CHECK(n.next == static_cast<std::uint32_t>(v >> 32));
// the other fields are untouched
CHECK(static_cast<std::uint8_t>(n.kind) == 0x5A);
CHECK(n.kind == nlohmann::detail::view::node_kind::link);
CHECK(n.flags == 0xA5);
CHECK(n.extra == 0x1234);
CHECK(n.off == 0x89ABCDEFu);
+4 -6
View File
@@ -326,8 +326,7 @@ TEST_CASE("json_view from_json")
CHECK(d.root().get<std::unordered_map<std::string, int>>().at("a") == 3);
}
// (the exceptions of views have neither the path nor the byte range of the value)
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS && !JSON_DIAGNOSTIC_POSITIONS
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS
SECTION("std types: the same errors as from the materialized value")
{
const json_document d = json_document::parse(R"([[1, "x"], {"a": 1}, "text", [1], 5, null])");
@@ -393,7 +392,7 @@ TEST_CASE("json_view from_json")
#if !defined(JSON_NOEXCEPTION)
const json j = d.root().materialize();
CHECK(exception_of_call([&] { static_cast<void>(d.root()[2].get<strict_color>()); }) == "[json.exception.out_of_range.410] enum value out of range for strict_color: \"purple\"");
#if !JSON_DIAGNOSTICS && !JSON_DIAGNOSTIC_POSITIONS
#if !JSON_DIAGNOSTICS
CHECK(exception_of_call([&] { static_cast<void>(d.root()[2].get<strict_color>()); }) == exception_of_call([&] { static_cast<void>(j[2].get<strict_color>()); }));
#endif
#endif
@@ -441,11 +440,10 @@ TEST_CASE("json_view from_json")
CHECK(o.root().get<shape>().center.y == 2);
}
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS && !JSON_DIAGNOSTIC_POSITIONS
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS
SECTION("errors are those of the same conversion of a json value")
{
// (with JSON_DIAGNOSTICS, those of json name the path of the value,
// and with JSON_DIAGNOSTIC_POSITIONS its bytes)
// (with JSON_DIAGNOSTICS, those of json name the path of the value)
const json_document d = json_document::parse(R"([{"x": 1}, {"x": "1", "y": 2}, [1, 2], {"id": "7", "c": "red"}])");
const json j = d.root().materialize();
for (std::size_t i = 0; i < 3; ++i)
-340
View File
@@ -1,340 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// source positions of materialize() (JSON_DIAGNOSTIC_POSITIONS, #5864)
#include "doctest_compatibility.h"
#ifndef JSON_DIAGNOSTICS
#define JSON_DIAGNOSTICS 1
#endif
#define JSON_DIAGNOSTIC_POSITIONS 1
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::json_editable_document;
#include <cstdint>
#include <cstring>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include "json_view_test_helpers.hpp"
#include <test_data.hpp>
namespace
{
constexpr std::size_t npos = std::string::npos;
// whether every value of a has the source positions of the value of b at the
// same place (a and b are equal; shallow test documents only)
template<typename BasicJsonType>
bool same_positions(const BasicJsonType& a, const BasicJsonType& b)
{
if (a.start_pos() != b.start_pos() || a.end_pos() != b.end_pos() || a.type() != b.type() || a.size() != b.size())
{
return false;
}
if (a.is_object())
{
for (auto it = a.begin(); it != a.end(); ++it)
{
if (!same_positions(it.value(), b.at(it.key())))
{
return false;
}
}
}
else if (a.is_array())
{
for (std::size_t i = 0; i < a.size(); ++i)
{
if (!same_positions(a[i], b[i]))
{
return false;
}
}
}
return true;
}
// the materialized root of text has the positions of parse()
template<typename BasicJsonType>
bool positions_as_parse(const std::string& text, bool comments = false, bool trailing_commas = false)
{
const auto j = BasicJsonType::parse(text, nullptr, true, comments, trailing_commas);
const auto d = nlohmann::basic_json_document<BasicJsonType>::parse(text, true, comments, trailing_commas);
const auto e = nlohmann::basic_json_document<BasicJsonType, nlohmann::view_mode::editable>::parse(text, true, comments, trailing_commas);
return same_positions(d.root().materialize(), j) && same_positions(e.root().materialize(), j);
}
std::string range(const json& j)
{
return j.start_pos() == npos && j.end_pos() == npos ? "none" : std::to_string(j.start_pos()) + "-" + std::to_string(j.end_pos());
}
} // namespace
TEST_CASE("json_view materialize() source positions")
{
SECTION("as parse()")
{
const std::vector<std::string> texts =
{
"null", "true", "false", "0", "-12", "3.25e-2", "18446744073709551615", "-9223372036854775809", R"("a")",
R"("\n")", R"("\u00e9")", R"("\ud83d\ude00")", "\"\xf0\x9f\x98\x80\"", R"("a\"b\\")", R"("x\\")",
" [ ] ", "{}", "[[[]]]", R"({"a":{"b":[1,{"c":[]}]}})",
R"({"\u0041": "v\n", "k": "\"", "\t":[ "\/" ]})",
"[1, 2.5, true, false, null, \"s\", \"\\u0000\", -0]",
"\n\t[\r\n 1 ,\n {\n \"a\" : [ ]\n }\n]\n",
"\xef\xbb\xbf[1, {\"a\": \"b\"}]", "\xef\xbb\xbf\"\\n\"",
};
for (const auto& text : texts)
{
CAPTURE(text)
CHECK(positions_as_parse<json>(text));
CHECK(positions_as_parse<ordered_json>(text));
}
}
SECTION("comments and trailing commas")
{
const std::vector<std::string> texts =
{
"[1 /* ] */]", "[1 // ]\n]", "{\"a\": 1 /* } */ , }", "[1, 2, ]", "[[1,],[2 , /**/ ] , ]",
"{\"a\": [] // }\n, \"b\": {} /* } ] */ }", "[ /* empty */ ]", "{ // }\n}", "/* a */ [1] // b",
"[\"\\n\" /* \" */ , ]",
};
for (const auto& text : texts)
{
CAPTURE(text)
CHECK(positions_as_parse<json>(text, true, true));
}
}
SECTION("generated documents")
{
json_view_test::generator gen;
for (int i = 0; i < 300; ++i)
{
std::string text;
gen.value(text, 0);
CAPTURE(text)
CHECK(positions_as_parse<json>(text));
CHECK(positions_as_parse<ordered_json>(text));
}
}
SECTION("test data")
{
for (const char* name :
{
"/json_testsuite/sample.json", "/nativejson-benchmark/canada.json", "/nativejson-benchmark/citm_catalog.json",
"/nativejson-benchmark/twitter.json", "/json_tests/pass1.json"
})
{
CAPTURE(name)
const std::ifstream f(std::string(TEST_DATA_DIRECTORY) + name, std::ios::binary);
std::stringstream ss;
ss << f.rdbuf();
CHECK(positions_as_parse<json>(ss.str()));
}
}
SECTION("subtrees")
{
const std::string text = R"({"a": [1, "\n", {"b": null}], "s": "x\ty", "n": -1.5})";
const json j = json::parse(text);
const json_document d = json_document::parse(text);
CHECK(same_positions(d.root()["a"].materialize(), j["a"]));
CHECK(same_positions(d.root()["a"][2].materialize(), j["a"][2]));
CHECK(same_positions(d.root()["a"][1].materialize(), j["a"][1]));
CHECK(same_positions(d.root()["s"].materialize(), j["s"]));
CHECK(same_positions(d.root()["n"].materialize(), j["n"]));
CHECK(range(d.root()["a"][1].materialize()) == "10-14");
CHECK(range(d.root()["s"].materialize()) == "35-41");
}
#if !defined(JSON_NOEXCEPTION)
SECTION("exception messages")
{
const std::string text = "{\n \"address\": {\"street\": \"Fake\\tStreet\", \"housenumber\": \"1\"},\n \"list\": [1, [true]]\n}";
const json j = json::parse(text);
const json_document d = json_document::parse(text);
const json m = d.root().materialize();
const std::string expected = json_view_test::exception_of_call([&]
{
static_cast<void>(j.at("address").at("housenumber").get<int>());
});
CHECK(expected.find("(bytes ") != std::string::npos);
CHECK(json_view_test::exception_of_call([&]
{
static_cast<void>(m.at("address").at("housenumber").get<int>());
}) == expected);
CHECK(json_view_test::exception_of_call([&]
{
static_cast<void>(m.at("address").at("street").get<int>());
}) == json_view_test::exception_of_call([&]
{
static_cast<void>(j.at("address").at("street").get<int>());
}));
CHECK(json_view_test::exception_of_call([&]
{
static_cast<void>(m.at("list").at(1).get<int>());
}) == json_view_test::exception_of_call([&]
{
static_cast<void>(j.at("list").at(1).get<int>());
}));
}
#endif
SECTION("edited documents")
{
const std::string text = R"({"a": [1, 2], "b": {"c": "\n", "d": true}, "e": [3, [4]], "f": "x"})";
const json j = json::parse(text);
SECTION("new scalars have none")
{
json_editable_document d = json_editable_document::parse(text);
d.set(d.root()["f"], 5);
d.set(d.root()["b"]["d"], "s");
const json m = d.root().materialize();
CHECK(range(m["f"]) == "none");
CHECK(range(m["b"]["d"]) == "none");
// the object whose last member is new has none; the others keep theirs
CHECK(range(m) == "none");
CHECK(range(m["b"]) == "none");
CHECK(same_positions(m["a"], j["a"]));
CHECK(same_positions(m["e"], j["e"]));
CHECK(same_positions(m["b"]["c"], j["b"]["c"]));
}
SECTION("containers whose elements change have none")
{
json_editable_document d = json_editable_document::parse(text);
d.push_back(d.root()["a"], 7);
d.erase(d.root()["e"], 0);
const json m = d.root().materialize();
CHECK(range(m["a"]) == "none");
CHECK(range(m["a"][2]) == "none");
CHECK(range(m["e"]) == "none");
// the elements kept keep theirs
CHECK(range(m["a"][0]) == range(j["a"][0]));
CHECK(range(m["a"][1]) == range(j["a"][1]));
CHECK(same_positions(m["e"][0], j["e"][1]));
CHECK(same_positions(m["b"], j["b"]));
// the root's elements did not change
CHECK(range(m) == range(j));
}
SECTION("new containers have none")
{
json_editable_document d = json_editable_document::parse(text);
d.set(d.root()["b"], json::parse(R"({"x": [1]})"));
const json m = d.root().materialize();
CHECK(range(m["b"]) == "none");
CHECK(range(m["b"]["x"]) == "none");
CHECK(range(m["b"]["x"][0]) == "none");
CHECK(range(m) == range(j));
}
SECTION("copies of scalars of the document keep theirs")
{
// (a copy of an array or object is a new one; its scalars are the
// parsed ones)
json_editable_document d = json_editable_document::parse(text);
d.set(d.root()["f"], d.root()["e"]);
d.set(d.root()["a"], d.root()["b"]["c"]);
const json m = d.root().materialize();
CHECK(range(m["a"]) == range(j["b"]["c"]));
CHECK(range(m["f"]) == "none");
CHECK(range(m["f"][1]) == "none");
CHECK(range(m["f"][0]) == range(j["e"][0]));
CHECK(range(m["f"][1][0]) == range(j["e"][1][0]));
}
}
#if !(defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)
SECTION("images")
{
const std::string text = R"({"a\n": ["\u00e9", 1.5, {"b": [true, null]}], "c": "d"})";
const json j = json::parse(text);
SECTION("round trip")
{
const json_document d = json_document::load(json_document::parse(text).save());
CHECK(same_positions(d.root().materialize(), j));
}
SECTION("edited")
{
json_editable_document e = json_editable_document::parse(text);
e.set(e.root()["c"], 1);
e.push_back(e.root()[std::string("a") + '\n'], false);
e.set(e.root()[std::string("a") + '\n'][2]["b"][0], "\"");
const json_document d = json_document::load(e.save());
const json m = d.root().materialize();
CHECK(range(m["c"]) == "none");
CHECK(range(m) == "none");
const json& a = m[std::string("a") + '\n'];
const json& ja = j[std::string("a") + '\n'];
CHECK(range(a) == "none");
CHECK(range(a[3]) == "none");
CHECK(range(a[0]) == range(ja[0]));
CHECK(range(a[1]) == range(ja[1]));
CHECK(range(a[2]["b"][0]) == "none");
CHECK(range(a[2]["b"][1]) == range(ja[2]["b"][1]));
// the last element of b is still the parsed one
CHECK(range(a[2]["b"]) == range(ja[2]["b"]));
CHECK(range(a[2]) == range(ja[2]));
}
SECTION("without positions")
{
// an image saved without JSON_DIAGNOSTIC_POSITIONS: next is 0 for
// the strings with escapes
std::vector<std::uint8_t> image = json_document::parse(text).save();
std::uint64_t count = 0;
std::memcpy(&count, image.data() + 8, sizeof(count));
for (std::size_t i = 0; i < count; ++i)
{
nlohmann::detail::view::node n{};
std::memcpy(&n, image.data() + 64 + (i * sizeof(n)), sizeof(n));
if (n.kind == nlohmann::detail::view::node_kind::string)
{
n.next = 0;
}
std::memcpy(image.data() + 64 + (i * sizeof(n)), &n, sizeof(n));
}
const json_document d = json_document::load(image);
const json m = d.root().materialize();
CHECK(range(m[std::string("a") + '\n'][0]) == "none");
CHECK(range(m["c"]) == range(j["c"]));
CHECK(same_positions(m[std::string("a") + '\n'][2], j[std::string("a") + '\n'][2]));
}
#if !defined(JSON_NOEXCEPTION)
SECTION("a decoded string's position must be within the text")
{
std::vector<std::uint8_t> image = json_document::parse(R"(["\n"])").save();
nlohmann::detail::view::node n{};
std::memcpy(&n, image.data() + 64 + sizeof(n), sizeof(n));
REQUIRE(n.next == 2);
n.next = 7;
std::memcpy(image.data() + 64 + sizeof(n), &n, sizeof(n));
CHECK_THROWS_AS(json_document::load(image), json::parse_error&);
// without the check, the position is out of bounds and ignored
const json_document d = json_document::load(image, json_document::image_check::none);
CHECK(range(d.root()[0].materialize()) == "none");
}
#endif
}
#endif
}
+17
View File
@@ -1499,6 +1499,23 @@ TEST_CASE("regression tests 1")
CHECK(order.dump() == R"({"a":2,"m":{"y":1,"b":2},"new_key":3})");
}
SECTION("operator[](json_pointer) when emplace() finds a key the lookup does not")
{
// a copied fifo_map restarts its timestamps, so a new key "k" gets
// the timestamp of "x": the lookup does not find "k", but emplace()
// finds "x" and inserts nothing; the pointer then continues with the
// member of "x", as basic_json::operator[] does
const my_json orig = my_json::parse(R"({"x":1})");
const std::array<const char*, 3> pointers = {{"/k/0", "/k/a/b", "/k/-"}};
for (const char* pointer : pointers)
{
CAPTURE(pointer)
my_json j = orig;
CHECK_THROWS_AS(j[my_json::json_pointer(pointer)], my_json::out_of_range&);
CHECK(j.dump() == R"({"x":1})");
}
}
SECTION("issue #977 - Assigning between different json types")
{
foo_json lj = ns::foo{3};