Compare commits

..
Author SHA1 Message Date
Claude 88a3c1bc15 json_view: pass accept()'s validate-only request as a parameter
Address review of #5970:
- build() takes validate_only as an argument; the validate_only members of
  document_data and basic_json_document are removed, so neither grows
- the internal read path takes an on_error mode (discard, raise, validate)
  instead of allow_exceptions; accept() calls it with on_error::validate
- build_with() has no default for Store
- drop the unneeded static_casts in push_node()
- name the NulIsEnd argument so astyle keeps build<...> intact
- explain in the docs and code why comments/trailing commas keep the index

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasF8Toq7uBWUMCrSjXV
2026-10-30 23:09:53 +00:00
Niels Lohmann 9ba25b1b66 json_view: accept() checks strings without decoding them
The builder that accept() runs stores no nodes, but still decoded every string
with escapes into the arena (slow_string(): memcpy of the runs between escapes
and the UTF-8 of every \u escape). It now calls check_string() instead, which
has the same checks and the same errors and error positions (control characters,
ill-formed UTF-8, bad escapes, bad hex digits, lone and unpaired surrogates, a
missing closing quote) and skips the text between them with scan_string_run(),
without writing anything. The builder that stores keeps slow_string().

The flag that passes the request from accept() to build() now sits next to
`discarded` in document_data, where it uses padding: no member of document_data
moves (the first version put it after `nodes`, which moved all members behind it
by eight bytes).

Tests: the new section of the accept test checks runs of text of every length from
0 to 40 between valid and invalid escapes, UTF-8 sequences and control characters,
in values and in keys, cut near the end of the text, against json::accept().
Checked against a checker that accepts a lone low surrogate.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 22:34:32 +01:00
Niels Lohmann dea2a4434a json_view: accept() checks the text without writing an index
json_document::accept() ran the builder that writes the node index and
discarded the index. For input without comments and trailing commas it now
runs a second instantiation of the builder, with a template parameter Store
that is false: the grammar, number, string, UTF-8 and nesting checks are the
same code, but no node is written, the container nodes are not patched when
they close, the node array is not reserved, and no list of large objects is
kept. What only builds the arguments of a node (integer conversion, the digit
counts of floats) is not run either. accept() with comments or trailing commas
and parse() are not changed; the builder that stores is the one parse() always
used. A flag of the document passes the request from accept() to build().

accept() on twitter, citm_catalog, canada and the two minified files, A/B in
one binary, 6 code layouts, time after / before (geometric mean of the five;
the other 14 files of the set are faster too, none is slower):

- Apple M1 Max, Apple clang 21: 0.89
- x86-64 Haswell, GCC 16.2: 0.75
- x86-64 Haswell, Clang 23: 0.78

(semanticscholar-corpus.json: 0.52 on both x86 compilers.) parse() runs the same
instructions as before; its timings moved by up to 3 % on a few small files with
unchanged instruction counts (code placement).

Tests: accept() against json::accept() for the same inputs with all four
option combinations and with and without a terminating NUL: 2,000 generated
documents and damaged copies of them, nesting from 63 to 1000 levels (closed and
not), objects with 300 members, a null byte in the text, and float overflow for
a float document. Checked against a deliberately broken validator.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 22:18:14 +01:00
6 changed files with 469 additions and 274 deletions

No files matched your search

@@ -1,194 +0,0 @@
---
name: review-like-gregmarr
description: Review a diff, branch, or PR of nlohmann/json the way long-time contributor gregmarr does — a line-by-line read focused on C++11 correctness, reference invalidation, backward/source compatibility, ODR/ABI, duplicated logic, hidden costs, repo conventions (amalgamation, astyle, macros) and precise docs, phrased as short courteous questions. Use when asked to "review like Greg", "gregmarr review", or for a careful maintainer-style review of changes to this library.
---
# Review like gregmarr
Distilled from ~1,400 inline review comments and ~1,300 issue/PR comments by
@gregmarr on nlohmann/json. He is a careful, courteous, line-by-line reviewer
who ("just a contributor") defers final decisions to @nlohmann.
## Workflow
1. Determine the target: a PR number (`gh pr diff <n>`), a branch
(`git diff $(git merge-base HEAD develop)...HEAD`), or the working tree
(`git diff`). Read the full diff, and open surrounding code in `include/nlohmann/`
whenever a hunk's context is not enough to judge it.
2. Walk the diff **file by file, hunk by hunk**, applying the checklist below.
Comment on specific lines; don't write a big essay.
3. Write the review in his voice (see "Voice"). Group nothing artificially —
emit one comment per location, in file/line order, with `path:line`.
4. If the same issue repeats, comment fully once, then "As above" /
"Same here (and N other places in this file)."
5. End with a 1–3 line summary only if useful (e.g. "Just this remaining and
then LGTM." or "@nlohmann thoughts on the naming?"). Don't post to GitHub
unless the user asks.
## Checklist (roughly in order of how often he raises them)
### Duplication and simplification (most frequent)
- Copy-pasted bodies, duplicated `#if/#elif` blocks, or "a copy of this function
that can get out of sync" → propose a shared helper or alias, with a code sketch.
- **Diverging limits/constants**: a new depth/recursion limit or magic number
where one already exists → "Should this reuse the existing recursion limit?
Different limits per operation invites them to diverge over time."
- Unneeded variables, parameters that can be deduced, values passed to a
function that the constructor already has, output parameter == return value.
- Verbose type spellings → `using` alias / alias template; `decltype()` where the
type is known.
- Disabling a lint warning instead of fixing the code ("why not just add the parens?").
### Lifetime, invalidation, UB
- A reference/pointer/iterator into a container (`stack.back()`,
`container_stack.back()`, `ordered_map` elements) held across `push_back`,
`pop_back`, `emplace`, or reallocation → "This invalidates `top`." Prefer a
`const` copy, or at least a comment warning future editors.
- Dangling refs from rvalue arguments stored in adapters; views outliving documents.
- Wrong logic operators (`&&` vs `||`), lost `!`, unreachable code, `continue`
that should be `break`, missing `return`, off-by-one in counters
(`chars_read` incremented twice).
- Uninitialized members → default member initializers; parameters that
"should not have a default so it's never left off by accident".
- Exception safety: leaks when `to_json` is called on a non-null value, partial
state, destructors that allocate, swallowed exceptions (`noexcept` that hides
throws — "The library shouldn't try to swallow the exception").
- Thread-unsafe calls (`setlocale`), specializations that redefine `std::` things
(ODR / UB).
- Narrowing/truncation: `static_cast<std::make_unsigned<...>>`, signed/unsigned
`char` cases, `n == 0` edge cases, buffer overruns guarded only by `JSON_ASSERT`.
### C++11 floor and portability
- The library requires only **C++11**: flag `if constexpr`, C++14 generic
lambdas/variable templates, `std::clamp`, C++17 library features, etc. unless
gated.
- Gate on the library's macros (`JSON_HAS_CPP_17`, `JSON_HAS_CPP_20`,
`JSON_HAS_THREE_WAY_COMPARISON`, `JSON_HAS_FILESYSTEM`) or feature-test macros
with `defined()` checks and exact values
(`#if defined(__cpp_lib_char8_t) && __cpp_lib_char8_t >= 201811`), not raw
`__cplusplus`. Don't silently disable a feature for an existing config.
- User-overridable macros: if externally defined, leave as-is — never clobber to
`1` (breaks `#define X 0`). `#if X` vs `#ifdef X` must match the convention.
- New macros start with `JSON_` / `NLOHMANN_JSON_` consistently with neighbors.
- `throw` must be `JSON_THROW` (and `try/catch` → `JSON_TRY/JSON_CATCH`) so
`JSON_NOEXCEPTION` builds work.
- `(std::min)(a, b)` / `(std::max)` for Windows macros; MSVC warnings (C4100
etc.); `string_t` not being `std::string`/`char`.
- Code relying on operations not required of user-supplied `array_t`/`object_t`
(range ctor, `emplace_hint`, allocator) → ask whether that's allowed.
- Does it work with `JSON_USE_IMPLICIT_CONVERSIONS=0`? With `JSON_DIAGNOSTICS`?
### Compatibility, ABI, API design
- Behavior changes to existing APIs/macros (e.g. `NLOHMANN_DEFINE_TYPE_*`,
released in 3.11.0) are breaking → needs opt-in macro or wait for 4.0; must be
prominently documented in release notes. Cite when the behavior shipped.
- Header-only: source compatibility matters, binary doesn't — adding a
**defaulted** parameter is fine; new overloads add ambiguity risk.
- Anything that changes layout or semantics per-config → should it be part of the
ABI tag (`json_abi_diag...`)? Mismatched config macros across TUs = ODR violation.
- Brand-new API shouldn't be added as deprecated — use `= delete` or omit.
- Naming consistency with existing options (`ignore_trailing_commas` vs
`ignore_comments`; singular "diagnostic"); names that read like functions;
single-letter or inconsistently `m_`-prefixed members.
- Prefer a type-safe template over passing a `value_t` that could be invalid.
- "Why would this belong in this library?" for very niche features.
### Hidden costs
- Extra copies (lost `std::move`, removed C++17 path), temporaries
(`push_back(std::string(...))` vs two `push_back`s), repeated `.back()` calls,
casting more than once, `reserve()` inside loops, quadratic recomputation for
deep nesting, dynamic allocation replacing stack allocation.
- `push_back` → `emplace_back`; `rbegin()`, `.data()`, `std::array`, `pop_back()`
to keep counters in sync.
- Struct member order/padding (put `bool` after `int`; "store value before type"),
`sizeof(basic_json)` growth, per-node pointer cost. "Don't pay for what you
don't use."
- Compile-time impact of new includes/templates.
- Ask for measurement instead of asserting a win.
### Style and repo conventions
- Formatting must match `make pretty` (astyle): spaced `template <`, `!` in traits
→ suggest `detail::negation<>`; 4-space indents; brace style; blank lines.
- Redundant keywords: `inline` inside class bodies or on templates (full
specializations do need it), `noexcept` on destructors, `public` in `struct`,
`{}` after defaulted init, C-style casts → `static_cast`.
- Sorted includes (cpplint: system before others), sorted lists in docs/CMake.
- **Amalgamation**: changes must be in `include/nlohmann/...` and
`single_include/` regenerated via `make amalgamate` — flag edits made only in
`single_include`, or amalgamated output that doesn't match / contains unrelated
changes.
- Merge debris, stray files at repo root, accidental changes to vendored files
(doctest.h), Makefile tweaks from local testing, missing trailing newline,
whitespace churn mixed with real changes ("separate commit or PR?").
- Comments: explain non-obvious code ("Comment about why this iterates
backwards?"); no change-log wording like "still" in live code; comment
blocks must stay attached to their function.
### Docs (`docs/mkdocs/docs/...`, README)
- Typos as `s/varry/vary/` or "typo: interal".
- Accuracy vs. actual behavior; state preconditions ("array is not empty"),
costs (one pointer per value with `JSON_DIAGNOSTICS`), defaults, which overloads
a sentence applies to, version numbers ("3.13.0").
- Consistency with sibling pages; unexplained claims ("Why is that a problem?
It's not obvious here."); markdown table column alignment.
- New macros/features need a docs page and an example.
### Tests
- Not his main focus, but: new macro/feature needs tests in `tests/src/unit-*.cpp`
that always build with the option (coverage build); test both with and
without the option; tests should use different values than the
initial ones; put tests in the existing file instead of duplicating a SAX class;
static_asserts where possible.
### Usually NOT commented on
Commit messages/PR titles, licensing, local variable naming bikeshedding,
demanding benchmarks, rejecting the overall feature idea. CI YAML gets
comments only for merge debris.
## Voice
- **Mostly questions**: "Is there a reason…?", "Should this…?", "Would it make
sense to…?", "Why not…?", "Any reason not to…?", "Is this needed?",
"Intentional?"
- **Flat statements only for clear bugs**, with the reason: "This `if` will
never be true. You want `||` not `&&`." "Remove the noexcept. The library
shouldn't try to swallow the exception."
- **Terse when trivial**: "`emplace_back`", "typo: alse", "astyle, negation",
"as above", "Lost a blank line here."
- **Concrete sketches** in plain fenced code blocks introduced with "Maybe like
this?" / "Something like this:" / "Untested:". Rarely ```suggestion blocks.
- **Hedge taste**: "minor nit:", "Personally I prefer…", "I'm on the fence…",
"not something for which I would hold up the PR", "(If this was just moved,
ignore this.)", "not the fault of this PR".
- Cite evidence: cppreference, godbolt, the standard, related issues/PRs by
number, existing code ("see `NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS`").
- Light humor allowed sparingly (":)", "astyle strikes again", "Ugh").
- Praise is rare and short ("Good change.", "Looks reasonable.").
- Defer design calls: "@nlohmann has the final say here."
- Never invent facts about the code: if unsure, phrase it as a question rather than a claim.
## Example output
```
include/nlohmann/detail/input/parser.hpp:212
This invalidates `top`. Make it a `const` copy so it can't be read after the
`push_back` by a future change?
include/nlohmann/detail/input/parser.hpp:240
Should this reuse the existing recursion limit rather than adding a new one?
Different limits per operation invites them to diverge over time.
include/nlohmann/detail/macro_scope.hpp:88
If this has been externally defined, we should leave it as-is. This will break
if someone has `#define JSON_HAS_FOO 0`.
include/nlohmann/json.hpp:1534
`throw` → `JSON_THROW`, or this can't be used by people that disable exceptions.
single_include/nlohmann/json.hpp:9001
Looks like this was edited directly. Changes go in `include/` and then
`make amalgamate`.
docs/mkdocs/docs/api/basic_json/back.md:12
Should this list the precondition that the array is not empty?
```
@@ -13,9 +13,15 @@ function never throws an exception for invalid input, and the returned `#!cpp bo
is returned.
The result is the same as that of `BasicJsonType::accept()` by design; the difference is how the check is made. This
function runs the document's index builder (and discards the index), while `BasicJsonType::accept()` runs the SAX
parser with a handler that ignores every event. Use this one if you go on to parse the same input with
[`parse()`](parse.md) anyway, so that both use the same code; either is fine for a plain yes/no answer.
function runs the document's builder, which checks the text as [`parse()`](parse.md) does, while
`BasicJsonType::accept()` runs the SAX parser with a handler that ignores every event. Use this one if you go on to
parse the same input with [`parse()`](parse.md) anyway, so that both use the same code; either is fine for a plain
yes/no answer.
When both `ignore_comments` and `ignore_trailing_commas` are `#!cpp false` (the default), the builder writes no index.
With either option set, it builds the index as [`parse()`](parse.md) does and discards it: the result is the same, only
the time differs. (Neither option needs the index; the variant that skips it exists only for the default options to
keep the code size down.)
## Template parameters
+128 -19
View File
@@ -16,6 +16,7 @@
#include <cstring> // memcmp, memcpy
#include <limits> // numeric_limits
#include <string> // string
#include <type_traits> // false_type, integral_constant, true_type
#include <vector> // vector
#include <nlohmann/json.hpp>
@@ -69,7 +70,8 @@ struct parse_failure
};
/// FloatType: the number_float_t of the document, whose overflow parse() rejects
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd, bool Sentinel>
/// Store: false for accept(): the grammar, number, string and UTF-8 checks run as for parse(), but no node is written
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd, bool Sentinel, bool Store>
class builder
{
public:
@@ -245,6 +247,88 @@ class builder
return p;
}
/// the string from p, the first backslash, control character, ill-formed UTF-8 byte, or the end, checked and not
/// decoded (accept() stores nothing, so the arena is not written): the checks, errors and error positions of
/// slow_string(), for everything up to the closing quote
NLOHMANN_VIEW_NOINLINE decoded check_string(const unsigned char* p)
{
const unsigned char* const end = m_end;
for (;;)
{
if (p == end)
{
return failed(error_code::string_missing_quote, p);
}
const unsigned char c = *p;
if (c == '"')
{
++p;
return decoded{p, 0, 0};
}
if (c != '\\')
{
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == end)
{
return failed(error_code::string_missing_quote, p);
}
const unsigned char d = *p++;
if (d == 'u')
{
const int cp = hex4(p); // (moves p past the four digits)
if (NLOHMANN_VIEW_UNLIKELY(cp < 0))
{
return failed(error_code::string_unicode_hex, p);
}
if (NLOHMANN_VIEW_UNLIKELY((cp & 0xF800) == 0xD800)) // a surrogate
{
if (cp >= 0xDC00)
{
return failed(error_code::string_surrogate_low, p);
}
if (end - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
p += 2;
const int lo = hex4(p); // (moves p past the four digits)
if (lo < 0)
{
return failed(error_code::string_unicode_hex, p);
}
if (lo < 0xDC00 || lo > 0xDFFF)
{
return failed(error_code::string_surrogate_high, p);
}
}
}
else if (!(d == '"' || d == '\\' || d == '/' || d == 'b' || d == 'f' || d == 'n' || d == 'r' || d == 't'))
{
--p;
return failed(error_code::string_escape, p);
}
p = scan_string_run(p, end); // the text up to the next backslash, quote, control character, or ill-formed UTF-8
}
}
/// the string from p, decoded into the arena, or only checked (Store == false)
decoded decode_string(const unsigned char* s, const unsigned char* p)
{
return decode_string(s, p, std::integral_constant<bool, Store>());
}
decoded decode_string(const unsigned char* s, const unsigned char* p, std::true_type /*unused*/)
{
return slow_string(s, p);
}
decoded decode_string(const unsigned char* /*unused*/, const unsigned char* p, std::false_type /*unused*/)
{
return check_string(p);
}
/// s points to the start of the string, p to a backslash, a control
/// character, an ill-formed UTF-8 byte, or the end: decode the string into
/// the arena
@@ -610,10 +694,13 @@ run_found:
NLOHMANN_VIEW_ALWAYS_INLINE bool run()
{
cold.m_doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.m_doc.nodes;
out = base;
cap = base + cold.m_doc.node_capacity;
if (enabled(Store))
{
cold.m_doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.m_doc.nodes;
out = base;
cap = base + cold.m_doc.node_capacity;
}
if (e - p >= 3 && p[0] == 0xEF && p[1] == 0xBB && p[2] == 0xBF)
{
@@ -797,7 +884,10 @@ close_object:
// an array pays nothing for this)
if (NLOHMANN_VIEW_UNLIKELY(cur_count >= document_data::index_min_members))
{
cold.note_large_object(cur_idx);
if (enabled(Store))
{
cold.note_large_object(cur_idx);
}
}
close_container:
@@ -957,6 +1047,12 @@ indent_done:
/// an integer's value); two stores on little-endian targets
NLOHMANN_VIEW_ALWAYS_INLINE node* push_node(node_kind k, std::uint8_t flags, std::uint16_t extra, std::size_t off, std::uint64_t second)
{
if (!enabled(Store))
{
// accept(): nothing is stored, so what computes the arguments is not run either. base and out stay
// null: their difference, which open() takes, is 0.
return out;
}
if (NLOHMANN_VIEW_UNLIKELY(out == cap))
{
const auto n = static_cast<std::size_t>(out - base);
@@ -1015,9 +1111,12 @@ indent_done:
NLOHMANN_VIEW_ALWAYS_INLINE void close()
{
node& n = base[cur_idx];
n.len = cur_count;
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
if (enabled(Store))
{
node& n = base[cur_idx];
n.len = cur_count;
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
}
if (--depth != 0)
{
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
@@ -1172,7 +1271,7 @@ indent_done:
++p;
return true;
}
const decoded r = cold.slow_string(s, p);
const decoded r = cold.decode_string(s, p);
if (r.p == nullptr)
{
return false;
@@ -1192,34 +1291,44 @@ indent_done:
};
/// run the builder with compile-time options
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd>
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd, bool Store>
inline bool build_with(document_data& d, const char* src, std::size_t size, bool sentinel, parse_failure& failure)
{
if (sentinel)
{
builder<FloatType, Comments, TrailingCommas, NulIsEnd, true> bld(d, src, size);
builder<FloatType, Comments, TrailingCommas, NulIsEnd, true, Store> bld(d, src, size);
const bool ok = bld.run();
failure = bld.failure();
return ok;
}
builder<FloatType, Comments, TrailingCommas, NulIsEnd, false> bld(d, src, size);
builder<FloatType, Comments, TrailingCommas, NulIsEnd, false, Store> bld(d, src, size);
const bool ok = bld.run();
failure = bld.failure();
return ok;
}
/// sentinel: src[size] is readable and 0 (e.g. std::string); FloatType: the
/// number_float_t of the document
/// number_float_t of the document; validate_only: accept(), only the result is
/// used
template<typename FloatType, bool NulIsEnd>
inline bool build(document_data& d, const char* src, std::size_t size, bool comments, bool trailing_commas, bool sentinel, parse_failure& failure)
inline bool build(document_data& d, const char* src, std::size_t size, bool comments, bool trailing_commas, bool sentinel, bool validate_only,
parse_failure& failure)
{
if (validate_only && !comments && !trailing_commas)
{
// accept() of plain JSON: nothing is stored. The other option sets
// could skip the stores just as well; they keep the builder that
// stores only so that there are two more instantiations of the parse
// loop, not eight.
return build_with<FloatType, false, false, NulIsEnd, false>(d, src, size, sentinel, failure);
}
if (comments)
{
return trailing_commas ? build_with<FloatType, true, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, true, false, NulIsEnd>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, true, true, NulIsEnd, true>(d, src, size, sentinel, failure)
: build_with<FloatType, true, false, NulIsEnd, true>(d, src, size, sentinel, failure);
}
return trailing_commas ? build_with<FloatType, false, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, false, false, NulIsEnd>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, false, true, NulIsEnd, true>(d, src, size, sentinel, failure)
: build_with<FloatType, false, false, NulIsEnd, true>(d, src, size, sentinel, failure);
}
} // namespace view
+37 -19
View File
@@ -1060,7 +1060,7 @@ class basic_json_document
const bool ignore_trailing_commas = false)
{
basic_json_document d;
d.read_range(first, last, allow_exceptions, ignore_comments, ignore_trailing_commas);
d.read_range(first, last, error_mode(allow_exceptions), ignore_comments, ignore_trailing_commas);
return d;
}
@@ -1075,7 +1075,7 @@ class basic_json_document
basic_json_document d;
std::string buf;
append_input(buf, std::forward<InputType>(input));
d.build_owned(std::move(buf), allow_exceptions, ignore_comments, ignore_trailing_commas);
d.build_owned(std::move(buf), error_mode(allow_exceptions), ignore_comments, ignore_trailing_commas);
return d;
}
@@ -1088,7 +1088,8 @@ class basic_json_document
static bool accept(InputType&& input, const bool ignore_comments = false, const bool ignore_trailing_commas = false)
{
basic_json_document d;
d.read(std::forward<InputType>(input), false, ignore_comments, ignore_trailing_commas);
d.read_kind(std::forward<InputType>(input), on_error::validate, ignore_comments, ignore_trailing_commas,
std::integral_constant<detail::view::input_kind, detail::view::classify_input<InputType>::value> {});
return !d.is_discarded();
}
@@ -1104,7 +1105,7 @@ class basic_json_document
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
read_kind(std::forward<InputType>(input), allow_exceptions, ignore_comments, ignore_trailing_commas,
read_kind(std::forward<InputType>(input), error_mode(allow_exceptions), ignore_comments, ignore_trailing_commas,
std::integral_constant<detail::view::input_kind, detail::view::classify_input<InputType>::value> {});
}
@@ -1463,6 +1464,22 @@ class basic_json_document
return idx;
}
/// what a failed parse does
enum class on_error : std::uint8_t
{
discard, ///< the document is discarded
raise, ///< parse_error is thrown
validate ///< accept(): the document is discarded, and no node is stored while parsing
};
static on_error error_mode(bool allow_exceptions) noexcept
{
return allow_exceptions ? on_error::raise : on_error::discard;
}
/// NulIsEnd of the builder: a null byte ends the input unless strict_nul_handling
static constexpr bool nul_is_end = !detail::abi_config::strict_nul_handling;
/// create the storage (sized for the input) on first use
void ensure_data(const char* src, std::size_t size)
{
@@ -1474,7 +1491,7 @@ class basic_json_document
/// parse a buffer the document takes ownership of (it replaces the
/// previous owned copy)
void build_owned(std::string&& buf, bool allow_exceptions, bool comments, bool trailing_commas)
void build_owned(std::string&& buf, on_error allow_exceptions, bool comments, bool trailing_commas)
{
ensure_data(buf.data(), buf.size());
m_data->owned = std::move(buf);
@@ -1484,7 +1501,7 @@ class basic_json_document
/// parse a copy of the input; the copy is appended to the cleared owned
/// buffer, so that its capacity is reused
template<typename T>
void build_copy(T&& input, bool allow_exceptions, bool comments, bool trailing_commas)
void build_copy(T&& input, on_error allow_exceptions, bool comments, bool trailing_commas)
{
if (!m_data)
{
@@ -1505,7 +1522,7 @@ class basic_json_document
}
/// sentinel: src[size] is readable and 0 (std::string, C strings)
void build(const char* src, std::size_t size, bool allow_exceptions, bool comments, bool trailing_commas, bool owned, bool sentinel)
void build(const char* src, std::size_t size, on_error allow_exceptions, bool comments, bool trailing_commas, bool owned, bool sentinel)
{
ensure_data(src, size);
document_data& d = *m_data;
@@ -1532,7 +1549,8 @@ class basic_json_document
}
else
{
ok = detail::view::build < typename BasicJsonType::number_float_t, !detail::abi_config::strict_nul_handling > (d, src, size, comments, trailing_commas, sentinel, failure);
ok = detail::view::build<typename BasicJsonType::number_float_t, nul_is_end>(d, src, size, comments, trailing_commas, sentinel,
allow_exceptions == on_error::validate, failure);
}
if (NLOHMANN_VIEW_LIKELY(ok))
{
@@ -1545,7 +1563,7 @@ class basic_json_document
return;
}
std::vector<std::uint32_t>().swap(d.large_objects);
if (allow_exceptions)
if (allow_exceptions == on_error::raise)
{
detail::view::throw_parse_failure<BasicJsonType>(failure, src, size, comments, trailing_commas);
}
@@ -1553,13 +1571,13 @@ class basic_json_document
// --- input dispatch (see detail::view::input_kind) ---
void read_kind(std::string&& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::move_string> /*unused*/)
void read_kind(std::string&& s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::move_string> /*unused*/)
{
build_owned(std::move(s), ae, c, tc);
}
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::c_string> /*unused*/)
void read_kind(CharT* s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::c_string> /*unused*/)
{
if (s == nullptr)
{
@@ -1573,7 +1591,7 @@ class basic_json_document
}
template<typename Array>
void read_kind(Array& a, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::char_array> /*unused*/)
void read_kind(Array& a, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::char_array> /*unused*/)
{
using CharT = typename std::remove_cv<typename std::remove_extent<Array>::type>::type;
const std::size_t n = std::extent<Array>::value;
@@ -1584,7 +1602,7 @@ class basic_json_document
}
template<typename T>
void read_kind(const T& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::borrow_range> /*unused*/)
void read_kind(const T& s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::borrow_range> /*unused*/)
{
// std::basic_string guarantees data()[size()] == 0: use it as sentinel
const auto size = static_cast<std::size_t>(s.size());
@@ -1593,13 +1611,13 @@ class basic_json_document
}
template<typename T>
void read_kind(const T& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::copy_range> /*unused*/)
void read_kind(const T& s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::copy_range> /*unused*/)
{
build_copy(s, ae, c, tc);
}
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::wide_c_string> /*unused*/)
void read_kind(CharT* s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::wide_c_string> /*unused*/)
{
if (s == nullptr)
{
@@ -1611,13 +1629,13 @@ class basic_json_document
}
template<typename T>
void read_kind(T&& input, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::adapter> /*unused*/)
void read_kind(T&& input, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::adapter> /*unused*/)
{
build_copy(std::forward<T>(input), ae, c, tc);
}
template<typename IteratorType>
void read_range(IteratorType first, IteratorType last, bool ae, bool c, bool tc)
void read_range(IteratorType first, IteratorType last, on_error ae, bool c, bool tc)
{
using value_type = typename std::remove_cv<typename std::iterator_traits<IteratorType>::value_type>::type;
// borrow the range where the library's input adapter scans it as one
@@ -1628,14 +1646,14 @@ class basic_json_document
}
template<typename IteratorType>
void read_range_impl(IteratorType first, IteratorType last, bool ae, bool c, bool tc, std::true_type /*contiguous bytes*/)
void read_range_impl(IteratorType first, IteratorType last, on_error ae, bool c, bool tc, std::true_type /*contiguous bytes*/)
{
const auto size = static_cast<std::size_t>(std::distance(first, last));
build(size == 0 ? "" : reinterpret_cast<const char*>(&*first), size, ae, c, tc, false, size == 0); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename IteratorType>
void read_range_impl(IteratorType first, IteratorType last, bool ae, bool c, bool tc, std::false_type /*other*/)
void read_range_impl(IteratorType first, IteratorType last, on_error ae, bool c, bool tc, std::false_type /*other*/)
{
build_copy(detail::view::make_adapter_input(detail::input_adapter(first, last)), ae, c, tc);
}
+165 -38
View File
@@ -88,6 +88,7 @@
#include <cstring> // memcmp, memcpy
#include <limits> // numeric_limits
#include <string> // string
#include <type_traits> // false_type, integral_constant, true_type
#include <vector> // vector
// #include <nlohmann/json.hpp>
@@ -1311,7 +1312,8 @@ struct parse_failure
};
/// FloatType: the number_float_t of the document, whose overflow parse() rejects
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd, bool Sentinel>
/// Store: false for accept(): the grammar, number, string and UTF-8 checks run as for parse(), but no node is written
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd, bool Sentinel, bool Store>
class builder
{
public:
@@ -1487,6 +1489,88 @@ class builder
return p;
}
/// the string from p, the first backslash, control character, ill-formed UTF-8 byte, or the end, checked and not
/// decoded (accept() stores nothing, so the arena is not written): the checks, errors and error positions of
/// slow_string(), for everything up to the closing quote
NLOHMANN_VIEW_NOINLINE decoded check_string(const unsigned char* p)
{
const unsigned char* const end = m_end;
for (;;)
{
if (p == end)
{
return failed(error_code::string_missing_quote, p);
}
const unsigned char c = *p;
if (c == '"')
{
++p;
return decoded{p, 0, 0};
}
if (c != '\\')
{
return failed(c < 0x20 ? error_code::string_control_character : error_code::string_utf8, p);
}
++p;
if (p == end)
{
return failed(error_code::string_missing_quote, p);
}
const unsigned char d = *p++;
if (d == 'u')
{
const int cp = hex4(p); // (moves p past the four digits)
if (NLOHMANN_VIEW_UNLIKELY(cp < 0))
{
return failed(error_code::string_unicode_hex, p);
}
if (NLOHMANN_VIEW_UNLIKELY((cp & 0xF800) == 0xD800)) // a surrogate
{
if (cp >= 0xDC00)
{
return failed(error_code::string_surrogate_low, p);
}
if (end - p < 2 || p[0] != '\\' || p[1] != 'u')
{
return failed(error_code::string_surrogate_high, p);
}
p += 2;
const int lo = hex4(p); // (moves p past the four digits)
if (lo < 0)
{
return failed(error_code::string_unicode_hex, p);
}
if (lo < 0xDC00 || lo > 0xDFFF)
{
return failed(error_code::string_surrogate_high, p);
}
}
}
else if (!(d == '"' || d == '\\' || d == '/' || d == 'b' || d == 'f' || d == 'n' || d == 'r' || d == 't'))
{
--p;
return failed(error_code::string_escape, p);
}
p = scan_string_run(p, end); // the text up to the next backslash, quote, control character, or ill-formed UTF-8
}
}
/// the string from p, decoded into the arena, or only checked (Store == false)
decoded decode_string(const unsigned char* s, const unsigned char* p)
{
return decode_string(s, p, std::integral_constant<bool, Store>());
}
decoded decode_string(const unsigned char* s, const unsigned char* p, std::true_type /*unused*/)
{
return slow_string(s, p);
}
decoded decode_string(const unsigned char* /*unused*/, const unsigned char* p, std::false_type /*unused*/)
{
return check_string(p);
}
/// s points to the start of the string, p to a backslash, a control
/// character, an ill-formed UTF-8 byte, or the end: decode the string into
/// the arena
@@ -1852,10 +1936,13 @@ run_found:
NLOHMANN_VIEW_ALWAYS_INLINE bool run()
{
cold.m_doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.m_doc.nodes;
out = base;
cap = base + cold.m_doc.node_capacity;
if (enabled(Store))
{
cold.m_doc.reserve(estimate_nodes(reinterpret_cast<const char*>(b), static_cast<std::size_t>(e - b)));
base = cold.m_doc.nodes;
out = base;
cap = base + cold.m_doc.node_capacity;
}
if (e - p >= 3 && p[0] == 0xEF && p[1] == 0xBB && p[2] == 0xBF)
{
@@ -2039,7 +2126,10 @@ close_object:
// an array pays nothing for this)
if (NLOHMANN_VIEW_UNLIKELY(cur_count >= document_data::index_min_members))
{
cold.note_large_object(cur_idx);
if (enabled(Store))
{
cold.note_large_object(cur_idx);
}
}
close_container:
@@ -2199,6 +2289,12 @@ indent_done:
/// an integer's value); two stores on little-endian targets
NLOHMANN_VIEW_ALWAYS_INLINE node* push_node(node_kind k, std::uint8_t flags, std::uint16_t extra, std::size_t off, std::uint64_t second)
{
if (!enabled(Store))
{
// accept(): nothing is stored, so what computes the arguments is not run either. base and out stay
// null: their difference, which open() takes, is 0.
return out;
}
if (NLOHMANN_VIEW_UNLIKELY(out == cap))
{
const auto n = static_cast<std::size_t>(out - base);
@@ -2257,9 +2353,12 @@ indent_done:
NLOHMANN_VIEW_ALWAYS_INLINE void close()
{
node& n = base[cur_idx];
n.len = cur_count;
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
if (enabled(Store))
{
node& n = base[cur_idx];
n.len = cur_count;
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
}
if (--depth != 0)
{
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
@@ -2414,7 +2513,7 @@ indent_done:
++p;
return true;
}
const decoded r = cold.slow_string(s, p);
const decoded r = cold.decode_string(s, p);
if (r.p == nullptr)
{
return false;
@@ -2434,34 +2533,44 @@ indent_done:
};
/// run the builder with compile-time options
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd>
template<typename FloatType, bool Comments, bool TrailingCommas, bool NulIsEnd, bool Store>
inline bool build_with(document_data& d, const char* src, std::size_t size, bool sentinel, parse_failure& failure)
{
if (sentinel)
{
builder<FloatType, Comments, TrailingCommas, NulIsEnd, true> bld(d, src, size);
builder<FloatType, Comments, TrailingCommas, NulIsEnd, true, Store> bld(d, src, size);
const bool ok = bld.run();
failure = bld.failure();
return ok;
}
builder<FloatType, Comments, TrailingCommas, NulIsEnd, false> bld(d, src, size);
builder<FloatType, Comments, TrailingCommas, NulIsEnd, false, Store> bld(d, src, size);
const bool ok = bld.run();
failure = bld.failure();
return ok;
}
/// sentinel: src[size] is readable and 0 (e.g. std::string); FloatType: the
/// number_float_t of the document
/// number_float_t of the document; validate_only: accept(), only the result is
/// used
template<typename FloatType, bool NulIsEnd>
inline bool build(document_data& d, const char* src, std::size_t size, bool comments, bool trailing_commas, bool sentinel, parse_failure& failure)
inline bool build(document_data& d, const char* src, std::size_t size, bool comments, bool trailing_commas, bool sentinel, bool validate_only,
parse_failure& failure)
{
if (validate_only && !comments && !trailing_commas)
{
// accept() of plain JSON: nothing is stored. The other option sets
// could skip the stores just as well; they keep the builder that
// stores only so that there are two more instantiations of the parse
// loop, not eight.
return build_with<FloatType, false, false, NulIsEnd, false>(d, src, size, sentinel, failure);
}
if (comments)
{
return trailing_commas ? build_with<FloatType, true, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, true, false, NulIsEnd>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, true, true, NulIsEnd, true>(d, src, size, sentinel, failure)
: build_with<FloatType, true, false, NulIsEnd, true>(d, src, size, sentinel, failure);
}
return trailing_commas ? build_with<FloatType, false, true, NulIsEnd>(d, src, size, sentinel, failure)
: build_with<FloatType, false, false, NulIsEnd>(d, src, size, sentinel, failure);
return trailing_commas ? build_with<FloatType, false, true, NulIsEnd, true>(d, src, size, sentinel, failure)
: build_with<FloatType, false, false, NulIsEnd, true>(d, src, size, sentinel, failure);
}
} // namespace view
@@ -10101,7 +10210,7 @@ class basic_json_document
const bool ignore_trailing_commas = false)
{
basic_json_document d;
d.read_range(first, last, allow_exceptions, ignore_comments, ignore_trailing_commas);
d.read_range(first, last, error_mode(allow_exceptions), ignore_comments, ignore_trailing_commas);
return d;
}
@@ -10116,7 +10225,7 @@ class basic_json_document
basic_json_document d;
std::string buf;
append_input(buf, std::forward<InputType>(input));
d.build_owned(std::move(buf), allow_exceptions, ignore_comments, ignore_trailing_commas);
d.build_owned(std::move(buf), error_mode(allow_exceptions), ignore_comments, ignore_trailing_commas);
return d;
}
@@ -10129,7 +10238,8 @@ class basic_json_document
static bool accept(InputType&& input, const bool ignore_comments = false, const bool ignore_trailing_commas = false)
{
basic_json_document d;
d.read(std::forward<InputType>(input), false, ignore_comments, ignore_trailing_commas);
d.read_kind(std::forward<InputType>(input), on_error::validate, ignore_comments, ignore_trailing_commas,
std::integral_constant<detail::view::input_kind, detail::view::classify_input<InputType>::value> {});
return !d.is_discarded();
}
@@ -10145,7 +10255,7 @@ class basic_json_document
const bool ignore_comments = false,
const bool ignore_trailing_commas = false)
{
read_kind(std::forward<InputType>(input), allow_exceptions, ignore_comments, ignore_trailing_commas,
read_kind(std::forward<InputType>(input), error_mode(allow_exceptions), ignore_comments, ignore_trailing_commas,
std::integral_constant<detail::view::input_kind, detail::view::classify_input<InputType>::value> {});
}
@@ -10504,6 +10614,22 @@ class basic_json_document
return idx;
}
/// what a failed parse does
enum class on_error : std::uint8_t
{
discard, ///< the document is discarded
raise, ///< parse_error is thrown
validate ///< accept(): the document is discarded, and no node is stored while parsing
};
static on_error error_mode(bool allow_exceptions) noexcept
{
return allow_exceptions ? on_error::raise : on_error::discard;
}
/// NulIsEnd of the builder: a null byte ends the input unless strict_nul_handling
static constexpr bool nul_is_end = !detail::abi_config::strict_nul_handling;
/// create the storage (sized for the input) on first use
void ensure_data(const char* src, std::size_t size)
{
@@ -10515,7 +10641,7 @@ class basic_json_document
/// parse a buffer the document takes ownership of (it replaces the
/// previous owned copy)
void build_owned(std::string&& buf, bool allow_exceptions, bool comments, bool trailing_commas)
void build_owned(std::string&& buf, on_error allow_exceptions, bool comments, bool trailing_commas)
{
ensure_data(buf.data(), buf.size());
m_data->owned = std::move(buf);
@@ -10525,7 +10651,7 @@ class basic_json_document
/// parse a copy of the input; the copy is appended to the cleared owned
/// buffer, so that its capacity is reused
template<typename T>
void build_copy(T&& input, bool allow_exceptions, bool comments, bool trailing_commas)
void build_copy(T&& input, on_error allow_exceptions, bool comments, bool trailing_commas)
{
if (!m_data)
{
@@ -10546,7 +10672,7 @@ class basic_json_document
}
/// sentinel: src[size] is readable and 0 (std::string, C strings)
void build(const char* src, std::size_t size, bool allow_exceptions, bool comments, bool trailing_commas, bool owned, bool sentinel)
void build(const char* src, std::size_t size, on_error allow_exceptions, bool comments, bool trailing_commas, bool owned, bool sentinel)
{
ensure_data(src, size);
document_data& d = *m_data;
@@ -10573,7 +10699,8 @@ class basic_json_document
}
else
{
ok = detail::view::build < typename BasicJsonType::number_float_t, !detail::abi_config::strict_nul_handling > (d, src, size, comments, trailing_commas, sentinel, failure);
ok = detail::view::build<typename BasicJsonType::number_float_t, nul_is_end>(d, src, size, comments, trailing_commas, sentinel,
allow_exceptions == on_error::validate, failure);
}
if (NLOHMANN_VIEW_LIKELY(ok))
{
@@ -10586,7 +10713,7 @@ class basic_json_document
return;
}
std::vector<std::uint32_t>().swap(d.large_objects);
if (allow_exceptions)
if (allow_exceptions == on_error::raise)
{
detail::view::throw_parse_failure<BasicJsonType>(failure, src, size, comments, trailing_commas);
}
@@ -10594,13 +10721,13 @@ class basic_json_document
// --- input dispatch (see detail::view::input_kind) ---
void read_kind(std::string&& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::move_string> /*unused*/)
void read_kind(std::string&& s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::move_string> /*unused*/)
{
build_owned(std::move(s), ae, c, tc);
}
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::c_string> /*unused*/)
void read_kind(CharT* s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::c_string> /*unused*/)
{
if (s == nullptr)
{
@@ -10614,7 +10741,7 @@ class basic_json_document
}
template<typename Array>
void read_kind(Array& a, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::char_array> /*unused*/)
void read_kind(Array& a, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::char_array> /*unused*/)
{
using CharT = typename std::remove_cv<typename std::remove_extent<Array>::type>::type;
const std::size_t n = std::extent<Array>::value;
@@ -10625,7 +10752,7 @@ class basic_json_document
}
template<typename T>
void read_kind(const T& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::borrow_range> /*unused*/)
void read_kind(const T& s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::borrow_range> /*unused*/)
{
// std::basic_string guarantees data()[size()] == 0: use it as sentinel
const auto size = static_cast<std::size_t>(s.size());
@@ -10634,13 +10761,13 @@ class basic_json_document
}
template<typename T>
void read_kind(const T& s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::copy_range> /*unused*/)
void read_kind(const T& s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::copy_range> /*unused*/)
{
build_copy(s, ae, c, tc);
}
template<typename CharT>
void read_kind(CharT* s, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::wide_c_string> /*unused*/)
void read_kind(CharT* s, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::wide_c_string> /*unused*/)
{
if (s == nullptr)
{
@@ -10652,13 +10779,13 @@ class basic_json_document
}
template<typename T>
void read_kind(T&& input, bool ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::adapter> /*unused*/)
void read_kind(T&& input, on_error ae, bool c, bool tc, std::integral_constant<input_kind, input_kind::adapter> /*unused*/)
{
build_copy(std::forward<T>(input), ae, c, tc);
}
template<typename IteratorType>
void read_range(IteratorType first, IteratorType last, bool ae, bool c, bool tc)
void read_range(IteratorType first, IteratorType last, on_error ae, bool c, bool tc)
{
using value_type = typename std::remove_cv<typename std::iterator_traits<IteratorType>::value_type>::type;
// borrow the range where the library's input adapter scans it as one
@@ -10669,14 +10796,14 @@ class basic_json_document
}
template<typename IteratorType>
void read_range_impl(IteratorType first, IteratorType last, bool ae, bool c, bool tc, std::true_type /*contiguous bytes*/)
void read_range_impl(IteratorType first, IteratorType last, on_error ae, bool c, bool tc, std::true_type /*contiguous bytes*/)
{
const auto size = static_cast<std::size_t>(std::distance(first, last));
build(size == 0 ? "" : reinterpret_cast<const char*>(&*first), size, ae, c, tc, false, size == 0); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
template<typename IteratorType>
void read_range_impl(IteratorType first, IteratorType last, bool ae, bool c, bool tc, std::false_type /*other*/)
void read_range_impl(IteratorType first, IteratorType last, on_error ae, bool c, bool tc, std::false_type /*other*/)
{
build_copy(detail::view::make_adapter_input(detail::input_adapter(first, last)), ae, c, tc);
}
+130 -1
View File
@@ -59,7 +59,7 @@ built build(const std::string& text, bool comments, bool trailing_commas, bool s
r.copy.assign(text.begin(), text.end());
src = r.copy.data();
}
r.ok = nlohmann::detail::view::build < FloatType, !nlohmann::detail::abi_config::strict_nul_handling > (*r.data, src, text.size(), comments, trailing_commas, sentinel, r.failure);
r.ok = nlohmann::detail::view::build < FloatType, !nlohmann::detail::abi_config::strict_nul_handling > (*r.data, src, text.size(), comments, trailing_commas, sentinel, false, r.failure);
r.data->src = src;
r.data->base[0] = src;
r.data->base[1] = r.data->arena.data();
@@ -172,6 +172,22 @@ void check_string(const std::string& content)
}
}
// accept() stores no nodes: it must accept and reject what json::accept does, for all options and for inputs with
// and without a terminating NUL
void check_accept(const std::string& text)
{
CAPTURE(text)
const std::vector<char> bytes(text.begin(), text.end());
for (int options = 0; options < 4; ++options)
{
const bool comments = (options & 1) != 0;
const bool trailing_commas = (options & 2) != 0;
const bool accepted = json::accept(text, comments, trailing_commas);
CHECK(nlohmann::json_document::accept(text, comments, trailing_commas) == accepted);
CHECK(nlohmann::json_document::accept(bytes, comments, trailing_commas) == accepted);
}
}
// a small deterministic generator of documents
struct generator
{
@@ -423,6 +439,119 @@ TEST_CASE("json_view builder")
}
}
SECTION("accept() without an index")
{
for (const char* text :
{
"", " ", "\n", "null", "nul", "true", "[", "]", "[]", "{}", "[1,2,3]", "[1,2,3", "[1,2,3,]", "{\"a\":1,}", "{\"a\"}", "{\"a\":}",
"[1] x", "[1][2]", "\xEF\xBB\xBF[1]", "[\"a\\", "[\"\\u12\"]", "[\"\\ud800\"]", "[\"\x01\"]", "[\"\xff\"]", "[01]", "[1.]",
"[.5]", "[-]", "[1e]", "// c\n[1]", "/* c */ {\"a\" /* x */ : 1}", "/* c", "[1] // c", "1e400", "-1e400", "123456789012345678901234567890",
"{\"a\":1,\"a\":2}", "[[[]]]", "\"\\u00e4\\n\\ud83d\\ude00\""
})
{
check_accept(text);
}
// a null byte inside the text, and after it
check_accept(std::string("[1,\0]", 5));
check_accept(std::string("[1]\0", 4));
check_accept(std::string("\0", 1));
// generated documents and damaged copies, as for the builder above
generator g;
for (int i = 0; i < 2000; ++i)
{
std::string text;
g.value(text, 0);
check_accept(text);
std::string damaged = text;
const auto at = static_cast<std::size_t>(g.r(static_cast<int>(damaged.size())));
static const char replacements[] = {'x', '"', '\\', ',', ':', ']', '}', '[', '{', '1', '-', '.', 'e', '\0', '\n', '/'}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
damaged[at] = replacements[g.r(16)];
check_accept(damaged);
check_accept(text.substr(0, at));
}
// nesting beyond the 64 levels that are kept on the builder's own stack, closed and not closed
for (const std::size_t depth :
{
63, 64, 65, 66, 200, 1000
})
{
check_accept(std::string(depth, '[') + std::string(depth, ']'));
check_accept(std::string(depth, '[') + std::string(depth - 1, ']'));
std::string objects;
for (std::size_t i = 0; i < depth; ++i)
{
objects += "{\"k\":";
}
objects += "1" + std::string(depth, '}');
check_accept(objects);
check_accept(objects.substr(0, objects.size() - 1));
}
// objects with enough members to get a hash index when they are indexed (and duplicate keys)
std::string large = "{";
for (int i = 0; i < 300; ++i)
{
large += (i == 0 ? "\"k" : ",\"k") + std::to_string(i % 200) + "\":" + std::to_string(i);
}
check_accept(large + "}");
check_accept(large);
check_accept("[" + large + "},{" + large.substr(1) + "}]");
// the overflow check uses the floating-point type of the document
using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
for (const char* text :
{
"1e39", "-1e39", "3.4028235e38", "3.4028236e38", "[1.5e38, 3.5e38]", "{\"a\": 1e-50, \"b\": 1e39}", "340282356779733661637539395458142568448"
})
{
CAPTURE(text)
CHECK(nlohmann::basic_json_document<float_json>::accept(text) == float_json::accept(text));
}
// strings are checked and not decoded: runs of text of every length from 0 to 40 between valid and invalid
// escapes, UTF-8 sequences and control characters, in values and in keys, cut near the end of the text
const std::array<const char*, 9> valid = {{"\\n", "\\\"", "\\\\", "\\/", "\\t", "\\u00e4", "\\u20ac", "\\ud83d\\ude00", "\\u0000"}};
const std::array<const char*, 12> invalid = {{"\\x", "\\ud800", "\\udc00", "\\u12G4", "\\u12", "\\ud83dx", "\\ud83d\\u0041", "\\ud83d\\ud83d", "\\", "\x01", "\x1f", "\x7f"}};
const std::array<const char*, 6> utf8 = {{"\xc3\xa4", "\xe2\x82\xac", "\xf0\x9f\x98\x80", "\xc3", "\xff", "\x80"}};
std::mt19937 rng(2029); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
for (std::size_t run = 0; run <= 40; ++run)
{
for (int i = 0; i < 30; ++i)
{
std::string content = std::string(rng() % 20, 'a');
const auto pieces = 1 + (rng() % 6);
for (unsigned k = 0; k < pieces; ++k)
{
content += valid[rng() % valid.size()];
content += std::string(k == 0 ? run : rng() % 41, static_cast<char>('b' + (rng() % 20)));
switch (rng() % 8)
{
case 0:
content += utf8[rng() % 3];
break;
case 1:
content += utf8[rng() % utf8.size()];
break;
case 2:
content += invalid[rng() % invalid.size()];
break;
default:
break;
}
}
for (const std::string& text :
{
"[\"" + content + "\"]", "{\"" + content + "\":1}", "[\"" + content, "[\"" + content.substr(0, content.size() - (rng() % 4)) + "\"]"
})
{
check_accept(text);
}
}
}
}
SECTION("test files")
{
for (const char* name :