Compare commits

..
Author SHA1 Message Date
Niels Lohmann d705cf0b63 Move dump_ascii() and dump_pretty() next to dump()
Address a review comment: the two dispatchers that dump() calls now follow
it directly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-23 06:45:57 +02:00
Niels Lohmann 2cf9df3eae Fix the clang-tidy findings in json_view's dump and its tests
- the SSE2 loop of the string scan is within NOLINTBEGIN/END
  (portability-simd-intrinsics), as in to_chars.hpp
- depth_counter<false>'s leave() and get() keep the interface of
  depth_counter<true> instead of being static
- write_ascii() compares the room left as a size_t variable
- the test's texts are a std::array, and a JSON fragment with escapes
  is a raw string literal

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 19:46:31 +02:00
Niels Lohmann 86049706a0 Rename depth_counter's open() and close() to enter() and leave()
Flawfinder took them for the file functions open() and close().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 13:18:11 +02:00
Niels Lohmann cc946adf68 Find the end of the plain run of a decoded string 16 bytes at a time with SSE2
The strings of a document that had escape sequences are written in runs
without a quote, backslash, or control character; the compact dump found the
end of a run 8 bytes at a time with a SWAR test. With SSE2 (x86-64) it now
compares 16 bytes per step. NEON keeps the SWAR code: there the mask of a
step costs more than it saves on the short runs that are typical here.

Instructions of dump(), Linux (GCC 13 / Clang 18):

  twitter.json                  1.99M -> 1.88M (-5.2 %) / 1.97M -> 1.92M (-3.0 %)
  CJK text with \u escapes      5.98M -> 4.52M (-24.5 %) / 5.30M -> 4.25M (-19.9 %)
  ASCII text with a few escapes 4.28M -> 4.09M (-4.4 %) / 4.24M -> 4.11M (-3.2 %)
  jeopardy (4,000 questions)    5.08M -> 4.78M (-5.8 %) / 4.97M -> 4.80M (-3.4 %)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:30:39 +02:00
Niels Lohmann 272d07b48b Scan the strings of json_view's dump() with ensure_ascii 16 bytes at a time
With ensure_ascii, a string of the source that has a byte >= 0x7F was written
by a generic path: put() per piece and a validate-and-decode step for every
character beyond ASCII. It now goes through a writer with the cursor in a
local variable. The part of the string that has nothing to escape is found
16 bytes at a time (SSE2 or NEON; the existing 8-byte scan without vector
support) and copied; strings that are ASCII throughout are still decided by
the 8-byte check of dump() with ensure_ascii (a 16-byte scan for them was
2-5 % slower for short strings). The two-byte sequences (lead C2..DF) and
the three-byte sequences whose continuation bytes may have any value (lead
E1..EF except ED) are decoded with a few compares; E0, ED, four-byte
sequences, and damaged input go through validate_one_utf8() as before, so
damaged images still throw type_error.316.

dump(-1, ' ', true) instructions per call, macOS arm64 (develop -> now);
the compact dump without ensure_ascii and dumps of ASCII text are unchanged:

  twitter 5.42M -> 3.57M    citm 3.28M -> 3.24M     jeopardy(4k) 5.78M -> 5.65M
  CJK text, raw 66.5M -> 35.3M    escaped \u 32.7M -> 17.0M
  Latin text with accents 11.5M -> 9.4M    ASCII text 4.40M -> 4.40M

Linux instructions (GCC 13 / Clang 18), dump(-1, ' ', true):

  twitter -9 % / -33 %    CJK text -14 % / -47 %    Latin text -8 % / -18 %
  citm -0.2 % / -0.4 %    ASCII text -0.1 % / -0.3 %

The compact dump without ensure_ascii is unchanged (canada 0.0 %).

The unit test covers every character of the Basic Multilingual Plane, and a
character that needs an escape at each position of strings of 0..40 bytes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:30:36 +02:00
Niels Lohmann ef45593bda Write the indented output of json_view's dump() with the fast writer
The compact writer of dump() (cursor in a register, fixed-size copies of
tokens) served dump(-1) only; every dump(indent), and everything with
ensure_ascii that was indented, went through a generic walk that calls put()
for each token and put_repeated() (a memset call) for each line break. The
fast writer now writes indented output, too, with or without ensure_ascii, and
the generic walk is gone. A line break is a copy of a 64-byte ruler (a line
break and 63 indentation characters) when the line is at most 64 bytes, and a
store plus memset beyond that. The depth is counted only in the indented
instantiations (counting it in the compact one cost an editable document 5 %
with Clang). The indented walks are out of line (dump_pretty()): inlined
into dump() next to the compact ones, they made compact dumps of integer
arrays 7 % slower.

A decoded string (one with escape sequences) of a damaged image loaded with
image_check::bounds can contain invalid UTF-8. As before, the compact dump
writes such bytes unchanged, and the indented dump throws type_error.316; its
runs are found by the lexer's validating scan (a scalar validation of every
sequence made dump(2) of decoded CJK text twice as slow as before).

dump(2) instructions per call, macOS arm64 (before -> after):

               twitter         citm            canada          jeopardy(4k)
  json         3.75M -> 1.85M  8.28M -> 3.93M  67.1M -> 56.0M  9.56M -> 4.81M
  editable     3.82M -> 1.84M  8.48M -> 4.02M  68.1M -> 56.2M  9.66M -> 4.75M

Linux instructions (GCC 13 / Clang 18), dump(2) and dump(2, ' ', true):

  twitter      -25 % / -30 %      -11 % / -11 %
  citm         -29 % / -28 %      -24 % / -24 %
  jeopardy(4k) -25 % / -32 %      -20 % / -24 %
  CJK text with \u escapes  -6 % / -11 %   -2 % / -2 %

The compact dump is unchanged (within +0.4 % on Linux, within noise on the Mac).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-22 09:30:34 +02:00
49 changed files with 4842 additions and 10042 deletions

No files matched your search

+3 -3
View File
@@ -38,14 +38,14 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/init@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
with:
languages: c-cpp
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/autobuild@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/analyze@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
@@ -37,14 +37,6 @@ jobs:
var matchArtifact = artifacts.data.artifacts.filter((artifact) => {
return artifact.name == "pr"
})[0];
// a run that failed before its jobs started (e.g. a stale queued
// run cancelled long after its push) has no artifact to report on
if (!matchArtifact) {
core.notice('Run ${{ github.event.workflow_run.id }} has no "pr" artifact; nothing to comment on.');
core.setOutput('found', 'false');
return;
}
core.setOutput('found', 'true');
var download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
@@ -59,11 +51,9 @@ jobs:
# Extract the untrusted PR artifact into a dedicated empty directory and
# read only the two expected files by fixed path afterwards. This avoids a
# malicious archive overwriting workspace files or escaping via ../ paths.
- if: steps.download.outputs.found == 'true'
run: unzip -o pr.zip -d ./pr_artifact
- run: unzip -o pr.zip -d ./pr_artifact
- name: 'Comment on PR'
if: steps.download.outputs.found == 'true'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
+1 -1
View File
@@ -47,6 +47,6 @@ jobs:
output: 'flawfinder_results.sarif'
- name: Upload analysis results to GitHub Security tab
uses: github/codeql-action/upload-sarif@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/upload-sarif@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
with:
sarif_file: ${{github.workspace}}/flawfinder_results.sarif
+2 -2
View File
@@ -54,7 +54,7 @@ jobs:
timeout-minutes: 60
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
with:
egress-policy: audit
@@ -78,7 +78,7 @@ jobs:
- name: Summary
run: cat tests/benchmarks/json_view/results/*.md >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: json_view-benchmarks-${{ matrix.runner }}-${{ inputs.compiler || 'g++' }}
path: tests/benchmarks/json_view/results/
+1 -1
View File
@@ -80,6 +80,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/upload-sarif@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
with:
sarif_file: results.sarif
+1 -1
View File
@@ -65,7 +65,7 @@ jobs:
# Upload SARIF file generated in previous step
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/upload-sarif@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
with:
sarif_file: semgrep.sarif
if: always()
+1 -2
View File
@@ -341,14 +341,13 @@ def download(url, docs) -> str:
raise ValueError(f'not an http(s) URL: {url}')
req = urllib.request.Request(u.geturl(), headers={'User-Agent': USER_AGENT})
# (the scheme is checked above)
# nosemgrep: python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected
with urllib.request.urlopen(req, timeout=20) as r: # nosec B310
data = r.read()
ctype = r.headers.get_content_type()
path = urllib.parse.unquote(u.path).lstrip('/')
ext = os.path.splitext(path)[1]
if u.query or not ext or path.endswith('/'):
digest = hashlib.sha256(url.encode()).hexdigest()[:12]
digest = hashlib.sha1(url.encode(), usedforsecurity=False).hexdigest()[:12]
path = os.path.join(os.path.dirname(path), digest + CONTENT_TYPE_EXT.get(ctype, ext or '.bin'))
rel = os.path.normpath(os.path.join('assets', 'external', u.hostname, path))
out = os.path.join(docs, rel)
-5
View File
@@ -339,11 +339,6 @@ The `serializer` (used by [`dump`](../api/basic_json/dump.md) and [`operator<<`]
- `output_stream_adapter` writes to a `std::ostream`.
- `output_string_adapter` appends to a string.
The `binary_writer` does not call an adapter for every byte. The `to_*` functions that return a `std::vector` write
into the vector through a sink (`output_vector_sink`) that is not virtual. The functions that take an output adapter
write through a sink (`output_adapter_sink`) that collects the output in a buffer of one kibibyte and passes it to the
adapter in blocks, so that the adapter is called (and a stream locked) once per block instead of once per value.
## Value conversion
Values are converted from and to other types with the `JSONSerializer` template parameter. The default,
+12 -12
View File
@@ -6,7 +6,7 @@
"": {
"name": "check-mermaid",
"dependencies": {
"jsdom": "30.1.3",
"jsdom": "30.1.2",
"mermaid": "11.17.2"
}
},
@@ -40,9 +40,9 @@
}
},
"node_modules/@asamuzakjp/dom-selector": {
"version": "10.0.3",
"resolved": "https://registry.npmjs.org/@asamuzakjp/dom-selector/-/dom-selector-10.0.3.tgz",
"integrity": "sha512-3+777rEPsOOjoy5/sRmIzHnoRDaLpm3XWY2dmWn/N7f/XwDG0bcpnFaKL3X9FYJUbNwLmpcJC1bUYexJRxsFlw==",
"version": "9.2.4",
"resolved": "https://registry.npmjs.org/@asamuzakjp/dom-selector/-/dom-selector-9.2.4.tgz",
"integrity": "sha512-YnVzxDxLaqL0t7Q4wfcgHZjg55Wm6y5EaO7Bhu3q9J7wq/8iw3PhsnWiOQhYekx4VexnbkgWpTT6n+QIV7kczw==",
"license": "MIT",
"dependencies": {
"bidi-js": "^1.1.0",
@@ -1202,13 +1202,13 @@
"license": "MIT"
},
"node_modules/jsdom": {
"version": "30.1.3",
"resolved": "https://registry.npmjs.org/jsdom/-/jsdom-30.1.3.tgz",
"integrity": "sha512-hQMUPvRgN7KxsfIytDQji7bimuJeRQa6Qo5+iuiZQk5CafeRxLIA10/cG5YCBFNpD9RO3oqIiN4CO1Y44kxqXg==",
"version": "30.1.2",
"resolved": "https://registry.npmjs.org/jsdom/-/jsdom-30.1.2.tgz",
"integrity": "sha512-0FFE/jE1rppmVfUrJUgxqXjcwolZYIGtAgj1pTussMhNZxIxi7W/PvfWaMNroGi2B36X1IKHbexpZ9DhkoOPiQ==",
"license": "MIT",
"dependencies": {
"@asamuzakjp/css-color": "^7.1.3",
"@asamuzakjp/dom-selector": "^10.0.3",
"@asamuzakjp/dom-selector": "^9.2.4",
"@bramus/specificity": "^2.4.2",
"@csstools/css-syntax-patches-for-csstree": "^1.1.15",
"@exodus/bytes": "^1.16.0",
@@ -1221,7 +1221,7 @@
"parse5": "^8.0.1",
"saxes": "^6.0.0",
"tough-cookie": "^6.0.2",
"undici": "^8.12.1",
"undici": "^8.11.2",
"w3c-xmlserializer": "^6.0.0",
"webidl-conversions": "^8.0.1",
"whatwg-mimetype": "^5.0.0",
@@ -1521,9 +1521,9 @@
}
},
"node_modules/undici": {
"version": "8.12.2",
"resolved": "https://registry.npmjs.org/undici/-/undici-8.12.2.tgz",
"integrity": "sha512-XtuDWVdqTLr3NAxm2ly9/hU6kUsJOgGh1RASzMBbsw6aWCOaQzHdW+QQDfdERjfJ2V5vL5pRPa4RetO9UfivDg==",
"version": "8.11.2",
"resolved": "https://registry.npmjs.org/undici/-/undici-8.11.2.tgz",
"integrity": "sha512-u4UB2/IrKdU6lFxumHmmo1a3fCQO5tzQllRorfoRS63txhrB7xTpSn1PftwC4qEHkOaqP95fCWW4lJzwErwzhQ==",
"license": "MIT",
"engines": {
"node": ">=22.19.0"
+1 -1
View File
@@ -4,7 +4,7 @@
"description": "Validate the Mermaid diagrams of the documentation (see check_mermaid.mjs)",
"type": "module",
"dependencies": {
"jsdom": "30.1.3",
"jsdom": "30.1.2",
"mermaid": "11.17.2"
}
}
@@ -8,7 +8,7 @@
#pragma once
#include <algorithm> // min, transform
#include <algorithm> // transform
#include <array> // array
#include <forward_list> // forward_list
#include <iterator> // inserter, front_inserter, end
@@ -69,10 +69,12 @@ void from_json(const BasicJsonType& j, std::optional<T>& opt)
}
#endif // JSON_HAS_CPP_17
// read a number value of any of the three number types into val;
// returns false (leaving val untouched) if j is not a number
template<typename BasicJsonType, typename ArithmeticType>
bool get_number_value(const BasicJsonType& j, ArithmeticType& val)
// overloads for basic_json template parameters
template < typename BasicJsonType, typename ArithmeticType,
enable_if_t < std::is_arithmetic<ArithmeticType>::value&&
!std::is_same<ArithmeticType, typename BasicJsonType::boolean_t>::value,
int > = 0 >
void get_arithmetic_value(const BasicJsonType& j, ArithmeticType& val)
{
switch (static_cast<value_t>(j))
{
@@ -83,17 +85,17 @@ bool get_number_value(const BasicJsonType& j, ArithmeticType& val)
const auto* ptr = j.template get_ptr<const typename BasicJsonType::number_unsigned_t*>();
JSON_ASSERT(ptr != nullptr);
val = static_cast<ArithmeticType>(*ptr);
return true;
break;
}
case value_t::number_integer:
{
val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_integer_t*>());
return true;
break;
}
case value_t::number_float:
{
val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_float_t*>());
return true;
break;
}
case value_t::null:
@@ -104,19 +106,6 @@ bool get_number_value(const BasicJsonType& j, ArithmeticType& val)
case value_t::binary:
case value_t::discarded:
default:
return false;
}
}
// overloads for basic_json template parameters
template < typename BasicJsonType, typename ArithmeticType,
enable_if_t < std::is_arithmetic<ArithmeticType>::value&&
!std::is_same<ArithmeticType, typename BasicJsonType::boolean_t>::value,
int > = 0 >
void get_arithmetic_value(const BasicJsonType& j, ArithmeticType& val)
{
if (JSON_HEDLEY_UNLIKELY(!get_number_value(j, val)))
{
throw_type_must_be("number", j);
}
}
@@ -256,44 +245,29 @@ inline void from_json_array_impl(const BasicJsonType& j, typename BasicJsonType:
arr = *j.template get_ptr<const typename BasicJsonType::array_t*>();
}
// fill a std::array from the JSON array j: the size is checked once and the
// elements are read in order, so a short array converts the elements it has and
// then throws the out_of_range error at() would throw for the first missing one
template<typename BasicJsonType, typename T, std::size_t N>
void from_json_std_array(const BasicJsonType& j, std::array<T, N>& arr)
{
const auto& inner_array = *j.template get_ptr<const typename BasicJsonType::array_t*>();
const std::size_t size = (std::min)(N, inner_array.size());
for (std::size_t i = 0; i < size; ++i)
{
arr[i] = inner_array[i].template get<T>();
}
if (JSON_HEDLEY_UNLIKELY(size < N))
{
static_cast<void>(j.at(size));
}
}
template<typename BasicJsonType, typename T, std::size_t N>
auto from_json_array_impl(const BasicJsonType& j, std::array<T, N>& arr,
priority_tag<2> /*unused*/)
-> decltype(j.template get<T>(), void())
{
from_json_std_array(j, arr);
for (std::size_t i = 0; i < N; ++i)
{
arr[i] = j.at(i).template get<T>();
}
}
// reserve() is called through this pair (used for arrays and objects alike)
// so from_json_array_impl and the object from_json have a single body for both
// containers that support reserve() and those that don't.
template<typename ConstructibleContainerType>
auto reserve_if_possible(ConstructibleContainerType& c, typename ConstructibleContainerType::size_type size, priority_tag<1> /*unused*/)
-> decltype(c.reserve(size), void())
// reserve() is called through this pair (modeled on from_json_object_reserve)
// so from_json_array_impl below has a single body for both ConstructibleArrayType
// that support reserve() and those that don't.
template<typename ConstructibleArrayType>
auto from_json_array_reserve(ConstructibleArrayType& arr, typename ConstructibleArrayType::size_type size, priority_tag<1> /*unused*/)
-> decltype(arr.reserve(size), void())
{
c.reserve(size);
arr.reserve(size);
}
template<typename ConstructibleContainerType>
void reserve_if_possible(ConstructibleContainerType& /*c*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
template<typename ConstructibleArrayType>
void from_json_array_reserve(ConstructibleArrayType& /*arr*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
{}
// append through push_back() where available; containers without it (such as
@@ -323,7 +297,7 @@ auto from_json_array_impl(const BasicJsonType& j, ConstructibleArrayType& arr, p
{
ConstructibleArrayType ret;
const auto* inner_array = j.template get_ptr<const typename BasicJsonType::array_t*>();
reserve_if_possible(ret, inner_array->size(), priority_tag<1> {});
from_json_array_reserve(ret, inner_array->size(), priority_tag<1> {});
for (const auto& elem : *inner_array)
{
// get<BasicJsonType>() returns *this, this won't call a from_json
@@ -362,35 +336,6 @@ std::array<T, sizeof...(Idx)> from_json_inplace_array_impl(const BasicJsonType&
return { { j.at(Idx).template get<T>()... } };
}
// std::is_trivial is deprecated in C++26 (see binary_writer::to_char_type)
#ifdef JSON_HAS_CPP_26
template<typename T>
using is_trivial_element = conjunction<std::is_trivially_copyable<T>, std::is_trivially_default_constructible<T>>;
#else
template<typename T>
using is_trivial_element = std::is_trivial<T>;
#endif
// the element type may not be default constructible: construct every element
// in place (a pack expansion that is unrolled N times)
template < typename BasicJsonType, typename T, std::size_t N >
std::array<T, N> from_json_inplace_array(const BasicJsonType& j, identity_tag<std::array<T, N>> tag,
std::false_type /*is_trivial_element*/)
{
return from_json_inplace_array_impl(j, tag, make_index_sequence<N> {});
}
// a trivial element type cannot tell default construction plus assignment from
// construction in place, so use a loop instead of unrolling N conversions
template < typename BasicJsonType, typename T, std::size_t N >
std::array<T, N> from_json_inplace_array(const BasicJsonType& j, identity_tag<std::array<T, N>> /*unused*/,
std::true_type /*is_trivial_element*/)
{
std::array<T, N> arr; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): every element is assigned or an exception is thrown
from_json_std_array(j, arr);
return arr;
}
template < typename BasicJsonType, typename T, std::size_t N >
auto from_json(const BasicJsonType& j, identity_tag<std::array<T, N>> tag)
-> decltype(from_json_inplace_array_impl(j, tag, make_index_sequence<N> {}))
@@ -400,7 +345,7 @@ auto from_json(const BasicJsonType& j, identity_tag<std::array<T, N>> tag)
throw_type_must_be("array", j);
}
return from_json_inplace_array(j, tag, is_trivial_element<T> {});
return from_json_inplace_array_impl(j, tag, make_index_sequence<N> {});
}
template<typename BasicJsonType>
@@ -433,6 +378,17 @@ inline void from_json(const BasicJsonType& j, CompatibleArrayType& bin)
}
}
template<typename ConstructibleObjectType>
auto from_json_object_reserve(ConstructibleObjectType& obj, typename ConstructibleObjectType::size_type size, priority_tag<1> /*unused*/)
-> decltype(obj.reserve(size), void())
{
obj.reserve(size);
}
template<typename ConstructibleObjectType>
inline void from_json_object_reserve(ConstructibleObjectType& /*obj*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
{}
// the keys of the source are sorted for the usual object types, so hinting the
// end makes every insertion constant-time (for containers that order their keys
// the same way); containers without emplace_hint() (such as ordered_map) fall
@@ -461,7 +417,7 @@ inline void from_json(const BasicJsonType& j, ConstructibleObjectType& obj)
ConstructibleObjectType ret;
const auto* inner_object = j.template get_ptr<const typename BasicJsonType::object_t*>();
reserve_if_possible(ret, inner_object->size(), priority_tag<1> {});
from_json_object_reserve(ret, inner_object->size(), priority_tag<1> {});
for (const auto& p : *inner_object)
{
from_json_object_emplace(ret, p.first, p.second.template get<typename ConstructibleObjectType::mapped_type>(), priority_tag<1> {});
@@ -481,17 +437,42 @@ template < typename BasicJsonType, typename ArithmeticType,
int > = 0 >
inline void from_json(const BasicJsonType& j, ArithmeticType& val)
{
if (get_number_value(j, val))
switch (static_cast<value_t>(j))
{
return;
case value_t::number_unsigned:
{
// never null: a basic_json whose number_unsigned_t is number_integer_t
// stores no number_unsigned values (#2573)
const auto* ptr = j.template get_ptr<const typename BasicJsonType::number_unsigned_t*>();
JSON_ASSERT(ptr != nullptr);
val = static_cast<ArithmeticType>(*ptr);
break;
}
case value_t::number_integer:
{
val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_integer_t*>());
break;
}
case value_t::number_float:
{
val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_float_t*>());
break;
}
case value_t::boolean:
{
val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::boolean_t*>());
break;
}
// unlike get_arithmetic_value(), a boolean is accepted here
if (JSON_HEDLEY_UNLIKELY(!j.is_boolean()))
{
case value_t::null:
case value_t::object:
case value_t::array:
case value_t::string:
case value_t::binary:
case value_t::discarded:
default:
throw_type_must_be("number", j);
}
val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::boolean_t*>());
}
template<typename BasicJsonType, typename Type>
+26 -22
View File
@@ -66,7 +66,6 @@ enum class exception_id : int
invalid_string_or_size = 113, ///< a binary input contains an invalid string or size specification
bson_unsupported_type = 114, ///< a BSON record type is not supported
invalid_high_precision_number = 115, ///< a UBJSON/BJData high-precision number cannot be parsed
invalid_document_image = 116, ///< json_document::load() rejects an image
// invalid_iterator
iterators_incompatible = 201, ///< the iterators of a range belong to different values
iterator_from_other_value = 202, ///< an iterator does not belong to the value it is used with
@@ -101,8 +100,6 @@ enum class exception_id : int
invalid_utf8 = 316, ///< dump() finds a string that is not valid UTF-8
type_not_serializable = 317, ///< a value cannot be serialized to the requested binary format
enum_key_duplicate = 318, ///< an enum-keyed map has two keys that serialize to the same string
document_binary_value = 319, ///< a binary value is stored in a json_document
document_image_unsupported = 320, ///< a json_document image cannot be saved or loaded (discarded document, big-endian target)
discarded_value_used = 321, ///< a discarded value is used
// out_of_range
array_index_out_of_range = 401, ///< an array index is out of range
@@ -120,7 +117,6 @@ enum class exception_id : int
patch_remove_parent_not_container = 413, ///< the parent of a JSON Patch 'remove' target is not a container
patch_move_into_child = 414, ///< a JSON Patch 'move' moves a value into one of its children
subtype_out_of_range = 415, ///< a binary subtype does not fit into one byte
document_too_large = 416, ///< a json_document input, its edit storage, or an image reaches the 32-bit (4 GiB) limit
// other_error
internal_error = 500, ///< unreachable code was reached
patch_test_failed = 501, ///< a JSON Patch 'test' operation failed
@@ -150,13 +146,6 @@ class exception : public std::exception
return concat("[json.exception.", ename, '.', std::to_string(id_), "] ");
}
/// build the message "[json.exception.<ename>.<id>] <diagnostics><what_arg>"
template<typename BasicJsonContext>
static std::string what_impl(const std::string& ename, int id_, const std::string& what_arg, BasicJsonContext context)
{
return concat(name(ename, id_), diagnostics(context), what_arg);
}
static std::string diagnostics(std::nullptr_t /*leaf_element*/)
{
return "";
@@ -169,21 +158,24 @@ class exception : public std::exception
std::vector<std::string> tokens;
for (const auto* current = leaf_element; current != nullptr && current->m_parent != nullptr; current = current->m_parent)
{
const auto* parent = current->m_parent;
if (parent->is_array())
switch (current->m_parent->type())
{
for (std::size_t i = 0; i < parent->m_data.m_value.array->size(); ++i)
case value_t::array:
{
if (&parent->m_data.m_value.array->operator[](i) == current)
for (std::size_t i = 0; i < current->m_parent->m_data.m_value.array->size(); ++i)
{
if (&current->m_parent->m_data.m_value.array->operator[](i) == current)
{
tokens.emplace_back(std::to_string(i));
break;
}
}
break;
}
else if (parent->is_object())
case value_t::object:
{
for (const auto& element : *parent->m_data.m_value.object)
for (const auto& element : *current->m_parent->m_data.m_value.object)
{
if (&element.second == current)
{
@@ -194,8 +186,20 @@ class exception : public std::exception
break;
}
}
break;
}
case value_t::null: // LCOV_EXCL_LINE
case value_t::string: // LCOV_EXCL_LINE
case value_t::boolean: // LCOV_EXCL_LINE
case value_t::number_integer: // LCOV_EXCL_LINE
case value_t::number_unsigned: // LCOV_EXCL_LINE
case value_t::number_float: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
// all other parent types (primitives) contribute no token
}
if (tokens.empty())
@@ -311,7 +315,7 @@ class invalid_iterator : public exception
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static invalid_iterator create(int id_, const std::string& what_arg, BasicJsonContext context)
{
const std::string w = exception::what_impl("invalid_iterator", id_, what_arg, context);
const std::string w = concat(exception::name("invalid_iterator", id_), exception::diagnostics(context), what_arg);
return {id_, w.c_str()};
}
@@ -335,7 +339,7 @@ class type_error : public exception
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static type_error create(int id_, const std::string& what_arg, BasicJsonContext context)
{
const std::string w = exception::what_impl("type_error", id_, what_arg, context);
const std::string w = concat(exception::name("type_error", id_), exception::diagnostics(context), what_arg);
return {id_, w.c_str()};
}
@@ -358,7 +362,7 @@ class out_of_range : public exception
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static out_of_range create(int id_, const std::string& what_arg, BasicJsonContext context)
{
const std::string w = exception::what_impl("out_of_range", id_, what_arg, context);
const std::string w = concat(exception::name("out_of_range", id_), exception::diagnostics(context), what_arg);
return {id_, w.c_str()};
}
@@ -381,7 +385,7 @@ class other_error : public exception
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static other_error create(int id_, const std::string& what_arg, BasicJsonContext context)
{
const std::string w = exception::what_impl("other_error", id_, what_arg, context);
const std::string w = concat(exception::name("other_error", id_), exception::diagnostics(context), what_arg);
return {id_, w.c_str()};
}
+82 -108
View File
@@ -293,40 +293,6 @@ class binary_reader
return is_object ? sax->end_object() : sax->end_array();
}
/*!
@brief close the innermost open container and report whether that
completed the value being read
The value begun by the driver is complete once its container is not
inside another one.
@param[out] value_complete whether no container is left open
@return whether the SAX parser accepted the end event
*/
bool leave_container_and_check(bool& value_complete)
{
if (JSON_HEDLEY_UNLIKELY(!leave_container()))
{
return false;
}
value_complete = container_stack.empty();
return true;
}
/*!
@brief read an object key and pass it to the SAX parser
@tparam GetKey member function that reads the key into a string
@param[in,out] key string the key is read into; cleared first
@return whether reading the key and the SAX event succeeded
*/
template<bool (binary_reader::*GetKey)(string_t&)>
bool read_object_key(string_t& key)
{
key.clear();
return (this->*GetKey)(key) && sax->key(key);
}
//////////
// BSON //
//////////
@@ -1098,34 +1064,6 @@ class binary_reader
return last_byte_error(exception_id::invalid_string_or_size, concat("indefinite-length ", type_name, " is not allowed inside indefinite-length ", type_name, "; last byte: 0x", get_token_string()), context);
}
/*!
@brief report an object key that is not a string
@param[in] found description of what was found instead of a string key
@return false
*/
bool unsupported_key_type(const char* found) const
{
return last_byte_error(exception_id::invalid_string_or_size, "only string keys are supported, but found ", "object key", found);
}
/*!
@brief read the length of a CBOR string or byte array and then its bytes
@tparam NumberType the type of the length field
@tparam ContainerType the destination container (string_t or binary_t)
@param[in] context further context information (for diagnostics)
@param[out] result container the bytes are appended to
@return whether reading completed
*/
template<typename NumberType, typename ContainerType>
bool get_cbor_sized_bytes(const char* context, ContainerType& result)
{
NumberType len{};
return get_number(len) && get_bytes(len, context, result);
}
/*!
@brief reads a definite-length CBOR string
@@ -1176,22 +1114,26 @@ class binary_reader
case 0x78: // UTF-8 string (one-byte uint8_t for n follows)
{
return get_cbor_sized_bytes<std::uint8_t>("string", result);
std::uint8_t len{};
return get_number(len) && get_string(len, result);
}
case 0x79: // UTF-8 string (two-byte uint16_t for n follow)
{
return get_cbor_sized_bytes<std::uint16_t>("string", result);
std::uint16_t len{};
return get_number(len) && get_string(len, result);
}
case 0x7A: // UTF-8 string (four-byte uint32_t for n follow)
{
return get_cbor_sized_bytes<std::uint32_t>("string", result);
std::uint32_t len{};
return get_number(len) && get_string(len, result);
}
case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow)
{
return get_cbor_sized_bytes<std::uint64_t>("string", result);
std::uint64_t len{};
return get_number(len) && get_string(len, result);
}
default:
@@ -1326,7 +1268,7 @@ class binary_reader
break;
}
return unsupported_key_type(found);
return last_byte_error(exception_id::invalid_string_or_size, "only string keys are supported, but found ", "object key", found);
}
/*!
@@ -1379,22 +1321,30 @@ class binary_reader
case 0x58: // Binary data (one-byte uint8_t for n follows)
{
return get_cbor_sized_bytes<std::uint8_t>("binary", result);
std::uint8_t len{};
return get_number(len) &&
get_binary(len, result);
}
case 0x59: // Binary data (two-byte uint16_t for n follow)
{
return get_cbor_sized_bytes<std::uint16_t>("binary", result);
std::uint16_t len{};
return get_number(len) &&
get_binary(len, result);
}
case 0x5A: // Binary data (four-byte uint32_t for n follow)
{
return get_cbor_sized_bytes<std::uint32_t>("binary", result);
std::uint32_t len{};
return get_number(len) &&
get_binary(len, result);
}
case 0x5B: // Binary data (eight-byte uint64_t for n follow)
{
return get_cbor_sized_bytes<std::uint64_t>("binary", result);
std::uint64_t len{};
return get_number(len) &&
get_binary(len, result);
}
default:
@@ -1609,12 +1559,12 @@ class binary_reader
if (at_end)
{
bool value_complete = false;
if (JSON_HEDLEY_UNLIKELY(!leave_container_and_check(value_complete)))
if (JSON_HEDLEY_UNLIKELY(!leave_container()))
{
return false;
}
if (value_complete)
// the value begun here is complete once its container is
if (container_stack.empty())
{
return true;
}
@@ -1623,7 +1573,8 @@ class binary_reader
if (top.is_object)
{
if (JSON_HEDLEY_UNLIKELY(!read_object_key<&binary_reader::get_cbor_object_key>(key)))
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key)))
{
return false;
}
@@ -2094,17 +2045,20 @@ class binary_reader
case 0xD9: // str 8
{
return get_string_with_length<std::uint8_t>(result, context);
std::uint8_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
case 0xDA: // str 16
{
return get_string_with_length<std::uint16_t>(result, context);
std::uint16_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
case 0xDB: // str 32
{
return get_string_with_length<std::uint32_t>(result, context);
std::uint32_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
default:
@@ -2201,7 +2155,7 @@ class binary_reader
break;
}
return unsupported_key_type(found);
return last_byte_error(exception_id::invalid_string_or_size, "only string keys are supported, but found ", "object key", found);
}
/*!
@@ -2346,12 +2300,12 @@ class binary_reader
if (container_stack.back().remaining == 0)
{
bool value_complete = false;
if (JSON_HEDLEY_UNLIKELY(!leave_container_and_check(value_complete)))
if (JSON_HEDLEY_UNLIKELY(!leave_container()))
{
return false;
}
if (value_complete)
// the value begun here is complete once its container is
if (container_stack.empty())
{
return true;
}
@@ -2364,7 +2318,8 @@ class binary_reader
if (is_object)
{
get();
if (JSON_HEDLEY_UNLIKELY(!read_object_key<&binary_reader::get_msgpack_object_key>(key)))
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key)))
{
return false;
}
@@ -2548,7 +2503,8 @@ class binary_reader
{
case 'U':
{
return get_string_with_length<std::uint8_t>(result, context);
std::uint8_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
case 'i':
@@ -2581,7 +2537,8 @@ class binary_reader
{
break;
}
return get_string_with_length<std::uint16_t>(result, context);
std::uint16_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
case 'm':
@@ -2590,7 +2547,8 @@ class binary_reader
{
break;
}
return get_string_with_length<std::uint32_t>(result, context);
std::uint32_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
case 'M':
@@ -2599,7 +2557,8 @@ class binary_reader
{
break;
}
return get_string_with_length<std::uint64_t>(result, context);
std::uint64_t len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
default:
@@ -3415,12 +3374,12 @@ class binary_reader
if (at_end)
{
bool value_complete = false;
if (JSON_HEDLEY_UNLIKELY(!leave_container_and_check(value_complete)))
if (JSON_HEDLEY_UNLIKELY(!leave_container()))
{
return false;
}
if (value_complete)
// the value begun here is complete once its container is
if (container_stack.empty())
{
return true;
}
@@ -3429,7 +3388,8 @@ class binary_reader
if (top.is_object)
{
if (JSON_HEDLEY_UNLIKELY(!read_object_key<&binary_reader::get_bon8_key>(key)))
key.clear();
if (JSON_HEDLEY_UNLIKELY(!get_bon8_key(key) || !sax->key(key)))
{
return false;
}
@@ -4169,22 +4129,6 @@ class binary_reader
return true;
}
/*!
@brief read a length of type @a NumberType, then a string of that many bytes
@tparam NumberType the type of the length field
@param[out] result string the bytes are appended to
@param[in] context further context information (for diagnostics)
@return whether reading and validating the string succeeded
*/
template<typename NumberType>
bool get_string_with_length(string_t& result, const char* context)
{
NumberType len{};
return get_number(len) && get_string(len, result) && check_string_utf8(result, context);
}
/*!
@brief create a byte array by reading bytes from the input
@@ -4394,7 +4338,37 @@ class binary_reader
const std::string& context) const
{
std::string error_msg = "syntax error while parsing ";
error_msg += input_format_name(input_format);
switch (input_format)
{
case input_format_t::cbor:
error_msg += "CBOR";
break;
case input_format_t::msgpack:
error_msg += "MessagePack";
break;
case input_format_t::ubjson:
error_msg += "UBJSON";
break;
case input_format_t::bson:
error_msg += "BSON";
break;
case input_format_t::bjdata:
error_msg += "BJData";
break;
case input_format_t::bon8:
error_msg += "BON8";
break;
case input_format_t::json: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
}
return concat(error_msg, ' ', context, ": ", detail);
}
+36 -225
View File
@@ -8,18 +8,17 @@
#pragma once
#include <algorithm> // count, find, max, min
#include <algorithm> // count, find, min
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint32_t
#include <cstring> // strlen
#include <iterator> // begin, end, iterator_traits, random_access_iterator_tag, distance, next, reverse_iterator
#include <limits> // numeric_limits
#include <memory> // unique_ptr
#include <streambuf> // streambuf
#include <string> // string, char_traits
#include <type_traits> // enable_if, is_base_of, is_pointer, is_integral, remove_pointer
#include <utility> // declval, move, pair
#include <utility> // pair, declval
#include <vector> // vector
#ifndef JSON_NO_IO
@@ -67,118 +66,12 @@ namespace detail
/// the supported input formats
enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata, bon8 };
/// @return the name of a binary input format, as used in exception messages
inline const char* input_format_name(const input_format_t format) noexcept
{
switch (format)
{
case input_format_t::cbor:
return "CBOR";
case input_format_t::msgpack:
return "MessagePack";
case input_format_t::ubjson:
return "UBJSON";
case input_format_t::bson:
return "BSON";
case input_format_t::bjdata:
return "BJData";
case input_format_t::bon8:
return "BON8";
case input_format_t::json: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
return ""; // LCOV_EXCL_LINE
}
}
/// @return the message of the parse_error thrown or reported for an empty input
inline const char* empty_input_message() noexcept
{
return "attempting to parse an empty input; check that your input string or stream contains the expected JSON";
}
////////////////////
// input adapters //
////////////////////
#ifndef JSON_NO_IO
/*!
A growable array of characters for the part of a token that an input adapter
copied out of its window: unlike std::vector<char>::push_back(), appending a
character takes no reference to a local, and is a few instructions when there
is room.
*/
class token_carry_buffer
{
public:
token_carry_buffer() = default;
token_carry_buffer(const token_carry_buffer&) = delete;
token_carry_buffer& operator=(const token_carry_buffer&) = delete;
token_carry_buffer(token_carry_buffer&& rhs) noexcept
: m_data(std::move(rhs.m_data)), m_end(rhs.m_end), m_capacity_end(rhs.m_capacity_end)
{
rhs.m_end = rhs.m_capacity_end = nullptr;
}
token_carry_buffer& operator=(token_carry_buffer&&) = delete;
~token_carry_buffer() = default;
void clear() noexcept
{
m_end = m_data.get();
}
void push_back(char c)
{
if (JSON_HEDLEY_UNLIKELY(m_end == m_capacity_end))
{
grow(1);
}
*m_end++ = c;
}
void append(const char* first, const char* last)
{
const auto n = static_cast<std::size_t>(last - first);
const auto room = static_cast<std::size_t>(m_capacity_end - m_end);
if (n > room)
{
grow(n);
}
if (n != 0)
{
std::memcpy(m_end, first, n);
}
m_end += n;
}
/// replace the contents of @a out
void copy_to(std::vector<char>& out) const
{
out.assign(m_data.get(), m_end);
}
private:
/// make room for @a extra more characters
JSON_HEDLEY_NEVER_INLINE void grow(std::size_t extra)
{
const auto size = static_cast<std::size_t>(m_end - m_data.get());
const auto capacity = static_cast<std::size_t>(m_capacity_end - m_data.get());
const std::size_t new_capacity = (std::max)(capacity < 32 ? static_cast<std::size_t>(32) : 2 * capacity, size + extra);
std::unique_ptr<char[]> data(new char[new_capacity]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
if (size != 0)
{
std::memcpy(data.get(), m_data.get(), size);
}
m_data = std::move(data);
m_end = m_data.get() + size;
m_capacity_end = m_data.get() + new_capacity;
}
std::unique_ptr<char[]> m_data{}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays,readability-redundant-member-init)
char* m_end = nullptr;
char* m_capacity_end = nullptr;
};
/*!
Input adapter for stdio file access. This adapter read only 1 byte and do not use any
buffer. This adapter is a very low level adapter.
*/
@@ -316,7 +209,6 @@ class input_stream_adapter
, token_segment(rhs.token_segment)
, current_in_get_area(rhs.current_in_get_area)
, tracking_token(rhs.tracking_token)
, no_get_area(rhs.no_get_area)
#endif
{
rhs.is = nullptr;
@@ -403,20 +295,37 @@ class input_stream_adapter
std::char_traits<char>::int_type get_character()
{
#if !JSON_DISABLE_STREAM_BULK_SCAN
// the character is in the get area: step over it, as sbumpc() does
const char* const next = streambuf_access::get_ptr(sb);
if (JSON_HEDLEY_LIKELY(next != streambuf_access::get_end(sb)))
// sbumpc() refills the get area if it is exhausted
const bool refill = streambuf_access::get_ptr(sb) == streambuf_access::get_end(sb);
if (JSON_HEDLEY_UNLIKELY(refill))
{
current_in_get_area = true;
streambuf_access::bump(sb, 1);
return std::char_traits<char>::to_int_type(*next);
}
if (no_get_area)
{
return get_character_no_get_area();
save_token_segment();
}
#endif
return get_character_slow();
auto res = sb->sbumpc();
// set eof manually, as we don't use the istream interface.
if (JSON_HEDLEY_UNLIKELY(res == std::char_traits<char>::eof()))
{
#if !JSON_DISABLE_STREAM_BULK_SCAN
current_in_get_area = false;
#endif
is->clear(is->rdstate() | std::ios::eofbit);
}
#if !JSON_DISABLE_STREAM_BULK_SCAN
else if (JSON_HEDLEY_UNLIKELY(refill))
{
// the consumed character is at gptr() - 1, unless the streambuf
// has no get area and returned it from uflow()
const char* const next = streambuf_access::get_ptr(sb);
after_refill(res, next != nullptr && next != streambuf_access::get_begin(sb));
}
else
{
current_in_get_area = true;
}
#endif
return res;
}
#endif
@@ -425,18 +334,6 @@ class input_stream_adapter
{
#if JSON_PRECISE_STREAM_POSITION
commit_lookahead();
#endif
#if !JSON_DISABLE_STREAM_BULK_SCAN
// the elements are in the get area: copy them, as sgetn() does, but
// without the virtual call
const char* const first = streambuf_access::get_ptr(sb);
const auto available = static_cast<std::size_t>(streambuf_access::get_end(sb) - first);
if (JSON_HEDLEY_LIKELY(count * sizeof(T) <= available && count * sizeof(T) <= static_cast<std::size_t>((std::numeric_limits<int>::max)())))
{
std::memcpy(dest, first, count * sizeof(T));
streambuf_access::bump(sb, static_cast<int>(count * sizeof(T)));
return count * sizeof(T);
}
#endif
auto res = static_cast<std::size_t>(sb->sgetn(reinterpret_cast<char*>(dest), static_cast<std::streamsize>(count * sizeof(T))));
if (JSON_HEDLEY_UNLIKELY(res < count * sizeof(T)))
@@ -524,7 +421,7 @@ class input_stream_adapter
// last one if @a drop_last.
void copy_token(std::vector<char>& out, bool drop_last) const
{
token_carry.copy_to(out);
out = token_carry;
if (token_segment != nullptr)
{
out.insert(out.end(), token_segment, current_end());
@@ -537,47 +434,6 @@ class input_stream_adapter
#endif
private:
#if !JSON_PRECISE_STREAM_POSITION
// the get area is exhausted, or there is none
#if !JSON_DISABLE_STREAM_BULK_SCAN
JSON_HEDLEY_NEVER_INLINE
#endif
std::char_traits<char>::int_type get_character_slow()
{
#if !JSON_DISABLE_STREAM_BULK_SCAN
// sbumpc() refills the get area if it is exhausted
const bool refill = streambuf_access::get_ptr(sb) == streambuf_access::get_end(sb);
if (JSON_HEDLEY_UNLIKELY(refill))
{
save_token_segment();
}
#endif
auto res = sb->sbumpc();
// set eof manually, as we don't use the istream interface.
if (JSON_HEDLEY_UNLIKELY(res == std::char_traits<char>::eof()))
{
#if !JSON_DISABLE_STREAM_BULK_SCAN
current_in_get_area = false;
#endif
is->clear(is->rdstate() | std::ios::eofbit);
}
#if !JSON_DISABLE_STREAM_BULK_SCAN
else if (JSON_HEDLEY_UNLIKELY(refill))
{
// the consumed character is at gptr() - 1, unless the streambuf
// has no get area and returned it from uflow()
const char* const next = streambuf_access::get_ptr(sb);
after_refill(res, next != nullptr && next != streambuf_access::get_begin(sb));
}
else
{
current_in_get_area = true;
}
#endif
return res;
}
#endif
#if !JSON_DISABLE_STREAM_BULK_SCAN
// Reads the protected get-area pointers of any std::streambuf. Naming the
// members through a derived class makes forming the member pointers
@@ -601,11 +457,6 @@ class input_stream_adapter
{
(s->*(&streambuf_access::gbump))(n);
}
// sbumpc() for an exhausted get area
static std::streambuf::int_type next_character(std::streambuf* s)
{
return (s->*(&streambuf_access::uflow))();
}
};
// one past the character last returned by get_character(), if that is in
@@ -626,53 +477,16 @@ class input_stream_adapter
{
if (token_segment != nullptr)
{
token_carry.append(token_segment, streambuf_access::get_end(sb));
token_carry.insert(token_carry.end(), token_segment, streambuf_access::get_end(sb));
token_segment = nullptr;
}
}
#if !JSON_PRECISE_STREAM_POSITION
// The streambuf has no get area (e.g., std::cin synchronized with stdio):
// every character comes from uflow(), and the token is kept by value.
JSON_HEDLEY_NEVER_INLINE std::char_traits<char>::int_type get_character_no_get_area()
{
// the get area is exhausted, so this is what sbumpc() does
const auto res = streambuf_access::next_character(sb);
if (JSON_HEDLEY_UNLIKELY(res == std::char_traits<char>::eof() || streambuf_access::get_end(sb) != nullptr))
{
return get_character_no_get_area_other(res);
}
// current_in_get_area is false: only the first character of a get
// area, or the end of the input, is read here
if (tracking_token)
{
token_carry.push_back(std::char_traits<char>::to_char_type(res));
}
return res;
}
// @a res came from uflow() of a streambuf without a get area: it is the
// end of the input, or the streambuf has a get area now
JSON_HEDLEY_NEVER_INLINE std::char_traits<char>::int_type get_character_no_get_area_other(std::char_traits<char>::int_type res)
{
if (res == std::char_traits<char>::eof())
{
current_in_get_area = false;
is->clear(is->rdstate() | std::ios::eofbit);
return res;
}
// the consumed character is at gptr() - 1
after_refill(res, streambuf_access::get_ptr(sb) != streambuf_access::get_begin(sb));
return res;
}
#endif
// character @a c was just read into a new get area (@a in_get_area) or
// returned without one
void after_refill(std::char_traits<char>::int_type c, bool in_get_area)
{
current_in_get_area = in_get_area;
no_get_area = !in_get_area;
if (!tracking_token)
{
return;
@@ -711,16 +525,13 @@ class input_stream_adapter
#endif
#if !JSON_DISABLE_STREAM_BULK_SCAN
/// the part of the current token that was in earlier get areas
token_carry_buffer token_carry{}; // NOLINT(readability-redundant-member-init)
std::vector<char> token_carry{}; // NOLINT(readability-redundant-member-init)
/// start of the current token's part in the get area, or nullptr
const char* token_segment = nullptr;
/// whether the character last returned by get_character() is in the get area
bool current_in_get_area = false;
/// whether a token is tracked at all; the binary readers never start one
bool tracking_token = false;
/// whether the streambuf had no get area when it was last asked for a
/// character
bool no_get_area = false;
#endif
};
#endif // JSON_NO_IO
@@ -1298,7 +1109,7 @@ inline file_input_adapter input_adapter(std::FILE* file)
{
if (file == nullptr)
{
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, empty_input_message(), nullptr));
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
return file_input_adapter(file);
}
@@ -1307,7 +1118,7 @@ inline input_stream_adapter input_adapter(std::istream& stream)
{
if (stream.rdbuf() == nullptr)
{
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, empty_input_message(), nullptr));
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
return input_stream_adapter(stream);
}
@@ -1332,7 +1143,7 @@ contiguous_bytes_input_adapter input_adapter(CharT b)
{
if (b == nullptr)
{
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, empty_input_message(), nullptr));
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
auto length = std::strlen(reinterpret_cast<const char*>(b));
const auto* ptr = reinterpret_cast<const char*>(b);
+48 -50
View File
@@ -315,52 +315,6 @@ struct diagnostic_positions
};
#endif
/*!
@brief error state shared by the DOM-building SAX parsers
Holds whether a parse error occurred and whether errors are thrown, and
implements the parse_error() and is_errored() members of the SAX interface for
json_sax_dom_parser and json_sax_dom_callback_parser.
*/
class json_sax_dom_error_state
{
public:
explicit json_sax_dom_error_state(const bool allow_exceptions_) noexcept
: allow_exceptions(allow_exceptions_)
{}
// make class move-only
json_sax_dom_error_state(const json_sax_dom_error_state&) = delete;
json_sax_dom_error_state(json_sax_dom_error_state&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor)
json_sax_dom_error_state& operator=(const json_sax_dom_error_state&) = delete;
json_sax_dom_error_state& operator=(json_sax_dom_error_state&&) = delete; // the const member prevents assignment
~json_sax_dom_error_state() = default;
template<class Exception>
bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
const Exception& ex)
{
errored = true;
static_cast<void>(ex);
if (allow_exceptions)
{
JSON_THROW(ex);
}
return false;
}
bool is_errored() const
{
return errored;
}
private:
/// whether a syntax error occurred
bool errored = false;
/// whether to throw exceptions in case of errors
const bool allow_exceptions = true;
};
/*!
@brief SAX implementation to create a JSON value from SAX events
@@ -375,7 +329,7 @@ constructor contains the parsed value.
@tparam BasicJsonType the JSON type
*/
template<typename BasicJsonType, typename InputAdapterType>
class json_sax_dom_parser : public json_sax_dom_error_state
class json_sax_dom_parser
{
public:
using number_integer_t = typename BasicJsonType::number_integer_t;
@@ -391,7 +345,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
@param[in] allow_exceptions_ whether parse errors yield exceptions
*/
explicit json_sax_dom_parser(BasicJsonType& r, const bool allow_exceptions_ = true, lexer_t* lexer_ = nullptr)
: json_sax_dom_error_state(allow_exceptions_), root(r), m_lexer_ref(lexer_)
: root(r), allow_exceptions(allow_exceptions_), m_lexer_ref(lexer_)
{}
// make class move-only
@@ -520,6 +474,24 @@ class json_sax_dom_parser : public json_sax_dom_error_state
return true;
}
template<class Exception>
bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
const Exception& ex)
{
errored = true;
static_cast<void>(ex);
if (allow_exceptions)
{
JSON_THROW(ex);
}
return false;
}
constexpr bool is_errored() const
{
return errored;
}
private:
/*!
@@ -573,12 +545,16 @@ class json_sax_dom_parser : public json_sax_dom_error_state
std::vector<BasicJsonType*> ref_stack {};
/// helper to hold the reference for the next object element
BasicJsonType* object_element = nullptr;
/// whether a syntax error occurred
bool errored = false;
/// whether to throw exceptions in case of errors
const bool allow_exceptions = true;
/// the lexer reference to obtain the current position
lexer_t* m_lexer_ref = nullptr;
};
template<typename BasicJsonType, typename InputAdapterType>
class json_sax_dom_callback_parser : public json_sax_dom_error_state
class json_sax_dom_callback_parser
{
public:
using number_integer_t = typename BasicJsonType::number_integer_t;
@@ -594,7 +570,7 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
parser_callback_t cb,
const bool allow_exceptions_ = true,
lexer_t* lexer_ = nullptr)
: json_sax_dom_error_state(allow_exceptions_), root(r), callback(std::move(cb)), m_lexer_ref(lexer_)
: root(r), callback(std::move(cb)), allow_exceptions(allow_exceptions_), m_lexer_ref(lexer_)
{
keep_stack.push_back(true);
}
@@ -861,6 +837,24 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
return true;
}
template<class Exception>
bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
const Exception& ex)
{
errored = true;
static_cast<void>(ex);
if (allow_exceptions)
{
JSON_THROW(ex);
}
return false;
}
constexpr bool is_errored() const
{
return errored;
}
private:
/// if there is a pending duplicate-key stash entry for this exact slot,
@@ -1102,8 +1096,12 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
/// value is later rejected by the callback, instead of erasing the
/// member entirely
std::vector<std::pair<BasicJsonType*, BasicJsonType>> duplicate_key_stash {};
/// whether a syntax error occurred
bool errored = false;
/// callback function
const parser_callback_t callback = nullptr;
/// whether to throw exceptions in case of errors
const bool allow_exceptions = true;
/// a discarded value for the callback
BasicJsonType discarded = BasicJsonType::value_t::discarded;
/// the lexer reference to obtain the current position
+156 -36
View File
@@ -656,74 +656,194 @@ class lexer : public lexer_base<BasicJsonType>
// invalid control characters
case 0x00:
{
error_message = "invalid string: control character U+0000 (NUL) must be escaped to \\u0000";
return token_type::parse_error;
}
case 0x01:
{
error_message = "invalid string: control character U+0001 (SOH) must be escaped to \\u0001";
return token_type::parse_error;
}
case 0x02:
{
error_message = "invalid string: control character U+0002 (STX) must be escaped to \\u0002";
return token_type::parse_error;
}
case 0x03:
{
error_message = "invalid string: control character U+0003 (ETX) must be escaped to \\u0003";
return token_type::parse_error;
}
case 0x04:
{
error_message = "invalid string: control character U+0004 (EOT) must be escaped to \\u0004";
return token_type::parse_error;
}
case 0x05:
{
error_message = "invalid string: control character U+0005 (ENQ) must be escaped to \\u0005";
return token_type::parse_error;
}
case 0x06:
{
error_message = "invalid string: control character U+0006 (ACK) must be escaped to \\u0006";
return token_type::parse_error;
}
case 0x07:
{
error_message = "invalid string: control character U+0007 (BEL) must be escaped to \\u0007";
return token_type::parse_error;
}
case 0x08:
{
error_message = "invalid string: control character U+0008 (BS) must be escaped to \\u0008 or \\b";
return token_type::parse_error;
}
case 0x09:
{
error_message = "invalid string: control character U+0009 (HT) must be escaped to \\u0009 or \\t";
return token_type::parse_error;
}
case 0x0A:
{
error_message = "invalid string: control character U+000A (LF) must be escaped to \\u000A or \\n";
return token_type::parse_error;
}
case 0x0B:
{
error_message = "invalid string: control character U+000B (VT) must be escaped to \\u000B";
return token_type::parse_error;
}
case 0x0C:
{
error_message = "invalid string: control character U+000C (FF) must be escaped to \\u000C or \\f";
return token_type::parse_error;
}
case 0x0D:
{
error_message = "invalid string: control character U+000D (CR) must be escaped to \\u000D or \\r";
return token_type::parse_error;
}
case 0x0E:
{
error_message = "invalid string: control character U+000E (SO) must be escaped to \\u000E";
return token_type::parse_error;
}
case 0x0F:
{
error_message = "invalid string: control character U+000F (SI) must be escaped to \\u000F";
return token_type::parse_error;
}
case 0x10:
{
error_message = "invalid string: control character U+0010 (DLE) must be escaped to \\u0010";
return token_type::parse_error;
}
case 0x11:
{
error_message = "invalid string: control character U+0011 (DC1) must be escaped to \\u0011";
return token_type::parse_error;
}
case 0x12:
{
error_message = "invalid string: control character U+0012 (DC2) must be escaped to \\u0012";
return token_type::parse_error;
}
case 0x13:
{
error_message = "invalid string: control character U+0013 (DC3) must be escaped to \\u0013";
return token_type::parse_error;
}
case 0x14:
{
error_message = "invalid string: control character U+0014 (DC4) must be escaped to \\u0014";
return token_type::parse_error;
}
case 0x15:
{
error_message = "invalid string: control character U+0015 (NAK) must be escaped to \\u0015";
return token_type::parse_error;
}
case 0x16:
{
error_message = "invalid string: control character U+0016 (SYN) must be escaped to \\u0016";
return token_type::parse_error;
}
case 0x17:
{
error_message = "invalid string: control character U+0017 (ETB) must be escaped to \\u0017";
return token_type::parse_error;
}
case 0x18:
{
error_message = "invalid string: control character U+0018 (CAN) must be escaped to \\u0018";
return token_type::parse_error;
}
case 0x19:
{
error_message = "invalid string: control character U+0019 (EM) must be escaped to \\u0019";
return token_type::parse_error;
}
case 0x1A:
{
error_message = "invalid string: control character U+001A (SUB) must be escaped to \\u001A";
return token_type::parse_error;
}
case 0x1B:
{
error_message = "invalid string: control character U+001B (ESC) must be escaped to \\u001B";
return token_type::parse_error;
}
case 0x1C:
{
error_message = "invalid string: control character U+001C (FS) must be escaped to \\u001C";
return token_type::parse_error;
}
case 0x1D:
{
error_message = "invalid string: control character U+001D (GS) must be escaped to \\u001D";
return token_type::parse_error;
}
case 0x1E:
{
error_message = "invalid string: control character U+001E (RS) must be escaped to \\u001E";
return token_type::parse_error;
}
case 0x1F:
{
static const char* const control_character_messages[32] = // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
{
"invalid string: control character U+0000 (NUL) must be escaped to \\u0000",
"invalid string: control character U+0001 (SOH) must be escaped to \\u0001",
"invalid string: control character U+0002 (STX) must be escaped to \\u0002",
"invalid string: control character U+0003 (ETX) must be escaped to \\u0003",
"invalid string: control character U+0004 (EOT) must be escaped to \\u0004",
"invalid string: control character U+0005 (ENQ) must be escaped to \\u0005",
"invalid string: control character U+0006 (ACK) must be escaped to \\u0006",
"invalid string: control character U+0007 (BEL) must be escaped to \\u0007",
"invalid string: control character U+0008 (BS) must be escaped to \\u0008 or \\b",
"invalid string: control character U+0009 (HT) must be escaped to \\u0009 or \\t",
"invalid string: control character U+000A (LF) must be escaped to \\u000A or \\n",
"invalid string: control character U+000B (VT) must be escaped to \\u000B",
"invalid string: control character U+000C (FF) must be escaped to \\u000C or \\f",
"invalid string: control character U+000D (CR) must be escaped to \\u000D or \\r",
"invalid string: control character U+000E (SO) must be escaped to \\u000E",
"invalid string: control character U+000F (SI) must be escaped to \\u000F",
"invalid string: control character U+0010 (DLE) must be escaped to \\u0010",
"invalid string: control character U+0011 (DC1) must be escaped to \\u0011",
"invalid string: control character U+0012 (DC2) must be escaped to \\u0012",
"invalid string: control character U+0013 (DC3) must be escaped to \\u0013",
"invalid string: control character U+0014 (DC4) must be escaped to \\u0014",
"invalid string: control character U+0015 (NAK) must be escaped to \\u0015",
"invalid string: control character U+0016 (SYN) must be escaped to \\u0016",
"invalid string: control character U+0017 (ETB) must be escaped to \\u0017",
"invalid string: control character U+0018 (CAN) must be escaped to \\u0018",
"invalid string: control character U+0019 (EM) must be escaped to \\u0019",
"invalid string: control character U+001A (SUB) must be escaped to \\u001A",
"invalid string: control character U+001B (ESC) must be escaped to \\u001B",
"invalid string: control character U+001C (FS) must be escaped to \\u001C",
"invalid string: control character U+001D (GS) must be escaped to \\u001D",
"invalid string: control character U+001E (RS) must be escaped to \\u001E",
"invalid string: control character U+001F (US) must be escaped to \\u001F"
};
error_message = control_character_messages[static_cast<std::size_t>(current)];
error_message = "invalid string: control character U+001F (US) must be escaped to \\u001F";
return token_type::parse_error;
}
+30 -37
View File
@@ -165,10 +165,23 @@ class parser
if (result)
{
return finish_input(*sax, strict);
if (strict)
{
// strict mode: next byte must be EOF
if (get_token() != token_type::end_of_input)
{
return syntax_error(*sax, exception_message(token_type::end_of_input, "value"));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
}
return false;
return result;
}
private:
@@ -189,39 +202,6 @@ class parser
return m_lexer.get_string();
}
/*!
@brief apply the end-of-input handling after a value was parsed
In strict mode, the next token must be the end of the input; otherwise
parse_error.101 is reported through @a sax. In non-strict mode, the
lookahead is released so the caller can keep reading the input right after
the parsed value.
@param[in,out] sax the SAX parser to report an error to
@param[in] strict whether to expect the last token to be EOF
@return true, or what @a sax returned for the error
*/
template<typename SAX>
bool finish_input(SAX& sax, const bool strict)
{
if (strict)
{
// strict mode: next byte must be EOF
if (get_token() != token_type::end_of_input)
{
return syntax_error(sax, exception_message(token_type::end_of_input, "value"));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
return true;
}
/*!
@brief run a DOM SAX parser to completion and position the lexer
@@ -241,7 +221,20 @@ class parser
{
sax_parse_internal(&sdp);
static_cast<void>(finish_input(sdp, strict));
if (strict)
{
// in strict mode, input must be completely read
if (get_token() != token_type::end_of_input)
{
syntax_error(sdp, exception_message(token_type::end_of_input, "value"));
}
}
else
{
// the caller keeps using the input: position it right after
// the value by leaving the character that terminated it
m_lexer.release_lookahead();
}
return !sdp.is_errored();
}
@@ -411,7 +404,7 @@ class parser
{
if (JSON_HEDLEY_UNLIKELY(m_lexer.get_chars_read_total() == 1))
{
return syntax_error(*sax, empty_input_message());
return syntax_error(*sax, "attempting to parse an empty input; check that your input string or stream contains the expected JSON");
}
return syntax_error(*sax, exception_message(token_type::literal_or_value, "value"));
@@ -96,7 +96,7 @@ JSON_HEDLEY_ALWAYS_INLINE std::size_t find_string_special(const unsigned char* d
// printable ASCII except '"' and '\\' - is emitted unchanged. Note this differs
// from is_string_special() only in that 0x7F is also a stop (it is escaped as
// \u007f under ensure_ascii).
constexpr bool is_ascii_copyable(unsigned char c) noexcept
inline bool is_ascii_copyable(unsigned char c) noexcept
{
return c >= 0x20u && c < 0x7Fu && c != '"' && c != '\\';
}
+90 -39
View File
@@ -108,18 +108,33 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{
JSON_ASSERT(m_object != nullptr);
if (m_object->is_object())
switch (m_object->m_data.m_type)
{
case value_t::object:
{
m_it.object_iterator = typename object_t::iterator();
break;
}
else if (m_object->is_array())
case value_t::array:
{
m_it.array_iterator = typename array_t::iterator();
break;
}
else
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// primitive (is_primitive(), including discarded)
m_it.primitive_iterator = primitive_iterator_t();
break;
}
}
}
@@ -234,28 +249,33 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{
JSON_ASSERT(m_object != nullptr);
if (m_object->is_object())
switch (m_object->m_data.m_type)
{
case value_t::object:
{
m_it.object_iterator = m_object->m_data.m_value.object->end();
}
else if (m_object->is_array())
{
m_it.array_iterator = m_object->m_data.m_value.array->end();
}
else
{
// primitive (is_primitive(), including discarded)
m_it.primitive_iterator.set_end();
}
break;
}
/// @throw invalid_iterator.212 if @a other points to another container
template<typename IterImpl>
JSON_HEDLEY_ALWAYS_INLINE void check_same_container(const IterImpl& other) const
case value_t::array:
{
if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
m_it.array_iterator = m_object->m_data.m_value.array->end();
break;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
throw_literal<invalid_iterator>(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object);
m_it.primitive_iterator.set_end();
break;
}
}
}
@@ -404,7 +424,10 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
bool operator==(const IterImpl& other) const
{
// if objects are not the same, the comparison is undefined
check_same_container(other);
if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
{
throw_literal<invalid_iterator>(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object);
}
// value-initialized forward iterators can be compared, and must compare equal to other value-initialized iterators of the same type #4493
if (m_object == nullptr)
@@ -443,7 +466,10 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
bool operator<(const iter_impl& other) const
{
// if objects are not the same, the comparison is undefined
check_same_container(other);
if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
{
throw_literal<invalid_iterator>(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object);
}
// value-initialized forward iterators can be compared, and must compare equal to other value-initialized iterators of the same type #4493
if (m_object == nullptr)
@@ -452,19 +478,26 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
return false;
}
if (m_object->is_object())
switch (m_object->m_data.m_type)
{
case value_t::object:
throw_literal<invalid_iterator>(exception_id::iterator_order_on_object, "cannot compare order of object iterators", m_object);
}
if (m_object->is_array())
{
case value_t::array:
return (m_it.array_iterator < other.m_it.array_iterator);
}
// primitive (is_primitive(), including discarded)
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
return (m_it.primitive_iterator < other.m_it.primitive_iterator);
}
}
/*!
@brief comparison: less than or equal
@@ -501,19 +534,30 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{
JSON_ASSERT(m_object != nullptr);
if (m_object->is_object())
switch (m_object->m_data.m_type)
{
case value_t::object:
throw_literal<invalid_iterator>(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object);
}
if (m_object->is_array())
case value_t::array:
{
std::advance(m_it.array_iterator, i);
break;
}
else
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// primitive (is_primitive(), including discarded)
m_it.primitive_iterator += i;
break;
}
}
return *this;
@@ -569,19 +613,26 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{
JSON_ASSERT(m_object != nullptr);
if (m_object->is_object())
switch (m_object->m_data.m_type)
{
case value_t::object:
throw_literal<invalid_iterator>(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object);
}
if (m_object->is_array())
{
case value_t::array:
return m_it.array_iterator - other.m_it.array_iterator;
}
// primitive (is_primitive(), including discarded)
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
return m_it.primitive_iterator - other.m_it.primitive_iterator;
}
}
/*!
@brief access to successor
@@ -109,9 +109,11 @@ template<typename IteratorType> class iteration_proxy_value
{
JSON_ASSERT(anchor.m_object != nullptr);
if (anchor.m_object->is_array())
switch (anchor.m_object->type())
{
// use integer array index as key
case value_t::array:
{
if (array_index != array_index_last)
{
int_to_string( array_index_str, array_index );
@@ -120,15 +122,23 @@ template<typename IteratorType> class iteration_proxy_value
return array_index_str;
}
if (anchor.m_object->is_object())
{
// use key from the object
case value_t::object:
return anchor.key();
}
// use an empty key for all primitive types
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
return empty_str;
}
}
/// return value of the iterator
typename IteratorType::reference value() const
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -55,66 +55,6 @@ using output_adapter_t = std::shared_ptr<output_adapter_protocol<CharType>>;
/// output_vector_sink also appends writes of at least this size with insert()
constexpr std::size_t max_reserve_size = 256;
/// @brief copy @a length characters, at most 32, without a call of memcpy()
///
/// Short strings are the typical payload of a binary value, and a call of
/// memcpy() costs more than copying them. A copy of 16 to 32 characters is two
/// possibly overlapping 16-byte copies, one of 8 to 15 two 8-byte copies, and
/// so on down to a single character; each is a fixed-size load and store.
/// Nothing outside of [src, src + length) is read or written.
template<typename CharType>
inline void copy_short(CharType* dst, const CharType* src, const std::size_t length, std::true_type /*is_byte*/) noexcept
{
JSON_ASSERT(length <= 32);
if (length >= 16)
{
std::array<char, 16> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<char, 16> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(first.data(), src, 16);
std::memcpy(last.data(), src + (length - 16), 16);
std::memcpy(dst, first.data(), 16);
std::memcpy(dst + (length - 16), last.data(), 16);
}
else if (length >= 8)
{
std::array<char, 8> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<char, 8> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(first.data(), src, 8);
std::memcpy(last.data(), src + (length - 8), 8);
std::memcpy(dst, first.data(), 8);
std::memcpy(dst + (length - 8), last.data(), 8);
}
else if (length >= 4)
{
std::array<char, 4> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<char, 4> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(first.data(), src, 4);
std::memcpy(last.data(), src + (length - 4), 4);
std::memcpy(dst, first.data(), 4);
std::memcpy(dst + (length - 4), last.data(), 4);
}
else if (length != 0)
{
// 1 to 3: the first, the middle, and the last character
const CharType a = src[0];
const CharType b = src[length >> 1];
const CharType c = src[length - 1];
dst[0] = a;
dst[length >> 1] = b;
dst[length - 1] = c;
}
}
/// @brief copy @a length characters, of a character type wider than a byte
template<typename CharType>
inline void copy_short(CharType* dst, const CharType* src, const std::size_t length, std::false_type /*is_byte*/) noexcept
{
std::memcpy(dst, src, length * sizeof(CharType));
}
/// the number of characters an empty vector is grown to by output_vector_sink
constexpr std::size_t min_vector_sink_capacity = 64;
/// @brief non-virtual output sink writing into a std::vector
///
/// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
@@ -138,7 +78,7 @@ class output_vector_sink
{
public:
explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
: v(&vec), cursor(vec.data() + vec.size()), limit(cursor)
: v(&vec), position(vec.size())
{}
output_vector_sink(const output_vector_sink&) = delete;
@@ -146,7 +86,7 @@ class output_vector_sink
output_vector_sink& operator=(output_vector_sink&&) = delete;
output_vector_sink(output_vector_sink&& other) noexcept
: v(other.v), cursor(other.cursor), limit(other.limit)
: v(other.v), position(other.position)
{
other.v = nullptr;
}
@@ -155,7 +95,7 @@ class output_vector_sink
{
if (v != nullptr)
{
v->resize(position());
v->resize(position);
}
}
@@ -164,11 +104,11 @@ class output_vector_sink
JSON_HEDLEY_ALWAYS_INLINE
void write_character(CharType c)
{
if (JSON_HEDLEY_UNLIKELY(cursor == limit))
if (JSON_HEDLEY_UNLIKELY(position == v->size()))
{
grow(1);
}
*cursor++ = c;
(*v)[position++] = c;
}
// no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
@@ -177,7 +117,7 @@ class output_vector_sink
JSON_HEDLEY_ALWAYS_INLINE
void write_characters(const CharType* s, std::size_t length)
{
if (JSON_HEDLEY_UNLIKELY(space() < length))
if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
{
if (length >= max_reserve_size)
{
@@ -186,15 +126,10 @@ class output_vector_sink
}
grow(length);
}
if (length <= 32)
if (length != 0)
{
copy_short(cursor, s, length, std::integral_constant<bool, sizeof(CharType) == 1>());
cursor += length;
}
else
{
std::memcpy(cursor, s, length * sizeof(CharType));
cursor += length;
std::memcpy(v->data() + position, s, length * sizeof(CharType));
position += length;
}
}
@@ -209,37 +144,21 @@ class output_vector_sink
CharType* reserve(const std::size_t length)
{
JSON_ASSERT(length <= max_reserve_size);
if (JSON_HEDLEY_UNLIKELY(space() < length))
if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
{
grow(length);
}
return cursor;
return v->data() + position;
}
/// keep the first @a length characters written to the space from reserve()
JSON_HEDLEY_ALWAYS_INLINE
void commit(const std::size_t length) noexcept
{
cursor += length;
position += length;
}
/// nothing to pass on: the characters are in the vector (see output_adapter_sink)
void flush() noexcept
{}
private:
/// number of characters of *v written so far
std::size_t position() const noexcept
{
return static_cast<std::size_t>(cursor - v->data());
}
/// number of characters that fit the vector behind the write position
std::size_t space() const noexcept
{
return static_cast<std::size_t>(limit - cursor);
}
/// make room for at least @a length more characters after the position;
/// kept out of line so that the write functions stay small enough for
/// GCC to inline them into the writer
@@ -247,15 +166,12 @@ class output_vector_sink
void grow(const std::size_t length)
{
// use the capacity the vector already has, else at least double it;
// never less than min_vector_sink_capacity, so that a small value does not
// never less than 64 characters, so that a small value does not
// reallocate for every few bytes
const std::size_t written = position();
const std::size_t needed = written + length;
v->resize((std::max)(min_vector_sink_capacity, needed <= v->capacity()
const std::size_t needed = position + length;
v->resize((std::max)(std::size_t{64}, needed <= v->capacity()
? v->capacity()
: (std::max)(needed, v->capacity() * 2)));
cursor = v->data() + written;
limit = v->data() + v->size();
}
/// append @a length (at least max_reserve_size) characters that do not fit
@@ -265,24 +181,20 @@ class output_vector_sink
JSON_HEDLEY_NEVER_INLINE
void append(const CharType* s, const std::size_t length)
{
const std::size_t written = position();
v->resize(written);
if (v->capacity() - written < length)
v->resize(position);
if (v->capacity() - position < length)
{
// grow geometrically, with room for the writes that follow, so
// that the next grow() need not reallocate right away
v->reserve((std::max)(v->capacity() * 2, written + length + max_reserve_size));
v->reserve((std::max)(v->capacity() * 2, position + length + max_reserve_size));
}
v->insert(v->end(), s, s + length);
cursor = v->data() + v->size();
limit = cursor;
position = v->size();
}
std::vector<CharType, AllocatorType>* v;
/// the write position: in *v, at the number of characters written so far
CharType* cursor;
/// the end of *v
CharType* limit;
/// number of characters of *v written so far
std::size_t position;
};
/// output adapter for byte vectors
@@ -365,17 +277,9 @@ class output_string_adapter : public output_adapter_protocol<CharType>
///
/// Wraps the polymorphic output_adapter_t so the same binary_writer template can
/// also target arbitrary adapters (output streams, strings, user-provided
/// adapters) via the `output_adapter`-based overloads.
///
/// A write through the adapter is a virtual call, and for a stream a call with
/// a sentry on top, which costs more than encoding the few bytes of most values.
/// The sink therefore collects what is written in a buffer and passes it on in
/// one piece when it is full, a write is too large for it, or @ref flush is
/// called. Everything written reaches the adapter in the same order and with
/// the same bytes; the destructor passes on what is left, so that a write that
/// throws leaves the adapter with exactly the bytes written before it. Only
/// @ref flush can report an error of the adapter (the destructor must not
/// throw), so it has to be called once the value is written.
/// adapters) via the `output_adapter`-based overloads. Each write still goes
/// through one virtual call, exactly as before; only the concrete sinks above
/// avoid it.
template<typename CharType>
class output_adapter_sink
{
@@ -386,121 +290,36 @@ class output_adapter_sink
JSON_ASSERT(oa);
}
output_adapter_sink(const output_adapter_sink&) = delete;
output_adapter_sink& operator=(const output_adapter_sink&) = delete;
output_adapter_sink& operator=(output_adapter_sink&&) = delete;
output_adapter_sink(output_adapter_sink&& other) noexcept
: oa(std::move(other.oa)), used(other.used)
{
std::memcpy(buffer.data(), other.buffer.data(), other.used * sizeof(CharType));
other.used = 0;
}
~output_adapter_sink()
{
// a normal return has flushed already, so this is for a write that threw
if (used != 0)
{
JSON_TRY
{
flush();
}
JSON_CATCH(...)
{
// nothing can be done about an adapter that throws here
}
}
}
JSON_HEDLEY_ALWAYS_INLINE
void write_character(CharType c)
{
if (JSON_HEDLEY_UNLIKELY(used == buffer_size))
{
flush();
}
buffer[used++] = c;
oa->write_character(c);
}
// no JSON_HEDLEY_NON_NULL: forwards (null, 0) for empty payloads, exactly as
// the type-erased path already did before this sink existed
JSON_HEDLEY_ALWAYS_INLINE
void write_characters(const CharType* s, std::size_t length)
{
if (JSON_HEDLEY_UNLIKELY(buffer_size - used < length))
{
write_large(s, length);
return;
}
if (length <= 32)
{
copy_short(buffer.data() + used, s, length, std::integral_constant<bool, sizeof(CharType) == 1>());
used += length;
}
else
{
std::memcpy(buffer.data() + used, s, length * sizeof(CharType));
used += length;
}
oa->write_characters(s, length);
}
/// space for writing up to @a length characters (see
/// output_vector_sink::reserve)
JSON_HEDLEY_ALWAYS_INLINE
CharType* reserve(const std::size_t length)
CharType* reserve(const std::size_t length) noexcept
{
JSON_ASSERT(length <= max_reserve_size);
if (JSON_HEDLEY_UNLIKELY(buffer_size - used < length))
{
flush();
}
return buffer.data() + used;
static_cast<void>(length);
return reserved.data();
}
/// keep the first @a length characters written to the space from reserve()
JSON_HEDLEY_ALWAYS_INLINE
void commit(const std::size_t length) noexcept
/// write the first @a length characters written to the space from reserve()
void commit(const std::size_t length)
{
used += length;
}
/// pass everything written so far on to the adapter
void flush()
{
if (used != 0)
{
// reset first: the characters count as written even if the adapter throws
const std::size_t length = used;
used = 0;
oa->write_characters(buffer.data(), length);
}
oa->write_characters(reserved.data(), length);
}
private:
/// a write that does not fit the space left in the buffer
JSON_HEDLEY_NEVER_INLINE
void write_large(const CharType* s, const std::size_t length)
{
flush();
if (length >= buffer_size)
{
oa->write_characters(s, length);
}
else
{
std::memcpy(buffer.data(), s, length * sizeof(CharType));
used = length;
}
}
/// large enough for reserve() at any time
static constexpr std::size_t buffer_size = 4 * max_reserve_size;
output_adapter_t<CharType> oa;
/// number of characters of @ref buffer not yet passed on
std::size_t used = 0;
std::array<CharType, buffer_size> buffer{};
std::array<CharType, max_reserve_size> reserved{};
};
/// @brief whether std::basic_string<CharType> has a non-deprecated std::char_traits
File diff suppressed because it is too large. Load diff
+4 -87
View File
@@ -48,24 +48,6 @@ inline std::string hex_byte(const std::uint8_t byte)
return result;
}
/*!
@brief write a UTF-16 code unit as four lowercase hexadecimal digits
The lowercase counterpart of hex_byte(), used for the `\\uXXXX` escapes of the
serializers, whose output is lowercase. Writes exactly four characters to
@a out and returns the pointer past them.
*/
JSON_HEDLEY_ALWAYS_INLINE char* hex4_lower(const std::uint16_t codeunit, char* out) noexcept
{
constexpr const char* nibble_to_hex = "0123456789abcdef";
const unsigned int unit = codeunit;
out[0] = nibble_to_hex[(unit >> 12u) & 0x0Fu];
out[1] = nibble_to_hex[(unit >> 8u) & 0x0Fu];
out[2] = nibble_to_hex[(unit >> 4u) & 0x0Fu];
out[3] = nibble_to_hex[unit & 0x0Fu];
return out + 4;
}
///////////////////
// UTF-8 encoding //
///////////////////
@@ -121,66 +103,6 @@ void encode_utf8(std::uint32_t cp, const Out& out)
}
}
/*!
@brief the shape of a well-formed UTF-8 sequence, by its lead byte
@a length is 1 for ASCII, 2 to 4 for a valid lead byte, and 0 for a byte that
cannot start a well-formed sequence (a continuation byte, 0xC0, 0xC1, or 0xF5
to 0xFF). For a multibyte sequence, the second byte must lie in
[@a second_lo, @a second_hi], and every later byte in [0x80, 0xBF]
(RFC 3629, section 4; Unicode Table 3-7). The narrower second-byte ranges
exclude overlong forms (0xE0, 0xF0), surrogates (0xED), and code points
beyond U+10FFFF (0xF4).
*/
struct utf8_sequence_shape
{
std::uint8_t length;
std::uint8_t second_lo;
std::uint8_t second_hi;
};
/// @return the shape of a UTF-8 sequence starting with @a lead, see utf8_sequence_shape
inline utf8_sequence_shape utf8_shape(const std::uint8_t lead) noexcept
{
if (lead < 0x80u)
{
return {1, 0, 0};
}
if (lead < 0xC2u)
{
return {0, 0, 0};
}
if (lead <= 0xDFu)
{
return {2, 0x80u, 0xBFu};
}
if (lead == 0xE0u)
{
return {3, 0xA0u, 0xBFu};
}
if (lead == 0xEDu)
{
return {3, 0x80u, 0x9Fu};
}
if (lead <= 0xEFu)
{
return {3, 0x80u, 0xBFu};
}
if (lead == 0xF0u)
{
return {4, 0x90u, 0xBFu};
}
if (lead <= 0xF3u)
{
return {4, 0x80u, 0xBFu};
}
if (lead == 0xF4u)
{
return {4, 0x80u, 0x8Fu};
}
return {0, 0, 0};
}
///////////////////
// UTF-8 decoding //
///////////////////
@@ -196,7 +118,7 @@ This is a single-byte step of a "shift-based" UTF-8 decoder originally
written by Björn Hoehrmann. See
http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.
The library checks UTF-8 well-formedness (RFC 3629, section 4) in five
The library checks UTF-8 well-formedness (RFC 3629, section 4) in three
places, which differ in speed, diagnostics, and how they read the input:
- decode() below: the serializer, to escape and, in strict mode, reject
@@ -210,15 +132,10 @@ places, which differ in speed, diagnostics, and how they read the input:
bulk string scan, the bulk path of the BON8 reader, and the BON8 writer.
They must accept exactly what the lexer's switch accepts.
- the byte path of binary_reader::get_bon8_string(): BON8 input without bulk
access, and the bytes the bulk path leaves to it. It keeps its own ranges:
taking them from utf8_shape() made BON8 decoding 2-3% slower (#5891).
- check_utf8() in detail/view/edit.hpp: strings stored in an editable
json_document, with the index of the offending byte. It also uses
utf8_shape().
access, and the bytes the bulk path leaves to it.
All five must accept the same set of sequences, which the test "UTF-8
validators accept the same sequences" in unit-class_lexer.cpp checks
exhaustively against decode() and utf8_shape().
All four must accept the same set of sequences, so a change to one needs a
matching change to the others.
@param[in,out] state the current decoder state
@param[in,out] codep codepoint (valid only if resulting state is UTF8_ACCEPT)
-29
View File
@@ -177,35 +177,6 @@ inline bool owns_memory(const value_t t) noexcept
}
}
/*!
@brief whether a value of the given type is neither an object nor an array
Unlike basic_json::is_primitive(), this includes value_t::discarded: it names
the types for which iterators, JSON Pointer resolution, and the serializer take
the "single value" path. Every enumerator is listed so that -Wswitch-enum flags
this function when a new type is added.
*/
inline bool is_primitive(const value_t t) noexcept
{
switch (t)
{
case value_t::object:
case value_t::array:
return false;
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
return true;
}
}
/*!
@brief compare an integer with a floating point number without precision loss
+41 -19
View File
@@ -51,6 +51,11 @@ struct first_true<false, Conditions...> : std::integral_constant < int, 1 + firs
inline void check_utf8(const char* s, std::size_t n)
{
const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const auto hex = [](unsigned char c)
{
constexpr const char* digits = "0123456789ABCDEF";
return std::string{digits[c >> 4u], digits[c & 0xFu]};
};
for (std::size_t i = 0; i < n;)
{
const unsigned char c = p[i];
@@ -59,22 +64,39 @@ inline void check_utf8(const char* s, std::size_t n)
++i;
continue;
}
const utf8_sequence_shape shape = utf8_shape(c);
const std::size_t len = shape.length;
if (len == 0)
std::size_t len = 0;
unsigned char lo = 0x80;
unsigned char hi = 0xBF;
if (c >= 0xC2 && c <= 0xDF)
{
throw_type_error(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex_byte(c)));
len = 2;
}
else if (c >= 0xE0 && c <= 0xEF)
{
len = 3;
lo = c == 0xE0 ? 0xA0 : 0x80;
hi = c == 0xED ? 0x9F : 0xBF;
}
else if (c >= 0xF0 && c <= 0xF4)
{
len = 4;
lo = c == 0xF0 ? 0x90 : 0x80;
hi = c == 0xF4 ? 0x8F : 0xBF;
}
else
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex(c)));
}
for (std::size_t k = 1; k < len; ++k)
{
if (i + k == n)
{
throw_type_error(exception_id::invalid_utf8, concat("incomplete UTF-8 string; last byte: 0x", hex_byte(p[n - 1])));
throw_type_error(316, concat("incomplete UTF-8 string; last byte: 0x", hex(p[n - 1])));
}
const unsigned char b = p[i + k];
if (b < (k == 1 ? shape.second_lo : 0x80) || b > (k == 1 ? shape.second_hi : 0xBF))
if (b < (k == 1 ? lo : 0x80) || b > (k == 1 ? hi : 0xBF))
{
throw_type_error(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex_byte(b)));
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex(b)));
}
}
i += len;
@@ -122,7 +144,7 @@ class editor
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object) && o->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(exception_id::subscript_wrong_type, "cannot use operator[] with a string argument with ", object.type_name());
throw_type_error(305, "cannot use operator[] with a string argument with ", object.type_name());
}
check_utf8(key.data(), key.size());
const encoded e = encode(std::forward<V>(value));
@@ -173,7 +195,7 @@ class editor
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(exception_id::subscript_wrong_type, "cannot use operator[] with a numeric argument with ", array.type_name());
throw_type_error(305, "cannot use operator[] with a numeric argument with ", array.type_name());
}
check_index(idx, a->len);
const encoded e = encode(std::forward<V>(value));
@@ -190,7 +212,7 @@ class editor
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array) && a->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(exception_id::push_back_wrong_type, "cannot use push_back() with ", array.type_name());
throw_type_error(308, "cannot use push_back() with ", array.type_name());
}
const encoded e = encode(std::forward<V>(value));
if (a->kind == static_cast<std::uint8_t>(value_t::null))
@@ -214,7 +236,7 @@ class editor
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(exception_id::insert_wrong_type, "cannot use insert() with ", array.type_name());
throw_type_error(309, "cannot use insert() with ", array.type_name());
}
check_index(idx, a->len + 1);
const encoded e = encode(std::forward<V>(value));
@@ -234,7 +256,7 @@ class editor
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object))
{
throw_type_error(exception_id::erase_wrong_type, "cannot use erase() with ", object.type_name());
throw_type_error(307, "cannot use erase() with ", object.type_name());
}
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
@@ -252,7 +274,7 @@ class editor
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(exception_id::erase_wrong_type, "cannot use erase() with ", array.type_name());
throw_type_error(307, "cannot use erase() with ", array.type_name());
}
check_index(idx, a->len);
node* const h = block_of(m_doc, a, 0);
@@ -275,7 +297,7 @@ class editor
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_doc != &m_doc || v.m_node == nullptr))
{
throw_invalid_iterator(exception_id::iterator_from_other_value, "view does not belong to this document");
throw_invalid_iterator(202, "view does not belong to this document");
}
edit_state_of(m_doc);
return const_cast<node*>(v.m_node); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
@@ -285,7 +307,7 @@ class editor
{
if (idx >= limit)
{
throw_array_index_out_of_range(idx);
throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
}
}
@@ -426,7 +448,7 @@ class editor
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_node == nullptr))
{
throw_type_error(exception_id::type_mismatch, "type must be a value, but is ", "discarded");
throw_type_error(302, "type must be a value, but is ", "discarded");
}
encoded r;
if (!is_container(*v.m_node))
@@ -576,7 +598,7 @@ class editor
{
if (NLOHMANN_VIEW_UNLIKELY(len >= 0xFFFFFFFFu))
{
throw_out_of_range(exception_id::document_too_large, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
throw_out_of_range(416, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
}
node n = plain_node(value_t::string);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
@@ -651,12 +673,12 @@ class editor
return string_node(s.data(), s.size());
}
case value_t::binary:
throw_type_error(exception_id::document_binary_value, "cannot store a binary value in a json_document", "");
throw_type_error(319, "cannot store a binary value in a json_document", "");
case value_t::discarded:
case value_t::object:
case value_t::array:
default:
throw_type_error(exception_id::type_mismatch, "type must be a value, but is ", "discarded");
throw_type_error(302, "type must be a value, but is ", "discarded");
}
}
@@ -84,7 +84,7 @@ inline std::size_t text_capacity(std::size_t cap, std::size_t used, std::size_t
constexpr std::size_t limit = 0xFFFFFFFFu;
if (NLOHMANN_VIEW_UNLIKELY(used > limit || n > limit - used))
{
throw_out_of_range(exception_id::document_too_large, "edits of 4 GiB or more are not supported by json_document");
throw_out_of_range(416, "edits of 4 GiB or more are not supported by json_document");
}
const std::size_t needed = used + n;
const std::size_t wanted = needed + (std::min)(limit - needed, std::size_t{256});
@@ -135,7 +135,7 @@ inline void reserve_moved(document_data& d)
document_data::edit_state& e = edit_state_of(d);
if (e.moved.size() >= 0xFFFFFFFFu)
{
throw_out_of_range(exception_id::document_too_large, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
throw_out_of_range(416, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
}
if (e.moved.size() == e.moved.capacity() || e.moved_cap.size() == e.moved_cap.capacity())
{
+7 -27
View File
@@ -25,52 +25,32 @@ namespace view
// Exceptions are thrown out of line, so that the accessors that may throw stay
// small enough to be inlined.
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(exception_id id, const char* prefix, const char* type)
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const char* prefix, const char* type)
{
NLOHMANN_VIEW_THROW(type_error::create(id, concat(prefix, type), nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(exception_id id, const std::string& msg)
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(type_error::create(id, msg, nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(exception_id id, const std::string& msg)
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(out_of_range::create(id, msg, nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_iterator(exception_id id, const char* msg)
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_iterator(int id, const char* msg)
{
NLOHMANN_VIEW_THROW(invalid_iterator::create(id, msg, nullptr));
}
/// a parse error without a position (as those of json_pointer)
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_parse_error(exception_id id, const std::string& msg)
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_parse_error(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(parse_error::create(id, 0, msg, nullptr));
}
/// out_of_range.401: the array index @a idx is not below the size
template<typename Index>
[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_array_index_out_of_range(const Index idx)
{
throw_out_of_range(exception_id::array_index_out_of_range, concat("array index ", std::to_string(idx), " is out of range"));
}
/// out_of_range.402: the JSON Pointer token '-' of an array of @a size elements
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_array_index_past_end(std::size_t size)
{
throw_out_of_range(exception_id::pointer_past_the_end_index, concat("array index '-' (", std::to_string(size), ") is out of range"));
}
/// out_of_range.403: the object has no member @a key
template<typename Key>
[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_key_not_found(const Key& key)
{
throw_out_of_range(exception_id::key_not_found, concat("key '", key, "' not found"));
}
/// SAX handler of the error replay: accepts every event without storing it,
/// and throws the exception the parser reports
template<typename BasicJsonType>
@@ -169,7 +149,7 @@ template<typename BasicJsonType>
if (f.code == error_code::input_too_large)
{
// (the limit is detail::view::max_input_size(): 4 GiB minus 16 bytes)
NLOHMANN_VIEW_THROW(out_of_range::create(exception_id::document_too_large, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
}
throwing_sax<BasicJsonType> sax;
const bool accepted = BasicJsonType::sax_parse(src, src + size, &sax, input_format_t::json, true, ignore_comments, ignore_trailing_commas);
@@ -189,7 +169,7 @@ template<typename BasicJsonType>
}
}
pos.chars_read_current_line = off + 1 - line_start;
NLOHMANN_VIEW_THROW(parse_error::create(exception_id::syntax_error, pos, "syntax error while parsing value", nullptr));
NLOHMANN_VIEW_THROW(parse_error::create(101, pos, "syntax error while parsing value", nullptr));
// LCOV_EXCL_STOP
}
+4 -4
View File
@@ -186,7 +186,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
inline std::vector<std::uint8_t> save_image(const document_data& d)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(exception_id::document_image_unsupported, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
const std::size_t arena_size = d.arena_size;
std::vector<node> compacted;
@@ -203,7 +203,7 @@ inline std::vector<std::uint8_t> save_image(const document_data& d)
if (NLOHMANN_VIEW_UNLIKELY(text_size >= image_limit || total_arena >= image_limit || count >= image_limit))
{
// LCOV_EXCL_START (4 GiB)
throw_out_of_range(exception_id::document_too_large, "images of 4 GiB or more are not supported by json_document");
throw_out_of_range(416, "images of 4 GiB or more are not supported by json_document");
// LCOV_EXCL_STOP
}
image_header h{};
@@ -660,7 +660,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_image(const char* what)
{
throw_parse_error(exception_id::invalid_document_image, concat("invalid json_document image: ", what));
throw_parse_error(116, concat("invalid json_document image: ", what));
}
/// Read an image into d. The text and the decoded strings stay in the image;
@@ -668,7 +668,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
inline void load_image(document_data& d, const std::uint8_t* image, std::size_t size, image_check check)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(exception_id::document_image_unsupported, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
if (image == nullptr || size < sizeof(image_header))
{
+1 -1
View File
@@ -110,7 +110,7 @@ class view_iterator
{
if (NLOHMANN_VIEW_UNLIKELY(m_value_offset == 0))
{
throw_invalid_iterator(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators");
throw_invalid_iterator(207, "cannot use key() for non-object iterators");
}
return string_view_t(m_doc->str(*m_pos), m_pos->len);
}
+50 -14
View File
@@ -32,15 +32,51 @@ enum class pointer_mode
contains, ///< as basic_json::contains(): no exceptions at all
};
/// the outcome of reading an array index: detail::parse_array_index(), shared with json_pointer
using index_status = detail::array_index_status;
/// the outcome of reading an array index from a reference token
enum class index_status
{
ok,
leading_zero, ///< parse_error.106
not_number, ///< parse_error.109
unresolved, ///< out_of_range.404
too_large, ///< out_of_range.410
};
/// reads an array index like json_pointer::array_index (RFC 6901, Sect. 4),
/// but reports errors instead of throwing them
template<typename StringType>
index_status array_index(const StringType& s, std::size_t& idx) noexcept
{
return detail::parse_array_index(s, idx);
// as basic_json: the empty token and "-" (past the last element) are
// unresolved rather than not a number
if (s.empty() || (s.size() == 1 && s[0] == '-'))
{
return index_status::unresolved;
}
if (s[0] < '0' || s[0] > '9')
{
return index_status::not_number;
}
if (s.size() > 1 && s[0] == '0')
{
return index_status::leading_zero;
}
std::uint64_t v = 0;
for (const auto c : s)
{
const auto d = static_cast<unsigned>(static_cast<unsigned char>(c)) - '0';
if (d > 9 || v > ((std::numeric_limits<std::uint64_t>::max)() - d) / 10)
{
return index_status::unresolved; // not a number, or beyond unsigned long long
}
v = (v * 10) + d;
}
if (v >= (std::numeric_limits<std::size_t>::max)()) // (std::size_t converts to std::uint64_t implicitly)
{
return index_status::too_large;
}
idx = static_cast<std::size_t>(v);
return index_status::ok;
}
/// throws the exception json_pointer::array_index throws for this status
@@ -50,15 +86,15 @@ template<typename StringType>
switch (status)
{
case index_status::leading_zero:
throw_parse_error(exception_id::pointer_index_leading_zero, concat("array index '", s, "' must not begin with '0'"));
case index_status::not_a_number:
throw_parse_error(exception_id::pointer_index_not_a_number, concat("array index '", s, "' is not a number"));
case index_status::exceeds_size_type:
throw_out_of_range(exception_id::value_out_of_range, concat("array index ", s, " exceeds size_type")); // LCOV_EXCL_LINE
throw_parse_error(106, concat("array index '", s, "' must not begin with '0'"));
case index_status::not_number:
throw_parse_error(109, concat("array index '", s, "' is not a number"));
case index_status::too_large:
throw_out_of_range(410, concat("array index ", s, " exceeds size_type")); // LCOV_EXCL_LINE
case index_status::unresolved:
case index_status::ok:
default:
throw_out_of_range(exception_id::pointer_unresolved, concat("unresolved reference token '", s, "'"));
throw_out_of_range(404, concat("unresolved reference token '", s, "'"));
}
}
@@ -83,7 +119,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
{
if (mode == pointer_mode::checked)
{
throw_key_not_found(token);
throw_out_of_range(403, concat("key '", token, "' not found"));
}
return View();
}
@@ -95,7 +131,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
{
if (throwing)
{
throw_array_index_past_end(cur.size());
throw_out_of_range(402, concat("array index '-' (", std::to_string(cur.size()), ") is out of range"));
}
return View();
}
@@ -103,7 +139,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
const index_status status = array_index(token, idx);
if (status != index_status::ok)
{
const bool parse_error = status == index_status::leading_zero || status == index_status::not_a_number;
const bool parse_error = status == index_status::leading_zero || status == index_status::not_number;
if (throwing || (mode == pointer_mode::value && parse_error))
{
throw_array_index_error(status, token);
@@ -114,7 +150,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
{
if (mode == pointer_mode::checked)
{
throw_array_index_out_of_range(idx);
throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
}
return View();
}
@@ -124,7 +160,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
{
if (throwing)
{
throw_out_of_range(exception_id::pointer_unresolved, concat("unresolved reference token '", token, "'"));
throw_out_of_range(404, concat("unresolved reference token '", token, "'"));
}
return View();
}
+378 -257
View File
@@ -23,6 +23,7 @@
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/simd.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -80,13 +81,6 @@ class output_buffer
put(s, N - 1);
}
void put_repeated(char c, std::size_t n)
{
reserve(n);
std::memset(m_pos, c, n);
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
@@ -143,9 +137,27 @@ class output_buffer
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
std::size_t i = 0;
#if NLOHMANN_VIEW_SSE2
// (16 bytes per step; with NEON, the mask of a step costs more than it saves on the
// short runs that are typical here)
// NOLINTBEGIN(portability-simd-intrinsics)
for (; i + 16 <= n; i += 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
// (a byte is below 0x20 if it is its own minimum with 0x1F)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (mask != 0)
{
return i + static_cast<std::size_t>(count_trailing_zeros(mask));
}
}
// NOLINTEND(portability-simd-intrinsics)
#endif
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
@@ -196,6 +208,70 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
return true;
}
#if NLOHMANN_VIEW_VECTOR
/// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
/// backslash, a control character, DEL, or not ASCII; a signed compare with
/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
/// byte (x86) or one nibble per byte (NEON)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
#endif
}
/// the width in bits of a byte in the mask of ascii_stop_mask()
constexpr unsigned ascii_stop_bits() noexcept
{
return NLOHMANN_VIEW_NEON ? 4u : 1u;
}
#endif
/// As find_ascii_copyable_run(), 16 bytes per step. end is the end of the
/// buffer that holds the n bytes at p (a short run is read as 16 bytes if
/// they are in the buffer).
inline std::size_t ascii_copyable_run(const unsigned char* p, std::size_t n, const unsigned char* end) noexcept
{
#if NLOHMANN_VIEW_VECTOR
if (n >= 16)
{
std::size_t i = 0;
for (; i + 16 <= n; i += 16)
{
const std::uint64_t mask = ascii_stop_mask(p + i);
if (mask != 0)
{
return i + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits());
}
}
if (i == n)
{
return n;
}
// the last 16 bytes: those before i are known to be copyable
const std::uint64_t mask = ascii_stop_mask(p + n - 16);
return mask != 0 ? n - 16 + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits()) : n;
}
if (end - p >= 16)
{
// (the bytes after the run are masked out)
const std::uint64_t mask = ascii_stop_mask(p) & ((std::uint64_t{1} << (n * ascii_stop_bits())) - 1u);
return mask != 0 ? static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits() : n;
}
#else
static_cast<void>(end);
#endif
return find_ascii_copyable_run(p, n);
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
@@ -275,6 +351,39 @@ NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexc
return w + N - 1;
}
/// the depth of the innermost open container, counted only for indented output
template<bool Counted>
struct depth_counter
{
std::size_t value = 0;
void enter() noexcept
{
++value;
}
std::size_t leave() noexcept
{
return --value;
}
std::size_t get() const noexcept
{
return value;
}
};
template<>
struct depth_counter<false>
{
void enter() noexcept {}
std::size_t leave() noexcept // NOLINT(readability-convert-member-functions-to-static): same interface as depth_counter<true>
{
return 0;
}
std::size_t get() const noexcept // NOLINT(readability-convert-member-functions-to-static): same interface as depth_counter<true>
{
return 0;
}
};
/// how the view's dump() writes a value
struct dump_style
{
@@ -310,140 +419,96 @@ class view_serializer
{}
void dump(const node* root)
{
if (!m_style.pretty)
{
if (m_style.ensure_ascii)
{
dump_compact_ascii(root);
dump_ascii(root);
}
else if (m_style.pretty)
{
dump_pretty(root);
}
else if (m_style.source_numbers)
{
dump_compact<true, false>(root);
dump_text<true, false, false>(root);
}
else
{
dump_compact<false, false>(root);
}
return;
}
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
bool first; ///< nothing written yet
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const node* n = root;
for (;;)
{
// write the value at n
if (is_container(*n))
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
if (n->len == 0)
{
m_out.put(object ? "{}" : "[]", 2);
}
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
}
}
else
{
write_scalar(*n);
}
// go to the next value: close finished containers, then separate
for (;;)
{
if (stack.empty())
{
m_out.finish();
return;
}
if (stack.back().pos == stack.back().end)
{
const bool object = stack.back().object;
stack.pop_back();
newline(stack.size());
m_out.put(object ? '}' : ']');
continue;
}
frame& f = stack.back();
if (!f.first)
{
m_out.put(',');
}
f.first = false;
newline(stack.size());
if (f.object)
{
write_string(*f.pos);
if (m_style.pretty)
{
m_out.put(": ");
}
else
{
m_out.put(':');
}
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
}
else
{
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
}
break;
}
dump_text<false, false, false>(root);
}
}
private:
/// dump_text() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_ascii(const node* root)
{
if (m_style.pretty)
{
if (m_style.source_numbers)
{
dump_text<true, true, true>(root);
}
else
{
dump_text<false, true, true>(root);
}
}
else if (m_style.source_numbers)
{
dump_text<true, false, true>(root);
}
else
{
dump_text<false, false, true>(root);
}
}
/// dump_text() with indentation, out of line: the compact walk is faster
/// when it is not merged with the indented ones into dump()
NLOHMANN_VIEW_NOINLINE void dump_pretty(const node* root)
{
if (m_style.source_numbers)
{
dump_text<true, true, false>(root);
}
else
{
dump_text<false, true, false>(root);
}
}
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
/// are copied. If validate is set, the string must be valid UTF-8: the
/// runs are found by the lexer's scan, which validates the sequences, and
/// invalid UTF-8 throws what basic_json::dump() throws. (The compact output
/// writes the bytes of a damaged image unchanged, and scans without
/// validating them.)
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n, bool validate)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
const std::size_t run = validate ? string_bulk_run(s + i, n.len - i) : plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
if (validate && s[i] >= 0x80)
{
invalid_utf8(s, n.len); // (the scan stopped at ill-formed UTF-8)
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// dump_compact() with ensure_ascii, out of line: the walk without it is
/// faster when it is not merged with this one into dump()
NLOHMANN_VIEW_NOINLINE void dump_compact_ascii(const node* root)
{
if (m_style.source_numbers)
{
dump_compact<true, true>(root);
}
else
{
dump_compact<false, true>(root);
}
}
/// whether dump_compact() copies a string or key from the source: one of
/// whether dump_text() copies a string or key from the source: one of
/// the source, and with ensure_ascii one without bytes >= 0x7F (an overload
/// each, so that the walk without ensure_ascii is compiled exactly as
/// before: with GCC, a condition that only folds away later changed its
@@ -458,13 +523,155 @@ class view_serializer
return (n.flags & node_flags::storage) == 0 && is_ascii_without_delete(src + n.off, n.len);
}
/// a string with ensure_ascii (dump_compact()), out of line
NLOHMANN_VIEW_NOINLINE void write_ascii_string(const node& n)
/*!
@brief a string with ensure_ascii: the characters that are not printable ASCII as \\uXXXX
(a character beyond U+FFFF as a surrogate pair); strings of a document
are valid UTF-8, except in a damaged image loaded with image_check::bounds,
for which this throws what basic_json::dump() throws for the string
*/
NLOHMANN_VIEW_NOINLINE void write_ascii(const node& n)
{
write_string(n);
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const std::size_t size = n.len;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for k bytes
const auto room = [&](std::size_t k)
{
const auto left = static_cast<std::size_t>(lim - w);
if (NLOHMANN_VIEW_UNLIKELY(left < k))
{
m_out.set_cursor(w);
m_out.reserve(k);
w = m_out.cursor();
lim = m_out.limit();
}
};
static constexpr const char* hex = "0123456789abcdef";
// \\uXXXX
const auto u_escape = [&](std::uint32_t u)
{
w[0] = '\\';
w[1] = 'u';
w[2] = hex[(u >> 12u) & 0xFu];
w[3] = hex[(u >> 8u) & 0xFu];
w[4] = hex[(u >> 4u) & 0xFu];
w[5] = hex[u & 0xFu];
w += 6;
};
room(1);
*w++ = '"';
std::size_t i = 0;
// (a string of the source starts with a run that has nothing to escape, 16 bytes per step)
const std::size_t prefix = (n.flags & node_flags::storage) == 0 ? ascii_copyable_run(s, size, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)) : 0; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (prefix != 0)
{
room(prefix);
std::memcpy(w, s, prefix);
w += prefix;
i = prefix;
}
while (i < size)
{
// (a character that needs an escape sequence is not worth a call)
const std::size_t run = is_ascii_copyable(s[i]) ? find_ascii_copyable_run(s + i, size - i) : 0;
if (run != 0)
{
room(run);
std::memcpy(w, s + i, run);
w += run;
i += run;
continue;
}
room(12);
if (s[i] >= 0x80)
{
const unsigned char lead = s[i];
// (the two-byte sequences of Latin text and the three-byte sequences of most other
// text, with continuation bytes of any value: the ranges that are restricted (E0,
// ED, and the sequences of four bytes) go through validate_one_utf8())
if (lead >= 0xE1 && lead <= 0xEF && lead != 0xED && size - i >= 3 && (s[i + 1] & 0xC0u) == 0x80 && (s[i + 2] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x0Fu) << 12u) | ((s[i + 1] & 0x3Fu) << 6u) | (s[i + 2] & 0x3Fu));
i += 3;
continue;
}
if (lead >= 0xC2 && lead <= 0xDF && size - i >= 2 && (s[i + 1] & 0xC0u) == 0x80)
{
u_escape(((lead & 0x1Fu) << 6u) | (s[i + 1] & 0x3Fu));
i += 2;
continue;
}
const std::size_t len = validate_one_utf8(s + i, size - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
m_out.set_cursor(w);
invalid_utf8(s, size);
return; // LCOV_EXCL_LINE (invalid_utf8 throws)
}
std::uint32_t codepoint = s[i] & (0xFFu >> (len + 1));
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
i += len;
if (codepoint <= 0xFFFF)
{
u_escape(codepoint);
}
else
{
u_escape(0xD7C0u + (codepoint >> 10u));
u_escape(0xDC00u + (codepoint & 0x3FFu));
}
continue;
}
// a quote, a backslash, or a control character (or DEL)
const unsigned char c = s[i++];
char escape = 0;
switch (c)
{
case 0x08:
escape = 'b';
break;
case 0x09:
escape = 't';
break;
case 0x0A:
escape = 'n';
break;
case 0x0C:
escape = 'f';
break;
case 0x0D:
escape = 'r';
break;
case 0x22:
escape = '"';
break;
case 0x5C:
escape = '\\';
break;
default:
break;
}
if (escape != 0)
{
w[0] = '\\';
w[1] = escape;
w += 2;
}
else
{
u_escape(c);
}
}
room(1);
*w++ = '"';
m_out.set_cursor(w);
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// the copies of dump_text() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
@@ -481,7 +688,7 @@ class view_serializer
}
/*!
@brief the compact output (the default dump(), or with ensure_ascii)
@brief the output, compact (the default dump()) or indented, with or without ensure_ascii
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
@@ -489,11 +696,11 @@ class view_serializer
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot. With ensure_ascii (Ascii), a
string of the source is copied only if it has no byte >= 0x7F (see
is_ascii_without_delete()); any other string is written by write_string().
string of the source is copied only if it has nothing to escape (see
copied()); any other string is written by write_ascii().
*/
template<bool SourceNumbers, bool Ascii>
void dump_compact(const node* root)
template<bool SourceNumbers, bool Pretty, bool Ascii>
void dump_text(const node* root)
{
struct frame
{
@@ -508,6 +715,16 @@ class view_serializer
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// (indented output) a line break and 63 indentation characters, from
// which the line break of a level of up to 63 characters is a copy
std::array<char, 64> ruler; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled below if used
if (Pretty)
{
ruler.fill(m_style.indent_char);
ruler[0] = '\n';
}
const std::size_t indent = m_style.indent;
depth_counter<Pretty> level; // depth of the innermost open container (indented output)
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
@@ -539,6 +756,22 @@ class view_serializer
}
w += n;
};
// (indented output) the line break and the indentation of a level
const auto newline = [&](std::size_t depth)
{
const std::size_t count = 1 + (depth * indent);
room(count);
if (count <= ruler.size())
{
std::memcpy(w, ruler.data(), ruler.size());
}
else
{
*w = '\n';
std::memset(w + 1, m_style.indent_char, count - 1);
}
w += count;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
@@ -546,11 +779,11 @@ class view_serializer
m_out.set_cursor(w);
if (enabled(Ascii))
{
write_ascii_string(n);
write_ascii(n);
}
else
{
write_decoded(n);
write_decoded(n, Pretty);
}
w = m_out.cursor();
lim = m_out.limit();
@@ -651,6 +884,7 @@ class view_serializer
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
level.enter();
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
@@ -693,6 +927,10 @@ class view_serializer
{
break;
}
if (Pretty)
{
newline(level.leave());
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
@@ -701,13 +939,17 @@ class view_serializer
room(1);
*w++ = ',';
}
if (Pretty)
{
newline(level.get());
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if (copied(key, src, std::integral_constant<bool, Ascii>()))
{
room(key.len + 3);
room(key.len + (Pretty ? 4 : 3));
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
@@ -717,9 +959,13 @@ class view_serializer
else
{
escaped(key);
room(1);
room(Pretty ? 2 : 1);
*w++ = ':';
}
if (Pretty)
{
*w++ = ' ';
}
if (Editable)
{
n = nav::value(at + 1);
@@ -738,69 +984,6 @@ class view_serializer
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
{
m_out.put('\n');
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
}
}
void write_scalar(const node& n)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
m_out.put("null");
break;
case value_t::boolean:
if ((n.flags & node_flags::is_true) != 0)
{
m_out.put("true");
}
else
{
m_out.put("false");
}
break;
case value_t::string:
write_string(n);
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const char* const token = m_doc.str(n);
const std::uint32_t len = number_length(n);
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
{
m_out.put('0'); // parse() reads -0 as the integer 0
}
else
{
m_out.put(token, len);
}
break;
}
case value_t::number_float:
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
{
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
}
else
{
write_float_node(n);
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
case value_t::array: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
}
/// a float node as dump() writes it
void write_float_node(const node& n)
{
@@ -890,89 +1073,27 @@ class view_serializer
m_out.put("null");
return;
}
write_float(x, std::integral_constant<bool, ::nlohmann::detail::has_native_float_format<number_float_t>::value>());
static constexpr bool is_ieee_single_or_double
= std::numeric_limits<number_float_t>::is_iec559
&& ((std::numeric_limits<number_float_t>::digits == 24 && std::numeric_limits<number_float_t>::max_exponent == 128)
|| (std::numeric_limits<number_float_t>::digits == 53 && std::numeric_limits<number_float_t>::max_exponent == 1024));
write_float(x, std::integral_constant<bool, is_ieee_single_or_double>());
}
void write_float(number_float_t x, std::true_type /*has_native_float_format*/)
void write_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
{
std::array<char, 64> buf{};
const char* const end = ::nlohmann::detail::to_chars(buf.data(), buf.data() + buf.size(), x);
m_out.put(buf.data(), static_cast<std::size_t>(end - buf.data()));
}
void write_float(number_float_t x, std::false_type /*has_native_float_format*/)
void write_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/)
{
// other types (e.g. long double) are rare: the library writes them
const string_t s = BasicJsonType(x).dump();
m_out.put(s.data(), s.size());
}
void write_string(const node& n)
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::storage) == 0 && (!m_style.ensure_ascii || is_ascii_without_delete(s, n.len)))
{
// a string of the source without escape sequences has nothing to
// escape (with ensure_ascii, if it is printable ASCII)
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
{
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
else
{
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
m_out.put('"');
}
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(s + i, n - i);
}
else if (is_ascii_copyable(s[i]))
{
run = find_ascii_copyable_run(s + i, n - i);
}
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0x80)
{
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
invalid_utf8(s, n);
return;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
}
write_codepoint<EnsureAscii>(codepoint, s + i, len);
i += len;
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
@@ -1027,8 +1148,8 @@ class view_serializer
void write_u_escape(std::uint32_t u)
{
std::array<char, 6> e = {{'\\', 'u', 0, 0, 0, 0}};
hex4_lower(static_cast<std::uint16_t>(u), &e[2]);
static constexpr const char* hex = "0123456789abcdef";
const std::array<char, 6> e = {{'\\', 'u', hex[(u >> 12u) & 0xFu], hex[(u >> 8u) & 0xFu], hex[(u >> 4u) & 0xFu], hex[u & 0xFu]}};
m_out.put(e.data(), e.size());
}
+3 -3
View File
@@ -60,7 +60,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
case value_t::binary:
case value_t::discarded:
default:
throw_type_error(exception_id::type_mismatch, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
throw_type_error(302, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
}
}
@@ -70,7 +70,7 @@ std::vector<U, A> vector_value(const View& v)
{
if (NLOHMANN_VIEW_UNLIKELY(!v.is_array()))
{
throw_type_error(exception_id::type_mismatch, "type must be array, but is ", v.type_name());
throw_type_error(302, "type must be array, but is ", v.type_name());
}
std::vector<U, A> r;
r.reserve(v.size());
@@ -88,7 +88,7 @@ Map map_value(const View& v)
{
if (NLOHMANN_VIEW_UNLIKELY(!v.is_object()))
{
throw_type_error(exception_id::type_mismatch, "type must be object, but is ", v.type_name());
throw_type_error(302, "type must be object, but is ", v.type_name());
}
Map r;
for (auto it = v.begin(); it != v.end(); ++it)
+326 -441
View File
File diff suppressed because it is too large. Load diff
+19 -19
View File
@@ -267,7 +267,7 @@ class basic_json_view
{
return {};
}
detail::view::throw_type_error(detail::exception_id::subscript_wrong_type, "cannot use operator[] with a string argument with ", type_name());
detail::view::throw_type_error(305, "cannot use operator[] with a string argument with ", type_name());
}
return lookup(key);
}
@@ -293,7 +293,7 @@ class basic_json_view
{
return {};
}
detail::view::throw_type_error(detail::exception_id::subscript_wrong_type, "cannot use operator[] with a numeric argument with ", type_name());
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
}
@@ -326,12 +326,12 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
{
detail::view::throw_type_error(detail::exception_id::at_wrong_type, "cannot use at() with ", type_name());
detail::view::throw_type_error(304, "cannot use at() with ", type_name());
}
const basic_json_view r = lookup(key);
if (NLOHMANN_VIEW_UNLIKELY(r.is_discarded()))
{
detail::view::throw_key_not_found(std::string(key.data(), key.size()));
detail::view::throw_out_of_range(403, detail::concat("key '", std::string(key.data(), key.size()), "' not found"));
}
return r;
}
@@ -352,11 +352,11 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_array()))
{
detail::view::throw_type_error(detail::exception_id::at_wrong_type, "cannot use at() with ", type_name());
detail::view::throw_type_error(304, "cannot use at() with ", type_name());
}
if (NLOHMANN_VIEW_UNLIKELY(idx >= m_node->len))
{
detail::view::throw_array_index_out_of_range(idx);
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
@@ -384,7 +384,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
{
detail::view::throw_type_error(detail::exception_id::value_wrong_type, "cannot use value() with ", type_name());
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
const basic_json_view r = lookup(key);
return r.is_discarded() ? default_value : r.template get<T>();
@@ -403,7 +403,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_structured()))
{
detail::view::throw_type_error(detail::exception_id::value_wrong_type, "cannot use value() with ", type_name());
detail::view::throw_type_error(306, "cannot use value() with ", type_name());
}
const basic_json_view r = detail::view::resolve_pointer(*this, detail::json_pointer_access::reference_tokens(ptr), detail::view::pointer_mode::value);
return r.is_discarded() ? default_value : r.template get<T>();
@@ -421,7 +421,7 @@ class basic_json_view
const iterator it = begin();
if (NLOHMANN_VIEW_UNLIKELY(it == end()))
{
detail::view::throw_invalid_iterator(detail::exception_id::iterator_value_unavailable, "cannot get value");
detail::view::throw_invalid_iterator(214, "cannot get value");
}
return *it;
}
@@ -573,7 +573,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_string()))
{
detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be string, but is ", type_name());
detail::view::throw_type_error(302, "type must be string, but is ", type_name());
}
return {m_doc->str(*m_node), m_node->len};
}
@@ -585,7 +585,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_number()))
{
detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be number, but is ", type_name());
detail::view::throw_type_error(302, "type must be number, but is ", type_name());
}
return {m_doc->str(*m_node), detail::view::number_length(*m_node)};
}
@@ -795,7 +795,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_boolean()))
{
detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be boolean, but is ", type_name());
detail::view::throw_type_error(302, "type must be boolean, but is ", type_name());
}
return (m_node->flags & detail::view::node_flags::is_true) != 0;
}
@@ -805,7 +805,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(m_node == nullptr))
{
detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be number, but is ", type_name());
detail::view::throw_type_error(302, "type must be number, but is ", type_name());
}
return detail::view::arithmetic_value<T, BasicJsonType>(*m_doc, *m_node);
}
@@ -814,7 +814,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_UNLIKELY(!is_null()))
{
detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be null, but is ", type_name());
detail::view::throw_type_error(302, "type must be null, but is ", type_name());
}
return nullptr;
}
@@ -1123,7 +1123,7 @@ class basic_json_document
{
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_type_error(detail::exception_id::document_image_unsupported, "cannot save a discarded json_document");
detail::view::throw_type_error(320, "cannot save a discarded json_document");
}
return detail::view::save_image(*m_data);
}
@@ -1224,7 +1224,7 @@ class basic_json_document
const std::size_t idx = pointer_index(token);
if (idx >= 0xFFFFFFFFu)
{
detail::view::throw_array_index_out_of_range(idx);
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
for (std::size_t i = 0; i < idx; ++i)
{
@@ -1280,7 +1280,7 @@ class basic_json_document
{
if (ptr.empty())
{
detail::view::throw_out_of_range(detail::exception_id::patch_on_root, "JSON pointer has no parent");
detail::view::throw_out_of_range(405, "JSON pointer has no parent");
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
@@ -1300,7 +1300,7 @@ class basic_json_document
static_assert(Editable, "only an editable document can be edited: use basic_json_document<BasicJsonType, true> (json_editable_document)");
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_invalid_iterator(detail::exception_id::iterator_from_other_value, "view does not belong to this document");
detail::view::throw_invalid_iterator(202, "view does not belong to this document");
}
return detail::view::editor<BasicJsonType, view_type>(*m_data);
}
@@ -1317,7 +1317,7 @@ class basic_json_document
{
if (idx < 0)
{
detail::view::throw_array_index_out_of_range(idx);
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return static_cast<std::size_t>(idx);
}
+14 -15
View File
@@ -125,19 +125,6 @@ private:
Container::pop_back();
}
/// @brief remove the entry with the given key; returns the number of removed entries
template<class K>
size_type erase_impl(const K& key)
{
const auto it = find_impl(*this, key);
if (it != this->end())
{
erase_at(it);
return 1;
}
return 0;
}
public:
template<class V, detail::enable_if_t<
detail::is_constructible<T, V>::value, int> = 0>
@@ -227,14 +214,26 @@ public:
size_type erase(const key_type& key)
{
return erase_impl(key);
const auto it = find_impl(*this, key);
if (it != this->end())
{
erase_at(it);
return 1;
}
return 0;
}
template<class KeyType, detail::enable_if_t<
detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
size_type erase(KeyType && key) // NOLINT(cppcoreguidelines-missing-std-forward)
{
return erase_impl(key);
const auto it = find_impl(*this, key);
if (it != this->end())
{
erase_at(it);
return 1;
}
return 0;
}
iterator erase(iterator pos)
+1827 -4358
View File
File diff suppressed because it is too large. Load diff
+506 -346
View File
File diff suppressed because it is too large. Load diff
-275
View File
@@ -492,278 +492,3 @@ TEST_CASE("binary_reserve_hint never over-reserves")
CHECK(nlohmann::detail::binary_reserve_hint(json({{"a", 1}, {"b", 2}})) == 5);
}
}
namespace
{
// numbers around every boundary of the integer encodings, and floats of each
// kind (exact in float, needing double, special values)
std::vector<json> boundary_numbers()
{
std::vector<json> numbers;
for (int k = 0; k < 64; ++k)
{
const std::uint64_t p = std::uint64_t{1} << k;
for (const std::uint64_t d :
{
p - 1, p, p + 1
})
{
numbers.emplace_back(d);
if (d <= static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()))
{
numbers.emplace_back(static_cast<std::int64_t>(d));
numbers.emplace_back(-static_cast<std::int64_t>(d));
numbers.emplace_back(-static_cast<std::int64_t>(d) - 1);
}
}
}
numbers.emplace_back((std::numeric_limits<std::int64_t>::min)());
numbers.emplace_back((std::numeric_limits<std::uint64_t>::max)());
for (const double f :
{
0.0, -0.0, 1.0, -1.0, 0.5, 0.1, 3.25, 1e300, -1e-300, 3.4028234663852886e38, 1e39,
std::numeric_limits<double>::infinity(), -std::numeric_limits<double>::infinity(),
std::numeric_limits<double>::quiet_NaN()
})
{
numbers.emplace_back(f);
}
return numbers;
}
// the concatenation of the encodings of the elements of @a numbers[first, first + count)
template<typename Single>
std::vector<std::uint8_t> concat_singles(const std::vector<json>& numbers, const std::size_t first, const std::size_t count, Single single)
{
std::vector<std::uint8_t> bytes;
for (std::size_t i = first; i < first + count; ++i)
{
const auto one = single(numbers[i]);
bytes.insert(bytes.end(), one.begin(), one.end());
}
return bytes;
}
} // namespace
// Runs of numbers in arrays are encoded straight into space reserved from the
// sink, by code that is separate from the encoding of a single value. Both have
// to agree on every number: an array is its head and the encodings of its
// elements, one after the other.
TEST_CASE("runs of numbers in arrays are encoded as the numbers one by one")
{
const std::vector<json> numbers = boundary_numbers();
// arrays of every length that a one-byte head covers, so that runs are as
// long as, and longer and shorter than, what fits one reservation
for (const std::size_t count :
{
1u, 2u, 3u, 5u, 14u, 15u, 23u
})
{
for (std::size_t first = 0; first + count <= numbers.size(); first += 7)
{
const json array(std::vector<json>(numbers.begin() + static_cast<std::ptrdiff_t>(first),
numbers.begin() + static_cast<std::ptrdiff_t>(first + count)));
CAPTURE(array.dump(-1, ' ', false, json::error_handler_t::replace))
std::vector<std::uint8_t> expected{static_cast<std::uint8_t>(0x80 + count)};
auto elements = concat_singles(numbers, first, count, [](const json & j)
{
return json::to_cbor(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
CHECK(json::to_cbor(array) == expected);
expected.assign(1, static_cast<std::uint8_t>(count <= 15 ? 0x90 + count : 0xDC));
if (count > 15)
{
expected.push_back(0);
expected.push_back(static_cast<std::uint8_t>(count));
}
elements = concat_singles(numbers, first, count, [](const json & j)
{
return json::to_msgpack(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
CHECK(json::to_msgpack(array) == expected);
for (const bool bjdata :
{
false, true
})
{
expected.assign(1, static_cast<std::uint8_t>('['));
elements = concat_singles(numbers, first, count, [bjdata](const json & j)
{
return bjdata ? json::to_bjdata(j) : json::to_ubjson(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
expected.push_back(static_cast<std::uint8_t>(']'));
CHECK((bjdata ? json::to_bjdata(array) : json::to_ubjson(array)) == expected);
}
// BON8 has no integers above the int64 range
bool representable = true;
for (std::size_t i = first; i < first + count; ++i)
{
representable = representable && bon8_representable(numbers[i]);
}
if (representable)
{
expected.assign(1, static_cast<std::uint8_t>(count <= 4 ? 0x80 + count : 0x85));
elements = concat_singles(numbers, first, count, [](const json & j)
{
return json::to_bon8(j);
});
expected.insert(expected.end(), elements.begin(), elements.end());
if (count > 4)
{
expected.push_back(0xFE);
}
CHECK(json::to_bon8(array) == expected);
}
else
{
CHECK_THROWS_AS(json::to_bon8(array), json::out_of_range&);
}
}
}
}
// An array that is written in more than one reservation (more numbers than fit
// one), and arrays of an index with more digits, round trip for every format.
TEST_CASE("large arrays and many digits of indices round trip")
{
json array = json::array();
for (int i = 0; i < 12000; ++i)
{
switch (i % 4)
{
case 0:
array.push_back(i);
break;
case 1:
array.push_back(-i);
break;
case 2:
array.push_back(i * 0.25);
break;
default:
array.push_back(i % 8 == 3);
break;
}
}
json document = json::object();
document["numbers"] = array;
document["nested"] = json::array({array, "text", array});
CHECK(json::from_cbor(json::to_cbor(document)) == document);
CHECK(json::from_msgpack(json::to_msgpack(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document, true, true)) == document);
CHECK(json::from_bjdata(json::to_bjdata(document)) == document);
CHECK(json::from_bson(json::to_bson(document)) == document);
CHECK(json::from_bon8(json::to_bon8(document)) == document);
}
// Strings and keys are written with their head in one reservation up to a
// length; around that length, and with every kind of head, they must read back.
TEST_CASE("strings of every head length round trip")
{
json document = json::object();
json array = json::array();
for (const std::size_t length :
{
0u, 1u, 2u, 3u, 4u, 7u, 8u, 15u, 16u, 17u, 23u, 24u, 25u, 31u, 32u, 33u, 63u, 127u, 128u, 129u, 200u, 245u, 246u, 247u, 248u, 249u, 250u, 251u, 252u, 253u, 254u, 255u, 256u, 257u, 300u, 1000u, 65535u, 65536u, 70000u
})
{
const std::string text(length, 'x');
array.push_back(text);
document[std::string(length, 'k')] = text;
}
document["array"] = array;
CHECK(json::from_cbor(json::to_cbor(document)) == document);
CHECK(json::from_msgpack(json::to_msgpack(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document)) == document);
CHECK(json::from_ubjson(json::to_ubjson(document, true, true)) == document);
CHECK(json::from_bjdata(json::to_bjdata(document)) == document);
CHECK(json::from_bson(json::to_bson(document)) == document);
CHECK(json::from_bon8(json::to_bon8(document)) == document);
}
#if !defined(JSON_NOEXCEPTION)
// What a write that throws leaves behind in an output adapter is what was written
// before it, whatever the adapter and however the adapter sink buffers. (Without
// exceptions, the writes in CHECK_THROWS_AS do not run.)
TEST_CASE("a write that throws leaves the same bytes in every kind of output")
{
json array = json::array();
for (int i = 0; i < 700; ++i)
{
array.push_back(i * 1000);
array.push_back("text " + std::to_string(i));
}
array.push_back("bad \xFF byte");
json document = json::object();
document["first"] = array;
document["second"] = 1;
const auto strict = json::error_handler_t::strict;
SECTION("cbor")
{
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_cbor(document, as_vector, strict), json::type_error&);
CHECK_THROWS_AS(json::to_cbor(document, as_string, strict), json::type_error&);
CHECK_THROWS_AS(json::to_cbor(document, as_stream, strict), json::type_error&);
CHECK(!as_vector.empty());
CHECK(std::string(reinterpret_cast<const char*>(as_vector.data()), as_vector.size()) == as_string); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
CHECK(as_stream.str() == as_string);
}
SECTION("msgpack")
{
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_msgpack(document, as_vector, strict), json::type_error&);
CHECK_THROWS_AS(json::to_msgpack(document, as_string, strict), json::type_error&);
CHECK_THROWS_AS(json::to_msgpack(document, as_stream, strict), json::type_error&);
CHECK(!as_vector.empty());
CHECK(std::string(reinterpret_cast<const char*>(as_vector.data()), as_vector.size()) == as_string); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
CHECK(as_stream.str() == as_string);
}
SECTION("ubjson")
{
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_ubjson(document, as_vector, false, false, strict), json::type_error&);
CHECK_THROWS_AS(json::to_ubjson(document, as_string, false, false, strict), json::type_error&);
CHECK_THROWS_AS(json::to_ubjson(document, as_stream, false, false, strict), json::type_error&);
CHECK(!as_vector.empty());
CHECK(std::string(reinterpret_cast<const char*>(as_vector.data()), as_vector.size()) == as_string); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
CHECK(as_stream.str() == as_string);
}
SECTION("bson")
{
// the size pass finds the string before anything is written
std::vector<std::uint8_t> as_vector;
std::string as_string;
std::ostringstream as_stream;
CHECK_THROWS_AS(json::to_bson(document, as_vector, strict), json::type_error&);
CHECK_THROWS_AS(json::to_bson(document, as_string, strict), json::type_error&);
CHECK_THROWS_AS(json::to_bson(document, as_stream, strict), json::type_error&);
CHECK(as_vector.empty());
CHECK(as_string.empty());
CHECK(as_stream.str().empty());
}
}
#endif
-115
View File
@@ -2042,118 +2042,3 @@ TEST_CASE("string scanning kernels")
CHECK(nlohmann::detail::read_eight_bytes(bytes.data() + 8) == 0xFF0F0E0D0C0B0A09u);
CHECK(nlohmann::detail::read_eight_bytes(reinterpret_cast<const char*>(bytes.data()) + 3) == 0x0B0A090807060504u);
}
TEST_CASE("find_ascii_copyable_run agrees with is_ascii_copyable for every byte")
{
// the SWAR mask in find_ascii_copyable_run() encodes the predicate a
// second time; check every byte value at every position of a 24-byte buffer
// (three words) and for several lengths, with the byte at the end or inside
for (unsigned int value = 0; value < 256; ++value)
{
for (std::size_t position = 0; position < 24; ++position)
{
std::array<unsigned char, 24> buffer{};
buffer.fill(static_cast<unsigned char>('a'));
buffer[position] = static_cast<unsigned char>(value);
const bool copyable = nlohmann::detail::is_ascii_copyable(static_cast<unsigned char>(value));
for (const std::size_t n :
{
position, position + 1, std::size_t{24}
})
{
if (n > 24)
{
continue;
}
const std::size_t expected = (copyable || n <= position) ? n : position;
CAPTURE(value)
CAPTURE(position)
CAPTURE(n)
CHECK(nlohmann::detail::find_ascii_copyable_run(buffer.data(), n) == expected);
}
}
}
}
TEST_CASE("UTF-8 validators accept the same sequences")
{
// The library checks UTF-8 in several places: detail::decode (serializer),
// the per-lead-byte switch of the lexer (json::accept), validate_one_utf8
// (bulk string scan), utf8_shape, and the byte path of the BON8 reader.
// They must accept exactly the same sequences (RFC 3629, section 4).
using nlohmann::json;
// bytes for the third and fourth position: the first two pairs are checked
// against all validators, the others only against the cheap ones
const std::array<std::array<unsigned char, 2>, 6> tails =
{
{
{{0x80, 0x80}}, {{0xBF, 0xBF}}, {{0x7F, 0x80}}, {{0xC0, 0x80}}, {{0x80, 0x7F}}, {{0x80, 0xC0}}
}
};
for (unsigned lead = 0x80; lead <= 0xFF; ++lead)
{
const auto shape = nlohmann::detail::utf8_shape(static_cast<std::uint8_t>(lead));
for (unsigned second = 0; second <= 0xFF; ++second)
{
for (std::size_t t = 0; t < tails.size(); ++t)
{
const std::array<unsigned char, 4> bytes = {{static_cast<unsigned char>(lead), static_cast<unsigned char>(second), tails[t][0], tails[t][1]}};
// expectation from the shape: the sequence is bytes[0..length)
const std::size_t length = shape.length;
bool expected = length >= 2 && second >= shape.second_lo && second <= shape.second_hi;
for (std::size_t i = 2; expected && i < length; ++i)
{
expected = bytes[i] >= 0x80 && bytes[i] <= 0xBF;
}
// the DFA of the serializer
bool dfa = length >= 2;
if (dfa)
{
std::uint8_t state = nlohmann::detail::UTF8_ACCEPT;
std::uint32_t codep = 0;
for (std::size_t i = 0; i < length; ++i)
{
nlohmann::detail::decode(state, codep, bytes[i]);
}
dfa = state == nlohmann::detail::UTF8_ACCEPT;
}
// the bulk path: exactly the sequence, and one byte short
const std::size_t bulk = nlohmann::detail::validate_one_utf8(bytes.data(), length);
const std::size_t bulk_short = length > 0 ? nlohmann::detail::validate_one_utf8(bytes.data(), length - 1) : 0;
CAPTURE(lead)
CAPTURE(second)
CAPTURE(t)
CHECK((dfa == expected && bulk == (expected ? length : 0) && bulk_short == 0));
if (t < 2 && length >= 2)
{
// the lexer's byte path
const std::string sequence(reinterpret_cast<const char*>(bytes.data()), length); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
CHECK(json::accept("\"" + sequence + "\"") == expected);
// the byte path of the BON8 reader (a string ends at 0xFF)
if (lead >= 0xC2 && lead <= 0xF7)
{
std::vector<std::uint8_t> packed(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length));
packed.push_back(0xFF);
const json j = json::from_bon8(packed, true, false);
if (expected)
{
CHECK(j == json(sequence));
}
else
{
CHECK_FALSE((j.is_string() && j.get<std::string>() == sequence));
}
}
}
}
}
}
}
-24
View File
@@ -361,30 +361,6 @@ TEST_CASE("constructors")
CHECK_THROWS_WITH_AS((j.get<std::pair<int, int>>()), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS((j.get<std::tuple<int, int>>()), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS((j.get<std::array<int, 3>>()), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS((j.get<std::array<std::string, 3>>()), "[json.exception.type_error.302] type must be string, but is number", json::type_error&);
// the elements are converted in order, so a type error comes before a missing element
json const j2{1, "two"};
CHECK_THROWS_WITH_AS((j2.get<std::array<int, 3>>()), "[json.exception.type_error.302] type must be number, but is string", json::type_error&);
std::array<int, 3> a{{0, 0, 0}};
CHECK_THROWS_WITH_AS(j2.get_to(a), "[json.exception.type_error.302] type must be number, but is string", json::type_error&);
CHECK_THROWS_WITH_AS(j.get_to(a), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
#if !defined(JSON_NOEXCEPTION) // without exceptions, the get_to calls in CHECK_THROWS_WITH_AS do not run
CHECK(a[0] == 1);
#endif
json const j3{"one", "two"};
CHECK_THROWS_WITH_AS((j3.get<std::array<std::string, 3>>()), "[json.exception.out_of_range.401] array index 2 is out of range", json::out_of_range&);
}
SECTION("std::array from a longer array or from booleans")
{
json const j{1, 2.5, true, 4u};
CHECK(j.get<std::array<int, 3>>() == (std::array<int, 3> {{1, 2, 1}}));
std::array<int, 4> a{{0, 0, 0, 0}};
j.get_to(a);
CHECK(a == (std::array<int, 4> {{1, 2, 1, 4}}));
CHECK(j.get<std::array<int, 0>>().empty());
}
SECTION("std::forward_list<json>")
-52
View File
@@ -285,58 +285,6 @@ TEST_CASE("custom number types: integers beyond 64 bits")
CHECK(wide_json::to_bjdata(wide_json::array({uint_big, uint_big}), true, true) == expected_bjdata_array);
}
SECTION("integers in arrays and objects")
{
// runs of numbers and scalar object members are encoded by loops of
// their own for the number types up to 64 bits; a wider number type
// takes the generic path, which checks the range of every integer
const wide_json run = wide_json::array({make_int(1), make_int(-3), make_uint(7), wide_json(1.5), int_big, make_int(5)});
const wide_json run_negative = wide_json::array({make_int(1), make_int(-3), int_big_negative});
const wide_json run_unsigned = wide_json::array({make_uint(1), make_uint(7), uint_big});
const wide_json members = wide_json::object({{"a", make_int(1)}, {"b", wide_json(1.5)}, {"c", int_big}});
const std::vector<wide_json> failing = {run, run_negative, run_unsigned, members, wrap(run), wrap(members)};
for (const auto& value : failing)
{
CHECK_THROWS_AS(_ = wide_json::to_cbor(value), out_of_range&);
CHECK_THROWS_AS(_ = wide_json::to_msgpack(value), out_of_range&);
CHECK_THROWS_AS(_ = wide_json::to_bson(wrap(value)), out_of_range&);
CHECK_THROWS_AS(_ = wide_json::to_bon8(value), out_of_range&);
}
#if !defined(JSON_NOEXCEPTION)
// a failing write leaves no more than the elements before the failing one
// (without exceptions, the writes in CHECK_THROWS_AS do not run)
std::vector<std::uint8_t> out;
CHECK_THROWS_AS(wide_json::to_cbor(run, out), out_of_range&);
// (the head of the array, 1, -3, 7, and 1.5 as a single-precision float)
CHECK(out == std::vector<std::uint8_t>({0x86, 0x01, 0x22, 0x07, 0xFA, 0x3F, 0xC0, 0x00, 0x00}));
out.clear();
CHECK_THROWS_AS(wide_json::to_bon8(run, out), out_of_range&);
CHECK(!out.empty());
#endif
// the same values that fit are written exactly as with 64-bit number types
const wide_json fits = wide_json::array({make_int(1), make_int(-3), make_uint(7), wide_json(1.5), make_int(min_int64), make_int(max_int64), make_int(200), make_int(-70000)});
const nlohmann::json fits64 = nlohmann::json::array({1, -3, 7, 1.5, (std::numeric_limits<std::int64_t>::min)(), (std::numeric_limits<std::int64_t>::max)(), 200, -70000});
CHECK(wide_json::to_cbor(fits) == nlohmann::json::to_cbor(fits64));
CHECK(wide_json::to_msgpack(fits) == nlohmann::json::to_msgpack(fits64));
CHECK(wide_json::to_ubjson(fits) == nlohmann::json::to_ubjson(fits64));
CHECK(wide_json::to_ubjson(fits, true, true) == nlohmann::json::to_ubjson(fits64, true, true));
CHECK(wide_json::to_bjdata(fits) == nlohmann::json::to_bjdata(fits64));
CHECK(wide_json::to_bjdata(fits, true, true) == nlohmann::json::to_bjdata(fits64, true, true));
CHECK(wide_json::to_bon8(fits) == nlohmann::json::to_bon8(fits64));
CHECK(wide_json::to_bson(wrap(fits)) == nlohmann::json::to_bson(nlohmann::json::object({{"v", fits64}})));
const wide_json fit_members = wide_json::object({{"a", make_int(1)}, {"b", make_int(-300)}, {"c", make_uint(max_uint64)}, {"d", wide_json(2.5)}});
const nlohmann::json fit_members64 = nlohmann::json::object({{"a", 1}, {"b", -300}, {"c", (std::numeric_limits<std::uint64_t>::max)()}, {"d", 2.5}});
CHECK(wide_json::to_cbor(fit_members) == nlohmann::json::to_cbor(fit_members64));
CHECK(wide_json::to_msgpack(fit_members) == nlohmann::json::to_msgpack(fit_members64));
CHECK(wide_json::to_ubjson(fit_members) == nlohmann::json::to_ubjson(fit_members64));
CHECK(wide_json::to_bjdata(fit_members) == nlohmann::json::to_bjdata(fit_members64));
CHECK(wide_json::to_bson(fit_members) == nlohmann::json::to_bson(fit_members64));
}
SECTION("BJData ND-array")
{
// an ND-array element beyond 64 bits does not fit any dtype, so the
+127
View File
@@ -186,6 +186,133 @@ TEST_CASE("json_view dump")
CHECK(deep_document.root().dump() == deep);
}
SECTION("ensure_ascii: a character that needs an escape at each position of a string")
{
// string lengths around the width of the vector steps, in a string value, a key, and a value near the end of the text
for (const char* const special :
{
"\xC3\xA9", "\xE4\xB8\xAD", "\xF0\x9F\x98\x80", "\x7F", "\\n", "\\u0001", "\\\"", "\\\\"
})
{
for (std::size_t length = 0; length <= 40; ++length)
{
for (std::size_t position = 0; position <= length; ++position)
{
std::string content(length, 'a');
content.insert(position, special);
const std::array<std::string, 3> texts =
{
{
"\"" + content + "\"",
"[\"" + content + "\"]",
"{\"" + content + "\":1}",
}
};
for (const std::string& text : texts)
{
CAPTURE(text)
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
}
}
}
}
}
SECTION("ensure_ascii: every character of the Basic Multilingual Plane and some beyond")
{
// (the two- and three-byte sequences, including the ranges that need a check: E0, ED, EF BF BF)
std::string text = "[";
std::size_t in_chunk = 0;
const auto append = [&](std::uint32_t cp)
{
if (cp < 0x80)
{
if (cp < 0x20 || cp == '"' || cp == '\\')
{
return;
}
text += static_cast<char>(cp);
}
else if (cp < 0x800)
{
text += static_cast<char>(0xC0 | (cp >> 6));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else if (cp < 0x10000)
{
text += static_cast<char>(0xE0 | (cp >> 12));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else
{
text += static_cast<char>(0xF0 | (cp >> 18));
text += static_cast<char>(0x80 | ((cp >> 12) & 0x3F));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
};
for (std::uint32_t cp = 0x20; cp < 0x10400; ++cp)
{
if (cp >= 0xD800 && cp < 0xE000)
{
continue;
}
if (in_chunk == 0)
{
text += text.size() > 1 ? ",\"" : "\"";
}
append(cp);
if (++in_chunk == 37)
{
text += "\"";
in_chunk = 0;
}
}
text += in_chunk != 0 ? "\"]" : "]";
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root().dump(2) == j.dump(2));
}
SECTION("deep nesting, indented")
{
// indentations of up to and beyond the width of the writer's fixed-size copy
for (const int depth :
{
1, 5, 20, 70, 300
})
{
std::string text;
for (int i = 0; i < depth; ++i)
{
text += i % 2 == 0 ? "[1," : R"({"k":2,"\u00e9":)";
}
text += "[]";
for (int i = depth - 1; i >= 0; --i)
{
text += i % 2 == 0 ? "]" : "}";
}
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
for (const int indent :
{
0, 1, 2, 3, 8
})
{
CHECK(d.root().dump(indent) == j.dump(indent));
CHECK(d.root().dump(indent, '\t') == j.dump(indent, '\t'));
}
}
}
SECTION("the output buffer of a small value is small")
{
// an escaped key after the value: its node lies in the arena, so the
+20
View File
@@ -682,6 +682,26 @@ TEST_CASE("json_view images: ownership")
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view images: errors")
{
SECTION("invalid UTF-8 of a decoded string: the compact dump writes it, the others throw")
{
const json_document d = json_document::parse(R"({"k\n":"\u00e9x"})");
std::vector<std::uint8_t> image = d.save();
const std::size_t arena = text_at(image) + header_size_field(image, 16) + 1;
image[arena + 3] = 0x28; // the key is "k\n"; the value starts at 3: 0xC3 0x28 is invalid
const json_document loaded = json_document::load(image, image_check::bounds);
CHECK(loaded.root().dump().size() == 13);
for (const int indent :
{
0, 2
})
{
CHECK_THROWS_AS(loaded.root().dump(indent), json::type_error);
CHECK_THROWS_AS(loaded.root().dump(indent, ' ', true), json::type_error);
}
CHECK_THROWS_AS(loaded.root().dump(-1, ' ', true), json::type_error);
CHECK_THROWS_WITH_AS(loaded.root().dump(2), "[json.exception.type_error.316] invalid UTF-8 byte at index 1: 0x28", json::type_error&);
}
SECTION("a literal as the root: dump() after loading")
{
for (const char* text :
-91
View File
@@ -1580,94 +1580,3 @@ TEST_CASE("invalid UTF-8 at every position is handled like the reference")
}
}
}
TEST_CASE("short strings are written like the reference, as values and as keys")
{
// strings of up to 16 characters (32 with vector instructions) that need no
// escaping are written as a whole (together with the quotes and, for a key,
// the colon); everything else, including a string that is merely short,
// takes the general path.
// Every length around the edges of the size classes of the fast path is
// combined with each kind of byte that must send a string to the general
// path, at each position, and compared with the reference for values and,
// via the value, for keys.
const std::vector<std::string> specials =
{
"\"", "\\", "\n", "\x1f", "\x7f", "\xc3\xa9", "\xe2\x82\xac", "\xf0\x9f\x98\x80", "\xff", "\x80", "\xc3"
};
for (std::size_t length = 0; length <= 36; ++length)
{
const std::string plain(length, 'k');
check_dump_against_reference(plain, false);
for (const auto& special : specials)
{
for (std::size_t pos = 0; pos <= length; ++pos)
{
std::string s = plain;
s.insert(pos, special);
check_dump_against_reference(s, false);
for (const bool ensure_ascii :
{
false, true
})
{
// a key and a value are written alike, in compact and
// in pretty-printed form
const json value = s;
const json object = json::object({{s, 1}});
for (const auto error_handler :
{
json::error_handler_t::replace, json::error_handler_t::keep
})
{
const std::string quoted = value.dump(-1, ' ', ensure_ascii, error_handler);
CHECK(object.dump(-1, ' ', ensure_ascii, error_handler) == "{" + quoted + ":1}");
CHECK(object.dump(2, ' ', ensure_ascii, error_handler) == "{\n " + quoted + ": 1\n}");
}
}
}
}
}
}
TEST_CASE("runs of multi-byte characters are escaped like the reference under ensure_ascii")
{
// a run of well-formed multi-byte characters is escaped straight into the
// write buffer; it has to flush that mid-run, stop at an ASCII character or
// an ill-formed sequence and leave those to the general path, and pick up
// behind what the general path has pending
const std::vector<std::string> characters =
{
"\xc2\x80", "\xdf\xbf", "\xe0\xa0\x80", "\xe2\x82\xac", "\xed\x9f\xbf", "\xef\xbf\xbf",
"\xf0\x90\x80\x80", "\xf0\x9f\x98\x80", "\xf4\x8f\xbf\xbf"
};
const std::vector<std::string> interruptions =
{
"", "a", "\n", "\"", "\x7f", "\xff", "\x80", "\xc3", "\xe2\x82", "\xf0\x9f\x98", "\xed\xa0\x80"
};
for (std::size_t shift = 0; shift <= 13; ++shift)
{
for (const auto& interruption : interruptions)
{
for (const std::size_t count :
{
std::size_t{1}, std::size_t{2}, std::size_t{85}, std::size_t{200}
})
{
std::string s(shift, 'x');
for (std::size_t i = 0; i < count; ++i)
{
s += characters[(i + shift) % characters.size()];
}
s += interruption;
s += characters[shift % characters.size()];
s += characters[(shift + 3) % characters.size()];
check_dump_against_reference(s, shift < 2);
}
}
}
}
+1 -2
View File
@@ -78,8 +78,7 @@ TEST_CASE("detail::bit_cast")
// to and from byte arrays
const auto bytes = bit_cast<std::array<std::uint8_t, 4>>(std::uint32_t{0x01020304u});
// qualified: with a std::array argument, ADL would also find std::bit_cast in C++20
CHECK(nlohmann::detail::bit_cast<std::uint32_t>(bytes) == 0x01020304u);
CHECK(bit_cast<std::uint32_t>(bytes) == 0x01020304u);
static_assert(noexcept(bit_cast<std::uint64_t>(1.0)), "bit_cast must be noexcept");
}
+6 -34
View File
@@ -48,16 +48,14 @@ namespace
class chunked_streambuf : public std::streambuf
{
public:
// The first @a unbuffered characters are served without a get area, the
// rest in get areas of @a chunk_size.
chunked_streambuf(std::string s, std::size_t chunk_size, std::size_t unbuffered = 0)
: m_data(std::move(s)), m_chunk_size(chunk_size), m_unbuffered(unbuffered)
chunked_streambuf(std::string s, std::size_t chunk_size)
: m_data(std::move(s)), m_chunk_size(chunk_size)
{}
protected:
int_type underflow() override
{
if (m_chunk_size == 0 || m_pos < m_unbuffered)
if (m_chunk_size == 0)
{
return m_pos < m_data.size() ? traits_type::to_int_type(m_data[m_pos]) : traits_type::eof();
}
@@ -79,7 +77,7 @@ class chunked_streambuf : public std::streambuf
int_type uflow() override
{
if (m_chunk_size == 0 || m_pos < m_unbuffered)
if (m_chunk_size == 0)
{
return m_pos < m_data.size() ? traits_type::to_int_type(m_data[m_pos++]) : traits_type::eof();
}
@@ -90,7 +88,6 @@ class chunked_streambuf : public std::streambuf
std::string m_data;
std::size_t m_pos = 0;
std::size_t m_chunk_size;
std::size_t m_unbuffered;
};
// how a test reads from the stream
@@ -104,9 +101,9 @@ enum class read_mode
// Read @a input from a chunked_streambuf and describe everything observable:
// the values or the exception, the stream state after each value, and the
// characters left in the stream.
std::string observe(const std::string& input, std::size_t chunk_size, read_mode mode, std::size_t unbuffered = 0)
std::string observe(const std::string& input, std::size_t chunk_size, read_mode mode)
{
chunked_streambuf buf(input, chunk_size, unbuffered);
chunked_streambuf buf(input, chunk_size);
std::istream is(&buf);
std::string result;
for (int i = 0; i < 4; ++i)
@@ -215,31 +212,6 @@ TEST_CASE("parsing from a std::istream scans the get area in bulk")
}
}
SECTION("results do not depend on a get area that appears late")
{
// the first characters come without a get area, then the streambuf
// starts to use one: the token that is read at that point is complete
for (const auto& input : corpus())
{
const std::string expected = observe(input, 0, read_mode::extract);
for (const std::size_t unbuffered :
{
1u, 2u, 3u, 5u, 8u, 13u, 21u
})
{
for (const std::size_t chunk_size :
{
1u, 2u, 3u, 7u, 64u
})
{
CAPTURE(input)
CAPTURE(unbuffered)
CAPTURE(chunk_size)
CHECK(observe(input, chunk_size, read_mode::extract, unbuffered) == expected);
}
}
}
}
#endif // !defined(JSON_NOEXCEPTION)
SECTION("values are the same as when parsing from a string")
-3
View File
@@ -139,12 +139,9 @@ if __name__ == '__main__':
namespaces += [f'{namespaces[0]}::{ns}' for ns in inline_namespaces]
# The template renders a local XML file, not a web response, and autoescape is enabled.
# nosemgrep: python.flask.security.xss.audit.direct-use-of-jinja2.direct-use-of-jinja2
env = jinja2.Environment(loader=jinja2.FileSystemLoader(searchpath=sys.path[0]), autoescape=True, trim_blocks=True,
lstrip_blocks=True, keep_trailing_newline=True)
template = env.get_template('nlohmann_json.natvis.j2')
# nosemgrep: python.flask.security.xss.audit.direct-use-of-jinja2.direct-use-of-jinja2
natvis = template.render(namespaces=namespaces)
output = args.output or os.path.join(args.repo_root, 'nlohmann_json.natvis')