mirror of
https://github.com/nlohmann/json.git
synced 2026-10-21 21:48:14 +00:00
* Add images of json_documents: save() and load()
An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().
load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.
image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.
Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.
New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.
A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC errors in image test and add image_check doc anchor
Move the raw string literal out of the CHECK macro (MSVC preprocessor),
cast 64-bit header fields to std::size_t (C4244 on 32-bit), and give the
image_check section in load.md a real anchor for the mkdocs strict build.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check each distinct string and float range once in the full image check
The full check scanned the contents of every string node and every float token over its own range, so many nodes pointing to one large range made load() quadratic. The structural walk now only collects the ranges; after it, each kind is sorted and checked once per distinct range. Ranges of one kind that overlap without being identical are rejected, as save() never writes them (nodes that share a value share the whole range). The cost is linear in the size of the image plus sorting the ranges.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Adapt the image tests to named documents and last-wins lookups
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test that the targets of links never reach an image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test lookups in loaded images with duplicate and colliding keys
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Release the list of large objects once load() has built the indexes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test that invalid UTF-8 of a loaded image is not copied into an editable document
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document that editable documents refuse invalid UTF-8 copied from a loaded image
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Expect first-wins lookups in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid useless casts of header fields in the image tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy 22 findings in unit-json_view_image.cpp and image.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C4244 and C2017 in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the indentation of a loop in the image test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.
- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
json_view.hpp only includes json.hpp, so including them here would duplicate
their definitions
- drop edit_storage.hpp, which edit.hpp includes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence a clang-tidy C++17 finding in the image test
bugprone-suspicious-stringview-data-usage flags data() of the source view
under C++17, although the length is passed and checked by the REQUIRE
before it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the image link test without exceptions
load_result() exists only with exceptions; with JSON_NOEXCEPTION the test
loads the image with the check directly (a failed check aborts).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Follow the renames from the review of the view builder
The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Rename the emit lambda in the image writer for Qt compatibility
Qt defines emit as an empty macro, so a Qt header included before
json_view.hpp broke the lambda's declaration (see #5865). The slots
variables in the image tests are renamed as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address the review of json_document images
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
442 lines
14 KiB
C++
442 lines
14 KiB
C++
// __ _____ _____ _____
|
|
// __| | __| | | | JSON for Modern C++
|
|
// | | |__ | | | | | | version 3.12.0
|
|
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
|
//
|
|
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
#pragma once
|
|
|
|
#include <algorithm> // max
|
|
#include <array> // array
|
|
#include <cmath> // isfinite
|
|
#include <cstddef> // size_t
|
|
#include <cstdint> // uint8_t, uint32_t
|
|
#include <cstring> // memcpy, memset
|
|
#include <limits> // numeric_limits
|
|
#include <type_traits> // integral_constant
|
|
#include <vector> // vector
|
|
|
|
#include <nlohmann/json.hpp>
|
|
#include <nlohmann/detail/view/document_data.hpp>
|
|
#include <nlohmann/detail/view/macro_scope.hpp>
|
|
#include <nlohmann/detail/view/node.hpp>
|
|
#include <nlohmann/detail/view/number.hpp>
|
|
|
|
NLOHMANN_JSON_NAMESPACE_BEGIN
|
|
namespace detail
|
|
{
|
|
namespace view
|
|
{
|
|
|
|
/// append-only output buffer: writes through a raw pointer into a string that
|
|
/// is resized ahead, and trimmed by finish()
|
|
template<typename StringType>
|
|
class output_buffer
|
|
{
|
|
public:
|
|
output_buffer(StringType& out, std::size_t estimate)
|
|
: m_out(sized(out, estimate))
|
|
, m_pos(&m_out[0])
|
|
, m_end(m_pos + m_out.size())
|
|
{}
|
|
|
|
void finish()
|
|
{
|
|
const auto size = static_cast<std::size_t>(m_pos - m_out.data());
|
|
m_out.resize(size);
|
|
}
|
|
|
|
NLOHMANN_VIEW_ALWAYS_INLINE void reserve(std::size_t n)
|
|
{
|
|
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_end - m_pos) < n))
|
|
{
|
|
grow(n);
|
|
}
|
|
}
|
|
|
|
NLOHMANN_VIEW_ALWAYS_INLINE void put(char c)
|
|
{
|
|
reserve(1);
|
|
*m_pos++ = c;
|
|
}
|
|
|
|
NLOHMANN_VIEW_ALWAYS_INLINE void put(const char* s, std::size_t n)
|
|
{
|
|
reserve(n);
|
|
std::memcpy(m_pos, s, n);
|
|
m_pos += n;
|
|
}
|
|
|
|
/// a string literal, without its terminating NUL
|
|
template<std::size_t N>
|
|
NLOHMANN_VIEW_ALWAYS_INLINE void put(const char (&s)[N]) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
|
|
{
|
|
put(s, N - 1);
|
|
}
|
|
|
|
void put_repeated(char c, std::size_t n)
|
|
{
|
|
reserve(n);
|
|
std::memset(m_pos, c, n);
|
|
m_pos += n;
|
|
}
|
|
|
|
private:
|
|
static StringType& sized(StringType& out, std::size_t estimate)
|
|
{
|
|
out.resize((std::max)(estimate, static_cast<std::size_t>(64)));
|
|
return out;
|
|
}
|
|
|
|
NLOHMANN_VIEW_NOINLINE void grow(std::size_t n)
|
|
{
|
|
const auto used = static_cast<std::size_t>(m_pos - m_out.data());
|
|
m_out.resize((std::max)(m_out.size() * 2, used + n + 256));
|
|
m_pos = &m_out[0] + used;
|
|
m_end = &m_out[0] + m_out.size();
|
|
}
|
|
|
|
StringType& m_out;
|
|
char* m_pos;
|
|
char* m_end;
|
|
};
|
|
|
|
/// how the view's dump() writes a value
|
|
struct dump_style
|
|
{
|
|
bool pretty = false; ///< indent >= 0
|
|
std::size_t indent = 0; ///< characters per level
|
|
char indent_char = ' ';
|
|
bool ensure_ascii = false;
|
|
bool source_numbers = false; ///< copy number tokens from the source
|
|
};
|
|
|
|
/*!
|
|
@brief write a view's subtree as basic_json::dump() writes the value
|
|
|
|
The output of a subtree equals ordered_json::parse(text).dump() of it for
|
|
the same arguments (members in document order): strings are escaped by the
|
|
same rules, with the library's scanning kernels; floats are written with
|
|
the library's conversion; integers are copied from the source, where they
|
|
are canonical (except "-0", which parse() reads as 0). The walk is
|
|
iterative, so the nesting depth is limited by memory only.
|
|
*/
|
|
template<typename BasicJsonType, bool Editable>
|
|
class view_serializer
|
|
{
|
|
using nav = navigation<Editable>;
|
|
using string_t = typename BasicJsonType::string_t;
|
|
using number_float_t = typename BasicJsonType::number_float_t;
|
|
|
|
public:
|
|
view_serializer(const document_data& d, string_t& out, std::size_t estimate, const dump_style& style)
|
|
: m_doc(d), m_out(out, estimate), m_style(style)
|
|
{}
|
|
|
|
void dump(const node* root)
|
|
{
|
|
struct frame
|
|
{
|
|
const node* pos; ///< next element, or key of the next member
|
|
const node* end;
|
|
bool object;
|
|
bool first; ///< nothing written yet
|
|
};
|
|
std::vector<frame> stack;
|
|
const node* n = root;
|
|
for (;;)
|
|
{
|
|
// write the value at n
|
|
if (is_container(*n))
|
|
{
|
|
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
|
|
if (n->len == 0)
|
|
{
|
|
m_out.put(object ? "{}" : "[]", 2);
|
|
}
|
|
else
|
|
{
|
|
m_out.put(object ? '{' : '[');
|
|
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
|
|
}
|
|
}
|
|
else
|
|
{
|
|
write_scalar(*n);
|
|
}
|
|
|
|
// go to the next value: close finished containers, then separate
|
|
for (;;)
|
|
{
|
|
if (stack.empty())
|
|
{
|
|
m_out.finish();
|
|
return;
|
|
}
|
|
if (stack.back().pos == stack.back().end)
|
|
{
|
|
const bool object = stack.back().object;
|
|
stack.pop_back();
|
|
newline(stack.size());
|
|
m_out.put(object ? '}' : ']');
|
|
continue;
|
|
}
|
|
frame& f = stack.back();
|
|
if (!f.first)
|
|
{
|
|
m_out.put(',');
|
|
}
|
|
f.first = false;
|
|
newline(stack.size());
|
|
if (f.object)
|
|
{
|
|
write_string(*f.pos);
|
|
if (m_style.pretty)
|
|
{
|
|
m_out.put(": ");
|
|
}
|
|
else
|
|
{
|
|
m_out.put(':');
|
|
}
|
|
n = nav::value(f.pos + 1);
|
|
f.pos = document_data::after(f.pos + 1);
|
|
}
|
|
else
|
|
{
|
|
n = nav::value(f.pos);
|
|
f.pos = document_data::after(f.pos);
|
|
}
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
|
|
private:
|
|
void newline(std::size_t level)
|
|
{
|
|
if (m_style.pretty)
|
|
{
|
|
m_out.put('\n');
|
|
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
|
|
}
|
|
}
|
|
|
|
void write_scalar(const node& n)
|
|
{
|
|
switch (static_cast<value_t>(n.kind))
|
|
{
|
|
case value_t::null:
|
|
m_out.put("null");
|
|
break;
|
|
case value_t::boolean:
|
|
if ((n.flags & node_flags::is_true) != 0)
|
|
{
|
|
m_out.put("true");
|
|
}
|
|
else
|
|
{
|
|
m_out.put("false");
|
|
}
|
|
break;
|
|
case value_t::string:
|
|
write_string(n);
|
|
break;
|
|
case value_t::number_integer:
|
|
case value_t::number_unsigned:
|
|
{
|
|
const char* const token = m_doc.str(n);
|
|
const std::uint32_t len = number_length(n);
|
|
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
|
|
{
|
|
m_out.put('0'); // parse() reads -0 as the integer 0
|
|
}
|
|
else
|
|
{
|
|
m_out.put(token, len);
|
|
}
|
|
break;
|
|
}
|
|
case value_t::number_float:
|
|
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
|
|
{
|
|
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
|
|
}
|
|
else
|
|
{
|
|
write_float(float_value<number_float_t>(m_doc, n));
|
|
}
|
|
break;
|
|
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
|
|
case value_t::array: // LCOV_EXCL_LINE
|
|
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
|
|
case value_t::discarded: // LCOV_EXCL_LINE
|
|
default: // LCOV_EXCL_LINE
|
|
break; // LCOV_EXCL_LINE
|
|
}
|
|
}
|
|
|
|
/// as serializer::dump_float()
|
|
void write_float(number_float_t x)
|
|
{
|
|
if (!std::isfinite(x))
|
|
{
|
|
m_out.put("null");
|
|
return;
|
|
}
|
|
static constexpr bool is_ieee_single_or_double
|
|
= std::numeric_limits<number_float_t>::is_iec559
|
|
&& ((std::numeric_limits<number_float_t>::digits == 24 && std::numeric_limits<number_float_t>::max_exponent == 128)
|
|
|| (std::numeric_limits<number_float_t>::digits == 53 && std::numeric_limits<number_float_t>::max_exponent == 1024));
|
|
write_float(x, std::integral_constant<bool, is_ieee_single_or_double>());
|
|
}
|
|
|
|
void write_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
|
|
{
|
|
std::array<char, 64> buf{};
|
|
const char* const end = ::nlohmann::detail::to_chars(buf.data(), buf.data() + buf.size(), x);
|
|
m_out.put(buf.data(), static_cast<std::size_t>(end - buf.data()));
|
|
}
|
|
|
|
void write_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/)
|
|
{
|
|
// other types (e.g. long double) are rare: the library writes them
|
|
const string_t s = BasicJsonType(x).dump();
|
|
m_out.put(s.data(), s.size());
|
|
}
|
|
|
|
void write_string(const node& n)
|
|
{
|
|
const char* const s = m_doc.str(n);
|
|
m_out.put('"');
|
|
if ((n.flags & node_flags::storage) == 0 && !m_style.ensure_ascii)
|
|
{
|
|
// a string of the source without escape sequences has nothing to escape
|
|
m_out.put(s, n.len);
|
|
}
|
|
else if (m_style.ensure_ascii)
|
|
{
|
|
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
|
|
}
|
|
else
|
|
{
|
|
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
|
|
}
|
|
m_out.put('"');
|
|
}
|
|
|
|
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
|
|
/// except in a damaged image loaded with image_check::bounds, for which
|
|
/// this throws what basic_json::dump() throws for the string
|
|
template<bool EnsureAscii>
|
|
void write_escaped(const unsigned char* s, std::size_t n)
|
|
{
|
|
std::size_t i = 0;
|
|
while (i < n)
|
|
{
|
|
std::size_t run = 0;
|
|
if (!EnsureAscii)
|
|
{
|
|
run = string_bulk_run(s + i, n - i);
|
|
}
|
|
else if (is_ascii_copyable(s[i]))
|
|
{
|
|
run = find_ascii_copyable_run(s + i, n - i);
|
|
}
|
|
if (run != 0)
|
|
{
|
|
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
|
|
i += run;
|
|
continue;
|
|
}
|
|
std::uint32_t codepoint = s[i];
|
|
std::size_t len = 1;
|
|
if (codepoint >= 0x80)
|
|
{
|
|
len = validate_one_utf8(s + i, n - i);
|
|
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
|
|
{
|
|
invalid_utf8(s, n);
|
|
return;
|
|
}
|
|
codepoint &= 0xFFu >> (len + 1);
|
|
for (std::size_t k = 1; k < len; ++k)
|
|
{
|
|
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
|
|
}
|
|
}
|
|
write_codepoint<EnsureAscii>(codepoint, s + i, len);
|
|
i += len;
|
|
}
|
|
}
|
|
|
|
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
|
|
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
|
|
{
|
|
const string_t dumped = BasicJsonType(string_t(reinterpret_cast<const char*>(s), n)).dump(); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
|
|
static_cast<void>(dumped);
|
|
}
|
|
|
|
template<bool EnsureAscii>
|
|
void write_codepoint(std::uint32_t codepoint, const unsigned char* bytes, std::size_t len)
|
|
{
|
|
switch (codepoint)
|
|
{
|
|
case 0x08:
|
|
m_out.put("\\b");
|
|
return;
|
|
case 0x09:
|
|
m_out.put("\\t");
|
|
return;
|
|
case 0x0A:
|
|
m_out.put("\\n");
|
|
return;
|
|
case 0x0C:
|
|
m_out.put("\\f");
|
|
return;
|
|
case 0x0D:
|
|
m_out.put("\\r");
|
|
return;
|
|
case 0x22:
|
|
m_out.put("\\\"");
|
|
return;
|
|
case 0x5C:
|
|
m_out.put("\\\\");
|
|
return;
|
|
default:
|
|
break;
|
|
}
|
|
if (codepoint <= 0x1F || (EnsureAscii && codepoint >= 0x7F))
|
|
{
|
|
if (codepoint <= 0xFFFF)
|
|
{
|
|
write_u_escape(codepoint);
|
|
}
|
|
else
|
|
{
|
|
write_u_escape(0xD7C0u + (codepoint >> 10u));
|
|
write_u_escape(0xDC00u + (codepoint & 0x3FFu));
|
|
}
|
|
return;
|
|
}
|
|
m_out.put(reinterpret_cast<const char*>(bytes), len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast) LCOV_EXCL_LINE (printable characters are copied in runs)
|
|
}
|
|
|
|
void write_u_escape(std::uint32_t u)
|
|
{
|
|
static constexpr const char* hex = "0123456789abcdef";
|
|
const std::array<char, 6> e = {{'\\', 'u', hex[(u >> 12u) & 0xFu], hex[(u >> 8u) & 0xFu], hex[(u >> 4u) & 0xFu], hex[u & 0xFu]}};
|
|
m_out.put(e.data(), e.size());
|
|
}
|
|
|
|
const document_data& m_doc;
|
|
output_buffer<string_t> m_out;
|
|
const dump_style m_style;
|
|
};
|
|
|
|
} // namespace view
|
|
} // namespace detail
|
|
NLOHMANN_JSON_NAMESPACE_END
|