State the exact input size limit of json_document

The check rejects inputs of 0xFFFFFFF0 bytes or more, but the exception
message and the documentation said 4 GiB. Name the limit once
(max_input_size), and state 4 GiB minus 16 bytes in the message and the
documentation. Test the limit with a container that only claims the size.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json_document/parse.md

@@ -79,8 +79,8 @@ discarded; see [`is_discarded`](is_discarded.md).
 Throws the same exception [`BasicJsonType::parse()`](../basic_json/parse.md) throws for the same input and options --
 the same exception id, message, and position -- because on a failing input the library's own parser is run on the
 same bytes to produce the diagnostic. Additionally throws
-[`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the input is 4 GiB or larger, a size
-[`BasicJsonType::parse()`](../basic_json/parse.md) does not reject.
+[`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the input is 4294967280 bytes (4 GiB
+minus 16 bytes) or larger, a size [`BasicJsonType::parse()`](../basic_json/parse.md) does not reject.
 
 ## Complexity
 

docs/mkdocs/docs/features/json_view.md

@@ -111,7 +111,7 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
 
 - **Only 64-bit integers.** `basic_json_document<BasicJsonType>` requires `BasicJsonType::number_integer_t` and
   `number_unsigned_t` to both be 64 bits wide; this is a compile-time `#!cpp static_assert`.
-- **A 4 GiB input limit.** An input of 4 GiB or more throws
+- **A 4 GiB input limit.** An input of 4294967280 bytes (4 GiB minus 16 bytes) or more throws
   [`out_of_range.416`](../home/exceptions.md#jsonexceptionout_of_range416), a limit
   `#!cpp basic_json::parse()` does not have.
 - **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`.

docs/mkdocs/docs/home/architecture.md

@@ -210,7 +210,8 @@ packet-beta
 - **Navigation** needs no pointers: the elements of an array or object follow its node, and the node after a value's
   subtree is `next` nodes further for an array or object, and the next node otherwise (`document_data::after`). Views
   step from element to element this way and skip whole subtrees in constant time.
-- **Offsets** are 32 bits wide, so a document is limited to 4 GiB (`out_of_range.416`).
+- **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
+  2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
 
 For example, `#!json {"a": [1, 2.5]}` becomes five nodes. Each node's elements follow it, and `next` leads from an
 array or object past its subtree:

docs/mkdocs/docs/home/exceptions.md

@@ -1048,12 +1048,12 @@ MessagePack's ext type and BSON's binary subtype are each stored in a single byt
 
 [`basic_json_document::parse()`](../api/basic_json_document/parse.md) and the other parsing functions of
 [`basic_json_document`](../api/basic_json_document/index.md) index a value's position in the source text in 32 bits,
-so they do not support an input of 4 GiB or more.
+so they do not support an input of 4294967280 bytes (4 GiB minus 16 bytes) or more.
 
 !!! failure "Example message"
 
     ```
-    [json.exception.out_of_range.416] input of 4 GiB or more is not supported by json_document
+    [json.exception.out_of_range.416] input of 4294967280 bytes or more is not supported by json_document
     ```
 
 !!! note

include/nlohmann/detail/view/errors.hpp

@@ -55,9 +55,8 @@ template<typename BasicJsonType>
 {
     if (f.code == error_code::input_too_large)
     {
-        // LCOV_EXCL_START (4 GiB)
-        NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4 GiB or more is not supported by json_document", nullptr));
-        // LCOV_EXCL_STOP
+        // (the limit is detail::view::max_input_size: 4 GiB minus 16 bytes)
+        NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
     }
     const BasicJsonType accepted = BasicJsonType::parse(src, src + size, nullptr, true, ignore_comments, ignore_trailing_commas);
     // LCOV_EXCL_START (only if parse() accepts what the view rejects: a bug)

include/nlohmann/detail/view/node.hpp

@@ -29,6 +29,11 @@ static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::
               && static_cast<std::uint8_t>(value_t::number_unsigned) == 6 && static_cast<std::uint8_t>(value_t::number_float) == 7,
               "the node format depends on the numbering of value_t");
 
+/// The largest input a document accepts, in bytes. Offsets and node counts are
+/// 32 bits wide; the limit keeps 16 bytes (the width of the scanner's steps)
+/// below 2^32, so that a position one step past the end of the text fits.
+static constexpr std::size_t max_input_size = 0xFFFFFFEFu;
+
 /// node flags
 struct node_flags
 {

include/nlohmann/json_view.hpp

@@ -475,9 +475,9 @@ class basic_json_document
         d.discarded = true;
         detail::view::parse_failure failure;
         bool ok = false;
-        if (NLOHMANN_VIEW_UNLIKELY(size >= 0xFFFFFFF0u))
+        if (NLOHMANN_VIEW_UNLIKELY(size > detail::view::max_input_size))
         {
-            failure.code = detail::view::error_code::input_too_large; // LCOV_EXCL_LINE (4 GiB)
+            failure.code = detail::view::error_code::input_too_large;
         }
         else
         {

tests/src/unit-json_view.cpp

@@ -48,6 +48,25 @@ auto materialized_copy(Input&& input) -> decltype(std::declval<typename Document
     return d.root().materialize();
 }
 
+// a "byte container" that claims to hold `size` bytes, to reach the limit on
+// the size of the input without allocating gigabytes; nothing past the first
+// bytes is ever read, because the size is checked before the parse starts
+struct oversized_input
+{
+    using value_type = char;
+    std::size_t claimed;
+
+    const char* data() const
+    {
+        return "[1]";
+    }
+
+    std::size_t size() const
+    {
+        return claimed;
+    }
+};
+
 // detection of calls that must not compile
 template<typename... Args>
 using parse_call_t = decltype(json_document::parse(std::declval<Args>()...));
@@ -427,6 +446,22 @@ TEST_CASE("json_view")
         CHECK(moved.root().size() == 1);
     }
 
+    SECTION("input size limit")
+    {
+        // 32-bit offsets: the limit is 4 GiB minus 16 bytes (a margin below 2^32),
+        // which is what the exception message and the documentation say
+        const std::size_t limit = nlohmann::detail::view::max_input_size;
+        CHECK(limit == std::size_t{4294967279u});
+
+        const oversized_input input{limit + 1};
+        CHECK(!json_document::accept(input));
+        CHECK(json_document::parse(input, false).is_discarded());
+#if !defined(JSON_NOEXCEPTION)
+        json_document d;
+        CHECK_THROWS_WITH_AS(d = json_document::parse(input), "[json.exception.out_of_range.416] input of 4294967280 bytes or more is not supported by json_document", json::out_of_range&);
+#endif
+    }
+
     SECTION("document lifetime and reuse")
     {
         json_document d;