Fix -Wuseless-cast in array_prefix_set::reseed

uintptr_t is already uint64_t on LP64 Linux, so the explicit cast is
useless there; rely on the implicit (widening) conversion instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/dependabot.yml

@@ -67,4 +67,3 @@ updates:
       - dependency-name: mcp
       - dependency-name: boltons
       - dependency-name: "opentelemetry-*"
-      - dependency-name: importlib-metadata

cmake/requirements/requirements-semgrep.txt

@@ -391,9 +391,9 @@ httpx-sse==0.4.3 \
 idna==3.20 \
     --hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \
     --hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c
-importlib-metadata==8.7.1 \
+importlib-metadata==10.1.0 \
-    --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \
+    --hash=sha256:82b04047fa09792c1a8cfeddfe08bec79e4b13c29c8469b0d3949860f61017e8 \
-    --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151
+    --hash=sha256:cf45ce66a929c64e994185a809c04840d0501db17e663810d993d0f4829265c7
 jsonschema==4.25.1 \
     --hash=sha256:3fba0169e345c7175110351d456342c364814cfcf3b964ba4587f22915230a63 \
     --hash=sha256:e4a9655ce0da0c0b67a085847e00a3a51449e1157f4f75e9fb5aa545e122eb85

docs/mkdocs/docs/api/basic_json/operator_ge.md

@@ -60,14 +60,6 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
-    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
-    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
-    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
-    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
-    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
-    same for all language standards. The same program can therefore order such values differently when it is compiled
-    with a different language standard.
-
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_gt.md

@@ -60,14 +60,6 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
-    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
-    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
-    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
-    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
-    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
-    same for all language standards. The same program can therefore order such values differently when it is compiled
-    with a different language standard.
-
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_le.md

@@ -61,14 +61,6 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
-    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
-    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
-    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
-    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
-    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
-    same for all language standards. The same program can therefore order such values differently when it is compiled
-    with a different language standard.
-
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_lt.md

@@ -71,14 +71,6 @@ Linear.
       2. Comparing a `NaN` with another `NaN`.
       3. Comparing a `NaN` and any other number.
 
-    For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
-    another `NaN`) before any other difference, the result depends on the language standard. Before C++20, the pair
-    counts as equivalent and the following elements (or the sizes) decide: for instance, `[NaN] < [1, NaN]` and
-    `[1] <= [NaN]` are `#!cpp true`. Since C++20, the whole comparison is unordered, and `<`, `<=`, `>`, and `>=` all
-    yield `#!cpp false`. Otherwise (for example `[1, NaN] < [2, NaN]`, or objects whose keys differ), the result is the
-    same for all language standards. The same program can therefore order such values differently when it is compiled
-    with a different language standard.
-
 !!! note "Operator overload resolution"
 
     Since C++20 overload resolution will consider the _rewritten candidate_ generated from

docs/mkdocs/docs/api/basic_json/operator_spaceship.md

@@ -55,10 +55,6 @@ Linear.
         1. Comparing a `NaN` with itself.
         2. Comparing a `NaN` with another `NaN`.
         3. Comparing a `NaN` and any other number.
-    - For arrays and objects, the elements are compared in order. If the comparison reaches a `NaN` and a number (or
-      another `NaN`) before any other difference, the result is `std::partial_ordering::unordered`. (Without C++20, the
-      comparison operators `<`, `<=`, `>`, and `>=` use the comparison of the containers, which treats such a pair as
-      equivalent; see [**operator<**](operator_lt.md).)
 
 ## Examples
 

docs/mkdocs/docs/api/basic_json/parse.md

@@ -130,11 +130,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
 
     Contiguous inputs (strings, string views, vectors, arrays, pointer ranges) are the fastest to parse. A
     `#!cpp std::istream` comes close: the parser scans the characters its stream buffer holds in bulk, without reading
-    ahead, so the stream is left exactly where it would be after reading one character at a time. With glibc
+    ahead, so the stream is left exactly where it would be after reading one character at a time. A `#!cpp FILE*` is
-    and the Apple C library, the characters the `#!cpp FILE` holds in its buffer are scanned in bulk like a stream's;
+    read one character at a time, but the file is locked only once, so other threads that use the same `FILE` object
-    elsewhere a `#!cpp FILE*` is read one character at a time. Where the C library has unlocked stdio functions, the
+    wait until parsing is done. See [input sources and performance](../../features/parsing/index.md#input-sources-and-performance),
-    file is locked only once, so other threads that use the same `FILE` object wait until parsing is done. See
-    [input sources and performance](../../features/parsing/index.md#input-sources-and-performance),
     [`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md), and
     [`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md).
 
@@ -313,10 +311,6 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
   it as end of input; planned to become the default in version 4.0.0.
 - Extended empty-input detection to also cover an `std::istream` without a stream buffer, and fixed a crash
   (`std::terminate`) when parsing from an `std::istream` with `eofbit` in its exception mask, in version 3.13.0.
-- Fixed the position that a [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) reports for an
-  unexpected number that is followed by a line feed in version 3.13.0: for the input `1`, line feed, `2`, line feed,
-  version 3.12.0 reported "line 2, column 0" and version 3.13.0 reports "line 2, column 1"; for `1 2` and a line feed,
-  the report was "line 1, column 0" and is now "line 1, column 3".
 - Parsing from an `std::istream` scans the characters in its stream buffer in bulk in version 3.13.0
   ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md) restores reading one character at a
   time).

docs/mkdocs/docs/api/basic_json/sax_parse.md

@@ -124,10 +124,9 @@ consumer `sax` has a super-linear complexity.
 
 A UTF-8 byte order mark is silently ignored.
 
-When reading from a `#!c FILE*`, where the C library has unlocked stdio functions (see
+When reading from a `#!c FILE*`, the file is locked while it is parsed. The lock is recursive, so the SAX handler may
-[`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)), the file is locked while it is parsed.
+use the same file with stdio functions, but it must not wait for another thread that uses the same `FILE` object; see
-The lock is recursive, so the SAX handler may use the same file with stdio functions, but it must not wait for another
+[`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md).
-thread that uses the same `FILE` object.
 
 ## Examples
 

docs/mkdocs/docs/api/basic_json/start_pos.md

@@ -67,7 +67,3 @@ Constant.
 ## Version history
 
 - Added in version 3.12.0.
-- Fixed in version 3.13.0: in version 3.12.0, the start position of a string that contains escape sequences (for
-  example `#!cpp "\n"` or `#!cpp "\u00e4"`) was too large, by the number of bytes the escape sequences add to the
-  source text. The position is now always that of the opening `"`. The "(bytes a-b)" range in the messages of
-  [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md) was wrong in the same way.

docs/mkdocs/docs/api/macros/json_diagnostic_positions.md

@@ -125,5 +125,3 @@ When the macro is not defined, the library will define it to its default value.
 ## Version history
 
 - Added in version 3.12.0.
-- Fixed in version 3.13.0: in version 3.12.0, [`start_pos()`](../basic_json/start_pos.md) of a string that contains
-  escape sequences was too large; it is now the position of the opening `"`.

docs/mkdocs/docs/features/binary_formats/bjdata.md

@@ -88,6 +88,7 @@ The library uses the following mapping from JSON values types to BJData types ac
 | number_integer  | 65536..2147483647                         | int32       | `l`                                                                     |
 | number_integer  | 2147483648..4294967295                    | uint32      | `m`                                                                     |
 | number_integer  | 4294967296..9223372036854775807           | int64       | `L`                                                                     |
+| number_integer  | 9223372036854775808..18446744073709551615 | uint64      | `M`                                                                     |
 | number_unsigned | 0..127                                    | int8        | `i`                                                                     |
 | number_unsigned | 128..255                                  | uint8       | `U`                                                                     |
 | number_unsigned | 256..32767                                | int16       | `I`                                                                     |

docs/mkdocs/docs/features/binary_formats/messagepack.md

@@ -61,7 +61,7 @@ specification:
 | number_integer  | 128..255                                 | uint 8           | 0xCC       |
 | number_integer  | 256..65535                               | uint 16          | 0xCD       |
 | number_integer  | 65536..4294967295                        | uint 32          | 0xCE       |
-| number_integer  | 4294967296..9223372036854775807          | uint 64          | 0xCF       |
+| number_integer  | 4294967296..18446744073709551615         | uint 64          | 0xCF       |
 | number_unsigned | 0..127                                   | positive fixint  | 0x00..0x7F |
 | number_unsigned | 128..255                                 | uint 8           | 0xCC       |
 | number_unsigned | 256..65535                               | uint 16          | 0xCD       |

docs/mkdocs/docs/features/parsing/index.md

@@ -77,7 +77,7 @@ them faster than others:
 |--------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
 | strings, string views, string literals, contiguous containers, pointer or iterator ranges over contiguous memory | the whole input is scanned in bulk: runs of string characters, numbers, and indentation are recognized several bytes at a time |
 | `#!cpp std::istream` (`#!cpp std::ifstream`, `#!cpp std::istringstream`, ...)            | the characters the stream buffer holds are scanned in bulk like a string; the stream buffer is asked for more when they are used up |
-| `#!cpp FILE*`                                                                              | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; where the C library has unlocked stdio functions, the file is locked once for the whole parse |
+| `#!cpp FILE*`                                                                              | with glibc and the Apple C library, the characters the `FILE` holds in its buffer are scanned in bulk like a stream's; elsewhere one character at a time; the file is locked once for the whole parse |
 | other iterator ranges (e.g., `#!cpp std::list<char>`, `#!cpp std::istreambuf_iterator<char>`), wide strings | one character at a time                                                             |
 
 Input in one block of memory is read even faster with the default options (no

docs/mkdocs/docs/features/performance.md

@@ -27,9 +27,8 @@ A `#!cpp std::istream` (including `#!cpp std::ifstream`) gets the same bulk scan
 characters its stream buffer already holds (the *get area*) and asks the stream buffer for more only when they are used
 up. Nothing is read ahead, so the stream is left exactly where it would be after reading one character at a time, and
 [`operator>>`](../api/operator_gtgt.md) can still read several values from one stream. A stream buffer without a get
-area is read one character at a time. Where the C library has unlocked stdio functions, a `#!cpp FILE*` is locked once
+area is read one character at a time. A `#!cpp FILE*` is locked once per parse instead of once per character, and with
-per parse instead of once per character, and with glibc and the Apple C library, the characters in its buffer are
+glibc and the Apple C library, the characters in its buffer are scanned in bulk as a stream's get area is. See
-scanned in bulk as a stream's get area is. See
 [input sources and performance](parsing/index.md#input-sources-and-performance) for the full table. Parsing from a
 stream takes about 1.15 to 1.45 times as long as parsing an equivalent `#!cpp std::string`, which `parse()` reads with
 a parser of its own (see there).

include/nlohmann/detail/json_pointer.hpp

@@ -852,139 +852,6 @@ class json_pointer
     }
 
   private:
-    /*!
-    @brief the pointer prefixes of a flattened object that denote arrays
-
-    @ref unflatten creates arrays exactly below the prefixes that have a
-    reference token 0 among their children, and objects everywhere else.
-    Deciding this up front keeps the result independent of the order in which
-    the flattened object is iterated, which is unspecified for some object
-    types.
-
-    A prefix is identified by the text of the key it is the beginning of:
-    escaping is unambiguous, so equal prefixes of valid keys are equal texts.
-    The set refers to the keys, which must outlive it.
-    */
-    class array_prefix_set
-    {
-      public:
-        void insert(const char* data, std::size_t size)
-        {
-            // flattened objects are usually sorted, so most keys that have a
-            // 0 yield the prefix of the previous one
-            if (last_data != nullptr && last_size == size && std::memcmp(last_data, data, size) == 0)
-            {
-                return;
-            }
-            last_data = data;
-            last_size = size;
-
-            if (table.empty())
-            {
-                table.resize(16);
-            }
-            const std::size_t hash = hash_of(data, size);
-            const std::size_t mask = table.size() - 1;
-            std::size_t i = hash & mask;
-            while (table[i].data != nullptr)
-            {
-                if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
-                {
-                    return;
-                }
-                i = (i + 1) & mask;
-            }
-            table[i] = slot(hash, data, size);
-
-            // keep the load factor below 1/2
-            if (++used * 2 > table.size())
-            {
-                grow();
-            }
-        }
-
-        bool empty() const noexcept
-        {
-            return used == 0;
-        }
-
-        bool contains(const char* data, std::size_t size) const noexcept
-        {
-            if (used == 0)
-            {
-                return false;
-            }
-            const std::size_t hash = hash_of(data, size);
-            const std::size_t mask = table.size() - 1;
-            for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
-            {
-                if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
-                {
-                    return true;
-                }
-            }
-            return false;
-        }
-
-      private:
-        struct slot
-        {
-            slot() noexcept = default;
-
-            slot(std::size_t h, const char* d, std::size_t s) noexcept
-                : hash(h), data(d), size(s)
-            {}
-
-            std::size_t hash = 0;
-            const char* data = nullptr; // nullptr: the slot is empty
-            std::size_t size = 0;
-        };
-
-        static std::size_t hash_of(const char* data, std::size_t size) noexcept
-        {
-            std::uint64_t h = 0x9E3779B97F4A7C15ULL ^ size;
-            std::size_t i = 0;
-            for (; i + 8 <= size; i += 8)
-            {
-                std::uint64_t w = 0;
-                std::memcpy(&w, data + i, 8);
-                h = (h ^ w) * 0x100000001B3ULL;
-                h ^= h >> 32u;
-            }
-            std::uint64_t w = 0;
-            if (i < size)
-            {
-                std::memcpy(&w, data + i, size - i);
-            }
-            h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
-            return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
-        }
-
-        void grow()
-        {
-            std::vector<slot> old(table.size() * 2);
-            old.swap(table);
-            const std::size_t mask = table.size() - 1;
-            for (const auto& s : old)
-            {
-                if (s.data != nullptr)
-                {
-                    std::size_t i = s.hash & mask;
-                    while (table[i].data != nullptr)
-                    {
-                        i = (i + 1) & mask;
-                    }
-                    table[i] = s;
-                }
-            }
-        }
-
-        std::vector<slot> table{};
-        std::size_t used = 0;
-        const char* last_data = nullptr;
-        std::size_t last_size = 0;
-    };
-
     /// @brief the length of the longest common prefix of the first @a limit bytes of @a a and @a b
     static std::size_t common_prefix_length(const char* a, const char* b, const std::size_t limit) noexcept
     {
@@ -1007,6 +874,355 @@ class json_pointer
         return i;
     }
 
+    /*!
+    @brief the prefixes of a flattened object that are marked as arrays, for
+           prefixes that are not short
+
+    A prefix is identified by the tokens it consists of: a node of a tree
+    stands for the prefix of its parent followed by a token, and is looked up
+    by its parent and the text of the token. The node of a prefix is found by
+    walking the tokens of the key from the root, so identifying all prefixes
+    of a key together costs one pass over the key, however long the prefixes
+    are, and no prefix is ever compared byte by byte as a whole. The nodes
+    along the key that was walked last are kept, and the walk of the next key
+    starts below the beginning it has in common with that key. The tree refers
+    to the keys, which must outlive it.
+    */
+    class prefix_tree
+    {
+      public:
+        /// @brief set the seed of the hash function, before anything is added
+        void reseed(const std::uint64_t seed_value) noexcept
+        {
+            seed = seed_value;
+        }
+
+        /// @brief mark the first @a prefix_size bytes of the key of @a key_size bytes at @a data
+        void mark(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            nodes[walk(data, key_size, prefix_size)].marked = true;
+            any_marked = true;
+        }
+
+        bool is_marked(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            return any_marked && nodes[walk(data, key_size, prefix_size)].marked;
+        }
+
+        bool empty() const noexcept
+        {
+            return !any_marked;
+        }
+
+        /// @brief the hash of the first @a size bytes at @a data, for the seed @a seed_value
+        static std::size_t hash_bytes(const char* data, std::size_t size, const std::uint64_t seed_value) noexcept
+        {
+            std::uint64_t h = seed_value ^ size;
+            std::size_t i = 0;
+            for (; i + 8 <= size; i += 8)
+            {
+                std::uint64_t w = 0;
+                std::memcpy(&w, data + i, 8);
+                h = (h ^ w) * 0x100000001B3ULL;
+                h ^= h >> 32u;
+            }
+            std::uint64_t w = 0;
+            if (i < size)
+            {
+                std::memcpy(&w, data + i, size - i);
+            }
+            h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
+            return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
+        }
+
+      private:
+        struct node
+        {
+            std::size_t parent = 0;
+            std::size_t hash = 0;
+            const char* token = nullptr;
+            std::size_t token_size = 0;
+            bool marked = false;
+        };
+
+        /// @brief a slot of the hash table; the index of a node, 0 if the slot is empty
+        struct bucket
+        {
+            std::size_t index = 0;
+        };
+
+        /// @brief the node after a token of the key that was walked last, and where the token ends
+        struct step
+        {
+            std::size_t node = 0;
+            std::size_t end = 0;
+        };
+
+        /// @brief the index of the node of the prefix that ends at the slash (or the end) at @a prefix_size
+        std::size_t walk(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            if (nodes.empty())
+            {
+                nodes.emplace_back();
+                table.resize(16);
+                chain.emplace_back();
+            }
+
+            if (data != last_key)
+            {
+                // keep the nodes within the beginning that this key has in common
+                // with the key they were found for; a prefix that ends inside a
+                // token of this key is not one of its prefixes
+                const std::size_t same = common_prefix_length(data, last_key, (std::min)(key_size, last_key_size));
+                std::size_t keep = chain.size();
+                while (keep > 1 && (chain[keep - 1].end > same || (chain[keep - 1].end == same && same < key_size && data[same] != '/')))
+                {
+                    --keep;
+                }
+                chain.resize(keep);
+                last_key = data;
+                last_key_size = key_size;
+            }
+
+            // walk the tokens that are not walked yet
+            while (chain.back().end < prefix_size && chain.back().end < key_size)
+            {
+                const std::size_t start = chain.back().end + 1;
+                const void* const slash = std::memchr(data + start, '/', key_size - start);
+                const std::size_t end = slash == nullptr ? key_size : static_cast<std::size_t>(static_cast<const char*>(slash) - data);
+                step next;
+                next.node = child(chain.back().node, data + start, end - start);
+                next.end = end;
+                chain.push_back(next);
+            }
+
+            if (chain.back().end == prefix_size)
+            {
+                return chain.back().node;
+            }
+            // an earlier call walked further
+            const auto it = std::lower_bound(chain.begin(), chain.end(), prefix_size, [](const step & s, std::size_t size)
+            {
+                return s.end < size;
+            });
+            return it->node;
+        }
+
+        std::size_t slot_of(std::size_t parent, std::size_t hash) const noexcept
+        {
+            std::uint64_t x = static_cast<std::uint64_t>(hash) ^ (static_cast<std::uint64_t>(parent) * 0x9E3779B97F4A7C15ULL);
+            x ^= x >> 32u;
+            x *= 0xD6E8FEB86659FD93ULL;
+            x ^= x >> 32u;
+            return static_cast<std::size_t>(x) & (table.size() - 1);
+        }
+
+        /// @brief the node of the prefix of the node @a parent followed by the token, which is added if it is not there
+        std::size_t child(std::size_t parent, const char* token, std::size_t token_size)
+        {
+            const std::size_t hash = hash_bytes(token, token_size, seed);
+            std::size_t i = slot_of(parent, hash);
+            while (table[i].index != 0)
+            {
+                const node& n = nodes[table[i].index];
+                if (n.parent == parent && n.hash == hash && n.token_size == token_size && std::memcmp(n.token, token, token_size) == 0)
+                {
+                    return table[i].index;
+                }
+                i = (i + 1) & (table.size() - 1);
+            }
+
+            node created;
+            created.parent = parent;
+            created.hash = hash;
+            created.token = token;
+            created.token_size = token_size;
+            nodes.push_back(created);
+            table[i].index = nodes.size() - 1;
+
+            // keep the load factor below 1/2
+            if (nodes.size() * 2 > table.size())
+            {
+                std::vector<bucket> old(table.size() * 2);
+                old.swap(table);
+                for (std::size_t k = 1; k < nodes.size(); ++k)
+                {
+                    std::size_t j = slot_of(nodes[k].parent, nodes[k].hash);
+                    while (table[j].index != 0)
+                    {
+                        j = (j + 1) & (table.size() - 1);
+                    }
+                    table[j].index = k;
+                }
+            }
+            return nodes.size() - 1;
+        }
+
+        std::uint64_t seed = 0;
+        /// @brief the nodes; the first is the root, the empty prefix
+        std::vector<node> nodes{};
+        /// @brief the nodes (not the root) in a hash table
+        std::vector<bucket> table{};
+        /// @brief the root, and the node after each token of the key that was walked last
+        std::vector<step> chain{};
+        const char* last_key = nullptr;
+        std::size_t last_key_size = 0;
+        bool any_marked = false;
+    };
+
+    /*!
+    @brief the pointer prefixes of a flattened object that denote arrays
+
+    @ref unflatten creates arrays exactly below the prefixes that have a
+    reference token 0 among their children, and objects everywhere else.
+    Deciding this up front keeps the result independent of the order in which
+    the flattened object is iterated, which is unspecified for some object
+    types.
+
+    A prefix is identified by the text of the key it is the beginning of:
+    escaping is unambiguous, so equal prefixes of valid keys are equal texts.
+    The set refers to the keys, which must outlive it. Short prefixes are
+    kept in a hash table and compared as texts, which costs at most the length
+    of the bound. Longer prefixes are kept in a @ref prefix_tree, because
+    hashing and comparing every one of the prefixes of a key with many tokens
+    from its beginning would take quadratic time.
+
+    The hash function depends on a seed that is derived from the address of a
+    local variable, which address space layout randomization changes between
+    processes, so keys cannot be crafted to collide in advance; the result
+    does not depend on the seed. Where the addresses are not randomized, or
+    when the seed is known, keys can collide: k short prefixes then cost
+    k^2 / 2 comparisons of fewer than 64 bytes each. The long prefixes are not
+    hashed as a whole.
+    */
+    class array_prefix_set
+    {
+      public:
+        /// @brief choose the seed of the hash function, before anything is added: it is derived from the address that is passed
+        void reseed(const void* address) noexcept
+        {
+            // NOLINTNEXTLINE(cppcoreguidelines-pro-type-reinterpret-cast): the address is only a source of entropy
+            std::uint64_t x = reinterpret_cast<std::uintptr_t>(address);
+            x ^= 0x9E3779B97F4A7C15ULL;
+            x *= 0xBF58476D1CE4E5B9ULL;
+            seed = x ^ (x >> 31u);
+            tree.reseed(seed);
+        }
+
+        /// @brief add the first @a prefix_size bytes of the key of @a key_size bytes at @a data
+        void insert(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            if (prefix_size >= short_limit)
+            {
+                tree.mark(data, key_size, prefix_size);
+                return;
+            }
+
+            // flattened objects are usually sorted, so most keys that have a
+            // 0 yield the prefix of the previous one
+            if (last_data != nullptr && last_size == prefix_size && std::memcmp(last_data, data, prefix_size) == 0)
+            {
+                return;
+            }
+            last_data = data;
+            last_size = prefix_size;
+
+            const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
+            if (table.empty())
+            {
+                table.resize(16);
+            }
+            const std::size_t mask = table.size() - 1;
+            std::size_t i = hash & mask;
+            while (table[i].data != nullptr)
+            {
+                if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
+                {
+                    return;
+                }
+                i = (i + 1) & mask;
+            }
+            table[i] = slot(hash, data, prefix_size);
+
+            // keep the load factor below 1/2
+            if (++used * 2 > table.size())
+            {
+                grow();
+            }
+        }
+
+        bool empty() const noexcept
+        {
+            return used == 0 && tree.empty();
+        }
+
+        /// @brief whether the first @a prefix_size bytes of the key of @a key_size bytes at @a data were added
+        bool contains(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            if (prefix_size >= short_limit)
+            {
+                return tree.is_marked(data, key_size, prefix_size);
+            }
+            if (used == 0)
+            {
+                return false;
+            }
+            const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
+            const std::size_t mask = table.size() - 1;
+            for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
+            {
+                if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
+                {
+                    return true;
+                }
+            }
+            return false;
+        }
+
+      private:
+        /// @brief prefixes of fewer bytes are kept in the hash table
+        static constexpr std::size_t short_limit = 64;
+
+        struct slot
+        {
+            slot() noexcept = default;
+
+            slot(std::size_t h, const char* d, std::size_t s) noexcept
+                : hash(h), data(d), size(s)
+            {}
+
+            std::size_t hash = 0;
+            const char* data = nullptr; // nullptr: the slot is empty
+            std::size_t size = 0;
+        };
+
+        void grow()
+        {
+            std::vector<slot> old(table.size() * 2);
+            old.swap(table);
+            const std::size_t mask = table.size() - 1;
+            for (const auto& s : old)
+            {
+                if (s.data != nullptr)
+                {
+                    std::size_t i = s.hash & mask;
+                    while (table[i].data != nullptr)
+                    {
+                        i = (i + 1) & mask;
+                    }
+                    table[i] = s;
+                }
+            }
+        }
+
+        std::uint64_t seed = 0;
+        prefix_tree tree{};
+        std::vector<slot> table{};
+        std::size_t used = 0;
+        const char* last_data = nullptr;
+        std::size_t last_size = 0;
+    };
+
     /*!
     @brief check that @a key is a valid JSON pointer, and add the prefixes of
            it that denote arrays to @a arrays
@@ -1052,7 +1268,7 @@ class json_pointer
                 // a token 0 makes the pointer before it an array
                 if (i - begin == 1 && data[begin] == '0')
                 {
-                    arrays.insert(data, begin - 1);
+                    arrays.insert(data, size, begin - 1);
                 }
                 begin = i + 1;
             }
@@ -2062,6 +2278,7 @@ class json_pointer
         // token 0 among their children; the values below them are arrays,
         // all others are objects (see array_prefix_set)
         array_prefix_set arrays;
+        arrays.reseed(&arrays);
         const string_t* previous_key = nullptr;
         for (const auto& element : *value.m_data.m_value.object)
         {
@@ -2153,7 +2370,7 @@ class json_pointer
                     case detail::value_t::null:
                     {
                         // a first child 0 makes the value an array without a look-up
-                        if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, pos))
+                        if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, key.size(), pos))
                         {
                             // some reference token below this position is 0, so the
                             // value is an array

single_include/nlohmann/json.hpp

@@ -27027,139 +27027,6 @@ class json_pointer
     }
 
   private:
-    /*!
-    @brief the pointer prefixes of a flattened object that denote arrays
-
-    @ref unflatten creates arrays exactly below the prefixes that have a
-    reference token 0 among their children, and objects everywhere else.
-    Deciding this up front keeps the result independent of the order in which
-    the flattened object is iterated, which is unspecified for some object
-    types.
-
-    A prefix is identified by the text of the key it is the beginning of:
-    escaping is unambiguous, so equal prefixes of valid keys are equal texts.
-    The set refers to the keys, which must outlive it.
-    */
-    class array_prefix_set
-    {
-      public:
-        void insert(const char* data, std::size_t size)
-        {
-            // flattened objects are usually sorted, so most keys that have a
-            // 0 yield the prefix of the previous one
-            if (last_data != nullptr && last_size == size && std::memcmp(last_data, data, size) == 0)
-            {
-                return;
-            }
-            last_data = data;
-            last_size = size;
-
-            if (table.empty())
-            {
-                table.resize(16);
-            }
-            const std::size_t hash = hash_of(data, size);
-            const std::size_t mask = table.size() - 1;
-            std::size_t i = hash & mask;
-            while (table[i].data != nullptr)
-            {
-                if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
-                {
-                    return;
-                }
-                i = (i + 1) & mask;
-            }
-            table[i] = slot(hash, data, size);
-
-            // keep the load factor below 1/2
-            if (++used * 2 > table.size())
-            {
-                grow();
-            }
-        }
-
-        bool empty() const noexcept
-        {
-            return used == 0;
-        }
-
-        bool contains(const char* data, std::size_t size) const noexcept
-        {
-            if (used == 0)
-            {
-                return false;
-            }
-            const std::size_t hash = hash_of(data, size);
-            const std::size_t mask = table.size() - 1;
-            for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
-            {
-                if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
-                {
-                    return true;
-                }
-            }
-            return false;
-        }
-
-      private:
-        struct slot
-        {
-            slot() noexcept = default;
-
-            slot(std::size_t h, const char* d, std::size_t s) noexcept
-                : hash(h), data(d), size(s)
-            {}
-
-            std::size_t hash = 0;
-            const char* data = nullptr; // nullptr: the slot is empty
-            std::size_t size = 0;
-        };
-
-        static std::size_t hash_of(const char* data, std::size_t size) noexcept
-        {
-            std::uint64_t h = 0x9E3779B97F4A7C15ULL ^ size;
-            std::size_t i = 0;
-            for (; i + 8 <= size; i += 8)
-            {
-                std::uint64_t w = 0;
-                std::memcpy(&w, data + i, 8);
-                h = (h ^ w) * 0x100000001B3ULL;
-                h ^= h >> 32u;
-            }
-            std::uint64_t w = 0;
-            if (i < size)
-            {
-                std::memcpy(&w, data + i, size - i);
-            }
-            h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
-            return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
-        }
-
-        void grow()
-        {
-            std::vector<slot> old(table.size() * 2);
-            old.swap(table);
-            const std::size_t mask = table.size() - 1;
-            for (const auto& s : old)
-            {
-                if (s.data != nullptr)
-                {
-                    std::size_t i = s.hash & mask;
-                    while (table[i].data != nullptr)
-                    {
-                        i = (i + 1) & mask;
-                    }
-                    table[i] = s;
-                }
-            }
-        }
-
-        std::vector<slot> table{};
-        std::size_t used = 0;
-        const char* last_data = nullptr;
-        std::size_t last_size = 0;
-    };
-
     /// @brief the length of the longest common prefix of the first @a limit bytes of @a a and @a b
     static std::size_t common_prefix_length(const char* a, const char* b, const std::size_t limit) noexcept
     {
@@ -27182,6 +27049,355 @@ class json_pointer
         return i;
     }
 
+    /*!
+    @brief the prefixes of a flattened object that are marked as arrays, for
+           prefixes that are not short
+
+    A prefix is identified by the tokens it consists of: a node of a tree
+    stands for the prefix of its parent followed by a token, and is looked up
+    by its parent and the text of the token. The node of a prefix is found by
+    walking the tokens of the key from the root, so identifying all prefixes
+    of a key together costs one pass over the key, however long the prefixes
+    are, and no prefix is ever compared byte by byte as a whole. The nodes
+    along the key that was walked last are kept, and the walk of the next key
+    starts below the beginning it has in common with that key. The tree refers
+    to the keys, which must outlive it.
+    */
+    class prefix_tree
+    {
+      public:
+        /// @brief set the seed of the hash function, before anything is added
+        void reseed(const std::uint64_t seed_value) noexcept
+        {
+            seed = seed_value;
+        }
+
+        /// @brief mark the first @a prefix_size bytes of the key of @a key_size bytes at @a data
+        void mark(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            nodes[walk(data, key_size, prefix_size)].marked = true;
+            any_marked = true;
+        }
+
+        bool is_marked(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            return any_marked && nodes[walk(data, key_size, prefix_size)].marked;
+        }
+
+        bool empty() const noexcept
+        {
+            return !any_marked;
+        }
+
+        /// @brief the hash of the first @a size bytes at @a data, for the seed @a seed_value
+        static std::size_t hash_bytes(const char* data, std::size_t size, const std::uint64_t seed_value) noexcept
+        {
+            std::uint64_t h = seed_value ^ size;
+            std::size_t i = 0;
+            for (; i + 8 <= size; i += 8)
+            {
+                std::uint64_t w = 0;
+                std::memcpy(&w, data + i, 8);
+                h = (h ^ w) * 0x100000001B3ULL;
+                h ^= h >> 32u;
+            }
+            std::uint64_t w = 0;
+            if (i < size)
+            {
+                std::memcpy(&w, data + i, size - i);
+            }
+            h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
+            return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
+        }
+
+      private:
+        struct node
+        {
+            std::size_t parent = 0;
+            std::size_t hash = 0;
+            const char* token = nullptr;
+            std::size_t token_size = 0;
+            bool marked = false;
+        };
+
+        /// @brief a slot of the hash table; the index of a node, 0 if the slot is empty
+        struct bucket
+        {
+            std::size_t index = 0;
+        };
+
+        /// @brief the node after a token of the key that was walked last, and where the token ends
+        struct step
+        {
+            std::size_t node = 0;
+            std::size_t end = 0;
+        };
+
+        /// @brief the index of the node of the prefix that ends at the slash (or the end) at @a prefix_size
+        std::size_t walk(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            if (nodes.empty())
+            {
+                nodes.emplace_back();
+                table.resize(16);
+                chain.emplace_back();
+            }
+
+            if (data != last_key)
+            {
+                // keep the nodes within the beginning that this key has in common
+                // with the key they were found for; a prefix that ends inside a
+                // token of this key is not one of its prefixes
+                const std::size_t same = common_prefix_length(data, last_key, (std::min)(key_size, last_key_size));
+                std::size_t keep = chain.size();
+                while (keep > 1 && (chain[keep - 1].end > same || (chain[keep - 1].end == same && same < key_size && data[same] != '/')))
+                {
+                    --keep;
+                }
+                chain.resize(keep);
+                last_key = data;
+                last_key_size = key_size;
+            }
+
+            // walk the tokens that are not walked yet
+            while (chain.back().end < prefix_size && chain.back().end < key_size)
+            {
+                const std::size_t start = chain.back().end + 1;
+                const void* const slash = std::memchr(data + start, '/', key_size - start);
+                const std::size_t end = slash == nullptr ? key_size : static_cast<std::size_t>(static_cast<const char*>(slash) - data);
+                step next;
+                next.node = child(chain.back().node, data + start, end - start);
+                next.end = end;
+                chain.push_back(next);
+            }
+
+            if (chain.back().end == prefix_size)
+            {
+                return chain.back().node;
+            }
+            // an earlier call walked further
+            const auto it = std::lower_bound(chain.begin(), chain.end(), prefix_size, [](const step & s, std::size_t size)
+            {
+                return s.end < size;
+            });
+            return it->node;
+        }
+
+        std::size_t slot_of(std::size_t parent, std::size_t hash) const noexcept
+        {
+            std::uint64_t x = static_cast<std::uint64_t>(hash) ^ (static_cast<std::uint64_t>(parent) * 0x9E3779B97F4A7C15ULL);
+            x ^= x >> 32u;
+            x *= 0xD6E8FEB86659FD93ULL;
+            x ^= x >> 32u;
+            return static_cast<std::size_t>(x) & (table.size() - 1);
+        }
+
+        /// @brief the node of the prefix of the node @a parent followed by the token, which is added if it is not there
+        std::size_t child(std::size_t parent, const char* token, std::size_t token_size)
+        {
+            const std::size_t hash = hash_bytes(token, token_size, seed);
+            std::size_t i = slot_of(parent, hash);
+            while (table[i].index != 0)
+            {
+                const node& n = nodes[table[i].index];
+                if (n.parent == parent && n.hash == hash && n.token_size == token_size && std::memcmp(n.token, token, token_size) == 0)
+                {
+                    return table[i].index;
+                }
+                i = (i + 1) & (table.size() - 1);
+            }
+
+            node created;
+            created.parent = parent;
+            created.hash = hash;
+            created.token = token;
+            created.token_size = token_size;
+            nodes.push_back(created);
+            table[i].index = nodes.size() - 1;
+
+            // keep the load factor below 1/2
+            if (nodes.size() * 2 > table.size())
+            {
+                std::vector<bucket> old(table.size() * 2);
+                old.swap(table);
+                for (std::size_t k = 1; k < nodes.size(); ++k)
+                {
+                    std::size_t j = slot_of(nodes[k].parent, nodes[k].hash);
+                    while (table[j].index != 0)
+                    {
+                        j = (j + 1) & (table.size() - 1);
+                    }
+                    table[j].index = k;
+                }
+            }
+            return nodes.size() - 1;
+        }
+
+        std::uint64_t seed = 0;
+        /// @brief the nodes; the first is the root, the empty prefix
+        std::vector<node> nodes{};
+        /// @brief the nodes (not the root) in a hash table
+        std::vector<bucket> table{};
+        /// @brief the root, and the node after each token of the key that was walked last
+        std::vector<step> chain{};
+        const char* last_key = nullptr;
+        std::size_t last_key_size = 0;
+        bool any_marked = false;
+    };
+
+    /*!
+    @brief the pointer prefixes of a flattened object that denote arrays
+
+    @ref unflatten creates arrays exactly below the prefixes that have a
+    reference token 0 among their children, and objects everywhere else.
+    Deciding this up front keeps the result independent of the order in which
+    the flattened object is iterated, which is unspecified for some object
+    types.
+
+    A prefix is identified by the text of the key it is the beginning of:
+    escaping is unambiguous, so equal prefixes of valid keys are equal texts.
+    The set refers to the keys, which must outlive it. Short prefixes are
+    kept in a hash table and compared as texts, which costs at most the length
+    of the bound. Longer prefixes are kept in a @ref prefix_tree, because
+    hashing and comparing every one of the prefixes of a key with many tokens
+    from its beginning would take quadratic time.
+
+    The hash function depends on a seed that is derived from the address of a
+    local variable, which address space layout randomization changes between
+    processes, so keys cannot be crafted to collide in advance; the result
+    does not depend on the seed. Where the addresses are not randomized, or
+    when the seed is known, keys can collide: k short prefixes then cost
+    k^2 / 2 comparisons of fewer than 64 bytes each. The long prefixes are not
+    hashed as a whole.
+    */
+    class array_prefix_set
+    {
+      public:
+        /// @brief choose the seed of the hash function, before anything is added: it is derived from the address that is passed
+        void reseed(const void* address) noexcept
+        {
+            // NOLINTNEXTLINE(cppcoreguidelines-pro-type-reinterpret-cast): the address is only a source of entropy
+            std::uint64_t x = reinterpret_cast<std::uintptr_t>(address);
+            x ^= 0x9E3779B97F4A7C15ULL;
+            x *= 0xBF58476D1CE4E5B9ULL;
+            seed = x ^ (x >> 31u);
+            tree.reseed(seed);
+        }
+
+        /// @brief add the first @a prefix_size bytes of the key of @a key_size bytes at @a data
+        void insert(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            if (prefix_size >= short_limit)
+            {
+                tree.mark(data, key_size, prefix_size);
+                return;
+            }
+
+            // flattened objects are usually sorted, so most keys that have a
+            // 0 yield the prefix of the previous one
+            if (last_data != nullptr && last_size == prefix_size && std::memcmp(last_data, data, prefix_size) == 0)
+            {
+                return;
+            }
+            last_data = data;
+            last_size = prefix_size;
+
+            const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
+            if (table.empty())
+            {
+                table.resize(16);
+            }
+            const std::size_t mask = table.size() - 1;
+            std::size_t i = hash & mask;
+            while (table[i].data != nullptr)
+            {
+                if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
+                {
+                    return;
+                }
+                i = (i + 1) & mask;
+            }
+            table[i] = slot(hash, data, prefix_size);
+
+            // keep the load factor below 1/2
+            if (++used * 2 > table.size())
+            {
+                grow();
+            }
+        }
+
+        bool empty() const noexcept
+        {
+            return used == 0 && tree.empty();
+        }
+
+        /// @brief whether the first @a prefix_size bytes of the key of @a key_size bytes at @a data were added
+        bool contains(const char* data, std::size_t key_size, std::size_t prefix_size)
+        {
+            if (prefix_size >= short_limit)
+            {
+                return tree.is_marked(data, key_size, prefix_size);
+            }
+            if (used == 0)
+            {
+                return false;
+            }
+            const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
+            const std::size_t mask = table.size() - 1;
+            for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
+            {
+                if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
+                {
+                    return true;
+                }
+            }
+            return false;
+        }
+
+      private:
+        /// @brief prefixes of fewer bytes are kept in the hash table
+        static constexpr std::size_t short_limit = 64;
+
+        struct slot
+        {
+            slot() noexcept = default;
+
+            slot(std::size_t h, const char* d, std::size_t s) noexcept
+                : hash(h), data(d), size(s)
+            {}
+
+            std::size_t hash = 0;
+            const char* data = nullptr; // nullptr: the slot is empty
+            std::size_t size = 0;
+        };
+
+        void grow()
+        {
+            std::vector<slot> old(table.size() * 2);
+            old.swap(table);
+            const std::size_t mask = table.size() - 1;
+            for (const auto& s : old)
+            {
+                if (s.data != nullptr)
+                {
+                    std::size_t i = s.hash & mask;
+                    while (table[i].data != nullptr)
+                    {
+                        i = (i + 1) & mask;
+                    }
+                    table[i] = s;
+                }
+            }
+        }
+
+        std::uint64_t seed = 0;
+        prefix_tree tree{};
+        std::vector<slot> table{};
+        std::size_t used = 0;
+        const char* last_data = nullptr;
+        std::size_t last_size = 0;
+    };
+
     /*!
     @brief check that @a key is a valid JSON pointer, and add the prefixes of
            it that denote arrays to @a arrays
@@ -27227,7 +27443,7 @@ class json_pointer
                 // a token 0 makes the pointer before it an array
                 if (i - begin == 1 && data[begin] == '0')
                 {
-                    arrays.insert(data, begin - 1);
+                    arrays.insert(data, size, begin - 1);
                 }
                 begin = i + 1;
             }
@@ -28237,6 +28453,7 @@ class json_pointer
         // token 0 among their children; the values below them are arrays,
         // all others are objects (see array_prefix_set)
         array_prefix_set arrays;
+        arrays.reseed(&arrays);
         const string_t* previous_key = nullptr;
         for (const auto& element : *value.m_data.m_value.object)
         {
@@ -28328,7 +28545,7 @@ class json_pointer
                     case detail::value_t::null:
                     {
                         // a first child 0 makes the value an array without a look-up
-                        if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, pos))
+                        if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, key.size(), pos))
                         {
                             // some reference token below this position is 0, so the
                             // value is an array

tests/src/unit-json_pointer.cpp

@@ -23,8 +23,10 @@ using nlohmann::json;
     using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
 #endif
 
+#include <algorithm>
 #include <cerrno>
 #include <cstdlib>
+#include <ctime>
 #include <limits>
 #include <map>
 #include <random>
@@ -1560,3 +1562,294 @@ TEST_CASE("JSON patch with moves and copies")
     CHECK_THROWS_AS(inplace.patch_inplace(json::parse(R"([{"op":"move","from":"","path":""}])")), json::out_of_range&);
     CHECK(inplace == doc);
 }
+
+namespace
+{
+// the keys of a shape of flattened objects with long keys; `zero` is the
+// token that is repeated `m` times
+// 0: one key, 1: one `zero` followed by tokens 1, 2: the token 1 followed by
+// tokens `zero`, 3 and 4: two keys with a long prefix, apart from other keys,
+// 5: the same for two long branches, 6: more of the keys of 3
+std::vector<std::string> shape_keys(const int shape, const std::size_t m, const std::string& zero)
+{
+    std::string repeated;
+    for (std::size_t i = 0; i < m; ++i)
+    {
+        repeated += "/" + zero;
+    }
+    std::vector<std::string> keys;
+    switch (shape)
+    {
+        case 0:
+            keys.push_back(repeated);
+            break;
+        case 1:
+        {
+            std::string key = "/" + zero;
+            for (std::size_t i = 1; i < m; ++i)
+            {
+                key += "/1";
+            }
+            keys.push_back(key);
+            break;
+        }
+        case 2:
+        {
+            std::string key = "/1";
+            for (std::size_t i = 1; i < m; ++i)
+            {
+                key += "/" + zero;
+            }
+            keys.push_back(key);
+            break;
+        }
+        case 3:
+        case 4:
+            keys = {repeated + "/a0", "/10", repeated + "/a1", "/11"};
+            break;
+        case 5:
+            keys = {"/x" + repeated + "/a0", "/y" + repeated + "/a0", "/x" + repeated + "/a1", "/y" + repeated + "/a1"};
+            break;
+        case 6:
+            for (int j = 0; j < 4; ++j)
+            {
+                keys.push_back(repeated + "/a" + std::to_string(j));
+                keys.push_back("/1" + std::to_string(j));
+            }
+            break;
+        default:
+            break;
+    }
+    return keys;
+}
+
+// the best of `runs` times in seconds that std::clock() measures to unflatten the keys,
+// which an ordered_json holds in the order they are given (shape 4 uses a
+// json, which sorts them)
+double unflatten_cpu_time(const int shape, const std::size_t m, const int runs, const std::string& zero)
+{
+    const std::vector<std::string> keys = shape_keys(shape, m, zero);
+    nlohmann::ordered_json ordered = nlohmann::ordered_json::object();
+    json sorted = json::object();
+    for (const auto& key : keys)
+    {
+        ordered[key] = 1;
+        sorted[key] = 1;
+    }
+
+    double best = 1e30;
+    for (int i = 0; i < runs; ++i)
+    {
+        const std::clock_t start = std::clock();
+        if (shape == 4)
+        {
+            const json nested = sorted.unflatten();
+        }
+        else
+        {
+            const nlohmann::ordered_json nested = ordered.unflatten();
+        }
+        best = (std::min)(best, static_cast<double>(std::clock() - start) / static_cast<double>(CLOCKS_PER_SEC));
+    }
+    return best;
+}
+} // namespace
+
+TEST_CASE("unflatten of keys with many reference tokens")
+{
+    using nlohmann::ordered_json;
+
+    SECTION("which prefixes become arrays")
+    {
+        // sorted keys that share a long beginning
+        CHECK(json({{"/x/y/q/1", 5}, {"/x/y/w/0", 4}, {"/x/y/w/1", 3}, {"/x/y/z/0", 1}, {"/x/y/z/1", 2}}).unflatten() ==
+        json::parse(R"({"x":{"y":{"q":{"1":5},"w":[4,3],"z":[1,2]}}})"));
+
+        // prefixes that differ in the position of a slash only
+        CHECK(json({{"/a/0/x", 1}, {"/a/1/y", 3}, {"/ab/1", 2}}).unflatten() ==
+        json::parse(R"({"a":[{"x":1},{"y":3}],"ab":{"1":2}})"));
+        CHECK(json({{"/a/b/1", 1}, {"/ab/0", 2}}).unflatten() ==
+        json::parse(R"({"a":{"b":{"1":1}},"ab":[2]})"));
+
+        // tokens longer than one word, escapes, and a token 0 below a token 1
+        const std::string t = "token_with_more_than_eight_bytes";
+        CHECK(json({{"/" + t + "/1/b/0", 1}, {"/" + t + "/0", 2}, {"/" + t + "/2/a~1b", 3}}).unflatten() ==
+        json::parse(R"({"token_with_more_than_eight_bytes":[2,{"b":[1]},{"a/b":3}]})"));
+
+        // prefixes of more than 64 bytes, that share a long beginning
+        std::string deep;
+        json expected = {{"a", {1, 2}}, {"b", {{"1", 3}}}, {"c", {{"1", {{"x", {4}}}}}}};
+        for (int i = 0; i < 30; ++i)
+        {
+            deep += "/seg";
+            expected = {{"seg", expected}};
+        }
+        CHECK(json({{deep + "/c/1/x/0", 4}, {deep + "/a/1", 2}, {deep + "/b/1", 3}, {deep + "/a/0", 1}}).unflatten() == expected);
+        CHECK(json(ordered_json({{deep + "/c/1/x/0", 4}, {deep + "/a/1", 2}, {deep + "/b/1", 3}, {deep + "/a/0", 1}}).unflatten()) == expected);
+        CHECK(json(ordered_json({{deep + "/a/1", 2}, {deep + "/a/0", 1}, {deep + "/b/1", 3}, {deep + "/c/1/x/0", 4}}).unflatten()) == expected);
+
+        // prefixes of every length around the limit up to which they are hashed directly
+        for (std::size_t length = 40; length < 90; ++length)
+        {
+            const std::string name(length, 'x');
+            const json flat = {{"/" + name + "/a/0", 1}, {"/" + name + "/a/1", 2}, {"/" + name + "/b/1", 3}, {"/" + name + "/c/1/0", 4}};
+            CHECK(flat.unflatten() == json({{name, {{"a", {1, 2}}, {"b", {{"1", 3}}}, {"c", {{"1", {4}}}}}}}));
+        }
+
+        // only the parent of a token 0 is an array, however deep the key is
+        CHECK(json({{"/0/1/1/1", 1}}).unflatten() == json::parse(R"([{"1":{"1":{"1":1}}}])"));
+        CHECK(json({{"/1/1/1/0", 1}}).unflatten() == json::parse(R"({"1":{"1":{"1":[1]}}})"));
+        CHECK(json({{"/1/0/1/0", 1}}).unflatten() == json::parse(R"({"1":[{"1":[1]}]})"));
+    }
+
+    SECTION("the result does not depend on the order of the keys")
+    {
+        // random documents without empty containers and without the key "0" in
+        // an object: flatten() and unflatten() are inverse to each other
+        std::mt19937 gen(20261110); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
+        const std::vector<std::string> names = {"a", "b", "ab", "1", "2", "10", "a/b", "a~b", "token_with_more_than_eight_bytes", "x"};
+
+        struct builder
+        {
+            std::mt19937& gen;
+            const std::vector<std::string>& names;
+
+            json operator()(int depth) const
+            {
+                const int kind = std::uniform_int_distribution<int>(0, 9)(gen);
+                if (depth == 0 || kind < 3)
+                {
+                    return kind;
+                }
+                json result = kind < 6 ? json::array() : json::object();
+                const int size = std::uniform_int_distribution<int>(1, 4)(gen);
+                for (int i = 0; i < size; ++i)
+                {
+                    if (kind < 6)
+                    {
+                        result.push_back((*this)(depth - 1));
+                    }
+                    else
+                    {
+                        result[names[std::uniform_int_distribution<std::size_t>(0, names.size() - 1)(gen)]] = (*this)(depth - 1);
+                    }
+                }
+                return result;
+            }
+        };
+        const builder build{gen, names};
+
+        for (int round = 0; round < 200; ++round)
+        {
+            const json doc = build(6);
+            const json flat = doc.flatten();
+            CHECK(flat.unflatten() == doc);
+
+            // the same entries in shuffled order, in an object that keeps it
+            std::vector<std::pair<std::string, json>> entries;
+            for (const auto& el : flat.items())
+            {
+                entries.emplace_back(el.key(), el.value());
+            }
+            std::shuffle(entries.begin(), entries.end(), gen);
+            ordered_json shuffled = ordered_json::object();
+            for (const auto& e : entries)
+            {
+                shuffled[e.first] = e.second;
+            }
+            CHECK(json(shuffled.unflatten()) == doc);
+        }
+    }
+
+    SECTION("keys with long prefixes, in every order")
+    {
+        // prefixes of more than 64 bytes that share a long beginning, and
+        // continue with tokens that start the same way
+        std::string deep;
+        for (int i = 0; i < 30; ++i)
+        {
+            deep += "/seg";
+        }
+        const std::string other = deep.substr(0, 70) + "/x" + deep.substr(70);
+        std::vector<std::string> keys = {deep + "/a/0", deep + "/ab/1", deep + "/ab/x", deep + "/abc/0/y/0", other + "/b/1/0", deep + "/abc/1"};
+        std::sort(keys.begin(), keys.end());
+
+        // the result for the keys in sorted order, which json iterates in
+        json sorted = json::object();
+        for (const auto& key : keys)
+        {
+            sorted[key] = 1;
+        }
+        const json expected = sorted.unflatten();
+        CHECK(expected.is_object());
+
+        // ordered_json iterates in the order the keys were added
+        std::vector<std::size_t> order(keys.size());
+        for (std::size_t i = 0; i < order.size(); ++i)
+        {
+            order[i] = i;
+        }
+        std::size_t permutations = 0;
+        bool all_equal = true;
+        do
+        {
+            nlohmann::ordered_json flat = nlohmann::ordered_json::object();
+            for (const auto i : order)
+            {
+                flat[keys[i]] = 1;
+            }
+            all_equal = all_equal && json(flat.unflatten()) == expected;
+            ++permutations;
+        }
+        while (std::next_permutation(order.begin(), order.end()));
+        CHECK(permutations == 720);
+        CHECK(all_equal);
+    }
+
+    SECTION("time is linear in the number of tokens")
+    {
+        // A bound in seconds would depend on the machine. Instead, the time is
+        // compared with the time for the same keys with the token 0 replaced by
+        // a token that is not a number: there is no array, so no prefix is
+        // looked up. The ratio is 1 to 3 for a linear algorithm, and between
+        // 12 and 100 when looking up a prefix takes time proportional to its
+        // length. The time that std::clock() measures is mostly the time of the
+        // processor, which a busy machine does not inflate (on MSVC, it is the
+        // time that has passed), and the best of two or three runs counts.
+        for (int shape = 0; shape < 7; ++shape)
+        {
+            // the shapes with several keys, in an order that does not sort them,
+            // need more tokens to be slow enough to tell
+            const bool many = shape == 3 || shape == 5 || shape == 6;
+            const std::size_t m = many ? 40000 : 10000;
+            const int runs = many ? 2 : 3;
+
+            // a busy machine may spoil one comparison, hardly two
+            bool linear = false;
+            double t_array = 0;
+            double t_object = 0;
+            for (int attempt = 0; attempt < 2 && !linear; ++attempt)
+            {
+                t_array = unflatten_cpu_time(shape, m, runs, "0");
+                t_object = (std::max)(unflatten_cpu_time(shape, m, runs, "a"), 1e-3);
+                linear = t_array < 6 * t_object;
+            }
+            CAPTURE(shape)
+            CAPTURE(t_array)
+            CAPTURE(t_object)
+            CHECK(linear);
+        }
+
+        // the nesting is as expected: an array exactly below each token 0
+        const json nested_deep = json::object({{shape_keys(0, 1000, "0")[0], 1}}).unflatten();
+        std::size_t depth = 0;
+        for (const json* p = &nested_deep; p->is_array(); p = &p->back())
+        {
+            ++depth;
+        }
+        CHECK(depth == 1000);
+        const json nested_ones = json::object({{shape_keys(1, 1000, "0")[0], 1}}).unflatten();
+        CHECK(nested_ones.is_array());
+        CHECK(nested_ones.back().is_object());
+    }
+}