Flatten deeply nested values without recursing per nesting level

json_pointer::flatten() called itself once per nesting level, so
flatten() on a value nested deeply enough exhausted the call stack.
#5547 and #5548 fixed merge_patch() and diff() from #5393, but flatten()
was left out.

flatten() now walks the value with an explicit stack and keeps the path
in one buffer that grows and shrinks with it. It has a single code path
and no depth limit: the old version built a new path string per child,
so the iterative one is no slower on shallow values and much faster on
deep ones. The output, including the order of an ordered_json result,
is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/macros/json_use_implicit_conversions.md

@@ -67,7 +67,7 @@ By default, implicit conversions are enabled.
     `JSON_USE_IMPLICIT_CONVERSIONS` is defined to `0`:
 
     ```cpp
-    using wjson = nlohmann::basic_json<std::map, std::vector, std::wstring>;
+    using wjson = nlohmann::json::with_string_t<std::wstring>;
 
     void load(const nlohmann::json& j);
 

docs/mkdocs/docs/community/roadmap.md

@@ -36,13 +36,26 @@ work items are tracked in the [GitHub milestones](https://github.com/nlohmann/js
 ## API stability
 
 Releases follow [semantic versioning](https://semver.org): a minor or patch release of version 3.x does not break code
-that uses the public API. In particular, a 3.x release does not:
+that uses the public API, unless that code opts in to a change with a macro as described [below](#version-40). In
+particular, a 3.x release does not:
 
-- change the signature of a function (its parameter types, return type, number of parameters, or the const-ness of a
+- make breaking changes to the signature of a function: the types or order of its existing parameters, its return type,
-  member function);
+  its `noexcept` or `constexpr` specifier, or the const-ness of a member function. New parameters may be added if they
-- remove or rename a function or class;
+  have a default value;
+- remove or rename a function or class, or change the template parameters of a public class template;
 - change which exceptions a function throws, or the [exception ids](../home/exceptions.md);
-- change access specifiers or default arguments.
+- change access specifiers, or change or remove existing default arguments. New default arguments may be added;
+- change the JSON type that a valid input parses to, or the text that `dump()` produces for a valid value;
+- accept input that was rejected before, or reject input that was accepted before;
+- change the order in which the keys of an object are iterated. The default type sorts keys, and
+  [`ordered_json`](../api/ordered_json.md) keeps insertion order;
+- change when iterators, pointers, or references are invalidated, or the state of a moved-from `basic_json`;
+- add or remove implicit conversions from `basic_json`;
+- change how `to_json` and `from_json` functions are found, or the behavior of
+  [`adl_serializer`](../api/adl_serializer/index.md);
+- add pure virtual functions to the [`json_sax`](../api/json_sax/index.md) interface;
+- remove, rename, renumber, or add enumerators of `value_t`;
+- remove or rename a documented macro, CMake option, CMake target, or header, or change what a documented macro does.
 
 Exceptions to these rules, for instance when fixing a bug requires changing the exception a function throws, are
 documented in the [release notes](../home/releases.md).
@@ -51,13 +64,14 @@ The following are **not** part of the public API and may change in any release,
 
 - The text of exception messages returned by `what()`. Use the [exception id](../home/exceptions.md) to tell errors
   apart.
-- The ABI, including `sizeof(basic_json)` and the memory layout of its values. Recompile your code when you upgrade the
+- The ABI, including `sizeof(basic_json)` and the memory layout of its values. The
-  library. The [versioned inline namespace](../features/namespace.md) turns mixing versions into a link error.
+  [versioned inline namespace](../features/namespace.md) turns mixing versions into a link error.
+- The hash values returned by `std::hash` for `basic_json`. Numbers that compare equal still hash equally.
 - Everything in namespace `nlohmann::detail`, and macros and type traits that are not documented in the
   [API reference](../api/basic_json/index.md).
 
-Changes that would break the public API are only added behind a macro whose default keeps the 3.x behavior, see
+Breaking changes are only added behind a macro whose default keeps the 3.x behavior. See [Version 4.0](#version-40) and
-[Version 4.0](#version-40).
+the [macro overview](../features/macros.md).
 
 ## Version 4.0
 

docs/mkdocs/docs/examples/contains__json_pointer.cpp

@@ -19,25 +19,9 @@ int main()
               << j.contains("/array/1"_json_pointer) << '\n'
               << j.contains("/array/-"_json_pointer) << '\n'
               << j.contains("/array/4"_json_pointer) << '\n'
-              << j.contains("/baz"_json_pointer) << std::endl;
+              << j.contains("/baz"_json_pointer) << '\n'
-
+              // an array index with a leading '0' is not found
-    try
+              << j.contains("/array/01"_json_pointer) << '\n'
-    {
+              // an array index that is not a number is not found
-        // try to use an array index with leading '0'
+              << j.contains("/array/one"_json_pointer) << std::endl;
-        j.contains("/array/01"_json_pointer);
-    }
-    catch (const json::parse_error& e)
-    {
-        std::cout << e.what() << '\n';
-    }
-
-    try
-    {
-        // try to use an array index that is not a number
-        j.contains("/array/one"_json_pointer);
-    }
-    catch (const json::parse_error& e)
-    {
-        std::cout << e.what() << '\n';
-    }
 }

docs/mkdocs/docs/examples/contains__json_pointer.output

@@ -5,3 +5,5 @@ true
 false
 false
 false
+false
+false

docs/mkdocs/docs/examples/custom_array_type.cpp

@@ -1,11 +1,10 @@
 #include <iostream>
-#include <map>
 
 #include <nlohmann/json.hpp>
 
 #include "custom_array_type.hpp"
 
-using custom_json = nlohmann::basic_json<std::map, custom_array_type>;
+using custom_json = nlohmann::json::with_array_t<custom_array_type>;
 
 int main()
 {

docs/mkdocs/docs/examples/custom_binary_type.cpp

@@ -1,16 +1,10 @@
-#include <cstdint>
 #include <iostream>
-#include <map>
-#include <string>
-#include <vector>
 
 #include <nlohmann/json.hpp>
 
 #include "custom_binary_type.hpp"
 
-using custom_json = nlohmann::basic_json<std::map, std::vector, std::string, bool,
+using custom_json = nlohmann::json::with_binary_t<custom_binary_type>;
-      std::int64_t, std::uint64_t, double, std::allocator,
-      nlohmann::adl_serializer, custom_binary_type>;
 
 int main()
 {

docs/mkdocs/docs/examples/custom_object_type.cpp

@@ -1,12 +1,11 @@
 #include <iostream>
 #include <type_traits>
-#include <vector>
 
 #include <nlohmann/json.hpp>
 
 #include "custom_object_type.hpp"
 
-using custom_json = nlohmann::basic_json<custom_object_type, std::vector>;
+using custom_json = nlohmann::json::with_object_t<custom_object_type>;
 
 int main()
 {

docs/mkdocs/docs/examples/custom_string_type.cpp

@@ -1,12 +1,10 @@
 #include <iostream>
-#include <map>
-#include <vector>
 
 #include <nlohmann/json.hpp>
 
 #include "custom_string_type.hpp"
 
-using custom_json = nlohmann::basic_json<std::map, std::vector, custom_string_type>;
+using custom_json = nlohmann::json::with_string_t<custom_string_type>;
 
 int main()
 {

docs/mkdocs/docs/examples/parse_error.cpp

@@ -8,7 +8,7 @@ int main()
     try
     {
         // parsing input with a syntax error
-        json::parse("[1,2,3,]");
+        json j = json::parse("[1,2,3,]");
     }
     catch (const json::parse_error& e)
     {

docs/mkdocs/docs/features/types/number_handling.md

@@ -351,9 +351,8 @@ The number types can be changed with template parameters.
 
     A `basic_json` type that uses `#!c long double` as floating-point type.
 
-    ```cpp hl_lines="2"
+    ```cpp hl_lines="1"
-    using json_ld = nlohmann::basic_json<std::map, std::vector, std::string, bool,
+    using json_ld = nlohmann::json::with_float_t<long double>;
-                                         std::int64_t, std::uint64_t, long double>;
     ```
 
     Note values should then be parsed with `json_ld::parse` rather than `json::parse` as the latter would parse

docs/mkdocs/docs/features/types/template_parameters.md

@@ -8,6 +8,10 @@ these requirements so they do not have to be discovered by trial and error. Each
 that are known to work for that parameter and the ones that do not, checked against Boost 1.83, Abseil 20250127.0,
 Folly, EASTL 3.21, `ankerl::unordered_dense`, `phmap`, `gtl`, `robin_hood`, `tsl::ordered_map`, and Qt 6.
 
+To change a single template parameter and keep the others, use the member alias templates
+[`with_*_t`](../../api/basic_json/with_t.md); for instance, `nlohmann::json::with_float_t<long double>` is `json` with
+`#!cpp long double` as [`number_float_t`](../../api/basic_json/number_float_t.md).
+
 ## How to read this page
 
 Requirements are split into two groups:
@@ -143,7 +147,7 @@ struct unordered_map_object
     using base_t::base_t;
 };
 
-using unordered_json = nlohmann::basic_json<unordered_map_object>;
+using unordered_json = nlohmann::json::with_object_t<unordered_map_object>;
 ```
 
 Whether `#!cpp std::unordered_map` can be instantiated at all depends on the standard library: `object_t` is formed
@@ -176,7 +180,7 @@ struct flat_hash_object
     using base_t::base_t;
 };
 
-using flat_hash_json = nlohmann::basic_json<flat_hash_object>;
+using flat_hash_json = nlohmann::json::with_object_t<flat_hash_object>;
 ```
 
 `absl::node_hash_map` keeps references to the mapped values valid across insertions; `absl::flat_hash_map` does not,

docs/mkdocs/docs/integration/migration_guide.md

@@ -116,7 +116,7 @@ function to use instead.
     === "Deprecated"
   
           ```cpp
-          using my_json = nlohmann::basic_json<std::map, std::vector, my_string_type>;
+          using my_json = nlohmann::json::with_string_t<my_string_type>;
           nlohmann::json_pointer<my_json> ptr("/foo/bar/1");
           ```
   

include/nlohmann/detail/bit_ops.hpp

@@ -9,12 +9,15 @@
 #pragma once
 
 #include <cstdint> // uint64_t
+#if !defined(__SIZEOF_INT128__) && defined(_MSC_VER) && (defined(_M_X64) || defined(_M_ARM64))
+    #include <intrin0.h> // __umulh, _umul128
+#endif
 
 #include <nlohmann/detail/abi_macros.hpp>
 
 // Portable bit-level helpers for the number and string scanners. They use
-// compiler builtins where available and plain C++ otherwise, so they need no
+// compiler builtins or platform-specific intrinsics where available and plain
-// platform headers and work regardless of byte order.
+// C++ otherwise, so they work regardless of byte order.
 
 NLOHMANN_JSON_NAMESPACE_BEGIN
 namespace detail
@@ -52,6 +55,12 @@ inline uint128_parts full_multiplication(std::uint64_t a, std::uint64_t b) noexc
     __extension__ using uint128 = unsigned __int128;
     const uint128 r = static_cast<uint128>(a) * b;
     return {static_cast<std::uint64_t>(r), static_cast<std::uint64_t>(r >> 64u)};
+#elif defined(_MSC_VER) && defined(_M_X64)
+    std::uint64_t high = 0;
+    const std::uint64_t low = _umul128(a, b, &high);
+    return {low, high};
+#elif defined(_MSC_VER) && defined(_M_ARM64)
+    return {a * b, __umulh(a, b)};
 #else
     const std::uint64_t a_lo = a & 0xFFFFFFFFu;
     const std::uint64_t a_hi = a >> 32u;

include/nlohmann/detail/json_pointer.hpp

@@ -878,49 +878,70 @@ class json_pointer
     @param[in,out] result        the result object to insert values to
 
     @note Empty objects or arrays are flattened to `null`.
+
+    The value is walked with an explicit stack rather than the call stack, so
+    arbitrarily deeply nested values can be flattened.
+
+    @sa https://github.com/nlohmann/json/issues/5393
     */
     template<typename BasicJsonType>
     static void flatten(const string_t& reference_string,
                         const BasicJsonType& value,
                         BasicJsonType& result)
     {
-        switch (value.type())
+        using object_const_iterator = typename BasicJsonType::object_t::const_iterator;
+
+        // an array or object being walked: the container, the array index or
+        // object iterator of the next child, and the length of the path of the
+        // container itself
+        struct frame
+        {
+            const BasicJsonType* container;
+            std::size_t index;
+            object_const_iterator member;
+            std::size_t path_length;
+        };
+
+        // The containers being flattened are kept on an explicit stack, and
+        // every child is flattened completely before the next one, so the
+        // entries come out in the same order as with a recursive walk. The
+        // path of the value being flattened is kept in one buffer that grows
+        // and shrinks with the stack, rather than in a new string per level.
+        std::vector<frame> stack;
+        string_t path = reference_string;
+
+        // flatten `v`, whose path is `path`: primitives and empty containers
+        // are added to the result right away; other containers get a frame
+        const auto enter = [&stack, &path, &result](const BasicJsonType & v)
+        {
+            switch (v.type())
             {
                 case detail::value_t::array:
                 {
-                if (value.m_data.m_value.array->empty())
+                    if (v.m_data.m_value.array->empty())
                     {
                         // flatten empty array as null
-                    result[reference_string] = nullptr;
+                        result[path] = nullptr;
                     }
                     else
                     {
-                    // iterate array and use index as a reference string
+                        stack.push_back({&v, 0, object_const_iterator(), path.size()});
-                    for (std::size_t i = 0; i < value.m_data.m_value.array->size(); ++i)
-                    {
-                        flatten(detail::concat<string_t>(reference_string, '/', std::to_string(i)),
-                                value.m_data.m_value.array->operator[](i), result);
                     }
-                }
+                    return;
-                break;
                 }
 
                 case detail::value_t::object:
                 {
-                if (value.m_data.m_value.object->empty())
+                    if (v.m_data.m_value.object->empty())
                     {
                         // flatten empty object as null
-                    result[reference_string] = nullptr;
+                        result[path] = nullptr;
                     }
                     else
                     {
-                    // iterate object and use keys as reference string
+                        stack.push_back({&v, 0, v.m_data.m_value.object->begin(), path.size()});
-                    for (const auto& element : *value.m_data.m_value.object)
-                    {
-                        flatten(detail::concat<string_t>(reference_string, '/', detail::escape(element.first)), element.second, result);
                     }
-                }
+                    return;
-                break;
                 }
 
                 case detail::value_t::null:
@@ -934,8 +955,50 @@ class json_pointer
                 default:
                 {
                     // add a primitive value with its reference string
-                result[reference_string] = value;
+                    result[path] = v;
-                break;
+                    return;
+                }
+            }
+        };
+
+        enter(value);
+        while (!stack.empty())
+        {
+            // the frame is changed through stack.back(): enter() may push a
+            // frame, which would invalidate a reference to it
+            const BasicJsonType* const container = stack.back().container;
+
+            // drop the path of the previous child
+            path.resize(stack.back().path_length);
+
+            if (container->is_array())
+            {
+                const auto& array = *container->m_data.m_value.array;
+                const std::size_t i = stack.back().index;
+                if (i == array.size())
+                {
+                    stack.pop_back();
+                    continue;
+                }
+
+                // iterate array and use index as a reference string
+                ++stack.back().index;
+                detail::concat_into(path, '/', detail::to_string<string_t>(i));
+                enter(array[i]);
+            }
+            else
+            {
+                const object_const_iterator it = stack.back().member;
+                if (it == container->m_data.m_value.object->end())
+                {
+                    stack.pop_back();
+                    continue;
+                }
+
+                // iterate object and use keys as reference string
+                ++stack.back().member;
+                detail::concat_into(path, '/', detail::escape(it->first));
+                enter(it->second);
             }
         }
     }

single_include/nlohmann/json.hpp

@@ -8791,13 +8791,16 @@ NLOHMANN_JSON_NAMESPACE_END
 
 
 #include <cstdint> // uint64_t
+#if !defined(__SIZEOF_INT128__) && defined(_MSC_VER) && (defined(_M_X64) || defined(_M_ARM64))
+    #include <intrin0.h> // __umulh, _umul128
+#endif
 
 // #include <nlohmann/detail/abi_macros.hpp>
 
 
 // Portable bit-level helpers for the number and string scanners. They use
-// compiler builtins where available and plain C++ otherwise, so they need no
+// compiler builtins or platform-specific intrinsics where available and plain
-// platform headers and work regardless of byte order.
+// C++ otherwise, so they work regardless of byte order.
 
 NLOHMANN_JSON_NAMESPACE_BEGIN
 namespace detail
@@ -8835,6 +8838,12 @@ inline uint128_parts full_multiplication(std::uint64_t a, std::uint64_t b) noexc
     __extension__ using uint128 = unsigned __int128;
     const uint128 r = static_cast<uint128>(a) * b;
     return {static_cast<std::uint64_t>(r), static_cast<std::uint64_t>(r >> 64u)};
+#elif defined(_MSC_VER) && defined(_M_X64)
+    std::uint64_t high = 0;
+    const std::uint64_t low = _umul128(a, b, &high);
+    return {low, high};
+#elif defined(_MSC_VER) && defined(_M_ARM64)
+    return {a * b, __umulh(a, b)};
 #else
     const std::uint64_t a_lo = a & 0xFFFFFFFFu;
     const std::uint64_t a_hi = a >> 32u;
@@ -20787,49 +20796,70 @@ class json_pointer
     @param[in,out] result        the result object to insert values to
 
     @note Empty objects or arrays are flattened to `null`.
+
+    The value is walked with an explicit stack rather than the call stack, so
+    arbitrarily deeply nested values can be flattened.
+
+    @sa https://github.com/nlohmann/json/issues/5393
     */
     template<typename BasicJsonType>
     static void flatten(const string_t& reference_string,
                         const BasicJsonType& value,
                         BasicJsonType& result)
     {
-        switch (value.type())
+        using object_const_iterator = typename BasicJsonType::object_t::const_iterator;
+
+        // an array or object being walked: the container, the array index or
+        // object iterator of the next child, and the length of the path of the
+        // container itself
+        struct frame
+        {
+            const BasicJsonType* container;
+            std::size_t index;
+            object_const_iterator member;
+            std::size_t path_length;
+        };
+
+        // The containers being flattened are kept on an explicit stack, and
+        // every child is flattened completely before the next one, so the
+        // entries come out in the same order as with a recursive walk. The
+        // path of the value being flattened is kept in one buffer that grows
+        // and shrinks with the stack, rather than in a new string per level.
+        std::vector<frame> stack;
+        string_t path = reference_string;
+
+        // flatten `v`, whose path is `path`: primitives and empty containers
+        // are added to the result right away; other containers get a frame
+        const auto enter = [&stack, &path, &result](const BasicJsonType & v)
+        {
+            switch (v.type())
             {
                 case detail::value_t::array:
                 {
-                if (value.m_data.m_value.array->empty())
+                    if (v.m_data.m_value.array->empty())
                     {
                         // flatten empty array as null
-                    result[reference_string] = nullptr;
+                        result[path] = nullptr;
                     }
                     else
                     {
-                    // iterate array and use index as a reference string
+                        stack.push_back({&v, 0, object_const_iterator(), path.size()});
-                    for (std::size_t i = 0; i < value.m_data.m_value.array->size(); ++i)
-                    {
-                        flatten(detail::concat<string_t>(reference_string, '/', std::to_string(i)),
-                                value.m_data.m_value.array->operator[](i), result);
                     }
-                }
+                    return;
-                break;
                 }
 
                 case detail::value_t::object:
                 {
-                if (value.m_data.m_value.object->empty())
+                    if (v.m_data.m_value.object->empty())
                     {
                         // flatten empty object as null
-                    result[reference_string] = nullptr;
+                        result[path] = nullptr;
                     }
                     else
                     {
-                    // iterate object and use keys as reference string
+                        stack.push_back({&v, 0, v.m_data.m_value.object->begin(), path.size()});
-                    for (const auto& element : *value.m_data.m_value.object)
-                    {
-                        flatten(detail::concat<string_t>(reference_string, '/', detail::escape(element.first)), element.second, result);
                     }
-                }
+                    return;
-                break;
                 }
 
                 case detail::value_t::null:
@@ -20843,8 +20873,50 @@ class json_pointer
                 default:
                 {
                     // add a primitive value with its reference string
-                result[reference_string] = value;
+                    result[path] = v;
-                break;
+                    return;
+                }
+            }
+        };
+
+        enter(value);
+        while (!stack.empty())
+        {
+            // the frame is changed through stack.back(): enter() may push a
+            // frame, which would invalidate a reference to it
+            const BasicJsonType* const container = stack.back().container;
+
+            // drop the path of the previous child
+            path.resize(stack.back().path_length);
+
+            if (container->is_array())
+            {
+                const auto& array = *container->m_data.m_value.array;
+                const std::size_t i = stack.back().index;
+                if (i == array.size())
+                {
+                    stack.pop_back();
+                    continue;
+                }
+
+                // iterate array and use index as a reference string
+                ++stack.back().index;
+                detail::concat_into(path, '/', detail::to_string<string_t>(i));
+                enter(array[i]);
+            }
+            else
+            {
+                const object_const_iterator it = stack.back().member;
+                if (it == container->m_data.m_value.object->end())
+                {
+                    stack.pop_back();
+                    continue;
+                }
+
+                // iterate object and use keys as reference string
+                ++stack.back().member;
+                detail::concat_into(path, '/', detail::escape(it->first));
+                enter(it->second);
             }
         }
     }

tests/CMakeLists.txt

@@ -138,7 +138,8 @@ json_test_set_test_options(test-disabled_exceptions
 # only the #972 regression test needs thirdparty/fifo_map on its include path
 json_test_set_test_options(test-regression1 LINK_LIBRARIES fifo_map_include)
 
-# GCC's false -Warray-bounds error with JSON_DIAGNOSTICS only shows up when optimizing (#5742).
+# Regression test for GCC's false -Warray-bounds error with JSON_DIAGNOSTICS (#5742, fixed in #5585). It only
+# showed up when optimizing, so build this test with -O3 and the warning as an error.
 # -O3 makes the optimizer-driven warnings of the ci_test_gcc flag set (-Winline,
 # -Wsuggest-attribute=...) fire on the library's inline functions; they are not
 # what this test checks, so turn them off for it.

tests/src/unit-allocator.cpp

@@ -370,14 +370,7 @@ TEST_CASE("copy of a deeply nested value survives a failing allocation (#5640)")
 #if !(defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
     SECTION("std::map-backed object_t")
     {
-        using bad_alloc_json = nlohmann::basic_json<std::map,
+        using bad_alloc_json = nlohmann::json::with_allocator_t<nth_alloc_fails_allocator>;
-              std::vector,
-              std::string,
-              bool,
-              std::int64_t,
-              std::uint64_t,
-              double,
-              nth_alloc_fails_allocator>;
 
         check_deep_copy_survives_failing_allocation<bad_alloc_json>(false);
         check_deep_copy_survives_failing_allocation<bad_alloc_json>(true);
@@ -385,14 +378,7 @@ TEST_CASE("copy of a deeply nested value survives a failing allocation (#5640)")
 
     SECTION("ordered_map-backed object_t")
     {
-        using bad_alloc_ordered_json = nlohmann::basic_json<nlohmann::ordered_map,
+        using bad_alloc_ordered_json = nlohmann::ordered_json::with_allocator_t<nth_alloc_fails_allocator>;
-              std::vector,
-              std::string,
-              bool,
-              std::int64_t,
-              std::uint64_t,
-              double,
-              nth_alloc_fails_allocator>;
 
         check_deep_copy_survives_failing_allocation<bad_alloc_ordered_json>(false);
         check_deep_copy_survives_failing_allocation<bad_alloc_ordered_json>(true);
@@ -450,14 +436,7 @@ struct scratch_counting_allocator : std::allocator<T>
 
 TEST_CASE("deep copy uses the provided allocator")
 {
-    using counting_json = nlohmann::basic_json<std::map,
+    using counting_json = nlohmann::json::with_allocator_t<scratch_counting_allocator>;
-          std::vector,
-          std::string,
-          bool,
-          std::int64_t,
-          std::uint64_t,
-          double,
-          scratch_counting_allocator>;
 
     // deeper than the 128 levels the copy constructor descends into, so the
     // innermost objects are copied by the iterative deep copy
@@ -516,14 +495,7 @@ TEST_CASE("converting a deeply nested value from another specialization fails cl
     // the allocator in noexcept constructors, so a failing construction crashes
     // the program there instead of throwing std::bad_alloc. Nothing to check.
 #if !(defined(_MSC_VER) && _MSC_VER < 1910 && defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
-    using countdown_json = nlohmann::basic_json<std::map,
+    using countdown_json = nlohmann::json::with_allocator_t<countdown_allocator>;
-          std::vector,
-          std::string,
-          bool,
-          std::int64_t,
-          std::uint64_t,
-          double,
-          countdown_allocator>;
 
     // deeper than the 128 levels the converting constructor descends into, so
     // that failures land on both sides of the bound - or, built with
@@ -631,14 +603,7 @@ TEST_CASE("destructor performs no allocation, only deallocation")
     // Since that stack could itself throw bad_alloc from inside the
     // noexcept destructor (#5135), destroy() no longer allocates anything:
     // it only ever frees what is already there.
-    using counting_json = nlohmann::basic_json<std::map,
+    using counting_json = nlohmann::json::with_allocator_t<counting_allocator>;
-          std::vector,
-          std::string,
-          bool,
-          std::int64_t,
-          std::uint64_t,
-          double,
-          counting_allocator>;
 
     SECTION("array")
     {
@@ -683,14 +648,7 @@ TEST_CASE("destructor performs no allocation, only deallocation")
 TEST_CASE("a failed allocation leaves the value unchanged")
 {
     // create JSON type using the throwing allocator
-    using my_json = nlohmann::basic_json<std::map,
+    using my_json = nlohmann::json::with_allocator_t<my_allocator>;
-          std::vector,
-          std::string,
-          bool,
-          std::int64_t,
-          std::uint64_t,
-          double,
-          my_allocator>;
 
     // Each of these creates a string, array, object, or binary value. The
     // value must be created before the type is changed: otherwise, a failed

tests/src/unit-binary_formats.cpp

@@ -237,7 +237,7 @@ namespace
 // the binary formats as function pointers for "Binary formats with narrow number types";
 // named functions rather than lambdas, because clang 3.5 cannot convert a lambda
 // to a function pointer in the braced initializer of the format table
-using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
+using narrow_json = nlohmann::json::with_integers_t<std::int32_t, std::uint32_t>::with_float_t<float>;
 using bytes = std::vector<std::uint8_t>;
 
 bytes encode_cbor(const json& j)

tests/src/unit-class_lexer.cpp

@@ -17,6 +17,7 @@ using nlohmann::json;
 #include <cstdint> // uint32_t, uint64_t
 #include <cstdlib> // strtod
 #include <cstring> // memcpy
+#include <limits> // numeric_limits
 #include <sstream> // stringstream
 #include <string> // string
 #include <utility> // pair
@@ -891,8 +892,39 @@ TEST_CASE("Eisel-Lemire float conversion")
 
     SECTION("128-bit products and leading zeros")
     {
+        const auto check_product = [](std::uint64_t a, std::uint64_t b)
+        {
+            const auto product = nlohmann::detail::full_multiplication(a, b);
+            CHECK(big_from(product.high, product.low) == big_mul(big_from(0, a), big_from(0, b)));
+        };
+
+        const std::uint64_t max = (std::numeric_limits<std::uint64_t>::max)();
+        const std::array<std::pair<std::uint64_t, std::uint64_t>, 13> edge_cases =
+        {
+            {
+                {0, 0},
+                {0, 1},
+                {1, 1},
+                {1, max},
+                {0xFFFFFFFFu, 0x100000000u},
+                {0x100000000u, 0x100000000u},
+                {0x100000001u, 0x100000001u},
+                {max, max},
+                {max, 2},
+                {0xFFFFFFFF00000000u, 0x100000001u},
+                {0x100000001u, 0xFFFFFFFF00000000u},
+                {max, 1},
+                {2, max},
+            }
+        };
+
+        for (const auto& test : edge_cases)
+        {
+            check_product(test.first, test.second);
+        }
+
         // whichever implementation the compiler gets (with or without a
-        // 128-bit integer type or a builtin)
+        // 128-bit integer type or a builtin / intrinsic)
         std::uint64_t state = 42;
         for (int i = 0; i < 10000; ++i)
         {
@@ -901,8 +933,7 @@ TEST_CASE("Eisel-Lemire float conversion")
             state ^= state << 17u;
             const std::uint64_t a = state;
             const std::uint64_t b = (state * 0x9E3779B97F4A7C15u) >> (i % 64);
-            const auto product = nlohmann::detail::full_multiplication(a, b);
+            check_product(a, b);
-            CHECK(big_from(product.high, product.low) == big_mul(big_from(0, a), big_from(0, b)));
 
             const int k = i % 64;
             const std::uint64_t x = (std::uint64_t{1} << k) | (a & ((std::uint64_t{1} << k) - 1));

tests/src/unit-comparison.cpp

@@ -824,7 +824,7 @@ struct unordered_object_t : std::map<Key, Value, directed_less<Key>, Allocator>
         return !(lhs == rhs);
     }
 };
-using unordered_json = nlohmann::basic_json<unordered_object_t>;
+using unordered_json = nlohmann::json::with_object_t<unordered_object_t>;
 
 // the entries "0" to "9", enumerated in ascending or in descending order
 unordered_json make_unordered_object(const bool descending)
@@ -875,7 +875,7 @@ struct key_case_less
 
 template<class Key, class Value, class /*Compare*/, class Allocator>
 using key_case_map = std::map<Key, Value, key_case_less, Allocator>;
-using key_case_json = nlohmann::basic_json<key_case_map>;
+using key_case_json = nlohmann::json::with_object_t<key_case_map>;
 
 // the innermost value of a chain of single-element arrays
 template<typename Json>
@@ -905,7 +905,7 @@ struct case_insensitive_less
 
 template<class Key, class Value, class /*Compare*/, class Allocator>
 using case_insensitive_map = std::map<Key, Value, case_insensitive_less, Allocator>;
-using ci_json = nlohmann::basic_json<case_insensitive_map>;
+using ci_json = nlohmann::json::with_object_t<case_insensitive_map>;
 } // namespace
 
 TEST_CASE("equality of objects whose entries have no fixed order")

tests/src/unit-custom-array-type.cpp

@@ -22,7 +22,7 @@ namespace
 
 // std::deque has no capacity() member function, which the library only needs
 // to detect a reallocation for JSON_DIAGNOSTICS
-using deque_json = nlohmann::basic_json<std::map, std::deque>;
+using deque_json = nlohmann::json::with_array_t<std::deque>;
 
 // a std::vector whose at() is hidden: the library performs its own bounds
 // check and must not fall back to the container's checked accessor
@@ -39,7 +39,7 @@ class vector_without_at : public std::vector<T, Allocator>
     void at() = delete;
 };
 
-using no_at_json = nlohmann::basic_json<std::map, vector_without_at>;
+using no_at_json = nlohmann::json::with_array_t<vector_without_at>;
 
 } // namespace
 

tests/src/unit-custom-object-type.cpp

@@ -179,7 +179,7 @@ class no_key_compare_map
     }
 };
 
-using no_key_compare_json = nlohmann::basic_json<no_key_compare_map>;
+using no_key_compare_json = nlohmann::json::with_object_t<no_key_compare_map>;
 
 // An ObjectType whose erase(iterator) returns void rather than the following
 // iterator, as for instance Abseil's hash maps do
@@ -196,7 +196,7 @@ struct void_erase_map : std::map<Key, T, Compare, Allocator>
     }
 };
 
-using void_erase_json = nlohmann::basic_json<void_erase_map>;
+using void_erase_json = nlohmann::json::with_object_t<void_erase_map>;
 
 // wraps an iterator, but only offers the LegacyForwardIterator operations,
 // like the iterators of std::unordered_map and other hash maps
@@ -388,7 +388,7 @@ class forward_only_map
     }
 };
 
-using forward_only_json = nlohmann::basic_json<forward_only_map>;
+using forward_only_json = nlohmann::json::with_object_t<forward_only_map>;
 
 } // namespace
 

tests/src/unit-hash.cpp

@@ -157,13 +157,13 @@ TEST_CASE("hash<nlohmann::json>")
     // the ends of the integer ranges, which equal floats exactly
     const auto int_min = (std::numeric_limits<json::number_integer_t>::min)();
     const auto int_max = (std::numeric_limits<json::number_integer_t>::max)();
-    const auto two_63 = json::number_unsigned_t(1) << 63U;
+    const auto two_63 = static_cast<json::number_unsigned_t>(1) << 63U;
     CHECK(json(int_min) == json(-9223372036854775808.0));
     CHECK(std::hash<json> {}(json(int_min)) == std::hash<json> {}(json(-9223372036854775808.0)));
     CHECK(json(two_63) == json(9223372036854775808.0));
     CHECK(std::hash<json> {}(json(two_63)) == std::hash<json> {}(json(9223372036854775808.0)));
-    CHECK(json(json::number_unsigned_t(int_max)) == json(int_max));
+    CHECK(json(static_cast<json::number_unsigned_t>(int_max)) == json(int_max));
-    CHECK(std::hash<json> {}(json(json::number_unsigned_t(int_max))) == std::hash<json> {}(json(int_max)));
+    CHECK(std::hash<json> {}(json(static_cast<json::number_unsigned_t>(int_max))) == std::hash<json> {}(json(int_max)));
 }
 
 TEST_CASE("hash<nlohmann::ordered_json>")

tests/src/unit-json_pointer.cpp

@@ -939,3 +939,81 @@ TEST_CASE("unescaping keeps a '~' that does not start an escape sequence")
     nlohmann::detail::unescape(s);
     CHECK(s == "~/~");
 }
+
+TEST_CASE("flatten of structured values")
+{
+    SECTION("values nested too deeply for the call stack (#5393)")
+    {
+        // flatten() used to recurse once per nesting level
+        const std::size_t depth = 100000;
+        for (const bool objects :
+                {
+                    false, true
+                })
+        {
+            CAPTURE(objects)
+            std::string text;
+            std::string path;
+            for (std::size_t i = 0; i < depth; ++i)
+            {
+                text += objects ? "{\"a\":" : "[";
+                path += objects ? "/a" : "/0";
+            }
+            text += "0";
+            text += std::string(depth, objects ? '}' : ']');
+            const auto value = json::parse(text);
+
+            const auto flat = value.flatten();
+            REQUIRE(flat.size() == 1);
+            REQUIRE(flat.begin().key().size() == path.size());
+            CHECK(flat.begin().key() == path);
+            CHECK(flat.begin().value() == 0);
+
+            // unflatten() is not iterative: it takes time and memory
+            // quadratic in the depth, so it is only roundtripped for a
+            // moderate depth
+            std::string small_text;
+            for (std::size_t i = 0; i < 500; ++i)
+            {
+                small_text += objects ? "{\"a\":" : "[";
+            }
+            small_text += "0";
+            small_text += std::string(500, objects ? '}' : ']');
+            const auto small_value = json::parse(small_text);
+            CHECK(small_value.flatten().unflatten() == small_value);
+        }
+    }
+
+    SECTION("objects and arrays interleaved")
+    {
+        const json value =
+        {
+            {"a", {1, {{"b", json::array()}, {"c", json::object()}}, json::array({{{"x~/", {true, nullptr}}}})}},
+            {"a/b", {{"~", 1}}},
+            {"z", "s"}
+        };
+
+        const json expected =
+        {
+            {"/a/0", 1},
+            {"/a/1/b", nullptr},
+            {"/a/1/c", nullptr},
+            {"/a/2/0/x~0~1/0", true},
+            {"/a/2/0/x~0~1/1", nullptr},
+            {"/a~1b/~0", 1},
+            {"/z", "s"}
+        };
+
+        CHECK(value.flatten() == expected);
+    }
+
+    SECTION("order of the entries of an ordered_json")
+    {
+        const auto value = nlohmann::ordered_json::parse(
+                               R"({"z":"s","a/b":{"~":1,"k":[]},"a":[1,{"c":{},"b":[]},[{"x~/":[true,null],"w":2}]]})");
+
+        const auto flat = value.flatten();
+        CHECK(flat.dump() ==
+              R"({"/z":"s","/a~1b/~0":1,"/a~1b/k":null,"/a/0":1,"/a/1/c":null,"/a/1/b":null,"/a/2/0/x~0~1/0":true,"/a/2/0/x~0~1/1":null,"/a/2/0/w":2})");
+    }
+}

tests/src/unit-large_json.cpp

@@ -426,7 +426,7 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
     {
         for (std::size_t depth = 120; depth <= 140; ++depth)
         {
-            CAPTURE(depth);
+            CAPTURE(depth)
 
             const json array = nested_array(depth, json(7));
             CHECK(json::from_cbor(json::to_cbor(array)) == array);
@@ -464,7 +464,7 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
                     nlohmann::detail::recursion_depth_limit() + 1, nlohmann::detail::recursion_depth_limit() + 2
                 })
         {
-            CAPTURE(depth);
+            CAPTURE(depth)
             const json array = nested_array(depth, json(0));
 
             std::vector<std::uint8_t> expected_cbor(depth, 0x81);
@@ -505,10 +505,22 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
         const json discarded_leaf(json::value_t::discarded);
         const json deep_discarded = nested_array(depth, discarded_leaf);
 
-        CHECK_THROWS_WITH_AS(json::to_cbor(deep_discarded), "[json.exception.type_error.321] cannot serialize discarded value to CBOR", json::type_error);
+        // with diagnostics, the message names the path to the discarded leaf
-        CHECK_THROWS_WITH_AS(json::to_msgpack(deep_discarded), "[json.exception.type_error.321] cannot serialize discarded value to MessagePack", json::type_error);
+#if JSON_DIAGNOSTICS
-        CHECK_THROWS_WITH_AS(json::to_ubjson(deep_discarded), "[json.exception.type_error.321] cannot serialize discarded value to UBJSON", json::type_error);
+        std::string path;
-        CHECK_THROWS_WITH_AS(json::to_bjdata(deep_discarded), "[json.exception.type_error.321] cannot serialize discarded value to BJData", json::type_error);
+        for (std::size_t i = 0; i < depth; ++i)
+        {
+            path += "/0";
+        }
+        const std::string prefix = "[json.exception.type_error.321] (" + path + ") ";
+#else
+        const std::string prefix = "[json.exception.type_error.321] ";
+#endif
+
+        CHECK_THROWS_WITH_AS(json::to_cbor(deep_discarded), (prefix + "cannot serialize discarded value to CBOR").c_str(), json::type_error);
+        CHECK_THROWS_WITH_AS(json::to_msgpack(deep_discarded), (prefix + "cannot serialize discarded value to MessagePack").c_str(), json::type_error);
+        CHECK_THROWS_WITH_AS(json::to_ubjson(deep_discarded), (prefix + "cannot serialize discarded value to UBJSON").c_str(), json::type_error);
+        CHECK_THROWS_WITH_AS(json::to_bjdata(deep_discarded), (prefix + "cannot serialize discarded value to BJData").c_str(), json::type_error);
     }
 
     SECTION("does not overflow the C++ stack")

tests/src/unit-locale-cpp.cpp

@@ -172,7 +172,7 @@ TEST_CASE("locale-dependent test (LC_NUMERIC=de_DE)")
             // a floating-point type that is not a float or a double is written
             // with snprintf, whose locale-specific decimal point and thousands
             // separator are undone afterwards
-            using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
+            using long_double_json = nlohmann::json::with_float_t<long double>;
             CHECK(long_double_json(12345.5L).dump() == "12345.5");
             CHECK(long_double_json(1.0L).dump() == "1.0");
             CHECK(long_double_json(-0.25L).dump() == "-0.25");
@@ -272,7 +272,7 @@ TEST_CASE("locale changes between lexer construction and number conversion (#519
     }
     text += "]";
 
-    using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
+    using long_double_json = nlohmann::json::with_float_t<long double>;
 
     // reference values, parsed without a locale switch
     REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
@@ -432,7 +432,7 @@ TEST_CASE("locale changes during a single dump() (#5709 item 3)")
     // long double on 64-bit Arm, where it is IEEE-754 double) takes the
     // locale-independent to_chars() path instead, and this test is a no-op
     // there.
-    using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
+    using long_double_json = nlohmann::json::with_float_t<long double>;
     using ld_limits = std::numeric_limits<long_double_json::number_float_t>;
     const bool is_ieee_single_or_double =
         (ld_limits::is_iec559 && ld_limits::digits == 24 && ld_limits::max_exponent == 128) ||

tests/src/unit-msgpack.cpp

@@ -2432,8 +2432,8 @@ TEST_CASE("MessagePack numbers use the active union member (see #5644)")
     // used to read the union member that was not the active one, writing
     // wrong bytes for some values; std::int64_t/std::uint64_t (the default
     // types, where both members have the same width) were not affected
-    using int32_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint64_t, double>;
+    using int32_json = nlohmann::json::with_integers_t<std::int32_t, std::uint64_t>;
-    using int16_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int16_t, std::uint64_t, double>;
+    using int16_json = nlohmann::json::with_integers_t<std::int16_t, std::uint64_t>;
 
     SECTION("number_integer_t = std::int32_t")
     {

tests/src/unit-regression1.cpp

@@ -39,7 +39,7 @@ using nlohmann::json;
 
 template<class K, class V, class dummy_compare, class A>
 using my_workaround_fifo_map = nlohmann::fifo_map<K, V, nlohmann::fifo_map_compare<K>, A>;
-using my_json = nlohmann::basic_json<my_workaround_fifo_map>;
+using my_json = nlohmann::json::with_object_t<my_workaround_fifo_map>;
 
 /////////////////////////////////////////////////////////////////////
 // for #977
@@ -86,8 +86,7 @@ struct foo_serializer < T, typename std::enable_if < !std::is_same<foo, T>::valu
 };
 } // namespace ns
 
-using foo_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t,
+using foo_json = nlohmann::json::with_json_serializer_t<ns::foo_serializer>;
-      std::uint64_t, double, std::allocator, ns::foo_serializer, std::vector<std::uint8_t>>;
 
 /////////////////////////////////////////////////////////////////////
 // for #805
@@ -254,7 +253,7 @@ TEST_CASE("regression tests 1")
     {
         // create JSON class with nonstandard integer number type
         using custom_json =
-            nlohmann::basic_json<std::map, std::vector, std::string, bool, int32_t, uint32_t, float>;
+            nlohmann::json::with_integers_t<std::int32_t, std::uint32_t>::with_float_t<float>;
         custom_json j;
         j["int_1"] = 1;
         CHECK(j["int_1"] == 1);
@@ -470,18 +469,17 @@ TEST_CASE("regression tests 1")
         // create JSON class with nonstandard float number type
 
         // float
-        nlohmann::basic_json<std::map, std::vector, std::string, bool, int32_t, uint32_t, float> const j_float =
+        nlohmann::json::with_integers_t<std::int32_t, std::uint32_t>::with_float_t<float> const j_float =
             1.23e25f;
         CHECK(j_float.get<float>() == 1.23e25f);
 
         // double
-        nlohmann::basic_json<std::map, std::vector, std::string, bool, int64_t, uint64_t, double> const j_double =
+        nlohmann::json const j_double =
             1.23e35;
         CHECK(j_double.get<double>() == 1.23e35);
 
         // long double
-        nlohmann::basic_json<std::map, std::vector, std::string, bool, int64_t, uint64_t, long double>
+        nlohmann::json::with_float_t<long double> const j_long_double = 1.23e45L;
-        const j_long_double = 1.23e45L;
         CHECK(j_long_double.get<long double>() == 1.23e45L);
     }
 

tests/src/unit-regression2.cpp

@@ -64,18 +64,7 @@ using ordered_json = nlohmann::ordered_json;
     /////////////////////////////////////////////////////////////////////
     // for #4804
     /////////////////////////////////////////////////////////////////////
-    using json_4804 = nlohmann::basic_json<std::map,        // ObjectType
+    using json_4804 = nlohmann::json::with_binary_t<std::vector<std::byte>>;
-    std::vector,     // ArrayType
-    std::string,     // StringType
-    bool,            // BooleanType
-    std::int64_t,    // NumberIntegerType
-    std::uint64_t,   // NumberUnsignedType
-    double,          // NumberFloatType
-    std::allocator,  // AllocatorType
-    nlohmann::adl_serializer,  // JSONSerializer
-    std::vector<std::byte>,    // BinaryType
-    void                       // CustomBaseClass
-    >;
 #endif
 
 #ifdef JSON_HAS_CPP_20
@@ -107,7 +96,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors")
 // for #1021
 /////////////////////////////////////////////////////////////////////
 
-using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
+using float_json = nlohmann::json::with_float_t<float>;
 
 #if (defined(__cpp_exceptions) || defined(__EXCEPTIONS) || defined(_CPPUNWIND)) && !defined(JSON_NOEXCEPTION)
 namespace
@@ -155,10 +144,8 @@ struct failing_allocator : std::allocator<T>
     };
 };
 
-using failing_json = nlohmann::basic_json<std::map, std::vector, std::string, bool,
+using failing_json = nlohmann::json::with_allocator_t<failing_allocator>;
-      std::int64_t, std::uint64_t, double, failing_allocator>;
+using failing_ordered_json = nlohmann::ordered_json::with_allocator_t<failing_allocator>;
-using failing_ordered_json = nlohmann::basic_json<nlohmann::ordered_map, std::vector, std::string, bool,
-      std::int64_t, std::uint64_t, double, failing_allocator>;
 
 // builds `depth` levels of nesting around a scalar, iteratively (never
 // recursing: each wrap only moves the previous, already-built value, which

tests/src/unit-regression3.cpp

@@ -62,18 +62,7 @@ using ordered_json = nlohmann::ordered_json;
     /////////////////////////////////////////////////////////////////////
     // for #4804
     /////////////////////////////////////////////////////////////////////
-    using json_4804 = nlohmann::basic_json<std::map,        // ObjectType
+    using json_4804 = nlohmann::json::with_binary_t<std::vector<std::byte>>;
-    std::vector,     // ArrayType
-    std::string,     // StringType
-    bool,            // BooleanType
-    std::int64_t,    // NumberIntegerType
-    std::uint64_t,   // NumberUnsignedType
-    double,          // NumberFloatType
-    std::allocator,  // AllocatorType
-    nlohmann::adl_serializer,  // JSONSerializer
-    std::vector<std::byte>,    // BinaryType
-    void                       // CustomBaseClass
-    >;
 #endif
 
 #ifdef JSON_HAS_CPP_20
@@ -930,7 +919,7 @@ TEST_CASE("regression test #5476 - array type without reserve()")
 {
     // the capacity reserved for definite-length arrays must not require the
     // array type to have a reserve() member function
-    using deque_json = nlohmann::basic_json<std::map, std::deque>;
+    using deque_json = nlohmann::json::with_array_t<std::deque>;
 
     SECTION("std::deque")
     {

tests/src/unit-serialization.cpp

@@ -367,8 +367,7 @@ TEST_CASE("dump for basic_json with long double number_float_t")
     // serializer::dump_float(x, std::false_type). That branch must use the
     // "%.*Lg" format specifier; using "%.*g" with a long double argument is
     // undefined behavior and corrupts the output.
-    using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string,
+    using long_double_json = nlohmann::json::with_float_t<long double>;
-          bool, std::int64_t, std::uint64_t, long double>;
 
     SECTION("round-trip dump/parse")
     {