Docs: compile times of JSON_HAS_STD_FORMAT=0 and JSON_HAS_RANGES=0

Add sections on switching off std::format and ranges support to the compile-times page, with measurements (Apple clang, GCC 16, -std=c++20), the functionality that is lost, and chart rows (issue #5294). Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Binary reader and writer: remove checks that invariants make dead (#6001 )
2026-11-05 12:19:21 +00:00 · 2026-11-05 09:18:04 +01:00 · 2026-11-04 23:42:17 +01:00 · 2026-11-04 23:38:57 +01:00 · 2026-11-04 23:38:47 +01:00 · 2026-11-04 17:46:20 +01:00
41 changed files with 611 additions and 260 deletions
@@ -28,7 +28,7 @@ jobs:

    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -22,7 +22,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -46,7 +46,7 @@ jobs:

    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -20,7 +20,7 @@ jobs:
    timeout-minutes: 30
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -13,7 +13,7 @@ jobs:
    runs-on: ubuntu-22.04
    steps:
    - name: Harden Runner
-      uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+      uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
      with:
        egress-policy: audit

@@ -27,7 +27,7 @@ jobs:

    steps:
    - name: Harden Runner
-      uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+      uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
      with:
        egress-policy: audit

@@ -20,7 +20,7 @@ jobs:
      pull-requests: write
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -21,7 +21,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -31,7 +31,7 @@ jobs:
      security-events: write
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -54,7 +54,7 @@ jobs:
    timeout-minutes: 60
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -23,7 +23,7 @@ jobs:

    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -37,7 +37,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -40,7 +40,7 @@ jobs:

    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -36,7 +36,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -16,7 +16,7 @@ jobs:

    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -25,7 +25,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -38,7 +38,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Check that Meson and pkg-config offer the CMake options
        run: make check_build_options
      - name: Install Meson
@@ -80,7 +80,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -92,7 +92,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -112,7 +112,7 @@ jobs:
        target: [ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind]
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -123,7 +123,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -142,7 +142,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -168,7 +168,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -178,7 +178,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -244,7 +244,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: CXX=g++-${{ matrix.compiler }} cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -262,7 +262,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -279,7 +279,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Disable filesystem support (std::filesystem is unusable) for clang 7 / 8 / 9 / 10
        run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV"
        if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }}
@@ -299,7 +299,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -319,7 +319,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build with libc++
@@ -346,7 +346,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -366,7 +366,7 @@ jobs:
      # import-std support. Its opt-in token is CMake-version-specific, so pin
      # CMake to the version whose token is set in tests/module_cpp20/CMakeLists.txt.
      - name: Get pinned CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
        with:
          cmakeVersion: 4.3.4
      # Clang: the std library module is provided by libc++ (the image's libstdc++
@@ -392,7 +392,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -409,7 +409,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -423,7 +423,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DJSON_CI=On
      - name: Build
@@ -433,7 +433,7 @@ jobs:
    runs-on: ubuntu-26.04
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -443,7 +443,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Run CMake
        run: cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -GNinja
      - name: Build
@@ -456,7 +456,7 @@ jobs:
        target: [ci_test_examples, ci_test_build_documentation, ci_test_documentation_mermaid]
    steps:
      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
        with:
          egress-policy: audit

@@ -484,7 +484,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Check that the job runs natively on arm64
        run: uname -m && test "$(uname -m)" = aarch64
      - name: Run CMake
@@ -90,7 +90,7 @@ jobs:
        with:
          persist-credentials: false
      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
      - name: Set extra CXX_FLAGS for latest std_version
        # /wd5285 silences C5285 emitted by the bundled third-party doctest.h, which
        # specializes std::tuple (newly diagnosed by the VS2026 v145 toolset)
@@ -216,7 +216,7 @@ jobs:
      # import-std support. Its opt-in token is CMake-version-specific, so pin
      # CMake to the version whose token is set in tests/module_cpp20/CMakeLists.txt.
      - name: Get pinned CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
+        uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
        with:
          cmakeVersion: 4.3.4
      # CMAKE_CXX_FLAGS replaces CMake's MSVC defaults (which include /EHsc), and
@@ -15,7 +15,9 @@ the patch. This function applies a JSON patch in place and returns void.

 ## Exception safety

-No guarantees, value may be corrupted by an unsuccessful patch operation.
+No guarantees: if an operation fails, the operations before it stay applied, and a failing `move` operation may already
+have removed the value at its "from" location. Use [`patch`](patch.md) if the value must be unchanged after a failed
+patch.

 ## Exceptions

@@ -62,8 +64,19 @@ the JSON value, the complexity can usually be neglected.
 ## Notes

 Unlike [`patch`](patch.md), `patch_inplace` applies the operation "in place" and no copy of the JSON value is created.
-That makes it faster for large documents by avoiding the copy. However, the JSON value might be corrupted if the
-function throws an exception.
+That makes it faster for large documents by avoiding the copy.
+
+!!! warning "Not atomic"
+
+    `patch_inplace` applies the operations one after the other and does not undo them when one fails. After an
+    exception, the value holds the result of the operations before the failing one, and a failing `move` operation
+    (a "remove" followed by an "add") may already have removed the value at its "from" location; see the second example
+    below. [RFC 6902](https://tools.ietf.org/html/rfc6902#section-5) requires that a patch with a failing operation is
+    not deemed successful, but leaves open what happens to the document.
+
+    If the value must stay unchanged when the patch fails, use [`patch`](patch.md), which applies the patch to a copy:
+    `#!cpp j = j.patch(json_patch);`. Validating the patch beforehand is not enough, as "test" operations and missing
+    paths are only detected while the patch is applied.

 Duplicate member names inside an operation object are resolved by the parser before the patch is applied (the last one
 is kept, see [`object_t`](object_t.md#behavior)), so `#!json {"op": "add", "op": "remove", "path": "/a"}` is a
@@ -0,0 +1,130 @@
+# Reading and writing files
+
+The library has no functions that take a file name. It reads from and writes to streams, so files are handled with
+`#!cpp std::ifstream` and `#!cpp std::ofstream` (or a `#!cpp FILE*` when reading). This page collects the recipes for
+the common cases and the mistakes that show up again and again.
+
+## Reading a JSON file
+
+Pass the stream to [`parse`](../api/basic_json/parse.md) directly, and check that the file was opened:
+
+```cpp
+#include <fstream>
+#include <stdexcept>
+#include <string>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+json read_json_file(const std::string& path)
+{
+    std::ifstream f(path);
+    if (!f.is_open())
+    {
+        throw std::runtime_error("cannot open " + path);
+    }
+    return json::parse(f);
+}
+```
+
+- **Check that the file is open.** A stream that failed to open looks like an empty input, so `parse` reports
+  [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101) ("unexpected end of input") at line 1,
+  column 1 instead of saying that the file is missing. A relative path is resolved against the working directory of the
+  running program, which is often not the directory of the source file (see the
+  [FAQ](../home/faq.md#parsing-a-file-fails-with-unexpected-end-of-input)).
+- **Don't read the file into a string first.** Parsing from a `#!cpp std::ifstream` is only about 10 to 20 % slower
+  than reading the whole file into a string with one `read()` and parsing that, and it needs no memory for the copy (see
+  [input sources and performance](parsing/index.md#input-sources-and-performance)). The common way to read a file with
+  `#!cpp std::istreambuf_iterator<char>` is so slow that it takes about twice as long as parsing from the stream. Read
+  the file into a string only if you need the text itself, for example for an error message. This advice is for
+  `#!cpp json::parse`. [`json_document`](json_view.md) reads a stream into a buffer that it owns anyway, so there is no
+  copy to save: reading the file into a `#!cpp std::string` yourself and passing that as an lvalue, which the document
+  borrows, was about 6 to 12 % faster than parsing from the stream, with the same single buffer.
+- `parse` requires the file to hold exactly one JSON value. Files with one value per line are read with
+  [JSON Lines](parsing/json_lines.md); files too large for memory are covered in
+  [Large, streamed, and partial input](parsing/large_and_partial_input.md).
+
+## Writing a JSON file
+
+[`dump`](../api/basic_json/dump.md) the value into a string first, then write the string:
+
+```cpp
+void write_json_file(const std::string& path, const json& j)
+{
+    const std::string text = j.dump(4) + '\n';  // may throw: nothing is written yet
+
+    std::ofstream f(path, std::ios::binary);
+    if (!f.is_open())
+    {
+        throw std::runtime_error("cannot open " + path);
+    }
+    f << text;
+    f.close();
+    if (f.fail())
+    {
+        throw std::runtime_error("cannot write " + path);
+    }
+}
+```
+
+- **Serialize before opening the file.** Opening a `#!cpp std::ofstream` truncates the file. `dump` and
+  [`operator<<`](../api/operator_ltlt.md) throw [`type_error.316`](../home/exceptions.md#jsonexceptiontype_error316)
+  when a string is not valid UTF-8 (see [handling invalid UTF-8](serialization.md#handling-invalid-utf-8)), and
+  `#!cpp f << std::setw(4) << j` may already have written part of the value when it throws. Serializing into a string
+  first leaves the old file untouched in that case.
+- **Check the stream after closing it.** A full disk or a missing permission sets the stream's error state; it does not
+  throw unless you enabled exceptions on the stream with `exceptions()`.
+- `std::ios::binary` writes the text as it is. Without it, Windows writes `\r\n` for every newline of a pretty-printed
+  value.
+
+## Replacing a file without corrupting it
+
+The library never writes a file itself, so this section is not about the library. It is general C++ advice for a
+problem users of the library keep running into.
+
+A program that crashes or is killed while it writes leaves a truncated file behind. To make sure readers see either the
+old or the new contents, write to a temporary file in the same directory and rename it over the original
+(C++17, `#!cpp <filesystem>`):
+
+```cpp
+#include <filesystem>
+
+void replace_json_file(const std::filesystem::path& path, const json& j)
+{
+    std::filesystem::path tmp = path;
+    tmp += ".tmp";  // use a unique name if several processes may write the file
+
+    write_json_file(tmp.string(), j);
+    std::filesystem::rename(tmp, path);  // replaces path
+}
+```
+
+The temporary file must be in the same directory (more precisely, on the same file system) as the target: a rename
+across file systems does not fall back to a copy, it fails and throws `#!cpp std::filesystem::filesystem_error`. On
+POSIX systems, the rename atomically replaces the file. Standard C++ has no way to flush a file to the disk (`fsync`),
+so this protects against a crashed program, but not necessarily against a power loss.
+
+## Binary formats
+
+The [binary formats](binary_formats/index.md) work the same way, with the streams opened in binary mode:
+
+```cpp
+// read
+std::ifstream in("data.cbor", std::ios::binary);
+json j = json::from_cbor(in);
+
+// write
+const std::vector<std::uint8_t> bytes = json::to_cbor(j);
+std::ofstream out("data.cbor", std::ios::binary);
+out.write(reinterpret_cast<const char*>(bytes.data()), static_cast<std::streamsize>(bytes.size()));
+```
+
+Without `std::ios::binary`, Windows turns the bytes `0x0D 0x0A` into `0x0A` and may stop at a byte `0x1A` while reading,
+and turns `0x0A` into `0x0D 0x0A` while writing, which corrupts the data.
+
+## See also
+
+- [Parsing](parsing/index.md) - the inputs `parse` accepts
+- [Serialization](serialization.md) - the options of `dump`
+- [`parse`](../api/basic_json/parse.md), [`dump`](../api/basic_json/dump.md)
+- [`operator>>`](../api/operator_gtgt.md), [`operator<<`](../api/operator_ltlt.md)
@@ -46,6 +46,8 @@ C++ types, and finally serialize it again.
  [BJData](binary_formats/bjdata.md), [BON8](binary_formats/bon8.md), [BSON](binary_formats/bson.md),
  [CBOR](binary_formats/cbor.md), [MessagePack](binary_formats/messagepack.md), or [UBJSON](binary_formats/ubjson.md).
 - [Binary values](binary_values.md) — store and exchange raw byte sequences.
+- [Reading and writing files](files.md) — read and write files safely, including replacing a file without
+  corrupting it.

 ## How values are stored and configured

@@ -29,7 +29,8 @@ other three continue on its result.

    [`patch`](../api/basic_json/patch.md) leaves the original value unchanged and returns the patched result as a copy.
    If the document is large and the original value is no longer needed,
-    [`patch_inplace`](../api/basic_json/patch_inplace.md) applies the same operations in place instead.
+    [`patch_inplace`](../api/basic_json/patch_inplace.md) applies the same operations in place instead. Unlike
+    `patch`, it is not atomic: if an operation fails, the operations before it stay applied.

 !!! note "Duplicate member names"

@@ -160,7 +160,9 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
  wide. The limit is 16 bytes below 2^32, because the parser reads in 16-byte steps, and a position one step past the
  end must still fit.
 - **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
-  is read to its end into a buffer the document owns, and parsing starts after that.
+  is read to its end into a buffer the document owns, and parsing starts after that. For a file, reading it into a
+  `#!cpp std::string` yourself and parsing that (an lvalue is borrowed) is therefore not more expensive in memory, and
+  was about 6 to 12 % faster than parsing from an `#!cpp std::ifstream`.
 - **Fewer source positions after edits.** With
  [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
  [`materialize()`](../api/basic_json_view/materialize.md) sets the same positions as `#!cpp basic_json::parse()`.
@@ -35,6 +35,8 @@ std::ifstream f("example.json");
 json data = json::parse(f);
 ```

+See [Reading and writing files](../files.md) for checking that the file was opened and for writing files safely.
+
 The input must be encoded in UTF-8; other encodings are not supported. The exception is wide-character input, which
 is read as UTF-16 or UTF-32 — depending on the character type — and converted to UTF-8; see the
 [`parse`](../../api/basic_json/parse.md) page for an example. A single input may contain only one JSON value.
@@ -84,6 +86,9 @@ ahead: after parsing, it is positioned exactly as if every character had been re
 [`operator>>`](../../api/operator_gtgt.md) can still read several values from one stream (see also
 [JSON Lines](json_lines.md)).

+This does not hold for [`json_document`](../json_view.md): it reads a stream into a buffer it owns, so the stream saves
+no memory, and reading the file into a `#!cpp std::string` yourself and borrowing it is about 6 to 12 % faster.
+
 Two macros switch these optimizations off without changing any result:
 [`JSON_DISABLE_STREAM_BULK_SCAN`](../../api/macros/json_disable_stream_bulk_scan.md) for stream buffers that do not
 follow the usual `std::streambuf` contract, and
@@ -39,7 +39,9 @@ iterator pair over non-contiguous but random-access storage (e.g. `#!cpp std::de
 optimization but not the second, since the byte-scanning fast path requires contiguous storage.

 Practically: a file can be passed directly to `parse`, which also avoids the memory for a copy of the file. Reading it
-into a string first is no longer necessary for speed:
+into a string first is no longer necessary for speed (this is for `#!cpp json::parse`; a
+[`json_document`](json_view.md) reads a stream into an owned buffer anyway, so for it reading the file into a string
+and borrowing that is the faster way):

 ```cpp
 // fast: the stream buffer's get area is scanned in bulk
@@ -17,6 +17,9 @@ To write a value directly to a stream (for example, a file or `#!cpp std::cout`)
 std::cout << j << std::endl;
 ```

+To write a file without leaving a truncated file behind when serialization fails, see
+[Reading and writing files](files.md#writing-a-json-file).
+
 !!! note "String, not raw value"

    `dump` always returns a **JSON text**. Serializing a JSON string therefore includes the surrounding quotes and
@@ -110,7 +110,7 @@ in the source files that need it, instantiate `basic_json` once, and consider pr
 The parser did not see any character: the file was most likely not opened. A relative path is resolved against the
 working directory of the running program, which is often the build directory of your IDE, not the source directory.
 Check `is_open()` and try an absolute path. Other causes are an empty file and a stream that was already read. See
-[`parse_error.101`](exceptions.md#jsonexceptionparse_error101).
+[`parse_error.101`](exceptions.md#jsonexceptionparse_error101) and [Reading and writing files](../features/files.md).

 ### Input that arrives in chunks

@@ -1,21 +1,21 @@
-<svg class="nj-fig nj-charts nj-charts--compile" viewBox="0 0 470 516" role="img" aria-labelledby="nj-charts-ct nj-charts-ct-d" xmlns="http://www.w3.org/2000/svg">
+<svg class="nj-fig nj-charts nj-charts--compile" viewBox="0 0 470 620" role="img" aria-labelledby="nj-charts-ct nj-charts-ct-d" xmlns="http://www.w3.org/2000/svg">
 <title id="nj-charts-ct">Change in compile time against the default configuration at -O0, Apple clang and G&#8203;CC 16</title>
-<desc id="nj-charts-ct-d">Grouped horizontal bars, negative is faster. json_fwd.hpp instead of json.hpp on a header-only translation unit: Apple clang -62 %, G&#8203;CC 16 -82 %; extern template on a parsing translation unit: Apple clang -7 %, G&#8203;CC 16 -21 %; JSON_NO_AUTOMATIC_UDLS on a model translation unit: Apple clang -11 %, G&#8203;CC 16 -11 %; JSON_NO_AUTOMATIC_UDLS on a parsing translation unit: Apple clang -1 %, G&#8203;CC 16 -2 %; JSON_HAS_FILESYSTEM=0 + JSON_NO_IO on a model translation unit: Apple clang -8 %, G&#8203;CC 16 -11 %; JSON_HAS_FILESYSTEM=0 on a model translation unit: Apple clang -5 %, G&#8203;CC 16 -9 %; JSON_NO_IO on a model translation unit: Apple clang -2 %, G&#8203;CC 16 -2 %; JSON_USE_GLOBAL_UDLS=0 on a model translation unit: Apple clang 0 %, G&#8203;CC 16 -1 %.</desc>
+<desc id="nj-charts-ct-d">Grouped horizontal bars, negative is faster. json_fwd.hpp instead of json.hpp on a header-only translation unit: Apple clang -62 %, G&#8203;CC 16 -82 %; extern template on a parsing translation unit: Apple clang -7 %, G&#8203;CC 16 -21 %; JSON_NO_AUTOMATIC_UDLS on a model translation unit: Apple clang -11 %, G&#8203;CC 16 -11 %; JSON_NO_AUTOMATIC_UDLS on a parsing translation unit: Apple clang -1 %, G&#8203;CC 16 -2 %; JSON_HAS_FILESYSTEM=0 + JSON_NO_IO on a model translation unit: Apple clang -8 %, G&#8203;CC 16 -11 %; JSON_HAS_FILESYSTEM=0 on a model translation unit: Apple clang -5 %, G&#8203;CC 16 -9 %; JSON_HAS_STD_FORMAT=0 on a model translation unit: Apple clang -1 %, G&#8203;CC 16 -4 %; JSON_HAS_RANGES=0 on a model translation unit: Apple clang -4 %, G&#8203;CC 16 -1 %; JSON_NO_IO on a model translation unit: Apple clang -2 %, G&#8203;CC 16 -2 %; JSON_USE_GLOBAL_UDLS=0 on a model translation unit: Apple clang 0 %, G&#8203;CC 16 -1 %.</desc>
 <rect class="nj-charts-clang" x="8" y="10" width="12" height="12" rx="3"/>
 <text class="nj-charts-leg" x="26" y="20.5">Apple clang</text>
 <rect class="nj-charts-gcc" x="126" y="10" width="12" height="12" rx="3"/>
 <text class="nj-charts-leg" x="144" y="20.5">G&#8203;CC 16</text>
-<line class="nj-charts-grid" x1="47.4" y1="58" x2="47.4" y2="472"/>
-<text class="nj-charts-tick" x="47.4" y="488">&#8722;80 %</text>
-<line class="nj-charts-grid" x1="126.2" y1="58" x2="126.2" y2="472"/>
-<text class="nj-charts-tick" x="126.2" y="488">&#8722;60 %</text>
-<line class="nj-charts-grid" x1="205" y1="58" x2="205" y2="472"/>
-<text class="nj-charts-tick" x="205" y="488">&#8722;40 %</text>
-<line class="nj-charts-grid" x1="283.8" y1="58" x2="283.8" y2="472"/>
-<text class="nj-charts-tick" x="283.8" y="488">&#8722;20 %</text>
-<text class="nj-charts-tick" x="362.6" y="488">0 %</text>
-<text class="nj-charts-tick nj-charts-tick--l" x="8" y="506">&#8592; faster</text>
-<text class="nj-charts-tick nj-charts-tick--r" x="462" y="506">slower &#8594;</text>
+<line class="nj-charts-grid" x1="47.4" y1="58" x2="47.4" y2="576"/>
+<text class="nj-charts-tick" x="47.4" y="592">&#8722;80 %</text>
+<line class="nj-charts-grid" x1="126.2" y1="58" x2="126.2" y2="576"/>
+<text class="nj-charts-tick" x="126.2" y="592">&#8722;60 %</text>
+<line class="nj-charts-grid" x1="205" y1="58" x2="205" y2="576"/>
+<text class="nj-charts-tick" x="205" y="592">&#8722;40 %</text>
+<line class="nj-charts-grid" x1="283.8" y1="58" x2="283.8" y2="576"/>
+<text class="nj-charts-tick" x="283.8" y="592">&#8722;20 %</text>
+<text class="nj-charts-tick" x="362.6" y="592">0 %</text>
+<text class="nj-charts-tick nj-charts-tick--l" x="8" y="610">&#8592; faster</text>
+<text class="nj-charts-tick nj-charts-tick--r" x="462" y="610">slower &#8594;</text>
 <text class="nj-charts-rowlabel" x="8" y="62"><tspan class="nj-charts-code">json_fwd.hpp instead of json.hpp</tspan><tspan class="nj-charts-sub"> &#183; header-only TU</tspan></text>
 <path class="nj-charts-clang" d="M362.6 70 H122.32 Q118.32 70 118.32 74 V78 Q118.32 82 122.32 82 H362.6 Z"><title>json_fwd.hpp instead of json.hpp, header-only TU (Apple clang): &#8722;62 %</title></path>
 <text class="nj-charts-val" x="367.6" y="80.5">&#8722;62 %</text>
@@ -46,15 +46,25 @@
 <text class="nj-charts-val" x="367.6" y="340.5">&#8722;5 %</text>
 <path class="nj-charts-gcc" d="M362.6 344 H331.14 Q327.14 344 327.14 348 V352 Q327.14 356 331.14 356 H362.6 Z"><title>JSON_HAS_FILESYSTEM=0, model TU (G&#8203;CC 16): &#8722;9 %</title></path>
 <text class="nj-charts-val" x="367.6" y="354.5">&#8722;9 %</text>
-<text class="nj-charts-rowlabel" x="8" y="374"><tspan class="nj-charts-code">JSON_NO_IO</tspan><tspan class="nj-charts-sub"> &#183; model TU*</tspan></text>
-<path class="nj-charts-clang" d="M362.6 382 H358.72 Q354.72 382 354.72 386 V390 Q354.72 394 358.72 394 H362.6 Z"><title>JSON_NO_IO, model TU (Apple clang): &#8722;2 %</title></path>
-<text class="nj-charts-val" x="367.6" y="392.5">&#8722;2 %</text>
-<path class="nj-charts-gcc" d="M362.6 396 H358.72 Q354.72 396 354.72 400 V404 Q354.72 408 358.72 408 H362.6 Z"><title>JSON_NO_IO, model TU (G&#8203;CC 16): &#8722;2 %</title></path>
-<text class="nj-charts-val" x="367.6" y="406.5">&#8722;2 %</text>
-<text class="nj-charts-rowlabel" x="8" y="426"><tspan class="nj-charts-code">JSON_USE_GLOBAL_UDLS=0</tspan><tspan class="nj-charts-sub"> &#183; model TU*</tspan></text>
-<path class="nj-charts-clang" d="M362.6 434 h0.5 v12 h-0.5 Z"><title>JSON_USE_GLOBAL_UDLS=0, model TU (Apple clang): 0 %</title></path>
-<text class="nj-charts-val" x="367.6" y="444.5">0 %</text>
-<path class="nj-charts-gcc" d="M362.6 448 H362.6 Q358.66 448 358.66 451.94 V456.06 Q358.66 460 362.6 460 H362.6 Z"><title>JSON_USE_GLOBAL_UDLS=0, model TU (G&#8203;CC 16): &#8722;1 %</title></path>
+<text class="nj-charts-rowlabel" x="8" y="374"><tspan class="nj-charts-code">JSON_HAS_STD_FORMAT=0</tspan><tspan class="nj-charts-sub"> &#183; model TU</tspan></text>
+<path class="nj-charts-clang" d="M362.6 382 H362.6 Q358.66 382 358.66 385.94 V390.06 Q358.66 394 362.6 394 H362.6 Z"><title>JSON_HAS_STD_FORMAT=0, model TU (Apple clang): &#8722;1 %</title></path>
+<text class="nj-charts-val" x="367.6" y="392.5">&#8722;1 %</text>
+<path class="nj-charts-gcc" d="M362.6 396 H350.84 Q346.84 396 346.84 400 V404 Q346.84 408 350.84 408 H362.6 Z"><title>JSON_HAS_STD_FORMAT=0, model TU (G&#8203;CC 16): &#8722;4 %</title></path>
+<text class="nj-charts-val" x="367.6" y="406.5">&#8722;4 %</text>
+<text class="nj-charts-rowlabel" x="8" y="426"><tspan class="nj-charts-code">JSON_HAS_RANGES=0</tspan><tspan class="nj-charts-sub"> &#183; model TU</tspan></text>
+<path class="nj-charts-clang" d="M362.6 434 H350.84 Q346.84 434 346.84 438 V442 Q346.84 446 350.84 446 H362.6 Z"><title>JSON_HAS_RANGES=0, model TU (Apple clang): &#8722;4 %</title></path>
+<text class="nj-charts-val" x="367.6" y="444.5">&#8722;4 %</text>
+<path class="nj-charts-gcc" d="M362.6 448 H362.6 Q358.66 448 358.66 451.94 V456.06 Q358.66 460 362.6 460 H362.6 Z"><title>JSON_HAS_RANGES=0, model TU (G&#8203;CC 16): &#8722;1 %</title></path>
 <text class="nj-charts-val" x="367.6" y="458.5">&#8722;1 %</text>
-<line class="nj-charts-zero" x1="362.6" y1="58" x2="362.6" y2="472"/>
+<text class="nj-charts-rowlabel" x="8" y="478"><tspan class="nj-charts-code">JSON_NO_IO</tspan><tspan class="nj-charts-sub"> &#183; model TU*</tspan></text>
+<path class="nj-charts-clang" d="M362.6 486 H358.72 Q354.72 486 354.72 490 V494 Q354.72 498 358.72 498 H362.6 Z"><title>JSON_NO_IO, model TU (Apple clang): &#8722;2 %</title></path>
+<text class="nj-charts-val" x="367.6" y="496.5">&#8722;2 %</text>
+<path class="nj-charts-gcc" d="M362.6 500 H358.72 Q354.72 500 354.72 504 V508 Q354.72 512 358.72 512 H362.6 Z"><title>JSON_NO_IO, model TU (G&#8203;CC 16): &#8722;2 %</title></path>
+<text class="nj-charts-val" x="367.6" y="510.5">&#8722;2 %</text>
+<text class="nj-charts-rowlabel" x="8" y="530"><tspan class="nj-charts-code">JSON_USE_GLOBAL_UDLS=0</tspan><tspan class="nj-charts-sub"> &#183; model TU*</tspan></text>
+<path class="nj-charts-clang" d="M362.6 538 h0.5 v12 h-0.5 Z"><title>JSON_USE_GLOBAL_UDLS=0, model TU (Apple clang): 0 %</title></path>
+<text class="nj-charts-val" x="367.6" y="548.5">0 %</text>
+<path class="nj-charts-gcc" d="M362.6 552 H362.6 Q358.66 552 358.66 555.94 V560.06 Q358.66 564 362.6 564 H362.6 Z"><title>JSON_USE_GLOBAL_UDLS=0, model TU (G&#8203;CC 16): &#8722;1 %</title></path>
+<text class="nj-charts-val" x="367.6" y="562.5">&#8722;1 %</text>
+<line class="nj-charts-zero" x1="362.6" y1="58" x2="362.6" y2="576"/>
 </svg>
@@ -11,6 +11,11 @@ that cost, ordered by how much they typically save.
    16.2.0 on macOS (Apple M1 Max). All configurations were run interleaved, in alternating order. They show the order
    of magnitude to expect; measure your own code before and after a change.

+    The two sections on `std::format` and ranges apply to C++20 only and were measured with `-std=c++20`, so their
+    baselines are slower than the others. Their differences are mostly within the run-to-run noise of a few percent:
+    two complete runs of the same configuration differed by up to 10% in the median. The tables show the later run (one
+    outlier cell was repeated).
+
    The translation units are listed below. They replace the ones used for earlier versions of this page, whose
    sources were not published, so the numbers differ from earlier versions.

@@ -66,7 +71,8 @@ that cost, ordered by how much they typically save.
 --8<-- "images/compile_times.svg"
 </div>
 <p class="nj-charts-caption" markdown>Change in compile time at `-O0` against the default configuration of the same
-translation unit, taken from the tables below (medians of nine runs, see above). * computed from the milliseconds in
+translation unit, taken from the tables below (medians of nine runs, see above; the `std::format` and ranges rows are
+measured with `-std=c++20`). * computed from the milliseconds in
 [Options without effect on compile times](#options-without-effect-on-compile-times).</p>

 ## Include `json_fwd.hpp` in headers
@@ -214,6 +220,88 @@ from and to `std::string`; whether that fallback exists depends on the platform'
 (`std::wstring` on Windows), so check code that converts paths. `JSON_NO_IO` additionally removes all stream
 functions (see [Options without effect](#options-without-effect-on-compile-times)).

+## Switch off `std::format` support
+
+With C++20 and a standard library that provides `<format>`, the library includes it to provide a
+[`std::formatter<basic_json>`](../api/basic_json/std_formatter.md) specialization (see
+[`JSON_HAS_STD_FORMAT`](../api/macros/json_has_std_format.md)). If you do not format JSON values with `std::format`,
+switch the detection off for the whole project:
+
+```cmake
+target_compile_definitions(my_target PRIVATE JSON_HAS_STD_FORMAT=0)
+```
+
+| Compiler    | Translation unit | Default (`-O0` / `-O2`) | `std::format` off (`-O0` / `-O2`) | Change     |
+|-------------|------------------|------------------------:|----------------------------------:|-----------:|
+| Apple clang | model            |       1283 ms / 1544 ms |                1269 ms / 1539 ms |  −1% / 0%  |
+| GCC 16      | model            |       2026 ms / 2482 ms |                1944 ms / 2416 ms | −4% / −3%  |
+| Apple clang | parsing          |       1350 ms / 2278 ms |                1347 ms / 2300 ms |  0% / +1%  |
+| GCC 16      | parsing          |       2715 ms / 4351 ms |                2644 ms / 4287 ms | −3% / −1%  |
+
+<p class="nj-table-caption" markdown>Compile time with `JSON_HAS_STD_FORMAT=0` against the default configuration, both with `-std=c++20`, at `-O0` and `-O2`.</p>
+
+The saving is small, and only GCC shows it: libstdc++'s `<format>` is an additional header that nothing else in the
+translation unit includes, so the macro removes it (and the headers it pulls in). With libc++ (Apple clang), other
+standard headers include `<format>` anyway, so the macro only removes the library's own formatter, and the difference is
+within the noise.
+
+The trade-off: the `std::formatter<basic_json>` specialization is removed, so `#!cpp std::format("{}", j)`,
+and `#!cpp std::format_to` with a JSON value no longer compile (in a test with both compilers on
+macOS, the error was that `std::formatter` must be specialized for the type). Use [`dump()`](../api/basic_json/dump.md),
+or the stream operators, instead.
+
+## Switch off ranges support
+
+With C++20 and a standard library that provides `<ranges>`, the library includes it to declare the
+[`items()`](../api/basic_json/items.md) proxy a *borrowed range*, and uses the `std::ranges` concepts and algorithms for
+construction from range views and for iterator input (see [`JSON_HAS_RANGES`](../api/macros/json_has_ranges.md)). If you
+do not need these, switch the detection off for the whole project:
+
+```cmake
+target_compile_definitions(my_target PRIVATE JSON_HAS_RANGES=0)
+```
+
+| Compiler    | Translation unit | Default (`-O0` / `-O2`) | Ranges off (`-O0` / `-O2`) | Change     |
+|-------------|------------------|------------------------:|---------------------------:|-----------:|
+| Apple clang | model            |       1283 ms / 1544 ms |           1237 ms / 1495 ms | −4% / −3%  |
+| GCC 16      | model            |       2026 ms / 2482 ms |           2004 ms / 2464 ms | −1% / −1%  |
+| Apple clang | parsing          |       1350 ms / 2278 ms |           1311 ms / 2284 ms |  −3% / 0%  |
+| GCC 16      | parsing          |       2715 ms / 4351 ms |           2705 ms / 4427 ms |  0% / +2%  |
+
+<p class="nj-table-caption" markdown>Compile time with `JSON_HAS_RANGES=0` against the default configuration, both with `-std=c++20`, at `-O0` and `-O2`.</p>
+
+The saving is small as well, and with GCC it is not distinguishable from the noise: its `<format>` includes `<span>`,
+so with `std::format` support left on, part of the ranges machinery is still parsed.
+Both macros together save more than either alone:
+
+```cmake
+target_compile_definitions(my_target PRIVATE JSON_HAS_STD_FORMAT=0 JSON_HAS_RANGES=0)
+```
+
+| Compiler    | Translation unit | Default (`-O0` / `-O2`) | Both off (`-O0` / `-O2`) | Change     |
+|-------------|------------------|------------------------:|-------------------------:|-----------:|
+| Apple clang | model            |       1283 ms / 1544 ms |         1223 ms / 1504 ms | −5% / −3%  |
+| GCC 16      | model            |       2026 ms / 2482 ms |         1914 ms / 2378 ms | −6% / −4%  |
+| Apple clang | parsing          |       1350 ms / 2278 ms |         1311 ms / 2271 ms |  −3% / 0%  |
+| GCC 16      | parsing          |       2715 ms / 4351 ms |         2635 ms / 4260 ms | −3% / −2%  |
+
+<p class="nj-table-caption" markdown>Compile time with `JSON_HAS_STD_FORMAT=0` and `JSON_HAS_RANGES=0` against the default configuration, both with `-std=c++20`, at `-O0` and `-O2`.</p>
+
+The trade-offs of `JSON_HAS_RANGES=0` are larger than those of the other options on this page:
+
+- `items()` is no longer a borrowed range. On a temporary, `#!cpp std::ranges::begin(make_json().items())` no longer
+  compiles, and `std::ranges` algorithms such as `#!cpp std::ranges::find_if(make_json().items(), pred)` return
+  `std::ranges::dangling` instead of an iterator, so code that uses the result no longer compiles. Calling them on
+  `#!cpp j.items()` for a named `j` is not affected.
+- [`JSON_HAS_RANGE_VIEW_CONVERSION`](../api/macros/json_has_range_view_conversion.md) becomes `0`, so a JSON array can no
+  longer be constructed from a `std::ranges::view` such as the result of `std::views::filter`. The assignment
+  `#!cpp json j = v | std::views::filter(pred);` no longer compiles.
+- Parsing from an iterator pair loses the C++20 fast path for contiguous iterators that are not pointers (for
+  example those of `std::vector`) and falls back to byte-at-a-time scanning. The result is the same; parsing may be
+  slower (not measured here).
+
+The first two were checked with both compilers on macOS; the third is read from the library's source.
+
 ## Use C++20 modules

 With a toolchain that supports named modules, `import nlohmann.json;` compiles the library once into a module and
@@ -343,6 +431,8 @@ and −2% / 0% (GCC). To get a saving, combine it with

 - [`JSON_HAS_FILESYSTEM`](../api/macros/json_has_filesystem.md) - control the conversions from and to
  `std::filesystem::path`
+- [`JSON_HAS_STD_FORMAT`](../api/macros/json_has_std_format.md) - control the `std::formatter` specialization
+- [`JSON_HAS_RANGES`](../api/macros/json_has_ranges.md) - control the use of `std::ranges`
 - [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) - do not include the user-defined string
  literals automatically
 - [Modules](../features/modules.md) - C++20 module support
@@ -130,6 +130,7 @@ nav:
          - features/element_access/unchecked_access.md
          - features/element_access/checked_access.md
          - features/element_access/default_value.md
+      - features/files.md
      - features/iterators.md
      - features/merge_patch.md
      - features/json_patch.md
@@ -73,6 +73,8 @@ COMPILE = [
    ("JSON_NO_AUTOMATIC_UDLS", "parsing TU", -1, -2, False),
    ("JSON_HAS_FILESYSTEM=0 + JSON_NO_IO", "model TU", -8, -11, False),
    ("JSON_HAS_FILESYSTEM=0", "model TU", -5, -9, False),
+    ("JSON_HAS_STD_FORMAT=0", "model TU", -1, -4, False),
+    ("JSON_HAS_RANGES=0", "model TU", -4, -1, False),
    # computed from the ms in "Options without effect on compile times": 1034/1051, 1756/1801, 1055/1051, 1783/1801
    ("JSON_NO_IO", "model TU", round((1034 / 1051 - 1) * 100), round((1756 / 1801 - 1) * 100), True),
    ("JSON_USE_GLOBAL_UDLS=0", "model TU", round((1055 / 1051 - 1) * 100), round((1783 / 1801 - 1) * 100), True),
@@ -344,8 +344,8 @@ class binary_reader
    nested document could declare a length that disagrees with where its
    terminator actually falls and quietly hand the bytes in between to the
    enclosing document. A well-formed document is at least 5 bytes (the prefix
-    plus the terminator); the equality also rejects those impossible sizes,
-    since at least 5 bytes are always consumed.
+    plus the terminator); open_bson_document() rejects a smaller size before
+    it is stored, so @a document_size is never negative here.

    @param[in] document_start  value of chars_read before the size prefix
    @param[in] document_size   the declared document size
@@ -353,7 +353,7 @@ class binary_reader
    */
    bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size)
    {
-        if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
+        if (JSON_HEDLEY_UNLIKELY(static_cast<std::size_t>(document_size) != chars_read - document_start))
        {
            return bson_document_size_error(document_size, chars_read - document_start);
        }
@@ -1024,7 +1024,7 @@ class binary_reader
            {
                std::uint64_t len{};
                std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_cbor_array(size);
+                return get_cbor_argument(len) && get_container_size(len, size, "array") && enter_cbor_array(size);
            }

            case 0x9F: // array (indefinite length)
@@ -1064,7 +1064,7 @@ class binary_reader
            {
                std::uint64_t len{};
                std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "map") && enter_object(size);
+                return get_cbor_argument(len) && get_container_size(len, size, "map") && enter_object(size);
            }

            case 0xBF: // map (indefinite length)
@@ -1700,7 +1700,7 @@ class binary_reader
    }

    /*!
-    @brief narrow a definite CBOR array/map length to std::size_t
+    @brief narrow a definite CBOR or MessagePack array/map length to std::size_t

    A definite length is rejected if it does not fit in std::size_t or if it
    equals detail::unknown_size(), which is reserved to mark an indefinite-
@@ -1713,7 +1713,7 @@ class binary_reader
    @param[in]  context  "array" or "map", for the error message
    @return whether the length is usable
    */
-    bool get_cbor_container_size(const std::uint64_t len, std::size_t& result, const char* context)
+    bool get_container_size(const std::uint64_t len, std::size_t& result, const char* context)
    {
        if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::size_t>(len) || len == detail::unknown_size()))
        {
@@ -2309,7 +2309,8 @@ class binary_reader
            case 0xDD: // array 32
            {
                std::uint32_t len{};
-                return get_number(len) && enter_msgpack_array(conditional_static_cast<std::size_t>(len));
+                std::size_t size{};
+                return get_number(len) && get_container_size(len, size, "array") && enter_msgpack_array(size);
            }

            case 0xDE: // map 16
@@ -2321,7 +2322,8 @@ class binary_reader
            case 0xDF: // map 32
            {
                std::uint32_t len{};
-                return get_number(len) && enter_object(conditional_static_cast<std::size_t>(len));
+                std::size_t size{};
+                return get_number(len) && get_container_size(len, size, "map") && enter_object(size);
            }

            // negative fixint
@@ -3092,16 +3094,13 @@ class binary_reader
        {
            if (size_and_type.second != 0)
            {
-                if (size_and_type.second != 'N')
+                for (std::size_t i = 0; i < size_and_type.first; ++i)
                {
-                    for (std::size_t i = 0; i < size_and_type.first; ++i)
+                    if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
                    {
-                        if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
-                        {
-                            return false;
-                        }
-                        dim.push_back(dimlen);
+                        return false;
                    }
+                    dim.push_back(dimlen);
                }
            }
            else
@@ -3131,6 +3130,27 @@ class binary_reader
        return true;
    }

+    /*!
+    @brief store the count of an optimized container (or the length of a string)
+
+    A count equal to npos is rejected: npos marks a container whose count is
+    not known, so a count of that value would be read as "no count" (compare
+    get_container_size()). No container can hold that many elements.
+
+    @param[in]  count   the count read from the input
+    @param[out] result  set to @a count unless it is rejected
+    @return whether @a count is usable
+    */
+    bool store_ubjson_count(const std::size_t count, std::size_t& result)
+    {
+        if (JSON_HEDLEY_UNLIKELY(count == npos))
+        {
+            return out_of_range_error(exception_id::container_too_large, "integer value overflow", "size");
+        }
+        result = count;
+        return true;
+    }
+
    /*!
    @brief read a UBJSON/BJData optimized-container count of a signed marker
           type ('i', 'I', 'l', 'L') and narrow it to std::size_t
@@ -3161,8 +3181,7 @@ class binary_reader
        {
            return out_of_range_error(exception_id::container_too_large, "integer value overflow", "size");
        }
-        result = static_cast<std::size_t>(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char
-        return true;
+        return store_ubjson_count(static_cast<std::size_t>(number), result); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char
    }

    /*!
@@ -3239,8 +3258,7 @@ class binary_reader
                {
                    return false;
                }
-                result = conditional_static_cast<std::size_t>(number);
-                return true;
+                return store_ubjson_count(conditional_static_cast<std::size_t>(number), result);
            }

            case 'M':
@@ -3258,8 +3276,7 @@ class binary_reader
                {
                    return out_of_range_error(exception_id::container_too_large, "integer value overflow", "size");
                }
-                result = detail::conditional_static_cast<std::size_t>(number);
-                return true;
+                return store_ubjson_count(detail::conditional_static_cast<std::size_t>(number), result);
            }

            case '[':
@@ -3303,11 +3320,9 @@ class binary_reader
                    // follow the documented _ArrayType_, _ArraySize_, _ArrayData_ order
                    if (ndarray_dtype != 0)
                    {
+                        // every optimized type marker is_valid_optimized_type() accepts for BJData has a name
                        const char* type_name = bjd_type_name(ndarray_dtype);
-                        if (JSON_HEDLEY_UNLIKELY(type_name == nullptr))
-                        {
-                            return unexpected_byte("invalid byte", "type");
-                        }
+                        JSON_ASSERT(type_name != nullptr);

                        string_t type_key = "_ArrayType_";
                        string_t type = type_name; // sax->string() takes a reference
@@ -3390,12 +3405,6 @@ class binary_reader
        if (current == '$')
        {
            result.second = get();  // must not ignore 'N', because 'N' maybe the type
-            if (input_format == input_format_t::bjdata
-                    && JSON_HEDLEY_UNLIKELY(is_bjd_excluded_optimized_type(result.second)))
-            {
-                return optimized_type_error();
-            }
-
            if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type")))
            {
                return false;
@@ -3423,7 +3432,7 @@ class binary_reader
            // an ndarray was read here only if the flag flipped; when it was
            // seeded true, get_ubjson_size_value() already rejected the nested
            // dimension vector
-            if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
+            if (is_ndarray && !inside_ndarray)
            {
                result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters
            }
@@ -3433,7 +3442,7 @@ class binary_reader
        if (current == '#')
        {
            const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
-            if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
+            if (is_ndarray && !inside_ndarray)
            {
                return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size");
            }
@@ -3607,7 +3616,6 @@ class binary_reader
    JSON_HEDLEY_NEVER_INLINE
    bool parse_ubjson_typed_run(std::size_t& remaining, const char_int_type type, std::true_type /*bulk*/)
    {
-        const bool bjdata = input_format == input_format_t::bjdata;
        switch (type)
        {
            case 'U':
@@ -3621,11 +3629,11 @@ class binary_reader
            case 'L':
                return parse_ubjson_typed_run<std::int64_t>(remaining);
            case 'u':
-                return !bjdata || parse_ubjson_typed_run<std::uint16_t>(remaining);
+                return parse_ubjson_typed_run<std::uint16_t>(remaining);
            case 'm':
-                return !bjdata || parse_ubjson_typed_run<std::uint32_t>(remaining);
+                return parse_ubjson_typed_run<std::uint32_t>(remaining);
            case 'M':
-                return !bjdata || parse_ubjson_typed_run<std::uint64_t>(remaining);
+                return parse_ubjson_typed_run<std::uint64_t>(remaining);
            case 'd':
                return parse_ubjson_typed_run<float>(remaining);
            case 'D':
@@ -3706,7 +3714,7 @@ class binary_reader
        // if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata):
        // {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]}

-        if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
+        if ((size_and_type.second & (1 << 8)) != 0)
        {
            size_and_type.second &= ~(static_cast<char_int_type>(1) << 8);  // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker

@@ -3765,7 +3773,7 @@ class binary_reader
        }

        // If BJData type marker is 'B' decode as binary
-        if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B')
+        if (size_and_type.first != npos && size_and_type.second == 'B')
        {
            binary_t result;
            return get_binary(size_and_type.first, result) && sax->binary(result);
@@ -3814,7 +3822,7 @@ class binary_reader
        }

        // do not accept ND-array size in objects in BJData
-        if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
+        if ((size_and_type.second & (1 << 8)) != 0)
        {
            return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object");
        }
@@ -5095,8 +5103,9 @@ class binary_reader
  JSON_PRIVATE_UNLESS_TESTED:
    /*!
    @brief check whether @a marker is a type marker that may follow '$'
-    UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C
-    (BJData's excluded markers are rejected separately).
+    UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C.
+    The markers BJData excludes (F H N S T Z [ {) are the ones
+    binary_writer::is_bjdata_excluded_type_marker() lists; keep the two in sync.
    */
    bool is_valid_optimized_type(const char_int_type marker) const noexcept
    {
@@ -5131,20 +5140,6 @@ class binary_reader
        }
    }

-    /*!
-    @brief whether @a marker is excluded from BJData's optimized ND-array types
-    @return whether @a marker is one of 'F', 'H', 'N', 'S', 'T', 'Z', '[', '{'
-
-    Mirrors binary_writer's @ref binary_writer::is_bjdata_excluded_type_marker
-    "is_bjdata_excluded_type_marker()`, which encodes the same list the other
-    way; keep the two in sync.
-    */
-    static constexpr bool is_bjd_excluded_optimized_type(const char_int_type marker) noexcept
-    {
-        return marker == '[' || marker == '{' || marker == 'S' || marker == 'H'
-               || marker == 'T' || marker == 'F' || marker == 'N' || marker == 'Z';
-    }
-
    /*!
    @brief look up the ND-array element type name for a BJData dtype marker
    @return the type name ("uint8", "int8", ...), or nullptr if @a marker does
@@ -904,7 +904,8 @@ class json_sax_dom_parser : public json_sax_dom_error_state
            {
                hashes.push_back(hasher(element.first));
            }
-            ordered_data.key_indexes.emplace_back(new object_key_index(std::move(hashes)));
+            std::unique_ptr<object_key_index> index(new object_key_index(std::move(hashes))); // NOLINT(cppcoreguidelines-owning-memory,modernize-make-unique): std::make_unique is C++14
+            ordered_data.key_indexes.push_back(std::move(index));
            f.ordered.index = ordered_data.key_indexes.back().get();
        }

@@ -3126,22 +3126,19 @@ class binary_writer
                // negative fixnum
                write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
            }
-            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)() &&
-                     j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
+            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)())
            {
                // int 8
                oa.write_character(to_char_type(0xD0));
                write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
            }
-            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)() &&
-                     j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
+            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)())
            {
                // int 16
                oa.write_character(to_char_type(0xD1));
                write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer));
            }
-            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)() &&
-                     j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
+            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)())
            {
                // int 32
                oa.write_character(to_char_type(0xD2));
@@ -4057,7 +4054,7 @@ class binary_writer
    Containers, strings, high-precision numbers, booleans and null cannot be
    declared as the single type of an optimized container in BJData; such a
    container is written unoptimized. The reader rejects them with the same
-    list (binary_reader::is_bjd_excluded_optimized_type()).
+    list (binary_reader::is_valid_optimized_type()).
    */
    static constexpr bool is_bjdata_excluded_type_marker(const CharType marker) noexcept
    {
@@ -116,6 +116,10 @@ inline void copy_short(CharType* dst, const CharType* src, const std::size_t len
 /// the number of characters an empty vector is grown to by output_vector_sink
 constexpr std::size_t min_vector_sink_capacity = 64;

+/// the number of characters output_vector_sink makes writable ahead of its
+/// position at a time: 64 KiB of the vector's capacity, not all of it
+constexpr std::size_t vector_sink_window = 65536;
+
 /// @brief non-virtual output sink writing into a std::vector
 ///
 /// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
@@ -129,7 +133,8 @@ constexpr std::size_t min_vector_sink_capacity = 64;
 /// encoding them: insert() is an out-of-line call into a routine that can insert
 /// anywhere, and resize() is one as well with libstdc++. The sink therefore keeps
 /// its own write position: it grows the vector geometrically ahead of the
-/// position and copies each write to it, so a write is a capacity check and a
+/// position (making @ref vector_sink_window characters of it writable at a
+/// time) and copies each write to it, so a write is a capacity check and a
 /// short copy. The destructor shrinks the vector to the bytes actually written,
 /// also when a write throws; shrinking a vector never allocates. The sink owns
 /// that trim, so it can be moved but not copied, and the vector must not be used
@@ -277,9 +282,17 @@ class output_vector_sink
        // reallocate for every few bytes
        const std::size_t written = position();
        const std::size_t needed = written + length;
-        v->resize((std::max)(min_vector_sink_capacity, needed <= v->capacity()
-                             ? v->capacity()
-                             : (std::max)(needed, v->capacity() * 2)));
+        const std::size_t capacity = (std::max)(min_vector_sink_capacity, needed <= v->capacity()
+                                                ? v->capacity()
+                                                : (std::max)(needed, v->capacity() * 2));
+        v->reserve(capacity);
+        // resize() zeroes the characters it adds. Zeroing all of a doubled
+        // capacity, megabytes of a fresh allocation, at once faults in its
+        // pages and evicts them from the cache before the writer gets there,
+        // so its stores miss again; a window ahead of the position is still
+        // cached when they come (the binary writers on a long array of
+        // numbers: 4-26 % faster on x86, 1-2.5 % on Apple M1)
+        v->resize((std::min)(capacity, (std::max)(needed, written + vector_sink_window)));
        cursor = v->data() + written;
        limit = v->data() + v->size();
    }
@@ -8,7 +8,7 @@

 #pragma once

-#include <algorithm> // remove, fill, find, none_of, min
+#include <algorithm> // copy, none_of, min, search
 #include <array> // array
 #include <clocale> // localeconv, lconv
 #include <cmath> // isfinite
@@ -2080,31 +2080,35 @@ class serializer
        // check if the buffer was large enough
        JSON_ASSERT(static_cast<std::size_t>(len) < number_buffer.size());

-        // look up the locale's thousands separator and decimal point now,
-        // matching what snprintf_float() just used (see lexer::get_decimal_point())
+        // look up the locale's decimal point now, matching what
+        // snprintf_float() just used (see get_decimal_point() in
+        // number_parse.hpp); it is a string, which can be longer than one
+        // byte (U+066B in ps_AF, #5993).
+        // A thousands separator needs no handling: %g never groups digits.
        const auto* loc = std::localeconv();
        JSON_ASSERT(loc != nullptr);
-        const char thousands_sep = (loc->thousands_sep == nullptr) ? '\0' : *loc->thousands_sep;
-        const char decimal_point = (loc->decimal_point == nullptr) ? '\0' : *loc->decimal_point;
-
-        // erase thousands separators
-        if (thousands_sep != '\0')
-        {
-            // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::remove returns an iterator, see https://github.com/nlohmann/json/issues/3081
-            const auto end = std::remove(number_buffer.begin(), number_buffer.begin() + len, thousands_sep);
-            std::fill(end, number_buffer.end(), '\0');
-            JSON_ASSERT((end - number_buffer.begin()) <= len);
-            len = (end - number_buffer.begin());
-        }
+        const char* const decimal_point = (loc->decimal_point == nullptr) ? "" : loc->decimal_point;
+        const std::size_t decimal_point_len = std::char_traits<char>::length(decimal_point);

        // convert decimal point to '.'
-        if (decimal_point != '\0' && decimal_point != '.')
+        if ((decimal_point_len > 1) || (decimal_point_len == 1 && decimal_point[0] != '.'))
        {
-            // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::find returns an iterator, see https://github.com/nlohmann/json/issues/3081
-            const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), decimal_point);
-            if (dec_pos != number_buffer.end())
+            const auto first = number_buffer.begin();
+            const auto last = first + len;
+            // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::search returns an iterator, see https://github.com/nlohmann/json/issues/3081
+            const auto dec_pos = std::search(first, last, decimal_point, decimal_point + decimal_point_len);
+            if (dec_pos != last)
            {
                *dec_pos = '.';
+                if (decimal_point_len > 1)
+                {
+                    // move the fraction and exponent behind the '.' (std::copy
+                    // allows the overlap, as the destination is before the source)
+                    // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::copy returns an iterator, see https://github.com/nlohmann/json/issues/3081
+                    const auto end = std::copy(dec_pos + static_cast<std::ptrdiff_t>(decimal_point_len), last, dec_pos + 1);
+                    *end = '\0';
+                    len = end - first;
+                }
            }
        }

@@ -16538,7 +16538,8 @@ class json_sax_dom_parser : public json_sax_dom_error_state
            {
                hashes.push_back(hasher(element.first));
            }
-            ordered_data.key_indexes.emplace_back(new object_key_index(std::move(hashes)));
+            std::unique_ptr<object_key_index> index(new object_key_index(std::move(hashes))); // NOLINT(cppcoreguidelines-owning-memory,modernize-make-unique): std::make_unique is C++14
+            ordered_data.key_indexes.push_back(std::move(index));
            f.ordered.index = ordered_data.key_indexes.back().get();
        }

@@ -18359,8 +18360,8 @@ class binary_reader
    nested document could declare a length that disagrees with where its
    terminator actually falls and quietly hand the bytes in between to the
    enclosing document. A well-formed document is at least 5 bytes (the prefix
-    plus the terminator); the equality also rejects those impossible sizes,
-    since at least 5 bytes are always consumed.
+    plus the terminator); open_bson_document() rejects a smaller size before
+    it is stored, so @a document_size is never negative here.

    @param[in] document_start  value of chars_read before the size prefix
    @param[in] document_size   the declared document size
@@ -18368,7 +18369,7 @@ class binary_reader
    */
    bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size)
    {
-        if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
+        if (JSON_HEDLEY_UNLIKELY(static_cast<std::size_t>(document_size) != chars_read - document_start))
        {
            return bson_document_size_error(document_size, chars_read - document_start);
        }
@@ -19039,7 +19040,7 @@ class binary_reader
            {
                std::uint64_t len{};
                std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_cbor_array(size);
+                return get_cbor_argument(len) && get_container_size(len, size, "array") && enter_cbor_array(size);
            }

            case 0x9F: // array (indefinite length)
@@ -19079,7 +19080,7 @@ class binary_reader
            {
                std::uint64_t len{};
                std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "map") && enter_object(size);
+                return get_cbor_argument(len) && get_container_size(len, size, "map") && enter_object(size);
            }

            case 0xBF: // map (indefinite length)
@@ -19715,7 +19716,7 @@ class binary_reader
    }

    /*!
-    @brief narrow a definite CBOR array/map length to std::size_t
+    @brief narrow a definite CBOR or MessagePack array/map length to std::size_t

    A definite length is rejected if it does not fit in std::size_t or if it
    equals detail::unknown_size(), which is reserved to mark an indefinite-
@@ -19728,7 +19729,7 @@ class binary_reader
    @param[in]  context  "array" or "map", for the error message
    @return whether the length is usable
    */
-    bool get_cbor_container_size(const std::uint64_t len, std::size_t& result, const char* context)
+    bool get_container_size(const std::uint64_t len, std::size_t& result, const char* context)
    {
        if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::size_t>(len) || len == detail::unknown_size()))
        {
@@ -20324,7 +20325,8 @@ class binary_reader
            case 0xDD: // array 32
            {
                std::uint32_t len{};
-                return get_number(len) && enter_msgpack_array(conditional_static_cast<std::size_t>(len));
+                std::size_t size{};
+                return get_number(len) && get_container_size(len, size, "array") && enter_msgpack_array(size);
            }

            case 0xDE: // map 16
@@ -20336,7 +20338,8 @@ class binary_reader
            case 0xDF: // map 32
            {
                std::uint32_t len{};
-                return get_number(len) && enter_object(conditional_static_cast<std::size_t>(len));
+                std::size_t size{};
+                return get_number(len) && get_container_size(len, size, "map") && enter_object(size);
            }

            // negative fixint
@@ -21107,16 +21110,13 @@ class binary_reader
        {
            if (size_and_type.second != 0)
            {
-                if (size_and_type.second != 'N')
+                for (std::size_t i = 0; i < size_and_type.first; ++i)
                {
-                    for (std::size_t i = 0; i < size_and_type.first; ++i)
+                    if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
                    {
-                        if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_value(dimlen, no_ndarray, size_and_type.second)))
-                        {
-                            return false;
-                        }
-                        dim.push_back(dimlen);
+                        return false;
                    }
+                    dim.push_back(dimlen);
                }
            }
            else
@@ -21146,6 +21146,27 @@ class binary_reader
        return true;
    }

+    /*!
+    @brief store the count of an optimized container (or the length of a string)
+
+    A count equal to npos is rejected: npos marks a container whose count is
+    not known, so a count of that value would be read as "no count" (compare
+    get_container_size()). No container can hold that many elements.
+
+    @param[in]  count   the count read from the input
+    @param[out] result  set to @a count unless it is rejected
+    @return whether @a count is usable
+    */
+    bool store_ubjson_count(const std::size_t count, std::size_t& result)
+    {
+        if (JSON_HEDLEY_UNLIKELY(count == npos))
+        {
+            return out_of_range_error(exception_id::container_too_large, "integer value overflow", "size");
+        }
+        result = count;
+        return true;
+    }
+
    /*!
    @brief read a UBJSON/BJData optimized-container count of a signed marker
           type ('i', 'I', 'l', 'L') and narrow it to std::size_t
@@ -21176,8 +21197,7 @@ class binary_reader
        {
            return out_of_range_error(exception_id::container_too_large, "integer value overflow", "size");
        }
-        result = static_cast<std::size_t>(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char
-        return true;
+        return store_ubjson_count(static_cast<std::size_t>(number), result); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char
    }

    /*!
@@ -21254,8 +21274,7 @@ class binary_reader
                {
                    return false;
                }
-                result = conditional_static_cast<std::size_t>(number);
-                return true;
+                return store_ubjson_count(conditional_static_cast<std::size_t>(number), result);
            }

            case 'M':
@@ -21273,8 +21292,7 @@ class binary_reader
                {
                    return out_of_range_error(exception_id::container_too_large, "integer value overflow", "size");
                }
-                result = detail::conditional_static_cast<std::size_t>(number);
-                return true;
+                return store_ubjson_count(detail::conditional_static_cast<std::size_t>(number), result);
            }

            case '[':
@@ -21318,11 +21336,9 @@ class binary_reader
                    // follow the documented _ArrayType_, _ArraySize_, _ArrayData_ order
                    if (ndarray_dtype != 0)
                    {
+                        // every optimized type marker is_valid_optimized_type() accepts for BJData has a name
                        const char* type_name = bjd_type_name(ndarray_dtype);
-                        if (JSON_HEDLEY_UNLIKELY(type_name == nullptr))
-                        {
-                            return unexpected_byte("invalid byte", "type");
-                        }
+                        JSON_ASSERT(type_name != nullptr);

                        string_t type_key = "_ArrayType_";
                        string_t type = type_name; // sax->string() takes a reference
@@ -21405,12 +21421,6 @@ class binary_reader
        if (current == '$')
        {
            result.second = get();  // must not ignore 'N', because 'N' maybe the type
-            if (input_format == input_format_t::bjdata
-                    && JSON_HEDLEY_UNLIKELY(is_bjd_excluded_optimized_type(result.second)))
-            {
-                return optimized_type_error();
-            }
-
            if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type")))
            {
                return false;
@@ -21438,7 +21448,7 @@ class binary_reader
            // an ndarray was read here only if the flag flipped; when it was
            // seeded true, get_ubjson_size_value() already rejected the nested
            // dimension vector
-            if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
+            if (is_ndarray && !inside_ndarray)
            {
                result.second |= (1 << 8); // use bit 8 to indicate ndarray, all UBJSON and BJData markers should be ASCII letters
            }
@@ -21448,7 +21458,7 @@ class binary_reader
        if (current == '#')
        {
            const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
-            if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
+            if (is_ndarray && !inside_ndarray)
            {
                return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size");
            }
@@ -21622,7 +21632,6 @@ class binary_reader
    JSON_HEDLEY_NEVER_INLINE
    bool parse_ubjson_typed_run(std::size_t& remaining, const char_int_type type, std::true_type /*bulk*/)
    {
-        const bool bjdata = input_format == input_format_t::bjdata;
        switch (type)
        {
            case 'U':
@@ -21636,11 +21645,11 @@ class binary_reader
            case 'L':
                return parse_ubjson_typed_run<std::int64_t>(remaining);
            case 'u':
-                return !bjdata || parse_ubjson_typed_run<std::uint16_t>(remaining);
+                return parse_ubjson_typed_run<std::uint16_t>(remaining);
            case 'm':
-                return !bjdata || parse_ubjson_typed_run<std::uint32_t>(remaining);
+                return parse_ubjson_typed_run<std::uint32_t>(remaining);
            case 'M':
-                return !bjdata || parse_ubjson_typed_run<std::uint64_t>(remaining);
+                return parse_ubjson_typed_run<std::uint64_t>(remaining);
            case 'd':
                return parse_ubjson_typed_run<float>(remaining);
            case 'D':
@@ -21721,7 +21730,7 @@ class binary_reader
        // if bit-8 of size_and_type.second is set to 1, encode bjdata ndarray as an object in JData annotated array format (https://github.com/NeuroJSON/jdata):
        // {"_ArrayType_" : "typeid", "_ArraySize_" : [n1, n2, ...], "_ArrayData_" : [v1, v2, ...]}

-        if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
+        if ((size_and_type.second & (1 << 8)) != 0)
        {
            size_and_type.second &= ~(static_cast<char_int_type>(1) << 8);  // use bit 8 to indicate ndarray, here we remove the bit to restore the type marker

@@ -21780,7 +21789,7 @@ class binary_reader
        }

        // If BJData type marker is 'B' decode as binary
-        if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B')
+        if (size_and_type.first != npos && size_and_type.second == 'B')
        {
            binary_t result;
            return get_binary(size_and_type.first, result) && sax->binary(result);
@@ -21829,7 +21838,7 @@ class binary_reader
        }

        // do not accept ND-array size in objects in BJData
-        if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
+        if ((size_and_type.second & (1 << 8)) != 0)
        {
            return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object");
        }
@@ -23110,8 +23119,9 @@ class binary_reader
  JSON_PRIVATE_UNLESS_TESTED:
    /*!
    @brief check whether @a marker is a type marker that may follow '$'
-    UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C
-    (BJData's excluded markers are rejected separately).
+    UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C.
+    The markers BJData excludes (F H N S T Z [ {) are the ones
+    binary_writer::is_bjdata_excluded_type_marker() lists; keep the two in sync.
    */
    bool is_valid_optimized_type(const char_int_type marker) const noexcept
    {
@@ -23146,20 +23156,6 @@ class binary_reader
        }
    }

-    /*!
-    @brief whether @a marker is excluded from BJData's optimized ND-array types
-    @return whether @a marker is one of 'F', 'H', 'N', 'S', 'T', 'Z', '[', '{'
-
-    Mirrors binary_writer's @ref binary_writer::is_bjdata_excluded_type_marker
-    "is_bjdata_excluded_type_marker()`, which encodes the same list the other
-    way; keep the two in sync.
-    */
-    static constexpr bool is_bjd_excluded_optimized_type(const char_int_type marker) noexcept
-    {
-        return marker == '[' || marker == '{' || marker == 'S' || marker == 'H'
-               || marker == 'T' || marker == 'F' || marker == 'N' || marker == 'Z';
-    }
-
    /*!
    @brief look up the ND-array element type name for a BJData dtype marker
    @return the type name ("uint8", "int8", ...), or nullptr if @a marker does
@@ -27491,6 +27487,10 @@ inline void copy_short(CharType* dst, const CharType* src, const std::size_t len
 /// the number of characters an empty vector is grown to by output_vector_sink
 constexpr std::size_t min_vector_sink_capacity = 64;

+/// the number of characters output_vector_sink makes writable ahead of its
+/// position at a time: 64 KiB of the vector's capacity, not all of it
+constexpr std::size_t vector_sink_window = 65536;
+
 /// @brief non-virtual output sink writing into a std::vector
 ///
 /// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
@@ -27504,7 +27504,8 @@ constexpr std::size_t min_vector_sink_capacity = 64;
 /// encoding them: insert() is an out-of-line call into a routine that can insert
 /// anywhere, and resize() is one as well with libstdc++. The sink therefore keeps
 /// its own write position: it grows the vector geometrically ahead of the
-/// position and copies each write to it, so a write is a capacity check and a
+/// position (making @ref vector_sink_window characters of it writable at a
+/// time) and copies each write to it, so a write is a capacity check and a
 /// short copy. The destructor shrinks the vector to the bytes actually written,
 /// also when a write throws; shrinking a vector never allocates. The sink owns
 /// that trim, so it can be moved but not copied, and the vector must not be used
@@ -27652,9 +27653,17 @@ class output_vector_sink
        // reallocate for every few bytes
        const std::size_t written = position();
        const std::size_t needed = written + length;
-        v->resize((std::max)(min_vector_sink_capacity, needed <= v->capacity()
-                             ? v->capacity()
-                             : (std::max)(needed, v->capacity() * 2)));
+        const std::size_t capacity = (std::max)(min_vector_sink_capacity, needed <= v->capacity()
+                                                ? v->capacity()
+                                                : (std::max)(needed, v->capacity() * 2));
+        v->reserve(capacity);
+        // resize() zeroes the characters it adds. Zeroing all of a doubled
+        // capacity, megabytes of a fresh allocation, at once faults in its
+        // pages and evicts them from the cache before the writer gets there,
+        // so its stores miss again; a window ahead of the position is still
+        // cached when they come (the binary writers on a long array of
+        // numbers: 4-26 % faster on x86, 1-2.5 % on Apple M1)
+        v->resize((std::min)(capacity, (std::max)(needed, written + vector_sink_window)));
        cursor = v->data() + written;
        limit = v->data() + v->size();
    }
@@ -31110,22 +31119,19 @@ class binary_writer
                // negative fixnum
                write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
            }
-            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)() &&
-                     j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
+            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)())
            {
                // int 8
                oa.write_character(to_char_type(0xD0));
                write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
            }
-            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)() &&
-                     j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
+            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)())
            {
                // int 16
                oa.write_character(to_char_type(0xD1));
                write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer));
            }
-            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)() &&
-                     j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
+            else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)())
            {
                // int 32
                oa.write_character(to_char_type(0xD2));
@@ -32041,7 +32047,7 @@ class binary_writer
    Containers, strings, high-precision numbers, booleans and null cannot be
    declared as the single type of an optimized container in BJData; such a
    container is written unoptimized. The reader rejects them with the same
-    list (binary_reader::is_bjd_excluded_optimized_type()).
+    list (binary_reader::is_valid_optimized_type()).
    */
    static constexpr bool is_bjdata_excluded_type_marker(const CharType marker) noexcept
    {
@@ -33446,7 +33452,7 @@ NLOHMANN_JSON_NAMESPACE_END



-#include <algorithm> // remove, fill, find, none_of, min
+#include <algorithm> // copy, none_of, min, search
 #include <array> // array
 #include <clocale> // localeconv, lconv
 #include <cmath> // isfinite
@@ -37300,31 +37306,35 @@ class serializer
        // check if the buffer was large enough
        JSON_ASSERT(static_cast<std::size_t>(len) < number_buffer.size());

-        // look up the locale's thousands separator and decimal point now,
-        // matching what snprintf_float() just used (see lexer::get_decimal_point())
+        // look up the locale's decimal point now, matching what
+        // snprintf_float() just used (see get_decimal_point() in
+        // number_parse.hpp); it is a string, which can be longer than one
+        // byte (U+066B in ps_AF, #5993).
+        // A thousands separator needs no handling: %g never groups digits.
        const auto* loc = std::localeconv();
        JSON_ASSERT(loc != nullptr);
-        const char thousands_sep = (loc->thousands_sep == nullptr) ? '\0' : *loc->thousands_sep;
-        const char decimal_point = (loc->decimal_point == nullptr) ? '\0' : *loc->decimal_point;
-
-        // erase thousands separators
-        if (thousands_sep != '\0')
-        {
-            // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::remove returns an iterator, see https://github.com/nlohmann/json/issues/3081
-            const auto end = std::remove(number_buffer.begin(), number_buffer.begin() + len, thousands_sep);
-            std::fill(end, number_buffer.end(), '\0');
-            JSON_ASSERT((end - number_buffer.begin()) <= len);
-            len = (end - number_buffer.begin());
-        }
+        const char* const decimal_point = (loc->decimal_point == nullptr) ? "" : loc->decimal_point;
+        const std::size_t decimal_point_len = std::char_traits<char>::length(decimal_point);

        // convert decimal point to '.'
-        if (decimal_point != '\0' && decimal_point != '.')
+        if ((decimal_point_len > 1) || (decimal_point_len == 1 && decimal_point[0] != '.'))
        {
-            // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::find returns an iterator, see https://github.com/nlohmann/json/issues/3081
-            const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), decimal_point);
-            if (dec_pos != number_buffer.end())
+            const auto first = number_buffer.begin();
+            const auto last = first + len;
+            // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::search returns an iterator, see https://github.com/nlohmann/json/issues/3081
+            const auto dec_pos = std::search(first, last, decimal_point, decimal_point + decimal_point_len);
+            if (dec_pos != last)
            {
                *dec_pos = '.';
+                if (decimal_point_len > 1)
+                {
+                    // move the fraction and exponent behind the '.' (std::copy
+                    // allows the overlap, as the destination is before the source)
+                    // NOLINTNEXTLINE(readability-qualified-auto,llvm-qualified-auto): std::copy returns an iterator, see https://github.com/nlohmann/json/issues/3081
+                    const auto end = std::copy(dec_pos + static_cast<std::ptrdiff_t>(decimal_point_len), last, dec_pos + 1);
+                    *end = '\0';
+                    len = end - first;
+                }
            }
        }

@@ -97,3 +97,23 @@ TEST_CASE("CBOR")
        }
    }
 }
+
+TEST_CASE("MessagePack")
+{
+    SECTION("parse errors")
+    {
+        SECTION("array/map size equal to the unknown-size sentinel")
+        {
+            // on 32-bit platforms an array 32/map 32 length of 0xFFFFFFFF aliases unknown_size()
+            std::vector<uint8_t> const varr = {0xDD, 0xFF, 0xFF, 0xFF, 0xFF};
+            std::vector<uint8_t> const vmap = {0xDF, 0xFF, 0xFF, 0xFF, 0xFF};
+
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(varr), "[json.exception.out_of_range.408] syntax error while parsing MessagePack size: excessive array size", json::out_of_range&);
+            CHECK(json::from_msgpack(varr, true, false).is_discarded());
+
+            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(vmap), "[json.exception.out_of_range.408] syntax error while parsing MessagePack size: excessive map size", json::out_of_range&);
+            CHECK(json::from_msgpack(vmap, true, false).is_discarded());
+        }
+    }
+}
@@ -50,6 +50,14 @@ std::vector<json> test_values()
        large_payloads.push_back(json::binary(std::vector<std::uint8_t>(static_cast<std::size_t>(100 * i), static_cast<std::uint8_t>(i))));
    }

+    // several times vector_sink_window characters in every format: the vector
+    // sink makes the capacity it grows to writable a window at a time
+    json many_windows = json::array();
+    for (std::uint64_t i = 0; i < 30000; ++i)
+    {
+        many_windows.push_back((i * 0x9E3779B97F4A7C15u) >> 1u);
+    }
+
    return
    {
        json(nullptr), json(true), json(false),
@@ -61,7 +69,7 @@ std::vector<json> test_values()
        json::array(), json::object(),
        json::array({1, 2, 3}), json({{"a", 1}, {"b", nullptr}}),
        json({{"nested", {{"deep", json::array({1, "two", 3.0, nullptr})}}}}),
-        big_array, big_object, large_payloads,
+        big_array, big_object, large_payloads, many_windows,
        json::binary(std::vector<std::uint8_t>(5000, 0x42))
    };
 }
@@ -78,23 +78,25 @@ TEST_CASE("BJData")
 {
    SECTION("binary_reader BJData lookup tables")
    {
-        // both lookups are static member functions
+        // bjd_type_name() is static; is_valid_optimized_type() needs a BJData reader
        std::vector<std::uint8_t> const data;
        using reader_t = nlohmann::detail::binary_reader<json, decltype(nlohmann::detail::input_adapter(data))>;

-        // the excluded optimized-type markers must match binary_writer's
-        // is_bjdata_excluded_type_marker(), which encodes the same 8 markers
+        // the markers BJData excludes as optimized types must match
+        // binary_writer's is_bjdata_excluded_type_marker(), which encodes the
+        // same 8 markers
+        const reader_t reader(nlohmann::detail::input_adapter(data), json::input_format_t::bjdata);
        for (const char marker :
-                {'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z'
+                {'[', '{', 'S', 'H', 'T', 'F', 'N', 'Z', 'x'
                })
        {
-            CHECK(reader_t::is_bjd_excluded_optimized_type(static_cast<unsigned char>(marker)));
+            CHECK(!reader.is_valid_optimized_type(static_cast<unsigned char>(marker)));
        }
        for (const char marker :
-                {'U', 'i', 'u', 'I', 'm', 'l', 'M', 'L', 'd', 'D', 'C', 'B', 'x'
+                {'U', 'i', 'u', 'I', 'm', 'l', 'M', 'L', 'd', 'D', 'C', 'B', 'h'
                })
        {
-            CHECK(!reader_t::is_bjd_excluded_optimized_type(static_cast<unsigned char>(marker)));
+            CHECK(reader.is_valid_optimized_type(static_cast<unsigned char>(marker)));
        }

        // every dtype marker must round-trip to its ND-array type name
@@ -3237,14 +3239,34 @@ TEST_CASE("BJData")
 #endif
                CHECK(json::from_bjdata(vM, true, false).is_discarded());

-#if SIZE_MAX != 0xffffffff
-                CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vMX), "[json.exception.out_of_range.408] syntax error while parsing BJData size: excessive ndarray size caused overflow", json::out_of_range&);
-#else
+                // the first dimension is 2^64-1, which is rejected as a count (see below)
                CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vMX), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
-#endif
                CHECK(json::from_bjdata(vMX, true, false).is_discarded());
            }

+            SECTION("optimized array: count equal to SIZE_MAX")
+            {
+                // the reader marks a container without a count by SIZE_MAX, so
+                // a count of that value must not be read as "no count"
+                // (it used to parse as [1])
+                json _;
+#if SIZE_MAX != 0xffffffff
+                std::vector<uint8_t> const vM = {'[', '#', 'M', 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
+                CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
+                CHECK(json::from_bjdata(vM, true, false).is_discarded());
+                // one less is an ordinary excessive count
+                std::vector<uint8_t> const vM1 = {'[', '#', 'M', 0xFE, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
+                CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vM1), "[json.exception.out_of_range.408] excessive array size: 18446744073709551614", json::out_of_range&);
+#else
+                std::vector<uint8_t> const vm = {'[', '#', 'm', 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
+                CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vm), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
+                CHECK(json::from_bjdata(vm, true, false).is_discarded());
+                std::vector<uint8_t> const vL = {'[', '#', 'L', 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 'i', 0x01, ']'};
+                CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vL), "[json.exception.out_of_range.408] syntax error while parsing BJData size: integer value overflow", json::out_of_range&);
+                CHECK(json::from_bjdata(vL, true, false).is_discarded());
+#endif
+            }
+
            SECTION("optimized array: integer value overflow")
            {
 #if SIZE_MAX == 0xffffffff
@@ -397,6 +397,14 @@ TEST_CASE("locale with a multi-byte decimal point")

        // a value the locale-independent paths convert is not affected
        CHECK(json::parse("12.5") == 12.5);
+
+        // dump() of a long double that takes the snprintf path replaces the
+        // whole decimal point, not only its first byte (#5993)
+        CHECK(long_double_json(1.5L).dump() == "1.5");
+        CHECK(long_double_json(2.0L).dump() == "2.0");
+        CHECK(long_double_json(-0.25L).dump() == "-0.25");
+        CHECK(long_double_json::parse(long_double_json(1.5e-7L).dump()) == long_double_json(1.5e-7L));
+        CHECK(long_double_json::parse(ld.dump()) == expected_long_double);
    }
    if (!tested)
    {
@@ -1009,7 +1009,7 @@ TEST_CASE("regression test - excessive binary container size honors allow_except
    // allow_exceptions=true (the default) must still throw exactly as before.
    // The exact message text is not checked here: on platforms where
    // std::size_t is 32-bit, the CBOR reader's own length-narrowing check
-    // (get_cbor_container_size(), unrelated to this fix) intercepts a
+    // (get_container_size(), unrelated to this fix) intercepts a
    // declared length of 2^63 before it ever reaches the check this test
    // targets, with different (but equally valid, and already correct)
    // wording -- see unit-cbor.cpp for coverage of that message.
@@ -2385,6 +2385,18 @@ TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
        }
    }

+    SECTION("a count equal to SIZE_MAX is rejected")
+    {
+        // the reader marks a container without a count by SIZE_MAX, so a
+        // count of that value must not be read as "no count"; UBJSON's
+        // largest count type (int64) can only reach it where size_t is 32 bits
+#if SIZE_MAX == 0xffffffff
+        const std::vector<uint8_t> input = {'[', '#', 'L', 0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0xFF, 'i', 0x01, ']'};
+        CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), "[json.exception.out_of_range.408] syntax error while parsing UBJSON size: integer value overflow", json::out_of_range&);
+        CHECK(json::from_ubjson(input, true, false).is_discarded());
+#endif
+    }
+
    SECTION("ordinary counts are unaffected")
    {
        CHECK(json::from_ubjson(std::vector<uint8_t>({'[', '$', 'Z', '#', 'i', 3})) == json({nullptr, nullptr, nullptr}));
Author	SHA1	Message	Date
Niels Lohmann	3558619847	Docs: compile times of JSON_HAS_STD_FORMAT=0 and JSON_HAS_RANGES=0 Add sections on switching off std::format and ranges support to the compile-times page, with measurements (Apple clang, GCC 16, -std=c++20), the functionality that is lost, and chart rows (issue #5294). Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-11-05 09:18:04 +01:00
Niels Lohmann	e48800f09d	Binary reader and writer: remove checks that invariants make dead (#6001 ) * Binary reader and writer: remove checks that invariants make dead Branch coverage of the test suite and of 24k mutated inputs showed these conditions as never taken; each is implied by a check made earlier on the same path: - check_bson_document_size: `document_size < 0` (the size was rejected below 5 before it was stored) - get_ubjson_ndarray_size: `second != 'N'` ('N' is rejected as a type marker before `second` can be set) - bjd_type_name() == nullptr for an ndarray dtype: every marker is_valid_optimized_type() accepts for BJData has a name (now an assert) - the early is_bjd_excluded_optimized_type() check in get_ubjson_size_type(): is_valid_optimized_type() rejects the same eight markers with the same error - `input_format == bjdata` next to `is_ndarray`, bit 8 of the type, or a 'B'/'u'/'m'/'M' marker: these only arise for BJData - `first != npos` next to bit 8: the bit is only set after a count - MessagePack writer: the `<= max` halves of the negative int8/16/32 ladder (the ladder is in the `else` of `n >= 0`) No change in behaviour: a build that aborts on each removed side passes the test suite and the fuzz corpora, and a 1.6M-input differential run of the reader against the original headers found no difference. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove is_bjd_excluded_optimized_type(), which has no caller left; fix two comments is_valid_optimized_type() rejects the markers BJData excludes, so the reader no longer needs its own list; the test now checks is_valid_optimized_type() of a BJData reader against the writer's list. The BSON size comment now says that open_bson_document() rejects a size below 5. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the BJData reader in the lookup test const (misc-const-correctness) Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-11-04 23:42:17 +01:00
dependabot[bot]	20fe34ceeb	build(deps): bump lukka/get-cmake from 4.4.4 to 4.4.5 (#6024 ) Bumps [lukka/get-cmake](https://github.com/lukka/get-cmake) from 4.4.4 to 4.4.5. - [Release notes](https://github.com/lukka/get-cmake/releases) - [Changelog](https://github.com/lukka/get-cmake/blob/main/RELEASE_PROCESS.md) - [Commits](https://github.com/lukka/get-cmake/compare/a89e43e22a96abafcd7a62db652a65ecd33052c9...f86dd60acae0d132056913e248bdabf0e0580b7a) --- updated-dependencies: - dependency-name: lukka/get-cmake dependency-version: 4.4.5 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-11-04 23:38:57 +01:00
Christoph Grüninger	1da8735154	Fix leak and crash when an allocation throws during parsing (#6023 ) json_sax_dom_parser::insert_key_with_index: own the new object_key_index in a unique_ptr before key_indexes may reallocate; a throwing reallocation leaked it. Signed-off-by: Christoph Grüninger <foss@grueninger.de>	2026-11-04 23:38:47 +01:00
Niels Lohmann	a49242f088	Docs: reading and writing files; patch_inplace() is not atomic (#6017 )	2026-11-04 17:46:20 +01:00
Niels Lohmann	42825fba07	Binary writers: zero only a window ahead of the vector sink's position (#6020 ) output_vector_sink grew the vector by resizing it to its whole new capacity. resize() zeroes the characters it adds, so every doubling zeroed megabytes of a fresh allocation at once: that faults in all its pages and evicts them from the cache before the writer reaches them, and the writer's stores then miss again. The sink now reserves the same capacity as before but resizes the vector only to 64 KiB (vector_sink_window) ahead of the position, which is still cached when the writer's stores come. to_cbor(), to_msgpack(), to_ubjson(), to_bjdata() and to_bon8() of an array of one million numbers into a fresh vector: 4-26 % faster on x86 (Skylake laptop with Apple clang 13, Haswell VPS with GCC 16 and Clang 23), 1-2.5 % on Apple M1 Max; other documents unchanged (within 0.4 % in reruns). The capacity and growth of the vector are as before. Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-11-04 17:13:25 +01:00
Niels Lohmann	efc437ef6e	UBJSON/BJData/MessagePack reader: reject a container count equal to SIZE_MAX (#6002 ) * UBJSON/BJData reader: reject a container count equal to SIZE_MAX The reader marks a container whose count is not known by npos, which is SIZE_MAX. A count of exactly that value was stored unchanged and then read as "no count", so the BJData input [ # M ff ff ff ff ff ff ff ff i 01 ] parsed as [1] instead of throwing out_of_range.408 as a count of 2^64-2 does. Where size_t is 32 bits, the same holds for 'm' and 'L' counts of 2^32-1, in UBJSON too. The counts of 'm', 'M' and the signed types now go through store_ubjson_count(), which rejects npos with "integer value overflow" (the message a count beyond size_t already gets), as the CBOR reader does in get_cbor_container_size(). An ND-array dimension of 2^64-1 gets the same message now, where it was "excessive ndarray size caused overflow" (the product check) on 64-bit platforms. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * MessagePack reader: reject an array/map 32 length equal to SIZE_MAX On a platform with a 32-bit std::size_t, an array 32 or map 32 length of 0xFFFFFFFF equals unknown_size(), so start_array()/start_object() were told the size was unknown and skipped the max_size() check. Route both through the CBOR length check, renamed get_container_size(), which already rejects that value. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>	2026-11-04 17:13:16 +01:00
Niels Lohmann	57b898ef99	Replace a multibyte decimal point in dump() of a long double (#5998 )	2026-11-03 19:17:41 +01:00
dependabot[bot]	92707f43f2	build(deps): bump step-security/harden-runner from 2.22.2 to 2.22.3 (#6021 )	2026-11-03 19:17:27 +01:00