CBOR reader: read runs of scalar array elements in one loop

When a definite-length CBOR array opens and the input is contiguous, read
the integers, floats, booleans, and nulls that follow in a tight loop
instead of going through parse_cbor_internal's container logic and
parse_cbor_value's dispatch for every element. The loop stops at any other
kind of value and when fewer than 9 bytes are left; those values are read
by the general path exactly as before. Non-contiguous inputs are not
affected (tag dispatch on the bulk-scan capability).

Also pass a member empty string as the number source to number_float
instead of constructing a std::string for every float.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/CONTRIBUTING.md

@@ -154,11 +154,6 @@ and each page's title (H1) and declaration by
 [`docs/docset/generate_docset.py`](https://github.com/nlohmann/json/blob/develop/docs/docset/generate_docset.py);
 `make build` reports API pages that cannot be classified.
 
-A pull request that changes only the documentation, Markdown files, GitHub metadata, or license files skips the
-compiler workflows (Ubuntu, macOS, Windows); the documentation and license checks still run. See
-[`tools/check_path_filters`](https://github.com/nlohmann/json/tree/develop/tools/check_path_filters) for the list of
-skipped files and what to do when a compiler job starts reading one of them.
-
 ### Amalgamate the source code
 
 The single-header files

.github/workflows/macos.yml

@@ -7,25 +7,6 @@ on:
       - master
       - release/*
   pull_request:
-    # Skip pull requests that change only files no job here reads. The list is checked by
-    # tools/check_path_filters/check_path_filters.py (repository_checks.yml); change both together.
-    paths:
-      - '**'
-      - '!docs/**'
-      - '!**.md'
-      - '!.github/*'
-      - '!.github/ISSUE_TEMPLATE/**'
-      - '!.github/external_ci/**'
-      - '!.github/workflows/**'
-      - '!.reuse/**'
-      - '!LICENSES/**'
-      - '!CITATION.cff'
-      - '!REUSE.toml'
-      # read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
-      - 'docs/mkdocs/docs/integration/package_managers.md'
-      # built by the swiftpm job (macos.yml)
-      - 'docs/mkdocs/docs/integration/swift/**'
-      - '.github/workflows/macos.yml'
   workflow_dispatch:
 
 concurrency:

.github/workflows/repository_checks.yml

@@ -1,78 +0,0 @@
-name: Repository checks
-
-# Checks that concern files the compiler workflows (ubuntu.yml, macos.yml,
-# windows.yml) skip with their path filter: the documentation, the license
-# information, and the path filter itself. This workflow has no path filter.
-on:
-  push:
-    branches:
-      - develop
-      - master
-      - release/*
-  pull_request:
-  workflow_dispatch:
-
-permissions:
-  contents: read
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
-  cancel-in-progress: true
-
-jobs:
-  check_path_filters:
-    runs-on: ubuntu-26.04
-    steps:
-      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
-        with:
-          egress-policy: audit
-
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-      - name: Check the path filters of the compiler workflows
-        run: python3 tools/check_path_filters/check_path_filters.py
-
-  ci_reuse_compliance:
-    runs-on: ubuntu-26.04
-    steps:
-      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
-        with:
-          egress-policy: audit
-
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-      - name: Get latest CMake and ninja
-        uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
-      - name: Run CMake
-        run: cmake -S . -B build -DJSON_CI=On
-      - name: Build
-        run: cmake --build build --target ci_reuse_compliance
-
-  ci_test_documentation:
-    runs-on: ubuntu-26.04
-    strategy:
-      matrix:
-        target: [ci_test_examples, ci_test_build_documentation, ci_test_documentation_mermaid]
-    steps:
-      - name: Harden Runner
-        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
-        with:
-          egress-policy: audit
-
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-          # the git-revision-date-localized plugin needs the history; a shallow clone makes the strict build fail
-          fetch-depth: ${{ matrix.target == 'ci_test_build_documentation' && '0' || '1' }}
-      - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
-        if: matrix.target == 'ci_test_documentation_mermaid'
-        with:
-          node-version: 24
-      - name: Run CMake
-        run: cmake -S . -B build -DJSON_CI=On
-      - name: Build
-        run: cmake --build build --target ${{ matrix.target }}

.github/workflows/ubuntu.yml

@@ -7,25 +7,6 @@ on:
       - master
       - release/*
   pull_request:
-    # Skip pull requests that change only files no job here reads. The list is checked by
-    # tools/check_path_filters/check_path_filters.py (repository_checks.yml); change both together.
-    paths:
-      - '**'
-      - '!docs/**'
-      - '!**.md'
-      - '!.github/*'
-      - '!.github/ISSUE_TEMPLATE/**'
-      - '!.github/external_ci/**'
-      - '!.github/workflows/**'
-      - '!.reuse/**'
-      - '!LICENSES/**'
-      - '!CITATION.cff'
-      - '!REUSE.toml'
-      # read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
-      - 'docs/mkdocs/docs/integration/package_managers.md'
-      # built by the swiftpm job (macos.yml)
-      - 'docs/mkdocs/docs/integration/swift/**'
-      - '.github/workflows/ubuntu.yml'
   workflow_dispatch:
 
 permissions:
@@ -128,7 +109,7 @@ jobs:
     runs-on: ubuntu-26.04
     strategy:
       matrix:
-        target: [ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_test_valgrind]
+        target: [ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind]
     steps:
       - name: Harden Runner
         uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
@@ -468,6 +449,31 @@ jobs:
       - name: Build
         run: cmake --build build
 
+  ci_test_documentation:
+    runs-on: ubuntu-26.04
+    strategy:
+      matrix:
+        target: [ci_test_examples, ci_test_build_documentation, ci_test_documentation_mermaid]
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+          # the git-revision-date-localized plugin needs the history; a shallow clone makes the strict build fail
+          fetch-depth: ${{ matrix.target == 'ci_test_build_documentation' && '0' || '1' }}
+      - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
+        if: matrix.target == 'ci_test_documentation_mermaid'
+        with:
+          node-version: 24
+      - name: Run CMake
+        run: cmake -S . -B build -DJSON_CI=On
+      - name: Build
+        run: cmake --build build --target ${{ matrix.target }}
+
   # Native Linux ARM64 runner with the latest GCC. The ubuntu-24.04-arm label is
   # only available for public repositories.
   ci_test_arm64:

.github/workflows/windows.yml

@@ -7,25 +7,6 @@ on:
       - master
       - release/*
   pull_request:
-    # Skip pull requests that change only files no job here reads. The list is checked by
-    # tools/check_path_filters/check_path_filters.py (repository_checks.yml); change both together.
-    paths:
-      - '**'
-      - '!docs/**'
-      - '!**.md'
-      - '!.github/*'
-      - '!.github/ISSUE_TEMPLATE/**'
-      - '!.github/external_ci/**'
-      - '!.github/workflows/**'
-      - '!.reuse/**'
-      - '!LICENSES/**'
-      - '!CITATION.cff'
-      - '!REUSE.toml'
-      # read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
-      - 'docs/mkdocs/docs/integration/package_managers.md'
-      # built by the swiftpm job (macos.yml)
-      - 'docs/mkdocs/docs/integration/swift/**'
-      - '.github/workflows/windows.yml'
   workflow_dispatch:
 
 permissions:

include/nlohmann/detail/input/binary_reader.hpp

@@ -741,11 +741,13 @@ class binary_reader
     bool get_cbor_negative_integer()
     {
         NumberType number{};
-        if (JSON_HEDLEY_UNLIKELY(!get_number(number)))
-        {
-            return false;
-        }
+        return get_number(number) && emit_cbor_negative_integer(number);
+    }
 
+    /// pass the CBOR negative integer -1 - @a number to the SAX parser
+    template<typename NumberType>
+    bool emit_cbor_negative_integer(const NumberType number)
+    {
         // the value is -1 - number, which fits into number_integer_t
         // whenever number does; the outer cast undoes the integral promotion
         // for number_integer_t types narrower than int
@@ -970,7 +972,7 @@ class binary_reader
             case 0x95:
             case 0x96:
             case 0x97:
-                return enter_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
+                return enter_cbor_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
 
             case 0x98: // array (one-byte uint8_t for n follows)
             case 0x99: // array (two-byte uint16_t for n follow)
@@ -979,7 +981,7 @@ class binary_reader
             {
                 std::uint64_t len{};
                 std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_array(size);
+                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_cbor_array(size);
             }
 
             case 0x9F: // array (indefinite length)
@@ -1676,6 +1678,171 @@ class binary_reader
         return true;
     }
 
+    /*!
+    @brief read a big-endian number of type @a NumberType at @a data
+    */
+    template<typename NumberType>
+    NumberType load_big_endian(const unsigned char* data) const noexcept
+    {
+        NumberType result{};
+        std::memcpy(&result, data, sizeof(NumberType));
+        if (is_little_endian)
+        {
+            byte_swap(result);
+        }
+        return result;
+    }
+
+    /*!
+    @brief read the integers, floats, booleans, and nulls that follow in a
+           definite-length array, without the per-element bookkeeping of
+           @ref parse_cbor_internal
+
+    Stops at the end of the array, at any other kind of value, and when fewer
+    than 9 bytes (the longest of these values) are left in the input, leaving
+    those to @ref parse_cbor_value, which reads them the same way.
+
+    @param[in,out] remaining  the number of elements of the array not yet read
+    @return whether the SAX parser accepted the values
+    */
+    bool parse_cbor_scalar_run(std::size_t& remaining)
+    {
+        while (remaining != 0 && ia.bulk_remaining() >= 9)
+        {
+            const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
+            const unsigned char head = data[0];
+            std::size_t size = 1;
+            bool accepted = true;
+            switch (head)
+            {
+                case 0x18:
+                    size = 2;
+                    accepted = emit_unsigned(data[1]);
+                    break;
+                case 0x19:
+                    size = 3;
+                    accepted = emit_unsigned(load_big_endian<std::uint16_t>(data + 1));
+                    break;
+                case 0x1A:
+                    size = 5;
+                    accepted = emit_unsigned(load_big_endian<std::uint32_t>(data + 1));
+                    break;
+                case 0x1B:
+                    size = 9;
+                    accepted = emit_unsigned(load_big_endian<std::uint64_t>(data + 1));
+                    break;
+                case 0x38:
+                    size = 2;
+                    accepted = emit_cbor_negative_integer(data[1]);
+                    break;
+                case 0x39:
+                    size = 3;
+                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint16_t>(data + 1));
+                    break;
+                case 0x3A:
+                    size = 5;
+                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint32_t>(data + 1));
+                    break;
+                case 0x3B:
+                    size = 9;
+                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint64_t>(data + 1));
+                    break;
+                case 0xF4:
+                    accepted = sax->boolean(false);
+                    break;
+                case 0xF5:
+                    accepted = sax->boolean(true);
+                    break;
+                case 0xF6:
+                    accepted = sax->null();
+                    break;
+                case 0xFA:
+                    size = 5;
+                    accepted = emit_float(load_big_endian<float>(data + 1));
+                    break;
+                case 0xFB:
+                    size = 9;
+                    accepted = emit_float(load_big_endian<double>(data + 1));
+                    break;
+                default:
+                    if (head <= 0x17)
+                    {
+                        accepted = sax->number_unsigned(static_cast<number_unsigned_t>(head));
+                    }
+                    else if (head >= 0x20 && head <= 0x37)
+                    {
+                        accepted = sax->number_integer(static_cast<std::int8_t>(0x20 - 1 - head));
+                    }
+                    else
+                    {
+                        // any other value is read by parse_cbor_value
+                        return true;
+                    }
+                    break;
+            }
+            // as if read by get() and get_number(): current is the head byte
+            ia.bulk_skip(size);
+            chars_read += size;
+            current = head;
+            --remaining;
+            if (JSON_HEDLEY_UNLIKELY(!accepted))
+            {
+                return false;
+            }
+        }
+        return true;
+    }
+
+    /*!
+    @brief open a definite-length array and read the scalars it begins with
+           (see @ref parse_cbor_scalar_run)
+
+    Only the start of an array is tried, which covers arrays of numbers
+    (e.g. coordinates) but costs arrays of other values only one test, not one
+    per element.
+
+    @param[in] len  the number of elements
+    @return whether the SAX parser accepted the array and the values read
+    */
+    bool enter_cbor_array(const std::size_t len)
+    {
+        return enter_array(len) && (len == 0 || parse_cbor_array_start(std::integral_constant<bool, bulk_scan> {}));
+    }
+
+    /// read the scalars the array just opened begins with; out of line, so
+    /// that @ref parse_cbor_value stays small enough to be inlined into
+    /// @ref parse_cbor_internal
+    JSON_HEDLEY_NEVER_INLINE
+    bool parse_cbor_array_start(std::true_type /*bulk*/)
+    {
+        return !cbor_scalar_follows() || parse_cbor_scalar_run(container_stack.back().remaining);
+    }
+
+    /// input that is not contiguous: every value is read by parse_cbor_value
+    bool parse_cbor_array_start(std::false_type /*bulk*/) const noexcept
+    {
+        return true;
+    }
+
+    /*!
+    @brief whether @ref parse_cbor_scalar_run can read the next value: one of
+           the heads it handles, with at least 9 bytes of input left
+    */
+    bool cbor_scalar_follows() const
+    {
+        if (ia.bulk_remaining() < 9)
+        {
+            return false;
+        }
+        const auto head = static_cast<unsigned int>(*reinterpret_cast<const unsigned char*>(ia.bulk_data()));
+        // major types 0 and 1 (unsigned and negative integers) up to an
+        // 8-byte argument, or false, true, null, float32, float64 (0xF4,
+        // 0xF5, 0xF6, 0xFA, 0xFB: bits 0, 1, 2, 6, 7 of 0xC7)
+        return ((head & 0xC0u) == 0 && (head & 0x1Fu) <= 0x1Bu)
+               || (head >= 0xF4u && ((0xC7u >> (head - 0xF4u)) & 1u) != 0);
+    }
+
+
     /*!
     @brief read a CBOR value and everything nested inside it
 
@@ -1701,10 +1868,8 @@ class binary_reader
         {
             if (!container_stack.empty())
             {
-                // a copy, not a reference: it must stay valid across the
-                // pop_back() below, which destroys the container_stack element
-                // it would otherwise alias
                 const container_frame top = container_stack.back();
+                const bool is_object = top.is_object;
                 bool at_end = false;
 
                 if (top.remaining != npos)
@@ -1716,7 +1881,7 @@ class binary_reader
                     {
                         // claim the element about to be read
                         --container_stack.back().remaining;
-                        if (top.is_object)
+                        if (is_object)
                         {
                             get();
                         }
@@ -1729,7 +1894,7 @@ class binary_reader
                     // Testing for it consumes a byte, which is the first byte
                     // of the next element when it is not one.
                     at_end = (get() == 0xFF);
-                    fetch = top.is_object;
+                    fetch = is_object;
                 }
 
                 if (at_end)
@@ -1746,7 +1911,7 @@ class binary_reader
                     continue;
                 }
 
-                if (top.is_object)
+                if (is_object)
                 {
                     if (JSON_HEDLEY_UNLIKELY(!skip_cbor_key_tags(tag_handler)))
                     {
@@ -3691,13 +3856,13 @@ class binary_reader
                 return sax->null();
 
             case 0xFB:
-                return sax->number_float(static_cast<number_float_t>(-1.0), "");
+                return sax->number_float(static_cast<number_float_t>(-1.0), no_source);
 
             case 0xFC:
-                return sax->number_float(static_cast<number_float_t>(0.0), "");
+                return sax->number_float(static_cast<number_float_t>(0.0), no_source);
 
             case 0xFD:
-                return sax->number_float(static_cast<number_float_t>(1.0), "");
+                return sax->number_float(static_cast<number_float_t>(1.0), no_source);
 
             case 0xFF: // empty string
             {
@@ -4201,7 +4366,7 @@ class binary_reader
         {
             return out_of_range_error(exception_id::number_overflow, "number overflow", "value");
         }
-        return sax->number_float(result, "");
+        return sax->number_float(result, no_source);
     }
 
     /*!
@@ -4261,7 +4426,7 @@ class binary_reader
         }();
         return sax->number_float((half & 0x8000u) != 0
                                  ? static_cast<number_float_t>(-val)
-                                 : static_cast<number_float_t>(val), "");
+                                 : static_cast<number_float_t>(val), no_source);
     }
 
     /*!
@@ -4586,6 +4751,10 @@ class binary_reader
     /// the SAX parser
     json_sax_t* sax = nullptr;
 
+    /// the (empty) source text of every number_float event: binary formats
+    /// have none, and a string_t built for each float would cost more
+    const string_t no_source{};
+
     /// the containers that have been opened and not closed yet; see @ref container_frame
     std::vector<container_frame> container_stack{};
 

include/nlohmann/json_view.hpp

@@ -766,9 +766,6 @@ class basic_json_view
     template<typename, typename> friend class detail::view::view_side;
     friend iterator;
 
-    /// whether the view is one of an editable document
-    static constexpr bool is_editable = Editable;
-
     basic_json_view(const document_data* d, const node* n) noexcept
         : m_doc(d), m_node(n)
     {}

single_include/nlohmann/json.hpp

@@ -18517,11 +18517,13 @@ class binary_reader
     bool get_cbor_negative_integer()
     {
         NumberType number{};
-        if (JSON_HEDLEY_UNLIKELY(!get_number(number)))
-        {
-            return false;
-        }
+        return get_number(number) && emit_cbor_negative_integer(number);
+    }
 
+    /// pass the CBOR negative integer -1 - @a number to the SAX parser
+    template<typename NumberType>
+    bool emit_cbor_negative_integer(const NumberType number)
+    {
         // the value is -1 - number, which fits into number_integer_t
         // whenever number does; the outer cast undoes the integral promotion
         // for number_integer_t types narrower than int
@@ -18746,7 +18748,7 @@ class binary_reader
             case 0x95:
             case 0x96:
             case 0x97:
-                return enter_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
+                return enter_cbor_array(conditional_static_cast<std::size_t>(static_cast<unsigned int>(current) & 0x1Fu));
 
             case 0x98: // array (one-byte uint8_t for n follows)
             case 0x99: // array (two-byte uint16_t for n follow)
@@ -18755,7 +18757,7 @@ class binary_reader
             {
                 std::uint64_t len{};
                 std::size_t size{};
-                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_array(size);
+                return get_cbor_argument(len) && get_cbor_container_size(len, size, "array") && enter_cbor_array(size);
             }
 
             case 0x9F: // array (indefinite length)
@@ -19452,6 +19454,171 @@ class binary_reader
         return true;
     }
 
+    /*!
+    @brief read a big-endian number of type @a NumberType at @a data
+    */
+    template<typename NumberType>
+    NumberType load_big_endian(const unsigned char* data) const noexcept
+    {
+        NumberType result{};
+        std::memcpy(&result, data, sizeof(NumberType));
+        if (is_little_endian)
+        {
+            byte_swap(result);
+        }
+        return result;
+    }
+
+    /*!
+    @brief read the integers, floats, booleans, and nulls that follow in a
+           definite-length array, without the per-element bookkeeping of
+           @ref parse_cbor_internal
+
+    Stops at the end of the array, at any other kind of value, and when fewer
+    than 9 bytes (the longest of these values) are left in the input, leaving
+    those to @ref parse_cbor_value, which reads them the same way.
+
+    @param[in,out] remaining  the number of elements of the array not yet read
+    @return whether the SAX parser accepted the values
+    */
+    bool parse_cbor_scalar_run(std::size_t& remaining)
+    {
+        while (remaining != 0 && ia.bulk_remaining() >= 9)
+        {
+            const auto* const data = reinterpret_cast<const unsigned char*>(ia.bulk_data());
+            const unsigned char head = data[0];
+            std::size_t size = 1;
+            bool accepted = true;
+            switch (head)
+            {
+                case 0x18:
+                    size = 2;
+                    accepted = emit_unsigned(data[1]);
+                    break;
+                case 0x19:
+                    size = 3;
+                    accepted = emit_unsigned(load_big_endian<std::uint16_t>(data + 1));
+                    break;
+                case 0x1A:
+                    size = 5;
+                    accepted = emit_unsigned(load_big_endian<std::uint32_t>(data + 1));
+                    break;
+                case 0x1B:
+                    size = 9;
+                    accepted = emit_unsigned(load_big_endian<std::uint64_t>(data + 1));
+                    break;
+                case 0x38:
+                    size = 2;
+                    accepted = emit_cbor_negative_integer(data[1]);
+                    break;
+                case 0x39:
+                    size = 3;
+                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint16_t>(data + 1));
+                    break;
+                case 0x3A:
+                    size = 5;
+                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint32_t>(data + 1));
+                    break;
+                case 0x3B:
+                    size = 9;
+                    accepted = emit_cbor_negative_integer(load_big_endian<std::uint64_t>(data + 1));
+                    break;
+                case 0xF4:
+                    accepted = sax->boolean(false);
+                    break;
+                case 0xF5:
+                    accepted = sax->boolean(true);
+                    break;
+                case 0xF6:
+                    accepted = sax->null();
+                    break;
+                case 0xFA:
+                    size = 5;
+                    accepted = emit_float(load_big_endian<float>(data + 1));
+                    break;
+                case 0xFB:
+                    size = 9;
+                    accepted = emit_float(load_big_endian<double>(data + 1));
+                    break;
+                default:
+                    if (head <= 0x17)
+                    {
+                        accepted = sax->number_unsigned(static_cast<number_unsigned_t>(head));
+                    }
+                    else if (head >= 0x20 && head <= 0x37)
+                    {
+                        accepted = sax->number_integer(static_cast<std::int8_t>(0x20 - 1 - head));
+                    }
+                    else
+                    {
+                        // any other value is read by parse_cbor_value
+                        return true;
+                    }
+                    break;
+            }
+            // as if read by get() and get_number(): current is the head byte
+            ia.bulk_skip(size);
+            chars_read += size;
+            current = head;
+            --remaining;
+            if (JSON_HEDLEY_UNLIKELY(!accepted))
+            {
+                return false;
+            }
+        }
+        return true;
+    }
+
+    /*!
+    @brief open a definite-length array and read the scalars it begins with
+           (see @ref parse_cbor_scalar_run)
+
+    Only the start of an array is tried, which covers arrays of numbers
+    (e.g. coordinates) but costs arrays of other values only one test, not one
+    per element.
+
+    @param[in] len  the number of elements
+    @return whether the SAX parser accepted the array and the values read
+    */
+    bool enter_cbor_array(const std::size_t len)
+    {
+        return enter_array(len) && (len == 0 || parse_cbor_array_start(std::integral_constant<bool, bulk_scan> {}));
+    }
+
+    /// read the scalars the array just opened begins with; out of line, so
+    /// that @ref parse_cbor_value stays small enough to be inlined into
+    /// @ref parse_cbor_internal
+    JSON_HEDLEY_NEVER_INLINE
+    bool parse_cbor_array_start(std::true_type /*bulk*/)
+    {
+        return !cbor_scalar_follows() || parse_cbor_scalar_run(container_stack.back().remaining);
+    }
+
+    /// input that is not contiguous: every value is read by parse_cbor_value
+    bool parse_cbor_array_start(std::false_type /*bulk*/) const noexcept
+    {
+        return true;
+    }
+
+    /*!
+    @brief whether @ref parse_cbor_scalar_run can read the next value: one of
+           the heads it handles, with at least 9 bytes of input left
+    */
+    bool cbor_scalar_follows() const
+    {
+        if (ia.bulk_remaining() < 9)
+        {
+            return false;
+        }
+        const auto head = static_cast<unsigned int>(*reinterpret_cast<const unsigned char*>(ia.bulk_data()));
+        // major types 0 and 1 (unsigned and negative integers) up to an
+        // 8-byte argument, or false, true, null, float32, float64 (0xF4,
+        // 0xF5, 0xF6, 0xFA, 0xFB: bits 0, 1, 2, 6, 7 of 0xC7)
+        return ((head & 0xC0u) == 0 && (head & 0x1Fu) <= 0x1Bu)
+               || (head >= 0xF4u && ((0xC7u >> (head - 0xF4u)) & 1u) != 0);
+    }
+
+
     /*!
     @brief read a CBOR value and everything nested inside it
 
@@ -19477,10 +19644,8 @@ class binary_reader
         {
             if (!container_stack.empty())
             {
-                // a copy, not a reference: it must stay valid across the
-                // pop_back() below, which destroys the container_stack element
-                // it would otherwise alias
                 const container_frame top = container_stack.back();
+                const bool is_object = top.is_object;
                 bool at_end = false;
 
                 if (top.remaining != npos)
@@ -19492,7 +19657,7 @@ class binary_reader
                     {
                         // claim the element about to be read
                         --container_stack.back().remaining;
-                        if (top.is_object)
+                        if (is_object)
                         {
                             get();
                         }
@@ -19505,7 +19670,7 @@ class binary_reader
                     // Testing for it consumes a byte, which is the first byte
                     // of the next element when it is not one.
                     at_end = (get() == 0xFF);
-                    fetch = top.is_object;
+                    fetch = is_object;
                 }
 
                 if (at_end)
@@ -19522,7 +19687,7 @@ class binary_reader
                     continue;
                 }
 
-                if (top.is_object)
+                if (is_object)
                 {
                     if (JSON_HEDLEY_UNLIKELY(!skip_cbor_key_tags(tag_handler)))
                     {
@@ -21467,13 +21632,13 @@ class binary_reader
                 return sax->null();
 
             case 0xFB:
-                return sax->number_float(static_cast<number_float_t>(-1.0), "");
+                return sax->number_float(static_cast<number_float_t>(-1.0), no_source);
 
             case 0xFC:
-                return sax->number_float(static_cast<number_float_t>(0.0), "");
+                return sax->number_float(static_cast<number_float_t>(0.0), no_source);
 
             case 0xFD:
-                return sax->number_float(static_cast<number_float_t>(1.0), "");
+                return sax->number_float(static_cast<number_float_t>(1.0), no_source);
 
             case 0xFF: // empty string
             {
@@ -21977,7 +22142,7 @@ class binary_reader
         {
             return out_of_range_error(exception_id::number_overflow, "number overflow", "value");
         }
-        return sax->number_float(result, "");
+        return sax->number_float(result, no_source);
     }
 
     /*!
@@ -22037,7 +22202,7 @@ class binary_reader
         }();
         return sax->number_float((half & 0x8000u) != 0
                                  ? static_cast<number_float_t>(-val)
-                                 : static_cast<number_float_t>(val), "");
+                                 : static_cast<number_float_t>(val), no_source);
     }
 
     /*!
@@ -22362,6 +22527,10 @@ class binary_reader
     /// the SAX parser
     json_sax_t* sax = nullptr;
 
+    /// the (empty) source text of every number_float event: binary formats
+    /// have none, and a string_t built for each float would cost more
+    const string_t no_source{};
+
     /// the containers that have been opened and not closed yet; see @ref container_frame
     std::vector<container_frame> container_stack{};
 

single_include/nlohmann/json_view.hpp

@@ -9807,9 +9807,6 @@ class basic_json_view
     template<typename, typename> friend class detail::view::view_side;
     friend iterator;
 
-    /// whether the view is one of an editable document
-    static constexpr bool is_editable = Editable;
-
     basic_json_view(const document_data* d, const node* n) noexcept
         : m_doc(d), m_node(n)
     {}

tests/src/unit-cbor.cpp

@@ -29,7 +29,9 @@ using nlohmann::json;
 #include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 #include "sax_countdown.hpp"
+#include "sax_event_loggers.hpp"
 using utils::SaxCountdown;
+using utils::SaxEventLogger;
 
 
 TEST_CASE("CBOR")
@@ -2446,6 +2448,78 @@ TEST_CASE("CBOR SAX parsing stops at every event")
     CHECK(count_events(std::vector<std::uint8_t>({0xBF, 0x61, 'a', 0x9F, 0x01, 0xFF, 0xFF})) == 6);
 }
 
+TEST_CASE("CBOR arrays of scalars from contiguous and non-contiguous input")
+{
+    // contiguous input reads the integers, floats, booleans, and nulls an
+    // array begins with in one loop, other input value by value; both must
+    // give the same events, value, or error for the complete document and for
+    // every truncation of it, and stop at every event a SAX handler rejects
+    const std::vector<std::uint8_t> cbor =
+    {
+        0xA3,                                                       // map(3)
+        0x61, 'a',                                                  // "a"
+        0x98, 0x16,                                                 // array(22)
+        0x00, 0x17, 0x18, 0xFF, 0x19, 0x01, 0x00,                   // 0, 23, 255, 256
+        0x1A, 0x00, 0x01, 0x00, 0x00,                               // 65536
+        0x1B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,       // 18446744073709551615
+        0x20, 0x37, 0x38, 0xFF, 0x39, 0x01, 0x00,                   // -1, -24, -256, -257
+        0x3A, 0x00, 0x01, 0x00, 0x00,                               // -65537
+        0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,       // -18446744073709551616 (a float)
+        0x3B, 0x7F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,       // -9223372036854775808
+        0xF4, 0xF5, 0xF6,                                           // false, true, null
+        0xFA, 0x3F, 0xC0, 0x00, 0x00,                               // 1.5 (float32)
+        0xFB, 0xBF, 0xB9, 0x99, 0x99, 0x99, 0x99, 0x99, 0x9A,       // -0.1
+        0xF9, 0x3C, 0x00,                                           // 1.0 (float16, ends the loop)
+        0x01, 0x61, 'a',                                            // 1, "a"
+        0xFB, 0x40, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,       // 2.5
+        0x61, 'b',                                                  // "b"
+        0x83,                                                       // array(3)
+        0x82, 0xFB, 0x3F, 0xF4, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // [1.25,
+        0xFB, 0xC0, 0x0E, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,       //  -3.75]
+        0x82, 0xFB, 0x3F, 0xF4, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // [1.25,
+        0xFB, 0xC0, 0x0E, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,       //  -3.75]
+        0x80,                                                       // []
+        0x61, 'c',                                                  // "c"
+        0x84, 0xA0, 0x01, 0xC1, 0x02, 0x81, 0x03                    // [{}, 1, tag(1) 2, [3]]
+    };
+    const auto tags = json::cbor_tag_handler_t::ignore;
+
+    const json j = json::from_cbor(cbor, true, true, tags);
+    CHECK(j["a"].size() == 22);
+    CHECK(j["a"][5] == 18446744073709551615u);
+    CHECK(j["a"][11].is_number_float());
+    CHECK(j["a"][11] == -18446744073709551616.0);
+    CHECK(j["a"][12] == (std::numeric_limits<std::int64_t>::min)());
+    CHECK(j["a"][17] == -0.1);
+    CHECK(j["b"] == json::parse("[[1.25, -3.75], [1.25, -3.75], []]"));
+    CHECK(j["c"] == json::parse("[{}, 1, 2, [3]]"));
+
+    for (std::size_t length = 0; length <= cbor.size(); ++length)
+    {
+        CAPTURE(length)
+        const std::vector<std::uint8_t> contiguous(cbor.begin(), cbor.begin() + static_cast<std::ptrdiff_t>(length));
+        const std::list<std::uint8_t> bytewise(contiguous.begin(), contiguous.end());
+
+        CHECK(json::from_cbor(contiguous, true, false, tags).dump() == json::from_cbor(bytewise.begin(), bytewise.end(), true, false, tags).dump());
+
+        SaxEventLogger from_contiguous;
+        SaxEventLogger from_bytewise;
+        CHECK(json::sax_parse(contiguous, &from_contiguous, json::input_format_t::cbor, true, false, false, tags)
+              == json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::cbor, true, false, false, tags));
+        CHECK(from_contiguous.events == from_bytewise.events);
+    }
+
+    for (int events = 0; events <= 60; ++events)
+    {
+        CAPTURE(events)
+        const std::list<std::uint8_t> bytewise(cbor.begin(), cbor.end());
+        SaxCountdown from_contiguous(events);
+        SaxCountdown from_bytewise(events);
+        CHECK(json::sax_parse(cbor, &from_contiguous, json::input_format_t::cbor, true, false, false, tags)
+              == json::sax_parse(bytewise.begin(), bytewise.end(), &from_bytewise, json::input_format_t::cbor, true, false, false, tags));
+    }
+}
+
 TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
 {
     // Reading an indefinite-length string or byte array used to call itself

tools/check_path_filters/README.md

@@ -1,34 +0,0 @@
-# check_path_filters
-
-The compiler workflows [`ubuntu.yml`](../../.github/workflows/ubuntu.yml),
-[`macos.yml`](../../.github/workflows/macos.yml), and [`windows.yml`](../../.github/workflows/windows.yml) skip pull
-requests that change only files no compiler job reads: the documentation, Markdown files, GitHub metadata, other
-workflows, and the license files. Pushes to `develop`, `master`, and `release/*` always run them. The documentation,
-license, and path-filter checks run in [`repository_checks.yml`](../../.github/workflows/repository_checks.yml), which
-has no path filter.
-
-The filter starts with `'**'` and excludes these files explicitly, so a new file runs the compiler workflows unless it
-is excluded on purpose. [`check_path_filters.py`](check_path_filters.py) holds the filter (`PATHS`) and checks that
-
-- each of the three workflows has exactly this filter, with its own workflow file included,
-- the CI targets that read excluded files (`UNFILTERED_TARGETS`) do not run in these workflows, and
-- no file the filter keeps (sources, tests, build files, tools, workflows) mentions an excluded file or directory
-  outside a comment line, unless the mention is listed in `KNOWN_REFERENCES` with a reason.
-
-When the check reports a mention, decide whether a compiler job reads that file:
-
-- If it does, include the file in `PATHS` after the exclusions (as `package_managers.md` and the Swift example are) and
-  copy the new filter into the three workflows.
-- If it does not, add the mention to `KNOWN_REFERENCES` with the reason.
-
-Run the check with
-
-```shell
-python3 tools/check_path_filters/check_path_filters.py
-```
-
-To see which workflows a pull request would run, pass the changed files:
-
-```shell
-python3 tools/check_path_filters/check_path_filters.py --changed $(git diff --name-only origin/develop)
-```

tools/check_path_filters/check_path_filters.py

@@ -1,269 +0,0 @@
-#!/usr/bin/env python3
-"""Check the path filters of the compiler workflows.
-
-ubuntu.yml, macos.yml and windows.yml skip pull requests that change only files
-no compiler job reads (documentation, Markdown files, GitHub metadata, license
-files). The filter starts with '**' and excludes these files explicitly, so a
-new file runs the workflows unless it is excluded on purpose.
-
-The script checks that
-
-- each of the three workflows has exactly the filter defined in PATHS below,
-  with its own workflow file included, and
-- no tracked file that the filter keeps (sources, tests, build files, tools,
-  workflows) mentions a file the filter excludes, unless the mention is listed
-  in KNOWN_REFERENCES with the reason why it does not matter.
-
-A mention found this way must either be added to PATHS as an included path
-(the file is read by a compiler job) or to KNOWN_REFERENCES (it is not).
-
-With --changed FILE..., the script prints instead which workflows a pull
-request changing these files would run.
-"""
-
-import argparse
-import os
-import re
-import subprocess  # nosec B404
-import sys
-
-REPO_ROOT = os.path.normpath(os.path.join(sys.path[0], '..', '..'))
-SELF = 'tools/check_path_filters/check_path_filters.py'
-
-WORKFLOWS = ['.github/workflows/ubuntu.yml', '.github/workflows/macos.yml', '.github/workflows/windows.yml']
-
-# The pull_request paths filter of the workflows above; '{workflow}' is replaced
-# with the workflow's own file. GitHub applies the patterns in order: a file
-# matching a later pattern is included ('pattern') or excluded ('!pattern').
-PATHS = [
-    '**',
-    '!docs/**',
-    '!**.md',
-    '!.github/*',
-    '!.github/ISSUE_TEMPLATE/**',
-    '!.github/external_ci/**',
-    '!.github/workflows/**',
-    '!.reuse/**',
-    '!LICENSES/**',
-    '!CITATION.cff',
-    '!REUSE.toml',
-    # read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
-    'docs/mkdocs/docs/integration/package_managers.md',
-    # built by the swiftpm job (macos.yml)
-    'docs/mkdocs/docs/integration/swift/**',
-    '{workflow}',
-]
-
-# Mentions of excluded files that no job of the three workflows depends on:
-# (file that mentions it, the mention, reason)
-KNOWN_REFERENCES = [
-    ('cmake/ci.cmake', 'docs/', 'ci_test_examples; runs in repository_checks.yml, which has no path filter'),
-    ('cmake/ci.cmake', 'docs/mkdocs/', 'ci_test_build_documentation and ci_test_documentation_mermaid; they run in '
-                                       'repository_checks.yml, which has no path filter'),
-    ('cmake/ci.cmake', 'docs/mkdocs/docs/examples/', 'indentation check of ci_test_amalgamation; the same check runs in '
-                                                     'check_amalgamation.yml, which has no path filter'),
-    ('Makefile', 'docs/mkdocs/docs/examples/', 'make pretty; runs in check_amalgamation.yml, which has no path filter'),
-    ('Makefile', 'ChangeLog.md', 'make ChangeLog.md writes the file; no CI job runs it'),
-    ('Makefile', 'tests/fuzzing.md', 'help text'),
-    ('Makefile', 'tools/generate_natvis/README.md', 'help text'),
-    ('tests/src/make_test_data_available.hpp', 'README.md', 'a file of the external test data repository'),
-    ('tools/check_build_options/check_build_options.py', 'docs/', 'builds the path of package_managers.md, which '
-                                                                  'PATHS includes'),
-]
-
-
-# CI targets that read excluded files; they must run in a workflow without a path filter
-UNFILTERED_TARGETS = ['ci_test_examples', 'ci_test_build_documentation', 'ci_test_documentation_mermaid',
-                      'ci_test_amalgamation', 'ci_reuse_compliance']
-
-
-def git_files(root):
-    # fixed command, no user input; includes new files not yet added, so a local run
-    # sees what CI sees after the commit
-    out = subprocess.check_output(['git', 'ls-files', '-z', '--cached', '--others', '--exclude-standard'], cwd=root)  # nosec B603, B607
-    return [f for f in out.decode('utf-8').split('\0') if f]
-
-
-def glob_to_regex(pattern):
-    """Translate a GitHub Actions path pattern to a regular expression."""
-    regex = ''
-    i = 0
-    while i < len(pattern):
-        if pattern.startswith('**', i):
-            regex += '.*'
-            i += 2
-        elif pattern[i] == '*':
-            regex += '[^/]*'
-            i += 1
-        elif pattern[i] == '?':
-            regex += '[^/]'
-            i += 1
-        else:
-            regex += re.escape(pattern[i])
-            i += 1
-    return re.compile(regex + r'\Z')
-
-
-def compile_filter(paths, workflow):
-    return [(p.startswith('!'), glob_to_regex(p.lstrip('!').replace('{workflow}', workflow))) for p in paths]
-
-
-def included(compiled, path):
-    result = False
-    for negated, regex in compiled:
-        if regex.match(path):
-            result = not negated
-    return result
-
-
-def read_filter(root, workflow):
-    """Return the pull_request paths list of a workflow file."""
-    with open(os.path.join(root, workflow), encoding='utf-8') as f:
-        lines = f.read().splitlines()
-    paths = None
-    in_pull_request = False
-    for line in lines:
-        if re.match(r'^  pull_request:', line):
-            in_pull_request = True
-            continue
-        if in_pull_request:
-            if re.match(r'^  \S', line) or re.match(r'^\S', line):
-                break
-            if re.match(r'^    paths:\s*$', line):
-                paths = []
-                continue
-            m = re.match(r"^      - '(.*)'\s*(#.*)?$", line)
-            if paths is not None and m:
-                paths.append(m.group(1))
-            elif paths is not None and line.strip() and not line.strip().startswith('#'):
-                break
-    return paths
-
-
-TOKEN = re.compile(r'(["\'/]?)([\w.-]+(?:/[\w.-]+)*)')
-
-
-COMMENT_LINE = re.compile(r'^[ \t]*(//|#|/?\*|--).*$', re.MULTILINE)
-
-
-class Mentions:
-    """Finds the words and paths in a text that name an excluded file or directory."""
-
-    def __init__(self, files, excluded, is_excluded):
-        self.files = set(files)
-        self.excluded = excluded
-        self.dirs = {'/'.join(f.split('/')[:n]) for f in files for n in range(1, f.count('/') + 1)}
-        # an excluded directory: a new file in it would be excluded
-        self.excluded_dirs = {d for d in self.dirs if is_excluded(d + '/new_file')}
-        # base names that only excluded files have, e.g. 'ChangeLog.md' but not 'CMakeLists.txt'
-        self.names = {os.path.basename(f) for f in excluded} - {os.path.basename(f) for f in files if f not in excluded}
-
-    def find(self, text):
-        """Yield (offset, mention) for every word or path in text that names an excluded file or directory."""
-        for m in TOKEN.finditer(text):
-            parts = m.group(2).rstrip('/').split('/')
-            # a single word names a directory only in a path context, e.g. 'docs' in os.path.join('docs', ...)
-            path_context = len(parts) > 1 or m.group(1)
-            for i in range(len(parts)):
-                path = '/'.join(parts[i:])
-                if path in self.files:
-                    if path in self.excluded:
-                        yield m.start(2), path
-                    break
-                if path in self.dirs and path_context:
-                    if path in self.excluded_dirs:
-                        yield m.start(2), path + '/'
-                    break
-            else:
-                if parts[-1] in self.names and '.' in parts[-1]:
-                    yield m.start(2), parts[-1]
-
-
-def known(path, mention):
-    return any(path == file and mention == known_mention for file, known_mention, _ in KNOWN_REFERENCES)
-
-
-def check_workflows(root):
-    """Return the errors in the filters of the workflows and in the targets they run."""
-    errors = []
-    for workflow in WORKFLOWS:
-        expected = [p.replace('{workflow}', workflow) for p in PATHS]
-        actual = read_filter(root, workflow)
-        if actual != expected:
-            errors.append('{}: the pull_request paths filter differs from tools/check_path_filters/check_path_filters.py\n'
-                          '  expected: {}\n  actual:   {}'.format(workflow, expected, actual))
-        with open(os.path.join(root, workflow), encoding='utf-8') as f:
-            text = f.read()
-        errors += ['{}: runs {}, which reads excluded files; run it in repository_checks.yml'.format(workflow, target)
-                   for target in UNFILTERED_TARGETS if re.search(r'\b' + target + r'\b', text)]
-    return errors
-
-
-def scanned_text(root, path):
-    """Return the text of a kept file without comment lines and without the filter itself, or None to skip it."""
-    # third-party code and Markdown files read nothing; this script lists the excluded files
-    if path.startswith('tests/thirdparty/') or path.endswith('.md') or path == SELF:
-        return None
-    try:
-        with open(os.path.join(root, path), encoding='utf-8') as f:
-            # a mention in a comment line reads nothing; keep the line numbers
-            text = COMMENT_LINE.sub('', f.read())
-    except (UnicodeDecodeError, OSError):
-        return None
-    if path in WORKFLOWS:
-        # the filter itself lists the excluded files
-        for p in PATHS:
-            text = text.replace("      - '{}'".format(p.replace('{workflow}', path)), '')
-    return text
-
-
-def check_mentions(root, files, excluded, finder):
-    """Return an error for every mention of an excluded file in a kept file that KNOWN_REFERENCES does not list."""
-    errors = []
-    for path in sorted(set(files) - excluded):
-        text = scanned_text(root, path)
-        reported = set()
-        for offset, mention in finder.find(text or ''):
-            if mention not in reported and not known(path, mention):
-                reported.add(mention)
-                line = text.count('\n', 0, offset) + 1
-                errors.append('{}:{}: mentions excluded {}; include it in PATHS if a compiler job reads it, '
-                              'or add it to KNOWN_REFERENCES'.format(path, line, mention))
-    return errors
-
-
-def check(root):
-    files = git_files(root)
-    union = [compile_filter(PATHS, w) for w in WORKFLOWS]
-
-    def is_excluded(path):
-        return not any(included(c, path) for c in union)
-
-    excluded = {f for f in files if is_excluded(f)}
-    errors = check_workflows(root) + check_mentions(root, files, excluded, Mentions(files, excluded, is_excluded))
-    for e in errors:
-        print(e)
-    if not errors:
-        print('path filters OK: {} of {} tracked files skip the compiler workflows'.format(len(excluded), len(files)))
-    return 1 if errors else 0
-
-
-def explain(changed):
-    for workflow in WORKFLOWS:
-        compiled = compile_filter(PATHS, workflow)
-        hits = [f for f in changed if included(compiled, f)]
-        print('{}: {}'.format(workflow, 'runs ({})'.format(', '.join(hits[:3]) + (' ...' if len(hits) > 3 else ''))
-                                       if hits else 'skipped'))
-    return 0
-
-
-def main():
-    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
-    parser.add_argument('--root', default=REPO_ROOT, help='repository root')
-    parser.add_argument('--changed', nargs='+', metavar='FILE', help='show which workflows these changed files run')
-    args = parser.parse_args()
-    return explain(args.changed) if args.changed else check(args.root)
-
-
-if __name__ == '__main__':
-    sys.exit(main())