Compare commits

..
Author SHA1 Message Date
Niels Lohmann 9505be15fd Merge branch 'develop' into claude/binary-reader-narrow-numbers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-28 22:22:22 +02:00
Niels Lohmann 44325873ea Check integer-to-float fallbacks for overflow in the binary readers
emit_signed, emit_unsigned, and the CBOR negative integer fallback now
pass their number_float_t fallback through emit_float, so a value that
overflows number_float_t is rejected with out_of_range.406 like a
floating-point value, instead of silently becoming infinity. This only
matters for a number_float_t that cannot represent 2^64, such as a
half-precision type. The CBOR value -1 - n is computed as long double so
that emit_float sees a finite value.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-28 20:19:28 +02:00
Niels Lohmann 0c630d4c30 Fix MSVC and clang 3.5 in the narrow number type test
MSVC types 3000000000 and 5000000000 as unsigned long, so
json(-3000000000) triggered C4146 (unary minus on an unsigned type),
which /WX turns into an error. Use LL literals, as elsewhere in the
tests.

clang 3.5 cannot convert the lambdas in the braced initializer of the
format table to function pointers. Use named functions instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-28 18:39:45 +02:00
Niels Lohmann 04ed4f593c Handle numbers that do not fit narrow number types in the binary readers
With custom number types narrower than the values in a binary document,
for example basic_json<..., std::int32_t, std::uint32_t, float>, every
binary reader (CBOR, MessagePack, UBJSON, BJData, BSON, BON8) passed the
decoded number to the SAX interface with an implicit conversion: the
integer 5000000000 silently became 705032704, and a finite double such as
1e300 became infinity. The lexer handles the same values in JSON text: an
integer that fits neither integer type is stored as number_float_t, and a
finite number that overflows number_float_t is rejected with
out_of_range.406.

Pass every number read from binary input through three helpers that
apply the lexer's rules:
- emit_signed(): number_integer_t, else number_unsigned_t for a
  non-negative value, else number_float_t
- emit_unsigned(): number_unsigned_t, else number_float_t
- emit_float(): out_of_range.406 if a finite value overflows
  number_float_t; infinity and NaN are passed on

For consistency, a CBOR negative integer below the range of
number_integer_t is now stored as number_float_t, like a too small
integer in JSON text, instead of being rejected with parse_error.112.
With the default number types, this is the only change in behavior.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 23:23:16 +02:00
19 changed files with 554 additions and 191 deletions
+17
View File
@@ -0,0 +1,17 @@
arm_container:
image: gcc:latest
check_task:
check_script:
# the gcc image ships an outdated CMake, so fetch a recent prebuilt binary
# instead of compiling CMake from source
- wget -q https://github.com/Kitware/CMake/releases/download/v4.3.4/cmake-4.3.4-linux-aarch64.tar.gz
- tar xfz cmake-4.3.4-linux-aarch64.tar.gz
- export PATH="$(pwd)/cmake-4.3.4-linux-aarch64/bin:$PATH"
- cmake --version
- mkdir build
- cd build
- cmake .. -DJSON_FastTests=ON
- make -j4
- cd tests
- ctest -j4
-4
View File
@@ -24,10 +24,6 @@ updates:
interval: daily
cooldown:
default-days: 7
ignore:
# astyle is deliberately pinned (see tools/astyle/requirements.txt): newer versions
# reformat the code, and the pinned version defines the formatting CI enforces
- dependency-name: astyle
- package-ecosystem: pip
directory: /tools/generate_natvis
+11
View File
@@ -30,6 +30,14 @@ environment:
CMAKE_OPTIONS: ""
GENERATOR: Visual Studio 14 2015
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
configuration: Release
platform: x86
name: with_win_header
CXX_FLAGS: "/W4 /WX"
CMAKE_OPTIONS: ""
GENERATOR: Visual Studio 14 2015
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
configuration: Release
platform: x86
@@ -66,6 +74,9 @@ install:
- if "%platform%"=="x86" set GENERATOR_PLATFORM=Win32
before_build:
# for with_win_header build, inject the inclusion of Windows.h to the single-header library
- ps: if ($env:name -Eq "with_win_header") { $header_path = "single_include\nlohmann\json.hpp" }
- ps: if ($env:name -Eq "with_win_header") { "#include <Windows.h>`n" + (Get-Content $header_path | Out-String) | Set-Content $header_path }
- cmake . -G "%GENERATOR%" -A "%GENERATOR_PLATFORM%" -DCMAKE_CXX_FLAGS="%CXX_FLAGS%" -DCMAKE_IGNORE_PATH="C:/Program Files/Git/usr/bin" -DJSON_BuildTests=On "%CMAKE_OPTIONS%"
build_script:
-18
View File
@@ -403,21 +403,3 @@ jobs:
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
# Native Linux ARM64 runner with the latest GCC. The ubuntu-24.04-arm label is
# only available for public repositories.
ci_test_arm64:
runs-on: ubuntu-24.04-arm
container: gcc:latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@fffaaafeea488556c2c12dad60690008bc1caacb # v4.4.2
- name: Check that the job runs natively on arm64
run: uname -m && test "$(uname -m)" = aarch64
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
+11
View File
@@ -33,6 +33,17 @@ Further documentation:
> [!IMPORTANT]
> The folder `.github/workflows` is predetermined by GitHub.
### `.cirrus.yml`
Configuration file for the pipeline at [Cirrus CI](https://cirrus-ci.com/github/nlohmann/json).
Further documentation:
- [Writing tasks](https://cirrus-ci.org/guide/writing-tasks/)
> [!IMPORTANT]
> The filename `.cirrus.yml` and position (root of the repository) are predetermined by Cirrus CI.
### `.github/external_ci/appveyor.yml`
Configuration for the pipelines at [AppVeyor](https://ci.appveyor.com/project/nlohmann/json).
+1
View File
@@ -7,6 +7,7 @@
[![Coverage Status](https://coveralls.io/repos/github/nlohmann/json/badge.svg?branch=develop)](https://coveralls.io/github/nlohmann/json?branch=develop)
[![Coverity Scan Build Status](https://scan.coverity.com/projects/5550/badge.svg)](https://scan.coverity.com/projects/nlohmann-json)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/e0d1a9d5d6fd46fcb655c4cb930bb3e8)](https://app.codacy.com/gh/nlohmann/json/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)
[![Cirrus CI](https://api.cirrus-ci.com/github/nlohmann/json.svg)](https://cirrus-ci.com/github/nlohmann/json)
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/json.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:json)
[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://wandbox.org/permlink/1mp10JbaANo6FUc7)
[![Documentation](https://img.shields.io/badge/docs-mkdocs-blue.svg)](https://json.nlohmann.me)
+69 -29
View File
@@ -55,6 +55,12 @@ execute_process(COMMAND ${NINJA_TOOL} --version OUTPUT_VARIABLE NINJA_TOOL_VERSI
string(REGEX MATCH "[0-9]+(\\.[0-9]+)+" NINJA_TOOL_VERSION "${NINJA_TOOL_VERSION}")
message(STATUS "🔖 Ninja ${NINJA_TOOL_VERSION} (${NINJA_TOOL})")
find_program(OCLINT_TOOL NAMES oclint-json-compilation-database)
find_program(OCLINT_VERSION_TOOL NAMES oclint)
execute_process(COMMAND ${OCLINT_VERSION_TOOL} --version OUTPUT_VARIABLE OCLINT_TOOL_VERSION ERROR_VARIABLE OCLINT_TOOL_VERSION)
string(REGEX MATCH "[0-9]+(\\.[0-9]+)+" OCLINT_TOOL_VERSION "${OCLINT_TOOL_VERSION}")
message(STATUS "🔖 OCLint ${OCLINT_TOOL_VERSION} (${OCLINT_TOOL})")
find_program(VALGRIND_TOOL NAMES valgrind)
execute_process(COMMAND ${VALGRIND_TOOL} --version OUTPUT_VARIABLE VALGRIND_TOOL_VERSION ERROR_VARIABLE VALGRIND_TOOL_VERSION)
string(REGEX MATCH "[0-9]+(\\.[0-9]+)+" VALGRIND_TOOL_VERSION "${VALGRIND_TOOL_VERSION}")
@@ -122,16 +128,16 @@ foreach(CXX_STANDARD 11 14 17 20 23 26)
COMMENT "Compile and test with Clang for C++${CXX_STANDARD}"
)
# pass -stdlib=libc++ through CXXFLAGS: an explicit -DCMAKE_CXX_FLAGS would make CMake ignore CXXFLAGS
add_custom_target(ci_test_clang_libcxx_cxx${CXX_STANDARD}
COMMAND CXX=${CLANG_TOOL} CXXFLAGS="${CLANG_CXXFLAGS};-stdlib=libc++" ${CMAKE_COMMAND}
COMMAND CXX=${CLANG_TOOL} CXXFLAGS="${CLANG_CXXFLAGS}" ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_FastTests=ON
-DJSON_TestStandards=${CXX_STANDARD}
-DCMAKE_CXX_FLAGS="-stdlib=libc++"
-DCMAKE_EXE_LINKER_FLAGS="-lc++abi"
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_clang_libcxx_cxx${CXX_STANDARD}
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_clang_libcxx_cxx${CXX_STANDARD}
COMMAND cd ${PROJECT_BINARY_DIR}/build_clang_libcxx_cxx${CXX_STANDARD} && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_clang_cxx${CXX_STANDARD}
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_clang_cxx${CXX_STANDARD}
COMMAND cd ${PROJECT_BINARY_DIR}/build_clang_cxx${CXX_STANDARD} && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
COMMENT "Compile and test with Clang for C++${CXX_STANDARD} (libc++)"
)
endforeach()
@@ -444,7 +450,7 @@ add_custom_target(ci_cppcheck
COMMAND venv_cppcheck/bin/cppcheck --enable=warning --check-level=exhaustive --inline-suppr --inconclusive --force
--std=c++11 ${PROJECT_SOURCE_DIR}/include/nlohmann/json.hpp -I ${CMAKE_SOURCE_DIR}/include
--error-exitcode=1 --relative-paths=${PROJECT_SOURCE_DIR} -j ${N} --include=default_defines.hpp
--cppcheck-build-dir=cppcheck
--cppcheck-build-dir=cppcheck --check-level=exhaustive
-UJSON_CATCH_USER -UJSON_TRY_USER -UJSON_ASSERT -UJSON_INTERNAL_CATCH -UJSON_THROW
-DJSON_HAS_CPP_11 -UJSON_HAS_CPP_14 -UJSON_HAS_CPP_17 -UJSON_HAS_CPP_20 -UJSON_HAS_THREE_WAY_COMPARISON
COMMENT "Check code with Cppcheck"
@@ -462,6 +468,34 @@ add_custom_target(ci_cpplint
WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
)
###############################################################################
# Check code with OCLint.
###############################################################################
file(COPY ${PROJECT_SOURCE_DIR}/single_include/nlohmann/json.hpp DESTINATION ${PROJECT_BINARY_DIR}/src_single)
file(RENAME ${PROJECT_BINARY_DIR}/src_single/json.hpp ${PROJECT_BINARY_DIR}/src_single/all.cpp)
file(APPEND "${PROJECT_BINARY_DIR}/src_single/all.cpp" "\n\nint main()\n{}\n")
add_executable(single_all ${PROJECT_BINARY_DIR}/src_single/all.cpp)
target_compile_features(single_all PRIVATE cxx_std_11)
add_custom_target(ci_oclint
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON
-DJSON_BuildTests=OFF -DJSON_CI=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_oclint
COMMAND ${OCLINT_TOOL} -i ${PROJECT_BINARY_DIR}/build_oclint/src_single/all.cpp -p ${PROJECT_BINARY_DIR}/build_oclint --
-report-type html -enable-global-analysis --max-priority-1=0 --max-priority-2=1000 --max-priority-3=2000
--disable-rule=MultipleUnaryOperator
--disable-rule=DoubleNegative
--disable-rule=ShortVariableName
--disable-rule=GotoStatement
--disable-rule=LongLine
-o ${PROJECT_BINARY_DIR}/build_oclint/oclint_report.html
COMMENT "Check code with OCLint"
)
###############################################################################
# Check code with Clang-Tidy.
###############################################################################
@@ -476,6 +510,21 @@ add_custom_target(ci_clang_tidy
COMMENT "Check code with Clang-Tidy"
)
###############################################################################
# Check code with PVS-Studio Analyzer <https://www.viva64.com/en/pvs-studio/>.
###############################################################################
add_custom_target(ci_pvs_studio
COMMAND CXX=${CLANG_TOOL} ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON
-DJSON_BuildTests=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_pvs_studio
COMMAND cd ${PROJECT_BINARY_DIR}/build_pvs_studio && ${PVS_STUDIO_ANALYZER_TOOL} analyze -j 10
COMMAND cd ${PROJECT_BINARY_DIR}/build_pvs_studio && ${PLOG_CONVERTER_TOOL} -a'GA:1,2;64:1;CS' -t fullhtml PVS-Studio.log -o pvs
COMMENT "Check code with PVS Studio"
)
###############################################################################
# Check code with Infer <https://fbinfer.com> static analyzer.
###############################################################################
@@ -583,8 +632,7 @@ add_custom_target(ci_benchmarks
# we test the project with different CMake versions:
# - CMake 3.5 (the earliest supported)
# - CMake 3.31.6 (the latest 3.x release)
# - CMake 4.0.0 (the first 4.x release)
# - the CMake version running this build (usually the latest release)
# - CMake 4.0.0 (the latest release)
function(ci_get_cmake version var)
set(${var} ${PROJECT_BINARY_DIR}/cmake-${version}/bin/cmake)
@@ -608,24 +656,30 @@ ci_get_cmake(4.0.0 CMAKE_4_0_0_BINARY)
# the tests require CMake 3.13 or later, so they are excluded for CMake 3.5.0
set(JSON_CMAKE_FLAGS_3_5_0 JSON_Diagnostics JSON_Diagnostic_Positions JSON_GlobalUDLs JSON_ImplicitConversions JSON_DisableEnumSerialization
JSON_LegacyDiscardedValueComparison JSON_Install JSON_MultipleHeaders JSON_SystemInclude JSON_Valgrind
JSON_StrictNulHandling)
set(JSON_CMAKE_FLAGS_3_31_6 JSON_BuildTests ${JSON_CMAKE_FLAGS_3_5_0})
JSON_LegacyDiscardedValueComparison JSON_Install JSON_MultipleHeaders JSON_SystemInclude JSON_Valgrind)
set(JSON_CMAKE_FLAGS_3_31_6 JSON_BuildTests ${JSON_CMAKE_FLAGS_3_31_6})
set(JSON_CMAKE_FLAGS_4_0_0 JSON_BuildTests ${JSON_CMAKE_FLAGS_3_5_0})
function(ci_add_cmake_flags_targets flag min_version)
string(TOLOWER "ci_cmake_flag_${flag}" flag_target)
string(REPLACE . _ min_version_var ${min_version})
set(cmake_binary ${CMAKE_${min_version_var}_BINARY})
add_custom_target(${flag_target}_${min_version}_2
COMMENT "Check CMake flag ${flag} (CMake ${CMAKE_VERSION})"
COMMAND ${CMAKE_COMMAND}
-Werror=dev
-D${flag}=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_${flag_target}
)
add_custom_target(${flag_target}_${min_version_var}
COMMENT "Check CMake flag ${flag} (CMake ${min_version})"
COMMENT "Check CMake flag ${JSON_CMAKE_FLAG} (CMake ${min_version})"
COMMAND mkdir -pv ${PROJECT_BINARY_DIR}/build_${flag_target}_${min_version_var}
COMMAND cd ${PROJECT_BINARY_DIR}/build_${flag_target}_${min_version_var}
&& ${cmake_binary} -Werror=dev ${PROJECT_SOURCE_DIR} -D${flag}=ON
DEPENDS ${cmake_binary}
)
list(APPEND JSON_CMAKE_FLAG_TARGETS ${flag_target}_${min_version_var})
list(APPEND JSON_CMAKE_FLAG_BUILD_DIRS ${PROJECT_BINARY_DIR}/build_${flag_target}_${min_version_var})
list(APPEND JSON_CMAKE_FLAG_TARGETS ${JSON_CMAKE_FLAG_TARGET} ${flag_target}_${min_version_var})
list(APPEND JSON_CMAKE_FLAG_BUILD_DIRS ${PROJECT_BINARY_DIR}/build_${flag_target} ${PROJECT_BINARY_DIR}/build_${flag_target}_${min_version_var})
set(JSON_CMAKE_FLAG_TARGETS ${JSON_CMAKE_FLAG_TARGETS} PARENT_SCOPE)
set(JSON_CMAKE_FLAG_BUILD_DIRS ${JSON_CMAKE_FLAG_BUILD_DIRS} PARENT_SCOPE)
endfunction()
@@ -642,20 +696,6 @@ foreach(JSON_CMAKE_FLAG ${JSON_CMAKE_FLAGS_4_0_0})
ci_add_cmake_flags_targets(${JSON_CMAKE_FLAG} 4.0.0)
endforeach()
# check the same flags with the CMake version running this build
foreach(JSON_CMAKE_FLAG ${JSON_CMAKE_FLAGS_4_0_0})
string(TOLOWER "ci_cmake_flag_${JSON_CMAKE_FLAG}" flag_target)
add_custom_target(${flag_target}
COMMENT "Check CMake flag ${JSON_CMAKE_FLAG} (CMake ${CMAKE_VERSION})"
COMMAND ${CMAKE_COMMAND}
-Werror=dev
-D${JSON_CMAKE_FLAG}=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_${flag_target}
)
list(APPEND JSON_CMAKE_FLAG_TARGETS ${flag_target})
list(APPEND JSON_CMAKE_FLAG_BUILD_DIRS ${PROJECT_BINARY_DIR}/build_${flag_target})
endforeach()
add_custom_target(ci_cmake_flags
DEPENDS ${JSON_CMAKE_FLAG_TARGETS}
COMMENT "Check CMake flags"
@@ -814,6 +854,6 @@ add_custom_target(ci_test_build_documentation
###############################################################################
add_custom_target(ci_clean
COMMAND rm -fr ${PROJECT_BINARY_DIR}/build_* ${PROJECT_BINARY_DIR}/cmake-3.5.0 ${PROJECT_BINARY_DIR}/cmake-3.31.6 ${PROJECT_BINARY_DIR}/cmake-4.0.0 ${JSON_CMAKE_FLAG_BUILD_DIRS} ${single_binaries}
COMMAND rm -fr ${PROJECT_BINARY_DIR}/build_* cmake-3.5.0-Darwin64 ${JSON_CMAKE_FLAG_BUILD_DIRS} ${single_binaries}
COMMENT "Clean generated directories"
)
@@ -55,6 +55,10 @@ This implementation does exactly follow this approach, as it uses double precisi
smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
and be serialized to `null`.
During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
#### Storage
Floating-point number values are stored directly inside a `basic_json` type.
@@ -47,8 +47,9 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va
When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).
range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
or [`number_float_t`](number_float_t.md).
[RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
> Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
@@ -48,8 +48,9 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default
When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).
when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
[`number_float_t`](number_float_t.md).
[RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
> Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are
@@ -90,7 +90,7 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
| GNU 14.2.0 | x86_64 | Ubuntu 22.04.1 LTS | GitHub |
| GNU 15.1.0 | x86_64 | Ubuntu 22.04.1 LTS | GitHub |
| GNU 16.1.0 | x86_64 | Ubuntu 22.04.1 LTS | GitHub |
| GNU 16.1.0 | arm64 | Ubuntu 24.04 | GitHub |
| GNU 16.1.0 | arm64 | Linux 6.1.100 | Cirrus CI |
| icpc (ICC) 2021.10.0 20230609 | x86_64 | Ubuntu 22.04 LTS | GitHub |
| icpx (Intel oneAPI DPC++/C++) 2025.3.2 | x86_64 | Ubuntu 24.04 LTS | GitHub |
| nvc++ (NVIDIA HPC SDK) 25.5-0 | x86_64 | Ubuntu 22.04 LTS | GitHub |
@@ -132,8 +132,7 @@ Note: Some modern features (like C++20 ranges or filesystem support) may be disa
[libstdc++](https://gcc.gnu.org/onlinedocs/libstdc++/) to detect subtle differences or incompatibilities.
- [x] The code checked with [Include What You Use (IWYU)](https://include-what-you-use.org) that all required standard
headers are included.
- [x] On Windows, the library is compiled with `<Windows.h>` being included to detect and avoid common bugs (see
[`unit-windows_h.cpp`](https://github.com/nlohmann/json/blob/develop/tests/src/unit-windows_h.cpp)).
- [x] On Windows, the library is compiled with `<Windows.h>` being included to detect and avoid common bugs.
- [x] The library is compiled with exceptions disabled to support alternative means of error handling.
## Stable public API
@@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows:
!!! warning "Negative integer overflow"
CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
silently.
result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
stored as `-1.8446744073709552e+19`.
!!! warning "Object keys"
+7 -5
View File
@@ -331,9 +331,6 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
[json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
```
```
[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
```
```
[json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
```
@@ -854,13 +851,18 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme
### json.exception.out_of_range.406
A parsed number could not be stored as without changing it to NaN or INF.
A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
double-precision number when `number_float_t` is `#!cpp float`.
!!! failure "Example message"
!!! failure "Example messages"
```
number overflow parsing '10E1000'
```
```
[json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
```
### json.exception.out_of_range.407
+133 -45
View File
@@ -559,7 +559,7 @@ class binary_reader
case 0x01: // double
{
double number{};
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
}
case 0x02: // string
@@ -600,19 +600,19 @@ class binary_reader
case 0x10: // int32
{
std::int32_t value{};
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
}
case 0x12: // int64
{
std::int64_t value{};
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
}
case 0x11: // uint64
{
std::uint64_t value{};
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(input_format_t::bson, value);
}
default: // anything else is not supported (yet)
@@ -638,14 +638,19 @@ class binary_reader
{
return false;
}
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
if (number > max_val)
// the value is -1 - number, which fits into number_integer_t
// whenever number does
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
{
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(112, chars_read,
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
}
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
// like the lexer does for JSON text, store a value too small for
// number_integer_t as number_float_t; compute it as long double so
// that emit_float sees a finite value and can detect an overflow of
// number_float_t
return emit_float(input_format_t::cbor, static_cast<long double>(-1) - static_cast<long double>(number));
}
/*!
@@ -702,25 +707,25 @@ class binary_reader
case 0x18: // Unsigned integer (one-byte uint8_t follows)
{
std::uint8_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
case 0x19: // Unsigned integer (two-byte uint16_t follows)
{
std::uint16_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
{
std::uint32_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
{
std::uint64_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
// Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -1165,13 +1170,13 @@ class binary_reader
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
{
float number{};
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
}
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
{
double number{};
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
}
default: // anything else (0xFF is handled inside the other types)
@@ -1935,61 +1940,61 @@ class binary_reader
case 0xCA: // float 32
{
float number{};
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
}
case 0xCB: // float 64
{
double number{};
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
}
case 0xCC: // uint 8
{
std::uint8_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xCD: // uint 16
{
std::uint16_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xCE: // uint 32
{
std::uint32_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xCF: // uint 64
{
std::uint64_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xD0: // int 8
{
std::int8_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xD1: // int 16
{
std::int16_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xD2: // int 32
{
std::int32_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xD3: // int 64
{
std::int64_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xDC: // array 16
@@ -2922,7 +2927,7 @@ class binary_reader
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(input_format, i)))
{
return false;
}
@@ -3054,37 +3059,37 @@ class binary_reader
break;
}
std::uint8_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'U':
{
std::uint8_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'i':
{
std::int8_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'I':
{
std::int16_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'l':
{
std::int32_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'L':
{
std::int64_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'u':
@@ -3094,7 +3099,7 @@ class binary_reader
break;
}
std::uint16_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'm':
@@ -3104,7 +3109,7 @@ class binary_reader
break;
}
std::uint32_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'M':
@@ -3114,7 +3119,7 @@ class binary_reader
break;
}
std::uint64_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'h':
@@ -3172,13 +3177,13 @@ class binary_reader
case 'd':
{
float number{};
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format, number) && emit_float(input_format, number);
}
case 'D':
{
double number{};
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format, number) && emit_float(input_format, number);
}
case 'H':
@@ -3645,13 +3650,13 @@ class binary_reader
case 0x8E: // binary32
{
float number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
}
case 0x8F: // binary64
{
double number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
}
case 0xF8:
@@ -3717,7 +3722,9 @@ class binary_reader
@brief pass an integer to the SAX parser
Non-negative integers are passed as unsigned, negative integers as signed
numbers, like the other binary formats do.
numbers, like the other binary formats do. A value that does not fit the
number type is passed as described for @ref emit_unsigned and
@ref emit_signed.
@param[in] number the integer
@return whether the SAX parser accepted the value
@@ -3726,9 +3733,9 @@ class binary_reader
{
if (number >= 0)
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
return emit_unsigned(input_format_t::bon8, static_cast<std::uint64_t>(number));
}
return sax->number_integer(static_cast<number_integer_t>(number));
return emit_signed(input_format_t::bon8, number);
}
/*!
@@ -3785,8 +3792,7 @@ class binary_reader
value = (value << 8) | static_cast<std::int64_t>(current);
}
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
return emit_bon8_integer(negative ? -(value + offset) : value + offset);
}
/*!
@@ -4083,6 +4089,88 @@ class binary_reader
return true;
}
/*!
@brief pass a signed integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_integer_t is passed as number_unsigned_t if it is non-negative and
fits there, and as number_float_t otherwise. With the default number
types, every integer the binary formats can encode fits, so this only
matters for narrower custom number types.
@tparam NumberType a signed integer type
@param[in] format the current format (for diagnostics)
@param[in] number the integer
@return whether the SAX parser accepted the value
@throw out_of_range.406 if @a number overflows number_float_t (see
@ref emit_float)
*/
template<typename NumberType>
bool emit_signed(const input_format_t format, const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
{
return sax->number_integer(static_cast<number_integer_t>(number));
}
if (value_in_range_of<number_unsigned_t>(number))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return emit_float(format, number);
}
/*!
@brief pass an unsigned integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_unsigned_t is passed as number_float_t.
@tparam NumberType an unsigned integer type
@param[in] format the current format (for diagnostics)
@param[in] number the integer
@return whether the SAX parser accepted the value
@throw out_of_range.406 if @a number overflows number_float_t (see
@ref emit_float)
*/
template<typename NumberType>
bool emit_unsigned(const input_format_t format, const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return emit_float(format, number);
}
/*!
@brief pass a floating-point number read from the input to the SAX parser
Like the lexer does for JSON text, a finite value that overflows
number_float_t is rejected instead of silently becoming infinity. Infinity
and NaN in the input are passed on unchanged. Integers only overflow if
number_float_t cannot represent 2^64, e.g., a half-precision type.
@tparam NumberType a floating-point or integer type
@param[in] format the current format (for diagnostics)
@param[in] number the number
@return whether the SAX parser accepted the value
@throw out_of_range.406 if a finite @a number overflows number_float_t
*/
template<typename NumberType>
bool emit_float(const input_format_t format, const NumberType number)
{
const auto result = static_cast<number_float_t>(number);
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
{
return sax->parse_error(chars_read, get_token_string(),
out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
}
return sax->number_float(result, "");
}
/*!
@brief create a string by reading characters from the input
+133 -45
View File
@@ -13326,7 +13326,7 @@ class binary_reader
case 0x01: // double
{
double number{};
return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
}
case 0x02: // string
@@ -13367,19 +13367,19 @@ class binary_reader
case 0x10: // int32
{
std::int32_t value{};
return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
}
case 0x12: // int64
{
std::int64_t value{};
return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
}
case 0x11: // uint64
{
std::uint64_t value{};
return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(input_format_t::bson, value);
}
default: // anything else is not supported (yet)
@@ -13405,14 +13405,19 @@ class binary_reader
{
return false;
}
const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
if (number > max_val)
// the value is -1 - number, which fits into number_integer_t
// whenever number does
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
{
return sax->parse_error(chars_read, get_token_string(),
parse_error::create(112, chars_read,
exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
}
return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
// like the lexer does for JSON text, store a value too small for
// number_integer_t as number_float_t; compute it as long double so
// that emit_float sees a finite value and can detect an overflow of
// number_float_t
return emit_float(input_format_t::cbor, static_cast<long double>(-1) - static_cast<long double>(number));
}
/*!
@@ -13469,25 +13474,25 @@ class binary_reader
case 0x18: // Unsigned integer (one-byte uint8_t follows)
{
std::uint8_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
case 0x19: // Unsigned integer (two-byte uint16_t follows)
{
std::uint16_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
case 0x1A: // Unsigned integer (four-byte uint32_t follows)
{
std::uint32_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
{
std::uint64_t number{};
return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
}
// Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -13932,13 +13937,13 @@ class binary_reader
case 0xFA: // Single-Precision Float (four-byte IEEE 754)
{
float number{};
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
}
case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
{
double number{};
return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
}
default: // anything else (0xFF is handled inside the other types)
@@ -14702,61 +14707,61 @@ class binary_reader
case 0xCA: // float 32
{
float number{};
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
}
case 0xCB: // float 64
{
double number{};
return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
}
case 0xCC: // uint 8
{
std::uint8_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xCD: // uint 16
{
std::uint16_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xCE: // uint 32
{
std::uint32_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xCF: // uint 64
{
std::uint64_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
}
case 0xD0: // int 8
{
std::int8_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xD1: // int 16
{
std::int16_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xD2: // int 32
{
std::int32_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xD3: // int 64
{
std::int64_t number{};
return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
}
case 0xDC: // array 16
@@ -15689,7 +15694,7 @@ class binary_reader
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(input_format, i)))
{
return false;
}
@@ -15821,37 +15826,37 @@ class binary_reader
break;
}
std::uint8_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'U':
{
std::uint8_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'i':
{
std::int8_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'I':
{
std::int16_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'l':
{
std::int32_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'L':
{
std::int64_t number{};
return get_number(input_format, number) && sax->number_integer(number);
return get_number(input_format, number) && emit_signed(input_format, number);
}
case 'u':
@@ -15861,7 +15866,7 @@ class binary_reader
break;
}
std::uint16_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'm':
@@ -15871,7 +15876,7 @@ class binary_reader
break;
}
std::uint32_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'M':
@@ -15881,7 +15886,7 @@ class binary_reader
break;
}
std::uint64_t number{};
return get_number(input_format, number) && sax->number_unsigned(number);
return get_number(input_format, number) && emit_unsigned(input_format, number);
}
case 'h':
@@ -15939,13 +15944,13 @@ class binary_reader
case 'd':
{
float number{};
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format, number) && emit_float(input_format, number);
}
case 'D':
{
double number{};
return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format, number) && emit_float(input_format, number);
}
case 'H':
@@ -16412,13 +16417,13 @@ class binary_reader
case 0x8E: // binary32
{
float number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
}
case 0x8F: // binary64
{
double number{};
return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
}
case 0xF8:
@@ -16484,7 +16489,9 @@ class binary_reader
@brief pass an integer to the SAX parser
Non-negative integers are passed as unsigned, negative integers as signed
numbers, like the other binary formats do.
numbers, like the other binary formats do. A value that does not fit the
number type is passed as described for @ref emit_unsigned and
@ref emit_signed.
@param[in] number the integer
@return whether the SAX parser accepted the value
@@ -16493,9 +16500,9 @@ class binary_reader
{
if (number >= 0)
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
return emit_unsigned(input_format_t::bon8, static_cast<std::uint64_t>(number));
}
return sax->number_integer(static_cast<number_integer_t>(number));
return emit_signed(input_format_t::bon8, number);
}
/*!
@@ -16552,8 +16559,7 @@ class binary_reader
value = (value << 8) | static_cast<std::int64_t>(current);
}
return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
: sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
return emit_bon8_integer(negative ? -(value + offset) : value + offset);
}
/*!
@@ -16850,6 +16856,88 @@ class binary_reader
return true;
}
/*!
@brief pass a signed integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_integer_t is passed as number_unsigned_t if it is non-negative and
fits there, and as number_float_t otherwise. With the default number
types, every integer the binary formats can encode fits, so this only
matters for narrower custom number types.
@tparam NumberType a signed integer type
@param[in] format the current format (for diagnostics)
@param[in] number the integer
@return whether the SAX parser accepted the value
@throw out_of_range.406 if @a number overflows number_float_t (see
@ref emit_float)
*/
template<typename NumberType>
bool emit_signed(const input_format_t format, const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
{
return sax->number_integer(static_cast<number_integer_t>(number));
}
if (value_in_range_of<number_unsigned_t>(number))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return emit_float(format, number);
}
/*!
@brief pass an unsigned integer read from the input to the SAX parser
Like the lexer does for JSON text, a value that does not fit into
number_unsigned_t is passed as number_float_t.
@tparam NumberType an unsigned integer type
@param[in] format the current format (for diagnostics)
@param[in] number the integer
@return whether the SAX parser accepted the value
@throw out_of_range.406 if @a number overflows number_float_t (see
@ref emit_float)
*/
template<typename NumberType>
bool emit_unsigned(const input_format_t format, const NumberType number)
{
if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
{
return sax->number_unsigned(static_cast<number_unsigned_t>(number));
}
return emit_float(format, number);
}
/*!
@brief pass a floating-point number read from the input to the SAX parser
Like the lexer does for JSON text, a finite value that overflows
number_float_t is rejected instead of silently becoming infinity. Infinity
and NaN in the input are passed on unchanged. Integers only overflow if
number_float_t cannot represent 2^64, e.g., a half-precision type.
@tparam NumberType a floating-point or integer type
@param[in] format the current format (for diagnostics)
@param[in] number the number
@return whether the SAX parser accepted the value
@throw out_of_range.406 if a finite @a number overflows number_float_t
*/
template<typename NumberType>
bool emit_float(const input_format_t format, const NumberType number)
{
const auto result = static_cast<number_float_t>(number);
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
{
return sax->parse_error(chars_read, get_token_string(),
out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
}
return sax->number_float(result, "");
}
/*!
@brief create a string by reading characters from the input
+141
View File
@@ -11,7 +11,12 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cmath>
#include <fstream>
#include <limits>
#include <map>
#include <string>
#include <vector>
#include "make_test_data_available.hpp"
TEST_CASE("Binary Formats" * doctest::skip())
@@ -224,3 +229,139 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
}
}
namespace
{
// the binary formats as function pointers for "Binary formats with narrow number types";
// named functions rather than lambdas, because clang 3.5 cannot convert a lambda
// to a function pointer in the braced initializer of the format table
using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
using bytes = std::vector<std::uint8_t>;
bytes encode_cbor(const json& j)
{
return json::to_cbor(j);
}
narrow_json decode_cbor(const bytes& v, bool allow_exceptions)
{
return narrow_json::from_cbor(v, true, allow_exceptions);
}
bytes encode_msgpack(const json& j)
{
return json::to_msgpack(j);
}
narrow_json decode_msgpack(const bytes& v, bool allow_exceptions)
{
return narrow_json::from_msgpack(v, true, allow_exceptions);
}
bytes encode_ubjson(const json& j)
{
return json::to_ubjson(j);
}
narrow_json decode_ubjson(const bytes& v, bool allow_exceptions)
{
return narrow_json::from_ubjson(v, true, allow_exceptions);
}
bytes encode_bjdata(const json& j)
{
return json::to_bjdata(j);
}
narrow_json decode_bjdata(const bytes& v, bool allow_exceptions)
{
return narrow_json::from_bjdata(v, true, allow_exceptions);
}
// BSON can only store numbers as object members
bytes encode_bson(const json& j)
{
return json::to_bson(json{{"a", j}});
}
narrow_json decode_bson(const bytes& v, bool allow_exceptions)
{
const auto result = narrow_json::from_bson(v, true, allow_exceptions);
return result.is_discarded() ? result : result.at("a");
}
bytes encode_bon8(const json& j)
{
return json::to_bon8(j);
}
narrow_json decode_bon8(const bytes& v, bool allow_exceptions)
{
return narrow_json::from_bon8(v, true, allow_exceptions);
}
} // namespace
TEST_CASE("Binary formats with narrow number types")
{
// Numbers that do not fit the number types are handled like the lexer
// handles them in JSON text: an integer that fits neither integer type is
// stored as a floating-point number, and a finite floating-point number
// that overflows number_float_t is rejected with out_of_range.406.
struct binary_format
{
const char* name;
bytes (*encode)(const json&);
narrow_json (*decode)(const bytes&, bool);
};
const std::vector<binary_format> formats =
{
{"CBOR", encode_cbor, decode_cbor},
{"MessagePack", encode_msgpack, decode_msgpack},
{"UBJSON", encode_ubjson, decode_ubjson},
{"BJData", encode_bjdata, decode_bjdata},
{"BSON", encode_bson, decode_bson},
{"BON8", encode_bon8, decode_bon8},
};
for (const auto& format : formats)
{
const std::string name = format.name;
INFO("format := ", name);
const auto roundtrip = [&format](const json & j)
{
return format.decode(format.encode(j), true);
};
// integers that fit keep their type
CHECK(roundtrip(json(-5)).is_number_integer());
CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
// integers that fit neither integer type are stored as float
CHECK(roundtrip(json(5000000000u)).is_number_float());
CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
{
CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
}
CHECK(roundtrip(json(-3000000000LL)).is_number_float());
CHECK(roundtrip(json(-3000000000LL)).get<float>() == -3000000000.0f);
CHECK(roundtrip(json(-5000000000LL)).is_number_float());
CHECK(roundtrip(json(-5000000000LL)).get<float>() == -5000000000.0f);
// floating-point numbers that fit
CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
std::numeric_limits<double>::infinity());
CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
// infinity and NaN are passed on
CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
// finite floating-point numbers that overflow number_float_t are rejected
const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
+ " value: number overflow";
CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
}
}
+16 -14
View File
@@ -3187,7 +3187,8 @@ TEST_CASE("Tagged values")
// CBOR encodes negative integers as: result = -1 - n
// For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
// the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
// When n > INT64_MAX, the result exceeds int64_t range and is rejected.
// When n > INT64_MAX, the result exceeds int64_t range and is stored
// as a floating-point number, as the lexer does for JSON text.
SECTION("n = 0 is valid (result = -1)")
{
@@ -3208,33 +3209,34 @@ TEST_CASE("Tagged values")
CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
}
SECTION("n = INT64_MAX + 1 is rejected (overflow)")
SECTION("n = INT64_MAX + 1 is stored as float")
{
// n = INT64_MAX + 1 (0x8000000000000000)
// result = -1 - n = -9223372036854775809, which exceeds int64_t range
// result = -1 - n = -9223372036854775809, which exceeds int64_t range;
// the nearest double is -9223372036854775808.0
const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
"[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
json::parse_error);
const auto result = json::from_cbor(input);
CHECK(result.is_number_float());
CHECK(result.get<double>() == -9223372036854775808.0);
CHECK(result == json::parse("-9223372036854775809"));
}
SECTION("n = UINT64_MAX is rejected (overflow)")
SECTION("n = UINT64_MAX is stored as float")
{
// n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
// result = -1 - n = -18446744073709551616, which exceeds int64_t range
const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
"[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
json::parse_error);
const auto result = json::from_cbor(input);
CHECK(result.is_number_float());
CHECK(result.get<double>() == -18446744073709551616.0);
CHECK(result == json::parse("-18446744073709551616"));
}
SECTION("overflow with allow_exceptions=false returns discarded")
SECTION("overflow with allow_exceptions=false is not an error")
{
const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
const auto result = json::from_cbor(input, true, false);
CHECK(result.is_discarded());
CHECK(result.is_number_float());
}
}
-18
View File
@@ -7,22 +7,4 @@
// SPDX-License-Identifier: MIT
#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
// libc++ annotates std::mutex for Clang's thread safety analysis, so -Weverything
// reports -Wthread-safety-negative for the locks in doctest's reporters, which are
// only compiled in this file. __has_warning keeps older Clang versions from
// reporting an unknown warning group.
#if defined(__clang__) && defined(__has_warning)
#if __has_warning("-Wthread-safety-negative")
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wthread-safety-negative"
#endif
#endif
#include "doctest_compatibility.h"
#if defined(__clang__) && defined(__has_warning)
#if __has_warning("-Wthread-safety-negative")
#pragma clang diagnostic pop
#endif
#endif
-3
View File
@@ -1,4 +1 @@
# astyle is deliberately pinned: newer versions reformat the code, and this version
# defines the formatting that `make amalgamate` produces and check_amalgamation.yml enforces.
# Dependabot is told to ignore it in .github/dependabot.yml.
astyle==3.4.13