Resolves a conflict in include/nlohmann/detail/input/input_adapters.hpp
between this branch's memcpy fast path for get_elements() and develop's
lazy-diagnostics feature (#5234), which added a `begin` iterator member,
`get_consumed_count()`, and `copy_consumed_range()` to the same class.
The two features are orthogonal (lexer.hpp uses the new diagnostics
helpers; binary_reader.hpp uses get_elements()) and both are preserved.
single_include/nlohmann/json.hpp is regenerated from the merged sources.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
`lexer::get()` copied every scanned character into `token_string` on the
whole successful-parse hot path, yet that buffer is consumed only by
`get_token_string()` when rendering the "last read" fragment of a parse
error. On well-formed input the per-byte copy (plus the `unget()` pop)
is pure overhead that is always discarded.
For seekable input adapters - random-access, single-byte iterators such
as those backing `std::string`, `const char*`, and `std::vector<char>` -
the offending token is now reconstructed on demand from the input when
an error is reported, using a saved start offset, and the eager copy is
skipped. Streaming adapters (file, istream, wide-string, and user-defined
adapters) keep the eager copy; the strategy is chosen at compile time via
`input_adapter_supports_seek`, so adapters without the capability are
unaffected.
Error messages are byte-for-byte identical across all adapters, verified
by a new parity regression test. Microbenchmark (4 MB mixed JSON, parsed
from a std::string): ~149 -> ~160 MB/s, about +8%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
std::vector::resize(n) (and string_t's resize) is required to make
size() exactly n; only capacity() is permitted to overshoot. The
destination-size assert in get_bytes should reflect that exact guarantee
rather than a weaker `>=`.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get_elements() only receives a raw pointer and cannot know the size of
its destination, so the destination-size guarantee is asserted at the
caller that owns the buffer: after resizing `result`, assert it has room
for `wanted` bytes at `old_size`. Together with the `copied <= wanted`
assert at the memcpy, this makes the "destination is large enough"
invariant explicit on both sides of the call.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Harden CI workflows: validate PR artifact inputs and migrate off deprecated Semgrep action
Address two CI/supply-chain hardening items from the 2026-07-03 security
audit:
- comment_check_amalgamation.yml (todo 117): the privileged `workflow_run`
job consumes an untrusted PR artifact. Validate `author` against a strict
GitHub-username pattern and `number` as a positive integer before use, and
extract the artifact into a dedicated directory (`unzip -o pr.zip -d
./pr_artifact`), reading only the two expected files by fixed path. This
prevents Markdown/mention injection via the attacker-controlled `author`
text and avoids a malicious archive touching the workspace.
- semgrep.yml (todo 118): `returntocorp/semgrep-action` is deprecated (the
org was renamed to `semgrep/*`). Replace it with an explicit `semgrep ci`
invocation via the maintained CLI; the deployment is inferred from
SEMGREP_APP_TOKEN.
Todo 116 (CIFuzz `@master` refs) already carries a comment documenting the
OSS-Fuzz-recommended exception, so no change is needed there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Semgrep step: use `semgrep scan` instead of token-gated `semgrep ci`
The CI `Scan` job failed with "Path does not exist: semgrep.sarif" because
`semgrep ci` requires a login token (SEMGREP_APP_TOKEN), which this repo does
not have configured, so it bailed without producing a SARIF file. The former
returntocorp/semgrep-action, given no token, fell back to plain
`semgrep scan --sarif`; match that with `semgrep scan --config auto`, which
needs no token and always produces the SARIF for upload.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Make the "buffers are large enough" reasoning explicit: before the copy,
assert that `copied` fits both the caller-provided destination (`wanted`
bytes) and the remaining input range (`available` bytes). These hold by
construction (`copied = min(wanted, available)`) but were previously only
implied; the asserts document the invariant and would catch a future
regression that breaks it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Flawfinder flags the bare identifier `read` as the POSIX read() syscall
(CWE-120/CWE-20), producing several false-positive code-scanning alerts.
Renaming the local variable removes them; the byte-copy itself is
unchanged and remains bounds-checked (get_elements caps the copy to the
bytes available). This is a pure rename with identical semantics.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- get_bytes(): replace `len -= static_cast<NumberType>(wanted)` with an
explicit `len = static_cast<NumberType>(len - ...)`. The compound
assignment promoted the operands to int and narrowed back, which GCC
rejects under -Werror=arith-conversion when NumberType is narrower than
int (e.g. UBJSON's int16 length). This also fixes the CodeQL build.
- get_bytes(): add JSON_ASSERT(read == wanted) documenting that
get_elements() never returns more than requested (per review).
- iterator_input_adapter: de-duplicate the iterator_is_contiguous
definition with the #if inside the initializer (per review).
- Add a comment explaining why the memcpy source is &*current (needed for
non-pointer contiguous iterators), and drop the redundant parentheses.
- Run astyle on the new unit test so the amalgamation/format check passes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The binary reader read CBOR/MessagePack/BSON/UBJSON strings and byte
arrays one byte at a time via get()/push_back(), and the iterator input
adapter's get_elements() fallback was itself a per-byte loop, so even
contiguous inputs never benefited from a block copy.
Two changes:
1. iterator_input_adapter::get_elements() gains a contiguous fast path
that copies the whole requested range with std::memcpy. Contiguity is
detected via std::is_pointer (all standards) and, in C++20,
std::contiguous_iterator (so std::vector/std::string iterators also
qualify). Non-contiguous iterators keep the element-by-element loop.
2. get_string()/get_binary() now share get_bytes(), which reads into the
result in bounded chunks through get_elements() instead of byte by
byte. Capping the chunk size preserves the deliberate "do not
reserve(len) for an untrusted length" DoS protection while turning the
inner loop into block copies. The min(chunk_size, len) computation is
width-safe so narrow length types (e.g. MessagePack ext-8's uint8_t)
cannot truncate chunk_size to zero.
Microbenchmark (2 MiB string + 2 MiB blob + 2000x1 KiB strings, Apple
clang, -O2):
C++20 from_cbor(vector) 20.1 ms -> 1.0 ms (~20x)
from_cbor(pointer) 18.9 ms -> 1.0 ms (~19x)
C++17 from_cbor(pointer) 18.7 ms -> 1.0 ms (~19x, memcpy)
from_cbor(vector) 18.7 ms -> 4.0 ms (~4.6x, tight loop)
Behavior is unchanged: truncated input still throws parse_error.110 at
the same byte offset, and all binary-format unit tests pass.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bumps init/analyze/autobuild together (previously split across #5226,
#5227, #5228, which each failed CI due to a version mismatch between
the CodeQL config and the running action). Also adds a dependabot
group so future codeql-action bumps land in a single PR.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When a PR is not amalgamated/formatted, the astyle version friction (see
the recurring blocker across many PRs) means contributors often struggle
to reproduce the exact fix locally. The check now regenerates the
amalgamation and formatting, captures the difference as a patch, and
uploads it as the `amalgamation-patch` artifact. The failure comment
links to that artifact and tells contributors to run
`git apply amalgamation.patch`, so they no longer need to install the
pinned astyle version themselves.
The pass/fail verdict is unchanged: the same PRs fail as before, and a
correctly amalgamated PR uploads nothing and passes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add /wd4702 to disable warning C4702: unreachable code in MSVC Release build.
Signed-off-by: Richard Musil <risa2000x@gmail.com>
Co-authored-by: Richard Musil <risa2000x@gmail.com>
* Added NLOHNMANN_JSON_SERIALIZE_ENUM_STRICT
- duplicate of NLOHMANN_JSON_SERIALIZE_ENUM
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* Added failing tests for NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* modified NLOHMANN_JSON_SERIALIZE_STRICT to throw
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* added documentation and changed readme to include NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* ran amalgamate
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* docs(macros): add page for JSON_SERIALIZE_ENUM_STRICT
- added page to nav
- added links to new page where appropriate
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* refactor(macros): make JSON_SERIALIZE_ENUM_STRICT use JSON_THROW
- added templated wrapper function to fix scope error in calling JSON_THROW
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* refactor(macros): make NLOHMANN_SERIALIZE_ENUM_STRICT use error code 410
- added error code 410 to docs
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* tests(macros): add test for to_json with enum value not mentioned
in mapping for NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* Apply suggestions from code review
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Caillin Nugent <nugentcaillin@gmail.com>
* fix(macro): prevent compilation error with -Werror and -Wunused-parameter
with NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
- casted exception to void to avoid warning
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* fix(docs): add link to NLOHMANN_SERIALIZE_ENUM_STRICT docs to exception page
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* docs(macros): add example of exception throwing for NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* refactor(macros): add more in-depth error message to NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
- changed error message to follow style of nlohmann/json#4989
- made description of throw wrapper more general
- updated tests and example of exceptions
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
---------
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
Signed-off-by: Caillin Nugent <nugentcaillin@gmail.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Fix for printing long doubles bug in dump_float
When you use long double as a floating point type with the current version of this file and try to dump json it prints trash instead of actual number. This if-else fixes the problem. On using long double you just need to add an 'L' modifier before 'g' in format string.
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
* C++11 compatibility
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
* Shorter solution
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
* Applied amalgamate
Signed-off-by: rusloker <klokotkov@ya.ru>
* Add unit tests for `dump()` with `long double` in custom `basic_json`
Signed-off-by: rusloker <klokotkov@ya.ru>
* Fix UB in `snprintf_float` by using `%.*Lg` for `long double`
Signed-off-by: rusloker <klokotkov@ya.ru>
* Use `std::array` for `values` in serialization unit tests to improve type safety
Signed-off-by: rusloker <klokotkov@ya.ru>
* Fix brace initialization for `std::array` in serialization unit tests
Signed-off-by: rusloker <klokotkov@ya.ru>
* Remove comments in `snprintf_float` regarding `%Lg` usage
Signed-off-by: rusloker <klokotkov@ya.ru>
* Skip `long double` infinity dump assertions under Valgrind
Signed-off-by: rusloker <klokotkov@ya.ru>
* Clarify Valgrind bug-tracker reference in `long double` test
Signed-off-by: rusloker <klokotkov@ya.ru>
* Satisfy clang-tidy in `long double` infinity probe
Signed-off-by: rusloker <klokotkov@ya.ru>
---------
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
Signed-off-by: rusloker <klokotkov@ya.ru>
* Add new macros for named conversions
* Unit tests for the named conversion macros
* Update the docs to include the new macros
* Fix the documentation for the macros
the correct maximum number of member variables is 63
* Fix CI tests
* update the named macros
* move the example files
* update the explicit macros expansion
* update documentation
* fix documentation hiccups
* astyle changes
* add static analysis exceptions
* change md header to explicit html to fit the length
* Small corrections to docs
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: George Sedov <radist.morse@gmail.com>
---------
Signed-off-by: George Sedov <radist.morse@gmail.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
Niels Lohmann
dependabot[bot]
Cosmin D.
risa2000andRichard Musil
Swastik Bose
Miko
Caillin NugentandNiels Lohmann
Kirill Lokotkov
George SedovandNiels Lohmann
Hariom Phulre