Files
json/docs/mkdocs/docs/api/macros/json_diagnostic_positions.md
T
Niels LohmannandClaude Sonnet 5.5 33f15727ce Parse JSON in one block of memory with a fused pointer-based parser (#6031)
* Parse JSON in one block of memory with a fused pointer-based parser

The parser and the lexer exchange one token at a time, and the lexer keeps
its state (the current character, a pending unget, where the token starts)
in members, which every character pays for: json::accept() of twitter.json
took 2.8x as long as json_document::accept(). detail::contiguous_parser
reads text in one block of memory through a local pointer, with the
grammar, the whitespace and the tokens in one loop (after yyjson's reader),
and drives a SAX handler: parse()'s DOM handler, accept()'s handler, or the
user's handler of sax_parse(). It is used with the default options (no
callback, no comments, no trailing commas; sax_parse() with strict = true).

Errors are reported by the library parser, so the exception, its message and
its position stay the same. contiguous_parser stops at the token that the
library parser would reject, before reporting it, so the events up to the
error are the same; then:
- parse() runs the library parser with a handler that only reports the
  error (json_sax_error_reporter), with the lexer of parse(), which is
  instantiated anyway;
- sax_parse() runs it with a handler that forwards only parse_error() to
  the user's handler (json_sax_error_forwarder), so no event arrives twice;
- accept() returns the verdict.
A handler that returns false stops the parse as before. An error at the end
of a large input costs parse() about 1.2x of what it cost before.

On an Apple M1 Max: accept() 1.6-2.0x as fast, parse() 10-16% faster,
ordered_json 11-16%, sax_parse() with a user's handler 1.36-1.46x, small
messages 12-22% (parse()) and 2.1-2.4x (accept()). On x86-64 (GCC 15,
Clang 21): accept() 1.6-1.9x, parse() 5-17%, ordered_json 12-19%,
sax_parse() 1.32-1.57x. A translation unit that parses a string compiles 3%
(-O0) and 9% (-O2) slower.

unit-contiguous_parser.cpp compares the contiguous path with the byte path
(std::list iterators) on corner cases, generated documents, their
truncations and mutations, deep nesting, and narrow number types: the same
values with the same number types and float bits, exception messages,
accept() verdicts and sax_parse() events.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: unused parameter without exceptions, cpplint, BUILD.bazel

- json_sax_error_reporter::parse_error: mark the exception unused, as
  JSON_THROW does not use it with JSON_NOEXCEPTION
- unit-contiguous_parser.cpp: compare the exceptions of both paths only
  when exceptions are enabled (invalid input aborted the test otherwise)
- silence cpplint's readability/braces false positives on the if
  statements inside the parser's macros
- add contiguous_parser.hpp to BUILD.bazel

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* accept(): skip digits without computing their value

accept() needs the value of a number only for a float whose exponent may
reach the largest value. skip_digits() finds the end of the digits without
the multiply-adds of accumulate_digits(), and once an 8-byte word fails the
digit test, the remaining loop needs no bounds check: a non-digit is in
that word. The value is computed afterwards for the rare floats that need
it.

accept() of canada.json: -16 % on Apple M1, -22 % with GCC 15 and -21 %
with Clang 21 on x86-64; files with many numbers -9 to -15 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: mark parser callback in unit-contiguous_parser noexcept

Fixes the -Werror=noexcept failure with GCC 26 (ci_test_standards_gcc (26)).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: sweep_allocator::construct() accepts pointers to const

Recent libc++ (emsdk latest) probes construct(const T*, const T&) in an
unevaluated context while checking for trivial copies; the static_cast to
void* in the noexcept specification then fails to compile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit 794bf70823c12a2a1cb2a8a95f34dfb6357746a8)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: guard sweep_allocator::construct() against const types instead of const_cast

The const_cast added in the previous commit is rejected by
cppcoreguidelines-pro-type-const-cast (clang-tidy). Make construct() not
viable for const U instead, which is what libc++ probes for.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
(cherry picked from commit de099fa607001b710862fd1bbecef3609c54cbc4)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI: astyle and clang-tidy for sweep_allocator::construct()

astyle 3.4.13 reformats the enable_if<!...> default template argument
(Check amalgamation). Delete the const overload instead, which makes the
libc++ probe for const objects fail in the SFINAE context just the same
and needs no <type_traits>.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasEsrzHhSNouxSoEwqz
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Contiguous parser: review fixes (wording, macros, diagnostic positions)

- "default parser" instead of "library parser" in comments and tests
- the three comments in json.hpp say the same thing
- current() replaces the repeated `p != end ? *p : 0`
- the backslashes of each macro are aligned
- the docs say that JSON_DIAGNOSTIC_POSITIONS turns the fast path off

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 07:08:44 +01:00

5.7 KiB

JSON_DIAGNOSTIC_POSITIONS

#define JSON_DIAGNOSTIC_POSITIONS /* value */

This macro enables position diagnostics for generated JSON objects.

When enabled, two new member functions start_pos() and end_pos() are added to basic_json values. If the value was created by calling theparse function, then these functions allow querying the byte positions of the value in the input it was parsed from. In case the value was constructed by other means, std::string::npos is returned.

start_pos() returns the position of the first character of a given value in the original JSON string, while end_pos() returns the position of the character following the last character. For objects and arrays, the first and last characters correspond to the opening or closing braces/brackets, respectively. For primitive values, the first and last character represents the opening and closing quotes (strings) or the first and last character of the field's numerical or predefined value (true, false, null), respectively.

JSON type return value start_pos() return value end_pos()
object position of the opening { position after the closing }
array position of the opening [ position after the closing ]
string position of the opening " position after the closing "
number position of the first character position after the last character
boolean position of t for true and f for false position after e
null position of n position after l

Given the above, end_pos()-start_pos() for a JSON value provides the length of the parsed JSON string for that value, including the opening or closing braces, brackets, or quotes.

Note that enabling this macro increases the size of every JSON value by two std::size_t fields and adds slight runtime overhead to parsing, copying JSON value objects, and the generation of error messages for exceptions. It also causes these values to be reported in those error messages. In addition, parse(), accept(), and sax_parse() no longer use the faster path for input in one block of memory, so parsing a string or a buffer is slower than without the macro.

If JSON_DIAGNOSTICS is also set, the message contains the path and the byte range, such as (/1) (bytes 4-8). An exception about the root value has no path and only the range, such as (bytes 0-4).

Default definition

The default value is 0 (position diagnostics are switched off).

#define JSON_DIAGNOSTIC_POSITIONS 0

When the macro is not defined, the library will define it to its default value.

Notes

!!! note "CMake option"

Diagnostic positions can also be controlled with the CMake option
[`JSON_Diagnostic_Positions`](../../integration/cmake.md#json_diagnostic_positions) (`OFF` by default)
which defines `JSON_DIAGNOSTIC_POSITIONS` accordingly.

!!! note "Availability"

Diagnostic positions are only available if the value was created by the [`parse`](../basic_json/parse.md) function.
The [`sax_parse`](../basic_json/sax_parse.md) function or all other means to create a JSON value **do not** set the
diagnostic positions and [`start_pos()`](../basic_json/start_pos.md) and [`end_pos()`](../basic_json/end_pos.md)
will only return `std::string::npos` for these values.

!!! warning "Invalidation"

The returned positions are only valid as long as the JSON value is not changed. The positions are *not* updated
when the JSON value is changed.

Examples

??? example "Example: retrieving positions"

```cpp
--8<-- "examples/diagnostic_positions.cpp"
```

Output:

```
--8<-- "examples/diagnostic_positions.output"
```

The output shows the start/end positions of all the objects and fields in the JSON string.

??? example "Example: using only diagnostic positions in exceptions"

```cpp
--8<-- "examples/diagnostic_positions_exception.cpp"
```

Output:

```
--8<-- "examples/diagnostic_positions_exception.output"
```

    The output shows the exception with start/end positions only.

??? example "Example: using extended diagnostics with positions enabled in exceptions"

```cpp
--8<-- "examples/diagnostics_extended_positions.cpp"
```

Output:

```
--8<-- "examples/diagnostics_extended_positions.output"
```

    The output shows the exception with diagnostic path info and start/end positions.

See also

Version history

  • Added in version 3.12.0.