Merge wip/5948 into ci/tiers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/patch_inplace.md

@@ -69,6 +69,19 @@ Duplicate member names inside an operation object are resolved by the parser bef
 is kept, see [`object_t`](object_t.md#behavior)), so `#!json {"op": "add", "op": "remove", "path": "/a"}` is a
 `remove` operation.
 
+The JSON patch may be the JSON value itself (`#!cpp j.patch_inplace(j)`): the operations are then read from a copy of
+the patch made before the first operation is applied, so the result is the same as that of
+`#!cpp j = j.patch(j)`, and the exception safety is the same as above.
+
+If [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md) is enabled, the same holds for a JSON patch stored inside the
+JSON value (e.g., `#!cpp j.patch_inplace(j["patch"])`): it is detected via the parent pointers.
+
+!!! danger "Undefined behavior"
+
+    Without [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md), the JSON patch must not be a value stored inside the
+    JSON value (e.g., `#!cpp j.patch_inplace(j["patch"])`), as the operations may change or destroy it while it is
+    read. Pass a copy (`#!cpp j.patch_inplace(json(j["patch"]))`) or use [`patch`](patch.md) instead.
+
 ## Examples
 
 ??? example "Example: apply a JSON patch in place"
@@ -122,3 +135,5 @@ is kept, see [`object_t`](object_t.md#behavior)), so `#!json {"op": "add", "op":
 - Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
   [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is
   not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.
+- Applying a JSON value as patch to itself (or, with `JSON_DIAGNOSTICS`, to a value stored inside it) no longer results
+  in undefined behavior in version 3.13.0.

docs/mkdocs/docs/api/basic_json_view/materialize.md

@@ -32,9 +32,24 @@ uses, so the result matches it exactly — including, for an object, that a repe
 last value. The replay is iterative, so it is not limited by the call stack the way a naive recursive conversion
 would be; the JSON nesting depth is limited only by available memory, as for `BasicJsonType::parse()` itself.
 
-Unlike parsing with [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values
-produced by `materialize()` do not carry source positions. The index has the start offset of every value, but not the
-end of arrays, objects, and escaped strings, which the positions also need.
+With [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values get the source
+positions [`BasicJsonType::parse()`](../basic_json/parse.md) would give them, also for the subtree of a view (the
+positions are those in the document's whole source text), and exceptions about them name the same bytes. For values of
+an [editable document](../json_editable_document.md), positions are all-or-nothing, and [`start_pos()`](../basic_json/start_pos.md)
+and [`end_pos()`](../basic_json/end_pos.md) return `std::string::npos` for
+
+- a value written by an edit (a copy of an array or object of the document is also new; copies of its scalars keep
+  their positions),
+- an array or object whose elements were changed (its elements keep theirs), and
+- an array or object whose last element has no positions (its end is found from that element).
+
+This differs from a `basic_json` value, whose positions are not updated when it is changed: they stay as they were
+parsed, even if they no longer match the value (see [`start_pos()`](../basic_json/start_pos.md)).
+
+Documents [loaded](../basic_json_document/load.md) from an image have the positions of the saved document. Its strings
+with escapes have positions only if the document was parsed with `JSON_DIAGNOSTIC_POSITIONS`: the position of their
+opening quote is recorded while parsing, and `save()` and `load()` keep it whether or not they are compiled with the
+macro.
 
 Calling `materialize()` on the same view repeatedly builds a new, independent `BasicJsonType` value each time; it
 never caches the result.
@@ -62,8 +77,7 @@ never caches the result.
 
 - [`root`](../basic_json_document/root.md) - the view of a document's root value
 - [`BasicJsonType::parse`](../basic_json/parse.md) - build a `BasicJsonType` value directly from a JSON text
-- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values (not
-  produced by `materialize()`)
+- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values
 - [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - when to build a basic_json value from a view
 
 ## Version history

docs/mkdocs/docs/api/json_pointer/operator_slasheq.md

@@ -18,7 +18,8 @@ json_pointer& operator/=(std::size_t array_idx)
 ## Parameters
 
 `ptr` (in)
-:    JSON pointer to append
+:    JSON pointer to append; may be the JSON pointer itself (`#!cpp p /= p`), which then appends a copy of its
+     reference tokens
 
 `token` (in)
 :    reference token to append
@@ -69,6 +70,7 @@ json_pointer& operator/=(std::size_t array_idx)
 
 ## Version history
 
-1. Added in version 3.6.0.
+1. Added in version 3.6.0. Appending a JSON pointer to itself no longer results in undefined behavior in version
+   3.13.0.
 2. Added in version 3.6.0. Changed type of `token` to `string_t` in version 3.11.0.
 3. Added in version 3.6.0.

docs/mkdocs/docs/api/macros/json_diagnostic_positions.md

@@ -113,6 +113,7 @@ When the macro is not defined, the library will define it to its default value.
 
 - [:simple-cmake: JSON_Diagnostic_Positions](../../integration/cmake.md#json_diagnostic_positions) - CMake option to control the macro
 - [`JSON_DIAGNOSTICS`](json_diagnostics.md) - macro to control extended diagnostics
+- [`materialize()`](../basic_json_view/materialize.md) - values of a JSON view, with the same positions
 - [Parsing and Exceptions](../../features/parsing/parse_exceptions.md) - handling parse errors and locating them in the input
 
 ## Version history

docs/mkdocs/docs/features/json_view.md

@@ -149,10 +149,12 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
   end must still fit.
 - **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
   is read to its end into a buffer the document owns, and parsing starts after that.
-- **No source positions on `materialize()`.** Even with
+- **Fewer source positions after edits.** With
   [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
-  [`materialize()`](../api/basic_json_view/materialize.md) does not set them. The index has the start offset of every
-  value, but not the end of arrays, objects, and escaped strings, which the positions also need.
+  [`materialize()`](../api/basic_json_view/materialize.md) sets the same positions as `#!cpp basic_json::parse()`.
+  In an [editable document](../api/json_editable_document.md), a value written by an edit has none, and neither has
+  an array or object whose elements changed or whose last element has none; a `basic_json` value keeps its positions
+  when it is changed. See [`materialize()`](../api/basic_json_view/materialize.md#notes).
 - **Objects iterate in document order.** [`begin()`](../api/basic_json_view/begin.md)/
   [`end()`](../api/basic_json_view/end.md) and [`items()`](../api/basic_json_view/items.md) visit an object's
   members in the order they appear in the source text. `basic_json`'s default `object_t` is a `std::map`, which

docs/mkdocs/docs/home/architecture.md

@@ -203,7 +203,7 @@ packet-beta
 | 2-3   | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their key hash index (1-based), or 0; otherwise 0 |
 | 4-7   | `off`   | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
 | 8-11  | `len`   | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
-| 12-15 | `next`  | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself                                              |
+| 12-15 | `next`  | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself; strings with escapes: the position of the opening quote in the source plus one if the document was parsed with [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md), or 0 if it was not; all other values: 0 |
 
 <p class="nj-table-caption" markdown>Fields of a node, by their byte offset within the 16-byte node. Integers use bytes 8-15 differently, see below.</p>
 
@@ -218,6 +218,12 @@ packet-beta
   step from element to element this way and skip whole subtrees in constant time.
 - **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
   2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
+- **Source positions** ([`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md)) are derived by
+  [`materialize()`](../api/basic_json_view/materialize.md) and cost almost nothing while parsing: the only work is
+  storing the offset of the opening quote of a string with escapes in its `next`, which is otherwise unused. A
+  scalar's range follows from `off` and its length, a string with escapes is scanned from its opening quote (`next`) to the closing one,
+  and the end of an array or object is found by scanning from the end of its last element over whitespace, comments,
+  and a trailing comma. `materialize()` hands these positions to the SAX builder of `parse()` in place of its lexer.
 - **Large objects** (128 members or more) get a hash index for their keys after parsing
   ([`detail/view/object_index.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/object_index.hpp)):
   an open-addressing table whose slots hold the distance from the object's node to a key's node, so that a lookup does
@@ -277,7 +283,7 @@ Images ([`save`](../api/basic_json_document/save.md) and [`load`](../api/basic_j
 the nodes as they are: a 64-byte header (the magic bytes `NJVI`, a format version, the sizes, and reserved bytes that
 must be zero), the nodes, the text, and the decoded strings. An edited document is first written in document order, as
 the parser would have written it (without links), and the numbers of the hash indexes are cleared, since `load`
-rebuilds the indexes. So a change of the node layout is a change of the image format: it must raise `image_version`,
+rebuilds the indexes. The flag *new* is kept, as these values have no position in the text. So a change of the node layout is a change of the image format: it must raise `image_version`,
 and `load` then rejects images of other versions (`parse_error.116`) instead of misreading them.
 
 ## Input adapters