mirror of
https://github.com/nlohmann/json.git
synced 2026-11-17 09:01:28 +00:00
Cut long tokens in parse error messages at UTF-8 character boundaries
A parse error message shows a token of more than 8192 bytes by its first and its last 4096 bytes around "...". The cuts were at fixed byte offsets, so they could split a UTF-8 character: for valid UTF-8 input, what() and the last token passed to SAX parse_error() were not valid UTF-8, and storing the message in a json value and dumping it threw type_error.316. Move the end of the first part back and the start of the last part forward to a character boundary, by at most three bytes each. The lexer always reads bytes (wide strings are converted to UTF-8 first), and the decision depends only on bytes inside the two parts, so the message stays the same for every kind of input. Both parts only get shorter, so the buffers that keep the token need no more room. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
4 files changed
+216
-5
No files matched your search
@@ -140,7 +140,7 @@ Exceptions have ids 1xx.
|
||||
|
||||
This error indicates a syntax error while deserializing a JSON text. The error message describes that an unexpected token (character) was encountered, and the member `byte` indicates the error position.
|
||||
|
||||
The message ends with the last read token (`last read: '...'`), with control characters written as `<U+XXXX>`. A token of more than 8192 characters is shown by its first and its last 4096 characters, separated by `...`.
|
||||
The message ends with the last read token (`last read: '...'`), with control characters written as `<U+XXXX>`. A token of more than 8192 bytes is shown by its first and its last 4096 bytes, separated by `...`; each part is shortened by up to three bytes so that it does not end or begin within a UTF-8 character.
|
||||
|
||||
!!! failure "Example message"
|
||||
|
||||
|
||||
Reference in new issue
Block a user