Throw type_error.321 when serializing discarded values to binary formats (#5761)

* Throw type_error.321 when serializing discarded values to binary formats

The CBOR, MessagePack, UBJSON, BJData, and BSON writers silently
skipped the payload of a value_t::discarded value nested in an array
or object, while still writing its slot in the element/member count
(and, for BSON, its entry header), producing a binary document whose
declared size does not match what was actually written.

Throw type_error.321 instead, for a discarded value anywhere in the
tree, including at the top level.

Rewritten from the original PR against the current (non-recursive
option aside) binary_writer.hpp, which has changed substantially since
this was first proposed; the out_of_range.412 MessagePack size check
and unrelated test reformatting from that PR are dropped as out of
scope here.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document and test type_error.321 for discarded binary values

Add docs for the new exception (home/exceptions.md and the Exceptions
sections of to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson) and test
coverage for a discarded value nested in an array or object, nested
deeper, and (for UBJSON/BJData) inside an optimized same-type array,
for each of CBOR, MessagePack, UBJSON, BJData, and BSON. Adjust the
three pre-existing "discarded" tests that asserted the old silent
behavior (empty/short output) to expect type_error.321 instead.

Co-authored-by: ameliabarnabyhub <312084480+ameliabarnabyhub@users.noreply.github.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: ameliabarnabyhub <ameliabarnabyhub@users.noreply.github.com>
Co-authored-by: ameliabarnabyhub <312084480+ameliabarnabyhub@users.noreply.github.com>

docs/mkdocs/docs/api/basic_json/to_bjdata.md

@@ -68,6 +68,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 - Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
   not valid UTF-8 and `error_handler` is `strict` (the default only if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
+- Throws [type_error.321](../../home/exceptions.md#jsonexceptiontype_error321) if `j` or a value nested in it is
+  discarded; example: `"cannot serialize discarded value to BJData"`
 
 ## Complexity
 
@@ -119,4 +121,6 @@ Linear in the size of the JSON value `j`.
 - BJData version parameter (for draft3 binary encoding) added in version 3.12.0.
 - Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
   that is not valid UTF-8 unchanged, as before; `strict` (the default if
-  [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
+  [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
+- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
+  array or object was silently skipped, producing invalid BJData.

docs/mkdocs/docs/api/basic_json/to_bson.md

@@ -58,6 +58,9 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 - Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key is
   not valid UTF-8 and `error_handler` is `strict` (the default only if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
+- Throws [type_error.321](../../home/exceptions.md#jsonexceptiontype_error321) if a value nested in `j` is discarded
+  (the top-level value itself is covered by `type_error.317` above, since it must be an object); example:
+  `"cannot serialize discarded value to BSON"`
 
 ## Complexity
 
@@ -110,6 +113,8 @@ pass before anything is written.
 - Throws `out_of_range.412` and `out_of_range.415` since version 3.13.0.
 - Linear in the size of `j`, and no longer limited by the call stack for deeply nested values, since version 3.13.0.
 - `out_of_range.415` is now detected before anything is written, like the other exceptions above, since version 3.13.0.
+- Throws `type_error.321` for a discarded value nested in `j` since version 3.13.0; previously, it was silently
+  skipped, producing a document whose declared size did not match what was actually written.
 - Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
   that is not valid UTF-8 unchanged, as before; `strict` (the default if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316` before anything

docs/mkdocs/docs/api/basic_json/to_cbor.md

@@ -49,6 +49,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 - Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
   not valid UTF-8 and `error_handler` is `strict` (the default only if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
+- Throws [type_error.321](../../home/exceptions.md#jsonexceptiontype_error321) if `j` or a value nested in it is
+  discarded; example: `"cannot serialize discarded value to CBOR"`
 
 ## Complexity
 
@@ -86,3 +88,5 @@ Linear in the size of the JSON value `j`.
 - Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
   that is not valid UTF-8 unchanged, as before; `strict` (the default if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
+- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
+  array or object was silently skipped, producing invalid CBOR.

docs/mkdocs/docs/api/basic_json/to_msgpack.md

@@ -54,6 +54,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
   `"subtype 70000 is too large for the MessagePack ext type (max 255)"`
 - Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
   not valid UTF-8 and `error_handler` is `strict`
+- Throws [type_error.321](../../home/exceptions.md#jsonexceptiontype_error321) if `j` or a value nested in it is
+  discarded; example: `"cannot serialize discarded value to MessagePack"`
 
 ## Complexity
 
@@ -108,3 +110,5 @@ Linear in the size of the JSON value `j`.
 - Fixed in version 3.13.0 to serialize `number_integer_t`/`number_unsigned_t` pairs of different width correctly;
   before, integers could be serialized with the wrong value if `number_integer_t` was narrower than
   `number_unsigned_t`.
+- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
+  array or object was silently skipped, producing invalid MessagePack.

docs/mkdocs/docs/api/basic_json/to_ubjson.md

@@ -61,6 +61,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 - Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
   not valid UTF-8 and `error_handler` is `strict` (the default only if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
+- Throws [type_error.321](../../home/exceptions.md#jsonexceptiontype_error321) if `j` or a value nested in it is
+  discarded; example: `"cannot serialize discarded value to UBJSON"`
 
 ## Complexity
 
@@ -112,3 +114,5 @@ Linear in the size of the JSON value `j`.
 - Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
   that is not valid UTF-8 unchanged, as before; `strict` (the default if
   [`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
+- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
+  array or object was silently skipped, producing invalid UBJSON.