Run the CBOR, MessagePack, BSON and BON8 round-trip invariants in CI

tests/src/round_trip_corpus.hpp exists so that the byte-stability
invariant the fuzzer drivers check also runs on a fixed corpus in CI,
instead of only at OSS-Fuzz. So far only the UBJSON and BJData drivers
had a matching unit test; the CBOR, MessagePack, BSON and BON8 drivers
assert the same invariant (assert(to_X(j2) == vec)) but nothing ran it
outside OSS-Fuzz.

Add "<FORMAT> round-trip invariants" test cases to unit-cbor.cpp,
unit-msgpack.cpp, unit-bson.cpp and unit-bon8.cpp, modeled on the
UBJSON case: seed j1 from the corpus (skipping values that do not
survive the format's own round trip, as the fuzzer drivers only ever
see values from_X() actually produced), then require from_X(to_X(j1))
not to throw and check to_X(j2) == to_X(j1). BSON only serializes
objects, so non-object corpus values are skipped. Update the comments
in round_trip_corpus.hpp and tests/fuzzing.md to name all six formats.

The stream-versus-contiguous check in the BON8 driver is left out, as
#5601 reworks it.

A local probe confirms no violations on the current corpus (CBOR 3849
checked, MessagePack 3909, BSON 2958, BON8 3841 - matching the counts
already recorded for this probe in the issue).

Closes #5714 item 1.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

tests/fuzzing.md

@@ -93,9 +93,10 @@ conventions:
   add it as a regression test to the unit test of the affected format (e.g., `tests/src/unit-bjdata.cpp`), with a
   comment naming the OSS-Fuzz issue. This way the input is checked by every CI run rather than only by OSS-Fuzz, and
   it stays covered even if OSS-Fuzz later closes the report as not reproducible.
-- **Keep the fuzzer drivers and the unit tests in sync.** The round-trip checks of the UBJSON and BJData drivers are
-  also run on a fixed corpus in the unit tests (see `tests/src/round_trip_corpus.hpp` and the "round-trip invariants"
-  test cases), so a regression shows up in CI first. When a driver's checks change, change the unit tests with them.
+- **Keep the fuzzer drivers and the unit tests in sync.** The round-trip checks of the BJData, BON8, BSON, CBOR,
+  MessagePack and UBJSON drivers are also run on a fixed corpus in the unit tests (see
+  `tests/src/round_trip_corpus.hpp` and the "round-trip invariants" test cases), so a regression shows up in CI
+  first. When a driver's checks change, change the unit tests with them.
 - **Record in the report whether the bug shipped.** OSS-Fuzz asks whether a crash was a short-lived regression or
   affects a released version; answer it when the fix is merged, as it decides whether the fix needs a release note or
   a security advisory (see the [security policy](../.github/SECURITY.md)).

tests/src/round_trip_corpus.hpp

@@ -19,13 +19,14 @@
 
 #include <nlohmann/json.hpp>
 
-// Values for the round-trip property tests of the UBJSON and BJData writers.
+// Values for the round-trip property tests of the binary format writers
+// (BJData, BON8, BSON, CBOR, MessagePack and UBJSON).
 //
-// The fuzzer drivers (tests/src/fuzzer-parse_ubjson.cpp and
-// fuzzer-parse_bjdata.cpp) check that anything the library parses can be
-// serialized, parsed back, and serialized again without loss. Those checks
-// only run at OSS-Fuzz, so a regression used to surface days later as an
-// external report. The unit tests run the same checks on this corpus in CI.
+// The fuzzer drivers (tests/src/fuzzer-parse_*.cpp) check that anything the
+// library parses can be serialized, parsed back, and serialized again
+// without loss. Those checks only run at OSS-Fuzz, so a regression used to
+// surface days later as an external report. The unit tests run the same
+// checks on this corpus in CI.
 //
 // The corpus is deterministic: std::mt19937's output sequence is fixed by
 // the standard, and it is used directly rather than through a distribution

tests/src/unit-bon8.cpp

@@ -21,6 +21,7 @@ using nlohmann::json;
 #include <string>
 #include <vector>
 #include "make_test_data_available.hpp"
+#include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 #include "sax_countdown.hpp"
 using utils::SaxCountdown;
@@ -744,6 +745,39 @@ TEST_CASE("Parse BON8 directly from a file using iterator and sentinel")
     CHECK((parsed.is_object() || parsed.is_array()));
 }
 
+TEST_CASE("BON8 round-trip invariants")
+{
+    // This checks what the parse_bon8_fuzzer driver checks (see
+    // tests/src/fuzzer-parse_bon8.cpp), so that a regression shows up in CI
+    // rather than as an OSS-Fuzz report: anything from_bon8() returns (j1)
+    // can be serialized, parsed back (j2), and serialized again to reproduce
+    // the exact bytes. The stream-versus-contiguous input check the driver
+    // also performs is not covered here (see #5601).
+    for (const auto& j0 : utils::round_trip_corpus::values())
+    {
+        json j1;
+        try
+        {
+            // turn the corpus value into a value as from_bon8() returns it
+            j1 = json::from_bon8(json::to_bon8(j0));
+        }
+        catch (const json::exception&)
+        {
+            // BON8 cannot represent an unsigned integer above INT64_MAX, and
+            // the fuzzer driver only ever sees values from_bon8() actually
+            // produced, so skip such corpus values here, too
+            continue;
+        }
+
+        INFO("j1 = " << j1.dump());
+        const std::vector<std::uint8_t> vec = json::to_bon8(j1);
+        json j2;
+        // anything the library writes must be parsable by the library
+        REQUIRE_NOTHROW(j2 = json::from_bon8(vec));
+        CHECK(json::to_bon8(j2) == vec);
+    }
+}
+
 TEST_CASE("BON8 roundtrips" * doctest::skip())
 {
     SECTION("input from HikoGUI")

tests/src/unit-bson.cpp

@@ -17,6 +17,7 @@ using nlohmann::json;
 #include <sstream>
 #include <vector>
 #include "make_test_data_available.hpp"
+#include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 #include "sax_countdown.hpp"
 using utils::SaxCountdown;
@@ -1613,6 +1614,44 @@ TEST_CASE("Parse BSON directly from a file using iterator and sentinel")
     CHECK(parsed == expected);
 }
 
+TEST_CASE("BSON round-trip invariants")
+{
+    // This checks what the parse_bson_fuzzer driver checks (see
+    // tests/src/fuzzer-parse_bson.cpp), so that a regression shows up in CI
+    // rather than as an OSS-Fuzz report: anything from_bson() returns (j1)
+    // can be serialized, parsed back (j2), and serialized again to reproduce
+    // the exact bytes. BSON only serializes objects, so non-object corpus
+    // values are skipped.
+    for (const auto& j0 : utils::round_trip_corpus::values())
+    {
+        if (!j0.is_object())
+        {
+            continue;
+        }
+
+        json j1;
+        try
+        {
+            // turn the corpus value into a value as from_bson() returns it
+            j1 = json::from_bson(json::to_bson(j0));
+        }
+        catch (const json::exception&)
+        {
+            // the fuzzer driver only ever sees values from_bson() actually
+            // produced, so skip corpus values that do not survive the
+            // round trip here, too
+            continue;
+        }
+
+        INFO("j1 = " << j1.dump());
+        const std::vector<std::uint8_t> vec = json::to_bson(j1);
+        json j2;
+        // anything the library writes must be parsable by the library
+        REQUIRE_NOTHROW(j2 = json::from_bson(vec));
+        CHECK(json::to_bson(j2) == vec);
+    }
+}
+
 TEST_CASE("BSON roundtrips" * doctest::skip())
 {
     SECTION("reference files")

tests/src/unit-cbor.cpp

@@ -18,6 +18,7 @@ using nlohmann::json;
 #include <list>
 #include <set>
 #include "make_test_data_available.hpp"
+#include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 #include "sax_countdown.hpp"
 using utils::SaxCountdown;
@@ -2340,6 +2341,39 @@ TEST_CASE("issue #5405 - array reserve for definite-length CBOR arrays")
     }
 }
 
+TEST_CASE("CBOR round-trip invariants")
+{
+    // This checks what the parse_cbor_fuzzer driver checks (see
+    // tests/src/fuzzer-parse_cbor.cpp), so that a regression shows up in CI
+    // rather than as an OSS-Fuzz report: anything from_cbor() returns (j1)
+    // can be serialized, parsed back (j2), and serialized again to reproduce
+    // the exact bytes.
+    for (const auto& j0 : utils::round_trip_corpus::values())
+    {
+        json j1;
+        try
+        {
+            // turn the corpus value into a value as from_cbor() returns it
+            j1 = json::from_cbor(json::to_cbor(j0));
+        }
+        catch (const json::exception&)
+        {
+            // not every corpus value survives a CBOR round trip (e.g., a
+            // binary subtype is written with a tag the default tag handler
+            // then rejects); the fuzzer driver only ever sees values
+            // from_cbor() actually produced, so skip those here, too
+            continue;
+        }
+
+        INFO("j1 = " << j1.dump());
+        const std::vector<std::uint8_t> vec = json::to_cbor(j1);
+        json j2;
+        // anything the library writes must be parsable by the library
+        REQUIRE_NOTHROW(j2 = json::from_cbor(vec));
+        CHECK(json::to_cbor(j2) == vec);
+    }
+}
+
 TEST_CASE("CBOR roundtrips" * doctest::skip())
 {
     SECTION("input from flynn")

tests/src/unit-msgpack.cpp

@@ -21,6 +21,7 @@ using nlohmann::json;
 #include <limits>
 #include <set>
 #include "make_test_data_available.hpp"
+#include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
 #include "sax_countdown.hpp"
 using utils::SaxCountdown;
@@ -1855,6 +1856,38 @@ TEST_CASE("Parse MessagePack directly from a file using iterator and sentinel")
     CHECK((parsed.is_object() || parsed.is_array()));
 }
 
+TEST_CASE("MessagePack round-trip invariants")
+{
+    // This checks what the parse_msgpack_fuzzer driver checks (see
+    // tests/src/fuzzer-parse_msgpack.cpp), so that a regression shows up in
+    // CI rather than as an OSS-Fuzz report: anything from_msgpack() returns
+    // (j1) can be serialized, parsed back (j2), and serialized again to
+    // reproduce the exact bytes.
+    for (const auto& j0 : utils::round_trip_corpus::values())
+    {
+        json j1;
+        try
+        {
+            // turn the corpus value into a value as from_msgpack() returns it
+            j1 = json::from_msgpack(json::to_msgpack(j0));
+        }
+        catch (const json::exception&)
+        {
+            // the fuzzer driver only ever sees values from_msgpack() actually
+            // produced, so skip corpus values that do not survive the
+            // round trip here, too
+            continue;
+        }
+
+        INFO("j1 = " << j1.dump());
+        const std::vector<std::uint8_t> vec = json::to_msgpack(j1);
+        json j2;
+        // anything the library writes must be parsable by the library
+        REQUIRE_NOTHROW(j2 = json::from_msgpack(vec));
+        CHECK(json::to_msgpack(j2) == vec);
+    }
+}
+
 TEST_CASE("MessagePack roundtrips" * doctest::skip())
 {
     SECTION("input from msgpack-python")