Restore a duplicate key's prior value when the callback rejects its new value

json_sax_dom_callback_parser::key() unconditionally overwrote the object
slot for a key with a `discarded` placeholder as soon as the key was
accepted by the parser callback. For a duplicate key (legal JSON), this
destroyed the pre-existing value from an earlier occurrence of the same
key before the new value was even parsed. If the new value was then
rejected by the callback, remove_discarded_value() erased the member
entirely instead of leaving the original value in place, contradicting
the documented behavior that a discarded value behaves as if it was
never read.

Add a small stash of (slot pointer, previous value) pairs so that when
key() overwrites an existing member with the discarded placeholder, the
previous value can be restored later if the corresponding value (scalar,
object, or array) is rejected, instead of being erased. The stash entry
is dropped without restoring once the new value is definitively
accepted (in handle_value() for scalars, end_object()/end_array() for
containers), so a duplicate key whose new value is accepted still keeps
the last value as before. Non-duplicate keys are unaffected: rejecting
their value still removes the member entirely, since there is nothing
to restore.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

tests/src/unit-regression2.cpp

@@ -1566,4 +1566,75 @@ TEST_CASE("issue #5402 - update(merge_objects=true) overwrites a primitive with
     CHECK(mixed == json({{"keep", {{"a", 1}, {"b", 2}}}, {"replace", {{"x", 2}}}}));
 }
 
+TEST_CASE("regression test - parser callback must not lose a duplicate key's prior value")
+{
+    // a callback that rejects only the scalar value 2
+    const json::parser_callback_t drop_value_2 = [](int /*depth*/, json::parse_event_t ev, json & v)
+    {
+        return !(ev == json::parse_event_t::value && v == 2);
+    };
+
+    SECTION("duplicate key, second (scalar) value rejected - prior value is restored")
+    {
+        const json j = json::parse(R"({"a":1,"a":2})", drop_value_2);
+        CHECK(j.dump() == "{\"a\":1}");
+    }
+
+    SECTION("duplicate key, second value is an object rejected at object_end - prior value is restored")
+    {
+        const json j = json::parse(R"({"a":1,"a":{"x":2}})",
+                                   [](int depth, json::parse_event_t ev, json& /*parsed*/)
+        {
+            return !(ev == json::parse_event_t::object_end && depth == 1);
+        });
+        CHECK(j.dump() == "{\"a\":1}");
+    }
+
+    SECTION("duplicate key, second value is an array rejected at array_end - prior value is restored")
+    {
+        const json j = json::parse(R"({"a":1,"a":[9,9]})",
+                                   [](int depth, json::parse_event_t ev, json& /*parsed*/)
+        {
+            return !(ev == json::parse_event_t::array_end && depth == 1);
+        });
+        CHECK(j.dump() == "{\"a\":1}");
+    }
+
+    SECTION("duplicate key, second value accepted (scalar) - last value wins")
+    {
+        const json j = json::parse(R"({"a":1,"a":2})", [](int, json::parse_event_t, json&) noexcept
+        {
+            return true;
+        });
+        CHECK(j.dump() == "{\"a\":2}");
+    }
+
+    SECTION("duplicate key, second value accepted (object) - last value wins")
+    {
+        const json j = json::parse(R"({"a":1,"a":{"x":2}})", [](int, json::parse_event_t, json&) noexcept
+        {
+            return true;
+        });
+        CHECK(j.dump() == "{\"a\":{\"x\":2}}");
+    }
+
+    SECTION("brand new (non-duplicate) key, value rejected - member is fully absent")
+    {
+        const json j = json::parse(R"({"a":1,"b":2})", drop_value_2);
+        CHECK(j.dump() == "{\"a\":1}");
+    }
+
+    SECTION("duplicate key nested two levels deep")
+    {
+        const json j = json::parse(R"({"outer":{"a":1,"a":2}})", drop_value_2);
+        CHECK(j.dump() == "{\"outer\":{\"a\":1}}");
+    }
+
+    SECTION("three occurrences of the same key - middle rejected, last accepted")
+    {
+        const json j = json::parse(R"({"k":1,"k":2,"k":3})", drop_value_2);
+        CHECK(j.dump() == "{\"k\":3}");
+    }
+}
+
 DOCTEST_CLANG_SUPPRESS_WARNING_POP