Fix blank SMTP TLS failure-detail fields crashing saves; read RFC 8460 additional-information (11.0.3) (#916)

* Treat blank SMTP TLS failure-detail fields as absent; read RFC 8460 additional-information

Some reporters send optional failure-details fields as empty strings
(e.g. "sending-mta-ip": "" on sts-policy-fetch-error). The parser copied
them verbatim, so the PostgreSQL save failed on the INET columns
("invalid input syntax for type inet") and the Elasticsearch/OpenSearch
saves failed Ip() validation, discarding the whole report. Blank or
whitespace-only optional values are now omitted.

RFC 8460 section 4.4 names the URI key "additional-information"; the
parser only read "additional-info-uri" (the schema's value placeholder),
so conformant reports lost the URI. The RFC key is now read, with the old
key kept as a fallback.

Bump version to 11.0.3.

Closes #915

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Import parse_report_file directly in the ES/OpenSearch tests

Addresses github-code-quality's "Module is imported with 'import' and
'import from'" findings on tests/test_elastic.py and
tests/test_opensearch.py.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sean WhalenandClaude Opus 5.5 authored and GitHub committed 2026-09-24 10:42:34 -04:00
1 parent 6648be7d02
commit e077136b60
8 files changed
+307 -23

No files matched your search

+42 -1
View File
@@ -8,10 +8,11 @@ queries, error wrapping — without needing a running Elasticsearch cluster.
import time
import unittest
from typing import cast
from unittest.mock import MagicMock, call, patch
import parsedmarc.elastic as elastic_module
from parsedmarc import InvalidFailureReport
from parsedmarc import InvalidFailureReport, parse_report_file
from parsedmarc.elastic import (
AlreadySaved,
ElasticsearchError,
@@ -1486,6 +1487,46 @@ class TestSaveSmtpTlsReport(unittest.TestCase):
"https://reports.example.com/tls-help",
)
def test_save_blank_failure_detail_fields_omitted_from_ip_fields(self):
"""Blank optional failure-detail fields must not reach the Ip()
fields (issue #915).
samples/smtp_tls/empty_failure_detail_fields.json reproduces a
real reporter's shape: sending-mta-ip/receiving-ip sent as "" on
an sts-policy-fetch-error. Verified directly against
elasticsearch.dsl: Document.save() validates by calling
full_clean() before any network call, and an Ip() field's
validation calls Python's ipaddress.ip_address(""), which raises a
bare ValueError ("'' does not appear to be an IPv4 or IPv6
address") rather than a caught ValidationException -- so an empty
string here previously discarded the whole report instead of
indexing it with a blank IP.
save() is mocked (per this module's convention, so no real
cluster is needed), which also means it never runs full_clean()
itself; call it explicitly on the captured document so this test
exercises the same validation a real save() would, and fails with
that ValueError on unfixed parsedmarc/__init__.py.
"""
result = parse_report_file(
"samples/smtp_tls/empty_failure_detail_fields.json", offline=True
)
report = cast(dict, result["report"])
with (
patch("parsedmarc.elastic.Search", return_value=_empty_search()),
patch("parsedmarc.elastic.Index"),
patch.object(
elastic_module._SMTPTLSReportDoc, "save", autospec=True
) as mock_save,
):
save_smtp_tls_report_to_elasticsearch(report)
doc = mock_save.call_args[0][0]
doc.full_clean()
failure_detail = doc.policies[0].failure_details[0]
self.assertIsNone(failure_detail.sending_mta_ip)
self.assertIsNone(failure_detail.receiving_ip)
self.assertEqual(failure_detail.failure_reason_code, "no-policy-served")
class TestBackwardCompatAlias(unittest.TestCase):
def test_save_forensic_alias_points_to_save_failure(self):