Fix DKIM/SPF alignment detail cross-product in dashboards (#169)

Elasticsearch and OpenSearch dynamic-map the dkim_results/spf_results
object arrays as `object` (create_indexes never registers the DSL
document mappings), so Lucene flattens each array into independent
multi-valued fields and stacked terms aggregations on
dkim_results.selector/.domain/.result return every combination of
values across a report's signatures — each phantom row repeating the
full message count.

Aggregate documents now also carry dkim_results_combined and
spf_results_combined: one "selector / domain / result"
("scope / domain / result") string per auth result, composed in
add_dkim_result/add_spf_result. The Kibana/OpenSearch Dashboards and
Grafana (Elasticsearch) alignment-detail tables aggregate those
instead, and the Splunk detail panels pair the values with
mvzip/mvexpand. A documented idempotent _update_by_query backfills
documents saved by older versions; the query matches only documents
that have auth results and lack the combined fields, because an
`exists` query cannot see an empty array.

Also corrects the dead _SPFResult.results (plural) declaration to
`result` (the save path always wrote the singular key), fixes the
result parameter annotations on add_dkim_result/add_spf_result, and
removes the Grafana dmarcian.com DKIM-checker data link, which
required the separate domain/selector columns.

The SMTP TLS visualizations have the same class of defect and are
tracked separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sean Whalen
2026-07-21 21:52:19 -04:00
co-authored by Claude Fable 5
parent 4fa8cfb1e7
commit 6148002a43
10 changed files with 335 additions and 64 deletions
+2
View File
@@ -12,6 +12,8 @@
- **The Elasticsearch/OpenSearch aggregate dashboards' over-time charts (and the Grafana ES dashboard's summary pies and time series) bucketed on the multi-valued `date_range` field**; a date histogram counts a report once per value, double-counting any report whose begin and end dates fall in different buckets. All date histograms and time-range filters now use the single-valued `date_begin`, matching the report-begin semantics of the PostgreSQL (`begin_date`) and Splunk (`_time` = interval begin) dashboards.
- **Aggregate-report policy and authentication result words are now normalized to lowercase** ([#288](https://github.com/domainaware/parsedmarc/issues/288)): reporters that emit mixed-case values such as `Pass` no longer create duplicate result categories in outputs and dashboards.
- **The results email (SMTP and Microsoft Graph) is no longer sent when no reports were parsed** ([#200](https://github.com/domainaware/parsedmarc/issues/200)): previously an empty run — an empty inbox, or one where every message was invalid — still emailed a zip of headers-only CSVs. The email step is now skipped with an INFO log when the run produced no aggregate, failure, or SMTP TLS reports.
- **The DKIM/SPF alignment-detail tables on the Kibana/OpenSearch Dashboards, Grafana (Elasticsearch), and Splunk aggregate dashboards no longer show a selector × domain × result cross-product** ([#169](https://github.com/domainaware/parsedmarc/issues/169)). Elasticsearch/OpenSearch flatten the `dkim_results`/`spf_results` object arrays (they are dynamic-mapped as `object`, not `nested`), so stacking terms aggregations on their subfields produced every combination of selector/domain/result across a report's signatures, each repeating the full message count. Aggregate documents now also carry `dkim_results_combined` and `spf_results_combined` — one `"selector / domain / result"` (`"scope / domain / result"`) string per auth result — and the dashboards aggregate those instead; the Splunk detail panels now pair the values with `mvzip`/`mvexpand`. Documents saved by older versions can be backfilled with a documented `_update_by_query` command (see the Elasticsearch docs page). The Grafana "DKIM Alignment Details" panel's dmarcian.com DKIM-checker data link was removed because it required the separate domain/selector columns. The SMTP TLS visualizations have the same class of defect and are tracked separately.
- **Corrected the dead `_SPFResult.results` (plural) field declaration to `result`**, matching what was always written to it.
## 10.2.4
+12 -52
View File
@@ -2598,7 +2598,7 @@
{
"$$hashKey": "object:412",
"fake": true,
"field": "spf_results.result.keyword",
"field": "spf_results_combined.keyword",
"id": "16",
"settings": {
"min_doc_count": "1",
@@ -2626,7 +2626,7 @@
{
"$$hashKey": "object:461",
"fake": true,
"field": "dkim_results.result.keyword",
"field": "dkim_results_combined.keyword",
"id": "10",
"settings": {
"min_doc_count": "1",
@@ -2666,7 +2666,7 @@
"Sum": 11,
"disposition.keyword": 4,
"dkim_aligned": 6,
"dkim_results.result.keyword": 9,
"dkim_results_combined.keyword": 9,
"header_from.keyword": 10,
"org_name.keyword": 7,
"source_base_domain.keyword": 0,
@@ -2674,7 +2674,7 @@
"source_ip_address.keyword": 2,
"source_reverse_dns.keyword": 1,
"spf_aligned": 5,
"spf_results.result.keyword": 8
"spf_results_combined.keyword": 8
},
"renameByName": {
"Sum": "Messages",
@@ -2682,7 +2682,7 @@
"disposition.keyword": "Disposition",
"dkim_aligned": "DKIM",
"dkim_results.domain.keyword": "DKIM Domain",
"dkim_results.result.keyword": "DKIM Auth Result",
"dkim_results_combined.keyword": "DKIM Auth Result",
"dkim_results.selector.keyword": "DKIM Selector",
"envelope_from.keyword": "Envelope From",
"header_from.keyword": "Email Domain",
@@ -2692,7 +2692,7 @@
"source_ip_address.keyword": "Source IP",
"source_reverse_dns.keyword": "PTR",
"spf_aligned": "SPF",
"spf_results.result.keyword": "SPF Auth Result"
"spf_results_combined.keyword": "SPF Auth Result"
}
}
}
@@ -3431,7 +3431,7 @@
{
"$$hashKey": "object:459",
"fake": true,
"field": "spf_results.result.keyword",
"field": "spf_results_combined.keyword",
"id": "8",
"settings": {
"min_doc_count": 1,
@@ -3502,7 +3502,7 @@
"header_from.keyword": "Header From",
"source_base_domain.keyword": "Reverse DNS Base",
"spf_aligned": "SPF Aligned",
"spf_results.result.keyword": "SPF Result"
"spf_results_combined.keyword": "SPF Scope / Domain / Result"
}
}
}
@@ -3590,22 +3590,12 @@
{
"matcher": {
"id": "byName",
"options": "DKIM Selector"
"options": "DKIM Selector / Domain / Result"
},
"properties": [
{
"id": "custom.width",
"value": 320
},
{
"id": "links",
"value": [
{
"targetBlank": true,
"title": "Open dmarcian.com DKIM Record Checker",
"url": "https://dmarcian.com/dkim-inspector/?domain=${__data.fields[\"dkim_results.domain.keyword\"]}&selector=${__data.fields[\"dkim_results.selector.keyword\"]}"
}
]
"value": 400
}
]
}
@@ -3649,7 +3639,7 @@
{
"$$hashKey": "object:458",
"fake": true,
"field": "dkim_results.selector.keyword",
"field": "dkim_results_combined.keyword",
"id": "7",
"settings": {
"min_doc_count": "1",
@@ -3660,34 +3650,6 @@
},
"type": "terms"
},
{
"$$hashKey": "object:459",
"fake": true,
"field": "dkim_results.domain.keyword",
"id": "8",
"settings": {
"min_doc_count": 1,
"missing": "-",
"order": "desc",
"orderBy": "4",
"size": "0"
},
"type": "terms"
},
{
"$$hashKey": "object:460",
"fake": true,
"field": "dkim_results.result.keyword",
"id": "9",
"settings": {
"min_doc_count": 1,
"missing": null,
"order": "desc",
"orderBy": "4",
"size": "0"
},
"type": "terms"
},
{
"$$hashKey": "object:798",
"fake": true,
@@ -3744,9 +3706,7 @@
"renameByName": {
"Sum": "Messages",
"dkim_aligned": "DKIM Aligned",
"dkim_results.domain.keyword": "DKIM Domain",
"dkim_results.result.keyword": "DKIM Result",
"dkim_results.selector.keyword": "DKIM Selector",
"dkim_results_combined.keyword": "DKIM Selector / Domain / Result",
"envelope_from.keyword": "Envelope From",
"header_from.keyword": "Header From",
"source_base_domain.keyword": "Reverse DNS Base",
File diff suppressed because one or more lines are too long
@@ -323,7 +323,12 @@
<title>SPF details</title>
<table>
<search base="base_search">
<query>| fillnull value="none" source_base_domain | stats sum(message_count) as message_count by header_from,envelope_from,spf_result,source_base_domain,spf_aligned
<query>| fillnull value="none" source_base_domain envelope_domain spf_scope spf_result
| eval spf_signature=mvzip(mvzip(spf_scope, envelope_domain, " / "), spf_result, " / ")
| mvexpand spf_signature
| stats sum(message_count) as message_count by header_from, envelope_from, spf_signature, source_base_domain, spf_aligned
| eval parts=split(spf_signature, " / "), spf_scope=mvindex(parts, 0), spf_domain=mvindex(parts, 1), spf_result=mvindex(parts, 2)
| table header_from, envelope_from, spf_scope, spf_domain, spf_result, spf_aligned, source_base_domain, message_count
| sort -message_count</query>
</search>
<option name="drilldown">none</option>
@@ -339,7 +344,12 @@
<title>DKIM details</title>
<table>
<search base="base_search">
<query>| fillnull value="none" source_base_domain | stats sum(message_count) as message_count by header_from,dkim_selector,dkim_domain,dkim_result,dkim_aligned,source_base_domain
<query>| fillnull value="none" source_base_domain
| eval dkim_signature=mvzip(mvzip(dkim_selector, dkim_domain, " / "), dkim_result, " / ")
| mvexpand dkim_signature
| stats sum(message_count) as message_count by header_from, dkim_signature, dkim_aligned, source_base_domain
| eval parts=split(dkim_signature, " / "), dkim_selector=mvindex(parts, 0), dkim_domain=mvindex(parts, 1), dkim_result=mvindex(parts, 2)
| table header_from, dkim_selector, dkim_domain, dkim_result, dkim_aligned, source_base_domain, message_count
| sort -message_count</query>
</search>
<option name="drilldown">none</option>
+54
View File
@@ -226,6 +226,60 @@ Kibana index patterns with versions that match the upgraded indexes:
7. Import `export.ndjson` by clicking Import from the Kibana
Saved Objects page
## Backfilling the combined DKIM/SPF result fields
As of the version fixing [#169](https://github.com/domainaware/parsedmarc/issues/169),
aggregate documents include `dkim_results_combined` and `spf_results_combined` —
scalar string arrays that keep each auth result's selector/scope, domain, and
result paired, which the dashboards' alignment-detail tables aggregate on.
Reports saved by older versions lack these fields and will not appear in
those tables.
Running the following once per cluster backfills the fields on existing
documents. It is idempotent (documents that already have the fields are
skipped), so it is safe to re-run. It works identically on OpenSearch;
just adjust the URL and credentials. The query matches only documents
that have at least one DKIM or SPF auth result and lack the corresponding
combined field; documents with no auth results are skipped, because an
`exists` query cannot see an empty array, and for search purposes an
empty `dkim_results_combined` is identical to an absent one.
```bash
curl -X POST "http://localhost:9200/dmarc_aggregate*/_update_by_query?conflicts=proceed&wait_for_completion=false" \
-H "Content-Type: application/json" -d '
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [{"exists": {"field": "dkim_results.domain"}}],
"must_not": [{"exists": {"field": "dkim_results_combined"}}]
}
},
{
"bool": {
"must": [{"exists": {"field": "spf_results.domain"}}],
"must_not": [{"exists": {"field": "spf_results_combined"}}]
}
}
]
}
},
"script": {
"lang": "painless",
"source": "List dk = new ArrayList(); def dr = ctx._source.dkim_results; if (dr != null) { if (!(dr instanceof List)) { dr = [dr]; } for (e in dr) { if (e == null) { continue; } def sel = e.selector != null ? e.selector : \"none\"; def dom = e.domain != null ? e.domain : \"none\"; def res = e.result != null ? e.result : \"none\"; dk.add(sel + \" / \" + dom + \" / \" + res); } } ctx._source.dkim_results_combined = dk; List sp = new ArrayList(); def sr = ctx._source.spf_results; if (sr != null) { if (!(sr instanceof List)) { sr = [sr]; } for (e in sr) { if (e == null) { continue; } def sc = e.scope != null ? e.scope : \"mfrom\"; def dom = e.domain != null ? e.domain : \"none\"; def res = e.result != null ? e.result : (e.results != null ? e.results : \"none\"); sp.add(sc + \" / \" + dom + \" / \" + res); } } ctx._source.spf_results_combined = sp;"
}
}'
```
`wait_for_completion=false` returns a task ID — check progress with
`GET _tasks/<task id>`. Adjust the index pattern if you use a custom
`index_prefix`/`index_suffix`. After backfilling, re-import the updated
dashboards ndjson (the index pattern saved object changed too) per the
import instructions above.
## Records retention
Starting in version 5.0.0, `parsedmarc` stores data in a separate
+5 -1
View File
@@ -82,7 +82,11 @@ Further down the dashboard, you can filter by source country or source IP
address.
Tables showing SPF and DKIM alignment details are located under the IP address
table.
table. Each row of the DKIM details table is one real DKIM signature, shown
as a combined `selector / domain / result` value; the SPF details table
shows `scope / domain / result` the same way. Combining the values into one
column keeps each signature's selector, domain, and result paired together,
rather than aggregating them as separate columns.
:::{note}
The alignment tables (SPF details, DKIM details) and the per-IP source
+26 -3
View File
@@ -99,7 +99,7 @@ class _SPFResult(InnerDoc):
domain = Text()
scope = Text()
results = Text()
result = Text()
human_result = Text()
@@ -146,6 +146,18 @@ class _AggregateReportDoc(Document):
envelope_to = Text()
dkim_results = Nested(_DKIMResult)
spf_results = Nested(_SPFResult)
# One "{selector} / {domain} / {result}" (DKIM) or "{scope} / {domain} /
# {result}" (SPF) string per auth result. Kibana/Grafana tables cannot
# terms-aggregate the subfields of an object array without producing a
# cross-product of values (issue #169), so dashboards aggregate these
# composed keywords instead. Declared to match what dynamic mapping
# produces for a string array (text + .keyword).
dkim_results_combined = Text(
multi=True, fields={"keyword": Keyword(ignore_above=256)}
)
spf_results_combined = Text(
multi=True, fields={"keyword": Keyword(ignore_above=256)}
)
np = Keyword()
testing = Keyword()
discovery_method = Keyword()
@@ -158,7 +170,7 @@ class _AggregateReportDoc(Document):
self,
domain: str,
selector: str,
result: _DKIMResult,
result: str,
human_result: str | None = None,
):
self.dkim_results.append(
@@ -169,12 +181,15 @@ class _AggregateReportDoc(Document):
human_result=human_result,
)
)
self.dkim_results_combined.append(
"{0} / {1} / {2}".format(selector, domain, result)
)
def add_spf_result(
self,
domain: str,
scope: str,
result: _SPFResult,
result: str,
human_result: str | None = None,
):
self.spf_results.append(
@@ -185,6 +200,9 @@ class _AggregateReportDoc(Document):
human_result=human_result,
)
)
self.spf_results_combined.append(
"{0} / {1} / {2}".format(scope, domain, result)
)
def save(self, **kwargs): # pyright: ignore[reportIncompatibleMethodOverride]
self.passed_dmarc = False
@@ -450,6 +468,11 @@ def create_indexes(names: list[str], settings: dict[str, Any] | None = None):
for name in names:
index = Index(name)
try:
# Deliberately no Index.document() registration: Kibana/OpenSearch
# Dashboards/Grafana cannot terms-aggregate fields inside a
# `nested` mapping, so the dynamic `object` mapping produced by a
# bare create is load-bearing for the shipped dashboards (issue
# #169; see the *_combined fields on _AggregateReportDoc).
if not index.exists():
logger.debug("Creating Elasticsearch index: {0}".format(name))
if effective_settings:
+26 -3
View File
@@ -62,7 +62,7 @@ class _DKIMResult(InnerDoc):
class _SPFResult(InnerDoc):
domain = Text()
scope = Text()
results = Text()
result = Text()
human_result = Text()
@@ -103,6 +103,18 @@ class _AggregateReportDoc(Document):
envelope_to = Text()
dkim_results = Nested(_DKIMResult)
spf_results = Nested(_SPFResult)
# One "{selector} / {domain} / {result}" (DKIM) or "{scope} / {domain} /
# {result}" (SPF) string per auth result. Kibana/Grafana tables cannot
# terms-aggregate the subfields of an object array without producing a
# cross-product of values (issue #169), so dashboards aggregate these
# composed keywords instead. Declared to match what dynamic mapping
# produces for a string array (text + .keyword).
dkim_results_combined = Text(
multi=True, fields={"keyword": Keyword(ignore_above=256)}
)
spf_results_combined = Text(
multi=True, fields={"keyword": Keyword(ignore_above=256)}
)
np = Keyword()
testing = Keyword()
discovery_method = Keyword()
@@ -115,7 +127,7 @@ class _AggregateReportDoc(Document):
self,
domain: str,
selector: str,
result: _DKIMResult,
result: str,
human_result: str | None = None,
):
self.dkim_results.append(
@@ -126,12 +138,15 @@ class _AggregateReportDoc(Document):
human_result=human_result,
)
)
self.dkim_results_combined.append(
"{0} / {1} / {2}".format(selector, domain, result)
)
def add_spf_result(
self,
domain: str,
scope: str,
result: _SPFResult,
result: str,
human_result: str | None = None,
):
self.spf_results.append(
@@ -142,6 +157,9 @@ class _AggregateReportDoc(Document):
human_result=human_result,
)
)
self.spf_results_combined.append(
"{0} / {1} / {2}".format(scope, domain, result)
)
def save(self, **kwargs): # pyright: ignore[reportIncompatibleMethodOverride]
self.passed_dmarc = False
@@ -367,6 +385,11 @@ def create_indexes(names: list[str], settings: dict[str, Any] | None = None):
for name in names:
index = Index(name)
try:
# Deliberately no Index.document() registration: Kibana/OpenSearch
# Dashboards/Grafana cannot terms-aggregate fields inside a
# `nested` mapping, so the dynamic `object` mapping produced by a
# bare create is load-bearing for the shipped dashboards (issue
# #169; see the *_combined fields on _AggregateReportDoc).
if not index.exists():
logger.debug("Creating OpenSearch index: {0}".format(name))
if settings is None:
+98
View File
@@ -549,6 +549,54 @@ class TestSaveAggregateReport(unittest.TestCase):
mock_doc_cls.call_args.kwargs["date_begin"].timestamp(), 1705276800
)
def test_save_populates_combined_dkim_and_spf_fields(self):
"""Regression guard for issue #169: two DKIM signatures on one
record must yield exactly two combined entries, not a 4-way
cross-product. autospec=True is required on the save patch so
mock_save.call_args captures the doc instance as ``self``."""
report = _aggregate_report()
report["records"][0]["auth_results"] = {
"dkim": [
{
"domain": "example.net",
"selector": "net1",
"result": "fail",
"human_result": None,
},
{
"domain": "example.org",
"selector": "org1",
"result": "pass",
"human_result": None,
},
],
"spf": [
{
"domain": "example.org",
"scope": "mfrom",
"result": "pass",
"human_result": None,
},
],
}
with (
patch("parsedmarc.elastic.Search", return_value=_empty_search()),
patch(
"parsedmarc.elastic.Index",
return_value=MagicMock(exists=MagicMock(return_value=True)),
),
patch.object(
elastic_module._AggregateReportDoc, "save", autospec=True
) as mock_save,
):
save_aggregate_report_to_elasticsearch(report)
doc = mock_save.call_args[0][0]
self.assertEqual(
list(doc.dkim_results_combined),
["net1 / example.net / fail", "org1 / example.org / pass"],
)
self.assertEqual(list(doc.spf_results_combined), ["mfrom / example.org / pass"])
class TestAggregateDocPassedDmarc(unittest.TestCase):
"""The _AggregateReportDoc.save() override derives passed_dmarc — the
@@ -576,6 +624,56 @@ class TestAggregateDocPassedDmarc(unittest.TestCase):
self.assertEqual(bool(doc.passed_dmarc), expected)
class TestAggregateDocCombinedResults(unittest.TestCase):
"""add_dkim_result/add_spf_result never touch the network, so these
construct _AggregateReportDoc directly rather than going through the
save_* entry point."""
def test_add_dkim_result_appends_combined_string(self):
"""Regression guard for issue #169: dkim_results/spf_results are
stored as nested object arrays, which Kibana/Grafana tables cannot
terms-aggregate without producing a cross-product of selector/
domain/result values. The composed "selector / domain / result"
string preserves per-signature pairing that the object-mapped
array loses."""
doc = elastic_module._AggregateReportDoc()
doc.add_dkim_result(
domain="example.net", selector="net1", result="fail", human_result=None
)
doc.add_dkim_result(
domain="example.org", selector="org1", result="pass", human_result=None
)
expected = ["net1 / example.net / fail", "org1 / example.org / pass"]
# dkim_results_combined is declared as Text(multi=True, ...); the SDK
# stub types the class attribute as Text (no Iterable protocol),
# even though the runtime value is an AttrList once multi=True is
# set. Same category of stub gap as the Q()/meta.index ignores in
# elastic.py.
self.assertEqual(list(doc.dkim_results_combined), expected) # pyright: ignore[reportArgumentType]
self.assertEqual(doc.to_dict()["dkim_results_combined"], expected)
def test_add_spf_result_appends_combined_string(self):
doc = elastic_module._AggregateReportDoc()
doc.add_spf_result(
domain="example.org", scope="mfrom", result="pass", human_result=None
)
expected = ["mfrom / example.org / pass"]
self.assertEqual(list(doc.spf_results_combined), expected) # pyright: ignore[reportArgumentType]
self.assertEqual(doc.to_dict()["spf_results_combined"], expected)
def test_spf_result_serializes_under_singular_result_key(self):
"""The _SPFResult class previously declared a dead ``results``
(plural) field while the save path wrote ``result``; verify the
serialized nested doc actually uses the singular key."""
doc = elastic_module._AggregateReportDoc()
doc.add_spf_result(
domain="example.org", scope="mfrom", result="pass", human_result=None
)
d = doc.to_dict()["spf_results"][0]
self.assertEqual(d["result"], "pass")
self.assertNotIn("results", d)
# ---------------------------------------------------------------------------
# save_failure_report_to_elasticsearch
# ---------------------------------------------------------------------------
+97
View File
@@ -602,6 +602,54 @@ class TestSaveAggregateReport(unittest.TestCase):
mock_doc_cls.call_args.kwargs["date_begin"].timestamp(), 1705276800
)
def test_save_populates_combined_dkim_and_spf_fields(self):
"""Regression guard for issue #169: two DKIM signatures on one
record must yield exactly two combined entries, not a 4-way
cross-product. autospec=True is required on the save patch so
mock_save.call_args captures the doc instance as ``self``."""
report = _aggregate_report()
report["records"][0]["auth_results"] = {
"dkim": [
{
"domain": "example.net",
"selector": "net1",
"result": "fail",
"human_result": None,
},
{
"domain": "example.org",
"selector": "org1",
"result": "pass",
"human_result": None,
},
],
"spf": [
{
"domain": "example.org",
"scope": "mfrom",
"result": "pass",
"human_result": None,
},
],
}
with (
patch("parsedmarc.opensearch.Search", return_value=_empty_search()),
patch(
"parsedmarc.opensearch.Index",
return_value=MagicMock(exists=MagicMock(return_value=True)),
),
patch.object(
opensearch_module._AggregateReportDoc, "save", autospec=True
) as mock_save,
):
save_aggregate_report_to_opensearch(report)
doc = mock_save.call_args[0][0]
self.assertEqual(
list(doc.dkim_results_combined),
["net1 / example.net / fail", "org1 / example.org / pass"],
)
self.assertEqual(list(doc.spf_results_combined), ["mfrom / example.org / pass"])
class TestAggregateDocPassedDmarc(unittest.TestCase):
"""The _AggregateReportDoc.save() override derives passed_dmarc — the
@@ -629,6 +677,55 @@ class TestAggregateDocPassedDmarc(unittest.TestCase):
self.assertEqual(bool(doc.passed_dmarc), expected)
class TestAggregateDocCombinedResults(unittest.TestCase):
"""add_dkim_result/add_spf_result never touch the network, so these
construct _AggregateReportDoc directly rather than going through the
save_* entry point."""
def test_add_dkim_result_appends_combined_string(self):
"""Regression guard for issue #169: dkim_results/spf_results are
stored as nested object arrays, which Kibana/Grafana tables cannot
terms-aggregate without producing a cross-product of selector/
domain/result values. The composed "selector / domain / result"
string preserves per-signature pairing that the object-mapped
array loses."""
doc = opensearch_module._AggregateReportDoc()
doc.add_dkim_result(
domain="example.net", selector="net1", result="fail", human_result=None
)
doc.add_dkim_result(
domain="example.org", selector="org1", result="pass", human_result=None
)
expected = ["net1 / example.net / fail", "org1 / example.org / pass"]
# dkim_results_combined is declared as Text(multi=True, ...); the SDK
# stub types the class attribute as Text (no Iterable protocol),
# even though the runtime value is an AttrList once multi=True is
# set.
self.assertEqual(list(doc.dkim_results_combined), expected) # pyright: ignore[reportArgumentType]
self.assertEqual(doc.to_dict()["dkim_results_combined"], expected)
def test_add_spf_result_appends_combined_string(self):
doc = opensearch_module._AggregateReportDoc()
doc.add_spf_result(
domain="example.org", scope="mfrom", result="pass", human_result=None
)
expected = ["mfrom / example.org / pass"]
self.assertEqual(list(doc.spf_results_combined), expected) # pyright: ignore[reportArgumentType]
self.assertEqual(doc.to_dict()["spf_results_combined"], expected)
def test_spf_result_serializes_under_singular_result_key(self):
"""The _SPFResult class previously declared a dead ``results``
(plural) field while the save path wrote ``result``; verify the
serialized nested doc actually uses the singular key."""
doc = opensearch_module._AggregateReportDoc()
doc.add_spf_result(
domain="example.org", scope="mfrom", result="pass", human_result=None
)
d = doc.to_dict()["spf_results"][0]
self.assertEqual(d["result"], "pass")
self.assertNotIn("results", d)
# ---------------------------------------------------------------------------
# save_failure_report_to_opensearch
# ---------------------------------------------------------------------------