mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-08-19 13:43:20 +00:00
Match backfill guard on either domain or result subfield
End-to-end upgrade testing (real parsedmarc 10.2.4 ingest, then a branch startup) surfaced that an exists query cannot see an empty string: a text field with no tokens is invisible to exists. The parsers we audited never store an auth result with an empty or missing domain — they drop such entries entirely, so the previous domain-only guard was sufficient for their data — but the storage shape of every historical parsedmarc version can't be audited, so the guard (and the documented manual command) now matches either the domain or the result subfield per protocol. Matching either costs nothing and cannot skip a document that has something to backfill. Verified by recomputing expected combined values from _source for all 2,299 documents on both engines: every document with stored auth results has exactly the recomputed pairs, and the legacy fo migration correctly did not fire on typeless indexes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
bbee148d2a
commit
5c0192e720
@@ -256,7 +256,11 @@ matches only documents that have at least one DKIM or SPF auth result and
|
||||
lack the corresponding combined field; documents with no auth results are
|
||||
skipped, because an `exists` query cannot see an empty array, and for
|
||||
search purposes an empty `dkim_results_combined` is identical to an
|
||||
absent one.
|
||||
absent one. Each result is matched on either its `domain` or its `result`
|
||||
subfield as defense in depth: an empty string indexes no text tokens and
|
||||
is invisible to `exists`, and the storage shape of every historical
|
||||
parsedmarc version can't be audited, so matching either subfield ensures
|
||||
no backfillable document is skipped.
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:9200/dmarc_aggregate*/_update_by_query?conflicts=proceed&wait_for_completion=false" \
|
||||
@@ -268,13 +272,33 @@ curl -X POST "http://localhost:9200/dmarc_aggregate*/_update_by_query?conflicts=
|
||||
"should": [
|
||||
{
|
||||
"bool": {
|
||||
"must": [{"exists": {"field": "dkim_results.domain"}}],
|
||||
"must": [
|
||||
{
|
||||
"bool": {
|
||||
"minimum_should_match": 1,
|
||||
"should": [
|
||||
{"exists": {"field": "dkim_results.domain"}},
|
||||
{"exists": {"field": "dkim_results.result"}}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"must_not": [{"exists": {"field": "dkim_results_combined"}}]
|
||||
}
|
||||
},
|
||||
{
|
||||
"bool": {
|
||||
"must": [{"exists": {"field": "spf_results.domain"}}],
|
||||
"must": [
|
||||
{
|
||||
"bool": {
|
||||
"minimum_should_match": 1,
|
||||
"should": [
|
||||
{"exists": {"field": "spf_results.domain"}},
|
||||
{"exists": {"field": "spf_results.result"}}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"must_not": [{"exists": {"field": "spf_results_combined"}}]
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user