Add a Domain policy dropdown filter to the Splunk aggregate dashboard (#854) (#855)

Adds a dropdown that filters every panel on the published DMARC policy
(published_policy.p) or subdomain policy (published_policy.sp), mirroring
the Message disposition dropdown's choices and default and sitting
immediately before it, per the issue spec. Both fields are always present
on parsedmarc-written events (sp defaults to p in the parser), so the
wildcard default matches everything, consistent with the other raw-search
filters.

Also fixes the Splunk docs page's XML-files link, which still pointed at
the pre-#736 splunk/ path and 404ed.

Closes #854

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sean Whalen
2026-07-25 20:48:44 -04:00
committed by GitHub
co-authored by Claude Fable 5
parent b02a7972b5
commit a421747a79
3 changed files with 12 additions and 2 deletions
+2
View File
@@ -6,9 +6,11 @@
- **`n_procs` parallel parsing now covers messages from mbox files and mailbox connections** ([#147](https://github.com/domainaware/parsedmarc/issues/147)), not just report files passed directly as CLI arguments: IMAP, Microsoft Graph, Gmail API, and Maildir connections, including watch mode. `get_dmarc_reports_from_mbox()`, `get_dmarc_reports_from_mailbox()`, and `watch_inbox()` all gained an `n_procs` keyword argument. Parallel workers are now a reused process pool for the whole run, with a bounded submission window that keeps at most roughly `2 * n_procs` messages in flight at a time so memory stays bounded even for huge mboxes; the main process sends periodic IMAP keepalives while workers parse.
- **Added `ParserConfig`, a single frozen dataclass carrying every parsing/enrichment option** ([#503](https://github.com/domainaware/parsedmarc/issues/503)): offline mode, IP database path, reverse DNS map and PSL overrides paths/URLs, DNS nameservers/timeout/retries, `strip_attachment_payloads`, `normalize_timespan_threshold_hours`, and the three caches the parser and enrichment code share across calls (IP address info, seen aggregate report IDs, and the reverse DNS map). `parse_aggregate_report_xml()`, `parse_aggregate_report_file()`, `parse_failure_report()`, `parse_report_email()`, `parse_report_file()`, `get_dmarc_reports_from_mbox()`, `get_dmarc_reports_from_mailbox()`, and `watch_inbox()` all accept a keyword-only `config=` argument; when it's provided, the individual option keyword arguments are ignored in favor of the config's values, and every existing per-option keyword argument continues to work unchanged when `config=` is omitted. Every explicitly constructed `ParserConfig` owns fresh, isolated caches; omitting `config=` falls back to the existing process-wide shared default caches, unchanged and identity-preserved (`parsedmarc.IP_ADDRESS_CACHE`, `parsedmarc.SEEN_AGGREGATE_REPORT_IDS`, `parsedmarc.REVERSE_DNS_MAP`). Caches never cross multiprocessing worker boundaries: `ParserConfig.__getstate__` drops the three cache fields when a config is pickled for a worker, and each worker accumulates its own from that point on. `keep_alive` and `n_procs` remain separate keyword arguments — they control process/worker orchestration, not parsing or enrichment behavior, so they're deliberately not `ParserConfig` fields. See the new "Using parsedmarc as a library" section of the usage docs for an example.
- **Added a "Domain policy" dropdown filter to the Splunk aggregate DMARC dashboard** ([#854](https://github.com/domainaware/parsedmarc/issues/854)): filters every panel on the published DMARC policy (`published_policy.p`) or subdomain policy (`published_policy.sp`), making it easy to see which domains with a `none` policy are ready to move to `quarantine` or `reject`. It sits immediately before the "Message disposition" dropdown and offers the same choices (`any`/`none`/`quarantine`/`reject`, defaulting to `any`).
### Bug fixes
- **Fixed the broken "XML files" link on the Splunk docs page**: it pointed at the old `splunk/` repository path instead of `dashboards/splunk/`, where the dashboards have lived since the `dashboards/` directory was introduced.
- **A report file whose parsing raised an unexpected non-parser exception no longer hangs the CLI forever.** The old direct-file parallel implementation spawned a fresh child process per file and blocked the parent on a pipe read; a child that crashed with anything other than a `ParserError` never wrote to that pipe, so the parent waited indefinitely. The new implementation returns the error as a value and logs `Failed to parse <path>` instead.
- **Direct-file parallel parsing no longer stalls a whole batch on one slow file, and no longer spawns a fresh interpreter per file.** The old implementation processed files in hard batches of `n_procs`, so a single slow file delayed every other file in its batch; the new pooled-worker implementation streams files through a reused pool instead.
- **A one-shot mailbox run (no `--watch`) now honors `[general] dns_timeout` and `dns_retries`** ([#503](https://github.com/domainaware/parsedmarc/issues/503)). The CLI's call to `get_dmarc_reports_from_mailbox()` never forwarded those two options, so every one-shot mailbox run silently used the library's own hardcoded default (`dns_timeout=6.0`, `dns_retries=0`) regardless of what the operator configured; watch-mode runs were unaffected since the `watch_inbox()` call site did pass them. The CLI now builds a single `ParserConfig` per run (via the new `_build_parser_config()` helper) from parsed options and passes it as `config=` to every parsing entry point — the direct-file, mbox, one-shot mailbox, and watch call sites, plus the SIGHUP config-reload path — instead of forwarding option keyword arguments by hand at each call site.
@@ -3,7 +3,7 @@
<description>A summary of aggregate DMARC report data</description>
<search id="base_search">
<query>
index="email" sourcetype="dmarc:aggregate" spf_aligned=$spf_aligned$ dkim_aligned=$dkim_aligned$ passed_dmarc=$passed_dmarc$ org_name=$org_name$ source_reverse_dns=$source_reverse_dns$ header_from=$header_from$ envelope_from=$envelope_from$ disposition=$disposition$ source_ip_address=$source_ip_address$ source_base_domain=$source_base_domain$ source_country=$source_country$
index="email" sourcetype="dmarc:aggregate" spf_aligned=$spf_aligned$ dkim_aligned=$dkim_aligned$ passed_dmarc=$passed_dmarc$ org_name=$org_name$ source_reverse_dns=$source_reverse_dns$ header_from=$header_from$ envelope_from=$envelope_from$ (published_policy.p=$domain_policy$ OR published_policy.sp=$domain_policy$) disposition=$disposition$ source_ip_address=$source_ip_address$ source_base_domain=$source_base_domain$ source_country=$source_country$
| rename spf_results{}.domain as envelope_domain spf_results{}.result as spf_result spf_results{}.scope as spf_scope dkim_results{}.selector as dkim_selector dkim_results{}.domain as dkim_domain dkim_results{}.result as dkim_result
| fillnull value=null source_reverse_dns source_base_domain dkim_selector dkim_domain dkim_result source_type source_name source_as_name
| search dkim_selector=$dkim_selector$ dkim_domain=$dkim_domain$ source_type="$source_type$" source_name="$source_name$" source_as_name="$source_as_name$"
@@ -46,6 +46,14 @@
<label>Envelope from</label>
<default>*</default>
</input>
<input type="dropdown" token="domain_policy" searchWhenChanged="true">
<label>Domain policy</label>
<choice value="*">any</choice>
<choice value="none">none</choice>
<choice value="quarantine">quarantine</choice>
<choice value="reject">reject</choice>
<default>*</default>
</input>
<input type="dropdown" token="disposition" searchWhenChanged="true">
<label>Message disposition</label>
<choice value="*">any</choice>
+1 -1
View File
@@ -18,5 +18,5 @@ The Splunk dashboards display the same content and layout as the
Kibana dashboards, although the Kibana dashboards have slightly
easier and more flexible filtering options.
[xml files]: https://github.com/domainaware/parsedmarc/tree/master/splunk
[xml files]: https://github.com/domainaware/parsedmarc/tree/master/dashboards/splunk
[http event collector (hec)]: http://docs.splunk.com/Documentation/Splunk/latest/Data/AboutHEC