mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-11-07 01:09:26 +00:00
Sean WhalenandClaude Opus 5.5
56e5a09923
* Add opt-in Elasticsearch/OpenSearch data streams (#842) New use_data_streams option in [elasticsearch] and [opensearch] saves each report type to a data stream named like today's index without the date suffix, instead of daily or monthly indexes. parsedmarc puts an exact-name index template (priority 501, @timestamp mapped as date, <stream>@custom component template on Elasticsearch) and creates the stream; retention is left to operator-managed ILM/ISM policies, with documented examples. Dated indexes are untouched and still searched for duplicates and matched by the dashboards. Every document now carries @timestamp and is saved with op_type=create in both modes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix data stream prose found by cross-check review (layer 1) Aggregate @timestamp is each record's begin date, not the report's; the cluster only rolls backing indices over under an attached policy; move the Elasticsearch 8.7 note after the template it refers to; clarify the name-collision paragraph, OpenSearch-side comments, the migrate_indexes docstring, and the test-module headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tighten names and wiring tests found by cross-check review (layer 2) Split the use_data_streams CLI passthrough test per backend so a call site reading the other section's flag fails; name the dated-mode tests for both changes they assert; compute index_date only in dated mode in every save function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Align data stream docs with Elastic/OpenSearch references (layer 3) Say the stream must exist before reindexing into it (ES reindex API: auto-creation needs a data-stream template); an existing stream's write index needs the ILM policy attached directly since nothing rolls it over otherwise; target the stream name in the _settings example; note OpenSearch's leading-dot rule for stream names and Fleet's template priorities. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cover the Loki client-initialization error wrapper _parse_config accepts a [loki] token alongside username/password; only the LokiClient constructor rejects the combination. Exercise that path through _init_output_clients with the real constructor so the RuntimeError("Loki: ...") wrapper Codecov flagged as uncovered is tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
parsedmarc
parsedmarc is a Python module and CLI utility for parsing DMARC
reports. When used with Elasticsearch and Kibana (or Splunk), or with
OpenSearch and Grafana, it works as a self-hosted open-source
alternative to commercial DMARC report processing services such as
Agari Brand Protection, Dmarcian, OnDMARC, ProofPoint Email Fraud
Defense, and Valimail.
Note
Domain-based Message Authentication, Reporting, and Conformance (DMARC) is an email authentication protocol.
Sponsors
This project is maintained by one developer. Please consider sponsoring my work if you or your organization benefit from it.
Features
- Parses aggregate/rua DMARC reports: the legacy draft and 1.0 schemas (RFC 7489) and the new RFC 9990 schema for the final DMARC standard (RFC 9989)
- Parses failure/ruf DMARC reports (RFC 6591 and RFC 9991; formerly called forensic reports)
- Parses reports from SMTP TLS Reporting (TLS-RPT, RFC 8460)
- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
- Transparently handles gzip or zip compressed reports
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch, OpenSearch, Splunk, or PostgreSQL, for use with premade dashboards
- Optionally send the results to Apache Kafka, Amazon S3, Azure Log Analytics (Microsoft Sentinel), a Graylog (GELF) endpoint, a syslog server, Grafana Loki, or an HTTP webhook
Python Compatibility
This project supports the following Python versions, which are either actively maintained or are the default versions for RHEL or Debian.
| Version | Supported | Reason |
|---|---|---|
| < 3.6 | ❌ | End of Life (EOL) |
| 3.6 | ❌ | Used in RHEL 8, but not supported by project dependencies |
| 3.7 | ❌ | End of Life (EOL) |
| 3.8 | ❌ | End of Life (EOL) |
| 3.9 | ❌ | Used in Debian 11 and RHEL 9, but not supported by project dependencies |
| 3.10 | ✅ | Actively maintained |
| 3.11 | ✅ | Actively maintained; supported until June 2028 (Debian 12) |
| 3.12 | ✅ | Actively maintained; supported until May 2035 (RHEL 10) |
| 3.13 | ✅ | Actively maintained; supported until June 2030 (Debian 13) |
| 3.14 | ✅ | Supported (requires imapclient>=3.1.0) |