mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-07-28 11:24:55 +00:00
Sean WhalenandClaude Fable 5
95e881bdb3
Review findings verified against the parsedmarc serializers, the UDM
field list, Google's parser troubleshooting doc, and the content-hub
parsers:
- Detect failure reports by feedback_type OR arrival_date_utc: the
text-format failure path synthesizes a feedback report with no
Feedback-Type field, so those rows had no feedback_type key and were
dropped as TAG_UNSUPPORTED.
- Detect SMTP TLS rows by policy_type OR result_type, and fall back to
receiving_mx_hostname for target.hostname, so failure-detail rows from
parsedmarc versions without the paired serializer fix still map.
- Merge (not replace) network.email.to and network.email.subject: both
are repeated fields per the UDM field list, so a scalar replace is a
type mismatch. The unguarded subject replace could have failed parsing
for every failure event with a subject.
- Guard testing with != "": parsedmarc emits the RFC 9990 t= flag as the
string "y"/"n", not a boolean, so the == "true"/"false" test could
never fire and dmarc_testing was never emitted. Drop testing from the
boolean convert list accordingly.
- Add on_error to the four numeric %{} interpolations (count,
source_asn, *_session_count) so a tenant where interpolating a
non-string field errors degrades to a missing additional.fields entry
instead of _failed_parsing_ (per the official parser tips doc).
- Drop the concatenated security_result.description strings: the tips
doc says not to pack multiple values into one UDM field, and every
value already lands individually in additional.fields.
- Use TAG_MALFORMED_MESSAGE for non-JSON input, matching the tips doc's
canonical not-JSON example (TAG_MALFORMED_ENCODING is for character
encoding problems).
- Map org_extra_contact_info/errors (aggregate) and the source
enrichment fields (failure) into additional.fields; the failure branch
previously dropped enrichment the aggregate branch kept.
- Replace the SMTP TLS success sample, which was not generated from this
repository's samples, with rfc8460.json output; regenerate all sample
events with the fixed serializer and name each event's source sample
file. All six samples are now byte-identical to real syslog output.
- Reorder the validation plan to start with the failure sample: it is
the only shape that emits JSON nulls, whose json{} interaction with
the pre-initialized "" fields is undocumented.
- Update Google doc links to docs.cloud.google.com (the old URLs 301)
and link the parser from the syslog section of the usage docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
parsedmarc
parsedmarc is a Python module and CLI utility for parsing DMARC
reports. When used with Elasticsearch and Kibana (or Splunk), it works
as a self-hosted open-source alternative to commercial DMARC report
processing services such as Agari Brand Protection, Dmarcian, OnDMARC,
ProofPoint Email Fraud Defense, and Valimail.
Note
Domain-based Message Authentication, Reporting, and Conformance (DMARC) is an email authentication protocol.
Sponsors
This project is maintained by one developer. Please consider sponsoring my work if you or your organization benefit from it.
Features
- Parses aggregate/rua DMARC reports: the legacy draft and 1.0 schemas (RFC 7489) and the new RFC 9990 schema for the final DMARC standard (RFC 9989)
- Parses failure/ruf DMARC reports (RFC 6591 and RFC 9991; formerly called forensic reports)
- Parses reports from SMTP TLS Reporting (TLS-RPT, RFC 8460)
- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
- Transparently handles gzip or zip compressed reports
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch, OpenSearch, Splunk, or PostgreSQL, for use with premade dashboards
- Optionally send the results to Apache Kafka, Amazon S3, Azure Log Analytics (Microsoft Sentinel), a Graylog (GELF) endpoint, a syslog server, or an HTTP webhook
Python Compatibility
This project supports the following Python versions, which are either actively maintained or are the default versions for RHEL or Debian.
| Version | Supported | Reason |
|---|---|---|
| < 3.6 | ❌ | End of Life (EOL) |
| 3.6 | ❌ | Used in RHEL 8, but not supported by project dependencies |
| 3.7 | ❌ | End of Life (EOL) |
| 3.8 | ❌ | End of Life (EOL) |
| 3.9 | ❌ | Used in Debian 11 and RHEL 9, but not supported by project dependencies |
| 3.10 | ✅ | Actively maintained |
| 3.11 | ✅ | Actively maintained; supported until June 2028 (Debian 12) |
| 3.12 | ✅ | Actively maintained; supported until May 2035 (RHEL 10) |
| 3.13 | ✅ | Actively maintained; supported until June 2030 (Debian 13) |
| 3.14 | ✅ | Supported (requires imapclient>=3.1.0) |