mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-07-28 11:24:55 +00:00
Sean WhalenandClaude Fable 5
fba083f351
Sends parsed reports to a Google SecOps instance as pre-normalized UDM
events through the GA v1 Chronicle API:
POST https://chronicle.{region}.rep.googleapis.com/v1/{parent}/events:import
Pre-normalized events bypass SecOps's server-side (CBN) parsing layer,
so no parser needs to be installed in the tenant; the CBN parser in
google_secops_parser/ remains the collector-based alternative for
deployments that want raw-log retention, and both paths emit the same
UDM shape and additional-field keys so searches port between them.
Google's ingestion docs recommend the UDM-events path when possible.
Implementation notes, all grounded in the v1 reference docs:
- Auth is standard Google Cloud IAM (Chronicle API Editor role /
chronicle.events.import permission): a service account key file when
[gsecops] credentials_file is set, Application Default Credentials
otherwise. google-auth was already a transitive requirement via
mailsuite[gmail]; it is now declared directly.
- Batches follow the documented best practices (1,000 events per
request, 60 s timeout). events.import is all-or-nothing — one invalid
event rejects the whole request — so a rejected batch is bisected to
isolate invalid events; valid events are still delivered and the drop
count is raised as an output error at the end.
- to and subject are repeated fields in the UDM Email message;
SecurityResult action and category are repeated enums.
- additional is a protobuf Struct, so counts and ASNs stay numbers and
alignment flags stay booleans — range-queryable without the CBN
string-hop.
New [gsecops] config section (project_id, instance_id, region,
credentials_file) is wired through the INI schema, _parse_config,
Namespace defaults, PARSEDMARC_GSECOPS_* env vars, the usage docs, and
per-batch client construction (SIGHUP-reload safe by construction).
Tests build events from real sample reports and assert on the payloads
sent through a mocked AuthorizedSession, including batching and the
400-bisect path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
parsedmarc
parsedmarc is a Python module and CLI utility for parsing DMARC
reports. When used with Elasticsearch and Kibana (or Splunk), it works
as a self-hosted open-source alternative to commercial DMARC report
processing services such as Agari Brand Protection, Dmarcian, OnDMARC,
ProofPoint Email Fraud Defense, and Valimail.
Note
Domain-based Message Authentication, Reporting, and Conformance (DMARC) is an email authentication protocol.
Sponsors
This project is maintained by one developer. Please consider sponsoring my work if you or your organization benefit from it.
Features
- Parses aggregate/rua DMARC reports: the legacy draft and 1.0 schemas (RFC 7489) and the new RFC 9990 schema for the final DMARC standard (RFC 9989)
- Parses failure/ruf DMARC reports (RFC 6591 and RFC 9991; formerly called forensic reports)
- Parses reports from SMTP TLS Reporting (TLS-RPT, RFC 8460)
- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
- Transparently handles gzip or zip compressed reports
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch, OpenSearch, Splunk, or PostgreSQL, for use with premade dashboards
- Optionally send the results to Apache Kafka, Amazon S3, Azure Log Analytics (Microsoft Sentinel), a Graylog (GELF) endpoint, a syslog server, or an HTTP webhook
Python Compatibility
This project supports the following Python versions, which are either actively maintained or are the default versions for RHEL or Debian.
| Version | Supported | Reason |
|---|---|---|
| < 3.6 | ❌ | End of Life (EOL) |
| 3.6 | ❌ | Used in RHEL 8, but not supported by project dependencies |
| 3.7 | ❌ | End of Life (EOL) |
| 3.8 | ❌ | End of Life (EOL) |
| 3.9 | ❌ | Used in Debian 11 and RHEL 9, but not supported by project dependencies |
| 3.10 | ✅ | Actively maintained |
| 3.11 | ✅ | Actively maintained; supported until June 2028 (Debian 12) |
| 3.12 | ✅ | Actively maintained; supported until May 2035 (RHEL 10) |
| 3.13 | ✅ | Actively maintained; supported until June 2030 (Debian 13) |
| 3.14 | ✅ | Supported (requires imapclient>=3.1.0) |