mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-05-06 03:55:25 +00:00
Sean Whalen
ec2db7238e
* README: declare base_reverse_dns_map.csv under CC BY-SA 4.0
The map is now a curated derivative of the bundled IPinfo Lite MMDB
(as_domain / as_name fields, walked for unmapped operators and
classified via the workflow in AGENTS.md). IPinfo Lite is licensed
under Creative Commons Attribution-ShareAlike 4.0, which propagates
to derivative works, so the CSV is distributed under CC BY-SA 4.0
with attribution to IPinfo for the underlying network identification
data.
Also updates the file-size estimate in the README from "over 1,400"
to "over 5,000" to reflect the current state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Alias redirect targets into the map and codify the practice in AGENTS.md
When a domain's homepage redirects to a different host *for the same
operator* (acquisition target's site, or a TLD/subdomain variant), PTR
reverse-DNS reports observed in the wild may reference either domain.
Mapping only the original loses attribution for the redirect target.
Adds 91 aliases discovered during the previous bulk PR's classification
work — every redirect target where the original was newly mapped, the
target wasn't already in the map, and the target was the same operator
(not a sister brand and not a placeholder/bot/parking page). Notable
examples: apogee.us + boldyn.com both -> Boldyn ISP; sungardas.com +
1111systems.com both -> 11:11 Systems MSP; vodafone.is + syn.is both
-> Sýn ISP; sendinblue.com + brevo.com both -> Brevo (Sendinblue)
Marketing; tigo.com + millicom.com both -> Tigo ISP; rockwellcollins.com
+ collinsaerospace.com both -> Collins Aerospace Defense.
Codifies the alias-target practice as a new paragraph under AGENTS.md
step 6 (the homepage-redirect disambiguation rule). Key guardrails:
- Alias only for case 1 (acquisition) and case 3 (TLD variant). Do
NOT alias for case 2 (sister brand / shared infra) -- aliasing the
redirect target there mis-attributes the redirect target's email.
Cited example: do not alias ziggo.nl to UPC after the chello.sk fix.
- Skip generic-placeholder, bot-management, and TLD/eTLD redirect
targets (example.com, perfdrive.com, umbler.com, co.uk, com.br...).
- When in doubt, drop the alias rather than commit it. A missing alias
is recoverable; a wrong one mis-attributes mail.
Also fixes four canonical-naming inconsistencies surfaced during the
brand-mismatch sweep, aligning recent additions to pre-existing entries:
- ga.gov: "Georgia Government" -> "State of Georgia" (matches existing
georgia.gov)
- goco.ca, radiant.net: "Telus" -> "TELUS" (matches existing telus.com)
- vee.com.tw: "VeeTime" -> "VeeTIME" (matches existing veetime.com)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Promote 21 inbound-redirect aliases from KU to map
Sweeping the session's collector TSVs for the inverse pattern of the
91 outbound aliases in commit ddf962e: domains that stayed in
known-unknown this session but whose homepage final_url redirected to
an entry that's now in the map. These are acquisitions and TLD/
subdomain variants where the operator can be inferred from the
redirect-target's existing mapping.
Notable acquisitions surfaced:
- nitelusa.com -> Comcast (NITEL was acquired by Comcast Business)
- level3.net -> Lumen (Level 3 rebranded)
- novis.pt -> NOS (Novis acquired by NOS Portugal)
- oxfordnetworks.net -> FirstLight Fiber (acquisition)
- saunalahti.fi -> Elisa (acquisition)
- omnicity.net, wcoil.com -> Watch Communications (acquisitions)
- servercentral.net -> Summit (acquisition)
TLD / subdomain variants:
- as29550.net (Simply Transit ASN domain) -> Simply Transit
- asahi-net.or.jp -> ASAHI Net (.jp variant)
- cyber-folks.pl -> cyber_Folks (cyberfolks.pl)
- digicelsr.com -> Digicel (Suriname variant)
- edpnet.net -> EDPnet (.be variant)
- la.net.ua -> Lanet
- pair.net -> Pair Networks (pair.com)
- twlakes.net -> Twin Lakes Communications
- megamailservers.eu -> MegaMailServers (.com variant)
Cloudflare email/SMTP family:
- cloudflare-email.org, cloudflare-smtp.com/.net/.org -> Cloudflare,
Email Security (matches cloudflare-email.com/.net, distinct from
the bare cloudflare.com/.net which use SaaS)
Of 32 redirect-to-mapped hits in the session TSVs, 21 cleared the
same-operator bar. The other 11 were excluded as case-2-equivalent
redirects (homepage hosted on Google/Wordpress/Aruba), registrar
parking pages (Dynadot), or ambiguous brand relationships requiring
research beyond what the redirect alone could justify (frontiernet.net
-> yahoo.com from Frontier's 2017 email-services migration to Yahoo,
dido.com -> socket.net, evo.uz -> tps.uz, ncport.ru -> avantel.ru).
Those are flagged in the PR comment for follow-up review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* AGENTS.md: document the inbound redirect-target alias sweep
After a batch lands, the same collector TSVs that drove the original
classifications are also the input to a free secondary pass: KU
domains whose final_url redirects to a host that's now mapped are
typically the inbound mirror of the outbound alias rule (step 6).
Each such pair is an acquisition or TLD/subdomain variant where the
operator is inferable from the redirect-target's existing mapping.
Adds a new bullet to "After a batch merge" describing the sweep and
the same case-2 exclusion list as the outbound rule (sister-brand,
generic hosting platform, bot-management proxy). Notes that the
sweep routinely surfaces 5-15% of the prior batch's KU additions as
legitimate map promotions, citing the actual examples that landed in
this PR (nitelusa.com -> Comcast, level3.net -> Lumen,
saunalahti.fi -> Elisa, oxfordnetworks.net -> FirstLight Fiber,
asahi-net.or.jp -> ASAHI Net, etc.).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
parsedmarc
parsedmarc is a Python module and CLI utility for parsing DMARC
reports. When used with Elasticsearch and Kibana (or Splunk), it works
as a self-hosted open-source alternative to commercial DMARC report
processing services such as Agari Brand Protection, Dmarcian, OnDMARC,
ProofPoint Email Fraud Defense, and Valimail.
Note
Domain-based Message Authentication, Reporting, and Conformance (DMARC) is an email authentication protocol.
Sponsors
This is a project is maintained by one developer. Please consider sponsoring my work if you or your organization benefit from it.
Features
- Parses draft and 1.0 standard aggregate/rua DMARC reports
- Parses forensic/failure/ruf DMARC reports
- Parses reports from SMTP TLS Reporting
- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
- Transparently handles gzip or zip compressed reports
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch, Opensearch, and/or Splunk, for use with premade dashboards
- Optionally send reports to Apache Kafka
Python Compatibility
This project supports the following Python versions, which are either actively maintained or are the default versions for RHEL or Debian.
| Version | Supported | Reason |
|---|---|---|
| < 3.6 | ❌ | End of Life (EOL) |
| 3.6 | ❌ | Used in RHEL 8, but not supported by project dependencies |
| 3.7 | ❌ | End of Life (EOL) |
| 3.8 | ❌ | End of Life (EOL) |
| 3.9 | ❌ | Used in Debian 11 and RHEL 9, but not supported by project dependencies |
| 3.10 | ✅ | Actively maintained |
| 3.11 | ✅ | Actively maintained; supported until June 2028 (Debian 12) |
| 3.12 | ✅ | Actively maintained; supported until May 2035 (RHEL 10) |
| 3.13 | ✅ | Actively maintained; supported until June 2030 (Debian 13) |
| 3.14 | ✅ | Supported (requires imapclient>=3.1.0) |