mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-09-16 02:47:59 +00:00
2978436d89
* Expand reverse-DNS map and PSL overrides from the live PSL Parses the private-domains section of the live Public Suffix List and adds 269 brand-owned suffixes as PSL overrides paired with map entries, so customer subdomains on shared hosting / SaaS / PaaS platforms fold to the operator's brand. Adds 33 ASN-domain entries for the subset of these brands whose IP space is registered under a different corporate domain in the MMDB, so both the PTR-derived lookup and the ASN-fallback lookup hit the same (name, type). Also normalizes ``a2hosting.com`` from ``A2Hosting`` to ``A2 Hosting`` for spelling consistency. PTR-path wins (overrides + map entries) - Web hosts: A2 Hosting, alwaysdata, Antagonist, Beget, bplaced, Bytemark, Combell, cyber_Folks, cyon, DreamHost, EasyWP, Gehirn, HelioHost, home.pl, HostyHosting, Hypernode, IONOS (6 suffixes), Jotelulu, JouwWeb, KaasHosting, Keyweb, LCube, LiquidNet, McHost, Memset, Mittwald, Mythic Beasts, NearlyFreeSpeech, Nimbus Hosting, One.com (20 ccTLD variants), OwnProvider, Pantheon, Planet-Work, prgmr, Rackmaze, Rad Web Hosting, Raidboxes, Servebolt, SpeedPartner, Uberspace, Whatbox, WP Engine, ZAP-Hosting, Zitcom. - Dynamic DNS: DuckDNS, DynDNS (24), No-IP (22), Now-DNS, dynv6, freemyip, nsupdate.info, ddnss.de, GoIP, DrayTek. - PaaS/SaaS/IaaS: Netlify, Vercel (6), Heroku, fly.io, Render, Firebase/GCP (4), Azure (5), AWS (4), DigitalOcean (2), Red Hat OpenShift, Hasura, Supabase, Snowflake/Streamlit, Read the Docs, PythonAnywhere, GitHub, GitLab, Adobe Magento. - Hosted sites/stores: Hatena (6), Notion, Figma, Webflow, Wix (4), Shopify, Shopware, Sellfy, Spreadshop (19 ccTLDs), Datto. - Email/Marketing: Fastmail, ActiveTrail, Leadpages, Heyflow, Carrd, Typeform. - CDN/Technology: Akamai (7), Fastly (3), Yandex Cloud. ASN-path wins (MMDB coverage now attributes 1,184,256 more IPv4 addresses to a named brand, 85.04% -> 85.08%): yandex.com, ya.ru, hosting.com (A2 Hosting), beget.com, cyberfolks.pl, fly.io, bytemark.co.uk, cyberfolks.ro, keyweb.de, mittwald.de, memset.com, zap-hosting.com, datto.com, jotelulu.com, yandex.cloud, github.com, asavie.com (Akamai), and 16 others. Entries are curated from the live PSL rather than any bundled copy; brand / as_name attribution was verified against the CLAUDE.md rule that the IP-WHOIS signal is only trusted when the domain name itself matches the host's name (name-collisions in MMDB were skipped — Hypernode AU, goipgroup.com, liquidnet.com, One.com substring noise, nimbusitsolutions.com, etc.). Types follow ``base_reverse_dns_types.txt``; ``sortlists.py`` re-sorts + dedupes + validates after the batch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Document PSL-derived override workflow and load_psl_overrides gotcha Adds three pieces of map-maintenance context learned while building this PR: - New subsection "Discovering overrides from the live PSL private-domains section" — distinct source from live DMARC data (unknown_base_reverse_dns.csv) and MMDB coverage-gap analysis. The private section is itself a list of brand-owned suffixes; each is a candidate (psl_override + map entry) pair. Emphasizes ruthless selectivity — most of the 600+ private-section orgs are dev sandboxes or hobby zones that will never appear in DMARC reports. - Two-path coverage as a single linked step, not two round-trips: when adding a PSL override for a hosted-content suffix (netlify.app), also add a map row for the brand's corporate as_domain (netlify.com) in the same pass. The override fixes the PTR path; the ASN-domain alias fixes the ASN-fallback path. - The load_psl_overrides() fetch-first gotcha. The no-arg form pulls the file from master on GitHub, so end-to-end testing of local overrides silently uses the old remote version. offline=True is required to test local changes against get_base_domain(). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>