- Auto-download psl_overrides.txt at startup (and whenever the reverse DNS
  map is reloaded) via load_psl_overrides(); add local_psl_overrides_path
  and psl_overrides_url config options
- Add collect_domain_info.py and detect_psl_overrides.py for bulk WHOIS/HTTP
  enrichment and automatic cluster-based PSL override detection
- Block full-IPv4 reverse-DNS entries from ever entering
  base_reverse_dns_map.csv, known_unknown_base_reverse_dns.txt, or
  unknown_base_reverse_dns.csv, and sweep pre-existing IP entries
- Add Religion and Utilities to the allowed service_type values
- Document the full map-maintenance workflow in AGENTS.md
- Substantial expansion of base_reverse_dns_map.csv (net ~+1,000 entries)
- Add 26 tests covering the new loader, IP filter, PSL fold logic, and
  cluster detection

Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com>
This commit is contained in:
Sean Whalen
2026-04-19 21:20:41 -04:00
committed by GitHub
co-authored by Sean Whalen
parent 10dd7c0459
commit 6effd80604
16 changed files with 5025 additions and 13 deletions
+2
View File
@@ -143,6 +143,8 @@ The full set of configuration options are:
IP-to-country database and reverse DNS map
- `local_reverse_dns_map_path` - Overrides the default local file path to use for the reverse DNS map
- `reverse_dns_map_url` - Overrides the default download URL for the reverse DNS map
- `local_psl_overrides_path` - Overrides the default local file path to use for the PSL overrides list
- `psl_overrides_url` - Overrides the default download URL for the PSL overrides list
- `nameservers` - str: A comma separated list of
DNS resolvers (Default: `[Cloudflare's public resolvers]`)
- `dns_test_address` - str: a dummy address used for DNS pre-flight checks