- Auto-download psl_overrides.txt at startup (and whenever the reverse DNS
map is reloaded) via load_psl_overrides(); add local_psl_overrides_path
and psl_overrides_url config options
- Add collect_domain_info.py and detect_psl_overrides.py for bulk WHOIS/HTTP
enrichment and automatic cluster-based PSL override detection
- Block full-IPv4 reverse-DNS entries from ever entering
base_reverse_dns_map.csv, known_unknown_base_reverse_dns.txt, or
unknown_base_reverse_dns.csv, and sweep pre-existing IP entries
- Add Religion and Utilities to the allowed service_type values
- Document the full map-maintenance workflow in AGENTS.md
- Substantial expansion of base_reverse_dns_map.csv (net ~+1,000 entries)
- Add 26 tests covering the new loader, IP filter, PSL fold logic, and
cluster detection
Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com>
Download the latest DB-IP Country Lite mmdb from GitHub on startup and
SIGHUP, caching it locally, with fallback to a previously cached or
bundled copy. Skipped when the offline flag is set. Adds ip_db_url
config option (PARSEDMARC_GENERAL_IP_DB_URL) to override the download
URL. Bumps version to 9.6.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Should have caught this on previous fix for since. the current time is used on line 2145: connection.fetch_messages(reports_folder, since=current_time)
if that code is called and it usually won't be depending upon configuration it will fail with the time format being wrong: yyyy-mm-ddThh:mm:ss.zzzzzz+00:00Z --- this removes the extra "Z" that is not needed since utc offset is already specified and becomes invalid.
- Fixed `FileNotFoundError` when using Maildir with Docker volume mounts. Python's `mailbox.Maildir(create=True)` only creates `cur/new/tmp` subdirectories when the top-level directory doesn't exist; Docker volume mounts pre-create the directory as empty, skipping subdirectory creation. parsedmarc now explicitly creates the subdirectories when `maildir_create` is enabled.
- Maildir UID mismatch no longer crashes the process. In Docker containers where volume ownership differs from the container UID, parsedmarc now logs a warning instead of raising an exception. Also handles `os.setuid` failures gracefully in containers without `CAP_SETUID`.
- Token file writes (MS Graph and Gmail) now create parent directories automatically, preventing `FileNotFoundError` when the token path points to a directory that doesn't yet exist.
- File paths from config (`token_file`, `credentials_file`, `cert_path`, `log_file`, `output`, `ip_db_path`, `maildir_path`, syslog cert paths, etc.) now expand `~` and `$VAR` references via `os.path.expanduser`/`os.path
- Fixed `FileNotFoundError` when using Maildir with Docker volume mounts. Python's `mailbox.Maildir(create=True)` only creates `cur/new/tmp` subdirectories when the top-level directory doesn't exist; Docker volume mounts pre-create the directory as empty, skipping subdirectory creation. parsedmarc now explicitly creates the subdirectories when `maildir_create` is enabled.
Fix formatting of ISO 8601 date strings for MSGraphConnection. format yyyy-dd-mmThh:MM:SS.zzzzzz+00:00 already has a timezone indicated. The extra Z is invalid in this format. specifying a "since" in config file causes msgraph to error due to invalid time stamp.
Add environment variable configuration support and update documentation
- Introduced support for configuration via environment variables using the `PARSEDMARC_{SECTION}_{KEY}` format.
- Added `PARSEDMARC_CONFIG_FILE` variable to specify the config file path.
- Enabled env-only mode for file-less Docker deployments.
- Implemented explicit read permission checks on config files.
- Updated changelog and usage documentation to reflect these changes.
### Added
- Extracted `load_reverse_dns_map()` utility function in `utils.py` for loading the reverse DNS map independently of individual IP lookups.
- SIGHUP reload now re-downloads/reloads the reverse DNS map, so changes take effect without restarting.
- Add premade OpenSearch index patterns, visualizations, and dashboards
### Changed
- When `index_prefix_domain_map` is configured, SMTP TLS reports for domains not in the map are now silently dropped instead of being output. Unlike DMARC, TLS-RPT has no DNS authorization records, so this filtering prevents processing reports for unrelated domains.
- Bump OpenSearch support to `< 4`
### Fixed
- Fixed `get_index_prefix` using wrong key (`domain` instead of `policy_domain`) for SMTP TLS reports, which prevented domain map matching from working for TLS reports.
- Domain matching in `get_index_prefix` now lowercases the domain for case-insensitive comparison.
Elasticsearch and OpenSearch now verify SSL certificates by default when `ssl = True`, even without a `cert_path`
- Added `skip_certificate_verification` option to the `elasticsearch` and `opensearch` configuration sections for consistency with `splunk_hec`
- Splunk HEC `skip_certificate_verification` now works correctly with self-signed certificates
- SMTP TLS reports no longer fail when saving to multiple output targets (e.g. Elasticsearch and OpenSearch) due to in-place mutation of the report dict
- Output client initialization errors now identify which module failed (e.g. "OpenSearch: ConnectionError..." instead of generic "Output client error")
- Enhanced error handling for output client initialization
- Better checking of `msconfig` configuration (PR #695)
- Updated `dbip-country-lite` database to version `2026-03`
- Changed - DNS query error logging level from `warning` to `debug`
* Add MS Graph certificate authentication support
* Preserve MS Graph constructor compatibility
---------
Co-authored-by: Sean Whalen <44679+seanthegeek@users.noreply.github.com>
