mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-07-28 19:34:55 +00:00
264b9a4556
* Ignore Claude Code agent worktrees under .claude/worktrees/ Untracked repo snapshots from agent sessions were making repo-root ruff check . fail on stale code and cluttering git status. ruff respects .gitignore, so ignoring the directory fixes both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add [general] archive_directory to archive processed local files (#570) Move successfully processed report files given as local path arguments into <archive_directory>/<year>/<month>/<Aggregate|Failure|SMTP-TLS>/, dated from the parsed report's own metadata (aggregate begin_date, failure arrival_date_utc, SMTP TLS begin_date) rather than the filename. Files that fail to parse as a report (ParserError) go to <archive_directory>/Invalid/; other failures (e.g. transient I/O errors) leave the file in place so a later run can retry it. Existing destination files are never overwritten: each candidate name is claimed atomically (O_CREAT | O_EXCL) and collisions get a numeric suffix before the extension. Files already inside the archive directory are excluded from processing (compared via realpath so symlinked spellings still match), so the archive can safely live inside an input directory, as the issue requests. mbox files and mailbox modes are unaffected; mailbox modes keep [mailbox] archive_folder. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review feedback on docstrings and the placeholder-cleanup except - _move_file_to_archive's docstring no longer claims the copy2 fallback replaces the placeholder atomically; only the same-filesystem os.rename path is atomic. The fallback is a plain copy-and-overwrite, which is still collision-safe because the placeholder already claimed the name. - The empty except OSError in the placeholder cleanup now carries a comment explaining that it is deliberate best-effort cleanup and the re-raised move failure is the actionable error. - test_general_archive_directory_unset_leaves_attribute_absent's docstring now describes what the assertion actually tests (the attribute staying absent from a bare Namespace) and moves the real-CLI None-default behavior to a parenthetical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Cover the two defensive exception branches Codecov flagged Codecov's patch report flagged four uncovered lines, all in the two platform-dependent exception branches of the archive helpers: - _exclude_archived_paths's except ValueError branch. Per the Python docs for os.path.commonpath, ValueError is raised when paths "are on the different drives" (Windows) or mix absolute and relative pathnames; both inputs are realpath()-resolved so only the different-drives case remains, which Linux CI can't produce naturally. The new test simulates the raise and asserts the non-comparable path is kept for parsing rather than excluded. - _move_file_to_archive's except OSError placeholder-cleanup branch. The new test fails the move with a non-OSError type and the cleanup with OSError, then asserts the move error is what propagates (the cleanup error is swallowed, not allowed to mask it) and that the zero-byte placeholder survives its failed cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Tag arrival_date_utc as UTC when parsing the archive date human_timestamp_to_datetime's docstring names arrival_date_utc as exactly the kind of known-UTC naive string that should be parsed with assume_utc=True; _archive_subdir_for_result was parsing it naive. The flag is scoped to the failure branch because the shared call also handles the other two report types: aggregate begin_date is a local-time wall-clock string (timestamp_to_human uses datetime.fromtimestamp), so tagging it UTC would be wrong, and SMTP TLS begin_date carries an RFC 3339 offset, making assume_utc a no-op. No observable behavior change: only the wall-clock year/month fields are read and assume_utc never shifts wall-clock time, so no new test can honestly distinguish the two versions — this aligns the call with its dependency's documented contract and hardens against a future edit adding a real timezone conversion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Create the archive placeholder with mode 0o600 os.open's mode parameter defaults to 0o777 (masked by the umask), so the O_CREAT|O_EXCL placeholder in _move_file_to_archive was created executable and group-accessible on typical umasks (0o775 under umask 002). Normally it's replaced immediately, but a placeholder that outlives a failed move+cleanup persisted with those permissions. Pass 0o600 explicitly. The mode never reaches the real archived file: os.rename replaces the placeholder's inode outright, and the copy2 fallback's copystat overwrites the mode with the source file's. The leftover-placeholder regression test now also asserts the surviving placeholder has no owner-exec or group/other bits (umask-independent, since the umask only clears bits); the assertion fails against the unfixed default-mode call. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
parsedmarc
parsedmarc is a Python module and CLI utility for parsing DMARC
reports. When used with Elasticsearch and Kibana (or Splunk), it works
as a self-hosted open-source alternative to commercial DMARC report
processing services such as Agari Brand Protection, Dmarcian, OnDMARC,
ProofPoint Email Fraud Defense, and Valimail.
Note
Domain-based Message Authentication, Reporting, and Conformance (DMARC) is an email authentication protocol.
Sponsors
This project is maintained by one developer. Please consider sponsoring my work if you or your organization benefit from it.
Features
- Parses aggregate/rua DMARC reports: the legacy draft and 1.0 schemas (RFC 7489) and the new RFC 9990 schema for the final DMARC standard (RFC 9989)
- Parses failure/ruf DMARC reports (RFC 6591 and RFC 9991; formerly called forensic reports)
- Parses reports from SMTP TLS Reporting (TLS-RPT, RFC 8460)
- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
- Transparently handles gzip or zip compressed reports
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch, OpenSearch, Splunk, or PostgreSQL, for use with premade dashboards
- Optionally send the results to Apache Kafka, Amazon S3, Azure Log Analytics (Microsoft Sentinel), a Graylog (GELF) endpoint, a syslog server, or an HTTP webhook
Python Compatibility
This project supports the following Python versions, which are either actively maintained or are the default versions for RHEL or Debian.
| Version | Supported | Reason |
|---|---|---|
| < 3.6 | ❌ | End of Life (EOL) |
| 3.6 | ❌ | Used in RHEL 8, but not supported by project dependencies |
| 3.7 | ❌ | End of Life (EOL) |
| 3.8 | ❌ | End of Life (EOL) |
| 3.9 | ❌ | Used in Debian 11 and RHEL 9, but not supported by project dependencies |
| 3.10 | ✅ | Actively maintained |
| 3.11 | ✅ | Actively maintained; supported until June 2028 (Debian 12) |
| 3.12 | ✅ | Actively maintained; supported until May 2035 (RHEL 10) |
| 3.13 | ✅ | Actively maintained; supported until June 2030 (Debian 13) |
| 3.14 | ✅ | Supported (requires imapclient>=3.1.0) |