Welcome to parsedmarc’s documentation!¶

pasedmarc is a Python module and CLI utility for parsing DMARC reports.

Features¶

Parses draft and 1.0 standard aggregate reports
Parses forensic reports
Can parse reports from an inbox over IMAP
Transparently handles gzip or zip compressed reports
Consistent data structures
Simple JSON and/or CSV output
Optionally email the results

CLI help¶

usage: parsedmarc [-h] [-o OUTPUT] [-n NAMESERVERS [NAMESERVERS ...]]
                  [-t TIMEOUT] [-H HOST] [-u USER] [-p PASSWORD]
                  [-a ARCHIVE_FOLDER] [-d] [-O OUTGOING_HOST]
                  [-U OUTGOING_USER] [-P OUTGOING_PASSWORD]
                  [-F OUTGOING_FROM] [-T OUTGOING_TO [OUTGOING_TO ...]]
                  [-S OUTGOING_SUBJECT] [-A OUTGOING_ATTACHMENT]
                  [-M OUTGOING_MESSAGE] [-i] [--test] [-v]
                  [file_path [file_path ...]]

Parses DMARC reports

positional arguments:
  file_path             one or more paths of aggregate report files
                        (compressed or uncompressed)

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Write output files to the given directory
  -n NAMESERVERS [NAMESERVERS ...], --nameservers NAMESERVERS [NAMESERVERS ...]
                        nameservers to query
  -t TIMEOUT, --timeout TIMEOUT
                        number of seconds to wait for an answer from DNS
                        (default 6.0)
  -H HOST, --host HOST  IMAP hostname or IP address
  -u USER, --user USER  IMAP user
  -p PASSWORD, --password PASSWORD
                        IMAP password
  -a ARCHIVE_FOLDER, --archive-folder ARCHIVE_FOLDER
                        Specifies the IMAP folder to move messages to after
                        processing them (default: Archive)
  -d, --delete          Delete the reports after processing them
  -O OUTGOING_HOST, --outgoing-host OUTGOING_HOST
                        Email the results using this host
  -U OUTGOING_USER, --outgoing-user OUTGOING_USER
                        Email the results using this user
  -P OUTGOING_PASSWORD, --outgoing-password OUTGOING_PASSWORD
                        Email the results using this password
  -F OUTGOING_FROM, --outgoing-from OUTGOING_FROM
                        Email the results using this from address
  -T OUTGOING_TO [OUTGOING_TO ...], --outgoing-to OUTGOING_TO [OUTGOING_TO ...]
                        Email the results to these addresses
  -S OUTGOING_SUBJECT, --outgoing-subject OUTGOING_SUBJECT
                        Email the results using this subject
  -A OUTGOING_ATTACHMENT, --outgoing-attachment OUTGOING_ATTACHMENT
                        Email the results using this filename
  -M OUTGOING_MESSAGE, --outgoing-message OUTGOING_MESSAGE
                        Email the results using this message
  -i, --idle            Use an IMAP IDLE connection to process reports as they
                        arrive in the inbox
  --test                Do not move or delete IMAP messages
  -v, --version         show program's version number and exit

SPF and DMARC record validation¶

If you are looking for SPF and DMARC record validation and parsing, check out the sister project, checkdmarc.

Sample aggregate report output¶

Here are the results from parsing the example report from the dmarc.org wiki. It’s actually an older draft of the the 1.0 report schema standardized in RFC 7480 Appendix C. This draft schema is still in wide use.

parsedmarc produces consistent, normalized output, regardless of the report schema.

JSON¶

{
  "xml_schema": "draft",
  "report_metadata": {
    "org_name": "acme.com",
    "org_email": "noreply-dmarc-support@acme.com",
    "org_extra_contact_info": "http://acme.com/dmarc/support",
    "report_id": "9391651994964116463",
    "begin_date": "2012-04-27 20:00:00",
    "end_date": "2012-04-28 19:59:59",
    "errors": []
  },
  "policy_published": {
    "domain": "example.com",
    "adkim": "r",
    "aspf": "r",
    "p": "none",
    "sp": "none",
    "pct": "100",
    "fo": "0"
  },
  "records": [
    {
      "source": {
        "ip_address": "72.150.241.94",
        "country": "US",
        "reverse_dns": "adsl-72-150-241-94.shv.bellsouth.net",
        "base_domain": "bellsouth.net"
      },
      "count": 2,
      "policy_evaluated": {
        "disposition": "none",
        "dkim": "fail",
        "spf": "pass",
        "policy_override_reasons": []
      },
      "identifiers": {
        "header_from": "example.com",
        "envelope_from": "example.com",
        "envelope_to": null
      },
      "auth_results": {
        "dkim": [
          {
            "domain": "example.com",
            "selector": "none",
            "result": "fail"
          }
        ],
        "spf": [
          {
            "domain": "example.com",
            "scope": "mfrom",
            "result": "pass"
          }
        ]
      }
    }
  ]
}

CSV¶

xml_schema,org_name,org_email,org_extra_contact_info,report_id,begin_date,end_date,errors,domain,adkim,aspf,p,sp,pct,fo,source_ip_address,source_country,source_reverse_dns,source_base_domain,count,disposition,dkim_alignment,spf_alignment,policy_override_reasons,policy_override_comments,envelope_from,header_from,envelope_to,dkim_domains,dkim_selectors,dkim_results,spf_domains,spf_scopes,spf_results
draft,acme.com,noreply-dmarc-support@acme.com,http://acme.com/dmarc/support,9391651994964116463,2012-04-27 20:00:00,2012-04-28 19:59:59,,example.com,r,r,none,none,100,0,72.150.241.94,US,adsl-72-150-241-94.shv.bellsouth.net,bellsouth.net,2,none,fail,pass,,,example.com,example.com,,example.com,none,fail,example.com,mfrom,pass

Sample forensic report output¶

I don’t have a sample I can share for privacy reasons. If you have a sample forensic report that you can share publicly, please contact me!

Bug reports¶

Please report bugs on the GitHub issue tracker

https://github.com/domainaware/parsedmarc/issues

Installation¶

parsedmarc works with Python 2 or 3, but Python 3 is preferred.

On Debian or Ubuntu systems, run:

$ sudo apt-get install python3-pip

Python 3 installers for Windows and macOS can be found at https://www.python.org/downloads/

To install or upgrade to the latest stable release of parsedmarc on macOS or Linux, run

$ sudo -H pip3 install -U parsedmarc

Or, install the latest development release directly from GitHub:

$ sudo -H pip3 install -U git+https://github.com/domainaware/parsedmarc.git

Note

On Windows, pip3 is pip, even with Python 3. So on Windows, simply substitute pip as an administrator in place of sudo pip3, in the above commands.

Optional dependencies¶

If you would like to be able to parse emails saved from Microsoft Outlook (i.e. OLE .msg files), install msgconvert:

On Debian or Ubuntu systems, run:

$ sudo apt-get install libemail-outlook-message-perl

API¶

A Python module and CLI for parsing DMARC reports

exception parsedmarc.IMAPError[source]¶: Raised when an IMAP error occurs

exception parsedmarc.InvalidAggregateReport[source]¶: Raised when an invalid DMARC aggregate report is encountered

exception parsedmarc.InvalidDMARCReport[source]¶: Raised when an invalid DMARC report is encountered

exception parsedmarc.InvalidForensicReport[source]¶: Raised when an invalid DMARC forensic report is encountered

exception parsedmarc.ParserError[source]¶: Raised whenever the parser fails for some reason

exception parsedmarc.SMTPError[source]¶: Raised when a SMTP error occurs

parsedmarc.email_results(results, host, mail_from, mail_to, port=0, starttls=True, use_ssl=False, user=None, password=None, subject=None, attachment_filename=None, message=None, ssl_context=None)[source]¶

Emails parsing results as a zip file

Parameters:

results (OrderedDict) – Parsing results
host – Mail server hostname or IP address
mail_from – The value of the message from header
mail_to – A list of addresses to mail to
port (int) – Port to use
starttls (bool) – use STARTTLS
use_ssl (bool) – Require a SSL connection from the start
user – An optional username
password – An optional password
subject – Overrides the default message subject
attachment_filename – Override the default attachment filename
message – Override the default plain text body
ssl_context – SSL context options

parsedmarc.extract_xml(input_)[source]¶

Extracts xml from a zip or gzip file at the given path, file-like object, or bytes.

Parameters:	input – A path to a file, a file like object, or bytes
Returns:	The extracted XML
Return type:	str

parsedmarc.get_dmarc_reports_from_inbox(host, user, password, archive_folder='Archive', delete=False, test=False, nameservers=None, dns_timeout=6.0)[source]¶

Fetches and parses DMARC reports from sn inbox

Parameters:	host – The mail server hostname or IP address user – The mail server user password – The mail server password archive_folder – The folder to move processed mail to delete (bool) – Delete messages after processing them test (bool) – Do not move or delete messages after processing them nameservers (list) – A list of DNS nameservers to query dns_timeout (float) – Set the DNS query timeout
Returns:	Lists of `aggregate_reports` and `forensic_reports`
Return type:	OrderedDict

parsedmarc.get_report_zip(results)[source]¶

Creates a zip file of parsed report output

Parameters:	results (OrderedDict) – The parsed results
Returns:	zip file bytes
Return type:	bytes

parsedmarc.human_timestamp_to_datetime(human_timestamp)[source]¶

Converts a human-readable timestamp into a Python DateTime object

Parameters:	human_timestamp (str) – A timestamp in YYYY-MM-DD HH:MM:SS` format
Returns:	The converted timestamp
Return type:	DateTime

parsedmarc.parse_aggregate_report_file(_input, nameservers=None, timeout=6.0)[source]¶

Parses a file at the given path, a file-like object. or bytes as a aggregate DMARC report

Parameters:	_input – A path to a file, a file like object, or bytes nameservers (list) – A list of one or more nameservers to use timeout (float) – Sets the DNS timeout in seconds
Returns:	The parsed DMARC aggregate report
Return type:	OrderedDict

parsedmarc.parse_aggregate_report_xml(xml, nameservers=None, timeout=6.0)[source]¶

Parses a DMARC XML report string and returns a consistent OrderedDict

Parameters:	xml (str) – A string of DMARC aggregate report XML nameservers (list) – A list of one or more nameservers to use timeout (float) – Sets the DNS timeout in seconds
Returns:	The parsed aggregate DMARC report
Return type:	OrderedDict

parsedmarc.parse_forensic_report(feedback_report, sample, sample_headers_only, nameservers=None, timeout=6.0)[source]¶

Converts a DMARC forensic report and sample to a OrderedDict

Parameters:	feedback_report – A message’s feedback report as a string sample – The RFC 822 headers or RFC 822 message sample sample_headers_only (bool) – Set true if the sample is only headers nameservers (list) – A list of one or more nameservers to use timeout (float) – Sets the DNS timeout in seconds
Returns:	An parsed report and sample
Return type:	OrderedDict

parsedmarc.parse_report_email(input_, nameservers=None, timeout=6.0)[source]¶

Parses a DMARC report from an email

Parameters:

input – An emailed DMARC report in RFC 822 format, as bytes or a string
nameservers (list) – A list of one or more nameservers to use
timeout (float) – Sets the DNS timeout in seconds

Returns:

report_type: aggregate or forensic
report: The parsed report

Return type:

OrderedDict

parsedmarc.parse_report_file(input_, nameservers=None, timeout=6.0)[source]¶

Parses a DMARC aggregate or forensic file at the given path, a file-like object. or bytes

Parameters:	input – A path to a file, a file like object, or bytes nameservers (list) – A list of one or more nameservers to use timeout (float) – Sets the DNS timeout in seconds
Returns:	The parsed DMARC report
Return type:	OrderedDict

parsedmarc.parsed_aggregate_report_to_csv(reports)[source]¶

Converts one or more parsed aggregate reports to flat CSV format, including headers

Parameters:	reports – A parsed aggregate report or list of parsed aggregate reports
Returns:	Parsed aggregate report data in flat CSV format, including headers
Return type:	str

parsedmarc.parsed_dmarc_forensic_reports_to_csv(reports)[source]¶

Converts one or more parsed forensic reports to flat CSV format, including headers

Parameters:	reports – A parsed forensic report or list of parsed forensic reports
Returns:	Parsed forensic report data in flat CSV format, including headers
Return type:	str

parsedmarc.save_output(results, output_directory='output')[source]¶

Save report data in the given directory

Parameters:	results (OrderedDict) – Parsing results output_directory – The patch to the directory to save in

parsedmarc.watch_inbox(host, username, password, callback, archive_folder='Archive', delete=False, test=False, wait=30, nameservers=None, dns_timeout=6.0)[source]¶

Use an IDLE IMAP connection to parse incoming emails, and pass the results to a callback function

Parameters:

host – The mail server hostname or IP address
username – The mail server username
password – The mail server password
callback – The callback function to receive the parsing results
archive_folder – The folder to move processed mail to
delete (bool) – Delete messages after processing them
test (bool) – Do not move or delete messages after processing them
wait (int) – Number of seconds to wait for a IMAP IDLE response
nameservers (list) – A list of DNS nameservers to query
dns_timeout (float) – Set the DNS query timeout

Welcome to parsedmarc’s documentation!¶

Features¶

CLI help¶

SPF and DMARC record validation¶

Sample aggregate report output¶

JSON¶

CSV¶

Sample forensic report output¶

Bug reports¶

Installation¶

Optional dependencies¶

API¶

Indices and tables¶