API reference
+parsedmarc
+A Python package for parsing DMARC reports
+-
+
- +exception parsedmarc.InvalidAggregateReport[source] +
Raised when an invalid DMARC aggregate report is encountered
+
-
+
- +exception parsedmarc.InvalidDMARCReport[source] +
Raised when an invalid DMARC report is encountered
+
-
+
- +exception parsedmarc.InvalidForensicReport[source] +
Raised when an invalid DMARC forensic report is encountered
+
-
+
- +parsedmarc.email_results(results, host, mail_from, mail_to, mail_cc=None, mail_bcc=None, port=0, require_encryption=False, verify=True, username=None, password=None, subject=None, attachment_filename=None, message=None)[source] +
Emails parsing results as a zip file
+-
+
- Parameters +
-
+
results (OrderedDict) – Parsing results
+host – Mail server hostname or IP address
+mail_from – The value of the message from header
+mail_to (list) – A list of addresses to mail to
+mail_cc (list) – A list of addresses to CC
+mail_bcc (list) – A list addresses to BCC
+port (int) – Port to use
+require_encryption (bool) – Require a secure connection from the start
+verify (bool) – verify the SSL/TLS certificate
+username (str) – An optional username
+password (str) – An optional password
+subject (str) – Overrides the default message subject
+attachment_filename (str) – Override the default attachment filename
+message (str) – Override the default plain text body
+
+
-
+
- +parsedmarc.extract_xml(input_)[source] +
Extracts xml from a zip or gzip file at the given path, file-like object, +or bytes.
+-
+
- Parameters +
input – A path to a file, a file like object, or bytes
+
+- Returns +
The extracted XML
+
+- Return type +
str
+
+
-
+
- +parsedmarc.get_dmarc_reports_from_mailbox(connection: MailboxConnection, reports_folder='INBOX', archive_folder='Archive', delete=False, test=False, ip_db_path=None, offline=False, nameservers=None, dns_timeout=6.0, strip_attachment_payloads=False, results=None, batch_size=10, create_folders=True)[source] +
Fetches and parses DMARC reports from a mailbox
+-
+
- Parameters +
-
+
connection – A Mailbox connection object
+reports_folder – The folder where reports can be found
+archive_folder – The folder to move processed mail to
+delete (bool) – Delete messages after processing them
+test (bool) – Do not move or delete messages after processing them
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not query online for geolocation or DNS
+nameservers (list) – A list of DNS nameservers to query
+dns_timeout (float) – Set the DNS query timeout
+strip_attachment_payloads (bool) – Remove attachment payloads from +forensic report results
+results (dict) – Results from the previous run
+batch_size (int) – Number of messages to read and process before saving +(use 0 for no limit)
+create_folders (bool) – Whether to create the destination folders +(not used in watch)
+
+- Returns +
Lists of
+aggregate_reportsandforensic_reports
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.get_dmarc_reports_from_mbox(input_, nameservers=None, dns_timeout=2.0, strip_attachment_payloads=False, ip_db_path=None, offline=False, parallel=False)[source] +
Parses a mailbox in mbox format containing e-mails with attached +DMARC reports
+-
+
- Parameters +
-
+
input – A path to a mbox file
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+dns_timeout (float) – Sets the DNS timeout in seconds
+strip_attachment_payloads (bool) – Remove attachment payloads from +forensic report results
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not make online queries for geolocation or DNS
+parallel (bool) – Parallel processing
+
+- Returns +
Lists of
+aggregate_reportsandforensic_reports
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.get_report_zip(results)[source] +
Creates a zip file of parsed report output
+-
+
- Parameters +
results (OrderedDict) – The parsed results
+
+- Returns +
zip file bytes
+
+- Return type +
bytes
+
+
-
+
- +parsedmarc.parse_aggregate_report_file(_input, offline=False, ip_db_path=None, nameservers=None, dns_timeout=2.0, parallel=False, keep_alive=None)[source] +
Parses a file at the given path, a file-like object. or bytes as an +aggregate DMARC report
+-
+
- Parameters +
-
+
_input – A path to a file, a file like object, or bytes
+offline (bool) – Do not query online for geolocation or DNS
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+dns_timeout (float) – Sets the DNS timeout in seconds
+parallel (bool) – Parallel processing
+keep_alive (callable) – Keep alive function
+
+- Returns +
The parsed DMARC aggregate report
+
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.parse_aggregate_report_xml(xml, ip_db_path=None, offline=False, nameservers=None, timeout=2.0, parallel=False, keep_alive=None)[source] +
Parses a DMARC XML report string and returns a consistent OrderedDict
+-
+
- Parameters +
-
+
xml (str) – A string of DMARC aggregate report XML
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not query online for geolocation or DNS
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+timeout (float) – Sets the DNS timeout in seconds
+parallel (bool) – Parallel processing
+keep_alive (callable) – Keep alive function
+
+- Returns +
The parsed aggregate DMARC report
+
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.parse_forensic_report(feedback_report, sample, msg_date, offline=False, ip_db_path=None, nameservers=None, dns_timeout=2.0, strip_attachment_payloads=False, parallel=False)[source] +
Converts a DMARC forensic report and sample to a
+OrderedDict-
+
- Parameters +
-
+
feedback_report (str) – A message’s feedback report as a string
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not query online for geolocation or DNS
+sample (str) – The RFC 822 headers or RFC 822 message sample
+msg_date (str) – The message’s date header
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+dns_timeout (float) – Sets the DNS timeout in seconds
+strip_attachment_payloads (bool) – Remove attachment payloads from +forensic report results
+parallel (bool) – Parallel processing
+
+- Returns +
A parsed report and sample
+
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.parse_report_email(input_, offline=False, ip_db_path=None, nameservers=None, dns_timeout=2.0, strip_attachment_payloads=False, parallel=False, keep_alive=None)[source] +
Parses a DMARC report from an email
+-
+
- Parameters +
-
+
input – An emailed DMARC report in RFC 822 format, as bytes or a string
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not query online for geolocation on DNS
+nameservers (list) – A list of one or more nameservers to use
+dns_timeout (float) – Sets the DNS timeout in seconds
+strip_attachment_payloads (bool) – Remove attachment payloads from +forensic report results
+parallel (bool) – Parallel processing
+keep_alive (callable) – keep alive function
+
+- Returns +
-
+
report_type:aggregateorforensic
+report: The parsed report
+
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.parse_report_file(input_, nameservers=None, dns_timeout=2.0, strip_attachment_payloads=False, ip_db_path=None, offline=False, parallel=False, keep_alive=None)[source] +
Parses a DMARC aggregate or forensic file at the given path, a +file-like object. or bytes
+-
+
- Parameters +
-
+
input – A path to a file, a file like object, or bytes
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+dns_timeout (float) – Sets the DNS timeout in seconds
+strip_attachment_payloads (bool) – Remove attachment payloads from +forensic report results
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not make online queries for geolocation or DNS
+parallel (bool) – Parallel processing
+keep_alive (callable) – Keep alive function
+
+- Returns +
The parsed DMARC report
+
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.parsed_aggregate_reports_to_csv(reports)[source] +
Converts one or more parsed aggregate reports to flat CSV format, including +headers
+-
+
- Parameters +
reports – A parsed aggregate report or list of parsed aggregate reports
+
+- Returns +
Parsed aggregate report data in flat CSV format, including headers
+
+- Return type +
str
+
+
-
+
- +parsedmarc.parsed_aggregate_reports_to_csv_rows(reports)[source] +
Converts one or more parsed aggregate reports to list of dicts in flat CSV +format
+-
+
- Parameters +
reports – A parsed aggregate report or list of parsed aggregate reports
+
+- Returns +
Parsed aggregate report data as a list of dicts in flat CSV +format
+
+- Return type +
list
+
+
-
+
- +parsedmarc.parsed_forensic_reports_to_csv(reports)[source] +
Converts one or more parsed forensic reports to flat CSV format, including +headers
+-
+
- Parameters +
reports – A parsed forensic report or list of parsed forensic reports
+
+- Returns +
Parsed forensic report data in flat CSV format, including headers
+
+- Return type +
str
+
+
-
+
- +parsedmarc.parsed_forensic_reports_to_csv_rows(reports)[source] +
Converts one or more parsed forensic reports to a list of dicts in flat CSV +format
+-
+
- Parameters +
reports – A parsed forensic report or list of parsed forensic reports
+
+- Returns +
Parsed forensic report data as a list of dicts in flat CSV format
+
+- Return type +
list
+
+
-
+
- +parsedmarc.save_output(results, output_directory='output', aggregate_json_filename='aggregate.json', forensic_json_filename='forensic.json', aggregate_csv_filename='aggregate.csv', forensic_csv_filename='forensic.csv')[source] +
Save report data in the given directory
+-
+
- Parameters +
-
+
results (OrderedDict) – Parsing results
+output_directory (str) – The path to the directory to save in
+aggregate_json_filename (str) – Filename for the aggregate JSON file
+forensic_json_filename (str) – Filename for the forensic JSON file
+aggregate_csv_filename (str) – Filename for the aggregate CSV file
+forensic_csv_filename (str) – Filename for the forensic CSV file
+
+
-
+
- +parsedmarc.watch_inbox(mailbox_connection: MailboxConnection, callback: Callable, reports_folder='INBOX', archive_folder='Archive', delete=False, test=False, check_timeout=30, ip_db_path=None, offline=False, nameservers=None, dns_timeout=6.0, strip_attachment_payloads=False, batch_size=None)[source] +
-
+
- Watches the mailbox for new messages and
sends the results to a callback function
+
+
-
+
- Parameters +
-
+
mailbox_connection – The mailbox connection object
+callback – The callback function to receive the parsing results
+reports_folder – The IMAP folder where reports can be found
+archive_folder – The folder to move processed mail to
+delete (bool) – Delete messages after processing them
+test (bool) – Do not move or delete messages after processing them
+check_timeout (int) – Number of seconds to wait for a IMAP IDLE response +or the number of seconds until the next mail check
+ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP
+offline (bool) – Do not query online for geolocation or DNS
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+dns_timeout (float) – Set the DNS query timeout
+strip_attachment_payloads (bool) – Replace attachment payloads in +forensic report samples with None
+batch_size (int) – Number of messages to read and process before saving
+
+
parsedmarc.elastic
+-
+
- +exception parsedmarc.elastic.AlreadySaved[source] +
Raised when a report to be saved matches an existing report
+
-
+
- +exception parsedmarc.elastic.ElasticsearchError[source] +
Raised when an Elasticsearch error occurs
+
-
+
- +parsedmarc.elastic.create_indexes(names, settings=None)[source] +
Create Elasticsearch indexes
+-
+
- Parameters +
-
+
names (list) – A list of index names
+settings (dict) – Index settings
+
+
-
+
- +parsedmarc.elastic.migrate_indexes(aggregate_indexes=None, forensic_indexes=None)[source] +
Updates index mappings
+-
+
- Parameters +
-
+
aggregate_indexes (list) – A list of aggregate index names
+forensic_indexes (list) – A list of forensic index names
+
+
-
+
- +parsedmarc.elastic.save_aggregate_report_to_elasticsearch(aggregate_report, index_suffix=None, monthly_indexes=False, number_of_shards=1, number_of_replicas=0)[source] +
Saves a parsed DMARC aggregate report to ElasticSearch
+-
+
- Parameters +
-
+
aggregate_report (OrderedDict) – A parsed forensic report
+index_suffix (str) – The suffix of the name of the index to save to
+monthly_indexes (bool) – Use monthly indexes instead of daily indexes
+number_of_shards (int) – The number of shards to use in the index
+number_of_replicas (int) – The number of replicas to use in the index
+
+- Raises +
- + +
-
+
- +parsedmarc.elastic.save_forensic_report_to_elasticsearch(forensic_report, index_suffix=None, monthly_indexes=False, number_of_shards=1, number_of_replicas=0)[source] +
Saves a parsed DMARC forensic report to ElasticSearch
+-
+
- Parameters +
-
+
forensic_report (OrderedDict) – A parsed forensic report
+index_suffix (str) – The suffix of the name of the index to save to
+monthly_indexes (bool) – Use monthly indexes instead of daily +indexes
+number_of_shards (int) – The number of shards to use in the index
+number_of_replicas (int) – The number of replicas to use in the +index
+
+- Raises +
- + +
-
+
- +parsedmarc.elastic.set_hosts(hosts, use_ssl=False, ssl_cert_path=None, username=None, password=None, timeout=60.0)[source] +
Sets the Elasticsearch hosts to use
+-
+
- Parameters +
-
+
hosts (str) – A single hostname or URL, or list of hostnames or URLs
+use_ssl (bool) – Use a HTTPS connection to the server
+ssl_cert_path (str) – Path to the certificate chain
+username (str) – The username to use for authentication
+password (str) – The password to use for authentication
+timeout (float) – Timeout in seconds
+
+
parsedmarc.splunk
+-
+
- +class parsedmarc.splunk.HECClient(url, access_token, index, source='parsedmarc', verify=True, timeout=60)[source] +
Initializes the HECClient
+-
+
- Parameters +
-
+
url (str) – The URL of the HEC
+access_token (str) – The HEC access token
+index (str) – The name of the index
+source (str) – The source name
+verify (bool) – Verify SSL certificates
+timeout (float) – Number of seconds to wait for the server to send +data before giving up
+
+
parsedmarc.utils
+Utility functions that might be useful for other projects
+-
+
- +exception parsedmarc.utils.DownloadError[source] +
Raised when an error occurs when downloading a file
+
-
+
- +exception parsedmarc.utils.EmailParserError[source] +
Raised when an error parsing the email occurs
+
-
+
- +parsedmarc.utils.convert_outlook_msg(msg_bytes)[source] +
Uses the
+msgconvertPerl utility to convert an Outlook MS file to +standard RFC 822 format-
+
- Parameters +
msg_bytes (bytes) – the content of the .msg file
+
+- Returns +
A RFC 822 string
+
+
-
+
- +parsedmarc.utils.decode_base64(data)[source] +
Decodes a base64 string, with padding being optional
+-
+
- Parameters +
data – A base64 encoded string
+
+- Returns +
The decoded bytes
+
+- Return type +
bytes
+
+
-
+
- +parsedmarc.utils.get_base_domain(domain, use_fresh_psl=False)[source] +
Gets the base domain name for the given domain
+++Note
+Results are based on a list of public domain suffixes at +https://publicsuffix.org/list/public_suffix_list.dat.
+-
+
- Parameters +
-
+
domain (str) – A domain or subdomain
+use_fresh_psl (bool) – Download a fresh Public Suffix List
+
+- Returns +
The base domain of the given domain
+
+- Return type +
str
+
+
-
+
- +parsedmarc.utils.get_filename_safe_string(string)[source] +
Converts a string to a string that is safe for a filename
+-
+
- Parameters +
string (str) – A string to make safe for a filename
+
+- Returns +
A string safe for a filename
+
+- Return type +
str
+
+
-
+
- +parsedmarc.utils.get_ip_address_country(ip_address, db_path=None)[source] +
Returns the ISO code for the country associated +with the given IPv4 or IPv6 address
+-
+
- Parameters +
-
+
ip_address (str) – The IP address to query for
+db_path (str) – Path to a MMDB file from MaxMind or DBIP
+
+- Returns +
And ISO country code associated with the given IP address
+
+- Return type +
str
+
+
-
+
- +parsedmarc.utils.get_ip_address_info(ip_address, ip_db_path=None, cache=None, offline=False, nameservers=None, timeout=2.0, parallel=False)[source] +
Returns reverse DNS and country information for the given IP address
+-
+
- Parameters +
-
+
ip_address (str) – The IP address to check
+ip_db_path (str) – path to a MMDB file from MaxMind or DBIP
+cache (ExpiringDict) – Cache storage
+offline (bool) – Do not make online queries for geolocation or DNS
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+timeout (float) – Sets the DNS timeout in seconds
+parallel (bool) – parallel processing
+
+- Returns +
+ip_address,reverse_dns
+- Return type +
OrderedDict
+
+
-
+
- +parsedmarc.utils.get_reverse_dns(ip_address, cache=None, nameservers=None, timeout=2.0)[source] +
Resolves an IP address to a hostname using a reverse DNS query
+-
+
- Parameters +
-
+
ip_address (str) – The IP address to resolve
+cache (ExpiringDict) – Cache storage
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+timeout (float) – Sets the DNS query timeout in seconds
+
+- Returns +
The reverse DNS hostname (if any)
+
+- Return type +
str
+
+
-
+
- +parsedmarc.utils.human_timestamp_to_datetime(human_timestamp, to_utc=False)[source] +
Converts a human-readable timestamp into a Python
+datetimeobject-
+
- Parameters +
-
+
human_timestamp (str) – A timestamp string
+to_utc (bool) – Convert the timestamp to UTC
+
+- Returns +
The converted timestamp
+
+- Return type +
datetime
+
+
-
+
- +parsedmarc.utils.human_timestamp_to_timestamp(human_timestamp)[source] +
Converts a human-readable timestamp into a UNIX timestamp
+-
+
- Parameters +
human_timestamp (str) – A timestamp in YYYY-MM-DD HH:MM:SS` format
+
+- Returns +
The converted timestamp
+
+- Return type +
float
+
+
-
+
- +parsedmarc.utils.is_mbox(path)[source] +
Checks if the given content is an MBOX mailbox file
+-
+
- Parameters +
path – Content to check
+
+- Returns +
A flag that indicates if the file is an MBOX mailbox file
+
+- Return type +
bool
+
+
-
+
- +parsedmarc.utils.is_outlook_msg(content)[source] +
Checks if the given content is an Outlook msg OLE/MSG file
+-
+
- Parameters +
content – Content to check
+
+- Returns +
A flag that indicates if the file is an Outlook MSG file
+
+- Return type +
bool
+
+
-
+
- +parsedmarc.utils.parse_email(data, strip_attachment_payloads=False)[source] +
A simplified email parser
+-
+
- Parameters +
-
+
data – The RFC 822 message string, or MSG binary
+strip_attachment_payloads (bool) – Remove attachment payloads
+
+- Returns +
Parsed email data
+
+- Return type +
dict
+
+
-
+
- +parsedmarc.utils.query_dns(domain, record_type, cache=None, nameservers=None, timeout=2.0)[source] +
Queries DNS
+-
+
- Parameters +
-
+
domain (str) – The domain or subdomain to query about
+record_type (str) – The record type to query for
+cache (ExpiringDict) – Cache storage
+nameservers (list) – A list of one or more nameservers to use +(Cloudflare’s public DNS resolvers by default)
+timeout (float) – Sets the DNS timeout in seconds
+
+- Returns +
A list of answers
+
+- Return type +
list
+
+
Indices and tables
+-
+
- +
- +
