From e5173a5e6f37b16d989ff0e00cd390455568c59b Mon Sep 17 00:00:00 2001 From: Sean Whalen Date: Sun, 14 May 2023 20:42:35 -0400 Subject: [PATCH] Update docs --- _modules/index.html | 2 +- _modules/parsedmarc.html | 10 ++++++++-- _modules/parsedmarc/elastic.html | 2 +- _modules/parsedmarc/splunk.html | 2 +- _modules/parsedmarc/utils.html | 2 +- _sources/elasticsearch.md.txt | 6 +++--- _sources/installation.md.txt | 7 +++---- _sources/kibana.md.txt | 2 +- _sources/mailing-lists.md.txt | 2 +- _sources/output.md.txt | 2 +- api.html | 2 +- contributing.html | 2 +- davmail.html | 2 +- dmarc.html | 8 ++++---- elasticsearch.html | 8 ++++---- genindex.html | 2 +- index.html | 2 +- installation.html | 9 ++++----- kibana.html | 4 ++-- mailing-lists.html | 6 +++--- output.html | 4 ++-- py-modindex.html | 2 +- search.html | 2 +- searchindex.js | 2 +- splunk.html | 2 +- usage.html | 2 +- 26 files changed, 50 insertions(+), 46 deletions(-) diff --git a/_modules/index.html b/_modules/index.html index 81976e8..8cbfe83 100644 --- a/_modules/index.html +++ b/_modules/index.html @@ -94,7 +94,7 @@
-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/_modules/parsedmarc.html b/_modules/parsedmarc.html index 0d60742..19ddbc8 100644 --- a/_modules/parsedmarc.html +++ b/_modules/parsedmarc.html @@ -335,7 +335,9 @@ "email"].split("@")[-1] org_name = report_metadata["org_name"] if org_name is not None and " " not in org_name: - org_name = get_base_domain(org_name) + new_org_name = get_base_domain(org_name) + if new_org_name is not None: + org_name = new_org_name new_report_metadata["org_name"] = org_name new_report_metadata["org_email"] = report_metadata["email"] extra = None @@ -348,6 +350,10 @@ "").replace(">", "").split("@")[0] new_report_metadata["report_id"] = report_id date_range = report["report_metadata"]["date_range"] + if (int(date_range["end"]) - int(date_range["begin"]) > 2*86400): + raise InvalidAggregateReport("The begin and end fields span too \ + many hours, should be max 24 hours \ + according to RFC 7489 section 7.2") date_range["begin"] = timestamp_to_human(date_range["begin"]) date_range["end"] = timestamp_to_human(date_range["end"]) new_report_metadata["begin_date"] = date_range["begin"] @@ -1543,7 +1549,7 @@
-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/_modules/parsedmarc/elastic.html b/_modules/parsedmarc/elastic.html index 893d7d4..60e858e 100644 --- a/_modules/parsedmarc/elastic.html +++ b/_modules/parsedmarc/elastic.html @@ -639,7 +639,7 @@
-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/_modules/parsedmarc/splunk.html b/_modules/parsedmarc/splunk.html index 6e39dab..9243aad 100644 --- a/_modules/parsedmarc/splunk.html +++ b/_modules/parsedmarc/splunk.html @@ -250,7 +250,7 @@
-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/_modules/parsedmarc/utils.html b/_modules/parsedmarc/utils.html index d51dc79..c1a02b3 100644 --- a/_modules/parsedmarc/utils.html +++ b/_modules/parsedmarc/utils.html @@ -646,7 +646,7 @@
-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/_sources/elasticsearch.md.txt b/_sources/elasticsearch.md.txt index 1c38f1e..856cf09 100644 --- a/_sources/elasticsearch.md.txt +++ b/_sources/elasticsearch.md.txt @@ -29,7 +29,7 @@ which will cause it to crash under a heavy load. To fix this, increase the minimum and maximum JVM heap sizes in `/etc/elasticsearch/jvm.options` to more reasonable levels, depending on your server's resources. -Make sure the system has at least 2 GB more RAM then the assigned JVM +Make sure the system has at least 2 GB more RAM than the assigned JVM heap size. Always set the minimum and maximum JVM heap sizes to the same @@ -180,7 +180,7 @@ sudo systemctl restart elasticsearch Now that Elasticsearch is up and running, use `parsedmarc` to send data to it. -Download (right click the link and click save as) [export.ndjson]. +Download (right-click the link and click save as) [export.ndjson]. Connect to kibana using the "elastic" user and the password you previously provide on the console ("End Kibana configuration" part). @@ -219,7 +219,7 @@ Kibana index patterns with versions that match the upgraded indexes: index patterns 4. Click Delete 5. Click Delete on the conformation message -6. Download (right click the link and click save as) +6. Download (right-click the link and click save as) the latest version of [export.ndjson] 7. Import `export.ndjson` by clicking Import from the Kibana Saved Objects page diff --git a/_sources/installation.md.txt b/_sources/installation.md.txt index cea325d..c34abfd 100644 --- a/_sources/installation.md.txt +++ b/_sources/installation.md.txt @@ -28,8 +28,7 @@ https_proxy=https://prox-server:3128 ftp_proxy=http://prox-server:3128 ``` -This will set the the proxy up for use system-wide, including for -`parsedmarc`. +This will set the proxy up for use system-wide, including for `parsedmarc`. ::: :::{warning} @@ -49,7 +48,7 @@ Starting in `parsedmarc` 7.1.0, a static copy of the `parsedmarc`, under the terms of the [Creative Commons Attribution 4.0 International License]. as a fallback if the [MaxMind GeoLite2 Country database] is not -installed However, `parsedmarc` cannot install updated versions of +installed. However, `parsedmarc` cannot install updated versions of these databases as they are released, so MaxMind's databases and the [geoipupdate] tool is still the preferable solution. @@ -91,7 +90,7 @@ regulations][to comply with various privacy regulations]. Start by [registering for a free GeoLite2 account], and signing in. -Then, navigate the to the [License Keys] page under your account, +Then, navigate to the [License Keys] page under your account, and create a new license key for the version of `geoipupdate` that was installed. diff --git a/_sources/kibana.md.txt b/_sources/kibana.md.txt index 67c224e..bf2cf1a 100644 --- a/_sources/kibana.md.txt +++ b/_sources/kibana.md.txt @@ -6,7 +6,7 @@ results from incoming DMARC reports. :::{note} The default dashboard is DMARC Summary. To switch between dashboards, -click on the Dashboard link in the left side menu of Kibana. +click on the Dashboard link on the left side menu of Kibana. ::: ## DMARC Summary diff --git a/_sources/mailing-lists.md.txt b/_sources/mailing-lists.md.txt index ede5c8a..71b8975 100644 --- a/_sources/mailing-lists.md.txt +++ b/_sources/mailing-lists.md.txt @@ -6,7 +6,7 @@ list is spoofing your from address, and modifying the subject, footer, or other part of the message, thereby breaking the DKIM signature. -### Mailing list list best practices +### Mailing list best practices Ideally, a mailing list should forward messages without altering the headers or body content at all. [Joe Nelson] does a fantastic job of diff --git a/_sources/output.md.txt b/_sources/output.md.txt index 4838ed3..f676891 100644 --- a/_sources/output.md.txt +++ b/_sources/output.md.txt @@ -89,7 +89,7 @@ draft,acme.com,noreply-dmarc-support@acme.com,http://acme.com/dmarc/support,9391 ## Sample forensic report output -Thanks to Github user [xennn](https://github.com/xennn) for the anonymized +Thanks to GitHub user [xennn](https://github.com/xennn) for the anonymized [forensic report email sample](). ### JSON forensic report diff --git a/api.html b/api.html index b77bcc9..0024b49 100644 --- a/api.html +++ b/api.html @@ -1045,7 +1045,7 @@ with the given IPv4 or IPv6 address

-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/contributing.html b/contributing.html index b5fe68d..fd70334 100644 --- a/contributing.html +++ b/contributing.html @@ -107,7 +107,7 @@
-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/davmail.html b/davmail.html index 1861545..0b628b1 100644 --- a/davmail.html +++ b/davmail.html @@ -266,7 +266,7 @@ config file:

-

© Copyright 2018, Sean Whalen.

+

© Copyright 2018 - 2023, Sean Whalen and contributors.

Built with Sphinx using a diff --git a/dmarc.html b/dmarc.html index 4ab526e..35af80a 100644 --- a/dmarc.html +++ b/dmarc.html @@ -65,7 +65,7 @@
  • DMARC Alignment Guide
  • What if a sender won’t support DKIM/DMARC?
  • What about mailing lists?
      -
    • Mailing list list best practices
        +
      • Mailing list best practices
        • Do
        • Do not
        • Mailman 2
          • @@ -203,8 +203,8 @@ relayed by mailing lists are failing DMARC, most likely because the mailing list is spoofing your from address, and modifying the subject, footer, or other part of the message, thereby breaking the DKIM signature.

          -
          -

          Mailing list list best practices

          +
          +

          Mailing list best practices

          Ideally, a mailing list should forward messages without altering the headers or body content at all. Joe Nelson does a fantastic job of explaining exactly what mailing lists should and shouldn’t do to be @@ -485,7 +485,7 @@ the original sender.

          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/elasticsearch.html b/elasticsearch.html index 0db2a59..16e4652 100644 --- a/elasticsearch.html +++ b/elasticsearch.html @@ -115,7 +115,7 @@ sudo apt-get install/etc/elasticsearch/jvm.options to more reasonable levels, depending on your server’s resources.

          -

          Make sure the system has at least 2 GB more RAM then the assigned JVM +

          Make sure the system has at least 2 GB more RAM than the assigned JVM heap size.

          Always set the minimum and maximum JVM heap sizes to the same value.

          @@ -229,7 +229,7 @@ sudo systemctl restart

          Now that Elasticsearch is up and running, use parsedmarc to send data to it.

          -

          Download (right click the link and click save as) export.ndjson.

          +

          Download (right-click the link and click save as) export.ndjson.

          Connect to kibana using the “elastic” user and the password you previously provide on the console (“End Kibana configuration” part).

          Import export.ndjson the Saved Objects tab of the Stack management @@ -254,7 +254,7 @@ Kibana index patterns with versions that match the upgraded indexes:

          index patterns

        • Click Delete

        • Click Delete on the conformation message

          • -

        • Download (right click the link and click save as) +

        • Download (right-click the link and click save as) the latest version of export.ndjson

        • Import export.ndjson by clicking Import from the Kibana Saved Objects page

          • @@ -280,7 +280,7 @@ check out the Elastic guide to -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with           Sphinx using a diff --git a/genindex.html b/genindex.html index 769da8a..a213046 100644 --- a/genindex.html +++ b/genindex.html @@ -341,7 +341,7 @@
          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/index.html b/index.html index 2f620ac..373ab25 100644 --- a/index.html +++ b/index.html @@ -187,7 +187,7 @@ premade dashboards

          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/installation.html b/installation.html index ae191af..a126edb 100644 --- a/installation.html +++ b/installation.html @@ -115,8 +115,7 @@ proxy details there, for example:

          ftp_proxy=http://prox-server:3128 -

          This will set the the proxy up for use system-wide, including for -parsedmarc.

          +

          This will set the proxy up for use system-wide, including for parsedmarc.

          @@ -139,7 +138,7 @@ least:

          parsedmarc, under the terms of the [Creative Commons Attribution 4.0 International License]. as a fallback if the MaxMind GeoLite2 Country database is not -installed However, parsedmarc cannot install updated versions of +installed. However, parsedmarc cannot install updated versions of these databases as they are released, so MaxMind’s databases and the geoipupdate tool is still the preferable solution.

          The location of the database file can be overridden by using the @@ -170,7 +169,7 @@ access the free Geolite2 databases, in order .

          Start by registering for a free GeoLite2 account, and signing in.

          -

          Then, navigate the to the License Keys page under your account, +

          Then, navigate to the License Keys page under your account, and create a new license key for the version of geoipupdate that was installed.

          @@ -259,7 +258,7 @@ Outlook (i.e. OLE .msg files), install -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/kibana.html b/kibana.html index f4c8bfc..97ade38 100644 --- a/kibana.html +++ b/kibana.html @@ -95,7 +95,7 @@ results from incoming DMARC reports.

          Note

          The default dashboard is DMARC Summary. To switch between dashboards, -click on the Dashboard link in the left side menu of Kibana.

          +click on the Dashboard link on the left side menu of Kibana.

          DMARC Summary

          @@ -179,7 +179,7 @@ supply the headers of sample emails. Very few provide the entire email.

          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/mailing-lists.html b/mailing-lists.html index 8d950b5..4508f0b 100644 --- a/mailing-lists.html +++ b/mailing-lists.html @@ -89,8 +89,8 @@ relayed by mailing lists are failing DMARC, most likely because the mailing list is spoofing your from address, and modifying the subject, footer, or other part of the message, thereby breaking the DKIM signature.

          -
          -

          Mailing list list best practices

          +
          +

          Mailing list best practices

          Ideally, a mailing list should forward messages without altering the headers or body content at all. Joe Nelson does a fantastic job of explaining exactly what mailing lists should and shouldn’t do to be @@ -367,7 +367,7 @@ the original sender.

          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/output.html b/output.html index d588f03..84f5286 100644 --- a/output.html +++ b/output.html @@ -186,7 +186,7 @@ draft,acme.com,noreply-dmarc-support@acme.com,http://acme.com/dmarc/support,9391

          Sample forensic report output

          -

          Thanks to Github user xennn for the anonymized +

          Thanks to GitHub user xennn for the anonymized forensic report email sample.

          JSON forensic report

          @@ -298,7 +298,7 @@ auth-failure,Lua/1.0,1.0,,sharepoint@domain.de,peter.pan@domain.de,"Mon, 01
          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/py-modindex.html b/py-modindex.html index d4d419d..9ff29ea 100644 --- a/py-modindex.html +++ b/py-modindex.html @@ -125,7 +125,7 @@
          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/search.html b/search.html index 60c3cdd..4fbe14b 100644 --- a/search.html +++ b/search.html @@ -103,7 +103,7 @@
          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/searchindex.js b/searchindex.js index 8f43470..d7d68e1 100644 --- a/searchindex.js +++ b/searchindex.js @@ -1 +1 @@ -Search.setIndex({"docnames": ["api", "contributing", "davmail", "dmarc", "elasticsearch", "index", "installation", "kibana", "mailing-lists", "output", "splunk", "usage"], "filenames": ["api.md", "contributing.md", "davmail.md", "dmarc.md", "elasticsearch.md", "index.md", "installation.md", "kibana.md", "mailing-lists.md", "output.md", "splunk.md", "usage.md"], "titles": ["API reference", "Contributing to parsedmarc", "Accessing an inbox using OWA/EWS", "Understanding DMARC", "Elasticsearch and Kibana", "parsedmarc documentation - Open source DMARC report analyzer and visualizer", "Installation", "Using the Kibana dashboards", "What about mailing lists?", "Sample outputs", "Splunk", "Using parsedmarc"], "terms": {"A": [0, 3, 11], "python": [0, 5, 6], "packag": [0, 4], "pars": [0, 3, 5, 6, 9, 11], "dmarc": [0, 4, 6, 8, 9, 10, 11], "report": [0, 4, 7, 10, 11], "except": [0, 11], "invalidaggregatereport": 0, "sourc": [0, 3, 4, 6, 7, 9], "rais": 0, "when": [0, 3, 5, 7, 8, 11], "an": [0, 3, 5, 7, 8, 9, 11], "invalid": 0, "aggreg": [0, 5, 7, 10, 11], "i": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11], "encount": 0, "invaliddmarcreport": 0, "invalidforensicreport": 0, "forens": [0, 5, 10, 11], "parsererror": 0, "whenev": [0, 2, 11], "parser": 0, "fail": [0, 3, 7, 8, 9, 11], "some": [0, 2, 3, 4, 7, 8], "reason": [0, 2, 4, 11], "email_result": 0, "result": [0, 5, 7, 9, 11], "host": [0, 2, 3, 4, 5, 8, 11], "mail_from": 0, "mail_to": 0, "mail_cc": 0, "none": [0, 3, 9, 11], "mail_bcc": 0, "port": [0, 2, 11], "0": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11], "require_encrypt": 0, "fals": [0, 2, 6, 9, 11], "verifi": 0, "true": [0, 2, 4, 9, 11], "usernam": [0, 11], "password": [0, 4, 6, 11], "subject": [0, 3, 8, 9, 11], "attachment_filenam": 0, "messag": [0, 2, 3, 4, 6, 7, 8, 9, 11], "email": [0, 3, 5, 6, 7, 8, 9, 10, 11], "zip": [0, 2, 5, 11], "file": [0, 2, 5, 6, 10], "paramet": 0, "ordereddict": 0, "mail": [0, 5, 6, 9, 11], "server": [0, 2, 3, 4, 6, 7, 9, 11], "hostnam": [0, 11], "ip": [0, 3, 4, 6, 7, 11], "address": [0, 2, 3, 4, 7, 8, 9, 11], "The": [0, 3, 6, 7, 10, 11], "valu": [0, 3, 4, 7, 8, 11], "from": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11], "header": [0, 3, 7, 8, 9, 11], "list": [0, 2, 4, 5, 7, 11], "cc": [0, 9], "bcc": [0, 9], "int": [0, 11], "us": [0, 3, 4, 5, 6, 8, 9], "bool": [0, 11], "requir": [0, 2, 3, 4, 6, 8, 11], "secur": [0, 4, 11], "connect": [0, 2, 4, 11], "start": [0, 2, 4, 6, 7, 10, 11], "ssl": [0, 2, 4, 11], "tl": [0, 11], "certif": [0, 4, 11], "str": [0, 11], "option": [0, 2, 3, 4, 5, 8, 10, 11], "overrid": 0, "default": [0, 2, 4, 6, 7, 11], "attach": [0, 3, 8, 9, 11], "filenam": [0, 11], "plain": 0, "text": [0, 9], "bodi": [0, 3, 8, 9, 11], "extract_xml": 0, "input_": 0, "extract": [0, 2], "xml": [0, 10], "gzip": [0, 5], "given": [0, 11], "path": [0, 4, 11], "like": [0, 3, 6, 8], "object": [0, 4], "byte": 0, "input": 0, "return": 0, "type": [0, 9, 11], "get_dmarc_reports_from_mailbox": 0, "mailboxconnect": 0, "reports_fold": [0, 11], "inbox": [0, 3, 5, 8, 11], "archive_fold": [0, 11], "archiv": [0, 11], "delet": [0, 2, 4, 11], "test": [0, 5, 11], "ip_db_path": [0, 6, 11], "offlin": [0, 11], "nameserv": [0, 11], "dns_timeout": [0, 11], "6": [0, 4, 6, 11], "strip_attachment_payload": [0, 11], "batch_siz": [0, 11], "10": [0, 6, 9, 11], "create_fold": 0, "fetch": 0, "mailbox": [0, 7, 11], "folder": [0, 2, 11], "where": [0, 2, 3, 8, 11], "can": [0, 2, 3, 4, 5, 6, 7, 8, 11], "found": [0, 6, 11], "move": [0, 4, 11], "process": [0, 2, 5, 6, 11], "after": [0, 2, 4, 11], "them": [0, 4, 7, 11], "do": [0, 2, 6, 7, 11], "mmdb": [0, 11], "maxmind": [0, 6, 11], "dbip": [0, 11], "queri": [0, 11], "onlin": [0, 2, 11], "geoloc": [0, 11], "dn": [0, 3, 7, 11], "float": [0, 11], "set": [0, 2, 3, 4, 6, 7, 8, 11], "timeout": [0, 2, 11], "remov": [0, 3, 4, 8, 11], "payload": [0, 11], "dict": 0, "previou": [0, 2, 4, 11], "run": [0, 4, 5, 6], "number": [0, 11], "read": [0, 11], "befor": [0, 11], "save": [0, 4, 6, 11], "limit": [0, 2, 11], "whether": 0, "creat": [0, 2, 3, 4, 6, 8, 11], "destin": 0, "watch": [0, 2, 4, 11], "aggregate_report": 0, "forensic_report": 0, "get_dmarc_reports_from_mbox": 0, "2": [0, 4, 9, 11], "parallel": [0, 11], "mbox": [0, 11], "format": [0, 6], "contain": [0, 7, 10, 11], "e": [0, 2, 3, 4, 6, 8, 11], "one": [0, 3, 5, 8, 11], "more": [0, 4, 6, 10, 11], "cloudflar": [0, 11], "": [0, 2, 3, 4, 6, 8, 9, 11], "public": [0, 3, 9, 11], "resolv": [0, 11], "second": [0, 2, 11], "make": [0, 3, 4, 8, 11], "get_report_zip": 0, "output": [0, 5, 11], "parse_aggregate_report_fil": 0, "_input": 0, "keep_al": 0, "callabl": 0, "keep": 0, "aliv": 0, "function": 0, "parse_aggregate_report_xml": 0, "string": 0, "consist": [0, 5, 9], "parse_forensic_report": 0, "feedback_report": 0, "sampl": [0, 5, 11], "msg_date": 0, "convert": [0, 3, 8], "feedback": 0, "rfc": [0, 3, 8, 9], "822": 0, "date": [0, 3, 8, 9], "parse_report_email": 0, "report_typ": 0, "parse_report_fil": 0, "parsed_aggregate_reports_to_csv": 0, "flat": 0, "csv": [0, 5, 11], "includ": [0, 3, 6, 7, 8, 11], "data": [0, 4, 5, 7, 10, 11], "parsed_aggregate_reports_to_csv_row": 0, "parsed_forensic_reports_to_csv": 0, "parsed_forensic_reports_to_csv_row": 0, "save_output": 0, "output_directori": 0, "aggregate_json_filenam": [0, 11], "json": [0, 5, 11], "forensic_json_filenam": [0, 11], "aggregate_csv_filenam": [0, 11], "forensic_csv_filenam": [0, 11], "directori": [0, 11], "watch_inbox": 0, "mailbox_connect": 0, "callback": 0, "check_timeout": [0, 11], "30": [0, 11], "new": [0, 2, 3, 6, 7, 11], "send": [0, 2, 3, 4, 5, 7, 8, 10, 11], "receiv": [0, 9, 11], "imap": [0, 2, 5, 11], "wait": [0, 11], "idl": [0, 2, 11], "respons": [0, 11], "until": [0, 11], "next": [0, 11], "check": [0, 2, 3, 4, 6, 11], "replac": [0, 3, 4, 8], "alreadysav": 0, "match": [0, 4, 10], "exist": [0, 3, 4, 8], "elasticsearcherror": 0, "elasticsearch": [0, 5, 11], "error": [0, 9, 11], "occur": [0, 7], "create_index": 0, "name": [0, 3, 4, 7, 9, 10, 11], "index": [0, 5, 10, 11], "migrate_index": 0, "aggregate_index": 0, "forensic_index": 0, "updat": [0, 4, 6, 11], "map": 0, "save_aggregate_report_to_elasticsearch": 0, "index_suffix": [0, 11], "monthly_index": [0, 11], "number_of_shard": [0, 11], "1": [0, 2, 4, 5, 6, 9, 11], "number_of_replica": [0, 11], "suffix": [0, 11], "monthli": [0, 11], "instead": [0, 3, 6, 8, 11], "daili": [0, 11], "shard": [0, 11], "replica": [0, 11], "save_forensic_report_to_elasticsearch": 0, "set_host": 0, "use_ssl": 0, "ssl_cert_path": 0, "60": 0, "singl": 0, "url": [0, 2, 11], "http": [0, 1, 2, 3, 4, 6, 8, 9, 10, 11], "chain": 0, "authent": [0, 2, 3, 4, 7, 11], "class": 0, "hecclient": 0, "access_token": 0, "initi": 0, "hec": [0, 10, 11], "access": [0, 4, 5, 6, 11], "token": [0, 4, 11], "give": [0, 4, 11], "up": [0, 2, 4, 6, 7], "save_aggregate_reports_to_splunk": 0, "dictionari": 0, "save_forensic_reports_to_splunk": 0, "splunkerror": 0, "might": [0, 3, 7, 8], "other": [0, 3, 4, 7, 8], "project": [0, 2, 3, 5, 10], "downloaderror": 0, "download": [0, 2, 4, 6], "emailparsererror": 0, "convert_outlook_msg": 0, "msg_byte": 0, "msgconvert": [0, 6], "perl": [0, 6], "outlook": [0, 2, 6], "m": [0, 6, 9, 11], "standard": [0, 5, 9], "content": [0, 3, 8, 9, 10], "msg": [0, 6], "decode_base64": 0, "decod": 0, "base64": 0, "pad": 0, "being": 0, "encod": [0, 9, 11], "get_base_domain": 0, "domain": [0, 4, 7, 8, 9], "get": [0, 2, 4, 6, 11], "base": [0, 2, 3, 4, 7, 8, 9], "ar": [0, 2, 3, 4, 6, 7, 8, 9, 11], "publicsuffix": 0, "org": [0, 6, 9], "public_suffix_list": 0, "dat": 0, "subdomain": [0, 3], "get_filename_safe_str": 0, "safe": 0, "get_ip_address_countri": 0, "ip_address": [0, 9], "db_path": 0, "iso": 0, "code": [0, 4, 5], "countri": [0, 6, 7, 9], "associ": 0, "ipv4": 0, "ipv6": 0, "And": 0, "get_ip_address_info": 0, "cach": [0, 11], "revers": [0, 7], "inform": [0, 4, 6, 7, 11], "expiringdict": 0, "storag": [0, 11], "reverse_dn": [0, 9], "get_reverse_dn": 0, "ani": [0, 3, 7, 8, 11], "human_timestamp_to_datetim": 0, "human_timestamp": 0, "to_utc": 0, "human": [0, 7], "readabl": 0, "timestamp": 0, "datetim": 0, "utc": 0, "human_timestamp_to_timestamp": 0, "unix": 0, "yyyi": 0, "mm": 0, "dd": 0, "hh": 0, "ss": 0, "is_mbox": 0, "flag": [0, 2], "is_outlook_msg": 0, "ol": [0, 6], "parse_email": 0, "simplifi": 0, "binari": 0, "query_dn": 0, "record_typ": 0, "about": [0, 5, 6], "record": [0, 5, 6, 9], "answer": [0, 11], "timestamp_to_datetim": 0, "timestamp_to_human": 0, "modul": [0, 5, 11], "pleas": [1, 5, 11], "github": [1, 6, 9, 11], "issu": [1, 5], "tracker": 1, "com": [1, 2, 3, 8, 9, 11], "domainawar": [1, 3, 11], "8": [2, 4, 6, 9, 11], "support": [2, 4, 5, 9, 10], "microsoft": [2, 5, 6, 9, 11], "offic": 2, "365": [2, 4], "via": 2, "graph": [2, 5, 7, 11], "api": [2, 4, 5], "which": [2, 4, 7, 11], "prefer": [2, 6], "over": [2, 5, 7], "organ": [2, 7, 11], "allow": [2, 3, 8, 11], "onli": [2, 3, 6, 7, 8, 11], "exchang": [2, 6, 9, 11], "web": [2, 4, 6], "In": [2, 3, 7, 8, 11], "case": [2, 3, 8], "need": [2, 3, 4, 6, 7, 8, 11], "local": [2, 4, 9, 11], "gatewai": 2, "It": [2, 4, 7, 9, 11], "even": [2, 3, 8, 11], "work": [2, 3, 5, 6, 7, 8], "modern": [2, 3, 8], "auth": [2, 9, 11], "multi": [2, 11], "factor": 2, "To": [2, 4, 6, 7, 9, 11], "thi": [2, 3, 4, 5, 6, 7, 8, 9, 11], "latest": [2, 4, 6], "version": [2, 4, 6, 9, 10, 11], "sourceforg": 2, "net": [2, 9], "unzip": 2, "command": [2, 3, 8, 11], "instal": [2, 4, 5, 11], "java": 2, "sudo": [2, 4, 6, 11], "apt": [2, 4, 6], "jre": 2, "headless": 2, "properti": 2, "see": [2, 3, 4, 5, 7, 11], "document": [2, 11], "basic": 2, "workstat": 2, "mode": [2, 4, 11], "auto": 2, "webdav": 2, "enableew": 2, "office365": 2, "asmx": 2, "listen": [2, 11], "imapport": 2, "1143": 2, "network": [2, 4, 11], "proxi": [2, 6], "enableproxi": 2, "usesystemproxi": 2, "proxyhost": 2, "proxyport": 2, "proxyus": 2, "proxypassword": 2, "exclud": 2, "noproxyfor": 2, "block": [2, 11], "remot": 2, "allowremot": 2, "bind": 2, "socket": 2, "loopback": 2, "bindaddress": 2, "127": [2, 4, 11], "disabl": [2, 11], "specifi": [2, 3], "nosecureimap": 2, "keepal": 2, "charact": [2, 11], "dure": 2, "larg": 2, "enablekeepal": 2, "count": [2, 9], "retriev": 2, "foldersizelimit": 2, "immedi": 2, "store": [2, 4], "imapautoexpung": 2, "enabl": [2, 4, 11], "poll": [2, 11], "delai": [2, 9], "minut": [2, 11], "imapidledelai": 2, "alwai": [2, 4, 11], "repli": [2, 3, 8], "rfc822": 2, "size": [2, 4], "request": [2, 4, 11], "approxim": 2, "perform": [2, 11], "imapalwaysapproxmsgs": 2, "client": [2, 3, 4, 8, 11], "300": 2, "clientsotimeout": 2, "system": [2, 3, 4, 6, 8, 11], "user": [2, 3, 4, 5, 6, 8, 9, 11], "useradd": [2, 6], "r": [2, 6, 9, 11], "bin": [2, 4, 6, 11], "protect": [2, 3, 5, 8, 11], "pry": [2, 11], "ey": [2, 11], "chown": [2, 11], "root": [2, 11], "opt": [2, 6, 11], "chmod": [2, 4, 11], "u": [2, 6, 9, 11], "rw": [2, 11], "g": [2, 3, 4, 8, 11], "o": [2, 4, 11], "nano": [2, 11], "etc": [2, 3, 4, 6, 8, 11], "unit": [2, 11], "descript": [2, 6, 11], "want": [2, 5, 11], "target": [2, 11], "syslog": [2, 11], "execstart": [2, 11], "group": [2, 7, 11], "restart": [2, 3, 4, 8, 11], "restartsec": [2, 11], "5m": [2, 11], "wantedbi": [2, 11], "Then": [2, 3, 4, 6, 8, 11], "systemctl": [2, 4, 11], "daemon": [2, 4, 11], "reload": [2, 4, 11], "you": [2, 3, 4, 5, 6, 7, 8, 11], "must": [2, 3, 8, 11], "also": [2, 3, 4, 7, 8, 11], "abov": [2, 11], "edit": [2, 6, 11], "everi": [2, 6, 11], "time": [2, 4, 6, 7, 11], "upgrad": [2, 5, 6, 11], "statu": [2, 11], "event": [2, 10, 11], "crash": [2, 4, 11], "5": [2, 4], "show": [2, 7, 11], "log": [2, 11], "current": [2, 4, 11], "vew": 2, "well": [2, 11], "newest": [2, 11], "oldest": [2, 11], "journalctl": [2, 11], "becaus": [2, 3, 7, 8, 11], "interact": [2, 4], "add": [2, 3, 4, 6, 7, 8, 11], "follow": [2, 4], "ini": [2, 11], "config": [2, 6, 11], "demystifi": 3, "complet": [3, 4], "If": [3, 4, 6, 7, 8, 11], "look": [3, 7], "out": [3, 4, 7], "sister": 3, "checkdmarc": 3, "against": [3, 8], "spoof": [3, 8], "open": 3, "monitor": [3, 11], "ensur": [3, 6, 8], "dkm": 3, "mechan": 3, "actual": [3, 9], "same": [3, 4, 6, 7, 10], "end": [3, 4], "pass": [3, 7, 9], "long": 3, "relat": 3, "indic": [3, 5], "signatur": [3, 7, 8], "kei": [3, 4, 6, 11], "publish": 3, "smtp": [3, 7, 9, 11], "envelop": 3, "sign": [3, 4, 6], "vendor": 3, "don": 3, "know": 3, "yet": 3, "ask": 3, "thei": [3, 6, 7, 8, 11], "through": 3, "your": [3, 4, 6, 7, 8, 10, 11], "relai": [3, 8], "theirs": 3, "realli": 3, "why": [3, 7], "displai": [3, 7, 10], "worst": 3, "have": [3, 4, 6, 7, 8, 10, 11], "specif": [3, 11], "norepli": [3, 9], "exampl": [3, 4, 6, 8, 9, 11], "separ": [3, 4, 6, 7, 10, 11], "p": [3, 6, 9], "alter": [3, 8], "sp": [3, 9], "top": [3, 7], "level": [3, 4], "tld": 3, "would": [3, 5, 6, 8], "leav": 3, "vulner": 3, "deploi": [3, 8], "find": [3, 7, 8], "most": [3, 4, 7, 8, 11], "modifi": [3, 8, 11], "footer": [3, 8], "part": [3, 4, 7, 8], "therebi": [3, 8], "break": [3, 4, 8], "ideal": [3, 8], "should": [3, 6, 7, 8, 11], "forward": [3, 7, 8], "without": [3, 4, 7, 8], "all": [3, 5, 7, 8, 10, 11], "joe": [3, 8], "nelson": [3, 8], "doe": [3, 8], "fantast": [3, 8], "job": [3, 6, 8], "explain": [3, 8], "exactli": [3, 8], "shouldn": [3, 8], "fulli": [3, 8], "compliant": [3, 8], "rather": [3, 8], "than": [3, 8, 11], "repeat": [3, 8], "hi": [3, 8], "fine": [3, 8], "here": [3, 8, 9, 11], "summari": [3, 5, 8], "retain": [3, 8], "origin": [3, 8, 11], "2369": [3, 8], "unsubscrib": [3, 8], "outgo": [3, 8, 11], "ad": [3, 6, 8, 11], "link": [3, 4, 7, 8], "2919": [3, 8], "id": [3, 8, 9, 11], "webmail": [3, 7, 8], "servic": [3, 4, 5, 7, 8], "gener": [3, 4, 6, 8, 9, 11], "button": [3, 8], "tradit": [3, 8], "disclaim": [3, 8], "addit": [3, 8], "compli": [3, 4, 6, 8], "configur": [3, 4, 5, 6, 7, 8], "action": [3, 8], "prefix": [3, 8], "still": [3, 6, 8, 9, 11], "tell": [3, 6, 7, 8], "came": [3, 8], "wa": [3, 4, 6, 8], "sent": [3, 8, 11], "post": [3, 8], "step": [3, 4, 8], "common": [3, 4, 6, 8], "platform": [3, 8], "below": [3, 8, 11], "navig": [3, 6, 8], "subject_prefix": [3, 8], "from_is_list": [3, 8], "No": [3, 8], "first_strip_reply_to": [3, 8], "reply_goes_to_list": [3, 8], "poster": [3, 8], "include_rfc2369_head": [3, 8], "ye": [3, 8], "include_list_post_head": [3, 8], "include_sender_head": [3, 8], "non": [3, 8, 11], "digest": [3, 8], "msg_header": [3, 8], "msg_footer": [3, 8], "scrub_nondigest": [3, 8], "privaci": [3, 6, 7, 8, 11], "filter": [3, 7, 8, 10], "dmarc_moderation_act": [3, 8], "accept": [3, 4, 8], "dmarc_quarantine_moderation_act": [3, 8], "dmarc_none_moderation_act": [3, 8], "ident": [3, 8, 11], "blank": [3, 8], "html": [3, 4, 8, 9], "plaintext": [3, 8], "rfc2369": [3, 8], "explicit": [3, 8], "first": [3, 6, 8, 11], "strip": [3, 8, 11], "replyto": [3, 8], "goe": [3, 8], "mung": [3, 8], "mitig": [3, 8], "uncondition": [3, 8], "templat": [3, 8], "unfortun": [3, 8], "postoriu": [3, 8], "admin": [3, 8, 11], "ui": [3, 8], "empti": [3, 8], "so": [3, 6, 7, 8, 11], "ll": [3, 8], "line": [3, 8], "touch": [3, 8], "var": [3, 8], "en": [3, 4, 8, 9], "member": [3, 8], "regular": [3, 8], "languag": [3, 8], "core": [3, 8], "16": [3, 8], "2017a": [3, 8], "higher": [3, 8], "rewrit": [3, 8], "enforc": [3, 8], "quarantin": [3, 8], "reject": [3, 8], "polici": [3, 8, 9, 11], "linux": [3, 6, 8], "go": [3, 8], "legal": [3, 8], "administr": [3, 8], "known": [3, 7, 8, 11], "longer": [3, 8], "wrap": [3, 8], "could": [3, 4, 8, 11], "interfer": [3, 8], "search": [3, 8, 11], "mobil": [3, 8], "On": [3, 4, 6, 7, 8], "hand": [3, 8], "caus": [3, 4, 7, 8], "accident": [3, 8], "entir": [3, 7, 8], "intend": [3, 8], "choos": [3, 8], "fit": [3, 8], "commun": [3, 8], "tab": [3, 4, 8], "page": [3, 4, 6, 7, 8], "splunk": [4, 5, 11], "parsedmarc": [4, 9, 10], "4": [4, 6, 10], "3": [4, 6, 9, 10, 11], "visual": 4, "dashboard": [4, 5, 10], "later": [4, 6, 11], "debian": [4, 6], "ubuntu": [4, 6], "y": [4, 6], "transport": 4, "wget": 4, "qo": 4, "artifact": 4, "elast": [4, 5], "co": 4, "gpg": 4, "dearmor": 4, "usr": 4, "share": [4, 11], "keyr": 4, "echo": 4, "deb": 4, "x": [4, 9], "stabl": 4, "main": 4, "tee": 4, "d": 4, "For": [4, 11], "cento": [4, 6], "rhel": [4, 6], "rpm": 4, "guid": [4, 5], "previous": [4, 7], "jvm": 4, "heap": 4, "veri": [4, 7, 11], "small": 4, "1g": 4, "under": [4, 6, 7], "heavi": 4, "load": 4, "fix": 4, "increas": [4, 11], "minimum": 4, "maximum": 4, "depend": [4, 5, 11], "resourc": [4, 5, 11], "sure": [4, 6], "ha": [4, 7, 11], "least": [4, 6, 11], "gb": 4, "ram": 4, "assign": 4, "xms4g": 4, "xmx4g": 4, "www": [4, 6, 11], "refer": [4, 5], "import": [4, 7], "As": [4, 7], "7": [4, 6], "activ": [4, 6], "xpack": 4, "vim": 4, "yml": 4, "featur": 4, "enrol": 4, "encrypt": [4, 11], "logstash": 4, "agent": 4, "keystor": 4, "cert": 4, "p12": 4, "mutual": 4, "between": [4, 7], "cluster": [4, 11], "node": 4, "verification_mod": 4, "truststor": 4, "self": [4, 5], "openssl": 4, "req": 4, "x509": 4, "dai": [4, 11], "newkei": 4, "rsa": 4, "4096": 4, "keyout": 4, "crt": 4, "Or": [4, 6], "csr": 4, "ca": 4, "fill": [4, 6], "prompt": 4, "fqdn": 4, "field": 4, "rm": 4, "f": 4, "place": [4, 7, 11], "mv": 4, "660": 4, "server_ip": 4, "publicbaseurl": 4, "note": 4, "connexion": 4, "9200": [4, 11], "5601": 4, "past": [4, 10], "verif": [4, 11], "put": [4, 11], "browser": 4, "setup": [4, 5, 11], "encryptedsavedobject": 4, "encryptionkei": 4, "xxxx": 4, "now": [4, 7], "right": [4, 7], "click": [4, 7], "export": 4, "ndjson": 4, "provid": [4, 7], "consol": [4, 11], "stack": 4, "manag": [4, 11], "hamburg": 4, "menu": [4, 7], "overwrit": 4, "restor": 4, "someon": 4, "els": 4, "permiss": [4, 11], "control": 4, "commerci": [4, 5], "pack": 4, "chang": [4, 7, 10, 11], "wai": [4, 7], "releas": [4, 6], "login": 4, "checkbox": 4, "dmarc_aggreg": 4, "dmarc_forens": 4, "conform": 4, "each": [4, 6, 10, 11], "easi": 4, "regul": [4, 6, 11], "gdpr": 4, "fore": 4, "effici": 4, "help": 5, "maintain": 5, "develop": 5, "consid": [5, 7], "review": [5, 7], "how": 5, "contribut": 5, "assist": 5, "pin": 5, "particularli": [5, 11], "thank": [5, 9], "contributor": 5, "cli": 5, "util": 5, "kibana": [5, 10], "altern": [5, 11], "agari": 5, "brand": [5, 7], "dmarcian": 5, "ondmarc": 5, "proofpoint": 5, "fraud": 5, "defens": 5, "valimail": 5, "draft": [5, 9], "rua": [5, 6], "failur": [5, 7, 9, 11], "ruf": [5, 6, 7, 11], "gmail": [5, 7, 11], "transpar": 5, "handl": [5, 11], "compress": 5, "structur": 5, "simpl": 5, "premad": [5, 10], "apach": 5, "kafka": [5, 11], "multipl": 5, "geoipupd": 5, "systemd": 5, "pattern": [5, 7], "retent": 5, "owa": 5, "ew": 5, "davmail": 5, "understand": [5, 7], "align": [5, 7, 9], "what": 5, "sender": [5, 7, 8], "won": 5, "t": [5, 8, 11], "dkim": [5, 7, 8, 9], "bug": 5, "tabl": [5, 7], "anoth": [6, 11], "solut": 6, "two": 6, "mailto": 6, "uri": 6, "tag": 6, "comma": [6, 11], "behind": 6, "environ": 6, "detail": [6, 7], "http_proxi": 6, "prox": 6, "3128": 6, "https_proxi": 6, "ftp_proxi": 6, "credenti": [6, 11], "wide": [6, 9], "patch": 6, "2010": [6, 9], "rollup": 6, "22": 6, "kb4295699": 6, "2013": 6, "cumul": 6, "21": 6, "kb4099855": 6, "2016": 6, "11": [6, 9], "kb4134118": 6, "static": 6, "copi": [6, 10], "lite": 6, "databas": 6, "ipdb": 6, "distribut": 6, "term": 6, "creativ": 6, "attribut": 6, "intern": 6, "licens": 6, "fallback": 6, "geolite2": 6, "howev": 6, "cannot": 6, "tool": [6, 11], "locat": [6, 7], "overridden": 6, "buster": 6, "compon": 6, "contrib": 6, "repositori": [6, 10], "ppa": 6, "dnf": 6, "build": 6, "maco": 6, "window": 6, "decemb": 6, "30th": 6, "2019": 6, "free": 6, "account": [6, 7], "order": 6, "variou": 6, "regist": 6, "differ": [6, 7, 11], "older": [6, 9], "newer": 6, "Be": 6, "select": 6, "correct": 6, "v": [6, 11], "onc": 6, "pre": 6, "geoip": 6, "conf": 6, "systemdr": 6, "programdata": 6, "citi": 6, "asn": 6, "weekli": 6, "tuesdai": 6, "cron": 6, "schedul": 6, "task": 6, "python3": 6, "pip": 6, "virtualenv": 6, "dev": [6, 11], "libxml2": 6, "libxslt": 6, "python39": 6, "setuptool": 6, "devel": 6, "mkdir": 6, "b": [6, 9], "venv": [6, 11], "those": 6, "explicitli": 6, "9": 6, "insid": 6, "abl": 6, "libemail": 6, "friendli": 7, "incom": [7, 11], "switch": 7, "left": 7, "side": 7, "suggest": 7, "best": 7, "across": 7, "three": 7, "pie": 7, "chart": 7, "percentag": 7, "spf": [7, 9], "segment": 7, "malici": [7, 11], "just": 7, "especi": 7, "collect": [7, 11], "mai": [7, 11], "legitim": [7, 11], "correctli": 7, "while": [7, 11], "remain": 7, "valid": [7, 11], "often": 7, "rule": [7, 11], "wherea": 7, "reli": 7, "session": 7, "underneath": 7, "passag": 7, "disposit": [7, 9], "center": 7, "sort": [7, 11], "volum": 7, "By": [7, 11], "hover": 7, "mous": 7, "magnifi": 7, "glass": 7, "icon": 7, "our": 7, "recogn": 7, "market": 7, "plu": 7, "That": 7, "busi": 7, "particular": 7, "With": 7, "contact": 7, "lot": 7, "b2c": 7, "custom": [7, 11], "high": 7, "come": 7, "consum": 7, "googl": [7, 11], "yahoo": 7, "old": 7, "mention": 7, "earlier": 7, "similar": 7, "observ": 7, "who": 7, "addresse": 7, "parent": 7, "subsidiari": 7, "outdat": 7, "further": 7, "down": 7, "were": [7, 11], "call": 7, "been": [7, 11], "consolid": 7, "view": [7, 11], "own": [7, 10], "temporari": 7, "upper": 7, "These": 7, "recipi": 7, "avoid": 7, "leak": 7, "notabl": 7, "chines": 7, "suppli": [7, 11], "few": [7, 11], "wiki": 9, "schema": 9, "7480": 9, "appendix": 9, "c": [9, 11], "produc": 9, "normal": [9, 11], "regardless": 9, "xml_schema": 9, "report_metadata": 9, "org_nam": 9, "acm": 9, "org_email": 9, "org_extra_contact_info": 9, "report_id": 9, "9391651994964116463": 9, "begin_d": 9, "2012": 9, "04": 9, "27": 9, "20": 9, "00": 9, "end_dat": 9, "28": 9, "19": 9, "59": 9, "policy_publish": 9, "adkim": 9, "aspf": 9, "pct": 9, "100": 9, "fo": 9, "72": 9, "150": 9, "241": 9, "94": 9, "adsl": 9, "shv": 9, "bellsouth": 9, "base_domain": 9, "policy_evalu": 9, "policy_override_reason": 9, "identifi": 9, "header_from": 9, "envelope_from": 9, "envelope_to": 9, "null": 9, "auth_result": 9, "selector": 9, "scope": [9, 11], "mfrom": 9, "source_ip_address": 9, "source_countri": 9, "source_reverse_dn": 9, "source_base_domain": 9, "spf_align": 9, "dkim_align": 9, "dmarc_align": 9, "policy_override_com": 9, "dkim_domain": 9, "dkim_selector": 9, "dkim_result": 9, "spf_domain": 9, "spf_scope": 9, "spf_result": 9, "xennn": 9, "anonym": 9, "feedback_typ": 9, "user_ag": 9, "lua": 9, "original_mail_from": 9, "sharepoint": 9, "de": 9, "original_rcpt_to": 9, "peter": 9, "pan": 9, "arrival_d": 9, "mon": 9, "01": 9, "oct": 9, "2018": 9, "0200": 9, "message_id": 9, "38": 9, "e7": 9, "30937": 9, "bd6e1bb5": 9, "mailrelai": 9, "authentication_result": 9, "di": 9, "delivery_result": 9, "auth_failur": 9, "reported_domain": 9, "arrival_date_utc": 9, "09": 9, "authentication_mechan": 9, "original_envelope_id": 9, "sample_headers_onli": 9, "servernameon": 9, "n": [9, 11], "tby": 9, "cest": 9, "ndate": 9, "nmessag": 9, "nto": 9, "nfrom": 9, "utf": 9, "sw50zxjha3rpdmugv2v0dgjld2vyymvylcocymvyc2ljahq": 9, "nsubject": 9, "nmime": 9, "nx": 9, "mailer": 9, "foundat": 9, "ncontent": 9, "charset": 9, "transfer": 9, "quot": 9, "printabl": 9, "head": 9, "href": 9, "3d": 9, "nwettbewerb": 9, "doctyp": 9, "w3c": 9, "dtd": 9, "meta": 9, "08": 9, "0240": 9, "003": 9, "parsed_sampl": 9, "display_nam": 9, "interakt": 9, "wettbewerb": 9, "\u00fcbersicht": 9, "to_domain": 9, "timezon": 9, "mime": 9, "hop": 9, "date_utc": 9, "has_defect": 9, "reply_to": 9, "filename_safe_subject": 9, "collector": [10, 11], "editor": 10, "occurr": 10, "layout": 10, "although": 10, "slightli": 10, "easier": 10, "flexibl": 10, "usag": 11, "h": 11, "config_fil": 11, "verbos": 11, "debug": 11, "log_fil": 11, "file_path": 11, "posit": 11, "argument": 11, "exit": 11, "silent": 11, "impli": 11, "write": 11, "print": 11, "warn": 11, "program": 11, "describ": 11, "comment": 11, "save_aggreg": 11, "save_forens": 11, "dmarcresport": 11, "upersecur": 11, "splunk_hec": 11, "splunkhec": 11, "hectokengoesher": 11, "s3": 11, "bucket": 11, "my": 11, "localhost": 11, "514": 11, "full": 11, "period": 11, "n_proc": 11, "chunk_siz": 11, "larger": 11, "improv": 11, "thousand": 11, "label": 11, "arriv": 11, "993": 11, "escap": 11, "wherev": 11, "section": 11, "recommend": 11, "try": 11, "skip_certificate_verif": 11, "skip": 11, "msgraph": 11, "auth_method": 11, "method": 11, "usernamepassword": 11, "devicecod": 11, "clientsecret": 11, "m365": 11, "client_id": 11, "app": 11, "registr": 11, "client_secret": 11, "secret": 11, "tenant_id": 11, "azur": 11, "tenant": 11, "token_fil": 11, "allow_unencrypted_storag": 11, "fall": 11, "back": 11, "unencrypt": 11, "grant": 11, "readwrit": 11, "deleg": 11, "applic": 11, "restrict": 11, "sinc": 11, "applicationaccesspolici": 11, "powershel": 11, "accessright": 11, "restrictaccess": 11, "appid": 11, "policyscopegroupid": 11, "special": 11, "cert_path": 11, "trust": 11, "appli": 11, "passsword": 11, "aggregate_top": 11, "topic": 11, "forensic_top": 11, "25": 11, "starttl": 11, "upload": 11, "region_nam": 11, "region": 11, "endpoint_url": 11, "endpoint": 11, "access_key_id": 11, "secret_access_kei": 11, "udp": 11, "gmail_api": 11, "credentials_fil": 11, "include_spam_trash": 11, "spam": 11, "trash": 11, "acquir": 11, "googleapi": 11, "oauth2_port": 11, "tcp": 11, "oauth2": 11, "8080": 11, "log_analyt": 11, "resid": 11, "dce": 11, "ingest": 11, "dcr_immutable_id": 11, "immut": 11, "dcr": 11, "dcr_aggregate_stream": 11, "stream": 11, "dcr_forensic_stream": 11, "regard": 11, "strongli": 11, "much": 11, "faster": 11, "reliabl": 11, "cisco": 11, "opendn": 11, "outsid": 11, "instanc": 11, "highli": 11, "industri": 11, "sensit": 11, "healthcar": 11, "financ": 11, "possibl": 11, "appear": 11, "sometim": 11, "kind": 11, "approach": 11, "manual": 11, "1000": 11, "per": 11, "analyz": 11, "year": 11, "_cluster": 11, "health": 11, "pretti": 11, "active_primary_shard": 11, "932": 11, "active_shard": 11, "2k": 11, "persist": 11, "max_shards_per_nod": 11, "2000": 11, "watcher": 11, "io": 11}, "objects": {"": [[0, 0, 0, "-", "parsedmarc"]], "parsedmarc": [[0, 1, 1, "", "InvalidAggregateReport"], [0, 1, 1, "", "InvalidDMARCReport"], [0, 1, 1, "", "InvalidForensicReport"], [0, 1, 1, "", "ParserError"], [0, 0, 0, "-", "elastic"], [0, 2, 1, "", "email_results"], [0, 2, 1, "", "extract_xml"], [0, 2, 1, "", "get_dmarc_reports_from_mailbox"], [0, 2, 1, "", "get_dmarc_reports_from_mbox"], [0, 2, 1, "", "get_report_zip"], [0, 2, 1, "", "parse_aggregate_report_file"], [0, 2, 1, "", "parse_aggregate_report_xml"], [0, 2, 1, "", "parse_forensic_report"], [0, 2, 1, "", "parse_report_email"], [0, 2, 1, "", "parse_report_file"], [0, 2, 1, "", "parsed_aggregate_reports_to_csv"], [0, 2, 1, "", "parsed_aggregate_reports_to_csv_rows"], [0, 2, 1, "", "parsed_forensic_reports_to_csv"], [0, 2, 1, "", "parsed_forensic_reports_to_csv_rows"], [0, 2, 1, "", "save_output"], [0, 0, 0, "-", "splunk"], [0, 0, 0, "-", "utils"], [0, 2, 1, "", "watch_inbox"]], "parsedmarc.elastic": [[0, 1, 1, "", "AlreadySaved"], [0, 1, 1, "", "ElasticsearchError"], [0, 2, 1, "", "create_indexes"], [0, 2, 1, "", "migrate_indexes"], [0, 2, 1, "", "save_aggregate_report_to_elasticsearch"], [0, 2, 1, "", "save_forensic_report_to_elasticsearch"], [0, 2, 1, "", "set_hosts"]], "parsedmarc.splunk": [[0, 3, 1, "", "HECClient"], [0, 1, 1, "", "SplunkError"]], "parsedmarc.splunk.HECClient": [[0, 4, 1, "", "save_aggregate_reports_to_splunk"], [0, 4, 1, "", "save_forensic_reports_to_splunk"]], "parsedmarc.utils": [[0, 1, 1, "", "DownloadError"], [0, 1, 1, "", "EmailParserError"], [0, 2, 1, "", "convert_outlook_msg"], [0, 2, 1, "", "decode_base64"], [0, 2, 1, "", "get_base_domain"], [0, 2, 1, "", "get_filename_safe_string"], [0, 2, 1, "", "get_ip_address_country"], [0, 2, 1, "", "get_ip_address_info"], [0, 2, 1, "", "get_reverse_dns"], [0, 2, 1, "", "human_timestamp_to_datetime"], [0, 2, 1, "", "human_timestamp_to_timestamp"], [0, 2, 1, "", "is_mbox"], [0, 2, 1, "", "is_outlook_msg"], [0, 2, 1, "", "parse_email"], [0, 2, 1, "", "query_dns"], [0, 2, 1, "", "timestamp_to_datetime"], [0, 2, 1, "", "timestamp_to_human"]]}, "objtypes": {"0": "py:module", "1": "py:exception", "2": "py:function", "3": "py:class", "4": "py:method"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "exception", "Python exception"], "2": ["py", "function", "Python function"], "3": ["py", "class", "Python class"], "4": ["py", "method", "Python method"]}, "titleterms": {"api": 0, "refer": 0, "parsedmarc": [0, 1, 2, 5, 6, 11], "elast": 0, "splunk": [0, 10], "util": 0, "indic": 0, "tabl": 0, "contribut": 1, "bug": 1, "report": [1, 5, 6, 9], "access": 2, "an": 2, "inbox": 2, "us": [2, 7, 11], "owa": 2, "ew": 2, "run": [2, 11], "davmail": 2, "systemd": [2, 11], "servic": [2, 11], "configur": [2, 11], "understand": 3, "dmarc": [3, 5, 7], "resourc": 3, "guid": 3, "spf": 3, "record": [3, 4], "valid": 3, "lookalik": 3, "domain": 3, "align": 3, "what": [3, 8], "sender": 3, "won": 3, "t": 3, "support": 3, "dkim": 3, "about": [3, 8], "mail": [3, 8], "list": [3, 8], "best": [3, 8], "practic": [3, 8], "do": [3, 8], "mailman": [3, 8], "2": [3, 8], "3": [3, 8], "listserv": [3, 8], "workaround": [3, 8], "elasticsearch": 4, "kibana": [4, 7], "upgrad": 4, "index": 4, "pattern": 4, "retent": 4, "document": 5, "open": 5, "sourc": 5, "analyz": [5, 6], "visual": 5, "featur": 5, "content": 5, "instal": 6, "test": 6, "multipl": 6, "geoipupd": 6, "setup": 6, "option": 6, "depend": 6, "dashboard": 7, "summari": 7, "forens": [7, 9], "sampl": [7, 9], "output": 9, "aggreg": 9, "json": 9, "csv": 9, "cli": 11, "help": 11, "file": 11}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.todo": 2, "sphinx.ext.viewcode": 1, "sphinx": 57}, "alltitles": {"API reference": [[0, "api-reference"]], "parsedmarc": [[0, "module-parsedmarc"]], "parsedmarc.elastic": [[0, "module-parsedmarc.elastic"]], "parsedmarc.splunk": [[0, "module-parsedmarc.splunk"]], "parsedmarc.utils": [[0, "module-parsedmarc.utils"]], "Indices and tables": [[0, "indices-and-tables"]], "Contributing to parsedmarc": [[1, "contributing-to-parsedmarc"]], "Bug reports": [[1, "bug-reports"]], "Accessing an inbox using OWA/EWS": [[2, "accessing-an-inbox-using-owa-ews"]], "Running DavMail as a systemd service": [[2, "running-davmail-as-a-systemd-service"]], "Configuring parsedmarc for DavMail": [[2, "configuring-parsedmarc-for-davmail"]], "Understanding DMARC": [[3, "understanding-dmarc"]], "Resources": [[3, "resources"]], "DMARC guides": [[3, "dmarc-guides"]], "SPF and DMARC record validation": [[3, "spf-and-dmarc-record-validation"]], "Lookalike domains": [[3, "lookalike-domains"]], "DMARC Alignment Guide": [[3, "dmarc-alignment-guide"]], "What if a sender won\u2019t support DKIM/DMARC?": [[3, "what-if-a-sender-wont-support-dkim-dmarc"]], "What about mailing lists?": [[3, "what-about-mailing-lists"], [8, "what-about-mailing-lists"]], "Mailing list list best practices": [[3, "mailing-list-list-best-practices"], [8, "mailing-list-list-best-practices"]], "Do": [[3, "do"], [8, "do"]], "Do not": [[3, "do-not"], [8, "do-not"]], "Mailman 2": [[3, "mailman-2"], [3, "id1"], [8, "mailman-2"], [8, "id1"]], "Mailman 3": [[3, "mailman-3"], [3, "id2"], [8, "mailman-3"], [8, "id2"]], "LISTSERV": [[3, "listserv"], [8, "listserv"]], "Workarounds": [[3, "workarounds"], [8, "workarounds"]], "Elasticsearch and Kibana": [[4, "elasticsearch-and-kibana"]], "Upgrading Kibana index patterns": [[4, "upgrading-kibana-index-patterns"]], "Records retention": [[4, "records-retention"]], "parsedmarc documentation - Open source DMARC report analyzer and visualizer": [[5, "parsedmarc-documentation-open-source-dmarc-report-analyzer-and-visualizer"]], "Features": [[5, "features"]], "Contents": [[5, null]], "Installation": [[6, "installation"]], "Testing multiple report analyzers": [[6, "testing-multiple-report-analyzers"]], "geoipupdate setup": [[6, "geoipupdate-setup"]], "Installing parsedmarc": [[6, "installing-parsedmarc"]], "Optional dependencies": [[6, "optional-dependencies"]], "Using the Kibana dashboards": [[7, "using-the-kibana-dashboards"]], "DMARC Summary": [[7, "dmarc-summary"]], "DMARC Forensic Samples": [[7, "dmarc-forensic-samples"]], "Sample outputs": [[9, "sample-outputs"]], "Sample aggregate report output": [[9, "sample-aggregate-report-output"]], "JSON aggregate report": [[9, "json-aggregate-report"]], "CSV aggregate report": [[9, "csv-aggregate-report"]], "Sample forensic report output": [[9, "sample-forensic-report-output"]], "JSON forensic report": [[9, "json-forensic-report"]], "CSV forensic report": [[9, "csv-forensic-report"]], "Splunk": [[10, "splunk"]], "Using parsedmarc": [[11, "using-parsedmarc"]], "CLI help": [[11, "cli-help"]], "Configuration file": [[11, "configuration-file"]], "Running parsedmarc as a systemd service": [[11, "running-parsedmarc-as-a-systemd-service"]]}, "indexentries": {"alreadysaved": [[0, "parsedmarc.elastic.AlreadySaved"]], "downloaderror": [[0, "parsedmarc.utils.DownloadError"]], "elasticsearcherror": [[0, "parsedmarc.elastic.ElasticsearchError"]], "emailparsererror": [[0, "parsedmarc.utils.EmailParserError"]], "hecclient (class in parsedmarc.splunk)": [[0, "parsedmarc.splunk.HECClient"]], "invalidaggregatereport": [[0, "parsedmarc.InvalidAggregateReport"]], "invaliddmarcreport": [[0, "parsedmarc.InvalidDMARCReport"]], "invalidforensicreport": [[0, "parsedmarc.InvalidForensicReport"]], "parsererror": [[0, "parsedmarc.ParserError"]], "splunkerror": [[0, "parsedmarc.splunk.SplunkError"]], "convert_outlook_msg() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.convert_outlook_msg"]], "create_indexes() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.create_indexes"]], "decode_base64() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.decode_base64"]], "email_results() (in module parsedmarc)": [[0, "parsedmarc.email_results"]], "extract_xml() (in module parsedmarc)": [[0, "parsedmarc.extract_xml"]], "get_base_domain() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_base_domain"]], "get_dmarc_reports_from_mailbox() (in module parsedmarc)": [[0, "parsedmarc.get_dmarc_reports_from_mailbox"]], "get_dmarc_reports_from_mbox() (in module parsedmarc)": [[0, "parsedmarc.get_dmarc_reports_from_mbox"]], "get_filename_safe_string() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_filename_safe_string"]], "get_ip_address_country() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_ip_address_country"]], "get_ip_address_info() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_ip_address_info"]], "get_report_zip() (in module parsedmarc)": [[0, "parsedmarc.get_report_zip"]], "get_reverse_dns() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_reverse_dns"]], "human_timestamp_to_datetime() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.human_timestamp_to_datetime"]], "human_timestamp_to_timestamp() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.human_timestamp_to_timestamp"]], "is_mbox() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.is_mbox"]], "is_outlook_msg() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.is_outlook_msg"]], "migrate_indexes() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.migrate_indexes"]], "module": [[0, "module-parsedmarc"], [0, "module-parsedmarc.elastic"], [0, "module-parsedmarc.splunk"], [0, "module-parsedmarc.utils"]], "parse_aggregate_report_file() (in module parsedmarc)": [[0, "parsedmarc.parse_aggregate_report_file"]], "parse_aggregate_report_xml() (in module parsedmarc)": [[0, "parsedmarc.parse_aggregate_report_xml"]], "parse_email() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.parse_email"]], "parse_forensic_report() (in module parsedmarc)": [[0, "parsedmarc.parse_forensic_report"]], "parse_report_email() (in module parsedmarc)": [[0, "parsedmarc.parse_report_email"]], "parse_report_file() (in module parsedmarc)": [[0, "parsedmarc.parse_report_file"]], "parsed_aggregate_reports_to_csv() (in module parsedmarc)": [[0, "parsedmarc.parsed_aggregate_reports_to_csv"]], "parsed_aggregate_reports_to_csv_rows() (in module parsedmarc)": [[0, "parsedmarc.parsed_aggregate_reports_to_csv_rows"]], "parsed_forensic_reports_to_csv() (in module parsedmarc)": [[0, "parsedmarc.parsed_forensic_reports_to_csv"]], "parsed_forensic_reports_to_csv_rows() (in module parsedmarc)": [[0, "parsedmarc.parsed_forensic_reports_to_csv_rows"]], "parsedmarc": [[0, "module-parsedmarc"]], "parsedmarc.elastic": [[0, "module-parsedmarc.elastic"]], "parsedmarc.splunk": [[0, "module-parsedmarc.splunk"]], "parsedmarc.utils": [[0, "module-parsedmarc.utils"]], "query_dns() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.query_dns"]], "save_aggregate_report_to_elasticsearch() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.save_aggregate_report_to_elasticsearch"]], "save_aggregate_reports_to_splunk() (parsedmarc.splunk.hecclient method)": [[0, "parsedmarc.splunk.HECClient.save_aggregate_reports_to_splunk"]], "save_forensic_report_to_elasticsearch() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.save_forensic_report_to_elasticsearch"]], "save_forensic_reports_to_splunk() (parsedmarc.splunk.hecclient method)": [[0, "parsedmarc.splunk.HECClient.save_forensic_reports_to_splunk"]], "save_output() (in module parsedmarc)": [[0, "parsedmarc.save_output"]], "set_hosts() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.set_hosts"]], "timestamp_to_datetime() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.timestamp_to_datetime"]], "timestamp_to_human() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.timestamp_to_human"]], "watch_inbox() (in module parsedmarc)": [[0, "parsedmarc.watch_inbox"]]}}) \ No newline at end of file +Search.setIndex({"docnames": ["api", "contributing", "davmail", "dmarc", "elasticsearch", "index", "installation", "kibana", "mailing-lists", "output", "splunk", "usage"], "filenames": ["api.md", "contributing.md", "davmail.md", "dmarc.md", "elasticsearch.md", "index.md", "installation.md", "kibana.md", "mailing-lists.md", "output.md", "splunk.md", "usage.md"], "titles": ["API reference", "Contributing to parsedmarc", "Accessing an inbox using OWA/EWS", "Understanding DMARC", "Elasticsearch and Kibana", "parsedmarc documentation - Open source DMARC report analyzer and visualizer", "Installation", "Using the Kibana dashboards", "What about mailing lists?", "Sample outputs", "Splunk", "Using parsedmarc"], "terms": {"A": [0, 3, 11], "python": [0, 5, 6], "packag": [0, 4], "pars": [0, 3, 5, 6, 9, 11], "dmarc": [0, 4, 6, 8, 9, 10, 11], "report": [0, 4, 7, 10, 11], "except": [0, 11], "invalidaggregatereport": 0, "sourc": [0, 3, 4, 6, 7, 9], "rais": 0, "when": [0, 3, 5, 7, 8, 11], "an": [0, 3, 5, 7, 8, 9, 11], "invalid": 0, "aggreg": [0, 5, 7, 10, 11], "i": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11], "encount": 0, "invaliddmarcreport": 0, "invalidforensicreport": 0, "forens": [0, 5, 10, 11], "parsererror": 0, "whenev": [0, 2, 11], "parser": 0, "fail": [0, 3, 7, 8, 9, 11], "some": [0, 2, 3, 4, 7, 8], "reason": [0, 2, 4, 11], "email_result": 0, "result": [0, 5, 7, 9, 11], "host": [0, 2, 3, 4, 5, 8, 11], "mail_from": 0, "mail_to": 0, "mail_cc": 0, "none": [0, 3, 9, 11], "mail_bcc": 0, "port": [0, 2, 11], "0": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11], "require_encrypt": 0, "fals": [0, 2, 6, 9, 11], "verifi": 0, "true": [0, 2, 4, 9, 11], "usernam": [0, 11], "password": [0, 4, 6, 11], "subject": [0, 3, 8, 9, 11], "attachment_filenam": 0, "messag": [0, 2, 3, 4, 6, 7, 8, 9, 11], "email": [0, 3, 5, 6, 7, 8, 9, 10, 11], "zip": [0, 2, 5, 11], "file": [0, 2, 5, 6, 10], "paramet": 0, "ordereddict": 0, "mail": [0, 5, 6, 9, 11], "server": [0, 2, 3, 4, 6, 7, 9, 11], "hostnam": [0, 11], "ip": [0, 3, 4, 6, 7, 11], "address": [0, 2, 3, 4, 7, 8, 9, 11], "The": [0, 3, 6, 7, 10, 11], "valu": [0, 3, 4, 7, 8, 11], "from": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11], "header": [0, 3, 7, 8, 9, 11], "list": [0, 2, 4, 5, 7, 11], "cc": [0, 9], "bcc": [0, 9], "int": [0, 11], "us": [0, 3, 4, 5, 6, 8, 9], "bool": [0, 11], "requir": [0, 2, 3, 4, 6, 8, 11], "secur": [0, 4, 11], "connect": [0, 2, 4, 11], "start": [0, 2, 4, 6, 7, 10, 11], "ssl": [0, 2, 4, 11], "tl": [0, 11], "certif": [0, 4, 11], "str": [0, 11], "option": [0, 2, 3, 4, 5, 8, 10, 11], "overrid": 0, "default": [0, 2, 4, 6, 7, 11], "attach": [0, 3, 8, 9, 11], "filenam": [0, 11], "plain": 0, "text": [0, 9], "bodi": [0, 3, 8, 9, 11], "extract_xml": 0, "input_": 0, "extract": [0, 2], "xml": [0, 10], "gzip": [0, 5], "given": [0, 11], "path": [0, 4, 11], "like": [0, 3, 6, 8], "object": [0, 4], "byte": 0, "input": 0, "return": 0, "type": [0, 9, 11], "get_dmarc_reports_from_mailbox": 0, "mailboxconnect": 0, "reports_fold": [0, 11], "inbox": [0, 3, 5, 8, 11], "archive_fold": [0, 11], "archiv": [0, 11], "delet": [0, 2, 4, 11], "test": [0, 5, 11], "ip_db_path": [0, 6, 11], "offlin": [0, 11], "nameserv": [0, 11], "dns_timeout": [0, 11], "6": [0, 4, 6, 11], "strip_attachment_payload": [0, 11], "batch_siz": [0, 11], "10": [0, 6, 9, 11], "create_fold": 0, "fetch": 0, "mailbox": [0, 7, 11], "folder": [0, 2, 11], "where": [0, 2, 3, 8, 11], "can": [0, 2, 3, 4, 5, 6, 7, 8, 11], "found": [0, 6, 11], "move": [0, 4, 11], "process": [0, 2, 5, 6, 11], "after": [0, 2, 4, 11], "them": [0, 4, 7, 11], "do": [0, 2, 6, 7, 11], "mmdb": [0, 11], "maxmind": [0, 6, 11], "dbip": [0, 11], "queri": [0, 11], "onlin": [0, 2, 11], "geoloc": [0, 11], "dn": [0, 3, 7, 11], "float": [0, 11], "set": [0, 2, 3, 4, 6, 7, 8, 11], "timeout": [0, 2, 11], "remov": [0, 3, 4, 8, 11], "payload": [0, 11], "dict": 0, "previou": [0, 2, 4, 11], "run": [0, 4, 5, 6], "number": [0, 11], "read": [0, 11], "befor": [0, 11], "save": [0, 4, 6, 11], "limit": [0, 2, 11], "whether": 0, "creat": [0, 2, 3, 4, 6, 8, 11], "destin": 0, "watch": [0, 2, 4, 11], "aggregate_report": 0, "forensic_report": 0, "get_dmarc_reports_from_mbox": 0, "2": [0, 4, 9, 11], "parallel": [0, 11], "mbox": [0, 11], "format": [0, 6], "contain": [0, 7, 10, 11], "e": [0, 2, 3, 4, 6, 8, 11], "one": [0, 3, 5, 8, 11], "more": [0, 4, 6, 10, 11], "cloudflar": [0, 11], "": [0, 2, 3, 4, 6, 8, 9, 11], "public": [0, 3, 9, 11], "resolv": [0, 11], "second": [0, 2, 11], "make": [0, 3, 4, 8, 11], "get_report_zip": 0, "output": [0, 5, 11], "parse_aggregate_report_fil": 0, "_input": 0, "keep_al": 0, "callabl": 0, "keep": 0, "aliv": 0, "function": 0, "parse_aggregate_report_xml": 0, "string": 0, "consist": [0, 5, 9], "parse_forensic_report": 0, "feedback_report": 0, "sampl": [0, 5, 11], "msg_date": 0, "convert": [0, 3, 8], "feedback": 0, "rfc": [0, 3, 8, 9], "822": 0, "date": [0, 3, 8, 9], "parse_report_email": 0, "report_typ": 0, "parse_report_fil": 0, "parsed_aggregate_reports_to_csv": 0, "flat": 0, "csv": [0, 5, 11], "includ": [0, 3, 6, 7, 8, 11], "data": [0, 4, 5, 7, 10, 11], "parsed_aggregate_reports_to_csv_row": 0, "parsed_forensic_reports_to_csv": 0, "parsed_forensic_reports_to_csv_row": 0, "save_output": 0, "output_directori": 0, "aggregate_json_filenam": [0, 11], "json": [0, 5, 11], "forensic_json_filenam": [0, 11], "aggregate_csv_filenam": [0, 11], "forensic_csv_filenam": [0, 11], "directori": [0, 11], "watch_inbox": 0, "mailbox_connect": 0, "callback": 0, "check_timeout": [0, 11], "30": [0, 11], "new": [0, 2, 3, 6, 7, 11], "send": [0, 2, 3, 4, 5, 7, 8, 10, 11], "receiv": [0, 9, 11], "imap": [0, 2, 5, 11], "wait": [0, 11], "idl": [0, 2, 11], "respons": [0, 11], "until": [0, 11], "next": [0, 11], "check": [0, 2, 3, 4, 6, 11], "replac": [0, 3, 4, 8], "alreadysav": 0, "match": [0, 4, 10], "exist": [0, 3, 4, 8], "elasticsearcherror": 0, "elasticsearch": [0, 5, 11], "error": [0, 9, 11], "occur": [0, 7], "create_index": 0, "name": [0, 3, 4, 7, 9, 10, 11], "index": [0, 5, 10, 11], "migrate_index": 0, "aggregate_index": 0, "forensic_index": 0, "updat": [0, 4, 6, 11], "map": 0, "save_aggregate_report_to_elasticsearch": 0, "index_suffix": [0, 11], "monthly_index": [0, 11], "number_of_shard": [0, 11], "1": [0, 2, 4, 5, 6, 9, 11], "number_of_replica": [0, 11], "suffix": [0, 11], "monthli": [0, 11], "instead": [0, 3, 6, 8, 11], "daili": [0, 11], "shard": [0, 11], "replica": [0, 11], "save_forensic_report_to_elasticsearch": 0, "set_host": 0, "use_ssl": 0, "ssl_cert_path": 0, "60": 0, "singl": 0, "url": [0, 2, 11], "http": [0, 1, 2, 3, 4, 6, 8, 9, 10, 11], "chain": 0, "authent": [0, 2, 3, 4, 7, 11], "class": 0, "hecclient": 0, "access_token": 0, "initi": 0, "hec": [0, 10, 11], "access": [0, 4, 5, 6, 11], "token": [0, 4, 11], "give": [0, 4, 11], "up": [0, 2, 4, 6, 7], "save_aggregate_reports_to_splunk": 0, "dictionari": 0, "save_forensic_reports_to_splunk": 0, "splunkerror": 0, "might": [0, 3, 7, 8], "other": [0, 3, 4, 7, 8], "project": [0, 2, 3, 5, 10], "downloaderror": 0, "download": [0, 2, 4, 6], "emailparsererror": 0, "convert_outlook_msg": 0, "msg_byte": 0, "msgconvert": [0, 6], "perl": [0, 6], "outlook": [0, 2, 6], "m": [0, 6, 9, 11], "standard": [0, 5, 9], "content": [0, 3, 8, 9, 10], "msg": [0, 6], "decode_base64": 0, "decod": 0, "base64": 0, "pad": 0, "being": 0, "encod": [0, 9, 11], "get_base_domain": 0, "domain": [0, 4, 7, 8, 9], "get": [0, 2, 4, 6, 11], "base": [0, 2, 3, 4, 7, 8, 9], "ar": [0, 2, 3, 4, 6, 7, 8, 9, 11], "publicsuffix": 0, "org": [0, 6, 9], "public_suffix_list": 0, "dat": 0, "subdomain": [0, 3], "get_filename_safe_str": 0, "safe": 0, "get_ip_address_countri": 0, "ip_address": [0, 9], "db_path": 0, "iso": 0, "code": [0, 4, 5], "countri": [0, 6, 7, 9], "associ": 0, "ipv4": 0, "ipv6": 0, "And": 0, "get_ip_address_info": 0, "cach": [0, 11], "revers": [0, 7], "inform": [0, 4, 6, 7, 11], "expiringdict": 0, "storag": [0, 11], "reverse_dn": [0, 9], "get_reverse_dn": 0, "ani": [0, 3, 7, 8, 11], "human_timestamp_to_datetim": 0, "human_timestamp": 0, "to_utc": 0, "human": [0, 7], "readabl": 0, "timestamp": 0, "datetim": 0, "utc": 0, "human_timestamp_to_timestamp": 0, "unix": 0, "yyyi": 0, "mm": 0, "dd": 0, "hh": 0, "ss": 0, "is_mbox": 0, "flag": [0, 2], "is_outlook_msg": 0, "ol": [0, 6], "parse_email": 0, "simplifi": 0, "binari": 0, "query_dn": 0, "record_typ": 0, "about": [0, 5, 6], "record": [0, 5, 6, 9], "answer": [0, 11], "timestamp_to_datetim": 0, "timestamp_to_human": 0, "modul": [0, 5, 11], "pleas": [1, 5, 11], "github": [1, 6, 9, 11], "issu": [1, 5], "tracker": 1, "com": [1, 2, 3, 8, 9, 11], "domainawar": [1, 3, 11], "8": [2, 4, 6, 9, 11], "support": [2, 4, 5, 9, 10], "microsoft": [2, 5, 6, 9, 11], "offic": 2, "365": [2, 4], "via": 2, "graph": [2, 5, 7, 11], "api": [2, 4, 5], "which": [2, 4, 7, 11], "prefer": [2, 6], "over": [2, 5, 7], "organ": [2, 7, 11], "allow": [2, 3, 8, 11], "onli": [2, 3, 6, 7, 8, 11], "exchang": [2, 6, 9, 11], "web": [2, 4, 6], "In": [2, 3, 7, 8, 11], "case": [2, 3, 8], "need": [2, 3, 4, 6, 7, 8, 11], "local": [2, 4, 9, 11], "gatewai": 2, "It": [2, 4, 7, 9, 11], "even": [2, 3, 8, 11], "work": [2, 3, 5, 6, 7, 8], "modern": [2, 3, 8], "auth": [2, 9, 11], "multi": [2, 11], "factor": 2, "To": [2, 4, 6, 7, 9, 11], "thi": [2, 3, 4, 5, 6, 7, 8, 9, 11], "latest": [2, 4, 6], "version": [2, 4, 6, 9, 10, 11], "sourceforg": 2, "net": [2, 9], "unzip": 2, "command": [2, 3, 8, 11], "instal": [2, 4, 5, 11], "java": 2, "sudo": [2, 4, 6, 11], "apt": [2, 4, 6], "jre": 2, "headless": 2, "properti": 2, "see": [2, 3, 4, 5, 7, 11], "document": [2, 11], "basic": 2, "workstat": 2, "mode": [2, 4, 11], "auto": 2, "webdav": 2, "enableew": 2, "office365": 2, "asmx": 2, "listen": [2, 11], "imapport": 2, "1143": 2, "network": [2, 4, 11], "proxi": [2, 6], "enableproxi": 2, "usesystemproxi": 2, "proxyhost": 2, "proxyport": 2, "proxyus": 2, "proxypassword": 2, "exclud": 2, "noproxyfor": 2, "block": [2, 11], "remot": 2, "allowremot": 2, "bind": 2, "socket": 2, "loopback": 2, "bindaddress": 2, "127": [2, 4, 11], "disabl": [2, 11], "specifi": [2, 3], "nosecureimap": 2, "keepal": 2, "charact": [2, 11], "dure": 2, "larg": 2, "enablekeepal": 2, "count": [2, 9], "retriev": 2, "foldersizelimit": 2, "immedi": 2, "store": [2, 4], "imapautoexpung": 2, "enabl": [2, 4, 11], "poll": [2, 11], "delai": [2, 9], "minut": [2, 11], "imapidledelai": 2, "alwai": [2, 4, 11], "repli": [2, 3, 8], "rfc822": 2, "size": [2, 4], "request": [2, 4, 11], "approxim": 2, "perform": [2, 11], "imapalwaysapproxmsgs": 2, "client": [2, 3, 4, 8, 11], "300": 2, "clientsotimeout": 2, "system": [2, 3, 4, 6, 8, 11], "user": [2, 3, 4, 5, 6, 8, 9, 11], "useradd": [2, 6], "r": [2, 6, 9, 11], "bin": [2, 4, 6, 11], "protect": [2, 3, 5, 8, 11], "pry": [2, 11], "ey": [2, 11], "chown": [2, 11], "root": [2, 11], "opt": [2, 6, 11], "chmod": [2, 4, 11], "u": [2, 6, 9, 11], "rw": [2, 11], "g": [2, 3, 4, 8, 11], "o": [2, 4, 11], "nano": [2, 11], "etc": [2, 3, 4, 6, 8, 11], "unit": [2, 11], "descript": [2, 6, 11], "want": [2, 5, 11], "target": [2, 11], "syslog": [2, 11], "execstart": [2, 11], "group": [2, 7, 11], "restart": [2, 3, 4, 8, 11], "restartsec": [2, 11], "5m": [2, 11], "wantedbi": [2, 11], "Then": [2, 3, 4, 6, 8, 11], "systemctl": [2, 4, 11], "daemon": [2, 4, 11], "reload": [2, 4, 11], "you": [2, 3, 4, 5, 6, 7, 8, 11], "must": [2, 3, 8, 11], "also": [2, 3, 4, 7, 8, 11], "abov": [2, 11], "edit": [2, 6, 11], "everi": [2, 6, 11], "time": [2, 4, 6, 7, 11], "upgrad": [2, 5, 6, 11], "statu": [2, 11], "event": [2, 10, 11], "crash": [2, 4, 11], "5": [2, 4], "show": [2, 7, 11], "log": [2, 11], "current": [2, 4, 11], "vew": 2, "well": [2, 11], "newest": [2, 11], "oldest": [2, 11], "journalctl": [2, 11], "becaus": [2, 3, 7, 8, 11], "interact": [2, 4], "add": [2, 3, 4, 6, 7, 8, 11], "follow": [2, 4], "ini": [2, 11], "config": [2, 6, 11], "demystifi": 3, "complet": [3, 4], "If": [3, 4, 6, 7, 8, 11], "look": [3, 7], "out": [3, 4, 7], "sister": 3, "checkdmarc": 3, "against": [3, 8], "spoof": [3, 8], "open": 3, "monitor": [3, 11], "ensur": [3, 6, 8], "dkm": 3, "mechan": 3, "actual": [3, 9], "same": [3, 4, 6, 7, 10], "end": [3, 4], "pass": [3, 7, 9], "long": 3, "relat": 3, "indic": [3, 5], "signatur": [3, 7, 8], "kei": [3, 4, 6, 11], "publish": 3, "smtp": [3, 7, 9, 11], "envelop": 3, "sign": [3, 4, 6], "vendor": 3, "don": 3, "know": 3, "yet": 3, "ask": 3, "thei": [3, 6, 7, 8, 11], "through": 3, "your": [3, 4, 6, 7, 8, 10, 11], "relai": [3, 8], "theirs": 3, "realli": 3, "why": [3, 7], "displai": [3, 7, 10], "worst": 3, "have": [3, 4, 6, 7, 8, 10, 11], "specif": [3, 11], "norepli": [3, 9], "exampl": [3, 4, 6, 8, 9, 11], "separ": [3, 4, 6, 7, 10, 11], "p": [3, 6, 9], "alter": [3, 8], "sp": [3, 9], "top": [3, 7], "level": [3, 4], "tld": 3, "would": [3, 5, 6, 8], "leav": 3, "vulner": 3, "deploi": [3, 8], "find": [3, 7, 8], "most": [3, 4, 7, 8, 11], "modifi": [3, 8, 11], "footer": [3, 8], "part": [3, 4, 7, 8], "therebi": [3, 8], "break": [3, 4, 8], "ideal": [3, 8], "should": [3, 6, 7, 8, 11], "forward": [3, 7, 8], "without": [3, 4, 7, 8], "all": [3, 5, 7, 8, 10, 11], "joe": [3, 8], "nelson": [3, 8], "doe": [3, 8], "fantast": [3, 8], "job": [3, 6, 8], "explain": [3, 8], "exactli": [3, 8], "shouldn": [3, 8], "fulli": [3, 8], "compliant": [3, 8], "rather": [3, 8], "than": [3, 4, 8, 11], "repeat": [3, 8], "hi": [3, 8], "fine": [3, 8], "here": [3, 8, 9, 11], "summari": [3, 5, 8], "retain": [3, 8], "origin": [3, 8, 11], "2369": [3, 8], "unsubscrib": [3, 8], "outgo": [3, 8, 11], "ad": [3, 6, 8, 11], "link": [3, 4, 7, 8], "2919": [3, 8], "id": [3, 8, 9, 11], "webmail": [3, 7, 8], "servic": [3, 4, 5, 7, 8], "gener": [3, 4, 6, 8, 9, 11], "button": [3, 8], "tradit": [3, 8], "disclaim": [3, 8], "addit": [3, 8], "compli": [3, 4, 6, 8], "configur": [3, 4, 5, 6, 7, 8], "action": [3, 8], "prefix": [3, 8], "still": [3, 6, 8, 9, 11], "tell": [3, 6, 7, 8], "came": [3, 8], "wa": [3, 4, 6, 8], "sent": [3, 8, 11], "post": [3, 8], "step": [3, 4, 8], "common": [3, 4, 6, 8], "platform": [3, 8], "below": [3, 8, 11], "navig": [3, 6, 8], "subject_prefix": [3, 8], "from_is_list": [3, 8], "No": [3, 8], "first_strip_reply_to": [3, 8], "reply_goes_to_list": [3, 8], "poster": [3, 8], "include_rfc2369_head": [3, 8], "ye": [3, 8], "include_list_post_head": [3, 8], "include_sender_head": [3, 8], "non": [3, 8, 11], "digest": [3, 8], "msg_header": [3, 8], "msg_footer": [3, 8], "scrub_nondigest": [3, 8], "privaci": [3, 6, 7, 8, 11], "filter": [3, 7, 8, 10], "dmarc_moderation_act": [3, 8], "accept": [3, 4, 8], "dmarc_quarantine_moderation_act": [3, 8], "dmarc_none_moderation_act": [3, 8], "ident": [3, 8, 11], "blank": [3, 8], "html": [3, 4, 8, 9], "plaintext": [3, 8], "rfc2369": [3, 8], "explicit": [3, 8], "first": [3, 6, 8, 11], "strip": [3, 8, 11], "replyto": [3, 8], "goe": [3, 8], "mung": [3, 8], "mitig": [3, 8], "uncondition": [3, 8], "templat": [3, 8], "unfortun": [3, 8], "postoriu": [3, 8], "admin": [3, 8, 11], "ui": [3, 8], "empti": [3, 8], "so": [3, 6, 7, 8, 11], "ll": [3, 8], "line": [3, 8], "touch": [3, 8], "var": [3, 8], "en": [3, 4, 8, 9], "member": [3, 8], "regular": [3, 8], "languag": [3, 8], "core": [3, 8], "16": [3, 8], "2017a": [3, 8], "higher": [3, 8], "rewrit": [3, 8], "enforc": [3, 8], "quarantin": [3, 8], "reject": [3, 8], "polici": [3, 8, 9, 11], "linux": [3, 6, 8], "go": [3, 8], "legal": [3, 8], "administr": [3, 8], "known": [3, 7, 8, 11], "longer": [3, 8], "wrap": [3, 8], "could": [3, 4, 8, 11], "interfer": [3, 8], "search": [3, 8, 11], "mobil": [3, 8], "On": [3, 4, 6, 7, 8], "hand": [3, 8], "caus": [3, 4, 7, 8], "accident": [3, 8], "entir": [3, 7, 8], "intend": [3, 8], "choos": [3, 8], "fit": [3, 8], "commun": [3, 8], "tab": [3, 4, 8], "page": [3, 4, 6, 7, 8], "splunk": [4, 5, 11], "parsedmarc": [4, 9, 10], "4": [4, 6, 10], "3": [4, 6, 9, 10, 11], "visual": 4, "dashboard": [4, 5, 10], "later": [4, 6, 11], "debian": [4, 6], "ubuntu": [4, 6], "y": [4, 6], "transport": 4, "wget": 4, "qo": 4, "artifact": 4, "elast": [4, 5], "co": 4, "gpg": 4, "dearmor": 4, "usr": 4, "share": [4, 11], "keyr": 4, "echo": 4, "deb": 4, "x": [4, 9], "stabl": 4, "main": 4, "tee": 4, "d": 4, "For": [4, 11], "cento": [4, 6], "rhel": [4, 6], "rpm": 4, "guid": [4, 5], "previous": [4, 7], "jvm": 4, "heap": 4, "veri": [4, 7, 11], "small": 4, "1g": 4, "under": [4, 6, 7], "heavi": 4, "load": 4, "fix": 4, "increas": [4, 11], "minimum": 4, "maximum": 4, "depend": [4, 5, 11], "resourc": [4, 5, 11], "sure": [4, 6], "ha": [4, 7, 11], "least": [4, 6, 11], "gb": 4, "ram": 4, "assign": 4, "xms4g": 4, "xmx4g": 4, "www": [4, 6, 11], "refer": [4, 5], "import": [4, 7], "As": [4, 7], "7": [4, 6], "activ": [4, 6], "xpack": 4, "vim": 4, "yml": 4, "featur": 4, "enrol": 4, "encrypt": [4, 11], "logstash": 4, "agent": 4, "keystor": 4, "cert": 4, "p12": 4, "mutual": 4, "between": [4, 7], "cluster": [4, 11], "node": 4, "verification_mod": 4, "truststor": 4, "self": [4, 5], "openssl": 4, "req": 4, "x509": 4, "dai": [4, 11], "newkei": 4, "rsa": 4, "4096": 4, "keyout": 4, "crt": 4, "Or": [4, 6], "csr": 4, "ca": 4, "fill": [4, 6], "prompt": 4, "fqdn": 4, "field": 4, "rm": 4, "f": 4, "place": [4, 7, 11], "mv": 4, "660": 4, "server_ip": 4, "publicbaseurl": 4, "note": 4, "connexion": 4, "9200": [4, 11], "5601": 4, "past": [4, 10], "verif": [4, 11], "put": [4, 11], "browser": 4, "setup": [4, 5, 11], "encryptedsavedobject": 4, "encryptionkei": 4, "xxxx": 4, "now": [4, 7], "right": [4, 7], "click": [4, 7], "export": 4, "ndjson": 4, "provid": [4, 7], "consol": [4, 11], "stack": 4, "manag": [4, 11], "hamburg": 4, "menu": [4, 7], "overwrit": 4, "restor": 4, "someon": 4, "els": 4, "permiss": [4, 11], "control": 4, "commerci": [4, 5], "pack": 4, "chang": [4, 7, 10, 11], "wai": [4, 7], "releas": [4, 6], "login": 4, "checkbox": 4, "dmarc_aggreg": 4, "dmarc_forens": 4, "conform": 4, "each": [4, 6, 10, 11], "easi": 4, "regul": [4, 6, 11], "gdpr": 4, "fore": 4, "effici": 4, "help": 5, "maintain": 5, "develop": 5, "consid": [5, 7], "review": [5, 7], "how": 5, "contribut": 5, "assist": 5, "pin": 5, "particularli": [5, 11], "thank": [5, 9], "contributor": 5, "cli": 5, "util": 5, "kibana": [5, 10], "altern": [5, 11], "agari": 5, "brand": [5, 7], "dmarcian": 5, "ondmarc": 5, "proofpoint": 5, "fraud": 5, "defens": 5, "valimail": 5, "draft": [5, 9], "rua": [5, 6], "failur": [5, 7, 9, 11], "ruf": [5, 6, 7, 11], "gmail": [5, 7, 11], "transpar": 5, "handl": [5, 11], "compress": 5, "structur": 5, "simpl": 5, "premad": [5, 10], "apach": 5, "kafka": [5, 11], "multipl": 5, "geoipupd": 5, "systemd": 5, "pattern": [5, 7], "retent": 5, "owa": 5, "ew": 5, "davmail": 5, "understand": [5, 7], "align": [5, 7, 9], "what": 5, "sender": [5, 7, 8], "won": 5, "t": [5, 8, 11], "dkim": [5, 7, 8, 9], "bug": 5, "tabl": [5, 7], "anoth": [6, 11], "solut": 6, "two": 6, "mailto": 6, "uri": 6, "tag": 6, "comma": [6, 11], "behind": 6, "environ": 6, "detail": [6, 7], "http_proxi": 6, "prox": 6, "3128": 6, "https_proxi": 6, "ftp_proxi": 6, "credenti": [6, 11], "wide": [6, 9], "patch": 6, "2010": [6, 9], "rollup": 6, "22": 6, "kb4295699": 6, "2013": 6, "cumul": 6, "21": 6, "kb4099855": 6, "2016": 6, "11": [6, 9], "kb4134118": 6, "static": 6, "copi": [6, 10], "lite": 6, "databas": 6, "ipdb": 6, "distribut": 6, "term": 6, "creativ": 6, "attribut": 6, "intern": 6, "licens": 6, "fallback": 6, "geolite2": 6, "howev": 6, "cannot": 6, "tool": [6, 11], "locat": [6, 7], "overridden": 6, "buster": 6, "compon": 6, "contrib": 6, "repositori": [6, 10], "ppa": 6, "dnf": 6, "build": 6, "maco": 6, "window": 6, "decemb": 6, "30th": 6, "2019": 6, "free": 6, "account": [6, 7], "order": 6, "variou": 6, "regist": 6, "differ": [6, 7, 11], "older": [6, 9], "newer": 6, "Be": 6, "select": 6, "correct": 6, "v": [6, 11], "onc": 6, "pre": 6, "geoip": 6, "conf": 6, "systemdr": 6, "programdata": 6, "citi": 6, "asn": 6, "weekli": 6, "tuesdai": 6, "cron": 6, "schedul": 6, "task": 6, "python3": 6, "pip": 6, "virtualenv": 6, "dev": [6, 11], "libxml2": 6, "libxslt": 6, "python39": 6, "setuptool": 6, "devel": 6, "mkdir": 6, "b": [6, 9], "venv": [6, 11], "those": 6, "explicitli": 6, "9": 6, "insid": 6, "abl": 6, "libemail": 6, "friendli": 7, "incom": [7, 11], "switch": 7, "left": 7, "side": 7, "suggest": 7, "best": 7, "across": 7, "three": 7, "pie": 7, "chart": 7, "percentag": 7, "spf": [7, 9], "segment": 7, "malici": [7, 11], "just": 7, "especi": 7, "collect": [7, 11], "mai": [7, 11], "legitim": [7, 11], "correctli": 7, "while": [7, 11], "remain": 7, "valid": [7, 11], "often": 7, "rule": [7, 11], "wherea": 7, "reli": 7, "session": 7, "underneath": 7, "passag": 7, "disposit": [7, 9], "center": 7, "sort": [7, 11], "volum": 7, "By": [7, 11], "hover": 7, "mous": 7, "magnifi": 7, "glass": 7, "icon": 7, "our": 7, "recogn": 7, "market": 7, "plu": 7, "That": 7, "busi": 7, "particular": 7, "With": 7, "contact": 7, "lot": 7, "b2c": 7, "custom": [7, 11], "high": 7, "come": 7, "consum": 7, "googl": [7, 11], "yahoo": 7, "old": 7, "mention": 7, "earlier": 7, "similar": 7, "observ": 7, "who": 7, "addresse": 7, "parent": 7, "subsidiari": 7, "outdat": 7, "further": 7, "down": 7, "were": [7, 11], "call": 7, "been": [7, 11], "consolid": 7, "view": [7, 11], "own": [7, 10], "temporari": 7, "upper": 7, "These": 7, "recipi": 7, "avoid": 7, "leak": 7, "notabl": 7, "chines": 7, "suppli": [7, 11], "few": [7, 11], "wiki": 9, "schema": 9, "7480": 9, "appendix": 9, "c": [9, 11], "produc": 9, "normal": [9, 11], "regardless": 9, "xml_schema": 9, "report_metadata": 9, "org_nam": 9, "acm": 9, "org_email": 9, "org_extra_contact_info": 9, "report_id": 9, "9391651994964116463": 9, "begin_d": 9, "2012": 9, "04": 9, "27": 9, "20": 9, "00": 9, "end_dat": 9, "28": 9, "19": 9, "59": 9, "policy_publish": 9, "adkim": 9, "aspf": 9, "pct": 9, "100": 9, "fo": 9, "72": 9, "150": 9, "241": 9, "94": 9, "adsl": 9, "shv": 9, "bellsouth": 9, "base_domain": 9, "policy_evalu": 9, "policy_override_reason": 9, "identifi": 9, "header_from": 9, "envelope_from": 9, "envelope_to": 9, "null": 9, "auth_result": 9, "selector": 9, "scope": [9, 11], "mfrom": 9, "source_ip_address": 9, "source_countri": 9, "source_reverse_dn": 9, "source_base_domain": 9, "spf_align": 9, "dkim_align": 9, "dmarc_align": 9, "policy_override_com": 9, "dkim_domain": 9, "dkim_selector": 9, "dkim_result": 9, "spf_domain": 9, "spf_scope": 9, "spf_result": 9, "xennn": 9, "anonym": 9, "feedback_typ": 9, "user_ag": 9, "lua": 9, "original_mail_from": 9, "sharepoint": 9, "de": 9, "original_rcpt_to": 9, "peter": 9, "pan": 9, "arrival_d": 9, "mon": 9, "01": 9, "oct": 9, "2018": 9, "0200": 9, "message_id": 9, "38": 9, "e7": 9, "30937": 9, "bd6e1bb5": 9, "mailrelai": 9, "authentication_result": 9, "di": 9, "delivery_result": 9, "auth_failur": 9, "reported_domain": 9, "arrival_date_utc": 9, "09": 9, "authentication_mechan": 9, "original_envelope_id": 9, "sample_headers_onli": 9, "servernameon": 9, "n": [9, 11], "tby": 9, "cest": 9, "ndate": 9, "nmessag": 9, "nto": 9, "nfrom": 9, "utf": 9, "sw50zxjha3rpdmugv2v0dgjld2vyymvylcocymvyc2ljahq": 9, "nsubject": 9, "nmime": 9, "nx": 9, "mailer": 9, "foundat": 9, "ncontent": 9, "charset": 9, "transfer": 9, "quot": 9, "printabl": 9, "head": 9, "href": 9, "3d": 9, "nwettbewerb": 9, "doctyp": 9, "w3c": 9, "dtd": 9, "meta": 9, "08": 9, "0240": 9, "003": 9, "parsed_sampl": 9, "display_nam": 9, "interakt": 9, "wettbewerb": 9, "\u00fcbersicht": 9, "to_domain": 9, "timezon": 9, "mime": 9, "hop": 9, "date_utc": 9, "has_defect": 9, "reply_to": 9, "filename_safe_subject": 9, "collector": [10, 11], "editor": 10, "occurr": 10, "layout": 10, "although": 10, "slightli": 10, "easier": 10, "flexibl": 10, "usag": 11, "h": 11, "config_fil": 11, "verbos": 11, "debug": 11, "log_fil": 11, "file_path": 11, "posit": 11, "argument": 11, "exit": 11, "silent": 11, "impli": 11, "write": 11, "print": 11, "warn": 11, "program": 11, "describ": 11, "comment": 11, "save_aggreg": 11, "save_forens": 11, "dmarcresport": 11, "upersecur": 11, "splunk_hec": 11, "splunkhec": 11, "hectokengoesher": 11, "s3": 11, "bucket": 11, "my": 11, "localhost": 11, "514": 11, "full": 11, "period": 11, "n_proc": 11, "chunk_siz": 11, "larger": 11, "improv": 11, "thousand": 11, "label": 11, "arriv": 11, "993": 11, "escap": 11, "wherev": 11, "section": 11, "recommend": 11, "try": 11, "skip_certificate_verif": 11, "skip": 11, "msgraph": 11, "auth_method": 11, "method": 11, "usernamepassword": 11, "devicecod": 11, "clientsecret": 11, "m365": 11, "client_id": 11, "app": 11, "registr": 11, "client_secret": 11, "secret": 11, "tenant_id": 11, "azur": 11, "tenant": 11, "token_fil": 11, "allow_unencrypted_storag": 11, "fall": 11, "back": 11, "unencrypt": 11, "grant": 11, "readwrit": 11, "deleg": 11, "applic": 11, "restrict": 11, "sinc": 11, "applicationaccesspolici": 11, "powershel": 11, "accessright": 11, "restrictaccess": 11, "appid": 11, "policyscopegroupid": 11, "special": 11, "cert_path": 11, "trust": 11, "appli": 11, "passsword": 11, "aggregate_top": 11, "topic": 11, "forensic_top": 11, "25": 11, "starttl": 11, "upload": 11, "region_nam": 11, "region": 11, "endpoint_url": 11, "endpoint": 11, "access_key_id": 11, "secret_access_kei": 11, "udp": 11, "gmail_api": 11, "credentials_fil": 11, "include_spam_trash": 11, "spam": 11, "trash": 11, "acquir": 11, "googleapi": 11, "oauth2_port": 11, "tcp": 11, "oauth2": 11, "8080": 11, "log_analyt": 11, "resid": 11, "dce": 11, "ingest": 11, "dcr_immutable_id": 11, "immut": 11, "dcr": 11, "dcr_aggregate_stream": 11, "stream": 11, "dcr_forensic_stream": 11, "regard": 11, "strongli": 11, "much": 11, "faster": 11, "reliabl": 11, "cisco": 11, "opendn": 11, "outsid": 11, "instanc": 11, "highli": 11, "industri": 11, "sensit": 11, "healthcar": 11, "financ": 11, "possibl": 11, "appear": 11, "sometim": 11, "kind": 11, "approach": 11, "manual": 11, "1000": 11, "per": 11, "analyz": 11, "year": 11, "_cluster": 11, "health": 11, "pretti": 11, "active_primary_shard": 11, "932": 11, "active_shard": 11, "2k": 11, "persist": 11, "max_shards_per_nod": 11, "2000": 11, "watcher": 11, "io": 11}, "objects": {"": [[0, 0, 0, "-", "parsedmarc"]], "parsedmarc": [[0, 1, 1, "", "InvalidAggregateReport"], [0, 1, 1, "", "InvalidDMARCReport"], [0, 1, 1, "", "InvalidForensicReport"], [0, 1, 1, "", "ParserError"], [0, 0, 0, "-", "elastic"], [0, 2, 1, "", "email_results"], [0, 2, 1, "", "extract_xml"], [0, 2, 1, "", "get_dmarc_reports_from_mailbox"], [0, 2, 1, "", "get_dmarc_reports_from_mbox"], [0, 2, 1, "", "get_report_zip"], [0, 2, 1, "", "parse_aggregate_report_file"], [0, 2, 1, "", "parse_aggregate_report_xml"], [0, 2, 1, "", "parse_forensic_report"], [0, 2, 1, "", "parse_report_email"], [0, 2, 1, "", "parse_report_file"], [0, 2, 1, "", "parsed_aggregate_reports_to_csv"], [0, 2, 1, "", "parsed_aggregate_reports_to_csv_rows"], [0, 2, 1, "", "parsed_forensic_reports_to_csv"], [0, 2, 1, "", "parsed_forensic_reports_to_csv_rows"], [0, 2, 1, "", "save_output"], [0, 0, 0, "-", "splunk"], [0, 0, 0, "-", "utils"], [0, 2, 1, "", "watch_inbox"]], "parsedmarc.elastic": [[0, 1, 1, "", "AlreadySaved"], [0, 1, 1, "", "ElasticsearchError"], [0, 2, 1, "", "create_indexes"], [0, 2, 1, "", "migrate_indexes"], [0, 2, 1, "", "save_aggregate_report_to_elasticsearch"], [0, 2, 1, "", "save_forensic_report_to_elasticsearch"], [0, 2, 1, "", "set_hosts"]], "parsedmarc.splunk": [[0, 3, 1, "", "HECClient"], [0, 1, 1, "", "SplunkError"]], "parsedmarc.splunk.HECClient": [[0, 4, 1, "", "save_aggregate_reports_to_splunk"], [0, 4, 1, "", "save_forensic_reports_to_splunk"]], "parsedmarc.utils": [[0, 1, 1, "", "DownloadError"], [0, 1, 1, "", "EmailParserError"], [0, 2, 1, "", "convert_outlook_msg"], [0, 2, 1, "", "decode_base64"], [0, 2, 1, "", "get_base_domain"], [0, 2, 1, "", "get_filename_safe_string"], [0, 2, 1, "", "get_ip_address_country"], [0, 2, 1, "", "get_ip_address_info"], [0, 2, 1, "", "get_reverse_dns"], [0, 2, 1, "", "human_timestamp_to_datetime"], [0, 2, 1, "", "human_timestamp_to_timestamp"], [0, 2, 1, "", "is_mbox"], [0, 2, 1, "", "is_outlook_msg"], [0, 2, 1, "", "parse_email"], [0, 2, 1, "", "query_dns"], [0, 2, 1, "", "timestamp_to_datetime"], [0, 2, 1, "", "timestamp_to_human"]]}, "objtypes": {"0": "py:module", "1": "py:exception", "2": "py:function", "3": "py:class", "4": "py:method"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "exception", "Python exception"], "2": ["py", "function", "Python function"], "3": ["py", "class", "Python class"], "4": ["py", "method", "Python method"]}, "titleterms": {"api": 0, "refer": 0, "parsedmarc": [0, 1, 2, 5, 6, 11], "elast": 0, "splunk": [0, 10], "util": 0, "indic": 0, "tabl": 0, "contribut": 1, "bug": 1, "report": [1, 5, 6, 9], "access": 2, "an": 2, "inbox": 2, "us": [2, 7, 11], "owa": 2, "ew": 2, "run": [2, 11], "davmail": 2, "systemd": [2, 11], "servic": [2, 11], "configur": [2, 11], "understand": 3, "dmarc": [3, 5, 7], "resourc": 3, "guid": 3, "spf": 3, "record": [3, 4], "valid": 3, "lookalik": 3, "domain": 3, "align": 3, "what": [3, 8], "sender": 3, "won": 3, "t": 3, "support": 3, "dkim": 3, "about": [3, 8], "mail": [3, 8], "list": [3, 8], "best": [3, 8], "practic": [3, 8], "do": [3, 8], "mailman": [3, 8], "2": [3, 8], "3": [3, 8], "listserv": [3, 8], "workaround": [3, 8], "elasticsearch": 4, "kibana": [4, 7], "upgrad": 4, "index": 4, "pattern": 4, "retent": 4, "document": 5, "open": 5, "sourc": 5, "analyz": [5, 6], "visual": 5, "featur": 5, "content": 5, "instal": 6, "test": 6, "multipl": 6, "geoipupd": 6, "setup": 6, "option": 6, "depend": 6, "dashboard": 7, "summari": 7, "forens": [7, 9], "sampl": [7, 9], "output": 9, "aggreg": 9, "json": 9, "csv": 9, "cli": 11, "help": 11, "file": 11}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.todo": 2, "sphinx.ext.viewcode": 1, "sphinx": 57}, "alltitles": {"API reference": [[0, "api-reference"]], "parsedmarc": [[0, "module-parsedmarc"]], "parsedmarc.elastic": [[0, "module-parsedmarc.elastic"]], "parsedmarc.splunk": [[0, "module-parsedmarc.splunk"]], "parsedmarc.utils": [[0, "module-parsedmarc.utils"]], "Indices and tables": [[0, "indices-and-tables"]], "Contributing to parsedmarc": [[1, "contributing-to-parsedmarc"]], "Bug reports": [[1, "bug-reports"]], "Accessing an inbox using OWA/EWS": [[2, "accessing-an-inbox-using-owa-ews"]], "Running DavMail as a systemd service": [[2, "running-davmail-as-a-systemd-service"]], "Configuring parsedmarc for DavMail": [[2, "configuring-parsedmarc-for-davmail"]], "Understanding DMARC": [[3, "understanding-dmarc"]], "Resources": [[3, "resources"]], "DMARC guides": [[3, "dmarc-guides"]], "SPF and DMARC record validation": [[3, "spf-and-dmarc-record-validation"]], "Lookalike domains": [[3, "lookalike-domains"]], "DMARC Alignment Guide": [[3, "dmarc-alignment-guide"]], "What if a sender won\u2019t support DKIM/DMARC?": [[3, "what-if-a-sender-wont-support-dkim-dmarc"]], "What about mailing lists?": [[3, "what-about-mailing-lists"], [8, "what-about-mailing-lists"]], "Mailing list best practices": [[3, "mailing-list-best-practices"], [8, "mailing-list-best-practices"]], "Do": [[3, "do"], [8, "do"]], "Do not": [[3, "do-not"], [8, "do-not"]], "Mailman 2": [[3, "mailman-2"], [3, "id1"], [8, "mailman-2"], [8, "id1"]], "Mailman 3": [[3, "mailman-3"], [3, "id2"], [8, "mailman-3"], [8, "id2"]], "LISTSERV": [[3, "listserv"], [8, "listserv"]], "Workarounds": [[3, "workarounds"], [8, "workarounds"]], "Elasticsearch and Kibana": [[4, "elasticsearch-and-kibana"]], "Upgrading Kibana index patterns": [[4, "upgrading-kibana-index-patterns"]], "Records retention": [[4, "records-retention"]], "parsedmarc documentation - Open source DMARC report analyzer and visualizer": [[5, "parsedmarc-documentation-open-source-dmarc-report-analyzer-and-visualizer"]], "Features": [[5, "features"]], "Contents": [[5, null]], "Installation": [[6, "installation"]], "Testing multiple report analyzers": [[6, "testing-multiple-report-analyzers"]], "geoipupdate setup": [[6, "geoipupdate-setup"]], "Installing parsedmarc": [[6, "installing-parsedmarc"]], "Optional dependencies": [[6, "optional-dependencies"]], "Using the Kibana dashboards": [[7, "using-the-kibana-dashboards"]], "DMARC Summary": [[7, "dmarc-summary"]], "DMARC Forensic Samples": [[7, "dmarc-forensic-samples"]], "Sample outputs": [[9, "sample-outputs"]], "Sample aggregate report output": [[9, "sample-aggregate-report-output"]], "JSON aggregate report": [[9, "json-aggregate-report"]], "CSV aggregate report": [[9, "csv-aggregate-report"]], "Sample forensic report output": [[9, "sample-forensic-report-output"]], "JSON forensic report": [[9, "json-forensic-report"]], "CSV forensic report": [[9, "csv-forensic-report"]], "Splunk": [[10, "splunk"]], "Using parsedmarc": [[11, "using-parsedmarc"]], "CLI help": [[11, "cli-help"]], "Configuration file": [[11, "configuration-file"]], "Running parsedmarc as a systemd service": [[11, "running-parsedmarc-as-a-systemd-service"]]}, "indexentries": {"alreadysaved": [[0, "parsedmarc.elastic.AlreadySaved"]], "downloaderror": [[0, "parsedmarc.utils.DownloadError"]], "elasticsearcherror": [[0, "parsedmarc.elastic.ElasticsearchError"]], "emailparsererror": [[0, "parsedmarc.utils.EmailParserError"]], "hecclient (class in parsedmarc.splunk)": [[0, "parsedmarc.splunk.HECClient"]], "invalidaggregatereport": [[0, "parsedmarc.InvalidAggregateReport"]], "invaliddmarcreport": [[0, "parsedmarc.InvalidDMARCReport"]], "invalidforensicreport": [[0, "parsedmarc.InvalidForensicReport"]], "parsererror": [[0, "parsedmarc.ParserError"]], "splunkerror": [[0, "parsedmarc.splunk.SplunkError"]], "convert_outlook_msg() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.convert_outlook_msg"]], "create_indexes() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.create_indexes"]], "decode_base64() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.decode_base64"]], "email_results() (in module parsedmarc)": [[0, "parsedmarc.email_results"]], "extract_xml() (in module parsedmarc)": [[0, "parsedmarc.extract_xml"]], "get_base_domain() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_base_domain"]], "get_dmarc_reports_from_mailbox() (in module parsedmarc)": [[0, "parsedmarc.get_dmarc_reports_from_mailbox"]], "get_dmarc_reports_from_mbox() (in module parsedmarc)": [[0, "parsedmarc.get_dmarc_reports_from_mbox"]], "get_filename_safe_string() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_filename_safe_string"]], "get_ip_address_country() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_ip_address_country"]], "get_ip_address_info() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_ip_address_info"]], "get_report_zip() (in module parsedmarc)": [[0, "parsedmarc.get_report_zip"]], "get_reverse_dns() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.get_reverse_dns"]], "human_timestamp_to_datetime() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.human_timestamp_to_datetime"]], "human_timestamp_to_timestamp() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.human_timestamp_to_timestamp"]], "is_mbox() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.is_mbox"]], "is_outlook_msg() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.is_outlook_msg"]], "migrate_indexes() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.migrate_indexes"]], "module": [[0, "module-parsedmarc"], [0, "module-parsedmarc.elastic"], [0, "module-parsedmarc.splunk"], [0, "module-parsedmarc.utils"]], "parse_aggregate_report_file() (in module parsedmarc)": [[0, "parsedmarc.parse_aggregate_report_file"]], "parse_aggregate_report_xml() (in module parsedmarc)": [[0, "parsedmarc.parse_aggregate_report_xml"]], "parse_email() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.parse_email"]], "parse_forensic_report() (in module parsedmarc)": [[0, "parsedmarc.parse_forensic_report"]], "parse_report_email() (in module parsedmarc)": [[0, "parsedmarc.parse_report_email"]], "parse_report_file() (in module parsedmarc)": [[0, "parsedmarc.parse_report_file"]], "parsed_aggregate_reports_to_csv() (in module parsedmarc)": [[0, "parsedmarc.parsed_aggregate_reports_to_csv"]], "parsed_aggregate_reports_to_csv_rows() (in module parsedmarc)": [[0, "parsedmarc.parsed_aggregate_reports_to_csv_rows"]], "parsed_forensic_reports_to_csv() (in module parsedmarc)": [[0, "parsedmarc.parsed_forensic_reports_to_csv"]], "parsed_forensic_reports_to_csv_rows() (in module parsedmarc)": [[0, "parsedmarc.parsed_forensic_reports_to_csv_rows"]], "parsedmarc": [[0, "module-parsedmarc"]], "parsedmarc.elastic": [[0, "module-parsedmarc.elastic"]], "parsedmarc.splunk": [[0, "module-parsedmarc.splunk"]], "parsedmarc.utils": [[0, "module-parsedmarc.utils"]], "query_dns() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.query_dns"]], "save_aggregate_report_to_elasticsearch() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.save_aggregate_report_to_elasticsearch"]], "save_aggregate_reports_to_splunk() (parsedmarc.splunk.hecclient method)": [[0, "parsedmarc.splunk.HECClient.save_aggregate_reports_to_splunk"]], "save_forensic_report_to_elasticsearch() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.save_forensic_report_to_elasticsearch"]], "save_forensic_reports_to_splunk() (parsedmarc.splunk.hecclient method)": [[0, "parsedmarc.splunk.HECClient.save_forensic_reports_to_splunk"]], "save_output() (in module parsedmarc)": [[0, "parsedmarc.save_output"]], "set_hosts() (in module parsedmarc.elastic)": [[0, "parsedmarc.elastic.set_hosts"]], "timestamp_to_datetime() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.timestamp_to_datetime"]], "timestamp_to_human() (in module parsedmarc.utils)": [[0, "parsedmarc.utils.timestamp_to_human"]], "watch_inbox() (in module parsedmarc)": [[0, "parsedmarc.watch_inbox"]]}}) \ No newline at end of file diff --git a/splunk.html b/splunk.html index b1174c3..dcbefea 100644 --- a/splunk.html +++ b/splunk.html @@ -113,7 +113,7 @@ easier and more flexible filtering options.

          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a diff --git a/usage.html b/usage.html index 7bf7aa8..2be4bbc 100644 --- a/usage.html +++ b/usage.html @@ -565,7 +565,7 @@ as well as the current process (newest to oldest), run:

          -

          © Copyright 2018, Sean Whalen.

          +

          © Copyright 2018 - 2023, Sean Whalen and contributors.

          Built with Sphinx using a