Improve test to explicitly demonstrate case-insensitive handling of folder names like 'Inbox'

Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>
Add unit test for MSGraph well-known folder name mapping
2026-02-17 23:16:26 +00:00 · 2025-12-31 21:00:38 +00:00 · 2025-12-31 20:47:47 +00:00 · 2025-12-31 20:44:56 +00:00 · 2025-12-31 20:43:44 +00:00 · 2025-12-31 20:39:17 +00:00
46 changed files with 3621 additions and 1650 deletions
--- a/.github/workflows/docker.yml
+++ b/.github/workflows/docker.yml
@@ -24,11 +24,11 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v5

      - name: Docker meta
        id: meta
-        uses: docker/metadata-action@v3
+        uses: docker/metadata-action@v5
        with:
          images: |
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
@@ -40,16 +40,14 @@ jobs:
            type=semver,pattern={{major}}.{{minor}}

      - name: Log in to the Container registry
-        # https://github.com/docker/login-action/releases/tag/v2.0.0
-        uses: docker/login-action@49ed152c8eca782a232dede0303416e8f356c37b
+        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push Docker image
-        # https://github.com/docker/build-push-action/releases/tag/v3.0.0
-        uses: docker/build-push-action@e551b19e49efd4e98792db7592c17c09b89db8d8
+        uses: docker/build-push-action@v6
        with:
          context: .
          push: ${{ github.event_name == 'release' }}
--- a/.github/workflows/python-tests.yml
+++ b/.github/workflows/python-tests.yml
@@ -15,7 +15,7 @@ jobs:

    services:
      elasticsearch:
-        image: elasticsearch:8.18.2
+        image: elasticsearch:8.19.7
        env:
          discovery.type: single-node
          cluster.name: parsedmarc-cluster
@@ -30,18 +30,18 @@ jobs:
    strategy:
      fail-fast: false
      matrix:
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]

    steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v5
    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v5
+      uses: actions/setup-python@v6
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install system dependencies
      run: |
-        sudo apt-get update
-        sudo apt-get install -y libemail-outlook-message-perl
+        sudo apt-get -q update
+        sudo apt-get -qy install libemail-outlook-message-perl
    - name: Install Python dependencies
      run: |
        python -m pip install --upgrade pip
@@ -65,6 +65,6 @@ jobs:
      run: |
        hatch build
    - name: Upload coverage to Codecov
-      uses: codecov/codecov-action@v4
+      uses: codecov/codecov-action@v5
      with:
          token: ${{ secrets.CODECOV_TOKEN }}
--- a/.gitignore
+++ b/.gitignore
@@ -106,7 +106,7 @@ ENV/
 .idea/

 # VS Code launch config
-.vscode/launch.json
+#.vscode/launch.json

 # Visual Studio Code settings
 #.vscode/
@@ -142,4 +142,6 @@ scratch.py

 parsedmarc/resources/maps/base_reverse_dns.csv
 parsedmarc/resources/maps/unknown_base_reverse_dns.csv
+parsedmarc/resources/maps/sus_domains.csv
+parsedmarc/resources/maps/unknown_domains.txt
 *.bak
--- a/.vscode/launch.json
+++ b/.vscode/launch.json
@@ -0,0 +1,45 @@
+{
+  // Use IntelliSense to learn about possible attributes.
+  // Hover to view descriptions of existing attributes.
+  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "name": "Python Debugger: Current File",
+      "type": "debugpy",
+      "request": "launch",
+      "program": "${file}",
+      "console": "integratedTerminal"
+    },
+    {
+      "name": "tests.py",
+      "type": "debugpy",
+      "request": "launch",
+      "program": "tests.py",
+      "console": "integratedTerminal"
+    },
+    {
+      "name": "sample",
+      "type": "debugpy",
+      "request": "launch",
+      "module": "parsedmarc.cli",
+      "args": ["samples/private/sample"]
+    },
+    {
+      "name": "sortlists.py",
+      "type": "debugpy",
+      "request": "launch",
+      "program": "sortlists.py",
+      "cwd": "${workspaceFolder}/parsedmarc/resources/maps",
+      "console": "integratedTerminal"
+    },
+    {
+      "name": "find_unknown_base_reverse_dns.py",
+      "type": "debugpy",
+      "request": "launch",
+      "program": "find_unknown_base_reverse_dns.py",
+      "cwd": "${workspaceFolder}/parsedmarc/resources/maps",
+      "console": "integratedTerminal"
+    }
+  ]
+}
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -1,143 +1,166 @@
 {
+  "[python]": {
+    "editor.defaultFormatter": "charliermarsh.ruff",
+    "editor.formatOnSave": true,
+
+    // Let Ruff handle lint fixes + import sorting on save
+    "editor.codeActionsOnSave": {
+      "source.fixAll.ruff": "explicit",
+      "source.organizeImports.ruff": "explicit"
+    }
+  },
    "markdownlint.config": {
        "MD024": false
    },
    "cSpell.words": [
-        "adkim",
-        "akamaiedge",
-        "amsmath",
-        "andrewmcgilvray",
-        "arcname",
-        "aspf",
-        "autoclass",
-        "automodule",
-        "backported",
-        "bellsouth",
-        "boto",
-        "brakhane",
-        "Brightmail",
-        "CEST",
-        "CHACHA",
-        "checkdmarc",
-        "Codecov",
-        "confnew",
-        "dateparser",
-        "dateutil",
-        "Davmail",
-        "DBIP",
-        "dearmor",
-        "deflist",
-        "devel",
-        "DMARC",
-        "Dmarcian",
-        "dnspython",
-        "dollarmath",
-        "dpkg",
-        "exampleuser",
-        "expiringdict",
-        "fieldlist",
-        "genindex",
-        "geoip",
-        "geoipupdate",
-        "Geolite",
-        "geolocation",
-        "githubpages",
-        "Grafana",
-        "hostnames",
-        "htpasswd",
-        "httpasswd",
-        "httplib",
-        "IMAP",
-        "imapclient",
-        "infile",
-        "Interaktive",
-        "IPDB",
-        "journalctl",
-        "keepalive",
-        "keyout",
-        "keyrings",
-        "Leeman",
-        "libemail",
-        "linkify",
-        "LISTSERV",
-        "lxml",
-        "mailparser",
-        "mailrelay",
-        "mailsuite",
-        "maxdepth",
-        "maxmind",
-        "mbox",
-        "mfrom",
-        "michaeldavie",
-        "mikesiegel",
-        "mitigations",
-        "MMDB",
-        "modindex",
-        "msgconvert",
-        "msgraph",
-        "MSSP",
-        "Munge",
-        "ndjson",
-        "newkey",
-        "Nhcm",
-        "nojekyll",
-        "nondigest",
-        "nosecureimap",
-        "nosniff",
-        "nwettbewerb",
-        "opensearch",
-        "parsedmarc",
-        "passsword",
-        "Postorius",
-        "premade",
-        "procs",
-        "publicsuffix",
-        "publicsuffixlist",
-        "publixsuffix",
-        "pygelf",
-        "pypy",
-        "pytest",
-        "quickstart",
-        "Reindex",
-        "replyto",
-        "reversename",
-        "Rollup",
-        "Rpdm",
-        "SAMEORIGIN",
-        "sdist",
-        "Servernameone",
-        "setuptools",
-        "smartquotes",
-        "SMTPTLS",
-        "sortmaps",
-        "sourcetype",
-        "STARTTLS",
-        "tasklist",
-        "timespan",
-        "tlsa",
-        "tlsrpt",
-        "toctree",
-        "TQDDM",
-        "tqdm",
-        "truststore",
-        "Übersicht",
-        "uids",
-        "unparasable",
-        "uper",
-        "urllib",
-        "Valimail",
-        "venv",
-        "Vhcw",
-        "viewcode",
-        "virtualenv",
-        "WBITS",
-        "webmail",
-        "Wettbewerber",
-        "Whalen",
-        "whitespaces",
-        "xennn",
-        "xmltodict",
-        "xpack",
-        "zscholl"
+      "adkim",
+      "akamaiedge",
+      "amsmath",
+      "andrewmcgilvray",
+      "arcname",
+      "aspf",
+      "autoclass",
+      "automodule",
+      "backported",
+      "bellsouth",
+      "boto",
+      "brakhane",
+      "Brightmail",
+      "CEST",
+      "CHACHA",
+      "checkdmarc",
+      "Codecov",
+      "confnew",
+      "dateparser",
+      "dateutil",
+      "Davmail",
+      "DBIP",
+      "dearmor",
+      "deflist",
+      "devel",
+      "DMARC",
+      "Dmarcian",
+      "dnspython",
+      "dollarmath",
+      "dpkg",
+      "exampleuser",
+      "expiringdict",
+      "fieldlist",
+      "GELF",
+      "genindex",
+      "geoip",
+      "geoipupdate",
+      "Geolite",
+      "geolocation",
+      "githubpages",
+      "Grafana",
+      "hostnames",
+      "htpasswd",
+      "httpasswd",
+      "httplib",
+      "ifhost",
+      "IMAP",
+      "imapclient",
+      "infile",
+      "Interaktive",
+      "IPDB",
+      "journalctl",
+      "kafkaclient",
+      "keepalive",
+      "keyout",
+      "keyrings",
+      "Leeman",
+      "libemail",
+      "linkify",
+      "LISTSERV",
+      "loganalytics",
+      "lxml",
+      "mailparser",
+      "mailrelay",
+      "mailsuite",
+      "maxdepth",
+      "MAXHEADERS",
+      "maxmind",
+      "mbox",
+      "mfrom",
+      "mhdw",
+      "michaeldavie",
+      "mikesiegel",
+      "Mimecast",
+      "mitigations",
+      "MMDB",
+      "modindex",
+      "msgconvert",
+      "msgraph",
+      "MSSP",
+      "multiprocess",
+      "Munge",
+      "ndjson",
+      "newkey",
+      "Nhcm",
+      "nojekyll",
+      "nondigest",
+      "nosecureimap",
+      "nosniff",
+      "nwettbewerb",
+      "opensearch",
+      "opensearchpy",
+      "parsedmarc",
+      "passsword",
+      "pbar",
+      "Postorius",
+      "premade",
+      "privatesuffix",
+      "procs",
+      "publicsuffix",
+      "publicsuffixlist",
+      "publixsuffix",
+      "pygelf",
+      "pypy",
+      "pytest",
+      "quickstart",
+      "Reindex",
+      "replyto",
+      "reversename",
+      "Rollup",
+      "Rpdm",
+      "SAMEORIGIN",
+      "sdist",
+      "Servernameone",
+      "setuptools",
+      "smartquotes",
+      "SMTPTLS",
+      "sortlists",
+      "sortmaps",
+      "sourcetype",
+      "STARTTLS",
+      "tasklist",
+      "timespan",
+      "tlsa",
+      "tlsrpt",
+      "toctree",
+      "TQDDM",
+      "tqdm",
+      "truststore",
+      "Übersicht",
+      "uids",
+      "Uncategorized",
+      "unparasable",
+      "uper",
+      "urllib",
+      "Valimail",
+      "venv",
+      "Vhcw",
+      "viewcode",
+      "virtualenv",
+      "WBITS",
+      "webmail",
+      "Wettbewerber",
+      "Whalen",
+      "whitespaces",
+      "xennn",
+      "xmltodict",
+      "xpack",
+      "zscholl"
    ],
 }
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
--- a/2
+++ b/2
@@ -1,4 +1,4 @@
-ARG BASE_IMAGE=python:3.9-slim
+ARG BASE_IMAGE=python:3.13-slim
 ARG USERNAME=parsedmarc
 ARG USER_UID=1000
 ARG USER_GID=$USER_UID
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@ Package](https://img.shields.io/pypi/v/parsedmarc.svg)](https://pypi.org/project
 [![PyPI - Downloads](https://img.shields.io/pypi/dm/parsedmarc?color=blue)](https://pypistats.org/packages/parsedmarc)

 <p align="center">
-  <img src="https://github.com/domainaware/parsedmarc/raw/master/docs/source/_static/screenshots/dmarc-summary-charts.png?raw=true" alt="A screenshot of DMARC summary charts in Kibana"/>
+  <img src="https://raw.githubusercontent.com/domainaware/parsedmarc/refs/heads/master/docs/source/_static/screenshots/dmarc-summary-charts.png?raw=true" alt="A screenshot of DMARC summary charts in Kibana"/>
 </p>

 `parsedmarc` is a Python module and CLI utility for parsing DMARC
@@ -23,25 +23,42 @@ ProofPoint Email Fraud Defense, and Valimail.

 ## Help Wanted

-This project is maintained by one developer. Please consider
-reviewing the open
-[issues](https://github.com/domainaware/parsedmarc/issues) to see how
-you can contribute code, documentation, or user support. Assistance on
-the pinned issues would be particularly helpful.
+This project is maintained by one developer. Please consider reviewing the open
+[issues](https://github.com/domainaware/parsedmarc/issues) to see how you can
+contribute code, documentation, or user support. Assistance on the pinned
+issues would be particularly helpful.

 Thanks to all
 [contributors](https://github.com/domainaware/parsedmarc/graphs/contributors)!

 ## Features

- Parses draft and 1.0 standard aggregate/rua reports
- Parses forensic/failure/ruf reports
- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail
-    API
+- Parses draft and 1.0 standard aggregate/rua DMARC reports
+- Parses forensic/failure/ruf DMARC reports
+- Parses reports from SMTP TLS Reporting
+- Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
 - Transparently handles gzip or zip compressed reports
 - Consistent data structures
 - Simple JSON and/or CSV output
 - Optionally email the results
- Optionally send the results to Elasticsearch, Opensearch, and/or Splunk, for use
-    with premade dashboards
+- Optionally send the results to Elasticsearch, Opensearch, and/or Splunk, for
+  use with premade dashboards
 - Optionally send reports to Apache Kafka
+
+## Python Compatibility
+
+This project supports the following Python versions, which are either actively maintained or are the default versions
+for RHEL or Debian.
+
+| Version | Supported | Reason                                                     |
+|---------|-----------|------------------------------------------------------------|
+| < 3.6   | ❌         | End of Life (EOL)                                          |
+| 3.6     | ❌         | Used in RHEL 8, but not supported by project dependencies |
+| 3.7     | ❌         | End of Life (EOL)                                          |
+| 3.8     | ❌         | End of Life (EOL)                                          |
+| 3.9     | ✅         | Supported until August 2026 (Debian 11); May 2032 (RHEL 9) |
+| 3.10    | ✅         | Actively maintained                                        |
+| 3.11    | ✅         | Actively maintained; supported until June 2028 (Debian 12) |
+| 3.12    | ✅         | Actively maintained; supported until May 2035 (RHEL 10)    |
+| 3.13    | ✅         | Actively maintained; supported until June 2030 (Debian 13) |
+| 3.14    | ❌         | Not currently supported due to [this imapclient bug](https://github.com/mjs/imapclient/issues/618)|
--- a/build.sh
+++ b/build.sh
@@ -9,17 +9,16 @@ fi
 . venv/bin/activate
 pip install .[build]
 ruff format .
-ruff check .
 cd docs
 make clean 
 make html
 touch build/html/.nojekyll
-if [  -d "./../parsedmarc-docs" ]; then
+if [  -d "../../parsedmarc-docs" ]; then
  cp -rf build/html/* ../../parsedmarc-docs/
 fi
 cd ..
 cd parsedmarc/resources/maps
-python3 sortmaps.py
+python3 sortlists.py
 echo "Checking for invalid UTF-8 bytes in base_reverse_dns_map.csv"
 python3 find_bad_utf8.py base_reverse_dns_map.csv
 cd ../../..
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,8 +1,6 @@
-version: '3.7'
-
 services:
  elasticsearch:
-    image: docker.elastic.co/elasticsearch/elasticsearch:8.3.1
+    image: docker.elastic.co/elasticsearch/elasticsearch:8.19.7
    environment:
      - network.host=127.0.0.1
      - http.host=0.0.0.0
@@ -14,7 +12,7 @@ services:
      - xpack.security.enabled=false
      - xpack.license.self_generated.type=basic
    ports:
-      - 127.0.0.1:9200:9200
+      - "127.0.0.1:9200:9200"
    ulimits:
      memlock:
        soft: -1
@@ -30,7 +28,7 @@ services:
      retries: 24

  opensearch:
-    image: opensearchproject/opensearch:2.18.0
+    image: opensearchproject/opensearch:2
    environment:
      - network.host=127.0.0.1
      - http.host=0.0.0.0
@@ -41,7 +39,7 @@ services:
      - bootstrap.memory_lock=true
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ports:
-      - 127.0.0.1:9201:9200
+      - "127.0.0.1:9201:9200"
    ulimits:
      memlock:
        soft: -1
--- a/docs/source/api.md
+++ b/docs/source/api.md
@@ -21,7 +21,6 @@
   :members:
 ```

-
 ## parsedmarc.splunk

 ```{eval-rst}
@@ -29,6 +28,13 @@
   :members:
 ```

+## parsedmarc.types
+
+```{eval-rst}
+.. automodule:: parsedmarc.types
+   :members:
+```
+
 ## parsedmarc.utils

 ```{eval-rst}
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -20,7 +20,7 @@ from parsedmarc import __version__
 # -- Project information -----------------------------------------------------

 project = "parsedmarc"
-copyright = "2018 - 2023, Sean Whalen and contributors"
+copyright = "2018 - 2025, Sean Whalen and contributors"
 author = "Sean Whalen and contributors"

 # The version info for the project you're documenting, acts as replacement for
--- a/docs/source/example.ini
+++ b/docs/source/example.ini
@@ -29,3 +29,14 @@ token_file = /etc/example/token.json
 include_spam_trash = True
 paginate_messages = True
 scopes = https://www.googleapis.com/auth/gmail.modify
+
+[msgraph]
+auth_method = ClientSecret
+client_id = 12345678-90ab-cdef-1234-567890abcdef
+client_secret = your-client-secret-here
+tenant_id = 12345678-90ab-cdef-1234-567890abcdef
+mailbox = dmarc-reports@example.com
+# Use standard folder names - they work across all locales
+# and avoid "Default folder Root not found" errors
+reports_folder = Inbox
+archive_folder = Archive
--- a/docs/source/index.md
+++ b/docs/source/index.md
@@ -33,17 +33,36 @@ and Valimail.

 ## Features

- Parses draft and 1.0 standard aggregate/rua reports
- Parses forensic/failure/ruf reports
+- Parses draft and 1.0 standard aggregate/rua DMARC reports
+- Parses forensic/failure/ruf DMARC reports
+- Parses reports from SMTP TLS Reporting
 - Can parse reports from an inbox over IMAP, Microsoft Graph, or Gmail API
 - Transparently handles gzip or zip compressed reports
 - Consistent data structures
 - Simple JSON and/or CSV output
 - Optionally email the results
- Optionally send the results to Elasticsearch/OpenSearch and/or Splunk, for use with
-  premade dashboards
+- Optionally send the results to Elasticsearch, Opensearch, and/or Splunk, for use
+    with premade dashboards
 - Optionally send reports to Apache Kafka

+## Python Compatibility
+
+This project supports the following Python versions, which are either actively maintained or are the default versions
+for RHEL or Debian.
+
+| Version | Supported | Reason                                                     |
+|---------|-----------|------------------------------------------------------------|
+| < 3.6   | ❌         | End of Life (EOL)                                          |
+| 3.6     | ❌         | Used in RHEL 8, but not supported by project dependencies |
+| 3.7     | ❌         | End of Life (EOL)                                          |
+| 3.8     | ❌         | End of Life (EOL)                                          |
+| 3.9     | ✅         | Supported until August 2026 (Debian 11); May 2032 (RHEL 9) |
+| 3.10    | ✅         | Actively maintained                                        |
+| 3.11    | ✅         | Actively maintained; supported until June 2028 (Debian 12) |
+| 3.12    | ✅         | Actively maintained; supported until May 2035 (RHEL 10)    |
+| 3.13    | ✅         | Actively maintained; supported until June 2030 (Debian 13) |
+| 3.14    | ❌         | Not currently supported due to [this imapclient bug](https://github.com/mjs/imapclient/issues/618)|
+
 ```{toctree}
 :caption: 'Contents'
 :maxdepth: 2
--- a/docs/source/installation.md
+++ b/docs/source/installation.md
@@ -199,7 +199,7 @@ sudo apt-get install libemail-outlook-message-perl
 [geoipupdate releases page on github]: https://github.com/maxmind/geoipupdate/releases
 [ip to country lite database]: https://db-ip.com/db/download/ip-to-country-lite
 [license keys]: https://www.maxmind.com/en/accounts/current/license-key
-[maxmind geoipupdate page]: https://dev.maxmind.com/geoip/geoipupdate/
+[maxmind geoipupdate page]: https://dev.maxmind.com/geoip/updating-databases/
 [maxmind geolite2 country database]: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
 [registering for a free geolite2 account]: https://www.maxmind.com/en/geolite2/signup
 [to comply with various privacy regulations]: https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/
--- a/docs/source/output.md
+++ b/docs/source/output.md
@@ -23,6 +23,8 @@ of the report schema.
    "report_id": "9391651994964116463",
    "begin_date": "2012-04-27 20:00:00",
    "end_date": "2012-04-28 19:59:59",
+    "timespan_requires_normalization": false,
+    "original_timespan_seconds": 86399,
    "errors": []
  },
  "policy_published": {
@@ -39,8 +41,10 @@ of the report schema.
      "source": {
        "ip_address": "72.150.241.94",
        "country": "US",
-        "reverse_dns": "adsl-72-150-241-94.shv.bellsouth.net",
-        "base_domain": "bellsouth.net"
+        "reverse_dns": null,
+        "base_domain": null,
+        "name": null,
+        "type": null
      },
      "count": 2,
      "alignment": {
@@ -74,7 +78,10 @@ of the report schema.
            "result": "pass"
          }
        ]
-      }
+      },
+      "normalized_timespan": false,
+      "interval_begin": "2012-04-28 00:00:00",
+      "interval_end": "2012-04-28 23:59:59"
    }
  ]
 }
@@ -83,8 +90,10 @@ of the report schema.
 ### CSV aggregate report

 ```text
-xml_schema,org_name,org_email,org_extra_contact_info,report_id,begin_date,end_date,errors,domain,adkim,aspf,p,sp,pct,fo,source_ip_address,source_country,source_reverse_dns,source_base_domain,count,spf_aligned,dkim_aligned,dmarc_aligned,disposition,policy_override_reasons,policy_override_comments,envelope_from,header_from,envelope_to,dkim_domains,dkim_selectors,dkim_results,spf_domains,spf_scopes,spf_results
-draft,acme.com,noreply-dmarc-support@acme.com,http://acme.com/dmarc/support,9391651994964116463,2012-04-27 20:00:00,2012-04-28 19:59:59,,example.com,r,r,none,none,100,0,72.150.241.94,US,adsl-72-150-241-94.shv.bellsouth.net,bellsouth.net,2,True,False,True,none,,,example.com,example.com,,example.com,none,fail,example.com,mfrom,pass
+xml_schema,org_name,org_email,org_extra_contact_info,report_id,begin_date,end_date,normalized_timespan,errors,domain,adkim,aspf,p,sp,pct,fo,source_ip_address,source_country,source_reverse_dns,source_base_domain,source_name,source_type,count,spf_aligned,dkim_aligned,dmarc_aligned,disposition,policy_override_reasons,policy_override_comments,envelope_from,header_from,envelope_to,dkim_domains,dkim_selectors,dkim_results,spf_domains,spf_scopes,spf_results
+draft,acme.com,noreply-dmarc-support@acme.com,http://acme.com/dmarc/support,9391651994964116463,2012-04-28 00:00:00,2012-04-28 23:59:59,False,,example.com,r,r,none,none,100,0,72.150.241.94,US,,,,,2,True,False,True,none,,,example.com,example.com,,example.com,none,fail,example.com,mfrom,pass
+draft,acme.com,noreply-dmarc-support@acme.com,http://acme.com/dmarc/support,9391651994964116463,2012-04-28 00:00:00,2012-04-28 23:59:59,False,,example.com,r,r,none,none,100,0,72.150.241.94,US,,,,,2,True,False,True,none,,,example.com,example.com,,example.com,none,fail,example.com,mfrom,pass
+
 ```

 ## Sample forensic report output
--- a/docs/source/usage.md
+++ b/docs/source/usage.md
@@ -4,47 +4,50 @@

 ```text
 usage: parsedmarc [-h] [-c CONFIG_FILE] [--strip-attachment-payloads] [-o OUTPUT]
-                   [--aggregate-json-filename AGGREGATE_JSON_FILENAME]
-                   [--forensic-json-filename FORENSIC_JSON_FILENAME]
-                   [--aggregate-csv-filename AGGREGATE_CSV_FILENAME]
-                   [--forensic-csv-filename FORENSIC_CSV_FILENAME]
-                   [-n NAMESERVERS [NAMESERVERS ...]] [-t DNS_TIMEOUT] [--offline]
-                   [-s] [--verbose] [--debug] [--log-file LOG_FILE] [-v]
-                   [file_path ...]
+                  [--aggregate-json-filename AGGREGATE_JSON_FILENAME] [--forensic-json-filename FORENSIC_JSON_FILENAME]
+                  [--smtp-tls-json-filename SMTP_TLS_JSON_FILENAME] [--aggregate-csv-filename AGGREGATE_CSV_FILENAME]
+                  [--forensic-csv-filename FORENSIC_CSV_FILENAME] [--smtp-tls-csv-filename SMTP_TLS_CSV_FILENAME]
+                  [-n NAMESERVERS [NAMESERVERS ...]] [-t DNS_TIMEOUT] [--offline] [-s] [-w] [--verbose] [--debug]
+                  [--log-file LOG_FILE] [--no-prettify-json] [-v]
+                  [file_path ...]

- Parses DMARC reports
+Parses DMARC reports

- positional arguments:
-   file_path             one or more paths to aggregate or forensic report
-                         files, emails, or mbox files'
+positional arguments:
+  file_path             one or more paths to aggregate or forensic report files, emails, or mbox files'

- optional arguments:
-   -h, --help            show this help message and exit
-   -c CONFIG_FILE, --config-file CONFIG_FILE
-                         a path to a configuration file (--silent implied)
-   --strip-attachment-payloads
-                         remove attachment payloads from forensic report output
-   -o OUTPUT, --output OUTPUT
-                         write output files to the given directory
-   --aggregate-json-filename AGGREGATE_JSON_FILENAME
-                         filename for the aggregate JSON output file
-   --forensic-json-filename FORENSIC_JSON_FILENAME
-                         filename for the forensic JSON output file
-   --aggregate-csv-filename AGGREGATE_CSV_FILENAME
-                         filename for the aggregate CSV output file
-   --forensic-csv-filename FORENSIC_CSV_FILENAME
-                         filename for the forensic CSV output file
-   -n NAMESERVERS [NAMESERVERS ...], --nameservers NAMESERVERS [NAMESERVERS ...]
-                         nameservers to query
-   -t DNS_TIMEOUT, --dns_timeout DNS_TIMEOUT
-                         number of seconds to wait for an answer from DNS
-                         (default: 2.0)
-   --offline             do not make online queries for geolocation or DNS
-   -s, --silent          only print errors and warnings
-   --verbose             more verbose output
-   --debug               print debugging information
-   --log-file LOG_FILE   output logging to a file
-   -v, --version         show program's version number and exit
+options:
+  -h, --help            show this help message and exit
+  -c CONFIG_FILE, --config-file CONFIG_FILE
+                        a path to a configuration file (--silent implied)
+  --strip-attachment-payloads
+                        remove attachment payloads from forensic report output
+  -o OUTPUT, --output OUTPUT
+                        write output files to the given directory
+  --aggregate-json-filename AGGREGATE_JSON_FILENAME
+                        filename for the aggregate JSON output file
+  --forensic-json-filename FORENSIC_JSON_FILENAME
+                        filename for the forensic JSON output file
+  --smtp-tls-json-filename SMTP_TLS_JSON_FILENAME
+                        filename for the SMTP TLS JSON output file
+  --aggregate-csv-filename AGGREGATE_CSV_FILENAME
+                        filename for the aggregate CSV output file
+  --forensic-csv-filename FORENSIC_CSV_FILENAME
+                        filename for the forensic CSV output file
+  --smtp-tls-csv-filename SMTP_TLS_CSV_FILENAME
+                        filename for the SMTP TLS CSV output file
+  -n NAMESERVERS [NAMESERVERS ...], --nameservers NAMESERVERS [NAMESERVERS ...]
+                        nameservers to query
+  -t DNS_TIMEOUT, --dns_timeout DNS_TIMEOUT
+                        number of seconds to wait for an answer from DNS (default: 2.0)
+  --offline             do not make online queries for geolocation or DNS
+  -s, --silent          only print errors
+  -w, --warnings        print warnings in addition to errors
+  --verbose             more verbose output
+  --debug               print debugging information
+  --log-file LOG_FILE   output logging to a file
+  --no-prettify-json    output JSON in a single line without indentation
+  -v, --version         show program's version number and exit
 ```

 :::{note}
@@ -120,8 +123,10 @@ The full set of configuration options are:
      Elasticsearch, Splunk and/or S3
  - `save_smtp_tls` - bool: Save SMTP-STS report data to
      Elasticsearch, Splunk and/or S3
+  - `index_prefix_domain_map` -  bool: A path mapping of Opensearch/Elasticsearch index prefixes to domain names
  - `strip_attachment_payloads` - bool: Remove attachment
      payloads from results
+  - `silent` - bool: Set this to `False` to output results to STDOUT
  - `output` - str: Directory to place JSON and CSV files in.  This is required if you set either of the JSON output file options.
  - `aggregate_json_filename` - str: filename for the aggregate
      JSON output file
@@ -167,7 +172,7 @@ The full set of configuration options are:
      IDLE response or the number of seconds until the next
      mail check (Default: `30`)
  - `since` - str: Search for messages since certain time. (Examples: `5m|3h|2d|1w`) 
-      Acceptable units - {"m":"minutes", "h":"hours", "d":"days", "w":"weeks"}). 
+      Acceptable units - {"m":"minutes", "h":"hours", "d":"days", "w":"weeks"}. 
      Defaults to `1d` if incorrect value is provided.
 - `imap`
  - `host` - str: The IMAP server hostname or IP address
@@ -224,6 +229,18 @@ The full set of configuration options are:
    username, you must grant the app `Mail.ReadWrite.Shared`.
    :::

+    :::{tip}
+    When configuring folder names (e.g., `reports_folder`, `archive_folder`),
+    you can use standard folder names like `Inbox`, `Archive`, `Sent Items`, etc.
+    These will be automatically mapped to Microsoft Graph's well-known folder names,
+    which works reliably across different mailbox locales and avoids issues with
+    uninitialized or shared mailboxes. Supported folder names include:
+    - English: Inbox, Sent Items, Deleted Items, Drafts, Junk Email, Archive, Outbox
+    - German: Posteingang, Gesendete Elemente, Gelöschte Elemente, Entwürfe, Junk-E-Mail, Archiv
+    - French: Boîte de réception, Éléments envoyés, Éléments supprimés, Brouillons, Courrier indésirable, Archives
+    - Spanish: Bandeja de entrada, Elementos enviados, Elementos eliminados, Borradores, Correo no deseado
+    :::
+
    :::{warning}
    If you are using the `ClientSecret` auth method, you need to
    grant the `Mail.ReadWrite` (application) permission to the
@@ -252,7 +269,7 @@ The full set of configuration options are:
    :::
  - `user` - str: Basic auth username
  - `password` - str: Basic auth password
-  - `apiKey` - str: API key
+  - `api_key` - str: API key
  - `ssl` - bool: Use an encrypted SSL/TLS connection
    (Default: `True`)
  - `timeout` - float: Timeout in seconds (Default: 60)
@@ -275,7 +292,7 @@ The full set of configuration options are:
    :::
  - `user` - str: Basic auth username
  - `password` - str: Basic auth password
-  - `apiKey` - str: API key
+  - `api_key` - str: API key
  - `ssl` - bool: Use an encrypted SSL/TLS connection
    (Default: `True`)
  - `timeout` - float: Timeout in seconds (Default: 60)
@@ -445,6 +462,28 @@ PUT _cluster/settings
 Increasing this value increases resource usage.
 :::

+## Multi-tenant support
+
+Starting in `8.19.0`, ParseDMARC provides multi-tenant support by placing data into separate OpenSearch or Elasticsearch index prefixes. To set this up, create a YAML file that is formatted where each key is a tenant name, and the value is a list of domains related to that tenant, not including subdomains, like this:
+
+```yaml
+example:
+  - example.com
+  - example.net
+  - example.org
+
+whalensolutions:
+  - whalensolutions.com
+```
+
+Save it to disk where the user running ParseDMARC can read it, then set `index_prefix_domain_map` to that filepath in the `[general]` section of the ParseDMARC configuration file and do not set an `index_prefix` option in the `[elasticsearch]` or `[opensearch]` sections.
+
+When configured correctly, if ParseDMARC finds that a report is related to a domain in the mapping, the report will be saved in an index name that has the tenant name prefixed to it with a trailing underscore. Then, you can use the security features of Opensearch or the ELK stack to only grant users access to the indexes that they need.
+
+ :::{note}
+ A domain cannot be used in multiple tenant lists. Only the first prefix list that contains the matching domain is used.
+:::
+
 ## Running parsedmarc as a systemd service

 Use systemd to run `parsedmarc` as a service and process reports as
--- a/kibana/export.ndjson
+++ b/kibana/export.ndjson
--- a/parsedmarc/init.py
+++ b/parsedmarc/init.py
--- a/parsedmarc/cli.py
+++ b/parsedmarc/cli.py
@@ -3,53 +3,55 @@

 """A CLI for parsing DMARC reports"""

-from argparse import Namespace, ArgumentParser
+import http.client
+import json
+import logging
 import os
+import sys
+from argparse import ArgumentParser, Namespace
 from configparser import ConfigParser
 from glob import glob
-import logging
-import math
-from collections import OrderedDict
-import json
-from ssl import CERT_NONE, create_default_context
 from multiprocessing import Pipe, Process
-import sys
-import http.client
+from ssl import CERT_NONE, create_default_context
+
+import yaml
 from tqdm import tqdm

 from parsedmarc import (
-    get_dmarc_reports_from_mailbox,
-    watch_inbox,
-    parse_report_file,
-    get_dmarc_reports_from_mbox,
-    elastic,
-    opensearch,
-    kafkaclient,
-    splunk,
-    save_output,
-    email_results,
+    SEEN_AGGREGATE_REPORT_IDS,
+    InvalidDMARCReport,
    ParserError,
    __version__,
-    InvalidDMARCReport,
-    s3,
-    syslog,
-    loganalytics,
+    elastic,
+    email_results,
    gelf,
+    get_dmarc_reports_from_mailbox,
+    get_dmarc_reports_from_mbox,
+    kafkaclient,
+    loganalytics,
+    opensearch,
+    parse_report_file,
+    s3,
+    save_output,
+    splunk,
+    syslog,
+    watch_inbox,
    webhook,
 )
+from parsedmarc.log import logger
 from parsedmarc.mail import (
-    IMAPConnection,
-    MSGraphConnection,
    GmailConnection,
+    IMAPConnection,
    MaildirConnection,
+    MSGraphConnection,
 )
 from parsedmarc.mail.graph import AuthMethod
+from parsedmarc.types import ParsingResults
+from parsedmarc.utils import get_base_domain, get_reverse_dns, is_mbox

-from parsedmarc.log import logger
-from parsedmarc.utils import is_mbox, get_reverse_dns
-from parsedmarc import SEEN_AGGREGATE_REPORT_IDS
-
-http.client._MAXHEADERS = 200  # pylint:disable=protected-access
+# Increase the max header limit for very large emails. `_MAXHEADERS` is a
+# private stdlib attribute and may not exist in type stubs.
+setattr(http.client, "_MAXHEADERS", 200)

 formatter = logging.Formatter(
    fmt="%(levelname)8s:%(filename)s:%(lineno)d:%(message)s",
@@ -66,6 +68,48 @@ def _str_to_list(s):
    return list(map(lambda i: i.lstrip(), _list))


+def _configure_logging(log_level, log_file=None):
+    """
+    Configure logging for the current process.
+    This is needed for child processes to properly log messages.
+
+    Args:
+        log_level: The logging level (e.g., logging.DEBUG, logging.WARNING)
+        log_file: Optional path to log file
+    """
+    # Get the logger
+    from parsedmarc.log import logger
+
+    # Set the log level
+    logger.setLevel(log_level)
+
+    # Add StreamHandler with formatter if not already present
+    # Check if we already have a StreamHandler to avoid duplicates
+    # Use exact type check to distinguish from FileHandler subclass
+    has_stream_handler = any(type(h) is logging.StreamHandler for h in logger.handlers)
+
+    if not has_stream_handler:
+        formatter = logging.Formatter(
+            fmt="%(levelname)8s:%(filename)s:%(lineno)d:%(message)s",
+            datefmt="%Y-%m-%d:%H:%M:%S",
+        )
+        handler = logging.StreamHandler()
+        handler.setFormatter(formatter)
+        logger.addHandler(handler)
+
+    # Add FileHandler if log_file is specified
+    if log_file:
+        try:
+            fh = logging.FileHandler(log_file, "a")
+            formatter = logging.Formatter(
+                "%(asctime)s - %(levelname)s - [%(filename)s:%(lineno)d] - %(message)s"
+            )
+            fh.setFormatter(formatter)
+            logger.addHandler(fh)
+        except (IOError, OSError, PermissionError) as error:
+            logger.warning("Unable to write to log file: {}".format(error))
+
+
 def cli_parse(
    file_path,
    sa,
@@ -76,9 +120,31 @@ def cli_parse(
    always_use_local_files,
    reverse_dns_map_path,
    reverse_dns_map_url,
+    normalize_timespan_threshold_hours,
    conn,
+    log_level=logging.ERROR,
+    log_file=None,
 ):
-    """Separated this function for multiprocessing"""
+    """Separated this function for multiprocessing
+
+    Args:
+        file_path: Path to the report file
+        sa: Strip attachment payloads flag
+        nameservers: List of nameservers
+        dns_timeout: DNS timeout
+        ip_db_path: Path to IP database
+        offline: Offline mode flag
+        always_use_local_files: Always use local files flag
+        reverse_dns_map_path: Path to reverse DNS map
+        reverse_dns_map_url: URL to reverse DNS map
+        normalize_timespan_threshold_hours: Timespan threshold
+        conn: Pipe connection for IPC
+        log_level: Logging level for this process
+        log_file: Optional path to log file
+    """
+    # Configure logging in this child process
+    _configure_logging(log_level, log_file)
+
    try:
        file_results = parse_report_file(
            file_path,
@@ -90,6 +156,7 @@ def cli_parse(
            nameservers=nameservers,
            dns_timeout=dns_timeout,
            strip_attachment_payloads=sa,
+            normalize_timespan_threshold_hours=normalize_timespan_threshold_hours,
        )
        conn.send([file_results, file_path])
    except ParserError as error:
@@ -101,14 +168,42 @@ def cli_parse(
 def _main():
    """Called when the module is executed"""

+    def get_index_prefix(report):
+        domain = None
+        if index_prefix_domain_map is None:
+            return None
+        if "policy_published" in report:
+            domain = report["policy_published"]["domain"]
+        elif "reported_domain" in report:
+            domain = report("reported_domain")
+        elif "policies" in report:
+            domain = report["policies"][0]["domain"]
+        if domain:
+            domain = get_base_domain(domain)
+            for prefix in index_prefix_domain_map:
+                if domain in index_prefix_domain_map[prefix]:
+                    prefix = (
+                        prefix.lower()
+                        .strip()
+                        .strip("_")
+                        .replace(" ", "_")
+                        .replace("-", "_")
+                    )
+                    prefix = f"{prefix}_"
+                    return prefix
+        return None
+
    def process_reports(reports_):
-        output_str = "{0}\n".format(json.dumps(reports_, ensure_ascii=False, indent=2))
+        indent_value = 2 if opts.prettify_json else None
+        output_str = "{0}\n".format(
+            json.dumps(reports_, ensure_ascii=False, indent=indent_value)
+        )

        if not opts.silent:
            print(output_str)
        if opts.output:
            save_output(
-                results,
+                reports_,
                output_directory=opts.output,
                aggregate_json_filename=opts.aggregate_json_filename,
                forensic_json_filename=opts.forensic_json_filename,
@@ -126,7 +221,8 @@ def _main():
                        elastic.save_aggregate_report_to_elasticsearch(
                            report,
                            index_suffix=opts.elasticsearch_index_suffix,
-                            index_prefix=opts.elasticsearch_index_prefix,
+                            index_prefix=opts.elasticsearch_index_prefix
+                            or get_index_prefix(report),
                            monthly_indexes=opts.elasticsearch_monthly_indexes,
                            number_of_shards=shards,
                            number_of_replicas=replicas,
@@ -147,7 +243,8 @@ def _main():
                        opensearch.save_aggregate_report_to_opensearch(
                            report,
                            index_suffix=opts.opensearch_index_suffix,
-                            index_prefix=opts.opensearch_index_prefix,
+                            index_prefix=opts.opensearch_index_prefix
+                            or get_index_prefix(report),
                            monthly_indexes=opts.opensearch_monthly_indexes,
                            number_of_shards=shards,
                            number_of_replicas=replicas,
@@ -189,8 +286,9 @@ def _main():

                try:
                    if opts.webhook_aggregate_url:
+                        indent_value = 2 if opts.prettify_json else None
                        webhook_client.save_aggregate_report_to_webhook(
-                            json.dumps(report, ensure_ascii=False, indent=2)
+                            json.dumps(report, ensure_ascii=False, indent=indent_value)
                        )
                except Exception as error_:
                    logger.error("Webhook Error: {0}".format(error_.__str__()))
@@ -212,7 +310,8 @@ def _main():
                        elastic.save_forensic_report_to_elasticsearch(
                            report,
                            index_suffix=opts.elasticsearch_index_suffix,
-                            index_prefix=opts.elasticsearch_index_prefix,
+                            index_prefix=opts.elasticsearch_index_prefix
+                            or get_index_prefix(report),
                            monthly_indexes=opts.elasticsearch_monthly_indexes,
                            number_of_shards=shards,
                            number_of_replicas=replicas,
@@ -231,7 +330,8 @@ def _main():
                        opensearch.save_forensic_report_to_opensearch(
                            report,
                            index_suffix=opts.opensearch_index_suffix,
-                            index_prefix=opts.opensearch_index_prefix,
+                            index_prefix=opts.opensearch_index_prefix
+                            or get_index_prefix(report),
                            monthly_indexes=opts.opensearch_monthly_indexes,
                            number_of_shards=shards,
                            number_of_replicas=replicas,
@@ -271,8 +371,9 @@ def _main():

                try:
                    if opts.webhook_forensic_url:
+                        indent_value = 2 if opts.prettify_json else None
                        webhook_client.save_forensic_report_to_webhook(
-                            json.dumps(report, ensure_ascii=False, indent=2)
+                            json.dumps(report, ensure_ascii=False, indent=indent_value)
                        )
                except Exception as error_:
                    logger.error("Webhook Error: {0}".format(error_.__str__()))
@@ -294,7 +395,8 @@ def _main():
                        elastic.save_smtp_tls_report_to_elasticsearch(
                            report,
                            index_suffix=opts.elasticsearch_index_suffix,
-                            index_prefix=opts.elasticsearch_index_prefix,
+                            index_prefix=opts.elasticsearch_index_prefix
+                            or get_index_prefix(report),
                            monthly_indexes=opts.elasticsearch_monthly_indexes,
                            number_of_shards=shards,
                            number_of_replicas=replicas,
@@ -313,7 +415,8 @@ def _main():
                        opensearch.save_smtp_tls_report_to_opensearch(
                            report,
                            index_suffix=opts.opensearch_index_suffix,
-                            index_prefix=opts.opensearch_index_prefix,
+                            index_prefix=opts.opensearch_index_prefix
+                            or get_index_prefix(report),
                            monthly_indexes=opts.opensearch_monthly_indexes,
                            number_of_shards=shards,
                            number_of_replicas=replicas,
@@ -353,8 +456,9 @@ def _main():

                try:
                    if opts.webhook_smtp_tls_url:
+                        indent_value = 2 if opts.prettify_json else None
                        webhook_client.save_smtp_tls_report_to_webhook(
-                            json.dumps(report, ensure_ascii=False, indent=2)
+                            json.dumps(report, ensure_ascii=False, indent=indent_value)
                        )
                except Exception as error_:
                    logger.error("Webhook Error: {0}".format(error_.__str__()))
@@ -475,6 +579,12 @@ def _main():
        "--debug", action="store_true", help="print debugging information"
    )
    arg_parser.add_argument("--log-file", default=None, help="output logging to a file")
+    arg_parser.add_argument(
+        "--no-prettify-json",
+        action="store_false",
+        dest="prettify_json",
+        help="output JSON in a single line without indentation",
+    )
    arg_parser.add_argument("-v", "--version", action="version", version=__version__)

    aggregate_reports = []
@@ -504,6 +614,7 @@ def _main():
        dns_timeout=args.dns_timeout,
        debug=args.debug,
        verbose=args.verbose,
+        prettify_json=args.prettify_json,
        save_aggregate=False,
        save_forensic=False,
        save_smtp_tls=False,
@@ -547,7 +658,7 @@ def _main():
        elasticsearch_monthly_indexes=False,
        elasticsearch_username=None,
        elasticsearch_password=None,
-        elasticsearch_apiKey=None,
+        elasticsearch_api_key=None,
        opensearch_hosts=None,
        opensearch_timeout=60,
        opensearch_number_of_shards=1,
@@ -559,7 +670,7 @@ def _main():
        opensearch_monthly_indexes=False,
        opensearch_username=None,
        opensearch_password=None,
-        opensearch_apiKey=None,
+        opensearch_api_key=None,
        kafka_hosts=None,
        kafka_username=None,
        kafka_password=None,
@@ -615,6 +726,7 @@ def _main():
        webhook_forensic_url=None,
        webhook_smtp_tls_url=None,
        webhook_timeout=60,
+        normalize_timespan_threshold_hours=24.0,
    )
    args = arg_parser.parse_args()

@@ -625,14 +737,24 @@ def _main():
            exit(-1)
        opts.silent = True
        config = ConfigParser()
+        index_prefix_domain_map = None
        config.read(args.config_file)
        if "general" in config.sections():
            general_config = config["general"]
+            if "silent" in general_config:
+                opts.silent = bool(general_config.getboolean("silent"))
+            if "normalize_timespan_threshold_hours" in general_config:
+                opts.normalize_timespan_threshold_hours = general_config.getfloat(
+                    "normalize_timespan_threshold_hours"
+                )
+            if "index_prefix_domain_map" in general_config:
+                with open(general_config["index_prefix_domain_map"]) as f:
+                    index_prefix_domain_map = yaml.safe_load(f)
            if "offline" in general_config:
-                opts.offline = general_config.getboolean("offline")
+                opts.offline = bool(general_config.getboolean("offline"))
            if "strip_attachment_payloads" in general_config:
-                opts.strip_attachment_payloads = general_config.getboolean(
-                    "strip_attachment_payloads"
+                opts.strip_attachment_payloads = bool(
+                    general_config.getboolean("strip_attachment_payloads")
                )
            if "output" in general_config:
                opts.output = general_config["output"]
@@ -650,6 +772,8 @@ def _main():
                opts.smtp_tls_csv_filename = general_config["smtp_tls_csv_filename"]
            if "dns_timeout" in general_config:
                opts.dns_timeout = general_config.getfloat("dns_timeout")
+                if opts.dns_timeout is None:
+                    opts.dns_timeout = 2
            if "dns_test_address" in general_config:
                opts.dns_test_address = general_config["dns_test_address"]
            if "nameservers" in general_config:
@@ -672,19 +796,19 @@ def _main():
                    )
                    exit(-1)
            if "save_aggregate" in general_config:
-                opts.save_aggregate = general_config["save_aggregate"]
+                opts.save_aggregate = bool(general_config.getboolean("save_aggregate"))
            if "save_forensic" in general_config:
-                opts.save_forensic = general_config["save_forensic"]
+                opts.save_forensic = bool(general_config.getboolean("save_forensic"))
            if "save_smtp_tls" in general_config:
-                opts.save_smtp_tls = general_config["save_smtp_tls"]
+                opts.save_smtp_tls = bool(general_config.getboolean("save_smtp_tls"))
            if "debug" in general_config:
-                opts.debug = general_config.getboolean("debug")
+                opts.debug = bool(general_config.getboolean("debug"))
            if "verbose" in general_config:
-                opts.verbose = general_config.getboolean("verbose")
+                opts.verbose = bool(general_config.getboolean("verbose"))
            if "silent" in general_config:
-                opts.silent = general_config.getboolean("silent")
+                opts.silent = bool(general_config.getboolean("silent"))
            if "warnings" in general_config:
-                opts.warnings = general_config.getboolean("warnings")
+                opts.warnings = bool(general_config.getboolean("warnings"))
            if "log_file" in general_config:
                opts.log_file = general_config["log_file"]
            if "n_procs" in general_config:
@@ -694,13 +818,15 @@ def _main():
            else:
                opts.ip_db_path = None
            if "always_use_local_files" in general_config:
-                opts.always_use_local_files = general_config.getboolean(
-                    "always_use_local_files"
+                opts.always_use_local_files = bool(
+                    general_config.getboolean("always_use_local_files")
                )
            if "reverse_dns_map_path" in general_config:
                opts.reverse_dns_map_path = general_config["reverse_dns_path"]
            if "reverse_dns_map_url" in general_config:
                opts.reverse_dns_map_url = general_config["reverse_dns_url"]
+            if "prettify_json" in general_config:
+                opts.prettify_json = bool(general_config.getboolean("prettify_json"))

        if "mailbox" in config.sections():
            mailbox_config = config["mailbox"]
@@ -711,11 +837,11 @@ def _main():
            if "archive_folder" in mailbox_config:
                opts.mailbox_archive_folder = mailbox_config["archive_folder"]
            if "watch" in mailbox_config:
-                opts.mailbox_watch = mailbox_config.getboolean("watch")
+                opts.mailbox_watch = bool(mailbox_config.getboolean("watch"))
            if "delete" in mailbox_config:
-                opts.mailbox_delete = mailbox_config.getboolean("delete")
+                opts.mailbox_delete = bool(mailbox_config.getboolean("delete"))
            if "test" in mailbox_config:
-                opts.mailbox_test = mailbox_config.getboolean("test")
+                opts.mailbox_test = bool(mailbox_config.getboolean("test"))
            if "batch_size" in mailbox_config:
                opts.mailbox_batch_size = mailbox_config.getint("batch_size")
            if "check_timeout" in mailbox_config:
@@ -739,14 +865,15 @@ def _main():
            if "port" in imap_config:
                opts.imap_port = imap_config.getint("port")
            if "timeout" in imap_config:
-                opts.imap_timeout = imap_config.getfloat("timeout")
+                opts.imap_timeout = imap_config.getint("timeout")
            if "max_retries" in imap_config:
                opts.imap_max_retries = imap_config.getint("max_retries")
            if "ssl" in imap_config:
-                opts.imap_ssl = imap_config.getboolean("ssl")
+                opts.imap_ssl = bool(imap_config.getboolean("ssl"))
            if "skip_certificate_verification" in imap_config:
-                imap_verify = imap_config.getboolean("skip_certificate_verification")
-                opts.imap_skip_certificate_verification = imap_verify
+                opts.imap_skip_certificate_verification = bool(
+                    imap_config.getboolean("skip_certificate_verification")
+                )
            if "user" in imap_config:
                opts.imap_user = imap_config["user"]
            else:
@@ -774,7 +901,7 @@ def _main():
                    "section instead."
                )
            if "watch" in imap_config:
-                opts.mailbox_watch = imap_config.getboolean("watch")
+                opts.mailbox_watch = bool(imap_config.getboolean("watch"))
                logger.warning(
                    "Use of the watch option in the imap "
                    "configuration section has been deprecated. "
@@ -789,7 +916,7 @@ def _main():
                    "section instead."
                )
            if "test" in imap_config:
-                opts.mailbox_test = imap_config.getboolean("test")
+                opts.mailbox_test = bool(imap_config.getboolean("test"))
                logger.warning(
                    "Use of the test option in the imap "
                    "configuration section has been deprecated. "
@@ -883,8 +1010,8 @@ def _main():
                opts.graph_url = graph_config["graph_url"]

            if "allow_unencrypted_storage" in graph_config:
-                opts.graph_allow_unencrypted_storage = graph_config.getboolean(
-                    "allow_unencrypted_storage"
+                opts.graph_allow_unencrypted_storage = bool(
+                    graph_config.getboolean("allow_unencrypted_storage")
                )

        if "elasticsearch" in config:
@@ -912,18 +1039,22 @@ def _main():
            if "index_prefix" in elasticsearch_config:
                opts.elasticsearch_index_prefix = elasticsearch_config["index_prefix"]
            if "monthly_indexes" in elasticsearch_config:
-                monthly = elasticsearch_config.getboolean("monthly_indexes")
+                monthly = bool(elasticsearch_config.getboolean("monthly_indexes"))
                opts.elasticsearch_monthly_indexes = monthly
            if "ssl" in elasticsearch_config:
-                opts.elasticsearch_ssl = elasticsearch_config.getboolean("ssl")
+                opts.elasticsearch_ssl = bool(elasticsearch_config.getboolean("ssl"))
            if "cert_path" in elasticsearch_config:
                opts.elasticsearch_ssl_cert_path = elasticsearch_config["cert_path"]
            if "user" in elasticsearch_config:
                opts.elasticsearch_username = elasticsearch_config["user"]
            if "password" in elasticsearch_config:
                opts.elasticsearch_password = elasticsearch_config["password"]
+            # Until 8.20
            if "apiKey" in elasticsearch_config:
                opts.elasticsearch_apiKey = elasticsearch_config["apiKey"]
+            # Since 8.20
+            if "api_key" in elasticsearch_config:
+                opts.elasticsearch_apiKey = elasticsearch_config["api_key"]

        if "opensearch" in config:
            opensearch_config = config["opensearch"]
@@ -948,18 +1079,22 @@ def _main():
            if "index_prefix" in opensearch_config:
                opts.opensearch_index_prefix = opensearch_config["index_prefix"]
            if "monthly_indexes" in opensearch_config:
-                monthly = opensearch_config.getboolean("monthly_indexes")
+                monthly = bool(opensearch_config.getboolean("monthly_indexes"))
                opts.opensearch_monthly_indexes = monthly
            if "ssl" in opensearch_config:
-                opts.opensearch_ssl = opensearch_config.getboolean("ssl")
+                opts.opensearch_ssl = bool(opensearch_config.getboolean("ssl"))
            if "cert_path" in opensearch_config:
                opts.opensearch_ssl_cert_path = opensearch_config["cert_path"]
            if "user" in opensearch_config:
                opts.opensearch_username = opensearch_config["user"]
            if "password" in opensearch_config:
                opts.opensearch_password = opensearch_config["password"]
+            # Until 8.20
            if "apiKey" in opensearch_config:
                opts.opensearch_apiKey = opensearch_config["apiKey"]
+            # Since 8.20
+            if "api_key" in opensearch_config:
+                opts.opensearch_apiKey = opensearch_config["api_key"]

        if "splunk_hec" in config.sections():
            hec_config = config["splunk_hec"]
@@ -1001,9 +1136,11 @@ def _main():
            if "password" in kafka_config:
                opts.kafka_password = kafka_config["password"]
            if "ssl" in kafka_config:
-                opts.kafka_ssl = kafka_config.getboolean("ssl")
+                opts.kafka_ssl = bool(kafka_config.getboolean("ssl"))
            if "skip_certificate_verification" in kafka_config:
-                kafka_verify = kafka_config.getboolean("skip_certificate_verification")
+                kafka_verify = bool(
+                    kafka_config.getboolean("skip_certificate_verification")
+                )
                opts.kafka_skip_certificate_verification = kafka_verify
            if "aggregate_topic" in kafka_config:
                opts.kafka_aggregate_topic = kafka_config["aggregate_topic"]
@@ -1035,9 +1172,11 @@ def _main():
            if "port" in smtp_config:
                opts.smtp_port = smtp_config.getint("port")
            if "ssl" in smtp_config:
-                opts.smtp_ssl = smtp_config.getboolean("ssl")
+                opts.smtp_ssl = bool(smtp_config.getboolean("ssl"))
            if "skip_certificate_verification" in smtp_config:
-                smtp_verify = smtp_config.getboolean("skip_certificate_verification")
+                smtp_verify = bool(
+                    smtp_config.getboolean("skip_certificate_verification")
+                )
                opts.smtp_skip_certificate_verification = smtp_verify
            if "user" in smtp_config:
                opts.smtp_user = smtp_config["user"]
@@ -1105,23 +1244,27 @@ def _main():
            gmail_api_config = config["gmail_api"]
            opts.gmail_api_credentials_file = gmail_api_config.get("credentials_file")
            opts.gmail_api_token_file = gmail_api_config.get("token_file", ".token")
-            opts.gmail_api_include_spam_trash = gmail_api_config.getboolean(
-                "include_spam_trash", False
+            opts.gmail_api_include_spam_trash = bool(
+                gmail_api_config.getboolean("include_spam_trash", False)
            )
-            opts.gmail_api_paginate_messages = gmail_api_config.getboolean(
-                "paginate_messages", True
+            opts.gmail_api_paginate_messages = bool(
+                gmail_api_config.getboolean("paginate_messages", True)
            )
            opts.gmail_api_scopes = gmail_api_config.get(
                "scopes", default_gmail_api_scope
            )
            opts.gmail_api_scopes = _str_to_list(opts.gmail_api_scopes)
            if "oauth2_port" in gmail_api_config:
-                opts.gmail_api_oauth2_port = gmail_api_config.get("oauth2_port", 8080)
+                opts.gmail_api_oauth2_port = gmail_api_config.getint(
+                    "oauth2_port", 8080
+                )

        if "maildir" in config.sections():
            maildir_api_config = config["maildir"]
            opts.maildir_path = maildir_api_config.get("maildir_path")
-            opts.maildir_create = maildir_api_config.get("maildir_create")
+            opts.maildir_create = bool(
+                maildir_api_config.getboolean("maildir_create", fallback=False)
+            )

        if "log_analytics" in config.sections():
            log_analytics_config = config["log_analytics"]
@@ -1167,7 +1310,7 @@ def _main():
            if "smtp_tls_url" in webhook_config:
                opts.webhook_smtp_tls_url = webhook_config["smtp_tls_url"]
            if "timeout" in webhook_config:
-                opts.webhook_timeout = webhook_config["timeout"]
+                opts.webhook_timeout = webhook_config.getint("timeout")

    logger.setLevel(logging.ERROR)

@@ -1216,14 +1359,19 @@ def _main():
                    es_aggregate_index = "{0}{1}".format(prefix, es_aggregate_index)
                    es_forensic_index = "{0}{1}".format(prefix, es_forensic_index)
                    es_smtp_tls_index = "{0}{1}".format(prefix, es_smtp_tls_index)
+                elastic_timeout_value = (
+                    float(opts.elasticsearch_timeout)
+                    if opts.elasticsearch_timeout is not None
+                    else 60.0
+                )
                elastic.set_hosts(
                    opts.elasticsearch_hosts,
-                    opts.elasticsearch_ssl,
-                    opts.elasticsearch_ssl_cert_path,
-                    opts.elasticsearch_username,
-                    opts.elasticsearch_password,
-                    opts.elasticsearch_apiKey,
-                    timeout=opts.elasticsearch_timeout,
+                    use_ssl=opts.elasticsearch_ssl,
+                    ssl_cert_path=opts.elasticsearch_ssl_cert_path,
+                    username=opts.elasticsearch_username,
+                    password=opts.elasticsearch_password,
+                    api_key=opts.elasticsearch_api_key,
+                    timeout=elastic_timeout_value,
                )
                elastic.migrate_indexes(
                    aggregate_indexes=[es_aggregate_index],
@@ -1248,14 +1396,19 @@ def _main():
                    os_aggregate_index = "{0}{1}".format(prefix, os_aggregate_index)
                    os_forensic_index = "{0}{1}".format(prefix, os_forensic_index)
                    os_smtp_tls_index = "{0}{1}".format(prefix, os_smtp_tls_index)
+                opensearch_timeout_value = (
+                    float(opts.opensearch_timeout)
+                    if opts.opensearch_timeout is not None
+                    else 60.0
+                )
                opensearch.set_hosts(
                    opts.opensearch_hosts,
-                    opts.opensearch_ssl,
-                    opts.opensearch_ssl_cert_path,
-                    opts.opensearch_username,
-                    opts.opensearch_password,
-                    opts.opensearch_apiKey,
-                    timeout=opts.opensearch_timeout,
+                    use_ssl=opts.opensearch_ssl,
+                    ssl_cert_path=opts.opensearch_ssl_cert_path,
+                    username=opts.opensearch_username,
+                    password=opts.opensearch_password,
+                    api_key=opts.opensearch_api_key,
+                    timeout=opensearch_timeout_value,
                )
                opensearch.migrate_indexes(
                    aggregate_indexes=[os_aggregate_index],
@@ -1364,16 +1517,23 @@ def _main():

    results = []

+    pbar = None
    if sys.stdout.isatty():
        pbar = tqdm(total=len(file_paths))

-    for batch_index in range(math.ceil(len(file_paths) / opts.n_procs)):
+    n_procs = int(opts.n_procs or 1)
+    if n_procs < 1:
+        n_procs = 1
+
+    # Capture the current log level to pass to child processes
+    current_log_level = logger.level
+    current_log_file = opts.log_file
+
+    for batch_index in range((len(file_paths) + n_procs - 1) // n_procs):
        processes = []
        connections = []

-        for proc_index in range(
-            opts.n_procs * batch_index, opts.n_procs * (batch_index + 1)
-        ):
+        for proc_index in range(n_procs * batch_index, n_procs * (batch_index + 1)):
            if proc_index >= len(file_paths):
                break

@@ -1392,7 +1552,10 @@ def _main():
                    opts.always_use_local_files,
                    opts.reverse_dns_map_path,
                    opts.reverse_dns_map_url,
+                    opts.normalize_timespan_threshold_hours,
                    child_conn,
+                    current_log_level,
+                    current_log_file,
                ),
            )
            processes.append(process)
@@ -1405,12 +1568,15 @@ def _main():

        for proc in processes:
            proc.join()
-            if sys.stdout.isatty():
+            if pbar is not None:
                counter += 1
-                pbar.update(counter - pbar.n)
+                pbar.update(1)
+
+    if pbar is not None:
+        pbar.close()

    for result in results:
-        if type(result[0]) is ParserError:
+        if isinstance(result[0], ParserError) or result[0] is None:
            logger.error("Failed to parse {0} - {1}".format(result[1], result[0]))
        else:
            if result[0]["report_type"] == "aggregate":
@@ -1431,6 +1597,11 @@ def _main():
                smtp_tls_reports.append(result[0]["report"])

    for mbox_path in mbox_paths:
+        normalize_timespan_threshold_hours_value = (
+            float(opts.normalize_timespan_threshold_hours)
+            if opts.normalize_timespan_threshold_hours is not None
+            else 24.0
+        )
        strip = opts.strip_attachment_payloads
        reports = get_dmarc_reports_from_mbox(
            mbox_path,
@@ -1442,12 +1613,17 @@ def _main():
            reverse_dns_map_path=opts.reverse_dns_map_path,
            reverse_dns_map_url=opts.reverse_dns_map_url,
            offline=opts.offline,
+            normalize_timespan_threshold_hours=normalize_timespan_threshold_hours_value,
        )
        aggregate_reports += reports["aggregate_reports"]
        forensic_reports += reports["forensic_reports"]
        smtp_tls_reports += reports["smtp_tls_reports"]

    mailbox_connection = None
+    mailbox_batch_size_value = 10
+    mailbox_check_timeout_value = 30
+    normalize_timespan_threshold_hours_value = 24.0
+
    if opts.imap_host:
        try:
            if opts.imap_user is None or opts.imap_password is None:
@@ -1460,16 +1636,23 @@ def _main():
            if opts.imap_skip_certificate_verification:
                logger.debug("Skipping IMAP certificate verification")
                verify = False
-            if opts.imap_ssl is False:
+            if not opts.imap_ssl:
                ssl = False

+            imap_timeout = (
+                int(opts.imap_timeout) if opts.imap_timeout is not None else 30
+            )
+            imap_max_retries = (
+                int(opts.imap_max_retries) if opts.imap_max_retries is not None else 4
+            )
+            imap_port_value = int(opts.imap_port) if opts.imap_port is not None else 993
            mailbox_connection = IMAPConnection(
                host=opts.imap_host,
-                port=opts.imap_port,
+                port=imap_port_value,
                ssl=ssl,
                verify=verify,
-                timeout=opts.imap_timeout,
-                max_retries=opts.imap_max_retries,
+                timeout=imap_timeout,
+                max_retries=imap_max_retries,
                user=opts.imap_user,
                password=opts.imap_password,
            )
@@ -1490,7 +1673,7 @@ def _main():
                username=opts.graph_user,
                password=opts.graph_password,
                token_file=opts.graph_token_file,
-                allow_unencrypted_storage=opts.graph_allow_unencrypted_storage,
+                allow_unencrypted_storage=bool(opts.graph_allow_unencrypted_storage),
                graph_url=opts.graph_url,
            )

@@ -1535,11 +1718,24 @@ def _main():
            exit(1)

    if mailbox_connection:
+        mailbox_batch_size_value = (
+            int(opts.mailbox_batch_size) if opts.mailbox_batch_size is not None else 10
+        )
+        mailbox_check_timeout_value = (
+            int(opts.mailbox_check_timeout)
+            if opts.mailbox_check_timeout is not None
+            else 30
+        )
+        normalize_timespan_threshold_hours_value = (
+            float(opts.normalize_timespan_threshold_hours)
+            if opts.normalize_timespan_threshold_hours is not None
+            else 24.0
+        )
        try:
            reports = get_dmarc_reports_from_mailbox(
                connection=mailbox_connection,
                delete=opts.mailbox_delete,
-                batch_size=opts.mailbox_batch_size,
+                batch_size=mailbox_batch_size_value,
                reports_folder=opts.mailbox_reports_folder,
                archive_folder=opts.mailbox_archive_folder,
                ip_db_path=opts.ip_db_path,
@@ -1551,6 +1747,7 @@ def _main():
                test=opts.mailbox_test,
                strip_attachment_payloads=opts.strip_attachment_payloads,
                since=opts.mailbox_since,
+                normalize_timespan_threshold_hours=normalize_timespan_threshold_hours_value,
            )

            aggregate_reports += reports["aggregate_reports"]
@@ -1561,31 +1758,36 @@ def _main():
            logger.exception("Mailbox Error")
            exit(1)

-    results = OrderedDict(
-        [
-            ("aggregate_reports", aggregate_reports),
-            ("forensic_reports", forensic_reports),
-            ("smtp_tls_reports", smtp_tls_reports),
-        ]
-    )
+    parsing_results: ParsingResults = {
+        "aggregate_reports": aggregate_reports,
+        "forensic_reports": forensic_reports,
+        "smtp_tls_reports": smtp_tls_reports,
+    }

-    process_reports(results)
+    process_reports(parsing_results)

    if opts.smtp_host:
        try:
            verify = True
            if opts.smtp_skip_certificate_verification:
                verify = False
+            smtp_port_value = int(opts.smtp_port) if opts.smtp_port is not None else 25
+            smtp_to_value = (
+                list(opts.smtp_to)
+                if isinstance(opts.smtp_to, list)
+                else _str_to_list(str(opts.smtp_to))
+            )
            email_results(
-                results,
+                parsing_results,
                opts.smtp_host,
                opts.smtp_from,
-                opts.smtp_to,
-                port=opts.smtp_port,
+                smtp_to_value,
+                port=smtp_port_value,
                verify=verify,
                username=opts.smtp_user,
                password=opts.smtp_password,
                subject=opts.smtp_subject,
+                require_encryption=opts.smtp_ssl,
            )
        except Exception:
            logger.exception("Failed to email results")
@@ -1602,16 +1804,17 @@ def _main():
                archive_folder=opts.mailbox_archive_folder,
                delete=opts.mailbox_delete,
                test=opts.mailbox_test,
-                check_timeout=opts.mailbox_check_timeout,
+                check_timeout=mailbox_check_timeout_value,
                nameservers=opts.nameservers,
                dns_timeout=opts.dns_timeout,
                strip_attachment_payloads=opts.strip_attachment_payloads,
-                batch_size=opts.mailbox_batch_size,
+                batch_size=mailbox_batch_size_value,
                ip_db_path=opts.ip_db_path,
                always_use_local_files=opts.always_use_local_files,
                reverse_dns_map_path=opts.reverse_dns_map_path,
                reverse_dns_map_url=opts.reverse_dns_map_url,
                offline=opts.offline,
+                normalize_timespan_threshold_hours=normalize_timespan_threshold_hours_value,
            )
        except FileExistsError as error:
            logger.error("{0}".format(error.__str__()))
--- a/parsedmarc/constants.py
+++ b/parsedmarc/constants.py
@@ -1,2 +1,3 @@
-__version__ = "8.18.6"
+__version__ = "9.0.8"
+
 USER_AGENT = f"parsedmarc/{__version__}"
--- a/parsedmarc/elastic.py
+++ b/parsedmarc/elastic.py
@@ -1,27 +1,29 @@
 # -*- coding: utf-8 -*-

-from collections import OrderedDict
+from __future__ import annotations

-from elasticsearch_dsl.search import Q
+from typing import Any, Optional, Union
+
+from elasticsearch.helpers import reindex
 from elasticsearch_dsl import (
-    connections,
-    Object,
+    Boolean,
+    Date,
    Document,
    Index,
-    Nested,
    InnerDoc,
    Integer,
-    Text,
-    Boolean,
    Ip,
-    Date,
+    Nested,
+    Object,
    Search,
+    Text,
+    connections,
 )
-from elasticsearch.helpers import reindex
+from elasticsearch_dsl.search import Q

+from parsedmarc import InvalidForensicReport
 from parsedmarc.log import logger
 from parsedmarc.utils import human_timestamp_to_datetime
-from parsedmarc import InvalidForensicReport


 class ElasticsearchError(Exception):
@@ -67,6 +69,8 @@ class _AggregateReportDoc(Document):
    date_range = Date()
    date_begin = Date()
    date_end = Date()
+    normalized_timespan = Boolean()
+    original_timespan_seconds = Integer
    errors = Text()
    published_policy = Object(_PublishedPolicy)
    source_ip_address = Ip()
@@ -87,18 +91,18 @@ class _AggregateReportDoc(Document):
    dkim_results = Nested(_DKIMResult)
    spf_results = Nested(_SPFResult)

-    def add_policy_override(self, type_, comment):
-        self.policy_overrides.append(_PolicyOverride(type=type_, comment=comment))
+    def add_policy_override(self, type_: str, comment: str):
+        self.policy_overrides.append(_PolicyOverride(type=type_, comment=comment))  # pyright: ignore[reportCallIssue]

-    def add_dkim_result(self, domain, selector, result):
+    def add_dkim_result(self, domain: str, selector: str, result: _DKIMResult):
        self.dkim_results.append(
            _DKIMResult(domain=domain, selector=selector, result=result)
-        )
+        )  # pyright: ignore[reportCallIssue]

-    def add_spf_result(self, domain, scope, result):
-        self.spf_results.append(_SPFResult(domain=domain, scope=scope, result=result))
+    def add_spf_result(self, domain: str, scope: str, result: _SPFResult):
+        self.spf_results.append(_SPFResult(domain=domain, scope=scope, result=result))  # pyright: ignore[reportCallIssue]

-    def save(self, **kwargs):
+    def save(self, **kwargs):  # pyright: ignore[reportIncompatibleMethodOverride]
        self.passed_dmarc = False
        self.passed_dmarc = self.spf_aligned or self.dkim_aligned

@@ -131,26 +135,26 @@ class _ForensicSampleDoc(InnerDoc):
    body = Text()
    attachments = Nested(_EmailAttachmentDoc)

-    def add_to(self, display_name, address):
-        self.to.append(_EmailAddressDoc(display_name=display_name, address=address))
+    def add_to(self, display_name: str, address: str):
+        self.to.append(_EmailAddressDoc(display_name=display_name, address=address))  # pyright: ignore[reportCallIssue]

-    def add_reply_to(self, display_name, address):
+    def add_reply_to(self, display_name: str, address: str):
        self.reply_to.append(
            _EmailAddressDoc(display_name=display_name, address=address)
-        )
+        )  # pyright: ignore[reportCallIssue]

-    def add_cc(self, display_name, address):
-        self.cc.append(_EmailAddressDoc(display_name=display_name, address=address))
+    def add_cc(self, display_name: str, address: str):
+        self.cc.append(_EmailAddressDoc(display_name=display_name, address=address))  # pyright: ignore[reportCallIssue]

-    def add_bcc(self, display_name, address):
-        self.bcc.append(_EmailAddressDoc(display_name=display_name, address=address))
+    def add_bcc(self, display_name: str, address: str):
+        self.bcc.append(_EmailAddressDoc(display_name=display_name, address=address))  # pyright: ignore[reportCallIssue]

-    def add_attachment(self, filename, content_type, sha256):
+    def add_attachment(self, filename: str, content_type: str, sha256: str):
        self.attachments.append(
            _EmailAttachmentDoc(
                filename=filename, content_type=content_type, sha256=sha256
            )
-        )
+        )  # pyright: ignore[reportCallIssue]


 class _ForensicReportDoc(Document):
@@ -197,15 +201,15 @@ class _SMTPTLSPolicyDoc(InnerDoc):

    def add_failure_details(
        self,
-        result_type,
-        ip_address,
-        receiving_ip,
-        receiving_mx_helo,
-        failed_session_count,
-        sending_mta_ip=None,
-        receiving_mx_hostname=None,
-        additional_information_uri=None,
-        failure_reason_code=None,
+        result_type: Optional[str] = None,
+        ip_address: Optional[str] = None,
+        receiving_ip: Optional[str] = None,
+        receiving_mx_helo: Optional[str] = None,
+        failed_session_count: Optional[int] = None,
+        sending_mta_ip: Optional[str] = None,
+        receiving_mx_hostname: Optional[str] = None,
+        additional_information_uri: Optional[str] = None,
+        failure_reason_code: Union[str, int, None] = None,
    ):
        _details = _SMTPTLSFailureDetailsDoc(
            result_type=result_type,
@@ -218,7 +222,7 @@ class _SMTPTLSPolicyDoc(InnerDoc):
            additional_information=additional_information_uri,
            failure_reason_code=failure_reason_code,
        )
-        self.failure_details.append(_details)
+        self.failure_details.append(_details)  # pyright: ignore[reportCallIssue]


 class _SMTPTLSReportDoc(Document):
@@ -235,13 +239,14 @@ class _SMTPTLSReportDoc(Document):

    def add_policy(
        self,
-        policy_type,
-        policy_domain,
-        successful_session_count,
-        failed_session_count,
-        policy_string=None,
-        mx_host_patterns=None,
-        failure_details=None,
+        policy_type: str,
+        policy_domain: str,
+        successful_session_count: int,
+        failed_session_count: int,
+        *,
+        policy_string: Optional[str] = None,
+        mx_host_patterns: Optional[list[str]] = None,
+        failure_details: Optional[str] = None,
    ):
        self.policies.append(
            policy_type=policy_type,
@@ -251,7 +256,7 @@ class _SMTPTLSReportDoc(Document):
            policy_string=policy_string,
            mx_host_patterns=mx_host_patterns,
            failure_details=failure_details,
-        )
+        )  # pyright: ignore[reportCallIssue]


 class AlreadySaved(ValueError):
@@ -259,24 +264,25 @@ class AlreadySaved(ValueError):


 def set_hosts(
-    hosts,
-    use_ssl=False,
-    ssl_cert_path=None,
-    username=None,
-    password=None,
-    apiKey=None,
-    timeout=60.0,
+    hosts: Union[str, list[str]],
+    *,
+    use_ssl: bool = False,
+    ssl_cert_path: Optional[str] = None,
+    username: Optional[str] = None,
+    password: Optional[str] = None,
+    api_key: Optional[str] = None,
+    timeout: float = 60.0,
 ):
    """
    Sets the Elasticsearch hosts to use

    Args:
-        hosts (str): A single hostname or URL, or list of hostnames or URLs
-        use_ssl (bool): Use a HTTPS connection to the server
+        hosts (str | list[str]): A single hostname or URL, or list of hostnames or URLs
+        use_ssl (bool): Use an HTTPS connection to the server
        ssl_cert_path (str): Path to the certificate chain
        username (str): The username to use for authentication
        password (str): The password to use for authentication
-        apiKey (str): The Base64 encoded API key to use for authentication
+        api_key (str): The Base64 encoded API key to use for authentication
        timeout (float): Timeout in seconds
    """
    if not isinstance(hosts, list):
@@ -289,14 +295,14 @@ def set_hosts(
            conn_params["ca_certs"] = ssl_cert_path
        else:
            conn_params["verify_certs"] = False
-    if username:
+    if username and password:
        conn_params["http_auth"] = username + ":" + password
-    if apiKey:
-        conn_params["api_key"] = apiKey
+    if api_key:
+        conn_params["api_key"] = api_key
    connections.create_connection(**conn_params)


-def create_indexes(names, settings=None):
+def create_indexes(names: list[str], settings: Optional[dict[str, Any]] = None):
    """
    Create Elasticsearch indexes

@@ -319,7 +325,10 @@ def create_indexes(names, settings=None):
            raise ElasticsearchError("Elasticsearch error: {0}".format(e.__str__()))


-def migrate_indexes(aggregate_indexes=None, forensic_indexes=None):
+def migrate_indexes(
+    aggregate_indexes: Optional[list[str]] = None,
+    forensic_indexes: Optional[list[str]] = None,
+):
    """
    Updates index mappings

@@ -358,7 +367,7 @@ def migrate_indexes(aggregate_indexes=None, forensic_indexes=None):
            }
            Index(new_index_name).create()
            Index(new_index_name).put_mapping(doc_type=doc, body=body)
-            reindex(connections.get_connection(), aggregate_index_name, new_index_name)
+            reindex(connections.get_connection(), aggregate_index_name, new_index_name)  # pyright: ignore[reportArgumentType]
            Index(aggregate_index_name).delete()

    for forensic_index in forensic_indexes:
@@ -366,18 +375,18 @@ def migrate_indexes(aggregate_indexes=None, forensic_indexes=None):


 def save_aggregate_report_to_elasticsearch(
-    aggregate_report,
-    index_suffix=None,
-    index_prefix=None,
-    monthly_indexes=False,
-    number_of_shards=1,
-    number_of_replicas=0,
+    aggregate_report: dict[str, Any],
+    index_suffix: Optional[str] = None,
+    index_prefix: Optional[str] = None,
+    monthly_indexes: Optional[bool] = False,
+    number_of_shards: int = 1,
+    number_of_replicas: int = 0,
 ):
    """
    Saves a parsed DMARC aggregate report to Elasticsearch

    Args:
-        aggregate_report (OrderedDict): A parsed forensic report
+        aggregate_report (dict): A parsed forensic report
        index_suffix (str): The suffix of the name of the index to save to
        index_prefix (str): The prefix of the name of the index to save to
        monthly_indexes (bool): Use monthly indexes instead of daily indexes
@@ -395,21 +404,17 @@ def save_aggregate_report_to_elasticsearch(
    domain = aggregate_report["policy_published"]["domain"]
    begin_date = human_timestamp_to_datetime(metadata["begin_date"], to_utc=True)
    end_date = human_timestamp_to_datetime(metadata["end_date"], to_utc=True)
-    begin_date_human = begin_date.strftime("%Y-%m-%d %H:%M:%SZ")
-    end_date_human = end_date.strftime("%Y-%m-%d %H:%M:%SZ")
+
    if monthly_indexes:
        index_date = begin_date.strftime("%Y-%m")
    else:
        index_date = begin_date.strftime("%Y-%m-%d")
-    aggregate_report["begin_date"] = begin_date
-    aggregate_report["end_date"] = end_date
-    date_range = [aggregate_report["begin_date"], aggregate_report["end_date"]]

-    org_name_query = Q(dict(match_phrase=dict(org_name=org_name)))
-    report_id_query = Q(dict(match_phrase=dict(report_id=report_id)))
-    domain_query = Q(dict(match_phrase={"published_policy.domain": domain}))
-    begin_date_query = Q(dict(match=dict(date_begin=begin_date)))
-    end_date_query = Q(dict(match=dict(date_end=end_date)))
+    org_name_query = Q(dict(match_phrase=dict(org_name=org_name)))  # type: ignore
+    report_id_query = Q(dict(match_phrase=dict(report_id=report_id)))  # pyright: ignore[reportArgumentType]
+    domain_query = Q(dict(match_phrase={"published_policy.domain": domain}))  # pyright: ignore[reportArgumentType]
+    begin_date_query = Q(dict(match=dict(date_begin=begin_date)))  # pyright: ignore[reportArgumentType]
+    end_date_query = Q(dict(match=dict(date_end=end_date)))  # pyright: ignore[reportArgumentType]

    if index_suffix is not None:
        search_index = "dmarc_aggregate_{0}*".format(index_suffix)
@@ -421,6 +426,8 @@ def save_aggregate_report_to_elasticsearch(
    query = org_name_query & report_id_query & domain_query
    query = query & begin_date_query & end_date_query
    search.query = query
+    begin_date_human = begin_date.strftime("%Y-%m-%d %H:%M:%SZ")
+    end_date_human = end_date.strftime("%Y-%m-%d %H:%M:%SZ")

    try:
        existing = search.execute()
@@ -450,6 +457,17 @@ def save_aggregate_report_to_elasticsearch(
    )

    for record in aggregate_report["records"]:
+        begin_date = human_timestamp_to_datetime(record["interval_begin"], to_utc=True)
+        end_date = human_timestamp_to_datetime(record["interval_end"], to_utc=True)
+        normalized_timespan = record["normalized_timespan"]
+
+        if monthly_indexes:
+            index_date = begin_date.strftime("%Y-%m")
+        else:
+            index_date = begin_date.strftime("%Y-%m-%d")
+        aggregate_report["begin_date"] = begin_date
+        aggregate_report["end_date"] = end_date
+        date_range = [aggregate_report["begin_date"], aggregate_report["end_date"]]
        agg_doc = _AggregateReportDoc(
            xml_schema=aggregate_report["xml_schema"],
            org_name=metadata["org_name"],
@@ -457,8 +475,9 @@ def save_aggregate_report_to_elasticsearch(
            org_extra_contact_info=metadata["org_extra_contact_info"],
            report_id=metadata["report_id"],
            date_range=date_range,
-            date_begin=aggregate_report["begin_date"],
-            date_end=aggregate_report["end_date"],
+            date_begin=begin_date,
+            date_end=end_date,
+            normalized_timespan=normalized_timespan,
            errors=metadata["errors"],
            published_policy=published_policy,
            source_ip_address=record["source"]["ip_address"],
@@ -508,7 +527,7 @@ def save_aggregate_report_to_elasticsearch(
            number_of_shards=number_of_shards, number_of_replicas=number_of_replicas
        )
        create_indexes([index], index_settings)
-        agg_doc.meta.index = index
+        agg_doc.meta.index = index  # pyright: ignore[reportOptionalMemberAccess, reportAttributeAccessIssue]

        try:
            agg_doc.save()
@@ -517,18 +536,18 @@ def save_aggregate_report_to_elasticsearch(


 def save_forensic_report_to_elasticsearch(
-    forensic_report,
-    index_suffix=None,
-    index_prefix=None,
-    monthly_indexes=False,
-    number_of_shards=1,
-    number_of_replicas=0,
+    forensic_report: dict[str, Any],
+    index_suffix: Optional[Any] = None,
+    index_prefix: Optional[str] = None,
+    monthly_indexes: Optional[bool] = False,
+    number_of_shards: int = 1,
+    number_of_replicas: int = 0,
 ):
    """
    Saves a parsed DMARC forensic report to Elasticsearch

    Args:
-        forensic_report (OrderedDict): A parsed forensic report
+        forensic_report (dict): A parsed forensic report
        index_suffix (str): The suffix of the name of the index to save to
        index_prefix (str): The prefix of the name of the index to save to
        monthly_indexes (bool): Use monthly indexes instead of daily
@@ -548,7 +567,7 @@ def save_forensic_report_to_elasticsearch(
        sample_date = forensic_report["parsed_sample"]["date"]
        sample_date = human_timestamp_to_datetime(sample_date)
    original_headers = forensic_report["parsed_sample"]["headers"]
-    headers = OrderedDict()
+    headers: dict[str, Any] = {}
    for original_header in original_headers:
        headers[original_header.lower()] = original_headers[original_header]

@@ -562,7 +581,7 @@ def save_forensic_report_to_elasticsearch(
    if index_prefix is not None:
        search_index = "{0}{1}".format(index_prefix, search_index)
    search = Search(index=search_index)
-    q = Q(dict(match=dict(arrival_date=arrival_date_epoch_milliseconds)))
+    q = Q(dict(match=dict(arrival_date=arrival_date_epoch_milliseconds)))  # pyright: ignore[reportArgumentType]

    from_ = None
    to_ = None
@@ -577,7 +596,7 @@ def save_forensic_report_to_elasticsearch(

        from_ = dict()
        from_["sample.headers.from"] = headers["from"]
-        from_query = Q(dict(match_phrase=from_))
+        from_query = Q(dict(match_phrase=from_))  # pyright: ignore[reportArgumentType]
        q = q & from_query
    if "to" in headers:
        # We convert the TO header from a string list to a flat string.
@@ -589,12 +608,12 @@ def save_forensic_report_to_elasticsearch(

        to_ = dict()
        to_["sample.headers.to"] = headers["to"]
-        to_query = Q(dict(match_phrase=to_))
+        to_query = Q(dict(match_phrase=to_))  # pyright: ignore[reportArgumentType]
        q = q & to_query
    if "subject" in headers:
        subject = headers["subject"]
        subject_query = {"match_phrase": {"sample.headers.subject": subject}}
-        q = q & Q(subject_query)
+        q = q & Q(subject_query)  # pyright: ignore[reportArgumentType]

    search.query = q
    existing = search.execute()
@@ -672,7 +691,7 @@ def save_forensic_report_to_elasticsearch(
            number_of_shards=number_of_shards, number_of_replicas=number_of_replicas
        )
        create_indexes([index], index_settings)
-        forensic_doc.meta.index = index
+        forensic_doc.meta.index = index  # pyright: ignore[reportAttributeAccessIssue, reportOptionalMemberAccess]
        try:
            forensic_doc.save()
        except Exception as e:
@@ -684,18 +703,18 @@ def save_forensic_report_to_elasticsearch(


 def save_smtp_tls_report_to_elasticsearch(
-    report,
-    index_suffix=None,
-    index_prefix=None,
-    monthly_indexes=False,
-    number_of_shards=1,
-    number_of_replicas=0,
+    report: dict[str, Any],
+    index_suffix: Optional[str] = None,
+    index_prefix: Optional[str] = None,
+    monthly_indexes: bool = False,
+    number_of_shards: int = 1,
+    number_of_replicas: int = 0,
 ):
    """
    Saves a parsed SMTP TLS report to Elasticsearch

    Args:
-        report (OrderedDict): A parsed SMTP TLS report
+        report (dict): A parsed SMTP TLS report
        index_suffix (str): The suffix of the name of the index to save to
        index_prefix (str): The prefix of the name of the index to save to
        monthly_indexes (bool): Use monthly indexes instead of daily indexes
@@ -719,10 +738,10 @@ def save_smtp_tls_report_to_elasticsearch(
    report["begin_date"] = begin_date
    report["end_date"] = end_date

-    org_name_query = Q(dict(match_phrase=dict(org_name=org_name)))
-    report_id_query = Q(dict(match_phrase=dict(report_id=report_id)))
-    begin_date_query = Q(dict(match=dict(date_begin=begin_date)))
-    end_date_query = Q(dict(match=dict(date_end=end_date)))
+    org_name_query = Q(dict(match_phrase=dict(org_name=org_name)))  # pyright: ignore[reportArgumentType]
+    report_id_query = Q(dict(match_phrase=dict(report_id=report_id)))  # pyright: ignore[reportArgumentType]
+    begin_date_query = Q(dict(match=dict(date_begin=begin_date)))  # pyright: ignore[reportArgumentType]
+    end_date_query = Q(dict(match=dict(date_end=end_date)))  # pyright: ignore[reportArgumentType]

    if index_suffix is not None:
        search_index = "smtp_tls_{0}*".format(index_suffix)
@@ -781,7 +800,7 @@ def save_smtp_tls_report_to_elasticsearch(
        policy_doc = _SMTPTLSPolicyDoc(
            policy_domain=policy["policy_domain"],
            policy_type=policy["policy_type"],
-            succesful_session_count=policy["successful_session_count"],
+            successful_session_count=policy["successful_session_count"],
            failed_session_count=policy["failed_session_count"],
            policy_string=policy_strings,
            mx_host_patterns=mx_host_patterns,
@@ -823,10 +842,10 @@ def save_smtp_tls_report_to_elasticsearch(
                    additional_information_uri=additional_information_uri,
                    failure_reason_code=failure_reason_code,
                )
-        smtp_tls_doc.policies.append(policy_doc)
+        smtp_tls_doc.policies.append(policy_doc)  # pyright: ignore[reportCallIssue]

    create_indexes([index], index_settings)
-    smtp_tls_doc.meta.index = index
+    smtp_tls_doc.meta.index = index  # pyright: ignore[reportOptionalMemberAccess, reportAttributeAccessIssue]

    try:
        smtp_tls_doc.save()
--- a/parsedmarc/gelf.py
+++ b/parsedmarc/gelf.py
@@ -1,17 +1,19 @@
 # -*- coding: utf-8 -*-

+from __future__ import annotations
+
 import logging
 import logging.handlers
-import json
 import threading
+from typing import Any
+
+from pygelf import GelfTcpHandler, GelfTlsHandler, GelfUdpHandler

 from parsedmarc import (
    parsed_aggregate_reports_to_csv_rows,
    parsed_forensic_reports_to_csv_rows,
    parsed_smtp_tls_reports_to_csv_rows,
 )
-from pygelf import GelfTcpHandler, GelfUdpHandler, GelfTlsHandler
-

 log_context_data = threading.local()

@@ -48,7 +50,7 @@ class GelfClient(object):
        )
        self.logger.addHandler(self.handler)

-    def save_aggregate_report_to_gelf(self, aggregate_reports):
+    def save_aggregate_report_to_gelf(self, aggregate_reports: list[dict[str, Any]]):
        rows = parsed_aggregate_reports_to_csv_rows(aggregate_reports)
        for row in rows:
            log_context_data.parsedmarc = row
@@ -56,12 +58,14 @@ class GelfClient(object):

        log_context_data.parsedmarc = None

-    def save_forensic_report_to_gelf(self, forensic_reports):
+    def save_forensic_report_to_gelf(self, forensic_reports: list[dict[str, Any]]):
        rows = parsed_forensic_reports_to_csv_rows(forensic_reports)
        for row in rows:
-            self.logger.info(json.dumps(row))
+            log_context_data.parsedmarc = row
+            self.logger.info("parsedmarc forensic report")

-    def save_smtp_tls_report_to_gelf(self, smtp_tls_reports):
+    def save_smtp_tls_report_to_gelf(self, smtp_tls_reports: dict[str, Any]):
        rows = parsed_smtp_tls_reports_to_csv_rows(smtp_tls_reports)
        for row in rows:
-            self.logger.info(json.dumps(row))
+            log_context_data.parsedmarc = row
+            self.logger.info("parsedmarc smtptls report")
--- a/parsedmarc/kafkaclient.py
+++ b/parsedmarc/kafkaclient.py
@@ -1,15 +1,17 @@
 # -*- coding: utf-8 -*-

+from __future__ import annotations
+
 import json
-from ssl import create_default_context
+from ssl import SSLContext, create_default_context
+from typing import Any, Optional, Union

 from kafka import KafkaProducer
 from kafka.errors import NoBrokersAvailable, UnknownTopicOrPartitionError
-from collections import OrderedDict
-from parsedmarc.utils import human_timestamp_to_datetime

 from parsedmarc import __version__
 from parsedmarc.log import logger
+from parsedmarc.utils import human_timestamp_to_datetime


 class KafkaError(RuntimeError):
@@ -18,7 +20,13 @@ class KafkaError(RuntimeError):

 class KafkaClient(object):
    def __init__(
-        self, kafka_hosts, ssl=False, username=None, password=None, ssl_context=None
+        self,
+        kafka_hosts: list[str],
+        *,
+        ssl: Optional[bool] = False,
+        username: Optional[str] = None,
+        password: Optional[str] = None,
+        ssl_context: Optional[SSLContext] = None,
    ):
        """
        Initializes the Kafka client
@@ -28,7 +36,7 @@ class KafkaClient(object):
            ssl (bool): Use a SSL/TLS connection
            username (str): An optional username
            password (str):  An optional password
-            ssl_context: SSL context options
+            ssl_context (SSLContext): SSL context options

        Notes:
            ``use_ssl=True`` is implied when a username or password are
@@ -38,7 +46,7 @@ class KafkaClient(object):
            ``$ConnectionString``, and the password is the
            Azure Event Hub connection string.
        """
-        config = dict(
+        config: dict[str, Any] = dict(
            value_serializer=lambda v: json.dumps(v).encode("utf-8"),
            bootstrap_servers=kafka_hosts,
            client_id="parsedmarc-{0}".format(__version__),
@@ -55,7 +63,7 @@ class KafkaClient(object):
            raise KafkaError("No Kafka brokers available")

    @staticmethod
-    def strip_metadata(report):
+    def strip_metadata(report: dict[str, Any]):
        """
        Duplicates org_name, org_email and report_id into JSON root
        and removes report_metadata key to bring it more inline
@@ -69,7 +77,7 @@ class KafkaClient(object):
        return report

    @staticmethod
-    def generate_daterange(report):
+    def generate_date_range(report: dict[str, Any]):
        """
        Creates a date_range timestamp with format YYYY-MM-DD-T-HH:MM:SS
        based on begin and end dates for easier parsing in Kibana.
@@ -86,7 +94,11 @@ class KafkaClient(object):
        logger.debug("date_range is {}".format(date_range))
        return date_range

-    def save_aggregate_reports_to_kafka(self, aggregate_reports, aggregate_topic):
+    def save_aggregate_reports_to_kafka(
+        self,
+        aggregate_reports: Union[dict[str, Any], list[dict[str, Any]]],
+        aggregate_topic: str,
+    ):
        """
        Saves aggregate DMARC reports to Kafka

@@ -96,16 +108,14 @@ class KafkaClient(object):
            aggregate_topic (str): The name of the Kafka topic

        """
-        if isinstance(aggregate_reports, dict) or isinstance(
-            aggregate_reports, OrderedDict
-        ):
+        if isinstance(aggregate_reports, dict):
            aggregate_reports = [aggregate_reports]

        if len(aggregate_reports) < 1:
            return

        for report in aggregate_reports:
-            report["date_range"] = self.generate_daterange(report)
+            report["date_range"] = self.generate_date_range(report)
            report = self.strip_metadata(report)

            for slice in report["records"]:
@@ -129,7 +139,11 @@ class KafkaClient(object):
                except Exception as e:
                    raise KafkaError("Kafka error: {0}".format(e.__str__()))

-    def save_forensic_reports_to_kafka(self, forensic_reports, forensic_topic):
+    def save_forensic_reports_to_kafka(
+        self,
+        forensic_reports: Union[dict[str, Any], list[dict[str, Any]]],
+        forensic_topic: str,
+    ):
        """
        Saves forensic DMARC reports to Kafka, sends individual
        records (slices) since Kafka requires messages to be <= 1MB
@@ -159,7 +173,11 @@ class KafkaClient(object):
        except Exception as e:
            raise KafkaError("Kafka error: {0}".format(e.__str__()))

-    def save_smtp_tls_reports_to_kafka(self, smtp_tls_reports, smtp_tls_topic):
+    def save_smtp_tls_reports_to_kafka(
+        self,
+        smtp_tls_reports: Union[list[dict[str, Any]], dict[str, Any]],
+        smtp_tls_topic: str,
+    ):
        """
        Saves SMTP TLS reports to Kafka, sends individual
        records (slices) since Kafka requires messages to be <= 1MB
--- a/parsedmarc/loganalytics.py
+++ b/parsedmarc/loganalytics.py
@@ -1,9 +1,15 @@
 # -*- coding: utf-8 -*-
-from parsedmarc.log import logger
+
+from __future__ import annotations
+
+from typing import Any
+
 from azure.core.exceptions import HttpResponseError
 from azure.identity import ClientSecretCredential
 from azure.monitor.ingestion import LogsIngestionClient

+from parsedmarc.log import logger
+

 class LogAnalyticsException(Exception):
    """Raised when an Elasticsearch error occurs"""
@@ -102,7 +108,12 @@ class LogAnalyticsClient(object):
                "Invalid configuration. " + "One or more required settings are missing."
            )

-    def publish_json(self, results, logs_client: LogsIngestionClient, dcr_stream: str):
+    def publish_json(
+        self,
+        results,
+        logs_client: LogsIngestionClient,
+        dcr_stream: str,
+    ):
        """
        Background function to publish given
        DMARC report to specific Data Collection Rule.
@@ -121,7 +132,11 @@ class LogAnalyticsClient(object):
            raise LogAnalyticsException("Upload failed: {error}".format(error=e))

    def publish_results(
-        self, results, save_aggregate: bool, save_forensic: bool, save_smtp_tls: bool
+        self,
+        results: dict[str, Any],
+        save_aggregate: bool,
+        save_forensic: bool,
+        save_smtp_tls: bool,
    ):
        """
        Function to publish DMARC and/or SMTP TLS reports to Log Analytics
--- a/parsedmarc/mail/gmail.py
+++ b/parsedmarc/mail/gmail.py
@@ -1,3 +1,7 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
 from base64 import urlsafe_b64decode
 from functools import lru_cache
 from pathlib import Path
@@ -112,14 +116,14 @@ class GmailConnection(MailboxConnection):
        else:
            return [id for id in self._fetch_all_message_ids(reports_label_id)]

-    def fetch_message(self, message_id):
+    def fetch_message(self, message_id) -> str:
        msg = (
            self.service.users()
            .messages()
            .get(userId="me", id=message_id, format="raw")
            .execute()
        )
-        return urlsafe_b64decode(msg["raw"])
+        return urlsafe_b64decode(msg["raw"]).decode(errors="replace")

    def delete_message(self, message_id: str):
        self.service.users().messages().delete(userId="me", id=message_id)
@@ -152,3 +156,4 @@ class GmailConnection(MailboxConnection):
        for label in labels:
            if label_name == label["id"] or label_name == label["name"]:
                return label["id"]
+        return ""
--- a/parsedmarc/mail/graph.py
+++ b/parsedmarc/mail/graph.py
@@ -1,8 +1,12 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
 from enum import Enum
 from functools import lru_cache
 from pathlib import Path
 from time import sleep
-from typing import List, Optional
+from typing import Any, List, Optional, Union

 from azure.identity import (
    UsernamePasswordCredential,
@@ -16,6 +20,59 @@ from msgraph.core import GraphClient
 from parsedmarc.log import logger
 from parsedmarc.mail.mailbox_connection import MailboxConnection

+# Mapping of common folder names to Microsoft Graph well-known folder names
+# This avoids the "Default folder Root not found" error on uninitialized mailboxes
+WELL_KNOWN_FOLDER_MAP = {
+    # English names
+    "inbox": "inbox",
+    "sent items": "sentitems",
+    "sent": "sentitems",
+    "sentitems": "sentitems",
+    "deleted items": "deleteditems",
+    "deleted": "deleteditems",
+    "deleteditems": "deleteditems",
+    "trash": "deleteditems",
+    "drafts": "drafts",
+    "junk email": "junkemail",
+    "junk": "junkemail",
+    "junkemail": "junkemail",
+    "spam": "junkemail",
+    "archive": "archive",
+    "outbox": "outbox",
+    "conversation history": "conversationhistory",
+    "conversationhistory": "conversationhistory",
+    # German names
+    "posteingang": "inbox",
+    "gesendete elemente": "sentitems",
+    "gesendet": "sentitems",
+    "gelöschte elemente": "deleteditems",
+    "gelöscht": "deleteditems",
+    "entwürfe": "drafts",
+    "junk-e-mail": "junkemail",
+    "archiv": "archive",
+    "postausgang": "outbox",
+    # French names
+    "boîte de réception": "inbox",
+    "éléments envoyés": "sentitems",
+    "envoyés": "sentitems",
+    "éléments supprimés": "deleteditems",
+    "supprimés": "deleteditems",
+    "brouillons": "drafts",
+    "courrier indésirable": "junkemail",
+    "archives": "archive",
+    "boîte d'envoi": "outbox",
+    # Spanish names
+    "bandeja de entrada": "inbox",
+    "elementos enviados": "sentitems",
+    "enviados": "sentitems",
+    "elementos eliminados": "deleteditems",
+    "eliminados": "deleteditems",
+    "borradores": "drafts",
+    "correo no deseado": "junkemail",
+    "archivar": "archive",
+    "bandeja de salida": "outbox",
+}
+

 class AuthMethod(Enum):
    DeviceCode = 1
@@ -24,7 +81,7 @@ class AuthMethod(Enum):


 def _get_cache_args(token_path: Path, allow_unencrypted_storage):
-    cache_args = {
+    cache_args: dict[str, Any] = {
        "cache_persistence_options": TokenCachePersistenceOptions(
            name="parsedmarc", allow_unencrypted_storage=allow_unencrypted_storage
        )
@@ -126,6 +183,13 @@ class MSGraphConnection(MailboxConnection):
        self.mailbox_name = mailbox

    def create_folder(self, folder_name: str):
+        # Check if this is a well-known folder - they already exist and cannot be created
+        if "/" not in folder_name:
+            well_known_name = WELL_KNOWN_FOLDER_MAP.get(folder_name.lower())
+            if well_known_name:
+                logger.debug(f"Folder '{folder_name}' is a well-known folder, skipping creation")
+                return
+        
        sub_url = ""
        path_parts = folder_name.split("/")
        if len(path_parts) > 1:  # Folder is a subFolder
@@ -147,9 +211,9 @@ class MSGraphConnection(MailboxConnection):
        else:
            logger.warning(f"Unknown response {resp.status_code} {resp.json()}")

-    def fetch_messages(self, folder_name: str, **kwargs) -> List[str]:
+    def fetch_messages(self, reports_folder: str, **kwargs) -> List[str]:
        """Returns a list of message UIDs in the specified folder"""
-        folder_id = self._find_folder_id_from_folder_path(folder_name)
+        folder_id = self._find_folder_id_from_folder_path(reports_folder)
        url = f"/users/{self.mailbox_name}/mailFolders/{folder_id}/messages"
        since = kwargs.get("since")
        if not since:
@@ -162,7 +226,7 @@ class MSGraphConnection(MailboxConnection):

    def _get_all_messages(self, url, batch_size, since):
        messages: list
-        params = {"$select": "id"}
+        params: dict[str, Union[str, int]] = {"$select": "id"}
        if since:
            params["$filter"] = f"receivedDateTime ge {since}"
        if batch_size and batch_size > 0:
@@ -242,6 +306,12 @@ class MSGraphConnection(MailboxConnection):
                parent_folder_id = folder_id
            return self._find_folder_id_with_parent(path_parts[-1], parent_folder_id)
        else:
+            # Check if this is a well-known folder name (case-insensitive)
+            well_known_name = WELL_KNOWN_FOLDER_MAP.get(folder_name.lower())
+            if well_known_name:
+                # Use well-known folder name directly to avoid querying uninitialized mailboxes
+                logger.debug(f"Using well-known folder name '{well_known_name}' for '{folder_name}'")
+                return well_known_name
            return self._find_folder_id_with_parent(folder_name, None)

    def _find_folder_id_with_parent(
--- a/parsedmarc/mail/imap.py
+++ b/parsedmarc/mail/imap.py
@@ -1,3 +1,9 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
+from typing import cast
+
 from time import sleep

 from imapclient.exceptions import IMAPClientError
@@ -11,14 +17,14 @@ from parsedmarc.mail.mailbox_connection import MailboxConnection
 class IMAPConnection(MailboxConnection):
    def __init__(
        self,
-        host=None,
-        user=None,
-        password=None,
-        port=None,
-        ssl=True,
-        verify=True,
-        timeout=30,
-        max_retries=4,
+        host: str,
+        user: str,
+        password: str,
+        port: int = 993,
+        ssl: bool = True,
+        verify: bool = True,
+        timeout: int = 30,
+        max_retries: int = 4,
    ):
        self._username = user
        self._password = password
@@ -40,18 +46,18 @@ class IMAPConnection(MailboxConnection):
    def fetch_messages(self, reports_folder: str, **kwargs):
        self._client.select_folder(reports_folder)
        since = kwargs.get("since")
-        if since:
-            return self._client.search(["SINCE", since])
+        if since is not None:
+            return self._client.search(f"SINCE {since}")
        else:
            return self._client.search()

-    def fetch_message(self, message_id):
-        return self._client.fetch_message(message_id, parse=False)
+    def fetch_message(self, message_id: int):
+        return cast(str, self._client.fetch_message(message_id, parse=False))

-    def delete_message(self, message_id: str):
+    def delete_message(self, message_id: int):
        self._client.delete_messages([message_id])

-    def move_message(self, message_id: str, folder_name: str):
+    def move_message(self, message_id: int, folder_name: str):
        self._client.move_messages([message_id], folder_name)

    def keepalive(self):
--- a/parsedmarc/mail/mailbox_connection.py
+++ b/parsedmarc/mail/mailbox_connection.py
@@ -1,5 +1,8 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
 from abc import ABC
-from typing import List


 class MailboxConnection(ABC):
@@ -10,16 +13,16 @@ class MailboxConnection(ABC):
    def create_folder(self, folder_name: str):
        raise NotImplementedError

-    def fetch_messages(self, reports_folder: str, **kwargs) -> List[str]:
+    def fetch_messages(self, reports_folder: str, **kwargs):
        raise NotImplementedError

    def fetch_message(self, message_id) -> str:
        raise NotImplementedError

-    def delete_message(self, message_id: str):
+    def delete_message(self, message_id):
        raise NotImplementedError

-    def move_message(self, message_id: str, folder_name: str):
+    def move_message(self, message_id, folder_name: str):
        raise NotImplementedError

    def keepalive(self):
--- a/parsedmarc/mail/maildir.py
+++ b/parsedmarc/mail/maildir.py
@@ -1,16 +1,21 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
+import mailbox
+import os
 from time import sleep
+from typing import Dict

 from parsedmarc.log import logger
 from parsedmarc.mail.mailbox_connection import MailboxConnection
-import mailbox
-import os


 class MaildirConnection(MailboxConnection):
    def __init__(
        self,
-        maildir_path=None,
-        maildir_create=False,
+        maildir_path: str,
+        maildir_create: bool = False,
    ):
        self._maildir_path = maildir_path
        self._maildir_create = maildir_create
@@ -27,27 +32,31 @@ class MaildirConnection(MailboxConnection):
                )
                raise Exception(ex)
        self._client = mailbox.Maildir(maildir_path, create=maildir_create)
-        self._subfolder_client = {}
+        self._subfolder_client: Dict[str, mailbox.Maildir] = {}

    def create_folder(self, folder_name: str):
        self._subfolder_client[folder_name] = self._client.add_folder(folder_name)
-        self._client.add_folder(folder_name)

    def fetch_messages(self, reports_folder: str, **kwargs):
        return self._client.keys()

-    def fetch_message(self, message_id):
-        return self._client.get(message_id).as_string()
+    def fetch_message(self, message_id: str) -> str:
+        msg = self._client.get(message_id)
+        if msg is not None:
+            msg = msg.as_string()
+            if msg is not None:
+                return msg
+        return ""

    def delete_message(self, message_id: str):
        self._client.remove(message_id)

    def move_message(self, message_id: str, folder_name: str):
        message_data = self._client.get(message_id)
-        if folder_name not in self._subfolder_client.keys():
-            self._subfolder_client = mailbox.Maildir(
-                os.join(self.maildir_path, folder_name), create=self.maildir_create
-            )
+        if message_data is None:
+            return
+        if folder_name not in self._subfolder_client:
+            self._subfolder_client[folder_name] = self._client.add_folder(folder_name)
        self._subfolder_client[folder_name].add(message_data)
        self._client.remove(message_id)

--- a/parsedmarc/opensearch.py
+++ b/parsedmarc/opensearch.py
@@ -1,27 +1,29 @@
 # -*- coding: utf-8 -*-

-from collections import OrderedDict
+from __future__ import annotations
+
+from typing import Any, Optional, Union

 from opensearchpy import (
-    Q,
-    connections,
-    Object,
+    Boolean,
+    Date,
    Document,
    Index,
-    Nested,
    InnerDoc,
    Integer,
-    Text,
-    Boolean,
    Ip,
-    Date,
+    Nested,
+    Object,
+    Q,
    Search,
+    Text,
+    connections,
 )
 from opensearchpy.helpers import reindex

+from parsedmarc import InvalidForensicReport
 from parsedmarc.log import logger
 from parsedmarc.utils import human_timestamp_to_datetime
-from parsedmarc import InvalidForensicReport


 class OpenSearchError(Exception):
@@ -67,6 +69,8 @@ class _AggregateReportDoc(Document):
    date_range = Date()
    date_begin = Date()
    date_end = Date()
+    normalized_timespan = Boolean()
+    original_timespan_seconds = Integer
    errors = Text()
    published_policy = Object(_PublishedPolicy)
    source_ip_address = Ip()
@@ -87,18 +91,18 @@ class _AggregateReportDoc(Document):
    dkim_results = Nested(_DKIMResult)
    spf_results = Nested(_SPFResult)

-    def add_policy_override(self, type_, comment):
+    def add_policy_override(self, type_: str, comment: str):
        self.policy_overrides.append(_PolicyOverride(type=type_, comment=comment))

-    def add_dkim_result(self, domain, selector, result):
+    def add_dkim_result(self, domain: str, selector: str, result: _DKIMResult):
        self.dkim_results.append(
            _DKIMResult(domain=domain, selector=selector, result=result)
        )

-    def add_spf_result(self, domain, scope, result):
+    def add_spf_result(self, domain: str, scope: str, result: _SPFResult):
        self.spf_results.append(_SPFResult(domain=domain, scope=scope, result=result))

-    def save(self, **kwargs):
+    def save(self, **kwargs):  # pyright: ignore[reportIncompatibleMethodOverride]
        self.passed_dmarc = False
        self.passed_dmarc = self.spf_aligned or self.dkim_aligned

@@ -131,21 +135,21 @@ class _ForensicSampleDoc(InnerDoc):
    body = Text()
    attachments = Nested(_EmailAttachmentDoc)

-    def add_to(self, display_name, address):
+    def add_to(self, display_name: str, address: str):
        self.to.append(_EmailAddressDoc(display_name=display_name, address=address))

-    def add_reply_to(self, display_name, address):
+    def add_reply_to(self, display_name: str, address: str):
        self.reply_to.append(
            _EmailAddressDoc(display_name=display_name, address=address)
        )

-    def add_cc(self, display_name, address):
+    def add_cc(self, display_name: str, address: str):
        self.cc.append(_EmailAddressDoc(display_name=display_name, address=address))

-    def add_bcc(self, display_name, address):
+    def add_bcc(self, display_name: str, address: str):
        self.bcc.append(_EmailAddressDoc(display_name=display_name, address=address))

-    def add_attachment(self, filename, content_type, sha256):
+    def add_attachment(self, filename: str, content_type: str, sha256: str):
        self.attachments.append(
            _EmailAttachmentDoc(
                filename=filename, content_type=content_type, sha256=sha256
@@ -197,15 +201,15 @@ class _SMTPTLSPolicyDoc(InnerDoc):

    def add_failure_details(
        self,
-        result_type,
-        ip_address,
-        receiving_ip,
-        receiving_mx_helo,
-        failed_session_count,
-        sending_mta_ip=None,
-        receiving_mx_hostname=None,
-        additional_information_uri=None,
-        failure_reason_code=None,
+        result_type: Optional[str] = None,
+        ip_address: Optional[str] = None,
+        receiving_ip: Optional[str] = None,
+        receiving_mx_helo: Optional[str] = None,
+        failed_session_count: Optional[int] = None,
+        sending_mta_ip: Optional[str] = None,
+        receiving_mx_hostname: Optional[str] = None,
+        additional_information_uri: Optional[str] = None,
+        failure_reason_code: Union[str, int, None] = None,
    ):
        _details = _SMTPTLSFailureDetailsDoc(
            result_type=result_type,
@@ -235,13 +239,14 @@ class _SMTPTLSReportDoc(Document):

    def add_policy(
        self,
-        policy_type,
-        policy_domain,
-        successful_session_count,
-        failed_session_count,
-        policy_string=None,
-        mx_host_patterns=None,
-        failure_details=None,
+        policy_type: str,
+        policy_domain: str,
+        successful_session_count: int,
+        failed_session_count: int,
+        *,
+        policy_string: Optional[str] = None,
+        mx_host_patterns: Optional[list[str]] = None,
+        failure_details: Optional[str] = None,
    ):
        self.policies.append(
            policy_type=policy_type,
@@ -259,24 +264,25 @@ class AlreadySaved(ValueError):


 def set_hosts(
-    hosts,
-    use_ssl=False,
-    ssl_cert_path=None,
-    username=None,
-    password=None,
-    apiKey=None,
-    timeout=60.0,
+    hosts: Union[str, list[str]],
+    *,
+    use_ssl: Optional[bool] = False,
+    ssl_cert_path: Optional[str] = None,
+    username: Optional[str] = None,
+    password: Optional[str] = None,
+    api_key: Optional[str] = None,
+    timeout: Optional[float] = 60.0,
 ):
    """
    Sets the OpenSearch hosts to use

    Args:
-        hosts (str|list): A hostname or URL, or list of hostnames or URLs
+        hosts (str|list[str]): A single hostname or URL, or list of hostnames or URLs
        use_ssl (bool): Use an HTTPS connection to the server
        ssl_cert_path (str): Path to the certificate chain
        username (str): The username to use for authentication
        password (str): The password to use for authentication
-        apiKey (str): The Base64 encoded API key to use for authentication
+        api_key (str): The Base64 encoded API key to use for authentication
        timeout (float): Timeout in seconds
    """
    if not isinstance(hosts, list):
@@ -289,14 +295,14 @@ def set_hosts(
            conn_params["ca_certs"] = ssl_cert_path
        else:
            conn_params["verify_certs"] = False
-    if username:
+    if username and password:
        conn_params["http_auth"] = username + ":" + password
-    if apiKey:
-        conn_params["api_key"] = apiKey
+    if api_key:
+        conn_params["api_key"] = api_key
    connections.create_connection(**conn_params)


-def create_indexes(names, settings=None):
+def create_indexes(names: list[str], settings: Optional[dict[str, Any]] = None):
    """
    Create OpenSearch indexes

@@ -319,7 +325,10 @@ def create_indexes(names, settings=None):
            raise OpenSearchError("OpenSearch error: {0}".format(e.__str__()))


-def migrate_indexes(aggregate_indexes=None, forensic_indexes=None):
+def migrate_indexes(
+    aggregate_indexes: Optional[list[str]] = None,
+    forensic_indexes: Optional[list[str]] = None,
+):
    """
    Updates index mappings

@@ -366,18 +375,18 @@ def migrate_indexes(aggregate_indexes=None, forensic_indexes=None):


 def save_aggregate_report_to_opensearch(
-    aggregate_report,
-    index_suffix=None,
-    index_prefix=None,
-    monthly_indexes=False,
-    number_of_shards=1,
-    number_of_replicas=0,
+    aggregate_report: dict[str, Any],
+    index_suffix: Optional[str] = None,
+    index_prefix: Optional[str] = None,
+    monthly_indexes: bool = False,
+    number_of_shards: int = 1,
+    number_of_replicas: int = 0,
 ):
    """
    Saves a parsed DMARC aggregate report to OpenSearch

    Args:
-        aggregate_report (OrderedDict): A parsed forensic report
+        aggregate_report (dict): A parsed forensic report
        index_suffix (str): The suffix of the name of the index to save to
        index_prefix (str): The prefix of the name of the index to save to
        monthly_indexes (bool): Use monthly indexes instead of daily indexes
@@ -395,15 +404,11 @@ def save_aggregate_report_to_opensearch(
    domain = aggregate_report["policy_published"]["domain"]
    begin_date = human_timestamp_to_datetime(metadata["begin_date"], to_utc=True)
    end_date = human_timestamp_to_datetime(metadata["end_date"], to_utc=True)
-    begin_date_human = begin_date.strftime("%Y-%m-%d %H:%M:%SZ")
-    end_date_human = end_date.strftime("%Y-%m-%d %H:%M:%SZ")
+
    if monthly_indexes:
        index_date = begin_date.strftime("%Y-%m")
    else:
        index_date = begin_date.strftime("%Y-%m-%d")
-    aggregate_report["begin_date"] = begin_date
-    aggregate_report["end_date"] = end_date
-    date_range = [aggregate_report["begin_date"], aggregate_report["end_date"]]

    org_name_query = Q(dict(match_phrase=dict(org_name=org_name)))
    report_id_query = Q(dict(match_phrase=dict(report_id=report_id)))
@@ -421,6 +426,8 @@ def save_aggregate_report_to_opensearch(
    query = org_name_query & report_id_query & domain_query
    query = query & begin_date_query & end_date_query
    search.query = query
+    begin_date_human = begin_date.strftime("%Y-%m-%d %H:%M:%SZ")
+    end_date_human = end_date.strftime("%Y-%m-%d %H:%M:%SZ")

    try:
        existing = search.execute()
@@ -450,6 +457,17 @@ def save_aggregate_report_to_opensearch(
    )

    for record in aggregate_report["records"]:
+        begin_date = human_timestamp_to_datetime(record["interval_begin"], to_utc=True)
+        end_date = human_timestamp_to_datetime(record["interval_end"], to_utc=True)
+        normalized_timespan = record["normalized_timespan"]
+
+        if monthly_indexes:
+            index_date = begin_date.strftime("%Y-%m")
+        else:
+            index_date = begin_date.strftime("%Y-%m-%d")
+        aggregate_report["begin_date"] = begin_date
+        aggregate_report["end_date"] = end_date
+        date_range = [aggregate_report["begin_date"], aggregate_report["end_date"]]
        agg_doc = _AggregateReportDoc(
            xml_schema=aggregate_report["xml_schema"],
            org_name=metadata["org_name"],
@@ -457,8 +475,9 @@ def save_aggregate_report_to_opensearch(
            org_extra_contact_info=metadata["org_extra_contact_info"],
            report_id=metadata["report_id"],
            date_range=date_range,
-            date_begin=aggregate_report["begin_date"],
-            date_end=aggregate_report["end_date"],
+            date_begin=begin_date,
+            date_end=end_date,
+            normalized_timespan=normalized_timespan,
            errors=metadata["errors"],
            published_policy=published_policy,
            source_ip_address=record["source"]["ip_address"],
@@ -517,18 +536,18 @@ def save_aggregate_report_to_opensearch(


 def save_forensic_report_to_opensearch(
-    forensic_report,
-    index_suffix=None,
-    index_prefix=None,
-    monthly_indexes=False,
-    number_of_shards=1,
-    number_of_replicas=0,
+    forensic_report: dict[str, Any],
+    index_suffix: Optional[str] = None,
+    index_prefix: Optional[str] = None,
+    monthly_indexes: bool = False,
+    number_of_shards: int = 1,
+    number_of_replicas: int = 0,
 ):
    """
    Saves a parsed DMARC forensic report to OpenSearch

    Args:
-        forensic_report (OrderedDict): A parsed forensic report
+        forensic_report (dict): A parsed forensic report
        index_suffix (str): The suffix of the name of the index to save to
        index_prefix (str): The prefix of the name of the index to save to
        monthly_indexes (bool): Use monthly indexes instead of daily
@@ -548,7 +567,7 @@ def save_forensic_report_to_opensearch(
        sample_date = forensic_report["parsed_sample"]["date"]
        sample_date = human_timestamp_to_datetime(sample_date)
    original_headers = forensic_report["parsed_sample"]["headers"]
-    headers = OrderedDict()
+    headers: dict[str, Any] = {}
    for original_header in original_headers:
        headers[original_header.lower()] = original_headers[original_header]

@@ -684,18 +703,18 @@ def save_forensic_report_to_opensearch(


 def save_smtp_tls_report_to_opensearch(
-    report,
-    index_suffix=None,
-    index_prefix=None,
-    monthly_indexes=False,
-    number_of_shards=1,
-    number_of_replicas=0,
+    report: dict[str, Any],
+    index_suffix: Optional[str] = None,
+    index_prefix: Optional[str] = None,
+    monthly_indexes: bool = False,
+    number_of_shards: int = 1,
+    number_of_replicas: int = 0,
 ):
    """
    Saves a parsed SMTP TLS report to OpenSearch

    Args:
-        report (OrderedDict): A parsed SMTP TLS report
+        report (dict): A parsed SMTP TLS report
        index_suffix (str): The suffix of the name of the index to save to
        index_prefix (str): The prefix of the name of the index to save to
        monthly_indexes (bool): Use monthly indexes instead of daily indexes
@@ -705,7 +724,7 @@ def save_smtp_tls_report_to_opensearch(
    Raises:
            AlreadySaved
    """
-    logger.info("Saving aggregate report to OpenSearch")
+    logger.info("Saving SMTP TLS report to OpenSearch")
    org_name = report["organization_name"]
    report_id = report["report_id"]
    begin_date = human_timestamp_to_datetime(report["begin_date"], to_utc=True)
@@ -781,7 +800,7 @@ def save_smtp_tls_report_to_opensearch(
        policy_doc = _SMTPTLSPolicyDoc(
            policy_domain=policy["policy_domain"],
            policy_type=policy["policy_type"],
-            succesful_session_count=policy["successful_session_count"],
+            successful_session_count=policy["successful_session_count"],
            failed_session_count=policy["failed_session_count"],
            policy_string=policy_strings,
            mx_host_patterns=mx_host_patterns,
--- a/parsedmarc/resources/maps/base_reverse_dns_map.csv
+++ b/parsedmarc/resources/maps/base_reverse_dns_map.csv
--- a/parsedmarc/resources/maps/base_reverse_dns_types.txt
+++ b/parsedmarc/resources/maps/base_reverse_dns_types.txt
@@ -0,0 +1,44 @@
+Agriculture
+Automotive
+Beauty
+Conglomerate
+Construction
+Consulting
+Defense
+Education
+Email Provider
+Email Security
+Entertainment
+Event Planning
+Finance
+Food
+Government
+Government Media
+Healthcare
+ISP
+IaaS
+Industrial
+Legal
+Logistics
+MSP
+MSSP
+Manufacturing
+Marketing
+News
+Nonprofit
+PaaS
+Photography
+Physical Security
+Print
+Publishing
+Real Estate
+Retail
+SaaS
+Science
+Search Engine
+Social Media
+Sports
+Staffing
+Technology
+Travel
+Web Host
--- a/parsedmarc/resources/maps/find_unknown_base_reverse_dns.py
+++ b/parsedmarc/resources/maps/find_unknown_base_reverse_dns.py
@@ -1,6 +1,5 @@
 #!/usr/bin/env python

-import logging
 import os
 import csv

@@ -14,62 +13,48 @@ def _main():

    csv_headers = ["source_name", "message_count"]

+    known_unknown_domains = []
+    psl_overrides = []
+    known_domains = []
    output_rows = []

-    logging.basicConfig()
-    logger = logging.getLogger(__name__)
-    logger.setLevel(logging.INFO)
+    def load_list(file_path, list_var):
+        if not os.path.exists(file_path):
+            print(f"Error: {file_path} does not exist")
+        print(f"Loading {file_path}")
+        with open(file_path) as f:
+            for line in f.readlines():
+                domain = line.lower().strip()
+                if domain in list_var:
+                    print(f"Error: {domain} is in {file_path} multiple times")
+                    exit(1)
+                elif domain != "":
+                    list_var.append(domain)

-    for p in [
-        input_csv_file_path,
-        base_reverse_dns_map_file_path,
-        known_unknown_list_file_path,
-        psl_overrides_file_path,
-    ]:
-        if not os.path.exists(p):
-            logger.error(f"{p} does not exist")
-            exit(1)
-    logger.info(f"Loading {known_unknown_list_file_path}")
-    known_unknown_domains = []
-    with open(known_unknown_list_file_path) as f:
-        for line in f.readlines():
-            domain = line.lower().strip()
-            if domain in known_unknown_domains:
-                logger.warning(
-                    f"{domain} is in {known_unknown_list_file_path} multiple times"
-                )
-            else:
-                known_unknown_domains.append(domain)
-    logger.info(f"Loading {psl_overrides_file_path}")
-    psl_overrides = []
-    with open(psl_overrides_file_path) as f:
-        for line in f.readlines():
-            domain = line.lower().strip()
-            if domain in psl_overrides:
-                logger.warning(
-                    f"{domain} is in {psl_overrides_file_path} multiple times"
-                )
-            else:
-                psl_overrides.append(domain)
-    logger.info(f"Loading {base_reverse_dns_map_file_path}")
-    known_domains = []
+    load_list(known_unknown_list_file_path, known_unknown_domains)
+    load_list(psl_overrides_file_path, psl_overrides)
+    if not os.path.exists(base_reverse_dns_map_file_path):
+        print(f"Error: {base_reverse_dns_map_file_path} does not exist")
+    print(f"Loading {base_reverse_dns_map_file_path}")
    with open(base_reverse_dns_map_file_path) as f:
        for row in csv.DictReader(f):
            domain = row["base_reverse_dns"].lower().strip()
            if domain in known_domains:
-                logger.warning(
-                    f"{domain} is in {base_reverse_dns_map_file_path} multiple times"
+                print(
+                    f"Error: {domain} is in {base_reverse_dns_map_file_path} multiple times"
                )
+                exit()
            else:
                known_domains.append(domain)
            if domain in known_unknown_domains and known_domains:
-                pass
-                logger.warning(
-                    f"{domain} is in {known_unknown_list_file_path} and \
+                print(
+                    f"Error:{domain} is in {known_unknown_list_file_path} and \
                        {base_reverse_dns_map_file_path}"
                )
-
-    logger.info(f"Checking domains against {base_reverse_dns_map_file_path}")
+                exit(1)
+    if not os.path.exists(input_csv_file_path):
+        print(f"Error: {base_reverse_dns_map_file_path} does not exist")
+        exit(1)
    with open(input_csv_file_path) as f:
        for row in csv.DictReader(f):
            domain = row["source_name"].lower().strip()
@@ -77,12 +62,12 @@ def _main():
                continue
            for psl_domain in psl_overrides:
                if domain.endswith(psl_domain):
-                    domain = psl_domain
+                    domain = psl_domain.strip(".").strip("-")
                    break
            if domain not in known_domains and domain not in known_unknown_domains:
-                logger.info(f"New unknown domain found: {domain}")
+                print(f"New unknown domain found: {domain}")
                output_rows.append(row)
-    logger.info(f"Writing {output_csv_file_path}")
+    print(f"Writing {output_csv_file_path}")
    with open(output_csv_file_path, "w") as f:
        writer = csv.DictWriter(f, fieldnames=csv_headers)
        writer.writeheader()
--- a/parsedmarc/resources/maps/known_unknown_base_reverse_dns.txt
+++ b/parsedmarc/resources/maps/known_unknown_base_reverse_dns.txt
@@ -1,10 +1,15 @@
-185.in-addr.arpa
-190.in-addr.arpa
-200.in-addr.arpa
+1jli.site
+26.107
+444qcuhilla.com
+4xr1.com
 9services.com
 a7e.ru
 a94434500-blog.com
+aams8.jp
 abv-10.top
+acemail.co.in
+activaicon.com
+adcritic.net
 adlucrumnewsletter.com
 admin.corpivensa.gob.ve
 advantageiq.com
@@ -15,6 +20,11 @@ aghories.com
 ai270.net
 albagroup-eg.com
 alchemy.net
+alohabeachcamp.net
+alsiscad.com
+aluminumpipetubing.com
+americanstorageca.com
+amplusserver.info
 anchorfundhub.com
 anglishment.com
 anteldata.net.uy
@@ -26,98 +36,186 @@ aosau.net
 arandomserver.com
 aransk.ru
 ardcs.cn
+armninl.met
 as29550.net
+asahachimaru.com
+aserv.co.za
 asmecam.it
+ateky.net.br
 aurelienvos.com
 automatech.lat
 avistaadvantage.com
 b8sales.com
+bahjs.com
+baliaura.com
+banaras.co
 bearandbullmarketnews.com
 bestinvestingtime.com
+bhjui.com
 biocorp.com
+biosophy.net
 bitter-echo.com
+bizhostingservices.com
 blguss.com
 bluenet.ch
 bluhosting.com
+bnasg.com
 bodiax.pp.ua
 bost-law.com
 brainity.com
+brazalnde.net
+brellatransplc.shop
 brnonet.cz
+broadwaycover.com
 brushinglegal.de
 brw.net
+btes.tv
 budgeteasehub.com
 buoytoys.com
+buyjapanese.jp
+c53dw7m24rj.com
+cahtelrandom.org
+casadelmarsamara.com
 cashflowmasterypro.com
 cavabeen.com
 cbti.net
+centralmalaysia.com
+chauffeurplan.co.uk
 checkpox.fun
 chegouseuvlache.org
+chinaxingyu.xyz
 christus.mx
+churchills.market
+ci-xyz.fit
+cisumrecords.com
 ckaik.cn
+clcktoact.com
+cli-eurosignal.cz
+cloud-admin.it
 cloud-edm.com
-cloudaccess.net
 cloudflare-email.org
 cloudhosting.rs
 cloudlogin.co
+cloudplatformpro.com
 cnode.io
+cntcloud.com
 code-it.net
+codefriend.top
 colombiaceropapel.org
 commerceinsurance.com
+comsharempc.com
+conexiona.com
 coolblaze.com
 coowo.com
 corpemail.net
 cp2-myorderbox.com
 cps.com.ar
+crnagora.net
+cross-d-bar-troutranch.com
 ctla.co.kr
+cumbalikonakhotel.com
+currencyexconverter.com
+daakbabu.com
+daikinmae.com
+dairyvalley.com.my
 dastans.ru
 datahost36.de
+ddii.network
+deep-sek.shop
+deetownsounds.com
 descarca-counter-strike.net
 detrot.xyz
+dettlaffinc.com
+dextoolse.net
+digestivedaily.com
 digi.net.my
 dinofelis.cn
 diwkyncbi.top
 dkginternet.com
+dnexpress.info
 dns-oid.com
+dnsindia.net
+domainserver.ne.jp
 domconfig.com
 doorsrv.com
 dreampox.fun
 dreamtechmedia.com
 ds.network
+dss-group.net
 dvj.theworkpc.com
 dwlcka.com
+dynamic-wiretel.in
 dyntcorp.com
+easternkingspei.com
 economiceagles.com
 egosimail.com
+eliotporterphotos.us
 emailgids.net
 emailperegrine.com
+entendercopilot.com
 entretothom.net
+epaycontrol.com
+epicinvestmentsreview.co
+epicinvestmentsreview.com
+epik.com
 epsilon-group.com
 erestaff.com
+euro-trade-gmbh.com
 example.com
 exposervers.com-new
+extendcp.co.uk
 eyecandyhosting.xyz
+fastwebnet.it
+fd9ing7wfn.com
+feipnghardware.com
 fetscorp.shop
+fewo-usedom.net
 fin-crime.com
 financeaimpoint.com
 financeupward.com
+firmflat.com
 flex-video.bnr.la
+flourishfusionlife.com
 formicidaehunt.net
 fosterheap.com
+fredi.shop
 frontiernet.net
+ftifb7tk3c.com
+gamersprotectionvpn.online
 gendns.com
+getgreencardsfast.com
 getthatroi.com
+gibbshosting.com
 gigidea.net
 giize.com
 ginous.eu.com
+gis.net
 gist-th.com
+globalglennpartners.com
+goldsboroughplace.com
 gophermedia.com
 gqlists.us.com
 gratzl.de
 greatestworldnews.com
 greennutritioncare.com
+gsbb.com
+gumbolimbo.net
 h-serv.co.uk
+haedefpartners.com
+halcyon-aboveboard.com
+hanzubon.org
+healthfuljourneyjoy.com
 hgnbroken.us.com
+highwey-diesel.com
+hirofactory.com
+hjd.asso.fr
+hongchenggco.pro
+hongkongtaxi.co
+hopsinthehanger.com
+hosted-by-worldstream.net
+hostelsucre.com
 hosting1337.com
+hostinghane.com
 hostinglotus.cloud
 hostingmichigan.com
 hostiran.name
@@ -125,8 +223,11 @@ hostmnl.com
 hostname.localhost
 hostnetwork.com
 hosts.net.nz
+hostserv.eu
 hostwhitelabel.com
 hpms1.jp
+hunariojmk.net
+hunriokinmuim.net
 hypericine.com
 i-mecca.net
 iaasdns.com
@@ -134,36 +235,88 @@ iam.net.ma
 iconmarketingguy.com
 idcfcloud.net
 idealconcept.live
+igmohji.com
 igppevents.org.uk
+ihglobaldns.com
+ilmessicano.com
 imjtmn.cn
 immenzaces.com
+in-addr-arpa
+in-addr.arpa
+indsalelimited.com
+indulgent-holistic.com
+industechint.org
 inshaaegypt.com
+intal.uz
+interfarma.kz
+intocpanel.com
 ip-147-135-108.us
 ip-178-33-109.eu
 ip-ptr.tech
+iswhatpercent.com
 itsidc.com
 itwebs.com
+iuon.net
 ivol.co
 jalanet.co.id
 jimishare.com
+jlccptt.net.cn
+jlenterprises.co.uk
+jmontalto.com
+joyomokei.com
 jumanra.org
+justlongshirts.com
 kahlaa.com
+kaw.theworkpc.com
 kbronet.com.tw
 kdnursing.org
+kielnet.net
 kihy.theworkpc.com
+kingschurchwirral.org
 kitchenaildbd.com
+klaomi.shop
+knkconsult.net
+kohshikai.com
+krhfund.org
+krillaglass.com
+lancorhomes.com
+landpedia.org
+lanzatuseo.es
 layerdns.cloud
+learninglinked.com
 legenditds.com
+levertechcentre.com
+lhost.no
+lideri.net.br
 lighthouse-media.com
+lightpath.net
+limogesporcelainboxes.com
+lindsaywalt.net
+linuxsunucum.com
 listertermoformadoa.com
 llsend.com
+local.net
 lohkal.com
+londionrtim.net
 lonestarmm.net
+longmarquis.com
 longwoodmgmt.com
+lse.kz
+lunvoy.com
+luxarpro.ru
 lwl-puehringer.at
 lynx.net.lb
+lyse.net
+m-sender.com.ua
+maggiolicloud.it
 magnetmail.net
+magnumgo.uz
+maia11.com
 mail-fire.com
+mailsentinel.net
+mailset.cn
+malardino.net
+managed-vps.net
 manhattanbulletpoint.com
 manpowerservices.com
 marketmysterycode.com
@@ -173,8 +326,23 @@ matroguel.cam
 maximpactipo.com
 mechanicalwalk.store
 mediavobis.com
+meqlobal.com
+mgts.by
+migrans.net
+miixta.com
+milleniumsrv.com
+mindworksunlimited.com
+mirth-gale.com
 misorpresa.com
+mitomobile.com
+mitsubachi-kibako.net
+mjinn.com
+mkegs.shop
+mobius.fr
+model-ac.ink
 moderntradingnews.com
+monnaiegroup.com
+monopolizeright.com
 moonjaws.com
 morningnewscatcher.com
 motion4ever.net
@@ -182,122 +350,245 @@ mschosting.com
 msdp1.com
 mspnet.pro
 mts-nn.ru
+multifamilydesign.com
 mxserver.ro
 mxthunder.net
+my-ihor.ru
+mycloudmailbox.com
+myfriendforum.com
 myrewards.net
 mysagestore.com
 mysecurewebserver.com
+myshanet.net
 myvps.jp
+mywedsite.net
+mywic.eu
 name.tools
 nanshenqfurniture.com
 nask.pl
+navertise.net
+ncbb.kz
 ncport.ru
 ncsdi.ws
 nebdig.com
 neovet-base.ru
 netbri.com
+netcentertelecom.net.br
+neti.ee
 netkl.org
 newinvestingguide.com
 newwallstreetcode.com
 ngvcv.cn
 nic.name
 nidix.net
+nieuwedagnetwerk.net
 nlscanme.com
 nmeuh.cn
 noisndametal.com
+nucleusemail.com
+nutriboostlife.com
 nwo.giize.com
+nwwhalewatchers.org
+ny.adsl
+nyt1.com
 offerslatedeals.com
 office365.us
 ogicom.net
+olivettilexikon.co.uk
 omegabrasil.inf.br
 onnet21.com
+onumubunumu.com
 oppt-ac.fit
 orbitel.net.co
+orfsurface.com
+orientalspot.com
+outsidences.com
 ovaltinalization.co
 overta.ru
+ox28vgrurc.com
+pamulang.net
 panaltyspot.space
+panolacountysheriffms.com
 passionatesmiles.com
+paulinelam.com
+pdi-corp.com
+peloquinbeck.com
 perimetercenter.net
 permanentscreen.com
+permasteellisagroup.com
+perumkijhyu.net
+pesnia.com.ua
+ph8ltwdi12o.com
+pharmada.com.de
 phdns3.es
+pigelixval1.com
+pipefittingsindia.com
 planethoster.net
+playamedia.io
 plesk.page
 pmnhost.net
+pokiloandhu.net
 pokupki5.ru
+polandi.net
 popiup.com
 ports.net
+posolstvostilya.com
+potia.net
 prima.com.ar
 prima.net.ar
+profsol.co.uk
 prohealthmotion.com
+promooffermarket.site
 proudserver.com
+proxado.com
 psnm.ru
 pvcwindowsprices.live
 qontenciplc.autos
+quakeclick.com
+quasarstate.store
 quatthonggiotico.com
+qxyxab44njd.com
+radianthealthrenaissance.com
 rapidns.com
 raxa.host
+reberte.com
+reethvikintl.com
+regruhosting.ru
 reliablepanel.com
 rgb365.eu
 riddlecamera.net
+riddletrends.com
+roccopugliese.com
+runnin-rebels.com
+rupar.puglia.it
 rwdhosting.ca
 s500host.com
+sageevents.co.ke
 sahacker-2020.com
 samsales.site
+sante-lorraine.fr
 saransk.ru
 satirogluet.com
+scioncontacts.com
+sdcc.my
 seaspraymta3.net
 secorp.mx
 securen.net
 securerelay.in
 securev.net
+seductiveeyes.com
+seizethedayconsulting.com
+serroplast.shop
+server290.com
+server342.com
+server3559.cc
 servershost.biz
+sfek.kz
+sgnetway.net
+shopfox.ca
 silvestrejaguar.sbs
 silvestreonca.sbs
+simplediagnostics.org
 siriuscloud.jp
 sisglobalresearch.com
+sixpacklink.net
+sjestyle.com
 smallvillages.com
 smartape-vps.com
 solusoftware.com
+sourcedns.com
+southcoastwebhosting12.com
+specialtvvs.com
 spiritualtechnologies.io
 sprout.org
+srv.cat
 stableserver.net
+statlerfa.co.uk
+stock-smtp.top
 stockepictigers.com
 stockexchangejournal.com
+subterranean-concave.com
 suksangroup.com
+swissbluetopaz.com
+switer.shop
 sysop4.com
 system.eu.com
 szhongbing.com
 t-jon.com
+tacaindo.net
+tacom.tj
+tankertelz.co
+tataidc.com
+teamveiw.com
 tecnoxia.net
 tel-xyz.fit
 tenkids.net
 terminavalley.com
 thaicloudsolutions.com
+thaikinghost.com
 thaimonster.com
+thegermainetruth.net
+thehandmaderose.com
 thepushcase.com
+ticdns.com
+tigo.bo
+toledofibra.net.br
+topdns.com
 totaal.net
+totalplay.net
+tqh.ro
 traderlearningcenter.com
+tradeukraine.site
+traveleza.com
+trwww.com
+tsuzakij.com
 tullostrucking.com
+turbinetrends.com
+twincitiesdistinctivehomes.com
+tylerfordonline.com
+uiyum.com
 ultragate.com
+uneedacollie.com
+unified.services
 unite.services
 urawasl.com
 us.servername.us
+vagebond.net
 varvia.de
+vbcploo.com
+vdc.vn
 vendimetry.com
 vibrantwellnesscorp.com
+virtualine.org
+visit.docotor
 viviotech.us
 vlflgl.com
 volganet.ru
+vrns.net
+vulterdi.edu
+vvondertex.com
 wallstreetsgossip.com
+wamego.net
+wanekoohost.com
 wealthexpertisepro.com
 web-login.eu
 weblinkinternational.com
 webnox.io
+websale.net
 welllivinghive.com
+westparkcom.com
+wetransfer-eu.com
+wheelch.me
+whoflew.com
+whpservers.com
 wisdomhard.com
 wisewealthcircle.com
+wisvis.com
+wodeniowa.com
+wordpresshosting.xyz
 wsiph2.com
 xnt.mx
+xodiax.com
 xpnuf.cn
 xsfati.us.com
 xspmail.jp
@@ -305,5 +596,6 @@ yourciviccompass.com
 yourinvestworkbook.com
 yoursitesecure.net
 zerowebhosting.net
+zmml.uk
 znlc.jp
 ztomy.com
--- a/parsedmarc/resources/maps/psl_overrides.txt
+++ b/parsedmarc/resources/maps/psl_overrides.txt
@@ -1,6 +1,23 @@
-akura.ne.jp
-amazonaws.com
-cloudaccess.net
-h-serv.co.uk
-linode.com
-plesk.page
+-applefibernet.com
+-c3.net.pl
+-celsiainternet.com
+-clientes-izzi.mx
+-clientes-zap-izzi.mx
+-imnet.com.br
+-mcnbd.com
+-smile.com.bd
+-tataidc.co.in
+-veloxfiber.com.br
+-wconect.com.br
+.amazonaws.com
+.cloudaccess.net
+.ddnsgeek.com
+.fastvps-server.com
+.in-addr-arpa
+.in-addr.arpa
+.kasserver.com
+.kinghost.net
+.linode.com
+.linodeusercontent.com
+.na4u.ru
+.sakura.ne.jp
--- a/parsedmarc/resources/maps/sortlists.py
+++ b/parsedmarc/resources/maps/sortlists.py
@@ -0,0 +1,184 @@
+#!/usr/bin/env python3
+
+from __future__ import annotations
+
+import os
+import csv
+from pathlib import Path
+from typing import Mapping, Iterable, Optional, Collection, Union, List, Dict
+
+
+class CSVValidationError(Exception):
+    def __init__(self, errors: list[str]):
+        super().__init__("\n".join(errors))
+        self.errors = errors
+
+
+def sort_csv(
+    filepath: Union[str, Path],
+    field: str,
+    *,
+    sort_field_value_must_be_unique: bool = True,
+    strip_whitespace: bool = True,
+    fields_to_lowercase: Optional[Iterable[str]] = None,
+    case_insensitive_sort: bool = False,
+    required_fields: Optional[Iterable[str]] = None,
+    allowed_values: Optional[Mapping[str, Collection[str]]] = None,
+) -> List[Dict[str, str]]:
+    """
+    Read a CSV, optionally normalize rows (strip whitespace, lowercase certain fields),
+    validate field values, and write the sorted CSV back to the same path.
+
+    - filepath: Path to the CSV to sort.
+    - field: The field name to sort by.
+    - fields_to_lowercase: Permanently lowercases these field(s) in the data.
+    - strip_whitespace: Remove all whitespace at the beginning and of field values.
+    - case_insensitive_sort: Ignore case when sorting without changing values.
+    - required_fields: A list of fields that must have data in all rows.
+    - allowed_values: A mapping of allowed values for fields.
+    """
+    path = Path(filepath)
+    required_fields = set(required_fields or [])
+    lower_set = set(fields_to_lowercase or [])
+    allowed_sets = {k: set(v) for k, v in (allowed_values or {}).items()}
+    if sort_field_value_must_be_unique:
+        seen_sort_field_values = []
+
+    with path.open("r", newline="") as infile:
+        reader = csv.DictReader(infile)
+        fieldnames = reader.fieldnames or []
+        if field not in fieldnames:
+            raise CSVValidationError([f"Missing sort column: {field!r}"])
+        missing_headers = required_fields - set(fieldnames)
+        if missing_headers:
+            raise CSVValidationError(
+                [f"Missing required header(s): {sorted(missing_headers)}"]
+            )
+        rows = list(reader)
+
+    def normalize_row(row: Dict[str, str]) -> None:
+        if strip_whitespace:
+            for k, v in row.items():
+                if isinstance(v, str):
+                    row[k] = v.strip()
+        for fld in lower_set:
+            if fld in row and isinstance(row[fld], str):
+                row[fld] = row[fld].lower()
+
+    def validate_row(
+        row: Dict[str, str], sort_field: str, line_no: int, errors: list[str]
+    ) -> None:
+        if sort_field_value_must_be_unique:
+            if row[sort_field] in seen_sort_field_values:
+                errors.append(f"Line {line_no}: Duplicate row for '{row[sort_field]}'")
+            else:
+                seen_sort_field_values.append(row[sort_field])
+        for rf in required_fields:
+            val = row.get(rf)
+            if val is None or val == "":
+                errors.append(
+                    f"Line {line_no}: Missing value for required field '{rf}'"
+                )
+        for field, allowed_values in allowed_sets.items():
+            if field in row:
+                val = row[field]
+                if val not in allowed_values:
+                    errors.append(
+                        f"Line {line_no}: '{val}' is not an allowed value for '{field}' "
+                        f"(allowed: {sorted(allowed_values)})"
+                    )
+
+    errors: list[str] = []
+    for idx, row in enumerate(rows, start=2):  # header is line 1
+        normalize_row(row)
+        validate_row(row, field, idx, errors)
+
+    if errors:
+        raise CSVValidationError(errors)
+
+    def sort_key(r: Dict[str, str]):
+        v = r.get(field, "")
+        if isinstance(v, str) and case_insensitive_sort:
+            return v.casefold()
+        return v
+
+    rows.sort(key=sort_key)
+
+    with open(filepath, "w", newline="") as outfile:
+        writer = csv.DictWriter(outfile, fieldnames=fieldnames)
+        writer.writeheader()
+        writer.writerows(rows)
+
+
+def sort_list_file(
+    filepath: Union[str, Path],
+    *,
+    lowercase: bool = True,
+    strip: bool = True,
+    deduplicate: bool = True,
+    remove_blank_lines: bool = True,
+    ending_newline: bool = True,
+    newline: Optional[str] = "\n",
+):
+    """Read a list from a file, sort it, optionally strip and deduplicate the values,
+    then write that list back to the file.
+
+    - Filepath: The path to the file.
+    - lowercase: Lowercase all values prior to sorting.
+    - remove_blank_lines: Remove any plank lines.
+    - ending_newline: End the file with a newline, even if remove_blank_lines is true.
+    - newline: The newline character to use.
+    """
+    with open(filepath, mode="r", newline=newline) as infile:
+        lines = infile.readlines()
+        for i in range(len(lines)):
+            if lowercase:
+                lines[i] = lines[i].lower()
+            if strip:
+                lines[i] = lines[i].strip()
+        if deduplicate:
+            lines = list(set(lines))
+        if remove_blank_lines:
+            while "" in lines:
+                lines.remove("")
+        lines = sorted(lines)
+        if ending_newline:
+            if lines[-1] != "":
+                lines.append("")
+    with open(filepath, mode="w", newline=newline) as outfile:
+        outfile.write("\n".join(lines))
+
+
+def _main():
+    map_file = "base_reverse_dns_map.csv"
+    map_key = "base_reverse_dns"
+    list_files = ["known_unknown_base_reverse_dns.txt", "psl_overrides.txt"]
+    types_file = "base_reverse_dns_types.txt"
+
+    with open(types_file) as f:
+        types = f.readlines()
+        while "" in types:
+            types.remove("")
+
+    map_allowed_values = {"Type": types}
+
+    for list_file in list_files:
+        if not os.path.exists(list_file):
+            print(f"Error: {list_file} does not exist")
+            exit(1)
+        sort_list_file(list_file)
+    if not os.path.exists(types_file):
+        print(f"Error: {types_file} does not exist")
+        exit(1)
+    sort_list_file(types_file, lowercase=False)
+    if not os.path.exists(map_file):
+        print(f"Error: {map_file} does not exist")
+        exit(1)
+    try:
+        sort_csv(map_file, map_key, allowed_values=map_allowed_values)
+    except CSVValidationError as e:
+        print(f"{map_file} did not validate: {e}")
+
+
+if __name__ == "__main__":
+    _main()
--- a/parsedmarc/resources/maps/sortmaps.py
+++ b/parsedmarc/resources/maps/sortmaps.py
@@ -1,59 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import csv
-
-maps_dir = os.path.join(".")
-map_files = ["base_reverse_dns_map.csv"]
-list_files = ["known_unknown_base_reverse_dns.txt", "psl_overrides.txt"]
-
-
-def sort_csv(filepath, column=0):
-    with open(filepath, mode="r", newline="") as infile:
-        reader = csv.reader(infile)
-        header = next(reader)
-        sorted_rows = sorted(reader, key=lambda row: row[column])
-        existing_values = []
-        for row in sorted_rows:
-            if row[column] in existing_values:
-                print(f"Warning: {row[column]} is in {filepath} multiple times")
-
-    with open(filepath, mode="w", newline="\n") as outfile:
-        writer = csv.writer(outfile)
-        writer.writerow(header)
-        writer.writerows(sorted_rows)
-
-
-def sort_list_file(
-    filepath,
-    lowercase=True,
-    strip=True,
-    deduplicate=True,
-    remove_blank_lines=True,
-    ending_newline=True,
-    newline="\n",
-):
-    with open(filepath, mode="r", newline=newline) as infile:
-        lines = infile.readlines()
-        for i in range(len(lines)):
-            if lowercase:
-                lines[i] = lines[i].lower()
-            if strip:
-                lines[i] = lines[i].strip()
-        if deduplicate:
-            lines = list(set(lines))
-        if remove_blank_lines:
-            while "" in lines:
-                lines.remove("")
-        lines = sorted(lines)
-        if ending_newline:
-            if lines[-1] != "":
-                lines.append("")
-    with open(filepath, mode="w", newline=newline) as outfile:
-        outfile.write("\n".join(lines))
-
-
-for csv_file in map_files:
-    sort_csv(os.path.join(maps_dir, csv_file))
-for list_file in list_files:
-    sort_list_file(os.path.join(maps_dir, list_file))
--- a/parsedmarc/s3.py
+++ b/parsedmarc/s3.py
@@ -1,6 +1,10 @@
 # -*- coding: utf-8 -*-

+from __future__ import annotations
+
 import json
+from typing import Any
+
 import boto3

 from parsedmarc.log import logger
@@ -8,16 +12,16 @@ from parsedmarc.utils import human_timestamp_to_datetime


 class S3Client(object):
-    """A client for a Amazon S3"""
+    """A client for interacting with Amazon S3"""

    def __init__(
        self,
-        bucket_name,
-        bucket_path,
-        region_name,
-        endpoint_url,
-        access_key_id,
-        secret_access_key,
+        bucket_name: str,
+        bucket_path: str,
+        region_name: str,
+        endpoint_url: str,
+        access_key_id: str,
+        secret_access_key: str,
    ):
        """
        Initializes the S3Client
@@ -47,18 +51,18 @@ class S3Client(object):
            aws_access_key_id=access_key_id,
            aws_secret_access_key=secret_access_key,
        )
-        self.bucket = self.s3.Bucket(self.bucket_name)
+        self.bucket = self.s3.Bucket(self.bucket_name)  # type: ignore

-    def save_aggregate_report_to_s3(self, report):
+    def save_aggregate_report_to_s3(self, report: dict[str, Any]):
        self.save_report_to_s3(report, "aggregate")

-    def save_forensic_report_to_s3(self, report):
+    def save_forensic_report_to_s3(self, report: dict[str, Any]):
        self.save_report_to_s3(report, "forensic")

-    def save_smtp_tls_report_to_s3(self, report):
+    def save_smtp_tls_report_to_s3(self, report: dict[str, Any]):
        self.save_report_to_s3(report, "smtp_tls")

-    def save_report_to_s3(self, report, report_type):
+    def save_report_to_s3(self, report: dict[str, Any], report_type: str):
        if report_type == "smtp_tls":
            report_date = report["begin_date"]
            report_id = report["report_id"]
--- a/parsedmarc/splunk.py
+++ b/parsedmarc/splunk.py
@@ -1,9 +1,14 @@
-from urllib.parse import urlparse
-import socket
-import json
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
+import json
+import socket
+from typing import Any, Union
+from urllib.parse import urlparse

-import urllib3
 import requests
+import urllib3

 from parsedmarc.constants import USER_AGENT
 from parsedmarc.log import logger
@@ -23,7 +28,13 @@ class HECClient(object):
    # http://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector

    def __init__(
-        self, url, access_token, index, source="parsedmarc", verify=True, timeout=60
+        self,
+        url: str,
+        access_token: str,
+        index: str,
+        source: str = "parsedmarc",
+        verify=True,
+        timeout=60,
    ):
        """
        Initializes the HECClient
@@ -37,9 +48,9 @@ class HECClient(object):
            timeout (float): Number of seconds to wait for the server to send
                data before giving up
        """
-        url = urlparse(url)
+        parsed_url = urlparse(url)
        self.url = "{0}://{1}/services/collector/event/1.0".format(
-            url.scheme, url.netloc
+            parsed_url.scheme, parsed_url.netloc
        )
        self.access_token = access_token.lstrip("Splunk ")
        self.index = index
@@ -48,14 +59,19 @@ class HECClient(object):
        self.session = requests.Session()
        self.timeout = timeout
        self.session.verify = verify
-        self._common_data = dict(host=self.host, source=self.source, index=self.index)
+        self._common_data: dict[str, Union[str, int, float, dict]] = dict(
+            host=self.host, source=self.source, index=self.index
+        )

        self.session.headers = {
            "User-Agent": USER_AGENT,
            "Authorization": "Splunk {0}".format(self.access_token),
        }

-    def save_aggregate_reports_to_splunk(self, aggregate_reports):
+    def save_aggregate_reports_to_splunk(
+        self,
+        aggregate_reports: Union[list[dict[str, Any]], dict[str, Any]],
+    ):
        """
        Saves aggregate DMARC reports to Splunk

@@ -75,9 +91,12 @@ class HECClient(object):
        json_str = ""
        for report in aggregate_reports:
            for record in report["records"]:
-                new_report = dict()
+                new_report: dict[str, Union[str, int, float, dict]] = dict()
                for metadata in report["report_metadata"]:
                    new_report[metadata] = report["report_metadata"][metadata]
+                new_report["interval_begin"] = record["interval_begin"]
+                new_report["interval_end"] = record["interval_end"]
+                new_report["normalized_timespan"] = record["normalized_timespan"]
                new_report["published_policy"] = report["policy_published"]
                new_report["source_ip_address"] = record["source"]["ip_address"]
                new_report["source_country"] = record["source"]["country"]
@@ -98,7 +117,9 @@ class HECClient(object):
                    new_report["spf_results"] = record["auth_results"]["spf"]

                data["sourcetype"] = "dmarc:aggregate"
-                timestamp = human_timestamp_to_unix_timestamp(new_report["begin_date"])
+                timestamp = human_timestamp_to_unix_timestamp(
+                    new_report["interval_begin"]
+                )
                data["time"] = timestamp
                data["event"] = new_report.copy()
                json_str += "{0}\n".format(json.dumps(data))
@@ -113,7 +134,10 @@ class HECClient(object):
        if response["code"] != 0:
            raise SplunkError(response["text"])

-    def save_forensic_reports_to_splunk(self, forensic_reports):
+    def save_forensic_reports_to_splunk(
+        self,
+        forensic_reports: Union[list[dict[str, Any]], dict[str, Any]],
+    ):
        """
        Saves forensic DMARC reports to Splunk

@@ -147,7 +171,9 @@ class HECClient(object):
        if response["code"] != 0:
            raise SplunkError(response["text"])

-    def save_smtp_tls_reports_to_splunk(self, reports):
+    def save_smtp_tls_reports_to_splunk(
+        self, reports: Union[list[dict[str, Any]], dict[str, Any]]
+    ):
        """
        Saves aggregate DMARC reports to Splunk

--- a/parsedmarc/syslog.py
+++ b/parsedmarc/syslog.py
@@ -1,8 +1,12 @@
 # -*- coding: utf-8 -*-

+
+from __future__ import annotations
+
+import json
 import logging
 import logging.handlers
-import json
+from typing import Any

 from parsedmarc import (
    parsed_aggregate_reports_to_csv_rows,
@@ -14,7 +18,7 @@ from parsedmarc import (
 class SyslogClient(object):
    """A client for Syslog"""

-    def __init__(self, server_name, server_port):
+    def __init__(self, server_name: str, server_port: int):
        """
        Initializes the SyslogClient
        Args:
@@ -28,17 +32,17 @@ class SyslogClient(object):
        log_handler = logging.handlers.SysLogHandler(address=(server_name, server_port))
        self.logger.addHandler(log_handler)

-    def save_aggregate_report_to_syslog(self, aggregate_reports):
+    def save_aggregate_report_to_syslog(self, aggregate_reports: list[dict[str, Any]]):
        rows = parsed_aggregate_reports_to_csv_rows(aggregate_reports)
        for row in rows:
            self.logger.info(json.dumps(row))

-    def save_forensic_report_to_syslog(self, forensic_reports):
+    def save_forensic_report_to_syslog(self, forensic_reports: list[dict[str, Any]]):
        rows = parsed_forensic_reports_to_csv_rows(forensic_reports)
        for row in rows:
            self.logger.info(json.dumps(row))

-    def save_smtp_tls_report_to_syslog(self, smtp_tls_reports):
+    def save_smtp_tls_report_to_syslog(self, smtp_tls_reports: list[dict[str, Any]]):
        rows = parsed_smtp_tls_reports_to_csv_rows(smtp_tls_reports)
        for row in rows:
            self.logger.info(json.dumps(row))
--- a/parsedmarc/types.py
+++ b/parsedmarc/types.py
@@ -0,0 +1,220 @@
+from __future__ import annotations
+
+from typing import Any, Dict, List, Literal, Optional, TypedDict, Union
+
+# NOTE: This module is intentionally Python 3.9 compatible.
+# - No PEP 604 unions (A | B)
+# - No typing.NotRequired / Required (3.11+) to avoid an extra dependency.
+#   For optional keys, use total=False TypedDicts.
+
+
+ReportType = Literal["aggregate", "forensic", "smtp_tls"]
+
+
+class AggregateReportMetadata(TypedDict):
+    org_name: str
+    org_email: str
+    org_extra_contact_info: Optional[str]
+    report_id: str
+    begin_date: str
+    end_date: str
+    timespan_requires_normalization: bool
+    original_timespan_seconds: int
+    errors: List[str]
+
+
+class AggregatePolicyPublished(TypedDict):
+    domain: str
+    adkim: str
+    aspf: str
+    p: str
+    sp: str
+    pct: str
+    fo: str
+
+
+class IPSourceInfo(TypedDict):
+    ip_address: str
+    country: Optional[str]
+    reverse_dns: Optional[str]
+    base_domain: Optional[str]
+    name: Optional[str]
+    type: Optional[str]
+
+
+class AggregateAlignment(TypedDict):
+    spf: bool
+    dkim: bool
+    dmarc: bool
+
+
+class AggregateIdentifiers(TypedDict):
+    header_from: str
+    envelope_from: Optional[str]
+    envelope_to: Optional[str]
+
+
+class AggregatePolicyOverrideReason(TypedDict):
+    type: Optional[str]
+    comment: Optional[str]
+
+
+class AggregateAuthResultDKIM(TypedDict):
+    domain: str
+    result: str
+    selector: str
+
+
+class AggregateAuthResultSPF(TypedDict):
+    domain: str
+    result: str
+    scope: str
+
+
+class AggregateAuthResults(TypedDict):
+    dkim: List[AggregateAuthResultDKIM]
+    spf: List[AggregateAuthResultSPF]
+
+
+class AggregatePolicyEvaluated(TypedDict):
+    disposition: str
+    dkim: str
+    spf: str
+    policy_override_reasons: List[AggregatePolicyOverrideReason]
+
+
+class AggregateRecord(TypedDict):
+    interval_begin: str
+    interval_end: str
+    source: IPSourceInfo
+    count: int
+    alignment: AggregateAlignment
+    policy_evaluated: AggregatePolicyEvaluated
+    disposition: str
+    identifiers: AggregateIdentifiers
+    auth_results: AggregateAuthResults
+
+
+class AggregateReport(TypedDict):
+    xml_schema: str
+    report_metadata: AggregateReportMetadata
+    policy_published: AggregatePolicyPublished
+    records: List[AggregateRecord]
+
+
+class EmailAddress(TypedDict):
+    display_name: Optional[str]
+    address: str
+    local: Optional[str]
+    domain: Optional[str]
+
+
+class EmailAttachment(TypedDict, total=False):
+    filename: Optional[str]
+    mail_content_type: Optional[str]
+    sha256: Optional[str]
+
+
+ParsedEmail = TypedDict(
+    "ParsedEmail",
+    {
+        # This is a lightly-specified version of mailsuite/mailparser JSON.
+        # It focuses on the fields parsedmarc uses in forensic handling.
+        "headers": Dict[str, Any],
+        "subject": Optional[str],
+        "filename_safe_subject": Optional[str],
+        "date": Optional[str],
+        "from": EmailAddress,
+        "to": List[EmailAddress],
+        "cc": List[EmailAddress],
+        "bcc": List[EmailAddress],
+        "attachments": List[EmailAttachment],
+        "body": Optional[str],
+        "has_defects": bool,
+        "defects": Any,
+        "defects_categories": Any,
+    },
+    total=False,
+)
+
+
+class ForensicReport(TypedDict):
+    feedback_type: Optional[str]
+    user_agent: Optional[str]
+    version: Optional[str]
+    original_envelope_id: Optional[str]
+    original_mail_from: Optional[str]
+    original_rcpt_to: Optional[str]
+    arrival_date: str
+    arrival_date_utc: str
+    authentication_results: Optional[str]
+    delivery_result: Optional[str]
+    auth_failure: List[str]
+    authentication_mechanisms: List[str]
+    dkim_domain: Optional[str]
+    reported_domain: str
+    sample_headers_only: bool
+    source: IPSourceInfo
+    sample: str
+    parsed_sample: ParsedEmail
+
+
+class SMTPTLSFailureDetails(TypedDict):
+    result_type: str
+    failed_session_count: int
+
+
+class SMTPTLSFailureDetailsOptional(SMTPTLSFailureDetails, total=False):
+    sending_mta_ip: str
+    receiving_ip: str
+    receiving_mx_hostname: str
+    receiving_mx_helo: str
+    additional_info_uri: str
+    failure_reason_code: str
+    ip_address: str
+
+
+class SMTPTLSPolicySummary(TypedDict):
+    policy_domain: str
+    policy_type: str
+    successful_session_count: int
+    failed_session_count: int
+
+
+class SMTPTLSPolicy(SMTPTLSPolicySummary, total=False):
+    policy_strings: List[str]
+    mx_host_patterns: List[str]
+    failure_details: List[SMTPTLSFailureDetailsOptional]
+
+
+class SMTPTLSReport(TypedDict):
+    organization_name: str
+    begin_date: str
+    end_date: str
+    contact_info: Union[str, List[str]]
+    report_id: str
+    policies: List[SMTPTLSPolicy]
+
+
+class AggregateParsedReport(TypedDict):
+    report_type: Literal["aggregate"]
+    report: AggregateReport
+
+
+class ForensicParsedReport(TypedDict):
+    report_type: Literal["forensic"]
+    report: ForensicReport
+
+
+class SMTPTLSParsedReport(TypedDict):
+    report_type: Literal["smtp_tls"]
+    report: SMTPTLSReport
+
+
+ParsedReport = Union[AggregateParsedReport, ForensicParsedReport, SMTPTLSParsedReport]
+
+
+class ParsingResults(TypedDict):
+    aggregate_reports: List[AggregateReport]
+    forensic_reports: List[ForensicReport]
+    smtp_tls_reports: List[SMTPTLSReport]
--- a/parsedmarc/utils.py
+++ b/parsedmarc/utils.py
@@ -1,22 +1,26 @@
+# -*- coding: utf-8 -*-
+
 """Utility functions that might be useful for other projects"""

-import logging
-import os
-from datetime import datetime
-from datetime import timezone
-from datetime import timedelta
-from collections import OrderedDict
-import tempfile
-import subprocess
-import shutil
-import mailparser
-import json
-import hashlib
+from __future__ import annotations
+
 import base64
-import mailbox
-import re
 import csv
+import hashlib
 import io
+import json
+import logging
+import mailbox
+import os
+import re
+import shutil
+import subprocess
+import tempfile
+from datetime import datetime, timedelta, timezone
+from typing import Optional, TypedDict, Union, cast
+
+import mailparser
+from expiringdict import ExpiringDict

 try:
    from importlib.resources import files
@@ -25,25 +29,31 @@ except ImportError:
    from importlib.resources import files


-from dateutil.parser import parse as parse_date
-import dns.reversename
-import dns.resolver
 import dns.exception
+import dns.resolver
+import dns.reversename
 import geoip2.database
 import geoip2.errors
 import publicsuffixlist
 import requests
+from dateutil.parser import parse as parse_date

-from parsedmarc.log import logger
 import parsedmarc.resources.dbip
 import parsedmarc.resources.maps
 from parsedmarc.constants import USER_AGENT
+from parsedmarc.log import logger

 parenthesis_regex = re.compile(r"\s*\(.*\)\s*")

 null_file = open(os.devnull, "w")
 mailparser_logger = logging.getLogger("mailparser")
 mailparser_logger.setLevel(logging.CRITICAL)
+psl = publicsuffixlist.PublicSuffixList()
+psl_overrides_path = str(files(parsedmarc.resources.maps).joinpath("psl_overrides.txt"))
+with open(psl_overrides_path) as f:
+    psl_overrides = [line.rstrip() for line in f.readlines()]
+    while "" in psl_overrides:
+        psl_overrides.remove("")


 class EmailParserError(RuntimeError):
@@ -54,31 +64,49 @@ class DownloadError(RuntimeError):
    """Raised when an error occurs when downloading a file"""


-def decode_base64(data):
+class ReverseDNSService(TypedDict):
+    name: str
+    type: Optional[str]
+
+
+ReverseDNSMap = dict[str, ReverseDNSService]
+
+
+class IPAddressInfo(TypedDict):
+    ip_address: str
+    reverse_dns: Optional[str]
+    country: Optional[str]
+    base_domain: Optional[str]
+    name: Optional[str]
+    type: Optional[str]
+
+
+def decode_base64(data: str) -> bytes:
    """
    Decodes a base64 string, with padding being optional

    Args:
-        data: A base64 encoded string
+        data (str): A base64 encoded string

    Returns:
        bytes: The decoded bytes

    """
-    data = bytes(data, encoding="ascii")
-    missing_padding = len(data) % 4
+    data_bytes = bytes(data, encoding="ascii")
+    missing_padding = len(data_bytes) % 4
    if missing_padding != 0:
-        data += b"=" * (4 - missing_padding)
-    return base64.b64decode(data)
+        data_bytes += b"=" * (4 - missing_padding)
+    return base64.b64decode(data_bytes)


-def get_base_domain(domain):
+def get_base_domain(domain: str) -> Optional[str]:
    """
    Gets the base domain name for the given domain

    .. note::
        Results are based on a list of public domain suffixes at
-        https://publicsuffix.org/list/public_suffix_list.dat.
+        https://publicsuffix.org/list/public_suffix_list.dat and overrides included in
+        parsedmarc.resources.maps.psl_overrides.txt

    Args:
        domain (str): A domain or subdomain
@@ -87,11 +115,22 @@ def get_base_domain(domain):
        str: The base domain of the given domain

    """
-    psl = publicsuffixlist.PublicSuffixList()
-    return psl.privatesuffix(domain)
+    domain = domain.lower()
+    publicsuffix = psl.privatesuffix(domain)
+    for override in psl_overrides:
+        if domain.endswith(override):
+            return override.strip(".").strip("-")
+    return publicsuffix


-def query_dns(domain, record_type, cache=None, nameservers=None, timeout=2.0):
+def query_dns(
+    domain: str,
+    record_type: str,
+    *,
+    cache: Optional[ExpiringDict] = None,
+    nameservers: Optional[list[str]] = None,
+    timeout: float = 2.0,
+) -> list[str]:
    """
    Queries DNS

@@ -110,9 +149,9 @@ def query_dns(domain, record_type, cache=None, nameservers=None, timeout=2.0):
    record_type = record_type.upper()
    cache_key = "{0}_{1}".format(domain, record_type)
    if cache:
-        records = cache.get(cache_key, None)
-        if records:
-            return records
+        cached_records = cache.get(cache_key, None)
+        if isinstance(cached_records, list):
+            return cast(list[str], cached_records)

    resolver = dns.resolver.Resolver()
    timeout = float(timeout)
@@ -126,33 +165,25 @@ def query_dns(domain, record_type, cache=None, nameservers=None, timeout=2.0):
    resolver.nameservers = nameservers
    resolver.timeout = timeout
    resolver.lifetime = timeout
-    if record_type == "TXT":
-        resource_records = list(
-            map(
-                lambda r: r.strings,
-                resolver.resolve(domain, record_type, lifetime=timeout),
-            )
-        )
-        _resource_record = [
-            resource_record[0][:0].join(resource_record)
-            for resource_record in resource_records
-            if resource_record
-        ]
-        records = [r.decode() for r in _resource_record]
-    else:
-        records = list(
-            map(
-                lambda r: r.to_text().replace('"', "").rstrip("."),
-                resolver.resolve(domain, record_type, lifetime=timeout),
-            )
+    records = list(
+        map(
+            lambda r: r.to_text().replace('"', "").rstrip("."),
+            resolver.resolve(domain, record_type, lifetime=timeout),
        )
+    )
    if cache:
        cache[cache_key] = records

    return records


-def get_reverse_dns(ip_address, cache=None, nameservers=None, timeout=2.0):
+def get_reverse_dns(
+    ip_address,
+    *,
+    cache: Optional[ExpiringDict] = None,
+    nameservers: Optional[list[str]] = None,
+    timeout: float = 2.0,
+) -> Optional[str]:
    """
    Resolves an IP address to a hostname using a reverse DNS query

@@ -170,7 +201,7 @@ def get_reverse_dns(ip_address, cache=None, nameservers=None, timeout=2.0):
    try:
        address = dns.reversename.from_address(ip_address)
        hostname = query_dns(
-            address, "PTR", cache=cache, nameservers=nameservers, timeout=timeout
+            str(address), "PTR", cache=cache, nameservers=nameservers, timeout=timeout
        )[0]

    except dns.exception.DNSException as e:
@@ -180,7 +211,7 @@ def get_reverse_dns(ip_address, cache=None, nameservers=None, timeout=2.0):
    return hostname


-def timestamp_to_datetime(timestamp):
+def timestamp_to_datetime(timestamp: int) -> datetime:
    """
    Converts a UNIX/DMARC timestamp to a Python ``datetime`` object

@@ -193,7 +224,7 @@ def timestamp_to_datetime(timestamp):
    return datetime.fromtimestamp(int(timestamp))


-def timestamp_to_human(timestamp):
+def timestamp_to_human(timestamp: int) -> str:
    """
    Converts a UNIX/DMARC timestamp to a human-readable string

@@ -206,7 +237,9 @@ def timestamp_to_human(timestamp):
    return timestamp_to_datetime(timestamp).strftime("%Y-%m-%d %H:%M:%S")


-def human_timestamp_to_datetime(human_timestamp, to_utc=False):
+def human_timestamp_to_datetime(
+    human_timestamp: str, *, to_utc: bool = False
+) -> datetime:
    """
    Converts a human-readable timestamp into a Python ``datetime`` object

@@ -225,7 +258,7 @@ def human_timestamp_to_datetime(human_timestamp, to_utc=False):
    return dt.astimezone(timezone.utc) if to_utc else dt


-def human_timestamp_to_unix_timestamp(human_timestamp):
+def human_timestamp_to_unix_timestamp(human_timestamp: str) -> int:
    """
    Converts a human-readable timestamp into a UNIX timestamp

@@ -236,10 +269,12 @@ def human_timestamp_to_unix_timestamp(human_timestamp):
        float: The converted timestamp
    """
    human_timestamp = human_timestamp.replace("T", " ")
-    return human_timestamp_to_datetime(human_timestamp).timestamp()
+    return int(human_timestamp_to_datetime(human_timestamp).timestamp())


-def get_ip_address_country(ip_address, db_path=None):
+def get_ip_address_country(
+    ip_address: str, *, db_path: Optional[str] = None
+) -> Optional[str]:
    """
    Returns the ISO code for the country associated
    with the given IPv4 or IPv6 address
@@ -266,7 +301,7 @@ def get_ip_address_country(ip_address, db_path=None):
    ]

    if db_path is not None:
-        if os.path.isfile(db_path) is False:
+        if not os.path.isfile(db_path):
            db_path = None
            logger.warning(
                f"No file exists at {db_path}. Falling back to an "
@@ -303,12 +338,13 @@ def get_ip_address_country(ip_address, db_path=None):

 def get_service_from_reverse_dns_base_domain(
    base_domain,
-    always_use_local_file=False,
-    local_file_path=None,
-    url=None,
-    offline=False,
-    reverse_dns_map=None,
-):
+    *,
+    always_use_local_file: bool = False,
+    local_file_path: Optional[str] = None,
+    url: Optional[str] = None,
+    offline: bool = False,
+    reverse_dns_map: Optional[ReverseDNSMap] = None,
+) -> ReverseDNSService:
    """
    Returns the service name of a given base domain name from reverse DNS.

@@ -325,12 +361,6 @@ def get_service_from_reverse_dns_base_domain(
        the supplied reverse_dns_base_domain and the type will be None
    """

-    def load_csv(_csv_file):
-        reader = csv.DictReader(_csv_file)
-        for row in reader:
-            key = row["base_reverse_dns"].lower().strip()
-            reverse_dns_map[key] = dict(name=row["name"], type=row["type"])
-
    base_domain = base_domain.lower().strip()
    if url is None:
        url = (
@@ -338,11 +368,24 @@ def get_service_from_reverse_dns_base_domain(
            "/parsedmarc/master/parsedmarc/"
            "resources/maps/base_reverse_dns_map.csv"
        )
+    reverse_dns_map_value: ReverseDNSMap
    if reverse_dns_map is None:
-        reverse_dns_map = dict()
+        reverse_dns_map_value = {}
+    else:
+        reverse_dns_map_value = reverse_dns_map
+
+    def load_csv(_csv_file):
+        reader = csv.DictReader(_csv_file)
+        for row in reader:
+            key = row["base_reverse_dns"].lower().strip()
+            reverse_dns_map_value[key] = {
+                "name": row["name"],
+                "type": row["type"],
+            }
+
    csv_file = io.StringIO()

-    if not (offline or always_use_local_file) and len(reverse_dns_map) == 0:
+    if not (offline or always_use_local_file) and len(reverse_dns_map_value) == 0:
        try:
            logger.debug(f"Trying to fetch reverse DNS map from {url}...")
            headers = {"User-Agent": USER_AGENT}
@@ -359,7 +402,7 @@ def get_service_from_reverse_dns_base_domain(
            logging.debug("Response body:")
            logger.debug(csv_file.read())

-    if len(reverse_dns_map) == 0:
+    if len(reverse_dns_map_value) == 0:
        logger.info("Loading included reverse DNS map...")
        path = str(
            files(parsedmarc.resources.maps).joinpath("base_reverse_dns_map.csv")
@@ -368,26 +411,28 @@ def get_service_from_reverse_dns_base_domain(
            path = local_file_path
        with open(path) as csv_file:
            load_csv(csv_file)
+    service: ReverseDNSService
    try:
-        service = reverse_dns_map[base_domain]
+        service = reverse_dns_map_value[base_domain]
    except KeyError:
-        service = dict(name=base_domain, type=None)
+        service = {"name": base_domain, "type": None}

    return service


 def get_ip_address_info(
    ip_address,
-    ip_db_path=None,
-    reverse_dns_map_path=None,
-    always_use_local_files=False,
-    reverse_dns_map_url=None,
-    cache=None,
-    reverse_dns_map=None,
-    offline=False,
-    nameservers=None,
-    timeout=2.0,
-):
+    *,
+    ip_db_path: Optional[str] = None,
+    reverse_dns_map_path: Optional[str] = None,
+    always_use_local_files: bool = False,
+    reverse_dns_map_url: Optional[str] = None,
+    cache: Optional[ExpiringDict] = None,
+    reverse_dns_map: Optional[ReverseDNSMap] = None,
+    offline: bool = False,
+    nameservers: Optional[list[str]] = None,
+    timeout: float = 2.0,
+) -> IPAddressInfo:
    """
    Returns reverse DNS and country information for the given IP address

@@ -405,17 +450,27 @@ def get_ip_address_info(
        timeout (float): Sets the DNS timeout in seconds

    Returns:
-        OrderedDict: ``ip_address``, ``reverse_dns``
+        dict: ``ip_address``, ``reverse_dns``, ``country``

    """
    ip_address = ip_address.lower()
    if cache is not None:
-        info = cache.get(ip_address, None)
-        if info:
+        cached_info = cache.get(ip_address, None)
+        if (
+            cached_info
+            and isinstance(cached_info, dict)
+            and "ip_address" in cached_info
+        ):
            logger.debug(f"IP address {ip_address} was found in cache")
-            return info
-    info = OrderedDict()
-    info["ip_address"] = ip_address
+            return cast(IPAddressInfo, cached_info)
+    info: IPAddressInfo = {
+        "ip_address": ip_address,
+        "reverse_dns": None,
+        "country": None,
+        "base_domain": None,
+        "name": None,
+        "type": None,
+    }
    if offline:
        reverse_dns = None
    else:
@@ -425,9 +480,6 @@ def get_ip_address_info(
    country = get_ip_address_country(ip_address, db_path=ip_db_path)
    info["country"] = country
    info["reverse_dns"] = reverse_dns
-    info["base_domain"] = None
-    info["name"] = None
-    info["type"] = None
    if reverse_dns is not None:
        base_domain = get_base_domain(reverse_dns)
        if base_domain is not None:
@@ -452,7 +504,7 @@ def get_ip_address_info(
    return info


-def parse_email_address(original_address):
+def parse_email_address(original_address: str) -> dict[str, Optional[str]]:
    if original_address[0] == "":
        display_name = None
    else:
@@ -465,17 +517,15 @@ def parse_email_address(original_address):
        local = address_parts[0].lower()
        domain = address_parts[-1].lower()

-    return OrderedDict(
-        [
-            ("display_name", display_name),
-            ("address", address),
-            ("local", local),
-            ("domain", domain),
-        ]
-    )
+    return {
+        "display_name": display_name,
+        "address": address,
+        "local": local,
+        "domain": domain,
+    }


-def get_filename_safe_string(string):
+def get_filename_safe_string(string: str) -> str:
    """
    Converts a string to a string that is safe for a filename

@@ -497,7 +547,7 @@ def get_filename_safe_string(string):
    return string


-def is_mbox(path):
+def is_mbox(path: str) -> bool:
    """
    Checks if the given content is an MBOX mailbox file

@@ -518,7 +568,7 @@ def is_mbox(path):
    return _is_mbox


-def is_outlook_msg(content):
+def is_outlook_msg(content) -> bool:
    """
    Checks if the given content is an Outlook msg OLE/MSG file

@@ -533,7 +583,7 @@ def is_outlook_msg(content):
    )


-def convert_outlook_msg(msg_bytes):
+def convert_outlook_msg(msg_bytes: bytes) -> bytes:
    """
    Uses the ``msgconvert`` Perl utility to convert an Outlook MS file to
    standard RFC 822 format
@@ -542,7 +592,7 @@ def convert_outlook_msg(msg_bytes):
        msg_bytes (bytes): the content of the .msg file

    Returns:
-        A RFC 822 string
+        A RFC 822 bytes payload
    """
    if not is_outlook_msg(msg_bytes):
        raise ValueError("The supplied bytes are not an Outlook MSG file")
@@ -569,7 +619,9 @@ def convert_outlook_msg(msg_bytes):
    return rfc822


-def parse_email(data, strip_attachment_payloads=False):
+def parse_email(
+    data: Union[bytes, str], *, strip_attachment_payloads: bool = False
+) -> dict:
    """
    A simplified email parser

--- a/parsedmarc/webhook.py
+++ b/parsedmarc/webhook.py
@@ -1,3 +1,9 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
+from typing import Any, Optional, Union
+
 import requests

 from parsedmarc import logger
@@ -7,7 +13,13 @@ from parsedmarc.constants import USER_AGENT
 class WebhookClient(object):
    """A client for webhooks"""

-    def __init__(self, aggregate_url, forensic_url, smtp_tls_url, timeout=60):
+    def __init__(
+        self,
+        aggregate_url: str,
+        forensic_url: str,
+        smtp_tls_url: str,
+        timeout: Optional[int] = 60,
+    ):
        """
        Initializes the WebhookClient
        Args:
@@ -26,25 +38,27 @@ class WebhookClient(object):
            "Content-Type": "application/json",
        }

-    def save_forensic_report_to_webhook(self, report):
+    def save_forensic_report_to_webhook(self, report: str):
        try:
            self._send_to_webhook(self.forensic_url, report)
        except Exception as error_:
            logger.error("Webhook Error: {0}".format(error_.__str__()))

-    def save_smtp_tls_report_to_webhook(self, report):
+    def save_smtp_tls_report_to_webhook(self, report: str):
        try:
            self._send_to_webhook(self.smtp_tls_url, report)
        except Exception as error_:
            logger.error("Webhook Error: {0}".format(error_.__str__()))

-    def save_aggregate_report_to_webhook(self, report):
+    def save_aggregate_report_to_webhook(self, report: str):
        try:
            self._send_to_webhook(self.aggregate_url, report)
        except Exception as error_:
            logger.error("Webhook Error: {0}".format(error_.__str__()))

-    def _send_to_webhook(self, webhook_url, payload):
+    def _send_to_webhook(
+        self, webhook_url: str, payload: Union[bytes, str, dict[str, Any]]
+    ):
        try:
            self.session.post(webhook_url, data=payload, timeout=self.timeout)
        except Exception as error_:
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -2,6 +2,7 @@
 requires = [
    "hatchling>=1.27.0",
 ]
+requires_python = ">=3.9,<3.14"
 build-backend = "hatchling.build"

 [project]
@@ -28,6 +29,7 @@ classifiers = [
    "Operating System :: OS Independent",
    "Programming Language :: Python :: 3"
 ]
+requires-python = ">=3.9, <3.14"
 dependencies = [
    "azure-identity>=1.8.0",
    "azure-monitor-ingestion>=1.0.0",
@@ -46,7 +48,7 @@ dependencies = [
    "imapclient>=2.1.0",
    "kafka-python-ng>=2.2.2",
    "lxml>=4.4.0",
-    "mailsuite>=1.9.18",
+    "mailsuite>=1.11.1",
    "msgraph-core==0.2.2",
    "opensearch-py>=2.4.2,<=3.0.0",
    "publicsuffixlist>=0.10.0",
@@ -55,6 +57,7 @@ dependencies = [
    "tqdm>=4.31.1",
    "urllib3>=1.25.7",
    "xmltodict>=0.12.0",
+    "PyYAML>=6.0.3"
 ]

 [project.optional-dependencies]
@@ -85,11 +88,11 @@ include = [

 [tool.hatch.build]
 exclude = [
-"base_reverse_dns.csv",
-"find_bad_utf8.py",
-"find_unknown_base_reverse_dns.py",
-"unknown_base_reverse_dns.csv",
-"sortmaps.py",
-"README.md",
-"*.bak"
+    "base_reverse_dns.csv",
+    "find_bad_utf8.py",
+    "find_unknown_base_reverse_dns.py",
+    "unknown_base_reverse_dns.csv",
+    "sortmaps.py",
+    "README.md",
+    "*.bak"
 ]
--- a/tests.py
+++ b/tests.py
@@ -1,3 +1,6 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
 from __future__ import absolute_import, print_function, unicode_literals

 import os
@@ -43,11 +46,12 @@ class Test(unittest.TestCase):

    def testExtractReportXMLComparator(self):
        """Test XML comparator function"""
-        print()
-        xmlnice = open("samples/extract_report/nice-input.xml").read()
-        print(xmlnice)
-        xmlchanged = minify_xml(open("samples/extract_report/changed-input.xml").read())
-        print(xmlchanged)
+        xmlnice_file = open("samples/extract_report/nice-input.xml")
+        xmlnice = xmlnice_file.read()
+        xmlnice_file.close()
+        xmlchanged_file = open("samples/extract_report/changed-input.xml")
+        xmlchanged = minify_xml(xmlchanged_file.read())
+        xmlchanged_file.close()
        self.assertTrue(compare_xml(xmlnice, xmlnice))
        self.assertTrue(compare_xml(xmlchanged, xmlchanged))
        self.assertFalse(compare_xml(xmlnice, xmlchanged))
@@ -62,7 +66,9 @@ class Test(unittest.TestCase):
            data = f.read()
        print("Testing {0}: ".format(file), end="")
        xmlout = parsedmarc.extract_report(data)
-        xmlin = open("samples/extract_report/nice-input.xml").read()
+        xmlin_file = open("samples/extract_report/nice-input.xml")
+        xmlin = xmlin_file.read()
+        xmlin_file.close()
        self.assertTrue(compare_xml(xmlout, xmlin))
        print("Passed!")

@@ -71,8 +77,10 @@ class Test(unittest.TestCase):
        print()
        file = "samples/extract_report/nice-input.xml"
        print("Testing {0}: ".format(file), end="")
-        xmlout = parsedmarc.extract_report(file)
-        xmlin = open("samples/extract_report/nice-input.xml").read()
+        xmlout = parsedmarc.extract_report_from_file_path(file)
+        xmlin_file = open("samples/extract_report/nice-input.xml")
+        xmlin = xmlin_file.read()
+        xmlin_file.close()
        self.assertTrue(compare_xml(xmlout, xmlin))
        print("Passed!")

@@ -82,7 +90,9 @@ class Test(unittest.TestCase):
        file = "samples/extract_report/nice-input.xml.gz"
        print("Testing {0}: ".format(file), end="")
        xmlout = parsedmarc.extract_report_from_file_path(file)
-        xmlin = open("samples/extract_report/nice-input.xml").read()
+        xmlin_file = open("samples/extract_report/nice-input.xml")
+        xmlin = xmlin_file.read()
+        xmlin_file.close()
        self.assertTrue(compare_xml(xmlout, xmlin))
        print("Passed!")

@@ -92,12 +102,13 @@ class Test(unittest.TestCase):
        file = "samples/extract_report/nice-input.xml.zip"
        print("Testing {0}: ".format(file), end="")
        xmlout = parsedmarc.extract_report_from_file_path(file)
-        print(xmlout)
-        xmlin = minify_xml(open("samples/extract_report/nice-input.xml").read())
-        print(xmlin)
+        xmlin_file = open("samples/extract_report/nice-input.xml")
+        xmlin = minify_xml(xmlin_file.read())
+        xmlin_file.close()
        self.assertTrue(compare_xml(xmlout, xmlin))
-        xmlin = minify_xml(open("samples/extract_report/changed-input.xml").read())
-        print(xmlin)
+        xmlin_file = open("samples/extract_report/changed-input.xml")
+        xmlin = xmlin_file.read()
+        xmlin_file.close()
        self.assertFalse(compare_xml(xmlout, xmlin))
        print("Passed!")

@@ -145,6 +156,32 @@ class Test(unittest.TestCase):
            parsedmarc.parsed_smtp_tls_reports_to_csv(parsed_report)
            print("Passed!")

+    def testMSGraphWellKnownFolders(self):
+        """Test MSGraph well-known folder name mapping"""
+        from parsedmarc.mail.graph import WELL_KNOWN_FOLDER_MAP
+        
+        # Test English folder names
+        assert WELL_KNOWN_FOLDER_MAP.get("inbox") == "inbox"
+        assert WELL_KNOWN_FOLDER_MAP.get("sent items") == "sentitems"
+        assert WELL_KNOWN_FOLDER_MAP.get("deleted items") == "deleteditems"
+        assert WELL_KNOWN_FOLDER_MAP.get("archive") == "archive"
+        
+        # Test case insensitivity - simulating how the code actually uses it
+        # This is what happens when user config has "reports_folder = Inbox"
+        assert WELL_KNOWN_FOLDER_MAP.get("inbox") == "inbox"
+        assert WELL_KNOWN_FOLDER_MAP.get("Inbox".lower()) == "inbox"  # User's exact config
+        assert WELL_KNOWN_FOLDER_MAP.get("INBOX".lower()) == "inbox"
+        assert WELL_KNOWN_FOLDER_MAP.get("Archive".lower()) == "archive"
+        
+        # Test German folder names
+        assert WELL_KNOWN_FOLDER_MAP.get("posteingang") == "inbox"
+        assert WELL_KNOWN_FOLDER_MAP.get("Posteingang".lower()) == "inbox"  # Capitalized
+        assert WELL_KNOWN_FOLDER_MAP.get("archiv") == "archive"
+        
+        # Test that custom folders don't match
+        assert WELL_KNOWN_FOLDER_MAP.get("custom_folder") is None
+        assert WELL_KNOWN_FOLDER_MAP.get("my_reports") is None
+

 if __name__ == "__main__":
    unittest.main(verbosity=2)
Author	SHA1	Message	Date
copilot-swe-agent[bot]	eb2218b6fc	Improve test to explicitly demonstrate case-insensitive handling of folder names like 'Inbox' Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>	2025-12-31 21:00:38 +00:00
copilot-swe-agent[bot]	3f2fc5f727	Add unit test for MSGraph well-known folder name mapping Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>	2025-12-31 20:47:47 +00:00
copilot-swe-agent[bot]	f94c28c770	Update documentation with MSGraph well-known folder names and add example configuration Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>	2025-12-31 20:44:56 +00:00
copilot-swe-agent[bot]	c0f05b81b8	Add well-known folder name support for MSGraph to avoid "Default folder Root not found" error Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>	2025-12-31 20:43:44 +00:00
copilot-swe-agent[bot]	9c9ef2fa50	Initial plan	2025-12-31 20:39:17 +00:00
Sean Whalen	1f3a1fc843	Better typing	2025-12-29 17:14:54 -05:00
Sean Whalen	34fa0c145d	9.0.8 - Fix logging configuration not propagating to child parser processes (#646). - Update `mailsuite` dependency to `?=1.11.1` to solve issues with iCloud IMAP (#493).	2025-12-29 17:07:38 -05:00
Copilot	6719a06388	Fix logging configuration not propagating to child parser processes (#646 ) * Initial plan * Fix logging configuration propagation to child parser processes - Add _configure_logging() helper function to set up logging in child processes - Modified cli_parse() to accept log_level and log_file parameters - Pass current logging configuration from parent to child processes - Logging warnings/errors from child processes now properly display Fixes issue where logging handlers in parent process were not inherited by child processes created via multiprocessing.Process(). Child processes now configure their own logging with the same settings as the parent. Tested with sample files and confirmed warnings from DNS exceptions in child processes are now visible. Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com> * Address code review feedback on logging configuration - Use exact type check (type(h) is logging.StreamHandler) instead of isinstance to avoid confusion with FileHandler subclass - Catch specific exceptions (IOError, OSError, PermissionError) instead of bare Exception when creating FileHandler - Kept logging.ERROR as default to maintain consistency with existing behavior Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>	2025-12-29 15:07:22 -05:00
Sean Whalen	eafa435868	Code cleanup	2025-12-29 14:32:05 -05:00
Sean Whalen	5d772c3b36	Bump version to 9.0.7 and update changelog with IMAP `since` option fix	2025-12-29 14:23:50 -05:00
Copilot	72cabbef23	Fix IMAP SEARCH SINCE date format to RFC 3501 DD-Mon-YYYY (#645 ) * Initial plan * Fix IMAP since option date format to use RFC 3501 compliant DD-Mon-YYYY format Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>	2025-12-29 14:18:48 -05:00
Sean Whalen	3d74cd6ac0	Update CHANGELOG with issue reference for email read status Added a reference to issue #625 regarding email read status.	2025-12-29 12:10:19 -05:00
Tomáš Kováčik	d1ac59a016	fix #641 (#642 ) * fix smtptls and forensic reports for GELF * add policy_domain, policy_type and failed_session_count to record row * Remove unused import of json in gelf.py --------- Co-authored-by: Sean Whalen <44679+seanthegeek@users.noreply.github.com>	2025-12-29 12:05:07 -05:00
Anael Mobilia	7fdd53008f	Update README.md (#644 )	2025-12-29 10:36:21 -05:00
Sean Whalen	35331d4b84	Add parsedmarc.types module to API reference documentation	2025-12-25 17:24:45 -05:00
Sean Whalen	de9edd3590	Add note about email read status in Microsoft 365 to changelog	2025-12-25 17:16:39 -05:00
Sean Whalen	abf4bdba13	Add type annotations for SMTP TLS and forensic report structures	2025-12-25 16:39:33 -05:00
Sean Whalen	7b842740f5	Change file permissions for tests.py to make it executable	2025-12-25 16:02:33 -05:00
Sean Whalen	ebe3ccf40a	Update changelog for version 9.0.6 and set version in constants.py	2025-12-25 16:01:25 -05:00
Sean Whalen	808285658f	Refactor function parameters to use non-Optional types where applicable	2025-12-25 16:01:12 -05:00
Sean Whalen	bc1dae29bd	Update mailsuite dependency version to 1.11.0	2025-12-25 15:32:27 -05:00
Sean Whalen	4b904444e5	Refactor and improve parsing and extraction functions - Updated `extract_report` to handle various input types more robustly, removing unnecessary complexity and improving error handling. - Simplified the handling of file-like objects and added checks for binary mode. - Enhanced the `parse_report_email` function to streamline input processing and improve type handling. - Introduced TypedDicts for better type safety in `utils.py`, specifically for reverse DNS and IP address information. - Refined the configuration loading in `cli.py` to ensure boolean values are consistently cast to `bool`. - Improved overall code readability and maintainability by restructuring and clarifying logic in several functions.	2025-12-25 15:30:20 -05:00
Sean Whalen	3608bce344	Remove unused import of Union and cast from cli.py	2025-12-24 16:53:22 -05:00
Sean Whalen	fe809c4c3f	Add type ignore comments for Pyright in elastic.py and opensearch.py	2025-12-24 16:49:42 -05:00
Sean Whalen	a76c2f9621	More code cleanup	2025-12-24 16:36:59 -05:00
Sean Whalen	bb8f4002bf	Use literal dicts instead of ordered dicts and other code cleanup	2025-12-24 15:04:10 -05:00
Sean Whalen	b5773c6b4a	Fix etree import to type checkers don't complain	2025-12-24 14:37:38 -05:00
Sean Whalen	b99bd67225	Fix get_base_domain() typing	2025-12-24 14:32:05 -05:00
Sean Whalen	af9ad568ec	Specify Python version requirements in pyproject.toml	2025-12-17 16:18:24 -05:00
Sean Whalen	748164d177	Fix #638	2025-12-17 16:09:26 -05:00
Sean Whalen	487e5e1149	Format on build	2025-12-12 15:56:52 -05:00
Sean Whalen	73010cf964	Use ruff for code formatting	2025-12-12 15:44:46 -05:00
Sean Whalen	a4a5475aa8	Fix another typo before releasing 9.0.5	2025-12-08 15:29:48 -05:00
Sean Whalen	dab78880df	Actual 9.0.5 release Fix typo	2025-12-08 15:26:58 -05:00
Sean Whalen	fb54e3b742	9.0.5 - Fix report type detection bug introduced in `9.0.4` (yanked).	2025-12-08 15:22:02 -05:00
Sean Whalen	6799f10364	9.0.4 Fixes - Fix saving reports to OpenSearch ([#637](https://github.com/domainaware/parsedmarc/issues/637)) - Fix parsing certain DMARC failure/forensic reports - Some fixes to type hints (incomplete, but published as-is due to the above bugs)	2025-12-08 13:26:59 -05:00
Sean Whalen	445c9565a4	Update bug link in docs	2025-12-06 15:05:19 -05:00
Sean Whalen	4b786846ae	Remove Python 3.14 from testing Until cpython bug https://github.com/python/cpython/issues/142307 is fixed	2025-12-05 11:05:29 -05:00
Sean Whalen	23ae563cd8	Update Python version support details in documentation	2025-12-05 10:48:04 -05:00
Sean Whalen	cdd000e675	9.0.3 - Set `requires-python` to `>=3.9, <3.14` to avoid [this bug](https://github.com/python/cpython/issues/142307)	2025-12-05 10:43:28 -05:00
Sean Whalen	7d58abc67b	Add shebang and encoding declaration to tests.py	2025-12-04 10:21:53 -05:00
Sean Whalen	a18ae439de	Fix typo in RHEL version support description in documentation	2025-12-04 10:18:15 -05:00
Sean Whalen	d7061330a8	Use None for blank fields in the Top 1000 Message Sources by Name DMARC Summary dashboard widget	2025-12-03 09:22:33 -05:00
Sean Whalen	9d5654b8ec	Fix bugs with the Top 1000 Message Sources by Name DMARC Summary dashboard widget	2025-12-03 09:14:52 -05:00
Sean Whalen	a0e0070dd0	Bump version to 9.0.2	2025-12-02 20:12:58 -05:00
Sean Whalen	cf3b7f2c29	## 9.0.2 ## Improvements - Type hinting is now used properly across the entire library. (#445) ## Fixes - Decompress report files as needed when passed via the CLI. - Fixed incomplete removal of the ability for `parsedmarc.utils.extract_report` to accept a file path directly in `8.15.0`. ## Breaking changes This version of the library requires consumers to pass certain arguments as keyword-only. Internally, the API uses a bare `*` in the function signature. This is standard per [PEP 3102](https://peps.python.org/pep-3102/) and as documented in the Python Language Reference. .	2025-12-02 19:41:14 -05:00
Sean Whalen	d312522ab7	Enhance type hints and argument formatting in multiple files for improved clarity and consistency	2025-12-02 17:06:57 -05:00
Sean Whalen	888d717476	Enhance type hints and argument formatting in utils.py for improved clarity and consistency	2025-12-02 16:21:30 -05:00
Sean Whalen	1127f65fbb	Enhance type hints and argument formatting in webhook.py for improved clarity and consistency	2025-12-02 15:52:31 -05:00
Sean Whalen	d017dfcddf	Enhance type hints and argument formatting across multiple files for improved clarity and consistency	2025-12-02 15:17:37 -05:00
Sean Whalen	5fae99aacc	Enhance type hints for improved clarity and consistency in __init__.py, elastic.py, and opensearch.py	2025-12-02 14:14:06 -05:00
Sean Whalen	ba57368ac3	Refactor argument formatting and type hints in elastic.py for consistency	2025-12-02 13:13:25 -05:00
Sean Whalen	dc6ee5de98	Add type hints to methods in opensearch.py for improved clarity and type checking	2025-12-02 13:11:59 -05:00
Sean Whalen	158d63d205	Complete annotations on elastic.py	2025-12-02 12:59:03 -05:00
Oscar Mattsson	f1933b906c	Fix 404 link to maxmind docs (#635 )	2025-12-02 09:26:01 -05:00
Anael Mobilia	4b98d795ff	Define minimal Python version on pyproject (#634 )	2025-12-01 20:22:49 -05:00
Sean Whalen	b1356f7dfc	9.0.1 - Allow multiple `records` for the same aggregate DMARC report in Elasticsearch and Opensearch (fixes issue in 9.0.0) - Fix typos	2025-12-01 18:57:23 -05:00
Sean Whalen	1969196e1a	Switch CHANGELOG headers	2025-12-01 18:01:54 -05:00
Sean Whalen	553f15f6a9	Code formatting	2025-12-01 17:24:10 -05:00
Sean Whalen	1fc9f638e2	9.0.0 (#629 ) * Normalize report volumes when a report timespan exceed 24 hours	2025-12-01 17:06:58 -05:00
Sean Whalen	48bff504b4	Fix build script to properly publish docs	2025-12-01 11:08:21 -05:00
Sean Whalen	681b7cbf85	Formatting	2025-12-01 10:56:08 -05:00
Sean Whalen	0922d6e83a	Add supported Python versions to the documentation index	2025-12-01 10:24:19 -05:00
Sean Whalen	baf3f95fb1	Update README with clarification on Python 3.6 support	2025-12-01 10:20:56 -05:00
Anael Mobilia	a51f945305	Clearly define supported Python versions policy (#633 ) * Clearly define supported Python versions. Support policy based on author's comment on https://github.com/domainaware/parsedmarc/pull/458#issuecomment-2002516299 #458 * Compile Python 3.6 as Ubuntu latest run against Ubuntu 24.04 which haven't Python3.6 + 20.04 is no longer available https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json * Use latest versions of GH Actions * Silent some technicals GH Actions steps * Elasticsearch / opensearch: use supported versions + align used versions * Delete .github/workflows/python-tests-3.6.yml Drop Python 3.6 test * Update Python 3.6 support status in README --------- Co-authored-by: Sean Whalen <44679+seanthegeek@users.noreply.github.com>	2025-12-01 10:02:47 -05:00
Sean Whalen	55dbf8e3db	Add sources my name table to the Kibana DMARC Summary dashboard This matches the table in the Splunk DMARC Aggregate reports dashboard	2025-11-30 19:44:14 -05:00
Anael Mobilia	00267c9847	Codestyle cleanup (#631 ) * Fix typos * Copyright - Update date * Codestyle xxx is False -> not xxx * Ensure "_find_label_id_for_label" always return str * PEP-8 : apiKey -> api_key + backward compatibility for config files * Duplicate variable initialization * Fix format	2025-11-30 19:13:57 -05:00
Anael Mobilia	51356175e1	Get option on the type described on documentation (#632 )	2025-11-30 19:00:04 -05:00
Anael Mobilia	3be10d30dd	Fix warnings in docker-compose.yml (#630 ) * Fix level=warning msg="...\parsedmarc\docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion" * Fix "Unquoted port mapping not recommended"	2025-11-30 18:59:01 -05:00
Sean Whalen	98342ecac6	8.19.1 (#627 ) - Ignore HTML content type in report email parsing (#626)	2025-11-29 11:37:31 -05:00
Sean Whalen	38a3d4eaae	Code formatting	2025-11-28 12:48:55 -05:00
Sean Whalen	a05c230152	8.19.0 (#622 ) 8.19.0 - Add multi-tenant support via an index-prefix domain mapping file - PSL overrides so that services like AWS are correctly identified - Additional improvements to report type detection - Fix webhook timeout parsing (PR #623) - Output to STDOUT when the new general config boolean `silent` is set to `False` (Close #614) - Additional services added to `base_reverse_dns_map.csv` --------- Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com> Co-authored-by: Félix <felix.debloisbeaucage@gmail.com>	2025-11-28 12:47:00 -05:00
Sean Whalen	17bdc3a134	More tests cleanup	2025-11-21 09:10:59 -05:00
Sean Whalen	858be00f22	Fix badge links and update image source branch	2025-11-21 09:03:04 -05:00
Sean Whalen	597ca64f9f	Clean up tests	2025-11-21 00:09:28 -05:00
Sean Whalen	c5dbe2c4dc	8.10.9 - Complete fix for #687 and more robust report type detection	2025-11-20 23:50:42 -05:00
Sean Whalen	082b3d355f	8.18.8 - Fix parsing emails with an uncompressed aggregate report attachment (Closes #607) - Add `--no-prettify-json` CLI option (PR #617)	2025-11-20 20:47:57 -05:00
Sean Whalen	2a7ce47bb1	Update code coverage badge link to main branch	2025-11-20 20:28:10 -05:00
daminoux	9882405d96	Update README.md fix url screenshot (#620 ) the url of screenshot is broken	2025-11-20 20:27:15 -05:00
Andrew	fce84763b9	add --no-prettify-json CLI option (#617 ) * updates process_reports to respect newly added prettify_json option * removes duplicate definition * removes redundant option * fixes typo	2025-11-02 15:54:59 -05:00
Rowan	8a299b8600	Updated default python docker base image to 3.13-slim (#618 ) * Updated default python docker base image to 3.13-slim * Added python 3.13 to tests	2025-10-29 22:34:06 -04:00
jandr	b4c2b21547	Sorted usage of TLS on SMTP (#613 ) Added a line for the `email_results` function to take into account the smtp_ssl setting.	2025-08-25 13:51:10 -04:00
Sean Whalen	865c249437	Update features list	2025-08-24 13:39:50 -04:00
Sean Whalen	013859f10e	Fix find_unknown_base_reverse_dns.py	2025-08-19 21:18:14 -04:00
Sean Whalen	6d4a31a120	Fix find_unknown_base_reverse_dns.py and sortlist.py	2025-08-19 20:59:42 -04:00
Sean Whalen	45d3dc3b2e	Fiz sortlists.py	2025-08-19 20:23:55 -04:00
Sean Whalen	4bbd97dbaa	Improve list verification	2025-08-19 20:02:55 -04:00
Sean Whalen	5df152d469	Refactor find_unknown_base_reverse_dns.py	2025-08-18 12:59:54 -04:00
Sean Whalen	d990bef342	Use \n here too	2025-08-17 21:08:28 -04:00
Sean Whalen	caf77ca6d4	Use \n when writing CSVs	2025-08-17 21:01:07 -04:00
Sean Whalen	4b3d32c5a6	Actual, actual Actual 6.18.7 release Revert back to using python csv instead of pandas to avoid conflicts with numpy in elasticsearch	2025-08-17 20:36:15 -04:00
Sean Whalen	5df5c10f80	Pin pandas an numpy versions	2025-08-17 19:59:53 -04:00
Sean Whalen	308d4657ab	Make sort_csv function more flexible	2025-08-17 19:43:19 -04:00
Sean Whalen	0f74e33094	Fix typo	2025-08-17 19:35:16 -04:00
Sean Whalen	9f339e11f5	Actual 6.18.7 release	2025-08-17 19:34:14 -04:00
Sean Whalen	391e84b717	Fix map sorting	2025-08-17 18:15:20 -04:00
Sean Whalen	8bf06ce5af	8.18.7 Removed improper spaces from `base_reverse_dns_map.csv` (Closes #612)	2025-08-17 18:13:49 -04:00