From 5915230d4befd1a359dc2ea7ef01e583e58ac367 Mon Sep 17 00:00:00 2001 From: Sean Whalen Date: Thu, 23 Apr 2026 00:37:54 -0400 Subject: [PATCH] Update docs --- _modules/index.html | 4 +-- _modules/parsedmarc.html | 22 ++++++------- _modules/parsedmarc/elastic.html | 4 +-- _modules/parsedmarc/opensearch.html | 4 +-- _modules/parsedmarc/splunk.html | 4 +-- _modules/parsedmarc/types.html | 4 +-- _modules/parsedmarc/utils.html | 49 ++++++++++++++++------------- _sources/installation.md.txt | 26 ++++++++------- _sources/usage.md.txt | 2 +- _static/documentation_options.js | 2 +- api.html | 22 ++++++------- contributing.html | 4 +-- davmail.html | 4 +-- dmarc.html | 4 +-- elasticsearch.html | 4 +-- genindex.html | 4 +-- index.html | 4 +-- installation.html | 27 ++++++++-------- kibana.html | 4 +-- mailing-lists.html | 4 +-- objects.inv | 2 +- opensearch.html | 4 +-- output.html | 4 +-- py-modindex.html | 4 +-- search.html | 4 +-- searchindex.js | 2 +- splunk.html | 4 +-- usage.html | 6 ++-- 28 files changed, 120 insertions(+), 112 deletions(-) diff --git a/_modules/index.html b/_modules/index.html index 1e91f15..8a93a14 100644 --- a/_modules/index.html +++ b/_modules/index.html @@ -5,14 +5,14 @@ - Overview: module code — parsedmarc 9.7.1 documentation + Overview: module code — parsedmarc 9.8.0 documentation - + diff --git a/_modules/parsedmarc.html b/_modules/parsedmarc.html index 755e24e..2e9e709 100644 --- a/_modules/parsedmarc.html +++ b/_modules/parsedmarc.html @@ -5,14 +5,14 @@ - parsedmarc — parsedmarc 9.7.1 documentation + parsedmarc — parsedmarc 9.8.0 documentation - + @@ -413,7 +413,7 @@ always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP offline (bool): Do not query online for geolocation or DNS nameservers (list): A list of one or more nameservers to use (Cloudflare's public DNS resolvers by default) @@ -792,7 +792,7 @@ Args: xml (str): A string of DMARC aggregate report XML - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file @@ -1125,7 +1125,7 @@ always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP nameservers (list): A list of one or more nameservers to use (Cloudflare's public DNS resolvers by default) dns_timeout (float): Sets the DNS timeout in seconds @@ -1382,7 +1382,7 @@ Args: feedback_report (str): A message's feedback report as a string sample (str): The RFC 822 headers or RFC 822 message sample - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file @@ -1627,7 +1627,7 @@ Args: input_: An emailed DMARC report in RFC 822 format, as bytes or a string - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map reverse_dns_map_url (str): URL to a reverse DNS map @@ -1852,7 +1852,7 @@ or other transient errors strip_attachment_payloads (bool): Remove attachment payloads from forensic report results - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map reverse_dns_map_url (str): URL to a reverse DNS map @@ -1954,7 +1954,7 @@ always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP offline (bool): Do not make online queries for geolocation or DNS normalize_timespan_threshold_hours (float): Normalize timespans beyond this @@ -2050,7 +2050,7 @@ archive_folder (str): The folder to move processed mail to delete (bool): Delete messages after processing them test (bool): Do not move or delete messages after processing them - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file @@ -2395,7 +2395,7 @@ test (bool): Do not move or delete messages after processing them check_timeout (int): Number of seconds to wait for a IMAP IDLE response or the number of seconds until the next mail check - ip_db_path (str): Path to a MMDB file from MaxMind or DBIP + ip_db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP always_use_local_files (bool): Do not download files reverse_dns_map_path (str): Path to a reverse DNS map file reverse_dns_map_url (str): URL to a reverse DNS map file diff --git a/_modules/parsedmarc/elastic.html b/_modules/parsedmarc/elastic.html index 0e3bbe1..9f19679 100644 --- a/_modules/parsedmarc/elastic.html +++ b/_modules/parsedmarc/elastic.html @@ -5,14 +5,14 @@ - parsedmarc.elastic — parsedmarc 9.7.1 documentation + parsedmarc.elastic — parsedmarc 9.8.0 documentation - + diff --git a/_modules/parsedmarc/opensearch.html b/_modules/parsedmarc/opensearch.html index b87c910..0bf04c2 100644 --- a/_modules/parsedmarc/opensearch.html +++ b/_modules/parsedmarc/opensearch.html @@ -5,14 +5,14 @@ - parsedmarc.opensearch — parsedmarc 9.7.1 documentation + parsedmarc.opensearch — parsedmarc 9.8.0 documentation - + diff --git a/_modules/parsedmarc/splunk.html b/_modules/parsedmarc/splunk.html index e83b8a7..9609c51 100644 --- a/_modules/parsedmarc/splunk.html +++ b/_modules/parsedmarc/splunk.html @@ -5,14 +5,14 @@ - parsedmarc.splunk — parsedmarc 9.7.1 documentation + parsedmarc.splunk — parsedmarc 9.8.0 documentation - + diff --git a/_modules/parsedmarc/types.html b/_modules/parsedmarc/types.html index b67a3ed..5ce947e 100644 --- a/_modules/parsedmarc/types.html +++ b/_modules/parsedmarc/types.html @@ -5,14 +5,14 @@ - parsedmarc.types — parsedmarc 9.7.1 documentation + parsedmarc.types — parsedmarc 9.8.0 documentation - + diff --git a/_modules/parsedmarc/utils.html b/_modules/parsedmarc/utils.html index 58f83c8..df777f4 100644 --- a/_modules/parsedmarc/utils.html +++ b/_modules/parsedmarc/utils.html @@ -5,14 +5,14 @@ - parsedmarc.utils — parsedmarc 9.7.1 documentation + parsedmarc.utils — parsedmarc 9.8.0 documentation - + @@ -114,13 +114,12 @@ import dns.exception import dns.resolver import dns.reversename -import geoip2.database -import geoip2.errors +import maxminddb import publicsuffixlist import requests from dateutil.parser import parse as parse_date -import parsedmarc.resources.dbip +import parsedmarc.resources.ipinfo import parsedmarc.resources.maps from parsedmarc.constants import ( DEFAULT_DNS_MAX_RETRIES, @@ -539,8 +538,8 @@ if url is None: url = ( "https://github.com/domainaware/parsedmarc/raw/" - "refs/heads/master/parsedmarc/resources/dbip/" - "dbip-country-lite.mmdb" + "refs/heads/master/parsedmarc/resources/ipinfo/" + "ipinfo_lite.mmdb" ) if local_file_path is not None and os.path.isfile(local_file_path): @@ -549,7 +548,7 @@ return cache_dir = os.path.join(tempfile.gettempdir(), "parsedmarc") - cached_path = os.path.join(cache_dir, "dbip-country-lite.mmdb") + cached_path = os.path.join(cache_dir, "ipinfo_lite.mmdb") if not (offline or always_use_local_file): try: @@ -577,9 +576,7 @@ return # Final fallback: bundled copy - _IP_DB_PATH = str( - files(parsedmarc.resources.dbip).joinpath("dbip-country-lite.mmdb") - ) + _IP_DB_PATH = str(files(parsedmarc.resources.ipinfo).joinpath("ipinfo_lite.mmdb")) logger.info("Using bundled IP database") @@ -595,12 +592,13 @@ Args: ip_address (str): The IP address to query for - db_path (str): Path to a MMDB file from MaxMind or DBIP + db_path (str): Path to a MMDB file from IPinfo, MaxMind, or DBIP Returns: str: And ISO country code associated with the given IP address """ db_paths = [ + "ipinfo_lite.mmdb", "GeoLite2-Country.mmdb", "/usr/local/share/GeoIP/GeoLite2-Country.mmdb", "/usr/share/GeoIP/GeoLite2-Country.mmdb", @@ -616,12 +614,12 @@ if db_path is not None: if not os.path.isfile(db_path): - db_path = None logger.warning( f"No file exists at {db_path}. Falling back to an " - "included copy of the IPDB IP to Country " + "included copy of the IPinfo IP to Country " "Lite database." ) + db_path = None if db_path is None: for system_path in db_paths: @@ -634,21 +632,28 @@ db_path = _IP_DB_PATH else: db_path = str( - files(parsedmarc.resources.dbip).joinpath("dbip-country-lite.mmdb") + files(parsedmarc.resources.ipinfo).joinpath("ipinfo_lite.mmdb") ) db_age = datetime.now() - datetime.fromtimestamp(os.stat(db_path).st_mtime) if db_age > timedelta(days=30): logger.warning("IP database is more than a month old") - db_reader = geoip2.database.Reader(db_path) + db_reader = maxminddb.open_database(db_path) + record = db_reader.get(ip_address) - country = None - - try: - country = db_reader.country(ip_address).country.iso_code - except geoip2.errors.AddressNotFoundError: - pass + # Support both the IPinfo schema (flat top-level ``country_code``) and the + # MaxMind/DBIP schema (nested ``country.iso_code``) so users dropping in + # their own MMDB from any of these providers keeps working. + country: Optional[str] = None + if isinstance(record, dict): + code = record.get("country_code") + if code is None: + nested = record.get("country") + if isinstance(nested, dict): + code = nested.get("iso_code") + if isinstance(code, str): + country = code return country diff --git a/_sources/installation.md.txt b/_sources/installation.md.txt index b1f1a61..530906d 100644 --- a/_sources/installation.md.txt +++ b/_sources/installation.md.txt @@ -44,19 +44,20 @@ least: ### geoipupdate setup :::{note} -Starting in `parsedmarc` 7.1.0, a static copy of the -[IP to Country Lite database] from IPDB is distributed with -`parsedmarc`, under the terms of the -[Creative Commons Attribution 4.0 International License]. +Starting in `parsedmarc` 9.8.0, a static copy of the +[IPinfo Lite] database is distributed with `parsedmarc`, under the +terms of the [Creative Commons Attribution-ShareAlike 4.0 License], as a fallback if the [MaxMind GeoLite2 Country database] is not -installed. +installed. Prior versions bundled the DB-IP Country Lite database +instead; both share the same MMDB format, so users who have installed +either (or a MaxMind GeoLite2) database locally will continue to work +without changes. -Starting in `parsedmarc` 9.6.0, the bundled DB-IP database is -automatically updated at startup by downloading the latest copy from -GitHub, unless the `offline` flag is set. The database is cached -locally and refreshed on each run (or on `SIGHUP` in watch mode). -If the download fails, a previously cached copy or the bundled -database is used as a fallback. +The bundled database is automatically updated at startup by downloading +the latest copy from GitHub, unless the `offline` flag is set. The +database is cached locally and refreshed on each run (or on `SIGHUP` +in watch mode). If the download fails, a previously cached copy or the +bundled database is used as a fallback. The download URL can be overridden with the `ip_db_url` setting, and the location of a local database file can be overridden with the @@ -203,7 +204,8 @@ sudo apt-get install libemail-outlook-message-perl [Component "contrib"]: https://wiki.debian.org/SourcesList#Component [geoipupdate]: https://github.com/maxmind/geoipupdate [geoipupdate releases page on github]: https://github.com/maxmind/geoipupdate/releases -[ip to country lite database]: https://db-ip.com/db/download/ip-to-country-lite +[ipinfo lite]: https://ipinfo.io/lite +[creative commons attribution-sharealike 4.0 license]: https://creativecommons.org/licenses/by-sa/4.0/deed.en [license keys]: https://www.maxmind.com/en/accounts/current/license-key [maxmind geoipupdate page]: https://dev.maxmind.com/geoip/updating-databases/ [maxmind geolite2 country database]: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data diff --git a/_sources/usage.md.txt b/_sources/usage.md.txt index c3b1584..27d682e 100644 --- a/_sources/usage.md.txt +++ b/_sources/usage.md.txt @@ -133,7 +133,7 @@ The full set of configuration options are: - `forensic_json_filename` - str: filename for the forensic JSON output file - `ip_db_path` - str: An optional custom path to a MMDB file - from MaxMind or DBIP + from IPinfo, MaxMind, or DBIP - `ip_db_url` - str: Overrides the default download URL for the IP-to-country database (env var: `PARSEDMARC_GENERAL_IP_DB_URL`) - `offline` - bool: Do not use online queries for geolocation diff --git a/_static/documentation_options.js b/_static/documentation_options.js index f3528d1..a795c44 100644 --- a/_static/documentation_options.js +++ b/_static/documentation_options.js @@ -1,5 +1,5 @@ const DOCUMENTATION_OPTIONS = { - VERSION: '9.7.1', + VERSION: '9.8.0', LANGUAGE: 'en', COLLAPSE_INDEX: false, BUILDER: 'html', diff --git a/api.html b/api.html index df4b7a7..42d054a 100644 --- a/api.html +++ b/api.html @@ -6,14 +6,14 @@ - API reference — parsedmarc 9.7.1 documentation + API reference — parsedmarc 9.8.0 documentation - + @@ -298,7 +298,7 @@ file-like object, or bytes.

  • archive_folder (str) – The folder to move processed mail to

  • delete (bool) – Delete messages after processing them

  • test (bool) – Do not move or delete messages after processing them

    • -

  • ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP

    • +

  • ip_db_path (str) – Path to a MMDB file from IPinfo, MaxMind, or DBIP

  • always_use_local_files (bool) – Do not download files

  • reverse_dns_map_path (str) – Path to a reverse DNS map file

  • reverse_dns_map_url (str) – URL to a reverse DNS map file

    • @@ -347,7 +347,7 @@ forensic report results

  • always_use_local_files (bool) – Do not download files

  • reverse_dns_map_path (str) – Path to a reverse DNS map file

  • reverse_dns_map_url (str) – URL to a reverse DNS map file

    • -

  • ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP

    • +

  • ip_db_path (str) – Path to a MMDB file from IPinfo, MaxMind, or DBIP

  • offline (bool) – Do not make online queries for geolocation or DNS

  • normalize_timespan_threshold_hours (float) – Normalize timespans beyond this

    • @@ -391,7 +391,7 @@ aggregate DMARC report

  • always_use_local_files (bool) – Do not download files

  • reverse_dns_map_path (str) – Path to a reverse DNS map file

  • reverse_dns_map_url (str) – URL to a reverse DNS map file

    • -

  • ip_db_path (str) – Path to a MMDB file from MaxMind or DBIP

    • +

  • ip_db_path (str) – Path to a MMDB file from IPinfo, MaxMind, or DBIP

  • nameservers (list) – A list of one or more nameservers to use (Cloudflare’s public DNS resolvers by default)

  • dns_timeout (float) – Sets the DNS timeout in seconds

    • @@ -418,7 +418,7 @@ or other transient errors

    Parameters: