mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-04-25 23:09:28 +00:00
576c68ed67
Rebased on top of master @ 2cda5bf (9.9.0), which added the ASN
source attribution work (#712, #713, #714, #715). Individual Copilot
iteration commits squashed into this single commit — the per-commit
history on the feature branch was iterative (add tests, fix lint,
move field, revert, etc.) and not worth preserving; GitHub squash-
merges PRs anyway.
New fields from the DMARCbis XSD, plumbed through types, parsing, CSV
output, and the Elasticsearch / OpenSearch mappings:
- ``np`` — non-existent subdomain policy (``none`` / ``quarantine`` /
``reject``)
- ``testing`` — testing mode flag (``n`` / ``y``), replaces RFC 7489
``pct``
- ``discovery_method`` — policy discovery method (``psl`` /
``treewalk``)
- ``generator`` — report generator software identifier (metadata)
- ``human_result`` — optional descriptive text on DKIM / SPF results
RFC 7489 reports parse with ``None`` for DMARCbis-only fields.
Forensic reports have been renamed to failure reports throughout the
project to reflect the proper naming since RFC 7489.
- Core: ``types.py``, ``__init__.py`` — ``ForensicReport`` →
``FailureReport``, ``parse_forensic_report`` →
``parse_failure_report``, report type ``"failure"``.
- Output modules: ``elastic.py``, ``opensearch.py``, ``splunk.py``,
``kafkaclient.py``, ``syslog.py``, ``gelf.py``, ``webhook.py``,
``loganalytics.py``, ``s3.py``.
- CLI: ``cli.py`` — args, config keys, index names
(``dmarc_failure``).
- Docs + dashboards: all markdown, Grafana JSON, Kibana NDJSON,
Splunk XML.
Backward compatibility preserved: old function / type names remain as
aliases (``parse_forensic_report = parse_failure_report``,
``ForensicReport = FailureReport``, etc.), CLI accepts both the old
(``save_forensic``, ``forensic_topic``) and new (``save_failure``,
``failure_topic``) config keys, and updated dashboards query both
old and new index / sourcetype names so data from before and after
the rename appears together.
Merge conflicts resolved in ``parsedmarc/constants.py`` (took bis's
10.0.0 bump), ``parsedmarc/__init__.py`` (combined bis's "failure"
wording with master's IPinfo MMDB mention), ``parsedmarc/elastic.py``
and ``parsedmarc/opensearch.py`` (kept master's ``source_asn`` /
``source_asn_name`` / ``source_asn_domain`` on the failure doc path
while renaming ``forensic_report`` → ``failure_report``), and
``CHANGELOG.md`` (10.0.0 entry now sits above the 9.9.0 entry).
All 324 tests pass; ``ruff check`` / ``ruff format --check`` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
89 lines
2.2 KiB
YAML
89 lines
2.2 KiB
YAML
name: Python tests
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
on:
|
|
push:
|
|
branches: [ master ]
|
|
pull_request:
|
|
branches: [ master ]
|
|
|
|
jobs:
|
|
lint-docs-build:
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.13"
|
|
- name: Install Python dependencies
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
pip install .[build]
|
|
- name: Check code style
|
|
run: |
|
|
ruff check .
|
|
- name: Test building documentation
|
|
run: |
|
|
cd docs
|
|
make html
|
|
- name: Test building packages
|
|
run: |
|
|
hatch build
|
|
|
|
test:
|
|
needs: lint-docs-build
|
|
runs-on: ubuntu-latest
|
|
|
|
services:
|
|
elasticsearch:
|
|
image: elasticsearch:8.19.7
|
|
env:
|
|
discovery.type: single-node
|
|
cluster.name: parsedmarc-cluster
|
|
discovery.seed_hosts: elasticsearch
|
|
bootstrap.memory_lock: true
|
|
xpack.security.enabled: false
|
|
xpack.license.self_generated.type: basic
|
|
ports:
|
|
- 9200:9200
|
|
- 9300:9300
|
|
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
|
|
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- name: Set up Python ${{ matrix.python-version }}
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: ${{ matrix.python-version }}
|
|
- name: Install system dependencies
|
|
run: |
|
|
sudo apt-get -q update
|
|
sudo apt-get -qy install libemail-outlook-message-perl
|
|
- name: Install Python dependencies
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
pip install .[build]
|
|
- name: Run unit tests
|
|
run: |
|
|
pytest --cov --cov-report=xml tests.py
|
|
- name: Test sample DMARC reports
|
|
run: |
|
|
pip install -e .
|
|
parsedmarc --debug -c ci.ini samples/aggregate/*
|
|
parsedmarc --debug -c ci.ini samples/failure/*
|
|
- name: Test building packages
|
|
run: |
|
|
hatch build
|
|
- name: Upload coverage to Codecov
|
|
uses: codecov/codecov-action@v5
|
|
with:
|
|
token: ${{ secrets.CODECOV_TOKEN }}
|