11.0.1 release: harden failure-report sample filenames and cap decompressed report size (#895)

* Harden failure-report sample filenames and cap decompressed report size

Fixes the two open security advisories.

GHSA-c284-w5m6-jhjm (path traversal, affects 9.0.6 through 11.0.0):
save_output() named each failure report's message sample after the
sample's Subject header, falling back to the raw subject whenever
sanitizing it produced an empty string. A subject of only path
separators and dots -- "../../../" or "/" -- sanitizes to "", so the raw
value reached os.path.join() and the .eml landed outside the samples
directory or at an absolute path. That subject comes from a message that
failed authentication, so any sender a monitored mailbox accepts
controls it. The name is now sanitized at write time and falls back to
"sample", and the caller-supplied filename_safe_subject key is no longer
trusted. get_filename_safe_string() also strips NUL (which would
otherwise make open() raise ValueError: embedded null byte and hold back
the whole mailbox batch), truncates before stripping trailing
characters, and strips trailing spaces along with trailing dots, since
Windows drops both when creating a file; its docstring now states the
guarantees callers depend on.

GHSA-43qf-f35w-2x4r (unbounded decompression, affects all versions
through 11.0.0): extract_report() inflated gzip with one unbounded
zlib.decompress() and read zip members with an unbounded .read(). The
attachment's content is chosen by its sender, and deflate reaches about
1000:1 on degenerate input, so a 100 KB attachment inflated to 100 MB
with a ~209 MiB peak. Extraction now stops at
MAX_DECOMPRESSED_REPORT_SIZE (100 MiB) and raises ParserError. The gzip
path moves to a zlib.decompressobj() bounded by max_length, which does
not raise on a stream that ends early the way the one-shot call did, so
the helper checks decompressor.eof itself; a stream with trailing bytes
after the gzip member still extracts, matching the old behavior. Both
behaviors were observed against the unmodified code first and are
pinned by tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Strip NUL from CSV fields so Python 3.10's csv writer accepts them

Python 3.10's csv writer raises _csv.Error: need to escape, but no
escapechar set on any field containing NUL (CPython issue 97503, a 3.10
regression fixed in 3.11+). Failure report text fields (subject, user
agent, authentication results, addresses, etc.) come from untrusted
mail, so a NUL byte in one made parsed_failure_reports_to_csv() -- and
therefore save_output() -- raise on 3.10, which the CLI's
except (OSError, ValueError) around save_output does not catch. NUL is
now stripped from every CSV field on all Python versions via a shared
_csv_safe() helper, applied in all three CSV writers, so output is
identical regardless of interpreter version.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* 11.0.1 release: bump version and finalize changelog

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Sean Whalen
2026-09-03 15:51:44 -04:00
committed by GitHub
co-authored by Claude Fable 5.1
parent 823b0a1811
commit 09c88ca2a3
6 changed files with 465 additions and 18 deletions
+11
View File
@@ -1,5 +1,16 @@
# Changelog
## 11.0.1
### Security
- **Fixed a path traversal in the failure report sample filenames written by `--output`** ([GHSA-c284-w5m6-jhjm](https://github.com/domainaware/parsedmarc/security/advisories/GHSA-c284-w5m6-jhjm), affects 9.0.6 through 11.0.0). `save_output()` named each failure report's message sample after the sample's `Subject` header, falling back to the raw, unsanitized subject whenever sanitizing it produced an empty string. A subject consisting only of path separators and dots — `../../../` or `/` — sanitizes to nothing, so the raw value reached `os.path.join()` and the `.eml` file was written outside the `samples` directory, or at an absolute path. The subject comes from a message that failed authentication, so anyone who can send mail to a monitored mailbox controls it. The filename is now sanitized at write time and falls back to `sample`, and the `filename_safe_subject` key that a library caller may supply alongside the subject is no longer trusted.
- **Fixed unbounded decompression of report attachments** ([GHSA-43qf-f35w-2x4r](https://github.com/domainaware/parsedmarc/security/advisories/GHSA-43qf-f35w-2x4r), affects all versions through 11.0.0). `extract_report()` inflated gzip and zip attachments with no limit on the output size. The attachment's content is chosen by whoever sent it, and deflate reaches compression ratios of about 1000:1 on degenerate input, so a 100 KB attachment from any sender a monitored mailbox accepts inflated to 100 MB, with a peak of about twice that. Extraction now stops at 100 MiB of decompressed data and raises `ParserError`. The limit is deliberately not configurable: real DMARC aggregate, failure, and SMTP TLS reports are orders of magnitude smaller.
### Bug fixes
- **CSV output (`failure.csv`, and the other CSV renderers) no longer raises `_csv.Error` on Python 3.10 when a report field contains a NUL character.** Python 3.10's `csv` writer rejects any field containing NUL unless an escapechar is set ([CPython issue 97503](https://github.com/python/cpython/issues/97503), a 3.10 regression fixed in 3.11+). Failure report text fields (subject, user agent, authentication results, addresses, and more) come from untrusted mail, so a NUL byte anywhere in one made `parsed_failure_reports_to_csv()` — and therefore `save_output()` — raise on 3.10, uncaught by the CLI's `except (OSError, ValueError)` around `save_output`. The character is now stripped from CSV fields on every Python version, matching the sanitizing already applied to sample filenames.
## 11.0.0
### Changes
+85 -10
View File
@@ -72,6 +72,7 @@ from parsedmarc.types import (
from parsedmarc.utils import (
convert_outlook_msg,
get_base_domain,
get_filename_safe_string,
get_ip_address_info,
human_timestamp_to_datetime,
is_outlook_msg,
@@ -123,6 +124,17 @@ MAGIC_XML_TAG = b"\x3c" # '<' - XML starting with an element tag (no declaratio
# or "{" check, which masked it.
MAGIC_JSON = b"\x7b"
# Maximum size, in bytes, of the data extracted from one compressed report
# attachment. Real reports compress at roughly 10:1, but the attachment's
# content is chosen by whoever sent it, and deflate (used by both gzip and
# zip) reaches about 1000:1 on degenerate input, so a 100 KB attachment from
# any sender a monitored mailbox accepts could inflate to 100 MB -- and the
# decoded str coexists with the decompressed bytes, so the peak is roughly
# twice that. Deliberately not configurable: real DMARC aggregate, failure,
# and SMTP TLS reports are orders of magnitude smaller than 100 MB, so no
# legitimate deployment needs to change it.
MAX_DECOMPRESSED_REPORT_SIZE = 100 * 1024 * 1024
# Per-message count of consecutive failed saves, keyed on
# ``(reports_folder, str(message_uid))``. Populated only when
# ``get_dmarc_reports_from_mailbox()`` is given a ``save_callback`` that
@@ -764,6 +776,20 @@ def parse_smtp_tls_report_json(report: str | bytes) -> SMTPTLSReport:
raise InvalidSMTPTLSReport(str(e) + _exc_origin(e)) from e
def _csv_safe(value: Any) -> Any:
"""Strips NUL characters from a string CSV field.
Python 3.10's ``csv`` writer raises ``_csv.Error: need to escape, but
no escapechar set`` on any field containing NUL (CPython issue 97503,
fixed in 3.11). Report text fields come from untrusted mail, so the
character is dropped here, matching ``get_filename_safe_string``.
Non-string values are returned unchanged.
"""
if isinstance(value, str):
return value.replace("\x00", "")
return value
def parsed_smtp_tls_reports_to_csv_rows(
reports: SMTPTLSReport | list[SMTPTLSReport],
) -> list[dict[str, Any]]:
@@ -845,7 +871,7 @@ def parsed_smtp_tls_reports_to_csv(
rows = parsed_smtp_tls_reports_to_csv_rows(reports)
for row in rows:
writer.writerow(row)
writer.writerow({key: _csv_safe(value) for key, value in row.items()})
csv_file_object.flush()
return csv_file_object.getvalue()
@@ -1152,6 +1178,46 @@ def parse_aggregate_report_xml(
) from error
def _decompress_gzip_bounded(data: bytes) -> bytes:
"""
Decompresses a gzip stream, refusing to produce more than
``MAX_DECOMPRESSED_REPORT_SIZE`` bytes.
The limit is read from the module global on every call so that it stays
a single source of truth (and can be patched in tests).
Args:
data: The gzip stream
Returns:
bytes: The decompressed data
Raises:
ParserError: The decompressed data exceeds the limit, or the gzip
stream ends before the decompressor reaches its end marker.
zlib.error: The stream is corrupt (bad header, deflate data, or
CRC); ``extract_report()`` wraps it in ``ParserError``.
"""
limit = MAX_DECOMPRESSED_REPORT_SIZE
decompressor = zlib.decompressobj(zlib.MAX_WBITS | 16)
decompressed = decompressor.decompress(data, max_length=limit + 1)
if len(decompressed) > limit:
raise ParserError(f"Decompressed report exceeds the {limit} byte limit")
# ``flush()`` ignores ``max_length``, so it must not run until the
# bounded read above has been accepted. It is bounded here: a bounded
# ``decompress()`` that returns fewer than ``max_length`` bytes has
# either consumed all of its input or hit the end of the stream (after
# which any trailing bytes are ``unused_data``, not input), so nothing
# unbounded is left for ``flush()`` to produce.
decompressed += decompressor.flush()
if not decompressor.eof:
# A one-shot zlib.decompress() raises on a stream that ends early;
# a decompressobj just stops, so the truncation is detected here.
raise ParserError("Incomplete or truncated gzip stream")
return decompressed
def extract_report(content: bytes | str | BinaryIO) -> str:
"""
Extracts report text from zip- or gzip-compressed content, and returns
@@ -1210,9 +1276,13 @@ def extract_report(content: bytes | str | BinaryIO) -> str:
if header[: len(MAGIC_ZIP)] == MAGIC_ZIP:
_zip = zipfile.ZipFile(file_object)
report = _zip.open(_zip.namelist()[0]).read().decode(errors="ignore")
limit = MAX_DECOMPRESSED_REPORT_SIZE
member = _zip.open(_zip.namelist()[0]).read(limit + 1)
if len(member) > limit:
raise ParserError(f"Decompressed report exceeds the {limit} byte limit")
report = member.decode(errors="ignore")
elif header[: len(MAGIC_GZIP)] == MAGIC_GZIP:
report = zlib.decompress(file_object.read(), zlib.MAX_WBITS | 16).decode(
report = _decompress_gzip_bounded(file_object.read()).decode(
errors="ignore"
)
elif (
@@ -1518,7 +1588,7 @@ def parsed_aggregate_reports_to_csv(
rows = parsed_aggregate_reports_to_csv_rows(reports)
for row in rows:
writer.writerow(row)
writer.writerow({key: _csv_safe(value) for key, value in row.items()})
csv_file_object.flush()
return csv_file_object.getvalue()
@@ -1837,7 +1907,7 @@ def parsed_failure_reports_to_csv(
for row in rows:
new_row: dict[str, Any] = {}
for key in fields:
new_row[key] = row.get(key)
new_row[key] = _csv_safe(row.get(key))
csv_writer.writerow(new_row)
return csv_file.getvalue()
@@ -3382,6 +3452,15 @@ def save_output(
"""
Save report data in the given directory
The message sample of each failure report is written to a ``samples``
subdirectory, named after the sample's own ``subject`` header run
through :func:`parsedmarc.utils.get_filename_safe_string`, falling back
to ``sample`` when sanitizing leaves nothing. Since a subject arrives
from an untrusted sender, sanitizing happens here, at write time; the
``filename_safe_subject`` key a caller may supply alongside it is not
trusted and not used. Names that collide get a ``(1)``, ``(2)``, …
suffix.
Args:
results: Parsing results
output_directory (str): The path to the directory to save in
@@ -3438,11 +3517,7 @@ def save_output(
sample = failure_report["sample"]
message_count = 0
parsed_sample = failure_report["parsed_sample"]
subject = (
parsed_sample.get("filename_safe_subject")
or parsed_sample.get("subject")
or "sample"
)
subject = get_filename_safe_string(parsed_sample.get("subject")) or "sample"
filename = subject
while filename in sample_filenames:
+1 -1
View File
@@ -1,4 +1,4 @@
__version__ = "11.0.0"
__version__ = "11.0.1"
USER_AGENT = f"parsedmarc/{__version__}"
+22 -5
View File
@@ -1241,24 +1241,41 @@ def parse_email_address(original_address: str) -> dict[str, str | None]:
}
def get_filename_safe_string(string: str) -> str:
def get_filename_safe_string(string: str | None) -> str:
"""
Converts a string to a string that is safe for a filename
Converts a string to a string that is safe to use as a filename
The returned string is a single path component, never a path: it
contains no path separator (``/`` or ``\\``), no drive separator
(``:``), and no NUL byte, so it cannot be absolute, drive-relative, or
escape the directory it is joined to. It never ends in ``.`` or a
space (Windows drops both when creating a file, which would make two
distinct subjects collide on one name), and it is never ``.`` or ``..``
(both consist only of stripped characters and collapse to ``""``). It
is at most 100 characters long. It can be empty -- when the input is
empty, consists only of stripped characters, or is truncated to a run
of dots and spaces -- so callers that need a non-empty name must supply
their own fallback (e.g. ``get_filename_safe_string(subject) or
"sample"``). Windows reserved device names such as ``CON`` are not
rewritten.
Args:
string (str): A string to make safe for a filename
string (str | None): A string to make safe for a filename.
``None`` is treated as the literal string ``"None"``.
Returns:
str: A string safe for a filename
"""
invalid_filename_chars = ["\\", "/", ":", '"', "*", "?", "|", "\n", "\r"]
invalid_filename_chars = ["\\", "/", ":", '"', "*", "?", "|", "\n", "\r", "\x00"]
if string is None:
string = "None"
for char in invalid_filename_chars:
string = string.replace(char, "")
string = string.rstrip(".")
# Truncate before stripping trailing dots and spaces, so that a name cut
# off just after one cannot end in it.
string = (string[:100]) if len(string) > 100 else string
string = string.rstrip(". ")
return string
+297 -1
View File
@@ -7,6 +7,7 @@ extract_report, get_dmarc_reports_from_mbox, and the CSV / JSON renderers.
import base64
import binascii
import csv
import email
import gzip
import inspect
@@ -22,7 +23,7 @@ from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart
from email.mime.text import MIMEText
from glob import glob
from io import BytesIO
from io import BytesIO, StringIO
from pathlib import Path
from shutil import rmtree
from tempfile import NamedTemporaryFile, mkdtemp
@@ -962,6 +963,29 @@ class Test(unittest.TestCase):
)
print("Passed!")
def testSmtpTlsCsvStripsNulFromFields(self):
"""A NUL character in an SMTP TLS report text field is stripped
from CSV output instead of reaching the ``csv`` writer.
Python 3.10's ``csv`` module raises ``_csv.Error: need to escape,
but no escapechar set`` on any field containing NUL
(https://github.com/python/cpython/issues/97503, a 3.10 regression
fixed in 3.11). This test fails on unfixed code on 3.10 with that
error, and on unfixed code on 3.11+ because the NUL passes through
to the CSV text uncleaned."""
result = parsedmarc.parse_report_file(
"samples/smtp_tls/smtp_tls.json", offline=True
)
report = cast(SMTPTLSReport, result["report"])
report["organization_name"] = "Example\x00Inc."
csv_text = parsedmarc.parsed_smtp_tls_reports_to_csv(report)
self.assertNotIn("\x00", csv_text)
reader = csv.DictReader(StringIO(csv_text))
row = next(reader)
self.assertEqual(row["organization_name"], "ExampleInc.")
def testAggregateCsvExposesASNColumns(self):
"""The aggregate CSV output should include source_asn, source_as_name,
and source_as_domain columns."""
@@ -2106,6 +2130,33 @@ class Test(unittest.TestCase):
self.assertTrue(len(rows) > 0)
print("Passed!")
def testFailureReportCsvStripsNulFromFields(self):
"""A NUL character in a failure report text field is stripped from
CSV output instead of reaching the ``csv`` writer.
Python 3.10's ``csv`` module raises ``_csv.Error: need to escape,
but no escapechar set`` on any field containing NUL
(https://github.com/python/cpython/issues/97503, a 3.10 regression
fixed in 3.11). This test fails on unfixed code on 3.10 with that
error, and on unfixed code on 3.11+ because the NUL passes through
to the CSV text uncleaned."""
parsed_report = cast(
FailureReport,
parsedmarc.parse_report_file(
"samples/failure/dmarc_ruf_report_linkedin.eml", offline=True
)["report"],
)
parsed_report["parsed_sample"]["subject"] = "re\x00port"
parsed_report["user_agent"] = "Agent\x00X"
csv_text = parsedmarc.parsed_failure_reports_to_csv(parsed_report)
self.assertNotIn("\x00", csv_text)
reader = csv.DictReader(StringIO(csv_text))
row = next(reader)
self.assertEqual(row["subject"], "report")
self.assertEqual(row["user_agent"], "AgentX")
class TestExtractReport(unittest.TestCase):
"""Tests for parsedmarc.extract_report()"""
@@ -2216,6 +2267,93 @@ class TestExtractReport(unittest.TestCase):
with self.assertRaises(parsedmarc.ParserError):
parsedmarc.extract_report(cast(BinaryIO, TextStream()))
def testExtractReportGzipOverLimitRejected(self):
"""A gzip attachment that inflates past
MAX_DECOMPRESSED_REPORT_SIZE raises ParserError instead of being
decompressed.
Regression test for GHSA-43qf-f35w-2x4r: extract_report inflated
gzip with a single unbounded zlib.decompress(), so a small
attachment from any sender a monitored mailbox accepts could
allocate hundreds of MB. The limit is patched down here so the test
stays fast; the real 100 MiB cap is the same code path."""
limit = 4096
compressed = gzip.compress(b"a" * (limit + 1))
# The attack is amplification: the wire form is far below the cap.
self.assertLess(len(compressed), limit)
with patch("parsedmarc.MAX_DECOMPRESSED_REPORT_SIZE", limit):
with self.assertRaises(parsedmarc.ParserError) as ctx:
parsedmarc.extract_report(compressed)
self.assertIn(
f"Decompressed report exceeds the {limit} byte limit", str(ctx.exception)
)
def testExtractReportGzipAtLimitIsExtracted(self):
"""A gzip attachment inflating to exactly the limit is still
extracted in full, so the cap rejects only what is over it"""
limit = 4096
with patch("parsedmarc.MAX_DECOMPRESSED_REPORT_SIZE", limit):
result = parsedmarc.extract_report(gzip.compress(b"a" * limit))
self.assertEqual(result, "a" * limit)
def testExtractReportZipOverLimitRejected(self):
"""A zip attachment whose member inflates past
MAX_DECOMPRESSED_REPORT_SIZE raises ParserError.
Regression test for GHSA-43qf-f35w-2x4r, zip half: the member was
read with an unbounded .read()."""
import zipfile
limit = 4096
buf = BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("report.xml", b"a" * (limit + 1))
compressed = buf.getvalue()
self.assertLess(len(compressed), limit)
with patch("parsedmarc.MAX_DECOMPRESSED_REPORT_SIZE", limit):
with self.assertRaises(parsedmarc.ParserError) as ctx:
parsedmarc.extract_report(compressed)
self.assertIn(
f"Decompressed report exceeds the {limit} byte limit", str(ctx.exception)
)
def testExtractReportZipAtLimitIsExtracted(self):
"""A zip member inflating to exactly the limit is still extracted in
full"""
import zipfile
limit = 4096
buf = BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("report.xml", b"a" * limit)
with patch("parsedmarc.MAX_DECOMPRESSED_REPORT_SIZE", limit):
result = parsedmarc.extract_report(buf.getvalue())
self.assertEqual(result, "a" * limit)
def testExtractReportTruncatedGzipRaises(self):
"""A gzip stream cut short still raises ParserError.
Behavior parity check for the bounded-decompression change: the old
one-shot zlib.decompress() raised "Error -5 ... incomplete or
truncated stream" here, while a zlib.decompressobj() stops silently,
so the replacement has to detect the missing end-of-stream marker
itself."""
xml = b'<?xml version="1.0"?><feedback></feedback>'
with self.assertRaises(parsedmarc.ParserError) as ctx:
parsedmarc.extract_report(gzip.compress(xml)[:-8])
self.assertIn("truncated", str(ctx.exception))
def testExtractReportGzipWithTrailingBytesIsExtracted(self):
"""Bytes after the end of the gzip member are ignored, exactly as
the old one-shot zlib.decompress() ignored them.
Behavior parity check for the bounded-decompression change: verified
against the unmodified code, which returned the report text and did
not raise."""
xml = b'<?xml version="1.0"?><feedback></feedback>'
result = parsedmarc.extract_report(gzip.compress(xml) + b"trailing garbage")
self.assertEqual(result, xml.decode())
class TestMalformedXmlRecovery(unittest.TestCase):
"""Tests for XML recovery in parse_aggregate_report_xml"""
@@ -4916,5 +5054,163 @@ class TestExtractReportStreams(unittest.TestCase):
self.assertIn("<feedback>", result)
class TestSaveOutput(unittest.TestCase):
"""save_output() writes each failure report's message sample to a file
named after the sample's Subject header.
That header comes from the message that failed authentication, i.e.
from an untrusted sender, so every test here asserts on the on-disk
layout of the whole temporary root -- not just of the samples
directory -- because the defect being guarded against
(GHSA-c284-w5m6-jhjm) wrote files above it."""
def setUp(self):
self.temp_root = mkdtemp()
self.addCleanup(rmtree, self.temp_root, True)
# Nested, so a traversal out of samples/ has somewhere to land
# inside the tree the assertions walk.
self.output_directory = os.path.join(self.temp_root, "parsedmarc", "output")
def _failure_report(self, subject, **parsed_sample_extra) -> FailureReport:
"""A failure report carrying only the keys save_output() reads"""
parsed_sample = {"subject": subject}
parsed_sample.update(parsed_sample_extra)
return cast(
FailureReport,
{
"sample": f"Subject: {subject!r}\r\n\r\nThe message body.\r\n",
"parsed_sample": parsed_sample,
# Read only by the CSV renderer save_output() also calls.
"auth_failure": ["dmarc"],
"authentication_mechanisms": [],
"source": {
"ip_address": "192.0.2.1",
"reverse_dns": None,
"base_domain": None,
"name": None,
"type": None,
"asn": None,
"as_name": None,
"as_domain": None,
"country": None,
},
},
)
def _save(self, *failure_reports: FailureReport) -> None:
results = cast(
ParsingResults,
{
"aggregate_reports": [],
"failure_reports": list(failure_reports),
"smtp_tls_reports": [],
},
)
parsedmarc.save_output(results, output_directory=self.output_directory)
def _written_files(self) -> list[str]:
"""Every file under the temp root, as a path relative to it"""
written = []
for root, _dirs, files in os.walk(self.temp_root):
for name in files:
written.append(
os.path.relpath(os.path.join(root, name), self.temp_root)
)
return sorted(written)
def testTraversalSubjectsAreConfinedToTheSamplesDirectory(self):
"""Subjects made only of path separators and dots all collapse to
the "sample" fallback inside samples/, instead of escaping it.
Regression test for GHSA-c284-w5m6-jhjm: save_output() fell back to
the raw subject whenever sanitizing it produced an empty string, so
os.path.join(samples_directory, "../../../.eml") escaped three
levels up and os.path.join(samples_directory, "/.eml") became the
absolute path /.eml."""
self._save(
self._failure_report("../../../"),
self._failure_report("/"),
self._failure_report(".."),
)
samples = os.path.join("parsedmarc", "output", "samples")
eml_files = [f for f in self._written_files() if f.endswith(".eml")]
self.assertEqual(
eml_files,
[
os.path.join(samples, "sample (1).eml"),
os.path.join(samples, "sample (2).eml"),
os.path.join(samples, "sample.eml"),
],
)
# Nothing at all was written above the output directory.
self.assertTrue(
all(
f.startswith(os.path.join("parsedmarc", "output") + os.sep)
for f in self._written_files()
),
self._written_files(),
)
def testCraftedFilenameSafeSubjectIsIgnored(self):
"""The filename is derived from the subject at write time, so a
caller-supplied filename_safe_subject cannot name the file.
parse_email() derives that key with the same sanitizer, but
save_output() is public API: a library caller assembling its own
ParsingResults can put anything there, and the value used to win
over the subject."""
self._save(
self._failure_report("monthly digest", filename_safe_subject="../evil")
)
written = self._written_files()
samples = os.path.join("parsedmarc", "output", "samples")
self.assertIn(os.path.join(samples, "monthly digest.eml"), written)
self.assertFalse([f for f in written if "evil" in f], written)
def testNullByteSubjectIsStrippedAndWritten(self):
"""A NUL in the subject is stripped rather than reaching open(),
which would raise ValueError: embedded null byte and, in the CLI,
hold back the whole mailbox batch"""
self._save(self._failure_report("re\x00port"))
samples = os.path.join("parsedmarc", "output", "samples")
self.assertEqual(
[f for f in self._written_files() if f.endswith(".eml")],
[os.path.join(samples, "report.eml")],
)
def testOrdinarySubjectNamesTheFileAndSampleIsWrittenVerbatim(self):
"""An ordinary subject is unchanged by sanitizing, and the file
holds the message sample"""
report = self._failure_report("DMARC failure report")
self._save(report)
path = os.path.join(
self.output_directory, "samples", "DMARC failure report.eml"
)
# newline="" so the sample's CRLFs are not translated on the way in.
with open(path, newline="", encoding="utf-8") as sample_file:
self.assertEqual(sample_file.read(), report["sample"])
def testParsedSampleReportIsNamedFromItsSubject(self):
"""End to end: a real failure report parsed from the sample corpus
is written under its own subject line"""
report = cast(
FailureReport,
parsedmarc.parse_report_file(
"samples/failure/dmarc_ruf_report_linkedin.eml", offline=True
)["report"],
)
self._save(report)
samples = os.path.join("parsedmarc", "output", "samples")
self.assertEqual(
[f for f in self._written_files() if f.endswith(".eml")],
[os.path.join(samples, "Subject line, could be UTF8 encoded.eml")],
)
if __name__ == "__main__":
unittest.main(verbosity=2)
+49 -1
View File
@@ -348,9 +348,57 @@ class Test(unittest.TestCase):
def testGetFilenameSafeStringNone(self):
"""get_filename_safe_string with None returns 'None'"""
result = parsedmarc.utils.get_filename_safe_string(None) # type: ignore[arg-type]
result = parsedmarc.utils.get_filename_safe_string(None)
self.assertEqual(result, "None")
def testGetFilenameSafeStringTraversalSequencesCollapseToEmpty(self):
"""A subject made only of path separators and dots sanitizes to the
empty string, so a caller's `or "sample"` fallback takes over rather
than a relative path reaching os.path.join()
Regression test for GHSA-c284-w5m6-jhjm: `../../../` and `/` are the
reporter's proof-of-concept subjects."""
for traversal in ("../../../", "/", "..", ".", "..\\..\\", "./../"):
with self.subTest(traversal=traversal):
self.assertEqual(
parsedmarc.utils.get_filename_safe_string(traversal), ""
)
def testGetFilenameSafeStringStripsNullByte(self):
"""A NUL byte is stripped, so the result cannot make open() raise
ValueError: embedded null byte"""
result = parsedmarc.utils.get_filename_safe_string("re\x00port")
self.assertEqual(result, "report")
def testGetFilenameSafeStringHasNoPathSeparators(self):
"""No os.sep or os.altsep survives a mixed-separator subject"""
result = parsedmarc.utils.get_filename_safe_string("a/b\\c:d..\\..\\e/../f")
self.assertEqual(result, "abcd....e..f")
self.assertNotIn(os.sep, result)
if os.altsep is not None:
self.assertNotIn(os.altsep, result)
def testGetFilenameSafeStringTruncatedNameHasNoTrailingDot(self):
"""Truncation happens before trailing dots are stripped, so a name
cut off just after a dot does not end in one
A trailing dot is silently dropped by Windows when the file is
created, which would make two distinct subjects collide on one
filename."""
subject = f"{'a' * 99}.{'b' * 50}"
result = parsedmarc.utils.get_filename_safe_string(subject)
self.assertEqual(result, "a" * 99)
self.assertFalse(result.endswith("."))
def testGetFilenameSafeStringStripsTrailingSpaces(self):
"""Trailing spaces are stripped like trailing dots, and mixed runs of
both, because Windows drops them when creating the file"""
for subject, expected in (("report ", "report"), ("report . .", "report")):
with self.subTest(subject=subject):
self.assertEqual(
parsedmarc.utils.get_filename_safe_string(subject), expected
)
def testGetFilenameSafeStringLong(self):
"""get_filename_safe_string truncates to 100 chars"""
result = parsedmarc.utils.get_filename_safe_string("a" * 200)