Fix ReDoS in failure-report, schema-tag, and timestamp regexes (11.0.4) (#921)

Three regular expressions that run on attacker-controlled input
backtracked super-linearly (CWE-1333):

GHSA-7grx-mj6g-qxmc (affects 8.2.0 through 11.0.3, reported by
mjatin2007): text_report_regex, used by the Exim/cPanel plain-text
failure-report fallback, was \s*([a-zA-Z\s]+):\s(.+). The leading \s*
and the \s inside the class overlap and the pattern is unanchored, so a
whitespace run is retried from every offset and every split: cubic,
about 8x per doubling (400 spaces 0.06 s, 1600 spaces 3.3 s on CPython
3.12). A ~100 KB message would stall the watch loop for days and be
re-parsed after every restart. The pattern is now anchored to a line
start with disjoint classes.

GHSA-4h9r-c435-j834 (affects 4.1.7 through 11.0.3, found while auditing
every regex in the package): xml_schema_regex was </??xs:schema.*>,
whose greedy .*> rescans to end of line for every <xs:schema
occurrence: quadratic, 1 MB in 19 s. It runs on every attachment that
decodes to "<" after extract_report() has inflated it up to the 100 MiB
cap, which 194 KB of gzip reaches. Now </?xs:schema[^<>]*>. This also
fixes the greedy match deleting a minified single-line report from the
schema tag to the last ">" on the line.

human_timestamp_to_datetime() stripped RFC 5322 comments with
\s*\(.*\)\s*, quadratic on whitespace (200,000 spaces 18.6 s) and
reached through every Date header; filed as a bug fix because MTAs cap
header size. Comments are now removed by a depth-counting scan, which is
linear and handles the nested comments and quoted-pairs RFC 5322
section 3.2.2 allows (a character-class regex would have regressed
them).

Adds regression tests sized so the old patterns fail the time bound in
minutes rather than hanging CI, a "Regular expressions on untrusted
input" section in AGENTS.md, and the 11.0.4 changelog and version bump.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Sean WhalenandClaude Fable 5.1 authored and GitHub committed 2026-10-18 12:01:32 -04:00
1 parent 1c335d951f
commit 14aa287dc1
7 files changed
+200 -8

No files matched your search

+46
View File
@@ -249,6 +249,52 @@ class Test(unittest.TestCase):
"Mon, 01 Jan 2024 00:00:00 +0000 (UTC)"
)
self.assertEqual(dt.year, 2024)
# A tab before the comment, and more than one comment, are stripped
# too.
self.assertEqual(
parsedmarc.utils.human_timestamp_to_datetime(
"Tue, 1 Jan 2019 00:00:00 +0000\t(GMT)"
),
datetime(2019, 1, 1, tzinfo=timezone.utc),
)
self.assertEqual(
parsedmarc.utils.human_timestamp_to_datetime(
"Mon, 01 Jan 2024 00:00:00 +0000 (x) (y)"
),
datetime(2024, 1, 1, tzinfo=timezone.utc),
)
# RFC 5322 section 3.2.2 lets comments nest and contain quoted-pairs;
# a regex cannot strip those, which is why a depth-counting scan does.
self.assertEqual(
parsedmarc.utils.human_timestamp_to_datetime(
"Mon, 01 Jan 2024 00:00:00 +0000 (a (b) c)"
),
datetime(2024, 1, 1, tzinfo=timezone.utc),
)
self.assertEqual(
parsedmarc.utils.human_timestamp_to_datetime(
"Mon, 01 Jan 2024 00:00:00 +0000 (\\) x)"
),
datetime(2024, 1, 1, tzinfo=timezone.utc),
)
def testHumanTimestampToDatetimeWhitespaceIsLinear(self):
"""A whitespace-padded timestamp is rejected in linear time.
Comments were stripped with the regex ``\\s*\\(.*\\)\\s*``, whose
leading ``\\s*`` was retried from every position in a whitespace run
— quadratic, so 200,000 spaces took 18.6 s. The function runs on
every message's Date header. The 5 s bound is not flaky: the
depth-counting scan handles this input in milliseconds, orders of
magnitude below both the bound and the old pattern's cost. The
comment is left unclosed so the old pattern can never match: with a
closing parenthesis its first attempt succeeded and the backtracking
never happened."""
start = time.perf_counter()
# dateutil's ParserError subclasses ValueError.
with self.assertRaises(ValueError):
parsedmarc.utils.human_timestamp_to_datetime(" " * 200_000 + "(x")
self.assertLess(time.perf_counter() - start, 5)
def testHumanTimestampToDatetimeNegativeZero(self):
"""-0000 timezone is handled"""