mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-10-24 03:28:58 +00:00
Fix ReDoS in failure-report, schema-tag, and timestamp regexes (11.0.4) (#921)
Three regular expressions that run on attacker-controlled input backtracked super-linearly (CWE-1333): GHSA-7grx-mj6g-qxmc (affects 8.2.0 through 11.0.3, reported by mjatin2007): text_report_regex, used by the Exim/cPanel plain-text failure-report fallback, was \s*([a-zA-Z\s]+):\s(.+). The leading \s* and the \s inside the class overlap and the pattern is unanchored, so a whitespace run is retried from every offset and every split: cubic, about 8x per doubling (400 spaces 0.06 s, 1600 spaces 3.3 s on CPython 3.12). A ~100 KB message would stall the watch loop for days and be re-parsed after every restart. The pattern is now anchored to a line start with disjoint classes. GHSA-4h9r-c435-j834 (affects 4.1.7 through 11.0.3, found while auditing every regex in the package): xml_schema_regex was </??xs:schema.*>, whose greedy .*> rescans to end of line for every <xs:schema occurrence: quadratic, 1 MB in 19 s. It runs on every attachment that decodes to "<" after extract_report() has inflated it up to the 100 MiB cap, which 194 KB of gzip reaches. Now </?xs:schema[^<>]*>. This also fixes the greedy match deleting a minified single-line report from the schema tag to the last ">" on the line. human_timestamp_to_datetime() stripped RFC 5322 comments with \s*\(.*\)\s*, quadratic on whitespace (200,000 spaces 18.6 s) and reached through every Date header; filed as a bug fix because MTAs cap header size. Comments are now removed by a depth-counting scan, which is linear and handles the nested comments and quoted-pairs RFC 5322 section 3.2.2 allows (a character-class regex would have regressed them). Adds regression tests sized so the old patterns fail the time bound in minutes rather than hanging CI, a "Regular expressions on untrusted input" section in AGENTS.md, and the 11.0.4 changelog and version bump. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
1c335d951f
commit
14aa287dc1
7 files changed
+200
-8
No files matched your search
@@ -249,6 +249,52 @@ class Test(unittest.TestCase):
|
||||
"Mon, 01 Jan 2024 00:00:00 +0000 (UTC)"
|
||||
)
|
||||
self.assertEqual(dt.year, 2024)
|
||||
# A tab before the comment, and more than one comment, are stripped
|
||||
# too.
|
||||
self.assertEqual(
|
||||
parsedmarc.utils.human_timestamp_to_datetime(
|
||||
"Tue, 1 Jan 2019 00:00:00 +0000\t(GMT)"
|
||||
),
|
||||
datetime(2019, 1, 1, tzinfo=timezone.utc),
|
||||
)
|
||||
self.assertEqual(
|
||||
parsedmarc.utils.human_timestamp_to_datetime(
|
||||
"Mon, 01 Jan 2024 00:00:00 +0000 (x) (y)"
|
||||
),
|
||||
datetime(2024, 1, 1, tzinfo=timezone.utc),
|
||||
)
|
||||
# RFC 5322 section 3.2.2 lets comments nest and contain quoted-pairs;
|
||||
# a regex cannot strip those, which is why a depth-counting scan does.
|
||||
self.assertEqual(
|
||||
parsedmarc.utils.human_timestamp_to_datetime(
|
||||
"Mon, 01 Jan 2024 00:00:00 +0000 (a (b) c)"
|
||||
),
|
||||
datetime(2024, 1, 1, tzinfo=timezone.utc),
|
||||
)
|
||||
self.assertEqual(
|
||||
parsedmarc.utils.human_timestamp_to_datetime(
|
||||
"Mon, 01 Jan 2024 00:00:00 +0000 (\\) x)"
|
||||
),
|
||||
datetime(2024, 1, 1, tzinfo=timezone.utc),
|
||||
)
|
||||
|
||||
def testHumanTimestampToDatetimeWhitespaceIsLinear(self):
|
||||
"""A whitespace-padded timestamp is rejected in linear time.
|
||||
|
||||
Comments were stripped with the regex ``\\s*\\(.*\\)\\s*``, whose
|
||||
leading ``\\s*`` was retried from every position in a whitespace run
|
||||
— quadratic, so 200,000 spaces took 18.6 s. The function runs on
|
||||
every message's Date header. The 5 s bound is not flaky: the
|
||||
depth-counting scan handles this input in milliseconds, orders of
|
||||
magnitude below both the bound and the old pattern's cost. The
|
||||
comment is left unclosed so the old pattern can never match: with a
|
||||
closing parenthesis its first attempt succeeded and the backtracking
|
||||
never happened."""
|
||||
start = time.perf_counter()
|
||||
# dateutil's ParserError subclasses ValueError.
|
||||
with self.assertRaises(ValueError):
|
||||
parsedmarc.utils.human_timestamp_to_datetime(" " * 200_000 + "(x")
|
||||
self.assertLess(time.perf_counter() - start, 5)
|
||||
|
||||
def testHumanTimestampToDatetimeNegativeZero(self):
|
||||
"""-0000 timezone is handled"""
|
||||
|
||||
Reference in new issue
Block a user