mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-11-11 03:11:33 +00:00
Reject reports dated in the future and bound what parse_report_file() reads (#954)
* Reject reports whose index-naming date is more than two days in the future (#947) The date that names a report's dated Elasticsearch/OpenSearch index (an aggregate report's begin date, a failure report's Arrival-Date, an SMTP TLS report's start-datetime) was never compared with the present, so a report dated 2100 named an index for 2100. Each parser now rejects such a report with its own Invalid*Report; past dates stay unbounded. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Refuse an input larger than the report size cap before parse_report_file() reads it (#949) The 32 MiB cap was applied only after the whole input was read. A path is now checked with os.path.getsize(), bytes-like input with len(), and a file object is read at most one byte past the cap. The error is the ParserError with the message the content check already gives. Three existing tests that fed over-cap content now see the earlier rejection. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Measure a memoryview input by its byte size len() of a memoryview counts items, which are not bytes for every format, so the size cap in parse_report_file() compares nbytes instead. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * Bound the aggregate end date and read file objects to their end Address the Copilot review of #954. A normalized report's per-day records name their indexes from their own interval starts, so the report's end date is bounded like its begin date. A raw stream may return fewer bytes than asked for before its end, so parse_report_file() now reads in a loop until end of file or one byte past the cap. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * Accumulate bounded reads in one buffer Address the second Copilot review of #954. Keeping every chunk in a list cost one bytes object per read, about 85 bytes per byte for a stream that returns a byte at a time (22.7 MiB for 256 KiB of input). One bytearray keeps memory proportional to the bytes read. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
fa905e41fd
commit
d9193eafea
6 files changed
+339
-12
No files matched your search
@@ -420,6 +420,10 @@ index with their own dates, and a warning is logged. Dated index names still com
|
||||
from each report's own dates, so reports with many different dates
|
||||
still create one index per date; data streams (`use_data_streams`) are
|
||||
affected by neither.
|
||||
A report whose begin or end date (aggregate), begin date (SMTP TLS) or
|
||||
`Arrival-Date` (failure) is more than two days in the future is rejected
|
||||
as invalid, because those dates name the dated indexes; past dates are
|
||||
not bounded.
|
||||
|
||||
Set `use_data_streams = True` in the `[elasticsearch]` section to write
|
||||
to [data streams] instead. Each report type then goes to one data stream
|
||||
|
||||
Reference in new issue
Block a user