Hardening for 11.2.2: attempts file, parse timeout, worker handling, three fixes (#945)

* Wrap the failure-report duplicate search in the output error

The aggregate and SMTP TLS saves turn a failed duplicate-check search
into ElasticsearchError or OpenSearchError; the failure-report saves
let the search library's exception escape instead.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Refuse date strings too long to be dates

human_timestamp_to_datetime() had no length limit; a digit run of a
million characters took 12 s in dateutil. Add a 128 character limit
(ValueError) and map it to the matching Invalid*Report at the failure
Arrival-Date, the message Date header and the failure-report sample
Date in the Elasticsearch and OpenSearch saves.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Skip failure sample headers with an empty name

The Elasticsearch and OpenSearch failure saves normalise each header
name; a name that is empty after that is not a valid field name, so
leave the header out and save the rest.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Add a persisted store of unfinished parse attempts

A message whose parse kills the process (the out-of-memory killer, a
container memory cap) is never moved out of the reports folder, so every
later run parses it and dies again. parsedmarc.attempts keeps a small
JSON file counting, per message, the parse attempts that started but
never finished, so the mailbox loop can set such a message aside after
MAX_PARSE_ATTEMPTS. The key hashes the first 64 KiB of the message
because mailsuite's IMAP backend does not expose UIDVALIDITY, so a
reused UID must not inherit an old count. Writes are atomic and owner
only, and a failed write turns the store off instead of stopping a run.

Refs GHSA-qcc2-4x3v-v4g7, GHSA-q7gm-8cj9-94hr, GHSA-4272-4r8g-6c5h

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Set aside mailbox messages that keep killing the parse

A message whose parse ends the process stays in the reports folder, so
every one-shot run or service restart parsed it again and died before
reaching the reports behind it. The new [mailbox] attempts_file option
records each message in an AttemptStore before it is parsed and clears
the entry once the parse returns a report or raises ParserError, so an
entry left behind marks a parse that never finished; a message with
MAX_PARSE_ATTEMPTS of them is disposed of as invalid without parsing.
The entry is cleared before the save, so a failed save is never counted
as a crash, and only after a successful move, so a failed move is
retried.

With n_procs > 1 a dead run had a whole submission window in flight, so
counting every message submitted beside the culprit would set innocent
reports aside on the third run. A message with one unfinished attempt is
therefore parsed alone in the calling process after the rest of the
batch, and a second death counts against it only.

PARSEDMARC_MAILBOX_ATTEMPTS_FILE names the file directly, so the key
joins _DIRECT_FILE_KEYS; without it the variable was read as a
Docker-secret reference.

Refs GHSA-qcc2-4x3v-v4g7, GHSA-q7gm-8cj9-94hr, GHSA-4272-4r8g-6c5h

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Stop a parallel_map job that runs past a timeout

The parse limits added for GHSA-qcc2-4x3v-v4g7 bound every input known
to parse slowly, but nothing bounds a parse they do not cover: with
n_procs = 1 the rest of the mailbox waits behind it. parallel_map() now
takes parse_timeout and timeout_result. A job that runs that long gets
timeout_result(job, parse_timeout) in its place; since a
ProcessPoolExecutor cannot stop one job, every worker is killed, a new
pool is started and the other unfinished jobs are resubmitted.

A job's clock starts when its worker reports that it began, through a
queue the pool initializer hands each worker. Future.running() is not
enough: the executor marks one job more than it has workers as running,
so a job queued behind a slow neighbour would be charged for the wait.

ParseTimeoutError subclasses InvalidDMARCReport so the mbox and mailbox
loops treat a timed-out message as invalid without changes.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add parse_timeout to the mbox, mailbox and watch functions

get_dmarc_reports_from_mbox(), get_dmarc_reports_from_mailbox() and
watch_inbox() take parse_timeout and pass it to parallel_map() with the
email job's timeout value. A timeout set with n_procs = 1 also takes the
parallel_map() branch, since only a parse in a worker process can be
stopped. The mailbox function and watch_inbox() reject an invalid value
before fetching or watching, as they do for max_unsaved_retries: raised
inside a watch check, the error would be retried on every check.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add the [general] parse_timeout option

The CLI reads parse_timeout as seconds, rejects a negative or
non-numeric value as a ConfigurationError, defaults it to 0 (no limit),
and passes it to the report-file, mbox, mailbox and watch calls. The
watch call reads it from opts, so a SIGHUP reload takes a new value.
The default stays 0 because the parse limits already bound the known
inputs and a timeout routes every n_procs = 1 run through a worker
process; the documentation suggests 300 seconds.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Document the attempts_file option

Describe [mailbox] attempts_file in the option list and in a new usage
section on messages that crash parsedmarc, say in the Unsaved note that
the failed-save counts are the in-memory ones, keep the file in a state
directory in the systemd example and in a named volume in the Docker
Compose example, and add the Security changelog entry for the
retry-loop backstop the three advisories deferred.

Refs GHSA-qcc2-4x3v-v4g7, GHSA-q7gm-8cj9-94hr, GHSA-4272-4r8g-6c5h

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* List parse_timeout and attempts_file as orchestration options

ParserConfig's docstring and the usage page's ParserConfig notes name the
keyword arguments that stay outside the dataclass. The two new options
are such arguments, so a library caller looking for them on ParserConfig
is told where they go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Bound a suspect message's parse by parse_timeout

With an attempts file, a message that was being parsed when an earlier
run died is parsed alone after the rest of its batch, so that a second
death is charged to it and not to the messages parsed beside it. That
parse ran in the main process, where parse_timeout cannot stop it, so the
one message most likely to be slow or fatal was the one the timeout did
not cover. With a timeout set, each suspect now gets a worker pool of its
own, one message per pool so a crash is still attributed to it alone,
and its result is taken the same way as the batch's. Without a timeout
the suspect is still parsed in this process.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Restore default signal handlers in parsing workers

Under the fork start method a worker inherits the parent's signal
handlers, and the CLI installs SIGTERM and SIGINT handlers that only set
a flag. When a worker dies, ProcessPoolExecutor terminates the others
with SIGTERM and joins them; a worker running the inherited handler
ignores that signal, so the teardown waited forever and the run hung
instead of failing. The pool initializer now resets both signals to
their defaults. The logging tests that call the initializer in the test
process put the process's own handlers back afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Order the Unreleased changelog and tie its entries together

The three branches each added to the Unreleased entry. The GHSA-q7gm
bullet still promised a follow-up for a persisted failed-attempt marker,
which the attempts_file option now is, so it names the option. The
attempts_file bullet now says that a suspect message is parsed in a
worker of its own when parse_timeout is set, matching the code. The
#942 fix moves up beside the other fix already on master, and the
worker signal-handler fix gets its own line.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Ignore SIGINT in workers and end the batch when one dies

Resetting SIGINT to its default in the workers made a Ctrl-C, which
the terminal sends to the whole process group, kill every worker, so
the run ended in BrokenProcessPool where the CLI's handler meant to
let the batch finish. Workers now ignore SIGINT and keep the default
SIGTERM action, which ProcessPoolExecutor needs to tear down a broken
pool.

A worker that dies for another reason (the out-of-memory killer, a
crash) raised BrokenProcessPool out of parallel_map, which no caller
handled: the file and mbox paths ended in a traceback. parallel_map
now kills the remaining workers, closes the start queue and raises
ParserError from it, and the CLI file and mbox paths log it and exit 1
as the mailbox paths already did. The in-flight messages keep their
attempts-file entries, so the next run parses each alone.

The systemd example gains KillMode=mixed so stopping the service
signals only the main process and the batch in progress finishes.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* State the parse_timeout bounds the code actually gives

A slow job running beside another that expires loses its partial run
and starts again, so P slow jobs cost one more round of
parse_timeout than the docstring said, plus the bounded joins of the
killed workers, during which no heartbeat is sent. The clocks are
also checked only while the parent waits for a result, so "at most
one second past the timeout" was not true while the caller is busy
between results; the docstring, usage.md and the changelog now say a
parse is stopped within one wait slice of the parent next waiting.

ParseTimeoutError's docstring says it is the value for every timed-out
input, SMTP TLS and unrecognised ones included. A new test closes the
results iterator mid-run and checks the workers are killed and the
start queue closed; the zero-timeout test now observes which pool ran
the job instead of patching _DeadlinePool.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Correct the attempts-file wording and test the parallel test mode

Suspect isolation happens whenever the mailbox takes the parallel path,
which parse_timeout selects even with n_procs = 1; the docstring,
comment, usage.md and changelog now say so. The set-aside log said a
message "has crashed parsedmarc {count-1} time(s)", which overstates
the crashes once a failed move has added attempts; it now reports the
unfinished attempts against the cap.

usage.md lists attempts_file among the _file keys read directly from
the environment and among the settings a SIGHUP reloads, and says that
a failed write clearing an entry after a good parse can lead to a
healthy message being set aside after two such failures. The test-mode
set-aside test now runs the parallel path too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Measure a date's length after its comments are removed

The length guard in human_timestamp_to_datetime() ran before the
RFC 5322 comments were stripped. RFC 5322 section 3.2.2 puts no limit
on a comment's length, so a valid Date header with a long comment was
refused, and through the sample Date path the whole report email with
it. The check now runs on the stripped value, which is linear to
produce; a long digit run survives stripping, so it is still refused
before dateutil sees it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Save parsed reports before exiting on a dead worker

A ParserError from parallel_map (a worker process that died) or from an
mbox made the CLI exit at once, discarding every report parsed so far;
report files already moved under archive_directory were then saved
nowhere. The CLI now stops reading input, saves what it gathered, and
exits with status 1.

Workers ignore SIGINT, so the plain pool path's executor shutdown on an
early exit (KeyboardInterrupt, closing the generator) waited for the
whole submission window. Any early exit now kills the workers, as the
timed path already did.

The Ctrl-C test ran under fork only, where a worker inherits the test's
own harmless handler, so it passed without the SIGINT line. It now runs
under every available start method and signals each worker once it has
started a job.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Stop counting a message that is already set aside

A message at the attempt limit that stayed in the folder (a failed move,
test mode) was recorded again on every later run, so the logged count
grew without bound. It is now set aside without being recorded, and the
log names the stored count and the limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Describe worker death per mode and the systemd stop window

A dead worker ends a one-shot run with status 1 after saving, but in
watch mode the IMAP and Maildir watchers log it and check again, while
Graph and Gmail exit. KillMode=mixed lets the batch finish only within
TimeoutStopSec. Also say that queued messages keep attempt entries too,
and re-wrap a parse_timeout note line.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Clear the seen report IDs around the worker-death CLI tests

The new save-before-exit test parses a sample in the parent, which
records it in the shared dedup set, so a later test of the same sample
saw it as a duplicate and sent no results email.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Save file reports when a mailbox fetch ends in a ParserError

A worker that died during the one-shot mailbox fetch reached the
ParserError handler, which exited before the reports from report files
and mbox inputs were saved; files already moved under archive_directory
were then kept nowhere. The handler now saves them first. A second test
covers the input-error path: no mailbox is opened and the parsed file
stays under archive_directory.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Say what a mailbox run keeps when a worker dies

Earlier batches are saved and the batch in progress stays in the
mailbox. Also say that a set-aside message is not recorded again, so its
entry ages out after 30 days, and rewrap two paragraphs.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Say which set-aside entries age out of the attempts file

The 30-day drop applies only to an entry that stays: a failed move, or
a message left in place in test mode. The sentence read as if every
set-aside message kept an entry, contradicting the line before it.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Carry the test attempts path in the message, not the environment

test_parallel_path_records_each_attempt_before_submitting_it passed the
attempts file's path to its worker job through os.environ. A forkserver
worker forks from a server started before the patch, so it never saw the
variable and the job raised KeyError (Python 3.14's Linux default). The
path now travels in a header line of the message itself, which reaches
the worker under every start method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test the worker-side helpers and the save-failure exits

Codecov listed added lines no test reached: the timed pool's initializer
and job wrapper, which run only in workers; _kill_workers on an executor
without _processes; the plain path's heartbeat; the CLI's progress bar;
the mbox loop's stop after a file worker dies; and a failed save before
exiting, on the file/mbox path and the mailbox path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Read start tokens every round, not only while waiting

With parse_timeout set, the parent read the workers' start tokens only
inside its wait loop. When results were already done at harvest time it
never waited, so each finished job stayed in the pool's token map with
its whole input, and the map grew with the run past the submission
window. The tokens are now read at the top of every round, and close()
clears the map.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Kill the parsing workers on a second Ctrl-C

The second Ctrl-C ends parsedmarc with os._exit(130), which skips the
process pool's cleanup. Workers now ignore SIGINT, so under every start
method a running worker survived as an orphan and kept parsing, without
end when parse_timeout is off. The handler now kills the active child
processes first; Python runs it in the main thread between bytecodes,
so active_children() and Process.kill() are safe there.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Hand timed results back through files, not the pool's pipe

With parse_timeout set, a timeout kills every worker, possibly one that
is sending a large result. The executor's management thread then waits
for the rest of that message forever; the bounded join returns with the
thread alive, and because it is not a daemon the interpreter hangs at
exit. Reproduced under fork, forkserver and spawn with a 256-400 MiB
result. Workers of the timed pool now pickle each outcome, error values
included, to a file in a private directory the pool owns and return
None; every message left on a pipe is far below PIPE_BUF, which POSIX
writes atomically. The pool without a timeout never kills a running
worker and is unchanged.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Report an unwritable temporary directory as a ParserError

With parse_timeout set the results travel through a temporary
directory. If it could not be created, or a worker could not write its
result file, a raw OSError escaped parallel_map, which the CLI does not
handle. The pool now raises ParserError naming the cause; a worker that
cannot write its file returns the error text through the executor, still
a small message, and the parent raises the same ParserError.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep every hand-back failure a short ParserError

A result the worker could not pickle (PicklingError, TypeError,
RecursionError, MemoryError) escaped through the executor with a
traceback of any size, breaking the short-message guarantee the timed
pool relies on, and a result file that did not unpickle raised a raw
exception in the parent. The worker now catches any failure to write
its outcome and returns a short marker, its text cut to 200 characters
so the message stays below the 512-byte minimum PIPE_BUF; the parent
turns it, and any failure to load a result file, into a ParserError.
Also corrects the PIPE_BUF figure in the module docstring, names the
temporary directory as TMPDIR, and avoids a second 128 MiB object in a
test.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Cap the hand-back error text in bytes and survive a failing __str__

The pipe's atomic-write bound is in bytes, so a 200-character text of
multibyte characters could exceed it; the cap now applies to the UTF-8
encoding. An exception whose __str__ raises is named by its type alone,
so nothing in the worker's error path can raise through the executor. A
failed write's .tmp file is removed with the pool's result directory;
the docstring says so.

Refs GHSA-qcc2-4x3v-v4g7

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Match the pickler's message across Python versions in a test

The unpicklable-result test asserted "Can't get local object", which is
the wording of Python 3.12 and 3.13 only; 3.10, 3.11 and 3.14 say
"Can't pickle local object". Assert the part they share.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Cover the worker-side hand-back failures in the test process

The three failure branches of _run_timed_job and _take_outcome run in
worker processes in the pool tests, where coverage cannot see them;
call them directly here with an unwritable directory, an unpicklable
result and an unreadable result file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Assert the unreadable-result message by its text, and format

The temporary-directory error carries the OSError's text, not its
class name, so the test asserts "Is a directory"; one long test line
is wrapped as ruff format wants.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Narrow the marker before unpacking it in two tests

pyright cannot narrow Optional through assertIsNotNone; a plain assert
does.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

docs/source/usage.md

@@ -247,6 +247,34 @@ The full set of configuration options is:
     cache.
     :::
 
+  - `parse_timeout` - float: Stop parsing a report file or message
+      after this many seconds, and treat it as invalid: a report file
+      is logged as failed (and moved to `Invalid` under
+      `archive_directory`), an mbox message is skipped with a warning,
+      and a mailbox message is moved to the `Invalid` archive folder
+      (or deleted with `delete_invalid`). `0` means no limit
+      (Default: `0`, suggested: `300`)
+
+    :::{note}
+    With a timeout set, parsing always runs in worker processes, even
+    with `n_procs = 1`, because a parse can only be stopped by ending
+    the process that runs it. When a parse times out, every worker
+    process is stopped and new ones are started, and the other
+    messages or files that were still being parsed start again from
+    the beginning. parsedmarc checks the time only while it waits for
+    a parse result, so a parse is stopped within one second of
+    parsedmarc next waiting for a result after its time is up; while
+    parsedmarc is busy between results (moving messages, for example)
+    a parse can run on past the timeout. Choose a value well above the
+    time your largest legitimate reports take, including DNS lookups,
+    each of which can take up to `dns_timeout` seconds per try. With a
+    timeout set, the worker processes hand each parsed report back
+    through a file in a directory readable only by the parsedmarc user,
+    inside the temporary directory (`TMPDIR`), which must therefore be
+    writable; the files are removed as they are read, and their
+    directory once the batch has been parsed or the workers stopped.
+    :::
+
 - `mailbox`
   - `reports_folder` - str: The mailbox folder (or label for
       Gmail) where the incoming reports can be found
@@ -298,6 +326,16 @@ The full set of configuration options is:
       runs. See
       [Mailbox messages are only archived once the reports are saved](#mailbox-messages-are-only-archived-once-the-reports-are-saved)
       below.
+  - `attempts_file` - str: Path to a JSON file in which parsedmarc
+      counts the messages whose parsing never finished because the
+      process died, so that a message that keeps killing parsedmarc is
+      moved to the `Invalid` archive subfolder (or deleted, per
+      `delete_invalid`) after two such attempts instead of being parsed
+      on every run. The directory must be writable by the user running
+      parsedmarc; the file is created on first use. Unset by default
+      (no record is kept). See
+      [Messages that crash parsedmarc are not retried forever](#messages-that-crash-parsedmarc-are-not-retried-forever)
+      below.
   - `since` - str: Search for messages since a certain time. (Examples: `5m|3h|2d|1w`)
       Acceptable units - {"m":"minutes", "h":"hours", "d":"days", "w":"weeks"}.
       Defaults to `1d` if an incorrect value is provided.
@@ -1150,14 +1188,17 @@ from `Archive/Unsaved` back into the reports folder, or run parsedmarc once
 with `reports_folder = Archive/Unsaved`.
 
 :::{note}
-The failure counts live in memory, so they are counted per parsedmarc
-process. In watch mode — a long-running process that checks the mailbox
-repeatedly — the cap works as described across checks. A one-shot run
-(`cron`, `systemd` timers) attempts each message exactly once and then
-exits, so its counts start over every time and the default cap is never
-reached: messages simply keep being retried on every run, which is the
-safe direction. Set `max_unsaved_retries = 0` if you want one-shot runs to
-move unsavable messages to `Unsaved` immediately instead.
+The failed-save counts live in memory, so they are counted per
+parsedmarc process. In watch mode — a long-running process that checks
+the mailbox repeatedly — the cap works as described across checks. A
+one-shot run (`cron`, `systemd` timers) attempts each message exactly once
+and then exits, so its counts start over every time and the default cap is
+never reached: messages simply keep being retried on every run, which is
+the safe direction. Set `max_unsaved_retries = 0` if you want one-shot
+runs to move unsavable messages to `Unsaved` immediately instead. A
+message that crashes parsedmarc while it is being parsed is a different
+case, counted on disk by `attempts_file`; see
+[Messages that crash parsedmarc are not retried forever](#messages-that-crash-parsedmarc-are-not-retried-forever).
 :::
 
 :::{warning}
@@ -1195,6 +1236,77 @@ reports folder until it is processed by a run with a wider (or no)
 `since` window.
 :::
 
+### Messages that crash parsedmarc are not retried forever
+
+A message is only moved out of the reports folder after it has been
+parsed. If parsing a message kills the parsedmarc process — the
+out-of-memory killer, a container memory limit, a crash in a library —
+the message stays where it is, and every later run, or every restart of
+the service, fetches it again and dies on it again, so the reports behind
+it in the mailbox are never processed. parsedmarc rejects the message shapes known to
+do this, but cannot rule out new ones.
+
+Set `[mailbox]` `attempts_file` to a path to guard against that loop:
+
+```ini
+[mailbox]
+attempts_file = /var/lib/parsedmarc/attempts.json
+```
+
+parsedmarc then writes an entry to that file before it parses each
+message, and removes the entry as soon as the parse returns, whether it
+found a report or rejected the message as invalid. An entry that is still
+there on a later run belongs to a message whose parse never finished. Once
+a message has two such unfinished attempts, the next run logs an error
+naming the folder, the message and the count, and moves the message to
+`<archive_folder>/Invalid` without parsing it (or deletes it, if
+`delete_invalid` is set). Its entry is removed once the message has been
+moved, so moving it back into the reports folder tries it again from the
+start; if the move fails, the entry stays and the next run tries the move
+again. A set-aside message whose move failed, or one left in place in
+`test` mode, is not recorded again, so its entry is not refreshed and is
+dropped after 30 days, after which the message is parsed up to twice more
+before it is set aside again. Two attempts are allowed
+so that one crash for an unrelated reason (a reboot, a deployment) never
+sets a message aside. With `n_procs` above 1, or with `parse_timeout` set
+(which parses in worker processes even with `n_procs = 1`), several
+messages are parsed at once, so every message that was being parsed or
+waiting for a worker when a run died has an entry; on the next run each of
+them is parsed on its own after the rest of its batch, so a second crash
+counts against the one message that caused it. It is parsed in the main
+process, or, when `parse_timeout` is set, in a worker process of its own,
+so the timeout still applies to it. If a worker process dies while it
+parses (the out-of-memory killer can end a worker without ending
+parsedmarc), the batch ends with a logged error. A one-shot run saves the
+reports parsed from report files and mbox files before it and exits with
+status 1; for a mailbox, earlier batches are saved and the batch in
+progress stays in the mailbox. In watch mode the IMAP and Maildir watchers
+log it as a warning and check the mailbox again, so the process keeps
+running, while the Microsoft Graph and Gmail watchers exit with status 1.
+Either way the entries of the messages that were being parsed or waiting
+for a worker stay in the file, so the next run or check parses each of
+them on its own in the same way.
+
+This count is separate from the in-memory failed-save counts of
+`max_unsaved_retries` above: an entry is removed when the parse
+returns, before the reports are saved, so a message whose batch could not
+be saved is retried and moved to `Unsaved` as described above, never to
+`Invalid`. A message is identified by the reports folder, its id and a
+hash of its beginning, so a new message that reuses an old id starts with
+no attempts.
+
+Use one file per configuration (per mailbox and reports folder): two
+parsedmarc processes writing one file overwrite each other's entries. The
+file is written atomically, readable by its owner only, and entries older
+than 30 days are dropped. A file that is not a valid attempts file is
+logged as a warning and replaced. If the file cannot be written, an error
+is logged and the protection is off for the rest of that batch; reports are
+still processed. If the write that removes an entry after a successful
+parse is the one that fails, the entry stays (and the protection is off
+for the rest of that batch), so a healthy message can be set aside as
+invalid after two such failures; the write error is logged each time. In
+`test` mode messages are recorded and set aside as usual but never moved.
+
 ## Environment variable configuration
 
 Any configuration option can be set via environment variables using the
@@ -1257,11 +1369,24 @@ services:
       PARSEDMARC_IMAP_USER: dmarc@example.com
       PARSEDMARC_IMAP_PASSWORD: ${IMAP_PASSWORD}
       PARSEDMARC_MAILBOX_WATCH: "true"
+      PARSEDMARC_MAILBOX_ATTEMPTS_FILE: /home/parsedmarc/attempts.json
       PARSEDMARC_ELASTICSEARCH_HOSTS: http://elasticsearch:9200
       PARSEDMARC_GENERAL_SAVE_AGGREGATE: "true"
       PARSEDMARC_GENERAL_SAVE_FAILURE: "true"
+    volumes:
+      - parsedmarc-state:/home/parsedmarc
+
+volumes:
+  parsedmarc-state:
 ```
 
+The `parsedmarc-state` volume keeps the
+[attempts file](#messages-that-crash-parsedmarc-are-not-retried-forever)
+when the container is recreated, as it is on an image upgrade. It is
+mounted on the home directory of the image's `parsedmarc` user because
+Docker gives a new named volume the owner of the directory it is mounted
+on, and that directory is one the user can write.
+
 ### Docker secrets (`_FILE` suffix)
 
 Any `PARSEDMARC_{SECTION}_{KEY}` environment variable can also be supplied
@@ -1300,7 +1425,8 @@ services:
 
 Note that a small set of config keys whose own names already end in
 `_file` (`[general] log_file`, `[msgraph] token_file`,
-`[gmail_api] credentials_file`, `[gmail_api] token_file`) keep their
+`[gmail_api] credentials_file`, `[gmail_api] token_file`,
+`[mailbox] attempts_file`) keep their
 pre-existing meaning when set via `PARSEDMARC_..._FILE` — that env var is
 the path itself, not a wrapper around a file containing the path. To pass
 *those* paths via a Docker secret, double up the suffix
@@ -1372,9 +1498,10 @@ A few things to keep in mind:
   (IP address info, seen aggregate report IDs, and the reverse DNS map).
   Omitting `config=` falls back to the process-wide caches shared by every
   call that doesn't pass one.
-- `keep_alive` and `n_procs` are not part of `ParserConfig` — they control
-  process/worker orchestration rather than parsing or enrichment behavior,
-  so they are always passed as separate keyword arguments.
+- `keep_alive`, `n_procs`, `parse_timeout` and `attempts_file` are not part
+  of `ParserConfig` — they control process/worker or mailbox orchestration
+  rather than parsing or enrichment behavior, so they are always passed as
+  separate keyword arguments.
 
 ## Performance tuning
 
@@ -1392,6 +1519,9 @@ imports more predictable:
   trade-off is memory and DNS load: at most roughly `2 * n_procs` messages
   are held in flight at once, each worker process keeps its own DNS/GeoIP
   cache, and DNS query volume can multiply by up to `n_procs`.
+- Set `parse_timeout` (for example to `300`) so that one message or file that
+  takes far too long to parse is set aside as invalid instead of holding up
+  the rest of the run.
 - Use `mailbox.since` to process reports in smaller time windows such as `1d`,
   `7d`, or another interval that fits the backlog. This makes it easier to catch
   up incrementally instead of loading an entire mailbox history in one run.
@@ -1466,9 +1596,11 @@ After=network.target network-online.target elasticsearch.service
 ExecStart=/opt/parsedmarc/venv/bin/parsedmarc -c /etc/parsedmarc.ini
 ExecReload=/bin/kill -HUP $MAINPID
 KillSignal=SIGTERM
+KillMode=mixed
 TimeoutStopSec=60
 User=parsedmarc
 Group=parsedmarc
+StateDirectory=parsedmarc
 Restart=always
 RestartSec=5m
 
@@ -1476,6 +1608,23 @@ RestartSec=5m
 WantedBy=multi-user.target
 ```
 
+`KillMode=mixed` sends the stop signal to the main parsedmarc process
+only, so the batch in progress can finish before it exits, within
+`TimeoutStopSec`, after which systemd kills the whole group; the default
+control-group mode would also stop its parsing worker processes at once
+and abandon the batch.
+
+`StateDirectory=parsedmarc` makes systemd create `/var/lib/parsedmarc`,
+owned by the `parsedmarc` user, before the service starts. Point the
+[attempts file](#messages-that-crash-parsedmarc-are-not-retried-forever)
+there in `/etc/parsedmarc.ini`, so a message that kills the service is
+not parsed again on every restart:
+
+```ini
+[mailbox]
+attempts_file = /var/lib/parsedmarc/attempts.json
+```
+
 Then, enable the service
 
 ```bash
@@ -1533,7 +1682,7 @@ for that batch have completed. The following settings are reloaded:
 - DNS and GeoIP settings (`nameservers`, `dns_timeout`, `ip_db_path`,
   `ipinfo_url`, `offline`, etc.)
 - Processing flags (`strip_attachment_payloads`, `batch_size`,
-  `check_timeout`, etc.)
+  `check_timeout`, `parse_timeout`, `attempts_file`, etc.)
 - Log level (`debug`, `verbose`, `warnings`, `silent`)
 
 Mailbox connection settings (IMAP host/credentials, Microsoft Graph,