Add opt-in Elasticsearch/OpenSearch data streams (#937)

* Add opt-in Elasticsearch/OpenSearch data streams (#842)

New use_data_streams option in [elasticsearch] and [opensearch] saves
each report type to a data stream named like today's index without the
date suffix, instead of daily or monthly indexes. parsedmarc puts an
exact-name index template (priority 501, @timestamp mapped as date,
<stream>@custom component template on Elasticsearch) and creates the
stream; retention is left to operator-managed ILM/ISM policies, with
documented examples. Dated indexes are untouched and still searched for
duplicates and matched by the dashboards. Every document now carries
@timestamp and is saved with op_type=create in both modes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix data stream prose found by cross-check review (layer 1)

Aggregate @timestamp is each record's begin date, not the report's; the
cluster only rolls backing indices over under an attached policy; move
the Elasticsearch 8.7 note after the template it refers to; clarify the
name-collision paragraph, OpenSearch-side comments, the migrate_indexes
docstring, and the test-module headers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Tighten names and wiring tests found by cross-check review (layer 2)

Split the use_data_streams CLI passthrough test per backend so a call
site reading the other section's flag fails; name the dated-mode tests
for both changes they assert; compute index_date only in dated mode in
every save function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Align data stream docs with Elastic/OpenSearch references (layer 3)

Say the stream must exist before reindexing into it (ES reindex API:
auto-creation needs a data-stream template); an existing stream's write
index needs the ILM policy attached directly since nothing rolls it over
otherwise; target the stream name in the _settings example; note
OpenSearch's leading-dot rule for stream names and Fleet's template
priorities.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Cover the Loki client-initialization error wrapper

_parse_config accepts a [loki] token alongside username/password; only
the LokiClient constructor rejects the combination. Exercise that path
through _init_output_clients with the real constructor so the
RuntimeError("Loki: ...") wrapper Codecov flagged as uncovered is tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

docs/source/usage.md

@@ -591,16 +591,27 @@ The full set of configuration options is:
   - `index_suffix` - str: A suffix to apply to the index names
   - `index_prefix` - str: A prefix to apply to the index names
   - `monthly_indexes` - bool: Use monthly indexes instead of daily indexes
+    (ignored, with a warning, when `use_data_streams` is `True`)
   - `number_of_shards` - int: The number of shards to use when
-    creating the index (Default: `1`)
+    creating the index, or each backing index of a data stream
+    (Default: `1`)
   - `number_of_replicas` - int: The number of replicas to use when
-    creating the index (Default: `0`)
+    creating the index, or each backing index of a data stream
+    (Default: `0`)
+  - `use_data_streams` - bool: Save each report type to a data stream
+    named like the index without the date suffix (e.g. `dmarc_aggregate`)
+    instead of to daily or monthly indexes. parsedmarc creates the stream
+    and its index template; retention is left to an ILM policy you attach. Requires Elasticsearch
+    8.7 or later.
+    See [Data streams and records retention](elasticsearch.md#data-streams-and-records-retention)
+    (Default: `False`)
   - `serverless` - bool: Set to `True` when targeting an Elastic Cloud
     Serverless project. Serverless manages sharding and replication itself
     and rejects the `number_of_shards` / `number_of_replicas` index settings
     with HTTP 400. With this flag set, parsedmarc strips those keys from the
-    settings sent at index creation; any other settings (e.g.
-    `refresh_interval`) are passed through unchanged (Default: `False`)
+    settings sent at index creation (or in a data stream's index template);
+    any other settings (e.g. `refresh_interval`) are passed through
+    unchanged (Default: `False`)
 - `opensearch`
 
   Requires the `opensearch` extra: `pip install "parsedmarc[opensearch]"`
@@ -629,10 +640,21 @@ The full set of configuration options is:
   - `index_suffix` - str: A suffix to apply to the index names
   - `index_prefix` - str: A prefix to apply to the index names
   - `monthly_indexes` - bool: Use monthly indexes instead of daily indexes
+    (ignored, with a warning, when `use_data_streams` is `True`)
   - `number_of_shards` - int: The number of shards to use when
-    creating the index (Default: `1`)
+    creating the index, or each backing index of a data stream
+    (Default: `1`)
   - `number_of_replicas` - int: The number of replicas to use when
-    creating the index (Default: `0`)
+    creating the index, or each backing index of a data stream
+    (Default: `0`)
+  - `use_data_streams` - bool: Save each report type to a data stream
+    named like the index without the date suffix (e.g. `dmarc_aggregate`)
+    instead of to daily or monthly indexes. parsedmarc creates the stream
+    and its index template; retention is left to an ISM policy you attach.
+    OpenSearch rejects data stream names that start with `.`, so an
+    `index_prefix` starting with `.` cannot be used with this option.
+    See [Data streams and records retention](opensearch.md#data-streams-and-records-retention)
+    (Default: `False`)
 - `splunk_hec`
   - `url` - str: The URL of the Splunk HTTP Events Collector (HEC)
   - `token` - str: The HEC token