mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-11-06 16:59:23 +00:00
Sean WhalenandClaude Opus 5.5
56e5a09923
* Add opt-in Elasticsearch/OpenSearch data streams (#842) New use_data_streams option in [elasticsearch] and [opensearch] saves each report type to a data stream named like today's index without the date suffix, instead of daily or monthly indexes. parsedmarc puts an exact-name index template (priority 501, @timestamp mapped as date, <stream>@custom component template on Elasticsearch) and creates the stream; retention is left to operator-managed ILM/ISM policies, with documented examples. Dated indexes are untouched and still searched for duplicates and matched by the dashboards. Every document now carries @timestamp and is saved with op_type=create in both modes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix data stream prose found by cross-check review (layer 1) Aggregate @timestamp is each record's begin date, not the report's; the cluster only rolls backing indices over under an attached policy; move the Elasticsearch 8.7 note after the template it refers to; clarify the name-collision paragraph, OpenSearch-side comments, the migrate_indexes docstring, and the test-module headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tighten names and wiring tests found by cross-check review (layer 2) Split the use_data_streams CLI passthrough test per backend so a call site reading the other section's flag fails; name the dated-mode tests for both changes they assert; compute index_date only in dated mode in every save function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Align data stream docs with Elastic/OpenSearch references (layer 3) Say the stream must exist before reindexing into it (ES reindex API: auto-creation needs a data-stream template); an existing stream's write index needs the ILM policy attached directly since nothing rolls it over otherwise; target the stream name in the _settings example; note OpenSearch's leading-dot rule for stream names and Fleet's template priorities. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cover the Loki client-initialization error wrapper _parse_config accepts a [loki] token alongside username/password; only the LokiClient constructor rejects the combination. Exercise that path through _init_output_clients with the real constructor so the RuntimeError("Loki: ...") wrapper Codecov flagged as uncovered is tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>