adding OpenSearch support, fixing minor typos, and code styling (#481)

* adding OpenSearch support, fixing minor typos and code styling

* documentation update
This commit is contained in:
Szasza Palmer
2024-03-05 02:06:26 +11:00
committed by GitHub
parent 77132b3fc5
commit 995bdbcd97
9 changed files with 957 additions and 6 deletions

View File

@@ -14,6 +14,14 @@
:members:
```
## parsedmarc.opensearch
```{eval-rst}
.. automodule:: parsedmarc.opensearch
:members:
```
## parsedmarc.splunk
```{eval-rst}

View File

@@ -26,7 +26,7 @@ Thanks to all [contributors]!
```
`parsedmarc` is a Python module and CLI utility for parsing DMARC reports.
When used with Elasticsearch and Kibana (or Splunk), it works as a self-hosted
When used with Elasticsearch and Kibana (or Splunk), or with OpenSearch and Grafana, it works as a self-hosted
open source alternative to commercial DMARC report processing services such
as Agari Brand Protection, Dmarcian, OnDMARC, ProofPoint Email Fraud Defense,
and Valimail.
@@ -40,7 +40,7 @@ and Valimail.
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch and/or Splunk, for use with
- Optionally send the results to Elasticsearch/OpenSearch and/or Splunk, for use with
premade dashboards
- Optionally send reports to Apache Kafka
@@ -52,6 +52,7 @@ installation
usage
output
elasticsearch
opensearch
kibana
splunk
davmail

14
docs/source/opensearch.md Normal file
View File

@@ -0,0 +1,14 @@
# OpenSearch and Grafana
To set up visual dashboards of DMARC data, install OpenSearch and Grafana.
## Installation
OpenSearch: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/
Grafana: https://grafana.com/docs/grafana/latest/setup-grafana/installation/
## Records retention
Starting in version 5.0.0, `parsedmarc` stores data in a separate
index for each day to make it easy to comply with records
retention regulations such as GDPR.

View File

@@ -82,6 +82,10 @@ delete = False
hosts = 127.0.0.1:9200
ssl = False
[opensearch]
hosts = https://admin:admin@127.0.0.1:9200
ssl = True
[splunk_hec]
url = https://splunkhec.example.com
token = HECTokenGoesHere
@@ -238,6 +242,28 @@ The full set of configuration options are:
creating the index (Default: `1`)
- `number_of_replicas` - int: The number of replicas to use when
creating the index (Default: `0`)
- `opensearch`
- `hosts` - str: A comma separated list of hostnames and ports
or URLs (e.g. `127.0.0.1:9200` or
`https://user:secret@localhost`)
:::{note}
Special characters in the username or password must be
[URL encoded].
:::
- `user` - str: Basic auth username
- `password` - str: Basic auth password
- `apiKey` - str: API key
- `ssl` - bool: Use an encrypted SSL/TLS connection
(Default: `True`)
- `timeout` - float: Timeout in seconds (Default: 60)
- `cert_path` - str: Path to a trusted certificates
- `index_suffix` - str: A suffix to apply to the index names
- `monthly_indexes` - bool: Use monthly indexes instead of daily indexes
- `number_of_shards` - int: The number of shards to use when
creating the index (Default: `1`)
- `number_of_replicas` - int: The number of replicas to use when
creating the index (Default: `0`)
- `splunk_hec`
- `url` - str: The URL of the Splunk HTTP Events Collector (HEC)
- `token` - str: The HEC token