From 597b13add2497edba40937d61654c82a3fd71bca Mon Sep 17 00:00:00 2001
From: Sean Whalen
Date: Sat, 10 Sep 2022 14:59:16 -0400
Subject: [PATCH] Update docs
---
_sources/index.md.txt | 68 +++++++++++-----------
index.html | 130 +++++++++++++++---------------------------
2 files changed, 82 insertions(+), 116 deletions(-)
diff --git a/_sources/index.md.txt b/_sources/index.md.txt
index 36ab5ba..6a8e7f7 100644
--- a/_sources/index.md.txt
+++ b/_sources/index.md.txt
@@ -159,7 +159,7 @@ port = 514
The full set of configuration options are:
- `general`
- : - `save_aggregate` - bool: Save aggregate report data to
+ - `save_aggregate` - bool: Save aggregate report data to
Elasticsearch, Splunk and/or S3
- `save_forensic` - bool: Save forensic report data to
Elasticsearch, Splunk and/or S3
@@ -190,23 +190,24 @@ The full set of configuration options are:
performance when processing thousands of files
:::
- `mailbox`
- : - `reports_folder` - str: The mailbox folder (or label for
- Gmail) where the incoming reports can be found (Default: INBOX)
+ - `reports_folder` - str: The mailbox folder (or label for
+ Gmail) where the incoming reports can be found
+ (Default: `INBOX`)
- `archive_folder` - str: The mailbox folder (or label for
- Gmail) to sort processed emails into (Default: Archive)
+ Gmail) to sort processed emails into (Default: `Archive`)
- `watch` - bool: Use the IMAP `IDLE` command to process
- messages as they arrive or poll MS Graph for new messages
- `delete` - bool: Delete messages after processing them,
- instead of archiving them
- `test` - bool: Do not move or delete messages
- `batch_size` - int: Number of messages to read and process
- before saving. Default 10. Use 0 for no limit.
+ before saving. Default `10`. Use `0` for no limit.
- `check_timeout` - int: Number of seconds to wait for a IMAP
IDLE response or the number of seconds until the next mai
- check (Default: 30)
+ check (Default: `30`)
- `imap`
- : - `host` - str: The IMAP server hostname or IP address
- - `port` - int: The IMAP server port (Default: 993)
+ - `host` - str: The IMAP server hostname or IP address
+ - `port` - int: The IMAP server port (Default: `993`)
:::{note}
`%` characters must be escaped with another `%` character,
@@ -229,9 +230,9 @@ The full set of configuration options are:
- `user` - str: The IMAP user
- `password` - str: The IMAP password
- `msgraph`
- : - `auth_method` - str: Authentication method, valid types are
- UsernamePassword, DeviceCode, or ClientSecret
- (Default: UsernamePassword).
+ - `auth_method` - str: Authentication method, valid types are
+ `UsernamePassword`, `DeviceCode`, or `ClientSecret`
+ (Default: `UsernamePassword`).
- `user` - str: The M365 user, required when the auth method is
UsernamePassword
- `password` - str: The user password, required when the auth
@@ -244,7 +245,7 @@ The full set of configuration options are:
current user if using the UsernamePassword auth method, but
could be a shared mailbox if the user has access to the mailbox
- `token_file` - str: Path to save the token file
- (Default: .token)
+ (Default: `.token`)
:::{note}
You must create an app registration in Azure AD and have an
@@ -272,7 +273,7 @@ The full set of configuration options are:
:::
- `elasticsearch`
- : - `hosts` - str: A comma separated list of hostnames and ports
+ - `hosts` - str: A comma separated list of hostnames and ports
or URLs (e.g. `127.0.0.1:9200` or
`https://user:secret@localhost`)
@@ -281,66 +282,70 @@ The full set of configuration options are:
[URL encoded].
:::
- - `ssl` - bool: Use an encrypted SSL/TLS connection (Default: True)
+ - `ssl` - bool: Use an encrypted SSL/TLS connection
+ (Default: `True`)
- `cert_path` - str: Path to a trusted certificates
- `index_suffix` - str: A suffix to apply to the index names
- `monthly_indexes` - bool: Use monthly indexes instead of daily indexes
- - `number_of_shards` - int: The number of shards to use when creating the index (Default: 1)
- - `number_of_replicas` - int: The number of replicas to use when creating the index (Default: 1)
+ - `number_of_shards` - int: The number of shards to use when
+ creating the index (Default: `1`)
+ - `number_of_replicas` - int: The number of replicas to use when
+ creating the index (Default: `1`)
- `splunk_hec`
- : - `url` - str: The URL of the Splunk HTTP Events Collector (HEC)
+ - `url` - str: The URL of the Splunk HTTP Events Collector (HEC)
- `token` - str: The HEC token
- `index` - str: The Splunk index to use
- `skip_certificate_verification` - bool: Skip certificate
- verification (not recommended)
+ verification (not recommended)
- `kafka`
- : - `hosts` - str: A comma separated list of Kafka hosts
+ - `hosts` - str: A comma separated list of Kafka hosts
- `user` - str: The Kafka user
- `passsword` - str: The Kafka password
- `ssl` - bool: Use an encrypted SSL/TLS connection (Default: True)
- `skip_certificate_verification` - bool: Skip certificate
- verification (not recommended)
+ verification (not recommended)
- `aggregate_topic` - str: The Kafka topic for aggregate reports
- `forensic_topic` - str: The Kafka topic for forensic reports
- `smtp`
- : - `host` - str: The SMTP hostname
+ - `host` - str: The SMTP hostname
- `port` - int: The SMTP port (Default: 25)
- `ssl` - bool: Require SSL/TLS instead of using STARTTLS
- `skip_certificate_verification` - bool: Skip certificate
- verification (not recommended)
+ verification (not recommended)
- `user` - str: the SMTP username
- `password` - str: the SMTP password
- `from` - str: The From header to use in the email
- `to` - list: A list of email addresses to send to
- `subject` - str: The Subject header to use in the email
- (Default: parsedmarc report)
+ (Default: `parsedmarc report`)
- `attachment` - str: The ZIP attachment filenames
- `message` - str: The email message
- (Default: Please see the attached parsedmarc report.)
+ (Default: `Please see the attached parsedmarc report.`)
:::{note}
`%` characters must be escaped with another `%` character,
so use `%%` wherever a `%` character is used.
:::
- `s3`
- : - `bucket` - str: The S3 bucket name
+ - `bucket` - str: The S3 bucket name
- `path` - str: The path to upload reports to (Default: /)
- `region_name` - str: The region name (Optional)
- `endpoint_url` - str: The endpoint URL (Optional)
- `access_key_id` - str: The access key id (Optional)
- `secret_access_key` - str: The secret access key (Optional)
- `syslog`
- : - `server` - str: The Syslog server name or IP address
+ - `server` - str: The Syslog server name or IP address
- `port` - int: The UDP port to use (Default: 514)
- `gmail_api`
- : - `credentials_file` - str: Path to file containing the
+ - `credentials_file` - str: Path to file containing the
credentials, None to disable (Default: None)
- `token_file` - str: Path to save the token file
(Default: .token)
- `include_spam_trash` - bool: Include messages in Spam and
Trash when searching reports (Default: False)
- `scopes` - str: Comma separated list of scopes to use when
- acquiring credentials (Default: )
+ acquiring credentials
+ (Default: `https://www.googleapis.com/auth/gmail.modify`)
- `oauth2_port` - int: The TCP port for the local server to
listen on for the OAuth2 response (Default: 8080)
@@ -382,9 +387,9 @@ known samples you want to save to that folder
## Sample aggregate report output
-Here are the results from parsing the [example](https://dmarc.org/wiki/FAQ#I_need_to_implement_aggregate_reports.2C_what_do_they_look_like.3F)
-report from the dmarc.org wiki. It's actually an older draft of the the 1.0
-report schema standardized in
+Here are the results from parsing the[example](https://dmarc.org/wiki/FAQ#I_need_to_implement_aggregate_reports.2C_what_do_they_look_like.3F)
+report from the dmarc.org wiki. It's actually an older draft of
+the 1.0 report schema standardized in
[RFC 7480 Appendix C](https://tools.ietf.org/html/rfc7489#appendix-C).
This draft schema is still in wide use.
@@ -1675,7 +1680,6 @@ Some additional steps are needed for Linux hosts.
[maxmind geoipupdate page]: https://dev.maxmind.com/geoip/geoipupdate/
[maxmind geolite2 country database]: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
[modern auth/multi-factor authentication]: http://davmail.sourceforge.net/faq.html
-[pypy3]: https://www.pypy.org/download.html
[readonlyrest]: https://readonlyrest.com/
[registering for a free geolite2 account]: https://www.maxmind.com/en/geolite2/signup
[rfc 2369]: https://tools.ietf.org/html/rfc2369
diff --git a/index.html b/index.html
index a191b31..9fd4f1d 100644
--- a/index.html
+++ b/index.html
@@ -287,14 +287,10 @@ configuration file, described below.
The full set of configuration options are:
-
-general
-
-
save_forensic - bool: Save forensic report data to
Elasticsearch, Splunk and/or S3
strip_attachment_payloads - bool: Remove attachment
@@ -326,36 +322,29 @@ performance when processing thousands of files
-
-mailbox-
-
-
+mailbox
+reports_folder - str: The mailbox folder (or label for
+Gmail) where the incoming reports can be found
+(Default: INBOX)
archive_folder - str: The mailbox folder (or label for
-Gmail) to sort processed emails into (Default: Archive)
+Gmail) to sort processed emails into (Default: Archive)
watch - bool: Use the IMAP IDLE command to process
messages as they arrive or poll MS Graph for new messages
delete - bool: Delete messages after processing them,
instead of archiving them
test - bool: Do not move or delete messages
batch_size - int: Number of messages to read and process
-before saving. Default 10. Use 0 for no limit.
+before saving. Default 10. Use 0 for no limit.
check_timeout - int: Number of seconds to wait for a IMAP
IDLE response or the number of seconds until the next mai
-check (Default: 30)
+check (Default: 30)
-
-imap-
-
-
+imap
-port - int: The IMAP server port (Default: 993)
+host - str: The IMAP server hostname or IP address
+port - int: The IMAP server port (Default: 993)
Note
% characters must be escaped with another % character,
@@ -379,15 +368,11 @@ verification (not recommended)
password - str: The IMAP password
-
-msgraph
-auth_method - str: Authentication method, valid types are
-UsernamePassword, DeviceCode, or ClientSecret
-(Default: UsernamePassword).
-
-
-
+msgraph
+auth_method - str: Authentication method, valid types are
+UsernamePassword, DeviceCode, or ClientSecret
+(Default: UsernamePassword).
user - str: The M365 user, required when the auth method is
UsernamePassword
password - str: The user password, required when the auth
@@ -400,7 +385,7 @@ for all auth methods except UsernamePassword.
current user if using the UsernamePassword auth method, but
could be a shared mailbox if the user has access to the mailbox
token_file - str: Path to save the token file
-(Default: .token)
+(Default: .token)
Note
You must create an app registration in Azure AD and have an
@@ -428,48 +413,40 @@ group and use that as the group id.
-
-elasticsearch
+elasticsearch
+
+https://user:secret@localhost)
Note
Special characters in the username or password must be
URL encoded.
-
-
-
+
ssl - bool: Use an encrypted SSL/TLS connection
+(Default: True)
cert_path - str: Path to a trusted certificates
index_suffix - str: A suffix to apply to the index names
monthly_indexes - bool: Use monthly indexes instead of daily indexes
-
number_of_shards - int: The number of shards to use when creating the index (Default: 1)
-
number_of_replicas - int: The number of replicas to use when creating the index (Default: 1)
+
number_of_shards - int: The number of shards to use when
+creating the index (Default: 1)
+
number_of_replicas - int: The number of replicas to use when
+creating the index (Default: 1)
-
-splunk_hec-
-
-
+splunk_hec
+url - str: The URL of the Splunk HTTP Events Collector (HEC)
token - str: The HEC token
index - str: The Splunk index to use
skip_certificate_verification - bool: Skip certificate
verification (not recommended)
-
-kafka-
-
-
+kafka
+hosts - str: A comma separated list of Kafka hosts
user - str: The Kafka user
passsword - str: The Kafka password
ssl - bool: Use an encrypted SSL/TLS connection (Default: True)
@@ -479,13 +456,9 @@ verification (not recommended)
forensic_topic - str: The Kafka topic for forensic reports
-
-smtp-
-
-
+smtp
+host - str: The SMTP hostname
port - int: The SMTP port (Default: 25)
ssl - bool: Require SSL/TLS instead of using STARTTLS
skip_certificate_verification - bool: Skip certificate
@@ -495,10 +468,10 @@ verification (not recommended)
from - str: The From header to use in the email
to - list: A list of email addresses to send to
subject - str: The Subject header to use in the email
-(Default: parsedmarc report)
+(Default: parsedmarc report)
attachment - str: The ZIP attachment filenames
message - str: The email message
-(Default: Please see the attached parsedmarc report.)
+(Default: Please see the attached parsedmarc report.)
Note
% characters must be escaped with another % character,
@@ -507,13 +480,9 @@ so use %%
-
-s3-
-
-
+s3
+bucket - str: The S3 bucket name
path - str: The path to upload reports to (Default: /)
region_name - str: The region name (Optional)
endpoint_url - str: The endpoint URL (Optional)
@@ -521,30 +490,23 @@ so use %%secret_access_key - str: The secret access key (Optional)
-
-syslog-
-
-
+syslog
-
-gmail_api
-
-
token_file - str: Path to save the token file
(Default: .token)
include_spam_trash - bool: Include messages in Spam and
Trash when searching reports (Default: False)
scopes - str: Comma separated list of scopes to use when
-acquiring credentials (Default: https://www.googleapis.com/auth/gmail.modify)
+acquiring credentials
+(Default: https://www.googleapis.com/auth/gmail.modify)
oauth2_port - int: The TCP port for the local server to
listen on for the OAuth2 response (Default: 8080)
@@ -584,9 +546,9 @@ known samples you want to save to that folder
Sample aggregate report output
-Here are the results from parsing the example
-report from the dmarc.org wiki. It’s actually an older draft of the the 1.0
-report schema standardized in
+
Here are the results from parsing theexample
+report from the dmarc.org wiki. It’s actually an older draft of
+the 1.0 report schema standardized in
RFC 7480 Appendix C.
This draft schema is still in wide use.
parsedmarc produces consistent, normalized output, regardless