Commit Graph

1026 Commits

Author SHA1 Message Date
Sean Whalen
722ff79e23 Move senders out of the package
until the database is more complete
2023-10-11 17:36:13 -04:00
Sean Whalen
2bee4fc591 Fix README typos 2023-09-05 21:17:04 -04:00
Sean Whalen
4d0974948d Update senders.sqlite 2023-09-05 21:10:12 -04:00
Sean Whalen
8b5834b00d Fix flake8 E721 2023-09-05 18:12:50 -04:00
Sean Whalen
31db7d2301 Add senders.sqlite 2023-09-05 15:15:30 -04:00
Sean Whalen
26027ef6b3 Add senders.sqlite 2023-09-05 15:14:11 -04:00
Sean Whalen
4ff44dcb0f Update dbip-country-lite.mmdb 2023-09-05 06:43:31 -04:00
Sean Whalen
557e2e0554 Code style fixes 2023-08-01 16:23:31 -04:00
Sean Whalen
6c84cfb7c4 Fix output in tests.py 2023-06-27 15:56:33 -04:00
Sean Whalen
a4b0aabdfb Update CHANGELOG.md 2023-06-27 15:54:21 -04:00
andersnauman
51760181b0 Fix: Less error-prone code to minimize the risk of program crash (#417)
- Double check if org_name exist. Empty name will crash Elastissearch's 'exist' search
- Move try-statement to include open() to catch if files do not exist
- Enclose Elasticsearch's execute in a try-statement to catch any invalid searches when variables are empty/missing
2023-06-27 15:45:40 -04:00
Sean Whalen
89872d78ac Fix testing 2023-06-27 15:43:09 -04:00
Sean Whalen
477a45d19b Acutally bump the version to 8.6.1 2023-05-14 20:43:42 -04:00
Sean Whalen
e5e7a6fe75 8.6.1
- Fix handling of non-domain organization names (PR #411 fixes issue #410)
- Ignore rua reports with a date range that is too long to be valid (PR #408 fixes issue #282)
2023-05-14 20:41:44 -04:00
Gaige B Paulsen
5a659ea578 fix: handling of text org_name without space #410 (#411) 2023-05-14 20:29:12 -04:00
Anael Mobilia
d2d62488f7 Add contributors on copyright (#414)
Co-authored-by: Anael Mobilia <anael.mobilia@mydsomanager.com>
2023-05-14 18:07:35 -04:00
Anael Mobilia
bf69ea8ccc Fix typos (#413)
Co-authored-by: Anael Mobilia <anael.mobilia@mydsomanager.com>
2023-05-14 18:07:07 -04:00
Michael Kliewe
af1e299dd4 Fix issue #282: Detect large date ranges in aggregate reports and skip processing (#408) 2023-05-14 18:06:07 -04:00
Sean Whalen
d426098b7e 8.6.0
- Replace publicsuffix2 with publicsuffixlist
2023-05-09 09:11:35 -04:00
Jed Laundry
a06fdc586f Change publicsuffix2 to publicsuffixlist (#406)
* change to publicsuffixlist

* update publicsuffixlist (now auto-updating)

* Fix unused imports
2023-05-09 08:49:41 -04:00
Anael Mobilia
062d6ea821 Use a local network connection to elasticsearch (#407) 2023-05-09 08:48:17 -04:00
Sean Whalen
b15e8d0aad Fix documentation headings 2023-05-03 18:25:27 -04:00
Sean Whalen
bf102f78aa 8.5.0
- Add support for Azure Log Analytics (PR #394)
- Fix a bug in the Microsoft Graph integration that caused a crash when an inbox has 10+ folders (PR #398)
- Documentation fixes
2023-05-03 17:10:49 -04:00
Sean Whalen
f6c0a4ecbc Update docs 2023-05-03 16:57:22 -04:00
Sean Whalen
044038a381 Fix mailing list diocumentation 2023-05-03 16:54:25 -04:00
Sean Whalen
cd475255c5 Documentation cleanup 2023-05-03 16:44:15 -04:00
Ben Companjen
2b35b785c6 Split and Organise documentation files (#404)
* Set global TOC collapse to false

* Split documentation

I tried to split the index.md file into logical parts, not changing the contents.
I did add a space and change one HTTP URL to HTTPS.

---------

Co-authored-by: Sean Whalen <44679+seanthegeek@users.noreply.github.com>
2023-05-03 16:11:58 -04:00
Sean Whalen
634631e326 Code style fixes 2023-05-03 16:07:54 -04:00
rubeste
a7280988eb Implemented Azure Log Analytics ingestion via Data Collection Rules (#394)
* Implemented Azure Log Analytics ingestion via Data Collection Rules

* Update loganalytics.py

* Update cli.py

* Update pyproject.toml

* Fixed config bug

Fixed a bug that causes the program to fail if you do not configure a Data stream.

* Fixed code format
2023-05-03 15:54:25 -04:00
Anael Mobilia
02e856a9bf From Elasticsearch 8.7, xpack security isn't on by default but is required (#395)
```
org.elasticsearch.ElasticsearchSecurityException: invalid configuration for xpack.security.transport.ssl - [xpack.security.transport.ssl.enabled] is not set, but the following settings have been configured in elasticsearch.yml : [xpack.security.transport.ssl.keystore.secure_password,xpack.security.transport.ssl.truststore.secure_password]
```
2023-05-03 15:39:46 -04:00
rubeste
70a9d11adb Fixed Bug in graph.py (#398)
* Fixed Bug in graph.py

Fixed Bug regarding the finding of a specific folder.
This Bug caused parsedmarc to crash if it could not find the folder in one Ms Graph request. This is only an issue if your MailBox contains 10+ folders.
It was solved by adding the `$filter=displayName eq '{folder_name}'` param so it would immediatly find the folder.

* Fixed MS Graph Search bug

Fixed bug that causes only 10 messages to be read from MS Graph if batch size is defined as 0.
Fixed formatting

* prevented $top from being 0
2023-05-03 15:36:23 -04:00
Ben Companjen
9c86e2df49 Fix docstring indentations and grammar (#405)
In several docstrings incorrect indentation caused errors in the
rendered documentation.
I also changed a few words to correct the grammar.
2023-05-03 15:30:05 -04:00
Sean Whalen
b75259c58c Merge branch 'master' of github.com:domainaware/parsedmarc 2023-01-21 10:09:39 -05:00
Sean Whalen
16318b9152 8.4.2
PR #386 closes issues #289 and #380

- Only initialize the syslog, S3 and Kafka clients once
2023-01-21 10:08:47 -05:00
kcyd
22a6d21362 initialize syslog, s3 and kafka clients only once (#386) 2023-01-21 10:02:05 -05:00
Sean Whalen
b10cc06441 8.4.1
- Fix bug introduced in 8.3.1 that caused `No such file or directory` errors if output files didn't exist (PR #385 closes issues #358 and #382)
- Make the `--silent` CLI option only print errors.
  Add the `--warnings` options to also print warnings (PR#383)
2023-01-16 15:06:31 -05:00
Aleksandar Ristic
a54cc3e6dd Add --warnings cli option; --silent now prints only errors (#383)
* Adding warnings argument; silent logs only errors.

* Fixing flake warning for trailing whitespace

* Change help for --warnings to match existing style
2023-01-16 14:45:27 -05:00
Anael Mobilia
8b8c8c15fe Fix markdown (#384) 2023-01-16 14:43:36 -05:00
Clayton Dukes
bb838bb11a fixes https://github.com/domainaware/parsedmarc/issues/382 (#385) 2023-01-16 14:43:06 -05:00
Sean Whalen
375aaf025d Better build script 2022-12-23 20:20:53 -05:00
Sean Whalen
f82445fa06 8.4.0 2022-12-23 20:08:00 -05:00
Anael Mobilia
70ff13bfae In case of timeout or exception, wait "check_timeout" before to try a new connexion (documentation : "or the number of seconds until the next mail check"). (#377)
Else in case of a mail server issue, we try to connect again each 5 seconds.
2022-12-23 18:23:01 -05:00
aroldxd
fcc64ed85a add option to allow unencrypted fallback for token cache (#375) 2022-12-23 18:21:22 -05:00
Bjoern
ea777d9d18 Fix Bug with wrong ip_db_path (#338 #287) (#369)
Fixes the problem when only a file path was specified instead of a file
Reportet with Issues:
Not a valid aggregate or forensic report after fresh Install with 8.3.x #338
Not a valid aggregate or forensic report after Update to 7.1.1 #287
2022-12-23 18:19:21 -05:00
Anael Mobilia
4217a076de Doc - Add info on how to update max shards (#368)
Add information on how to fix "Elasticsearch error: RequestError(400, 'validation_exception', 'Validation Failed: 1: this action would add [1] shards, but this cluster currently has [1000]/[1000] maximum normal shards open;"
2022-12-23 18:15:11 -05:00
nmourad
0a0e4beb27 Update documentation default value for ES replica setting (#376)
Change made in 7.1.0 "Set Elasticsearch shard replication to 0 (PR #274)"
Documentation was not updated

Co-authored-by: n.mourad <n.mourad@criteo.com>
2022-12-23 18:14:41 -05:00
ykcab
304ebaaa42 Updated Grafana query template (#366)
Co-authored-by: Alain Mbuku <git@almb.me>
2022-12-23 18:13:56 -05:00
Anael Mobilia
bcf242b0ab Fix typo (#364) 2022-12-23 18:13:10 -05:00
Anael Mobilia
1380eed2b8 Doc - Update install documentation to Elasticsearch/Kibana 8 (#363)
* Update elasticsearch/kibana instructions

[From elastisearch notes](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#heap-size-settings) : 
```
By default, Elasticsearch automatically sets the JVM heap size based on a node’s roles and total memory. We recommend the default sizing for most production environments.
```

* Update nginx conf to TLSv1.3 and IPv6

* Replace nginx proxy by native https server

Kibana now provide https web server, remove the nginx proxy part and directly use kibana

* Fix typo

* Add infos how to login to kibana

* Add interface details
2022-12-23 18:12:39 -05:00
Anael Mobilia
69c2c6bdb6 Add details on virtualenv / package installation (#361) 2022-12-23 18:10:35 -05:00