mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-07-27 02:44:57 +00:00
Sean WhalenandClaude Fable 5
47fb50e76a
The from-domain volume table on every provider's aggregate dashboard is now "Message volume and DMARC compliance by from domain" with columns From Domain | Messages | % DMARC Compliant: - OpenSearch Dashboards/Kibana: the agg-based data table is replaced by a TSVB table using a Filter Ratio metric (passed_dmarc:true over all, sum of message_count), pivoted on header_from.keyword. The time field is date_begin rather than the multi-valued date_range, which TSVB's per-value date histogram would double-count. Editing (not rendering) the panel on Kibana 8.x requires the metrics:allowStringIndices advanced setting. - Grafana (Elasticsearch): a second passed_dmarc:true query joined by field with a binary calculation (Sum 2 / Sum 1) rendered as percentunit. - Grafana (PostgreSQL): compliance column via an aggregate FILTER clause, COALESCEd so zero-pass domains show 0 instead of NULL. - Splunk: sum(eval(if(passed_dmarc="true", message_count, 0))) inside stats, per the SPL eval-in-stats syntax. All four providers were verified against the same seeded sample data in the dashboard dev stack; each returns identical per-domain values (example.com: 2425 messages, 5.3% compliant). Dev stack fixes found along the way: cap Elasticsearch heap at 2g (the unset heap auto-sized to 50% of host RAM and was OOM-killed with bootstrap.memory_lock on large hosts), and install the elasticsearch datasource plugin in Grafana, which is no longer bundled as of Grafana 13. Closes #112 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
59 lines
1.7 KiB
YAML
59 lines
1.7 KiB
YAML
services:
|
|
elasticsearch:
|
|
image: docker.elastic.co/elasticsearch/elasticsearch:8.19.7
|
|
environment:
|
|
- network.host=127.0.0.1
|
|
- http.host=0.0.0.0
|
|
- node.name=elasticsearch
|
|
- discovery.type=single-node
|
|
- cluster.name=parsedmarc-cluster
|
|
- discovery.seed_hosts=elasticsearch
|
|
- bootstrap.memory_lock=true
|
|
- xpack.security.enabled=false
|
|
- xpack.license.self_generated.type=basic
|
|
# Without an explicit heap, ES sizes it to 50% of host RAM and mlocks it
|
|
# (bootstrap.memory_lock), which OOM-kills the container on large hosts.
|
|
- ES_JAVA_OPTS=-Xms2g -Xmx2g
|
|
ports:
|
|
- "127.0.0.1:9200:9200"
|
|
ulimits:
|
|
memlock:
|
|
soft: -1
|
|
hard: -1
|
|
healthcheck:
|
|
test:
|
|
[
|
|
"CMD-SHELL",
|
|
"curl -s -XGET http://localhost:9200/_cluster/health?pretty | grep status | grep -q '\\(green\\|yellow\\)'"
|
|
]
|
|
interval: 10s
|
|
timeout: 10s
|
|
retries: 24
|
|
|
|
opensearch:
|
|
image: opensearchproject/opensearch:3
|
|
environment:
|
|
- network.host=127.0.0.1
|
|
- http.host=0.0.0.0
|
|
- node.name=opensearch
|
|
- discovery.type=single-node
|
|
- cluster.name=parsedmarc-cluster
|
|
- discovery.seed_hosts=opensearch
|
|
- bootstrap.memory_lock=true
|
|
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
|
|
ports:
|
|
- "127.0.0.1:9201:9200"
|
|
ulimits:
|
|
memlock:
|
|
soft: -1
|
|
hard: -1
|
|
healthcheck:
|
|
test:
|
|
[
|
|
"CMD-SHELL",
|
|
"curl -sk -u admin:${OPENSEARCH_INITIAL_ADMIN_PASSWORD} -XGET https://localhost:9200/_cluster/health?pretty | grep status | grep -q '\\(green\\|yellow\\)'"
|
|
]
|
|
interval: 10s
|
|
timeout: 10s
|
|
retries: 24
|