mirror of
https://github.com/domainaware/parsedmarc.git
synced 2026-05-21 11:25:23 +00:00
Sean Whalen
4e8c28bbc0
* Align Kibana dashboards with OpenSearch Dashboards source-of-truth
OSD is a fork of Kibana 7.10 and Kibana 8.x's saved-object migration
handlers accept OSD's saved-object format directly. Replace the legacy
Kibana export with a byte-identical copy of the OSD ndjson, so the two
backends ship the same panels, metric aggregations, panel titles, and
field assignments instead of drifting independently.
Verified against Kibana 8.19.7: import returns successCount=26 with no
errors and Kibana auto-migrates each viz / dashboard to its current
saved-object schema (typeMigrationVersion 8.5.0 for visualizations,
10.3.0 for dashboards) on import.
Net effects for Kibana users on import:
- Picks up the metric-aggregation fix from 9.10.3 — pies, tables, and
the choropleth now sum(message_count) instead of counting OS docs,
giving real message volume rather than distinct source-row counts.
- Adds "Message sources by Autonomous System" and "Message sources by
name and type" panels (previously only on OSD).
- Forensic dashboard simplified to OSD's two-panel layout (markdown
intro + samples table) — drops the Kibana-only IP-address and
country-ISO tables and the choropleth.
- Adds the "SMTP TLS reporting" dashboard (was absent from the bundled
Kibana export).
- Drops the extraneous "Evolution DMARC par source_reverse_DNS" Lens
visualization that snuck in via a community contribution.
Updates docs/source/kibana.md to reflect the new dashboard names
("DMARC aggregate reports" / "DMARC failure reports") and adds a brief
section on the SMTP TLS reporting dashboard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Drop the duplicate Kibana ndjson; point Kibana users at the OSD file
Kibana 8.x's saved-object migration handlers accept the OpenSearch
Dashboards saved-object format directly (verified by import returning
successCount=26 with no errors), so a separate kibana/export.ndjson
was just two copies of the same bytes that would inevitably drift. Drop
it and update the bootstrap script and docs to point at the existing
dashboards/opensearch/opensearch_dashboards.ndjson.
Add a path-filtered CI workflow (.github/workflows/dashboards.yml) that
fires only when the OSD ndjson changes. It stands up an Elasticsearch +
Kibana 8.19.7 service pair, POSTs the file at the saved-objects import
endpoint, and asserts success=true with no errors. That keeps the
single-file source compatible with Kibana on every change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
104 lines
4.6 KiB
Markdown
104 lines
4.6 KiB
Markdown
|
|
# Using the Kibana dashboards
|
|
|
|
The Kibana DMARC dashboards are a human-friendly way to understand the
|
|
results from incoming DMARC reports.
|
|
|
|
There is no separate Kibana export — Kibana 8.x's saved-object migration
|
|
handlers accept the OpenSearch Dashboards format directly, so Kibana
|
|
users import the bundled
|
|
[`dashboards/opensearch/opensearch_dashboards.ndjson`](https://raw.githubusercontent.com/domainaware/parsedmarc/master/dashboards/opensearch/opensearch_dashboards.ndjson)
|
|
in *Stack Management → Saved Objects → Import*. A CI check imports the
|
|
same file into a Kibana 8.x container on every change so this stays
|
|
compatible.
|
|
|
|
:::{note}
|
|
The default dashboard is DMARC aggregate reports. To switch between
|
|
dashboards, click on the Dashboard link on the left side menu of Kibana.
|
|
:::
|
|
|
|
## DMARC aggregate reports
|
|
|
|
As the name suggests, this dashboard is the best place to start
|
|
reviewing your aggregate DMARC data.
|
|
|
|
Across the top of the dashboard, three pie charts display the percentage of
|
|
alignment pass/fail for SPF, DKIM, and DMARC. Clicking on any chart segment
|
|
will filter for that value.
|
|
|
|
:::{note}
|
|
Messages should not be considered malicious just because they failed to pass
|
|
DMARC; especially if you have just started collecting data. It may be a
|
|
legitimate service that needs SPF and DKIM configured correctly.
|
|
:::
|
|
|
|
Start by filtering the results to only show failed DKIM alignment. While DMARC
|
|
passes if a message passes SPF or DKIM alignment, only DKIM alignment remains
|
|
valid when a message is forwarded without changing the from address, which is
|
|
often caused by a mailbox forwarding rule. This is because DKIM signatures are
|
|
part of the message headers, whereas SPF relies on SMTP session headers.
|
|
|
|
Underneath the pie charts. you can see graphs of DMARC passage and message
|
|
disposition over time.
|
|
|
|
Under the graphs you will find the most useful data tables on the dashboard. On
|
|
the left, there is a list of organizations that are sending you DMARC reports.
|
|
In the center, there is a list of sending servers grouped by the base domain
|
|
in their reverse DNS. On the right, there is a list of email from domains,
|
|
sorted by message volume.
|
|
|
|
By hovering your mouse over a data table value and using the magnifying glass
|
|
icons, you can filter on our filter out different values. Start by looking at
|
|
the Message Sources by Reverse DNS table. Find a sender that you recognize,
|
|
such as an email marketing service, hover over it, and click on the plus (+)
|
|
magnifying glass icon, to add a filter that only shows results for that sender.
|
|
Now, look at the Message From Header table to the right. That shows you the
|
|
domains that a sender is sending as, which might tell you which brand/business
|
|
is using a particular service. With that information, you can contact them and
|
|
have them set up DKIM.
|
|
|
|
:::{note}
|
|
If you have a lot of B2C customers, you may see a high volume of emails as
|
|
your domains coming from consumer email services, such as Google/Gmail and
|
|
Yahoo! This occurs when customers have mailbox rules in place that forward
|
|
emails from an old account to a new account, which is why DKIM
|
|
authentication is so important, as mentioned earlier. Similar patterns may
|
|
be observed with businesses who send from reverse DNS addressees of
|
|
parent, subsidiary, and outdated brands.
|
|
:::
|
|
|
|
Further down the dashboard, you can filter by source country or source IP
|
|
address.
|
|
|
|
Tables showing SPF and DKIM alignment details are located under the IP address
|
|
table.
|
|
|
|
:::{note}
|
|
The alignment tables (SPF details, DKIM details) and the per-IP source
|
|
table live on the same dashboard, further down. To view failures only,
|
|
use the pie chart at the top of the page as a filter.
|
|
:::
|
|
|
|
Any other filters work the same way. You can also add your own custom temporary
|
|
filters by clicking on Add Filter at the upper right of the page.
|
|
|
|
## DMARC failure reports
|
|
|
|
The DMARC failure reports dashboard (formerly DMARC Forensic Samples) contains
|
|
information on DMARC failure reports (also known as forensic or ruf reports).
|
|
These reports contain samples of emails that have failed to pass DMARC.
|
|
|
|
:::{note}
|
|
Most recipients do not send forensic/failure/ruf reports at all to avoid
|
|
privacy leaks. Some recipients (notably Chinese webmail services) will only
|
|
supply the headers of sample emails. Very few provide the entire email.
|
|
:::
|
|
|
|
## SMTP TLS reporting
|
|
|
|
The SMTP TLS reporting dashboard surfaces aggregate counts of TLS-RPT
|
|
reporting organizations, the policy domains they report on, and the
|
|
specific failure types — certificate expiry, STARTTLS not supported,
|
|
STS policy fetch errors, validation failures, and similar — together with
|
|
the sending and receiving MTA addresses involved.
|