Rework the SMTP TLS dashboards onto the combined fields

Kibana/OSD: "SMTP TLS domains" replaces its stacked policy_domain ×
policy_type terms with one terms agg on policies_combined.keyword;
"SMTP TLS failure details" replaces six stacked terms spanning both
array levels with one on failure_details_combined.keyword; the
smtp_tls* index-pattern field cache gains the new fields. The
"reporting organizations" table only buckets on doc-level org_name and
needed no change.

Splunk: the base search now expands policies at the JSON level (spath +
mvexpand) so policy fields are scalars per event, and the failure
details panel expands the second level the same way — sums are the
detail's own failed_session_count, correctly paired. Verified via the
search REST API: a two-policy repro returns exactly one row per real
failure detail with per-detail counts.

kibana.md documents the per-policy/per-detail row semantics and the
honest caveat that session-count sums remain per report document.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sean Whalen
2026-07-24 23:04:26 -04:00
co-authored by Claude Fable 5
parent 9be85409d0
commit 8f92a36622
3 changed files with 30 additions and 16 deletions
File diff suppressed because one or more lines are too long
+15 -13
View File
@@ -2,18 +2,14 @@
<label>SMTP TLS Reporting</label>
<search id="base_search">
<query>
index=email sourcetype=smtp:tls organization_name=$organization_name$ policies{}.policy_domain=$policy_domain$ policies{}.policy_type=$policy_type$
| rename policies{}.policy_domain as policy_domain
| rename policies{}.policy_type as policy_type
| rename policies{}.failed_session_count as failed_sessions
| rename policies{}.successful_session_count as successful_sessions
| rename policies{}.failure_details{}.receiving_mx_hostname as receiving_mx_hostname
| rename policies{}.failure_details{}.result_type as failure_type
| rename policies{}.failure_details{}.sending_mta_ip as sending_mta_ip
| rename policies{}.failure_details{}.receiving_ip as receiving_mta_ip
index=email sourcetype=smtp:tls organization_name=$organization_name$
| spath policies{} output=policy
| mvexpand policy
| spath input=policy
| search policy_domain=$policy_domain$ policy_type=$policy_type$
| rename successful_session_count as successful_sessions
| rename failed_session_count as failed_sessions
| fillnull value=0 failed_sessions successful_sessions
| table *
| table *
</query>
<earliest>$time_range.earliest$</earliest>
<latest>$time_range.latest$</latest>
@@ -78,8 +74,14 @@ index=email sourcetype=smtp:tls organization_name=$organization_name$ policies{}
<table>
<search base="base_search">
<query>
where failed_sessions &gt; 0
| stats sum(failed_sessions) as failed_sessions by organization_name, policy_domain, policy_type, failure_type, sending_mta_ip, receiving_mta_ip, receiving_mx_hostname
| spath input=policy path=failure_details{} output=detail
| mvexpand detail
| spath input=detail
| where failed_session_count &gt; 0
| rename result_type as failure_type
| rename receiving_ip as receiving_mta_ip
| fillnull value="none" failure_type sending_mta_ip receiving_mta_ip receiving_mx_hostname
| stats sum(failed_session_count) as failed_sessions by organization_name, policy_domain, policy_type, failure_type, sending_mta_ip, receiving_mta_ip, receiving_mx_hostname
</query>
</search>
<option name="drilldown">none</option>
+12
View File
@@ -128,3 +128,15 @@ reporting organizations, the policy domains they report on, and the
specific failure types — certificate expiry, STARTTLS not supported,
STS policy fetch errors, validation failures, and similar — together with
the sending and receiving MTA addresses involved.
Like the DKIM and SPF details tables above, the "SMTP TLS domains" and
"SMTP TLS failure details" tables show one row per policy and one row per
failure detail, respectively, using combined `policy (domain / type)` and
`failure detail (domain / type / result / sending mta / receiving ip / mx)`
columns so that each policy's or failure detail's fields stay paired
together, rather than aggregating them as separate columns. The
`successful_sessions` and `failed_sessions` columns are summed per report
document, though, not per policy: when a single report carries multiple
policies, a row's session sums include the sibling policies from that
report as well as its own. Fully attributing session counts to a single
policy would require restructuring the stored documents.