Align Kibana dashboards with OpenSearch Dashboards source-of-truth (#737)

* Align Kibana dashboards with OpenSearch Dashboards source-of-truth OSD is a fork of Kibana 7.10 and Kibana 8.x's saved-object migration handlers accept OSD's saved-object format directly. Replace the legacy Kibana export with a byte-identical copy of the OSD ndjson, so the two backends ship the same panels, metric aggregations, panel titles, and field assignments instead of drifting independently. Verified against Kibana 8.19.7: import returns successCount=26 with no errors and Kibana auto-migrates each viz / dashboard to its current saved-object schema (typeMigrationVersion 8.5.0 for visualizations, 10.3.0 for dashboards) on import. Net effects for Kibana users on import: - Picks up the metric-aggregation fix from 9.10.3 — pies, tables, and the choropleth now sum(message_count) instead of counting OS docs, giving real message volume rather than distinct source-row counts. - Adds "Message sources by Autonomous System" and "Message sources by name and type" panels (previously only on OSD). - Forensic dashboard simplified to OSD's two-panel layout (markdown intro + samples table) — drops the Kibana-only IP-address and country-ISO tables and the choropleth. - Adds the "SMTP TLS reporting" dashboard (was absent from the bundled Kibana export). - Drops the extraneous "Evolution DMARC par source_reverse_DNS" Lens visualization that snuck in via a community contribution. Updates docs/source/kibana.md to reflect the new dashboard names ("DMARC aggregate reports" / "DMARC failure reports") and adds a brief section on the SMTP TLS reporting dashboard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Drop the duplicate Kibana ndjson; point Kibana users at the OSD file Kibana 8.x's saved-object migration handlers accept the OpenSearch Dashboards saved-object format directly (verified by import returning successCount=26 with no errors), so a separate kibana/export.ndjson was just two copies of the same bytes that would inevitably drift. Drop it and update the bootstrap script and docs to point at the existing dashboards/opensearch/opensearch_dashboards.ndjson. Add a path-filtered CI workflow (.github/workflows/dashboards.yml) that fires only when the OSD ndjson changes. It stands up an Elasticsearch + Kibana 8.19.7 service pair, POSTs the file at the saved-objects import endpoint, and asserts success=true with no errors. That keeps the single-file source compatible with Kibana on every change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Sean Whalen <seanthegeek@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 19:05:24 +00:00 · 2026-04-27 01:30:48 -04:00
parent 826e78c390
commit 4e8c28bbc0
5 changed files with 116 additions and 37 deletions
@@ -0,0 +1,88 @@
+name: Validate dashboards
+
+permissions:
+  contents: read
+
+# Kibana 8.x's saved-object migration handlers accept the OpenSearch
+# Dashboards saved-object format directly, so we ship the OSD ndjson as the
+# single source for both backends. This workflow guards that compatibility:
+# any change to the OSD ndjson must still import cleanly into a Kibana 8.x
+# container before the change is mergeable.
+#
+# The job is path-filtered to only run when the ndjson itself changes —
+# every other PR skips it.
+on:
+  push:
+    branches: [master]
+    paths: ['dashboards/opensearch/opensearch_dashboards.ndjson']
+  pull_request:
+    branches: [master]
+    paths: ['dashboards/opensearch/opensearch_dashboards.ndjson']
+
+jobs:
+  kibana-import:
+    name: Verify ndjson imports into Kibana 8.x
+    runs-on: ubuntu-latest
+
+    services:
+      elasticsearch:
+        image: docker.elastic.co/elasticsearch/elasticsearch:8.19.7
+        env:
+          discovery.type: single-node
+          xpack.security.enabled: "false"
+          xpack.license.self_generated.type: basic
+          ES_JAVA_OPTS: "-Xms512m -Xmx512m"
+        ports:
+          - 9200:9200
+        options: >-
+          --health-cmd "curl -sf http://localhost:9200/_cluster/health"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 24
+
+      kibana:
+        image: docker.elastic.co/kibana/kibana:8.19.7
+        env:
+          ELASTICSEARCH_HOSTS: http://elasticsearch:9200
+        ports:
+          - 5601:5601
+        options: >-
+          --health-cmd "curl -sf http://localhost:5601/api/status"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 30
+
+    steps:
+      - uses: actions/checkout@v5
+
+      - name: Wait for Kibana to report ready
+        run: |
+          for i in $(seq 1 60); do
+            if curl -sf http://localhost:5601/api/status >/dev/null; then
+              echo "Kibana is ready"
+              exit 0
+            fi
+            sleep 5
+          done
+          echo "Kibana failed to come up within 5 minutes" >&2
+          exit 1
+
+      - name: Import OSD ndjson and assert success
+        run: |
+          response=$(curl -sS -X POST \
+            'http://localhost:5601/api/saved_objects/_import?overwrite=true' \
+            -H 'kbn-xsrf: true' \
+            --form file=@dashboards/opensearch/opensearch_dashboards.ndjson)
+          echo "$response" | python3 -m json.tool
+          echo "$response" | python3 - <<'PY'
+          import json, sys
+          d = json.load(sys.stdin)
+          if not d.get("success"):
+              sys.exit(f"Kibana import failed: {d}")
+          if d.get("errors"):
+              sys.exit(f"Kibana import had errors: {d['errors']}")
+          n = d.get("successCount", 0)
+          if n < 1:
+              sys.exit(f"Expected at least 1 imported object, got {n}")
+          print(f"OK: {n} saved objects imported and migrated by Kibana")
+          PY