Dashboard development
This directory holds the dashboard sources that ship with parsedmarc:
- opensearch/opensearch_dashboards.ndjson — the source-of-truth saved-objects export. It is imported into both OpenSearch Dashboards and Kibana (the file format is compatible with both).
- grafana/Grafana-DMARC_Reports.json — the Grafana dashboard, with two Elasticsearch datasources (
dmarc-ag,dmarc-fo). - splunk/ — three Splunk dashboard XML views (
dmarc_aggregate,dmarc_forensic,smtp_tls).
Edits to any of these files should be exported from a running instance after authoring the change in the UI, not hand-edited (with the occasional exception of small XML tweaks for Splunk).
The dev stack
docker-compose.dashboard-dev.yml brings up every viz target at once so a single dashboard change can be authored and re-exported across all four UIs in one session. It include:s docker-compose.yml for the Elasticsearch and OpenSearch backends, then layers on Kibana, OpenSearch Dashboards, Grafana, and Splunk.
| Service | URL | Credentials |
|---|---|---|
| Elasticsearch | http://localhost:9200 | (security disabled) |
| OpenSearch | https://localhost:9201 | admin / $OPENSEARCH_INITIAL_ADMIN_PASSWORD |
| Kibana | http://localhost:5601 | (security disabled) |
| OpenSearch Dashboards | http://localhost:5602 | admin / $OPENSEARCH_INITIAL_ADMIN_PASSWORD |
| Grafana | http://localhost:3000 | admin / $GRAFANA_PASSWORD |
| Splunk Web / HEC | http://localhost:8000 / https://localhost:8088 | admin / $SPLUNK_PASSWORD, HEC token $SPLUNK_HEC_TOKEN |
All ports bind to 127.0.0.1 only.
Prerequisites
-
Docker with the Compose v2 plugin.
-
A repo-root
.envdefining the secrets the compose file references:OPENSEARCH_INITIAL_ADMIN_PASSWORD=... SPLUNK_PASSWORD=... SPLUNK_HEC_TOKEN=... GRAFANA_PASSWORD=...Pick any values you like — these are local-only dev secrets. Both
.envandparsedmarc*.iniare gitignored. The matching values must also appear in parsedmarc-dev.ini, which the bootstrap script feeds to the parsedmarc CLI for sample-data ingestion. -
The parsedmarc CLI on
PATH(or in./venv/bin/) —pip install -e .[build]from the repo root works. Override the lookup withPARSEDMARC_BIN=/path/to/parsedmarcif needed.
One-shot bootstrap
dashboard-dev-bootstrap.sh is the normal entry point. It is idempotent — re-run it any time:
./dashboard-dev-bootstrap.sh
It does, in order:
docker compose -f docker-compose.dashboard-dev.yml up -dand waits for every service's health endpoint.- Provisions Splunk: creates the
emailindex, creates theDMARCapp, configures the auto-created HEC token to allow theemailindex, and scopes the search-app's "scheduled export" announcement view away fromglobalso it stops appearing in the DMARC app's dashboard list. - Seeds Elasticsearch, OpenSearch, and Splunk with parsedmarc-parsed sample reports (from samples/) so the dashboards render against real data. Skipped when ES already has aggregate docs — pass
RESEED=1to wipe and re-seed all three backends. - Imports the dashboard files from this directory into the running services. This step always runs, so the typical edit loop is edit in the UI → export → save into this directory → re-run the bootstrap script to verify the file imports cleanly into a fresh service.
VS Code users can run this via the Dev Dashboard: Bootstrap task in .vscode/tasks.json. Dev Dashboard: Up brings the stack up without importing or seeding.
Editing a dashboard
After running the bootstrap script once, the round trip for each platform is:
OpenSearch Dashboards (and Kibana)
- Edit the dashboard at http://localhost:5602/ (OpenSearch Dashboards) — this is the canonical authoring surface.
- Stack Management → Saved Objects → Export, select the DMARC dashboard, include related objects, and save the resulting
.ndjsonover opensearch/opensearch_dashboards.ndjson. - Re-run
./dashboard-dev-bootstrap.shto confirm it re-imports cleanly into both OSD and Kibana. The Kibana CI workflow (.github/workflows/dashboards.yml) also imports the same file on every PR that touches it.
OSD imports default to the global_tenant so other admins on the instance can see the result. Set OSD_TENANT=... to import elsewhere.
Grafana
- Edit the dashboard at http://localhost:3000/.
- Dashboard settings → JSON Model, copy the JSON, save it to grafana/Grafana-DMARC_Reports.json.
- Re-run the bootstrap script.
The bootstrap script provisions two elasticsearch datasources (dmarc-ag for dmarc_aggregate*, dmarc-fo for dmarc_forensic*) on first run; existing datasources are left alone.
Splunk
- Edit the dashboard at http://localhost:8000/ inside the DMARC app.
- Open the dashboard's Source view, copy the XML, and paste it over the matching file in splunk/ (
dmarc_aggregate_dashboard.xml,dmarc_forensic_dashboard.xml, orsmtp_tls_dashboard.xml). - Re-run the bootstrap script. It re-imports each view via
DELETE+POSTto the splunkd management API.
Reseeding sample data
RESEED=1 ./dashboard-dev-bootstrap.sh
Wipes every dmarc_aggregate* / dmarc_forensic* / smtp_tls* index from ES and OS, drops and recreates the Splunk email index, then re-runs the parsedmarc CLI against the curated sample list. Use this after changing parsedmarc's enrichment or output schemas.
Tearing the stack down
docker compose -f docker-compose.dashboard-dev.yml down # stop containers, keep volumes
docker compose -f docker-compose.dashboard-dev.yml down -v # also drop volumes (full reset)