Add dashboard screenshot harness to the dev tooling (#840)

* Add dashboard screenshot harness to the dev tooling

dashboard-dev-screenshots.py drives headless Chromium (Playwright)
against the dashboard dev stack and captures how Kibana, OpenSearch
Dashboards, Grafana, and Splunk actually render the current sample
data — for end-to-end verification of dashboard changes and PR
evidence. It encodes the platform quirks that otherwise cost time to
rediscover: fixed render waits instead of networkidle (Kibana/OSD
dashboards poll forever), pinning OSD to the global tenant (a stale
private-tenant copy silently screenshots old dashboards), and driving
the Grafana and Splunk login forms rather than HTTP basic auth. The
output directory is gitignored; usage is documented in
dashboards/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address Copilot review findings on the screenshot harness

- Only require OPENSEARCH_INITIAL_ADMIN_PASSWORD / SPLUNK_PASSWORD when
  the osd / splunk targets are selected, and fail fast before launching
  Playwright with a clear message naming the missing variable(s).
- Honor GRAFANA_USER (defaulting to admin) to match the dev bootstrap
  script instead of hard-coding the Grafana username.
- Print a full traceback to stderr when a target fails, instead of just
  the exception message, while still continuing to the next target.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Always close the browser when a screenshot target fails

Copilot round 2: each target only closed its Chromium instance on the
happy path, so a mid-run Playwright failure leaked a headless browser
while the script continued to the next target. Wrap each target body in
try/finally; closing the browser also closes its contexts and pages, so
the single b.close() covers the OSD context too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Exit with an install hint when playwright is missing

Copilot round 3: playwright is deliberately not a project dependency, so
a bare import failure produced a raw ModuleNotFoundError traceback.
Catch ImportError and exit with the same one-time install command the
module docstring documents.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sean Whalen
2026-07-22 17:01:07 -04:00
committed by GitHub
co-authored by Claude Fable 5
parent 4fa8cfb1e7
commit 72bece4686
3 changed files with 214 additions and 0 deletions
+21
View File
@@ -82,6 +82,27 @@ The bootstrap script provisions two `elasticsearch` datasources (`dmarc-ag` for
2. Open the dashboard's **Source** view, copy the XML, and paste it over the matching file in [splunk/](splunk/) (`dmarc_aggregate_dashboard.xml`, `dmarc_failure_dashboard.xml`, or `smtp_tls_dashboard.xml`).
3. Re-run the bootstrap script. It re-imports each view via `DELETE` + `POST` to the splunkd management API.
## Screenshotting the dashboards
[dashboard-dev-screenshots.py](../dashboard-dev-screenshots.py) captures how
each UI actually renders the current sample data — useful for verifying a
dashboard change end-to-end and for PR evidence. One-time setup, then run
against the live stack:
```bash
pip install playwright && playwright install chromium
set -a; . ./.env; set +a
./dashboard-dev-screenshots.py # all four UIs
./dashboard-dev-screenshots.py grafana # or a subset
```
Output lands in `dashboard-screenshots/` (gitignored). The script encodes
the platform quirks that otherwise cost time to rediscover: Kibana/OSD
dashboards never reach Playwright's "networkidle" (fixed render waits are
used instead), OSD must be pinned to `?security_tenant=global` or a stale
private-tenant copy may be captured, and Grafana/Splunk need their login
forms driven rather than HTTP basic auth.
## Reseeding sample data
```bash