* Add dashboard screenshot harness to the dev tooling
dashboard-dev-screenshots.py drives headless Chromium (Playwright)
against the dashboard dev stack and captures how Kibana, OpenSearch
Dashboards, Grafana, and Splunk actually render the current sample
data — for end-to-end verification of dashboard changes and PR
evidence. It encodes the platform quirks that otherwise cost time to
rediscover: fixed render waits instead of networkidle (Kibana/OSD
dashboards poll forever), pinning OSD to the global tenant (a stale
private-tenant copy silently screenshots old dashboards), and driving
the Grafana and Splunk login forms rather than HTTP basic auth. The
output directory is gitignored; usage is documented in
dashboards/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address Copilot review findings on the screenshot harness
- Only require OPENSEARCH_INITIAL_ADMIN_PASSWORD / SPLUNK_PASSWORD when
the osd / splunk targets are selected, and fail fast before launching
Playwright with a clear message naming the missing variable(s).
- Honor GRAFANA_USER (defaulting to admin) to match the dev bootstrap
script instead of hard-coding the Grafana username.
- Print a full traceback to stderr when a target fails, instead of just
the exception message, while still continuing to the next target.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Always close the browser when a screenshot target fails
Copilot round 2: each target only closed its Chromium instance on the
happy path, so a mid-run Playwright failure leaked a headless browser
while the script continued to the next target. Wrap each target body in
try/finally; closing the browser also closes its contexts and pages, so
the single b.close() covers the OSD context too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Exit with an install hint when playwright is missing
Copilot round 3: playwright is deliberately not a project dependency, so
a bare import failure produced a raw ModuleNotFoundError traceback.
Catch ImportError and exit with the same one-time install command the
module docstring documents.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The PostgreSQL dashboard shipped at the repo-root grafana/ by oversight;
every other dashboard source lives under dashboards/ (opensearch/,
grafana/, splunk/). Move it next to the existing Grafana dashboard, list
it in dashboards/README.md, and fix the CHANGELOG path reference.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The forensic→failure rename (#659) left a few loose ends and one deliberate
hold-back. This closes them.
Leftover rename misses (broken paths / stale canonical names):
- CONTRIBUTING.md, dashboard-dev-bootstrap.sh: samples/forensic/* → samples/failure/*
- dashboard-dev-bootstrap.sh, dashboards/README.md: dmarc_forensic_dashboard.xml
→ dmarc_failure_dashboard.xml (the file was already renamed; the import path
and view name were not)
- docs/source/usage.md: PARSEDMARC_GENERAL_SAVE_FORENSIC → ..._SAVE_FAILURE example
- samples/parsedmarc.ini: save_forensic → save_failure
- pyproject.toml, README.md: canonical "failure" naming
(ci.ini intentionally keeps save_forensic to smoke-test the deprecated alias.)
Archive subfolder rename + on-startup migration:
- New failure reports now archive to <archive>/Failure (was <archive>/Forensic).
- _migrate_forensic_archive_folder() runs once on startup (best-effort):
renames Forensic→Failure when no Failure folder exists yet, merges the two
when both exist, no-ops when there's no legacy folder, and logs-and-skips a
mailbox it can't reorganize (warn, don't crash). This consolidates pre- and
post-rename failure reports into one folder, replacing the previously
documented decision to keep the folder named Forensic to avoid a split
archive. Uses the folder-management API (folder_exists / rename_folder /
merge_folders) added in mailsuite 2.1.0; the pin is bumped to >=2.1.0.
Grafana dashboard (the rename PR updated OSD/Splunk/ES-OS but not Grafana):
- Forensic panel titles + the datasource label → Failure; the fo-column display
label and its linked byName field-override matcher both → "Failure Policy"
(changed together so the column-width override keeps matching).
- dev-bootstrap Grafana ES datasource: dmarc_forensic* → dmarc_f* (matches both
pre-rename dmarc_forensic* and post-rename dmarc_failure*, like the OSD/Kibana
dashboards); RESEED wipe loop now also clears dmarc_failure* indices.
- Removed dashboards/grafana/Grafana-DMARC_Reports.json-new_panel.json, an
orphan export accidentally committed in #736 and referenced by nothing.
Tests (tests/test_init.py):
- TestMigrateForensicArchiveFolderMaildir: real on-disk Maildir round-trips via
mailsuite's MaildirConnection (no mocks) — rename, merge, no-op, and the full
get_dmarc_reports_from_mailbox orchestration. Runs in CI (no network/creds).
- TestMigrateForensicArchiveFolderErrorHandling: the one path a real Maildir
can't reproduce — a backend that raises mid-operation must warn, not crash.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sean Whalen