Fix blank below-fold panels in Grafana full-dashboard screenshots (#841)

* Grow the viewport so Grafana full-dashboard captures render every panel

Grafana only runs a panel's queries once the panel enters the viewport,
so the full-page screenshot taken at a fixed 1720x1200 viewport captured
everything below the fold as an empty placeholder — the tables in the
bottom two-thirds of the DMARC dashboard came out blank. Scroll-through
approaches are unreliable in headless Chromium (Grafana's custom scroll
container ignores synthetic wheel events), so instead measure the
dashboard's full scroll height and grow the viewport to cover it before
capturing, then restore the normal viewport for the per-panel viewPanel
captures (which fill the viewport and would otherwise be distorted).

Verified against the live dashboard-dev stack: all panels, including the
bottom alignment-detail tables, now render in the capture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Pin ruff exactly, matching the existing pyright pin rationale

CI installs the [build] extra fresh on every run, and ruff was the one
lint tool left unpinned. ruff 0.16.0 (released this week) began
flagging this codebase's str.format() house style, so every PR started
failing lint on lines it never touched. Pin to 0.15.21 — the version
the codebase is clean under — with the same bump-deliberately comment
pyright carries. Upgrading to 0.16 and converting to f-strings can be
its own PR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Capture the grown viewport, not the full page (Copilot finding)

full_page=True screenshots the entire scrollable page, so a dashboard
taller than the 12000px viewport cap would still include unrendered
blank panels below the cap. Capture the viewport exactly instead — a
taller-than-cap dashboard now yields a truncated-but-fully-rendered
image, with a printed note about the truncation instead of silence.

Verified against the live dashboard-dev stack: capture is 1720x6241,
matching the measured dashboard height plus buffer, all panels
rendered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

dashboard-dev-screenshots.py

@@ -20,6 +20,12 @@ Hard-won details encoded here:
   copies; pin ?security_tenant=global in the URL or you will screenshot
   old dashboards.
 - Grafana ignores HTTP basic auth for its UI; drive the login form.
+- Grafana lazy-renders panels only when they enter the viewport, so the
+  full-dashboard capture grows the viewport to the dashboard's full height
+  before screenshotting; per-panel viewPanel captures keep the normal size.
+  The capture itself is viewport-sized (not full-page) and capped at
+  12000px, so extremely tall dashboards are truncated at the cap rather
+  than padded out with unrendered blank panels.
 - Splunk panels are the slowest to populate; wait ~25s before capturing.
 """
 
@@ -124,7 +130,32 @@ def grafana(pw):
             timeout=120000,
         )
         page.wait_for_timeout(10000)
-        shot(page, "grafana_dashboard.png")
+        # Grafana only renders a panel's queries once the panel scrolls into
+        # the viewport, so a fixed 1720x1200 viewport leaves everything
+        # below the fold as an empty placeholder in the full-page capture.
+        # Grow the viewport to cover the whole dashboard first so every
+        # panel is "visible" and runs its queries before we screenshot, then
+        # capture that viewport exactly (not full-page) so a dashboard
+        # taller than the cap below is truncated at the cap instead of
+        # having the full-page capture reach past it into unrendered panels.
+        height = page.evaluate(
+            "() => Math.max(document.documentElement.scrollHeight,"
+            " document.body ? document.body.scrollHeight : 0)"
+        )
+        capture_height = min(height + 400, 12000)
+        if height + 400 > 12000:
+            print(
+                "note: dashboard is taller than the 12000px viewport cap; "
+                "grafana_dashboard.png will be truncated at 12000px"
+            )
+        page.set_viewport_size({"width": VIEWPORT["width"], "height": capture_height})
+        page.wait_for_timeout(20000)
+        shot(page, "grafana_dashboard.png", full=False)
+        # Restore the normal viewport for the per-panel captures below; a
+        # viewPanel view fills the viewport, so a grown viewport would
+        # distort those screenshots.
+        page.set_viewport_size(VIEWPORT)
+        page.wait_for_timeout(2000)
         # Zoomed views of the alignment-detail panels.
         for pid, name in ((40, "dkim_details"), (16, "spf_details"), (41, "overview")):
             page.goto(

pyproject.toml

@@ -91,7 +91,12 @@ build = [
     # stays on requests because its permissive-TLS fallback is built on
     # urllib3's HTTPAdapter machinery.
     "requests>=2.22.0",
-    "ruff",
+    # Pinned exactly for the same reason as pyright: ruff's default rule
+    # set evolves between releases (0.16.0 began flagging this codebase's
+    # str.format() style), so an unpinned version breaks CI without any
+    # code change. Bump deliberately and fix any new findings in the same
+    # PR as the bump.
+    "ruff==0.15.21",
     "sphinx",
     "sphinx_rtd_theme",
 ]