From 14d881ef74a49eaf2c70514e03750e87887d86c1 Mon Sep 17 00:00:00 2001 From: Sean Whalen <44679+seanthegeek@users.noreply.github.com> Date: Sat, 25 Jul 2026 12:24:01 -0400 Subject: [PATCH] Stop seeding the large synthetic report into the dev dashboards (#844) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 2,286-record stress-test report was ~99% of the seeded corpus, so every unfiltered dev dashboard showed its skew instead of a realistic mix — most visibly the almost-entirely-blank envelope_from column, since its records carry no envelope_from and empty SPF domains, which defeats the parser's SPF-domain fallback. Exclude it from the default seed with a comment explaining why and how to load it manually for scale or backfill testing. Co-authored-by: Claude Fable 5 --- dashboard-dev-bootstrap.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/dashboard-dev-bootstrap.sh b/dashboard-dev-bootstrap.sh index 698b9785..6adeaa57 100755 --- a/dashboard-dev-bootstrap.sh +++ b/dashboard-dev-bootstrap.sh @@ -266,9 +266,14 @@ else # 2001:db8::, etc.) that won't resolve, so cap retries/timeout to bound # the cost of those NXDOMAIN-bound lookups. Intentionally invalid samples # (empty_reason.xml, invalid_xml.xml, etc.) are skipped from the list. + # samples/aggregate/!large-example.com!1711897200!1711983600.xml is + # deliberately NOT seeded: its 2,286 synthetic records would be ~99% of + # the corpus, drowning the realistic mix on every unfiltered dashboard + # (and its records carry no envelope_from and empty SPF domains, so that + # column reads almost entirely blank). To load it for scale or backfill + # testing, run the seed command below manually with that file appended. SAMPLE_FILES=( samples/aggregate/!example.com!1538204542!1538463818.xml - samples/aggregate/!large-example.com!1711897200!1711983600.xml 'samples/aggregate/Report domain- borschow.com Submitter- google.com Report-ID- 949348866075514174.eml' samples/aggregate/addisonfoods.com!example.com!1536105600!1536191999.xml samples/aggregate/estadocuenta1.infonacot.gob.mx!example.com!1536853302!1536939702!2940.xml.zip