Final whole-branch review before this tooling branch settles: add a
required --yes-i-know-this-wipes-the-database confirmation flag ahead of
--reset (it deletes ALL users/groups/documents in the target DB, not just
benchmark-created ones); spread seeded documents' created dates over a
3-year window instead of leaving them all on one default date; fix
capture_explain() to use MariaDB's ANALYZE syntax instead of Postgres-only
EXPLAIN ANALYZE (verified against real MariaDB 12.3 -- the old code was a
silent 1064 syntax error); record db_vendor/document_count in run/profile
history entries; shrink SeededData's memory footprint at scale by
returning counts instead of full ORM instance tuples; and a handful of
minor fixes (storage_path assignment, --explain warning instead of silent
no-op, harness.py repeat<1 guard, type annotations). All changes verified
against real Postgres and MariaDB containers on the VM, not just SQLite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The skill documentation stated that run benchmarks /api/tags/ but
src/paperless_benchmark/endpoints.py actually uses
/api/tags/?page_size=100000. Update the skill's Command reference
section to match the actual endpoint definition.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_handle_profile unconditionally called seed_benchmark_dataset every
run, colliding with an IntegrityError against data seeded by a prior
seed/profile call. Look up the existing perf_target user instead,
matching _handle_run's already-correct pattern, and raise a
CommandError pointing at `seed` if no dataset exists.
Scenario.run/queryset_for_explain now take a User directly instead of
a SeededData, since only data.users[0] (perf_target) was ever used.
profile no longer seeds, so it drops the --tier dependency: the
printed line and history entry no longer include a tier/(tier=...)
suffix, and --tier's help text now only mentions `seed`.
Merges the standalone run_benchmarks.py/seed_benchmark_data.py scripts
and the tools/profiling branch's Postgres-only harness into one
cross-backend `manage.py benchmark` command suite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Fix: exclude next-period start from relative date-range filters
Tantivy's [lo TO hi] range is inclusive on both ends, but computed upper
bounds (keyword ranges, YYYY/YYYYMM/YYYYMMDD tokens) represent the start of
the next period. Use half-open [lo TO hi} for those so e.g. "previous month"
no longer matches the 1st of the current month.
* Adds a regression test down to the second check for the hi range
Addresses discussion #13373. Documents that password removal creates a
new document version via re-consumption of the decrypted file rather
than editing in place, and explains why the Consumption Started trigger
produces an initial un-OCR'd version followed by a properly processed
one.
build_llm_index_text queried Note and CustomFieldInstance (plus its
field FK) per document, uncovered by the earlier correspondent/type/
storage_path/tags prefetch fix. Add notes and custom_fields__field to
the rebuild and scoped document querysets, and read from doc.notes.all()
instead of a fresh Note.objects.filter() so the prefetch is actually used.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
The permission filter OR'd three querysets together on top of a
queryset that could already carry two independent tags__id__all
joins, letting a document that matched more than one branch (e.g.
unowned + group-permissioned) come back twice. Replaced it with a
single id__in filter against the existing permitted_document_ids
helper, which is join-free and can't hit this.
* Perf: scope bulk_update_documents' LLM index refresh to the edited document ids instead of the whole library
* Perf: select_related/prefetch_related for the scoped LLM index batch
* Perf: select_related/prefetch_related for the full LLM index rebuild path
* Fix: address Copilot review findings on LLM index scoping
stumpylogandClaude Sonnet 5
shamoonandGitHub
GitHub Actions
SandroandGitHub