chore: remove benchmark-commands spec/plan docs now that work is complete

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fix: apply final-review fix wave (safety guard, date spread, MariaDB explain, history fields, memory footprint, minors)
2026-07-30 23:55:59 +00:00 · 2026-07-30 13:43:01 -07:00 · 2026-07-30 13:33:28 -07:00 · 2026-07-30 11:20:14 -07:00 · 2026-07-30 11:07:33 -07:00 · 2026-07-30 10:53:02 -07:00
35 changed files with 1682 additions and 238 deletions
@@ -0,0 +1,187 @@
+---
+name: paperless-benchmarking
+description: Use when profiling paperless-ngx performance, running `manage.py benchmark`, investigating a slow query or endpoint, or deciding whether a profiling finding should become a permanent registered scenario. Covers command reference, the fork/merge-back branch workflow for perf investigations, and how to read query-plan output.
+---
+
+# Paperless-ngx Benchmarking
+
+This repo has a built-in benchmarking/profiling tool: `manage.py benchmark`, in
+the `paperless_benchmark` Django app. It replaces ad hoc standalone scripts —
+use it instead of writing new one-off seed/timing scripts.
+
+## Command reference
+
+```
+manage.py benchmark seed --tier {home,medium,large} [--reset --yes-i-know-this-wipes-the-database] [--seed N]
+manage.py benchmark run --repeat 5 [--label baseline]
+manage.py benchmark profile <scenario_name> [--repeat 5] [--explain]
+manage.py benchmark list-scenarios
+```
+
+- **`seed`** builds a realistic dataset at one of three scales: `home` (500
+  documents — fast, use this for iteration), `medium` (20,000 — the default
+  when `--tier` is omitted; a multi-minute seed), `large` (360,000 — matches
+  the scale reported in real large-install bug reports; slow, only use it
+  when a finding needs confirming at real scale). `--reset` wipes any
+  previously-seeded benchmark data first — but it is destructive and
+  irreversible: it deletes **all** users, **all** groups, and **all**
+  documents/tags/correspondents/document types/storage paths in the target
+  database, not just benchmark-created rows. Only run it against a disposable
+  benchmark database, never a real install. Because of that, `--reset` also
+  requires passing `--yes-i-know-this-wipes-the-database` in the same
+  invocation, or the command raises an error and does nothing. Omit both
+  flags if you want to layer more data onto an existing seed instead. `seed`
+  creates two named users, `perf_target` (mixed owned/shared documents,
+  realistic guardian permission grants) and `perf_admin` (superuser), plus a
+  general user/group pool with realistic permission-row ratios.
+- **`run`** times the 3 built-in API endpoint benchmarks
+  (`/api/documents/`, `/api/documents/?page_size=50`, `/api/tags/?page_size=100000`) for both
+  `perf_target` and `perf_admin`, reporting min/median/max wall-clock and SQL
+  query count. Requires `seed` to have already run — it reuses that data, it
+  does not seed its own.
+- **`profile`** times one named scenario from the registry (see
+  `list-scenarios`) via best-of-N repeat timing and SQL query count, against
+  `perf_target`. `--explain` additionally captures and prints the query plan:
+  real `EXPLAIN ANALYZE` execution stats on PostgreSQL/MariaDB, or
+  `EXPLAIN QUERY PLAN` (plan only, no real timing/row counts — clearly labeled
+  as such) on SQLite. Like `run`, `profile` requires `seed` to have already
+  run — it does not seed its own data either.
+- Every `run`/`profile` invocation appends a JSON line to
+  `benchmark_results/history.jsonl` at the repo root (local-only, gitignored —
+  never commit this file). Use it to compare a `before`/`after` pair across
+  two invocations without hand-copying numbers.
+- Full chain example: `seed --reset` once, then `run` and `profile` as many
+  times as you want against that same seeded data — no need to reseed between
+  them.
+
+## Adding a new scenario
+
+A "scenario" is a named, registered query/operation that `profile` can time
+and explain. To add one, edit `src/paperless_benchmark/scenarios.py`: write a
+`_<name>_run(user)` function (returns whatever `run_profile` should time) and
+optionally a `_<name>_queryset(user)` function (returns the `QuerySet` for
+`--explain` to analyze), then `register(Scenario(name=..., describe=...,
+run=..., queryset_for_explain=...))` at module level. Both functions receive
+the already-seeded `perf_target` user — they should not seed their own data.
+
+## Branch workflow
+
+`tools/benchmark-management-commands` is a **long-lived tooling branch**, not
+a feature branch that gets merged and closed:
+
+1. It is periodically brought up to date with `dev` (merge `dev` into it) so
+   the tooling doesn't drift from the schema/codebase it profiles. Do this
+   before starting a new investigation if it's been a while since the last
+   sync.
+2. **Every performance investigation forks its own branch from
+   `tools/benchmark-management-commands`** (not from `dev`). Do the
+   investigation there: write throwaway profiling code, try fixes, capture
+   before/after numbers.
+3. **That investigation branch never merges into `dev` or production.** Its
+   only job is to produce evidence and, optionally, a reusable scenario.
+4. If the investigation turns up a scenario worth keeping permanently (see
+   "When to graduate a scenario" below), open a PR that adds **just that
+   scenario** back into `tools/benchmark-management-commands` — not the rest
+   of the investigation branch's throwaway code.
+5. Any actual production fix the investigation motivates (e.g. an ORM query
+   change) goes into its own normal feature branch off `dev`, following the
+   project's regular contribution process — profiling evidence informs that
+   PR's description, but the profiling code itself does not travel with it.
+
+## Reading query-plan output
+
+- **PostgreSQL** `EXPLAIN ANALYZE`: look for `Seq Scan` on a large table
+  (missing index), a large gap between `rows=N` (planner's estimate) and the
+  actual row count in parentheses (stale statistics or a bad cardinality
+  estimate), and nested-loop joins driven by an outer relation with many
+  rows (usually the N+1 pattern this tool exists to catch).
+- **MariaDB**: verified against a real MariaDB 12.3 container that MariaDB
+  does NOT accept MySQL 8.0.18+'s `EXPLAIN ANALYZE` syntax (it's a 1064
+  syntax error) -- `capture_explain()` instead runs MariaDB's own
+  `ANALYZE <statement>` form (no `EXPLAIN` keyword), which returns a
+  tabular plan with real per-row execution columns: `rows` (estimate) vs.
+  `r_rows` (actual), and `filtered` vs. `r_filtered`. A large gap between
+  `rows` and `r_rows`, or `type: ALL` (full table scan) on a large table,
+  are the signals to look for -- the same underlying concerns as Postgres's
+  `Seq Scan`/estimate-vs-actual gap, just in MariaDB's column-based output
+  instead of Postgres's nested-tree text format.
+- **SQLite** `EXPLAIN QUERY PLAN`: no real timing/row-count data, only the
+  chosen access path (`SCAN` vs `SEARCH`, which index if any). Useful for
+  confirming an index is even being considered, not for judging real-world
+  cost — corroborate any SQLite finding against Postgres/MariaDB before
+  trusting it, since planner behavior differs meaningfully between them.
+- Compare query **count**, not just timing, between before/after: a fix that
+  keeps the same wall-clock time but drops query count from O(n) to O(1) is
+  still a real, durable improvement — timing alone is noisy and
+  environment-dependent, query count is not.
+
+## Cleaning up after an interrupted run
+
+If a `seed`/`run`/`profile` invocation gets killed mid-run (Ctrl-C, `kill -9`,
+a timed-out SSH session, etc.), check whether it left anything behind before
+trusting the next benchmark's numbers. This was verified for real: a
+`benchmark seed --tier large --reset ...` was started against both a fresh
+PostgreSQL 18 container and a fresh MariaDB 12.3 container and `kill -9`'d a
+few seconds into document seeding. In both cases, the database-side
+connection disappeared immediately -- no stuck backend, no lingering query,
+no held lock was observed in either backend once the killed process's PID
+was confirmed gone. That said, this was one interruption point (mid
+bulk-seed, between chunks); a run killed mid-query, or a driver/network
+hiccup that doesn't cleanly close the socket, could behave differently, so
+still check before trusting a number if any run in the session was
+interrupted:
+
+- **PostgreSQL**: look for leftover connections against the benchmark
+  database:
+
+  ```sql
+  SELECT pid, state, query, query_start
+  FROM pg_stat_activity
+  WHERE datname = current_database() AND pid <> pg_backend_pid();
+  ```
+
+  If a stuck backend shows up, clear it with:
+
+  ```sql
+  SELECT pg_terminate_backend(pid)
+  FROM pg_stat_activity
+  WHERE datname = current_database() AND pid <> pg_backend_pid();
+  ```
+
+- **MariaDB**: look for leftover connections/queries:
+
+  ```sql
+  SHOW FULL PROCESSLIST;
+  ```
+
+  If a stuck connection shows up (anything other than your current admin
+  session), clear it with:
+
+  ```sql
+  KILL <id>;
+  ```
+
+A stray connection left running concurrently with a subsequent benchmark run
+would add real, contaminating load (extra queries competing for the same
+rows, possibly held locks slowing the next run's timings) -- cheap to rule
+out, expensive to silently trust a number that was actually measured
+alongside a zombie connection.
+
+## When to graduate a one-off finding into a permanent scenario
+
+Register a scenario (rather than leaving it as throwaway code on the
+investigation branch) when **both** are true:
+
+- The query pattern is one this codebase is likely to regress on again (e.g.
+  it involves a permission-check join, a bulk operation, or anything else
+  with an easy-to-reintroduce N+1) — not a one-time fluke specific to this
+  investigation.
+- Re-running it later, against a fresh seed, would still produce a
+  meaningful signal (it doesn't depend on investigation-specific throwaway
+  data or a fix that's already permanently landed and can't regress the same
+  way).
+
+If a finding doesn't meet both bars, keep it as disposable code on the
+investigation branch and let the branch's evidence (captured in the PR
+description of whatever production fix it motivates) be the permanent
+record instead.
@@ -115,3 +115,6 @@ celerybeat-schedule*

 # Git worktree local folder
 .worktrees
+
+# Benchmark tooling output (local only, never committed)
+/benchmark_results/
@@ -620,6 +620,34 @@ no other workflow will be executed on the document.
 If a "Move to Trash" action is executed in a consume pipeline, the consumption
 will be aborted and the file will be deleted.

+##### Password Removal {#workflow-action-password-removal}
+
+"Password Removal" actions attempt to remove password protection from encrypted PDF documents. You can specify:
+
+- One or more passwords to try, separated by commas or new lines
+- Each password is tried in order until one successfully unlocks the document
+
+Password removal never modifies a file in place. Instead, once a working password is found, the
+decrypted content is consumed as a new [document version](#document-file-versions), leaving the
+original (still encrypted) version in the document's version history.
+
+**Consumption Started**: because this trigger fires before the document exists yet, the password
+removal itself is deferred until after the initial consumption of the encrypted file has completed.
+OCR engines cannot process an encrypted PDF, so this first version is typically stored with no
+extracted text (unless the file already contained extractable text outside of OCR). Immediately
+afterwards, the password is removed and the decrypted file is automatically re-consumed as a second,
+new version of the same document, this time with normal OCR/text extraction applied. In other words,
+a password-protected file added with this trigger will briefly exist as an un-OCR'd version before
+the properly processed version is created.
+
+**Document Added**, **Document Updated**, **Scheduled**: these triggers run against a document that
+already exists, so password removal happens immediately: the decrypted content is queued for
+consumption as a new version right away. Note that if the document's initial consumption also
+happened while it was still encrypted, that original version will likewise be missing OCR text.
+
+**Current limitation**: Passwords are stored as a simple list without descriptions. To handle
+multiple PDF types with different passwords, create separate workflows for each use case.
+
 #### Workflow placeholders

 Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
@@ -659,7 +659,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">445</context>
+          <context context-type="linenumber">449</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/bulk-editor/custom-fields-bulk-edit-dialog/custom-fields-bulk-edit-dialog.component.html</context>
@@ -831,7 +831,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">464</context>
+          <context context-type="linenumber">468</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
@@ -1355,7 +1355,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">390</context>
+          <context context-type="linenumber">394</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.html</context>
@@ -1596,7 +1596,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">165</context>
+          <context context-type="linenumber">169</context>
        </context-group>
      </trans-unit>
      <trans-unit id="2691296884221415710" datatype="html">
@@ -1607,7 +1607,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">170</context>
+          <context context-type="linenumber">174</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.html</context>
@@ -1638,7 +1638,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">174</context>
+          <context context-type="linenumber">178</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.html</context>
@@ -1669,7 +1669,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">178</context>
+          <context context-type="linenumber">182</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.html</context>
@@ -4914,7 +4914,7 @@
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">356</context>
+          <context context-type="linenumber">360</context>
        </context-group>
      </trans-unit>
      <trans-unit id="8057014866157903311" datatype="html">
@@ -7760,14 +7760,14 @@
        <source>Details</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">160</context>
+          <context context-type="linenumber">164</context>
        </context-group>
      </trans-unit>
      <trans-unit id="5701618810648052610" datatype="html">
        <source>Title</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">163</context>
+          <context context-type="linenumber">167</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
@@ -7790,14 +7790,14 @@
        <source>Date created</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">167</context>
+          <context context-type="linenumber">171</context>
        </context-group>
      </trans-unit>
      <trans-unit id="5607669932062416162" datatype="html">
        <source>Default</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">179</context>
+          <context context-type="linenumber">183</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/manage/saved-views/saved-views.component.html</context>
@@ -7808,14 +7808,14 @@
        <source>Content</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">286</context>
+          <context context-type="linenumber">290</context>
        </context-group>
      </trans-unit>
      <trans-unit id="218403386307979629" datatype="html">
        <source>Metadata</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">295</context>
+          <context context-type="linenumber">299</context>
        </context-group>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/metadata-collapse/metadata-collapse.component.ts</context>
@@ -7826,147 +7826,147 @@
        <source>Date modified</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">302</context>
+          <context context-type="linenumber">306</context>
        </context-group>
      </trans-unit>
      <trans-unit id="6392918669949841614" datatype="html">
        <source>Date added</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">306</context>
+          <context context-type="linenumber">310</context>
        </context-group>
      </trans-unit>
      <trans-unit id="146828917013192897" datatype="html">
        <source>Media filename</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">310</context>
+          <context context-type="linenumber">314</context>
        </context-group>
      </trans-unit>
      <trans-unit id="4500855521601039868" datatype="html">
        <source>Original filename</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">314</context>
+          <context context-type="linenumber">318</context>
        </context-group>
      </trans-unit>
      <trans-unit id="2659735245739197634" datatype="html">
        <source>Original SHA256 checksum</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">318</context>
+          <context context-type="linenumber">322</context>
        </context-group>
      </trans-unit>
      <trans-unit id="5888243105821763422" datatype="html">
        <source>Original file size</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">322</context>
+          <context context-type="linenumber">326</context>
        </context-group>
      </trans-unit>
      <trans-unit id="2696647325713149563" datatype="html">
        <source>Original mime type</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">326</context>
+          <context context-type="linenumber">330</context>
        </context-group>
      </trans-unit>
      <trans-unit id="6714358112223607756" datatype="html">
        <source>Archive SHA256 checksum</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">331</context>
+          <context context-type="linenumber">335</context>
        </context-group>
      </trans-unit>
      <trans-unit id="6033581412811562084" datatype="html">
        <source>Archive file size</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">337</context>
+          <context context-type="linenumber">341</context>
        </context-group>
      </trans-unit>
      <trans-unit id="6992781481378431874" datatype="html">
        <source>Original document metadata</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">346</context>
+          <context context-type="linenumber">350</context>
        </context-group>
      </trans-unit>
      <trans-unit id="2846565152091361585" datatype="html">
        <source>Archived document metadata</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">349</context>
+          <context context-type="linenumber">353</context>
        </context-group>
      </trans-unit>
      <trans-unit id="7206723502037428235" datatype="html">
        <source>Notes <x id="START_BLOCK_IF" equiv-text="@if (document()?.notes.length) {"/><x id="START_TAG_SPAN" ctype="x-span" equiv-text="&lt;span class=&quot;badge text-bg-secondary ms-1&quot;&gt;"/><x id="INTERPOLATION" equiv-text="length}}"/><x id="CLOSE_TAG_SPAN" ctype="x-span"/><x id="CLOSE_BLOCK_IF" equiv-text="}"/></source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">368,371</context>
+          <context context-type="linenumber">372,375</context>
        </context-group>
      </trans-unit>
      <trans-unit id="186236568870281953" datatype="html">
        <source>History</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">379</context>
+          <context context-type="linenumber">383</context>
        </context-group>
      </trans-unit>
      <trans-unit id="8236092845697214347" datatype="html">
        <source> Duplicates <x id="START_TAG_SPAN" ctype="x-span" equiv-text="&lt;span class=&quot;badge text-bg-secondary ms-1&quot;&gt;"/><x id="INTERPOLATION" equiv-text="cate_documents.length }}"/><x id="CLOSE_TAG_SPAN" ctype="x-span"/></source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">401,405</context>
+          <context context-type="linenumber">405,409</context>
        </context-group>
      </trans-unit>
      <trans-unit id="6449374629822973702" datatype="html">
        <source>Duplicate documents detected:</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">407</context>
+          <context context-type="linenumber">411</context>
        </context-group>
      </trans-unit>
      <trans-unit id="14058600336670816" datatype="html">
        <source>In trash</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">418</context>
+          <context context-type="linenumber">422</context>
        </context-group>
      </trans-unit>
      <trans-unit id="5129524307369213584" datatype="html">
        <source>Save &amp; next</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">447</context>
+          <context context-type="linenumber">451</context>
        </context-group>
      </trans-unit>
      <trans-unit id="4910102545766233758" datatype="html">
        <source>Save &amp; close</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">449</context>
+          <context context-type="linenumber">453</context>
        </context-group>
      </trans-unit>
      <trans-unit id="3823219296477075982" datatype="html">
        <source>Discard</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">451</context>
+          <context context-type="linenumber">455</context>
        </context-group>
      </trans-unit>
      <trans-unit id="1309556917227148591" datatype="html">
        <source>Document loading...</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">459</context>
+          <context context-type="linenumber">463</context>
        </context-group>
      </trans-unit>
      <trans-unit id="8191371354890763172" datatype="html">
        <source>Enter Password</source>
        <context-group purpose="location">
          <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.html</context>
-          <context context-type="linenumber">513</context>
+          <context context-type="linenumber">517</context>
        </context-group>
      </trans-unit>
      <trans-unit id="5758784066858623886" datatype="html">
@@ -21,7 +21,7 @@
  </div>
 </pngx-page-header>

-@if (!tasksService.completedFileTasks && tasksService.loading) {
+@if (loading() && pagedTasks().length === 0) {
  <div class="spinner-border spinner-border-sm fw-normal ms-2 me-auto" role="status"></div>
  <div class="visually-hidden" i18n>Loading...</div>
 }
@@ -129,13 +129,25 @@ describe('PngxPdfViewerComponent', () => {
    ;(component as any).applyScale()
    expect(viewer.currentScaleValue).toBe(PdfZoomScale.PageFit)
    expect(viewer.currentScale).toBe(2)
+  })

+  it('does not reapply scale for page-only changes', async () => {
+    await initComponent()
+
+    const pdf = (component as any).pdf as { numPages: number }
+    pdf.numPages = 3
+    const viewer = (component as any).pdfViewer as PDFViewer
+    viewer.setDocument(pdf)
    const applyScaleSpy = jest.spyOn(component as any, 'applyScale')
    component.page = 2
-    ;(component as any).lastViewerPage = 2
-    ;(component as any).applyViewerState()
+
+    component.ngOnChanges({
+      page: new SimpleChange(1, 2, false),
+    })
+
+    expect(viewer.currentPageNumber).toBe(2)
    expect((component as any).lastViewerPage).toBeUndefined()
-    expect(applyScaleSpy).toHaveBeenCalled()
+    expect(applyScaleSpy).not.toHaveBeenCalled()
  })

  it('does not reset the viewer when it is already on the requested page', async () => {
@@ -116,7 +116,10 @@ export class PngxPdfViewerComponent
      changes['zoomScale'] ||
      changes['rotation']
    ) {
-      this.applyViewerState()
+      // Prevent loop with page / scale application see https://github.com/paperless-ngx/paperless-ngx/issues/13404
+      this.applyViewerState(
+        !!(changes['zoom'] || changes['zoomScale'] || changes['rotation'])
+      )
    }

    if (changes['searchQuery']) {
@@ -240,7 +243,7 @@ export class PngxPdfViewerComponent
    }
  }

-  private applyViewerState(): void {
+  private applyViewerState(applyScale = true): void {
    if (!this.pdfViewer) {
      return
    }
@@ -264,7 +267,7 @@ export class PngxPdfViewerComponent
    if (this.page === this.lastViewerPage) {
      this.lastViewerPage = undefined
    }
-    if (hasPages) {
+    if (hasPages && applyScale) {
      this.applyScale()
    }
    this.dispatchFindIfReady()
@@ -113,8 +113,8 @@

    <form [formGroup]='documentForm' (ngSubmit)="save()">

-      <div class="btn-toolbar mb-1 border-bottom">
-        <div class="btn-group pb-3">
+      <div class="btn-toolbar justify-content-end mb-1 pb-3 gap-2 row-gap-2 border-bottom">
+        <div class="btn-group me-auto">
          <button type="button" class="btn btn-sm btn-outline-secondary" i18n-title title="Close" (click)="close()">
            <i-bs width="1.2em" height="1.2em" name="x"></i-bs>
          </button>
@@ -127,32 +127,36 @@
        </div>

        <ng-container *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }">
-          <div class="btn-group pb-3 ms-auto">
-            <pngx-suggestions-dropdown *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }"
-              [disabled]="!userCanEdit || suggestionsLoading()"
-              [loading]="suggestionsLoading()"
-              [suggestions]="suggestions()"
-              [aiEnabled]="aiEnabled"
-              (getSuggestions)="getSuggestions()"
-              (addTag)="createTag($event)"
-              (addDocumentType)="createDocumentType($event)"
-              (addCorrespondent)="createCorrespondent($event)">
-            </pngx-suggestions-dropdown>
+          <div class="d-flex gap-2">
+            <div class="btn-group">
+              <pngx-suggestions-dropdown *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }"
+                [disabled]="!userCanEdit || suggestionsLoading()"
+                [loading]="suggestionsLoading()"
+                [suggestions]="suggestions()"
+                [aiEnabled]="aiEnabled"
+                (getSuggestions)="getSuggestions()"
+                (addTag)="createTag($event)"
+                (addDocumentType)="createDocumentType($event)"
+                (addCorrespondent)="createCorrespondent($event)">
+              </pngx-suggestions-dropdown>
+            </div>
+
+            <div class="btn-group">
+              <pngx-custom-fields-dropdown
+                *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.CustomField }"
+                [documentId]="documentId()"
+                [disabled]="!userCanEdit"
+                [existingFields]="document()?.custom_fields"
+                (created)="refreshCustomFields()"
+                (added)="addField($event)">
+              </pngx-custom-fields-dropdown>
+            </div>
          </div>

-          <div class="btn-group pb-3 ms-2">
-            <pngx-custom-fields-dropdown
-              *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.CustomField }"
-              [documentId]="documentId()"
-              [disabled]="!userCanEdit"
-              [existingFields]="document()?.custom_fields"
-              (created)="refreshCustomFields()"
-              (added)="addField($event)">
-            </pngx-custom-fields-dropdown>
+          <div class="ps-3">
+            <ng-container *ngTemplateOutlet="saveButtons"></ng-container>
          </div>
        </ng-container>
-
-        <ng-container *ngTemplateOutlet="saveButtons"></ng-container>
      </div>

      <ul ngbNav #nav="ngbNav" class="nav-underline flex-nowrap flex-md-wrap overflow-auto" (navChange)="onNavChange($event)" [activeId]="activeNavID()" (activeIdChange)="activeNavID.set($event)">
@@ -276,7 +280,7 @@
              }
            </div>

-            <div class="d-flex border-top pt-3">
+            <div class="d-flex justify-content-end border-top pt-3">
              <ng-container *ngTemplateOutlet="saveButtons"></ng-container>
            </div>
          </ng-template>
@@ -440,7 +444,7 @@
 </div>

 <ng-template #saveButtons>
-  <div class="btn-group pb-3 ms-4">
+  <div class="btn-group">
    <ng-container *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }">
      <button type="submit" class="order-3 btn btn-sm btn-primary" i18n [disabled]="!userCanEdit || networkActive() || (isDirty$ | async) !== true">Save</button>
      @if (hasNext()) {
@@ -2269,4 +2269,34 @@ describe('FilterEditorComponent', () => {
    component.itemSelected({ item: 'world', preventDefault: () => true })
    expect(component.textFilter).toEqual('hello world ')
  })
+
+  it('should choose the active autocomplete item with Enter', () => {
+    component.textFilterTarget = 'fulltext-query'
+    jest
+      .spyOn(searchService, 'autocomplete')
+      .mockReturnValue(of(['hello', 'help']))
+
+    const input = component.textFilterInput.nativeElement as HTMLInputElement
+    input.value = 'he'
+    input.dispatchEvent(new Event('input'))
+    tick(250)
+
+    input.dispatchEvent(
+      new KeyboardEvent('keydown', {
+        key: 'ArrowDown',
+        bubbles: true,
+        cancelable: true,
+      })
+    )
+    input.dispatchEvent(
+      new KeyboardEvent('keydown', {
+        key: 'Enter',
+        bubbles: true,
+        cancelable: true,
+      })
+    )
+    fixture.detectChanges()
+
+    expect(component.textFilter).toEqual('help ')
+  })
 })
@@ -1314,6 +1314,10 @@ export class FilterEditorComponent

  textFilterKeydown(event: KeyboardEvent) {
    if (event.key == 'Enter') {
+      if (event.defaultPrevented) {
+        // NgbTypeahead calls preventDefault, so use that to detect if the Enter key was for the dropdown
+        return
+      }
      const filterString = (
        this.textFilterInput.nativeElement as HTMLInputElement
      ).value
@@ -50,13 +50,66 @@ describe('TasksService', () => {
    req.flush({ count: 0, results: [] })
  })

-  it('does not call tasks api endpoint on reload if already loading', () => {
-    tasksService.loading = true
+  it('cancels an in-progress reload when reloading again', () => {
    tasksService.reload()
-    httpTestingController.expectNone(
+    const staleReload = httpTestingController.expectOne(
      (req: HttpRequest<unknown>) =>
        req.url === `${environment.apiBaseUrl}tasks/`
    )
+    tasksService.reload()
+
+    expect(staleReload.cancelled).toBe(true)
+    httpTestingController
+      .expectOne(
+        (req: HttpRequest<unknown>) =>
+          req.url === `${environment.apiBaseUrl}tasks/`
+      )
+      .flush({ count: 0, results: [] })
+  })
+
+  it('continues reloading after a reload request fails', () => {
+    tasksService.reload()
+    httpTestingController
+      .expectOne(
+        (req: HttpRequest<unknown>) =>
+          req.url === `${environment.apiBaseUrl}tasks/`
+      )
+      .flush('error', { status: 500, statusText: 'error' })
+
+    expect(tasksService.loading).toBe(false)
+
+    tasksService.reload()
+    httpTestingController
+      .expectOne(
+        (req: HttpRequest<unknown>) =>
+          req.url === `${environment.apiBaseUrl}tasks/`
+      )
+      .flush({ count: 0, results: [] })
+  })
+
+  it('reloads after dismissing a task while a reload is already in progress', () => {
+    tasksService.reload()
+    const staleReload = httpTestingController.expectOne(
+      (req: HttpRequest<unknown>) =>
+        req.url === `${environment.apiBaseUrl}tasks/` &&
+        req.params.get('acknowledged') === 'false'
+    )
+
+    tasksService.dismissTasks(new Set([1])).subscribe()
+    httpTestingController
+      .expectOne(`${environment.apiBaseUrl}tasks/acknowledge/`)
+      .flush([])
+
+    expect(staleReload.cancelled).toBe(true)
+    httpTestingController
+      .expectOne(
+        (req: HttpRequest<unknown>) =>
+          req.url === `${environment.apiBaseUrl}tasks/` &&
+          req.params.get('acknowledged') === 'false'
+      )
+      .flush({ count: 0, results: [] })
+
+    expect(tasksService.needsAttentionTasks).toHaveLength(0)
  })

  it('calls acknowledge_tasks api endpoint on dismiss and reloads', () => {
@@ -101,79 +154,6 @@ describe('TasksService', () => {
      .flush({ count: 0, results: [] })
  })

-  it('groups mixed task types by status when reloading', () => {
-    expect(tasksService.total).toEqual(0)
-    const mockTasks = [
-      {
-        task_type: PaperlessTaskType.ConsumeFile,
-        trigger_source: PaperlessTaskTriggerSource.FolderConsume,
-        status: PaperlessTaskStatus.Success,
-        acknowledged: false,
-        task_id: '1234',
-        input_data: { filename: 'file1.pdf' },
-        date_created: new Date(),
-        related_document_ids: [],
-      },
-      {
-        task_type: PaperlessTaskType.SanityCheck,
-        trigger_source: PaperlessTaskTriggerSource.System,
-        status: PaperlessTaskStatus.Failure,
-        acknowledged: false,
-        task_id: '1235',
-        input_data: {},
-        date_created: new Date(),
-        related_document_ids: [],
-      },
-      {
-        task_type: PaperlessTaskType.MailFetch,
-        trigger_source: PaperlessTaskTriggerSource.Scheduled,
-        status: PaperlessTaskStatus.Pending,
-        acknowledged: false,
-        task_id: '1236',
-        input_data: {},
-        date_created: new Date(),
-        related_document_ids: [],
-      },
-      {
-        task_type: PaperlessTaskType.LlmIndex,
-        trigger_source: PaperlessTaskTriggerSource.WebUI,
-        status: PaperlessTaskStatus.Started,
-        acknowledged: false,
-        task_id: '1237',
-        input_data: {},
-        date_created: new Date(),
-        related_document_ids: [],
-      },
-      {
-        task_type: PaperlessTaskType.EmptyTrash,
-        trigger_source: PaperlessTaskTriggerSource.Manual,
-        status: PaperlessTaskStatus.Success,
-        acknowledged: false,
-        task_id: '1238',
-        input_data: {},
-        date_created: new Date(),
-        related_document_ids: [],
-      },
-    ]
-
-    tasksService.reload()
-
-    const req = httpTestingController.expectOne(
-      (req: HttpRequest<unknown>) =>
-        req.url === `${environment.apiBaseUrl}tasks/` &&
-        req.params.get('acknowledged') === 'false' &&
-        req.params.get('page_size') === '1000'
-    )
-
-    req.flush({ count: mockTasks.length, results: mockTasks })
-
-    expect(tasksService.allFileTasks).toHaveLength(5)
-    expect(tasksService.completedFileTasks).toHaveLength(2)
-    expect(tasksService.failedFileTasks).toHaveLength(1)
-    expect(tasksService.queuedFileTasks).toHaveLength(1)
-    expect(tasksService.startedFileTasks).toHaveLength(1)
-  })
-
  it('includes revoked tasks in needs attention', () => {
    const mockTasks = [
      {
@@ -1,7 +1,15 @@
 import { HttpClient } from '@angular/common/http'
 import { Injectable, inject, signal } from '@angular/core'
-import { Observable, Subject } from 'rxjs'
-import { first, map, takeUntil, tap } from 'rxjs/operators'
+import { EMPTY, Observable, Subject } from 'rxjs'
+import {
+  catchError,
+  finalize,
+  first,
+  map,
+  switchMap,
+  takeUntil,
+  tap,
+} from 'rxjs/operators'
 import {
  PaperlessTask,
  PaperlessTaskStatus,
@@ -23,44 +31,40 @@ export class TasksService {

  public loading: boolean = false

-  private readonly fileTasks = signal<PaperlessTask[]>([])
+  private readonly tasks = signal<PaperlessTask[]>([])
+  private readonly reloadNotifier = new Subject<void>()

  private unsubscribeNotifer: Subject<any> = new Subject()

-  public get total(): number {
-    return this.fileTasks().length
-  }
-
-  public get allFileTasks(): PaperlessTask[] {
-    return this.fileTasks().slice(0)
-  }
-
-  public get queuedFileTasks(): PaperlessTask[] {
-    return this.fileTasks().filter(
-      (t) => t.status === PaperlessTaskStatus.Pending
-    )
-  }
-
-  public get startedFileTasks(): PaperlessTask[] {
-    return this.fileTasks().filter(
-      (t) => t.status === PaperlessTaskStatus.Started
-    )
-  }
-
-  public get completedFileTasks(): PaperlessTask[] {
-    return this.fileTasks().filter(
-      (t) => t.status === PaperlessTaskStatus.Success
-    )
-  }
-
-  public get failedFileTasks(): PaperlessTask[] {
-    return this.fileTasks().filter(
-      (t) => t.status === PaperlessTaskStatus.Failure
-    )
+  constructor() {
+    this.reloadNotifier
+      .pipe(
+        switchMap(() => {
+          this.loading = true
+          return this.http
+            .get<Results<PaperlessTask>>(`${this.baseUrl}${this.endpoint}/`, {
+              params: {
+                acknowledged: 'false',
+                page_size: this.defaultReloadPageSize,
+              },
+            })
+            .pipe(
+              map((response) => response.results),
+              takeUntil(this.unsubscribeNotifer),
+              catchError(() => EMPTY),
+              finalize(() => {
+                this.loading = false
+              })
+            )
+        })
+      )
+      .subscribe((tasks) => {
+        this.tasks.set(tasks)
+      })
  }

  public get needsAttentionTasks(): PaperlessTask[] {
-    return this.fileTasks().filter((t) =>
+    return this.tasks().filter((t) =>
      [PaperlessTaskStatus.Failure, PaperlessTaskStatus.Revoked].includes(
        t.status
      )
@@ -68,22 +72,7 @@ export class TasksService {
  }

  public reload() {
-    if (this.loading) return
-    this.loading = true
-
-    this.http
-      .get<Results<PaperlessTask>>(`${this.baseUrl}${this.endpoint}/`, {
-        params: {
-          acknowledged: 'false',
-          page_size: this.defaultReloadPageSize,
-        },
-      })
-      .pipe(map((r) => r.results))
-      .pipe(takeUntil(this.unsubscribeNotifer), first())
-      .subscribe((r) => {
-        this.fileTasks.set(r)
-        this.loading = false
-      })
+    this.reloadNotifier.next()
  }

  public list(
@@ -36,6 +36,9 @@ def send_email(

    TODO: re-evaluate this pending https://code.djangoproject.com/ticket/35581 / https://github.com/django/django/pull/18966
    """
+    if "\r" in subject or "\n" in subject:
+        subject = " ".join(line.strip(" \t") for line in subject.splitlines())
+
    email = EmailMessage(
        subject=subject,
        body=body,
@@ -386,10 +386,19 @@ class Command(CryptMixin, PaperlessCommand):
                raise DeserializationError(
                    f"{model.__name__} has no updatable fields; PK-only models are not supported by the importer",
                )
+            # MySQL/MariaDB support upserts via ON DUPLICATE KEY UPDATE but,
+            # unlike PostgreSQL/SQLite, cannot target a specific unique field
+            # for the conflict -- passing unique_fields there raises
+            # NotSupportedError.
+            unique_fields = (
+                [model._meta.pk.attname]
+                if connection.features.supports_update_conflicts_with_target
+                else None
+            )
            model.objects.bulk_create(  # type: ignore[attr-defined]
                instances,
                update_conflicts=True,
-                unique_fields=[model._meta.pk.attname],
+                unique_fields=unique_fields,
                update_fields=update_fields,
            )
            loaded_models.add(model)
@@ -43,8 +43,16 @@ def _fmt(dt: datetime) -> str:


 def _iso_range(lo: datetime, hi: datetime) -> str:
-    """Format a [lo TO hi] range string in ISO 8601 for Tantivy query syntax."""
-    return f"[{_fmt(lo)} TO {_fmt(hi)}]"
+    """
+    Format a half-open ``[lo TO hi)`` range in ISO 8601 for Tantivy query syntax.
+
+    ``hi`` is always the exclusive ceiling of a computed period (the start of
+    the *next* day/week/month/quarter/year), so the closing bracket must be
+    the Tantivy exclusive-range brace ``}`` rather than ``]`` — otherwise the
+    first instant of the following period (e.g. the 1st of next month) is
+    incorrectly included in the match.
+    """
+    return f"[{_fmt(lo)} TO {_fmt(hi)}}}"


 def _quarter_start(d: date) -> date:
@@ -566,6 +566,17 @@ def translate_range(field: str, lo: str, hi: str, tz: tzinfo) -> str:
        lo_pair, hi_pair = hi_pair, lo_pair

    lo_iso = _fmt(lo_pair[0]) if lo_pair is not None else OPEN_LO
-    hi_iso = _fmt(hi_pair[1]) if hi_pair is not None else OPEN_HI

-    return f"{field}:[{lo_iso} TO {hi_iso}]"
+    # A bound resolves to (floor, ceil) where floor == ceil for an exact instant
+    # (a full ISO datetime, "now", or a "+/-N unit" offset) and floor != ceil for
+    # a coarser period token (year/month/day precision). Only the latter needs a
+    # half-open close: its ceil is the start of the *next* period and must be
+    # excluded, or that instant (e.g. the 1st of next month) wrongly matches.
+    if hi_pair is not None:
+        hi_iso = _fmt(hi_pair[1])
+        hi_close = "]" if hi_pair[0] == hi_pair[1] else "}"
+    else:
+        hi_iso = OPEN_HI
+        hi_close = "]"
+
+    return f"{field}:[{lo_iso} TO {hi_iso}{hi_close}"
@@ -1134,6 +1134,11 @@ def before_task_publish_handler(
        return

    try:
+        # Close stale connections without disrupting a transaction publishing a task
+        for connection in connections.all(initialized_only=True):
+            if not connection.in_atomic_block:
+                connection.close_if_unusable_or_obsolete()
+
        _, task_kwargs, _ = body
        task_id = headers["id"]

@@ -32,7 +32,9 @@ AUCKLAND = ZoneInfo("Pacific/Auckland")  # UTC+13 in southern-hemisphere summer


 def _range(result: str, field: str) -> tuple[str, str]:
-    m = re.search(rf"{field}:\[(.+?) TO (.+?)\]", result)
+    # Half-open period ranges close with "}" (exclusive); exact-instant ranges
+    # (full ISO datetimes, "now", relative offsets) close with "]" (inclusive).
+    m = re.search(rf"{field}:\[(.+?) TO (.+?)[\]}}]", result)
    assert m, f"No range for {field!r} in: {result!r}"
    return m.group(1), m.group(2)

@@ -214,27 +214,27 @@ class TestTranslateScalar:
            (
                "created",
                "2020",
-                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
            ),
            (
                "created",
                "202003",
-                "created:[2020-03-01T00:00:00Z TO 2020-04-01T00:00:00Z]",
+                "created:[2020-03-01T00:00:00Z TO 2020-04-01T00:00:00Z}",
            ),
            (
                "created",
                "20200115",
-                "created:[2020-01-15T00:00:00Z TO 2020-01-16T00:00:00Z]",
+                "created:[2020-01-15T00:00:00Z TO 2020-01-16T00:00:00Z}",
            ),
            (
                "created",
                "2020-01-15",
-                "created:[2020-01-15T00:00:00Z TO 2020-01-16T00:00:00Z]",
+                "created:[2020-01-15T00:00:00Z TO 2020-01-16T00:00:00Z}",
            ),
            (
                "created",
                "2020-03",
-                "created:[2020-03-01T00:00:00Z TO 2020-04-01T00:00:00Z]",
+                "created:[2020-03-01T00:00:00Z TO 2020-04-01T00:00:00Z}",
            ),
        ],
    )
@@ -248,9 +248,9 @@ class TestTranslateScalar:
        assert exc_info.value.value == "202023"

    def test_keyword_delegates(self) -> None:
-        # keyword path produces a range; just assert it is a created range
+        # keyword path produces a half-open range; just assert it is a created range
        out = translate_scalar("created", "today", UTC)
-        assert out.startswith("created:[") and out.endswith("]")
+        assert out.startswith("created:[") and out.endswith("}")

    def test_14digit_compact_datetime(self) -> None:
        out = translate_scalar("created", "20240115120000", UTC)
@@ -279,21 +279,21 @@ class TestTranslateRange:
    @pytest.mark.parametrize(
        ("lo", "hi", "expected"),
        [
-            ("2005", "2009", "created:[2005-01-01T00:00:00Z TO 2010-01-01T00:00:00Z]"),
+            ("2005", "2009", "created:[2005-01-01T00:00:00Z TO 2010-01-01T00:00:00Z}"),
            (
                "202001",
                "202006",
-                "created:[2020-01-01T00:00:00Z TO 2020-07-01T00:00:00Z]",
+                "created:[2020-01-01T00:00:00Z TO 2020-07-01T00:00:00Z}",
            ),
            (
                "20200101",
                "20201231",
-                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
            ),
            (
                "2020-01-01",
                "2020-12-31",
-                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
            ),
        ],
    )
@@ -302,7 +302,7 @@ class TestTranslateRange:

    def test_reversed_swaps(self):
        assert translate_range("created", "2009", "2005", UTC) == (
-            "created:[2005-01-01T00:00:00Z TO 2010-01-01T00:00:00Z]"
+            "created:[2005-01-01T00:00:00Z TO 2010-01-01T00:00:00Z}"
        )

    def test_open_upper(self):
@@ -311,7 +311,7 @@ class TestTranslateRange:

    def test_open_lower(self):
        out = translate_range("created", "", "2020", UTC)
-        assert out == f"created:[{OPEN_LO} TO 2021-01-01T00:00:00Z]"
+        assert out == f"created:[{OPEN_LO} TO 2021-01-01T00:00:00Z}}"

    def test_invalid_bound_raises(self):
        with pytest.raises(InvalidDateQuery) as exc_info:
@@ -334,16 +334,16 @@ class TestTranslateQuery:
        [
            (
                "created:2020",
-                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
            ),
            ("tag:foo,bar", "tag:foo AND tag:bar"),
            # 'type' is a user-facing alias rewritten to 'document_type' (the real schema field)
            ("tag:foo,type:bar", "tag:foo AND document_type:bar"),
            (
                "created:[2020 TO 2021],added:[2022 TO 2023]",
-                "created:[2020-01-01T00:00:00Z TO 2022-01-01T00:00:00Z]"
+                "created:[2020-01-01T00:00:00Z TO 2022-01-01T00:00:00Z}"
                " AND "
-                "added:[2022-01-01T00:00:00Z TO 2024-01-01T00:00:00Z]",
+                "added:[2022-01-01T00:00:00Z TO 2024-01-01T00:00:00Z}",
            ),
            # correspondent is not multi-value: comma stays literal inside the value
            ("correspondent:foo,bar", "correspondent:foo,bar"),
@@ -506,7 +506,7 @@ class TestOperatorNormalization:
    def test_date_range_preserved(self) -> None:
        out = translate_query("created:[2020 TO 2021]", UTC)
        # Must not corrupt the ISO range
-        assert out == "created:[2020-01-01T00:00:00Z TO 2022-01-01T00:00:00Z]"
+        assert out == "created:[2020-01-01T00:00:00Z TO 2022-01-01T00:00:00Z}"

    def test_date_scalar_with_or(self) -> None:
        out = translate_query("created:2020 OR foo", UTC)
@@ -581,42 +581,42 @@ class TestKeywordDateResolution:
        [
            pytest.param(
                "today",
-                "created:[2026-03-28T00:00:00Z TO 2026-03-29T00:00:00Z]",
+                "created:[2026-03-28T00:00:00Z TO 2026-03-29T00:00:00Z}",
                id="today",
            ),
            pytest.param(
                "yesterday",
-                "created:[2026-03-27T00:00:00Z TO 2026-03-28T00:00:00Z]",
+                "created:[2026-03-27T00:00:00Z TO 2026-03-28T00:00:00Z}",
                id="yesterday",
            ),
            pytest.param(
                "previous week",
-                "created:[2026-03-16T00:00:00Z TO 2026-03-23T00:00:00Z]",
+                "created:[2026-03-16T00:00:00Z TO 2026-03-23T00:00:00Z}",
                id="previous-week",
            ),
            pytest.param(
                "this month",
-                "created:[2026-03-01T00:00:00Z TO 2026-04-01T00:00:00Z]",
+                "created:[2026-03-01T00:00:00Z TO 2026-04-01T00:00:00Z}",
                id="this-month",
            ),
            pytest.param(
                "previous month",
-                "created:[2026-02-01T00:00:00Z TO 2026-03-01T00:00:00Z]",
+                "created:[2026-02-01T00:00:00Z TO 2026-03-01T00:00:00Z}",
                id="previous-month",
            ),
            pytest.param(
                "this year",
-                "created:[2026-01-01T00:00:00Z TO 2027-01-01T00:00:00Z]",
+                "created:[2026-01-01T00:00:00Z TO 2027-01-01T00:00:00Z}",
                id="this-year",
            ),
            pytest.param(
                "previous year",
-                "created:[2025-01-01T00:00:00Z TO 2026-01-01T00:00:00Z]",
+                "created:[2025-01-01T00:00:00Z TO 2026-01-01T00:00:00Z}",
                id="previous-year",
            ),
            pytest.param(
                "previous quarter",
-                "created:[2025-10-01T00:00:00Z TO 2026-01-01T00:00:00Z]",
+                "created:[2025-10-01T00:00:00Z TO 2026-01-01T00:00:00Z}",
                id="previous-quarter",
            ),
        ],
@@ -637,42 +637,42 @@ class TestKeywordDateResolution:
        [
            pytest.param(
                "today",
-                "added:[2026-03-27T15:00:00Z TO 2026-03-28T15:00:00Z]",
+                "added:[2026-03-27T15:00:00Z TO 2026-03-28T15:00:00Z}",
                id="today",
            ),
            pytest.param(
                "yesterday",
-                "added:[2026-03-26T15:00:00Z TO 2026-03-27T15:00:00Z]",
+                "added:[2026-03-26T15:00:00Z TO 2026-03-27T15:00:00Z}",
                id="yesterday",
            ),
            pytest.param(
                "previous week",
-                "added:[2026-03-15T15:00:00Z TO 2026-03-22T15:00:00Z]",
+                "added:[2026-03-15T15:00:00Z TO 2026-03-22T15:00:00Z}",
                id="previous-week",
            ),
            pytest.param(
                "this month",
-                "added:[2026-02-28T15:00:00Z TO 2026-03-31T15:00:00Z]",
+                "added:[2026-02-28T15:00:00Z TO 2026-03-31T15:00:00Z}",
                id="this-month",
            ),
            pytest.param(
                "previous month",
-                "added:[2026-01-31T15:00:00Z TO 2026-02-28T15:00:00Z]",
+                "added:[2026-01-31T15:00:00Z TO 2026-02-28T15:00:00Z}",
                id="previous-month",
            ),
            pytest.param(
                "this year",
-                "added:[2025-12-31T15:00:00Z TO 2026-12-31T15:00:00Z]",
+                "added:[2025-12-31T15:00:00Z TO 2026-12-31T15:00:00Z}",
                id="this-year",
            ),
            pytest.param(
                "previous year",
-                "added:[2024-12-31T15:00:00Z TO 2025-12-31T15:00:00Z]",
+                "added:[2024-12-31T15:00:00Z TO 2025-12-31T15:00:00Z}",
                id="previous-year",
            ),
            pytest.param(
                "previous quarter",
-                "added:[2025-09-30T15:00:00Z TO 2025-12-31T15:00:00Z]",
+                "added:[2025-09-30T15:00:00Z TO 2025-12-31T15:00:00Z}",
                id="previous-quarter",
            ),
        ],
@@ -719,7 +719,7 @@ class TestISODatetimeBounds:
    def test_translate_query_text_before_comma_separated_date_clause(self) -> None:
        result = translate_query("schäfersee,created:previous year", UTC)
        assert result == (
-            "schäfersee AND created:[2025-01-01T00:00:00Z TO 2026-01-01T00:00:00Z]"
+            "schäfersee AND created:[2025-01-01T00:00:00Z TO 2026-01-01T00:00:00Z}"
        )

    def test_invalid_iso_datetime_raises(self) -> None:
@@ -75,7 +75,7 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
                {
                    "documents": [self.doc1.pk, self.doc2.pk],
                    "addresses": "hello@paperless-ngx.com,test@example.com",
-                    "subject": "Bulk email test",
+                    "subject": "Bulk email\n test",
                    "message": "Here are your documents",
                },
            ),
@@ -720,6 +720,48 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
        self.assertEqual(results[0]["id"], 3)
        self.assertEqual(results[0]["title"], "bank statement 3")

+    def test_search_added_previous_month_excludes_next_period_start(self) -> None:
+        """
+        GIVEN:
+            - One document added at the last instant of last month
+            - One document added exactly at the first instant of this month
+        WHEN:
+            - Query for documents added in the previous month
+        THEN:
+            - Only the document from last month is returned; the document dated
+              exactly at the start of this month (the exclusive upper bound of
+              the range) is not
+        """
+        d1 = DocumentFactory.create(
+            title="end of last month",
+            content="last instant of last month",
+            checksum="A",
+            pk=1,
+            added=timezone.make_aware(datetime.datetime(2024, 1, 31, 23, 59, 59)),
+        )
+        d2 = DocumentFactory.create(
+            title="start of this month",
+            content="first instant of this month",
+            checksum="B",
+            pk=2,
+            added=timezone.make_aware(datetime.datetime(2024, 2, 1, 0, 0, 0)),
+        )
+
+        backend = get_backend()
+        backend.add_or_update(d1)
+        backend.add_or_update(d2)
+
+        with time_machine.travel(
+            timezone.make_aware(datetime.datetime(2024, 2, 15, 12, 0, 0)),
+            tick=False,
+        ):
+            response = self.client.get("/api/documents/?query=added:previous month")
+        results = response.data["results"]
+
+        self.assertEqual(len(results), 1)
+        self.assertEqual(results[0]["id"], 1)
+        self.assertEqual(results[0]["title"], "end of last month")
+
    def test_search_added_invalid_date(self) -> None:
        """
        GIVEN:
@@ -56,6 +56,32 @@ def send_publish(

@pytest.mark.django_db
 class TestBeforeTaskPublishHandler:
+    @mock.patch("documents.signals.handlers.connections.all")
+    def test_closes_old_connections_outside_atomic_blocks(
+        self,
+        connections_all,
+    ) -> None:
+        connection = mock.Mock(in_atomic_block=False)
+        connections_all.return_value = [connection]
+
+        task_id = send_publish("documents.tasks.train_classifier", (), {})
+
+        connection.close_if_unusable_or_obsolete.assert_called_once_with()
+        assert PaperlessTask.objects.filter(task_id=task_id).exists()
+
+    @mock.patch("documents.signals.handlers.connections.all")
+    def test_keeps_connections_open_inside_atomic_blocks(
+        self,
+        connections_all,
+    ) -> None:
+        connection = mock.Mock(in_atomic_block=True)
+        connections_all.return_value = [connection]
+
+        task_id = send_publish("documents.tasks.train_classifier", (), {})
+
+        connection.close_if_unusable_or_obsolete.assert_not_called()
+        assert PaperlessTask.objects.filter(task_id=task_id).exists()
+
    def test_creates_task_for_consume_file(
        self,
        consume_input_doc,
@@ -150,6 +150,7 @@ INSTALLED_APPS = [
    "drf_spectacular",
    "drf_spectacular_sidecar",
    "treenode",
+    "paperless_benchmark.apps.PaperlessBenchmarkConfig",
    *env_apps,
 ]

@@ -0,0 +1,8 @@
+from django.apps import AppConfig
+from django.utils.translation import gettext_lazy as _
+
+
+class PaperlessBenchmarkConfig(AppConfig):
+    name = "paperless_benchmark"
+
+    verbose_name = _("Paperless benchmark")
@@ -0,0 +1,136 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from django.db import connection
+
+if TYPE_CHECKING:
+    from django.db.models import QuerySet
+
+
+def _reset_table_names() -> list[str]:
+    from guardian.models import GroupObjectPermission
+    from guardian.models import UserObjectPermission
+
+    from documents.models import Correspondent
+    from documents.models import Document
+    from documents.models import DocumentType
+    from documents.models import StoragePath
+    from documents.models import Tag
+
+    return [
+        Document.tags.through._meta.db_table,
+        Document._meta.db_table,
+        Tag._meta.db_table,
+        Correspondent._meta.db_table,
+        DocumentType._meta.db_table,
+        StoragePath._meta.db_table,
+        UserObjectPermission._meta.db_table,
+        GroupObjectPermission._meta.db_table,
+    ]
+
+
+def _delete_all_users_and_groups() -> None:
+    # ASSUMPTION: this tool assumes a disposable benchmark database, never
+    # point it at a real install. This deletes EVERY user and group in the
+    # database (not just benchmark-created ones) -- there is no way to
+    # distinguish "real" users from seeded ones, so this is only safe against
+    # a database that exists solely to run this benchmarking tool. The
+    # `benchmark seed --reset` CLI path requires an explicit
+    # `--yes-i-know-this-wipes-the-database` flag before reaching here; do
+    # not remove that guard.
+    from django.contrib.auth import get_user_model
+    from django.contrib.auth.models import Group
+
+    get_user_model().objects.all().delete()
+    Group.objects.all().delete()
+
+
+def _reset_postgresql() -> None:
+    tables = _reset_table_names()
+    with connection.cursor() as cursor:
+        cursor.execute(f"TRUNCATE TABLE {', '.join(tables)} RESTART IDENTITY CASCADE;")
+    _delete_all_users_and_groups()
+
+
+def _reset_mariadb() -> None:
+    # MariaDB's TRUNCATE has no CASCADE clause and refuses to truncate a
+    # table referenced by a foreign key while checks are enabled, so
+    # checks are disabled for the duration of the reset.
+    tables = _reset_table_names()
+    with connection.cursor() as cursor:
+        cursor.execute("SET FOREIGN_KEY_CHECKS = 0;")
+        try:
+            for table in tables:
+                cursor.execute(f"TRUNCATE TABLE {table};")
+        finally:
+            cursor.execute("SET FOREIGN_KEY_CHECKS = 1;")
+    _delete_all_users_and_groups()
+
+
+def _reset_sqlite() -> None:
+    from documents.models import Correspondent
+    from documents.models import Document
+    from documents.models import DocumentType
+    from documents.models import StoragePath
+    from documents.models import Tag
+
+    Document.global_objects.all().delete()
+    Tag.objects.all().delete()
+    Correspondent.objects.all().delete()
+    DocumentType.objects.all().delete()
+    StoragePath.objects.all().delete()
+    _delete_all_users_and_groups()
+
+
+def reset_benchmark_data() -> None:
+    """
+    Remove all previously-seeded benchmark data (documents, tags,
+    correspondents, document types, storage paths, guardian permission
+    rows, users, and groups) so a fresh `benchmark seed` run starts
+    from an empty slate. Dispatches per-backend because TRUNCATE syntax
+    and cascade behavior differ across the 3 supported databases.
+    """
+    if connection.vendor == "postgresql":
+        _reset_postgresql()
+    elif connection.vendor == "mysql":
+        # MariaDB also reports vendor == "mysql" under Django's mysql backend.
+        _reset_mariadb()
+    else:
+        _reset_sqlite()
+
+
+def capture_explain(queryset: QuerySet) -> str:
+    """
+    Return the query plan for `queryset` using the current backend's
+    explain facility. PostgreSQL supports `EXPLAIN ANALYZE {sql}` (real
+    execution stats). MariaDB does NOT accept that syntax -- verified
+    against a real MariaDB 12.3 container: `EXPLAIN ANALYZE {sql}` raises a
+    1064 syntax error, while MariaDB's own `ANALYZE {sql}` form (no
+    `EXPLAIN` keyword) works and returns real per-row execution stats
+    (`r_rows`, `r_filtered`, etc. columns) -- this is MariaDB's
+    EXPLAIN-ANALYZE-equivalent, distinct from MySQL 8.0.18+'s
+    `EXPLAIN ANALYZE` syntax, which MariaDB does not implement. SQLite only
+    supports EXPLAIN QUERY PLAN (the chosen plan, not real timing/row
+    counts) -- that output is clearly labeled rather than silently looking
+    equivalent to the other two backends' output.
+    """
+    sql, params = queryset.query.sql_with_params()
+    with connection.cursor() as cursor:
+        if connection.vendor == "postgresql":
+            cursor.execute(f"EXPLAIN ANALYZE {sql}", params)
+            return "\n".join(str(row[0]) for row in cursor.fetchall())
+        if connection.vendor == "mysql":
+            # MariaDB also reports vendor == "mysql" under Django's mysql
+            # backend. Unlike MySQL 8.0.18+, MariaDB has no `EXPLAIN
+            # ANALYZE` syntax -- its equivalent is `ANALYZE <statement>`.
+            cursor.execute(f"ANALYZE {sql}", params)
+            columns = [c[0] for c in cursor.description]
+            header = " | ".join(columns)
+            rows = "\n".join(
+                " | ".join(str(c) for c in row) for row in cursor.fetchall()
+            )
+            return f"{header}\n{rows}"
+        cursor.execute(f"EXPLAIN QUERY PLAN {sql}", params)
+        rows = "\n".join(" | ".join(str(c) for c in row) for row in cursor.fetchall())
+        return f"(plan only -- no execution stats on SQLite)\n{rows}"
@@ -0,0 +1,81 @@
+# src/paperless_benchmark/endpoints.py
+from __future__ import annotations
+
+import statistics
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from django.contrib.auth.models import User
+    from rest_framework.test import APIClient
+
+ENDPOINTS: tuple[tuple[str, str], ...] = (
+    ("documents_default", "/api/documents/"),
+    ("documents_page50", "/api/documents/?page_size=50"),
+    ("tags_all", "/api/tags/?page_size=100000"),
+)
+
+
+@dataclass(frozen=True, slots=True)
+class EndpointTiming:
+    user_label: str
+    endpoint_name: str
+    query_count: int
+    min_ms: float
+    median_ms: float
+    max_ms: float
+
+
+def _timed_requests(client: APIClient, url: str, n: int) -> list[float]:
+    times = []
+    for _ in range(n):
+        t0 = time.perf_counter()
+        resp = client.get(url)
+        t1 = time.perf_counter()
+        if resp.status_code != 200:
+            raise RuntimeError(
+                f"GET {url} -> {resp.status_code}: {resp.content[:300]!r}",
+            )
+        times.append(t1 - t0)
+    return times
+
+
+def _query_count(client: APIClient, url: str) -> int:
+    from django.db import connection
+    from django.test.utils import CaptureQueriesContext
+
+    with CaptureQueriesContext(connection) as ctx:
+        resp = client.get(url)
+    if resp.status_code != 200:
+        raise RuntimeError(f"GET {url} -> {resp.status_code}: {resp.content[:300]!r}")
+    return len(ctx.captured_queries)
+
+
+def run_endpoint_benchmarks(
+    *,
+    perf_target: User,
+    perf_admin: User,
+    repeat: int,
+) -> list[EndpointTiming]:
+    from rest_framework.test import APIClient
+
+    results: list[EndpointTiming] = []
+    for user_label, user in (("target", perf_target), ("admin", perf_admin)):
+        client = APIClient()
+        client.force_authenticate(user=user)
+        for name, url in ENDPOINTS:
+            client.get(url)  # warm-up request, not counted
+            qcount = _query_count(client, url)
+            times_ms = [t * 1000 for t in _timed_requests(client, url, repeat)]
+            results.append(
+                EndpointTiming(
+                    user_label=user_label,
+                    endpoint_name=name,
+                    query_count=qcount,
+                    min_ms=min(times_ms),
+                    median_ms=statistics.median(times_ms),
+                    max_ms=max(times_ms),
+                ),
+            )
+    return results
@@ -0,0 +1,56 @@
+# src/paperless_benchmark/harness.py
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+from typing import Generic
+from typing import TypeVar
+
+from django.db import connection
+from django.test.utils import CaptureQueriesContext
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+T = TypeVar("T")
+
+
+@dataclass(frozen=True, slots=True)
+class ProfileResult(Generic[T]):
+    best_seconds: float
+    all_seconds: tuple[float, ...]
+    query_count: int
+    result: T
+
+
+def run_profile(fn: Callable[[], T], *, repeat: int = 5) -> ProfileResult[T]:
+    """
+    Call `fn` `repeat` times, capturing wall-clock time for every call and
+    the SQL query count for the final call. Returns the best (minimum)
+    time across all repeats, since the first call(s) can be skewed by
+    connection warm-up or cold caches.
+    """
+    if repeat < 1:
+        raise ValueError("repeat must be >= 1")
+
+    all_seconds: list[float] = []
+    result: T | None = None
+    query_count = 0
+    for i in range(repeat):
+        with CaptureQueriesContext(connection) as ctx:
+            start = time.perf_counter()
+            result = fn()
+            all_seconds.append(time.perf_counter() - start)
+        if i == repeat - 1:
+            query_count = len(ctx.captured_queries)
+    # Purely a type-narrowing aid for the type checker: the `repeat < 1`
+    # guard above already turns the one case that could leave `result`
+    # unset into a clear ValueError, so this is unreachable in practice.
+    assert result is not None
+    return ProfileResult(
+        best_seconds=min(all_seconds),
+        all_seconds=tuple(all_seconds),
+        query_count=query_count,
+        result=result,
+    )
@@ -0,0 +1,222 @@
+# src/paperless_benchmark/management/commands/benchmark.py
+from __future__ import annotations
+
+from typing import Any
+
+from django.core.management.base import BaseCommand
+from django.core.management.base import CommandError
+from django.core.management.base import CommandParser
+
+
+class Command(BaseCommand):
+    help = "Seed, run, and profile paperless-ngx performance benchmarks."
+
+    def add_arguments(self, parser: CommandParser) -> None:
+        parser.add_argument(
+            "action",
+            choices=["seed", "run", "profile", "list-scenarios"],
+            help="Which benchmark action to perform.",
+        )
+        parser.add_argument(
+            "scenario",
+            nargs="?",
+            default=None,
+            help="Scenario name (required for `profile`; see `list-scenarios`).",
+        )
+        parser.add_argument(
+            "--tier",
+            choices=["home", "medium", "large"],
+            default="medium",
+            help="Dataset scale tier for `seed` (default: medium).",
+        )
+        parser.add_argument(
+            "--reset",
+            action="store_true",
+            default=False,
+            help="For `seed`: truncate existing benchmark data first.",
+        )
+        parser.add_argument(
+            "--yes-i-know-this-wipes-the-database",
+            action="store_true",
+            default=False,
+            help=(
+                "Required alongside --reset: confirms you understand `seed "
+                "--reset` deletes ALL users, ALL groups, and ALL documents/"
+                "tags/correspondents/document types/storage paths in this "
+                "database, not just benchmark-created ones."
+            ),
+        )
+        parser.add_argument(
+            "--seed",
+            type=int,
+            default=42,
+            help="RNG seed for reproducible datasets (default: 42).",
+        )
+        parser.add_argument(
+            "--repeat",
+            type=int,
+            default=5,
+            help="Number of timed repetitions for `run`/`profile` (default: 5).",
+        )
+        parser.add_argument(
+            "--label",
+            default="baseline",
+            help="Free-text tag for a `run`, printed and recorded in history only.",
+        )
+        parser.add_argument(
+            "--explain",
+            action="store_true",
+            default=False,
+            help="For `profile`: also capture and print the query plan.",
+        )
+
+    def handle(self, *args: Any, **options: Any) -> None:
+        action = options["action"]
+        if action == "seed":
+            self._handle_seed(options)
+        elif action == "run":
+            self._handle_run(options)
+        elif action == "profile":
+            self._handle_profile(options)
+        else:
+            self._handle_list_scenarios()
+
+    def _handle_seed(self, options: dict[str, Any]) -> None:
+        from paperless_benchmark.db import reset_benchmark_data
+        from paperless_benchmark.seeding import seed_benchmark_dataset
+
+        if options["reset"]:
+            if not options["yes_i_know_this_wipes_the_database"]:
+                raise CommandError(
+                    "--reset requires --yes-i-know-this-wipes-the-database. "
+                    "This deletes ALL users, ALL groups, and ALL documents, "
+                    "tags, correspondents, document types, and storage paths "
+                    "in this database -- not just benchmark-created ones. "
+                    "Only run this against a disposable benchmark database, "
+                    "never a real install. Re-run with "
+                    "--reset --yes-i-know-this-wipes-the-database to proceed.",
+                )
+            self.stdout.write("Resetting existing benchmark data...")
+            reset_benchmark_data()
+
+        data = seed_benchmark_dataset(options["tier"], seed=options["seed"])
+        self.stdout.write(
+            self.style.SUCCESS(
+                f"Seeded tier={options['tier']!r}: {data.documents} documents, "
+                f"{len(data.users)} users, {len(data.groups)} groups.",
+            ),
+        )
+
+    def _handle_run(self, options: dict[str, Any]) -> None:
+        from django.contrib.auth import get_user_model
+        from django.db import connection
+
+        from documents.models import Document
+        from paperless_benchmark.endpoints import run_endpoint_benchmarks
+        from paperless_benchmark.results import append_history
+
+        user_model = get_user_model()
+        try:
+            perf_target = user_model.objects.get(username="perf_target")
+            perf_admin = user_model.objects.get(username="perf_admin")
+        except user_model.DoesNotExist as e:
+            raise CommandError(
+                "No benchmark dataset found. Run `manage.py benchmark seed` first.",
+            ) from e
+
+        db_vendor = connection.vendor
+        document_count = Document.objects.count()
+
+        results = run_endpoint_benchmarks(
+            perf_target=perf_target,
+            perf_admin=perf_admin,
+            repeat=options["repeat"],
+        )
+
+        self.stdout.write(f"# label={options['label']} repeat={options['repeat']}")
+        self.stdout.write(
+            f"{'user':7s} {'endpoint':20s} {'queries':>8s} "
+            f"{'min_ms':>9s} {'median_ms':>10s} {'max_ms':>9s}",
+        )
+        for r in results:
+            self.stdout.write(
+                f"{r.user_label:7s} {r.endpoint_name:20s} {r.query_count:8d} "
+                f"{r.min_ms:9.1f} {r.median_ms:10.1f} {r.max_ms:9.1f}",
+            )
+            append_history(
+                {
+                    "mode": "run",
+                    "label": options["label"],
+                    "user": r.user_label,
+                    "endpoint": r.endpoint_name,
+                    "query_count": r.query_count,
+                    "min_ms": r.min_ms,
+                    "median_ms": r.median_ms,
+                    "max_ms": r.max_ms,
+                    "db_vendor": db_vendor,
+                    "document_count": document_count,
+                },
+            )
+
+    def _handle_profile(self, options: dict[str, Any]) -> None:
+        from django.contrib.auth import get_user_model
+        from django.db import connection
+
+        from documents.models import Document
+        from paperless_benchmark.db import capture_explain
+        from paperless_benchmark.harness import run_profile
+        from paperless_benchmark.results import append_history
+        from paperless_benchmark.scenarios import get as get_scenario
+
+        if not options["scenario"]:
+            raise CommandError(
+                "`profile` requires a scenario name; see `list-scenarios`.",
+            )
+
+        scenario = get_scenario(options["scenario"])
+
+        user_model = get_user_model()
+        try:
+            perf_target = user_model.objects.get(username="perf_target")
+        except user_model.DoesNotExist as e:
+            raise CommandError(
+                "No benchmark dataset found. Run `manage.py benchmark seed` first.",
+            ) from e
+
+        profile = run_profile(
+            lambda: scenario.run(perf_target),
+            repeat=options["repeat"],
+        )
+        self.stdout.write(
+            f"{scenario.name}: best={profile.best_seconds:.4f}s "
+            f"queries={profile.query_count}",
+        )
+
+        if options["explain"]:
+            if scenario.queryset_for_explain is not None:
+                plan = capture_explain(scenario.queryset_for_explain(perf_target))
+                self.stdout.write(plan)
+            else:
+                self.stdout.write(
+                    self.style.WARNING(
+                        f"--explain was requested but scenario {scenario.name!r} "
+                        "does not support it (no queryset_for_explain); skipping.",
+                    ),
+                )
+
+        append_history(
+            {
+                "mode": "profile",
+                "scenario": scenario.name,
+                "best_seconds": profile.best_seconds,
+                "query_count": profile.query_count,
+                "db_vendor": connection.vendor,
+                "document_count": Document.objects.count(),
+            },
+        )
+
+    def _handle_list_scenarios(self) -> None:
+        from paperless_benchmark.scenarios import all_scenarios
+
+        for scenario in all_scenarios():
+            self.stdout.write(f"{scenario.name}: {scenario.describe}")
@@ -0,0 +1,39 @@
+# src/paperless_benchmark/results.py
+from __future__ import annotations
+
+import json
+import subprocess
+from datetime import UTC
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+RESULTS_DIR = Path(__file__).resolve().parent.parent.parent / "benchmark_results"
+
+
+def _current_git_ref() -> str:
+    result = subprocess.run(
+        ["git", "rev-parse", "--short", "HEAD"],
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    return result.stdout.strip() or "unknown"
+
+
+def append_history(entry: dict[str, Any], *, code_ref: str | None = None) -> None:
+    """
+    Append one line to benchmark_results/history.jsonl -- a local-only,
+    append-only, cross-session record of every `benchmark run`/`profile`
+    invocation. Unlike a single overwritten snapshot file, this survives
+    across sessions so a benchmarking effort picked back up days later has
+    a full timeline instead of only the most recent result.
+    """
+    RESULTS_DIR.mkdir(exist_ok=True)
+    record = {
+        "timestamp": datetime.now(UTC).isoformat(),
+        "code_ref": code_ref or _current_git_ref(),
+        **entry,
+    }
+    with (RESULTS_DIR / "history.jsonl").open("a") as f:
+        f.write(json.dumps(record) + "\n")
@@ -0,0 +1,110 @@
+# src/paperless_benchmark/scenarios.py
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+from typing import Any
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    from django.contrib.auth.models import User
+    from django.db.models import QuerySet
+
+
+@dataclass(frozen=True, slots=True)
+class Scenario:
+    name: str
+    describe: str
+    run: Callable[[User], Any]
+    queryset_for_explain: Callable[[User], QuerySet] | None = None
+
+
+_SCENARIOS: dict[str, Scenario] = {}
+
+
+def register(scenario: Scenario) -> None:
+    _SCENARIOS[scenario.name] = scenario
+
+
+def get(name: str) -> Scenario:
+    from django.core.management.base import CommandError
+
+    try:
+        return _SCENARIOS[name]
+    except KeyError:
+        available = ", ".join(sorted(_SCENARIOS)) or "(none registered)"
+        raise CommandError(
+            f"Unknown benchmark scenario {name!r}. Available: {available}",
+        ) from None
+
+
+def all_scenarios() -> tuple[Scenario, ...]:
+    return tuple(_SCENARIOS.values())
+
+
+def _guardian_visibility_query_run(user: User) -> list[int]:
+    from documents.models import Document
+    from documents.permissions import get_objects_for_user_owner_aware
+
+    return list(
+        get_objects_for_user_owner_aware(
+            user,
+            "documents.view_document",
+            Document,
+        ).values_list("id", flat=True),
+    )
+
+
+def _guardian_visibility_query_queryset(user: User) -> QuerySet:
+    from documents.models import Document
+    from documents.permissions import get_objects_for_user_owner_aware
+
+    return get_objects_for_user_owner_aware(user, "documents.view_document", Document)
+
+
+register(
+    Scenario(
+        name="guardian_visibility_query",
+        describe=(
+            "Document-visibility queryset for a user with mixed owned/shared "
+            "documents -- exercises documents.permissions."
+            "get_objects_for_user_owner_aware's guardian permission join."
+        ),
+        run=_guardian_visibility_query_run,
+        queryset_for_explain=_guardian_visibility_query_queryset,
+    ),
+)
+
+
+def _permitted_document_ids_run(user: User) -> list[int]:
+    from documents.models import Document
+    from documents.permissions import permitted_document_ids
+
+    return list(
+        Document.objects.filter(id__in=permitted_document_ids(user)).values_list(
+            "id",
+            flat=True,
+        ),
+    )
+
+
+def _permitted_document_ids_queryset(user: User) -> QuerySet:
+    from documents.models import Document
+    from documents.permissions import permitted_document_ids
+
+    return Document.objects.filter(id__in=permitted_document_ids(user))
+
+
+register(
+    Scenario(
+        name="permitted_document_ids",
+        describe=(
+            "Document-visibility query built from documents.permissions."
+            "permitted_document_ids -- the resolved-ID-set alternative to "
+            "guardian_visibility_query, for side-by-side comparison."
+        ),
+        run=_permitted_document_ids_run,
+        queryset_for_explain=_permitted_document_ids_queryset,
+    ),
+)
@@ -0,0 +1,445 @@
+# src/paperless_benchmark/seeding.py
+from __future__ import annotations
+
+import datetime
+import random
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+from typing import Literal
+
+if TYPE_CHECKING:
+    from django.contrib.auth.models import Group
+    from django.contrib.auth.models import User
+
+Tier = Literal["home", "medium", "large"]
+
+CHUNK_SIZE = 5_000
+
+MIME_TYPES = (
+    "application/pdf",
+    "image/png",
+    "image/jpeg",
+    "text/plain",
+)
+
+# Ownership split for documents, mirroring the shape used in the #11950
+# perf-benchmark dataset (owned-by-target / owned-by-other / unowned).
+OWNED_BY_TARGET_FRACTION = 0.60
+OWNED_BY_OTHER_FRACTION = 0.30
+# remainder (0.10) is unowned
+
+# Of documents owned by "other" users, the fraction explicitly shared
+# (view, or view+change) with perf_target via guardian permissions --
+# this is what exercises the get_user_can_change() per-row N+1 that the
+# `run` endpoint benchmarks measure.
+SHARED_WITH_TARGET_FRACTION = 0.5
+SHARED_WITH_CHANGE_FRACTION = 0.5
+
+# Guardian permission-row ratios measured from a real install (discussion
+# #13276): 1,414 user-perm rows / 27,232 group-perm rows over 12,000
+# documents. Layered across ALL owned documents for the general user/group
+# pool (not just perf_target's shares), so `profile` scenarios exercise a
+# realistic permission-join shape for arbitrary users, not only perf_target.
+USER_PERM_ROWS_PER_DOC = 1_414 / 12_000
+GROUP_PERM_ROWS_PER_DOC = 27_232 / 12_000
+
+
+@dataclass(frozen=True, slots=True)
+class _TierCounts:
+    documents: int
+    tags: int
+    correspondents: int
+    document_types: int
+    storage_paths: int
+    other_users: int
+    groups: int
+    tags_per_doc: tuple[int, int]
+
+
+TIERS: dict[Tier, _TierCounts] = {
+    "home": _TierCounts(
+        documents=500,
+        tags=20,
+        correspondents=10,
+        document_types=8,
+        storage_paths=5,
+        other_users=3,
+        groups=2,
+        tags_per_doc=(1, 3),
+    ),
+    "medium": _TierCounts(
+        documents=20_000,
+        tags=100,
+        correspondents=300,
+        document_types=50,
+        storage_paths=20,
+        other_users=10,
+        groups=5,
+        tags_per_doc=(2, 6),
+    ),
+    "large": _TierCounts(
+        documents=360_000,
+        tags=1_000,
+        correspondents=5_000,
+        document_types=300,
+        storage_paths=50,
+        other_users=25,
+        groups=10,
+        tags_per_doc=(3, 7),
+    ),
+}
+
+
+def log(msg: str) -> None:
+    print(f"[{time.strftime('%H:%M:%S')}] {msg}", flush=True)  # noqa: T201
+
+
+@dataclass(frozen=True, slots=True)
+class SeededData:
+    """
+    Summary of a completed seed run. `documents`/`tags`/`correspondents`/
+    `document_types`/`storage_paths` are counts, not the seeded ORM
+    instances: at the `large` tier (360,000 documents) holding every
+    instance in memory simultaneously is a real risk for zero benefit, since
+    no caller consumes anything but the counts. `perf_target`/`perf_admin`/
+    `users`/`groups` stay as real objects -- at most ~26 users/12 groups even
+    at `large` tier, and small enough to be useful to a future caller.
+    """
+
+    perf_target: User
+    perf_admin: User
+    users: tuple[User, ...]
+    groups: tuple[Group, ...]
+    documents: int
+    tags: int
+    correspondents: int
+    document_types: int
+    storage_paths: int
+
+
+def _grant_model_level_permissions(user: User) -> None:
+    """
+    Grant perf_target Django model-level view/add/change permissions on
+    Document and Tag, on top of the per-object guardian grants seeding
+    creates elsewhere. DRF's PaperlessObjectPermissions checks model-level
+    permissions before guardian's object-level ones are ever consulted, so
+    without this perf_target gets a blanket 403 on /api/documents/ and
+    /api/tags/ regardless of which documents guardian says it can see.
+    """
+    from django.contrib.auth.models import Permission
+    from django.contrib.contenttypes.models import ContentType
+
+    from documents.models import Document
+    from documents.models import Tag
+
+    for model in (Document, Tag):
+        content_type = ContentType.objects.get_for_model(model)
+        codenames = [
+            f"{action}_{model._meta.model_name}" for action in ("view", "add", "change")
+        ]
+        perms = Permission.objects.filter(
+            content_type=content_type,
+            codename__in=codenames,
+        )
+        user.user_permissions.add(*perms)
+
+
+def _create_users_and_groups(
+    counts: _TierCounts,
+) -> tuple[User, User, tuple[User, ...], tuple[Group, ...]]:
+    from django.contrib.auth.models import Group
+
+    from documents.tests.factories import UserFactory
+
+    perf_target = UserFactory.create(username="perf_target")
+    _grant_model_level_permissions(perf_target)
+    perf_admin = UserFactory.create(username="perf_admin", superuser=True)
+    other_users = tuple(UserFactory.create_batch(counts.other_users))
+    groups = tuple(
+        Group.objects.create(name=f"benchmark_group_{i}") for i in range(counts.groups)
+    )
+    log(
+        f"Created users: 1 target, 1 superuser, {len(other_users)} other, "
+        f"{len(groups)} groups.",
+    )
+    return perf_target, perf_admin, other_users, groups
+
+
+def _create_lookup_tables(counts: _TierCounts):
+    from documents.models import Correspondent
+    from documents.models import DocumentType
+    from documents.models import StoragePath
+    from documents.models import Tag
+    from documents.tests.factories import CorrespondentFactory
+    from documents.tests.factories import DocumentTypeFactory
+    from documents.tests.factories import StoragePathFactory
+    from documents.tests.factories import TagFactory
+
+    tags = tuple(Tag.objects.bulk_create(TagFactory.build_batch(counts.tags)))
+    correspondents = tuple(
+        Correspondent.objects.bulk_create(
+            CorrespondentFactory.build_batch(counts.correspondents),
+        ),
+    )
+    document_types = tuple(
+        DocumentType.objects.bulk_create(
+            DocumentTypeFactory.build_batch(counts.document_types),
+        ),
+    )
+    storage_paths = tuple(
+        StoragePath.objects.bulk_create(
+            StoragePathFactory.build_batch(counts.storage_paths),
+        ),
+    )
+    log(
+        f"Created {len(tags)} tags, {len(correspondents)} correspondents, "
+        f"{len(document_types)} document types, {len(storage_paths)} storage paths.",
+    )
+    return tags, correspondents, document_types, storage_paths
+
+
+def _assign_owner(rng: random.Random, perf_target: User, other_users: tuple[User, ...]):
+    roll = rng.random()
+    if roll < OWNED_BY_TARGET_FRACTION:
+        return perf_target, "target"
+    if roll < OWNED_BY_TARGET_FRACTION + OWNED_BY_OTHER_FRACTION:
+        return rng.choice(other_users), "other"
+    return None, "unowned"
+
+
+def _seed_documents(
+    rng: random.Random,
+    counts: _TierCounts,
+    tags,
+    correspondents,
+    document_types,
+    storage_paths,
+    perf_target,
+    other_users,
+) -> tuple[int, list[int]]:
+    """
+    Bulk-create `counts.documents` documents in chunks. Returns the total
+    document count plus a lightweight list of pks for documents that ended
+    up with an owner (target or other) -- that's all
+    `_grant_general_permissions` needs to sample from, so full `Document`
+    instances aren't accumulated across chunks (a real memory concern at the
+    `large` tier's 360,000 documents).
+    """
+    from django.contrib.auth.models import Permission
+    from django.contrib.contenttypes.models import ContentType
+    from guardian.models import UserObjectPermission
+
+    from documents.models import Document
+    from documents.tests.factories import DocumentFactory
+
+    tag_ids = [t.pk for t in tags]
+    correspondent_ids = [c.pk for c in correspondents]
+    document_type_ids = [d.pk for d in document_types]
+    storage_path_ids = [s.pk for s in storage_paths]
+
+    doc_content_type = ContentType.objects.get_for_model(Document)
+    view_perm = Permission.objects.get(
+        codename="view_document",
+        content_type=doc_content_type,
+    )
+    change_perm = Permission.objects.get(
+        codename="change_document",
+        content_type=doc_content_type,
+    )
+    through_model = Document.tags.through
+
+    document_count = 0
+    owned_document_pks: list[int] = []
+    remaining = counts.documents
+    while remaining > 0:
+        chunk_n = min(CHUNK_SIZE, remaining)
+        remaining -= chunk_n
+
+        batch = []
+        owner_buckets = []
+        for _ in range(chunk_n):
+            doc = DocumentFactory.build(
+                mime_type=rng.choice(MIME_TYPES),
+                page_count=rng.randint(1, 30),
+                correspondent_id=(
+                    rng.choice(correspondent_ids)
+                    if correspondent_ids and rng.random() < 0.8
+                    else None
+                ),
+                document_type_id=(
+                    rng.choice(document_type_ids)
+                    if document_type_ids and rng.random() < 0.6
+                    else None
+                ),
+                storage_path_id=(
+                    rng.choice(storage_path_ids)
+                    if storage_path_ids and rng.random() < 0.8
+                    else None
+                ),
+                # Document.created is a plain DateField (default: today).
+                # Leaving it unset would give every seeded document the same
+                # date, collapsing Document's ("-created",) ordering index
+                # into a single-valued sort key -- spread it over a
+                # realistic multi-year window instead.
+                created=datetime.date.today()
+                - datetime.timedelta(days=rng.randint(0, 365 * 3)),
+            )
+            owner, bucket = _assign_owner(rng, perf_target, other_users)
+            doc.owner_id = owner.pk if owner else None
+            batch.append(doc)
+            owner_buckets.append(bucket)
+
+        created = Document.objects.bulk_create(batch, batch_size=CHUNK_SIZE)
+
+        through_rows = []
+        for doc in created:
+            k = rng.randint(*counts.tags_per_doc)
+            for tag_id in rng.sample(tag_ids, min(k, len(tag_ids))):
+                through_rows.append(through_model(document_id=doc.pk, tag_id=tag_id))
+        if through_rows:
+            through_model.objects.bulk_create(through_rows, batch_size=CHUNK_SIZE)
+
+        perm_rows = []
+        for doc, bucket in zip(created, owner_buckets, strict=True):
+            if bucket == "unowned":
+                continue
+            owned_document_pks.append(doc.pk)
+
+            if bucket != "other":
+                continue
+            if rng.random() >= SHARED_WITH_TARGET_FRACTION:
+                continue
+            perm_rows.append(
+                UserObjectPermission(
+                    permission=view_perm,
+                    content_type=doc_content_type,
+                    object_pk=str(doc.pk),
+                    user=perf_target,
+                ),
+            )
+            if rng.random() < SHARED_WITH_CHANGE_FRACTION:
+                perm_rows.append(
+                    UserObjectPermission(
+                        permission=change_perm,
+                        content_type=doc_content_type,
+                        object_pk=str(doc.pk),
+                        user=perf_target,
+                    ),
+                )
+        if perm_rows:
+            UserObjectPermission.objects.bulk_create(perm_rows, batch_size=CHUNK_SIZE)
+
+        document_count += len(created)
+        log(f"  {document_count}/{counts.documents} documents seeded")
+
+    return document_count, owned_document_pks
+
+
+def _grant_general_permissions(
+    rng: random.Random,
+    owned_document_pks: list[int],
+    users,
+    groups,
+) -> None:
+    """
+    Layer realistic (issue #13276-derived) guardian permission-row ratios
+    across owned documents for the general user/group pool, so `profile`
+    scenarios exercise the same permission-join shape regardless of which
+    user they check visibility for (not just perf_target).
+    """
+    from django.contrib.auth.models import Permission
+    from django.contrib.contenttypes.models import ContentType
+    from guardian.models import GroupObjectPermission
+    from guardian.models import UserObjectPermission
+
+    from documents.models import Document
+
+    if not owned_document_pks or not users:
+        return
+
+    doc_content_type = ContentType.objects.get_for_model(Document)
+    view_perm = Permission.objects.get(
+        codename="view_document",
+        content_type=doc_content_type,
+    )
+
+    n_user_perms = round(len(owned_document_pks) * USER_PERM_ROWS_PER_DOC)
+    n_group_perms = (
+        round(len(owned_document_pks) * GROUP_PERM_ROWS_PER_DOC) if groups else 0
+    )
+
+    user_rows = [
+        UserObjectPermission(
+            permission=view_perm,
+            content_type=doc_content_type,
+            object_pk=str(rng.choice(owned_document_pks)),
+            user=rng.choice(users),
+        )
+        for _ in range(n_user_perms)
+    ]
+    if user_rows:
+        UserObjectPermission.objects.bulk_create(
+            user_rows,
+            batch_size=CHUNK_SIZE,
+            ignore_conflicts=True,
+        )
+
+    group_rows = [
+        GroupObjectPermission(
+            permission=view_perm,
+            content_type=doc_content_type,
+            object_pk=str(rng.choice(owned_document_pks)),
+            group=rng.choice(groups),
+        )
+        for _ in range(n_group_perms)
+    ]
+    if group_rows:
+        GroupObjectPermission.objects.bulk_create(
+            group_rows,
+            batch_size=CHUNK_SIZE,
+            ignore_conflicts=True,
+        )
+
+    log(f"  Granted {len(user_rows)} user perms, {len(group_rows)} group perms.")
+
+
+def seed_benchmark_dataset(tier: Tier, *, seed: int = 42) -> SeededData:
+    """
+    Build a benchmark dataset at the given scale tier: a named perf_target
+    (mixed owned/shared documents) and perf_admin (superuser) for endpoint
+    benchmarking, plus a general user/group pool with realistic guardian
+    permission-row ratios for profile scenarios.
+    """
+    counts = TIERS[tier]
+    rng = random.Random(seed)
+
+    log(f"Seeding tier={tier!r}")
+    perf_target, perf_admin, other_users, groups = _create_users_and_groups(counts)
+    tags, correspondents, document_types, storage_paths = _create_lookup_tables(counts)
+    document_count, owned_document_pks = _seed_documents(
+        rng,
+        counts,
+        tags,
+        correspondents,
+        document_types,
+        storage_paths,
+        perf_target,
+        other_users,
+    )
+    all_users = (perf_target, *other_users)
+    _grant_general_permissions(rng, owned_document_pks, all_users, groups)
+
+    log(f"Done. {document_count} documents seeded for tier={tier!r}.")
+
+    return SeededData(
+        perf_target=perf_target,
+        perf_admin=perf_admin,
+        users=all_users,
+        groups=groups,
+        documents=document_count,
+        tags=len(tags),
+        correspondents=len(correspondents),
+        document_types=len(document_types),
+        storage_paths=len(storage_paths),
+    )
@@ -3735,15 +3735,15 @@ crypto = [

 [[package]]
 name = "pymdown-extensions"
-version = "10.21.3"
+version = "11.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
    { name = "markdown", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
    { name = "pyyaml", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/9e/26/d1015444da4d952a1ca487a236b522eb979766f0295a0bd0c5fc089989a9/pymdown_extensions-10.21.3.tar.gz", hash = "sha256:72cfcf55f07aea0d4af2c4f11dd4e52466ddfb1bb819673146398e0bd3a77354", size = 854140, upload-time = "2026-05-13T12:57:32.267Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/47/67/f1e79672a5f91985577c7984c9709ca110e4fd37fe7fd167b60422e6ccc2/pymdown_extensions-11.0.tar.gz", hash = "sha256:8269cef0247f9e2d0a62fcea10860aba05c1cbab5470fd4b63230b96434dc589", size = 857049, upload-time = "2026-06-23T02:27:45.146Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/7e/85/545a951eecc270fcd688288c600017e2050a1aacb56c711d208586d3e470/pymdown_extensions-10.21.3-py3-none-any.whl", hash = "sha256:d7a5d08014fc571e80ca21dd6f854e31f94c489800350564d55d15b3c41e76b6", size = 269002, upload-time = "2026-05-13T12:57:30.296Z" },
+    { url = "https://files.pythonhosted.org/packages/af/b6/1ae53367e28b9cffa3be7574e13fbe4589694272fd47710fbdbafd3d63c6/pymdown_extensions-11.0-py3-none-any.whl", hash = "sha256:fbc4acb641814fa9d17521bbd21a5240ef739a662f11c06330c4b78c93e954d6", size = 269415, upload-time = "2026-06-23T02:27:43.826Z" },
 ]

 [[package]]
Author	SHA1	Message	Date
stumpylogandClaude Sonnet 5	cfb9125d9e	chore: remove benchmark-commands spec/plan docs now that work is complete Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-30 13:43:01 -07:00
stumpylogandClaude Sonnet 5	fa2d7f660b	Fix: apply final-review fix wave (safety guard, date spread, MariaDB explain, history fields, memory footprint, minors) Final whole-branch review before this tooling branch settles: add a required --yes-i-know-this-wipes-the-database confirmation flag ahead of --reset (it deletes ALL users/groups/documents in the target DB, not just benchmark-created ones); spread seeded documents' created dates over a 3-year window instead of leaving them all on one default date; fix capture_explain() to use MariaDB's ANALYZE syntax instead of Postgres-only EXPLAIN ANALYZE (verified against real MariaDB 12.3 -- the old code was a silent 1064 syntax error); record db_vendor/document_count in run/profile history entries; shrink SeededData's memory footprint at scale by returning counts instead of full ORM instance tuples; and a handful of minor fixes (storage_path assignment, --explain warning instead of silent no-op, harness.py repeat<1 guard, type annotations). All changes verified against real Postgres and MariaDB containers on the VM, not just SQLite. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-30 13:33:28 -07:00
stumpylogandClaude Sonnet 5	2712c3fe48	Fix: correct /api/tags/ endpoint URL in benchmarking skill The skill documentation stated that run benchmarks /api/tags/ but src/paperless_benchmark/endpoints.py actually uses /api/tags/?page_size=100000. Update the skill's Command reference section to match the actual endpoint definition. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-30 11:20:14 -07:00
stumpylog	684c4ef392	docs(benchmark): add paperless-benchmarking skill	2026-07-30 11:07:33 -07:00
stumpylog	410e440bd6	fix(benchmark): fully clear users/groups on reset, grant perf_target model perms	2026-07-30 10:53:02 -07:00
stumpylogandClaude Sonnet 5	ef9ee435e8	docs: add Task 10 (quirk fixes) and Task 11 (skill) to benchmark plan Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-30 10:44:34 -07:00
stumpylog	5d8287f120	fix(benchmark): reuse seeded data in profile instead of reseeding _handle_profile unconditionally called seed_benchmark_dataset every run, colliding with an IntegrityError against data seeded by a prior seed/profile call. Look up the existing perf_target user instead, matching _handle_run's already-correct pattern, and raise a CommandError pointing at `seed` if no dataset exists. Scenario.run/queryset_for_explain now take a User directly instead of a SeededData, since only data.users[0] (perf_target) was ever used. profile no longer seeds, so it drops the --tier dependency: the printed line and history entry no longer include a tier/(tier=...) suffix, and --tier's help text now only mentions `seed`.	2026-07-30 10:15:12 -07:00
stumpylog	acfdafbff9	feat(benchmark): add manage.py benchmark command (seed/run/profile/list-scenarios)	2026-07-30 09:52:21 -07:00
stumpylog	cd02cfe02c	feat(benchmark): add pluggable profile-scenario registry	2026-07-30 08:31:14 -07:00
stumpylog	ff9cde7f9c	feat(benchmark): add timed API endpoint scenarios	2026-07-30 07:49:05 -07:00
stumpylogandClaude Sonnet 5	259731074c	docs: add implementation plan for benchmark management commands Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-30 07:16:32 -07:00
stumpylog	02aaf9b3a2	feat(benchmark): add local JSONL results history log	2026-07-29 14:58:11 -07:00
stumpylog	8a34385199	feat(benchmark): add tiered dataset seeding	2026-07-29 14:17:52 -07:00
stumpylog	3e8656a3ed	feat(benchmark): add per-backend reset and explain helpers	2026-07-29 13:27:39 -07:00
stumpylog	8ec322f9aa	feat(benchmark): add run_profile timing harness	2026-07-29 12:29:53 -07:00
stumpylogandClaude Sonnet 5	22e4be1548	feat(benchmark): scaffold paperless_benchmark app Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-29 12:11:53 -07:00
stumpylogandClaude Sonnet 5	17e0b70da5	docs: add design spec for benchmark management commands Merges the standalone run_benchmarks.py/seed_benchmark_data.py scripts and the tools/profiling branch's Postgres-only harness into one cross-backend `manage.py benchmark` command suite. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-29 11:15:28 -07:00
shamoonandGitHub	668fa77428	Fix: prevent scale vs page loop in pngx PDF viewer (#13406 )	2026-07-29 09:25:37 -07:00
shamoonandGitHub	5bd72014a6	Fix: handle hidden line breaks in email subjects when sending email (#13402 )	2026-07-29 14:35:30 +00:00
Trenton HandGitHub	bbb9c86ba4	Fix: avoid NotSupportedError from document_importer on MariaDB (#13400 )	2026-07-29 07:14:13 -07:00
GitHub Actions	09c9fa03cf	Auto translate strings	2026-07-28 18:28:12 +00:00
shamoonandGitHub	04779d72bb	Tweak: adjust doc details button toolbar flow (#13382 )	2026-07-28 11:26:37 -07:00
Trenton HandGitHub	1b32b9d678	Fix: exclude next-period start from relative date-range filters (#13381 ) * Fix: exclude next-period start from relative date-range filters Tantivy's [lo TO hi] range is inclusive on both ends, but computed upper bounds (keyword ranges, YYYY/YYYYMM/YYYYMMDD tokens) represent the start of the next period. Use half-open [lo TO hi} for those so e.g. "previous month" no longer matches the 1st of the current month. * Adds a regression test down to the second check for the hi range	2026-07-28 16:58:38 +00:00
Trenton HandGitHub	12d318deff	Documentation: Add Password Removal workflow action documentation (#13377 ) Addresses discussion #13373. Documents that password removal creates a new document version via re-consumption of the decrypted file rather than editing in place, and explains why the Consumption Started trigger produces an initial un-OCR'd version followed by a properly processed one.	2026-07-28 09:21:14 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	9c98f30d3f	Chore(deps): Bump pymdown-extensions in the uv group across 1 directory (#13378 ) Bumps the uv group with 1 update in the / directory: [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions). Updates `pymdown-extensions` from 10.21.3 to 11.0 - [Release notes](https://github.com/facelessuser/pymdown-extensions/releases) - [Commits](https://github.com/facelessuser/pymdown-extensions/compare/10.21.3...11.0) --- updated-dependencies: - dependency-name: pymdown-extensions dependency-version: '11.0' dependency-type: indirect dependency-group: uv ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-28 15:29:20 +00:00
shamoonandGitHub	6a1d7b1bca	Fix: close non-atomic db connections in before_task_publish (#13366 )	2026-07-28 07:21:02 -07:00
shamoonandGitHub	de9f520143	Chore/fix: refactor frontend task service (#13365 )	2026-07-28 00:34:30 -07:00
shamoonandGitHub	a68c487d60	Fix: fix Enter selection in search autocomplete (#13361 )	2026-07-27 23:33:13 -07:00