chore: remove benchmark-commands spec/plan docs now that work is complete

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fix: apply final-review fix wave (safety guard, date spread, MariaDB explain, history fields, memory footprint, minors)
2026-09-10 11:48:00 +00:00 · 2026-09-09 07:42:25 -07:00 · 2026-09-09 07:42:25 -07:00 · 2026-09-09 07:42:25 -07:00 · 2026-09-09 07:42:25 -07:00 · 2026-09-09 07:42:25 -07:00
53 changed files with 2171 additions and 2318 deletions
@@ -0,0 +1,187 @@
+---
+name: paperless-benchmarking
+description: Use when profiling paperless-ngx performance, running `manage.py benchmark`, investigating a slow query or endpoint, or deciding whether a profiling finding should become a permanent registered scenario. Covers command reference, the fork/merge-back branch workflow for perf investigations, and how to read query-plan output.
+---
+
+# Paperless-ngx Benchmarking
+
+This repo has a built-in benchmarking/profiling tool: `manage.py benchmark`, in
+the `paperless_benchmark` Django app. It replaces ad hoc standalone scripts —
+use it instead of writing new one-off seed/timing scripts.
+
+## Command reference
+
+```
+manage.py benchmark seed --tier {home,medium,large} [--reset --yes-i-know-this-wipes-the-database] [--seed N]
+manage.py benchmark run --repeat 5 [--label baseline]
+manage.py benchmark profile <scenario_name> [--repeat 5] [--explain]
+manage.py benchmark list-scenarios
+```
+
+- **`seed`** builds a realistic dataset at one of three scales: `home` (500
+  documents — fast, use this for iteration), `medium` (20,000 — the default
+  when `--tier` is omitted; a multi-minute seed), `large` (360,000 — matches
+  the scale reported in real large-install bug reports; slow, only use it
+  when a finding needs confirming at real scale). `--reset` wipes any
+  previously-seeded benchmark data first — but it is destructive and
+  irreversible: it deletes **all** users, **all** groups, and **all**
+  documents/tags/correspondents/document types/storage paths in the target
+  database, not just benchmark-created rows. Only run it against a disposable
+  benchmark database, never a real install. Because of that, `--reset` also
+  requires passing `--yes-i-know-this-wipes-the-database` in the same
+  invocation, or the command raises an error and does nothing. Omit both
+  flags if you want to layer more data onto an existing seed instead. `seed`
+  creates two named users, `perf_target` (mixed owned/shared documents,
+  realistic guardian permission grants) and `perf_admin` (superuser), plus a
+  general user/group pool with realistic permission-row ratios.
+- **`run`** times the 3 built-in API endpoint benchmarks
+  (`/api/documents/`, `/api/documents/?page_size=50`, `/api/tags/?page_size=100000`) for both
+  `perf_target` and `perf_admin`, reporting min/median/max wall-clock and SQL
+  query count. Requires `seed` to have already run — it reuses that data, it
+  does not seed its own.
+- **`profile`** times one named scenario from the registry (see
+  `list-scenarios`) via best-of-N repeat timing and SQL query count, against
+  `perf_target`. `--explain` additionally captures and prints the query plan:
+  real `EXPLAIN ANALYZE` execution stats on PostgreSQL/MariaDB, or
+  `EXPLAIN QUERY PLAN` (plan only, no real timing/row counts — clearly labeled
+  as such) on SQLite. Like `run`, `profile` requires `seed` to have already
+  run — it does not seed its own data either.
+- Every `run`/`profile` invocation appends a JSON line to
+  `benchmark_results/history.jsonl` at the repo root (local-only, gitignored —
+  never commit this file). Use it to compare a `before`/`after` pair across
+  two invocations without hand-copying numbers.
+- Full chain example: `seed --reset` once, then `run` and `profile` as many
+  times as you want against that same seeded data — no need to reseed between
+  them.
+
+## Adding a new scenario
+
+A "scenario" is a named, registered query/operation that `profile` can time
+and explain. To add one, edit `src/paperless_benchmark/scenarios.py`: write a
+`_<name>_run(user)` function (returns whatever `run_profile` should time) and
+optionally a `_<name>_queryset(user)` function (returns the `QuerySet` for
+`--explain` to analyze), then `register(Scenario(name=..., describe=...,
+run=..., queryset_for_explain=...))` at module level. Both functions receive
+the already-seeded `perf_target` user — they should not seed their own data.
+
+## Branch workflow
+
+`tools/benchmark-management-commands` is a **long-lived tooling branch**, not
+a feature branch that gets merged and closed:
+
+1. It is periodically brought up to date with `dev` (merge `dev` into it) so
+   the tooling doesn't drift from the schema/codebase it profiles. Do this
+   before starting a new investigation if it's been a while since the last
+   sync.
+2. **Every performance investigation forks its own branch from
+   `tools/benchmark-management-commands`** (not from `dev`). Do the
+   investigation there: write throwaway profiling code, try fixes, capture
+   before/after numbers.
+3. **That investigation branch never merges into `dev` or production.** Its
+   only job is to produce evidence and, optionally, a reusable scenario.
+4. If the investigation turns up a scenario worth keeping permanently (see
+   "When to graduate a scenario" below), open a PR that adds **just that
+   scenario** back into `tools/benchmark-management-commands` — not the rest
+   of the investigation branch's throwaway code.
+5. Any actual production fix the investigation motivates (e.g. an ORM query
+   change) goes into its own normal feature branch off `dev`, following the
+   project's regular contribution process — profiling evidence informs that
+   PR's description, but the profiling code itself does not travel with it.
+
+## Reading query-plan output
+
+- **PostgreSQL** `EXPLAIN ANALYZE`: look for `Seq Scan` on a large table
+  (missing index), a large gap between `rows=N` (planner's estimate) and the
+  actual row count in parentheses (stale statistics or a bad cardinality
+  estimate), and nested-loop joins driven by an outer relation with many
+  rows (usually the N+1 pattern this tool exists to catch).
+- **MariaDB**: verified against a real MariaDB 12.3 container that MariaDB
+  does NOT accept MySQL 8.0.18+'s `EXPLAIN ANALYZE` syntax (it's a 1064
+  syntax error) -- `capture_explain()` instead runs MariaDB's own
+  `ANALYZE <statement>` form (no `EXPLAIN` keyword), which returns a
+  tabular plan with real per-row execution columns: `rows` (estimate) vs.
+  `r_rows` (actual), and `filtered` vs. `r_filtered`. A large gap between
+  `rows` and `r_rows`, or `type: ALL` (full table scan) on a large table,
+  are the signals to look for -- the same underlying concerns as Postgres's
+  `Seq Scan`/estimate-vs-actual gap, just in MariaDB's column-based output
+  instead of Postgres's nested-tree text format.
+- **SQLite** `EXPLAIN QUERY PLAN`: no real timing/row-count data, only the
+  chosen access path (`SCAN` vs `SEARCH`, which index if any). Useful for
+  confirming an index is even being considered, not for judging real-world
+  cost — corroborate any SQLite finding against Postgres/MariaDB before
+  trusting it, since planner behavior differs meaningfully between them.
+- Compare query **count**, not just timing, between before/after: a fix that
+  keeps the same wall-clock time but drops query count from O(n) to O(1) is
+  still a real, durable improvement — timing alone is noisy and
+  environment-dependent, query count is not.
+
+## Cleaning up after an interrupted run
+
+If a `seed`/`run`/`profile` invocation gets killed mid-run (Ctrl-C, `kill -9`,
+a timed-out SSH session, etc.), check whether it left anything behind before
+trusting the next benchmark's numbers. This was verified for real: a
+`benchmark seed --tier large --reset ...` was started against both a fresh
+PostgreSQL 18 container and a fresh MariaDB 12.3 container and `kill -9`'d a
+few seconds into document seeding. In both cases, the database-side
+connection disappeared immediately -- no stuck backend, no lingering query,
+no held lock was observed in either backend once the killed process's PID
+was confirmed gone. That said, this was one interruption point (mid
+bulk-seed, between chunks); a run killed mid-query, or a driver/network
+hiccup that doesn't cleanly close the socket, could behave differently, so
+still check before trusting a number if any run in the session was
+interrupted:
+
+- **PostgreSQL**: look for leftover connections against the benchmark
+  database:
+
+  ```sql
+  SELECT pid, state, query, query_start
+  FROM pg_stat_activity
+  WHERE datname = current_database() AND pid <> pg_backend_pid();
+  ```
+
+  If a stuck backend shows up, clear it with:
+
+  ```sql
+  SELECT pg_terminate_backend(pid)
+  FROM pg_stat_activity
+  WHERE datname = current_database() AND pid <> pg_backend_pid();
+  ```
+
+- **MariaDB**: look for leftover connections/queries:
+
+  ```sql
+  SHOW FULL PROCESSLIST;
+  ```
+
+  If a stuck connection shows up (anything other than your current admin
+  session), clear it with:
+
+  ```sql
+  KILL <id>;
+  ```
+
+A stray connection left running concurrently with a subsequent benchmark run
+would add real, contaminating load (extra queries competing for the same
+rows, possibly held locks slowing the next run's timings) -- cheap to rule
+out, expensive to silently trust a number that was actually measured
+alongside a zombie connection.
+
+## When to graduate a one-off finding into a permanent scenario
+
+Register a scenario (rather than leaving it as throwaway code on the
+investigation branch) when **both** are true:
+
+- The query pattern is one this codebase is likely to regress on again (e.g.
+  it involves a permission-check join, a bulk operation, or anything else
+  with an easy-to-reintroduce N+1) — not a one-time fluke specific to this
+  investigation.
+- Re-running it later, against a fresh seed, would still produce a
+  meaningful signal (it doesn't depend on investigation-specific throwaway
+  data or a fix that's already permanently landed and can't regress the same
+  way).
+
+If a finding doesn't meet both bars, keep it as disposable code on the
+investigation branch and let the branch's evidence (captured in the PR
+description of whatever production fix it motivates) be the permanent
+record instead.
@@ -115,3 +115,6 @@ celerybeat-schedule*

 # Git worktree local folder
 .worktrees
+
+# Benchmark tooling output (local only, never committed)
+/benchmark_results/
@@ -1200,15 +1200,6 @@ still perform some basic text pre-processing before matching.

    Defaults to true, enabling the feature.

-#### [`PAPERLESS_CLASSIFIER_MATCH_THRESHOLD=<float>`](#PAPERLESS_CLASSIFIER_MATCH_THRESHOLD) {#PAPERLESS_CLASSIFIER_MATCH_THRESHOLD}
-
-: Sets the minimum confidence score (0.0-1.0) required for the automatic
-classifier to assign a correspondent, document type, or storage path to a
-document. Predictions below this threshold are discarded and the field is
-left unassigned, preventing low-confidence guesses from being applied.
-
-    Defaults to 0.6.
-
 #### [`PAPERLESS_DATE_PARSER_LANGUAGES=<lang>`](#PAPERLESS_DATE_PARSER_LANGUAGES) {#PAPERLESS_DATE_PARSER_LANGUAGES}

 : Specifies which language Paperless should use when parsing dates from documents.
@@ -112,22 +112,6 @@

                <pngx-input-check i18n-title title="Use 'slim' sidebar (icons only)" formControlName="slimSidebarEnabled"></pngx-input-check>

-                <p class="mb-2 mt-3" i18n>Sidebar items to show:</p>
-                @for (option of sidebarItemOptions; track option.id) {
-                  <div class="form-check">
-                    <input
-                      class="form-check-input"
-                      type="checkbox"
-                      [id]="'sidebar-item-setting-' + option.id"
-                      [checked]="isSidebarItemShown(option.id)"
-                      (change)="toggleSidebarItem(option.id, $event.target.checked)"
-                    />
-                    <label class="form-check-label" [for]="'sidebar-item-setting-' + option.id">
-                      {{ option.label }}
-                    </label>
-                  </div>
-                }
-
              </div>
            </div>

@@ -24,7 +24,7 @@ import {
  SystemStatus,
  SystemStatusItemStatus,
 } from 'src/app/data/system-status'
-import { HideableSidebarItemID, SETTINGS_KEYS } from 'src/app/data/ui-settings'
+import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
 import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
 import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
 import { PermissionsGuard } from 'src/app/guards/permissions.guard'
@@ -209,45 +209,6 @@ describe('SettingsComponent', () => {
    fixture.detectChanges()
  }

-  it('supports configuring sidebar items and canceling changes', () => {
-    completeSetup()
-
-    component.toggleSidebarItem(HideableSidebarItemID.Workflows, false)
-    fixture.detectChanges()
-
-    expect(component.settingsForm.value.sidebarHiddenItems).toContain(
-      HideableSidebarItemID.Workflows
-    )
-
-    settingsService.updateSidebarItemVisibility(
-      HideableSidebarItemID.Mail,
-      false
-    )
-
-    expect(component.settingsForm.value.sidebarHiddenItems).toContain(
-      HideableSidebarItemID.Mail
-    )
-
-    component.reset()
-
-    expect(component.settingsForm.value.sidebarHiddenItems).not.toContain(
-      HideableSidebarItemID.Workflows
-    )
-    expect(component.settingsForm.value.sidebarHiddenItems).not.toContain(
-      HideableSidebarItemID.Mail
-    )
-  })
-
-  it('enables sidebar item controls on general settings until destroyed', () => {
-    completeSetup()
-
-    expect(settingsService.organizingSidebarItems()).toBe(true)
-
-    component.ngOnDestroy()
-
-    expect(settingsService.organizingSidebarItems()).toBe(false)
-  })
-
  it('should support tabbed settings & change URL, prevent navigation if dirty confirmation rejected', async () => {
    completeSetup()
    const navigateSpy = jest.spyOn(router, 'navigate')
@@ -288,7 +249,6 @@ describe('SettingsComponent', () => {

  it('should support save local settings updating appearance settings and calling API, show error', () => {
    completeSetup()
-    component.toggleSidebarItem(HideableSidebarItemID.Workflows, false)
    const toastErrorSpy = jest.spyOn(toastService, 'showError')
    const toastSpy = jest.spyOn(toastService, 'show')
    const storeSpy = jest.spyOn(settingsService, 'storeSettings')
@@ -307,10 +267,7 @@ describe('SettingsComponent', () => {
    expect(toastErrorSpy).toHaveBeenCalled()
    expect(storeSpy).toHaveBeenCalled()
    expect(appearanceSettingsSpy).not.toHaveBeenCalled()
-    expect(setSpy).toHaveBeenCalledTimes(34)
-    expect(setSpy).toHaveBeenCalledWith(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
-      HideableSidebarItemID.Workflows,
-    ])
+    expect(setSpy).toHaveBeenCalledTimes(33)

    // succeed
    storeSpy.mockReturnValueOnce(of(true))
@@ -39,12 +39,7 @@ import {
  SystemStatus,
  SystemStatusItemStatus,
 } from 'src/app/data/system-status'
-import {
-  GlobalSearchType,
-  HIDEABLE_SIDEBAR_ITEM_IDS,
-  HideableSidebarItemID,
-  SETTINGS_KEYS,
-} from 'src/app/data/ui-settings'
+import { GlobalSearchType, SETTINGS_KEYS } from 'src/app/data/ui-settings'
 import { User } from 'src/app/data/user'
 import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
 import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
@@ -107,14 +102,6 @@ const documentDetailFieldOptions = [
  { id: DocumentDetailFieldID.Tags, label: $localize`Tags` },
 ]

-const sidebarItemLabels: Record<HideableSidebarItemID, string> = {
-  [HideableSidebarItemID.Dashboard]: $localize`Dashboard`,
-  [HideableSidebarItemID.SavedViews]: $localize`Saved Views`,
-  [HideableSidebarItemID.Workflows]: $localize`Workflows`,
-  [HideableSidebarItemID.Mail]: $localize`Mail`,
-  [HideableSidebarItemID.Documentation]: $localize`Documentation`,
-}
-
@Component({
  selector: 'pngx-settings',
  templateUrl: './settings.component.html',
@@ -162,7 +149,6 @@ export class SettingsComponent
    bulkEditApplyOnClose: new FormControl(null),
    documentListItemPerPage: new FormControl(null),
    slimSidebarEnabled: new FormControl(null),
-    sidebarHiddenItems: new FormControl<HideableSidebarItemID[]>([]),
    darkModeUseSystem: new FormControl(null),
    darkModeEnabled: new FormControl(null),
    darkModeInvertThumbs: new FormControl(null),
@@ -200,7 +186,6 @@ export class SettingsComponent

  store: BehaviorSubject<any>
  storeSub: Subscription
-  sidebarItemsSub: Subscription
  isDirty$: Observable<boolean>
  isDirty: boolean = false
  unsubscribeNotifier: Subject<any> = new Subject()
@@ -218,10 +203,6 @@ export class SettingsComponent
  public readonly PdfEditorEditMode = PdfEditorEditMode

  public readonly documentDetailFieldOptions = documentDetailFieldOptions
-  public readonly sidebarItemOptions = HIDEABLE_SIDEBAR_ITEM_IDS.map((id) => ({
-    id,
-    label: sidebarItemLabels[id],
-  }))

  get systemStatusHasErrors(): boolean {
    const status = this.systemStatus()
@@ -249,10 +230,6 @@ export class SettingsComponent

  constructor() {
    super()
-    this.sidebarItemsSub =
-      this.settings.sidebarHiddenItemsEditingChanged.subscribe((hiddenItems) =>
-        this.settingsForm.controls.sidebarHiddenItems.setValue(hiddenItems)
-      )
    this.settings.settingsSaved.subscribe(() => {
      if (!this.savePending) this.initialize()
      this.savedViewsService.maybeRefreshDocumentCounts()
@@ -302,21 +279,14 @@ export class SettingsComponent

    this.activatedRoute.paramMap.subscribe((paramMap) => {
      const section = paramMap.get('section')
-      let navID = SettingsNavIDs.General
      if (section) {
        const navIDKey: string = Object.keys(SettingsNavIDs).find(
          (navID) => navID.toLowerCase() == section
        )
        if (navIDKey) {
-          navID = SettingsNavIDs[navIDKey]
+          this.activeNavID.set(SettingsNavIDs[navIDKey])
        }
      }
-      this.activeNavID.set(navID)
-      this.settings.sidebarHiddenItemsEditing.set(
-        navID === SettingsNavIDs.General
-          ? [...this.settingsForm.controls.sidebarHiddenItems.value]
-          : null
-      )
    })
  }

@@ -340,7 +310,6 @@ export class SettingsComponent
        SETTINGS_KEYS.DOCUMENT_LIST_SIZE
      ),
      slimSidebarEnabled: this.settings.get(SETTINGS_KEYS.SLIM_SIDEBAR),
-      sidebarHiddenItems: this.settings.get(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS),
      darkModeUseSystem: this.settings.get(SETTINGS_KEYS.DARK_MODE_USE_SYSTEM),
      darkModeEnabled: this.settings.get(SETTINGS_KEYS.DARK_MODE_ENABLED),
      darkModeInvertThumbs: this.settings.get(
@@ -467,12 +436,6 @@ export class SettingsComponent
      this.settingsForm.patchValue(currentFormValue)
    }

-    if (this.settings.organizingSidebarItems()) {
-      this.settings.sidebarHiddenItemsEditing.set([
-        ...this.settingsForm.controls.sidebarHiddenItems.value,
-      ])
-    }
-
    if (this.canViewSystemStatus) {
      this.systemStatusService.get().subscribe((status) => {
        this.systemStatus.set(status)
@@ -481,18 +444,8 @@ export class SettingsComponent
  }

  ngOnDestroy() {
-    this.settings.sidebarHiddenItemsEditing.set(null)
    if (this.isDirty) this.settings.updateAppearanceSettings() // in case user changed appearance but didn't save
    this.storeSub && this.storeSub.unsubscribe()
-    this.sidebarItemsSub.unsubscribe()
-  }
-
-  isSidebarItemShown(item: HideableSidebarItemID): boolean {
-    return !(this.settingsForm.value.sidebarHiddenItems || []).includes(item)
-  }
-
-  toggleSidebarItem(item: HideableSidebarItemID, checked: boolean): void {
-    this.settings.updateSidebarItemVisibility(item, checked)
  }

  public saveSettings() {
@@ -520,10 +473,6 @@ export class SettingsComponent
      SETTINGS_KEYS.SLIM_SIDEBAR,
      this.settingsForm.value.slimSidebarEnabled
    )
-    this.settings.set(
-      SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS,
-      this.settingsForm.value.sidebarHiddenItems
-    )
    this.settings.set(
      SETTINGS_KEYS.DARK_MODE_USE_SYSTEM,
      this.settingsForm.value.darkModeUseSystem
@@ -683,11 +632,6 @@ export class SettingsComponent

  reset() {
    this.settingsForm.patchValue(this.store.getValue())
-    if (this.settings.organizingSidebarItems()) {
-      this.settings.sidebarHiddenItemsEditing.set([
-        ...this.settingsForm.controls.sidebarHiddenItems.value,
-      ])
-    }
  }

  clearThemeColor() {
@@ -86,15 +86,12 @@
      }
      <div class="sidebar-sticky pt-3 pb-1 d-flex flex-column justify-space-around">
        <ul class="nav flex-column">
-          <li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Dashboard) && !settingsService.organizingSidebarItems()">
-            <a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Dashboard)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
+          <li class="nav-item app-link">
+            <a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
              ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
              container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
              <i-bs class="me-2" name="house"></i-bs><span class="nav-link-label"><ng-container i18n>Dashboard</ng-container></span>
            </a>
-            @if (settingsService.organizingSidebarItems()) {
-              <pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Dashboard" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.Dashboard)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.Dashboard, $event)"></pngx-input-switch>
-            }
          </li>
          <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }">
            <a class="nav-link" routerLink="documents" routerLinkActive="active"
@@ -240,38 +237,29 @@
                </div>
              </li>
            }
-            <li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.SavedViews) && !settingsService.organizingSidebarItems()" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
-              <a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.SavedViews)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="savedviews" routerLinkActive="active" (click)="closeMenu()"
+            <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
+              <a class="nav-link" routerLink="savedviews" routerLinkActive="active" (click)="closeMenu()"
                ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                <i-bs class="me-2" name="window-stack"></i-bs><span class="nav-link-label"><ng-container i18n>Saved Views</ng-container></span>
              </a>
-              @if (settingsService.organizingSidebarItems()) {
-                <pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Saved Views" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.SavedViews)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.SavedViews, $event)"></pngx-input-switch>
-              }
            </li>
-            <li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows) && !settingsService.organizingSidebarItems()"
+            <li class="nav-item app-link"
              *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Workflow }"
              tourAnchor="tour.workflows">
-              <a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="workflows" routerLinkActive="active" (click)="closeMenu()"
+              <a class="nav-link" routerLink="workflows" routerLinkActive="active" (click)="closeMenu()"
                ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                <i-bs class="me-2" name="boxes"></i-bs><span class="nav-link-label"><ng-container i18n>Workflows</ng-container></span>
              </a>
-              @if (settingsService.organizingSidebarItems()) {
-                <pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Workflows" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.Workflows, $event)"></pngx-input-switch>
-              }
            </li>
-            <li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail) && !settingsService.organizingSidebarItems()" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
+            <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
              tourAnchor="tour.mail">
-              <a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
+              <a class="nav-link" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
                i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                <i-bs class="me-2" name="envelope"></i-bs><span class="nav-link-label"><ng-container i18n>Mail</ng-container></span>
              </a>
-              @if (settingsService.organizingSidebarItems()) {
-                <pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Mail" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.Mail, $event)"></pngx-input-switch>
-              }
            </li>
            <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
              <a class="nav-link" routerLink="trash" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Trash"
@@ -334,16 +322,13 @@
                </a>
              </li>
            }
-            <li class="nav-item mt-2 position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Documentation) && !settingsService.organizingSidebarItems()" tourAnchor="tour.outro">
-              <a class="text-muted small d-flex align-items-center flex-wrap text-decoration-none nav-anchor" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Documentation)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()"
+            <li class="nav-item mt-2" tourAnchor="tour.outro">
+              <a class="text-muted small d-flex align-items-center flex-wrap text-decoration-none nav-anchor"
                target="_blank" rel="noopener noreferrer" href="https://docs.paperless-ngx.com" ngbPopover="Documentation"
                i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                <i-bs class="d-flex me-2" name="question-circle"></i-bs><span><ng-container i18n>Documentation</ng-container></span>
              </a>
-              @if (settingsService.organizingSidebarItems()) {
-                <pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Documentation" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.Documentation)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.Documentation, $event)"></pngx-input-switch>
-              }
            </li>
            <li class="nav-item" [class.visually-hidden]="slimSidebarEnabled">
              <div class="text-muted small d-flex align-items-center flex-wrap nav-label">
@@ -15,7 +15,7 @@ import { provideUiTour } from 'ngx-ui-tour-ng-bootstrap'
 import { of, throwError } from 'rxjs'
 import { routes } from 'src/app/app-routing.module'
 import { SavedView } from 'src/app/data/saved-view'
-import { HideableSidebarItemID, SETTINGS_KEYS } from 'src/app/data/ui-settings'
+import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
 import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
 import { PermissionsGuard } from 'src/app/guards/permissions.guard'
 import {
@@ -287,82 +287,6 @@ describe('AppFrameComponent', () => {
    jest.useRealTimers()
  })

-  it('should hide configured sidebar items', () => {
-    settingsService.set(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
-      HideableSidebarItemID.Dashboard,
-      HideableSidebarItemID.Workflows,
-    ])
-    fixture.detectChanges()
-
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="dashboard"]')
-        .parentElement.classList
-    ).toContain('d-none')
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="workflows"]')
-        .parentElement.classList
-    ).toContain('d-none')
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="mail"]').parentElement
-        .classList
-    ).not.toContain('d-none')
-  })
-
-  it('should show hidden items and visibility switches while customizing', () => {
-    settingsService.set(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
-      HideableSidebarItemID.Dashboard,
-    ])
-    settingsService.sidebarHiddenItemsEditing.set([
-      HideableSidebarItemID.Dashboard,
-    ])
-    fixture.detectChanges()
-
-    expect(
-      fixture.nativeElement.querySelectorAll('pngx-input-switch').length
-    ).toBe(5)
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="dashboard"]')
-        .parentElement.classList
-    ).not.toContain('d-none')
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="dashboard"]').classList
-    ).toContain('opacity-50')
-
-    settingsService.set(SETTINGS_KEYS.SLIM_SIDEBAR, true)
-    fixture.detectChanges()
-
-    expect(
-      Array.from(
-        fixture.nativeElement.querySelectorAll('pngx-input-switch')
-      ).every((toggle: HTMLElement) => toggle.classList.contains('d-none'))
-    ).toBe(true)
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="dashboard"]').classList
-    ).not.toContain('pe-5')
-
-    settingsService.set(SETTINGS_KEYS.SLIM_SIDEBAR, false)
-    component.slimSidebarAnimating.set(true)
-    fixture.detectChanges()
-
-    expect(
-      Array.from(
-        fixture.nativeElement.querySelectorAll('pngx-input-switch')
-      ).every((toggle: HTMLElement) => toggle.classList.contains('d-none'))
-    ).toBe(true)
-
-    component.slimSidebarAnimating.set(false)
-    fixture.detectChanges()
-
-    expect(
-      Array.from(
-        fixture.nativeElement.querySelectorAll('pngx-input-switch')
-      ).every((toggle: HTMLElement) => !toggle.classList.contains('d-none'))
-    ).toBe(true)
-    expect(
-      fixture.nativeElement.querySelector('[routerLink="dashboard"]').classList
-    ).toContain('pe-5')
-  })
-
  it('should show error on toggle slim sidebar if store settings fails', () => {
    jest.spyOn(console, 'warn').mockImplementation(() => {})
    const toastSpy = jest.spyOn(toastService, 'showError')
@@ -7,7 +7,6 @@ import {
 } from '@angular/cdk/drag-drop'
 import { NgClass } from '@angular/common'
 import { Component, HostListener, inject, OnInit, signal } from '@angular/core'
-import { FormsModule } from '@angular/forms'
 import { ActivatedRoute, Router, RouterModule } from '@angular/router'
 import {
  NgbCollapseModule,
@@ -22,11 +21,7 @@ import { Observable } from 'rxjs'
 import { first } from 'rxjs/operators'
 import { Document } from 'src/app/data/document'
 import { SavedView } from 'src/app/data/saved-view'
-import {
-  CollapsibleSection,
-  HideableSidebarItemID,
-  SETTINGS_KEYS,
-} from 'src/app/data/ui-settings'
+import { CollapsibleSection, SETTINGS_KEYS } from 'src/app/data/ui-settings'
 import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
 import { ComponentCanDeactivate } from 'src/app/guards/dirty-doc.guard'
 import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -53,7 +48,6 @@ import { ChatComponent } from '../chat/chat/chat.component'
 import { BrandMarkComponent } from '../common/logo/brand-mark/brand-mark.component'
 import { LogoComponent } from '../common/logo/logo.component'
 import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
-import { SwitchComponent } from '../common/input/switch/switch.component'
 import { DocumentDetailComponent } from '../document-detail/document-detail.component'
 import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
 import { GlobalSearchComponent } from './global-search/global-search.component'
@@ -82,8 +76,6 @@ const SCROLL_THRESHOLD = 16
    NgxBootstrapIconsModule,
    DragDropModule,
    TourNgBootstrap,
-    FormsModule,
-    SwitchComponent,
  ],
 })
 export class AppFrameComponent
@@ -106,7 +98,6 @@ export class AppFrameComponent
  readonly isMenuCollapsed = signal(true)
  readonly slimSidebarAnimating = signal(false)
  readonly mobileSearchHidden = signal(false)
-  readonly HideableSidebarItemID = HideableSidebarItemID
  private readonly versionSetting = this.settingsService.getSignal<string>(
    SETTINGS_KEYS.VERSION
  )
@@ -204,10 +195,6 @@ export class AppFrameComponent
    }, 200) // slightly longer than css animation for slim sidebar
  }

-  toggleSidebarItem(item: HideableSidebarItemID, visible: boolean): void {
-    this.settingsService.updateSidebarItemVisibility(item, visible)
-  }
-
  toggleAttributesSections(event?: Event): void {
    event?.preventDefault()
    event?.stopPropagation()
@@ -1,6 +1,6 @@
-<div [class.mb-3]="!compact">
-  <div [class.row]="!compact">
-    @if (!horizontal && !compact) {
+<div class="mb-3">
+  <div class="row">
+    @if (!horizontal) {
      <div class="d-flex align-items-center position-relative hidden-button-container col-md-3">
        <label class="form-label" [for]="inputId" [ngbTooltip]="showUnsetNote && isUnset ? tipContent: null" placement="end">
          {{title}}
@@ -17,8 +17,8 @@
    }
    <div [ngClass]="{'align-items-center': horizontal, 'd-flex': horizontal}">
      <div class="form-check form-switch">
-        <input #inputField type="checkbox" class="form-check-input" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" (blur)="onTouched()" [disabled]="disabled" [attr.aria-label]="compact ? title : null">
-        @if (horizontal && !compact) {
+        <input #inputField type="checkbox" class="form-check-input" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" (blur)="onTouched()" [disabled]="disabled">
+        @if (horizontal) {
          <label class="form-check-label" [class.text-muted]="showUnsetNote && isUnset" [for]="inputId" [ngbTooltip]="showUnsetNote && isUnset ? tipContent: null" placement="end">
            {{title}}
            @if (showUnsetNote && isUnset) {
@@ -48,14 +48,4 @@ describe('SwitchComponent', () => {
    component.value = undefined
    expect(component.isUnset).toBeTruthy()
  })
-
-  it('should support a compact layout', () => {
-    component.compact = true
-    component.title = 'Test switch'
-    fixture.detectChanges()
-
-    expect(fixture.nativeElement.querySelector('.mb-3')).toBeNull()
-    expect(fixture.nativeElement.querySelector('.row')).toBeNull()
-    expect(input.getAttribute('aria-label')).toEqual('Test switch')
-  })
 })
@@ -25,9 +25,6 @@ export class SwitchComponent extends AbstractInputComponent<boolean> {
  @Input()
  showUnsetNote: boolean = false

-  @Input()
-  compact: boolean = false
-
  constructor() {
    super()
  }
@@ -24,16 +24,6 @@ export enum CollapsibleSection {
  ATTRIBUTES = 'attributes',
 }

-export enum HideableSidebarItemID {
-  Dashboard = 'dashboard',
-  SavedViews = 'saved_views',
-  Workflows = 'workflows',
-  Mail = 'mail',
-  Documentation = 'documentation',
-}
-
-export const HIDEABLE_SIDEBAR_ITEM_IDS = Object.values(HideableSidebarItemID)
-
 export const PAPERLESS_GREEN_HEX = '#17541f'

 export const SETTINGS_KEYS = {
@@ -66,7 +56,6 @@ export const SETTINGS_KEYS = {
  NOTES_ENABLED: 'general-settings:notes-enabled',
  AUDITLOG_ENABLED: 'general-settings:auditlog-enabled',
  SLIM_SIDEBAR: 'general-settings:slim-sidebar',
-  SIDEBAR_HIDDEN_ITEMS: 'general-settings:sidebar:hidden-items',
  ATTRIBUTES_SECTIONS_COLLAPSED:
    'general-settings:attributes-sections-collapsed',
  UPDATE_CHECKING_ENABLED: 'general-settings:update-checking:enabled',
@@ -138,11 +127,6 @@ export const SETTINGS: UiSetting[] = [
    type: 'boolean',
    default: false,
  },
-  {
-    key: SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS,
-    type: 'array',
-    default: [],
-  },
  {
    key: SETTINGS_KEYS.ATTRIBUTES_SECTIONS_COLLAPSED,
    type: 'array',
@@ -14,11 +14,7 @@ import { CustomFieldDataType } from '../data/custom-field'
 import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
 import { SavedView } from '../data/saved-view'
 import { RemoteOCRModeConfig } from '../data/paperless-config'
-import {
-  HideableSidebarItemID,
-  SETTINGS_KEYS,
-  UiSettings,
-} from '../data/ui-settings'
+import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings'
 import { PermissionsService } from './permissions.service'
 import { CustomFieldsService } from './rest/custom-fields.service'
 import { SettingsService } from './settings.service'
@@ -234,35 +230,6 @@ describe('SettingsService', () => {
    expect(notesEnabled()).toBeFalsy()
  })

-  it('updates sidebar item visibility', () => {
-    httpTestingController
-      .expectOne(`${environment.apiBaseUrl}ui_settings/`)
-      .flush(ui_settings)
-
-    expect(
-      settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)
-    ).toBe(false)
-
-    settingsService.updateSidebarItemVisibility(
-      HideableSidebarItemID.Workflows,
-      false
-    )
-
-    expect(
-      settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)
-    ).toBe(true)
-    expect(settingsService.get(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS)).toEqual([])
-
-    settingsService.updateSidebarItemVisibility(
-      HideableSidebarItemID.Workflows,
-      true
-    )
-
-    expect(
-      settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)
-    ).toBe(false)
-  })
-
  it('updates setting signals when settings are reinitialized', () => {
    let req = httpTestingController.expectOne(
      `${environment.apiBaseUrl}ui_settings/`
@@ -24,7 +24,6 @@ import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
 import { RemoteOCRModeConfig } from '../data/paperless-config'
 import { SavedView } from '../data/saved-view'
 import {
-  HideableSidebarItemID,
  PAPERLESS_GREEN_HEX,
  SETTINGS,
  SETTINGS_KEYS,
@@ -314,18 +313,6 @@ export class SettingsService {
  readonly globalDropzoneEnabled = signal(true)
  readonly globalDropzoneActive = signal(false)
  readonly organizingSidebarSavedViews = signal(false)
-  readonly sidebarHiddenItemsEditing = signal<HideableSidebarItemID[] | null>(
-    null
-  )
-  readonly organizingSidebarItems = computed(
-    () => this.sidebarHiddenItemsEditing() !== null
-  )
-  readonly sidebarHiddenItemsEditingChanged = new EventEmitter<
-    HideableSidebarItemID[]
-  >()
-  readonly hiddenSidebarItems = this.getSignal<HideableSidebarItemID[]>(
-    SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS
-  )

  readonly allDisplayFields = signal<Array<{ id: DisplayField; name: string }>>(
    DEFAULT_DISPLAY_FIELDS
@@ -762,29 +749,6 @@ export class SettingsService {
    return this.storeSettings()
  }

-  sidebarItemIsHidden(item: HideableSidebarItemID): boolean {
-    return (
-      this.sidebarHiddenItemsEditing() ?? this.hiddenSidebarItems()
-    ).includes(item)
-  }
-
-  updateSidebarItemVisibility(
-    item: HideableSidebarItemID,
-    visible: boolean
-  ): void {
-    const hiddenItems = new Set(
-      this.sidebarHiddenItemsEditing() ?? this.hiddenSidebarItems()
-    )
-    if (visible) {
-      hiddenItems.delete(item)
-    } else {
-      hiddenItems.add(item)
-    }
-    const updatedHiddenItems = [...hiddenItems]
-    this.sidebarHiddenItemsEditing.set(updatedHiddenItems)
-    this.sidebarHiddenItemsEditingChanged.emit(updatedHiddenItems)
-  }
-
  updateSavedViewsVisibility(
    dashboardVisibleViewIds: number[],
    sidebarVisibleViewIds: number[]
@@ -2,7 +2,6 @@ from __future__ import annotations

 import logging
 import tempfile
-import uuid
 from pathlib import Path
 from typing import TYPE_CHECKING
 from typing import Literal
@@ -299,55 +298,53 @@ def modify_custom_fields(
 ) -> Literal["OK"]:
    qs = Document.objects.filter(id__in=doc_ids).only("pk")
    affected_docs = list(qs.values_list("pk", flat=True))
-    # Ensure add_custom_fields is a list of (int, value) tuples, supports old API
+    # Ensure add_custom_fields is a list of tuples, supports old API
    add_custom_fields = (
-        [(int(field), value) for field, value in add_custom_fields.items()]
+        add_custom_fields.items()
        if isinstance(add_custom_fields, dict)
-        else [(int(field), None) for field in add_custom_fields]
+        else [(field, None) for field in add_custom_fields]
    )

-    # Resolved once, instead of re-querying the same field for every document
-    custom_fields_by_id: dict[int, CustomField] = CustomField.objects.in_bulk(
-        [field_id for field_id, _ in add_custom_fields],
-    )
-    # Passed to update_or_create() below rather than a bare id, so the FK is
-    # cached on the created instance and auditlog's post_save receiver does
-    # not reload it per row. Only needed for additions. content is deferred:
-    # the one field here that is both large and unused.
-    docs_by_id: dict[int, Document] = (
-        Document.objects.defer("content").in_bulk(affected_docs)
-        if add_custom_fields
-        else {}
-    )
+    custom_fields = CustomField.objects.filter(
+        id__in=[int(field) for field, _ in add_custom_fields],
+    ).distinct()
    for field_id, value in add_custom_fields:
-        custom_field = custom_fields_by_id[field_id]
-        value_field = CustomFieldInstance.TYPE_TO_DATA_STORE_NAME_MAP[
-            custom_field.data_type
-        ]
-        is_doclink = custom_field.data_type == CustomField.FieldDataType.DOCUMENTLINK
        for doc_id in affected_docs:
-            if is_doclink and value and doc_id in value:
-                # Prevent self-linking
-                continue
+            defaults = {}
+            custom_field = custom_fields.get(id=field_id)
+            if custom_field:
+                value_field = CustomFieldInstance.TYPE_TO_DATA_STORE_NAME_MAP[
+                    custom_field.data_type
+                ]
+                defaults[value_field] = value
+                if (
+                    custom_field.data_type == CustomField.FieldDataType.DOCUMENTLINK
+                    and value
+                    and doc_id in value
+                ):
+                    # Prevent self-linking
+                    continue
            CustomFieldInstance.objects.update_or_create(
-                document=docs_by_id[doc_id],
-                field=custom_field,
-                defaults={value_field: value},
+                document_id=doc_id,
+                field_id=field_id,
+                defaults=defaults,
            )
-            if is_doclink:
-                reflect_doclinks(docs_by_id[doc_id], custom_field, value)
+            if custom_field.data_type == CustomField.FieldDataType.DOCUMENTLINK:
+                doc = Document.objects.get(id=doc_id)
+                reflect_doclinks(doc, custom_field, value)

-    # For doc link fields that are being removed, remove symmetrical links.
-    # select_related avoids a per-instance reload of the document and field.
+    # For doc link fields that are being removed, remove symmetrical links
    for doclink_being_removed_instance in CustomFieldInstance.objects.filter(
        document_id__in=affected_docs,
        field__id__in=remove_custom_fields,
        field__data_type=CustomField.FieldDataType.DOCUMENTLINK,
        value_document_ids__isnull=False,
-    ).select_related("field", "document"):
+    ):
        for target_doc_id in doclink_being_removed_instance.value:
            remove_doclink(
-                document=doclink_being_removed_instance.document,
+                document=Document.objects.get(
+                    id=doclink_being_removed_instance.document.id,
+                ),
                field=doclink_being_removed_instance.field,
                target_doc_id=target_doc_id,
            )
@@ -382,7 +379,7 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
        )
        delete_ids = list({*doc_ids, *version_ids})

-        Document.objects.filter(id__in=delete_ids).delete(transaction_id=uuid.uuid4())
+        Document.objects.filter(id__in=delete_ids).delete()

        from documents.search import get_backend

@@ -1180,13 +1177,10 @@ def remove_doclink(
    """
    Removes a 'symmetrical' link to `document` from the target document's existing custom field instance
    """
-    # select_related: a signal receiver (auditlog) touches .document/.field on
-    # the save() below, without this that is a per-call reload query
-    target_doc_field_instance = (
-        CustomFieldInstance.objects.filter(document_id=target_doc_id, field=field)
-        .select_related("document", "field")
-        .first()
-    )
+    target_doc_field_instance = CustomFieldInstance.objects.filter(
+        document_id=target_doc_id,
+        field=field,
+    ).first()
    if (
        target_doc_field_instance is not None
        and document.id in target_doc_field_instance.value
@@ -34,27 +34,6 @@ from paperless.signed_pickle import signed_pickle_loads

 logger = logging.getLogger("paperless.classifier")

-
-def _predict_with_threshold(classifier, X, threshold: float) -> int | None:
-    """
-    Return the predicted class id, or None if:
-    - the prediction is -1 (no match), or
-    - the winning class probability is below the configured threshold.
-
-    Using predict_proba() instead of predict() lets us apply a minimum-confidence
-    cutoff so that uncertain predictions are discarded rather than assigned.
-    """
-    probas = classifier.predict_proba(X)[0]
-    best_idx = int(probas.argmax())
-    best_class = int(classifier.classes_[best_idx])
-
-    if best_class == -1:
-        return None
-    if threshold > 0.0 and probas[best_idx] < threshold:
-        return None
-    return best_class
-
-
 ADVANCED_TEXT_PROCESSING_ENABLED = (
    settings.NLTK_LANGUAGE is not None and settings.NLTK_ENABLED
 )
@@ -123,8 +102,7 @@ class DocumentClassifier:
    # v8 - Added storage path classifier
    # v9 - Changed from hashing to time/ids for re-train check
    # v10 - HMAC-signed model file
-    # v11 - Use sample_weight for balanced training; predict_proba with threshold
-    FORMAT_VERSION = 11
+    FORMAT_VERSION = 10

    HMAC_SIZE = 32  # SHA-256 digest length

@@ -346,13 +324,6 @@ class DocumentClassifier:
        from sklearn.preprocessing import LabelBinarizer
        from sklearn.preprocessing import MultiLabelBinarizer

-        # MLPClassifier does not support class_weight directly
-        # (https://github.com/scikit-learn/scikit-learn/issues/9113), so we use
-        # compute_sample_weight to balance classes during training and prevent
-        # over-represented correspondents from dominating predictions.
-        # https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_sample_weight.html
-        from sklearn.utils.class_weight import compute_sample_weight
-
        # Step 2: vectorize data
        logger.debug("Vectorizing data...")
        notify("Vectorizing document content...")
@@ -398,7 +369,7 @@ class DocumentClassifier:
                self.tags_binarizer = MultiLabelBinarizer()
                labels_tags_vectorized = self.tags_binarizer.fit_transform(labels_tags)

-            self.tags_classifier = MLPClassifier(tol=0.01, random_state=0)
+            self.tags_classifier = MLPClassifier(tol=0.01)
            self.tags_classifier.fit(data_vectorized, labels_tags_vectorized)
        else:
            self.tags_classifier = None
@@ -409,12 +380,8 @@ class DocumentClassifier:
            notify(
                f"Training correspondent classifier ({num_correspondents} correspondent(s))...",
            )
-            self.correspondent_classifier = MLPClassifier(tol=0.01, random_state=0)
-            self.correspondent_classifier.fit(
-                data_vectorized,
-                labels_correspondent,
-                sample_weight=compute_sample_weight("balanced", labels_correspondent),
-            )
+            self.correspondent_classifier = MLPClassifier(tol=0.01)
+            self.correspondent_classifier.fit(data_vectorized, labels_correspondent)
        else:
            self.correspondent_classifier = None
            logger.debug(
@@ -426,12 +393,8 @@ class DocumentClassifier:
            notify(
                f"Training document type classifier ({num_document_types} type(s))...",
            )
-            self.document_type_classifier = MLPClassifier(tol=0.01, random_state=0)
-            self.document_type_classifier.fit(
-                data_vectorized,
-                labels_document_type,
-                sample_weight=compute_sample_weight("balanced", labels_document_type),
-            )
+            self.document_type_classifier = MLPClassifier(tol=0.01)
+            self.document_type_classifier.fit(data_vectorized, labels_document_type)
        else:
            self.document_type_classifier = None
            logger.debug(
@@ -443,11 +406,10 @@ class DocumentClassifier:
                "Training storage paths classifier...",
            )
            notify(f"Training storage path classifier ({num_storage_paths} path(s))...")
-            self.storage_path_classifier = MLPClassifier(tol=0.01, random_state=0)
+            self.storage_path_classifier = MLPClassifier(tol=0.01)
            self.storage_path_classifier.fit(
                data_vectorized,
                labels_storage_path,
-                sample_weight=compute_sample_weight("balanced", labels_storage_path),
            )
        else:
            self.storage_path_classifier = None
@@ -584,24 +546,24 @@ class DocumentClassifier:
    def predict_correspondent(self, content: str) -> int | None:
        if self.correspondent_classifier:
            X = self._vectorize(content)
-            predicted_id = _predict_with_threshold(
-                self.correspondent_classifier,
-                X,
-                settings.CLASSIFIER_MATCH_THRESHOLD,
-            )
-            return predicted_id
-        return None
+            correspondent_id = self.correspondent_classifier.predict(X)
+            if correspondent_id != -1:
+                return correspondent_id
+            else:
+                return None
+        else:
+            return None

    def predict_document_type(self, content: str) -> int | None:
        if self.document_type_classifier:
            X = self._vectorize(content)
-            predicted_id = _predict_with_threshold(
-                self.document_type_classifier,
-                X,
-                settings.CLASSIFIER_MATCH_THRESHOLD,
-            )
-            return predicted_id
-        return None
+            document_type_id = self.document_type_classifier.predict(X)
+            if document_type_id != -1:
+                return document_type_id
+            else:
+                return None
+        else:
+            return None

    def predict_tags(self, content: str) -> list[int]:
        from sklearn.utils.multiclass import type_of_target
@@ -627,10 +589,10 @@ class DocumentClassifier:
    def predict_storage_path(self, content: str) -> int | None:
        if self.storage_path_classifier:
            X = self._vectorize(content)
-            predicted_id = _predict_with_threshold(
-                self.storage_path_classifier,
-                X,
-                settings.CLASSIFIER_MATCH_THRESHOLD,
-            )
-            return predicted_id
-        return None
+            storage_path_id = self.storage_path_classifier.predict(X)
+            if storage_path_id != -1:
+                return storage_path_id
+            else:
+                return None
+        else:
+            return None
@@ -156,15 +156,6 @@ class FileStabilityTracker:
                    logger.debug(f"File disappeared during stability check: {path}")
                continue

-            # Stable, but empty: some scanners create a zero byte placeholder
-            # and only write the page some time later. Consuming it now can
-            # only fail  so drop it and let the writer's next event
-            # (or the periodic rescan) bring it back once it has content
-            if not tracked.last_size:
-                to_remove.append(path)
-                logger.debug("Ignoring stable but empty file: %s", path)
-                continue
-
            # File is stable, we can return it
            to_yield.append(path)
            logger.info(f"File is stable: {path}")
@@ -1,5 +1,4 @@
 import datetime
-import uuid
 from pathlib import Path
 from typing import Final

@@ -515,20 +514,13 @@ class Document(SoftDeleteModel, ModelWithOwner):  # type: ignore[django-manager-
    def delete(
        self,
        *args,
-        transaction_id=None,
        **kwargs,
    ):
-        # Versions must share the root's transaction ID so they are restored
-        # together by django-softdelete.
-        if transaction_id is None:
-            transaction_id = uuid.uuid4()
+        # If deleting a root document, move all its versions to trash as well.
        if self.root_document_id is None:
-            Document.objects.filter(root_document=self).delete(
-                transaction_id=transaction_id,
-            )
+            Document.objects.filter(root_document=self).delete()
        return super().delete(
            *args,
-            transaction_id=transaction_id,
            **kwargs,
        )

@@ -1256,26 +1256,20 @@ class DocumentSerializer(
                if "tags" in validated_data
                else []
            )
-            tags_being_added = Tag.objects.filter(id__in=tag_ids_being_added)
-            required_by_add_tags = set(tags_being_added)
-            for tag in tags_being_added:
-                required_by_add_tags.update(tag.get_ancestors())
-
-            # Remove its descendants too, except any that is being added in this same update
-            tags_to_remove = set()
-            for tag in Tag.objects.filter(is_inbox_tag=True):
-                if tag in required_by_add_tags:
-                    continue
-                tags_to_remove.add(tag)
-                tags_to_remove.update(tag.get_descendants())
-
+            inbox_tags_not_being_added = Tag.objects.filter(is_inbox_tag=True).exclude(
+                id__in=tag_ids_being_added,
+            )
            if "tags" in validated_data:
                validated_data["tags"] = [
-                    tag for tag in validated_data["tags"] if tag not in tags_to_remove
+                    tag
+                    for tag in validated_data["tags"]
+                    if tag not in inbox_tags_not_being_added
                ]
            else:
                validated_data["tags"] = [
-                    tag for tag in instance.tags.all() if tag not in tags_to_remove
+                    tag
+                    for tag in instance.tags.all()
+                    if tag not in inbox_tags_not_being_added
                ]

        if settings.AUDIT_LOG_ENABLED:
@@ -207,65 +207,3 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
        )
        self.assertEqual(resp.status_code, status.HTTP_400_BAD_REQUEST)
        self.assertIn("have not yet been deleted", resp.data["documents"][0])
-
-    def _make_versioned_document(self) -> tuple[Document, list[Document]]:
-        root = Document.objects.create(
-            title="root",
-            content="root-content",
-            checksum="root",
-            mime_type="application/pdf",
-        )
-        versions = [
-            Document.objects.create(
-                title=f"v{index}",
-                content=f"v{index}-content",
-                checksum=f"v{index}",
-                mime_type="application/pdf",
-                root_document=root,
-                version_index=index,
-            )
-            for index in range(1, 3)
-        ]
-        return root, versions
-
-    def test_api_trash_restore_document_restores_its_versions(self) -> None:
-        """
-        GIVEN:
-            - Existing document with two versions
-        WHEN:
-            - API request to delete the document
-            - API request to restore it from the trash
-        THEN:
-            - Only the document itself is listed in the trash
-            - A version cannot be restored without its root
-            - The document is restored together with all of its versions
-        """
-        root, versions = self._make_versioned_document()
-
-        self.client.force_login(user=self.user)
-        self.client.delete(f"/api/documents/{root.pk}/")
-        self.assertEqual(Document.deleted_objects.count(), 3)
-
-        resp = self.client.get("/api/trash/")
-        self.assertEqual(resp.status_code, status.HTTP_200_OK)
-        self.assertEqual(resp.data["count"], 1)
-        self.assertEqual(resp.data["results"][0]["id"], root.pk)
-
-        # A version cannot be restored while its root remains in the trash.
-        resp = self.client.post(
-            "/api/trash/",
-            {"action": "restore", "documents": [versions[0].pk]},
-        )
-        self.assertEqual(resp.status_code, status.HTTP_400_BAD_REQUEST)
-        self.assertIn("Restore the root document", resp.data["documents"][0])
-
-        resp = self.client.post(
-            "/api/trash/",
-            {"action": "restore", "documents": [root.pk]},
-        )
-        self.assertEqual(resp.status_code, status.HTTP_200_OK)
-        self.assertEqual(Document.deleted_objects.count(), 0)
-        self.assertCountEqual(
-            Document.objects.filter(root_document=root).values_list("id", flat=True),
-            [version.pk for version in versions],
-        )
@@ -392,11 +392,6 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
        self.assertFalse(Document.objects.filter(id=self.doc1.id).exists())
        self.assertFalse(Document.objects.filter(id=version.id).exists())

-        Document.deleted_objects.get(id=self.doc1.id).restore(strict=False)
-
-        self.assertTrue(Document.objects.filter(id=self.doc1.id).exists())
-        self.assertTrue(Document.objects.filter(id=version.id).exists())
-
    def test_delete_version_document_keeps_root(self) -> None:
        version = Document.objects.create(
            checksum="A-v1",
@@ -3,7 +3,6 @@ import warnings
 from pathlib import Path
 from unittest import mock

-import numpy as np
 import pytest
 from django.conf import settings
 from django.test import TestCase
@@ -12,7 +11,6 @@ from django.test import override_settings
 from documents.classifier import ClassifierModelCorruptError
 from documents.classifier import DocumentClassifier
 from documents.classifier import IncompatibleClassifierVersionError
-from documents.classifier import _predict_with_threshold
 from documents.classifier import load_classifier
 from documents.models import Correspondent
 from documents.models import Document
@@ -627,103 +625,6 @@ class TestClassifier(DirectoriesMixin, TestCase):
        self.assertEqual(self.classifier.predict_storage_path(doc1.content), sp.pk)
        self.assertIsNone(self.classifier.predict_storage_path(doc2.content))

-    def test_predict_rejects_prediction_below_match_threshold(self) -> None:
-        """
-        GIVEN:
-            - Classifiers trained against test data with confident predictions
-        WHEN:
-            - CLASSIFIER_MATCH_THRESHOLD exceeds the model's confidence
-        THEN:
-            - Every predict_* method discards the match in favor of no match
-        """
-        c1 = Correspondent.objects.create(
-            name="c1",
-            matching_algorithm=Correspondent.MATCH_AUTO,
-        )
-        dt1 = DocumentType.objects.create(
-            name="dt1",
-            matching_algorithm=DocumentType.MATCH_AUTO,
-        )
-        sp1 = StoragePath.objects.create(
-            name="sp1",
-            matching_algorithm=StoragePath.MATCH_AUTO,
-        )
-
-        doc1 = Document.objects.create(
-            title="doc1",
-            content="this is a document from c1",
-            correspondent=c1,
-            document_type=dt1,
-            storage_path=sp1,
-            checksum="A",
-        )
-        Document.objects.create(
-            title="doc2",
-            content="this is a document from no one",
-            checksum="B",
-        )
-
-        self.classifier.train()
-
-        predictors = {
-            "correspondent": self.classifier.predict_correspondent,
-            "document_type": self.classifier.predict_document_type,
-            "storage_path": self.classifier.predict_storage_path,
-        }
-        # No real prediction can reach a confidence this high, so this
-        # isolates the threshold check from the model's actual output.
-        with override_settings(CLASSIFIER_MATCH_THRESHOLD=0.999999):
-            for name, predict in predictors.items():
-                with self.subTest(field=name):
-                    self.assertIsNone(predict(doc1.content))
-
-    def test_train_uses_balanced_sample_weight(self) -> None:
-        """
-        GIVEN:
-            - A training set with correspondents, document types and storage paths
-        WHEN:
-            - The classifier is trained
-        THEN:
-            - Each MLP classifier is fit with balanced sample weights, so that
-              over-represented classes don't dominate predictions
-        """
-        c1 = Correspondent.objects.create(
-            name="c1",
-            matching_algorithm=Correspondent.MATCH_AUTO,
-        )
-        dt1 = DocumentType.objects.create(
-            name="dt1",
-            matching_algorithm=DocumentType.MATCH_AUTO,
-        )
-        sp1 = StoragePath.objects.create(
-            name="sp1",
-            matching_algorithm=StoragePath.MATCH_AUTO,
-        )
-
-        Document.objects.create(
-            title="doc1",
-            content="this is a document from c1",
-            correspondent=c1,
-            document_type=dt1,
-            storage_path=sp1,
-            checksum="A",
-        )
-        Document.objects.create(
-            title="doc2",
-            content="this is a document from no one",
-            checksum="B",
-        )
-
-        with mock.patch(
-            "sklearn.utils.class_weight.compute_sample_weight",
-            return_value=None,
-        ) as mocked_compute_sample_weight:
-            self.classifier.train()
-
-        self.assertEqual(mocked_compute_sample_weight.call_count, 3)
-        for call in mocked_compute_sample_weight.call_args_list:
-            self.assertEqual(call.args[0], "balanced")
-
    def test_one_tag_predict(self) -> None:
        t1 = Tag.objects.create(name="t1", matching_algorithm=Tag.MATCH_AUTO, pk=12)

@@ -909,52 +810,6 @@ class TestClassifier(DirectoriesMixin, TestCase):
            load_classifier(raise_exception=True)


-class _StubProbaClassifier:
-    """
-    A fake scikit-learn classifier exposing just enough of the API for
-    `_predict_with_threshold`: `classes_` and `predict_proba`.
-    """
-
-    def __init__(self, classes: list[int], probabilities: list[float]) -> None:
-        self.classes_ = np.array(classes)
-        self._probabilities = np.array([probabilities])
-
-    def predict_proba(self, X) -> np.ndarray:
-        return self._probabilities
-
-
-@pytest.mark.parametrize(
-    ("classes", "probabilities", "threshold", "expected"),
-    [
-        # confident prediction above the threshold is returned
-        ([-1, 3], [0.1, 0.9], 0.6, 3),
-        # prediction below the threshold is discarded
-        ([-1, 3], [0.45, 0.55], 0.6, None),
-        # boundary: exactly at the threshold is accepted, not discarded
-        ([-1, 3], [0.4, 0.6], 0.6, 3),
-        # the winning class is the "no match" pseudo-class, regardless of its
-        # own confidence
-        ([-1, 3], [0.99, 0.01], 0.0, None),
-        # threshold of 0.0 disables the confidence check entirely
-        ([-1, 3], [0.45, 0.55], 0.0, 3),
-    ],
-)
-def test_predict_with_threshold(classes, probabilities, threshold, expected) -> None:
-    classifier = _StubProbaClassifier(classes, probabilities)
-    result = _predict_with_threshold(classifier, X=None, threshold=threshold)
-    assert result == expected
-
-
-def test_classifier_match_threshold_default() -> None:
-    """
-    GIVEN:
-        - No PAPERLESS_CLASSIFIER_MATCH_THRESHOLD environment variable is set
-    THEN:
-        - The classifier match threshold defaults to 0.6
-    """
-    assert settings.CLASSIFIER_MATCH_THRESHOLD == 0.6
-
-
 def test_preprocess_content() -> None:
    """
    GIVEN:
@@ -110,7 +110,7 @@ class TestDocument(TestCase):
            checksum="checksum",
            mime_type="application/pdf",
        )
-        version = Document.objects.create(
+        Document.objects.create(
            root_document=root,
            correspondent=root.correspondent,
            title="Version",
@@ -124,10 +124,6 @@ class TestDocument(TestCase):
        self.assertEqual(Document.objects.count(), 0)
        self.assertEqual(Document.deleted_objects.count(), 2)

-        root.restore(strict=False)
-
-        self.assertTrue(Document.objects.filter(pk=version.pk).exists())
-
    def test_file_name(self) -> None:
        doc = Document(
            mime_type="application/pdf",
@@ -136,23 +136,6 @@ def wait_for_mock_call(
    return False


-def sleep_past_stability(
-    owner: FileStabilityTracker | ConsumerThread,
-    *,
-    windows: float = 1.5,
-) -> None:
-    """
-    Block until a tracked file's stability window has certainly elapsed.
-
-    Args:
-        owner: The tracker, or the consumer thread running one, whose
-               configured stability delay sets the wait.
-        windows: How many stability windows to wait, giving slop for a slow
-                 or loaded test runner.
-    """
-    sleep(owner.stability_delay * windows)
-
-
 class TestTrackedFile:
    """Tests for the TrackedFile dataclass."""

@@ -278,56 +261,6 @@ class TestFileStabilityTracker:
        assert len(stable) == 0
        assert stability_tracker.pending_count == 1

-    def test_get_stable_files_skips_empty_file(
-        self,
-        stability_tracker: FileStabilityTracker,
-        tmp_path: Path,
-    ) -> None:
-        """
-        GIVEN:
-            - A zero byte file, tracked and past its stability delay
-        WHEN:
-            - Stable files are collected
-        THEN:
-            - The file is not yielded for consumption
-            - The file is dropped from tracking rather than held, so an
-              abandoned placeholder does not keep the watch loop awake
-        """
-        empty = tmp_path / "scan.pdf"
-        empty.write_bytes(b"")
-        stability_tracker.track(empty, Change.added)
-        sleep_past_stability(stability_tracker)
-
-        stable = list(stability_tracker.get_stable_files())
-
-        assert stable == []
-        assert stability_tracker.pending_count == 0
-
-    def test_empty_file_is_yielded_once_content_arrives(
-        self,
-        stability_tracker: FileStabilityTracker,
-        tmp_path: Path,
-    ) -> None:
-        """
-        GIVEN:
-            - A zero byte file which was dropped from tracking while empty
-        WHEN:
-            - The writer fills the file and a new event re-tracks it
-        THEN:
-            - The file is yielded for consumption once it is stable
-        """
-        target = tmp_path / "scan.pdf"
-        target.write_bytes(b"")
-        stability_tracker.track(target, Change.added)
-        sleep_past_stability(stability_tracker)
-        assert list(stability_tracker.get_stable_files()) == []
-
-        target.write_bytes(b"%PDF-1.4 content")
-        stability_tracker.track(target, Change.modified)
-        sleep_past_stability(stability_tracker)
-
-        assert list(stability_tracker.get_stable_files()) == [target]
-
    def test_get_stable_files_deleted_during_check(self, temp_file: Path) -> None:
        """Test deleted file is not returned during stability check."""
        tracker = FileStabilityTracker(stability_delay=0.1)
@@ -946,51 +879,6 @@ class TestCommandWatch:

        mock_consume_file_delay.apply_async.assert_called()

-    def test_scanner_placeholder_is_not_consumed_while_empty(
-        self,
-        consumption_dir: Path,
-        sample_pdf: Path,
-        mock_consume_file_delay: MagicMock,
-        start_consumer: Callable[..., ConsumerThread],
-    ) -> None:
-        """
-        GIVEN:
-            - A scanner which creates a zero byte placeholder and only writes
-              the page some time later (GH discussion #13969)
-        WHEN:
-            - The placeholder sits untouched well past the stability delay
-            - The scanner then writes the real content
-        THEN:
-            - The empty placeholder is never queued, as it could only fail
-              with "Unsupported mime type inode/x-empty"
-            - The file is queued exactly once, when the content lands
-        """
-        thread = start_consumer(stability_delay=0.2)
-
-        target = consumption_dir / "scan.pdf"
-        target.write_bytes(b"")  # the scanner's placeholder
-
-        # Well past the stability delay: the old behaviour queued it here.
-        sleep_past_stability(thread, windows=5)
-        if thread.exception:
-            raise thread.exception
-        assert mock_consume_file_delay.apply_async.call_count == 0
-
-        shutil.copy(sample_pdf, target)  # the scanner finishes the page
-
-        assert wait_for_mock_call(
-            mock_consume_file_delay.apply_async,
-            timeout_s=5.0,
-        )
-        if thread.exception:
-            raise thread.exception
-
-        assert mock_consume_file_delay.apply_async.call_count == 1
-        queued_doc = mock_consume_file_delay.apply_async.call_args.kwargs["kwargs"][
-            "input_doc"
-        ]
-        assert queued_doc.original_file.name == "scan.pdf"
-
    def test_ignores_macos_files(
        self,
        consumption_dir: Path,
@@ -2,7 +2,6 @@ from unittest import mock

 from django.contrib.auth.models import Permission
 from django.contrib.auth.models import User
-from rest_framework import status
 from rest_framework.test import APITestCase

 from documents import bulk_edit
@@ -109,44 +108,6 @@ class TestTagHierarchy(DirectoriesMixin, APITestCase):
        self.document.refresh_from_db()
        assert self.document.tags.count() == 0

-    def test_remove_inbox_tags_removes_nested_children(self) -> None:
-        inbox = Tag.objects.create(name="Inbox", is_inbox_tag=True)
-        nested = Tag.objects.create(name="Nested", tn_parent=inbox)
-        self.document.add_nested_tags([nested])
-
-        resp = self.client.patch(
-            f"/api/documents/{self.document.pk}/",
-            {"title": "new title", "remove_inbox_tags": True},
-            format="json",
-        )
-        assert resp.status_code == status.HTTP_200_OK
-        self.document.refresh_from_db()
-        assert self.document.tags.count() == 0
-
-        # A subsequent save must not re-add the inbox tag as an ancestor
-        resp = self.client.patch(
-            f"/api/documents/{self.document.pk}/",
-            {"title": "another title", "tags": [], "remove_inbox_tags": True},
-            format="json",
-        )
-        assert resp.status_code == status.HTTP_200_OK
-        self.document.refresh_from_db()
-        assert self.document.tags.count() == 0
-
-    def test_remove_inbox_tags_keeps_inbox_when_nested_child_added(self) -> None:
-        inbox = Tag.objects.create(name="Inbox", is_inbox_tag=True)
-        nested = Tag.objects.create(name="Nested", tn_parent=inbox)
-        self.document.add_nested_tags([inbox])
-
-        self.client.patch(
-            f"/api/documents/{self.document.pk}/",
-            {"tags": [nested.pk], "remove_inbox_tags": True},
-            format="json",
-        )
-        self.document.refresh_from_db()
-        tags = set(self.document.tags.values_list("pk", flat=True))
-        assert tags == {inbox.pk, nested.pk}
-
    def test_bulk_edit_respects_hierarchy(self) -> None:
        bulk_edit.add_tag([self.document.pk], self.child.pk)
        self.document.refresh_from_db()
@@ -32,7 +32,6 @@ from documents.signals.handlers import update_llm_suggestions_cache
 from documents.tests.utils import DirectoriesMixin
 from documents.tests.utils import read_streaming_response
 from paperless.models import ApplicationConfiguration
-from paperless_ai.exceptions import LLMProviderError
 from paperless_ai.exceptions import LLMTimeoutError


@@ -738,38 +737,6 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
            get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
        )

-    @patch("documents.views.get_ai_document_classification")
-    @override_settings(
-        AI_ENABLED=True,
-        LLM_BACKEND="openai-like",
-    )
-    def test_ai_suggestions_with_llm_provider_error(
-        self,
-        mock_get_ai_classification,
-    ) -> None:
-        mock_get_ai_classification.side_effect = LLMProviderError(
-            "confidential provider response",
-        )
-
-        self.client.force_login(user=self.user)
-        response = self.client.get(
-            f"/api/documents/{self.document.pk}/ai_suggestions/",
-        )
-
-        self.assertEqual(response.status_code, status.HTTP_502_BAD_GATEWAY)
-        self.assertEqual(
-            response.json(),
-            {
-                "ai": [
-                    "AI backend rejected the request. Check logs for details.",
-                ],
-            },
-        )
-        self.assertNotIn("confidential provider response", response.content.decode())
-        self.assertIsNone(
-            get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
-        )
-
    @patch("documents.views.get_ai_document_classification")
    @override_settings(
        AI_ENABLED=True,
@@ -252,7 +252,6 @@ from paperless.views import StandardPagination
 from paperless_ai.ai_classifier import get_ai_document_classification
 from paperless_ai.ai_classifier import get_llm_output_language
 from paperless_ai.chat import stream_chat_with_documents
-from paperless_ai.exceptions import LLMProviderError
 from paperless_ai.exceptions import LLMTimeoutError
 from paperless_ai.matching import extract_unmatched_names
 from paperless_ai.matching import match_correspondents_by_name
@@ -1604,22 +1603,6 @@ class DocumentViewSet(
                    {"ai": [_("AI backend request timed out.")]},
                    status=status.HTTP_503_SERVICE_UNAVAILABLE,
                )
-            except LLMProviderError:
-                logger.exception(
-                    "AI backend rejected the request for document %s",
-                    doc.pk,
-                )
-                return Response(
-                    {
-                        "ai": [
-                            _(
-                                "AI backend rejected the request. "
-                                "Check logs for details.",
-                            ),
-                        ],
-                    },
-                    status=status.HTTP_502_BAD_GATEWAY,
-                )
            set_llm_suggestions_cache(
                doc.pk,
                llm_suggestions,
@@ -5454,10 +5437,7 @@ class TrashView(ListModelMixin, PassUserMixin):

    model = Document

-    # A version is listed separately only when its root is not in the trash.
-    queryset = Document.deleted_objects.exclude(
-        root_document_id__in=Document.deleted_objects.values("id"),
-    )
+    queryset = Document.deleted_objects.all()

    def get(self, request: Request, format: str | None = None) -> Response:
        self.serializer_class = DocumentSerializer
@@ -5488,15 +5468,7 @@ class TrashView(ListModelMixin, PassUserMixin):
            return HttpResponseForbidden("Insufficient permissions")
        action = serializer.validated_data.get("action")
        if action == "restore":
-            restored = list(self.get_queryset().filter(id__in=doc_ids))
-            if len(restored) != len(doc_ids):
-                raise ValidationError(
-                    {
-                        "documents": [
-                            "Restore the root document instead of one of its versions.",
-                        ],
-                    },
-                )
+            restored = list(Document.deleted_objects.filter(id__in=doc_ids))
            for doc in restored:
                doc.restore(strict=False)
            if restored:
@@ -96,13 +96,6 @@ MODEL_FILE = get_path_from_env(
    "PAPERLESS_MODEL_FILE",
    DATA_DIR / "classification_model.pickle",
 )
-
-# Minimum confidence (0.0-1.0) for the ML classifier to assign a correspondent,
-# document type, or storage path. 0.0 disables the threshold.
-CLASSIFIER_MATCH_THRESHOLD: Final[float] = get_float_from_env(
-    "PAPERLESS_CLASSIFIER_MATCH_THRESHOLD",
-    0.6,
-)
 LLM_INDEX_DIR = DATA_DIR / "llm_index"
 LLM_INDEX_LOCK = LLM_INDEX_DIR / "index.lock"
 # Cross-process read/write lock guarding the LLM index compaction/migration
@@ -157,6 +150,7 @@ INSTALLED_APPS = [
    "drf_spectacular",
    "drf_spectacular_sidecar",
    "treenode",
+    "paperless_benchmark.apps.PaperlessBenchmarkConfig",
    *env_apps,
 ]

@@ -4,24 +4,21 @@ from django.conf import settings
 from django.contrib.auth.models import User

 from documents.models import Document
-from documents.permissions import permitted_object_ids
-from documents.permissions import restrict_queryset_to_visible
-from documents.permissions import user_is_unrestricted
+from documents.permissions import get_objects_for_user_owner_aware
 from paperless.config import AIConfig
 from paperless_ai.base_model import ClassificationSuggestions
 from paperless_ai.base_model import TaxonomyChoiceDict
 from paperless_ai.base_model import classification_suggestions_to_model
 from paperless_ai.client import AIClient
 from paperless_ai.db import db_connection_released
+from paperless_ai.indexing import _node_document_ids
 from paperless_ai.indexing import retrieve_similar_nodes
 from paperless_ai.indexing import truncate_content
 from paperless_ai.prompts.context import ClassificationPromptContext
 from paperless_ai.prompts.context import LocalizationPromptContext
 from paperless_ai.prompts.context import RagContextPromptContext
 from paperless_ai.prompts.render import render_prompt
-from paperless_ai.taxonomy import SimilarDocument
 from paperless_ai.taxonomy import TaxonomyCandidates
-from paperless_ai.taxonomy import _node_document_weights
 from paperless_ai.taxonomy import build_taxonomy_candidates
 from paperless_ai.taxonomy import empty_taxonomy_candidates
 from paperless_ai.taxonomy import format_taxonomy_for_prompt
@@ -40,48 +37,6 @@ logger = logging.getLogger("paperless_ai.rag_classifier")
 TAXONOMY_CANDIDATE_TOP_K = 15


-def _fulltext_similar_documents(
-    document: Document,
-    user: User | None,
-    top_k: int,
-) -> list[SimilarDocument]:
-    """Rank-based fallback when no embedding backend is configured. Uses
-    Tantivy's "More Like This" (term-overlap similarity) instead of vector
-    similarity - cruder, but far better than no candidates at all.
-    more_like_this_ids returns only a ranked ID list, no scores, so weight is
-    synthesized from rank (descending from top_k) rather than claiming a
-    similarity magnitude that doesn't exist. An unrestricted user (none, or an
-    active superuser - see user_is_unrestricted) is normalized to ``None``
-    before calling, since the backend's permission filter has no superuser
-    short-circuit of its own. Results are re-checked with
-    restrict_queryset_to_visible() since Tantivy's indexed permission fields
-    lag the DB via async reindexing.
-    """
-    from documents.search import get_backend
-
-    unrestricted = user_is_unrestricted(user)
-    search_user = None if unrestricted else user
-    backend = get_backend()
-    similar_ids = backend.more_like_this_ids(
-        document.pk,
-        user=search_user,
-        limit=top_k,
-    )
-    if not unrestricted:
-        allowed_ids = set(
-            restrict_queryset_to_visible(
-                Document.objects.filter(pk__in=similar_ids),
-                user,
-                "view_document",
-            ).values_list("pk", flat=True),
-        )
-        similar_ids = [doc_id for doc_id in similar_ids if doc_id in allowed_ids]
-    return [
-        SimilarDocument(document_id=doc_id, weight=float(top_k - rank))
-        for rank, doc_id in enumerate(similar_ids)
-    ]
-
-
 def get_language_name(language_code: str) -> str:
    normalized_language_code = language_code.lower()
    for code, name in settings.LANGUAGES:
@@ -181,52 +136,43 @@ def get_taxonomy_context(
    user: User | None = None,
    max_docs: int = 5,
 ) -> tuple[TaxonomyCandidates, str]:
-    """One retrieval feeds both taxonomy candidates and RAG text context. Uses
-    vector similarity when an embedding backend is configured, otherwise
-    falls back to Tantivy full-text "More Like This" similarity - see
-    _fulltext_similar_documents. On any retrieval failure, degrades to empty
-    candidates/context rather than propagating the exception - neither a
-    vector-store outage nor a search-index issue should block classification,
-    only its context-assisted enrichment.
+    """One retrieval feeds both taxonomy candidates and RAG text context.
+    On any retrieval failure, degrades to empty candidates/context rather than
+    propagating the exception - a vector-store outage should not block
+    classification, only its RAG-assisted enrichment.
    """
-    ai_config = AIConfig()
    try:
-        if ai_config.llm_embedding_backend:
-            # None means "no restriction" to retrieve_similar_nodes. An
-            # unrestricted user (no user at all, or an active superuser -- see
-            # user_is_unrestricted) can see every document, so skip
-            # materializing every visible pk into a Python list and passing it
-            # through as an IN filter: for a large library that is a wasted
-            # quadratic scan in the vector store at best, and past ~32,763
-            # documents a hard sqlite3.OperationalError (SQLite's
-            # bound-parameter limit) at worst.
-            # permitted_object_ids() has its own superuser shortcut that would
-            # return every Document's id anyway, so this changes nothing about
-            # which documents are considered -- only how we get there.
-            visible_document_ids = (
-                None
-                if user_is_unrestricted(user)
-                else list(permitted_object_ids(user, Document, "view_document"))
-            )
-            nodes = retrieve_similar_nodes(
-                document,
-                top_k=TAXONOMY_CANDIDATE_TOP_K,
-                document_ids=visible_document_ids,
-            )
-            similar_documents = _node_document_weights(nodes)
-        else:
-            # See _fulltext_similar_documents: it applies its own permission
-            # filter via `user`, so no visible-document-id list is needed here.
-            similar_documents = _fulltext_similar_documents(
-                document,
-                user,
-                top_k=TAXONOMY_CANDIDATE_TOP_K,
+        # None means "no restriction" to retrieve_similar_nodes. A superuser
+        # (like no user at all) can see every document, so skip materializing
+        # every visible pk into a Python list and passing it through as an IN
+        # filter: for a large library that is a wasted quadratic scan in the
+        # vector store at best, and past ~32,763 documents a hard
+        # sqlite3.OperationalError (SQLite's bound-parameter limit) at worst.
+        # get_objects_for_user_owner_aware() would return every Document for a
+        # superuser anyway (guardian's own with_superuser shortcut), so this
+        # changes nothing about which documents are considered -- only how we
+        # get there.
+        visible_document_ids = (
+            None
+            if user is None or user.is_superuser
+            else list(
+                get_objects_for_user_owner_aware(
+                    user,
+                    "view_document",
+                    Document,
+                ).values_list("pk", flat=True),
            )
+        )
+        nodes = retrieve_similar_nodes(
+            document,
+            top_k=TAXONOMY_CANDIDATE_TOP_K,
+            document_ids=visible_document_ids,
+        )

-        candidates = build_taxonomy_candidates(similar_documents, user)
+        candidates = build_taxonomy_candidates(nodes, user)

-        # similar_documents is already ordered by descending weight; don't lose it.
-        similar_document_ids = [s["document_id"] for s in similar_documents]
+        # ``nodes`` are already ordered by descending vector similarity; don't lose it.
+        similar_document_ids = list(dict.fromkeys(_node_document_ids(nodes)))
        similar_documents_by_id = Document.objects.in_bulk(similar_document_ids)
        similar_docs = [
            similar_documents_by_id[document_id]
@@ -240,8 +186,8 @@ def get_taxonomy_context(
            context_blocks.append(f"TITLE: {title}\n{text}")
    except Exception:
        logger.exception(
-            "Failed to retrieve similar-document context for document %s; "
-            "continuing without taxonomy candidates or similar-document context.",
+            "Failed to retrieve RAG neighbours for document %s; continuing "
+            "without taxonomy candidates or similar-document context.",
            document.pk,
        )
        return empty_taxonomy_candidates(), ""
@@ -295,13 +241,17 @@ def get_ai_document_classification(
 ) -> ClassificationSuggestions:
    ai_config = AIConfig()

-    candidates, context = get_taxonomy_context(document, user)
-    prompt = build_prompt_with_rag(
-        document,
-        ai_config,
-        candidates=candidates,
-        context=context,
-    )
+    if ai_config.llm_embedding_backend:
+        candidates, context = get_taxonomy_context(document, user)
+        prompt = build_prompt_with_rag(
+            document,
+            ai_config,
+            candidates=candidates,
+            context=context,
+        )
+    else:
+        candidates = empty_taxonomy_candidates()
+        prompt = build_prompt_without_rag(document, ai_config, candidates=candidates)

    client = AIClient()
    # Hand the pooled DB connection back while the (slow) LLM query runs so it
@@ -22,7 +22,6 @@ from paperless.network import validate_outbound_http_url
 from paperless_ai.base_model import ClassificationSuggestions
 from paperless_ai.base_model import DocumentClassifierSchema
 from paperless_ai.base_model import model_to_classification_suggestions
-from paperless_ai.exceptions import LLMProviderError
 from paperless_ai.exceptions import LLMTimeoutError

 logger = logging.getLogger("paperless_ai.client")
@@ -133,7 +132,7 @@ class AIClient:
        from llama_index.core.llms import ChatMessage

        if self.settings.llm_backend == LLMBackend.OLLAMA:
-            with self._normalize_errors():
+            with self._normalize_timeouts():
                result = self.llm.chat(
                    [ChatMessage(role="user", content=prompt)],
                    format=DocumentClassifierSchema.model_json_schema(),
@@ -154,7 +153,7 @@ class AIClient:
            content=f"{prompt}\n\n"
            f"Answer by calling the {tool.metadata.name} tool. Do not write the answer as text.",
        )
-        with self._normalize_errors():
+        with self._normalize_timeouts():
            result = self.llm.chat_with_tools(
                tools=[tool],
                user_msg=user_msg,
@@ -174,7 +173,7 @@ class AIClient:
        )

    @contextmanager
-    def _normalize_errors(self) -> Iterator[None]:
+    def _normalize_timeouts(self) -> Iterator[None]:
        try:
            yield
        except httpx.TimeoutException as exc:
@@ -182,23 +181,8 @@ class AIClient:
        except Exception as exc:
            if self._is_openai_timeout(exc):
                raise LLMTimeoutError from exc
-            if self._is_provider_error(exc):
-                raise LLMProviderError from exc
            raise

-    def _is_provider_error(self, exc: Exception) -> bool:
-        if self.settings.llm_backend == LLMBackend.OLLAMA:
-            from ollama import ResponseError
-
-            return isinstance(exc, ResponseError)
-
-        if self.settings.llm_backend == LLMBackend.OPENAI_LIKE:
-            from openai import APIStatusError
-
-            return isinstance(exc, APIStatusError)
-
-        return False
-
    def _is_openai_timeout(self, exc: Exception) -> bool:
        if self.settings.llm_backend != LLMBackend.OPENAI_LIKE:
            return False
@@ -1,6 +1,2 @@
 class LLMTimeoutError(Exception):
    pass
-
-
-class LLMProviderError(Exception):
-    """The LLM backend rejected the request."""
@@ -721,3 +721,20 @@ def retrieve_similar_nodes(
            continue
        filtered.append(node)
    return filtered
+
+
+def _node_document_ids(nodes: list["NodeWithScore"]) -> list[int]:
+    document_ids: list[int] = []
+    for node in nodes:
+        document_id = node.metadata.get("document_id")
+        if document_id is None:  # pragma: no cover
+            # See the matching guard in retrieve_similar_nodes() above.
+            continue
+        try:
+            document_ids.append(int(document_id))
+        except ValueError:  # pragma: no cover
+            logger.warning(
+                "Skipping LLM index result with invalid document_id %r.",
+                document_id,
+            )
+    return document_ids
@@ -31,11 +31,6 @@ class TaxonomyCandidate(TypedDict):
    weight: float


-class SimilarDocument(TypedDict):
-    document_id: int
-    weight: float
-
-
 class TaxonomyCandidates(TypedDict):
    tags: list[TaxonomyCandidate]
    document_types: list[TaxonomyCandidate]
@@ -54,10 +49,10 @@ def empty_taxonomy_candidates() -> TaxonomyCandidates:
    )


-def _node_document_weights(nodes: list["NodeWithScore"]) -> list[SimilarDocument]:
-    """Sum each node's similarity score into its document_id (a document can
-    appear via multiple chunks/nodes) and return one SimilarDocument per
-    distinct document_id."""
+def _node_document_weights(nodes: list["NodeWithScore"]) -> dict[int, float]:
+    """document_id -> that node's similarity score, summed if a document_id
+    appears more than once across the retrieved nodes (e.g. multiple chunks
+    of the same source document)."""
    weights: dict[int, float] = defaultdict(float)
    for node in nodes:
        document_id = node.metadata.get("document_id")
@@ -70,14 +65,7 @@ def _node_document_weights(nodes: list["NodeWithScore"]) -> list[SimilarDocument
            weights[int(document_id)] += float(node.score or 0.0)
        except (TypeError, ValueError):  # pragma: no cover
            continue
-    return sorted(
-        (
-            SimilarDocument(document_id=document_id, weight=weight)
-            for document_id, weight in weights.items()
-        ),
-        key=lambda similar: similar["weight"],
-        reverse=True,
-    )
+    return weights


 def _visible_ranked_candidates(
@@ -113,25 +101,20 @@ def _visible_ranked_candidates(


 def build_taxonomy_candidates(
-    similar_documents: list[SimilarDocument],
+    nodes: list["NodeWithScore"],
    user: User | None,
 ) -> TaxonomyCandidates:
-    """Resolve each similar document's id to a live Document, read its
-    *current* tags/type/correspondent/storage_path via the ORM (never any
-    possibly-stale names an adapter's source might have cached), weight each
-    distinct taxonomy object by aggregate similarity weight, permission-filter
+    """Resolve each neighbour node's document_id to a live Document, read its
+    *current* tags/type/correspondent/storage_path via the ORM (never the
+    possibly-stale names cached in vector-index node metadata), weight each
+    distinct taxonomy object by aggregate neighbour similarity, permission-filter
    against what ``user`` can see, and return each category ranked by weight
-    and capped. ``similar_documents`` may come from either the vector-RAG
-    adapter or the full-text fallback adapter - both produce this same shape.
+    and capped.
    """
-    if not similar_documents:
-        return empty_taxonomy_candidates()

-    # Both adapters guarantee at most one SimilarDocument per document_id, so
-    # this never silently drops a duplicate's weight.
-    document_weights: dict[int, float] = {
-        s["document_id"]: s["weight"] for s in similar_documents
-    }
+    document_weights = _node_document_weights(nodes)
+    if not document_weights:
+        return empty_taxonomy_candidates()

    # Only .tags.all() needs prefetching (a reverse M2M, one extra query for
    # the whole batch). document_type/correspondent/storage_path are read
@@ -1,5 +1,4 @@
 import datetime
-from collections.abc import Generator
 from types import SimpleNamespace
 from unittest.mock import MagicMock
 from unittest.mock import patch
@@ -7,24 +6,18 @@ from unittest.mock import patch
 import pytest
 import pytest_mock
 from django.test import override_settings
-from guardian.shortcuts import assign_perm
-from guardian.shortcuts import remove_perm

 from documents.models import Document
-from documents.search import TantivyBackend
 from documents.tests.factories import DocumentFactory
 from documents.tests.factories import TagFactory
 from documents.tests.factories import UserFactory
 from paperless.config import AIConfig
-from paperless_ai.ai_classifier import TAXONOMY_CANDIDATE_TOP_K
-from paperless_ai.ai_classifier import _fulltext_similar_documents
 from paperless_ai.ai_classifier import build_localization_prompt
 from paperless_ai.ai_classifier import build_prompt_with_rag
 from paperless_ai.ai_classifier import build_prompt_without_rag
 from paperless_ai.ai_classifier import get_ai_document_classification
 from paperless_ai.ai_classifier import get_language_name
 from paperless_ai.ai_classifier import get_taxonomy_context
-from paperless_ai.taxonomy import SimilarDocument
 from paperless_ai.taxonomy import TaxonomyCandidate
 from paperless_ai.taxonomy import TaxonomyCandidates

@@ -227,10 +220,12 @@ def test_use_rag_if_configured(

@pytest.mark.django_db
@patch("paperless_ai.client.AIClient.run_llm_query")
-@patch("paperless_ai.ai_classifier.build_prompt_with_rag")
+@patch("paperless_ai.ai_classifier.build_prompt_without_rag")
+@patch("paperless_ai.ai_classifier.AIConfig")
@override_settings(LLM_BACKEND="ollama", LLM_MODEL="some_model")
-def test_use_rag_prompt_even_without_embedding_backend(
-    mock_build_prompt_with_rag,
+def test_use_without_rag_if_not_configured(
+    mock_ai_config,
+    mock_build_prompt_without_rag,
    mock_run_llm_query,
    mock_document,
 ):
@@ -240,13 +235,13 @@ def test_use_rag_prompt_even_without_embedding_backend(
    WHEN:
        - get_ai_document_classification() is called
    THEN:
-        - The RAG-context prompt builder is still used (fed by the full-text
-          fallback's context/candidates instead of the vector store's)
+        - The non-RAG prompt builder is used
    """
-    mock_build_prompt_with_rag.return_value = "Prompt with RAG"
+    mock_ai_config.return_value.llm_embedding_backend = None
+    mock_build_prompt_without_rag.return_value = "Prompt without RAG"
    mock_run_llm_query.return_value = NESTED_SUGGESTIONS
    get_ai_document_classification(mock_document)
-    mock_build_prompt_with_rag.assert_called_once()
+    mock_build_prompt_without_rag.assert_called_once()


@pytest.mark.django_db
@@ -325,7 +320,6 @@ def test_build_localization_prompt_preserves_unicode_characters():


@pytest.mark.django_db
-@override_settings(LLM_EMBEDDING_BACKEND="huggingface")
 def test_get_taxonomy_context_assembles_rag_text_and_candidates():
    """
    GIVEN:
@@ -360,7 +354,6 @@ def test_get_taxonomy_context_assembles_rag_text_and_candidates():


@pytest.mark.django_db
-@override_settings(LLM_EMBEDDING_BACKEND="huggingface")
 def test_get_taxonomy_context_preserves_similarity_order_and_distinct_documents():
    """
    GIVEN:
@@ -431,7 +424,6 @@ def test_get_taxonomy_context_preserves_similarity_order_and_distinct_documents(


@pytest.mark.django_db
-@override_settings(LLM_EMBEDDING_BACKEND="huggingface")
 def test_get_taxonomy_context_no_similar_docs():
    """
    GIVEN:
@@ -455,67 +447,6 @@ def test_get_taxonomy_context_no_similar_docs():
    }


-@pytest.mark.django_db
-def test_get_taxonomy_context_uses_fulltext_fallback_when_no_embedding_backend(
-    mocker: pytest_mock.MockerFixture,
-) -> None:
-    """
-    GIVEN:
-        - No LLM embedding backend is configured (the default test settings)
-    WHEN:
-        - get_taxonomy_context() is called
-    THEN:
-        - _fulltext_similar_documents() is called with the document, the user
-          and TAXONOMY_CANDIDATE_TOP_K
-        - retrieve_similar_nodes() (the vector path) is never called
-    """
-    document = DocumentFactory.create(content="Some content")
-    mock_fulltext = mocker.patch(
-        "paperless_ai.ai_classifier._fulltext_similar_documents",
-        return_value=[],
-    )
-    mock_retrieve = mocker.patch("paperless_ai.ai_classifier.retrieve_similar_nodes")
-
-    get_taxonomy_context(document, user=None)
-
-    mock_fulltext.assert_called_once_with(
-        document,
-        None,
-        top_k=TAXONOMY_CANDIDATE_TOP_K,
-    )
-    mock_retrieve.assert_not_called()
-
-
-@pytest.mark.django_db
-@override_settings(LLM_EMBEDDING_BACKEND="huggingface")
-def test_get_taxonomy_context_uses_vector_path_when_embedding_backend_configured(
-    mocker: pytest_mock.MockerFixture,
-) -> None:
-    """
-    GIVEN:
-        - An LLM embedding backend is configured
-    WHEN:
-        - get_taxonomy_context() is called
-    THEN:
-        - retrieve_similar_nodes() (the vector path) is called
-        - _fulltext_similar_documents() (the no-embedding-backend fallback)
-          is never called
-    """
-    document = DocumentFactory.create(content="Some content")
-    mock_retrieve = mocker.patch(
-        "paperless_ai.ai_classifier.retrieve_similar_nodes",
-        return_value=[],
-    )
-    mock_fulltext = mocker.patch(
-        "paperless_ai.ai_classifier._fulltext_similar_documents",
-    )
-
-    get_taxonomy_context(document, user=None)
-
-    mock_retrieve.assert_called_once()
-    mock_fulltext.assert_not_called()
-
-
 class TestGetTaxonomyContextVisibility:
    """get_taxonomy_context must not materialize every visible document id
    for a user who can already see the whole library: a superuser (like no
@@ -528,7 +459,6 @@ class TestGetTaxonomyContextVisibility:
    """

    @pytest.mark.django_db
-    @override_settings(LLM_EMBEDDING_BACKEND="huggingface")
    def test_skips_permission_lookup_for_superuser(
        self,
        mocker: pytest_mock.MockerFixture,
@@ -547,18 +477,17 @@ class TestGetTaxonomyContextVisibility:
            "paperless_ai.ai_classifier.retrieve_similar_nodes",
            return_value=[],
        )
-        mock_permitted = mocker.patch(
-            "paperless_ai.ai_classifier.permitted_object_ids",
+        mock_get_objects = mocker.patch(
+            "paperless_ai.ai_classifier.get_objects_for_user_owner_aware",
        )
        user = UserFactory.create(is_superuser=True)

        get_taxonomy_context(document, user)

-        mock_permitted.assert_not_called()
+        mock_get_objects.assert_not_called()
        assert mock_retrieve.call_args.kwargs["document_ids"] is None

    @pytest.mark.django_db
-    @override_settings(LLM_EMBEDDING_BACKEND="huggingface")
    def test_skips_permission_lookup_when_no_user(
        self,
        mocker: pytest_mock.MockerFixture,
@@ -577,17 +506,16 @@ class TestGetTaxonomyContextVisibility:
            "paperless_ai.ai_classifier.retrieve_similar_nodes",
            return_value=[],
        )
-        mock_permitted = mocker.patch(
-            "paperless_ai.ai_classifier.permitted_object_ids",
+        mock_get_objects = mocker.patch(
+            "paperless_ai.ai_classifier.get_objects_for_user_owner_aware",
        )

        get_taxonomy_context(document, None)

-        mock_permitted.assert_not_called()
+        mock_get_objects.assert_not_called()
        assert mock_retrieve.call_args.kwargs["document_ids"] is None

    @pytest.mark.django_db
-    @override_settings(LLM_EMBEDDING_BACKEND="huggingface")
    def test_restricts_to_visible_documents_for_non_superuser(
        self,
        mocker: pytest_mock.MockerFixture,
@@ -598,7 +526,7 @@ class TestGetTaxonomyContextVisibility:
        WHEN:
            - get_taxonomy_context() is called
        THEN:
-            - The user's permitted document ids are looked up and passed to
+            - The user's visible document ids are looked up and passed to
              retrieve_similar_nodes() as a restriction
        """
        document = DocumentFactory.create(content="Some content")
@@ -606,232 +534,21 @@ class TestGetTaxonomyContextVisibility:
            "paperless_ai.ai_classifier.retrieve_similar_nodes",
            return_value=[],
        )
-        mock_permitted = mocker.patch(
-            "paperless_ai.ai_classifier.permitted_object_ids",
-            return_value=[1, 2, 3],
+        mock_queryset = mocker.MagicMock()
+        mock_queryset.values_list.return_value = [1, 2, 3]
+        mock_get_objects = mocker.patch(
+            "paperless_ai.ai_classifier.get_objects_for_user_owner_aware",
+            return_value=mock_queryset,
        )
        user = UserFactory.create(is_superuser=False)

        get_taxonomy_context(document, user)

-        mock_permitted.assert_called_once_with(user, Document, "view_document")
+        mock_get_objects.assert_called_once_with(user, "view_document", Document)
        assert mock_retrieve.call_args.kwargs["document_ids"] == [1, 2, 3]


@pytest.mark.django_db
-class TestFulltextSimilarDocuments:
-    """_fulltext_similar_documents is the no-embedding-backend fallback: it
-    asks the Tantivy full-text index for "More Like This" neighbours instead
-    of the vector store, and synthesizes a rank-based weight since Tantivy's
-    more_like_this_ids returns only an ordered id list, no scores.
-    """
-
-    @pytest.fixture
-    def fulltext_backend(
-        self,
-        mocker: pytest_mock.MockerFixture,
-    ) -> Generator[TantivyBackend, None, None]:
-        """An in-memory Tantivy backend, wired up as the module-level
-        singleton _fulltext_similar_documents resolves via get_backend()."""
-        backend = TantivyBackend(path=None)
-        backend.open()
-        mocker.patch("documents.search.get_backend", return_value=backend)
-        try:
-            yield backend
-        finally:
-            backend.close()
-
-    def test_ranks_by_rank_based_weight_descending(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A source document and two similar documents indexed in Tantivy
-        WHEN:
-            - _fulltext_similar_documents() is called
-        THEN:
-            - Each result's weight reflects its rank (first result weighted
-              higher than the second), not a raw similarity score
-        """
-        source = DocumentFactory.create(content="quarterly financial report details")
-        first = DocumentFactory.create(content="quarterly financial report details")
-        second = DocumentFactory.create(content="financial report")
-        for doc in (source, first, second):
-            fulltext_backend.add_or_update(doc)
-
-        result = _fulltext_similar_documents(source, user=None, top_k=5)
-
-        assert len(result) == 2
-        weight_by_id = {s["document_id"]: s["weight"] for s in result}
-        assert weight_by_id[first.pk] > weight_by_id[second.pk]
-
-    def test_excludes_source_document(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A source document indexed in Tantivy with no other documents
-        WHEN:
-            - _fulltext_similar_documents() is called
-        THEN:
-            - An empty list is returned - the source document is never its
-              own similar document
-        """
-        source = DocumentFactory.create(content="unique unrelated content")
-        fulltext_backend.add_or_update(source)
-
-        result = _fulltext_similar_documents(source, user=None, top_k=5)
-
-        assert result == []
-
-    def test_empty_index_returns_empty_list(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A document that has never been indexed (fresh/empty Tantivy index)
-        WHEN:
-            - _fulltext_similar_documents() is called
-        THEN:
-            - An empty list is returned rather than raising
-        """
-        source = DocumentFactory.create(content="never indexed")
-
-        result = _fulltext_similar_documents(source, user=None, top_k=5)
-
-        assert result == []
-
-    def test_respects_top_k_limit(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A source document and four similar documents indexed
-        WHEN:
-            - _fulltext_similar_documents() is called with top_k=2
-        THEN:
-            - At most 2 results are returned
-        """
-        source = DocumentFactory.create(content="shared overlapping keyword text")
-        fulltext_backend.add_or_update(source)
-        for _ in range(4):
-            fulltext_backend.add_or_update(
-                DocumentFactory.create(content="shared overlapping keyword text"),
-            )
-
-        result = _fulltext_similar_documents(source, user=None, top_k=2)
-
-        assert len(result) == 2
-
-    def test_result_shape_is_similar_document(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A source document and one similar document indexed
-        WHEN:
-            - _fulltext_similar_documents() is called
-        THEN:
-            - Each result is a SimilarDocument (document_id + weight only)
-        """
-        source = DocumentFactory.create(content="shared content phrase")
-        other = DocumentFactory.create(content="shared content phrase")
-        fulltext_backend.add_or_update(source)
-        fulltext_backend.add_or_update(other)
-
-        result = _fulltext_similar_documents(source, user=None, top_k=5)
-
-        # rank 0 (the only/best result) with top_k=5 -> weight = top_k - rank = 5.0,
-        # per the "first result gets top_k, the last gets 1" formula.
-        assert result == [SimilarDocument(document_id=other.pk, weight=5.0)]
-
-    def test_superuser_sees_other_users_documents(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A source document owned by one user and a similar document
-              owned by a different user, with no sharing between them
-        WHEN:
-            - _fulltext_similar_documents() is called with a superuser
-        THEN:
-            - The other user's document is still returned as a similar
-              document - a superuser must not be narrowed by the backend's
-              owner-based permission filter
-        """
-        owner = UserFactory.create()
-        other_owner = UserFactory.create()
-        superuser = UserFactory.create(is_superuser=True)
-        source = DocumentFactory.create(
-            content="shared content phrase",
-            owner=owner,
-        )
-        other = DocumentFactory.create(
-            content="shared content phrase",
-            owner=other_owner,
-        )
-        fulltext_backend.add_or_update(source)
-        fulltext_backend.add_or_update(other)
-
-        result = _fulltext_similar_documents(source, user=superuser, top_k=5)
-
-        assert [s["document_id"] for s in result] == [other.pk]
-
-    def test_excludes_stale_permitted_document_for_regular_user(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A regular (non-superuser) user
-            - A similar document the user is permitted to view, and another
-              similar document indexed while the user still had view
-              permission but which has since had that permission revoked in
-              the database, i.e. the Tantivy index has stale permission data
-        WHEN:
-            - _fulltext_similar_documents() is called with that user
-        THEN:
-            - Only the still-permitted document is returned - the DB
-              re-check via restrict_queryset_to_visible() must catch the
-              document Tantivy's stale index still thinks is visible
-        """
-        owner = UserFactory.create()
-        viewer = UserFactory.create(is_superuser=False)
-        source = DocumentFactory.create(
-            content="shared content phrase",
-            owner=owner,
-        )
-        permitted = DocumentFactory.create(
-            content="shared content phrase",
-            owner=owner,
-        )
-        now_private = DocumentFactory.create(
-            content="shared content phrase",
-            owner=owner,
-        )
-        assign_perm("view_document", viewer, permitted)
-        assign_perm("view_document", viewer, now_private)
-        fulltext_backend.add_or_update(source)
-        fulltext_backend.add_or_update(permitted)
-        fulltext_backend.add_or_update(now_private)
-
-        # Revoke access after indexing, without reindexing: the index still
-        # carries viewer as a permitted viewer for `now_private`.
-        remove_perm("view_document", viewer, now_private)
-
-        result = _fulltext_similar_documents(source, user=viewer, top_k=5)
-
-        assert [s["document_id"] for s in result] == [permitted.pk]
-
-
-@pytest.mark.django_db
-@override_settings(LLM_EMBEDDING_BACKEND="huggingface")
@patch("paperless_ai.ai_classifier.retrieve_similar_nodes")
 def test_get_taxonomy_context_retrieval_failure_degrades_to_no_hints(mock_retrieve):
    """
@@ -858,7 +575,6 @@ def test_get_taxonomy_context_retrieval_failure_degrades_to_no_hints(mock_retrie


@pytest.mark.django_db
-@override_settings(LLM_EMBEDDING_BACKEND="huggingface")
@patch("paperless_ai.ai_classifier.build_taxonomy_candidates")
@patch("paperless_ai.ai_classifier.retrieve_similar_nodes")
 def test_get_taxonomy_context_candidate_building_failure_degrades_to_no_hints(
@@ -1188,7 +1188,9 @@ class TestRetrieveSimilarNodesAgainstRealIndex:

        nodes = indexing.retrieve_similar_nodes(a, document_ids=[b.id])

-        assert all(int(node.metadata["document_id"]) == b.id for node in nodes)
+        assert all(
+            document_id == b.id for document_id in indexing._node_document_ids(nodes)
+        )

    def test_excludes_self(
        self,
@@ -1210,7 +1212,7 @@ class TestRetrieveSimilarNodesAgainstRealIndex:

        nodes = indexing.retrieve_similar_nodes(a, top_k=5)

-        assert {int(node.metadata["document_id"]) for node in nodes} == {b.id}
+        assert set(indexing._node_document_ids(nodes)) == {b.id}

    def test_excludes_self_with_multiple_chunks(
        self,
@@ -1233,4 +1235,4 @@ class TestRetrieveSimilarNodesAgainstRealIndex:

        nodes = indexing.retrieve_similar_nodes(a, top_k=3)

-        assert {int(node.metadata["document_id"]) for node in nodes} == {b.id}
+        assert set(indexing._node_document_ids(nodes)) == {b.id}
@@ -4,7 +4,6 @@ from unittest.mock import MagicMock
 from unittest.mock import patch

 import httpx
-import ollama
 import openai
 import pytest
 from llama_index.core.llms.llm import ToolSelection
@@ -12,7 +11,6 @@ from llama_index.core.llms.llm import ToolSelection
 from paperless_ai.client import LLM_SYSTEM_PROMPT
 from paperless_ai.client import PLACEHOLDER_API_KEY
 from paperless_ai.client import AIClient
-from paperless_ai.exceptions import LLMProviderError
 from paperless_ai.exceptions import LLMTimeoutError


@@ -216,52 +214,6 @@ def test_run_llm_query_openai_timeout_raises_local_error(
        client.run_llm_query("test_prompt")


-def test_run_llm_query_openai_status_error_raises_provider_error(
-    mock_ai_config,
-    mock_openai_llm,
-):
-    mock_ai_config.llm_backend = "openai-like"
-    mock_ai_config.llm_model = "test_model"
-    mock_ai_config.llm_endpoint = "http://test-url"
-
-    request = httpx.Request("POST", "http://test-url/v1/chat/completions")
-    body = {"error": {"message": "Thinking mode does not support this tool_choice"}}
-    mock_openai_llm.return_value.chat_with_tools.side_effect = openai.BadRequestError(
-        "Error code: 400",
-        response=httpx.Response(400, request=request, json=body),
-        body=body,
-    )
-
-    client = AIClient()
-
-    with pytest.raises(LLMProviderError) as exc_info:
-        client.run_llm_query("test_prompt")
-    assert str(exc_info.value) == ""
-    assert isinstance(exc_info.value.__cause__, openai.BadRequestError)
-
-
-def test_run_llm_query_ollama_response_error_raises_provider_error(
-    mock_ai_config,
-    mock_ollama_llm,
-):
-    mock_ai_config.llm_backend = "ollama"
-    mock_ai_config.llm_model = "test_model"
-    mock_ai_config.llm_endpoint = "http://test-url"
-
-    response_error = ollama.ResponseError(
-        "confidential provider response",
-        status_code=400,
-    )
-    mock_ollama_llm.return_value.chat.side_effect = response_error
-
-    client = AIClient()
-
-    with pytest.raises(LLMProviderError) as exc_info:
-        client.run_llm_query("test_prompt")
-    assert str(exc_info.value) == ""
-    assert exc_info.value.__cause__ is response_error
-
-
 def test_run_llm_query_httpx_timeout_raises_local_error(
    mock_ai_config,
    mock_ollama_llm,
@@ -1,4 +1,5 @@
 import json
+from types import SimpleNamespace

 import pytest
 import pytest_mock
@@ -9,14 +10,14 @@ from documents.tests.factories import DocumentTypeFactory
 from documents.tests.factories import StoragePathFactory
 from documents.tests.factories import TagFactory
 from documents.tests.factories import UserFactory
-from paperless_ai.taxonomy import SimilarDocument
 from paperless_ai.taxonomy import TaxonomyCandidates
 from paperless_ai.taxonomy import build_taxonomy_candidates
 from paperless_ai.taxonomy import format_taxonomy_for_prompt


-def make_similar(document_id: int, weight: float) -> SimilarDocument:
-    return SimilarDocument(document_id=document_id, weight=weight)
+def make_node(document_id: int, score: float) -> SimpleNamespace:
+    """A stand-in for NodeWithScore: only ``.metadata``/``.score`` are read."""
+    return SimpleNamespace(metadata={"document_id": str(document_id)}, score=score)


@pytest.mark.django_db
@@ -52,9 +53,9 @@ class TestBuildTaxonomyCandidates:
        doc_a.tags.add(tag)
        doc_b = DocumentFactory.create()
        doc_b.tags.add(tag)
-        similar_documents = [make_similar(doc_a.pk, 0.9), make_similar(doc_b.pk, 0.4)]
+        nodes = [make_node(doc_a.pk, 0.9), make_node(doc_b.pk, 0.4)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["tags"]) == 1
        assert result["tags"][0]["id"] == tag.pk
@@ -79,9 +80,9 @@ class TestBuildTaxonomyCandidates:
        document.tags.add(tag)
        tag.name = "New Name"
        tag.save()
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert result["tags"][0]["name"] == "New Name"

@@ -101,9 +102,9 @@ class TestBuildTaxonomyCandidates:
        document = DocumentFactory.create()
        document.tags.add(tag)
        tag.delete()
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert result["tags"] == []

@@ -122,12 +123,9 @@ class TestBuildTaxonomyCandidates:
        strong_doc.tags.add(strong_tag)
        weak_doc = DocumentFactory.create()
        weak_doc.tags.add(weak_tag)
-        similar_documents = [
-            make_similar(strong_doc.pk, 0.9),
-            make_similar(weak_doc.pk, 0.1),
-        ]
+        nodes = [make_node(strong_doc.pk, 0.9), make_node(weak_doc.pk, 0.1)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert [c["name"] for c in result["tags"]] == ["Strong", "Weak"]

@@ -143,9 +141,9 @@ class TestBuildTaxonomyCandidates:
        document = DocumentFactory.create()
        for i in range(15):
            document.tags.add(TagFactory.create(name=f"Tag{i}"))
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["tags"]) == 10

@@ -159,12 +157,12 @@ class TestBuildTaxonomyCandidates:
            - Only 5 correspondents are returned
        """
        correspondents = CorrespondentFactory.create_batch(7)
-        similar_documents = [
-            make_similar(DocumentFactory.create(correspondent=c).pk, 0.5)
+        nodes = [
+            make_node(DocumentFactory.create(correspondent=c).pk, 0.5)
            for c in correspondents
        ]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["correspondents"]) == 5

@@ -179,9 +177,9 @@ class TestBuildTaxonomyCandidates:
        """
        document_type = DocumentTypeFactory.create(name="Invoice")
        document = DocumentFactory.create(document_type=document_type)
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["document_types"]) == 1
        assert result["document_types"][0]["id"] == document_type.pk
@@ -197,12 +195,12 @@ class TestBuildTaxonomyCandidates:
            - Only 5 document_types are returned
        """
        document_types = DocumentTypeFactory.create_batch(7)
-        similar_documents = [
-            make_similar(DocumentFactory.create(document_type=dt).pk, 0.5)
+        nodes = [
+            make_node(DocumentFactory.create(document_type=dt).pk, 0.5)
            for dt in document_types
        ]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["document_types"]) == 5

@@ -217,9 +215,9 @@ class TestBuildTaxonomyCandidates:
        """
        storage_path = StoragePathFactory.create(name="Invoices")
        document = DocumentFactory.create(storage_path=storage_path)
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["storage_paths"]) == 1
        assert result["storage_paths"][0]["id"] == storage_path.pk
@@ -235,12 +233,12 @@ class TestBuildTaxonomyCandidates:
            - Only 5 storage_paths are returned
        """
        storage_paths = StoragePathFactory.create_batch(7)
-        similar_documents = [
-            make_similar(DocumentFactory.create(storage_path=sp).pk, 0.5)
+        nodes = [
+            make_node(DocumentFactory.create(storage_path=sp).pk, 0.5)
            for sp in storage_paths
        ]

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert len(result["storage_paths"]) == 5

@@ -260,14 +258,14 @@ class TestBuildTaxonomyCandidates:
        tag = TagFactory.create(name="Restricted")
        document = DocumentFactory.create()
        document.tags.add(tag)
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]
        user = UserFactory.create()
        mocker.patch(
            "documents.permissions.permitted_object_ids",
            return_value=[],  # user cannot see this tag
        )

-        result = build_taxonomy_candidates(similar_documents, user=user)
+        result = build_taxonomy_candidates(nodes, user=user)

        assert result["tags"] == []

@@ -297,10 +295,10 @@ class TestBuildTaxonomyCandidates:
        tag.save()
        document = DocumentFactory.create()
        document.tags.add(tag)
-        similar_documents = [make_similar(document.pk, 0.5)]
+        nodes = [make_node(document.pk, 0.5)]
        spy = mocker.patch("documents.permissions.permitted_object_ids")

-        result = build_taxonomy_candidates(similar_documents, user=None)
+        result = build_taxonomy_candidates(nodes, user=None)

        assert result["tags"][0]["name"] == "Owned"
        spy.assert_not_called()
@@ -0,0 +1,8 @@
+from django.apps import AppConfig
+from django.utils.translation import gettext_lazy as _
+
+
+class PaperlessBenchmarkConfig(AppConfig):
+    name = "paperless_benchmark"
+
+    verbose_name = _("Paperless benchmark")
@@ -0,0 +1,136 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from django.db import connection
+
+if TYPE_CHECKING:
+    from django.db.models import QuerySet
+
+
+def _reset_table_names() -> list[str]:
+    from guardian.models import GroupObjectPermission
+    from guardian.models import UserObjectPermission
+
+    from documents.models import Correspondent
+    from documents.models import Document
+    from documents.models import DocumentType
+    from documents.models import StoragePath
+    from documents.models import Tag
+
+    return [
+        Document.tags.through._meta.db_table,
+        Document._meta.db_table,
+        Tag._meta.db_table,
+        Correspondent._meta.db_table,
+        DocumentType._meta.db_table,
+        StoragePath._meta.db_table,
+        UserObjectPermission._meta.db_table,
+        GroupObjectPermission._meta.db_table,
+    ]
+
+
+def _delete_all_users_and_groups() -> None:
+    # ASSUMPTION: this tool assumes a disposable benchmark database, never
+    # point it at a real install. This deletes EVERY user and group in the
+    # database (not just benchmark-created ones) -- there is no way to
+    # distinguish "real" users from seeded ones, so this is only safe against
+    # a database that exists solely to run this benchmarking tool. The
+    # `benchmark seed --reset` CLI path requires an explicit
+    # `--yes-i-know-this-wipes-the-database` flag before reaching here; do
+    # not remove that guard.
+    from django.contrib.auth import get_user_model
+    from django.contrib.auth.models import Group
+
+    get_user_model().objects.all().delete()
+    Group.objects.all().delete()
+
+
+def _reset_postgresql() -> None:
+    tables = _reset_table_names()
+    with connection.cursor() as cursor:
+        cursor.execute(f"TRUNCATE TABLE {', '.join(tables)} RESTART IDENTITY CASCADE;")
+    _delete_all_users_and_groups()
+
+
+def _reset_mariadb() -> None:
+    # MariaDB's TRUNCATE has no CASCADE clause and refuses to truncate a
+    # table referenced by a foreign key while checks are enabled, so
+    # checks are disabled for the duration of the reset.
+    tables = _reset_table_names()
+    with connection.cursor() as cursor:
+        cursor.execute("SET FOREIGN_KEY_CHECKS = 0;")
+        try:
+            for table in tables:
+                cursor.execute(f"TRUNCATE TABLE {table};")
+        finally:
+            cursor.execute("SET FOREIGN_KEY_CHECKS = 1;")
+    _delete_all_users_and_groups()
+
+
+def _reset_sqlite() -> None:
+    from documents.models import Correspondent
+    from documents.models import Document
+    from documents.models import DocumentType
+    from documents.models import StoragePath
+    from documents.models import Tag
+
+    Document.global_objects.all().delete()
+    Tag.objects.all().delete()
+    Correspondent.objects.all().delete()
+    DocumentType.objects.all().delete()
+    StoragePath.objects.all().delete()
+    _delete_all_users_and_groups()
+
+
+def reset_benchmark_data() -> None:
+    """
+    Remove all previously-seeded benchmark data (documents, tags,
+    correspondents, document types, storage paths, guardian permission
+    rows, users, and groups) so a fresh `benchmark seed` run starts
+    from an empty slate. Dispatches per-backend because TRUNCATE syntax
+    and cascade behavior differ across the 3 supported databases.
+    """
+    if connection.vendor == "postgresql":
+        _reset_postgresql()
+    elif connection.vendor == "mysql":
+        # MariaDB also reports vendor == "mysql" under Django's mysql backend.
+        _reset_mariadb()
+    else:
+        _reset_sqlite()
+
+
+def capture_explain(queryset: QuerySet) -> str:
+    """
+    Return the query plan for `queryset` using the current backend's
+    explain facility. PostgreSQL supports `EXPLAIN ANALYZE {sql}` (real
+    execution stats). MariaDB does NOT accept that syntax -- verified
+    against a real MariaDB 12.3 container: `EXPLAIN ANALYZE {sql}` raises a
+    1064 syntax error, while MariaDB's own `ANALYZE {sql}` form (no
+    `EXPLAIN` keyword) works and returns real per-row execution stats
+    (`r_rows`, `r_filtered`, etc. columns) -- this is MariaDB's
+    EXPLAIN-ANALYZE-equivalent, distinct from MySQL 8.0.18+'s
+    `EXPLAIN ANALYZE` syntax, which MariaDB does not implement. SQLite only
+    supports EXPLAIN QUERY PLAN (the chosen plan, not real timing/row
+    counts) -- that output is clearly labeled rather than silently looking
+    equivalent to the other two backends' output.
+    """
+    sql, params = queryset.query.sql_with_params()
+    with connection.cursor() as cursor:
+        if connection.vendor == "postgresql":
+            cursor.execute(f"EXPLAIN ANALYZE {sql}", params)
+            return "\n".join(str(row[0]) for row in cursor.fetchall())
+        if connection.vendor == "mysql":
+            # MariaDB also reports vendor == "mysql" under Django's mysql
+            # backend. Unlike MySQL 8.0.18+, MariaDB has no `EXPLAIN
+            # ANALYZE` syntax -- its equivalent is `ANALYZE <statement>`.
+            cursor.execute(f"ANALYZE {sql}", params)
+            columns = [c[0] for c in cursor.description]
+            header = " | ".join(columns)
+            rows = "\n".join(
+                " | ".join(str(c) for c in row) for row in cursor.fetchall()
+            )
+            return f"{header}\n{rows}"
+        cursor.execute(f"EXPLAIN QUERY PLAN {sql}", params)
+        rows = "\n".join(" | ".join(str(c) for c in row) for row in cursor.fetchall())
+        return f"(plan only -- no execution stats on SQLite)\n{rows}"
@@ -0,0 +1,81 @@
+# src/paperless_benchmark/endpoints.py
+from __future__ import annotations
+
+import statistics
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from django.contrib.auth.models import User
+    from rest_framework.test import APIClient
+
+ENDPOINTS: tuple[tuple[str, str], ...] = (
+    ("documents_default", "/api/documents/"),
+    ("documents_page50", "/api/documents/?page_size=50"),
+    ("tags_all", "/api/tags/?page_size=100000"),
+)
+
+
+@dataclass(frozen=True, slots=True)
+class EndpointTiming:
+    user_label: str
+    endpoint_name: str
+    query_count: int
+    min_ms: float
+    median_ms: float
+    max_ms: float
+
+
+def _timed_requests(client: APIClient, url: str, n: int) -> list[float]:
+    times = []
+    for _ in range(n):
+        t0 = time.perf_counter()
+        resp = client.get(url)
+        t1 = time.perf_counter()
+        if resp.status_code != 200:
+            raise RuntimeError(
+                f"GET {url} -> {resp.status_code}: {resp.content[:300]!r}",
+            )
+        times.append(t1 - t0)
+    return times
+
+
+def _query_count(client: APIClient, url: str) -> int:
+    from django.db import connection
+    from django.test.utils import CaptureQueriesContext
+
+    with CaptureQueriesContext(connection) as ctx:
+        resp = client.get(url)
+    if resp.status_code != 200:
+        raise RuntimeError(f"GET {url} -> {resp.status_code}: {resp.content[:300]!r}")
+    return len(ctx.captured_queries)
+
+
+def run_endpoint_benchmarks(
+    *,
+    perf_target: User,
+    perf_admin: User,
+    repeat: int,
+) -> list[EndpointTiming]:
+    from rest_framework.test import APIClient
+
+    results: list[EndpointTiming] = []
+    for user_label, user in (("target", perf_target), ("admin", perf_admin)):
+        client = APIClient()
+        client.force_authenticate(user=user)
+        for name, url in ENDPOINTS:
+            client.get(url)  # warm-up request, not counted
+            qcount = _query_count(client, url)
+            times_ms = [t * 1000 for t in _timed_requests(client, url, repeat)]
+            results.append(
+                EndpointTiming(
+                    user_label=user_label,
+                    endpoint_name=name,
+                    query_count=qcount,
+                    min_ms=min(times_ms),
+                    median_ms=statistics.median(times_ms),
+                    max_ms=max(times_ms),
+                ),
+            )
+    return results
@@ -0,0 +1,56 @@
+# src/paperless_benchmark/harness.py
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+from typing import Generic
+from typing import TypeVar
+
+from django.db import connection
+from django.test.utils import CaptureQueriesContext
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+T = TypeVar("T")
+
+
+@dataclass(frozen=True, slots=True)
+class ProfileResult(Generic[T]):
+    best_seconds: float
+    all_seconds: tuple[float, ...]
+    query_count: int
+    result: T
+
+
+def run_profile(fn: Callable[[], T], *, repeat: int = 5) -> ProfileResult[T]:
+    """
+    Call `fn` `repeat` times, capturing wall-clock time for every call and
+    the SQL query count for the final call. Returns the best (minimum)
+    time across all repeats, since the first call(s) can be skewed by
+    connection warm-up or cold caches.
+    """
+    if repeat < 1:
+        raise ValueError("repeat must be >= 1")
+
+    all_seconds: list[float] = []
+    result: T | None = None
+    query_count = 0
+    for i in range(repeat):
+        with CaptureQueriesContext(connection) as ctx:
+            start = time.perf_counter()
+            result = fn()
+            all_seconds.append(time.perf_counter() - start)
+        if i == repeat - 1:
+            query_count = len(ctx.captured_queries)
+    # Purely a type-narrowing aid for the type checker: the `repeat < 1`
+    # guard above already turns the one case that could leave `result`
+    # unset into a clear ValueError, so this is unreachable in practice.
+    assert result is not None
+    return ProfileResult(
+        best_seconds=min(all_seconds),
+        all_seconds=tuple(all_seconds),
+        query_count=query_count,
+        result=result,
+    )
@@ -0,0 +1,222 @@
+# src/paperless_benchmark/management/commands/benchmark.py
+from __future__ import annotations
+
+from typing import Any
+
+from django.core.management.base import BaseCommand
+from django.core.management.base import CommandError
+from django.core.management.base import CommandParser
+
+
+class Command(BaseCommand):
+    help = "Seed, run, and profile paperless-ngx performance benchmarks."
+
+    def add_arguments(self, parser: CommandParser) -> None:
+        parser.add_argument(
+            "action",
+            choices=["seed", "run", "profile", "list-scenarios"],
+            help="Which benchmark action to perform.",
+        )
+        parser.add_argument(
+            "scenario",
+            nargs="?",
+            default=None,
+            help="Scenario name (required for `profile`; see `list-scenarios`).",
+        )
+        parser.add_argument(
+            "--tier",
+            choices=["home", "medium", "large"],
+            default="medium",
+            help="Dataset scale tier for `seed` (default: medium).",
+        )
+        parser.add_argument(
+            "--reset",
+            action="store_true",
+            default=False,
+            help="For `seed`: truncate existing benchmark data first.",
+        )
+        parser.add_argument(
+            "--yes-i-know-this-wipes-the-database",
+            action="store_true",
+            default=False,
+            help=(
+                "Required alongside --reset: confirms you understand `seed "
+                "--reset` deletes ALL users, ALL groups, and ALL documents/"
+                "tags/correspondents/document types/storage paths in this "
+                "database, not just benchmark-created ones."
+            ),
+        )
+        parser.add_argument(
+            "--seed",
+            type=int,
+            default=42,
+            help="RNG seed for reproducible datasets (default: 42).",
+        )
+        parser.add_argument(
+            "--repeat",
+            type=int,
+            default=5,
+            help="Number of timed repetitions for `run`/`profile` (default: 5).",
+        )
+        parser.add_argument(
+            "--label",
+            default="baseline",
+            help="Free-text tag for a `run`, printed and recorded in history only.",
+        )
+        parser.add_argument(
+            "--explain",
+            action="store_true",
+            default=False,
+            help="For `profile`: also capture and print the query plan.",
+        )
+
+    def handle(self, *args: Any, **options: Any) -> None:
+        action = options["action"]
+        if action == "seed":
+            self._handle_seed(options)
+        elif action == "run":
+            self._handle_run(options)
+        elif action == "profile":
+            self._handle_profile(options)
+        else:
+            self._handle_list_scenarios()
+
+    def _handle_seed(self, options: dict[str, Any]) -> None:
+        from paperless_benchmark.db import reset_benchmark_data
+        from paperless_benchmark.seeding import seed_benchmark_dataset
+
+        if options["reset"]:
+            if not options["yes_i_know_this_wipes_the_database"]:
+                raise CommandError(
+                    "--reset requires --yes-i-know-this-wipes-the-database. "
+                    "This deletes ALL users, ALL groups, and ALL documents, "
+                    "tags, correspondents, document types, and storage paths "
+                    "in this database -- not just benchmark-created ones. "
+                    "Only run this against a disposable benchmark database, "
+                    "never a real install. Re-run with "
+                    "--reset --yes-i-know-this-wipes-the-database to proceed.",
+                )
+            self.stdout.write("Resetting existing benchmark data...")
+            reset_benchmark_data()
+
+        data = seed_benchmark_dataset(options["tier"], seed=options["seed"])
+        self.stdout.write(
+            self.style.SUCCESS(
+                f"Seeded tier={options['tier']!r}: {data.documents} documents, "
+                f"{len(data.users)} users, {len(data.groups)} groups.",
+            ),
+        )
+
+    def _handle_run(self, options: dict[str, Any]) -> None:
+        from django.contrib.auth import get_user_model
+        from django.db import connection
+
+        from documents.models import Document
+        from paperless_benchmark.endpoints import run_endpoint_benchmarks
+        from paperless_benchmark.results import append_history
+
+        user_model = get_user_model()
+        try:
+            perf_target = user_model.objects.get(username="perf_target")
+            perf_admin = user_model.objects.get(username="perf_admin")
+        except user_model.DoesNotExist as e:
+            raise CommandError(
+                "No benchmark dataset found. Run `manage.py benchmark seed` first.",
+            ) from e
+
+        db_vendor = connection.vendor
+        document_count = Document.objects.count()
+
+        results = run_endpoint_benchmarks(
+            perf_target=perf_target,
+            perf_admin=perf_admin,
+            repeat=options["repeat"],
+        )
+
+        self.stdout.write(f"# label={options['label']} repeat={options['repeat']}")
+        self.stdout.write(
+            f"{'user':7s} {'endpoint':20s} {'queries':>8s} "
+            f"{'min_ms':>9s} {'median_ms':>10s} {'max_ms':>9s}",
+        )
+        for r in results:
+            self.stdout.write(
+                f"{r.user_label:7s} {r.endpoint_name:20s} {r.query_count:8d} "
+                f"{r.min_ms:9.1f} {r.median_ms:10.1f} {r.max_ms:9.1f}",
+            )
+            append_history(
+                {
+                    "mode": "run",
+                    "label": options["label"],
+                    "user": r.user_label,
+                    "endpoint": r.endpoint_name,
+                    "query_count": r.query_count,
+                    "min_ms": r.min_ms,
+                    "median_ms": r.median_ms,
+                    "max_ms": r.max_ms,
+                    "db_vendor": db_vendor,
+                    "document_count": document_count,
+                },
+            )
+
+    def _handle_profile(self, options: dict[str, Any]) -> None:
+        from django.contrib.auth import get_user_model
+        from django.db import connection
+
+        from documents.models import Document
+        from paperless_benchmark.db import capture_explain
+        from paperless_benchmark.harness import run_profile
+        from paperless_benchmark.results import append_history
+        from paperless_benchmark.scenarios import get as get_scenario
+
+        if not options["scenario"]:
+            raise CommandError(
+                "`profile` requires a scenario name; see `list-scenarios`.",
+            )
+
+        scenario = get_scenario(options["scenario"])
+
+        user_model = get_user_model()
+        try:
+            perf_target = user_model.objects.get(username="perf_target")
+        except user_model.DoesNotExist as e:
+            raise CommandError(
+                "No benchmark dataset found. Run `manage.py benchmark seed` first.",
+            ) from e
+
+        profile = run_profile(
+            lambda: scenario.run(perf_target),
+            repeat=options["repeat"],
+        )
+        self.stdout.write(
+            f"{scenario.name}: best={profile.best_seconds:.4f}s "
+            f"queries={profile.query_count}",
+        )
+
+        if options["explain"]:
+            if scenario.queryset_for_explain is not None:
+                plan = capture_explain(scenario.queryset_for_explain(perf_target))
+                self.stdout.write(plan)
+            else:
+                self.stdout.write(
+                    self.style.WARNING(
+                        f"--explain was requested but scenario {scenario.name!r} "
+                        "does not support it (no queryset_for_explain); skipping.",
+                    ),
+                )
+
+        append_history(
+            {
+                "mode": "profile",
+                "scenario": scenario.name,
+                "best_seconds": profile.best_seconds,
+                "query_count": profile.query_count,
+                "db_vendor": connection.vendor,
+                "document_count": Document.objects.count(),
+            },
+        )
+
+    def _handle_list_scenarios(self) -> None:
+        from paperless_benchmark.scenarios import all_scenarios
+
+        for scenario in all_scenarios():
+            self.stdout.write(f"{scenario.name}: {scenario.describe}")
@@ -0,0 +1,39 @@
+# src/paperless_benchmark/results.py
+from __future__ import annotations
+
+import json
+import subprocess
+from datetime import UTC
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+RESULTS_DIR = Path(__file__).resolve().parent.parent.parent / "benchmark_results"
+
+
+def _current_git_ref() -> str:
+    result = subprocess.run(
+        ["git", "rev-parse", "--short", "HEAD"],
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    return result.stdout.strip() or "unknown"
+
+
+def append_history(entry: dict[str, Any], *, code_ref: str | None = None) -> None:
+    """
+    Append one line to benchmark_results/history.jsonl -- a local-only,
+    append-only, cross-session record of every `benchmark run`/`profile`
+    invocation. Unlike a single overwritten snapshot file, this survives
+    across sessions so a benchmarking effort picked back up days later has
+    a full timeline instead of only the most recent result.
+    """
+    RESULTS_DIR.mkdir(exist_ok=True)
+    record = {
+        "timestamp": datetime.now(UTC).isoformat(),
+        "code_ref": code_ref or _current_git_ref(),
+        **entry,
+    }
+    with (RESULTS_DIR / "history.jsonl").open("a") as f:
+        f.write(json.dumps(record) + "\n")
@@ -0,0 +1,110 @@
+# src/paperless_benchmark/scenarios.py
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+from typing import Any
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    from django.contrib.auth.models import User
+    from django.db.models import QuerySet
+
+
+@dataclass(frozen=True, slots=True)
+class Scenario:
+    name: str
+    describe: str
+    run: Callable[[User], Any]
+    queryset_for_explain: Callable[[User], QuerySet] | None = None
+
+
+_SCENARIOS: dict[str, Scenario] = {}
+
+
+def register(scenario: Scenario) -> None:
+    _SCENARIOS[scenario.name] = scenario
+
+
+def get(name: str) -> Scenario:
+    from django.core.management.base import CommandError
+
+    try:
+        return _SCENARIOS[name]
+    except KeyError:
+        available = ", ".join(sorted(_SCENARIOS)) or "(none registered)"
+        raise CommandError(
+            f"Unknown benchmark scenario {name!r}. Available: {available}",
+        ) from None
+
+
+def all_scenarios() -> tuple[Scenario, ...]:
+    return tuple(_SCENARIOS.values())
+
+
+def _guardian_visibility_query_run(user: User) -> list[int]:
+    from documents.models import Document
+    from documents.permissions import get_objects_for_user_owner_aware
+
+    return list(
+        get_objects_for_user_owner_aware(
+            user,
+            "documents.view_document",
+            Document,
+        ).values_list("id", flat=True),
+    )
+
+
+def _guardian_visibility_query_queryset(user: User) -> QuerySet:
+    from documents.models import Document
+    from documents.permissions import get_objects_for_user_owner_aware
+
+    return get_objects_for_user_owner_aware(user, "documents.view_document", Document)
+
+
+register(
+    Scenario(
+        name="guardian_visibility_query",
+        describe=(
+            "Document-visibility queryset for a user with mixed owned/shared "
+            "documents -- exercises documents.permissions."
+            "get_objects_for_user_owner_aware's guardian permission join."
+        ),
+        run=_guardian_visibility_query_run,
+        queryset_for_explain=_guardian_visibility_query_queryset,
+    ),
+)
+
+
+def _permitted_document_ids_run(user: User) -> list[int]:
+    from documents.models import Document
+    from documents.permissions import permitted_document_ids
+
+    return list(
+        Document.objects.filter(id__in=permitted_document_ids(user)).values_list(
+            "id",
+            flat=True,
+        ),
+    )
+
+
+def _permitted_document_ids_queryset(user: User) -> QuerySet:
+    from documents.models import Document
+    from documents.permissions import permitted_document_ids
+
+    return Document.objects.filter(id__in=permitted_document_ids(user))
+
+
+register(
+    Scenario(
+        name="permitted_document_ids",
+        describe=(
+            "Document-visibility query built from documents.permissions."
+            "permitted_document_ids -- the resolved-ID-set alternative to "
+            "guardian_visibility_query, for side-by-side comparison."
+        ),
+        run=_permitted_document_ids_run,
+        queryset_for_explain=_permitted_document_ids_queryset,
+    ),
+)
@@ -0,0 +1,445 @@
+# src/paperless_benchmark/seeding.py
+from __future__ import annotations
+
+import datetime
+import random
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+from typing import Literal
+
+if TYPE_CHECKING:
+    from django.contrib.auth.models import Group
+    from django.contrib.auth.models import User
+
+Tier = Literal["home", "medium", "large"]
+
+CHUNK_SIZE = 5_000
+
+MIME_TYPES = (
+    "application/pdf",
+    "image/png",
+    "image/jpeg",
+    "text/plain",
+)
+
+# Ownership split for documents, mirroring the shape used in the #11950
+# perf-benchmark dataset (owned-by-target / owned-by-other / unowned).
+OWNED_BY_TARGET_FRACTION = 0.60
+OWNED_BY_OTHER_FRACTION = 0.30
+# remainder (0.10) is unowned
+
+# Of documents owned by "other" users, the fraction explicitly shared
+# (view, or view+change) with perf_target via guardian permissions --
+# this is what exercises the get_user_can_change() per-row N+1 that the
+# `run` endpoint benchmarks measure.
+SHARED_WITH_TARGET_FRACTION = 0.5
+SHARED_WITH_CHANGE_FRACTION = 0.5
+
+# Guardian permission-row ratios measured from a real install (discussion
+# #13276): 1,414 user-perm rows / 27,232 group-perm rows over 12,000
+# documents. Layered across ALL owned documents for the general user/group
+# pool (not just perf_target's shares), so `profile` scenarios exercise a
+# realistic permission-join shape for arbitrary users, not only perf_target.
+USER_PERM_ROWS_PER_DOC = 1_414 / 12_000
+GROUP_PERM_ROWS_PER_DOC = 27_232 / 12_000
+
+
+@dataclass(frozen=True, slots=True)
+class _TierCounts:
+    documents: int
+    tags: int
+    correspondents: int
+    document_types: int
+    storage_paths: int
+    other_users: int
+    groups: int
+    tags_per_doc: tuple[int, int]
+
+
+TIERS: dict[Tier, _TierCounts] = {
+    "home": _TierCounts(
+        documents=500,
+        tags=20,
+        correspondents=10,
+        document_types=8,
+        storage_paths=5,
+        other_users=3,
+        groups=2,
+        tags_per_doc=(1, 3),
+    ),
+    "medium": _TierCounts(
+        documents=20_000,
+        tags=100,
+        correspondents=300,
+        document_types=50,
+        storage_paths=20,
+        other_users=10,
+        groups=5,
+        tags_per_doc=(2, 6),
+    ),
+    "large": _TierCounts(
+        documents=360_000,
+        tags=1_000,
+        correspondents=5_000,
+        document_types=300,
+        storage_paths=50,
+        other_users=25,
+        groups=10,
+        tags_per_doc=(3, 7),
+    ),
+}
+
+
+def log(msg: str) -> None:
+    print(f"[{time.strftime('%H:%M:%S')}] {msg}", flush=True)  # noqa: T201
+
+
+@dataclass(frozen=True, slots=True)
+class SeededData:
+    """
+    Summary of a completed seed run. `documents`/`tags`/`correspondents`/
+    `document_types`/`storage_paths` are counts, not the seeded ORM
+    instances: at the `large` tier (360,000 documents) holding every
+    instance in memory simultaneously is a real risk for zero benefit, since
+    no caller consumes anything but the counts. `perf_target`/`perf_admin`/
+    `users`/`groups` stay as real objects -- at most ~26 users/12 groups even
+    at `large` tier, and small enough to be useful to a future caller.
+    """
+
+    perf_target: User
+    perf_admin: User
+    users: tuple[User, ...]
+    groups: tuple[Group, ...]
+    documents: int
+    tags: int
+    correspondents: int
+    document_types: int
+    storage_paths: int
+
+
+def _grant_model_level_permissions(user: User) -> None:
+    """
+    Grant perf_target Django model-level view/add/change permissions on
+    Document and Tag, on top of the per-object guardian grants seeding
+    creates elsewhere. DRF's PaperlessObjectPermissions checks model-level
+    permissions before guardian's object-level ones are ever consulted, so
+    without this perf_target gets a blanket 403 on /api/documents/ and
+    /api/tags/ regardless of which documents guardian says it can see.
+    """
+    from django.contrib.auth.models import Permission
+    from django.contrib.contenttypes.models import ContentType
+
+    from documents.models import Document
+    from documents.models import Tag
+
+    for model in (Document, Tag):
+        content_type = ContentType.objects.get_for_model(model)
+        codenames = [
+            f"{action}_{model._meta.model_name}" for action in ("view", "add", "change")
+        ]
+        perms = Permission.objects.filter(
+            content_type=content_type,
+            codename__in=codenames,
+        )
+        user.user_permissions.add(*perms)
+
+
+def _create_users_and_groups(
+    counts: _TierCounts,
+) -> tuple[User, User, tuple[User, ...], tuple[Group, ...]]:
+    from django.contrib.auth.models import Group
+
+    from documents.tests.factories import UserFactory
+
+    perf_target = UserFactory.create(username="perf_target")
+    _grant_model_level_permissions(perf_target)
+    perf_admin = UserFactory.create(username="perf_admin", superuser=True)
+    other_users = tuple(UserFactory.create_batch(counts.other_users))
+    groups = tuple(
+        Group.objects.create(name=f"benchmark_group_{i}") for i in range(counts.groups)
+    )
+    log(
+        f"Created users: 1 target, 1 superuser, {len(other_users)} other, "
+        f"{len(groups)} groups.",
+    )
+    return perf_target, perf_admin, other_users, groups
+
+
+def _create_lookup_tables(counts: _TierCounts):
+    from documents.models import Correspondent
+    from documents.models import DocumentType
+    from documents.models import StoragePath
+    from documents.models import Tag
+    from documents.tests.factories import CorrespondentFactory
+    from documents.tests.factories import DocumentTypeFactory
+    from documents.tests.factories import StoragePathFactory
+    from documents.tests.factories import TagFactory
+
+    tags = tuple(Tag.objects.bulk_create(TagFactory.build_batch(counts.tags)))
+    correspondents = tuple(
+        Correspondent.objects.bulk_create(
+            CorrespondentFactory.build_batch(counts.correspondents),
+        ),
+    )
+    document_types = tuple(
+        DocumentType.objects.bulk_create(
+            DocumentTypeFactory.build_batch(counts.document_types),
+        ),
+    )
+    storage_paths = tuple(
+        StoragePath.objects.bulk_create(
+            StoragePathFactory.build_batch(counts.storage_paths),
+        ),
+    )
+    log(
+        f"Created {len(tags)} tags, {len(correspondents)} correspondents, "
+        f"{len(document_types)} document types, {len(storage_paths)} storage paths.",
+    )
+    return tags, correspondents, document_types, storage_paths
+
+
+def _assign_owner(rng: random.Random, perf_target: User, other_users: tuple[User, ...]):
+    roll = rng.random()
+    if roll < OWNED_BY_TARGET_FRACTION:
+        return perf_target, "target"
+    if roll < OWNED_BY_TARGET_FRACTION + OWNED_BY_OTHER_FRACTION:
+        return rng.choice(other_users), "other"
+    return None, "unowned"
+
+
+def _seed_documents(
+    rng: random.Random,
+    counts: _TierCounts,
+    tags,
+    correspondents,
+    document_types,
+    storage_paths,
+    perf_target,
+    other_users,
+) -> tuple[int, list[int]]:
+    """
+    Bulk-create `counts.documents` documents in chunks. Returns the total
+    document count plus a lightweight list of pks for documents that ended
+    up with an owner (target or other) -- that's all
+    `_grant_general_permissions` needs to sample from, so full `Document`
+    instances aren't accumulated across chunks (a real memory concern at the
+    `large` tier's 360,000 documents).
+    """
+    from django.contrib.auth.models import Permission
+    from django.contrib.contenttypes.models import ContentType
+    from guardian.models import UserObjectPermission
+
+    from documents.models import Document
+    from documents.tests.factories import DocumentFactory
+
+    tag_ids = [t.pk for t in tags]
+    correspondent_ids = [c.pk for c in correspondents]
+    document_type_ids = [d.pk for d in document_types]
+    storage_path_ids = [s.pk for s in storage_paths]
+
+    doc_content_type = ContentType.objects.get_for_model(Document)
+    view_perm = Permission.objects.get(
+        codename="view_document",
+        content_type=doc_content_type,
+    )
+    change_perm = Permission.objects.get(
+        codename="change_document",
+        content_type=doc_content_type,
+    )
+    through_model = Document.tags.through
+
+    document_count = 0
+    owned_document_pks: list[int] = []
+    remaining = counts.documents
+    while remaining > 0:
+        chunk_n = min(CHUNK_SIZE, remaining)
+        remaining -= chunk_n
+
+        batch = []
+        owner_buckets = []
+        for _ in range(chunk_n):
+            doc = DocumentFactory.build(
+                mime_type=rng.choice(MIME_TYPES),
+                page_count=rng.randint(1, 30),
+                correspondent_id=(
+                    rng.choice(correspondent_ids)
+                    if correspondent_ids and rng.random() < 0.8
+                    else None
+                ),
+                document_type_id=(
+                    rng.choice(document_type_ids)
+                    if document_type_ids and rng.random() < 0.6
+                    else None
+                ),
+                storage_path_id=(
+                    rng.choice(storage_path_ids)
+                    if storage_path_ids and rng.random() < 0.8
+                    else None
+                ),
+                # Document.created is a plain DateField (default: today).
+                # Leaving it unset would give every seeded document the same
+                # date, collapsing Document's ("-created",) ordering index
+                # into a single-valued sort key -- spread it over a
+                # realistic multi-year window instead.
+                created=datetime.date.today()
+                - datetime.timedelta(days=rng.randint(0, 365 * 3)),
+            )
+            owner, bucket = _assign_owner(rng, perf_target, other_users)
+            doc.owner_id = owner.pk if owner else None
+            batch.append(doc)
+            owner_buckets.append(bucket)
+
+        created = Document.objects.bulk_create(batch, batch_size=CHUNK_SIZE)
+
+        through_rows = []
+        for doc in created:
+            k = rng.randint(*counts.tags_per_doc)
+            for tag_id in rng.sample(tag_ids, min(k, len(tag_ids))):
+                through_rows.append(through_model(document_id=doc.pk, tag_id=tag_id))
+        if through_rows:
+            through_model.objects.bulk_create(through_rows, batch_size=CHUNK_SIZE)
+
+        perm_rows = []
+        for doc, bucket in zip(created, owner_buckets, strict=True):
+            if bucket == "unowned":
+                continue
+            owned_document_pks.append(doc.pk)
+
+            if bucket != "other":
+                continue
+            if rng.random() >= SHARED_WITH_TARGET_FRACTION:
+                continue
+            perm_rows.append(
+                UserObjectPermission(
+                    permission=view_perm,
+                    content_type=doc_content_type,
+                    object_pk=str(doc.pk),
+                    user=perf_target,
+                ),
+            )
+            if rng.random() < SHARED_WITH_CHANGE_FRACTION:
+                perm_rows.append(
+                    UserObjectPermission(
+                        permission=change_perm,
+                        content_type=doc_content_type,
+                        object_pk=str(doc.pk),
+                        user=perf_target,
+                    ),
+                )
+        if perm_rows:
+            UserObjectPermission.objects.bulk_create(perm_rows, batch_size=CHUNK_SIZE)
+
+        document_count += len(created)
+        log(f"  {document_count}/{counts.documents} documents seeded")
+
+    return document_count, owned_document_pks
+
+
+def _grant_general_permissions(
+    rng: random.Random,
+    owned_document_pks: list[int],
+    users,
+    groups,
+) -> None:
+    """
+    Layer realistic (issue #13276-derived) guardian permission-row ratios
+    across owned documents for the general user/group pool, so `profile`
+    scenarios exercise the same permission-join shape regardless of which
+    user they check visibility for (not just perf_target).
+    """
+    from django.contrib.auth.models import Permission
+    from django.contrib.contenttypes.models import ContentType
+    from guardian.models import GroupObjectPermission
+    from guardian.models import UserObjectPermission
+
+    from documents.models import Document
+
+    if not owned_document_pks or not users:
+        return
+
+    doc_content_type = ContentType.objects.get_for_model(Document)
+    view_perm = Permission.objects.get(
+        codename="view_document",
+        content_type=doc_content_type,
+    )
+
+    n_user_perms = round(len(owned_document_pks) * USER_PERM_ROWS_PER_DOC)
+    n_group_perms = (
+        round(len(owned_document_pks) * GROUP_PERM_ROWS_PER_DOC) if groups else 0
+    )
+
+    user_rows = [
+        UserObjectPermission(
+            permission=view_perm,
+            content_type=doc_content_type,
+            object_pk=str(rng.choice(owned_document_pks)),
+            user=rng.choice(users),
+        )
+        for _ in range(n_user_perms)
+    ]
+    if user_rows:
+        UserObjectPermission.objects.bulk_create(
+            user_rows,
+            batch_size=CHUNK_SIZE,
+            ignore_conflicts=True,
+        )
+
+    group_rows = [
+        GroupObjectPermission(
+            permission=view_perm,
+            content_type=doc_content_type,
+            object_pk=str(rng.choice(owned_document_pks)),
+            group=rng.choice(groups),
+        )
+        for _ in range(n_group_perms)
+    ]
+    if group_rows:
+        GroupObjectPermission.objects.bulk_create(
+            group_rows,
+            batch_size=CHUNK_SIZE,
+            ignore_conflicts=True,
+        )
+
+    log(f"  Granted {len(user_rows)} user perms, {len(group_rows)} group perms.")
+
+
+def seed_benchmark_dataset(tier: Tier, *, seed: int = 42) -> SeededData:
+    """
+    Build a benchmark dataset at the given scale tier: a named perf_target
+    (mixed owned/shared documents) and perf_admin (superuser) for endpoint
+    benchmarking, plus a general user/group pool with realistic guardian
+    permission-row ratios for profile scenarios.
+    """
+    counts = TIERS[tier]
+    rng = random.Random(seed)
+
+    log(f"Seeding tier={tier!r}")
+    perf_target, perf_admin, other_users, groups = _create_users_and_groups(counts)
+    tags, correspondents, document_types, storage_paths = _create_lookup_tables(counts)
+    document_count, owned_document_pks = _seed_documents(
+        rng,
+        counts,
+        tags,
+        correspondents,
+        document_types,
+        storage_paths,
+        perf_target,
+        other_users,
+    )
+    all_users = (perf_target, *other_users)
+    _grant_general_permissions(rng, owned_document_pks, all_users, groups)
+
+    log(f"Done. {document_count} documents seeded for tier={tier!r}.")
+
+    return SeededData(
+        perf_target=perf_target,
+        perf_admin=perf_admin,
+        users=all_users,
+        groups=groups,
+        documents=document_count,
+        tags=len(tags),
+        correspondents=len(correspondents),
+        document_types=len(document_types),
+        storage_paths=len(storage_paths),
+    )
@@ -1,9 +1,7 @@
 import logging

-from celery import Task
 from celery import shared_task

-from documents.models import PaperlessTask
 from paperless_mail.mail import MailAccountHandler
 from paperless_mail.mail import MailError
 from paperless_mail.models import MailAccount
@@ -12,26 +10,8 @@ from paperless_mail.models import MailRule
 logger = logging.getLogger("paperless.mail.tasks")


-@shared_task(bind=True)
-def process_mail_accounts(self: Task, account_ids: list[int] | None = None) -> str:
-    # A scheduled check can still be running (or queued) when the next one
-    # ProcessedMail dedup only records a message once its
-    # handling has finished, so an overlapping run can still pick up the same
-    # not-yet-recorded message. Skip outright rather than race it.
-    other_mail_fetch_running = (
-        PaperlessTask.objects.filter(
-            task_type=PaperlessTask.TaskType.MAIL_FETCH,
-            status__in=[PaperlessTask.Status.PENDING, PaperlessTask.Status.STARTED],
-        )
-        .exclude(task_id=self.request.id)
-        .exists()
-    )
-    if other_mail_fetch_running:
-        logger.info(
-            "Mail account processing is already running; skipping this run.",
-        )
-        return "Skipped: mail account processing already in progress."
-
+@shared_task
+def process_mail_accounts(account_ids: list[int] | None = None) -> str:
    total_new_documents = 0
    accounts = (
        MailAccount.objects.filter(pk__in=account_ids)
@@ -1,134 +0,0 @@
-from typing import Final
-
-import pytest
-import pytest_mock
-
-from documents.models import PaperlessTask
-from documents.tests.factories import PaperlessTaskFactory
-from paperless_mail import tasks
-from paperless_mail.tests.factories import MailAccountFactory
-from paperless_mail.tests.factories import MailRuleFactory
-
-NO_DOCUMENTS_ADDED: Final = "No new documents were added."
-SKIPPED: Final = "Skipped: mail account processing already in progress."
-
-
-@pytest.mark.django_db
-@pytest.mark.usefixtures("account_with_rule")
-class TestProcessMailAccountsOverlap:
-    @pytest.fixture
-    def account_with_rule(self) -> None:
-        """An enabled mail account with a single enabled rule."""
-        account = MailAccountFactory.create()
-        MailRuleFactory.create(account=account, enabled=True)
-
-    @pytest.mark.parametrize(
-        ("status", "expected_result", "expected_call_count"),
-        [
-            pytest.param(
-                PaperlessTask.Status.PENDING,
-                SKIPPED,
-                0,
-                id="pending-task-blocks",
-            ),
-            pytest.param(
-                PaperlessTask.Status.STARTED,
-                SKIPPED,
-                0,
-                id="started-task-blocks",
-            ),
-            pytest.param(
-                PaperlessTask.Status.SUCCESS,
-                NO_DOCUMENTS_ADDED,
-                1,
-                id="finished-task-does-not-block",
-            ),
-        ],
-    )
-    def test_skips_only_while_another_mail_fetch_task_runs(
-        self,
-        mocker: pytest_mock.MockerFixture,
-        status: PaperlessTask.Status,
-        expected_result: str,
-        expected_call_count: int,
-    ) -> None:
-        """
-        GIVEN:
-            - An enabled mail account with a rule
-            - Another mail fetch task row in the given status
-        WHEN:
-            - Mail accounts are processed
-        THEN:
-            - Processing is skipped only if that other task is pending or running
-        """
-        PaperlessTaskFactory.create(
-            task_type=PaperlessTask.TaskType.MAIL_FETCH,
-            trigger_source=PaperlessTask.TriggerSource.SCHEDULED,
-            status=status,
-        )
-
-        mocked_handle = mocker.patch.object(
-            tasks.MailAccountHandler,
-            "handle_mail_account",
-            return_value=0,
-        )
-
-        result = tasks.process_mail_accounts()
-
-        assert mocked_handle.call_count == expected_call_count
-        assert result == expected_result
-
-    def test_runs_when_no_other_mail_fetch_task_exists(
-        self,
-        mocker: pytest_mock.MockerFixture,
-    ) -> None:
-        """
-        GIVEN:
-            - An enabled mail account with a rule
-            - No other mail fetch task rows
-        WHEN:
-            - Mail accounts are processed
-        THEN:
-            - The account is handled
-        """
-        mocked_handle = mocker.patch.object(
-            tasks.MailAccountHandler,
-            "handle_mail_account",
-            return_value=0,
-        )
-
-        result = tasks.process_mail_accounts()
-
-        mocked_handle.assert_called_once()
-        assert result == NO_DOCUMENTS_ADDED
-
-    def test_does_not_skip_due_to_its_own_task_row(
-        self,
-        mocker: pytest_mock.MockerFixture,
-    ) -> None:
-        """
-        GIVEN:
-            - An enabled mail account with a rule
-            - A running mail fetch task row belonging to this very task
-        WHEN:
-            - Mail accounts are processed under that task id
-        THEN:
-            - The task does not skip itself and handles the account
-        """
-        PaperlessTaskFactory.create(
-            task_id="self-task-id",
-            task_type=PaperlessTask.TaskType.MAIL_FETCH,
-            trigger_source=PaperlessTask.TriggerSource.SCHEDULED,
-            status=PaperlessTask.Status.STARTED,
-        )
-
-        mocked_handle = mocker.patch.object(
-            tasks.MailAccountHandler,
-            "handle_mail_account",
-            return_value=0,
-        )
-
-        result = tasks.process_mail_accounts.apply(task_id="self-task-id").result
-
-        mocked_handle.assert_called_once()
-        assert result == NO_DOCUMENTS_ADDED
Author	SHA1	Message	Date
stumpylogandClaude Sonnet 5	a10f8fc81f	chore: remove benchmark-commands spec/plan docs now that work is complete Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:25 -07:00
stumpylogandClaude Sonnet 5	5ac6ae52de	Fix: apply final-review fix wave (safety guard, date spread, MariaDB explain, history fields, memory footprint, minors) Final whole-branch review before this tooling branch settles: add a required --yes-i-know-this-wipes-the-database confirmation flag ahead of --reset (it deletes ALL users/groups/documents in the target DB, not just benchmark-created ones); spread seeded documents' created dates over a 3-year window instead of leaving them all on one default date; fix capture_explain() to use MariaDB's ANALYZE syntax instead of Postgres-only EXPLAIN ANALYZE (verified against real MariaDB 12.3 -- the old code was a silent 1064 syntax error); record db_vendor/document_count in run/profile history entries; shrink SeededData's memory footprint at scale by returning counts instead of full ORM instance tuples; and a handful of minor fixes (storage_path assignment, --explain warning instead of silent no-op, harness.py repeat<1 guard, type annotations). All changes verified against real Postgres and MariaDB containers on the VM, not just SQLite. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:25 -07:00
stumpylogandClaude Sonnet 5	235be8d15f	Fix: correct /api/tags/ endpoint URL in benchmarking skill The skill documentation stated that run benchmarks /api/tags/ but src/paperless_benchmark/endpoints.py actually uses /api/tags/?page_size=100000. Update the skill's Command reference section to match the actual endpoint definition. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:25 -07:00
stumpylog	69af169575	docs(benchmark): add paperless-benchmarking skill	2026-09-09 07:42:25 -07:00
stumpylog	6aecbac2c4	fix(benchmark): fully clear users/groups on reset, grant perf_target model perms	2026-09-09 07:42:25 -07:00
stumpylogandClaude Sonnet 5	0bc7221e12	docs: add Task 10 (quirk fixes) and Task 11 (skill) to benchmark plan Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:25 -07:00
stumpylog	95474bf1ae	fix(benchmark): reuse seeded data in profile instead of reseeding _handle_profile unconditionally called seed_benchmark_dataset every run, colliding with an IntegrityError against data seeded by a prior seed/profile call. Look up the existing perf_target user instead, matching _handle_run's already-correct pattern, and raise a CommandError pointing at `seed` if no dataset exists. Scenario.run/queryset_for_explain now take a User directly instead of a SeededData, since only data.users[0] (perf_target) was ever used. profile no longer seeds, so it drops the --tier dependency: the printed line and history entry no longer include a tier/(tier=...) suffix, and --tier's help text now only mentions `seed`.	2026-09-09 07:42:24 -07:00
stumpylog	e9bedd9343	feat(benchmark): add manage.py benchmark command (seed/run/profile/list-scenarios)	2026-09-09 07:42:24 -07:00
stumpylog	cee6519a06	feat(benchmark): add pluggable profile-scenario registry	2026-09-09 07:42:24 -07:00
stumpylog	c7b94952d2	feat(benchmark): add timed API endpoint scenarios	2026-09-09 07:42:24 -07:00
stumpylogandClaude Sonnet 5	bfae3ef82d	docs: add implementation plan for benchmark management commands Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:24 -07:00
stumpylog	7b90eb5069	feat(benchmark): add local JSONL results history log	2026-09-09 07:42:24 -07:00
stumpylog	e15dad04bb	feat(benchmark): add tiered dataset seeding	2026-09-09 07:42:24 -07:00
stumpylog	d0731ce892	feat(benchmark): add per-backend reset and explain helpers	2026-09-09 07:42:24 -07:00
stumpylog	dda0c93e80	feat(benchmark): add run_profile timing harness	2026-09-09 07:42:24 -07:00
stumpylogandClaude Sonnet 5	a09ea28636	feat(benchmark): scaffold paperless_benchmark app Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:24 -07:00
stumpylogandClaude Sonnet 5	d5a9c502fc	docs: add design spec for benchmark management commands Merges the standalone run_benchmarks.py/seed_benchmark_data.py scripts and the tools/profiling branch's Postgres-only harness into one cross-backend `manage.py benchmark` command suite. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-09 07:42:24 -07:00