chore: add whoosh-compat transition skill

Encodes the settled integration decisions for replacing the
hand-maintained search translation layer with whoosh-compat:
user-typed query surface policy, analyzer seam, diagnostics-before-emit
contract, mandatory date parity audit, test churn, and rollout plan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.claude/skills/whoosh-compat-transition/SKILL.md

@@ -0,0 +1,55 @@
+---
+name: whoosh-compat-transition
+description: Use when integrating the whoosh-compat library into paperless-ngx search, replacing src/documents/search/_translate.py or _dates.py, building the search FieldRegistry, or changing user query parsing during the whoosh-to-tantivy transition
+---
+
+# whoosh-compat transition
+
+## Overview
+
+whoosh-compat (github.com/stumpylog/whoosh-compat; local checkout usually at `../whoosh-compat`) replaces the hand-maintained translation layer (`src/documents/search/_translate.py`, `_dates.py`): it parses user queries with a faithful fork of whoosh's real grammar into a typed AST and emits programmatic tantivy queries. Read its README and ARCHITECTURE.md before wiring anything; its DIVERGENCES.md lists intended behavior differences and is the authority on "is this difference a bug".
+
+## Decisions already made (do not re-derive)
+
+- **Queries are user-typed free text.** The advanced search box passes whatever the user types straight to the parser (that is how the issue #13568 queries exist). Do NOT try to infer the supported field surface from frontend code; the frontend only generates a few date filter strings, everything else is typed by users.
+- **The field surface is a policy decision, not `KNOWN_FIELDS`.** Today's `KNOWN_FIELDS` accepts internal ID fields (`tag_id`, `owner_id`, `viewer_id`, other `*_id`) that are undocumented in `docs/usage.md` and were ruled not user-searchable by the maintainer: exclude them from the `FieldRegistry` (they stay as programmatic permission/filter fields in `build_permission_filter`, which never touches user query text). The registry is built from documented syntax in `docs/usage.md` plus the v2-compat aliases (`type`, `path`, `type_id`-style aliases follow their canonical field's fate). Undocumented-but-working fields (`asn`, `page_count`, `num_notes`, `original_filename`, `checksum`) need an explicit maintainer yes/no; since users type freely, silently dropping one breaks any saved view using it, so a drop must be a visible, documented decision.
+- **Analyzer seam:** `FieldSpec.analyzer` binds the live registered tantivy analyzer's `.analyze` (the same Rust analyzer used at index time; language-keyed, so rebuild the registry when `SEARCH_LANGUAGE` changes, on the same trigger as `register_tokenizers`). `pattern_normalizer` is `_tokenizer.ascii_fold`: character-level lowercase+fold only, NEVER stemming.
+- **Diagnostics before emit:** `whoosh_compat.parse()` never raises on bad input. Check `ParseResult.diagnostics` and map to `SearchQueryError`/`InvalidDateQuery` (HTTP 400) BEFORE calling `emit()`; also catch the emitter's `UnsupportedQueryError` into a 400. Never carry forward the legacy raw-string fallback (`except Exception: query_str = raw_query`) into the new path; it masks integration bugs.
+- **`notes` and `custom_fields` are JSON fields** with fixed subpaths (`notes.user`/`notes.note`, `custom_fields.name`/`custom_fields.value`); the registry stays a static, language-keyed singleton, never per-request.
+
+## Mandatory before deleting old code
+
+- Date-grammar parity audit, line by line: every keyword, relative unit, and abbreviation `_dates.py` and `_translate.py` accept today (including the whoosh-era abbreviations kept for old saved views) must have an accepted form in whoosh-compat's dateparse grammar. Silent keyword loss is the saved-view breakage class behind issue #13568.
+- Acceptance corpus compared by matched-document-ID sets, not query strings: the #13568 queries verbatim, real saved-view strings, every date keyword, field aliases, comma lists, date and numeric ranges, wildcards with bracket classes, boosts, JSON subpaths.
+
+## Tests: what goes, what comes
+
+Removed with their modules (do not port their string-level assertions):
+
+- `src/documents/tests/search/test_translate.py`: its subject is deleted; string-translation unit cases are whoosh-compat's own responsibility now. Cases that encode real user-visible behavior get reincarnated as result-level acceptance cases, not string assertions.
+- Date-keyword unit tests tied to `_dates.py` internals: same treatment.
+- `test_query.py` cases asserting `parse_user_query` internals or intermediate query strings: rewritten against the new pipeline, asserting on matched results.
+
+Kept: `test_migration_fulltext_query_field_prefixes.py` (data migration, orthogonal), `test_schema.py`, `test_tokenizer.py`, permission-filter and simple-search tests.
+
+Added:
+
+- A result-level acceptance module (paperless's analogue of whoosh-compat's `test_acceptance_e2e.py`): the corpus above against a real index built from `build_schema()`, asserting document-ID sets. Use `pytest.param(..., id="...")` for every case.
+- Registry unit tests: internal `*_id` names rejected, aliases resolve to canonical fields, JSON subpaths match `docs/usage.md`, construction deterministic per language.
+- One `Multitoken` case nested inside a top-level `OR` (whoosh-compat DIVERGENCES entry on Multitoken.DEFAULT) to prove it does not matter for paperless's data.
+- If acceptance work surfaces a new whoosh-compat divergence, that is a whoosh-compat-repo change (its `differential-triage` skill applies), not a silent paperless workaround.
+
+## Coordination
+
+- whoosh-compat is pre-1.0: pin an exact version or git SHA; upgrades are deliberate, reviewed changes.
+- JSON subpath emission depends on the installed tantivy-py version (fallback until quickwit-oss/tantivy-py#716 ships). The whoosh-compat repo has a `carve-out-retirement` skill; coordinate tantivy pin bumps with it, in a separate PR from the parser migration.
+- Rollout: settings flag defaulting to the legacy path plus shadow-compare logging (log when old and new paths return different ID sets; sample if cost matters) for one release; delete `_translate.py`/`_dates.py` only after the flag defaults to the new path with no material reports.
+
+## Common mistakes
+
+- Inferring the field surface from frontend code (users type queries directly).
+- Copying `KNOWN_FIELDS` into the registry wholesale (resurfaces internal fields).
+- Wiring stemming into `pattern_normalizer`.
+- Calling `emit()` unconditionally, or porting the legacy raw-string fallback.
+- Deleting `_dates.py` without the parity audit.
+- Porting `test_translate.py`'s string assertions instead of writing result-level tests.

.github/workflows/ci-frontend.yml

@@ -129,8 +129,8 @@ jobs:
             ~/.pnpm-store
             ~/.cache
           key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
-      - name: Install dependencies
+      - name: Re-link Angular CLI
-        run: cd src-ui && pnpm install --frozen-lockfile
+        run: cd src-ui && pnpm link @angular/cli
       - name: Run lint
         run: cd src-ui && pnpm run lint
   unit-tests:
@@ -168,8 +168,8 @@ jobs:
             ~/.pnpm-store
             ~/.cache
           key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
-      - name: Install dependencies
+      - name: Re-link Angular CLI
-        run: cd src-ui && pnpm install --frozen-lockfile
+        run: cd src-ui && pnpm link @angular/cli
       - name: Run Jest unit tests
         run: cd src-ui && pnpm run test --max-workers=2 --shard=${{ matrix.shard-index }}/${{ matrix.shard-count }}
       - name: Upload test results to Codecov
@@ -223,15 +223,18 @@ jobs:
             ~/.pnpm-store
             ~/.cache
           key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
+      - name: Re-link Angular CLI
+        run: cd src-ui && pnpm link @angular/cli
       - name: Install dependencies
-        run: cd src-ui && pnpm install --frozen-lockfile
+        run: cd src-ui && pnpm install --no-frozen-lockfile
       - name: Run Playwright E2E tests
         run: cd src-ui && pnpm exec playwright test --shard ${{ matrix.shard-index }}/${{ matrix.shard-count }}
-  frontend-build:
+  bundle-analysis:
-    name: Frontend Build
+    name: Bundle Analysis
     needs: [changes, unit-tests, e2e-tests]
     if: needs.changes.outputs.frontend_changed == 'true'
     runs-on: ubuntu-24.04
+    environment: bundle-analysis
     permissions:
       contents: read
     steps:
@@ -257,19 +260,21 @@ jobs:
             ~/.pnpm-store
             ~/.cache
           key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
-      - name: Install dependencies
+      - name: Re-link Angular CLI
-        run: cd src-ui && pnpm install --frozen-lockfile
+        run: cd src-ui && pnpm link @angular/cli
-      - name: Build
+      - name: Build and analyze
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
         run: cd src-ui && pnpm run build --configuration=production
   gate:
     name: Frontend CI Gate
-    needs: [changes, install-dependencies, lint, unit-tests, e2e-tests, frontend-build]
+    needs: [changes, install-dependencies, lint, unit-tests, e2e-tests, bundle-analysis]
     if: always()
     runs-on: ubuntu-slim
     steps:
       - name: Check gate
         env:
-          BUILD_RESULT: ${{ needs['frontend-build'].result }}
+          BUNDLE_ANALYSIS_RESULT: ${{ needs['bundle-analysis'].result }}
           E2E_RESULT: ${{ needs['e2e-tests'].result }}
           FRONTEND_CHANGED: ${{ needs.changes.outputs.frontend_changed }}
           INSTALL_RESULT: ${{ needs['install-dependencies'].result }}
@@ -301,8 +306,8 @@ jobs:
             exit 1
           fi
 
-          if [[ "${BUILD_RESULT}" != "success" ]]; then
+          if [[ "${BUNDLE_ANALYSIS_RESULT}" != "success" ]]; then
-            echo "::error::Frontend build job result: ${BUILD_RESULT}"
+            echo "::error::Frontend bundle-analysis job result: ${BUNDLE_ANALYSIS_RESULT}"
             exit 1
           fi
 

.github/workflows/translate-strings.yml

@@ -61,7 +61,10 @@ jobs:
             ~/.cache
           key: ${{ runner.os }}-frontenddeps-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
       - name: Install frontend dependencies
-        run: cd src-ui && pnpm install --frozen-lockfile
+        if: steps.cache-frontend-deps.outputs.cache-hit != 'true'
+        run: cd src-ui && pnpm install
+      - name: Re-link Angular cli
+        run: cd src-ui && pnpm link @angular/cli
       - name: Generate frontend translation strings
         run: |
           cd src-ui

pyproject.toml

@@ -38,6 +38,7 @@ dependencies = [
   "django-soft-delete~=1.0.18",
   "django-treenode>=0.24",
   "djangorestframework~=3.16",
+  "djangorestframework-guardian~=0.4.0",
   "drf-spectacular~=0.30",
   "drf-spectacular-sidecar~=2026.7.1",
   "drf-writable-nested~=0.7.1",

src-ui/angular.json

@@ -56,13 +56,13 @@
       },
       "architect": {
         "build": {
-          "builder": "@angular/build:application",
+          "builder": "@angular-builders/custom-webpack:browser",
           "options": {
-            "outputPath": {
+            "customWebpackConfig": {
-              "base": "dist/paperless-ui",
+                "path": "./extra-webpack.config.ts"
-              "browser": ""
             },
-            "browser": "src/main.ts",
+            "outputPath": "dist/paperless-ui",
+            "main": "src/main.ts",
             "outputHashing": "none",
             "index": "src/index.html",
             "polyfills": [
@@ -97,7 +97,6 @@
             "scripts": [],
             "allowedCommonJsDependencies": [
               "file-saver",
-              "mime-names",
               "utif"
             ],
             "extractLicenses": false,
@@ -118,13 +117,11 @@
                   "with": "src/environments/environment.prod.ts"
                 }
               ],
-              "outputPath": {
+              "outputPath": "../src/documents/static/frontend/",
-                "base": "../src/documents/static/frontend/",
-                "browser": ""
-              },
               "optimization": true,
               "outputHashing": "none",
               "sourceMap": false,
+              "namedChunks": false,
               "extractLicenses": true,
               "budgets": [
                 {
@@ -148,7 +145,7 @@
           "defaultConfiguration": ""
         },
         "serve": {
-          "builder": "@angular/build:dev-server",
+          "builder": "@angular-builders/custom-webpack:dev-server",
           "options": {
             "buildTarget": "paperless-ui:build:en-US"
           },
@@ -159,7 +156,7 @@
           }
         },
         "extract-i18n": {
-          "builder": "@angular/build:extract-i18n",
+          "builder": "@angular-builders/custom-webpack:extract-i18n",
           "options": {
             "buildTarget": "paperless-ui:build"
           }

src-ui/extra-webpack.config.ts

@@ -0,0 +1,24 @@
+import {
+  CustomWebpackBrowserSchema,
+  TargetOptions,
+} from '@angular-builders/custom-webpack'
+import * as webpack from 'webpack'
+const { codecovWebpackPlugin } = require('@codecov/webpack-plugin')
+
+export default (
+  config: webpack.Configuration,
+  options: CustomWebpackBrowserSchema,
+  targetOptions: TargetOptions
+) => {
+  if (config.plugins) {
+    config.plugins.push(
+      codecovWebpackPlugin({
+        enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
+        bundleName: 'paperless-ngx',
+        uploadToken: process.env.CODECOV_TOKEN,
+      })
+    )
+  }
+
+  return config
+}

src-ui/package.json

@@ -12,13 +12,13 @@
   "private": true,
   "dependencies": {
     "@angular/cdk": "^22.0.6",
-    "@angular/common": "~22.1.0",
+    "@angular/common": "~22.0.8",
-    "@angular/compiler": "~22.1.0",
+    "@angular/compiler": "~22.0.8",
-    "@angular/core": "~22.1.0",
+    "@angular/core": "~22.0.8",
-    "@angular/forms": "~22.1.0",
+    "@angular/forms": "~22.0.8",
-    "@angular/localize": "~22.1.0",
+    "@angular/localize": "~22.0.8",
-    "@angular/platform-browser": "~22.1.0",
+    "@angular/platform-browser": "~22.0.8",
-    "@angular/router": "~22.1.0",
+    "@angular/router": "~22.0.8",
     "@ng-bootstrap/ng-bootstrap": "^21.0.0",
     "@ng-select/ng-select": "^23.5.0",
     "@ngneat/dirty-check-forms": "^3.0.3",
@@ -32,24 +32,26 @@
     "ngx-device-detector": "^12.0.0",
     "ngx-ui-tour-ng-bootstrap": "^19.0.0",
     "normalize-diacritics": "^5.0.0",
-    "pdfjs-dist": "^6.2.108",
+    "pdfjs-dist": "^6.0.227",
     "rxjs": "^7.8.2",
     "tslib": "^2.8.1",
     "utif": "^3.1.0",
     "uuid": "^14.0.1"
   },
   "devDependencies": {
+    "@angular-builders/custom-webpack": "^22.0.1",
     "@angular-builders/jest": "^22.0.1",
-    "@angular-devkit/core": "^22.1.2",
+    "@angular-devkit/core": "^22.0.8",
-    "@angular-devkit/schematics": "^22.1.2",
+    "@angular-devkit/schematics": "^22.0.8",
     "@angular-eslint/builder": "22.1.0",
     "@angular-eslint/eslint-plugin": "22.1.0",
     "@angular-eslint/eslint-plugin-template": "22.1.0",
     "@angular-eslint/schematics": "22.1.0",
     "@angular-eslint/template-parser": "22.1.0",
-    "@angular/build": "22.1.2",
+    "@angular/build": "^22.0.8",
-    "@angular/cli": "22.1.2",
+    "@angular/cli": "~22.0.5",
-    "@angular/compiler-cli": "~22.1.0",
+    "@angular/compiler-cli": "~22.0.8",
+    "@codecov/webpack-plugin": "^2.0.1",
     "@playwright/test": "^1.62.0",
     "@types/jest": "^30.0.0",
     "@types/node": "^26.1.1",
@@ -64,7 +66,8 @@
     "jest-websocket-mock": "^2.5.0",
     "prettier-plugin-organize-imports": "^4.3.0",
     "ts-node": "~10.9.1",
-    "typescript": "^6.0.3"
+    "typescript": "^6.0.3",
+    "webpack": "^5.107.2"
   },
   "packageManager": "pnpm@10.26.0"
 }

src-ui/src/app/components/common/pdf-viewer/pdf-viewer.component.scss

@@ -151,13 +151,6 @@
   inset: 0;
   pointer-events: none;
 
-  & section {
-    position: absolute;
-    text-align: initial;
-    box-sizing: border-box;
-    transform-origin: 0 0;
-  }
-
   & .annotationTextContent {
     opacity: 0;
   }

src-ui/src/app/components/common/pdf-viewer/pdf-viewer.component.ts

@@ -13,7 +13,6 @@ import {
   ViewChild,
 } from '@angular/core'
 import {
-  AnnotationMode,
   getDocument,
   GlobalWorkerOptions,
   PDFDocumentLoadingTask,
@@ -222,7 +221,6 @@ export class PngxPdfViewerComponent
       linkService: this.linkService,
       findController: this.findController,
       textLayerMode,
-      annotationMode: AnnotationMode.ENABLE,
       enableSelectionRendering: false,
       removePageBorders: true,
     }

src-ui/src/app/components/document-list/filter-editor/filter-editor.component.spec.ts

@@ -2213,20 +2213,6 @@ describe('FilterEditorComponent', () => {
     expect(blurSpy).toHaveBeenCalled()
   })
 
-  it('should only dismiss open autocomplete suggestions on Escape, keeping the query', () => {
-    component.textFilter = 'foo bar'
-    component.textFilterInput.nativeElement.value = 'foo bar'
-    jest.spyOn(component.searchTypeahead, 'isPopupOpen').mockReturnValue(true)
-    const dismissSpy = jest
-      .spyOn(component.searchTypeahead, 'dismissPopup')
-      .mockImplementation(() => {})
-    component.textFilterInput.nativeElement.dispatchEvent(
-      new KeyboardEvent('keydown', { key: 'Escape' })
-    )
-    expect(dismissSpy).toHaveBeenCalled()
-    expect(component.textFilter).toEqual('foo bar')
-  })
-
   it('should adjust text filter targets if more like search', () => {
     const TEXT_FILTER_TARGET_FULLTEXT_MORELIKE = 'fulltext-morelike' // private const
     component.textFilterTarget = TEXT_FILTER_TARGET_FULLTEXT_MORELIKE

src-ui/src/app/components/document-list/filter-editor/filter-editor.component.ts

@@ -15,7 +15,6 @@ import {
 import { FormsModule, ReactiveFormsModule } from '@angular/forms'
 import {
   NgbDropdownModule,
-  NgbTypeahead,
   NgbTypeaheadModule,
 } from '@ng-bootstrap/ng-bootstrap'
 import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
@@ -352,9 +351,6 @@ export class FilterEditorComponent
   @ViewChild('textFilterInput')
   textFilterInput: ElementRef
 
-  @ViewChild(NgbTypeahead)
-  searchTypeahead: NgbTypeahead
-
   readonly customFields = signal<CustomField[]>([])
 
   tagDocumentCounts: SelectionDataItem[]
@@ -1154,7 +1150,6 @@ export class FilterEditorComponent
   }
 
   set textFilter(value) {
-    this._textFilter = value // set immediately to prevent loss of keystrokes
     this.textFilterDebounce.next(value)
   }
 
@@ -1247,9 +1242,9 @@ export class FilterEditorComponent
         distinctUntilChanged(),
         filter((query) => !query.length || query.length > 2)
       )
-      .subscribe(() =>
+      .subscribe((text) =>
         this.updateTextFilter(
-          this._textFilter, // use the current value, not the debounced (possibly stale) one
+          text,
           this.textFilterTarget !== TEXT_FILTER_TARGET_FULLTEXT_QUERY
         )
       )
@@ -1325,11 +1320,6 @@ export class FilterEditorComponent
         this.updateTextFilter(filterString)
       }
     } else if (event.key === 'Escape') {
-      if (this.searchTypeahead?.isPopupOpen()) {
-        // only dismiss the suggestions, so longer query can use Enter
-        this.searchTypeahead.dismissPopup()
-        return
-      }
       if (this._textFilter?.length) {
         this.resetTextField()
       } else {

src-ui/src/app/components/manage/document-attributes/management-list/management-list.component.html

@@ -88,7 +88,7 @@
       @if (depth > 0) {
         <div class="indicator"></div>
       }
-      <button class="btn btn-link ms-0 ps-0 text-start" style="user-select: text;" [disabled]="!userCanEdit(object)" (click)="userCanEdit(object) ? openEditDialog(object) : null; $event.stopPropagation()">{{ object.name }}</button>
+      <button class="btn btn-link ms-0 ps-0 text-start" (click)="userCanEdit(object) ? openEditDialog(object) : null; $event.stopPropagation()">{{ object.name }}</button>
     </td>
     <td class="d-none d-sm-table-cell">{{ getMatching(object) }}</td>
     <td>{{ getDocumentCount(object) }}</td>

src-ui/src/test/mocks/pdfjs-legacy-build-pdf.ts

@@ -19,13 +19,6 @@ export const GlobalWorkerOptions = {
   workerSrc: '',
 }
 
-export const AnnotationMode = {
-  DISABLE: 0,
-  ENABLE: 1,
-  ENABLE_FORMS: 2,
-  ENABLE_STORAGE: 3,
-}
-
 export const getDocument = (_src: unknown): PDFDocumentLoadingTask => {
   return new PDFDocumentLoadingTask(Promise.resolve(new PDFDocumentProxy()))
 }

src/documents/export/sinks.py

@@ -1,346 +0,0 @@
-from __future__ import annotations
-
-import abc
-import hashlib
-import json
-import os
-import shutil
-import tempfile
-import zipfile
-from contextlib import AbstractContextManager
-from contextlib import contextmanager
-from pathlib import Path
-from pathlib import PurePosixPath
-from typing import TYPE_CHECKING
-
-from django.conf import settings
-from django.core.serializers.json import DjangoJSONEncoder
-
-from documents.file_handling import delete_empty_directories
-from documents.utils import compute_checksum
-from documents.utils import copy_file_with_basic_stats
-
-if TYPE_CHECKING:
-    from collections.abc import Iterator
-    from typing import TextIO
-
-
-def _dumps(content: list | dict) -> str:
-    """Serialize export JSON consistently across all sinks."""
-    return json.dumps(content, cls=DjangoJSONEncoder, indent=2, ensure_ascii=False)
-
-
-class StreamingManifestWriter:
-    """Incrementally writes a JSON array to a text handle, one record at a time.
-
-    Knows nothing about folders or zips: it writes the array framing and records
-    to whatever handle the sink's ``stream()`` yields. The sink owns the handle's
-    lifecycle (atomic rename, compare, spooling).
-    """
-
-    def __init__(self, handle: TextIO) -> None:
-        self._file = handle
-        self._first = True
-        self._file.write("[")
-
-    def write_record(self, record: dict) -> None:
-        if not self._first:
-            self._file.write(",\n")
-        else:
-            self._first = False
-        self._file.write(_dumps(record))
-
-    def write_batch(self, records: list[dict]) -> None:
-        for record in records:
-            self.write_record(record)
-
-    def close(self) -> None:
-        """Write the closing bracket. Does NOT close the handle (the sink owns it)."""
-        self._file.write("\n]")
-
-
-class ExportSink(AbstractContextManager, abc.ABC):
-    """Destination for a document export.
-
-    The command declares export contents via three verbs; the sink decides how to
-    persist each. ``arcname`` is always a relative POSIX path
-    (e.g. ``"manifest.json"``, ``"originals/foo.pdf"``).
-
-    Contract:
-      * At most one ``stream()`` open at a time (it is the manifest);
-        ``add_file``/``add_json`` may be called while it is open.
-      * Context-manager: normal exit finalizes, an exception aborts. No partial or
-        failed run leaves a complete-looking artifact.
-    """
-
-    @abc.abstractmethod
-    def add_file(
-        self,
-        source: Path,
-        arcname: str,
-        *,
-        checksum: str | None = None,
-    ) -> None: ...
-
-    @abc.abstractmethod
-    def add_json(self, content: list | dict, arcname: str) -> None: ...
-
-    @abc.abstractmethod
-    def stream(self, arcname: str) -> AbstractContextManager[TextIO]: ...
-
-    def _open(self) -> None:
-        """Hook called on context entry. Override as needed."""
-
-    @abc.abstractmethod
-    def _finalize(self) -> None:
-        """Commit on clean exit."""
-
-    @abc.abstractmethod
-    def _abort(self) -> None:
-        """Roll back on exception."""
-
-    def __enter__(self) -> ExportSink:
-        self._open()
-        return self
-
-    def __exit__(self, exc_type, exc_val, exc_tb) -> None:
-        if exc_type is not None:
-            self._abort()
-        else:
-            self._finalize()
-
-
-class DirectoryExportSink(ExportSink):
-    """Writes loose files into a target directory, with incremental sync.
-
-    Owns the snapshot/skip/compare/prune machinery that used to live in the
-    command (``files_in_export_dir``, ``check_and_copy``, ``check_and_write_json``,
-    and the ``--delete`` pass).
-    """
-
-    def __init__(
-        self,
-        target: Path,
-        *,
-        compare_checksums: bool,
-        compare_json: bool,
-        delete: bool,
-    ) -> None:
-        self._target = target.resolve()
-        self._compare_checksums = compare_checksums
-        self._compare_json = compare_json
-        self._delete = delete
-        self._snapshot: set[Path] = set()
-        self._stream_open = False
-
-    def _open(self) -> None:
-        for x in self._target.glob("**/*"):
-            if x.is_file():
-                self._snapshot.add(x.resolve())
-
-    def add_file(
-        self,
-        source: Path,
-        arcname: str,
-        *,
-        checksum: str | None = None,
-    ) -> None:
-        target = (self._target / arcname).resolve()
-        self._snapshot.discard(target)
-        perform_copy = False
-        if target.exists():
-            source_stat = source.stat()
-            target_stat = target.stat()
-            if self._compare_checksums and checksum:
-                perform_copy = compute_checksum(target) != checksum
-            elif (
-                source_stat.st_mtime != target_stat.st_mtime
-                or source_stat.st_size != target_stat.st_size
-            ):
-                perform_copy = True
-        else:
-            perform_copy = True
-        if perform_copy:
-            target.parent.mkdir(parents=True, exist_ok=True)
-            copy_file_with_basic_stats(source, target)
-
-    @staticmethod
-    def _content_unchanged(target: Path, new_bytes: bytes) -> bool:
-        """True if ``target`` already holds byte-identical content (BLAKE2b)."""
-        return (
-            hashlib.blake2b(target.read_bytes()).hexdigest()
-            == hashlib.blake2b(new_bytes).hexdigest()
-        )
-
-    def add_json(self, content: list | dict, arcname: str) -> None:
-        target = (self._target / arcname).resolve()
-        json_str = _dumps(content)
-        perform_write = True
-        if target in self._snapshot:
-            self._snapshot.discard(target)
-            if self._compare_json and self._content_unchanged(
-                target,
-                json_str.encode("utf-8"),
-            ):
-                perform_write = False
-        if perform_write:
-            target.parent.mkdir(parents=True, exist_ok=True)
-            target.write_text(json_str, encoding="utf-8")
-
-    @contextmanager
-    def stream(self, arcname: str) -> Iterator[TextIO]:
-        if self._stream_open:
-            raise RuntimeError("A stream is already open on this sink")
-        target = (self._target / arcname).resolve()
-        tmp = target.with_suffix(target.suffix + ".tmp")
-        target.parent.mkdir(parents=True, exist_ok=True)
-        handle = tmp.open("w", encoding="utf-8")
-        self._stream_open = True
-        try:
-            yield handle
-        except BaseException:
-            handle.close()
-            tmp.unlink(missing_ok=True)
-            raise
-        else:
-            handle.close()
-            self._commit_streamed_file(target, tmp)
-        finally:
-            self._stream_open = False
-
-    def _commit_streamed_file(self, target: Path, tmp: Path) -> None:
-        if target in self._snapshot:
-            self._snapshot.discard(target)
-            if self._compare_json and self._content_unchanged(
-                target,
-                tmp.read_bytes(),
-            ):
-                tmp.unlink()
-                return
-        tmp.rename(target)
-
-    def _finalize(self) -> None:
-        if self._delete:
-            for f in self._snapshot:
-                if not f.is_relative_to(self._target):  # pragma: no cover
-                    # Defense in depth: a symlink inside the export dir can
-                    # resolve outside of it; never delete outside the target.
-                    continue
-                f.unlink()
-                delete_empty_directories(f.parent, self._target)
-
-    def _abort(self) -> None:
-        # Folder mode is in-place/incremental: streamed .tmp files are already
-        # cleaned in stream(); leave everything else intact and skip the prune.
-        return None
-
-
-class ZipExportSink(ExportSink):
-    """Writes a single zip archive, produced atomically only on success.
-
-    Builds into ``<target>/<zip_name>.zip.tmp`` and renames to ``.zip`` on clean
-    finalize. The manifest stream is spooled to a temp file in SCRATCH_DIR and
-    added as an entry at finalize (a zip entry cannot be interleaved with others).
-    """
-
-    def __init__(self, target: Path, zip_name: str, *, delete: bool = False) -> None:
-        self._target = target.resolve()
-        self._zip_path = (self._target / zip_name).with_suffix(".zip")
-        self._tmp_path = self._zip_path.with_name(self._zip_path.name + ".tmp")
-        self._delete = delete
-        self._zip: zipfile.ZipFile | None = None
-        self._dirs: set[str] = set()
-        self._pending_manifest: tuple[Path, str] | None = None
-        self._stream_open = False
-
-    def _open(self) -> None:
-        settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
-        self._zip = zipfile.ZipFile(
-            self._tmp_path,
-            "w",
-            compression=zipfile.ZIP_DEFLATED,
-            allowZip64=True,
-        )
-
-    def _ensure_dirs(self, arcname: str) -> None:
-        assert self._zip is not None
-        dir_arc = ""
-        for part in PurePosixPath(arcname).parts[:-1]:
-            dir_arc += f"{part}/"
-            if dir_arc not in self._dirs:
-                self._dirs.add(dir_arc)
-                self._zip.mkdir(dir_arc)
-
-    def add_file(
-        self,
-        source: Path,
-        arcname: str,
-        *,
-        checksum: str | None = None,
-    ) -> None:
-        assert self._zip is not None
-        self._ensure_dirs(arcname)
-        self._zip.write(source, arcname=arcname)
-
-    def add_json(self, content: list | dict, arcname: str) -> None:
-        assert self._zip is not None
-        self._ensure_dirs(arcname)
-        self._zip.writestr(arcname, _dumps(content))
-
-    @contextmanager
-    def stream(self, arcname: str) -> Iterator[TextIO]:
-        if self._stream_open:
-            raise RuntimeError("A stream is already open on this sink")
-        settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
-        fd, tmp_name = tempfile.mkstemp(
-            dir=settings.SCRATCH_DIR,
-            prefix="export-manifest-",
-            suffix=".json",
-        )
-        tmp = Path(tmp_name)
-        handle = os.fdopen(fd, "w", encoding="utf-8")
-        self._stream_open = True
-        try:
-            yield handle
-        except BaseException:
-            handle.close()
-            tmp.unlink(missing_ok=True)
-            raise
-        else:
-            handle.close()
-            self._pending_manifest = (tmp, arcname)
-        finally:
-            self._stream_open = False
-
-    def _finalize(self) -> None:
-        assert self._zip is not None
-        if self._pending_manifest is not None:
-            tmp, arcname = self._pending_manifest
-            self._ensure_dirs(arcname)
-            self._zip.write(tmp, arcname=arcname)
-            tmp.unlink(missing_ok=True)
-            self._pending_manifest = None
-        self._zip.close()
-        self._zip = None
-        if self._delete:
-            self._wipe_destination()
-        self._tmp_path.replace(self._zip_path)
-
-    def _wipe_destination(self) -> None:
-        skip = {self._zip_path.resolve(), self._tmp_path.resolve()}
-        for item in self._target.glob("*"):
-            if item.resolve() in skip:
-                continue
-            if item.is_dir():
-                shutil.rmtree(item)
-            else:
-                item.unlink()
-
-    def _abort(self) -> None:
-        if self._zip is not None:
-            self._zip.close()
-            self._zip = None
-        self._tmp_path.unlink(missing_ok=True)
-        if self._pending_manifest is not None:
-            self._pending_manifest[0].unlink(missing_ok=True)
-            self._pending_manifest = None

src/documents/filters.py

@@ -39,6 +39,7 @@ from guardian.utils import get_user_obj_perms_model
 from rest_framework import serializers
 from rest_framework.filters import BaseFilterBackend
 from rest_framework.filters import OrderingFilter
+from rest_framework_guardian.filters import ObjectPermissionsFilter
 
 from documents.models import Correspondent
 from documents.models import CustomField
@@ -50,7 +51,7 @@ from documents.models import ShareLink
 from documents.models import ShareLinkBundle
 from documents.models import StoragePath
 from documents.models import Tag
-from documents.permissions import permitted_object_ids
+from documents.permissions import permitted_document_ids
 
 if TYPE_CHECKING:
     from collections.abc import Callable
@@ -1027,35 +1028,59 @@ class PaperlessTaskFilterSet(FilterSet):
         return queryset.exclude(status__in=PaperlessTask.COMPLETE_STATUSES)
 
 
-class PermittedObjectsFilter(BaseFilterBackend):
+class ObjectOwnedOrGrantedPermissionsFilter(ObjectPermissionsFilter):
     """
-    Filters a queryset down to objects the requesting user owns, are
+    A filter backend that limits results to those where the requesting user
-    unowned, or (when ``include_granted`` is True) has an explicit
+    has read object level permissions, owns the objects, or objects without
-    user/group guardian permission on. Backed by ``permitted_object_ids``
+    an owner (for backwards compat)
-    -- a single ``id__in`` subquery, not a join -- so it can't produce
-    duplicate rows even when the base queryset already carries independent
-    joins (e.g. multi-value ``tags__id__all`` filtering), and stays
-    index-friendly at scale instead of falling back to guardian's
-    varchar-cast join.
-
-    Set ``include_granted = False`` on a subclass for endpoints that
-    intentionally only show owned/unowned objects regardless of explicit
-    shares (e.g. ``TrashView``).
     """
 
-    include_granted: bool = True
-    perm_codename: str | None = None
-
     def filter_queryset(self, request, queryset, view):
         if request.user.is_superuser:
             return queryset
-        if not self.include_granted:
+        objects_with_perms = super().filter_queryset(request, queryset, view)
-            return queryset.filter(Q(owner=request.user) | Q(owner__isnull=True))
+        objects_owned = queryset.filter(owner=request.user)
-        model = queryset.model
+        objects_unowned = queryset.filter(owner__isnull=True)
-        perm = self.perm_codename or f"view_{model._meta.model_name}"
+        return objects_with_perms | objects_owned | objects_unowned
-        return queryset.filter(
+
-            id__in=permitted_object_ids(request.user, model, perm),
+
-        )
+class DocumentPermissionsFilter(BaseFilterBackend):
+    """
+    A filter backend limiting Document results to those the requesting user
+    owns, are unowned, or has explicit (user- or group-level) view
+    permission on.
+
+    Unlike ``ObjectOwnedOrGrantedPermissionsFilter``, this does not build an
+    ``objects_with_perms | objects_owned | objects_unowned`` union of
+    querysets derived from the same base queryset. When that base queryset
+    already carries independent joins on a multi-valued relation (e.g. two
+    separate joins from ``tags__id__all`` filtering on two tags), each
+    OR-ed branch can end up pairing those joins' aliases differently,
+    letting more than one row out of the join's cross product satisfy the
+    combined WHERE -- returning the same document more than once. Filtering
+    via a single ``id__in`` against ``permitted_document_ids`` (a plain
+    subquery, not a join) sidesteps that entirely and is also cheaper than
+    guardian's join-based permission check.
+    """
+
+    def filter_queryset(self, request, queryset, view):
+        if request.user.is_superuser:
+            return queryset
+        return queryset.filter(id__in=permitted_document_ids(request.user))
+
+
+class ObjectOwnedPermissionsFilter(ObjectPermissionsFilter):
+    """
+    A filter backend that limits results to those where the requesting user
+    owns the objects or objects without an owner (for backwards compat)
+    """
+
+    def filter_queryset(self, request, queryset, view):
+        if request.user.is_superuser:
+            return queryset
+        objects_owned = queryset.filter(owner=request.user)
+        objects_unowned = queryset.filter(owner__isnull=True)
+        return objects_owned | objects_unowned
 
 
 class DocumentsOrderingFilter(OrderingFilter):

src/documents/management/commands/document_exporter.py

@@ -1,4 +1,8 @@
+import hashlib
+import json
 import os
+import shutil
+import tempfile
 from itertools import islice
 from pathlib import Path
 from typing import TYPE_CHECKING
@@ -15,6 +19,7 @@ from django.contrib.auth.models import User
 from django.contrib.contenttypes.models import ContentType
 from django.core import serializers
 from django.core.management.base import CommandError
+from django.core.serializers.json import DjangoJSONEncoder
 from django.db import transaction
 from django.utils import timezone
 from filelock import FileLock
@@ -29,10 +34,7 @@ if TYPE_CHECKING:
 if settings.AUDIT_LOG_ENABLED:
     from auditlog.models import LogEntry
 
-from documents.export.sinks import DirectoryExportSink
+from documents.file_handling import delete_empty_directories
-from documents.export.sinks import ExportSink
-from documents.export.sinks import StreamingManifestWriter
-from documents.export.sinks import ZipExportSink
 from documents.file_handling import generate_filename
 from documents.management.commands.base import PaperlessCommand
 from documents.management.commands.mixins import CryptMixin
@@ -58,7 +60,8 @@ from documents.settings import EXPORTER_ARCHIVE_NAME
 from documents.settings import EXPORTER_FILE_NAME
 from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
 from documents.settings import EXPORTER_THUMBNAIL_NAME
-from documents.utils import QuerySetStream
+from documents.utils import compute_checksum
+from documents.utils import copy_file_with_basic_stats
 from paperless import version
 from paperless.models import ApplicationConfiguration
 from paperless_mail.models import MailAccount
@@ -81,6 +84,87 @@ def serialize_queryset_batched(
         yield serializers.serialize("python", chunk)
 
 
+class StreamingManifestWriter:
+    """Incrementally writes a JSON array to a file, one record at a time.
+
+    Writes to <target>.tmp first; on close(), optionally BLAKE2b-compares
+    with the existing file (--compare-json) and renames or discards accordingly.
+    On exception, discard() deletes the tmp file and leaves the original intact.
+    """
+
+    def __init__(
+        self,
+        path: Path,
+        *,
+        compare_json: bool = False,
+        files_in_export_dir: "set[Path] | None" = None,
+    ) -> None:
+        self._path = path.resolve()
+        self._tmp_path = self._path.with_suffix(self._path.suffix + ".tmp")
+        self._compare_json = compare_json
+        self._files_in_export_dir: set[Path] = (
+            files_in_export_dir if files_in_export_dir is not None else set()
+        )
+        self._file = None
+        self._first = True
+
+    def open(self) -> None:
+        self._path.parent.mkdir(parents=True, exist_ok=True)
+        self._file = self._tmp_path.open("w", encoding="utf-8")
+        self._file.write("[")
+        self._first = True
+
+    def write_record(self, record: dict) -> None:
+        if not self._first:
+            self._file.write(",\n")
+        else:
+            self._first = False
+        self._file.write(
+            json.dumps(record, cls=DjangoJSONEncoder, indent=2, ensure_ascii=False),
+        )
+
+    def write_batch(self, records: list[dict]) -> None:
+        for record in records:
+            self.write_record(record)
+
+    def close(self) -> None:
+        if self._file is None:
+            return
+        self._file.write("\n]")
+        self._file.close()
+        self._file = None
+        self._finalize()
+
+    def discard(self) -> None:
+        if self._file is not None:
+            self._file.close()
+            self._file = None
+        if self._tmp_path.exists():
+            self._tmp_path.unlink()
+
+    def _finalize(self) -> None:
+        """Compare with existing file (if --compare-json) then rename or discard tmp."""
+        if self._path in self._files_in_export_dir:
+            self._files_in_export_dir.remove(self._path)
+            if self._compare_json:
+                existing_hash = hashlib.blake2b(self._path.read_bytes()).hexdigest()
+                new_hash = hashlib.blake2b(self._tmp_path.read_bytes()).hexdigest()
+                if existing_hash == new_hash:
+                    self._tmp_path.unlink()
+                    return
+        self._tmp_path.rename(self._path)
+
+    def __enter__(self) -> "StreamingManifestWriter":
+        self.open()
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb) -> None:
+        if exc_type is not None:
+            self.discard()
+        else:
+            self.close()
+
+
 class Command(CryptMixin, PaperlessCommand):
     help = (
         "Decrypt and rename all files in our collection into a given target "
@@ -230,13 +314,20 @@ class Command(CryptMixin, PaperlessCommand):
         self.passphrase: str | None = options.get("passphrase")
         self.batch_size: int = options["batch_size"]
 
+        self.files_in_export_dir: set[Path] = set()
         self.exported_files: set[str] = set()
 
-        if self.zip_export and (self.compare_checksums or self.compare_json):
+        # If zipping, save the original target for later and
-            raise CommandError(
+        # get a temporary directory for the target instead
-                "--compare-checksums and --compare-json have no effect when "
+        temp_dir = None
-                "used with --zip",
+        self.original_target = self.target
+        if self.zip_export:
+            settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
+            temp_dir = tempfile.TemporaryDirectory(
+                dir=settings.SCRATCH_DIR,
+                prefix="paperless-export",
             )
+            self.target = Path(temp_dir.name).resolve()
 
         if not self.target.exists():
             raise CommandError("That path doesn't exist")
@@ -247,28 +338,33 @@ class Command(CryptMixin, PaperlessCommand):
         if not os.access(self.target, os.W_OK):
             raise CommandError("That path doesn't appear to be writable")
 
-        sink: ExportSink
+        try:
-        if self.zip_export:
+            # Prevent any ongoing changes in the documents
-            sink = ZipExportSink(
+            with FileLock(settings.MEDIA_LOCK):
-                self.target,
+                self.dump()
-                options["zip_name"],
-                delete=self.delete,
-            )
-        else:
-            sink = DirectoryExportSink(
-                self.target,
-                compare_checksums=self.compare_checksums,
-                compare_json=self.compare_json,
-                delete=self.delete,
-            )
 
-        # Prevent any ongoing changes in the documents while exporting
+                # We've written everything to the temporary directory in this case,
-        with FileLock(settings.MEDIA_LOCK), sink:
+                # now make an archive in the original target, with all files stored
-            self.dump(sink)
+                if self.zip_export and temp_dir is not None:
+                    shutil.make_archive(
+                        self.original_target / options["zip_name"],
+                        format="zip",
+                        root_dir=temp_dir.name,
+                    )
 
-    def dump(self, sink: ExportSink) -> None:
+        finally:
-        # 1. Create manifest, containing all correspondents, types, tags, storage
+            # Always cleanup the temporary directory, if one was created
-        #    paths, note, documents and ui_settings
+            if self.zip_export and temp_dir is not None:
+                temp_dir.cleanup()
+
+    def dump(self) -> None:
+        # 1. Take a snapshot of what files exist in the current export folder
+        for x in self.target.glob("**/*"):
+            if x.is_file():
+                self.files_in_export_dir.add(x.resolve())
+
+        # 2. Create manifest, containing all correspondents, types, tags, storage paths
+        # note, documents and ui_settings
         _excluded_usernames = ["consumer", "AnonymousUser"]
         manifest_key_to_object_query: dict[str, QuerySet[Any]] = {
             "correspondents": Correspondent.objects.all(),
@@ -331,9 +427,13 @@ class Command(CryptMixin, PaperlessCommand):
 
         document_manifest: list[dict] = []
         share_link_bundle_manifest: list[dict] = []
+        manifest_path = (self.target / "manifest.json").resolve()
 
-        with sink.stream("manifest.json") as handle:
+        with StreamingManifestWriter(
-            writer = StreamingManifestWriter(handle)
+            manifest_path,
+            compare_json=self.compare_json,
+            files_in_export_dir=self.files_in_export_dir,
+        ) as writer:
             with transaction.atomic():
                 for key, qs in manifest_key_to_object_query.items():
                     if key == "documents":
@@ -369,6 +469,9 @@ class Command(CryptMixin, PaperlessCommand):
                                 self._encrypt_record_inline(record)
                             writer.write_batch(batch)
 
+            document_map: dict[int, Document] = {
+                d.pk: d for d in Document.global_objects.order_by("id")
+            }
             share_link_bundle_map: dict[int, ShareLinkBundle] = {
                 b.pk: b
                 for b in ShareLinkBundle.objects.order_by("id").prefetch_related(
@@ -376,72 +479,84 @@ class Command(CryptMixin, PaperlessCommand):
                 )
             }
 
-            # 2. Export files from each document
+            # 3. Export files from each document
-            # document_manifest and this stream are both ordered by id from the
+            for index, document_dict in enumerate(
-            # same underlying rows, so zip them in lockstep instead of building
+                self.track(
-            # a dict of every Document instance up front (QuerySetStream keeps
+                    document_manifest,
-            # only one batch of documents resident at a time).
+                    description="Exporting documents...",
-            documents_stream = QuerySetStream(
+                    total=len(document_manifest),
-                Document.global_objects.order_by("id"),
+                ),
-                chunk_size=self.batch_size,
-            )
-            for document_dict, document in self.track(
-                zip(document_manifest, documents_stream, strict=True),
-                description="Exporting documents...",
-                total=len(document_manifest),
             ):
-                # Both document_manifest and documents_stream come from the same
+                document = document_map[document_dict["pk"]]
-                # Document.global_objects.order_by("id") query, taken while
-                # MEDIA_LOCK is held, so this should be unreachable -- it guards
-                # against silent data corruption if that invariant ever breaks.
-                if document.pk != document_dict["pk"]:  # pragma: no cover
-                    raise CommandError(
-                        "Document export ordering mismatch: expected "
-                        f"pk={document_dict['pk']}, got pk={document.pk}. "
-                        "Documents may have changed during export.",
-                    )
 
-                # generate a unique filename, then the arcnames for its files
+                # 3.1. generate a unique filename
                 base_name = self.generate_base_name(document)
-                original_arc, thumbnail_arc, archive_arc = (
+
+                # 3.2. write filenames into manifest
+                original_target, thumbnail_target, archive_target = (
                     self.generate_document_targets(document, base_name, document_dict)
                 )
 
+                # 3.3. write files to target folder
                 if not self.data_only:
                     self.copy_document_files(
                         document,
-                        sink,
+                        original_target,
-                        original_arc,
+                        thumbnail_target,
-                        thumbnail_arc,
+                        archive_target,
-                        archive_arc,
                     )
 
                 if self.split_manifest:
-                    self._write_split_manifest(sink, document_dict, document, base_name)
+                    self._write_split_manifest(document_dict, document, base_name)
                 else:
                     writer.write_record(document_dict)
 
             for bundle_dict in share_link_bundle_manifest:
                 bundle = share_link_bundle_map[bundle_dict["pk"]]
-                bundle_arc = self.generate_share_link_bundle_target(
+
+                bundle_target = self.generate_share_link_bundle_target(
                     bundle,
                     bundle_dict,
                 )
-                if not self.data_only and bundle_arc is not None:
+
-                    self.copy_share_link_bundle_file(bundle, sink, bundle_arc)
+                if not self.data_only and bundle_target is not None:
+                    self.copy_share_link_bundle_file(bundle, bundle_target)
+
                 writer.write_record(bundle_dict)
 
-            writer.close()
+        # 4.2 write version information to target folder
-
+        extra_metadata_path = (self.target / "metadata.json").resolve()
-        # 3. Write version (and crypto params) to metadata.json
-        # Django stores most crypto values in the field itself; we store
-        # them once here for the whole export
         metadata: dict[str, str | int | dict[str, str | int]] = {
             "version": version.__full_version_str__,
         }
+
+        # 4.2.1 If needed, write the crypto values into the metadata
+        # Django stores most of these in the field itself, we store them once here
         if self.passphrase:
             metadata.update(self.get_crypt_params())
-        sink.add_json(metadata, "metadata.json")
+
+        self.check_and_write_json(
+            metadata,
+            extra_metadata_path,
+        )
+
+        if self.delete:
+            # 5. Remove files which we did not explicitly export in this run
+            if not self.zip_export:
+                for f in self.files_in_export_dir:
+                    f.unlink()
+
+                    delete_empty_directories(
+                        f.parent,
+                        self.target,
+                    )
+            else:
+                # 5. Remove anything in the original location (before moving the zip)
+                for item in self.original_target.glob("*"):
+                    if item.is_dir():
+                        shutil.rmtree(item)
+                    else:
+                        item.unlink()
 
     def generate_base_name(self, document: Document) -> Path:
         """
@@ -469,69 +584,73 @@ class Command(CryptMixin, PaperlessCommand):
         document: Document,
         base_name: Path,
         document_dict: dict,
-    ) -> tuple[str, str | None, str | None]:
+    ) -> tuple[Path, Path | None, Path | None]:
         """
-        Generates the relative POSIX arcnames for a document's original, thumbnail
+        Generates the targets for a given document, including the original file, archive file and thumbnail (depending on settings).
-        and archive files (depending on settings), and records them in the manifest.
         """
         original_name = base_name
         if self.use_folder_prefix:
             original_name = Path("originals") / original_name
-        original_arc = original_name.as_posix()
+        original_target = (self.target / original_name).resolve()
-        document_dict[EXPORTER_FILE_NAME] = original_arc
+        document_dict[EXPORTER_FILE_NAME] = str(original_name)
 
         if not self.no_thumbnail:
             thumbnail_name = base_name.parent / (base_name.stem + "-thumbnail.webp")
             if self.use_folder_prefix:
                 thumbnail_name = Path("thumbnails") / thumbnail_name
-            thumbnail_arc = thumbnail_name.as_posix()
+            thumbnail_target = (self.target / thumbnail_name).resolve()
-            document_dict[EXPORTER_THUMBNAIL_NAME] = thumbnail_arc
+            document_dict[EXPORTER_THUMBNAIL_NAME] = str(thumbnail_name)
         else:
-            thumbnail_arc = None
+            thumbnail_target = None
 
         if not self.no_archive and document.has_archive_version:
             archive_name = base_name.parent / (base_name.stem + "-archive.pdf")
             if self.use_folder_prefix:
                 archive_name = Path("archive") / archive_name
-            archive_arc = archive_name.as_posix()
+            archive_target = (self.target / archive_name).resolve()
-            document_dict[EXPORTER_ARCHIVE_NAME] = archive_arc
+            document_dict[EXPORTER_ARCHIVE_NAME] = str(archive_name)
         else:
-            archive_arc = None
+            archive_target = None
 
-        return original_arc, thumbnail_arc, archive_arc
+        return original_target, thumbnail_target, archive_target
 
     def copy_document_files(
         self,
         document: Document,
-        sink: ExportSink,
+        original_target: Path,
-        original_arc: str,
+        thumbnail_target: Path | None,
-        thumbnail_arc: str | None,
+        archive_target: Path | None,
-        archive_arc: str | None,
     ) -> None:
         """
-        Hands the document's files to the sink (original, thumbnail, archive).
+        Copies files from the document storage location to the specified target location.
+
+        If the document is encrypted, the files are decrypted before copying them to the target location.
         """
-        sink.add_file(document.source_path, original_arc, checksum=document.checksum)
+        self.check_and_copy(
+            document.source_path,
+            document.checksum,
+            original_target,
+        )
 
-        if thumbnail_arc:
+        if thumbnail_target:
-            sink.add_file(document.thumbnail_path, thumbnail_arc)
+            self.check_and_copy(document.thumbnail_path, None, thumbnail_target)
 
-        if archive_arc:
+        if archive_target:
             if TYPE_CHECKING:
                 assert isinstance(document.archive_path, Path)
-            sink.add_file(
+            self.check_and_copy(
                 document.archive_path,
-                archive_arc,
+                document.archive_checksum,
-                checksum=document.archive_checksum,
+                archive_target,
             )
 
     def generate_share_link_bundle_target(
         self,
         bundle: ShareLinkBundle,
         bundle_dict: dict,
-    ) -> str | None:
+    ) -> Path | None:
         """
-        Generates the relative POSIX arcname for a share link bundle file, if any.
+        Generates the export target for a share link bundle file, when present.
         """
         if not bundle.file_path:
             return None
@@ -547,22 +666,25 @@ class Command(CryptMixin, PaperlessCommand):
         bundle_dict["fields"]["file_path"] = portable_bundle_path.as_posix()
         bundle_dict[EXPORTER_SHARE_LINK_BUNDLE_NAME] = export_bundle_path.as_posix()
 
-        return export_bundle_path.as_posix()
+        return (self.target / export_bundle_path).resolve()
 
     def copy_share_link_bundle_file(
         self,
         bundle: ShareLinkBundle,
-        sink: ExportSink,
+        bundle_target: Path,
-        bundle_arc: str,
     ) -> None:
         """
-        Hands a share link bundle ZIP to the sink.
+        Copies a share link bundle ZIP into the export directory.
         """
         bundle_source_path = bundle.absolute_file_path
         if bundle_source_path is None:
             raise FileNotFoundError(f"Share link bundle {bundle.pk} has no file path")
 
-        sink.add_file(bundle_source_path, bundle_arc)
+        self.check_and_copy(
+            bundle_source_path,
+            None,
+            bundle_target,
+        )
 
     def _encrypt_record_inline(self, record: dict) -> None:
         """Encrypt sensitive fields in a single record, if passphrase is set."""
@@ -578,7 +700,6 @@ class Command(CryptMixin, PaperlessCommand):
 
     def _write_split_manifest(
         self,
-        sink: ExportSink,
         document_dict: dict,
         document: Document,
         base_name: Path,
@@ -600,4 +721,81 @@ class Command(CryptMixin, PaperlessCommand):
         manifest_name = base_name.with_name(f"{base_name.stem}-manifest.json")
         if self.use_folder_prefix:
             manifest_name = Path("json") / manifest_name
-        sink.add_json(content, manifest_name.as_posix())
+        manifest_name = (self.target / manifest_name).resolve()
+        manifest_name.parent.mkdir(parents=True, exist_ok=True)
+        self.check_and_write_json(content, manifest_name)
+
+    def check_and_write_json(
+        self,
+        content: list[dict] | dict,
+        target: Path,
+    ) -> None:
+        """
+        Writes the source content to the target json file.
+        If --compare-json arg was used, don't write to target file if
+        the file exists and checksum is identical to content checksum.
+        This preserves the file timestamps when no changes are made.
+        """
+
+        target = target.resolve()
+        perform_write = True
+        if target in self.files_in_export_dir:
+            self.files_in_export_dir.remove(target)
+            if self.compare_json:
+                target_checksum = hashlib.blake2b(target.read_bytes()).hexdigest()
+                src_str = json.dumps(
+                    content,
+                    cls=DjangoJSONEncoder,
+                    indent=2,
+                    ensure_ascii=False,
+                )
+                src_checksum = hashlib.blake2b(src_str.encode("utf-8")).hexdigest()
+                if src_checksum == target_checksum:
+                    perform_write = False
+
+        if perform_write:
+            target.write_text(
+                json.dumps(
+                    content,
+                    cls=DjangoJSONEncoder,
+                    indent=2,
+                    ensure_ascii=False,
+                ),
+                encoding="utf-8",
+            )
+
+    def check_and_copy(
+        self,
+        source: Path,
+        source_checksum: str | None,
+        target: Path,
+    ) -> None:
+        """
+        Copies the source to the target, if target doesn't exist or the target doesn't seem to match
+        the source attributes
+        """
+
+        target = target.resolve()
+        if target in self.files_in_export_dir:
+            self.files_in_export_dir.remove(target)
+
+        perform_copy = False
+
+        if target.exists():
+            source_stat = source.stat()
+            target_stat = target.stat()
+            if self.compare_checksums and source_checksum:
+                target_checksum = compute_checksum(target)
+                perform_copy = target_checksum != source_checksum
+            elif (
+                source_stat.st_mtime != target_stat.st_mtime
+                or source_stat.st_size != target_stat.st_size
+            ):
+                perform_copy = True
+        else:
+            # Copy if it does not exist
+            perform_copy = True
+
+        if perform_copy:
+            target.parent.mkdir(parents=True, exist_ok=True)
+            copy_file_with_basic_stats(source, target)

src/documents/matching.py

@@ -19,7 +19,7 @@ from documents.models import StoragePath
 from documents.models import Tag
 from documents.models import Workflow
 from documents.models import WorkflowTrigger
-from documents.permissions import permitted_object_ids
+from documents.permissions import get_objects_for_user_owner_aware
 from documents.regex import safe_regex_search
 
 if TYPE_CHECKING:
@@ -55,8 +55,10 @@ def match_correspondents(document: Document, classifier: DocumentClassifier, use
         user = document.owner
 
     if user is not None:
-        correspondents = Correspondent.objects.filter(
+        correspondents = get_objects_for_user_owner_aware(
-            id__in=permitted_object_ids(user, Correspondent, "view_correspondent"),
+            user,
+            "documents.view_correspondent",
+            Correspondent,
         )
     else:
         correspondents = Correspondent.objects.all()
@@ -84,8 +86,10 @@ def match_document_types(document: Document, classifier: DocumentClassifier, use
         user = document.owner
 
     if user is not None:
-        document_types = DocumentType.objects.filter(
+        document_types = get_objects_for_user_owner_aware(
-            id__in=permitted_object_ids(user, DocumentType, "view_documenttype"),
+            user,
+            "documents.view_documenttype",
+            DocumentType,
         )
     else:
         document_types = DocumentType.objects.all()
@@ -112,9 +116,7 @@ def match_tags(document: Document, classifier: DocumentClassifier, user=None):
         user = document.owner
 
     if user is not None:
-        tags = Tag.objects.filter(
+        tags = get_objects_for_user_owner_aware(user, "documents.view_tag", Tag)
-            id__in=permitted_object_ids(user, Tag, "view_tag"),
-        )
     else:
         tags = Tag.objects.all()
 
@@ -143,8 +145,10 @@ def match_storage_paths(document: Document, classifier: DocumentClassifier, user
         user = document.owner
 
     if user is not None:
-        storage_paths = StoragePath.objects.filter(
+        storage_paths = get_objects_for_user_owner_aware(
-            id__in=permitted_object_ids(user, StoragePath, "view_storagepath"),
+            user,
+            "documents.view_storagepath",
+            StoragePath,
         )
     else:
         storage_paths = StoragePath.objects.all()

src/documents/permissions.py

@@ -7,7 +7,6 @@ from django.contrib.contenttypes.models import ContentType
 from django.db.models import Case
 from django.db.models import Count
 from django.db.models import IntegerField
-from django.db.models import Model
 from django.db.models import Q
 from django.db.models import QuerySet
 from django.db.models import Value
@@ -164,32 +163,30 @@ def set_permissions_for_object(
                         )
 
 
-def permitted_object_ids(
+def permitted_document_ids(
-    user: User | None,
+    user,
-    model: type[Model],
-    perm: str,
     *,
+    perm: str = "view_document",
     include_deleted: bool = False,
-) -> QuerySet[int]:
+):
     """
-    Generic version of ``permitted_document_ids`` for any model with an
+    Return a queryset of document IDs the user has ``perm`` on (default
-    ``owner`` field and guardian object-level permissions. ``include_deleted``
+    ``"view_document"``). By default limited to non-deleted documents; pass
-    only has an effect for models exposing a ``global_objects``/``deleted_at``
+    ``include_deleted=True`` for callers that need to check permission on
-    soft-delete pattern (currently only ``Document``); for every other model
+    soft-deleted documents (e.g. trash restore). This intentionally avoids
-    it is accepted but has no effect, since those models have no soft-delete
+    ``get_objects_for_user`` to keep the subquery small and index-friendly.
-    concept.
     """
-    has_soft_delete = hasattr(model, "global_objects")
+
-    manager = (
+    manager = Document.global_objects if include_deleted else Document.objects
-        model.global_objects if include_deleted and has_soft_delete else model.objects
+    base_docs = manager.all()
-    )
+    base_docs = base_docs.only("id", "owner")
-    base_qs = manager.all().only("id", "owner")
 
     if user is None or not getattr(user, "is_authenticated", False):
-        return base_qs.filter(owner__isnull=True).values_list("id", flat=True)
+        # Just Anonymous user e.g. for drf-spectacular
+        return base_docs.filter(owner__isnull=True).values_list("id", flat=True)
 
     if getattr(user, "is_superuser", False):
-        return base_qs.values_list("id", flat=True)
+        return base_docs.values_list("id", flat=True)
 
     # Guardian's UserObjectPermission/GroupObjectPermission always store a bare
     # codename, but has_perm()-style callers commonly pass the qualified
@@ -197,46 +194,31 @@ def permitted_object_ids(
     # codename, so just drop any prefix rather than silently under-permitting.
     perm = perm.rsplit(".", 1)[-1]
 
-    content_type = ContentType.objects.get_for_model(model)
+    document_ct = ContentType.objects.get_for_model(Document)
     perm_filter = {
         "permission__codename": perm,
-        "permission__content_type": content_type,
+        "permission__content_type": document_ct,
     }
 
-    user_perm_ids = (
+    user_perm_docs = (
         UserObjectPermission.objects.filter(user=user, **perm_filter)
         .annotate(object_pk_int=Cast("object_pk", IntegerField()))
         .values_list("object_pk_int", flat=True)
     )
-    group_perm_ids = (
+
+    group_perm_docs = (
         GroupObjectPermission.objects.filter(group__user=user, **perm_filter)
         .annotate(object_pk_int=Cast("object_pk", IntegerField()))
         .values_list("object_pk_int", flat=True)
     )
-    permitted_ids = user_perm_ids.union(group_perm_ids)
 
-    return base_qs.filter(
+    permitted_documents = user_perm_docs.union(group_perm_docs)
-        Q(owner=user) | Q(owner__isnull=True) | Q(id__in=permitted_ids),
+
+    return base_docs.filter(
+        Q(owner=user) | Q(owner__isnull=True) | Q(id__in=permitted_documents),
     ).values_list("id", flat=True)
 
 
-def permitted_document_ids(
-    user: User | None,
-    *,
-    perm: str = "view_document",
-    include_deleted: bool = False,
-) -> QuerySet[int]:
-    """
-    Document-specific convenience wrapper around ``permitted_object_ids``.
-    Return a queryset of document IDs the user has ``perm`` on (default
-    ``"view_document"``). By default limited to non-deleted documents; pass
-    ``include_deleted=True`` for callers that need to check permission on
-    soft-deleted documents (e.g. trash restore). This intentionally avoids
-    ``get_objects_for_user`` to keep the subquery small and index-friendly.
-    """
-    return permitted_object_ids(user, Document, perm, include_deleted=include_deleted)
-
-
 def get_document_count_filter_for_user(user, related_name: str = "documents"):
     """
     Return the Q object used to filter document counts for the given user.
@@ -359,13 +341,6 @@ def get_objects_for_user_owner_aware(
     """
     Returns objects the user owns, are unowned, or has explicit perms.
     When include_deleted is True, soft-deleted items are also included.
-
-    Legacy slow path (guardian-backed, O(n) style permission resolution).
-    Most queryset-filtering call sites have migrated onto
-    ``PermittedObjectsFilter``/``permitted_object_ids()``, but this function
-    is kept because production callers still remain. Several callers remain
-    across ``documents/``, ``paperless_mail/``, and ``paperless_ai/`` --
-    grep for this function name before removing it.
     """
     manager = (
         Model.global_objects
@@ -385,15 +360,6 @@ def get_objects_for_user_owner_aware(
 
 
 def has_perms_owner_aware(user, perms, obj):
-    """
-    Legacy slow path (guardian-backed) single-object permission check.
-
-    The queryset-filtering side of this migrated onto
-    ``PermittedObjectsFilter``/``permitted_object_ids()``, but this
-    single-object check still has many production callers. Several callers
-    remain across ``documents/``, ``paperless_mail/``, and ``paperless_ai/``
-    -- grep for this function name before removing it.
-    """
     checker = ObjectPermissionChecker(user)
     return obj.owner is None or obj.owner == user or checker.has_perm(perms, obj)
 

src/documents/templates/index.html

@@ -70,7 +70,8 @@
 			]
 		</script>
 	</pngx-root>
-	<script src="{% static polyfills_js %}" type="module"></script>
+	<script src="{% static runtime_js %}" defer></script>
-	<script src="{% static main_js %}" type="module"></script>
+	<script src="{% static polyfills_js %}" defer></script>
+	<script src="{% static main_js %}" defer></script>
 </body>
 </html>

src/documents/tests/export/test_sinks.py

@@ -1,327 +0,0 @@
-import io
-import json
-import os
-import zipfile
-from pathlib import Path
-
-import pytest
-from pytest_django.fixtures import SettingsWrapper
-
-from documents.export.sinks import DirectoryExportSink
-from documents.export.sinks import ExportSink
-from documents.export.sinks import StreamingManifestWriter
-from documents.export.sinks import ZipExportSink
-from documents.export.sinks import _dumps
-
-
-@pytest.fixture()
-def source_file(tmp_path: Path) -> Path:
-    src: Path = tmp_path / "src" / "doc.pdf"
-    src.parent.mkdir(parents=True)
-    src.write_bytes(b"PDF-CONTENT")
-    return src
-
-
-class TestDumps:
-    def test_dumps_is_indented_unicode_json(self) -> None:
-        result: str = _dumps({"a": "é", "b": 1})
-        assert '"é"' in result  # ensure_ascii=False keeps unicode literal
-        assert "\n" in result  # indent=2 produces newlines
-        assert json.loads(result) == {"a": "é", "b": 1}
-
-
-class TestStreamingManifestWriter:
-    def test_writes_json_array_of_records(self) -> None:
-        handle: io.StringIO = io.StringIO()
-        writer: StreamingManifestWriter = StreamingManifestWriter(handle)
-        writer.write_batch([{"pk": 1}, {"pk": 2}])
-        writer.write_record({"pk": 3})
-        writer.close()
-        assert json.loads(handle.getvalue()) == [{"pk": 1}, {"pk": 2}, {"pk": 3}]
-
-    def test_empty_manifest_is_valid_empty_array(self) -> None:
-        handle: io.StringIO = io.StringIO()
-        writer: StreamingManifestWriter = StreamingManifestWriter(handle)
-        writer.close()
-        assert json.loads(handle.getvalue()) == []
-
-
-class TestDirectoryExportSink:
-    def test_add_file_copies_to_relative_arcname(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with DirectoryExportSink(
-            target,
-            compare_checksums=False,
-            compare_json=False,
-            delete=False,
-        ) as sink:
-            sink.add_file(source_file, "originals/doc.pdf")
-        assert (target / "originals" / "doc.pdf").read_bytes() == b"PDF-CONTENT"
-
-    def test_add_json_writes_file(self, tmp_path: Path) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with DirectoryExportSink(
-            target,
-            compare_checksums=False,
-            compare_json=False,
-            delete=False,
-        ) as sink:
-            sink.add_json({"version": "x"}, "metadata.json")
-        assert json.loads((target / "metadata.json").read_text()) == {"version": "x"}
-
-    def test_stream_writes_manifest(self, tmp_path: Path) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with DirectoryExportSink(
-            target,
-            compare_checksums=False,
-            compare_json=False,
-            delete=False,
-        ) as sink:
-            with sink.stream("manifest.json") as handle:
-                writer: StreamingManifestWriter = StreamingManifestWriter(handle)
-                writer.write_record({"pk": 1})
-                writer.close()
-        assert json.loads((target / "manifest.json").read_text()) == [{"pk": 1}]
-
-    def test_add_file_skips_when_size_and_mtime_match(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        # Pre-existing target with identical size+mtime but DIFFERENT content:
-        # if add_file skips (no compare-checksums), the old content survives.
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        existing: Path = target / "originals" / "doc.pdf"
-        existing.parent.mkdir(parents=True)
-        # Same byte length as the source but different content + matching mtime,
-        # so a size/mtime comparison treats it as unchanged and skips the copy.
-        existing.write_bytes(b"X" * len(b"PDF-CONTENT"))
-        stat = source_file.stat()
-        os.utime(existing, (stat.st_atime, stat.st_mtime))
-        with DirectoryExportSink(
-            target,
-            compare_checksums=False,
-            compare_json=False,
-            delete=False,
-        ) as sink:
-            sink.add_file(source_file, "originals/doc.pdf", checksum="abc")
-        assert existing.read_bytes() == b"X" * len(b"PDF-CONTENT")  # skipped
-
-    def test_add_file_recopies_when_compare_checksums_differ(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        existing: Path = target / "originals" / "doc.pdf"
-        existing.parent.mkdir(parents=True)
-        existing.write_bytes(b"X" * len(b"PDF-CONTENT"))
-        stat = source_file.stat()
-        os.utime(existing, (stat.st_atime, stat.st_mtime))
-        with DirectoryExportSink(
-            target,
-            compare_checksums=True,
-            compare_json=False,
-            delete=False,
-        ) as sink:
-            # wrong checksum forces recopy despite matching size/mtime
-            sink.add_file(source_file, "originals/doc.pdf", checksum="not-the-real-sum")
-        assert existing.read_bytes() == b"PDF-CONTENT"  # recopied
-
-    def test_delete_prunes_unwritten_snapshot_files(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        stale: Path = target / "stale.pdf"
-        stale.write_bytes(b"STALE")
-        with DirectoryExportSink(
-            target,
-            compare_checksums=False,
-            compare_json=False,
-            delete=True,
-        ) as sink:
-            sink.add_file(source_file, "originals/doc.pdf")
-        assert not stale.exists()
-        assert (target / "originals" / "doc.pdf").exists()
-
-    def test_no_delete_keeps_unwritten_files(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        stale: Path = target / "stale.pdf"
-        stale.write_bytes(b"STALE")
-        with DirectoryExportSink(
-            target,
-            compare_checksums=False,
-            compare_json=False,
-            delete=False,
-        ) as sink:
-            sink.add_file(source_file, "originals/doc.pdf")
-        assert stale.exists()
-
-
-class TestZipExportSink:
-    def test_round_trip_files_json_and_stream(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with ZipExportSink(target, "export", delete=False) as sink:
-            sink.add_file(source_file, "originals/doc.pdf")
-            sink.add_json({"version": "x"}, "metadata.json")
-            with sink.stream("manifest.json") as handle:
-                writer = StreamingManifestWriter(handle)
-                writer.write_record({"pk": 1})
-                writer.close()
-        zip_path: Path = target / "export.zip"
-        assert zip_path.exists()
-        assert not (target / "export.zip.tmp").exists()
-        with zipfile.ZipFile(zip_path) as zf:
-            names = set(zf.namelist())
-            assert {"originals/doc.pdf", "metadata.json", "manifest.json"} <= names
-            assert zf.read("originals/doc.pdf") == b"PDF-CONTENT"
-            assert json.loads(zf.read("manifest.json")) == [{"pk": 1}]
-
-    def test_nested_arcname_emits_directory_marker(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with ZipExportSink(target, "export", delete=False) as sink:
-            sink.add_file(source_file, "originals/doc.pdf")
-        with zipfile.ZipFile(target / "export.zip") as zf:
-            assert "originals/" in zf.namelist()
-
-    def test_flat_arcname_has_no_directory_markers(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with ZipExportSink(target, "export", delete=False) as sink:
-            sink.add_file(source_file, "doc.pdf")
-        with zipfile.ZipFile(target / "export.zip") as zf:
-            assert all(not n.endswith("/") for n in zf.namelist())
-
-    def test_exception_leaves_no_zip_and_no_tmp(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with pytest.raises(RuntimeError):
-            with ZipExportSink(target, "export", delete=False) as sink:
-                sink.add_file(source_file, "doc.pdf")
-                raise RuntimeError("boom")
-        assert not (target / "export.zip").exists()
-        assert not (target / "export.zip.tmp").exists()
-
-    def test_exception_inside_stream_cleans_up_manifest_tmp(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-        settings: SettingsWrapper,
-    ) -> None:
-        scratch_dir = tmp_path / "scratch"
-        settings.SCRATCH_DIR = scratch_dir
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with pytest.raises(RuntimeError):
-            with ZipExportSink(target, "export", delete=False) as sink:
-                sink.add_file(source_file, "doc.pdf")
-                with sink.stream("manifest.json") as handle:
-                    handle.write("[")
-                    raise RuntimeError("boom")
-        assert list(scratch_dir.glob("export-manifest-*")) == []
-        assert not (target / "export.zip").exists()
-        assert not (target / "export.zip.tmp").exists()
-
-    def test_abort_after_manifest_written_cleans_up_pending_tmp(
-        self,
-        tmp_path: Path,
-        settings: SettingsWrapper,
-    ) -> None:
-        scratch_dir = tmp_path / "scratch"
-        settings.SCRATCH_DIR = scratch_dir
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        with pytest.raises(RuntimeError):
-            with ZipExportSink(target, "export", delete=False) as sink:
-                with sink.stream("manifest.json") as handle:
-                    handle.write("[]")
-                raise RuntimeError("boom")
-        assert list(scratch_dir.glob("export-manifest-*")) == []
-        assert not (target / "export.zip").exists()
-
-    def test_delete_wipes_destination_on_success(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        (target / "preexisting.txt").write_text("old")
-        (target / "olddir").mkdir()
-        with ZipExportSink(target, "export", delete=True) as sink:
-            sink.add_file(source_file, "doc.pdf")
-        assert (target / "export.zip").exists()
-        assert not (target / "preexisting.txt").exists()
-        assert not (target / "olddir").exists()
-
-    def test_abort_with_delete_does_not_wipe_destination(
-        self,
-        tmp_path: Path,
-        source_file: Path,
-    ) -> None:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        (target / "preexisting.txt").write_text("old")
-        with pytest.raises(RuntimeError):
-            with ZipExportSink(target, "export", delete=True) as sink:
-                sink.add_file(source_file, "doc.pdf")
-                raise RuntimeError("boom")
-        assert (target / "preexisting.txt").exists()
-        assert not (target / "export.zip").exists()
-
-
-class TestStreamContract:
-    @pytest.fixture(params=["dir", "zip"])
-    def sink(self, request: pytest.FixtureRequest, tmp_path: Path) -> ExportSink:
-        target: Path = tmp_path / "out"
-        target.mkdir()
-        if request.param == "dir":
-            return DirectoryExportSink(
-                target,
-                compare_checksums=False,
-                compare_json=False,
-                delete=False,
-            )
-        return ZipExportSink(target, "export", delete=False)
-
-    def test_second_concurrent_stream_is_rejected(self, sink: ExportSink) -> None:
-        with sink:
-            with sink.stream("manifest.json"):
-                with pytest.raises(RuntimeError, match="already open"):
-                    with sink.stream("other.json"):
-                        pass

src/documents/tests/test_api_search.py

@@ -1057,52 +1057,33 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
         THEN:
             - The similar documents are returned from the API request
         """
-        # Distinct created/added/modified dates: documents sharing a timestamp
+        # Distinct created/added dates: documents created at the same instant
-        # term (down to the second) would be matched on it by more_like_this
+        # share a timestamp term, and more_like_this (which cannot be scoped to
-        # (which cannot be scoped to content fields), surfacing unrelated
+        # content fields) would then match on it, surfacing unrelated documents.
-        # documents. `modified` is auto_now, so it can't be set via factory
+        d1 = DocumentFactory(
-        # kwargs like created/added - freeze time per document instead so all
+            title="invoice",
-        # three date fields land on distinct seconds.
+            content="the thing i bought at a shop and paid with bank account",
-        with time_machine.travel(
+            created=datetime.date(2018, 1, 1),
-            timezone.make_aware(datetime.datetime(2018, 1, 1)),
+            added=timezone.make_aware(datetime.datetime(2018, 1, 1)),
-            tick=False,
+        )
-        ):
+        d2 = DocumentFactory(
-            d1 = DocumentFactory(
+            title="bank statement 1",
-                title="invoice",
+            content="things i paid for in august",
-                content="the thing i bought at a shop and paid with bank account",
+            created=datetime.date(2019, 3, 4),
-                created=datetime.date(2018, 1, 1),
+            added=timezone.make_aware(datetime.datetime(2019, 3, 4)),
-                added=timezone.make_aware(datetime.datetime(2018, 1, 1)),
+        )
-            )
+        d3 = DocumentFactory(
-        with time_machine.travel(
+            title="bank statement 3",
-            timezone.make_aware(datetime.datetime(2019, 3, 4)),
+            content="things i paid for in september",
-            tick=False,
+            created=datetime.date(2020, 7, 9),
-        ):
+            added=timezone.make_aware(datetime.datetime(2020, 7, 9)),
-            d2 = DocumentFactory(
+        )
-                title="bank statement 1",
+        d4 = DocumentFactory(
-                content="things i paid for in august",
+            title="Quarterly Report",
-                created=datetime.date(2019, 3, 4),
+            content="quarterly revenue profit margin earnings growth",
-                added=timezone.make_aware(datetime.datetime(2019, 3, 4)),
+            created=datetime.date(2021, 11, 30),
-            )
+            added=timezone.make_aware(datetime.datetime(2021, 11, 30)),
-        with time_machine.travel(
+        )
-            timezone.make_aware(datetime.datetime(2020, 7, 9)),
-            tick=False,
-        ):
-            d3 = DocumentFactory(
-                title="bank statement 3",
-                content="things i paid for in september",
-                created=datetime.date(2020, 7, 9),
-                added=timezone.make_aware(datetime.datetime(2020, 7, 9)),
-            )
-        with time_machine.travel(
-            timezone.make_aware(datetime.datetime(2021, 11, 30)),
-            tick=False,
-        ):
-            d4 = DocumentFactory(
-                title="Quarterly Report",
-                content="quarterly revenue profit margin earnings growth",
-                created=datetime.date(2021, 11, 30),
-                added=timezone.make_aware(datetime.datetime(2021, 11, 30)),
-            )
         backend = get_backend()
         backend.add_or_update(d1)
         backend.add_or_update(d2)

src/documents/tests/test_management_exporter.py

@@ -426,7 +426,7 @@ class TestExportImport(
         st_mtime_1 = (self.target / "manifest.json").stat().st_mtime
 
         with mock.patch(
-            "documents.export.sinks.copy_file_with_basic_stats",
+            "documents.management.commands.document_exporter.copy_file_with_basic_stats",
         ) as m:
             self._do_export()
             m.assert_not_called()
@@ -437,7 +437,7 @@ class TestExportImport(
         Path(self.d1.source_path).touch()
 
         with mock.patch(
-            "documents.export.sinks.copy_file_with_basic_stats",
+            "documents.management.commands.document_exporter.copy_file_with_basic_stats",
         ) as m:
             self._do_export()
             self.assertEqual(m.call_count, 1)
@@ -464,7 +464,7 @@ class TestExportImport(
         self.assertIsFile(self.target / "manifest.json")
 
         with mock.patch(
-            "documents.export.sinks.copy_file_with_basic_stats",
+            "documents.management.commands.document_exporter.copy_file_with_basic_stats",
         ) as m:
             self._do_export()
             m.assert_not_called()
@@ -475,7 +475,7 @@ class TestExportImport(
         self.d2.save()
 
         with mock.patch(
-            "documents.export.sinks.copy_file_with_basic_stats",
+            "documents.management.commands.document_exporter.copy_file_with_basic_stats",
         ) as m:
             self._do_export(compare_checksums=True)
             self.assertEqual(m.call_count, 1)
@@ -1058,26 +1058,6 @@ class TestExportImport(
 
         self.assertEqual(Document.objects.all().count(), 4)
 
-    def test_zip_with_compare_flags_raises(self) -> None:
-        """
-        GIVEN:
-            - A request to export to a zip file
-        WHEN:
-            - --compare-checksums or --compare-json is also passed
-        THEN:
-            - A CommandError is raised (the flags are no-ops in zip mode)
-        """
-        for flag in ("--compare-checksums", "--compare-json"):
-            with self.subTest(flag=flag):
-                with self.assertRaises(CommandError):
-                    call_command(
-                        "document_exporter",
-                        self.target,
-                        "--zip",
-                        flag,
-                        skip_checks=True,
-                    )
-
 
 @pytest.mark.management
 class TestCryptExportImport(

src/documents/tests/test_permission_filtering_security.py

@@ -12,22 +12,9 @@ from django.test import override_settings
 from guardian.shortcuts import assign_perm
 from rest_framework.test import APIClient
 
-from documents.matching import match_correspondents
-from documents.matching import match_document_types
-from documents.matching import match_storage_paths
-from documents.matching import match_tags
-from documents.models import Correspondent
-from documents.models import DocumentType
-from documents.models import StoragePath
-from documents.models import Tag
 from documents.permissions import permitted_document_ids
-from documents.permissions import permitted_object_ids
 from documents.serialisers import _get_viewable_duplicates
-from documents.tests.factories import CorrespondentFactory
 from documents.tests.factories import DocumentFactory
-from documents.tests.factories import DocumentTypeFactory
-from documents.tests.factories import StoragePathFactory
-from documents.tests.factories import TagFactory
 
 
 def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden):
@@ -444,320 +431,3 @@ class TestTrashRestorePermissionBoundary:
             format="json",
         )
         assert response.status_code == HTTPStatus.OK
-
-
-@pytest.mark.django_db
-class TestTrashViewExcludesExplicitlyGrantedDocuments:
-    """
-    Regression test pinning TrashView's use of
-    ``_TrashPermittedObjectsFilter`` (``include_granted = False``). If that
-    flag were ever flipped to the default ``True``, or the subclass removed
-    in favor of the base ``PermittedObjectsFilter``, a trashed document
-    would leak into ``/api/trash/`` results for any user holding an
-    explicit guardian grant on it, even though they are neither the owner
-    nor a superuser.
-    """
-
-    def test_explicit_grant_does_not_leak_trashed_document(self, rest_api_client):
-        owner = User.objects.create_user(username="trash_owner")
-        grantee = User.objects.create_user(username="trash_grantee")
-        doc = DocumentFactory(owner=owner)
-        doc.delete()  # soft delete
-        assign_perm("view_document", grantee, doc)
-
-        rest_api_client.force_authenticate(user=grantee)
-        response = rest_api_client.get("/api/trash/")
-
-        assert response.status_code == HTTPStatus.OK
-        result_ids = {result["id"] for result in response.data["results"]}
-        assert doc.pk not in result_ids
-
-
-@pytest.mark.django_db
-@pytest.mark.parametrize(
-    ("model", "factory", "perm"),
-    [
-        (Tag, TagFactory, "view_tag"),
-        (Correspondent, CorrespondentFactory, "view_correspondent"),
-        (DocumentType, DocumentTypeFactory, "view_documenttype"),
-        (StoragePath, StoragePathFactory, "view_storagepath"),
-    ],
-)
-class TestPermittedObjectIdsGenericModels:
-    def test_owner_sees_own_object(self, model, factory, perm):
-        owner = User.objects.create_user(username=f"owner_{model.__name__}")
-        stranger = User.objects.create_user(username=f"stranger_{model.__name__}")
-        owned = factory(owner=owner)
-        strangers = factory(owner=stranger)
-
-        assert_visible_document_ids(
-            permitted_object_ids(owner, model, perm),
-            expected_visible=[owned.pk],
-            expected_hidden=[strangers.pk],
-        )
-
-    def test_unowned_object_visible_to_everyone(self, model, factory, perm):
-        user = User.objects.create_user(username=f"user_{model.__name__}")
-        unowned = factory(owner=None)
-
-        assert_visible_document_ids(
-            permitted_object_ids(user, model, perm),
-            expected_visible=[unowned.pk],
-            expected_hidden=[],
-        )
-
-    def test_explicit_permission_grants_visibility(self, model, factory, perm):
-        owner = User.objects.create_user(username=f"owner2_{model.__name__}")
-        grantee = User.objects.create_user(username=f"grantee_{model.__name__}")
-        stranger = User.objects.create_user(username=f"stranger2_{model.__name__}")
-        shared = factory(owner=owner)
-        not_shared = factory(owner=owner)
-        assign_perm(perm, grantee, shared)
-
-        assert_visible_document_ids(
-            permitted_object_ids(grantee, model, perm),
-            expected_visible=[shared.pk],
-            expected_hidden=[not_shared.pk],
-        )
-        assert_visible_document_ids(
-            permitted_object_ids(stranger, model, perm),
-            expected_visible=[],
-            expected_hidden=[shared.pk, not_shared.pk],
-        )
-
-    def test_group_permission_grants_visibility_to_members_only(
-        self,
-        model,
-        factory,
-        perm,
-    ):
-        owner = User.objects.create_user(username=f"owner3_{model.__name__}")
-        member = User.objects.create_user(username=f"member_{model.__name__}")
-        non_member = User.objects.create_user(username=f"nonmember_{model.__name__}")
-        group = Group.objects.create(name=f"group_{model.__name__}")
-        member.groups.add(group)
-        shared = factory(owner=owner)
-        assign_perm(perm, group, shared)
-
-        assert_visible_document_ids(
-            permitted_object_ids(member, model, perm),
-            expected_visible=[shared.pk],
-            expected_hidden=[],
-        )
-        assert_visible_document_ids(
-            permitted_object_ids(non_member, model, perm),
-            expected_visible=[],
-            expected_hidden=[shared.pk],
-        )
-
-    def test_superuser_sees_everything(self, model, factory, perm):
-        superuser = User.objects.create_superuser(username=f"root_{model.__name__}")
-        owner = User.objects.create_user(username=f"owner4_{model.__name__}")
-        obj = factory(owner=owner)
-
-        assert_visible_document_ids(
-            permitted_object_ids(superuser, model, perm),
-            expected_visible=[obj.pk],
-            expected_hidden=[],
-        )
-
-
-@pytest.mark.django_db
-class TestMatchingRespectsObjectPermissions:
-    def test_match_tags_only_considers_tags_visible_to_user(self):
-        owner = User.objects.create_user(username="tag_owner")
-        classifying_user = User.objects.create_user(username="classifier_user")
-        visible_tag = TagFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=Tag.MATCH_LITERAL,
-        )
-        hidden_tag = TagFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=Tag.MATCH_LITERAL,
-        )
-        assign_perm("view_tag", classifying_user, visible_tag)
-        doc = DocumentFactory(owner=classifying_user, content="an invoice document")
-
-        matched = match_tags(doc, classifier=None, user=classifying_user)
-        matched_ids = {t.pk for t in matched}
-        assert visible_tag.pk in matched_ids
-        assert hidden_tag.pk not in matched_ids
-
-    def test_match_correspondents_only_considers_correspondents_visible_to_user(self):
-        owner = User.objects.create_user(username="correspondent_owner")
-        classifying_user = User.objects.create_user(username="classifier_user2")
-        visible_correspondent = CorrespondentFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=Correspondent.MATCH_LITERAL,
-        )
-        hidden_correspondent = CorrespondentFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=Correspondent.MATCH_LITERAL,
-        )
-        assign_perm("view_correspondent", classifying_user, visible_correspondent)
-        doc = DocumentFactory(owner=classifying_user, content="an invoice document")
-
-        matched = match_correspondents(doc, classifier=None, user=classifying_user)
-        matched_ids = {c.pk for c in matched}
-        assert visible_correspondent.pk in matched_ids
-        assert hidden_correspondent.pk not in matched_ids
-
-    def test_match_document_types_only_considers_document_types_visible_to_user(self):
-        owner = User.objects.create_user(username="document_type_owner")
-        classifying_user = User.objects.create_user(username="classifier_user3")
-        visible_document_type = DocumentTypeFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=DocumentType.MATCH_LITERAL,
-        )
-        hidden_document_type = DocumentTypeFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=DocumentType.MATCH_LITERAL,
-        )
-        assign_perm("view_documenttype", classifying_user, visible_document_type)
-        doc = DocumentFactory(owner=classifying_user, content="an invoice document")
-
-        matched = match_document_types(doc, classifier=None, user=classifying_user)
-        matched_ids = {dt.pk for dt in matched}
-        assert visible_document_type.pk in matched_ids
-        assert hidden_document_type.pk not in matched_ids
-
-    def test_match_storage_paths_only_considers_storage_paths_visible_to_user(self):
-        owner = User.objects.create_user(username="storage_path_owner")
-        classifying_user = User.objects.create_user(username="classifier_user4")
-        visible_storage_path = StoragePathFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=StoragePath.MATCH_LITERAL,
-        )
-        hidden_storage_path = StoragePathFactory(
-            owner=owner,
-            match="invoice",
-            matching_algorithm=StoragePath.MATCH_LITERAL,
-        )
-        assign_perm("view_storagepath", classifying_user, visible_storage_path)
-        doc = DocumentFactory(owner=classifying_user, content="an invoice document")
-
-        matched = match_storage_paths(doc, classifier=None, user=classifying_user)
-        matched_ids = {sp.pk for sp in matched}
-        assert visible_storage_path.pk in matched_ids
-        assert hidden_storage_path.pk not in matched_ids
-
-
-@pytest.mark.django_db
-class TestBulkEditObjectsApplyToAllPermissionBoundary:
-    def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client):
-        owner = User.objects.create_user(username="tags_owner")
-        requester = User.objects.create_user(username="tags_requester")
-        # grant the global change_tag permission so the object-level
-        # filtering (not the global has_perm check) is what's under test
-        requester.user_permissions.add(
-            Permission.objects.get(codename="change_tag"),
-        )
-        rest_api_client.force_authenticate(user=requester)
-        visible = TagFactory(owner=owner)
-        hidden = TagFactory(owner=owner)
-        assign_perm("view_tag", requester, visible)
-        assign_perm("change_tag", requester, visible)
-
-        response = rest_api_client.post(
-            "/api/bulk_edit_objects/",
-            {
-                "object_type": "tags",
-                "operation": "set_permissions",
-                "all": True,
-                "filters": {},
-                "owner": requester.pk,
-            },
-            format="json",
-        )
-        assert response.status_code == HTTPStatus.OK
-
-        # The apply_to_all dispatch must resolve permitted objects up front:
-        # the visible tag (object-level change_tag granted) gets its owner
-        # reassigned, while the hidden tag (no object-level grant) is
-        # excluded entirely and keeps its original owner.
-        visible.refresh_from_db()
-        hidden.refresh_from_db()
-        assert visible.owner == requester
-        assert hidden.owner == owner
-
-
-@pytest.mark.django_db
-class TestBulkEditObjectsTagDescendantPartialPermission:
-    def test_apply_to_all_descendant_expansion_respects_per_object_permissions(
-        self,
-        rest_api_client,
-    ):
-        """
-        GIVEN:
-            - A tag hierarchy (parent -> permitted_child, unpermitted_child)
-            - A non-superuser requester with object-level change_tag granted
-              on the parent and on only ONE of the two children
-        WHEN:
-            - bulk_edit_objects is called with all=True and a filter that
-              matches only the root (parent) tag, engaging the
-              tag-descendant-expansion logic in BulkEditObjectsView.post
-        THEN:
-            - The descendant expansion only pulls in descendants the
-              requester actually has permission on: the permitted child's
-              owner is reassigned alongside the parent's, while the
-              unpermitted child keeps its original owner. This pins that the
-              expansion checks per-object permissions (editable_ids), not
-              merely "is a descendant of a filter match".
-
-        NOTE: this uses ``set_permissions`` (owner reassignment) rather than
-        ``delete`` as the operation, because Tag.tn_parent (django-treenode)
-        cascades deletes to descendants at the database/ORM level regardless
-        of which tags the view resolved into ``objs`` -- a delete-based test
-        would pass/fail based on FK cascade behavior, not on whether the
-        descendant-expansion logic itself respected per-object permissions.
-        """
-        owner = User.objects.create_user(username="tag_hierarchy_owner")
-        requester = User.objects.create_user(username="tag_hierarchy_requester")
-        # global change_tag permission so the has_perm() gate passes and the
-        # object-level permitted_object_ids filtering is what's under test
-        requester.user_permissions.add(
-            Permission.objects.get(codename="change_tag"),
-        )
-        rest_api_client.force_authenticate(user=requester)
-
-        parent = TagFactory(owner=owner, name="parent-tag")
-        permitted_child = TagFactory(
-            owner=owner,
-            name="permitted-child-tag",
-            tn_parent=parent,
-        )
-        unpermitted_child = TagFactory(
-            owner=owner,
-            name="unpermitted-child-tag",
-            tn_parent=parent,
-        )
-        assign_perm("change_tag", requester, parent)
-        assign_perm("change_tag", requester, permitted_child)
-        # unpermitted_child is intentionally NOT granted change_tag
-
-        response = rest_api_client.post(
-            "/api/bulk_edit_objects/",
-            {
-                "object_type": "tags",
-                "operation": "set_permissions",
-                "all": True,
-                "filters": {"is_root": True},
-                "owner": requester.pk,
-            },
-            format="json",
-        )
-        assert response.status_code == HTTPStatus.OK
-
-        parent.refresh_from_db()
-        permitted_child.refresh_from_db()
-        unpermitted_child.refresh_from_db()
-        assert parent.owner == requester
-        assert permitted_child.owner == requester
-        assert unpermitted_child.owner == owner

src/documents/tests/test_permitted_objects_filter.py

@@ -1,70 +0,0 @@
-import pytest
-from django.contrib.auth.models import User
-from guardian.shortcuts import assign_perm
-from rest_framework.test import APIRequestFactory
-
-from documents.filters import PermittedObjectsFilter
-from documents.models import Tag
-from documents.tests.factories import TagFactory
-
-
-class _DummyView:
-    queryset = Tag.objects.all()
-
-
-@pytest.mark.django_db
-class TestPermittedObjectsFilter:
-    def test_superuser_bypasses_filtering_entirely(self):
-        superuser = User.objects.create_superuser(username="root")
-        owner = User.objects.create_user(username="owner")
-        TagFactory(owner=owner)
-        request = APIRequestFactory().get("/")
-        request.user = superuser
-
-        result = PermittedObjectsFilter().filter_queryset(
-            request,
-            Tag.objects.all(),
-            _DummyView(),
-        )
-        assert result.count() == Tag.objects.count()
-
-    def test_non_superuser_sees_only_owned_unowned_and_granted(self):
-        owner = User.objects.create_user(username="owner")
-        grantee = User.objects.create_user(username="grantee")
-        owned = TagFactory(owner=grantee)
-        unowned = TagFactory(owner=None)
-        granted = TagFactory(owner=owner)
-        hidden = TagFactory(owner=owner)
-        assign_perm("view_tag", grantee, granted)
-        request = APIRequestFactory().get("/")
-        request.user = grantee
-
-        result = PermittedObjectsFilter().filter_queryset(
-            request,
-            Tag.objects.all(),
-            _DummyView(),
-        )
-        visible_ids = set(result.values_list("id", flat=True))
-        assert visible_ids == {owned.pk, unowned.pk, granted.pk}
-        assert hidden.pk not in visible_ids
-
-    def test_include_granted_false_excludes_explicitly_shared_objects(self):
-        owner = User.objects.create_user(username="owner2")
-        grantee = User.objects.create_user(username="grantee2")
-        owned = TagFactory(owner=grantee)
-        granted = TagFactory(owner=owner)
-        assign_perm("view_tag", grantee, granted)
-        request = APIRequestFactory().get("/")
-        request.user = grantee
-
-        class _OwnerOnlyFilter(PermittedObjectsFilter):
-            include_granted = False
-
-        result = _OwnerOnlyFilter().filter_queryset(
-            request,
-            Tag.objects.all(),
-            _DummyView(),
-        )
-        visible_ids = set(result.values_list("id", flat=True))
-        assert visible_ids == {owned.pk}
-        assert granted.pk not in visible_ids

src/documents/tests/test_views.py

@@ -78,6 +78,10 @@ class TestViews(DirectoriesMixin, TestCase):
                 response.context_data["styles_css"],
                 f"frontend/{language_actual}/styles.css",
             )
+            self.assertEqual(
+                response.context_data["runtime_js"],
+                f"frontend/{language_actual}/runtime.js",
+            )
             self.assertEqual(
                 response.context_data["polyfills_js"],
                 f"frontend/{language_actual}/polyfills.js",

src/documents/views.py

@@ -133,10 +133,12 @@ from documents.file_handling import format_filename
 from documents.filters import CorrespondentFilterSet
 from documents.filters import CustomFieldFilterSet
 from documents.filters import DocumentFilterSet
+from documents.filters import DocumentPermissionsFilter
 from documents.filters import DocumentsOrderingFilter
 from documents.filters import DocumentTypeFilterSet
+from documents.filters import ObjectOwnedOrGrantedPermissionsFilter
+from documents.filters import ObjectOwnedPermissionsFilter
 from documents.filters import PaperlessTaskFilterSet
-from documents.filters import PermittedObjectsFilter
 from documents.filters import ShareLinkBundleFilterSet
 from documents.filters import ShareLinkFilterSet
 from documents.filters import StoragePathFilterSet
@@ -176,7 +178,6 @@ from documents.permissions import has_global_statistics_permission
 from documents.permissions import has_perms_owner_aware
 from documents.permissions import has_system_status_permission
 from documents.permissions import permitted_document_ids
-from documents.permissions import permitted_object_ids
 from documents.permissions import set_permissions_for_object
 from documents.plugins.date_parsing import get_date_parser
 from documents.schema import generate_object_with_permissions_schema
@@ -347,6 +348,7 @@ class IndexView(TemplateView):
         context["username"] = self.request.user.username
         context["full_name"] = self.request.user.get_full_name()
         context["styles_css"] = f"frontend/{self.get_frontend_language()}/styles.css"
+        context["runtime_js"] = f"frontend/{self.get_frontend_language()}/runtime.js"
         context["polyfills_js"] = (
             f"frontend/{self.get_frontend_language()}/polyfills.js"
         )
@@ -549,7 +551,7 @@ class CorrespondentViewSet(
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = CorrespondentFilterSet
     ordering_fields = (
@@ -590,7 +592,7 @@ class TagViewSet(PermissionsAwareDocumentCountMixin, ModelViewSet[Tag]):
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = TagFilterSet
     ordering_fields = ("color", "name", "matching_algorithm", "match", "document_count")
@@ -682,7 +684,7 @@ class DocumentTypeViewSet(
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = DocumentTypeFilterSet
     ordering_fields = ("name", "matching_algorithm", "match", "document_count")
@@ -986,7 +988,7 @@ class DocumentViewSet(
         DjangoFilterBackend,
         SearchFilter,
         DocumentsOrderingFilter,
-        PermittedObjectsFilter,
+        DocumentPermissionsFilter,
     )
     filterset_class = DocumentFilterSet
     search_fields = ("title", "correspondent__name", "effective_content")
@@ -2672,7 +2674,7 @@ class SavedViewViewSet(BulkPermissionMixin, PassUserMixin, ModelViewSet[SavedVie
     permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
     filter_backends = (
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     ordering_fields = ("name",)
 
@@ -3919,7 +3921,7 @@ class StoragePathViewSet(PermissionsAwareDocumentCountMixin, ModelViewSet[Storag
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = StoragePathFilterSet
     ordering_fields = ("name", "path", "matching_algorithm", "match", "document_count")
@@ -4450,7 +4452,7 @@ class ShareLinkViewSet(
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = ShareLinkFilterSet
     ordering_fields = ("created", "expiration", "document")
@@ -4480,7 +4482,7 @@ class ShareLinkBundleViewSet(PassUserMixin, ModelViewSet[ShareLinkBundle]):
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = ShareLinkBundleFilterSet
     ordering_fields = ("created", "expiration", "status")
@@ -4763,8 +4765,10 @@ class BulkEditObjectsView(PassUserMixin):
                 "document_types": DocumentTypeFilterSet,
                 "storage_paths": StoragePathFilterSet,
             }[object_type]
-            user_permitted_objects = object_class.objects.filter(
+            user_permitted_objects = get_objects_for_user_owner_aware(
-                id__in=permitted_object_ids(user, object_class, perm_codename),
+                user,
+                perm_codename,
+                object_class,
             )
             objs = filterset_class(
                 data=filters,
@@ -4789,11 +4793,8 @@ class BulkEditObjectsView(PassUserMixin):
 
         if not user.is_superuser:
             perm = f"documents.{perm_codename}"
-            has_perms = (
+            has_perms = user.has_perm(perm) and all(
-                user.has_perm(perm)
+                has_perms_owner_aware(user, perm_codename, obj) for obj in objs
-                and not objs.exclude(
-                    pk__in=permitted_object_ids(user, object_class, perm_codename),
-                ).exists()
             )
 
             if not has_perms:
@@ -5294,11 +5295,7 @@ class SystemStatusView(PassUserMixin):
 class TrashView(ListModelMixin, PassUserMixin):
     permission_classes = (IsAuthenticated,)
     serializer_class = TrashSerializer
-
+    filter_backends = (ObjectOwnedPermissionsFilter,)
-    class _TrashPermittedObjectsFilter(PermittedObjectsFilter):
-        include_granted = False
-
-    filter_backends = (_TrashPermittedObjectsFilter,)
     pagination_class = StandardPagination
 
     model = Document

src/locale/en_US/LC_MESSAGES/django.po

@@ -2,7 +2,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: paperless-ngx\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2026-08-08 14:28+0000\n"
+"POT-Creation-Date: 2026-08-05 14:50+0000\n"
 "PO-Revision-Date: 2022-02-17 04:17\n"
 "Last-Translator: \n"
 "Language-Team: English\n"
@@ -21,39 +21,39 @@ msgstr ""
 msgid "Documents"
 msgstr ""
 
-#: documents/filters.py:471
+#: documents/filters.py:472
 msgid "Value must be valid JSON."
 msgstr ""
 
-#: documents/filters.py:490
+#: documents/filters.py:491
 msgid "Invalid custom field query expression"
 msgstr ""
 
-#: documents/filters.py:500
+#: documents/filters.py:501
 msgid "Invalid expression list. Must be nonempty."
 msgstr ""
 
-#: documents/filters.py:521
+#: documents/filters.py:522
 msgid "Invalid logical operator {op!r}"
 msgstr ""
 
-#: documents/filters.py:535
+#: documents/filters.py:536
 msgid "Maximum number of query conditions exceeded."
 msgstr ""
 
-#: documents/filters.py:599
+#: documents/filters.py:600
 msgid "{name!r} is not a valid custom field."
 msgstr ""
 
-#: documents/filters.py:636
+#: documents/filters.py:637
 msgid "{data_type} does not support query expr {expr!r}."
 msgstr ""
 
-#: documents/filters.py:755 documents/models.py:136
+#: documents/filters.py:756 documents/models.py:136
 msgid "Maximum nesting depth exceeded."
 msgstr ""
 
-#: documents/filters.py:1073
+#: documents/filters.py:1098
 msgid "Custom field not found"
 msgstr ""
 
@@ -1352,7 +1352,7 @@ msgid "workflow runs"
 msgstr ""
 
 #: documents/serialisers.py:521 documents/serialisers.py:873
-#: documents/serialisers.py:2767 documents/views.py:299 documents/views.py:2555
+#: documents/serialisers.py:2767 documents/views.py:300 documents/views.py:2557
 #: paperless_mail/serialisers.py:155
 msgid "Insufficient permissions."
 msgstr ""
@@ -1393,7 +1393,7 @@ msgstr ""
 msgid "Duplicate document identifiers are not allowed."
 msgstr ""
 
-#: documents/serialisers.py:2853 documents/views.py:4509
+#: documents/serialisers.py:2853 documents/views.py:4511
 #, python-format
 msgid "Documents not found: %(ids)s"
 msgstr ""
@@ -1661,36 +1661,36 @@ msgstr ""
 msgid "Unable to parse URI {value}"
 msgstr ""
 
-#: documents/views.py:292 documents/views.py:2552
+#: documents/views.py:293 documents/views.py:2554
 msgid "Invalid more_like_id"
 msgstr ""
 
-#: documents/views.py:1566
+#: documents/views.py:1568
 msgid "Invalid AI configuration."
 msgstr ""
 
-#: documents/views.py:1575
+#: documents/views.py:1577
 msgid "AI backend request timed out."
 msgstr ""
 
-#: documents/views.py:2377 documents/views.py:2698
+#: documents/views.py:2379 documents/views.py:2700
 msgid "Specify only one of text, title_search, query, or more_like_id."
 msgstr ""
 
-#: documents/views.py:4522
+#: documents/views.py:4524
 #, python-format
 msgid "Insufficient permissions to share document %(id)s."
 msgstr ""
 
-#: documents/views.py:4568
+#: documents/views.py:4570
 msgid "Bundle is already being processed."
 msgstr ""
 
-#: documents/views.py:4629
+#: documents/views.py:4631
 msgid "The share link bundle is still being prepared. Please try again later."
 msgstr ""
 
-#: documents/views.py:4639
+#: documents/views.py:4641
 msgid "The share link bundle is unavailable."
 msgstr ""
 

src/paperless/serialisers.py

@@ -217,7 +217,6 @@ class ApplicationConfigurationSerializer(
     llm_api_key = ObfuscatedPasswordField(
         required=False,
         allow_null=True,
-        max_length=1024,
     )
 
     def run_validation(self, data):

src/paperless_mail/views.py

@@ -23,7 +23,7 @@ from rest_framework.response import Response
 from rest_framework.viewsets import ModelViewSet
 from rest_framework.viewsets import ReadOnlyModelViewSet
 
-from documents.filters import PermittedObjectsFilter
+from documents.filters import ObjectOwnedOrGrantedPermissionsFilter
 from documents.models import PaperlessTask
 from documents.permissions import PaperlessObjectPermissions
 from documents.permissions import has_perms_owner_aware
@@ -75,7 +75,7 @@ class MailAccountViewSet(PassUserMixin, ModelViewSet[MailAccount]):
     serializer_class = MailAccountSerializer
     pagination_class = StandardPagination
     permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
-    filter_backends = (PermittedObjectsFilter,)
+    filter_backends = (ObjectOwnedOrGrantedPermissionsFilter,)
 
     def get_permissions(self):
         if self.action == "test":
@@ -197,7 +197,7 @@ class ProcessedMailViewSet(PassUserMixin, ReadOnlyModelViewSet[ProcessedMail]):
     filter_backends = (
         DjangoFilterBackend,
         OrderingFilter,
-        PermittedObjectsFilter,
+        ObjectOwnedOrGrantedPermissionsFilter,
     )
     filterset_class = ProcessedMailFilterSet
 
@@ -225,7 +225,7 @@ class MailRuleViewSet(PassUserMixin, ModelViewSet[MailRule]):
     serializer_class = MailRuleSerializer
     pagination_class = StandardPagination
     permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
-    filter_backends = (PermittedObjectsFilter,)
+    filter_backends = (ObjectOwnedOrGrantedPermissionsFilter,)
 
 
 @extend_schema_view(