Chore: Supersample the PDF thumbnail render 2x

Render the PDF page at twice the target size and downsample, with edge case
handling for small and odd sized pages, simplified helpers and tests, and
trimmed comments.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

.github/workflows/ci-backend.yml

@@ -111,7 +111,7 @@ jobs:
         timeout-minutes: 12
         uses: $/.github/actions/apt-install
         with:
-          packages: unpaper tesseract-ocr imagemagick ghostscript poppler-utils
+          packages: unpaper tesseract-ocr imagemagick ghostscript poppler-utils qpdf
       - name: Configure ImageMagick
         run: |
           sudo cp docker/rootfs/etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-7/policy.xml

docs/configuration.md

@@ -1315,29 +1315,17 @@ valid crontab(5) expression describing when to run.
 
 #### [`PAPERLESS_CONVERT_MEMORY_LIMIT=<num>`](#PAPERLESS_CONVERT_MEMORY_LIMIT) {#PAPERLESS_CONVERT_MEMORY_LIMIT}
 
-: On smaller systems, or even in the case of Very Large Documents, the
-consumer may explode, complaining about how it's "unable to extend
-pixel cache". In such cases, try setting this to a reasonably low
-value, like 32. The default is to use whatever is necessary to do
-everything without writing to disk, and units are in megabytes.
+!!! warning
 
-    For more information on how to use this value, you should search the
-    web for "MAGICK_MEMORY_LIMIT".
-
-    Defaults to 0, which disables the limit.
+    Deprecated and has no effect, since PDF thumbnails no longer use
+    ImageMagick. It will be removed in a future release.
 
 #### [`PAPERLESS_CONVERT_TMPDIR=<path>`](#PAPERLESS_CONVERT_TMPDIR) {#PAPERLESS_CONVERT_TMPDIR}
 
-: Similar to the memory limit, if you've got a small system and your
-OS mounts /tmp as tmpfs, you should set this to a path that's on a
-physical disk, like /home/your_user/tmp or something. ImageMagick
-will use this as scratch space when crunching through very large
-documents.
+!!! warning
 
-    For more information on how to use this value, you should search the
-    web for "MAGICK_TMPDIR".
-
-    Default is none, which disables the temporary directory.
+    Deprecated and has no effect, since PDF thumbnails no longer use
+    ImageMagick. It will be removed in a future release.
 
 #### [`PAPERLESS_APPS=<string>`](#PAPERLESS_APPS) {#PAPERLESS_APPS}
 

docs/setup.md

@@ -177,12 +177,12 @@ to a positive number to enable polling and disable native filesystem notificatio
     - `pkg-config` for mysqlclient (python dependency)
     - `fonts-liberation` for generating thumbnails for plain text
       files
-    - `imagemagick` >= 6 for PDF conversion
+    - `imagemagick` >= 6 for image alpha handling
     - `gnupg` for decrypting GPG-encrypted email
     - `libpq-dev` for PostgreSQL
     - `libmagic-dev` for mime type detection
     - `mariadb-client` for MariaDB compile time
-    - `poppler-utils` for barcode detection
+    - `poppler-utils` for thumbnail generation and barcode detection
 
     Use this list for your preferred package management:
 
@@ -416,11 +416,8 @@ to a positive number to enable polling and disable native filesystem notificatio
         You may need to change the path in the files. Example:
         `ExecStart=/opt/paperless/.local/bin/celery --app paperless worker --loglevel INFO`
 
-12. Configure ImageMagick to allow processing of PDF documents and disable
-    formats that Paperless-ngx does not use. Most distributions disable PDF
-    processing by default, since PDF documents can contain malware. If you
-    don't enable it, Paperless-ngx will fall back to Ghostscript for certain
-    steps such as thumbnail generation.
+12. Harden ImageMagick by disabling formats that Paperless-ngx does not use.
+    PDF processing is not needed and should stay disabled.
 
     Configure the active ImageMagick policy file (commonly
     `/etc/ImageMagick-6/policy.xml` or `/etc/ImageMagick-7/policy.xml`) and

paperless.conf.example

@@ -50,8 +50,6 @@ PAPERLESS_SECRET_KEY=change-me
 #PAPERLESS_OCR_ROTATE_PAGES=true
 #PAPERLESS_OCR_ROTATE_PAGES_THRESHOLD=12.0
 #PAPERLESS_OCR_USER_ARGS={}
-#PAPERLESS_CONVERT_MEMORY_LIMIT=0
-#PAPERLESS_CONVERT_TMPDIR=/var/tmp/paperless
 
 # Software tweaks
 

pyproject.toml

@@ -256,8 +256,8 @@ isort.force-single-line = true
 [tool.codespell]
 ignore-words-list = "criterias,afterall,valeu,ureue,equest,ure,assertIn,Oktober,commitish,NIN,nin,reprot"
 skip = """\
-  src-ui/src/locale/*,src-ui/pnpm-lock.yaml,src-ui/e2e/*,src/paperless/tests/samples/mail/*,src/documents/tests/samples\
-  /*,src/paperless_testing/sample_files/*,*.po,*.json\
+  src-ui/src/locale/*,src-ui/pnpm-lock.yaml,src-ui/e2e/*,src/paperless_mail/tests/samples/*,src/paperless/tests/samples\
+  /mail/*,src/documents/tests/samples/*,*.po,*.json\
   """
 write-changes = true
 

src-ui/tsconfig.json

@@ -15,7 +15,7 @@
     "target": "ES2022",
     "module": "es2020",
     "lib": [
-      "es2022",
+      "es2020",
       "dom"
     ],
     "useDefineForClassFields": false

src/conftest.py

@@ -11,8 +11,6 @@ from typing import TYPE_CHECKING
 
 import pytest
 
-from paperless_testing import samples
-
 if TYPE_CHECKING:
     from collections.abc import Generator
     from pathlib import Path
@@ -151,39 +149,3 @@ def fake_progress_manager(
 
     monkeypatch.setattr("documents.tasks.ProgressManager", FakeProgressManager)
     return FakeProgressManager
-
-
-@pytest.fixture(scope="session")
-def shared_samples_dir() -> Path:
-    """Directory of sample files used by more than one app's tests."""
-    return samples.SHARED_SAMPLES_DIR
-
-
-@pytest.fixture(scope="session")
-def simple_digital_pdf_file() -> Path:
-    """One-page PDF with a text layer."""
-    return samples.SIMPLE_DIGITAL_PDF
-
-
-@pytest.fixture(scope="session")
-def multi_page_digital_pdf_file() -> Path:
-    """Three-page PDF with a text layer."""
-    return samples.MULTI_PAGE_DIGITAL_PDF
-
-
-@pytest.fixture(scope="session")
-def with_form_pdf_file() -> Path:
-    """PDF containing a fillable form."""
-    return samples.WITH_FORM_PDF
-
-
-@pytest.fixture(scope="session")
-def multi_page_images_pdf_file() -> Path:
-    """Multi-page PDF of scanned images, no text layer."""
-    return samples.MULTI_PAGE_IMAGES_PDF
-
-
-@pytest.fixture(scope="session")
-def thumbnail_webp_file() -> Path:
-    """Small WebP thumbnail."""
-    return samples.THUMBNAIL_WEBP

src/documents/parsers.py

@@ -1,8 +1,8 @@
 from __future__ import annotations
 
 import logging
+import math
 import mimetypes
-import os
 import shutil
 import subprocess
 import tempfile
@@ -68,58 +68,6 @@ def get_supported_file_extensions() -> set[str]:
     return extensions
 
 
-def run_convert(
-    input_file,
-    output_file,
-    *,
-    density=None,
-    scale=None,
-    alpha=None,
-    strip=False,
-    trim=False,
-    type=None,
-    depth=None,
-    auto_orient=False,
-    use_cropbox=False,
-    extra=None,
-    logging_group=None,
-) -> None:
-    environment = os.environ.copy()
-    if settings.CONVERT_MEMORY_LIMIT:
-        # MAGICK_MEMORY_LIMIT sets the maximum amount of RAM the pixel cache can use.
-        # MAGICK_MAP_LIMIT sets the maximum amount of memory-mapped I/O allowed.
-        #
-        # For large-format documents  ImageMagick will hit the RAM limit and
-        # immediately try to "map" the remaining data. If MAGICK_MAP_LIMIT isn't
-        # also set, the process may trigger an OOM kill because the default
-        # system/policy map limit is often too restrictive for these massive bitmaps.
-        environment["MAGICK_MEMORY_LIMIT"] = settings.CONVERT_MEMORY_LIMIT
-        environment["MAGICK_MAP_LIMIT"] = settings.CONVERT_MEMORY_LIMIT
-    if settings.CONVERT_TMPDIR:
-        environment["MAGICK_TMPDIR"] = settings.CONVERT_TMPDIR
-
-    args = [settings.CONVERT_BINARY]
-    args += ["-density", str(density)] if density else []
-    args += ["-scale", str(scale)] if scale else []
-    args += ["-alpha", str(alpha)] if alpha else []
-    args += ["-strip"] if strip else []
-    args += ["-trim"] if trim else []
-    args += ["-type", str(type)] if type else []
-    args += ["-depth", str(depth)] if depth else []
-    args += ["-auto-orient"] if auto_orient else []
-    args += ["-define", "pdf:use-cropbox=true"] if use_cropbox else []
-    args += [str(input_file), str(output_file)]
-
-    logger.debug("Execute: " + " ".join(args), extra={"group": logging_group})
-
-    try:
-        run_subprocess(args, environment, logger)
-    except subprocess.CalledProcessError as e:
-        raise ParseError(f"Convert failed at {args}") from e
-    except Exception as e:  # pragma: no cover
-        raise ParseError("Unknown error running convert") from e
-
-
 def get_default_thumbnail() -> Path:
     """
     Returns the path to a generic thumbnail
@@ -127,46 +75,168 @@ def get_default_thumbnail() -> Path:
     return (Path(__file__).parent / "resources" / "document.webp").resolve()
 
 
-def make_thumbnail_from_pdf_gs_fallback(in_path, temp_dir, logging_group=None) -> Path:
-    out_path: Path = Path(temp_dir) / "convert_gs.webp"
+_THUMBNAIL_MAX_WIDTH = 500
+_THUMBNAIL_MAX_HEIGHT = 5000
+# Used only when the page geometry cannot be read
+_THUMBNAIL_FALLBACK_DPI = 150
+# Applied before supersampling, so tiny pages are not enlarged
+_THUMBNAIL_MAX_DPI = 300
+# Rendering at a multiple and downsampling keeps text crisper
+_THUMBNAIL_SUPERSAMPLE = 2
 
-    # if convert fails, fall back to extracting
-    # the first PDF page as a PNG using Ghostscript
-    logger.warning(
-        "Thumbnail generation with ImageMagick failed, falling back "
-        "to ghostscript. Check your /etc/ImageMagick-x/policy.xml!",
-        extra={"group": logging_group},
-    )
-    # Ghostscript doesn't handle WebP outputs
-    gs_out_path: Path = Path(temp_dir) / "gs_out.png"
-    cmd = [settings.GS_BINARY, "-q", "-sDEVICE=pngalpha", "-o", gs_out_path, in_path]
+
+def rasterize_pdf_page_to_png(
+    in_path: Path,
+    out_path: Path,
+    *,
+    dpi: int,
+    logging_group=None,
+) -> None:
+    """
+    Rasterizes the first page of a PDF to a PNG with pdftoppm.
+    """
+    # -singlefile drops the page number and -png appends ".png", so pass the
+    # path without its suffix
+    args = [
+        "pdftoppm",
+        "-f",
+        "1",
+        "-l",
+        "1",
+        "-r",
+        str(dpi),
+        "-png",
+        "-singlefile",
+        "-cropbox",
+        str(in_path),
+        str(out_path.with_suffix("")),
+    ]
+
+    logger.debug("Execute: " + " ".join(args), extra={"group": logging_group})
 
     try:
-        try:
-            run_subprocess(cmd, logger=logger)
-        except subprocess.CalledProcessError as e:
-            raise ParseError(f"Thumbnail (gs) failed at {cmd}") from e
-        # then run convert on the output from gs to make WebP
-        run_convert(
-            density=300,
-            scale="500x5000>",
-            alpha="remove",
-            strip=True,
-            trim=False,
-            auto_orient=True,
-            input_file=gs_out_path,
-            output_file=out_path,
-            logging_group=logging_group,
-        )
+        run_subprocess(args, logger=logger)
+    except subprocess.CalledProcessError as e:
+        raise ParseError(f"pdftoppm failed at {args}") from e
+    except Exception as e:  # pragma: no cover
+        raise ParseError("Unknown error running pdftoppm") from e
 
+
+def encode_thumbnail_webp(
+    png_path: Path,
+    out_path: Path,
+    *,
+    supersample: int = 1,
+) -> None:
+    """
+    Flattens alpha onto white, undoes supersampling, shrinks to fit and saves as WebP.
+    """
+    from PIL import Image
+
+    try:
+        with Image.open(png_path) as im:
+            if im.mode in ("RGBA", "LA"):
+                flattened = Image.new("RGB", im.size, (255, 255, 255))
+                flattened.paste(im, mask=im.split()[-1])
+            else:
+                flattened = im.convert("RGB")
+
+        if supersample > 1:
+            flattened = flattened.resize(
+                (
+                    max(1, round(flattened.width / supersample)),
+                    max(1, round(flattened.height / supersample)),
+                ),
+                Image.Resampling.LANCZOS,
+            )
+
+        flattened.thumbnail((_THUMBNAIL_MAX_WIDTH, _THUMBNAIL_MAX_HEIGHT))
+        flattened.save(out_path, format="WEBP")
+    except (OSError, Image.DecompressionBombError) as e:
+        raise ParseError(f"Unable to encode thumbnail from {png_path}") from e
+
+
+def _compute_thumbnail_dpi(in_path: Path, logging_group=None) -> tuple[int, int]:
+    """
+    Returns (dpi, supersample). Unknown geometry is not supersampled, since
+    the render size cannot be bounded.
+    """
+    from paperless.parsers.utils import get_pdf_first_page_size_points
+
+    size = get_pdf_first_page_size_points(in_path)
+    if size is None:
+        logger.debug(
+            "Could not read PDF page size, using fallback DPI",
+            extra={"group": logging_group},
+        )
+        return _THUMBNAIL_FALLBACK_DPI, 1
+
+    width_pts, height_pts = size
+    dpi_for_width = _THUMBNAIL_MAX_WIDTH * 72 / width_pts
+    dpi_for_height = _THUMBNAIL_MAX_HEIGHT * 72 / height_pts
+    # Round up so the downsampled render is never a few pixels short of the
+    # target; the shrink-only clamp in encode_thumbnail_webp trims the excess.
+    dpi = max(
+        1,
+        math.ceil(min(_THUMBNAIL_MAX_DPI, dpi_for_width, dpi_for_height)),
+    )
+    # At the 1 DPI floor the page is already oversized, so do not supersample
+    return dpi, 1 if dpi == 1 else _THUMBNAIL_SUPERSAMPLE
+
+
+def _render_pdf_thumbnail(
+    in_path: Path,
+    png_path: Path,
+    out_path: Path,
+    logging_group=None,
+) -> None:
+    dpi, supersample = _compute_thumbnail_dpi(in_path, logging_group=logging_group)
+    rasterize_pdf_page_to_png(
+        in_path,
+        png_path,
+        dpi=dpi * supersample,
+        logging_group=logging_group,
+    )
+    encode_thumbnail_webp(png_path, out_path, supersample=supersample)
+
+
+def _repair_pdf_with_qpdf(in_path: Path, out_path: Path) -> None:
+    # qpdf exits 3 after a repair; --warning-exit-0 keeps that from failing
+    try:
+        shutil.copy(in_path, out_path)
+        run_subprocess(
+            ["qpdf", "--warning-exit-0", "--replace-input", str(out_path)],
+            logger=logger,
+        )
+    except (subprocess.CalledProcessError, OSError) as e:
+        raise ParseError(f"qpdf repair failed for {in_path}") from e
+
+
+def make_thumbnail_from_pdf_qpdf_fallback(
+    in_path: Path,
+    temp_dir: Path,
+    logging_group=None,
+) -> Path:
+    png_path = temp_dir / "page1_repaired.png"
+    out_path = temp_dir / "convert_qpdf.webp"
+    repaired_path = temp_dir / "repaired.pdf"
+
+    logger.warning(
+        "Thumbnail generation with pdftoppm failed, attempting qpdf repair and retry.",
+        extra={"group": logging_group},
+    )
+
+    try:
+        _repair_pdf_with_qpdf(in_path, repaired_path)
+        _render_pdf_thumbnail(repaired_path, png_path, out_path, logging_group)
         return out_path
 
     except ParseError as e:
-        logger.error(f"Unable to make thumbnail with Ghostscript: {e}")
+        logger.error(f"Unable to make thumbnail after qpdf repair: {e}")
         # The caller might expect a generated thumbnail that can be moved,
         # so we need to copy it before it gets moved.
         # https://github.com/paperless-ngx/paperless-ngx/issues/3631
-        default_thumbnail_path: Path = Path(temp_dir) / "document.webp"
+        default_thumbnail_path = temp_dir / "document.webp"
         copy_file_with_basic_stats(get_default_thumbnail(), default_thumbnail_path)
         return default_thumbnail_path
 
@@ -175,25 +245,18 @@ def make_thumbnail_from_pdf(in_path: Path, temp_dir: Path, logging_group=None) -
     """
     The thumbnail of a PDF is just a 500px wide image of the first page.
     """
+    png_path: Path = temp_dir / "page1.png"
     out_path: Path = temp_dir / "convert.webp"
 
-    # Run convert to get a decent thumbnail
     try:
-        run_convert(
-            density=300,
-            scale="500x5000>",
-            alpha="remove",
-            strip=True,
-            trim=False,
-            auto_orient=True,
-            use_cropbox=True,
-            input_file=f"{in_path}[0]",
-            output_file=str(out_path),
-            logging_group=logging_group,
-        )
+        _render_pdf_thumbnail(in_path, png_path, out_path, logging_group)
     except ParseError as e:
-        logger.error(f"Unable to make thumbnail with convert: {e}")
-        out_path = make_thumbnail_from_pdf_gs_fallback(in_path, temp_dir, logging_group)
+        logger.error(f"Unable to make thumbnail with pdftoppm: {e}")
+        out_path = make_thumbnail_from_pdf_qpdf_fallback(
+            in_path,
+            temp_dir,
+            logging_group,
+        )
 
     return out_path
 

src/documents/permissions.py

@@ -6,9 +6,7 @@ from django.contrib.auth.models import Permission
 from django.contrib.auth.models import User
 from django.contrib.contenttypes.models import ContentType
 from django.db.models import Case
-from django.db.models import CharField
 from django.db.models import Count
-from django.db.models import F
 from django.db.models import IntegerField
 from django.db.models import Model
 from django.db.models import Q
@@ -351,7 +349,6 @@ def permitted_object_ids(
     perm: str,
     *,
     include_deleted: bool = False,
-    parent_field: str | None = None,
 ) -> QuerySet[int]:
     """
     Generic version of ``permitted_document_ids`` for any model with an
@@ -360,24 +357,6 @@ def permitted_object_ids(
     soft-delete pattern (currently only ``Document``); for every other model
     it is accepted but has no effect, since those models have no soft-delete
     concept.
-
-    ``parent_field`` names a self-referencing foreign key whose target
-    authorizes the row (``Document.root_document``). A row with a parent is
-    visible exactly when its parent is, judged by the parent's owner and
-    grants, so the row's own owner and grants are ignored.
-
-    Guardian stores ``object_pk`` as a string, so the row key is cast to a
-    string and tested against the user's and groups' grants with a single
-    uncorrelated ``IN``. Postgres and SQLite build that set once. MariaDB
-    evaluates it as an index probe per row, which is cheap because the
-    lookups use guardian's unique indexes. Casting every ``object_pk`` to an
-    integer instead cannot use an index, and MariaDB cannot materialize it
-    inside the owner ``OR``, so it re-scans the user's grants for every row.
-    A correlated ``EXISTS`` per grant fixes MariaDB too, but Postgres and
-    SQLite re-run it for every row and end up slower than the original. The
-    user's groups are matched with an ``IN`` subquery rather than a join
-    through the membership table, which SQLite plans badly once the grant
-    tables grow.
     """
     has_soft_delete = hasattr(model, "global_objects")
     manager = (
@@ -385,21 +364,8 @@ def permitted_object_ids(
     )
     base_qs = manager.all().only("id", "owner")
 
-    owner_field, key_field = "owner", "pk"
-    if parent_field is not None:
-        owner_field, key_field = "authorizing_owner", "authorizing_id"
-        base_qs = base_qs.annotate(
-            authorizing_id=Coalesce(f"{parent_field}_id", "id"),
-            authorizing_owner=Case(
-                When(**{f"{parent_field}_id__isnull": True}, then=F("owner_id")),
-                default=F(f"{parent_field}__owner_id"),
-                output_field=IntegerField(),
-            ),
-        )
-    unowned = Q(**{f"{owner_field}__isnull": True})
-
     if user is None or not getattr(user, "is_authenticated", False):
-        return base_qs.filter(unowned).values_list("id", flat=True)
+        return base_qs.filter(owner__isnull=True).values_list("id", flat=True)
 
     # Deactivated users get nothing, deactivated superusers included, so this
     # has to come before the superuser shortcut. guardian's
@@ -423,26 +389,21 @@ def permitted_object_ids(
         "permission__content_type": content_type,
     }
 
-    # Both grant sets are compared to the row key as strings, exactly as
-    # guardian stores them, and are uncorrelated, so each engine can build the
-    # set once instead of probing per row.
-    user_keys = UserObjectPermission.objects.filter(
-        user=user,
-        **perm_filter,
-    ).values_list("object_pk", flat=True)
-    group_keys = GroupObjectPermission.objects.filter(
-        group_id__in=user.groups.values("id"),
-        **perm_filter,
-    ).values_list("object_pk", flat=True)
-    permitted_keys = user_keys.union(group_keys, all=True)
-
-    return (
-        base_qs.annotate(permitted_key=Cast(key_field, CharField(max_length=64)))
-        .filter(
-            Q(**{owner_field: user.pk}) | unowned | Q(permitted_key__in=permitted_keys),
-        )
-        .values_list("id", flat=True)
+    user_perm_ids = (
+        UserObjectPermission.objects.filter(user=user, **perm_filter)
+        .annotate(object_pk_int=Cast("object_pk", IntegerField()))
+        .values_list("object_pk_int", flat=True)
     )
+    group_perm_ids = (
+        GroupObjectPermission.objects.filter(group__user=user, **perm_filter)
+        .annotate(object_pk_int=Cast("object_pk", IntegerField()))
+        .values_list("object_pk_int", flat=True)
+    )
+    permitted_ids = user_perm_ids.union(group_perm_ids)
+
+    return base_qs.filter(
+        Q(owner=user) | Q(owner__isnull=True) | Q(id__in=permitted_ids),
+    ).values_list("id", flat=True)
 
 
 ModelT = TypeVar("ModelT", bound=Model)
@@ -510,16 +471,30 @@ def permitted_document_ids(
     ``include_deleted=True`` for callers that need to check permission on
     soft-deleted documents (e.g. trash restore). This intentionally avoids
     ``get_objects_for_user`` to keep the subquery small and index-friendly.
-
-    A version is authorized by its root document, so a version's own owner and
-    grants never matter.
     """
-    return permitted_object_ids(
-        user,
-        Document,
-        perm,
-        include_deleted=include_deleted,
-        parent_field="root_document",
+    return permitted_object_ids(user, Document, perm, include_deleted=include_deleted)
+
+
+def documents_without_permitted_root(
+    documents: QuerySet[Document],
+    user: User | None,
+    *,
+    perm: str = "view_document",
+    include_deleted: bool = False,
+) -> QuerySet[Document]:
+    """
+    The documents the user lacks ``perm`` on. Versions are authorized by their
+    root document, so a version's own owner is ignored. A single query, without
+    loading the documents or joining the root.
+    """
+    return documents.annotate(
+        root_id=Coalesce("root_document_id", "id"),
+    ).exclude(
+        root_id__in=permitted_document_ids(
+            user,
+            perm=perm,
+            include_deleted=include_deleted,
+        ),
     )
 
 

src/documents/tests/conftest.py

@@ -12,25 +12,29 @@ if TYPE_CHECKING:
     from paperless_testing.dirs import PaperlessDirs
 
 
+@pytest.fixture(scope="session")
+def document_samples_dir() -> Path:
+    """Path to the shared test sample documents."""
+    return Path(__file__).parent / "samples" / "documents"
+
+
 @pytest.fixture()
 def sample_doc(
     paperless_dirs: "PaperlessDirs",
-    simple_digital_pdf_file: Path,
-    multi_page_images_pdf_file: Path,
-    thumbnail_webp_file: Path,
+    document_samples_dir: Path,
 ) -> "Document":
     """Create a document with valid files and matching checksums."""
     with filelock.FileLock(paperless_dirs.media_lock):
         shutil.copy(
-            simple_digital_pdf_file,
+            document_samples_dir / "originals" / "0000001.pdf",
             paperless_dirs.originals_dir / "0000001.pdf",
         )
         shutil.copy(
-            multi_page_images_pdf_file,
+            document_samples_dir / "archive" / "0000001.pdf",
             paperless_dirs.archive_dir / "0000001.pdf",
         )
         shutil.copy(
-            thumbnail_webp_file,
+            document_samples_dir / "thumbnails" / "0000001.webp",
             paperless_dirs.thumbnail_dir / "0000001.webp",
         )
 

src/documents/tests/samples/simple.txt

@@ -0,0 +1 @@
+This is a test file.

src/documents/tests/test_api_bulk_download.py

@@ -53,7 +53,7 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
             archive_checksum="D",
         )
 
-        shutil.copy(self.SIMPLE_PDF, self.doc2.source_path)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", self.doc2.source_path)
         shutil.copy(self.SAMPLE_DIR / "simple.png", self.doc2b.source_path)
         shutil.copy(self.SAMPLE_DIR / "simple.jpg", self.doc3.source_path)
         shutil.copy(self.SAMPLE_DIR / "test_with_bom.pdf", self.doc3.archive_path)

src/documents/tests/test_api_documents.py

@@ -56,8 +56,6 @@ from paperless_testing.http import read_streaming_response
 from paperless_testing.permissions import grant_all_global
 from paperless_testing.permissions import grant_global
 from paperless_testing.permissions import grant_object
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
-from paperless_testing.samples import THUMBNAIL_WEBP
 
 
 class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
@@ -1854,11 +1852,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SimpleUploadedFile(
-            "simple.pdf",
-            SIMPLE_DIGITAL_PDF.read_bytes(),
-            content_type="application/pdf",
-        ) as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f},
@@ -1886,7 +1880,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
 
         payload = SimpleUploadedFile(
             "../../outside.pdf",
-            SIMPLE_DIGITAL_PDF.read_bytes(),
+            (Path(__file__).parent / "samples" / "simple.pdf").read_bytes(),
             content_type="application/pdf",
         )
 
@@ -1915,7 +1909,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        pdf_bytes = SIMPLE_DIGITAL_PDF.read_bytes()
+        pdf_bytes = (Path(__file__).parent / "samples" / "simple.pdf").read_bytes()
         boundary = "paperless-boundary"
         payload = (
             (
@@ -1991,7 +1985,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
     def test_upload_insufficient_permissions(self) -> None:
         self.client.force_authenticate(user=UserFactory(username="testuser2"))
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f},
@@ -2004,11 +1998,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SimpleUploadedFile(
-            "simple.pdf",
-            SIMPLE_DIGITAL_PDF.read_bytes(),
-            content_type="application/pdf",
-        ) as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {
@@ -2041,7 +2031,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"documenst": f},
@@ -2067,7 +2057,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "title": "my custom title"},
@@ -2087,7 +2077,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
         )
 
         c = Correspondent.objects.create(name="test-corres")
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "correspondent": c.id},
@@ -2106,7 +2096,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "correspondent": 3456},
@@ -2121,7 +2111,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
         )
 
         dt = DocumentType.objects.create(name="invoice")
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "document_type": dt.id},
@@ -2140,7 +2130,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "document_type": 34578},
@@ -2155,7 +2145,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
         )
 
         sp = StoragePath.objects.create(name="invoices")
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "storage_path": sp.id},
@@ -2174,7 +2164,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "storage_path": 34578},
@@ -2190,7 +2180,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
 
         t1 = Tag.objects.create(name="tag1")
         t2 = Tag.objects.create(name="tag2")
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "tags": [t2.id, t1.id]},
@@ -2211,7 +2201,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
 
         t1 = Tag.objects.create(name="tag1")
         t2 = Tag.objects.create(name="tag2")
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "tags": [t2.id, t1.id, 734563]},
@@ -2235,7 +2225,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             0,
             tzinfo=zoneinfo.ZoneInfo("America/Los_Angeles"),
         )
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "created": created},
@@ -2251,11 +2241,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SimpleUploadedFile(
-            "simple.pdf",
-            SIMPLE_DIGITAL_PDF.read_bytes(),
-            content_type="application/pdf",
-        ) as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "archive_serial_number": 500},
@@ -2282,11 +2268,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             data_type=CustomField.FieldDataType.STRING,
         )
 
-        with SimpleUploadedFile(
-            "simple.pdf",
-            SIMPLE_DIGITAL_PDF.read_bytes(),
-            content_type="application/pdf",
-        ) as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {
@@ -2341,7 +2323,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
         w1.actions.add(action1)
         w1.save()
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {
@@ -2382,11 +2364,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             data_type=CustomField.FieldDataType.INT,
         )
 
-        with SimpleUploadedFile(
-            "simple.pdf",
-            SIMPLE_DIGITAL_PDF.read_bytes(),
-            content_type="application/pdf",
-        ) as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {
@@ -2435,7 +2413,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
         ]
 
         for payload in error_payloads:
-            with SIMPLE_DIGITAL_PDF.open("rb") as f:
+            with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
                 data = {"document": f, **payload}
                 response = self.client.post(
                     "/api/documents/post_document/",
@@ -2493,7 +2471,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             id=str(uuid.uuid4()),
         )
 
-        with SIMPLE_DIGITAL_PDF.open("rb") as f:
+        with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
             response = self.client.post(
                 "/api/documents/post_document/",
                 {"document": f, "from_webui": True},
@@ -2532,8 +2510,14 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             archive_filename="archive.pdf",
         )
 
-        source_file: Path = THUMBNAIL_WEBP
-        archive_file: Path = SIMPLE_DIGITAL_PDF
+        source_file: Path = (
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "thumbnails"
+            / "0000001.webp"
+        )
+        archive_file: Path = Path(__file__).parent / "samples" / "simple.pdf"
 
         shutil.copy(source_file, doc.source_path)
         shutil.copy(archive_file, doc.archive_path)
@@ -2566,7 +2550,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             mime_type="application/pdf",
         )
 
-        shutil.copy(SIMPLE_DIGITAL_PDF, doc.source_path)
+        shutil.copy(Path(__file__).parent / "samples" / "simple.pdf", doc.source_path)
 
         response = self.client.get(f"/api/documents/{doc.pk}/metadata/")
         self.assertEqual(response.status_code, status.HTTP_200_OK)
@@ -4304,8 +4288,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
             filename="test2.pdf",
         )
 
-        archive_file = SIMPLE_DIGITAL_PDF
-        source_file = SIMPLE_DIGITAL_PDF
+        archive_file = Path(__file__).parent / "samples" / "simple.pdf"
+        source_file = Path(__file__).parent / "samples" / "simple.pdf"
 
         shutil.copy(archive_file, doc.archive_path)
         shutil.copy(source_file, doc2.source_path)

src/documents/tests/test_api_email.py

@@ -12,9 +12,6 @@ from documents.tests.utils import SampleDirMixin
 from paperless_testing.dirs import DirectoriesMixin
 from paperless_testing.factories import UserFactory
 from paperless_testing.permissions import grant_global
-from paperless_testing.samples import MULTI_PAGE_DIGITAL_PDF
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
-from paperless_testing.samples import WITH_FORM_PDF
 
 
 class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
@@ -45,15 +42,15 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
 
         # Copy sample files to document paths (using different files to distinguish versions)
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            self.SAMPLE_DIR / "documents" / "originals" / "0000001.pdf",
             self.doc1.archive_path,
         )
         shutil.copy(
-            MULTI_PAGE_DIGITAL_PDF,
+            self.SAMPLE_DIR / "documents" / "originals" / "0000002.pdf",
             self.doc1.source_path,
         )
         shutil.copy(
-            WITH_FORM_PDF,
+            self.SAMPLE_DIR / "documents" / "originals" / "0000003.pdf",
             self.doc2.source_path,
         )
 
@@ -380,7 +377,7 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
             checksum="3",
             filename="test3.pdf",
         )
-        shutil.copy(self.SIMPLE_PDF, doc3.source_path)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", doc3.source_path)
 
         doc4 = Document.objects.create(
             title="test1",
@@ -389,7 +386,7 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
             checksum="4",
             filename="test4.pdf",
         )
-        shutil.copy(self.SIMPLE_PDF, doc4.source_path)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", doc4.source_path)
 
         response = self.client.post(
             self.ENDPOINT,

src/documents/tests/test_barcodes.py

@@ -167,7 +167,7 @@ class TestBarcode(
             - No barcodes detected
             - No pages to split on
         """
-        test_file = self.SIMPLE_PDF
+        test_file = self.SAMPLE_DIR / "simple.pdf"
         with self.get_reader(test_file) as reader:
             reader.detect()
             separator_page_numbers = reader.get_separation_pages()
@@ -459,7 +459,7 @@ class TestBarcode(
         THEN:
             - Nothing happens
         """
-        test_file = self.SIMPLE_PDF
+        test_file = self.SAMPLE_DIR / "simple.pdf"
 
         with self.get_reader(test_file) as reader:
             try:

src/documents/tests/test_bulk_edit.py

@@ -27,9 +27,6 @@ from documents.permissions import set_permissions_for_objects
 from paperless_testing.dirs import DirectoriesMixin
 from paperless_testing.factories import DocumentFactory
 from paperless_testing.permissions import grant_object
-from paperless_testing.samples import MULTI_PAGE_DIGITAL_PDF
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
-from paperless_testing.samples import WITH_FORM_PDF
 
 
 class TestBulkEdit(DirectoriesMixin, TestCase):
@@ -707,27 +704,47 @@ class TestPDFActions(DirectoriesMixin, TestCase):
         super().setUp()
         sample1 = self.dirs.scratch_dir / "sample.pdf"
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
             sample1,
         )
         sample1_archive = self.dirs.archive_dir / "sample_archive.pdf"
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
             sample1_archive,
         )
         sample2 = self.dirs.scratch_dir / "sample2.pdf"
         shutil.copy(
-            MULTI_PAGE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000002.pdf",
             sample2,
         )
         sample2_archive = self.dirs.archive_dir / "sample2_archive.pdf"
         shutil.copy(
-            MULTI_PAGE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000002.pdf",
             sample2_archive,
         )
         sample3 = self.dirs.scratch_dir / "sample3.pdf"
         shutil.copy(
-            WITH_FORM_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000003.pdf",
             sample3,
         )
         self.doc1 = Document.objects.create(
@@ -761,7 +778,11 @@ class TestPDFActions(DirectoriesMixin, TestCase):
         )
         img_doc_archive = self.dirs.archive_dir / "sample_image.pdf"
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
             img_doc_archive,
         )
         self.img_doc = Document.objects.create(

src/documents/tests/test_consumer.py

@@ -36,9 +36,6 @@ from paperless_testing.assertions import FileSystemAssertsMixin
 from paperless_testing.dirs import DirectoriesMixin
 from paperless_testing.factories import UserFactory
 from paperless_testing.fakes.progress import FakeProgressManager
-from paperless_testing.samples import MULTI_PAGE_DIGITAL_PDF
-from paperless_testing.samples import MULTI_PAGE_IMAGES_PDF
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
 
 
 class _BaseNewStyleParser:
@@ -115,7 +112,9 @@ class _BaseNewStyleParser:
 
 
 class DummyParser(_BaseNewStyleParser):
-    _ARCHIVE_SRC = MULTI_PAGE_IMAGES_PDF
+    _ARCHIVE_SRC = (
+        Path(__file__).parent / "samples" / "documents" / "archive" / "0000001.pdf"
+    )
 
     def parse(self, document_path, mime_type, *, produce_archive: bool = True) -> None:
         self._text = "The Text"
@@ -201,19 +200,33 @@ class TestConsumer(
         self.addCleanup(patcher.stop)
 
     def get_test_file(self):
-        src = SIMPLE_DIGITAL_PDF
+        src = (
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf"
+        )
         dst = self.dirs.scratch_dir / "sample.pdf"
         shutil.copy(src, dst)
         return dst
 
     def get_test_file2(self):
-        src = MULTI_PAGE_DIGITAL_PDF
+        src = (
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000002.pdf"
+        )
         dst = self.dirs.scratch_dir / "sample2.pdf"
         shutil.copy(src, dst)
         return dst
 
     def get_test_archive_file(self):
-        src = MULTI_PAGE_IMAGES_PDF
+        src = (
+            Path(__file__).parent / "samples" / "documents" / "archive" / "0000001.pdf"
+        )
         dst = self.dirs.scratch_dir / "sample_archive.pdf"
         shutil.copy(src, dst)
         return dst
@@ -1049,7 +1062,7 @@ class TestConsumer(
     @mock.patch("documents.consumer.get_parser_registry")
     def test_similar_filenames(self, m) -> None:
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent / "samples" / "simple.pdf",
             settings.CONSUMPTION_DIR / "simple.pdf",
         )
         shutil.copy(
@@ -1590,7 +1603,13 @@ class TestConsumerRemoteOCR(
         self.addCleanup(patcher.stop)
 
     def _consume(self, *, overrides: DocumentMetadataOverrides | None = None) -> bool:
-        src = SIMPLE_DIGITAL_PDF
+        src = (
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf"
+        )
         dst = self.dirs.scratch_dir / "sample.pdf"
         shutil.copy(src, dst)
 

src/documents/tests/test_double_sided.py

@@ -37,15 +37,12 @@ class TestDoubleSided(
         self.double_sided_dir.mkdir()
         self.staging_file = self.dirs.scratch_dir / STAGING_FILE_NAME
 
-    def _sample(self, name: str) -> Path:
-        return self.SIMPLE_PDF if name == "simple.pdf" else self.SAMPLE_DIR / name
-
     def consume_file(self, srcname, dstname: str | Path = "foo.pdf"):
         """
         Starts the consume process and also ensures the
         destination file does not exist afterwards
         """
-        src = self._sample(srcname)
+        src = self.SAMPLE_DIR / srcname
         dst = self.double_sided_dir / dstname
         dst.parent.mkdir(parents=True, exist_ok=True)
         shutil.copy(src, dst)
@@ -60,7 +57,7 @@ class TestDoubleSided(
         return msg
 
     def create_staging_file(self, src="double-sided-odd.pdf", datetime=None) -> None:
-        shutil.copy(self._sample(src), self.staging_file)
+        shutil.copy(self.SAMPLE_DIR / src, self.staging_file)
         if datetime is None:
             datetime = dt.datetime.now()
         os.utime(str(self.staging_file), (datetime.timestamp(),) * 2)

src/documents/tests/test_management.py

@@ -22,9 +22,8 @@ from documents.models import Document
 from documents.tasks import update_document_content_maybe_archive_file
 from paperless_testing.assertions import FileSystemAssertsMixin
 from paperless_testing.dirs import DirectoriesMixin
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
 
-sample_file: Path = SIMPLE_DIGITAL_PDF
+sample_file: Path = Path(__file__).parent / "samples" / "simple.pdf"
 
 
 @pytest.mark.management

src/documents/tests/test_management_exporter.py

@@ -53,7 +53,6 @@ from paperless_testing.dirs import DirectoriesMixin
 from paperless_testing.dirs import paperless_environment
 from paperless_testing.factories import DocumentBarcodeFactory
 from paperless_testing.permissions import grant_object
-from paperless_testing.samples import install_document_samples
 
 
 @pytest.mark.management
@@ -199,7 +198,10 @@ class TestExportImport(
 
     def test_exporter(self, *, use_filename_format=False) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         num_permission_objects = Permission.objects.count()
 
@@ -303,7 +305,10 @@ class TestExportImport(
 
     def test_exporter_with_filename_format(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         with override_settings(
             FILENAME_FORMAT="{created_year}/{correspondent}/{title}",
@@ -312,7 +317,10 @@ class TestExportImport(
 
     def test_exporter_includes_share_links_and_bundles(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         share_link = ShareLink.objects.create(
             slug="share-link-slug",
@@ -411,7 +419,10 @@ class TestExportImport(
 
     def test_update_export_changed_time(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         self._do_export()
         self.assertIsFile(self.target / "manifest.json")
@@ -447,7 +458,10 @@ class TestExportImport(
 
     def test_update_export_changed_checksum(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         self._do_export()
 
@@ -474,7 +488,10 @@ class TestExportImport(
 
     def test_update_export_deleted_document(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         manifest = self._do_export()
 
@@ -506,7 +523,10 @@ class TestExportImport(
     @override_settings(FILENAME_FORMAT="{title}/{correspondent}")
     def test_update_export_changed_location(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         self._do_export(use_filename_format=True)
         self.assertIsFile(self.target / "wow1" / "c.pdf")
@@ -548,7 +568,10 @@ class TestExportImport(
             - Zipfile contains exported files
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         args = ["document_exporter", self.target, "--zip"]
 
@@ -577,7 +600,10 @@ class TestExportImport(
             - Zipfile contains exported files
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         args = ["document_exporter", self.target, "--zip", "--use-filename-format"]
 
@@ -613,7 +639,10 @@ class TestExportImport(
             - The existing file and directory in target are removed
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         # Create stuff in target directory
         existing_file = self.target / "test.txt"
@@ -709,7 +738,10 @@ class TestExportImport(
             - Documents can be imported again
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         manifest = self._do_export()
         has_archive = False
@@ -752,7 +784,10 @@ class TestExportImport(
             - Documents can be imported again
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         manifest = self._do_export()
         has_thumbnail = False
@@ -797,7 +832,10 @@ class TestExportImport(
             - Documents can be imported again
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         manifest = self._do_export(split_manifest=True)
         has_document = False
@@ -822,7 +860,10 @@ class TestExportImport(
 
     def _export_import_barcodes(self, *, split_manifest: bool) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
         DocumentBarcodeFactory(document=self.d1, value="https://example.com")
         DocumentBarcodeFactory(document=self.d2, page=2, value="DE8937")
 
@@ -858,7 +899,10 @@ class TestExportImport(
             - Documents can be imported again
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         self._do_export(use_folder_prefix=True)
 
@@ -885,7 +929,10 @@ class TestExportImport(
             - Documents can be imported again
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         self._do_export(use_folder_prefix=True, split_manifest=True)
 
@@ -912,7 +959,10 @@ class TestExportImport(
         """
 
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         num_content_type_objects = ContentType.objects.count()
         num_permission_objects = Permission.objects.count()
@@ -949,7 +999,10 @@ class TestExportImport(
 
     def test_exporter_with_auditlog_disabled(self) -> None:
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         with override_settings(
             AUDIT_LOG_ENABLED=False,
@@ -969,7 +1022,10 @@ class TestExportImport(
               survive the round-trip with deleted_at preserved
         """
         shutil.rmtree(Path(self.dirs.media_dir) / "documents")
-        install_document_samples(Path(self.dirs.media_dir) / "documents")
+        shutil.copytree(
+            Path(__file__).parent / "samples" / "documents",
+            Path(self.dirs.media_dir) / "documents",
+        )
 
         # d1 has self.note and self.cfi1 attached via setUp
         self.d1.delete()
@@ -1013,7 +1069,10 @@ class TestExportImport(
         """
 
         shutil.rmtree(self.dirs.media_dir / "documents")
-        install_document_samples(self.dirs.media_dir / "documents")
+        shutil.copytree(
+            self.SAMPLE_DIR / "documents",
+            self.dirs.media_dir / "documents",
+        )
 
         _ = self._do_export(data_only=True)
 

src/documents/tests/test_management_thumbnails.py

@@ -11,7 +11,6 @@ from documents.models import Document
 from documents.parsers import get_default_thumbnail
 from paperless_testing.assertions import FileSystemAssertsMixin
 from paperless_testing.dirs import DirectoriesMixin
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
 
 
 @pytest.mark.management
@@ -25,7 +24,7 @@ class TestMakeThumbnails(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
             filename="test.pdf",
         )
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent / "samples" / "simple.pdf",
             self.d1.source_path,
         )
 
@@ -37,7 +36,7 @@ class TestMakeThumbnails(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
             filename="test2.pdf",
         )
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent / "samples" / "simple.pdf",
             self.d2.source_path,
         )
 

src/documents/tests/test_parsers.py

@@ -1,11 +1,22 @@
+import subprocess
 from collections.abc import Generator
+from pathlib import Path
 
+import pikepdf
 import pytest
+from PIL import Image
 from pytest_django.fixtures import Settings
+from pytest_mock import MockerFixture
 
+from documents.parsers import ParseError
+from documents.parsers import _compute_thumbnail_dpi
+from documents.parsers import encode_thumbnail_webp
 from documents.parsers import get_default_file_extension
+from documents.parsers import get_default_thumbnail
 from documents.parsers import get_supported_file_extensions
 from documents.parsers import is_file_ext_supported
+from documents.parsers import make_thumbnail_from_pdf
+from documents.parsers import rasterize_pdf_page_to_png
 from paperless.parsers.registry import get_parser_registry
 from paperless.parsers.registry import reset_parser_registry
 from paperless.parsers.tesseract import RasterisedDocumentParser
@@ -125,3 +136,374 @@ class TestParserAvailability:
         assert is_file_ext_supported(".pdf")
         assert not is_file_ext_supported(".hsdfh")
         assert not is_file_ext_supported("")
+
+
+class TestComputeThumbnailDpi:
+    @pytest.mark.parametrize(
+        ("size", "expected"),
+        [
+            pytest.param((612.0, 792.0), (59, 2), id="letter-width-bound"),
+            pytest.param((792.0, 612.0), (46, 2), id="landscape-rounded-up"),
+            pytest.param((612.0, 100000.0), (4, 2), id="tall-strip-height-bound"),
+            pytest.param((200.0, 300.0), (180, 2), id="small-page-width-bound"),
+            pytest.param((72.0, 72.0), (300, 2), id="tiny-page-capped-at-300"),
+            pytest.param(
+                (1000000.0, 1000000.0),
+                (1, 1),
+                id="huge-page-minimum-one-unsupersampled",
+            ),
+            pytest.param(None, (150, 1), id="unreadable-geometry-fallback"),
+        ],
+    )
+    def test_dpi_from_page_size(
+        self,
+        mocker: MockerFixture,
+        tmp_path: Path,
+        size: tuple[float, float] | None,
+        expected: tuple[int, int],
+    ) -> None:
+        """
+        GIVEN:
+            - A first page of the given size, or unreadable geometry
+        WHEN:
+            - The thumbnail DPI is computed
+        THEN:
+            - The expected DPI and supersample factor are returned
+        """
+        mocker.patch(
+            "paperless.parsers.utils.get_pdf_first_page_size_points",
+            return_value=size,
+        )
+        assert _compute_thumbnail_dpi(tmp_path / "doc.pdf") == expected
+
+
+class TestMakeThumbnailFromPdf:
+    @pytest.fixture
+    def work_dir(self, tmp_path: Path) -> Path:
+        path = tmp_path / "work"
+        path.mkdir()
+        return path
+
+    @staticmethod
+    def _write_blank_pdf(path: Path, page_size: tuple[int, int] = (612, 792)) -> Path:
+        pdf = pikepdf.new()
+        pdf.add_blank_page(page_size=page_size)
+        pdf.save(path, object_stream_mode=pikepdf.ObjectStreamMode.disable)
+        return path
+
+    @pytest.mark.parametrize(
+        ("size", "expected_dpi", "expected_supersample"),
+        [
+            pytest.param((612.0, 792.0), 118, 2, id="known-geometry-2x"),
+            pytest.param(None, 150, 1, id="unreadable-geometry-plain-fallback"),
+        ],
+    )
+    def test_render_dpi_requested(
+        self,
+        mocker: MockerFixture,
+        tmp_path: Path,
+        work_dir: Path,
+        size: tuple[float, float] | None,
+        expected_dpi: int,
+        expected_supersample: int,
+    ) -> None:
+        """
+        GIVEN:
+            - Readable or unreadable page geometry
+        WHEN:
+            - A thumbnail is made
+        THEN:
+            - Rasterize and encode get the matching DPI and supersample factor
+        """
+        mocker.patch(
+            "paperless.parsers.utils.get_pdf_first_page_size_points",
+            return_value=size,
+        )
+        rasterize = mocker.patch("documents.parsers.rasterize_pdf_page_to_png")
+        encode = mocker.patch("documents.parsers.encode_thumbnail_webp")
+
+        make_thumbnail_from_pdf(tmp_path / "in.pdf", work_dir)
+
+        assert rasterize.call_args.kwargs["dpi"] == expected_dpi
+        assert encode.call_args.kwargs["supersample"] == expected_supersample
+
+    @pytest.mark.parametrize(
+        ("page_size", "expected_width"),
+        [
+            pytest.param((612, 792), 500, id="letter"),
+            pytest.param((792, 612), 500, id="landscape-letter"),
+            pytest.param((595, 842), 500, id="a4"),
+            pytest.param((200, 300), 500, id="small-page"),
+            pytest.param((72, 72), 300, id="tiny-page-capped"),
+        ],
+    )
+    def test_thumbnail_width(
+        self,
+        tmp_path: Path,
+        work_dir: Path,
+        page_size: tuple[int, int],
+        expected_width: int,
+    ) -> None:
+        """
+        GIVEN:
+            - A PDF whose first page has the given size in points
+        WHEN:
+            - A thumbnail is made from it
+        THEN:
+            - The thumbnail has the expected width
+        """
+        pdf_path = self._write_blank_pdf(tmp_path / "in.pdf", page_size)
+
+        thumb = make_thumbnail_from_pdf(pdf_path, work_dir)
+
+        assert thumb == work_dir / "convert.webp"
+        with Image.open(thumb) as im:
+            assert im.format == "WEBP"
+            assert im.width == expected_width
+
+    @classmethod
+    def _write_pdf_without_xref(cls, path: Path) -> Path:
+        """
+        Cuts off the xref and trailer, which pdftoppm cannot recover from but qpdf can.
+        """
+        cls._write_blank_pdf(path)
+        data = path.read_bytes()
+        path.write_bytes(data[: data.rindex(b"\nxref")])
+        return path
+
+    def test_qpdf_repair_produces_real_thumbnail(
+        self,
+        tmp_path: Path,
+        work_dir: Path,
+    ) -> None:
+        """
+        GIVEN:
+            - A PDF with its xref table and trailer cut off
+        WHEN:
+            - A thumbnail is made from it
+        THEN:
+            - The thumbnail is rendered from a qpdf repaired copy
+            - The original file is unchanged
+        """
+        pdf_path = self._write_pdf_without_xref(tmp_path / "broken.pdf")
+        original_bytes = pdf_path.read_bytes()
+
+        with pytest.raises(ParseError):
+            rasterize_pdf_page_to_png(pdf_path, work_dir / "probe.png", dpi=50)
+
+        thumb = make_thumbnail_from_pdf(pdf_path, work_dir)
+
+        assert thumb == work_dir / "convert_qpdf.webp"
+        with Image.open(thumb) as im:
+            assert im.format == "WEBP"
+            assert im.width == 500
+        assert pdf_path.read_bytes() == original_bytes
+
+    @pytest.mark.parametrize(
+        "qpdf_error",
+        [
+            pytest.param(subprocess.CalledProcessError(2, "qpdf"), id="qpdf-fails"),
+            pytest.param(None, id="repaired-still-unrenderable"),
+        ],
+    )
+    def test_double_failure_uses_default_thumbnail(
+        self,
+        mocker: MockerFixture,
+        tmp_path: Path,
+        work_dir: Path,
+        qpdf_error: subprocess.CalledProcessError | None,
+    ) -> None:
+        """
+        GIVEN:
+            - A PDF that cannot be rendered, even after qpdf repair
+        WHEN:
+            - A thumbnail is made from it
+        THEN:
+            - A copy of the default thumbnail is returned
+        """
+        mocker.patch(
+            "documents.parsers.rasterize_pdf_page_to_png",
+            side_effect=ParseError("Does not compute."),
+        )
+        if qpdf_error is not None:
+            mocker.patch("documents.parsers.run_subprocess", side_effect=qpdf_error)
+        pdf_path = self._write_blank_pdf(tmp_path / "in.pdf")
+
+        thumb = make_thumbnail_from_pdf(pdf_path, work_dir)
+
+        assert thumb == work_dir / "document.webp"
+        assert thumb.read_bytes() == get_default_thumbnail().read_bytes()
+
+
+class TestRasterizePdfPageToPng:
+    @staticmethod
+    def _write_pdf(
+        path: Path,
+        *,
+        crop_box: tuple[float, float, float, float] | None = None,
+        rotate: int | None = None,
+    ) -> Path:
+        pdf = pikepdf.new()
+        pdf.add_blank_page(page_size=(144, 72))
+        pdf.add_blank_page(page_size=(300, 300))
+        page = pdf.pages[0]
+        if crop_box is not None:
+            page.obj.CropBox = pikepdf.Array(crop_box)
+        if rotate is not None:
+            page.obj.Rotate = rotate
+        pdf.save(path)
+        return path
+
+    @pytest.mark.parametrize(
+        ("crop_box", "rotate", "expected_size"),
+        [
+            pytest.param(None, None, (144, 72), id="plain"),
+            pytest.param(None, 90, (72, 144), id="rotated-90"),
+            pytest.param((0, 0, 72, 36), None, (72, 36), id="crop-box"),
+        ],
+    )
+    def test_renders_first_page_to_exact_path(
+        self,
+        tmp_path: Path,
+        crop_box: tuple[float, float, float, float] | None,
+        rotate: int | None,
+        expected_size: tuple[int, int],
+    ) -> None:
+        """
+        GIVEN:
+            - A two page PDF, first page optionally cropped or rotated
+        WHEN:
+            - The first page is rasterized at 72 DPI
+        THEN:
+            - Only out_path is written, sized to the first page's crop and rotation
+        """
+        pdf_path = self._write_pdf(
+            tmp_path / "in.pdf",
+            crop_box=crop_box,
+            rotate=rotate,
+        )
+        out_dir = tmp_path / "out"
+        out_dir.mkdir()
+        out_path = out_dir / "page1.png"
+
+        rasterize_pdf_page_to_png(pdf_path, out_path, dpi=72)
+
+        assert list(out_dir.iterdir()) == [out_path]
+        with Image.open(out_path) as im:
+            assert im.format == "PNG"
+            assert im.size == expected_size
+
+    def test_failure_raises_parse_error(self, tmp_path: Path) -> None:
+        """
+        GIVEN:
+            - A file that is not a PDF
+        WHEN:
+            - Rasterization is attempted
+        THEN:
+            - A ParseError is raised
+        """
+        bad = tmp_path / "bad.pdf"
+        bad.write_bytes(b"not a pdf")
+
+        with pytest.raises(ParseError):
+            rasterize_pdf_page_to_png(bad, tmp_path / "page1.png", dpi=72)
+
+
+class TestEncodeThumbnailWebp:
+    @pytest.mark.parametrize(
+        ("mode", "color"),
+        [
+            pytest.param("RGBA", (0, 0, 0, 0), id="rgba"),
+            pytest.param("LA", (0, 0), id="la"),
+        ],
+    )
+    def test_alpha_flattened_onto_white(
+        self,
+        tmp_path: Path,
+        mode: str,
+        color: tuple[int, ...],
+    ) -> None:
+        """
+        GIVEN:
+            - A fully transparent PNG with an alpha channel
+        WHEN:
+            - It is encoded as a thumbnail
+        THEN:
+            - The WebP output is RGB with the transparency flattened to white
+        """
+        png_path = tmp_path / "in.png"
+        Image.new(mode, (20, 10), color).save(png_path)
+        out_path = tmp_path / "out.webp"
+
+        encode_thumbnail_webp(png_path, out_path)
+
+        with Image.open(out_path) as im:
+            assert im.format == "WEBP"
+            assert im.mode == "RGB"
+            assert im.size == (20, 10)
+            red, green, blue = im.getpixel((10, 5))
+            assert min(red, green, blue) >= 250
+
+    @pytest.mark.parametrize(
+        ("in_size", "supersample", "expected_size"),
+        [
+            pytest.param((1000, 2000), 1, (500, 1000), id="too-wide-shrunk"),
+            pytest.param((100, 10000), 1, (50, 5000), id="too-tall-shrunk"),
+            pytest.param((100, 200), 1, (100, 200), id="small-not-enlarged"),
+            pytest.param((1000, 1400), 2, (500, 700), id="2x-halved"),
+            pytest.param((1001, 1401), 2, (500, 700), id="2x-odd-rounded"),
+            pytest.param((600, 800), 2, (300, 400), id="2x-small-not-enlarged"),
+            pytest.param((900, 1200), 2, (450, 600), id="2x-below-clamp"),
+        ],
+    )
+    def test_size_clamped_and_downsampled(
+        self,
+        tmp_path: Path,
+        in_size: tuple[int, int],
+        supersample: int,
+        expected_size: tuple[int, int],
+    ) -> None:
+        """
+        GIVEN:
+            - A rendered image and its supersample factor
+        WHEN:
+            - It is encoded as a thumbnail with that factor
+        THEN:
+            - It is downsampled, fit within 500x5000 and never enlarged
+        """
+        png_path = tmp_path / "in.png"
+        Image.new("RGB", in_size, (255, 255, 255)).save(png_path)
+        out_path = tmp_path / "out.webp"
+
+        encode_thumbnail_webp(png_path, out_path, supersample=supersample)
+
+        with Image.open(out_path) as im:
+            assert im.size == expected_size
+
+    @pytest.mark.parametrize(
+        "error",
+        [
+            pytest.param(OSError("broken image"), id="os-error"),
+            pytest.param(Image.DecompressionBombError("too large"), id="bomb"),
+        ],
+    )
+    def test_decode_failure_raises_parse_error(
+        self,
+        tmp_path: Path,
+        mocker: MockerFixture,
+        error: Exception,
+    ) -> None:
+        """
+        GIVEN:
+            - Opening the rendered image fails
+        WHEN:
+            - It is encoded as a thumbnail
+        THEN:
+            - A ParseError is raised
+        """
+        png_path = tmp_path / "in.png"
+        Image.new("RGB", (10, 10)).save(png_path)
+        mocker.patch("PIL.Image.open", side_effect=error)
+
+        with pytest.raises(ParseError):
+            encode_thumbnail_webp(png_path, tmp_path / "out.webp")

src/documents/tests/test_permission_filtering_security.py

@@ -32,8 +32,6 @@ from paperless_testing.permissions import grant_global
 from paperless_testing.permissions import grant_object
 
 if TYPE_CHECKING:
-    from django.contrib.auth.models import User
-
     from paperless_testing.dirs import PaperlessDirs
 
 
@@ -180,296 +178,6 @@ class TestPermittedDocumentIdsIncludeDeleted:
         )
 
 
-@pytest.mark.django_db
-class TestPermittedDocumentIdsVersions:
-    """
-    A version is authorized by its root document: the version's own owner and
-    grants never matter.
-    """
-
-    @pytest.mark.parametrize(
-        ("root_owner", "version_owner", "expected_visible"),
-        [
-            pytest.param(
-                "other",
-                "nobody",
-                False,
-                id="unowned-version-of-private-root",
-            ),
-            pytest.param("other", "user", False, id="own-version-of-private-root"),
-            pytest.param("user", "other", True, id="foreign-version-of-own-root"),
-            pytest.param("user", "nobody", True, id="unowned-version-of-own-root"),
-            pytest.param("nobody", "other", True, id="private-version-of-unowned-root"),
-        ],
-    )
-    def test_version_follows_root_owner(
-        self,
-        root_owner: str,
-        version_owner: str,
-        *,
-        expected_visible: bool,
-    ) -> None:
-        """
-        GIVEN:
-            - A root document and a version with differing owners
-        WHEN:
-            - The permitted document ids are resolved for the user
-        THEN:
-            - The version is visible exactly when its root is
-        """
-        user = UserFactory()
-        owners = {"user": user, "other": UserFactory(), "nobody": None}
-        root = DocumentFactory(owner=owners[root_owner])
-        version = DocumentFactory(root_document=root, owner=owners[version_owner])
-
-        visible = set(permitted_document_ids(user))
-
-        assert (version.pk in visible) is expected_visible
-        assert (root.pk in visible) is expected_visible
-
-    @staticmethod
-    def grantee(user: User, kind: str) -> User | Group:
-        """The user itself, or a new group the user belongs to."""
-        if kind == "user":
-            return user
-        group = Group.objects.create(name="shared")
-        user.groups.add(group)
-        return group
-
-    @pytest.mark.parametrize(
-        "grantee_kind",
-        [pytest.param("user", id="user"), pytest.param("group", id="group")],
-    )
-    def test_grant_on_root_applies_to_version(self, grantee_kind: str) -> None:
-        """
-        GIVEN:
-            - A private root document shared with a user or one of their groups
-            - A version of it owned by someone else
-        WHEN:
-            - The permitted document ids are resolved for the user
-        THEN:
-            - Both the root and the version are visible
-            - A user without the grant sees neither
-        """
-        user = UserFactory()
-        stranger = UserFactory()
-        root = DocumentFactory(owner=UserFactory())
-        version = DocumentFactory(root_document=root, owner=UserFactory())
-        grant_object(self.grantee(user, grantee_kind), root, "view_document")
-
-        assert_visible_document_ids(
-            permitted_document_ids(user),
-            expected_visible=[root.pk, version.pk],
-            expected_hidden=[],
-        )
-        assert_visible_document_ids(
-            permitted_document_ids(stranger),
-            expected_visible=[],
-            expected_hidden=[root.pk, version.pk],
-        )
-
-    @pytest.mark.parametrize(
-        "grantee_kind",
-        [pytest.param("user", id="user"), pytest.param("group", id="group")],
-    )
-    def test_grant_on_version_is_ignored(self, grantee_kind: str) -> None:
-        """
-        GIVEN:
-            - A private root document
-            - A version with an explicit grant for the user or one of their groups
-        WHEN:
-            - The permitted document ids are resolved for the user
-        THEN:
-            - Neither the root nor the version is visible
-        """
-        user = UserFactory()
-        root = DocumentFactory(owner=UserFactory())
-        version = DocumentFactory(root_document=root, owner=UserFactory())
-        grant_object(self.grantee(user, grantee_kind), version, "view_document")
-
-        assert_visible_document_ids(
-            permitted_document_ids(user),
-            expected_visible=[],
-            expected_hidden=[root.pk, version.pk],
-        )
-
-    def test_grant_on_one_root_does_not_reach_another_roots_version(self) -> None:
-        """
-        GIVEN:
-            - Two private roots, each with a version
-            - The user may view only the first root
-        WHEN:
-            - The permitted document ids are resolved for the user
-        THEN:
-            - Only the first root and its version are visible
-        """
-        user = UserFactory()
-        first = DocumentFactory(owner=UserFactory())
-        first_version = DocumentFactory(root_document=first, owner=UserFactory())
-        second = DocumentFactory(owner=UserFactory())
-        second_version = DocumentFactory(root_document=second, owner=user)
-        grant_object(user, first, "view_document")
-
-        assert_visible_document_ids(
-            permitted_document_ids(user),
-            expected_visible=[first.pk, first_version.pk],
-            expected_hidden=[second.pk, second_version.pk],
-        )
-
-    def test_user_in_several_groups(self) -> None:
-        """
-        GIVEN:
-            - A user in two groups
-            - Two private roots shared with one group each, and a third shared with nobody
-            - A version of each root
-        WHEN:
-            - The permitted document ids are resolved for the user
-        THEN:
-            - The two shared roots and their versions are visible
-            - The third root and its version are not
-        """
-        user = UserFactory()
-        groups = [Group.objects.create(name=f"group{i}") for i in range(2)]
-        user.groups.add(*groups)
-        shared = [DocumentFactory(owner=UserFactory()) for _ in groups]
-        for root, group in zip(shared, groups, strict=True):
-            grant_object(group, root, "view_document")
-        unshared = DocumentFactory(owner=UserFactory())
-        shared_versions = [
-            DocumentFactory(root_document=root, owner=UserFactory()) for root in shared
-        ]
-        unshared_version = DocumentFactory(root_document=unshared, owner=None)
-
-        assert_visible_document_ids(
-            permitted_document_ids(user),
-            expected_visible=[
-                *(root.pk for root in shared),
-                *(version.pk for version in shared_versions),
-            ],
-            expected_hidden=[unshared.pk, unshared_version.pk],
-        )
-
-    def test_permission_is_resolved_through_the_root(self) -> None:
-        """
-        GIVEN:
-            - A private root document where the user may view and change
-        WHEN:
-            - The permitted ids are resolved for view, change and delete
-        THEN:
-            - The version is visible for view and change only
-        """
-        user = UserFactory()
-        root = DocumentFactory(owner=UserFactory())
-        version = DocumentFactory(root_document=root, owner=UserFactory())
-        grant_object(user, root, "view_document", "change_document")
-
-        assert version.pk in set(permitted_document_ids(user))
-        assert version.pk in set(permitted_document_ids(user, perm="change_document"))
-        assert version.pk in set(
-            permitted_document_ids(user, perm="documents.change_document"),
-        )
-        assert version.pk not in set(
-            permitted_document_ids(user, perm="delete_document"),
-        )
-
-    def test_anonymous_sees_versions_of_unowned_roots_only(self) -> None:
-        """
-        GIVEN:
-            - A version owned by nobody under a private root
-            - A version owned by someone under an unowned root
-        WHEN:
-            - The permitted document ids are resolved for an anonymous user
-        THEN:
-            - Only the version of the unowned root is visible
-        """
-        private_root = DocumentFactory(owner=UserFactory())
-        private_version = DocumentFactory(root_document=private_root, owner=None)
-        open_root = DocumentFactory(owner=None)
-        open_version = DocumentFactory(root_document=open_root, owner=UserFactory())
-
-        assert_visible_document_ids(
-            permitted_document_ids(AnonymousUser()),
-            expected_visible=[open_root.pk, open_version.pk],
-            expected_hidden=[private_root.pk, private_version.pk],
-        )
-
-    def test_deleted_versions_follow_their_deleted_root(self) -> None:
-        """
-        GIVEN:
-            - A soft-deleted root document and its version, which deleting the
-              root soft-deletes too; the version is owned by someone else
-        WHEN:
-            - The permitted document ids are resolved with and without deleted
-              documents
-        THEN:
-            - Nothing is visible by default
-            - With deleted documents included, the version is visible to the
-              root's owner and not to the version's own owner
-        """
-        owner = UserFactory()
-        version_owner = UserFactory()
-        root = DocumentFactory(owner=owner)
-        version = DocumentFactory(root_document=root, owner=version_owner)
-        root.delete()
-
-        assert not {root.pk, version.pk} & set(permitted_document_ids(owner))
-        assert_visible_document_ids(
-            permitted_document_ids(owner, include_deleted=True),
-            expected_visible=[root.pk, version.pk],
-            expected_hidden=[],
-        )
-        assert_visible_document_ids(
-            permitted_document_ids(version_owner, include_deleted=True),
-            expected_visible=[],
-            expected_hidden=[root.pk, version.pk],
-        )
-
-    @pytest.mark.parametrize(
-        "is_superuser",
-        [
-            pytest.param(False, id="regular-user"),
-            pytest.param(True, id="superuser"),
-        ],
-    )
-    def test_inactive_user_sees_no_versions(self, *, is_superuser: bool) -> None:
-        """
-        GIVEN:
-            - An inactive user, possibly a superuser, who owns a root and its version
-        WHEN:
-            - The permitted document ids are resolved for them
-        THEN:
-            - Nothing is visible
-        """
-        user = UserFactory(is_active=False, is_superuser=is_superuser)
-        root = DocumentFactory(owner=user)
-        version = DocumentFactory(root_document=root, owner=user)
-
-        assert_visible_document_ids(
-            permitted_document_ids(user),
-            expected_visible=[],
-            expected_hidden=[root.pk, version.pk],
-        )
-
-    def test_superuser_sees_all_versions(self) -> None:
-        """
-        GIVEN:
-            - A private root owned by someone else, with a version
-        WHEN:
-            - The permitted document ids are resolved for a superuser
-        THEN:
-            - Both the root and the version are visible
-        """
-        superuser = UserFactory(superuser=True)
-        root = DocumentFactory(owner=UserFactory())
-        version = DocumentFactory(root_document=root, owner=UserFactory())
-
-        assert_visible_document_ids(
-            permitted_document_ids(superuser),
-            expected_visible=[root.pk, version.pk],
-            expected_hidden=[],
-        )
-
-
 @pytest.mark.django_db
 class TestAiChatAllDocumentsPermissionBoundary:
     """

src/documents/tests/test_sample_files.py

@@ -1,61 +0,0 @@
-import hashlib
-from collections import defaultdict
-from pathlib import Path
-
-from paperless_testing.samples import SHARED_SAMPLES_DIR
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
-from paperless_testing.samples import install_document_samples
-
-EXPECTED_MEDIA_FILES = [
-    "originals/0000001.pdf",
-    "originals/0000002.pdf",
-    "originals/0000003.pdf",
-    "originals/0000004.pdf",
-    "originals/0000005.pdf",
-    "originals/0000006.pdf",
-    "archive/0000001.pdf",
-    "thumbnails/0000001.webp",
-    "thumbnails/0000002.webp",
-    "thumbnails/0000003.webp",
-    "thumbnails/0000004.webp",
-]
-
-
-def _sha(path: Path) -> str:
-    return hashlib.sha256(path.read_bytes()).hexdigest()
-
-
-def test_install_recreates_every_historical_opaque_name(tmp_path: Path) -> None:
-    install_document_samples(tmp_path)
-    for relpath in EXPECTED_MEDIA_FILES:
-        assert (tmp_path / relpath).is_file(), relpath
-
-
-def test_install_uses_the_shared_bytes(tmp_path: Path) -> None:
-    install_document_samples(tmp_path)
-    assert _sha(tmp_path / "originals" / "0000001.pdf") == _sha(SIMPLE_DIGITAL_PDF)
-
-
-APP_SAMPLE_TREES = [
-    Path(__file__).parent / "samples",
-    Path(__file__).parents[2] / "paperless" / "tests" / "samples",
-    SHARED_SAMPLES_DIR,
-]
-
-
-def test_no_two_sample_files_have_identical_bytes() -> None:
-    by_hash: dict[str, list[Path]] = defaultdict(list)
-    for tree in APP_SAMPLE_TREES:
-        assert tree.is_dir(), f"Sample tree does not exist: {tree}"
-        for path in tree.rglob("*"):
-            # Skip empty placeholders, which would all hash identically
-            if path.is_file() and path.stat().st_size > 0:
-                by_hash[_sha(path)].append(path)
-    assert by_hash, "No sample files were found to compare"
-    duplicates = {h: ps for h, ps in by_hash.items() if len(ps) > 1}
-    lines = ["  " + ", ".join(str(p) for p in ps) for ps in duplicates.values()]
-    message = (
-        "Duplicate sample files, move one to paperless_testing/sample_files:\n"
-        + "\n".join(lines)
-    )
-    assert not duplicates, message

src/documents/tests/test_tasks.py

@@ -21,7 +21,6 @@ from documents.tests.helpers import dummy_preprocess
 from paperless_testing.assertions import FileSystemAssertsMixin
 from paperless_testing.dirs import DirectoriesMixin
 from paperless_testing.factories import DocumentFactory
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
 
 
 @pytest.mark.django_db
@@ -230,12 +229,20 @@ class TestUpdateContent(DirectoriesMixin, TestCase):
         """
         sample1 = self.dirs.scratch_dir / "sample.pdf"
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
             sample1,
         )
         sample1_archive = self.dirs.archive_dir / "sample_archive.pdf"
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
             sample1_archive,
         )
         doc = Document.objects.create(
@@ -263,7 +270,11 @@ class TestUpdateContent(DirectoriesMixin, TestCase):
         """
         sample1 = self.dirs.scratch_dir / "sample.pdf"
         shutil.copy(
-            SIMPLE_DIGITAL_PDF,
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
             sample1,
         )
         doc = Document.objects.create(
@@ -279,7 +290,14 @@ class TestUpdateContent(DirectoriesMixin, TestCase):
 
     def _create_root_with_version(self) -> tuple[Document, Document]:
         sample1 = self.dirs.scratch_dir / "sample.pdf"
-        shutil.copy(SIMPLE_DIGITAL_PDF, sample1)
+        shutil.copy(
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf",
+            sample1,
+        )
         root = DocumentFactory(content="root content", mime_type="application/pdf")
         version = DocumentFactory(
             content="my document",

src/documents/tests/test_workflows.py

@@ -178,7 +178,7 @@ class TestWorkflows(
         self.assertEqual(action.__str__(), "WorkflowAction 1")
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -290,7 +290,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -409,7 +409,7 @@ class TestWorkflows(
         w2.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -478,7 +478,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -531,7 +531,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -608,7 +608,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -687,7 +687,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -765,7 +765,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -876,7 +876,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -928,7 +928,7 @@ class TestWorkflows(
         generated = generate_unique_filename(doc)
         destination = (settings.ORIGINALS_DIR / generated).resolve()
         create_source_path_directory(destination)
-        shutil.copy(self.SIMPLE_PDF, destination)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", destination)
         Document.objects.filter(pk=doc.pk).update(filename=generated.as_posix())
         doc.refresh_from_db()
 
@@ -2023,7 +2023,7 @@ class TestWorkflows(
         superuser = UserFactory(username="superuser", superuser=True)
         self.client.force_authenticate(user=superuser)
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
         tasks.consume_file(
@@ -3009,7 +3009,7 @@ class TestWorkflows(
         generated = generate_unique_filename(doc)
         destination = (settings.ORIGINALS_DIR / generated).resolve()
         create_source_path_directory(destination)
-        shutil.copy(self.SIMPLE_PDF, destination)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", destination)
         Document.objects.filter(pk=doc.pk).update(filename=generated.as_posix())
         doc.refresh_from_db()
         doc.tags.set([self.t1, self.t2])
@@ -3073,7 +3073,7 @@ class TestWorkflows(
         generated = generate_unique_filename(doc)
         destination = (settings.ORIGINALS_DIR / generated).resolve()
         create_source_path_directory(destination)
-        shutil.copy(self.SIMPLE_PDF, destination)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", destination)
         Document.objects.filter(pk=doc.pk).update(filename=generated.as_posix())
         doc.refresh_from_db()
         doc.tags.set([self.t1])
@@ -3222,7 +3222,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -3346,7 +3346,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -3496,7 +3496,7 @@ class TestWorkflows(
         workflow.actions.set([assignment_action, email_action])
 
         temp_working_copy = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "working-copy.pdf",
         )
 
@@ -3596,7 +3596,7 @@ class TestWorkflows(
 
         # move the file
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -3704,7 +3704,7 @@ class TestWorkflows(
         generated = generate_unique_filename(doc)
         destination = (settings.ORIGINALS_DIR / generated).resolve()
         create_source_path_directory(destination)
-        shutil.copy(self.SIMPLE_PDF, destination)
+        shutil.copy(self.SAMPLE_DIR / "simple.pdf", destination)
         Document.objects.filter(pk=doc.pk).update(filename=generated.as_posix())
 
         run_workflows(WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED, doc)
@@ -3915,7 +3915,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -4036,7 +4036,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -4387,7 +4387,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -4835,7 +4835,7 @@ class TestWorkflows(
         w.save()
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 
@@ -5027,7 +5027,7 @@ class TestWorkflows(
 
         # Create a test file to be consumed
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
         test_file_path = Path(test_file)
@@ -5091,7 +5091,7 @@ class TestWorkflows(
 
         # Create a test file to be consumed
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple2.pdf",
         )
         test_file_path = Path(test_file)
@@ -5529,7 +5529,7 @@ class TestDateWorkflowLocalization(
         )
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             tmp_path / "simple.pdf",
         )
 
@@ -5596,7 +5596,7 @@ class TestRemoteOCRWorkflowAction(DirectoriesMixin, SampleDirMixin, APITestCase)
         self._make_workflow(WorkflowTrigger.WorkflowTriggerType.CONSUMPTION)
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
         overrides = DocumentMetadataOverrides()
@@ -5814,7 +5814,7 @@ class TestApplyAISuggestionsWorkflowAction(
         )
 
         test_file = shutil.copy(
-            self.SIMPLE_PDF,
+            self.SAMPLE_DIR / "simple.pdf",
             self.dirs.scratch_dir / "simple.pdf",
         )
 

src/documents/tests/utils.py

@@ -11,7 +11,6 @@ from documents.data_models import ConsumableDocument
 from documents.data_models import DocumentMetadataOverrides
 from documents.data_models import DocumentSource
 from paperless_testing.fakes.progress import FakeProgressManager
-from paperless_testing.samples import SIMPLE_DIGITAL_PDF
 
 
 class ConsumeTaskMixin:
@@ -54,8 +53,6 @@ class SampleDirMixin:
 
     BARCODE_SAMPLE_DIR = SAMPLE_DIR / "barcodes"
 
-    SIMPLE_PDF = SIMPLE_DIGITAL_PDF
-
 
 class GetConsumerMixin:
     @contextmanager

src/documents/views.py

@@ -174,6 +174,7 @@ from documents.permissions import TrashPermissions
 from documents.permissions import ViewDocumentsPermissions
 from documents.permissions import annotate_document_count_by_ids
 from documents.permissions import annotate_document_count_for_related_queryset
+from documents.permissions import documents_without_permitted_root
 from documents.permissions import get_document_count_filter_for_user
 from documents.permissions import get_objects_for_user_owner_aware
 from documents.permissions import has_global_statistics_permission
@@ -2116,7 +2117,7 @@ class DocumentViewSet(
         documents = Document.objects.filter(pk__in=document_ids)
         if (
             request.user is not None
-            and documents.exclude(id__in=permitted_document_ids(request.user)).exists()
+            and documents_without_permitted_root(documents, request.user).exists()
         ):
             return HttpResponseForbidden("Insufficient permissions")
 
@@ -3008,8 +3009,12 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
             user.has_perm(
                 "documents.change_document",
             )
-            and not Document.global_objects.filter(pk__in=documents)
-            .exclude(pk__in=permitted_document_ids(user, perm="change_document"))
+            and not Document.global_objects.filter(
+                pk__in=[doc.pk for doc in root_docs],
+            )
+            .exclude(
+                pk__in=permitted_document_ids(user, perm="change_document"),
+            )
             .exists()
         )
 
@@ -3622,7 +3627,7 @@ class SelectionDataView(DocumentSelectionMixin, GenericAPIView[Any]):
         documents = Document.objects.filter(pk__in=ids)
         if (
             documents.count() != len(ids)
-            or documents.exclude(id__in=permitted_document_ids(request.user)).exists()
+            or documents_without_permitted_root(documents, request.user).exists()
         ):
             return HttpResponseForbidden("Insufficient permissions")
 
@@ -4119,16 +4124,21 @@ class BulkDownloadView(DocumentSelectionMixin, GenericAPIView[Any]):
             validated_data=serializer.validated_data,
         )
         documents = Document.objects.filter(pk__in=ids)
+        versioned_documents = []
         compression = serializer.validated_data.get("compression")
         content = serializer.validated_data.get("content")
         follow_filename_format = serializer.validated_data.get("follow_formatting")
 
-        if documents.exclude(id__in=permitted_document_ids(request.user)).exists():
-            return HttpResponseForbidden("Insufficient permissions")
-        versioned_documents = [
-            get_latest_version_for_root(get_root_document(document))
-            for document in documents
-        ]
+        permitted_ids = set(permitted_document_ids(request.user))
+        for document in documents:
+            root_doc = get_root_document(document)
+            if root_doc.pk not in permitted_ids:
+                return HttpResponseForbidden("Insufficient permissions")
+            versioned_documents.append(
+                get_latest_version_for_root(
+                    root_doc,
+                ),
+            )
 
         if content == "both":
             strategy_class = OriginalAndArchiveStrategy
@@ -4801,7 +4811,7 @@ class ShareLinkBundleViewSet(PassUserMixin, ModelViewSet[ShareLinkBundle]):
             )
 
         denied_id = (
-            documents_qs.exclude(id__in=permitted_document_ids(request.user))
+            documents_without_permitted_root(documents_qs, request.user)
             .order_by("pk")
             .values_list("pk", flat=True)
             .first()
@@ -5647,12 +5657,11 @@ class TrashView(ListModelMixin, PassUserMixin):
             if doc_ids is not None
             else self.filter_queryset(self.get_queryset()).all()
         )
-        if docs.exclude(
-            id__in=permitted_document_ids(
-                request.user,
-                perm="delete_document",
-                include_deleted=True,
-            ),
+        if documents_without_permitted_root(
+            docs,
+            request.user,
+            perm="delete_document",
+            include_deleted=True,
         ).exists():
             return HttpResponseForbidden("Insufficient permissions")
         action = serializer.validated_data.get("action")

src/locale/en_US/LC_MESSAGES/django.po

@@ -2,7 +2,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: paperless-ngx\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2026-10-10 22:04+0000\n"
+"POT-Creation-Date: 2026-10-08 18:39+0000\n"
 "PO-Revision-Date: 2022-02-17 04:17\n"
 "Last-Translator: \n"
 "Language-Team: English\n"
@@ -1653,7 +1653,7 @@ msgid "workflow runs"
 msgstr ""
 
 #: documents/serialisers.py:516 documents/serialisers.py:873
-#: documents/serialisers.py:2903 documents/views.py:343 documents/views.py:2750
+#: documents/serialisers.py:2903 documents/views.py:344 documents/views.py:2751
 #: paperless_mail/serialisers.py:156
 msgid "Insufficient permissions."
 msgstr ""
@@ -1694,7 +1694,7 @@ msgstr ""
 msgid "Duplicate document identifiers are not allowed."
 msgstr ""
 
-#: documents/serialisers.py:2989 documents/views.py:4797
+#: documents/serialisers.py:2989 documents/views.py:4807
 #, python-format
 msgid "Documents not found: %(ids)s"
 msgstr ""
@@ -1945,40 +1945,40 @@ msgstr ""
 msgid ", "
 msgstr ""
 
-#: documents/views.py:336 documents/views.py:2747
+#: documents/views.py:337 documents/views.py:2748
 msgid "Invalid more_like_id"
 msgstr ""
 
-#: documents/views.py:1682
+#: documents/views.py:1683
 msgid "Invalid AI configuration."
 msgstr ""
 
-#: documents/views.py:1693
+#: documents/views.py:1694
 msgid "AI backend request timed out."
 msgstr ""
 
-#: documents/views.py:1705
+#: documents/views.py:1706
 msgid "AI backend rejected the request. Check logs for details."
 msgstr ""
 
-#: documents/views.py:2572 documents/views.py:2888
+#: documents/views.py:2573 documents/views.py:2889
 msgid "Specify only one of text, title_search, query, or more_like_id."
 msgstr ""
 
-#: documents/views.py:4813
+#: documents/views.py:4823
 #, python-format
 msgid "Insufficient permissions to share document %(id)s."
 msgstr ""
 
-#: documents/views.py:4860
+#: documents/views.py:4870
 msgid "Bundle is already being processed."
 msgstr ""
 
-#: documents/views.py:4924
+#: documents/views.py:4934
 msgid "The share link bundle is still being prepared. Please try again later."
 msgstr ""
 
-#: documents/views.py:4938
+#: documents/views.py:4948
 msgid "The share link bundle is unavailable."
 msgstr ""
 

src/paperless/parsers/utils.py

@@ -265,6 +265,52 @@ def get_page_count_for_pdf(
         return None
 
 
+def get_pdf_first_page_size_points(
+    path: Path,
+    log: logging.Logger | None = None,
+) -> tuple[float, float] | None:
+    """Return the first page's (width, height) in PDF points, post-rotation.
+
+    Uses the CropBox (MediaBox if absent), which must match pdftoppm's
+    ``-cropbox`` or the computed DPI targets the wrong box.
+
+    Swaps width and height for 90/270 rotation. ``page.rotation`` resolves
+    inherited and negative ``/Rotate`` values, a raw lookup does not.
+
+    Parameters
+    ----------
+    path:
+        Absolute path to the PDF file.
+    log:
+        Logger for warnings.  Falls back to the module-level logger when omitted.
+
+    Returns
+    -------
+    tuple[float, float] | None
+        ``(width_points, height_points)``, or ``None`` if the file cannot be
+        opened, has no pages, or the page box is degenerate.
+    """
+    import pikepdf
+
+    _log = log or logger
+    try:
+        with pikepdf.Pdf.open(path) as pdf:
+            if len(pdf.pages) == 0:
+                return None
+            page = pdf.pages[0]
+            llx, lly, urx, ury = (float(v) for v in page.cropbox)
+            width = abs(urx - llx)
+            height = abs(ury - lly)
+            if width <= 0 or height <= 0:
+                return None
+            if page.rotation in (90, 270):
+                width, height = height, width
+            return width, height
+    except Exception as e:
+        _log.warning("Could not determine PDF page size for %s: %s", path, e)
+        return None
+
+
 def extract_pdf_metadata(
     document_path: Path,
     log: logging.Logger | None = None,

src/paperless/settings/__init__.py

@@ -986,8 +986,6 @@ GNUPG_HOME = os.getenv("HOME", "/tmp")
 
 # Convert is part of the ImageMagick package
 CONVERT_BINARY = os.getenv("PAPERLESS_CONVERT_BINARY", "convert")
-CONVERT_TMPDIR = os.getenv("PAPERLESS_CONVERT_TMPDIR")
-CONVERT_MEMORY_LIMIT = os.getenv("PAPERLESS_CONVERT_MEMORY_LIMIT")
 
 GS_BINARY = os.getenv("PAPERLESS_GS_BINARY", "gs")
 

src/paperless/tests/parsers/conftest.py

@@ -432,6 +432,30 @@ def tesseract_samples_dir(parser_samples_dir: Path) -> Path:
     return parser_samples_dir / "tesseract"
 
 
+@pytest.fixture(scope="session")
+def multi_page_images_pdf_file(tesseract_samples_dir: Path) -> Path:
+    """Path to a multi-page PDF with images.
+
+    Returns
+    -------
+    Path
+        Absolute path to ``tesseract/multi-page-images.pdf``.
+    """
+    return tesseract_samples_dir / "multi-page-images.pdf"
+
+
+@pytest.fixture(scope="session")
+def simple_digital_pdf_file(tesseract_samples_dir: Path) -> Path:
+    """Path to a simple digital PDF sample file.
+
+    Returns
+    -------
+    Path
+        Absolute path to ``tesseract/simple-digital.pdf``.
+    """
+    return tesseract_samples_dir / "simple-digital.pdf"
+
+
 @pytest.fixture(scope="session")
 def simple_no_dpi_png_file(tesseract_samples_dir: Path) -> Path:
     """Path to a simple PNG without DPI information.

src/paperless/tests/parsers/test_tesseract_parser.py

@@ -15,9 +15,10 @@ from typing import TYPE_CHECKING
 
 import pytest
 from ocrmypdf import SubprocessOutputError
+from PIL import Image
 
 from documents.parsers import ParseError
-from documents.parsers import run_convert
+from documents.parsers import rasterize_pdf_page_to_png
 from paperless.models import ModeChoices
 from paperless.parsers import ParserProtocol
 from paperless.parsers.tesseract import RasterisedDocumentParser
@@ -160,11 +161,11 @@ class TestGetPageCount:
     def test_single_page_pdf(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         assert (
             tesseract_parser.get_page_count(
-                simple_digital_pdf_file,
+                tesseract_samples_dir / "simple-digital.pdf",
                 "application/pdf",
             )
             == 1
@@ -187,7 +188,7 @@ class TestGetPageCount:
         self,
         mocker: MockerFixture,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
         caplog,
     ) -> None:
         """
@@ -203,7 +204,7 @@ class TestGetPageCount:
 
         with caplog.at_level(logging.WARNING):
             page_count = tesseract_parser.get_page_count(
-                simple_digital_pdf_file,
+                tesseract_samples_dir / "simple-digital.pdf",
                 "application/pdf",
             )
         assert page_count is None
@@ -272,32 +273,79 @@ class TestGetThumbnail:
     def test_thumbnail_is_file(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         thumb = tesseract_parser.get_thumbnail(
-            simple_digital_pdf_file,
+            tesseract_samples_dir / "simple-digital.pdf",
             "application/pdf",
         )
         assert thumb.is_file()
 
-    def test_thumbnail_fallback_on_convert_error(
+    @pytest.mark.parametrize(
+        ("filename", "expected_height"),
+        [
+            pytest.param("simple-digital.pdf", 647, id="portrait-letter"),
+            pytest.param("rotated.pdf", 386, id="landscape"),
+        ],
+    )
+    def test_thumbnail_is_correct_format_and_size(
+        self,
+        tesseract_parser: RasterisedDocumentParser,
+        tesseract_samples_dir: Path,
+        filename: str,
+        expected_height: int,
+    ) -> None:
+        """
+        GIVEN:
+            - A portrait or landscape PDF
+        WHEN:
+            - A thumbnail is generated
+        THEN:
+            - A 500px wide WebP keeping the page's aspect ratio
+        """
+        thumb = tesseract_parser.get_thumbnail(
+            tesseract_samples_dir / filename,
+            "application/pdf",
+        )
+        with Image.open(thumb) as im:
+            assert im.format == "WEBP"
+            assert im.width == 500
+            assert im.height == pytest.approx(expected_height, abs=2)
+
+    def test_thumbnail_fallback_on_pdftoppm_error(
         self,
         mocker: MockerFixture,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
-        def _raise_on_pdf(input_file, output_file, **kwargs) -> None:
-            if ".pdf" in str(input_file):
+        """
+        GIVEN:
+            - Rasterizing the original PDF fails
+        WHEN:
+            - A thumbnail is generated
+        THEN:
+            - The PDF is repaired with qpdf and a real thumbnail is rendered
+        """
+        original = tesseract_samples_dir / "simple-digital.pdf"
+
+        def _fail_on_original(in_path: Path, out_path: Path, **kwargs) -> None:
+            if in_path == original:
                 raise ParseError("Does not compute.")
-            run_convert(input_file=input_file, output_file=output_file, **kwargs)
+            rasterize_pdf_page_to_png(in_path, out_path, **kwargs)
 
-        mocker.patch("documents.parsers.run_convert", side_effect=_raise_on_pdf)
-
-        thumb = tesseract_parser.get_thumbnail(
-            simple_digital_pdf_file,
-            "application/pdf",
+        rasterize = mocker.patch(
+            "documents.parsers.rasterize_pdf_page_to_png",
+            side_effect=_fail_on_original,
         )
+
+        thumb = tesseract_parser.get_thumbnail(original, "application/pdf")
+
+        assert rasterize.call_count == 2
         assert thumb.is_file()
+        assert thumb.name == "convert_qpdf.webp"
+        with Image.open(thumb) as im:
+            assert im.format == "WEBP"
+            assert im.width == 500
 
     def test_thumbnail_encrypted_pdf(
         self,
@@ -320,11 +368,11 @@ class TestExtractText:
     def test_extract_text_from_digital_pdf(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         text = tesseract_parser.extract_text(
             None,
-            simple_digital_pdf_file,
+            tesseract_samples_dir / "simple-digital.pdf",
         )
         assert text is not None
         assert "This is a test document." in text.strip()
@@ -339,7 +387,7 @@ class TestParsePdf:
     def test_simple_digital_creates_archive(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -353,7 +401,7 @@ class TestParsePdf:
             - Text is extracted
         """
         tesseract_parser.parse(
-            multi_page_digital_pdf_file,
+            tesseract_samples_dir / "multi-page-digital.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -368,10 +416,10 @@ class TestParsePdf:
     def test_with_form_default(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        with_form_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         tesseract_parser.parse(
-            with_form_pdf_file,
+            tesseract_samples_dir / "with-form.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -384,11 +432,11 @@ class TestParsePdf:
     def test_with_form_redo_no_archive_when_not_requested(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        with_form_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         tesseract_parser.settings.mode = ModeChoices.REDO
         tesseract_parser.parse(
-            with_form_pdf_file,
+            tesseract_samples_dir / "with-form.pdf",
             "application/pdf",
             produce_archive=False,
         )
@@ -401,11 +449,11 @@ class TestParsePdf:
     def test_with_form_force(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        with_form_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         tesseract_parser.settings.mode = ModeChoices.FORCE
         tesseract_parser.parse(
-            with_form_pdf_file,
+            tesseract_samples_dir / "with-form.pdf",
             "application/pdf",
         )
         assert_ordered_substrings(
@@ -446,7 +494,7 @@ class TestParsePdf:
         self,
         mocker: MockerFixture,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         mocker.patch(
             "ocrmypdf.ocr",
@@ -454,7 +502,7 @@ class TestParsePdf:
         )
         with pytest.raises(ParseError):
             tesseract_parser.parse(
-                simple_digital_pdf_file,
+                tesseract_samples_dir / "simple-digital.pdf",
                 "application/pdf",
             )
 
@@ -530,10 +578,10 @@ class TestParseMultiPage:
     def test_multi_page_digital(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         tesseract_parser.parse(
-            multi_page_digital_pdf_file,
+            tesseract_samples_dir / "multi-page-digital.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -557,12 +605,12 @@ class TestParseMultiPage:
         self,
         mode: str,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         tesseract_parser.settings.pages = 2
         tesseract_parser.settings.mode = mode
         tesseract_parser.parse(
-            multi_page_digital_pdf_file,
+            tesseract_samples_dir / "multi-page-digital.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -576,11 +624,11 @@ class TestParseMultiPage:
     def test_multi_page_images_skip(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_images_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         tesseract_parser.settings.mode = ModeChoices.AUTO
         tesseract_parser.parse(
-            multi_page_images_pdf_file,
+            tesseract_samples_dir / "multi-page-images.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -594,7 +642,7 @@ class TestParseMultiPage:
     def test_multi_page_images_redo_pages_2(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_images_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -609,7 +657,7 @@ class TestParseMultiPage:
         tesseract_parser.settings.pages = 2
         tesseract_parser.settings.mode = ModeChoices.REDO
         tesseract_parser.parse(
-            multi_page_images_pdf_file,
+            tesseract_samples_dir / "multi-page-images.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -622,7 +670,7 @@ class TestParseMultiPage:
     def test_multi_page_images_force_page_1(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_images_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -637,7 +685,7 @@ class TestParseMultiPage:
         tesseract_parser.settings.pages = 1
         tesseract_parser.settings.mode = ModeChoices.FORCE
         tesseract_parser.parse(
-            multi_page_images_pdf_file,
+            tesseract_samples_dir / "multi-page-images.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -733,7 +781,7 @@ class TestSkipArchive:
     def test_skip_noarchive_with_text_layer(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -747,7 +795,7 @@ class TestSkipArchive:
         """
         tesseract_parser.settings.mode = ModeChoices.AUTO
         tesseract_parser.parse(
-            multi_page_digital_pdf_file,
+            tesseract_samples_dir / "multi-page-digital.pdf",
             "application/pdf",
             produce_archive=False,
         )
@@ -762,7 +810,7 @@ class TestSkipArchive:
     def test_skip_noarchive_image_only_creates_archive(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        multi_page_images_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -775,7 +823,7 @@ class TestSkipArchive:
         """
         tesseract_parser.settings.mode = ModeChoices.AUTO
         tesseract_parser.parse(
-            multi_page_images_pdf_file,
+            tesseract_samples_dir / "multi-page-images.pdf",
             "application/pdf",
         )
         assert tesseract_parser.archive_path is not None
@@ -787,29 +835,29 @@ class TestSkipArchive:
         )
 
     @pytest.mark.parametrize(
-        ("produce_archive", "sample_fixture", "expect_archive"),
+        ("produce_archive", "filename", "expect_archive"),
         [
             pytest.param(
                 True,
-                "multi_page_digital_pdf_file",
+                "multi-page-digital.pdf",
                 True,
                 id="produce-archive-with-text",
             ),
             pytest.param(
                 True,
-                "multi_page_images_pdf_file",
+                "multi-page-images.pdf",
                 True,
                 id="produce-archive-no-text",
             ),
             pytest.param(
                 False,
-                "multi_page_digital_pdf_file",
+                "multi-page-digital.pdf",
                 False,
                 id="no-archive-with-text-layer",
             ),
             pytest.param(
                 False,
-                "multi_page_images_pdf_file",
+                "multi-page-images.pdf",
                 False,
                 id="no-archive-no-text-layer",
             ),
@@ -818,10 +866,10 @@ class TestSkipArchive:
     def test_produce_archive_flag(
         self,
         produce_archive: bool,  # noqa: FBT001
-        sample_fixture: str,
+        filename: str,
         expect_archive: bool,  # noqa: FBT001
         tesseract_parser: RasterisedDocumentParser,
-        request: pytest.FixtureRequest,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -834,9 +882,8 @@ class TestSkipArchive:
             - Text is always extracted
         """
         tesseract_parser.settings.mode = ModeChoices.AUTO
-        sample = request.getfixturevalue(sample_fixture)
         tesseract_parser.parse(
-            sample,
+            tesseract_samples_dir / filename,
             "application/pdf",
             produce_archive=produce_archive,
         )
@@ -853,7 +900,7 @@ class TestSkipArchive:
         self,
         mocker: MockerFixture,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -869,7 +916,7 @@ class TestSkipArchive:
         tesseract_parser.settings.mode = ModeChoices.AUTO
         mock_ocr = mocker.patch("ocrmypdf.ocr")
         tesseract_parser.parse(
-            simple_digital_pdf_file,
+            tesseract_samples_dir / "simple-digital.pdf",
             "application/pdf",
             produce_archive=False,
         )
@@ -908,7 +955,7 @@ class TestSkipArchive:
     def test_tagged_pdf_produces_pdfa_archive_without_ocr(
         self,
         tesseract_parser: RasterisedDocumentParser,
-        simple_digital_pdf_file: Path,
+        tesseract_samples_dir: Path,
     ) -> None:
         """
         GIVEN:
@@ -923,7 +970,7 @@ class TestSkipArchive:
         """
         tesseract_parser.settings.mode = ModeChoices.AUTO
         tesseract_parser.parse(
-            simple_digital_pdf_file,
+            tesseract_samples_dir / "simple-digital.pdf",
             "application/pdf",
             produce_archive=True,
         )

src/paperless/tests/samples/mail/sample.html

@@ -0,0 +1,19 @@
+ <html>
+    <head>
+      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
+    </head>
+    <body>
+      <p>Some Text</p>
+      <p>
+				<img src="cid:part1.pNdUSz0s.D3NqVtPg@example.de" alt="Has to be rewritten to work..">
+				<img src="http://localhost:8080/assets/logo_full_white.svg" alt="This image should not be shown.">
+			</p>
+
+      <p>and an embedded image.<br>
+      </p>
+      <p id="changeme">Paragraph unchanged.</p>
+      <scRipt>
+          document.getElementById("changeme").innerHTML = "Paragraph changed via Java Script.";
+      </script>
+    </body>
+  </html>

src/paperless/tests/test_parser_utils.py

@@ -6,8 +6,10 @@ import codecs
 from pathlib import Path
 from typing import TYPE_CHECKING
 
+import pikepdf
 import pytest
 
+from paperless.parsers.utils import get_pdf_first_page_size_points
 from paperless.parsers.utils import is_tagged_pdf
 from paperless.parsers.utils import pdf_born_digital_text
 from paperless.parsers.utils import post_process_text
@@ -16,6 +18,8 @@ from paperless.parsers.utils import read_file_handle_unicode_errors
 if TYPE_CHECKING:
     from pytest_mock import MockerFixture
 
+SAMPLES = Path(__file__).parent / "samples" / "tesseract"
+
 
 class TestReadFileHandleUnicodeErrors:
     def test_plain_utf8(self, tmp_path: Path) -> None:
@@ -53,11 +57,11 @@ class TestReadFileHandleUnicodeErrors:
 
 
 class TestIsTaggedPdf:
-    def test_tagged_pdf_returns_true(self, simple_digital_pdf_file: Path) -> None:
-        assert is_tagged_pdf(simple_digital_pdf_file) is True
+    def test_tagged_pdf_returns_true(self) -> None:
+        assert is_tagged_pdf(SAMPLES / "simple-digital.pdf") is True
 
-    def test_untagged_pdf_returns_false(self, multi_page_images_pdf_file: Path) -> None:
-        assert is_tagged_pdf(multi_page_images_pdf_file) is False
+    def test_untagged_pdf_returns_false(self) -> None:
+        assert is_tagged_pdf(SAMPLES / "multi-page-images.pdf") is False
 
     def test_nonexistent_path_returns_false(self) -> None:
         assert is_tagged_pdf(Path("/nonexistent/file.pdf")) is False
@@ -68,6 +72,149 @@ class TestIsTaggedPdf:
         assert is_tagged_pdf(bad) is False
 
 
+class TestGetPdfFirstPageSizePoints:
+    @staticmethod
+    def _write_pdf(
+        path: Path,
+        *,
+        media_box: tuple[float, float, float, float] = (0, 0, 600, 800),
+        crop_box: tuple[float, float, float, float] | None = None,
+        page_rotate: int | None = None,
+        inherited_rotate: int | None = None,
+    ) -> Path:
+        pdf = pikepdf.new()
+        pdf.add_blank_page(page_size=(media_box[2], media_box[3]))
+        page = pdf.pages[0]
+        page.obj.MediaBox = pikepdf.Array(media_box)
+        if crop_box is not None:
+            page.obj.CropBox = pikepdf.Array(crop_box)
+        if page_rotate is not None:
+            page.obj.Rotate = page_rotate
+        if inherited_rotate is not None:
+            pdf.Root.Pages.Rotate = inherited_rotate
+        pdf.save(path)
+        return path
+
+    def test_letter_sample(self) -> None:
+        """
+        GIVEN:
+            - A US Letter sample PDF with no CropBox and no rotation
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - The MediaBox size in points is returned
+        """
+        assert get_pdf_first_page_size_points(SAMPLES / "simple-digital.pdf") == (
+            612.0,
+            792.0,
+        )
+
+    @pytest.mark.parametrize(
+        ("rotate", "expected"),
+        [
+            pytest.param(0, (600.0, 800.0), id="rotate-0"),
+            pytest.param(90, (800.0, 600.0), id="rotate-90"),
+            pytest.param(180, (600.0, 800.0), id="rotate-180"),
+            pytest.param(270, (800.0, 600.0), id="rotate-270"),
+            pytest.param(-90, (800.0, 600.0), id="rotate-negative-90"),
+        ],
+    )
+    def test_page_rotation_swaps_dimensions(
+        self,
+        tmp_path: Path,
+        rotate: int,
+        expected: tuple[float, float],
+    ) -> None:
+        """
+        GIVEN:
+            - A portrait PDF page with /Rotate set directly on the page
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - Width and height are swapped for quarter-turn rotations only
+        """
+        pdf_path = self._write_pdf(tmp_path / "rotated.pdf", page_rotate=rotate)
+        assert get_pdf_first_page_size_points(pdf_path) == expected
+
+    def test_inherited_rotation_swaps_dimensions(self, tmp_path: Path) -> None:
+        """
+        GIVEN:
+            - A page inheriting /Rotate 90 from the /Pages node
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - Width and height are swapped
+        """
+        pdf_path = self._write_pdf(tmp_path / "inherited.pdf", inherited_rotate=90)
+        assert get_pdf_first_page_size_points(pdf_path) == (800.0, 600.0)
+
+    def test_crop_box_preferred_over_media_box(self, tmp_path: Path) -> None:
+        """
+        GIVEN:
+            - A PDF page with a CropBox smaller than its MediaBox
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - The CropBox size is returned
+        """
+        pdf_path = self._write_pdf(
+            tmp_path / "cropped.pdf",
+            crop_box=(50, 100, 350, 500),
+        )
+        assert get_pdf_first_page_size_points(pdf_path) == (300.0, 400.0)
+
+    def test_degenerate_box_returns_none(self, tmp_path: Path) -> None:
+        """
+        GIVEN:
+            - A PDF page whose box has zero width
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - None is returned
+        """
+        pdf_path = self._write_pdf(
+            tmp_path / "degenerate.pdf",
+            media_box=(0, 0, 600, 800),
+            crop_box=(100, 0, 100, 800),
+        )
+        assert get_pdf_first_page_size_points(pdf_path) is None
+
+    def test_nonexistent_path_returns_none(self) -> None:
+        """
+        GIVEN:
+            - A path that does not exist
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - None is returned and nothing is raised
+        """
+        assert get_pdf_first_page_size_points(Path("/nonexistent/file.pdf")) is None
+
+    def test_corrupt_pdf_returns_none(self, tmp_path: Path) -> None:
+        """
+        GIVEN:
+            - A file that is not a PDF
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - None is returned and nothing is raised
+        """
+        bad = tmp_path / "bad.pdf"
+        bad.write_bytes(b"not a pdf")
+        assert get_pdf_first_page_size_points(bad) is None
+
+    def test_encrypted_pdf_returns_none(self) -> None:
+        """
+        GIVEN:
+            - A password protected PDF
+        WHEN:
+            - The first page size is requested
+        THEN:
+            - None is returned and nothing is raised
+        """
+        assert get_pdf_first_page_size_points(SAMPLES / "encrypted.pdf") is None
+
+
 class TestPostProcessText:
     @pytest.mark.parametrize(
         ("source", "expected"),

src/paperless_ai/ai_classifier.py

@@ -4,7 +4,8 @@ from django.conf import settings
 from django.contrib.auth.models import User
 
 from documents.models import Document
-from documents.permissions import permitted_document_ids
+from documents.permissions import permitted_object_ids
+from documents.permissions import restrict_queryset_to_visible
 from documents.permissions import user_is_unrestricted
 from paperless.config import AIConfig
 from paperless_ai.base_model import ClassificationSuggestions
@@ -53,8 +54,8 @@ def _fulltext_similar_documents(
     active superuser - see user_is_unrestricted) is normalized to ``None``
     before calling, since the backend's permission filter has no superuser
     short-circuit of its own. Results are re-checked with
-    permitted_document_ids() since Tantivy's indexed permission fields lag the
-    DB via async reindexing and judge a version by its own owner.
+    restrict_queryset_to_visible() since Tantivy's indexed permission fields
+    lag the DB via async reindexing.
     """
     from documents.search import get_backend
 
@@ -68,9 +69,10 @@ def _fulltext_similar_documents(
     )
     if not unrestricted:
         allowed_ids = set(
-            Document.objects.filter(
-                pk__in=similar_ids,
-                id__in=permitted_document_ids(user),
+            restrict_queryset_to_visible(
+                Document.objects.filter(pk__in=similar_ids),
+                user,
+                "view_document",
             ).values_list("pk", flat=True),
         )
         similar_ids = [doc_id for doc_id in similar_ids if doc_id in allowed_ids]
@@ -198,13 +200,13 @@ def get_taxonomy_context(
             # quadratic scan in the vector store at best, and past ~32,763
             # documents a hard sqlite3.OperationalError (SQLite's
             # bound-parameter limit) at worst.
-            # permitted_document_ids() has its own superuser shortcut that would
+            # permitted_object_ids() has its own superuser shortcut that would
             # return every Document's id anyway, so this changes nothing about
             # which documents are considered -- only how we get there.
             visible_document_ids = (
                 None
                 if user_is_unrestricted(user)
-                else list(permitted_document_ids(user))
+                else list(permitted_object_ids(user, Document, "view_document"))
             )
             nodes = retrieve_similar_nodes(
                 document,

src/paperless_ai/tests/test_ai_classifier.py

@@ -552,7 +552,7 @@ class TestGetTaxonomyContextVisibility:
             return_value=[],
         )
         mock_permitted = mocker.patch(
-            "paperless_ai.ai_classifier.permitted_document_ids",
+            "paperless_ai.ai_classifier.permitted_object_ids",
         )
         user = UserFactory.create(is_superuser=True)
 
@@ -582,7 +582,7 @@ class TestGetTaxonomyContextVisibility:
             return_value=[],
         )
         mock_permitted = mocker.patch(
-            "paperless_ai.ai_classifier.permitted_document_ids",
+            "paperless_ai.ai_classifier.permitted_object_ids",
         )
 
         get_taxonomy_context(document, None)
@@ -611,55 +611,16 @@ class TestGetTaxonomyContextVisibility:
             return_value=[],
         )
         mock_permitted = mocker.patch(
-            "paperless_ai.ai_classifier.permitted_document_ids",
+            "paperless_ai.ai_classifier.permitted_object_ids",
             return_value=[1, 2, 3],
         )
         user = UserFactory.create(is_superuser=False)
 
         get_taxonomy_context(document, user)
 
-        mock_permitted.assert_called_once_with(user)
+        mock_permitted.assert_called_once_with(user, Document, "view_document")
         assert mock_retrieve.call_args.kwargs["document_ids"] == [1, 2, 3]
 
-    @pytest.mark.django_db
-    @override_settings(LLM_EMBEDDING_BACKEND="huggingface")
-    def test_version_of_private_root_is_not_visible(
-        self,
-        mocker: pytest_mock.MockerFixture,
-    ) -> None:
-        """
-        GIVEN:
-            - A private root document owned by someone else
-            - A version of it whose own owner is unset, as when the root
-              changed hands after the version was created
-        WHEN:
-            - get_taxonomy_context() is called for a non-superuser
-        THEN:
-            - Neither the root nor the version is in the visible ids passed to
-              retrieve_similar_nodes(), since a version follows its root
-        """
-        owner = UserFactory.create()
-        viewer = UserFactory.create(is_superuser=False)
-        root = DocumentFactory.create(content="private", owner=owner)
-        version = DocumentFactory.create(
-            content="private",
-            owner=None,
-            root_document=root,
-            version_index=1,
-        )
-        source = DocumentFactory.create(content="Some content", owner=viewer)
-        mock_retrieve = mocker.patch(
-            "paperless_ai.ai_classifier.retrieve_similar_nodes",
-            return_value=[],
-        )
-
-        get_taxonomy_context(source, viewer)
-
-        visible = mock_retrieve.call_args.kwargs["document_ids"]
-        assert source.pk in visible
-        assert root.pk not in visible
-        assert version.pk not in visible
-
 
 @pytest.mark.django_db
 class TestFulltextSimilarDocuments:
@@ -842,7 +803,7 @@ class TestFulltextSimilarDocuments:
             - _fulltext_similar_documents() is called with that user
         THEN:
             - Only the still-permitted document is returned - the DB
-              re-check via permitted_document_ids() must catch the
+              re-check via restrict_queryset_to_visible() must catch the
               document Tantivy's stale index still thinks is visible
         """
         owner = UserFactory.create()
@@ -873,45 +834,6 @@ class TestFulltextSimilarDocuments:
 
         assert [s["document_id"] for s in result] == [permitted.pk]
 
-    def test_excludes_version_of_private_root_for_regular_user(
-        self,
-        fulltext_backend: TantivyBackend,
-    ) -> None:
-        """
-        GIVEN:
-            - A regular user and a private root owned by someone else
-            - A version of that root with no owner of its own, which the
-              Tantivy index therefore treats as visible to everyone
-        WHEN:
-            - _fulltext_similar_documents() is called with that user
-        THEN:
-            - The version is not returned, since the DB re-check judges it by
-              its root
-        """
-        owner = UserFactory.create()
-        viewer = UserFactory.create(is_superuser=False)
-        source = DocumentFactory.create(
-            content="shared content phrase",
-            owner=viewer,
-        )
-        root = DocumentFactory.create(
-            content="shared content phrase",
-            owner=owner,
-        )
-        version = DocumentFactory.create(
-            content="shared content phrase",
-            owner=None,
-            root_document=root,
-            version_index=1,
-        )
-        fulltext_backend.add_or_update(source)
-        fulltext_backend.add_or_update(root)
-        fulltext_backend.add_or_update(version)
-
-        result = _fulltext_similar_documents(source, user=viewer, top_k=5)
-
-        assert version.pk not in [s["document_id"] for s in result]
-
 
 @pytest.mark.django_db
 @override_settings(LLM_EMBEDDING_BACKEND="huggingface")

src/paperless_testing/samples.py

@@ -1,48 +0,0 @@
-"""Test sample files shared across tests.
-
-A sample used by a single app stays in that app's ``tests/samples`` tree.
-Samples needed by more than one app, or stored under more than one name, live
-here exactly once.
-"""
-
-import shutil
-from pathlib import Path
-
-SHARED_SAMPLES_DIR = (Path(__file__).parent / "sample_files").resolve()
-
-SIMPLE_DIGITAL_PDF = SHARED_SAMPLES_DIR / "simple-digital.pdf"
-MULTI_PAGE_DIGITAL_PDF = SHARED_SAMPLES_DIR / "multi-page-digital.pdf"
-WITH_FORM_PDF = SHARED_SAMPLES_DIR / "with-form.pdf"
-MULTI_PAGE_IMAGES_PDF = SHARED_SAMPLES_DIR / "multi-page-images.pdf"
-THUMBNAIL_WEBP = SHARED_SAMPLES_DIR / "thumbnail.webp"
-
-# The fake media tree under documents/tests/samples/documents keeps only the
-# files unique to it. The files that duplicate a shared sample are laid back
-# down under their historical opaque names (DB rows and exporter manifests
-# refer to them) by install_document_samples().
-_DOCUMENT_SAMPLES_DIR = (
-    Path(__file__).parent.parent / "documents" / "tests" / "samples" / "documents"
-).resolve()
-
-_SHARED_AS_MEDIA = (
-    ("originals/0000001.pdf", SIMPLE_DIGITAL_PDF),
-    ("originals/0000002.pdf", MULTI_PAGE_DIGITAL_PDF),
-    ("originals/0000003.pdf", WITH_FORM_PDF),
-    ("archive/0000001.pdf", MULTI_PAGE_IMAGES_PDF),
-    ("thumbnails/0000001.webp", THUMBNAIL_WEBP),
-    ("thumbnails/0000002.webp", THUMBNAIL_WEBP),
-    ("thumbnails/0000003.webp", THUMBNAIL_WEBP),
-    ("thumbnails/0000004.webp", THUMBNAIL_WEBP),
-)
-
-
-def install_document_samples(dest: Path) -> None:
-    """Populate ``dest`` with the full fake media tree.
-
-    Replaces ``shutil.copytree(<samples>/documents, dest)`` in tests.
-    """
-    shutil.copytree(_DOCUMENT_SAMPLES_DIR, dest, dirs_exist_ok=True)
-    for relpath, source in _SHARED_AS_MEDIA:
-        target = dest / relpath
-        target.parent.mkdir(parents=True, exist_ok=True)
-        shutil.copy2(source, target)