* Mark the thumbnail constants Final
* Comment each pdftoppm option
* Feature/Chore: Replaces all usages of ImageMagik's convert binary with a simple Pillow based helper
This removes ImageMagik as a dependency entirly and all documentation is cleaned up from it.
We no longer need to install it, have a policy for it to allow PDFs and no longer shell
out to it either.
* Fix: Save WebP images losslessly when removing the alpha channel
Pillow's default WebP save is lossy, so a WebP with an alpha channel came
back from remove_alpha_channel with slightly shifted pixel values before
it reached OCR. Save WebP lossless; other formats are unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* Chore: Generate PDF thumbnails with pdftoppm and Pillow instead of ImageMagick
Replace the ImageMagick convert based PDF thumbnail path with pdftoppm and
Pillow. Keeps thumbnails at 500px wide for small and odd sized pages, covers
the qpdf repair and double failure paths, removes the unused run_convert
helper, hardens thumbnail encoding, deprecates PAPERLESS_CONVERT_MEMORY_LIMIT
and PAPERLESS_CONVERT_TMPDIR, and corrects the related docs and CI.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Chore: Supersample the PDF thumbnail render 2x
Render the PDF page at twice the target size and downsample, with edge case
handling for small and odd sized pages, simplified helpers and tests, and
trimmed comments.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Fix: use a fractional thumbnail DPI so oversized pages render a bounded raster
* Deprecate the unused PAPERLESS_GS_BINARY setting too
* Warn at startup when pdftoppm is missing
* Resample PDF thumbnails once instead of twice with LANCZOS, still never enlarging
* Read PDF page size from pdfinfo so it matches pdftoppm
* Mark the thumbnail constants Final
* Comment each pdftoppm option
* Preprocesses classifier content with Tantivy instead of NLTK
Tokenizing and stemming now happen in one Rust call instead of NLTK's
Python tokenizer and per word stemming, which also removes the Redis
backed stem cache from every preprocessing call. The output matches the
NLTK pipeline closely; tokens containing digits are now stemmed, and the
English stop words follow Snowball's list.
Stemming and stop word removal apply whenever the OCR language is one of
the supported classifier languages, so PAPERLESS_ENABLE_NLTK and
PAPERLESS_NLTK_DIR are removed.
* Copies packages instead of hardlinking them in backend CI, some NLTK thing
* Adds a normalization to NFC to better fit what Tantivy expects
To be consistent, let's provide an easily copy-pastable list of packages
even for the final set of build dependencies.
Signed-off-by: martin f. krafft <madduck@madduck.net>
Trenton HandClaude Sonnet 5.5
shamoon
Albert Mikaelyan
dependabot[bot]andshamoon
Katrin Leinweber
Colin Yatesandshamoon
Malte Breitzmann
Johan Ohly
marph91andshamoon
fusselkater
martin f. krafft
Lino