Commit Graph
299 Commits
Author SHA1 Message Date
Trenton HandClaude Sonnet 5.5 c089bba32c Chore: Drop ImageMagik as a dependency (#14440)
* Mark the thumbnail constants Final

* Comment each pdftoppm option

* Feature/Chore: Replaces all usages of ImageMagik's convert binary with a simple Pillow based helper

This removes ImageMagik as a dependency entirly and all documentation is cleaned up from it.
We no longer need to install it, have a policy for it to allow PDFs and no longer shell
out to it either.

* Fix: Save WebP images losslessly when removing the alpha channel

Pillow's default WebP save is lossy, so a WebP with an alpha channel came
back from remove_alpha_channel with slightly shifted pixel values before
it reached OCR. Save WebP lossless; other formats are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 11:22:43 -07:00
Sven Lankes 1fb463b55c Chore: silence gotenberg startup warning (#14405) 2026-10-12 16:54:47 +00:00
shamoon f8876c55e3 Documentation: remove MARIADB_HOST from MariaDB compose files (#14378) 2026-10-07 07:32:37 -07:00
dependabot[bot] 4254100fae docker-compose(deps): bump greenmail/standalone in /docker/compose (#14281)
Bumps greenmail/standalone from 2.1.13 to 2.1.14.

---
updated-dependencies:
- dependency-name: greenmail/standalone
  dependency-version: 2.1.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 11:31:43 -07:00
Trenton H 4f777d7438 Feature: propagate resolved secrets to interactive container shells (#14254)
docker exec bash bypasses s6, so *_FILE secrets resolved by init-env-file
into /run/s6/container_environment aren't visible even though s6 services
and management wrappers already see them via with-contenv.

Add /etc/profile.d/contenv.sh, sourced by both login and non-login
interactive bash shells, to export those resolved values into the shell.
2026-09-24 13:07:41 -07:00
Bitfoo 54e332d259 Fix: only pass --conf to flower when flowerconfig.py exists (#14182)
* Fix: only pass --conf to flower when flowerconfig.py exists

The service passes --conf=${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py
unconditionally, but the image does not ship that file. flower 2.0.1 tolerated
this because it ignored a missing config whose basename matched its own default
name; 2.1.0 dropped that basename() call (mher/flower#1391) and now raises
FileNotFoundError, so flower crash-loops and never binds 5555.

Pass the flag only when the file is present.
2026-09-19 21:10:52 -07:00
dependabot[bot] 24b157622d docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#14137)
Bumps gotenberg/gotenberg from 8.36 to 8.37.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.37'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-17 15:52:45 -07:00
dependabot[bot] 51f3266014 docker-compose(deps): Bump nginx in /docker/compose (#14138)
Bumps nginx from 1.31.5-alpine to 1.31.6-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.6-alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-17 22:32:40 +00:00
Trenton H a00755907e Performance: Precompile scoped bytecode cache once per container start (#13819)
* Performance: Precompile scoped bytecode cache once per container start

* Skips this unit during non-root user startup too
2026-09-10 08:28:21 -07:00
dependabot[bot] a07e1cea75 docker-compose(deps): Bump gotenberg/gotenberg in /docker/compose (#13910)
Bumps gotenberg/gotenberg from 8.34 to 8.36.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.36'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 09:18:04 -07:00
dependabot[bot] 1806141983 docker-compose(deps): bump nginx in /docker/compose (#13909)
Bumps nginx from 1.31.3-alpine to 1.31.5-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.4-alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 16:07:06 +00:00
dependabot[bot] 593a65c157 docker-compose(deps): Bump greenmail/standalone in /docker/compose (#13907)
Bumps greenmail/standalone from 2.1.11 to 2.1.13.

---
updated-dependencies:
- dependency-name: greenmail/standalone
  dependency-version: 2.1.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 08:53:52 -07:00
shamoon a510d03c77 Merge branch 'main' into dev 2026-08-22 05:27:10 -07:00
shamoon d7b3612a41 Chore: pin Apache Tika images to 3.3.1 (#13758) 2026-08-22 05:25:37 -07:00
shamoon 41953c7846 Chore: harden ImageMagick policy 2026-08-18 09:38:54 -07:00
dependabot[bot] 4692de5595 docker-compose(deps): Bump nginx in /docker/compose (#13470)
Bumps nginx from 1.31.2-alpine to 1.31.3-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.3-alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-02 03:41:25 +00:00
dependabot[bot] 141d74464b docker-compose(deps): Bump greenmail/standalone in /docker/compose (#13469)
Bumps greenmail/standalone from 2.1.9 to 2.1.11.

---
updated-dependencies:
- dependency-name: greenmail/standalone
  dependency-version: 2.1.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-02 03:31:10 +00:00
Trenton H 486b4babac Performance: sqlite-vec point-delete for document chunks (#13438)
* Refactor: extract migration infrastructure, add has_pending_migration() (#13410)

* Empty commit to try and get Codecov going
2026-07-31 19:10:59 -07:00
Fabian 50c88dab7a Fix: correct database engine from postgres to postgresql in the Porttainer Compose file (#13213) 2026-07-23 07:40:39 -07:00
dependabot[bot] e2cf3633fe docker-compose(deps): bump greenmail/standalone in /docker/compose (#13067)
Bumps greenmail/standalone from 2.1.8 to 2.1.9.

---
updated-dependencies:
- dependency-name: greenmail/standalone
  dependency-version: 2.1.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 09:23:10 -07:00
dependabot[bot] c0589a9ce9 docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#13066)
Bumps gotenberg/gotenberg from 8.33 to 8.34.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.34'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 09:09:30 -07:00
dependabot[bot] 2135572adb docker-compose(deps): bump nginx in /docker/compose (#13065)
Bumps nginx from 1.31.1-alpine to 1.31.2-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.2-alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 15:55:33 +00:00
Trenton Handupmcplanetracker a009ea1f04 Chore(beta): Update suggested broker to Valkey + Redis agnostic documentation (#13032)
Co-authored-by: upmcplanetracker <219436948+upmcplanetracker@users.noreply.github.com>
2026-06-18 19:54:15 +00:00
dependabot[bot] f3eb8d4f58 docker-compose(deps): bump apache/tika in /docker/compose (#12912)
Bumps apache/tika from 3.2.3.0 to 3.3.1.0.

---
updated-dependencies:
- dependency-name: apache/tika
  dependency-version: 3.3.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-03 13:13:14 -07:00
dependabot[bot] eab964124d docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#12910)
Bumps gotenberg/gotenberg from 8.27 to 8.33.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.33'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-03 12:40:18 -07:00
dependabot[bot] 2e9b07b77f docker-compose(deps): Bump nginx in /docker/compose (#12911)
Bumps nginx from 1.29.5-alpine to 1.31.1-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.1-alpine
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-03 11:41:13 -07:00
Trenton H dda05a7c00 Security: Improve overall security in a few ways (#12501)
- Make sure we're always using regex with timeouts for user controlled data
- Adds rate limiting to the token endpoint (configurable)
- Signs the classifier pickle file with the SECRET_KEY and refuse to load one which doesn't verify.
- Require the user to set a secret key, instead of falling back to our old hard coded one
2026-04-02 15:30:26 -07:00
aed9abe48c Feature: Replace Whoosh with tantivy search backend (#12471)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
2026-04-02 12:38:22 -07:00
Trenton HandClaude Sonnet 4.6 aea2927a02 Feature: Convert Tika parser to the plugin system (#12333)
* Chore: move Tika parser and tests to paperless/

Move TikaDocumentParser and its tests to the canonical parser package
location, matching the pattern established for TextDocumentParser:

- src/paperless_tika/parsers.py → src/paperless/parsers/tika.py
- src/paperless_tika/tests/test_tika_parser.py → src/paperless/tests/parsers/test_tika_parser.py
- src/paperless_tika/tests/samples/ → src/paperless/tests/samples/tika/

Merge tika fixtures (tika_parser, sample_odt_file, sample_docx_file,
sample_doc_file, sample_broken_odt) into the shared parsers conftest.
Remove the now-empty src/paperless_tika/tests/conftest.py.

Content is unchanged — this commit is rename-only so git history is
preserved on the moved files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Feature: Phase 3 — migrate TikaDocumentParser to ParserProtocol

Refactor TikaDocumentParser to satisfy ParserProtocol without subclassing
the legacy DocumentParser ABC:

- Add ClassVars: name, version, author, url
- Add supported_mime_types() classmethod (12 Office/ODF/RTF MIME types)
- Add score() classmethod — returns None when TIKA_ENABLED is False, 10 otherwise
- can_produce_archive = False (PDF is for display, not an OCR archive)
- requires_pdf_rendition = True (Office formats need PDF for browser display)
- __enter__/__exit__ via ExitStack: TikaClient opened once per parser
  lifetime and shared across parse() and extract_metadata() calls
- extract_metadata() falls back to a short-lived TikaClient when called
  outside a context manager (legacy view-layer metadata path)
- _convert_to_pdf() uses OutputTypeConfig() to honour the database-stored
  ApplicationConfiguration before falling back to the env-var setting
- Rename convert_to_pdf → _convert_to_pdf (private helper)

Update paperless_tika/signals.py shim to import from the new module path
and drop the legacy logging_group/progress_callback kwargs.

Update documents/consumer.py to extend the existing TextDocumentParser
special cases to also cover TikaDocumentParser (parse/get_thumbnail
signatures, __exit__ cleanup).

Add TestTikaParserRegistryInterface (7 tests) covering score(), properties,
and ParserProtocol isinstance check.  Update existing tests to use the new
accessor API (get_text, get_date, get_archive_path, _convert_to_pdf).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: update remaining imports and move live Tika tests after parser migration

- src/documents/tests/test_parsers.py: import TikaDocumentParser from
  paperless.parsers.tika (old paperless_tika.parsers no longer exists)
- git mv paperless_tika/tests/test_live_tika.py →
  paperless/tests/parsers/test_live_tika.py to co-locate all Tika tests
  with the parser; update import and replace old attribute API
  (tika_parser.text/.archive_path) with accessor methods
  (get_text/get_archive_path)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: satisfy mypy and pyrefly for TikaDocumentParser

Use a TYPE_CHECKING-guarded assert to narrow self._tika_client from
TikaClient | None to TikaClient at the point of use in parse().  The
assert is visible to type checkers (TYPE_CHECKING=True) so both mypy
and pyrefly accept the subsequent attribute accesses without error;
at runtime TYPE_CHECKING is False so the assert never executes and no
ruff S101 suppression is required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: require context manager for TikaDocumentParser; clean up client lifecycle

- consumer.py: call __enter__ for new-style parsers so _tika_client and
  _gotenberg_client are set before parse() is invoked
- views.py: use `with parser` (via nullcontext for old-style parsers) in
  get_metadata so extract_metadata always runs inside a context manager
- tika.py: GotenbergClient added to ExitStack alongside TikaClient;
  inline client creation removed from extract_metadata and _convert_to_pdf;
  __exit__ uses ExitStack.close() instead of __exit__ pass-through

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 15:43:28 -07:00
Trenton H 85fecac401 Fix: don't try to usermod/groupmod when non-root + update docs (#12365) 2026-03-17 05:15:03 +00:00
Trenton H f15394fa5c Fix: Removes the double exec that prevented migrations from running (#12317) 2026-03-12 12:46:12 -07:00
Trenton H 86fa74c115 Fix: Postgres selection, DBENGINE and migrations (#12299) 2026-03-11 11:54:24 -07:00
dependabot[bot] 484bef00c1 docker-compose(deps): Bump gotenberg/gotenberg in /docker/compose (#12190)
Bumps gotenberg/gotenberg from 8.26 to 8.27.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.27'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 10:14:48 -08:00
dependabot[bot] b9b90ec9f7 docker-compose(deps): Bump nginx in /docker/compose (#12018)
Bumps nginx from 1.29-alpine to 1.29.5-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.29.5-alpine
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-06 12:26:29 -08:00
dependabot[bot] 4a5116adf8 docker-compose(deps): Bump gotenberg/gotenberg in /docker/compose (#11979)
Bumps gotenberg/gotenberg from 8.25 to 8.26.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.26'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-04 13:24:19 -08:00
shamoon 72b861b5eb Fix: fix broken docker create_classifier command in 2.20.6 (#11965) 2026-02-01 12:09:30 -08:00
shamoon c3b036e0d3 Merge branch 'main' into dev 2026-01-31 09:10:33 -08:00
shamoon 6913f9d79c Fix: fix user checks in management scripts (#11928) 2026-01-28 13:45:12 -08:00
Trenton H 01b21377af Chore: Use a local http server instead of external to reduce flakiness (#11916) 2026-01-28 03:57:12 +00:00
Trenton H c84f2f04b3 Chore: Switch to a local IMAP server instead of a real email service (#11913) 2026-01-27 11:35:12 -08:00
shamoon 444ff6951e Merge branch 'release/v2.20.x' into dev 2026-01-25 16:58:04 -08:00
Trenton H d0032c18be Breaking: Remove support for document and thumbnail encryption (#11850) 2026-01-24 19:29:54 -08:00
Trenton H 94f6b8d36d Fixes the management scripts under a non-root install where the user ID is something besides 1000 (#11870) 2026-01-23 16:08:28 -08:00
shamoon e940764fe0 Feature: Paperless AI (#10319) 2026-01-13 16:24:42 +00:00
Daniel Rheinbay 67d079fe14 fix: Skip SSL for MariaDB ping in init script (#11491)
Restore compatibility with MariaDB server versions < 11.4, which do not use SSL by default.
2025-11-28 14:25:57 -08:00
Daniel Rheinbay ffc56bddda fix: Add user parameter to MariaDB connection check (#11441) 2025-11-23 15:03:35 -08:00
Trenton H a96db50b0a Feature: Replace duplicated static files with symlinks (#11418) 2025-11-21 20:07:57 +00:00
Trenton H bc622d67fc Chore: Configure pre-commit to format our s6-overlay files (#11414) 2025-11-19 21:34:29 +00:00
Trenton H 25b5e8fede Improves the MariaDB wait command to use mariadb-admin ping for a better check if the server is up (#11396) 2025-11-18 23:45:49 +00:00
dependabot[bot] 4bf681387a docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#11393)
Bumps gotenberg/gotenberg from 8.24 to 8.25.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.25'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-18 09:29:39 -08:00