Commit Graph
94 Commits
Author SHA1 Message Date
Trenton H 530059c5c0 Performance: Preprocess classifier text with Tantivy instead of NLTK (#14127)
* Preprocesses classifier content with Tantivy instead of NLTK

Tokenizing and stemming now happen in one Rust call instead of NLTK's
Python tokenizer and per word stemming, which also removes the Redis
backed stem cache from every preprocessing call. The output matches the
NLTK pipeline closely; tokens containing digits are now stemmed, and the
English stop words follow Snowball's list.

Stemming and stop word removal apply whenever the OCR language is one of
the supported classifier languages, so PAPERLESS_ENABLE_NLTK and
PAPERLESS_NLTK_DIR are removed.

* Copies packages instead of hardlinking them in backend CI, some NLTK thing

* Adds a normalization to NFC to better fit what Tantivy expects
2026-09-16 07:35:26 -07:00
shamoon 41953c7846 Chore: harden ImageMagick policy 2026-08-18 09:38:54 -07:00
Trenton H 1646756c6c Docs: warn bare-metal users about stale files when upgrading (#13296) 2026-07-25 14:06:31 -07:00
shamoon 6cf3c47814 Documentation: fix a couple broken links 2026-07-22 11:37:56 -07:00
Albert Mikaelyan 735c3b073a Fix: allow setting any UID and GID when running rootless (#13090) 2026-07-15 10:19:22 -07:00
Trenton Handshamoon bb5d7438b1 Documentation (beta): Updates documentation for new v3 features (#13033)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-06-18 16:20:31 -07:00
Trenton Handupmcplanetracker a009ea1f04 Chore(beta): Update suggested broker to Valkey + Redis agnostic documentation (#13032)
Co-authored-by: upmcplanetracker <219436948+upmcplanetracker@users.noreply.github.com>
2026-06-18 19:54:15 +00:00
c232d443fa Breaking: Decouple OCR control from archive file control (#12448)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-04-06 15:50:21 -07:00
Trenton H 85fecac401 Fix: don't try to usermod/groupmod when non-root + update docs (#12365) 2026-03-17 05:15:03 +00:00
dependabot[bot]andshamoon e2947ccff2 Chore(deps): Bump the pre-commit-dependencies group with 4 updates (#12323)
* Chore(deps): Bump the pre-commit-dependencies group with 4 updates

---
updated-dependencies:
- dependency-name: https://github.com/codespell-project/codespell
  dependency-version: 2.4.2
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
- dependency-name: prettier
  dependency-version: 3.8.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: pre-commit-dependencies
- dependency-name: prettier-plugin-organize-imports
  dependency-version: 4.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: pre-commit-dependencies
- dependency-name: https://github.com/lovesegfault/beautysh
  dependency-version: 6.4.3
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Drop this, it seems more trouble than its worth

* Re-run prek with new prettier

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-03-12 16:29:57 +00:00
Trenton H 1e21bcd26e Breaking: Drop support for Python 3.10 (#12234) 2026-03-04 15:03:33 -08:00
shamoon 709bcfd30d Fix merge thing 2026-02-28 02:34:50 -08:00
shamoon f65807b906 Merge branch 'main' into dev
# Conflicts:
#	docs/setup.md
#	src-ui/src/app/components/manage/document-attributes/management-list/management-list.component.ts
#	src/documents/tests/test_api_documents.py
2026-02-28 02:31:20 -08:00
Trenton H 5988d5896b Breaking: Refactor advanced database settings to allow more user configuration (#12165) 2026-02-27 14:37:26 -08:00
shamoon c431370bde Update setup.md 2026-02-24 15:11:21 -08:00
shamoon be82fcb70a Documentation: docs cleanup (#12158) 2026-02-24 15:10:38 -08:00
shamoon ffbbe7986f Merge branch 'main' into dev
# Conflicts:
#	docs/setup.md
2026-02-24 15:05:49 -08:00
shamoon 6de7283cbf Documentation: docs cleanup (#12158) 2026-02-24 14:59:40 -08:00
Trenton H 8db1c4e08b Breaking: Remove pybzar as a barcode reader (#12065) 2026-02-13 08:14:00 -08:00
shamoon ab328e0212 Chore: move to Zensical for docs (#12011)
(cherry picked from commit 3c51b3f9cd)
2026-02-07 10:58:55 -08:00
shamoon 3c51b3f9cd Chore: move to Zensical for docs (#12011) 2026-02-06 08:34:15 -08:00
Trenton H 51b466a86b Feature: Simplify and improve the consumer (#11753) 2026-01-21 14:37:48 -08:00
shamoon b5df90156e Documentation: update redis security link (#11373) 2025-11-15 13:31:03 -08:00
Katrin Leinweber 5410074062 Documentation: copy-edits (#10417) 2025-07-20 17:27:04 +00:00
shamoon 32a7f9cd5a Enhancement: allow webUI first account signup (#9500) 2025-03-29 17:12:34 +00:00
Trenton H 9c68100dc0 Fix: Make management commands aware of the container environment (#9499) 2025-03-26 14:17:10 -07:00
shamoon 0c9d615f56 Merge branch 'main' into beta 2025-03-14 13:07:03 -07:00
Trenton H 0faa9e8865 Chore: Split out some items into extras (#9297) 2025-03-04 22:16:09 +00:00
Trenton Handshamoon 654c9ca273 Feature: Switch webserver to granian (#9218)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2025-02-28 19:37:45 +00:00
shamoon 90561857e8 Documentation: add note about WAL mode with SQLite 2025-02-24 14:08:49 -08:00
shamoon a548c32c1f Enhancement: allow disabling the filesystem consumer (#9199) 2025-02-23 13:52:41 -08:00
Colin Yatesandshamoon 29726c3ce1 Documentation: note wget requirement for install script on macOS (#8793)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2025-01-18 00:46:47 +00:00
shamoon 5821033e3d Merge branch 'dev' 2025-01-13 07:47:08 -08:00
Malte Breitzmann 2a1c6bf0ca Documentation: remove dollar sign in code snippets (#8600) 2025-01-04 00:30:25 +00:00
Johan Ohly fbeaed930e Documentation: fix docker hub image name (#8398) 2024-12-01 07:38:49 -08:00
shamoon fcf532f13e Documentation: documentation updates 2024-11-24 14:20:20 -08:00
marph91andshamoon 605aa50b00 Documentation: Fix list indentation (#8050)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-10-28 00:02:06 +00:00
Trenton H e6f59472e4 Chore: Drop Python 3.9 support (#7774) 2024-09-26 12:22:24 -07:00
shamoon 4f8e59030e Documentation: move usually-unneeded 'mime-support' from bare-metal docs (#7689) 2024-09-11 12:58:53 -07:00
fusselkater fa19a8975e Fix: correct nltk data path (#7338) 2024-07-29 07:27:08 -07:00
Trenton H 28db7e84e6 Documentation: Corrections and clarifications for Python support (#6995)
* Clarifies Python version support and a rough policy of what versions are supported
2024-06-13 11:53:34 -07:00
Trenton H 5fc6736666 Fixes up various doc sections (#6883) 2024-06-02 03:18:12 +00:00
martin f. krafft d6191d2f2b Add copy-paste list of Python build dependencies (#6824)
To be consistent, let's provide an easily copy-pastable list of packages
even for the final set of build dependencies.

Signed-off-by: martin f. krafft <madduck@madduck.net>
2024-05-29 09:03:51 -07:00
shamoon 5d937cf639 Merge branch 'dev' 2024-05-06 13:41:08 -07:00
Lino 93555cf2e7 Documentation: add link to list of hosting providers (#6577) 2024-05-06 20:33:13 +00:00
shamoon ea8127202d Documentation: Remove SQLite to Postgres in favor of export/import (#6401) 2024-04-15 11:50:04 -07:00
shamoon f525ac0af6 Chore: add pre-commit hook for codespell (#5324) 2024-01-08 13:03:05 -08:00
shamoon f4e75c7fb7 Merge branch 'dev' 2024-01-05 21:36:01 -08:00
ChrisRBe 5963dfe41b Fix: correctly format tip admonition (#5229)
format for material admonition requires a line break and another
level of indentation which was not correctly applied on this line.
2024-01-03 14:34:59 -08:00
Christoph LoyandTrenton H 3f4f4444f7 Document invalid options in rootless containers (#5122)
Specifying `PAPERLESS_OCR_LANGUAGES` in rootless containers is not possible since the extra languages are currently installed with apt which requires root.

Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2024-01-01 08:00:05 -08:00