Performance: Preprocess classifier text with Tantivy instead of NLTK (#14127)

* Preprocesses classifier content with Tantivy instead of NLTK

Tokenizing and stemming now happen in one Rust call instead of NLTK's
Python tokenizer and per word stemming, which also removes the Redis
backed stem cache from every preprocessing call. The output matches the
NLTK pipeline closely; tokens containing digits are now stemmed, and the
English stop words follow Snowball's list.

Stemming and stop word removal apply whenever the OCR language is one of
the supported classifier languages, so PAPERLESS_ENABLE_NLTK and
PAPERLESS_NLTK_DIR are removed.

* Copies packages instead of hardlinking them in backend CI, some NLTK thing

* Adds a normalization to NFC to better fit what Tantivy expects

.github/dependabot.yml

@@ -81,7 +81,6 @@ updates:
       # Data, NLP, and Search
       data-nlp-search:
         patterns:
-          - "nltk"
           - "scikit-learn"
           - "langdetect"
           - "rapidfuzz"

.github/workflows/ci-backend.yml

@@ -12,7 +12,9 @@ concurrency:
   cancel-in-progress: true
 env:
   DEFAULT_UV_VERSION: "0.12.x"
-  NLTK_DATA: "/usr/share/nltk_data"
+  # Match the Docker image: nltk refuses to read hardlinked data files, such as
+  # the copy bundled with llama-index when uv links packages from its cache
+  UV_LINK_MODE: copy
 permissions: {}
 jobs:
   changes:
@@ -125,12 +127,8 @@ jobs:
       - name: List installed Python dependencies
         run: |
           uv pip list
-      - name: Install NLTK data
-        run: |
-          uv run python -m nltk.downloader punkt punkt_tab snowball_data stopwords -d "${NLTK_DATA}"
       - name: Run tests
         env:
-          NLTK_DATA: ${{ env.NLTK_DATA }}
           PAPERLESS_CI_TEST: 1
           PYTHON_VERSION: ${{ steps.setup-python.outputs.python-version }}
         run: |