* Fix: match Apple Mail tag colors case-insensitively
* Fix: reject OAuth callbacks with an unrecognized scope
* Fix: refuse to refresh OAuth tokens for non-OAuth accounts or missing refresh tokens
* Fix: parse datetime strings in the format_datetime filter and reject invalid ones
* Fix: reject a PAPERLESS_URL without a hostname
* Fix: report a database error in the system status instead of failing with a 500
* Fix: surface the database error when refreshing a document before moving its files
* Fix: report no unapplied migrations when the applied migrations cannot be read
* Fix: return the parsed value directly in format_datetime for typing
* Chore: annotate the paperless_dirs fixture in the refresh failure test
* Allow null for the latest_migration system status type
---------
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
* Preprocesses classifier content with Tantivy instead of NLTK
Tokenizing and stemming now happen in one Rust call instead of NLTK's
Python tokenizer and per word stemming, which also removes the Redis
backed stem cache from every preprocessing call. The output matches the
NLTK pipeline closely; tokens containing digits are now stemmed, and the
English stop words follow Snowball's list.
Stemming and stop word removal apply whenever the OCR language is one of
the supported classifier languages, so PAPERLESS_ENABLE_NLTK and
PAPERLESS_NLTK_DIR are removed.
* Copies packages instead of hardlinking them in backend CI, some NLTK thing
* Adds a normalization to NFC to better fit what Tantivy expects
Trenton Handshamoon
