* Fix: redirect SHARE_LINK_BUNDLE_DIR to the test temp layout instead of the real media directory
* Fix: include f_to in test_filters subTest labels so each of the 8 cases reports distinctly
* Fix: run the post_consume error-log assertion after the raising call and match the actual paperless_mail logger name
* Fix: rename the blank-password workflow test to match its behavior and add a real wrong-password-fails test
* Fix: use the created social account's actual pk and remove an accidental tuple wrapping the mock provider
* Fix: assert against the created documents' actual pks instead of hardcoded 1 and 2
* Fix: assert test_compression actually produces a valid LZMA-compressed zip
* Fix: clear os.environ when patching PAPERLESS_ADMIN_* vars so a host-set value can't leak into the no-user test
* Fix: restore MIDDLEWARE, AUTHENTICATION_BACKENDS and REST_FRAMEWORK auth classes after each remote-user settings test instead of leaking the mutation into later tests
* Fix: use a guaranteed-nonexistent temp path instead of hardcoded /tmp/foo/bar in test_export_target_not_exists
Snapshot only the edited field, before and after the operation, gathering tags
and custom field instances into sorted id lists per document (empty when there
are none).
The mail message and mailbox builders, the fake libmagic and the classifier preprocessor stub lived inside test_mail.py and test_classifier.py, so other test modules imported them by importing a test module.
They now live in helpers modules beside the tests that use them.
util_call_with_backoff made every call wait 20 seconds even when the first attempt succeeded
The helper now sleeps only after a failure that has another attempt left.
Test modules in paperless, paperless_mail and documents imported filesystem assertions, the migration test base, the retry helper and the streaming-response reader out of documents/tests/utils.py, which kept each app's tests coupled to another app's test package.
They now live in paperless_testing, and the progress manager fake is renamed FakeProgressManager and now subclasses the real ProgressManager, overriding only the transport, so the payload it records is built by the production code. The twenty places that patched documents.tasks.ProgressManager by hand now use a fake_progress_manager fixture.
Two fixtures created a temporary index directory and pointed INDEX_DIR at it, and paperless_dirs did the same, so a test that requested more than one got whichever assignment ran last. The search conftest no longer defines its own index_dir fixture, the tests that took it read paperless_dirs.index_dir instead, and _search_index is now a thin wrapper that requests paperless_dirs. The fixture that yields a Document is renamed from indexed_document to searchable_document so it no longer differs by one character from the index_document factory next to it.
TantivyBackend(path=None) built an in-memory index that is not used by production ever used. The backend now requires a path, the open, write-batch and rebuild branches are gone, and the shared backend fixture and the fulltext similar-documents fixture build a real index under the per-test directory layout.
* Chore: Speed up test setup by hashing passwords with MD5 and batching index writes
Don't use Django's default PBKDF2, about 600 ms per use and 110 uses across the suite. Switches to MD5 instead.
Also fixes a test that didn't batch update the search index
* Chore: Stop the invalid webhook params test from waiting on a Celery broker
test_workflow_webhook_action_url_invalid_params_headers left send_webhook.apply_async unpatched, so it tried to actually enqueu and waited for the timeout.
Nineteen single-file fixtures in the parsers conftest had no consumers anywhere in the test tree.
Two fixtures were both called samples_dir and resolved one directory apart They are now document_samples_dir and parser_samples_dir
Enables pytest-randomly, which has sat commented out in pyproject.toml
since the Pytest 9 upgrade. Tests now run in a different order every
session, so a test cannot quietly depend on another having run first.
The unittest side of the suite built its temp directory tree with
tempfile.mkdtemp and a manually enabled override_settings, cleaned up only if
tearDown ran. That is now gone. DirectoriesMixin lives alongside the layout it
bridges and does nothing but hand the paperless_dirs fixture to TestCase
subclasses as self.dirs, so both halves of the suite get the same twelve
settings, the same directory shapes and cleanup owned by tmp_path.
The mixin moves to paperless_testing.dirs rather than staying in the documents
test utilities, because modules in paperless and paperless_mail import it
across the app boundary. The thirty-eight consuming modules change only their
import line; self.dirs.scratch_dir and its siblings keep working.
The temp directory layout used by the tests was written out four separate
times: once in the documents conftest, once in the paperless checks tests,
once in a fixture local to the NFC upload tests, and once in the helper
behind the old paperless_environment context manager. Each copy covered a
different subset of the settings, so which directories a test actually got
depended on which copy it happened to reach.
The layout now lives in paperless_testing.dirs. build_paperless_dirs owns
where things go and creates them, dirs_settings maps them onto Django
setting names and is pure, and a paperless_dirs fixture in the root conftest
applies that mapping through pytest-django's settings fixture so every app
can reach it. Tests that need a second environment part way through a test
body use the paperless_environment context manager from the same module,
which expresses the identical layout through override_settings. The three
redundant implementations and the old media settings fixture are gone, and
their consumers now take paperless_dirs.
The model factories lived in the documents test package, but three other
apps needed them. The AI, mail and testing suites all reached across an app
boundary to import from documents.tests.factories, which made a private test
package into a shared dependency.
The factories now live in the shared testing package, where cross-app use is
the intended use.
All four Django apps have test code in common, but the only place to put it
was the documents app's own tests package, so paperless, paperless_ai and
paperless_mail each reached across an app boundary to import helpers and
relied on fixtures that were only defined for the documents test path.
This adds a root src/conftest.py holding the fixtures every app needs and an
new src/paperless_testing package for shared helpers a test names
Images with an out-of-spec EXIF orientation value (e.g. 0) fail
conversion with img2pdf.ExifOrientationError, which aborts archive
generation and, during consumption with OCR disabled, fails the whole
document.
Pass rotation=img2pdf.Rotation.ifvalid so invalid orientation values
are ignored while valid values (1, 3, 6, 8) are still applied.
Co-authored-by: zhzy0077 <zhzy0077@users.noreply.github.com>
* Feature: match fuzzy terms in place inside the parsed query
Fuzzy matching was a separate clause OR'd in above the query: a flat bag
of the query's words, re-parsed through tantivy's own parser, blended
beside the exact clause. Nothing around a term reached it, so a fielded
term fuzzed across every default field, a filter did not constrain it, and
an exclusion had to be hoisted back over the whole blend to stop the
clause re-admitting what the query had just excluded.
Widen each leaf where it sits instead, through emit()'s rewrite_leaf hook,
so fielding, negation, AND, REQUIRE and positive filters constrain the
fuzzy match exactly as they constrain the exact one. Each of a leaf's
words becomes a Fuzzy leaf on the leaf's own field, boosted to 0.1, beside
the leaf and any CJK alternative it already had.
* Hello?
QUERY-mode searches blended a separate bigram clause in at the top of the
query, built from the parsed AST's free-text tokens. Because it sat beside
the exact clause rather than inside the query, nothing around a CJK term
constrained its bigram match: an exclusion that was one OR branch's own
condition could never reach it, so "(東京 AND NOT secret) OR bill" still
returned the secret document.
Widen each CJK leaf where it sits instead, through emit()'s rewrite_leaf
hook, so every AND, NOT, REQUIRE, boost and field restriction around the
leaf applies to its bigram match too. Negated leaves are widened on
purpose, so "NOT X" excludes exactly what "X" matches.
* Preprocesses classifier content with Tantivy instead of NLTK
Tokenizing and stemming now happen in one Rust call instead of NLTK's
Python tokenizer and per word stemming, which also removes the Redis
backed stem cache from every preprocessing call. The output matches the
NLTK pipeline closely; tokens containing digits are now stemmed, and the
English stop words follow Snowball's list.
Stemming and stop word removal apply whenever the OCR language is one of
the supported classifier languages, so PAPERLESS_ENABLE_NLTK and
PAPERLESS_NLTK_DIR are removed.
* Copies packages instead of hardlinking them in backend CI, some NLTK thing
* Adds a normalization to NFC to better fit what Tantivy expects
Trenton H
GitHub Actions
shamoon
Zhiyuan Zhengandzhzy0077
Boris Rybalkin