* Fix: redirect SHARE_LINK_BUNDLE_DIR to the test temp layout instead of the real media directory
* Fix: include f_to in test_filters subTest labels so each of the 8 cases reports distinctly
* Fix: run the post_consume error-log assertion after the raising call and match the actual paperless_mail logger name
* Fix: rename the blank-password workflow test to match its behavior and add a real wrong-password-fails test
* Fix: use the created social account's actual pk and remove an accidental tuple wrapping the mock provider
* Fix: assert against the created documents' actual pks instead of hardcoded 1 and 2
* Fix: assert test_compression actually produces a valid LZMA-compressed zip
* Fix: clear os.environ when patching PAPERLESS_ADMIN_* vars so a host-set value can't leak into the no-user test
* Fix: restore MIDDLEWARE, AUTHENTICATION_BACKENDS and REST_FRAMEWORK auth classes after each remote-user settings test instead of leaking the mutation into later tests
* Fix: use a guaranteed-nonexistent temp path instead of hardcoded /tmp/foo/bar in test_export_target_not_exists
Test modules in paperless, paperless_mail and documents imported filesystem assertions, the migration test base, the retry helper and the streaming-response reader out of documents/tests/utils.py, which kept each app's tests coupled to another app's test package.
They now live in paperless_testing, and the progress manager fake is renamed FakeProgressManager and now subclasses the real ProgressManager, overriding only the transport, so the payload it records is built by the production code. The twenty places that patched documents.tasks.ProgressManager by hand now use a fake_progress_manager fixture.
The unittest side of the suite built its temp directory tree with
tempfile.mkdtemp and a manually enabled override_settings, cleaned up only if
tearDown ran. That is now gone. DirectoriesMixin lives alongside the layout it
bridges and does nothing but hand the paperless_dirs fixture to TestCase
subclasses as self.dirs, so both halves of the suite get the same twelve
settings, the same directory shapes and cleanup owned by tmp_path.
The mixin moves to paperless_testing.dirs rather than staying in the documents
test utilities, because modules in paperless and paperless_mail import it
across the app boundary. The thirty-eight consuming modules change only their
import line; self.dirs.scratch_dir and its siblings keep working.
The temp directory layout used by the tests was written out four separate
times: once in the documents conftest, once in the paperless checks tests,
once in a fixture local to the NFC upload tests, and once in the helper
behind the old paperless_environment context manager. Each copy covered a
different subset of the settings, so which directories a test actually got
depended on which copy it happened to reach.
The layout now lives in paperless_testing.dirs. build_paperless_dirs owns
where things go and creates them, dirs_settings maps them onto Django
setting names and is pure, and a paperless_dirs fixture in the root conftest
applies that mapping through pytest-django's settings fixture so every app
can reach it. Tests that need a second environment part way through a test
body use the paperless_environment context manager from the same module,
which expresses the identical layout through override_settings. The three
redundant implementations and the old media settings fixture are gone, and
their consumers now take paperless_dirs.
* Feature: Allow configuring the compression type and compression levels during export
Building on the zip export improvements, this now allows users to further configure the
zip to fit their needs. A simple stored zip for speed, or a high compression zstd for
the smallest archive. Full validation of the method and levels at the command line
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
* Perf: streaming manifest writer for document exporter (Phase 3)
Replaces the in-memory manifest dict accumulation with a
StreamingManifestWriter that writes records to manifest.json
incrementally, keeping only one batch resident in memory at a time.
Key changes:
- Add StreamingManifestWriter: writes to .tmp atomically, BLAKE2b
compare for --compare-json, discard() on exception
- Add _encrypt_record_inline(): per-record encryption replacing the
bulk encrypt_secret_fields() call; crypto setup moved before streaming
- Add _write_split_manifest(): extracted per-document manifest writing
- Refactor dump(): non-doc records streamed during transaction, documents
accumulated then written after filenames are assigned
- Upgrade check_and_write_json() from MD5 to BLAKE2b
- Remove encrypt_secret_fields() and unused itertools.chain import
- Add profiling marker to pyproject.toml
Measured improvement (200 docs + 200 CustomFieldInstances, same
dump() code path, only writer differs):
- Peak memory: ~50% reduction
- Memory delta: ~70% reduction
- Wall time and query count: unchanged
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Refactor: O(1) lookup table for CRYPT_FIELDS in per-record encryption
Add CRYPT_FIELDS_BY_MODEL to CryptMixin, derived from CRYPT_FIELDS at
class definition time. _encrypt_record_inline() now does a single dict
lookup instead of a linear scan per record, eliminating the loop and
break pattern.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* tests: general cleanup and fixes for runnning under docker
This now allows tests to be run under a locally built or production
docker image with something like:
`docker run --rm -v $PWD:/usr/src/paperless --entrypoint=bash paperlessngx/paperless-ngx:latest -c "uv run pytest"`
Specific fixes:
- fix unreachable code around `assertRaises` blocks
- fix `assertInt` typos
- fix `str(e)` vs `str(e.exception)` issues
- skip permission-based checks when root (in a docker container)
- catch `OSError` problems when instantiating `INotify` and
skip inotify-based tests when it's unavailable.
* Reverts most files to dev while keeping the exception assert fixes
---------
Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
* Saving some start on this
* At least partially working for the tesseract parser
* Problems with migration testing need to figure out
* Work around that error
* Fixes max m_pixels
* Moving the settings to main paperless application
* Starting some consumer options
* More fixes and work
* Fixes these last tests
* Fix max_length on OcrSettings.mode field
* Fix all fields on Common & Ocr settings serializers
* Umbrellla config view
* Revert "Umbrellla config view"
This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.
* Updates to use a single configuration object for all settings
* Squashed commit of the following:
commit 8a0a49dd57
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 23:02:47 2023 -0800
Fix formatting
commit 66b2d90c50
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 22:36:35 2023 -0800
Refactor frontend data models
commit 5723bd8dd8
Author: Adam Bogdał <adam@bogdal.pl>
Date: Wed Dec 20 01:17:43 2023 +0100
Fix: speed up admin panel for installs with a large number of documents (#5052)
commit 9b08ce1761
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 15:18:51 2023 -0800
Update PULL_REQUEST_TEMPLATE.md
commit a6248bec2d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 15:02:05 2023 -0800
Chore: Update Angular to v17 (#4980)
commit b1f6f52486
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 13:53:56 2023 -0800
Fix: Dont allow null custom_fields property via API (#5063)
commit 638d9970fd
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 13:43:50 2023 -0800
Enhancement: symmetric document links (#4907)
commit 5e8de4c1da
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date: Tue Dec 19 12:45:04 2023 -0800
Enhancement: shared icon & shared by me filter (#4859)
commit 088bad9030
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date: Tue Dec 19 12:04:03 2023 -0800
Bulk updates all the backend libraries (#5061)
* Saving some work on frontend config
* Very basic but dynamically-generated config form
* Saving work on slightly less ugly frontend config
* JSON validation for user_args field
* Fully dynamic config form
* Adds in some additional validators for a nicer error message
* Cleaning up the testing and coverage more
* Reverts unintentional change
* Adds documentation about the settings and the precedence
* Couple more commenting and style fixes
---------
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
* Initial implementation of consumption templates
* Frontend implementation of consumption templates
Testing
* Support consumption template source
* order templates, automatically add permissions
* Support title assignment in consumption templates
* Refactoring, filters to and, show sources on list
Show sources on template list, update some translation strings
Make filters and
minor testing
* Update strings
* Only update django-multiselectfield
* Basic docs, document some methods
* Improve testing coverage, template multi-assignment merges
* Implement share links
Basic implementation of share links
Make certain share link fields not editable, automatically grant permissions on migrate
Updated styling, error messages from expired / deleted links
frontend code linting, reversable sharelink migration
testing coverage
Update translation strings
No links message
* Consolidate file response methods
* improvements to share links on mobile devices
* Refactor share links file_version
* Add docs for share links
* Apply suggestions from code review
* When filtering share links, use the timezone aware now()
* Removes extra call to setup directories for usage in testing
* FIx copied badge display on some browsers
* Move copy to ngx-clipboard library
---------
Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
Trenton H
Jan Kleine
Sebastian Steinbeißer
Antoine Mérino
Ed BardsleyandTrenton H
dependabot[bot]andshamoon
Kevin Doren
