Compare commits

..
Author SHA1 Message Date
Trenton HolmesandClaude Sonnet 5 ff0ce4d123 Fix: register the filter_selection_data e2e stub after routeFromHAR, not before
Per Playwright's own docs, routeFromHAR's notFound: 'fallback' sends
unmatched requests straight to the network -- it does not chain to
other, earlier-registered page.route() handlers via route.fallback()
the way I'd assumed. Since Playwright checks routes in
reverse-registration order, a handler registered in beforeEach (i.e.
before the test body's routeFromHAR call) is checked AFTER routeFromHAR,
whose catch-all pattern intercepts everything first and sends any miss
straight to the (nonexistent, in e2e) network -- my stub never got a
chance to run. Confirmed by a second CI failure with the same "Failed
to fetch" symptom even after the CORS-header fix.

Moved the mockFilterSelectionData(page) call to immediately after each
test's own routeFromHAR(...) call instead, so it's registered later and
checked first for that specific URL, with routeFromHAR's broader
pattern still handling everything else as before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 14:54:58 -07:00
Trenton HolmesandClaude Sonnet 5 8b3665b32d Fix: add CORS header to the e2e filter_selection_data stub response
The e2e app (localhost:4200) calls the backend at localhost:8000, a
cross-origin request from the browser's perspective. Recorded HAR
responses carry the real backend's Access-Control-Allow-Origin header,
which is why they work; the stub added in the previous commit didn't
set one, so the browser rejected the fulfilled response as a CORS
violation -- same symptom as the original bug (TypeError: Failed to
fetch), confirmed via a fresh CI run after the first e2e fix attempt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 14:32:59 -07:00
Trenton HolmesandClaude Sonnet 5 9cce6d1b68 Fix: mock the new filter_selection_data request in Playwright e2e tests
CI (https://github.com/paperless-ngx/paperless-ngx/actions/runs/29777331059)
failed every e2e test that reloads the document list, with the browser
throwing "TypeError: Failed to fetch". Root cause: the e2e harness only
starts the Angular dev server (no real backend), and relies entirely on
page.routeFromHAR(..., { notFound: 'fallback' }) to mock API responses,
falling through to the real network for anything not in a recorded HAR.
The new GET /api/documents/filter_selection_data/ request added by this
branch fires on every non-search reload() and isn't in any pre-recorded
HAR fixture, so it fell through to a real network call with nothing
listening on the other end.

Added a shared mockFilterSelectionData() helper that stubs an empty
response for that endpoint, registered via test.beforeEach() before each
affected test's own routeFromHAR() call in every spec file that
exercises document-list reload (document-list, dashboard,
global-permissions, settings, document-detail). Playwright resolves
routes in reverse-registration order, so the later-registered HAR route
checks first, calls route.fallback() on a miss, and defers back to this
earlier-registered stub instead of hitting the network.

Not verified locally (Playwright browser binaries aren't installed in
this environment and the user preferred not to install them); relying
on CI to confirm.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 14:21:51 -07:00
Trenton HolmesandClaude Sonnet 5 bb77e65d52 Simplify: dedup HttpParams building, fire selection-data concurrently
Cleanups from a code-simplification pass over the prior commit:

- Extract the repeated "build HttpParams from a plain object, skipping
  null/undefined" loop (previously duplicated in list(), getFew(), and the
  new getFilterSelectionData()) into a single shared
  AbstractPaperlessService.withParams() helper.

- document-list-view.service.ts: fire the filter_selection_data request
  concurrently with the primary list request instead of nesting it inside
  the list request's success callback. They're independent (the follow-up
  only needs filterRules, not the list response), so there's no reason to
  wait on one before starting the other. Extracted into
  loadFilterSelectionData() for readability, and guarded the error path so
  a failed list request cancels the concurrent follow-up too rather than
  letting it resolve afterward and clobber the error-state reset.

- views.py: give filter_selection_data a lean base queryset
  (_base_document_queryset) instead of the full get_queryset(), which
  carries select_related/prefetch_related meant for serializing full
  Document objects. Benchmarked this specifically: no measured query-time
  impact in this path, since those calls were never iterating the
  queryset directly and were already inert here -- keeping it for
  clarity, not as a performance claim.

- Test fallout: since filter_selection_data now fires unconditionally on
  every non-search reload() rather than only after a successful primary
  flush, added a generic drain step to both spec files' afterEach so
  individual tests don't each need to know about this follow-up request
  unless they specifically assert on its response.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 10:00:08 -07:00
Trenton HolmesandClaude Sonnet 5 eb5bf53476 Fix (beta): stop blocking the document list on selection-data aggregation
The document overview sent include_selection_data=true on every list
request (page/filter/sort change), computing 5 correlated Count(DISTINCT)
aggregations over the full matching queryset inline before the list could
render. At scale (hundreds of thousands of documents, unfiltered) this is
the confirmed root cause of the document overview "eternally loading" in
paperless-ngx/paperless-ngx#13161.

Split the aggregation into its own endpoint, GET
/api/documents/filter_selection_data/, filter-scoped the same way the
list endpoint already resolves matches (no document ID enumeration
needed). The frontend now fetches it as a separate, non-blocking request
after the list has already rendered, for plain (non-search) browsing.
Full-text search keeps computing it inline since results are already
narrowed by the search backend first.

include_selection_data never shipped in a stable release (introduced
this beta cycle, not present on main), so the plain list endpoint simply
stops acting on it rather than needing a deprecation path.

Also closes a latent localStorage leak between
document-list-view.service.spec.ts tests (filterRules persisted via
localStorage were never cleared, only sessionStorage was) that this
change's new URL-dependent follow-up request exposed, and fixes two
tests that were unknowingly relying on that leaked state to pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 10:00:08 -07:00
149 changed files with 25097 additions and 30533 deletions
+1 -783
View File
@@ -1,787 +1,5 @@
# Changelog
## paperless-ngx 3.0.0
### Breaking Changes
- [BREAKING] Remove the positional arguments from the pre/post consume scripts [@stumpylog](https://github.com/stumpylog) ([#12573](https://github.com/paperless-ngx/paperless-ngx/pull/12573))
- [BREAKING] Decouple OCR control from archive file control [@stumpylog](https://github.com/stumpylog) ([#12448](https://github.com/paperless-ngx/paperless-ngx/pull/12448))
- [BREAKING] Chore: drop support for api versions \< 9 [@shamoon](https://github.com/shamoon) ([#12284](https://github.com/paperless-ngx/paperless-ngx/pull/12284))
- [BREAKING] Chore: Drop support for Python 3.10 [@stumpylog](https://github.com/stumpylog) ([#12234](https://github.com/paperless-ngx/paperless-ngx/pull/12234))
- [BREAKING] Chore: Refactor advanced database settings to allow more user configuration [@stumpylog](https://github.com/stumpylog) ([#12165](https://github.com/paperless-ngx/paperless-ngx/pull/12165))
- [BREAKING] Remove API v1 compatibility [@stumpylog](https://github.com/stumpylog) ([#12166](https://github.com/paperless-ngx/paperless-ngx/pull/12166))
- [BREAKING] Remove pybzar as a barcode reader [@stumpylog](https://github.com/stumpylog) ([#12065](https://github.com/paperless-ngx/paperless-ngx/pull/12065))
- [BREAKING] Remove support for document and thumbnail encryption [@stumpylog](https://github.com/stumpylog) ([#11850](https://github.com/paperless-ngx/paperless-ngx/pull/11850))
- [BREAKING] Feature: Simplify and improve the consumer [@stumpylog](https://github.com/stumpylog) ([#11753](https://github.com/paperless-ngx/paperless-ngx/pull/11753))
### Notable Changes
- Feature: Replace Whoosh with tantivy search backend [@stumpylog](https://github.com/stumpylog) ([#12471](https://github.com/paperless-ngx/paperless-ngx/pull/12471))
- Feature: document parser plugin framework [@stumpylog](https://github.com/stumpylog) ([#12294](https://github.com/paperless-ngx/paperless-ngx/pull/12294))
- Feature: document file versions [@shamoon](https://github.com/shamoon) ([#12061](https://github.com/paperless-ngx/paperless-ngx/pull/12061))
- Feature: sharelink bundles [@shamoon](https://github.com/shamoon) ([#11682](https://github.com/paperless-ngx/paperless-ngx/pull/11682))
- Feature: Paperless AI [@shamoon](https://github.com/shamoon) ([#10319](https://github.com/paperless-ngx/paperless-ngx/pull/10319))
- Feature: Remote OCR (Azure AI) [@shamoon](https://github.com/shamoon) ([#10320](https://github.com/paperless-ngx/paperless-ngx/pull/10320))
### Features / Enhancements
- [Beta] Paperless-ngx v3.0.0 Beta [@shamoon](https://github.com/shamoon) ([#12713](https://github.com/paperless-ngx/paperless-ngx/pull/12713))
- Tweakhancement (beta): allow some bulk operations with all [@shamoon](https://github.com/shamoon) ([#13193](https://github.com/paperless-ngx/paperless-ngx/pull/13193))
- Security (beta): enforce current permissions in autocomplete [@shamoon](https://github.com/shamoon) ([#13188](https://github.com/paperless-ngx/paperless-ngx/pull/13188))
- Security (beta): bound email linkification [@shamoon](https://github.com/shamoon) ([#13187](https://github.com/paperless-ngx/paperless-ngx/pull/13187))
- Change: update root modified timestamp on version changes [@shamoon](https://github.com/shamoon) ([#13170](https://github.com/paperless-ngx/paperless-ngx/pull/13170))
- Performance: add DB indexes for common query/sort patterns [@stumpylog](https://github.com/stumpylog) ([#13167](https://github.com/paperless-ngx/paperless-ngx/pull/13167))
- Performance: use NgOptimizedImage for thumbnail lazy loading [@shamoon](https://github.com/shamoon) ([#13169](https://github.com/paperless-ngx/paperless-ngx/pull/13169))
- Performance: Tantivy indexing optimization [@Merinorus](https://github.com/Merinorus) ([#13053](https://github.com/paperless-ngx/paperless-ngx/pull/13053))
- Chorehancement: update to Angular v22, 'zoneless' / 'reactive' [@shamoon](https://github.com/shamoon) ([#13114](https://github.com/paperless-ngx/paperless-ngx/pull/13114))
- Enhancement (beta): support LLM timeout config [@shamoon](https://github.com/shamoon) ([#13002](https://github.com/paperless-ngx/paperless-ngx/pull/13002))
- Enhancement: ignore diacritics, support multiple substring matching for UI filtering [@shamoon](https://github.com/shamoon) ([#13021](https://github.com/paperless-ngx/paperless-ngx/pull/13021))
- Enhancement(beta): replace LanceDB vector store with sqlite-vec [@stumpylog](https://github.com/stumpylog) ([#12990](https://github.com/paperless-ngx/paperless-ngx/pull/12990))
- Enhancement (beta): Switch the AI vector store to LanceDB [@stumpylog](https://github.com/stumpylog) ([#12944](https://github.com/paperless-ngx/paperless-ngx/pull/12944))
- Fixhancement (beta): tasks dismiss all [@shamoon](https://github.com/shamoon) ([#12949](https://github.com/paperless-ngx/paperless-ngx/pull/12949))
- Enhancement (beta): add direct LLM language setting [@shamoon](https://github.com/shamoon) ([#12906](https://github.com/paperless-ngx/paperless-ngx/pull/12906))
- Enhancement: try to respect language for AI suggestions [@shamoon](https://github.com/shamoon) ([#12894](https://github.com/paperless-ngx/paperless-ngx/pull/12894))
- Enhancement: AI LLM chunk size and context window config [@shamoon](https://github.com/shamoon) ([#12891](https://github.com/paperless-ngx/paperless-ngx/pull/12891))
- Enhancement: support ollama embeddings [@shamoon](https://github.com/shamoon) ([#12753](https://github.com/paperless-ngx/paperless-ngx/pull/12753))
- Tweakhancment: Include the last applied 'documents' migration in the log [@stumpylog](https://github.com/stumpylog) ([#12757](https://github.com/paperless-ngx/paperless-ngx/pull/12757))
- Enhancement: version-aware thumbnail etag [@shamoon](https://github.com/shamoon) ([#12754](https://github.com/paperless-ngx/paperless-ngx/pull/12754))
- Feature: Further reduce document importer memory usage [@stumpylog](https://github.com/stumpylog) ([#12707](https://github.com/paperless-ngx/paperless-ngx/pull/12707))
- Tweakhancement: use fixed position instead of display none for printing [@shamoon](https://github.com/shamoon) ([#12706](https://github.com/paperless-ngx/paperless-ngx/pull/12706))
- Enhancement: Paperless-ngx v3 Logo [@shamoon](https://github.com/shamoon) ([#12673](https://github.com/paperless-ngx/paperless-ngx/pull/12673))
- Tweakhancement: localize some more task result messages [@shamoon](https://github.com/shamoon) ([#12672](https://github.com/paperless-ngx/paperless-ngx/pull/12672))
- Enhancement: chat message document links [@shamoon](https://github.com/shamoon) ([#12670](https://github.com/paperless-ngx/paperless-ngx/pull/12670))
- Tweakhancement: make upload notification open an anchor link [@shamoon](https://github.com/shamoon) ([#12659](https://github.com/paperless-ngx/paperless-ngx/pull/12659))
- Tweakhancement: tweak tasks UI, make open doc an anchor [@shamoon](https://github.com/shamoon) ([#12658](https://github.com/paperless-ngx/paperless-ngx/pull/12658))
- Enhancement: show small task summary in system status [@shamoon](https://github.com/shamoon) ([#12634](https://github.com/paperless-ngx/paperless-ngx/pull/12634))
- Enhancement: new Tasks UI [@shamoon](https://github.com/shamoon) ([#12614](https://github.com/paperless-ngx/paperless-ngx/pull/12614))
- Feature: Allow monitoring access to tasks summary [@stumpylog](https://github.com/stumpylog) ([#12624](https://github.com/paperless-ngx/paperless-ngx/pull/12624))
- Tweak: remove stale index warning [@shamoon](https://github.com/shamoon) ([#12616](https://github.com/paperless-ngx/paperless-ngx/pull/12616))
- Enhancement: add highlighting to title + content searches [@shamoon](https://github.com/shamoon) ([#12593](https://github.com/paperless-ngx/paperless-ngx/pull/12593))
- Feature: Redesign the task system [@stumpylog](https://github.com/stumpylog) ([#12584](https://github.com/paperless-ngx/paperless-ngx/pull/12584))
- Feature: Tune SQLite default settings for increased speed [@stumpylog](https://github.com/stumpylog) ([#12580](https://github.com/paperless-ngx/paperless-ngx/pull/12580))
- Feature: Document fuzzy match improvements [@stumpylog](https://github.com/stumpylog) ([#12579](https://github.com/paperless-ngx/paperless-ngx/pull/12579))
- Enhancement: validate and sanitize uploaded logos [@shamoon](https://github.com/shamoon) ([#12551](https://github.com/paperless-ngx/paperless-ngx/pull/12551))
- Enhancement: add view\_global\_statistics and view\_system\_status permissions [@shamoon](https://github.com/shamoon) ([#12530](https://github.com/paperless-ngx/paperless-ngx/pull/12530))
- Fixhancement: include sharelinks + bundles in export/import [@shamoon](https://github.com/shamoon) ([#12479](https://github.com/paperless-ngx/paperless-ngx/pull/12479))
- Enhancement: unify text search to use tantivy [@shamoon](https://github.com/shamoon) ([#12485](https://github.com/paperless-ngx/paperless-ngx/pull/12485))
- Feature: Consumer logging correlation [@stumpylog](https://github.com/stumpylog) ([#12510](https://github.com/paperless-ngx/paperless-ngx/pull/12510))
- Security enhancement: allow opt-in blocking internal mail hosts [@shamoon](https://github.com/shamoon) ([#12502](https://github.com/paperless-ngx/paperless-ngx/pull/12502))
- Fixhancement: include trashed documents in document exporter/importer [@JanKleine](https://github.com/JanKleine) ([#12425](https://github.com/paperless-ngx/paperless-ngx/pull/12425))
- Tweakhancement: show file extension in StoragePath test [@shamoon](https://github.com/shamoon) ([#12452](https://github.com/paperless-ngx/paperless-ngx/pull/12452))
- Feature: Transition all checksums to use SHA256 [@stumpylog](https://github.com/stumpylog) ([#12432](https://github.com/paperless-ngx/paperless-ngx/pull/12432))
- Enhancement: auto-hide the search bar on mobile [@shamoon](https://github.com/shamoon) ([#12404](https://github.com/paperless-ngx/paperless-ngx/pull/12404))
- Change: sort custom fields alphabetically by default [@shamoon](https://github.com/shamoon) ([#12358](https://github.com/paperless-ngx/paperless-ngx/pull/12358))
- Feature: Add progress information to the classifier training for a better ux [@stumpylog](https://github.com/stumpylog) ([#12331](https://github.com/paperless-ngx/paperless-ngx/pull/12331))
- Enhancement: Make the StatusConsumer truly async [@stumpylog](https://github.com/stumpylog) ([#12298](https://github.com/paperless-ngx/paperless-ngx/pull/12298))
- Feature: Migrate import/export to rich progress [@stumpylog](https://github.com/stumpylog) ([#12260](https://github.com/paperless-ngx/paperless-ngx/pull/12260))
- Enhancement: Show more document details in merge dialog [@svenstaro](https://github.com/svenstaro) ([#12271](https://github.com/paperless-ngx/paperless-ngx/pull/12271))
- Enhancement: saved view sharing [@shamoon](https://github.com/shamoon) ([#12142](https://github.com/paperless-ngx/paperless-ngx/pull/12142))
- Enhancement: “live” document updates [@shamoon](https://github.com/shamoon) ([#12141](https://github.com/paperless-ngx/paperless-ngx/pull/12141))
- Enhancement: Improve the retagger output using rich [@stumpylog](https://github.com/stumpylog) ([#12194](https://github.com/paperless-ngx/paperless-ngx/pull/12194))
- Enhancement: Transition sanity check to rich and improve output [@stumpylog](https://github.com/stumpylog) ([#12182](https://github.com/paperless-ngx/paperless-ngx/pull/12182))
- Enhancement: Switch all indexing to use rich [@stumpylog](https://github.com/stumpylog) ([#12193](https://github.com/paperless-ngx/paperless-ngx/pull/12193))
- Feature: Switch progress bar library to rich [@stumpylog](https://github.com/stumpylog) ([#12169](https://github.com/paperless-ngx/paperless-ngx/pull/12169))
- Enhancement: Formatted filename for single document downloads [@JanKleine](https://github.com/JanKleine) ([#12095](https://github.com/paperless-ngx/paperless-ngx/pull/12095))
- Enhancement: prevent duplicate mail processing across rules [@shamoon](https://github.com/shamoon) ([#12159](https://github.com/paperless-ngx/paperless-ngx/pull/12159))
- Feature: move to trash action for workflows [@JanKleine](https://github.com/JanKleine) ([#11176](https://github.com/paperless-ngx/paperless-ngx/pull/11176))
- Tweakhancement: reset to page 1 on reset filters [@shamoon](https://github.com/shamoon) ([#12143](https://github.com/paperless-ngx/paperless-ngx/pull/12143))
- Enhancement: consolidate management lists into document attributes section [@shamoon](https://github.com/shamoon) ([#12045](https://github.com/paperless-ngx/paperless-ngx/pull/12045))
- Enhancement: option to stop processing further mail rules [@shamoon](https://github.com/shamoon) ([#12053](https://github.com/paperless-ngx/paperless-ngx/pull/12053))
- Tweak: improve 2-digit year parsing [@shamoon](https://github.com/shamoon) ([#12044](https://github.com/paperless-ngx/paperless-ngx/pull/12044))
- Enhancement: pngx pdf viewer [@shamoon](https://github.com/shamoon) ([#12043](https://github.com/paperless-ngx/paperless-ngx/pull/12043))
- Fixhancement: improve ASN handling with PDF operations [@shamoon](https://github.com/shamoon) ([#11689](https://github.com/paperless-ngx/paperless-ngx/pull/11689))
- Feature: Enable users to customize date parsing via plugins [@stumpylog](https://github.com/stumpylog) ([#11931](https://github.com/paperless-ngx/paperless-ngx/pull/11931))
- Feature: password removal workflow action [@shamoon](https://github.com/shamoon) ([#11665](https://github.com/paperless-ngx/paperless-ngx/pull/11665))
- Enhancement: per-type object page sizing [@shamoon](https://github.com/shamoon) ([#11977](https://github.com/paperless-ngx/paperless-ngx/pull/11977))
- Enhancement: improve filter drop-down performance with virtual scrolling [@shamoon](https://github.com/shamoon) ([#11973](https://github.com/paperless-ngx/paperless-ngx/pull/11973))
- Tweakhancement: tweak bulk delete text [@shamoon](https://github.com/shamoon) ([#11967](https://github.com/paperless-ngx/paperless-ngx/pull/11967))
- Feature: support split documents based on tag barcodes @schober-ch ([#11645](https://github.com/paperless-ngx/paperless-ngx/pull/11645))
- Enhancement: Add setting for default PDF Editor mode [@JanKleine](https://github.com/JanKleine) ([#11927](https://github.com/paperless-ngx/paperless-ngx/pull/11927))
- Fixhancement: auto-queue llm index if needed [@shamoon](https://github.com/shamoon) ([#11891](https://github.com/paperless-ngx/paperless-ngx/pull/11891))
- Enhancement: user control of doc details fields [@shamoon](https://github.com/shamoon) ([#11906](https://github.com/paperless-ngx/paperless-ngx/pull/11906))
- Enhancement: improve relative dates in date filter [@JanKleine](https://github.com/JanKleine) ([#11899](https://github.com/paperless-ngx/paperless-ngx/pull/11899))
- Enhancement: allow duplicates with warnings, UI for discovery [@shamoon](https://github.com/shamoon) ([#11815](https://github.com/paperless-ngx/paperless-ngx/pull/11815))
- Enhancement: configurable SSO groups claim [@Gabgobie](https://github.com/Gabgobie) ([#11841](https://github.com/paperless-ngx/paperless-ngx/pull/11841))
- Enhancement: support select all for management lists [@shamoon](https://github.com/shamoon) ([#11889](https://github.com/paperless-ngx/paperless-ngx/pull/11889))
- Tweakhancement: display document id, with copy [@shamoon](https://github.com/shamoon) ([#11896](https://github.com/paperless-ngx/paperless-ngx/pull/11896))
- Enhancement: Add support for app oidc [@paulgessinger](https://github.com/paulgessinger) ([#11756](https://github.com/paperless-ngx/paperless-ngx/pull/11756))
- Enhancement: Add 'any of' workflow trigger filters [@shamoon](https://github.com/shamoon) ([#11683](https://github.com/paperless-ngx/paperless-ngx/pull/11683))
- Enhancement: support doc\_id placeholder in workflow templates [@shamoon](https://github.com/shamoon) ([#11847](https://github.com/paperless-ngx/paperless-ngx/pull/11847))
- Performance: improve treenode inefficiencies [@shamoon](https://github.com/shamoon) ([#11606](https://github.com/paperless-ngx/paperless-ngx/pull/11606))
- Feature: password removal action [@shamoon](https://github.com/shamoon) ([#11656](https://github.com/paperless-ngx/paperless-ngx/pull/11656))
- Tweakhancement: use anchor element for management list quick filter buttons [@shamoon](https://github.com/shamoon) ([#11692](https://github.com/paperless-ngx/paperless-ngx/pull/11692))
- Feature: Indonesian translation [@shamoon](https://github.com/shamoon) ([#11641](https://github.com/paperless-ngx/paperless-ngx/pull/11641))
### Bug Fixes
- Fix (beta): prevent pdfjs offsetParent warning [@shamoon](https://github.com/shamoon) ([#13197](https://github.com/paperless-ngx/paperless-ngx/pull/13197))
- Fix (beta): dont include hidden buttons in virtual scroll container height [@shamoon](https://github.com/shamoon) ([#13194](https://github.com/paperless-ngx/paperless-ngx/pull/13194))
- Fix (beta): fix custom field bulk editing for 'all' [@shamoon](https://github.com/shamoon) ([#13191](https://github.com/paperless-ngx/paperless-ngx/pull/13191))
- Security (beta): two small permission check fixes [@shamoon](https://github.com/shamoon) ([#13186](https://github.com/paperless-ngx/paperless-ngx/pull/13186))
- Fix (beta): short-circuit ObjectOwnedOrGrantedPermissionsFilter for superusers [@stumpylog](https://github.com/stumpylog) ([#13183](https://github.com/paperless-ngx/paperless-ngx/pull/13183))
- Fix (beta): compute num\_notes via a subquery instead of Count()+distinct [@stumpylog](https://github.com/stumpylog) ([#13182](https://github.com/paperless-ngx/paperless-ngx/pull/13182))
- Fix: prevent tag assignment from reverting other pending workflow assignments [@stumpylog](https://github.com/stumpylog) ([#13178](https://github.com/paperless-ngx/paperless-ngx/pull/13178))
- Fix(beta): cache per-request effective-document resolution for thumb/metadata/preview [@stumpylog](https://github.com/stumpylog) ([#13166](https://github.com/paperless-ngx/paperless-ngx/pull/13166))
- Fix: fallback from archive version for share links [@shamoon](https://github.com/shamoon) ([#13163](https://github.com/paperless-ngx/paperless-ngx/pull/13163))
- Fix: better handle saved view errors in dashboard widgets [@shamoon](https://github.com/shamoon) ([#13164](https://github.com/paperless-ngx/paperless-ngx/pull/13164))
- Fix (beta): convert chat component to signal-backed Angular [@shamoon](https://github.com/shamoon) ([#13152](https://github.com/paperless-ngx/paperless-ngx/pull/13152))
- Fix: HTTP 500 filtering documents by tags + custom field query on MariaDB [@stumpylog](https://github.com/stumpylog) ([#13143](https://github.com/paperless-ngx/paperless-ngx/pull/13143))
- Fix: scope ProcessedMail UID deduplication by IMAP UIDVALIDITY [@chouquette](https://github.com/chouquette) ([#12839](https://github.com/paperless-ngx/paperless-ngx/pull/12839))
- Fix: allow setting any UID and GID when running rootless [@Tahvok](https://github.com/Tahvok) ([#13090](https://github.com/paperless-ngx/paperless-ngx/pull/13090))
- Fix (beta): fix sidebar document close button [@shamoon](https://github.com/shamoon) ([#13136](https://github.com/paperless-ngx/paperless-ngx/pull/13136))
- Fix: include wasm assets in pdf.js-based viewer [@shamoon](https://github.com/shamoon) ([#13122](https://github.com/paperless-ngx/paperless-ngx/pull/13122))
- Fix: fix ui labels from md5 to sha256 [@shamoon](https://github.com/shamoon) ([#13119](https://github.com/paperless-ngx/paperless-ngx/pull/13119))
- Fixhancement: dont assign empty title in workflow with broken template [@shamoon](https://github.com/shamoon) ([#13112](https://github.com/paperless-ngx/paperless-ngx/pull/13112))
- Fix: clamp mailrule max age before migration [@shamoon](https://github.com/shamoon) ([#13093](https://github.com/paperless-ngx/paperless-ngx/pull/13093))
- Fix: use latest document version for bulk download [@shamoon](https://github.com/shamoon) ([#13103](https://github.com/paperless-ngx/paperless-ngx/pull/13103))
- Fix (beta): handle relative date with comma query [@shamoon](https://github.com/shamoon) ([#13087](https://github.com/paperless-ngx/paperless-ngx/pull/13087))
- Fix (beta): fix sanity checker affected document counts [@NgoQuocViet2001](https://github.com/NgoQuocViet2001) ([#13052](https://github.com/paperless-ngx/paperless-ngx/pull/13052))
- Fix (beta): preserve Unicode in localization prompt [@shamoon](https://github.com/shamoon) ([#13055](https://github.com/paperless-ngx/paperless-ngx/pull/13055))
- Fix: change tantivy token limit to title field max length [@shamoon](https://github.com/shamoon) ([#13046](https://github.com/paperless-ngx/paperless-ngx/pull/13046))
- Fix: include last-modified in doc etag [@shamoon](https://github.com/shamoon) ([#13044](https://github.com/paperless-ngx/paperless-ngx/pull/13044))
- Fix (beta): Stream chunks during compaction to prevent oom on smaller installs [@stumpylog](https://github.com/stumpylog) ([#13014](https://github.com/paperless-ngx/paperless-ngx/pull/13014))
- Fixhancement (beta): truncate embedding queries for small chunk size [@shamoon](https://github.com/shamoon) ([#13028](https://github.com/paperless-ngx/paperless-ngx/pull/13028))
- Fix(beta): Workaround a Tantivy panic in more like this searching [@stumpylog](https://github.com/stumpylog) ([#13026](https://github.com/paperless-ngx/paperless-ngx/pull/13026))
- Fix (beta): Catch consumer files created during watcher re-creations [@stumpylog](https://github.com/stumpylog) ([#13013](https://github.com/paperless-ngx/paperless-ngx/pull/13013))
- Fix (beta): restore v2 (Whoosh) advanced-search query compatibility [@stumpylog](https://github.com/stumpylog) ([#13010](https://github.com/paperless-ngx/paperless-ngx/pull/13010))
- Fix (beta): retry celery ping and report warning on no response [@shamoon](https://github.com/shamoon) ([#13012](https://github.com/paperless-ngx/paperless-ngx/pull/13012))
- Fix (beta): don't send chat message on Enter while composing with IME (CJK) [@mzyy94](https://github.com/mzyy94) ([#12999](https://github.com/paperless-ngx/paperless-ngx/pull/12999))
- Fix: Apply unicode normalization to all paths and path components [@stumpylog](https://github.com/stumpylog) ([#12993](https://github.com/paperless-ngx/paperless-ngx/pull/12993))
- Fix: Handle the UTF 16 and BOM text files better [@stumpylog](https://github.com/stumpylog) ([#12994](https://github.com/paperless-ngx/paperless-ngx/pull/12994))
- Fix (beta): avoid DRF update calling `save` on all fields [@shamoon](https://github.com/shamoon) ([#12992](https://github.com/paperless-ngx/paperless-ngx/pull/12992))
- Fix: release pooled DB connection during AI LLM/embedding calls [@stumpylog](https://github.com/stumpylog) ([#12983](https://github.com/paperless-ngx/paperless-ngx/pull/12983))
- Fix: health-check pooled DB connections and close the pool on worker shutdown [@stumpylog](https://github.com/stumpylog) ([#12977](https://github.com/paperless-ngx/paperless-ngx/pull/12977))
- Fix: wrap long titles in delete confirm dialog [@shamoon](https://github.com/shamoon) ([#12973](https://github.com/paperless-ngx/paperless-ngx/pull/12973))
- Fix (beta): fix re-ordering in merge dialog [@shamoon](https://github.com/shamoon) ([#12967](https://github.com/paperless-ngx/paperless-ngx/pull/12967))
- Fix (beta): move task filtering to backend fully [@shamoon](https://github.com/shamoon) ([#12956](https://github.com/paperless-ngx/paperless-ngx/pull/12956))
- Fix (beta): avoid unnecessary creating new PDF with pw removal workflow [@shamoon](https://github.com/shamoon) ([#12948](https://github.com/paperless-ngx/paperless-ngx/pull/12948))
- Fix (beta): correct chat message bg color [@shamoon](https://github.com/shamoon) ([#12955](https://github.com/paperless-ngx/paperless-ngx/pull/12955))
- Fix (beta): respect disable state for suggest endpoint, require change perms [@shamoon](https://github.com/shamoon) ([#12942](https://github.com/paperless-ngx/paperless-ngx/pull/12942))
- Fix: Ensure checksum comparison is using SHA256 [@stumpylog](https://github.com/stumpylog) ([#12939](https://github.com/paperless-ngx/paperless-ngx/pull/12939))
- Fix: Restrict the AI backend at settings configure time [@stumpylog](https://github.com/stumpylog) ([#12903](https://github.com/paperless-ngx/paperless-ngx/pull/12903))
- Fix (beta): correctly apply i18n in suggestions dropdown [@shamoon](https://github.com/shamoon) ([#12905](https://github.com/paperless-ngx/paperless-ngx/pull/12905))
- Fix: Lock AI index during reading and don't index documents many times during a bulk update [@stumpylog](https://github.com/stumpylog) ([#12899](https://github.com/paperless-ngx/paperless-ngx/pull/12899))
- Fix: Minor fixes for the AI indexing [@stumpylog](https://github.com/stumpylog) ([#12893](https://github.com/paperless-ngx/paperless-ngx/pull/12893))
- Fix (beta): dont use tool calling with ollama [@shamoon](https://github.com/shamoon) ([#12896](https://github.com/paperless-ngx/paperless-ngx/pull/12896))
- Fix: Improvements for security around the AI [@stumpylog](https://github.com/stumpylog) ([#12895](https://github.com/paperless-ngx/paperless-ngx/pull/12895))
- Fix: correctly show timestamp tooltip on history item [@nathanaelhoun](https://github.com/nathanaelhoun) ([#12879](https://github.com/paperless-ngx/paperless-ngx/pull/12879))
- Fix: Fold query and autocomplete terms with Tantivy's ascii\_fold so special letters match [@stumpylog](https://github.com/stumpylog) ([#12868](https://github.com/paperless-ngx/paperless-ngx/pull/12868))
- Fix: Don't store autocomplete\_word, only index it [@stumpylog](https://github.com/stumpylog) ([#12867](https://github.com/paperless-ngx/paperless-ngx/pull/12867))
- Fix: Preserve Whoosh date range swapping in Tantviy [@stumpylog](https://github.com/stumpylog) ([#12866](https://github.com/paperless-ngx/paperless-ngx/pull/12866))
- Fix: Always release search index writer [@stumpylog](https://github.com/stumpylog) ([#12865](https://github.com/paperless-ngx/paperless-ngx/pull/12865))
- Fix: Handle CJK title, content and metadata searching [@stumpylog](https://github.com/stumpylog) ([#12862](https://github.com/paperless-ngx/paperless-ngx/pull/12862))
- Fix: Restrict date query rewrites to date or datetime fields only [@stumpylog](https://github.com/stumpylog) ([#12864](https://github.com/paperless-ngx/paperless-ngx/pull/12864))
- Fix: Missing call to tanvity wait\_merging\_threads [@stumpylog](https://github.com/stumpylog) ([#12863](https://github.com/paperless-ngx/paperless-ngx/pull/12863))
- Fix: Handle tanvity index lock contention [@stumpylog](https://github.com/stumpylog) ([#12856](https://github.com/paperless-ngx/paperless-ngx/pull/12856))
- Fix (beta): better catch chat errors [@shamoon](https://github.com/shamoon) ([#12854](https://github.com/paperless-ngx/paperless-ngx/pull/12854))
- Security (beta): fixes for v3 beta [@shamoon](https://github.com/shamoon) ([#12838](https://github.com/paperless-ngx/paperless-ngx/pull/12838))
- Fix (beta): normalize long punctuation chunks to improve embedding [@shamoon](https://github.com/shamoon) ([#12848](https://github.com/paperless-ngx/paperless-ngx/pull/12848))
- Fix: use chord.on\_error before apply\_async [@shamoon](https://github.com/shamoon) ([#12842](https://github.com/paperless-ngx/paperless-ngx/pull/12842))
- Fix: Don't use smaller integer fields for some workflow fields [@stumpylog](https://github.com/stumpylog) ([#12834](https://github.com/paperless-ngx/paperless-ngx/pull/12834))
- Fix: Password removal source file location [@stumpylog](https://github.com/stumpylog) ([#12830](https://github.com/paperless-ngx/paperless-ngx/pull/12830))
- Fix (beta): use correct html button type for custom field buttons [@shamoon](https://github.com/shamoon) ([#12819](https://github.com/paperless-ngx/paperless-ngx/pull/12819))
- Fix: Defer password removal workflow action until the file is in place [@stumpylog](https://github.com/stumpylog) ([#12814](https://github.com/paperless-ngx/paperless-ngx/pull/12814))
- Fix: Allow setting allauth rate limit configuration settings [@stumpylog](https://github.com/stumpylog) ([#12798](https://github.com/paperless-ngx/paperless-ngx/pull/12798))
- Fix: Don't embed the metadata which is already embedded into the context [@stumpylog](https://github.com/stumpylog) ([#12795](https://github.com/paperless-ngx/paperless-ngx/pull/12795))
- Fix: Sanitize dash or plus from the text search path [@stumpylog](https://github.com/stumpylog) ([#12789](https://github.com/paperless-ngx/paperless-ngx/pull/12789))
- Fix: improve new tasks ui layout across screen sizes [@shamoon](https://github.com/shamoon) ([#12784](https://github.com/paperless-ngx/paperless-ngx/pull/12784))
- Fix: Update parser contract to require empty strings, not None [@shamoon](https://github.com/shamoon) ([#12775](https://github.com/paperless-ngx/paperless-ngx/pull/12775))
- Fix: Use a persistent, writeable location for hugging face models [@stumpylog](https://github.com/stumpylog) ([#12771](https://github.com/paperless-ngx/paperless-ngx/pull/12771))
- Fix: create LLM\_INDEX\_DIR before writing meta.json on first run [@pReya](https://github.com/pReya) ([#12759](https://github.com/paperless-ngx/paperless-ngx/pull/12759))
- Fix: use response synthesizer for RAG doc chat [@shamoon](https://github.com/shamoon) ([#12751](https://github.com/paperless-ngx/paperless-ngx/pull/12751))
- Fix: only update modified field in notes actions [@shamoon](https://github.com/shamoon) ([#12750](https://github.com/paperless-ngx/paperless-ngx/pull/12750))
- Fix: Handle dash or plus operators in search queries [@stumpylog](https://github.com/stumpylog) ([#12734](https://github.com/paperless-ngx/paperless-ngx/pull/12734))
- Fixes trash preview when a document has deleted versions [@stumpylog](https://github.com/stumpylog) ([#12742](https://github.com/paperless-ngx/paperless-ngx/pull/12742))
- Fix: exclude versions from stats count [@shamoon](https://github.com/shamoon) ([#12738](https://github.com/paperless-ngx/paperless-ngx/pull/12738))
- Fixes celery chords by using Redis as our result backend [@stumpylog](https://github.com/stumpylog) ([#12741](https://github.com/paperless-ngx/paperless-ngx/pull/12741))
- Fix: bump version.py to 3.0.0 also [@shamoon](https://github.com/shamoon) ([#12736](https://github.com/paperless-ngx/paperless-ngx/pull/12736))
- Fix: makes the font cache folder writeable to all users [@stumpylog](https://github.com/stumpylog) ([#12726](https://github.com/paperless-ngx/paperless-ngx/pull/12726))
- Fixes Whoosh year only queries to be rewritten to Tantivy date syntax [@stumpylog](https://github.com/stumpylog) ([#12725](https://github.com/paperless-ngx/paperless-ngx/pull/12725))
- Fix: pass allow parallel tool calls in LLM client [@shamoon](https://github.com/shamoon) ([#12718](https://github.com/paperless-ngx/paperless-ngx/pull/12718))
- Fix: Handle passwords for removal action as lists [@stumpylog](https://github.com/stumpylog) ([#12716](https://github.com/paperless-ngx/paperless-ngx/pull/12716))
- Bumps all our versions to 3.0.0 [@stumpylog](https://github.com/stumpylog) ([#12715](https://github.com/paperless-ngx/paperless-ngx/pull/12715))
- Fix: avoid unnecessary close\_old\_connections in Celery task dispatch [@stumpylog](https://github.com/stumpylog) ([#12701](https://github.com/paperless-ngx/paperless-ngx/pull/12701))
- Fix: apply tag changes directly to document in db [@shamoon](https://github.com/shamoon) ([#12664](https://github.com/paperless-ngx/paperless-ngx/pull/12664))
- Fix: Use FileResponse for file API responses [@stumpylog](https://github.com/stumpylog) ([#12638](https://github.com/paperless-ngx/paperless-ngx/pull/12638))
- Fix: Changes bare metal webserver to use uvloop [@stumpylog](https://github.com/stumpylog) ([#12626](https://github.com/paperless-ngx/paperless-ngx/pull/12626))
- Fixhancement: Exact custom field monetary exact searching [@stumpylog](https://github.com/stumpylog) ([#12592](https://github.com/paperless-ngx/paperless-ngx/pull/12592))
- Fix: relative added date check visibility [@shamoon](https://github.com/shamoon) ([#12600](https://github.com/paperless-ngx/paperless-ngx/pull/12600))
- Fix: prevent intermediate change event when CustomFieldQueryAtom operator changes type [@ggouzi](https://github.com/ggouzi) ([#12597](https://github.com/paperless-ngx/paperless-ngx/pull/12597))
- Fix: reject invalid requests to API notes endpoint [@ggouzi](https://github.com/ggouzi) ([#12582](https://github.com/paperless-ngx/paperless-ngx/pull/12582))
- Fix: Respect workflow change\_groups during consumption [@stumpylog](https://github.com/stumpylog) ([#12431](https://github.com/paperless-ngx/paperless-ngx/pull/12431))
- Fix: Don't try to update the user when running as non-root [@stumpylog](https://github.com/stumpylog) ([#12365](https://github.com/paperless-ngx/paperless-ngx/pull/12365))
- Fix: use maxsplit=1 in Redis URL parsing to handle URLs with multiple colons [@cryptomilk](https://github.com/cryptomilk) ([#12239](https://github.com/paperless-ngx/paperless-ngx/pull/12239))
- Fixhancement: show sequential + id version labels, fix padding [@shamoon](https://github.com/shamoon) ([#12196](https://github.com/paperless-ngx/paperless-ngx/pull/12196))
- Fix: correct user dropdown button icon styling [@shamoon](https://github.com/shamoon) ([#12092](https://github.com/paperless-ngx/paperless-ngx/pull/12092))
- Fix: Handle an OOM kill of celery workers in the task handler [@stumpylog](https://github.com/stumpylog) ([#12040](https://github.com/paperless-ngx/paperless-ngx/pull/12040))
- Fix: re-run ASN check after barcode detection [@shamoon](https://github.com/shamoon) ([#11681](https://github.com/paperless-ngx/paperless-ngx/pull/11681))
- Fix: prevent infinite loading crash in mail component [@shamoon](https://github.com/shamoon) ([#11978](https://github.com/paperless-ngx/paperless-ngx/pull/11978))
- Fix: fix broken docker create\_classifier command in 2.20.6 [@shamoon](https://github.com/shamoon) ([#11965](https://github.com/paperless-ngx/paperless-ngx/pull/11965))
- Fix: ensure css color-scheme for dark mode [@shamoon](https://github.com/shamoon) ([#11855](https://github.com/paperless-ngx/paperless-ngx/pull/11855))
- Fix: fix tag list horizontal scroll, again [@shamoon](https://github.com/shamoon) ([#11839](https://github.com/paperless-ngx/paperless-ngx/pull/11839))
- Fix: ensure horizontal scroll for long tag names in list, wrap tags without parent [@shamoon](https://github.com/shamoon) ([#11811](https://github.com/paperless-ngx/paperless-ngx/pull/11811))
- Fix: use explicit order field for workflow actions [@shamoon](https://github.com/shamoon) ([#11781](https://github.com/paperless-ngx/paperless-ngx/pull/11781))
- Fix: fix recurring workflow to respect latest run time [@shamoon](https://github.com/shamoon) ([#11735](https://github.com/paperless-ngx/paperless-ngx/pull/11735))
- Fixhancement: add error handling and retry when opening index [@shamoon](https://github.com/shamoon) ([#11731](https://github.com/paperless-ngx/paperless-ngx/pull/11731))
- Fix: validate cf integer values within PostgreSQL range [@shamoon](https://github.com/shamoon) ([#11666](https://github.com/paperless-ngx/paperless-ngx/pull/11666))
- Fix: support ordering by storage path name [@shamoon](https://github.com/shamoon) ([#11661](https://github.com/paperless-ngx/paperless-ngx/pull/11661))
- Fix: propagate metadata override created value [@shamoon](https://github.com/shamoon) ([#11659](https://github.com/paperless-ngx/paperless-ngx/pull/11659))
- Fix: prevent ASN collisions for merge operations [@shamoon](https://github.com/shamoon) ([#11634](https://github.com/paperless-ngx/paperless-ngx/pull/11634))
### Documentation
- Documentation (beta): Updates documentation for new v3 features [@stumpylog](https://github.com/stumpylog) ([#13033](https://github.com/paperless-ngx/paperless-ngx/pull/13033))
- Documentation: fix PAPERLESS\_DATE\_PARSER\_LANGUAGES formatting [@shamoon](https://github.com/shamoon) ([#12997](https://github.com/paperless-ngx/paperless-ngx/pull/12997))
- Documentation: remove duplicate words in three files [@mvanhorn](https://github.com/mvanhorn) ([#12852](https://github.com/paperless-ngx/paperless-ngx/pull/12852))
- Documentation: Update v3 migration docs [@shamoon](https://github.com/shamoon) ([#12752](https://github.com/paperless-ngx/paperless-ngx/pull/12752))
### Maintenance
- Chore: separate actions from bulk edit endpoint [@shamoon](https://github.com/shamoon) ([#12286](https://github.com/paperless-ngx/paperless-ngx/pull/12286))
- Chore: address more zizmor flags [@shamoon](https://github.com/shamoon) ([#12529](https://github.com/paperless-ngx/paperless-ngx/pull/12529))
- Chore: Resolves some zizmor reported code scan findings [@stumpylog](https://github.com/stumpylog) ([#12516](https://github.com/paperless-ngx/paperless-ngx/pull/12516))
- Chore: Don't upload digets file as a zip [@stumpylog](https://github.com/stumpylog) ([#12264](https://github.com/paperless-ngx/paperless-ngx/pull/12264))
- Chore: Switches to use prek in place of pre-commit [@stumpylog](https://github.com/stumpylog) ([#12002](https://github.com/paperless-ngx/paperless-ngx/pull/12002))
- Chore: Enable mypy checking in CI [@stumpylog](https://github.com/stumpylog) ([#11991](https://github.com/paperless-ngx/paperless-ngx/pull/11991))
- Chore: upgrade to node v24 [@shamoon](https://github.com/shamoon) ([#11747](https://github.com/paperless-ngx/paperless-ngx/pull/11747))
- Performance: pre-compress static files on ARM64 [@stumpylog](https://github.com/stumpylog) ([#11721](https://github.com/paperless-ngx/paperless-ngx/pull/11721))
### Dependencies
<details>
<summary>160 changes</summary>
- Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13137](https://github.com/paperless-ngx/paperless-ngx/pull/13137))
- Chore(deps-dev): Bump the development group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13068](https://github.com/paperless-ngx/paperless-ngx/pull/13068))
- docker-compose(deps): bump greenmail/standalone from 2.1.8 to 2.1.9 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#13067](https://github.com/paperless-ngx/paperless-ngx/pull/13067))
- docker-compose(deps): bump gotenberg/gotenberg from 8.33 to 8.34 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#13066](https://github.com/paperless-ngx/paperless-ngx/pull/13066))
- docker-compose(deps): bump nginx from 1.31.1-alpine to 1.31.2-alpine in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#13065](https://github.com/paperless-ngx/paperless-ngx/pull/13065))
- docker(deps): Bump astral-sh/uv from 0.11.19-python3.12-trixie-slim to 0.11.28-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#13070](https://github.com/paperless-ngx/paperless-ngx/pull/13070))
- Chore(deps): Bump the actions group across 1 directory with 14 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13079](https://github.com/paperless-ngx/paperless-ngx/pull/13079))
- Chore(deps): Bump pdfjs-dist from 5.7.284 to 6.0.227 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13080](https://github.com/paperless-ngx/paperless-ngx/pull/13080))
- Chore(deps-dev): Bump @playwright/test from 1.60.0 to 1.61.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13078](https://github.com/paperless-ngx/paperless-ngx/pull/13078))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13076](https://github.com/paperless-ngx/paperless-ngx/pull/13076))
- Chore(deps): Bump uuid from 14.0.0 to 14.0.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13077](https://github.com/paperless-ngx/paperless-ngx/pull/13077))
- Chore(deps-dev): Bump jest-preset-angular from 16.1.5 to 17.0.0 in /src-ui in the frontend-jest-dependencies group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13075](https://github.com/paperless-ngx/paperless-ngx/pull/13075))
- Chore(deps-dev): Bump @types/node from 25.9.1 to 26.0.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13081](https://github.com/paperless-ngx/paperless-ngx/pull/13081))
- Chore(deps): Bump the npm\_and\_yarn group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13016](https://github.com/paperless-ngx/paperless-ngx/pull/13016))
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12995](https://github.com/paperless-ngx/paperless-ngx/pull/12995))
- Chore(deps): Bump the utilities-patch group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12931](https://github.com/paperless-ngx/paperless-ngx/pull/12931))
- Chore(deps-dev): Bump types-markdown from 3.10.2.20260211 to 3.10.2.20260518 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12927](https://github.com/paperless-ngx/paperless-ngx/pull/12927))
- Chore(deps): Bump aiohttp from 3.13.4 to 3.14.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12930](https://github.com/paperless-ngx/paperless-ngx/pull/12930))
- Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12923](https://github.com/paperless-ngx/paperless-ngx/pull/12923))
- docker(deps): Bump astral-sh/uv from 0.11.6-python3.12-trixie-slim to 0.11.19-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#12920](https://github.com/paperless-ngx/paperless-ngx/pull/12920))
- Chore(deps): Bump the actions group across 1 directory with 12 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12909](https://github.com/paperless-ngx/paperless-ngx/pull/12909))
- docker-compose(deps): bump apache/tika from 3.2.3.0 to 3.3.1.0 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#12912](https://github.com/paperless-ngx/paperless-ngx/pull/12912))
- docker-compose(deps): bump gotenberg/gotenberg from 8.27 to 8.33 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#12910](https://github.com/paperless-ngx/paperless-ngx/pull/12910))
- docker-compose(deps): Bump nginx from 1.29.5-alpine to 1.31.1-alpine in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#12911](https://github.com/paperless-ngx/paperless-ngx/pull/12911))
- Chore(deps-dev): Bump @playwright/test from 1.59.1 to 1.60.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12919](https://github.com/paperless-ngx/paperless-ngx/pull/12919))
- Chore(deps-dev): Bump @types/node from 25.6.0 to 25.9.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12915](https://github.com/paperless-ngx/paperless-ngx/pull/12915))
- Chore(deps): Bump pdfjs-dist from 5.6.205 to 5.7.284 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12918](https://github.com/paperless-ngx/paperless-ngx/pull/12918))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12913](https://github.com/paperless-ngx/paperless-ngx/pull/12913))
- Chore(deps-dev): Bump webpack from 5.106.2 to 5.107.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12917](https://github.com/paperless-ngx/paperless-ngx/pull/12917))
- Chore(deps): Bump zone.js from 0.16.1 to 0.16.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12916](https://github.com/paperless-ngx/paperless-ngx/pull/12916))
- Chore(deps-dev): Bump the frontend-jest-dependencies group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12908](https://github.com/paperless-ngx/paperless-ngx/pull/12908))
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12907](https://github.com/paperless-ngx/paperless-ngx/pull/12907))
- Chore(deps): Bump hono from 4.12.16 to 4.12.18 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12767](https://github.com/paperless-ngx/paperless-ngx/pull/12767))
- Chore(deps): Bump @babel/plugin-transform-modules-systemjs from 7.29.0 to 7.29.4 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12764](https://github.com/paperless-ngx/paperless-ngx/pull/12764))
- Chore(deps): Bump fast-uri from 3.1.1 to 3.1.2 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12763](https://github.com/paperless-ngx/paperless-ngx/pull/12763))
- Chore: Targeted dependency upgrades [@stumpylog](https://github.com/stumpylog) ([#12731](https://github.com/paperless-ngx/paperless-ngx/pull/12731))
- Chore(deps-dev): Bump @playwright/test from 1.59.0 to 1.59.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12692](https://github.com/paperless-ngx/paperless-ngx/pull/12692))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12689](https://github.com/paperless-ngx/paperless-ngx/pull/12689))
- Chore(deps-dev): Bump @codecov/webpack-plugin from 1.9.1 to 2.0.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12691](https://github.com/paperless-ngx/paperless-ngx/pull/12691))
- Chore(deps-dev): Bump the frontend-jest-dependencies group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12685](https://github.com/paperless-ngx/paperless-ngx/pull/12685))
- Chore(deps-dev): Bump @types/node from 25.5.0 to 25.6.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12690](https://github.com/paperless-ngx/paperless-ngx/pull/12690))
- Chore(deps-dev): Bump webpack from 5.105.3 to 5.106.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12693](https://github.com/paperless-ngx/paperless-ngx/pull/12693))
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 15 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12684](https://github.com/paperless-ngx/paperless-ngx/pull/12684))
- Chore(deps): Bump the utilities-patch group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12702](https://github.com/paperless-ngx/paperless-ngx/pull/12702))
- Chore(deps): Bump the utilities-minor group across 1 directory with 9 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12696](https://github.com/paperless-ngx/paperless-ngx/pull/12696))
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12686](https://github.com/paperless-ngx/paperless-ngx/pull/12686))
- Chore(deps): Bump ocrmypdf from 17.4.0 to 17.4.2 in the document-processing group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12687](https://github.com/paperless-ngx/paperless-ngx/pull/12687))
- Chore(deps-dev): Bump the development group with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12683](https://github.com/paperless-ngx/paperless-ngx/pull/12683))
- Chore(deps): Bump the pre-commit-dependencies group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12694](https://github.com/paperless-ngx/paperless-ngx/pull/12694))
- Chore(deps): Bump uuid from 13.0.0 to 14.0.0 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12627](https://github.com/paperless-ngx/paperless-ngx/pull/12627))
- Chore(deps): Bump lxml from 6.0.2 to 6.1.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12619](https://github.com/paperless-ngx/paperless-ngx/pull/12619))
- Chore(deps): Bump hono from 4.12.12 to 4.12.14 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12585](https://github.com/paperless-ngx/paperless-ngx/pull/12585))
- Chore(deps): Bump follow-redirects from 1.15.11 to 1.16.0 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12575](https://github.com/paperless-ngx/paperless-ngx/pull/12575))
- Chore(deps): Bump pillow from 12.1.1 to 12.2.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12574](https://github.com/paperless-ngx/paperless-ngx/pull/12574))
- Chore(deps-dev): Bump pytest from 9.0.2 to 9.0.3 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12568](https://github.com/paperless-ngx/paperless-ngx/pull/12568))
- Chore(deps): Bump the utilities-minor group across 1 directory with 19 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12540](https://github.com/paperless-ngx/paperless-ngx/pull/12540))
- Chore(deps): Bump cryptography from 46.0.6 to 46.0.7 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12546](https://github.com/paperless-ngx/paperless-ngx/pull/12546))
- Chore(deps-dev): Bump types-python-dateutil from 2.9.0.20260305 to 2.9.0.20260323 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12542](https://github.com/paperless-ngx/paperless-ngx/pull/12542))
- Chore(deps-dev): Bump types-pytz from 2025.2.0.20251108 to 2026.1.1.20260304 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12541](https://github.com/paperless-ngx/paperless-ngx/pull/12541))
- Chore(deps): Bump django-guardian from 3.3.0 to 3.3.1 in the utilities-patch group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12539](https://github.com/paperless-ngx/paperless-ngx/pull/12539))
- Chore(deps-dev): Bump zensical from 0.0.29 to 0.0.31 in the development group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12532](https://github.com/paperless-ngx/paperless-ngx/pull/12532))
- Chore(deps): Bump pdfjs-dist from 5.4.624 to 5.6.205 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12536](https://github.com/paperless-ngx/paperless-ngx/pull/12536))
- Chore(deps): Bump the actions group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12538](https://github.com/paperless-ngx/paperless-ngx/pull/12538))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12535](https://github.com/paperless-ngx/paperless-ngx/pull/12535))
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 5 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12533](https://github.com/paperless-ngx/paperless-ngx/pull/12533))
- Chore(deps-dev): Bump jest-preset-angular from 16.1.1 to 16.1.2 in /src-ui in the frontend-jest-dependencies group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12534](https://github.com/paperless-ngx/paperless-ngx/pull/12534))
- Chore(deps-dev): Bump @playwright/test from 1.58.2 to 1.59.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12537](https://github.com/paperless-ngx/paperless-ngx/pull/12537))
- Chore(deps): Bump the npm\_and\_yarn group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12531](https://github.com/paperless-ngx/paperless-ngx/pull/12531))
- Chore(deps): Bump djangorestframework from 3.16.1 to 3.17.1 in the django-ecosystem group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12488](https://github.com/paperless-ngx/paperless-ngx/pull/12488))
- Chore(deps): Bump the document-processing group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12489](https://github.com/paperless-ngx/paperless-ngx/pull/12489))
- Chore(deps): Bump the utilities-patch group across 1 directory with 5 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12499](https://github.com/paperless-ngx/paperless-ngx/pull/12499))
- Chore(deps): Bump aiohttp from 3.13.3 to 3.13.4 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12486](https://github.com/paperless-ngx/paperless-ngx/pull/12486))
- Chore(deps): Bump the pre-commit-dependencies group with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12495](https://github.com/paperless-ngx/paperless-ngx/pull/12495))
- Chore(deps): Bump lodash from 4.17.23 to 4.18.1 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12498](https://github.com/paperless-ngx/paperless-ngx/pull/12498))
- Chore(deps): Bump the actions group across 1 directory with 9 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12490](https://github.com/paperless-ngx/paperless-ngx/pull/12490))
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12491](https://github.com/paperless-ngx/paperless-ngx/pull/12491))
- Chore(deps-dev): Bump @types/node from 25.4.0 to 25.5.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12494](https://github.com/paperless-ngx/paperless-ngx/pull/12494))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12493](https://github.com/paperless-ngx/paperless-ngx/pull/12493))
- Chore(deps): Bump cryptography from 46.0.5 to 46.0.6 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12458](https://github.com/paperless-ngx/paperless-ngx/pull/12458))
- Chore(deps): Bump requests from 2.32.5 to 2.33.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12441](https://github.com/paperless-ngx/paperless-ngx/pull/12441))
- Chore(deps): Bump cbor2 from 5.8.0 to 5.9.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12424](https://github.com/paperless-ngx/paperless-ngx/pull/12424))
- Chore(deps): Bump tinytag from 2.2.0 to 2.2.1 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12396](https://github.com/paperless-ngx/paperless-ngx/pull/12396))
- Chore(deps): Bump ujson from 5.11.0 to 5.12.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12387](https://github.com/paperless-ngx/paperless-ngx/pull/12387))
- Chore(deps): Bump pyasn1 from 0.6.2 to 0.6.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12370](https://github.com/paperless-ngx/paperless-ngx/pull/12370))
- Chore(deps): Bump https://github.com/astral-sh/ruff-pre-commit from v0.15.5 to 0.15.6 in the pre-commit-dependencies group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12371](https://github.com/paperless-ngx/paperless-ngx/pull/12371))
- Chore(deps): Bump the actions group with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12377](https://github.com/paperless-ngx/paperless-ngx/pull/12377))
- Chore(deps): Bump the utilities-patch group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12382](https://github.com/paperless-ngx/paperless-ngx/pull/12382))
- Chore(deps): Bump openai from 2.24.0 to 2.26.0 in the utilities-minor group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12379](https://github.com/paperless-ngx/paperless-ngx/pull/12379))
- Chore(deps-dev): Bump types-python-dateutil from 2.9.0.20260124 to 2.9.0.20260305 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12380](https://github.com/paperless-ngx/paperless-ngx/pull/12380))
- Chore(deps): Update django-allauth[mfa,socialaccount] requirement from ~=65.14.0 to ~=65.15.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12381](https://github.com/paperless-ngx/paperless-ngx/pull/12381))
- Chore(deps-dev): Bump the frontend-jest-dependencies group in /src-ui with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12374](https://github.com/paperless-ngx/paperless-ngx/pull/12374))
- Chore(deps): Bump @ng-select/ng-select from 21.4.1 to 21.5.2 in /src-ui in the frontend-angular-dependencies group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12373](https://github.com/paperless-ngx/paperless-ngx/pull/12373))
- Chore(deps-dev): Bump @types/node from 25.3.3 to 25.4.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12376](https://github.com/paperless-ngx/paperless-ngx/pull/12376))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12375](https://github.com/paperless-ngx/paperless-ngx/pull/12375))
- Chore(deps): Bump pyopenssl from 25.3.0 to 26.0.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12363](https://github.com/paperless-ngx/paperless-ngx/pull/12363))
- Chore(deps): Bump pyjwt from 2.10.1 to 2.12.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12335](https://github.com/paperless-ngx/paperless-ngx/pull/12335))
- Chore: bump Angular dependencies to 21.2.x [@shamoon](https://github.com/shamoon) ([#12338](https://github.com/paperless-ngx/paperless-ngx/pull/12338))
- Bump ocrmypdf from 16.13.0 to 17.3.0 in the document-processing group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12267](https://github.com/paperless-ngx/paperless-ngx/pull/12267))
- Chore(deps): Bump tornado from 6.5.4 to 6.5.5 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12327](https://github.com/paperless-ngx/paperless-ngx/pull/12327))
- Chore(deps): Bump the utilities-minor group across 1 directory with 5 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12324](https://github.com/paperless-ngx/paperless-ngx/pull/12324))
- Chore(deps): Bump the pre-commit-dependencies group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12323](https://github.com/paperless-ngx/paperless-ngx/pull/12323))
- Chore(deps): Bump the actions group with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12322](https://github.com/paperless-ngx/paperless-ngx/pull/12322))
- docker(deps): bump astral-sh/uv from 0.10.7-python3.12-trixie-slim to 0.10.9-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#12265](https://github.com/paperless-ngx/paperless-ngx/pull/12265))
- Chore(deps): Bump the utilities-patch group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12291](https://github.com/paperless-ngx/paperless-ngx/pull/12291))
- Chore: update ESLint to v10 [@shamoon](https://github.com/shamoon) ([#12256](https://github.com/paperless-ngx/paperless-ngx/pull/12256))
- Bump django-allauth from 65.14.0 to 65.14.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12253](https://github.com/paperless-ngx/paperless-ngx/pull/12253))
- Bump django from 5.2.11 to 5.2.12 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12249](https://github.com/paperless-ngx/paperless-ngx/pull/12249))
- docker(deps): Bump astral-sh/uv from 0.10.5-python3.12-trixie-slim to 0.10.7-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#12191](https://github.com/paperless-ngx/paperless-ngx/pull/12191))
- Chore(deps): Bump the actions group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12224](https://github.com/paperless-ngx/paperless-ngx/pull/12224))
- Chore: Updates actions to the most specific version released [@stumpylog](https://github.com/stumpylog) ([#12222](https://github.com/paperless-ngx/paperless-ngx/pull/12222))
- Chore(deps): Bump whitenoise from 6.11.0 to 6.12.0 in the django-ecosystem group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12192](https://github.com/paperless-ngx/paperless-ngx/pull/12192))
- docker-compose(deps): Bump gotenberg/gotenberg from 8.26 to 8.27 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#12190](https://github.com/paperless-ngx/paperless-ngx/pull/12190))
- Chore: Updates s6-overlay to 3.2.2.0 [@stumpylog](https://github.com/stumpylog) ([#12189](https://github.com/paperless-ngx/paperless-ngx/pull/12189))
- Chore(deps): Bump the utilities-patch group across 1 directory with 11 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12179](https://github.com/paperless-ngx/paperless-ngx/pull/12179))
- Chore(deps): Bump nltk from 3.9.2 to 3.9.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12177](https://github.com/paperless-ngx/paperless-ngx/pull/12177))
- Chore(deps): Bump the utilities-minor group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12174](https://github.com/paperless-ngx/paperless-ngx/pull/12174))
- docker(deps): Bump astral-sh/uv from 0.10.0-python3.12-trixie-slim to 0.10.4-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#12125](https://github.com/paperless-ngx/paperless-ngx/pull/12125))
- Chore(deps): Bump pillow from 12.1.0 to 12.1.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12064](https://github.com/paperless-ngx/paperless-ngx/pull/12064))
- Chore(deps): Bump cryptography from 46.0.3 to 46.0.5 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12060](https://github.com/paperless-ngx/paperless-ngx/pull/12060))
- Chore(deps): Bump the utilities-patch group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12051](https://github.com/paperless-ngx/paperless-ngx/pull/12051))
- Chore(deps): Bump j178/prek-action from 1.1.0 to 1.1.1 in the actions group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12033](https://github.com/paperless-ngx/paperless-ngx/pull/12033))
- docker(deps): Bump astral-sh/uv from 0.9.29-python3.12-trixie-slim to 0.10.0-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#12019](https://github.com/paperless-ngx/paperless-ngx/pull/12019))
- Chore(deps-dev): Bump @playwright/test from 1.58.1 to 1.58.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12032](https://github.com/paperless-ngx/paperless-ngx/pull/12032))
- Chore(deps): Bump the utilities-minor group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12020](https://github.com/paperless-ngx/paperless-ngx/pull/12020))
- docker-compose(deps): Bump nginx from 1.29-alpine to 1.29.5-alpine in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#12018](https://github.com/paperless-ngx/paperless-ngx/pull/12018))
- Upgrade: Bump @types/node from 25.2.0 to 25.2.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12008](https://github.com/paperless-ngx/paperless-ngx/pull/12008))
- Upgrade: Bump webpack from 5.103.0 to 5.105.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12007](https://github.com/paperless-ngx/paperless-ngx/pull/12007))
- docker(deps): Bump astral-sh/uv from 0.9.26-python3.12-trixie-slim to 0.9.28-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#11980](https://github.com/paperless-ngx/paperless-ngx/pull/11980))
- docker-compose(deps): Bump gotenberg/gotenberg from 8.25 to 8.26 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#11979](https://github.com/paperless-ngx/paperless-ngx/pull/11979))
- Chore(deps): Bump the utilities-patch group with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11981](https://github.com/paperless-ngx/paperless-ngx/pull/11981))
- Chore(deps): Bump the utilities-minor group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11993](https://github.com/paperless-ngx/paperless-ngx/pull/11993))
- Chore(deps): Bump drf-spectacular-sidecar from 2025.10.1 to 2026.1.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11985](https://github.com/paperless-ngx/paperless-ngx/pull/11985))
- Chore(deps): Update django-allauth[mfa,socialaccount] requirement from ~=65.13.1 to ~=65.14.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11984](https://github.com/paperless-ngx/paperless-ngx/pull/11984))
- Chore(deps): Update granian[uvloop] requirement from ~=2.6.0 to ~=2.7.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11983](https://github.com/paperless-ngx/paperless-ngx/pull/11983))
- Chore(deps): Bump django from 5.2.10 to 5.2.11 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11988](https://github.com/paperless-ngx/paperless-ngx/pull/11988))
- Chore(deps): Bump the actions group with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11966](https://github.com/paperless-ngx/paperless-ngx/pull/11966))
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 22 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11968](https://github.com/paperless-ngx/paperless-ngx/pull/11968))
- Chore(deps): Bump zone.js from 0.15.1 to 0.16.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#11970](https://github.com/paperless-ngx/paperless-ngx/pull/11970))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11969](https://github.com/paperless-ngx/paperless-ngx/pull/11969))
- Chore(deps-dev): Bump @types/node from 24.10.1 to 25.2.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#11972](https://github.com/paperless-ngx/paperless-ngx/pull/11972))
- Chore(deps-dev): Bump @playwright/test from 1.57.0 to 1.58.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#11971](https://github.com/paperless-ngx/paperless-ngx/pull/11971))
- Chore: Bulk backend updates [@stumpylog](https://github.com/stumpylog) ([#11543](https://github.com/paperless-ngx/paperless-ngx/pull/11543))
- Chore: Manually upgrades allauth to resolve a security issue with it [@stumpylog](https://github.com/stumpylog) ([#11853](https://github.com/paperless-ngx/paperless-ngx/pull/11853))
- Chore(deps): Bump the utilities-patch group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11793](https://github.com/paperless-ngx/paperless-ngx/pull/11793))
- Chore(deps): Bump the utilities-minor group across 1 directory with 10 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11799](https://github.com/paperless-ngx/paperless-ngx/pull/11799))
- Chore(deps): Bump pyasn1 from 0.6.1 to 0.6.2 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11801](https://github.com/paperless-ngx/paperless-ngx/pull/11801))
- Chore(deps): Bump torch from 2.7.1 to 2.8.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11800](https://github.com/paperless-ngx/paperless-ngx/pull/11800))
- docker(deps): Bump astral-sh/uv from 0.9.15-python3.12-trixie-slim to 0.9.24-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#11762](https://github.com/paperless-ngx/paperless-ngx/pull/11762))
- Chore(deps): Bump brotli from 1.1.0 to 1.2.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11796](https://github.com/paperless-ngx/paperless-ngx/pull/11796))
- Chore(deps): Bump transformers from 4.51.3 to 4.53.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11797](https://github.com/paperless-ngx/paperless-ngx/pull/11797))
- Chore(deps): Bump django from 5.2.7 to 5.2.9 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11794](https://github.com/paperless-ngx/paperless-ngx/pull/11794))
- Chore(deps): Bump the llama-index group @[dependabot[bot]](https://github.com/apps/dependabot) ([#11798](https://github.com/paperless-ngx/paperless-ngx/pull/11798))
- Chore(deps): Bump marshmallow from 3.26.1 to 3.26.2 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11790](https://github.com/paperless-ngx/paperless-ngx/pull/11790))
- Chore(deps): Bump uv from 0.9.3 to 0.9.6 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11795](https://github.com/paperless-ngx/paperless-ngx/pull/11795))
- Chore(deps): Bump aiohttp from 3.11.18 to 3.13.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11789](https://github.com/paperless-ngx/paperless-ngx/pull/11789))
- Chore(deps): Bump urllib3 from 2.5.0 to 2.6.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11792](https://github.com/paperless-ngx/paperless-ngx/pull/11792))
- Chore(deps): Bump virtualenv from 20.34.0 to 20.36.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11774](https://github.com/paperless-ngx/paperless-ngx/pull/11774))
- Chore(deps): Bump azure-core from 1.33.0 to 1.38.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11776](https://github.com/paperless-ngx/paperless-ngx/pull/11776))
- Chore: upgrade to Angular v21 [@shamoon](https://github.com/shamoon) ([#11746](https://github.com/paperless-ngx/paperless-ngx/pull/11746))
- Chore(deps): Bump the actions group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11695](https://github.com/paperless-ngx/paperless-ngx/pull/11695))
</details>
### All App Changes
<details>
<summary>360 changes</summary>
- [Beta] Paperless-ngx v3.0.0 Beta [@shamoon](https://github.com/shamoon) ([#12713](https://github.com/paperless-ngx/paperless-ngx/pull/12713))
- Chore (beta): mark first batch of thumbnails as priority for LCP warning [@shamoon](https://github.com/shamoon) ([#13196](https://github.com/paperless-ngx/paperless-ngx/pull/13196))
- Fix (beta): prevent pdfjs offsetParent warning [@shamoon](https://github.com/shamoon) ([#13197](https://github.com/paperless-ngx/paperless-ngx/pull/13197))
- Fix (beta): dont include hidden buttons in virtual scroll container height [@shamoon](https://github.com/shamoon) ([#13194](https://github.com/paperless-ngx/paperless-ngx/pull/13194))
- Tweakhancement (beta): allow some bulk operations with all [@shamoon](https://github.com/shamoon) ([#13193](https://github.com/paperless-ngx/paperless-ngx/pull/13193))
- Fix (beta): fix custom field bulk editing for 'all' [@shamoon](https://github.com/shamoon) ([#13191](https://github.com/paperless-ngx/paperless-ngx/pull/13191))
- Security (beta): enforce current permissions in autocomplete [@shamoon](https://github.com/shamoon) ([#13188](https://github.com/paperless-ngx/paperless-ngx/pull/13188))
- Security (beta): bound email linkification [@shamoon](https://github.com/shamoon) ([#13187](https://github.com/paperless-ngx/paperless-ngx/pull/13187))
- Security (beta): two small permission check fixes [@shamoon](https://github.com/shamoon) ([#13186](https://github.com/paperless-ngx/paperless-ngx/pull/13186))
- Fix (beta): short-circuit ObjectOwnedOrGrantedPermissionsFilter for superusers [@stumpylog](https://github.com/stumpylog) ([#13183](https://github.com/paperless-ngx/paperless-ngx/pull/13183))
- Fix (beta): compute num\_notes via a subquery instead of Count()+distinct [@stumpylog](https://github.com/stumpylog) ([#13182](https://github.com/paperless-ngx/paperless-ngx/pull/13182))
- Change: update root modified timestamp on version changes [@shamoon](https://github.com/shamoon) ([#13170](https://github.com/paperless-ngx/paperless-ngx/pull/13170))
- Fix: prevent tag assignment from reverting other pending workflow assignments [@stumpylog](https://github.com/stumpylog) ([#13178](https://github.com/paperless-ngx/paperless-ngx/pull/13178))
- Performance: add DB indexes for common query/sort patterns [@stumpylog](https://github.com/stumpylog) ([#13167](https://github.com/paperless-ngx/paperless-ngx/pull/13167))
- Performance: use NgOptimizedImage for thumbnail lazy loading [@shamoon](https://github.com/shamoon) ([#13169](https://github.com/paperless-ngx/paperless-ngx/pull/13169))
- Fix(beta): cache per-request effective-document resolution for thumb/metadata/preview [@stumpylog](https://github.com/stumpylog) ([#13166](https://github.com/paperless-ngx/paperless-ngx/pull/13166))
- Fix: fallback from archive version for share links [@shamoon](https://github.com/shamoon) ([#13163](https://github.com/paperless-ngx/paperless-ngx/pull/13163))
- Fix: better handle saved view errors in dashboard widgets [@shamoon](https://github.com/shamoon) ([#13164](https://github.com/paperless-ngx/paperless-ngx/pull/13164))
- Fix (beta): convert chat component to signal-backed Angular [@shamoon](https://github.com/shamoon) ([#13152](https://github.com/paperless-ngx/paperless-ngx/pull/13152))
- Fix: HTTP 500 filtering documents by tags + custom field query on MariaDB [@stumpylog](https://github.com/stumpylog) ([#13143](https://github.com/paperless-ngx/paperless-ngx/pull/13143))
- Performance: Tantivy indexing optimization [@Merinorus](https://github.com/Merinorus) ([#13053](https://github.com/paperless-ngx/paperless-ngx/pull/13053))
- Chore(deps-dev): Bump the development group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13068](https://github.com/paperless-ngx/paperless-ngx/pull/13068))
- Fix: scope ProcessedMail UID deduplication by IMAP UIDVALIDITY [@chouquette](https://github.com/chouquette) ([#12839](https://github.com/paperless-ngx/paperless-ngx/pull/12839))
- Fix (beta): fix sidebar document close button [@shamoon](https://github.com/shamoon) ([#13136](https://github.com/paperless-ngx/paperless-ngx/pull/13136))
- Fix: include wasm assets in pdf.js-based viewer [@shamoon](https://github.com/shamoon) ([#13122](https://github.com/paperless-ngx/paperless-ngx/pull/13122))
- Fix: fix ui labels from md5 to sha256 [@shamoon](https://github.com/shamoon) ([#13119](https://github.com/paperless-ngx/paperless-ngx/pull/13119))
- Chorehancement: update to Angular v22, 'zoneless' / 'reactive' [@shamoon](https://github.com/shamoon) ([#13114](https://github.com/paperless-ngx/paperless-ngx/pull/13114))
- Fixhancement: dont assign empty title in workflow with broken template [@shamoon](https://github.com/shamoon) ([#13112](https://github.com/paperless-ngx/paperless-ngx/pull/13112))
- Fix: clamp mailrule max age before migration [@shamoon](https://github.com/shamoon) ([#13093](https://github.com/paperless-ngx/paperless-ngx/pull/13093))
- Fix: use latest document version for bulk download [@shamoon](https://github.com/shamoon) ([#13103](https://github.com/paperless-ngx/paperless-ngx/pull/13103))
- Fix (beta): handle relative date with comma query [@shamoon](https://github.com/shamoon) ([#13087](https://github.com/paperless-ngx/paperless-ngx/pull/13087))
- Chore(deps): Bump pdfjs-dist from 5.7.284 to 6.0.227 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13080](https://github.com/paperless-ngx/paperless-ngx/pull/13080))
- Chore(deps-dev): Bump @playwright/test from 1.60.0 to 1.61.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13078](https://github.com/paperless-ngx/paperless-ngx/pull/13078))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13076](https://github.com/paperless-ngx/paperless-ngx/pull/13076))
- Chore(deps): Bump uuid from 14.0.0 to 14.0.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13077](https://github.com/paperless-ngx/paperless-ngx/pull/13077))
- Chore(deps-dev): Bump jest-preset-angular from 16.1.5 to 17.0.0 in /src-ui in the frontend-jest-dependencies group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13075](https://github.com/paperless-ngx/paperless-ngx/pull/13075))
- Chore(deps-dev): Bump @types/node from 25.9.1 to 26.0.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13081](https://github.com/paperless-ngx/paperless-ngx/pull/13081))
- Fix (beta): fix sanity checker affected document counts [@NgoQuocViet2001](https://github.com/NgoQuocViet2001) ([#13052](https://github.com/paperless-ngx/paperless-ngx/pull/13052))
- Fix (beta): preserve Unicode in localization prompt [@shamoon](https://github.com/shamoon) ([#13055](https://github.com/paperless-ngx/paperless-ngx/pull/13055))
- Fix: change tantivy token limit to title field max length [@shamoon](https://github.com/shamoon) ([#13046](https://github.com/paperless-ngx/paperless-ngx/pull/13046))
- Fix: include last-modified in doc etag [@shamoon](https://github.com/shamoon) ([#13044](https://github.com/paperless-ngx/paperless-ngx/pull/13044))
- Fix (beta): Stream chunks during compaction to prevent oom on smaller installs [@stumpylog](https://github.com/stumpylog) ([#13014](https://github.com/paperless-ngx/paperless-ngx/pull/13014))
- Fixhancement (beta): truncate embedding queries for small chunk size [@shamoon](https://github.com/shamoon) ([#13028](https://github.com/paperless-ngx/paperless-ngx/pull/13028))
- Enhancement (beta): support LLM timeout config [@shamoon](https://github.com/shamoon) ([#13002](https://github.com/paperless-ngx/paperless-ngx/pull/13002))
- Fix(beta): Workaround a Tantivy panic in more like this searching [@stumpylog](https://github.com/stumpylog) ([#13026](https://github.com/paperless-ngx/paperless-ngx/pull/13026))
- Chore (beta): set tool\_required to openai-like llm calls [@shamoon](https://github.com/shamoon) ([#13025](https://github.com/paperless-ngx/paperless-ngx/pull/13025))
- Enhancement: ignore diacritics, support multiple substring matching for UI filtering [@shamoon](https://github.com/shamoon) ([#13021](https://github.com/paperless-ngx/paperless-ngx/pull/13021))
- Fix (beta): Catch consumer files created during watcher re-creations [@stumpylog](https://github.com/stumpylog) ([#13013](https://github.com/paperless-ngx/paperless-ngx/pull/13013))
- Chore(deps): Bump the npm\_and\_yarn group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13016](https://github.com/paperless-ngx/paperless-ngx/pull/13016))
- Fix (beta): restore v2 (Whoosh) advanced-search query compatibility [@stumpylog](https://github.com/stumpylog) ([#13010](https://github.com/paperless-ngx/paperless-ngx/pull/13010))
- Fix (beta): retry celery ping and report warning on no response [@shamoon](https://github.com/shamoon) ([#13012](https://github.com/paperless-ngx/paperless-ngx/pull/13012))
- Enhancement(beta): replace LanceDB vector store with sqlite-vec [@stumpylog](https://github.com/stumpylog) ([#12990](https://github.com/paperless-ngx/paperless-ngx/pull/12990))
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12995](https://github.com/paperless-ngx/paperless-ngx/pull/12995))
- Chore(deps): Bump the utilities-patch group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12931](https://github.com/paperless-ngx/paperless-ngx/pull/12931))
- Fix (beta): don't send chat message on Enter while composing with IME (CJK) [@mzyy94](https://github.com/mzyy94) ([#12999](https://github.com/paperless-ngx/paperless-ngx/pull/12999))
- Fix: Apply unicode normalization to all paths and path components [@stumpylog](https://github.com/stumpylog) ([#12993](https://github.com/paperless-ngx/paperless-ngx/pull/12993))
- Fix: Handle the UTF 16 and BOM text files better [@stumpylog](https://github.com/stumpylog) ([#12994](https://github.com/paperless-ngx/paperless-ngx/pull/12994))
- Fix (beta): avoid DRF update calling `save` on all fields [@shamoon](https://github.com/shamoon) ([#12992](https://github.com/paperless-ngx/paperless-ngx/pull/12992))
- Fix: release pooled DB connection during AI LLM/embedding calls [@stumpylog](https://github.com/stumpylog) ([#12983](https://github.com/paperless-ngx/paperless-ngx/pull/12983))
- Fix: health-check pooled DB connections and close the pool on worker shutdown [@stumpylog](https://github.com/stumpylog) ([#12977](https://github.com/paperless-ngx/paperless-ngx/pull/12977))
- Fix: wrap long titles in delete confirm dialog [@shamoon](https://github.com/shamoon) ([#12973](https://github.com/paperless-ngx/paperless-ngx/pull/12973))
- Fix (beta): fix re-ordering in merge dialog [@shamoon](https://github.com/shamoon) ([#12967](https://github.com/paperless-ngx/paperless-ngx/pull/12967))
- Fix (beta): move task filtering to backend fully [@shamoon](https://github.com/shamoon) ([#12956](https://github.com/paperless-ngx/paperless-ngx/pull/12956))
- Chore(deps-dev): Bump types-markdown from 3.10.2.20260211 to 3.10.2.20260518 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12927](https://github.com/paperless-ngx/paperless-ngx/pull/12927))
- Chore(deps): Bump aiohttp from 3.13.4 to 3.14.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12930](https://github.com/paperless-ngx/paperless-ngx/pull/12930))
- Fix (beta): avoid unnecessary creating new PDF with pw removal workflow [@shamoon](https://github.com/shamoon) ([#12948](https://github.com/paperless-ngx/paperless-ngx/pull/12948))
- Enhancement (beta): Switch the AI vector store to LanceDB [@stumpylog](https://github.com/stumpylog) ([#12944](https://github.com/paperless-ngx/paperless-ngx/pull/12944))
- Fixhancement (beta): tasks dismiss all [@shamoon](https://github.com/shamoon) ([#12949](https://github.com/paperless-ngx/paperless-ngx/pull/12949))
- Fix (beta): correct chat message bg color [@shamoon](https://github.com/shamoon) ([#12955](https://github.com/paperless-ngx/paperless-ngx/pull/12955))
- Fix (beta): respect disable state for suggest endpoint, require change perms [@shamoon](https://github.com/shamoon) ([#12942](https://github.com/paperless-ngx/paperless-ngx/pull/12942))
- Fix: Ensure checksum comparison is using SHA256 [@stumpylog](https://github.com/stumpylog) ([#12939](https://github.com/paperless-ngx/paperless-ngx/pull/12939))
- Fix: Restrict the AI backend at settings configure time [@stumpylog](https://github.com/stumpylog) ([#12903](https://github.com/paperless-ngx/paperless-ngx/pull/12903))
- Enhancement (beta): add direct LLM language setting [@shamoon](https://github.com/shamoon) ([#12906](https://github.com/paperless-ngx/paperless-ngx/pull/12906))
- Chore(deps-dev): Bump @playwright/test from 1.59.1 to 1.60.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12919](https://github.com/paperless-ngx/paperless-ngx/pull/12919))
- Fix (beta): correctly apply i18n in suggestions dropdown [@shamoon](https://github.com/shamoon) ([#12905](https://github.com/paperless-ngx/paperless-ngx/pull/12905))
- Chore(deps-dev): Bump @types/node from 25.6.0 to 25.9.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12915](https://github.com/paperless-ngx/paperless-ngx/pull/12915))
- Chore(deps): Bump pdfjs-dist from 5.6.205 to 5.7.284 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12918](https://github.com/paperless-ngx/paperless-ngx/pull/12918))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12913](https://github.com/paperless-ngx/paperless-ngx/pull/12913))
- Chore(deps-dev): Bump webpack from 5.106.2 to 5.107.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12917](https://github.com/paperless-ngx/paperless-ngx/pull/12917))
- Chore(deps): Bump zone.js from 0.16.1 to 0.16.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12916](https://github.com/paperless-ngx/paperless-ngx/pull/12916))
- Chore(deps-dev): Bump the frontend-jest-dependencies group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12908](https://github.com/paperless-ngx/paperless-ngx/pull/12908))
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12907](https://github.com/paperless-ngx/paperless-ngx/pull/12907))
- Fix: Lock AI index during reading and don't index documents many times during a bulk update [@stumpylog](https://github.com/stumpylog) ([#12899](https://github.com/paperless-ngx/paperless-ngx/pull/12899))
- Fix: Minor fixes for the AI indexing [@stumpylog](https://github.com/stumpylog) ([#12893](https://github.com/paperless-ngx/paperless-ngx/pull/12893))
- Fix (beta): dont use tool calling with ollama [@shamoon](https://github.com/shamoon) ([#12896](https://github.com/paperless-ngx/paperless-ngx/pull/12896))
- Enhancement: try to respect language for AI suggestions [@shamoon](https://github.com/shamoon) ([#12894](https://github.com/paperless-ngx/paperless-ngx/pull/12894))
- Enhancement: AI LLM chunk size and context window config [@shamoon](https://github.com/shamoon) ([#12891](https://github.com/paperless-ngx/paperless-ngx/pull/12891))
- Fix: Improvements for security around the AI [@stumpylog](https://github.com/stumpylog) ([#12895](https://github.com/paperless-ngx/paperless-ngx/pull/12895))
- Fix: correctly show timestamp tooltip on history item [@nathanaelhoun](https://github.com/nathanaelhoun) ([#12879](https://github.com/paperless-ngx/paperless-ngx/pull/12879))
- Fix: Fold query and autocomplete terms with Tantivy's ascii\_fold so special letters match [@stumpylog](https://github.com/stumpylog) ([#12868](https://github.com/paperless-ngx/paperless-ngx/pull/12868))
- Fix: Don't store autocomplete\_word, only index it [@stumpylog](https://github.com/stumpylog) ([#12867](https://github.com/paperless-ngx/paperless-ngx/pull/12867))
- Fix: Preserve Whoosh date range swapping in Tantviy [@stumpylog](https://github.com/stumpylog) ([#12866](https://github.com/paperless-ngx/paperless-ngx/pull/12866))
- Fix: Always release search index writer [@stumpylog](https://github.com/stumpylog) ([#12865](https://github.com/paperless-ngx/paperless-ngx/pull/12865))
- Fix: Handle CJK title, content and metadata searching [@stumpylog](https://github.com/stumpylog) ([#12862](https://github.com/paperless-ngx/paperless-ngx/pull/12862))
- Fix: Restrict date query rewrites to date or datetime fields only [@stumpylog](https://github.com/stumpylog) ([#12864](https://github.com/paperless-ngx/paperless-ngx/pull/12864))
- Fix: Missing call to tanvity wait\_merging\_threads [@stumpylog](https://github.com/stumpylog) ([#12863](https://github.com/paperless-ngx/paperless-ngx/pull/12863))
- Fix: Handle tanvity index lock contention [@stumpylog](https://github.com/stumpylog) ([#12856](https://github.com/paperless-ngx/paperless-ngx/pull/12856))
- Fix (beta): better catch chat errors [@shamoon](https://github.com/shamoon) ([#12854](https://github.com/paperless-ngx/paperless-ngx/pull/12854))
- Performance (beta): dont re-build vector index with each chat [@shamoon](https://github.com/shamoon) ([#12847](https://github.com/paperless-ngx/paperless-ngx/pull/12847))
- Security (beta): fixes for v3 beta [@shamoon](https://github.com/shamoon) ([#12838](https://github.com/paperless-ngx/paperless-ngx/pull/12838))
- Fix (beta): normalize long punctuation chunks to improve embedding [@shamoon](https://github.com/shamoon) ([#12848](https://github.com/paperless-ngx/paperless-ngx/pull/12848))
- Fix: use chord.on\_error before apply\_async [@shamoon](https://github.com/shamoon) ([#12842](https://github.com/paperless-ngx/paperless-ngx/pull/12842))
- Fix: Don't use smaller integer fields for some workflow fields [@stumpylog](https://github.com/stumpylog) ([#12834](https://github.com/paperless-ngx/paperless-ngx/pull/12834))
- Fix: Password removal source file location [@stumpylog](https://github.com/stumpylog) ([#12830](https://github.com/paperless-ngx/paperless-ngx/pull/12830))
- Fix (beta): use correct html button type for custom field buttons [@shamoon](https://github.com/shamoon) ([#12819](https://github.com/paperless-ngx/paperless-ngx/pull/12819))
- Fix: Defer password removal workflow action until the file is in place [@stumpylog](https://github.com/stumpylog) ([#12814](https://github.com/paperless-ngx/paperless-ngx/pull/12814))
- Fix: Allow setting allauth rate limit configuration settings [@stumpylog](https://github.com/stumpylog) ([#12798](https://github.com/paperless-ngx/paperless-ngx/pull/12798))
- Fix: Don't embed the metadata which is already embedded into the context [@stumpylog](https://github.com/stumpylog) ([#12795](https://github.com/paperless-ngx/paperless-ngx/pull/12795))
- Fix: Sanitize dash or plus from the text search path [@stumpylog](https://github.com/stumpylog) ([#12789](https://github.com/paperless-ngx/paperless-ngx/pull/12789))
- Fix: improve new tasks ui layout across screen sizes [@shamoon](https://github.com/shamoon) ([#12784](https://github.com/paperless-ngx/paperless-ngx/pull/12784))
- Fix: Update parser contract to require empty strings, not None [@shamoon](https://github.com/shamoon) ([#12775](https://github.com/paperless-ngx/paperless-ngx/pull/12775))
- Chore: Further dependency security updates [@stumpylog](https://github.com/stumpylog) ([#12780](https://github.com/paperless-ngx/paperless-ngx/pull/12780))
- Fix: Use a persistent, writeable location for hugging face models [@stumpylog](https://github.com/stumpylog) ([#12771](https://github.com/paperless-ngx/paperless-ngx/pull/12771))
- Chore(deps): Bump hono from 4.12.16 to 4.12.18 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12767](https://github.com/paperless-ngx/paperless-ngx/pull/12767))
- Enhancement: support ollama embeddings [@shamoon](https://github.com/shamoon) ([#12753](https://github.com/paperless-ngx/paperless-ngx/pull/12753))
- Fix: create LLM\_INDEX\_DIR before writing meta.json on first run [@pReya](https://github.com/pReya) ([#12759](https://github.com/paperless-ngx/paperless-ngx/pull/12759))
- Chore(deps): Bump @babel/plugin-transform-modules-systemjs from 7.29.0 to 7.29.4 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12764](https://github.com/paperless-ngx/paperless-ngx/pull/12764))
- Chore(deps): Bump fast-uri from 3.1.1 to 3.1.2 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12763](https://github.com/paperless-ngx/paperless-ngx/pull/12763))
- Tweakhancment: Include the last applied 'documents' migration in the log [@stumpylog](https://github.com/stumpylog) ([#12757](https://github.com/paperless-ngx/paperless-ngx/pull/12757))
- Fix: use response synthesizer for RAG doc chat [@shamoon](https://github.com/shamoon) ([#12751](https://github.com/paperless-ngx/paperless-ngx/pull/12751))
- Fix: only update modified field in notes actions [@shamoon](https://github.com/shamoon) ([#12750](https://github.com/paperless-ngx/paperless-ngx/pull/12750))
- Enhancement: version-aware thumbnail etag [@shamoon](https://github.com/shamoon) ([#12754](https://github.com/paperless-ngx/paperless-ngx/pull/12754))
- Documentation: Update v3 migration docs [@shamoon](https://github.com/shamoon) ([#12752](https://github.com/paperless-ngx/paperless-ngx/pull/12752))
- Fix: Handle dash or plus operators in search queries [@stumpylog](https://github.com/stumpylog) ([#12734](https://github.com/paperless-ngx/paperless-ngx/pull/12734))
- Fixes trash preview when a document has deleted versions [@stumpylog](https://github.com/stumpylog) ([#12742](https://github.com/paperless-ngx/paperless-ngx/pull/12742))
- Fix: exclude versions from stats count [@shamoon](https://github.com/shamoon) ([#12738](https://github.com/paperless-ngx/paperless-ngx/pull/12738))
- Fixes celery chords by using Redis as our result backend [@stumpylog](https://github.com/stumpylog) ([#12741](https://github.com/paperless-ngx/paperless-ngx/pull/12741))
- Fix: bump version.py to 3.0.0 also [@shamoon](https://github.com/shamoon) ([#12736](https://github.com/paperless-ngx/paperless-ngx/pull/12736))
- Chore: Targeted dependency upgrades [@stumpylog](https://github.com/stumpylog) ([#12731](https://github.com/paperless-ngx/paperless-ngx/pull/12731))
- Fixes Whoosh year only queries to be rewritten to Tantivy date syntax [@stumpylog](https://github.com/stumpylog) ([#12725](https://github.com/paperless-ngx/paperless-ngx/pull/12725))
- Fix: pass allow parallel tool calls in LLM client [@shamoon](https://github.com/shamoon) ([#12718](https://github.com/paperless-ngx/paperless-ngx/pull/12718))
- Fix: Handle passwords for removal action as lists [@stumpylog](https://github.com/stumpylog) ([#12716](https://github.com/paperless-ngx/paperless-ngx/pull/12716))
- Bumps all our versions to 3.0.0 [@stumpylog](https://github.com/stumpylog) ([#12715](https://github.com/paperless-ngx/paperless-ngx/pull/12715))
- Feature: Further reduce document importer memory usage [@stumpylog](https://github.com/stumpylog) ([#12707](https://github.com/paperless-ngx/paperless-ngx/pull/12707))
- Chore(deps-dev): Bump @playwright/test from 1.59.0 to 1.59.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12692](https://github.com/paperless-ngx/paperless-ngx/pull/12692))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12689](https://github.com/paperless-ngx/paperless-ngx/pull/12689))
- Chore(deps-dev): Bump @codecov/webpack-plugin from 1.9.1 to 2.0.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12691](https://github.com/paperless-ngx/paperless-ngx/pull/12691))
- Chore(deps-dev): Bump the frontend-jest-dependencies group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12685](https://github.com/paperless-ngx/paperless-ngx/pull/12685))
- Chore(deps-dev): Bump @types/node from 25.5.0 to 25.6.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12690](https://github.com/paperless-ngx/paperless-ngx/pull/12690))
- Chore(deps-dev): Bump webpack from 5.105.3 to 5.106.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12693](https://github.com/paperless-ngx/paperless-ngx/pull/12693))
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 15 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12684](https://github.com/paperless-ngx/paperless-ngx/pull/12684))
- Tweakhancement: use fixed position instead of display none for printing [@shamoon](https://github.com/shamoon) ([#12706](https://github.com/paperless-ngx/paperless-ngx/pull/12706))
- Fix: avoid unnecessary close\_old\_connections in Celery task dispatch [@stumpylog](https://github.com/stumpylog) ([#12701](https://github.com/paperless-ngx/paperless-ngx/pull/12701))
- Chore(deps): Bump the utilities-patch group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12702](https://github.com/paperless-ngx/paperless-ngx/pull/12702))
- Chore(deps): Bump the utilities-minor group across 1 directory with 9 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12696](https://github.com/paperless-ngx/paperless-ngx/pull/12696))
- Chore(deps): Bump ocrmypdf from 17.4.0 to 17.4.2 in the document-processing group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12687](https://github.com/paperless-ngx/paperless-ngx/pull/12687))
- Chore(deps-dev): Bump the development group with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12683](https://github.com/paperless-ngx/paperless-ngx/pull/12683))
- Enhancement: Paperless-ngx v3 Logo [@shamoon](https://github.com/shamoon) ([#12673](https://github.com/paperless-ngx/paperless-ngx/pull/12673))
- Tweakhancement: localize some more task result messages [@shamoon](https://github.com/shamoon) ([#12672](https://github.com/paperless-ngx/paperless-ngx/pull/12672))
- Enhancement: chat message document links [@shamoon](https://github.com/shamoon) ([#12670](https://github.com/paperless-ngx/paperless-ngx/pull/12670))
- Fix: apply tag changes directly to document in db [@shamoon](https://github.com/shamoon) ([#12664](https://github.com/paperless-ngx/paperless-ngx/pull/12664))
- Tweakhancement: make upload notification open an anchor link [@shamoon](https://github.com/shamoon) ([#12659](https://github.com/paperless-ngx/paperless-ngx/pull/12659))
- Tweakhancement: tweak tasks UI, make open doc an anchor [@shamoon](https://github.com/shamoon) ([#12658](https://github.com/paperless-ngx/paperless-ngx/pull/12658))
- Fix: Use FileResponse for file API responses [@stumpylog](https://github.com/stumpylog) ([#12638](https://github.com/paperless-ngx/paperless-ngx/pull/12638))
- Chore: Paginate the task listing [@stumpylog](https://github.com/stumpylog) ([#12633](https://github.com/paperless-ngx/paperless-ngx/pull/12633))
- Enhancement: show small task summary in system status [@shamoon](https://github.com/shamoon) ([#12634](https://github.com/paperless-ngx/paperless-ngx/pull/12634))
- Security: Don't allow the example secret key as a secret key [@stumpylog](https://github.com/stumpylog) ([#12630](https://github.com/paperless-ngx/paperless-ngx/pull/12630))
- Chore(deps): Bump uuid from 13.0.0 to 14.0.0 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12627](https://github.com/paperless-ngx/paperless-ngx/pull/12627))
- Enhancement: new Tasks UI [@shamoon](https://github.com/shamoon) ([#12614](https://github.com/paperless-ngx/paperless-ngx/pull/12614))
- Feature: Allow monitoring access to tasks summary [@stumpylog](https://github.com/stumpylog) ([#12624](https://github.com/paperless-ngx/paperless-ngx/pull/12624))
- Chore(deps): Bump lxml from 6.0.2 to 6.1.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12619](https://github.com/paperless-ngx/paperless-ngx/pull/12619))
- Performance: Increases workflow related M2M prefetching [@stumpylog](https://github.com/stumpylog) ([#12618](https://github.com/paperless-ngx/paperless-ngx/pull/12618))
- Tweak: remove stale index warning [@shamoon](https://github.com/shamoon) ([#12616](https://github.com/paperless-ngx/paperless-ngx/pull/12616))
- Fixhancement: Exact custom field monetary exact searching [@stumpylog](https://github.com/stumpylog) ([#12592](https://github.com/paperless-ngx/paperless-ngx/pull/12592))
- Performance: Resolves another N+1 query related to custom fields [@stumpylog](https://github.com/stumpylog) ([#12617](https://github.com/paperless-ngx/paperless-ngx/pull/12617))
- Enhancement: add highlighting to title + content searches [@shamoon](https://github.com/shamoon) ([#12593](https://github.com/paperless-ngx/paperless-ngx/pull/12593))
- Feature: Redesign the task system [@stumpylog](https://github.com/stumpylog) ([#12584](https://github.com/paperless-ngx/paperless-ngx/pull/12584))
- Fix: relative added date check visibility [@shamoon](https://github.com/shamoon) ([#12600](https://github.com/paperless-ngx/paperless-ngx/pull/12600))
- Fix: prevent intermediate change event when CustomFieldQueryAtom operator changes type [@ggouzi](https://github.com/ggouzi) ([#12597](https://github.com/paperless-ngx/paperless-ngx/pull/12597))
- Chore(deps): Bump hono from 4.12.12 to 4.12.14 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12585](https://github.com/paperless-ngx/paperless-ngx/pull/12585))
- Feature: Tune SQLite default settings for increased speed [@stumpylog](https://github.com/stumpylog) ([#12580](https://github.com/paperless-ngx/paperless-ngx/pull/12580))
- Feature: Document fuzzy match improvements [@stumpylog](https://github.com/stumpylog) ([#12579](https://github.com/paperless-ngx/paperless-ngx/pull/12579))
- Fix: reject invalid requests to API notes endpoint [@ggouzi](https://github.com/ggouzi) ([#12582](https://github.com/paperless-ngx/paperless-ngx/pull/12582))
- [BREAKING] Remove the positional arguments from the pre/post consume scripts [@stumpylog](https://github.com/stumpylog) ([#12573](https://github.com/paperless-ngx/paperless-ngx/pull/12573))
- Chore(deps): Bump follow-redirects from 1.15.11 to 1.16.0 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12575](https://github.com/paperless-ngx/paperless-ngx/pull/12575))
- Chore(deps): Bump pillow from 12.1.1 to 12.2.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12574](https://github.com/paperless-ngx/paperless-ngx/pull/12574))
- Chore(deps-dev): Bump pytest from 9.0.2 to 9.0.3 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12568](https://github.com/paperless-ngx/paperless-ngx/pull/12568))
- Enhancement: validate and sanitize uploaded logos [@shamoon](https://github.com/shamoon) ([#12551](https://github.com/paperless-ngx/paperless-ngx/pull/12551))
- Chore(deps): Bump the utilities-minor group across 1 directory with 19 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12540](https://github.com/paperless-ngx/paperless-ngx/pull/12540))
- Chore(deps): Bump cryptography from 46.0.6 to 46.0.7 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12546](https://github.com/paperless-ngx/paperless-ngx/pull/12546))
- Chore(deps-dev): Bump types-python-dateutil from 2.9.0.20260305 to 2.9.0.20260323 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12542](https://github.com/paperless-ngx/paperless-ngx/pull/12542))
- Chore(deps-dev): Bump types-pytz from 2025.2.0.20251108 to 2026.1.1.20260304 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12541](https://github.com/paperless-ngx/paperless-ngx/pull/12541))
- Chore(deps): Bump django-guardian from 3.3.0 to 3.3.1 in the utilities-patch group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12539](https://github.com/paperless-ngx/paperless-ngx/pull/12539))
- Chore(deps-dev): Bump zensical from 0.0.29 to 0.0.31 in the development group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12532](https://github.com/paperless-ngx/paperless-ngx/pull/12532))
- Chore(deps): Bump pdfjs-dist from 5.4.624 to 5.6.205 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12536](https://github.com/paperless-ngx/paperless-ngx/pull/12536))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12535](https://github.com/paperless-ngx/paperless-ngx/pull/12535))
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 5 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12533](https://github.com/paperless-ngx/paperless-ngx/pull/12533))
- Chore(deps-dev): Bump jest-preset-angular from 16.1.1 to 16.1.2 in /src-ui in the frontend-jest-dependencies group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12534](https://github.com/paperless-ngx/paperless-ngx/pull/12534))
- Chore(deps-dev): Bump @playwright/test from 1.58.2 to 1.59.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12537](https://github.com/paperless-ngx/paperless-ngx/pull/12537))
- Enhancement: add view\_global\_statistics and view\_system\_status permissions [@shamoon](https://github.com/shamoon) ([#12530](https://github.com/paperless-ngx/paperless-ngx/pull/12530))
- Chore(deps): Bump the npm\_and\_yarn group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12531](https://github.com/paperless-ngx/paperless-ngx/pull/12531))
- [BREAKING] Decouple OCR control from archive file control [@stumpylog](https://github.com/stumpylog) ([#12448](https://github.com/paperless-ngx/paperless-ngx/pull/12448))
- Fixhancement: include sharelinks + bundles in export/import [@shamoon](https://github.com/shamoon) ([#12479](https://github.com/paperless-ngx/paperless-ngx/pull/12479))
- Chore: Better typed status manager messages [@stumpylog](https://github.com/stumpylog) ([#12509](https://github.com/paperless-ngx/paperless-ngx/pull/12509))
- Enhancement: unify text search to use tantivy [@shamoon](https://github.com/shamoon) ([#12485](https://github.com/paperless-ngx/paperless-ngx/pull/12485))
- Feature: Consumer logging correlation [@stumpylog](https://github.com/stumpylog) ([#12510](https://github.com/paperless-ngx/paperless-ngx/pull/12510))
- Chore(deps): Bump djangorestframework from 3.16.1 to 3.17.1 in the django-ecosystem group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12488](https://github.com/paperless-ngx/paperless-ngx/pull/12488))
- Chore(deps): Bump the document-processing group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12489](https://github.com/paperless-ngx/paperless-ngx/pull/12489))
- Chore(deps): Bump the utilities-patch group across 1 directory with 5 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12499](https://github.com/paperless-ngx/paperless-ngx/pull/12499))
- Security: Sign the Celery pickled tasks [@stumpylog](https://github.com/stumpylog) ([#12504](https://github.com/paperless-ngx/paperless-ngx/pull/12504))
- Security enhancement: allow opt-in blocking internal mail hosts [@shamoon](https://github.com/shamoon) ([#12502](https://github.com/paperless-ngx/paperless-ngx/pull/12502))
- Feature: Replace Whoosh with tantivy search backend [@stumpylog](https://github.com/stumpylog) ([#12471](https://github.com/paperless-ngx/paperless-ngx/pull/12471))
- Chore(deps): Bump aiohttp from 3.13.3 to 3.13.4 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12486](https://github.com/paperless-ngx/paperless-ngx/pull/12486))
- Chore(deps): Bump lodash from 4.17.23 to 4.18.1 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12498](https://github.com/paperless-ngx/paperless-ngx/pull/12498))
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12491](https://github.com/paperless-ngx/paperless-ngx/pull/12491))
- Chore(deps-dev): Bump @types/node from 25.4.0 to 25.5.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12494](https://github.com/paperless-ngx/paperless-ngx/pull/12494))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12493](https://github.com/paperless-ngx/paperless-ngx/pull/12493))
- Performance: support bulk edit without id lists [@shamoon](https://github.com/shamoon) ([#12355](https://github.com/paperless-ngx/paperless-ngx/pull/12355))
- Performance: deprecate and remove usage of `all` in API results [@shamoon](https://github.com/shamoon) ([#12309](https://github.com/paperless-ngx/paperless-ngx/pull/12309))
- Performance: support passing selection data with filtered document requests [@shamoon](https://github.com/shamoon) ([#12300](https://github.com/paperless-ngx/paperless-ngx/pull/12300))
- Chore(deps): Bump cryptography from 46.0.5 to 46.0.6 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12458](https://github.com/paperless-ngx/paperless-ngx/pull/12458))
- Tweakhancement: show file extension in StoragePath test [@shamoon](https://github.com/shamoon) ([#12452](https://github.com/paperless-ngx/paperless-ngx/pull/12452))
- Feature: Transition all checksums to use SHA256 [@stumpylog](https://github.com/stumpylog) ([#12432](https://github.com/paperless-ngx/paperless-ngx/pull/12432))
- Chore(deps): Bump requests from 2.32.5 to 2.33.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12441](https://github.com/paperless-ngx/paperless-ngx/pull/12441))
- Enhancement: auto-hide the search bar on mobile [@shamoon](https://github.com/shamoon) ([#12404](https://github.com/paperless-ngx/paperless-ngx/pull/12404))
- Chore: logger, response and template sanitization cleanup [@shamoon](https://github.com/shamoon) ([#12439](https://github.com/paperless-ngx/paperless-ngx/pull/12439))
- Fix: Respect workflow change\_groups during consumption [@stumpylog](https://github.com/stumpylog) ([#12431](https://github.com/paperless-ngx/paperless-ngx/pull/12431))
- Security: prevent prototype pollution in frontend settings and list view [@shamoon](https://github.com/shamoon) ([#12438](https://github.com/paperless-ngx/paperless-ngx/pull/12438))
- Chore(deps): Bump cbor2 from 5.8.0 to 5.9.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12424](https://github.com/paperless-ngx/paperless-ngx/pull/12424))
- Chore(deps): Bump tinytag from 2.2.0 to 2.2.1 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#12396](https://github.com/paperless-ngx/paperless-ngx/pull/12396))
- Chore(deps): Bump ujson from 5.11.0 to 5.12.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12387](https://github.com/paperless-ngx/paperless-ngx/pull/12387))
- Chore(deps): Bump pyasn1 from 0.6.2 to 0.6.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12370](https://github.com/paperless-ngx/paperless-ngx/pull/12370))
- Chore(deps): Bump the utilities-patch group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12382](https://github.com/paperless-ngx/paperless-ngx/pull/12382))
- Chore(deps): Bump openai from 2.24.0 to 2.26.0 in the utilities-minor group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12379](https://github.com/paperless-ngx/paperless-ngx/pull/12379))
- Chore(deps-dev): Bump types-python-dateutil from 2.9.0.20260124 to 2.9.0.20260305 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12380](https://github.com/paperless-ngx/paperless-ngx/pull/12380))
- Chore(deps): Update django-allauth[mfa,socialaccount] requirement from ~=65.14.0 to ~=65.15.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12381](https://github.com/paperless-ngx/paperless-ngx/pull/12381))
- Chore(deps-dev): Bump the frontend-jest-dependencies group in /src-ui with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12374](https://github.com/paperless-ngx/paperless-ngx/pull/12374))
- Chore(deps): Bump @ng-select/ng-select from 21.4.1 to 21.5.2 in /src-ui in the frontend-angular-dependencies group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12373](https://github.com/paperless-ngx/paperless-ngx/pull/12373))
- Chore(deps-dev): Bump @types/node from 25.3.3 to 25.4.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12376](https://github.com/paperless-ngx/paperless-ngx/pull/12376))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12375](https://github.com/paperless-ngx/paperless-ngx/pull/12375))
- Chore: Reduce system status test times [@stumpylog](https://github.com/stumpylog) ([#12354](https://github.com/paperless-ngx/paperless-ngx/pull/12354))
- Chore(deps): Bump pyopenssl from 25.3.0 to 26.0.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12363](https://github.com/paperless-ngx/paperless-ngx/pull/12363))
- Chore(deps): Bump pyjwt from 2.10.1 to 2.12.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12335](https://github.com/paperless-ngx/paperless-ngx/pull/12335))
- Change: sort custom fields alphabetically by default [@shamoon](https://github.com/shamoon) ([#12358](https://github.com/paperless-ngx/paperless-ngx/pull/12358))
- Feature: Add progress information to the classifier training for a better ux [@stumpylog](https://github.com/stumpylog) ([#12331](https://github.com/paperless-ngx/paperless-ngx/pull/12331))
- Chore: bump Angular dependencies to 21.2.x [@shamoon](https://github.com/shamoon) ([#12338](https://github.com/paperless-ngx/paperless-ngx/pull/12338))
- Bump ocrmypdf from 16.13.0 to 17.3.0 in the document-processing group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12267](https://github.com/paperless-ngx/paperless-ngx/pull/12267))
- Feature: document parser plugin framework [@stumpylog](https://github.com/stumpylog) ([#12294](https://github.com/paperless-ngx/paperless-ngx/pull/12294))
- Chore(deps): Bump tornado from 6.5.4 to 6.5.5 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12327](https://github.com/paperless-ngx/paperless-ngx/pull/12327))
- Enhancement: Make the StatusConsumer truly async [@stumpylog](https://github.com/stumpylog) ([#12298](https://github.com/paperless-ngx/paperless-ngx/pull/12298))
- Chore(deps): Bump the utilities-minor group across 1 directory with 5 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12324](https://github.com/paperless-ngx/paperless-ngx/pull/12324))
- Chore: mark document detail email action as deprecated [@shamoon](https://github.com/shamoon) ([#12308](https://github.com/paperless-ngx/paperless-ngx/pull/12308))
- Chore: separate actions from bulk edit endpoint [@shamoon](https://github.com/shamoon) ([#12286](https://github.com/paperless-ngx/paperless-ngx/pull/12286))
- [BREAKING] Chore: drop support for api versions \< 9 [@shamoon](https://github.com/shamoon) ([#12284](https://github.com/paperless-ngx/paperless-ngx/pull/12284))
- Chore(deps): Bump the utilities-patch group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12291](https://github.com/paperless-ngx/paperless-ngx/pull/12291))
- Performance: Stream JSON during import for memory improvements [@stumpylog](https://github.com/stumpylog) ([#12276](https://github.com/paperless-ngx/paperless-ngx/pull/12276))
- Feature: Migrate import/export to rich progress [@stumpylog](https://github.com/stumpylog) ([#12260](https://github.com/paperless-ngx/paperless-ngx/pull/12260))
- Enhancement: Show more document details in merge dialog [@svenstaro](https://github.com/svenstaro) ([#12271](https://github.com/paperless-ngx/paperless-ngx/pull/12271))
- Performance: Further export memory improvements [@stumpylog](https://github.com/stumpylog) ([#12273](https://github.com/paperless-ngx/paperless-ngx/pull/12273))
- Chore: pytest style paperless tests [@stumpylog](https://github.com/stumpylog) ([#12254](https://github.com/paperless-ngx/paperless-ngx/pull/12254))
- Chore: update ESLint to v10 [@shamoon](https://github.com/shamoon) ([#12256](https://github.com/paperless-ngx/paperless-ngx/pull/12256))
- Bump django-allauth from 65.14.0 to 65.14.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12253](https://github.com/paperless-ngx/paperless-ngx/pull/12253))
- Bump django from 5.2.11 to 5.2.12 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12249](https://github.com/paperless-ngx/paperless-ngx/pull/12249))
- [BREAKING] Chore: Drop support for Python 3.10 [@stumpylog](https://github.com/stumpylog) ([#12234](https://github.com/paperless-ngx/paperless-ngx/pull/12234))
- Performance: Improve exporter memory efficiency [@stumpylog](https://github.com/stumpylog) ([#12236](https://github.com/paperless-ngx/paperless-ngx/pull/12236))
- Enhancement: saved view sharing [@shamoon](https://github.com/shamoon) ([#12142](https://github.com/paperless-ngx/paperless-ngx/pull/12142))
- Fix: use maxsplit=1 in Redis URL parsing to handle URLs with multiple colons [@cryptomilk](https://github.com/cryptomilk) ([#12239](https://github.com/paperless-ngx/paperless-ngx/pull/12239))
- Enhancement: “live” document updates [@shamoon](https://github.com/shamoon) ([#12141](https://github.com/paperless-ngx/paperless-ngx/pull/12141))
- Enhancement: Improve the retagger output using rich [@stumpylog](https://github.com/stumpylog) ([#12194](https://github.com/paperless-ngx/paperless-ngx/pull/12194))
- Enhancement: Transition sanity check to rich and improve output [@stumpylog](https://github.com/stumpylog) ([#12182](https://github.com/paperless-ngx/paperless-ngx/pull/12182))
- Enhancement: Switch all indexing to use rich [@stumpylog](https://github.com/stumpylog) ([#12193](https://github.com/paperless-ngx/paperless-ngx/pull/12193))
- Chore(deps): Bump whitenoise from 6.11.0 to 6.12.0 in the django-ecosystem group @[dependabot[bot]](https://github.com/apps/dependabot) ([#12192](https://github.com/paperless-ngx/paperless-ngx/pull/12192))
- Fixhancement: show sequential + id version labels, fix padding [@shamoon](https://github.com/shamoon) ([#12196](https://github.com/paperless-ngx/paperless-ngx/pull/12196))
- [BREAKING] Chore: Refactor advanced database settings to allow more user configuration [@stumpylog](https://github.com/stumpylog) ([#12165](https://github.com/paperless-ngx/paperless-ngx/pull/12165))
- Chore(deps): Bump the utilities-patch group across 1 directory with 11 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12179](https://github.com/paperless-ngx/paperless-ngx/pull/12179))
- Feature: Switch progress bar library to rich [@stumpylog](https://github.com/stumpylog) ([#12169](https://github.com/paperless-ngx/paperless-ngx/pull/12169))
- Chore(deps): Bump nltk from 3.9.2 to 3.9.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12177](https://github.com/paperless-ngx/paperless-ngx/pull/12177))
- Enhancement: Formatted filename for single document downloads [@JanKleine](https://github.com/JanKleine) ([#12095](https://github.com/paperless-ngx/paperless-ngx/pull/12095))
- Chore(deps): Bump the utilities-minor group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12174](https://github.com/paperless-ngx/paperless-ngx/pull/12174))
- Feature: document file versions [@shamoon](https://github.com/shamoon) ([#12061](https://github.com/paperless-ngx/paperless-ngx/pull/12061))
- [BREAKING] Remove API v1 compatibility [@stumpylog](https://github.com/stumpylog) ([#12166](https://github.com/paperless-ngx/paperless-ngx/pull/12166))
- Enhancement: prevent duplicate mail processing across rules [@shamoon](https://github.com/shamoon) ([#12159](https://github.com/paperless-ngx/paperless-ngx/pull/12159))
- Feature: move to trash action for workflows [@JanKleine](https://github.com/JanKleine) ([#11176](https://github.com/paperless-ngx/paperless-ngx/pull/11176))
- Tweakhancement: reset to page 1 on reset filters [@shamoon](https://github.com/shamoon) ([#12143](https://github.com/paperless-ngx/paperless-ngx/pull/12143))
- Fix: correct user dropdown button icon styling [@shamoon](https://github.com/shamoon) ([#12092](https://github.com/paperless-ngx/paperless-ngx/pull/12092))
- Enhancement: consolidate management lists into document attributes section [@shamoon](https://github.com/shamoon) ([#12045](https://github.com/paperless-ngx/paperless-ngx/pull/12045))
- Enhancement: option to stop processing further mail rules [@shamoon](https://github.com/shamoon) ([#12053](https://github.com/paperless-ngx/paperless-ngx/pull/12053))
- [BREAKING] Remove pybzar as a barcode reader [@stumpylog](https://github.com/stumpylog) ([#12065](https://github.com/paperless-ngx/paperless-ngx/pull/12065))
- Chore(deps): Bump pillow from 12.1.0 to 12.1.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12064](https://github.com/paperless-ngx/paperless-ngx/pull/12064))
- Chore(deps): Bump cryptography from 46.0.3 to 46.0.5 @[dependabot[bot]](https://github.com/apps/dependabot) ([#12060](https://github.com/paperless-ngx/paperless-ngx/pull/12060))
- Chore: Optimizes the integer fields for choice types [@stumpylog](https://github.com/stumpylog) ([#12057](https://github.com/paperless-ngx/paperless-ngx/pull/12057))
- Chore(deps): Bump the utilities-patch group across 1 directory with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12051](https://github.com/paperless-ngx/paperless-ngx/pull/12051))
- Tweak: improve 2-digit year parsing [@shamoon](https://github.com/shamoon) ([#12044](https://github.com/paperless-ngx/paperless-ngx/pull/12044))
- Enhancement: pngx pdf viewer [@shamoon](https://github.com/shamoon) ([#12043](https://github.com/paperless-ngx/paperless-ngx/pull/12043))
- Fix: Handle an OOM kill of celery workers in the task handler [@stumpylog](https://github.com/stumpylog) ([#12040](https://github.com/paperless-ngx/paperless-ngx/pull/12040))
- Chore(deps-dev): Bump @playwright/test from 1.58.1 to 1.58.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12032](https://github.com/paperless-ngx/paperless-ngx/pull/12032))
- Chore(deps): Bump the utilities-minor group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#12020](https://github.com/paperless-ngx/paperless-ngx/pull/12020))
- Fixhancement: improve ASN handling with PDF operations [@shamoon](https://github.com/shamoon) ([#11689](https://github.com/paperless-ngx/paperless-ngx/pull/11689))
- Upgrade: Bump @types/node from 25.2.0 to 25.2.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12008](https://github.com/paperless-ngx/paperless-ngx/pull/12008))
- Upgrade: Bump webpack from 5.103.0 to 5.105.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#12007](https://github.com/paperless-ngx/paperless-ngx/pull/12007))
- Chore: bump Angular to 21.1.3, ngx-ui-tour-ng-bootstrap to v18 [@shamoon](https://github.com/shamoon) ([#12015](https://github.com/paperless-ngx/paperless-ngx/pull/12015))
- Chore: Switches to use prek in place of pre-commit [@stumpylog](https://github.com/stumpylog) ([#12002](https://github.com/paperless-ngx/paperless-ngx/pull/12002))
- Chore(deps): Bump the utilities-patch group with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11981](https://github.com/paperless-ngx/paperless-ngx/pull/11981))
- Chore(deps): Bump the utilities-minor group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11993](https://github.com/paperless-ngx/paperless-ngx/pull/11993))
- Chore(deps): Bump drf-spectacular-sidecar from 2025.10.1 to 2026.1.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11985](https://github.com/paperless-ngx/paperless-ngx/pull/11985))
- Chore(deps): Update django-allauth[mfa,socialaccount] requirement from ~=65.13.1 to ~=65.14.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11984](https://github.com/paperless-ngx/paperless-ngx/pull/11984))
- Chore(deps): Update granian[uvloop] requirement from ~=2.6.0 to ~=2.7.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11983](https://github.com/paperless-ngx/paperless-ngx/pull/11983))
- Chore(deps): Bump django from 5.2.10 to 5.2.11 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11988](https://github.com/paperless-ngx/paperless-ngx/pull/11988))
- Feature: Enable users to customize date parsing via plugins [@stumpylog](https://github.com/stumpylog) ([#11931](https://github.com/paperless-ngx/paperless-ngx/pull/11931))
- Chore: Fix further test flakiness [@stumpylog](https://github.com/stumpylog) ([#11987](https://github.com/paperless-ngx/paperless-ngx/pull/11987))
- Feature: password removal workflow action [@shamoon](https://github.com/shamoon) ([#11665](https://github.com/paperless-ngx/paperless-ngx/pull/11665))
- Fix: re-run ASN check after barcode detection [@shamoon](https://github.com/shamoon) ([#11681](https://github.com/paperless-ngx/paperless-ngx/pull/11681))
- Fix: prevent infinite loading crash in mail component [@shamoon](https://github.com/shamoon) ([#11978](https://github.com/paperless-ngx/paperless-ngx/pull/11978))
- Enhancement: per-type object page sizing [@shamoon](https://github.com/shamoon) ([#11977](https://github.com/paperless-ngx/paperless-ngx/pull/11977))
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 22 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11968](https://github.com/paperless-ngx/paperless-ngx/pull/11968))
- Chore(deps): Bump zone.js from 0.15.1 to 0.16.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#11970](https://github.com/paperless-ngx/paperless-ngx/pull/11970))
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11969](https://github.com/paperless-ngx/paperless-ngx/pull/11969))
- Chore(deps-dev): Bump @types/node from 24.10.1 to 25.2.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#11972](https://github.com/paperless-ngx/paperless-ngx/pull/11972))
- Chore(deps-dev): Bump @playwright/test from 1.57.0 to 1.58.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#11971](https://github.com/paperless-ngx/paperless-ngx/pull/11971))
- Enhancement: improve filter drop-down performance with virtual scrolling [@shamoon](https://github.com/shamoon) ([#11973](https://github.com/paperless-ngx/paperless-ngx/pull/11973))
- Tweakhancement: tweak bulk delete text [@shamoon](https://github.com/shamoon) ([#11967](https://github.com/paperless-ngx/paperless-ngx/pull/11967))
- Feature: support split documents based on tag barcodes @schober-ch ([#11645](https://github.com/paperless-ngx/paperless-ngx/pull/11645))
- Chore: Bulk backend updates [@stumpylog](https://github.com/stumpylog) ([#11543](https://github.com/paperless-ngx/paperless-ngx/pull/11543))
- Tweak: increase minimum screen width before inserting padding [@JanKleine](https://github.com/JanKleine) ([#11926](https://github.com/paperless-ngx/paperless-ngx/pull/11926))
- Enhancement: Add setting for default PDF Editor mode [@JanKleine](https://github.com/JanKleine) ([#11927](https://github.com/paperless-ngx/paperless-ngx/pull/11927))
- Fixhancement: auto-queue llm index if needed [@shamoon](https://github.com/shamoon) ([#11891](https://github.com/paperless-ngx/paperless-ngx/pull/11891))
- Feature: sharelink bundles [@shamoon](https://github.com/shamoon) ([#11682](https://github.com/paperless-ngx/paperless-ngx/pull/11682))
- Enhancement: user control of doc details fields [@shamoon](https://github.com/shamoon) ([#11906](https://github.com/paperless-ngx/paperless-ngx/pull/11906))
- Enhancement: improve relative dates in date filter [@JanKleine](https://github.com/JanKleine) ([#11899](https://github.com/paperless-ngx/paperless-ngx/pull/11899))
- Performance: faster statistics panel on dashboard [@Merinorus](https://github.com/Merinorus) ([#11760](https://github.com/paperless-ngx/paperless-ngx/pull/11760))
- Enhancement: allow duplicates with warnings, UI for discovery [@shamoon](https://github.com/shamoon) ([#11815](https://github.com/paperless-ngx/paperless-ngx/pull/11815))
- Enhancement: configurable SSO groups claim [@Gabgobie](https://github.com/Gabgobie) ([#11841](https://github.com/paperless-ngx/paperless-ngx/pull/11841))
- Enhancement: support select all for management lists [@shamoon](https://github.com/shamoon) ([#11889](https://github.com/paperless-ngx/paperless-ngx/pull/11889))
- Tweakhancement: display document id, with copy [@shamoon](https://github.com/shamoon) ([#11896](https://github.com/paperless-ngx/paperless-ngx/pull/11896))
- Enhancement: Add support for app oidc [@paulgessinger](https://github.com/paulgessinger) ([#11756](https://github.com/paperless-ngx/paperless-ngx/pull/11756))
- Enhancement: Add 'any of' workflow trigger filters [@shamoon](https://github.com/shamoon) ([#11683](https://github.com/paperless-ngx/paperless-ngx/pull/11683))
- [BREAKING] Remove support for document and thumbnail encryption [@stumpylog](https://github.com/stumpylog) ([#11850](https://github.com/paperless-ngx/paperless-ngx/pull/11850))
- Fix: ensure css color-scheme for dark mode [@shamoon](https://github.com/shamoon) ([#11855](https://github.com/paperless-ngx/paperless-ngx/pull/11855))
- Enhancement: support doc\_id placeholder in workflow templates [@shamoon](https://github.com/shamoon) ([#11847](https://github.com/paperless-ngx/paperless-ngx/pull/11847))
- [BREAKING] Feature: Simplify and improve the consumer [@stumpylog](https://github.com/stumpylog) ([#11753](https://github.com/paperless-ngx/paperless-ngx/pull/11753))
- Fix: fix tag list horizontal scroll, again [@shamoon](https://github.com/shamoon) ([#11839](https://github.com/paperless-ngx/paperless-ngx/pull/11839))
- Fix: ensure horizontal scroll for long tag names in list, wrap tags without parent [@shamoon](https://github.com/shamoon) ([#11811](https://github.com/paperless-ngx/paperless-ngx/pull/11811))
- Chore(deps): Bump the utilities-patch group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11793](https://github.com/paperless-ngx/paperless-ngx/pull/11793))
- Chore(deps): Bump the utilities-minor group across 1 directory with 10 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#11799](https://github.com/paperless-ngx/paperless-ngx/pull/11799))
- Chore(deps): Bump pyasn1 from 0.6.1 to 0.6.2 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11801](https://github.com/paperless-ngx/paperless-ngx/pull/11801))
- Chore(deps): Bump torch from 2.7.1 to 2.8.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11800](https://github.com/paperless-ngx/paperless-ngx/pull/11800))
- Chore(deps): Bump brotli from 1.1.0 to 1.2.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11796](https://github.com/paperless-ngx/paperless-ngx/pull/11796))
- Chore(deps): Bump transformers from 4.51.3 to 4.53.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11797](https://github.com/paperless-ngx/paperless-ngx/pull/11797))
- Chore(deps): Bump django from 5.2.7 to 5.2.9 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11794](https://github.com/paperless-ngx/paperless-ngx/pull/11794))
- Chore(deps): Bump the llama-index group @[dependabot[bot]](https://github.com/apps/dependabot) ([#11798](https://github.com/paperless-ngx/paperless-ngx/pull/11798))
- Chore(deps): Bump marshmallow from 3.26.1 to 3.26.2 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11790](https://github.com/paperless-ngx/paperless-ngx/pull/11790))
- Chore(deps): Bump uv from 0.9.3 to 0.9.6 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11795](https://github.com/paperless-ngx/paperless-ngx/pull/11795))
- Chore(deps): Bump aiohttp from 3.11.18 to 3.13.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11789](https://github.com/paperless-ngx/paperless-ngx/pull/11789))
- Chore(deps): Bump urllib3 from 2.5.0 to 2.6.3 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11792](https://github.com/paperless-ngx/paperless-ngx/pull/11792))
- Chore(deps): Bump virtualenv from 20.34.0 to 20.36.1 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11774](https://github.com/paperless-ngx/paperless-ngx/pull/11774))
- Fix: use explicit order field for workflow actions [@shamoon](https://github.com/shamoon) ([#11781](https://github.com/paperless-ngx/paperless-ngx/pull/11781))
- Chore(deps): Bump azure-core from 1.33.0 to 1.38.0 @[dependabot[bot]](https://github.com/apps/dependabot) ([#11776](https://github.com/paperless-ngx/paperless-ngx/pull/11776))
- Feature: Paperless AI [@shamoon](https://github.com/shamoon) ([#10319](https://github.com/paperless-ngx/paperless-ngx/pull/10319))
- Performance: improve treenode inefficiencies [@shamoon](https://github.com/shamoon) ([#11606](https://github.com/paperless-ngx/paperless-ngx/pull/11606))
- Chore: upgrade to node v24 [@shamoon](https://github.com/shamoon) ([#11747](https://github.com/paperless-ngx/paperless-ngx/pull/11747))
- Chore: upgrade to Angular v21 [@shamoon](https://github.com/shamoon) ([#11746](https://github.com/paperless-ngx/paperless-ngx/pull/11746))
- Feature: Remote OCR (Azure AI) [@shamoon](https://github.com/shamoon) ([#10320](https://github.com/paperless-ngx/paperless-ngx/pull/10320))
- Feature: password removal action [@shamoon](https://github.com/shamoon) ([#11656](https://github.com/paperless-ngx/paperless-ngx/pull/11656))
- Fix: fix recurring workflow to respect latest run time [@shamoon](https://github.com/shamoon) ([#11735](https://github.com/paperless-ngx/paperless-ngx/pull/11735))
- Fixhancement: add error handling and retry when opening index [@shamoon](https://github.com/shamoon) ([#11731](https://github.com/paperless-ngx/paperless-ngx/pull/11731))
- Tweakhancement: use anchor element for management list quick filter buttons [@shamoon](https://github.com/shamoon) ([#11692](https://github.com/paperless-ngx/paperless-ngx/pull/11692))
- Fix: validate cf integer values within PostgreSQL range [@shamoon](https://github.com/shamoon) ([#11666](https://github.com/paperless-ngx/paperless-ngx/pull/11666))
- Fix: support ordering by storage path name [@shamoon](https://github.com/shamoon) ([#11661](https://github.com/paperless-ngx/paperless-ngx/pull/11661))
- Fix: propagate metadata override created value [@shamoon](https://github.com/shamoon) ([#11659](https://github.com/paperless-ngx/paperless-ngx/pull/11659))
- Fix: prevent ASN collisions for merge operations [@shamoon](https://github.com/shamoon) ([#11634](https://github.com/paperless-ngx/paperless-ngx/pull/11634))
</details>
## paperless-ngx 2.20.15
### Security
@@ -7827,7 +7045,7 @@ primarily.
### paperless-ng 0.9.0
- **Deprecated:** GnuPG. [See this note on the state of GnuPG in paperless-ng.](#paperless-ng-093)
- **Deprecated:** GnuPG. [See this note on the state of GnuPG in paperless-ng.](administration.md#encryption)
This features will most likely be removed in future versions.
- **Added:** New frontend. Features:
- Single page application: It's much more responsive than the
+9 -11
View File
@@ -1159,21 +1159,19 @@ still perform some basic text pre-processing before matching.
#### [`PAPERLESS_DATE_PARSER_LANGUAGES=<lang>`](#PAPERLESS_DATE_PARSER_LANGUAGES) {#PAPERLESS_DATE_PARSER_LANGUAGES}
: Specifies which language Paperless should use when parsing dates from documents.
Specifies which language Paperless should use when parsing dates from documents.
: This should be a language code supported by the dateparser library,
for example: "en", or a combination such as "en+de".
Locales are also supported (e.g., "en-AU").
Multiple languages can be combined using "+", for example: "en+de" or "en-AU+de".
For valid values, refer to the list of supported languages and locales in the [dateparser documentation](https://dateparser.readthedocs.io/en/latest/supported_locales.html).
: Set this to match the languages in which most of your documents are written.
This should be a language code supported by the dateparser library,
for example: "en", or a combination such as "en+de".
Locales are also supported (e.g., "en-AU").
Multiple languages can be combined using "+", for example: "en+de" or "en-AU+de".
For valid values, refer to the list of supported languages and locales in the [dateparser documentation](https://dateparser.readthedocs.io/en/latest/supported_locales.html).
Set this to match the languages in which most of your documents are written.
If not set, Paperless will attempt to infer the language(s) from the OCR configuration (`PAPERLESS_OCR_LANGUAGE`).
!!! note
This format differs from the `PAPERLESS_OCR_LANGUAGE` setting, which uses ISO 639-2 codes (3 letters, e.g., "eng+deu" for Tesseract OCR).
!!! note
This format differs from the `PAPERLESS_OCR_LANGUAGE` setting, which uses ISO 639-2 codes (3 letters, e.g., "eng+deu" for Tesseract OCR).
#### [`PAPERLESS_EMAIL_TASK_CRON=<cron expression>`](#PAPERLESS_EMAIL_TASK_CRON) {#PAPERLESS_EMAIL_TASK_CRON}
+1 -1
View File
@@ -192,7 +192,7 @@ searchable PDF and stores it as the archive copy. `ARCHIVE_FILE_GENERATION=never
has no effect for documents handled by the remote parser — the archive is produced
unconditionally by the remote engine.
## Search Index (Whoosh -> Tantivy)
# Search Index (Whoosh -> Tantivy)
The full-text search backend has been replaced with [Tantivy](https://github.com/quickwit-oss/tantivy).
The index format is incompatible with Whoosh, so **the search index is automatically rebuilt from
+2 -2
View File
@@ -157,8 +157,8 @@ Do not combine this with `USERMAP_UID` or `USERMAP_GID`, which are intended for
Some file systems, such as NFS network shares, don't support file system
notifications with `inotify`. When the consumption directory is on such a
file system, Paperless-ngx will not pick up new files with the default
configuration. Set [`PAPERLESS_CONSUMER_POLLING_INTERVAL`](configuration.md#PAPERLESS_CONSUMER_POLLING_INTERVAL)
to a positive number to enable polling and disable native filesystem notifications.
configuration. Use [`PAPERLESS_CONSUMER_POLLING`](configuration.md#PAPERLESS_CONSUMER_POLLING)
to enable polling and disable inotify. See [here](configuration.md#polling).
## Bare Metal Install {#bare_metal}
File diff suppressed because it is too large Load Diff
@@ -1,839 +0,0 @@
# Export Zip Compression Control Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add `--zip-compression {stored,deflated,bzip2,lzma,zstd}` and `--zip-compression-level N` flags to `document_exporter`, threaded into `ZipExportSink`, with import-side safety for codecs the running Python can't read.
**Architecture:** A new pure-data module `documents/export/compression.py` owns the method↔constant map, per-method level bounds, the runtime availability probe, and a compress-type readability check. `ZipExportSink` gains `compression`/`compresslevel` constructor params. The command validates flags up front (fail-fast `CommandError`) and constructs the sink; the importer pre-checks entry compress types before extracting.
**Tech Stack:** Python ≥3.11 (zstd only on 3.14+), `zipfile`, `compression.zstd` (PEP 784), pytest + pytest-mock + factory-boy. Backend tests run on the Linux VM (Python 3.11 — zstd positive tests are `skipif`-guarded); `ruff` runs locally.
**Spec:** `docs/superpowers/specs/2026-06-16-export-zip-compression-design.md`
**PREREQUISITE:** The base refactor `docs/superpowers/plans/2026-06-16-export-sink-architecture.md` MUST be merged first. This plan assumes `src/documents/export/sinks.py` exists with `ZipExportSink(target, zip_name, *, delete=False)` opening its `ZipFile` in `_open()`.
---
## Verified facts (CPython 3.14.3, via `uv run --python 3.14 --no-project`)
- Constants: `ZIP_STORED=0`, `ZIP_DEFLATED=8`, `ZIP_BZIP2=12`, `ZIP_LZMA=14`, `ZIP_ZSTANDARD=93` (zstd added 3.14; absent on < 3.14).
- `ZipFile(file, "w", compression=…, compresslevel=…)` applies both as the default for every `write`/`writestr` — no per-entry args needed (verified).
- Level bounds: `deflated` 09, `bzip2` 19, `lzma`/`stored` ignore level, `zstd` -131072…22 (`compression.zstd.CompressionParameter.compression_level.bounds() == (-131072, 22)`).
- An invalid level fails at the **first write** (`ValueError: Invalid initialization option` / `compresslevel must be between 1 and 9`), plus GC-time `AttributeError` noise on close — hence up-front validation.
- zstd is backed by `compression.zstd`; `zipfile` raises `RuntimeError` if it's unavailable.
## Conventions for every task
- **Run backend tests on the VM:** `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "<targets>"` (never locally).
- **Lint locally:** `ruff check <paths> && ruff format <paths>` (global ruff, not `uv run`).
- **Tests are pytest-style:** classes, `@pytest.mark.django_db` on the class only where DB is needed (the `compression.py` and sink tests need no DB), factory-boy, `mocker`, `parametrize`, full type annotations.
- The VM runs Python 3.11, so **zstd positive tests must be `@pytest.mark.skipif(...)`-guarded**; they will simply not run there. zstd _rejection_ tests (the < 3.14 path) DO run on the VM.
## File structure
- **Create** `src/documents/export/compression.py` — method map, CLI choices, level bounds, `compression_available()`, `level_error()`, `compress_type_readable()`, `unreadable_method_names()`. Pure, no Django.
- **Create** `src/documents/tests/export/test_compression.py` — unit tests for the above.
- **Modify** `src/documents/export/sinks.py``ZipExportSink.__init__` gains `compression`/`compresslevel`; `_open()` passes them to `ZipFile`.
- **Modify** `src/documents/tests/export/test_sinks.py` — assert the chosen `compress_type` is applied.
- **Modify** `src/documents/management/commands/document_exporter.py` — add the two CLI flags, up-front validation, and pass resolved values to `ZipExportSink`.
- **Modify** `src/documents/tests/test_management_exporter.py` — flag validation + default-unchanged tests.
- **Modify** `src/documents/management/commands/document_importer.py` — pre-extract compress-type check.
- **Modify** `src/documents/tests/test_management_importer.py` — unsupported-codec → `CommandError`.
- **Modify** `docs/administration.md` — document both flags + zstd portability caveat.
---
## Task 1: `documents/export/compression.py` (pure compression policy)
**Files:**
- Create: `src/documents/export/compression.py`
- Test: `src/documents/tests/export/test_compression.py`
- [ ] **Step 1: Write the failing tests**
Create `src/documents/tests/export/test_compression.py`:
```python
import sys
import zipfile
import pytest
from documents.export import compression
class TestCompressionMethods:
def test_choices_always_include_zstd(self) -> None:
# zstd is offered regardless of runtime; availability is checked separately
assert compression.COMPRESSION_CHOICES == (
"stored",
"deflated",
"bzip2",
"lzma",
"zstd",
)
@pytest.mark.parametrize(
("name", "constant"),
[
("stored", zipfile.ZIP_STORED),
("deflated", zipfile.ZIP_DEFLATED),
("bzip2", zipfile.ZIP_BZIP2),
("lzma", zipfile.ZIP_LZMA),
],
)
def test_method_maps_to_zipfile_constant(self, name: str, constant: int) -> None:
assert compression.COMPRESSION_METHODS[name] == constant
def test_stored_and_deflated_always_available(self) -> None:
assert compression.compression_available("stored")
assert compression.compression_available("deflated")
def test_zstd_availability_tracks_runtime(self) -> None:
expected: bool = sys.version_info >= (3, 14)
assert compression.compression_available("zstd") == expected
class TestLevelError:
@pytest.mark.parametrize(
("method", "level"),
[
("deflated", 0),
("deflated", 9),
("bzip2", 1),
("bzip2", 9),
("deflated", None),
("stored", None),
],
)
def test_valid_levels_return_none(self, method: str, level: int | None) -> None:
assert compression.level_error(method, level) is None
@pytest.mark.parametrize(
("method", "level"),
[
("deflated", 10),
("deflated", -1),
("bzip2", 0),
("bzip2", 10),
],
)
def test_out_of_range_levels_return_message(
self,
method: str,
level: int,
) -> None:
msg: str | None = compression.level_error(method, level)
assert msg is not None
assert "between" in msg
@pytest.mark.parametrize("method", ["stored", "lzma"])
def test_level_on_levelless_method_is_rejected(self, method: str) -> None:
msg: str | None = compression.level_error(method, 5)
assert msg is not None
assert "no effect" in msg
class TestCompressTypeReadable:
@pytest.mark.parametrize("ct", [zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED])
def test_stored_and_deflated_always_readable(self, ct: int) -> None:
assert compression.compress_type_readable(ct)
def test_zstd_compress_type_readability_tracks_runtime(self) -> None:
# 93 = ZIP_ZSTANDARD; 20 = legacy zstd method id (read-only)
expected: bool = sys.version_info >= (3, 14)
assert compression.compress_type_readable(93) == expected
assert compression.compress_type_readable(20) == expected
def test_unknown_compress_type_is_unreadable(self) -> None:
assert not compression.compress_type_readable(9999)
def test_unreadable_method_names_lists_methods(self) -> None:
# An unknown method id maps to no name and is reported generically.
names: set[str] = compression.unreadable_method_names({9999})
assert names == {"method 9999"}
```
- [ ] **Step 2: Run to verify it fails**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/export/test_compression.py -v"`
Expected: FAIL with `ModuleNotFoundError: No module named 'documents.export.compression'`.
- [ ] **Step 3: Implement `compression.py`**
Create `src/documents/export/compression.py`:
```python
from __future__ import annotations
import importlib
import zipfile
# ZIP_ZSTANDARD exists only on Python 3.14+ (PEP 784). None elsewhere.
ZSTD: int | None = getattr(zipfile, "ZIP_ZSTANDARD", None)
# CLI choices are fixed across runtimes so argparse never hides zstd; runtime
# availability is enforced separately in compression_available().
COMPRESSION_CHOICES: tuple[str, ...] = (
"stored",
"deflated",
"bzip2",
"lzma",
"zstd",
)
# Method name -> zipfile compression constant (zstd only when supported).
COMPRESSION_METHODS: dict[str, int] = {
"stored": zipfile.ZIP_STORED,
"deflated": zipfile.ZIP_DEFLATED,
"bzip2": zipfile.ZIP_BZIP2,
"lzma": zipfile.ZIP_LZMA,
}
if ZSTD is not None:
COMPRESSION_METHODS["zstd"] = ZSTD
# Inclusive (min, max) level bounds per method; None => level not applicable.
# Verified on CPython 3.14.3.
LEVEL_BOUNDS: dict[str, tuple[int, int] | None] = {
"stored": None,
"deflated": (0, 9),
"bzip2": (1, 9),
"lzma": None,
"zstd": (-131072, 22),
}
# zipfile compress_type id -> method name. 93 = current zstd id, 20 = legacy
# zstd id that zipfile can still read.
_COMPRESS_TYPE_TO_METHOD: dict[int, str] = {
zipfile.ZIP_STORED: "stored",
zipfile.ZIP_DEFLATED: "deflated",
zipfile.ZIP_BZIP2: "bzip2",
zipfile.ZIP_LZMA: "lzma",
93: "zstd",
20: "zstd",
}
def compression_available(method: str) -> bool:
"""Whether the running interpreter can actually use the given method."""
if method in ("stored", "deflated"):
# zlib is a hard CPython dependency; stored needs nothing.
return True
if method == "bzip2":
return _module_importable("bz2")
if method == "lzma":
return _module_importable("lzma")
if method == "zstd":
return ZSTD is not None and _module_importable("compression.zstd")
return False
def _module_importable(name: str) -> bool:
try:
importlib.import_module(name)
except ImportError:
return False
return True
def level_error(method: str, level: int | None) -> str | None:
"""Return a human message if (method, level) is invalid, else None."""
if level is None:
return None
bounds = LEVEL_BOUNDS[method]
if bounds is None:
return f"--zip-compression-level has no effect for '{method}'"
low, high = bounds
if not (low <= level <= high):
return (
f"--zip-compression-level for '{method}' must be between "
f"{low} and {high}"
)
return None
def compress_type_readable(compress_type: int) -> bool:
"""Whether this interpreter can decompress an entry of the given type."""
method = _COMPRESS_TYPE_TO_METHOD.get(compress_type)
if method is None:
return False
return compression_available(method)
def unreadable_method_names(compress_types: set[int]) -> set[str]:
"""Map a set of compress_type ids to human method names for error messages."""
names: set[str] = set()
for ct in compress_types:
names.add(_COMPRESS_TYPE_TO_METHOD.get(ct, f"method {ct}"))
return names
```
- [ ] **Step 4: Run to verify it passes**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/export/test_compression.py -v"`
Expected: PASS (on the 3.11 VM, `test_zstd_availability_tracks_runtime` and `test_zstd_compress_type_readability_tracks_runtime` assert `False`).
- [ ] **Step 5: Lint**
Run: `ruff check src/documents/export/compression.py src/documents/tests/export/test_compression.py && ruff format src/documents/export/compression.py src/documents/tests/export/test_compression.py`
Expected: no errors.
- [ ] **Step 6: Commit**
```bash
git add src/documents/export/compression.py src/documents/tests/export/test_compression.py
git commit -m "Feature: add export compression policy module"
```
---
## Task 2: `ZipExportSink` accepts compression method + level
**Files:**
- Modify: `src/documents/export/sinks.py`
- Test: `src/documents/tests/export/test_sinks.py`
- [ ] **Step 1: Write the failing test**
Append to `src/documents/tests/export/test_sinks.py` (the top-of-file block already imports `zipfile`, `Path`, `pytest`, `ZipExportSink`, `StreamingManifestWriter` from the base-refactor plan):
```python
class TestZipExportSinkCompression:
@pytest.fixture()
def source_file(self, tmp_path: Path) -> Path:
src: Path = tmp_path / "src" / "doc.pdf"
src.parent.mkdir(parents=True)
src.write_bytes(b"PDF-CONTENT" * 100)
return src
@pytest.mark.parametrize(
("method", "constant"),
[
("stored", zipfile.ZIP_STORED),
("deflated", zipfile.ZIP_DEFLATED),
("bzip2", zipfile.ZIP_BZIP2),
("lzma", zipfile.ZIP_LZMA),
],
)
def test_compression_method_is_applied_to_file_entries(
self,
tmp_path: Path,
source_file: Path,
method: str,
constant: int,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with ZipExportSink(
target,
"export",
delete=False,
compression=constant,
) as sink:
sink.add_file(source_file, "doc.pdf")
with zipfile.ZipFile(target / "export.zip") as zf:
info = zf.getinfo("doc.pdf")
assert info.compress_type == constant
def test_compressing_method_beats_stored(
self,
tmp_path: Path,
source_file: Path,
) -> None:
# Robust size invariant: a compressing method must be <= stored on
# compressible content (avoids flaky level-9-vs-level-1 comparisons).
sizes: dict[str, int] = {}
for name, constant in (("stored", zipfile.ZIP_STORED), ("deflated", zipfile.ZIP_DEFLATED)):
target: Path = tmp_path / name
target.mkdir()
with ZipExportSink(target, "export", delete=False, compression=constant) as sink:
sink.add_file(source_file, "doc.pdf")
sizes[name] = (target / "export.zip").stat().st_size
assert sizes["deflated"] <= sizes["stored"]
```
- [ ] **Step 2: Run to verify it fails**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/export/test_sinks.py::TestZipExportSinkCompression -v"`
Expected: FAIL with `TypeError: __init__() got an unexpected keyword argument 'compression'`.
- [ ] **Step 3: Add the params to `ZipExportSink`**
In `src/documents/export/sinks.py`, change `ZipExportSink.__init__` to accept the new keyword-only params and store them, and pass them in `_open()`:
```python
def __init__(
self,
target: Path,
zip_name: str,
*,
delete: bool = False,
compression: int = zipfile.ZIP_DEFLATED,
compresslevel: int | None = None,
) -> None:
self._target = target.resolve()
self._zip_path = (self._target / zip_name).with_suffix(".zip")
self._tmp_path = self._zip_path.with_name(self._zip_path.name + ".tmp")
self._delete = delete
self._compression = compression
self._compresslevel = compresslevel
self._zip: zipfile.ZipFile | None = None
self._dirs: set[str] = set()
self._pending_manifest: tuple[Path, str] | None = None
self._stream_open = False
```
And in `_open()`:
```python
def _open(self) -> None:
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
self._zip = zipfile.ZipFile(
self._tmp_path,
"w",
compression=self._compression,
compresslevel=self._compresslevel,
allowZip64=True,
)
```
- [ ] **Step 4: Run to verify it passes**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/export/test_sinks.py -v"`
Expected: PASS (all sink tests, including the four method params and the size invariant). `bzip2`/`lzma` are present on the VM's CPython, so those params pass.
- [ ] **Step 5: Lint**
Run: `ruff check src/documents/export/sinks.py && ruff format src/documents/export/sinks.py`
Expected: no errors.
- [ ] **Step 6: Commit**
```bash
git add src/documents/export/sinks.py src/documents/tests/export/test_sinks.py
git commit -m "Feature: ZipExportSink accepts compression method and level"
```
---
## Task 3: Wire CLI flags + validation into `document_exporter`
**Files:**
- Modify: `src/documents/management/commands/document_exporter.py`
- Test: `src/documents/tests/test_management_exporter.py`
- [ ] **Step 1: Add the argparse flags**
In `document_exporter.py`, add the import near the other `documents.export` import:
```python
from documents.export.compression import COMPRESSION_CHOICES
from documents.export.compression import COMPRESSION_METHODS
from documents.export.compression import compression_available
from documents.export.compression import level_error
from documents.export.compression import ZSTD
```
In `add_arguments`, after the `--zip-name` argument, add:
```python
parser.add_argument(
"--zip-compression",
choices=COMPRESSION_CHOICES,
default=None,
help=(
"Compression method for the export zip (requires --zip). "
"Default: deflated. 'zstd' requires Python 3.14+ on both the "
"exporting and importing machine."
),
)
parser.add_argument(
"--zip-compression-level",
type=int,
default=None,
help=(
"Compression level for the export zip (requires --zip). "
"deflated: 0-9, bzip2: 1-9, zstd: -131072..22; ignored for "
"stored/lzma."
),
)
```
- [ ] **Step 2: Read + validate the flags in `handle()`**
In `handle()`, after the existing `--compare-*` + `--zip` guard, add the compression flag handling. Insert before the sink construction:
```python
zip_compression: str | None = options["zip_compression"]
zip_compression_level: int | None = options["zip_compression_level"]
if not self.zip_export and (
zip_compression is not None or zip_compression_level is not None
):
raise CommandError(
"--zip-compression and --zip-compression-level require --zip",
)
compression_method = zip_compression or "deflated"
if self.zip_export:
if not compression_available(compression_method):
if compression_method == "zstd" and ZSTD is None:
raise CommandError(
"zstd compression requires Python 3.14 or newer",
)
raise CommandError(
f"Compression method '{compression_method}' is not "
f"available on this Python runtime",
)
level_msg = level_error(compression_method, zip_compression_level)
if level_msg is not None:
raise CommandError(level_msg)
```
- [ ] **Step 3: Pass the resolved values into `ZipExportSink`**
Change the `ZipExportSink(...)` construction in `handle()` to:
```python
if self.zip_export:
sink = ZipExportSink(
self.target,
options["zip_name"],
delete=self.delete,
compression=COMPRESSION_METHODS[compression_method],
compresslevel=zip_compression_level,
)
else:
sink = DirectoryExportSink(
self.target,
compare_checksums=self.compare_checksums,
compare_json=self.compare_json,
delete=self.delete,
)
```
- [ ] **Step 4: Write the command-level tests**
Add to the `TestExportImport` class in `src/documents/tests/test_management_exporter.py` (imports `call_command`, `CommandError`, `ZipFile`, `timezone` already present):
```python
def test_compression_flags_require_zip(self) -> None:
for args in (
["--zip-compression", "lzma"],
["--zip-compression-level", "5"],
):
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
*args,
skip_checks=True,
)
def test_zip_compression_level_out_of_range_raises(self) -> None:
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"deflated",
"--zip-compression-level",
"99",
skip_checks=True,
)
def test_zip_compression_level_rejected_for_stored(self) -> None:
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"stored",
"--zip-compression-level",
"5",
skip_checks=True,
)
def test_zip_lzma_compression_round_trips(self) -> None:
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"lzma",
skip_checks=True,
)
expected = str(
self.target / f"export-{timezone.localdate().isoformat()}.zip",
)
self.assertIsFile(expected)
with ZipFile(expected) as zip_file:
info = zip_file.getinfo("manifest.json")
# manifest.json carries the chosen method; deflated is the default
self.assertEqual(info.compress_type, 14) # ZIP_LZMA
def test_default_zip_uses_deflate(self) -> None:
call_command(
"document_exporter",
self.target,
"--zip",
skip_checks=True,
)
expected = str(
self.target / f"export-{timezone.localdate().isoformat()}.zip",
)
with ZipFile(expected) as zip_file:
info = zip_file.getinfo("manifest.json")
self.assertEqual(info.compress_type, 8) # ZIP_DEFLATED
```
- [ ] **Step 5: Run the tests**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/test_management_exporter.py -v"`
Expected: PASS — the new tests plus all existing exporter tests stay green.
- [ ] **Step 6: Lint**
Run: `ruff check src/documents/management/commands/document_exporter.py src/documents/tests/test_management_exporter.py && ruff format src/documents/management/commands/document_exporter.py src/documents/tests/test_management_exporter.py`
Expected: no errors.
- [ ] **Step 7: Commit**
```bash
git add src/documents/management/commands/document_exporter.py src/documents/tests/test_management_exporter.py
git commit -m "Feature: add --zip-compression and --zip-compression-level flags"
```
---
## Task 4: Importer pre-check for unreadable codecs
**Files:**
- Modify: `src/documents/management/commands/document_importer.py`
- Test: `src/documents/tests/test_management_importer.py`
- [ ] **Step 1: Write the failing test**
The importer test file `src/documents/tests/test_management_importer.py` is
`TestCase`-style (`class TestCommandImport(... TestCase)`, `self.assertRaises`,
`DirectoriesMixin` gives `self.dirs.scratch_dir`). Match that style. Add this
method to `TestCommandImport`. It builds a valid zip and patches the readability
probe so the check fires deterministically on any runtime:
```python
def test_import_rejects_unreadable_compression(self) -> None:
"""
GIVEN:
- A zip archive with an entry whose compression this Python can't read
WHEN:
- Import is attempted
THEN:
- A CommandError naming the issue is raised, before extraction
"""
import zipfile
from unittest import mock
archive = Path(self.dirs.scratch_dir) / "export.zip"
with zipfile.ZipFile(archive, "w") as zf:
zf.writestr("manifest.json", "[]")
with mock.patch(
"documents.management.commands.document_importer.compress_type_readable",
return_value=False,
):
with self.assertRaises(CommandError) as e:
call_command(
"document_importer",
str(archive),
"--no-progress-bar",
skip_checks=True,
)
self.assertIn("compression", str(e.exception))
```
- [ ] **Step 2: Run to verify it fails**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/test_management_importer.py -k unreadable_compression -v"`
Expected: FAIL — no pre-check exists yet, so the import proceeds (or fails with a different error).
- [ ] **Step 3: Implement the pre-check**
In `document_importer.py`, add the import:
```python
from documents.export.compression import compress_type_readable
from documents.export.compression import unreadable_method_names
```
Find the zip-handling block (around `document_importer.py:453`):
```python
with ZipFile(self.source) as zf:
zf.extractall(tmp_dir)
```
Replace it with a pre-check before extraction:
```python
with ZipFile(self.source) as zf:
unsupported = {
info.compress_type
for info in zf.infolist()
if not compress_type_readable(info.compress_type)
}
if unsupported:
names = ", ".join(sorted(unreadable_method_names(unsupported)))
raise CommandError(
f"This archive uses compression this Python cannot "
f"read ({names}). zstd archives require Python 3.14+.",
)
zf.extractall(tmp_dir)
```
Confirm `CommandError` is imported in `document_importer.py` (it is used elsewhere; if not, add `from django.core.management.base import CommandError`).
- [ ] **Step 4: Run to verify it passes**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/test_management_importer.py -v"`
Expected: PASS — the new test plus all existing importer tests (normal deflated/stored archives still import).
- [ ] **Step 5: Lint**
Run: `ruff check src/documents/management/commands/document_importer.py src/documents/tests/test_management_importer.py && ruff format src/documents/management/commands/document_importer.py src/documents/tests/test_management_importer.py`
Expected: no errors.
- [ ] **Step 6: Commit**
```bash
git add src/documents/management/commands/document_importer.py src/documents/tests/test_management_importer.py
git commit -m "Feature: importer rejects archives with unreadable compression"
```
---
## Task 5: Document the flags
**Files:**
- Modify: `docs/administration.md`
- [ ] **Step 1: Add the flags to the option list**
In `docs/administration.md`, update the usage block (around line 257) to include the new flags:
```
document_exporter target [-c] [-d] [-f] [-na] [-nt] [-p] [-sm] [-z]
optional arguments:
-c, --compare-checksums
-cj, --compare-json
-d, --delete
-f, --use-filename-format
-na, --no-archive
-nt, --no-thumbnail
-p, --use-folder-prefix
-sm, --split-manifest
-z, --zip
-zn, --zip-name
--zip-compression
--zip-compression-level
--data-only
--no-progress-bar
--passphrase
```
- [ ] **Step 2: Add the prose**
After the `-z`/`--zip` paragraph (around line 330), add:
```markdown
The compression method for the zip can be set with `--zip-compression`
(`stored`, `deflated` (default), `bzip2`, `lzma`, or `zstd`) and tuned with
`--zip-compression-level` (deflated: 09, bzip2: 19, zstd: -13107222; ignored
for `stored` and `lzma`). Both options require `--zip`.
!!! warning
`zstd` compression requires Python 3.14 or newer on **both** the machine
creating the export and any machine importing it. An archive compressed with
`zstd` (or `lzma`/`bzip2` where those modules are unavailable) cannot be
imported on a runtime that lacks the codec; the importer will refuse it with
a clear error. The default `deflated` is universally readable.
```
- [ ] **Step 3: Verify the docs build is not broken (lint markdown)**
Run: `ruff check docs/ 2>/dev/null; echo "docs are markdown; rely on prettier pre-commit"`
(No code to test. The prettier pre-commit hook will reformat on commit.)
- [ ] **Step 4: Commit**
```bash
git add docs/administration.md
git commit -m "Docs: document --zip-compression and --zip-compression-level"
```
---
## Task 6: Final verification
**Files:** none (verification only).
- [ ] **Step 1: Full backend suites on the VM**
Run: `bash /c/Users/tholmes/Documents/Coding/paperless/vmtest.sh "src/documents/tests/export/ src/documents/tests/test_management_exporter.py src/documents/tests/test_management_importer.py -v"`
Expected: PASS, no failures.
- [ ] **Step 2: Spot-check the zstd happy path on Python 3.14 (cannot run under Django on the 3.11 VM)**
The zstd positive round-trip can't run in the 3.11 test env. Confirm the policy module behaves on a real 3.14 interpreter with a standalone check (no Django needed):
Run:
```bash
uv run --python 3.14 --no-project python -c "import sys; sys.path.insert(0,'src'); import django; print('skip')" 2>/dev/null || \
uv run --python 3.14 --no-project python -c "
import zipfile, io
from compression.zstd import CompressionParameter as CP
print('zstd const', zipfile.ZIP_ZSTANDARD, 'bounds', CP.compression_level.bounds())
buf = io.BytesIO()
with zipfile.ZipFile(buf,'w',compression=zipfile.ZIP_ZSTANDARD,compresslevel=19) as zf:
zf.writestr('a.txt','x'*1000)
with zipfile.ZipFile(buf) as zf:
assert zf.getinfo('a.txt').compress_type == zipfile.ZIP_ZSTANDARD
assert zf.read('a.txt') == b'x'*1000
print('zstd round-trip OK')
"
```
Expected: prints `zstd const 93 bounds (-131072, 22)` and `zstd round-trip OK`. This validates the constant, bounds, and that a zstd archive round-trips — the parts the 3.11 CI cannot exercise.
- [ ] **Step 3: Type-check on the VM (pyrefly)**
```bash
tar czf - src pyproject.toml uv.lock .pyrefly-baseline.json | ssh -o BatchMode=yes -p 2244 trenton@localhost 'tar xzf - -C ~/projects/paperless-ngx'
ssh -o BatchMode=yes -p 2244 trenton@localhost 'bash -lc "cd ~/projects/paperless-ngx && uv run pyrefly check"'
```
Expected: no new type errors beyond the baseline. (Note: `import compression.zstd` is guarded behind `importlib.import_module`, so it is never statically resolved on the 3.11 baseline.)
- [ ] **Step 4: Final lint**
Run: `ruff check src/documents/export/ src/documents/management/commands/document_exporter.py src/documents/management/commands/document_importer.py && ruff format --check src/documents/export/ src/documents/management/commands/document_exporter.py src/documents/management/commands/document_importer.py`
Expected: clean.
---
## Notes for the implementer
- **Default behavior is unchanged:** with no flags, the sink is constructed with `compression=ZIP_DEFLATED, compresslevel=None` — byte-method-identical to today (`shutil.make_archive` used `ZIP_DEFLATED` with no level). `test_default_zip_uses_deflate` pins this.
- **zstd availability is gated three ways and never imported statically:** the constant via `getattr`, the codec via `importlib.import_module("compression.zstd")`, and the CLI value rejected with a friendly message on < 3.14. The choices list always contains `zstd` so argparse doesn't hide it.
- **The importer pre-check is the safety net** for portability foot-guns — without it an unreadable entry raises a bare `NotImplementedError` mid-`extractall`. The check runs on `infolist()` (metadata only) before any extraction.
- **Why `--zip-compression` defaults to `None`, not `"deflated"`:** so `handle()` can detect "user passed it without `--zip`" and fail fast. The effective default is resolved as `zip_compression or "deflated"`.
@@ -1,337 +0,0 @@
# Export Sink Architecture — Design
**Date:** 2026-06-16
**Branch base:** `dev`
**Status:** Approved design, pending implementation plan
## Problem
The `document_exporter` management command can export to a folder or to a zip
file, but the zip support is bolted on rather than designed in:
- **Zip mode is a temp-dir detour.** `handle()` redirects `self.target` to a
`tempfile.TemporaryDirectory` in `SCRATCH_DIR`, runs the entire export against
that directory, then calls `shutil.make_archive` to zip the whole tree and
cleans the temp dir up (`document_exporter.py:322-358`). The export is written
to disk twice (loose files, then the zip).
- **An attempted "direct to zip" refactor leaks the destination everywhere.**
The prior work on `feature-direct-zip-export` threads `if self.zip_export:`
branches through `check_and_copy`, `check_and_write_json`,
`_write_split_manifest`, `dump`, `handle`, and `StreamingManifestWriter`. Each
write site grew a second code path plus a `.resolve().relative_to(self.target)`
arcname dance. The destination became a cross-cutting concern smeared across
the command.
- **The command owns logic that isn't about the export contents.** Incremental
sync — the `files_in_export_dir` snapshot, the `--compare-checksums` /
`--compare-json` skip-if-unchanged checks, and the `--delete` stale-file prune —
is interleaved with the logic that decides _what_ to export. These behaviors
only make sense for a folder destination, yet they live in the command body.
- **Atomicity is informal.** A backup must never look complete when it isn't.
The temp-dir approach happens to be safe (the zip is built last), but there is
no explicit "produce the archive only if the whole run succeeded" contract, and
the direct-to-zip branch had to hand-manage a `.tmp` file inline.
## Goal
Separate **what** is exported (the command's job) from **where/how** it lands
(the destination's job), behind a small `ExportSink` abstraction. The command
declares files, JSON blobs, and a streamed manifest; the sink decides whether and
how to persist each one. Folder and zip become two interchangeable sinks, and a
future `S3ExportSink` is a third implementation rather than a fourth set of
branches. The zip is produced **only** if the entire export succeeds.
## Scope
In scope:
- New `documents/export/` package with the `ExportSink` ABC and two concrete
sinks (`DirectoryExportSink`, `ZipExportSink`).
- Move all incremental-sync machinery (snapshot, compare, prune) out of the
command and into `DirectoryExportSink`.
- Rewrite `document_exporter.handle()` / `dump()` to be destination-agnostic.
- Simplify `StreamingManifestWriter` to write to a sink-provided handle.
- Unit tests for each sink; keep existing command-level tests green.
Out of scope:
- `bulk_download.py` / `BulkArchiveStrategy` and share-link bundle zipping. Those
select _which document files_ go in and stream to an HTTP response with no
atomic-finalize requirement — a different axis from the backup sink. Untouched.
- Actually implementing an S3 (or any cloud) sink. The interface is designed to
_allow_ one; we do not build one (YAGNI).
- Changing the export's on-disk/in-zip layout, manifest schema, crypto, or any
CLI flag's meaning. Behavior is preserved; only the destination plumbing moves.
- Zip compression control (method / level). The `ZipExportSink` keeps today's
fixed `ZIP_DEFLATED` here; making compression configurable is a follow-up —
see `2026-06-16-export-zip-compression-design.md`, which depends on this
refactor landing first. The sink is the single seam that makes it a small,
isolated change.
## Decisions
These were settled during brainstorming:
1. **Scope is the `document_exporter` command only.** Design the interface so an
S3 sink could be added later; do not refactor `bulk_download` or share bundles.
2. **`--compare-*` are folder-only (hard error with `--zip`); `--delete` is kept
for both.** `--compare-checksums` / `--compare-json` are genuine no-ops in zip
mode today (the temp dir is always empty, so the compare always copies), so
combining either with `--zip` raises a `CommandError` up front. **`--delete`,
however, is an existing tested feature in zip mode** — it wipes the destination
directory of pre-existing files/dirs before the archive lands
(`test_export_zipped_with_delete`). Its meaning differs by destination: folder
`--delete` prunes stale exported files; zip `--delete` clears the target dir.
Both are preserved — `--delete` is a parameter of _both_ sinks, not an error.
3. **The zip manifest spools to a temp file, not memory.** The sink exposes a
streaming-write handle. The zip sink streams the manifest to a single temp
file in `SCRATCH_DIR` and adds it as the manifest entry at finalize, keeping
peak memory flat regardless of library size. The only "temp" artifact is one
manifest file, not a whole export tree.
## Architecture
### The `ExportSink` interface
New module `documents/export/sinks.py`:
```python
class ExportSink(AbstractContextManager):
"""Destination for a document export.
The command declares export contents via the three verbs below; the sink
decides whether and how to persist each item. arcname is always a relative
POSIX path (e.g. "manifest.json", "originals/foo.pdf").
"""
def add_file(
self,
source: Path,
arcname: str,
*,
checksum: str | None = None,
) -> None:
"""Persist an existing file at the relative arcname."""
def add_json(self, content: list | dict, arcname: str) -> None:
"""Persist JSON-serializable content at the relative arcname."""
def stream(self, arcname: str) -> ContextManager[TextIO]:
"""Yield a writable text handle for incrementally produced content.
Reserved for the bulk manifest. At most one stream may be open at a
time; add_file/add_json may be called freely while it is open.
"""
# __enter__ opens the sink and returns self.
# __exit__ calls finalize() on success, abort() on exception.
```
**Contract / invariants** (the checklist a future sink author honors):
- `arcname` is relative and **POSIX-style (forward slashes)**; the sink maps it to
its own namespace (folder: joined under the target; zip: the entry name). The
command must build arcnames with `Path(...).as_posix()``str(Path(...))`
yields backslashes on Windows, which corrupts zip entry names and makes the
manifest's stored paths non-portable. The same string is used both as the sink
key and as the value stored in the manifest (`EXPORTER_FILE_NAME` etc.), so it
must be POSIX at the point of construction. (The share-link bundle path already
uses `.as_posix()`; the document targets currently do not and must be fixed.)
- At most one `stream()` is open at a time. It is the manifest. `add_file` /
`add_json` may be interleaved with an open stream — implementations that can't
interleave a real stream (zip, S3) must spool the stream to a side buffer and
emit it at `finalize()`.
- The sink is a context manager. Normal exit finalizes; an exception aborts.
**No partial or failed run may leave a "complete-looking" artifact.**
### `DirectoryExportSink(target, *, compare_checksums, compare_json, delete)`
Owns everything the command currently does for folder mode:
- On open: snapshot existing files under `target` (today's `files_in_export_dir`).
- `add_file`: the `check_and_copy` skip logic (mtime/size, or checksum when
`compare_checksums`), then copy with stat preservation. Records the arcname as
"seen this run".
- `add_json`: the `check_and_write_json` blake2b compare-or-write (honoring
`compare_json`). Records the arcname as seen.
- `stream`: yields a handle writing to `<arcname>.tmp`; on context close, applies
the `compare_json` blake2b compare and renames-or-discards (today's
`StreamingManifestWriter` finalize). Records the arcname as seen.
- `finalize()` (success only): if `delete`, prune every snapshot file not seen
this run and clean up emptied directories (today's stale-delete pass).
- `abort()` (on exception): discard any in-flight `.tmp`; leave existing files
intact; do **not** run the prune.
The folder sink is inherently in-place/incremental, not atomic — that is its
nature and is unchanged. Its safety is the per-file `.tmp`+rename it already does.
### `ZipExportSink(target, zip_name, *, delete)`
- On open: ensure `SCRATCH_DIR` exists (`mkdir(parents=True, exist_ok=True)`
today's `handle()` does this before using it; the sink must do it now), then
open a `zipfile.ZipFile` at `<target>/<zip_name>.zip.tmp` (`ZIP_DEFLATED`,
`allowZip64=True`). The `.zip.tmp` lives in the same directory as the final
`.zip` so the finalize rename is atomic (same filesystem).
- `add_file` / `add_json`: write the entry directly, first emitting directory
marker entries for parent paths so every zip viewer shows the folder structure
(today's `_ensure_zip_dirs`). A _flat_ export (no `--use-folder-prefix`, no
nested arcnames) has no parent dirs, so it emits **zero** markers — matching
today's `make_archive` output for flat trees (keeps the `namelist()` count
assertions in `test_export_zipped` valid). Nested/prefixed exports gain marker
entries; any count assertion on those must be audited.
- `stream`: yields a handle writing to a single temp file in `SCRATCH_DIR`.
- `finalize()` (success only): add the spooled manifest temp file as its entry,
close the zip, then **if `delete`, wipe the destination directory** of every
pre-existing file/dir except the in-progress `.zip.tmp` and any prior `.zip`
(today's zip `--delete` behavior), then atomically rename `.zip.tmp``.zip`.
- `abort()` (on exception): close the zip, unlink the `.zip.tmp`, delete the
manifest temp file. **A `.zip` therefore exists only after a fully successful
run**, and on abort the destination is never wiped.
- Rejects `compare_*` (the command guards this before constructing the sink). It
does **not** reject `delete` — that is a supported zip behavior (see above).
### Command changes (`document_exporter.py`)
- **`handle()`**: validate the target, then _up front_ raise `CommandError` if
`--compare-checksums` or `--compare-json` is combined with `--zip` (those are
no-ops in zip mode). `--delete` is **not** rejected — it is passed to whichever
sink is built. Construct the appropriate sink (`delete=` passed to both). Run
the export as `with sink: self.dump(sink)`. Delete the temp-dir /
`shutil.make_archive` block entirely.
- **`--data-only`**: unchanged in meaning — it simply skips every `sink.add_file`
call (no document/thumbnail/archive/bundle files) while the manifest stream and
`metadata.json` are still written. Works identically for both sinks; no sink
code is data-only-aware. (`test_export_data_only` and its zip equivalent stay
green.)
- **`dump(sink)`**: destination-agnostic. Builds relative arcnames and calls
`sink.add_file(...)`, `sink.add_json(...)`, and `sink.stream("manifest.json")`.
`self.files_in_export_dir`, `check_and_copy`, `check_and_write_json`, and the
stale-delete pass are removed (their logic now lives in the folder sink).
- **`generate_document_targets`**: returns relative arcnames
(`originals/<name>`, `<name>-thumbnail.webp`, `archive/<name>-archive.pdf`)
instead of absolute `self.target / ...` paths. It already writes the relative
name into `document_dict[EXPORTER_FILE_NAME]` etc.; we just drop the absolute
half.
- **`StreamingManifestWriter`**: simplified to write JSON-array records to the
text handle returned by `sink.stream("manifest.json")`. It no longer knows
folder vs zip, owns no `.tmp` logic, and has no compare/zip parameters — that
behavior moved into each sink's `stream()`.
- **Crypto / passphrase** handling stays in the command: it transforms record
_contents_ before they reach the sink, which is independent of destination.
- **Progress tracking stays in the command — the sinks know nothing about it.**
`PaperlessCommand.track()` wraps the _document iterable_ in `dump()` and ticks
the Rich bar once per document. That loop stays in the command; each iteration
calls `sink.add_file(...)`, so the per-document progress is preserved
unchanged. The sinks deliberately do **not** depend on `PaperlessCommand`,
`track()`, or Rich — coupling the destination abstraction to the command
framework would defeat the isolation goal and make the sinks impossible to unit
-test without a full command. (A sink is a plain context-managed I/O object; it
is constructed by `handle()` and exercised directly in `test_sinks.py`.) If
finer-grained progress is ever wanted for a single very large file, that is a
future enhancement layered via an optional callback — not a `PaperlessCommand`
dependency, and out of scope here.
### How `--split-manifest` fits (no sink special-casing)
`--split-manifest` is purely a command-level choice and touches no sink code:
- The single bulk `manifest.json` is always the one and only `sink.stream(...)`
handle. In split mode it simply carries fewer record types (document records,
notes, and custom-field-instances are redirected out).
- Per-document `<base>-manifest.json` files are small _complete_ JSON blobs — they
were never streamed. `_write_split_manifest` collapses to building the content
list and one `sink.add_json(content, "<base>-manifest.json")` call, exactly
like `metadata.json`.
Because the manifest stream is backed by its own handle (a `.tmp` file in the
folder sink, a `SCRATCH_DIR` temp file in the zip sink) and never an open zip
entry, the per-document `add_json` / `add_file` calls made _while the bulk
manifest stream is open_ never collide with it.
## Data flow
```
handle(options)
├─ validate target; reject --compare-* + --zip → CommandError (--delete allowed)
├─ sink = DirectoryExportSink(..., delete=…) | ZipExportSink(..., delete=…)
└─ with FileLock(MEDIA_LOCK), sink:
dump(sink)
├─ with sink.stream("manifest.json") as mh:
│ writer = StreamingManifestWriter(mh)
│ ├─ global querysets → writer.write_batch(...) (encrypted inline)
│ ├─ per document:
│ │ ├─ sink.add_file(source, "originals/…", checksum=…)
│ │ ├─ sink.add_file(thumb, "…-thumbnail.webp")
│ │ ├─ sink.add_file(archive,"archive/…-archive.pdf", checksum=…)
│ │ └─ split? sink.add_json(doc_bundle, "…-manifest.json")
│ │ : writer.write_record(doc_record)
│ └─ per share-link bundle: sink.add_file(...) + writer.write_record(...)
└─ sink.add_json(metadata, "metadata.json")
(success → sink.finalize(); exception → sink.abort())
```
## Error handling & atomicity
- Any exception in `dump()` propagates through `with sink:``__exit__`
`abort()`. Zip: the `.zip.tmp` and the manifest temp file are deleted, and the
destination is **not** wiped; **no `.zip` is produced.** Folder: in-flight
`.tmp` files are discarded, existing files are left intact, and the stale-prune
does not run.
- `finalize()` runs only on clean exit, after all contents are written. For the
zip: optionally wipe the destination (`--delete`), then the single `.zip.tmp`
`.zip` rename (atomic on the same filesystem). For the folder: the optional
stale-delete prune.
- **Honest limits of the atomicity guarantee.** The guarantee is "no
_complete-looking_ `.zip` after a failed run," not "no leftovers." If the
process is `SIGKILL`ed or the rename itself fails _after_ the zip is closed, a
`.zip.tmp` may be orphaned — that is the safe direction (no false-complete
`.zip`), but stale `.zip.tmp` files are **not** auto-cleaned on a later run
(matching the prior branch). `KeyboardInterrupt` is a `BaseException` but
`__exit__` still runs, so `abort()` fires normally. The rename being atomic and
these runs not racing each other both rely on `FileLock(settings.MEDIA_LOCK)`,
which serializes exports; concurrent same-`--zip-name` runs are out of scope.
- The `FileLock(settings.MEDIA_LOCK)` wrapping is unchanged.
## Testing
New `documents/export/tests/test_sinks.py`, unit-testing each sink in isolation
(pytest classes, factory-boy factories, the `mocker` fixture, `parametrize`, full
type annotations; run on the Linux VM):
- **Round-trip** (both sinks, parametrized): `add_file` + `add_json` + a streamed
manifest produce the expected files/entries with correct relative arcnames.
- **Folder incremental**: unchanged file is skipped under `compare_checksums` and
under `compare_json`; `delete` prunes a snapshot file not written this run and
removes emptied directories; without `delete`, stale files remain.
- **Zip atomicity**: injecting an exception mid-export (via `mocker`) leaves no
`.zip` and no leftover `.zip.tmp`, and does not wipe the destination even with
`--delete`; a clean run yields exactly the `.zip`. A nested/prefixed export has
directory marker entries; a flat export has none.
- **Zip `--delete`**: a clean `--zip --delete` run wipes pre-existing
files/dirs in the destination and produces the `.zip` (preserves
`test_export_zipped_with_delete`).
- **POSIX arcnames**: nested arcnames are stored with forward slashes in both the
zip entry names and the manifest values, regardless of host OS (guards the
Windows backslash bug).
- **`--data-only`**: both sinks produce only `manifest.json` + `metadata.json`,
no document files.
- **Stream contract**: opening a second concurrent `stream()` is rejected;
`add_file`/`add_json` while a stream is open succeed.
- **Command guard**: `--zip` with `--compare-checksums` or `--compare-json`
raises `CommandError`; `--zip --delete` does **not** error.
Existing `test_management_exporter.py` and `test_management_importer.py` stay
green unchanged — the export's external behavior (layout, manifest, round-trip
import, `--zip --delete`, `--data-only`) is preserved.
## Risks
- **Behavior drift in the folder path.** The incremental logic is subtle
(mtime/size vs checksum, blake2b json compare, empty-dir cleanup). Mitigation:
move it verbatim into the sink and lean on the unchanged command-level tests
plus new focused sink tests.
- **Manifest interleaving in zip mode.** Relies on the spool-to-temp-file
decision; the stream contract makes this explicit and the stream-contract test
guards it.
@@ -1,236 +0,0 @@
# Export Zip Compression Control — Design
**Date:** 2026-06-16
**Branch base:** `dev`
**Status:** Design complete (zstd facts verified on CPython 3.14.3) — **depends on**
`2026-06-16-export-sink-architecture-design.md` being implemented first.
## Prerequisite
This builds directly on the export sink refactor. It assumes `ZipExportSink`
already exists and is the single place that owns `zipfile.ZipFile` creation and
entry writes. Do not start this until that refactor has landed; without it, the
change would have to touch the command's zip branches again.
## Problem
Zip export is hardwired to `ZIP_DEFLATED` at the library default level. Users
have no way to trade speed against archive size — a fast `ZIP_STORED` pass for a
quick local copy, or a maximal `ZIP_LZMA` pass for the smallest off-site backup.
The sink refactor turns "which compression" into a single constructor argument,
so exposing it is now a small, isolated change.
## Goal
Let the operator choose the zip compression method and level from the CLI, with
behavior identical to today when the flags are omitted. All knowledge of
compression stays inside `ZipExportSink`; the command only parses flags and maps
them to sink arguments.
## Scope
In scope:
- `ZipExportSink` gains `compression: int` and `compresslevel: int | None`
constructor parameters (default `ZIP_DEFLATED`, `None` → library default),
passed straight to `zipfile.ZipFile(...)`.
- New `document_exporter` flags: `--zip-compression` and
`--zip-compression-level`, valid only with `--zip`.
- Validation: method availability, level range per method, and the
requires-`--zip` guard.
- Import-side: a pre-extract support check in `document_importer` that turns an
unsupported codec into a clear `CommandError` (the importer otherwise decompresses
transparently via `ZipFile.extractall`).
- Docs: add both flags and the zstd-portability caveat to `docs/administration.md`
(the `document_exporter` option list, lines ~257-270 and the `-z`/`-zn` section,
lines ~328-330). New flags are long-form only (`--zip-compression`,
`--zip-compression-level`) — no short aliases, to avoid `-zc`/`-zl` collisions
with the existing `-z`/`-zn`.
Out of scope:
- Compression for any non-zip sink (folder has none; a future S3 sink would
handle its own object storage compression separately).
- Changing the default. Omitting the flags must produce a byte-compatible-method
archive to today's (`ZIP_DEFLATED`, default level).
## Design
### `ZipExportSink` changes
The base sink's signature is `ZipExportSink(target, zip_name, *, delete)`; this
adds two keyword-only params after `delete`:
```python
def __init__(
self,
target: Path,
zip_name: str,
*,
delete: bool = False,
compression: int = zipfile.ZIP_DEFLATED,
compresslevel: int | None = None,
) -> None:
...
# opened in __enter__:
self._zip = zipfile.ZipFile(
self._tmp_path,
"w",
compression=compression,
compresslevel=compresslevel,
allowZip64=True,
)
```
`ZipFile` applies `compression`/`compresslevel` as the default for every
`write`/`writestr` (verified: a `ZipFile(..., compression=ZIP_BZIP2)` yields
entries with `compress_type == ZIP_BZIP2` without per-call args), so `add_file` /
`add_json` / the manifest entry need no changes. Directory marker entries are
empty so their compressed payload is zero, but they are still _tagged_ with the
chosen `compress_type` — harmless, but tests that read `infolist()` should filter
or account for marker entries (see Testing).
### CLI flags (`document_exporter`)
- `--zip-compression {stored,deflated,bzip2,lzma}` — and `zstd` **when the
runtime supports it** (see below). Maps to the matching `zipfile.ZIP_*`
constant. Default `deflated`.
- `--zip-compression-level N` — integer. Per-method accepted ranges (verified
against the [3.14 `zipfile` docs](https://docs.python.org/3.14/library/zipfile.html#zipfile.ZipFile)):
- `deflated`: **09** (`zlib` also accepts `-1` = "default", identical to
omitting the flag / `compresslevel=None`).
- `bzip2`: **19** (`0` is invalid for bzip2).
- `lzma`, `stored`: level has **no effect** — passing `--zip-compression-level`
with either is a `CommandError`, not a silent accept (consistent with the
base refactor's fail-fast posture).
- `zstd`: **-131072 … 22** (the documented commonly-accepted range; the
authoritative bounds are
`compression.zstd.CompressionParameter.compression_level.bounds()`).
Default: unset → library default (`compresslevel=None`).
Both flags require `--zip`; passing either without `--zip` raises a
`CommandError`, matching the incremental-flag rule from the base refactor.
**Why validate up front (not let `zipfile` raise) — verified on 3.14.3:** an
invalid level does _not_ fail at `ZipFile(...)` construction — it fails at the
**first `write`/`writestr` call**, with an opaque message
(`ValueError: Invalid initialization option` for deflated > 9, or
`ValueError: compresslevel must be between 1 and 9` for bzip2). Worse, on context
exit the half-initialized write handle emits a secondary
`AttributeError: '_ZipWriteFile' object has no attribute '_compressor'` during GC
finalization, so the user sees stack-trace noise unrelated to the real cause.
Up-front validation turns all of that into a single clean `CommandError`.
### Validation (in `handle()`, before constructing the sink)
1. **Requires `--zip`.** Either flag without `--zip``CommandError`.
2. **Method availability — via a named, patchable seam.** Expose a module-level
helper `compression_available(method: str) -> bool` that does
`try: import bz2 / import lzma / from compression import zstd except ImportError:
return False`**not** `importlib.util.find_spec`, which can report a stdlib
C-extension as present when importing it actually fails. `stored`/`deflated`
are always available (`zlib` is a hard CPython dependency). For `zstd` the probe
must import `compression.zstd` (3.14+), not merely check that
`zipfile.ZIP_ZSTANDARD` exists. Making this a named function is also what lets
the test patch "method unavailable" with `mocker`. If the chosen method is
unavailable, raise a `CommandError` naming the missing capability — `zipfile`
itself would otherwise raise a bare `RuntimeError`
("Compression requires the (missing) … module").
3. **Level range.** Reject an out-of-range `--zip-compression-level` for the
chosen method with a clear `CommandError`; reject the flag entirely for
`stored`/`lzma` (see above).
### zstd (Python 3.14+)
**Verified empirically on CPython 3.14.3** (via `uv run --python 3.14 --no-project`)
and against [PEP 784](https://peps.python.org/pep-0784/) +
[the 3.14 `zipfile` docs](https://docs.python.org/3.14/library/zipfile.html):
- The compression-method constant is **`zipfile.ZIP_ZSTANDARD`** (added 3.14; its
numeric value is `93`). It does **not** exist on < 3.14.
- It is backed by the new **`compression.zstd`** stdlib module (PEP 784 added a
`compression` namespace package; legacy `bz2`/`lzma`/`zlib` imports are
unchanged). `zipfile` raises `RuntimeError` if `compression.zstd` is
unavailable when zstd is requested.
- Accepted `compresslevel` is **`-131072 … 22`**, confirmed at runtime via
`compression.zstd.CompressionParameter.compression_level.bounds() == (-131072, 22)`.
Gate everything zstd-related at runtime so nothing is imported or referenced on
< 3.14 (the project targets Python ≥ 3.11):
```python
_ZSTD: int | None = getattr(zipfile, "ZIP_ZSTANDARD", None) # None before 3.14
```
Presence of the _constant_ does not guarantee the _codec_ is usable, so the
availability probe (validation step 2) imports `compression.zstd`, not merely
checks the constant.
Keep `zstd` in the `--zip-compression` `choices` **always** (even on < 3.14), and
reject it in validation with a friendly "zstd requires Python 3.14+" message. If
it were dropped from `choices` on older runtimes, argparse would emit a generic
"invalid choice" that reads as though the option never existed — worse UX.
### Import-side compatibility
`document_importer` reads zips with `ZipFile(self.source).extractall(...)`
(`document_importer.py:453`), which decompresses each entry transparently using
whatever method it was stored with — **provided the matching module exists on the
importing machine.**
The failure mode when it doesn't is unfriendly and must be handled: a zstd (or
otherwise unsupported) entry raises a bare `NotImplementedError` **per-entry,
during `extractall`** — _not_ at `ZipFile(self.source)` open, and `is_zipfile()`
still returns true (a zstd archive is a valid zip container). So the importer
enters the zip branch, creates its temp dir, may partially extract other entries,
then blows up mid-extract with no context. **Mitigation (in scope here):** before
extracting, inspect `ZipFile(self.source).infolist()` compress types and, if any
is unsupported on this runtime, raise a `CommandError` naming the method and the
requirement (e.g. "this archive uses zstd, which needs Python 3.14+") instead of
letting `NotImplementedError` escape.
Per-method summary (document in help text + `administration.md`):
- `deflated`/`stored`: universally importable.
- `bzip2`/`lzma`: importable wherever the `bz2`/`lzma` modules are present
(essentially always).
- `zstd`: importable only on Python 3.14+. An archive compressed with `zstd` is
**not** importable on older runtimes.
## Testing
New cases in the sink tests and an export→import round-trip
(pytest classes, factory-boy, `mocker`, `parametrize`, typed; run on the VM):
- **Round-trip per method.** Parametrize over the available methods (skip `zstd`
below 3.14, skip `bzip2`/`lzma` if the module is somehow absent): export a
small library, import it back, assert documents/manifest match.
- **Method is applied.** Assert each written _file_ entry's `compress_type`
equals the requested method (read back via `ZipFile.infolist()`), filtering out
directory marker entries (which are tagged but empty).
- **Level affects size — robustly.** Do **not** compare deflate level 9 vs 1
(on small or incompressible fixtures level 9 can equal or slightly exceed level
1, causing flaky CI). Instead assert that a compressing method on a
moderately-compressible fixture yields a total smaller than `stored`
(`ZIP_STORED`), which is a stable invariant.
- **Validation.** Each flag without `--zip``CommandError`; out-of-range level
(`--zip-compression-level 99`) → a clean `CommandError` from validation
(asserting we never reach the `writestr` that would raise the masked
`ValueError`); `--zip-compression-level` with `stored`/`lzma``CommandError`;
unavailable method (patch the named availability seam with `mocker`) →
`CommandError`; on < 3.14, `--zip-compression zstd` → the friendly
"requires 3.14+" `CommandError`.
- **Import pre-check.** An archive containing an unsupported compress type
produces a `CommandError` from the importer naming the method, not a raw
`NotImplementedError` (simulate by patching the importer's support probe).
- **Default unchanged.** Omitting both flags yields file entries with
`compress_type == ZIP_DEFLATED`, identical to pre-feature behavior.
## Risks
- **Foot-gun archives.** A user could produce a `zstd`/`lzma` archive their
import target can't read. Mitigation: explicit help text and the import-side
notes above; the default stays the universally-readable `deflated`.
- **Optional-module assumptions.** Don't assume `bz2`/`lzma` are always compiled
in; probe and error clearly. Mitigation: the availability validation step.
+2 -2
View File
@@ -458,8 +458,8 @@ For related metadata such as tags, correspondents, document types, and storage p
### Password reset
In order to enable the password reset feature you will need to setup an SMTP backend, see
[`PAPERLESS_EMAIL_HOST`](configuration.md#PAPERLESS_EMAIL_HOST). You should also set
[`PAPERLESS_URL`](configuration.md#PAPERLESS_URL) and / or its corresponding configuration settings.
[`PAPERLESS_EMAIL_HOST`](configuration.md#PAPERLESS_EMAIL_HOST). If your installation does not have
[`PAPERLESS_URL`](configuration.md#PAPERLESS_URL) set, the reset link included in emails will use the server host.
### Two-factor authentication
+4
View File
@@ -1,5 +1,6 @@
import { expect, test } from '@playwright/test'
import path from 'node:path'
import { mockFilterSelectionData } from '../mock-filter-selection-data'
const REQUESTS_HAR = path.join(__dirname, 'requests/api-settings.har')
@@ -7,6 +8,7 @@ test('should activate / deactivate save button when settings change', async ({
page,
}) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/settings')
await expect(page.getByRole('button', { name: 'Save' })).toBeDisabled()
await page.getByLabel('Use system setting').click()
@@ -16,6 +18,7 @@ test('should activate / deactivate save button when settings change', async ({
test('should warn on unsaved changes', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/settings')
await page.getByLabel('Use system setting').click()
await page.getByRole('link', { name: 'Dashboard' }).click()
@@ -28,6 +31,7 @@ test('should warn on unsaved changes', async ({ page }) => {
test('should apply appearance changes when set', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/settings')
await expect(page.locator('html')).toHaveAttribute('data-bs-theme', /auto/)
await page.getByLabel('Use system setting').click()
+6
View File
@@ -1,5 +1,6 @@
import { expect, test } from '@playwright/test'
import path from 'node:path'
import { mockFilterSelectionData } from '../mock-filter-selection-data'
const REQUESTS_HAR1 = path.join(__dirname, 'requests/api-dashboard1.har')
const REQUESTS_HAR2 = path.join(__dirname, 'requests/api-dashboard2.har')
@@ -8,6 +9,7 @@ const REQUESTS_HAR4 = path.join(__dirname, 'requests/api-dashboard4.har')
test('dashboard inbox link', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR1, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await page.getByRole('link', { name: 'Documents in inbox' }).click()
await expect(page).toHaveURL(/tags__id__in=9/)
@@ -16,6 +18,7 @@ test('dashboard inbox link', async ({ page }) => {
test('dashboard total documents link', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR2, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await page.getByRole('link').filter({ hasText: 'Total documents' }).click()
await expect(page).toHaveURL(/documents/)
@@ -25,6 +28,7 @@ test('dashboard total documents link', async ({ page }) => {
test('dashboard saved view show all', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR3, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await page
.locator('pngx-widget-frame')
@@ -38,6 +42,7 @@ test('dashboard saved view show all', async ({ page }) => {
test('dashboard saved view document links', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR4, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await page
.locator('pngx-widget-frame')
@@ -51,6 +56,7 @@ test('dashboard saved view document links', async ({ page }) => {
test('test slim sidebar', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR1, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await page.locator('.sidebar-slim-toggler').click()
await expect(
@@ -1,5 +1,6 @@
import { expect, test } from '@playwright/test'
import path from 'node:path'
import { mockFilterSelectionData } from '../mock-filter-selection-data'
const REQUESTS_HAR = path.join(__dirname, 'requests/api-document-detail.har')
const REQUESTS_HAR2 = path.join(__dirname, 'requests/api-document-detail2.har')
@@ -8,6 +9,7 @@ test('should activate / deactivate save button when changes are saved', async ({
page,
}) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents/175/')
await page.waitForSelector('pngx-document-detail pngx-input-text:first-child')
await expect(page.getByTitle('Storage path', { exact: true })).toHaveText(
@@ -20,6 +22,7 @@ test('should activate / deactivate save button when changes are saved', async ({
test('should warn on unsaved changes', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents/175/')
await expect(page.getByTitle('Correspondent', { exact: true })).toHaveText(
/\w+/
@@ -39,6 +42,7 @@ test('should warn on unsaved changes', async ({ page }) => {
test('should support tab direct navigation', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents/175/details')
await expect(page.getByRole('tab', { name: 'Details' })).toHaveAttribute(
'aria-selected',
@@ -68,6 +72,7 @@ test('should support tab direct navigation', async ({ page }) => {
test('should show a mobile preview', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents/175/')
await page.setViewportSize({ width: 400, height: 1000 })
await expect(page.getByRole('tab', { name: 'Preview' })).toBeVisible()
@@ -77,6 +82,7 @@ test('should show a mobile preview', async ({ page }) => {
test('should show a list of notes', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents/175/notes')
await expect(page.locator('pngx-document-notes')).toBeVisible()
await expect(
@@ -89,6 +95,7 @@ test('should show a list of notes', async ({ page }) => {
test('should support quick filters', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR2, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents/175/details')
await page
.getByRole('button', { name: 'Filter documents with these Tags' })
@@ -1,5 +1,6 @@
import { expect, test } from '@playwright/test'
import path from 'node:path'
import { mockFilterSelectionData } from '../mock-filter-selection-data'
const REQUESTS_HAR1 = path.join(__dirname, 'requests/api-document-list1.har')
const REQUESTS_HAR2 = path.join(__dirname, 'requests/api-document-list2.har')
@@ -10,6 +11,7 @@ const REQUESTS_HAR6 = path.join(__dirname, 'requests/api-document-list6.har')
test('basic filtering', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR1, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents')
await page.getByRole('button', { name: 'Tags' }).click()
await page.getByRole('menuitem', { name: 'Inbox' }).click()
@@ -45,6 +47,7 @@ test('basic filtering', async ({ page }) => {
test('text filtering', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR2, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents')
await page.getByRole('main').getByRole('combobox').click()
await page.getByRole('main').getByRole('combobox').fill('test')
@@ -81,6 +84,7 @@ test('text filtering', async ({ page }) => {
test('date filtering', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR3, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents')
await page.getByRole('button', { name: 'Dates' }).click()
await page.locator('.ng-arrow-wrapper').first().click()
@@ -103,6 +107,7 @@ test('date filtering', async ({ page }) => {
test('sorting', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR4, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents')
await page.getByRole('button', { name: 'Sort' }).click()
await page.getByRole('button', { name: 'ASN' }).click()
@@ -141,6 +146,7 @@ test('sorting', async ({ page }) => {
test('change views', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR5, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents')
await page.locator('.btn-group > label').first().click()
await expect(page.locator('pngx-document-list table')).toBeVisible()
@@ -152,6 +158,7 @@ test('change views', async ({ page }) => {
test('bulk edit', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR6, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/documents')
await page.locator('pngx-document-card-small').nth(0).click()
+35
View File
@@ -0,0 +1,35 @@
import { Page } from '@playwright/test'
const EMPTY_SELECTION_DATA = {
selected_correspondents: [],
selected_tags: [],
selected_document_types: [],
selected_storage_paths: [],
selected_custom_fields: [],
}
/**
* The document list now fires a GET to filter_selection_data on every
* non-search reload(), independent of and concurrent with the main list
* request. It's not present in any of the recorded HAR fixtures, so with
* `notFound: 'fallback'` it would otherwise fall through to the real
* network (nothing listens there in e2e, since only the frontend dev
* server is started) and fail every test that reloads the list.
*
* Playwright checks routes in reverse-registration order, so this must be
* registered before a test's own page.routeFromHAR() call for the HAR
* route's `notFound: 'fallback'` to defer back to this one.
*/
export async function mockFilterSelectionData(page: Page) {
await page.route('**/api/documents/filter_selection_data/**', (route) =>
route.fulfill({
json: EMPTY_SELECTION_DATA,
// The app calls the (cross-origin, from the e2e app's perspective)
// backend at http://localhost:8000 while served from :4200, so a
// fulfilled response needs the same CORS header the real backend
// sends (and that recorded HAR responses already carry) or the
// browser rejects it as a cross-origin failure.
headers: { 'Access-Control-Allow-Origin': 'http://localhost:4200' },
})
)
}
@@ -1,10 +1,12 @@
import { expect, test } from '@playwright/test'
import path from 'node:path'
import { mockFilterSelectionData } from '../mock-filter-selection-data'
const REQUESTS_HAR = path.join(__dirname, 'requests/api-global-permissions.har')
test('should not allow user to edit settings', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(page.getByRole('link', { name: 'Settings' })).not.toBeAttached()
await page.goto('/settings')
@@ -15,6 +17,7 @@ test('should not allow user to edit settings', async ({ page }) => {
test('should not allow user to view documents', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(
page.locator('nav').getByRole('link', { name: 'Documents' })
@@ -31,6 +34,7 @@ test('should not allow user to view documents', async ({ page }) => {
test('should not allow user to view correspondents', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(
page.getByRole('link', { name: 'Attributes' })
@@ -43,6 +47,7 @@ test('should not allow user to view correspondents', async ({ page }) => {
test('should not allow user to view tags', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(
page.getByRole('link', { name: 'Attributes' })
@@ -55,6 +60,7 @@ test('should not allow user to view tags', async ({ page }) => {
test('should not allow user to view document types', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(
page.getByRole('link', { name: 'Attributes' })
@@ -67,6 +73,7 @@ test('should not allow user to view document types', async ({ page }) => {
test('should not allow user to view storage paths', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(
page.getByRole('link', { name: 'Attributes' })
@@ -79,6 +86,7 @@ test('should not allow user to view storage paths', async ({ page }) => {
test('should not allow user to view logs', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(page.getByRole('link', { name: 'Logs' })).not.toBeAttached()
await page.goto('/logs')
@@ -89,6 +97,7 @@ test('should not allow user to view logs', async ({ page }) => {
test('should not allow user to view tasks', async ({ page }) => {
await page.routeFromHAR(REQUESTS_HAR, { notFound: 'fallback' })
await mockFilterSelectionData(page)
await page.goto('/dashboard')
await expect(page.getByRole('link', { name: 'Tasks' })).not.toBeAttached()
await page.goto('/tasks')
+68 -87
View File
@@ -1317,7 +1317,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">237</context>
<context context-type="linenumber">236</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -1615,7 +1615,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">206</context>
<context context-type="linenumber">205</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -1646,7 +1646,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">246</context>
<context context-type="linenumber">245</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -1677,7 +1677,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">255</context>
<context context-type="linenumber">254</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -1716,7 +1716,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">219</context>
<context context-type="linenumber">218</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -2065,7 +2065,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">264</context>
<context context-type="linenumber">263</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -2236,11 +2236,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">76</context>
<context context-type="linenumber">72</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">147</context>
<context context-type="linenumber">143</context>
</context-group>
</trans-unit>
<trans-unit id="3278307631146748151" datatype="html">
@@ -3477,11 +3477,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">82</context>
<context context-type="linenumber">78</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">153</context>
<context context-type="linenumber">149</context>
</context-group>
</trans-unit>
<trans-unit id="searchResults.noResults" datatype="html">
@@ -3833,7 +3833,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">79</context>
<context context-type="linenumber">75</context>
</context-group>
</trans-unit>
<trans-unit id="3188389494264426470" datatype="html">
@@ -4067,7 +4067,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">273</context>
<context context-type="linenumber">272</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -5906,7 +5906,7 @@
<source>Create</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.html</context>
<context context-type="linenumber">54</context>
<context context-type="linenumber">56</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/share-links-dialog/share-links-dialog.component.html</context>
@@ -5921,14 +5921,14 @@
<source>Apply</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.html</context>
<context context-type="linenumber">60</context>
<context context-type="linenumber">62</context>
</context-group>
</trans-unit>
<trans-unit id="7780041345210191160" datatype="html">
<source>Click again to exclude items.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.html</context>
<context context-type="linenumber">73</context>
<context context-type="linenumber">75</context>
</context-group>
</trans-unit>
<trans-unit id="7593728289020204896" datatype="html">
@@ -5943,7 +5943,7 @@
<source>Open <x id="PH" equiv-text="this.title"/> filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.ts</context>
<context context-type="linenumber">828</context>
<context context-type="linenumber">824</context>
</context-group>
</trans-unit>
<trans-unit id="7005745151564974365" datatype="html">
@@ -7400,11 +7400,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">29</context>
<context context-type="linenumber">25</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">318</context>
<context context-type="linenumber">317</context>
</context-group>
</trans-unit>
<trans-unit id="78870852467682010" datatype="html">
@@ -7415,11 +7415,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">100</context>
<context context-type="linenumber">96</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">358</context>
<context context-type="linenumber">357</context>
</context-group>
</trans-unit>
<trans-unit id="157572966557284263" datatype="html">
@@ -7430,11 +7430,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">106</context>
<context context-type="linenumber">102</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">365</context>
<context context-type="linenumber">364</context>
</context-group>
</trans-unit>
<trans-unit id="883965278435032344" datatype="html">
@@ -7452,7 +7452,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">386</context>
<context context-type="linenumber">385</context>
</context-group>
</trans-unit>
<trans-unit id="3542042671420335679" datatype="html">
@@ -7463,7 +7463,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">386</context>
<context context-type="linenumber">385</context>
</context-group>
</trans-unit>
<trans-unit id="872092479747931526" datatype="html">
@@ -7713,7 +7713,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">73</context>
<context context-type="linenumber">69</context>
</context-group>
</trans-unit>
<trans-unit id="2336375155355449543" datatype="html">
@@ -7767,7 +7767,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">216</context>
<context context-type="linenumber">215</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
@@ -8739,28 +8739,28 @@
<source>Custom fields updated.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1025</context>
<context context-type="linenumber">1024</context>
</context-group>
</trans-unit>
<trans-unit id="3873496751167944011" datatype="html">
<source>Error updating custom fields.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1034</context>
<context context-type="linenumber">1033</context>
</context-group>
</trans-unit>
<trans-unit id="6144801143088984138" datatype="html">
<source>Share link bundle creation requested.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1082</context>
<context context-type="linenumber">1073</context>
</context-group>
</trans-unit>
<trans-unit id="46019676931295023" datatype="html">
<source>Share link bundle creation is not available yet.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1089</context>
<context context-type="linenumber">1080</context>
</context-group>
</trans-unit>
<trans-unit id="6307402210351946694" datatype="html">
@@ -8785,108 +8785,89 @@
<context context-type="linenumber">73,78</context>
</context-group>
</trans-unit>
<trans-unit id="7575628227893972164" datatype="html">
<source><x id="INTERPOLATION" equiv-text="document().title | documentTitle }}"/> thumbnail</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">6</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">8</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">6</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">8</context>
</context-group>
</trans-unit>
<trans-unit id="2784168796433474565" datatype="html">
<source>Filter by tag</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">40</context>
<context context-type="linenumber">36</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">334</context>
<context context-type="linenumber">333</context>
</context-group>
</trans-unit>
<trans-unit id="106713086593101376" datatype="html">
<source>View notes</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">95</context>
<context context-type="linenumber">91</context>
</context-group>
</trans-unit>
<trans-unit id="3727324658595204357" datatype="html">
<source>Created: <x id="INTERPOLATION" equiv-text="{{ document().created | customDate }}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">119,120</context>
<context context-type="linenumber">115,116</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">80,81</context>
<context context-type="linenumber">76,77</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">95,96</context>
<context context-type="linenumber">91,92</context>
</context-group>
</trans-unit>
<trans-unit id="2030261243264601523" datatype="html">
<source>Added: <x id="INTERPOLATION" equiv-text="{{ document().added | customDate }}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">120,121</context>
<context context-type="linenumber">116,117</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">81,82</context>
<context context-type="linenumber">77,78</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">96,97</context>
<context context-type="linenumber">92,93</context>
</context-group>
</trans-unit>
<trans-unit id="4235671847487610290" datatype="html">
<source>Modified: <x id="INTERPOLATION" equiv-text="{{ document().modified | customDate }}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">121,122</context>
<context context-type="linenumber">117,118</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">82,83</context>
<context context-type="linenumber">78,79</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">97,98</context>
<context context-type="linenumber">93,94</context>
</context-group>
</trans-unit>
<trans-unit id="197162226430950645" datatype="html">
<source>{VAR_PLURAL, plural, =1 {1 page} other {<x id="INTERPOLATION"/> pages}}</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">138</context>
<context context-type="linenumber">134</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">110</context>
<context context-type="linenumber">106</context>
</context-group>
</trans-unit>
<trans-unit id="5739581984228459958" datatype="html">
<source>Shared</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">148</context>
<context context-type="linenumber">144</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">129</context>
<context context-type="linenumber">125</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -8901,35 +8882,35 @@
<source>Score:</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">153</context>
<context context-type="linenumber">149</context>
</context-group>
</trans-unit>
<trans-unit id="3661756380991326939" datatype="html">
<source>Toggle tag filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">24</context>
<context context-type="linenumber">20</context>
</context-group>
</trans-unit>
<trans-unit id="4648526799630820486" datatype="html">
<source>Toggle correspondent filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">47</context>
<context context-type="linenumber">43</context>
</context-group>
</trans-unit>
<trans-unit id="5319701482646590642" datatype="html">
<source>Toggle document type filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">63</context>
<context context-type="linenumber">59</context>
</context-group>
</trans-unit>
<trans-unit id="8950368321707344185" datatype="html">
<source>Toggle storage path filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">70</context>
<context context-type="linenumber">66</context>
</context-group>
</trans-unit>
<trans-unit id="3797570084942068182" datatype="html">
@@ -9078,14 +9059,14 @@
<source>Sort by ASN</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">193</context>
<context context-type="linenumber">192</context>
</context-group>
</trans-unit>
<trans-unit id="7517688192215738656" datatype="html">
<source>ASN</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">197</context>
<context context-type="linenumber">196</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
@@ -9104,28 +9085,28 @@
<source>Sort by correspondent</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">202</context>
<context context-type="linenumber">201</context>
</context-group>
</trans-unit>
<trans-unit id="2066713941761361709" datatype="html">
<source>Sort by title</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">211</context>
<context context-type="linenumber">210</context>
</context-group>
</trans-unit>
<trans-unit id="6232673011753681091" datatype="html">
<source>Sort by owner</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">224</context>
<context context-type="linenumber">223</context>
</context-group>
</trans-unit>
<trans-unit id="3715596725146409911" datatype="html">
<source>Owner</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">228</context>
<context context-type="linenumber">227</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -9140,49 +9121,49 @@
<source>Sort by notes</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">233</context>
<context context-type="linenumber">232</context>
</context-group>
</trans-unit>
<trans-unit id="5499001829734502606" datatype="html">
<source>Sort by document type</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">242</context>
<context context-type="linenumber">241</context>
</context-group>
</trans-unit>
<trans-unit id="6213829731736042759" datatype="html">
<source>Sort by storage path</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">251</context>
<context context-type="linenumber">250</context>
</context-group>
</trans-unit>
<trans-unit id="3406167410329973166" datatype="html">
<source>Sort by created date</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">260</context>
<context context-type="linenumber">259</context>
</context-group>
</trans-unit>
<trans-unit id="3769035778779263084" datatype="html">
<source>Sort by added date</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">269</context>
<context context-type="linenumber">268</context>
</context-group>
</trans-unit>
<trans-unit id="4874754501044009042" datatype="html">
<source>Sort by number of pages</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">278</context>
<context context-type="linenumber">277</context>
</context-group>
</trans-unit>
<trans-unit id="3817498941817715969" datatype="html">
<source>Pages</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">282</context>
<context context-type="linenumber">281</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -9201,28 +9182,28 @@
<source> Shared </source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">285,287</context>
<context context-type="linenumber">284,286</context>
</context-group>
</trans-unit>
<trans-unit id="5083658411133224968" datatype="html">
<source>Sort by <x id="INTERPOLATION" equiv-text="{{getDisplayCustomFieldTitle(field_id)}}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">292,293</context>
<context context-type="linenumber">291,292</context>
</context-group>
</trans-unit>
<trans-unit id="2179847500064178686" datatype="html">
<source>Edit document</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">326</context>
<context context-type="linenumber">325</context>
</context-group>
</trans-unit>
<trans-unit id="3420321797707163677" datatype="html">
<source>Preview document</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">327</context>
<context context-type="linenumber">326</context>
</context-group>
</trans-unit>
<trans-unit id="4512084577073831437" datatype="html">
@@ -34,28 +34,30 @@
</div>
@if (selectionModel.items) {
<cdk-virtual-scroll-viewport class="items" [itemSize]="FILTERABLE_BUTTON_HEIGHT_PX" #buttonsViewport [style.height.px]="scrollViewportHeight">
<div *cdkVirtualFor="let item of filteredItems; trackBy: trackByItem; let i = index">
<pngx-toggleable-dropdown-button
[item]="item"
[hideCount]="hideCount(item)"
[opacifyCount]="!editing"
[state]="selectionModel.get(item.id)"
[count]="getUpdatedDocumentCount(item.id)"
(toggled)="selectionModel.toggle(item.id)"
(exclude)="excludeClicked(item.id)"
[disabled]="disabled">
</pngx-toggleable-dropdown-button>
<div *cdkVirtualFor="let item of selectionModel.items | filter: filterText:'name'; trackBy: trackByItem; let i = index">
@if (allowSelectNone || item.id) {
<pngx-toggleable-dropdown-button
[item]="item"
[hideCount]="hideCount(item)"
[opacifyCount]="!editing"
[state]="selectionModel.get(item.id)"
[count]="getUpdatedDocumentCount(item.id)"
(toggled)="selectionModel.toggle(item.id)"
(exclude)="excludeClicked(item.id)"
[disabled]="disabled">
</pngx-toggleable-dropdown-button>
}
</div>
</cdk-virtual-scroll-viewport>
}
@if (editing) {
@if (filteredItems.length === 0 && createRef !== undefined) {
@if ((selectionModel.items | filter: filterText:'name').length === 0 && createRef !== undefined) {
<button class="list-group-item list-group-item-action bg-light" (click)="createClicked()" [disabled]="disabled">
<small class="ms-2"><ng-container i18n>Create</ng-container> "{{filterText}}"</small>
<i-bs width="1.5em" height="1em" name="plus"></i-bs>
</button>
}
@if (filteredItems.length > 0) {
@if ((selectionModel.items | filter: filterText:'name').length > 0) {
<button class="list-group-item list-group-item-action bg-light d-flex align-items-center" (click)="applyClicked()" [disabled]="!modelIsDirty() || disabled">
<small class="ms-2" [ngClass]="{'fw-bold': modelIsDirty()}" i18n>Apply</small>
<i-bs width="1.5em" height="1em" name="arrow-right"></i-bs>
@@ -265,9 +265,7 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
expect(document.activeElement).toEqual(
component.listFilterTextInput.nativeElement
)
expect(component.buttonsViewport.getRenderedRange().end).toEqual(
items.length
) // all selectable items shown
expect(component.buttonsViewport.getRenderedRange().end).toEqual(3) // all items shown
component.filterText = 'Tag2'
fixture.detectChanges()
@@ -280,29 +278,6 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
expect(component.filterText).toHaveLength(0)
})
it('should omit disallowed null items from the virtual scroll viewport', async () => {
component.selectionModel.items = items
component.icon = 'tag-fill'
fixture.nativeElement
.querySelector('button')
.dispatchEvent(new MouseEvent('click')) // open
fixture.detectChanges()
await wait(100)
fixture.detectChanges()
component.buttonsViewport?.checkViewportSize()
fixture.detectChanges()
expect(component.filteredItems).toEqual(items)
expect(component.scrollViewportHeight).toEqual(
items.length * component.FILTERABLE_BUTTON_HEIGHT_PX
)
expect(
component.buttonsViewport.elementRef.nativeElement.querySelectorAll(
'.cdk-virtual-scroll-content-wrapper > div'
)
).toHaveLength(items.length)
})
it('should toggle & close on enter inside filter field if 1 item remains', async () => {
component.selectionModel.items = items
component.icon = 'tag-fill'
@@ -661,6 +661,7 @@ export class FilterableDropdownSelectionModel {
imports: [
ClearableBadgeComponent,
ToggleableDropdownButtonComponent,
FilterPipe,
FormsModule,
ReactiveFormsModule,
NgxBootstrapIconsModule,
@@ -800,17 +801,12 @@ export class FilterableDropdownComponent
private keyboardIndex: number
public get filteredItems(): MatchingModel[] {
return this.filterPipe
.transform(this.items, this.filterText, 'name')
.filter((item) => this.allowSelectNone || Boolean(item.id))
}
public get scrollViewportHeight(): number {
return Math.min(
this.filteredItems.length * this.FILTERABLE_BUTTON_HEIGHT_PX,
400
)
const filteredLength = this.filterPipe.transform(
this.items,
this.filterText
).length
return Math.min(filteredLength * this.FILTERABLE_BUTTON_HEIGHT_PX, 400)
}
constructor() {
@@ -882,7 +878,7 @@ export class FilterableDropdownComponent
}
listFilterEnter(): void {
const filtered = this.filteredItems
let filtered = this.filterPipe.transform(this.items, this.filterText)
if (filtered.length == 1) {
this.selectionModel.toggle(filtered[0].id)
setTimeout(() => {
@@ -138,18 +138,6 @@ describe('PngxPdfViewerComponent', () => {
expect(applyScaleSpy).toHaveBeenCalled()
})
it('does not reset the viewer when it is already on the requested page', async () => {
await initComponent()
const viewer = (component as any).pdfViewer as PDFViewer
const currentPageSpy = jest.spyOn(viewer, 'currentPageNumber', 'set')
component.page = viewer.currentPageNumber
;(component as any).applyViewerState()
expect(currentPageSpy).not.toHaveBeenCalled()
})
it('dispatches find when search query changes after render', async () => {
await initComponent()
@@ -256,9 +256,7 @@ export class PngxPdfViewerComponent
Math.max(Math.trunc(this.page), 1),
this.pdfViewer.pagesCount
)
if (nextPage !== this.pdfViewer.currentPageNumber) {
this.pdfViewer.currentPageNumber = nextPage
}
this.pdfViewer.currentPageNumber = nextPage
}
if (this.page === this.lastViewerPage) {
this.lastViewerPage = undefined
@@ -103,13 +103,13 @@
class="btn btn-sm btn-outline-primary"
id="dropdownSend"
ngbDropdownToggle
[disabled]="disabled || !canSendSelection"
[disabled]="disabled || !list.hasSelection || list.allSelected"
>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container>
</div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownSend" class="shadow">
<button ngbDropdownItem (click)="createShareLinkBundle()" [disabled]="!canSendSelection">
<button ngbDropdownItem (click)="createShareLinkBundle()" [disabled]="list.allSelected">
<i-bs name="link" class="me-1"></i-bs><ng-container i18n>Create a share link bundle</ng-container>
</button>
<button ngbDropdownItem (click)="manageShareLinkBundles()">
@@ -117,7 +117,7 @@
</button>
<div class="dropdown-divider"></div>
@if (emailEnabled) {
<button ngbDropdownItem (click)="emailSelected()" [disabled]="!canSendSelection">
<button ngbDropdownItem (click)="emailSelected()" [disabled]="list.allSelected">
<i-bs name="envelope" class="me-1"></i-bs><ng-container i18n>Email</ng-container>
</button>
}
@@ -191,6 +191,14 @@ describe('BulkEditorComponent', () => {
})
afterEach(async () => {
// A filter_selection_data request now fires concurrently with every
// non-search reload(), independent of whether a given test flushes or
// even inspects the primary list response. Drain any left unclaimed.
httpTestingController.match(
(request) =>
request.url ===
`${environment.apiBaseUrl}documents/filter_selection_data/`
)
httpTestingController.verify()
})
@@ -208,50 +216,6 @@ describe('BulkEditorComponent', () => {
expect(component.tagSelectionModel.selectionSize()).toEqual(1)
})
it('should allow sending an all-filtered selection that fits on the current page', () => {
jest
.spyOn(documentListViewService, 'hasSelection', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'allSelected', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'selectedCount', 'get')
.mockReturnValue(5)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([1, 2, 3, 4, 5]))
fixture.detectChanges()
expect(component.canSendSelection).toBe(true)
expect(
fixture.debugElement.query(By.css('#dropdownSend')).nativeElement.disabled
).toBe(false)
})
it('should prevent sending an all-filtered selection spanning multiple pages', () => {
jest
.spyOn(documentListViewService, 'hasSelection', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'allSelected', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'selectedCount', 'get')
.mockReturnValue(6)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([1, 2, 3, 4, 5]))
fixture.detectChanges()
expect(component.canSendSelection).toBe(false)
expect(
fixture.debugElement.query(By.css('#dropdownSend')).nativeElement.disabled
).toBe(true)
})
it('should apply selection data to correspondents menu', () => {
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
fixture.detectChanges()
@@ -430,7 +394,7 @@ describe('BulkEditorComponent', () => {
parameters: { add_tags: [101], remove_tags: [] },
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -476,7 +440,7 @@ describe('BulkEditorComponent', () => {
parameters: { add_tags: [101], remove_tags: [] },
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
})
@@ -505,7 +469,7 @@ describe('BulkEditorComponent', () => {
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -596,7 +560,7 @@ describe('BulkEditorComponent', () => {
parameters: { correspondent: 101 },
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -628,7 +592,7 @@ describe('BulkEditorComponent', () => {
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -694,7 +658,7 @@ describe('BulkEditorComponent', () => {
parameters: { document_type: 101 },
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -726,7 +690,7 @@ describe('BulkEditorComponent', () => {
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -792,7 +756,7 @@ describe('BulkEditorComponent', () => {
parameters: { storage_path: 101 },
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -824,7 +788,7 @@ describe('BulkEditorComponent', () => {
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -890,7 +854,7 @@ describe('BulkEditorComponent', () => {
parameters: { add_custom_fields: [101], remove_custom_fields: [102] },
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -922,7 +886,7 @@ describe('BulkEditorComponent', () => {
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1031,7 +995,7 @@ describe('BulkEditorComponent', () => {
documents: [3, 4],
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1124,7 +1088,7 @@ describe('BulkEditorComponent', () => {
documents: [3, 4],
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1159,7 +1123,7 @@ describe('BulkEditorComponent', () => {
source_mode: 'latest_version',
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1200,7 +1164,7 @@ describe('BulkEditorComponent', () => {
metadata_document_id: 3,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1219,7 +1183,7 @@ describe('BulkEditorComponent', () => {
delete_originals: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1240,7 +1204,7 @@ describe('BulkEditorComponent', () => {
archive_fallback: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1343,7 +1307,7 @@ describe('BulkEditorComponent', () => {
},
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1641,7 +1605,6 @@ describe('BulkEditorComponent', () => {
expect(modal.componentInstance.customFields.length).toEqual(2)
expect(modal.componentInstance.fieldsToAddIds).toEqual([1, 2])
expect(modal.componentInstance.selection).toEqual({ documents: [3, 4] })
expect(modal.componentInstance.selectionCount).toEqual(2)
expect(modal.componentInstance.documents).toEqual([3, 4])
modal.componentInstance.failed.emit()
@@ -1652,7 +1615,7 @@ describe('BulkEditorComponent', () => {
expect(toastServiceShowInfoSpy).toHaveBeenCalled()
expect(listReloadSpy).toHaveBeenCalled()
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
@@ -1020,7 +1020,6 @@ export class BulkEditorComponent
)
dialog.selection = this.getSelectionQuery()
dialog.selectionCount = this.getSelectionSize()
dialog.succeeded.subscribe((result) => {
this.toastService.showInfo($localize`Custom fields updated.`)
this.list.reload()
@@ -1041,14 +1040,6 @@ export class BulkEditorComponent
return this.settings.get(SETTINGS_KEYS.EMAIL_ENABLED)
}
public get canSendSelection(): boolean {
return (
this.list.hasSelection &&
(!this.list.allSelected ||
this.list.selectedCount === this.list.selected.size)
)
}
createShareLinkBundle() {
const modal = this.modalService.open(ShareLinkBundleDialogComponent, {
backdrop: 'static',
@@ -1,9 +1,9 @@
<form [formGroup]="form" (ngSubmit)="save()" autocomplete="off">
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title" i8n>{
documentCount,
documents.length,
plural,
=1 {Set custom fields for 1 document} other {Set custom fields for {{documentCount}} documents}
=1 {Set custom fields for 1 document} other {Set custom fields for {{documents.length}} documents}
}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
@@ -42,20 +42,6 @@ describe('CustomFieldsBulkEditDialogComponent', () => {
expect(component.form.contains('2')).toBeTruthy()
})
it('should render the document count for a filtered selection', () => {
component.selection = {
all: true,
filters: { title__icontains: 'invoice' },
}
component.selectionCount = 42
fixture.detectChanges()
expect(component.documents).toEqual([])
expect(
fixture.nativeElement.querySelector('.modal-title').textContent
).toContain('Set custom fields for 42 documents')
})
it('should emit succeeded event and close modal on successful save', () => {
const editSpy = jest
.spyOn(documentService, 'bulkEdit')
@@ -81,14 +81,8 @@ export class CustomFieldsBulkEditDialogComponent {
public selection: DocumentSelectionQuery = { documents: [] }
public selectionCount: number
public get documents(): number[] {
return this.selection.documents ?? []
}
public get documentCount(): number {
return this.selectionCount ?? this.documents.length
return this.selection.documents
}
initForm() {
@@ -2,11 +2,7 @@
<div class="row g-0">
<div class="col-md-2 doc-img-container rounded-start" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit()">
@if (document()) {
@if (priority()) {
<img [ngSrc]="getThumbUrl()" fill priority class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()" alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
} @else {
<img [ngSrc]="getThumbUrl()" fill class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()" alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
}
<img [ngSrc]="getThumbUrl()" fill class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()">
<div class="border-end border-bottom bg-light document-card-check">
<div class="form-check">
@@ -94,15 +94,6 @@ describe('DocumentCardLargeComponent', () => {
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should prioritize the thumbnail when requested', () => {
fixture.componentRef.setInput('priority', true)
fixture.detectChanges()
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('eager')
expect(thumbnail.getAttribute('fetchpriority')).toEqual('high')
})
it('should trim content', () => {
expect(component.contentTrimmed).toHaveLength(503) // includes ...
})
@@ -66,7 +66,6 @@ export class DocumentCardLargeComponent
private documentService = inject(DocumentService)
settingsService = inject(SettingsService)
readonly selected = input(false)
readonly priority = input(false)
readonly displayFields = input<string[]>(
DEFAULT_DISPLAY_FIELDS.map((f) => f.id)
)
@@ -2,11 +2,7 @@
<div class="card h-100 shadow-sm document-card" [class.placeholder-glow]="!document()" [class.card-selected]="selected()" (mouseleave)="mouseLeaveCard()">
<div class="border-bottom doc-img-container rounded-top" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit(this)">
@if (document()) {
@if (priority()) {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill priority alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
} @else {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
}
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill>
<div class="border-end border-bottom bg-light py-1 px-2 document-card-check">
<div class="form-check">
@@ -67,15 +67,6 @@ describe('DocumentCardSmallComponent', () => {
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should prioritize the thumbnail when requested', () => {
fixture.componentRef.setInput('priority', true)
fixture.detectChanges()
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('eager')
expect(thumbnail.getAttribute('fetchpriority')).toEqual('high')
})
it('should display a document, limit tags to 5', () => {
expect(fixture.nativeElement.textContent).toContain('Document 10')
expect(
@@ -66,7 +66,6 @@ export class DocumentCardSmallComponent
private documentService = inject(DocumentService)
settingsService = inject(SettingsService)
readonly selected = input(false)
readonly priority = input(false)
readonly document = input<Document>(undefined)
readonly displayFields = input<string[]>(
DEFAULT_DISPLAY_FIELDS.map((f) => f.id)
@@ -164,10 +164,9 @@
} @else {
@if (list.displayMode === DisplayMode.LARGE_CARDS) {
<div>
@for (d of list.documents; track d.id; let i = $index) {
@for (d of list.documents; track d.id) {
<pngx-document-card-large
[selected]="list.isSelected(d)"
[priority]="i < 2"
(toggleSelected)="toggleSelected(d, $event)"
(dblClickDocument)="openDocumentDetail(d)"
[document]="d"
@@ -399,10 +398,9 @@
}
@if (list.displayMode === DisplayMode.SMALL_CARDS) {
<div class="row row-cols-paperless-cards">
@for (d of list.documents; track d.id; let i = $index) {
@for (d of list.documents; track d.id) {
<pngx-document-card-small class="p-0"
[selected]="list.isSelected(d)"
[priority]="i < 6"
(toggleSelected)="toggleSelected(d, $event)"
(dblClickDocument)="openDocumentDetail(d)"
[document]="d"
@@ -84,6 +84,28 @@ const view: SavedView = {
filter_rules: filterRules,
}
const emptySelectionData = {
selected_correspondents: [],
selected_tags: [],
selected_document_types: [],
selected_storage_paths: [],
selected_custom_fields: [],
}
// A successful (non-search) list response now triggers a separate,
// non-blocking request for filter dropdown counts. Tests that flush a
// successful list response need to also flush this follow-up request.
function flushSelectionDataRequest(
httpTestingController: HttpTestingController,
querySuffix: string = ''
) {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/filter_selection_data/${querySuffix}`
)
expect(req.request.method).toEqual('GET')
req.flush(emptySelectionData)
}
describe('DocumentListViewService', () => {
let httpTestingController: HttpTestingController
let documentListViewService: DocumentListViewService
@@ -105,6 +127,7 @@ describe('DocumentListViewService', () => {
})
sessionStorage.clear()
localStorage.clear()
httpTestingController = TestBed.inject(HttpTestingController)
documentListViewService = TestBed.inject(DocumentListViewService)
settingsService = TestBed.inject(SettingsService)
@@ -114,8 +137,19 @@ describe('DocumentListViewService', () => {
afterEach(() => {
documentListViewService.cancelPending()
// A filter_selection_data request now fires concurrently with every
// non-search reload(), independent of whether the test cares about or
// flushes the primary list response. Drain any that a test didn't
// explicitly claim via flushSelectionDataRequest, so unrelated tests
// don't have to know about this follow-up request to pass verify().
httpTestingController.match(
(request) =>
request.url ===
`${environment.apiBaseUrl}documents/filter_selection_data/`
)
httpTestingController.verify()
sessionStorage.clear()
localStorage.clear()
})
afterAll(() => {
@@ -128,10 +162,11 @@ describe('DocumentListViewService', () => {
expect(documentListViewService.currentPage).toEqual(1)
documentListViewService.reload()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
expect(req.request.method).toEqual('GET')
expect(documentListViewService.isReloading).toBeFalsy()
expect(documentListViewService.activeSavedViewId).toBeNull()
@@ -143,12 +178,12 @@ describe('DocumentListViewService', () => {
it('should handle error on page request out of range', () => {
documentListViewService.currentPage = 50
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=50&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=50&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush([], { status: 404, statusText: 'Unexpected error' })
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
expect(documentListViewService.currentPage).toEqual(1)
@@ -165,21 +200,20 @@ describe('DocumentListViewService', () => {
]
documentListViewService.setFilterRules(filterRulesAny)
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__in=${tags__id__in}`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false&tags__id__in=${tags__id__in}`
)
expect(req.request.method).toEqual('GET')
req.flush(
{ archive_serial_number: 'hello' },
{ status: 404, statusText: 'Unexpected error' }
)
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
expect(req.request.method).toEqual('GET')
// the error is a plain field error (not a page-out-of-range or deleted
// custom-field-sort case), so no automatic retry request is sent here
expect(documentListViewService.error).toBeTruthy()
// reset the list
documentListViewService.setFilterRules([])
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
})
@@ -187,7 +221,7 @@ describe('DocumentListViewService', () => {
documentListViewService.currentPage = 1
documentListViewService.sortField = 'custom_field_999'
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-custom_field_999&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-custom_field_999&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush(
@@ -196,7 +230,7 @@ describe('DocumentListViewService', () => {
)
// resets itself
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
})
@@ -211,7 +245,7 @@ describe('DocumentListViewService', () => {
]
documentListViewService.setFilterRules(filterRulesAny)
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__in=${tags__id__in}`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false&tags__id__in=${tags__id__in}`
)
expect(req.request.method).toEqual('GET')
req.flush('Generic error', { status: 404, statusText: 'Unexpected error' })
@@ -219,7 +253,7 @@ describe('DocumentListViewService', () => {
// reset the list
documentListViewService.setFilterRules([])
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
})
@@ -228,7 +262,7 @@ describe('DocumentListViewService', () => {
expect(documentListViewService.sortReverse).toBeTruthy()
documentListViewService.setSort('added', false)
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=added&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=added&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
expect(documentListViewService.sortField).toEqual('added')
@@ -236,12 +270,12 @@ describe('DocumentListViewService', () => {
documentListViewService.sortField = 'created'
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=created&truncate_content=true&include_selection_data=false`
)
expect(documentListViewService.sortField).toEqual('created')
documentListViewService.sortReverse = true
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
expect(documentListViewService.sortReverse).toBeTruthy()
@@ -284,7 +318,7 @@ describe('DocumentListViewService', () => {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=${page}&page_size=${
documentListViewService.pageSize
}&ordering=${reverse ? '-' : ''}${sort}&truncate_content=true&include_selection_data=true`
}&ordering=${reverse ? '-' : ''}${sort}&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
expect(documentListViewService.currentPage).toEqual(page)
@@ -301,7 +335,7 @@ describe('DocumentListViewService', () => {
}
documentListViewService.loadFromQueryParams(convertToParamMap(params))
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&include_selection_data=false&tags__id__all=${tags__id__all}`
)
expect(req.request.method).toEqual('GET')
expect(documentListViewService.filterRules).toEqual([
@@ -311,12 +345,16 @@ describe('DocumentListViewService', () => {
},
])
req.flush(full_results)
flushSelectionDataRequest(
httpTestingController,
`?tags__id__all=${tags__id__all}`
)
})
it('should use filter rules to update query params', () => {
documentListViewService.setFilterRules(filterRules)
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&include_selection_data=false&tags__id__all=${tags__id__all}`
)
expect(req.request.method).toEqual('GET')
})
@@ -325,26 +363,31 @@ describe('DocumentListViewService', () => {
documentListViewService.currentPage = 2
let req = httpTestingController.expectOne((request) =>
request.urlWithParams.startsWith(
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
)
expect(req.request.method).toEqual('GET')
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.setFilterRules(filterRules, true)
const filteredReqs = httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false&tags__id__all=${tags__id__all}`
)
expect(filteredReqs).toHaveLength(1)
filteredReqs[0].flush(full_results)
flushSelectionDataRequest(
httpTestingController,
`?tags__id__all=${tags__id__all}`
)
expect(documentListViewService.currentPage).toEqual(1)
})
it('should support quick filter', () => {
documentListViewService.quickFilter(filterRules)
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&include_selection_data=false&tags__id__all=${tags__id__all}`
)
expect(req.request.method).toEqual('GET')
})
@@ -367,21 +410,21 @@ describe('DocumentListViewService', () => {
convertToParamMap(params)
)
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=${page}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
`${environment.apiBaseUrl}documents/?page=${page}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&include_selection_data=false&tags__id__all=${tags__id__all}`
)
expect(req.request.method).toEqual('GET')
// reset the list
documentListViewService.currentPage = 1
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&include_selection_data=true&tags__id__all=9`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&include_selection_data=false&tags__id__all=9`
)
documentListViewService.setFilterRules([])
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&include_selection_data=false`
)
documentListViewService.sortField = 'created'
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
documentListViewService.activateSavedView(null)
})
@@ -389,18 +432,22 @@ describe('DocumentListViewService', () => {
it('should support navigating next / previous', () => {
documentListViewService.setFilterRules([])
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(documentListViewService.currentPage).toEqual(1)
documentListViewService.pageSize = 3
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush({
count: 3,
results: documents.slice(0, 3),
})
// two reload()s ran above (setFilterRules, then pageSize), each firing
// its own concurrent filter_selection_data request with an identical
// (unfiltered) URL; this test doesn't assert on selectionData, so let
// afterEach's drain step clean both up rather than disambiguating here.
expect(documentListViewService.hasNext(documents[0].id)).toBeTruthy()
expect(documentListViewService.hasPrevious(documents[0].id)).toBeFalsy()
documentListViewService.getNext(documents[0].id).subscribe((docId) => {
@@ -447,7 +494,7 @@ describe('DocumentListViewService', () => {
expect(documentListViewService.currentPage).toEqual(1)
documentListViewService.pageSize = 3
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=false`
)
jest
.spyOn(documentListViewService, 'getLastPage')
@@ -462,7 +509,7 @@ describe('DocumentListViewService', () => {
expect(reloadSpy).toHaveBeenCalled()
expect(documentListViewService.currentPage).toEqual(2)
const reqs = httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(reqs.length).toBeGreaterThan(0)
})
@@ -497,11 +544,11 @@ describe('DocumentListViewService', () => {
.mockReturnValue(documents)
documentListViewService.currentPage = 2
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
documentListViewService.pageSize = 3
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true&include_selection_data=false`
)
const reloadSpy = jest.spyOn(documentListViewService, 'reload')
documentListViewService.getPrevious(1).subscribe({
@@ -511,7 +558,7 @@ describe('DocumentListViewService', () => {
expect(reloadSpy).toHaveBeenCalled()
expect(documentListViewService.currentPage).toEqual(1)
const reqs = httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(reqs.length).toBeGreaterThan(0)
})
@@ -524,10 +571,11 @@ describe('DocumentListViewService', () => {
it('should support select a document', () => {
documentListViewService.reload()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.toggleSelected(documents[0])
expect(documentListViewService.isSelected(documents[0])).toBeTruthy()
documentListViewService.toggleSelected(documents[0])
@@ -537,10 +585,11 @@ describe('DocumentListViewService', () => {
it('should support select all', () => {
documentListViewService.reload()
const reloadReq = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(reloadReq.request.method).toEqual('GET')
reloadReq.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.selectAll()
expect(documentListViewService.allSelected).toBeTruthy()
@@ -553,13 +602,14 @@ describe('DocumentListViewService', () => {
it('should support select page', () => {
documentListViewService.pageSize = 3
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush({
count: 3,
results: documents.slice(0, 3),
})
flushSelectionDataRequest(httpTestingController)
documentListViewService.selectPage()
expect(documentListViewService.selected.size).toEqual(3)
expect(documentListViewService.isSelected(documents[5])).toBeFalsy()
@@ -568,10 +618,11 @@ describe('DocumentListViewService', () => {
it('should support select range', () => {
documentListViewService.reload()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.toggleSelected(documents[0])
expect(documentListViewService.isSelected(documents[0])).toBeTruthy()
documentListViewService.selectRangeTo(documents[2])
@@ -583,9 +634,10 @@ describe('DocumentListViewService', () => {
it('should clear all-selected mode when toggling a single document', () => {
documentListViewService.reload()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.selectAll()
expect(documentListViewService.allSelected).toBeTruthy()
@@ -599,9 +651,10 @@ describe('DocumentListViewService', () => {
it('should clear all-selected mode when selecting a range', () => {
documentListViewService.reload()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.selectAll()
documentListViewService.toggleSelected(documents[1])
@@ -619,22 +672,24 @@ describe('DocumentListViewService', () => {
it('should support selection range reduction', () => {
documentListViewService.reload()
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(req.request.method).toEqual('GET')
req.flush(full_results)
flushSelectionDataRequest(httpTestingController)
documentListViewService.selectAll()
expect(documentListViewService.selected.size).toEqual(6)
documentListViewService.setFilterRules(filterRules)
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=9`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false&tags__id__all=9`
)
req.flush({
count: 3,
results: documents.slice(0, 3),
})
flushSelectionDataRequest(httpTestingController, '?tags__id__all=9')
expect(documentListViewService.allSelected).toBeTruthy()
expect(documentListViewService.selected.size).toEqual(3)
})
@@ -643,7 +698,7 @@ describe('DocumentListViewService', () => {
const cancelSpy = jest.spyOn(documentListViewService, 'cancelPending')
documentListViewService.reload()
httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=9`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(cancelSpy).toHaveBeenCalled()
})
@@ -662,7 +717,7 @@ describe('DocumentListViewService', () => {
documentListViewService.setFilterRules([])
expect(documentListViewService.sortField).toEqual('created')
httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
})
@@ -689,11 +744,11 @@ describe('DocumentListViewService', () => {
expect(localStorageSpy).toHaveBeenCalled()
// reload triggered
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
documentListViewService.displayFields = null
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
expect(documentListViewService.displayFields).toEqual(
DEFAULT_DISPLAY_FIELDS.filter((f) => f.id !== DisplayField.ADDED).map(
@@ -738,7 +793,7 @@ describe('DocumentListViewService', () => {
it('should generate quick filter URL preserving default state', () => {
documentListViewService.reload()
httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=false`
)
const urlTree = documentListViewService.getQuickFilterUrl(filterRules)
expect(urlTree).toBeDefined()
@@ -314,12 +314,39 @@ export class DocumentListViewService {
}
}
private loadFilterSelectionData(filterRules: FilterRule[]) {
this.documentService
.getFilterSelectionData(filterRules)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (selectionData) => {
this.selectionData = selectionData
this.markChanged()
},
error: () => {
this.selectionData = null
this.markChanged()
},
})
}
reload(onFinish?, updateQueryParams: boolean = true) {
this.cancelPending()
this.isReloading = true
this.error = null
this.markChanged()
let activeListViewState = this.activeListViewState
// Full-text search results are already narrowed by the search backend, so
// computing selection data inline there is cheap. A plain (unfiltered or
// ORM-filtered) browse can span the entire document set, so its selection
// data is fetched separately -- concurrently with the list itself, rather
// than blocking or waiting on it.
const isFullTextSearch = isFullTextFilterRule(
activeListViewState.filterRules
)
if (!isFullTextSearch) {
this.loadFilterSelectionData(activeListViewState.filterRules)
}
this.documentService
.listFiltered(
activeListViewState.currentPage,
@@ -327,17 +354,22 @@ export class DocumentListViewService {
activeListViewState.sortField,
activeListViewState.sortReverse,
activeListViewState.filterRules,
{ truncate_content: true, include_selection_data: true }
{
truncate_content: true,
include_selection_data: isFullTextSearch,
}
)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (result) => {
const resultWithSelectionData = result as DocumentResults
this.initialized = true
this.isReloading = false
activeListViewState.collectionSize = result.count
activeListViewState.documents = result.results
this.selectionData = resultWithSelectionData.selection_data ?? null
if (isFullTextSearch) {
this.selectionData =
(result as DocumentResults).selection_data ?? null
}
this.syncSelectedToCurrentPage()
this.markChanged()
@@ -376,6 +408,9 @@ export class DocumentListViewService {
// e.g. field was deleted
this.sortField = 'created'
} else {
// cancel the concurrently-fired selection-data request too, so it
// can't resolve afterward and clobber this reset with stale data
this.cancelPending()
this.selectionData = null
let errorMessage
if (
@@ -41,6 +41,24 @@ export abstract class AbstractPaperlessService<T extends ObjectWithId> {
}
}
/**
* Merges a plain params object into an HttpParams instance, skipping
* null/undefined values so they aren't serialized as literal "null" /
* "undefined" query string entries.
*/
protected withParams(
params,
base: HttpParams = new HttpParams()
): HttpParams {
let httpParams = base
for (let key in params) {
if (params[key] != null) {
httpParams = httpParams.set(key, params[key])
}
}
return httpParams
}
list(
page?: number,
pageSize?: number,
@@ -60,11 +78,7 @@ export abstract class AbstractPaperlessService<T extends ObjectWithId> {
if (ordering) {
httpParams = httpParams.set('ordering', ordering)
}
for (let extraParamKey in extraParams) {
if (extraParams[extraParamKey] != null) {
httpParams = httpParams.set(extraParamKey, extraParams[extraParamKey])
}
}
httpParams = this.withParams(extraParams, httpParams)
return this.http
.get<Results<T>>(this.getResourceUrl(), {
params: httpParams,
@@ -113,11 +127,7 @@ export abstract class AbstractPaperlessService<T extends ObjectWithId> {
httpParams = httpParams.set('id__in', ids.join(','))
httpParams = httpParams.set('ordering', '-id')
httpParams = httpParams.set('page_size', 1000)
for (let extraParamKey in extraParams) {
if (extraParams[extraParamKey] != null) {
httpParams = httpParams.set(extraParamKey, extraParams[extraParamKey])
}
}
httpParams = this.withParams(extraParams, httpParams)
return this.http
.get<Results<T>>(this.getResourceUrl(), {
params: httpParams,
@@ -398,6 +398,13 @@ export class DocumentService extends AbstractPaperlessService<Document> {
)
}
getFilterSelectionData(filterRules: FilterRule[]): Observable<SelectionData> {
return this.http.get<SelectionData>(
this.getResourceUrl(null, 'filter_selection_data'),
{ params: this.withParams(queryParamsFromFilterRules(filterRules)) }
)
}
getSuggestions(id: number): Observable<DocumentSuggestions> {
return this.http.get<DocumentSuggestions>(
this.getResourceUrl(id, 'suggestions')
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
View File
-335
View File
@@ -1,335 +0,0 @@
from __future__ import annotations
import abc
import hashlib
import json
import os
import shutil
import tempfile
import zipfile
from contextlib import AbstractContextManager
from contextlib import contextmanager
from pathlib import Path
from pathlib import PurePosixPath
from typing import TYPE_CHECKING
from django.conf import settings
from django.core.serializers.json import DjangoJSONEncoder
from documents.file_handling import delete_empty_directories
from documents.utils import compute_checksum
from documents.utils import copy_file_with_basic_stats
if TYPE_CHECKING:
from collections.abc import Iterator
from typing import TextIO
def _dumps(content: list | dict) -> str:
"""Serialize export JSON consistently across all sinks."""
return json.dumps(content, cls=DjangoJSONEncoder, indent=2, ensure_ascii=False)
class StreamingManifestWriter:
"""Incrementally writes a JSON array to a text handle, one record at a time.
Knows nothing about folders or zips: it writes the array framing and records
to whatever handle the sink's ``stream()`` yields. The sink owns the handle's
lifecycle (atomic rename, compare, spooling).
"""
def __init__(self, handle: TextIO) -> None:
self._file = handle
self._first = True
self._file.write("[")
def write_record(self, record: dict) -> None:
if not self._first:
self._file.write(",\n")
else:
self._first = False
self._file.write(_dumps(record))
def write_batch(self, records: list[dict]) -> None:
for record in records:
self.write_record(record)
def close(self) -> None:
"""Write the closing bracket. Does NOT close the handle (the sink owns it)."""
self._file.write("\n]")
class ExportSink(AbstractContextManager, abc.ABC):
"""Destination for a document export.
The command declares export contents via three verbs; the sink decides how to
persist each. ``arcname`` is always a relative POSIX path
(e.g. ``"manifest.json"``, ``"originals/foo.pdf"``).
Contract:
* At most one ``stream()`` open at a time (it is the manifest);
``add_file``/``add_json`` may be called while it is open.
* Context-manager: normal exit finalizes, an exception aborts. No partial or
failed run leaves a complete-looking artifact.
"""
@abc.abstractmethod
def add_file(
self,
source: Path,
arcname: str,
*,
checksum: str | None = None,
) -> None: ...
@abc.abstractmethod
def add_json(self, content: list | dict, arcname: str) -> None: ...
@abc.abstractmethod
def stream(self, arcname: str): # -> contextmanager yielding TextIO
...
def _open(self) -> None:
"""Hook called on context entry. Override as needed."""
@abc.abstractmethod
def _finalize(self) -> None:
"""Commit on clean exit."""
@abc.abstractmethod
def _abort(self) -> None:
"""Roll back on exception."""
def __enter__(self) -> ExportSink:
self._open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
if exc_type is not None:
self._abort()
else:
self._finalize()
class DirectoryExportSink(ExportSink):
"""Writes loose files into a target directory, with incremental sync.
Owns the snapshot/skip/compare/prune machinery that used to live in the
command (``files_in_export_dir``, ``check_and_copy``, ``check_and_write_json``,
and the ``--delete`` pass).
"""
def __init__(
self,
target: Path,
*,
compare_checksums: bool,
compare_json: bool,
delete: bool,
) -> None:
self._target = target.resolve()
self._compare_checksums = compare_checksums
self._compare_json = compare_json
self._delete = delete
self._snapshot: set[Path] = set()
self._stream_open = False
def _open(self) -> None:
for x in self._target.glob("**/*"):
if x.is_file():
self._snapshot.add(x.resolve())
def add_file(
self,
source: Path,
arcname: str,
*,
checksum: str | None = None,
) -> None:
target = (self._target / arcname).resolve()
self._snapshot.discard(target)
perform_copy = False
if target.exists():
source_stat = source.stat()
target_stat = target.stat()
if self._compare_checksums and checksum:
perform_copy = compute_checksum(target) != checksum
elif (
source_stat.st_mtime != target_stat.st_mtime
or source_stat.st_size != target_stat.st_size
):
perform_copy = True
else:
perform_copy = True
if perform_copy:
target.parent.mkdir(parents=True, exist_ok=True)
copy_file_with_basic_stats(source, target)
def add_json(self, content: list | dict, arcname: str) -> None:
target = (self._target / arcname).resolve()
json_str = _dumps(content)
perform_write = True
if target in self._snapshot:
self._snapshot.discard(target)
if self._compare_json:
target_checksum = hashlib.blake2b(target.read_bytes()).hexdigest()
src_checksum = hashlib.blake2b(json_str.encode("utf-8")).hexdigest()
if src_checksum == target_checksum:
perform_write = False
if perform_write:
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(json_str, encoding="utf-8")
@contextmanager
def stream(self, arcname: str) -> Iterator[TextIO]:
if self._stream_open:
raise RuntimeError("A stream is already open on this sink")
target = (self._target / arcname).resolve()
tmp = target.with_suffix(target.suffix + ".tmp")
target.parent.mkdir(parents=True, exist_ok=True)
handle = tmp.open("w", encoding="utf-8")
self._stream_open = True
try:
yield handle
except BaseException:
handle.close()
tmp.unlink(missing_ok=True)
raise
else:
handle.close()
self._commit_streamed_file(target, tmp)
finally:
self._stream_open = False
def _commit_streamed_file(self, target: Path, tmp: Path) -> None:
if target in self._snapshot:
self._snapshot.discard(target)
if self._compare_json:
existing = hashlib.blake2b(target.read_bytes()).hexdigest()
new = hashlib.blake2b(tmp.read_bytes()).hexdigest()
if existing == new:
tmp.unlink()
return
tmp.rename(target)
def _finalize(self) -> None:
if self._delete:
for f in self._snapshot:
f.unlink()
delete_empty_directories(f.parent, self._target)
def _abort(self) -> None:
# Folder mode is in-place/incremental: streamed .tmp files are already
# cleaned in stream(); leave everything else intact and skip the prune.
return None
class ZipExportSink(ExportSink):
"""Writes a single zip archive, produced atomically only on success.
Builds into ``<target>/<zip_name>.zip.tmp`` and renames to ``.zip`` on clean
finalize. The manifest stream is spooled to a temp file in SCRATCH_DIR and
added as an entry at finalize (a zip entry cannot be interleaved with others).
"""
def __init__(self, target: Path, zip_name: str, *, delete: bool = False) -> None:
self._target = target.resolve()
self._zip_path = (self._target / zip_name).with_suffix(".zip")
self._tmp_path = self._zip_path.with_name(self._zip_path.name + ".tmp")
self._delete = delete
self._zip: zipfile.ZipFile | None = None
self._dirs: set[str] = set()
self._pending_manifest: tuple[Path, str] | None = None
self._stream_open = False
def _open(self) -> None:
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
self._zip = zipfile.ZipFile(
self._tmp_path,
"w",
compression=zipfile.ZIP_DEFLATED,
allowZip64=True,
)
def _ensure_dirs(self, arcname: str) -> None:
parts = PurePosixPath(arcname).parts[:-1]
for i in range(len(parts)):
dir_arc = "/".join(parts[: i + 1]) + "/"
if dir_arc not in self._dirs:
self._dirs.add(dir_arc)
assert self._zip is not None
self._zip.mkdir(dir_arc)
def add_file(
self,
source: Path,
arcname: str,
*,
checksum: str | None = None,
) -> None:
assert self._zip is not None
self._ensure_dirs(arcname)
self._zip.write(source, arcname=arcname)
def add_json(self, content: list | dict, arcname: str) -> None:
assert self._zip is not None
self._ensure_dirs(arcname)
self._zip.writestr(arcname, _dumps(content))
@contextmanager
def stream(self, arcname: str) -> Iterator[TextIO]:
if self._stream_open:
raise RuntimeError("A stream is already open on this sink")
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(
dir=settings.SCRATCH_DIR,
prefix="export-manifest-",
suffix=".json",
)
tmp = Path(tmp_name)
handle = os.fdopen(fd, "w", encoding="utf-8")
self._stream_open = True
try:
yield handle
except BaseException:
handle.close()
tmp.unlink(missing_ok=True)
raise
else:
handle.close()
self._pending_manifest = (tmp, arcname)
finally:
self._stream_open = False
def _finalize(self) -> None:
assert self._zip is not None
if self._pending_manifest is not None:
tmp, arcname = self._pending_manifest
self._ensure_dirs(arcname)
self._zip.write(tmp, arcname=arcname)
tmp.unlink(missing_ok=True)
self._pending_manifest = None
self._zip.close()
self._zip = None
if self._delete:
self._wipe_destination()
self._tmp_path.rename(self._zip_path)
def _wipe_destination(self) -> None:
skip = {self._zip_path.resolve(), self._tmp_path.resolve()}
for item in self._target.glob("*"):
if item.resolve() in skip:
continue
if item.is_dir():
shutil.rmtree(item)
else:
item.unlink()
def _abort(self) -> None:
if self._zip is not None:
self._zip.close()
self._zip = None
self._tmp_path.unlink(missing_ok=True)
if self._pending_manifest is not None:
self._pending_manifest[0].unlink(missing_ok=True)
self._pending_manifest = None
-2
View File
@@ -1024,8 +1024,6 @@ class ObjectOwnedOrGrantedPermissionsFilter(ObjectPermissionsFilter):
"""
def filter_queryset(self, request, queryset, view):
if request.user.is_superuser:
return queryset
objects_with_perms = super().filter_queryset(request, queryset, view)
objects_owned = queryset.filter(owner=request.user)
objects_unowned = queryset.filter(owner__isnull=True)
+34 -87
View File
@@ -11,7 +11,6 @@ from enum import StrEnum
from itertools import islice
from typing import TYPE_CHECKING
from typing import Final
from typing import NamedTuple
from typing import Self
from typing import TypedDict
from typing import TypeVar
@@ -22,7 +21,6 @@ import tantivy
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.utils.timezone import get_current_timezone
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
from documents.search._query import build_permission_filter
@@ -47,8 +45,6 @@ if TYPE_CHECKING:
from pathlib import Path
from django.contrib.auth.models import AbstractUser
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from django.db.models import QuerySet
from documents.models import Document
@@ -63,18 +59,6 @@ _LOCK_BACKOFF_CAP: Final[float] = 10.0 # seconds
T = TypeVar("T")
class ViewerGrant(NamedTuple):
"""Direct user and group view grants for a single document.
Named fields (rather than a bare 2-tuple) so ``viewer_ids`` and
``viewer_group_ids`` can't be silently transposed at a call site — both
are ``list[int]``, so a positional swap would type-check cleanly.
"""
viewer_ids: list[int]
viewer_group_ids: list[int]
class SearchMode(StrEnum):
QUERY = "query"
TEXT = "text"
@@ -383,7 +367,7 @@ class TantivyBackend:
) -> tantivy.Query:
"""Wrap a query with a permission filter if the user is not a superuser."""
if user is not None:
permission_filter = self._build_permission_filter(user)
permission_filter = build_permission_filter(self._schema, user)
return tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, query),
@@ -392,21 +376,11 @@ class TantivyBackend:
)
return query
def _build_permission_filter(self, user: AbstractUser) -> tantivy.Query:
"""Build a filter using the user's current group memberships."""
group_ids = user.groups.values_list("pk", flat=True)
return build_permission_filter(
self._schema,
user,
viewer_group_ids=group_ids,
)
def _build_tantivy_doc(
self,
document: Document,
effective_content: str | None = None,
viewer_ids: list[int] | None = None,
viewer_group_ids: list[int] | None = None,
) -> tantivy.Document:
"""Build a tantivy Document from a Django Document instance.
@@ -523,26 +497,10 @@ class TantivyBackend:
users_with_perms = get_users_with_perms(
document,
only_with_perms_in=["view_document"],
with_group_users=False,
)
viewer_ids = list(
cast("QuerySet[User]", users_with_perms).values_list("id", flat=True),
)
viewer_ids = [int(u.id) for u in users_with_perms]
for viewer_id in viewer_ids:
doc.add_unsigned("viewer_id", viewer_id)
if viewer_group_ids is None:
groups_with_perms = get_groups_with_perms(
document,
only_with_perms_in=["view_document"],
)
viewer_group_ids = list(
cast("QuerySet[Group]", groups_with_perms).values_list(
"id",
flat=True,
),
)
for viewer_group_id in viewer_group_ids:
doc.add_unsigned("viewer_group_id", viewer_group_id)
# Autocomplete words
text_sources = [document.title, content]
@@ -855,7 +813,7 @@ class TantivyBackend:
# Intersect with permission filter so autocomplete words from
# invisible documents don't leak to other users.
if user is not None and not user.is_superuser:
permission_query = self._build_permission_filter(user)
permission_query = build_permission_filter(self._schema, user)
matches = searcher.terms_with_prefix(
"autocomplete_word",
@@ -957,7 +915,7 @@ class TantivyBackend:
def rebuild(
self,
documents: QuerySet[Document],
iter_wrapper: IterWrapper[tuple[Document, ViewerGrant]] = identity,
iter_wrapper: IterWrapper[tuple[Document, list[int]]] = identity,
writer_heap_bytes: int = 512_000_000,
) -> None:
"""
@@ -970,8 +928,8 @@ class TantivyBackend:
documents: QuerySet of Document instances to index
iter_wrapper: Optional wrapper function for progress tracking
(e.g., progress bar). Wraps an iterable of
``(document, (viewer_ids, viewer_group_ids))`` pairs and should yield
each unchanged, advancing one step per document.
``(document, viewer_ids)`` pairs and should yield each pair
unchanged, advancing one step per document.
writer_heap_bytes: Tantivy writer memory budget (split across the
writer's threads). Larger values buffer more docs in RAM before
flushing a segment, deferring merge work; they do not avoid it.
@@ -995,14 +953,11 @@ class TantivyBackend:
documents_stream = _DocumentViewerStream(documents, chunk_size=1000)
try:
writer = new_index.writer(heap_size=writer_heap_bytes)
for document, (viewer_ids, viewer_group_ids) in iter_wrapper(
documents_stream,
):
for document, viewer_ids in iter_wrapper(documents_stream):
doc = self._build_tantivy_doc(
document,
document.get_effective_content(),
viewer_ids=viewer_ids,
viewer_group_ids=viewer_group_ids,
)
writer.add_document(doc)
writer.commit()
@@ -1023,26 +978,19 @@ def chunked(iterable, size):
yield chunk
_EMPTY_VIEWER_GRANT: Final[ViewerGrant] = ViewerGrant(
viewer_ids=[],
viewer_group_ids=[],
)
class _DocumentViewerStream:
"""Yield document permission data while batch-loading grants.
"""Yield (document, viewer_ids) pairs while batch-loading viewer ids.
Viewer permissions are fetched in batches (see
``_bulk_get_viewer_permissions``), but documents are yielded individually so a
Viewer permissions are fetched one SQL query per chunk (see
``_bulk_get_viewer_ids``), but documents are yielded individually so a
progress bar wrapped around this stream advances per document rather than
jumping a whole chunk at a time. ``__len__`` lets the progress helper still
discover the total (it inspects ``QuerySet``/``Sized``).
The viewer and group ids travel with each document in the yielded pair
rather than through a separate mutable attribute, so the pairing survives
regardless of how ``iter_wrapper`` consumes the stream (buffering,
batching, etc.) — there is no reliance on the caller advancing this
generator in lock-step.
The viewer ids travel with each document in the yielded pair rather than
through a separate mutable attribute, so the pairing survives regardless
of how ``iter_wrapper`` consumes the stream (buffering, batching, etc.) —
there is no reliance on the caller advancing this generator in lock-step.
"""
def __init__(self, documents: QuerySet[Document], *, chunk_size: int) -> None:
@@ -1052,25 +1000,25 @@ class _DocumentViewerStream:
def __len__(self) -> int:
return self._documents.count()
def __iter__(self) -> Iterator[tuple[Document, ViewerGrant]]:
def __iter__(self) -> Iterator[tuple[Document, list[int]]]:
# iterator(chunk_size=…) streams from a server-side cursor instead of
# materialising the whole queryset in memory; since Django 4.1 it still
# honours prefetch_related, running the prefetches one batch at a time.
documents = self._documents.iterator(chunk_size=self._chunk_size)
for chunk in chunked(documents, self._chunk_size):
grants_by_pk = _bulk_get_viewer_permissions([doc.pk for doc in chunk])
viewer_ids_by_pk = _bulk_get_viewer_ids([doc.pk for doc in chunk])
for doc in chunk:
yield doc, grants_by_pk.get(doc.pk, _EMPTY_VIEWER_GRANT)
yield doc, viewer_ids_by_pk.get(doc.pk, [])
def _bulk_get_viewer_permissions(
doc_pks: Sequence[int],
) -> dict[int, ViewerGrant]:
"""Fetch direct user and group view grants for a batch of documents, keyed by pk.
def _bulk_get_viewer_ids(doc_pks: Sequence[int]) -> dict[int, list[int]]:
"""Fetch all view_document permissions for a batch of documents in one query.
Group grants remain group IDs in the index so permission checks use the
requesting user's current memberships. Expanding groups to user IDs here
would leave stale access behind after a user is removed from a group.
Mirrors get_users_with_perms(doc, only_with_perms_in=["view_document"])
(with_group_users defaults to True there): a user counts as a viewer if
they hold the permission directly, OR via membership in a group that
holds the permission. Missing the group case would silently drop search
access for group-only viewers.
"""
from collections import defaultdict
@@ -1084,7 +1032,6 @@ def _bulk_get_viewer_permissions(
str_pks = [str(pk) for pk in doc_pks]
viewer_map: dict[int, set[int]] = defaultdict(set)
viewer_group_map: dict[int, set[int]] = defaultdict(set)
# Fold the permission lookup into the query via a join on codename instead
# of a separate Permission.objects.get(), which would otherwise run once per
@@ -1098,22 +1045,22 @@ def _bulk_get_viewer_permissions(
for object_pk, user_id in user_qs:
viewer_map[int(object_pk)].add(user_id)
# User.groups has related_query_name="user", so group__user__id joins
# through the group membership m2m to the member users' ids in the same
# single-query fashion as user_qs above (values_list compiles to one SQL
# JOIN; no per-row Python-side lookups follow, so select_related /
# prefetch_related do not apply here).
group_qs = GroupObjectPermission.objects.filter(
content_type=ct,
permission__content_type=ct,
permission__codename="view_document",
object_pk__in=str_pks,
).values_list("object_pk", "group_id")
for object_pk, group_id in group_qs:
viewer_group_map[int(object_pk)].add(group_id)
).values_list("object_pk", "group__user__id")
for object_pk, user_id in group_qs:
if user_id is not None:
viewer_map[int(object_pk)].add(user_id)
return {
object_pk: ViewerGrant(
viewer_ids=list(viewer_map.get(object_pk, ())),
viewer_group_ids=list(viewer_group_map.get(object_pk, ())),
)
for object_pk in viewer_map.keys() | viewer_group_map.keys()
}
return {object_pk: list(user_ids) for object_pk, user_ids in viewer_map.items()}
# Module-level singleton with proper thread safety
+1 -11
View File
@@ -20,7 +20,6 @@ from documents.search._translate import SearchQueryError
from documents.search._translate import translate_query
if TYPE_CHECKING:
from collections.abc import Iterable
from datetime import tzinfo
from django.contrib.auth.base_user import AbstractBaseUser
@@ -115,7 +114,6 @@ def normalize_query(query: str) -> str:
def build_permission_filter(
schema: tantivy.Schema,
user: AbstractBaseUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
@@ -125,12 +123,10 @@ def build_permission_filter(
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
@@ -144,13 +140,7 @@ def build_permission_filter(
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
return tantivy.Query.disjunction_max_query([no_owner, owned, shared])
DEFAULT_SEARCH_FIELDS = [
-1
View File
@@ -102,7 +102,6 @@ def build_schema() -> tantivy.Schema:
"tag_id",
"owner_id",
"viewer_id",
"viewer_group_id",
):
sb.add_unsigned_field(field, stored=False, indexed=True, fast=True)
-289
View File
@@ -1,289 +0,0 @@
import io
import json
import os
import zipfile
from pathlib import Path
import pytest
from documents.export.sinks import DirectoryExportSink
from documents.export.sinks import ExportSink
from documents.export.sinks import StreamingManifestWriter
from documents.export.sinks import ZipExportSink
from documents.export.sinks import _dumps
@pytest.fixture()
def source_file(tmp_path: Path) -> Path:
src: Path = tmp_path / "src" / "doc.pdf"
src.parent.mkdir(parents=True)
src.write_bytes(b"PDF-CONTENT")
return src
class TestDumps:
def test_dumps_is_indented_unicode_json(self) -> None:
result: str = _dumps({"a": "é", "b": 1})
assert '"é"' in result # ensure_ascii=False keeps unicode literal
assert "\n" in result # indent=2 produces newlines
assert json.loads(result) == {"a": "é", "b": 1}
class TestStreamingManifestWriter:
def test_writes_json_array_of_records(self) -> None:
handle: io.StringIO = io.StringIO()
writer: StreamingManifestWriter = StreamingManifestWriter(handle)
writer.write_batch([{"pk": 1}, {"pk": 2}])
writer.write_record({"pk": 3})
writer.close()
assert json.loads(handle.getvalue()) == [{"pk": 1}, {"pk": 2}, {"pk": 3}]
def test_empty_manifest_is_valid_empty_array(self) -> None:
handle: io.StringIO = io.StringIO()
writer: StreamingManifestWriter = StreamingManifestWriter(handle)
writer.close()
assert json.loads(handle.getvalue()) == []
class TestDirectoryExportSink:
def test_add_file_copies_to_relative_arcname(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=False,
) as sink:
sink.add_file(source_file, "originals/doc.pdf")
assert (target / "originals" / "doc.pdf").read_bytes() == b"PDF-CONTENT"
def test_add_json_writes_file(self, tmp_path: Path) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=False,
) as sink:
sink.add_json({"version": "x"}, "metadata.json")
assert json.loads((target / "metadata.json").read_text()) == {"version": "x"}
def test_stream_writes_manifest(self, tmp_path: Path) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=False,
) as sink:
with sink.stream("manifest.json") as handle:
writer: StreamingManifestWriter = StreamingManifestWriter(handle)
writer.write_record({"pk": 1})
writer.close()
assert json.loads((target / "manifest.json").read_text()) == [{"pk": 1}]
def test_add_file_skips_when_size_and_mtime_match(
self,
tmp_path: Path,
source_file: Path,
) -> None:
# Pre-existing target with identical size+mtime but DIFFERENT content:
# if add_file skips (no compare-checksums), the old content survives.
target: Path = tmp_path / "out"
target.mkdir()
existing: Path = target / "originals" / "doc.pdf"
existing.parent.mkdir(parents=True)
# Same byte length as the source but different content + matching mtime,
# so a size/mtime comparison treats it as unchanged and skips the copy.
existing.write_bytes(b"X" * len(b"PDF-CONTENT"))
stat = source_file.stat()
os.utime(existing, (stat.st_atime, stat.st_mtime))
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=False,
) as sink:
sink.add_file(source_file, "originals/doc.pdf", checksum="abc")
assert existing.read_bytes() == b"X" * len(b"PDF-CONTENT") # skipped
def test_add_file_recopies_when_compare_checksums_differ(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
existing: Path = target / "originals" / "doc.pdf"
existing.parent.mkdir(parents=True)
existing.write_bytes(b"X" * len(b"PDF-CONTENT"))
stat = source_file.stat()
os.utime(existing, (stat.st_atime, stat.st_mtime))
with DirectoryExportSink(
target,
compare_checksums=True,
compare_json=False,
delete=False,
) as sink:
# wrong checksum forces recopy despite matching size/mtime
sink.add_file(source_file, "originals/doc.pdf", checksum="not-the-real-sum")
assert existing.read_bytes() == b"PDF-CONTENT" # recopied
def test_delete_prunes_unwritten_snapshot_files(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
stale: Path = target / "stale.pdf"
stale.write_bytes(b"STALE")
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=True,
) as sink:
sink.add_file(source_file, "originals/doc.pdf")
assert not stale.exists()
assert (target / "originals" / "doc.pdf").exists()
def test_no_delete_keeps_unwritten_files(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
stale: Path = target / "stale.pdf"
stale.write_bytes(b"STALE")
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=False,
) as sink:
sink.add_file(source_file, "originals/doc.pdf")
assert stale.exists()
class TestZipExportSink:
def test_round_trip_files_json_and_stream(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with ZipExportSink(target, "export", delete=False) as sink:
sink.add_file(source_file, "originals/doc.pdf")
sink.add_json({"version": "x"}, "metadata.json")
with sink.stream("manifest.json") as handle:
writer = StreamingManifestWriter(handle)
writer.write_record({"pk": 1})
writer.close()
zip_path: Path = target / "export.zip"
assert zip_path.exists()
assert not (target / "export.zip.tmp").exists()
with zipfile.ZipFile(zip_path) as zf:
names = set(zf.namelist())
assert {"originals/doc.pdf", "metadata.json", "manifest.json"} <= names
assert zf.read("originals/doc.pdf") == b"PDF-CONTENT"
assert json.loads(zf.read("manifest.json")) == [{"pk": 1}]
def test_nested_arcname_emits_directory_marker(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with ZipExportSink(target, "export", delete=False) as sink:
sink.add_file(source_file, "originals/doc.pdf")
with zipfile.ZipFile(target / "export.zip") as zf:
assert "originals/" in zf.namelist()
def test_flat_arcname_has_no_directory_markers(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with ZipExportSink(target, "export", delete=False) as sink:
sink.add_file(source_file, "doc.pdf")
with zipfile.ZipFile(target / "export.zip") as zf:
assert all(not n.endswith("/") for n in zf.namelist())
def test_exception_leaves_no_zip_and_no_tmp(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
with pytest.raises(RuntimeError):
with ZipExportSink(target, "export", delete=False) as sink:
sink.add_file(source_file, "doc.pdf")
raise RuntimeError("boom")
assert not (target / "export.zip").exists()
assert not (target / "export.zip.tmp").exists()
def test_delete_wipes_destination_on_success(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
(target / "preexisting.txt").write_text("old")
(target / "olddir").mkdir()
with ZipExportSink(target, "export", delete=True) as sink:
sink.add_file(source_file, "doc.pdf")
assert (target / "export.zip").exists()
assert not (target / "preexisting.txt").exists()
assert not (target / "olddir").exists()
def test_abort_with_delete_does_not_wipe_destination(
self,
tmp_path: Path,
source_file: Path,
) -> None:
target: Path = tmp_path / "out"
target.mkdir()
(target / "preexisting.txt").write_text("old")
with pytest.raises(RuntimeError):
with ZipExportSink(target, "export", delete=True) as sink:
sink.add_file(source_file, "doc.pdf")
raise RuntimeError("boom")
assert (target / "preexisting.txt").exists()
assert not (target / "export.zip").exists()
class TestStreamContract:
@pytest.fixture(params=["dir", "zip"])
def sink(self, request: pytest.FixtureRequest, tmp_path: Path) -> ExportSink:
target: Path = tmp_path / "out"
target.mkdir()
if request.param == "dir":
return DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=False,
)
return ZipExportSink(target, "export", delete=False)
def test_second_concurrent_stream_is_rejected(self, sink: ExportSink) -> None:
with sink:
with sink.stream("manifest.json"):
with pytest.raises(RuntimeError, match="already open"):
with sink.stream("other.json"):
pass
-92
View File
@@ -1419,30 +1419,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
@mock.patch("documents.views.bulk_edit.merge")
def test_merge_and_delete_requires_change_permission(self, m) -> None:
self.setup_mock(m, "merge")
user = User.objects.create_user(username="no-change")
user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="delete_document"),
)
self.client.force_authenticate(user=user)
response = self.client.post(
"/api/documents/merge/",
json.dumps(
{
"documents": [self.doc2.id, self.doc3.id],
"delete_originals": True,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.merge")
def test_merge_invalid_parameters(self, m) -> None:
self.setup_mock(m, "merge")
@@ -1692,74 +1668,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
@mock.patch("documents.views.bulk_edit.edit_pdf")
def test_edit_pdf_update_requires_change_permission(self, m) -> None:
self.setup_mock(m, "edit_pdf")
user = User.objects.create_user(username="no-change")
self.client.force_authenticate(user=user)
response = self.client.post(
"/api/documents/edit_pdf/",
json.dumps(
{
"documents": [self.doc2.id],
"operations": [{"page": 1}],
"update_document": True,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.remove_password")
@mock.patch("documents.views.bulk_edit.edit_pdf")
def test_delete_original_requires_delete_permission(
self,
edit_pdf_mock,
remove_password_mock,
) -> None:
self.setup_mock(edit_pdf_mock, "edit_pdf")
self.setup_mock(remove_password_mock, "remove_password")
user = User.objects.create_user(username="no-delete")
user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="change_document"),
)
self.client.force_authenticate(user=user)
cases = [
(
"/api/documents/edit_pdf/",
{
"documents": [self.doc2.id],
"operations": [{"page": 1}],
"delete_original": True,
},
edit_pdf_mock,
),
(
"/api/documents/remove_password/",
{
"documents": [self.doc2.id],
"password": "secret",
"delete_original": True,
},
remove_password_mock,
),
]
for endpoint, payload, operation_mock in cases:
with self.subTest(endpoint=endpoint):
response = self.client.post(
endpoint,
json.dumps(payload),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
operation_mock.assert_not_called()
@mock.patch("documents.views.bulk_edit.remove_password")
def test_remove_password(self, m) -> None:
self.setup_mock(m, "remove_password")
@@ -669,26 +669,6 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
def test_update_version_requires_global_change_permission(self) -> None:
user = User.objects.create_user(username="add-only")
user.user_permissions.add(Permission.objects.get(codename="add_document"))
root = Document.objects.create(
title="root",
checksum="root",
mime_type="application/pdf",
)
self.client.force_authenticate(user=user)
with mock.patch("documents.views.consume_file") as consume_mock:
resp = self.client.post(
f"/api/documents/{root.id}/update_version/",
{"document": self._make_pdf_upload()},
format="multipart",
)
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
consume_mock.apply_async.assert_not_called()
def test_update_version_returns_404_for_missing_document(self) -> None:
resp = self.client.post(
"/api/documents/9999/update_version/",
+18 -9
View File
@@ -1241,7 +1241,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
],
)
def test_list_with_include_selection_data(self) -> None:
def test_selection_data_endpoint(self) -> None:
correspondent = Correspondent.objects.create(name="c1")
doc_type = DocumentType.objects.create(name="dt1")
storage_path = StoragePath.objects.create(name="sp1")
@@ -1259,30 +1259,28 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
non_matching_doc.tags.add(Tag.objects.create(name="other"))
response = self.client.get(
f"/api/documents/?tags__id__in={tag.id}&include_selection_data=true",
f"/api/documents/filter_selection_data/?tags__id__in={tag.id}",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertIn("selection_data", response.data)
self.assertNotIn("results", response.data)
selected_correspondent = next(
item
for item in response.data["selection_data"]["selected_correspondents"]
for item in response.data["selected_correspondents"]
if item["id"] == correspondent.id
)
selected_tag = next(
item
for item in response.data["selection_data"]["selected_tags"]
if item["id"] == tag.id
item for item in response.data["selected_tags"] if item["id"] == tag.id
)
selected_type = next(
item
for item in response.data["selection_data"]["selected_document_types"]
for item in response.data["selected_document_types"]
if item["id"] == doc_type.id
)
selected_storage_path = next(
item
for item in response.data["selection_data"]["selected_storage_paths"]
for item in response.data["selected_storage_paths"]
if item["id"] == storage_path.id
)
@@ -1291,6 +1289,17 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(selected_type["document_count"], 1)
self.assertEqual(selected_storage_path["document_count"], 1)
def test_list_no_longer_supports_include_selection_data(self) -> None:
"""
include_selection_data was never part of a stable release (beta-only,
introduced and removed within the 3.0.0-beta cycle) -- the plain list
endpoint should just ignore the param now rather than compute it inline.
"""
response = self.client.get("/api/documents/?include_selection_data=true")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertNotIn("selection_data", response.data)
def test_statistics(self) -> None:
doc1 = Document.objects.create(
title="none1",
@@ -131,10 +131,6 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.client.get("/api/saved_views/").status_code,
status.HTTP_403_FORBIDDEN,
)
self.assertEqual(
self.client.get("/api/search/autocomplete/?term=test").status_code,
status.HTTP_403_FORBIDDEN,
)
def test_api_sufficient_permissions(self) -> None:
user = User.objects.create_user(username="test")
-25
View File
@@ -832,7 +832,6 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
"""
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1.user_permissions.add(Permission.objects.get(codename="view_document"))
self.client.force_authenticate(user=u1)
@@ -879,30 +878,6 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data, ["applebaum", "apples", "appletini"])
def test_search_autocomplete_group_revocation_is_immediate(self) -> None:
user = User.objects.create_user("group-user")
owner = User.objects.create_user("document-owner")
group = Group.objects.create(name="temporary-viewers")
user.user_permissions.add(Permission.objects.get(codename="view_document"))
user.groups.add(group)
document = Document.objects.create(
title="private",
content="canarysecretautocomplete",
checksum="group-revocation",
owner=owner,
)
assign_perm("view_document", group, document)
get_backend().add_or_update(document)
self.client.force_authenticate(user=user)
response = self.client.get("/api/search/autocomplete/?term=canarysecret")
self.assertEqual(response.data, ["canarysecretautocomplete"])
user.groups.remove(group)
response = self.client.get("/api/search/autocomplete/?term=canarysecret")
self.assertEqual(response.data, [])
def test_search_autocomplete_field_name_match(self) -> None:
"""
GIVEN:
+32 -45
View File
@@ -1034,29 +1034,27 @@ class DocumentViewSet(
],
}
def get_queryset(self):
def _base_document_queryset(self):
# Root documents only, with the annotations that filter_backends rely
# on (effective_content for SearchFilter, num_notes for ordering) --
# but no select_related/prefetch_related, since those only matter for
# serializing documents, not for filtering, ordering, or aggregating.
latest_version_content = Subquery(
Document.objects.filter(root_document=OuterRef("pk"))
.order_by("-id")
.values("content")[:1],
)
# A correlated subquery avoids the LEFT JOIN + Count() this used to
# be, which forced a GROUP BY aggregate over every matching document
# before the query could even be sorted or limited.
note_count = Subquery(
Note.objects.filter(document=OuterRef("pk"))
.order_by()
.values("document")
.annotate(count=Count("pk"))
.values("count"),
output_field=IntegerField(),
)
return (
Document.objects.filter(root_document__isnull=True)
.distinct()
.order_by("-created", "-id")
.annotate(effective_content=Coalesce(latest_version_content, F("content")))
.annotate(num_notes=Coalesce(note_count, 0))
.annotate(num_notes=Count("notes"))
)
def get_queryset(self):
return (
self._base_document_queryset()
.select_related("correspondent", "storage_path", "document_type", "owner")
.prefetch_related(
Prefetch(
@@ -1195,24 +1193,21 @@ class DocumentViewSet(
return response
def list(self, request, *args, **kwargs):
if not get_boolean(
str(request.query_params.get("include_selection_data", "false")),
):
return super().list(request, *args, **kwargs)
queryset = self.filter_queryset(self.get_queryset())
selection_data = self._get_selection_data_for_queryset(queryset)
page = self.paginate_queryset(queryset)
if page is not None:
serializer = self.get_serializer(page, many=True)
response = self.get_paginated_response(serializer.data)
response.data["selection_data"] = selection_data
return response
serializer = self.get_serializer(queryset, many=True)
return Response({"results": serializer.data, "selection_data": selection_data})
@extend_schema(
operation_id="documents_filter_selection_data",
description=(
"Returns per-tag/correspondent/document-type/storage-path/custom-field "
"document counts for the current filter, without paginating or "
"serializing the matching documents themselves. Split out from the "
"plain document list so that browsing the (potentially huge) unfiltered "
"document list doesn't pay for this aggregation on every request."
),
responses={200: inline_serializer(name="SelectionData", fields={})},
)
@action(detail=False, methods=["get"], url_path="filter_selection_data")
def filter_selection_data(self, request, *args, **kwargs):
queryset = self.filter_queryset(self._base_document_queryset())
return Response(self._get_selection_data_for_queryset(queryset))
def destroy(self, request, *args, **kwargs):
from documents.search import get_backend
@@ -1947,13 +1942,10 @@ class DocumentViewSet(
"root_document",
).get(pk=pk)
root_doc = get_root_document(request_doc)
if request.user is not None and (
not request.user.has_perm("documents.change_document")
or not has_perms_owner_aware(
request.user,
"change_document",
root_doc,
)
if request.user is not None and not has_perms_owner_aware(
request.user,
"change_document",
root_doc,
):
return HttpResponseForbidden("Insufficient permissions")
except Document.DoesNotExist:
@@ -2759,7 +2751,7 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
)
or (method == bulk_edit.edit_pdf and parameters.get("update_document"))
):
has_perms = has_perms and user_is_owner_of_all_documents
has_perms = user_is_owner_of_all_documents
# check global add permissions for methods that create documents
if (
@@ -2784,11 +2776,6 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
method in [bulk_edit.merge, bulk_edit.split]
and parameters.get("delete_originals")
)
or (
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
and parameters.get("delete_original")
and not parameters.get("update_document")
)
)
and not user.has_perm("documents.delete_document")
):
@@ -3389,7 +3376,7 @@ class SelectionDataView(GenericAPIView[Any]):
),
)
class SearchAutoCompleteView(GenericAPIView[Any]):
permission_classes = (IsAuthenticated, ViewDocumentsPermissions)
permission_classes = (IsAuthenticated,)
def get(self, request, format=None):
user = self.request.user if hasattr(self.request, "user") else None
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More