mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-08-29 22:17:14 +00:00
Compare commits
43
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b89fb0f978 | ||
|
|
535975e2fd | ||
|
|
9c475e0b27 | ||
|
|
7ddc1c9801 | ||
|
|
6f3945f11f | ||
|
|
a784a642ee | ||
|
|
8ded82ea23 | ||
|
|
3a5312ba6f | ||
|
|
00d9bf474a | ||
|
|
8079690f33 | ||
|
|
c6252a0234 | ||
|
|
4a69c47bdd | ||
|
|
d30ee1d620 | ||
|
|
e1aa9997d7 | ||
|
|
05917a04aa | ||
|
|
02b2f55248 | ||
|
|
a2ea3b9e76 | ||
|
|
a298568962 | ||
|
|
f6f37898e8 | ||
|
|
0cfe33720a | ||
|
|
14b17ddf8d | ||
|
|
9009fbb4e9 | ||
|
|
94651d817d | ||
|
|
1630e78aac | ||
|
|
4e79553489 | ||
|
|
f24d58a383 | ||
|
|
d486dd1205 | ||
|
|
406bc1a233 | ||
|
|
808746c28c | ||
|
|
ff6d73ce17 | ||
|
|
b4068d38bc | ||
|
|
139739c2a2 | ||
|
|
cafed919a1 | ||
|
|
549afde1fd | ||
|
|
a8576d827e | ||
|
|
7330c4d9cb | ||
|
|
aad75b4981 | ||
|
|
2e7cd14964 | ||
|
|
5090ffe39e | ||
|
|
35d40b51cb | ||
|
|
78025df405 | ||
|
|
e24db7023a | ||
|
|
f5ddc14588 |
+4
-1
@@ -59,10 +59,13 @@ The following are not generally considered vulnerabilities unless accompanied by
|
|||||||
- large uploads or resource usage that do not bypass documented limits or privileges
|
- large uploads or resource usage that do not bypass documented limits or privileges
|
||||||
- IDOR / access control claims regarding the ability to attach an un-viewable object to a document. This is expected behavior.
|
- IDOR / access control claims regarding the ability to attach an un-viewable object to a document. This is expected behavior.
|
||||||
- claims based solely on the presence of a library, framework feature or code pattern without a working exploit
|
- claims based solely on the presence of a library, framework feature or code pattern without a working exploit
|
||||||
- reports that rely on admin-level access, workflow-editing privileges, shell access, or other high-trust roles unless they demonstrate an unintended privilege boundary bypass
|
- pickle deserialization of internal data from trusted components such as the Redis-compatible broker or Paperless-ngx data directory
|
||||||
|
- users with permission to edit users granting themselves additional privileges; this is expected behavior for that trusted permission
|
||||||
|
- reports that rely on admin-level access, application-configuration access, workflow-editing privileges, shell access, or other high-trust roles unless they demonstrate an unintended privilege boundary bypass
|
||||||
- optional webhook, mail, AI, OCR, or integration behavior described without a product-level vulnerability
|
- optional webhook, mail, AI, OCR, or integration behavior described without a product-level vulnerability
|
||||||
- missing limits or hardening settings presented without concrete impact
|
- missing limits or hardening settings presented without concrete impact
|
||||||
- generic AI or static-analysis output that is not confirmed against the current codebase and a real deployment scenario
|
- generic AI or static-analysis output that is not confirmed against the current codebase and a real deployment scenario
|
||||||
|
- metadata names visible in a document's custom storage path, even when the user cannot access the underlying metadata object; this is expected behavior
|
||||||
- the ability to attach objects that a user cannot access to a document by ID is an intentional design choice, and not considered a vulnerability
|
- the ability to attach objects that a user cannot access to a document by ID is an intentional design choice, and not considered a vulnerability
|
||||||
|
|
||||||
## Transparency
|
## Transparency
|
||||||
|
|||||||
@@ -1,5 +1,220 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## paperless-ngx 3.1.0
|
||||||
|
|
||||||
|
### Features / Enhancements
|
||||||
|
|
||||||
|
- Enhancement: Apply AI suggestions workflow action [@shamoon](https://github.com/shamoon) ([#13639](https://github.com/paperless-ngx/paperless-ngx/pull/13639))
|
||||||
|
- Enhancement: welcome widget visual tweaks [@shamoon](https://github.com/shamoon) ([#13794](https://github.com/paperless-ngx/paperless-ngx/pull/13794))
|
||||||
|
- Enhancement: support using remote OCR engines selectively [@shamoon](https://github.com/shamoon) ([#13633](https://github.com/paperless-ngx/paperless-ngx/pull/13633))
|
||||||
|
- Enhancement: websocket heartbeat [@oktupol](https://github.com/oktupol) ([#13739](https://github.com/paperless-ngx/paperless-ngx/pull/13739))
|
||||||
|
- Enhancement: more v3 ui tweaks [@shamoon](https://github.com/shamoon) ([#13774](https://github.com/paperless-ngx/paperless-ngx/pull/13774))
|
||||||
|
- Tweak: better support long list of views in documents list [@shamoon](https://github.com/shamoon) ([#13769](https://github.com/paperless-ngx/paperless-ngx/pull/13769))
|
||||||
|
- QoL: add count badge to versions dropdown [@shamoon](https://github.com/shamoon) ([#13753](https://github.com/paperless-ngx/paperless-ngx/pull/13753))
|
||||||
|
- Tweakhancement: add jitter to IMAP polling schedule [@shamoon](https://github.com/shamoon) ([#13734](https://github.com/paperless-ngx/paperless-ngx/pull/13734))
|
||||||
|
- Enhancement: sync OIDC groups to superuser and staff roles [@BeSovereign](https://github.com/BeSovereign) ([#13060](https://github.com/paperless-ngx/paperless-ngx/pull/13060))
|
||||||
|
- Enhancement: merge documents as versions [@shamoon](https://github.com/shamoon) ([#13515](https://github.com/paperless-ngx/paperless-ngx/pull/13515))
|
||||||
|
- Tweak: adjust modal proportions for small screens [@shamoon](https://github.com/shamoon) ([#13728](https://github.com/paperless-ngx/paperless-ngx/pull/13728))
|
||||||
|
- Refactor: render paperless\_ai prompts via Jinja2 templates instead of f-strings [@stumpylog](https://github.com/stumpylog) ([#13698](https://github.com/paperless-ngx/paperless-ngx/pull/13698))
|
||||||
|
- Tweak: small visual tweaks / improvements \& fixes [@shamoon](https://github.com/shamoon) ([#13700](https://github.com/paperless-ngx/paperless-ngx/pull/13700))
|
||||||
|
- Enhancement: prefer existing tags, types, correspondents, and storage paths in AI suggestions [@stumpylog](https://github.com/stumpylog) ([#13676](https://github.com/paperless-ngx/paperless-ngx/pull/13676))
|
||||||
|
- Tweak: tweak permissions menu labels for shared user-dependent views [@shamoon](https://github.com/shamoon) ([#13685](https://github.com/paperless-ngx/paperless-ngx/pull/13685))
|
||||||
|
- Feature: Allow selection of compression type and and level during export [@stumpylog](https://github.com/stumpylog) ([#13661](https://github.com/paperless-ngx/paperless-ngx/pull/13661))
|
||||||
|
- Enhancement: customizable icons for saved views [@shamoon](https://github.com/shamoon) ([#13388](https://github.com/paperless-ngx/paperless-ngx/pull/13388))
|
||||||
|
- Enhancement: Add --url argument to document\_fuzzy\_match to improve output [@lukyjay](https://github.com/lukyjay) ([#13123](https://github.com/paperless-ngx/paperless-ngx/pull/13123))
|
||||||
|
- Feature: Updates remote OCR parser to respect the OCR mode setting [@stumpylog](https://github.com/stumpylog) ([#13408](https://github.com/paperless-ngx/paperless-ngx/pull/13408))
|
||||||
|
- Tweak: improve no ML suggestions UX [@shamoon](https://github.com/shamoon) ([#13621](https://github.com/paperless-ngx/paperless-ngx/pull/13621))
|
||||||
|
- Performance: reduce memory and I/O overhead of the document exporter during zip exports [@stumpylog](https://github.com/stumpylog) ([#13490](https://github.com/paperless-ngx/paperless-ngx/pull/13490))
|
||||||
|
- QoL: make name button text on attribute pages selectable [@shamoon](https://github.com/shamoon) ([#13592](https://github.com/paperless-ngx/paperless-ngx/pull/13592))
|
||||||
|
- Performance: More efficient mail fetching [@stumpylog](https://github.com/stumpylog) ([#13432](https://github.com/paperless-ngx/paperless-ngx/pull/13432))
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
- Fix: prevent config autocomplete craziness [@shamoon](https://github.com/shamoon) ([#13808](https://github.com/paperless-ngx/paperless-ngx/pull/13808))
|
||||||
|
- Fix/performance: prevent token reuse in dropdown filtering, also a perf thing [@shamoon](https://github.com/shamoon) ([#13804](https://github.com/paperless-ngx/paperless-ngx/pull/13804))
|
||||||
|
- Fix: exclude version documents from bulk edit "all" [@shamoon](https://github.com/shamoon) ([#13791](https://github.com/paperless-ngx/paperless-ngx/pull/13791))
|
||||||
|
- Fix: fix bottom mobile nav buttons on Android [@shamoon](https://github.com/shamoon) ([#13780](https://github.com/paperless-ngx/paperless-ngx/pull/13780))
|
||||||
|
- Fix: lazy import guardian modules to fix search language setting [@shamoon](https://github.com/shamoon) ([#13768](https://github.com/paperless-ngx/paperless-ngx/pull/13768))
|
||||||
|
- Fix: version indexing fixes [@shamoon](https://github.com/shamoon) ([#13737](https://github.com/paperless-ngx/paperless-ngx/pull/13737))
|
||||||
|
- Fix: append charset to file response for text files [@shamoon](https://github.com/shamoon) ([#13759](https://github.com/paperless-ngx/paperless-ngx/pull/13759))
|
||||||
|
- Chore: pin Apache Tika images to 3.3.1 [@shamoon](https://github.com/shamoon) ([#13758](https://github.com/paperless-ngx/paperless-ngx/pull/13758))
|
||||||
|
- Fix: align bulk edit object perms with document model [@shamoon](https://github.com/shamoon) ([#13757](https://github.com/paperless-ngx/paperless-ngx/pull/13757))
|
||||||
|
- Fix: use selected version for doc detail emailing [@shamoon](https://github.com/shamoon) ([#13738](https://github.com/paperless-ngx/paperless-ngx/pull/13738))
|
||||||
|
- Fix: hide version delete button without global perms [@shamoon](https://github.com/shamoon) ([#13735](https://github.com/paperless-ngx/paperless-ngx/pull/13735))
|
||||||
|
- Fix: dont re-render path template when checking collisions [@shamoon](https://github.com/shamoon) ([#13718](https://github.com/paperless-ngx/paperless-ngx/pull/13718))
|
||||||
|
- Fix: DocumentClassifierSchema bounds [@shamoon](https://github.com/shamoon) ([#13707](https://github.com/paperless-ngx/paperless-ngx/pull/13707))
|
||||||
|
- Fix: remove shadow around attribute pages [@shamoon](https://github.com/shamoon) ([#13696](https://github.com/paperless-ngx/paperless-ngx/pull/13696))
|
||||||
|
- Zen: correct dropdown corner radius visual defect [@shamoon](https://github.com/shamoon) ([#13695](https://github.com/paperless-ngx/paperless-ngx/pull/13695))
|
||||||
|
- Fix: handle Android keyboard popper overlay [@shamoon](https://github.com/shamoon) ([#13694](https://github.com/paperless-ngx/paperless-ngx/pull/13694))
|
||||||
|
- Fix: only show create when there is text, hide set values if no fields in cf bulk edit dropdown [@shamoon](https://github.com/shamoon) ([#13688](https://github.com/paperless-ngx/paperless-ngx/pull/13688))
|
||||||
|
- Fix: reopen a fresh Tantivy index per write to prevent orphaned segment files [@stumpylog](https://github.com/stumpylog) ([#13682](https://github.com/paperless-ngx/paperless-ngx/pull/13682))
|
||||||
|
- Fix: fix validation of workflow title assignment [@maxtruxa](https://github.com/maxtruxa) ([#13659](https://github.com/paperless-ngx/paperless-ngx/pull/13659))
|
||||||
|
- Fix: dont clip search dropdown on mobile [@shamoon](https://github.com/shamoon) ([#13675](https://github.com/paperless-ngx/paperless-ngx/pull/13675))
|
||||||
|
- Fix: include sharelink bundle perms in WebUI [@shamoon](https://github.com/shamoon) ([#13664](https://github.com/paperless-ngx/paperless-ngx/pull/13664))
|
||||||
|
- Fix: add pagination to saved views management page [@shamoon](https://github.com/shamoon) ([#13646](https://github.com/paperless-ngx/paperless-ngx/pull/13646))
|
||||||
|
- Fix: fixes for workflow assign custom field values [@shamoon](https://github.com/shamoon) ([#13630](https://github.com/paperless-ngx/paperless-ngx/pull/13630))
|
||||||
|
- Fix: deny deactivated users in permission filtering and auto-login [@stumpylog](https://github.com/stumpylog) ([#13623](https://github.com/paperless-ngx/paperless-ngx/pull/13623))
|
||||||
|
- Fix: check bulk mail delete permissions for the whole batch up front [@stumpylog](https://github.com/stumpylog) ([#13620](https://github.com/paperless-ngx/paperless-ngx/pull/13620))
|
||||||
|
- Fix: Allow DRF to validate the maximum API key length [@stumpylog](https://github.com/stumpylog) ([#13614](https://github.com/paperless-ngx/paperless-ngx/pull/13614))
|
||||||
|
- Fix: render PDF form values in annotation layer [@shamoon](https://github.com/shamoon) ([#13607](https://github.com/paperless-ngx/paperless-ngx/pull/13607))
|
||||||
|
- QoL: disable name button without perms [@shamoon](https://github.com/shamoon) ([#13606](https://github.com/paperless-ngx/paperless-ngx/pull/13606))
|
||||||
|
- Fix: prevent debounce overwrites in advanced search field, also improve Esc behavior [@shamoon](https://github.com/shamoon) ([#13602](https://github.com/paperless-ngx/paperless-ngx/pull/13602))
|
||||||
|
- Fix: raise ParseError on remote OCR failure instead of silently continuing [@stumpylog](https://github.com/stumpylog) ([#13574](https://github.com/paperless-ngx/paperless-ngx/pull/13574))
|
||||||
|
- Fix: use selected version when creating share links [@shamoon](https://github.com/shamoon) ([#13571](https://github.com/paperless-ngx/paperless-ngx/pull/13571))
|
||||||
|
- Fix: reject bulk edit permissions requests without the correct key [@shamoon](https://github.com/shamoon) ([#13563](https://github.com/paperless-ngx/paperless-ngx/pull/13563))
|
||||||
|
- Fix: correctly serve app logo specified in env [@shamoon](https://github.com/shamoon) ([#13561](https://github.com/paperless-ngx/paperless-ngx/pull/13561))
|
||||||
|
- Fix: prevent workflow passwords field type error [@shamoon](https://github.com/shamoon) ([#13552](https://github.com/paperless-ngx/paperless-ngx/pull/13552))
|
||||||
|
- Fix: correct Firefox print regression [@shamoon](https://github.com/shamoon) ([#13543](https://github.com/paperless-ngx/paperless-ngx/pull/13543))
|
||||||
|
- Fix: hide some saved view operations on management page without permissions [@shamoon](https://github.com/shamoon) ([#13542](https://github.com/paperless-ngx/paperless-ngx/pull/13542))
|
||||||
|
- Fix: disable pdfjs selection rendering [@shamoon](https://github.com/shamoon) ([#13538](https://github.com/paperless-ngx/paperless-ngx/pull/13538))
|
||||||
|
- Fix: hide sidebar drag grips with insufficient permissions [@shamoon](https://github.com/shamoon) ([#13536](https://github.com/paperless-ngx/paperless-ngx/pull/13536))
|
||||||
|
- Fix: don't re-queue a consume-folder file that is already queued and awaiting consumption [@stumpylog](https://github.com/stumpylog) ([#13526](https://github.com/paperless-ngx/paperless-ngx/pull/13526))
|
||||||
|
- Fix: correct multi-search non-adjacent queries [@shamoon](https://github.com/shamoon) ([#13504](https://github.com/paperless-ngx/paperless-ngx/pull/13504))
|
||||||
|
- Fix: Content-Disposition filename normalization [@shamoon](https://github.com/shamoon) ([#13514](https://github.com/paperless-ngx/paperless-ngx/pull/13514))
|
||||||
|
- Fix: prevent duplicated text query with multiple date queries [@shamoon](https://github.com/shamoon) ([#13522](https://github.com/paperless-ngx/paperless-ngx/pull/13522))
|
||||||
|
- Fix: crash filtering document link custom fields with an unset or unrelated field present [@ggouzi](https://github.com/ggouzi) ([#13518](https://github.com/paperless-ngx/paperless-ngx/pull/13518))
|
||||||
|
- Fix: parse unpadded yyyy-mm-dd date input regardless of locale [@Se1foo](https://github.com/Se1foo) ([#13501](https://github.com/paperless-ngx/paperless-ngx/pull/13501))
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
|
||||||
|
- Documentation: clarify default OCR mode changes in v3 [@shamoon](https://github.com/shamoon) ([#13666](https://github.com/paperless-ngx/paperless-ngx/pull/13666))
|
||||||
|
- Fix: fixes for workflow assign custom field values [@shamoon](https://github.com/shamoon) ([#13630](https://github.com/paperless-ngx/paperless-ngx/pull/13630))
|
||||||
|
- Documentation: add wiki links for AI stuff and parser plugins [@shamoon](https://github.com/shamoon) ([#13626](https://github.com/paperless-ngx/paperless-ngx/pull/13626))
|
||||||
|
|
||||||
|
### Maintenance
|
||||||
|
|
||||||
|
- Chore(deps): Bump the actions group across 1 directory with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13481](https://github.com/paperless-ngx/paperless-ngx/pull/13481))
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>23 changes</summary>
|
||||||
|
|
||||||
|
- Chore: Upgrade Docker image to Python 3.14 [@stumpylog](https://github.com/stumpylog) ([#13721](https://github.com/paperless-ngx/paperless-ngx/pull/13721))
|
||||||
|
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13709](https://github.com/paperless-ngx/paperless-ngx/pull/13709))
|
||||||
|
- Chore: update, reorg some npm deps [@shamoon](https://github.com/shamoon) ([#13716](https://github.com/paperless-ngx/paperless-ngx/pull/13716))
|
||||||
|
- Chore: update fpdf2 to 2.8.8 [@shamoon](https://github.com/shamoon) ([#13629](https://github.com/paperless-ngx/paperless-ngx/pull/13629))
|
||||||
|
- Chore: update pnpm, add blockExoticSubdeps [@shamoon](https://github.com/shamoon) ([#13628](https://github.com/paperless-ngx/paperless-ngx/pull/13628))
|
||||||
|
- Chore(deps): Bump h2 from 4.3.0 to 4.4.1 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13593](https://github.com/paperless-ngx/paperless-ngx/pull/13593))
|
||||||
|
- Chore(deps): Bump pdfjs-dist from 6.1.200 to 6.2.108 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13594](https://github.com/paperless-ngx/paperless-ngx/pull/13594))
|
||||||
|
- Chore(deps): Bump cryptography from 48.0.1 to 50.0.0 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13588](https://github.com/paperless-ngx/paperless-ngx/pull/13588))
|
||||||
|
- Chore(deps): Bump the utilities-patch group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13539](https://github.com/paperless-ngx/paperless-ngx/pull/13539))
|
||||||
|
- Chore(deps): Bump the utilities-minor group across 1 directory with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13535](https://github.com/paperless-ngx/paperless-ngx/pull/13535))
|
||||||
|
- Chore(deps): Bump aiohttp from 3.14.1 to 3.14.3 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13537](https://github.com/paperless-ngx/paperless-ngx/pull/13537))
|
||||||
|
- Chore(deps-dev): Bump zensical from 0.0.47 to 0.0.51 in the development group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13533](https://github.com/paperless-ngx/paperless-ngx/pull/13533))
|
||||||
|
- Chore(deps-dev): Bump postcss from 8.5.22 to 8.5.25 in /src/paperless\_mail/templates in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13534](https://github.com/paperless-ngx/paperless-ngx/pull/13534))
|
||||||
|
- Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13532](https://github.com/paperless-ngx/paperless-ngx/pull/13532))
|
||||||
|
- Chore: ruff 0.16 upgrade [@stumpylog](https://github.com/stumpylog) ([#13531](https://github.com/paperless-ngx/paperless-ngx/pull/13531))
|
||||||
|
- Chore(deps): Bump the actions group across 1 directory with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13481](https://github.com/paperless-ngx/paperless-ngx/pull/13481))
|
||||||
|
- docker(deps): bump astral-sh/uv from 0.11.28-python3.12-trixie-slim to 0.11.32-python3.12-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#13472](https://github.com/paperless-ngx/paperless-ngx/pull/13472))
|
||||||
|
- docker-compose(deps): bump nginx from 1.31.2-alpine to 1.31.3-alpine in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#13470](https://github.com/paperless-ngx/paperless-ngx/pull/13470))
|
||||||
|
- docker-compose(deps): Bump greenmail/standalone from 2.1.9 to 2.1.11 in /docker/compose @[dependabot[bot]](https://github.com/apps/dependabot) ([#13469](https://github.com/paperless-ngx/paperless-ngx/pull/13469))
|
||||||
|
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 18 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13476](https://github.com/paperless-ngx/paperless-ngx/pull/13476))
|
||||||
|
- Chore(deps-dev): Bump @playwright/test from 1.61.1 to 1.62.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13480](https://github.com/paperless-ngx/paperless-ngx/pull/13480))
|
||||||
|
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13477](https://github.com/paperless-ngx/paperless-ngx/pull/13477))
|
||||||
|
- Chore(deps-dev): Bump @types/node from 26.1.0 to 26.1.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13479](https://github.com/paperless-ngx/paperless-ngx/pull/13479))
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
### All App Changes
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>91 changes</summary>
|
||||||
|
|
||||||
|
- Fix: prevent config autocomplete craziness [@shamoon](https://github.com/shamoon) ([#13808](https://github.com/paperless-ngx/paperless-ngx/pull/13808))
|
||||||
|
- Fix/performance: prevent token reuse in dropdown filtering, also a perf thing [@shamoon](https://github.com/shamoon) ([#13804](https://github.com/paperless-ngx/paperless-ngx/pull/13804))
|
||||||
|
- Enhancement: Apply AI suggestions workflow action [@shamoon](https://github.com/shamoon) ([#13639](https://github.com/paperless-ngx/paperless-ngx/pull/13639))
|
||||||
|
- Fix: exclude version documents from bulk edit "all" [@shamoon](https://github.com/shamoon) ([#13791](https://github.com/paperless-ngx/paperless-ngx/pull/13791))
|
||||||
|
- Enhancement: welcome widget visual tweaks [@shamoon](https://github.com/shamoon) ([#13794](https://github.com/paperless-ngx/paperless-ngx/pull/13794))
|
||||||
|
- Performance: fetch note authors with prefetch instead of one query each [@shamoon](https://github.com/shamoon) ([#13790](https://github.com/paperless-ngx/paperless-ngx/pull/13790))
|
||||||
|
- Tweak: more misc UI tweaks [@shamoon](https://github.com/shamoon) ([#13783](https://github.com/paperless-ngx/paperless-ngx/pull/13783))
|
||||||
|
- Enhancement: support using remote OCR engines selectively [@shamoon](https://github.com/shamoon) ([#13633](https://github.com/paperless-ngx/paperless-ngx/pull/13633))
|
||||||
|
- Fix: fix bottom mobile nav buttons on Android [@shamoon](https://github.com/shamoon) ([#13780](https://github.com/paperless-ngx/paperless-ngx/pull/13780))
|
||||||
|
- Enhancement: websocket heartbeat [@oktupol](https://github.com/oktupol) ([#13739](https://github.com/paperless-ngx/paperless-ngx/pull/13739))
|
||||||
|
- Fix: lazy import guardian modules to fix search language setting [@shamoon](https://github.com/shamoon) ([#13768](https://github.com/paperless-ngx/paperless-ngx/pull/13768))
|
||||||
|
- Enhancement: more v3 ui tweaks [@shamoon](https://github.com/shamoon) ([#13774](https://github.com/paperless-ngx/paperless-ngx/pull/13774))
|
||||||
|
- Chore: add some missing UI accessibility labels [@shamoon](https://github.com/shamoon) ([#13772](https://github.com/paperless-ngx/paperless-ngx/pull/13772))
|
||||||
|
- Chore: refactor permission checkbox live changes [@shamoon](https://github.com/shamoon) ([#13771](https://github.com/paperless-ngx/paperless-ngx/pull/13771))
|
||||||
|
- Tweak: better support long list of views in documents list [@shamoon](https://github.com/shamoon) ([#13769](https://github.com/paperless-ngx/paperless-ngx/pull/13769))
|
||||||
|
- Fix: version indexing fixes [@shamoon](https://github.com/shamoon) ([#13737](https://github.com/paperless-ngx/paperless-ngx/pull/13737))
|
||||||
|
- Fix: append charset to file response for text files [@shamoon](https://github.com/shamoon) ([#13759](https://github.com/paperless-ngx/paperless-ngx/pull/13759))
|
||||||
|
- Fix: align bulk edit object perms with document model [@shamoon](https://github.com/shamoon) ([#13757](https://github.com/paperless-ngx/paperless-ngx/pull/13757))
|
||||||
|
- QoL: add count badge to versions dropdown [@shamoon](https://github.com/shamoon) ([#13753](https://github.com/paperless-ngx/paperless-ngx/pull/13753))
|
||||||
|
- Tweakhancement: add jitter to IMAP polling schedule [@shamoon](https://github.com/shamoon) ([#13734](https://github.com/paperless-ngx/paperless-ngx/pull/13734))
|
||||||
|
- Fix: use selected version for doc detail emailing [@shamoon](https://github.com/shamoon) ([#13738](https://github.com/paperless-ngx/paperless-ngx/pull/13738))
|
||||||
|
- Fix: hide version delete button without global perms [@shamoon](https://github.com/shamoon) ([#13735](https://github.com/paperless-ngx/paperless-ngx/pull/13735))
|
||||||
|
- Enhancement: sync OIDC groups to superuser and staff roles [@BeSovereign](https://github.com/BeSovereign) ([#13060](https://github.com/paperless-ngx/paperless-ngx/pull/13060))
|
||||||
|
- Enhancement: merge documents as versions [@shamoon](https://github.com/shamoon) ([#13515](https://github.com/paperless-ngx/paperless-ngx/pull/13515))
|
||||||
|
- Tweak: adjust modal proportions for small screens [@shamoon](https://github.com/shamoon) ([#13728](https://github.com/paperless-ngx/paperless-ngx/pull/13728))
|
||||||
|
- Refactor: render paperless\_ai prompts via Jinja2 templates instead of f-strings [@stumpylog](https://github.com/stumpylog) ([#13698](https://github.com/paperless-ngx/paperless-ngx/pull/13698))
|
||||||
|
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13709](https://github.com/paperless-ngx/paperless-ngx/pull/13709))
|
||||||
|
- Fix: dont re-render path template when checking collisions [@shamoon](https://github.com/shamoon) ([#13718](https://github.com/paperless-ngx/paperless-ngx/pull/13718))
|
||||||
|
- Chore: update, reorg some npm deps [@shamoon](https://github.com/shamoon) ([#13716](https://github.com/paperless-ngx/paperless-ngx/pull/13716))
|
||||||
|
- Fix: DocumentClassifierSchema bounds [@shamoon](https://github.com/shamoon) ([#13707](https://github.com/paperless-ngx/paperless-ngx/pull/13707))
|
||||||
|
- Tweak: small visual tweaks / improvements \& fixes [@shamoon](https://github.com/shamoon) ([#13700](https://github.com/paperless-ngx/paperless-ngx/pull/13700))
|
||||||
|
- Fix: remove shadow around attribute pages [@shamoon](https://github.com/shamoon) ([#13696](https://github.com/paperless-ngx/paperless-ngx/pull/13696))
|
||||||
|
- Zen: correct dropdown corner radius visual defect [@shamoon](https://github.com/shamoon) ([#13695](https://github.com/paperless-ngx/paperless-ngx/pull/13695))
|
||||||
|
- Fix: handle Android keyboard popper overlay [@shamoon](https://github.com/shamoon) ([#13694](https://github.com/paperless-ngx/paperless-ngx/pull/13694))
|
||||||
|
- Enhancement: prefer existing tags, types, correspondents, and storage paths in AI suggestions [@stumpylog](https://github.com/stumpylog) ([#13676](https://github.com/paperless-ngx/paperless-ngx/pull/13676))
|
||||||
|
- Fix: only show create when there is text, hide set values if no fields in cf bulk edit dropdown [@shamoon](https://github.com/shamoon) ([#13688](https://github.com/paperless-ngx/paperless-ngx/pull/13688))
|
||||||
|
- Fix: reopen a fresh Tantivy index per write to prevent orphaned segment files [@stumpylog](https://github.com/stumpylog) ([#13682](https://github.com/paperless-ngx/paperless-ngx/pull/13682))
|
||||||
|
- Tweak: tweak permissions menu labels for shared user-dependent views [@shamoon](https://github.com/shamoon) ([#13685](https://github.com/paperless-ngx/paperless-ngx/pull/13685))
|
||||||
|
- Fix: fix validation of workflow title assignment [@maxtruxa](https://github.com/maxtruxa) ([#13659](https://github.com/paperless-ngx/paperless-ngx/pull/13659))
|
||||||
|
- Feature: Allow selection of compression type and and level during export [@stumpylog](https://github.com/stumpylog) ([#13661](https://github.com/paperless-ngx/paperless-ngx/pull/13661))
|
||||||
|
- Fix: dont clip search dropdown on mobile [@shamoon](https://github.com/shamoon) ([#13675](https://github.com/paperless-ngx/paperless-ngx/pull/13675))
|
||||||
|
- Fix: include sharelink bundle perms in WebUI [@shamoon](https://github.com/shamoon) ([#13664](https://github.com/paperless-ngx/paperless-ngx/pull/13664))
|
||||||
|
- Enhancement: customizable icons for saved views [@shamoon](https://github.com/shamoon) ([#13388](https://github.com/paperless-ngx/paperless-ngx/pull/13388))
|
||||||
|
- Enhancement: Add --url argument to document\_fuzzy\_match to improve output [@lukyjay](https://github.com/lukyjay) ([#13123](https://github.com/paperless-ngx/paperless-ngx/pull/13123))
|
||||||
|
- Feature: Updates remote OCR parser to respect the OCR mode setting [@stumpylog](https://github.com/stumpylog) ([#13408](https://github.com/paperless-ngx/paperless-ngx/pull/13408))
|
||||||
|
- Performance: pass document chat queries as a QuerySet instead of a materialized.list [@stumpylog](https://github.com/stumpylog) ([#13638](https://github.com/paperless-ngx/paperless-ngx/pull/13638))
|
||||||
|
- Fix: add pagination to saved views management page [@shamoon](https://github.com/shamoon) ([#13646](https://github.com/paperless-ngx/paperless-ngx/pull/13646))
|
||||||
|
- Fix: fixes for workflow assign custom field values [@shamoon](https://github.com/shamoon) ([#13630](https://github.com/paperless-ngx/paperless-ngx/pull/13630))
|
||||||
|
- Fix: deny deactivated users in permission filtering and auto-login [@stumpylog](https://github.com/stumpylog) ([#13623](https://github.com/paperless-ngx/paperless-ngx/pull/13623))
|
||||||
|
- Chore: update fpdf2 to 2.8.8 [@shamoon](https://github.com/shamoon) ([#13629](https://github.com/paperless-ngx/paperless-ngx/pull/13629))
|
||||||
|
- Chore: update pnpm, add blockExoticSubdeps [@shamoon](https://github.com/shamoon) ([#13628](https://github.com/paperless-ngx/paperless-ngx/pull/13628))
|
||||||
|
- Fix: check bulk mail delete permissions for the whole batch up front [@stumpylog](https://github.com/stumpylog) ([#13620](https://github.com/paperless-ngx/paperless-ngx/pull/13620))
|
||||||
|
- Tweak: improve no ML suggestions UX [@shamoon](https://github.com/shamoon) ([#13621](https://github.com/paperless-ngx/paperless-ngx/pull/13621))
|
||||||
|
- Fix: Allow DRF to validate the maximum API key length [@stumpylog](https://github.com/stumpylog) ([#13614](https://github.com/paperless-ngx/paperless-ngx/pull/13614))
|
||||||
|
- Performance: unify permission-filtering backends, fixes Correspondent/Tag list slowness [@stumpylog](https://github.com/stumpylog) ([#13601](https://github.com/paperless-ngx/paperless-ngx/pull/13601))
|
||||||
|
- Performance: generalize permitted\_document\_ids into permitted\_object\_ids for any model [@stumpylog](https://github.com/stumpylog) ([#13578](https://github.com/paperless-ngx/paperless-ngx/pull/13578))
|
||||||
|
- Fix: render PDF form values in annotation layer [@shamoon](https://github.com/shamoon) ([#13607](https://github.com/paperless-ngx/paperless-ngx/pull/13607))
|
||||||
|
- QoL: disable name button without perms [@shamoon](https://github.com/shamoon) ([#13606](https://github.com/paperless-ngx/paperless-ngx/pull/13606))
|
||||||
|
- Fix: prevent debounce overwrites in advanced search field, also improve Esc behavior [@shamoon](https://github.com/shamoon) ([#13602](https://github.com/paperless-ngx/paperless-ngx/pull/13602))
|
||||||
|
- Performance: reduce memory and I/O overhead of the document exporter during zip exports [@stumpylog](https://github.com/stumpylog) ([#13490](https://github.com/paperless-ngx/paperless-ngx/pull/13490))
|
||||||
|
- Chore(deps): Bump h2 from 4.3.0 to 4.4.1 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13593](https://github.com/paperless-ngx/paperless-ngx/pull/13593))
|
||||||
|
- Chore(deps): Bump pdfjs-dist from 6.1.200 to 6.2.108 in /src-ui in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13594](https://github.com/paperless-ngx/paperless-ngx/pull/13594))
|
||||||
|
- QoL: make name button text on attribute pages selectable [@shamoon](https://github.com/shamoon) ([#13592](https://github.com/paperless-ngx/paperless-ngx/pull/13592))
|
||||||
|
- Fix: raise ParseError on remote OCR failure instead of silently continuing [@stumpylog](https://github.com/stumpylog) ([#13574](https://github.com/paperless-ngx/paperless-ngx/pull/13574))
|
||||||
|
- Fix: use selected version when creating share links [@shamoon](https://github.com/shamoon) ([#13571](https://github.com/paperless-ngx/paperless-ngx/pull/13571))
|
||||||
|
- Chore: specify AI chat refine template [@shamoon](https://github.com/shamoon) ([#13564](https://github.com/paperless-ngx/paperless-ngx/pull/13564))
|
||||||
|
- Fix: reject bulk edit permissions requests without the correct key [@shamoon](https://github.com/shamoon) ([#13563](https://github.com/paperless-ngx/paperless-ngx/pull/13563))
|
||||||
|
- Fix: correctly serve app logo specified in env [@shamoon](https://github.com/shamoon) ([#13561](https://github.com/paperless-ngx/paperless-ngx/pull/13561))
|
||||||
|
- Fix: prevent workflow passwords field type error [@shamoon](https://github.com/shamoon) ([#13552](https://github.com/paperless-ngx/paperless-ngx/pull/13552))
|
||||||
|
- Chore(deps): Bump the utilities-patch group across 1 directory with 6 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13539](https://github.com/paperless-ngx/paperless-ngx/pull/13539))
|
||||||
|
- Chore(deps): Bump the utilities-minor group across 1 directory with 20 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13535](https://github.com/paperless-ngx/paperless-ngx/pull/13535))
|
||||||
|
- Performance: More efficient mail fetching [@stumpylog](https://github.com/stumpylog) ([#13432](https://github.com/paperless-ngx/paperless-ngx/pull/13432))
|
||||||
|
- Performance: eliminate per-document guardian permission-check causing high CPU on document lists [@stumpylog](https://github.com/stumpylog) ([#13505](https://github.com/paperless-ngx/paperless-ngx/pull/13505))
|
||||||
|
- Fix: correct Firefox print regression [@shamoon](https://github.com/shamoon) ([#13543](https://github.com/paperless-ngx/paperless-ngx/pull/13543))
|
||||||
|
- Fix: hide some saved view operations on management page without permissions [@shamoon](https://github.com/shamoon) ([#13542](https://github.com/paperless-ngx/paperless-ngx/pull/13542))
|
||||||
|
- Chore(deps): Bump aiohttp from 3.14.1 to 3.14.3 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13537](https://github.com/paperless-ngx/paperless-ngx/pull/13537))
|
||||||
|
- Chore(deps-dev): Bump zensical from 0.0.47 to 0.0.51 in the development group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13533](https://github.com/paperless-ngx/paperless-ngx/pull/13533))
|
||||||
|
- Fix: disable pdfjs selection rendering [@shamoon](https://github.com/shamoon) ([#13538](https://github.com/paperless-ngx/paperless-ngx/pull/13538))
|
||||||
|
- Fix: hide sidebar drag grips with insufficient permissions [@shamoon](https://github.com/shamoon) ([#13536](https://github.com/paperless-ngx/paperless-ngx/pull/13536))
|
||||||
|
- Chore(deps-dev): Bump postcss from 8.5.22 to 8.5.25 in /src/paperless\_mail/templates in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13534](https://github.com/paperless-ngx/paperless-ngx/pull/13534))
|
||||||
|
- Chore: ruff 0.16 upgrade [@stumpylog](https://github.com/stumpylog) ([#13531](https://github.com/paperless-ngx/paperless-ngx/pull/13531))
|
||||||
|
- Fix: don't re-queue a consume-folder file that is already queued and awaiting consumption [@stumpylog](https://github.com/stumpylog) ([#13526](https://github.com/paperless-ngx/paperless-ngx/pull/13526))
|
||||||
|
- Fix: correct multi-search non-adjacent queries [@shamoon](https://github.com/shamoon) ([#13504](https://github.com/paperless-ngx/paperless-ngx/pull/13504))
|
||||||
|
- Fix: Content-Disposition filename normalization [@shamoon](https://github.com/shamoon) ([#13514](https://github.com/paperless-ngx/paperless-ngx/pull/13514))
|
||||||
|
- Fix: prevent duplicated text query with multiple date queries [@shamoon](https://github.com/shamoon) ([#13522](https://github.com/paperless-ngx/paperless-ngx/pull/13522))
|
||||||
|
- Fix: crash filtering document link custom fields with an unset or unrelated field present [@ggouzi](https://github.com/ggouzi) ([#13518](https://github.com/paperless-ngx/paperless-ngx/pull/13518))
|
||||||
|
- Fix: parse unpadded yyyy-mm-dd date input regardless of locale [@Se1foo](https://github.com/Se1foo) ([#13501](https://github.com/paperless-ngx/paperless-ngx/pull/13501))
|
||||||
|
- Chore(deps): Bump the frontend-angular-dependencies group across 1 directory with 18 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13476](https://github.com/paperless-ngx/paperless-ngx/pull/13476))
|
||||||
|
- Chore(deps-dev): Bump @playwright/test from 1.61.1 to 1.62.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13480](https://github.com/paperless-ngx/paperless-ngx/pull/13480))
|
||||||
|
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13477](https://github.com/paperless-ngx/paperless-ngx/pull/13477))
|
||||||
|
- Chore(deps-dev): Bump @types/node from 26.1.0 to 26.1.1 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#13479](https://github.com/paperless-ngx/paperless-ngx/pull/13479))
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
## paperless-ngx 3.0.5
|
## paperless-ngx 3.0.5
|
||||||
|
|
||||||
### Bug Fixes
|
### Bug Fixes
|
||||||
|
|||||||
@@ -8,6 +8,12 @@ common [OCR](#ocr) related settings and some frontend settings. If set, these wi
|
|||||||
preference over the settings via environment variables. If not set, the environment setting
|
preference over the settings via environment variables. If not set, the environment setting
|
||||||
or applicable default will be utilized instead.
|
or applicable default will be utilized instead.
|
||||||
|
|
||||||
|
!!! warning
|
||||||
|
|
||||||
|
Changing configuration from the UI requires the `AppConfig` permission, which applies
|
||||||
|
instance-wide and should be treated as an admin-level permission. See
|
||||||
|
[global permissions](usage.md#global-permissions).
|
||||||
|
|
||||||
- If you run paperless on docker, `paperless.conf` is not used.
|
- If you run paperless on docker, `paperless.conf` is not used.
|
||||||
Rather, configure paperless by copying necessary options to
|
Rather, configure paperless by copying necessary options to
|
||||||
`docker-compose.env`.
|
`docker-compose.env`.
|
||||||
@@ -1125,6 +1131,10 @@ they use underscores instead of dashes.
|
|||||||
so specifying invalid options may prevent paperless from consuming
|
so specifying invalid options may prevent paperless from consuming
|
||||||
any documents. Use with caution!
|
any documents. Use with caution!
|
||||||
|
|
||||||
|
These arguments are passed directly to OCRmyPDF, so this setting should only
|
||||||
|
be changed by trusted users. This applies to the `AppConfig` permission as well,
|
||||||
|
which allows setting these arguments from the UI.
|
||||||
|
|
||||||
Specify arguments as a JSON dictionary. Keep note of lower case
|
Specify arguments as a JSON dictionary. Keep note of lower case
|
||||||
booleans and double quoted parameter names and strings. Examples:
|
booleans and double quoted parameter names and strings. Examples:
|
||||||
|
|
||||||
|
|||||||
+50
-21
@@ -427,27 +427,27 @@ Global permissions define what areas of the app and API endpoints users can acce
|
|||||||
determine if a user can create, edit, delete or view _any_ documents, but individual documents themselves
|
determine if a user can create, edit, delete or view _any_ documents, but individual documents themselves
|
||||||
still have "object-level" permissions.
|
still have "object-level" permissions.
|
||||||
|
|
||||||
| Type | Details |
|
| Type | Details |
|
||||||
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| AppConfig | _Change_ or higher permissions grants access to the "Application Configuration" area. |
|
| AppConfig | _Change_ or higher permissions grants access to the "Application Configuration" area.<br/>:warning: **This is a trusted, admin-level permission.** Application configuration applies instance-wide and some settings, such as OCR arguments, are passed directly to the underlying processing tools. |
|
||||||
| Correspondent | Add, edit, delete or view Correspondents. |
|
| Correspondent | Add, edit, delete or view Correspondents. |
|
||||||
| CustomField | Add, edit, delete or view Custom Fields. |
|
| CustomField | Add, edit, delete or view Custom Fields. |
|
||||||
| Document | Add, edit, delete or view Documents. |
|
| Document | Add, edit, delete or view Documents. |
|
||||||
| DocumentType | Add, edit, delete or view Document Types. |
|
| DocumentType | Add, edit, delete or view Document Types. |
|
||||||
| Group | Add, edit, delete or view Groups. |
|
| Group | Add, edit, delete or view Groups. |
|
||||||
| GlobalStatistics | View aggregate object counts and statistics. This does not grant access to view individual documents. |
|
| GlobalStatistics | View aggregate object counts and statistics. This does not grant access to view individual documents. |
|
||||||
| MailAccount | Add, edit, delete or view Mail Accounts. |
|
| MailAccount | Add, edit, delete or view Mail Accounts. |
|
||||||
| MailRule | Add, edit, delete or view Mail Rules. |
|
| MailRule | Add, edit, delete or view Mail Rules. |
|
||||||
| Note | Add, edit, delete or view Notes. |
|
| Note | Add, edit, delete or view Notes. |
|
||||||
| PaperlessTask | View or dismiss (_Change_) File Tasks. |
|
| PaperlessTask | View or dismiss (_Change_) File Tasks. |
|
||||||
| SavedView | Add, edit, delete or view Saved Views. |
|
| SavedView | Add, edit, delete or view Saved Views. |
|
||||||
| ShareLink | Add, delete or view Share Links. |
|
| ShareLink | Add, delete or view Share Links. |
|
||||||
| StoragePath | Add, edit, delete or view Storage Paths. |
|
| StoragePath | Add, edit, delete or view Storage Paths. |
|
||||||
| SystemMonitoring | View the system status dialog, tasks summary and their API endpoints. Admin users also retain system status access. |
|
| SystemMonitoring | View the system status dialog, tasks summary and their API endpoints. Admin users also retain system status access. |
|
||||||
| Tag | Add, edit, delete or view Tags. |
|
| Tag | Add, edit, delete or view Tags. |
|
||||||
| UISettings | Add, edit, delete or view the UI settings that are used by the web app.<br/>:warning: **Users that will access the web UI must be granted at least _View_ permissions.** |
|
| UISettings | Add, edit, delete or view the UI settings that are used by the web app.<br/>:warning: **Users that will access the web UI must be granted at least _View_ permissions.** |
|
||||||
| User | Add, edit, delete or view other user accounts via Settings > Users & Groups and `/api/users/`. These permissions are not needed for users to edit their own profile via "My Profile" or `/api/profile/`. |
|
| User | Add, edit, delete or view other user accounts via Settings > Users & Groups and `/api/users/`. These permissions are not needed for users to edit their own profile via "My Profile" or `/api/profile/`. |
|
||||||
| Workflow | Add, edit, delete or view Workflows.<br/>Note that Workflows are global; all users who can access workflows see the same set. Workflows have other permission implications — see [Workflow permissions](#workflow-permissions). |
|
| Workflow | Add, edit, delete or view Workflows.<br/>Note that Workflows are global; all users who can access workflows see the same set. Workflows have other permission implications — see [Workflow permissions](#workflow-permissions). |
|
||||||
|
|
||||||
#### Detailed Explanation of Object Permissions {#object-permissions}
|
#### Detailed Explanation of Object Permissions {#object-permissions}
|
||||||
|
|
||||||
@@ -667,6 +667,35 @@ The action takes no options, its presence is what enables remote OCR for a match
|
|||||||
If the remote engine is not configured, or does not support the document's file type, the document is
|
If the remote engine is not configured, or does not support the document's file type, the document is
|
||||||
processed locally instead and a warning is written to the log.
|
processed locally instead and a warning is written to the log.
|
||||||
|
|
||||||
|
##### Apply AI Suggestions {#workflow-action-apply-ai-suggestions}
|
||||||
|
|
||||||
|
"Apply AI Suggestions" actions ask the configured AI service for title and metadata suggestions,
|
||||||
|
the same as the AI suggestions shown on the document detail page, except applied automatically and in bulk.
|
||||||
|
It requires [AI features](configuration.md#ai) to be enabled. You can specify:
|
||||||
|
|
||||||
|
- Which suggestions to apply: title, tags, correspondent, document type, storage path and / or created
|
||||||
|
date. Suggestions for fields you did not select are discarded.
|
||||||
|
- Whether to create missing items. By default only tags, correspondents and document types that
|
||||||
|
already exist are assigned and any other suggestion is dropped. With this enabled, suggested items
|
||||||
|
that do not exist are created. Storage paths are never created.
|
||||||
|
- Whether to overwrite existing values. By default a field is only filled in if it is currently empty.
|
||||||
|
Note that documents almost always already have a title and created date, so if you select those you
|
||||||
|
will usually want to enable this too. Tags are an exception: suggested tags are always added and
|
||||||
|
never replace the document's existing tags.
|
||||||
|
|
||||||
|
The action works with every trigger **except Consumption Started**, because suggestions are made from
|
||||||
|
the document's text, which does not exist until after the document has been processed.
|
||||||
|
|
||||||
|
Because the query to the AI service is slow, the action is queued and runs in the background rather
|
||||||
|
than as part of the workflow run itself. The document is updated once the suggestions come back.
|
||||||
|
|
||||||
|
!!! warning
|
||||||
|
|
||||||
|
Every matching document results in a query to the AI service, which may incur costs and have privacy
|
||||||
|
implications. Queries can be slow, so a workflow matching a large number of documents can occupy the
|
||||||
|
task queue, and delay consumption of new documents, etc. Consider narrowing the trigger filters,
|
||||||
|
running in small batches and / or increasing workers.
|
||||||
|
|
||||||
#### Workflow placeholders
|
#### Workflow placeholders
|
||||||
|
|
||||||
Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
|
Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "paperless-ngx"
|
name = "paperless-ngx"
|
||||||
version = "3.0.5"
|
version = "3.1.0"
|
||||||
description = "A community-supported supercharged document management system: scan, index and archive all your physical documents"
|
description = "A community-supported supercharged document management system: scan, index and archive all your physical documents"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
|
|||||||
+245
-159
File diff suppressed because it is too large
Load Diff
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "paperless-ngx-ui",
|
"name": "paperless-ngx-ui",
|
||||||
"version": "3.0.5",
|
"version": "3.1.0",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"preinstall": "npx only-allow pnpm",
|
"preinstall": "npx only-allow pnpm",
|
||||||
"ng": "ng",
|
"ng": "ng",
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
infoLink="configuration">
|
infoLink="configuration">
|
||||||
</pngx-page-header>
|
</pngx-page-header>
|
||||||
|
|
||||||
<form [formGroup]="configForm" (ngSubmit)="saveConfig()" class="pb-4">
|
<form [formGroup]="configForm" (ngSubmit)="saveConfig()" class="pb-4" autocomplete="off">
|
||||||
|
|
||||||
<ul ngbNav #nav="ngbNav" class="nav-tabs">
|
<ul ngbNav #nav="ngbNav" class="nav-tabs">
|
||||||
@for (category of optionCategories; track category) {
|
@for (category of optionCategories; track category) {
|
||||||
@@ -44,7 +44,7 @@
|
|||||||
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
|
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
|
||||||
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
|
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
|
||||||
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
|
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
|
||||||
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
|
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]" autocomplete="new-password"></pngx-input-password> }
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
@if (option.note) {
|
@if (option.note) {
|
||||||
|
|||||||
@@ -4,31 +4,29 @@
|
|||||||
(click)="closeMobileSearch(); toggleMenuCollapsed()">
|
(click)="closeMobileSearch(); toggleMenuCollapsed()">
|
||||||
<i-bs width="1.5em" height="1.5em" name="list"></i-bs>
|
<i-bs width="1.5em" height="1.5em" name="list"></i-bs>
|
||||||
</button>
|
</button>
|
||||||
<a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0 order-sm-0"
|
<div class="navbar-brand-container order-sm-0">
|
||||||
|
<a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0"
|
||||||
[ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
|
[ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
|
||||||
routerLink="/dashboard"
|
routerLink="/dashboard"
|
||||||
tourAnchor="tour.intro">
|
tourAnchor="tour.intro">
|
||||||
@if (!hasCustomBranding) {
|
@if (!hasCustomBranding) {
|
||||||
<pngx-logo extra_classes="navbar-official-logo px-1" height="2.4rem"></pngx-logo>
|
<pngx-logo extra_classes="navbar-official-logo px-1" height="2.4rem"></pngx-logo>
|
||||||
<svg class="brand-mark brand-mark-slim d-none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
|
<pngx-brand-mark class="brand-mark brand-mark-slim d-none"></pngx-brand-mark>
|
||||||
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
|
|
||||||
</svg>
|
|
||||||
} @else {
|
|
||||||
@if (customAppLogo) {
|
|
||||||
<img class="brand-logo" [src]="customAppLogo" alt="" />
|
|
||||||
} @else {
|
} @else {
|
||||||
<svg class="brand-mark" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
|
@if (customAppLogo) {
|
||||||
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
|
<img class="brand-logo" [src]="customAppLogo" alt="" />
|
||||||
</svg>
|
} @else {
|
||||||
}
|
<pngx-brand-mark class="brand-mark"></pngx-brand-mark>
|
||||||
<div class="brand-copy ms-2 text-truncate" [class.d-md-none]="slimSidebarEnabled">
|
|
||||||
<span class="brand-title text-truncate">{{ appTitle }}</span>
|
|
||||||
@if (customAppTitle) {
|
|
||||||
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
|
|
||||||
}
|
}
|
||||||
</div>
|
<div class="brand-copy ms-2 text-truncate" [class.d-md-none]="slimSidebarEnabled">
|
||||||
}
|
<span class="brand-title text-truncate">{{ appTitle }}</span>
|
||||||
</a>
|
@if (customAppTitle) {
|
||||||
|
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
|
||||||
|
}
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 me-sm-auto order-3 order-sm-1"
|
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 me-sm-auto order-3 order-sm-1"
|
||||||
[class.mobile-hidden]="mobileSearchHidden()">
|
[class.mobile-hidden]="mobileSearchHidden()">
|
||||||
<div class="col-12 header-search mx-auto">
|
<div class="col-12 header-search mx-auto">
|
||||||
@@ -41,7 +39,7 @@
|
|||||||
}
|
}
|
||||||
<pngx-toasts-dropdown></pngx-toasts-dropdown>
|
<pngx-toasts-dropdown></pngx-toasts-dropdown>
|
||||||
<li ngbDropdown class="nav-item dropdown">
|
<li ngbDropdown class="nav-item dropdown">
|
||||||
<button class="btn navbar-action border-0" id="userDropdown" ngbDropdownToggle aria-label="User menu" i18n-aria-label>
|
<button class="btn navbar-action border-0 me-1 me-md-2" id="userDropdown" ngbDropdownToggle aria-label="User menu" i18n-aria-label>
|
||||||
<i-bs width="1.3em" height="1.3em" name="person-circle"></i-bs>
|
<i-bs width="1.3em" height="1.3em" name="person-circle"></i-bs>
|
||||||
<span class="small ms-2 d-none d-sm-inline">
|
<span class="small ms-2 d-none d-sm-inline">
|
||||||
{{this.settingsService.displayName}}
|
{{this.settingsService.displayName}}
|
||||||
|
|||||||
@@ -109,6 +109,16 @@ main {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@media(min-width: 768px) {
|
@media(min-width: 768px) {
|
||||||
|
// hide scrollbars on browsers that take up layout width
|
||||||
|
// :host-context since <html> is outside the component
|
||||||
|
:host-context(.pngx-classic-scrollbars) .sidebar.slim {
|
||||||
|
scrollbar-width: none;
|
||||||
|
|
||||||
|
&::-webkit-scrollbar {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
.sidebar.slim {
|
.sidebar.slim {
|
||||||
max-width: 55px;
|
max-width: 55px;
|
||||||
|
|
||||||
@@ -125,6 +135,19 @@ main {
|
|||||||
.sidebar-heading span {
|
.sidebar-heading span {
|
||||||
display: none;
|
display: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.nav-link,
|
||||||
|
.nav-anchor {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
padding-left: 0;
|
||||||
|
padding-right: 0;
|
||||||
|
|
||||||
|
i-bs {
|
||||||
|
margin-right: 0 !important;
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
.sidebar.slim:not(.animating) ~ main.col-slim {
|
.sidebar.slim:not(.animating) ~ main.col-slim {
|
||||||
@@ -362,6 +385,7 @@ main {
|
|||||||
}
|
}
|
||||||
|
|
||||||
::ng-deep .navbar-official-logo {
|
::ng-deep .navbar-official-logo {
|
||||||
|
will-change: filter; // Safari repaints the whole navbar on filter change without this
|
||||||
filter: drop-shadow(0 1px 2px rgba(var(--pngx-navbar-brand-shadow-rgb), .3));
|
filter: drop-shadow(0 1px 2px rgba(var(--pngx-navbar-brand-shadow-rgb), .3));
|
||||||
transition: filter .15s ease-in-out;
|
transition: filter .15s ease-in-out;
|
||||||
|
|
||||||
@@ -417,7 +441,7 @@ main {
|
|||||||
|
|
||||||
:host ::ng-deep .navbar-official-logo {
|
:host ::ng-deep .navbar-official-logo {
|
||||||
.leaf {
|
.leaf {
|
||||||
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 70%, var(--bs-primary)) !important;
|
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 85%, var(--bs-primary)) !important;
|
||||||
}
|
}
|
||||||
|
|
||||||
.text {
|
.text {
|
||||||
@@ -436,6 +460,19 @@ main {
|
|||||||
padding-right: .5rem;
|
padding-right: .5rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// true center the search with equal flex widths
|
||||||
|
@media (min-width: 768px) {
|
||||||
|
.navbar-brand-container,
|
||||||
|
.navbar > ul {
|
||||||
|
display: flex; // so the brand link stays as wide as its contents
|
||||||
|
flex: 1 1 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.navbar > ul {
|
||||||
|
justify-content: flex-end;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
:host ::ng-deep .navbar-action {
|
:host ::ng-deep .navbar-action {
|
||||||
display: inline-flex;
|
display: inline-flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
@@ -476,6 +513,7 @@ main {
|
|||||||
text-align: left;
|
text-align: left;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.navbar-brand-container,
|
||||||
.navbar-brand {
|
.navbar-brand {
|
||||||
grid-area: brand;
|
grid-area: brand;
|
||||||
min-width: 0;
|
min-width: 0;
|
||||||
|
|||||||
@@ -543,6 +543,27 @@ describe('AppFrameComponent', () => {
|
|||||||
)
|
)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should only flag scrollbars that take up layout width', () => {
|
||||||
|
const offsetWidth = jest.spyOn(HTMLElement.prototype, 'offsetWidth', 'get')
|
||||||
|
jest.spyOn(HTMLElement.prototype, 'clientWidth', 'get').mockReturnValue(100)
|
||||||
|
|
||||||
|
offsetWidth.mockReturnValue(115)
|
||||||
|
component['detectClassicScrollbars']()
|
||||||
|
expect(
|
||||||
|
window.document.documentElement.classList.contains(
|
||||||
|
'pngx-classic-scrollbars'
|
||||||
|
)
|
||||||
|
).toBeTruthy()
|
||||||
|
|
||||||
|
offsetWidth.mockReturnValue(100)
|
||||||
|
component['detectClassicScrollbars']()
|
||||||
|
expect(
|
||||||
|
window.document.documentElement.classList.contains(
|
||||||
|
'pngx-classic-scrollbars'
|
||||||
|
)
|
||||||
|
).toBeFalsy()
|
||||||
|
})
|
||||||
|
|
||||||
it('should collapse attributes sections when enabling slim sidebar', () => {
|
it('should collapse attributes sections when enabling slim sidebar', () => {
|
||||||
jest.spyOn(settingsService, 'storeSettings').mockReturnValue(of(true))
|
jest.spyOn(settingsService, 'storeSettings').mockReturnValue(of(true))
|
||||||
settingsService.set(SETTINGS_KEYS.ATTRIBUTES_SECTIONS_COLLAPSED, [])
|
settingsService.set(SETTINGS_KEYS.ATTRIBUTES_SECTIONS_COLLAPSED, [])
|
||||||
|
|||||||
@@ -45,6 +45,7 @@ import { TasksService } from 'src/app/services/tasks.service'
|
|||||||
import { ToastService } from 'src/app/services/toast.service'
|
import { ToastService } from 'src/app/services/toast.service'
|
||||||
import { environment } from 'src/environments/environment'
|
import { environment } from 'src/environments/environment'
|
||||||
import { ChatComponent } from '../chat/chat/chat.component'
|
import { ChatComponent } from '../chat/chat/chat.component'
|
||||||
|
import { BrandMarkComponent } from '../common/logo/brand-mark/brand-mark.component'
|
||||||
import { LogoComponent } from '../common/logo/logo.component'
|
import { LogoComponent } from '../common/logo/logo.component'
|
||||||
import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
|
import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
|
||||||
import { DocumentDetailComponent } from '../document-detail/document-detail.component'
|
import { DocumentDetailComponent } from '../document-detail/document-detail.component'
|
||||||
@@ -61,6 +62,7 @@ const SCROLL_THRESHOLD = 16
|
|||||||
imports: [
|
imports: [
|
||||||
GlobalSearchComponent,
|
GlobalSearchComponent,
|
||||||
LogoComponent,
|
LogoComponent,
|
||||||
|
BrandMarkComponent,
|
||||||
DocumentTitlePipe,
|
DocumentTitlePipe,
|
||||||
IfPermissionsDirective,
|
IfPermissionsDirective,
|
||||||
ToastsDropdownComponent,
|
ToastsDropdownComponent,
|
||||||
@@ -116,6 +118,7 @@ export class AppFrameComponent
|
|||||||
|
|
||||||
ngOnInit(): void {
|
ngOnInit(): void {
|
||||||
this.lastScrollY = window.scrollY
|
this.lastScrollY = window.scrollY
|
||||||
|
this.detectClassicScrollbars()
|
||||||
|
|
||||||
if (this.settingsService.get(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)) {
|
if (this.settingsService.get(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)) {
|
||||||
this.checkForUpdates()
|
this.checkForUpdates()
|
||||||
@@ -341,6 +344,22 @@ export class AppFrameComponent
|
|||||||
this.lastScrollY = currentScrollY
|
this.lastScrollY = currentScrollY
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Flag for browsers whose scrollbars take up layout width. Remove me
|
||||||
|
* some day, I hope.
|
||||||
|
*/
|
||||||
|
private detectClassicScrollbars(): void {
|
||||||
|
const probe = document.createElement('div')
|
||||||
|
probe.style.cssText =
|
||||||
|
'position:absolute;top:-9999px;width:100px;height:100px;overflow:scroll'
|
||||||
|
document.body.appendChild(probe)
|
||||||
|
document.documentElement.classList.toggle(
|
||||||
|
'pngx-classic-scrollbars',
|
||||||
|
probe.offsetWidth > probe.clientWidth
|
||||||
|
)
|
||||||
|
probe.remove()
|
||||||
|
}
|
||||||
|
|
||||||
private isMobileViewport(): boolean {
|
private isMobileViewport(): boolean {
|
||||||
return window.innerWidth < 768
|
return window.innerWidth < 768
|
||||||
}
|
}
|
||||||
|
|||||||
+1
@@ -94,6 +94,7 @@ export class MailAccountEditDialogComponent extends EditDialogComponent<MailAcco
|
|||||||
this.testActive = false
|
this.testActive = false
|
||||||
this.testResult.set('danger')
|
this.testResult.set('danger')
|
||||||
this.alertTimeout = setTimeout(() => this.testResultAlert.close(), 5000)
|
this.alertTimeout = setTimeout(() => this.testResultAlert.close(), 5000)
|
||||||
|
this.error = e.error
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
+39
@@ -462,6 +462,45 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
|
@case (WorkflowActionType.ApplyAiSuggestions) {
|
||||||
|
<div class="row">
|
||||||
|
<div class="col">
|
||||||
|
<p class="text-muted small" i18n>The document will be sent to the configured AI service for suggestions. Consider costs and privacy.</p>
|
||||||
|
<pngx-input-select
|
||||||
|
i18n-title
|
||||||
|
title="Apply suggestions for"
|
||||||
|
[items]="aiSuggestionFieldOptions"
|
||||||
|
[multiple]="true"
|
||||||
|
formControlName="ai_suggestion_fields"
|
||||||
|
[error]="error?.actions?.[i]?.ai_suggestion_fields"
|
||||||
|
hint="Suggestions for fields that are not selected are discarded."
|
||||||
|
i18n-hint
|
||||||
|
></pngx-input-select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="row">
|
||||||
|
<div class="col-md-6">
|
||||||
|
<pngx-input-switch
|
||||||
|
[horizontal]="true"
|
||||||
|
i18n-title
|
||||||
|
title="Create missing items"
|
||||||
|
formControlName="ai_create_missing"
|
||||||
|
hint="Create suggested tags, correspondents and document types that do not exist yet."
|
||||||
|
i18n-hint
|
||||||
|
></pngx-input-switch>
|
||||||
|
</div>
|
||||||
|
<div class="col-md-6">
|
||||||
|
<pngx-input-switch
|
||||||
|
[horizontal]="true"
|
||||||
|
i18n-title
|
||||||
|
title="Overwrite existing values"
|
||||||
|
formControlName="ai_overwrite_existing"
|
||||||
|
hint="Apply suggestions even if the document already has a value. Tags are always added, never replaced."
|
||||||
|
i18n-hint
|
||||||
|
></pngx-input-switch>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
</ng-template>
|
</ng-template>
|
||||||
|
|||||||
+124
-2
@@ -22,6 +22,7 @@ import {
|
|||||||
} from 'src/app/data/matching-model'
|
} from 'src/app/data/matching-model'
|
||||||
import { Workflow } from 'src/app/data/workflow'
|
import { Workflow } from 'src/app/data/workflow'
|
||||||
import {
|
import {
|
||||||
|
AISuggestionField,
|
||||||
WorkflowAction,
|
WorkflowAction,
|
||||||
WorkflowActionType,
|
WorkflowActionType,
|
||||||
} from 'src/app/data/workflow-action'
|
} from 'src/app/data/workflow-action'
|
||||||
@@ -49,6 +50,7 @@ import { TagsComponent } from '../../input/tags/tags.component'
|
|||||||
import { TextComponent } from '../../input/text/text.component'
|
import { TextComponent } from '../../input/text/text.component'
|
||||||
import { EditDialogMode } from '../edit-dialog.component'
|
import { EditDialogMode } from '../edit-dialog.component'
|
||||||
import {
|
import {
|
||||||
|
AI_SUGGESTION_FIELD_OPTIONS,
|
||||||
DOCUMENT_SOURCE_OPTIONS,
|
DOCUMENT_SOURCE_OPTIONS,
|
||||||
SCHEDULE_DATE_FIELD_OPTIONS,
|
SCHEDULE_DATE_FIELD_OPTIONS,
|
||||||
TriggerFilterType,
|
TriggerFilterType,
|
||||||
@@ -239,14 +241,15 @@ describe('WorkflowEditDialogComponent', () => {
|
|||||||
SCHEDULE_DATE_FIELD_OPTIONS
|
SCHEDULE_DATE_FIELD_OPTIONS
|
||||||
)
|
)
|
||||||
|
|
||||||
// Email disabled
|
// Email, remote OCR and AI all disabled
|
||||||
jest.spyOn(settingsService, 'get').mockReturnValue(false)
|
jest.spyOn(settingsService, 'get').mockReturnValue(false)
|
||||||
component.ngOnInit()
|
component.ngOnInit()
|
||||||
expect(component.actionTypeOptions).toEqual(
|
expect(component.actionTypeOptions).toEqual(
|
||||||
WORKFLOW_ACTION_OPTIONS.filter(
|
WORKFLOW_ACTION_OPTIONS.filter(
|
||||||
(a) =>
|
(a) =>
|
||||||
a.id !== WorkflowActionType.Email &&
|
a.id !== WorkflowActionType.Email &&
|
||||||
a.id !== WorkflowActionType.RemoteOcr
|
a.id !== WorkflowActionType.RemoteOcr &&
|
||||||
|
a.id !== WorkflowActionType.ApplyAiSuggestions
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
})
|
})
|
||||||
@@ -344,6 +347,125 @@ describe('WorkflowEditDialogComponent', () => {
|
|||||||
)
|
)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should offer apply AI suggestions unless every trigger is consumption', () => {
|
||||||
|
jest.spyOn(settingsService, 'get').mockReturnValue(true)
|
||||||
|
|
||||||
|
// Consumption runs before the document has been parsed, so there would be
|
||||||
|
// no content to make suggestions from
|
||||||
|
component.object = {
|
||||||
|
name: 'Workflow 1',
|
||||||
|
order: 0,
|
||||||
|
enabled: true,
|
||||||
|
triggers: [{ type: WorkflowTriggerType.Consumption }],
|
||||||
|
actions: [],
|
||||||
|
} as Workflow
|
||||||
|
component.ngOnInit()
|
||||||
|
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
|
||||||
|
WorkflowActionType.ApplyAiSuggestions
|
||||||
|
)
|
||||||
|
|
||||||
|
// A second, usable trigger is enough
|
||||||
|
component.object = {
|
||||||
|
name: 'Workflow 2',
|
||||||
|
order: 0,
|
||||||
|
enabled: true,
|
||||||
|
triggers: [
|
||||||
|
{ type: WorkflowTriggerType.Consumption },
|
||||||
|
{ type: WorkflowTriggerType.DocumentAdded },
|
||||||
|
],
|
||||||
|
actions: [],
|
||||||
|
} as Workflow
|
||||||
|
component.ngOnInit()
|
||||||
|
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
|
||||||
|
WorkflowActionType.ApplyAiSuggestions
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('should keep apply AI suggestions listed when an action already uses it', () => {
|
||||||
|
jest.spyOn(settingsService, 'get').mockReturnValue(true)
|
||||||
|
|
||||||
|
// Otherwise changing the trigger would silently blank the selection
|
||||||
|
component.object = {
|
||||||
|
name: 'Workflow 1',
|
||||||
|
order: 0,
|
||||||
|
enabled: true,
|
||||||
|
triggers: [{ type: WorkflowTriggerType.Consumption }],
|
||||||
|
actions: [{ type: WorkflowActionType.ApplyAiSuggestions }],
|
||||||
|
} as Workflow
|
||||||
|
component.ngOnInit()
|
||||||
|
|
||||||
|
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
|
||||||
|
WorkflowActionType.ApplyAiSuggestions
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('should not offer apply AI suggestions when AI is disabled', () => {
|
||||||
|
jest
|
||||||
|
.spyOn(settingsService, 'get')
|
||||||
|
.mockImplementation((key) => key !== SETTINGS_KEYS.AI_ENABLED)
|
||||||
|
|
||||||
|
component.object = {
|
||||||
|
name: 'Workflow 1',
|
||||||
|
order: 0,
|
||||||
|
enabled: true,
|
||||||
|
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
|
||||||
|
actions: [],
|
||||||
|
} as Workflow
|
||||||
|
component.ngOnInit()
|
||||||
|
|
||||||
|
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
|
||||||
|
WorkflowActionType.ApplyAiSuggestions
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('should create form fields for apply AI suggestions options', () => {
|
||||||
|
component.object = {
|
||||||
|
name: 'Workflow 1',
|
||||||
|
order: 0,
|
||||||
|
enabled: true,
|
||||||
|
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
|
||||||
|
actions: [
|
||||||
|
{
|
||||||
|
type: WorkflowActionType.ApplyAiSuggestions,
|
||||||
|
ai_suggestion_fields: [
|
||||||
|
AISuggestionField.Title,
|
||||||
|
AISuggestionField.Tags,
|
||||||
|
],
|
||||||
|
ai_create_missing: true,
|
||||||
|
ai_overwrite_existing: true,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
} as Workflow
|
||||||
|
component.ngOnInit()
|
||||||
|
|
||||||
|
const action = component.actionFields.at(0)
|
||||||
|
expect(action.get('ai_suggestion_fields').value).toEqual([
|
||||||
|
AISuggestionField.Title,
|
||||||
|
AISuggestionField.Tags,
|
||||||
|
])
|
||||||
|
expect(action.get('ai_create_missing').value).toBeTruthy()
|
||||||
|
expect(action.get('ai_overwrite_existing').value).toBeTruthy()
|
||||||
|
expect(component.aiSuggestionFieldOptions).toEqual(
|
||||||
|
AI_SUGGESTION_FIELD_OPTIONS
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('should default apply AI suggestions options on a new action', () => {
|
||||||
|
component.object = {
|
||||||
|
name: 'Workflow 1',
|
||||||
|
order: 0,
|
||||||
|
enabled: true,
|
||||||
|
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
|
||||||
|
actions: [],
|
||||||
|
} as Workflow
|
||||||
|
component.addAction()
|
||||||
|
|
||||||
|
const action = component.actionFields.at(component.actionFields.length - 1)
|
||||||
|
expect(action.get('ai_suggestion_fields').value).toEqual([])
|
||||||
|
expect(action.get('ai_create_missing').value).toBeFalsy()
|
||||||
|
expect(action.get('ai_overwrite_existing').value).toBeFalsy()
|
||||||
|
})
|
||||||
|
|
||||||
it('should support add and remove triggers and actions', () => {
|
it('should support add and remove triggers and actions', () => {
|
||||||
component.object = workflow
|
component.object = workflow
|
||||||
component.addTrigger()
|
component.addTrigger()
|
||||||
|
|||||||
+62
@@ -30,6 +30,7 @@ import { StoragePath } from 'src/app/data/storage-path'
|
|||||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||||
import { Workflow } from 'src/app/data/workflow'
|
import { Workflow } from 'src/app/data/workflow'
|
||||||
import {
|
import {
|
||||||
|
AISuggestionField,
|
||||||
WorkflowAction,
|
WorkflowAction,
|
||||||
WorkflowActionType,
|
WorkflowActionType,
|
||||||
} from 'src/app/data/workflow-action'
|
} from 'src/app/data/workflow-action'
|
||||||
@@ -152,6 +153,37 @@ export const WORKFLOW_ACTION_OPTIONS = [
|
|||||||
id: WorkflowActionType.RemoteOcr,
|
id: WorkflowActionType.RemoteOcr,
|
||||||
name: $localize`Remote OCR`,
|
name: $localize`Remote OCR`,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
id: WorkflowActionType.ApplyAiSuggestions,
|
||||||
|
name: $localize`Apply AI suggestions`,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
export const AI_SUGGESTION_FIELD_OPTIONS = [
|
||||||
|
{
|
||||||
|
id: AISuggestionField.Title,
|
||||||
|
name: $localize`Title`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: AISuggestionField.Tags,
|
||||||
|
name: $localize`Tags`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: AISuggestionField.Correspondent,
|
||||||
|
name: $localize`Correspondent`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: AISuggestionField.DocumentType,
|
||||||
|
name: $localize`Document type`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: AISuggestionField.StoragePath,
|
||||||
|
name: $localize`Storage path`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: AISuggestionField.Created,
|
||||||
|
name: $localize`Created date`,
|
||||||
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
export enum TriggerFilterType {
|
export enum TriggerFilterType {
|
||||||
@@ -576,6 +608,24 @@ export class WorkflowEditDialogComponent
|
|||||||
allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
|
allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Only available after consumption. Unlike remote OCR this is hidden only
|
||||||
|
// once every trigger is consumption, so it stays offered on a workflow
|
||||||
|
// that has no triggers yet.
|
||||||
|
const aiSuggestionsUsable =
|
||||||
|
this.settingsService.get(SETTINGS_KEYS.AI_ENABLED) &&
|
||||||
|
(!formWorkflow?.triggers?.length ||
|
||||||
|
formWorkflow.triggers.some(
|
||||||
|
(trigger) => trigger.type !== WorkflowTriggerType.Consumption
|
||||||
|
) ||
|
||||||
|
formWorkflow.actions?.some(
|
||||||
|
(action) => action.type === WorkflowActionType.ApplyAiSuggestions
|
||||||
|
))
|
||||||
|
if (!aiSuggestionsUsable) {
|
||||||
|
allowed = allowed.filter(
|
||||||
|
(a) => a.id !== WorkflowActionType.ApplyAiSuggestions
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
if (
|
if (
|
||||||
this.allowedActionTypes?.length === allowed.length &&
|
this.allowedActionTypes?.length === allowed.length &&
|
||||||
this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
|
this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
|
||||||
@@ -1227,6 +1277,11 @@ export class WorkflowEditDialogComponent
|
|||||||
passwords: new FormControl(
|
passwords: new FormControl(
|
||||||
this.formatPasswords(action.passwords ?? [])
|
this.formatPasswords(action.passwords ?? [])
|
||||||
),
|
),
|
||||||
|
ai_suggestion_fields: new FormControl(
|
||||||
|
action.ai_suggestion_fields ?? []
|
||||||
|
),
|
||||||
|
ai_create_missing: new FormControl(!!action.ai_create_missing),
|
||||||
|
ai_overwrite_existing: new FormControl(!!action.ai_overwrite_existing),
|
||||||
}),
|
}),
|
||||||
{ emitEvent }
|
{ emitEvent }
|
||||||
)
|
)
|
||||||
@@ -1316,6 +1371,10 @@ export class WorkflowEditDialogComponent
|
|||||||
return this.actionTypeOptions.find((t) => t.id === type)?.name ?? ''
|
return this.actionTypeOptions.find((t) => t.id === type)?.name ?? ''
|
||||||
}
|
}
|
||||||
|
|
||||||
|
get aiSuggestionFieldOptions() {
|
||||||
|
return AI_SUGGESTION_FIELD_OPTIONS
|
||||||
|
}
|
||||||
|
|
||||||
addAction() {
|
addAction() {
|
||||||
if (!this.object) {
|
if (!this.object) {
|
||||||
this.object = Object.assign({}, this.objectForm.value)
|
this.object = Object.assign({}, this.objectForm.value)
|
||||||
@@ -1369,6 +1428,9 @@ export class WorkflowEditDialogComponent
|
|||||||
include_document: false,
|
include_document: false,
|
||||||
},
|
},
|
||||||
passwords: [],
|
passwords: [],
|
||||||
|
ai_suggestion_fields: [],
|
||||||
|
ai_create_missing: false,
|
||||||
|
ai_overwrite_existing: false,
|
||||||
}
|
}
|
||||||
this.object.actions.push(action)
|
this.object.actions.push(action)
|
||||||
this.createActionField(action)
|
this.createActionField(action)
|
||||||
|
|||||||
@@ -47,8 +47,8 @@
|
|||||||
<i-bs width="0.9em" height="0.9em" name="file-text" class="me-1"></i-bs><span>{{document.title}}</span>
|
<i-bs width="0.9em" height="0.9em" name="file-text" class="me-1"></i-bs><span>{{document.title}}</span>
|
||||||
</a>
|
</a>
|
||||||
} @else {
|
} @else {
|
||||||
<span class="badge bg-light text-muted" (click)="unselect(document)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove link" i18n-title>
|
<span class="badge bg-light text-muted">
|
||||||
<i-bs width="0.9em" height="0.9em" name="exclamation-triangle-fill" class="me-1"></i-bs><span i18n>Not found</span>
|
<i-bs width="0.9em" height="0.9em" name="exclamation-triangle-fill" class="me-1"></i-bs><span i18n>Unavailable</span>
|
||||||
</span>
|
</span>
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -151,6 +151,23 @@ describe('DocumentLinkComponent', () => {
|
|||||||
expect(component.selectedDocuments).toEqual([])
|
expect(component.selectedDocuments).toEqual([])
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should preserve and neutrally label unavailable document IDs', async () => {
|
||||||
|
jest.spyOn(documentService, 'getFew').mockReturnValue(
|
||||||
|
of({
|
||||||
|
count: 0,
|
||||||
|
all: [],
|
||||||
|
results: [],
|
||||||
|
})
|
||||||
|
)
|
||||||
|
|
||||||
|
component.writeValue([99])
|
||||||
|
await fixture.whenStable()
|
||||||
|
|
||||||
|
expect(component.selectedDocuments).toEqual([{ id: 99 }])
|
||||||
|
expect(fixture.nativeElement.textContent).toContain('Unavailable')
|
||||||
|
expect(fixture.nativeElement.textContent).not.toContain('Not found')
|
||||||
|
})
|
||||||
|
|
||||||
it('should support unselect', () => {
|
it('should support unselect', () => {
|
||||||
const getSpy = jest.spyOn(documentService, 'getFew')
|
const getSpy = jest.spyOn(documentService, 'getFew')
|
||||||
getSpy.mockImplementation((ids) => {
|
getSpy.mockImplementation((ids) => {
|
||||||
@@ -167,6 +184,15 @@ describe('DocumentLinkComponent', () => {
|
|||||||
expect(component.selectedDocuments).toEqual([documents[1]])
|
expect(component.selectedDocuments).toEqual([documents[1]])
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should not unselect documents when disabled', () => {
|
||||||
|
component.disabled = true
|
||||||
|
component.selectedDocuments = [documents[0]]
|
||||||
|
|
||||||
|
component.unselect(documents[0])
|
||||||
|
|
||||||
|
expect(component.selectedDocuments).toEqual([documents[0]])
|
||||||
|
})
|
||||||
|
|
||||||
it('should use correct compare, trackBy functions', () => {
|
it('should use correct compare, trackBy functions', () => {
|
||||||
expect(component.compareDocuments(documents[0], { id: 1 })).toBeTruthy()
|
expect(component.compareDocuments(documents[0], { id: 1 })).toBeTruthy()
|
||||||
expect(component.compareDocuments(documents[0], { id: 2 })).toBeFalsy()
|
expect(component.compareDocuments(documents[0], { id: 2 })).toBeFalsy()
|
||||||
|
|||||||
@@ -101,7 +101,7 @@ export class DocumentLinkComponent
|
|||||||
.subscribe((documentResults) => {
|
.subscribe((documentResults) => {
|
||||||
this.loading.set(false)
|
this.loading.set(false)
|
||||||
this.selectedDocuments = documentIDs.map(
|
this.selectedDocuments = documentIDs.map(
|
||||||
(id) => documentResults.results.find((d) => d.id === id) ?? {}
|
(id) => documentResults.results.find((d) => d.id === id) ?? { id }
|
||||||
)
|
)
|
||||||
super.writeValue(documentIDs)
|
super.writeValue(documentIDs)
|
||||||
})
|
})
|
||||||
@@ -142,6 +142,8 @@ export class DocumentLinkComponent
|
|||||||
}
|
}
|
||||||
|
|
||||||
unselect(document: Document): void {
|
unselect(document: Document): void {
|
||||||
|
if (this.disabled) return
|
||||||
|
|
||||||
this.selectedDocuments = this.selectedDocuments.filter(
|
this.selectedDocuments = this.selectedDocuments.filter(
|
||||||
(d) => d && d.id !== document.id
|
(d) => d && d.id !== document.id
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -40,6 +40,10 @@ describe('PasswordComponent', () => {
|
|||||||
// expect(component.value).toEqual('foo')
|
// expect(component.value).toEqual('foo')
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should not offer itself to browser autofill by default', () => {
|
||||||
|
expect(input.getAttribute('autocomplete')).toEqual('off')
|
||||||
|
})
|
||||||
|
|
||||||
it('should support toggling field visibility', () => {
|
it('should support toggling field visibility', () => {
|
||||||
expect(input.type).toEqual('password')
|
expect(input.type).toEqual('password')
|
||||||
component.showReveal = true
|
component.showReveal = true
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ export class PasswordComponent extends AbstractInputComponent<string> {
|
|||||||
showReveal: boolean = false
|
showReveal: boolean = false
|
||||||
|
|
||||||
@Input()
|
@Input()
|
||||||
autocomplete: string
|
autocomplete: string = 'off'
|
||||||
|
|
||||||
public textVisible: boolean = false
|
public textVisible: boolean = false
|
||||||
|
|
||||||
|
|||||||
@@ -36,11 +36,16 @@
|
|||||||
(focus)="clearLastSearchTerm()"
|
(focus)="clearLastSearchTerm()"
|
||||||
(clear)="clearLastSearchTerm()"
|
(clear)="clearLastSearchTerm()"
|
||||||
(blur)="onBlur()">
|
(blur)="onBlur()">
|
||||||
<ng-template ng-label-tmp let-item="item">
|
<ng-template ng-label-tmp let-item="item" let-clear="clear">
|
||||||
@if (iconField && item[iconField]) {
|
@if (multiple && !disabled) {
|
||||||
<i-bs class="me-2" [name]="item[iconField]"></i-bs>
|
<span class="ng-value-icon left" role="button" tabindex="0" (click)="clear(item)" (keydown.enter)="clear(item)" aria-label="Remove item" i18n-aria-label>×</span>
|
||||||
}
|
}
|
||||||
<span [title]="item[bindLabel]">{{item[bindLabel]}}</span>
|
<span class="ng-value-label" [title]="item[bindLabel]">
|
||||||
|
@if (iconField && item[iconField]) {
|
||||||
|
<i-bs class="me-2" [name]="item[iconField]"></i-bs>
|
||||||
|
}
|
||||||
|
{{item[bindLabel]}}
|
||||||
|
</span>
|
||||||
</ng-template>
|
</ng-template>
|
||||||
<ng-template ng-option-tmp let-item="item">
|
<ng-template ng-option-tmp let-item="item">
|
||||||
@if (iconField && item[iconField]) {
|
@if (iconField && item[iconField]) {
|
||||||
|
|||||||
@@ -24,6 +24,12 @@ describe('TextComponent', () => {
|
|||||||
input = component.inputField.nativeElement
|
input = component.inputField.nativeElement
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should not offer itself to browser autofill by default', () => {
|
||||||
|
expect(
|
||||||
|
component.inputField.nativeElement.getAttribute('autocomplete')
|
||||||
|
).toEqual('off')
|
||||||
|
})
|
||||||
|
|
||||||
it('should support use of input field', () => {
|
it('should support use of input field', () => {
|
||||||
expect(component.value).toBeUndefined()
|
expect(component.value).toBeUndefined()
|
||||||
input.value = 'foo'
|
input.value = 'foo'
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ import { AbstractInputComponent } from '../abstract-input'
|
|||||||
})
|
})
|
||||||
export class TextComponent extends AbstractInputComponent<string> {
|
export class TextComponent extends AbstractInputComponent<string> {
|
||||||
@Input()
|
@Input()
|
||||||
autocomplete: string
|
autocomplete: string = 'off'
|
||||||
|
|
||||||
@Input()
|
@Input()
|
||||||
placeholder: string = ''
|
placeholder: string = ''
|
||||||
|
|||||||
@@ -0,0 +1,3 @@
|
|||||||
|
<svg class="w-100 h-100 d-block" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" fill="currentColor" aria-hidden="true">
|
||||||
|
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 545 B |
@@ -0,0 +1,14 @@
|
|||||||
|
import { Component, input } from '@angular/core'
|
||||||
|
|
||||||
|
@Component({
|
||||||
|
selector: 'pngx-brand-mark',
|
||||||
|
templateUrl: './brand-mark.component.html',
|
||||||
|
host: {
|
||||||
|
'[style.width]': 'width()',
|
||||||
|
'[style.height]': 'height()',
|
||||||
|
},
|
||||||
|
})
|
||||||
|
export class BrandMarkComponent {
|
||||||
|
readonly width = input<string>(null)
|
||||||
|
readonly height = input<string>(null)
|
||||||
|
}
|
||||||
+16
-8
@@ -1,10 +1,18 @@
|
|||||||
<ngb-alert class="pe-3" type="primary" [dismissible]="true" (closed)="dismiss.emit(true)">
|
<ngb-alert class="welcome-widget bg-light shadow-sm pe-3 position-relative overflow-hidden" type="primary" [dismissible]="true" (closed)="dismiss.emit(true)">
|
||||||
<h4 class="alert-heading"><ng-container i18n>Paperless-ngx is running!</ng-container> 🎉</h4>
|
<pngx-brand-mark class="welcome-watermark d-none d-md-block position-absolute top-50 translate-middle-y text-primary pe-none" width="26rem" height="26rem"></pngx-brand-mark>
|
||||||
<p i18n>You're ready to start uploading documents! Explore the various features of this web app on your own, or start a quick tour using the button below.</p>
|
<div class="d-flex align-items-start gap-3 gap-md-4 position-relative">
|
||||||
<p i18n>More detail on how to use and configure Paperless-ngx is always available in the <a href="https://docs.paperless-ngx.com" target="_blank">documentation</a>.</p>
|
<pngx-brand-mark class="d-none d-sm-block flex-shrink-0 text-primary" width="3rem" height="3rem"></pngx-brand-mark>
|
||||||
<hr>
|
<div class="flex-grow-1">
|
||||||
<div class="d-flex align-items-end">
|
<h4 class="alert-heading text-primary fw-semibold"><ng-container i18n>Paperless-ngx is running!</ng-container> 🎉</h4>
|
||||||
<p class="lead fs-6 m-0"><em i18n>Thanks for being a part of the Paperless-ngx community!</em></p>
|
<div class="welcome-copy text-dark">
|
||||||
<button class="btn btn-primary ms-auto flex-shrink-0" (click)="tourService.start()"><ng-container i18n>Start the tour</ng-container> →</button>
|
<p i18n>You're ready to start uploading documents! Explore the various features of this web app on your own, or start a quick tour using the button below.</p>
|
||||||
|
<p class="mb-0" i18n>More detail on how to use and configure Paperless-ngx is always available in the <a href="https://docs.paperless-ngx.com" target="_blank">documentation</a>.</p>
|
||||||
|
</div>
|
||||||
|
<hr class="opacity-100">
|
||||||
|
<div class="d-flex flex-column flex-sm-row align-items-sm-center gap-3">
|
||||||
|
<p class="m-0 text-secondary"><em i18n>Thanks for being a part of the Paperless-ngx community!</em></p>
|
||||||
|
<button class="btn btn-primary ms-sm-auto flex-shrink-0 align-self-start align-self-sm-auto" (click)="tourService.start()"><ng-container i18n>Start the tour</ng-container> →</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</ngb-alert>
|
</ngb-alert>
|
||||||
|
|||||||
+16
@@ -0,0 +1,16 @@
|
|||||||
|
.welcome-widget {
|
||||||
|
border-color: color-mix(in srgb, var(--bs-primary) 50%, transparent);
|
||||||
|
|
||||||
|
hr {
|
||||||
|
border-top-color: color-mix(in srgb, var(--bs-primary) 25%, transparent);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.welcome-copy {
|
||||||
|
max-width: 68ch;
|
||||||
|
}
|
||||||
|
|
||||||
|
.welcome-watermark {
|
||||||
|
right: -6rem;
|
||||||
|
opacity: .07;
|
||||||
|
}
|
||||||
|
|||||||
+2
-1
@@ -1,12 +1,13 @@
|
|||||||
import { Component, EventEmitter, Output, inject } from '@angular/core'
|
import { Component, EventEmitter, Output, inject } from '@angular/core'
|
||||||
import { NgbAlertModule } from '@ng-bootstrap/ng-bootstrap'
|
import { NgbAlertModule } from '@ng-bootstrap/ng-bootstrap'
|
||||||
import { TourService } from 'ngx-ui-tour-ng-bootstrap'
|
import { TourService } from 'ngx-ui-tour-ng-bootstrap'
|
||||||
|
import { BrandMarkComponent } from '../../../common/logo/brand-mark/brand-mark.component'
|
||||||
|
|
||||||
@Component({
|
@Component({
|
||||||
selector: 'pngx-welcome-widget',
|
selector: 'pngx-welcome-widget',
|
||||||
templateUrl: './welcome-widget.component.html',
|
templateUrl: './welcome-widget.component.html',
|
||||||
styleUrls: ['./welcome-widget.component.scss'],
|
styleUrls: ['./welcome-widget.component.scss'],
|
||||||
imports: [NgbAlertModule],
|
imports: [NgbAlertModule, BrandMarkComponent],
|
||||||
})
|
})
|
||||||
export class WelcomeWidgetComponent {
|
export class WelcomeWidgetComponent {
|
||||||
readonly tourService = inject(TourService)
|
readonly tourService = inject(TourService)
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ import {
|
|||||||
} from '@ng-bootstrap/ng-bootstrap'
|
} from '@ng-bootstrap/ng-bootstrap'
|
||||||
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
|
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
|
||||||
import { DeviceDetectorService } from 'ngx-device-detector'
|
import { DeviceDetectorService } from 'ngx-device-detector'
|
||||||
import { of, throwError } from 'rxjs'
|
import { Subject, of, throwError } from 'rxjs'
|
||||||
import { routes } from 'src/app/app-routing.module'
|
import { routes } from 'src/app/app-routing.module'
|
||||||
import { Correspondent } from 'src/app/data/correspondent'
|
import { Correspondent } from 'src/app/data/correspondent'
|
||||||
import { CustomFieldDataType } from 'src/app/data/custom-field'
|
import { CustomFieldDataType } from 'src/app/data/custom-field'
|
||||||
@@ -1444,6 +1444,26 @@ describe('DocumentDetailComponent', () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should reset the suggestions loading state if the document changes mid-request', () => {
|
||||||
|
const getSetting = settingsService.get.bind(settingsService)
|
||||||
|
jest
|
||||||
|
.spyOn(settingsService, 'get')
|
||||||
|
.mockImplementation((key) =>
|
||||||
|
key === SETTINGS_KEYS.AI_ENABLED ? true : getSetting(key)
|
||||||
|
)
|
||||||
|
const pending = new Subject<any>()
|
||||||
|
jest
|
||||||
|
.spyOn(documentService, 'getAiSuggestions')
|
||||||
|
.mockReturnValue(pending.asObservable())
|
||||||
|
initNormally()
|
||||||
|
expect(component.suggestionsLoading()).toBeTruthy()
|
||||||
|
|
||||||
|
// the in-flight request is cancelled, e.g. by a websocket-driven reload
|
||||||
|
component.docChangeNotifier.next(component.documentId())
|
||||||
|
|
||||||
|
expect(component.suggestionsLoading()).toBeFalsy()
|
||||||
|
})
|
||||||
|
|
||||||
it('should show error if needed for get suggestions', () => {
|
it('should show error if needed for get suggestions', () => {
|
||||||
const suggestionsSpy = jest.spyOn(documentService, 'getSuggestions')
|
const suggestionsSpy = jest.spyOn(documentService, 'getSuggestions')
|
||||||
const errorSpy = jest.spyOn(toastService, 'showError')
|
const errorSpy = jest.spyOn(toastService, 'showError')
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ import {
|
|||||||
debounceTime,
|
debounceTime,
|
||||||
distinctUntilChanged,
|
distinctUntilChanged,
|
||||||
filter,
|
filter,
|
||||||
|
finalize,
|
||||||
first,
|
first,
|
||||||
map,
|
map,
|
||||||
switchMap,
|
switchMap,
|
||||||
@@ -1016,16 +1017,15 @@ export class DocumentDetailComponent
|
|||||||
.pipe(
|
.pipe(
|
||||||
first(),
|
first(),
|
||||||
takeUntil(this.unsubscribeNotifier),
|
takeUntil(this.unsubscribeNotifier),
|
||||||
takeUntil(this.docChangeNotifier)
|
takeUntil(this.docChangeNotifier),
|
||||||
|
finalize(() => this.suggestionsLoading.set(false))
|
||||||
)
|
)
|
||||||
.subscribe({
|
.subscribe({
|
||||||
next: (result) => {
|
next: (result) => {
|
||||||
this.suggestions.set(result)
|
this.suggestions.set(result)
|
||||||
this.suggestionsLoading.set(false)
|
|
||||||
},
|
},
|
||||||
error: (error) => {
|
error: (error) => {
|
||||||
this.suggestions.set(null)
|
this.suggestions.set(null)
|
||||||
this.suggestionsLoading.set(false)
|
|
||||||
this.toastService.showError(
|
this.toastService.showError(
|
||||||
$localize`Error retrieving suggestions.`,
|
$localize`Error retrieving suggestions.`,
|
||||||
error
|
error
|
||||||
|
|||||||
@@ -114,13 +114,23 @@
|
|||||||
</div>
|
</div>
|
||||||
</button>
|
</button>
|
||||||
<div ngbDropdownMenu aria-labelledby="dropdownSend" class="shadow">
|
<div ngbDropdownMenu aria-labelledby="dropdownSend" class="shadow">
|
||||||
<button ngbDropdownItem (click)="createShareLinkBundle()" [disabled]="!canSendSelection">
|
@if (permissionService.currentUserCan(PermissionAction.Add, PermissionType.ShareLinkBundle)) {
|
||||||
<i-bs name="link" class="me-1"></i-bs><ng-container i18n>Create a share link bundle</ng-container>
|
<button ngbDropdownItem (click)="createShareLinkBundle()" [disabled]="!canSendSelection">
|
||||||
</button>
|
<i-bs name="link" class="me-1"></i-bs><ng-container i18n>Create a share link bundle</ng-container>
|
||||||
<button ngbDropdownItem (click)="manageShareLinkBundles()">
|
</button>
|
||||||
<i-bs name="list-ul" class="me-1"></i-bs><ng-container i18n>Manage share link bundles</ng-container>
|
}
|
||||||
</button>
|
@if (permissionService.currentUserCan(PermissionAction.View, PermissionType.ShareLinkBundle)) {
|
||||||
<div class="dropdown-divider"></div>
|
<button ngbDropdownItem (click)="manageShareLinkBundles()">
|
||||||
|
<i-bs name="list-ul" class="me-1"></i-bs><ng-container i18n>Manage share link bundles</ng-container>
|
||||||
|
</button>
|
||||||
|
}
|
||||||
|
@if (
|
||||||
|
emailEnabled &&
|
||||||
|
(permissionService.currentUserCan(PermissionAction.Add, PermissionType.ShareLinkBundle) ||
|
||||||
|
permissionService.currentUserCan(PermissionAction.View, PermissionType.ShareLinkBundle))
|
||||||
|
) {
|
||||||
|
<div class="dropdown-divider"></div>
|
||||||
|
}
|
||||||
@if (emailEnabled) {
|
@if (emailEnabled) {
|
||||||
<button ngbDropdownItem (click)="emailSelected()" [disabled]="!canSendSelection">
|
<button ngbDropdownItem (click)="emailSelected()" [disabled]="!canSendSelection">
|
||||||
<i-bs name="envelope" class="me-1"></i-bs><ng-container i18n>Email</ng-container>
|
<i-bs name="envelope" class="me-1"></i-bs><ng-container i18n>Email</ng-container>
|
||||||
|
|||||||
@@ -19,7 +19,11 @@ import { StoragePath } from 'src/app/data/storage-path'
|
|||||||
import { Tag } from 'src/app/data/tag'
|
import { Tag } from 'src/app/data/tag'
|
||||||
import { FilterPipe } from 'src/app/pipes/filter.pipe'
|
import { FilterPipe } from 'src/app/pipes/filter.pipe'
|
||||||
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
|
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
|
||||||
import { PermissionsService } from 'src/app/services/permissions.service'
|
import {
|
||||||
|
PermissionAction,
|
||||||
|
PermissionsService,
|
||||||
|
PermissionType,
|
||||||
|
} from 'src/app/services/permissions.service'
|
||||||
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
|
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
|
||||||
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
|
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
|
||||||
import { DocumentTypeService } from 'src/app/services/rest/document-type.service'
|
import { DocumentTypeService } from 'src/app/services/rest/document-type.service'
|
||||||
@@ -252,6 +256,54 @@ describe('BulkEditorComponent', () => {
|
|||||||
).toBe(true)
|
).toBe(true)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('should only show permitted share link bundle actions', () => {
|
||||||
|
permissionsService.initialize(
|
||||||
|
[
|
||||||
|
permissionsService.getPermissionCode(
|
||||||
|
PermissionAction.Add,
|
||||||
|
PermissionType.ShareLinkBundle
|
||||||
|
),
|
||||||
|
],
|
||||||
|
{ is_superuser: false } as any
|
||||||
|
)
|
||||||
|
fixture.detectChanges()
|
||||||
|
|
||||||
|
expect(fixture.nativeElement.textContent).toContain(
|
||||||
|
'Create a share link bundle'
|
||||||
|
)
|
||||||
|
expect(fixture.nativeElement.textContent).not.toContain(
|
||||||
|
'Manage share link bundles'
|
||||||
|
)
|
||||||
|
|
||||||
|
permissionsService.initialize(
|
||||||
|
[
|
||||||
|
permissionsService.getPermissionCode(
|
||||||
|
PermissionAction.View,
|
||||||
|
PermissionType.ShareLinkBundle
|
||||||
|
),
|
||||||
|
],
|
||||||
|
{ is_superuser: false } as any
|
||||||
|
)
|
||||||
|
fixture.detectChanges()
|
||||||
|
|
||||||
|
expect(fixture.nativeElement.textContent).not.toContain(
|
||||||
|
'Create a share link bundle'
|
||||||
|
)
|
||||||
|
expect(fixture.nativeElement.textContent).toContain(
|
||||||
|
'Manage share link bundles'
|
||||||
|
)
|
||||||
|
|
||||||
|
permissionsService.initialize([], { is_superuser: false } as any)
|
||||||
|
fixture.detectChanges()
|
||||||
|
|
||||||
|
expect(fixture.nativeElement.textContent).not.toContain(
|
||||||
|
'Create a share link bundle'
|
||||||
|
)
|
||||||
|
expect(fixture.nativeElement.textContent).not.toContain(
|
||||||
|
'Manage share link bundles'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
it('should apply selection data to correspondents menu', () => {
|
it('should apply selection data to correspondents menu', () => {
|
||||||
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
|
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
|
||||||
fixture.detectChanges()
|
fixture.detectChanges()
|
||||||
|
|||||||
@@ -101,7 +101,7 @@ export class BulkEditorComponent
|
|||||||
private toastService = inject(ToastService)
|
private toastService = inject(ToastService)
|
||||||
private storagePathService = inject(StoragePathService)
|
private storagePathService = inject(StoragePathService)
|
||||||
private customFieldService = inject(CustomFieldsService)
|
private customFieldService = inject(CustomFieldsService)
|
||||||
private permissionService = inject(PermissionsService)
|
public readonly permissionService = inject(PermissionsService)
|
||||||
private savedViewService = inject(SavedViewService)
|
private savedViewService = inject(SavedViewService)
|
||||||
private readonly shareLinkBundleService = inject(ShareLinkBundleService)
|
private readonly shareLinkBundleService = inject(ShareLinkBundleService)
|
||||||
|
|
||||||
|
|||||||
@@ -8,6 +8,17 @@ export enum WorkflowActionType {
|
|||||||
PasswordRemoval = 5,
|
PasswordRemoval = 5,
|
||||||
MoveToTrash = 6,
|
MoveToTrash = 6,
|
||||||
RemoteOcr = 7,
|
RemoteOcr = 7,
|
||||||
|
ApplyAiSuggestions = 8,
|
||||||
|
}
|
||||||
|
|
||||||
|
// see src/documents/models.py AISuggestionField
|
||||||
|
export enum AISuggestionField {
|
||||||
|
Title = 'title',
|
||||||
|
Tags = 'tags',
|
||||||
|
Correspondent = 'correspondent',
|
||||||
|
DocumentType = 'document_type',
|
||||||
|
StoragePath = 'storage_path',
|
||||||
|
Created = 'created',
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface WorkflowActionEmail extends ObjectWithId {
|
export interface WorkflowActionEmail extends ObjectWithId {
|
||||||
@@ -102,4 +113,10 @@ export interface WorkflowAction extends ObjectWithId {
|
|||||||
webhook?: WorkflowActionWebhook
|
webhook?: WorkflowActionWebhook
|
||||||
|
|
||||||
passwords?: string[]
|
passwords?: string[]
|
||||||
|
|
||||||
|
ai_suggestion_fields?: AISuggestionField[]
|
||||||
|
|
||||||
|
ai_create_missing?: boolean
|
||||||
|
|
||||||
|
ai_overwrite_existing?: boolean
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,6 +15,35 @@ describe('text search utilities', () => {
|
|||||||
expect(matchesSearchText('taxes 2026', 'tax receipt')).toBeFalsy()
|
expect(matchesSearchText('taxes 2026', 'tax receipt')).toBeFalsy()
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('does not let two terms match the same word', () => {
|
||||||
|
expect(matchesSearchText('Another Tag', 'another tag th')).toBeFalsy()
|
||||||
|
expect(matchesSearchText('Another Tag', 'another tag ag')).toBeFalsy()
|
||||||
|
expect(matchesSearchText('Another Tag', 'another tag e')).toBeFalsy()
|
||||||
|
expect(matchesSearchText('Another Tag', 'another tag')).toBeTruthy()
|
||||||
|
expect(matchesSearchText('Another Tag', 'tag another')).toBeTruthy()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('matches a single term anywhere in the value', () => {
|
||||||
|
expect(matchesSearchText('Another Tag', 'anoth')).toBeTruthy()
|
||||||
|
expect(matchesSearchText('Another Tag', 'th')).toBeTruthy()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('treats punctuation as a separator on both sides', () => {
|
||||||
|
expect(matchesSearchText('medical-history', 'medical history')).toBeTruthy()
|
||||||
|
expect(matchesSearchText('medical history', 'medical-history')).toBeTruthy()
|
||||||
|
expect(matchesSearchText('medical-history', 'medical dental')).toBeFalsy()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('matches longer terms first so they cannot be starved', () => {
|
||||||
|
expect(matchesSearchText('tagger tag', 'tag tagger')).toBeTruthy()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('handles a query with no usable terms', () => {
|
||||||
|
expect(matchesSearchText('Another Tag', '')).toBeTruthy()
|
||||||
|
// Still filters, so the dropdown can offer to create a tag named "---"
|
||||||
|
expect(matchesSearchText('Another Tag', '---')).toBeFalsy()
|
||||||
|
})
|
||||||
|
|
||||||
it('matches a large set of tag names without blocking input', () => {
|
it('matches a large set of tag names without blocking input', () => {
|
||||||
const tagNames = Array.from(
|
const tagNames = Array.from(
|
||||||
{ length: 1280 },
|
{ length: 1280 },
|
||||||
|
|||||||
@@ -3,13 +3,18 @@ import { diacritics } from 'normalize-diacritics/diacritics'
|
|||||||
export type SearchTextValue =
|
export type SearchTextValue =
|
||||||
string | number | boolean | bigint | null | undefined
|
string | number | boolean | bigint | null | undefined
|
||||||
|
|
||||||
|
const NON_ASCII = /[^\x00-\x7F]/
|
||||||
|
const SEPARATORS = /[^\p{L}\p{N}]+/u
|
||||||
|
|
||||||
export function normalizeSearchText(value: SearchTextValue): string {
|
export function normalizeSearchText(value: SearchTextValue): string {
|
||||||
const normalized = diacritics.reduce(
|
const text = String(value ?? '')
|
||||||
(text, replacement) => {
|
|
||||||
return text.replace(replacement.diacritics, replacement.letter)
|
// Nothing in the table matches ASCII, so skip normaliation
|
||||||
},
|
if (!NON_ASCII.test(text)) return text.toLocaleLowerCase()
|
||||||
String(value ?? '')
|
|
||||||
)
|
const normalized = diacritics.reduce((text, replacement) => {
|
||||||
|
return text.replace(replacement.diacritics, replacement.letter)
|
||||||
|
}, text)
|
||||||
|
|
||||||
return normalized.toLocaleLowerCase()
|
return normalized.toLocaleLowerCase()
|
||||||
}
|
}
|
||||||
@@ -18,8 +23,28 @@ export function matchesSearchText(
|
|||||||
value: SearchTextValue,
|
value: SearchTextValue,
|
||||||
searchText: SearchTextValue
|
searchText: SearchTextValue
|
||||||
): boolean {
|
): boolean {
|
||||||
const normalizedValue = normalizeSearchText(value)
|
const query = normalizeSearchText(searchText)
|
||||||
const searchTerms = normalizeSearchText(searchText).trim().split(/\s+/)
|
const terms = query.split(SEPARATORS).filter(Boolean)
|
||||||
|
|
||||||
return searchTerms.every((term) => normalizedValue.includes(term))
|
// Empty or punctuation-only query, nothing to split into terms
|
||||||
|
if (terms.length === 0) {
|
||||||
|
return normalizeSearchText(value).includes(query.trim())
|
||||||
|
}
|
||||||
|
|
||||||
|
const words = normalizeSearchText(value).split(SEPARATORS).filter(Boolean)
|
||||||
|
const claimed = new Array<boolean>(words.length).fill(false)
|
||||||
|
|
||||||
|
// Each term takes a word of its own, longest first, so that "another tag th"
|
||||||
|
// doesn't match "Another Tag" by finding the "th" inside "another"
|
||||||
|
return terms
|
||||||
|
.sort((a, b) => b.length - a.length)
|
||||||
|
.every((term) => {
|
||||||
|
for (let i = 0; i < words.length; i++) {
|
||||||
|
if (!claimed[i] && words[i].includes(term)) {
|
||||||
|
claimed[i] = true
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ export const environment = {
|
|||||||
apiVersion: '10', // match src/paperless/settings.py
|
apiVersion: '10', // match src/paperless/settings.py
|
||||||
appTitle: DEFAULT_APP_TITLE,
|
appTitle: DEFAULT_APP_TITLE,
|
||||||
tag: 'prod',
|
tag: 'prod',
|
||||||
version: '3.0.5',
|
version: '3.1.0',
|
||||||
webSocketHost: window.location.host,
|
webSocketHost: window.location.host,
|
||||||
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
|
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
|
||||||
webSocketBaseUrl: base_url.pathname + 'ws/',
|
webSocketBaseUrl: base_url.pathname + 'ws/',
|
||||||
|
|||||||
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2310
-1139
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2309
-1138
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2336
-1165
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2499
-1328
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+3555
-2385
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2529
-1359
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2344
-1173
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+2309
-1138
File diff suppressed because it is too large
Load Diff
+2310
-1139
File diff suppressed because it is too large
Load Diff
+2307
-1136
File diff suppressed because it is too large
Load Diff
+56
-10
@@ -1,7 +1,9 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
import logging
|
import logging
|
||||||
import pickle
|
import pickle
|
||||||
|
import uuid
|
||||||
from binascii import hexlify
|
from binascii import hexlify
|
||||||
from collections import OrderedDict
|
from collections import OrderedDict
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
@@ -55,6 +57,8 @@ LLM_CACHE_CLASSIFIER_VERSION: Final[int] = 1001
|
|||||||
CACHE_1_MINUTE: Final[int] = 60
|
CACHE_1_MINUTE: Final[int] = 60
|
||||||
CACHE_5_MINUTES: Final[int] = 5 * CACHE_1_MINUTE
|
CACHE_5_MINUTES: Final[int] = 5 * CACHE_1_MINUTE
|
||||||
CACHE_50_MINUTES: Final[int] = 50 * CACHE_1_MINUTE
|
CACHE_50_MINUTES: Final[int] = 50 * CACHE_1_MINUTE
|
||||||
|
# Deliberately longer than any entry it names
|
||||||
|
LLM_CACHE_GENERATION_TIMEOUT: Final[int] = 2 * CACHE_50_MINUTES
|
||||||
|
|
||||||
read_cache = caches["read-cache"]
|
read_cache = caches["read-cache"]
|
||||||
|
|
||||||
@@ -206,12 +210,40 @@ def refresh_suggestions_cache(
|
|||||||
cache.touch(doc_key, timeout)
|
cache.touch(doc_key, timeout)
|
||||||
|
|
||||||
|
|
||||||
|
def invalidate_suggestions_cache(document_id: int) -> None:
|
||||||
|
"""Invalidate classifier-generated suggestions for a document."""
|
||||||
|
cache.delete(get_suggestion_cache_key(document_id))
|
||||||
|
|
||||||
|
|
||||||
|
def _llm_generation_key(document_id: int) -> str:
|
||||||
|
return f"{get_suggestion_cache_key(document_id)}_llm_generation"
|
||||||
|
|
||||||
|
|
||||||
|
def _llm_variant_key(document_id: int, backend: str) -> str:
|
||||||
|
"""Cache key for one LLM configuration and permission scope.
|
||||||
|
|
||||||
|
``backend`` identifies the variant - model, endpoint, output language and
|
||||||
|
requesting user.
|
||||||
|
|
||||||
|
Generating the token on first use lets invalidate_llm_suggestions_cache()
|
||||||
|
be no-op for documents that never had AI suggestions.
|
||||||
|
"""
|
||||||
|
generation_key = _llm_generation_key(document_id)
|
||||||
|
generation = cache.get_or_set(
|
||||||
|
generation_key,
|
||||||
|
lambda: uuid.uuid4().hex,
|
||||||
|
timeout=LLM_CACHE_GENERATION_TIMEOUT,
|
||||||
|
)
|
||||||
|
cache.touch(generation_key, LLM_CACHE_GENERATION_TIMEOUT)
|
||||||
|
backend_hash = hashlib.sha256(backend.encode()).hexdigest()[:16]
|
||||||
|
return f"{get_suggestion_cache_key(document_id)}_llm_{generation}_{backend_hash}"
|
||||||
|
|
||||||
|
|
||||||
def get_llm_suggestion_cache(
|
def get_llm_suggestion_cache(
|
||||||
document_id: int,
|
document_id: int,
|
||||||
backend: str,
|
backend: str,
|
||||||
) -> SuggestionCacheData | None:
|
) -> SuggestionCacheData | None:
|
||||||
doc_key = get_suggestion_cache_key(document_id)
|
data: SuggestionCacheData = cache.get(_llm_variant_key(document_id, backend))
|
||||||
data: SuggestionCacheData = cache.get(doc_key)
|
|
||||||
|
|
||||||
if (
|
if (
|
||||||
data
|
data
|
||||||
@@ -234,9 +266,8 @@ def set_llm_suggestions_cache(
|
|||||||
Cache LLM-generated suggestions using a backend-specific identifier
|
Cache LLM-generated suggestions using a backend-specific identifier
|
||||||
(e.g. 'openai-like:gpt-4').
|
(e.g. 'openai-like:gpt-4').
|
||||||
"""
|
"""
|
||||||
doc_key = get_suggestion_cache_key(document_id)
|
|
||||||
cache.set(
|
cache.set(
|
||||||
doc_key,
|
_llm_variant_key(document_id, backend),
|
||||||
SuggestionCacheData(
|
SuggestionCacheData(
|
||||||
classifier_version=LLM_CACHE_CLASSIFIER_VERSION,
|
classifier_version=LLM_CACHE_CLASSIFIER_VERSION,
|
||||||
classifier_hash=backend,
|
classifier_hash=backend,
|
||||||
@@ -246,17 +277,31 @@ def set_llm_suggestions_cache(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def refresh_llm_suggestions_cache(
|
||||||
|
document_id: int,
|
||||||
|
backend: str,
|
||||||
|
*,
|
||||||
|
timeout: int = CACHE_50_MINUTES,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
Refreshes the expiration of one cached LLM suggestion variant.
|
||||||
|
"""
|
||||||
|
cache.touch(_llm_variant_key(document_id, backend), timeout)
|
||||||
|
|
||||||
|
|
||||||
def invalidate_llm_suggestions_cache(
|
def invalidate_llm_suggestions_cache(
|
||||||
document_id: int,
|
document_id: int,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""
|
"""
|
||||||
Invalidate the LLM suggestions cache for a specific document and backend.
|
Invalidate every LLM suggestion variant for a document.
|
||||||
"""
|
"""
|
||||||
doc_key = get_suggestion_cache_key(document_id)
|
generation_key = _llm_generation_key(document_id)
|
||||||
data: SuggestionCacheData = cache.get(doc_key)
|
if cache.get(generation_key) is not None:
|
||||||
|
cache.set(
|
||||||
if data:
|
generation_key,
|
||||||
cache.delete(doc_key)
|
uuid.uuid4().hex,
|
||||||
|
timeout=LLM_CACHE_GENERATION_TIMEOUT,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def get_metadata_cache_key(document_id: int) -> str:
|
def get_metadata_cache_key(document_id: int) -> str:
|
||||||
@@ -357,3 +402,4 @@ def clear_document_caches(document_id: int) -> None:
|
|||||||
get_thumbnail_modified_key(document_id),
|
get_thumbnail_modified_key(document_id),
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
|
invalidate_llm_suggestions_cache(document_id)
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
# Generated by Django 5.2.16 on 2026-08-10 18:26
|
||||||
|
|
||||||
|
from django.db import migrations
|
||||||
|
from django.db import models
|
||||||
|
|
||||||
|
|
||||||
|
class Migration(migrations.Migration):
|
||||||
|
dependencies = [
|
||||||
|
("documents", "0024_alter_workflowaction_type"),
|
||||||
|
]
|
||||||
|
|
||||||
|
operations = [
|
||||||
|
migrations.AddField(
|
||||||
|
model_name="workflowaction",
|
||||||
|
name="ai_create_missing",
|
||||||
|
field=models.BooleanField(
|
||||||
|
default=False,
|
||||||
|
help_text="Create suggested tags, correspondents, document types and storage paths that do not already exist instead of skipping them.",
|
||||||
|
verbose_name="create missing objects",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
migrations.AddField(
|
||||||
|
model_name="workflowaction",
|
||||||
|
name="ai_overwrite_existing",
|
||||||
|
field=models.BooleanField(
|
||||||
|
default=False,
|
||||||
|
help_text="Apply suggestions even if the document already has a value for that field. Tags are always added to, never replaced.",
|
||||||
|
verbose_name="overwrite existing values",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
migrations.AddField(
|
||||||
|
model_name="workflowaction",
|
||||||
|
name="ai_suggestion_fields",
|
||||||
|
field=models.JSONField(
|
||||||
|
blank=True,
|
||||||
|
help_text="Which of the AI-suggested fields to apply to the document.",
|
||||||
|
null=True,
|
||||||
|
verbose_name="AI suggestion fields",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
migrations.AlterField(
|
||||||
|
model_name="workflowaction",
|
||||||
|
name="type",
|
||||||
|
field=models.PositiveSmallIntegerField(
|
||||||
|
choices=[
|
||||||
|
(1, "Assignment"),
|
||||||
|
(2, "Removal"),
|
||||||
|
(3, "Email"),
|
||||||
|
(4, "Webhook"),
|
||||||
|
(5, "Password removal"),
|
||||||
|
(6, "Move to trash"),
|
||||||
|
(7, "Remote OCR"),
|
||||||
|
(8, "Apply AI suggestions"),
|
||||||
|
],
|
||||||
|
default=1,
|
||||||
|
verbose_name="Workflow Action Type",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
migrations.AlterField(
|
||||||
|
model_name="paperlesstask",
|
||||||
|
name="task_type",
|
||||||
|
field=models.CharField(
|
||||||
|
choices=[
|
||||||
|
("consume_file", "Consume File"),
|
||||||
|
("train_classifier", "Train Classifier"),
|
||||||
|
("sanity_check", "Sanity Check"),
|
||||||
|
("index_optimize", "Index Optimize"),
|
||||||
|
("mail_fetch", "Mail Fetch"),
|
||||||
|
("llm_index", "LLM Index"),
|
||||||
|
("empty_trash", "Empty Trash"),
|
||||||
|
("check_workflows", "Check Workflows"),
|
||||||
|
("bulk_update", "Bulk Update"),
|
||||||
|
("reprocess_document", "Reprocess Document"),
|
||||||
|
("build_share_link", "Build Share Link"),
|
||||||
|
("bulk_delete", "Bulk Delete"),
|
||||||
|
("apply_ai_suggestions", "Apply AI Suggestions"),
|
||||||
|
],
|
||||||
|
db_index=True,
|
||||||
|
help_text="The kind of work being performed",
|
||||||
|
max_length=50,
|
||||||
|
verbose_name="Task Type",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
]
|
||||||
@@ -766,6 +766,7 @@ class PaperlessTask(ModelWithOwner):
|
|||||||
REPROCESS_DOCUMENT = "reprocess_document", _("Reprocess Document")
|
REPROCESS_DOCUMENT = "reprocess_document", _("Reprocess Document")
|
||||||
BUILD_SHARE_LINK = "build_share_link", _("Build Share Link")
|
BUILD_SHARE_LINK = "build_share_link", _("Build Share Link")
|
||||||
BULK_DELETE = "bulk_delete", _("Bulk Delete")
|
BULK_DELETE = "bulk_delete", _("Bulk Delete")
|
||||||
|
APPLY_AI_SUGGESTIONS = "apply_ai_suggestions", _("Apply AI Suggestions")
|
||||||
|
|
||||||
COMPLETE_STATUSES = (
|
COMPLETE_STATUSES = (
|
||||||
Status.SUCCESS,
|
Status.SUCCESS,
|
||||||
@@ -1674,6 +1675,18 @@ class WorkflowAction(models.Model):
|
|||||||
7,
|
7,
|
||||||
_("Remote OCR"),
|
_("Remote OCR"),
|
||||||
)
|
)
|
||||||
|
APPLY_AI_SUGGESTIONS = (
|
||||||
|
8,
|
||||||
|
_("Apply AI suggestions"),
|
||||||
|
)
|
||||||
|
|
||||||
|
class AISuggestionField(models.TextChoices):
|
||||||
|
TITLE = ("title", _("Title"))
|
||||||
|
TAGS = ("tags", _("Tags"))
|
||||||
|
CORRESPONDENT = ("correspondent", _("Correspondent"))
|
||||||
|
DOCUMENT_TYPE = ("document_type", _("Document type"))
|
||||||
|
STORAGE_PATH = ("storage_path", _("Storage path"))
|
||||||
|
CREATED = ("created", _("Created date"))
|
||||||
|
|
||||||
type = models.PositiveSmallIntegerField(
|
type = models.PositiveSmallIntegerField(
|
||||||
_("Workflow Action Type"),
|
_("Workflow Action Type"),
|
||||||
@@ -1912,6 +1925,33 @@ class WorkflowAction(models.Model):
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
ai_suggestion_fields = models.JSONField(
|
||||||
|
_("AI suggestion fields"),
|
||||||
|
null=True,
|
||||||
|
blank=True,
|
||||||
|
help_text=_(
|
||||||
|
"Which of the AI-suggested fields to apply to the document.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
ai_create_missing = models.BooleanField(
|
||||||
|
_("create missing objects"),
|
||||||
|
default=False,
|
||||||
|
help_text=_(
|
||||||
|
"Create suggested tags, correspondents, document types and storage "
|
||||||
|
"paths that do not already exist instead of skipping them.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
ai_overwrite_existing = models.BooleanField(
|
||||||
|
_("overwrite existing values"),
|
||||||
|
default=False,
|
||||||
|
help_text=_(
|
||||||
|
"Apply suggestions even if the document already has a value for that "
|
||||||
|
"field. Tags are always added to, never replaced.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
class Meta:
|
class Meta:
|
||||||
verbose_name = _("workflow action")
|
verbose_name = _("workflow action")
|
||||||
verbose_name_plural = _("workflow actions")
|
verbose_name_plural = _("workflow actions")
|
||||||
|
|||||||
@@ -3235,6 +3235,9 @@ class WorkflowActionSerializer(serializers.ModelSerializer[WorkflowAction]):
|
|||||||
"email",
|
"email",
|
||||||
"webhook",
|
"webhook",
|
||||||
"passwords",
|
"passwords",
|
||||||
|
"ai_suggestion_fields",
|
||||||
|
"ai_create_missing",
|
||||||
|
"ai_overwrite_existing",
|
||||||
]
|
]
|
||||||
|
|
||||||
def validate(self, attrs):
|
def validate(self, attrs):
|
||||||
@@ -3292,6 +3295,23 @@ class WorkflowActionSerializer(serializers.ModelSerializer[WorkflowAction]):
|
|||||||
"Passwords are required for password removal actions",
|
"Passwords are required for password removal actions",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
"type" in attrs
|
||||||
|
and attrs["type"] == WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS
|
||||||
|
):
|
||||||
|
fields = attrs.get("ai_suggestion_fields")
|
||||||
|
valid_fields = set(WorkflowAction.AISuggestionField.values)
|
||||||
|
if (
|
||||||
|
fields is None
|
||||||
|
or not isinstance(fields, list)
|
||||||
|
or len(fields) == 0
|
||||||
|
or any(field not in valid_fields for field in fields)
|
||||||
|
):
|
||||||
|
raise serializers.ValidationError(
|
||||||
|
"At least one valid field is required for apply AI "
|
||||||
|
f"suggestions actions, options are: {sorted(valid_fields)}",
|
||||||
|
)
|
||||||
|
|
||||||
return attrs
|
return attrs
|
||||||
|
|
||||||
|
|
||||||
@@ -3320,24 +3340,43 @@ class WorkflowSerializer(serializers.ModelSerializer[Workflow]):
|
|||||||
action.get("type") == WorkflowAction.WorkflowActionType.REMOTE_OCR
|
action.get("type") == WorkflowAction.WorkflowActionType.REMOTE_OCR
|
||||||
for action in attrs["actions"]
|
for action in attrs["actions"]
|
||||||
)
|
)
|
||||||
|
has_ai_suggestions_action = any(
|
||||||
|
action.get("type")
|
||||||
|
== WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS
|
||||||
|
for action in attrs["actions"]
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
has_remote_ocr_action = self.instance is not None and (
|
has_remote_ocr_action = self.instance is not None and (
|
||||||
self.instance.actions.filter(
|
self.instance.actions.filter(
|
||||||
type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
|
type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
|
||||||
).exists()
|
).exists()
|
||||||
)
|
)
|
||||||
|
has_ai_suggestions_action = self.instance is not None and (
|
||||||
|
self.instance.actions.filter(
|
||||||
|
type=WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
|
||||||
|
).exists()
|
||||||
|
)
|
||||||
|
|
||||||
if "triggers" in attrs:
|
if "triggers" in attrs:
|
||||||
has_consumption_trigger = any(
|
has_consumption_trigger = any(
|
||||||
trigger.get("type") == WorkflowTrigger.WorkflowTriggerType.CONSUMPTION
|
trigger.get("type") == WorkflowTrigger.WorkflowTriggerType.CONSUMPTION
|
||||||
for trigger in attrs["triggers"]
|
for trigger in attrs["triggers"]
|
||||||
)
|
)
|
||||||
|
has_non_consumption_trigger = any(
|
||||||
|
trigger.get("type") != WorkflowTrigger.WorkflowTriggerType.CONSUMPTION
|
||||||
|
for trigger in attrs["triggers"]
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
has_consumption_trigger = self.instance is not None and (
|
has_consumption_trigger = self.instance is not None and (
|
||||||
self.instance.triggers.filter(
|
self.instance.triggers.filter(
|
||||||
type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
|
type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
|
||||||
).exists()
|
).exists()
|
||||||
)
|
)
|
||||||
|
has_non_consumption_trigger = self.instance is not None and (
|
||||||
|
self.instance.triggers.exclude(
|
||||||
|
type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
|
||||||
|
).exists()
|
||||||
|
)
|
||||||
|
|
||||||
# Remote OCR can only work with consumption triggers
|
# Remote OCR can only work with consumption triggers
|
||||||
if has_remote_ocr_action and not has_consumption_trigger:
|
if has_remote_ocr_action and not has_consumption_trigger:
|
||||||
@@ -3345,6 +3384,14 @@ class WorkflowSerializer(serializers.ModelSerializer[Workflow]):
|
|||||||
"Remote OCR actions require a consumption started trigger",
|
"Remote OCR actions require a consumption started trigger",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Suggestions are made from the document content, which does not exist
|
||||||
|
# until after consumption has finished
|
||||||
|
if has_ai_suggestions_action and not has_non_consumption_trigger:
|
||||||
|
raise serializers.ValidationError(
|
||||||
|
"Apply AI suggestions actions require a trigger other than "
|
||||||
|
"consumption started",
|
||||||
|
)
|
||||||
|
|
||||||
return attrs
|
return attrs
|
||||||
|
|
||||||
def update_triggers_and_actions(
|
def update_triggers_and_actions(
|
||||||
|
|||||||
@@ -32,6 +32,7 @@ from rest_framework import serializers
|
|||||||
from documents import matching
|
from documents import matching
|
||||||
from documents.caching import clear_document_caches
|
from documents.caching import clear_document_caches
|
||||||
from documents.caching import invalidate_llm_suggestions_cache
|
from documents.caching import invalidate_llm_suggestions_cache
|
||||||
|
from documents.caching import invalidate_suggestions_cache
|
||||||
from documents.data_models import ConsumableDocument
|
from documents.data_models import ConsumableDocument
|
||||||
from documents.file_handling import create_source_path_directory
|
from documents.file_handling import create_source_path_directory
|
||||||
from documents.file_handling import delete_empty_directories
|
from documents.file_handling import delete_empty_directories
|
||||||
@@ -740,9 +741,9 @@ def cleanup_custom_field_deletion(sender, instance: CustomField, **kwargs) -> No
|
|||||||
@receiver(models.signals.post_save, sender=Document)
|
@receiver(models.signals.post_save, sender=Document)
|
||||||
def update_llm_suggestions_cache(sender, instance, **kwargs):
|
def update_llm_suggestions_cache(sender, instance, **kwargs):
|
||||||
"""
|
"""
|
||||||
Invalidate the LLM suggestions cache when a document is saved.
|
Invalidate suggestions caches when a document is saved.
|
||||||
"""
|
"""
|
||||||
# Invalidate the cache for the document
|
invalidate_suggestions_cache(instance.pk)
|
||||||
invalidate_llm_suggestions_cache(instance.pk)
|
invalidate_llm_suggestions_cache(instance.pk)
|
||||||
|
|
||||||
|
|
||||||
@@ -984,6 +985,28 @@ def run_workflows(
|
|||||||
"triggers, ignoring",
|
"triggers, ignoring",
|
||||||
extra={"group": logging_group},
|
extra={"group": logging_group},
|
||||||
)
|
)
|
||||||
|
elif (
|
||||||
|
action.type
|
||||||
|
== WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS
|
||||||
|
):
|
||||||
|
if use_overrides:
|
||||||
|
# The document has not been parsed yet, so there is no
|
||||||
|
# content for the LLM to make suggestions from
|
||||||
|
logger.debug(
|
||||||
|
"Apply AI suggestions action does not apply to "
|
||||||
|
"consumption triggers, ignoring",
|
||||||
|
extra={"group": logging_group},
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# Queued rather than run sync
|
||||||
|
from documents.tasks import apply_ai_suggestions
|
||||||
|
|
||||||
|
# kwargs so the PaperlessTask record can note the
|
||||||
|
# document, see _extract_input_data
|
||||||
|
apply_ai_suggestions.delay(
|
||||||
|
action_id=action.pk,
|
||||||
|
document_id=document.pk,
|
||||||
|
)
|
||||||
|
|
||||||
if not use_overrides:
|
if not use_overrides:
|
||||||
# limit title to 128 characters
|
# limit title to 128 characters
|
||||||
@@ -1039,6 +1062,7 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
|
|||||||
"documents.tasks.update_document_content_maybe_archive_file": PaperlessTask.TaskType.REPROCESS_DOCUMENT,
|
"documents.tasks.update_document_content_maybe_archive_file": PaperlessTask.TaskType.REPROCESS_DOCUMENT,
|
||||||
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
|
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
|
||||||
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
|
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
|
||||||
|
"documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
|
||||||
}
|
}
|
||||||
|
|
||||||
_CELERY_STATE_TO_STATUS: dict[str, PaperlessTask.Status] = {
|
_CELERY_STATE_TO_STATUS: dict[str, PaperlessTask.Status] = {
|
||||||
@@ -1092,6 +1116,12 @@ def _extract_input_data(
|
|||||||
return {"account_ids": account_ids}
|
return {"account_ids": account_ids}
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
|
if task_type == PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS:
|
||||||
|
document_id = task_kwargs.get("document_id")
|
||||||
|
if document_id is not None:
|
||||||
|
return {"document_id": document_id}
|
||||||
|
return {}
|
||||||
|
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -71,6 +71,7 @@ from paperless.config import RemoteOCRConfig
|
|||||||
from paperless.logging import consume_task_id
|
from paperless.logging import consume_task_id
|
||||||
from paperless.parsers import ParserContext
|
from paperless.parsers import ParserContext
|
||||||
from paperless.parsers.registry import get_parser_registry
|
from paperless.parsers.registry import get_parser_registry
|
||||||
|
from paperless_ai.exceptions import LLMTimeoutError
|
||||||
from paperless_ai.indexing import llm_index_add_or_update_document
|
from paperless_ai.indexing import llm_index_add_or_update_document
|
||||||
from paperless_ai.indexing import llm_index_remove_document
|
from paperless_ai.indexing import llm_index_remove_document
|
||||||
from paperless_ai.indexing import update_llm_index
|
from paperless_ai.indexing import update_llm_index
|
||||||
@@ -714,6 +715,45 @@ def llmindex_index(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@shared_task(
|
||||||
|
bind=True,
|
||||||
|
autoretry_for=(LLMTimeoutError,),
|
||||||
|
max_retries=3,
|
||||||
|
retry_backoff=60,
|
||||||
|
retry_backoff_max=600,
|
||||||
|
retry_jitter=True,
|
||||||
|
)
|
||||||
|
def apply_ai_suggestions(self, action_id: int, document_id: int) -> None:
|
||||||
|
"""
|
||||||
|
Deferred "apply AI suggestions" workflow action.
|
||||||
|
"""
|
||||||
|
from documents.models import WorkflowAction
|
||||||
|
from documents.workflows.ai import apply_ai_suggestions_to_document
|
||||||
|
|
||||||
|
try:
|
||||||
|
action = WorkflowAction.objects.get(pk=action_id)
|
||||||
|
document = Document.objects.select_related("owner").get(pk=document_id)
|
||||||
|
except (WorkflowAction.DoesNotExist, Document.DoesNotExist):
|
||||||
|
logger.warning(
|
||||||
|
"Workflow action %s or document %s no longer exists, "
|
||||||
|
"not applying AI suggestions",
|
||||||
|
action_id,
|
||||||
|
document_id,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
if not apply_ai_suggestions_to_document(action, document):
|
||||||
|
return
|
||||||
|
|
||||||
|
# No document_updated signal to avoid loop
|
||||||
|
clear_document_caches(document.pk)
|
||||||
|
index_document.delay(document.pk)
|
||||||
|
|
||||||
|
ai_config = AIConfig()
|
||||||
|
if ai_config.llm_index_enabled:
|
||||||
|
update_document_in_llm_index.apply_async(kwargs={"document": document})
|
||||||
|
|
||||||
|
|
||||||
@shared_task
|
@shared_task
|
||||||
def update_document_in_llm_index(document) -> None:
|
def update_document_in_llm_index(document) -> None:
|
||||||
llm_index_add_or_update_document(document)
|
llm_index_add_or_update_document(document)
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ from django.contrib.auth.models import User
|
|||||||
from django.contrib.contenttypes.models import ContentType
|
from django.contrib.contenttypes.models import ContentType
|
||||||
from django.core.exceptions import FieldError
|
from django.core.exceptions import FieldError
|
||||||
from django.core.files.uploadedfile import SimpleUploadedFile
|
from django.core.files.uploadedfile import SimpleUploadedFile
|
||||||
|
from django.test import TestCase as DjangoTestCase
|
||||||
from django.utils import timezone
|
from django.utils import timezone
|
||||||
from rest_framework import status
|
from rest_framework import status
|
||||||
from rest_framework.test import APITestCase
|
from rest_framework.test import APITestCase
|
||||||
@@ -21,6 +22,7 @@ from documents.filters import TitleContentFilter
|
|||||||
from documents.models import Document
|
from documents.models import Document
|
||||||
from documents.tests.utils import DirectoriesMixin
|
from documents.tests.utils import DirectoriesMixin
|
||||||
from documents.tests.utils import read_streaming_response
|
from documents.tests.utils import read_streaming_response
|
||||||
|
from documents.views import DocumentSelectionMixin
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -923,3 +925,36 @@ class TestVersionAwareFilters(TestCase):
|
|||||||
|
|
||||||
self.assertIs(result, queryset)
|
self.assertIs(result, queryset)
|
||||||
queryset.filter.assert_not_called()
|
queryset.filter.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
class TestBulkSelectionExcludesVersions(DjangoTestCase):
|
||||||
|
def test_select_all_matching_does_not_select_version_documents(self) -> None:
|
||||||
|
"""
|
||||||
|
"Select all matching" reconstructs the document list, which never
|
||||||
|
contains version documents as rows of their own.
|
||||||
|
"""
|
||||||
|
user = User.objects.create_superuser(username="bulk_versions")
|
||||||
|
root = Document.objects.create(
|
||||||
|
title="shared-title root",
|
||||||
|
checksum="bulk-root",
|
||||||
|
mime_type="application/pdf",
|
||||||
|
content="root",
|
||||||
|
)
|
||||||
|
Document.objects.create(
|
||||||
|
title="shared-title version",
|
||||||
|
checksum="bulk-version",
|
||||||
|
mime_type="application/pdf",
|
||||||
|
root_document=root,
|
||||||
|
version_index=1,
|
||||||
|
content="version",
|
||||||
|
)
|
||||||
|
|
||||||
|
selected = DocumentSelectionMixin()._resolve_document_ids(
|
||||||
|
user=user,
|
||||||
|
validated_data={
|
||||||
|
"all": True,
|
||||||
|
"filters": {"title__icontains": "shared-title"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(selected, [root.id])
|
||||||
|
|||||||
@@ -93,6 +93,36 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
|
|||||||
self.assertEqual(response.data["count"], 0)
|
self.assertEqual(response.data["count"], 0)
|
||||||
self.assertEqual(len(results), 0)
|
self.assertEqual(len(results), 0)
|
||||||
|
|
||||||
|
def test_search_after_restore_from_trash(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- Indexed document that was moved to the trash
|
||||||
|
WHEN:
|
||||||
|
- The document is restored from the trash
|
||||||
|
THEN:
|
||||||
|
- The document is searchable again without a reindex
|
||||||
|
"""
|
||||||
|
doc = Document.objects.create(
|
||||||
|
title="invoice",
|
||||||
|
content="the thing i bought at a shop and paid with bank account",
|
||||||
|
checksum="A",
|
||||||
|
pk=1,
|
||||||
|
)
|
||||||
|
get_backend().add_or_update(doc)
|
||||||
|
|
||||||
|
self.assertEqual(self.client.get("/api/documents/?query=shop").data["count"], 1)
|
||||||
|
|
||||||
|
self.client.delete(f"/api/documents/{doc.pk}/")
|
||||||
|
self.assertEqual(self.client.get("/api/documents/?query=shop").data["count"], 0)
|
||||||
|
|
||||||
|
response = self.client.post(
|
||||||
|
"/api/trash/",
|
||||||
|
{"action": "restore", "documents": [doc.pk]},
|
||||||
|
)
|
||||||
|
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||||
|
|
||||||
|
self.assertEqual(self.client.get("/api/documents/?query=shop").data["count"], 1)
|
||||||
|
|
||||||
def test_simple_text_search(self) -> None:
|
def test_simple_text_search(self) -> None:
|
||||||
tagged = Tag.objects.create(name="invoice")
|
tagged = Tag.objects.create(name="invoice")
|
||||||
matching_doc = Document.objects.create(
|
matching_doc = Document.objects.create(
|
||||||
|
|||||||
@@ -641,6 +641,145 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
|
|||||||
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
|
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
|
||||||
self.assertEqual(self.workflow.triggers.get(), self.trigger)
|
self.assertEqual(self.workflow.triggers.get(), self.trigger)
|
||||||
|
|
||||||
|
def _post_ai_suggestions_workflow(self, *, trigger_types, action: dict):
|
||||||
|
def trigger(trigger_type):
|
||||||
|
# consumption triggers require a filter of their own
|
||||||
|
if trigger_type == WorkflowTrigger.WorkflowTriggerType.CONSUMPTION:
|
||||||
|
return {"type": trigger_type, "filter_filename": "*.pdf"}
|
||||||
|
return {"type": trigger_type}
|
||||||
|
|
||||||
|
return self.client.post(
|
||||||
|
self.ENDPOINT,
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"name": "Apply AI suggestions",
|
||||||
|
"order": 1,
|
||||||
|
"triggers": [trigger(t) for t in trigger_types],
|
||||||
|
"actions": [
|
||||||
|
{
|
||||||
|
"type": WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
|
||||||
|
**action,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
content_type="application/json",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_api_create_apply_ai_suggestions_action(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- API request to create a workflow with an apply AI suggestions
|
||||||
|
action and a valid set of fields
|
||||||
|
WHEN:
|
||||||
|
- API is called
|
||||||
|
THEN:
|
||||||
|
- The workflow is created with the chosen options
|
||||||
|
"""
|
||||||
|
response = self._post_ai_suggestions_workflow(
|
||||||
|
trigger_types=[WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED],
|
||||||
|
action={
|
||||||
|
"ai_suggestion_fields": ["title", "tags", "correspondent"],
|
||||||
|
"ai_create_missing": True,
|
||||||
|
"ai_overwrite_existing": True,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
|
||||||
|
action = Workflow.objects.get(name="Apply AI suggestions").actions.first()
|
||||||
|
self.assertEqual(
|
||||||
|
action.ai_suggestion_fields,
|
||||||
|
["title", "tags", "correspondent"],
|
||||||
|
)
|
||||||
|
self.assertTrue(action.ai_create_missing)
|
||||||
|
self.assertTrue(action.ai_overwrite_existing)
|
||||||
|
|
||||||
|
def test_api_create_apply_ai_suggestions_action_requires_fields(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- API request to create an apply AI suggestions action with no
|
||||||
|
fields selected, which could never do anything
|
||||||
|
WHEN:
|
||||||
|
- API is called
|
||||||
|
THEN:
|
||||||
|
- Correct HTTP 400 response
|
||||||
|
- No objects are created
|
||||||
|
"""
|
||||||
|
existing_count = Workflow.objects.count()
|
||||||
|
|
||||||
|
response = self._post_ai_suggestions_workflow(
|
||||||
|
trigger_types=[WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED],
|
||||||
|
action={"ai_suggestion_fields": []},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
|
||||||
|
self.assertEqual(Workflow.objects.count(), existing_count)
|
||||||
|
|
||||||
|
def test_api_create_apply_ai_suggestions_action_rejects_unknown_field(
|
||||||
|
self,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- API request to create an apply AI suggestions action naming a
|
||||||
|
field that does not exist
|
||||||
|
WHEN:
|
||||||
|
- API is called
|
||||||
|
THEN:
|
||||||
|
- Correct HTTP 400 response
|
||||||
|
"""
|
||||||
|
response = self._post_ai_suggestions_workflow(
|
||||||
|
trigger_types=[WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED],
|
||||||
|
action={"ai_suggestion_fields": ["title", "not_a_field"]},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
|
||||||
|
|
||||||
|
def test_api_create_apply_ai_suggestions_action_rejects_consumption_only(
|
||||||
|
self,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- API request to create an apply AI suggestions action whose only
|
||||||
|
trigger is consumption started, so there is no document content
|
||||||
|
to make suggestions from yet
|
||||||
|
WHEN:
|
||||||
|
- API is called
|
||||||
|
THEN:
|
||||||
|
- Correct HTTP 400 response
|
||||||
|
- No objects are created
|
||||||
|
"""
|
||||||
|
existing_count = Workflow.objects.count()
|
||||||
|
|
||||||
|
response = self._post_ai_suggestions_workflow(
|
||||||
|
trigger_types=[WorkflowTrigger.WorkflowTriggerType.CONSUMPTION],
|
||||||
|
action={"ai_suggestion_fields": ["title"]},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
|
||||||
|
self.assertEqual(Workflow.objects.count(), existing_count)
|
||||||
|
|
||||||
|
def test_api_create_apply_ai_suggestions_action_allows_extra_consumption_trigger(
|
||||||
|
self,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- API request to create an apply AI suggestions action with a
|
||||||
|
consumption trigger alongside a usable one
|
||||||
|
WHEN:
|
||||||
|
- API is called
|
||||||
|
THEN:
|
||||||
|
- The workflow is created, the action applies to the other trigger
|
||||||
|
"""
|
||||||
|
response = self._post_ai_suggestions_workflow(
|
||||||
|
trigger_types=[
|
||||||
|
WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
|
||||||
|
WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
|
||||||
|
],
|
||||||
|
action={"ai_suggestion_fields": ["title"]},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
|
||||||
|
|
||||||
def test_api_create_workflow_trigger_action_empty_fields(self) -> None:
|
def test_api_create_workflow_trigger_action_empty_fields(self) -> None:
|
||||||
"""
|
"""
|
||||||
GIVEN:
|
GIVEN:
|
||||||
|
|||||||
@@ -385,6 +385,25 @@ class TestTaskFailureHandler:
|
|||||||
task_failure_handler(task_id=None, exception=ValueError("x"), traceback=None)
|
task_failure_handler(task_id=None, exception=ValueError("x"), traceback=None)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
class TestApplyAiSuggestionsTracking:
|
||||||
|
def test_records_the_document_it_is_for(self) -> None:
|
||||||
|
"""
|
||||||
|
The action queues one task per document, so the tracked record notes
|
||||||
|
which document it is for -- otherwise a bulk run is an indistinguishable
|
||||||
|
wall of identical entries in the tasks list.
|
||||||
|
"""
|
||||||
|
task_id = send_publish(
|
||||||
|
"documents.tasks.apply_ai_suggestions",
|
||||||
|
(),
|
||||||
|
{"action_id": 1, "document_id": 42},
|
||||||
|
)
|
||||||
|
|
||||||
|
task = PaperlessTask.objects.get(task_id=task_id)
|
||||||
|
assert task.task_type == PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS
|
||||||
|
assert task.input_data == {"document_id": 42}
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.django_db
|
@pytest.mark.django_db
|
||||||
class TestTaskRevokedHandler:
|
class TestTaskRevokedHandler:
|
||||||
def test_marks_task_revoked(self, mocker: pytest_mock.MockerFixture) -> None:
|
def test_marks_task_revoked(self, mocker: pytest_mock.MockerFixture) -> None:
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ from documents.models import Correspondent
|
|||||||
from documents.models import Document
|
from documents.models import Document
|
||||||
from documents.models import DocumentType
|
from documents.models import DocumentType
|
||||||
from documents.models import Tag
|
from documents.models import Tag
|
||||||
|
from documents.models import WorkflowAction
|
||||||
from documents.sanity_checker import SanityCheckFailedException
|
from documents.sanity_checker import SanityCheckFailedException
|
||||||
from documents.sanity_checker import SanityCheckMessages
|
from documents.sanity_checker import SanityCheckMessages
|
||||||
from documents.tests.test_classifier import dummy_preprocess
|
from documents.tests.test_classifier import dummy_preprocess
|
||||||
@@ -447,3 +448,110 @@ class TestAIIndex(DirectoriesMixin, TestCase):
|
|||||||
rebuild=False,
|
rebuild=False,
|
||||||
document_ids=doc_ids,
|
document_ids=doc_ids,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestApplyAISuggestionsTask(DirectoriesMixin, TestCase):
|
||||||
|
def setUp(self) -> None:
|
||||||
|
super().setUp()
|
||||||
|
self.doc = Document.objects.create(
|
||||||
|
title="doc",
|
||||||
|
content="content",
|
||||||
|
checksum="apply-ai-suggestions",
|
||||||
|
)
|
||||||
|
self.action = WorkflowAction.objects.create(
|
||||||
|
type=WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
|
||||||
|
ai_suggestion_fields=[WorkflowAction.AISuggestionField.TITLE],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_reindexes_without_sending_document_updated(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An apply AI suggestions action that changes the document
|
||||||
|
WHEN:
|
||||||
|
- The task runs
|
||||||
|
THEN:
|
||||||
|
- The search index and caches are refreshed directly, deliberately
|
||||||
|
not via the document_updated signal: that re-runs updated
|
||||||
|
workflows, which for this action means queueing another LLM
|
||||||
|
query for a document it just changed, forever
|
||||||
|
"""
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"documents.workflows.ai.apply_ai_suggestions_to_document",
|
||||||
|
return_value=["title"],
|
||||||
|
),
|
||||||
|
mock.patch("documents.tasks.index_document") as index_document,
|
||||||
|
mock.patch("documents.tasks.clear_document_caches") as clear_caches,
|
||||||
|
mock.patch("documents.tasks.document_updated") as document_updated,
|
||||||
|
):
|
||||||
|
tasks.apply_ai_suggestions(self.action.pk, self.doc.pk)
|
||||||
|
|
||||||
|
index_document.delay.assert_called_once_with(self.doc.pk)
|
||||||
|
clear_caches.assert_called_once_with(self.doc.pk)
|
||||||
|
document_updated.send.assert_not_called()
|
||||||
|
|
||||||
|
def test_no_changes_skips_reindex(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An apply AI suggestions action that changes nothing
|
||||||
|
WHEN:
|
||||||
|
- The task runs
|
||||||
|
THEN:
|
||||||
|
- No reindexing work is queued
|
||||||
|
"""
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"documents.workflows.ai.apply_ai_suggestions_to_document",
|
||||||
|
return_value=[],
|
||||||
|
),
|
||||||
|
mock.patch("documents.tasks.index_document") as index_document,
|
||||||
|
):
|
||||||
|
tasks.apply_ai_suggestions(self.action.pk, self.doc.pk)
|
||||||
|
|
||||||
|
index_document.delay.assert_not_called()
|
||||||
|
|
||||||
|
@override_settings(AI_ENABLED=True, LLM_EMBEDDING_BACKEND="huggingface")
|
||||||
|
def test_updates_llm_index_when_enabled(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An apply AI suggestions action that changes the document
|
||||||
|
- The LLM index is enabled
|
||||||
|
WHEN:
|
||||||
|
- The task runs
|
||||||
|
THEN:
|
||||||
|
- The document is updated in the LLM index too
|
||||||
|
"""
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"documents.workflows.ai.apply_ai_suggestions_to_document",
|
||||||
|
return_value=["title"],
|
||||||
|
),
|
||||||
|
mock.patch("documents.tasks.index_document"),
|
||||||
|
mock.patch(
|
||||||
|
"documents.tasks.update_document_in_llm_index",
|
||||||
|
) as update_in_llm_index,
|
||||||
|
):
|
||||||
|
tasks.apply_ai_suggestions(self.action.pk, self.doc.pk)
|
||||||
|
|
||||||
|
update_in_llm_index.apply_async.assert_called_once()
|
||||||
|
|
||||||
|
def test_deleted_document_is_a_noop(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- A document that was deleted between the workflow running and the
|
||||||
|
queued task starting
|
||||||
|
WHEN:
|
||||||
|
- The task runs
|
||||||
|
THEN:
|
||||||
|
- It logs and exits rather than raising
|
||||||
|
"""
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"documents.workflows.ai.apply_ai_suggestions_to_document",
|
||||||
|
) as apply_suggestions,
|
||||||
|
self.assertLogs("paperless.tasks", level="WARNING") as cm,
|
||||||
|
):
|
||||||
|
tasks.apply_ai_suggestions(self.action.pk, self.doc.pk + 1000)
|
||||||
|
|
||||||
|
apply_suggestions.assert_not_called()
|
||||||
|
self.assertIn("no longer exists", "".join(cm.output))
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from django.conf import settings
|
|||||||
from django.contrib.auth.models import Group
|
from django.contrib.auth.models import Group
|
||||||
from django.contrib.auth.models import Permission
|
from django.contrib.auth.models import Permission
|
||||||
from django.contrib.auth.models import User
|
from django.contrib.auth.models import User
|
||||||
|
from django.core.cache import cache
|
||||||
from django.db import connection
|
from django.db import connection
|
||||||
from django.test import TestCase
|
from django.test import TestCase
|
||||||
from django.test import override_settings
|
from django.test import override_settings
|
||||||
@@ -18,6 +19,7 @@ from guardian.shortcuts import assign_perm
|
|||||||
from rest_framework import status
|
from rest_framework import status
|
||||||
|
|
||||||
from documents.caching import get_llm_suggestion_cache
|
from documents.caching import get_llm_suggestion_cache
|
||||||
|
from documents.caching import get_suggestion_cache_key
|
||||||
from documents.caching import set_llm_suggestions_cache
|
from documents.caching import set_llm_suggestions_cache
|
||||||
from documents.models import Correspondent
|
from documents.models import Correspondent
|
||||||
from documents.models import Document
|
from documents.models import Document
|
||||||
@@ -342,7 +344,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
super().setUp()
|
super().setUp()
|
||||||
|
|
||||||
@patch("documents.views.get_llm_suggestion_cache")
|
@patch("documents.views.get_llm_suggestion_cache")
|
||||||
@patch("documents.views.refresh_suggestions_cache")
|
@patch("documents.views.refresh_llm_suggestions_cache")
|
||||||
@override_settings(
|
@override_settings(
|
||||||
AI_ENABLED=True,
|
AI_ENABLED=True,
|
||||||
LLM_BACKEND="mock_backend",
|
LLM_BACKEND="mock_backend",
|
||||||
@@ -383,12 +385,15 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
self.assertEqual(response.json()["tags"], [self.tag1.pk])
|
self.assertEqual(response.json()["tags"], [self.tag1.pk])
|
||||||
mock_get_cache.assert_called_once_with(
|
mock_get_cache.assert_called_once_with(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
backend="mock_backend",
|
backend=f"mock_backend:user={self.user.pk}",
|
||||||
|
)
|
||||||
|
mock_refresh_cache.assert_called_once_with(
|
||||||
|
self.document.pk,
|
||||||
|
backend=f"mock_backend:user={self.user.pk}",
|
||||||
)
|
)
|
||||||
mock_refresh_cache.assert_called_once_with(self.document.pk)
|
|
||||||
|
|
||||||
@patch("documents.views.get_llm_suggestion_cache")
|
@patch("documents.views.get_llm_suggestion_cache")
|
||||||
@patch("documents.views.refresh_suggestions_cache")
|
@patch("documents.views.refresh_llm_suggestions_cache")
|
||||||
@override_settings(
|
@override_settings(
|
||||||
AI_ENABLED=True,
|
AI_ENABLED=True,
|
||||||
LLM_BACKEND="mock_backend",
|
LLM_BACKEND="mock_backend",
|
||||||
@@ -524,7 +529,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
get_llm_suggestion_cache(
|
get_llm_suggestion_cache(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
backend="mock_backend:de-de",
|
backend=f"mock_backend:de-de:user={self.user.pk}",
|
||||||
).suggestions["title"],
|
).suggestions["title"],
|
||||||
"KI Title",
|
"KI Title",
|
||||||
)
|
)
|
||||||
@@ -563,7 +568,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
get_llm_suggestion_cache(
|
get_llm_suggestion_cache(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
backend="mock_backend:fr-fr",
|
backend=f"mock_backend:fr-fr:user={self.user.pk}",
|
||||||
).suggestions["title"],
|
).suggestions["title"],
|
||||||
"Titre IA",
|
"Titre IA",
|
||||||
)
|
)
|
||||||
@@ -600,7 +605,79 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
self.assertIsNotNone(
|
self.assertIsNotNone(
|
||||||
get_llm_suggestion_cache(
|
get_llm_suggestion_cache(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
backend="mock_backend:model-a:http://endpoint-a",
|
backend=(f"mock_backend:model-a:http://endpoint-a:user={self.user.pk}"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
@patch("documents.views.get_ai_document_classification")
|
||||||
|
@override_settings(
|
||||||
|
AI_ENABLED=True,
|
||||||
|
LLM_BACKEND="mock_backend",
|
||||||
|
)
|
||||||
|
def test_ai_suggestions_cache_variants_coexist_per_requesting_user(
|
||||||
|
self,
|
||||||
|
mock_get_ai_classification,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- One user has populated the document's LLM suggestion cache
|
||||||
|
- A second user requests suggestions for the same document and
|
||||||
|
backend
|
||||||
|
WHEN:
|
||||||
|
- The second request is made
|
||||||
|
THEN:
|
||||||
|
- The first user's prompt-derived result is not reused
|
||||||
|
- The classification runs with the second user's visibility
|
||||||
|
context without evicting the first user's result
|
||||||
|
"""
|
||||||
|
second_user = User.objects.create_superuser(username="second_user")
|
||||||
|
empty_choices = {
|
||||||
|
"tags": {"existing_ids": [], "new_names": []},
|
||||||
|
"correspondents": {"existing_ids": [], "new_names": []},
|
||||||
|
"document_types": {"existing_ids": [], "new_names": []},
|
||||||
|
"storage_paths": {"existing_ids": [], "new_names": []},
|
||||||
|
"dates": [],
|
||||||
|
}
|
||||||
|
mock_get_ai_classification.side_effect = [
|
||||||
|
{"title": "First user's result", **empty_choices},
|
||||||
|
{"title": "Second user's result", **empty_choices},
|
||||||
|
]
|
||||||
|
|
||||||
|
self.client.force_login(user=self.user)
|
||||||
|
first_response = self.client.get(
|
||||||
|
f"/api/documents/{self.document.pk}/ai_suggestions/",
|
||||||
|
)
|
||||||
|
self.client.force_login(user=second_user)
|
||||||
|
second_response = self.client.get(
|
||||||
|
f"/api/documents/{self.document.pk}/ai_suggestions/",
|
||||||
|
)
|
||||||
|
self.client.force_login(user=self.user)
|
||||||
|
first_cached_response = self.client.get(
|
||||||
|
f"/api/documents/{self.document.pk}/ai_suggestions/",
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(first_response.json()["title"], "First user's result")
|
||||||
|
self.assertEqual(second_response.json()["title"], "Second user's result")
|
||||||
|
self.assertEqual(
|
||||||
|
first_cached_response.json()["title"],
|
||||||
|
"First user's result",
|
||||||
|
)
|
||||||
|
self.assertEqual(mock_get_ai_classification.call_count, 2)
|
||||||
|
mock_get_ai_classification.assert_called_with(
|
||||||
|
self.document,
|
||||||
|
second_user,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
self.assertIsNotNone(
|
||||||
|
get_llm_suggestion_cache(
|
||||||
|
self.document.pk,
|
||||||
|
backend=f"mock_backend:user={second_user.pk}",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self.assertIsNotNone(
|
||||||
|
get_llm_suggestion_cache(
|
||||||
|
self.document.pk,
|
||||||
|
backend=f"mock_backend:user={self.user.pk}",
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -786,8 +863,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
self.assertEqual(response.json()["tags"], [])
|
self.assertEqual(response.json()["tags"], [])
|
||||||
self.assertEqual(response.json()["suggested_tags"], [])
|
self.assertEqual(response.json()["suggested_tags"], [])
|
||||||
|
|
||||||
def test_invalidate_suggestions_cache(self) -> None:
|
def test_document_save_invalidates_all_suggestion_caches(self) -> None:
|
||||||
self.client.force_login(user=self.user)
|
|
||||||
suggestions = {
|
suggestions = {
|
||||||
"title": "AI Title",
|
"title": "AI Title",
|
||||||
"tags": ["tag1", "tag2"],
|
"tags": ["tag1", "tag2"],
|
||||||
@@ -796,11 +872,18 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
"storage_paths": ["path1"],
|
"storage_paths": ["path1"],
|
||||||
"dates": ["2023-01-01"],
|
"dates": ["2023-01-01"],
|
||||||
}
|
}
|
||||||
|
standard_cache_key = get_suggestion_cache_key(self.document.pk)
|
||||||
|
cache.set(standard_cache_key, "classifier suggestions")
|
||||||
set_llm_suggestions_cache(
|
set_llm_suggestions_cache(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
suggestions,
|
suggestions,
|
||||||
backend="mock_backend",
|
backend="mock_backend",
|
||||||
)
|
)
|
||||||
|
set_llm_suggestions_cache(
|
||||||
|
self.document.pk,
|
||||||
|
{**suggestions, "title": "Other Variant"},
|
||||||
|
backend="other_backend:user=2",
|
||||||
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
get_llm_suggestion_cache(
|
get_llm_suggestion_cache(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
@@ -808,17 +891,26 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
|
|||||||
).suggestions,
|
).suggestions,
|
||||||
suggestions,
|
suggestions,
|
||||||
)
|
)
|
||||||
# post_save signal triggered
|
self.assertEqual(cache.get(standard_cache_key), "classifier suggestions")
|
||||||
|
|
||||||
update_llm_suggestions_cache(
|
update_llm_suggestions_cache(
|
||||||
sender=None,
|
sender=None,
|
||||||
instance=self.document,
|
instance=self.document,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self.assertIsNone(cache.get(standard_cache_key))
|
||||||
self.assertIsNone(
|
self.assertIsNone(
|
||||||
get_llm_suggestion_cache(
|
get_llm_suggestion_cache(
|
||||||
self.document.pk,
|
self.document.pk,
|
||||||
backend="mock_backend",
|
backend="mock_backend",
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
self.assertIsNone(
|
||||||
|
get_llm_suggestion_cache(
|
||||||
|
self.document.pk,
|
||||||
|
backend="other_backend:user=2",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class TestAIChatStreamingView(DirectoriesMixin, TestCase):
|
class TestAIChatStreamingView(DirectoriesMixin, TestCase):
|
||||||
|
|||||||
@@ -31,7 +31,10 @@ from documents.file_handling import create_source_path_directory
|
|||||||
from documents.file_handling import generate_filename
|
from documents.file_handling import generate_filename
|
||||||
from documents.file_handling import generate_unique_filename
|
from documents.file_handling import generate_unique_filename
|
||||||
from documents.signals.handlers import run_workflows
|
from documents.signals.handlers import run_workflows
|
||||||
|
from documents.workflows.ai import apply_ai_suggestions_to_document
|
||||||
from documents.workflows.webhooks import send_webhook
|
from documents.workflows.webhooks import send_webhook
|
||||||
|
from paperless_ai.base_model import ClassificationSuggestions
|
||||||
|
from paperless_ai.exceptions import LLMTimeoutError
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from django.db.models import QuerySet
|
from django.db.models import QuerySet
|
||||||
@@ -5488,3 +5491,516 @@ class TestRemoteOCRWorkflowAction(DirectoriesMixin, SampleDirMixin, APITestCase)
|
|||||||
)
|
)
|
||||||
|
|
||||||
self.assertIn("only applies to consumption triggers", "".join(cm.output))
|
self.assertIn("only applies to consumption triggers", "".join(cm.output))
|
||||||
|
|
||||||
|
|
||||||
|
SUGGESTIONS: ClassificationSuggestions = {
|
||||||
|
"title": "Suggested Title",
|
||||||
|
"tags": {
|
||||||
|
"existing_ids": [],
|
||||||
|
"new_names": ["Existing Tag", "Suggested Tag"],
|
||||||
|
},
|
||||||
|
"correspondents": {
|
||||||
|
"existing_ids": [],
|
||||||
|
"new_names": ["Existing Correspondent", "Suggested Correspondent"],
|
||||||
|
},
|
||||||
|
"document_types": {
|
||||||
|
"existing_ids": [],
|
||||||
|
"new_names": ["Suggested Document Type"],
|
||||||
|
},
|
||||||
|
"storage_paths": {
|
||||||
|
"existing_ids": [],
|
||||||
|
"new_names": ["Suggested Storage Path"],
|
||||||
|
},
|
||||||
|
"dates": ["2024-03-05"],
|
||||||
|
}
|
||||||
|
|
||||||
|
ALL_SUGGESTION_FIELDS = [
|
||||||
|
WorkflowAction.AISuggestionField.TITLE,
|
||||||
|
WorkflowAction.AISuggestionField.TAGS,
|
||||||
|
WorkflowAction.AISuggestionField.CORRESPONDENT,
|
||||||
|
WorkflowAction.AISuggestionField.DOCUMENT_TYPE,
|
||||||
|
WorkflowAction.AISuggestionField.STORAGE_PATH,
|
||||||
|
WorkflowAction.AISuggestionField.CREATED,
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@override_settings(AI_ENABLED=True)
|
||||||
|
class TestApplyAISuggestionsWorkflowAction(
|
||||||
|
DirectoriesMixin,
|
||||||
|
SampleDirMixin,
|
||||||
|
APITestCase,
|
||||||
|
):
|
||||||
|
def setUp(self) -> None:
|
||||||
|
super().setUp()
|
||||||
|
self.user = User.objects.create(username="ai-user")
|
||||||
|
self.doc = Document.objects.create(
|
||||||
|
title="original.pdf",
|
||||||
|
content="the document content",
|
||||||
|
checksum="ai-suggestions-checksum",
|
||||||
|
mime_type="application/pdf",
|
||||||
|
created=datetime.date(2020, 1, 1),
|
||||||
|
owner=self.user,
|
||||||
|
)
|
||||||
|
|
||||||
|
def make_action(self, **kwargs) -> WorkflowAction:
|
||||||
|
return WorkflowAction.objects.create(
|
||||||
|
type=WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
|
||||||
|
ai_suggestion_fields=kwargs.pop(
|
||||||
|
"ai_suggestion_fields",
|
||||||
|
ALL_SUGGESTION_FIELDS,
|
||||||
|
),
|
||||||
|
**kwargs,
|
||||||
|
)
|
||||||
|
|
||||||
|
def make_workflow(self, action: WorkflowAction, trigger_type) -> Workflow:
|
||||||
|
trigger = WorkflowTrigger.objects.create(type=trigger_type)
|
||||||
|
w = Workflow.objects.create(name="Apply AI suggestions", order=0)
|
||||||
|
w.triggers.add(trigger)
|
||||||
|
w.actions.add(action)
|
||||||
|
w.save()
|
||||||
|
return w
|
||||||
|
|
||||||
|
def apply(
|
||||||
|
self,
|
||||||
|
action: WorkflowAction,
|
||||||
|
suggestions: ClassificationSuggestions = SUGGESTIONS,
|
||||||
|
) -> list[str]:
|
||||||
|
with mock.patch(
|
||||||
|
"documents.workflows.ai.get_ai_document_classification",
|
||||||
|
return_value=suggestions,
|
||||||
|
):
|
||||||
|
changed = apply_ai_suggestions_to_document(action, self.doc)
|
||||||
|
self.doc.refresh_from_db()
|
||||||
|
return changed
|
||||||
|
|
||||||
|
def test_fields_persist_when_tags_are_applied_in_the_same_run(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- A document that already has a filename, as any consumed document does
|
||||||
|
- Suggestions carrying both a document type and tags
|
||||||
|
WHEN:
|
||||||
|
- The suggestions are applied
|
||||||
|
THEN:
|
||||||
|
- The document type is still set after the tags are added
|
||||||
|
|
||||||
|
Adding tags fires m2m_changed, and update_filename_and_move_files
|
||||||
|
refreshes the document from the database. Assigning fields and then
|
||||||
|
adding tags before saving loses those assignments, and only for
|
||||||
|
documents with a filename, so it does not reproduce on a bare
|
||||||
|
Document.objects.create().
|
||||||
|
"""
|
||||||
|
self.doc.filename = "originals/original.pdf"
|
||||||
|
self.doc.save(update_fields=["filename"])
|
||||||
|
|
||||||
|
action = self.make_action(ai_create_missing=True)
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertIn("document_type", changed)
|
||||||
|
self.assertIn("tags", changed)
|
||||||
|
self.assertIsNotNone(
|
||||||
|
self.doc.document_type,
|
||||||
|
"document_type was reported as applied but did not persist",
|
||||||
|
)
|
||||||
|
self.assertEqual(self.doc.document_type.name, "Suggested Document Type")
|
||||||
|
self.assertEqual(self.doc.correspondent.name, "Existing Correspondent")
|
||||||
|
self.assertCountEqual(
|
||||||
|
[t.name for t in self.doc.tags.all()],
|
||||||
|
["Existing Tag", "Suggested Tag"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_document_added_trigger_queues_task(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- A document added workflow with an apply AI suggestions action
|
||||||
|
WHEN:
|
||||||
|
- A matching document is added
|
||||||
|
THEN:
|
||||||
|
- The work is queued rather than run inline, so a slow LLM query
|
||||||
|
cannot stall the rest of the workflow run
|
||||||
|
"""
|
||||||
|
action = self.make_action()
|
||||||
|
self.make_workflow(action, WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED)
|
||||||
|
|
||||||
|
with mock.patch("documents.tasks.apply_ai_suggestions.delay") as delay:
|
||||||
|
run_workflows(
|
||||||
|
WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
|
||||||
|
self.doc,
|
||||||
|
)
|
||||||
|
|
||||||
|
delay.assert_called_once_with(action_id=action.pk, document_id=self.doc.pk)
|
||||||
|
|
||||||
|
def test_consumption_trigger_is_ignored(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- A workflow with an apply AI suggestions action and a consumption
|
||||||
|
trigger alongside a valid one
|
||||||
|
WHEN:
|
||||||
|
- The consumption trigger fires
|
||||||
|
THEN:
|
||||||
|
- The action is skipped, since the document has not been parsed
|
||||||
|
yet and so has no content to make suggestions from
|
||||||
|
"""
|
||||||
|
action = self.make_action()
|
||||||
|
w = self.make_workflow(
|
||||||
|
action,
|
||||||
|
WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
|
||||||
|
)
|
||||||
|
w.triggers.add(
|
||||||
|
WorkflowTrigger.objects.create(
|
||||||
|
type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
test_file = shutil.copy(
|
||||||
|
self.SAMPLE_DIR / "simple.pdf",
|
||||||
|
self.dirs.scratch_dir / "simple.pdf",
|
||||||
|
)
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch("documents.tasks.apply_ai_suggestions.delay") as delay,
|
||||||
|
self.assertLogs("paperless.handlers", level="DEBUG") as cm,
|
||||||
|
):
|
||||||
|
run_workflows(
|
||||||
|
WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
|
||||||
|
ConsumableDocument(
|
||||||
|
source=DocumentSource.ConsumeFolder,
|
||||||
|
original_file=test_file,
|
||||||
|
),
|
||||||
|
overrides=DocumentMetadataOverrides(),
|
||||||
|
)
|
||||||
|
|
||||||
|
delay.assert_not_called()
|
||||||
|
self.assertIn("does not apply to consumption triggers", "".join(cm.output))
|
||||||
|
|
||||||
|
def test_no_selected_fields_does_nothing(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action with no suggestion fields selected
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- Nothing is changed and it is logged
|
||||||
|
"""
|
||||||
|
action = self.make_action(ai_suggestion_fields=[])
|
||||||
|
|
||||||
|
with self.assertLogs("paperless.workflows.ai", level="WARNING") as cm:
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(changed, [])
|
||||||
|
self.assertIn("no AI suggestion fields selected", "".join(cm.output))
|
||||||
|
|
||||||
|
@override_settings(AI_ENABLED=False)
|
||||||
|
def test_ai_disabled_does_nothing(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action on an install where AI has since been disabled
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- Nothing is changed and it is logged
|
||||||
|
"""
|
||||||
|
action = self.make_action()
|
||||||
|
|
||||||
|
with self.assertLogs("paperless.workflows.ai", level="ERROR") as cm:
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(changed, [])
|
||||||
|
self.assertIn("AI is not enabled", "".join(cm.output))
|
||||||
|
|
||||||
|
def test_invalid_configuration_leaves_document_untouched(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An AI backend that is misconfigured
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- The failure is logged and the document is left alone. It is not
|
||||||
|
re-raised, because retrying will not fix a bad configuration
|
||||||
|
"""
|
||||||
|
action = self.make_action()
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"documents.workflows.ai.get_ai_document_classification",
|
||||||
|
side_effect=ValueError("nope"),
|
||||||
|
),
|
||||||
|
self.assertLogs("paperless.workflows.ai", level="ERROR") as cm,
|
||||||
|
):
|
||||||
|
changed = apply_ai_suggestions_to_document(action, self.doc)
|
||||||
|
|
||||||
|
self.assertEqual(changed, [])
|
||||||
|
self.doc.refresh_from_db()
|
||||||
|
self.assertEqual(self.doc.title, "original.pdf")
|
||||||
|
self.assertIn("Invalid AI configuration", "".join(cm.output))
|
||||||
|
|
||||||
|
def test_transient_llm_failure_is_raised_for_retry(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An LLM backend that times out, or rate limits the request
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- The error propagates so the queued task can back off and retry,
|
||||||
|
rather than silently dropping this document's suggestions
|
||||||
|
"""
|
||||||
|
action = self.make_action()
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"documents.workflows.ai.get_ai_document_classification",
|
||||||
|
side_effect=LLMTimeoutError(),
|
||||||
|
),
|
||||||
|
self.assertRaises(LLMTimeoutError),
|
||||||
|
):
|
||||||
|
apply_ai_suggestions_to_document(action, self.doc)
|
||||||
|
|
||||||
|
self.doc.refresh_from_db()
|
||||||
|
self.assertEqual(self.doc.title, "original.pdf")
|
||||||
|
|
||||||
|
def test_only_matching_objects_are_applied(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action without create missing, and only some of the suggested
|
||||||
|
objects existing
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- Only the existing objects are assigned, unmatched suggestions are
|
||||||
|
dropped rather than creating anything
|
||||||
|
"""
|
||||||
|
tag = Tag.objects.create(name="Existing Tag", owner=self.user)
|
||||||
|
correspondent = Correspondent.objects.create(
|
||||||
|
name="Existing Correspondent",
|
||||||
|
owner=self.user,
|
||||||
|
)
|
||||||
|
action = self.make_action(ai_overwrite_existing=True)
|
||||||
|
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(self.doc.correspondent, correspondent)
|
||||||
|
self.assertEqual(list(self.doc.tags.all()), [tag])
|
||||||
|
# Nothing matched for these and create missing is off
|
||||||
|
self.assertIsNone(self.doc.document_type)
|
||||||
|
self.assertIsNone(self.doc.storage_path)
|
||||||
|
self.assertNotIn("document_type", changed)
|
||||||
|
self.assertEqual(Tag.objects.count(), 1)
|
||||||
|
self.assertEqual(Correspondent.objects.count(), 1)
|
||||||
|
|
||||||
|
def test_existing_id_suggestions_are_applied(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- AI suggestions that select existing taxonomy candidates by ID
|
||||||
|
WHEN:
|
||||||
|
- The suggestions are applied
|
||||||
|
THEN:
|
||||||
|
- Each selected object is assigned to the document
|
||||||
|
"""
|
||||||
|
tag = Tag.objects.create(name="Existing Tag", owner=self.user)
|
||||||
|
correspondent = Correspondent.objects.create(
|
||||||
|
name="Existing Correspondent",
|
||||||
|
owner=self.user,
|
||||||
|
)
|
||||||
|
document_type = DocumentType.objects.create(
|
||||||
|
name="Existing Document Type",
|
||||||
|
owner=self.user,
|
||||||
|
)
|
||||||
|
storage_path = StoragePath.objects.create(
|
||||||
|
name="Existing Storage Path",
|
||||||
|
path="{{ title }}",
|
||||||
|
owner=self.user,
|
||||||
|
)
|
||||||
|
action = self.make_action(ai_overwrite_existing=True)
|
||||||
|
suggestions: ClassificationSuggestions = {
|
||||||
|
**SUGGESTIONS,
|
||||||
|
"tags": {"existing_ids": [tag.pk], "new_names": []},
|
||||||
|
"correspondents": {
|
||||||
|
"existing_ids": [correspondent.pk],
|
||||||
|
"new_names": [],
|
||||||
|
},
|
||||||
|
"document_types": {
|
||||||
|
"existing_ids": [document_type.pk],
|
||||||
|
"new_names": [],
|
||||||
|
},
|
||||||
|
"storage_paths": {
|
||||||
|
"existing_ids": [storage_path.pk],
|
||||||
|
"new_names": [],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
changed = self.apply(action, suggestions)
|
||||||
|
|
||||||
|
self.assertEqual(list(self.doc.tags.all()), [tag])
|
||||||
|
self.assertEqual(self.doc.correspondent, correspondent)
|
||||||
|
self.assertEqual(self.doc.document_type, document_type)
|
||||||
|
self.assertEqual(self.doc.storage_path, storage_path)
|
||||||
|
self.assertTrue(
|
||||||
|
{"tags", "correspondent", "document_type", "storage_path"} <= set(changed),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_create_missing_creates_objects_owned_by_document_owner(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action with create missing enabled
|
||||||
|
WHEN:
|
||||||
|
- The action is applied and suggestions match nothing
|
||||||
|
THEN:
|
||||||
|
- Tags, correspondents and document types are created, owned by the
|
||||||
|
document owner so they stay private to them
|
||||||
|
- Storage paths are never created, since a path template cannot be
|
||||||
|
inferred from a name
|
||||||
|
"""
|
||||||
|
action = self.make_action(
|
||||||
|
ai_create_missing=True,
|
||||||
|
ai_overwrite_existing=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
sorted(t.name for t in self.doc.tags.all()),
|
||||||
|
["Existing Tag", "Suggested Tag"],
|
||||||
|
)
|
||||||
|
self.assertEqual(self.doc.correspondent.name, "Existing Correspondent")
|
||||||
|
self.assertEqual(self.doc.correspondent.owner, self.user)
|
||||||
|
self.assertEqual(self.doc.document_type.name, "Suggested Document Type")
|
||||||
|
self.assertEqual(self.doc.document_type.owner, self.user)
|
||||||
|
|
||||||
|
self.assertIsNone(self.doc.storage_path)
|
||||||
|
self.assertFalse(StoragePath.objects.exists())
|
||||||
|
self.assertNotIn("storage_path", changed)
|
||||||
|
|
||||||
|
def test_overwrite_disabled_keeps_existing_values(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action without overwrite existing
|
||||||
|
- A document that already has a title, created date and
|
||||||
|
correspondent
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- The existing values are kept, only the empty document type is
|
||||||
|
filled in
|
||||||
|
"""
|
||||||
|
existing = Correspondent.objects.create(name="Mine", owner=self.user)
|
||||||
|
self.doc.correspondent = existing
|
||||||
|
self.doc.save()
|
||||||
|
action = self.make_action(ai_create_missing=True)
|
||||||
|
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(self.doc.title, "original.pdf")
|
||||||
|
self.assertEqual(self.doc.created, datetime.date(2020, 1, 1))
|
||||||
|
self.assertEqual(self.doc.correspondent, existing)
|
||||||
|
self.assertEqual(self.doc.document_type.name, "Suggested Document Type")
|
||||||
|
self.assertNotIn("title", changed)
|
||||||
|
self.assertNotIn("correspondent", changed)
|
||||||
|
|
||||||
|
def test_overwrite_enabled_replaces_existing_values(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action with overwrite existing
|
||||||
|
- A document that already has a title and created date
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- The suggested values replace them
|
||||||
|
"""
|
||||||
|
action = self.make_action(
|
||||||
|
ai_create_missing=True,
|
||||||
|
ai_overwrite_existing=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(self.doc.title, "Suggested Title")
|
||||||
|
self.assertEqual(self.doc.created, datetime.date(2024, 3, 5))
|
||||||
|
self.assertIn("title", changed)
|
||||||
|
self.assertIn("created", changed)
|
||||||
|
|
||||||
|
def test_tags_are_added_not_replaced(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- A document that already has a tag unrelated to the suggestions
|
||||||
|
WHEN:
|
||||||
|
- The action is applied with overwrite existing enabled
|
||||||
|
THEN:
|
||||||
|
- The existing tag is kept, since suggested tags are always
|
||||||
|
additive regardless of the overwrite setting
|
||||||
|
"""
|
||||||
|
kept = Tag.objects.create(name="Do Not Remove", owner=self.user)
|
||||||
|
self.doc.tags.add(kept)
|
||||||
|
Tag.objects.create(name="Existing Tag", owner=self.user)
|
||||||
|
action = self.make_action(ai_overwrite_existing=True)
|
||||||
|
|
||||||
|
self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
sorted(t.name for t in self.doc.tags.all()),
|
||||||
|
["Do Not Remove", "Existing Tag"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_unselected_fields_are_untouched(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- An action that only selects the title
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- Only the title changes, even though the LLM suggested everything
|
||||||
|
"""
|
||||||
|
action = self.make_action(
|
||||||
|
ai_suggestion_fields=[WorkflowAction.AISuggestionField.TITLE],
|
||||||
|
ai_create_missing=True,
|
||||||
|
ai_overwrite_existing=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
changed = self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(changed, ["title"])
|
||||||
|
self.assertEqual(self.doc.title, "Suggested Title")
|
||||||
|
self.assertEqual(self.doc.tags.count(), 0)
|
||||||
|
self.assertIsNone(self.doc.correspondent)
|
||||||
|
self.assertEqual(self.doc.created, datetime.date(2020, 1, 1))
|
||||||
|
|
||||||
|
def test_another_users_private_objects_are_not_matched(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- A suggested tag name that exists, but is owned by someone else
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- It is not assigned, because the document owner cannot see it
|
||||||
|
"""
|
||||||
|
other = User.objects.create(username="someone-else")
|
||||||
|
Tag.objects.create(name="Existing Tag", owner=other)
|
||||||
|
action = self.make_action(
|
||||||
|
ai_suggestion_fields=[WorkflowAction.AISuggestionField.TAGS],
|
||||||
|
)
|
||||||
|
|
||||||
|
self.apply(action)
|
||||||
|
|
||||||
|
self.assertEqual(self.doc.tags.count(), 0)
|
||||||
|
|
||||||
|
def test_unparsable_dates_are_skipped(self) -> None:
|
||||||
|
"""
|
||||||
|
GIVEN:
|
||||||
|
- Suggested dates that are not all valid
|
||||||
|
WHEN:
|
||||||
|
- The action is applied
|
||||||
|
THEN:
|
||||||
|
- The first usable date is applied and the rest ignored
|
||||||
|
"""
|
||||||
|
action = self.make_action(
|
||||||
|
ai_suggestion_fields=[WorkflowAction.AISuggestionField.CREATED],
|
||||||
|
ai_overwrite_existing=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
with mock.patch(
|
||||||
|
"documents.workflows.ai.get_ai_document_classification",
|
||||||
|
return_value={**SUGGESTIONS, "dates": ["not a date", "2019-07-04"]},
|
||||||
|
):
|
||||||
|
changed = apply_ai_suggestions_to_document(action, self.doc)
|
||||||
|
|
||||||
|
self.doc.refresh_from_db()
|
||||||
|
self.assertEqual(changed, ["created"])
|
||||||
|
self.assertEqual(self.doc.created, datetime.date(2019, 7, 4))
|
||||||
|
|||||||
+29
-20
@@ -113,6 +113,7 @@ from documents.bulk_download import OriginalsOnlyStrategy
|
|||||||
from documents.caching import get_llm_suggestion_cache
|
from documents.caching import get_llm_suggestion_cache
|
||||||
from documents.caching import get_metadata_cache
|
from documents.caching import get_metadata_cache
|
||||||
from documents.caching import get_suggestion_cache
|
from documents.caching import get_suggestion_cache
|
||||||
|
from documents.caching import refresh_llm_suggestions_cache
|
||||||
from documents.caching import refresh_metadata_cache
|
from documents.caching import refresh_metadata_cache
|
||||||
from documents.caching import refresh_suggestions_cache
|
from documents.caching import refresh_suggestions_cache
|
||||||
from documents.caching import set_llm_suggestions_cache
|
from documents.caching import set_llm_suggestions_cache
|
||||||
@@ -247,6 +248,7 @@ from paperless.serialisers import GroupSerializer
|
|||||||
from paperless.serialisers import UserSerializer
|
from paperless.serialisers import UserSerializer
|
||||||
from paperless.views import StandardPagination
|
from paperless.views import StandardPagination
|
||||||
from paperless_ai.ai_classifier import get_ai_document_classification
|
from paperless_ai.ai_classifier import get_ai_document_classification
|
||||||
|
from paperless_ai.ai_classifier import get_llm_output_language
|
||||||
from paperless_ai.chat import stream_chat_with_documents
|
from paperless_ai.chat import stream_chat_with_documents
|
||||||
from paperless_ai.exceptions import LLMTimeoutError
|
from paperless_ai.exceptions import LLMTimeoutError
|
||||||
from paperless_ai.matching import extract_unmatched_names
|
from paperless_ai.matching import extract_unmatched_names
|
||||||
@@ -665,20 +667,6 @@ class TagViewSet(PermissionsAwareDocumentCountMixin, ModelViewSet[Tag]):
|
|||||||
update_document_parent_tags(tag, new_parent)
|
update_document_parent_tags(tag, new_parent)
|
||||||
|
|
||||||
|
|
||||||
def _get_llm_output_language(ai_config: AIConfig, request) -> str | None:
|
|
||||||
output_language = ai_config.llm_output_language
|
|
||||||
if (
|
|
||||||
not output_language
|
|
||||||
and hasattr(request.user, "ui_settings")
|
|
||||||
and isinstance(
|
|
||||||
request.user.ui_settings.settings,
|
|
||||||
dict,
|
|
||||||
)
|
|
||||||
):
|
|
||||||
output_language = request.user.ui_settings.settings.get("language")
|
|
||||||
return output_language
|
|
||||||
|
|
||||||
|
|
||||||
@extend_schema_view(**generate_object_with_permissions_schema(DocumentTypeSerializer))
|
@extend_schema_view(**generate_object_with_permissions_schema(DocumentTypeSerializer))
|
||||||
class DocumentTypeViewSet(
|
class DocumentTypeViewSet(
|
||||||
PermissionsAwareDocumentCountMixin,
|
PermissionsAwareDocumentCountMixin,
|
||||||
@@ -1133,7 +1121,8 @@ class DocumentViewSet(
|
|||||||
"custom_fields",
|
"custom_fields",
|
||||||
queryset=CustomFieldInstance.objects.select_related("field"),
|
queryset=CustomFieldInstance.objects.select_related("field"),
|
||||||
),
|
),
|
||||||
"notes",
|
# NotesSerializer nests the author, this avoids query per note
|
||||||
|
Prefetch("notes", queryset=Note.objects.select_related("user")),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1541,7 +1530,10 @@ class DocumentViewSet(
|
|||||||
if not ai_config.ai_enabled:
|
if not ai_config.ai_enabled:
|
||||||
return HttpResponseBadRequest("AI is required for this feature")
|
return HttpResponseBadRequest("AI is required for this feature")
|
||||||
|
|
||||||
output_language = _get_llm_output_language(ai_config=ai_config, request=request)
|
output_language = get_llm_output_language(
|
||||||
|
ai_config=ai_config,
|
||||||
|
user=request.user,
|
||||||
|
)
|
||||||
llm_cache_backend = ":".join(
|
llm_cache_backend = ":".join(
|
||||||
part
|
part
|
||||||
for part in (
|
for part in (
|
||||||
@@ -1549,6 +1541,7 @@ class DocumentViewSet(
|
|||||||
ai_config.llm_model,
|
ai_config.llm_model,
|
||||||
ai_config.llm_endpoint,
|
ai_config.llm_endpoint,
|
||||||
output_language,
|
output_language,
|
||||||
|
f"user={request.user.pk}",
|
||||||
)
|
)
|
||||||
if part
|
if part
|
||||||
)
|
)
|
||||||
@@ -1564,8 +1557,11 @@ class DocumentViewSet(
|
|||||||
# freshly for this requester on every request, cache hit or not,
|
# freshly for this requester on every request, cache hit or not,
|
||||||
# so a resolved id cached for one user's visibility can never be
|
# so a resolved id cached for one user's visibility can never be
|
||||||
# handed unfiltered to a second, less-privileged requester of
|
# handed unfiltered to a second, less-privileged requester of
|
||||||
# the same (backend-keyed, not user-keyed) cache entry.
|
# the same (backend + user-keyed) cache entry.
|
||||||
refresh_suggestions_cache(doc.pk)
|
refresh_llm_suggestions_cache(
|
||||||
|
doc.pk,
|
||||||
|
backend=llm_cache_backend,
|
||||||
|
)
|
||||||
llm_suggestions = cached_llm_suggestions.suggestions
|
llm_suggestions = cached_llm_suggestions.suggestions
|
||||||
else:
|
else:
|
||||||
try:
|
try:
|
||||||
@@ -2330,7 +2326,10 @@ class ChatStreamingView(GenericAPIView[Any]):
|
|||||||
id__in=permitted_document_ids(request.user),
|
id__in=permitted_document_ids(request.user),
|
||||||
)
|
)
|
||||||
|
|
||||||
output_language = _get_llm_output_language(ai_config=ai_config, request=request)
|
output_language = get_llm_output_language(
|
||||||
|
ai_config=ai_config,
|
||||||
|
user=request.user,
|
||||||
|
)
|
||||||
|
|
||||||
response = StreamingHttpResponse(
|
response = StreamingHttpResponse(
|
||||||
stream_chat_with_documents(
|
stream_chat_with_documents(
|
||||||
@@ -2794,8 +2793,11 @@ class DocumentSelectionMixin:
|
|||||||
for key, value in filters.items()
|
for key, value in filters.items()
|
||||||
if key not in _TANTIVY_SEARCH_PARAM_NAMES
|
if key not in _TANTIVY_SEARCH_PARAM_NAMES
|
||||||
}
|
}
|
||||||
|
# Operations are addressed to roots, a caller that wants
|
||||||
|
# to act on a specific version passes its id explicitly instead
|
||||||
permitted_documents = Document.objects.filter(
|
permitted_documents = Document.objects.filter(
|
||||||
id__in=permitted_document_ids(user),
|
id__in=permitted_document_ids(user),
|
||||||
|
root_document__isnull=True,
|
||||||
)
|
)
|
||||||
# orm-filtered docs
|
# orm-filtered docs
|
||||||
filtered_documents = DocumentFilterSet(
|
filtered_documents = DocumentFilterSet(
|
||||||
@@ -5435,8 +5437,15 @@ class TrashView(ListModelMixin, PassUserMixin):
|
|||||||
return HttpResponseForbidden("Insufficient permissions")
|
return HttpResponseForbidden("Insufficient permissions")
|
||||||
action = serializer.validated_data.get("action")
|
action = serializer.validated_data.get("action")
|
||||||
if action == "restore":
|
if action == "restore":
|
||||||
for doc in Document.deleted_objects.filter(id__in=doc_ids).all():
|
restored = list(Document.deleted_objects.filter(id__in=doc_ids))
|
||||||
|
for doc in restored:
|
||||||
doc.restore(strict=False)
|
doc.restore(strict=False)
|
||||||
|
if restored:
|
||||||
|
from documents.search import get_backend
|
||||||
|
|
||||||
|
with get_backend().batch_update() as batch:
|
||||||
|
for doc in restored:
|
||||||
|
batch.add_or_update(doc)
|
||||||
elif action == "empty":
|
elif action == "empty":
|
||||||
if doc_ids is None:
|
if doc_ids is None:
|
||||||
doc_ids = [doc.id for doc in docs]
|
doc_ids = [doc.id for doc in docs]
|
||||||
|
|||||||
@@ -0,0 +1,263 @@
|
|||||||
|
import logging
|
||||||
|
from datetime import date
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import TypeVar
|
||||||
|
|
||||||
|
from django.contrib.auth.models import User
|
||||||
|
|
||||||
|
from documents.models import Correspondent
|
||||||
|
from documents.models import Document
|
||||||
|
from documents.models import DocumentType
|
||||||
|
from documents.models import MatchingModel
|
||||||
|
from documents.models import StoragePath
|
||||||
|
from documents.models import Tag
|
||||||
|
from documents.models import WorkflowAction
|
||||||
|
from paperless.config import AIConfig
|
||||||
|
from paperless_ai.ai_classifier import get_ai_document_classification
|
||||||
|
from paperless_ai.ai_classifier import get_llm_output_language
|
||||||
|
from paperless_ai.matching import extract_unmatched_names
|
||||||
|
from paperless_ai.matching import match_correspondents_by_name
|
||||||
|
from paperless_ai.matching import match_document_types_by_name
|
||||||
|
from paperless_ai.matching import match_storage_paths_by_name
|
||||||
|
from paperless_ai.matching import match_tags_by_name
|
||||||
|
from paperless_ai.matching import resolve_correspondent_ids
|
||||||
|
from paperless_ai.matching import resolve_document_type_ids
|
||||||
|
from paperless_ai.matching import resolve_storage_path_ids
|
||||||
|
from paperless_ai.matching import resolve_tag_ids
|
||||||
|
|
||||||
|
logger = logging.getLogger("paperless.workflows.ai")
|
||||||
|
|
||||||
|
AISuggestionField = WorkflowAction.AISuggestionField
|
||||||
|
ObjT = TypeVar("ObjT", bound=MatchingModel)
|
||||||
|
|
||||||
|
# Tags use m2m relation instead
|
||||||
|
DIRECT_FIELDS: dict[str, str] = {
|
||||||
|
AISuggestionField.TITLE: "title",
|
||||||
|
AISuggestionField.CORRESPONDENT: "correspondent",
|
||||||
|
AISuggestionField.DOCUMENT_TYPE: "document_type",
|
||||||
|
AISuggestionField.STORAGE_PATH: "storage_path",
|
||||||
|
AISuggestionField.CREATED: "created",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_date(dates: list[str]) -> date | None:
|
||||||
|
"""
|
||||||
|
First usable date out of the suggestions, which are expected as
|
||||||
|
YYYY-MM-DD. Document.created is a DateField, so only one can be applied.
|
||||||
|
"""
|
||||||
|
for value in dates:
|
||||||
|
try:
|
||||||
|
return datetime.strptime(value, "%Y-%m-%d").date()
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
logger.debug("Ignoring unparsable suggested date %s", value)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_object(
|
||||||
|
model: type[ObjT],
|
||||||
|
names: list[str],
|
||||||
|
matched: list[ObjT],
|
||||||
|
*,
|
||||||
|
create_missing: bool,
|
||||||
|
owner: User | None,
|
||||||
|
) -> ObjT | None:
|
||||||
|
"""
|
||||||
|
Single object from a suggestion list. The best match if there was one, else
|
||||||
|
optionally a newly-created object. StoragePaths are excluded.
|
||||||
|
"""
|
||||||
|
if matched:
|
||||||
|
return matched[0]
|
||||||
|
|
||||||
|
if not create_missing or model is StoragePath:
|
||||||
|
return None
|
||||||
|
|
||||||
|
unmatched = extract_unmatched_names(names, matched)
|
||||||
|
if not unmatched:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# (name, owner) is what MatchingModel is unique on
|
||||||
|
obj, created = model.objects.get_or_create(
|
||||||
|
name=unmatched[0][:128],
|
||||||
|
owner=owner,
|
||||||
|
)
|
||||||
|
if created:
|
||||||
|
logger.info("Created %s '%s' from AI suggestion", model.__name__, obj.name)
|
||||||
|
return obj
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_tags(
|
||||||
|
names: list[str],
|
||||||
|
matched: list[Tag],
|
||||||
|
*,
|
||||||
|
create_missing: bool,
|
||||||
|
owner: User | None,
|
||||||
|
) -> list[Tag]:
|
||||||
|
"""
|
||||||
|
Matched tags, plus newly created ones if create_missing is set.
|
||||||
|
"""
|
||||||
|
tags = list(matched)
|
||||||
|
if not create_missing:
|
||||||
|
return tags
|
||||||
|
|
||||||
|
for name in extract_unmatched_names(names, matched):
|
||||||
|
tag, created = Tag.objects.get_or_create(
|
||||||
|
name=name[:128],
|
||||||
|
owner=owner,
|
||||||
|
)
|
||||||
|
if created:
|
||||||
|
logger.info("Created tag '%s' from AI suggestion", tag.name)
|
||||||
|
tags.append(tag)
|
||||||
|
return tags
|
||||||
|
|
||||||
|
|
||||||
|
def apply_ai_suggestions_to_document(
|
||||||
|
action: WorkflowAction,
|
||||||
|
document: Document,
|
||||||
|
logging_group=None,
|
||||||
|
) -> list[str]:
|
||||||
|
"""
|
||||||
|
Get suggestions about `document` and write the chosen fields.
|
||||||
|
|
||||||
|
Returns the names of the fields that were actually changed.
|
||||||
|
"""
|
||||||
|
selected = set(action.ai_suggestion_fields or [])
|
||||||
|
if not selected:
|
||||||
|
logger.warning(
|
||||||
|
"Workflow action %s has no AI suggestion fields selected, skipping",
|
||||||
|
action.pk,
|
||||||
|
extra={"group": logging_group},
|
||||||
|
)
|
||||||
|
return []
|
||||||
|
|
||||||
|
ai_config = AIConfig()
|
||||||
|
if not ai_config.ai_enabled:
|
||||||
|
logger.error(
|
||||||
|
"AI is not enabled, cannot apply AI suggestions for document %s",
|
||||||
|
document.pk,
|
||||||
|
extra={"group": logging_group},
|
||||||
|
)
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Workflows run without a user, so we use the document owner
|
||||||
|
owner = document.owner
|
||||||
|
|
||||||
|
try:
|
||||||
|
suggestions = get_ai_document_classification(
|
||||||
|
document,
|
||||||
|
owner,
|
||||||
|
get_llm_output_language(ai_config, owner),
|
||||||
|
)
|
||||||
|
except ValueError:
|
||||||
|
# A bad AI config will not fix itself, so swallow it rather than
|
||||||
|
# letting the caller retry. Timeouts, rate limits, network errors etc
|
||||||
|
# propagate so the queued task can back off and try again.
|
||||||
|
logger.exception(
|
||||||
|
"Invalid AI configuration, cannot get suggestions for document %s",
|
||||||
|
document.pk,
|
||||||
|
extra={"group": logging_group},
|
||||||
|
)
|
||||||
|
return []
|
||||||
|
|
||||||
|
overwrite = action.ai_overwrite_existing
|
||||||
|
create_missing = action.ai_create_missing
|
||||||
|
updated_fields: list[str] = []
|
||||||
|
|
||||||
|
def should_set(field: str) -> bool:
|
||||||
|
# The field is selected and (overwrite or it's empty)
|
||||||
|
return field in selected and (
|
||||||
|
overwrite or getattr(document, DIRECT_FIELDS[field]) in (None, "")
|
||||||
|
)
|
||||||
|
|
||||||
|
if should_set(AISuggestionField.TITLE):
|
||||||
|
title = suggestions["title"].strip()
|
||||||
|
if title:
|
||||||
|
# title is capped at 128 characters
|
||||||
|
document.title = title[:128]
|
||||||
|
updated_fields.append("title")
|
||||||
|
|
||||||
|
if should_set(AISuggestionField.CORRESPONDENT):
|
||||||
|
choice = suggestions["correspondents"]
|
||||||
|
names = choice["new_names"]
|
||||||
|
correspondent = resolve_object(
|
||||||
|
Correspondent,
|
||||||
|
names,
|
||||||
|
resolve_correspondent_ids(choice["existing_ids"], owner)
|
||||||
|
+ match_correspondents_by_name(names, owner),
|
||||||
|
create_missing=create_missing,
|
||||||
|
owner=owner,
|
||||||
|
)
|
||||||
|
if correspondent:
|
||||||
|
document.correspondent = correspondent
|
||||||
|
updated_fields.append("correspondent")
|
||||||
|
|
||||||
|
if should_set(AISuggestionField.DOCUMENT_TYPE):
|
||||||
|
choice = suggestions["document_types"]
|
||||||
|
names = choice["new_names"]
|
||||||
|
document_type = resolve_object(
|
||||||
|
DocumentType,
|
||||||
|
names,
|
||||||
|
resolve_document_type_ids(choice["existing_ids"], owner)
|
||||||
|
+ match_document_types_by_name(names, owner),
|
||||||
|
create_missing=create_missing,
|
||||||
|
owner=owner,
|
||||||
|
)
|
||||||
|
if document_type:
|
||||||
|
document.document_type = document_type
|
||||||
|
updated_fields.append("document_type")
|
||||||
|
|
||||||
|
if should_set(AISuggestionField.STORAGE_PATH):
|
||||||
|
choice = suggestions["storage_paths"]
|
||||||
|
names = choice["new_names"]
|
||||||
|
storage_path = resolve_object(
|
||||||
|
StoragePath,
|
||||||
|
names,
|
||||||
|
resolve_storage_path_ids(choice["existing_ids"], owner)
|
||||||
|
+ match_storage_paths_by_name(names, owner),
|
||||||
|
create_missing=create_missing,
|
||||||
|
owner=owner,
|
||||||
|
)
|
||||||
|
if storage_path:
|
||||||
|
document.storage_path = storage_path
|
||||||
|
updated_fields.append("storage_path")
|
||||||
|
|
||||||
|
if should_set(AISuggestionField.CREATED):
|
||||||
|
created = resolve_date(suggestions["dates"])
|
||||||
|
if created:
|
||||||
|
document.created = created
|
||||||
|
updated_fields.append("created")
|
||||||
|
|
||||||
|
tags_to_add: list[Tag] = []
|
||||||
|
if AISuggestionField.TAGS in selected:
|
||||||
|
choice = suggestions["tags"]
|
||||||
|
names = choice["new_names"]
|
||||||
|
tags_to_add = resolve_tags(
|
||||||
|
names,
|
||||||
|
resolve_tag_ids(choice["existing_ids"], owner)
|
||||||
|
+ match_tags_by_name(names, owner),
|
||||||
|
create_missing=create_missing,
|
||||||
|
owner=owner,
|
||||||
|
)
|
||||||
|
if tags_to_add:
|
||||||
|
# Suggested tags are always added, so overwrite_existing
|
||||||
|
# does not really apply here
|
||||||
|
updated_fields.append("tags")
|
||||||
|
|
||||||
|
if updated_fields:
|
||||||
|
# save fields and update modified (excluding m2m tags from update_fields)
|
||||||
|
direct_updated_fields = [
|
||||||
|
field for field in updated_fields if field in DIRECT_FIELDS.values()
|
||||||
|
]
|
||||||
|
document.save(update_fields=[*direct_updated_fields, "modified"])
|
||||||
|
|
||||||
|
# Tags at the end so m2m_changed doesn't trigger db and overwrite other changes
|
||||||
|
if tags_to_add:
|
||||||
|
document.add_nested_tags(tags_to_add)
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Applied AI suggestions %s to document %s",
|
||||||
|
updated_fields or "(none)",
|
||||||
|
document.pk,
|
||||||
|
extra={"group": logging_group},
|
||||||
|
)
|
||||||
|
|
||||||
|
return updated_fields
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user