Compare commits

..
10 Commits
Author SHA1 Message Date
shamoon 331fa13088 Fix: drop redundant change perms checks on cards open button (#14601) 2026-10-30 16:16:48 -07:00
shamoon ebddd5687c Chore: fix a couple e2e warnings / flakiness (#14596) 2026-10-30 15:45:46 -07:00
Trenton H 2925b8de81 Chore: Skip repeating Django system checks in the Docker consumer, scheduler and worker (#14598)
The init-system-checks step already runs them once and all three services
depend on init-complete. Measured idle anon PSS saved on
Python 3.14: consumer 14 MiB, beat 26 MiB, worker 27 MiB.
2026-10-30 22:37:37 +00:00
GitHub Actions 5e397e1222 Auto translate strings 2026-10-30 22:26:54 +00:00
Trenton H 6f99ad9469 Chore: Improve idle memory via more lazy imports (#14599)
* Chore: Import dateparser lazily so processes that never parse dates skip it

* Chore: Import turbohtml lazily in the mail parser

* Chore: Import lxml lazily in the SVG validator

* Chore: Import paperless_ai.indexing lazily in document tasks

* Chore: Import paperless_ai helpers lazily in views

* Chore: Read the OCRmyPDF sRGB profile on first use instead of at import

* Chore: Import pikepdf and img2pdf lazily so startup does not load them

* Chore: Reuse the initialized Django ASGI application for HTTP routing
2026-10-30 22:25:42 +00:00
shamoon a5d3247ca7 Tweakhancement: make new pdf controls mostly hidden (#14595) 2026-10-30 10:22:50 -07:00
GitHub Actions 8a67481a9d Auto translate strings 2026-10-30 16:59:59 +00:00
shamoon 652f964f5d Tweak: move preview button in doc list (#14594) 2026-10-30 09:58:29 -07:00
shamoon 80a1827fa0 Fix: checksum light mode display and cf dropdown mobile position (#14590) 2026-10-30 07:03:08 -07:00
github-actions[bot]andshamoon e240c89f27 Documentation: Add v3.4.0 changelog (#14582)
* Changelog v3.4.0 - GHA

* Remove 3.3.4 PR

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-10-29 15:45:43 -07:00
43 changed files with 288 additions and 137 deletions

No files matched your search

@@ -10,9 +10,11 @@ if [[ -n "${PAPERLESS_CONSUMER_DISABLE}" ]]; then
else else
cd ${PAPERLESS_SRC_DIR} cd ${PAPERLESS_SRC_DIR}
# Django system checks already ran once in init-system-checks, skip repeating them
if [[ -n "${USER_IS_NON_ROOT}" ]]; then if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec python3 manage.py document_consumer exec python3 manage.py document_consumer --skip-checks
else else
exec s6-setuidgid paperless python3 manage.py document_consumer exec s6-setuidgid paperless python3 manage.py document_consumer --skip-checks
fi fi
fi fi
@@ -3,8 +3,9 @@
cd ${PAPERLESS_SRC_DIR} cd ${PAPERLESS_SRC_DIR}
# Django system checks already ran once in init-system-checks, skip repeating them
if [[ -n "${USER_IS_NON_ROOT}" ]]; then if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec /usr/local/bin/celery --app paperless beat --loglevel INFO exec /usr/local/bin/celery --skip-checks --app paperless beat --loglevel INFO
else else
exec s6-setuidgid paperless /usr/local/bin/celery --app paperless beat --loglevel INFO exec s6-setuidgid paperless /usr/local/bin/celery --skip-checks --app paperless beat --loglevel INFO
fi fi
@@ -3,8 +3,9 @@
cd ${PAPERLESS_SRC_DIR} cd ${PAPERLESS_SRC_DIR}
# Django system checks already ran once in init-system-checks, skip repeating them
if [[ -n "${USER_IS_NON_ROOT}" ]]; then if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec /usr/local/bin/celery --app paperless worker --loglevel INFO --without-mingle --without-gossip exec /usr/local/bin/celery --skip-checks --app paperless worker --loglevel INFO --without-mingle --without-gossip
else else
exec s6-setuidgid paperless /usr/local/bin/celery --app paperless worker --loglevel INFO --without-mingle --without-gossip exec s6-setuidgid paperless /usr/local/bin/celery --skip-checks --app paperless worker --loglevel INFO --without-mingle --without-gossip
fi fi
+86
View File
@@ -1,5 +1,91 @@
# Changelog # Changelog
## paperless-ngx 3.4.0
### Features / Enhancements
- Enhancement: support custom fields in workflow title templates [@shamoon](https://github.com/shamoon) ([#14564](https://github.com/paperless-ngx/paperless-ngx/pull/14564))
- Enhancement: relative date custom field queries [@shamoon](https://github.com/shamoon) ([#14549](https://github.com/paperless-ngx/paperless-ngx/pull/14549))
- Performance: reduce tasks badge load times [@JanKleine](https://github.com/JanKleine) ([#14552](https://github.com/paperless-ngx/paperless-ngx/pull/14552))
- Enhancement: skip local 2FA for trusted OIDC AMR [@shamoon](https://github.com/shamoon) ([#14403](https://github.com/paperless-ngx/paperless-ngx/pull/14403))
- Feature: Generate PDF thumbnails via poppler utils [@stumpylog](https://github.com/stumpylog) ([#14420](https://github.com/paperless-ngx/paperless-ngx/pull/14420))
- Enhancement: add upload count to button, link to tray, reduce notifications [@shamoon](https://github.com/shamoon) ([#14501](https://github.com/paperless-ngx/paperless-ngx/pull/14501))
- Enhancement: notifications improvements [@shamoon](https://github.com/shamoon) ([#14500](https://github.com/paperless-ngx/paperless-ngx/pull/14500))
- Enhancement: doc details ux improvements [@shamoon](https://github.com/shamoon) ([#14485](https://github.com/paperless-ngx/paperless-ngx/pull/14485))
- Enhancement (QoL): allow independently toggling child + parent tags in WF tag fields [@shamoon](https://github.com/shamoon) ([#14400](https://github.com/paperless-ngx/paperless-ngx/pull/14400))
- Tweak: drop the user dropdown caret [@shamoon](https://github.com/shamoon) ([#14444](https://github.com/paperless-ngx/paperless-ngx/pull/14444))
- Enhancement: navbar upload button [@shamoon](https://github.com/shamoon) ([#14442](https://github.com/paperless-ngx/paperless-ngx/pull/14442))
- Enhancement: AI chat UX tweaks [@shamoon](https://github.com/shamoon) ([#14415](https://github.com/paperless-ngx/paperless-ngx/pull/14415))
- Enhancement: support markdown in AI chat [@shamoon](https://github.com/shamoon) ([#14409](https://github.com/paperless-ngx/paperless-ngx/pull/14409))
### Bug Fixes
- Fix: truncate long version labels in dropdown [@shamoon](https://github.com/shamoon) ([#14578](https://github.com/paperless-ngx/paperless-ngx/pull/14578))
- Fixhancement: show better result for revoked / expired tasks [@shamoon](https://github.com/shamoon) ([#14566](https://github.com/paperless-ngx/paperless-ngx/pull/14566))
- Fix: move re-index after workflow runs on doc edit [@shamoon](https://github.com/shamoon) ([#14558](https://github.com/paperless-ngx/paperless-ngx/pull/14558))
- Fix: folder export with --delete fails on a stale manifest.json.tmp [@benpetty](https://github.com/benpetty) ([#14526](https://github.com/paperless-ngx/paperless-ngx/pull/14526))
- Fix: fix list type for api schema document object filters [@shamoon](https://github.com/shamoon) ([#14524](https://github.com/paperless-ngx/paperless-ngx/pull/14524))
- Fix: standardize some dropdown sizing [@shamoon](https://github.com/shamoon) ([#14510](https://github.com/paperless-ngx/paperless-ngx/pull/14510))
- Fix: prevent doc permissions queries on schema view [@shamoon](https://github.com/shamoon) ([#14503](https://github.com/paperless-ngx/paperless-ngx/pull/14503))
- Fix: dont defer tag assignment when barcode will not result in a split [@shamoon](https://github.com/shamoon) ([#14497](https://github.com/paperless-ngx/paperless-ngx/pull/14497))
### Dependencies
<details>
<summary>6 changes</summary>
- Chore: bump some frontend deps for security fixes [@shamoon](https://github.com/shamoon) ([#14581](https://github.com/paperless-ngx/paperless-ngx/pull/14581))
- Chore: bump brotli, djangorestframework, filelock, msgpack, setuptools for security fixes [@stumpylog](https://github.com/stumpylog) ([#14575](https://github.com/paperless-ngx/paperless-ngx/pull/14575))
- Chore: bump django-allauth to 65.20 and oauthlib to 4.0 [@stumpylog](https://github.com/stumpylog) ([#14570](https://github.com/paperless-ngx/paperless-ngx/pull/14570))
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14517](https://github.com/paperless-ngx/paperless-ngx/pull/14517))
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14493](https://github.com/paperless-ngx/paperless-ngx/pull/14493))
- Chore: Drop ImageMagik as a dependency [@stumpylog](https://github.com/stumpylog) ([#14440](https://github.com/paperless-ngx/paperless-ngx/pull/14440))
</details>
### All App Changes
<details>
<summary>35 changes</summary>
- Chore: Consolidate test sample files [@stumpylog](https://github.com/stumpylog) ([#14430](https://github.com/paperless-ngx/paperless-ngx/pull/14430))
- Chore: bump some frontend deps for security fixes [@shamoon](https://github.com/shamoon) ([#14581](https://github.com/paperless-ngx/paperless-ngx/pull/14581))
- Enhancement: support custom fields in workflow title templates [@shamoon](https://github.com/shamoon) ([#14564](https://github.com/paperless-ngx/paperless-ngx/pull/14564))
- Enhancement: relative date custom field queries [@shamoon](https://github.com/shamoon) ([#14549](https://github.com/paperless-ngx/paperless-ngx/pull/14549))
- Fix: truncate long version labels in dropdown [@shamoon](https://github.com/shamoon) ([#14578](https://github.com/paperless-ngx/paperless-ngx/pull/14578))
- Chore: bump brotli, djangorestframework, filelock, msgpack, setuptools for security fixes [@stumpylog](https://github.com/stumpylog) ([#14575](https://github.com/paperless-ngx/paperless-ngx/pull/14575))
- Chore: bump django-allauth to 65.20 and oauthlib to 4.0 [@stumpylog](https://github.com/stumpylog) ([#14570](https://github.com/paperless-ngx/paperless-ngx/pull/14570))
- Fixhancement: show better result for revoked / expired tasks [@shamoon](https://github.com/shamoon) ([#14566](https://github.com/paperless-ngx/paperless-ngx/pull/14566))
- Fix: move re-index after workflow runs on doc edit [@shamoon](https://github.com/shamoon) ([#14558](https://github.com/paperless-ngx/paperless-ngx/pull/14558))
- Performance: reduce tasks badge load times [@JanKleine](https://github.com/JanKleine) ([#14552](https://github.com/paperless-ngx/paperless-ngx/pull/14552))
- Enhancement: skip local 2FA for trusted OIDC AMR [@shamoon](https://github.com/shamoon) ([#14403](https://github.com/paperless-ngx/paperless-ngx/pull/14403))
- Chore: consolidate direct permission grant lookups into one helper [@stumpylog](https://github.com/stumpylog) ([#14456](https://github.com/paperless-ngx/paperless-ngx/pull/14456))
- Chore: drop get\_groups\_with\_only\_permission [@stumpylog](https://github.com/stumpylog) ([#14454](https://github.com/paperless-ngx/paperless-ngx/pull/14454))
- Chore: Consolidate to a single way to set permission on an object [@stumpylog](https://github.com/stumpylog) ([#14453](https://github.com/paperless-ngx/paperless-ngx/pull/14453))
- Chore: Refactor bulk edit to remove PDF knowledge [@stumpylog](https://github.com/stumpylog) ([#14445](https://github.com/paperless-ngx/paperless-ngx/pull/14445))
- Fix: folder export with --delete fails on a stale manifest.json.tmp [@benpetty](https://github.com/benpetty) ([#14526](https://github.com/paperless-ngx/paperless-ngx/pull/14526))
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14517](https://github.com/paperless-ngx/paperless-ngx/pull/14517))
- Fix: fix list type for api schema document object filters [@shamoon](https://github.com/shamoon) ([#14524](https://github.com/paperless-ngx/paperless-ngx/pull/14524))
- Chore(deps): Bump the uv group across 1 directory with 2 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14493](https://github.com/paperless-ngx/paperless-ngx/pull/14493))
- Chore: drop get\_objects\_for\_user\_owner\_aware [@stumpylog](https://github.com/stumpylog) ([#14457](https://github.com/paperless-ngx/paperless-ngx/pull/14457))
- Chore: update Angular components to 22.2.2 [@shamoon](https://github.com/shamoon) ([#14515](https://github.com/paperless-ngx/paperless-ngx/pull/14515))
- Chore: share one base class for the method-mapped permission classes [@stumpylog](https://github.com/stumpylog) ([#14458](https://github.com/paperless-ngx/paperless-ngx/pull/14458))
- Fix: standardize some dropdown sizing [@shamoon](https://github.com/shamoon) ([#14510](https://github.com/paperless-ngx/paperless-ngx/pull/14510))
- Chore: Drop ImageMagik as a dependency [@stumpylog](https://github.com/stumpylog) ([#14440](https://github.com/paperless-ngx/paperless-ngx/pull/14440))
- Feature: Generate PDF thumbnails via poppler utils [@stumpylog](https://github.com/stumpylog) ([#14420](https://github.com/paperless-ngx/paperless-ngx/pull/14420))
- Fix: prevent doc permissions queries on schema view [@shamoon](https://github.com/shamoon) ([#14503](https://github.com/paperless-ngx/paperless-ngx/pull/14503))
- Enhancement: add upload count to button, link to tray, reduce notifications [@shamoon](https://github.com/shamoon) ([#14501](https://github.com/paperless-ngx/paperless-ngx/pull/14501))
- Enhancement: notifications improvements [@shamoon](https://github.com/shamoon) ([#14500](https://github.com/paperless-ngx/paperless-ngx/pull/14500))
- Enhancement: doc details ux improvements [@shamoon](https://github.com/shamoon) ([#14485](https://github.com/paperless-ngx/paperless-ngx/pull/14485))
- Enhancement (QoL): allow independently toggling child + parent tags in WF tag fields [@shamoon](https://github.com/shamoon) ([#14400](https://github.com/paperless-ngx/paperless-ngx/pull/14400))
- Tweak: drop the user dropdown caret [@shamoon](https://github.com/shamoon) ([#14444](https://github.com/paperless-ngx/paperless-ngx/pull/14444))
- Enhancement: navbar upload button [@shamoon](https://github.com/shamoon) ([#14442](https://github.com/paperless-ngx/paperless-ngx/pull/14442))
- Enhancement: AI chat UX tweaks [@shamoon](https://github.com/shamoon) ([#14415](https://github.com/paperless-ngx/paperless-ngx/pull/14415))
- Enhancement: support markdown in AI chat [@shamoon](https://github.com/shamoon) ([#14409](https://github.com/paperless-ngx/paperless-ngx/pull/14409))
- Fix: dont defer tag assignment when barcode will not result in a split [@shamoon](https://github.com/shamoon) ([#14497](https://github.com/paperless-ngx/paperless-ngx/pull/14497))
</details>
## paperless-ngx 3.3.4 ## paperless-ngx 3.3.4
### All App Changes ### All App Changes
+9 -1
View File
@@ -43,6 +43,7 @@ def configure_environment(instance_root: Path) -> None:
def seed_database() -> None: def seed_database() -> None:
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.core.management import call_command from django.core.management import call_command
from django.utils import timezone from django.utils import timezone
@@ -63,11 +64,18 @@ def seed_database() -> None:
username="playwright", username="playwright",
password="playwright", # NOSONAR password="playwright", # NOSONAR
) )
User.objects.create_user( viewer = User.objects.create_user(
username="viewer", username="viewer",
password="viewer", # NOSONAR password="viewer", # NOSONAR
) )
viewer.user_permissions.add(
Permission.objects.get(
content_type__app_label="documents",
codename="view_uisettings",
),
)
inbox = Tag.objects.create(name="Inbox", is_inbox_tag=True, owner=admin) inbox = Tag.objects.create(name="Inbox", is_inbox_tag=True, owner=admin)
quick_filter = Tag.objects.create(name="Another Sample Tag", owner=admin) quick_filter = Tag.objects.create(name="Another Sample Tag", owner=admin)
Tag.objects.create(name="TagWithPartial", owner=admin) Tag.objects.create(name="TagWithPartial", owner=admin)
@@ -80,12 +80,21 @@ test('should show a list of notes', async ({ page }) => {
test('should support quick filters', async ({ page }) => { test('should support quick filters', async ({ page }) => {
await page.goto('/documents/1/details') await page.goto('/documents/1/details')
// filter button is available before tags have loaded
const selectedTags = page.locator('pngx-input-tags .ng-value pngx-tag')
await expect(selectedTags).toHaveCount(2)
await expect(
selectedTags.getByText('Another Sample Tag', { exact: true })
).toBeVisible()
await expect(selectedTags.getByText('Inbox', { exact: true })).toBeVisible()
await page await page
.getByRole('button', { name: 'Filter documents with these Tags' }) .getByRole('button', { name: 'Filter documents with these Tags' })
.click() .click()
await expect(page).toHaveURL( await expect(page).toHaveURL((url) => {
/tags__id__all=2,1&sort=created&reverse=1&page=1/ const tags = url.searchParams.get('tags__id__all')?.split(',').sort()
) return url.pathname === '/documents' && tags?.join(',') === '1,2'
})
await expect(page.locator('pngx-document-list')).toHaveText(/one document/i)
}) })
test('should finish reloading the preview after a remote document update', async ({ test('should finish reloading the preview after a remote document update', async ({
@@ -6,8 +6,15 @@ test.use({
}, },
}) })
test('should not allow user to edit settings', async ({ page }) => { test.beforeEach(async ({ page }) => {
const settingsLoaded = page.waitForResponse((response) =>
response.url().endsWith('/api/ui_settings/')
)
await page.goto('/dashboard') await page.goto('/dashboard')
expect((await settingsLoaded).status()).toBe(200)
})
test('should not allow user to edit settings', async ({ page }) => {
await expect(page.getByRole('link', { name: 'Settings' })).not.toBeAttached() await expect(page.getByRole('link', { name: 'Settings' })).not.toBeAttached()
await page.goto('/settings') await page.goto('/settings')
await expect(page.locator('body')).toHaveText( await expect(page.locator('body')).toHaveText(
@@ -16,7 +23,6 @@ test('should not allow user to edit settings', async ({ page }) => {
}) })
test('should not allow user to view documents', async ({ page }) => { test('should not allow user to view documents', async ({ page }) => {
await page.goto('/dashboard')
await expect( await expect(
page.locator('nav').getByRole('link', { name: 'Documents' }) page.locator('nav').getByRole('link', { name: 'Documents' })
).not.toBeAttached() ).not.toBeAttached()
@@ -31,7 +37,6 @@ test('should not allow user to view documents', async ({ page }) => {
}) })
test('should not allow user to view correspondents', async ({ page }) => { test('should not allow user to view correspondents', async ({ page }) => {
await page.goto('/dashboard')
await expect( await expect(
page.getByRole('link', { name: 'Attributes' }) page.getByRole('link', { name: 'Attributes' })
).not.toBeAttached() ).not.toBeAttached()
@@ -42,7 +47,6 @@ test('should not allow user to view correspondents', async ({ page }) => {
}) })
test('should not allow user to view tags', async ({ page }) => { test('should not allow user to view tags', async ({ page }) => {
await page.goto('/dashboard')
await expect( await expect(
page.getByRole('link', { name: 'Attributes' }) page.getByRole('link', { name: 'Attributes' })
).not.toBeAttached() ).not.toBeAttached()
@@ -53,7 +57,6 @@ test('should not allow user to view tags', async ({ page }) => {
}) })
test('should not allow user to view document types', async ({ page }) => { test('should not allow user to view document types', async ({ page }) => {
await page.goto('/dashboard')
await expect( await expect(
page.getByRole('link', { name: 'Attributes' }) page.getByRole('link', { name: 'Attributes' })
).not.toBeAttached() ).not.toBeAttached()
@@ -64,7 +67,6 @@ test('should not allow user to view document types', async ({ page }) => {
}) })
test('should not allow user to view storage paths', async ({ page }) => { test('should not allow user to view storage paths', async ({ page }) => {
await page.goto('/dashboard')
await expect( await expect(
page.getByRole('link', { name: 'Attributes' }) page.getByRole('link', { name: 'Attributes' })
).not.toBeAttached() ).not.toBeAttached()
@@ -75,7 +77,6 @@ test('should not allow user to view storage paths', async ({ page }) => {
}) })
test('should not allow user to view logs', async ({ page }) => { test('should not allow user to view logs', async ({ page }) => {
await page.goto('/dashboard')
await expect(page.getByRole('link', { name: 'Logs' })).not.toBeAttached() await expect(page.getByRole('link', { name: 'Logs' })).not.toBeAttached()
await page.goto('/logs') await page.goto('/logs')
await expect(page.locator('body')).toHaveText( await expect(page.locator('body')).toHaveText(
@@ -84,7 +85,6 @@ test('should not allow user to view logs', async ({ page }) => {
}) })
test('should not allow user to view tasks', async ({ page }) => { test('should not allow user to view tasks', async ({ page }) => {
await page.goto('/dashboard')
await expect(page.getByRole('link', { name: 'Tasks' })).not.toBeAttached() await expect(page.getByRole('link', { name: 'Tasks' })).not.toBeAttached()
await page.goto('/tasks') await page.goto('/tasks')
await expect(page.locator('body')).toHaveText( await expect(page.locator('body')).toHaveText(
+5 -5
View File
@@ -10334,18 +10334,18 @@
<context context-type="linenumber">294,295</context> <context context-type="linenumber">294,295</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="2179847500064178686" datatype="html"> <trans-unit id="3420321797707163677" datatype="html">
<source>Edit document</source> <source>Preview document</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">328</context> <context context-type="linenumber">328</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3420321797707163677" datatype="html"> <trans-unit id="2179847500064178686" datatype="html">
<source>Preview document</source> <source>Edit document</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">329</context> <context context-type="linenumber">331</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4512084577073831437" datatype="html"> <trans-unit id="4512084577073831437" datatype="html">
@@ -1,10 +1,3 @@
.custom-fields-dropdown { .custom-fields-dropdown {
min-width: 300px; min-width: 300px;
// correct position on mobile
@media (max-width: 575.98px) {
&.show {
margin-left: -245px !important;
}
}
} }
@@ -308,7 +308,7 @@
</tr> </tr>
<tr class="align-middle"> <tr class="align-middle">
<td i18n>Original SHA256 checksum</td> <td i18n>Original SHA256 checksum</td>
<td><span class="badge bg-light font-monospace text-wrap text-break text-start">{{metadata()?.original_checksum}}</span></td> <td><span class="badge bg-light text-dark font-monospace text-wrap text-break text-start">{{metadata()?.original_checksum}}</span></td>
</tr> </tr>
<tr> <tr>
<td i18n>Original file size</td> <td i18n>Original file size</td>
@@ -465,7 +465,7 @@
@switch (archiveContentRenderType) { @switch (archiveContentRenderType) {
@case (ContentRenderType.PDF) { @case (ContentRenderType.PDF) {
@if (!useNativePdfViewer) { @if (!useNativePdfViewer) {
<div class="preview-sticky pdf-viewer-container"> <div class="preview-sticky pdf-viewer-container" tabindex="-1">
<pngx-pdf-viewer <pngx-pdf-viewer
[src]="pdfSource()" [src]="pdfSource()"
[sourceRevision]="previewRevision()" [sourceRevision]="previewRevision()"
@@ -477,7 +477,7 @@
(loadError)="onError($event)" (loadError)="onError($event)"
(afterLoadComplete)="pdfPreviewLoaded($event)"> (afterLoadComplete)="pdfPreviewLoaded($event)">
</pngx-pdf-viewer> </pngx-pdf-viewer>
<div class="position-absolute bottom-0 start-50 translate-middle-x z-1 mb-3 d-flex gap-2 p-1 bg-body border rounded shadow"> <div class="preview-controls position-absolute bottom-0 start-50 translate-middle-x z-1 mb-3 d-flex gap-2 p-1 bg-body border rounded shadow">
@if (previewNumPages()) { @if (previewNumPages()) {
<div class="input-group input-group-sm w-auto flex-nowrap"> <div class="input-group input-group-sm w-auto flex-nowrap">
<input class="form-control flex-grow-0 w-auto" type="number" min="1" [max]="previewNumPages()" [ngModelOptions]="{standalone: true}" [ngModel]="previewCurrentPage()" (ngModelChange)="previewCurrentPage.set($event)" aria-label="Page" i18n-aria-label /> <input class="form-control flex-grow-0 w-auto" type="number" min="1" [max]="previewNumPages()" [ngModelOptions]="{standalone: true}" [ngModel]="previewCurrentPage()" (ngModelChange)="previewCurrentPage.set($event)" aria-label="Page" i18n-aria-label />
@@ -12,6 +12,22 @@
width: 100%; width: 100%;
height: 100%; height: 100%;
} }
&:focus {
outline: none;
}
.preview-controls {
opacity: 0;
pointer-events: none;
transition: opacity 0.15s ease-in-out;
}
&:hover .preview-controls,
&:focus-within .preview-controls {
opacity: 1;
pointer-events: auto;
}
} }
.btn-group .dropdown-toggle-split { .btn-group .dropdown-toggle-split {
@@ -72,7 +72,7 @@
<a class="btn btn-sm btn-outline-secondary" (click)="clickMoreLike.emit()"> <a class="btn btn-sm btn-outline-secondary" (click)="clickMoreLike.emit()">
<i-bs name="diagram-3" class="me-1"></i-bs><span class="d-none d-md-inline" i18n>More like this</span> <i-bs name="diagram-3" class="me-1"></i-bs><span class="d-none d-md-inline" i18n>More like this</span>
</a> </a>
<a routerLink="/documents/{{document().id}}" class="btn btn-sm btn-outline-secondary" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }"> <a routerLink="/documents/{{document().id}}" class="btn btn-sm btn-outline-secondary">
<i-bs name="file-earmark-richtext" class="me-1"></i-bs><span class="d-none d-md-inline" i18n>Open</span> <i-bs name="file-earmark-richtext" class="me-1"></i-bs><span class="d-none d-md-inline" i18n>Open</span>
</a> </a>
<pngx-preview-popup [document]="document()" #popupPreview> <pngx-preview-popup [document]="document()" #popupPreview>
@@ -10,7 +10,6 @@ import {
NgbTooltipModule, NgbTooltipModule,
} from '@ng-bootstrap/ng-bootstrap' } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe' import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe' import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
import { IsNumberPipe } from 'src/app/pipes/is-number.pipe' import { IsNumberPipe } from 'src/app/pipes/is-number.pipe'
@@ -52,7 +51,6 @@ describe('DocumentCardLargeComponent', () => {
DocumentCardLargeComponent, DocumentCardLargeComponent,
DocumentTitlePipe, DocumentTitlePipe,
CustomDatePipe, CustomDatePipe,
IfPermissionsDirective,
SafeUrlPipe, SafeUrlPipe,
IsNumberPipe, IsNumberPipe,
PreviewPopupComponent, PreviewPopupComponent,
@@ -20,7 +20,6 @@ import {
Document, Document,
} from 'src/app/data/document' } from 'src/app/data/document'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings' import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe' import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe' import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe' import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -51,7 +50,6 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
CorrespondentNamePipe, CorrespondentNamePipe,
DocumentTypeNamePipe, DocumentTypeNamePipe,
StoragePathNamePipe, StoragePathNamePipe,
IfPermissionsDirective,
CustomDatePipe, CustomDatePipe,
RouterModule, RouterModule,
NgbTooltipModule, NgbTooltipModule,
@@ -144,7 +144,7 @@
<div class="d-flex justify-content-between align-items-center"> <div class="d-flex justify-content-between align-items-center">
<div class="btn-group w-100"> <div class="btn-group w-100">
@if (document()) { @if (document()) {
<a routerLink="/documents/{{document().id}}" class="btn btn-sm btn-outline-secondary" title="Open" i18n-title *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }" i18n-title> <a routerLink="/documents/{{document().id}}" class="btn btn-sm btn-outline-secondary" title="Open" i18n-title>
<i-bs name="file-earmark-richtext"></i-bs> <i-bs name="file-earmark-richtext"></i-bs>
</a> </a>
<pngx-preview-popup [document]="document()" #popupPreview> <pngx-preview-popup [document]="document()" #popupPreview>
@@ -20,7 +20,6 @@ import {
Document, Document,
} from 'src/app/data/document' } from 'src/app/data/document'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings' import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe' import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe' import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe' import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -51,7 +50,6 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
CorrespondentNamePipe, CorrespondentNamePipe,
DocumentTypeNamePipe, DocumentTypeNamePipe,
StoragePathNamePipe, StoragePathNamePipe,
IfPermissionsDirective,
CustomDatePipe, CustomDatePipe,
RouterModule, RouterModule,
NgbTooltipModule, NgbTooltipModule,
@@ -208,7 +208,7 @@
i18n>Correspondent</th> i18n>Correspondent</th>
} }
@if (activeDisplayFields.includes(DisplayField.TITLE)) { @if (activeDisplayFields.includes(DisplayField.TITLE)) {
<th class="cursor-pointer" <th class="cursor-pointer ps-5"
pngxSortable="title" pngxSortable="title"
title="Sort by title" i18n-title title="Sort by title" i18n-title
[currentSortField]="list.sortField" [currentSortField]="list.sortField"
@@ -322,13 +322,13 @@
</td> </td>
} }
@if (activeDisplayFields.includes(DisplayField.TITLE) || activeDisplayFields.includes(DisplayField.TAGS)) { @if (activeDisplayFields.includes(DisplayField.TITLE) || activeDisplayFields.includes(DisplayField.TAGS)) {
<td width="30%"> <td width="30%" class="position-relative" [class.ps-5]="activeDisplayFields.includes(DisplayField.TITLE)">
@if (activeDisplayFields.includes(DisplayField.TITLE)) { @if (activeDisplayFields.includes(DisplayField.TITLE)) {
<div class="d-inline-block" (mouseleave)="popupPreview.close()"> <div class="d-inline-block" (mouseleave)="popupPreview.close()">
<a routerLink="/documents/{{d.id}}" title="Edit document" i18n-title style="overflow-wrap: anywhere;">{{d.title | documentTitle}}</a> <pngx-preview-popup class="position-absolute start-0" [document]="d" linkClasses="btn btn-sm btn-link text-secondary" linkTitle="Preview document" (click)="$event.stopPropagation()" i18n-linkTitle #popupPreview>
<pngx-preview-popup [document]="d" linkClasses="btn btn-sm btn-link text-secondary" linkTitle="Preview document" (click)="$event.stopPropagation()" i18n-linkTitle #popupPreview>
<i-bs name="eye"></i-bs> <i-bs name="eye"></i-bs>
</pngx-preview-popup> </pngx-preview-popup>
<a routerLink="/documents/{{d.id}}" title="Edit document" i18n-title style="overflow-wrap: anywhere;">{{d.title | documentTitle}}</a>
</div> </div>
} }
@if (activeDisplayFields.includes(DisplayField.TAGS)) { @if (activeDisplayFields.includes(DisplayField.TAGS)) {
@@ -87,6 +87,10 @@ $paperless-card-breakpoints: (
} }
} }
.table .ps-5 {
padding-left: 2.2rem !important;
}
a { a {
cursor: pointer; cursor: pointer;
} }
+7 -3
View File
@@ -10,9 +10,6 @@ from typing import TYPE_CHECKING
import regex as regex_mod import regex as regex_mod
from django.conf import settings from django.conf import settings
from pdf2image import convert_from_path from pdf2image import convert_from_path
from pikepdf import Page
from pikepdf import PasswordError
from pikepdf import Pdf
from documents.converters import convert_from_tiff_to_pdf from documents.converters import convert_from_tiff_to_pdf
from documents.data_models import ConsumableDocument from documents.data_models import ConsumableDocument
@@ -34,6 +31,7 @@ from documents.utils import maybe_override_pixel_limit
from paperless.config import BarcodeConfig from paperless.config import BarcodeConfig
if TYPE_CHECKING: if TYPE_CHECKING:
from pikepdf import Page
from PIL import Image from PIL import Image
logger = logging.getLogger("paperless.barcodes") logger = logging.getLogger("paperless.barcodes")
@@ -282,6 +280,8 @@ class BarcodePlugin(ConsumeTaskPlugin):
# No op if not a TIFF # No op if not a TIFF
self.convert_from_tiff_to_pdf() self.convert_from_tiff_to_pdf()
from pikepdf import PasswordError
try: try:
self.barcodes = scan_pdf( self.barcodes = scan_pdf(
self.pdf_file, self.pdf_file,
@@ -433,6 +433,8 @@ class BarcodePlugin(ConsumeTaskPlugin):
document_paths = [] document_paths = []
fname: str = self.input_doc.original_file.stem fname: str = self.input_doc.original_file.stem
from pikepdf import Pdf
with Pdf.open(self.pdf_file) as input_pdf: with Pdf.open(self.pdf_file) as input_pdf:
# Start with an empty document # Start with an empty document
current_document: list[Page] = [] current_document: list[Page] = []
@@ -516,6 +518,8 @@ def scan_pdf(pdf_path: Path, settings: BarcodeConfig, work_dir: Path) -> list[Ba
""" """
barcodes: list[Barcode] = [] barcodes: list[Barcode] = []
from pikepdf import Pdf
with Pdf.open(pdf_path) as pdf: with Pdf.open(pdf_path) as pdf:
num_of_pages = len(pdf.pages) num_of_pages = len(pdf.pages)
logger.debug(f"PDF has {num_of_pages} pages") logger.debug(f"PDF has {num_of_pages} pages")
+2 -1
View File
@@ -1,6 +1,5 @@
from pathlib import Path from pathlib import Path
import img2pdf
from PIL import Image from PIL import Image
from documents.utils import copy_basic_file_stats from documents.utils import copy_basic_file_stats
@@ -17,6 +16,8 @@ def convert_from_tiff_to_pdf(tiff_path: Path, target_directory: Path) -> Path:
Returns the path of the PDF created. Returns the path of the PDF created.
""" """
import img2pdf
# override pixel setting if needed # override pixel setting if needed
maybe_override_pixel_limit() maybe_override_pixel_limit()
+2 -1
View File
@@ -6,7 +6,6 @@ from pathlib import Path
from typing import Final from typing import Final
from django.conf import settings from django.conf import settings
from pikepdf import Pdf
from documents.consumer import ConsumerError from documents.consumer import ConsumerError
from documents.converters import convert_from_tiff_to_pdf from documents.converters import convert_from_tiff_to_pdf
@@ -85,6 +84,8 @@ class CollatePlugin(NoCleanupPluginMixin, NoSetupPluginMixin, ConsumeTaskPlugin)
valid_staging_exists = True valid_staging_exists = True
if valid_staging_exists: if valid_staging_exists:
from pikepdf import Pdf
try: try:
# Collate pages from second PDF in reverse order # Collate pages from second PDF in reverse order
with Pdf.open(staging) as pdf1, Pdf.open(pdf_file) as pdf2: with Pdf.open(staging) as pdf1, Pdf.open(pdf_file) as pdf2:
+2 -1
View File
@@ -11,7 +11,6 @@ from decimal import InvalidOperation
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from typing import Any from typing import Any
import dateparser
from django.conf import settings from django.conf import settings
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from django.db.models import Case from django.db.models import Case
@@ -350,6 +349,8 @@ class DateQueryField(serializers.DateField):
if not isinstance(value, str): if not isinstance(value, str):
raise raise
import dateparser
try: try:
parsed = dateparser.parse( parsed = dateparser.parse(
value, value,
+14 -2
View File
@@ -14,8 +14,6 @@ from __future__ import annotations
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from typing import NamedTuple from typing import NamedTuple
import pikepdf
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
from collections.abc import Iterable from collections.abc import Iterable
@@ -43,6 +41,8 @@ def rotate_pdf(src: Path, dst: Path, degrees: int) -> None:
Rotate every page relatively on the opened document, not a rebuild, so Info, Rotate every page relatively on the opened document, not a rebuild, so Info,
XMP and outlines are kept. ``src`` is not modified. XMP and outlines are kept. ``src`` is not modified.
""" """
import pikepdf
with pikepdf.open(src) as pdf: with pikepdf.open(src) as pdf:
for page in pdf.pages: for page in pdf.pages:
page.rotate(degrees, relative=True) page.rotate(degrees, relative=True)
@@ -58,6 +58,8 @@ def remove_pages(src: Path, dst: Path, pages: Iterable[int]) -> None:
Duplicates are ignored. Pages are removed highest first so earlier removals Duplicates are ignored. Pages are removed highest first so earlier removals
never shift the index of later ones. never shift the index of later ones.
""" """
import pikepdf
unique = sorted(set(pages)) unique = sorted(set(pages))
_require_positive(unique) _require_positive(unique)
with pikepdf.open(src) as pdf: with pikepdf.open(src) as pdf:
@@ -81,6 +83,8 @@ def build_pdfs(
(Info, XMP, outlines) is not carried over. Returns the written paths in output (Info, XMP, outlines) is not carried over. Returns the written paths in output
order. order.
""" """
import pikepdf
for specs, _ in outputs: for specs, _ in outputs:
_require_positive(spec.page for spec in specs) _require_positive(spec.page for spec in specs)
@@ -140,6 +144,8 @@ def needs_decrypt(src: Path) -> bool:
True if ``src`` is encrypted. A PDF that needs a password to open at all True if ``src`` is encrypted. A PDF that needs a password to open at all
counts as encrypted. counts as encrypted.
""" """
import pikepdf
try: try:
with pikepdf.open(src) as pdf: with pikepdf.open(src) as pdf:
return bool(pdf.is_encrypted) return bool(pdf.is_encrypted)
@@ -154,6 +160,8 @@ def decrypt_pdf(src: Path, make_dst: Callable[[], Path], password: str) -> Path:
``make_dst`` is only called once the password has been accepted, so a wrong ``make_dst`` is only called once the password has been accepted, so a wrong
password never leaves a destination behind. password never leaves a destination behind.
""" """
import pikepdf
with pikepdf.open(src, password=password) as pdf: with pikepdf.open(src, password=password) as pdf:
pdf.remove_unreferenced_resources() pdf.remove_unreferenced_resources()
dst = make_dst() dst = make_dst()
@@ -170,6 +178,8 @@ class PdfMerger:
""" """
def __init__(self) -> None: def __init__(self) -> None:
import pikepdf
self._pdf = pikepdf.new() self._pdf = pikepdf.new()
self._version: str = self._pdf.pdf_version self._version: str = self._pdf.pdf_version
@@ -185,6 +195,8 @@ class PdfMerger:
self._pdf.close() self._pdf.close()
def add(self, path: Path) -> None: def add(self, path: Path) -> None:
import pikepdf
with pikepdf.open(str(path)) as pdf: with pikepdf.open(str(path)) as pdf:
self._version = max(self._version, pdf.pdf_version) self._version = max(self._version, pdf.pdf_version)
self._pdf.pages.extend(pdf.pages) self._pdf.pages.extend(pdf.pages)
+2 -2
View File
@@ -7,8 +7,6 @@ from dataclasses import dataclass
from types import TracebackType from types import TracebackType
from typing import Self from typing import Self
import dateparser
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -80,6 +78,8 @@ class DateParserPluginBase(ABC):
Uses configuration from `self.config`. Uses configuration from `self.config`.
""" """
import dateparser
try: try:
return dateparser.parse( return dateparser.parse(
date_string, date_string,
+8 -3
View File
@@ -76,9 +76,6 @@ from paperless.logging import consume_task_id
from paperless.parsers import ParserContext from paperless.parsers import ParserContext
from paperless.parsers.registry import get_parser_registry from paperless.parsers.registry import get_parser_registry
from paperless_ai.exceptions import LLMTimeoutError from paperless_ai.exceptions import LLMTimeoutError
from paperless_ai.indexing import llm_index_add_or_update_document
from paperless_ai.indexing import llm_index_remove_document
from paperless_ai.indexing import update_llm_index
if settings.AUDIT_LOG_ENABLED: if settings.AUDIT_LOG_ENABLED:
from auditlog.models import LogEntry from auditlog.models import LogEntry
@@ -339,6 +336,8 @@ def bulk_update_documents(document_ids) -> None:
ai_config = AIConfig() ai_config = AIConfig()
if ai_config.llm_index_enabled: if ai_config.llm_index_enabled:
from paperless_ai.indexing import update_llm_index
update_llm_index( update_llm_index(
rebuild=False, rebuild=False,
document_ids=document_ids, document_ids=document_ids,
@@ -499,6 +498,8 @@ def update_document_content_maybe_archive_file(
ai_config = AIConfig() ai_config = AIConfig()
if ai_config.llm_index_enabled: if ai_config.llm_index_enabled:
from paperless_ai.indexing import llm_index_add_or_update_document
llm_index_add_or_update_document(document) llm_index_add_or_update_document(document)
clear_document_caches(document.pk) clear_document_caches(document.pk)
@@ -800,11 +801,15 @@ def apply_ai_suggestions(self, action_id: int, document_id: int) -> None:
@shared_task @shared_task
def update_document_in_llm_index(document) -> None: def update_document_in_llm_index(document) -> None:
from paperless_ai.indexing import llm_index_add_or_update_document
llm_index_add_or_update_document(document) llm_index_add_or_update_document(document)
@shared_task @shared_task
def remove_document_from_llm_index(document) -> None: def remove_document_from_llm_index(document) -> None:
from paperless_ai.indexing import llm_index_remove_document
llm_index_remove_document(document) llm_index_remove_document(document)
@@ -71,7 +71,7 @@ class TestParseString:
): ):
# We still need to mock dateparser.parse to force the exception # We still need to mock dateparser.parse to force the exception
mocker.patch( mocker.patch(
"documents.plugins.date_parsing.base.dateparser.parse", "dateparser.parse",
side_effect=ValueError( side_effect=ValueError(
"Parsing error: 01/01/2024", "Parsing error: 01/01/2024",
), ),
@@ -260,7 +260,7 @@ class TestRegexDateParser:
parser = RegexDateParserPlugin(base_config) parser = RegexDateParserPlugin(base_config)
# Patch the dateparser.parse # Patch the dateparser.parse
target = "documents.plugins.date_parsing.base.dateparser.parse" target = "dateparser.parse"
def fake_parse( def fake_parse(
date_string: str, date_string: str,
@@ -335,7 +335,7 @@ class TestRegexDateParser:
) )
parser = RegexDateParserPlugin(cfg) parser = RegexDateParserPlugin(cfg)
target = "documents.plugins.date_parsing.base.dateparser.parse" target = "dateparser.parse"
def fake_parse( def fake_parse(
date_string: str, date_string: str,
@@ -400,7 +400,7 @@ class TestRegexDateParser:
parser = RegexDateParserPlugin(cfg) parser = RegexDateParserPlugin(cfg)
# Patch the module's dateparser.parse so we can inspect calls # Patch the module's dateparser.parse so we can inspect calls
target = "documents.plugins.date_parsing.base.dateparser.parse" target = "dateparser.parse"
def fake_parse( def fake_parse(
date_string: str, date_string: str,
+5 -5
View File
@@ -1070,7 +1070,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
with ( with (
patch("documents.tasks.llmindex_index.apply_async") as mock_update, patch("documents.tasks.llmindex_index.apply_async") as mock_update,
patch("paperless.views.llm_index_exists") as mock_exists, patch("paperless_ai.indexing.llm_index_exists") as mock_exists,
): ):
mock_exists.return_value = False mock_exists.return_value = False
self.client.patch( self.client.patch(
@@ -1095,7 +1095,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
with ( with (
patch("documents.tasks.llmindex_index.apply_async") as mock_update, patch("documents.tasks.llmindex_index.apply_async") as mock_update,
patch("paperless.views.llm_index_exists", return_value=False), patch("paperless_ai.indexing.llm_index_exists", return_value=False),
): ):
self.client.patch( self.client.patch(
f"{self.ENDPOINT}1/", f"{self.ENDPOINT}1/",
@@ -1115,7 +1115,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
with ( with (
patch("documents.tasks.llmindex_index.apply_async") as mock_update, patch("documents.tasks.llmindex_index.apply_async") as mock_update,
patch("paperless.views.llm_index_exists") as mock_exists, patch("paperless_ai.indexing.llm_index_exists") as mock_exists,
): ):
mock_exists.return_value = True mock_exists.return_value = True
self.client.patch( self.client.patch(
@@ -1136,7 +1136,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
with ( with (
patch("documents.tasks.llmindex_index.apply_async") as mock_update, patch("documents.tasks.llmindex_index.apply_async") as mock_update,
patch("paperless.views.llm_index_exists") as mock_exists, patch("paperless_ai.indexing.llm_index_exists") as mock_exists,
): ):
mock_exists.return_value = True mock_exists.return_value = True
self.client.patch( self.client.patch(
@@ -1174,7 +1174,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
with ( with (
patch("documents.tasks.llmindex_index.apply_async") as mock_update, patch("documents.tasks.llmindex_index.apply_async") as mock_update,
patch("paperless.views.llm_index_exists") as mock_exists, patch("paperless_ai.indexing.llm_index_exists") as mock_exists,
): ):
mock_exists.return_value = True mock_exists.return_value = True
self.client.patch( self.client.patch(
+2 -2
View File
@@ -61,7 +61,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
return_value=self._mock_ai_enabled(), return_value=self._mock_ai_enabled(),
), ),
mock.patch( mock.patch(
"documents.views.stream_chat_with_documents", "paperless_ai.chat.stream_chat_with_documents",
return_value=iter(chunks), return_value=iter(chunks),
), ),
): ):
@@ -104,7 +104,7 @@ class TestChatStreamingViewUnrestrictedFlag:
""" """
mocker.patch("documents.views.AIConfig").return_value.ai_enabled = True mocker.patch("documents.views.AIConfig").return_value.ai_enabled = True
return mocker.patch( return mocker.patch(
"documents.views.stream_chat_with_documents", "paperless_ai.chat.stream_chat_with_documents",
return_value=iter(()), return_value=iter(()),
) )
@@ -1110,7 +1110,7 @@ class TestVersionActionPermissions(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, 403) self.assertEqual(response.status_code, 403)
with ( with (
mock.patch("documents.views.AIConfig") as ai_config, mock.patch("documents.views.AIConfig") as ai_config,
mock.patch("documents.views.stream_chat_with_documents") as chat, mock.patch("paperless_ai.chat.stream_chat_with_documents") as chat,
): ):
ai_config.return_value.ai_enabled = True ai_config.return_value.ai_enabled = True
response = self.client.post( response = self.client.post(
+1 -1
View File
@@ -2686,7 +2686,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
response = self.client.get("/api/documents/34676/suggestions/") response = self.client.get("/api/documents/34676/suggestions/")
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND) self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
@mock.patch("documents.views.get_ai_document_classification") @mock.patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
def test_suggestions_still_uses_classifier_when_ai_enabled( def test_suggestions_still_uses_classifier_when_ai_enabled(
self, self,
@@ -572,7 +572,7 @@ class TestAiChatAllDocumentsPermissionBoundary:
ENDPOINT = "/api/documents/chat/" ENDPOINT = "/api/documents/chat/"
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
@patch("documents.views.stream_chat_with_documents") @patch("paperless_ai.chat.stream_chat_with_documents")
def test_chat_all_documents_excludes_unshared_document(self, mock_stream_chat): def test_chat_all_documents_excludes_unshared_document(self, mock_stream_chat):
mock_stream_chat.return_value = iter([b"data"]) mock_stream_chat.return_value = iter([b"data"])
+4 -4
View File
@@ -322,7 +322,7 @@ class TestUpdateContent(DirectoriesMixin, TestCase):
) )
@override_settings(AI_ENABLED=True, LLM_EMBEDDING_BACKEND="huggingface") @override_settings(AI_ENABLED=True, LLM_EMBEDDING_BACKEND="huggingface")
@mock.patch("documents.tasks.llm_index_add_or_update_document") @mock.patch("paperless_ai.indexing.llm_index_add_or_update_document")
@mock.patch("documents.search.get_backend") @mock.patch("documents.search.get_backend")
def test_update_content_version_updates_llm_index( def test_update_content_version_updates_llm_index(
self, self,
@@ -450,7 +450,7 @@ class TestAIIndex(DirectoriesMixin, TestCase):
checksum="wow", checksum="wow",
) )
with mock.patch( with mock.patch(
"documents.tasks.llm_index_add_or_update_document", "paperless_ai.indexing.llm_index_add_or_update_document",
) as llm_index_add_or_update_document: ) as llm_index_add_or_update_document:
tasks.update_document_in_llm_index(doc) tasks.update_document_in_llm_index(doc)
llm_index_add_or_update_document.assert_called_once_with(doc) llm_index_add_or_update_document.assert_called_once_with(doc)
@@ -470,7 +470,7 @@ class TestAIIndex(DirectoriesMixin, TestCase):
checksum="wow", checksum="wow",
) )
with mock.patch( with mock.patch(
"documents.tasks.llm_index_remove_document", "paperless_ai.indexing.llm_index_remove_document",
) as llm_index_remove_document: ) as llm_index_remove_document:
tasks.remove_document_from_llm_index(doc) tasks.remove_document_from_llm_index(doc)
llm_index_remove_document.assert_called_once_with(doc) llm_index_remove_document.assert_called_once_with(doc)
@@ -495,7 +495,7 @@ class TestAIIndex(DirectoriesMixin, TestCase):
"documents.tasks.update_document_in_llm_index", "documents.tasks.update_document_in_llm_index",
) as update_document_in_llm_index, ) as update_document_in_llm_index,
mock.patch( mock.patch(
"documents.tasks.update_llm_index", "paperless_ai.indexing.update_llm_index",
) as update_llm_index, ) as update_llm_index,
): ):
doc_ids = [doc.pk for doc in docs] doc_ids = [doc.pk for doc in docs]
+16 -16
View File
@@ -474,7 +474,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
self.assertEqual(response.json()["tags"], []) self.assertEqual(response.json()["tags"], [])
self.assertEqual(response.json()["suggested_tags"], []) self.assertEqual(response.json()["suggested_tags"], [])
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -524,7 +524,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
None, None,
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -562,7 +562,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
"KI Title", "KI Title",
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -601,7 +601,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
"Titre IA", "Titre IA",
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -637,7 +637,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
), ),
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -709,7 +709,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
), ),
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="openai-like", LLM_BACKEND="openai-like",
@@ -738,7 +738,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
get_llm_suggestion_cache(self.document.pk, backend="openai-like"), get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="openai-like", LLM_BACKEND="openai-like",
@@ -765,7 +765,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
get_llm_suggestion_cache(self.document.pk, backend="openai-like"), get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="openai-like", LLM_BACKEND="openai-like",
@@ -797,7 +797,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
get_llm_suggestion_cache(self.document.pk, backend="openai-like"), get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
) )
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -835,7 +835,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
self.assertEqual(response.json()["tags"], [self.tag1.pk]) self.assertEqual(response.json()["tags"], [self.tag1.pk])
self.assertEqual(response.json()["suggested_tags"], ["Follow-up"]) self.assertEqual(response.json()["suggested_tags"], ["Follow-up"])
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -874,7 +874,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
self.assertEqual(response.json()["tags"], [self.tag1.pk]) self.assertEqual(response.json()["tags"], [self.tag1.pk])
self.assertEqual(response.json()["suggested_tags"], []) self.assertEqual(response.json()["suggested_tags"], [])
@patch("documents.views.get_ai_document_classification") @patch("paperless_ai.ai_classifier.get_ai_document_classification")
@override_settings( @override_settings(
AI_ENABLED=True, AI_ENABLED=True,
LLM_BACKEND="mock_backend", LLM_BACKEND="mock_backend",
@@ -996,7 +996,7 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
self.assertEqual(response.status_code, 400) self.assertEqual(response.status_code, 400)
self.assertIn(b"AI is required for this feature", response.content) self.assertIn(b"AI is required for this feature", response.content)
@patch("documents.views.stream_chat_with_documents") @patch("paperless_ai.chat.stream_chat_with_documents")
@patch("documents.views.permitted_document_ids") @patch("documents.views.permitted_document_ids")
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
def test_post_no_document_id(self, mock_permitted_ids, mock_stream_chat) -> None: def test_post_no_document_id(self, mock_permitted_ids, mock_stream_chat) -> None:
@@ -1016,7 +1016,7 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
self.assertEqual(list(call_kwargs["documents"]), [self.document]) self.assertEqual(list(call_kwargs["documents"]), [self.document])
self.assertIsNone(call_kwargs["output_language"]) self.assertIsNone(call_kwargs["output_language"])
@patch("documents.views.stream_chat_with_documents") @patch("paperless_ai.chat.stream_chat_with_documents")
@patch("documents.views.permitted_document_ids") @patch("documents.views.permitted_document_ids")
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
def test_post_uses_user_display_language( def test_post_uses_user_display_language(
@@ -1042,7 +1042,7 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
self.assertEqual(list(call_kwargs["documents"]), [self.document]) self.assertEqual(list(call_kwargs["documents"]), [self.document])
self.assertEqual(call_kwargs["output_language"], "de-de") self.assertEqual(call_kwargs["output_language"], "de-de")
@patch("documents.views.stream_chat_with_documents") @patch("paperless_ai.chat.stream_chat_with_documents")
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
def test_post_with_document_id(self, mock_stream_chat) -> None: def test_post_with_document_id(self, mock_stream_chat) -> None:
self.grant_view_document_permission() self.grant_view_document_permission()
@@ -1079,7 +1079,7 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
self.assertEqual(response.status_code, 403) self.assertEqual(response.status_code, 403)
self.assertIn(b"Insufficient permissions", response.content) self.assertIn(b"Insufficient permissions", response.content)
@patch("documents.views.stream_chat_with_documents") @patch("paperless_ai.chat.stream_chat_with_documents")
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
def test_post_no_document_id_requires_view_document_permission( def test_post_no_document_id_requires_view_document_permission(
self, self,
@@ -1093,7 +1093,7 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
self.assertEqual(response.status_code, 403) self.assertEqual(response.status_code, 403)
mock_stream_chat.assert_not_called() mock_stream_chat.assert_not_called()
@patch("documents.views.stream_chat_with_documents") @patch("paperless_ai.chat.stream_chat_with_documents")
@override_settings(AI_ENABLED=True) @override_settings(AI_ENABLED=True)
def test_post_with_document_id_requires_view_document_permission( def test_post_with_document_id_requires_view_document_permission(
self, self,
+15 -12
View File
@@ -250,20 +250,8 @@ from paperless.parsers.remote import RemoteEngineConfig
from paperless.serialisers import GroupSerializer from paperless.serialisers import GroupSerializer
from paperless.serialisers import UserSerializer from paperless.serialisers import UserSerializer
from paperless.views import StandardPagination from paperless.views import StandardPagination
from paperless_ai.ai_classifier import get_ai_document_classification
from paperless_ai.ai_classifier import get_llm_output_language
from paperless_ai.chat import stream_chat_with_documents
from paperless_ai.exceptions import LLMProviderError from paperless_ai.exceptions import LLMProviderError
from paperless_ai.exceptions import LLMTimeoutError from paperless_ai.exceptions import LLMTimeoutError
from paperless_ai.matching import extract_unmatched_names
from paperless_ai.matching import match_correspondents_by_name
from paperless_ai.matching import match_document_types_by_name
from paperless_ai.matching import match_storage_paths_by_name
from paperless_ai.matching import match_tags_by_name
from paperless_ai.matching import resolve_correspondent_ids
from paperless_ai.matching import resolve_document_type_ids
from paperless_ai.matching import resolve_storage_path_ids
from paperless_ai.matching import resolve_tag_ids
from paperless_mail.models import MailAccount from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule from paperless_mail.models import MailRule
from paperless_mail.oauth import PaperlessMailOAuth2Manager from paperless_mail.oauth import PaperlessMailOAuth2Manager
@@ -1613,6 +1601,18 @@ class DocumentViewSet(
if not ai_config.ai_enabled: if not ai_config.ai_enabled:
return HttpResponseBadRequest("AI is required for this feature") return HttpResponseBadRequest("AI is required for this feature")
from paperless_ai.ai_classifier import get_ai_document_classification
from paperless_ai.ai_classifier import get_llm_output_language
from paperless_ai.matching import extract_unmatched_names
from paperless_ai.matching import match_correspondents_by_name
from paperless_ai.matching import match_document_types_by_name
from paperless_ai.matching import match_storage_paths_by_name
from paperless_ai.matching import match_tags_by_name
from paperless_ai.matching import resolve_correspondent_ids
from paperless_ai.matching import resolve_document_type_ids
from paperless_ai.matching import resolve_storage_path_ids
from paperless_ai.matching import resolve_tag_ids
output_language = get_llm_output_language( output_language = get_llm_output_language(
ai_config=ai_config, ai_config=ai_config,
user=request.user, user=request.user,
@@ -2446,6 +2446,9 @@ class ChatStreamingView(GenericAPIView[Any]):
) )
unrestricted = user_is_unrestricted(request.user) unrestricted = user_is_unrestricted(request.user)
from paperless_ai.ai_classifier import get_llm_output_language
from paperless_ai.chat import stream_chat_with_documents
output_language = get_llm_output_language( output_language = get_llm_output_language(
ai_config=ai_config, ai_config=ai_config,
user=request.user, user=request.user,
+19 -19
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr "" msgstr ""
"Project-Id-Version: paperless-ngx\n" "Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n" "Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-10-29 17:25+0000\n" "POT-Creation-Date: 2026-10-30 22:26+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n" "PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n" "Last-Translator: \n"
"Language-Team: English\n" "Language-Team: English\n"
@@ -21,43 +21,43 @@ msgstr ""
msgid "Documents" msgid "Documents"
msgstr "" msgstr ""
#: documents/filters.py:337 #: documents/filters.py:336
msgid "Expected a date or a relative date such as \"today\" or \"in 30 days\"." msgid "Expected a date or a relative date such as \"today\" or \"in 30 days\"."
msgstr "" msgstr ""
#: documents/filters.py:513 #: documents/filters.py:514
msgid "Value must be valid JSON." msgid "Value must be valid JSON."
msgstr "" msgstr ""
#: documents/filters.py:534 #: documents/filters.py:535
msgid "Invalid custom field query expression" msgid "Invalid custom field query expression"
msgstr "" msgstr ""
#: documents/filters.py:544 #: documents/filters.py:545
msgid "Invalid expression list. Must be nonempty." msgid "Invalid expression list. Must be nonempty."
msgstr "" msgstr ""
#: documents/filters.py:565 #: documents/filters.py:566
msgid "Invalid logical operator {op!r}" msgid "Invalid logical operator {op!r}"
msgstr "" msgstr ""
#: documents/filters.py:579 #: documents/filters.py:580
msgid "Maximum number of query conditions exceeded." msgid "Maximum number of query conditions exceeded."
msgstr "" msgstr ""
#: documents/filters.py:643 #: documents/filters.py:644
msgid "{name!r} is not a valid custom field." msgid "{name!r} is not a valid custom field."
msgstr "" msgstr ""
#: documents/filters.py:680 #: documents/filters.py:681
msgid "{data_type} does not support query expr {expr!r}." msgid "{data_type} does not support query expr {expr!r}."
msgstr "" msgstr ""
#: documents/filters.py:801 documents/models.py:137 #: documents/filters.py:802 documents/models.py:137
msgid "Maximum nesting depth exceeded." msgid "Maximum nesting depth exceeded."
msgstr "" msgstr ""
#: documents/filters.py:1178 #: documents/filters.py:1179
msgid "Custom field not found" msgid "Custom field not found"
msgstr "" msgstr ""
@@ -1657,7 +1657,7 @@ msgid "workflow runs"
msgstr "" msgstr ""
#: documents/serialisers.py:483 documents/serialisers.py:840 #: documents/serialisers.py:483 documents/serialisers.py:840
#: documents/serialisers.py:2872 documents/views.py:360 documents/views.py:2730 #: documents/serialisers.py:2872 documents/views.py:348 documents/views.py:2733
#: paperless_mail/serialisers.py:156 #: paperless_mail/serialisers.py:156
msgid "Insufficient permissions." msgid "Insufficient permissions."
msgstr "" msgstr ""
@@ -1698,7 +1698,7 @@ msgstr ""
msgid "Duplicate document identifiers are not allowed." msgid "Duplicate document identifiers are not allowed."
msgstr "" msgstr ""
#: documents/serialisers.py:2958 documents/views.py:4780 #: documents/serialisers.py:2958 documents/views.py:4783
#, python-format #, python-format
msgid "Documents not found: %(ids)s" msgid "Documents not found: %(ids)s"
msgstr "" msgstr ""
@@ -1949,11 +1949,11 @@ msgstr ""
msgid ", " msgid ", "
msgstr "" msgstr ""
#: documents/views.py:315 #: documents/views.py:303
msgid "Specify only one of text, title_search, query, or more_like_id." msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr "" msgstr ""
#: documents/views.py:353 documents/views.py:2727 #: documents/views.py:341 documents/views.py:2730
msgid "Invalid more_like_id" msgid "Invalid more_like_id"
msgstr "" msgstr ""
@@ -1969,20 +1969,20 @@ msgstr ""
msgid "AI backend rejected the request. Check logs for details." msgid "AI backend rejected the request. Check logs for details."
msgstr "" msgstr ""
#: documents/views.py:4796 #: documents/views.py:4799
#, python-format #, python-format
msgid "Insufficient permissions to share document %(id)s." msgid "Insufficient permissions to share document %(id)s."
msgstr "" msgstr ""
#: documents/views.py:4843 #: documents/views.py:4846
msgid "Bundle is already being processed." msgid "Bundle is already being processed."
msgstr "" msgstr ""
#: documents/views.py:4909 #: documents/views.py:4912
msgid "The share link bundle is still being prepared. Please try again later." msgid "The share link bundle is still being prepared. Please try again later."
msgstr "" msgstr ""
#: documents/views.py:4923 #: documents/views.py:4926
msgid "The share link bundle is unavailable." msgid "The share link bundle is unavailable."
msgstr "" msgstr ""
+1 -1
View File
@@ -17,7 +17,7 @@ from paperless.urls import websocket_urlpatterns # noqa: E402
application = ProtocolTypeRouter( application = ProtocolTypeRouter(
{ {
"http": get_asgi_application(), "http": django_asgi_app,
"websocket": AuthMiddlewareStack(URLRouter(websocket_urlpatterns)), "websocket": AuthMiddlewareStack(URLRouter(websocket_urlpatterns)),
}, },
) )
+4 -3
View File
@@ -36,9 +36,6 @@ from humanize import naturalsize
from imap_tools import MailAttachment from imap_tools import MailAttachment
from imap_tools import MailMessage from imap_tools import MailMessage
from tika_client import TikaClient from tika_client import TikaClient
from turbohtml.clean import Linkify
from turbohtml.clean import linkify
from turbohtml.migration.bleach import clean
from documents.parsers import ParseError from documents.parsers import ParseError
from documents.parsers import make_thumbnail_from_pdf from documents.parsers import make_thumbnail_from_pdf
@@ -622,6 +619,10 @@ class MailDocumentParser:
def clean_html(text: str) -> str: def clean_html(text: str) -> str:
"""Attempt to clean, escape, and linkify the given HTML string.""" """Attempt to clean, escape, and linkify the given HTML string."""
from turbohtml.clean import Linkify
from turbohtml.clean import linkify
from turbohtml.migration.bleach import clean
if isinstance(text, list): if isinstance(text, list):
text = "\n".join([str(e) for e in text]) text = "\n".join([str(e) for e in text])
if not isinstance(text, str): if not isinstance(text, str):
+8 -4
View File
@@ -1,5 +1,6 @@
from __future__ import annotations from __future__ import annotations
import functools
import importlib.resources import importlib.resources
import logging import logging
import os import os
@@ -39,9 +40,12 @@ if TYPE_CHECKING:
logger = logging.getLogger("paperless.parsing.tesseract") logger = logging.getLogger("paperless.parsing.tesseract")
_SRGB_ICC_DATA: Final[bytes] = (
importlib.resources.files("ocrmypdf.data").joinpath("sRGB.icc").read_bytes() @functools.cache
) def _srgb_icc_data() -> bytes:
# Read on first use, importing ocrmypdf.data pulls in the whole ocrmypdf package
return importlib.resources.files("ocrmypdf.data").joinpath("sRGB.icc").read_bytes()
_SUPPORTED_MIME_TYPES: Final[dict[str, str]] = { _SUPPORTED_MIME_TYPES: Final[dict[str, str]] = {
"application/pdf": ".pdf", "application/pdf": ".pdf",
@@ -403,7 +407,7 @@ class RasterisedDocumentParser:
pdfa_path = Path(self.tempdir) / "archive.pdf" pdfa_path = Path(self.tempdir) / "archive.pdf"
try: try:
with pikepdf.open(plain_pdf_path) as pdf: with pikepdf.open(plain_pdf_path) as pdf:
cs = pdf.make_stream(_SRGB_ICC_DATA) cs = pdf.make_stream(_srgb_icc_data())
cs["/N"] = 3 cs["/N"] = 3
output_intent = pikepdf.Dictionary( output_intent = pikepdf.Dictionary(
Type=pikepdf.Name("/OutputIntent"), Type=pikepdf.Name("/OutputIntent"),
+2 -1
View File
@@ -6,7 +6,6 @@ from pathlib import Path
from typing import Any from typing import Any
from celery.schedules import crontab from celery.schedules import crontab
from dateparser.languages.loader import LocaleDataLoader
from paperless.settings.parsers import get_choice_from_env from paperless.settings.parsers import get_choice_from_env
from paperless.settings.parsers import get_int_from_env from paperless.settings.parsers import get_int_from_env
@@ -352,6 +351,8 @@ def parse_dateparser_languages(languages: str | None) -> list[str]:
) )
language_list.append("zh") language_list.append("zh")
from dateparser.languages.loader import LocaleDataLoader
return list(LocaleDataLoader().get_locale_map(locales=language_list)) return list(LocaleDataLoader().get_locale_map(locales=language_list))
+4 -1
View File
@@ -59,7 +59,10 @@ def test_ocr_to_dateparser_languages_exception(
raise RuntimeError("Simulated error") raise RuntimeError("Simulated error")
with caplog.at_level(logging.WARNING): with caplog.at_level(logging.WARNING):
monkeypatch.setattr(utils, "LocaleDataLoader", lambda: DummyLoader()) monkeypatch.setattr(
"dateparser.languages.loader.LocaleDataLoader",
lambda: DummyLoader(),
)
result = utils.ocr_to_dateparser_languages("eng+fra") result = utils.ocr_to_dateparser_languages("eng+fra")
assert result == [] assert result == []
assert ( assert (
+2 -2
View File
@@ -1,7 +1,5 @@
import logging import logging
from dateparser.languages.loader import LocaleDataLoader
logger = logging.getLogger("paperless.utils") logger = logging.getLogger("paperless.utils")
OCR_TO_DATEPARSER_LANGUAGES = { OCR_TO_DATEPARSER_LANGUAGES = {
@@ -125,6 +123,8 @@ def ocr_to_dateparser_languages(ocr_languages: str) -> list[str]:
- If a language cannot be mapped or validated, it is skipped with a warning. - If a language cannot be mapped or validated, it is skipped with a warning.
- Returns a list of valid locales, or an empty list if none could be converted. - Returns a list of valid locales, or an empty list if none could be converted.
""" """
from dateparser.languages.loader import LocaleDataLoader
loader = LocaleDataLoader() loader = LocaleDataLoader()
result = [] result = []
try: try:
+1 -1
View File
@@ -3,7 +3,6 @@ from io import BytesIO
from django.conf import settings from django.conf import settings
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.core.files.uploadedfile import UploadedFile from django.core.files.uploadedfile import UploadedFile
from lxml import etree
from PIL import Image from PIL import Image
ALLOWED_SVG_TAGS: set[str] = { ALLOWED_SVG_TAGS: set[str] = {
@@ -186,6 +185,7 @@ def reject_dangerous_svg(file: UploadedFile) -> None:
Raises ValidationError if unsafe content is found. Raises ValidationError if unsafe content is found.
See GHSA-6p53-hqqw-8j62 See GHSA-6p53-hqqw-8j62
""" """
from lxml import etree
try: try:
parser = etree.XMLParser(resolve_entities=False) parser = etree.XMLParser(resolve_entities=False)
+2 -1
View File
@@ -49,7 +49,6 @@ from paperless.serialisers import GroupSerializer
from paperless.serialisers import PaperlessAuthTokenSerializer from paperless.serialisers import PaperlessAuthTokenSerializer
from paperless.serialisers import ProfileSerializer from paperless.serialisers import ProfileSerializer
from paperless.serialisers import UserSerializer from paperless.serialisers import UserSerializer
from paperless_ai.indexing import llm_index_exists
class PaperlessObtainAuthTokenView(ObtainAuthToken): class PaperlessObtainAuthTokenView(ObtainAuthToken):
@@ -471,6 +470,8 @@ class ApplicationConfigurationViewSet(ModelViewSet[ApplicationConfiguration]):
or old_llm_embedding_model != new_llm_embedding_model or old_llm_embedding_model != new_llm_embedding_model
or old_llm_context_size != new_llm_context_size or old_llm_context_size != new_llm_context_size
) )
from paperless_ai.indexing import llm_index_exists
rebuild_needed = new_ai_index_enabled and ( rebuild_needed = new_ai_index_enabled and (
not llm_index_exists() or embedding_config_changed not llm_index_exists() or embedding_config_changed
) )