Compare commits

...
Author SHA1 Message Date
stumpylog 5865e96a06 Handles a bad client sending malformed JSON or non-int primary keys 2026-08-25 14:26:33 -07:00
Trenton Holmes 172af8362d Perf: avoid per-instance CustomField reload in DocumentMetadataOverrides
send_websocket_document_updated calls document.refresh_from_db()
before building overrides, which drops the custom_fields prefetch
(and its select_related("field")) set up by the view's queryset.
DocumentMetadataOverrides.from_document() then lazily reloads field
once per custom field instance. Since from_document() can't rely on
the caller having a prefetched document, select_related explicitly at
the point of use instead.
2026-08-25 14:26:33 -07:00
Trenton Holmes e46163bfdf Perf: reuse resolved CustomField objects across drf-writable-nested's per-item revalidation
drf-writable-nested's update_or_create_reverse_relations rebuilds a
fresh serializer -- and fresh field instances -- per custom_fields item
while matching existing vs. new instances during save(), so the
per-instance lookup cache alone only helped the first validation pass.
It passes the same context dict (by reference) to every one of those
serializers, so stash resolved CustomField objects there instead:
later passes reuse them for free rather than re-querying.
2026-08-25 14:26:33 -07:00
Trenton Holmes f8e5aea8dd Perf: batch CustomField lookups when validating a document's custom_fields
DocumentSerializer.custom_fields validates each item's field id via a
plain PrimaryKeyRelatedField, which issues one SELECT per custom field
per validation pass (discussion #13690). Batch-resolve all field ids in
one query and cache them on the field instance so per-item validation
is free instead of re-querying.
2026-08-25 14:26:33 -07:00
shamoon e24db7023a Tweak: adjust brand leaf opacity 2026-08-25 10:44:49 -07:00
shamoon f5ddc14588 Fix: also check global change_mailaccount with test 2026-08-25 10:44:49 -07:00
GitHub Actions 492424f7f2 Auto translate strings 2026-08-25 00:07:03 +00:00
shamoonandGitHub cd525819d7 Tweak: more misc UI tweaks (#13783) 2026-08-24 17:05:29 -07:00
GitHub Actions 2609327e9c Auto translate strings 2026-08-24 21:44:25 +00:00
shamoonandGitHub b90ccf910f Finally, the remote ocr workflow (#13637)
* Ok! Backend stuff for the remote ocr workflow

* Frotnend workflow stuff

* And docs

* Fix dynamic action fields thing

* Actually, fix the action dropdown thing

* Fix this validation thing, and we have to check existing actions

* Fix migration
2026-08-24 14:43:05 -07:00
shamoonandGitHub c93c996edf Remote ocr reprocess (#13636)
* Backend stuff for remote ocr reprocess, add to bulk edit pass in from ui settings

* Ok, frontend reprocess remote option

* Docs
2026-08-24 14:43:05 -07:00
shamoonandGitHub 7f1609332a Allow parsers to declare uses remote, and remote ocr_mode (#13634)
* uses_remote_service + allow_remote to allow opt-in / out of remote OCR

* Add to parser dev docs

* remote_ocr_mode config setting

* Checks for remote_ocr_mode and fix import

* Update config.component.spec.ts

* More tests for remote_ocr_mode

* Docs for remote_ocr_mode

* Ok, wire up the remote_ocr_mode with allow_remote for consumer

* Update consumer.py

* Format remote OCR mode check tests

* Use get_choice_from_env
2026-08-24 14:43:04 -07:00
GitHub Actions a1f20c9fe7 Auto translate strings 2026-08-24 21:19:19 +00:00
shamoonandGitHub 4fd1c60731 Enhancement: support using remote OCR engines selectively (#13633)
* Backend changes and migration for remote OCR Config

* Backend tests

* Frontend stuff, with sections

* Docs

* Update test_tesseract_parser.py

* Actually we cant use this any more, in case settings are in app config

* Dont mark entire test file for db, use a mock for empty engine settings
2026-08-24 14:17:52 -07:00
GitHub Actions ba83e5b39a Auto translate strings 2026-08-24 18:28:51 +00:00
shamoonandGitHub 1497bd33a1 Fix: fix bottom mobile nav buttons on Android (#13780) 2026-08-24 11:27:26 -07:00
71 changed files with 2627 additions and 512 deletions
+2 -1
View File
@@ -302,7 +302,8 @@ The following methods are supported:
- `delete`
- No `parameters` required
- `reprocess`
- No `parameters` required
- Optional `parameters`: `{ "remote_ocr": true }` to send the documents to the
remote OCR engine, see [Remote OCR](usage.md#remote-ocr). Defaults to false.
- `set_permissions`
- Requires `parameters`:
- `"set_permissions": PERMISSIONS_OBJ` (see format [above](#permissions)) and / or
+12
View File
@@ -2066,6 +2066,18 @@ password. All of these options come from their similarly-named [Django settings]
Defaults to None.
#### [`PAPERLESS_REMOTE_OCR_MODE=<str>`](#PAPERLESS_REMOTE_OCR_MODE) {#PAPERLESS_REMOTE_OCR_MODE}
: Which documents are sent to the remote OCR engine.
- `always`: every document of a supported file type is sent to the remote
engine, bypassing the local OCR engine.
- `workflow_only`: documents are processed locally unless a workflow
explicitly enables remote OCR for them, letting you use the remote engine
selectively.
Defaults to "always".
## AI {#ai}
#### [`PAPERLESS_AI_ENABLED=<bool>`](#PAPERLESS_AI_ENABLED) {#PAPERLESS_AI_ENABLED}
+14
View File
@@ -456,6 +456,20 @@ def score(
return 10
```
**Remote services**
If your parser sends document content to a remote service, declare it:
```python
class MyCustomParser:
uses_remote_service = True
```
Paperless-ngx excludes such parsers when the document being consumed has not
been marked for remote processing, so users can keep remote OCR off by default
and enable it selectively with a workflow. Parsers that do not declare the
attribute are treated as fully local and are always considered.
**Archive and rendition flags**
```python
+22 -1
View File
@@ -654,6 +654,19 @@ happened while it was still encrypted, that original version will likewise be mi
**Current limitation**: Passwords are stored as a simple list without descriptions. To handle
multiple PDF types with different passwords, create separate workflows for each use case.
##### Remote OCR {#workflow-action-remote-ocr}
"Remote OCR" actions send the document to the configured remote OCR engine instead of processing it
locally. To use remote OCR selectively, set the [remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE)
to `workflow_only` then add this action to a workflow that matches only the documents you
want sent to the remote engine. See [Remote OCR](#remote-ocr) for the engine setup. The action only works with
a **Consumption Started** trigger.
The action takes no options, its presence is what enables remote OCR for a matching document.
If the remote engine is not configured, or does not support the document's file type, the document is
processed locally instead and a warning is written to the log.
#### Workflow placeholders
Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
@@ -1090,11 +1103,19 @@ Paperless-ngx supports performing OCR on documents using remote services. At the
[Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence).
This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with
Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details.
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details. These
settings can be supplied as environment variables or via **Application Configuration**.
Additionally, when using a commercial service with this feature, consider both potential costs as well as any associated file size
or page limitations (e.g. with a free tier).
By default, every document of a supported file type is sent to the remote engine. To use it more selectively, set the
[remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE) to `workflow_only`. Documents are then processed locally
unless a [remote OCR workflow action](#workflow-action-remote-ocr) enables it for them, so you can limit the remote
engine to particular documents.
Setting the mode to `workflow_only` also allows the **Reprocess** actions to selectively use remote OCR for individual documents.
## Architecture
Paperless-ngx consists of the following components:
+301 -220
View File
File diff suppressed because it is too large Load Diff
@@ -14,43 +14,48 @@
<a ngbNavLink>{{category}}</a>
<ng-template ngbNavContent>
<div class="p-3">
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
@for (option of getCategoryOptions(category); track option.key) {
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
@for (section of getCategorySections(category); track section) {
@if (section) {
<h5 class="mt-4 mb-3">{{section}}</h5>
}
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
@for (option of getCategoryOptions(category, section); track option.key) {
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div>
</div>
</div>
}
</div>
}
</div>
}
</div>
</ng-template>
</li>
@@ -8,7 +8,11 @@ import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectModule } from '@ng-select/ng-select'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { OutputTypeConfig } from 'src/app/data/paperless-config'
import {
ConfigCategory,
ConfigSection,
OutputTypeConfig,
} from 'src/app/data/paperless-config'
import { ConfigService } from 'src/app/services/config.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
@@ -158,4 +162,24 @@ describe('ConfigComponent', () => {
component.resetOption('barcodes_enabled')
expect(component.configForm.get('barcodes_enabled').value).toBeNull()
})
it('should group options into sections within a category, or not', () => {
const sections = component.getCategorySections(ConfigCategory.OCR)
expect(sections).toEqual([null, ConfigSection.RemoteOCR])
expect(
component
.getCategoryOptions(ConfigCategory.OCR)
.map((option) => option.key)
).toContain('output_type')
expect(
component
.getCategoryOptions(ConfigCategory.OCR, ConfigSection.RemoteOCR)
.map((option) => option.key)
).toEqual([
'remote_ocr_engine',
'remote_ocr_api_key',
'remote_ocr_endpoint',
'remote_ocr_mode',
])
})
})
@@ -74,8 +74,20 @@ export class ConfigComponent
return Object.values(ConfigCategory)
}
getCategoryOptions(category: string): ConfigOption[] {
return PaperlessConfigOptions.filter((o) => o.category === category)
getCategorySections(category: string): string[] {
return [
...new Set(
PaperlessConfigOptions.filter((o) => o.category === category).map(
(o) => o.section ?? null // null means no section
)
),
]
}
getCategoryOptions(category: string, section: string = null): ConfigOption[] {
return PaperlessConfigOptions.filter(
(o) => o.category === category && (o.section ?? null) === section
)
}
initialConfig: PaperlessConfig
@@ -2,7 +2,7 @@
<button class="navbar-toggler d-md-none collapsed border-0" type="button" data-toggle="collapse"
data-target="#sidebarMenu" aria-controls="sidebarMenu" aria-expanded="false" aria-label="Toggle navigation"
(click)="closeMobileSearch(); toggleMenuCollapsed()">
<span class="navbar-toggler-icon"></span>
<i-bs width="1.5em" height="1.5em" name="list"></i-bs>
</button>
<a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0 order-sm-0"
[ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
@@ -29,9 +29,9 @@
</div>
}
</a>
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 px-3 ps-md-3 me-sm-auto order-3 order-sm-1"
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 me-sm-auto order-3 order-sm-1"
[class.mobile-hidden]="mobileSearchHidden()">
<div class="col-12 header-search">
<div class="col-12 header-search mx-auto">
<pngx-global-search></pngx-global-search>
</div>
</div>
@@ -90,7 +90,7 @@
<ul class="nav flex-column">
<li class="nav-item app-link">
<a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="house"></i-bs><span><ng-container i18n>Dashboard</ng-container></span>
</a>
@@ -99,7 +99,7 @@
<a class="nav-link" routerLink="documents" routerLinkActive="active"
[routerLinkActiveOptions]="{ paths: 'exact', queryParams: 'ignored', matrixParams: 'ignored', fragment: 'ignored' }"
(click)="closeMenu()"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="files"></i-bs><span><ng-container i18n>Documents</ng-container></span>
</a>
@@ -118,7 +118,7 @@
(cdkDragEnded)="onDragEnd($event)">
<a class="nav-link" routerLink="view/{{view.id}}"
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
@if (showSidebarCounts && !slimSidebarEnabled) {
@@ -156,7 +156,7 @@
<li class="nav-item w-100 app-link">
<a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" routerLink="documents/{{d.id}}"
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="d.title | documentTitle"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" name="file-text"></i-bs><span>{{d.title | documentTitle}}</span>
<span class="close flex-column justify-content-center"
@@ -170,7 +170,7 @@
<li class="nav-item w-100 app-link">
<button type="button" class="nav-link nav-link-action app-link w-100 text-start"
[class.text-truncate]="!slimSidebarEnabled" (click)="closeAll()"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="x"></i-bs><span><ng-container i18n>Close all</ng-container></span>
</button>
@@ -189,7 +189,7 @@
<div class="d-flex align-items-center attributes-row">
<a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active"
[routerLinkActiveOptions]="{ exact: !(slimSidebarEnabled || attributesSectionsCollapsed) }" (click)="closeMenu()"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs name="stack"></i-bs><span class="ms-2"><ng-container i18n>Attributes</ng-container></span>
</a>
@@ -241,7 +241,7 @@
}
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
<a class="nav-link" routerLink="savedviews" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="window-stack"></i-bs><span><ng-container i18n>Saved Views</ng-container></span>
</a>
@@ -250,7 +250,7 @@
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Workflow }"
tourAnchor="tour.workflows">
<a class="nav-link" routerLink="workflows" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="boxes"></i-bs><span><ng-container i18n>Workflows</ng-container></span>
</a>
@@ -258,14 +258,14 @@
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
tourAnchor="tour.mail">
<a class="nav-link" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="envelope"></i-bs><span><ng-container i18n>Mail</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
<a class="nav-link" routerLink="trash" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Trash"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="trash"></i-bs><span><ng-container i18n>Trash</ng-container></span>
</a>
@@ -281,21 +281,21 @@
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }"
tourAnchor="tour.settings">
<a class="nav-link" routerLink="settings" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="gear"></i-bs><span><ng-container i18n>Settings</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.AppConfig }">
<a class="nav-link" routerLink="config" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="sliders2-vertical"></i-bs><span><ng-container i18n>Configuration</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.User }">
<a class="nav-link" routerLink="usersgroups" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="people"></i-bs><span><ng-container i18n>Users & Groups</ng-container></span>
</a>
@@ -304,7 +304,7 @@
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.PaperlessTask }"
tourAnchor="tour.file-tasks">
<a class="nav-link" routerLink="tasks" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="list-task"></i-bs><span><ng-container i18n>Tasks</ng-container>@if (tasksService.needsAttentionTasks.length > 0) {
<span><span class="badge bg-danger ms-2 d-inline">{{tasksService.needsAttentionTasks.length}}</span></span>
@@ -317,26 +317,26 @@
@if (permissionsService.isAdmin()) {
<li class="nav-item app-link">
<a class="nav-link" routerLink="logs" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Logs"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="text-left"></i-bs><span><ng-container i18n>Logs</ng-container></span>
</a>
</li>
}
<li class="nav-item mt-2" tourAnchor="tour.outro">
<a class="px-3 py-2 text-muted small d-flex align-items-center flex-wrap text-decoration-none"
<a class="text-muted small d-flex align-items-center flex-wrap text-decoration-none nav-anchor"
target="_blank" rel="noopener noreferrer" href="https://docs.paperless-ngx.com" ngbPopover="Documentation"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="d-flex me-2" name="question-circle"></i-bs><span><ng-container i18n>Documentation</ng-container></span>
</a>
</li>
<li class="nav-item" [class.visually-hidden]="slimSidebarEnabled">
<div class="px-3 py-0 text-muted small d-flex align-items-center flex-wrap">
<div class="text-muted small d-flex align-items-center flex-wrap nav-label">
<div class="me-3">
<a class="text-muted text-decoration-none" target="_blank" rel="noopener noreferrer"
href="https://github.com/paperless-ngx/paperless-ngx" ngbPopover="GitHub" i18n-ngbPopover
[disablePopover]="!slimSidebarEnabled" placement="end" container="body"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
{{ versionString }}
</a>
@@ -67,6 +67,23 @@
main.mobile-search-hidden {
padding-top: 56px;
}
.navbar-toggler {
padding-left: calc(12px - (1.5em * 2.5 / 16));
}
}
.search-container {
padding-left: 1rem;
padding-right: 1rem;
}
// Below sm the search gets its own full-width row, so line it up with main's content edge
@media (max-width: 575.98px) {
.search-container {
padding-left: 12px;
padding-right: 12px;
}
}
main {
@@ -97,7 +114,7 @@ main {
li.nav-item span.badge {
display: inline-block;
margin-right: 2px;
margin-right: -4px;
}
}
@@ -141,18 +158,11 @@ main {
}
}
.sidebar.slim {
li.nav-item span.badge {
display: inline-block;
margin-right: 2px;
}
}
.sidebar-slim-toggler {
display: block;
position: fixed;
left: calc(var(--pngx-sidebar-width) - 12px);
top: 72px;
bottom: 16px;
z-index: 996;
--bs-btn-padding-x: 0.35rem;
--bs-btn-padding-y: 0.125rem;
@@ -227,6 +237,10 @@ main {
}
}
.sidebar .nav-anchor, .sidebar .nav-label {
padding: .25rem .7rem;
}
.attributes-row {
border-radius: .55rem;
margin: .1rem 0;
@@ -403,7 +417,7 @@ main {
:host ::ng-deep .navbar-official-logo {
.leaf {
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 70%, var(--bs-primary)) !important;
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 85%, var(--bs-primary)) !important;
}
.text {
@@ -444,7 +458,7 @@ main {
.header-search {
width: 100%;
max-width: 44rem;
max-width: 55rem;
}
@media screen and (max-width: 575.98px) {
@@ -458,6 +472,7 @@ main {
.navbar-toggler {
grid-area: toggler;
color: var(--pngx-primary-text-contrast);
text-align: left;
}
@@ -278,6 +278,10 @@ export class AppFrameComponent
})
}
get slimSidebarPopoversEnabled(): boolean {
return this.slimSidebarEnabled && !this.isMobileViewport()
}
get attributesSectionsCollapsed(): boolean {
this.settingsService.trackChanges()
return this.settingsService
@@ -0,0 +1,28 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
</div>
<div class="modal-body">
@if (messageBold) {
<p class="text-break"><b>{{messageBold}}</b></p>
}
@if (message) {
<p class="mb-0 text-break" [innerHTML]="message"></p>
}
@if (showRemoteOcr) {
<div class="form-check mt-3">
<input class="form-check-input" type="checkbox" id="reprocessRemoteOcr" [(ngModel)]="remoteOcr" />
<label class="form-check-label" for="reprocessRemoteOcr" i18n>Use remote OCR</label>
<div class="form-text" i18n>Sends the document to the configured remote OCR service, which may incur costs.</div>
</div>
}
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,72 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { RemoteOCRModeConfig } from 'src/app/data/paperless-config'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { SettingsService } from 'src/app/services/settings.service'
import { ReprocessConfirmDialogComponent } from './reprocess-confirm-dialog.component'
describe('ReprocessConfirmDialogComponent', () => {
let component: ReprocessConfirmDialogComponent
let fixture: ComponentFixture<ReprocessConfirmDialogComponent>
let settingsService: SettingsService
const createComponent = (configured: boolean, mode: string) => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, configured)
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_MODE, mode)
fixture = TestBed.createComponent(ReprocessConfirmDialogComponent)
component = fixture.componentInstance
fixture.detectChanges()
}
beforeEach(async () => {
TestBed.configureTestingModule({
providers: [
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
imports: [ReprocessConfirmDialogComponent],
}).compileComponents()
settingsService = TestBed.inject(SettingsService)
})
it('should not request remote OCR by default', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.remoteOcr).toBeFalsy()
})
it('should not offer remote OCR when no engine is configured', () => {
createComponent(false, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should not offer remote OCR when it already handles every document', () => {
createComponent(true, RemoteOCRModeConfig.ALWAYS)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should offer remote OCR when configured and selective', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeTruthy()
const checkbox = fixture.nativeElement.querySelector('#reprocessRemoteOcr')
expect(checkbox).not.toBeNull()
checkbox.click()
fixture.detectChanges()
expect(component.remoteOcr).toBeTruthy()
})
})
@@ -0,0 +1,20 @@
import { Component, inject } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { SettingsService } from 'src/app/services/settings.service'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-reprocess-confirm-dialog',
templateUrl: './reprocess-confirm-dialog.component.html',
imports: [FormsModule],
})
export class ReprocessConfirmDialogComponent extends ConfirmDialogComponent {
private settings = inject(SettingsService)
remoteOcr: boolean = false
public get showRemoteOcr(): boolean {
// Hidden when it is not configured, or when it already handles every document anyway.
return this.settings.remoteOCRIsSelectable
}
}
@@ -455,6 +455,13 @@
</div>
</div>
}
@case (WorkflowActionType.RemoteOcr) {
<div class="row">
<div class="col">
<p class="text-muted small" i18n>The document will be sent to the configured remote OCR service. May incur costs.</p>
</div>
</div>
}
}
</div>
</ng-template>
@@ -29,6 +29,7 @@ import {
DocumentSource,
WorkflowTriggerType,
} from 'src/app/data/workflow-trigger'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
@@ -224,7 +225,12 @@ describe('WorkflowEditDialogComponent', () => {
).toEqual('Document Added')
expect(component.getTriggerTypeOptionName(null)).toEqual('')
expect(component.sourceOptions).toEqual(DOCUMENT_SOURCE_OPTIONS)
expect(component.actionTypeOptions).toEqual(WORKFLOW_ACTION_OPTIONS)
// Remote OCR is absent until the workflow has a consumption trigger
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.RemoteOcr
)
)
expect(
component.getActionTypeOptionName(WorkflowActionType.Assignment)
).toEqual('Assignment')
@@ -237,7 +243,104 @@ describe('WorkflowEditDialogComponent', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(false)
component.ngOnInit()
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter((a) => a.id !== WorkflowActionType.Email)
WORKFLOW_ACTION_OPTIONS.filter(
(a) =>
a.id !== WorkflowActionType.Email &&
a.id !== WorkflowActionType.RemoteOcr
)
)
})
it('should offer remote OCR only for consumption workflows', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// A consumption trigger makes the action reachable
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Any other trigger type runs after the document has been parsed
component.object = {
name: 'Workflow 2',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should offer remote OCR on a trigger added to a new workflow', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
component.ngOnInit()
// Nothing for the action to apply to yet
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
// addTrigger creates the form field with emitEvent false, so the options
// have to be computed on read rather than cached from valueChanges
component.addTrigger()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Switching that trigger to a type that runs after parsing removes it
component.triggerFields
.at(0)
.get('type')
.setValue(WorkflowTriggerType.DocumentAdded)
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should keep remote OCR listed when an action already uses it', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// Otherwise changing the trigger would silently blank the selection
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [{ type: WorkflowActionType.RemoteOcr }],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
})
it('should not offer remote OCR when no engine is configured', () => {
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) => key !== SETTINGS_KEYS.REMOTE_OCR_CONFIGURED)
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
@@ -148,6 +148,10 @@ export const WORKFLOW_ACTION_OPTIONS = [
id: WorkflowActionType.MoveToTrash,
name: $localize`Move to trash`,
},
{
id: WorkflowActionType.RemoteOcr,
name: $localize`Remote OCR`,
},
]
export enum TriggerFilterType {
@@ -504,8 +508,6 @@ export class WorkflowEditDialogComponent
expandedItem: number = null
readonly allowedActionTypes = signal([])
private readonly triggerFilterOptionsMap = new WeakMap<
FormArray,
TriggerFilterOption[]
@@ -548,13 +550,40 @@ export class WorkflowEditDialogComponent
this.checkRemovalActionFields.bind(this)
)
this.checkRemovalActionFields(this.objectForm.value)
this.allowedActionTypes.set(
this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)
? WORKFLOW_ACTION_OPTIONS
: WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.Email
)
)
}
private allowedActionTypes: typeof WORKFLOW_ACTION_OPTIONS = null
private getAllowedActionTypes() {
let allowed = WORKFLOW_ACTION_OPTIONS
if (!this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.Email)
}
// Remote OCR is decided before the document is parsed, so it is only
// offered for workflows that run at consumption.
const formWorkflow: Workflow = this.objectForm?.value
const remoteOcrUsable =
this.settingsService.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
(formWorkflow?.triggers?.some(
(trigger) => trigger.type === WorkflowTriggerType.Consumption
) ||
formWorkflow?.actions?.some(
(action) => action.type === WorkflowActionType.RemoteOcr
))
if (!remoteOcrUsable) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
}
if (
this.allowedActionTypes?.length === allowed.length &&
this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
) {
return this.allowedActionTypes
}
this.allowedActionTypes = allowed
return allowed
}
private checkRemovalActionFields(formWorkflow: Workflow) {
@@ -1279,7 +1308,8 @@ export class WorkflowEditDialogComponent
get actionTypeOptions() {
this.settingsService.trackChanges()
return this.allowedActionTypes()
// Computed on read rather than cached
return this.getAllowedActionTypes()
}
getActionTypeOptionName(type: WorkflowActionType): string {
@@ -1,6 +1,6 @@
<div class="row pt-3 pb-3 pb-md-2 align-items-center">
<div class="row pt-3 pb-2 align-items-center">
<div class="col-md text-truncate">
<h3 class="d-flex align-items-center mb-1" style="line-height: 1.4">
<h3 class="d-flex align-items-center mb-2 mb-md-1" style="line-height: 1.4">
<span class="text-truncate">{{title()}}</span>
@if (id()) {
<span class="badge bg-primary text-primary-text-contrast ms-3 small fs-normal cursor-pointer" (click)="copyID()">
@@ -12,10 +12,10 @@
</span>
}
@if (subTitle()) {
<span class="h6 mb-0 mt-1 d-block d-md-inline fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
<span class="page-subtitle h6 mb-0 mt-1 fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
}
@if (info()) {
<button class="btn btn-sm btn-link text-muted p-0 p-md-2" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<button class="btn btn-sm btn-link text-muted p-0 ms-2 p-md-2 ms-md-0" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<i-bs name="question-circle"></i-bs>
</button>
<ng-template #infoPopover>
@@ -1,5 +1,6 @@
h3 {
min-height: calc(1.325rem + 0.9vw);
flex-wrap: wrap;
.badge {
font-size: 0.65rem;
@@ -7,6 +8,16 @@ h3 {
}
}
// Drop the subtitle onto its own and shrink it
@media (max-width: 767.98px) {
h3 > .page-subtitle {
flex: 0 0 100%;
margin-top: 0 !important;
font-size: .7rem;
line-height: 1.3 !important;
}
}
:host {
display: block;
margin-bottom: .35rem;
@@ -963,12 +963,24 @@ describe('DocumentDetailComponent', () => {
component.reprocess()
const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] })
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, false)
expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
})
it('should pass remote OCR choice when reprocessing', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
reprocessSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component.reprocess()
openModal.componentInstance.remoteOcr = true
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, true)
})
it('should show error if redo ocr call fails', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
@@ -97,6 +97,7 @@ import { ISODateAdapter } from 'src/app/utils/ngb-iso-date-adapter'
import * as UTIF from 'utif'
import { DocumentDetailFieldID } from '../admin/settings/settings.component'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
@@ -1398,7 +1399,7 @@ export class DocumentDetailComponent
}
reprocess() {
let modal = this.modalService.open(ConfirmDialogComponent, {
let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Reprocess confirm`
@@ -1409,7 +1410,10 @@ export class DocumentDetailComponent
modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled.set(false)
this.documentsService
.reprocessDocuments({ documents: [this.document().id] })
.reprocessDocuments(
{ documents: [this.document().id] },
modal.componentInstance.remoteOcr
)
.subscribe({
next: () => {
this.toastService.showInfo(
@@ -1122,6 +1122,7 @@ describe('BulkEditorComponent', () => {
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
remote_ocr: false,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -52,6 +52,7 @@ import { flattenTags } from 'src/app/utils/flatten-tags'
import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dialog/merge-as-versions-confirm-dialog/merge-as-versions-confirm-dialog.component'
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
@@ -916,7 +917,7 @@ export class BulkEditorComponent
}
reprocessSelected() {
let modal = this.modalService.open(ConfirmDialogComponent, {
let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Reprocess confirm`
@@ -930,7 +931,10 @@ export class BulkEditorComponent
modal.componentInstance.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
this.documentService.reprocessDocuments(this.getSelectionQuery())
this.documentService.reprocessDocuments(
this.getSelectionQuery(),
modal.componentInstance.remoteOcr
)
)
})
}
+55
View File
@@ -54,6 +54,10 @@ export const ConfigCategory = {
AI: $localize`AI Settings`,
}
export const ConfigSection = {
RemoteOCR: $localize`Remote OCR`,
}
export const LLMEmbeddingBackendConfig = {
OPENAI_LIKE: 'openai-like',
HUGGINGFACE: 'huggingface',
@@ -65,6 +69,15 @@ export const LLMBackendConfig = {
OLLAMA: 'ollama',
}
export const RemoteOCREngineConfig = {
AZURE_AI: 'azureai',
}
export const RemoteOCRModeConfig = {
ALWAYS: 'always',
WORKFLOW_ONLY: 'workflow_only',
}
export interface ConfigOption {
key: string
title: string
@@ -72,6 +85,7 @@ export interface ConfigOption {
choices?: Array<{ id: string; name: string }>
config_key?: string
category: string
section?: string
note?: string
}
@@ -181,6 +195,43 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_OCR_USER_ARGS',
category: ConfigCategory.OCR,
},
{
key: 'remote_ocr_engine',
title: $localize`Remote OCR Engine`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCREngineConfig),
config_key: 'PAPERLESS_REMOTE_OCR_ENGINE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Enabling remote OCR sends documents to a third-party service for processing. Consider the privacy implications as well as potential costs before enabling.`,
},
{
key: 'remote_ocr_api_key',
title: $localize`Remote OCR API Key`,
type: ConfigOptionType.Password,
config_key: 'PAPERLESS_REMOTE_OCR_API_KEY',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
},
{
key: 'remote_ocr_endpoint',
title: $localize`Remote OCR Endpoint`,
type: ConfigOptionType.String,
config_key: 'PAPERLESS_REMOTE_OCR_ENDPOINT',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Required when using the Azure AI engine.`,
},
{
key: 'remote_ocr_mode',
title: $localize`Remote OCR Mode`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCRModeConfig),
config_key: 'PAPERLESS_REMOTE_OCR_MODE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Which documents are sent to the remote engine. Use 'workflow_only' to keep remote OCR off unless a workflow enables it for a document.`,
},
{
key: 'app_logo',
title: $localize`Application Logo`,
@@ -398,6 +449,10 @@ export interface PaperlessConfig extends ObjectWithId {
barcode_enable_tag: boolean
barcode_tag_mapping: object
barcode_tag_split: boolean
remote_ocr_engine: string
remote_ocr_api_key: string
remote_ocr_endpoint: string
remote_ocr_mode: string
ai_enabled: boolean
llm_embedding_backend: string
llm_embedding_model: string
+13
View File
@@ -1,5 +1,6 @@
import { PdfEditorEditMode } from '../components/common/pdf-editor/pdf-editor-edit-mode'
import { PdfZoomScale } from '../components/common/pdf-viewer/pdf-viewer.types'
import { RemoteOCRModeConfig } from './paperless-config'
import { User } from './user'
export interface UiSettings {
@@ -94,6 +95,8 @@ export const SETTINGS_KEYS = {
OUTLOOK_OAUTH_URL: 'outlook_oauth_url',
EMAIL_ENABLED: 'email_enabled',
AI_ENABLED: 'ai_enabled',
REMOTE_OCR_CONFIGURED: 'remote_ocr:configured',
REMOTE_OCR_MODE: 'remote_ocr:mode',
}
export const SETTINGS: UiSetting[] = [
@@ -347,4 +350,14 @@ export const SETTINGS: UiSetting[] = [
type: 'string',
default: PdfEditorEditMode.Create,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_CONFIGURED,
type: 'boolean',
default: false,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_MODE,
type: 'string',
default: RemoteOCRModeConfig.ALWAYS,
},
]
+1
View File
@@ -7,6 +7,7 @@ export enum WorkflowActionType {
Webhook = 4,
PasswordRemoval = 5,
MoveToTrash = 6,
RemoteOcr = 7,
}
export interface WorkflowActionEmail extends ObjectWithId {
@@ -284,6 +284,21 @@ describe(`DocumentService`, () => {
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: false,
})
})
it('should request remote OCR when reprocessing with it enabled', () => {
const ids = [1, 2, 3]
subscription = service
.reprocessDocuments({ documents: ids }, true)
.subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/reprocess/`
)
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: true,
})
})
@@ -349,9 +349,13 @@ export class DocumentService extends AbstractPaperlessService<Document> {
})
}
reprocessDocuments(selection: DocumentSelectionQuery) {
reprocessDocuments(
selection: DocumentSelectionQuery,
remoteOcr: boolean = false
) {
return this.http.post(this.getResourceUrl(null, 'reprocess'), {
...selection,
remote_ocr: remoteOcr,
})
}
@@ -13,6 +13,7 @@ import { environment } from 'src/environments/environment'
import { CustomFieldDataType } from '../data/custom-field'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { SavedView } from '../data/saved-view'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings'
import { PermissionsService } from './permissions.service'
import { CustomFieldsService } from './rest/custom-fields.service'
@@ -434,4 +435,26 @@ describe('SettingsService', () => {
).name
).toEqual(customFields[0].name)
})
it('should offer remote OCR only when configured and selective', () => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, false)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
// configured, but already handling every document
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, true)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.ALWAYS
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeTruthy()
})
})
@@ -19,6 +19,7 @@ import {
} from 'src/app/utils/color'
import { DEFAULT_APP_TITLE, environment } from 'src/environments/environment'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SavedView } from '../data/saved-view'
import {
PAPERLESS_GREEN_HEX,
@@ -687,6 +688,17 @@ export class SettingsService {
return this.settingIsSet(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)
}
/**
* Offering remote OCR as a choice only makes sense when an engine
* is configured but is not already handling every document.
*/
get remoteOCRIsSelectable(): boolean {
return (
this.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
this.get(SETTINGS_KEYS.REMOTE_OCR_MODE) !== RemoteOCRModeConfig.ALWAYS
)
}
offerTour(): boolean {
return this.dashboardIsEmpty() && !this.get(SETTINGS_KEYS.TOUR_COMPLETE)
}
+2
View File
@@ -118,6 +118,7 @@ import {
journalBookmarkFill,
journals,
link,
list,
listNested,
listTask,
listUl,
@@ -365,6 +366,7 @@ const icons = {
journalBookmarkFill,
journals,
link,
list,
listNested,
listTask,
listUl,
+1
View File
@@ -536,6 +536,7 @@ ul.pagination {
}
.nav-underline {
gap: calc(0.5 * var(--bs-nav-underline-gap)) var(--bs-nav-underline-gap);
.nav-link {
&.active, &:hover, &:focus {
color: var(--bs-primary);
+8 -2
View File
@@ -399,10 +399,16 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
return "OK"
def reprocess(doc_ids: list[int]) -> Literal["OK"]:
def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
"""
Re-run parsing for the given documents.
Consumption workflows do not run here, so ``remote_ocr`` is how the user
asks for the remote engine when it is not configured to handle everything.
"""
for document_id in doc_ids:
update_document_content_maybe_archive_file.apply_async(
kwargs={"document_id": document_id},
kwargs={"document_id": document_id, "remote_ocr": remote_ocr},
headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
)
+18
View File
@@ -53,6 +53,7 @@ from documents.utils import copy_basic_file_stats
from documents.utils import copy_file_with_basic_stats
from documents.utils import run_subprocess
from paperless.config import OcrConfig
from paperless.config import RemoteOCRConfig
from paperless.models import ArchiveFileGenerationChoices
from paperless.parsers import ParserContext
from paperless.parsers import ParserProtocol
@@ -451,12 +452,19 @@ class ConsumerPlugin(
except Exception as e:
self.log.error(f"Error attempting to clean PDF: {e}")
# Workflows have already run at this point, so the metadata knows
# whether this document was singled out for remote OCR
allow_remote = (
self.metadata.remote_ocr or RemoteOCRConfig().remote_ocr_by_default
)
# Based on the mime type, get the parser for that type
parser_class: type[ParserProtocol] | None = (
get_parser_registry().get_parser_for_file(
mime_type,
self.filename,
self.working_copy,
allow_remote=allow_remote,
)
)
if not parser_class:
@@ -465,6 +473,16 @@ class ConsumerPlugin(
f"Unsupported mime type {mime_type}",
)
if self.metadata.remote_ocr and not getattr(
parser_class,
"uses_remote_service",
False,
):
self.log.warning(
"Remote OCR was requested for this document but no remote "
"parser is available for it, processing locally instead.",
)
# Notify all listeners that we're going to do some work.
document_consumption_started.send(
+4 -1
View File
@@ -34,6 +34,7 @@ class DocumentMetadataOverrides:
skip_asn_if_exists: bool = False
version_label: str | None = None
actor_id: int | None = None
remote_ocr: bool = False
def update(self, other: "DocumentMetadataOverrides") -> "DocumentMetadataOverrides":
"""
@@ -57,6 +58,8 @@ class DocumentMetadataOverrides:
self.actor_id = other.actor_id
if other.skip_asn_if_exists:
self.skip_asn_if_exists = True
if other.remote_ocr:
self.remote_ocr = True
if other.version_label is not None:
self.version_label = other.version_label
@@ -126,7 +129,7 @@ class DocumentMetadataOverrides:
)
overrides.custom_fields = {
custom_field.field.id: custom_field.value
for custom_field in doc.custom_fields.all()
for custom_field in doc.custom_fields.select_related("field").all()
}
groups_with_perms = get_groups_with_perms(
@@ -0,0 +1,30 @@
# Generated by Django 5.2.16 on 2026-08-10 17:27
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0023_savedview_icon"),
]
operations = [
migrations.AlterField(
model_name="workflowaction",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Assignment"),
(2, "Removal"),
(3, "Email"),
(4, "Webhook"),
(5, "Password removal"),
(6, "Move to trash"),
(7, "Remote OCR"),
],
default=1,
verbose_name="Workflow Action Type",
),
),
]
+4
View File
@@ -1670,6 +1670,10 @@ class WorkflowAction(models.Model):
6,
_("Move to trash"),
)
REMOTE_OCR = (
7,
_("Remote OCR"),
)
type = models.PositiveSmallIntegerField(
_("Workflow Action Type"),
+146 -2
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import logging
import math
import re
from collections.abc import Iterable
from datetime import datetime
from datetime import timedelta
from decimal import Decimal
@@ -876,8 +877,106 @@ def validate_documentlink_targets(user, doc_ids):
)
# drf-writable-nested revalidates a document's custom_fields more than once
# per request: once as the ordinary nested list, then again per-item while
# matching existing vs. new CustomFieldInstance rows during save() -- and
# that second pass builds a brand new serializer (and field) instance per
# item (see its update_or_create_reverse_relations / _get_serializer_for_field),
# so a cache on the field instance alone only helps the first pass. It does,
# however, explicitly pass `context=self.context` to every one of those
# fresh serializers -- the *same* dict object the outer DocumentSerializer
# is using, not a copy. That context dict is already request-scoped (DRF
# builds it fresh per request via get_serializer_context()), so stashing the
# resolved CustomField objects there -- rather than in some new global/
# thread-local cache -- lets every later pass reuse them for free while
# staying entirely within DRF's existing, already-request-scoped machinery.
_CUSTOM_FIELD_CONTEXT_CACHE_KEY = "_custom_field_lookup_cache"
class _CachingCustomFieldPrimaryKeyField(serializers.PrimaryKeyRelatedField):
"""
Resolves CustomField ids with as few queries as possible: a per-instance
cache for repeat lookups on this exact field instance, backed by a
shared cache on the serializer context (see _CUSTOM_FIELD_CONTEXT_CACHE_KEY
above) so later, separately-instantiated fields for the same request
reuse what was already resolved instead of re-querying.
"""
def __init__(self, **kwargs: Any) -> None:
super().__init__(**kwargs)
self._cache: dict[int, CustomField] = {}
def _shared_cache(self) -> dict[int, CustomField]:
return self.context.setdefault(_CUSTOM_FIELD_CONTEXT_CACHE_KEY, {})
@staticmethod
def _normalize_pk(data: Any) -> int | None:
"""
Returns `data` coerced to the int a valid CustomField pk would be,
or None if `data` isn't a plausible pk (wrong type, unhashable,
non-numeric, or a bool -- DRF itself rejects bools as pks since
`True == 1` would otherwise silently match). None tells callers to
leave `data` alone and let `super().to_internal_value()` report the
normal validation error instead of touching the cache/queryset with
it directly.
"""
if isinstance(data, bool):
return None
try:
return int(data)
except (TypeError, ValueError):
return None
def prefetch(self, ids: Iterable[Any]) -> None:
shared_cache = self._shared_cache()
candidates = {pk for i in ids if (pk := self._normalize_pk(i)) is not None}
missing = {
i for i in candidates if i not in self._cache and i not in shared_cache
}
if missing:
for obj in self.get_queryset().filter(pk__in=missing):
shared_cache[obj.pk] = obj
for i in candidates:
obj = shared_cache.get(i)
if obj is not None:
self._cache[i] = obj
def to_internal_value(self, data: Any) -> CustomField:
pk = self._normalize_pk(data)
if pk is None:
return super().to_internal_value(data)
if pk in self._cache:
return self._cache[pk]
shared_cache = self._shared_cache()
if pk in shared_cache:
obj = shared_cache[pk]
self._cache[pk] = obj
return obj
obj: CustomField = super().to_internal_value(data)
self._cache[obj.pk] = obj
shared_cache[obj.pk] = obj
return obj
class CustomFieldInstanceListSerializer(serializers.ListSerializer):
def to_internal_value(self, data: Any) -> list[Any]:
if isinstance(data, list):
field_ids = []
for item in data:
if not isinstance(item, dict) or "field" not in item:
continue
try:
hash(item["field"])
except TypeError:
continue
field_ids.append(item["field"])
if field_ids:
self.child.fields["field"].prefetch(field_ids)
return super().to_internal_value(data)
class CustomFieldInstanceSerializer(serializers.ModelSerializer[CustomFieldInstance]):
field = serializers.PrimaryKeyRelatedField(queryset=CustomField.objects.all())
field = _CachingCustomFieldPrimaryKeyField(queryset=CustomField.objects.all())
value = ReadWriteSerializerMethodField(allow_null=True)
def create(self, validated_data):
@@ -978,6 +1077,7 @@ class CustomFieldInstanceSerializer(serializers.ModelSerializer[CustomFieldInsta
class Meta:
model = CustomFieldInstance
list_serializer_class = CustomFieldInstanceListSerializer
fields = [
"value",
"field",
@@ -1795,7 +1895,7 @@ class DeleteDocumentsSerializer(DocumentSelectionSerializer):
class ReprocessDocumentsSerializer(DocumentSelectionSerializer):
pass
remote_ocr = serializers.BooleanField(required=False, default=False)
class BulkEditSerializer(
@@ -2137,6 +2237,13 @@ class BulkEditSerializer(
f"Page {op['page']} is out of bounds for document with {doc.page_count} pages.",
)
def _validate_parameters_reprocess(self, parameters) -> None:
if "remote_ocr" in parameters:
if not isinstance(parameters["remote_ocr"], bool):
raise serializers.ValidationError("remote_ocr must be a boolean")
else:
parameters["remote_ocr"] = False
def validate_parameters_remove_password(self, parameters):
if "password" not in parameters:
raise serializers.ValidationError("password not specified")
@@ -2201,6 +2308,8 @@ class BulkEditSerializer(
self._validate_parameters_edit_pdf(parameters, attrs["documents"][0])
elif method == bulk_edit.remove_password:
self.validate_parameters_remove_password(parameters)
elif method == bulk_edit.reprocess:
self._validate_parameters_reprocess(parameters)
return attrs
@@ -3303,6 +3412,41 @@ class WorkflowSerializer(serializers.ModelSerializer[Workflow]):
"actions",
]
def validate(self, attrs):
attrs = super().validate(attrs)
if "actions" in attrs:
has_remote_ocr_action = any(
action.get("type") == WorkflowAction.WorkflowActionType.REMOTE_OCR
for action in attrs["actions"]
)
else:
has_remote_ocr_action = self.instance is not None and (
self.instance.actions.filter(
type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
).exists()
)
if "triggers" in attrs:
has_consumption_trigger = any(
trigger.get("type") == WorkflowTrigger.WorkflowTriggerType.CONSUMPTION
for trigger in attrs["triggers"]
)
else:
has_consumption_trigger = self.instance is not None and (
self.instance.triggers.filter(
type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
).exists()
)
# Remote OCR can only work with consumption triggers
if has_remote_ocr_action and not has_consumption_trigger:
raise serializers.ValidationError(
"Remote OCR actions require a consumption started trigger",
)
return attrs
def update_triggers_and_actions(
self,
instance: Workflow,
+11
View File
@@ -973,6 +973,17 @@ def run_workflows(
)
elif action.type == WorkflowAction.WorkflowActionType.MOVE_TO_TRASH:
has_move_to_trash_action = True
elif action.type == WorkflowAction.WorkflowActionType.REMOTE_OCR:
if use_overrides and overrides:
overrides.remote_ocr = True
else:
# If a workflow has a consumption trigger *and* another type,
# the document has already been parsed by the time the other one fires
logger.debug(
"Remote OCR action only applies to consumption "
"triggers, ignoring",
extra={"group": logging_group},
)
if not use_overrides:
# limit title to 128 characters
+10 -1
View File
@@ -67,6 +67,7 @@ from documents.utils import identity
from documents.versioning import annotate_effective_content
from documents.workflows.utils import get_workflows_for_trigger
from paperless.config import AIConfig
from paperless.config import RemoteOCRConfig
from paperless.logging import consume_task_id
from paperless.parsers import ParserContext
from paperless.parsers.registry import get_parser_registry
@@ -338,10 +339,17 @@ def bulk_update_documents(document_ids) -> None:
@shared_task
def update_document_content_maybe_archive_file(document_id) -> None:
def update_document_content_maybe_archive_file(
document_id,
*,
remote_ocr: bool = False,
) -> None:
"""
Re-creates OCR content and thumbnail for a document, and archive file if
it exists.
Remote OCR is used only when the engine is configured to handle everything
or if explicitly asked for via ``remote_ocr``.
"""
document = Document.objects.get(id=document_id)
@@ -351,6 +359,7 @@ def update_document_content_maybe_archive_file(document_id) -> None:
mime_type,
document.original_filename or "",
document.source_path,
allow_remote=remote_ocr or RemoteOCRConfig().remote_ocr_by_default,
)
if not parser_class:
@@ -72,6 +72,10 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
"barcode_enable_tag": None,
"barcode_tag_mapping": None,
"barcode_tag_split": None,
"remote_ocr_engine": None,
"remote_ocr_api_key": None,
"remote_ocr_endpoint": None,
"remote_ocr_mode": None,
"ai_enabled": False,
"llm_embedding_backend": None,
"llm_embedding_model": None,
@@ -870,6 +874,49 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
config.refresh_from_db()
self.assertEqual(config.llm_api_key, None)
def test_update_remote_ocr_api_key(self) -> None:
"""
GIVEN:
- Existing config with remote_ocr_api_key specified
WHEN:
- API to update remote_ocr_api_key is called with all *s
- API to update remote_ocr_api_key is called with empty string
THEN:
- remote_ocr_api_key is unchanged
- remote_ocr_api_key is set to None
"""
config = ApplicationConfiguration.objects.first()
assert config is not None
config.remote_ocr_api_key = "1234567890"
config.save()
# Test with all *
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"remote_ocr_api_key": "*" * 32,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.remote_ocr_api_key, "1234567890")
# Test with empty string
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"remote_ocr_api_key": "",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.remote_ocr_api_key, None)
def test_enable_ai_index_triggers_update(self) -> None:
"""
GIVEN:
+46 -1
View File
@@ -532,7 +532,29 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m.assert_called_once()
args, kwargs = m.call_args
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(len(kwargs), 0)
self.assertEqual(kwargs, {"remote_ocr": False})
@mock.patch("documents.views.bulk_edit.reprocess")
def test_reprocess_documents_endpoint_remote_ocr(self, m) -> None:
"""
GIVEN:
- API data to reprocess a document with remote OCR requested
WHEN:
- API is called
THEN:
- reprocess is called with remote_ocr=True
"""
self.setup_mock(m, "reprocess")
response = self.client.post(
"/api/documents/reprocess/",
json.dumps({"documents": [self.doc1.id], "remote_ocr": True}),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {"remote_ocr": True})
@mock.patch("documents.serialisers.bulk_edit.set_storage_path")
def test_api_set_storage_path(self, m) -> None:
@@ -1553,6 +1575,29 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
),
)
def test_legacy_bulk_edit_reprocess_invalid_remote_ocr(self) -> None:
"""
GIVEN:
- The deprecated bulk_edit endpoint with a non-boolean remote_ocr
WHEN:
- API is called
THEN:
- The request is rejected rather than passed through to the task
"""
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc1.id],
"method": "reprocess",
"parameters": {"remote_ocr": "yes please"},
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
@mock.patch("documents.views.bulk_edit.edit_pdf")
def test_edit_pdf(self, m) -> None:
self.setup_mock(m, "edit_pdf")
@@ -5,7 +5,9 @@ from unittest.mock import ANY
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.db import connection
from django.test import override_settings
from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APITestCase
@@ -13,6 +15,9 @@ from rest_framework.test import APITestCase
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.serialisers import CustomFieldInstanceSerializer
from documents.serialisers import DocumentSerializer
from documents.tests.factories import DocumentFactory
from documents.tests.utils import DirectoriesMixin
@@ -530,6 +535,136 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
doc.refresh_from_db()
self.assertEqual(len(doc.custom_fields.all()), 10)
def test_document_serializer_custom_fields_validation_batches_field_lookup(
self,
) -> None:
"""
GIVEN:
- A document is being validated with several custom field values
at once (as happens on every PATCH/PUT/POST)
WHEN:
- The serializer is validated
THEN:
- The referenced CustomField objects are resolved with a single
query, not one query per custom field
"""
doc = DocumentFactory(mime_type="application/pdf")
custom_fields = [
CustomField.objects.create(
name=f"Test Custom Field {i}",
data_type=CustomField.FieldDataType.STRING,
)
for i in range(5)
]
serializer = DocumentSerializer(
doc,
data={
"custom_fields": [
{"field": custom_field.id, "value": "test value"}
for custom_field in custom_fields
],
},
partial=True,
)
with CaptureQueriesContext(connection) as ctx:
self.assertTrue(serializer.is_valid(), serializer.errors)
custom_field_lookups = [
query
for query in ctx.captured_queries
if 'FROM "documents_customfield" WHERE "documents_customfield"."id"'
in query["sql"]
]
self.assertEqual(
len(custom_field_lookups),
1,
"Expected a single batched query to resolve the custom fields, "
f"got {len(custom_field_lookups)}: {custom_field_lookups}",
)
def test_custom_field_lookup_reuses_shared_context_cache(self) -> None:
"""
GIVEN:
- A CustomField has already been resolved once, by a serializer
sharing a given `context` dict
WHEN:
- A second, separately-instantiated CustomFieldInstanceSerializer
validates the same field id, sharing that same context
(this is what drf-writable-nested does: it rebuilds a fresh
serializer -- and fresh field instances -- per item while
matching existing vs. new instances during save())
THEN:
- No additional query is issued to resolve the CustomField
"""
custom_field = CustomField.objects.create(
name="Test Custom Field",
data_type=CustomField.FieldDataType.STRING,
)
context: dict = {}
first_pass = CustomFieldInstanceSerializer(
data={"field": custom_field.id, "value": "a"},
context=context,
)
self.assertTrue(first_pass.is_valid(), first_pass.errors)
second_pass = CustomFieldInstanceSerializer(
data={"field": custom_field.id, "value": "b"},
context=context,
)
with CaptureQueriesContext(connection) as ctx:
self.assertTrue(second_pass.is_valid(), second_pass.errors)
custom_field_lookups = [
query
for query in ctx.captured_queries
if 'FROM "documents_customfield" WHERE "documents_customfield"."id"'
in query["sql"]
]
self.assertEqual(
len(custom_field_lookups),
0,
"Expected the second, separately-instantiated serializer to reuse "
f"the already-resolved CustomField, got: {custom_field_lookups}",
)
def test_custom_field_validation_rejects_malformed_field_value(self) -> None:
"""
GIVEN:
- A document is being validated with a malformed custom_fields
entry whose "field" value is neither a valid CustomField id
nor a type DRF's own PrimaryKeyRelatedField can safely reject
on its own (unhashable, or a non-numeric scalar)
WHEN:
- The serializer is validated
THEN:
- A normal validation error is raised, not an unhandled
TypeError/ValueError escaping past DRF's validation layer
"""
doc = DocumentFactory(mime_type="application/pdf")
bad_field_values = {
"unhashable-list": [],
"unhashable-dict": {},
"non-numeric-scalar": "abc",
}
for case_id, bad_field_value in bad_field_values.items():
with self.subTest(case_id):
serializer = DocumentSerializer(
doc,
data={
"custom_fields": [
{"field": bad_field_value, "value": "test value"},
],
},
partial=True,
)
self.assertFalse(serializer.is_valid())
self.assertIn("custom_fields", serializer.errors)
def test_change_custom_field_instance_value(self) -> None:
"""
GIVEN:
@@ -60,6 +60,10 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
},
"email_enabled": False,
"ai_enabled": False,
"remote_ocr": {
"configured": False,
"mode": "always",
},
},
)
@@ -154,6 +158,50 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
str(response.data["settings"]),
)
@override_settings(
REMOTE_OCR_ENGINE="azureai",
REMOTE_OCR_API_KEY="somekey",
REMOTE_OCR_ENDPOINT="https://example.cognitiveservices.azure.com",
REMOTE_OCR_MODE="workflow_only",
)
def test_settings_reports_remote_ocr_when_configured(self) -> None:
"""
GIVEN:
- A fully configured remote OCR engine in workflow_only mode
WHEN:
- The ui_settings endpoint is called
THEN:
- The UI is told remote OCR is available and selective, so it can
offer it where it would actually change something
"""
response = self.client.get(self.ENDPOINT, format="json")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(
response.data["settings"]["remote_ocr"],
{"configured": True, "mode": "workflow_only"},
)
@override_settings(
REMOTE_OCR_ENGINE="azureai",
REMOTE_OCR_API_KEY=None,
REMOTE_OCR_ENDPOINT=None,
)
def test_settings_reports_remote_ocr_incompletely_configured(self) -> None:
"""
GIVEN:
- An engine named but missing its endpoint and API key
WHEN:
- The ui_settings endpoint is called
THEN:
- It is reported as not configured, matching what the parser
registry will actually do
"""
response = self.client.get(self.ENDPOINT, format="json")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertFalse(response.data["settings"]["remote_ocr"]["configured"])
@override_settings(
OAUTH_CALLBACK_BASE_URL="http://localhost:8000",
GMAIL_OAUTH_CLIENT_ID="abc123",
+135
View File
@@ -506,6 +506,141 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
self.assertEqual(Workflow.objects.count(), 1)
def test_api_create_remote_ocr_action_requires_consumption_trigger(
self,
) -> None:
"""
GIVEN:
- API request to create a workflow with a remote OCR action
- No consumption started trigger, so the action could never run
WHEN:
- API is called
THEN:
- Correct HTTP 400 response
- No objects are created
"""
existing_count = Workflow.objects.count()
response = self.client.post(
self.ENDPOINT,
json.dumps(
{
"name": "Remote OCR too late",
"order": 1,
"triggers": [
{
"type": WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
},
],
"actions": [
{
"type": WorkflowAction.WorkflowActionType.REMOTE_OCR,
},
],
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertEqual(Workflow.objects.count(), existing_count)
def test_api_create_remote_ocr_action_with_consumption_trigger(self) -> None:
"""
GIVEN:
- API request to create a workflow with a remote OCR action
- A consumption started trigger alongside another trigger type
WHEN:
- API is called
THEN:
- The workflow is created, the action applies to consumption only
"""
response = self.client.post(
self.ENDPOINT,
json.dumps(
{
"name": "Remote OCR on consume",
"order": 1,
"triggers": [
{
"type": WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
"filter_filename": "*.pdf",
},
{
"type": WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
},
],
"actions": [
{
"type": WorkflowAction.WorkflowActionType.REMOTE_OCR,
},
],
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
def test_api_partial_update_adds_remote_ocr_action(self) -> None:
"""
GIVEN:
- An existing workflow with a consumption started trigger
WHEN:
- A partial update adds a remote OCR action without resubmitting triggers
THEN:
- The existing trigger is considered and the update succeeds
"""
response = self.client.patch(
f"{self.ENDPOINT}{self.workflow.id}/",
json.dumps(
{
"actions": [
{
"type": WorkflowAction.WorkflowActionType.REMOTE_OCR,
},
],
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(
self.workflow.actions.get().type,
WorkflowAction.WorkflowActionType.REMOTE_OCR,
)
def test_api_partial_update_cannot_remove_remote_ocr_trigger(self) -> None:
"""
GIVEN:
- An existing workflow with a remote OCR action
- An existing consumption started trigger
WHEN:
- A partial update replaces the trigger without resubmitting actions
THEN:
- The existing action is considered and the update is rejected
"""
self.action.type = WorkflowAction.WorkflowActionType.REMOTE_OCR
self.action.save()
response = self.client.patch(
f"{self.ENDPOINT}{self.workflow.id}/",
json.dumps(
{
"triggers": [
{
"type": WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
},
],
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertEqual(self.workflow.triggers.get(), self.trigger)
def test_api_create_workflow_trigger_action_empty_fields(self) -> None:
"""
GIVEN:
+53
View File
@@ -1782,3 +1782,56 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertIn("wrong password", str(exc.exception))
self.assertIn("Error removing password from document", cm.output[0])
class TestBulkEditReprocess(DirectoriesMixin, TestCase):
def setUp(self) -> None:
super().setUp()
self.doc = Document.objects.create(
title="test",
checksum="A",
mime_type="application/pdf",
)
@mock.patch("documents.bulk_edit.update_document_content_maybe_archive_file")
def test_reprocess_defaults_to_local(self, mock_task: mock.Mock) -> None:
"""
GIVEN:
- A reprocess request that says nothing about remote OCR
WHEN:
- reprocess is called
THEN:
- The task is queued without asking for the remote engine
"""
result = bulk_edit.reprocess([self.doc.id])
self.assertEqual(result, "OK")
mock_task.apply_async.assert_called_once()
_, kwargs = mock_task.apply_async.call_args
self.assertEqual(
kwargs["kwargs"],
{"document_id": self.doc.id, "remote_ocr": False},
)
@mock.patch("documents.bulk_edit.update_document_content_maybe_archive_file")
def test_reprocess_passes_remote_ocr(self, mock_task: mock.Mock) -> None:
"""
GIVEN:
- A reprocess request that explicitly asks for remote OCR
WHEN:
- reprocess is called
THEN:
- The request is forwarded to the task for every document
"""
other = Document.objects.create(
title="test2",
checksum="B",
mime_type="application/pdf",
)
bulk_edit.reprocess([self.doc.id, other.id], remote_ocr=True)
self.assertEqual(mock_task.apply_async.call_count, 2)
for call in mock_task.apply_async.call_args_list:
self.assertTrue(call.kwargs["kwargs"]["remote_ocr"])
+80
View File
@@ -1559,6 +1559,72 @@ class PostConsumeTestCase(DirectoriesMixin, GetConsumerMixin, TestCase):
consumer.run_post_consume_script(doc)
class TestConsumerRemoteOCR(
DirectoriesMixin,
FileSystemAssertsMixin,
GetConsumerMixin,
TestCase,
):
"""
The consumer resolves the remote OCR mode and the per-document request from
workflows into the allow_remote flag it hands to the parser registry.
"""
def setUp(self) -> None:
super().setUp()
patcher = mock.patch("documents.consumer.get_parser_registry")
self.mock_registry = patcher.start()
self.mock_registry.return_value.get_parser_for_file.return_value = DummyParser
self.addCleanup(patcher.stop)
def _consume(self, *, overrides: DocumentMetadataOverrides | None = None) -> bool:
src = (
Path(__file__).parent
/ "samples"
/ "documents"
/ "originals"
/ "0000001.pdf"
)
dst = self.dirs.scratch_dir / "sample.pdf"
shutil.copy(src, dst)
with self.get_consumer(dst, overrides=overrides) as consumer:
consumer.run()
_, kwargs = self.mock_registry.return_value.get_parser_for_file.call_args
return kwargs["allow_remote"]
@override_settings(REMOTE_OCR_MODE="always")
def test_always_mode_allows_remote(self) -> None:
"""
GIVEN: Remote OCR mode is 'always'.
WHEN: A document is consumed without any workflow asking for it.
THEN: The registry is allowed to pick the remote parser.
"""
self.assertTrue(self._consume())
@override_settings(REMOTE_OCR_MODE="workflow_only")
def test_workflow_only_mode_denies_remote_by_default(self) -> None:
"""
GIVEN: Remote OCR mode is 'workflow_only'.
WHEN: A document is consumed and nothing asked for remote OCR.
THEN: The remote parser is excluded.
"""
self.assertFalse(self._consume())
@override_settings(REMOTE_OCR_MODE="workflow_only")
def test_workflow_only_mode_allows_remote_when_requested(self) -> None:
"""
GIVEN: Remote OCR mode is 'workflow_only'.
WHEN: A workflow set remote_ocr on the metadata overrides.
THEN: The registry is allowed to pick the remote parser.
"""
self.assertTrue(
self._consume(overrides=DocumentMetadataOverrides(remote_ocr=True)),
)
class TestMetadataOverrides(TestCase):
def test_update_skip_asn_if_exists(self) -> None:
base = DocumentMetadataOverrides()
@@ -1566,6 +1632,20 @@ class TestMetadataOverrides(TestCase):
base.update(incoming)
self.assertTrue(base.skip_asn_if_exists)
def test_update_remote_ocr(self) -> None:
base = DocumentMetadataOverrides()
base.update(DocumentMetadataOverrides(remote_ocr=True))
self.assertTrue(base.remote_ocr)
def test_update_remote_ocr_is_not_unset(self) -> None:
"""
A later workflow that says nothing must not undo an earlier one that
asked for remote OCR.
"""
base = DocumentMetadataOverrides(remote_ocr=True)
base.update(DocumentMetadataOverrides())
self.assertTrue(base.remote_ocr)
def test_update_actor_and_version_label(self) -> None:
base = DocumentMetadataOverrides(
actor_id=1,
+58
View File
@@ -0,0 +1,58 @@
from django.db import connection
from django.test import TestCase
from django.test.utils import CaptureQueriesContext
from documents.data_models import DocumentMetadataOverrides
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.tests.factories import DocumentFactory
from documents.tests.utils import DirectoriesMixin
class TestDocumentMetadataOverridesFromDocument(DirectoriesMixin, TestCase):
def test_from_document_batches_custom_field_lookup_after_refresh_from_db(
self,
) -> None:
"""
GIVEN:
- A document has several custom field values
- The document instance has just been refreshed from the database,
which drops any prefetched related objects (as
send_websocket_document_updated does before building overrides)
WHEN:
- DocumentMetadataOverrides.from_document() reads the document's
custom field values
THEN:
- The referenced CustomField objects are resolved with a single
query, not one query per custom field
"""
doc = DocumentFactory(mime_type="application/pdf")
for i in range(5):
CustomFieldInstance.objects.create(
document=doc,
field=CustomField.objects.create(
name=f"Test Custom Field {i}",
data_type=CustomField.FieldDataType.STRING,
),
value_text="value",
)
doc.refresh_from_db()
with CaptureQueriesContext(connection) as ctx:
overrides = DocumentMetadataOverrides.from_document(doc)
self.assertEqual(len(overrides.custom_fields), 5)
unbatched_field_lookups = [
query
for query in ctx.captured_queries
if 'FROM "documents_customfield" WHERE "documents_customfield"."id"'
in query["sql"]
]
self.assertEqual(
unbatched_field_lookups,
[],
"Expected CustomField data to come from the CustomFieldInstance "
"join, not a separate per-instance lookup, "
f"got: {unbatched_field_lookups}",
)
+39
View File
@@ -287,6 +287,45 @@ class TestUpdateContent(DirectoriesMixin, TestCase):
self.assertNotEqual(Document.objects.get(pk=doc.pk).content, "test")
class TestUpdateContentRemoteOCR(DirectoriesMixin, TestCase):
"""
Consumption workflows do not run on reprocess, so the remote parser is
used only in 'always' mode or when the caller explicitly asks for it.
"""
def setUp(self) -> None:
super().setUp()
patcher = mock.patch("documents.tasks.get_parser_registry")
self.mock_registry = patcher.start()
self.mock_registry.return_value.get_parser_for_file.return_value = None
self.addCleanup(patcher.stop)
self.doc = Document.objects.create(
title="test",
content="my document",
checksum="wow",
mime_type="application/pdf",
)
def _allow_remote(self, **kwargs) -> bool:
tasks.update_document_content_maybe_archive_file(self.doc.pk, **kwargs)
_, call_kwargs = self.mock_registry.return_value.get_parser_for_file.call_args
return call_kwargs["allow_remote"]
@override_settings(REMOTE_OCR_MODE="always")
def test_always_mode_allows_remote(self) -> None:
self.assertTrue(self._allow_remote())
@override_settings(REMOTE_OCR_MODE="workflow_only")
def test_workflow_only_mode_denies_remote_by_default(self) -> None:
self.assertFalse(self._allow_remote())
@override_settings(REMOTE_OCR_MODE="workflow_only")
def test_workflow_only_mode_allows_remote_when_requested(self) -> None:
self.assertTrue(self._allow_remote(remote_ocr=True))
class TestAIIndex(DirectoriesMixin, TestCase):
@override_settings(
AI_ENABLED=True,
+79
View File
@@ -5409,3 +5409,82 @@ class TestDateWorkflowLocalization(
document = Document.objects.first()
assert document is not None
assert document.title == expected_title
class TestRemoteOCRWorkflowAction(DirectoriesMixin, SampleDirMixin, APITestCase):
def _make_workflow(self, trigger_type) -> None:
trigger = WorkflowTrigger.objects.create(type=trigger_type)
action = WorkflowAction.objects.create(
type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
)
w = Workflow.objects.create(name="Remote OCR", order=0)
w.triggers.add(trigger)
w.actions.add(action)
w.save()
def test_consumption_trigger_requests_remote_ocr(self) -> None:
"""
GIVEN:
- A consumption workflow with a remote OCR action
WHEN:
- A matching document is consumed
THEN:
- The overrides ask for remote OCR, which is what the consumer
reads when choosing a parser
"""
self._make_workflow(WorkflowTrigger.WorkflowTriggerType.CONSUMPTION)
test_file = shutil.copy(
self.SAMPLE_DIR / "simple.pdf",
self.dirs.scratch_dir / "simple.pdf",
)
overrides = DocumentMetadataOverrides()
run_workflows(
WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=test_file,
),
overrides=overrides,
)
self.assertTrue(overrides.remote_ocr)
def test_other_trigger_types_are_ignored(self) -> None:
"""
GIVEN:
- A workflow with a remote OCR action that also has a
non-consumption trigger, which is a valid combination
WHEN:
- The non-consumption trigger fires
THEN:
- The action is skipped, since the document has already been
parsed by this point
"""
trigger = WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
)
updated_trigger = WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
)
action = WorkflowAction.objects.create(
type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
)
w = Workflow.objects.create(name="Remote OCR", order=0)
w.triggers.add(trigger, updated_trigger)
w.actions.add(action)
w.save()
doc = Document.objects.create(
title="sample test",
original_filename="sample.pdf",
)
with self.assertLogs("paperless.handlers", level="DEBUG") as cm:
run_workflows(
WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
doc,
)
self.assertIn("only applies to consumption triggers", "".join(cm.output))
+7
View File
@@ -239,8 +239,10 @@ from paperless import version
from paperless.celery import app as celery_app
from paperless.config import AIConfig
from paperless.config import GeneralConfig
from paperless.config import RemoteOCRConfig
from paperless.models import ApplicationConfiguration
from paperless.parsers.registry import get_parser_registry
from paperless.parsers.remote import RemoteEngineConfig
from paperless.serialisers import GroupSerializer
from paperless.serialisers import UserSerializer
from paperless.views import StandardPagination
@@ -4100,6 +4102,11 @@ class UiSettingsView(GenericAPIView[Any]):
ui_settings["auditlog_enabled"] = settings.AUDIT_LOG_ENABLED
ui_settings["remote_ocr"] = {
"configured": RemoteEngineConfig.from_app_config().engine_is_valid(),
"mode": RemoteOCRConfig().remote_ocr_mode,
}
if settings.GMAIL_OAUTH_ENABLED or settings.OUTLOOK_OAUTH_ENABLED:
manager = PaperlessMailOAuth2Manager()
if settings.GMAIL_OAUTH_ENABLED:
+157 -125
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr ""
"Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-08-19 18:18+0000\n"
"POT-Creation-Date: 2026-08-24 21:43+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n"
"Language-Team: English\n"
@@ -89,7 +89,7 @@ msgstr ""
msgid "Automatic"
msgstr ""
#: documents/models.py:65 documents/models.py:606 documents/models.py:1920
#: documents/models.py:65 documents/models.py:606 documents/models.py:1924
#: paperless_mail/models.py:23 paperless_mail/models.py:143
msgid "name"
msgstr ""
@@ -272,7 +272,7 @@ msgid "Optional short label for a document version."
msgstr ""
#: documents/models.py:340 documents/models.py:922 documents/models.py:976
#: documents/models.py:1963
#: documents/models.py:1967
msgid "document"
msgstr ""
@@ -1385,198 +1385,202 @@ msgid "Move to trash"
msgstr ""
#: documents/models.py:1675
msgid "Remote OCR"
msgstr ""
#: documents/models.py:1679
msgid "Workflow Action Type"
msgstr ""
#: documents/models.py:1680 documents/models.py:1922
#: documents/models.py:1684 documents/models.py:1926
#: paperless_mail/models.py:145
msgid "order"
msgstr ""
#: documents/models.py:1683
#: documents/models.py:1687
msgid "assign title"
msgstr ""
#: documents/models.py:1687
#: documents/models.py:1691
msgid "Assign a document title, must be a Jinja2 template, see documentation."
msgstr ""
#: documents/models.py:1695 paperless_mail/models.py:274
#: documents/models.py:1699 paperless_mail/models.py:274
msgid "assign this tag"
msgstr ""
#: documents/models.py:1704 paperless_mail/models.py:282
#: documents/models.py:1708 paperless_mail/models.py:282
msgid "assign this document type"
msgstr ""
#: documents/models.py:1713 paperless_mail/models.py:296
#: documents/models.py:1717 paperless_mail/models.py:296
msgid "assign this correspondent"
msgstr ""
#: documents/models.py:1722
#: documents/models.py:1726
msgid "assign this storage path"
msgstr ""
#: documents/models.py:1731
#: documents/models.py:1735
msgid "assign this owner"
msgstr ""
#: documents/models.py:1738
#: documents/models.py:1742
msgid "grant view permissions to these users"
msgstr ""
#: documents/models.py:1745
#: documents/models.py:1749
msgid "grant view permissions to these groups"
msgstr ""
#: documents/models.py:1752
#: documents/models.py:1756
msgid "grant change permissions to these users"
msgstr ""
#: documents/models.py:1759
#: documents/models.py:1763
msgid "grant change permissions to these groups"
msgstr ""
#: documents/models.py:1766
#: documents/models.py:1770
msgid "assign these custom fields"
msgstr ""
#: documents/models.py:1770
#: documents/models.py:1774
msgid "custom field values"
msgstr ""
#: documents/models.py:1774
#: documents/models.py:1778
msgid "Optional values to assign to the custom fields."
msgstr ""
#: documents/models.py:1783
#: documents/models.py:1787
msgid "remove these tag(s)"
msgstr ""
#: documents/models.py:1788
#: documents/models.py:1792
msgid "remove all tags"
msgstr ""
#: documents/models.py:1795
#: documents/models.py:1799
msgid "remove these document type(s)"
msgstr ""
#: documents/models.py:1800
#: documents/models.py:1804
msgid "remove all document types"
msgstr ""
#: documents/models.py:1807
#: documents/models.py:1811
msgid "remove these correspondent(s)"
msgstr ""
#: documents/models.py:1812
#: documents/models.py:1816
msgid "remove all correspondents"
msgstr ""
#: documents/models.py:1819
#: documents/models.py:1823
msgid "remove these storage path(s)"
msgstr ""
#: documents/models.py:1824
#: documents/models.py:1828
msgid "remove all storage paths"
msgstr ""
#: documents/models.py:1831
#: documents/models.py:1835
msgid "remove these owner(s)"
msgstr ""
#: documents/models.py:1836
#: documents/models.py:1840
msgid "remove all owners"
msgstr ""
#: documents/models.py:1843
#: documents/models.py:1847
msgid "remove view permissions for these users"
msgstr ""
#: documents/models.py:1850
#: documents/models.py:1854
msgid "remove view permissions for these groups"
msgstr ""
#: documents/models.py:1857
#: documents/models.py:1861
msgid "remove change permissions for these users"
msgstr ""
#: documents/models.py:1864
#: documents/models.py:1868
msgid "remove change permissions for these groups"
msgstr ""
#: documents/models.py:1869
#: documents/models.py:1873
msgid "remove all permissions"
msgstr ""
#: documents/models.py:1876
#: documents/models.py:1880
msgid "remove these custom fields"
msgstr ""
#: documents/models.py:1881
#: documents/models.py:1885
msgid "remove all custom fields"
msgstr ""
#: documents/models.py:1890
#: documents/models.py:1894
msgid "email"
msgstr ""
#: documents/models.py:1899
#: documents/models.py:1903
msgid "webhook"
msgstr ""
#: documents/models.py:1903
#: documents/models.py:1907
msgid "passwords"
msgstr ""
#: documents/models.py:1907
#: documents/models.py:1911
msgid ""
"Passwords to try when removing PDF protection. Separate with commas or new "
"lines."
msgstr ""
#: documents/models.py:1912
#: documents/models.py:1916
msgid "workflow action"
msgstr ""
#: documents/models.py:1913
#: documents/models.py:1917
msgid "workflow actions"
msgstr ""
#: documents/models.py:1928
#: documents/models.py:1932
msgid "triggers"
msgstr ""
#: documents/models.py:1935
#: documents/models.py:1939
msgid "actions"
msgstr ""
#: documents/models.py:1938 paperless_mail/models.py:154
#: documents/models.py:1942 paperless_mail/models.py:154
msgid "enabled"
msgstr ""
#: documents/models.py:1949
#: documents/models.py:1953
msgid "workflow"
msgstr ""
#: documents/models.py:1953
#: documents/models.py:1957
msgid "workflow trigger type"
msgstr ""
#: documents/models.py:1967
#: documents/models.py:1971
msgid "date run"
msgstr ""
#: documents/models.py:1973
#: documents/models.py:1977
msgid "workflow run"
msgstr ""
#: documents/models.py:1974
#: documents/models.py:1978
msgid "workflow runs"
msgstr ""
#: documents/serialisers.py:523 documents/serialisers.py:875
#: documents/serialisers.py:2818 documents/views.py:309 documents/views.py:2610
#: documents/serialisers.py:2827 documents/views.py:311 documents/views.py:2612
#: paperless_mail/serialisers.py:155
msgid "Insufficient permissions."
msgstr ""
@@ -1585,39 +1589,39 @@ msgstr ""
msgid "Invalid color."
msgstr ""
#: documents/serialisers.py:2295
#: documents/serialisers.py:2304
#, python-format
msgid "File type %(type)s not supported"
msgstr ""
#: documents/serialisers.py:2339
#: documents/serialisers.py:2348
#, python-format
msgid "Custom field id must be an integer: %(id)s"
msgstr ""
#: documents/serialisers.py:2346
#: documents/serialisers.py:2355
#, python-format
msgid "Custom field with id %(id)s does not exist"
msgstr ""
#: documents/serialisers.py:2363 documents/serialisers.py:2373
#: documents/serialisers.py:2372 documents/serialisers.py:2382
msgid ""
"Custom fields must be a list of integers or an object mapping ids to values."
msgstr ""
#: documents/serialisers.py:2368
#: documents/serialisers.py:2377
msgid "Some custom fields don't exist or were specified twice."
msgstr ""
#: documents/serialisers.py:2515
#: documents/serialisers.py:2524
msgid "Invalid variable detected."
msgstr ""
#: documents/serialisers.py:2874
#: documents/serialisers.py:2883
msgid "Duplicate document identifiers are not allowed."
msgstr ""
#: documents/serialisers.py:2904 documents/views.py:4599
#: documents/serialisers.py:2913 documents/views.py:4606
#, python-format
msgid "Documents not found: %(ids)s"
msgstr ""
@@ -1885,36 +1889,36 @@ msgstr ""
msgid "Unable to parse URI {value}"
msgstr ""
#: documents/views.py:302 documents/views.py:2607
#: documents/views.py:304 documents/views.py:2609
msgid "Invalid more_like_id"
msgstr ""
#: documents/views.py:1584
#: documents/views.py:1586
msgid "Invalid AI configuration."
msgstr ""
#: documents/views.py:1595
#: documents/views.py:1597
msgid "AI backend request timed out."
msgstr ""
#: documents/views.py:2432 documents/views.py:2753
#: documents/views.py:2434 documents/views.py:2755
msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr ""
#: documents/views.py:4612
#: documents/views.py:4619
#, python-format
msgid "Insufficient permissions to share document %(id)s."
msgstr ""
#: documents/views.py:4658
#: documents/views.py:4665
msgid "Bundle is already being processed."
msgstr ""
#: documents/views.py:4719
#: documents/views.py:4726
msgid "The share link bundle is still being prepared. Please try again later."
msgstr ""
#: documents/views.py:4729
#: documents/views.py:4736
msgid "The share link bundle is unavailable."
msgstr ""
@@ -1998,175 +2002,203 @@ msgstr ""
msgid "CMYK"
msgstr ""
#: paperless/models.py:78 paperless/models.py:88
#: paperless/models.py:82
msgid "Azure AI Document Intelligence"
msgstr ""
#: paperless/models.py:90
msgid "All supported documents"
msgstr ""
#: paperless/models.py:91
msgid "Only when a workflow enables it"
msgstr ""
#: paperless/models.py:95 paperless/models.py:105
msgid "OpenAI-compatible"
msgstr ""
#: paperless/models.py:79
#: paperless/models.py:96
msgid "Huggingface"
msgstr ""
#: paperless/models.py:80 paperless/models.py:89
#: paperless/models.py:97 paperless/models.py:106
msgid "Ollama"
msgstr ""
#: paperless/models.py:98
#: paperless/models.py:115
msgid "Sets the output PDF type"
msgstr ""
#: paperless/models.py:110
#: paperless/models.py:127
msgid "Do OCR from page 1 to this value"
msgstr ""
#: paperless/models.py:116
#: paperless/models.py:133
msgid "Do OCR using these languages"
msgstr ""
#: paperless/models.py:123
#: paperless/models.py:140
msgid "Sets the OCR mode"
msgstr ""
#: paperless/models.py:131
#: paperless/models.py:148
msgid "Controls archive file generation"
msgstr ""
#: paperless/models.py:139
#: paperless/models.py:156
msgid "Sets image DPI fallback value"
msgstr ""
#: paperless/models.py:146
#: paperless/models.py:163
msgid "Controls the unpaper cleaning"
msgstr ""
#: paperless/models.py:153
#: paperless/models.py:170
msgid "Enables deskew"
msgstr ""
#: paperless/models.py:156
#: paperless/models.py:173
msgid "Enables page rotation"
msgstr ""
#: paperless/models.py:161
#: paperless/models.py:178
msgid "Sets the threshold for rotation of pages"
msgstr ""
#: paperless/models.py:167
#: paperless/models.py:184
msgid "Sets the maximum image size for decompression"
msgstr ""
#: paperless/models.py:173
#: paperless/models.py:190
msgid "Sets the Ghostscript color conversion strategy"
msgstr ""
#: paperless/models.py:181
#: paperless/models.py:198
msgid "Adds additional user arguments for OCRMyPDF"
msgstr ""
#: paperless/models.py:190
#: paperless/models.py:207
msgid "Application title"
msgstr ""
#: paperless/models.py:197
#: paperless/models.py:214
msgid "Application logo"
msgstr ""
#: paperless/models.py:212
#: paperless/models.py:229
msgid "Enables barcode scanning"
msgstr ""
#: paperless/models.py:218
#: paperless/models.py:235
msgid "Enables barcode TIFF support"
msgstr ""
#: paperless/models.py:224
#: paperless/models.py:241
msgid "Sets the barcode string"
msgstr ""
#: paperless/models.py:232
#: paperless/models.py:249
msgid "Retains split pages"
msgstr ""
#: paperless/models.py:238
#: paperless/models.py:255
msgid "Enables ASN barcode"
msgstr ""
#: paperless/models.py:244
#: paperless/models.py:261
msgid "Sets the ASN barcode prefix"
msgstr ""
#: paperless/models.py:252
#: paperless/models.py:269
msgid "Sets the barcode upscale factor"
msgstr ""
#: paperless/models.py:259
#: paperless/models.py:276
msgid "Sets the barcode DPI"
msgstr ""
#: paperless/models.py:266
#: paperless/models.py:283
msgid "Sets the maximum pages for barcode"
msgstr ""
#: paperless/models.py:273
#: paperless/models.py:290
msgid "Enables tag barcode"
msgstr ""
#: paperless/models.py:279
#: paperless/models.py:296
msgid "Sets the tag barcode mapping"
msgstr ""
#: paperless/models.py:285
#: paperless/models.py:302
msgid "Enables splitting on tag barcodes"
msgstr ""
#: paperless/models.py:294
msgid "Enables AI features"
#: paperless/models.py:312
msgid "Sets the remote OCR engine"
msgstr ""
#: paperless/models.py:300
msgid "Sets the LLM embedding backend"
#: paperless/models.py:321
msgid "Sets the remote OCR API key"
msgstr ""
#: paperless/models.py:308
msgid "Sets the LLM embedding model"
#: paperless/models.py:329
msgid "Sets the remote OCR endpoint"
msgstr ""
#: paperless/models.py:315
msgid "Sets the LLM embedding endpoint, optional"
msgstr ""
#: paperless/models.py:322
msgid "Sets the LLM embedding chunk size"
msgstr ""
#: paperless/models.py:328
msgid "Sets the LLM context size"
msgstr ""
#: paperless/models.py:334
msgid "Sets the LLM backend"
msgstr ""
#: paperless/models.py:342
msgid "Sets the LLM model"
#: paperless/models.py:337
msgid "Sets which documents are sent to the remote OCR engine"
msgstr ""
#: paperless/models.py:349
msgid "Sets the LLM API key"
msgid "Enables AI features"
msgstr ""
#: paperless/models.py:356
msgid "Sets the LLM endpoint, optional"
#: paperless/models.py:355
msgid "Sets the LLM embedding backend"
msgstr ""
#: paperless/models.py:363
msgid "Sets the LLM output language"
msgid "Sets the LLM embedding model"
msgstr ""
#: paperless/models.py:370
msgid "Sets the LLM embedding endpoint, optional"
msgstr ""
#: paperless/models.py:377
msgid "Sets the LLM embedding chunk size"
msgstr ""
#: paperless/models.py:383
msgid "Sets the LLM context size"
msgstr ""
#: paperless/models.py:389
msgid "Sets the LLM backend"
msgstr ""
#: paperless/models.py:397
msgid "Sets the LLM model"
msgstr ""
#: paperless/models.py:404
msgid "Sets the LLM API key"
msgstr ""
#: paperless/models.py:411
msgid "Sets the LLM endpoint, optional"
msgstr ""
#: paperless/models.py:418
msgid "Sets the LLM output language"
msgstr ""
#: paperless/models.py:425
msgid "Sets the LLM timeout in seconds"
msgstr ""
#: paperless/models.py:376
#: paperless/models.py:431
msgid "paperless application settings"
msgstr ""
-14
View File
@@ -337,20 +337,6 @@ def check_deprecated_v2_ocr_env_vars(
return warnings
@register()
def check_remote_parser_configured(app_configs: Any, **kwargs: Any) -> list[Error]:
if settings.REMOTE_OCR_ENGINE == "azureai" and not (
settings.REMOTE_OCR_ENDPOINT and settings.REMOTE_OCR_API_KEY
):
return [
Error(
"Azure AI remote parser requires endpoint and API key to be configured.",
),
]
return []
def get_tesseract_langs():
proc = subprocess.run(
[shutil.which("tesseract"), "--list-langs"],
+40
View File
@@ -9,6 +9,7 @@ from paperless.models import CleanChoices
from paperless.models import ColorConvertChoices
from paperless.models import ModeChoices
from paperless.models import OutputTypeChoices
from paperless.models import RemoteOCRMode
@dataclasses.dataclass
@@ -185,6 +186,45 @@ class GeneralConfig(BaseConfig):
self.app_logo = app_config.app_logo.url if app_config.app_logo else None
@dataclasses.dataclass
class RemoteOCRConfig(BaseConfig):
"""
Settings for the remote (cloud) OCR parser
"""
remote_ocr_engine: str | None = dataclasses.field(init=False)
remote_ocr_api_key: str | None = dataclasses.field(init=False)
remote_ocr_endpoint: str | None = dataclasses.field(init=False)
remote_ocr_mode: RemoteOCRMode = dataclasses.field(init=False)
def __post_init__(self) -> None:
app_config = self._get_config_instance()
self.remote_ocr_engine = (
app_config.remote_ocr_engine or settings.REMOTE_OCR_ENGINE
)
self.remote_ocr_api_key = (
app_config.remote_ocr_api_key or settings.REMOTE_OCR_API_KEY
)
self.remote_ocr_endpoint = (
app_config.remote_ocr_endpoint or settings.REMOTE_OCR_ENDPOINT
)
self.remote_ocr_mode = app_config.remote_ocr_mode or RemoteOCRMode(
settings.REMOTE_OCR_MODE,
)
@property
def remote_ocr_by_default(self) -> bool:
"""
Whether every supported document goes to the remote engine.
When False the remote engine is used only for documents that
explicitly asked for it, i.e. a workflow matched during consumption or
the user ticked the box when reprocessing.
"""
return self.remote_ocr_mode == RemoteOCRMode.ALWAYS
@dataclasses.dataclass
class AIConfig(BaseConfig):
"""
@@ -0,0 +1,44 @@
# Generated by Django 5.2.16 on 2026-08-10 14:37
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("paperless", "0013_applicationconfiguration_llm_request_timeout"),
]
operations = [
migrations.AddField(
model_name="applicationconfiguration",
name="remote_ocr_api_key",
field=models.CharField(
blank=True,
max_length=1024,
null=True,
verbose_name="Sets the remote OCR API key",
),
),
migrations.AddField(
model_name="applicationconfiguration",
name="remote_ocr_endpoint",
field=models.CharField(
blank=True,
max_length=256,
null=True,
verbose_name="Sets the remote OCR endpoint",
),
),
migrations.AddField(
model_name="applicationconfiguration",
name="remote_ocr_engine",
field=models.CharField(
blank=True,
choices=[("azureai", "Azure AI Document Intelligence")],
max_length=32,
null=True,
verbose_name="Sets the remote OCR engine",
),
),
]
@@ -0,0 +1,27 @@
# Generated by Django 5.2.16 on 2026-08-10 15:43
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("paperless", "0014_applicationconfiguration_remote_ocr_api_key_and_more"),
]
operations = [
migrations.AddField(
model_name="applicationconfiguration",
name="remote_ocr_mode",
field=models.CharField(
blank=True,
choices=[
("always", "All supported documents"),
("workflow_only", "Only when a workflow enables it"),
],
max_length=32,
null=True,
verbose_name="Sets which documents are sent to the remote OCR engine",
),
),
]
+55
View File
@@ -74,6 +74,23 @@ class ColorConvertChoices(models.TextChoices):
CMYK = ("CMYK", _("CMYK"))
class RemoteOCREngine(models.TextChoices):
"""
Matches to PAPERLESS_REMOTE_OCR_ENGINE
"""
AZURE_AI = ("azureai", _("Azure AI Document Intelligence"))
class RemoteOCRMode(models.TextChoices):
"""
Matches to PAPERLESS_REMOTE_OCR_MODE
"""
ALWAYS = ("always", _("All supported documents"))
WORKFLOW_ONLY = ("workflow_only", _("Only when a workflow enables it"))
class LLMEmbeddingBackend(models.TextChoices):
OPENAI_LIKE = ("openai-like", _("OpenAI-compatible"))
HUGGINGFACE = ("huggingface", _("Huggingface"))
@@ -286,6 +303,44 @@ class ApplicationConfiguration(AbstractSingletonModel):
null=True,
)
"""
Settings for the remote OCR parser
"""
# PAPERLESS_REMOTE_OCR_ENGINE
remote_ocr_engine = models.CharField(
verbose_name=_("Sets the remote OCR engine"),
blank=True,
null=True,
max_length=32,
choices=RemoteOCREngine.choices,
)
# PAPERLESS_REMOTE_OCR_API_KEY
remote_ocr_api_key = models.CharField(
verbose_name=_("Sets the remote OCR API key"),
blank=True,
null=True,
max_length=1024,
)
# PAPERLESS_REMOTE_OCR_ENDPOINT
remote_ocr_endpoint = models.CharField(
verbose_name=_("Sets the remote OCR endpoint"),
blank=True,
null=True,
max_length=256,
)
# PAPERLESS_REMOTE_OCR_MODE
remote_ocr_mode = models.CharField(
verbose_name=_("Sets which documents are sent to the remote OCR engine"),
blank=True,
null=True,
max_length=32,
choices=RemoteOCRMode.choices,
)
"""
AI related settings
"""
+9
View File
@@ -134,6 +134,11 @@ class ParserProtocol(Protocol):
Author or organisation name.
url : str
URL for documentation, source code, or issue tracker.
Parsers that send document content to a remote service should additionally
set ``uses_remote_service = True`` so the registry can exclude them when
remote processing has not been requested for a document. The attribute is
optional so a parser that omits it is treated as fully local.
"""
# ------------------------------------------------------------------
@@ -145,6 +150,10 @@ class ParserProtocol(Protocol):
author: str
url: str
# NOTE: uses_remote_service is not declared here, the registry reads it
# with getattr(cls, ..., False) for backwards-compatibility with existing
# parsers
# ------------------------------------------------------------------
# Class methods
# ------------------------------------------------------------------
+14
View File
@@ -334,6 +334,8 @@ class ParserRegistry:
mime_type: str,
filename: str,
path: Path | None = None,
*,
allow_remote: bool = True,
) -> type[ParserProtocol] | None:
"""Return the best parser class for the given file, or None.
@@ -359,6 +361,11 @@ class ParserRegistry:
path:
Optional filesystem path to the file. Forwarded to each
parser's score method.
allow_remote:
When False, parsers that declare ``uses_remote_service = True``
are excluded from consideration, so a document is never sent to
a remote service. Parsers that do not declare the attribute
are treated as local and are always considered.
Returns
-------
@@ -374,6 +381,13 @@ class ParserRegistry:
if mime_type not in parser_class.supported_mime_types():
continue
if not allow_remote and getattr(
parser_class,
"uses_remote_service",
False,
):
continue
score = parser_class.score(mime_type, filename, path)
if score is None:
continue
+19 -10
View File
@@ -61,6 +61,18 @@ class RemoteEngineConfig:
self.api_key = api_key
self.endpoint = endpoint
@classmethod
def from_app_config(cls) -> Self:
"""Build the config from the app config, falling back to the env."""
from paperless.config import RemoteOCRConfig
app_config = RemoteOCRConfig()
return cls(
engine=app_config.remote_ocr_engine,
api_key=app_config.remote_ocr_api_key,
endpoint=app_config.remote_ocr_endpoint,
)
def engine_is_valid(self) -> bool:
"""Return True when the engine is known and fully configured."""
return (
@@ -90,6 +102,9 @@ class RemoteDocumentParser:
Maintainer name.
url : str
Issue tracker / source URL.
uses_remote_service : bool
Content is sent to a remote service, True so that the registry
can skip this parser if remote processing was not requested.
"""
name: str = "Paperless-ngx Remote OCR Parser"
@@ -97,6 +112,8 @@ class RemoteDocumentParser:
author: str = "Paperless-ngx Contributors"
url: str = "https://github.com/paperless-ngx/paperless-ngx"
uses_remote_service: bool = True
# ------------------------------------------------------------------
# Class methods
# ------------------------------------------------------------------
@@ -145,11 +162,7 @@ class RemoteDocumentParser:
20 when the remote engine is configured and the MIME type is
supported, otherwise None.
"""
config = RemoteEngineConfig(
engine=settings.REMOTE_OCR_ENGINE,
api_key=settings.REMOTE_OCR_API_KEY,
endpoint=settings.REMOTE_OCR_ENDPOINT,
)
config = RemoteEngineConfig.from_app_config()
if not config.engine_is_valid():
return None
if mime_type not in _SUPPORTED_MIME_TYPES:
@@ -244,11 +257,7 @@ class RemoteDocumentParser:
Whether an archive copy is wanted. For PDFs, False skips the
remote engine and uses locally-extracted text instead.
"""
config = RemoteEngineConfig(
engine=settings.REMOTE_OCR_ENGINE,
api_key=settings.REMOTE_OCR_API_KEY,
endpoint=settings.REMOTE_OCR_ENDPOINT,
)
config = RemoteEngineConfig.from_app_config()
if not config.engine_is_valid():
logger.warning(
+14 -5
View File
@@ -219,6 +219,13 @@ class ApplicationConfigurationSerializer(
allow_null=True,
max_length=1024,
)
remote_ocr_api_key = ObfuscatedPasswordField(
required=False,
allow_null=True,
max_length=1024,
)
OBFUSCATED_FIELDS = ("llm_api_key", "remote_ocr_api_key")
def run_validation(self, data):
# Empty strings treated as None to avoid unexpected behavior
@@ -230,11 +237,13 @@ class ApplicationConfigurationSerializer(
data["language"] = None
if "llm_output_language" in data and data["llm_output_language"] == "":
data["llm_output_language"] = None
if "llm_api_key" in data and data["llm_api_key"] is not None:
if data["llm_api_key"] == "":
data["llm_api_key"] = None
elif len(data["llm_api_key"].replace("*", "")) == 0:
del data["llm_api_key"]
for field in self.OBFUSCATED_FIELDS:
if field in data and data[field] is not None:
if data[field] == "":
data[field] = None
# Not a real value, don't overwrite the stored one
elif len(data[field].replace("*", "")) == 0:
del data[field]
return super().run_validation(data)
def update(self, instance, validated_data):
+5
View File
@@ -1203,6 +1203,11 @@ WEBHOOKS_ALLOW_INTERNAL_REQUESTS = get_bool_from_env(
REMOTE_OCR_ENGINE = os.getenv("PAPERLESS_REMOTE_OCR_ENGINE")
REMOTE_OCR_API_KEY = os.getenv("PAPERLESS_REMOTE_OCR_API_KEY")
REMOTE_OCR_ENDPOINT = os.getenv("PAPERLESS_REMOTE_OCR_ENDPOINT")
REMOTE_OCR_MODE = get_choice_from_env(
"PAPERLESS_REMOTE_OCR_MODE",
{"always", "workflow_only"},
default="always",
)
################################################################################
# AI Settings #
+24 -2
View File
@@ -114,7 +114,26 @@ def remote_parser() -> Generator[RemoteDocumentParser, None, None]:
@pytest.fixture()
def azure_settings(settings: SettingsWrapper) -> SettingsWrapper:
def empty_remote_ocr_app_config(mocker: MockerFixture) -> MagicMock:
# empty app config without accessing db
app_config = mocker.MagicMock(
remote_ocr_engine=None,
remote_ocr_api_key=None,
remote_ocr_endpoint=None,
remote_ocr_mode=None,
)
mocker.patch(
"paperless.config.BaseConfig._get_config_instance",
return_value=app_config,
)
return app_config
@pytest.fixture()
def azure_settings(
settings: SettingsWrapper,
empty_remote_ocr_app_config: MagicMock,
) -> SettingsWrapper:
"""Configure Django settings for a valid Azure AI OCR engine.
Sets ``REMOTE_OCR_ENGINE``, ``REMOTE_OCR_API_KEY``, and
@@ -133,7 +152,10 @@ def azure_settings(settings: SettingsWrapper) -> SettingsWrapper:
@pytest.fixture()
def no_engine_settings(settings: SettingsWrapper) -> SettingsWrapper:
def no_engine_settings(
settings: SettingsWrapper,
empty_remote_ocr_app_config: MagicMock,
) -> SettingsWrapper:
"""Configure Django settings with no remote engine configured.
Returns
@@ -21,6 +21,7 @@ from unittest.mock import Mock
import pytest
from documents.parsers import ParseError
from paperless.models import ApplicationConfiguration
from paperless.parsers import ParserContext
from paperless.parsers import ParserProtocol
from paperless.parsers.remote import RemoteDocumentParser
@@ -198,21 +199,21 @@ class TestRemoteParserScore:
def test_score_returns_none_when_api_key_missing(
self,
settings: SettingsWrapper,
no_engine_settings: SettingsWrapper,
) -> None:
settings.REMOTE_OCR_ENGINE = "azureai"
settings.REMOTE_OCR_API_KEY = None
settings.REMOTE_OCR_ENDPOINT = "https://test.cognitiveservices.azure.com"
no_engine_settings.REMOTE_OCR_ENGINE = "azureai"
no_engine_settings.REMOTE_OCR_ENDPOINT = (
"https://test.cognitiveservices.azure.com"
)
result = RemoteDocumentParser.score("application/pdf", "doc.pdf")
assert result is None
def test_score_returns_none_when_endpoint_missing(
self,
settings: SettingsWrapper,
no_engine_settings: SettingsWrapper,
) -> None:
settings.REMOTE_OCR_ENGINE = "azureai"
settings.REMOTE_OCR_API_KEY = "key"
settings.REMOTE_OCR_ENDPOINT = None
no_engine_settings.REMOTE_OCR_ENGINE = "azureai"
no_engine_settings.REMOTE_OCR_API_KEY = "key"
result = RemoteDocumentParser.score("application/pdf", "doc.pdf")
assert result is None
@@ -227,6 +228,24 @@ class TestRemoteParserScore:
score = RemoteDocumentParser.score("application/pdf", "doc.pdf")
assert score is not None and score > 10
@pytest.mark.django_db
def test_score_uses_app_config_when_env_unset(
self,
settings: SettingsWrapper,
) -> None:
"""The app config alone is enough to activate the parser."""
settings.REMOTE_OCR_ENGINE = None
settings.REMOTE_OCR_API_KEY = None
settings.REMOTE_OCR_ENDPOINT = None
config = ApplicationConfiguration.objects.first()
assert config is not None
config.remote_ocr_engine = "azureai"
config.remote_ocr_api_key = "app-config-key"
config.remote_ocr_endpoint = "https://config.cognitiveservices.azure.com"
config.save()
assert RemoteDocumentParser.score("application/pdf", "doc.pdf") == 20
# ---------------------------------------------------------------------------
# Properties
@@ -1277,6 +1277,8 @@ class TestParserFileTypes:
# ---------------------------------------------------------------------------
# Remote ocr config from ApplicationConfiguration needs DB access
@pytest.mark.django_db
class TestRasterisedDocumentParserRegistry:
def test_registered_in_defaults(self) -> None:
from paperless.parsers.registry import ParserRegistry
-26
View File
@@ -15,7 +15,6 @@ from paperless.checks import audit_log_check
from paperless.checks import binaries_check
from paperless.checks import check_default_language_available
from paperless.checks import check_deprecated_db_settings
from paperless.checks import check_remote_parser_configured
from paperless.checks import check_v3_minimum_upgrade_version
from paperless.checks import debug_mode_check
from paperless.checks import paths_check
@@ -631,31 +630,6 @@ class TestV3MinimumUpgradeVersionCheck:
assert check_v3_minimum_upgrade_version(None) == []
class TestRemoteParserChecks:
def test_no_engine(self, settings: SettingsWrapper) -> None:
settings.REMOTE_OCR_ENGINE = None
msgs = check_remote_parser_configured(None)
assert len(msgs) == 0
def test_azure_no_endpoint(self, settings: SettingsWrapper) -> None:
settings.REMOTE_OCR_ENGINE = "azureai"
settings.REMOTE_OCR_API_KEY = "somekey"
settings.REMOTE_OCR_ENDPOINT = None
msgs = check_remote_parser_configured(None)
assert len(msgs) == 1
msg = msgs[0]
assert (
"Azure AI remote parser requires endpoint and API key to be configured."
in msg.msg
)
class TestTesseractChecks:
def test_default_language(self) -> None:
check_default_language_available(None)
+118
View File
@@ -468,6 +468,124 @@ class TestParserRegistryGetParserForFile:
assert result is AcceptingBuiltin
class TestParserRegistryRemoteParsers:
"""Verify the allow_remote filter in ParserRegistry.get_parser_for_file()."""
@staticmethod
def _remote_parser_cls() -> type:
class RemoteParser:
name = "remote"
version = "1.0"
author = "A"
url = "https://example.com/remote"
uses_remote_service = True
@classmethod
def supported_mime_types(cls):
return {"text/plain": ".txt"}
@classmethod
def score(cls, mime_type, filename, path=None):
return 20
return RemoteParser
def test_remote_parser_wins_when_remote_allowed(
self,
dummy_parser_cls: type,
) -> None:
"""
GIVEN: A remote parser scoring 20 and a local parser scoring 10.
WHEN: get_parser_for_file() is called with allow_remote=True.
THEN: The remote parser is returned.
"""
remote_parser_cls = self._remote_parser_cls()
registry = ParserRegistry()
registry.register_builtin(dummy_parser_cls)
registry.register_builtin(remote_parser_cls)
result = registry.get_parser_for_file(
"text/plain",
"readme.txt",
allow_remote=True,
)
assert result is remote_parser_cls
def test_remote_parser_skipped_when_remote_not_allowed(
self,
dummy_parser_cls: type,
) -> None:
"""
GIVEN: A remote parser scoring 20 and a local parser scoring 10.
WHEN: get_parser_for_file() is called with allow_remote=False.
THEN: The local parser is returned despite its lower score.
"""
registry = ParserRegistry()
registry.register_builtin(dummy_parser_cls)
registry.register_builtin(self._remote_parser_cls())
result = registry.get_parser_for_file(
"text/plain",
"readme.txt",
allow_remote=False,
)
assert result is dummy_parser_cls
def test_no_parser_when_only_remote_available_and_not_allowed(self) -> None:
"""
GIVEN: A registry whose only candidate declares uses_remote_service.
WHEN: get_parser_for_file() is called with allow_remote=False.
THEN: None is returned the remote parser is never used as a
fallback when remote processing was not requested.
"""
registry = ParserRegistry()
registry.register_builtin(self._remote_parser_cls())
result = registry.get_parser_for_file(
"text/plain",
"readme.txt",
allow_remote=False,
)
assert result is None
def test_parser_without_attribute_treated_as_local(
self,
dummy_parser_cls: type,
) -> None:
"""
GIVEN: A third-party parser predating uses_remote_service, so it does
not declare the attribute at all.
WHEN: get_parser_for_file() is called with allow_remote=False.
THEN: It is still considered, i.e. treated as fully local, rather
than raising AttributeError.
"""
assert not hasattr(dummy_parser_cls, "uses_remote_service")
registry = ParserRegistry()
registry.register_builtin(dummy_parser_cls)
result = registry.get_parser_for_file(
"text/plain",
"readme.txt",
allow_remote=False,
)
assert result is dummy_parser_cls
def test_remote_allowed_by_default(self) -> None:
"""
GIVEN: A registry containing only a remote parser.
WHEN: get_parser_for_file() is called without allow_remote.
THEN: The remote parser is returned callers that do not opt in to
the filter keep the previous behaviour.
"""
remote_parser_cls = self._remote_parser_cls()
registry = ParserRegistry()
registry.register_builtin(remote_parser_cls)
result = registry.get_parser_for_file("text/plain", "readme.txt")
assert result is remote_parser_cls
class TestDiscover:
"""Verify entrypoint discovery in ParserRegistry.discover()."""
@@ -0,0 +1,113 @@
"""Tests for RemoteOCRConfig precedence between app config and Django settings."""
from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
from django.test import override_settings
from paperless.config import RemoteOCRConfig
from paperless.models import RemoteOCRMode
if TYPE_CHECKING:
from unittest.mock import MagicMock
@pytest.fixture()
def null_app_config(mocker) -> MagicMock:
"""Mock ApplicationConfiguration with all fields None → falls back to Django settings."""
return mocker.MagicMock(
remote_ocr_engine=None,
remote_ocr_api_key=None,
remote_ocr_endpoint=None,
remote_ocr_mode=None,
)
@pytest.fixture()
def make_remote_ocr_config(mocker):
def _make(app_config, **django_settings_overrides):
mocker.patch(
"paperless.config.BaseConfig._get_config_instance",
return_value=app_config,
)
with override_settings(**django_settings_overrides):
return RemoteOCRConfig()
return _make
class TestRemoteOCRConfig:
def test_falls_back_to_settings(
self,
make_remote_ocr_config,
null_app_config,
) -> None:
cfg = make_remote_ocr_config(
null_app_config,
REMOTE_OCR_ENGINE="azureai",
REMOTE_OCR_API_KEY="env-key",
REMOTE_OCR_ENDPOINT="https://env.cognitiveservices.azure.com",
REMOTE_OCR_MODE=RemoteOCRMode.WORKFLOW_ONLY,
)
assert cfg.remote_ocr_engine == "azureai"
assert cfg.remote_ocr_api_key == "env-key"
assert cfg.remote_ocr_endpoint == "https://env.cognitiveservices.azure.com"
assert cfg.remote_ocr_mode == RemoteOCRMode.WORKFLOW_ONLY
def test_app_config_takes_precedence(
self,
make_remote_ocr_config,
mocker,
) -> None:
app_config = mocker.MagicMock(
remote_ocr_engine="azureai",
remote_ocr_api_key="db-key",
remote_ocr_endpoint="https://db.cognitiveservices.azure.com",
remote_ocr_mode=RemoteOCRMode.WORKFLOW_ONLY,
)
cfg = make_remote_ocr_config(
app_config,
REMOTE_OCR_ENGINE=None,
REMOTE_OCR_API_KEY="env-key",
REMOTE_OCR_ENDPOINT="https://env.cognitiveservices.azure.com",
REMOTE_OCR_MODE=RemoteOCRMode.ALWAYS,
)
assert cfg.remote_ocr_engine == "azureai"
assert cfg.remote_ocr_api_key == "db-key"
assert cfg.remote_ocr_endpoint == "https://db.cognitiveservices.azure.com"
assert cfg.remote_ocr_mode == RemoteOCRMode.WORKFLOW_ONLY
def test_unset_everywhere(
self,
make_remote_ocr_config,
null_app_config,
) -> None:
cfg = make_remote_ocr_config(
null_app_config,
REMOTE_OCR_ENGINE=None,
REMOTE_OCR_API_KEY=None,
REMOTE_OCR_ENDPOINT=None,
)
assert cfg.remote_ocr_engine is None
assert cfg.remote_ocr_api_key is None
assert cfg.remote_ocr_endpoint is None
class TestRemoteOCRByDefault:
def test_always_mode(self, make_remote_ocr_config, null_app_config) -> None:
cfg = make_remote_ocr_config(
null_app_config,
REMOTE_OCR_MODE=RemoteOCRMode.ALWAYS,
)
assert cfg.remote_ocr_by_default is True
def test_workflow_only_mode(self, make_remote_ocr_config, null_app_config) -> None:
cfg = make_remote_ocr_config(
null_app_config,
REMOTE_OCR_MODE=RemoteOCRMode.WORKFLOW_ONLY,
)
assert cfg.remote_ocr_by_default is False
+75
View File
@@ -253,6 +253,81 @@ class TestAPIMailAccounts(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["success"], True)
def test_mail_account_test_existing_no_global_perms(self) -> None:
"""
GIVEN:
- Existing account without an owner
- User without any mail account permissions
WHEN:
- API call is made to test the account by id
THEN:
- API returns forbidden
"""
account = MailAccountFactory(
username="admin",
password="secret",
imap_server="server.example.com",
imap_port=443,
owner=None,
)
user = User.objects.create_user(username="no_perms")
self.client.force_authenticate(user=user)
response = self.client.post(
f"{self.ENDPOINT}test/",
json.dumps(
{
"id": account.pk,
"imap_server": "server.example.com",
"imap_port": 443,
"imap_security": MailAccount.ImapSecurity.SSL,
"username": "admin",
"password": "******",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertEqual(response.content.decode(), "Insufficient permissions")
def test_mail_account_test_existing_object_perms_only(self) -> None:
"""
GIVEN:
- Existing account owned by another user
- User with an object level grant but no global change permission
WHEN:
- API call is made to test the account by id
THEN:
- API returns forbidden
"""
owner = User.objects.create_user(username="account_owner")
account = MailAccountFactory(
username="admin",
password="secret",
imap_server="server.example.com",
imap_port=443,
owner=owner,
)
user = User.objects.create_user(username="object_perms_only")
assign_perm("change_mailaccount", user, account)
self.client.force_authenticate(user=user)
response = self.client.post(
f"{self.ENDPOINT}test/",
json.dumps(
{
"id": account.pk,
"imap_server": "server.example.com",
"imap_port": 443,
"imap_security": MailAccount.ImapSecurity.SSL,
"username": "admin",
"password": "******",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_mail_account_test_existing_nonexistent_id_forbidden(self) -> None:
response = self.client.post(
f"{self.ENDPOINT}test/",
+3 -1
View File
@@ -2195,7 +2195,9 @@ class TestMailAccountTestView(APITestCase):
password="testpassword",
)
self.user.user_permissions.add(
*Permission.objects.filter(codename__in=["add_mailaccount"]),
*Permission.objects.filter(
codename__in=["add_mailaccount", "change_mailaccount"],
),
)
self.user.save()
self.client.force_authenticate(user=self.user)
+3 -1
View File
@@ -106,7 +106,9 @@ class MailAccountViewSet(PassUserMixin, ModelViewSet[MailAccount]):
except (TypeError, ValueError, MailAccount.DoesNotExist):
return HttpResponseForbidden("Insufficient permissions")
if not has_perms_owner_aware(
if not request.user.has_perms(
["paperless_mail.change_mailaccount"],
) or not has_perms_owner_aware(
request.user,
"change_mailaccount",
existing_account,