Compare commits

..
Author SHA1 Message Date
shamoon 6380933b96 Note the large qty 2026-08-10 20:56:57 -07:00
shamoon 9de12ff1c5 Backoff/retry and tweak the error bubbling 2026-08-10 20:56:57 -07:00
shamoon d24fcb92b7 Show these in tasks 2026-08-10 20:56:56 -07:00
shamoon 4df2656623 Docs 2026-08-10 20:56:56 -07:00
shamoon dd5d93e84a Frontend AI suggestion workflow stuff 2026-08-10 20:56:56 -07:00
shamoon 013b8dbc32 Codex please write backend tests thx 2026-08-10 20:31:41 -07:00
shamoon c61ea2b398 Ok wire up the apply ai suggestions action to the wf 2026-08-10 20:31:41 -07:00
shamoon 9d673645b8 Ok the ai workflow module, actually does the thing 2026-08-10 20:31:41 -07:00
shamoon 4b1ca62349 Extract get_output_language for re-use 2026-08-10 20:31:41 -07:00
shamoon 93e6d25bb7 The object getters in matching need to accept none for workflows 2026-08-10 20:31:40 -07:00
shamoon 116cced6a6 Pull the ai workflow through serializer and check for required triggers 2026-08-10 20:31:40 -07:00
shamoon 1ce7d62b66 Model and migration 2026-08-10 20:31:40 -07:00
shamoon 44a822cb10 Actually, fix the action dropdown thing 2026-08-10 20:29:33 -07:00
shamoon 6df70e62f7 Fix dynamic action fields thing 2026-08-10 11:11:32 -07:00
shamoon c478d7bb5d And docs 2026-08-10 10:57:39 -07:00
shamoon ef54e0564a Frotnend workflow stuff 2026-08-10 10:57:35 -07:00
shamoon 67628401c8 Ok! Backend stuff for the remote ocr workflow 2026-08-10 10:43:26 -07:00
shamoon 5c96b38f4e Docs 2026-08-10 10:13:24 -07:00
shamoon 21ac856e3f Ok, frontend reprocess remote option 2026-08-10 10:13:23 -07:00
shamoon 37d6b02ebc Backend stuff for remote ocr reprocess, add to bulk edit pass in from ui settings 2026-08-10 10:13:23 -07:00
shamoon 7456b52e84 Update consumer.py 2026-08-10 10:13:18 -07:00
shamoon 6a392ea099 Ok, wire up the remote_ocr_mode with allow_remote for consumer 2026-08-10 10:13:18 -07:00
shamoon 2032ad1341 Docs for remote_ocr_mode 2026-08-10 10:13:18 -07:00
shamoon cc8fee91c4 More tests for remote_ocr_mode 2026-08-10 10:13:18 -07:00
shamoon a5d46a883e Update config.component.spec.ts 2026-08-10 10:13:18 -07:00
shamoon b0e1793093 Checks for remote_ocr_mode and fix import 2026-08-10 10:13:17 -07:00
shamoon 7e466d1f71 remote_ocr_mode config setting 2026-08-10 10:13:17 -07:00
shamoon 7b69a178c0 Add to parser dev docs 2026-08-10 10:13:17 -07:00
shamoon 22cd13a8a9 uses_remote_service + allow_remote to allow opt-in / out of remote OCR 2026-08-10 10:13:17 -07:00
shamoon 72a4676be0 Update test_tesseract_parser.py 2026-08-10 10:11:14 -07:00
shamoon 6673144d23 Docs 2026-08-10 10:11:13 -07:00
shamoon 994a84cf92 Frontend stuff, with sections 2026-08-10 10:11:13 -07:00
shamoon 654ce5d8f3 Backend tests 2026-08-10 10:11:13 -07:00
shamoon 5d5e9b6db4 Backend changes and migration for remote OCR Config 2026-08-10 10:11:13 -07:00
186 changed files with 59849 additions and 89809 deletions
-1
View File
@@ -699,7 +699,6 @@ document_fuzzy_match [--ratio] [--processes N]
| --ratio | No | 85.0 | a number between 0 and 100, setting how similar a document must be for it to be reported. Higher numbers mean more similarity. | | --ratio | No | 85.0 | a number between 0 and 100, setting how similar a document must be for it to be reported. Higher numbers mean more similarity. |
| --processes | No | 1/4 of system cores | Number of processes to use for matching. Setting 1 disables multiple processes | | --processes | No | 1/4 of system cores | Number of processes to use for matching. Setting 1 disables multiple processes |
| --delete | No | False | If provided, one document of a matched pair above the ratio will be deleted. | | --delete | No | False | If provided, one document of a matched pair above the ratio will be deleted. |
| --url | No | blank | If an instance URL is provided, the output table will show URLs to each documents instead of the document ID and name. |
!!! warning !!! warning
+2 -1
View File
@@ -301,7 +301,8 @@ The following methods are supported:
- `delete` - `delete`
- No `parameters` required - No `parameters` required
- `reprocess` - `reprocess`
- No `parameters` required - Optional `parameters`: `{ "remote_ocr": true }` to send the documents to the
remote OCR engine, see [Remote OCR](usage.md#remote-ocr). Defaults to false.
- `set_permissions` - `set_permissions`
- Requires `parameters`: - Requires `parameters`:
- `"set_permissions": PERMISSIONS_OBJ` (see format [above](#permissions)) and / or - `"set_permissions": PERMISSIONS_OBJ` (see format [above](#permissions)) and / or
+16 -5
View File
@@ -948,11 +948,10 @@ for display in the web interface.
!!! note !!! note
The **remote OCR parser** (Azure AI) also honors this setting: when The **remote OCR parser** (Azure AI) always produces a searchable
no archive is requested (`never`, or `auto` with a born-digital PDF), PDF and stores it as the archive copy, regardless of this setting.
the remote engine is skipped entirely and locally-extracted text is `ARCHIVE_FILE_GENERATION=never` has no effect when the remote
used instead, avoiding an unnecessary API call and a duplicate text parser handles a document.
layer.
#### [`PAPERLESS_OCR_CLEAN=<mode>`](#PAPERLESS_OCR_CLEAN) {#PAPERLESS_OCR_CLEAN} #### [`PAPERLESS_OCR_CLEAN=<mode>`](#PAPERLESS_OCR_CLEAN) {#PAPERLESS_OCR_CLEAN}
@@ -2048,6 +2047,18 @@ password. All of these options come from their similarly-named [Django settings]
Defaults to None. Defaults to None.
#### [`PAPERLESS_REMOTE_OCR_MODE=<str>`](#PAPERLESS_REMOTE_OCR_MODE) {#PAPERLESS_REMOTE_OCR_MODE}
: Which documents are sent to the remote OCR engine.
- `always`: every document of a supported file type is sent to the remote
engine, bypassing the local OCR engine.
- `workflow_only`: documents are processed locally unless a workflow
explicitly enables remote OCR for them, letting you use the remote engine
selectively.
Defaults to "always".
## AI {#ai} ## AI {#ai}
#### [`PAPERLESS_AI_ENABLED=<bool>`](#PAPERLESS_AI_ENABLED) {#PAPERLESS_AI_ENABLED} #### [`PAPERLESS_AI_ENABLED=<bool>`](#PAPERLESS_AI_ENABLED) {#PAPERLESS_AI_ENABLED}
+14
View File
@@ -456,6 +456,20 @@ def score(
return 10 return 10
``` ```
**Remote services**
If your parser sends document content to a remote service, declare it:
```python
class MyCustomParser:
uses_remote_service = True
```
Paperless-ngx excludes such parsers when the document being consumed has not
been marked for remote processing, so users can keep remote OCR off by default
and enable it selectively with a workflow. Parsers that do not declare the
attribute are treated as fully local and are always considered.
**Archive and rendition flags** **Archive and rendition flags**
```python ```python
+4 -5
View File
@@ -187,11 +187,10 @@ PAPERLESS_ARCHIVE_FILE_GENERATION=auto
### Remote OCR parser ### Remote OCR parser
If you use the **remote OCR parser** (Azure AI), `ARCHIVE_FILE_GENERATION` is If you use the **remote OCR parser** (Azure AI), note that it always produces a
honored the same way as for the local engine: when no archive is requested searchable PDF and stores it as the archive copy. `ARCHIVE_FILE_GENERATION=never`
(`never`, or `auto` with a born-digital PDF), the remote engine is skipped has no effect for documents handled by the remote parser - the archive is produced
entirely and locally-extracted text is used instead, avoiding an unnecessary unconditionally by the remote engine.
API call and a duplicate text layer.
## Search Index (Whoosh -> Tantivy) ## Search Index (Whoosh -> Tantivy)
+52 -4
View File
@@ -576,9 +576,7 @@ The following workflow action types are available:
- Tags, correspondent, document type and storage path - Tags, correspondent, document type and storage path
- Document owner - Document owner
- View and / or edit permissions to users or groups - View and / or edit permissions to users or groups
- Custom fields, optionally with a value. If no value is set, the field is only added to the - Custom fields. Note that no value for the field will be set
document and any value it may already have is left untouched. If a value is set, it will
overwrite an existing value of that field on the document.
##### Removal {#workflow-action-removal} ##### Removal {#workflow-action-removal}
@@ -650,6 +648,48 @@ happened while it was still encrypted, that original version will likewise be mi
**Current limitation**: Passwords are stored as a simple list without descriptions. To handle **Current limitation**: Passwords are stored as a simple list without descriptions. To handle
multiple PDF types with different passwords, create separate workflows for each use case. multiple PDF types with different passwords, create separate workflows for each use case.
##### Remote OCR {#workflow-action-remote-ocr}
"Remote OCR" actions send the document to the configured remote OCR engine instead of processing it
locally. To use remote OCR selectively, set the [remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE)
to `workflow_only` then add this action to a workflow that matches only the documents you
want sent to the remote engine. See [Remote OCR](#remote-ocr) for the engine setup. The action only works with
a **Consumption Started** trigger.
The action takes no options, its presence is what enables remote OCR for a matching document.
If the remote engine is not configured, or does not support the document's file type, the document is
processed locally instead and a warning is written to the log.
##### Apply AI Suggestions {#workflow-action-apply-ai-suggestions}
"Apply AI Suggestions" actions ask the configured AI service for title and metadata suggestions,
the same as the AI suggestions shown on the document detail page, except applied automatically and in bulk.
It requires [AI features](configuration.md#ai) to be enabled. You can specify:
- Which suggestions to apply: title, tags, correspondent, document type, storage path and / or created
date. Suggestions for fields you did not select are discarded.
- Whether to create missing items. By default only tags, correspondents and document types that
already exist are assigned and any other suggestion is dropped. With this enabled, suggested items
that do not exist are created. Storage paths are never created.
- Whether to overwrite existing values. By default a field is only filled in if it is currently empty.
Note that documents almost always already have a title and created date, so if you select those you
will usually want to enable this too. Tags are an exception: suggested tags are always added and
never replace the document's existing tags.
The action works with every trigger **except Consumption Started**, because suggestions are made from
the document's text, which does not exist until after the document has been processed.
Because the query to the AI service is slow, the action is queued and runs in the background rather
than as part of the workflow run itself. The document is updated once the suggestions come back.
!!! warning
Every matching document results in a query to the AI service, which may incur costs and have privacy
implications. Queries can be slow, so a workflow matching a large number of documents can occupy the
task queue, and delay consumption of new documents, etc. Consider narrowing the trigger filters,
running in small batches and / or increasing workers.
#### Workflow placeholders #### Workflow placeholders
Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/). Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
@@ -1086,11 +1126,19 @@ Paperless-ngx supports performing OCR on documents using remote services. At the
[Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence). [Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence).
This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with
Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details. the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details. These
settings can be supplied as environment variables or via **Application Configuration**.
Additionally, when using a commercial service with this feature, consider both potential costs as well as any associated file size Additionally, when using a commercial service with this feature, consider both potential costs as well as any associated file size
or page limitations (e.g. with a free tier). or page limitations (e.g. with a free tier).
By default, every document of a supported file type is sent to the remote engine. To use it more selectively, set the
[remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE) to `workflow_only`. Documents are then processed locally
unless a [remote OCR workflow action](#workflow-action-remote-ocr) enables it for them, so you can limit the remote
engine to particular documents.
Setting the mode to `workflow_only` also allows the **Reprocess** actions to selectively use remote OCR for individual documents.
## Architecture ## Architecture
Paperless-ngx consists of the following components: Paperless-ngx consists of the following components:
+99 -491
View File
File diff suppressed because it is too large Load Diff
@@ -14,43 +14,48 @@
<a ngbNavLink>{{category}}</a> <a ngbNavLink>{{category}}</a>
<ng-template ngbNavContent> <ng-template ngbNavContent>
<div class="p-3"> <div class="p-3">
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2"> @for (section of getCategorySections(category); track section) {
@for (option of getCategoryOptions(category); track option.key) { @if (section) {
<div class="col"> <h5 class="mt-4 mb-3">{{section}}</h5>
<div class="card bg-light"> }
<div class="card-body"> <div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
<div class="card-title d-flex align-items-center"> @for (option of getCategoryOptions(category, section); track option.key) {
<h6 class="mb-0"> <div class="col">
{{option.title}} <div class="card bg-light">
</h6> <div class="card-body">
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer"> <div class="card-title d-flex align-items-center">
<i-bs name="info-circle"></i-bs> <h6 class="mb-0">
</a> {{option.title}}
@if (isSet(option.key)) { </h6>
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)"> <a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container> <i-bs name="info-circle"></i-bs>
</button> </a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
} }
</div> </div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div> </div>
</div> </div>
</div> }
} </div>
</div> }
</div> </div>
</ng-template> </ng-template>
</li> </li>
@@ -8,7 +8,11 @@ import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectModule } from '@ng-select/ng-select' import { NgSelectModule } from '@ng-select/ng-select'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs' import { of, throwError } from 'rxjs'
import { OutputTypeConfig } from 'src/app/data/paperless-config' import {
ConfigCategory,
ConfigSection,
OutputTypeConfig,
} from 'src/app/data/paperless-config'
import { ConfigService } from 'src/app/services/config.service' import { ConfigService } from 'src/app/services/config.service'
import { SettingsService } from 'src/app/services/settings.service' import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service' import { ToastService } from 'src/app/services/toast.service'
@@ -158,4 +162,24 @@ describe('ConfigComponent', () => {
component.resetOption('barcodes_enabled') component.resetOption('barcodes_enabled')
expect(component.configForm.get('barcodes_enabled').value).toBeNull() expect(component.configForm.get('barcodes_enabled').value).toBeNull()
}) })
it('should group options into sections within a category, or not', () => {
const sections = component.getCategorySections(ConfigCategory.OCR)
expect(sections).toEqual([null, ConfigSection.RemoteOCR])
expect(
component
.getCategoryOptions(ConfigCategory.OCR)
.map((option) => option.key)
).toContain('output_type')
expect(
component
.getCategoryOptions(ConfigCategory.OCR, ConfigSection.RemoteOCR)
.map((option) => option.key)
).toEqual([
'remote_ocr_engine',
'remote_ocr_api_key',
'remote_ocr_endpoint',
'remote_ocr_mode',
])
})
}) })
@@ -74,8 +74,20 @@ export class ConfigComponent
return Object.values(ConfigCategory) return Object.values(ConfigCategory)
} }
getCategoryOptions(category: string): ConfigOption[] { getCategorySections(category: string): string[] {
return PaperlessConfigOptions.filter((o) => o.category === category) return [
...new Set(
PaperlessConfigOptions.filter((o) => o.category === category).map(
(o) => o.section ?? null // null means no section
)
),
]
}
getCategoryOptions(category: string, section: string = null): ConfigOption[] {
return PaperlessConfigOptions.filter(
(o) => o.category === category && (o.section ?? null) === section
)
} }
initialConfig: PaperlessConfig initialConfig: PaperlessConfig
@@ -111,7 +111,7 @@
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name" routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave" [disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim"> popoverClass="popover-slim">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div> <i-bs class="me-2" name="funnel"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
@if (showSidebarCounts && !slimSidebarEnabled) { @if (showSidebarCounts && !slimSidebarEnabled) {
<span class="badge bg-info text-dark ms-2 d-inline">{{ savedViewService.getDocumentCount(view) }}</span> <span class="badge bg-info text-dark ms-2 d-inline">{{ savedViewService.getDocumentCount(view) }}</span>
} }
@@ -0,0 +1,28 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
</div>
<div class="modal-body">
@if (messageBold) {
<p class="text-break"><b>{{messageBold}}</b></p>
}
@if (message) {
<p class="mb-0 text-break" [innerHTML]="message"></p>
}
@if (showRemoteOcr) {
<div class="form-check mt-3">
<input class="form-check-input" type="checkbox" id="reprocessRemoteOcr" [(ngModel)]="remoteOcr" />
<label class="form-check-label" for="reprocessRemoteOcr" i18n>Use remote OCR</label>
<div class="form-text" i18n>Sends the document to the configured remote OCR service, which may incur costs.</div>
</div>
}
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,72 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { RemoteOCRModeConfig } from 'src/app/data/paperless-config'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { SettingsService } from 'src/app/services/settings.service'
import { ReprocessConfirmDialogComponent } from './reprocess-confirm-dialog.component'
describe('ReprocessConfirmDialogComponent', () => {
let component: ReprocessConfirmDialogComponent
let fixture: ComponentFixture<ReprocessConfirmDialogComponent>
let settingsService: SettingsService
const createComponent = (configured: boolean, mode: string) => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, configured)
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_MODE, mode)
fixture = TestBed.createComponent(ReprocessConfirmDialogComponent)
component = fixture.componentInstance
fixture.detectChanges()
}
beforeEach(async () => {
TestBed.configureTestingModule({
providers: [
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
imports: [ReprocessConfirmDialogComponent],
}).compileComponents()
settingsService = TestBed.inject(SettingsService)
})
it('should not request remote OCR by default', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.remoteOcr).toBeFalsy()
})
it('should not offer remote OCR when no engine is configured', () => {
createComponent(false, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should not offer remote OCR when it already handles every document', () => {
createComponent(true, RemoteOCRModeConfig.ALWAYS)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should offer remote OCR when configured and selective', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeTruthy()
const checkbox = fixture.nativeElement.querySelector('#reprocessRemoteOcr')
expect(checkbox).not.toBeNull()
checkbox.click()
fixture.detectChanges()
expect(component.remoteOcr).toBeTruthy()
})
})
@@ -0,0 +1,20 @@
import { Component, inject } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { SettingsService } from 'src/app/services/settings.service'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-reprocess-confirm-dialog',
templateUrl: './reprocess-confirm-dialog.component.html',
imports: [FormsModule],
})
export class ReprocessConfirmDialogComponent extends ConfirmDialogComponent {
private settings = inject(SettingsService)
remoteOcr: boolean = false
public get showRemoteOcr(): boolean {
// Hidden when it is not configured, or when it already handles every document anyway.
return this.settings.remoteOCRIsSelectable
}
}
@@ -455,6 +455,52 @@
</div> </div>
</div> </div>
} }
@case (WorkflowActionType.RemoteOcr) {
<div class="row">
<div class="col">
<p class="text-muted small" i18n>The document will be sent to the configured remote OCR service. May incur costs.</p>
</div>
</div>
}
@case (WorkflowActionType.ApplyAiSuggestions) {
<div class="row">
<div class="col">
<p class="text-muted small" i18n>The document will be sent to the configured AI service for suggestions. Consider costs and privacy.</p>
<pngx-input-select
i18n-title
title="Apply suggestions for"
[items]="aiSuggestionFieldOptions"
[multiple]="true"
formControlName="ai_suggestion_fields"
[error]="error?.actions?.[i]?.ai_suggestion_fields"
hint="Suggestions for fields that are not selected are discarded."
i18n-hint
></pngx-input-select>
</div>
</div>
<div class="row">
<div class="col-md-6">
<pngx-input-switch
[horizontal]="true"
i18n-title
title="Create missing items"
formControlName="ai_create_missing"
hint="Create suggested tags, correspondents and document types that do not exist yet."
i18n-hint
></pngx-input-switch>
</div>
<div class="col-md-6">
<pngx-input-switch
[horizontal]="true"
i18n-title
title="Overwrite existing values"
formControlName="ai_overwrite_existing"
hint="Apply suggestions even if the document already has a value. Tags are always added, never replaced."
i18n-hint
></pngx-input-switch>
</div>
</div>
}
} }
</div> </div>
</ng-template> </ng-template>
@@ -22,6 +22,7 @@ import {
} from 'src/app/data/matching-model' } from 'src/app/data/matching-model'
import { Workflow } from 'src/app/data/workflow' import { Workflow } from 'src/app/data/workflow'
import { import {
AISuggestionField,
WorkflowAction, WorkflowAction,
WorkflowActionType, WorkflowActionType,
} from 'src/app/data/workflow-action' } from 'src/app/data/workflow-action'
@@ -29,6 +30,7 @@ import {
DocumentSource, DocumentSource,
WorkflowTriggerType, WorkflowTriggerType,
} from 'src/app/data/workflow-trigger' } from 'src/app/data/workflow-trigger'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive' import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive' import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CorrespondentService } from 'src/app/services/rest/correspondent.service' import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
@@ -48,6 +50,7 @@ import { TagsComponent } from '../../input/tags/tags.component'
import { TextComponent } from '../../input/text/text.component' import { TextComponent } from '../../input/text/text.component'
import { EditDialogMode } from '../edit-dialog.component' import { EditDialogMode } from '../edit-dialog.component'
import { import {
AI_SUGGESTION_FIELD_OPTIONS,
DOCUMENT_SOURCE_OPTIONS, DOCUMENT_SOURCE_OPTIONS,
SCHEDULE_DATE_FIELD_OPTIONS, SCHEDULE_DATE_FIELD_OPTIONS,
TriggerFilterType, TriggerFilterType,
@@ -224,7 +227,12 @@ describe('WorkflowEditDialogComponent', () => {
).toEqual('Document Added') ).toEqual('Document Added')
expect(component.getTriggerTypeOptionName(null)).toEqual('') expect(component.getTriggerTypeOptionName(null)).toEqual('')
expect(component.sourceOptions).toEqual(DOCUMENT_SOURCE_OPTIONS) expect(component.sourceOptions).toEqual(DOCUMENT_SOURCE_OPTIONS)
expect(component.actionTypeOptions).toEqual(WORKFLOW_ACTION_OPTIONS) // Remote OCR is absent until the workflow has a consumption trigger
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.RemoteOcr
)
)
expect( expect(
component.getActionTypeOptionName(WorkflowActionType.Assignment) component.getActionTypeOptionName(WorkflowActionType.Assignment)
).toEqual('Assignment') ).toEqual('Assignment')
@@ -233,14 +241,231 @@ describe('WorkflowEditDialogComponent', () => {
SCHEDULE_DATE_FIELD_OPTIONS SCHEDULE_DATE_FIELD_OPTIONS
) )
// Email disabled // Email, remote OCR and AI all disabled
jest.spyOn(settingsService, 'get').mockReturnValue(false) jest.spyOn(settingsService, 'get').mockReturnValue(false)
component.ngOnInit() component.ngOnInit()
expect(component.actionTypeOptions).toEqual( expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter((a) => a.id !== WorkflowActionType.Email) WORKFLOW_ACTION_OPTIONS.filter(
(a) =>
a.id !== WorkflowActionType.Email &&
a.id !== WorkflowActionType.RemoteOcr &&
a.id !== WorkflowActionType.ApplyAiSuggestions
)
) )
}) })
it('should offer remote OCR only for consumption workflows', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// A consumption trigger makes the action reachable
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Any other trigger type runs after the document has been parsed
component.object = {
name: 'Workflow 2',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should offer remote OCR on a trigger added to a new workflow', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
component.ngOnInit()
// Nothing for the action to apply to yet
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
// addTrigger creates the form field with emitEvent false, so the options
// have to be computed on read rather than cached from valueChanges
component.addTrigger()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Switching that trigger to a type that runs after parsing removes it
component.triggerFields
.at(0)
.get('type')
.setValue(WorkflowTriggerType.DocumentAdded)
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should keep remote OCR listed when an action already uses it', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// Otherwise changing the trigger would silently blank the selection
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [{ type: WorkflowActionType.RemoteOcr }],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
})
it('should not offer remote OCR when no engine is configured', () => {
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) => key !== SETTINGS_KEYS.REMOTE_OCR_CONFIGURED)
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should offer apply AI suggestions unless every trigger is consumption', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// Consumption runs before the document has been parsed, so there would be
// no content to make suggestions from
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.ApplyAiSuggestions
)
// A second, usable trigger is enough
component.object = {
name: 'Workflow 2',
order: 0,
enabled: true,
triggers: [
{ type: WorkflowTriggerType.Consumption },
{ type: WorkflowTriggerType.DocumentAdded },
],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.ApplyAiSuggestions
)
})
it('should keep apply AI suggestions listed when an action already uses it', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// Otherwise changing the trigger would silently blank the selection
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [{ type: WorkflowActionType.ApplyAiSuggestions }],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.ApplyAiSuggestions
)
})
it('should not offer apply AI suggestions when AI is disabled', () => {
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) => key !== SETTINGS_KEYS.AI_ENABLED)
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.ApplyAiSuggestions
)
})
it('should create form fields for apply AI suggestions options', () => {
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [
{
type: WorkflowActionType.ApplyAiSuggestions,
ai_suggestion_fields: [
AISuggestionField.Title,
AISuggestionField.Tags,
],
ai_create_missing: true,
ai_overwrite_existing: true,
},
],
} as Workflow
component.ngOnInit()
const action = component.actionFields.at(0)
expect(action.get('ai_suggestion_fields').value).toEqual([
AISuggestionField.Title,
AISuggestionField.Tags,
])
expect(action.get('ai_create_missing').value).toBeTruthy()
expect(action.get('ai_overwrite_existing').value).toBeTruthy()
expect(component.aiSuggestionFieldOptions).toEqual(
AI_SUGGESTION_FIELD_OPTIONS
)
})
it('should default apply AI suggestions options on a new action', () => {
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [],
} as Workflow
component.addAction()
const action = component.actionFields.at(component.actionFields.length - 1)
expect(action.get('ai_suggestion_fields').value).toEqual([])
expect(action.get('ai_create_missing').value).toBeFalsy()
expect(action.get('ai_overwrite_existing').value).toBeFalsy()
})
it('should support add and remove triggers and actions', () => { it('should support add and remove triggers and actions', () => {
component.object = workflow component.object = workflow
component.addTrigger() component.addTrigger()
@@ -30,6 +30,7 @@ import { StoragePath } from 'src/app/data/storage-path'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings' import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { Workflow } from 'src/app/data/workflow' import { Workflow } from 'src/app/data/workflow'
import { import {
AISuggestionField,
WorkflowAction, WorkflowAction,
WorkflowActionType, WorkflowActionType,
} from 'src/app/data/workflow-action' } from 'src/app/data/workflow-action'
@@ -148,6 +149,41 @@ export const WORKFLOW_ACTION_OPTIONS = [
id: WorkflowActionType.MoveToTrash, id: WorkflowActionType.MoveToTrash,
name: $localize`Move to trash`, name: $localize`Move to trash`,
}, },
{
id: WorkflowActionType.RemoteOcr,
name: $localize`Remote OCR`,
},
{
id: WorkflowActionType.ApplyAiSuggestions,
name: $localize`Apply AI suggestions`,
},
]
export const AI_SUGGESTION_FIELD_OPTIONS = [
{
id: AISuggestionField.Title,
name: $localize`Title`,
},
{
id: AISuggestionField.Tags,
name: $localize`Tags`,
},
{
id: AISuggestionField.Correspondent,
name: $localize`Correspondent`,
},
{
id: AISuggestionField.DocumentType,
name: $localize`Document type`,
},
{
id: AISuggestionField.StoragePath,
name: $localize`Storage path`,
},
{
id: AISuggestionField.Created,
name: $localize`Created date`,
},
] ]
export enum TriggerFilterType { export enum TriggerFilterType {
@@ -504,8 +540,6 @@ export class WorkflowEditDialogComponent
expandedItem: number = null expandedItem: number = null
readonly allowedActionTypes = signal([])
private readonly triggerFilterOptionsMap = new WeakMap< private readonly triggerFilterOptionsMap = new WeakMap<
FormArray, FormArray,
TriggerFilterOption[] TriggerFilterOption[]
@@ -548,13 +582,58 @@ export class WorkflowEditDialogComponent
this.checkRemovalActionFields.bind(this) this.checkRemovalActionFields.bind(this)
) )
this.checkRemovalActionFields(this.objectForm.value) this.checkRemovalActionFields(this.objectForm.value)
this.allowedActionTypes.set( }
this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)
? WORKFLOW_ACTION_OPTIONS private allowedActionTypes: typeof WORKFLOW_ACTION_OPTIONS = null
: WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.Email private getAllowedActionTypes() {
) let allowed = WORKFLOW_ACTION_OPTIONS
)
if (!this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.Email)
}
// Remote OCR is decided before the document is parsed, so it is only
// offered for workflows that run at consumption.
const formWorkflow: Workflow = this.objectForm?.value
const remoteOcrUsable =
this.settingsService.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
(formWorkflow?.triggers?.some(
(trigger) => trigger.type === WorkflowTriggerType.Consumption
) ||
formWorkflow?.actions?.some(
(action) => action.type === WorkflowActionType.RemoteOcr
))
if (!remoteOcrUsable) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
}
// Only available after consumption. Unlike remote OCR this is hidden only
// once every trigger is consumption, so it stays offered on a workflow
// that has no triggers yet.
const aiSuggestionsUsable =
this.settingsService.get(SETTINGS_KEYS.AI_ENABLED) &&
(!formWorkflow?.triggers?.length ||
formWorkflow.triggers.some(
(trigger) => trigger.type !== WorkflowTriggerType.Consumption
) ||
formWorkflow.actions?.some(
(action) => action.type === WorkflowActionType.ApplyAiSuggestions
))
if (!aiSuggestionsUsable) {
allowed = allowed.filter(
(a) => a.id !== WorkflowActionType.ApplyAiSuggestions
)
}
if (
this.allowedActionTypes?.length === allowed.length &&
this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
) {
return this.allowedActionTypes
}
this.allowedActionTypes = allowed
return allowed
} }
private checkRemovalActionFields(formWorkflow: Workflow) { private checkRemovalActionFields(formWorkflow: Workflow) {
@@ -1198,6 +1277,11 @@ export class WorkflowEditDialogComponent
passwords: new FormControl( passwords: new FormControl(
this.formatPasswords(action.passwords ?? []) this.formatPasswords(action.passwords ?? [])
), ),
ai_suggestion_fields: new FormControl(
action.ai_suggestion_fields ?? []
),
ai_create_missing: new FormControl(!!action.ai_create_missing),
ai_overwrite_existing: new FormControl(!!action.ai_overwrite_existing),
}), }),
{ emitEvent } { emitEvent }
) )
@@ -1279,13 +1363,18 @@ export class WorkflowEditDialogComponent
get actionTypeOptions() { get actionTypeOptions() {
this.settingsService.trackChanges() this.settingsService.trackChanges()
return this.allowedActionTypes() // Computed on read rather than cached
return this.getAllowedActionTypes()
} }
getActionTypeOptionName(type: WorkflowActionType): string { getActionTypeOptionName(type: WorkflowActionType): string {
return this.actionTypeOptions.find((t) => t.id === type)?.name ?? '' return this.actionTypeOptions.find((t) => t.id === type)?.name ?? ''
} }
get aiSuggestionFieldOptions() {
return AI_SUGGESTION_FIELD_OPTIONS
}
addAction() { addAction() {
if (!this.object) { if (!this.object) {
this.object = Object.assign({}, this.objectForm.value) this.object = Object.assign({}, this.objectForm.value)
@@ -1339,6 +1428,9 @@ export class WorkflowEditDialogComponent
include_document: false, include_document: false,
}, },
passwords: [], passwords: [],
ai_suggestion_fields: [],
ai_create_missing: false,
ai_overwrite_existing: false,
} }
this.object.actions.push(action) this.object.actions.push(action)
this.createActionField(action) this.createActionField(action)
@@ -52,10 +52,10 @@ describe('CustomFieldsValuesComponent', () => {
}) })
it('should set selectedFields and map values correctly', () => { it('should set selectedFields and map values correctly', () => {
component.value = { 1: 'value1', 3: 0, 4: false } component.value = { 1: 'value1' }
component.selectedFields = [1, 2, 3, 4] component.selectedFields = [1, 2]
expect(component.selectedFields).toEqual([1, 2, 3, 4]) expect(component.selectedFields).toEqual([1, 2])
expect(component.value).toEqual({ 1: 'value1', 2: null, 3: 0, 4: false }) expect(component.value).toEqual({ 1: 'value1', 2: null })
}) })
it('should return the correct custom field by id', () => { it('should return the correct custom field by id', () => {
@@ -77,7 +77,7 @@ export class CustomFieldsValuesComponent extends AbstractInputComponent<Object>
this._selectedFields = newFields this._selectedFields = newFields
// map the selected fields to an object with field_id as key and value as value // map the selected fields to an object with field_id as key and value as value
this.value = newFields.reduce((acc, fieldId) => { this.value = newFields.reduce((acc, fieldId) => {
acc[fieldId] = this.value?.[fieldId] ?? null acc[fieldId] = this.value?.[fieldId] || null
return acc return acc
}, {}) }, {})
this.onChange(this.value) this.onChange(this.value)
@@ -36,16 +36,7 @@
(focus)="clearLastSearchTerm()" (focus)="clearLastSearchTerm()"
(clear)="clearLastSearchTerm()" (clear)="clearLastSearchTerm()"
(blur)="onBlur()"> (blur)="onBlur()">
<ng-template ng-label-tmp let-item="item">
@if (iconField && item[iconField]) {
<i-bs class="me-2" [name]="item[iconField]"></i-bs>
}
<span [title]="item[bindLabel]">{{item[bindLabel]}}</span>
</ng-template>
<ng-template ng-option-tmp let-item="item"> <ng-template ng-option-tmp let-item="item">
@if (iconField && item[iconField]) {
<i-bs class="me-2" [name]="item[iconField]"></i-bs>
}
<span [title]="item[bindLabel]">{{item[bindLabel]}}</span> <span [title]="item[bindLabel]">{{item[bindLabel]}}</span>
</ng-template> </ng-template>
</ng-select> </ng-select>
@@ -35,7 +35,7 @@ import { AbstractInputComponent } from '../abstract-input'
NgxBootstrapIconsModule, NgxBootstrapIconsModule,
], ],
}) })
export class SelectComponent extends AbstractInputComponent<number | string> { export class SelectComponent extends AbstractInputComponent<number> {
constructor() { constructor() {
super() super()
this.addItemRef = this.addItem.bind(this) this.addItemRef = this.addItem.bind(this)
@@ -100,9 +100,6 @@ export class SelectComponent extends AbstractInputComponent<number | string> {
@Input() @Input()
bindLabel: string = 'name' bindLabel: string = 'name'
@Input()
iconField: string
public searchFn = (term: string, item: any): boolean => public searchFn = (term: string, item: any): boolean =>
matchesSearchText(item?.[this.bindLabel], term) matchesSearchText(item?.[this.bindLabel], term)
@@ -1,7 +1,6 @@
<pngx-widget-frame <pngx-widget-frame
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }"
[title]="savedView.name" [title]="savedView.name"
[titleIcon]="savedView.icon || 'funnel'"
[loading]="false" [loading]="false"
[draggable]="savedView" [draggable]="savedView"
> >
@@ -8,12 +8,7 @@
<i-bs name="grip-vertical"></i-bs> <i-bs name="grip-vertical"></i-bs>
</div> </div>
} }
<h6 class="card-title mb-0"> <h6 class="card-title mb-0">{{title()}}</h6>
@if (titleIcon()) {
<i-bs class="me-2" [name]="titleIcon()"></i-bs>
}
{{title()}}
</h6>
<ng-content select="[title-badge]"></ng-content> <ng-content select="[title-badge]"></ng-content>
@if (badge() !== null && badge() !== undefined) { @if (badge() !== null && badge() !== undefined) {
<span class="badge bg-info text-dark ms-2">{{badge()}}</span> <span class="badge bg-info text-dark ms-2">{{badge()}}</span>
@@ -16,8 +16,6 @@ export class WidgetFrameComponent implements AfterViewInit {
title = input<string>() title = input<string>()
titleIcon = input<string>()
draggable = input<any>() draggable = input<any>()
cardless = input(false) cardless = input(false)
@@ -963,12 +963,24 @@ describe('DocumentDetailComponent', () => {
component.reprocess() component.reprocess()
const modalCloseSpy = jest.spyOn(openModal, 'close') const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.confirmClicked.next() openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }) expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, false)
expect(modalSpy).toHaveBeenCalled() expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled() expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled() expect(modalCloseSpy).toHaveBeenCalled()
}) })
it('should pass remote OCR choice when reprocessing', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
reprocessSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component.reprocess()
openModal.componentInstance.remoteOcr = true
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, true)
})
it('should show error if redo ocr call fails', () => { it('should show error if redo ocr call fails', () => {
initNormally() initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments') const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
@@ -97,6 +97,7 @@ import { ISODateAdapter } from 'src/app/utils/ngb-iso-date-adapter'
import * as UTIF from 'utif' import * as UTIF from 'utif'
import { DocumentDetailFieldID } from '../admin/settings/settings.component' import { DocumentDetailFieldID } from '../admin/settings/settings.component'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component' import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component' import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component' import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component' import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
@@ -1402,7 +1403,7 @@ export class DocumentDetailComponent
} }
reprocess() { reprocess() {
let modal = this.modalService.open(ConfirmDialogComponent, { let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static', backdrop: 'static',
}) })
modal.componentInstance.title = $localize`Reprocess confirm` modal.componentInstance.title = $localize`Reprocess confirm`
@@ -1413,7 +1414,10 @@ export class DocumentDetailComponent
modal.componentInstance.confirmClicked.subscribe(() => { modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled = false modal.componentInstance.buttonsEnabled = false
this.documentsService this.documentsService
.reprocessDocuments({ documents: [this.document().id] }) .reprocessDocuments(
{ documents: [this.document().id] },
modal.componentInstance.remoteOcr
)
.subscribe({ .subscribe({
next: () => { next: () => {
this.toastService.showInfo( this.toastService.showInfo(
@@ -1122,6 +1122,7 @@ describe('BulkEditorComponent', () => {
req.flush(true) req.flush(true)
expect(req.request.body).toEqual({ expect(req.request.body).toEqual({
documents: [3, 4], documents: [3, 4],
remote_ocr: false,
}) })
httpTestingController.match( httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -51,6 +51,7 @@ import { ToastService } from 'src/app/services/toast.service'
import { flattenTags } from 'src/app/utils/flatten-tags' import { flattenTags } from 'src/app/utils/flatten-tags'
import { queryParamsFromFilterRules } from 'src/app/utils/query-params' import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component' import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component' import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component' import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component' import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
@@ -909,7 +910,7 @@ export class BulkEditorComponent
} }
reprocessSelected() { reprocessSelected() {
let modal = this.modalService.open(ConfirmDialogComponent, { let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static', backdrop: 'static',
}) })
modal.componentInstance.title = $localize`Reprocess confirm` modal.componentInstance.title = $localize`Reprocess confirm`
@@ -923,7 +924,10 @@ export class BulkEditorComponent
modal.componentInstance.buttonsEnabled = false modal.componentInstance.buttonsEnabled = false
this.executeDocumentAction( this.executeDocumentAction(
modal, modal,
this.documentService.reprocessDocuments(this.getSelectionQuery()) this.documentService.reprocessDocuments(
this.getSelectionQuery(),
modal.componentInstance.remoteOcr
)
) )
}) })
} }
@@ -97,9 +97,7 @@
<div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu> <div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu>
@if (!list.activeSavedViewId) { @if (!list.activeSavedViewId) {
@for (view of savedViewService.allViews; track view) { @for (view of savedViewService.allViews; track view) {
<button ngbDropdownItem (click)="loadViewConfig(view.id)"> <button ngbDropdownItem (click)="loadViewConfig(view.id)">{{view.name}}</button>
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
</button>
} }
@if (savedViewService.allViews.length > 0) { @if (savedViewService.allViews.length > 0) {
<div class="dropdown-divider"></div> <div class="dropdown-divider"></div>
@@ -457,7 +457,6 @@ export class DocumentListComponent
modal.componentInstance.buttonsEnabled.set(false) modal.componentInstance.buttonsEnabled.set(false)
let savedView: SavedView = { let savedView: SavedView = {
name: formValue.name, name: formValue.name,
icon: formValue.icon,
filter_rules: this.list.filterRules, filter_rules: this.list.filterRules,
sort_reverse: this.list.sortReverse, sort_reverse: this.list.sortReverse,
sort_field: this.list.sortField, sort_field: this.list.sortField,
@@ -6,14 +6,6 @@
</div> </div>
<div class="modal-body"> <div class="modal-body">
<pngx-input-text i18n-title title="Name" formControlName="name" [error]="error()?.name" autocomplete="off"></pngx-input-text> <pngx-input-text i18n-title title="Name" formControlName="name" [error]="error()?.name" autocomplete="off"></pngx-input-text>
<pngx-input-select
i18n-title
title="Icon"
formControlName="icon"
[items]="savedViewIcons"
iconField="icon"
[error]="error()?.icon">
</pngx-input-select>
<pngx-input-check i18n-title title="Show in sidebar" formControlName="showInSideBar"></pngx-input-check> <pngx-input-check i18n-title title="Show in sidebar" formControlName="showInSideBar"></pngx-input-check>
<pngx-input-check i18n-title title="Show on dashboard" formControlName="showOnDashboard"></pngx-input-check> <pngx-input-check i18n-title title="Show on dashboard" formControlName="showOnDashboard"></pngx-input-check>
<pngx-permissions-form accordion="true" formControlName="permissions_form"></pngx-permissions-form> <pngx-permissions-form accordion="true" formControlName="permissions_form"></pngx-permissions-form>
@@ -9,7 +9,6 @@ import { CheckComponent } from '../../common/input/check/check.component'
import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component' import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component'
import { PermissionsGroupComponent } from '../../common/input/permissions/permissions-group/permissions-group.component' import { PermissionsGroupComponent } from '../../common/input/permissions/permissions-group/permissions-group.component'
import { PermissionsUserComponent } from '../../common/input/permissions/permissions-user/permissions-user.component' import { PermissionsUserComponent } from '../../common/input/permissions/permissions-user/permissions-user.component'
import { SelectComponent } from '../../common/input/select/select.component'
import { TextComponent } from '../../common/input/text/text.component' import { TextComponent } from '../../common/input/text/text.component'
import { SaveViewConfigDialogComponent } from './save-view-config-dialog.component' import { SaveViewConfigDialogComponent } from './save-view-config-dialog.component'
@@ -41,7 +40,6 @@ describe('SaveViewConfigDialogComponent', () => {
ReactiveFormsModule, ReactiveFormsModule,
SaveViewConfigDialogComponent, SaveViewConfigDialogComponent,
TextComponent, TextComponent,
SelectComponent,
CheckComponent, CheckComponent,
PermissionsFormComponent, PermissionsFormComponent,
PermissionsUserComponent, PermissionsUserComponent,
@@ -65,7 +63,6 @@ describe('SaveViewConfigDialogComponent', () => {
expect(component.defaultName()).toEqual(name) expect(component.defaultName()).toEqual(name)
expect(result).toEqual({ expect(result).toEqual({
name, name,
icon: 'funnel',
showInSideBar: false, showInSideBar: false,
showOnDashboard: false, showOnDashboard: false,
}) })
@@ -97,7 +94,6 @@ describe('SaveViewConfigDialogComponent', () => {
component.save() component.save()
expect(result).toEqual({ expect(result).toEqual({
name, name,
icon: 'funnel',
showInSideBar: true, showInSideBar: true,
showOnDashboard: true, showOnDashboard: true,
}) })
@@ -117,7 +113,6 @@ describe('SaveViewConfigDialogComponent', () => {
component.save() component.save()
expect(result).toEqual({ expect(result).toEqual({
name: '', name: '',
icon: 'funnel',
showInSideBar: false, showInSideBar: false,
showOnDashboard: false, showOnDashboard: false,
permissions_form: permissions, permissions_form: permissions,
@@ -13,14 +13,9 @@ import {
ReactiveFormsModule, ReactiveFormsModule,
} from '@angular/forms' } from '@angular/forms'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap' import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import {
DEFAULT_SAVED_VIEW_ICON,
SAVED_VIEW_ICONS,
} from 'src/app/data/saved-view-icons'
import { User } from 'src/app/data/user' import { User } from 'src/app/data/user'
import { CheckComponent } from '../../common/input/check/check.component' import { CheckComponent } from '../../common/input/check/check.component'
import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component' import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component'
import { SelectComponent } from '../../common/input/select/select.component'
import { TextComponent } from '../../common/input/text/text.component' import { TextComponent } from '../../common/input/text/text.component'
@Component({ @Component({
@@ -29,7 +24,6 @@ import { TextComponent } from '../../common/input/text/text.component'
styleUrls: ['./save-view-config-dialog.component.scss'], styleUrls: ['./save-view-config-dialog.component.scss'],
imports: [ imports: [
CheckComponent, CheckComponent,
SelectComponent,
TextComponent, TextComponent,
PermissionsFormComponent, PermissionsFormComponent,
FormsModule, FormsModule,
@@ -47,7 +41,6 @@ export class SaveViewConfigDialogComponent implements OnInit {
public saveClicked = new EventEmitter() public saveClicked = new EventEmitter()
users: User[] users: User[]
readonly savedViewIcons = SAVED_VIEW_ICONS
setDefaultName(value: string) { setDefaultName(value: string) {
this.defaultName.set(value) this.defaultName.set(value)
@@ -56,7 +49,6 @@ export class SaveViewConfigDialogComponent implements OnInit {
saveViewConfigForm = new FormGroup({ saveViewConfigForm = new FormGroup({
name: new FormControl(''), name: new FormControl(''),
icon: new FormControl(DEFAULT_SAVED_VIEW_ICON),
showInSideBar: new FormControl(false), showInSideBar: new FormControl(false),
showOnDashboard: new FormControl(false), showOnDashboard: new FormControl(false),
permissions_form: new FormControl(null), permissions_form: new FormControl(null),
@@ -73,7 +65,6 @@ export class SaveViewConfigDialogComponent implements OnInit {
const formValue = this.saveViewConfigForm.value const formValue = this.saveViewConfigForm.value
const saveViewConfig = { const saveViewConfig = {
name: formValue.name, name: formValue.name,
icon: formValue.icon,
showInSideBar: formValue.showInSideBar, showInSideBar: formValue.showInSideBar,
showOnDashboard: formValue.showOnDashboard, showOnDashboard: formValue.showOnDashboard,
} }
@@ -7,24 +7,15 @@
</pngx-page-header> </pngx-page-header>
<form [formGroup]="savedViewsForm" (ngSubmit)="save()"> <form [formGroup]="savedViewsForm" (ngSubmit)="save()">
<ul class="list-group mb-3" formGroupName="savedViews"> <ul class="list-group mb-3" formGroupName="savedViews">
@for (view of pagedSavedViews(); track view) { @for (view of savedViews(); track view) {
<li class="list-group-item py-3"> <li class="list-group-item py-3">
<div [formGroupName]="view.id"> <div [formGroupName]="view.id">
<div class="row"> <div class="row">
<div class="col-md"> <div class="col">
<pngx-input-text title="Name" formControlName="name"></pngx-input-text> <pngx-input-text title="Name" formControlName="name"></pngx-input-text>
</div> </div>
<div class="col-md">
<pngx-input-select
i18n-title
title="Icon"
formControlName="icon"
[items]="savedViewIcons"
iconField="icon">
</pngx-input-select>
</div>
@if (canSaveSettings) { @if (canSaveSettings) {
<div class="col-md"> <div class="col">
<div class="form-check form-switch mt-3"> <div class="form-check form-switch mt-3">
<input type="checkbox" class="form-check-input" id="show_on_dashboard_{{view.id}}" formControlName="show_on_dashboard"> <input type="checkbox" class="form-check-input" id="show_on_dashboard_{{view.id}}" formControlName="show_on_dashboard">
<label class="form-check-label" for="show_on_dashboard_{{view.id}}" i18n>Show on dashboard</label> <label class="form-check-label" for="show_on_dashboard_{{view.id}}" i18n>Show on dashboard</label>
@@ -90,11 +81,6 @@
} }
</ul> </ul>
<div class="d-flex align-items-center mb-3"> <button type="button" (click)="reset()" class="btn btn-outline-secondary mb-2" [disabled]="(isDirty$ | async) === false" i18n>Cancel</button>
<button type="button" (click)="reset()" class="btn btn-outline-secondary mb-2" [disabled]="(isDirty$ | async) === false" i18n>Cancel</button> <button type="submit" class="btn btn-primary ms-2 mb-2" [disabled]="(isDirty$ | async) === false" i18n>Save</button>
<button type="submit" class="btn btn-primary ms-2 mb-2" [disabled]="(isDirty$ | async) === false" i18n>Save</button>
@if (savedViews()?.length > pageSize) {
<ngb-pagination class="ms-auto" [pageSize]="pageSize" [collectionSize]="savedViews().length" [page]="page()" [maxSize]="5" (pageChange)="page.set($event)" size="sm" aria-label="Pagination"></ngb-pagination>
}
</div>
</form> </form>
@@ -4,7 +4,6 @@ import { provideHttpClientTesting } from '@angular/common/http/testing'
import { signal } from '@angular/core' import { signal } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing' import { ComponentFixture, TestBed } from '@angular/core/testing'
import { FormsModule, ReactiveFormsModule } from '@angular/forms' import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { By } from '@angular/platform-browser'
import { NgbModal, NgbModule } from '@ng-bootstrap/ng-bootstrap' import { NgbModal, NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { Subject, of, throwError } from 'rxjs' import { Subject, of, throwError } from 'rxjs'
@@ -26,20 +25,8 @@ import { PageHeaderComponent } from '../../common/page-header/page-header.compon
import { SavedViewsComponent } from './saved-views.component' import { SavedViewsComponent } from './saved-views.component'
const savedViews = [ const savedViews = [
{ { id: 1, name: 'view1', show_in_sidebar: true, show_on_dashboard: true },
id: 1, { id: 2, name: 'view2', show_in_sidebar: false, show_on_dashboard: false },
name: 'view1',
icon: 'archive',
show_in_sidebar: true,
show_on_dashboard: true,
},
{
id: 2,
name: 'view2',
icon: 'funnel',
show_in_sidebar: false,
show_on_dashboard: false,
},
] ]
describe('SavedViewsComponent', () => { describe('SavedViewsComponent', () => {
@@ -170,24 +157,6 @@ describe('SavedViewsComponent', () => {
expect(patchBody.show_in_sidebar).toBeUndefined() expect(patchBody.show_in_sidebar).toBeUndefined()
}) })
it('should persist a changed icon', () => {
const patchSpy = jest.spyOn(savedViewService, 'patchMany')
const view = savedViews[0]
const iconControl = component.savedViewsForm
.get('savedViews')
.get(view.id.toString())
.get('icon')
iconControl.setValue('bell')
iconControl.markAsDirty()
component.save()
expect(patchSpy.mock.calls[0][0][0]).toMatchObject({
id: view.id,
icon: 'bell',
})
})
it('should persist visibility changes to user settings', () => { it('should persist visibility changes to user settings', () => {
const patchSpy = jest.spyOn(savedViewService, 'patchMany') const patchSpy = jest.spyOn(savedViewService, 'patchMany')
const updateVisibilitySpy = jest const updateVisibilitySpy = jest
@@ -253,44 +222,6 @@ describe('SavedViewsComponent', () => {
).toEqual(view.show_on_dashboard) ).toEqual(view.show_on_dashboard)
}) })
it('should page saved views, clamp the page if views are removed', () => {
const manyViews = Array.from({ length: 30 }, (_, i) => ({
id: i + 1,
name: `view${i + 1}`,
})) as SavedView[]
const listSpy = jest.spyOn(savedViewService, 'list').mockReturnValue(
of({
all: manyViews.map((v) => v.id),
count: manyViews.length,
results: manyViews.concat([]),
})
)
component.ngOnInit()
fixture.detectChanges()
expect(listSpy).toHaveBeenCalledWith(1, 100000, null, false, {
full_perms: true,
})
expect(component.pagedSavedViews()).toHaveLength(25)
expect(fixture.debugElement.query(By.css('ngb-pagination'))).not.toBeNull()
// all views have controls, not just the current page
expect(
Object.keys(component.savedViewsForm.get('savedViews').value)
).toHaveLength(30)
component.page.set(2)
expect(component.pagedSavedViews()).toHaveLength(5)
listSpy.mockReturnValue(
of({
all: manyViews.slice(0, 25).map((v) => v.id),
count: 25,
results: manyViews.slice(0, 25),
})
)
component.ngOnInit()
expect(component.page()).toEqual(1)
})
it('should support editing permissions', () => { it('should support editing permissions', () => {
const confirmClicked = new Subject<any>() const confirmClicked = new Subject<any>()
const modalRef = { const modalRef = {
@@ -1,29 +1,18 @@
import { AsyncPipe } from '@angular/common' import { AsyncPipe } from '@angular/common'
import { import { Component, OnDestroy, OnInit, inject, signal } from '@angular/core'
Component,
OnDestroy,
OnInit,
computed,
inject,
signal,
} from '@angular/core'
import { import {
FormControl, FormControl,
FormGroup, FormGroup,
FormsModule, FormsModule,
ReactiveFormsModule, ReactiveFormsModule,
} from '@angular/forms' } from '@angular/forms'
import { NgbModal, NgbPaginationModule } from '@ng-bootstrap/ng-bootstrap' import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { dirtyCheck } from '@ngneat/dirty-check-forms' import { dirtyCheck } from '@ngneat/dirty-check-forms'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { BehaviorSubject, Observable, of, switchMap, takeUntil } from 'rxjs' import { BehaviorSubject, Observable, of, switchMap, takeUntil } from 'rxjs'
import { PermissionsDialogComponent } from 'src/app/components/common/permissions-dialog/permissions-dialog.component' import { PermissionsDialogComponent } from 'src/app/components/common/permissions-dialog/permissions-dialog.component'
import { DisplayMode } from 'src/app/data/document' import { DisplayMode } from 'src/app/data/document'
import { SavedView } from 'src/app/data/saved-view' import { SavedView } from 'src/app/data/saved-view'
import {
DEFAULT_SAVED_VIEW_ICON,
SAVED_VIEW_ICONS,
} from 'src/app/data/saved-view-icons'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive' import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { import {
PermissionAction, PermissionAction,
@@ -36,7 +25,6 @@ import { ToastService } from 'src/app/services/toast.service'
import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-button.component' import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-button.component'
import { DragDropSelectComponent } from '../../common/input/drag-drop-select/drag-drop-select.component' import { DragDropSelectComponent } from '../../common/input/drag-drop-select/drag-drop-select.component'
import { NumberComponent } from '../../common/input/number/number.component' import { NumberComponent } from '../../common/input/number/number.component'
import { SelectComponent } from '../../common/input/select/select.component'
import { TextComponent } from '../../common/input/text/text.component' import { TextComponent } from '../../common/input/text/text.component'
import { PageHeaderComponent } from '../../common/page-header/page-header.component' import { PageHeaderComponent } from '../../common/page-header/page-header.component'
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component' import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
@@ -48,14 +36,12 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
PageHeaderComponent, PageHeaderComponent,
ConfirmButtonComponent, ConfirmButtonComponent,
NumberComponent, NumberComponent,
SelectComponent,
TextComponent, TextComponent,
IfPermissionsDirective, IfPermissionsDirective,
DragDropSelectComponent, DragDropSelectComponent,
FormsModule, FormsModule,
ReactiveFormsModule, ReactiveFormsModule,
AsyncPipe, AsyncPipe,
NgbPaginationModule,
NgxBootstrapIconsModule, NgxBootstrapIconsModule,
], ],
}) })
@@ -70,17 +56,8 @@ export class SavedViewsComponent
private readonly modalService = inject(NgbModal) private readonly modalService = inject(NgbModal)
DisplayMode = DisplayMode DisplayMode = DisplayMode
readonly savedViewIcons = SAVED_VIEW_ICONS
readonly savedViews = signal<SavedView[]>(undefined) readonly savedViews = signal<SavedView[]>(undefined)
readonly page = signal(1)
public readonly pageSize = 25
// All views are loaded at init, so paging is only for display
readonly pagedSavedViews = computed(() => {
const start = (this.page() - 1) * this.pageSize
return this.savedViews()?.slice(start, start + this.pageSize)
})
private savedViewsGroup = new FormGroup({}) private savedViewsGroup = new FormGroup({})
public savedViewsForm: FormGroup = new FormGroup({ public savedViewsForm: FormGroup = new FormGroup({
savedViews: this.savedViewsGroup, savedViews: this.savedViewsGroup,
@@ -107,11 +84,9 @@ export class SavedViewsComponent
private reloadViews(): void { private reloadViews(): void {
this.loading.set(true) this.loading.set(true)
this.savedViewService this.savedViewService
.list(1, 100000, null, false, { full_perms: true }) .list(null, null, null, false, { full_perms: true })
.subscribe((r) => { .subscribe((r) => {
this.savedViews.set(r.results) this.savedViews.set(r.results)
const pageCount = Math.ceil(r.results.length / this.pageSize)
this.page.update((page) => Math.min(page, Math.max(1, pageCount)))
this.initialize() this.initialize()
}) })
} }
@@ -135,7 +110,6 @@ export class SavedViewsComponent
storeData.savedViews[view.id.toString()] = { storeData.savedViews[view.id.toString()] = {
id: view.id, id: view.id,
name: view.name, name: view.name,
icon: view.icon ?? DEFAULT_SAVED_VIEW_ICON,
show_on_dashboard: view.show_on_dashboard, show_on_dashboard: view.show_on_dashboard,
show_in_sidebar: view.show_in_sidebar, show_in_sidebar: view.show_in_sidebar,
page_size: view.page_size, page_size: view.page_size,
@@ -148,7 +122,6 @@ export class SavedViewsComponent
new FormGroup({ new FormGroup({
id: new FormControl({ value: null, disabled: !canEdit }), id: new FormControl({ value: null, disabled: !canEdit }),
name: new FormControl({ value: null, disabled: !canEdit }), name: new FormControl({ value: null, disabled: !canEdit }),
icon: new FormControl({ value: null, disabled: !canEdit }),
show_on_dashboard: new FormControl({ show_on_dashboard: new FormControl({
value: null, value: null,
disabled: false, disabled: false,
@@ -227,7 +200,6 @@ export class SavedViewsComponent
const modelFieldsChanged = const modelFieldsChanged =
group.get('name')?.dirty || group.get('name')?.dirty ||
group.get('icon')?.dirty ||
group.get('page_size')?.dirty || group.get('page_size')?.dirty ||
group.get('display_mode')?.dirty || group.get('display_mode')?.dirty ||
group.get('display_fields')?.dirty group.get('display_fields')?.dirty
+55
View File
@@ -54,6 +54,10 @@ export const ConfigCategory = {
AI: $localize`AI Settings`, AI: $localize`AI Settings`,
} }
export const ConfigSection = {
RemoteOCR: $localize`Remote OCR`,
}
export const LLMEmbeddingBackendConfig = { export const LLMEmbeddingBackendConfig = {
OPENAI_LIKE: 'openai-like', OPENAI_LIKE: 'openai-like',
HUGGINGFACE: 'huggingface', HUGGINGFACE: 'huggingface',
@@ -65,6 +69,15 @@ export const LLMBackendConfig = {
OLLAMA: 'ollama', OLLAMA: 'ollama',
} }
export const RemoteOCREngineConfig = {
AZURE_AI: 'azureai',
}
export const RemoteOCRModeConfig = {
ALWAYS: 'always',
WORKFLOW_ONLY: 'workflow_only',
}
export interface ConfigOption { export interface ConfigOption {
key: string key: string
title: string title: string
@@ -72,6 +85,7 @@ export interface ConfigOption {
choices?: Array<{ id: string; name: string }> choices?: Array<{ id: string; name: string }>
config_key?: string config_key?: string
category: string category: string
section?: string
note?: string note?: string
} }
@@ -181,6 +195,43 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_OCR_USER_ARGS', config_key: 'PAPERLESS_OCR_USER_ARGS',
category: ConfigCategory.OCR, category: ConfigCategory.OCR,
}, },
{
key: 'remote_ocr_engine',
title: $localize`Remote OCR Engine`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCREngineConfig),
config_key: 'PAPERLESS_REMOTE_OCR_ENGINE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Enabling remote OCR sends documents to a third-party service for processing. Consider the privacy implications as well as potential costs before enabling.`,
},
{
key: 'remote_ocr_api_key',
title: $localize`Remote OCR API Key`,
type: ConfigOptionType.Password,
config_key: 'PAPERLESS_REMOTE_OCR_API_KEY',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
},
{
key: 'remote_ocr_endpoint',
title: $localize`Remote OCR Endpoint`,
type: ConfigOptionType.String,
config_key: 'PAPERLESS_REMOTE_OCR_ENDPOINT',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Required when using the Azure AI engine.`,
},
{
key: 'remote_ocr_mode',
title: $localize`Remote OCR Mode`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCRModeConfig),
config_key: 'PAPERLESS_REMOTE_OCR_MODE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Which documents are sent to the remote engine. Use 'workflow_only' to keep remote OCR off unless a workflow enables it for a document.`,
},
{ {
key: 'app_logo', key: 'app_logo',
title: $localize`Application Logo`, title: $localize`Application Logo`,
@@ -398,6 +449,10 @@ export interface PaperlessConfig extends ObjectWithId {
barcode_enable_tag: boolean barcode_enable_tag: boolean
barcode_tag_mapping: object barcode_tag_mapping: object
barcode_tag_split: boolean barcode_tag_split: boolean
remote_ocr_engine: string
remote_ocr_api_key: string
remote_ocr_endpoint: string
remote_ocr_mode: string
ai_enabled: boolean ai_enabled: boolean
llm_embedding_backend: string llm_embedding_backend: string
llm_embedding_model: string llm_embedding_model: string
-89
View File
@@ -1,89 +0,0 @@
export const DEFAULT_SAVED_VIEW_ICON = 'funnel'
export const SAVED_VIEW_ICONS = [
{ id: 'archive', name: $localize`Archive`, icon: 'archive' },
{ id: 'bank', name: $localize`Bank`, icon: 'bank' },
{ id: 'basket', name: $localize`Basket`, icon: 'basket' },
{ id: 'bell', name: $localize`Bell`, icon: 'bell' },
{ id: 'bookmark', name: $localize`Bookmark`, icon: 'bookmark' },
{ id: 'boxes', name: $localize`Boxes`, icon: 'boxes' },
{ id: 'briefcase', name: $localize`Briefcase`, icon: 'briefcase' },
{ id: 'building', name: $localize`Building`, icon: 'building' },
{ id: 'calculator', name: $localize`Calculator`, icon: 'calculator' },
{ id: 'calendar', name: $localize`Calendar`, icon: 'calendar' },
{ id: 'camera', name: $localize`Camera`, icon: 'camera' },
{
id: 'card-checklist',
name: $localize`Checklist`,
icon: 'card-checklist',
},
{ id: 'cash', name: $localize`Cash`, icon: 'cash' },
{ id: 'chat-left-text', name: $localize`Chat`, icon: 'chat-left-text' },
{ id: 'check-circle', name: $localize`Check`, icon: 'check-circle' },
{ id: 'clipboard', name: $localize`Clipboard`, icon: 'clipboard' },
{ id: 'clock-history', name: $localize`Clock`, icon: 'clock-history' },
{ id: 'credit-card', name: $localize`Credit card`, icon: 'credit-card' },
{ id: 'download', name: $localize`Download`, icon: 'download' },
{ id: 'envelope', name: $localize`Envelope`, icon: 'envelope' },
{
id: 'exclamation-triangle',
name: $localize`Warning`,
icon: 'exclamation-triangle',
},
{ id: 'file-earmark', name: $localize`File`, icon: 'file-earmark' },
{
id: 'file-earmark-check',
name: $localize`Checked file`,
icon: 'file-earmark-check',
},
{
id: 'file-earmark-lock',
name: $localize`Locked file`,
icon: 'file-earmark-lock',
},
{
id: 'file-earmark-medical',
name: $localize`Medical file`,
icon: 'file-earmark-medical',
},
{
id: 'file-earmark-person',
name: $localize`Person file`,
icon: 'file-earmark-person',
},
{
id: 'file-earmark-spreadsheet',
name: $localize`Spreadsheet`,
icon: 'file-earmark-spreadsheet',
},
{ id: 'file-text', name: $localize`Text file`, icon: 'file-text' },
{ id: 'files', name: $localize`Files`, icon: 'files' },
{ id: 'folder', name: $localize`Folder`, icon: 'folder' },
{ id: 'funnel', name: $localize`Filter`, icon: 'funnel' },
{ id: 'gear', name: $localize`Gear`, icon: 'gear' },
{ id: 'globe2', name: $localize`Globe`, icon: 'globe2' },
{ id: 'hash', name: $localize`Hash`, icon: 'hash' },
{ id: 'heart', name: $localize`Heart`, icon: 'heart' },
{ id: 'house', name: $localize`House`, icon: 'house' },
{ id: 'inbox', name: $localize`Inbox`, icon: 'inbox' },
{ id: 'journals', name: $localize`Journals`, icon: 'journals' },
{ id: 'list-task', name: $localize`Task list`, icon: 'list-task' },
{ id: 'newspaper', name: $localize`Newspaper`, icon: 'newspaper' },
{ id: 'paperclip', name: $localize`Attachment`, icon: 'paperclip' },
{ id: 'people', name: $localize`People`, icon: 'people' },
{ id: 'person', name: $localize`Person`, icon: 'person' },
{ id: 'printer', name: $localize`Printer`, icon: 'printer' },
{ id: 'receipt', name: $localize`Receipt`, icon: 'receipt' },
{ id: 'safe', name: $localize`Safe`, icon: 'safe' },
{ id: 'search', name: $localize`Search`, icon: 'search' },
{ id: 'send', name: $localize`Send`, icon: 'send' },
{ id: 'shop', name: $localize`Shop`, icon: 'shop' },
{ id: 'stack', name: $localize`Stack`, icon: 'stack' },
{ id: 'stars', name: $localize`Stars`, icon: 'stars' },
{ id: 'tag', name: $localize`Tag`, icon: 'tag' },
{ id: 'tags', name: $localize`Tags`, icon: 'tags' },
{ id: 'telephone', name: $localize`Telephone`, icon: 'telephone' },
{ id: 'truck', name: $localize`Truck`, icon: 'truck' },
{ id: 'upc-scan', name: $localize`Barcode`, icon: 'upc-scan' },
{ id: 'wallet2', name: $localize`Wallet`, icon: 'wallet2' },
]
-2
View File
@@ -5,8 +5,6 @@ import { ObjectWithPermissions } from './object-with-permissions'
export interface SavedView extends ObjectWithPermissions { export interface SavedView extends ObjectWithPermissions {
name?: string name?: string
icon?: string
show_on_dashboard?: boolean show_on_dashboard?: boolean
show_in_sidebar?: boolean show_in_sidebar?: boolean
+13
View File
@@ -1,5 +1,6 @@
import { PdfEditorEditMode } from '../components/common/pdf-editor/pdf-editor-edit-mode' import { PdfEditorEditMode } from '../components/common/pdf-editor/pdf-editor-edit-mode'
import { PdfZoomScale } from '../components/common/pdf-viewer/pdf-viewer.types' import { PdfZoomScale } from '../components/common/pdf-viewer/pdf-viewer.types'
import { RemoteOCRModeConfig } from './paperless-config'
import { User } from './user' import { User } from './user'
export interface UiSettings { export interface UiSettings {
@@ -94,6 +95,8 @@ export const SETTINGS_KEYS = {
OUTLOOK_OAUTH_URL: 'outlook_oauth_url', OUTLOOK_OAUTH_URL: 'outlook_oauth_url',
EMAIL_ENABLED: 'email_enabled', EMAIL_ENABLED: 'email_enabled',
AI_ENABLED: 'ai_enabled', AI_ENABLED: 'ai_enabled',
REMOTE_OCR_CONFIGURED: 'remote_ocr:configured',
REMOTE_OCR_MODE: 'remote_ocr:mode',
} }
export const SETTINGS: UiSetting[] = [ export const SETTINGS: UiSetting[] = [
@@ -347,4 +350,14 @@ export const SETTINGS: UiSetting[] = [
type: 'string', type: 'string',
default: PdfEditorEditMode.Create, default: PdfEditorEditMode.Create,
}, },
{
key: SETTINGS_KEYS.REMOTE_OCR_CONFIGURED,
type: 'boolean',
default: false,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_MODE,
type: 'string',
default: RemoteOCRModeConfig.ALWAYS,
},
] ]
+18
View File
@@ -7,6 +7,18 @@ export enum WorkflowActionType {
Webhook = 4, Webhook = 4,
PasswordRemoval = 5, PasswordRemoval = 5,
MoveToTrash = 6, MoveToTrash = 6,
RemoteOcr = 7,
ApplyAiSuggestions = 8,
}
// see src/documents/models.py AISuggestionField
export enum AISuggestionField {
Title = 'title',
Tags = 'tags',
Correspondent = 'correspondent',
DocumentType = 'document_type',
StoragePath = 'storage_path',
Created = 'created',
} }
export interface WorkflowActionEmail extends ObjectWithId { export interface WorkflowActionEmail extends ObjectWithId {
@@ -101,4 +113,10 @@ export interface WorkflowAction extends ObjectWithId {
webhook?: WorkflowActionWebhook webhook?: WorkflowActionWebhook
passwords?: string[] passwords?: string[]
ai_suggestion_fields?: AISuggestionField[]
ai_create_missing?: boolean
ai_overwrite_existing?: boolean
} }
@@ -284,6 +284,21 @@ describe(`DocumentService`, () => {
expect(req.request.method).toEqual('POST') expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({ expect(req.request.body).toEqual({
documents: ids, documents: ids,
remote_ocr: false,
})
})
it('should request remote OCR when reprocessing with it enabled', () => {
const ids = [1, 2, 3]
subscription = service
.reprocessDocuments({ documents: ids }, true)
.subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/reprocess/`
)
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: true,
}) })
}) })
@@ -349,9 +349,13 @@ export class DocumentService extends AbstractPaperlessService<Document> {
}) })
} }
reprocessDocuments(selection: DocumentSelectionQuery) { reprocessDocuments(
selection: DocumentSelectionQuery,
remoteOcr: boolean = false
) {
return this.http.post(this.getResourceUrl(null, 'reprocess'), { return this.http.post(this.getResourceUrl(null, 'reprocess'), {
...selection, ...selection,
remote_ocr: remoteOcr,
}) })
} }
@@ -13,6 +13,7 @@ import { environment } from 'src/environments/environment'
import { CustomFieldDataType } from '../data/custom-field' import { CustomFieldDataType } from '../data/custom-field'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document' import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { SavedView } from '../data/saved-view' import { SavedView } from '../data/saved-view'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings' import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings'
import { PermissionsService } from './permissions.service' import { PermissionsService } from './permissions.service'
import { CustomFieldsService } from './rest/custom-fields.service' import { CustomFieldsService } from './rest/custom-fields.service'
@@ -434,4 +435,26 @@ describe('SettingsService', () => {
).name ).name
).toEqual(customFields[0].name) ).toEqual(customFields[0].name)
}) })
it('should offer remote OCR only when configured and selective', () => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, false)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
// configured, but already handling every document
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, true)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.ALWAYS
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeTruthy()
})
}) })
@@ -19,6 +19,7 @@ import {
} from 'src/app/utils/color' } from 'src/app/utils/color'
import { DEFAULT_APP_TITLE, environment } from 'src/environments/environment' import { DEFAULT_APP_TITLE, environment } from 'src/environments/environment'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document' import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SavedView } from '../data/saved-view' import { SavedView } from '../data/saved-view'
import { import {
PAPERLESS_GREEN_HEX, PAPERLESS_GREEN_HEX,
@@ -687,6 +688,17 @@ export class SettingsService {
return this.settingIsSet(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED) return this.settingIsSet(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)
} }
/**
* Offering remote OCR as a choice only makes sense when an engine
* is configured but is not already handling every document.
*/
get remoteOCRIsSelectable(): boolean {
return (
this.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
this.get(SETTINGS_KEYS.REMOTE_OCR_MODE) !== RemoteOCRModeConfig.ALWAYS
)
}
offerTour(): boolean { offerTour(): boolean {
return this.dashboardIsEmpty() && !this.get(SETTINGS_KEYS.TOUR_COMPLETE) return this.dashboardIsEmpty() && !this.get(SETTINGS_KEYS.TOUR_COMPLETE)
} }
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-46
View File
@@ -35,27 +35,19 @@ import {
arrowRightShort, arrowRightShort,
arrowUpRight, arrowUpRight,
asterisk, asterisk,
bank,
basket,
bell, bell,
bodyText, bodyText,
bookmark,
boxArrowUp, boxArrowUp,
boxArrowUpRight, boxArrowUpRight,
boxes, boxes,
braces, braces,
briefcase,
building,
calculator,
calendar, calendar,
calendarEvent, calendarEvent,
calendarEventFill, calendarEventFill,
camera,
cardChecklist, cardChecklist,
cardHeading, cardHeading,
caretDown, caretDown,
caretUp, caretUp,
cash,
chatLeftText, chatLeftText,
chatSquareDots, chatSquareDots,
check, check,
@@ -73,7 +65,6 @@ import {
clipboardCheckFill, clipboardCheckFill,
clipboardFill, clipboardFill,
clockHistory, clockHistory,
creditCard,
dash, dash,
dashCircle, dashCircle,
diagram3, diagram3,
@@ -92,12 +83,9 @@ import {
fileEarmarkDiff, fileEarmarkDiff,
fileEarmarkFill, fileEarmarkFill,
fileEarmarkLock, fileEarmarkLock,
fileEarmarkMedical,
fileEarmarkMinus, fileEarmarkMinus,
fileEarmarkPerson,
fileEarmarkPlus, fileEarmarkPlus,
fileEarmarkRichtext, fileEarmarkRichtext,
fileEarmarkSpreadsheet,
fileText, fileText,
files, files,
filter, filter,
@@ -105,15 +93,12 @@ import {
folderFill, folderFill,
funnel, funnel,
gear, gear,
globe2,
google, google,
grid, grid,
gripVertical, gripVertical,
hash, hash,
hddStack, hddStack,
heart,
house, house,
inbox,
infoCircle, infoCircle,
journals, journals,
link, link,
@@ -121,9 +106,7 @@ import {
listTask, listTask,
listUl, listUl,
microsoft, microsoft,
newspaper,
nodePlus, nodePlus,
paperclip,
pencil, pencil,
people, people,
peopleFill, peopleFill,
@@ -138,12 +121,9 @@ import {
plusCircle, plusCircle,
printer, printer,
questionCircle, questionCircle,
receipt,
safe,
scissors, scissors,
search, search,
send, send,
shop,
slashCircle, slashCircle,
sliders2Vertical, sliders2Vertical,
sortAlphaDown, sortAlphaDown,
@@ -153,17 +133,14 @@ import {
tag, tag,
tagFill, tagFill,
tags, tags,
telephone,
textIndentLeft, textIndentLeft,
textLeft, textLeft,
threeDots, threeDots,
threeDotsVertical, threeDotsVertical,
trash, trash,
truck,
uiRadios, uiRadios,
unlock, unlock,
upcScan, upcScan,
wallet2,
windowStack, windowStack,
x, x,
xCircle, xCircle,
@@ -281,22 +258,15 @@ const icons = {
arrowRightShort, arrowRightShort,
arrowUpRight, arrowUpRight,
asterisk, asterisk,
bank,
basket,
bell, bell,
braces, braces,
bodyText, bodyText,
bookmark,
boxArrowUp, boxArrowUp,
boxArrowUpRight, boxArrowUpRight,
boxes, boxes,
briefcase,
building,
calculator,
calendar, calendar,
calendarEvent, calendarEvent,
calendarEventFill, calendarEventFill,
camera,
cardChecklist, cardChecklist,
cardHeading, cardHeading,
caretDown, caretDown,
@@ -318,8 +288,6 @@ const icons = {
clipboardCheckFill, clipboardCheckFill,
clipboardFill, clipboardFill,
clockHistory, clockHistory,
cash,
creditCard,
dash, dash,
dashCircle, dashCircle,
diagram3, diagram3,
@@ -338,12 +306,9 @@ const icons = {
fileEarmarkDiff, fileEarmarkDiff,
fileEarmarkFill, fileEarmarkFill,
fileEarmarkLock, fileEarmarkLock,
fileEarmarkMedical,
fileEarmarkMinus, fileEarmarkMinus,
fileEarmarkPerson,
fileEarmarkPlus, fileEarmarkPlus,
fileEarmarkRichtext, fileEarmarkRichtext,
fileEarmarkSpreadsheet,
files, files,
fileText, fileText,
filter, filter,
@@ -351,15 +316,12 @@ const icons = {
folderFill, folderFill,
funnel, funnel,
gear, gear,
globe2,
google, google,
grid, grid,
gripVertical, gripVertical,
hash, hash,
hddStack, hddStack,
heart,
house, house,
inbox,
infoCircle, infoCircle,
journals, journals,
link, link,
@@ -367,10 +329,8 @@ const icons = {
listTask, listTask,
listUl, listUl,
microsoft, microsoft,
newspaper,
nodePlus, nodePlus,
pencil, pencil,
paperclip,
people, people,
peopleFill, peopleFill,
person, person,
@@ -384,13 +344,10 @@ const icons = {
plusCircle, plusCircle,
printer, printer,
questionCircle, questionCircle,
receipt,
safe,
scissors, scissors,
search, search,
send, send,
slashCircle, slashCircle,
shop,
sliders2Vertical, sliders2Vertical,
sortAlphaDown, sortAlphaDown,
sortAlphaUpAlt, sortAlphaUpAlt,
@@ -401,15 +358,12 @@ const icons = {
tags, tags,
textIndentLeft, textIndentLeft,
textLeft, textLeft,
telephone,
threeDots, threeDots,
threeDotsVertical, threeDotsVertical,
trash, trash,
truck,
uiRadios, uiRadios,
unlock, unlock,
upcScan, upcScan,
wallet2,
windowStack, windowStack,
x, x,
xCircle, xCircle,
+8 -2
View File
@@ -394,10 +394,16 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
return "OK" return "OK"
def reprocess(doc_ids: list[int]) -> Literal["OK"]: def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
"""
Re-run parsing for the given documents.
Consumption workflows do not run here, so ``remote_ocr`` is how the user
asks for the remote engine when it is not configured to handle everything.
"""
for document_id in doc_ids: for document_id in doc_ids:
update_document_content_maybe_archive_file.apply_async( update_document_content_maybe_archive_file.apply_async(
kwargs={"document_id": document_id}, kwargs={"document_id": document_id, "remote_ocr": remote_ocr},
headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL}, headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
) )
+18
View File
@@ -53,6 +53,7 @@ from documents.utils import copy_basic_file_stats
from documents.utils import copy_file_with_basic_stats from documents.utils import copy_file_with_basic_stats
from documents.utils import run_subprocess from documents.utils import run_subprocess
from paperless.config import OcrConfig from paperless.config import OcrConfig
from paperless.config import RemoteOCRConfig
from paperless.models import ArchiveFileGenerationChoices from paperless.models import ArchiveFileGenerationChoices
from paperless.parsers import ParserContext from paperless.parsers import ParserContext
from paperless.parsers import ParserProtocol from paperless.parsers import ParserProtocol
@@ -451,12 +452,19 @@ class ConsumerPlugin(
except Exception as e: except Exception as e:
self.log.error(f"Error attempting to clean PDF: {e}") self.log.error(f"Error attempting to clean PDF: {e}")
# Workflows have already run at this point, so the metadata knows
# whether this document was singled out for remote OCR
allow_remote = (
self.metadata.remote_ocr or RemoteOCRConfig().remote_ocr_by_default
)
# Based on the mime type, get the parser for that type # Based on the mime type, get the parser for that type
parser_class: type[ParserProtocol] | None = ( parser_class: type[ParserProtocol] | None = (
get_parser_registry().get_parser_for_file( get_parser_registry().get_parser_for_file(
mime_type, mime_type,
self.filename, self.filename,
self.working_copy, self.working_copy,
allow_remote=allow_remote,
) )
) )
if not parser_class: if not parser_class:
@@ -465,6 +473,16 @@ class ConsumerPlugin(
f"Unsupported mime type {mime_type}", f"Unsupported mime type {mime_type}",
) )
if self.metadata.remote_ocr and not getattr(
parser_class,
"uses_remote_service",
False,
):
self.log.warning(
"Remote OCR was requested for this document but no remote "
"parser is available for it, processing locally instead.",
)
# Notify all listeners that we're going to do some work. # Notify all listeners that we're going to do some work.
document_consumption_started.send( document_consumption_started.send(
+3
View File
@@ -34,6 +34,7 @@ class DocumentMetadataOverrides:
skip_asn_if_exists: bool = False skip_asn_if_exists: bool = False
version_label: str | None = None version_label: str | None = None
actor_id: int | None = None actor_id: int | None = None
remote_ocr: bool = False
def update(self, other: "DocumentMetadataOverrides") -> "DocumentMetadataOverrides": def update(self, other: "DocumentMetadataOverrides") -> "DocumentMetadataOverrides":
""" """
@@ -57,6 +58,8 @@ class DocumentMetadataOverrides:
self.actor_id = other.actor_id self.actor_id = other.actor_id
if other.skip_asn_if_exists: if other.skip_asn_if_exists:
self.skip_asn_if_exists = True self.skip_asn_if_exists = True
if other.remote_ocr:
self.remote_ocr = True
if other.version_label is not None: if other.version_label is not None:
self.version_label = other.version_label self.version_label = other.version_label
@@ -55,7 +55,7 @@ class Command(PaperlessCommand):
"--ratio", "--ratio",
default=85.0, default=85.0,
type=float, type=float,
help="Ratio to consider documents a match (0.0 - 100.0)", help="Ratio to consider documents a match",
) )
parser.add_argument( parser.add_argument(
"--delete", "--delete",
@@ -69,17 +69,6 @@ class Command(PaperlessCommand):
action="store_true", action="store_true",
help="Skip the confirmation prompt when used with --delete", help="Skip the confirmation prompt when used with --delete",
) )
parser.add_argument(
"--url",
default=None,
type=str,
help=(
"Base URL of the Paperless instance (e.g. "
"http://localhost:8000 or https://paperless.local). If set, matched "
"documents are shown as clickable (usually ctrl+click) links to "
"<url>/documents/<id>/details instead of by title."
),
)
def _render_results( def _render_results(
self, self,
@@ -87,7 +76,6 @@ class Command(PaperlessCommand):
*, *,
opt_ratio: float, opt_ratio: float,
do_delete: bool, do_delete: bool,
base_url: str | None = None,
) -> list[int]: ) -> list[int]:
"""Render match results as a Rich table. Returns list of PKs to delete.""" """Render match results as a Rich table. Returns list of PKs to delete."""
if not matches: if not matches:
@@ -100,22 +88,13 @@ class Command(PaperlessCommand):
) )
return [] return []
# Fetch titles for matched documents in a single query, unless we're # Fetch titles for matched documents in a single query.
# going to show URLs instead. all_pks = {pk for m in matches for pk in (m.doc_one_pk, m.doc_two_pk)}
titles: dict[int, str] = {} titles: dict[int, str] = dict(
if not base_url: Document.objects.filter(pk__in=all_pks)
all_pks = {pk for m in matches for pk in (m.doc_one_pk, m.doc_two_pk)} .only("pk", "title")
titles = dict( .values_list("pk", "title"),
Document.objects.filter(pk__in=all_pks) )
.only("pk", "title")
.values_list("pk", "title"),
)
def _cell(pk: int) -> str:
if base_url:
doc_url = f"{base_url.rstrip('/')}/documents/{pk}/details"
return f"[link={doc_url}]{doc_url}[/link]"
return f"[dim]#{pk}[/dim] {titles.get(pk, 'Unknown')}"
table = Table( table = Table(
title=f"Fuzzy Matches (threshold: {opt_ratio:.1f}%)", title=f"Fuzzy Matches (threshold: {opt_ratio:.1f}%)",
@@ -145,8 +124,8 @@ class Command(PaperlessCommand):
table.add_row( table.add_row(
str(i), str(i),
_cell(pk_a), f"[dim]#{pk_a}[/dim] {titles.get(pk_a, 'Unknown')}",
_cell(pk_b), f"[dim]#{pk_b}[/dim] {titles.get(pk_b, 'Unknown')}",
Text(f"{ratio:.1f}%", style=ratio_style), Text(f"{ratio:.1f}%", style=ratio_style),
) )
maybe_delete_ids.append(pk_b) maybe_delete_ids.append(pk_b)
@@ -229,7 +208,6 @@ class Command(PaperlessCommand):
matches, matches,
opt_ratio=opt_ratio, opt_ratio=opt_ratio,
do_delete=options["delete"], do_delete=options["delete"],
base_url=options["url"],
) )
if options["delete"] and maybe_delete_ids: if options["delete"] and maybe_delete_ids:
@@ -0,0 +1,30 @@
# Generated by Django 5.2.16 on 2026-08-10 17:27
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0022_add_perf_indexes"),
]
operations = [
migrations.AlterField(
model_name="workflowaction",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Assignment"),
(2, "Removal"),
(3, "Email"),
(4, "Webhook"),
(5, "Password removal"),
(6, "Move to trash"),
(7, "Remote OCR"),
],
default=1,
verbose_name="Workflow Action Type",
),
),
]
@@ -1,79 +0,0 @@
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0022_add_perf_indexes"),
]
operations = [
migrations.AddField(
model_name="savedview",
name="icon",
field=models.CharField(
choices=[
("archive", "Archive"),
("bank", "Bank"),
("basket", "Basket"),
("bell", "Bell"),
("bookmark", "Bookmark"),
("boxes", "Boxes"),
("briefcase", "Briefcase"),
("building", "Building"),
("calculator", "Calculator"),
("calendar", "Calendar"),
("camera", "Camera"),
("card-checklist", "Checklist"),
("cash", "Cash"),
("chat-left-text", "Chat"),
("check-circle", "Check"),
("clipboard", "Clipboard"),
("clock-history", "Clock"),
("credit-card", "Credit card"),
("download", "Download"),
("envelope", "Envelope"),
("exclamation-triangle", "Warning"),
("file-earmark", "File"),
("file-earmark-check", "Checked file"),
("file-earmark-lock", "Locked file"),
("file-earmark-medical", "Medical file"),
("file-earmark-person", "Person file"),
("file-earmark-spreadsheet", "Spreadsheet"),
("file-text", "Text file"),
("files", "Files"),
("folder", "Folder"),
("funnel", "Filter"),
("gear", "Gear"),
("globe2", "Globe"),
("hash", "Hash"),
("heart", "Heart"),
("house", "House"),
("inbox", "Inbox"),
("journals", "Journals"),
("list-task", "Task list"),
("newspaper", "Newspaper"),
("paperclip", "Attachment"),
("people", "People"),
("person", "Person"),
("printer", "Printer"),
("receipt", "Receipt"),
("safe", "Safe"),
("search", "Search"),
("send", "Send"),
("shop", "Shop"),
("stack", "Stack"),
("stars", "Stars"),
("tag", "Tag"),
("tags", "Tags"),
("telephone", "Telephone"),
("truck", "Truck"),
("upc-scan", "Barcode"),
("wallet2", "Wallet"),
],
default="funnel",
max_length=64,
verbose_name="icon",
),
),
]
@@ -0,0 +1,84 @@
# Generated by Django 5.2.16 on 2026-08-10 18:26
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0023_alter_workflowaction_type"),
]
operations = [
migrations.AddField(
model_name="workflowaction",
name="ai_create_missing",
field=models.BooleanField(
default=False,
help_text="Create suggested tags, correspondents, document types and storage paths that do not already exist instead of skipping them.",
verbose_name="create missing objects",
),
),
migrations.AddField(
model_name="workflowaction",
name="ai_overwrite_existing",
field=models.BooleanField(
default=False,
help_text="Apply suggestions even if the document already has a value for that field. Tags are always added to, never replaced.",
verbose_name="overwrite existing values",
),
),
migrations.AddField(
model_name="workflowaction",
name="ai_suggestion_fields",
field=models.JSONField(
blank=True,
help_text="Which of the AI-suggested fields to apply to the document.",
null=True,
verbose_name="AI suggestion fields",
),
),
migrations.AlterField(
model_name="workflowaction",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Assignment"),
(2, "Removal"),
(3, "Email"),
(4, "Webhook"),
(5, "Password removal"),
(6, "Move to trash"),
(7, "Remote OCR"),
(8, "Apply AI suggestions"),
],
default=1,
verbose_name="Workflow Action Type",
),
),
migrations.AlterField(
model_name="paperlesstask",
name="task_type",
field=models.CharField(
choices=[
("consume_file", "Consume File"),
("train_classifier", "Train Classifier"),
("sanity_check", "Sanity Check"),
("index_optimize", "Index Optimize"),
("mail_fetch", "Mail Fetch"),
("llm_index", "LLM Index"),
("empty_trash", "Empty Trash"),
("check_workflows", "Check Workflows"),
("bulk_update", "Bulk Update"),
("reprocess_document", "Reprocess Document"),
("build_share_link", "Build Share Link"),
("bulk_delete", "Bulk Delete"),
("apply_ai_suggestions", "Apply AI Suggestions"),
],
db_index=True,
help_text="The kind of work being performed",
max_length=50,
verbose_name="Task Type",
),
),
]

Some files were not shown because too many files have changed in this diff Show More