mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-11-12 17:20:48 +00:00
Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 stumpylog
|
e02f6fff56 | ||
|
Trenton H
|
a1d76f3a19 | ||
 GitHub Actions
|
f5ca9bbcb6 | ||
 shamoon
|
60a35edee1 | ||
|
Trenton H
|
7cdcd71a6d | ||
|
shamoon
|
759f9cd22b | ||
|
Trenton H
|
d05e96cda2 | ||
|
shamoon
|
fefce7be1b | ||
|
GitHub Actions
|
cc65226d6d | ||
|
shamoon
|
00096e0252 | ||
|
shamoon
|
2829e78009 | ||
 dependabot[bot]andstumpylog
|
d7d4014bbd | ||
|
shamoon
|
a139bdff46 |
No files matched your search
@@ -8,7 +8,7 @@ ARG DEBIAN_FRONTEND=noninteractive
|
||||
ARG TARGETARCH
|
||||
|
||||
# Can be workflow provided, defaults set for manual building
|
||||
ARG JBIG2ENC_VERSION=0.30
|
||||
ARG JBIG2ENC_VERSION=0.32
|
||||
|
||||
# Set Python environment variables
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
|
||||
@@ -1,5 +1,23 @@
|
||||
title: "[Feature Request] "
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
#### Use of AI tools
|
||||
|
||||
If this request was generated, in whole or in part, by an AI tool or agent you **must** say so using the field below. Failing to clearly disclose this is a violation of our [Code of Conduct](https://github.com/paperless-ngx/paperless-ngx/blob/main/CODE_OF_CONDUCT.md).
|
||||
|
||||
The project prefers thoughtful, well-described ideas to AI-generated content. Please describe the problem, your use case, the desired behavior, and who would benefit.
|
||||
- type: dropdown
|
||||
id: ai-usage
|
||||
attributes:
|
||||
label: AI usage
|
||||
description: Was any part of this request written by an AI tool or agent?
|
||||
options:
|
||||
- No AI tools were used
|
||||
- Yes, AI tools helped write this request
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
|
||||
@@ -1,5 +1,21 @@
|
||||
title: "[Support] "
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
#### Use of AI tools
|
||||
|
||||
If this post was generated, in whole or in part, by an AI tool or agent you **must** say so using the field below. Failing to clearly disclose this is a violation of our [Code of Conduct](https://github.com/paperless-ngx/paperless-ngx/blob/main/CODE_OF_CONDUCT.md).
|
||||
- type: dropdown
|
||||
id: ai-usage
|
||||
attributes:
|
||||
label: AI usage
|
||||
description: Was any part of this post written by an AI tool or agent?
|
||||
options:
|
||||
- No AI tools were used
|
||||
- Yes, AI tools helped write this post
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
|
||||
@@ -38,7 +38,7 @@ repos:
|
||||
- json
|
||||
# See https://github.com/prettier/prettier/issues/15742 for the fork reason
|
||||
- repo: https://github.com/rbubley/mirrors-prettier
|
||||
rev: 'v3.9.9'
|
||||
rev: 'v3.9.10'
|
||||
hooks:
|
||||
- id: prettier
|
||||
types_or:
|
||||
@@ -46,16 +46,16 @@ repos:
|
||||
- ts
|
||||
- markdown
|
||||
additional_dependencies:
|
||||
- prettier@3.9.9
|
||||
- prettier@3.9.10
|
||||
- 'prettier-plugin-organize-imports@4.3.0'
|
||||
# Python hooks
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.16.9
|
||||
rev: v0.17.2
|
||||
hooks:
|
||||
- id: ruff-check
|
||||
- id: ruff-format
|
||||
- repo: https://github.com/tox-dev/pyproject-fmt
|
||||
rev: "v2.29.4"
|
||||
rev: "v2.30.3"
|
||||
hooks:
|
||||
- id: pyproject-fmt
|
||||
additional_dependencies: [tomli]
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# Guidance for AI agents
|
||||
|
||||
Read [CONTRIBUTING.md](CONTRIBUTING.md) before working on this repository, especially [Use of AI Tools](CONTRIBUTING.md#use-of-ai-tools). These contribution rules apply to all agent-assisted work.
|
||||
|
||||
## Contribution boundaries
|
||||
|
||||
- Do not open or submit pull requests, including "on behalf of" a human. The project does not accept PRs from AI agents. Prepare local changes for a human to review, verify, and submit personally.
|
||||
- The project does not accept PRs that are entirely or mostly AI-derived. Human submission and disclosure do not override this rule.
|
||||
- AI-generated content must be clearly disclosed as described in the contributing guidelines. When handing work back to a human, identify what was generated or changed with AI assistance so they can disclose it accurately.
|
||||
- Trust matters. Encourage first-time contributors to discuss their proposed change before preparing a PR, especially for large changes or changes involving AI-generated code. New features and enhancements should almost always target an existing feature request with community interest and discussion.
|
||||
- The project prefers thoughtful, well-described ideas to AI-generated pull requests. Help contributors describe their problem, use case, desired behavior, and who would benefit. An implementation is not required for a valuable contribution, and a flawed implementation can obscure a useful idea before it receives proper discussion.
|
||||
- AI tools have lowered the effort needed to produce a PR, but reviewing and maintaining it still requires substantial work. Encourage discussion of the idea before investing in implementation and review.
|
||||
- Issue reports must describe observed behavior, reproduction steps, and relevant logs. Do not add speculative root causes, code analysis, or proposed fixes to agent-generated reports.
|
||||
|
||||
## Security reports
|
||||
|
||||
- AI-assisted security reports must meet the legitimacy and verification requirements in [SECURITY.md](SECURITY.md#ai-assisted-submissions). A human must verify a real Paperless-ngx code path, concrete reproduction steps, affected versions, and realistic security impact before submission.
|
||||
- Account for existing permission checks, configuration requirements, and trust boundaries. Speculation, scanner output, or concerns about third-party software or deployment choices without a demonstrated Paperless-ngx vulnerability do not meet this threshold and may be closed without extended discussion.
|
||||
- Disclose AI assistance and use the private vulnerability reporting channel specified in [SECURITY.md](SECURITY.md#reporting-a-vulnerability).
|
||||
|
||||
## Working on changes
|
||||
|
||||
- Read the relevant code, tests, and documentation before editing. Paperless-ngx is complex: trace interactions between affected components rather than assuming a change is isolated.
|
||||
- Keep changes focused on the requested work and follow existing patterns. Avoid unrelated refactoring, formatting, dependency updates, or generated-file changes.
|
||||
- Consider effects on permissions, document processing, background tasks, storage, and the API when those areas are involved.
|
||||
- Add or update meaningful tests for behavior changes and regressions. Do not weaken tests or checks to make a change pass.
|
||||
- Update user or developer documentation when behavior or setup changes.
|
||||
- Keep credentials and private documents out of commits, fixtures, logs, and examples.
|
||||
|
||||
## Repository and validation
|
||||
|
||||
- `src/`: Django back end.
|
||||
- `src-ui/`: Angular front end, managed with `pnpm`.
|
||||
- `docs/`: Documentation.
|
||||
- `docker/` and `scripts/`: Deployment and development tooling.
|
||||
- Base functional changes on `dev`; documentation-only changes may target `main`.
|
||||
|
||||
Follow [the development documentation](docs/development.md) for environment setup and testing. Common checks after setup are:
|
||||
|
||||
- Back end: run `uv run pytest` from `src/` (use a test path to focus on the affected area).
|
||||
- Front end: run `pnpm test` and `pnpm lint` from `src-ui/`.
|
||||
- Formatting and other pre-commit checks: run `uv run prek run --files <changed-files>` from the repository root.
|
||||
|
||||
When handing work back, summarize the changes, the checks actually run and their results, and any remaining uncertainty. Clearly state when a check could not be run; do not claim verification that did not happen. The human contributor must understand and independently verify the contribution before submitting it.
|
||||
+16
-3
@@ -4,6 +4,18 @@ If you feel like contributing to the project, please do! Bug fixes and improveme
|
||||
|
||||
⚠️ Please note: Pull requests that implement a new feature or enhancement _should almost always target an existing feature request_ with evidence of community interest and discussion. This is in order to balance the work of implementing and maintaining new features / enhancements. Pull requests that are opened without meeting this requirement may not be merged.
|
||||
|
||||
## Trust and collaboration
|
||||
|
||||
Paperless-ngx places a high value on trust between contributors and maintainers. If you have not contributed before, consider starting with a discussion before preparing a pull request, especially for a large contribution or one involving AI-generated code.
|
||||
|
||||
Paperless-ngx is complex, and understanding how its components interact takes time. A change that appears correct in isolation can have consequences elsewhere in the application. Contributors should understand the code they submit, explain how it fits into the project, and work with maintainers through review. Starting with smaller, focused contributions can help build that understanding and trust.
|
||||
|
||||
We prefer thoughtful, well-described ideas to AI-generated pull requests. You do not need to supply an implementation for your contribution to be valuable. Explain the problem you have encountered, your use case, the behavior you would like to see, and who would benefit. Concrete examples, alternatives you have considered, and relevant constraints help the community discuss and refine the idea.
|
||||
|
||||
AI tools have lowered the effort needed to produce a pull request, but evaluating one still requires substantial maintainer time. Maintainers still need to establish whether the implementation is correct, fits the project, and can be maintained over time.
|
||||
|
||||
A flawed implementation can also obscure a good idea: discussion becomes focused on problems with the generated code before the community has considered whether the underlying proposal is useful. Starting with the idea gives it a fair discussion on its own merits and lets contributors and maintainers agree on an approach before investing in implementation and review.
|
||||
|
||||
If you want to implement something big:
|
||||
|
||||
- As above, please start with a discussion! Maybe something similar is already in development and we can make it happen together.
|
||||
@@ -61,9 +73,10 @@ This project does not specifically prohibit the use of AI tools or agents _durin
|
||||
|
||||
1. Any content — code, pull request descriptions, issue reports or comments — that was generated in whole or in part by an AI tool or agent **must be clearly disclosed as such**. Failing to clearly disclose this is considered a violation of our [Code of Conduct](CODE_OF_CONDUCT.md) and may result in your contributions being closed and further action being taken.
|
||||
2. AI-generated code must not violate copyright protections.
|
||||
3. We will not accept PRs that are entirely or mostly AI-derived.
|
||||
4. Issue reports must describe _observed behavior only_: what happened, how to reproduce it and the relevant logs. Reports generated by an AI agent should not include code analysis, speculation about the root cause or suggest fixes.
|
||||
5. Contributors are responsible for everything they submit, regardless of how it was produced. Please do not open issues or PRs you have not verified yourself.
|
||||
3. We will not accept PRs opened or submitted by AI agents, including agents acting "on behalf of" a human. A human contributor must personally review, verify, and submit the contribution and take responsibility for it throughout review.
|
||||
4. We will not accept PRs that are entirely or mostly AI-derived.
|
||||
5. Issue reports must describe _observed behavior only_: what happened, how to reproduce it and the relevant logs. Reports generated by an AI agent should not include code analysis, speculation about the root cause or suggest fixes.
|
||||
6. Contributors are responsible for everything they submit, regardless of how it was produced. Please do not open issues or PRs you have not verified yourself.
|
||||
|
||||
# Translating Paperless-ngx
|
||||
|
||||
|
||||
+1
-1
@@ -99,7 +99,7 @@ ARG DEBIAN_FRONTEND=noninteractive
|
||||
ARG TARGETARCH
|
||||
|
||||
# Can be workflow provided, defaults set for manual building
|
||||
ARG JBIG2ENC_VERSION=0.30
|
||||
ARG JBIG2ENC_VERSION=0.32
|
||||
|
||||
# Set Python environment variables
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
|
||||
@@ -565,6 +565,8 @@ Specify `rebuild` to build the index from scratch from all documents in the data
|
||||
this the first time you enable the feature, or after changing the embedding backend or
|
||||
model.
|
||||
|
||||
When using the `huggingface` embedding backend, a model that is already downloaded is loaded from the local cache without contacting the Hugging Face Hub. Only `rebuild` checks the Hub for a newer revision of the model, so a model update is picked up together with re-embedding every document. If the model is not cached yet, it is downloaded on first use.
|
||||
|
||||
Specify `update` to incrementally index new and changed documents. This is what the
|
||||
scheduled task runs.
|
||||
|
||||
|
||||
+22
-11
@@ -501,20 +501,20 @@ See the [Django project documentation on the settings](https://docs.djangoprojec
|
||||
|
||||
#### [`PAPERLESS_ALLOWED_HOSTS=<comma-separated-list>`](#PAPERLESS_ALLOWED_HOSTS) {#PAPERLESS_ALLOWED_HOSTS}
|
||||
|
||||
: If you're planning on putting Paperless on the open internet, then
|
||||
you really should set this value to the domain name you're using.
|
||||
Failing to do so leaves you open to HTTP host header attacks.
|
||||
: If you expose Paperless to the internet, set `PAPERLESS_URL` to its public URL
|
||||
or list its public hostname in `PAPERLESS_ALLOWED_HOSTS` to help protect against
|
||||
HTTP Host header attacks.
|
||||
You can read more about this in [the Django project's documentation](https://docs.djangoproject.com/en/4.1/topics/security/#host-header-validation)
|
||||
|
||||
Just remember that this is a comma-separated list, so
|
||||
"example.com" is fine, as is "example.com,www.example.com", but
|
||||
NOT " example.com" or "example.com,"
|
||||
Separate multiple hostnames with commas, as in `example.com,www.example.com`.
|
||||
|
||||
Can also be set using PAPERLESS_URL (see above).
|
||||
|
||||
"localhost" is always allowed for docker healthcheck
|
||||
|
||||
Defaults to "\*", which is all hosts.
|
||||
- With neither setting configured, Paperless accepts any host.
|
||||
- With only `PAPERLESS_URL` set, Paperless accepts the hostname from that URL
|
||||
and `localhost` (used by the Docker health check).
|
||||
- `PAPERLESS_ALLOWED_HOSTS` adds hosts to `localhost` and the hostname from
|
||||
`PAPERLESS_URL`, if set. A blank list adds no hosts.
|
||||
- With `PAPERLESS_ALLOWED_HOSTS=*`, Paperless accepts any host, even when
|
||||
`PAPERLESS_URL` is set.
|
||||
|
||||
#### [`PAPERLESS_CORS_ALLOWED_HOSTS=<comma-separated-list>`](#PAPERLESS_CORS_ALLOWED_HOSTS) {#PAPERLESS_CORS_ALLOWED_HOSTS}
|
||||
|
||||
@@ -873,6 +873,14 @@ If both the [PAPERLESS_ACCOUNT_DEFAULT_GROUPS](#PAPERLESS_ACCOUNT_DEFAULT_GROUPS
|
||||
|
||||
Defaults to 1209600 (2 weeks)
|
||||
|
||||
#### [`PAPERLESS_SESSION_CLEANUP_TASK_CRON=<cron expression>`](#PAPERLESS_SESSION_CLEANUP_TASK_CRON) {#PAPERLESS_SESSION_CLEANUP_TASK_CRON}
|
||||
|
||||
: Configures the schedule to remove expired sessions from the session store.
|
||||
Django does not prune these automatically, so without this they would
|
||||
accumulate indefinitely.
|
||||
|
||||
Defaults to `0 3 * * *`, once per day at 03:00.
|
||||
|
||||
#### [`PAPERLESS_TOKEN_THROTTLE_RATE=<rate>`](#PAPERLESS_TOKEN_THROTTLE_RATE) {#PAPERLESS_TOKEN_THROTTLE_RATE}
|
||||
|
||||
: Rate limit for the API token authentication endpoint (`/api/token/`), used to mitigate brute-force login attempts.
|
||||
@@ -1149,6 +1157,9 @@ they use underscores instead of dashes.
|
||||
be changed by trusted users. This applies to the `AppConfig` permission as well,
|
||||
which allows setting these arguments from the UI.
|
||||
|
||||
Arguments paperless manages itself (`input_file_or_options`, `output_file`
|
||||
and `sidecar`) are rejected by the API and ignored if set here.
|
||||
|
||||
Specify arguments as a JSON dictionary. Keep note of lower case
|
||||
booleans and double quoted parameter names and strings. Examples:
|
||||
|
||||
|
||||
+1
-82
@@ -109,7 +109,7 @@ Think of versions as **file history** for a document.
|
||||
Paperless-ngx includes management lists for tags, correspondents, document types
|
||||
and more. These areas allow you to view, add, edit, delete and manage permissions
|
||||
for these objects. You can also manage saved views, mail accounts, mail rules,
|
||||
workflows, [OCR templates](#ocr-templates) and more from the management sections.
|
||||
workflows and more from the management sections.
|
||||
|
||||
### Nested Tags
|
||||
|
||||
@@ -795,87 +795,6 @@ The following custom field types are supported:
|
||||
- `Document Link`: reference(s) to other document(s) displayed as links, automatically creates a symmetrical link in reverse
|
||||
- `Select`: a pre-defined list of strings from which the user can choose
|
||||
|
||||
## OCR Templates {#ocr-templates}
|
||||
|
||||
OCR templates read text from fixed areas ("zones") of a document and write it to a
|
||||
[custom field](#custom-fields), or to the document's title, archive serial number or created date. They are
|
||||
useful for documents that always share the same layout, such as bills from a particular utility. They allow you
|
||||
to draw a zone around the account number once, and every new bill gets its account number filled in.
|
||||
|
||||
Like tags or document types, each template has a [matching algorithm](advanced_usage.md#matching) that
|
||||
decides which documents it automatically applies to (or not), based on their content.
|
||||
|
||||
### Creating a template
|
||||
|
||||
1. Open **OCR Templates** under "Manage" and choose **Add Template**.
|
||||
2. Give the template a name and set up its matching, for example **Any word** with the utility's name. Choose
|
||||
**None** for a template that should only ever be run manually.
|
||||
3. Search for a sample document with the layout the template is for. Its pages are shown next to the template
|
||||
settings.
|
||||
4. Draw zones on the document. Click a zone to select and edit it.
|
||||
5. Choose the field each zone fills.
|
||||
6. Use **Test** to run OCR on the selected zone of the sample document and see the text it finds and the value
|
||||
that would be written.
|
||||
|
||||
Zones are positioned relative to the sample document's page, so templates work best when the documents they
|
||||
match have an identical layout. Keep the matching specific enough that it only catches documents with that
|
||||
layout.
|
||||
|
||||
### Zone settings
|
||||
|
||||
- **Page**: the page the zone is on. Turn on **Last page** for zones that are always on the final page,
|
||||
regardless of how many pages a document has.
|
||||
- **OCR language**: the Tesseract languages used to read the zone. Languages other than the defaults must be
|
||||
installed, see [`PAPERLESS_OCR_LANGUAGES`](configuration.md#PAPERLESS_OCR_LANGUAGES).
|
||||
- **Transform**: post-processing applied to the text, such as stripping whitespace, keeping only numbers,
|
||||
parsing a date or reading a QR code or barcode instead of text.
|
||||
- **Date format**: for the "Parse date" transform, the format of the date using
|
||||
[Python date codes](https://docs.python.org/3/library/datetime.html#format-codes), e.g. `%d.%m.%Y`.
|
||||
Leave it empty to detect the format automatically.
|
||||
- **Validation regex**: a regular expression the whole value must match. Values that do not match are not
|
||||
written.
|
||||
|
||||
Zones can fill custom fields of type text, long text, URL, date, integer, number, monetary or boolean.
|
||||
Document link and select fields are not supported.
|
||||
|
||||
### Combining zones
|
||||
|
||||
When several zones fill the same field, a **Combined value** setting appears for that field. Use it to arrange
|
||||
the zones' values with `{Zone name}` tokens and any text in between, for example `{Start} - {End}`. If it
|
||||
is left empty, the values are joined with spaces in zone order.
|
||||
|
||||
### When templates run
|
||||
|
||||
When a document is added, every enabled template that matches its content runs automatically. A template is
|
||||
only used for a document if the document's owner can view the template, so users' templates apply to their own
|
||||
documents, and unowned templates apply to everyone's. Templates run in name order, so if two templates fill the
|
||||
same field, the later one wins.
|
||||
|
||||
Turn off **Enabled** to pause a template without deleting it. Disabled templates never run automatically, but
|
||||
can still be run manually.
|
||||
|
||||
### Running templates manually
|
||||
|
||||
To run templates on documents that are already in Paperless-ngx, open the **Actions** menu on the document
|
||||
details page:
|
||||
|
||||
- **Run Zone OCR** runs every enabled template that matches the document.
|
||||
- The button next to it opens a dialog where you can choose one template to run
|
||||
instead. A template you choose runs even if it does not match the document or is disabled.
|
||||
|
||||
Both actions use the file version selected in document details. The latest version is selected by default.
|
||||
Extracted values update the document's metadata, which is shared across its versions.
|
||||
|
||||
The bulk editor's **Run Zone OCR** action opens the same dialog for the selected documents. Bulk runs use
|
||||
the latest version of each document and are processed in the background.
|
||||
|
||||
!!! warning
|
||||
|
||||
Zone OCR overwrites the fields it fills. A zone that finds no value leaves its field unchanged.
|
||||
|
||||
Managing OCR templates requires the OCR template permissions. Running zone OCR on documents requires
|
||||
permission to change them, and choosing a template requires permission to view it.
|
||||
|
||||
## PDF Actions
|
||||
|
||||
Paperless-ngx supports basic editing operations for PDFs (these operations currently cannot be performed on non-PDF files). When viewing an individual document you can
|
||||
|
||||
+1
-6
@@ -212,12 +212,7 @@ extend-select = [
|
||||
"UP", # https://docs.astral.sh/ruff/rules/#pyupgrade-up
|
||||
"W", # https://docs.astral.sh/ruff/rules/#pycodestyle-e-w
|
||||
]
|
||||
ignore = [
|
||||
"DJ001",
|
||||
"PLC0415",
|
||||
"RUF012",
|
||||
"SIM105",
|
||||
]
|
||||
ignore = [ "DJ001", "ICN001", "PLC0415", "RUF012", "SIM105" ]
|
||||
# Migrations
|
||||
per-file-ignores."*/migrations/*.py" = [
|
||||
"E501",
|
||||
|
||||
+160
-125
@@ -1989,15 +1989,15 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">214</context>
|
||||
<context context-type="linenumber">260</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">332</context>
|
||||
<context context-type="linenumber">378</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">361</context>
|
||||
<context context-type="linenumber">407</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6904866445262015585" datatype="html">
|
||||
@@ -2052,7 +2052,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">58</context>
|
||||
<context context-type="linenumber">60</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/custom-fields-query-dropdown/custom-fields-query-dropdown.component.html</context>
|
||||
@@ -2098,7 +2098,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">224</context>
|
||||
<context context-type="linenumber">227</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="131016739441837046" datatype="html">
|
||||
@@ -2109,7 +2109,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">236</context>
|
||||
<context context-type="linenumber">239</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3643866539136239218" datatype="html">
|
||||
@@ -2178,7 +2178,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">53</context>
|
||||
<context context-type="linenumber">54</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/trash/trash.component.html</context>
|
||||
@@ -2471,11 +2471,11 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">326</context>
|
||||
<context context-type="linenumber">340</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">360</context>
|
||||
<context context-type="linenumber">374</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-detail/document-detail.component.ts</context>
|
||||
@@ -2563,56 +2563,63 @@
|
||||
<source>Result</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">54</context>
|
||||
<context context-type="linenumber">55</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7839508346035554208" datatype="html">
|
||||
<source>Task ID</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">56</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2952127331314675199" datatype="html">
|
||||
<source>Needs attention</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">59</context>
|
||||
<context context-type="linenumber">61</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3307214558906712688" datatype="html">
|
||||
<source>In progress</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">60</context>
|
||||
<context context-type="linenumber">62</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1371178633976505412" datatype="html">
|
||||
<source>Recently completed</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">61</context>
|
||||
<context context-type="linenumber">63</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2936452645673957909" datatype="html">
|
||||
<source>Consume File</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">70</context>
|
||||
<context context-type="linenumber">72</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5622386857946132209" datatype="html">
|
||||
<source>Train Classifier</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">74</context>
|
||||
<context context-type="linenumber">76</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="776309955507105337" datatype="html">
|
||||
<source>Sanity Check</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">78</context>
|
||||
<context context-type="linenumber">80</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7502272564743467653" datatype="html">
|
||||
<source>Mail Fetch</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">80</context>
|
||||
<context context-type="linenumber">82</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.ts</context>
|
||||
@@ -2623,63 +2630,63 @@
|
||||
<source>LLM Index</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">81</context>
|
||||
<context context-type="linenumber">83</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6402092370576716734" datatype="html">
|
||||
<source>Empty Trash</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">84</context>
|
||||
<context context-type="linenumber">86</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4173754487295857311" datatype="html">
|
||||
<source>Check Workflows</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">88</context>
|
||||
<context context-type="linenumber">90</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2065831805515896240" datatype="html">
|
||||
<source>Bulk Update</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">92</context>
|
||||
<context context-type="linenumber">94</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5062348912423871240" datatype="html">
|
||||
<source>Reprocess Document</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">96</context>
|
||||
<context context-type="linenumber">98</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6903696804811148799" datatype="html">
|
||||
<source>Build Share Link</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">100</context>
|
||||
<context context-type="linenumber">102</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6581973658756561124" datatype="html">
|
||||
<source>Bulk Delete</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">104</context>
|
||||
<context context-type="linenumber">106</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5672013191898828327" datatype="html">
|
||||
<source>Apply AI Suggestions</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">108</context>
|
||||
<context context-type="linenumber">110</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="9172233176401579786" datatype="html">
|
||||
<source>Scheduled</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">118</context>
|
||||
<context context-type="linenumber">120</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.ts</context>
|
||||
@@ -2690,7 +2697,7 @@
|
||||
<source>Web UI</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">120</context>
|
||||
<context context-type="linenumber">122</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.ts</context>
|
||||
@@ -2701,7 +2708,7 @@
|
||||
<source>API Upload</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">123</context>
|
||||
<context context-type="linenumber">125</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.ts</context>
|
||||
@@ -2712,106 +2719,106 @@
|
||||
<source>Folder Consume</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">127</context>
|
||||
<context context-type="linenumber">129</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1456901843819961615" datatype="html">
|
||||
<source>Email Consume</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">131</context>
|
||||
<context context-type="linenumber">133</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="29832309535656200" datatype="html">
|
||||
<source>System</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">133</context>
|
||||
<context context-type="linenumber">135</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2233560223291461480" datatype="html">
|
||||
<source>Manual</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">134</context>
|
||||
<context context-type="linenumber">136</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5404910960991552159" datatype="html">
|
||||
<source>Dismiss selected</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">248</context>
|
||||
<context context-type="linenumber">251</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="9169677036332103838" datatype="html">
|
||||
<source>Dismiss visible</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">249</context>
|
||||
<context context-type="linenumber">252</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3169751690815214293" datatype="html">
|
||||
<source>Confirm Dismiss</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">323</context>
|
||||
<context context-type="linenumber">337</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5029621907742319073" datatype="html">
|
||||
<source>Dismiss <x id="PH" equiv-text="tasks.size"/> tasks?</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">324</context>
|
||||
<context context-type="linenumber">338</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3597309129998924778" datatype="html">
|
||||
<source>Error dismissing tasks</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">335</context>
|
||||
<context context-type="linenumber">349</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">369</context>
|
||||
<context context-type="linenumber">383</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2132179171926568807" datatype="html">
|
||||
<source>Error dismissing task</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">347</context>
|
||||
<context context-type="linenumber">361</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1323591410517879795" datatype="html">
|
||||
<source>Confirm Dismiss All</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">357</context>
|
||||
<context context-type="linenumber">371</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4157200209636243740" datatype="html">
|
||||
<source>Dismiss all <x id="PH" equiv-text="this.totalTasks()"/> tasks?</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">358</context>
|
||||
<context context-type="linenumber">372</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8149502458056418229" datatype="html">
|
||||
<source>Success. New document id <x id="PH" equiv-text="documentId"/> created</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">417</context>
|
||||
<context context-type="linenumber">431</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8760066891202884337" datatype="html">
|
||||
<source>Duplicate of document #<x id="PH" equiv-text="duplicateOf"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">427</context>
|
||||
<context context-type="linenumber">441</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
|
||||
<context context-type="linenumber">461</context>
|
||||
<context context-type="linenumber">475</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3418677553313974490" datatype="html">
|
||||
@@ -3409,19 +3416,19 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">954</context>
|
||||
<context context-type="linenumber">976</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">993</context>
|
||||
<context context-type="linenumber">1015</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1016</context>
|
||||
<context context-type="linenumber">1038</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1052</context>
|
||||
<context context-type="linenumber">1074</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/manage/document-attributes/custom-fields/custom-fields.component.ts</context>
|
||||
@@ -3764,25 +3771,46 @@
|
||||
<context context-type="linenumber">418</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3562016764120788499" datatype="html">
|
||||
<source>Bulk operation completed.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">202</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1633341269280627980" datatype="html">
|
||||
<source>Bulk operation revoked.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">204</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8606600295919857288" datatype="html">
|
||||
<source>Bulk operation failed. Some changes may already have been applied.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">207</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1542489069631984294" datatype="html">
|
||||
<source>Sidebar views updated</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">497</context>
|
||||
<context context-type="linenumber">543</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3547923076537026828" datatype="html">
|
||||
<source>Error updating sidebar views</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">500</context>
|
||||
<context context-type="linenumber">546</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2526035785704676448" datatype="html">
|
||||
<source>An error occurred while saving update checking settings.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/app-frame/app-frame.component.ts</context>
|
||||
<context context-type="linenumber">521</context>
|
||||
<context context-type="linenumber">567</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4580988005648117665" datatype="html">
|
||||
@@ -4163,27 +4191,27 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">572</context>
|
||||
<context context-type="linenumber">594</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">612</context>
|
||||
<context context-type="linenumber">634</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">650</context>
|
||||
<context context-type="linenumber">672</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">688</context>
|
||||
<context context-type="linenumber">710</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">750</context>
|
||||
<context context-type="linenumber">772</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">888</context>
|
||||
<context context-type="linenumber">910</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6088768251338567957" datatype="html">
|
||||
@@ -6586,7 +6614,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">892</context>
|
||||
<context context-type="linenumber">914</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1616074213645042277" datatype="html">
|
||||
@@ -8363,7 +8391,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">517</context>
|
||||
<context context-type="linenumber">539</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">this string is used to separate processing, failed and added on the file upload widget</note>
|
||||
</trans-unit>
|
||||
@@ -9160,7 +9188,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">890</context>
|
||||
<context context-type="linenumber">912</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5992101894268263750" datatype="html">
|
||||
@@ -9185,7 +9213,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">950</context>
|
||||
<context context-type="linenumber">972</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2951161989614003846" datatype="html">
|
||||
@@ -9571,7 +9599,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1050</context>
|
||||
<context context-type="linenumber">1072</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7251533806673754434" datatype="html">
|
||||
@@ -9602,29 +9630,43 @@
|
||||
<context context-type="linenumber">165,166</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3401633333450192408" datatype="html">
|
||||
<source>Bulk operation queued.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">315</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5709555629190115111" datatype="html">
|
||||
<source>View task</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">319</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1215215387232313677" datatype="html">
|
||||
<source>Error executing bulk operation</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">349</context>
|
||||
<context context-type="linenumber">371</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7894972847287473517" datatype="html">
|
||||
<source>"<x id="PH" equiv-text="items[0].name"/>"</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">509</context>
|
||||
<context context-type="linenumber">531</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">515</context>
|
||||
<context context-type="linenumber">537</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8639884465898458690" datatype="html">
|
||||
<source>"<x id="PH" equiv-text="items[0].name"/>" and "<x id="PH_1" equiv-text="items[1].name"/>"</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">511</context>
|
||||
<context context-type="linenumber">533</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">This is for messages like 'modify "tag1" and "tag2"'</note>
|
||||
</trans-unit>
|
||||
@@ -9632,7 +9674,7 @@
|
||||
<source><x id="PH" equiv-text="list"/> and "<x id="PH_1" equiv-text="items[items.length - 1].name"/>"</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">519,521</context>
|
||||
<context context-type="linenumber">541,543</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">this is for messages like 'modify "tag1", "tag2" and "tag3"'</note>
|
||||
</trans-unit>
|
||||
@@ -9640,14 +9682,14 @@
|
||||
<source>Confirm tags assignment</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">536</context>
|
||||
<context context-type="linenumber">558</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6619516195038467207" datatype="html">
|
||||
<source>This operation will add the tag "<x id="PH" equiv-text="tag.name"/>" to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">542</context>
|
||||
<context context-type="linenumber">564</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1894412783609570695" datatype="html">
|
||||
@@ -9656,14 +9698,14 @@
|
||||
)"/> to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">547,549</context>
|
||||
<context context-type="linenumber">569,571</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7181166515756808573" datatype="html">
|
||||
<source>This operation will remove the tag "<x id="PH" equiv-text="tag.name"/>" from <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">555</context>
|
||||
<context context-type="linenumber">577</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3819792277998068944" datatype="html">
|
||||
@@ -9672,7 +9714,7 @@
|
||||
)"/> from <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">560,562</context>
|
||||
<context context-type="linenumber">582,584</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2739066218579571288" datatype="html">
|
||||
@@ -9683,84 +9725,84 @@
|
||||
)"/> on <x id="PH_2" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">564,568</context>
|
||||
<context context-type="linenumber">586,590</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2996713129519325161" datatype="html">
|
||||
<source>Confirm correspondent assignment</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">605</context>
|
||||
<context context-type="linenumber">627</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6900893559485781849" datatype="html">
|
||||
<source>This operation will assign the correspondent "<x id="PH" equiv-text="correspondent.name"/>" to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">607</context>
|
||||
<context context-type="linenumber">629</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1257522660364398440" datatype="html">
|
||||
<source>This operation will remove the correspondent from <x id="PH" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">609</context>
|
||||
<context context-type="linenumber">631</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5393409374423140648" datatype="html">
|
||||
<source>Confirm document type assignment</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">643</context>
|
||||
<context context-type="linenumber">665</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="332180123895325027" datatype="html">
|
||||
<source>This operation will assign the document type "<x id="PH" equiv-text="documentType.name"/>" to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">645</context>
|
||||
<context context-type="linenumber">667</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2236642492594872779" datatype="html">
|
||||
<source>This operation will remove the document type from <x id="PH" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">647</context>
|
||||
<context context-type="linenumber">669</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6386555513013840736" datatype="html">
|
||||
<source>Confirm storage path assignment</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">681</context>
|
||||
<context context-type="linenumber">703</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8750527458618415924" datatype="html">
|
||||
<source>This operation will assign the storage path "<x id="PH" equiv-text="storagePath.name"/>" to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">683</context>
|
||||
<context context-type="linenumber">705</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="60728365335056946" datatype="html">
|
||||
<source>This operation will remove the storage path from <x id="PH" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">685</context>
|
||||
<context context-type="linenumber">707</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4187352575310415704" datatype="html">
|
||||
<source>Confirm custom field assignment</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">714</context>
|
||||
<context context-type="linenumber">736</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7966494636326273856" datatype="html">
|
||||
<source>This operation will assign the custom field "<x id="PH" equiv-text="customField.name"/>" to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">720</context>
|
||||
<context context-type="linenumber">742</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5789455969634598553" datatype="html">
|
||||
@@ -9769,14 +9811,14 @@
|
||||
)"/> to <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">725,727</context>
|
||||
<context context-type="linenumber">747,749</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5648572354333199245" datatype="html">
|
||||
<source>This operation will remove the custom field "<x id="PH" equiv-text="customField.name"/>" from <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">733</context>
|
||||
<context context-type="linenumber">755</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6666899594015948817" datatype="html">
|
||||
@@ -9785,7 +9827,7 @@
|
||||
)"/> from <x id="PH_1" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">738,740</context>
|
||||
<context context-type="linenumber">760,762</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8050047262594964176" datatype="html">
|
||||
@@ -9796,112 +9838,105 @@
|
||||
)"/> on <x id="PH_2" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">742,746</context>
|
||||
<context context-type="linenumber">764,768</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8615059324209654051" datatype="html">
|
||||
<source>Move <x id="PH" equiv-text="this.getSelectionSize()"/> selected document(s) to the trash?</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">889</context>
|
||||
<context context-type="linenumber">911</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="135114644550399974" datatype="html">
|
||||
<source>Moved <x id="PH" equiv-text="count"/> document(s) to the trash</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">916</context>
|
||||
<context context-type="linenumber">938</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8585195717323764335" datatype="html">
|
||||
<source>This operation will permanently recreate the archive files for <x id="PH" equiv-text="this.getSelectionSize()"/> selected document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">951</context>
|
||||
<context context-type="linenumber">973</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7366623494074776040" datatype="html">
|
||||
<source>The archive files will be re-generated with the current settings.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">952</context>
|
||||
<context context-type="linenumber">974</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6555329262222566158" datatype="html">
|
||||
<source>Rotate confirm</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">990</context>
|
||||
<context context-type="linenumber">1012</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5203024009814367559" datatype="html">
|
||||
<source>This operation will add rotated versions of the <x id="PH" equiv-text="this.getSelectionSize()"/> document(s).</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">991</context>
|
||||
<context context-type="linenumber">1013</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7910756456450124185" datatype="html">
|
||||
<source>Merge confirm</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1014</context>
|
||||
<context context-type="linenumber">1036</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7643543647233874431" datatype="html">
|
||||
<source>This operation will merge <x id="PH" equiv-text="this.getSelectionSize()"/> selected documents into a new document.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1015</context>
|
||||
<context context-type="linenumber">1037</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7869008840945899895" datatype="html">
|
||||
<source>Merged document will be queued for consumption.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1038</context>
|
||||
<context context-type="linenumber">1060</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1549333914563613027" datatype="html">
|
||||
<source>The selected documents will become versions of the root document.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1051</context>
|
||||
<context context-type="linenumber">1073</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7099667366799629283" datatype="html">
|
||||
<source>Documents merged as versions.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1066</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="476913782630693351" datatype="html">
|
||||
<source>Custom fields updated.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1092</context>
|
||||
<context context-type="linenumber">1088</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3873496751167944011" datatype="html">
|
||||
<source>Error updating custom fields.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1101</context>
|
||||
<context context-type="linenumber">1118</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6144801143088984138" datatype="html">
|
||||
<source>Share link bundle creation requested.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1149</context>
|
||||
<context context-type="linenumber">1166</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="46019676931295023" datatype="html">
|
||||
<source>Share link bundle creation is not available yet.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
|
||||
<context context-type="linenumber">1156</context>
|
||||
<context context-type="linenumber">1173</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6307402210351946694" datatype="html">
|
||||
@@ -13369,42 +13404,42 @@
|
||||
<source>Document already exists.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">27</context>
|
||||
<context context-type="linenumber">29</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5103087968344279314" datatype="html">
|
||||
<source>Document already exists. Note: existing document is in the trash.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">28</context>
|
||||
<context context-type="linenumber">30</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6108404046106249255" datatype="html">
|
||||
<source>Document with ASN already exists.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">29</context>
|
||||
<context context-type="linenumber">31</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="65951081560571094" datatype="html">
|
||||
<source>Document with ASN already exists. Note: existing document is in the trash.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">30</context>
|
||||
<context context-type="linenumber">32</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="148389968432135849" datatype="html">
|
||||
<source>File not found.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">31</context>
|
||||
<context context-type="linenumber">33</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1520671543092565667" datatype="html">
|
||||
<source>Pre-consume script does not exist.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">32</context>
|
||||
<context context-type="linenumber">34</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">Pre-Consume is a term that appears like that in the documentation as well and does not need a specific translation</note>
|
||||
</trans-unit>
|
||||
@@ -13412,7 +13447,7 @@
|
||||
<source>Error while executing pre-consume script.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">33</context>
|
||||
<context context-type="linenumber">35</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">Pre-Consume is a term that appears like that in the documentation as well and does not need a specific translation</note>
|
||||
</trans-unit>
|
||||
@@ -13420,7 +13455,7 @@
|
||||
<source>Post-consume script does not exist.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">34</context>
|
||||
<context context-type="linenumber">36</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">Post-Consume is a term that appears like that in the documentation as well and does not need a specific translation</note>
|
||||
</trans-unit>
|
||||
@@ -13428,7 +13463,7 @@
|
||||
<source>Error while executing post-consume script.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">35</context>
|
||||
<context context-type="linenumber">37</context>
|
||||
</context-group>
|
||||
<note priority="1" from="description">Post-Consume is a term that appears like that in the documentation as well and does not need a specific translation</note>
|
||||
</trans-unit>
|
||||
@@ -13436,49 +13471,49 @@
|
||||
<source>Received new file.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">36</context>
|
||||
<context context-type="linenumber">38</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7337565919209746135" datatype="html">
|
||||
<source>File type not supported.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">37</context>
|
||||
<context context-type="linenumber">39</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5002399167376099234" datatype="html">
|
||||
<source>Processing document...</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">38</context>
|
||||
<context context-type="linenumber">40</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1085975194762600381" datatype="html">
|
||||
<source>Generating thumbnail...</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">39</context>
|
||||
<context context-type="linenumber">41</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3280851677698431426" datatype="html">
|
||||
<source>Retrieving date from document...</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">40</context>
|
||||
<context context-type="linenumber">42</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7162102384876037296" datatype="html">
|
||||
<source>Saving document...</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">41</context>
|
||||
<context context-type="linenumber">43</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4550450765009165976" datatype="html">
|
||||
<source>Finished.</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/services/websocket-status.service.ts</context>
|
||||
<context context-type="linenumber">42</context>
|
||||
<context context-type="linenumber">44</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2501522447884928778" datatype="html">
|
||||
|
||||
@@ -13,7 +13,6 @@ import { DocumentDetailComponent } from './components/document-detail/document-d
|
||||
import { DocumentListComponent } from './components/document-list/document-list.component'
|
||||
import { DocumentAttributesComponent } from './components/manage/document-attributes/document-attributes.component'
|
||||
import { MailComponent } from './components/manage/mail/mail.component'
|
||||
import { OcrTemplatesComponent } from './components/manage/ocr-templates/ocr-templates.component'
|
||||
import { SavedViewsComponent } from './components/manage/saved-views/saved-views.component'
|
||||
import { ShareLinksComponent } from './components/manage/share-links/share-links.component'
|
||||
import { WorkflowsComponent } from './components/manage/workflows/workflows.component'
|
||||
@@ -276,18 +275,6 @@ export const routes: Routes = [
|
||||
componentName: 'WorkflowsComponent',
|
||||
},
|
||||
},
|
||||
{
|
||||
path: 'ocr-templates',
|
||||
component: OcrTemplatesComponent,
|
||||
canActivate: [PermissionsGuard],
|
||||
data: {
|
||||
requiredPermission: {
|
||||
action: PermissionAction.View,
|
||||
type: PermissionType.OcrTemplate,
|
||||
},
|
||||
componentName: 'OcrTemplatesComponent',
|
||||
},
|
||||
},
|
||||
{
|
||||
path: 'mail',
|
||||
component: MailComponent,
|
||||
|
||||
@@ -113,7 +113,6 @@ const sidebarItemLabels: Record<HideableSidebarItemID, string> = {
|
||||
[HideableSidebarItemID.SavedViews]: $localize`Saved Views`,
|
||||
[HideableSidebarItemID.ShareLinks]: $localize`Share Links`,
|
||||
[HideableSidebarItemID.Workflows]: $localize`Workflows`,
|
||||
[HideableSidebarItemID.OcrTemplates]: $localize`OCR Templates`,
|
||||
[HideableSidebarItemID.Mail]: $localize`Mail`,
|
||||
[HideableSidebarItemID.Documentation]: $localize`Documentation`,
|
||||
}
|
||||
|
||||
@@ -7,10 +7,15 @@ import {
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { FormsModule } from '@angular/forms'
|
||||
import { By } from '@angular/platform-browser'
|
||||
import { Router } from '@angular/router'
|
||||
import {
|
||||
ActivatedRoute,
|
||||
convertToParamMap,
|
||||
ParamMap,
|
||||
Router,
|
||||
} from '@angular/router'
|
||||
import { RouterTestingModule } from '@angular/router/testing'
|
||||
import { NgbModal, NgbModalRef, NgbModule } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { of, throwError } from 'rxjs'
|
||||
import { BehaviorSubject, of, throwError } from 'rxjs'
|
||||
import { routes } from 'src/app/app-routing.module'
|
||||
import {
|
||||
PaperlessTask,
|
||||
@@ -159,6 +164,7 @@ const sectionCountResponse = {
|
||||
}
|
||||
|
||||
describe('TasksComponent', () => {
|
||||
let queryParams: BehaviorSubject<ParamMap>
|
||||
let component: TasksComponent
|
||||
let fixture: ComponentFixture<TasksComponent>
|
||||
let tasksService: TasksService
|
||||
@@ -169,6 +175,7 @@ describe('TasksComponent', () => {
|
||||
let toastService: ToastService
|
||||
|
||||
beforeEach(async () => {
|
||||
queryParams = new BehaviorSubject(convertToParamMap({}))
|
||||
TestBed.configureTestingModule({
|
||||
imports: [
|
||||
NgbModule,
|
||||
@@ -181,6 +188,17 @@ describe('TasksComponent', () => {
|
||||
ConfirmDialogComponent,
|
||||
],
|
||||
providers: [
|
||||
{
|
||||
provide: ActivatedRoute,
|
||||
useValue: {
|
||||
queryParamMap: queryParams.asObservable(),
|
||||
snapshot: {
|
||||
get queryParamMap() {
|
||||
return queryParams.value
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: PermissionsService,
|
||||
useValue: {
|
||||
@@ -219,7 +237,11 @@ describe('TasksComponent', () => {
|
||||
})
|
||||
|
||||
// answers task list requests with the fixture tasks matching their status filter
|
||||
function flushTaskLists(pageSize: number, count?: number) {
|
||||
function flushTaskLists(
|
||||
pageSize: number,
|
||||
count?: number,
|
||||
filteredTasks: PaperlessTask[] = tasks
|
||||
) {
|
||||
httpTestingController
|
||||
.match(
|
||||
(req) =>
|
||||
@@ -228,7 +250,9 @@ describe('TasksComponent', () => {
|
||||
)
|
||||
.forEach((req) => {
|
||||
const statuses = req.request.params.getAll('status')
|
||||
const results = tasks.filter((task) => statuses.includes(task.status))
|
||||
const results = filteredTasks.filter((task) =>
|
||||
statuses.includes(task.status)
|
||||
)
|
||||
req.flush({ count: count ?? results.length, results })
|
||||
})
|
||||
}
|
||||
@@ -238,6 +262,71 @@ describe('TasksComponent', () => {
|
||||
flushTaskLists(25, count)
|
||||
}
|
||||
|
||||
it('consumes a task link on load into the normal resettable Task ID filter', () => {
|
||||
jest
|
||||
.spyOn(tasksService, 'list')
|
||||
.mockReturnValue(of({ count: 0, results: [] }))
|
||||
jest
|
||||
.spyOn(tasksService, 'statusCounts')
|
||||
.mockReturnValue(of(sectionCountResponse))
|
||||
jest.spyOn(router, 'navigate').mockResolvedValue(true)
|
||||
queryParams.next(convertToParamMap({ task_id: 'linked-task' }))
|
||||
fixture.destroy()
|
||||
fixture = TestBed.createComponent(TasksComponent)
|
||||
component = fixture.componentInstance
|
||||
fixture.detectChanges()
|
||||
expect(router.navigate).toHaveBeenCalledTimes(1)
|
||||
expect(router.navigate).toHaveBeenCalledWith(
|
||||
[],
|
||||
expect.objectContaining({
|
||||
queryParams: { task_id: null },
|
||||
replaceUrl: true,
|
||||
})
|
||||
)
|
||||
expect(component.filterTargetName).toBe('Task ID')
|
||||
expect(component.filterText).toBe('linked-task')
|
||||
expect(component.isFiltered).toBe(true)
|
||||
component.setSection(TaskSection.InProgress)
|
||||
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
|
||||
task_id: 'linked-task',
|
||||
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
|
||||
})
|
||||
component.resetFilters()
|
||||
expect(component.filterText).toBe('')
|
||||
expect(component.isFiltered).toBe(false)
|
||||
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
|
||||
acknowledged: false,
|
||||
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
|
||||
})
|
||||
expect(router.navigate).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('supports manually filtering by Task ID and clearing the text filter', () => {
|
||||
jest
|
||||
.spyOn(tasksService, 'list')
|
||||
.mockReturnValue(of({ count: 0, results: [] }))
|
||||
jest
|
||||
.spyOn(tasksService, 'statusCounts')
|
||||
.mockReturnValue(of(sectionCountResponse))
|
||||
component.setSection(TaskSection.InProgress)
|
||||
component.setFilterTarget(TaskFilterTargetID.TaskId)
|
||||
component.filterText = tasks[0].task_id
|
||||
jest.advanceTimersByTime(150)
|
||||
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
|
||||
task_id: tasks[0].task_id,
|
||||
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
|
||||
})
|
||||
component.pagedTasks.set(tasks)
|
||||
expect(component.tasksForSection(TaskSection.NeedsAttention)).toEqual([
|
||||
tasks[0],
|
||||
])
|
||||
component.resetFilter()
|
||||
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
|
||||
acknowledged: false,
|
||||
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
|
||||
})
|
||||
})
|
||||
|
||||
it('should display task sections with counts', () => {
|
||||
expect(component.selectedSection).toBe(TaskSection.All)
|
||||
expect(component.selectedTaskType).toBeNull()
|
||||
@@ -771,6 +860,7 @@ describe('TasksComponent', () => {
|
||||
input.nativeElement.value = '191092'
|
||||
input.nativeElement.dispatchEvent(new Event('input'))
|
||||
jest.advanceTimersByTime(150)
|
||||
flushTaskLists(10, undefined, [tasks[1]])
|
||||
fixture.detectChanges()
|
||||
expect(component.filterText).toEqual('191092')
|
||||
expect(component.tasksForSection(TaskSection.NeedsAttention)).toHaveLength(
|
||||
@@ -782,6 +872,7 @@ describe('TasksComponent', () => {
|
||||
component.setSection(TaskSection.InProgress)
|
||||
component.filterText = 'system'
|
||||
jest.advanceTimersByTime(150)
|
||||
flushTaskLists(25, undefined, [tasks[6]])
|
||||
|
||||
expect(component.tasksForSection(TaskSection.InProgress)).toHaveLength(1)
|
||||
expect(component.tasksForSection(TaskSection.InProgress)[0].task_type).toBe(
|
||||
@@ -815,6 +906,7 @@ describe('TasksComponent', () => {
|
||||
input.nativeElement.value = 'duplicate'
|
||||
input.nativeElement.dispatchEvent(new Event('input'))
|
||||
jest.advanceTimersByTime(150)
|
||||
flushTaskLists(25, undefined, [tasks[0], tasks[1]])
|
||||
fixture.detectChanges()
|
||||
expect(component.filterText).toEqual('duplicate')
|
||||
expect(component.tasksForSection(TaskSection.NeedsAttention)).toHaveLength(
|
||||
@@ -822,6 +914,53 @@ describe('TasksComponent', () => {
|
||||
)
|
||||
})
|
||||
|
||||
it('should show server-matched names when the displayed source is translated', () => {
|
||||
component.setSection(TaskSection.InProgress)
|
||||
flushTaskLists(25)
|
||||
component.filterText = 'email'
|
||||
jest.advanceTimersByTime(150)
|
||||
|
||||
const request = httpTestingController.expectOne(
|
||||
(req) =>
|
||||
req.url === `${environment.apiBaseUrl}tasks/` &&
|
||||
req.params.get('name') === 'email' &&
|
||||
req.params.get('page_size') === '25'
|
||||
)
|
||||
const translatedTask = {
|
||||
...tasks[5],
|
||||
trigger_source_display: 'E-Mail',
|
||||
}
|
||||
request.flush({ count: 1, results: [translatedTask] })
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(component.totalTasks()).toBe(1)
|
||||
expect(component.visibleTasks).toEqual([translatedTask])
|
||||
})
|
||||
|
||||
it('should show server-matched duplicates when the displayed result is translated', () => {
|
||||
component.setSection(TaskSection.NeedsAttention)
|
||||
flushTaskLists(25)
|
||||
jest
|
||||
.spyOn(component, 'taskResultMessage')
|
||||
.mockReturnValue('Duplikat von Document #311')
|
||||
component.setFilterTarget(TaskFilterTargetID.Result)
|
||||
component.filterText = 'duplicate'
|
||||
jest.advanceTimersByTime(150)
|
||||
|
||||
const request = httpTestingController.expectOne(
|
||||
(req) =>
|
||||
req.url === `${environment.apiBaseUrl}tasks/` &&
|
||||
req.params.get('result') === 'duplicate' &&
|
||||
req.params.get('page_size') === '25'
|
||||
)
|
||||
request.flush({ count: 1, results: [tasks[1]] })
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(component.totalTasks()).toBe(1)
|
||||
expect(component.visibleTasks).toEqual([tasks[1]])
|
||||
expect(fixture.nativeElement.textContent).toContain('191092.pdf')
|
||||
})
|
||||
|
||||
it('should prefer explicit reason in the result message', () => {
|
||||
expect(
|
||||
component.taskResultMessage({
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { JsonPipe, NgTemplateOutlet } from '@angular/common'
|
||||
import { Component, inject, OnDestroy, OnInit, signal } from '@angular/core'
|
||||
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
|
||||
import { Router, RouterLink } from '@angular/router'
|
||||
import { ActivatedRoute, Router, RouterLink } from '@angular/router'
|
||||
import {
|
||||
NgbCollapseModule,
|
||||
NgbDropdownModule,
|
||||
@@ -47,11 +47,13 @@ export enum TaskSection {
|
||||
export enum TaskFilterTargetID {
|
||||
Name,
|
||||
Result,
|
||||
TaskId,
|
||||
}
|
||||
|
||||
const FILTER_TARGETS = [
|
||||
{ id: TaskFilterTargetID.Name, name: $localize`Name` },
|
||||
{ id: TaskFilterTargetID.Result, name: $localize`Result` },
|
||||
{ id: TaskFilterTargetID.TaskId, name: $localize`Task ID` },
|
||||
]
|
||||
|
||||
const SECTION_LABELS = {
|
||||
@@ -161,6 +163,7 @@ export class TasksComponent
|
||||
tasksService = inject(TasksService)
|
||||
private modalService = inject(NgbModal)
|
||||
private readonly router = inject(Router)
|
||||
private readonly route = inject(ActivatedRoute)
|
||||
private readonly toastService = inject(ToastService)
|
||||
|
||||
readonly TaskSection = TaskSection
|
||||
@@ -275,6 +278,17 @@ export class TasksComponent
|
||||
}
|
||||
|
||||
ngOnInit() {
|
||||
const taskId = this.route.snapshot.queryParamMap.get('task_id')
|
||||
if (taskId) {
|
||||
this.filterTargetID = TaskFilterTargetID.TaskId
|
||||
this._filterText = taskId
|
||||
this.router.navigate([], {
|
||||
relativeTo: this.route,
|
||||
queryParams: { task_id: null },
|
||||
queryParamsHandling: 'merge',
|
||||
replaceUrl: true,
|
||||
})
|
||||
}
|
||||
this.tasksService.reload()
|
||||
this.reloadPage()
|
||||
timer(5000, 5000)
|
||||
@@ -636,17 +650,12 @@ export class TasksComponent
|
||||
|
||||
const query = this._filterText.toLowerCase()
|
||||
|
||||
if (this.filterTargetID == TaskFilterTargetID.Name) {
|
||||
return [
|
||||
this.taskDisplayName(task),
|
||||
task.task_type_display,
|
||||
task.trigger_source_display,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.some((value) => value.toLowerCase().includes(query))
|
||||
if (this.filterTargetID === TaskFilterTargetID.TaskId) {
|
||||
return task.task_id.toLowerCase() === query
|
||||
}
|
||||
|
||||
return this.taskResultMessage(task)?.toLowerCase().includes(query) ?? false
|
||||
// Text filtering now handled API-side
|
||||
return true
|
||||
}
|
||||
|
||||
private tasksForOptionCounts({
|
||||
@@ -706,9 +715,15 @@ export class TasksComponent
|
||||
}
|
||||
|
||||
if (this._filterText.length) {
|
||||
params[
|
||||
this.filterTargetID === TaskFilterTargetID.Name ? 'name' : 'result'
|
||||
] = this._filterText
|
||||
const parameter = {
|
||||
[TaskFilterTargetID.Name]: 'name',
|
||||
[TaskFilterTargetID.Result]: 'result',
|
||||
[TaskFilterTargetID.TaskId]: 'task_id',
|
||||
}[this.filterTargetID]
|
||||
params[parameter] = this._filterText
|
||||
if (this.filterTargetID === TaskFilterTargetID.TaskId) {
|
||||
delete params.acknowledged
|
||||
}
|
||||
}
|
||||
|
||||
return params
|
||||
|
||||
@@ -289,17 +289,6 @@
|
||||
<pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Workflows" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.Workflows, $event)"></pngx-input-switch>
|
||||
}
|
||||
</li>
|
||||
<li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.OcrTemplates) && !settingsService.organizingSidebarItems()"
|
||||
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.OcrTemplate }">
|
||||
<a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.OcrTemplates)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="ocr-templates" routerLinkActive="active" (click)="closeMenu()"
|
||||
ngbPopover="OCR Templates" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
|
||||
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
|
||||
<pngx-bs-icon class="me-2" name="bounding-box-circles"></pngx-bs-icon><span class="nav-link-label"><ng-container i18n>OCR Templates</ng-container></span>
|
||||
</a>
|
||||
@if (settingsService.organizingSidebarItems()) {
|
||||
<pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="OCR Templates" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.OcrTemplates)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.OcrTemplates, $event)"></pngx-input-switch>
|
||||
}
|
||||
</li>
|
||||
<li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail) && !settingsService.organizingSidebarItems()" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
|
||||
tourAnchor="tour.mail">
|
||||
<a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
|
||||
|
||||
@@ -11,8 +11,10 @@ import { ActivatedRoute, Router } from '@angular/router'
|
||||
import { RouterTestingModule } from '@angular/router/testing'
|
||||
import { NgbModal, NgbModalModule, NgbModule } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { provideUiTour } from 'ngx-ui-tour-ng-bootstrap'
|
||||
import { of, throwError } from 'rxjs'
|
||||
import { of, Subject, throwError } from 'rxjs'
|
||||
import { routes } from 'src/app/app-routing.module'
|
||||
import { BulkOperationStatus } from 'src/app/data/bulk-operation'
|
||||
import { PaperlessTaskStatus } from 'src/app/data/paperless-task'
|
||||
import { SavedView } from 'src/app/data/saved-view'
|
||||
import { HideableSidebarItemID, SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
|
||||
@@ -21,10 +23,11 @@ import {
|
||||
DjangoMessageLevel,
|
||||
DjangoMessagesService,
|
||||
} from 'src/app/services/django-messages.service'
|
||||
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
|
||||
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
|
||||
import {
|
||||
PermissionType,
|
||||
PermissionsService,
|
||||
PermissionType,
|
||||
} from 'src/app/services/permissions.service'
|
||||
import { RemoteVersionService } from 'src/app/services/rest/remote-version.service'
|
||||
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
|
||||
@@ -95,6 +98,7 @@ describe('AppFrameComponent', () => {
|
||||
let savedViewSpy
|
||||
let modalService: NgbModal
|
||||
let maybeRefreshSpy
|
||||
let bulkEvents: Subject<BulkOperationStatus>
|
||||
let tasksService: TasksService
|
||||
|
||||
beforeEach(async () => {
|
||||
@@ -182,6 +186,10 @@ describe('AppFrameComponent', () => {
|
||||
savedViewSpy = jest.spyOn(savedViewService, 'reload')
|
||||
maybeRefreshSpy = jest.spyOn(savedViewService, 'maybeRefreshDocumentCounts')
|
||||
|
||||
bulkEvents = new Subject<BulkOperationStatus>()
|
||||
jest
|
||||
.spyOn(TestBed.inject(WebsocketStatusService), 'onBulkOperationStatus')
|
||||
.mockReturnValue(bulkEvents.asObservable())
|
||||
fixture = TestBed.createComponent(AppFrameComponent)
|
||||
component = fixture.componentInstance
|
||||
|
||||
@@ -190,6 +198,78 @@ describe('AppFrameComponent', () => {
|
||||
fixture.detectChanges()
|
||||
})
|
||||
|
||||
it('refreshes and notifies on completion without client-side task tracking', () => {
|
||||
const list = TestBed.inject(DocumentListViewService)
|
||||
const reload = jest.spyOn(list, 'reload').mockImplementation(() => {})
|
||||
jest.spyOn(list, 'reduceSelectionToFilter').mockImplementation(() => {})
|
||||
const tasksReload = jest
|
||||
.spyOn(tasksService, 'reload')
|
||||
.mockImplementation(() => {})
|
||||
const success = jest.spyOn(toastService, 'showSuccess')
|
||||
const refresh = jest
|
||||
.spyOn(openDocumentsService, 'refreshDocument')
|
||||
.mockImplementation(() => {})
|
||||
jest
|
||||
.spyOn(openDocumentsService, 'getOpenDocuments')
|
||||
.mockReturnValue([document])
|
||||
tasksReload.mockClear()
|
||||
maybeRefreshSpy.mockClear()
|
||||
bulkEvents.next({
|
||||
task_id: 'bulk-task',
|
||||
operation: 'set_permissions',
|
||||
status: PaperlessTaskStatus.Started,
|
||||
})
|
||||
expect(reload).not.toHaveBeenCalled()
|
||||
bulkEvents.next({
|
||||
task_id: 'bulk-task',
|
||||
operation: 'set_permissions',
|
||||
status: PaperlessTaskStatus.Success,
|
||||
})
|
||||
expect(reload).toHaveBeenCalledTimes(1)
|
||||
expect(reload).toHaveBeenCalledWith(null, false)
|
||||
expect(refresh).toHaveBeenCalledWith(document.id)
|
||||
expect(maybeRefreshSpy).toHaveBeenCalledTimes(1)
|
||||
expect(tasksReload).toHaveBeenCalledTimes(1)
|
||||
expect(success).toHaveBeenCalledWith('Bulk operation completed.')
|
||||
fixture.destroy()
|
||||
bulkEvents.next({
|
||||
task_id: 'bulk-task',
|
||||
operation: 'set_permissions',
|
||||
status: PaperlessTaskStatus.Success,
|
||||
})
|
||||
expect(reload).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('refreshes and reports failed or revoked bulk operations', () => {
|
||||
const list = TestBed.inject(DocumentListViewService)
|
||||
const reload = jest.spyOn(list, 'reload').mockImplementation(() => {})
|
||||
jest.spyOn(list, 'reduceSelectionToFilter').mockImplementation(() => {})
|
||||
jest.spyOn(tasksService, 'reload').mockImplementation(() => {})
|
||||
const error = jest.spyOn(toastService, 'showError')
|
||||
const success = jest.spyOn(toastService, 'showSuccess')
|
||||
for (const status of [
|
||||
PaperlessTaskStatus.Failure,
|
||||
PaperlessTaskStatus.Revoked,
|
||||
]) {
|
||||
bulkEvents.next({
|
||||
task_id: 'bulk-task',
|
||||
operation: 'set_permissions',
|
||||
status,
|
||||
error:
|
||||
status === PaperlessTaskStatus.Revoked
|
||||
? 'Task expired before execution.'
|
||||
: 'Could not update',
|
||||
})
|
||||
}
|
||||
expect(reload).toHaveBeenCalledTimes(2)
|
||||
expect(error).toHaveBeenCalledWith(expect.any(String), 'Could not update')
|
||||
expect(error).toHaveBeenCalledWith(
|
||||
'Bulk operation revoked.',
|
||||
'Task expired before execution.'
|
||||
)
|
||||
expect(success).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should initialize the saved view service', () => {
|
||||
expect(savedViewSpy).toHaveBeenCalled()
|
||||
})
|
||||
@@ -293,7 +373,6 @@ describe('AppFrameComponent', () => {
|
||||
HideableSidebarItemID.Dashboard,
|
||||
HideableSidebarItemID.Workflows,
|
||||
HideableSidebarItemID.ShareLinks,
|
||||
HideableSidebarItemID.OcrTemplates,
|
||||
])
|
||||
fixture.detectChanges()
|
||||
|
||||
@@ -309,10 +388,6 @@ describe('AppFrameComponent', () => {
|
||||
fixture.nativeElement.querySelector('[routerLink="share-links"]')
|
||||
.parentElement.classList
|
||||
).toContain('d-none')
|
||||
expect(
|
||||
fixture.nativeElement.querySelector('[routerLink="ocr-templates"]')
|
||||
.parentElement.classList
|
||||
).toContain('d-none')
|
||||
expect(
|
||||
fixture.nativeElement.querySelector('[routerLink="mail"]').parentElement
|
||||
.classList
|
||||
@@ -325,21 +400,12 @@ describe('AppFrameComponent', () => {
|
||||
])
|
||||
settingsService.sidebarHiddenItemsEditing.set([
|
||||
HideableSidebarItemID.Dashboard,
|
||||
HideableSidebarItemID.OcrTemplates,
|
||||
])
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(
|
||||
fixture.nativeElement.querySelectorAll('pngx-input-switch').length
|
||||
).toBe(7)
|
||||
const ocrTemplatesLink = fixture.nativeElement.querySelector(
|
||||
'[routerLink="ocr-templates"]'
|
||||
)
|
||||
expect(ocrTemplatesLink.parentElement.classList).not.toContain('d-none')
|
||||
expect(ocrTemplatesLink.classList).toContain('opacity-50')
|
||||
expect(
|
||||
ocrTemplatesLink.parentElement.querySelector('pngx-input-switch')
|
||||
).not.toBeNull()
|
||||
).toBe(6)
|
||||
expect(
|
||||
fixture.nativeElement.querySelector('[routerLink="dashboard"]')
|
||||
.parentElement.classList
|
||||
|
||||
@@ -8,12 +8,14 @@ import {
|
||||
import { NgClass } from '@angular/common'
|
||||
import {
|
||||
Component,
|
||||
DestroyRef,
|
||||
HostListener,
|
||||
inject,
|
||||
OnInit,
|
||||
signal,
|
||||
viewChild,
|
||||
} from '@angular/core'
|
||||
import { takeUntilDestroyed } from '@angular/core/rxjs-interop'
|
||||
import { FormsModule } from '@angular/forms'
|
||||
import { ActivatedRoute, Router, RouterModule } from '@angular/router'
|
||||
import {
|
||||
@@ -24,10 +26,11 @@ import {
|
||||
NgbPopoverModule,
|
||||
} from '@ng-bootstrap/ng-bootstrap'
|
||||
import { TourNgBootstrap } from 'ngx-ui-tour-ng-bootstrap'
|
||||
import { Observable } from 'rxjs'
|
||||
import { filter, Observable } from 'rxjs'
|
||||
import { first } from 'rxjs/operators'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { Document } from 'src/app/data/document'
|
||||
import { PaperlessTaskStatus } from 'src/app/data/paperless-task'
|
||||
import { SavedView } from 'src/app/data/saved-view'
|
||||
import {
|
||||
CollapsibleSection,
|
||||
@@ -41,6 +44,7 @@ import {
|
||||
DjangoMessageLevel,
|
||||
DjangoMessagesService,
|
||||
} from 'src/app/services/django-messages.service'
|
||||
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
|
||||
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
|
||||
import {
|
||||
PermissionAction,
|
||||
@@ -101,6 +105,8 @@ export class AppFrameComponent
|
||||
extends ComponentWithPermissions
|
||||
implements OnInit, ComponentCanDeactivate
|
||||
{
|
||||
private destroyRef = inject(DestroyRef)
|
||||
private documentList = inject(DocumentListViewService)
|
||||
router = inject(Router)
|
||||
private activatedRoute = inject(ActivatedRoute)
|
||||
private openDocumentsService = inject(OpenDocumentsService)
|
||||
@@ -164,6 +170,46 @@ export class AppFrameComponent
|
||||
}
|
||||
|
||||
ngOnInit(): void {
|
||||
this.websocketStatusService
|
||||
.onBulkOperationStatus()
|
||||
.pipe(
|
||||
filter(({ status }) =>
|
||||
[
|
||||
PaperlessTaskStatus.Success,
|
||||
PaperlessTaskStatus.Failure,
|
||||
PaperlessTaskStatus.Revoked,
|
||||
].includes(status)
|
||||
),
|
||||
takeUntilDestroyed(this.destroyRef)
|
||||
)
|
||||
.subscribe(({ status, error }) => {
|
||||
this.documentList.reload(null, false)
|
||||
this.documentList.reduceSelectionToFilter()
|
||||
// maybe someday we can optimize to only refresh affected documents
|
||||
this.openDocumentsService.getOpenDocuments().forEach((doc) => {
|
||||
this.openDocumentsService.refreshDocument(doc.id)
|
||||
})
|
||||
this.savedViewService.maybeRefreshDocumentCounts()
|
||||
if (
|
||||
this.permissionsService.currentUserCan(
|
||||
PermissionAction.View,
|
||||
PermissionType.PaperlessTask
|
||||
)
|
||||
) {
|
||||
this.tasksService.reload()
|
||||
}
|
||||
if (status === PaperlessTaskStatus.Success) {
|
||||
this.toastService.showSuccess($localize`Bulk operation completed.`)
|
||||
} else if (status === PaperlessTaskStatus.Revoked) {
|
||||
this.toastService.showError($localize`Bulk operation revoked.`, error)
|
||||
} else {
|
||||
this.toastService.showError(
|
||||
$localize`Bulk operation failed. Some changes may already have been applied.`,
|
||||
error
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
this.lastScrollY = window.scrollY
|
||||
this.detectClassicScrollbars()
|
||||
|
||||
|
||||
@@ -16,7 +16,6 @@ import basket from 'bootstrap-icons/icons/basket.svg'
|
||||
import bell from 'bootstrap-icons/icons/bell.svg'
|
||||
import bodyText from 'bootstrap-icons/icons/body-text.svg'
|
||||
import bookmark from 'bootstrap-icons/icons/bookmark.svg'
|
||||
import boundingBoxCircles from 'bootstrap-icons/icons/bounding-box-circles.svg'
|
||||
import boxArrowInRight from 'bootstrap-icons/icons/box-arrow-in-right.svg'
|
||||
import boxArrowUpRight from 'bootstrap-icons/icons/box-arrow-up-right.svg'
|
||||
import boxArrowUp from 'bootstrap-icons/icons/box-arrow-up.svg'
|
||||
@@ -78,7 +77,6 @@ import fileEarmarkMinus from 'bootstrap-icons/icons/file-earmark-minus.svg'
|
||||
import fileEarmarkPerson from 'bootstrap-icons/icons/file-earmark-person.svg'
|
||||
import fileEarmarkPlus from 'bootstrap-icons/icons/file-earmark-plus.svg'
|
||||
import fileEarmarkRichtext from 'bootstrap-icons/icons/file-earmark-richtext.svg'
|
||||
import fileEarmarkRuled from 'bootstrap-icons/icons/file-earmark-ruled.svg'
|
||||
import fileEarmarkSpreadsheet from 'bootstrap-icons/icons/file-earmark-spreadsheet.svg'
|
||||
import fileEarmark from 'bootstrap-icons/icons/file-earmark.svg'
|
||||
import fileText from 'bootstrap-icons/icons/file-text.svg'
|
||||
@@ -132,7 +130,6 @@ import search from 'bootstrap-icons/icons/search.svg'
|
||||
import send from 'bootstrap-icons/icons/send.svg'
|
||||
import shop from 'bootstrap-icons/icons/shop.svg'
|
||||
import slashCircle from 'bootstrap-icons/icons/slash-circle.svg'
|
||||
import sliders2 from 'bootstrap-icons/icons/sliders2.svg'
|
||||
import sliders2Vertical from 'bootstrap-icons/icons/sliders2-vertical.svg'
|
||||
import sortAlphaDown from 'bootstrap-icons/icons/sort-alpha-down.svg'
|
||||
import sortAlphaUpAlt from 'bootstrap-icons/icons/sort-alpha-up-alt.svg'
|
||||
@@ -181,7 +178,6 @@ export const icons: Record<string, string> = {
|
||||
braces,
|
||||
bodyText,
|
||||
bookmark,
|
||||
boundingBoxCircles,
|
||||
boxArrowInRight,
|
||||
boxArrowUp,
|
||||
boxArrowUpRight,
|
||||
@@ -243,7 +239,6 @@ export const icons: Record<string, string> = {
|
||||
fileEarmarkPerson,
|
||||
fileEarmarkPlus,
|
||||
fileEarmarkRichtext,
|
||||
fileEarmarkRuled,
|
||||
fileEarmarkSpreadsheet,
|
||||
files,
|
||||
fileText,
|
||||
@@ -295,7 +290,6 @@ export const icons: Record<string, string> = {
|
||||
search,
|
||||
send,
|
||||
slashCircle,
|
||||
sliders2,
|
||||
shop,
|
||||
sliders2Vertical,
|
||||
sortAlphaDown,
|
||||
|
||||
-30
@@ -1,30 +0,0 @@
|
||||
<div class="modal-header">
|
||||
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
|
||||
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
|
||||
</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
@if (messageBold) {
|
||||
<p class="text-break"><b>{{messageBold}}</b></p>
|
||||
}
|
||||
@if (message) {
|
||||
<p class="text-break" [innerHTML]="message"></p>
|
||||
}
|
||||
<pngx-input-select
|
||||
[items]="templates()"
|
||||
[(ngModel)]="template"
|
||||
[allowNull]="true"
|
||||
title="OCR template"
|
||||
i18n-title
|
||||
placeholder="All matching templates"
|
||||
i18n-placeholder
|
||||
></pngx-input-select>
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled()">
|
||||
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
|
||||
</button>
|
||||
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled()">
|
||||
{{btnCaption}}
|
||||
</button>
|
||||
</div>
|
||||
-52
@@ -1,52 +0,0 @@
|
||||
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
|
||||
import { provideHttpClientTesting } from '@angular/common/http/testing'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { of } from 'rxjs'
|
||||
import { MATCH_NONE } from 'src/app/data/matching-model'
|
||||
import { OcrTemplate } from 'src/app/data/ocr-template'
|
||||
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
|
||||
import { ZoneOcrConfirmDialogComponent } from './zone-ocr-confirm-dialog.component'
|
||||
|
||||
const templates: OcrTemplate[] = [
|
||||
{
|
||||
id: 1,
|
||||
name: 'Invoices',
|
||||
match: '',
|
||||
matching_algorithm: MATCH_NONE,
|
||||
sample_document: null,
|
||||
source_width: 0,
|
||||
source_height: 0,
|
||||
enabled: true,
|
||||
zones: [],
|
||||
},
|
||||
]
|
||||
|
||||
describe('ZoneOcrConfirmDialogComponent', () => {
|
||||
let component: ZoneOcrConfirmDialogComponent
|
||||
let fixture: ComponentFixture<ZoneOcrConfirmDialogComponent>
|
||||
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
providers: [
|
||||
NgbActiveModal,
|
||||
provideHttpClient(withInterceptorsFromDi()),
|
||||
provideHttpClientTesting(),
|
||||
],
|
||||
imports: [ZoneOcrConfirmDialogComponent],
|
||||
})
|
||||
jest
|
||||
.spyOn(TestBed.inject(OcrTemplateService), 'listAll')
|
||||
.mockReturnValue(of({ count: 1, all: [1], results: templates }))
|
||||
|
||||
fixture = TestBed.createComponent(ZoneOcrConfirmDialogComponent)
|
||||
component = fixture.componentInstance
|
||||
fixture.detectChanges()
|
||||
})
|
||||
|
||||
it('should offer templates and default to all matching templates', () => {
|
||||
expect(component.templates()).toEqual(templates)
|
||||
expect(component.template).toBeNull()
|
||||
expect(fixture.nativeElement.textContent).toContain('OCR template')
|
||||
})
|
||||
})
|
||||
-31
@@ -1,31 +0,0 @@
|
||||
import { Component, OnInit, inject, signal } from '@angular/core'
|
||||
import { FormsModule } from '@angular/forms'
|
||||
import { takeUntil } from 'rxjs'
|
||||
import { OcrTemplate } from 'src/app/data/ocr-template'
|
||||
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
|
||||
import { SelectComponent } from '../../input/select/select.component'
|
||||
import { ConfirmDialogComponent } from '../confirm-dialog.component'
|
||||
|
||||
@Component({
|
||||
selector: 'pngx-zone-ocr-confirm-dialog',
|
||||
templateUrl: './zone-ocr-confirm-dialog.component.html',
|
||||
imports: [FormsModule, SelectComponent],
|
||||
})
|
||||
export class ZoneOcrConfirmDialogComponent
|
||||
extends ConfirmDialogComponent
|
||||
implements OnInit
|
||||
{
|
||||
private readonly ocrTemplateService = inject(OcrTemplateService)
|
||||
|
||||
public readonly templates = signal<OcrTemplate[]>([])
|
||||
|
||||
/** The template to run, or null for every template that matches. */
|
||||
template: number | null = null
|
||||
|
||||
ngOnInit() {
|
||||
this.ocrTemplateService
|
||||
.listAll()
|
||||
.pipe(takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe((r) => this.templates.set(r.results))
|
||||
}
|
||||
}
|
||||
@@ -40,7 +40,7 @@
|
||||
(change)="onChange(selectedDocumentIDs)">
|
||||
<ng-template ng-label-tmp let-document="item">
|
||||
<div class="d-flex align-items-center">
|
||||
@if (!disabled && multiple) {
|
||||
@if (!disabled) {
|
||||
<button class="btn p-0 lh-1" (click)="unselect(document)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove link" i18n-title><pngx-bs-icon name="x"></pngx-bs-icon></button>
|
||||
}
|
||||
@if (document.title) {
|
||||
|
||||
@@ -81,9 +81,6 @@ export class DocumentLinkComponent
|
||||
@Input()
|
||||
placeholder: string = $localize`Search for documents`
|
||||
|
||||
@Input()
|
||||
multiple: boolean = true
|
||||
|
||||
/**
|
||||
* Parent for ng-select dropdown, needed to prevent close on click.
|
||||
*/
|
||||
|
||||
@@ -61,22 +61,6 @@
|
||||
<pngx-bs-icon name="pencil" class="me-2"></pngx-bs-icon><ng-container i18n>PDF Editor</ng-container>
|
||||
</button>
|
||||
|
||||
<div class="d-flex" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.OcrTemplate }">
|
||||
<button ngbDropdownItem (click)="runZoneOcr()" [disabled]="!userCanEdit || networkActive()">
|
||||
<pngx-bs-icon width="1em" height="1em" name="bounding-box-circles" class="me-2"></pngx-bs-icon><span i18n>Run Zone OCR</span>
|
||||
</button>
|
||||
<button
|
||||
ngbDropdownItem
|
||||
class="w-auto flex-shrink-0 px-2"
|
||||
(click)="runZoneOcrWithOptions()"
|
||||
[disabled]="!userCanEdit || networkActive()"
|
||||
title="Choose OCR template"
|
||||
i18n-title
|
||||
>
|
||||
<pngx-bs-icon width="1em" height="1em" name="sliders2"></pngx-bs-icon>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
@if (userIsOwner && (requiresPassword || password)) {
|
||||
<button ngbDropdownItem (click)="removePassword()" [disabled]="!password">
|
||||
<pngx-bs-icon name="unlock" class="me-2"></pngx-bs-icon><ng-container i18n>Remove Password</ng-container>
|
||||
@@ -106,7 +90,7 @@
|
||||
<div class="row">
|
||||
<div class="col-md-6 col-xl-5 mb-4">
|
||||
|
||||
<form [formGroup]='documentForm' (ngSubmit)="save()" [attr.inert]="zoneOcrRunning() ? '' : null" [attr.aria-busy]="zoneOcrRunning()">
|
||||
<form [formGroup]='documentForm' (ngSubmit)="save()">
|
||||
|
||||
<div class="btn-toolbar mb-1 pb-3 gap-2 row-gap-2 border-bottom">
|
||||
<div class="btn-group">
|
||||
|
||||
@@ -1909,203 +1909,6 @@ describe('DocumentDetailComponent', () => {
|
||||
expect(toastSpy).toHaveBeenCalledWith('Document reloaded.')
|
||||
})
|
||||
|
||||
it('should run a chosen zone OCR template on the selected version and reload', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({ ...doc, versions: docVersions })
|
||||
component.selectedVersionId.set(10)
|
||||
jest.spyOn(documentService, 'runZoneOcr').mockReturnValue(
|
||||
of({
|
||||
results: [
|
||||
{ zone: 'Total', value: '102.10' },
|
||||
{ zone: 'Due', value: '' },
|
||||
],
|
||||
} as any)
|
||||
)
|
||||
const loadSpy = jest
|
||||
.spyOn(component as any, 'loadDocument')
|
||||
.mockImplementation(() => {})
|
||||
const notifySpy = jest.spyOn(component.docChangeNotifier, 'next')
|
||||
const toastSpy = jest.spyOn(toastService, 'showInfo')
|
||||
const modalSpy = jest.spyOn(modalService, 'open')
|
||||
|
||||
component.runZoneOcrWithOptions()
|
||||
const dialog = modalSpy.mock.results[0].value.componentInstance
|
||||
expect(dialog.message).toBe(
|
||||
'Fields filled by the template will be overwritten.'
|
||||
)
|
||||
dialog.template = 3
|
||||
dialog.confirmClicked.emit()
|
||||
|
||||
expect(documentService.runZoneOcr).toHaveBeenCalledWith(doc.id, 3, 10)
|
||||
expect(toastSpy).toHaveBeenCalledWith(
|
||||
'Filled 1 of 2 fields. No value found for: Due'
|
||||
)
|
||||
expect(notifySpy).toHaveBeenCalledWith(doc.id)
|
||||
expect(loadSpy).toHaveBeenCalledWith(doc.id, true, expect.any(Function))
|
||||
expect(component.zoneOcrRunning()).toBe(true)
|
||||
loadSpy.mock.calls[0][2]()
|
||||
expect(component.zoneOcrRunning()).toBe(false)
|
||||
expect(component.networkActive()).toBe(false)
|
||||
})
|
||||
|
||||
it('should summarize zone OCR results', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({ ...doc, versions: docVersions })
|
||||
component.selectedVersionId.set(12)
|
||||
jest.spyOn(component as any, 'loadDocument').mockImplementation(() => {})
|
||||
const runSpy = jest.spyOn(documentService, 'runZoneOcr')
|
||||
const toastSpy = jest.spyOn(toastService, 'showInfo')
|
||||
const modalSpy = jest.spyOn(modalService, 'open')
|
||||
|
||||
runSpy.mockReturnValueOnce(of({ status: 'completed', results: [] }))
|
||||
component.runZoneOcr()
|
||||
expect(modalSpy).not.toHaveBeenCalled()
|
||||
expect(runSpy).toHaveBeenCalledWith(doc.id, null, null)
|
||||
expect(toastSpy).toHaveBeenLastCalledWith(
|
||||
'Zone OCR found no fields to fill.'
|
||||
)
|
||||
|
||||
component.networkActive.set(false)
|
||||
runSpy.mockReturnValueOnce(
|
||||
of({
|
||||
status: 'completed',
|
||||
results: [
|
||||
{ template: 'T', zone: 'Total', custom_field: 'Amount', value: 0 },
|
||||
],
|
||||
})
|
||||
)
|
||||
component.runZoneOcr()
|
||||
expect(toastSpy).toHaveBeenLastCalledWith('Filled 1 of 1 fields.')
|
||||
})
|
||||
|
||||
it('should confirm before running zone OCR over unsaved changes', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({ ...doc, __changedFields: ['title'] })
|
||||
component.documentForm.patchValue({ title: 'Unsaved title' })
|
||||
const runSpy = jest.spyOn(documentService, 'runZoneOcr')
|
||||
const modalSpy = jest.spyOn(modalService, 'open')
|
||||
|
||||
component.runZoneOcr()
|
||||
|
||||
const modal = modalSpy.mock.results[0].value
|
||||
expect(modal.componentInstance.message).toBe(
|
||||
'Running Zone OCR will discard them.'
|
||||
)
|
||||
expect(runSpy).not.toHaveBeenCalled()
|
||||
modal.componentInstance.cancel()
|
||||
expect(runSpy).not.toHaveBeenCalled()
|
||||
expect(component.documentForm.get('title').value).toBe('Unsaved title')
|
||||
expect(component.zoneOcrRunning()).toBe(false)
|
||||
})
|
||||
|
||||
it('should warn about unsaved changes in the zone OCR options dialog', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({ ...doc, __changedFields: ['title'] })
|
||||
const runSpy = jest.spyOn(documentService, 'runZoneOcr')
|
||||
const modalSpy = jest.spyOn(modalService, 'open')
|
||||
|
||||
component.runZoneOcrWithOptions()
|
||||
|
||||
const dialog = modalSpy.mock.results[0].value.componentInstance
|
||||
expect(dialog.message).toBe('Running Zone OCR will discard them.')
|
||||
expect(dialog.btnClass).toBe('btn-warning')
|
||||
dialog.cancel()
|
||||
expect(runSpy).not.toHaveBeenCalled()
|
||||
component.networkActive.set(true)
|
||||
component.runZoneOcrWithOptions()
|
||||
expect(modalSpy).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('should not allow saving or a second zone OCR run while one is running', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({
|
||||
...doc,
|
||||
versions: docVersions,
|
||||
__changedFields: ['title'],
|
||||
})
|
||||
component.selectedVersionId.set(doc.id)
|
||||
const pending = new Subject<any>()
|
||||
const runSpy = jest
|
||||
.spyOn(documentService, 'runZoneOcr')
|
||||
.mockReturnValue(pending)
|
||||
const modalSpy = jest.spyOn(modalService, 'open')
|
||||
|
||||
component.runZoneOcr()
|
||||
modalSpy.mock.results[0].value.componentInstance.confirmClicked.emit()
|
||||
|
||||
expect(runSpy).toHaveBeenCalledWith(doc.id, null, doc.id)
|
||||
expect(component.zoneOcrRunning()).toBe(true)
|
||||
expect(component.networkActive()).toBe(true)
|
||||
const saveSpy = jest.spyOn(documentService, 'patch')
|
||||
component.save()
|
||||
expect(saveSpy).not.toHaveBeenCalled()
|
||||
component.runZoneOcr()
|
||||
expect(modalSpy).toHaveBeenCalledTimes(1)
|
||||
expect(runSpy).toHaveBeenCalledTimes(1)
|
||||
pending.error(new Error('failed'))
|
||||
expect(component.zoneOcrRunning()).toBe(false)
|
||||
expect(component.networkActive()).toBe(false)
|
||||
expect(component.document().__changedFields).toEqual(['title'])
|
||||
})
|
||||
|
||||
it('should not reload after zone OCR if another document is open', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({ ...doc, __changedFields: [] })
|
||||
const pending = new Subject<any>()
|
||||
jest.spyOn(documentService, 'runZoneOcr').mockReturnValue(pending)
|
||||
const loadSpy = jest.spyOn(component as any, 'loadDocument')
|
||||
|
||||
component.runZoneOcr()
|
||||
component.documentId.set(99)
|
||||
pending.next({ results: [] })
|
||||
|
||||
expect(loadSpy).not.toHaveBeenCalled()
|
||||
expect(component.zoneOcrRunning()).toBe(false)
|
||||
pending.complete()
|
||||
})
|
||||
|
||||
it('should keep unsaved changes when zone OCR is skipped', () => {
|
||||
component.documentId.set(doc.id)
|
||||
component.document.set({ ...doc, __changedFields: ['title'] })
|
||||
component.documentForm.patchValue({ title: 'Unsaved title' })
|
||||
const modalSpy = jest.spyOn(modalService, 'open')
|
||||
jest
|
||||
.spyOn(documentService, 'runZoneOcr')
|
||||
.mockReturnValue(of({ status: 'skipped', results: [] }))
|
||||
const loadSpy = jest.spyOn(component as any, 'loadDocument')
|
||||
const infoSpy = jest.spyOn(toastService, 'showInfo')
|
||||
|
||||
component.runZoneOcr()
|
||||
modalSpy.mock.results[0].value.componentInstance.confirm()
|
||||
|
||||
expect(loadSpy).not.toHaveBeenCalled()
|
||||
expect(component.documentForm.get('title').value).toBe('Unsaved title')
|
||||
expect(component.zoneOcrRunning()).toBe(false)
|
||||
expect(component.networkActive()).toBe(false)
|
||||
expect(infoSpy).toHaveBeenCalledWith(
|
||||
'No OCR template applies to this document.'
|
||||
)
|
||||
})
|
||||
|
||||
it('should show an error when zone OCR fails', () => {
|
||||
component.document.set(doc)
|
||||
jest
|
||||
.spyOn(documentService, 'runZoneOcr')
|
||||
.mockReturnValue(throwError(() => new Error('nope')))
|
||||
const loadSpy = jest.spyOn(component as any, 'loadDocument')
|
||||
const errorSpy = jest.spyOn(toastService, 'showError')
|
||||
|
||||
component.runZoneOcr()
|
||||
|
||||
expect(errorSpy).toHaveBeenCalledWith(
|
||||
'Error running zone OCR.',
|
||||
expect.any(Error)
|
||||
)
|
||||
expect(loadSpy).not.toHaveBeenCalled()
|
||||
expect(component.zoneOcrRunning()).toBe(false)
|
||||
expect(component.networkActive()).toBe(false)
|
||||
})
|
||||
|
||||
it('should change preview element by render type', () => {
|
||||
initNormally()
|
||||
component.document.update((document) => ({
|
||||
|
||||
@@ -64,7 +64,6 @@ import {
|
||||
FILTER_STORAGE_PATH,
|
||||
} from 'src/app/data/filter-rule-type'
|
||||
import { ObjectWithId } from 'src/app/data/object-with-id'
|
||||
import { OcrZoneRunResult } from 'src/app/data/ocr-template'
|
||||
import { StoragePath } from 'src/app/data/storage-path'
|
||||
import { Tag } from 'src/app/data/tag'
|
||||
import { SETTINGS_KEYS, SuggestionSource } from 'src/app/data/ui-settings'
|
||||
@@ -105,7 +104,6 @@ import { DocumentDetailFieldID } from '../admin/settings/settings.component'
|
||||
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
|
||||
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
|
||||
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
|
||||
import { ZoneOcrConfirmDialogComponent } from '../common/confirm-dialog/zone-ocr-confirm-dialog/zone-ocr-confirm-dialog.component'
|
||||
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
|
||||
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
|
||||
import { DocumentTypeEditDialogComponent } from '../common/edit-dialog/document-type-edit-dialog/document-type-edit-dialog.component'
|
||||
@@ -268,7 +266,6 @@ export class DocumentDetailComponent
|
||||
readonly error = signal<any>(undefined)
|
||||
|
||||
readonly networkActive = signal(false)
|
||||
readonly zoneOcrRunning = signal(false)
|
||||
|
||||
readonly documentId = signal<number>(undefined)
|
||||
readonly document = signal<Document>(undefined)
|
||||
@@ -546,11 +543,7 @@ export class DocumentDetailComponent
|
||||
)
|
||||
}
|
||||
|
||||
private loadDocument(
|
||||
documentId: number,
|
||||
forceRemote: boolean = false,
|
||||
onComplete?: () => void
|
||||
): void {
|
||||
private loadDocument(documentId: number, forceRemote: boolean = false): void {
|
||||
let redirectedToRoot = false
|
||||
this.closeIncomingUpdateModal()
|
||||
this.pendingIncomingUpdate = null
|
||||
@@ -607,8 +600,7 @@ export class DocumentDetailComponent
|
||||
}),
|
||||
first(),
|
||||
takeUntil(this.unsubscribeNotifier),
|
||||
takeUntil(this.docChangeNotifier),
|
||||
finalize(() => onComplete?.())
|
||||
takeUntil(this.docChangeNotifier)
|
||||
)
|
||||
.subscribe({
|
||||
next: (doc) => {
|
||||
@@ -1342,7 +1334,6 @@ export class DocumentDetailComponent
|
||||
}
|
||||
|
||||
save(close: boolean = false, savedCallback: () => void = null) {
|
||||
if (this.zoneOcrRunning()) return
|
||||
this.networkActive.set(true)
|
||||
;(document.activeElement as HTMLElement)?.dispatchEvent(new Event('change'))
|
||||
this.documentsService
|
||||
@@ -1568,113 +1559,6 @@ export class DocumentDetailComponent
|
||||
})
|
||||
}
|
||||
|
||||
/** Runs every template that matches, confirming only over unsaved changes. */
|
||||
runZoneOcr() {
|
||||
if (this.networkActive()) return
|
||||
const documentId = this.documentId()
|
||||
const versionId = this.getSelectedNonLatestVersionId()
|
||||
if (!this.hasLocalEdits(this.document())) {
|
||||
this.executeZoneOcr(documentId, null, versionId)
|
||||
return
|
||||
}
|
||||
const modal = this.modalService.open(ConfirmDialogComponent, {
|
||||
backdrop: 'static',
|
||||
})
|
||||
modal.componentInstance.title = $localize`Run Zone OCR`
|
||||
modal.componentInstance.messageBold = $localize`You have unsaved changes.`
|
||||
modal.componentInstance.message = $localize`Running Zone OCR will discard them.`
|
||||
modal.componentInstance.btnClass = 'btn-warning'
|
||||
modal.componentInstance.btnCaption = $localize`Run Zone OCR`
|
||||
modal.componentInstance.confirmClicked
|
||||
.pipe(first(), takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe(() => {
|
||||
modal.close()
|
||||
if (this.documentId() === documentId && !this.networkActive()) {
|
||||
this.executeZoneOcr(documentId, null, versionId)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/** Opens the dialog to pick which template to run. */
|
||||
runZoneOcrWithOptions() {
|
||||
if (this.networkActive()) return
|
||||
const documentId = this.documentId()
|
||||
const versionId = this.getSelectedNonLatestVersionId()
|
||||
const modal = this.modalService.open(ZoneOcrConfirmDialogComponent, {
|
||||
backdrop: 'static',
|
||||
})
|
||||
const dialog = modal.componentInstance as ZoneOcrConfirmDialogComponent
|
||||
dialog.title = $localize`Run Zone OCR`
|
||||
dialog.btnCaption = $localize`Run Zone OCR`
|
||||
if (this.hasLocalEdits(this.document())) {
|
||||
dialog.messageBold = $localize`You have unsaved changes.`
|
||||
dialog.message = $localize`Running Zone OCR will discard them.`
|
||||
dialog.btnClass = 'btn-warning'
|
||||
} else {
|
||||
dialog.message = $localize`Fields filled by the template will be overwritten.`
|
||||
}
|
||||
dialog.confirmClicked
|
||||
.pipe(first(), takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe(() => {
|
||||
modal.close()
|
||||
if (this.documentId() === documentId && !this.networkActive()) {
|
||||
this.executeZoneOcr(documentId, dialog.template, versionId)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
private executeZoneOcr(
|
||||
documentId: number,
|
||||
templateId: number | null,
|
||||
versionId: number | null
|
||||
) {
|
||||
this.zoneOcrRunning.set(true)
|
||||
this.networkActive.set(true)
|
||||
this.documentsService
|
||||
.runZoneOcr(documentId, templateId, versionId)
|
||||
.pipe(takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe({
|
||||
next: (response) => {
|
||||
if (
|
||||
this.documentId() !== documentId ||
|
||||
response.status === 'skipped'
|
||||
) {
|
||||
this.finishZoneOcr()
|
||||
if (response.status === 'skipped') {
|
||||
this.toastService.showInfo(
|
||||
$localize`No OCR template applies to this document.`
|
||||
)
|
||||
}
|
||||
return
|
||||
}
|
||||
this.toastService.showInfo(this.zoneOcrSummary(response.results))
|
||||
this.docChangeNotifier.next(documentId)
|
||||
this.loadDocument(documentId, true, () => this.finishZoneOcr())
|
||||
},
|
||||
error: (error) => {
|
||||
this.finishZoneOcr()
|
||||
this.toastService.showError($localize`Error running zone OCR.`, error)
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
private zoneOcrSummary(results: OcrZoneRunResult[]): string {
|
||||
if (!results.length) return $localize`Zone OCR found no fields to fill.`
|
||||
const missed = results
|
||||
.filter((result) => !`${result.value ?? ''}`.trim())
|
||||
.map((result) => result.zone)
|
||||
const filled = results.length - missed.length
|
||||
return missed.length
|
||||
? $localize`Filled ${filled} of ${results.length} fields. No value found for: ${missed.join(', ')}`
|
||||
: $localize`Filled ${filled} of ${results.length} fields.`
|
||||
}
|
||||
|
||||
private finishZoneOcr() {
|
||||
this.zoneOcrRunning.set(false)
|
||||
this.networkActive.set(false)
|
||||
this.flushPendingIncomingUpdate()
|
||||
}
|
||||
|
||||
private getSelectedNonLatestVersionId(): number | null {
|
||||
const versions = this.document()?.versions ?? []
|
||||
if (!versions.length || !this.selectedVersionId()) {
|
||||
|
||||
@@ -98,11 +98,6 @@
|
||||
<button ngbDropdownItem (click)="mergeSelectedAsVersions()" [disabled]="!userOwnsAll || !userCanEditAll || !userCanDelete || list.allSelected || list.selectedCount < 2">
|
||||
<pngx-bs-icon name="journal-bookmark-fill" class="me-2"></pngx-bs-icon><ng-container i18n>Merge as versions</ng-container>
|
||||
</button>
|
||||
@if (permissionService.currentUserCan(PermissionAction.View, PermissionType.OcrTemplate)) {
|
||||
<button ngbDropdownItem (click)="runZoneOcrSelected()" [disabled]="!userCanEditAll">
|
||||
<pngx-bs-icon name="bounding-box-circles" class="me-2"></pngx-bs-icon><ng-container i18n>Run Zone OCR</ng-container>
|
||||
</button>
|
||||
}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
+111
-107
@@ -559,18 +559,19 @@ describe('BulkEditorComponent', () => {
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: [3, 4],
|
||||
method: 'modify_tags',
|
||||
parameters: { add_tags: [101], remove_tags: [] },
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should execute modify tags bulk operation for all filtered documents', () => {
|
||||
@@ -607,17 +608,18 @@ describe('BulkEditorComponent', () => {
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
all: true,
|
||||
filters: { title_search: 'apple' },
|
||||
excluded_documents: [4],
|
||||
method: 'modify_tags',
|
||||
parameters: { add_tags: [101], remove_tags: [] },
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
)
|
||||
})
|
||||
|
||||
it('should execute modify tags bulk operation with confirmation dialog if enabled', () => {
|
||||
@@ -644,12 +646,12 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should set modal dialog text accordingly for tag edit confirmation', () => {
|
||||
@@ -729,18 +731,19 @@ describe('BulkEditorComponent', () => {
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: [3, 4],
|
||||
method: 'set_correspondent',
|
||||
parameters: { correspondent: 101 },
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should execute modify correspondent bulk operation with confirmation dialog if enabled', () => {
|
||||
@@ -767,12 +770,12 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should set modal dialog text accordingly for correspondent edit confirmation', () => {
|
||||
@@ -827,18 +830,19 @@ describe('BulkEditorComponent', () => {
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: [3, 4],
|
||||
method: 'set_document_type',
|
||||
parameters: { document_type: 101 },
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should execute modify document type bulk operation with confirmation dialog if enabled', () => {
|
||||
@@ -865,12 +869,12 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should set modal dialog text accordingly for document type edit confirmation', () => {
|
||||
@@ -925,18 +929,19 @@ describe('BulkEditorComponent', () => {
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: [3, 4],
|
||||
method: 'set_storage_path',
|
||||
parameters: { storage_path: 101 },
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should execute modify storage path bulk operation with confirmation dialog if enabled', () => {
|
||||
@@ -963,12 +968,12 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should set modal dialog text accordingly for storage path edit confirmation', () => {
|
||||
@@ -1023,18 +1028,19 @@ describe('BulkEditorComponent', () => {
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: [3, 4],
|
||||
method: 'modify_custom_fields',
|
||||
parameters: { add_custom_fields: [101], remove_custom_fields: [102] },
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should execute modify custom fields bulk operation with confirmation dialog if enabled', () => {
|
||||
@@ -1061,12 +1067,12 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
|
||||
// coverage for modal messages
|
||||
component.setCustomFields({
|
||||
@@ -1285,53 +1291,6 @@ describe('BulkEditorComponent', () => {
|
||||
) // listAllFilteredIds
|
||||
})
|
||||
|
||||
it('should support run zone ocr', () => {
|
||||
let modal: NgbModalRef
|
||||
modalService.activeInstances.subscribe((m) => (modal = m[0]))
|
||||
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
|
||||
jest
|
||||
.spyOn(documentListViewService, 'documents', 'get')
|
||||
.mockReturnValue([{ id: 3 }, { id: 4 }])
|
||||
jest
|
||||
.spyOn(documentListViewService, 'selected', 'get')
|
||||
.mockReturnValue(new Set([3, 4]))
|
||||
jest
|
||||
.spyOn(permissionsService, 'currentUserHasObjectPermissions')
|
||||
.mockReturnValue(true)
|
||||
fixture.detectChanges()
|
||||
component.runZoneOcrSelected()
|
||||
expect(modal).not.toBeUndefined()
|
||||
modal.componentInstance.confirm()
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/run_zone_ocr/`
|
||||
)
|
||||
req.flush(true)
|
||||
expect(req.request.body).toEqual({
|
||||
documents: [3, 4],
|
||||
})
|
||||
|
||||
component.runZoneOcrSelected()
|
||||
modal.componentInstance.template = 3
|
||||
modal.componentInstance.confirm()
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/run_zone_ocr/`
|
||||
)
|
||||
req.flush(true)
|
||||
expect(req.request.body).toEqual({
|
||||
documents: [3, 4],
|
||||
template: 3,
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}ocr_templates/?page=1&page_size=100000`
|
||||
) // template list
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
})
|
||||
|
||||
it('should support rotate', () => {
|
||||
let modal: NgbModalRef
|
||||
modalService.activeInstances.subscribe((m) => (modal = m[0]))
|
||||
@@ -1587,6 +1546,47 @@ describe('BulkEditorComponent', () => {
|
||||
)
|
||||
})
|
||||
|
||||
it.each([true, false])(
|
||||
'shows queued feedback and gates the task link on permission (%s)',
|
||||
(canViewTasks) => {
|
||||
jest
|
||||
.spyOn(permissionsService, 'currentUserCan')
|
||||
.mockReturnValue(canViewTasks)
|
||||
const queued = jest.spyOn(toastService, 'show')
|
||||
let modal: NgbModalRef
|
||||
modalService.activeInstances.subscribe((instances) => {
|
||||
if (instances.length) modal = instances[0]
|
||||
})
|
||||
const reload = jest.spyOn(documentListViewService, 'reload')
|
||||
component.setPermissions()
|
||||
const close = jest.spyOn(modal, 'close')
|
||||
modal.componentInstance.confirmClicked.emit({
|
||||
permissions: {},
|
||||
merge: true,
|
||||
})
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush({ result: 'OK', task_id: 'queued-operation' })
|
||||
expect(close).toHaveBeenCalledTimes(1)
|
||||
const notification = queued.mock.calls.at(-1)[0]
|
||||
expect(notification.content).toBe('Bulk operation queued.')
|
||||
if (canViewTasks) {
|
||||
expect(notification.actionName).toBe('View task')
|
||||
notification.action()
|
||||
expect(router.navigate).toHaveBeenCalledWith(['/tasks'], {
|
||||
queryParams: { task_id: 'queued-operation' },
|
||||
})
|
||||
} else {
|
||||
expect(notification.action).toBeUndefined()
|
||||
expect(notification.actionName).toBeUndefined()
|
||||
}
|
||||
expect(reload).not.toHaveBeenCalled()
|
||||
expect(documentListViewService.selected.size).toBe(0)
|
||||
expect(documentListViewService.allSelected).toBe(false)
|
||||
}
|
||||
)
|
||||
|
||||
it('should support bulk permissions update', () => {
|
||||
let modal: NgbModalRef
|
||||
modalService.activeInstances.subscribe((m) => (modal = m[0]))
|
||||
@@ -1619,8 +1619,9 @@ describe('BulkEditorComponent', () => {
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/bulk_edit/`
|
||||
)
|
||||
req.flush(true)
|
||||
req.flush({ result: 'OK', task_id: 'queued-task' })
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: [3, 4],
|
||||
method: 'set_permissions',
|
||||
parameters: {
|
||||
@@ -1628,12 +1629,12 @@ describe('BulkEditorComponent', () => {
|
||||
merge: true,
|
||||
},
|
||||
})
|
||||
httpTestingController.match(
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should not attempt to retrieve objects if user does not have permissions', () => {
|
||||
@@ -1934,15 +1935,18 @@ describe('BulkEditorComponent', () => {
|
||||
expect(toastServiceShowErrorSpy).toHaveBeenCalled()
|
||||
expect(listReloadSpy).not.toHaveBeenCalled()
|
||||
|
||||
modal.componentInstance.succeeded.emit()
|
||||
expect(toastServiceShowInfoSpy).toHaveBeenCalled()
|
||||
expect(listReloadSpy).toHaveBeenCalled()
|
||||
httpTestingController.match(
|
||||
modal.componentInstance.succeeded.emit({
|
||||
result: 'OK',
|
||||
task_id: 'custom-fields-task',
|
||||
})
|
||||
expect(toastServiceShowInfoSpy).not.toHaveBeenCalled()
|
||||
expect(listReloadSpy).not.toHaveBeenCalled()
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
)
|
||||
httpTestingController.expectNone(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
)
|
||||
})
|
||||
|
||||
it('should create share link bundle and enable manage callback', () => {
|
||||
|
||||
@@ -64,7 +64,6 @@ import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dial
|
||||
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
|
||||
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
|
||||
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
|
||||
import { ZoneOcrConfirmDialogComponent } from '../../common/confirm-dialog/zone-ocr-confirm-dialog/zone-ocr-confirm-dialog.component'
|
||||
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
|
||||
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
|
||||
import { DocumentTypeEditDialogComponent } from '../../common/edit-dialog/document-type-edit-dialog/document-type-edit-dialog.component'
|
||||
@@ -105,11 +104,11 @@ export class BulkEditorComponent
|
||||
private correspondentService = inject(CorrespondentService)
|
||||
list = inject(DocumentListViewService)
|
||||
private documentService = inject(DocumentService)
|
||||
private trashService = inject(TrashService)
|
||||
private modalService = inject(NgbModal)
|
||||
private openDocumentService = inject(OpenDocumentsService)
|
||||
private settings = inject(SettingsService)
|
||||
private toastService = inject(ToastService)
|
||||
private trashService = inject(TrashService)
|
||||
private storagePathService = inject(StoragePathService)
|
||||
private customFieldService = inject(CustomFieldsService)
|
||||
public readonly permissionService = inject(PermissionsService)
|
||||
@@ -300,11 +299,33 @@ export class BulkEditorComponent
|
||||
.bulkEdit(overrideSelection ?? this.getSelectionQuery(), method, args)
|
||||
.pipe(first())
|
||||
.subscribe({
|
||||
next: () => this.handleOperationSuccess(modal),
|
||||
next: ({ task_id }) => this.handleOperationQueued(modal, task_id),
|
||||
error: (error) => this.handleOperationError(modal, error),
|
||||
})
|
||||
}
|
||||
|
||||
private handleOperationQueued(modal: NgbModalRef, taskId: string) {
|
||||
modal?.close()
|
||||
this.list.selectNone()
|
||||
const canViewTasks = this.permissionService.currentUserCan(
|
||||
PermissionAction.View,
|
||||
PermissionType.PaperlessTask
|
||||
)
|
||||
this.toastService.show({
|
||||
content: $localize`Bulk operation queued.`,
|
||||
delay: 5000,
|
||||
...(canViewTasks
|
||||
? {
|
||||
actionName: $localize`View task`,
|
||||
action: () =>
|
||||
this.router.navigate(['/tasks'], {
|
||||
queryParams: { task_id: taskId },
|
||||
}),
|
||||
}
|
||||
: {}),
|
||||
})
|
||||
}
|
||||
|
||||
private executeDocumentAction(
|
||||
modal: NgbModalRef,
|
||||
request: Observable<any>,
|
||||
@@ -967,30 +988,6 @@ export class BulkEditorComponent
|
||||
})
|
||||
}
|
||||
|
||||
runZoneOcrSelected() {
|
||||
const modal = this.modalService.open(ZoneOcrConfirmDialogComponent, {
|
||||
backdrop: 'static',
|
||||
})
|
||||
const dialog = modal.componentInstance as ZoneOcrConfirmDialogComponent
|
||||
dialog.title = $localize`Run zone OCR confirm`
|
||||
dialog.messageBold = $localize`This operation will run zone OCR on ${this.getSelectionSize()} selected document(s).`
|
||||
dialog.message = $localize`Fields filled by the template will be overwritten.`
|
||||
dialog.btnClass = 'btn-warning'
|
||||
dialog.btnCaption = $localize`Proceed`
|
||||
dialog.confirmClicked
|
||||
.pipe(takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe(() => {
|
||||
dialog.buttonsEnabled.set(false)
|
||||
this.executeDocumentAction(
|
||||
modal,
|
||||
this.documentService.runZoneOcrDocuments(
|
||||
this.getSelectionQuery(),
|
||||
dialog.template
|
||||
)
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
setPermissions() {
|
||||
let modal = this.modalService.open(PermissionsDialogComponent, {
|
||||
backdrop: 'static',
|
||||
@@ -1113,14 +1110,9 @@ export class BulkEditorComponent
|
||||
|
||||
dialog.selection = this.getSelectionQuery()
|
||||
dialog.selectionCount = this.getSelectionSize()
|
||||
dialog.succeeded.subscribe((result) => {
|
||||
this.toastService.showSuccess($localize`Custom fields updated.`)
|
||||
this.list.reload()
|
||||
this.list.reduceSelectionToFilter()
|
||||
this.list.selected.forEach((id) => {
|
||||
this.openDocumentService.refreshDocument(id)
|
||||
})
|
||||
})
|
||||
dialog.succeeded.subscribe(({ task_id }) =>
|
||||
this.handleOperationQueued(null, task_id)
|
||||
)
|
||||
dialog.failed.subscribe((error) => {
|
||||
this.toastService.showError(
|
||||
$localize`Error updating custom fields.`,
|
||||
|
||||
+1
-1
@@ -59,7 +59,7 @@ describe('CustomFieldsBulkEditDialogComponent', () => {
|
||||
it('should emit succeeded event and close modal on successful save', () => {
|
||||
const editSpy = jest
|
||||
.spyOn(documentService, 'bulkEdit')
|
||||
.mockReturnValue(of('Success'))
|
||||
.mockReturnValue(of({ result: 'OK', task_id: 'custom-fields-task' }))
|
||||
const successSpy = jest.spyOn(component.succeeded, 'emit')
|
||||
|
||||
component.selection = [1, 2]
|
||||
|
||||
+7
-3
@@ -16,6 +16,7 @@ import { NumberComponent } from 'src/app/components/common/input/number/number.c
|
||||
import { SelectComponent } from 'src/app/components/common/input/select/select.component'
|
||||
import { TextComponent } from 'src/app/components/common/input/text/text.component'
|
||||
import { UrlComponent } from 'src/app/components/common/input/url/url.component'
|
||||
import { BulkOperationResponse } from 'src/app/data/bulk-operation'
|
||||
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
|
||||
import {
|
||||
DocumentSelectionQuery,
|
||||
@@ -49,7 +50,7 @@ export class CustomFieldsBulkEditDialogComponent {
|
||||
CustomFieldDataType = CustomFieldDataType
|
||||
|
||||
@Output()
|
||||
succeeded = new EventEmitter()
|
||||
succeeded = new EventEmitter<BulkOperationResponse>()
|
||||
|
||||
@Output()
|
||||
failed = new EventEmitter()
|
||||
@@ -103,6 +104,8 @@ export class CustomFieldsBulkEditDialogComponent {
|
||||
}
|
||||
|
||||
public save() {
|
||||
if (this.networkActive) return
|
||||
this.networkActive = true
|
||||
this.documentService
|
||||
.bulkEdit(this.selection, 'modify_custom_fields', {
|
||||
add_custom_fields: this.form.value,
|
||||
@@ -110,11 +113,12 @@ export class CustomFieldsBulkEditDialogComponent {
|
||||
})
|
||||
.pipe(first())
|
||||
.subscribe({
|
||||
next: () => {
|
||||
next: (response) => {
|
||||
this.activeModal.close()
|
||||
this.succeeded.emit()
|
||||
this.succeeded.emit(response)
|
||||
},
|
||||
error: (error) => {
|
||||
this.networkActive = false
|
||||
this.failed.emit(error)
|
||||
},
|
||||
})
|
||||
|
||||
-36
@@ -1,36 +0,0 @@
|
||||
@if (zones.length === 0) {
|
||||
<p class="text-muted" i18n>No zones yet. Drag anywhere on the sample document to create one.</p>
|
||||
} @else {
|
||||
<div class="list-group">
|
||||
@for (zone of zones; track $index; let i = $index) {
|
||||
<div class="list-group-item list-group-item-action d-flex align-items-center gap-3 position-relative">
|
||||
<span class="p-2 rounded flex-shrink-0" [style.background-color]="zoneColor(i)" aria-hidden="true"></span>
|
||||
<div class="flex-grow-1 text-truncate">
|
||||
<button type="button" class="btn btn-link p-0 fw-semibold text-reset text-decoration-none text-truncate mw-100 stretched-link" (click)="zoneSelected.emit(i)">{{ zone.name }}</button>
|
||||
@if (!hasField(zone)) {
|
||||
<div class="small text-warning text-truncate">
|
||||
<pngx-bs-icon name="exclamation-triangle" class="me-1" aria-hidden="true"></pngx-bs-icon><span i18n>No field selected</span>
|
||||
</div>
|
||||
} @else {
|
||||
<div class="small text-muted text-truncate">
|
||||
{{ getZoneTargetName(zone) }}
|
||||
@if (isCombined(zone)) {
|
||||
<pngx-bs-icon name="braces" class="ms-1" title="Combined with other zones" i18n-title></pngx-bs-icon>
|
||||
}
|
||||
</div>
|
||||
}
|
||||
</div>
|
||||
<small class="text-muted text-nowrap">
|
||||
@if (zone.page === -1) {
|
||||
<ng-container i18n>Last page</ng-container>
|
||||
} @else {
|
||||
<ng-container i18n>Page {{ zone.page ?? 1 }}</ng-container>
|
||||
}
|
||||
</small>
|
||||
<button type="button" class="btn btn-sm btn-link text-danger position-relative z-2" (click)="zoneRemoved.emit(i)" title="Delete zone" i18n-title aria-label="Delete zone" i18n-aria-label>
|
||||
<pngx-bs-icon name="trash"></pngx-bs-icon>
|
||||
</button>
|
||||
</div>
|
||||
}
|
||||
</div>
|
||||
}
|
||||
-115
@@ -1,115 +0,0 @@
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { CustomField } from 'src/app/data/custom-field'
|
||||
import {
|
||||
OcrTemplateZone,
|
||||
OcrZoneTarget,
|
||||
OcrZoneTransform,
|
||||
} from 'src/app/data/ocr-template'
|
||||
import { OcrTemplateEditorZoneListComponent } from './ocr-template-editor-zone-list.component'
|
||||
|
||||
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
|
||||
return {
|
||||
name: 'Zone 1',
|
||||
target: OcrZoneTarget.CustomField,
|
||||
custom_field: 7,
|
||||
x: 10,
|
||||
y: 20,
|
||||
width: 30,
|
||||
height: 40,
|
||||
page: 1,
|
||||
ocr_language: 'eng',
|
||||
transform: OcrZoneTransform.Strip,
|
||||
validation_regex: '',
|
||||
order: 0,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
describe('OcrTemplateEditorZoneListComponent', () => {
|
||||
let fixture: ComponentFixture<OcrTemplateEditorZoneListComponent>
|
||||
let component: OcrTemplateEditorZoneListComponent
|
||||
|
||||
beforeEach(async () => {
|
||||
await TestBed.configureTestingModule({
|
||||
imports: [OcrTemplateEditorZoneListComponent, BootstrapIconComponent],
|
||||
}).compileComponents()
|
||||
|
||||
fixture = TestBed.createComponent(OcrTemplateEditorZoneListComponent)
|
||||
component = fixture.componentInstance
|
||||
})
|
||||
|
||||
it('shows empty state when no zones are defined', () => {
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(fixture.nativeElement.textContent).toContain('No zones yet')
|
||||
})
|
||||
|
||||
it('renders zone name, target, and page', () => {
|
||||
component.zones = [zone(), zone({ name: 'Zone 2', page: -1 })]
|
||||
component.customFields = [{ id: 7, name: 'Invoice Number' } as CustomField]
|
||||
fixture.detectChanges()
|
||||
|
||||
const text = fixture.nativeElement.textContent
|
||||
expect(text).toContain('Zone 1')
|
||||
expect(text).toContain('Invoice Number')
|
||||
expect(text).toContain('Page 1')
|
||||
expect(text).toContain('Last page')
|
||||
})
|
||||
|
||||
it('marks zones that share a field', () => {
|
||||
component.zones = [
|
||||
zone(),
|
||||
zone({ name: 'Zone 2' }),
|
||||
zone({ name: 'Zone 3', custom_field: 8 }),
|
||||
]
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(component.isCombined(component.zones[0])).toBe(true)
|
||||
expect(component.isCombined(component.zones[2])).toBe(false)
|
||||
expect(
|
||||
fixture.nativeElement.querySelectorAll(
|
||||
'[title="Combined with other zones"]'
|
||||
)
|
||||
).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('warns only for zones without a field and clears the warning when assigned', () => {
|
||||
component.zones = [
|
||||
zone({ custom_field: null }),
|
||||
zone({ target: OcrZoneTarget.Title, custom_field: null }),
|
||||
zone(),
|
||||
]
|
||||
fixture.detectChanges()
|
||||
|
||||
const warnings = () =>
|
||||
fixture.nativeElement.querySelectorAll('.text-warning')
|
||||
expect(warnings()).toHaveLength(1)
|
||||
expect(warnings()[0].textContent).toContain('No field selected')
|
||||
|
||||
component.zones[0].custom_field = 7
|
||||
fixture.componentRef.setInput('zones', [...component.zones])
|
||||
fixture.detectChanges()
|
||||
expect(warnings()).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('emits select and remove events', () => {
|
||||
component.zones = [zone()]
|
||||
const selectSpy = jest.spyOn(component.zoneSelected, 'emit')
|
||||
const removeSpy = jest.spyOn(component.zoneRemoved, 'emit')
|
||||
fixture.detectChanges()
|
||||
|
||||
const buttons = fixture.nativeElement.querySelectorAll('button')
|
||||
buttons[0].click()
|
||||
buttons[1].click()
|
||||
|
||||
expect(selectSpy).toHaveBeenCalledWith(0)
|
||||
expect(removeSpy).toHaveBeenCalledWith(0)
|
||||
})
|
||||
|
||||
it('does not label an unknown built-in target', () => {
|
||||
expect(
|
||||
component.getZoneTargetName(zone({ target: 'unknown' as OcrZoneTarget }))
|
||||
).toBe('')
|
||||
})
|
||||
})
|
||||
-50
@@ -1,50 +0,0 @@
|
||||
import { Component, EventEmitter, Input, Output } from '@angular/core'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { CustomField } from 'src/app/data/custom-field'
|
||||
import {
|
||||
getZoneFieldKey,
|
||||
OCR_BUILTIN_TARGETS,
|
||||
OcrTemplateZone,
|
||||
OcrZoneTarget,
|
||||
} from 'src/app/data/ocr-template'
|
||||
import { getZoneColor } from '../zone-geometry'
|
||||
|
||||
@Component({
|
||||
selector: 'pngx-ocr-template-zone-list',
|
||||
imports: [BootstrapIconComponent],
|
||||
templateUrl: './ocr-template-editor-zone-list.component.html',
|
||||
})
|
||||
export class OcrTemplateEditorZoneListComponent {
|
||||
@Input() zones: OcrTemplateZone[] = []
|
||||
@Input() customFields: CustomField[] = []
|
||||
|
||||
@Output() public readonly zoneSelected = new EventEmitter<number>()
|
||||
@Output() public readonly zoneRemoved = new EventEmitter<number>()
|
||||
|
||||
public readonly zoneColor = getZoneColor
|
||||
|
||||
hasField(zone: OcrTemplateZone): boolean {
|
||||
return getZoneFieldKey(zone) !== null
|
||||
}
|
||||
|
||||
isCombined(zone: OcrTemplateZone): boolean {
|
||||
const key = getZoneFieldKey(zone)
|
||||
return (
|
||||
key !== null &&
|
||||
this.zones.filter((z) => getZoneFieldKey(z) === key).length > 1
|
||||
)
|
||||
}
|
||||
|
||||
getZoneTargetName(zone: OcrTemplateZone): string {
|
||||
if (zone.target === OcrZoneTarget.CustomField) {
|
||||
return (
|
||||
this.customFields.find((field) => field.id === zone.custom_field)
|
||||
?.name ?? ''
|
||||
)
|
||||
}
|
||||
return (
|
||||
OCR_BUILTIN_TARGETS.find((target) => target.id === zone.target)?.name ??
|
||||
''
|
||||
)
|
||||
}
|
||||
}
|
||||
-313
@@ -1,313 +0,0 @@
|
||||
<div class="modal-header">
|
||||
<h4 class="modal-title" id="ocr-template-editor-title">{{ dialogTitle }}</h4>
|
||||
@if (template.id) {
|
||||
<span class="badge bg-primary text-primary-text-contrast ms-2">ID: {{ template.id }}</span>
|
||||
}
|
||||
<button type="button" class="btn-close" (click)="activeModal.dismiss()" [disabled]="saving()" aria-label="Close" i18n-aria-label></button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body d-flex flex-column">
|
||||
<div class="row gx-3 editor-layout flex-grow-1 flex-lg-nowrap" [attr.inert]="saving() ? '' : null" [attr.aria-busy]="saving()">
|
||||
<div class="col-lg-4 mb-3 mb-lg-0 overflow-auto">
|
||||
<ul ngbNav #nav="ngbNav" [(activeId)]="activeTab" class="nav-underline flex-nowrap flex-md-wrap overflow-auto">
|
||||
<li ngbNavItem="template">
|
||||
<a ngbNavLink i18n>Template</a>
|
||||
<ng-template ngbNavContent>
|
||||
<div class="row mb-3">
|
||||
<div class="col-9">
|
||||
<pngx-input-text [(ngModel)]="template.name" title="Template name" i18n-title></pngx-input-text>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<pngx-input-switch [(ngModel)]="template.enabled" title="Enabled" i18n-title></pngx-input-switch>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<pngx-input-select [(ngModel)]="template.matching_algorithm" [items]="matchingAlgorithms" title="Matching algorithm" i18n-title></pngx-input-select>
|
||||
@if (patternRequired) {
|
||||
<pngx-input-text [(ngModel)]="template.match" title="Matching pattern" i18n-title></pngx-input-text>
|
||||
<pngx-input-check [(ngModel)]="template.is_insensitive" title="Case insensitive" i18n-title></pngx-input-check>
|
||||
}
|
||||
|
||||
<div *pngxIfOwner="object">
|
||||
<hr />
|
||||
<pngx-permissions-form [users]="users()" [accordion]="true" [(ngModel)]="permissionsForm"></pngx-permissions-form>
|
||||
</div>
|
||||
|
||||
</ng-template>
|
||||
</li>
|
||||
|
||||
<li ngbNavItem="zones">
|
||||
<a ngbNavLink><ng-container i18n>Zones</ng-container> <span class="badge bg-primary ms-2">{{ template.zones.length }}</span></a>
|
||||
<ng-template ngbNavContent>
|
||||
@if (selectedZone; as zone) {
|
||||
<div class="d-flex align-items-center gap-2 mb-3">
|
||||
<button type="button" class="btn btn-sm btn-link px-0 me-auto" (click)="deselectZone()">
|
||||
<pngx-bs-icon name="arrow-left" class="me-1"></pngx-bs-icon><ng-container i18n>All zones</ng-container>
|
||||
</button>
|
||||
<button type="button" class="btn btn-sm btn-outline-primary" (click)="testZone()" [disabled]="!previewDocId || zoneTesting">
|
||||
@if (zoneTesting) {
|
||||
<span class="spinner-border spinner-border-sm me-1" aria-hidden="true"></span>
|
||||
} @else {
|
||||
<pngx-bs-icon name="play-fill" class="me-1"></pngx-bs-icon>
|
||||
}
|
||||
<ng-container i18n>Test</ng-container>
|
||||
</button>
|
||||
<button type="button" class="btn btn-sm btn-outline-danger" (click)="removeZone(selectedZoneIndex)" title="Delete zone" i18n-title aria-label="Delete zone" i18n-aria-label>
|
||||
<pngx-bs-icon name="trash"></pngx-bs-icon>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
@if (zoneTestResult; as result) {
|
||||
@if (result.error) {
|
||||
<div class="alert alert-warning py-2 mb-3 small" role="alert">{{ result.error }}</div>
|
||||
} @else {
|
||||
<dl class="row small border rounded mx-0 py-2 mb-3">
|
||||
<dt class="col-4" i18n>OCR text</dt>
|
||||
<dd class="col-8">
|
||||
@if (result.raw_text) {
|
||||
<code>{{ result.raw_text }}</code>
|
||||
} @else {
|
||||
<span class="text-muted" i18n>Nothing detected</span>
|
||||
}
|
||||
</dd>
|
||||
<dt class="col-4" i18n>Value</dt>
|
||||
<dd class="col-8" [class.mb-0]="!result.regex">
|
||||
@if (result.value) {
|
||||
<code>{{ result.value }}</code>
|
||||
} @else {
|
||||
<span class="text-muted" i18n>Empty</span>
|
||||
}
|
||||
</dd>
|
||||
@if (result.regex) {
|
||||
<dt class="col-4" i18n>Validation</dt>
|
||||
<dd class="col-8 mb-0">
|
||||
@if (result.regex_match) {
|
||||
<span class="badge bg-success" i18n>Regex matches</span>
|
||||
} @else {
|
||||
<span class="badge bg-danger" i18n>Regex does not match</span>
|
||||
}
|
||||
</dd>
|
||||
}
|
||||
</dl>
|
||||
}
|
||||
}
|
||||
|
||||
<pngx-input-text [ngModel]="zone.name" (ngModelChange)="renameZone(zone, $event)" title="Name" i18n-title></pngx-input-text>
|
||||
|
||||
<pngx-input-select
|
||||
[items]="fieldItems()"
|
||||
[ngModel]="zoneFieldValue(zone)"
|
||||
(ngModelChange)="setZoneField(zone, $event)"
|
||||
(createNew)="createField($event)"
|
||||
title="Field"
|
||||
i18n-title
|
||||
placeholder="Choose a field"
|
||||
i18n-placeholder
|
||||
hint="Where the extracted value is written."
|
||||
i18n-hint
|
||||
></pngx-input-select>
|
||||
|
||||
@if (isFieldShared(zone)) {
|
||||
<pngx-input-text
|
||||
[ngModel]="getCombineFormat(zone)"
|
||||
(ngModelChange)="setCombineFormat(zone, $event)"
|
||||
[placeholder]="defaultCombineFormat(zone)"
|
||||
title="Combined value"
|
||||
i18n-title
|
||||
hint="Several zones write to this field. Set the format or leave empty to join them with spaces."
|
||||
i18n-hint
|
||||
></pngx-input-text>
|
||||
<div class="d-flex flex-wrap gap-1 mt-n2 mb-3">
|
||||
@for (z of zonesForField(zone); track z) {
|
||||
<button type="button" class="btn btn-sm btn-outline-secondary font-monospace" (click)="insertCombineToken(zone, z)">{{ '{' + z.name + '}' }}</button>
|
||||
}
|
||||
</div>
|
||||
}
|
||||
|
||||
<div class="row align-items-end">
|
||||
<div class="col">
|
||||
@if (!isLastPageZone(zone)) {
|
||||
<pngx-input-number [(ngModel)]="zone.page" [showAdd]="false" title="Page" i18n-title></pngx-input-number>
|
||||
}
|
||||
</div>
|
||||
<div class="col-auto">
|
||||
<pngx-input-switch [ngModel]="isLastPageZone(zone)" (ngModelChange)="setLastPageZone(zone, $event)" [horizontal]="true" title="Last page" i18n-title></pngx-input-switch>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h6 class="border-top pt-3 mt-1 mb-3" i18n>Processing</h6>
|
||||
|
||||
<pngx-input-select
|
||||
[items]="languageOptions"
|
||||
[multiple]="true"
|
||||
[ngModel]="ocrLanguageArray(zone)"
|
||||
(ngModelChange)="setOcrLanguages(zone, $event)"
|
||||
title="OCR language"
|
||||
i18n-title
|
||||
placeholder="Select languages"
|
||||
i18n-placeholder
|
||||
></pngx-input-select>
|
||||
|
||||
<pngx-input-select [items]="transformOptions" [(ngModel)]="zone.transform" title="Transform" i18n-title></pngx-input-select>
|
||||
|
||||
@if (zone.transform === OcrZoneTransform.Date) {
|
||||
<pngx-input-text
|
||||
[(ngModel)]="zone.date_format"
|
||||
title="Date format"
|
||||
i18n-title
|
||||
placeholder="Auto-detect"
|
||||
i18n-placeholder
|
||||
hint="Python date codes, e.g. %d.%m.%Y. Leave empty to detect the format automatically."
|
||||
i18n-hint
|
||||
></pngx-input-text>
|
||||
}
|
||||
|
||||
<pngx-input-text [(ngModel)]="zone.validation_regex" title="Validation regex" i18n-title placeholder="\d{2}\.\d{2}\.\d{4}" hint="The value must match this pattern to be applied." i18n-hint></pngx-input-text>
|
||||
} @else {
|
||||
<pngx-ocr-template-zone-list
|
||||
[zones]="template.zones"
|
||||
[customFields]="customFields()"
|
||||
(zoneSelected)="selectZone($event)"
|
||||
(zoneRemoved)="removeZone($event)"
|
||||
></pngx-ocr-template-zone-list>
|
||||
}
|
||||
</ng-template>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div [ngbNavOutlet]="nav" class="mt-3"></div>
|
||||
</div>
|
||||
|
||||
<div class="col-lg-8 d-flex flex-column">
|
||||
<div class="d-flex flex-wrap gap-2 mb-2 flex-shrink-0">
|
||||
<pngx-input-document-link
|
||||
class="flex-grow-1"
|
||||
[ngModel]="sampleDocumentIds"
|
||||
(ngModelChange)="setSampleDocument($event)"
|
||||
[minimal]="true"
|
||||
[multiple]="false"
|
||||
placeholder="Search for a sample document"
|
||||
i18n-placeholder
|
||||
></pngx-input-document-link>
|
||||
@if (previewDocId) {
|
||||
<div class="input-group w-auto flex-nowrap">
|
||||
<input class="form-control flex-grow-0 preview-page-input" type="number" min="1" [max]="previewPageCount()" [(ngModel)]="previewPageDisplay" [disabled]="!pageLoaded()" aria-label="Page" i18n-aria-label />
|
||||
<span class="input-group-text" i18n>of {{ previewPageCount() ?? '?' }}</span>
|
||||
</div>
|
||||
<fieldset class="input-group w-auto flex-nowrap" aria-label="Zoom" i18n-aria-label>
|
||||
<button type="button" class="btn btn-outline-secondary" aria-label="Zoom out" i18n-aria-label [disabled]="!pageLoaded() || zoom <= 0.5" (click)="zoomOut()"><pngx-bs-icon name="dash"></pngx-bs-icon></button>
|
||||
<button type="button" class="btn btn-outline-secondary preview-zoom" title="Reset zoom" i18n-title [disabled]="!pageLoaded()" (click)="resetZoom()">{{ zoom | percent: '1.0-0' }}</button>
|
||||
<button type="button" class="btn btn-outline-secondary" aria-label="Zoom in" i18n-aria-label [disabled]="!pageLoaded() || zoom >= 4" (click)="zoomIn()"><pngx-bs-icon name="plus"></pngx-bs-icon></button>
|
||||
</fieldset>
|
||||
}
|
||||
</div>
|
||||
<div class="flex-grow-1 overflow-hidden">
|
||||
<div class="h-100 overflow-auto border" [attr.aria-busy]="previewDocId && !pageLoaded()">
|
||||
@if (pageFailed()) {
|
||||
<div class="py-5 px-3 text-center" role="alert">
|
||||
<pngx-bs-icon name="exclamation-triangle" width="32" height="32" class="text-warning"></pngx-bs-icon>
|
||||
<p class="my-3" i18n>Could not load this page.</p>
|
||||
<button type="button" class="btn btn-sm btn-outline-secondary" (click)="setSampleDocument(sampleDocumentIds)"><pngx-bs-icon name="arrow-clockwise" class="me-1"></pngx-bs-icon><span i18n>Retry</span></button>
|
||||
</div>
|
||||
} @else if (previewDocId) {
|
||||
<div class="position-relative" [class.d-none]="!pageLoaded()" [style.width.%]="zoom * 100">
|
||||
<img
|
||||
#pageImage
|
||||
[src]="pageImageUrl"
|
||||
(load)="onImageLoad()"
|
||||
(error)="onImageError()"
|
||||
alt="Sample document page"
|
||||
i18n-alt
|
||||
class="d-block w-100"
|
||||
crossorigin="use-credentials"
|
||||
/>
|
||||
@if (pageLoaded()) {
|
||||
<svg
|
||||
#zoneOverlay
|
||||
class="zone-overlay position-absolute top-0 start-0 w-100 h-100"
|
||||
[attr.viewBox]="overlayViewBox()"
|
||||
preserveAspectRatio="none"
|
||||
[style.cursor]="overlayCursor"
|
||||
(pointerdown)="onOverlayPointerDown($event)"
|
||||
(pointermove)="onOverlayPointerMove($event)"
|
||||
(pointerup)="onOverlayPointerUp($event)"
|
||||
(pointercancel)="onOverlayPointerCancel($event)"
|
||||
(lostpointercapture)="onOverlayPointerCancel($event)"
|
||||
aria-label="Zone overlay" i18n-aria-label
|
||||
>
|
||||
@for (zone of template.zones; track $index; let i = $index) {
|
||||
@if (zoneDisplayRect(i); as rect) {
|
||||
<g>
|
||||
<rect
|
||||
class="zone-rect"
|
||||
[class.zone-rect-selected]="selectedZoneIndex === i"
|
||||
[attr.x]="rect.x"
|
||||
[attr.y]="rect.y"
|
||||
[attr.width]="rect.w"
|
||||
[attr.height]="rect.h"
|
||||
[attr.stroke]="zoneColor(i)"
|
||||
[attr.fill]="zoneColor(i)"
|
||||
></rect>
|
||||
<text
|
||||
class="zone-label fw-semibold pe-none"
|
||||
[attr.x]="rect.x + overlayUnitSize(6)"
|
||||
[attr.y]="zoneLabelY(rect)"
|
||||
[attr.font-size]="overlayFontSize()"
|
||||
[attr.fill]="zoneColor(i)"
|
||||
>{{ zoneLabel(zone, i) }}</text>
|
||||
|
||||
@if (selectedZoneIndex === i) {
|
||||
@for (handle of resizeHandles(rect); track handle.handle) {
|
||||
<rect
|
||||
class="zone-resize-handle"
|
||||
[attr.x]="handle.x - overlayHandleSize() / 2"
|
||||
[attr.y]="handle.y - overlayHandleSize() / 2"
|
||||
[attr.width]="overlayHandleSize()"
|
||||
[attr.height]="overlayHandleSize()"
|
||||
[attr.fill]="zoneColor(i)"
|
||||
></rect>
|
||||
}
|
||||
}
|
||||
</g>
|
||||
}
|
||||
}
|
||||
|
||||
@if (drawingRect(); as rect) {
|
||||
<rect
|
||||
class="zone-drawing-rect"
|
||||
[attr.x]="rect.x"
|
||||
[attr.y]="rect.y"
|
||||
[attr.width]="rect.w"
|
||||
[attr.height]="rect.h"
|
||||
></rect>
|
||||
}
|
||||
</svg>
|
||||
}
|
||||
</div>
|
||||
@if (!pageLoaded()) {
|
||||
<div class="p-5 d-flex justify-content-center align-items-center">
|
||||
<output class="spinner-border">
|
||||
<span class="visually-hidden" i18n>Loading page...</span>
|
||||
</output>
|
||||
</div>
|
||||
}
|
||||
} @else {
|
||||
<div class="py-5 px-3 text-center text-muted">
|
||||
<pngx-bs-icon name="file-text" width="32" height="32"></pngx-bs-icon>
|
||||
<p class="mt-3 mb-0" i18n>No sample document selected.</p>
|
||||
</div>
|
||||
}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button type="button" class="btn btn-outline-secondary" (click)="activeModal.dismiss()" [disabled]="saving()" i18n>Cancel</button>
|
||||
<button type="button" class="btn btn-primary" (click)="save()" [disabled]="saving() || !isDirty">
|
||||
@if (saving()) {
|
||||
<span class="spinner-border spinner-border-sm me-1" aria-hidden="true"></span>
|
||||
}
|
||||
<span i18n>Save</span>
|
||||
</button>
|
||||
</div>
|
||||
-64
@@ -1,64 +0,0 @@
|
||||
:host {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
@media (min-width: 992px) {
|
||||
.modal-body {
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.editor-layout {
|
||||
min-height: 0;
|
||||
}
|
||||
}
|
||||
|
||||
.preview-page-input {
|
||||
width: 3.5rem;
|
||||
}
|
||||
|
||||
.preview-zoom {
|
||||
min-width: 4rem;
|
||||
}
|
||||
|
||||
.zone-overlay {
|
||||
touch-action: none;
|
||||
}
|
||||
|
||||
.zone-rect,
|
||||
.zone-drawing-rect {
|
||||
vector-effect: non-scaling-stroke;
|
||||
}
|
||||
|
||||
.zone-rect {
|
||||
fill-opacity: 0.2;
|
||||
stroke-width: 2;
|
||||
}
|
||||
|
||||
.zone-rect-selected {
|
||||
stroke-width: 3;
|
||||
}
|
||||
|
||||
.zone-label {
|
||||
font-family: var(--bs-font-sans-serif);
|
||||
paint-order: stroke;
|
||||
stroke: #fff;
|
||||
stroke-linejoin: round;
|
||||
stroke-width: 4px;
|
||||
vector-effect: non-scaling-stroke;
|
||||
}
|
||||
|
||||
.zone-resize-handle {
|
||||
stroke: #fff;
|
||||
stroke-width: 1;
|
||||
vector-effect: non-scaling-stroke;
|
||||
}
|
||||
|
||||
.zone-drawing-rect {
|
||||
fill: var(--bs-primary);
|
||||
fill-opacity: 0.2;
|
||||
stroke: var(--bs-primary);
|
||||
stroke-dasharray: 5 5;
|
||||
stroke-width: 2;
|
||||
}
|
||||
-875
@@ -1,875 +0,0 @@
|
||||
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
|
||||
import { provideHttpClientTesting } from '@angular/common/http/testing'
|
||||
import { ElementRef } from '@angular/core'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { By } from '@angular/platform-browser'
|
||||
import { NgbActiveModal, NgbModal } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { of, Subject, throwError } from 'rxjs'
|
||||
import { PermissionsFormComponent } from 'src/app/components/common/input/permissions/permissions-form/permissions-form.component'
|
||||
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
|
||||
import { Document } from 'src/app/data/document'
|
||||
import {
|
||||
MATCH_ANY,
|
||||
MATCH_AUTO,
|
||||
MATCH_LITERAL,
|
||||
MATCH_NONE,
|
||||
} from 'src/app/data/matching-model'
|
||||
import {
|
||||
OcrTemplate,
|
||||
OcrTemplateZone,
|
||||
OcrZoneTarget,
|
||||
OcrZoneTestResult,
|
||||
OcrZoneTransform,
|
||||
} from 'src/app/data/ocr-template'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { User } from 'src/app/data/user'
|
||||
import { PermissionsService } from 'src/app/services/permissions.service'
|
||||
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
|
||||
import { DocumentService } from 'src/app/services/rest/document.service'
|
||||
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
|
||||
import { UserService } from 'src/app/services/rest/user.service'
|
||||
import { SettingsService } from 'src/app/services/settings.service'
|
||||
import { ToastService } from 'src/app/services/toast.service'
|
||||
import { OcrTemplateEditorComponent } from './ocr-template-editor.component'
|
||||
|
||||
const customFields: CustomField[] = [
|
||||
{ id: 1, name: 'Invoice number', data_type: CustomFieldDataType.String },
|
||||
{ id: 2, name: 'Status', data_type: CustomFieldDataType.Select },
|
||||
]
|
||||
|
||||
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
|
||||
return {
|
||||
name: 'Invoice number',
|
||||
target: OcrZoneTarget.CustomField,
|
||||
custom_field: 1,
|
||||
x: 100,
|
||||
y: 100,
|
||||
width: 200,
|
||||
height: 100,
|
||||
page: 1,
|
||||
ocr_language: 'eng',
|
||||
transform: OcrZoneTransform.Strip,
|
||||
validation_regex: '',
|
||||
order: 0,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function pointer(x: number, y: number, pointerId = 1): PointerEvent {
|
||||
return Object.assign(
|
||||
new MouseEvent('pointermove', { clientX: x, clientY: y, button: 0 }),
|
||||
{ pointerId }
|
||||
) as PointerEvent
|
||||
}
|
||||
|
||||
describe('OcrTemplateEditorComponent', () => {
|
||||
let component: OcrTemplateEditorComponent
|
||||
let fixture: ComponentFixture<OcrTemplateEditorComponent>
|
||||
let documentService: DocumentService
|
||||
let templateService: OcrTemplateService
|
||||
let modalService: NgbModal
|
||||
const activeModal = { close: jest.fn(), dismiss: jest.fn() }
|
||||
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
imports: [BootstrapIconComponent, OcrTemplateEditorComponent],
|
||||
providers: [
|
||||
{ provide: NgbActiveModal, useValue: activeModal },
|
||||
provideHttpClient(withInterceptorsFromDi()),
|
||||
provideHttpClientTesting(),
|
||||
],
|
||||
})
|
||||
jest
|
||||
.spyOn(TestBed.inject(CustomFieldsService), 'listAll')
|
||||
.mockReturnValue(of({ count: 2, all: [1, 2], results: customFields }))
|
||||
jest
|
||||
.spyOn(TestBed.inject(UserService), 'listAll')
|
||||
.mockReturnValue(of({ count: 0, all: [], results: [] }))
|
||||
TestBed.inject(PermissionsService).initialize([], { id: 1 } as User)
|
||||
documentService = TestBed.inject(DocumentService)
|
||||
jest
|
||||
.spyOn(documentService, 'get')
|
||||
.mockReturnValue(of({ id: 5, page_count: 3 }))
|
||||
templateService = TestBed.inject(OcrTemplateService)
|
||||
modalService = TestBed.inject(NgbModal)
|
||||
activeModal.close.mockReset()
|
||||
fixture = TestBed.createComponent(OcrTemplateEditorComponent)
|
||||
component = fixture.componentInstance
|
||||
})
|
||||
|
||||
function prepareOverlay() {
|
||||
const image = document.createElement('img')
|
||||
Object.defineProperties(image, {
|
||||
naturalWidth: { value: 1000 },
|
||||
naturalHeight: { value: 1000 },
|
||||
clientWidth: { value: 1000 },
|
||||
})
|
||||
const overlay = document.createElementNS(
|
||||
'http://www.w3.org/2000/svg',
|
||||
'svg'
|
||||
)
|
||||
Object.assign(overlay, {
|
||||
getScreenCTM: () => ({ inverse: () => ({}) }),
|
||||
createSVGPoint: () => ({
|
||||
x: 0,
|
||||
y: 0,
|
||||
matrixTransform() {
|
||||
return { x: this.x, y: this.y }
|
||||
},
|
||||
}),
|
||||
setPointerCapture: jest.fn(),
|
||||
hasPointerCapture: () => true,
|
||||
releasePointerCapture: jest.fn(),
|
||||
})
|
||||
component.imageRef = new ElementRef(image)
|
||||
component.overlayRef = new ElementRef(overlay)
|
||||
component.pageLoaded.set(true)
|
||||
}
|
||||
|
||||
function fieldDialog() {
|
||||
const dialog = {
|
||||
dialogMode: { set: jest.fn() },
|
||||
object: null,
|
||||
succeeded: new Subject<CustomField>(),
|
||||
failed: new Subject(),
|
||||
}
|
||||
jest
|
||||
.spyOn(modalService, 'open')
|
||||
.mockReturnValue({ componentInstance: dialog } as any)
|
||||
return dialog
|
||||
}
|
||||
|
||||
it('updates screen-sized overlay units when the preview image resizes', () => {
|
||||
let resize: ResizeObserverCallback
|
||||
const observer = {
|
||||
observe: jest.fn(),
|
||||
unobserve: jest.fn(),
|
||||
disconnect: jest.fn(),
|
||||
}
|
||||
const resizeObserver = jest
|
||||
.spyOn(globalThis, 'ResizeObserver')
|
||||
.mockImplementation((callback) => {
|
||||
resize = callback
|
||||
return observer
|
||||
})
|
||||
const image = document.createElement('img')
|
||||
const ref = new ElementRef(image)
|
||||
component.pageImage = ref
|
||||
expect(observer.observe).toHaveBeenCalledWith(image)
|
||||
resize([], observer)
|
||||
expect(component.overlayUnitSize(8)).toBe(8)
|
||||
|
||||
Object.defineProperties(image, {
|
||||
naturalWidth: { value: 1000 },
|
||||
clientWidth: { value: 500 },
|
||||
})
|
||||
resize([], observer)
|
||||
expect(component.overlayUnitSize(8)).toBe(16)
|
||||
expect(component.overlayFontSize()).toBe(24)
|
||||
|
||||
component.pageImage = undefined
|
||||
expect(observer.disconnect).toHaveBeenCalledTimes(1)
|
||||
fixture.destroy()
|
||||
expect(observer.disconnect).toHaveBeenCalledTimes(2)
|
||||
resizeObserver.mockRestore()
|
||||
})
|
||||
|
||||
it('renders loaded image dimensions, zone labels, and selection handles', () => {
|
||||
component.object = {
|
||||
...component.template,
|
||||
id: 3,
|
||||
sample_document: 5,
|
||||
zones: [zone()],
|
||||
}
|
||||
component.selectedZoneIndex = 0
|
||||
fixture.detectChanges()
|
||||
const image = fixture.nativeElement.querySelector('img')
|
||||
Object.defineProperties(image, {
|
||||
naturalWidth: { value: 1000 },
|
||||
naturalHeight: { value: 1500 },
|
||||
})
|
||||
image.dispatchEvent(new Event('load'))
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(component.template).toMatchObject({
|
||||
source_width: 1000,
|
||||
source_height: 1500,
|
||||
})
|
||||
expect(
|
||||
fixture.nativeElement
|
||||
.querySelector('.zone-overlay')
|
||||
.getAttribute('viewBox')
|
||||
).toBe('0 0 1000 1500')
|
||||
expect(
|
||||
fixture.nativeElement.querySelector('.zone-label').textContent
|
||||
).toContain('Invoice number')
|
||||
expect(
|
||||
fixture.nativeElement.querySelectorAll('.zone-resize-handle')
|
||||
).toHaveLength(8)
|
||||
expect(component.zoneLabel(zone({ name: '' }), 2)).toBe('Zone 3')
|
||||
expect(component.zoneLabelY({ x: 0, y: 0, w: 100, h: 100 })).toBe(14)
|
||||
})
|
||||
|
||||
it('offers the supported language and transform choices', () => {
|
||||
expect(component.languageOptions.map((option) => option.id)).toContain(
|
||||
'eng'
|
||||
)
|
||||
expect(component.transformOptions.map((option) => option.id)).toEqual(
|
||||
Object.values(OcrZoneTransform)
|
||||
)
|
||||
})
|
||||
|
||||
it('clamps page navigation and supports one-based page input', () => {
|
||||
component.setSampleDocument([5])
|
||||
component.previewPageDisplay = 2
|
||||
expect(component.previewPage).toBe(1)
|
||||
expect(component.previewPageDisplay).toBe(2)
|
||||
expect(component.pageImageUrl).toContain('/5/1/')
|
||||
component.pageLoaded.set(true)
|
||||
component.goToPage(1)
|
||||
expect(component.pageLoaded()).toBe(true)
|
||||
component.goToPage(Infinity)
|
||||
component.goToPage(NaN)
|
||||
expect(component.previewPage).toBe(1)
|
||||
component.goToPage(99)
|
||||
expect(component.previewPage).toBe(2)
|
||||
expect(component.pageLoaded()).toBe(false)
|
||||
component.goToPage(-5)
|
||||
expect(component.previewPage).toBe(0)
|
||||
component.previewPageCount.set(null)
|
||||
component.goToPage(4.7)
|
||||
expect(component.previewPage).toBe(4)
|
||||
})
|
||||
|
||||
it('limits zoom and resets it to the original size', () => {
|
||||
for (let i = 0; i < 20; i++) component.zoomIn()
|
||||
expect(component.zoom).toBe(4)
|
||||
for (let i = 0; i < 20; i++) component.zoomOut()
|
||||
expect(component.zoom).toBe(0.5)
|
||||
component.resetZoom()
|
||||
expect(component.zoom).toBe(1)
|
||||
})
|
||||
|
||||
it('moves last-page zones to the last page and can pin them to a page', () => {
|
||||
component.setSampleDocument([5])
|
||||
const selected = zone()
|
||||
component.template.zones = [selected]
|
||||
expect(component.isLastPageZone(selected)).toBe(false)
|
||||
component.setLastPageZone(selected, true)
|
||||
expect(selected.page).toBe(-1)
|
||||
expect(component.isLastPageZone(selected)).toBe(true)
|
||||
component.selectZone(0)
|
||||
expect(component.previewPage).toBe(2)
|
||||
component.setLastPageZone(selected, false)
|
||||
expect(selected.page).toBe(3)
|
||||
})
|
||||
|
||||
it('ignores page-count errors from a previous sample document', () => {
|
||||
const pending = new Subject<Document>()
|
||||
jest
|
||||
.spyOn(documentService, 'get')
|
||||
.mockReturnValueOnce(pending)
|
||||
.mockReturnValueOnce(of({ id: 6 }))
|
||||
component.setSampleDocument([5])
|
||||
component.setSampleDocument([6])
|
||||
pending.error(new Error('Old document failed'))
|
||||
expect(component.pageFailed()).toBe(false)
|
||||
expect(component.previewPageCount()).toBeNull()
|
||||
})
|
||||
|
||||
it('ignores non-primary, unloaded, and saving pointer gestures', () => {
|
||||
prepareOverlay()
|
||||
component.onOverlayPointerDown(
|
||||
new MouseEvent('pointerdown', { button: 2 }) as PointerEvent
|
||||
)
|
||||
expect(component.drawingRect()).toBeNull()
|
||||
component.pageLoaded.set(false)
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
expect(component.drawingRect()).toBeNull()
|
||||
component.pageLoaded.set(true)
|
||||
component.saving.set(true)
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
expect(component.drawingRect()).toBeNull()
|
||||
})
|
||||
|
||||
it('ignores pointer events when the SVG coordinate transform is unavailable', () => {
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
expect(component.drawingRect()).toBeNull()
|
||||
prepareOverlay()
|
||||
jest
|
||||
.spyOn(component.overlayRef.nativeElement, 'getScreenCTM')
|
||||
.mockReturnValue(null)
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
component.onOverlayPointerMove(pointer(200, 200))
|
||||
expect(component.drawingRect()).toBeNull()
|
||||
})
|
||||
|
||||
it('updates a drawing only for its captured pointer', () => {
|
||||
prepareOverlay()
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
component.onOverlayPointerDown(pointer(500, 500, 2))
|
||||
component.onOverlayPointerMove(pointer(900, 900, 2))
|
||||
expect(component.drawingRect()).toEqual({ x: 100, y: 100, w: 0, h: 0 })
|
||||
component.onOverlayPointerMove(pointer(250, 200))
|
||||
expect(component.drawingRect()).toEqual({ x: 100, y: 100, w: 150, h: 100 })
|
||||
component.onOverlayPointerUp(pointer(250, 200))
|
||||
expect(component.template.zones[0]).toMatchObject({
|
||||
x: 100,
|
||||
y: 100,
|
||||
width: 150,
|
||||
height: 100,
|
||||
})
|
||||
})
|
||||
|
||||
it('resizes a selected zone using its corner handle', () => {
|
||||
prepareOverlay()
|
||||
const selected = zone()
|
||||
component.template.zones = [selected]
|
||||
component.selectZone(0)
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
component.onOverlayPointerMove(pointer(50, 75))
|
||||
component.onOverlayPointerUp(pointer(60, 80))
|
||||
expect(selected).toMatchObject({ x: 60, y: 80, width: 240, height: 120 })
|
||||
expect(component.template.zones).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('shows the appropriate cursor over handles, zones, and empty space', () => {
|
||||
prepareOverlay()
|
||||
component.template.zones = [zone()]
|
||||
component.onOverlayPointerMove(pointer(150, 150))
|
||||
expect(component.overlayCursor).toBe('move')
|
||||
component.selectZone(0)
|
||||
component.onOverlayPointerMove(pointer(100, 100))
|
||||
expect(component.overlayCursor).toBe('nw-resize')
|
||||
component.onOverlayPointerMove(pointer(500, 500))
|
||||
expect(component.overlayCursor).toBe('crosshair')
|
||||
component.template.zones[0].page = 2
|
||||
component.onOverlayPointerMove(pointer(100, 100))
|
||||
expect(component.zoneDisplayRect(0)).toBeNull()
|
||||
expect(component.overlayCursor).toBe('crosshair')
|
||||
})
|
||||
|
||||
it('cancels only the captured pointer and tolerates an already-lost capture', () => {
|
||||
prepareOverlay()
|
||||
component.onOverlayPointerCancel(pointer(100, 100))
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
component.onOverlayPointerCancel(pointer(150, 150, 2))
|
||||
expect(component.drawingRect()).not.toBeNull()
|
||||
jest
|
||||
.spyOn(component.overlayRef.nativeElement, 'hasPointerCapture')
|
||||
.mockReturnValue(false)
|
||||
component.onOverlayPointerCancel(pointer(150, 150))
|
||||
expect(component.drawingRect()).toBeNull()
|
||||
expect(
|
||||
component.overlayRef.nativeElement.releasePointerCapture
|
||||
).not.toHaveBeenCalled()
|
||||
expect(component.template.zones).toEqual([])
|
||||
})
|
||||
|
||||
it('preserves the selected zone when an earlier or later zone is removed', () => {
|
||||
component.template.zones = [zone(), zone(), zone(), zone()]
|
||||
component.selectZone(1)
|
||||
const selected = component.selectedZone
|
||||
component.removeZone(3)
|
||||
expect(component.selectedZone).toBe(selected)
|
||||
component.removeZone(0)
|
||||
expect(component.selectedZoneIndex).toBe(0)
|
||||
expect(component.selectedZone).toBe(selected)
|
||||
component.selectZone(99)
|
||||
expect(component.selectedZone).toBeNull()
|
||||
})
|
||||
|
||||
it('does not test zones without a selection or a sample document', () => {
|
||||
const test = jest.spyOn(templateService, 'testZone')
|
||||
component.testZone()
|
||||
component.template.zones = [zone()]
|
||||
component.selectZone(0)
|
||||
component.testZone()
|
||||
expect(test).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it.each([
|
||||
{
|
||||
response: { error: { error: 'OCR unavailable' } },
|
||||
message: 'OCR unavailable',
|
||||
},
|
||||
{ response: new Error('Network failure'), message: 'Test failed' },
|
||||
])('shows zone-test errors: $message', ({ response, message }) => {
|
||||
component.template.zones = [zone({ page: undefined })]
|
||||
component.selectZone(0)
|
||||
component.setSampleDocument([5])
|
||||
jest
|
||||
.spyOn(templateService, 'testZone')
|
||||
.mockReturnValue(throwError(() => response))
|
||||
component.testZone()
|
||||
expect(component.zoneTestResult).toEqual({ error: message })
|
||||
expect(component.zoneTesting).toBe(false)
|
||||
expect(templateService.testZone).toHaveBeenCalledWith(
|
||||
5,
|
||||
expect.objectContaining({ page: 1 })
|
||||
)
|
||||
})
|
||||
|
||||
it('switches between built-in targets, custom fields, and an unassigned field', () => {
|
||||
const selected = zone()
|
||||
expect(component.zoneFieldValue(selected)).toBe(1)
|
||||
component.setZoneField(selected, OcrZoneTarget.Title)
|
||||
expect(component.zoneFieldValue(selected)).toBe(OcrZoneTarget.Title)
|
||||
expect(selected.custom_field).toBeNull()
|
||||
component.setZoneField(selected, 7)
|
||||
expect(selected).toMatchObject({
|
||||
target: OcrZoneTarget.CustomField,
|
||||
custom_field: 7,
|
||||
})
|
||||
component.setZoneField(selected, null)
|
||||
expect(component.zoneFieldValue(selected)).toBeNull()
|
||||
expect(component.zonesForField(selected)).toEqual([])
|
||||
component.setCombineFormat(selected, 'Ignored')
|
||||
expect(component.template.combine_formats).toEqual({})
|
||||
})
|
||||
|
||||
it('creates combine formats when absent and leaves implicit formats alone on rename', () => {
|
||||
const selected = zone({ target: OcrZoneTarget.Title, custom_field: null })
|
||||
component.template.combine_formats = undefined
|
||||
component.renameZone(selected, 'Renamed')
|
||||
expect(selected.name).toBe('Renamed')
|
||||
expect(component.getCombineFormat(selected)).toBe('')
|
||||
component.setCombineFormat(selected, '{Renamed} ')
|
||||
component.insertCombineToken(selected, selected)
|
||||
expect(component.template.combine_formats).toEqual({
|
||||
title: '{Renamed} {Renamed}',
|
||||
})
|
||||
})
|
||||
|
||||
it('saves templates without combine formats', () => {
|
||||
component.template.combine_formats = undefined
|
||||
fixture.detectChanges()
|
||||
expect(component.isDirty).toBe(false)
|
||||
jest
|
||||
.spyOn(templateService, 'create')
|
||||
.mockReturnValue(of(component.template))
|
||||
component.save()
|
||||
expect(component.template.combine_formats).toBeUndefined()
|
||||
})
|
||||
|
||||
it('does not assign unsupported custom field types and surfaces creation failures', () => {
|
||||
component.template.zones = [zone({ custom_field: null })]
|
||||
component.selectZone(0)
|
||||
const dialog = fieldDialog()
|
||||
const error = jest.spyOn(TestBed.inject(ToastService), 'showError')
|
||||
component.createField('Status')
|
||||
expect(dialog.object).toEqual({ name: 'Status' })
|
||||
dialog.succeeded.next(customFields[1])
|
||||
expect(component.selectedZone.custom_field).toBeNull()
|
||||
const failure = new Error('Could not save')
|
||||
dialog.failed.next(failure)
|
||||
expect(error).toHaveBeenCalledWith('Error saving field.', failure)
|
||||
})
|
||||
|
||||
it('refreshes field choices when a field is created without a selected zone', () => {
|
||||
const dialog = fieldDialog()
|
||||
component.createField('Total')
|
||||
dialog.succeeded.next(customFields[0])
|
||||
expect(component.customFields()).toEqual(customFields)
|
||||
expect(component.selectedZone).toBeNull()
|
||||
})
|
||||
|
||||
it('should edit a copy of the template and load its sample document', () => {
|
||||
const original: OcrTemplate = {
|
||||
...component.template,
|
||||
id: 3,
|
||||
name: 'Invoices',
|
||||
match: 'Northwind',
|
||||
matching_algorithm: MATCH_LITERAL,
|
||||
sample_document: 5,
|
||||
zones: [zone()],
|
||||
combine_formats: { '1': '{Invoice number}' },
|
||||
}
|
||||
component.object = original
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(component.dialogTitle).toEqual('Edit OCR Template')
|
||||
expect(component.previewDocId).toEqual(5)
|
||||
expect(component.previewPageCount()).toEqual(3)
|
||||
expect(component.pageImageUrl).toContain('document-page-image/5/0/')
|
||||
expect(component.template.match).toEqual('Northwind')
|
||||
expect(component.isDirty).toBeFalsy()
|
||||
|
||||
component.template.zones[0].x = 500
|
||||
component.template.combine_formats['1'] = 'Edited'
|
||||
expect(original.zones[0].x).toEqual(100)
|
||||
expect(original.combine_formats['1']).toEqual('{Invoice number}')
|
||||
})
|
||||
|
||||
it('should offer every matching algorithm except automatic', () => {
|
||||
expect(component.matchingAlgorithms.map((a) => a.id)).not.toContain(
|
||||
MATCH_AUTO
|
||||
)
|
||||
expect(component.template.matching_algorithm).toEqual(MATCH_ANY)
|
||||
expect(component.patternRequired).toBeTruthy()
|
||||
|
||||
component.template.matching_algorithm = MATCH_NONE
|
||||
expect(component.patternRequired).toBeFalsy()
|
||||
})
|
||||
|
||||
it('should initialize new template permissions from saved defaults', () => {
|
||||
const defaults = {
|
||||
[SETTINGS_KEYS.DEFAULT_PERMS_OWNER]: 1,
|
||||
[SETTINGS_KEYS.DEFAULT_PERMS_VIEW_USERS]: [2],
|
||||
[SETTINGS_KEYS.DEFAULT_PERMS_VIEW_GROUPS]: [3],
|
||||
[SETTINGS_KEYS.DEFAULT_PERMS_EDIT_USERS]: [],
|
||||
[SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS]: [4],
|
||||
}
|
||||
jest
|
||||
.spyOn(TestBed.inject(SettingsService), 'get')
|
||||
.mockImplementation((key) => defaults[key])
|
||||
fixture.detectChanges()
|
||||
expect(component.permissionsForm).toEqual({
|
||||
owner: 1,
|
||||
set_permissions: {
|
||||
view: { users: [2], groups: [3] },
|
||||
change: { users: [], groups: [4] },
|
||||
},
|
||||
})
|
||||
expect(
|
||||
fixture.debugElement.query(By.directive(PermissionsFormComponent))
|
||||
).toBeTruthy()
|
||||
expect(component.isDirty).toBe(false)
|
||||
const create = jest
|
||||
.spyOn(templateService, 'create')
|
||||
.mockReturnValue(of({ ...component.template, id: 3 }))
|
||||
component.save()
|
||||
expect(create).toHaveBeenCalledWith(
|
||||
expect.objectContaining(component.permissionsForm)
|
||||
)
|
||||
})
|
||||
|
||||
it('should copy existing permissions and track changes without mutating the list item', () => {
|
||||
component.object = {
|
||||
...component.template,
|
||||
id: 3,
|
||||
owner: 1,
|
||||
permissions: {
|
||||
view: { users: [2], groups: [] },
|
||||
change: { users: [], groups: [3] },
|
||||
},
|
||||
}
|
||||
fixture.detectChanges()
|
||||
expect(component.permissionsForm.owner).toBe(1)
|
||||
expect(component.isDirty).toBe(false)
|
||||
component.permissionsForm.set_permissions.view.users.push(4)
|
||||
expect(component.isDirty).toBe(true)
|
||||
expect(component.object.permissions.view.users).toEqual([2])
|
||||
const update = jest
|
||||
.spyOn(templateService, 'update')
|
||||
.mockReturnValue(of(component.template))
|
||||
component.save()
|
||||
expect(update).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
owner: 1,
|
||||
set_permissions: {
|
||||
view: { users: [2, 4], groups: [] },
|
||||
change: { users: [], groups: [3] },
|
||||
},
|
||||
})
|
||||
)
|
||||
})
|
||||
|
||||
it('should hide sharing controls and omit permission changes for a shared editor', () => {
|
||||
component.object = {
|
||||
...component.template,
|
||||
id: 3,
|
||||
owner: 2,
|
||||
user_can_change: true,
|
||||
}
|
||||
fixture.detectChanges()
|
||||
expect(
|
||||
fixture.debugElement.query(By.directive(PermissionsFormComponent))
|
||||
).toBeNull()
|
||||
component.permissionsForm = {
|
||||
owner: 1,
|
||||
set_permissions: { view: { users: [1] } },
|
||||
}
|
||||
const update = jest
|
||||
.spyOn(templateService, 'update')
|
||||
.mockReturnValue(of(component.template))
|
||||
component.save()
|
||||
expect(update.mock.calls[0][0].owner).toBe(2)
|
||||
expect(update.mock.calls[0][0]).not.toHaveProperty('set_permissions')
|
||||
})
|
||||
|
||||
it('should keep only the last picked sample document', () => {
|
||||
component.setSampleDocument([5, 6])
|
||||
expect(component.sampleDocumentIds).toEqual([6])
|
||||
|
||||
component.setSampleDocument([])
|
||||
expect(component.previewDocId).toBeNull()
|
||||
expect(component.pageImageUrl).toBeNull()
|
||||
})
|
||||
|
||||
it('should ignore a stale sample document response', () => {
|
||||
const pending = new Subject<Document>()
|
||||
jest.spyOn(documentService, 'get').mockReturnValueOnce(pending)
|
||||
component.setSampleDocument([5])
|
||||
component.setSampleDocument([6])
|
||||
pending.next({ id: 5, page_count: 99 })
|
||||
|
||||
expect(component.previewPageCount()).toEqual(3)
|
||||
})
|
||||
|
||||
it('should show a failed page and retry it', () => {
|
||||
jest
|
||||
.spyOn(documentService, 'get')
|
||||
.mockReturnValueOnce(throwError(() => new Error('offline')))
|
||||
component.setSampleDocument([5])
|
||||
expect(component.pageFailed()).toBeTruthy()
|
||||
|
||||
component.setSampleDocument(component.sampleDocumentIds)
|
||||
expect(component.pageFailed()).toBeFalsy()
|
||||
|
||||
component.onImageError()
|
||||
expect(component.pageFailed()).toBeTruthy()
|
||||
})
|
||||
|
||||
it('should support create and update', () => {
|
||||
const createSpy = jest
|
||||
.spyOn(templateService, 'create')
|
||||
.mockReturnValue(of({ ...component.template, id: 3 }))
|
||||
const updateSpy = jest
|
||||
.spyOn(templateService, 'update')
|
||||
.mockReturnValue(of(component.template))
|
||||
const succeededSpy = jest.spyOn(component.succeeded, 'emit')
|
||||
|
||||
component.save()
|
||||
expect(createSpy).toHaveBeenCalled()
|
||||
expect(succeededSpy).toHaveBeenCalledWith({ ...component.template, id: 3 })
|
||||
expect(activeModal.close).toHaveBeenCalled()
|
||||
|
||||
component.saving.set(false)
|
||||
component.template.id = 3
|
||||
component.save()
|
||||
expect(updateSpy).toHaveBeenCalledWith(component.template)
|
||||
})
|
||||
|
||||
it('should emit failed and allow editing again after a failed save', () => {
|
||||
prepareOverlay()
|
||||
component.template.zones = [zone()]
|
||||
const pending = new Subject<OcrTemplate>()
|
||||
jest.spyOn(templateService, 'create').mockReturnValue(pending)
|
||||
const failedSpy = jest.spyOn(component.failed, 'emit')
|
||||
component.onOverlayPointerDown(pointer(150, 150))
|
||||
|
||||
component.save()
|
||||
component.save()
|
||||
expect(templateService.create).toHaveBeenCalledTimes(1)
|
||||
component.onOverlayPointerMove(pointer(250, 250))
|
||||
component.onOverlayPointerUp(pointer(250, 250))
|
||||
expect(component.template.zones[0].x).toEqual(100)
|
||||
|
||||
pending.error(new Error('Save failed'))
|
||||
expect(failedSpy).toHaveBeenCalled()
|
||||
expect(component.saving()).toBeFalsy()
|
||||
component.onOverlayPointerDown(pointer(150, 150))
|
||||
component.onOverlayPointerUp(pointer(250, 250))
|
||||
expect(component.template.zones[0].x).toEqual(200)
|
||||
})
|
||||
|
||||
it('should track unsaved changes', () => {
|
||||
fixture.detectChanges()
|
||||
expect(component.isDirty).toBeFalsy()
|
||||
|
||||
component.template.combine_formats = { '1': '' }
|
||||
expect(component.isDirty).toBeFalsy()
|
||||
|
||||
component.template.name = 'Invoices'
|
||||
expect(component.isDirty).toBeTruthy()
|
||||
component.template.name = ''
|
||||
expect(component.isDirty).toBeFalsy()
|
||||
|
||||
component.setSampleDocument([5])
|
||||
expect(component.isDirty).toBeTruthy()
|
||||
})
|
||||
|
||||
it('should draw a zone with no field in empty space', () => {
|
||||
prepareOverlay()
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
component.onOverlayPointerUp(pointer(1200, 250))
|
||||
|
||||
expect(component.template.zones[0]).toMatchObject({
|
||||
x: 100,
|
||||
y: 100,
|
||||
width: 900,
|
||||
height: 150,
|
||||
custom_field: null,
|
||||
})
|
||||
expect(component.selectedZoneIndex).toEqual(0)
|
||||
})
|
||||
|
||||
it('should clear the selection on a click in empty space', () => {
|
||||
prepareOverlay()
|
||||
component.template.zones = [zone()]
|
||||
component.selectZone(0)
|
||||
component.onOverlayPointerDown(pointer(600, 600))
|
||||
component.onOverlayPointerUp(pointer(602, 601))
|
||||
|
||||
expect(component.template.zones).toHaveLength(1)
|
||||
expect(component.selectedZoneIndex).toBeNull()
|
||||
})
|
||||
|
||||
it('should move a zone when dragging on it', () => {
|
||||
prepareOverlay()
|
||||
component.template.zones = [zone()]
|
||||
component.onOverlayPointerDown(pointer(150, 150))
|
||||
component.onOverlayPointerUp(pointer(250, 200))
|
||||
|
||||
expect(component.template.zones).toHaveLength(1)
|
||||
expect(component.template.zones[0]).toMatchObject({ x: 200, y: 150 })
|
||||
})
|
||||
|
||||
it('should cancel a drawing on escape and ignore other pointers', () => {
|
||||
prepareOverlay()
|
||||
component.onOverlayPointerDown(pointer(100, 100))
|
||||
component.onOverlayPointerUp(pointer(300, 200, 2))
|
||||
expect(component.drawingRect()).not.toBeNull()
|
||||
|
||||
component.cancelDrawing()
|
||||
component.onOverlayPointerUp(pointer(300, 200))
|
||||
expect(component.template.zones).toEqual([])
|
||||
})
|
||||
|
||||
it('should open a zone in the zones tab and go back to the list', () => {
|
||||
component.template.zones = [zone(), zone()]
|
||||
component.selectZone(1)
|
||||
expect(component.activeTab).toEqual('zones')
|
||||
expect(component.selectedZone).toBe(component.template.zones[1])
|
||||
|
||||
component.deselectZone()
|
||||
expect(component.selectedZone).toBeNull()
|
||||
|
||||
component.selectZone(0)
|
||||
component.removeZone(0)
|
||||
expect(component.template.zones).toHaveLength(1)
|
||||
expect(component.selectedZone).toBeNull()
|
||||
})
|
||||
|
||||
it('should only offer custom fields OCR can write to', () => {
|
||||
fixture.detectChanges()
|
||||
expect(component.fieldItems().map((item) => item.id)).toEqual([
|
||||
OcrZoneTarget.Title,
|
||||
OcrZoneTarget.Asn,
|
||||
OcrZoneTarget.Created,
|
||||
1,
|
||||
])
|
||||
})
|
||||
|
||||
it('should create a custom field and assign it to the zone', () => {
|
||||
const succeeded = new Subject<CustomField>()
|
||||
const dialog = {
|
||||
dialogMode: { set: jest.fn() },
|
||||
object: null,
|
||||
succeeded,
|
||||
failed: new Subject(),
|
||||
}
|
||||
jest
|
||||
.spyOn(modalService, 'open')
|
||||
.mockReturnValue({ componentInstance: dialog } as any)
|
||||
component.template.zones = [zone({ custom_field: null })]
|
||||
component.selectZone(0)
|
||||
|
||||
component.createField()
|
||||
expect(dialog.object).toEqual({ name: 'Invoice number' })
|
||||
succeeded.next(customFields[0])
|
||||
|
||||
expect(component.template.zones[0].custom_field).toEqual(1)
|
||||
})
|
||||
|
||||
it('should build and prune combine formats', () => {
|
||||
const first = zone({ name: 'First' })
|
||||
const second = zone({ name: 'Second' })
|
||||
const other = zone({ name: 'Other', custom_field: 2 })
|
||||
component.template.zones = [first, second, other]
|
||||
component.template.combine_formats = { '2': '{Other}' }
|
||||
|
||||
expect(component.isFieldShared(first)).toBeTruthy()
|
||||
expect(component.isFieldShared(other)).toBeFalsy()
|
||||
expect(component.defaultCombineFormat(first)).toEqual('{First} {Second}')
|
||||
|
||||
component.insertCombineToken(first, second)
|
||||
component.insertCombineToken(first, first)
|
||||
expect(component.getCombineFormat(second)).toEqual('{Second} {First}')
|
||||
|
||||
jest.spyOn(templateService, 'create').mockReturnValue(of(null))
|
||||
component.save()
|
||||
expect(component.template.combine_formats).toEqual({
|
||||
'1': '{Second} {First}',
|
||||
})
|
||||
})
|
||||
|
||||
it('should update exact tokens when a zone is renamed', () => {
|
||||
const first = zone({ name: 'First [value]' })
|
||||
component.template.zones = [first, zone({ name: 'Second' })]
|
||||
component.template.combine_formats = {
|
||||
'1': 'First [value]: {First [value]} / { First [value] } / {Second} / {First [value] extra}',
|
||||
'2': '{First [value]}',
|
||||
}
|
||||
|
||||
component.renameZone(first, '')
|
||||
component.renameZone(first, 'Invoice number')
|
||||
|
||||
expect(first.name).toEqual('Invoice number')
|
||||
expect(component.template.combine_formats).toEqual({
|
||||
'1': 'First [value]: {Invoice number} / {Invoice number} / {Second} / {First [value] extra}',
|
||||
'2': '{First [value]}',
|
||||
})
|
||||
})
|
||||
|
||||
it('should keep the language array stable until the value changes', () => {
|
||||
const selected = zone({ ocr_language: 'deu+eng' })
|
||||
const languages = component.ocrLanguageArray(selected)
|
||||
expect(languages).toEqual(['deu', 'eng'])
|
||||
expect(component.ocrLanguageArray(selected)).toBe(languages)
|
||||
|
||||
component.setOcrLanguages(selected, ['fra'])
|
||||
expect(selected.ocr_language).toEqual('fra')
|
||||
expect(component.ocrLanguageArray(selected)).toEqual(['fra'])
|
||||
|
||||
component.setOcrLanguages(selected, null)
|
||||
expect(component.ocrLanguageArray(selected)).toEqual([])
|
||||
})
|
||||
|
||||
it('should hide a test result once the zone changes', () => {
|
||||
const selected = zone()
|
||||
component.template.zones = [selected]
|
||||
component.selectZone(0)
|
||||
component.setSampleDocument([5])
|
||||
jest
|
||||
.spyOn(templateService, 'testZone')
|
||||
.mockReturnValue(of({ value: '123' }))
|
||||
|
||||
component.testZone()
|
||||
expect(component.zoneTestResult.value).toEqual('123')
|
||||
|
||||
selected.transform = OcrZoneTransform.Numeric
|
||||
expect(component.zoneTestResult).toBeNull()
|
||||
})
|
||||
|
||||
it('should drop a pending test when the zone or sample document changes', () => {
|
||||
const pending = new Subject<OcrZoneTestResult>()
|
||||
component.template.zones = [zone(), zone()]
|
||||
component.selectZone(0)
|
||||
component.setSampleDocument([5])
|
||||
jest.spyOn(templateService, 'testZone').mockReturnValue(pending)
|
||||
|
||||
component.testZone()
|
||||
expect(component.zoneTesting).toBeTruthy()
|
||||
component.selectZone(1)
|
||||
pending.next({ value: 'Old zone' })
|
||||
expect(component.zoneTestResult).toBeNull()
|
||||
|
||||
component.testZone()
|
||||
component.setSampleDocument([])
|
||||
pending.next({ value: 'Old document' })
|
||||
expect(component.zoneTestResult).toBeNull()
|
||||
})
|
||||
})
|
||||
-963
@@ -1,963 +0,0 @@
|
||||
import { CommonModule } from '@angular/common'
|
||||
import {
|
||||
Component,
|
||||
computed,
|
||||
ElementRef,
|
||||
EventEmitter,
|
||||
HostListener,
|
||||
inject,
|
||||
Input,
|
||||
OnDestroy,
|
||||
OnInit,
|
||||
Output,
|
||||
signal,
|
||||
ViewChild,
|
||||
} from '@angular/core'
|
||||
import { FormsModule } from '@angular/forms'
|
||||
import {
|
||||
NgbActiveModal,
|
||||
NgbModal,
|
||||
NgbNavModule,
|
||||
} from '@ng-bootstrap/ng-bootstrap'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { Subject, takeUntil } from 'rxjs'
|
||||
import { CustomFieldEditDialogComponent } from 'src/app/components/common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
|
||||
import { EditDialogMode } from 'src/app/components/common/edit-dialog/edit-dialog.component'
|
||||
import { DocumentLinkComponent } from 'src/app/components/common/input/document-link/document-link.component'
|
||||
import { CheckComponent } from 'src/app/components/common/input/check/check.component'
|
||||
import { NumberComponent } from 'src/app/components/common/input/number/number.component'
|
||||
import {
|
||||
PermissionsFormComponent,
|
||||
PermissionsFormObject,
|
||||
} from 'src/app/components/common/input/permissions/permissions-form/permissions-form.component'
|
||||
import { SelectComponent } from 'src/app/components/common/input/select/select.component'
|
||||
import { SwitchComponent } from 'src/app/components/common/input/switch/switch.component'
|
||||
import { TextComponent } from 'src/app/components/common/input/text/text.component'
|
||||
import { CustomField } from 'src/app/data/custom-field'
|
||||
import {
|
||||
MATCH_ANY,
|
||||
MATCH_AUTO,
|
||||
MATCH_NONE,
|
||||
MATCHING_ALGORITHMS,
|
||||
} from 'src/app/data/matching-model'
|
||||
import {
|
||||
DEFAULT_OCR_ZONE_LANGUAGE,
|
||||
getZoneFieldKey,
|
||||
OCR_BUILTIN_TARGETS,
|
||||
OCR_SUPPORTED_FIELD_TYPES,
|
||||
OcrTemplate,
|
||||
OcrTemplateZone,
|
||||
OcrZoneTarget,
|
||||
OcrZoneTestResult,
|
||||
OcrZoneTransform,
|
||||
ZoneTestRequest,
|
||||
} from 'src/app/data/ocr-template'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { User } from 'src/app/data/user'
|
||||
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
|
||||
import { PermissionsService } from 'src/app/services/permissions.service'
|
||||
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
|
||||
import { DocumentService } from 'src/app/services/rest/document.service'
|
||||
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
|
||||
import { UserService } from 'src/app/services/rest/user.service'
|
||||
import { SettingsService } from 'src/app/services/settings.service'
|
||||
import { ToastService } from 'src/app/services/toast.service'
|
||||
import { OcrTemplateEditorZoneListComponent } from './ocr-template-editor-zone-list/ocr-template-editor-zone-list.component'
|
||||
import {
|
||||
DisplayRect,
|
||||
DrawingRect,
|
||||
findHandleAt,
|
||||
findZoneAt,
|
||||
getResizeHandles,
|
||||
getZoneColor,
|
||||
getZoneDisplayRect,
|
||||
getZonePage,
|
||||
HANDLE_SIZE,
|
||||
isZoneOnPage,
|
||||
MIN_ZONE_SIZE,
|
||||
MoveStart,
|
||||
moveZone,
|
||||
Point,
|
||||
ResizeHandle,
|
||||
resizeZone,
|
||||
} from './zone-geometry'
|
||||
|
||||
type ActiveTab = 'template' | 'zones'
|
||||
type ZoneFieldSelection = OcrZoneTarget | number | null
|
||||
type OverlayInteraction =
|
||||
| { kind: 'idle' }
|
||||
| { kind: 'drawing'; pointerId: number; rect: DrawingRect }
|
||||
| { kind: 'moving'; pointerId: number; zoneIndex: number; start: MoveStart }
|
||||
| {
|
||||
kind: 'resizing'
|
||||
pointerId: number
|
||||
zoneIndex: number
|
||||
handle: ResizeHandle
|
||||
}
|
||||
interface ZoneTest {
|
||||
zone: OcrTemplateZone
|
||||
/** The sample document and zone settings the result was produced from. */
|
||||
key: string
|
||||
result: OcrZoneTestResult | null
|
||||
}
|
||||
|
||||
export const ZONE_TRANSFORM_OPTIONS = [
|
||||
{ id: OcrZoneTransform.None, name: $localize`None` },
|
||||
{ id: OcrZoneTransform.Strip, name: $localize`Strip whitespace` },
|
||||
{ id: OcrZoneTransform.Uppercase, name: $localize`Uppercase` },
|
||||
{ id: OcrZoneTransform.Lowercase, name: $localize`Lowercase` },
|
||||
{ id: OcrZoneTransform.Numeric, name: $localize`Numeric only` },
|
||||
{
|
||||
id: OcrZoneTransform.StripPunctuation,
|
||||
name: $localize`Remove leading/trailing punctuation`,
|
||||
},
|
||||
{ id: OcrZoneTransform.Date, name: $localize`Parse date` },
|
||||
{ id: OcrZoneTransform.QrCode, name: $localize`Read QR/barcode` },
|
||||
]
|
||||
|
||||
export const ZONE_LANGUAGE_OPTIONS = [
|
||||
{ id: 'eng', name: $localize`English` },
|
||||
{ id: 'deu', name: $localize`German` },
|
||||
{ id: 'fra', name: $localize`French` },
|
||||
{ id: 'ita', name: $localize`Italian` },
|
||||
{ id: 'spa', name: $localize`Spanish` },
|
||||
{ id: 'por', name: $localize`Portuguese` },
|
||||
{ id: 'nld', name: $localize`Dutch` },
|
||||
]
|
||||
|
||||
const TEMPLATE_MATCHING_ALGORITHMS = MATCHING_ALGORITHMS.filter(
|
||||
(algorithm) => algorithm.id !== MATCH_AUTO
|
||||
)
|
||||
|
||||
const NO_OVERLAY_INTERACTION: OverlayInteraction = { kind: 'idle' }
|
||||
const RESIZE_CURSOR: Record<ResizeHandle, string> = {
|
||||
nw: 'nw-resize',
|
||||
ne: 'ne-resize',
|
||||
sw: 'sw-resize',
|
||||
se: 'se-resize',
|
||||
n: 'n-resize',
|
||||
s: 's-resize',
|
||||
w: 'w-resize',
|
||||
e: 'e-resize',
|
||||
}
|
||||
|
||||
@Component({
|
||||
selector: 'pngx-ocr-template-editor',
|
||||
imports: [
|
||||
DocumentLinkComponent,
|
||||
NumberComponent,
|
||||
TextComponent,
|
||||
CheckComponent,
|
||||
SelectComponent,
|
||||
SwitchComponent,
|
||||
CommonModule,
|
||||
FormsModule,
|
||||
NgbNavModule,
|
||||
BootstrapIconComponent,
|
||||
OcrTemplateEditorZoneListComponent,
|
||||
PermissionsFormComponent,
|
||||
IfOwnerDirective,
|
||||
],
|
||||
templateUrl: './ocr-template-editor.component.html',
|
||||
styleUrls: ['./ocr-template-editor.component.scss'],
|
||||
})
|
||||
export class OcrTemplateEditorComponent implements OnInit, OnDestroy {
|
||||
public readonly activeModal = inject(NgbActiveModal)
|
||||
private readonly templateService = inject(OcrTemplateService)
|
||||
private readonly customFieldsService = inject(CustomFieldsService)
|
||||
private readonly documentService = inject(DocumentService)
|
||||
private readonly toastService = inject(ToastService)
|
||||
private readonly modalService = inject(NgbModal)
|
||||
private readonly userService = inject(UserService)
|
||||
private readonly settingsService = inject(SettingsService)
|
||||
private readonly permissionsService = inject(PermissionsService)
|
||||
private readonly destroy$ = new Subject<void>()
|
||||
private readonly cancelZoneTest$ = new Subject<void>()
|
||||
|
||||
@Input() object: OcrTemplate
|
||||
@Output() public readonly succeeded = new EventEmitter<OcrTemplate>()
|
||||
@Output() public readonly failed = new EventEmitter()
|
||||
|
||||
@ViewChild('zoneOverlay') overlayRef: ElementRef<SVGSVGElement>
|
||||
imageRef: ElementRef<HTMLImageElement>
|
||||
private previewResizeObserver: ResizeObserver | null = null
|
||||
private readonly overlayScale = signal(1)
|
||||
|
||||
@ViewChild('pageImage')
|
||||
set pageImage(ref: ElementRef<HTMLImageElement> | undefined) {
|
||||
this.previewResizeObserver?.disconnect()
|
||||
this.imageRef = ref
|
||||
if (!ref) return
|
||||
this.previewResizeObserver = new ResizeObserver(() => {
|
||||
const image = ref.nativeElement
|
||||
if (image.naturalWidth && image.clientWidth) {
|
||||
this.overlayScale.set(image.naturalWidth / image.clientWidth)
|
||||
}
|
||||
})
|
||||
this.previewResizeObserver.observe(ref.nativeElement)
|
||||
}
|
||||
|
||||
template: OcrTemplate = {
|
||||
id: null,
|
||||
name: '',
|
||||
match: '',
|
||||
matching_algorithm: MATCH_ANY,
|
||||
is_insensitive: true,
|
||||
sample_document: null,
|
||||
source_width: 0,
|
||||
source_height: 0,
|
||||
enabled: true,
|
||||
combine_formats: {},
|
||||
zones: [],
|
||||
}
|
||||
|
||||
public readonly customFields = signal<CustomField[]>([])
|
||||
public readonly fieldItems = computed<
|
||||
{ id: ZoneFieldSelection; name: string }[]
|
||||
>(() => [
|
||||
...OCR_BUILTIN_TARGETS,
|
||||
...this.customFields()
|
||||
.filter((field) => OCR_SUPPORTED_FIELD_TYPES.includes(field.data_type))
|
||||
.map((field) => ({ id: field.id, name: field.name })),
|
||||
])
|
||||
public readonly users = signal<User[]>([])
|
||||
permissionsForm: PermissionsFormObject = {}
|
||||
public readonly OcrZoneTransform = OcrZoneTransform
|
||||
public readonly saving = signal(false)
|
||||
|
||||
/** Bound to the document picker, which works with id arrays. */
|
||||
sampleDocumentIds: number[] = []
|
||||
previewPage = 0
|
||||
public readonly previewPageCount = signal<number | null>(null)
|
||||
public readonly pageLoaded = signal(false)
|
||||
public readonly pageFailed = signal(false)
|
||||
zoom = 1
|
||||
|
||||
get previewDocId(): number | null {
|
||||
return this.sampleDocumentIds[0] ?? null
|
||||
}
|
||||
|
||||
get pageImageUrl(): string | null {
|
||||
return this.previewDocId
|
||||
? this.templateService.getPageImageUrl(
|
||||
this.previewDocId,
|
||||
this.previewPage
|
||||
)
|
||||
: null
|
||||
}
|
||||
|
||||
get transformOptions() {
|
||||
return ZONE_TRANSFORM_OPTIONS
|
||||
}
|
||||
|
||||
get languageOptions() {
|
||||
return ZONE_LANGUAGE_OPTIONS
|
||||
}
|
||||
|
||||
get matchingAlgorithms() {
|
||||
return TEMPLATE_MATCHING_ALGORITHMS
|
||||
}
|
||||
|
||||
get patternRequired(): boolean {
|
||||
return (
|
||||
this.template.matching_algorithm !== MATCH_AUTO &&
|
||||
this.template.matching_algorithm !== MATCH_NONE
|
||||
)
|
||||
}
|
||||
|
||||
public get previewPageDisplay(): number {
|
||||
return this.previewPage + 1
|
||||
}
|
||||
|
||||
public set previewPageDisplay(value: number) {
|
||||
this.goToPage(value - 1)
|
||||
}
|
||||
|
||||
activeTab: ActiveTab = 'template'
|
||||
|
||||
selectedZoneIndex: number | null = null
|
||||
private overlayInteraction: OverlayInteraction = NO_OVERLAY_INTERACTION
|
||||
overlayCursor = 'crosshair'
|
||||
|
||||
private readonly zoneTest = signal<ZoneTest | null>(null)
|
||||
|
||||
get zoneTestResult(): OcrZoneTestResult | null {
|
||||
return this.isCurrentZoneTest() ? this.zoneTest().result : null
|
||||
}
|
||||
|
||||
get zoneTesting(): boolean {
|
||||
return this.isCurrentZoneTest() && this.zoneTest().result === null
|
||||
}
|
||||
|
||||
get selectedZone(): OcrTemplateZone | null {
|
||||
return this.selectedZoneIndex !== null
|
||||
? (this.template.zones[this.selectedZoneIndex] ?? null)
|
||||
: null
|
||||
}
|
||||
|
||||
private initialState = ''
|
||||
|
||||
get isDirty(): boolean {
|
||||
return this.editorState() !== this.initialState
|
||||
}
|
||||
|
||||
private editorState(): string {
|
||||
const formats = Object.entries(this.template.combine_formats ?? {}).filter(
|
||||
([, format]) => format
|
||||
)
|
||||
return JSON.stringify({
|
||||
...this.template,
|
||||
combine_formats: Object.fromEntries(formats),
|
||||
sample_document: this.previewDocId ?? null,
|
||||
permissions_form: this.permissionsForm,
|
||||
})
|
||||
}
|
||||
|
||||
get dialogTitle(): string {
|
||||
return this.template.id
|
||||
? $localize`Edit OCR Template`
|
||||
: $localize`New OCR Template`
|
||||
}
|
||||
|
||||
ngOnInit() {
|
||||
this.userService
|
||||
.listAll()
|
||||
.pipe(takeUntil(this.destroy$))
|
||||
.subscribe((r) => this.users.set(r.results))
|
||||
this.customFieldsService
|
||||
.listAll()
|
||||
.pipe(takeUntil(this.destroy$))
|
||||
.subscribe((r) => {
|
||||
this.customFields.set(r.results)
|
||||
})
|
||||
|
||||
if (this.object) {
|
||||
this.template = {
|
||||
...this.object,
|
||||
zones: this.object.zones.map((zone) => ({ ...zone })),
|
||||
combine_formats: { ...this.object.combine_formats },
|
||||
}
|
||||
if (this.template.sample_document) {
|
||||
this.setSampleDocument([this.template.sample_document])
|
||||
}
|
||||
}
|
||||
const permissions = this.object?.permissions
|
||||
this.permissionsForm = this.object
|
||||
? {
|
||||
owner: this.object.owner,
|
||||
set_permissions: {
|
||||
view: {
|
||||
users: [...(permissions?.view.users ?? [])],
|
||||
groups: [...(permissions?.view.groups ?? [])],
|
||||
},
|
||||
change: {
|
||||
users: [...(permissions?.change.users ?? [])],
|
||||
groups: [...(permissions?.change.groups ?? [])],
|
||||
},
|
||||
},
|
||||
}
|
||||
: {
|
||||
owner: this.settingsService.get(SETTINGS_KEYS.DEFAULT_PERMS_OWNER),
|
||||
set_permissions: {
|
||||
view: {
|
||||
users: this.settingsService.get(
|
||||
SETTINGS_KEYS.DEFAULT_PERMS_VIEW_USERS
|
||||
),
|
||||
groups: this.settingsService.get(
|
||||
SETTINGS_KEYS.DEFAULT_PERMS_VIEW_GROUPS
|
||||
),
|
||||
},
|
||||
change: {
|
||||
users: this.settingsService.get(
|
||||
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_USERS
|
||||
),
|
||||
groups: this.settingsService.get(
|
||||
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
this.initialState = this.editorState()
|
||||
}
|
||||
|
||||
setSampleDocument(ids: number[]) {
|
||||
const id = ids.at(-1)
|
||||
this.sampleDocumentIds = id ? [id] : []
|
||||
this.previewPage = 0
|
||||
this.previewPageCount.set(null)
|
||||
this.resetPage()
|
||||
if (!id) return
|
||||
this.documentService
|
||||
.get(id)
|
||||
.pipe(takeUntil(this.destroy$))
|
||||
.subscribe({
|
||||
next: (doc) => {
|
||||
if (this.previewDocId !== id) return
|
||||
this.previewPageCount.set(doc.page_count ?? null)
|
||||
},
|
||||
error: () => {
|
||||
if (this.previewDocId === id) this.pageFailed.set(true)
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
private resetPage() {
|
||||
this.stopOverlayInteraction()
|
||||
this.clearZoneTest()
|
||||
this.pageLoaded.set(false)
|
||||
this.pageFailed.set(false)
|
||||
}
|
||||
|
||||
goToPage(page: number) {
|
||||
if (!Number.isFinite(page)) return
|
||||
const pageCount = this.previewPageCount()
|
||||
const max = pageCount ? pageCount - 1 : page
|
||||
const clamped = Math.max(0, Math.min(Math.trunc(page), max))
|
||||
if (clamped === this.previewPage) return
|
||||
this.previewPage = clamped
|
||||
this.resetPage()
|
||||
}
|
||||
|
||||
zoomIn() {
|
||||
this.zoom = Math.min(4, Math.round((this.zoom + 0.25) * 100) / 100)
|
||||
}
|
||||
|
||||
zoomOut() {
|
||||
this.zoom = Math.max(0.5, Math.round((this.zoom - 0.25) * 100) / 100)
|
||||
}
|
||||
|
||||
resetZoom() {
|
||||
this.zoom = 1
|
||||
}
|
||||
|
||||
zonePage(zone: OcrTemplateZone): number {
|
||||
return getZonePage(zone, this.previewPage, this.previewPageCount())
|
||||
}
|
||||
|
||||
isLastPageZone(zone: OcrTemplateZone): boolean {
|
||||
return zone.page === -1
|
||||
}
|
||||
|
||||
setLastPageZone(zone: OcrTemplateZone, lastPage: boolean) {
|
||||
zone.page = lastPage ? -1 : this.previewPage + 1
|
||||
}
|
||||
|
||||
private isOnCurrentPage(zone: OcrTemplateZone): boolean {
|
||||
return isZoneOnPage(zone, this.previewPage, this.previewPageCount())
|
||||
}
|
||||
|
||||
onImageLoad() {
|
||||
this.pageLoaded.set(true)
|
||||
const img = this.imageRef.nativeElement
|
||||
this.template.source_width = img.naturalWidth
|
||||
this.template.source_height = img.naturalHeight
|
||||
}
|
||||
|
||||
onImageError() {
|
||||
this.stopOverlayInteraction()
|
||||
this.pageFailed.set(true)
|
||||
}
|
||||
|
||||
@HostListener('document:keydown.escape')
|
||||
cancelDrawing() {
|
||||
this.stopOverlayInteraction()
|
||||
}
|
||||
|
||||
onOverlayPointerDown(event: PointerEvent) {
|
||||
if (this.saving()) return
|
||||
if (event.button !== 0 || this.overlayInteraction.kind !== 'idle') return
|
||||
const point = this.svgPointFromEvent(event)
|
||||
if (!this.pageLoaded() || !point) return
|
||||
event.preventDefault()
|
||||
|
||||
if (this.selectedZoneIndex !== null) {
|
||||
const handle = this.handleAt(point, this.selectedZoneIndex)
|
||||
if (handle) {
|
||||
this.overlayInteraction = {
|
||||
kind: 'resizing',
|
||||
pointerId: event.pointerId,
|
||||
zoneIndex: this.selectedZoneIndex,
|
||||
handle,
|
||||
}
|
||||
this.overlayRef.nativeElement.setPointerCapture(event.pointerId)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
const clickedIdx = this.zoneIndexAt(point)
|
||||
if (clickedIdx !== null) {
|
||||
this.selectZone(clickedIdx)
|
||||
const zone = this.template.zones[clickedIdx]
|
||||
this.overlayInteraction = {
|
||||
kind: 'moving',
|
||||
pointerId: event.pointerId,
|
||||
zoneIndex: clickedIdx,
|
||||
start: {
|
||||
mouseX: point.x,
|
||||
mouseY: point.y,
|
||||
zoneX: zone.x,
|
||||
zoneY: zone.y,
|
||||
},
|
||||
}
|
||||
this.overlayRef.nativeElement.setPointerCapture(event.pointerId)
|
||||
return
|
||||
}
|
||||
|
||||
// Empty space: start drawing a new zone. A click without a drag just clears
|
||||
// the selection (see onOverlayPointerUp).
|
||||
this.clearZoneTest()
|
||||
this.selectedZoneIndex = null
|
||||
this.overlayInteraction = {
|
||||
kind: 'drawing',
|
||||
pointerId: event.pointerId,
|
||||
rect: {
|
||||
startX: point.x,
|
||||
startY: point.y,
|
||||
endX: point.x,
|
||||
endY: point.y,
|
||||
},
|
||||
}
|
||||
this.overlayRef.nativeElement.setPointerCapture(event.pointerId)
|
||||
}
|
||||
|
||||
onOverlayPointerMove(event: PointerEvent) {
|
||||
if (
|
||||
this.overlayInteraction.kind !== 'idle' &&
|
||||
this.overlayInteraction.pointerId !== event.pointerId
|
||||
)
|
||||
return
|
||||
const point = this.svgPointFromEvent(event)
|
||||
if (!point) return
|
||||
|
||||
if (this.overlayInteraction.kind === 'resizing') {
|
||||
resizeZone(
|
||||
this.template.zones[this.overlayInteraction.zoneIndex],
|
||||
this.overlayInteraction.handle,
|
||||
point,
|
||||
this.imageNaturalSize()
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
if (this.overlayInteraction.kind === 'moving') {
|
||||
moveZone(
|
||||
this.template.zones[this.overlayInteraction.zoneIndex],
|
||||
point,
|
||||
this.overlayInteraction.start,
|
||||
this.imageNaturalSize()
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
if (this.overlayInteraction.kind === 'drawing') {
|
||||
this.overlayInteraction.rect.endX = point.x
|
||||
this.overlayInteraction.rect.endY = point.y
|
||||
return
|
||||
}
|
||||
|
||||
this.updateOverlayCursor(point)
|
||||
}
|
||||
|
||||
private updateOverlayCursor(point: Point) {
|
||||
if (this.selectedZoneIndex !== null) {
|
||||
const handle = this.handleAt(point, this.selectedZoneIndex)
|
||||
if (handle) {
|
||||
this.overlayCursor = RESIZE_CURSOR[handle]
|
||||
return
|
||||
}
|
||||
}
|
||||
this.overlayCursor = this.zoneIndexAt(point) !== null ? 'move' : 'crosshair'
|
||||
}
|
||||
|
||||
onOverlayPointerUp(event: PointerEvent) {
|
||||
if (this.overlayInteraction.kind === 'idle') return
|
||||
if (this.overlayInteraction.pointerId !== event.pointerId) return
|
||||
this.onOverlayPointerMove(event)
|
||||
const interaction = this.overlayInteraction
|
||||
this.stopOverlayInteraction()
|
||||
if (interaction.kind !== 'drawing') return
|
||||
|
||||
const rect = this.sourceRectFromDrawing(interaction.rect)
|
||||
|
||||
// Ignore tiny accidental clicks.
|
||||
if (rect.w < MIN_ZONE_SIZE || rect.h < MIN_ZONE_SIZE) {
|
||||
return
|
||||
}
|
||||
|
||||
this.template.zones.push(this.createZoneFromRect(rect))
|
||||
this.selectZone(this.template.zones.length - 1)
|
||||
}
|
||||
|
||||
private createZoneFromRect(rect: DisplayRect): OcrTemplateZone {
|
||||
const imageSize = this.imageNaturalSize()
|
||||
return {
|
||||
name: `Zone ${this.template.zones.length + 1}`,
|
||||
target: OcrZoneTarget.CustomField,
|
||||
custom_field: null,
|
||||
x: rect.x,
|
||||
y: rect.y,
|
||||
width: rect.w,
|
||||
height: rect.h,
|
||||
page: this.previewPageDisplay,
|
||||
ocr_language: DEFAULT_OCR_ZONE_LANGUAGE,
|
||||
transform: OcrZoneTransform.Strip,
|
||||
date_format: '',
|
||||
validation_regex: '',
|
||||
order: this.template.zones.length,
|
||||
zone_source_width: imageSize.width,
|
||||
zone_source_height: imageSize.height,
|
||||
}
|
||||
}
|
||||
|
||||
onOverlayPointerCancel(event: PointerEvent) {
|
||||
if (this.overlayInteraction.kind === 'idle') return
|
||||
if (this.overlayInteraction.pointerId !== event.pointerId) return
|
||||
this.stopOverlayInteraction()
|
||||
}
|
||||
|
||||
private stopOverlayInteraction() {
|
||||
const interaction = this.overlayInteraction
|
||||
this.overlayInteraction = NO_OVERLAY_INTERACTION
|
||||
this.overlayCursor = 'crosshair'
|
||||
const overlay = this.overlayRef?.nativeElement
|
||||
if (
|
||||
interaction.kind !== 'idle' &&
|
||||
overlay?.hasPointerCapture(interaction.pointerId)
|
||||
) {
|
||||
overlay.releasePointerCapture(interaction.pointerId)
|
||||
}
|
||||
}
|
||||
|
||||
drawingRect(): DisplayRect | null {
|
||||
return this.overlayInteraction.kind === 'drawing'
|
||||
? this.displayRectFromDrawing(this.overlayInteraction.rect)
|
||||
: null
|
||||
}
|
||||
|
||||
/** The zone's rectangle on the overlay, or null if it's on another page. */
|
||||
zoneDisplayRect(zoneIndex: number): DisplayRect | null {
|
||||
const zone = this.template.zones[zoneIndex]
|
||||
if (!this.isOnCurrentPage(zone)) return null
|
||||
return getZoneDisplayRect(zone, this.imageNaturalSize())
|
||||
}
|
||||
|
||||
private handleAt(point: Point, zoneIndex: number): ResizeHandle | null {
|
||||
const rect = this.zoneDisplayRect(zoneIndex)
|
||||
return rect ? findHandleAt(point, rect, this.overlayHandleSize()) : null
|
||||
}
|
||||
|
||||
private zoneIndexAt(point: Point): number | null {
|
||||
return findZoneAt(
|
||||
point,
|
||||
this.template.zones,
|
||||
this.previewPage,
|
||||
this.previewPageCount(),
|
||||
this.imageNaturalSize()
|
||||
)
|
||||
}
|
||||
|
||||
overlayViewBox(): string {
|
||||
const imageSize = this.imageNaturalSize()
|
||||
return `0 0 ${imageSize.width} ${imageSize.height}`
|
||||
}
|
||||
|
||||
public readonly zoneColor = getZoneColor
|
||||
|
||||
zoneLabel(zone: OcrTemplateZone, index: number): string {
|
||||
return zone.name || `Zone ${index + 1}`
|
||||
}
|
||||
|
||||
zoneLabelY(rect: DisplayRect): number {
|
||||
return Math.max(this.overlayUnitSize(14), rect.y - this.overlayUnitSize(4))
|
||||
}
|
||||
|
||||
public readonly resizeHandles = getResizeHandles
|
||||
|
||||
overlayHandleSize(): number {
|
||||
return this.overlayUnitSize(HANDLE_SIZE)
|
||||
}
|
||||
|
||||
overlayFontSize(): number {
|
||||
return this.overlayUnitSize(12)
|
||||
}
|
||||
|
||||
overlayUnitSize(screenPixels: number): number {
|
||||
return screenPixels * this.overlayScale()
|
||||
}
|
||||
|
||||
private svgPointFromEvent(event: MouseEvent): Point | null {
|
||||
const svg = this.overlayRef?.nativeElement
|
||||
const matrix = svg?.getScreenCTM()
|
||||
if (!svg || !matrix) return null
|
||||
|
||||
const point = svg.createSVGPoint()
|
||||
point.x = event.clientX
|
||||
point.y = event.clientY
|
||||
|
||||
const svgPoint = point.matrixTransform(matrix.inverse())
|
||||
const size = this.imageNaturalSize()
|
||||
return {
|
||||
x: Math.max(0, Math.min(svgPoint.x, size.width)),
|
||||
y: Math.max(0, Math.min(svgPoint.y, size.height)),
|
||||
}
|
||||
}
|
||||
|
||||
private displayRectFromDrawing(rect: DrawingRect): DisplayRect {
|
||||
return {
|
||||
x: Math.min(rect.startX, rect.endX),
|
||||
y: Math.min(rect.startY, rect.endY),
|
||||
w: Math.abs(rect.endX - rect.startX),
|
||||
h: Math.abs(rect.endY - rect.startY),
|
||||
}
|
||||
}
|
||||
|
||||
private sourceRectFromDrawing(rect: DrawingRect): DisplayRect {
|
||||
const displayRect = this.displayRectFromDrawing(rect)
|
||||
return {
|
||||
x: Math.round(displayRect.x),
|
||||
y: Math.round(displayRect.y),
|
||||
w: Math.round(displayRect.w),
|
||||
h: Math.round(displayRect.h),
|
||||
}
|
||||
}
|
||||
|
||||
private imageNaturalSize() {
|
||||
const img = this.imageRef.nativeElement
|
||||
return { width: img.naturalWidth, height: img.naturalHeight }
|
||||
}
|
||||
|
||||
removeZone(index: number) {
|
||||
this.stopOverlayInteraction()
|
||||
this.clearZoneTest()
|
||||
this.template.zones.splice(index, 1)
|
||||
if (this.selectedZoneIndex === index) {
|
||||
this.selectedZoneIndex = null
|
||||
} else if (this.selectedZoneIndex > index) {
|
||||
this.selectedZoneIndex--
|
||||
}
|
||||
}
|
||||
|
||||
selectZone(index: number) {
|
||||
this.selectedZoneIndex = index
|
||||
this.activeTab = 'zones'
|
||||
this.clearZoneTest()
|
||||
const zone = this.template.zones[index]
|
||||
if (zone) {
|
||||
this.goToPage(this.zonePage(zone) - 1)
|
||||
}
|
||||
}
|
||||
|
||||
deselectZone() {
|
||||
this.selectedZoneIndex = null
|
||||
this.clearZoneTest()
|
||||
}
|
||||
|
||||
testZone() {
|
||||
const zone = this.selectedZone
|
||||
if (!zone || !this.previewDocId) return
|
||||
this.clearZoneTest()
|
||||
const test: ZoneTest = { zone, key: this.zoneTestKey(zone), result: null }
|
||||
this.zoneTest.set(test)
|
||||
this.templateService
|
||||
.testZone(this.previewDocId, this.zoneTestRequest(zone))
|
||||
.pipe(takeUntil(this.destroy$), takeUntil(this.cancelZoneTest$))
|
||||
.subscribe({
|
||||
next: (res) => this.zoneTest.set({ ...test, result: res }),
|
||||
error: (err) =>
|
||||
this.zoneTest.set({
|
||||
...test,
|
||||
result: { error: err.error?.error || $localize`Test failed` },
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
private clearZoneTest() {
|
||||
this.cancelZoneTest$.next()
|
||||
this.zoneTest.set(null)
|
||||
}
|
||||
|
||||
/** A result is stale once its zone, settings or sample document change. */
|
||||
private isCurrentZoneTest(): boolean {
|
||||
const test = this.zoneTest()
|
||||
return (
|
||||
test?.zone === this.selectedZone &&
|
||||
test.key === this.zoneTestKey(test.zone)
|
||||
)
|
||||
}
|
||||
|
||||
private zoneTestKey(zone: OcrTemplateZone): string {
|
||||
return JSON.stringify([this.previewDocId, this.zoneTestRequest(zone)])
|
||||
}
|
||||
|
||||
private zoneTestRequest(zone: OcrTemplateZone): ZoneTestRequest {
|
||||
return {
|
||||
name: zone.name,
|
||||
x: zone.x,
|
||||
y: zone.y,
|
||||
width: zone.width,
|
||||
height: zone.height,
|
||||
page: zone.page ?? 1,
|
||||
ocr_language: zone.ocr_language,
|
||||
transform: zone.transform,
|
||||
date_format: zone.date_format,
|
||||
validation_regex: zone.validation_regex,
|
||||
zone_source_width: zone.zone_source_width,
|
||||
zone_source_height: zone.zone_source_height,
|
||||
}
|
||||
}
|
||||
|
||||
save() {
|
||||
if (this.saving()) return
|
||||
this.stopOverlayInteraction()
|
||||
this.saving.set(true)
|
||||
this.pruneCombineFormats()
|
||||
this.template.sample_document = this.previewDocId
|
||||
const payload = this.permissionsService.currentUserOwnsObject(this.object)
|
||||
? { ...this.template, ...this.permissionsForm }
|
||||
: this.template
|
||||
const obs = this.template.id
|
||||
? this.templateService.update(payload)
|
||||
: this.templateService.create(payload)
|
||||
|
||||
obs.pipe(takeUntil(this.destroy$)).subscribe({
|
||||
next: (saved) => {
|
||||
this.activeModal.close()
|
||||
this.succeeded.emit(saved)
|
||||
},
|
||||
error: (e) => {
|
||||
this.saving.set(false)
|
||||
this.failed.emit(e)
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
// ng-select needs a stable array between change detection runs, so the
|
||||
// split language list is cached per zone until ocr_language changes.
|
||||
private readonly ocrLanguageCache = new WeakMap<
|
||||
OcrTemplateZone,
|
||||
{ language: string; languages: string[] }
|
||||
>()
|
||||
|
||||
ocrLanguageArray(zone: OcrTemplateZone): string[] {
|
||||
const language = zone.ocr_language || ''
|
||||
const cached = this.ocrLanguageCache.get(zone)
|
||||
if (cached?.language === language) return cached.languages
|
||||
const languages = language.split('+').filter(Boolean)
|
||||
this.ocrLanguageCache.set(zone, { language, languages })
|
||||
return languages
|
||||
}
|
||||
|
||||
setOcrLanguages(zone: OcrTemplateZone, langs: string[]) {
|
||||
zone.ocr_language = (langs ?? []).join('+')
|
||||
}
|
||||
|
||||
/** The field select's value: a built-in target, or a custom field id. */
|
||||
zoneFieldValue(zone: OcrTemplateZone): ZoneFieldSelection {
|
||||
return zone.target === OcrZoneTarget.CustomField
|
||||
? zone.custom_field
|
||||
: zone.target
|
||||
}
|
||||
|
||||
setZoneField(zone: OcrTemplateZone, value: ZoneFieldSelection) {
|
||||
if (typeof value === 'string') {
|
||||
zone.target = value
|
||||
zone.custom_field = null
|
||||
} else {
|
||||
zone.target = OcrZoneTarget.CustomField
|
||||
zone.custom_field = value
|
||||
}
|
||||
}
|
||||
|
||||
zonesForField(zone: OcrTemplateZone): OcrTemplateZone[] {
|
||||
const key = getZoneFieldKey(zone)
|
||||
if (!key) return []
|
||||
return this.template.zones.filter((z) => getZoneFieldKey(z) === key)
|
||||
}
|
||||
|
||||
isFieldShared(zone: OcrTemplateZone): boolean {
|
||||
return this.zonesForField(zone).length > 1
|
||||
}
|
||||
|
||||
renameZone(zone: OcrTemplateZone, name: string) {
|
||||
const oldName = zone.name.trim()
|
||||
const format = this.getCombineFormat(zone)
|
||||
if (format) {
|
||||
this.setCombineFormat(
|
||||
zone,
|
||||
format.replace(/\{([^{}]*)\}/g, (token, tokenName: string) =>
|
||||
tokenName.trim() === oldName ? `{${name}}` : token
|
||||
)
|
||||
)
|
||||
}
|
||||
zone.name = name
|
||||
}
|
||||
|
||||
getCombineFormat(zone: OcrTemplateZone): string {
|
||||
return this.template.combine_formats?.[getZoneFieldKey(zone)] ?? ''
|
||||
}
|
||||
|
||||
setCombineFormat(zone: OcrTemplateZone, value: string) {
|
||||
const key = getZoneFieldKey(zone)
|
||||
if (!key) return
|
||||
this.template.combine_formats ??= {}
|
||||
this.template.combine_formats[key] = value
|
||||
}
|
||||
|
||||
defaultCombineFormat(zone: OcrTemplateZone): string {
|
||||
return this.zonesForField(zone)
|
||||
.map((z) => `{${z.name}}`)
|
||||
.join(' ')
|
||||
}
|
||||
|
||||
insertCombineToken(zone: OcrTemplateZone, tokenZone: OcrTemplateZone) {
|
||||
const current = this.getCombineFormat(zone)
|
||||
const sep = current && !current.endsWith(' ') ? ' ' : ''
|
||||
this.setCombineFormat(zone, `${current}${sep}{${tokenZone.name}}`)
|
||||
}
|
||||
|
||||
/** Drop formats that are empty or whose field no longer has several zones. */
|
||||
private pruneCombineFormats() {
|
||||
const formats = this.template.combine_formats
|
||||
if (!formats) return
|
||||
const keys = this.template.zones.map(getZoneFieldKey)
|
||||
for (const [key, format] of Object.entries(formats)) {
|
||||
if (!format || keys.filter((k) => k === key).length < 2) {
|
||||
delete formats[key]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
createField(name?: string) {
|
||||
const zone = this.selectedZone
|
||||
const modal = this.modalService.open(CustomFieldEditDialogComponent)
|
||||
modal.componentInstance.dialogMode.set(EditDialogMode.CREATE)
|
||||
modal.componentInstance.object = { name: name || zone?.name }
|
||||
modal.componentInstance.succeeded
|
||||
.pipe(takeUntil(this.destroy$))
|
||||
.subscribe((field: CustomField) => {
|
||||
this.toastService.showSuccess($localize`Saved field "${field.name}".`)
|
||||
this.customFieldsService.clearCache()
|
||||
this.customFieldsService
|
||||
.listAll()
|
||||
.pipe(takeUntil(this.destroy$))
|
||||
.subscribe((r) => {
|
||||
if (zone && OCR_SUPPORTED_FIELD_TYPES.includes(field.data_type)) {
|
||||
this.setZoneField(zone, field.id)
|
||||
}
|
||||
this.customFields.set(r.results)
|
||||
})
|
||||
})
|
||||
modal.componentInstance.failed
|
||||
.pipe(takeUntil(this.destroy$))
|
||||
.subscribe((e) => {
|
||||
this.toastService.showError($localize`Error saving field.`, e)
|
||||
})
|
||||
}
|
||||
|
||||
ngOnDestroy() {
|
||||
this.previewResizeObserver?.disconnect()
|
||||
this.cancelZoneTest$.complete()
|
||||
this.destroy$.next()
|
||||
this.destroy$.complete()
|
||||
}
|
||||
}
|
||||
-197
@@ -1,197 +0,0 @@
|
||||
import {
|
||||
OcrTemplateZone,
|
||||
OcrZoneTarget,
|
||||
OcrZoneTransform,
|
||||
} from 'src/app/data/ocr-template'
|
||||
import {
|
||||
findHandleAt,
|
||||
findZoneAt,
|
||||
getResizeHandles,
|
||||
getZoneDisplayRect,
|
||||
getZonePage,
|
||||
isZoneOnPage,
|
||||
moveZone,
|
||||
resizeZone,
|
||||
} from './zone-geometry'
|
||||
|
||||
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
|
||||
return {
|
||||
name: 'Zone',
|
||||
target: OcrZoneTarget.CustomField,
|
||||
custom_field: 1,
|
||||
x: 100,
|
||||
y: 200,
|
||||
width: 300,
|
||||
height: 400,
|
||||
page: 1,
|
||||
ocr_language: 'eng',
|
||||
transform: OcrZoneTransform.Strip,
|
||||
validation_regex: '',
|
||||
order: 0,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
describe('OCR template editor geometry', () => {
|
||||
it('normalizes zone pages', () => {
|
||||
expect(getZonePage(zone({ page: 2 }), 0, 5)).toBe(2)
|
||||
expect(getZonePage(zone({ page: -1 }), 0, 5)).toBe(5)
|
||||
expect(getZonePage(zone({ page: -1 }), 2, null)).toBe(3)
|
||||
expect(getZonePage(zone({ page: 0 }), 0, 5)).toBe(1)
|
||||
expect(getZonePage(zone({ page: undefined }), 0, 5)).toBe(1)
|
||||
})
|
||||
|
||||
it('checks whether a zone is on the current preview page', () => {
|
||||
expect(isZoneOnPage(zone({ page: 2 }), 1, 5)).toBe(true)
|
||||
expect(isZoneOnPage(zone({ page: 2 }), 0, 5)).toBe(false)
|
||||
expect(isZoneOnPage(zone({ page: -1 }), 4, 5)).toBe(true)
|
||||
})
|
||||
|
||||
it('uses natural image coordinates when source dimensions are unset', () => {
|
||||
expect(
|
||||
getZoneDisplayRect(zone({ x: 100, y: 200, width: 300, height: 400 }), {
|
||||
width: 1000,
|
||||
height: 2000,
|
||||
})
|
||||
).toEqual({ x: 100, y: 200, w: 300, h: 400 })
|
||||
})
|
||||
|
||||
it('uses per-zone source dimensions when present', () => {
|
||||
expect(
|
||||
getZoneDisplayRect(
|
||||
zone({
|
||||
x: 100,
|
||||
y: 100,
|
||||
width: 100,
|
||||
height: 100,
|
||||
zone_source_width: 1000,
|
||||
zone_source_height: 1000,
|
||||
}),
|
||||
{ width: 500, height: 750 }
|
||||
)
|
||||
).toEqual({ x: 50, y: 75, w: 50, h: 75 })
|
||||
})
|
||||
|
||||
it('finds zones from topmost to bottommost on the current page', () => {
|
||||
const zones = [
|
||||
zone({ name: 'first', x: 0, y: 0, width: 100, height: 100, page: 1 }),
|
||||
zone({ name: 'second', x: 0, y: 0, width: 50, height: 50, page: 1 }),
|
||||
zone({ name: 'third', x: 0, y: 0, width: 50, height: 50, page: 2 }),
|
||||
]
|
||||
|
||||
expect(
|
||||
findZoneAt({ x: 25, y: 25 }, zones, 0, 2, { width: 100, height: 100 })
|
||||
).toBe(1)
|
||||
})
|
||||
|
||||
it('finds resize handles around a display rect', () => {
|
||||
const rect = { x: 10, y: 20, w: 100, h: 200 }
|
||||
|
||||
expect(findHandleAt({ x: 10, y: 20 }, rect)).toBe('nw')
|
||||
expect(findHandleAt({ x: 110, y: 220 }, rect)).toBe('se')
|
||||
expect(findHandleAt({ x: 60, y: 20 }, rect)).toBe('n')
|
||||
expect(findHandleAt({ x: 90, y: 160 }, rect)).toBeNull()
|
||||
expect(findHandleAt({ x: 19, y: 20 }, rect, 8)).toBeNull()
|
||||
expect(findHandleAt({ x: 19, y: 20 }, rect, 10)).toBe('nw')
|
||||
})
|
||||
|
||||
it('returns no zone when the point is outside every rectangle', () => {
|
||||
expect(
|
||||
findZoneAt({ x: 900, y: 900 }, [zone()], 0, 1, {
|
||||
width: 1000,
|
||||
height: 1000,
|
||||
})
|
||||
).toBeNull()
|
||||
})
|
||||
|
||||
it('resizes north-west edges while keeping the opposite corner fixed', () => {
|
||||
const z = zone({ x: 100, y: 200, width: 300, height: 400 })
|
||||
resizeZone(z, 'nw', { x: 50, y: 100 }, { width: 1000, height: 1000 })
|
||||
expect(z).toMatchObject({ x: 50, y: 100, width: 350, height: 500 })
|
||||
resizeZone(z, 'nw', { x: 900, y: 900 }, { width: 1000, height: 1000 })
|
||||
expect(z).toMatchObject({ x: 390, y: 590, width: 10, height: 10 })
|
||||
})
|
||||
|
||||
it('clamps resized north-west edges to the source image bounds', () => {
|
||||
const z = zone({
|
||||
x: 50,
|
||||
y: 50,
|
||||
width: 100,
|
||||
height: 100,
|
||||
zone_source_width: 200,
|
||||
zone_source_height: 300,
|
||||
})
|
||||
resizeZone(z, 'nw', { x: -50, y: -50 }, { width: 400, height: 600 })
|
||||
expect(z).toMatchObject({ x: 0, y: 0, width: 150, height: 150 })
|
||||
})
|
||||
|
||||
it('uses the rendered handle positions for hit testing', () => {
|
||||
const rect = { x: 10, y: 20, w: 100, h: 200 }
|
||||
const handles = getResizeHandles(rect)
|
||||
|
||||
expect(handles).toEqual([
|
||||
{ handle: 'nw', x: 10, y: 20 },
|
||||
{ handle: 'n', x: 60, y: 20 },
|
||||
{ handle: 'ne', x: 110, y: 20 },
|
||||
{ handle: 'w', x: 10, y: 120 },
|
||||
{ handle: 'e', x: 110, y: 120 },
|
||||
{ handle: 'sw', x: 10, y: 220 },
|
||||
{ handle: 's', x: 60, y: 220 },
|
||||
{ handle: 'se', x: 110, y: 220 },
|
||||
])
|
||||
for (const marker of handles) {
|
||||
expect(findHandleAt(marker, rect)).toBe(marker.handle)
|
||||
}
|
||||
})
|
||||
|
||||
it('moves zones without leaving source image bounds', () => {
|
||||
const z = zone({ x: 50, y: 50, width: 100, height: 100 })
|
||||
|
||||
moveZone(
|
||||
z,
|
||||
{ x: 500, y: 500 },
|
||||
{ mouseX: 50, mouseY: 50, zoneX: 50, zoneY: 50 },
|
||||
{ width: 500, height: 500 }
|
||||
)
|
||||
|
||||
expect(z.x).toBe(400)
|
||||
expect(z.y).toBe(400)
|
||||
})
|
||||
|
||||
it('resizes zones without leaving source image bounds', () => {
|
||||
const z = zone({
|
||||
x: 50,
|
||||
y: 50,
|
||||
width: 100,
|
||||
height: 100,
|
||||
zone_source_width: 200,
|
||||
zone_source_height: 200,
|
||||
})
|
||||
|
||||
resizeZone(z, 'se', { x: 500, y: 500 }, { width: 500, height: 500 })
|
||||
|
||||
expect(z.width).toBe(150)
|
||||
expect(z.height).toBe(150)
|
||||
})
|
||||
|
||||
it('moves zones in their original source coordinates on a different-size preview', () => {
|
||||
const z = zone({
|
||||
x: 50,
|
||||
y: 50,
|
||||
width: 100,
|
||||
height: 100,
|
||||
zone_source_width: 1000,
|
||||
zone_source_height: 2000,
|
||||
})
|
||||
|
||||
moveZone(
|
||||
z,
|
||||
{ x: 75, y: 75 },
|
||||
{ mouseX: 25, mouseY: 25, zoneX: 50, zoneY: 50 },
|
||||
{ width: 500, height: 500 }
|
||||
)
|
||||
|
||||
expect(z.x).toBe(150)
|
||||
expect(z.y).toBe(250)
|
||||
})
|
||||
})
|
||||
@@ -1,204 +0,0 @@
|
||||
import { OcrTemplateZone } from 'src/app/data/ocr-template'
|
||||
|
||||
export interface DrawingRect {
|
||||
startX: number
|
||||
startY: number
|
||||
endX: number
|
||||
endY: number
|
||||
}
|
||||
|
||||
export interface Dimensions {
|
||||
width: number
|
||||
height: number
|
||||
}
|
||||
|
||||
export interface Point {
|
||||
x: number
|
||||
y: number
|
||||
}
|
||||
|
||||
export interface DisplayRect {
|
||||
x: number
|
||||
y: number
|
||||
w: number
|
||||
h: number
|
||||
}
|
||||
|
||||
export interface MoveStart {
|
||||
mouseX: number
|
||||
mouseY: number
|
||||
zoneX: number
|
||||
zoneY: number
|
||||
}
|
||||
|
||||
export type ResizeHandle = 'n' | 's' | 'e' | 'w' | 'ne' | 'nw' | 'se' | 'sw'
|
||||
|
||||
export interface ResizeHandleMarker extends Point {
|
||||
handle: ResizeHandle
|
||||
}
|
||||
|
||||
export const HANDLE_SIZE = 8
|
||||
export const MIN_ZONE_SIZE = 10
|
||||
|
||||
const ZONE_COLORS = [
|
||||
'#4f8ff7',
|
||||
'#ff6b6b',
|
||||
'#51cf66',
|
||||
'#ffd43b',
|
||||
'#cc5de8',
|
||||
'#ff922b',
|
||||
'#20c997',
|
||||
'#e599f7',
|
||||
]
|
||||
|
||||
/** Outline color for a zone, shared by the preview overlay and the zone list. */
|
||||
export function getZoneColor(index: number): string {
|
||||
return ZONE_COLORS[index % ZONE_COLORS.length]
|
||||
}
|
||||
|
||||
export function getZonePage(
|
||||
zone: OcrTemplateZone,
|
||||
previewPage: number,
|
||||
previewPageCount: number | null
|
||||
): number {
|
||||
const page = zone.page ?? 1
|
||||
if (page === -1) return previewPageCount ?? previewPage + 1
|
||||
return Math.max(page, 1)
|
||||
}
|
||||
|
||||
export function isZoneOnPage(
|
||||
zone: OcrTemplateZone,
|
||||
previewPage: number,
|
||||
previewPageCount: number | null
|
||||
): boolean {
|
||||
return getZonePage(zone, previewPage, previewPageCount) === previewPage + 1
|
||||
}
|
||||
|
||||
function getZoneSourceSize(
|
||||
zone: OcrTemplateZone,
|
||||
imageSize: Dimensions
|
||||
): Dimensions {
|
||||
return {
|
||||
width: zone.zone_source_width || imageSize.width,
|
||||
height: zone.zone_source_height || imageSize.height,
|
||||
}
|
||||
}
|
||||
|
||||
export function getZoneDisplayRect(
|
||||
zone: OcrTemplateZone,
|
||||
imageSize: Dimensions
|
||||
): DisplayRect {
|
||||
const sourceSize = getZoneSourceSize(zone, imageSize)
|
||||
// The SVG viewBox uses the preview image's natural dimensions.
|
||||
const scaleX = imageSize.width / sourceSize.width
|
||||
const scaleY = imageSize.height / sourceSize.height
|
||||
|
||||
return {
|
||||
x: zone.x * scaleX,
|
||||
y: zone.y * scaleY,
|
||||
w: zone.width * scaleX,
|
||||
h: zone.height * scaleY,
|
||||
}
|
||||
}
|
||||
|
||||
export function getResizeHandles(rect: DisplayRect): ResizeHandleMarker[] {
|
||||
return [
|
||||
{ handle: 'nw', x: rect.x, y: rect.y },
|
||||
{ handle: 'n', x: rect.x + rect.w / 2, y: rect.y },
|
||||
{ handle: 'ne', x: rect.x + rect.w, y: rect.y },
|
||||
{ handle: 'w', x: rect.x, y: rect.y + rect.h / 2 },
|
||||
{ handle: 'e', x: rect.x + rect.w, y: rect.y + rect.h / 2 },
|
||||
{ handle: 'sw', x: rect.x, y: rect.y + rect.h },
|
||||
{ handle: 's', x: rect.x + rect.w / 2, y: rect.y + rect.h },
|
||||
{ handle: 'se', x: rect.x + rect.w, y: rect.y + rect.h },
|
||||
]
|
||||
}
|
||||
|
||||
export function findHandleAt(
|
||||
point: Point,
|
||||
rect: DisplayRect,
|
||||
handleSize = HANDLE_SIZE
|
||||
): ResizeHandle | null {
|
||||
return (
|
||||
getResizeHandles(rect).find(
|
||||
({ x, y }) =>
|
||||
Math.abs(point.x - x) <= handleSize &&
|
||||
Math.abs(point.y - y) <= handleSize
|
||||
)?.handle ?? null
|
||||
)
|
||||
}
|
||||
|
||||
export function findZoneAt(
|
||||
point: Point,
|
||||
zones: OcrTemplateZone[],
|
||||
previewPage: number,
|
||||
previewPageCount: number | null,
|
||||
imageSize: Dimensions
|
||||
): number | null {
|
||||
for (let i = zones.length - 1; i >= 0; i--) {
|
||||
const zone = zones[i]
|
||||
if (!isZoneOnPage(zone, previewPage, previewPageCount)) continue
|
||||
const rect = getZoneDisplayRect(zone, imageSize)
|
||||
|
||||
if (
|
||||
point.x >= rect.x &&
|
||||
point.x <= rect.x + rect.w &&
|
||||
point.y >= rect.y &&
|
||||
point.y <= rect.y + rect.h
|
||||
) {
|
||||
return i
|
||||
}
|
||||
}
|
||||
|
||||
return null
|
||||
}
|
||||
|
||||
export function moveZone(
|
||||
zone: OcrTemplateZone,
|
||||
point: Point,
|
||||
moveStart: MoveStart,
|
||||
imageSize: Dimensions
|
||||
) {
|
||||
const sourceSize = getZoneSourceSize(zone, imageSize)
|
||||
const scaleX = sourceSize.width / imageSize.width
|
||||
const scaleY = sourceSize.height / imageSize.height
|
||||
const dx = Math.round((point.x - moveStart.mouseX) * scaleX)
|
||||
const dy = Math.round((point.y - moveStart.mouseY) * scaleY)
|
||||
|
||||
zone.x = clamp(moveStart.zoneX + dx, 0, sourceSize.width - zone.width)
|
||||
zone.y = clamp(moveStart.zoneY + dy, 0, sourceSize.height - zone.height)
|
||||
}
|
||||
|
||||
export function resizeZone(
|
||||
zone: OcrTemplateZone,
|
||||
handle: ResizeHandle,
|
||||
point: Point,
|
||||
imageSize: Dimensions
|
||||
) {
|
||||
const sourceSize = getZoneSourceSize(zone, imageSize)
|
||||
const scaleX = sourceSize.width / imageSize.width
|
||||
const scaleY = sourceSize.height / imageSize.height
|
||||
const imageX = clamp(Math.round(point.x * scaleX), 0, sourceSize.width)
|
||||
const imageY = clamp(Math.round(point.y * scaleY), 0, sourceSize.height)
|
||||
|
||||
if (handle.includes('w')) {
|
||||
const right = Math.min(zone.x + zone.width, sourceSize.width)
|
||||
zone.x = clamp(imageX, 0, right - MIN_ZONE_SIZE)
|
||||
zone.width = right - zone.x
|
||||
}
|
||||
if (handle.includes('e')) {
|
||||
zone.width = Math.max(MIN_ZONE_SIZE, imageX - zone.x)
|
||||
}
|
||||
if (handle.includes('n')) {
|
||||
const bottom = Math.min(zone.y + zone.height, sourceSize.height)
|
||||
zone.y = clamp(imageY, 0, bottom - MIN_ZONE_SIZE)
|
||||
zone.height = bottom - zone.y
|
||||
}
|
||||
if (handle.includes('s')) {
|
||||
zone.height = Math.max(MIN_ZONE_SIZE, imageY - zone.y)
|
||||
}
|
||||
}
|
||||
|
||||
function clamp(value: number, min: number, max: number): number {
|
||||
return Math.max(min, Math.min(value, max))
|
||||
}
|
||||
@@ -1,75 +0,0 @@
|
||||
<pngx-page-header
|
||||
title="OCR Templates"
|
||||
i18n-title
|
||||
info="Define extraction zones on document types to automatically populate custom fields via OCR."
|
||||
i18n-info
|
||||
>
|
||||
<button type="button" class="btn btn-sm btn-outline-primary" (click)="editTemplate()" *pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.OcrTemplate }">
|
||||
<pngx-bs-icon name="plus-circle" class="me-1"></pngx-bs-icon><ng-container i18n>Add Template</ng-container>
|
||||
</button>
|
||||
</pngx-page-header>
|
||||
|
||||
<ul class="list-group">
|
||||
|
||||
<li class="list-group-item">
|
||||
<div class="row">
|
||||
<div class="col" i18n>Name</div>
|
||||
<div class="col d-none d-sm-flex" i18n>Matching</div>
|
||||
<div class="col d-none d-sm-flex" i18n>Zones</div>
|
||||
<div class="col" i18n>Status</div>
|
||||
<div class="col" i18n>Actions</div>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
@if (loading() && templates().length === 0) {
|
||||
<li class="list-group-item">
|
||||
<output class="spinner-border spinner-border-sm me-2"></output>
|
||||
<ng-container i18n>Loading...</ng-container>
|
||||
</li>
|
||||
}
|
||||
|
||||
@for (template of templates(); track template.id) {
|
||||
<li class="list-group-item">
|
||||
<div class="row fade" [class.show]="show()">
|
||||
<div class="col d-flex align-items-center"><button class="btn btn-link p-0 text-start" type="button" (click)="editTemplate(template)" [disabled]="!userCanEdit(template)">{{template.name}}</button></div>
|
||||
<div class="col d-flex align-items-center d-none d-sm-flex">{{getMatching(template)}}</div>
|
||||
<div class="col d-flex align-items-center d-none d-sm-flex"><code>{{template.zones.length}}</code></div>
|
||||
<div class="col d-flex align-items-center">
|
||||
<div class="form-check form-switch mb-0">
|
||||
<input type="checkbox" class="form-check-input cursor-pointer" [id]="template.id+'_enable'" [(ngModel)]="template.enabled" (change)="toggleTemplateEnabled(template)" [disabled]="!userCanEdit(template)" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }">
|
||||
<label class="form-check-label cursor-pointer" [for]="template.id+'_enable'">
|
||||
<code> @if(template.enabled) { <ng-container i18n>Enabled</ng-container> } @else { <span i18n class="text-muted">Disabled</span> }</code>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col">
|
||||
|
||||
<div class="btn-group d-block d-sm-none">
|
||||
<div ngbDropdown container="body" class="d-inline-block">
|
||||
<button type="button" class="btn btn-link" id="actionsMenuMobile{{template.id}}" (click)="$event.stopPropagation()" ngbDropdownToggle>
|
||||
<pngx-bs-icon name="three-dots-vertical"></pngx-bs-icon>
|
||||
</button>
|
||||
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile{{template.id}}">
|
||||
<button (click)="editTemplate(template)" [disabled]="!userCanEdit(template)" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }" ngbDropdownItem i18n>Edit</button>
|
||||
<button (click)="deleteTemplate(template)" [disabled]="!permissionsService.currentUserOwnsObject(template)" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.OcrTemplate }" ngbDropdownItem i18n>Delete</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="btn-toolbar d-none d-sm-flex gap-2" role="toolbar">
|
||||
<div class="btn-group">
|
||||
<button *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }" [disabled]="!userCanEdit(template)" class="btn btn-sm btn-outline-secondary" type="button" (click)="editTemplate(template)">
|
||||
<pngx-bs-icon width="1em" height="1em" name="pencil" class="me-1"></pngx-bs-icon><ng-container i18n>Edit</ng-container>
|
||||
</button>
|
||||
<button *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.OcrTemplate }" [disabled]="!permissionsService.currentUserOwnsObject(template)" class="btn btn-sm btn-outline-danger" type="button" (click)="deleteTemplate(template)">
|
||||
<pngx-bs-icon width="1em" height="1em" name="trash" class="me-1"></pngx-bs-icon><ng-container i18n>Delete</ng-container>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</li>
|
||||
}
|
||||
@if (!loading() && templates().length === 0) {
|
||||
<li class="list-group-item" [class.show]="show()" i18n>No OCR templates defined.</li>
|
||||
}
|
||||
</ul>
|
||||
@@ -1,300 +0,0 @@
|
||||
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
|
||||
import { provideHttpClientTesting } from '@angular/common/http/testing'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { By } from '@angular/platform-browser'
|
||||
import { NgbModal, NgbModalRef } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { of, throwError } from 'rxjs'
|
||||
import { MATCH_LITERAL, MATCH_NONE } from 'src/app/data/matching-model'
|
||||
import { OcrTemplate } from 'src/app/data/ocr-template'
|
||||
import { User } from 'src/app/data/user'
|
||||
import { DirtyFormGuard } from 'src/app/guards/dirty-form.guard'
|
||||
import { PermissionsService } from 'src/app/services/permissions.service'
|
||||
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
|
||||
import { UserService } from 'src/app/services/rest/user.service'
|
||||
import { ToastService } from 'src/app/services/toast.service'
|
||||
import { ConfirmDialogComponent } from '../../common/confirm-dialog/confirm-dialog.component'
|
||||
import { OcrTemplateEditorComponent } from './ocr-template-editor/ocr-template-editor.component'
|
||||
import { OcrTemplatesComponent } from './ocr-templates.component'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
|
||||
const templates: OcrTemplate[] = [
|
||||
{
|
||||
id: 1,
|
||||
name: 'Invoices',
|
||||
match: 'Northwind',
|
||||
matching_algorithm: MATCH_LITERAL,
|
||||
sample_document: null,
|
||||
source_width: 0,
|
||||
source_height: 0,
|
||||
enabled: true,
|
||||
zones: [],
|
||||
},
|
||||
]
|
||||
|
||||
describe('OcrTemplatesComponent', () => {
|
||||
let component: OcrTemplatesComponent
|
||||
let fixture: ComponentFixture<OcrTemplatesComponent>
|
||||
let ocrTemplateService: OcrTemplateService
|
||||
let modalService: NgbModal
|
||||
let toastService: ToastService
|
||||
let modal: NgbModalRef
|
||||
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
imports: [OcrTemplatesComponent, BootstrapIconComponent],
|
||||
providers: [
|
||||
provideHttpClient(withInterceptorsFromDi()),
|
||||
provideHttpClientTesting(),
|
||||
],
|
||||
})
|
||||
|
||||
TestBed.inject(PermissionsService).initialize(
|
||||
[
|
||||
'view_ocrtemplate',
|
||||
'add_ocrtemplate',
|
||||
'change_ocrtemplate',
|
||||
'delete_ocrtemplate',
|
||||
],
|
||||
{ id: 1, groups: [], is_superuser: false } as User
|
||||
)
|
||||
jest
|
||||
.spyOn(TestBed.inject(UserService), 'listAll')
|
||||
.mockReturnValue(of({ count: 0, all: [], results: [] }))
|
||||
|
||||
ocrTemplateService = TestBed.inject(OcrTemplateService)
|
||||
jest
|
||||
.spyOn(ocrTemplateService, 'listAll')
|
||||
.mockReturnValue(of({ count: 1, all: [1], results: templates }))
|
||||
modalService = TestBed.inject(NgbModal)
|
||||
modalService.activeInstances.subscribe((m) => (modal = m[m.length - 1]))
|
||||
toastService = TestBed.inject(ToastService)
|
||||
fixture = TestBed.createComponent(OcrTemplatesComponent)
|
||||
component = fixture.componentInstance
|
||||
fixture.detectChanges()
|
||||
})
|
||||
|
||||
afterEach(() => modalService.dismissAll())
|
||||
|
||||
it('should show templates with their matching', () => {
|
||||
const text = fixture.nativeElement.textContent
|
||||
expect(text).toContain('Invoices')
|
||||
expect(text).toContain('Exact match: Northwind')
|
||||
|
||||
expect(
|
||||
component.getMatching({ ...templates[0], matching_algorithm: MATCH_NONE })
|
||||
).toEqual('None')
|
||||
expect(component.getMatching({ ...templates[0], match: '' })).toEqual('-')
|
||||
})
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: 'owned',
|
||||
owner: 1,
|
||||
user_can_change: true,
|
||||
editable: true,
|
||||
deletable: true,
|
||||
},
|
||||
{
|
||||
name: 'unowned',
|
||||
owner: null,
|
||||
user_can_change: true,
|
||||
editable: true,
|
||||
deletable: true,
|
||||
},
|
||||
{
|
||||
name: 'view-only shared',
|
||||
owner: 2,
|
||||
user_can_change: false,
|
||||
editable: false,
|
||||
deletable: false,
|
||||
},
|
||||
{
|
||||
name: 'editable shared',
|
||||
owner: 2,
|
||||
user_can_change: true,
|
||||
editable: true,
|
||||
deletable: false,
|
||||
},
|
||||
])(
|
||||
'should respect permissions for $name templates',
|
||||
async ({ owner, user_can_change, editable, deletable }) => {
|
||||
const template = { ...templates[0], owner, user_can_change }
|
||||
component.templates.set([template])
|
||||
fixture.detectChanges()
|
||||
await fixture.whenStable()
|
||||
|
||||
expect(fixture.nativeElement.querySelector('.text-start').disabled).toBe(
|
||||
!editable
|
||||
)
|
||||
expect(
|
||||
fixture.nativeElement.querySelector('.form-switch input').disabled
|
||||
).toBe(!editable)
|
||||
const buttons = Array.from<HTMLButtonElement>(
|
||||
fixture.nativeElement.querySelectorAll('button')
|
||||
)
|
||||
for (const button of buttons.filter(
|
||||
(button) => button.textContent.trim() === 'Edit'
|
||||
)) {
|
||||
expect(button.disabled).toBe(!editable)
|
||||
}
|
||||
for (const button of buttons.filter(
|
||||
(button) => button.textContent.trim() === 'Delete'
|
||||
)) {
|
||||
expect(button.disabled).toBe(!deletable)
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
it('should allow a superuser to manage another users template', () => {
|
||||
TestBed.inject(PermissionsService).initialize([], {
|
||||
id: 1,
|
||||
is_superuser: true,
|
||||
} as User)
|
||||
component.templates.set([
|
||||
{ ...templates[0], owner: 2, user_can_change: false },
|
||||
])
|
||||
fixture.detectChanges()
|
||||
expect(fixture.nativeElement.querySelector('.text-start').disabled).toBe(
|
||||
false
|
||||
)
|
||||
expect(
|
||||
fixture.nativeElement.querySelector('.btn-outline-danger').disabled
|
||||
).toBe(false)
|
||||
})
|
||||
|
||||
it('should not open mutation dialogs or patch a view-only template', () => {
|
||||
const template = { ...templates[0], owner: 2, user_can_change: false }
|
||||
const open = jest.spyOn(modalService, 'open')
|
||||
const patch = jest.spyOn(ocrTemplateService, 'patch')
|
||||
component.editTemplate(template)
|
||||
component.toggleTemplateEnabled(template)
|
||||
component.deleteTemplate(template)
|
||||
expect(open).not.toHaveBeenCalled()
|
||||
expect(patch).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should require global edit permission even for an owned template', () => {
|
||||
TestBed.inject(PermissionsService).initialize(['view_ocrtemplate'], {
|
||||
id: 1,
|
||||
} as User)
|
||||
fixture.destroy()
|
||||
fixture = TestBed.createComponent(OcrTemplatesComponent)
|
||||
component = fixture.componentInstance
|
||||
fixture.detectChanges()
|
||||
expect(fixture.nativeElement.querySelector('.text-start').disabled).toBe(
|
||||
true
|
||||
)
|
||||
expect(fixture.nativeElement.querySelector('.form-switch input')).toBeNull()
|
||||
})
|
||||
|
||||
it('should support create, show notification on error / success', () => {
|
||||
const toastErrorSpy = jest.spyOn(toastService, 'showError')
|
||||
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
|
||||
const reloadSpy = jest.spyOn(component, 'reload')
|
||||
|
||||
fixture.debugElement
|
||||
.queryAll(By.css('pngx-page-header button'))
|
||||
.find((button) => button.nativeElement.textContent.includes('Add'))
|
||||
.triggerEventHandler('click')
|
||||
|
||||
const editor = modal.componentInstance as OcrTemplateEditorComponent
|
||||
expect(editor).toBeInstanceOf(OcrTemplateEditorComponent)
|
||||
expect(editor.object).toBeUndefined()
|
||||
|
||||
editor.failed.emit({ error: 'error creating template' })
|
||||
expect(toastErrorSpy).toHaveBeenCalled()
|
||||
expect(reloadSpy).not.toHaveBeenCalled()
|
||||
|
||||
editor.succeeded.emit(templates[0])
|
||||
expect(toastSuccessSpy).toHaveBeenCalled()
|
||||
expect(reloadSpy).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should pass the template to the editor', () => {
|
||||
component.editTemplate(templates[0])
|
||||
expect(modal.componentInstance.object).toBe(templates[0])
|
||||
})
|
||||
|
||||
it('should confirm closing the editor with unsaved changes', async () => {
|
||||
const confirmSpy = jest
|
||||
.spyOn(TestBed.inject(DirtyFormGuard), 'confirmChanges')
|
||||
.mockReturnValue(of(false))
|
||||
component.editTemplate()
|
||||
const editor = modal.componentInstance as OcrTemplateEditorComponent
|
||||
editor.ngOnInit()
|
||||
|
||||
modal.dismiss()
|
||||
await fixture.whenStable()
|
||||
expect(confirmSpy).not.toHaveBeenCalled()
|
||||
|
||||
component.editTemplate()
|
||||
const dirtyEditor = modal.componentInstance as OcrTemplateEditorComponent
|
||||
dirtyEditor.ngOnInit()
|
||||
dirtyEditor.template.name = 'Changed'
|
||||
modal.dismiss()
|
||||
await fixture.whenStable()
|
||||
expect(confirmSpy).toHaveBeenCalled()
|
||||
expect(modalService.hasOpenModals()).toBe(true)
|
||||
|
||||
dirtyEditor.saving.set(true)
|
||||
confirmSpy.mockClear()
|
||||
modal.dismiss()
|
||||
await fixture.whenStable()
|
||||
expect(confirmSpy).not.toHaveBeenCalled()
|
||||
expect(modalService.hasOpenModals()).toBe(true)
|
||||
})
|
||||
|
||||
it('should only send the enabled flag when toggling', () => {
|
||||
const patchSpy = jest
|
||||
.spyOn(ocrTemplateService, 'patch')
|
||||
.mockReturnValueOnce(throwError(() => new Error('error toggling')))
|
||||
.mockReturnValueOnce(of(templates[0]))
|
||||
const toastErrorSpy = jest.spyOn(toastService, 'showError')
|
||||
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
|
||||
const reloadSpy = jest.spyOn(component, 'reload')
|
||||
|
||||
const toggle = fixture.debugElement.query(By.css('.form-switch input'))
|
||||
toggle.nativeElement.click()
|
||||
expect(patchSpy).toHaveBeenCalledWith({ id: 1, enabled: false })
|
||||
expect(toastErrorSpy).toHaveBeenCalled()
|
||||
expect(reloadSpy).toHaveBeenCalledTimes(1)
|
||||
|
||||
component.toggleTemplateEnabled(templates[0])
|
||||
expect(toastSuccessSpy).toHaveBeenCalled()
|
||||
expect(reloadSpy).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('should support delete, show notification on error / success', () => {
|
||||
const toastErrorSpy = jest.spyOn(toastService, 'showError')
|
||||
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
|
||||
const deleteSpy = jest.spyOn(ocrTemplateService, 'delete')
|
||||
const reloadSpy = jest.spyOn(component, 'reload')
|
||||
|
||||
component.deleteTemplate(templates[0])
|
||||
const dialog = modal.componentInstance as ConfirmDialogComponent
|
||||
|
||||
deleteSpy.mockReturnValueOnce(throwError(() => new Error('error deleting')))
|
||||
dialog.confirmClicked.emit()
|
||||
expect(toastErrorSpy).toHaveBeenCalled()
|
||||
expect(dialog.buttonsEnabled()).toBe(true)
|
||||
expect(reloadSpy).not.toHaveBeenCalled()
|
||||
|
||||
deleteSpy.mockReturnValueOnce(of(true))
|
||||
dialog.confirmClicked.emit()
|
||||
expect(toastSuccessSpy).toHaveBeenCalled()
|
||||
expect(reloadSpy).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it.each([true, false])(
|
||||
'reports the enabled state %s after the update succeeds',
|
||||
(enabled) => {
|
||||
const template = { ...templates[0], enabled }
|
||||
jest.spyOn(ocrTemplateService, 'patch').mockReturnValue(of(template))
|
||||
const toast = jest.spyOn(toastService, 'showSuccess')
|
||||
component.toggleTemplateEnabled(template)
|
||||
expect(toast).toHaveBeenCalledWith(
|
||||
`${enabled ? 'Enabled' : 'Disabled'} OCR template "${template.name}"`
|
||||
)
|
||||
}
|
||||
)
|
||||
})
|
||||
@@ -1,185 +0,0 @@
|
||||
import { Component, OnInit, inject, signal } from '@angular/core'
|
||||
import { FormsModule } from '@angular/forms'
|
||||
import { NgbDropdownModule, NgbModal } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { firstValueFrom, takeUntil, tap } from 'rxjs'
|
||||
import { MATCH_NONE, MATCHING_ALGORITHMS } from 'src/app/data/matching-model'
|
||||
import { OcrTemplate } from 'src/app/data/ocr-template'
|
||||
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
|
||||
import { DirtyFormGuard } from 'src/app/guards/dirty-form.guard'
|
||||
import {
|
||||
PermissionAction,
|
||||
PermissionType,
|
||||
PermissionsService,
|
||||
} from 'src/app/services/permissions.service'
|
||||
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
|
||||
import { ToastService } from 'src/app/services/toast.service'
|
||||
import { ConfirmDialogComponent } from '../../common/confirm-dialog/confirm-dialog.component'
|
||||
import { PageHeaderComponent } from '../../common/page-header/page-header.component'
|
||||
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
|
||||
import { OcrTemplateEditorComponent } from './ocr-template-editor/ocr-template-editor.component'
|
||||
|
||||
@Component({
|
||||
selector: 'pngx-ocr-templates',
|
||||
templateUrl: './ocr-templates.component.html',
|
||||
imports: [
|
||||
PageHeaderComponent,
|
||||
IfPermissionsDirective,
|
||||
FormsModule,
|
||||
NgbDropdownModule,
|
||||
BootstrapIconComponent,
|
||||
],
|
||||
})
|
||||
export class OcrTemplatesComponent
|
||||
extends LoadingComponentWithPermissions
|
||||
implements OnInit
|
||||
{
|
||||
private readonly ocrTemplateService = inject(OcrTemplateService)
|
||||
public readonly permissionsService = inject(PermissionsService)
|
||||
private readonly modalService = inject(NgbModal)
|
||||
private readonly dirtyFormGuard = inject(DirtyFormGuard)
|
||||
private readonly toastService = inject(ToastService)
|
||||
|
||||
public readonly templates = signal<OcrTemplate[]>([])
|
||||
|
||||
ngOnInit() {
|
||||
this.reload()
|
||||
}
|
||||
|
||||
reload() {
|
||||
this.loading.set(true)
|
||||
this.ocrTemplateService
|
||||
.listAll()
|
||||
.pipe(
|
||||
takeUntil(this.unsubscribeNotifier),
|
||||
tap((r) => this.templates.set(r.results))
|
||||
)
|
||||
.subscribe(() => {
|
||||
this.show.set(true)
|
||||
this.loading.set(false)
|
||||
})
|
||||
}
|
||||
|
||||
getMatching(template: OcrTemplate): string {
|
||||
if (template.matching_algorithm === MATCH_NONE) {
|
||||
return $localize`None`
|
||||
}
|
||||
if (!template.match) {
|
||||
return '-'
|
||||
}
|
||||
const algorithm = MATCHING_ALGORITHMS.find(
|
||||
(a) => a.id === template.matching_algorithm
|
||||
)
|
||||
return `${algorithm?.shortName}: ${template.match}`
|
||||
}
|
||||
|
||||
editTemplate(template?: OcrTemplate) {
|
||||
if (template && !this.userCanEdit(template)) return
|
||||
const modal = this.modalService.open(OcrTemplateEditorComponent, {
|
||||
backdrop: 'static',
|
||||
keyboard: false,
|
||||
size: 'xxl',
|
||||
fullscreen: 'lg',
|
||||
scrollable: true,
|
||||
ariaLabelledBy: 'ocr-template-editor-title',
|
||||
beforeDismiss: () => {
|
||||
const editor: OcrTemplateEditorComponent = modal.componentInstance
|
||||
if (editor.saving()) return false
|
||||
if (!editor.isDirty) return true
|
||||
return firstValueFrom(
|
||||
this.dirtyFormGuard.confirmChanges(undefined, $localize`Close`)
|
||||
)
|
||||
},
|
||||
})
|
||||
modal.componentInstance.object = template
|
||||
modal.componentInstance.succeeded
|
||||
.pipe(takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe((saved: OcrTemplate) => {
|
||||
this.toastService.showSuccess(
|
||||
$localize`Saved OCR template "${saved.name}".`
|
||||
)
|
||||
this.ocrTemplateService.clearCache()
|
||||
this.reload()
|
||||
})
|
||||
modal.componentInstance.failed
|
||||
.pipe(takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe((e) => {
|
||||
this.toastService.showError($localize`Error saving OCR template.`, e)
|
||||
})
|
||||
}
|
||||
|
||||
toggleTemplateEnabled(template: OcrTemplate) {
|
||||
if (!this.userCanEdit(template)) return
|
||||
this.ocrTemplateService
|
||||
.patch({ id: template.id, enabled: template.enabled } as OcrTemplate)
|
||||
.subscribe({
|
||||
next: () => {
|
||||
this.toastService.showSuccess(
|
||||
template.enabled
|
||||
? $localize`Enabled OCR template "${template.name}"`
|
||||
: $localize`Disabled OCR template "${template.name}"`
|
||||
)
|
||||
this.reload()
|
||||
},
|
||||
error: (e) => {
|
||||
this.toastService.showError(
|
||||
$localize`Error toggling OCR template "${template.name}".`,
|
||||
e
|
||||
)
|
||||
this.reload()
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
deleteTemplate(template: OcrTemplate) {
|
||||
if (
|
||||
!this.permissionsService.currentUserCan(
|
||||
PermissionAction.Delete,
|
||||
PermissionType.OcrTemplate
|
||||
) ||
|
||||
!this.permissionsService.currentUserOwnsObject(template)
|
||||
)
|
||||
return
|
||||
const modal = this.modalService.open(ConfirmDialogComponent, {
|
||||
backdrop: 'static',
|
||||
})
|
||||
modal.componentInstance.title = $localize`Confirm delete OCR template`
|
||||
modal.componentInstance.messageBold = $localize`This operation will permanently delete this OCR template.`
|
||||
modal.componentInstance.message = $localize`This operation cannot be undone.`
|
||||
modal.componentInstance.btnClass = 'btn-danger'
|
||||
modal.componentInstance.btnCaption = $localize`Proceed`
|
||||
modal.componentInstance.confirmClicked.subscribe(() => {
|
||||
modal.componentInstance.buttonsEnabled.set(false)
|
||||
this.ocrTemplateService.delete(template).subscribe({
|
||||
next: () => {
|
||||
modal.close()
|
||||
this.toastService.showSuccess(
|
||||
$localize`Deleted OCR template "${template.name}".`
|
||||
)
|
||||
this.ocrTemplateService.clearCache()
|
||||
this.reload()
|
||||
},
|
||||
error: (e) => {
|
||||
modal.componentInstance.buttonsEnabled.set(true)
|
||||
this.toastService.showError(
|
||||
$localize`Error deleting OCR template "${template.name}".`,
|
||||
e
|
||||
)
|
||||
},
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
userCanEdit(template: OcrTemplate): boolean {
|
||||
return (
|
||||
this.permissionsService.currentUserCan(
|
||||
PermissionAction.Change,
|
||||
PermissionType.OcrTemplate
|
||||
) &&
|
||||
this.permissionsService.currentUserHasObjectPermissions(
|
||||
PermissionAction.Change,
|
||||
template
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
import { PaperlessTaskStatus } from './paperless-task'
|
||||
|
||||
export interface BulkOperationResponse {
|
||||
result: string
|
||||
task_id: string
|
||||
}
|
||||
|
||||
export interface BulkOperationStatus {
|
||||
task_id: string
|
||||
operation: string
|
||||
status: PaperlessTaskStatus
|
||||
error?: string
|
||||
}
|
||||
@@ -1,115 +0,0 @@
|
||||
import { CustomFieldDataType } from './custom-field'
|
||||
import { MatchingModel } from './matching-model'
|
||||
import { ObjectWithPermissions } from './object-with-permissions'
|
||||
|
||||
export enum OcrZoneTarget {
|
||||
CustomField = 'custom_field',
|
||||
Title = 'title',
|
||||
Asn = 'asn',
|
||||
Created = 'created',
|
||||
}
|
||||
|
||||
export enum OcrZoneTransform {
|
||||
None = 'none',
|
||||
Strip = 'strip',
|
||||
Uppercase = 'uppercase',
|
||||
Lowercase = 'lowercase',
|
||||
Numeric = 'numeric',
|
||||
StripPunctuation = 'strip_punctuation',
|
||||
Date = 'date',
|
||||
QrCode = 'qr_code',
|
||||
}
|
||||
|
||||
export const DEFAULT_OCR_ZONE_LANGUAGE = 'eng'
|
||||
|
||||
/** The field a zone writes to, keyed by OcrTemplate.combine_formats. */
|
||||
export function getZoneFieldKey(zone: OcrTemplateZone): string | null {
|
||||
if (zone.target !== OcrZoneTarget.CustomField) return zone.target
|
||||
return zone.custom_field === null ? null : String(zone.custom_field)
|
||||
}
|
||||
|
||||
/** Custom field types zone OCR can write to. See models.py OCR_SUPPORTED_FIELD_TYPES. */
|
||||
export const OCR_SUPPORTED_FIELD_TYPES: CustomFieldDataType[] = [
|
||||
CustomFieldDataType.String,
|
||||
CustomFieldDataType.Url,
|
||||
CustomFieldDataType.Date,
|
||||
CustomFieldDataType.Integer,
|
||||
CustomFieldDataType.Float,
|
||||
CustomFieldDataType.Monetary,
|
||||
CustomFieldDataType.LongText,
|
||||
CustomFieldDataType.Boolean,
|
||||
]
|
||||
|
||||
export const OCR_BUILTIN_TARGETS = [
|
||||
{ id: OcrZoneTarget.Title, name: $localize`Title` },
|
||||
{ id: OcrZoneTarget.Asn, name: $localize`Archive serial number` },
|
||||
{ id: OcrZoneTarget.Created, name: $localize`Date created` },
|
||||
]
|
||||
|
||||
export interface OcrTemplateZone {
|
||||
id?: number
|
||||
name: string
|
||||
target: OcrZoneTarget
|
||||
custom_field: number | null
|
||||
page?: number
|
||||
x: number
|
||||
y: number
|
||||
width: number
|
||||
height: number
|
||||
ocr_language: string
|
||||
transform: OcrZoneTransform
|
||||
date_format?: string
|
||||
validation_regex: string
|
||||
order: number
|
||||
zone_source_width?: number
|
||||
zone_source_height?: number
|
||||
}
|
||||
|
||||
export interface OcrTemplate extends MatchingModel, ObjectWithPermissions {
|
||||
name: string
|
||||
sample_document: number | null
|
||||
source_width: number
|
||||
source_height: number
|
||||
enabled: boolean
|
||||
combine_formats?: Record<string, string>
|
||||
created?: string
|
||||
updated?: string
|
||||
zones: OcrTemplateZone[]
|
||||
}
|
||||
|
||||
export interface ZoneTestRequest {
|
||||
name: string
|
||||
x: number
|
||||
y: number
|
||||
width: number
|
||||
height: number
|
||||
page: number
|
||||
ocr_language: string
|
||||
transform: OcrZoneTransform
|
||||
date_format?: string
|
||||
validation_regex: string
|
||||
zone_source_width?: number
|
||||
zone_source_height?: number
|
||||
}
|
||||
|
||||
export interface OcrZoneTestResult {
|
||||
raw_text?: string | null
|
||||
value?: string | null
|
||||
regex?: string
|
||||
regex_match?: boolean | null
|
||||
error?: string
|
||||
}
|
||||
|
||||
export interface OcrZoneRunResult {
|
||||
template: string
|
||||
zone: string
|
||||
custom_field: string
|
||||
value: string | number | boolean | null
|
||||
reason?: string | null
|
||||
}
|
||||
|
||||
export interface OcrZoneRunResponse {
|
||||
status: 'completed' | 'skipped'
|
||||
reason?: 'no_template'
|
||||
results: OcrZoneRunResult[]
|
||||
}
|
||||
@@ -35,7 +35,6 @@ export enum HideableSidebarItemID {
|
||||
SavedViews = 'saved_views',
|
||||
ShareLinks = 'share_links',
|
||||
Workflows = 'workflows',
|
||||
OcrTemplates = 'ocr_templates',
|
||||
Mail = 'mail',
|
||||
Documentation = 'documentation',
|
||||
}
|
||||
|
||||
@@ -3,16 +3,12 @@ import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
|
||||
import { DirtyCheckGuard } from '@ngneat/dirty-check-forms'
|
||||
import { Observable, Subject } from 'rxjs'
|
||||
import { ConfirmDialogComponent } from 'src/app/components/common/confirm-dialog/confirm-dialog.component'
|
||||
import { ActivatedRouteSnapshot } from '@angular/router'
|
||||
|
||||
@Injectable({ providedIn: 'root' })
|
||||
export class DirtyFormGuard extends DirtyCheckGuard {
|
||||
private modalService = inject(NgbModal)
|
||||
|
||||
confirmChanges(
|
||||
_currentRoute?: ActivatedRouteSnapshot,
|
||||
leaveLabel: string = $localize`Leave page`
|
||||
): Observable<boolean> {
|
||||
confirmChanges(): Observable<boolean> {
|
||||
let modal = this.modalService.open(ConfirmDialogComponent, {
|
||||
backdrop: 'static',
|
||||
})
|
||||
@@ -20,7 +16,7 @@ export class DirtyFormGuard extends DirtyCheckGuard {
|
||||
modal.componentInstance.messageBold = $localize`You have unsaved changes.`
|
||||
modal.componentInstance.message = $localize`Are you sure you want to leave?`
|
||||
modal.componentInstance.btnClass = 'btn-warning'
|
||||
modal.componentInstance.btnCaption = leaveLabel
|
||||
modal.componentInstance.btnCaption = $localize`Leave page`
|
||||
modal.componentInstance.confirmClicked.subscribe(() => {
|
||||
modal.componentInstance.buttonsEnabled.set(false)
|
||||
modal.close()
|
||||
|
||||
@@ -124,6 +124,42 @@ describe('DocumentListViewService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it.each([false, true])(
|
||||
'preserves the current route on a background refresh (saved view: %s)',
|
||||
(savedView) => {
|
||||
if (savedView) documentListViewService.activateSavedView(view)
|
||||
const navigate = jest.mocked(router.navigate)
|
||||
navigate.mockClear()
|
||||
documentListViewService.reload(null, false)
|
||||
httpTestingController
|
||||
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
|
||||
.flush(full_results)
|
||||
expect(documentListViewService.documents).toEqual(full_results.results)
|
||||
expect(navigate).not.toHaveBeenCalled()
|
||||
}
|
||||
)
|
||||
|
||||
it('preserves the route when a background refresh falls back to the first page', () => {
|
||||
jest
|
||||
.spyOn(documentListViewService, 'saveDocumentListView')
|
||||
.mockImplementation(() => {})
|
||||
documentListViewService.currentPage = 2
|
||||
httpTestingController
|
||||
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
|
||||
.flush(full_results)
|
||||
const navigate = jest.mocked(router.navigate)
|
||||
navigate.mockClear()
|
||||
documentListViewService.reload(null, false)
|
||||
httpTestingController
|
||||
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
|
||||
.flush('Not found', { status: 404, statusText: 'Not found' })
|
||||
httpTestingController
|
||||
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
|
||||
.flush(full_results)
|
||||
expect(documentListViewService.currentPage).toBe(1)
|
||||
expect(navigate).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should reload the list', () => {
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
documentListViewService.reload()
|
||||
|
||||
@@ -380,7 +380,7 @@ export class DocumentListViewService {
|
||||
queryParams: paramsFromViewState(activeListViewState),
|
||||
replaceUrl: !this.router.routerState.snapshot.url.includes('?'), // in case navigating from params-less /documents
|
||||
})
|
||||
} else if (this._activeSavedViewId) {
|
||||
} else if (updateQueryParams && this._activeSavedViewId) {
|
||||
this.router.navigate([], {
|
||||
queryParams: paramsFromViewState(activeListViewState, true),
|
||||
queryParamsHandling: 'merge',
|
||||
@@ -399,7 +399,7 @@ export class DocumentListViewService {
|
||||
// this happens when applying a filter: the current page might not be available anymore due to the reduced result set.
|
||||
activeListViewState.currentPage = 1
|
||||
this.markChanged()
|
||||
this.reload()
|
||||
this.reload(onFinish, updateQueryParams)
|
||||
} else if (
|
||||
activeListViewState.sortField.indexOf('custom_field') === 0 &&
|
||||
!this.settings
|
||||
|
||||
@@ -283,10 +283,6 @@ describe('PermissionsService', () => {
|
||||
'view_workflow',
|
||||
'change_workflow',
|
||||
'delete_workflow',
|
||||
'add_ocrtemplate',
|
||||
'view_ocrtemplate',
|
||||
'change_ocrtemplate',
|
||||
'delete_ocrtemplate',
|
||||
'add_customfield',
|
||||
'view_customfield',
|
||||
'change_customfield',
|
||||
|
||||
@@ -29,7 +29,6 @@ export enum PermissionType {
|
||||
ShareLinkBundle = '%s_sharelinkbundle',
|
||||
CustomField = '%s_customfield',
|
||||
Workflow = '%s_workflow',
|
||||
OcrTemplate = '%s_ocrtemplate',
|
||||
ProcessedMail = '%s_processedmail',
|
||||
GlobalStatistics = '%s_global_statistics',
|
||||
SystemMonitoring = '%s_system_monitoring',
|
||||
|
||||
@@ -248,6 +248,7 @@ describe(`DocumentService`, () => {
|
||||
)
|
||||
expect(req.request.method).toEqual('POST')
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
documents: ids,
|
||||
method,
|
||||
parameters,
|
||||
@@ -271,6 +272,7 @@ describe(`DocumentService`, () => {
|
||||
)
|
||||
expect(req.request.method).toEqual('POST')
|
||||
expect(req.request.body).toEqual({
|
||||
from_webui: true,
|
||||
all: true,
|
||||
filters: { title__icontains: 'apple' },
|
||||
excluded_documents: [2, 3],
|
||||
@@ -563,20 +565,6 @@ describe(`DocumentService`, () => {
|
||||
|
||||
req.flush('task-id')
|
||||
})
|
||||
|
||||
it('should call appropriate api endpoint for running zone OCR', () => {
|
||||
const url = `${environment.apiBaseUrl}${endpoint}/${documents[0].id}/run-zone-ocr/`
|
||||
subscription = service.runZoneOcr(documents[0].id).subscribe()
|
||||
let req = httpTestingController.expectOne(url)
|
||||
expect(req.request.method).toEqual('POST')
|
||||
expect(req.request.body).toEqual({})
|
||||
req.flush({ status: 'completed', results: [] })
|
||||
|
||||
subscription = service.runZoneOcr(documents[0].id, 3, 10).subscribe()
|
||||
req = httpTestingController.expectOne(`${url}?version=10`)
|
||||
expect(req.request.body).toEqual({ template: 3 })
|
||||
req.flush({ status: 'completed', results: [] })
|
||||
})
|
||||
})
|
||||
|
||||
it('should construct sort fields respecting permissions', () => {
|
||||
|
||||
@@ -12,9 +12,9 @@ import {
|
||||
import { DocumentMetadata } from 'src/app/data/document-metadata'
|
||||
import { DocumentSuggestions } from 'src/app/data/document-suggestions'
|
||||
import { FilterRule } from 'src/app/data/filter-rule'
|
||||
import { OcrZoneRunResponse } from 'src/app/data/ocr-template'
|
||||
import { Results, SelectionData } from 'src/app/data/results'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { BulkOperationResponse } from '../../data/bulk-operation'
|
||||
import { queryParamsFromFilterRules } from '../../utils/query-params'
|
||||
import {
|
||||
PermissionAction,
|
||||
@@ -338,11 +338,15 @@ export class DocumentService extends AbstractPaperlessService<Document> {
|
||||
method: DocumentBulkEditMethod,
|
||||
args: any
|
||||
) {
|
||||
return this.http.post(this.getResourceUrl(null, 'bulk_edit'), {
|
||||
...selection,
|
||||
method: method,
|
||||
parameters: args,
|
||||
})
|
||||
return this.http.post<BulkOperationResponse>(
|
||||
this.getResourceUrl(null, 'bulk_edit'),
|
||||
{
|
||||
...selection,
|
||||
method: method,
|
||||
from_webui: true,
|
||||
parameters: args,
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
deleteDocuments(selection: DocumentSelectionQuery) {
|
||||
@@ -361,28 +365,6 @@ export class DocumentService extends AbstractPaperlessService<Document> {
|
||||
})
|
||||
}
|
||||
|
||||
runZoneOcrDocuments(
|
||||
selection: DocumentSelectionQuery,
|
||||
templateId: number | null = null
|
||||
) {
|
||||
return this.http.post(this.getResourceUrl(null, 'run_zone_ocr'), {
|
||||
...selection,
|
||||
...(templateId ? { template: templateId } : {}),
|
||||
})
|
||||
}
|
||||
|
||||
runZoneOcr(
|
||||
id: number,
|
||||
templateId: number | null = null,
|
||||
versionId: number | null = null
|
||||
): Observable<OcrZoneRunResponse> {
|
||||
return this.http.post<OcrZoneRunResponse>(
|
||||
this.getResourceUrl(id, 'run-zone-ocr'),
|
||||
templateId ? { template: templateId } : {},
|
||||
{ params: versionId ? { version: versionId.toString() } : {} }
|
||||
)
|
||||
}
|
||||
|
||||
rotateDocuments(
|
||||
selection: DocumentSelectionQuery,
|
||||
degrees: number,
|
||||
|
||||
@@ -1,78 +0,0 @@
|
||||
import { provideHttpClient } from '@angular/common/http'
|
||||
import {
|
||||
HttpTestingController,
|
||||
provideHttpClientTesting,
|
||||
} from '@angular/common/http/testing'
|
||||
import { TestBed } from '@angular/core/testing'
|
||||
import { OcrZoneTransform, ZoneTestRequest } from 'src/app/data/ocr-template'
|
||||
import { environment } from 'src/environments/environment'
|
||||
import { OcrTemplateService } from './ocr-template.service'
|
||||
|
||||
describe('OcrTemplateService', () => {
|
||||
let service: OcrTemplateService
|
||||
let http: HttpTestingController
|
||||
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
providers: [provideHttpClient(), provideHttpClientTesting()],
|
||||
})
|
||||
service = TestBed.inject(OcrTemplateService)
|
||||
http = TestBed.inject(HttpTestingController)
|
||||
})
|
||||
|
||||
afterEach(() => http.verify())
|
||||
|
||||
it('builds page image URLs with zero-based page numbers', () => {
|
||||
expect(service.getPageImageUrl(12, 0)).toBe(
|
||||
`${environment.apiBaseUrl}ocr_templates/document-page-image/12/0/`
|
||||
)
|
||||
expect(service.getPageImageUrl(12, 2)).toBe(
|
||||
`${environment.apiBaseUrl}ocr_templates/document-page-image/12/2/`
|
||||
)
|
||||
})
|
||||
|
||||
it('posts a zone test and returns the extraction result', () => {
|
||||
const zone: ZoneTestRequest = {
|
||||
name: 'Total',
|
||||
x: 10,
|
||||
y: 20,
|
||||
width: 100,
|
||||
height: 50,
|
||||
page: -1,
|
||||
ocr_language: 'eng',
|
||||
transform: OcrZoneTransform.Numeric,
|
||||
validation_regex: '\\d+',
|
||||
zone_source_width: 1000,
|
||||
zone_source_height: 1500,
|
||||
}
|
||||
const result = {
|
||||
raw_text: 'Total: 123',
|
||||
value: '123',
|
||||
regex: '\\d+',
|
||||
regex_match: true,
|
||||
}
|
||||
const next = jest.fn()
|
||||
service.testZone(12, zone).subscribe(next)
|
||||
const request = http.expectOne(
|
||||
`${environment.apiBaseUrl}ocr_templates/test-zone/`
|
||||
)
|
||||
expect(request.request.method).toBe('POST')
|
||||
expect(request.request.body).toEqual({ document: 12, zone })
|
||||
request.flush(result)
|
||||
expect(next).toHaveBeenCalledWith(result)
|
||||
})
|
||||
|
||||
it('passes zone-test failures to the caller', () => {
|
||||
const error = jest.fn()
|
||||
service.testZone(12, {} as ZoneTestRequest).subscribe({ error })
|
||||
http
|
||||
.expectOne(`${environment.apiBaseUrl}ocr_templates/test-zone/`)
|
||||
.flush(
|
||||
{ error: 'Invalid zone' },
|
||||
{ status: 400, statusText: 'Bad Request' }
|
||||
)
|
||||
expect(error).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ status: 400, error: { error: 'Invalid zone' } })
|
||||
)
|
||||
})
|
||||
})
|
||||
@@ -1,30 +0,0 @@
|
||||
import { Injectable } from '@angular/core'
|
||||
import { Observable } from 'rxjs'
|
||||
import {
|
||||
OcrTemplate,
|
||||
OcrZoneTestResult,
|
||||
ZoneTestRequest,
|
||||
} from '../../data/ocr-template'
|
||||
import { AbstractPaperlessService } from './abstract-paperless-service'
|
||||
|
||||
@Injectable({ providedIn: 'root' })
|
||||
export class OcrTemplateService extends AbstractPaperlessService<OcrTemplate> {
|
||||
constructor() {
|
||||
super()
|
||||
this.resourceName = 'ocr_templates'
|
||||
}
|
||||
|
||||
getPageImageUrl(docId: number, page: number): string {
|
||||
return `${this.baseUrl}${this.resourceName}/document-page-image/${docId}/${page}/`
|
||||
}
|
||||
|
||||
testZone(
|
||||
docId: number,
|
||||
zone: ZoneTestRequest
|
||||
): Observable<OcrZoneTestResult> {
|
||||
return this.http.post<OcrZoneTestResult>(
|
||||
`${this.baseUrl}${this.resourceName}/test-zone/`,
|
||||
{ document: docId, zone }
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -438,6 +438,20 @@ describe('ConsumerStatusService', () => {
|
||||
expect(updated).toBeTruthy()
|
||||
})
|
||||
|
||||
it('forwards bulk operation completion events', () => {
|
||||
const received = jest.fn()
|
||||
websocketStatusService.onBulkOperationStatus().subscribe(received)
|
||||
const data = {
|
||||
task_id: 'bulk-task',
|
||||
operation: 'set_permissions',
|
||||
status: 'success',
|
||||
}
|
||||
websocketStatusService.connect()
|
||||
server.send({ type: WebsocketStatusType.BULK_OPERATION_STATUS, data })
|
||||
websocketStatusService.disconnect()
|
||||
expect(received).toHaveBeenCalledWith(data)
|
||||
})
|
||||
|
||||
it('should ignore keep-alive heartbeat messages from the server', () => {
|
||||
let updated = false
|
||||
let deleted = false
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Injectable, inject, signal } from '@angular/core'
|
||||
import { Subject } from 'rxjs'
|
||||
import { environment } from 'src/environments/environment'
|
||||
import { BulkOperationStatus } from '../data/bulk-operation'
|
||||
import { User } from '../data/user'
|
||||
import { WebsocketDocumentUpdatedMessage } from '../data/websocket-document-updated-message'
|
||||
import { WebsocketDocumentsDeletedMessage } from '../data/websocket-documents-deleted-message'
|
||||
@@ -12,6 +13,7 @@ export enum WebsocketStatusType {
|
||||
DOCUMENTS_DELETED = 'documents_deleted',
|
||||
DOCUMENT_UPDATED = 'document_updated',
|
||||
HEARTBEAT = 'heartbeat',
|
||||
BULK_OPERATION_STATUS = 'bulk_operation_status',
|
||||
}
|
||||
|
||||
// see ProgressStatusOptions in src/documents/plugins/helpers.py
|
||||
@@ -116,6 +118,13 @@ export class WebsocketStatusService {
|
||||
private readonly documentDeletedSubject = new Subject<boolean>()
|
||||
private readonly documentUpdatedSubject =
|
||||
new Subject<WebsocketDocumentUpdatedMessage>()
|
||||
private readonly bulkOperationStatusSubject =
|
||||
new Subject<BulkOperationStatus>()
|
||||
|
||||
onBulkOperationStatus() {
|
||||
return this.bulkOperationStatusSubject.asObservable()
|
||||
}
|
||||
|
||||
private readonly connectionStatusSubject = new Subject<boolean>()
|
||||
|
||||
private get(taskId: string, filename?: string) {
|
||||
@@ -192,6 +201,7 @@ export class WebsocketStatusService {
|
||||
| WebsocketProgressMessage
|
||||
| WebsocketDocumentsDeletedMessage
|
||||
| WebsocketDocumentUpdatedMessage
|
||||
| BulkOperationStatus
|
||||
} = JSON.parse(ev.data)
|
||||
|
||||
switch (type) {
|
||||
@@ -209,6 +219,12 @@ export class WebsocketStatusService {
|
||||
this.handleProgressUpdate(messageData as WebsocketProgressMessage)
|
||||
break
|
||||
|
||||
case WebsocketStatusType.BULK_OPERATION_STATUS:
|
||||
this.bulkOperationStatusSubject.next(
|
||||
messageData as BulkOperationStatus
|
||||
)
|
||||
break
|
||||
|
||||
case WebsocketStatusType.HEARTBEAT:
|
||||
// keep-alive from the server, see paperless.consumers.StatusConsumer
|
||||
break
|
||||
|
||||
@@ -691,12 +691,6 @@ table.table {
|
||||
}
|
||||
}
|
||||
|
||||
@media (min-width: 992px) {
|
||||
.modal-xxl {
|
||||
--bs-modal-width: min(1440px, calc(100vw - 2 * var(--bs-modal-margin)));
|
||||
}
|
||||
}
|
||||
|
||||
.toast {
|
||||
--bs-toast-max-width: var(--pngx-toast-max-width);
|
||||
}
|
||||
|
||||
@@ -15,7 +15,6 @@ class DocumentsConfig(AppConfig):
|
||||
from documents.signals.handlers import add_to_index
|
||||
from documents.signals.handlers import run_workflows_added
|
||||
from documents.signals.handlers import run_workflows_updated
|
||||
from documents.signals.handlers import run_zone_ocr_extraction
|
||||
from documents.signals.handlers import send_websocket_document_updated
|
||||
from documents.signals.handlers import set_correspondent
|
||||
from documents.signals.handlers import set_document_type
|
||||
@@ -28,10 +27,8 @@ class DocumentsConfig(AppConfig):
|
||||
document_consumption_finished.connect(set_tags)
|
||||
document_consumption_finished.connect(set_storage_path)
|
||||
document_consumption_finished.connect(run_workflows_added)
|
||||
document_consumption_finished.connect(run_zone_ocr_extraction)
|
||||
document_consumption_finished.connect(add_to_index)
|
||||
document_consumption_finished.connect(add_or_update_document_in_llm_index)
|
||||
|
||||
document_updated.connect(run_workflows_updated)
|
||||
document_updated.connect(send_websocket_document_updated)
|
||||
document_updated.connect(add_or_update_document_in_llm_index)
|
||||
|
||||
+53
-57
@@ -33,7 +33,6 @@ from documents.models import CustomField
|
||||
from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentType
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import PaperlessTask
|
||||
from documents.models import StoragePath
|
||||
from documents.models import Tag
|
||||
@@ -42,7 +41,6 @@ from documents.permissions import set_permissions_for_objects
|
||||
from documents.plugins.helpers import DocumentsStatusManager
|
||||
from documents.tasks import bulk_update_documents
|
||||
from documents.tasks import consume_file
|
||||
from documents.tasks import extract_zones
|
||||
from documents.tasks import remove_document_from_index
|
||||
from documents.tasks import update_document_content_maybe_archive_file
|
||||
from documents.versioning import get_latest_version_for_root
|
||||
@@ -50,6 +48,7 @@ from documents.versioning import get_root_document
|
||||
from documents.versioning import versions_newest_first
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
from collections.abc import Mapping
|
||||
|
||||
from django.contrib.auth.models import User
|
||||
@@ -133,9 +132,24 @@ def _scratch_path(name: str) -> Path:
|
||||
return Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR)) / name
|
||||
|
||||
|
||||
def _finish_metadata_edit(
|
||||
document_ids: list[int],
|
||||
update_callback: Callable[[list[int]], None] | None,
|
||||
) -> None:
|
||||
if update_callback is not None:
|
||||
update_callback(document_ids)
|
||||
else:
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": document_ids},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
|
||||
|
||||
def set_correspondent(
|
||||
doc_ids: list[int],
|
||||
correspondent: Correspondent,
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
if correspondent:
|
||||
correspondent = Correspondent.objects.only("pk").get(id=correspondent)
|
||||
@@ -148,15 +162,17 @@ def set_correspondent(
|
||||
affected_docs = list(qs.values_list("pk", flat=True))
|
||||
qs.update(correspondent=correspondent)
|
||||
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
|
||||
def set_storage_path(doc_ids: list[int], storage_path: StoragePath) -> Literal["OK"]:
|
||||
def set_storage_path(
|
||||
doc_ids: list[int],
|
||||
storage_path: StoragePath,
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
if storage_path:
|
||||
storage_path = StoragePath.objects.only("pk").get(id=storage_path)
|
||||
|
||||
@@ -170,15 +186,17 @@ def set_storage_path(doc_ids: list[int], storage_path: StoragePath) -> Literal["
|
||||
affected_docs = list(qs.values_list("pk", flat=True))
|
||||
qs.update(storage_path=storage_path)
|
||||
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
|
||||
def set_document_type(doc_ids: list[int], document_type: DocumentType) -> Literal["OK"]:
|
||||
def set_document_type(
|
||||
doc_ids: list[int],
|
||||
document_type: DocumentType,
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
if document_type:
|
||||
document_type = DocumentType.objects.only("pk").get(id=document_type)
|
||||
|
||||
@@ -190,15 +208,17 @@ def set_document_type(doc_ids: list[int], document_type: DocumentType) -> Litera
|
||||
affected_docs = list(qs.values_list("pk", flat=True))
|
||||
qs.update(document_type=document_type)
|
||||
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
|
||||
def add_tag(doc_ids: list[int], tag: int) -> Literal["OK"]:
|
||||
def add_tag(
|
||||
doc_ids: list[int],
|
||||
tag: int,
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
tag_obj = Tag.objects.get(pk=tag)
|
||||
tags_to_add = [tag_obj, *tag_obj.get_ancestors()]
|
||||
|
||||
@@ -219,15 +239,17 @@ def add_tag(doc_ids: list[int], tag: int) -> Literal["OK"]:
|
||||
DocumentTagRelationship.objects.bulk_create(to_create)
|
||||
|
||||
if affected_docs:
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": list(affected_docs)},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(list(affected_docs), update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
|
||||
def remove_tag(doc_ids: list[int], tag: int) -> Literal["OK"]:
|
||||
def remove_tag(
|
||||
doc_ids: list[int],
|
||||
tag: int,
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
tag_obj = Tag.objects.get(pk=tag)
|
||||
tag_ids = [tag_obj.id, *tag_obj.get_descendants_pks()]
|
||||
|
||||
@@ -240,10 +262,7 @@ def remove_tag(doc_ids: list[int], tag: int) -> Literal["OK"]:
|
||||
qs.delete()
|
||||
|
||||
if affected_docs:
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
@@ -252,6 +271,8 @@ def modify_tags(
|
||||
doc_ids: list[int],
|
||||
add_tags: list[int],
|
||||
remove_tags: list[int],
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
qs = Document.objects.filter(id__in=doc_ids).only("pk")
|
||||
affected_docs = list(qs.values_list("pk", flat=True))
|
||||
@@ -295,10 +316,7 @@ def modify_tags(
|
||||
)
|
||||
|
||||
if affected_docs:
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
@@ -307,6 +325,8 @@ def modify_custom_fields(
|
||||
doc_ids: list[int],
|
||||
add_custom_fields: list[int] | dict,
|
||||
remove_custom_fields: list[int],
|
||||
*,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
qs = Document.objects.filter(id__in=doc_ids).only("pk")
|
||||
affected_docs = list(qs.values_list("pk", flat=True))
|
||||
@@ -369,10 +389,7 @@ def modify_custom_fields(
|
||||
field_id__in=remove_custom_fields,
|
||||
).hard_delete()
|
||||
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
@@ -453,6 +470,7 @@ def set_permissions(
|
||||
*,
|
||||
owner: User | None = None,
|
||||
merge: bool = False,
|
||||
update_callback: Callable[[list[int]], None] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
qs = Document.objects.filter(id__in=doc_ids).select_related("owner")
|
||||
|
||||
@@ -470,10 +488,7 @@ def set_permissions(
|
||||
merge=merge,
|
||||
)
|
||||
|
||||
bulk_update_documents.apply_async(
|
||||
kwargs={"document_ids": affected_docs},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
|
||||
)
|
||||
_finish_metadata_edit(affected_docs, update_callback)
|
||||
|
||||
return "OK"
|
||||
|
||||
@@ -1193,22 +1208,3 @@ def remove_doclink(
|
||||
target_doc_field_instance.value.remove(document.id)
|
||||
target_doc_field_instance.save()
|
||||
Document.objects.filter(id=target_doc_id).update(modified=timezone.now())
|
||||
|
||||
|
||||
def run_zone_ocr(
|
||||
doc_ids: list[int],
|
||||
*,
|
||||
template: OcrTemplate | None = None,
|
||||
) -> Literal["OK"]:
|
||||
"""
|
||||
Queue zone OCR for the given documents, with the given template or with the
|
||||
templates that match each document's content when its task runs.
|
||||
"""
|
||||
template_id = template.pk if template is not None else None
|
||||
for document_id in doc_ids:
|
||||
extract_zones.apply_async(
|
||||
kwargs={"document_id": document_id, "template_id": template_id},
|
||||
headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
|
||||
)
|
||||
|
||||
return "OK"
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Queued document metadata edits."""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from auditlog.context import set_actor
|
||||
from celery import shared_task
|
||||
from django.conf import settings
|
||||
from django.contrib.auth.models import User
|
||||
from django.db import transaction
|
||||
from rest_framework.exceptions import PermissionDenied
|
||||
|
||||
from documents import bulk_edit
|
||||
from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.permissions import has_document_operation_permissions
|
||||
from documents.tasks import bulk_update_documents
|
||||
|
||||
MODIFIED_FIELDS = {
|
||||
"set_correspondent": "correspondent",
|
||||
"set_document_type": "document_type",
|
||||
"set_storage_path": "storage_path",
|
||||
"add_tag": "tags",
|
||||
"remove_tag": "tags",
|
||||
"modify_tags": "tags",
|
||||
"modify_custom_fields": "custom_fields",
|
||||
"set_permissions": None,
|
||||
}
|
||||
QUEUED_OPERATIONS = frozenset(MODIFIED_FIELDS)
|
||||
|
||||
|
||||
def snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
|
||||
"""
|
||||
Returns each document's current value of field, for the audit log.
|
||||
|
||||
Tags and custom fields are one row per value, so they are gathered
|
||||
into a sorted list of pks per document (empty when there are none).
|
||||
Reading them through Document.values() instead would join those rows
|
||||
and return one arbitrary value per document.
|
||||
"""
|
||||
if field == "tags":
|
||||
rows = (
|
||||
Document.tags.through.objects.filter(document_id__in=doc_ids)
|
||||
.order_by("tag_id")
|
||||
.values_list("document_id", "tag_id")
|
||||
)
|
||||
elif field == "custom_fields":
|
||||
rows = (
|
||||
CustomFieldInstance.objects.filter(document_id__in=doc_ids)
|
||||
.order_by("pk")
|
||||
.values_list("document_id", "pk")
|
||||
)
|
||||
else:
|
||||
return dict(
|
||||
Document.objects.filter(pk__in=doc_ids).values_list("pk", field),
|
||||
)
|
||||
|
||||
values: dict[int, list[int]] = {doc_id: [] for doc_id in doc_ids}
|
||||
for doc_id, pk in rows:
|
||||
values[doc_id].append(pk)
|
||||
return values
|
||||
|
||||
|
||||
@shared_task
|
||||
def execute_bulk_operation(
|
||||
operation: str,
|
||||
document_ids: list[int],
|
||||
parameters: dict,
|
||||
user_id: int,
|
||||
) -> dict:
|
||||
if operation not in QUEUED_OPERATIONS:
|
||||
raise ValueError(f"Unsupported bulk operation: {operation}")
|
||||
method = getattr(bulk_edit, operation)
|
||||
user = User.objects.get(pk=user_id)
|
||||
if not has_document_operation_permissions(
|
||||
user=user,
|
||||
documents=document_ids,
|
||||
method=method,
|
||||
parameters=parameters,
|
||||
):
|
||||
raise PermissionDenied("Insufficient permissions to execute bulk operation")
|
||||
|
||||
affected_ids: list[int] = []
|
||||
modified_field = MODIFIED_FIELDS[operation]
|
||||
# Keep metadata writes and their audit entries together. Follow-up processing
|
||||
# runs after commit, within this same tracked task.
|
||||
with set_actor(user), transaction.atomic():
|
||||
old_values = (
|
||||
snapshot_field(document_ids, modified_field)
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field
|
||||
else {}
|
||||
)
|
||||
method(document_ids, **parameters, update_callback=affected_ids.extend)
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field:
|
||||
from auditlog.models import LogEntry
|
||||
|
||||
new_values = snapshot_field(document_ids, modified_field)
|
||||
for doc in Document.objects.filter(pk__in=document_ids):
|
||||
LogEntry.objects.log_create(
|
||||
instance=doc,
|
||||
changes={modified_field: [old_values[doc.pk], new_values[doc.pk]]},
|
||||
action=LogEntry.Action.UPDATE,
|
||||
actor=user,
|
||||
additional_data={"reason": f"Bulk edit: {operation}"},
|
||||
)
|
||||
if affected_ids:
|
||||
bulk_update_documents(list(dict.fromkeys(affected_ids)))
|
||||
return {"operation": operation, "document_count": len(document_ids)}
|
||||
@@ -631,9 +631,7 @@ class ConsumerPlugin(
|
||||
pk=self.metadata.actor_id,
|
||||
).first()
|
||||
if actor is not None:
|
||||
from auditlog.context import ( # type: ignore[import-untyped]
|
||||
set_actor,
|
||||
)
|
||||
from auditlog.context import set_actor # type: ignore[import-untyped]
|
||||
|
||||
with set_actor(actor):
|
||||
original_document.save()
|
||||
@@ -651,9 +649,7 @@ class ConsumerPlugin(
|
||||
|
||||
# Create a log entry for the version addition, if enabled
|
||||
if settings.AUDIT_LOG_ENABLED:
|
||||
from auditlog.models import ( # type: ignore[import-untyped]
|
||||
LogEntry,
|
||||
)
|
||||
from auditlog.models import LogEntry # type: ignore[import-untyped]
|
||||
|
||||
LogEntry.objects.log_create(
|
||||
instance=root_doc,
|
||||
|
||||
@@ -1,311 +0,0 @@
|
||||
# Generated by Django 5.2.17 on 2026-10-20 20:29
|
||||
|
||||
import django.core.validators
|
||||
import django.db.models.deletion
|
||||
import django.utils.timezone
|
||||
from django.conf import settings
|
||||
from django.db import migrations
|
||||
from django.db import models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
dependencies = [
|
||||
("documents", "0028_replace_tag_tree_fields"),
|
||||
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.CreateModel(
|
||||
name="OcrTemplate",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.AutoField(
|
||||
auto_created=True,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
verbose_name="ID",
|
||||
),
|
||||
),
|
||||
("name", models.CharField(max_length=128, verbose_name="name")),
|
||||
(
|
||||
"match",
|
||||
models.CharField(blank=True, max_length=256, verbose_name="match"),
|
||||
),
|
||||
(
|
||||
"matching_algorithm",
|
||||
models.PositiveSmallIntegerField(
|
||||
choices=[
|
||||
(0, "None"),
|
||||
(1, "Any word"),
|
||||
(2, "All words"),
|
||||
(3, "Exact match"),
|
||||
(4, "Regular expression"),
|
||||
(5, "Fuzzy word"),
|
||||
(6, "Automatic"),
|
||||
],
|
||||
default=1,
|
||||
verbose_name="matching algorithm",
|
||||
),
|
||||
),
|
||||
(
|
||||
"is_insensitive",
|
||||
models.BooleanField(default=True, verbose_name="is insensitive"),
|
||||
),
|
||||
(
|
||||
"source_width",
|
||||
models.PositiveIntegerField(
|
||||
help_text="Width of the image the zones were drawn on (px)",
|
||||
validators=[django.core.validators.MinValueValidator(1)],
|
||||
verbose_name="source width",
|
||||
),
|
||||
),
|
||||
(
|
||||
"source_height",
|
||||
models.PositiveIntegerField(
|
||||
help_text="Height of the image the zones were drawn on (px)",
|
||||
validators=[django.core.validators.MinValueValidator(1)],
|
||||
verbose_name="source height",
|
||||
),
|
||||
),
|
||||
("enabled", models.BooleanField(default=True, verbose_name="enabled")),
|
||||
(
|
||||
"combine_formats",
|
||||
models.JSONField(
|
||||
blank=True,
|
||||
default=dict,
|
||||
help_text="Per-target format strings for combining several zones into one field, keyed by target (custom field id, or 'title'/'asn'/'created'). Tokens like {Zone Name} are replaced with that zone's value.",
|
||||
verbose_name="combine formats",
|
||||
),
|
||||
),
|
||||
(
|
||||
"created",
|
||||
models.DateTimeField(
|
||||
db_index=True,
|
||||
default=django.utils.timezone.now,
|
||||
editable=False,
|
||||
verbose_name="created",
|
||||
),
|
||||
),
|
||||
(
|
||||
"updated",
|
||||
models.DateTimeField(auto_now=True, verbose_name="updated"),
|
||||
),
|
||||
(
|
||||
"owner",
|
||||
models.ForeignKey(
|
||||
blank=True,
|
||||
default=None,
|
||||
null=True,
|
||||
on_delete=django.db.models.deletion.SET_NULL,
|
||||
to=settings.AUTH_USER_MODEL,
|
||||
verbose_name="owner",
|
||||
),
|
||||
),
|
||||
(
|
||||
"sample_document",
|
||||
models.ForeignKey(
|
||||
blank=True,
|
||||
help_text="Document used for previewing zones in the editor",
|
||||
null=True,
|
||||
on_delete=django.db.models.deletion.SET_NULL,
|
||||
related_name="+",
|
||||
to="documents.document",
|
||||
verbose_name="sample document",
|
||||
),
|
||||
),
|
||||
],
|
||||
options={
|
||||
"verbose_name": "OCR template",
|
||||
"verbose_name_plural": "OCR templates",
|
||||
"ordering": ("name",),
|
||||
"abstract": False,
|
||||
},
|
||||
),
|
||||
migrations.CreateModel(
|
||||
name="OcrTemplateZone",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.AutoField(
|
||||
auto_created=True,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
verbose_name="ID",
|
||||
),
|
||||
),
|
||||
(
|
||||
"name",
|
||||
models.CharField(
|
||||
help_text="Descriptive name for this zone (e.g. 'Invoice Number')",
|
||||
max_length=128,
|
||||
verbose_name="zone name",
|
||||
),
|
||||
),
|
||||
(
|
||||
"target",
|
||||
models.CharField(
|
||||
choices=[
|
||||
("custom_field", "Custom field"),
|
||||
("title", "Title"),
|
||||
("asn", "Archive serial number"),
|
||||
("created", "Date created"),
|
||||
],
|
||||
default="custom_field",
|
||||
help_text="Where the extracted value is written: a custom field, or a built-in document field (title, ASN, created date)",
|
||||
max_length=20,
|
||||
verbose_name="target",
|
||||
),
|
||||
),
|
||||
(
|
||||
"page",
|
||||
models.IntegerField(
|
||||
blank=True,
|
||||
help_text="Page (1 = first, -1 = last; blank uses the template default)",
|
||||
null=True,
|
||||
verbose_name="page",
|
||||
),
|
||||
),
|
||||
(
|
||||
"x",
|
||||
models.PositiveIntegerField(
|
||||
help_text="Left edge (px)",
|
||||
verbose_name="x",
|
||||
),
|
||||
),
|
||||
(
|
||||
"y",
|
||||
models.PositiveIntegerField(
|
||||
help_text="Top edge (px)",
|
||||
verbose_name="y",
|
||||
),
|
||||
),
|
||||
(
|
||||
"width",
|
||||
models.PositiveIntegerField(
|
||||
help_text="Zone width (px)",
|
||||
validators=[django.core.validators.MinValueValidator(1)],
|
||||
verbose_name="width",
|
||||
),
|
||||
),
|
||||
(
|
||||
"height",
|
||||
models.PositiveIntegerField(
|
||||
help_text="Zone height (px)",
|
||||
validators=[django.core.validators.MinValueValidator(1)],
|
||||
verbose_name="height",
|
||||
),
|
||||
),
|
||||
(
|
||||
"zone_source_width",
|
||||
models.PositiveIntegerField(
|
||||
blank=True,
|
||||
help_text="Width of the page image this zone was drawn on (px). Falls back to template source_width if unset.",
|
||||
null=True,
|
||||
verbose_name="zone source width",
|
||||
),
|
||||
),
|
||||
(
|
||||
"zone_source_height",
|
||||
models.PositiveIntegerField(
|
||||
blank=True,
|
||||
help_text="Height of the page image this zone was drawn on (px). Falls back to template source_height if unset.",
|
||||
null=True,
|
||||
verbose_name="zone source height",
|
||||
),
|
||||
),
|
||||
(
|
||||
"ocr_language",
|
||||
models.CharField(
|
||||
default="eng",
|
||||
help_text="Tesseract language code(s), e.g. 'deu+eng'",
|
||||
max_length=20,
|
||||
verbose_name="OCR language",
|
||||
),
|
||||
),
|
||||
(
|
||||
"transform",
|
||||
models.CharField(
|
||||
choices=[
|
||||
("none", "None"),
|
||||
("strip", "Strip whitespace"),
|
||||
("uppercase", "Uppercase"),
|
||||
("lowercase", "Lowercase"),
|
||||
("numeric", "Numeric only"),
|
||||
(
|
||||
"strip_punctuation",
|
||||
"Remove leading/trailing punctuation",
|
||||
),
|
||||
("date", "Parse date"),
|
||||
("qr_code", "Read QR/barcode"),
|
||||
],
|
||||
default="strip",
|
||||
max_length=20,
|
||||
verbose_name="transform",
|
||||
),
|
||||
),
|
||||
(
|
||||
"date_format",
|
||||
models.CharField(
|
||||
blank=True,
|
||||
default="",
|
||||
help_text="Python strptime format for the 'Parse date' transform (e.g. %d.%m.%Y). Blank = auto-detect.",
|
||||
max_length=64,
|
||||
verbose_name="date format",
|
||||
),
|
||||
),
|
||||
(
|
||||
"validation_regex",
|
||||
models.CharField(
|
||||
blank=True,
|
||||
default="",
|
||||
help_text="Optional regex pattern — extracted text is only accepted if it matches",
|
||||
max_length=256,
|
||||
verbose_name="validation regex",
|
||||
),
|
||||
),
|
||||
("order", models.PositiveIntegerField(default=0, verbose_name="order")),
|
||||
(
|
||||
"custom_field",
|
||||
models.ForeignKey(
|
||||
blank=True,
|
||||
help_text="Target custom field (only used when target is 'custom_field')",
|
||||
null=True,
|
||||
on_delete=django.db.models.deletion.CASCADE,
|
||||
related_name="ocr_zones",
|
||||
to="documents.customfield",
|
||||
verbose_name="custom field",
|
||||
),
|
||||
),
|
||||
(
|
||||
"template",
|
||||
models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE,
|
||||
related_name="zones",
|
||||
to="documents.ocrtemplate",
|
||||
verbose_name="template",
|
||||
),
|
||||
),
|
||||
],
|
||||
options={
|
||||
"verbose_name": "OCR template zone",
|
||||
"verbose_name_plural": "OCR template zones",
|
||||
"ordering": ("template", "order"),
|
||||
},
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="ocrtemplate",
|
||||
constraint=models.UniqueConstraint(
|
||||
fields=("name", "owner"),
|
||||
name="documents_ocrtemplate_unique_name_owner",
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="ocrtemplate",
|
||||
constraint=models.UniqueConstraint(
|
||||
condition=models.Q(("owner__isnull", True)),
|
||||
fields=("name",),
|
||||
name="documents_ocrtemplate_name_uniq",
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -2204,215 +2204,3 @@ class WorkflowRun(SoftDeleteModel):
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"WorkflowRun of {self.workflow} at {self.run_at} on {self.document}"
|
||||
|
||||
|
||||
class OcrTemplate(MatchingModel):
|
||||
"""Matching rules and extraction zones for documents with a common layout."""
|
||||
|
||||
source_width = models.PositiveIntegerField(
|
||||
_("source width"),
|
||||
validators=[MinValueValidator(1)],
|
||||
help_text=_("Width of the image the zones were drawn on (px)"),
|
||||
)
|
||||
|
||||
source_height = models.PositiveIntegerField(
|
||||
_("source height"),
|
||||
validators=[MinValueValidator(1)],
|
||||
help_text=_("Height of the image the zones were drawn on (px)"),
|
||||
)
|
||||
|
||||
sample_document = models.ForeignKey(
|
||||
"documents.Document",
|
||||
on_delete=models.SET_NULL,
|
||||
null=True,
|
||||
blank=True,
|
||||
related_name="+",
|
||||
verbose_name=_("sample document"),
|
||||
help_text=_("Document used for previewing zones in the editor"),
|
||||
)
|
||||
|
||||
enabled = models.BooleanField(_("enabled"), default=True)
|
||||
|
||||
combine_formats = models.JSONField(
|
||||
_("combine formats"),
|
||||
default=dict,
|
||||
blank=True,
|
||||
help_text=_(
|
||||
"Per-target format strings for combining several zones into one "
|
||||
"field, keyed by target (custom field id, or 'title'/'asn'/'created'). "
|
||||
"Tokens like {Zone Name} are replaced with that zone's value.",
|
||||
),
|
||||
)
|
||||
|
||||
created = models.DateTimeField(
|
||||
_("created"),
|
||||
default=timezone.now,
|
||||
db_index=True,
|
||||
editable=False,
|
||||
)
|
||||
|
||||
updated = models.DateTimeField(
|
||||
_("updated"),
|
||||
auto_now=True,
|
||||
)
|
||||
|
||||
class Meta(MatchingModel.Meta):
|
||||
verbose_name = _("OCR template")
|
||||
verbose_name_plural = _("OCR templates")
|
||||
|
||||
|
||||
class OcrTemplateZone(models.Model):
|
||||
"""A page region and the field its extracted value is written to."""
|
||||
|
||||
template = models.ForeignKey(
|
||||
OcrTemplate,
|
||||
on_delete=models.CASCADE,
|
||||
related_name="zones",
|
||||
verbose_name=_("template"),
|
||||
)
|
||||
|
||||
name = models.CharField(
|
||||
_("zone name"),
|
||||
max_length=128,
|
||||
help_text=_("Descriptive name for this zone (e.g. 'Invoice Number')"),
|
||||
)
|
||||
|
||||
class TargetType(models.TextChoices):
|
||||
CUSTOM_FIELD = ("custom_field", _("Custom field"))
|
||||
TITLE = ("title", _("Title"))
|
||||
ASN = ("asn", _("Archive serial number"))
|
||||
CREATED = ("created", _("Date created"))
|
||||
|
||||
target = models.CharField(
|
||||
_("target"),
|
||||
max_length=20,
|
||||
choices=TargetType.choices,
|
||||
default=TargetType.CUSTOM_FIELD,
|
||||
help_text=_(
|
||||
"Where the extracted value is written: a custom field, or a "
|
||||
"built-in document field (title, ASN, created date)",
|
||||
),
|
||||
)
|
||||
|
||||
custom_field = models.ForeignKey(
|
||||
"documents.CustomField",
|
||||
on_delete=models.CASCADE,
|
||||
related_name="ocr_zones",
|
||||
verbose_name=_("custom field"),
|
||||
null=True,
|
||||
blank=True,
|
||||
help_text=_("Target custom field (only used when target is 'custom_field')"),
|
||||
)
|
||||
|
||||
page = models.IntegerField(
|
||||
_("page"),
|
||||
null=True,
|
||||
blank=True,
|
||||
help_text=_("Page (1 = first, -1 = last; blank uses the template default)"),
|
||||
)
|
||||
|
||||
x = models.PositiveIntegerField(_("x"), help_text=_("Left edge (px)"))
|
||||
y = models.PositiveIntegerField(_("y"), help_text=_("Top edge (px)"))
|
||||
width = models.PositiveIntegerField(
|
||||
_("width"),
|
||||
validators=[MinValueValidator(1)],
|
||||
help_text=_("Zone width (px)"),
|
||||
)
|
||||
height = models.PositiveIntegerField(
|
||||
_("height"),
|
||||
validators=[MinValueValidator(1)],
|
||||
help_text=_("Zone height (px)"),
|
||||
)
|
||||
|
||||
# Mixed-size PDFs need each zone's original page dimensions for scaling.
|
||||
zone_source_width = models.PositiveIntegerField(
|
||||
_("zone source width"),
|
||||
null=True,
|
||||
blank=True,
|
||||
help_text=_(
|
||||
"Width of the page image this zone was drawn on (px). "
|
||||
"Falls back to template source_width if unset.",
|
||||
),
|
||||
)
|
||||
zone_source_height = models.PositiveIntegerField(
|
||||
_("zone source height"),
|
||||
null=True,
|
||||
blank=True,
|
||||
help_text=_(
|
||||
"Height of the page image this zone was drawn on (px). "
|
||||
"Falls back to template source_height if unset.",
|
||||
),
|
||||
)
|
||||
|
||||
ocr_language = models.CharField(
|
||||
_("OCR language"),
|
||||
max_length=20,
|
||||
default="eng",
|
||||
help_text=_("Tesseract language code(s), e.g. 'deu+eng'"),
|
||||
)
|
||||
|
||||
class TransformType(models.TextChoices):
|
||||
NONE = ("none", _("None"))
|
||||
STRIP = ("strip", _("Strip whitespace"))
|
||||
UPPERCASE = ("uppercase", _("Uppercase"))
|
||||
LOWERCASE = ("lowercase", _("Lowercase"))
|
||||
NUMERIC = ("numeric", _("Numeric only"))
|
||||
STRIP_PUNCTUATION = (
|
||||
"strip_punctuation",
|
||||
_("Remove leading/trailing punctuation"),
|
||||
)
|
||||
DATE = ("date", _("Parse date"))
|
||||
QR_CODE = ("qr_code", _("Read QR/barcode"))
|
||||
|
||||
transform = models.CharField(
|
||||
_("transform"),
|
||||
max_length=20,
|
||||
choices=TransformType.choices,
|
||||
default=TransformType.STRIP,
|
||||
)
|
||||
|
||||
date_format = models.CharField(
|
||||
_("date format"),
|
||||
max_length=64,
|
||||
blank=True,
|
||||
default="",
|
||||
help_text=_(
|
||||
"Python strptime format for the 'Parse date' transform "
|
||||
"(e.g. %d.%m.%Y). Blank = auto-detect.",
|
||||
),
|
||||
)
|
||||
|
||||
validation_regex = models.CharField(
|
||||
_("validation regex"),
|
||||
max_length=256,
|
||||
blank=True,
|
||||
default="",
|
||||
help_text=_(
|
||||
"Optional regex pattern — extracted text is only accepted if it matches",
|
||||
),
|
||||
)
|
||||
|
||||
order = models.PositiveIntegerField(_("order"), default=0)
|
||||
|
||||
class Meta:
|
||||
ordering = ("template", "order")
|
||||
verbose_name = _("OCR template zone")
|
||||
verbose_name_plural = _("OCR template zones")
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.template.name} -> {self.name}"
|
||||
|
||||
|
||||
# Document links and select fields require object references, not OCR text.
|
||||
OCR_SUPPORTED_FIELD_TYPES = frozenset(
|
||||
{
|
||||
CustomField.FieldDataType.STRING,
|
||||
CustomField.FieldDataType.URL,
|
||||
CustomField.FieldDataType.DATE,
|
||||
CustomField.FieldDataType.INT,
|
||||
CustomField.FieldDataType.FLOAT,
|
||||
CustomField.FieldDataType.MONETARY,
|
||||
CustomField.FieldDataType.LONG_TEXT,
|
||||
CustomField.FieldDataType.BOOL,
|
||||
},
|
||||
)
|
||||
@@ -739,3 +739,100 @@ class AcknowledgeTasksPermissions(GlobalPermissionsByMethod):
|
||||
"OPTIONS": ["documents.change_paperlesstask"],
|
||||
"POST": ["documents.change_paperlesstask"],
|
||||
}
|
||||
|
||||
|
||||
def has_document_operation_permissions(
|
||||
*,
|
||||
user: User,
|
||||
documents: list[int],
|
||||
method,
|
||||
parameters: dict[str, Any],
|
||||
) -> bool:
|
||||
from documents import bulk_edit
|
||||
|
||||
if not user.is_active:
|
||||
return False
|
||||
if user.is_superuser:
|
||||
return True
|
||||
|
||||
root_docs = {
|
||||
get_root_document(doc)
|
||||
for doc in Document.objects.select_related(
|
||||
"owner",
|
||||
"root_document__owner",
|
||||
).filter(pk__in=documents)
|
||||
}
|
||||
user_is_owner_of_all_documents = all(
|
||||
(doc.owner == user or doc.owner is None) for doc in root_docs
|
||||
)
|
||||
|
||||
# check global and object permissions for all documents
|
||||
has_perms = (
|
||||
user.has_perm(
|
||||
"documents.change_document",
|
||||
)
|
||||
and not Document.global_objects.filter(pk__in=documents)
|
||||
.exclude(pk__in=permitted_document_ids(user, perm="change_document"))
|
||||
.exists()
|
||||
)
|
||||
|
||||
# check ownership for methods that change original document
|
||||
if (
|
||||
(
|
||||
has_perms
|
||||
and method
|
||||
in [
|
||||
bulk_edit.set_permissions,
|
||||
bulk_edit.delete,
|
||||
bulk_edit.rotate,
|
||||
bulk_edit.delete_pages,
|
||||
bulk_edit.edit_pdf,
|
||||
bulk_edit.merge_as_versions,
|
||||
bulk_edit.remove_password,
|
||||
]
|
||||
)
|
||||
or (
|
||||
method in [bulk_edit.merge, bulk_edit.split]
|
||||
and parameters.get("delete_originals")
|
||||
)
|
||||
or (method == bulk_edit.edit_pdf and parameters.get("update_document"))
|
||||
):
|
||||
has_perms = has_perms and user_is_owner_of_all_documents
|
||||
|
||||
# check global add permissions for methods that create documents
|
||||
if (
|
||||
has_perms
|
||||
and (
|
||||
method in [bulk_edit.split, bulk_edit.merge]
|
||||
or (
|
||||
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
|
||||
and not parameters.get("update_document")
|
||||
)
|
||||
)
|
||||
and not user.has_perm("documents.add_document")
|
||||
):
|
||||
has_perms = False
|
||||
|
||||
# check global delete permissions for methods that delete documents
|
||||
if (
|
||||
has_perms
|
||||
and (
|
||||
method == bulk_edit.delete
|
||||
# Sources stop being documents of their own, and removing one
|
||||
# again afterwards needs delete_document
|
||||
or method == bulk_edit.merge_as_versions
|
||||
or (
|
||||
method in [bulk_edit.merge, bulk_edit.split]
|
||||
and parameters.get("delete_originals")
|
||||
)
|
||||
or (
|
||||
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
|
||||
and parameters.get("delete_original")
|
||||
and not parameters.get("update_document")
|
||||
)
|
||||
)
|
||||
and not user.has_perm("documents.delete_document")
|
||||
):
|
||||
has_perms = False
|
||||
|
||||
return has_perms
|
||||
@@ -77,33 +77,6 @@ def safe_regex_match(pattern: str, text: str, *, flags: int = 0):
|
||||
return None
|
||||
|
||||
|
||||
def safe_regex_fullmatch(pattern: str, text: str, *, flags: int = 0):
|
||||
"""
|
||||
Run a regex full match with a timeout. Returns a match object or None.
|
||||
Validation errors and timeouts are logged and treated as no match.
|
||||
"""
|
||||
|
||||
try:
|
||||
validate_regex_pattern(pattern)
|
||||
compiled = regex.compile(pattern, flags=flags)
|
||||
except (regex.error, ValueError) as exc:
|
||||
logger.exception(
|
||||
"Error while processing regular expression %s: %s",
|
||||
textwrap.shorten(pattern, width=80, placeholder="…"),
|
||||
exc,
|
||||
)
|
||||
return None
|
||||
|
||||
try:
|
||||
return compiled.fullmatch(text, timeout=REGEX_TIMEOUT_SECONDS)
|
||||
except TimeoutError:
|
||||
logger.warning(
|
||||
"Regular expression matching timed out for pattern %s",
|
||||
textwrap.shorten(pattern, width=80, placeholder="…"),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def safe_regex_sub(pattern: str, repl: str, text: str, *, flags: int = 0) -> str | None:
|
||||
"""
|
||||
Run a regex substitution with a timeout. Returns the substituted string,
|
||||
|
||||
@@ -23,7 +23,6 @@ from django.core.validators import MaxValueValidator
|
||||
from django.core.validators import MinValueValidator
|
||||
from django.core.validators import RegexValidator
|
||||
from django.core.validators import integer_validator
|
||||
from django.db import transaction
|
||||
from django.db.models import Count
|
||||
from django.db.models import Q
|
||||
from django.db.models.functions import Lower
|
||||
@@ -55,7 +54,6 @@ if settings.AUDIT_LOG_ENABLED:
|
||||
from documents import bulk_edit
|
||||
from documents.data_models import DocumentSource
|
||||
from documents.filters import CustomFieldQueryParser
|
||||
from documents.models import OCR_SUPPORTED_FIELD_TYPES
|
||||
from documents.models import Correspondent
|
||||
from documents.models import CustomField
|
||||
from documents.models import CustomFieldInstance
|
||||
@@ -64,8 +62,6 @@ from documents.models import DocumentBarcode
|
||||
from documents.models import DocumentType
|
||||
from documents.models import MatchingModel
|
||||
from documents.models import Note
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import OcrTemplateZone
|
||||
from documents.models import PaperlessTask
|
||||
from documents.models import SavedView
|
||||
from documents.models import SavedViewFilterRule
|
||||
@@ -1803,18 +1799,6 @@ class ReprocessDocumentsSerializer(DocumentSelectionSerializer):
|
||||
remote_ocr = serializers.BooleanField(required=False, default=False)
|
||||
|
||||
|
||||
class RunZoneOcrSerializer(serializers.Serializer):
|
||||
template = serializers.PrimaryKeyRelatedField(
|
||||
queryset=OcrTemplate.objects.prefetch_related("zones", "zones__custom_field"),
|
||||
required=False,
|
||||
allow_null=True,
|
||||
)
|
||||
|
||||
|
||||
class RunZoneOcrDocumentsSerializer(DocumentSelectionSerializer, RunZoneOcrSerializer):
|
||||
pass
|
||||
|
||||
|
||||
class BulkEditSerializer(
|
||||
SerializerWithPerms,
|
||||
DocumentSelectionSerializer,
|
||||
@@ -3692,181 +3676,3 @@ class StoragePathTestSerializer(SerializerWithPerms):
|
||||
document_field.queryset = Document.objects.filter(
|
||||
id__in=permitted_document_ids(user),
|
||||
)
|
||||
|
||||
|
||||
class OcrTemplateZoneSerializer(serializers.ModelSerializer):
|
||||
class Meta:
|
||||
model = OcrTemplateZone
|
||||
fields = [
|
||||
"id",
|
||||
"name",
|
||||
"target",
|
||||
"custom_field",
|
||||
"page",
|
||||
"x",
|
||||
"y",
|
||||
"width",
|
||||
"height",
|
||||
"ocr_language",
|
||||
"transform",
|
||||
"date_format",
|
||||
"order",
|
||||
"zone_source_width",
|
||||
"zone_source_height",
|
||||
"validation_regex",
|
||||
]
|
||||
|
||||
def validate_custom_field(self, value):
|
||||
if value is None:
|
||||
return value
|
||||
if value.data_type not in OCR_SUPPORTED_FIELD_TYPES:
|
||||
raise serializers.ValidationError(
|
||||
f"Custom field type '{value.data_type}' is not supported for OCR extraction. "
|
||||
f"Use string, integer, float, date, monetary, boolean, URL, or long text.",
|
||||
)
|
||||
return value
|
||||
|
||||
def validate_page(self, value):
|
||||
if value is not None and value != -1 and value < 1:
|
||||
raise serializers.ValidationError(
|
||||
"Page must be positive or -1 for the last page.",
|
||||
)
|
||||
return value
|
||||
|
||||
def validate(self, data):
|
||||
# A template PATCH still replaces each zone in full.
|
||||
if self.root.partial:
|
||||
for name, field in self.fields.items():
|
||||
if field.required and name not in data:
|
||||
raise serializers.ValidationError(
|
||||
{name: field.error_messages["required"]},
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
class OcrTemplateZoneTestSerializer(OcrTemplateZoneSerializer):
|
||||
name = serializers.CharField(default="test")
|
||||
x = serializers.IntegerField(min_value=0, default=0)
|
||||
y = serializers.IntegerField(min_value=0, default=0)
|
||||
width = serializers.IntegerField(min_value=2)
|
||||
height = serializers.IntegerField(min_value=2)
|
||||
|
||||
class Meta(OcrTemplateZoneSerializer.Meta):
|
||||
fields = [
|
||||
"name",
|
||||
"x",
|
||||
"y",
|
||||
"width",
|
||||
"height",
|
||||
"page",
|
||||
"ocr_language",
|
||||
"transform",
|
||||
"date_format",
|
||||
"validation_regex",
|
||||
"zone_source_width",
|
||||
"zone_source_height",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"zone_source_width": {"min_value": 1},
|
||||
"zone_source_height": {"min_value": 1},
|
||||
}
|
||||
|
||||
|
||||
class OcrTemplateTestSerializer(serializers.Serializer):
|
||||
document = serializers.PrimaryKeyRelatedField(
|
||||
queryset=Document.objects.select_related("root_document__owner"),
|
||||
)
|
||||
zone = OcrTemplateZoneTestSerializer()
|
||||
|
||||
|
||||
class OcrTemplateSerializer(MatchingModelSerializer, OwnedObjectSerializer):
|
||||
zones = OcrTemplateZoneSerializer(many=True, required=False)
|
||||
combine_formats = serializers.DictField(
|
||||
child=serializers.CharField(allow_blank=True, trim_whitespace=False),
|
||||
required=False,
|
||||
)
|
||||
|
||||
# Templates are never assigned to documents
|
||||
slug = None
|
||||
document_count = None
|
||||
|
||||
class Meta:
|
||||
model = OcrTemplate
|
||||
fields = [
|
||||
"id",
|
||||
"name",
|
||||
"match",
|
||||
"matching_algorithm",
|
||||
"is_insensitive",
|
||||
"source_width",
|
||||
"source_height",
|
||||
"sample_document",
|
||||
"enabled",
|
||||
"combine_formats",
|
||||
"created",
|
||||
"updated",
|
||||
"zones",
|
||||
"owner",
|
||||
"permissions",
|
||||
"user_can_change",
|
||||
"set_permissions",
|
||||
]
|
||||
read_only_fields = ["created", "updated"]
|
||||
|
||||
def validate_matching_algorithm(self, value):
|
||||
if value == MatchingModel.MATCH_AUTO:
|
||||
raise serializers.ValidationError(
|
||||
"Automatic matching is not supported for OCR templates.",
|
||||
)
|
||||
return value
|
||||
|
||||
def validate(self, data):
|
||||
data = super().validate(data)
|
||||
zones = data.get("zones", [])
|
||||
if (
|
||||
self.instance
|
||||
and "zones" not in data
|
||||
and ("source_width" in data or "source_height" in data)
|
||||
):
|
||||
zones = self.instance.zones.values()
|
||||
source_width = data.get(
|
||||
"source_width",
|
||||
getattr(self.instance, "source_width", None),
|
||||
)
|
||||
source_height = data.get(
|
||||
"source_height",
|
||||
getattr(self.instance, "source_height", None),
|
||||
)
|
||||
for zone in zones:
|
||||
width = zone.get("zone_source_width") or source_width
|
||||
height = zone.get("zone_source_height") or source_height
|
||||
if (
|
||||
zone.get("x", 0) + zone.get("width", 0) > width
|
||||
or zone.get("y", 0) + zone.get("height", 0) > height
|
||||
):
|
||||
raise serializers.ValidationError(
|
||||
{
|
||||
"zones": f"Zone '{zone.get('name', '?')}' extends beyond its source dimensions.",
|
||||
},
|
||||
)
|
||||
return data
|
||||
|
||||
@transaction.atomic
|
||||
def create(self, validated_data):
|
||||
zones_data = validated_data.pop("zones", [])
|
||||
template = super().create(validated_data)
|
||||
for zone_data in zones_data:
|
||||
OcrTemplateZone.objects.create(template=template, **zone_data)
|
||||
return template
|
||||
|
||||
@transaction.atomic
|
||||
def update(self, instance, validated_data):
|
||||
zones_data = validated_data.pop("zones", None)
|
||||
instance = super().update(instance, validated_data)
|
||||
|
||||
if zones_data is not None:
|
||||
instance.zones.all().delete()
|
||||
for zone_data in zones_data:
|
||||
OcrTemplateZone.objects.create(template=instance, **zone_data)
|
||||
|
||||
return instance
|
||||
@@ -8,6 +8,7 @@ from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
from typing import Any
|
||||
|
||||
from asgiref.sync import async_to_sync
|
||||
from celery import shared_task
|
||||
from celery.signals import before_task_publish
|
||||
from celery.signals import task_failure
|
||||
@@ -16,6 +17,7 @@ from celery.signals import task_prerun
|
||||
from celery.signals import task_revoked
|
||||
from celery.signals import worker_process_init
|
||||
from celery.signals import worker_process_shutdown
|
||||
from channels.layers import get_channel_layer
|
||||
from django.conf import settings
|
||||
from django.contrib.auth.models import Group
|
||||
from django.contrib.auth.models import User
|
||||
@@ -1051,6 +1053,8 @@ def run_workflows(
|
||||
# Task tracking -- Celery signal handlers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
BULK_OPERATION_TASK_NAME = "documents.bulk_operations.execute_bulk_operation"
|
||||
|
||||
TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
|
||||
"documents.tasks.consume_file": PaperlessTask.TaskType.CONSUME_FILE,
|
||||
"documents.tasks.train_classifier": PaperlessTask.TaskType.TRAIN_CLASSIFIER,
|
||||
@@ -1064,6 +1068,7 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
|
||||
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
|
||||
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
|
||||
"documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
|
||||
BULK_OPERATION_TASK_NAME: PaperlessTask.TaskType.BULK_UPDATE,
|
||||
}
|
||||
|
||||
_CELERY_STATE_TO_STATUS: dict[str, PaperlessTask.Status] = {
|
||||
@@ -1081,8 +1086,7 @@ def _extract_input_data(
|
||||
|
||||
For consume_file tasks this includes the filename, MIME type, and any
|
||||
non-null overrides from the DocumentMetadataOverrides object. For
|
||||
mail_fetch tasks it captures the account_ids list. All other task
|
||||
types store no input data and return {}.
|
||||
mail_fetch tasks it captures the account_ids list.
|
||||
"""
|
||||
if task_type == PaperlessTask.TaskType.CONSUME_FILE:
|
||||
input_doc = task_kwargs.get("input_doc")
|
||||
@@ -1147,7 +1151,7 @@ def _extract_owner_id(
|
||||
task_type: PaperlessTask.TaskType,
|
||||
task_kwargs: dict,
|
||||
) -> int | None:
|
||||
"""Return the owner_id from consume_file overrides, or None for all other task types."""
|
||||
"""Return the document owner for consumption tasks."""
|
||||
if task_type != PaperlessTask.TaskType.CONSUME_FILE:
|
||||
return None
|
||||
overrides = task_kwargs.get("overrides")
|
||||
@@ -1186,9 +1190,16 @@ def before_task_publish_handler(
|
||||
_, task_kwargs, _ = body
|
||||
task_id = headers["id"]
|
||||
|
||||
input_data = _extract_input_data(task_type, task_kwargs)
|
||||
if task_name == BULK_OPERATION_TASK_NAME:
|
||||
input_data = {
|
||||
"operation": task_kwargs["operation"],
|
||||
"document_count": len(task_kwargs["document_ids"]),
|
||||
}
|
||||
owner_id = task_kwargs["user_id"]
|
||||
else:
|
||||
input_data = _extract_input_data(task_type, task_kwargs)
|
||||
owner_id = _extract_owner_id(task_type, task_kwargs)
|
||||
trigger_source = _determine_trigger_source(headers)
|
||||
owner_id = _extract_owner_id(task_type, task_kwargs)
|
||||
|
||||
# A retried task is republished with the same task_id, so this fires
|
||||
# again for it; get_or_create keeps the original PENDING record
|
||||
@@ -1228,6 +1239,30 @@ def task_prerun_handler(sender=None, task_id=None, task=None, **kwargs) -> None:
|
||||
logger.exception("Setting PaperlessTask started failed")
|
||||
|
||||
|
||||
def _send_bulk_operation_status(task: PaperlessTask | None) -> None:
|
||||
"""Send the recorded outcome to the initiator."""
|
||||
try:
|
||||
if task is None or task.owner_id is None:
|
||||
return
|
||||
result_data = task.result_data or {}
|
||||
async_to_sync(get_channel_layer().group_send)(
|
||||
f"bulk_operations_{task.owner_id}",
|
||||
{
|
||||
"type": "bulk_operation_status",
|
||||
"data": {
|
||||
"task_id": task.task_id,
|
||||
"status": task.status,
|
||||
"operation": task.input_data["operation"],
|
||||
"error": result_data.get("error_message")
|
||||
or result_data.get("reason", ""),
|
||||
},
|
||||
},
|
||||
)
|
||||
except Exception:
|
||||
# Just a notification failure
|
||||
logger.exception("Sending bulk operation status failed")
|
||||
|
||||
|
||||
@task_postrun.connect
|
||||
def task_postrun_handler(
|
||||
sender=None,
|
||||
@@ -1285,6 +1320,8 @@ def task_postrun_handler(
|
||||
changed_fields.append("status")
|
||||
|
||||
task_instance.save(update_fields=changed_fields)
|
||||
if task and task.name == BULK_OPERATION_TASK_NAME:
|
||||
_send_bulk_operation_status(task_instance)
|
||||
except Exception: # pragma: no cover
|
||||
logger.exception("Updating PaperlessTask failed")
|
||||
|
||||
@@ -1350,6 +1387,8 @@ def task_failure_handler(
|
||||
date_started - date_created
|
||||
).total_seconds()
|
||||
task_qs.update(**update_fields)
|
||||
if sender and sender.name == BULK_OPERATION_TASK_NAME:
|
||||
_send_bulk_operation_status(task_qs.first())
|
||||
except Exception: # pragma: no cover
|
||||
logger.exception("Updating PaperlessTask on failure failed")
|
||||
|
||||
@@ -1386,13 +1425,16 @@ def task_revoked_handler(
|
||||
reason = "Task expired before execution."
|
||||
elif terminated:
|
||||
reason = "Task terminated during execution."
|
||||
PaperlessTask.objects.filter(task_id=task_id).update(
|
||||
task_qs = PaperlessTask.objects.filter(task_id=task_id)
|
||||
task_qs.update(
|
||||
status=PaperlessTask.Status.REVOKED,
|
||||
date_done=timezone.now(),
|
||||
result_data={
|
||||
"reason": (reason + " Check worker logs for details."),
|
||||
},
|
||||
)
|
||||
if sender and sender.name == BULK_OPERATION_TASK_NAME:
|
||||
_send_bulk_operation_status(task_qs.first())
|
||||
except Exception: # pragma: no cover
|
||||
logger.exception("Updating PaperlessTask on revocation failed")
|
||||
|
||||
@@ -1412,19 +1454,6 @@ def close_connection_pool_on_worker_init(**kwargs) -> None:
|
||||
conn.close_pool()
|
||||
|
||||
|
||||
def run_zone_ocr_extraction(sender, document, original_file=None, **kwargs):
|
||||
"""Apply matching templates without failing consumption if extraction fails."""
|
||||
try:
|
||||
from documents.zone_ocr import run_zone_extraction
|
||||
|
||||
run_zone_extraction(document, Path(original_file) if original_file else None)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Zone OCR extraction failed for document %s",
|
||||
document.pk,
|
||||
)
|
||||
|
||||
|
||||
@worker_process_shutdown.connect
|
||||
def close_connection_pool_on_worker_shutdown(**kwargs) -> None: # pragma: no cover
|
||||
"""
|
||||
|
||||
+15
-28
@@ -12,6 +12,7 @@ from celery import Task
|
||||
from celery import shared_task
|
||||
from django.conf import settings
|
||||
from django.contrib.contenttypes.models import ContentType
|
||||
from django.contrib.sessions.models import Session
|
||||
from django.db import models
|
||||
from django.db import transaction
|
||||
from django.db.models.signals import post_save
|
||||
@@ -47,7 +48,6 @@ from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentBarcode
|
||||
from documents.models import DocumentType
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import PaperlessTask
|
||||
from documents.models import ShareLink
|
||||
from documents.models import ShareLinkBundle
|
||||
@@ -69,9 +69,7 @@ from documents.utils import IterWrapper
|
||||
from documents.utils import compute_checksum
|
||||
from documents.utils import identity
|
||||
from documents.versioning import annotate_effective_content
|
||||
from documents.versioning import latest_version
|
||||
from documents.workflows.utils import get_workflows_for_trigger
|
||||
from documents.zone_ocr import run_zone_extraction
|
||||
from paperless.config import AIConfig
|
||||
from paperless.config import BarcodeConfig
|
||||
from paperless.config import RemoteOCRConfig
|
||||
@@ -370,31 +368,6 @@ def _read_barcodes_for_reprocess(document: Document) -> list[StoredBarcode] | No
|
||||
return None
|
||||
|
||||
|
||||
@shared_task
|
||||
def extract_zones(document_id: int, template_id: int | None = None) -> None:
|
||||
"""
|
||||
Run an OCR template against a stored document, or every template that
|
||||
matches it when no template is given.
|
||||
"""
|
||||
document = Document.objects.get(id=document_id)
|
||||
templates = None
|
||||
if template_id is not None:
|
||||
try:
|
||||
template = OcrTemplate.objects.prefetch_related(
|
||||
"zones",
|
||||
"zones__custom_field",
|
||||
).get(id=template_id)
|
||||
except OcrTemplate.DoesNotExist:
|
||||
logger.warning(
|
||||
"OCR template %s no longer exists, not running zone OCR",
|
||||
template_id,
|
||||
)
|
||||
return
|
||||
templates = [template]
|
||||
|
||||
run_zone_extraction(latest_version(document), templates=templates, reindex=True)
|
||||
|
||||
|
||||
@shared_task
|
||||
def update_document_content_maybe_archive_file(
|
||||
document_id,
|
||||
@@ -953,3 +926,17 @@ def cleanup_expired_share_link_bundles() -> None:
|
||||
)
|
||||
if count:
|
||||
logger.info("Deleted %s expired share link bundle(s)", count)
|
||||
|
||||
|
||||
@shared_task
|
||||
def clear_expired_sessions() -> None:
|
||||
"""
|
||||
Remove expired database sessions in batches to limit the size of each delete.
|
||||
"""
|
||||
expired = Session.objects.filter(expire_date__lt=timezone.now())
|
||||
while session_keys := list(
|
||||
expired.order_by("expire_date").values_list("pk", flat=True)[:500],
|
||||
):
|
||||
expired.filter(pk__in=session_keys).delete()
|
||||
|
||||
logger.info("Cleared expired sessions")
|
||||
@@ -287,6 +287,28 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(config.user_args, user_args)
|
||||
self.assertEqual(config.barcode_tag_mapping, barcode_tag_mapping)
|
||||
|
||||
def test_api_update_config_rejects_reserved_user_args(self) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- API request to update app config with user_args which set an
|
||||
argument the parser manages itself
|
||||
WHEN:
|
||||
- API is called
|
||||
THEN:
|
||||
- Request is rejected with a 400 naming the argument
|
||||
- Config is not updated
|
||||
"""
|
||||
response = self.client.patch(
|
||||
f"{self.ENDPOINT}1/",
|
||||
json.dumps({"user_args": json.dumps({"sidecar": "/tmp/x.py"})}),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
|
||||
self.assertIn("sidecar", str(response.data["user_args"]))
|
||||
config = ApplicationConfiguration.objects.first()
|
||||
assert config is not None
|
||||
self.assertIsNone(config.user_args)
|
||||
|
||||
def test_api_update_config_rejects_invalid_json_objects(self) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
|
||||
@@ -7,12 +7,12 @@ from django.test import override_settings
|
||||
from rest_framework import status
|
||||
from rest_framework.test import APITestCase
|
||||
|
||||
from documents.bulk_operations import execute_bulk_operation
|
||||
from documents.models import Correspondent
|
||||
from documents.models import CustomField
|
||||
from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentType
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import StoragePath
|
||||
from documents.models import Tag
|
||||
from paperless_testing.dirs import DirectoriesMixin
|
||||
@@ -30,9 +30,18 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.user = user
|
||||
self.client.force_authenticate(user=user)
|
||||
|
||||
patcher = mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
patcher = mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
self.async_task = patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
# Exercise accepted requests and worker execution together in these
|
||||
# operation/validation tests. Queue-only behavior is tested separately.
|
||||
enqueue = mock.patch(
|
||||
"documents.views.execute_bulk_operation.apply_async",
|
||||
side_effect=lambda *, kwargs, headers: self.run_operation(kwargs),
|
||||
)
|
||||
enqueue.start()
|
||||
self.addCleanup(enqueue.stop)
|
||||
|
||||
self.c1 = Correspondent.objects.create(name="c1")
|
||||
self.c2 = Correspondent.objects.create(name="c2")
|
||||
self.dt1 = DocumentType.objects.create(name="dt1")
|
||||
@@ -62,11 +71,16 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.cf1 = CustomField.objects.create(name="cf1", data_type="string")
|
||||
self.cf2 = CustomField.objects.create(name="cf2", data_type="string")
|
||||
|
||||
@staticmethod
|
||||
def run_operation(kwargs):
|
||||
execute_bulk_operation(**kwargs)
|
||||
return mock.Mock(id="test-bulk-operation")
|
||||
|
||||
def setup_mock(self, m, method_name, return_value="OK") -> None:
|
||||
m.return_value = return_value
|
||||
m.__name__ = method_name
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
def test_api_set_correspondent(self, bulk_update_task_mock) -> None:
|
||||
self.assertNotEqual(self.doc1.correspondent, self.c1)
|
||||
response = self.client.post(
|
||||
@@ -85,11 +99,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(self.doc1.correspondent, self.c1)
|
||||
bulk_update_task_mock.assert_called_once()
|
||||
self.assertCountEqual(
|
||||
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
|
||||
bulk_update_task_mock.call_args.args[0],
|
||||
[self.doc1.pk],
|
||||
)
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
def test_api_unset_correspondent(self, bulk_update_task_mock) -> None:
|
||||
self.doc1.correspondent = self.c1
|
||||
self.doc1.save()
|
||||
@@ -111,7 +125,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.doc1.refresh_from_db()
|
||||
self.assertIsNone(self.doc1.correspondent)
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
def test_api_set_type(self, bulk_update_task_mock) -> None:
|
||||
self.assertNotEqual(self.doc1.document_type, self.dt1)
|
||||
response = self.client.post(
|
||||
@@ -130,11 +144,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(self.doc1.document_type, self.dt1)
|
||||
bulk_update_task_mock.assert_called_once()
|
||||
self.assertCountEqual(
|
||||
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
|
||||
bulk_update_task_mock.call_args.args[0],
|
||||
[self.doc1.pk],
|
||||
)
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
def test_api_unset_type(self, bulk_update_task_mock) -> None:
|
||||
self.doc1.document_type = self.dt1
|
||||
self.doc1.save()
|
||||
@@ -155,11 +169,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertIsNone(self.doc1.document_type)
|
||||
bulk_update_task_mock.assert_called_once()
|
||||
self.assertCountEqual(
|
||||
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
|
||||
bulk_update_task_mock.call_args.args[0],
|
||||
[self.doc1.pk],
|
||||
)
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
def test_api_add_tag(self, bulk_update_task_mock) -> None:
|
||||
self.assertFalse(self.doc1.tags.filter(pk=self.t1.pk).exists())
|
||||
|
||||
@@ -181,11 +195,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
|
||||
bulk_update_task_mock.assert_called_once()
|
||||
self.assertCountEqual(
|
||||
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
|
||||
bulk_update_task_mock.call_args.args[0],
|
||||
[self.doc1.pk],
|
||||
)
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.bulk_operations.bulk_update_documents")
|
||||
def test_api_remove_tag(self, bulk_update_task_mock) -> None:
|
||||
self.doc1.tags.add(self.t1)
|
||||
|
||||
@@ -224,6 +238,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertListEqual(args[0], [self.doc1.id, self.doc3.id])
|
||||
self.assertEqual(kwargs["add_tags"], [self.t1.id])
|
||||
self.assertEqual(kwargs["remove_tags"], [self.t2.id])
|
||||
@@ -278,6 +295,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertListEqual(args[0], [self.doc1.id, self.doc3.id])
|
||||
self.assertEqual(kwargs["add_custom_fields"], [self.cf1.id])
|
||||
self.assertEqual(kwargs["remove_custom_fields"], [self.cf2.id])
|
||||
@@ -346,6 +366,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertListEqual(args[0], [self.doc1.id, self.doc3.id])
|
||||
self.assertEqual(kwargs["add_custom_fields"], {self.cf1.id: "foo"})
|
||||
self.assertEqual(kwargs["remove_custom_fields"], [self.cf2.id])
|
||||
@@ -507,6 +530,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(len(kwargs), 0)
|
||||
|
||||
@@ -521,6 +547,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(len(kwargs), 0)
|
||||
|
||||
@@ -545,6 +574,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(len(kwargs), 0)
|
||||
|
||||
@@ -559,6 +591,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(kwargs, {"remote_ocr": False})
|
||||
|
||||
@@ -581,90 +616,12 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(kwargs, {"remote_ocr": True})
|
||||
|
||||
@mock.patch("documents.views.bulk_edit.run_zone_ocr")
|
||||
def test_run_zone_ocr_documents_endpoint(self, m) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- API data to run zone OCR on a document
|
||||
WHEN:
|
||||
- API is called
|
||||
THEN:
|
||||
- run_zone_ocr is called with the document
|
||||
"""
|
||||
self.setup_mock(m, "run_zone_ocr")
|
||||
response = self.client.post(
|
||||
"/api/documents/run_zone_ocr/",
|
||||
json.dumps({"documents": [self.doc1.id]}),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(kwargs, {})
|
||||
|
||||
@mock.patch("documents.views.bulk_edit.run_zone_ocr")
|
||||
def test_run_zone_ocr_documents_endpoint_with_template(self, m) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- API data to run a chosen OCR template on a document
|
||||
WHEN:
|
||||
- API is called
|
||||
THEN:
|
||||
- run_zone_ocr is called with the template
|
||||
"""
|
||||
self.setup_mock(m, "run_zone_ocr")
|
||||
template = OcrTemplate.objects.create(
|
||||
name="Bills",
|
||||
source_width=100,
|
||||
source_height=100,
|
||||
)
|
||||
response = self.client.post(
|
||||
"/api/documents/run_zone_ocr/",
|
||||
json.dumps({"documents": [self.doc1.id], "template": template.pk}),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
self.assertEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(kwargs, {"template": template})
|
||||
|
||||
@mock.patch("documents.views.bulk_edit.run_zone_ocr")
|
||||
def test_run_zone_ocr_documents_endpoint_rejects_private_template(
|
||||
self,
|
||||
m,
|
||||
) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A user who can change the documents but not view the template
|
||||
WHEN:
|
||||
- API is called with that template
|
||||
THEN:
|
||||
- The whole request is refused
|
||||
"""
|
||||
user = UserFactory(username="editor")
|
||||
grant_global(user, "view_document", "change_document")
|
||||
template = OcrTemplate.objects.create(
|
||||
name="Private",
|
||||
owner=UserFactory(username="other"),
|
||||
source_width=100,
|
||||
source_height=100,
|
||||
)
|
||||
self.client.force_authenticate(user=user)
|
||||
|
||||
response = self.client.post(
|
||||
"/api/documents/run_zone_ocr/",
|
||||
json.dumps({"documents": [self.doc1.id], "template": template.pk}),
|
||||
content_type="application/json",
|
||||
)
|
||||
|
||||
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
|
||||
m.assert_not_called()
|
||||
|
||||
@mock.patch("documents.serialisers.bulk_edit.set_storage_path")
|
||||
def test_api_set_storage_path(self, m) -> None:
|
||||
"""
|
||||
@@ -691,6 +648,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
|
||||
self.assertListEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(kwargs["storage_path"], self.sp1.id)
|
||||
@@ -721,6 +681,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
|
||||
self.assertListEqual(args[0], [self.doc1.id])
|
||||
self.assertEqual(kwargs["storage_path"], None)
|
||||
@@ -843,6 +806,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc2.id])
|
||||
self.assertEqual(kwargs["storage_path"], self.sp1.id)
|
||||
|
||||
@@ -866,6 +832,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [self.doc1.id, self.doc3.id, self.doc5.id])
|
||||
self.assertEqual(kwargs["storage_path"], self.sp1.id)
|
||||
|
||||
@@ -902,6 +871,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [first_duplicate.id, second_duplicate.id])
|
||||
self.assertEqual(kwargs["storage_path"], self.sp1.id)
|
||||
|
||||
@@ -938,6 +910,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
get_backend.return_value.search_ids.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value
|
||||
for key, value in kwargs.items()
|
||||
if key != "update_callback"
|
||||
}
|
||||
self.assertEqual(args[0], [self.doc2.id])
|
||||
self.assertEqual(kwargs["storage_path"], self.sp1.id)
|
||||
|
||||
@@ -1314,6 +1291,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [self.doc2.id, self.doc3.id])
|
||||
self.assertEqual(len(kwargs["set_permissions"]["view"]["users"]), 2)
|
||||
|
||||
@@ -1726,6 +1706,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [self.doc2.id, self.doc3.id])
|
||||
self.assertEqual(kwargs["degrees"], 90)
|
||||
self.assertEqual(kwargs["source_mode"], "latest_version")
|
||||
@@ -2012,6 +1995,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [self.doc2.id, self.doc3.id])
|
||||
self.assertEqual(kwargs["metadata_document_id"], self.doc3.id)
|
||||
self.assertEqual(kwargs["source_mode"], "latest_version")
|
||||
@@ -2219,6 +2205,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [self.doc2.id])
|
||||
self.assertEqual(kwargs["operations"], [{"page": 1}])
|
||||
self.assertEqual(kwargs["source_mode"], "explicit_selection")
|
||||
@@ -2564,6 +2553,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
m.assert_called_once()
|
||||
args, kwargs = m.call_args
|
||||
kwargs = {
|
||||
key: value for key, value in kwargs.items() if key != "update_callback"
|
||||
}
|
||||
self.assertCountEqual(args[0], [self.doc2.id])
|
||||
self.assertEqual(kwargs["password"], "secret")
|
||||
self.assertTrue(kwargs["update_document"])
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -149,10 +149,10 @@ class TestPostSelectionPathsEnforceTheCap:
|
||||
assert str(_MAX_QUERY_LENGTH + 1) in message
|
||||
parse_spy.assert_not_called()
|
||||
|
||||
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
|
||||
@mock.patch("documents.views.execute_bulk_operation.apply_async")
|
||||
def test_bulk_edit_query_at_exactly_the_cap_is_accepted(
|
||||
self,
|
||||
bulk_update_task_mock: mock.MagicMock,
|
||||
enqueue_mock: mock.MagicMock,
|
||||
admin_client: APIClient,
|
||||
searchable_document: Document,
|
||||
) -> None:
|
||||
@@ -163,12 +163,12 @@ class TestPostSelectionPathsEnforceTheCap:
|
||||
- Its `filters.query` is exactly `_MAX_QUERY_LENGTH` characters
|
||||
long
|
||||
THEN:
|
||||
- The cap check accepts it and the request reaches the real
|
||||
bulk-edit method (its Celery dispatch is mocked out here,
|
||||
same as every other bulk-edit test, since nothing here is
|
||||
testing that method itself)
|
||||
- The cap check accepts it and queues the metadata operation.
|
||||
Mock the executor dispatch so this exercises query validation
|
||||
without requiring a broker.
|
||||
"""
|
||||
query = "a" * _MAX_QUERY_LENGTH
|
||||
enqueue_mock.return_value.id = "query-cap-task"
|
||||
|
||||
response = admin_client.post(
|
||||
"/api/documents/bulk_edit/",
|
||||
@@ -183,6 +183,11 @@ class TestPostSelectionPathsEnforceTheCap:
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.data == {"result": "OK", "task_id": "query-cap-task"}
|
||||
enqueue_mock.assert_called_once()
|
||||
assert (
|
||||
enqueue_mock.call_args.kwargs["kwargs"]["operation"] == "set_document_type"
|
||||
)
|
||||
|
||||
def test_bulk_download_query_one_over_the_cap_is_a_400(
|
||||
self,
|
||||
|
||||
@@ -21,13 +21,11 @@ from documents.models import CustomField
|
||||
from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentType
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import StoragePath
|
||||
from documents.models import Tag
|
||||
from documents.pdf_ops import PageSpec
|
||||
from documents.permissions import PermissionsSpec
|
||||
from documents.permissions import set_permissions_for_objects
|
||||
from documents.tasks import extract_zones
|
||||
from paperless_testing.dirs import DirectoriesMixin
|
||||
from paperless_testing.factories import DocumentFactory
|
||||
from paperless_testing.permissions import grant_object
|
||||
@@ -1876,111 +1874,6 @@ class TestPDFActions(DirectoriesMixin, TestCase):
|
||||
self.assertIn("Error removing password from document", cm.output[0])
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestBulkEditRunZoneOcr:
|
||||
@pytest.fixture
|
||||
def mock_task(self, mocker: MockerFixture) -> mock.MagicMock:
|
||||
return mocker.patch("documents.bulk_edit.extract_zones")
|
||||
|
||||
def test_queues_every_selected_document(
|
||||
self,
|
||||
mock_task: mock.MagicMock,
|
||||
) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- Selected documents
|
||||
WHEN:
|
||||
- run_zone_ocr is called with them
|
||||
THEN:
|
||||
- Each document is queued once, templates are matched when its
|
||||
task runs
|
||||
"""
|
||||
docs = DocumentFactory.create_batch(2)
|
||||
|
||||
result = bulk_edit.run_zone_ocr([doc.id for doc in docs])
|
||||
|
||||
assert result == "OK"
|
||||
assert [
|
||||
call.kwargs["kwargs"] for call in mock_task.apply_async.call_args_list
|
||||
] == [{"document_id": doc.id, "template_id": None} for doc in docs]
|
||||
|
||||
def test_queues_the_chosen_template(self, mock_task: mock.MagicMock) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A chosen OCR template
|
||||
WHEN:
|
||||
- run_zone_ocr is called with it
|
||||
THEN:
|
||||
- The template is queued with the document
|
||||
"""
|
||||
doc = DocumentFactory()
|
||||
template = OcrTemplate.objects.create(
|
||||
name="Bills",
|
||||
source_width=100,
|
||||
source_height=100,
|
||||
)
|
||||
|
||||
bulk_edit.run_zone_ocr([doc.id], template=template)
|
||||
|
||||
assert mock_task.apply_async.call_args.kwargs["kwargs"] == {
|
||||
"document_id": doc.id,
|
||||
"template_id": template.pk,
|
||||
}
|
||||
|
||||
def test_task_runs_extraction(self, mocker: MockerFixture) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A queued zone OCR task
|
||||
WHEN:
|
||||
- The task runs
|
||||
THEN:
|
||||
- Zone extraction runs on the stored document
|
||||
"""
|
||||
extract = mocker.patch("documents.tasks.run_zone_extraction")
|
||||
doc = DocumentFactory()
|
||||
|
||||
extract_zones(doc.id)
|
||||
|
||||
extract.assert_called_once_with(doc, templates=None, reindex=True)
|
||||
|
||||
def test_task_runs_the_chosen_template(self, mocker: MockerFixture) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A queued zone OCR task with a chosen template
|
||||
WHEN:
|
||||
- The task runs
|
||||
THEN:
|
||||
- Zone extraction runs with only that template
|
||||
"""
|
||||
extract = mocker.patch("documents.tasks.run_zone_extraction")
|
||||
doc = DocumentFactory()
|
||||
template = OcrTemplate.objects.create(
|
||||
name="Bills",
|
||||
source_width=100,
|
||||
source_height=100,
|
||||
)
|
||||
|
||||
extract_zones(doc.id, template.pk)
|
||||
|
||||
extract.assert_called_once_with(doc, templates=[template], reindex=True)
|
||||
|
||||
def test_task_skips_a_deleted_template(self, mocker: MockerFixture) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A template deleted between queueing and running
|
||||
WHEN:
|
||||
- The task runs
|
||||
THEN:
|
||||
- Nothing is extracted
|
||||
"""
|
||||
extract = mocker.patch("documents.tasks.run_zone_extraction")
|
||||
doc = DocumentFactory()
|
||||
|
||||
extract_zones(doc.id, 999)
|
||||
|
||||
extract.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestBulkEditReprocess:
|
||||
@pytest.fixture
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
"""Request acceptance, worker execution, and task tracking."""
|
||||
|
||||
import uuid
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
from auditlog.models import LogEntry
|
||||
from django.contrib.auth.models import User
|
||||
from rest_framework.exceptions import PermissionDenied
|
||||
from rest_framework.test import APIClient
|
||||
|
||||
from documents.bulk_operations import execute_bulk_operation
|
||||
from documents.models import Correspondent
|
||||
from documents.models import Document
|
||||
from documents.models import PaperlessTask
|
||||
from documents.signals.handlers import before_task_publish_handler
|
||||
from documents.signals.handlers import task_failure_handler
|
||||
from documents.signals.handlers import task_postrun_handler
|
||||
from documents.signals.handlers import task_revoked_handler
|
||||
from paperless_testing.factories import UserFactory
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def bulk_setup(paperless_dirs):
|
||||
user = UserFactory(superuser=True)
|
||||
client = APIClient()
|
||||
client.force_authenticate(user)
|
||||
doc = Document.objects.create(title="Example", checksum="bulk-op-example")
|
||||
correspondent = Correspondent.objects.create(name="Example correspondent")
|
||||
return user, client, doc, correspondent
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize(
|
||||
"endpoint,operation,parameters",
|
||||
[
|
||||
("bulk_edit", "set_correspondent", {"correspondent": None}),
|
||||
],
|
||||
)
|
||||
def test_request_only_queues(bulk_setup, endpoint, operation, parameters):
|
||||
user, client, doc, _ = bulk_setup
|
||||
task_id = str(uuid.uuid4())
|
||||
data = {"documents": [doc.pk], "from_webui": True}
|
||||
if endpoint == "bulk_edit":
|
||||
data.update(method=operation, parameters=parameters)
|
||||
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
|
||||
enqueue.return_value.id = task_id
|
||||
response = client.post(f"/api/documents/{endpoint}/", data, format="json")
|
||||
assert response.status_code == 200
|
||||
assert response.data == {"result": "OK", "task_id": task_id}
|
||||
assert Document.objects.filter(pk=doc.pk).exists()
|
||||
assert enqueue.call_args.kwargs["kwargs"] == {
|
||||
"operation": operation,
|
||||
"document_ids": [doc.pk],
|
||||
"parameters": parameters,
|
||||
"user_id": user.pk,
|
||||
}
|
||||
assert enqueue.call_args.kwargs["headers"]["trigger_source"] == "web_ui"
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_metadata_task_includes_audit_and_followup(bulk_setup, settings):
|
||||
user, _, doc, correspondent = bulk_setup
|
||||
settings.AUDIT_LOG_ENABLED = True
|
||||
with (
|
||||
mock.patch("documents.bulk_operations.bulk_update_documents") as followup,
|
||||
mock.patch(
|
||||
"documents.bulk_edit.bulk_update_documents.apply_async",
|
||||
) as extra_task,
|
||||
):
|
||||
|
||||
def check_followup(ids):
|
||||
doc.refresh_from_db()
|
||||
assert doc.correspondent_id == correspondent.pk
|
||||
assert LogEntry.objects.filter(object_pk=str(doc.pk), actor=user).exists()
|
||||
assert ids == [doc.pk]
|
||||
|
||||
followup.side_effect = check_followup
|
||||
result = execute_bulk_operation(
|
||||
"set_correspondent",
|
||||
[doc.pk],
|
||||
{"correspondent": correspondent.pk},
|
||||
user.pk,
|
||||
)
|
||||
followup.assert_called_once()
|
||||
extra_task.assert_not_called()
|
||||
assert result["document_count"] == 1
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_permissions_rechecked_on_worker(bulk_setup):
|
||||
user, _, doc, correspondent = bulk_setup
|
||||
User.objects.filter(pk=user.pk).update(is_superuser=False)
|
||||
with pytest.raises(PermissionDenied):
|
||||
execute_bulk_operation(
|
||||
"set_correspondent",
|
||||
[doc.pk],
|
||||
{"correspondent": correspondent.pk},
|
||||
user.pk,
|
||||
)
|
||||
doc.refresh_from_db()
|
||||
assert doc.correspondent_id is None
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_queue_failure_does_not_change_documents(bulk_setup):
|
||||
_, client, doc, correspondent = bulk_setup
|
||||
with mock.patch(
|
||||
"documents.views.execute_bulk_operation.apply_async",
|
||||
side_effect=RuntimeError("broker unavailable"),
|
||||
):
|
||||
response = client.post(
|
||||
"/api/documents/bulk_edit/",
|
||||
{
|
||||
"documents": [doc.pk],
|
||||
"method": "set_correspondent",
|
||||
"parameters": {"correspondent": correspondent.pk},
|
||||
},
|
||||
format="json",
|
||||
)
|
||||
assert response.status_code == 400
|
||||
doc.refresh_from_db()
|
||||
assert doc.correspondent_id is None
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize("failed", [False, True])
|
||||
def test_tracked_outcome_records_initiator_and_result(bulk_setup, failed):
|
||||
user, _, doc, _ = bulk_setup
|
||||
task_id = str(uuid.uuid4())
|
||||
before_task_publish_handler(
|
||||
headers={
|
||||
"task": "documents.bulk_operations.execute_bulk_operation",
|
||||
"id": task_id,
|
||||
},
|
||||
body=(
|
||||
(),
|
||||
{
|
||||
"operation": "set_permissions",
|
||||
"document_ids": [doc.pk],
|
||||
"user_id": user.pk,
|
||||
},
|
||||
{},
|
||||
),
|
||||
)
|
||||
task = PaperlessTask.objects.get(task_id=task_id)
|
||||
assert task.owner_id == user.pk
|
||||
assert task.task_type == PaperlessTask.TaskType.BULK_UPDATE
|
||||
if failed:
|
||||
task_failure_handler(task_id=task_id, exception=ValueError("Could not update"))
|
||||
else:
|
||||
task_postrun_handler(
|
||||
task_id=task_id,
|
||||
state="SUCCESS",
|
||||
retval={"document_count": 1},
|
||||
)
|
||||
task.refresh_from_db()
|
||||
assert task.status == ("failure" if failed else "success")
|
||||
if failed:
|
||||
assert task.result_data["error_message"] == "Could not update"
|
||||
else:
|
||||
assert task.result_data == {"document_count": 1}
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_large_filtered_selection_is_one_task(bulk_setup):
|
||||
_, client, _, _ = bulk_setup
|
||||
Document.objects.bulk_create(
|
||||
[
|
||||
Document(title="bulk-large-selection", checksum=f"large-{i}")
|
||||
for i in range(10_000)
|
||||
],
|
||||
)
|
||||
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
|
||||
enqueue.return_value.id = str(uuid.uuid4())
|
||||
response = client.post(
|
||||
"/api/documents/bulk_edit/",
|
||||
{
|
||||
"all": True,
|
||||
"filters": {"title__icontains": "bulk-large-selection"},
|
||||
"method": "set_correspondent",
|
||||
"parameters": {"correspondent": None},
|
||||
},
|
||||
format="json",
|
||||
)
|
||||
assert response.status_code == 200
|
||||
enqueue.assert_called_once()
|
||||
assert len(enqueue.call_args.kwargs["kwargs"]["document_ids"]) == 10_000
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize(
|
||||
("outcome", "expected_status", "expected_error"),
|
||||
[
|
||||
("success", "success", ""),
|
||||
("failure", "failure", "Could not update"),
|
||||
(
|
||||
"expired",
|
||||
"revoked",
|
||||
"Task expired before execution. Check worker logs for details.",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_tracked_outcome_notifies_only_initiator(
|
||||
bulk_setup,
|
||||
outcome,
|
||||
expected_status,
|
||||
expected_error,
|
||||
):
|
||||
user, _, doc, _ = bulk_setup
|
||||
task_id = str(uuid.uuid4())
|
||||
before_task_publish_handler(
|
||||
headers={
|
||||
"task": execute_bulk_operation.name,
|
||||
"id": task_id,
|
||||
},
|
||||
body=(
|
||||
(),
|
||||
{
|
||||
"operation": "set_permissions",
|
||||
"document_ids": [doc.pk],
|
||||
"user_id": user.pk,
|
||||
},
|
||||
{},
|
||||
),
|
||||
)
|
||||
task = PaperlessTask.objects.get(task_id=task_id)
|
||||
assert task.owner_id == user.pk
|
||||
assert task.task_type == PaperlessTask.TaskType.BULK_UPDATE
|
||||
with mock.patch("documents.signals.handlers.async_to_sync") as send:
|
||||
if outcome == "expired":
|
||||
task_revoked_handler(
|
||||
sender=execute_bulk_operation,
|
||||
request=mock.Mock(id=task_id),
|
||||
expired=True,
|
||||
)
|
||||
elif outcome == "failure":
|
||||
task_failure_handler(
|
||||
sender=execute_bulk_operation,
|
||||
task_id=task_id,
|
||||
exception=ValueError("Could not update"),
|
||||
)
|
||||
else:
|
||||
task_postrun_handler(
|
||||
task=execute_bulk_operation,
|
||||
task_id=task_id,
|
||||
state="SUCCESS",
|
||||
retval={"document_count": 1},
|
||||
)
|
||||
group, event = send.return_value.call_args.args
|
||||
assert group == f"bulk_operations_{user.pk}"
|
||||
assert event["data"]["status"] == expected_status
|
||||
assert event["data"]["error"] == expected_error
|
||||
assert event["data"]["task_id"] == task_id
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
@pytest.mark.parametrize("anyio_backend", ["asyncio"])
|
||||
async def test_completion_socket_is_private(anyio_backend):
|
||||
from types import SimpleNamespace
|
||||
|
||||
from channels.layers import get_channel_layer
|
||||
from channels.testing import WebsocketCommunicator
|
||||
|
||||
from paperless.consumers import StatusConsumer
|
||||
|
||||
own = WebsocketCommunicator(StatusConsumer.as_asgi(), "/ws/status/")
|
||||
other = WebsocketCommunicator(StatusConsumer.as_asgi(), "/ws/status/")
|
||||
own.scope["user"] = SimpleNamespace(id=1, is_authenticated=True)
|
||||
other.scope["user"] = SimpleNamespace(id=2, is_authenticated=True)
|
||||
assert (await own.connect())[0]
|
||||
assert (await other.connect())[0]
|
||||
try:
|
||||
event = {
|
||||
"type": "bulk_operation_status",
|
||||
"data": {"task_id": "example", "status": "success"},
|
||||
}
|
||||
await get_channel_layer().group_send("bulk_operations_1", event)
|
||||
assert await own.receive_json_from() == event
|
||||
assert await other.receive_nothing(timeout=0.02)
|
||||
finally:
|
||||
await own.disconnect()
|
||||
await other.disconnect()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize("endpoint", ["delete", "bulk_edit"])
|
||||
def test_trash_remains_synchronous(bulk_setup, endpoint):
|
||||
_, client, doc, _ = bulk_setup
|
||||
data = {"documents": [doc.pk]}
|
||||
if endpoint == "bulk_edit":
|
||||
data.update(method="delete", parameters={})
|
||||
with (
|
||||
mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue,
|
||||
mock.patch("documents.search.get_backend"),
|
||||
):
|
||||
response = client.post(
|
||||
f"/api/documents/{endpoint}/",
|
||||
data,
|
||||
format="json",
|
||||
HTTP_ACCEPT="application/json; version=9",
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.data == {"result": "OK"}
|
||||
assert not Document.objects.filter(pk=doc.pk).exists()
|
||||
assert Document.deleted_objects.filter(pk=doc.pk).exists()
|
||||
enqueue.assert_not_called()
|
||||
@@ -4,7 +4,6 @@ from django.conf import settings
|
||||
from pytest_mock import MockerFixture
|
||||
|
||||
from documents.regex import safe_regex_finditer
|
||||
from documents.regex import safe_regex_fullmatch
|
||||
from documents.regex import safe_regex_match
|
||||
from documents.regex import safe_regex_search
|
||||
from documents.regex import safe_regex_sub
|
||||
@@ -27,7 +26,7 @@ class TestValidateRegexPattern:
|
||||
|
||||
|
||||
class TestSafeRegexSearchAndMatch:
|
||||
"""Tests for safe_regex_search, _match and _fullmatch (same contract)."""
|
||||
"""Tests for safe_regex_search and safe_regex_match (same contract)."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("func", "pattern", "text", "expected_group"),
|
||||
@@ -46,13 +45,6 @@ class TestSafeRegexSearchAndMatch:
|
||||
"123",
|
||||
id="match-match-found",
|
||||
),
|
||||
pytest.param(
|
||||
safe_regex_fullmatch,
|
||||
r"\d+",
|
||||
"123",
|
||||
"123",
|
||||
id="fullmatch-match-found",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_match_found(self, func, pattern, text, expected_group) -> None:
|
||||
@@ -65,12 +57,6 @@ class TestSafeRegexSearchAndMatch:
|
||||
[
|
||||
pytest.param(safe_regex_search, r"\d+", "abcdef", id="search-no-match"),
|
||||
pytest.param(safe_regex_match, r"\d+", "abc123", id="match-no-match"),
|
||||
pytest.param(
|
||||
safe_regex_fullmatch,
|
||||
r"\d+",
|
||||
"123abc",
|
||||
id="fullmatch-no-match",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_no_match(self, func, pattern, text) -> None:
|
||||
@@ -81,7 +67,6 @@ class TestSafeRegexSearchAndMatch:
|
||||
[
|
||||
pytest.param(safe_regex_search, id="search"),
|
||||
pytest.param(safe_regex_match, id="match"),
|
||||
pytest.param(safe_regex_fullmatch, id="fullmatch"),
|
||||
],
|
||||
)
|
||||
def test_invalid_pattern_returns_none(self, func) -> None:
|
||||
@@ -92,7 +77,6 @@ class TestSafeRegexSearchAndMatch:
|
||||
[
|
||||
pytest.param(safe_regex_search, id="search"),
|
||||
pytest.param(safe_regex_match, id="match"),
|
||||
pytest.param(safe_regex_fullmatch, id="fullmatch"),
|
||||
],
|
||||
)
|
||||
def test_flags_respected(self, func) -> None:
|
||||
@@ -103,7 +87,6 @@ class TestSafeRegexSearchAndMatch:
|
||||
[
|
||||
pytest.param(safe_regex_search, "search", id="search"),
|
||||
pytest.param(safe_regex_match, "match", id="match"),
|
||||
pytest.param(safe_regex_fullmatch, "fullmatch", id="fullmatch"),
|
||||
],
|
||||
)
|
||||
def test_timeout_returns_none(
|
||||
|
||||
@@ -13,52 +13,14 @@ from documents.data_models import DocumentMetadataOverrides
|
||||
from documents.data_models import DocumentSource
|
||||
from documents.models import PaperlessTask
|
||||
from documents.signals.handlers import before_task_publish_handler
|
||||
from documents.signals.handlers import run_zone_ocr_extraction
|
||||
from documents.signals.handlers import task_failure_handler
|
||||
from documents.signals.handlers import task_postrun_handler
|
||||
from documents.signals.handlers import task_prerun_handler
|
||||
from documents.signals.handlers import task_revoked_handler
|
||||
from paperless_testing.factories import DocumentFactory
|
||||
from paperless_testing.factories import PaperlessTaskFactory
|
||||
from paperless_testing.factories import UserFactory
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize(
|
||||
"original_file",
|
||||
[None, "/tmp/original.pdf", Path("/tmp/original.pdf")],
|
||||
)
|
||||
def test_consumption_runs_zone_extraction(
|
||||
original_file,
|
||||
mocker: pytest_mock.MockerFixture,
|
||||
):
|
||||
extract = mocker.patch("documents.zone_ocr.run_zone_extraction")
|
||||
document = DocumentFactory()
|
||||
|
||||
run_zone_ocr_extraction(sender=None, document=document, original_file=original_file)
|
||||
|
||||
extract.assert_called_once_with(
|
||||
document,
|
||||
Path(original_file) if original_file else None,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_zone_extraction_failure_does_not_fail_consumption(
|
||||
mocker: pytest_mock.MockerFixture,
|
||||
caplog,
|
||||
):
|
||||
mocker.patch(
|
||||
"documents.zone_ocr.run_zone_extraction",
|
||||
side_effect=RuntimeError("OCR failed"),
|
||||
)
|
||||
document = DocumentFactory()
|
||||
|
||||
run_zone_ocr_extraction(sender=None, document=document)
|
||||
|
||||
assert f"Zone OCR extraction failed for document {document.pk}" in caplog.text
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def consume_input_doc():
|
||||
doc = mock.MagicMock(spec=ConsumableDocument)
|
||||
|
||||
@@ -5,8 +5,11 @@ from unittest import mock
|
||||
|
||||
import pytest
|
||||
from django.conf import settings
|
||||
from django.contrib.sessions.models import Session
|
||||
from django.db import connection
|
||||
from django.test import TestCase
|
||||
from django.test import override_settings
|
||||
from django.test.utils import CaptureQueriesContext
|
||||
from django.utils import timezone
|
||||
|
||||
from documents import tasks
|
||||
@@ -612,3 +615,36 @@ class TestApplyAISuggestionsTask(DirectoriesMixin, TestCase):
|
||||
|
||||
apply_suggestions.assert_not_called()
|
||||
self.assertIn("no longer exists", "".join(cm.output))
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestClearExpiredSessions:
|
||||
def test_clears_only_expired_sessions_in_batches(self) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- More than one batch of expired sessions and one still-valid session
|
||||
WHEN:
|
||||
- The clear_expired_sessions task runs
|
||||
THEN:
|
||||
- Only expired sessions are removed, using multiple deletes
|
||||
"""
|
||||
now = timezone.now()
|
||||
Session.objects.bulk_create(
|
||||
Session(
|
||||
session_key=f"expired-{i}",
|
||||
session_data="",
|
||||
expire_date=now - timedelta(days=1),
|
||||
)
|
||||
for i in range(501)
|
||||
)
|
||||
Session.objects.create(
|
||||
session_key="valid",
|
||||
session_data="",
|
||||
expire_date=now + timedelta(days=1),
|
||||
)
|
||||
|
||||
with CaptureQueriesContext(connection) as queries:
|
||||
tasks.clear_expired_sessions()
|
||||
|
||||
assert set(Session.objects.values_list("session_key", flat=True)) == {"valid"}
|
||||
assert sum(query["sql"].startswith("DELETE FROM") for query in queries) == 2
|
||||
File diff suppressed because it is too large.
Load diff
+38
-419
@@ -3,16 +3,15 @@ import logging
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
import zipfile
|
||||
from collections import Counter
|
||||
from collections import deque
|
||||
from collections.abc import Callable
|
||||
from dataclasses import asdict
|
||||
from datetime import datetime
|
||||
from datetime import timedelta
|
||||
from http import HTTPStatus
|
||||
from inspect import signature
|
||||
from itertools import islice
|
||||
from pathlib import Path
|
||||
from time import mktime
|
||||
@@ -108,6 +107,9 @@ from documents import bulk_edit
|
||||
from documents.bulk_download import ArchiveOnlyStrategy
|
||||
from documents.bulk_download import OriginalAndArchiveStrategy
|
||||
from documents.bulk_download import OriginalsOnlyStrategy
|
||||
from documents.bulk_operations import QUEUED_OPERATIONS
|
||||
from documents.bulk_operations import execute_bulk_operation
|
||||
from documents.bulk_operations import snapshot_field
|
||||
from documents.caching import get_llm_suggestion_cache
|
||||
from documents.caching import get_metadata_cache
|
||||
from documents.caching import get_suggestion_cache
|
||||
@@ -156,8 +158,6 @@ from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentType
|
||||
from documents.models import Note
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import OcrTemplateZone
|
||||
from documents.models import PaperlessTask
|
||||
from documents.models import SavedView
|
||||
from documents.models import ShareLink
|
||||
@@ -177,6 +177,7 @@ from documents.permissions import ViewDocumentsPermissions
|
||||
from documents.permissions import annotate_document_count_for_related_queryset
|
||||
from documents.permissions import get_document_count_filter_for_user
|
||||
from documents.permissions import get_permission_grantees
|
||||
from documents.permissions import has_document_operation_permissions
|
||||
from documents.permissions import has_global_statistics_permission
|
||||
from documents.permissions import has_perms_owner_aware
|
||||
from documents.permissions import has_system_status_permission
|
||||
@@ -186,7 +187,6 @@ from documents.permissions import restrict_queryset_to_visible
|
||||
from documents.permissions import set_permissions_for_objects
|
||||
from documents.permissions import user_is_unrestricted
|
||||
from documents.plugins.date_parsing import get_date_parser
|
||||
from documents.regex import safe_regex_fullmatch
|
||||
from documents.schema import generate_object_with_permissions_schema
|
||||
from documents.search import SearchHit
|
||||
from documents.serialisers import AcknowledgeTasksViewSerializer
|
||||
@@ -207,15 +207,11 @@ from documents.serialisers import EmailSerializer
|
||||
from documents.serialisers import MergeDocumentsAsVersionsSerializer
|
||||
from documents.serialisers import MergeDocumentsSerializer
|
||||
from documents.serialisers import NotesSerializer
|
||||
from documents.serialisers import OcrTemplateSerializer
|
||||
from documents.serialisers import OcrTemplateTestSerializer
|
||||
from documents.serialisers import PostDocumentSerializer
|
||||
from documents.serialisers import RemovePasswordDocumentsSerializer
|
||||
from documents.serialisers import ReprocessDocumentsSerializer
|
||||
from documents.serialisers import RotateDocumentsSerializer
|
||||
from documents.serialisers import RunTaskSerializer
|
||||
from documents.serialisers import RunZoneOcrDocumentsSerializer
|
||||
from documents.serialisers import RunZoneOcrSerializer
|
||||
from documents.serialisers import SavedViewSerializer
|
||||
from documents.serialisers import SearchResultSerializer
|
||||
from documents.serialisers import SerializerWithPerms
|
||||
@@ -247,7 +243,6 @@ from documents.versioning import get_latest_version_for_root
|
||||
from documents.versioning import get_request_version_param
|
||||
from documents.versioning import get_root_document
|
||||
from documents.versioning import latest_version_content_prefetch
|
||||
from documents.versioning import resolve_effective_document
|
||||
from documents.versioning import resolve_requested_version_for_root
|
||||
from documents.versioning import versions_newest_first
|
||||
from paperless import version
|
||||
@@ -2260,92 +2255,6 @@ class DocumentViewSet(
|
||||
raise Http404
|
||||
return version_doc
|
||||
|
||||
@extend_schema(
|
||||
operation_id="documents_run_zone_ocr",
|
||||
parameters=[
|
||||
OpenApiParameter(
|
||||
name="version",
|
||||
type=OpenApiTypes.INT,
|
||||
location=OpenApiParameter.QUERY,
|
||||
description="File version ID. Defaults to the latest version.",
|
||||
),
|
||||
],
|
||||
request=RunZoneOcrSerializer,
|
||||
responses=inline_serializer(
|
||||
name="RunZoneOcrResult",
|
||||
fields={
|
||||
"status": serializers.ChoiceField(choices=["completed", "skipped"]),
|
||||
"reason": serializers.ChoiceField(
|
||||
choices=["no_template"],
|
||||
required=False,
|
||||
),
|
||||
"results": serializers.ListField(child=serializers.DictField()),
|
||||
},
|
||||
),
|
||||
)
|
||||
@action(
|
||||
methods=["post"],
|
||||
detail=True,
|
||||
url_path="run-zone-ocr",
|
||||
permission_classes=[IsAuthenticated],
|
||||
)
|
||||
def run_zone_ocr(self, request, pk=None):
|
||||
"""
|
||||
Run zone-based OCR extraction on this document, with the given template
|
||||
or with every template that matches it.
|
||||
"""
|
||||
serializer = RunZoneOcrSerializer(data=request.data)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
template = serializer.validated_data.get("template")
|
||||
|
||||
document = get_object_or_404(
|
||||
Document.objects.select_related("root_document__owner"),
|
||||
pk=pk,
|
||||
)
|
||||
if not request.user.has_perm(
|
||||
"documents.change_document",
|
||||
) or not has_perms_owner_aware(request.user, "change_document", document):
|
||||
return HttpResponseForbidden("Insufficient permissions")
|
||||
|
||||
if template is not None and not has_perms_owner_aware(
|
||||
request.user,
|
||||
"view_ocrtemplate",
|
||||
template,
|
||||
):
|
||||
return HttpResponseForbidden("Insufficient permissions")
|
||||
|
||||
from documents.zone_ocr import get_templates_for_document
|
||||
from documents.zone_ocr import run_zone_extraction
|
||||
|
||||
resolution = resolve_effective_document(document, request)
|
||||
if resolution.error == VersionResolutionError.INVALID:
|
||||
raise NotFound("Invalid version parameter")
|
||||
file_document = resolution.document
|
||||
if file_document is None:
|
||||
raise Http404
|
||||
|
||||
templates = (
|
||||
[template]
|
||||
if template is not None
|
||||
else get_templates_for_document(file_document)
|
||||
)
|
||||
if not templates:
|
||||
return Response(
|
||||
{"status": "skipped", "reason": "no_template", "results": []},
|
||||
)
|
||||
|
||||
doc_path = file_document.archive_path or file_document.source_path
|
||||
if not doc_path or not Path(doc_path).is_file():
|
||||
return Response(
|
||||
{"error": "Document file not found"},
|
||||
status=status.HTTP_404_NOT_FOUND,
|
||||
)
|
||||
|
||||
results = run_zone_extraction(file_document, templates=templates, reindex=True)
|
||||
return Response(
|
||||
{"status": "completed", "results": [asdict(result) for result in results]},
|
||||
)
|
||||
|
||||
@extend_schema(
|
||||
operation_id="documents_delete_version",
|
||||
parameters=[
|
||||
@@ -3078,98 +2987,26 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
|
||||
"merge_as_versions",
|
||||
}
|
||||
|
||||
def _has_document_permissions(
|
||||
self,
|
||||
*,
|
||||
user: User,
|
||||
documents: list[int],
|
||||
method,
|
||||
parameters: dict[str, Any],
|
||||
) -> bool:
|
||||
if user.is_superuser:
|
||||
return True
|
||||
|
||||
root_docs = {
|
||||
get_root_document(doc)
|
||||
for doc in Document.objects.select_related(
|
||||
"owner",
|
||||
"root_document__owner",
|
||||
).filter(pk__in=documents)
|
||||
}
|
||||
user_is_owner_of_all_documents = all(
|
||||
(doc.owner == user or doc.owner is None) for doc in root_docs
|
||||
def _queue_bulk_operation(self, method, documents, parameters, from_webui):
|
||||
if "update_callback" in parameters:
|
||||
raise ValueError("Unsupported bulk operation parameter: update_callback")
|
||||
signature(method).bind(documents, **parameters)
|
||||
task = execute_bulk_operation.apply_async(
|
||||
kwargs={
|
||||
"operation": method.__name__,
|
||||
"document_ids": documents,
|
||||
"parameters": parameters,
|
||||
"user_id": self.request.user.pk,
|
||||
},
|
||||
headers={
|
||||
"trigger_source": (
|
||||
PaperlessTask.TriggerSource.WEB_UI
|
||||
if from_webui
|
||||
else PaperlessTask.TriggerSource.API_UPLOAD
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
# check global and object permissions for all documents
|
||||
has_perms = (
|
||||
user.has_perm(
|
||||
"documents.change_document",
|
||||
)
|
||||
and not Document.global_objects.filter(pk__in=documents)
|
||||
.exclude(pk__in=permitted_document_ids(user, perm="change_document"))
|
||||
.exists()
|
||||
)
|
||||
|
||||
# check ownership for methods that change original document
|
||||
if (
|
||||
(
|
||||
has_perms
|
||||
and method
|
||||
in [
|
||||
bulk_edit.set_permissions,
|
||||
bulk_edit.delete,
|
||||
bulk_edit.rotate,
|
||||
bulk_edit.delete_pages,
|
||||
bulk_edit.edit_pdf,
|
||||
bulk_edit.merge_as_versions,
|
||||
bulk_edit.remove_password,
|
||||
]
|
||||
)
|
||||
or (
|
||||
method in [bulk_edit.merge, bulk_edit.split]
|
||||
and parameters.get("delete_originals")
|
||||
)
|
||||
or (method == bulk_edit.edit_pdf and parameters.get("update_document"))
|
||||
):
|
||||
has_perms = has_perms and user_is_owner_of_all_documents
|
||||
|
||||
# check global add permissions for methods that create documents
|
||||
if (
|
||||
has_perms
|
||||
and (
|
||||
method in [bulk_edit.split, bulk_edit.merge]
|
||||
or (
|
||||
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
|
||||
and not parameters.get("update_document")
|
||||
)
|
||||
)
|
||||
and not user.has_perm("documents.add_document")
|
||||
):
|
||||
has_perms = False
|
||||
|
||||
# check global delete permissions for methods that delete documents
|
||||
if (
|
||||
has_perms
|
||||
and (
|
||||
method == bulk_edit.delete
|
||||
# Sources stop being documents of their own, and removing one
|
||||
# again afterwards needs delete_document
|
||||
or method == bulk_edit.merge_as_versions
|
||||
or (
|
||||
method in [bulk_edit.merge, bulk_edit.split]
|
||||
and parameters.get("delete_originals")
|
||||
)
|
||||
or (
|
||||
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
|
||||
and parameters.get("delete_original")
|
||||
and not parameters.get("update_document")
|
||||
)
|
||||
)
|
||||
and not user.has_perm("documents.delete_document")
|
||||
):
|
||||
has_perms = False
|
||||
|
||||
return has_perms
|
||||
return Response({"result": "OK", "task_id": task.id})
|
||||
|
||||
def _execute_document_action(
|
||||
self,
|
||||
@@ -3206,7 +3043,7 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
|
||||
else PaperlessTask.TriggerSource.API_UPLOAD
|
||||
)
|
||||
|
||||
if not self._has_document_permissions(
|
||||
if not has_document_operation_permissions(
|
||||
user=user,
|
||||
documents=documents,
|
||||
method=method,
|
||||
@@ -3237,67 +3074,20 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
|
||||
name="BulkEditDocumentsResult",
|
||||
fields={
|
||||
"result": serializers.CharField(),
|
||||
"task_id": serializers.CharField(required=False),
|
||||
},
|
||||
),
|
||||
},
|
||||
),
|
||||
)
|
||||
class BulkEditView(DocumentOperationPermissionMixin):
|
||||
MODIFIED_FIELD_BY_METHOD = {
|
||||
"set_correspondent": "correspondent",
|
||||
"set_document_type": "document_type",
|
||||
"set_storage_path": "storage_path",
|
||||
"add_tag": "tags",
|
||||
"remove_tag": "tags",
|
||||
"modify_tags": "tags",
|
||||
"modify_custom_fields": "custom_fields",
|
||||
"set_permissions": None,
|
||||
LEGACY_MODIFIED_FIELDS = {
|
||||
"delete": "deleted_at",
|
||||
# These operations create new documents/versions no longer altering
|
||||
# fields on the selected document in place
|
||||
"rotate": None,
|
||||
"delete_pages": None,
|
||||
"split": None,
|
||||
"merge": None,
|
||||
"edit_pdf": None,
|
||||
"reprocess": "checksum",
|
||||
"remove_password": None,
|
||||
}
|
||||
|
||||
serializer_class = BulkEditSerializer
|
||||
|
||||
@staticmethod
|
||||
def _snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
|
||||
"""
|
||||
Returns each document's current value of field, for the audit log.
|
||||
|
||||
Tags and custom fields are one row per value, so they are gathered
|
||||
into a sorted list of pks per document (empty when there are none).
|
||||
Reading them through Document.values() instead would join those rows
|
||||
and return one arbitrary value per document.
|
||||
"""
|
||||
if field == "tags":
|
||||
rows = (
|
||||
Document.tags.through.objects.filter(document_id__in=doc_ids)
|
||||
.order_by("tag_id")
|
||||
.values_list("document_id", "tag_id")
|
||||
)
|
||||
elif field == "custom_fields":
|
||||
rows = (
|
||||
CustomFieldInstance.objects.filter(document_id__in=doc_ids)
|
||||
.order_by("pk")
|
||||
.values_list("document_id", "pk")
|
||||
)
|
||||
else:
|
||||
return dict(
|
||||
Document.objects.filter(pk__in=doc_ids).values_list("pk", field),
|
||||
)
|
||||
|
||||
values: dict[int, list[int]] = {doc_id: [] for doc_id in doc_ids}
|
||||
for doc_id, pk in rows:
|
||||
values[doc_id].append(pk)
|
||||
return values
|
||||
|
||||
def post(self, request, *args, **kwargs):
|
||||
request_method = request.data.get("method")
|
||||
api_version = int(request.version or settings.REST_FRAMEWORK["DEFAULT_VERSION"])
|
||||
@@ -3333,7 +3123,7 @@ class BulkEditView(DocumentOperationPermissionMixin):
|
||||
if from_webui
|
||||
else PaperlessTask.TriggerSource.API_UPLOAD
|
||||
)
|
||||
if not self._has_document_permissions(
|
||||
if not has_document_operation_permissions(
|
||||
user=user,
|
||||
documents=documents,
|
||||
method=method,
|
||||
@@ -3342,14 +3132,21 @@ class BulkEditView(DocumentOperationPermissionMixin):
|
||||
return HttpResponseForbidden("Insufficient permissions")
|
||||
|
||||
try:
|
||||
modified_field = self.MODIFIED_FIELD_BY_METHOD.get(method.__name__, None)
|
||||
if method.__name__ in QUEUED_OPERATIONS:
|
||||
return self._queue_bulk_operation(
|
||||
method,
|
||||
documents,
|
||||
parameters,
|
||||
from_webui,
|
||||
)
|
||||
modified_field = self.LEGACY_MODIFIED_FIELDS.get(method.__name__)
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field:
|
||||
old_values = self._snapshot_field(documents, modified_field)
|
||||
old_values = snapshot_field(documents, modified_field)
|
||||
|
||||
result = method(documents, **parameters)
|
||||
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field:
|
||||
new_values = self._snapshot_field(documents, modified_field)
|
||||
new_values = snapshot_field(documents, modified_field)
|
||||
for doc in Document.objects.filter(pk__in=documents):
|
||||
LogEntry.objects.log_create(
|
||||
instance=doc,
|
||||
@@ -3509,40 +3306,6 @@ class ReprocessDocumentsView(DocumentOperationPermissionMixin):
|
||||
)
|
||||
|
||||
|
||||
@extend_schema_view(
|
||||
post=extend_schema(
|
||||
operation_id="documents_run_zone_ocr_bulk",
|
||||
description="Run OCR template zone extraction on selected documents",
|
||||
responses={
|
||||
200: inline_serializer(
|
||||
name="RunZoneOcrDocumentsResult",
|
||||
fields={
|
||||
"result": serializers.CharField(),
|
||||
},
|
||||
),
|
||||
},
|
||||
),
|
||||
)
|
||||
class RunZoneOcrDocumentsView(DocumentOperationPermissionMixin):
|
||||
serializer_class = RunZoneOcrDocumentsSerializer
|
||||
|
||||
def post(self, request, *args, **kwargs):
|
||||
serializer = self.get_serializer(data=request.data)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
template = serializer.validated_data.get("template")
|
||||
if template is not None and not has_perms_owner_aware(
|
||||
request.user,
|
||||
"view_ocrtemplate",
|
||||
template,
|
||||
):
|
||||
return HttpResponseForbidden("Insufficient permissions")
|
||||
return self._execute_document_action(
|
||||
method=bulk_edit.run_zone_ocr,
|
||||
validated_data=serializer.validated_data,
|
||||
operation_label="document zone OCR",
|
||||
)
|
||||
|
||||
|
||||
@extend_schema_view(
|
||||
post=extend_schema(
|
||||
operation_id="documents_edit_pdf",
|
||||
@@ -5840,147 +5603,3 @@ def serve_logo(request: HttpRequest, filename: str | None = None) -> FileRespons
|
||||
filename=logo_name,
|
||||
as_attachment=True,
|
||||
)
|
||||
|
||||
|
||||
class OcrTemplateViewSet(BulkPermissionMixin, PassUserMixin, ModelViewSet[OcrTemplate]):
|
||||
"""CRUD for OCR templates with zone definitions."""
|
||||
|
||||
model = OcrTemplate
|
||||
|
||||
queryset = (
|
||||
OcrTemplate.objects.select_related("owner")
|
||||
.prefetch_related(
|
||||
"zones",
|
||||
"zones__custom_field",
|
||||
)
|
||||
.order_by("name")
|
||||
)
|
||||
serializer_class = OcrTemplateSerializer
|
||||
pagination_class = StandardPagination
|
||||
permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
|
||||
filter_backends = (
|
||||
OrderingFilter,
|
||||
PermittedObjectsFilter,
|
||||
)
|
||||
ordering_fields = ("name",)
|
||||
|
||||
def get_permissions(self):
|
||||
permissions = super().get_permissions()
|
||||
if self.action in {"document_page_image", "test_zone"}:
|
||||
permissions.append(ViewDocumentsPermissions())
|
||||
return permissions
|
||||
|
||||
@action(
|
||||
detail=False,
|
||||
methods=["get"],
|
||||
url_path=r"document-page-image/(?P<doc_id>[0-9]+)/(?P<page>[0-9]+)",
|
||||
)
|
||||
def document_page_image(self, request, doc_id=None, page=None):
|
||||
"""Render a page for the template editor's zone overlay."""
|
||||
document = get_object_or_404(
|
||||
Document.objects.select_related("root_document__owner"),
|
||||
pk=doc_id,
|
||||
)
|
||||
|
||||
if not has_perms_owner_aware(request.user, "view_document", document):
|
||||
return HttpResponseForbidden("Insufficient permissions")
|
||||
|
||||
page_num = int(page)
|
||||
|
||||
if document.page_count and page_num >= document.page_count:
|
||||
raise Http404(
|
||||
f"Page {page_num} out of range (document has {document.page_count} pages)",
|
||||
)
|
||||
|
||||
archive_path = document.archive_path
|
||||
doc_path = archive_path or document.source_path
|
||||
if not doc_path.is_file():
|
||||
raise Http404("Document file not found")
|
||||
|
||||
if archive_path is None and document.mime_type.startswith("image/"):
|
||||
return HttpResponse(doc_path.read_bytes(), content_type=document.mime_type)
|
||||
|
||||
with tempfile.TemporaryDirectory(dir=settings.SCRATCH_DIR) as tmp_dir:
|
||||
output_prefix = Path(tmp_dir) / "page"
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"pdftoppm",
|
||||
"-png",
|
||||
"-r",
|
||||
"150",
|
||||
"-f",
|
||||
str(page_num + 1),
|
||||
"-l",
|
||||
str(page_num + 1),
|
||||
str(doc_path),
|
||||
str(output_prefix),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
)
|
||||
except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as error:
|
||||
logger.warning(
|
||||
"Failed to render page %s of document %s: %s",
|
||||
page_num,
|
||||
document.pk,
|
||||
error,
|
||||
)
|
||||
raise Http404("Failed to render page") from error
|
||||
except FileNotFoundError as error:
|
||||
logger.warning("pdftoppm is not available")
|
||||
raise Http404("Failed to render page") from error
|
||||
|
||||
rendered = sorted(Path(tmp_dir).glob("page-*.png"))
|
||||
if not rendered:
|
||||
raise Http404("No rendered page found")
|
||||
|
||||
content = rendered[0].read_bytes()
|
||||
|
||||
return HttpResponse(content, content_type="image/png")
|
||||
|
||||
@extend_schema(request=OcrTemplateTestSerializer)
|
||||
@action(detail=False, methods=["post"], url_path="test-zone")
|
||||
def test_zone(self, request):
|
||||
"""Extract an ad-hoc zone without changing the document."""
|
||||
from documents.zone_ocr import extract_zone_preview
|
||||
|
||||
serializer = OcrTemplateTestSerializer(data=request.data)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
document = serializer.validated_data["document"]
|
||||
|
||||
if not has_perms_owner_aware(request.user, "view_document", document):
|
||||
return HttpResponseForbidden("Insufficient permissions")
|
||||
|
||||
doc_path = document.archive_path or document.source_path
|
||||
if not doc_path or not Path(doc_path).is_file():
|
||||
return Response(
|
||||
{"error": "Document file not found"},
|
||||
status=status.HTTP_404_NOT_FOUND,
|
||||
)
|
||||
|
||||
zone = OcrTemplateZone(**serializer.validated_data["zone"])
|
||||
|
||||
result = extract_zone_preview(
|
||||
Path(doc_path),
|
||||
zone,
|
||||
zone.zone_source_width or 0,
|
||||
zone.zone_source_height or 0,
|
||||
document.page_count,
|
||||
)
|
||||
|
||||
regex_match = None
|
||||
if zone.validation_regex and result.get("value") is not None:
|
||||
regex_match = (
|
||||
safe_regex_fullmatch(zone.validation_regex, result["value"]) is not None
|
||||
)
|
||||
|
||||
return Response(
|
||||
{
|
||||
"raw_text": result.get("raw_text"),
|
||||
"value": result.get("value"),
|
||||
"regex": zone.validation_regex,
|
||||
"regex_match": regex_match,
|
||||
},
|
||||
)
|
||||
@@ -1,697 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import math
|
||||
import re
|
||||
import string
|
||||
import subprocess
|
||||
import tempfile
|
||||
from dataclasses import dataclass
|
||||
from datetime import date
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from decimal import InvalidOperation
|
||||
from functools import partial
|
||||
from pathlib import Path
|
||||
|
||||
import dateparser
|
||||
import magic
|
||||
import zxingcpp
|
||||
from django.conf import settings
|
||||
from django.core.exceptions import ValidationError
|
||||
from django.db import transaction
|
||||
from django.db.models import F
|
||||
from PIL import Image
|
||||
|
||||
from documents.matching import matches
|
||||
from documents.models import CustomField
|
||||
from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import OcrTemplate
|
||||
from documents.models import OcrTemplateZone
|
||||
from documents.permissions import permitted_object_ids
|
||||
from documents.regex import safe_regex_fullmatch
|
||||
from documents.signals.handlers import add_to_index
|
||||
from documents.versioning import get_root_document
|
||||
|
||||
logger = logging.getLogger("paperless.zone_ocr")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ZoneExtraction:
|
||||
raw_text: str | None = None
|
||||
value: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FieldUpdate:
|
||||
field_name: str
|
||||
value: object
|
||||
custom_field: CustomField | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ZoneOcrResult:
|
||||
template: str
|
||||
zone: str
|
||||
custom_field: str
|
||||
value: object = None
|
||||
reason: str | None = None
|
||||
|
||||
|
||||
def _zone_result(
|
||||
template: OcrTemplate,
|
||||
zone: OcrTemplateZone,
|
||||
*,
|
||||
value: object = None,
|
||||
reason: str | None = None,
|
||||
) -> ZoneOcrResult:
|
||||
if zone.target == OcrTemplateZone.TargetType.CUSTOM_FIELD:
|
||||
label = zone.custom_field.name if zone.custom_field_id else "(no field)"
|
||||
else:
|
||||
label = {"title": "Title", "asn": "ASN", "created": "Created"}[zone.target]
|
||||
return ZoneOcrResult(template.name, zone.name, label, value, reason)
|
||||
|
||||
|
||||
def get_templates_for_document(document: Document) -> list[OcrTemplate]:
|
||||
"""
|
||||
The enabled templates that match the selected file's content and that the
|
||||
root document's owner can view.
|
||||
"""
|
||||
owner = get_root_document(document).owner
|
||||
templates = (
|
||||
OcrTemplate.objects.filter(
|
||||
enabled=True,
|
||||
id__in=permitted_object_ids(owner, OcrTemplate, "view_ocrtemplate"),
|
||||
)
|
||||
.order_by("name", "pk")
|
||||
.prefetch_related("zones", "zones__custom_field")
|
||||
)
|
||||
if document.root_document_id is None and templates:
|
||||
# The original file can be selected even when newer versions exist.
|
||||
document = (
|
||||
Document.objects.filter(pk=document.pk)
|
||||
.annotate(effective_content=F("content"))
|
||||
.first()
|
||||
)
|
||||
if document is None:
|
||||
return []
|
||||
return [template for template in templates if matches(template, document)]
|
||||
|
||||
|
||||
def run_zone_extraction(
|
||||
document: Document,
|
||||
original_file: Path | None = None,
|
||||
templates: list[OcrTemplate] | None = None,
|
||||
*,
|
||||
reindex: bool = False,
|
||||
) -> list[ZoneOcrResult]:
|
||||
"""
|
||||
Extract zones from the selected file version and update its root document.
|
||||
Run all matching templates when no templates are given.
|
||||
|
||||
Manual runs can request a search refresh after successful writes commit.
|
||||
"""
|
||||
if templates is None:
|
||||
templates = get_templates_for_document(document)
|
||||
if not templates:
|
||||
return []
|
||||
|
||||
doc_path = _resolve_doc_path(document, original_file)
|
||||
if doc_path is None:
|
||||
logger.warning(
|
||||
"Zone OCR: no accessible file for document %d",
|
||||
document.pk,
|
||||
)
|
||||
return [
|
||||
_zone_result(template, zone, reason="missing_file")
|
||||
for template in templates
|
||||
for zone in template.zones.all()
|
||||
]
|
||||
|
||||
template_zones = [(template, list(template.zones.all())) for template in templates]
|
||||
pages = {
|
||||
_resolve_page_idx(zone.page, document.page_count)
|
||||
for _, zones in template_zones
|
||||
for zone in zones
|
||||
}
|
||||
if not pages:
|
||||
return []
|
||||
results = []
|
||||
with tempfile.TemporaryDirectory(dir=settings.SCRATCH_DIR) as tmp_dir:
|
||||
tmp_path = Path(tmp_dir)
|
||||
page_images = _render_pages(doc_path, pages, tmp_path, document.page_count)
|
||||
for template, zones in template_zones:
|
||||
try:
|
||||
results.extend(
|
||||
_process_template(document, page_images, template, zones, tmp_path),
|
||||
)
|
||||
except Exception:
|
||||
# One failed template must not prevent the remaining templates from running.
|
||||
logger.exception(
|
||||
"Zone OCR: error processing template '%s' for document %d",
|
||||
template.name,
|
||||
document.pk,
|
||||
)
|
||||
results.extend(
|
||||
_zone_result(template, zone, reason="template_failed")
|
||||
for zone in zones
|
||||
)
|
||||
if reindex and any(result.reason is None for result in results):
|
||||
# queue a reindex of the document after the transaction commits
|
||||
root_document = get_root_document(document)
|
||||
transaction.on_commit(partial(_reindex_document, root_document.pk))
|
||||
return results
|
||||
|
||||
|
||||
def _reindex_document(document_id: int) -> None:
|
||||
document = Document.objects.filter(pk=document_id).first()
|
||||
if document is not None:
|
||||
add_to_index(sender=None, document=document)
|
||||
|
||||
|
||||
def _resolve_doc_path(
|
||||
document: Document,
|
||||
original_file: Path | None,
|
||||
) -> Path | None:
|
||||
"""Prefer the archive, then the source, then the ingestion signal's file."""
|
||||
candidates = []
|
||||
if document.has_archive_version:
|
||||
candidates.append(document.archive_path)
|
||||
candidates.append(document.source_path)
|
||||
if original_file is not None:
|
||||
candidates.append(original_file)
|
||||
|
||||
for path in candidates:
|
||||
if path is not None and Path(path).is_file():
|
||||
return Path(path)
|
||||
return None
|
||||
|
||||
|
||||
def _resolve_page_idx(page_value: int | None, page_count: int | None) -> int:
|
||||
"""Resolve a 1-based page or -1 (last page), defaulting to the first page."""
|
||||
if page_value == -1:
|
||||
return max((page_count or 1) - 1, 0)
|
||||
return max((page_value or 1) - 1, 0)
|
||||
|
||||
|
||||
def _process_template(
|
||||
document: Document,
|
||||
page_images: dict[int, Path],
|
||||
template: OcrTemplate,
|
||||
zones: list[OcrTemplateZone],
|
||||
tmp_path: Path,
|
||||
) -> list[ZoneOcrResult]:
|
||||
"""Extract and combine zone values before applying a template's updates."""
|
||||
grouped: dict[str, list[tuple[OcrTemplateZone, ZoneOcrResult]]] = {}
|
||||
results = []
|
||||
for zone in zones:
|
||||
page_idx = _resolve_page_idx(zone.page, document.page_count)
|
||||
extracted = None
|
||||
reason = "missing_page"
|
||||
if page_idx in page_images:
|
||||
extraction = _extract_zone(
|
||||
page_images[page_idx],
|
||||
zone,
|
||||
zone.zone_source_width or template.source_width,
|
||||
zone.zone_source_height or template.source_height,
|
||||
tmp_path,
|
||||
)
|
||||
extracted = extraction.value
|
||||
reason = None if extracted else "no_value"
|
||||
if (
|
||||
extracted is not None
|
||||
and zone.validation_regex
|
||||
and not safe_regex_fullmatch(zone.validation_regex, extracted)
|
||||
):
|
||||
reason = "regex_mismatch"
|
||||
result = _zone_result(
|
||||
template,
|
||||
zone,
|
||||
value=extracted if reason is None else None,
|
||||
reason=reason,
|
||||
)
|
||||
results.append(result)
|
||||
grouped.setdefault(_field_key(zone), []).append((zone, result))
|
||||
|
||||
updates = []
|
||||
combine_formats = template.combine_formats or {}
|
||||
for key, field_zones in grouped.items():
|
||||
accepted = [result for _, result in field_zones if result.reason is None]
|
||||
if not accepted:
|
||||
continue
|
||||
value = _combine_field_value(
|
||||
combine_formats.get(key, ""),
|
||||
[(zone.name, result.value) for zone, result in field_zones],
|
||||
)
|
||||
target_zone = field_zones[0][0]
|
||||
update = _prepare_field_update(target_zone, value) if value else None
|
||||
if update is not None:
|
||||
updates.append(update)
|
||||
for result in accepted:
|
||||
result.value = update.value if update else None
|
||||
if update is None:
|
||||
result.reason = (
|
||||
"unassigned" if key == "custom_field" else "invalid_value"
|
||||
)
|
||||
_apply_field_updates(get_root_document(document), updates)
|
||||
return results
|
||||
|
||||
|
||||
def _field_key(zone: OcrTemplateZone) -> str:
|
||||
"""Use the same target keys as the editor's combination formats."""
|
||||
target = zone.target
|
||||
if target == OcrTemplateZone.TargetType.CUSTOM_FIELD and zone.custom_field_id:
|
||||
return str(zone.custom_field_id)
|
||||
return target
|
||||
|
||||
|
||||
def _combine_field_value(
|
||||
fmt: str,
|
||||
field_zones: list[tuple[str, str | None]],
|
||||
) -> str:
|
||||
"""Combine the OCR values of all zones targeting one field.
|
||||
|
||||
With a format string, `{Zone Name}` tokens are replaced by that zone's value
|
||||
and literal text is kept; separators left dangling by an empty token are
|
||||
cleaned up. Without a format, the zone values are joined in order by a space.
|
||||
"""
|
||||
values = {name: value or "" for name, value in field_zones}
|
||||
|
||||
if not fmt:
|
||||
return " ".join(value for _, value in field_zones if value).strip()
|
||||
|
||||
def _replace(match: re.Match) -> str:
|
||||
return values.get(match.group(1).strip(), "")
|
||||
|
||||
combined = re.sub(r"\{([^{}]+)\}", _replace, fmt)
|
||||
# Tidy up separators an empty token may have left behind.
|
||||
combined = re.sub(r"\s{2,}", " ", combined)
|
||||
combined = re.sub(r"([^\w\s])\s*\1+", r"\1", combined)
|
||||
return combined.strip().strip("-/.,;:| \t")
|
||||
|
||||
|
||||
def _render_pages(
|
||||
doc_path: Path,
|
||||
pages: set[int],
|
||||
tmp_dir: Path,
|
||||
page_count: int | None,
|
||||
) -> dict[int, Path]:
|
||||
"""Render specific PDF pages as PNG images using pdftoppm (poppler-utils)."""
|
||||
result: dict[int, Path] = {}
|
||||
mime = _detect_mime(doc_path)
|
||||
|
||||
if mime and mime.startswith("image/"):
|
||||
if 0 in pages:
|
||||
result[0] = doc_path
|
||||
return result
|
||||
|
||||
for actual_page in sorted(pages):
|
||||
if actual_page < 0 or (page_count and actual_page >= page_count):
|
||||
continue
|
||||
|
||||
output_prefix = tmp_dir / f"page_{actual_page}"
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"pdftoppm",
|
||||
"-png",
|
||||
"-singlefile",
|
||||
"-r",
|
||||
"300",
|
||||
"-f",
|
||||
str(actual_page + 1), # pdftoppm is 1-indexed
|
||||
"-l",
|
||||
str(actual_page + 1),
|
||||
str(doc_path),
|
||||
str(output_prefix),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
timeout=60,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error("Zone OCR: pdftoppm timed out for page %d", actual_page)
|
||||
continue
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.exception(
|
||||
"Zone OCR: pdftoppm failed for page %d: %s",
|
||||
actual_page,
|
||||
e.stderr.decode(errors="replace") if e.stderr else str(e),
|
||||
)
|
||||
continue
|
||||
except FileNotFoundError:
|
||||
logger.error("Zone OCR: pdftoppm not found")
|
||||
return result
|
||||
|
||||
rendered = output_prefix.with_suffix(".png")
|
||||
if rendered.is_file():
|
||||
result[actual_page] = rendered
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _crop_zone(
|
||||
page_img: Path,
|
||||
zone: OcrTemplateZone,
|
||||
source_width: int,
|
||||
source_height: int,
|
||||
) -> Image.Image | None:
|
||||
"""Crop a zone from the page image and return the PIL Image."""
|
||||
try:
|
||||
with Image.open(page_img) as img:
|
||||
img_width, img_height = img.size
|
||||
if not source_width or not source_height:
|
||||
source_width, source_height = img.size
|
||||
|
||||
scale_x = img_width / source_width
|
||||
scale_y = img_height / source_height
|
||||
|
||||
crop_left = int(zone.x * scale_x)
|
||||
crop_top = int(zone.y * scale_y)
|
||||
crop_right = int((zone.x + zone.width) * scale_x)
|
||||
crop_bottom = int((zone.y + zone.height) * scale_y)
|
||||
|
||||
# Clamp to the image so an oversized zone can't crop out of bounds.
|
||||
crop_left = max(0, min(crop_left, img_width))
|
||||
crop_top = max(0, min(crop_top, img_height))
|
||||
crop_right = min(crop_right, img_width)
|
||||
crop_bottom = min(crop_bottom, img_height)
|
||||
|
||||
if crop_right - crop_left < 2 or crop_bottom - crop_top < 2:
|
||||
logger.warning("Zone OCR: crop too small for zone '%s'", zone.name)
|
||||
return None
|
||||
|
||||
return img.crop((crop_left, crop_top, crop_right, crop_bottom))
|
||||
except (OSError, ValueError, Image.DecompressionBombError):
|
||||
logger.exception("Zone OCR: crop failed for zone '%s'", zone.name)
|
||||
return None
|
||||
|
||||
|
||||
def _read_barcode(cropped: Image.Image, zone_name: str) -> str | None:
|
||||
"""Read QR/barcode from a cropped image using zxingcpp."""
|
||||
try:
|
||||
results = zxingcpp.read_barcodes(cropped)
|
||||
return results[0].text if results else None
|
||||
except (RuntimeError, ValueError, TypeError):
|
||||
logger.exception("Zone OCR: barcode read failed for zone '%s'", zone_name)
|
||||
return None
|
||||
|
||||
|
||||
def _ocr_text(cropped: Image.Image, zone: OcrTemplateZone, tmp_dir: Path) -> str | None:
|
||||
"""OCR a cropped image with Tesseract."""
|
||||
crop_path = tmp_dir / f"zone_{zone.pk}.png"
|
||||
cropped.save(crop_path)
|
||||
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[
|
||||
"tesseract",
|
||||
str(crop_path),
|
||||
"stdout",
|
||||
"-l",
|
||||
zone.ocr_language,
|
||||
"--psm",
|
||||
"6", # Assume uniform block of text
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
check=True,
|
||||
)
|
||||
return proc.stdout.strip() or None
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error("Zone OCR: Tesseract timed out for zone '%s'", zone.name)
|
||||
return None
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.exception(
|
||||
"Zone OCR: Tesseract failed for zone '%s': %s",
|
||||
zone.name,
|
||||
e.stderr[:200] if e.stderr else str(e),
|
||||
)
|
||||
return None
|
||||
except FileNotFoundError:
|
||||
logger.error("Zone OCR: Tesseract not found")
|
||||
return None
|
||||
|
||||
|
||||
def _extract_zone(
|
||||
page_img: Path,
|
||||
zone: OcrTemplateZone,
|
||||
source_width: int,
|
||||
source_height: int,
|
||||
tmp_dir: Path,
|
||||
) -> ZoneExtraction:
|
||||
"""Read and transform a zone, retaining the text for the editor's preview."""
|
||||
cropped = _crop_zone(page_img, zone, source_width, source_height)
|
||||
if cropped is None:
|
||||
return ZoneExtraction()
|
||||
|
||||
try:
|
||||
raw_text = (
|
||||
_read_barcode(cropped, zone.name)
|
||||
if zone.transform == OcrTemplateZone.TransformType.QR_CODE
|
||||
else _ocr_text(cropped, zone, tmp_dir)
|
||||
)
|
||||
finally:
|
||||
cropped.close()
|
||||
return ZoneExtraction(
|
||||
raw_text=raw_text,
|
||||
value=_apply_transform(raw_text, zone.transform, zone.date_format)
|
||||
if raw_text
|
||||
else None,
|
||||
)
|
||||
|
||||
|
||||
def extract_zone_preview(
|
||||
doc_path: Path,
|
||||
zone: OcrTemplateZone,
|
||||
source_width: int,
|
||||
source_height: int,
|
||||
page_count: int | None,
|
||||
) -> dict:
|
||||
"""Extract a zone without writing its value to the document."""
|
||||
page_idx = _resolve_page_idx(zone.page, page_count)
|
||||
with tempfile.TemporaryDirectory(dir=settings.SCRATCH_DIR) as tmp_dir:
|
||||
tmp_path = Path(tmp_dir)
|
||||
page_images = _render_pages(doc_path, {page_idx}, tmp_path, page_count)
|
||||
if page_idx not in page_images:
|
||||
return {"raw_text": None, "value": None}
|
||||
|
||||
result = _extract_zone(
|
||||
page_images[page_idx],
|
||||
zone,
|
||||
source_width,
|
||||
source_height,
|
||||
tmp_path,
|
||||
)
|
||||
return {"raw_text": result.raw_text, "value": result.value}
|
||||
|
||||
|
||||
def _parse_iso_date(text: str) -> date | None:
|
||||
match = re.search(r"\d{4}-\d{2}-\d{2}", text)
|
||||
return date.fromisoformat(match[0]) if match else None
|
||||
|
||||
|
||||
def _parse_date(text: str, fmt: str = "") -> date | None:
|
||||
"""Try the zone's date format, ISO dates, then automatic date detection."""
|
||||
if fmt:
|
||||
try:
|
||||
return datetime.strptime(text, fmt).date()
|
||||
except ValueError:
|
||||
pass
|
||||
try:
|
||||
iso_date = _parse_iso_date(text)
|
||||
except ValueError:
|
||||
iso_date = None
|
||||
if iso_date is not None:
|
||||
return iso_date
|
||||
try:
|
||||
parsed = dateparser.parse(
|
||||
text,
|
||||
settings={
|
||||
"PREFER_DAY_OF_MONTH": "first",
|
||||
"RETURN_AS_TIMEZONE_AWARE": False,
|
||||
},
|
||||
)
|
||||
if parsed:
|
||||
return parsed.date()
|
||||
except (ValueError, OverflowError):
|
||||
logger.debug("Zone OCR: date parsing failed")
|
||||
return None
|
||||
|
||||
|
||||
def _apply_transform(text: str, transform: str, date_format: str = "") -> str:
|
||||
"""Apply post-processing transform to extracted text."""
|
||||
text = text.strip()
|
||||
if not text:
|
||||
return text
|
||||
|
||||
if transform in ("strip", "none"):
|
||||
return text
|
||||
elif transform == "date":
|
||||
parsed = _parse_date(text, date_format)
|
||||
return parsed.isoformat() if parsed else text
|
||||
elif transform == "uppercase":
|
||||
return text.upper()
|
||||
elif transform == "lowercase":
|
||||
return text.lower()
|
||||
elif transform == "numeric":
|
||||
result = re.sub(r"[^\d.,\-]", "", text)
|
||||
return result if result else text
|
||||
elif transform == "strip_punctuation":
|
||||
return text.strip(string.punctuation + " \t\r\n")
|
||||
return text
|
||||
|
||||
|
||||
def _prepare_field_update(
|
||||
zone: OcrTemplateZone,
|
||||
value: str,
|
||||
) -> FieldUpdate | None:
|
||||
"""Convert extracted text and enforce the destination field's constraints."""
|
||||
custom_field = None
|
||||
if zone.target == OcrTemplateZone.TargetType.CUSTOM_FIELD:
|
||||
if not zone.custom_field_id:
|
||||
return None
|
||||
custom_field = zone.custom_field
|
||||
field_name = CustomFieldInstance.get_value_field_name(custom_field.data_type)
|
||||
typed_value = _convert_value(value, custom_field.data_type)
|
||||
model = CustomFieldInstance
|
||||
elif zone.target == OcrTemplateZone.TargetType.TITLE:
|
||||
field_name, typed_value = "title", value[:128]
|
||||
model = Document
|
||||
elif zone.target == OcrTemplateZone.TargetType.ASN:
|
||||
digits = re.sub(r"[^\d]", "", value)
|
||||
field_name = "archive_serial_number"
|
||||
typed_value = int(digits) if digits else None
|
||||
model = Document
|
||||
elif zone.target == OcrTemplateZone.TargetType.CREATED:
|
||||
field_name, typed_value = "created", _parse_date(value)
|
||||
model = Document
|
||||
else:
|
||||
return None
|
||||
if typed_value is None:
|
||||
return None
|
||||
try:
|
||||
typed_value = model._meta.get_field(field_name).clean(typed_value, None)
|
||||
except ValidationError:
|
||||
logger.info("Zone OCR: invalid value for zone '%s'", zone.name)
|
||||
return None
|
||||
return FieldUpdate(field_name, typed_value, custom_field)
|
||||
|
||||
|
||||
def _apply_field_updates(document: Document, updates: list[FieldUpdate]) -> None:
|
||||
builtin_values = {
|
||||
update.field_name: update.value
|
||||
for update in updates
|
||||
if update.custom_field is None
|
||||
}
|
||||
original_values = {name: getattr(document, name) for name in builtin_values}
|
||||
try:
|
||||
with transaction.atomic():
|
||||
for update in updates:
|
||||
if update.custom_field is not None:
|
||||
CustomFieldInstance.objects.update_or_create(
|
||||
document=document,
|
||||
field=update.custom_field,
|
||||
defaults={update.field_name: update.value},
|
||||
)
|
||||
if builtin_values:
|
||||
for name, value in builtin_values.items():
|
||||
setattr(document, name, value)
|
||||
document.save(update_fields=list(builtin_values))
|
||||
except Exception:
|
||||
# Database rollback does not restore attributes on the model instance.
|
||||
for name, value in original_values.items():
|
||||
setattr(document, name, value)
|
||||
raise
|
||||
|
||||
|
||||
def _normalize_number(value: str) -> str:
|
||||
cleaned = re.sub(r"[^\d.,\-]", "", value)
|
||||
# When both separators occur, the rightmost one is the decimal separator.
|
||||
if "," in cleaned and "." in cleaned:
|
||||
if cleaned.rindex(",") > cleaned.rindex("."):
|
||||
cleaned = cleaned.replace(".", "").replace(",", ".")
|
||||
else:
|
||||
cleaned = cleaned.replace(",", "")
|
||||
else:
|
||||
cleaned = cleaned.replace(",", ".")
|
||||
Decimal(cleaned)
|
||||
return cleaned
|
||||
|
||||
|
||||
def _convert_value(value: str, data_type: str) -> object | None:
|
||||
"""Convert an extracted OCR string to the appropriate type for the custom field."""
|
||||
if not value:
|
||||
return None
|
||||
|
||||
try:
|
||||
if data_type in (
|
||||
CustomField.FieldDataType.STRING,
|
||||
CustomField.FieldDataType.URL,
|
||||
):
|
||||
return value[:128]
|
||||
|
||||
elif data_type == CustomField.FieldDataType.LONG_TEXT:
|
||||
return value
|
||||
|
||||
elif data_type == CustomField.FieldDataType.INT:
|
||||
digits = re.sub(r"[^\d\-]", "", value)
|
||||
digits = digits.lstrip("-")
|
||||
if not digits:
|
||||
return None
|
||||
if value.strip().startswith("-"):
|
||||
digits = "-" + digits
|
||||
return int(digits)
|
||||
|
||||
elif data_type == CustomField.FieldDataType.FLOAT:
|
||||
number = float(_normalize_number(value))
|
||||
return number if math.isfinite(number) else None
|
||||
|
||||
elif data_type == CustomField.FieldDataType.DATE:
|
||||
parsed = _parse_iso_date(value)
|
||||
return parsed.isoformat() if parsed else None
|
||||
|
||||
elif data_type == CustomField.FieldDataType.MONETARY:
|
||||
return _normalize_number(value)
|
||||
|
||||
elif data_type == CustomField.FieldDataType.BOOL:
|
||||
lower = value.lower().strip()
|
||||
if lower in ("true", "yes", "1", "ja", "oui", "si", "x"):
|
||||
return True
|
||||
elif lower in ("false", "no", "0", "nein", "non"):
|
||||
return False
|
||||
return None
|
||||
|
||||
else:
|
||||
logger.debug(
|
||||
"Zone OCR: unsupported custom field type %s for OCR extraction",
|
||||
data_type,
|
||||
)
|
||||
return None
|
||||
|
||||
except (ValueError, InvalidOperation):
|
||||
logger.info("Zone OCR: could not convert value to %s", data_type)
|
||||
return None
|
||||
|
||||
|
||||
def _detect_mime(path: Path) -> str | None:
|
||||
"""Detect MIME type of a file."""
|
||||
try:
|
||||
return magic.from_file(str(path), mime=True)
|
||||
except (OSError, magic.MagicException):
|
||||
logger.debug("Zone OCR: magic failed for %s, falling back to extension", path)
|
||||
|
||||
suffix = path.suffix.lower()
|
||||
return {
|
||||
".pdf": "application/pdf",
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".tiff": "image/tiff",
|
||||
".tif": "image/tiff",
|
||||
".webp": "image/webp",
|
||||
".bmp": "image/bmp",
|
||||
".gif": "image/gif",
|
||||
}.get(suffix)
|
||||
@@ -2,7 +2,7 @@ msgid ""
|
||||
msgstr ""
|
||||
"Project-Id-Version: paperless-ngx\n"
|
||||
"Report-Msgid-Bugs-To: \n"
|
||||
"POT-Creation-Date: 2026-11-07 02:45+0000\n"
|
||||
"POT-Creation-Date: 2026-11-11 23:50+0000\n"
|
||||
"PO-Revision-Date: 2022-02-17 04:17\n"
|
||||
"Last-Translator: \n"
|
||||
"Language-Team: English\n"
|
||||
@@ -1661,8 +1661,8 @@ msgid "workflow runs"
|
||||
msgstr ""
|
||||
|
||||
#: documents/serialisers.py:483 documents/serialisers.py:844
|
||||
#: documents/serialisers.py:2850 documents/views.py:349 documents/views.py:615
|
||||
#: documents/views.py:2755 paperless_mail/serialisers.py:156
|
||||
#: documents/serialisers.py:2850 documents/views.py:354 documents/views.py:620
|
||||
#: documents/views.py:2760 paperless_mail/serialisers.py:156
|
||||
msgid "Insufficient permissions."
|
||||
msgstr ""
|
||||
|
||||
@@ -1702,7 +1702,7 @@ msgstr ""
|
||||
msgid "Duplicate document identifiers are not allowed."
|
||||
msgstr ""
|
||||
|
||||
#: documents/serialisers.py:2936 documents/views.py:4805
|
||||
#: documents/serialisers.py:2936 documents/views.py:4698
|
||||
#, python-format
|
||||
msgid "Documents not found: %(ids)s"
|
||||
msgstr ""
|
||||
@@ -1953,40 +1953,40 @@ msgstr ""
|
||||
msgid ", "
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:304
|
||||
#: documents/views.py:309
|
||||
msgid "Specify only one of text, title_search, query, or more_like_id."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:342 documents/views.py:2752
|
||||
#: documents/views.py:347 documents/views.py:2757
|
||||
msgid "Invalid more_like_id"
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:1679
|
||||
#: documents/views.py:1684
|
||||
msgid "Invalid AI configuration."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:1690
|
||||
#: documents/views.py:1695
|
||||
msgid "AI backend request timed out."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:1702
|
||||
#: documents/views.py:1707
|
||||
msgid "AI backend rejected the request. Check logs for details."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:4821
|
||||
#: documents/views.py:4714
|
||||
#, python-format
|
||||
msgid "Insufficient permissions to share document %(id)s."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:4868
|
||||
#: documents/views.py:4761
|
||||
msgid "Bundle is already being processed."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:4934
|
||||
#: documents/views.py:4827
|
||||
msgid "The share link bundle is still being prepared. Please try again later."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:4948
|
||||
#: documents/views.py:4841
|
||||
msgid "The share link bundle is unavailable."
|
||||
msgstr ""
|
||||
|
||||
@@ -2278,151 +2278,151 @@ msgstr ""
|
||||
msgid "paperless application settings"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:570
|
||||
#: paperless/settings/__init__.py:581
|
||||
msgid "English (US)"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:571
|
||||
#: paperless/settings/__init__.py:582
|
||||
msgid "Arabic"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:572
|
||||
#: paperless/settings/__init__.py:583
|
||||
msgid "Afrikaans"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:573
|
||||
#: paperless/settings/__init__.py:584
|
||||
msgid "Belarusian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:574
|
||||
#: paperless/settings/__init__.py:585
|
||||
msgid "Bulgarian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:575
|
||||
#: paperless/settings/__init__.py:586
|
||||
msgid "Catalan"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:576
|
||||
#: paperless/settings/__init__.py:587
|
||||
msgid "Czech"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:577
|
||||
#: paperless/settings/__init__.py:588
|
||||
msgid "Danish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:578
|
||||
#: paperless/settings/__init__.py:589
|
||||
msgid "German"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:579
|
||||
#: paperless/settings/__init__.py:590
|
||||
msgid "Greek"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:580
|
||||
#: paperless/settings/__init__.py:591
|
||||
msgid "English (GB)"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:581
|
||||
#: paperless/settings/__init__.py:592
|
||||
msgid "Spanish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:582
|
||||
#: paperless/settings/__init__.py:593
|
||||
msgid "Persian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:583
|
||||
#: paperless/settings/__init__.py:594
|
||||
msgid "Finnish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:584
|
||||
#: paperless/settings/__init__.py:595
|
||||
msgid "French"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:585
|
||||
#: paperless/settings/__init__.py:596
|
||||
msgid "Hungarian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:586
|
||||
#: paperless/settings/__init__.py:597
|
||||
msgid "Indonesian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:587
|
||||
#: paperless/settings/__init__.py:598
|
||||
msgid "Italian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:588
|
||||
#: paperless/settings/__init__.py:599
|
||||
msgid "Japanese"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:589
|
||||
#: paperless/settings/__init__.py:600
|
||||
msgid "Korean"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:590
|
||||
#: paperless/settings/__init__.py:601
|
||||
msgid "Luxembourgish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:591
|
||||
#: paperless/settings/__init__.py:602
|
||||
msgid "Norwegian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:592
|
||||
#: paperless/settings/__init__.py:603
|
||||
msgid "Dutch"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:593
|
||||
#: paperless/settings/__init__.py:604
|
||||
msgid "Polish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:594
|
||||
#: paperless/settings/__init__.py:605
|
||||
msgid "Portuguese (Brazil)"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:595
|
||||
#: paperless/settings/__init__.py:606
|
||||
msgid "Portuguese"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:596
|
||||
#: paperless/settings/__init__.py:607
|
||||
msgid "Romanian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:597
|
||||
#: paperless/settings/__init__.py:608
|
||||
msgid "Russian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:598
|
||||
#: paperless/settings/__init__.py:609
|
||||
msgid "Slovak"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:599
|
||||
#: paperless/settings/__init__.py:610
|
||||
msgid "Slovenian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:600
|
||||
#: paperless/settings/__init__.py:611
|
||||
msgid "Serbian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:601
|
||||
#: paperless/settings/__init__.py:612
|
||||
msgid "Swedish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:602
|
||||
#: paperless/settings/__init__.py:613
|
||||
msgid "Turkish"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:603
|
||||
#: paperless/settings/__init__.py:614
|
||||
msgid "Ukrainian"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:604
|
||||
#: paperless/settings/__init__.py:615
|
||||
msgid "Vietnamese"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:605
|
||||
#: paperless/settings/__init__.py:616
|
||||
msgid "Chinese Simplified"
|
||||
msgstr ""
|
||||
|
||||
#: paperless/settings/__init__.py:606
|
||||
#: paperless/settings/__init__.py:617
|
||||
msgid "Chinese Traditional"
|
||||
msgstr ""
|
||||
|
||||
|
||||
@@ -12,6 +12,17 @@ from paperless.models import ModeChoices
|
||||
from paperless.models import OutputTypeChoices
|
||||
from paperless.models import RemoteOCRMode
|
||||
|
||||
# OCRmyPDF arguments the parser sets itself, pointing into its own temporary
|
||||
# directory. The parser reads these files back after OCR, so user supplied
|
||||
# values can only break consumption (or write to arbitrary paths), never help.
|
||||
OCR_RESERVED_USER_ARGS: frozenset[str] = frozenset(
|
||||
{
|
||||
"input_file_or_options",
|
||||
"output_file",
|
||||
"sidecar",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class BaseConfig:
|
||||
|
||||
@@ -45,12 +45,21 @@ class StatusConsumer(AsyncWebsocketConsumer):
|
||||
await self.close()
|
||||
return
|
||||
await self.channel_layer.group_add("status_updates", self.channel_name)
|
||||
await self.channel_layer.group_add(
|
||||
f"bulk_operations_{self.scope['user'].id}",
|
||||
self.channel_name,
|
||||
)
|
||||
await self.accept()
|
||||
self._start_heartbeat()
|
||||
|
||||
async def disconnect(self, code: int) -> None:
|
||||
await self._stop_heartbeat()
|
||||
await self.channel_layer.group_discard("status_updates", self.channel_name)
|
||||
if self._authenticated():
|
||||
await self.channel_layer.group_discard(
|
||||
f"bulk_operations_{self.scope['user'].id}",
|
||||
self.channel_name,
|
||||
)
|
||||
|
||||
def _start_heartbeat(self) -> None:
|
||||
self.heartbeat_task = asyncio.create_task(self._heartbeat_loop())
|
||||
@@ -84,3 +93,9 @@ class StatusConsumer(AsyncWebsocketConsumer):
|
||||
await self.close()
|
||||
elif await self._can_view(event["data"]):
|
||||
await self.send(json.dumps(event))
|
||||
|
||||
async def bulk_operation_status(self, event: dict) -> None:
|
||||
if not self._authenticated():
|
||||
await self.close()
|
||||
else:
|
||||
await self.send(json.dumps(event))
|
||||
@@ -17,10 +17,12 @@ from django.conf import settings
|
||||
from PIL import Image
|
||||
|
||||
from documents.parsers import ParseError
|
||||
from documents.parsers import encode_thumbnail_webp
|
||||
from documents.parsers import make_thumbnail_from_pdf
|
||||
from documents.utils import copy_file_with_basic_stats
|
||||
from documents.utils import maybe_override_pixel_limit
|
||||
from documents.utils import remove_alpha_channel
|
||||
from paperless.config import OCR_RESERVED_USER_ARGS
|
||||
from paperless.config import OcrConfig
|
||||
from paperless.models import CleanChoices
|
||||
from paperless.models import ModeChoices
|
||||
@@ -161,6 +163,10 @@ class RasterisedDocumentParser:
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_thumbnail(self, document_path: Path, mime_type: str) -> Path:
|
||||
if self.archive_path is None and self.is_image(mime_type):
|
||||
thumbnail_path = self.tempdir / "convert.webp"
|
||||
encode_thumbnail_webp(Path(document_path), thumbnail_path)
|
||||
return thumbnail_path
|
||||
return make_thumbnail_from_pdf(
|
||||
self.archive_path or Path(document_path),
|
||||
self.tempdir,
|
||||
@@ -351,7 +357,22 @@ class RasterisedDocumentParser:
|
||||
|
||||
if self.settings.user_args is not None:
|
||||
try:
|
||||
ocrmypdf_args = {**ocrmypdf_args, **self.settings.user_args}
|
||||
reserved = sorted(
|
||||
OCR_RESERVED_USER_ARGS & self.settings.user_args.keys(),
|
||||
)
|
||||
if reserved:
|
||||
self.log.warning(
|
||||
f"Ignoring OCR user arguments managed by paperless: "
|
||||
f"{', '.join(reserved)}",
|
||||
)
|
||||
ocrmypdf_args = {
|
||||
**ocrmypdf_args,
|
||||
**{
|
||||
k: v
|
||||
for k, v in self.settings.user_args.items()
|
||||
if k not in OCR_RESERVED_USER_ARGS
|
||||
},
|
||||
}
|
||||
except Exception as e:
|
||||
self.log.warning(
|
||||
f"There is an issue with PAPERLESS_OCR_USER_ARGS, so "
|
||||
|
||||
@@ -19,6 +19,7 @@ from PIL import Image
|
||||
from rest_framework import serializers
|
||||
from rest_framework.authtoken.serializers import AuthTokenSerializer
|
||||
|
||||
from paperless.config import OCR_RESERVED_USER_ARGS
|
||||
from paperless.models import ApplicationConfiguration
|
||||
from paperless.network import validate_outbound_http_url
|
||||
from paperless.validators import reject_dangerous_svg
|
||||
@@ -251,6 +252,13 @@ class ApplicationConfigurationSerializer(
|
||||
|
||||
def validate_user_args(self, value):
|
||||
self._require_json_object("user_args", value)
|
||||
if value is not None:
|
||||
reserved = sorted(OCR_RESERVED_USER_ARGS & value.keys())
|
||||
if reserved:
|
||||
raise serializers.ValidationError(
|
||||
f"user_args may not set arguments managed by paperless: "
|
||||
f"{', '.join(reserved)}.",
|
||||
)
|
||||
return value
|
||||
|
||||
def validate_barcode_tag_mapping(self, value):
|
||||
|
||||
@@ -450,23 +450,34 @@ CORS_EXPOSE_HEADERS = [
|
||||
"Content-Disposition",
|
||||
]
|
||||
|
||||
ALLOWED_HOSTS = get_list_from_env("PAPERLESS_ALLOWED_HOSTS", default=["*"])
|
||||
if ALLOWED_HOSTS != ["*"]:
|
||||
# always allow localhost. Necessary e.g. for healthcheck in docker.
|
||||
ALLOWED_HOSTS.append("localhost")
|
||||
|
||||
def _get_allowed_hosts() -> list[str]:
|
||||
if "PAPERLESS_ALLOWED_HOSTS" in os.environ or os.getenv("PAPERLESS_URL"):
|
||||
allowed_hosts = get_list_from_env("PAPERLESS_ALLOWED_HOSTS")
|
||||
else:
|
||||
allowed_hosts = ["*"]
|
||||
if allowed_hosts != ["*"]:
|
||||
# Always allow localhost for the Docker health check.
|
||||
allowed_hosts.append("localhost")
|
||||
return allowed_hosts
|
||||
|
||||
|
||||
ALLOWED_HOSTS = _get_allowed_hosts()
|
||||
|
||||
|
||||
def _parse_paperless_url():
|
||||
global CSRF_TRUSTED_ORIGINS, CORS_ALLOWED_ORIGINS, ALLOWED_HOSTS
|
||||
url = os.getenv("PAPERLESS_URL")
|
||||
if url:
|
||||
hostname = urlparse(url).hostname
|
||||
if not hostname:
|
||||
hostname = (urlparse(url).hostname or "").removesuffix(".")
|
||||
if not hostname or hostname == "*" or hostname.startswith("."):
|
||||
raise ImproperlyConfigured(
|
||||
f"PAPERLESS_URL {url!r} has no hostname, expected a URL like https://example.com",
|
||||
f"PAPERLESS_URL {url!r} has no valid hostname, expected a URL like https://example.com",
|
||||
)
|
||||
CSRF_TRUSTED_ORIGINS.append(url)
|
||||
CORS_ALLOWED_ORIGINS.append(url)
|
||||
if ":" in hostname and not hostname.startswith("["):
|
||||
hostname = f"[{hostname}]" # Add brackets around IPv6 addresses
|
||||
ALLOWED_HOSTS.append(hostname)
|
||||
|
||||
return url
|
||||
@@ -690,7 +701,7 @@ logging.config.dictConfig(LOGGING)
|
||||
# https://docs.celeryq.dev/en/stable/userguide/configuration.html
|
||||
|
||||
# Tasks outside tasks.py must be loaded even when Django skips checks
|
||||
CELERY_IMPORTS = ("documents.bulk_edit",)
|
||||
CELERY_IMPORTS = ("documents.bulk_edit", "documents.bulk_operations")
|
||||
|
||||
CELERY_BROKER_URL = _CELERY_REDIS_URL
|
||||
CELERY_RESULT_BACKEND = _CELERY_REDIS_URL
|
||||
|
||||
@@ -165,6 +165,17 @@ def parse_beat_schedule() -> dict:
|
||||
"expires": 23.0 * 60.0 * 60.0,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "Clear expired sessions",
|
||||
"env_key": "PAPERLESS_SESSION_CLEANUP_TASK_CRON",
|
||||
# Default daily at 03:00
|
||||
"env_default": "0 3 * * *",
|
||||
"task": "documents.tasks.clear_expired_sessions",
|
||||
"options": {
|
||||
# 1 hour before default schedule sends again
|
||||
"expires": 23.0 * 60.0 * 60.0,
|
||||
},
|
||||
},
|
||||
]
|
||||
for task in tasks:
|
||||
# Either get the environment setting or use the default
|
||||
|
||||
@@ -243,6 +243,34 @@ class TestParserSettingsFromDb(DirectoriesMixin, FileSystemAssertsMixin, TestCas
|
||||
"UseDeviceIndependentColor",
|
||||
)
|
||||
|
||||
def test_ocr_user_args_cannot_override_reserved(self) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- User arguments which attempt to override arguments the parser
|
||||
manages itself, alongside a legitimate argument
|
||||
WHEN:
|
||||
- OCR parameters are constructed
|
||||
THEN:
|
||||
- Reserved arguments keep the parser supplied values
|
||||
- The legitimate argument is still applied
|
||||
"""
|
||||
instance = ApplicationConfiguration.objects.all().first()
|
||||
assert instance is not None
|
||||
instance.user_args = {
|
||||
"sidecar": "/tmp/evil.py",
|
||||
"output_file": "/tmp/out.pdf",
|
||||
"input_file_or_options": "/etc/passwd",
|
||||
"unpaper_args": "--pre-rotate 90",
|
||||
}
|
||||
instance.save()
|
||||
|
||||
params = self.get_params()
|
||||
|
||||
self.assertEqual(params["sidecar"], "sidecar.txt")
|
||||
self.assertEqual(params["output_file"], "output.pdf")
|
||||
self.assertEqual(params["input_file_or_options"], "input.pdf")
|
||||
self.assertEqual(params["unpaper_args"], "--pre-rotate 90")
|
||||
|
||||
def test_ocr_user_args(self) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
|
||||
@@ -18,6 +18,7 @@ from ocrmypdf import SubprocessOutputError
|
||||
from PIL import Image
|
||||
|
||||
from documents.parsers import ParseError
|
||||
from documents.parsers import get_default_thumbnail
|
||||
from documents.parsers import rasterize_pdf_page_to_png
|
||||
from paperless.models import ModeChoices
|
||||
from paperless.parsers import ParserProtocol
|
||||
@@ -270,6 +271,57 @@ class TestDpiHelpers:
|
||||
|
||||
|
||||
class TestGetThumbnail:
|
||||
@pytest.mark.parametrize(
|
||||
("filename", "mime_type"),
|
||||
[
|
||||
pytest.param("simple.jpg", "image/jpeg", id="jpeg"),
|
||||
pytest.param("simple.png", "image/png", id="png"),
|
||||
pytest.param("simple.tif", "image/tiff", id="tiff"),
|
||||
pytest.param("simple.gif", "image/gif", id="gif"),
|
||||
pytest.param("simple.bmp", "image/bmp", id="bmp"),
|
||||
pytest.param("document.webp", "image/webp", id="webp"),
|
||||
pytest.param("simple.heic", "image/heic", id="heic"),
|
||||
],
|
||||
)
|
||||
def test_image_without_archive_has_image_thumbnail(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
tesseract_parser: RasterisedDocumentParser,
|
||||
tesseract_samples_dir: Path,
|
||||
filename: str,
|
||||
mime_type: str,
|
||||
) -> None:
|
||||
original = tesseract_samples_dir / filename
|
||||
render_pdf = mocker.patch("documents.parsers.rasterize_pdf_page_to_png")
|
||||
|
||||
thumb = tesseract_parser.get_thumbnail(original, mime_type)
|
||||
|
||||
render_pdf.assert_not_called()
|
||||
assert thumb.read_bytes() != get_default_thumbnail().read_bytes()
|
||||
with Image.open(thumb) as im:
|
||||
assert im.format == "WEBP"
|
||||
assert 0 < im.width <= 500
|
||||
assert 0 < im.height <= 5000
|
||||
|
||||
def test_image_with_archive_uses_pdf_thumbnail(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
tesseract_parser: RasterisedDocumentParser,
|
||||
tesseract_samples_dir: Path,
|
||||
simple_digital_pdf_file: Path,
|
||||
) -> None:
|
||||
tesseract_parser.archive_path = simple_digital_pdf_file
|
||||
target = "paperless.parsers.tesseract.make_thumbnail_from_pdf"
|
||||
make_thumbnail = mocker.patch(target)
|
||||
original = tesseract_samples_dir / "simple.jpg"
|
||||
|
||||
tesseract_parser.get_thumbnail(original, "image/jpeg")
|
||||
|
||||
make_thumbnail.assert_called_once_with(
|
||||
simple_digital_pdf_file,
|
||||
tesseract_parser.tempdir,
|
||||
)
|
||||
|
||||
def test_thumbnail_is_file(
|
||||
self,
|
||||
tesseract_parser: RasterisedDocumentParser,
|
||||
|
||||
@@ -182,6 +182,7 @@ def make_expected_schedule(
|
||||
workflow_expire = 59.0 * 60.0
|
||||
llm_index_expire = 23.0 * 60.0 * 60.0
|
||||
share_link_cleanup_expire = 23.0 * 60.0 * 60.0
|
||||
session_cleanup_expire = 23.0 * 60.0 * 60.0
|
||||
|
||||
schedule: dict[str, Any] = {
|
||||
"Check all e-mail accounts": {
|
||||
@@ -248,6 +249,14 @@ def make_expected_schedule(
|
||||
"headers": {"trigger_source": "scheduled"},
|
||||
},
|
||||
},
|
||||
"Clear expired sessions": {
|
||||
"task": "documents.tasks.clear_expired_sessions",
|
||||
"schedule": crontab(minute=0, hour="3"),
|
||||
"options": {
|
||||
"expires": session_cleanup_expire,
|
||||
"headers": {"trigger_source": "scheduled"},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
overrides = overrides or {}
|
||||
@@ -298,6 +307,7 @@ class TestParseBeatSchedule:
|
||||
"PAPERLESS_WORKFLOW_SCHEDULED_TASK_CRON": "disable",
|
||||
"PAPERLESS_LLM_INDEX_TASK_CRON": "disable",
|
||||
"PAPERLESS_SHARE_LINK_BUNDLE_CLEANUP_CRON": "disable",
|
||||
"PAPERLESS_SESSION_CLEANUP_TASK_CRON": "disable",
|
||||
},
|
||||
{},
|
||||
id="all-disabled",
|
||||
|
||||
@@ -3,9 +3,14 @@ from unittest import TestCase
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
from django.core.exceptions import DisallowedHost
|
||||
from django.core.exceptions import ImproperlyConfigured
|
||||
from django.http import HttpRequest
|
||||
from django.test import override_settings
|
||||
|
||||
import paperless.settings as paperless_settings
|
||||
from paperless.settings import _get_allauth_trusted_proxy_count
|
||||
from paperless.settings import _get_allowed_hosts
|
||||
from paperless.settings import _get_classifier_language_setting
|
||||
from paperless.settings import _get_search_language_setting
|
||||
from paperless.settings import _parse_paperless_url
|
||||
@@ -168,8 +173,88 @@ class TestPaperlessURLSettings(TestCase):
|
||||
self.assertIn(url, settings.CORS_ALLOWED_ORIGINS)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", ["example.com", "https://", "/just/a/path"])
|
||||
def test_paperless_url_without_hostname_is_rejected(
|
||||
@pytest.mark.parametrize(
|
||||
("url", "allowed_hosts", "expected", "accepted_host", "rejected_host"),
|
||||
[
|
||||
(None, None, ["*"], "other.example.com", None),
|
||||
(None, "", ["localhost"], "localhost:8000", "other.example.com"),
|
||||
(
|
||||
None,
|
||||
"lan.example.com",
|
||||
["lan.example.com", "localhost"],
|
||||
"lan.example.com",
|
||||
"other.example.com",
|
||||
),
|
||||
(None, "*", ["*"], "other.example.com", None),
|
||||
(
|
||||
"https://example.com:8443",
|
||||
None,
|
||||
["localhost", "example.com"],
|
||||
"example.com:8443",
|
||||
"other.example.com",
|
||||
),
|
||||
(
|
||||
"https://example.com.",
|
||||
"",
|
||||
["localhost", "example.com"],
|
||||
"example.com.",
|
||||
"other.example.com",
|
||||
),
|
||||
(
|
||||
"http://[::1]:8000",
|
||||
None,
|
||||
["localhost", "[::1]"],
|
||||
"[::1]:8000",
|
||||
"other.example.com",
|
||||
),
|
||||
("https://example.com", "*", ["*", "example.com"], "other.example.com", None),
|
||||
(
|
||||
"https://example.com",
|
||||
"lan.example.com",
|
||||
["lan.example.com", "localhost", "example.com"],
|
||||
"lan.example.com",
|
||||
"other.example.com",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_allowed_hosts_at_startup(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
url: str | None,
|
||||
allowed_hosts: str | None,
|
||||
expected: list[str],
|
||||
accepted_host: str,
|
||||
rejected_host: str | None,
|
||||
) -> None:
|
||||
for name, value in (
|
||||
("PAPERLESS_URL", url),
|
||||
("PAPERLESS_ALLOWED_HOSTS", allowed_hosts),
|
||||
):
|
||||
if value is None:
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
else:
|
||||
monkeypatch.setenv(name, value)
|
||||
|
||||
monkeypatch.setattr(paperless_settings, "ALLOWED_HOSTS", _get_allowed_hosts())
|
||||
monkeypatch.setattr(paperless_settings, "CSRF_TRUSTED_ORIGINS", [])
|
||||
monkeypatch.setattr(paperless_settings, "CORS_ALLOWED_ORIGINS", [])
|
||||
assert _parse_paperless_url() == url
|
||||
assert expected == paperless_settings.ALLOWED_HOSTS
|
||||
|
||||
request = HttpRequest()
|
||||
with override_settings(ALLOWED_HOSTS=expected):
|
||||
request.META["HTTP_HOST"] = accepted_host
|
||||
assert request.get_host() == accepted_host
|
||||
if rejected_host:
|
||||
request.META["HTTP_HOST"] = rejected_host
|
||||
with pytest.raises(DisallowedHost):
|
||||
request.get_host()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"url",
|
||||
["example.com", "https://", "/just/a/path", "https://*", "https://.example.com"],
|
||||
)
|
||||
def test_paperless_url_without_valid_hostname_is_rejected(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
url: str,
|
||||
) -> None:
|
||||
@@ -179,9 +264,9 @@ def test_paperless_url_without_hostname_is_rejected(
|
||||
WHEN:
|
||||
- The URL is parsed
|
||||
THEN:
|
||||
- A configuration error is raised instead of adding None to ALLOWED_HOSTS
|
||||
- A configuration error is raised for missing or wildcard hostnames
|
||||
"""
|
||||
monkeypatch.setenv("PAPERLESS_URL", url)
|
||||
|
||||
with pytest.raises(ImproperlyConfigured, match="no hostname"):
|
||||
with pytest.raises(ImproperlyConfigured, match="no valid hostname"):
|
||||
_parse_paperless_url()
|
||||
@@ -30,13 +30,11 @@ from documents.views import IndexView
|
||||
from documents.views import LogViewSet
|
||||
from documents.views import MergeDocumentsAsVersionsView
|
||||
from documents.views import MergeDocumentsView
|
||||
from documents.views import OcrTemplateViewSet
|
||||
from documents.views import PostDocumentView
|
||||
from documents.views import RemoteVersionView
|
||||
from documents.views import RemovePasswordDocumentsView
|
||||
from documents.views import ReprocessDocumentsView
|
||||
from documents.views import RotateDocumentsView
|
||||
from documents.views import RunZoneOcrDocumentsView
|
||||
from documents.views import SavedViewViewSet
|
||||
from documents.views import SearchAutoCompleteView
|
||||
from documents.views import SelectionDataView
|
||||
@@ -92,7 +90,6 @@ api_router.register(r"workflow_triggers", WorkflowTriggerViewSet)
|
||||
api_router.register(r"workflow_actions", WorkflowActionViewSet)
|
||||
api_router.register(r"workflows", WorkflowViewSet)
|
||||
api_router.register(r"custom_fields", CustomFieldViewSet)
|
||||
api_router.register(r"ocr_templates", OcrTemplateViewSet)
|
||||
api_router.register(r"config", ApplicationConfigurationViewSet)
|
||||
api_router.register(r"processed_mail", ProcessedMailViewSet)
|
||||
|
||||
@@ -169,11 +166,6 @@ urlpatterns = [
|
||||
ReprocessDocumentsView.as_view(),
|
||||
name="reprocess_documents",
|
||||
),
|
||||
re_path(
|
||||
"^run_zone_ocr/",
|
||||
RunZoneOcrDocumentsView.as_view(),
|
||||
name="run_zone_ocr_documents",
|
||||
),
|
||||
re_path(
|
||||
"^rotate/",
|
||||
RotateDocumentsView.as_view(),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import re
|
||||
from contextlib import suppress
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from django.conf import settings
|
||||
@@ -20,7 +21,17 @@ OCR_LEADER_REGEX = re.compile(r"[._\-\u00b7]{4,}")
|
||||
HORIZONTAL_WHITESPACE_REGEX = re.compile(r"[ \t\u00a0]+")
|
||||
|
||||
|
||||
def get_embedding_model(config: AIConfig) -> "BaseEmbedding":
|
||||
def get_embedding_model(
|
||||
config: AIConfig,
|
||||
*,
|
||||
refresh: bool = False,
|
||||
) -> "BaseEmbedding":
|
||||
"""
|
||||
Build the embedding model for the configured backend.
|
||||
|
||||
For Hugging Face, a cached model loads offline by default, saving seconds per
|
||||
call. Pass ``refresh=True`` to check the Hub for a newer model revision.
|
||||
"""
|
||||
match config.llm_embedding_backend:
|
||||
case LLMEmbeddingBackend.OPENAI_LIKE:
|
||||
from llama_index.embeddings.openai_like import OpenAILikeEmbedding
|
||||
@@ -52,11 +63,15 @@ def get_embedding_model(config: AIConfig) -> "BaseEmbedding":
|
||||
case LLMEmbeddingBackend.HUGGINGFACE:
|
||||
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
|
||||
|
||||
return HuggingFaceEmbedding(
|
||||
model_name=config.llm_embedding_model
|
||||
hf_kwargs = {
|
||||
"model_name": config.llm_embedding_model
|
||||
or "sentence-transformers/all-MiniLM-L6-v2",
|
||||
cache_folder=str(settings.DATA_DIR / "hf_cache"),
|
||||
)
|
||||
"cache_folder": str(settings.DATA_DIR / "hf_cache"),
|
||||
}
|
||||
if not refresh:
|
||||
with suppress(OSError):
|
||||
return HuggingFaceEmbedding(**hf_kwargs, local_files_only=True)
|
||||
return HuggingFaceEmbedding(**hf_kwargs)
|
||||
case LLMEmbeddingBackend.OLLAMA:
|
||||
from llama_index.embeddings.ollama import OllamaEmbedding
|
||||
from ollama import AsyncClient
|
||||
|
||||
@@ -470,7 +470,7 @@ def update_llm_index(
|
||||
logger.warning("No documents found to index.")
|
||||
|
||||
chunk_size = config.llm_embedding_chunk_size
|
||||
embed_model = get_embedding_model(config)
|
||||
embed_model = get_embedding_model(config, refresh=rebuild)
|
||||
|
||||
with write_store(embed_model_name=model_name) as store:
|
||||
if rebuild or not store.table_exists():
|
||||
|
||||
@@ -212,6 +212,10 @@ def test_update_llm_index(
|
||||
|
||||
ai_config.assert_called_once()
|
||||
build_document_node.assert_called_once_with(real_document, chunk_size=512)
|
||||
indexing.get_embedding_model.assert_called_once_with(
|
||||
mock_config,
|
||||
refresh=True,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@@ -336,6 +340,8 @@ def test_update_llm_index_partial_update(
|
||||
|
||||
indexing.update_llm_index(rebuild=False)
|
||||
|
||||
assert indexing.get_embedding_model.call_args.kwargs == {"refresh": False}
|
||||
|
||||
with indexing.get_vector_store() as store:
|
||||
assert store.table_exists(), (
|
||||
"Expected the vector store table to exist after incremental update"
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
from unittest.mock import ANY
|
||||
from unittest.mock import MagicMock
|
||||
from unittest.mock import call
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
@@ -167,10 +168,45 @@ def test_get_embedding_model_huggingface(mock_ai_config):
|
||||
MockHuggingFaceEmbedding.assert_called_once_with(
|
||||
model_name="sentence-transformers/all-MiniLM-L6-v2",
|
||||
cache_folder=str(settings.DATA_DIR / "hf_cache"),
|
||||
local_files_only=True,
|
||||
)
|
||||
assert model == MockHuggingFaceEmbedding.return_value
|
||||
|
||||
|
||||
def test_get_embedding_model_huggingface_downloads_when_not_cached(mock_ai_config):
|
||||
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.HUGGINGFACE
|
||||
mock_ai_config.return_value.llm_embedding_model = "some/model"
|
||||
downloaded = MagicMock()
|
||||
|
||||
with patch(
|
||||
"llama_index.embeddings.huggingface.HuggingFaceEmbedding",
|
||||
side_effect=[OSError("not cached"), downloaded],
|
||||
) as MockHuggingFaceEmbedding:
|
||||
model = get_embedding_model(mock_ai_config.return_value)
|
||||
|
||||
cache_folder = str(settings.DATA_DIR / "hf_cache")
|
||||
assert MockHuggingFaceEmbedding.call_args_list == [
|
||||
call(model_name="some/model", cache_folder=cache_folder, local_files_only=True),
|
||||
call(model_name="some/model", cache_folder=cache_folder),
|
||||
]
|
||||
assert model is downloaded
|
||||
|
||||
|
||||
def test_get_embedding_model_huggingface_refresh_checks_the_hub(mock_ai_config):
|
||||
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.HUGGINGFACE
|
||||
mock_ai_config.return_value.llm_embedding_model = "some/model"
|
||||
|
||||
with patch(
|
||||
"llama_index.embeddings.huggingface.HuggingFaceEmbedding",
|
||||
) as MockHuggingFaceEmbedding:
|
||||
get_embedding_model(mock_ai_config.return_value, refresh=True)
|
||||
|
||||
MockHuggingFaceEmbedding.assert_called_once_with(
|
||||
model_name="some/model",
|
||||
cache_folder=str(settings.DATA_DIR / "hf_cache"),
|
||||
)
|
||||
|
||||
|
||||
def test_get_embedding_model_ollama(mock_ai_config):
|
||||
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.OLLAMA
|
||||
mock_ai_config.return_value.llm_embedding_model = "embeddinggemma"
|
||||
|
||||
@@ -1159,7 +1159,13 @@ class MailAccountHandler(LoggingMixin):
|
||||
new_headers += message.obj._headers
|
||||
message.obj._headers = new_headers
|
||||
|
||||
f.write(message.obj.as_bytes())
|
||||
# RFC 5322 requires CRLF; the default compat32 policy writes bare LF,
|
||||
# which Outlook misreads as literal "=" in quoted-printable soft breaks
|
||||
f.write(
|
||||
message.obj.as_bytes(
|
||||
policy=message.obj.policy.clone(linesep="\r\n"),
|
||||
),
|
||||
)
|
||||
|
||||
correspondent = self._get_correspondent(message, rule)
|
||||
|
||||
|
||||
Reference in new issue
Block a user