Compare commits

...
Author SHA1 Message Date
shamoon 182b0b4712 Merge branch 'dev' into feature/ocr-templates 2026-10-22 15:39:03 -07:00
shamoon f71a9a6eda Merge branch 'main' into dev 2026-10-22 15:38:13 -07:00
github-actions[bot] f319b05e0f Documentation: Add v3.3.4 changelog (#14534) 2026-10-22 15:36:38 -07:00
shamoon 558cb01a1d Bump version to 3.3.4 2026-10-22 14:58:47 -07:00
shamoon 59e037fed2 Chore: bump pillow + aiohttp for security fixes 2026-10-22 14:56:06 -07:00
shamoon 56ae227de3 Chore: Update certifi for eMudhra CA incident (#14533) 2026-10-22 14:51:54 -07:00
Trenton H de61c20197 Chore: Update uv to 0.13 for the eMudhra CA fixes (#14532) 2026-10-22 14:47:05 -07:00
Trenton H c0dbe1f040 Chore: Update uv to 0.13 for the eMudhra CA fixes (#14532) 2026-10-22 14:46:08 -07:00
GitHub Actions 35ba0b36a8 Auto translate strings 2026-10-22 19:30:17 +00:00
Trenton H 7bdbcca051 Chore: Refactor bulk edit to remove PDF knowledge (#14445)
* Refactor: add pure pdf_ops module with real-PDF tests
* Drops tests we don't relly need + adds the typing annotation
* Now that I'm sure it works, re-work these tests to mock pdf_ops, rather than pikepdf
* Validates pages ahead of time in build_pdfs rather than during the building
* Adds the given/when/then and some additional cleanups of pytest style
* This is redundant now
2026-10-22 19:25:39 +00:00
Ben Petty 9b56dc2f61 Fix: folder export with --delete fails on a stale manifest.json.tmp (#14526) 2026-10-22 11:50:31 -07:00
dependabot[bot] d872e456ab Chore(deps): Bump the uv group across 1 directory with 2 updates (#14517)
Bumps the uv group with 2 updates in the / directory: [certifi](https://github.com/certifi/python-certifi) and [pillow](https://github.com/python-pillow/Pillow).


Updates `certifi` from 2026.1.4 to 2026.10.19
- [Commits](https://github.com/certifi/python-certifi/compare/2026.01.04...2026.10.19)

Updates `pillow` from 12.3.0 to 13.0.0
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/12.3.0...13.0.0)

---
updated-dependencies:
- dependency-name: certifi
  dependency-version: 2026.10.19
  dependency-type: indirect
- dependency-name: pillow
  dependency-version: 13.0.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-22 09:15:00 -07:00
GitHub Actions e2b884d353 Auto translate strings 2026-10-22 12:55:09 +00:00
shamoon 97c98ee44c Fix: fix list type for api schema document object filters (#14524) 2026-10-22 05:53:17 -07:00
shamoon c718636f79 Update usage.md 2026-10-20 17:07:09 -07:00
shamoonandOpenAI Codex 54f7f43f35 Reuse OCR page renders and simplify value conversion
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:07:00 -07:00
shamoonandOpenAI Codex 94da3d4c57 Report actual OCR extraction outcomes instead of existing field values
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:02:23 -07:00
shamoonandOpenAI Codex 453e89d6dd Prepare and atomically apply OCR template field updates
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:00:03 -07:00
shamoonandOpenAI Codex 6562415894 Unify OCR zone extraction for preview and saved templates
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 16:58:10 -07:00
shamoon 257c33453d Update test_api_ocr_templates.py 2026-10-20 16:49:39 -07:00
shamoon 87913411ef Update serialisers.py 2026-10-20 16:49:35 -07:00
shamoon cec393dcae Update views.py 2026-10-20 16:49:32 -07:00
shamoon f5526d5700 Improve backend tests 2026-10-20 16:47:48 -07:00
shamoon 2534d3c16b Absurd comments 2026-10-20 16:46:21 -07:00
shamoon 95c8bf2058 Cleanup the serializers and views 2026-10-20 16:45:04 -07:00
shamoon 930d23956b Up frontend coverage 2026-10-20 16:34:22 -07:00
shamoon d2e6834602 Make sort order stable 2026-10-20 16:28:31 -07:00
shamoon a7bf04ddee Update spacing 2026-10-20 16:17:56 -07:00
shamoon 8ab2bf63aa Merge branch 'dev' into feature/ocr-templates 2026-10-20 16:16:31 -07:00
shamoon c63a7e25e7 Update docs 2026-10-20 15:37:08 -07:00
shamoon 9171a1aa6a Changeup the icon 2026-10-20 15:33:01 -07:00
shamoon 55c5d9121f Yes, cute little split button that opens the dialog 2026-10-20 15:31:48 -07:00
shamoon e4efb62667 And bulk edit 2026-10-20 14:02:04 -07:00
shamoon f0597a7c54 doc detail use it this way 2026-10-20 14:00:25 -07:00
shamoon bbdc016a32 Backend for manual tempalte choice 2026-10-20 13:51:54 -07:00
shamoon eae392469f Update the frontend for the matching stuff 2026-10-20 13:41:55 -07:00
shamoon bedefd85f2 Remove the handlers, make the matching work 2026-10-20 13:39:09 -07:00
shamoon bd73fe1e80 Update the serializer 2026-10-20 13:31:08 -07:00
shamoon 8b77a924dc Make it a matchingmodel 2026-10-20 13:29:42 -07:00
shamoon 1bdeb7ebcd Mas sonar 2026-10-20 13:15:17 -07:00
shamoon fa3acfe13f Sure sonar 2026-10-20 13:12:26 -07:00
shamoon 2bc22b14bd Make perms accordion 2026-10-20 13:03:04 -07:00
shamoon dbce8c1409 User perms on frontend 2026-10-20 11:25:48 -07:00
shamoon e976b2ec7a Tests 2026-10-20 11:23:44 -07:00
shamoon 05da0ba5a3 Apply only visible templates 2026-10-20 11:15:31 -07:00
shamoon 4ff2cb9c72 Make them owned, duh
Update 0028_ocrtemplate_ocrtemplatezone.py
2026-10-20 11:15:30 -07:00
shamoon 77a7fb40ec safe regex 2026-10-20 11:05:49 -07:00
shamoon 8100a8e1eb Lol and run zone ocr 2026-10-20 11:04:17 -07:00
shamoon 1e334903f3 And test zone 2026-10-20 11:01:11 -07:00
shamoon a958d89bf4 Add perms to document_page_image 2026-10-20 10:58:54 -07:00
shamoon 8b3295fce0 Docs 2026-10-20 10:42:16 -07:00
shamoon 9783c4995f Fix the tests for now 2026-10-20 10:42:12 -07:00
shamoon e89495dc02 Frontend use the bulk edit 2026-10-20 10:32:48 -07:00
shamoon 8bb2f88015 Backend work for proper bulk zone ocr 2026-10-20 10:20:28 -07:00
shamoon 7d9b8c770d Cleanup doc detail run zone ocr 2026-10-20 10:16:58 -07:00
shamoon 3296fd7cb5 Formatting, remove hard-coded alpha stuff 2026-10-20 10:14:03 -07:00
shamoon bff41907c2 Move back controls, drop (lol) dnd 2026-10-20 10:08:43 -07:00
shamoon 592f882394 Get rid of the x for the picker 2026-10-20 10:06:40 -07:00
shamoon 29ea9cec66 Gosh so much cleanup 2026-10-20 10:03:55 -07:00
shamoon 123df2e826 Pngx conventions 2026-10-20 09:50:29 -07:00
shamoon 639827ec65 Lol, no not deu+eng default 2026-10-20 09:45:30 -07:00
shamoon b3106e408c Lots of enum cleanup 2026-10-20 09:44:37 -07:00
shamoon ea31b6a043 Make it close not leave page 2026-10-20 09:38:29 -07:00
shamoon 0572b54579 Proper zoneless stuff 2026-10-20 09:35:39 -07:00
shamoon 2afec3e0b7 Lots of dead code 2026-10-20 09:31:59 -07:00
shamoon 57019bccbe Cleanup save now that we're modal 2026-10-20 09:30:12 -07:00
shamoon 26b3964a20 Cleanup the geometry stuff 2026-10-20 09:26:42 -07:00
shamoon a2eeadca7f Fix the delete handler 2026-10-20 09:24:26 -07:00
shamoon ba5474ba81 Lol, just one icon 2026-10-20 09:20:06 -07:00
shamoon e3592804bd Embiggen 2026-10-20 09:18:36 -07:00
shamoon 2d54927be8 Fuck it, make it a modal like literally everything else, drop the routes 2026-10-20 09:14:52 -07:00
shamoon 1d2832ac30 Update ocr-template-editor.component.ts 2026-10-20 09:05:24 -07:00
shamoon 3321ba46d6 Automatically update the combine string when changing zone name 2026-10-20 09:00:17 -07:00
shamoon bf316cba8a Dont fail if one bulk fails, collect results separately 2026-10-20 08:57:24 -07:00
shamoon 36c31b3a73 Also prevent editing template while saving 2026-10-20 08:47:26 -07:00
shamoon d10d303980 Warn about losing changes, disable while zone ocr running 2026-10-20 08:36:22 -07:00
shamoon a51e6a71f4 Fix reload after running zone ocr 2026-10-20 08:20:44 -07:00
shamoon ebd8dba3bb Fix the no zones note 2026-10-20 08:20:25 -07:00
shamoon 3ed9586347 Remove the create link from doc details 2026-10-20 08:11:40 -07:00
shamoon 7055f7a745 Fix perms test 2026-10-20 08:07:46 -07:00
shamoon 8f7500ba64 Fix the signal thing 2026-10-20 08:03:33 -07:00
shamoon 816774815c Add a little warning for empty zones 2026-10-20 07:39:47 -07:00
shamoon c5e6748771 Use braces icon instead 2026-10-20 07:18:01 -07:00
shamoon 9b42170554 Make the floating controls movable 2026-10-20 07:15:18 -07:00
shamoon d2ed0c55a4 dirtycheck 2026-10-19 23:12:09 -07:00
shamoon 4d31063082 Default to empty field 2026-10-19 23:07:12 -07:00
shamoon a4fb9bf2b0 Drop the silly mode toggle 2026-10-19 23:04:35 -07:00
shamoon e028257da5 Cleanup the zone combine stuff 2026-10-19 22:55:23 -07:00
shamoon 68eb4fe96d Make the zone list more useful, show colors 2026-10-19 22:43:12 -07:00
shamoon d2d434535b Remove the slop quick create CF stuff 2026-10-19 22:36:17 -07:00
shamoon 1d75c0c99b Use pngx inputs, move test button, generally make less terrible 2026-10-19 22:31:33 -07:00
shamoon 1de1261728 Ok thats a better start, two tabs 2026-10-19 22:23:50 -07:00
shamoon f5f33e0744 make the ui usable 2026-10-19 22:06:11 -07:00
shamoon c9d62afa50 Make it hideable too 2026-10-19 21:43:10 -07:00
shamoon b5a9353780 Move migration 2026-10-19 21:39:26 -07:00
shamoon cf6d16edcd Merge branch 'dev' into feature/ocr-templates 2026-10-19 21:37:23 -07:00
Nathanaël Houn a5c8b82da5 Documentation: add scratch_dir setting (#14498) 2026-10-19 15:00:28 -07:00
shamoon 551445773a Use SVG, get rid of all the manual redraw stuff 2026-09-09 12:43:11 -07:00
shamoon c54cb4a84b Use a real canvas interaction model
[skip ci]
2026-09-09 12:43:11 -07:00
shamoon af61044b63 Fix some permissions stuff, show error on toggle 2026-09-09 12:43:11 -07:00
shamoon 1ef5a6d3cf Get rid of absurd alert 2026-09-09 12:43:11 -07:00
shamoon 4da311f91f Proper data types
[skip ci]
2026-09-09 12:43:10 -07:00
shamoon cbb84649c7 Pull out the zone list into a sub-component 2026-09-09 12:43:10 -07:00
shamoon 8da734a495 Extract the geometry stuff a bit 2026-09-09 12:43:10 -07:00
shamoon 351671011d Remove afterviewinit, fix page change, custom date format per zone 2026-09-09 12:43:09 -07:00
15c95c251d Feature: OCR Templates (#13043)
[skip ci]

Signed-off-by: dependabot[bot] <support@github.com>
Co-Authored-By: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-Authored-By: stumpylog <797416+stumpylog@users.noreply.github.com>
Co-Authored-By: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-Authored-By: shamoon <4887959+shamoon@users.noreply.github.com>
2026-09-09 12:43:09 -07:00
77 changed files with 10157 additions and 571 deletions

No files matched your search

+3 -3
View File
@@ -11,7 +11,7 @@ concurrency:
group: backend-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
DEFAULT_UV_VERSION: "0.12.x"
DEFAULT_UV_VERSION: "0.13.x"
# Match the Docker image: nltk refuses to read hardlinked data files, such as
# the copy bundled with llama-index when uv links packages from its cache
UV_LINK_MODE: copy
@@ -102,7 +102,7 @@ jobs:
with:
python-version: "${{ matrix.python-version }}"
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
@@ -172,7 +172,7 @@ jobs:
with:
python-version: "${{ env.DEFAULT_PYTHON }}"
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
+2 -2
View File
@@ -11,7 +11,7 @@ concurrency:
permissions:
contents: read
env:
DEFAULT_UV_VERSION: "0.12.x"
DEFAULT_UV_VERSION: "0.13.x"
DEFAULT_PYTHON_VERSION: "3.12"
jobs:
changes:
@@ -78,7 +78,7 @@ jobs:
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
+2 -2
View File
@@ -216,9 +216,9 @@ jobs:
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: '0.12.x'
version: '0.13.x'
enable-cache: false
python-version: ${{ steps.setup-python.outputs.python-version }}
- name: Cache frontend dependencies
+3 -3
View File
@@ -8,7 +8,7 @@ concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
env:
DEFAULT_UV_VERSION: "0.12.x"
DEFAULT_UV_VERSION: "0.13.x"
DEFAULT_PYTHON_VERSION: "3.12"
permissions: {}
jobs:
@@ -59,7 +59,7 @@ jobs:
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: false
@@ -213,7 +213,7 @@ jobs:
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: false
+2 -2
View File
@@ -4,7 +4,7 @@ on:
branches:
- dev
env:
DEFAULT_UV_VERSION: "0.12.x"
DEFAULT_UV_VERSION: "0.13.x"
jobs:
generate-translate-strings:
name: Generate Translation Strings
@@ -30,7 +30,7 @@ jobs:
with:
packages: gettext
- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
-1
View File
@@ -38,7 +38,6 @@ src/documents/bulk_edit.py:0: error: Incompatible types in assignment (expressio
src/documents/bulk_edit.py:0: error: Invalid index type "str" for "dict[FieldDataType, str]"; expected type "FieldDataType" [index]
src/documents/bulk_edit.py:0: error: List comprehension has incompatible type List[tuple[int, Any]]; expected List[int] [misc]
src/documents/bulk_edit.py:0: error: List comprehension has incompatible type List[tuple[int, None]]; expected List[int] [misc]
src/documents/bulk_edit.py:0: error: Missing named argument "p" for "remove" of "PageList" [call-arg]
src/documents/bulk_edit.py:0: error: Missing type arguments for generic type "dict" [type-arg]
src/documents/bulk_edit.py:0: error: Missing type arguments for generic type "dict" [type-arg]
src/documents/bulk_edit.py:0: error: Need type annotation for "to_create" (hint: "to_create: list[<type>] = ...") [var-annotated]
-7
View File
@@ -91,13 +91,6 @@
"concise_description": "Argument `list[int]` is not assignable to parameter `args` with type `tuple[Any, ...] | None` in function `celery.app.task.Task.apply_async`",
"severity": "error"
},
{
"column": 33,
"path": "src/documents/bulk_edit.py",
"name": "missing-argument",
"concise_description": "Missing argument `p` in function `pikepdf._core.PageList.remove`",
"severity": "error"
},
{
"column": 25,
"path": "src/documents/caching.py",
+1 -1
View File
@@ -30,7 +30,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs
# Comments:
# - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.12.23-python3.14-trixie-slim AS s6-overlay-base
FROM ghcr.io/astral-sh/uv:0.13.5-python3.14-trixie-slim@sha256:d8e732839dfc4225a1ff698803b478b2223c05636c055d3add41ab9dbec0f315 AS s6-overlay-base
WORKDIR /usr/src/s6
+7
View File
@@ -1,5 +1,12 @@
# Changelog
## paperless-ngx 3.3.4
### All App Changes
- Chore: Update certifi for eMudhra CA incident [@shamoon](https://github.com/shamoon) ([#14533](https://github.com/paperless-ngx/paperless-ngx/pull/14533))
- Chore: Update uv to 0.13 for the eMudhra CA fixes [@stumpylog](https://github.com/stumpylog) ([#14532](https://github.com/paperless-ngx/paperless-ngx/pull/14532))
## paperless-ngx 3.3.3
### Bug Fixes
+9
View File
@@ -370,6 +370,15 @@ instead of being removed from the filesystem. Only the original version is kept.
This setting was previously named PAPERLESS_TRASH_DIR.
#### [`PAPERLESS_SCRATCH_DIR=<path>`](#PAPERLESS_SCRATCH_DIR) {#PAPERLESS_SCRATCH_DIR}
: This folder is used as a temporary workspace for multiple paperless operations, including
consuming documents, editing, and the document exporter.
This must be writeable by the user running paperless.
Defaults to "/tmp/paperless".
#### [`PAPERLESS_MEDIA_ROOT=<path>`](#PAPERLESS_MEDIA_ROOT) {#PAPERLESS_MEDIA_ROOT}
: This is where your documents and thumbnails are stored.
+79 -1
View File
@@ -109,7 +109,7 @@ Think of versions as **file history** for a document.
Paperless-ngx includes management lists for tags, correspondents, document types
and more. These areas allow you to view, add, edit, delete and manage permissions
for these objects. You can also manage saved views, mail accounts, mail rules,
workflows and more from the management sections.
workflows, [OCR templates](#ocr-templates) and more from the management sections.
### Nested Tags
@@ -792,6 +792,84 @@ The following custom field types are supported:
- `Document Link`: reference(s) to other document(s) displayed as links, automatically creates a symmetrical link in reverse
- `Select`: a pre-defined list of strings from which the user can choose
## OCR Templates {#ocr-templates}
OCR templates read text from fixed areas ("zones") of a document and write it to a
[custom field](#custom-fields), or to the document's title, archive serial number or created date. They are
useful for documents that always share the same layout, such as bills from a particular utility. They allow you
to draw a zone around the account number once, and every new bill gets its account number filled in.
Like tags or document types, each template has a [matching algorithm](advanced_usage.md#matching) that
decides which documents it automatically applies to (or not), based on their content.
### Creating a template
1. Open **OCR Templates** under "Manage" and choose **Add Template**.
2. Give the template a name and set up its matching, for example **Any word** with the utility's name. Choose
**None** for a template that should only ever be run manually.
3. Search for a sample document with the layout the template is for. Its pages are shown next to the template
settings.
4. Draw zones on the document. Click a zone to select and edit it.
5. Choose the field each zone fills.
6. Use **Test** to run OCR on the selected zone of the sample document and see the text it finds and the value
that would be written.
Zones are positioned relative to the sample document's page, so templates work best when the documents they
match have an identical layout. Keep the matching specific enough that it only catches documents with that
layout.
### Zone settings
- **Page**: the page the zone is on. Turn on **Last page** for zones that are always on the final page,
regardless of how many pages a document has.
- **OCR language**: the Tesseract languages used to read the zone. Languages other than the defaults must be
installed, see [`PAPERLESS_OCR_LANGUAGES`](configuration.md#PAPERLESS_OCR_LANGUAGES).
- **Transform**: post-processing applied to the text, such as stripping whitespace, keeping only numbers,
parsing a date or reading a QR code or barcode instead of text.
- **Date format**: for the "Parse date" transform, the format of the date using
[Python date codes](https://docs.python.org/3/library/datetime.html#format-codes), e.g. `%d.%m.%Y`.
Leave it empty to detect the format automatically.
- **Validation regex**: a regular expression the whole value must match. Values that do not match are not
written.
Zones can fill custom fields of type text, long text, URL, date, integer, number, monetary or boolean.
Document link and select fields are not supported.
### Combining zones
When several zones fill the same field, a **Combined value** setting appears for that field. Use it to arrange
the zones' values with `{Zone name}` tokens and any text in between, for example `{Start} - {End}`. If it
is left empty, the values are joined with spaces in zone order.
### When templates run
When a document is added, every enabled template that matches its content runs automatically. A template is
only used for a document if the document's owner can view the template, so users' templates apply to their own
documents, and unowned templates apply to everyone's. Templates run in name order, so if two templates fill the
same field, the later one wins.
Turn off **Enabled** to pause a template without deleting it. Disabled templates never run automatically, but
can still be run manually.
### Running templates manually
To run templates on documents that are already in Paperless-ngx, open the **Actions** menu on the document
details page:
- **Run Zone OCR** runs every enabled template that matches the document.
- The button next to it opens a dialog where you can choose one template to run
instead. A template you choose runs even if it does not match the document or is disabled.
The bulk editor's **Run Zone OCR** action opens the same dialog for the selected documents. Bulk runs are
processed in the background.
!!! warning
Zone OCR overwrites the fields it fills. A zone that finds no value leaves its field unchanged.
Managing OCR templates requires the OCR template permissions. Running zone OCR on documents requires
permission to change them, and choosing a template requires permission to view it.
## PDF Actions
Paperless-ngx supports basic editing operations for PDFs (these operations currently cannot be performed on non-PDF files). When viewing an individual document you can
+1
View File
@@ -16,6 +16,7 @@
#PAPERLESS_CONSUMPTION_DIR=../consume
#PAPERLESS_DATA_DIR=../data
#PAPERLESS_EMPTY_TRASH_DIR=
#PAPERLESS_SCRATCH_DIR=/tmp/paperless
#PAPERLESS_MEDIA_ROOT=../media
#PAPERLESS_STATICDIR=../static
#PAPERLESS_FILENAME_FORMAT=
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "paperless-ngx"
version = "3.3.3"
version = "3.3.4"
description = """\
A community-supported supercharged document management system: scan, index and archive all your physical documents\
"""
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "paperless-ngx-ui",
"version": "3.3.3",
"version": "3.3.4",
"scripts": {
"preinstall": "npx only-allow pnpm",
"ng": "ng",
+13
View File
@@ -13,6 +13,7 @@ import { DocumentDetailComponent } from './components/document-detail/document-d
import { DocumentListComponent } from './components/document-list/document-list.component'
import { DocumentAttributesComponent } from './components/manage/document-attributes/document-attributes.component'
import { MailComponent } from './components/manage/mail/mail.component'
import { OcrTemplatesComponent } from './components/manage/ocr-templates/ocr-templates.component'
import { SavedViewsComponent } from './components/manage/saved-views/saved-views.component'
import { ShareLinksComponent } from './components/manage/share-links/share-links.component'
import { WorkflowsComponent } from './components/manage/workflows/workflows.component'
@@ -275,6 +276,18 @@ export const routes: Routes = [
componentName: 'WorkflowsComponent',
},
},
{
path: 'ocr-templates',
component: OcrTemplatesComponent,
canActivate: [PermissionsGuard],
data: {
requiredPermission: {
action: PermissionAction.View,
type: PermissionType.OcrTemplate,
},
componentName: 'OcrTemplatesComponent',
},
},
{
path: 'mail',
component: MailComponent,
@@ -113,6 +113,7 @@ const sidebarItemLabels: Record<HideableSidebarItemID, string> = {
[HideableSidebarItemID.SavedViews]: $localize`Saved Views`,
[HideableSidebarItemID.ShareLinks]: $localize`Share Links`,
[HideableSidebarItemID.Workflows]: $localize`Workflows`,
[HideableSidebarItemID.OcrTemplates]: $localize`OCR Templates`,
[HideableSidebarItemID.Mail]: $localize`Mail`,
[HideableSidebarItemID.Documentation]: $localize`Documentation`,
}
@@ -289,6 +289,17 @@
<pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="Workflows" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.Workflows)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.Workflows, $event)"></pngx-input-switch>
}
</li>
<li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.OcrTemplates) && !settingsService.organizingSidebarItems()"
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.OcrTemplate }">
<a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.OcrTemplates)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="ocr-templates" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="OCR Templates" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="bounding-box-circles"></i-bs><span class="nav-link-label"><ng-container i18n>OCR Templates</ng-container></span>
</a>
@if (settingsService.organizingSidebarItems()) {
<pngx-input-switch class="position-absolute top-50 end-0 translate-middle-y me-1" [class.d-none]="slimSidebarEnabled || slimSidebarAnimating()" [compact]="true" title="OCR Templates" i18n-title [ngModel]="!settingsService.sidebarItemIsHidden(HideableSidebarItemID.OcrTemplates)" (ngModelChange)="toggleSidebarItem(HideableSidebarItemID.OcrTemplates, $event)"></pngx-input-switch>
}
</li>
<li class="nav-item app-link position-relative" [class.d-none]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail) && !settingsService.organizingSidebarItems()" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
tourAnchor="tour.mail">
<a class="nav-link" [class.opacity-50]="settingsService.sidebarItemIsHidden(HideableSidebarItemID.Mail)" [class.pe-5]="settingsService.organizingSidebarItems() && !slimSidebarEnabled && !slimSidebarAnimating()" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
@@ -295,6 +295,7 @@ describe('AppFrameComponent', () => {
HideableSidebarItemID.Dashboard,
HideableSidebarItemID.Workflows,
HideableSidebarItemID.ShareLinks,
HideableSidebarItemID.OcrTemplates,
])
fixture.detectChanges()
@@ -310,6 +311,10 @@ describe('AppFrameComponent', () => {
fixture.nativeElement.querySelector('[routerLink="share-links"]')
.parentElement.classList
).toContain('d-none')
expect(
fixture.nativeElement.querySelector('[routerLink="ocr-templates"]')
.parentElement.classList
).toContain('d-none')
expect(
fixture.nativeElement.querySelector('[routerLink="mail"]').parentElement
.classList
@@ -322,12 +327,21 @@ describe('AppFrameComponent', () => {
])
settingsService.sidebarHiddenItemsEditing.set([
HideableSidebarItemID.Dashboard,
HideableSidebarItemID.OcrTemplates,
])
fixture.detectChanges()
expect(
fixture.nativeElement.querySelectorAll('pngx-input-switch').length
).toBe(6)
).toBe(7)
const ocrTemplatesLink = fixture.nativeElement.querySelector(
'[routerLink="ocr-templates"]'
)
expect(ocrTemplatesLink.parentElement.classList).not.toContain('d-none')
expect(ocrTemplatesLink.classList).toContain('opacity-50')
expect(
ocrTemplatesLink.parentElement.querySelector('pngx-input-switch')
).not.toBeNull()
expect(
fixture.nativeElement.querySelector('[routerLink="dashboard"]')
.parentElement.classList
@@ -0,0 +1,30 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
</div>
<div class="modal-body">
@if (messageBold) {
<p class="text-break"><b>{{messageBold}}</b></p>
}
@if (message) {
<p class="text-break" [innerHTML]="message"></p>
}
<pngx-input-select
[items]="templates()"
[(ngModel)]="template"
[allowNull]="true"
title="OCR template"
i18n-title
placeholder="All matching templates"
i18n-placeholder
></pngx-input-select>
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled()">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled()">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,52 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { of } from 'rxjs'
import { MATCH_NONE } from 'src/app/data/matching-model'
import { OcrTemplate } from 'src/app/data/ocr-template'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { ZoneOcrConfirmDialogComponent } from './zone-ocr-confirm-dialog.component'
const templates: OcrTemplate[] = [
{
id: 1,
name: 'Invoices',
match: '',
matching_algorithm: MATCH_NONE,
sample_document: null,
source_width: 0,
source_height: 0,
enabled: true,
zones: [],
},
]
describe('ZoneOcrConfirmDialogComponent', () => {
let component: ZoneOcrConfirmDialogComponent
let fixture: ComponentFixture<ZoneOcrConfirmDialogComponent>
beforeEach(() => {
TestBed.configureTestingModule({
providers: [
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
imports: [ZoneOcrConfirmDialogComponent],
})
jest
.spyOn(TestBed.inject(OcrTemplateService), 'listAll')
.mockReturnValue(of({ count: 1, all: [1], results: templates }))
fixture = TestBed.createComponent(ZoneOcrConfirmDialogComponent)
component = fixture.componentInstance
fixture.detectChanges()
})
it('should offer templates and default to all matching templates', () => {
expect(component.templates()).toEqual(templates)
expect(component.template).toBeNull()
expect(fixture.nativeElement.textContent).toContain('OCR template')
})
})
@@ -0,0 +1,31 @@
import { Component, OnInit, inject, signal } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { takeUntil } from 'rxjs'
import { OcrTemplate } from 'src/app/data/ocr-template'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { SelectComponent } from '../../input/select/select.component'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-zone-ocr-confirm-dialog',
templateUrl: './zone-ocr-confirm-dialog.component.html',
imports: [FormsModule, SelectComponent],
})
export class ZoneOcrConfirmDialogComponent
extends ConfirmDialogComponent
implements OnInit
{
private readonly ocrTemplateService = inject(OcrTemplateService)
public readonly templates = signal<OcrTemplate[]>([])
/** The template to run, or null for every template that matches. */
template: number | null = null
ngOnInit() {
this.ocrTemplateService
.listAll()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((r) => this.templates.set(r.results))
}
}
@@ -40,7 +40,7 @@
(change)="onChange(selectedDocumentIDs)">
<ng-template ng-label-tmp let-document="item">
<div class="d-flex align-items-center">
@if (!disabled) {
@if (!disabled && multiple) {
<button class="btn p-0 lh-1" (click)="unselect(document)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove link" i18n-title><i-bs name="x"></i-bs></button>
}
@if (document.title) {
@@ -81,6 +81,9 @@ export class DocumentLinkComponent
@Input()
placeholder: string = $localize`Search for documents`
@Input()
multiple: boolean = true
/**
* Parent for ng-select dropdown, needed to prevent close on click.
*/
@@ -61,6 +61,22 @@
<i-bs name="pencil" class="me-2"></i-bs><ng-container i18n>PDF Editor</ng-container>
</button>
<div class="d-flex" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.OcrTemplate }">
<button ngbDropdownItem (click)="runZoneOcr()" [disabled]="!userCanEdit || networkActive()">
<i-bs width="1em" height="1em" name="bounding-box-circles" class="me-2"></i-bs><span i18n>Run Zone OCR</span>
</button>
<button
ngbDropdownItem
class="w-auto flex-shrink-0 px-2"
(click)="runZoneOcrWithOptions()"
[disabled]="!userCanEdit || networkActive()"
title="Choose OCR template"
i18n-title
>
<i-bs width="1em" height="1em" name="sliders2"></i-bs>
</button>
</div>
@if (userIsOwner && (requiresPassword || password)) {
<button ngbDropdownItem (click)="removePassword()" [disabled]="!password">
<i-bs name="unlock" class="me-2"></i-bs><ng-container i18n>Remove Password</ng-container>
@@ -90,7 +106,7 @@
<div class="row">
<div class="col-md-6 col-xl-5 mb-4">
<form [formGroup]='documentForm' (ngSubmit)="save()">
<form [formGroup]='documentForm' (ngSubmit)="save()" [attr.inert]="zoneOcrRunning() ? '' : null" [attr.aria-busy]="zoneOcrRunning()">
<div class="btn-toolbar mb-1 pb-3 gap-2 row-gap-2 border-bottom">
<div class="btn-group">
@@ -1900,6 +1900,196 @@ describe('DocumentDetailComponent', () => {
expect(toastSpy).toHaveBeenCalledWith('Document reloaded.')
})
it('should run zone OCR, report results and reload the document', () => {
component.documentId.set(doc.id)
component.document.set(doc)
jest.spyOn(documentService, 'runZoneOcr').mockReturnValue(
of({
results: [
{ zone: 'Total', value: '102.10' },
{ zone: 'Due', value: '' },
],
} as any)
)
const loadSpy = jest
.spyOn(component as any, 'loadDocument')
.mockImplementation(() => {})
const notifySpy = jest.spyOn(component.docChangeNotifier, 'next')
const toastSpy = jest.spyOn(toastService, 'showInfo')
const modalSpy = jest.spyOn(modalService, 'open')
component.runZoneOcrWithOptions()
const dialog = modalSpy.mock.results[0].value.componentInstance
expect(dialog.message).toBe(
'Fields filled by the template will be overwritten.'
)
dialog.template = 3
dialog.confirmClicked.emit()
expect(documentService.runZoneOcr).toHaveBeenCalledWith(doc.id, 3)
expect(toastSpy).toHaveBeenCalledWith(
'Filled 1 of 2 fields. No value found for: Due'
)
expect(notifySpy).toHaveBeenCalledWith(doc.id)
expect(loadSpy).toHaveBeenCalledWith(doc.id, true, expect.any(Function))
expect(component.zoneOcrRunning()).toBe(true)
loadSpy.mock.calls[0][2]()
expect(component.zoneOcrRunning()).toBe(false)
expect(component.networkActive()).toBe(false)
})
it('should summarize zone OCR results', () => {
component.documentId.set(doc.id)
component.document.set(doc)
jest.spyOn(component as any, 'loadDocument').mockImplementation(() => {})
const runSpy = jest.spyOn(documentService, 'runZoneOcr')
const toastSpy = jest.spyOn(toastService, 'showInfo')
const modalSpy = jest.spyOn(modalService, 'open')
runSpy.mockReturnValueOnce(of({ status: 'completed', results: [] }))
component.runZoneOcr()
expect(modalSpy).not.toHaveBeenCalled()
expect(runSpy).toHaveBeenCalledWith(doc.id, null)
expect(toastSpy).toHaveBeenLastCalledWith(
'Zone OCR found no fields to fill.'
)
component.networkActive.set(false)
runSpy.mockReturnValueOnce(
of({
status: 'completed',
results: [
{ template: 'T', zone: 'Total', custom_field: 'Amount', value: 0 },
],
})
)
component.runZoneOcr()
expect(toastSpy).toHaveBeenLastCalledWith('Filled 1 of 1 fields.')
})
it('should confirm before running zone OCR over unsaved changes', () => {
component.documentId.set(doc.id)
component.document.set({ ...doc, __changedFields: ['title'] })
component.documentForm.patchValue({ title: 'Unsaved title' })
const runSpy = jest.spyOn(documentService, 'runZoneOcr')
const modalSpy = jest.spyOn(modalService, 'open')
component.runZoneOcr()
const modal = modalSpy.mock.results[0].value
expect(modal.componentInstance.message).toBe(
'Running Zone OCR will discard them.'
)
expect(runSpy).not.toHaveBeenCalled()
modal.componentInstance.cancel()
expect(runSpy).not.toHaveBeenCalled()
expect(component.documentForm.get('title').value).toBe('Unsaved title')
expect(component.zoneOcrRunning()).toBe(false)
})
it('should warn about unsaved changes in the zone OCR options dialog', () => {
component.documentId.set(doc.id)
component.document.set({ ...doc, __changedFields: ['title'] })
const runSpy = jest.spyOn(documentService, 'runZoneOcr')
const modalSpy = jest.spyOn(modalService, 'open')
component.runZoneOcrWithOptions()
const dialog = modalSpy.mock.results[0].value.componentInstance
expect(dialog.message).toBe('Running Zone OCR will discard them.')
expect(dialog.btnClass).toBe('btn-warning')
dialog.cancel()
expect(runSpy).not.toHaveBeenCalled()
component.networkActive.set(true)
component.runZoneOcrWithOptions()
expect(modalSpy).toHaveBeenCalledTimes(1)
})
it('should not allow saving or a second zone OCR run while one is running', () => {
component.documentId.set(doc.id)
component.document.set({ ...doc, __changedFields: ['title'] })
const pending = new Subject<any>()
const runSpy = jest
.spyOn(documentService, 'runZoneOcr')
.mockReturnValue(pending)
const modalSpy = jest.spyOn(modalService, 'open')
component.runZoneOcr()
modalSpy.mock.results[0].value.componentInstance.confirmClicked.emit()
expect(runSpy).toHaveBeenCalledWith(doc.id, null)
expect(component.zoneOcrRunning()).toBe(true)
expect(component.networkActive()).toBe(true)
const saveSpy = jest.spyOn(documentService, 'patch')
component.save()
expect(saveSpy).not.toHaveBeenCalled()
component.runZoneOcr()
expect(modalSpy).toHaveBeenCalledTimes(1)
expect(runSpy).toHaveBeenCalledTimes(1)
pending.error(new Error('failed'))
expect(component.zoneOcrRunning()).toBe(false)
expect(component.networkActive()).toBe(false)
expect(component.document().__changedFields).toEqual(['title'])
})
it('should not reload after zone OCR if another document is open', () => {
component.documentId.set(doc.id)
component.document.set({ ...doc, __changedFields: [] })
const pending = new Subject<any>()
jest.spyOn(documentService, 'runZoneOcr').mockReturnValue(pending)
const loadSpy = jest.spyOn(component as any, 'loadDocument')
component.runZoneOcr()
component.documentId.set(99)
pending.next({ results: [] })
expect(loadSpy).not.toHaveBeenCalled()
expect(component.zoneOcrRunning()).toBe(false)
pending.complete()
})
it('should keep unsaved changes when zone OCR is skipped', () => {
component.documentId.set(doc.id)
component.document.set({ ...doc, __changedFields: ['title'] })
component.documentForm.patchValue({ title: 'Unsaved title' })
const modalSpy = jest.spyOn(modalService, 'open')
jest
.spyOn(documentService, 'runZoneOcr')
.mockReturnValue(of({ status: 'skipped', results: [] }))
const loadSpy = jest.spyOn(component as any, 'loadDocument')
const infoSpy = jest.spyOn(toastService, 'showInfo')
component.runZoneOcr()
modalSpy.mock.results[0].value.componentInstance.confirm()
expect(loadSpy).not.toHaveBeenCalled()
expect(component.documentForm.get('title').value).toBe('Unsaved title')
expect(component.zoneOcrRunning()).toBe(false)
expect(component.networkActive()).toBe(false)
expect(infoSpy).toHaveBeenCalledWith(
'No OCR template applies to this document.'
)
})
it('should show an error when zone OCR fails', () => {
component.document.set(doc)
jest
.spyOn(documentService, 'runZoneOcr')
.mockReturnValue(throwError(() => new Error('nope')))
const loadSpy = jest.spyOn(component as any, 'loadDocument')
const errorSpy = jest.spyOn(toastService, 'showError')
component.runZoneOcr()
expect(errorSpy).toHaveBeenCalledWith(
'Error running zone OCR.',
expect.any(Error)
)
expect(loadSpy).not.toHaveBeenCalled()
expect(component.zoneOcrRunning()).toBe(false)
expect(component.networkActive()).toBe(false)
})
it('should change preview element by render type', () => {
initNormally()
component.document.update((document) => ({
@@ -64,6 +64,7 @@ import {
FILTER_STORAGE_PATH,
} from 'src/app/data/filter-rule-type'
import { ObjectWithId } from 'src/app/data/object-with-id'
import { OcrZoneRunResult } from 'src/app/data/ocr-template'
import { StoragePath } from 'src/app/data/storage-path'
import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS, SuggestionSource } from 'src/app/data/ui-settings'
@@ -104,6 +105,7 @@ import { DocumentDetailFieldID } from '../admin/settings/settings.component'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { ZoneOcrConfirmDialogComponent } from '../common/confirm-dialog/zone-ocr-confirm-dialog/zone-ocr-confirm-dialog.component'
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { DocumentTypeEditDialogComponent } from '../common/edit-dialog/document-type-edit-dialog/document-type-edit-dialog.component'
@@ -266,6 +268,7 @@ export class DocumentDetailComponent
readonly error = signal<any>(undefined)
readonly networkActive = signal(false)
readonly zoneOcrRunning = signal(false)
readonly documentId = signal<number>(undefined)
readonly document = signal<Document>(undefined)
@@ -543,7 +546,11 @@ export class DocumentDetailComponent
)
}
private loadDocument(documentId: number, forceRemote: boolean = false): void {
private loadDocument(
documentId: number,
forceRemote: boolean = false,
onComplete?: () => void
): void {
let redirectedToRoot = false
this.closeIncomingUpdateModal()
this.pendingIncomingUpdate = null
@@ -600,7 +607,8 @@ export class DocumentDetailComponent
}),
first(),
takeUntil(this.unsubscribeNotifier),
takeUntil(this.docChangeNotifier)
takeUntil(this.docChangeNotifier),
finalize(() => onComplete?.())
)
.subscribe({
next: (doc) => {
@@ -1334,6 +1342,7 @@ export class DocumentDetailComponent
}
save(close: boolean = false, savedCallback: () => void = null) {
if (this.zoneOcrRunning()) return
this.networkActive.set(true)
;(document.activeElement as HTMLElement)?.dispatchEvent(new Event('change'))
this.documentsService
@@ -1554,6 +1563,107 @@ export class DocumentDetailComponent
})
}
/** Runs every template that matches, confirming only over unsaved changes. */
runZoneOcr() {
if (this.networkActive()) return
const documentId = this.documentId()
if (!this.hasLocalEdits(this.document())) {
this.executeZoneOcr(documentId, null)
return
}
const modal = this.modalService.open(ConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Run Zone OCR`
modal.componentInstance.messageBold = $localize`You have unsaved changes.`
modal.componentInstance.message = $localize`Running Zone OCR will discard them.`
modal.componentInstance.btnClass = 'btn-warning'
modal.componentInstance.btnCaption = $localize`Run Zone OCR`
modal.componentInstance.confirmClicked
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
modal.close()
if (this.documentId() === documentId && !this.networkActive()) {
this.executeZoneOcr(documentId, null)
}
})
}
/** Opens the dialog to pick which template to run. */
runZoneOcrWithOptions() {
if (this.networkActive()) return
const documentId = this.documentId()
const modal = this.modalService.open(ZoneOcrConfirmDialogComponent, {
backdrop: 'static',
})
const dialog = modal.componentInstance as ZoneOcrConfirmDialogComponent
dialog.title = $localize`Run Zone OCR`
dialog.btnCaption = $localize`Run Zone OCR`
if (this.hasLocalEdits(this.document())) {
dialog.messageBold = $localize`You have unsaved changes.`
dialog.message = $localize`Running Zone OCR will discard them.`
dialog.btnClass = 'btn-warning'
} else {
dialog.message = $localize`Fields filled by the template will be overwritten.`
}
dialog.confirmClicked
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
modal.close()
if (this.documentId() === documentId && !this.networkActive()) {
this.executeZoneOcr(documentId, dialog.template)
}
})
}
private executeZoneOcr(documentId: number, templateId: number | null) {
this.zoneOcrRunning.set(true)
this.networkActive.set(true)
this.documentsService
.runZoneOcr(documentId, templateId)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (response) => {
if (
this.documentId() !== documentId ||
response.status === 'skipped'
) {
this.finishZoneOcr()
if (response.status === 'skipped') {
this.toastService.showInfo(
$localize`No OCR template applies to this document.`
)
}
return
}
this.toastService.showInfo(this.zoneOcrSummary(response.results))
this.docChangeNotifier.next(documentId)
this.loadDocument(documentId, true, () => this.finishZoneOcr())
},
error: (error) => {
this.finishZoneOcr()
this.toastService.showError($localize`Error running zone OCR.`, error)
},
})
}
private zoneOcrSummary(results: OcrZoneRunResult[]): string {
if (!results.length) return $localize`Zone OCR found no fields to fill.`
const missed = results
.filter((result) => !`${result.value ?? ''}`.trim())
.map((result) => result.zone)
const filled = results.length - missed.length
return missed.length
? $localize`Filled ${filled} of ${results.length} fields. No value found for: ${missed.join(', ')}`
: $localize`Filled ${filled} of ${results.length} fields.`
}
private finishZoneOcr() {
this.zoneOcrRunning.set(false)
this.networkActive.set(false)
this.flushPendingIncomingUpdate()
}
private getSelectedNonLatestVersionId(): number | null {
const versions = this.document()?.versions ?? []
if (!versions.length || !this.selectedVersionId()) {
@@ -98,6 +98,11 @@
<button ngbDropdownItem (click)="mergeSelectedAsVersions()" [disabled]="!userOwnsAll || !userCanEditAll || !userCanDelete || list.allSelected || list.selectedCount < 2">
<i-bs name="journal-bookmark-fill" class="me-2"></i-bs><ng-container i18n>Merge as versions</ng-container>
</button>
@if (permissionService.currentUserCan(PermissionAction.View, PermissionType.OcrTemplate)) {
<button ngbDropdownItem (click)="runZoneOcrSelected()" [disabled]="!userCanEditAll">
<i-bs name="bounding-box-circles" class="me-2"></i-bs><ng-container i18n>Run Zone OCR</ng-container>
</button>
}
</div>
</div>
</div>
@@ -1286,6 +1286,53 @@ describe('BulkEditorComponent', () => {
) // listAllFilteredIds
})
it('should support run zone ocr', () => {
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest
.spyOn(documentListViewService, 'documents', 'get')
.mockReturnValue([{ id: 3 }, { id: 4 }])
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([3, 4]))
jest
.spyOn(permissionsService, 'currentUserHasObjectPermissions')
.mockReturnValue(true)
fixture.detectChanges()
component.runZoneOcrSelected()
expect(modal).not.toBeUndefined()
modal.componentInstance.confirm()
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/run_zone_ocr/`
)
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
})
component.runZoneOcrSelected()
modal.componentInstance.template = 3
modal.componentInstance.confirm()
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/run_zone_ocr/`
)
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
template: 3,
})
httpTestingController.match(
`${environment.apiBaseUrl}ocr_templates/?page=1&page_size=100000`
) // template list
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
})
it('should support rotate', () => {
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
@@ -64,6 +64,7 @@ import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dial
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { ZoneOcrConfirmDialogComponent } from '../../common/confirm-dialog/zone-ocr-confirm-dialog/zone-ocr-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
import { DocumentTypeEditDialogComponent } from '../../common/edit-dialog/document-type-edit-dialog/document-type-edit-dialog.component'
@@ -966,6 +967,30 @@ export class BulkEditorComponent
})
}
runZoneOcrSelected() {
const modal = this.modalService.open(ZoneOcrConfirmDialogComponent, {
backdrop: 'static',
})
const dialog = modal.componentInstance as ZoneOcrConfirmDialogComponent
dialog.title = $localize`Run zone OCR confirm`
dialog.messageBold = $localize`This operation will run zone OCR on ${this.getSelectionSize()} selected document(s).`
dialog.message = $localize`Fields filled by the template will be overwritten.`
dialog.btnClass = 'btn-warning'
dialog.btnCaption = $localize`Proceed`
dialog.confirmClicked
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
dialog.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
this.documentService.runZoneOcrDocuments(
this.getSelectionQuery(),
dialog.template
)
)
})
}
setPermissions() {
let modal = this.modalService.open(PermissionsDialogComponent, {
backdrop: 'static',
@@ -0,0 +1,36 @@
@if (zones.length === 0) {
<p class="text-muted" i18n>No zones yet. Drag anywhere on the sample document to create one.</p>
} @else {
<div class="list-group">
@for (zone of zones; track $index; let i = $index) {
<div class="list-group-item list-group-item-action d-flex align-items-center gap-3 position-relative">
<span class="p-2 rounded flex-shrink-0" [style.background-color]="zoneColor(i)" aria-hidden="true"></span>
<div class="flex-grow-1 text-truncate">
<button type="button" class="btn btn-link p-0 fw-semibold text-reset text-decoration-none text-truncate mw-100 stretched-link" (click)="zoneSelected.emit(i)">{{ zone.name }}</button>
@if (!hasField(zone)) {
<div class="small text-warning text-truncate">
<i-bs name="exclamation-triangle" class="me-1" aria-hidden="true"></i-bs><span i18n>No field selected</span>
</div>
} @else {
<div class="small text-muted text-truncate">
{{ getZoneTargetName(zone) }}
@if (isCombined(zone)) {
<i-bs name="braces" class="ms-1" title="Combined with other zones" i18n-title></i-bs>
}
</div>
}
</div>
<small class="text-muted text-nowrap">
@if (zone.page === -1) {
<ng-container i18n>Last page</ng-container>
} @else {
<ng-container i18n>Page {{ zone.page ?? 1 }}</ng-container>
}
</small>
<button type="button" class="btn btn-sm btn-link text-danger position-relative z-2" (click)="zoneRemoved.emit(i)" title="Delete zone" i18n-title aria-label="Delete zone" i18n-aria-label>
<i-bs name="trash"></i-bs>
</button>
</div>
}
</div>
}
@@ -0,0 +1,118 @@
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { CustomField } from 'src/app/data/custom-field'
import {
OcrTemplateZone,
OcrZoneTarget,
OcrZoneTransform,
} from 'src/app/data/ocr-template'
import { OcrTemplateEditorZoneListComponent } from './ocr-template-editor-zone-list.component'
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
return {
name: 'Zone 1',
target: OcrZoneTarget.CustomField,
custom_field: 7,
x: 10,
y: 20,
width: 30,
height: 40,
page: 1,
ocr_language: 'eng',
transform: OcrZoneTransform.Strip,
validation_regex: '',
order: 0,
...overrides,
}
}
describe('OcrTemplateEditorZoneListComponent', () => {
let fixture: ComponentFixture<OcrTemplateEditorZoneListComponent>
let component: OcrTemplateEditorZoneListComponent
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
OcrTemplateEditorZoneListComponent,
NgxBootstrapIconsModule.pick(allIcons),
],
}).compileComponents()
fixture = TestBed.createComponent(OcrTemplateEditorZoneListComponent)
component = fixture.componentInstance
})
it('shows empty state when no zones are defined', () => {
fixture.detectChanges()
expect(fixture.nativeElement.textContent).toContain('No zones yet')
})
it('renders zone name, target, and page', () => {
component.zones = [zone(), zone({ name: 'Zone 2', page: -1 })]
component.customFields = [{ id: 7, name: 'Invoice Number' } as CustomField]
fixture.detectChanges()
const text = fixture.nativeElement.textContent
expect(text).toContain('Zone 1')
expect(text).toContain('Invoice Number')
expect(text).toContain('Page 1')
expect(text).toContain('Last page')
})
it('marks zones that share a field', () => {
component.zones = [
zone(),
zone({ name: 'Zone 2' }),
zone({ name: 'Zone 3', custom_field: 8 }),
]
fixture.detectChanges()
expect(component.isCombined(component.zones[0])).toBe(true)
expect(component.isCombined(component.zones[2])).toBe(false)
expect(
fixture.nativeElement.querySelectorAll(
'[title="Combined with other zones"]'
)
).toHaveLength(2)
})
it('warns only for zones without a field and clears the warning when assigned', () => {
component.zones = [
zone({ custom_field: null }),
zone({ target: OcrZoneTarget.Title, custom_field: null }),
zone(),
]
fixture.detectChanges()
const warnings = () =>
fixture.nativeElement.querySelectorAll('.text-warning')
expect(warnings()).toHaveLength(1)
expect(warnings()[0].textContent).toContain('No field selected')
component.zones[0].custom_field = 7
fixture.componentRef.setInput('zones', [...component.zones])
fixture.detectChanges()
expect(warnings()).toHaveLength(0)
})
it('emits select and remove events', () => {
component.zones = [zone()]
const selectSpy = jest.spyOn(component.zoneSelected, 'emit')
const removeSpy = jest.spyOn(component.zoneRemoved, 'emit')
fixture.detectChanges()
const buttons = fixture.nativeElement.querySelectorAll('button')
buttons[0].click()
buttons[1].click()
expect(selectSpy).toHaveBeenCalledWith(0)
expect(removeSpy).toHaveBeenCalledWith(0)
})
it('does not label an unknown built-in target', () => {
expect(
component.getZoneTargetName(zone({ target: 'unknown' as OcrZoneTarget }))
).toBe('')
})
})
@@ -0,0 +1,50 @@
import { Component, EventEmitter, Input, Output } from '@angular/core'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { CustomField } from 'src/app/data/custom-field'
import {
getZoneFieldKey,
OCR_BUILTIN_TARGETS,
OcrTemplateZone,
OcrZoneTarget,
} from 'src/app/data/ocr-template'
import { getZoneColor } from '../zone-geometry'
@Component({
selector: 'pngx-ocr-template-zone-list',
imports: [NgxBootstrapIconsModule],
templateUrl: './ocr-template-editor-zone-list.component.html',
})
export class OcrTemplateEditorZoneListComponent {
@Input() zones: OcrTemplateZone[] = []
@Input() customFields: CustomField[] = []
@Output() public readonly zoneSelected = new EventEmitter<number>()
@Output() public readonly zoneRemoved = new EventEmitter<number>()
public readonly zoneColor = getZoneColor
hasField(zone: OcrTemplateZone): boolean {
return getZoneFieldKey(zone) !== null
}
isCombined(zone: OcrTemplateZone): boolean {
const key = getZoneFieldKey(zone)
return (
key !== null &&
this.zones.filter((z) => getZoneFieldKey(z) === key).length > 1
)
}
getZoneTargetName(zone: OcrTemplateZone): string {
if (zone.target === OcrZoneTarget.CustomField) {
return (
this.customFields.find((field) => field.id === zone.custom_field)
?.name ?? ''
)
}
return (
OCR_BUILTIN_TARGETS.find((target) => target.id === zone.target)?.name ??
''
)
}
}
@@ -0,0 +1,313 @@
<div class="modal-header">
<h4 class="modal-title" id="ocr-template-editor-title">{{ dialogTitle }}</h4>
@if (template.id) {
<span class="badge bg-primary text-primary-text-contrast ms-2">ID: {{ template.id }}</span>
}
<button type="button" class="btn-close" (click)="activeModal.dismiss()" [disabled]="saving()" aria-label="Close" i18n-aria-label></button>
</div>
<div class="modal-body d-flex flex-column">
<div class="row gx-3 editor-layout flex-grow-1 flex-lg-nowrap" [attr.inert]="saving() ? '' : null" [attr.aria-busy]="saving()">
<div class="col-lg-4 mb-3 mb-lg-0 overflow-auto">
<ul ngbNav #nav="ngbNav" [(activeId)]="activeTab" class="nav-underline flex-nowrap flex-md-wrap overflow-auto">
<li ngbNavItem="template">
<a ngbNavLink i18n>Template</a>
<ng-template ngbNavContent>
<div class="row mb-3">
<div class="col-9">
<pngx-input-text [(ngModel)]="template.name" title="Template name" i18n-title></pngx-input-text>
</div>
<div class="col-3">
<pngx-input-switch [(ngModel)]="template.enabled" title="Enabled" i18n-title></pngx-input-switch>
</div>
</div>
<pngx-input-select [(ngModel)]="template.matching_algorithm" [items]="matchingAlgorithms" title="Matching algorithm" i18n-title></pngx-input-select>
@if (patternRequired) {
<pngx-input-text [(ngModel)]="template.match" title="Matching pattern" i18n-title></pngx-input-text>
<pngx-input-check [(ngModel)]="template.is_insensitive" title="Case insensitive" i18n-title></pngx-input-check>
}
<div *pngxIfOwner="object">
<hr />
<pngx-permissions-form [users]="users()" [accordion]="true" [(ngModel)]="permissionsForm"></pngx-permissions-form>
</div>
</ng-template>
</li>
<li ngbNavItem="zones">
<a ngbNavLink><ng-container i18n>Zones</ng-container> <span class="badge bg-primary ms-2">{{ template.zones.length }}</span></a>
<ng-template ngbNavContent>
@if (selectedZone; as zone) {
<div class="d-flex align-items-center gap-2 mb-3">
<button type="button" class="btn btn-sm btn-link px-0 me-auto" (click)="deselectZone()">
<i-bs name="arrow-left" class="me-1"></i-bs><ng-container i18n>All zones</ng-container>
</button>
<button type="button" class="btn btn-sm btn-outline-primary" (click)="testZone()" [disabled]="!previewDocId || zoneTesting">
@if (zoneTesting) {
<span class="spinner-border spinner-border-sm me-1" aria-hidden="true"></span>
} @else {
<i-bs name="play-fill" class="me-1"></i-bs>
}
<ng-container i18n>Test</ng-container>
</button>
<button type="button" class="btn btn-sm btn-outline-danger" (click)="removeZone(selectedZoneIndex)" title="Delete zone" i18n-title aria-label="Delete zone" i18n-aria-label>
<i-bs name="trash"></i-bs>
</button>
</div>
@if (zoneTestResult; as result) {
@if (result.error) {
<div class="alert alert-warning py-2 mb-3 small" role="alert">{{ result.error }}</div>
} @else {
<dl class="row small border rounded mx-0 py-2 mb-3">
<dt class="col-4" i18n>OCR text</dt>
<dd class="col-8">
@if (result.raw_text) {
<code>{{ result.raw_text }}</code>
} @else {
<span class="text-muted" i18n>Nothing detected</span>
}
</dd>
<dt class="col-4" i18n>Value</dt>
<dd class="col-8" [class.mb-0]="!result.regex">
@if (result.value) {
<code>{{ result.value }}</code>
} @else {
<span class="text-muted" i18n>Empty</span>
}
</dd>
@if (result.regex) {
<dt class="col-4" i18n>Validation</dt>
<dd class="col-8 mb-0">
@if (result.regex_match) {
<span class="badge bg-success" i18n>Regex matches</span>
} @else {
<span class="badge bg-danger" i18n>Regex does not match</span>
}
</dd>
}
</dl>
}
}
<pngx-input-text [ngModel]="zone.name" (ngModelChange)="renameZone(zone, $event)" title="Name" i18n-title></pngx-input-text>
<pngx-input-select
[items]="fieldItems()"
[ngModel]="zoneFieldValue(zone)"
(ngModelChange)="setZoneField(zone, $event)"
(createNew)="createField($event)"
title="Field"
i18n-title
placeholder="Choose a field"
i18n-placeholder
hint="Where the extracted value is written."
i18n-hint
></pngx-input-select>
@if (isFieldShared(zone)) {
<pngx-input-text
[ngModel]="getCombineFormat(zone)"
(ngModelChange)="setCombineFormat(zone, $event)"
[placeholder]="defaultCombineFormat(zone)"
title="Combined value"
i18n-title
hint="Several zones write to this field. Set the format or leave empty to join them with spaces."
i18n-hint
></pngx-input-text>
<div class="d-flex flex-wrap gap-1 mt-n2 mb-3">
@for (z of zonesForField(zone); track z) {
<button type="button" class="btn btn-sm btn-outline-secondary font-monospace" (click)="insertCombineToken(zone, z)">{{ '{' + z.name + '}' }}</button>
}
</div>
}
<div class="row align-items-end">
<div class="col">
@if (!isLastPageZone(zone)) {
<pngx-input-number [(ngModel)]="zone.page" [showAdd]="false" title="Page" i18n-title></pngx-input-number>
}
</div>
<div class="col-auto">
<pngx-input-switch [ngModel]="isLastPageZone(zone)" (ngModelChange)="setLastPageZone(zone, $event)" [horizontal]="true" title="Last page" i18n-title></pngx-input-switch>
</div>
</div>
<h6 class="border-top pt-3 mt-1 mb-3" i18n>Processing</h6>
<pngx-input-select
[items]="languageOptions"
[multiple]="true"
[ngModel]="ocrLanguageArray(zone)"
(ngModelChange)="setOcrLanguages(zone, $event)"
title="OCR language"
i18n-title
placeholder="Select languages"
i18n-placeholder
></pngx-input-select>
<pngx-input-select [items]="transformOptions" [(ngModel)]="zone.transform" title="Transform" i18n-title></pngx-input-select>
@if (zone.transform === OcrZoneTransform.Date) {
<pngx-input-text
[(ngModel)]="zone.date_format"
title="Date format"
i18n-title
placeholder="Auto-detect"
i18n-placeholder
hint="Python date codes, e.g. %d.%m.%Y. Leave empty to detect the format automatically."
i18n-hint
></pngx-input-text>
}
<pngx-input-text [(ngModel)]="zone.validation_regex" title="Validation regex" i18n-title placeholder="\d{2}\.\d{2}\.\d{4}" hint="The value must match this pattern to be applied." i18n-hint></pngx-input-text>
} @else {
<pngx-ocr-template-zone-list
[zones]="template.zones"
[customFields]="customFields()"
(zoneSelected)="selectZone($event)"
(zoneRemoved)="removeZone($event)"
></pngx-ocr-template-zone-list>
}
</ng-template>
</li>
</ul>
<div [ngbNavOutlet]="nav" class="mt-3"></div>
</div>
<div class="col-lg-8 d-flex flex-column">
<div class="d-flex flex-wrap gap-2 mb-2 flex-shrink-0">
<pngx-input-document-link
class="flex-grow-1"
[ngModel]="sampleDocumentIds"
(ngModelChange)="setSampleDocument($event)"
[minimal]="true"
[multiple]="false"
placeholder="Search for a sample document"
i18n-placeholder
></pngx-input-document-link>
@if (previewDocId) {
<div class="input-group w-auto flex-nowrap">
<input class="form-control flex-grow-0 preview-page-input" type="number" min="1" [max]="previewPageCount()" [(ngModel)]="previewPageDisplay" [disabled]="!pageLoaded()" aria-label="Page" i18n-aria-label />
<span class="input-group-text" i18n>of {{ previewPageCount() ?? '?' }}</span>
</div>
<fieldset class="input-group w-auto flex-nowrap" aria-label="Zoom" i18n-aria-label>
<button type="button" class="btn btn-outline-secondary" aria-label="Zoom out" i18n-aria-label [disabled]="!pageLoaded() || zoom <= 0.5" (click)="zoomOut()"><i-bs name="dash"></i-bs></button>
<button type="button" class="btn btn-outline-secondary preview-zoom" title="Reset zoom" i18n-title [disabled]="!pageLoaded()" (click)="resetZoom()">{{ zoom | percent: '1.0-0' }}</button>
<button type="button" class="btn btn-outline-secondary" aria-label="Zoom in" i18n-aria-label [disabled]="!pageLoaded() || zoom >= 4" (click)="zoomIn()"><i-bs name="plus"></i-bs></button>
</fieldset>
}
</div>
<div class="flex-grow-1 overflow-hidden">
<div class="h-100 overflow-auto border" [attr.aria-busy]="previewDocId && !pageLoaded()">
@if (pageFailed()) {
<div class="py-5 px-3 text-center" role="alert">
<i-bs name="exclamation-triangle" width="32" height="32" class="text-warning"></i-bs>
<p class="my-3" i18n>Could not load this page.</p>
<button type="button" class="btn btn-sm btn-outline-secondary" (click)="setSampleDocument(sampleDocumentIds)"><i-bs name="arrow-clockwise" class="me-1"></i-bs><span i18n>Retry</span></button>
</div>
} @else if (previewDocId) {
<div class="position-relative" [class.d-none]="!pageLoaded()" [style.width.%]="zoom * 100">
<img
#pageImage
[src]="pageImageUrl"
(load)="onImageLoad()"
(error)="onImageError()"
alt="Sample document page"
i18n-alt
class="d-block w-100"
crossorigin="use-credentials"
/>
@if (pageLoaded()) {
<svg
#zoneOverlay
class="zone-overlay position-absolute top-0 start-0 w-100 h-100"
[attr.viewBox]="overlayViewBox()"
preserveAspectRatio="none"
[style.cursor]="overlayCursor"
(pointerdown)="onOverlayPointerDown($event)"
(pointermove)="onOverlayPointerMove($event)"
(pointerup)="onOverlayPointerUp($event)"
(pointercancel)="onOverlayPointerCancel($event)"
(lostpointercapture)="onOverlayPointerCancel($event)"
aria-label="Zone overlay" i18n-aria-label
>
@for (zone of template.zones; track $index; let i = $index) {
@if (zoneDisplayRect(i); as rect) {
<g>
<rect
class="zone-rect"
[class.zone-rect-selected]="selectedZoneIndex === i"
[attr.x]="rect.x"
[attr.y]="rect.y"
[attr.width]="rect.w"
[attr.height]="rect.h"
[attr.stroke]="zoneColor(i)"
[attr.fill]="zoneColor(i)"
></rect>
<text
class="zone-label fw-semibold pe-none"
[attr.x]="rect.x + overlayUnitSize(6)"
[attr.y]="zoneLabelY(rect)"
[attr.font-size]="overlayFontSize()"
[attr.fill]="zoneColor(i)"
>{{ zoneLabel(zone, i) }}</text>
@if (selectedZoneIndex === i) {
@for (handle of resizeHandles(rect); track handle.handle) {
<rect
class="zone-resize-handle"
[attr.x]="handle.x - overlayHandleSize() / 2"
[attr.y]="handle.y - overlayHandleSize() / 2"
[attr.width]="overlayHandleSize()"
[attr.height]="overlayHandleSize()"
[attr.fill]="zoneColor(i)"
></rect>
}
}
</g>
}
}
@if (drawingRect(); as rect) {
<rect
class="zone-drawing-rect"
[attr.x]="rect.x"
[attr.y]="rect.y"
[attr.width]="rect.w"
[attr.height]="rect.h"
></rect>
}
</svg>
}
</div>
@if (!pageLoaded()) {
<div class="p-5 d-flex justify-content-center align-items-center">
<output class="spinner-border">
<span class="visually-hidden" i18n>Loading page...</span>
</output>
</div>
}
} @else {
<div class="py-5 px-3 text-center text-muted">
<i-bs name="file-text" width="32" height="32"></i-bs>
<p class="mt-3 mb-0" i18n>No sample document selected.</p>
</div>
}
</div>
</div>
</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-outline-secondary" (click)="activeModal.dismiss()" [disabled]="saving()" i18n>Cancel</button>
<button type="button" class="btn btn-primary" (click)="save()" [disabled]="saving() || !isDirty">
@if (saving()) {
<span class="spinner-border spinner-border-sm me-1" aria-hidden="true"></span>
}
<span i18n>Save</span>
</button>
</div>
@@ -0,0 +1,64 @@
:host {
display: flex;
flex-direction: column;
min-height: 0;
}
@media (min-width: 992px) {
.modal-body {
overflow: hidden;
}
.editor-layout {
min-height: 0;
}
}
.preview-page-input {
width: 3.5rem;
}
.preview-zoom {
min-width: 4rem;
}
.zone-overlay {
touch-action: none;
}
.zone-rect,
.zone-drawing-rect {
vector-effect: non-scaling-stroke;
}
.zone-rect {
fill-opacity: 0.2;
stroke-width: 2;
}
.zone-rect-selected {
stroke-width: 3;
}
.zone-label {
font-family: var(--bs-font-sans-serif);
paint-order: stroke;
stroke: #fff;
stroke-linejoin: round;
stroke-width: 4px;
vector-effect: non-scaling-stroke;
}
.zone-resize-handle {
stroke: #fff;
stroke-width: 1;
vector-effect: non-scaling-stroke;
}
.zone-drawing-rect {
fill: var(--bs-primary);
fill-opacity: 0.2;
stroke: var(--bs-primary);
stroke-dasharray: 5 5;
stroke-width: 2;
}
@@ -0,0 +1,878 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ElementRef } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
import { NgbActiveModal, NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { allIcons, NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { of, Subject, throwError } from 'rxjs'
import { PermissionsFormComponent } from 'src/app/components/common/input/permissions/permissions-form/permissions-form.component'
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
import { Document } from 'src/app/data/document'
import {
MATCH_ANY,
MATCH_AUTO,
MATCH_LITERAL,
MATCH_NONE,
} from 'src/app/data/matching-model'
import {
OcrTemplate,
OcrTemplateZone,
OcrZoneTarget,
OcrZoneTestResult,
OcrZoneTransform,
} from 'src/app/data/ocr-template'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { User } from 'src/app/data/user'
import { PermissionsService } from 'src/app/services/permissions.service'
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
import { DocumentService } from 'src/app/services/rest/document.service'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { UserService } from 'src/app/services/rest/user.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { OcrTemplateEditorComponent } from './ocr-template-editor.component'
const customFields: CustomField[] = [
{ id: 1, name: 'Invoice number', data_type: CustomFieldDataType.String },
{ id: 2, name: 'Status', data_type: CustomFieldDataType.Select },
]
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
return {
name: 'Invoice number',
target: OcrZoneTarget.CustomField,
custom_field: 1,
x: 100,
y: 100,
width: 200,
height: 100,
page: 1,
ocr_language: 'eng',
transform: OcrZoneTransform.Strip,
validation_regex: '',
order: 0,
...overrides,
}
}
function pointer(x: number, y: number, pointerId = 1): PointerEvent {
return Object.assign(
new MouseEvent('pointermove', { clientX: x, clientY: y, button: 0 }),
{ pointerId }
) as PointerEvent
}
describe('OcrTemplateEditorComponent', () => {
let component: OcrTemplateEditorComponent
let fixture: ComponentFixture<OcrTemplateEditorComponent>
let documentService: DocumentService
let templateService: OcrTemplateService
let modalService: NgbModal
const activeModal = { close: jest.fn(), dismiss: jest.fn() }
beforeEach(() => {
TestBed.configureTestingModule({
imports: [
NgxBootstrapIconsModule.pick(allIcons),
OcrTemplateEditorComponent,
],
providers: [
{ provide: NgbActiveModal, useValue: activeModal },
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
})
jest
.spyOn(TestBed.inject(CustomFieldsService), 'listAll')
.mockReturnValue(of({ count: 2, all: [1, 2], results: customFields }))
jest
.spyOn(TestBed.inject(UserService), 'listAll')
.mockReturnValue(of({ count: 0, all: [], results: [] }))
TestBed.inject(PermissionsService).initialize([], { id: 1 } as User)
documentService = TestBed.inject(DocumentService)
jest
.spyOn(documentService, 'get')
.mockReturnValue(of({ id: 5, page_count: 3 }))
templateService = TestBed.inject(OcrTemplateService)
modalService = TestBed.inject(NgbModal)
activeModal.close.mockReset()
fixture = TestBed.createComponent(OcrTemplateEditorComponent)
component = fixture.componentInstance
})
function prepareOverlay() {
const image = document.createElement('img')
Object.defineProperties(image, {
naturalWidth: { value: 1000 },
naturalHeight: { value: 1000 },
clientWidth: { value: 1000 },
})
const overlay = document.createElementNS(
'http://www.w3.org/2000/svg',
'svg'
)
Object.assign(overlay, {
getScreenCTM: () => ({ inverse: () => ({}) }),
createSVGPoint: () => ({
x: 0,
y: 0,
matrixTransform() {
return { x: this.x, y: this.y }
},
}),
setPointerCapture: jest.fn(),
hasPointerCapture: () => true,
releasePointerCapture: jest.fn(),
})
component.imageRef = new ElementRef(image)
component.overlayRef = new ElementRef(overlay)
component.pageLoaded.set(true)
}
function fieldDialog() {
const dialog = {
dialogMode: { set: jest.fn() },
object: null,
succeeded: new Subject<CustomField>(),
failed: new Subject(),
}
jest
.spyOn(modalService, 'open')
.mockReturnValue({ componentInstance: dialog } as any)
return dialog
}
it('updates screen-sized overlay units when the preview image resizes', () => {
let resize: ResizeObserverCallback
const observer = {
observe: jest.fn(),
unobserve: jest.fn(),
disconnect: jest.fn(),
}
const resizeObserver = jest
.spyOn(globalThis, 'ResizeObserver')
.mockImplementation((callback) => {
resize = callback
return observer
})
const image = document.createElement('img')
const ref = new ElementRef(image)
component.pageImage = ref
expect(observer.observe).toHaveBeenCalledWith(image)
resize([], observer)
expect(component.overlayUnitSize(8)).toBe(8)
Object.defineProperties(image, {
naturalWidth: { value: 1000 },
clientWidth: { value: 500 },
})
resize([], observer)
expect(component.overlayUnitSize(8)).toBe(16)
expect(component.overlayFontSize()).toBe(24)
component.pageImage = undefined
expect(observer.disconnect).toHaveBeenCalledTimes(1)
fixture.destroy()
expect(observer.disconnect).toHaveBeenCalledTimes(2)
resizeObserver.mockRestore()
})
it('renders loaded image dimensions, zone labels, and selection handles', () => {
component.object = {
...component.template,
id: 3,
sample_document: 5,
zones: [zone()],
}
component.selectedZoneIndex = 0
fixture.detectChanges()
const image = fixture.nativeElement.querySelector('img')
Object.defineProperties(image, {
naturalWidth: { value: 1000 },
naturalHeight: { value: 1500 },
})
image.dispatchEvent(new Event('load'))
fixture.detectChanges()
expect(component.template).toMatchObject({
source_width: 1000,
source_height: 1500,
})
expect(
fixture.nativeElement
.querySelector('.zone-overlay')
.getAttribute('viewBox')
).toBe('0 0 1000 1500')
expect(
fixture.nativeElement.querySelector('.zone-label').textContent
).toContain('Invoice number')
expect(
fixture.nativeElement.querySelectorAll('.zone-resize-handle')
).toHaveLength(8)
expect(component.zoneLabel(zone({ name: '' }), 2)).toBe('Zone 3')
expect(component.zoneLabelY({ x: 0, y: 0, w: 100, h: 100 })).toBe(14)
})
it('offers the supported language and transform choices', () => {
expect(component.languageOptions.map((option) => option.id)).toContain(
'eng'
)
expect(component.transformOptions.map((option) => option.id)).toEqual(
Object.values(OcrZoneTransform)
)
})
it('clamps page navigation and supports one-based page input', () => {
component.setSampleDocument([5])
component.previewPageDisplay = 2
expect(component.previewPage).toBe(1)
expect(component.previewPageDisplay).toBe(2)
expect(component.pageImageUrl).toContain('/5/1/')
component.pageLoaded.set(true)
component.goToPage(1)
expect(component.pageLoaded()).toBe(true)
component.goToPage(Infinity)
component.goToPage(NaN)
expect(component.previewPage).toBe(1)
component.goToPage(99)
expect(component.previewPage).toBe(2)
expect(component.pageLoaded()).toBe(false)
component.goToPage(-5)
expect(component.previewPage).toBe(0)
component.previewPageCount.set(null)
component.goToPage(4.7)
expect(component.previewPage).toBe(4)
})
it('limits zoom and resets it to the original size', () => {
for (let i = 0; i < 20; i++) component.zoomIn()
expect(component.zoom).toBe(4)
for (let i = 0; i < 20; i++) component.zoomOut()
expect(component.zoom).toBe(0.5)
component.resetZoom()
expect(component.zoom).toBe(1)
})
it('moves last-page zones to the last page and can pin them to a page', () => {
component.setSampleDocument([5])
const selected = zone()
component.template.zones = [selected]
expect(component.isLastPageZone(selected)).toBe(false)
component.setLastPageZone(selected, true)
expect(selected.page).toBe(-1)
expect(component.isLastPageZone(selected)).toBe(true)
component.selectZone(0)
expect(component.previewPage).toBe(2)
component.setLastPageZone(selected, false)
expect(selected.page).toBe(3)
})
it('ignores page-count errors from a previous sample document', () => {
const pending = new Subject<Document>()
jest
.spyOn(documentService, 'get')
.mockReturnValueOnce(pending)
.mockReturnValueOnce(of({ id: 6 }))
component.setSampleDocument([5])
component.setSampleDocument([6])
pending.error(new Error('Old document failed'))
expect(component.pageFailed()).toBe(false)
expect(component.previewPageCount()).toBeNull()
})
it('ignores non-primary, unloaded, and saving pointer gestures', () => {
prepareOverlay()
component.onOverlayPointerDown(
new MouseEvent('pointerdown', { button: 2 }) as PointerEvent
)
expect(component.drawingRect()).toBeNull()
component.pageLoaded.set(false)
component.onOverlayPointerDown(pointer(100, 100))
expect(component.drawingRect()).toBeNull()
component.pageLoaded.set(true)
component.saving.set(true)
component.onOverlayPointerDown(pointer(100, 100))
expect(component.drawingRect()).toBeNull()
})
it('ignores pointer events when the SVG coordinate transform is unavailable', () => {
component.onOverlayPointerDown(pointer(100, 100))
expect(component.drawingRect()).toBeNull()
prepareOverlay()
jest
.spyOn(component.overlayRef.nativeElement, 'getScreenCTM')
.mockReturnValue(null)
component.onOverlayPointerDown(pointer(100, 100))
component.onOverlayPointerMove(pointer(200, 200))
expect(component.drawingRect()).toBeNull()
})
it('updates a drawing only for its captured pointer', () => {
prepareOverlay()
component.onOverlayPointerDown(pointer(100, 100))
component.onOverlayPointerDown(pointer(500, 500, 2))
component.onOverlayPointerMove(pointer(900, 900, 2))
expect(component.drawingRect()).toEqual({ x: 100, y: 100, w: 0, h: 0 })
component.onOverlayPointerMove(pointer(250, 200))
expect(component.drawingRect()).toEqual({ x: 100, y: 100, w: 150, h: 100 })
component.onOverlayPointerUp(pointer(250, 200))
expect(component.template.zones[0]).toMatchObject({
x: 100,
y: 100,
width: 150,
height: 100,
})
})
it('resizes a selected zone using its corner handle', () => {
prepareOverlay()
const selected = zone()
component.template.zones = [selected]
component.selectZone(0)
component.onOverlayPointerDown(pointer(100, 100))
component.onOverlayPointerMove(pointer(50, 75))
component.onOverlayPointerUp(pointer(60, 80))
expect(selected).toMatchObject({ x: 60, y: 80, width: 240, height: 120 })
expect(component.template.zones).toHaveLength(1)
})
it('shows the appropriate cursor over handles, zones, and empty space', () => {
prepareOverlay()
component.template.zones = [zone()]
component.onOverlayPointerMove(pointer(150, 150))
expect(component.overlayCursor).toBe('move')
component.selectZone(0)
component.onOverlayPointerMove(pointer(100, 100))
expect(component.overlayCursor).toBe('nw-resize')
component.onOverlayPointerMove(pointer(500, 500))
expect(component.overlayCursor).toBe('crosshair')
component.template.zones[0].page = 2
component.onOverlayPointerMove(pointer(100, 100))
expect(component.zoneDisplayRect(0)).toBeNull()
expect(component.overlayCursor).toBe('crosshair')
})
it('cancels only the captured pointer and tolerates an already-lost capture', () => {
prepareOverlay()
component.onOverlayPointerCancel(pointer(100, 100))
component.onOverlayPointerDown(pointer(100, 100))
component.onOverlayPointerCancel(pointer(150, 150, 2))
expect(component.drawingRect()).not.toBeNull()
jest
.spyOn(component.overlayRef.nativeElement, 'hasPointerCapture')
.mockReturnValue(false)
component.onOverlayPointerCancel(pointer(150, 150))
expect(component.drawingRect()).toBeNull()
expect(
component.overlayRef.nativeElement.releasePointerCapture
).not.toHaveBeenCalled()
expect(component.template.zones).toEqual([])
})
it('preserves the selected zone when an earlier or later zone is removed', () => {
component.template.zones = [zone(), zone(), zone(), zone()]
component.selectZone(1)
const selected = component.selectedZone
component.removeZone(3)
expect(component.selectedZone).toBe(selected)
component.removeZone(0)
expect(component.selectedZoneIndex).toBe(0)
expect(component.selectedZone).toBe(selected)
component.selectZone(99)
expect(component.selectedZone).toBeNull()
})
it('does not test zones without a selection or a sample document', () => {
const test = jest.spyOn(templateService, 'testZone')
component.testZone()
component.template.zones = [zone()]
component.selectZone(0)
component.testZone()
expect(test).not.toHaveBeenCalled()
})
it.each([
{
response: { error: { error: 'OCR unavailable' } },
message: 'OCR unavailable',
},
{ response: new Error('Network failure'), message: 'Test failed' },
])('shows zone-test errors: $message', ({ response, message }) => {
component.template.zones = [zone({ page: undefined })]
component.selectZone(0)
component.setSampleDocument([5])
jest
.spyOn(templateService, 'testZone')
.mockReturnValue(throwError(() => response))
component.testZone()
expect(component.zoneTestResult).toEqual({ error: message })
expect(component.zoneTesting).toBe(false)
expect(templateService.testZone).toHaveBeenCalledWith(
5,
expect.objectContaining({ page: 1 })
)
})
it('switches between built-in targets, custom fields, and an unassigned field', () => {
const selected = zone()
expect(component.zoneFieldValue(selected)).toBe(1)
component.setZoneField(selected, OcrZoneTarget.Title)
expect(component.zoneFieldValue(selected)).toBe(OcrZoneTarget.Title)
expect(selected.custom_field).toBeNull()
component.setZoneField(selected, 7)
expect(selected).toMatchObject({
target: OcrZoneTarget.CustomField,
custom_field: 7,
})
component.setZoneField(selected, null)
expect(component.zoneFieldValue(selected)).toBeNull()
expect(component.zonesForField(selected)).toEqual([])
component.setCombineFormat(selected, 'Ignored')
expect(component.template.combine_formats).toEqual({})
})
it('creates combine formats when absent and leaves implicit formats alone on rename', () => {
const selected = zone({ target: OcrZoneTarget.Title, custom_field: null })
component.template.combine_formats = undefined
component.renameZone(selected, 'Renamed')
expect(selected.name).toBe('Renamed')
expect(component.getCombineFormat(selected)).toBe('')
component.setCombineFormat(selected, '{Renamed} ')
component.insertCombineToken(selected, selected)
expect(component.template.combine_formats).toEqual({
title: '{Renamed} {Renamed}',
})
})
it('saves templates without combine formats', () => {
component.template.combine_formats = undefined
fixture.detectChanges()
expect(component.isDirty).toBe(false)
jest
.spyOn(templateService, 'create')
.mockReturnValue(of(component.template))
component.save()
expect(component.template.combine_formats).toBeUndefined()
})
it('does not assign unsupported custom field types and surfaces creation failures', () => {
component.template.zones = [zone({ custom_field: null })]
component.selectZone(0)
const dialog = fieldDialog()
const error = jest.spyOn(TestBed.inject(ToastService), 'showError')
component.createField('Status')
expect(dialog.object).toEqual({ name: 'Status' })
dialog.succeeded.next(customFields[1])
expect(component.selectedZone.custom_field).toBeNull()
const failure = new Error('Could not save')
dialog.failed.next(failure)
expect(error).toHaveBeenCalledWith('Error saving field.', failure)
})
it('refreshes field choices when a field is created without a selected zone', () => {
const dialog = fieldDialog()
component.createField('Total')
dialog.succeeded.next(customFields[0])
expect(component.customFields()).toEqual(customFields)
expect(component.selectedZone).toBeNull()
})
it('should edit a copy of the template and load its sample document', () => {
const original: OcrTemplate = {
...component.template,
id: 3,
name: 'Invoices',
match: 'Northwind',
matching_algorithm: MATCH_LITERAL,
sample_document: 5,
zones: [zone()],
combine_formats: { '1': '{Invoice number}' },
}
component.object = original
fixture.detectChanges()
expect(component.dialogTitle).toEqual('Edit OCR Template')
expect(component.previewDocId).toEqual(5)
expect(component.previewPageCount()).toEqual(3)
expect(component.pageImageUrl).toContain('document-page-image/5/0/')
expect(component.template.match).toEqual('Northwind')
expect(component.isDirty).toBeFalsy()
component.template.zones[0].x = 500
component.template.combine_formats['1'] = 'Edited'
expect(original.zones[0].x).toEqual(100)
expect(original.combine_formats['1']).toEqual('{Invoice number}')
})
it('should offer every matching algorithm except automatic', () => {
expect(component.matchingAlgorithms.map((a) => a.id)).not.toContain(
MATCH_AUTO
)
expect(component.template.matching_algorithm).toEqual(MATCH_ANY)
expect(component.patternRequired).toBeTruthy()
component.template.matching_algorithm = MATCH_NONE
expect(component.patternRequired).toBeFalsy()
})
it('should initialize new template permissions from saved defaults', () => {
const defaults = {
[SETTINGS_KEYS.DEFAULT_PERMS_OWNER]: 1,
[SETTINGS_KEYS.DEFAULT_PERMS_VIEW_USERS]: [2],
[SETTINGS_KEYS.DEFAULT_PERMS_VIEW_GROUPS]: [3],
[SETTINGS_KEYS.DEFAULT_PERMS_EDIT_USERS]: [],
[SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS]: [4],
}
jest
.spyOn(TestBed.inject(SettingsService), 'get')
.mockImplementation((key) => defaults[key])
fixture.detectChanges()
expect(component.permissionsForm).toEqual({
owner: 1,
set_permissions: {
view: { users: [2], groups: [3] },
change: { users: [], groups: [4] },
},
})
expect(
fixture.debugElement.query(By.directive(PermissionsFormComponent))
).toBeTruthy()
expect(component.isDirty).toBe(false)
const create = jest
.spyOn(templateService, 'create')
.mockReturnValue(of({ ...component.template, id: 3 }))
component.save()
expect(create).toHaveBeenCalledWith(
expect.objectContaining(component.permissionsForm)
)
})
it('should copy existing permissions and track changes without mutating the list item', () => {
component.object = {
...component.template,
id: 3,
owner: 1,
permissions: {
view: { users: [2], groups: [] },
change: { users: [], groups: [3] },
},
}
fixture.detectChanges()
expect(component.permissionsForm.owner).toBe(1)
expect(component.isDirty).toBe(false)
component.permissionsForm.set_permissions.view.users.push(4)
expect(component.isDirty).toBe(true)
expect(component.object.permissions.view.users).toEqual([2])
const update = jest
.spyOn(templateService, 'update')
.mockReturnValue(of(component.template))
component.save()
expect(update).toHaveBeenCalledWith(
expect.objectContaining({
owner: 1,
set_permissions: {
view: { users: [2, 4], groups: [] },
change: { users: [], groups: [3] },
},
})
)
})
it('should hide sharing controls and omit permission changes for a shared editor', () => {
component.object = {
...component.template,
id: 3,
owner: 2,
user_can_change: true,
}
fixture.detectChanges()
expect(
fixture.debugElement.query(By.directive(PermissionsFormComponent))
).toBeNull()
component.permissionsForm = {
owner: 1,
set_permissions: { view: { users: [1] } },
}
const update = jest
.spyOn(templateService, 'update')
.mockReturnValue(of(component.template))
component.save()
expect(update.mock.calls[0][0].owner).toBe(2)
expect(update.mock.calls[0][0]).not.toHaveProperty('set_permissions')
})
it('should keep only the last picked sample document', () => {
component.setSampleDocument([5, 6])
expect(component.sampleDocumentIds).toEqual([6])
component.setSampleDocument([])
expect(component.previewDocId).toBeNull()
expect(component.pageImageUrl).toBeNull()
})
it('should ignore a stale sample document response', () => {
const pending = new Subject<Document>()
jest.spyOn(documentService, 'get').mockReturnValueOnce(pending)
component.setSampleDocument([5])
component.setSampleDocument([6])
pending.next({ id: 5, page_count: 99 })
expect(component.previewPageCount()).toEqual(3)
})
it('should show a failed page and retry it', () => {
jest
.spyOn(documentService, 'get')
.mockReturnValueOnce(throwError(() => new Error('offline')))
component.setSampleDocument([5])
expect(component.pageFailed()).toBeTruthy()
component.setSampleDocument(component.sampleDocumentIds)
expect(component.pageFailed()).toBeFalsy()
component.onImageError()
expect(component.pageFailed()).toBeTruthy()
})
it('should support create and update', () => {
const createSpy = jest
.spyOn(templateService, 'create')
.mockReturnValue(of({ ...component.template, id: 3 }))
const updateSpy = jest
.spyOn(templateService, 'update')
.mockReturnValue(of(component.template))
const succeededSpy = jest.spyOn(component.succeeded, 'emit')
component.save()
expect(createSpy).toHaveBeenCalled()
expect(succeededSpy).toHaveBeenCalledWith({ ...component.template, id: 3 })
expect(activeModal.close).toHaveBeenCalled()
component.saving.set(false)
component.template.id = 3
component.save()
expect(updateSpy).toHaveBeenCalledWith(component.template)
})
it('should emit failed and allow editing again after a failed save', () => {
prepareOverlay()
component.template.zones = [zone()]
const pending = new Subject<OcrTemplate>()
jest.spyOn(templateService, 'create').mockReturnValue(pending)
const failedSpy = jest.spyOn(component.failed, 'emit')
component.onOverlayPointerDown(pointer(150, 150))
component.save()
component.save()
expect(templateService.create).toHaveBeenCalledTimes(1)
component.onOverlayPointerMove(pointer(250, 250))
component.onOverlayPointerUp(pointer(250, 250))
expect(component.template.zones[0].x).toEqual(100)
pending.error(new Error('Save failed'))
expect(failedSpy).toHaveBeenCalled()
expect(component.saving()).toBeFalsy()
component.onOverlayPointerDown(pointer(150, 150))
component.onOverlayPointerUp(pointer(250, 250))
expect(component.template.zones[0].x).toEqual(200)
})
it('should track unsaved changes', () => {
fixture.detectChanges()
expect(component.isDirty).toBeFalsy()
component.template.combine_formats = { '1': '' }
expect(component.isDirty).toBeFalsy()
component.template.name = 'Invoices'
expect(component.isDirty).toBeTruthy()
component.template.name = ''
expect(component.isDirty).toBeFalsy()
component.setSampleDocument([5])
expect(component.isDirty).toBeTruthy()
})
it('should draw a zone with no field in empty space', () => {
prepareOverlay()
component.onOverlayPointerDown(pointer(100, 100))
component.onOverlayPointerUp(pointer(1200, 250))
expect(component.template.zones[0]).toMatchObject({
x: 100,
y: 100,
width: 900,
height: 150,
custom_field: null,
})
expect(component.selectedZoneIndex).toEqual(0)
})
it('should clear the selection on a click in empty space', () => {
prepareOverlay()
component.template.zones = [zone()]
component.selectZone(0)
component.onOverlayPointerDown(pointer(600, 600))
component.onOverlayPointerUp(pointer(602, 601))
expect(component.template.zones).toHaveLength(1)
expect(component.selectedZoneIndex).toBeNull()
})
it('should move a zone when dragging on it', () => {
prepareOverlay()
component.template.zones = [zone()]
component.onOverlayPointerDown(pointer(150, 150))
component.onOverlayPointerUp(pointer(250, 200))
expect(component.template.zones).toHaveLength(1)
expect(component.template.zones[0]).toMatchObject({ x: 200, y: 150 })
})
it('should cancel a drawing on escape and ignore other pointers', () => {
prepareOverlay()
component.onOverlayPointerDown(pointer(100, 100))
component.onOverlayPointerUp(pointer(300, 200, 2))
expect(component.drawingRect()).not.toBeNull()
component.cancelDrawing()
component.onOverlayPointerUp(pointer(300, 200))
expect(component.template.zones).toEqual([])
})
it('should open a zone in the zones tab and go back to the list', () => {
component.template.zones = [zone(), zone()]
component.selectZone(1)
expect(component.activeTab).toEqual('zones')
expect(component.selectedZone).toBe(component.template.zones[1])
component.deselectZone()
expect(component.selectedZone).toBeNull()
component.selectZone(0)
component.removeZone(0)
expect(component.template.zones).toHaveLength(1)
expect(component.selectedZone).toBeNull()
})
it('should only offer custom fields OCR can write to', () => {
fixture.detectChanges()
expect(component.fieldItems().map((item) => item.id)).toEqual([
OcrZoneTarget.Title,
OcrZoneTarget.Asn,
OcrZoneTarget.Created,
1,
])
})
it('should create a custom field and assign it to the zone', () => {
const succeeded = new Subject<CustomField>()
const dialog = {
dialogMode: { set: jest.fn() },
object: null,
succeeded,
failed: new Subject(),
}
jest
.spyOn(modalService, 'open')
.mockReturnValue({ componentInstance: dialog } as any)
component.template.zones = [zone({ custom_field: null })]
component.selectZone(0)
component.createField()
expect(dialog.object).toEqual({ name: 'Invoice number' })
succeeded.next(customFields[0])
expect(component.template.zones[0].custom_field).toEqual(1)
})
it('should build and prune combine formats', () => {
const first = zone({ name: 'First' })
const second = zone({ name: 'Second' })
const other = zone({ name: 'Other', custom_field: 2 })
component.template.zones = [first, second, other]
component.template.combine_formats = { '2': '{Other}' }
expect(component.isFieldShared(first)).toBeTruthy()
expect(component.isFieldShared(other)).toBeFalsy()
expect(component.defaultCombineFormat(first)).toEqual('{First} {Second}')
component.insertCombineToken(first, second)
component.insertCombineToken(first, first)
expect(component.getCombineFormat(second)).toEqual('{Second} {First}')
jest.spyOn(templateService, 'create').mockReturnValue(of(null))
component.save()
expect(component.template.combine_formats).toEqual({
'1': '{Second} {First}',
})
})
it('should update exact tokens when a zone is renamed', () => {
const first = zone({ name: 'First [value]' })
component.template.zones = [first, zone({ name: 'Second' })]
component.template.combine_formats = {
'1': 'First [value]: {First [value]} / { First [value] } / {Second} / {First [value] extra}',
'2': '{First [value]}',
}
component.renameZone(first, '')
component.renameZone(first, 'Invoice number')
expect(first.name).toEqual('Invoice number')
expect(component.template.combine_formats).toEqual({
'1': 'First [value]: {Invoice number} / {Invoice number} / {Second} / {First [value] extra}',
'2': '{First [value]}',
})
})
it('should keep the language array stable until the value changes', () => {
const selected = zone({ ocr_language: 'deu+eng' })
const languages = component.ocrLanguageArray(selected)
expect(languages).toEqual(['deu', 'eng'])
expect(component.ocrLanguageArray(selected)).toBe(languages)
component.setOcrLanguages(selected, ['fra'])
expect(selected.ocr_language).toEqual('fra')
expect(component.ocrLanguageArray(selected)).toEqual(['fra'])
component.setOcrLanguages(selected, null)
expect(component.ocrLanguageArray(selected)).toEqual([])
})
it('should hide a test result once the zone changes', () => {
const selected = zone()
component.template.zones = [selected]
component.selectZone(0)
component.setSampleDocument([5])
jest
.spyOn(templateService, 'testZone')
.mockReturnValue(of({ value: '123' }))
component.testZone()
expect(component.zoneTestResult.value).toEqual('123')
selected.transform = OcrZoneTransform.Numeric
expect(component.zoneTestResult).toBeNull()
})
it('should drop a pending test when the zone or sample document changes', () => {
const pending = new Subject<OcrZoneTestResult>()
component.template.zones = [zone(), zone()]
component.selectZone(0)
component.setSampleDocument([5])
jest.spyOn(templateService, 'testZone').mockReturnValue(pending)
component.testZone()
expect(component.zoneTesting).toBeTruthy()
component.selectZone(1)
pending.next({ value: 'Old zone' })
expect(component.zoneTestResult).toBeNull()
component.testZone()
component.setSampleDocument([])
pending.next({ value: 'Old document' })
expect(component.zoneTestResult).toBeNull()
})
})
@@ -0,0 +1,963 @@
import { CommonModule } from '@angular/common'
import {
Component,
computed,
ElementRef,
EventEmitter,
HostListener,
inject,
Input,
OnDestroy,
OnInit,
Output,
signal,
ViewChild,
} from '@angular/core'
import { FormsModule } from '@angular/forms'
import {
NgbActiveModal,
NgbModal,
NgbNavModule,
} from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { Subject, takeUntil } from 'rxjs'
import { CustomFieldEditDialogComponent } from 'src/app/components/common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
import { EditDialogMode } from 'src/app/components/common/edit-dialog/edit-dialog.component'
import { DocumentLinkComponent } from 'src/app/components/common/input/document-link/document-link.component'
import { CheckComponent } from 'src/app/components/common/input/check/check.component'
import { NumberComponent } from 'src/app/components/common/input/number/number.component'
import {
PermissionsFormComponent,
PermissionsFormObject,
} from 'src/app/components/common/input/permissions/permissions-form/permissions-form.component'
import { SelectComponent } from 'src/app/components/common/input/select/select.component'
import { SwitchComponent } from 'src/app/components/common/input/switch/switch.component'
import { TextComponent } from 'src/app/components/common/input/text/text.component'
import { CustomField } from 'src/app/data/custom-field'
import {
MATCH_ANY,
MATCH_AUTO,
MATCH_NONE,
MATCHING_ALGORITHMS,
} from 'src/app/data/matching-model'
import {
DEFAULT_OCR_ZONE_LANGUAGE,
getZoneFieldKey,
OCR_BUILTIN_TARGETS,
OCR_SUPPORTED_FIELD_TYPES,
OcrTemplate,
OcrTemplateZone,
OcrZoneTarget,
OcrZoneTestResult,
OcrZoneTransform,
ZoneTestRequest,
} from 'src/app/data/ocr-template'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { User } from 'src/app/data/user'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { PermissionsService } from 'src/app/services/permissions.service'
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
import { DocumentService } from 'src/app/services/rest/document.service'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { UserService } from 'src/app/services/rest/user.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { OcrTemplateEditorZoneListComponent } from './ocr-template-editor-zone-list/ocr-template-editor-zone-list.component'
import {
DisplayRect,
DrawingRect,
findHandleAt,
findZoneAt,
getResizeHandles,
getZoneColor,
getZoneDisplayRect,
getZonePage,
HANDLE_SIZE,
isZoneOnPage,
MIN_ZONE_SIZE,
MoveStart,
moveZone,
Point,
ResizeHandle,
resizeZone,
} from './zone-geometry'
type ActiveTab = 'template' | 'zones'
type ZoneFieldSelection = OcrZoneTarget | number | null
type OverlayInteraction =
| { kind: 'idle' }
| { kind: 'drawing'; pointerId: number; rect: DrawingRect }
| { kind: 'moving'; pointerId: number; zoneIndex: number; start: MoveStart }
| {
kind: 'resizing'
pointerId: number
zoneIndex: number
handle: ResizeHandle
}
interface ZoneTest {
zone: OcrTemplateZone
/** The sample document and zone settings the result was produced from. */
key: string
result: OcrZoneTestResult | null
}
export const ZONE_TRANSFORM_OPTIONS = [
{ id: OcrZoneTransform.None, name: $localize`None` },
{ id: OcrZoneTransform.Strip, name: $localize`Strip whitespace` },
{ id: OcrZoneTransform.Uppercase, name: $localize`Uppercase` },
{ id: OcrZoneTransform.Lowercase, name: $localize`Lowercase` },
{ id: OcrZoneTransform.Numeric, name: $localize`Numeric only` },
{
id: OcrZoneTransform.StripPunctuation,
name: $localize`Remove leading/trailing punctuation`,
},
{ id: OcrZoneTransform.Date, name: $localize`Parse date` },
{ id: OcrZoneTransform.QrCode, name: $localize`Read QR/barcode` },
]
export const ZONE_LANGUAGE_OPTIONS = [
{ id: 'eng', name: $localize`English` },
{ id: 'deu', name: $localize`German` },
{ id: 'fra', name: $localize`French` },
{ id: 'ita', name: $localize`Italian` },
{ id: 'spa', name: $localize`Spanish` },
{ id: 'por', name: $localize`Portuguese` },
{ id: 'nld', name: $localize`Dutch` },
]
const TEMPLATE_MATCHING_ALGORITHMS = MATCHING_ALGORITHMS.filter(
(algorithm) => algorithm.id !== MATCH_AUTO
)
const NO_OVERLAY_INTERACTION: OverlayInteraction = { kind: 'idle' }
const RESIZE_CURSOR: Record<ResizeHandle, string> = {
nw: 'nw-resize',
ne: 'ne-resize',
sw: 'sw-resize',
se: 'se-resize',
n: 'n-resize',
s: 's-resize',
w: 'w-resize',
e: 'e-resize',
}
@Component({
selector: 'pngx-ocr-template-editor',
imports: [
DocumentLinkComponent,
NumberComponent,
TextComponent,
CheckComponent,
SelectComponent,
SwitchComponent,
CommonModule,
FormsModule,
NgbNavModule,
NgxBootstrapIconsModule,
OcrTemplateEditorZoneListComponent,
PermissionsFormComponent,
IfOwnerDirective,
],
templateUrl: './ocr-template-editor.component.html',
styleUrls: ['./ocr-template-editor.component.scss'],
})
export class OcrTemplateEditorComponent implements OnInit, OnDestroy {
public readonly activeModal = inject(NgbActiveModal)
private readonly templateService = inject(OcrTemplateService)
private readonly customFieldsService = inject(CustomFieldsService)
private readonly documentService = inject(DocumentService)
private readonly toastService = inject(ToastService)
private readonly modalService = inject(NgbModal)
private readonly userService = inject(UserService)
private readonly settingsService = inject(SettingsService)
private readonly permissionsService = inject(PermissionsService)
private readonly destroy$ = new Subject<void>()
private readonly cancelZoneTest$ = new Subject<void>()
@Input() object: OcrTemplate
@Output() public readonly succeeded = new EventEmitter<OcrTemplate>()
@Output() public readonly failed = new EventEmitter()
@ViewChild('zoneOverlay') overlayRef: ElementRef<SVGSVGElement>
imageRef: ElementRef<HTMLImageElement>
private previewResizeObserver: ResizeObserver | null = null
private readonly overlayScale = signal(1)
@ViewChild('pageImage')
set pageImage(ref: ElementRef<HTMLImageElement> | undefined) {
this.previewResizeObserver?.disconnect()
this.imageRef = ref
if (!ref) return
this.previewResizeObserver = new ResizeObserver(() => {
const image = ref.nativeElement
if (image.naturalWidth && image.clientWidth) {
this.overlayScale.set(image.naturalWidth / image.clientWidth)
}
})
this.previewResizeObserver.observe(ref.nativeElement)
}
template: OcrTemplate = {
id: null,
name: '',
match: '',
matching_algorithm: MATCH_ANY,
is_insensitive: true,
sample_document: null,
source_width: 0,
source_height: 0,
enabled: true,
combine_formats: {},
zones: [],
}
public readonly customFields = signal<CustomField[]>([])
public readonly fieldItems = computed<
{ id: ZoneFieldSelection; name: string }[]
>(() => [
...OCR_BUILTIN_TARGETS,
...this.customFields()
.filter((field) => OCR_SUPPORTED_FIELD_TYPES.includes(field.data_type))
.map((field) => ({ id: field.id, name: field.name })),
])
public readonly users = signal<User[]>([])
permissionsForm: PermissionsFormObject = {}
public readonly OcrZoneTransform = OcrZoneTransform
public readonly saving = signal(false)
/** Bound to the document picker, which works with id arrays. */
sampleDocumentIds: number[] = []
previewPage = 0
public readonly previewPageCount = signal<number | null>(null)
public readonly pageLoaded = signal(false)
public readonly pageFailed = signal(false)
zoom = 1
get previewDocId(): number | null {
return this.sampleDocumentIds[0] ?? null
}
get pageImageUrl(): string | null {
return this.previewDocId
? this.templateService.getPageImageUrl(
this.previewDocId,
this.previewPage
)
: null
}
get transformOptions() {
return ZONE_TRANSFORM_OPTIONS
}
get languageOptions() {
return ZONE_LANGUAGE_OPTIONS
}
get matchingAlgorithms() {
return TEMPLATE_MATCHING_ALGORITHMS
}
get patternRequired(): boolean {
return (
this.template.matching_algorithm !== MATCH_AUTO &&
this.template.matching_algorithm !== MATCH_NONE
)
}
public get previewPageDisplay(): number {
return this.previewPage + 1
}
public set previewPageDisplay(value: number) {
this.goToPage(value - 1)
}
activeTab: ActiveTab = 'template'
selectedZoneIndex: number | null = null
private overlayInteraction: OverlayInteraction = NO_OVERLAY_INTERACTION
overlayCursor = 'crosshair'
private readonly zoneTest = signal<ZoneTest | null>(null)
get zoneTestResult(): OcrZoneTestResult | null {
return this.isCurrentZoneTest() ? this.zoneTest().result : null
}
get zoneTesting(): boolean {
return this.isCurrentZoneTest() && this.zoneTest().result === null
}
get selectedZone(): OcrTemplateZone | null {
return this.selectedZoneIndex !== null
? (this.template.zones[this.selectedZoneIndex] ?? null)
: null
}
private initialState = ''
get isDirty(): boolean {
return this.editorState() !== this.initialState
}
private editorState(): string {
const formats = Object.entries(this.template.combine_formats ?? {}).filter(
([, format]) => format
)
return JSON.stringify({
...this.template,
combine_formats: Object.fromEntries(formats),
sample_document: this.previewDocId ?? null,
permissions_form: this.permissionsForm,
})
}
get dialogTitle(): string {
return this.template.id
? $localize`Edit OCR Template`
: $localize`New OCR Template`
}
ngOnInit() {
this.userService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => this.users.set(r.results))
this.customFieldsService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => {
this.customFields.set(r.results)
})
if (this.object) {
this.template = {
...this.object,
zones: this.object.zones.map((zone) => ({ ...zone })),
combine_formats: { ...this.object.combine_formats },
}
if (this.template.sample_document) {
this.setSampleDocument([this.template.sample_document])
}
}
const permissions = this.object?.permissions
this.permissionsForm = this.object
? {
owner: this.object.owner,
set_permissions: {
view: {
users: [...(permissions?.view.users ?? [])],
groups: [...(permissions?.view.groups ?? [])],
},
change: {
users: [...(permissions?.change.users ?? [])],
groups: [...(permissions?.change.groups ?? [])],
},
},
}
: {
owner: this.settingsService.get(SETTINGS_KEYS.DEFAULT_PERMS_OWNER),
set_permissions: {
view: {
users: this.settingsService.get(
SETTINGS_KEYS.DEFAULT_PERMS_VIEW_USERS
),
groups: this.settingsService.get(
SETTINGS_KEYS.DEFAULT_PERMS_VIEW_GROUPS
),
},
change: {
users: this.settingsService.get(
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_USERS
),
groups: this.settingsService.get(
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS
),
},
},
}
this.initialState = this.editorState()
}
setSampleDocument(ids: number[]) {
const id = ids.at(-1)
this.sampleDocumentIds = id ? [id] : []
this.previewPage = 0
this.previewPageCount.set(null)
this.resetPage()
if (!id) return
this.documentService
.get(id)
.pipe(takeUntil(this.destroy$))
.subscribe({
next: (doc) => {
if (this.previewDocId !== id) return
this.previewPageCount.set(doc.page_count ?? null)
},
error: () => {
if (this.previewDocId === id) this.pageFailed.set(true)
},
})
}
private resetPage() {
this.stopOverlayInteraction()
this.clearZoneTest()
this.pageLoaded.set(false)
this.pageFailed.set(false)
}
goToPage(page: number) {
if (!Number.isFinite(page)) return
const pageCount = this.previewPageCount()
const max = pageCount ? pageCount - 1 : page
const clamped = Math.max(0, Math.min(Math.trunc(page), max))
if (clamped === this.previewPage) return
this.previewPage = clamped
this.resetPage()
}
zoomIn() {
this.zoom = Math.min(4, Math.round((this.zoom + 0.25) * 100) / 100)
}
zoomOut() {
this.zoom = Math.max(0.5, Math.round((this.zoom - 0.25) * 100) / 100)
}
resetZoom() {
this.zoom = 1
}
zonePage(zone: OcrTemplateZone): number {
return getZonePage(zone, this.previewPage, this.previewPageCount())
}
isLastPageZone(zone: OcrTemplateZone): boolean {
return zone.page === -1
}
setLastPageZone(zone: OcrTemplateZone, lastPage: boolean) {
zone.page = lastPage ? -1 : this.previewPage + 1
}
private isOnCurrentPage(zone: OcrTemplateZone): boolean {
return isZoneOnPage(zone, this.previewPage, this.previewPageCount())
}
onImageLoad() {
this.pageLoaded.set(true)
const img = this.imageRef.nativeElement
this.template.source_width = img.naturalWidth
this.template.source_height = img.naturalHeight
}
onImageError() {
this.stopOverlayInteraction()
this.pageFailed.set(true)
}
@HostListener('document:keydown.escape')
cancelDrawing() {
this.stopOverlayInteraction()
}
onOverlayPointerDown(event: PointerEvent) {
if (this.saving()) return
if (event.button !== 0 || this.overlayInteraction.kind !== 'idle') return
const point = this.svgPointFromEvent(event)
if (!this.pageLoaded() || !point) return
event.preventDefault()
if (this.selectedZoneIndex !== null) {
const handle = this.handleAt(point, this.selectedZoneIndex)
if (handle) {
this.overlayInteraction = {
kind: 'resizing',
pointerId: event.pointerId,
zoneIndex: this.selectedZoneIndex,
handle,
}
this.overlayRef.nativeElement.setPointerCapture(event.pointerId)
return
}
}
const clickedIdx = this.zoneIndexAt(point)
if (clickedIdx !== null) {
this.selectZone(clickedIdx)
const zone = this.template.zones[clickedIdx]
this.overlayInteraction = {
kind: 'moving',
pointerId: event.pointerId,
zoneIndex: clickedIdx,
start: {
mouseX: point.x,
mouseY: point.y,
zoneX: zone.x,
zoneY: zone.y,
},
}
this.overlayRef.nativeElement.setPointerCapture(event.pointerId)
return
}
// Empty space: start drawing a new zone. A click without a drag just clears
// the selection (see onOverlayPointerUp).
this.clearZoneTest()
this.selectedZoneIndex = null
this.overlayInteraction = {
kind: 'drawing',
pointerId: event.pointerId,
rect: {
startX: point.x,
startY: point.y,
endX: point.x,
endY: point.y,
},
}
this.overlayRef.nativeElement.setPointerCapture(event.pointerId)
}
onOverlayPointerMove(event: PointerEvent) {
if (
this.overlayInteraction.kind !== 'idle' &&
this.overlayInteraction.pointerId !== event.pointerId
)
return
const point = this.svgPointFromEvent(event)
if (!point) return
if (this.overlayInteraction.kind === 'resizing') {
resizeZone(
this.template.zones[this.overlayInteraction.zoneIndex],
this.overlayInteraction.handle,
point,
this.imageNaturalSize()
)
return
}
if (this.overlayInteraction.kind === 'moving') {
moveZone(
this.template.zones[this.overlayInteraction.zoneIndex],
point,
this.overlayInteraction.start,
this.imageNaturalSize()
)
return
}
if (this.overlayInteraction.kind === 'drawing') {
this.overlayInteraction.rect.endX = point.x
this.overlayInteraction.rect.endY = point.y
return
}
this.updateOverlayCursor(point)
}
private updateOverlayCursor(point: Point) {
if (this.selectedZoneIndex !== null) {
const handle = this.handleAt(point, this.selectedZoneIndex)
if (handle) {
this.overlayCursor = RESIZE_CURSOR[handle]
return
}
}
this.overlayCursor = this.zoneIndexAt(point) !== null ? 'move' : 'crosshair'
}
onOverlayPointerUp(event: PointerEvent) {
if (this.overlayInteraction.kind === 'idle') return
if (this.overlayInteraction.pointerId !== event.pointerId) return
this.onOverlayPointerMove(event)
const interaction = this.overlayInteraction
this.stopOverlayInteraction()
if (interaction.kind !== 'drawing') return
const rect = this.sourceRectFromDrawing(interaction.rect)
// Ignore tiny accidental clicks.
if (rect.w < MIN_ZONE_SIZE || rect.h < MIN_ZONE_SIZE) {
return
}
this.template.zones.push(this.createZoneFromRect(rect))
this.selectZone(this.template.zones.length - 1)
}
private createZoneFromRect(rect: DisplayRect): OcrTemplateZone {
const imageSize = this.imageNaturalSize()
return {
name: `Zone ${this.template.zones.length + 1}`,
target: OcrZoneTarget.CustomField,
custom_field: null,
x: rect.x,
y: rect.y,
width: rect.w,
height: rect.h,
page: this.previewPageDisplay,
ocr_language: DEFAULT_OCR_ZONE_LANGUAGE,
transform: OcrZoneTransform.Strip,
date_format: '',
validation_regex: '',
order: this.template.zones.length,
zone_source_width: imageSize.width,
zone_source_height: imageSize.height,
}
}
onOverlayPointerCancel(event: PointerEvent) {
if (this.overlayInteraction.kind === 'idle') return
if (this.overlayInteraction.pointerId !== event.pointerId) return
this.stopOverlayInteraction()
}
private stopOverlayInteraction() {
const interaction = this.overlayInteraction
this.overlayInteraction = NO_OVERLAY_INTERACTION
this.overlayCursor = 'crosshair'
const overlay = this.overlayRef?.nativeElement
if (
interaction.kind !== 'idle' &&
overlay?.hasPointerCapture(interaction.pointerId)
) {
overlay.releasePointerCapture(interaction.pointerId)
}
}
drawingRect(): DisplayRect | null {
return this.overlayInteraction.kind === 'drawing'
? this.displayRectFromDrawing(this.overlayInteraction.rect)
: null
}
/** The zone's rectangle on the overlay, or null if it's on another page. */
zoneDisplayRect(zoneIndex: number): DisplayRect | null {
const zone = this.template.zones[zoneIndex]
if (!this.isOnCurrentPage(zone)) return null
return getZoneDisplayRect(zone, this.imageNaturalSize())
}
private handleAt(point: Point, zoneIndex: number): ResizeHandle | null {
const rect = this.zoneDisplayRect(zoneIndex)
return rect ? findHandleAt(point, rect, this.overlayHandleSize()) : null
}
private zoneIndexAt(point: Point): number | null {
return findZoneAt(
point,
this.template.zones,
this.previewPage,
this.previewPageCount(),
this.imageNaturalSize()
)
}
overlayViewBox(): string {
const imageSize = this.imageNaturalSize()
return `0 0 ${imageSize.width} ${imageSize.height}`
}
public readonly zoneColor = getZoneColor
zoneLabel(zone: OcrTemplateZone, index: number): string {
return zone.name || `Zone ${index + 1}`
}
zoneLabelY(rect: DisplayRect): number {
return Math.max(this.overlayUnitSize(14), rect.y - this.overlayUnitSize(4))
}
public readonly resizeHandles = getResizeHandles
overlayHandleSize(): number {
return this.overlayUnitSize(HANDLE_SIZE)
}
overlayFontSize(): number {
return this.overlayUnitSize(12)
}
overlayUnitSize(screenPixels: number): number {
return screenPixels * this.overlayScale()
}
private svgPointFromEvent(event: MouseEvent): Point | null {
const svg = this.overlayRef?.nativeElement
const matrix = svg?.getScreenCTM()
if (!svg || !matrix) return null
const point = svg.createSVGPoint()
point.x = event.clientX
point.y = event.clientY
const svgPoint = point.matrixTransform(matrix.inverse())
const size = this.imageNaturalSize()
return {
x: Math.max(0, Math.min(svgPoint.x, size.width)),
y: Math.max(0, Math.min(svgPoint.y, size.height)),
}
}
private displayRectFromDrawing(rect: DrawingRect): DisplayRect {
return {
x: Math.min(rect.startX, rect.endX),
y: Math.min(rect.startY, rect.endY),
w: Math.abs(rect.endX - rect.startX),
h: Math.abs(rect.endY - rect.startY),
}
}
private sourceRectFromDrawing(rect: DrawingRect): DisplayRect {
const displayRect = this.displayRectFromDrawing(rect)
return {
x: Math.round(displayRect.x),
y: Math.round(displayRect.y),
w: Math.round(displayRect.w),
h: Math.round(displayRect.h),
}
}
private imageNaturalSize() {
const img = this.imageRef.nativeElement
return { width: img.naturalWidth, height: img.naturalHeight }
}
removeZone(index: number) {
this.stopOverlayInteraction()
this.clearZoneTest()
this.template.zones.splice(index, 1)
if (this.selectedZoneIndex === index) {
this.selectedZoneIndex = null
} else if (this.selectedZoneIndex > index) {
this.selectedZoneIndex--
}
}
selectZone(index: number) {
this.selectedZoneIndex = index
this.activeTab = 'zones'
this.clearZoneTest()
const zone = this.template.zones[index]
if (zone) {
this.goToPage(this.zonePage(zone) - 1)
}
}
deselectZone() {
this.selectedZoneIndex = null
this.clearZoneTest()
}
testZone() {
const zone = this.selectedZone
if (!zone || !this.previewDocId) return
this.clearZoneTest()
const test: ZoneTest = { zone, key: this.zoneTestKey(zone), result: null }
this.zoneTest.set(test)
this.templateService
.testZone(this.previewDocId, this.zoneTestRequest(zone))
.pipe(takeUntil(this.destroy$), takeUntil(this.cancelZoneTest$))
.subscribe({
next: (res) => this.zoneTest.set({ ...test, result: res }),
error: (err) =>
this.zoneTest.set({
...test,
result: { error: err.error?.error || $localize`Test failed` },
}),
})
}
private clearZoneTest() {
this.cancelZoneTest$.next()
this.zoneTest.set(null)
}
/** A result is stale once its zone, settings or sample document change. */
private isCurrentZoneTest(): boolean {
const test = this.zoneTest()
return (
test?.zone === this.selectedZone &&
test.key === this.zoneTestKey(test.zone)
)
}
private zoneTestKey(zone: OcrTemplateZone): string {
return JSON.stringify([this.previewDocId, this.zoneTestRequest(zone)])
}
private zoneTestRequest(zone: OcrTemplateZone): ZoneTestRequest {
return {
name: zone.name,
x: zone.x,
y: zone.y,
width: zone.width,
height: zone.height,
page: zone.page ?? 1,
ocr_language: zone.ocr_language,
transform: zone.transform,
date_format: zone.date_format,
validation_regex: zone.validation_regex,
zone_source_width: zone.zone_source_width,
zone_source_height: zone.zone_source_height,
}
}
save() {
if (this.saving()) return
this.stopOverlayInteraction()
this.saving.set(true)
this.pruneCombineFormats()
this.template.sample_document = this.previewDocId
const payload = this.permissionsService.currentUserOwnsObject(this.object)
? { ...this.template, ...this.permissionsForm }
: this.template
const obs = this.template.id
? this.templateService.update(payload)
: this.templateService.create(payload)
obs.pipe(takeUntil(this.destroy$)).subscribe({
next: (saved) => {
this.activeModal.close()
this.succeeded.emit(saved)
},
error: (e) => {
this.saving.set(false)
this.failed.emit(e)
},
})
}
// ng-select needs a stable array between change detection runs, so the
// split language list is cached per zone until ocr_language changes.
private readonly ocrLanguageCache = new WeakMap<
OcrTemplateZone,
{ language: string; languages: string[] }
>()
ocrLanguageArray(zone: OcrTemplateZone): string[] {
const language = zone.ocr_language || ''
const cached = this.ocrLanguageCache.get(zone)
if (cached?.language === language) return cached.languages
const languages = language.split('+').filter(Boolean)
this.ocrLanguageCache.set(zone, { language, languages })
return languages
}
setOcrLanguages(zone: OcrTemplateZone, langs: string[]) {
zone.ocr_language = (langs ?? []).join('+')
}
/** The field select's value: a built-in target, or a custom field id. */
zoneFieldValue(zone: OcrTemplateZone): ZoneFieldSelection {
return zone.target === OcrZoneTarget.CustomField
? zone.custom_field
: zone.target
}
setZoneField(zone: OcrTemplateZone, value: ZoneFieldSelection) {
if (typeof value === 'string') {
zone.target = value
zone.custom_field = null
} else {
zone.target = OcrZoneTarget.CustomField
zone.custom_field = value
}
}
zonesForField(zone: OcrTemplateZone): OcrTemplateZone[] {
const key = getZoneFieldKey(zone)
if (!key) return []
return this.template.zones.filter((z) => getZoneFieldKey(z) === key)
}
isFieldShared(zone: OcrTemplateZone): boolean {
return this.zonesForField(zone).length > 1
}
renameZone(zone: OcrTemplateZone, name: string) {
const oldName = zone.name.trim()
const format = this.getCombineFormat(zone)
if (format) {
this.setCombineFormat(
zone,
format.replace(/\{([^{}]*)\}/g, (token, tokenName: string) =>
tokenName.trim() === oldName ? `{${name}}` : token
)
)
}
zone.name = name
}
getCombineFormat(zone: OcrTemplateZone): string {
return this.template.combine_formats?.[getZoneFieldKey(zone)] ?? ''
}
setCombineFormat(zone: OcrTemplateZone, value: string) {
const key = getZoneFieldKey(zone)
if (!key) return
this.template.combine_formats ??= {}
this.template.combine_formats[key] = value
}
defaultCombineFormat(zone: OcrTemplateZone): string {
return this.zonesForField(zone)
.map((z) => `{${z.name}}`)
.join(' ')
}
insertCombineToken(zone: OcrTemplateZone, tokenZone: OcrTemplateZone) {
const current = this.getCombineFormat(zone)
const sep = current && !current.endsWith(' ') ? ' ' : ''
this.setCombineFormat(zone, `${current}${sep}{${tokenZone.name}}`)
}
/** Drop formats that are empty or whose field no longer has several zones. */
private pruneCombineFormats() {
const formats = this.template.combine_formats
if (!formats) return
const keys = this.template.zones.map(getZoneFieldKey)
for (const [key, format] of Object.entries(formats)) {
if (!format || keys.filter((k) => k === key).length < 2) {
delete formats[key]
}
}
}
createField(name?: string) {
const zone = this.selectedZone
const modal = this.modalService.open(CustomFieldEditDialogComponent)
modal.componentInstance.dialogMode.set(EditDialogMode.CREATE)
modal.componentInstance.object = { name: name || zone?.name }
modal.componentInstance.succeeded
.pipe(takeUntil(this.destroy$))
.subscribe((field: CustomField) => {
this.toastService.showSuccess($localize`Saved field "${field.name}".`)
this.customFieldsService.clearCache()
this.customFieldsService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => {
if (zone && OCR_SUPPORTED_FIELD_TYPES.includes(field.data_type)) {
this.setZoneField(zone, field.id)
}
this.customFields.set(r.results)
})
})
modal.componentInstance.failed
.pipe(takeUntil(this.destroy$))
.subscribe((e) => {
this.toastService.showError($localize`Error saving field.`, e)
})
}
ngOnDestroy() {
this.previewResizeObserver?.disconnect()
this.cancelZoneTest$.complete()
this.destroy$.next()
this.destroy$.complete()
}
}
@@ -0,0 +1,197 @@
import {
OcrTemplateZone,
OcrZoneTarget,
OcrZoneTransform,
} from 'src/app/data/ocr-template'
import {
findHandleAt,
findZoneAt,
getResizeHandles,
getZoneDisplayRect,
getZonePage,
isZoneOnPage,
moveZone,
resizeZone,
} from './zone-geometry'
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
return {
name: 'Zone',
target: OcrZoneTarget.CustomField,
custom_field: 1,
x: 100,
y: 200,
width: 300,
height: 400,
page: 1,
ocr_language: 'eng',
transform: OcrZoneTransform.Strip,
validation_regex: '',
order: 0,
...overrides,
}
}
describe('OCR template editor geometry', () => {
it('normalizes zone pages', () => {
expect(getZonePage(zone({ page: 2 }), 0, 5)).toBe(2)
expect(getZonePage(zone({ page: -1 }), 0, 5)).toBe(5)
expect(getZonePage(zone({ page: -1 }), 2, null)).toBe(3)
expect(getZonePage(zone({ page: 0 }), 0, 5)).toBe(1)
expect(getZonePage(zone({ page: undefined }), 0, 5)).toBe(1)
})
it('checks whether a zone is on the current preview page', () => {
expect(isZoneOnPage(zone({ page: 2 }), 1, 5)).toBe(true)
expect(isZoneOnPage(zone({ page: 2 }), 0, 5)).toBe(false)
expect(isZoneOnPage(zone({ page: -1 }), 4, 5)).toBe(true)
})
it('uses natural image coordinates when source dimensions are unset', () => {
expect(
getZoneDisplayRect(zone({ x: 100, y: 200, width: 300, height: 400 }), {
width: 1000,
height: 2000,
})
).toEqual({ x: 100, y: 200, w: 300, h: 400 })
})
it('uses per-zone source dimensions when present', () => {
expect(
getZoneDisplayRect(
zone({
x: 100,
y: 100,
width: 100,
height: 100,
zone_source_width: 1000,
zone_source_height: 1000,
}),
{ width: 500, height: 750 }
)
).toEqual({ x: 50, y: 75, w: 50, h: 75 })
})
it('finds zones from topmost to bottommost on the current page', () => {
const zones = [
zone({ name: 'first', x: 0, y: 0, width: 100, height: 100, page: 1 }),
zone({ name: 'second', x: 0, y: 0, width: 50, height: 50, page: 1 }),
zone({ name: 'third', x: 0, y: 0, width: 50, height: 50, page: 2 }),
]
expect(
findZoneAt({ x: 25, y: 25 }, zones, 0, 2, { width: 100, height: 100 })
).toBe(1)
})
it('finds resize handles around a display rect', () => {
const rect = { x: 10, y: 20, w: 100, h: 200 }
expect(findHandleAt({ x: 10, y: 20 }, rect)).toBe('nw')
expect(findHandleAt({ x: 110, y: 220 }, rect)).toBe('se')
expect(findHandleAt({ x: 60, y: 20 }, rect)).toBe('n')
expect(findHandleAt({ x: 90, y: 160 }, rect)).toBeNull()
expect(findHandleAt({ x: 19, y: 20 }, rect, 8)).toBeNull()
expect(findHandleAt({ x: 19, y: 20 }, rect, 10)).toBe('nw')
})
it('returns no zone when the point is outside every rectangle', () => {
expect(
findZoneAt({ x: 900, y: 900 }, [zone()], 0, 1, {
width: 1000,
height: 1000,
})
).toBeNull()
})
it('resizes north-west edges while keeping the opposite corner fixed', () => {
const z = zone({ x: 100, y: 200, width: 300, height: 400 })
resizeZone(z, 'nw', { x: 50, y: 100 }, { width: 1000, height: 1000 })
expect(z).toMatchObject({ x: 50, y: 100, width: 350, height: 500 })
resizeZone(z, 'nw', { x: 900, y: 900 }, { width: 1000, height: 1000 })
expect(z).toMatchObject({ x: 390, y: 590, width: 10, height: 10 })
})
it('clamps resized north-west edges to the source image bounds', () => {
const z = zone({
x: 50,
y: 50,
width: 100,
height: 100,
zone_source_width: 200,
zone_source_height: 300,
})
resizeZone(z, 'nw', { x: -50, y: -50 }, { width: 400, height: 600 })
expect(z).toMatchObject({ x: 0, y: 0, width: 150, height: 150 })
})
it('uses the rendered handle positions for hit testing', () => {
const rect = { x: 10, y: 20, w: 100, h: 200 }
const handles = getResizeHandles(rect)
expect(handles).toEqual([
{ handle: 'nw', x: 10, y: 20 },
{ handle: 'n', x: 60, y: 20 },
{ handle: 'ne', x: 110, y: 20 },
{ handle: 'w', x: 10, y: 120 },
{ handle: 'e', x: 110, y: 120 },
{ handle: 'sw', x: 10, y: 220 },
{ handle: 's', x: 60, y: 220 },
{ handle: 'se', x: 110, y: 220 },
])
for (const marker of handles) {
expect(findHandleAt(marker, rect)).toBe(marker.handle)
}
})
it('moves zones without leaving source image bounds', () => {
const z = zone({ x: 50, y: 50, width: 100, height: 100 })
moveZone(
z,
{ x: 500, y: 500 },
{ mouseX: 50, mouseY: 50, zoneX: 50, zoneY: 50 },
{ width: 500, height: 500 }
)
expect(z.x).toBe(400)
expect(z.y).toBe(400)
})
it('resizes zones without leaving source image bounds', () => {
const z = zone({
x: 50,
y: 50,
width: 100,
height: 100,
zone_source_width: 200,
zone_source_height: 200,
})
resizeZone(z, 'se', { x: 500, y: 500 }, { width: 500, height: 500 })
expect(z.width).toBe(150)
expect(z.height).toBe(150)
})
it('moves zones in their original source coordinates on a different-size preview', () => {
const z = zone({
x: 50,
y: 50,
width: 100,
height: 100,
zone_source_width: 1000,
zone_source_height: 2000,
})
moveZone(
z,
{ x: 75, y: 75 },
{ mouseX: 25, mouseY: 25, zoneX: 50, zoneY: 50 },
{ width: 500, height: 500 }
)
expect(z.x).toBe(150)
expect(z.y).toBe(250)
})
})
@@ -0,0 +1,204 @@
import { OcrTemplateZone } from 'src/app/data/ocr-template'
export interface DrawingRect {
startX: number
startY: number
endX: number
endY: number
}
export interface Dimensions {
width: number
height: number
}
export interface Point {
x: number
y: number
}
export interface DisplayRect {
x: number
y: number
w: number
h: number
}
export interface MoveStart {
mouseX: number
mouseY: number
zoneX: number
zoneY: number
}
export type ResizeHandle = 'n' | 's' | 'e' | 'w' | 'ne' | 'nw' | 'se' | 'sw'
export interface ResizeHandleMarker extends Point {
handle: ResizeHandle
}
export const HANDLE_SIZE = 8
export const MIN_ZONE_SIZE = 10
const ZONE_COLORS = [
'#4f8ff7',
'#ff6b6b',
'#51cf66',
'#ffd43b',
'#cc5de8',
'#ff922b',
'#20c997',
'#e599f7',
]
/** Outline color for a zone, shared by the preview overlay and the zone list. */
export function getZoneColor(index: number): string {
return ZONE_COLORS[index % ZONE_COLORS.length]
}
export function getZonePage(
zone: OcrTemplateZone,
previewPage: number,
previewPageCount: number | null
): number {
const page = zone.page ?? 1
if (page === -1) return previewPageCount ?? previewPage + 1
return Math.max(page, 1)
}
export function isZoneOnPage(
zone: OcrTemplateZone,
previewPage: number,
previewPageCount: number | null
): boolean {
return getZonePage(zone, previewPage, previewPageCount) === previewPage + 1
}
function getZoneSourceSize(
zone: OcrTemplateZone,
imageSize: Dimensions
): Dimensions {
return {
width: zone.zone_source_width || imageSize.width,
height: zone.zone_source_height || imageSize.height,
}
}
export function getZoneDisplayRect(
zone: OcrTemplateZone,
imageSize: Dimensions
): DisplayRect {
const sourceSize = getZoneSourceSize(zone, imageSize)
// The SVG viewBox uses the preview image's natural dimensions.
const scaleX = imageSize.width / sourceSize.width
const scaleY = imageSize.height / sourceSize.height
return {
x: zone.x * scaleX,
y: zone.y * scaleY,
w: zone.width * scaleX,
h: zone.height * scaleY,
}
}
export function getResizeHandles(rect: DisplayRect): ResizeHandleMarker[] {
return [
{ handle: 'nw', x: rect.x, y: rect.y },
{ handle: 'n', x: rect.x + rect.w / 2, y: rect.y },
{ handle: 'ne', x: rect.x + rect.w, y: rect.y },
{ handle: 'w', x: rect.x, y: rect.y + rect.h / 2 },
{ handle: 'e', x: rect.x + rect.w, y: rect.y + rect.h / 2 },
{ handle: 'sw', x: rect.x, y: rect.y + rect.h },
{ handle: 's', x: rect.x + rect.w / 2, y: rect.y + rect.h },
{ handle: 'se', x: rect.x + rect.w, y: rect.y + rect.h },
]
}
export function findHandleAt(
point: Point,
rect: DisplayRect,
handleSize = HANDLE_SIZE
): ResizeHandle | null {
return (
getResizeHandles(rect).find(
({ x, y }) =>
Math.abs(point.x - x) <= handleSize &&
Math.abs(point.y - y) <= handleSize
)?.handle ?? null
)
}
export function findZoneAt(
point: Point,
zones: OcrTemplateZone[],
previewPage: number,
previewPageCount: number | null,
imageSize: Dimensions
): number | null {
for (let i = zones.length - 1; i >= 0; i--) {
const zone = zones[i]
if (!isZoneOnPage(zone, previewPage, previewPageCount)) continue
const rect = getZoneDisplayRect(zone, imageSize)
if (
point.x >= rect.x &&
point.x <= rect.x + rect.w &&
point.y >= rect.y &&
point.y <= rect.y + rect.h
) {
return i
}
}
return null
}
export function moveZone(
zone: OcrTemplateZone,
point: Point,
moveStart: MoveStart,
imageSize: Dimensions
) {
const sourceSize = getZoneSourceSize(zone, imageSize)
const scaleX = sourceSize.width / imageSize.width
const scaleY = sourceSize.height / imageSize.height
const dx = Math.round((point.x - moveStart.mouseX) * scaleX)
const dy = Math.round((point.y - moveStart.mouseY) * scaleY)
zone.x = clamp(moveStart.zoneX + dx, 0, sourceSize.width - zone.width)
zone.y = clamp(moveStart.zoneY + dy, 0, sourceSize.height - zone.height)
}
export function resizeZone(
zone: OcrTemplateZone,
handle: ResizeHandle,
point: Point,
imageSize: Dimensions
) {
const sourceSize = getZoneSourceSize(zone, imageSize)
const scaleX = sourceSize.width / imageSize.width
const scaleY = sourceSize.height / imageSize.height
const imageX = clamp(Math.round(point.x * scaleX), 0, sourceSize.width)
const imageY = clamp(Math.round(point.y * scaleY), 0, sourceSize.height)
if (handle.includes('w')) {
const right = Math.min(zone.x + zone.width, sourceSize.width)
zone.x = clamp(imageX, 0, right - MIN_ZONE_SIZE)
zone.width = right - zone.x
}
if (handle.includes('e')) {
zone.width = Math.max(MIN_ZONE_SIZE, imageX - zone.x)
}
if (handle.includes('n')) {
const bottom = Math.min(zone.y + zone.height, sourceSize.height)
zone.y = clamp(imageY, 0, bottom - MIN_ZONE_SIZE)
zone.height = bottom - zone.y
}
if (handle.includes('s')) {
zone.height = Math.max(MIN_ZONE_SIZE, imageY - zone.y)
}
}
function clamp(value: number, min: number, max: number): number {
return Math.max(min, Math.min(value, max))
}
@@ -0,0 +1,75 @@
<pngx-page-header
title="OCR Templates"
i18n-title
info="Define extraction zones on document types to automatically populate custom fields via OCR."
i18n-info
>
<button type="button" class="btn btn-sm btn-outline-primary" (click)="editTemplate()" *pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.OcrTemplate }">
<i-bs name="plus-circle" class="me-1"></i-bs><ng-container i18n>Add Template</ng-container>
</button>
</pngx-page-header>
<ul class="list-group">
<li class="list-group-item">
<div class="row">
<div class="col" i18n>Name</div>
<div class="col d-none d-sm-flex" i18n>Matching</div>
<div class="col d-none d-sm-flex" i18n>Zones</div>
<div class="col" i18n>Status</div>
<div class="col" i18n>Actions</div>
</div>
</li>
@if (loading() && templates().length === 0) {
<li class="list-group-item">
<output class="spinner-border spinner-border-sm me-2"></output>
<ng-container i18n>Loading...</ng-container>
</li>
}
@for (template of templates(); track template.id) {
<li class="list-group-item">
<div class="row fade" [class.show]="show()">
<div class="col d-flex align-items-center"><button class="btn btn-link p-0 text-start" type="button" (click)="editTemplate(template)" [disabled]="!userCanEdit(template)">{{template.name}}</button></div>
<div class="col d-flex align-items-center d-none d-sm-flex">{{getMatching(template)}}</div>
<div class="col d-flex align-items-center d-none d-sm-flex"><code>{{template.zones.length}}</code></div>
<div class="col d-flex align-items-center">
<div class="form-check form-switch mb-0">
<input type="checkbox" class="form-check-input cursor-pointer" [id]="template.id+'_enable'" [(ngModel)]="template.enabled" (change)="toggleTemplateEnabled(template)" [disabled]="!userCanEdit(template)" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }">
<label class="form-check-label cursor-pointer" [for]="template.id+'_enable'">
<code> @if(template.enabled) { <ng-container i18n>Enabled</ng-container> } @else { <span i18n class="text-muted">Disabled</span> }</code>
</label>
</div>
</div>
<div class="col">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile{{template.id}}" (click)="$event.stopPropagation()" ngbDropdownToggle>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile{{template.id}}">
<button (click)="editTemplate(template)" [disabled]="!userCanEdit(template)" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }" ngbDropdownItem i18n>Edit</button>
<button (click)="deleteTemplate(template)" [disabled]="!permissionsService.currentUserOwnsObject(template)" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.OcrTemplate }" ngbDropdownItem i18n>Delete</button>
</div>
</div>
</div>
<div class="btn-toolbar d-none d-sm-flex gap-2" role="toolbar">
<div class="btn-group">
<button *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }" [disabled]="!userCanEdit(template)" class="btn btn-sm btn-outline-secondary" type="button" (click)="editTemplate(template)">
<i-bs width="1em" height="1em" name="pencil" class="me-1"></i-bs><ng-container i18n>Edit</ng-container>
</button>
<button *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.OcrTemplate }" [disabled]="!permissionsService.currentUserOwnsObject(template)" class="btn btn-sm btn-outline-danger" type="button" (click)="deleteTemplate(template)">
<i-bs width="1em" height="1em" name="trash" class="me-1"></i-bs><ng-container i18n>Delete</ng-container>
</button>
</div>
</div>
</div>
</div>
</li>
}
@if (!loading() && templates().length === 0) {
<li class="list-group-item" [class.show]="show()" i18n>No OCR templates defined.</li>
}
</ul>
@@ -0,0 +1,300 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
import { NgbModal, NgbModalRef } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { MATCH_LITERAL, MATCH_NONE } from 'src/app/data/matching-model'
import { OcrTemplate } from 'src/app/data/ocr-template'
import { User } from 'src/app/data/user'
import { DirtyFormGuard } from 'src/app/guards/dirty-form.guard'
import { PermissionsService } from 'src/app/services/permissions.service'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { UserService } from 'src/app/services/rest/user.service'
import { ToastService } from 'src/app/services/toast.service'
import { ConfirmDialogComponent } from '../../common/confirm-dialog/confirm-dialog.component'
import { OcrTemplateEditorComponent } from './ocr-template-editor/ocr-template-editor.component'
import { OcrTemplatesComponent } from './ocr-templates.component'
const templates: OcrTemplate[] = [
{
id: 1,
name: 'Invoices',
match: 'Northwind',
matching_algorithm: MATCH_LITERAL,
sample_document: null,
source_width: 0,
source_height: 0,
enabled: true,
zones: [],
},
]
describe('OcrTemplatesComponent', () => {
let component: OcrTemplatesComponent
let fixture: ComponentFixture<OcrTemplatesComponent>
let ocrTemplateService: OcrTemplateService
let modalService: NgbModal
let toastService: ToastService
let modal: NgbModalRef
beforeEach(() => {
TestBed.configureTestingModule({
imports: [NgxBootstrapIconsModule.pick(allIcons), OcrTemplatesComponent],
providers: [
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
})
TestBed.inject(PermissionsService).initialize(
[
'view_ocrtemplate',
'add_ocrtemplate',
'change_ocrtemplate',
'delete_ocrtemplate',
],
{ id: 1, groups: [], is_superuser: false } as User
)
jest
.spyOn(TestBed.inject(UserService), 'listAll')
.mockReturnValue(of({ count: 0, all: [], results: [] }))
ocrTemplateService = TestBed.inject(OcrTemplateService)
jest
.spyOn(ocrTemplateService, 'listAll')
.mockReturnValue(of({ count: 1, all: [1], results: templates }))
modalService = TestBed.inject(NgbModal)
modalService.activeInstances.subscribe((m) => (modal = m[m.length - 1]))
toastService = TestBed.inject(ToastService)
fixture = TestBed.createComponent(OcrTemplatesComponent)
component = fixture.componentInstance
fixture.detectChanges()
})
afterEach(() => modalService.dismissAll())
it('should show templates with their matching', () => {
const text = fixture.nativeElement.textContent
expect(text).toContain('Invoices')
expect(text).toContain('Exact match: Northwind')
expect(
component.getMatching({ ...templates[0], matching_algorithm: MATCH_NONE })
).toEqual('None')
expect(component.getMatching({ ...templates[0], match: '' })).toEqual('-')
})
it.each([
{
name: 'owned',
owner: 1,
user_can_change: true,
editable: true,
deletable: true,
},
{
name: 'unowned',
owner: null,
user_can_change: true,
editable: true,
deletable: true,
},
{
name: 'view-only shared',
owner: 2,
user_can_change: false,
editable: false,
deletable: false,
},
{
name: 'editable shared',
owner: 2,
user_can_change: true,
editable: true,
deletable: false,
},
])(
'should respect permissions for $name templates',
async ({ owner, user_can_change, editable, deletable }) => {
const template = { ...templates[0], owner, user_can_change }
component.templates.set([template])
fixture.detectChanges()
await fixture.whenStable()
expect(fixture.nativeElement.querySelector('.text-start').disabled).toBe(
!editable
)
expect(
fixture.nativeElement.querySelector('.form-switch input').disabled
).toBe(!editable)
const buttons = Array.from<HTMLButtonElement>(
fixture.nativeElement.querySelectorAll('button')
)
for (const button of buttons.filter(
(button) => button.textContent.trim() === 'Edit'
)) {
expect(button.disabled).toBe(!editable)
}
for (const button of buttons.filter(
(button) => button.textContent.trim() === 'Delete'
)) {
expect(button.disabled).toBe(!deletable)
}
}
)
it('should allow a superuser to manage another users template', () => {
TestBed.inject(PermissionsService).initialize([], {
id: 1,
is_superuser: true,
} as User)
component.templates.set([
{ ...templates[0], owner: 2, user_can_change: false },
])
fixture.detectChanges()
expect(fixture.nativeElement.querySelector('.text-start').disabled).toBe(
false
)
expect(
fixture.nativeElement.querySelector('.btn-outline-danger').disabled
).toBe(false)
})
it('should not open mutation dialogs or patch a view-only template', () => {
const template = { ...templates[0], owner: 2, user_can_change: false }
const open = jest.spyOn(modalService, 'open')
const patch = jest.spyOn(ocrTemplateService, 'patch')
component.editTemplate(template)
component.toggleTemplateEnabled(template)
component.deleteTemplate(template)
expect(open).not.toHaveBeenCalled()
expect(patch).not.toHaveBeenCalled()
})
it('should require global edit permission even for an owned template', () => {
TestBed.inject(PermissionsService).initialize(['view_ocrtemplate'], {
id: 1,
} as User)
fixture.destroy()
fixture = TestBed.createComponent(OcrTemplatesComponent)
component = fixture.componentInstance
fixture.detectChanges()
expect(fixture.nativeElement.querySelector('.text-start').disabled).toBe(
true
)
expect(fixture.nativeElement.querySelector('.form-switch input')).toBeNull()
})
it('should support create, show notification on error / success', () => {
const toastErrorSpy = jest.spyOn(toastService, 'showError')
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
const reloadSpy = jest.spyOn(component, 'reload')
fixture.debugElement
.queryAll(By.css('pngx-page-header button'))
.find((button) => button.nativeElement.textContent.includes('Add'))
.triggerEventHandler('click')
const editor = modal.componentInstance as OcrTemplateEditorComponent
expect(editor).toBeInstanceOf(OcrTemplateEditorComponent)
expect(editor.object).toBeUndefined()
editor.failed.emit({ error: 'error creating template' })
expect(toastErrorSpy).toHaveBeenCalled()
expect(reloadSpy).not.toHaveBeenCalled()
editor.succeeded.emit(templates[0])
expect(toastSuccessSpy).toHaveBeenCalled()
expect(reloadSpy).toHaveBeenCalled()
})
it('should pass the template to the editor', () => {
component.editTemplate(templates[0])
expect(modal.componentInstance.object).toBe(templates[0])
})
it('should confirm closing the editor with unsaved changes', async () => {
const confirmSpy = jest
.spyOn(TestBed.inject(DirtyFormGuard), 'confirmChanges')
.mockReturnValue(of(false))
component.editTemplate()
const editor = modal.componentInstance as OcrTemplateEditorComponent
editor.ngOnInit()
modal.dismiss()
await fixture.whenStable()
expect(confirmSpy).not.toHaveBeenCalled()
component.editTemplate()
const dirtyEditor = modal.componentInstance as OcrTemplateEditorComponent
dirtyEditor.ngOnInit()
dirtyEditor.template.name = 'Changed'
modal.dismiss()
await fixture.whenStable()
expect(confirmSpy).toHaveBeenCalled()
expect(modalService.hasOpenModals()).toBe(true)
dirtyEditor.saving.set(true)
confirmSpy.mockClear()
modal.dismiss()
await fixture.whenStable()
expect(confirmSpy).not.toHaveBeenCalled()
expect(modalService.hasOpenModals()).toBe(true)
})
it('should only send the enabled flag when toggling', () => {
const patchSpy = jest
.spyOn(ocrTemplateService, 'patch')
.mockReturnValueOnce(throwError(() => new Error('error toggling')))
.mockReturnValueOnce(of(templates[0]))
const toastErrorSpy = jest.spyOn(toastService, 'showError')
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
const reloadSpy = jest.spyOn(component, 'reload')
const toggle = fixture.debugElement.query(By.css('.form-switch input'))
toggle.nativeElement.click()
expect(patchSpy).toHaveBeenCalledWith({ id: 1, enabled: false })
expect(toastErrorSpy).toHaveBeenCalled()
expect(reloadSpy).toHaveBeenCalledTimes(1)
component.toggleTemplateEnabled(templates[0])
expect(toastSuccessSpy).toHaveBeenCalled()
expect(reloadSpy).toHaveBeenCalledTimes(2)
})
it('should support delete, show notification on error / success', () => {
const toastErrorSpy = jest.spyOn(toastService, 'showError')
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
const deleteSpy = jest.spyOn(ocrTemplateService, 'delete')
const reloadSpy = jest.spyOn(component, 'reload')
component.deleteTemplate(templates[0])
const dialog = modal.componentInstance as ConfirmDialogComponent
deleteSpy.mockReturnValueOnce(throwError(() => new Error('error deleting')))
dialog.confirmClicked.emit()
expect(toastErrorSpy).toHaveBeenCalled()
expect(dialog.buttonsEnabled()).toBe(true)
expect(reloadSpy).not.toHaveBeenCalled()
deleteSpy.mockReturnValueOnce(of(true))
dialog.confirmClicked.emit()
expect(toastSuccessSpy).toHaveBeenCalled()
expect(reloadSpy).toHaveBeenCalled()
})
it.each([true, false])(
'reports the enabled state %s after the update succeeds',
(enabled) => {
const template = { ...templates[0], enabled }
jest.spyOn(ocrTemplateService, 'patch').mockReturnValue(of(template))
const toast = jest.spyOn(toastService, 'showSuccess')
component.toggleTemplateEnabled(template)
expect(toast).toHaveBeenCalledWith(
`${enabled ? 'Enabled' : 'Disabled'} OCR template "${template.name}"`
)
}
)
})
@@ -0,0 +1,185 @@
import { Component, OnInit, inject, signal } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { NgbDropdownModule, NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { firstValueFrom, takeUntil, tap } from 'rxjs'
import { MATCH_NONE, MATCHING_ALGORITHMS } from 'src/app/data/matching-model'
import { OcrTemplate } from 'src/app/data/ocr-template'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { DirtyFormGuard } from 'src/app/guards/dirty-form.guard'
import {
PermissionAction,
PermissionType,
PermissionsService,
} from 'src/app/services/permissions.service'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { ToastService } from 'src/app/services/toast.service'
import { ConfirmDialogComponent } from '../../common/confirm-dialog/confirm-dialog.component'
import { PageHeaderComponent } from '../../common/page-header/page-header.component'
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
import { OcrTemplateEditorComponent } from './ocr-template-editor/ocr-template-editor.component'
@Component({
selector: 'pngx-ocr-templates',
templateUrl: './ocr-templates.component.html',
imports: [
PageHeaderComponent,
IfPermissionsDirective,
FormsModule,
NgbDropdownModule,
NgxBootstrapIconsModule,
],
})
export class OcrTemplatesComponent
extends LoadingComponentWithPermissions
implements OnInit
{
private readonly ocrTemplateService = inject(OcrTemplateService)
public readonly permissionsService = inject(PermissionsService)
private readonly modalService = inject(NgbModal)
private readonly dirtyFormGuard = inject(DirtyFormGuard)
private readonly toastService = inject(ToastService)
public readonly templates = signal<OcrTemplate[]>([])
ngOnInit() {
this.reload()
}
reload() {
this.loading.set(true)
this.ocrTemplateService
.listAll()
.pipe(
takeUntil(this.unsubscribeNotifier),
tap((r) => this.templates.set(r.results))
)
.subscribe(() => {
this.show.set(true)
this.loading.set(false)
})
}
getMatching(template: OcrTemplate): string {
if (template.matching_algorithm === MATCH_NONE) {
return $localize`None`
}
if (!template.match) {
return '-'
}
const algorithm = MATCHING_ALGORITHMS.find(
(a) => a.id === template.matching_algorithm
)
return `${algorithm?.shortName}: ${template.match}`
}
editTemplate(template?: OcrTemplate) {
if (template && !this.userCanEdit(template)) return
const modal = this.modalService.open(OcrTemplateEditorComponent, {
backdrop: 'static',
keyboard: false,
size: 'xxl',
fullscreen: 'lg',
scrollable: true,
ariaLabelledBy: 'ocr-template-editor-title',
beforeDismiss: () => {
const editor: OcrTemplateEditorComponent = modal.componentInstance
if (editor.saving()) return false
if (!editor.isDirty) return true
return firstValueFrom(
this.dirtyFormGuard.confirmChanges(undefined, $localize`Close`)
)
},
})
modal.componentInstance.object = template
modal.componentInstance.succeeded
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((saved: OcrTemplate) => {
this.toastService.showSuccess(
$localize`Saved OCR template "${saved.name}".`
)
this.ocrTemplateService.clearCache()
this.reload()
})
modal.componentInstance.failed
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((e) => {
this.toastService.showError($localize`Error saving OCR template.`, e)
})
}
toggleTemplateEnabled(template: OcrTemplate) {
if (!this.userCanEdit(template)) return
this.ocrTemplateService
.patch({ id: template.id, enabled: template.enabled } as OcrTemplate)
.subscribe({
next: () => {
this.toastService.showSuccess(
template.enabled
? $localize`Enabled OCR template "${template.name}"`
: $localize`Disabled OCR template "${template.name}"`
)
this.reload()
},
error: (e) => {
this.toastService.showError(
$localize`Error toggling OCR template "${template.name}".`,
e
)
this.reload()
},
})
}
deleteTemplate(template: OcrTemplate) {
if (
!this.permissionsService.currentUserCan(
PermissionAction.Delete,
PermissionType.OcrTemplate
) ||
!this.permissionsService.currentUserOwnsObject(template)
)
return
const modal = this.modalService.open(ConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Confirm delete OCR template`
modal.componentInstance.messageBold = $localize`This operation will permanently delete this OCR template.`
modal.componentInstance.message = $localize`This operation cannot be undone.`
modal.componentInstance.btnClass = 'btn-danger'
modal.componentInstance.btnCaption = $localize`Proceed`
modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled.set(false)
this.ocrTemplateService.delete(template).subscribe({
next: () => {
modal.close()
this.toastService.showSuccess(
$localize`Deleted OCR template "${template.name}".`
)
this.ocrTemplateService.clearCache()
this.reload()
},
error: (e) => {
modal.componentInstance.buttonsEnabled.set(true)
this.toastService.showError(
$localize`Error deleting OCR template "${template.name}".`,
e
)
},
})
})
}
userCanEdit(template: OcrTemplate): boolean {
return (
this.permissionsService.currentUserCan(
PermissionAction.Change,
PermissionType.OcrTemplate
) &&
this.permissionsService.currentUserHasObjectPermissions(
PermissionAction.Change,
template
)
)
}
}
+115
View File
@@ -0,0 +1,115 @@
import { CustomFieldDataType } from './custom-field'
import { MatchingModel } from './matching-model'
import { ObjectWithPermissions } from './object-with-permissions'
export enum OcrZoneTarget {
CustomField = 'custom_field',
Title = 'title',
Asn = 'asn',
Created = 'created',
}
export enum OcrZoneTransform {
None = 'none',
Strip = 'strip',
Uppercase = 'uppercase',
Lowercase = 'lowercase',
Numeric = 'numeric',
StripPunctuation = 'strip_punctuation',
Date = 'date',
QrCode = 'qr_code',
}
export const DEFAULT_OCR_ZONE_LANGUAGE = 'eng'
/** The field a zone writes to, keyed by OcrTemplate.combine_formats. */
export function getZoneFieldKey(zone: OcrTemplateZone): string | null {
if (zone.target !== OcrZoneTarget.CustomField) return zone.target
return zone.custom_field === null ? null : String(zone.custom_field)
}
/** Custom field types zone OCR can write to. See models.py OCR_SUPPORTED_FIELD_TYPES. */
export const OCR_SUPPORTED_FIELD_TYPES: CustomFieldDataType[] = [
CustomFieldDataType.String,
CustomFieldDataType.Url,
CustomFieldDataType.Date,
CustomFieldDataType.Integer,
CustomFieldDataType.Float,
CustomFieldDataType.Monetary,
CustomFieldDataType.LongText,
CustomFieldDataType.Boolean,
]
export const OCR_BUILTIN_TARGETS = [
{ id: OcrZoneTarget.Title, name: $localize`Title` },
{ id: OcrZoneTarget.Asn, name: $localize`Archive serial number` },
{ id: OcrZoneTarget.Created, name: $localize`Date created` },
]
export interface OcrTemplateZone {
id?: number
name: string
target: OcrZoneTarget
custom_field: number | null
page?: number
x: number
y: number
width: number
height: number
ocr_language: string
transform: OcrZoneTransform
date_format?: string
validation_regex: string
order: number
zone_source_width?: number
zone_source_height?: number
}
export interface OcrTemplate extends MatchingModel, ObjectWithPermissions {
name: string
sample_document: number | null
source_width: number
source_height: number
enabled: boolean
combine_formats?: Record<string, string>
created?: string
updated?: string
zones: OcrTemplateZone[]
}
export interface ZoneTestRequest {
name: string
x: number
y: number
width: number
height: number
page: number
ocr_language: string
transform: OcrZoneTransform
date_format?: string
validation_regex: string
zone_source_width?: number
zone_source_height?: number
}
export interface OcrZoneTestResult {
raw_text?: string | null
value?: string | null
regex?: string
regex_match?: boolean | null
error?: string
}
export interface OcrZoneRunResult {
template: string
zone: string
custom_field: string
value: string | number | boolean | null
reason?: string | null
}
export interface OcrZoneRunResponse {
status: 'completed' | 'skipped'
reason?: 'no_template'
results: OcrZoneRunResult[]
}
+1
View File
@@ -35,6 +35,7 @@ export enum HideableSidebarItemID {
SavedViews = 'saved_views',
ShareLinks = 'share_links',
Workflows = 'workflows',
OcrTemplates = 'ocr_templates',
Mail = 'mail',
Documentation = 'documentation',
}
+6 -2
View File
@@ -3,12 +3,16 @@ import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { DirtyCheckGuard } from '@ngneat/dirty-check-forms'
import { Observable, Subject } from 'rxjs'
import { ConfirmDialogComponent } from 'src/app/components/common/confirm-dialog/confirm-dialog.component'
import { ActivatedRouteSnapshot } from '@angular/router'
@Injectable({ providedIn: 'root' })
export class DirtyFormGuard extends DirtyCheckGuard {
private modalService = inject(NgbModal)
confirmChanges(): Observable<boolean> {
confirmChanges(
_currentRoute?: ActivatedRouteSnapshot,
leaveLabel: string = $localize`Leave page`
): Observable<boolean> {
let modal = this.modalService.open(ConfirmDialogComponent, {
backdrop: 'static',
})
@@ -16,7 +20,7 @@ export class DirtyFormGuard extends DirtyCheckGuard {
modal.componentInstance.messageBold = $localize`You have unsaved changes.`
modal.componentInstance.message = $localize`Are you sure you want to leave?`
modal.componentInstance.btnClass = 'btn-warning'
modal.componentInstance.btnCaption = $localize`Leave page`
modal.componentInstance.btnCaption = leaveLabel
modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled.set(false)
modal.close()
@@ -283,6 +283,10 @@ describe('PermissionsService', () => {
'view_workflow',
'change_workflow',
'delete_workflow',
'add_ocrtemplate',
'view_ocrtemplate',
'change_ocrtemplate',
'delete_ocrtemplate',
'add_customfield',
'view_customfield',
'change_customfield',
@@ -29,6 +29,7 @@ export enum PermissionType {
ShareLinkBundle = '%s_sharelinkbundle',
CustomField = '%s_customfield',
Workflow = '%s_workflow',
OcrTemplate = '%s_ocrtemplate',
ProcessedMail = '%s_processedmail',
GlobalStatistics = '%s_global_statistics',
SystemMonitoring = '%s_system_monitoring',
@@ -563,6 +563,20 @@ describe(`DocumentService`, () => {
req.flush('task-id')
})
it('should call appropriate api endpoint for running zone OCR', () => {
const url = `${environment.apiBaseUrl}${endpoint}/${documents[0].id}/run-zone-ocr/`
subscription = service.runZoneOcr(documents[0].id).subscribe()
let req = httpTestingController.expectOne(url)
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({})
req.flush({ status: 'completed', results: [] })
subscription = service.runZoneOcr(documents[0].id, 3).subscribe()
req = httpTestingController.expectOne(url)
expect(req.request.body).toEqual({ template: 3 })
req.flush({ status: 'completed', results: [] })
})
})
it('should construct sort fields respecting permissions', () => {
@@ -12,6 +12,7 @@ import {
import { DocumentMetadata } from 'src/app/data/document-metadata'
import { DocumentSuggestions } from 'src/app/data/document-suggestions'
import { FilterRule } from 'src/app/data/filter-rule'
import { OcrZoneRunResponse } from 'src/app/data/ocr-template'
import { Results, SelectionData } from 'src/app/data/results'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { queryParamsFromFilterRules } from '../../utils/query-params'
@@ -360,6 +361,26 @@ export class DocumentService extends AbstractPaperlessService<Document> {
})
}
runZoneOcrDocuments(
selection: DocumentSelectionQuery,
templateId: number | null = null
) {
return this.http.post(this.getResourceUrl(null, 'run_zone_ocr'), {
...selection,
...(templateId ? { template: templateId } : {}),
})
}
runZoneOcr(
id: number,
templateId: number | null = null
): Observable<OcrZoneRunResponse> {
return this.http.post<OcrZoneRunResponse>(
this.getResourceUrl(id, 'run-zone-ocr'),
templateId ? { template: templateId } : {}
)
}
rotateDocuments(
selection: DocumentSelectionQuery,
degrees: number,
@@ -0,0 +1,78 @@
import { provideHttpClient } from '@angular/common/http'
import {
HttpTestingController,
provideHttpClientTesting,
} from '@angular/common/http/testing'
import { TestBed } from '@angular/core/testing'
import { OcrZoneTransform, ZoneTestRequest } from 'src/app/data/ocr-template'
import { environment } from 'src/environments/environment'
import { OcrTemplateService } from './ocr-template.service'
describe('OcrTemplateService', () => {
let service: OcrTemplateService
let http: HttpTestingController
beforeEach(() => {
TestBed.configureTestingModule({
providers: [provideHttpClient(), provideHttpClientTesting()],
})
service = TestBed.inject(OcrTemplateService)
http = TestBed.inject(HttpTestingController)
})
afterEach(() => http.verify())
it('builds page image URLs with zero-based page numbers', () => {
expect(service.getPageImageUrl(12, 0)).toBe(
`${environment.apiBaseUrl}ocr_templates/document-page-image/12/0/`
)
expect(service.getPageImageUrl(12, 2)).toBe(
`${environment.apiBaseUrl}ocr_templates/document-page-image/12/2/`
)
})
it('posts a zone test and returns the extraction result', () => {
const zone: ZoneTestRequest = {
name: 'Total',
x: 10,
y: 20,
width: 100,
height: 50,
page: -1,
ocr_language: 'eng',
transform: OcrZoneTransform.Numeric,
validation_regex: '\\d+',
zone_source_width: 1000,
zone_source_height: 1500,
}
const result = {
raw_text: 'Total: 123',
value: '123',
regex: '\\d+',
regex_match: true,
}
const next = jest.fn()
service.testZone(12, zone).subscribe(next)
const request = http.expectOne(
`${environment.apiBaseUrl}ocr_templates/test-zone/`
)
expect(request.request.method).toBe('POST')
expect(request.request.body).toEqual({ document: 12, zone })
request.flush(result)
expect(next).toHaveBeenCalledWith(result)
})
it('passes zone-test failures to the caller', () => {
const error = jest.fn()
service.testZone(12, {} as ZoneTestRequest).subscribe({ error })
http
.expectOne(`${environment.apiBaseUrl}ocr_templates/test-zone/`)
.flush(
{ error: 'Invalid zone' },
{ status: 400, statusText: 'Bad Request' }
)
expect(error).toHaveBeenCalledWith(
expect.objectContaining({ status: 400, error: { error: 'Invalid zone' } })
)
})
})
@@ -0,0 +1,30 @@
import { Injectable } from '@angular/core'
import { Observable } from 'rxjs'
import {
OcrTemplate,
OcrZoneTestResult,
ZoneTestRequest,
} from '../../data/ocr-template'
import { AbstractPaperlessService } from './abstract-paperless-service'
@Injectable({ providedIn: 'root' })
export class OcrTemplateService extends AbstractPaperlessService<OcrTemplate> {
constructor() {
super()
this.resourceName = 'ocr_templates'
}
getPageImageUrl(docId: number, page: number): string {
return `${this.baseUrl}${this.resourceName}/document-page-image/${docId}/${page}/`
}
testZone(
docId: number,
zone: ZoneTestRequest
): Observable<OcrZoneTestResult> {
return this.http.post<OcrZoneTestResult>(
`${this.baseUrl}${this.resourceName}/test-zone/`,
{ document: docId, zone }
)
}
}
+1 -1
View File
@@ -8,7 +8,7 @@ export const environment = {
apiVersion: '10', // match src/paperless/settings.py
appTitle: DEFAULT_APP_TITLE,
tag: 'prod',
version: '3.3.3',
version: '3.3.4',
webSocketHost: window.location.host,
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
webSocketBaseUrl: base_url.pathname + 'ws/',
+6
View File
@@ -42,6 +42,7 @@ import {
bell,
bodyText,
bookmark,
boundingBoxCircles,
boxArrowInRight,
boxArrowUp,
boxArrowUpRight,
@@ -105,6 +106,7 @@ import {
fileEarmarkPlus,
fileEarmarkRichtext,
fileEarmarkSpreadsheet,
fileEarmarkRuled,
fileText,
files,
filter,
@@ -155,6 +157,7 @@ import {
send,
shop,
slashCircle,
sliders2,
sliders2Vertical,
sortAlphaDown,
sortAlphaUpAlt,
@@ -299,6 +302,7 @@ const icons = {
braces,
bodyText,
bookmark,
boundingBoxCircles,
boxArrowInRight,
boxArrowUp,
boxArrowUpRight,
@@ -361,6 +365,7 @@ const icons = {
fileEarmarkPlus,
fileEarmarkRichtext,
fileEarmarkSpreadsheet,
fileEarmarkRuled,
files,
fileText,
filter,
@@ -411,6 +416,7 @@ const icons = {
send,
slashCircle,
shop,
sliders2,
sliders2Vertical,
sortAlphaDown,
sortAlphaUpAlt,
+6
View File
@@ -691,6 +691,12 @@ table.table {
}
}
@media (min-width: 992px) {
.modal-xxl {
--bs-modal-width: min(1440px, calc(100vw - 2 * var(--bs-modal-margin)));
}
}
.toast {
--bs-toast-max-width: var(--pngx-toast-max-width);
}
+3
View File
@@ -15,6 +15,7 @@ class DocumentsConfig(AppConfig):
from documents.signals.handlers import add_to_index
from documents.signals.handlers import run_workflows_added
from documents.signals.handlers import run_workflows_updated
from documents.signals.handlers import run_zone_ocr_extraction
from documents.signals.handlers import send_websocket_document_updated
from documents.signals.handlers import set_correspondent
from documents.signals.handlers import set_document_type
@@ -29,6 +30,8 @@ class DocumentsConfig(AppConfig):
document_consumption_finished.connect(run_workflows_added)
document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(add_or_update_document_in_llm_index)
document_consumption_finished.connect(run_zone_ocr_extraction)
document_updated.connect(run_workflows_updated)
document_updated.connect(send_websocket_document_updated)
document_updated.connect(add_or_update_document_in_llm_index)
+205 -219
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import logging
import tempfile
import uuid
from functools import partial
from pathlib import Path
from typing import TYPE_CHECKING
from typing import Literal
@@ -23,6 +24,7 @@ from django.db.models import When
from django.db.models.functions import Coalesce
from django.utils import timezone
from documents import pdf_ops
from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
from documents.data_models import DocumentSource
@@ -31,6 +33,7 @@ from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentType
from documents.models import OcrTemplate
from documents.models import PaperlessTask
from documents.models import StoragePath
from documents.models import Tag
@@ -38,6 +41,7 @@ from documents.permissions import set_permissions_for_objects
from documents.plugins.helpers import DocumentsStatusManager
from documents.tasks import bulk_update_documents
from documents.tasks import consume_file
from documents.tasks import extract_zones
from documents.tasks import remove_document_from_index
from documents.tasks import update_document_content_maybe_archive_file
from documents.versioning import get_latest_version_for_root
@@ -123,6 +127,11 @@ def _resolve_root_and_source_doc(
)
def _scratch_path(name: str) -> Path:
"""A path inside a fresh directory under SCRATCH_DIR."""
return Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR)) / name
def set_correspondent(
doc_ids: list[int],
correspondent: Correspondent,
@@ -499,8 +508,6 @@ def rotate(
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
docs_by_root_id.setdefault(pair.root_doc.id, pair)
import pikepdf
for pair in docs_by_root_id.values():
if pair.source_doc.mime_type != "application/pdf":
logger.warning(
@@ -513,11 +520,7 @@ def rotate(
Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR))
/ f"{pair.root_doc.id}_rotated.pdf"
)
with pikepdf.open(pair.source_doc.source_path) as pdf:
for page in pdf.pages:
page.rotate(degrees, relative=True)
pdf.remove_unreferenced_resources()
pdf.save(filepath)
pdf_ops.rotate_pdf(pair.source_doc.source_path, filepath, degrees)
# Preserve metadata/permissions via overrides; mark as new version
overrides = DocumentMetadataOverrides().from_document(pair.root_doc)
@@ -560,48 +563,45 @@ def merge(
qs = Document.objects.select_related("root_document").filter(id__in=doc_ids)
docs_by_id = {doc.id: doc for doc in qs}
affected_docs: list[int] = []
import pikepdf
merged_pdf = pikepdf.new()
version: str = merged_pdf.pdf_version
handoff_asn: int | None = None
# use doc_ids to preserve order
for doc_id in doc_ids:
doc = docs_by_id.get(doc_id)
if doc is None:
continue
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
try:
doc_path = (
pair.source_doc.archive_path
if archive_fallback
and pair.source_doc.mime_type != "application/pdf"
and pair.source_doc.has_archive_version
else pair.source_doc.source_path
)
with pikepdf.open(str(doc_path)) as pdf:
version = max(version, pdf.pdf_version)
merged_pdf.pages.extend(pdf.pages)
affected_docs.append(doc.id)
if handoff_asn is None and doc.archive_serial_number is not None:
handoff_asn = doc.archive_serial_number
except Exception as e:
logger.exception(
f"Error merging document {doc.id}, it will not be included in the merge: {e}",
)
if len(affected_docs) == 0:
logger.warning("No documents were merged")
return "OK"
with pdf_ops.PdfMerger() as merger:
# use doc_ids to preserve order
for doc_id in doc_ids:
doc = docs_by_id.get(doc_id)
if doc is None:
continue
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
try:
# archive_path is None when there is no archive version
archive_path = (
pair.source_doc.archive_path
if archive_fallback
and pair.source_doc.mime_type != "application/pdf"
else None
)
merger.add(
archive_path
if archive_path is not None
else pair.source_doc.source_path,
)
affected_docs.append(doc.id)
if handoff_asn is None and doc.archive_serial_number is not None:
handoff_asn = doc.archive_serial_number
except Exception as e:
logger.exception(
f"Error merging document {doc.id}, it will not be included in the merge: {e}",
)
if len(affected_docs) == 0:
logger.warning("No documents were merged")
return "OK"
filepath = (
Path(
tempfile.mkdtemp(dir=settings.SCRATCH_DIR),
filepath = (
Path(
tempfile.mkdtemp(dir=settings.SCRATCH_DIR),
)
/ f"{'_'.join([str(doc_id) for doc_id in affected_docs])[:100]}_merged.pdf"
)
/ f"{'_'.join([str(doc_id) for doc_id in affected_docs])[:100]}_merged.pdf"
)
merged_pdf.remove_unreferenced_resources()
merged_pdf.save(filepath, min_version=version)
merged_pdf.close()
merger.save(filepath)
if metadata_document_id:
metadata_document = qs.get(id=metadata_document_id)
@@ -777,64 +777,60 @@ def split(
)
doc = Document.objects.select_related("root_document").get(id=doc_ids[0])
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
import pikepdf
consume_tasks = []
try:
with pikepdf.open(pair.source_doc.source_path) as pdf:
for idx, split_doc in enumerate(pages):
dst: pikepdf.Pdf = pikepdf.new()
for page in split_doc:
dst.pages.append(pdf.pages[page - 1])
filepath: Path = (
Path(
tempfile.mkdtemp(dir=settings.SCRATCH_DIR),
)
/ f"{doc.id}_{split_doc[0]}-{split_doc[-1]}.pdf"
)
dst.remove_unreferenced_resources()
dst.save(filepath)
dst.close()
outputs = [
(
[pdf_ops.PageSpec(page) for page in split_doc],
partial(_scratch_path, f"{doc.id}_{split_doc[0]}-{split_doc[-1]}.pdf"),
)
for split_doc in pages
]
filepaths = pdf_ops.build_pdfs(pair.source_doc.source_path, outputs)
overrides: DocumentMetadataOverrides = (
DocumentMetadataOverrides().from_document(doc)
)
overrides.title = f"{doc.title} (split {idx + 1})"
if user is not None:
overrides.owner_id = user.id
if not delete_originals:
overrides.skip_asn_if_exists = True
logger.info(
f"Adding split document with pages {split_doc} to the task queue.",
)
consume_tasks.append(
consume_file.s(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
),
overrides=overrides,
).set(headers={"trigger_source": trigger_source}),
)
for idx, (split_doc, filepath) in enumerate(
zip(pages, filepaths, strict=True),
):
overrides: DocumentMetadataOverrides = (
DocumentMetadataOverrides().from_document(doc)
)
overrides.title = f"{doc.title} (split {idx + 1})"
if user is not None:
overrides.owner_id = user.id
if not delete_originals:
overrides.skip_asn_if_exists = True
logger.info(
f"Adding split document with pages {split_doc} to the task queue.",
)
consume_tasks.append(
consume_file.s(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
),
overrides=overrides,
).set(headers={"trigger_source": trigger_source}),
)
if delete_originals:
backup = release_archive_serial_numbers([doc.id])
logger.info(
"Queueing removal of original document after consumption of the split documents",
)
try:
chord(
header=consume_tasks,
body=delete.si([doc.id]),
).on_error(
restore_archive_serial_numbers_task.s(backup),
).apply_async()
except Exception:
restore_archive_serial_numbers(backup)
raise
else:
group(consume_tasks).delay()
if delete_originals:
backup = release_archive_serial_numbers([doc.id])
logger.info(
"Queueing removal of original document after consumption of the split documents",
)
try:
chord(
header=consume_tasks,
body=delete.si([doc.id]),
).on_error(
restore_archive_serial_numbers_task.s(backup),
).apply_async()
except Exception:
restore_archive_serial_numbers(backup)
raise
else:
group(consume_tasks).delay()
except Exception as e:
logger.exception(f"Error splitting document {doc.id}: {e}")
@@ -855,8 +851,6 @@ def delete_pages(
)
doc = Document.objects.select_related("root_document").get(id=doc_ids[0])
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
pages = sorted(pages) # sort pages to avoid index issues
import pikepdf
try:
# Produce edited PDF to a temp file and create a new version
@@ -864,13 +858,7 @@ def delete_pages(
Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR))
/ f"{pair.root_doc.id}_pages_deleted.pdf"
)
with pikepdf.open(pair.source_doc.source_path) as pdf:
offset = 1 # pages are 1-indexed
for page_num in pages:
pdf.pages.remove(pdf.pages[page_num - offset])
offset += 1 # remove() changes the index of the pages
pdf.remove_unreferenced_resources()
pdf.save(filepath)
pdf_ops.remove_pages(pair.source_doc.source_path, filepath, pages)
overrides = DocumentMetadataOverrides().from_document(pair.root_doc)
if user is not None:
@@ -919,47 +907,28 @@ def edit_pdf(
)
doc = Document.objects.select_related("root_document").get(id=doc_ids[0])
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
import pikepdf
pdf_docs: list[pikepdf.Pdf] = []
try:
if not operations:
raise ValueError("Output document index is out of bounds")
max_idx = max(op.get("doc", 0) for op in operations)
if update_document and max_idx > 0:
logger.error(
"Update requested but multiple output documents specified",
output_count = pdf_ops.validate_page_operations(
operations,
single_output=update_document,
)
page_specs: list[list[pdf_ops.PageSpec]] = [[] for _ in range(output_count)]
for op in operations:
page_specs[op.get("doc", 0)].append(
pdf_ops.PageSpec(op["page"], op.get("rotate", 0)),
)
raise ValueError("Multiple output documents specified")
if any(
op.get("doc", 0) < 0 or op.get("doc", 0) >= len(operations)
for op in operations
):
raise ValueError("Output document index is out of bounds")
with pikepdf.open(pair.source_doc.source_path) as src:
# prepare output documents
pdf_docs = [pikepdf.new() for _ in range(max_idx + 1)]
for op in operations:
dst = pdf_docs[op.get("doc", 0)]
page = src.pages[op["page"] - 1]
dst.pages.append(page)
if op.get("rotate"):
dst.pages[-1].rotate(op["rotate"], relative=True)
if update_document:
# Create a new version from the edited PDF rather than replacing in-place
pdf = pdf_docs[0]
pdf.remove_unreferenced_resources()
filepath: Path = (
Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR))
/ f"{pair.root_doc.id}_edited.pdf"
(filepath,) = pdf_ops.build_pdfs(
pair.source_doc.source_path,
[
(
page_specs[0],
partial(_scratch_path, f"{pair.root_doc.id}_edited.pdf"),
),
],
)
pdf.save(filepath)
overrides = (
DocumentMetadataOverrides().from_document(pair.root_doc)
if include_metadata
@@ -980,6 +949,19 @@ def edit_pdf(
headers={"trigger_source": trigger_source},
)
else:
version_filepaths = pdf_ops.build_pdfs(
pair.source_doc.source_path,
[
(
specs,
partial(
_scratch_path,
f"{pair.root_doc.id}_edit_{idx}.pdf",
),
)
for idx, specs in enumerate(page_specs, start=1)
],
)
consume_tasks = []
overrides = (
DocumentMetadataOverrides().from_document(pair.root_doc)
@@ -991,15 +973,9 @@ def edit_pdf(
overrides.actor_id = user.id
if not delete_original:
overrides.skip_asn_if_exists = True
if delete_original and len(pdf_docs) == 1:
if delete_original and output_count == 1:
overrides.asn = pair.root_doc.archive_serial_number
for idx, pdf in enumerate(pdf_docs, start=1):
version_filepath: Path = (
Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR))
/ f"{pair.root_doc.id}_edit_{idx}.pdf"
)
pdf.remove_unreferenced_resources()
pdf.save(version_filepath)
for version_filepath in version_filepaths:
consume_tasks.append(
consume_file.s(
input_doc=ConsumableDocument(
@@ -1049,8 +1025,6 @@ def remove_password(
"""
Remove password protection from PDF documents.
"""
import pikepdf
for doc_id in doc_ids:
doc = Document.objects.select_related("root_document").get(id=doc_id)
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
@@ -1064,76 +1038,69 @@ def remove_password(
doc.id,
pair.source_doc.source_path,
)
try:
with pikepdf.open(source_path) as pdf:
if not pdf.is_encrypted:
logger.info(
"Skipping password removal for document %s because the "
"source PDF is not encrypted",
pair.root_doc.id,
)
continue
except pikepdf.PasswordError:
# Password-protected PDFs need the supplied password below.
pass
with pikepdf.open(source_path, password=password) as pdf:
filepath: Path = (
Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR))
/ f"{pair.root_doc.id}_unprotected.pdf"
if not pdf_ops.needs_decrypt(source_path):
logger.info(
"Skipping password removal for document %s because the "
"source PDF is not encrypted",
pair.root_doc.id,
)
pdf.remove_unreferenced_resources()
pdf.save(filepath)
continue
if update_document:
# Create a new version rather than modifying the root/original in place.
overrides = (
DocumentMetadataOverrides().from_document(pair.root_doc)
if include_metadata
else DocumentMetadataOverrides()
)
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
)
filepath = pdf_ops.decrypt_pdf(
source_path,
partial(_scratch_path, f"{pair.root_doc.id}_unprotected.pdf"),
password,
)
if update_document:
# Create a new version rather than modifying the root/original in place.
overrides = (
DocumentMetadataOverrides().from_document(pair.root_doc)
if include_metadata
else DocumentMetadataOverrides()
)
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
)
else:
consume_tasks = []
overrides = (
DocumentMetadataOverrides().from_document(pair.root_doc)
if include_metadata
else DocumentMetadataOverrides()
)
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
consume_tasks.append(
consume_file.s(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
),
overrides=overrides,
).set(headers={"trigger_source": trigger_source}),
)
if delete_original:
chord(
header=consume_tasks,
body=delete.si([doc.id]),
).delay()
else:
consume_tasks = []
overrides = (
DocumentMetadataOverrides().from_document(pair.root_doc)
if include_metadata
else DocumentMetadataOverrides()
)
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
consume_tasks.append(
consume_file.s(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
),
overrides=overrides,
).set(headers={"trigger_source": trigger_source}),
)
if delete_original:
chord(
header=consume_tasks,
body=delete.si([doc.id]),
).delay()
else:
group(consume_tasks).delay()
group(consume_tasks).delay()
except Exception as e:
logger.exception(
@@ -1231,3 +1198,22 @@ def remove_doclink(
target_doc_field_instance.value.remove(document.id)
target_doc_field_instance.save()
Document.objects.filter(id=target_doc_id).update(modified=timezone.now())
def run_zone_ocr(
doc_ids: list[int],
*,
template: OcrTemplate | None = None,
) -> Literal["OK"]:
"""
Queue zone OCR for the given documents, with the given template or with the
templates that match each document's content when its task runs.
"""
template_id = template.pk if template is not None else None
for document_id in doc_ids:
extract_zones.apply_async(
kwargs={"document_id": document_id, "template_id": template_id},
headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
)
return "OK"
+3
View File
@@ -193,6 +193,9 @@ class DirectoryExportSink(ExportSink):
raise RuntimeError("A stream is already open on this sink")
target = (self._target / arcname).resolve()
tmp = target.with_suffix(target.suffix + ".tmp")
# A stale tmp from an interrupted run is overwritten and consumed here,
# so it must not be pruned as a leftover at finalize.
self._snapshot.discard(tmp)
target.parent.mkdir(parents=True, exist_ok=True)
handle = tmp.open("w", encoding="utf-8")
self._stream_open = True
@@ -0,0 +1,311 @@
# Generated by Django 5.2.17 on 2026-10-20 20:29
import django.core.validators
import django.db.models.deletion
import django.utils.timezone
from django.conf import settings
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0027_documentbarcode"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name="OcrTemplate",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("name", models.CharField(max_length=128, verbose_name="name")),
(
"match",
models.CharField(blank=True, max_length=256, verbose_name="match"),
),
(
"matching_algorithm",
models.PositiveSmallIntegerField(
choices=[
(0, "None"),
(1, "Any word"),
(2, "All words"),
(3, "Exact match"),
(4, "Regular expression"),
(5, "Fuzzy word"),
(6, "Automatic"),
],
default=1,
verbose_name="matching algorithm",
),
),
(
"is_insensitive",
models.BooleanField(default=True, verbose_name="is insensitive"),
),
(
"source_width",
models.PositiveIntegerField(
help_text="Width of the image the zones were drawn on (px)",
validators=[django.core.validators.MinValueValidator(1)],
verbose_name="source width",
),
),
(
"source_height",
models.PositiveIntegerField(
help_text="Height of the image the zones were drawn on (px)",
validators=[django.core.validators.MinValueValidator(1)],
verbose_name="source height",
),
),
("enabled", models.BooleanField(default=True, verbose_name="enabled")),
(
"combine_formats",
models.JSONField(
blank=True,
default=dict,
help_text="Per-target format strings for combining several zones into one field, keyed by target (custom field id, or 'title'/'asn'/'created'). Tokens like {Zone Name} are replaced with that zone's value.",
verbose_name="combine formats",
),
),
(
"created",
models.DateTimeField(
db_index=True,
default=django.utils.timezone.now,
editable=False,
verbose_name="created",
),
),
(
"updated",
models.DateTimeField(auto_now=True, verbose_name="updated"),
),
(
"owner",
models.ForeignKey(
blank=True,
default=None,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
to=settings.AUTH_USER_MODEL,
verbose_name="owner",
),
),
(
"sample_document",
models.ForeignKey(
blank=True,
help_text="Document used for previewing zones in the editor",
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="+",
to="documents.document",
verbose_name="sample document",
),
),
],
options={
"verbose_name": "OCR template",
"verbose_name_plural": "OCR templates",
"ordering": ("name",),
"abstract": False,
},
),
migrations.CreateModel(
name="OcrTemplateZone",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"name",
models.CharField(
help_text="Descriptive name for this zone (e.g. 'Invoice Number')",
max_length=128,
verbose_name="zone name",
),
),
(
"target",
models.CharField(
choices=[
("custom_field", "Custom field"),
("title", "Title"),
("asn", "Archive serial number"),
("created", "Date created"),
],
default="custom_field",
help_text="Where the extracted value is written: a custom field, or a built-in document field (title, ASN, created date)",
max_length=20,
verbose_name="target",
),
),
(
"page",
models.IntegerField(
blank=True,
help_text="Page (1 = first, -1 = last; blank uses the template default)",
null=True,
verbose_name="page",
),
),
(
"x",
models.PositiveIntegerField(
help_text="Left edge (px)",
verbose_name="x",
),
),
(
"y",
models.PositiveIntegerField(
help_text="Top edge (px)",
verbose_name="y",
),
),
(
"width",
models.PositiveIntegerField(
help_text="Zone width (px)",
validators=[django.core.validators.MinValueValidator(1)],
verbose_name="width",
),
),
(
"height",
models.PositiveIntegerField(
help_text="Zone height (px)",
validators=[django.core.validators.MinValueValidator(1)],
verbose_name="height",
),
),
(
"zone_source_width",
models.PositiveIntegerField(
blank=True,
help_text="Width of the page image this zone was drawn on (px). Falls back to template source_width if unset.",
null=True,
verbose_name="zone source width",
),
),
(
"zone_source_height",
models.PositiveIntegerField(
blank=True,
help_text="Height of the page image this zone was drawn on (px). Falls back to template source_height if unset.",
null=True,
verbose_name="zone source height",
),
),
(
"ocr_language",
models.CharField(
default="eng",
help_text="Tesseract language code(s), e.g. 'deu+eng'",
max_length=20,
verbose_name="OCR language",
),
),
(
"transform",
models.CharField(
choices=[
("none", "None"),
("strip", "Strip whitespace"),
("uppercase", "Uppercase"),
("lowercase", "Lowercase"),
("numeric", "Numeric only"),
(
"strip_punctuation",
"Remove leading/trailing punctuation",
),
("date", "Parse date"),
("qr_code", "Read QR/barcode"),
],
default="strip",
max_length=20,
verbose_name="transform",
),
),
(
"date_format",
models.CharField(
blank=True,
default="",
help_text="Python strptime format for the 'Parse date' transform (e.g. %d.%m.%Y). Blank = auto-detect.",
max_length=64,
verbose_name="date format",
),
),
(
"validation_regex",
models.CharField(
blank=True,
default="",
help_text="Optional regex pattern — extracted text is only accepted if it matches",
max_length=256,
verbose_name="validation regex",
),
),
("order", models.PositiveIntegerField(default=0, verbose_name="order")),
(
"custom_field",
models.ForeignKey(
blank=True,
help_text="Target custom field (only used when target is 'custom_field')",
null=True,
on_delete=django.db.models.deletion.CASCADE,
related_name="ocr_zones",
to="documents.customfield",
verbose_name="custom field",
),
),
(
"template",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="zones",
to="documents.ocrtemplate",
verbose_name="template",
),
),
],
options={
"verbose_name": "OCR template zone",
"verbose_name_plural": "OCR template zones",
"ordering": ("template", "order"),
},
),
migrations.AddConstraint(
model_name="ocrtemplate",
constraint=models.UniqueConstraint(
fields=("name", "owner"),
name="documents_ocrtemplate_unique_name_owner",
),
),
migrations.AddConstraint(
model_name="ocrtemplate",
constraint=models.UniqueConstraint(
condition=models.Q(("owner__isnull", True)),
fields=("name",),
name="documents_ocrtemplate_name_uniq",
),
),
]
+212
View File
@@ -2149,3 +2149,215 @@ class WorkflowRun(SoftDeleteModel):
def __str__(self) -> str:
return f"WorkflowRun of {self.workflow} at {self.run_at} on {self.document}"
class OcrTemplate(MatchingModel):
"""Matching rules and extraction zones for documents with a common layout."""
source_width = models.PositiveIntegerField(
_("source width"),
validators=[MinValueValidator(1)],
help_text=_("Width of the image the zones were drawn on (px)"),
)
source_height = models.PositiveIntegerField(
_("source height"),
validators=[MinValueValidator(1)],
help_text=_("Height of the image the zones were drawn on (px)"),
)
sample_document = models.ForeignKey(
"documents.Document",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="+",
verbose_name=_("sample document"),
help_text=_("Document used for previewing zones in the editor"),
)
enabled = models.BooleanField(_("enabled"), default=True)
combine_formats = models.JSONField(
_("combine formats"),
default=dict,
blank=True,
help_text=_(
"Per-target format strings for combining several zones into one "
"field, keyed by target (custom field id, or 'title'/'asn'/'created'). "
"Tokens like {Zone Name} are replaced with that zone's value.",
),
)
created = models.DateTimeField(
_("created"),
default=timezone.now,
db_index=True,
editable=False,
)
updated = models.DateTimeField(
_("updated"),
auto_now=True,
)
class Meta(MatchingModel.Meta):
verbose_name = _("OCR template")
verbose_name_plural = _("OCR templates")
class OcrTemplateZone(models.Model):
"""A page region and the field its extracted value is written to."""
template = models.ForeignKey(
OcrTemplate,
on_delete=models.CASCADE,
related_name="zones",
verbose_name=_("template"),
)
name = models.CharField(
_("zone name"),
max_length=128,
help_text=_("Descriptive name for this zone (e.g. 'Invoice Number')"),
)
class TargetType(models.TextChoices):
CUSTOM_FIELD = ("custom_field", _("Custom field"))
TITLE = ("title", _("Title"))
ASN = ("asn", _("Archive serial number"))
CREATED = ("created", _("Date created"))
target = models.CharField(
_("target"),
max_length=20,
choices=TargetType.choices,
default=TargetType.CUSTOM_FIELD,
help_text=_(
"Where the extracted value is written: a custom field, or a "
"built-in document field (title, ASN, created date)",
),
)
custom_field = models.ForeignKey(
"documents.CustomField",
on_delete=models.CASCADE,
related_name="ocr_zones",
verbose_name=_("custom field"),
null=True,
blank=True,
help_text=_("Target custom field (only used when target is 'custom_field')"),
)
page = models.IntegerField(
_("page"),
null=True,
blank=True,
help_text=_("Page (1 = first, -1 = last; blank uses the template default)"),
)
x = models.PositiveIntegerField(_("x"), help_text=_("Left edge (px)"))
y = models.PositiveIntegerField(_("y"), help_text=_("Top edge (px)"))
width = models.PositiveIntegerField(
_("width"),
validators=[MinValueValidator(1)],
help_text=_("Zone width (px)"),
)
height = models.PositiveIntegerField(
_("height"),
validators=[MinValueValidator(1)],
help_text=_("Zone height (px)"),
)
# Mixed-size PDFs need each zone's original page dimensions for scaling.
zone_source_width = models.PositiveIntegerField(
_("zone source width"),
null=True,
blank=True,
help_text=_(
"Width of the page image this zone was drawn on (px). "
"Falls back to template source_width if unset.",
),
)
zone_source_height = models.PositiveIntegerField(
_("zone source height"),
null=True,
blank=True,
help_text=_(
"Height of the page image this zone was drawn on (px). "
"Falls back to template source_height if unset.",
),
)
ocr_language = models.CharField(
_("OCR language"),
max_length=20,
default="eng",
help_text=_("Tesseract language code(s), e.g. 'deu+eng'"),
)
class TransformType(models.TextChoices):
NONE = ("none", _("None"))
STRIP = ("strip", _("Strip whitespace"))
UPPERCASE = ("uppercase", _("Uppercase"))
LOWERCASE = ("lowercase", _("Lowercase"))
NUMERIC = ("numeric", _("Numeric only"))
STRIP_PUNCTUATION = (
"strip_punctuation",
_("Remove leading/trailing punctuation"),
)
DATE = ("date", _("Parse date"))
QR_CODE = ("qr_code", _("Read QR/barcode"))
transform = models.CharField(
_("transform"),
max_length=20,
choices=TransformType.choices,
default=TransformType.STRIP,
)
date_format = models.CharField(
_("date format"),
max_length=64,
blank=True,
default="",
help_text=_(
"Python strptime format for the 'Parse date' transform "
"(e.g. %d.%m.%Y). Blank = auto-detect.",
),
)
validation_regex = models.CharField(
_("validation regex"),
max_length=256,
blank=True,
default="",
help_text=_(
"Optional regex pattern — extracted text is only accepted if it matches",
),
)
order = models.PositiveIntegerField(_("order"), default=0)
class Meta:
ordering = ("template", "order")
verbose_name = _("OCR template zone")
verbose_name_plural = _("OCR template zones")
def __str__(self) -> str:
return f"{self.template.name} -> {self.name}"
# Document links and select fields require object references, not OCR text.
OCR_SUPPORTED_FIELD_TYPES = frozenset(
{
CustomField.FieldDataType.STRING,
CustomField.FieldDataType.URL,
CustomField.FieldDataType.DATE,
CustomField.FieldDataType.INT,
CustomField.FieldDataType.FLOAT,
CustomField.FieldDataType.MONETARY,
CustomField.FieldDataType.LONG_TEXT,
CustomField.FieldDataType.BOOL,
},
)
+194
View File
@@ -0,0 +1,194 @@
"""
Pure PDF page operations used by documents.bulk_edit.
This module deliberately knows nothing about Django, Celery or the documents
app: callers resolve documents, choose output paths and queue work. Every
function that writes a PDF removes unreferenced resources before saving.
pikepdf is always called as ``pikepdf.open(...)`` / ``pikepdf.new()`` (never
``from pikepdf import open``) so tests can patch those module attributes.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from typing import NamedTuple
import pikepdf
if TYPE_CHECKING:
from collections.abc import Callable
from collections.abc import Iterable
from collections.abc import Mapping
from collections.abc import Sequence
from pathlib import Path
from types import TracebackType
class PageSpec(NamedTuple):
"""One page of an output PDF: a 1-indexed source page, optionally rotated."""
page: int
rotate: int = 0 # relative degrees, 0 leaves the page alone
def _require_positive(pages: Iterable[int]) -> None:
for page in pages:
if page < 1:
raise ValueError(f"Page numbers start at 1, got {page}")
def rotate_pdf(src: Path, dst: Path, degrees: int) -> None:
"""
Rotate every page relatively on the opened document, not a rebuild, so Info,
XMP and outlines are kept. ``src`` is not modified.
"""
with pikepdf.open(src) as pdf:
for page in pdf.pages:
page.rotate(degrees, relative=True)
pdf.remove_unreferenced_resources()
pdf.save(dst)
def remove_pages(src: Path, dst: Path, pages: Iterable[int]) -> None:
"""
Remove 1-indexed pages from the opened document, not a rebuild, so Info, XMP
and outlines are kept. ``src`` is not modified.
Duplicates are ignored. Pages are removed highest first so earlier removals
never shift the index of later ones.
"""
unique = sorted(set(pages))
_require_positive(unique)
with pikepdf.open(src) as pdf:
for page_num in reversed(unique):
del pdf.pages[page_num - 1]
pdf.remove_unreferenced_resources()
pdf.save(dst)
def build_pdfs(
src: Path,
outputs: Sequence[tuple[Sequence[PageSpec], Callable[[], Path]]],
) -> list[Path]:
"""
Build one new PDF per output from pages of ``src``, opening ``src`` once.
Each output is ``(page_specs, make_dst)``. Every page number is checked against
``src`` before any output is built, and ``make_dst`` is called after that
output's pages are copied and immediately before it is saved, so a bad page
number in any output never leaves a destination behind. Document-level data
(Info, XMP, outlines) is not carried over. Returns the written paths in output
order.
"""
for specs, _ in outputs:
_require_positive(spec.page for spec in specs)
written: list[Path] = []
with pikepdf.open(src) as source:
page_count = len(source.pages)
for specs, _ in outputs:
for spec in specs:
if spec.page > page_count:
raise IndexError(
f"Page {spec.page} is out of range, the PDF has "
f"{page_count} pages",
)
for specs, make_dst in outputs:
dst = pikepdf.new()
for spec in specs:
dst.pages.append(source.pages[spec.page - 1])
if spec.rotate:
dst.pages[-1].rotate(spec.rotate, relative=True)
dst.remove_unreferenced_resources()
path = make_dst()
dst.save(path)
dst.close()
written.append(path)
return written
def validate_page_operations(
operations: Sequence[Mapping[str, int]],
*,
single_output: bool,
) -> int:
"""
Validate ``edit_pdf`` style operations and return the output document count.
Each operation has ``page`` and optionally ``rotate`` and ``doc`` (the output
document index, default 0). The bounds rule is kept as it was: a ``doc`` index
must be below the number of operations.
"""
if not operations:
raise ValueError("Output document index is out of bounds")
max_idx = max(op.get("doc", 0) for op in operations)
if single_output and max_idx > 0:
raise ValueError("Multiple output documents specified")
if any(
op.get("doc", 0) < 0 or op.get("doc", 0) >= len(operations) for op in operations
):
raise ValueError("Output document index is out of bounds")
return max_idx + 1
def needs_decrypt(src: Path) -> bool:
"""
True if ``src`` is encrypted. A PDF that needs a password to open at all
counts as encrypted.
"""
try:
with pikepdf.open(src) as pdf:
return bool(pdf.is_encrypted)
except pikepdf.PasswordError:
return True
def decrypt_pdf(src: Path, make_dst: Callable[[], Path], password: str) -> Path:
"""
Write an unencrypted copy of ``src`` and return its path.
``make_dst`` is only called once the password has been accepted, so a wrong
password never leaves a destination behind.
"""
with pikepdf.open(src, password=password) as pdf:
pdf.remove_unreferenced_resources()
dst = make_dst()
pdf.save(dst)
return dst
class PdfMerger:
"""
Accumulates the pages of several PDFs into one new PDF.
``add`` raises if a source cannot be read; deciding whether to skip it is the
caller's policy. Use as a context manager so the merged PDF is closed.
"""
def __init__(self) -> None:
self._pdf = pikepdf.new()
self._version: str = self._pdf.pdf_version
def __enter__(self) -> PdfMerger:
return self
def __exit__(
self,
exc_type: type[BaseException] | None,
exc: BaseException | None,
tb: TracebackType | None,
) -> None:
self._pdf.close()
def add(self, path: Path) -> None:
with pikepdf.open(str(path)) as pdf:
self._version = max(self._version, pdf.pdf_version)
self._pdf.pages.extend(pdf.pages)
def save(self, dst: Path) -> None:
self._pdf.remove_unreferenced_resources()
self._pdf.save(dst, min_version=self._version)
+27
View File
@@ -77,6 +77,33 @@ def safe_regex_match(pattern: str, text: str, *, flags: int = 0):
return None
def safe_regex_fullmatch(pattern: str, text: str, *, flags: int = 0):
"""
Run a regex full match with a timeout. Returns a match object or None.
Validation errors and timeouts are logged and treated as no match.
"""
try:
validate_regex_pattern(pattern)
compiled = regex.compile(pattern, flags=flags)
except (regex.error, ValueError) as exc:
logger.exception(
"Error while processing regular expression %s: %s",
textwrap.shorten(pattern, width=80, placeholder="…"),
exc,
)
return None
try:
return compiled.fullmatch(text, timeout=REGEX_TIMEOUT_SECONDS)
except TimeoutError:
logger.warning(
"Regular expression matching timed out for pattern %s",
textwrap.shorten(pattern, width=80, placeholder="…"),
)
return None
def safe_regex_sub(pattern: str, repl: str, text: str, *, flags: int = 0) -> str | None:
"""
Run a regex substitution with a timeout. Returns the substituted string,
+196
View File
@@ -24,6 +24,7 @@ from django.core.validators import MaxValueValidator
from django.core.validators import MinValueValidator
from django.core.validators import RegexValidator
from django.core.validators import integer_validator
from django.db import transaction
from django.db.models import Count
from django.db.models import Q
from django.db.models.functions import Lower
@@ -56,6 +57,7 @@ if settings.AUDIT_LOG_ENABLED:
from documents import bulk_edit
from documents.data_models import DocumentSource
from documents.filters import CustomFieldQueryParser
from documents.models import OCR_SUPPORTED_FIELD_TYPES
from documents.models import Correspondent
from documents.models import CustomField
from documents.models import CustomFieldInstance
@@ -64,6 +66,8 @@ from documents.models import DocumentBarcode
from documents.models import DocumentType
from documents.models import MatchingModel
from documents.models import Note
from documents.models import OcrTemplate
from documents.models import OcrTemplateZone
from documents.models import PaperlessTask
from documents.models import SavedView
from documents.models import SavedViewFilterRule
@@ -1853,6 +1857,18 @@ class ReprocessDocumentsSerializer(DocumentSelectionSerializer):
remote_ocr = serializers.BooleanField(required=False, default=False)
class RunZoneOcrSerializer(serializers.Serializer):
template = serializers.PrimaryKeyRelatedField(
queryset=OcrTemplate.objects.prefetch_related("zones", "zones__custom_field"),
required=False,
allow_null=True,
)
class RunZoneOcrDocumentsSerializer(DocumentSelectionSerializer, RunZoneOcrSerializer):
pass
class BulkEditSerializer(
SerializerWithPerms,
DocumentSelectionSerializer,
@@ -2144,6 +2160,8 @@ class BulkEditSerializer(
raise serializers.ValidationError("pages must be a list")
if not all(isinstance(i, int) for i in parameters["pages"]):
raise serializers.ValidationError("pages must be a list of integers")
if any(i < 1 for i in parameters["pages"]):
raise serializers.ValidationError("pages must be positive integers")
def _validate_parameters_merge(self, parameters) -> None:
if "delete_originals" in parameters:
@@ -3728,3 +3746,181 @@ class StoragePathTestSerializer(SerializerWithPerms):
document_field.queryset = Document.objects.filter(
id__in=permitted_document_ids(user),
)
class OcrTemplateZoneSerializer(serializers.ModelSerializer):
class Meta:
model = OcrTemplateZone
fields = [
"id",
"name",
"target",
"custom_field",
"page",
"x",
"y",
"width",
"height",
"ocr_language",
"transform",
"date_format",
"order",
"zone_source_width",
"zone_source_height",
"validation_regex",
]
def validate_custom_field(self, value):
if value is None:
return value
if value.data_type not in OCR_SUPPORTED_FIELD_TYPES:
raise serializers.ValidationError(
f"Custom field type '{value.data_type}' is not supported for OCR extraction. "
f"Use string, integer, float, date, monetary, boolean, URL, or long text.",
)
return value
def validate_page(self, value):
if value is not None and value != -1 and value < 1:
raise serializers.ValidationError(
"Page must be positive or -1 for the last page.",
)
return value
def validate(self, data):
# A template PATCH still replaces each zone in full.
if self.root.partial:
for name, field in self.fields.items():
if field.required and name not in data:
raise serializers.ValidationError(
{name: field.error_messages["required"]},
)
return data
class OcrTemplateZoneTestSerializer(OcrTemplateZoneSerializer):
name = serializers.CharField(default="test")
x = serializers.IntegerField(min_value=0, default=0)
y = serializers.IntegerField(min_value=0, default=0)
width = serializers.IntegerField(min_value=2)
height = serializers.IntegerField(min_value=2)
class Meta(OcrTemplateZoneSerializer.Meta):
fields = [
"name",
"x",
"y",
"width",
"height",
"page",
"ocr_language",
"transform",
"date_format",
"validation_regex",
"zone_source_width",
"zone_source_height",
]
extra_kwargs = {
"zone_source_width": {"min_value": 1},
"zone_source_height": {"min_value": 1},
}
class OcrTemplateTestSerializer(serializers.Serializer):
document = serializers.PrimaryKeyRelatedField(
queryset=Document.objects.select_related("root_document__owner"),
)
zone = OcrTemplateZoneTestSerializer()
class OcrTemplateSerializer(MatchingModelSerializer, OwnedObjectSerializer):
zones = OcrTemplateZoneSerializer(many=True, required=False)
combine_formats = serializers.DictField(
child=serializers.CharField(allow_blank=True, trim_whitespace=False),
required=False,
)
# Templates are never assigned to documents
slug = None
document_count = None
class Meta:
model = OcrTemplate
fields = [
"id",
"name",
"match",
"matching_algorithm",
"is_insensitive",
"source_width",
"source_height",
"sample_document",
"enabled",
"combine_formats",
"created",
"updated",
"zones",
"owner",
"permissions",
"user_can_change",
"set_permissions",
]
read_only_fields = ["created", "updated"]
def validate_matching_algorithm(self, value):
if value == MatchingModel.MATCH_AUTO:
raise serializers.ValidationError(
"Automatic matching is not supported for OCR templates.",
)
return value
def validate(self, data):
data = super().validate(data)
zones = data.get("zones", [])
if (
self.instance
and "zones" not in data
and ("source_width" in data or "source_height" in data)
):
zones = self.instance.zones.values()
source_width = data.get(
"source_width",
getattr(self.instance, "source_width", None),
)
source_height = data.get(
"source_height",
getattr(self.instance, "source_height", None),
)
for zone in zones:
width = zone.get("zone_source_width") or source_width
height = zone.get("zone_source_height") or source_height
if (
zone.get("x", 0) + zone.get("width", 0) > width
or zone.get("y", 0) + zone.get("height", 0) > height
):
raise serializers.ValidationError(
{
"zones": f"Zone '{zone.get('name', '?')}' extends beyond its source dimensions.",
},
)
return data
@transaction.atomic
def create(self, validated_data):
zones_data = validated_data.pop("zones", [])
template = super().create(validated_data)
for zone_data in zones_data:
OcrTemplateZone.objects.create(template=template, **zone_data)
return template
@transaction.atomic
def update(self, instance, validated_data):
zones_data = validated_data.pop("zones", None)
instance = super().update(instance, validated_data)
if zones_data is not None:
instance.zones.all().delete()
for zone_data in zones_data:
OcrTemplateZone.objects.create(template=instance, **zone_data)
return instance
+13
View File
@@ -1404,6 +1404,19 @@ def close_connection_pool_on_worker_init(**kwargs) -> None:
conn.close_pool()
def run_zone_ocr_extraction(sender, document, original_file=None, **kwargs):
"""Apply matching templates without failing consumption if extraction fails."""
try:
from documents.zone_ocr import run_zone_extraction
run_zone_extraction(document, Path(original_file) if original_file else None)
except Exception:
logger.exception(
"Zone OCR extraction failed for document %s",
document.pk,
)
@worker_process_shutdown.connect
def close_connection_pool_on_worker_shutdown(**kwargs) -> None: # pragma: no cover
"""
+27
View File
@@ -47,6 +47,7 @@ from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentBarcode
from documents.models import DocumentType
from documents.models import OcrTemplate
from documents.models import PaperlessTask
from documents.models import ShareLink
from documents.models import ShareLinkBundle
@@ -69,6 +70,7 @@ from documents.utils import compute_checksum
from documents.utils import identity
from documents.versioning import annotate_effective_content
from documents.workflows.utils import get_workflows_for_trigger
from documents.zone_ocr import run_zone_extraction
from paperless.config import AIConfig
from paperless.config import BarcodeConfig
from paperless.config import RemoteOCRConfig
@@ -368,6 +370,31 @@ def _read_barcodes_for_reprocess(document: Document) -> list[StoredBarcode] | No
return None
@shared_task
def extract_zones(document_id: int, template_id: int | None = None) -> None:
"""
Run an OCR template against a stored document, or every template that
matches it when no template is given.
"""
document = Document.objects.get(id=document_id)
if template_id is None:
run_zone_extraction(document, None)
return
try:
template = OcrTemplate.objects.prefetch_related(
"zones",
"zones__custom_field",
).get(id=template_id)
except OcrTemplate.DoesNotExist:
logger.warning(
"OCR template %s no longer exists, not running zone OCR",
template_id,
)
return
run_zone_extraction(document, None, templates=[template])
@shared_task
def update_document_content_maybe_archive_file(
document_id,
+22
View File
@@ -157,6 +157,28 @@ class TestDirectoryExportSink:
assert not stale.exists()
assert (target / "originals" / "doc.pdf").exists()
def test_delete_with_stale_stream_tmp_from_interrupted_run(
self,
tmp_path: Path,
) -> None:
# A previous run killed mid-stream leaves manifest.json.tmp behind; the
# next run reuses that path, so pruning must not trip over it.
target: Path = tmp_path / "out"
target.mkdir()
(target / "manifest.json.tmp").write_text("[", encoding="utf-8")
with DirectoryExportSink(
target,
compare_checksums=False,
compare_json=False,
delete=True,
) as sink:
with sink.stream("manifest.json") as handle:
writer: StreamingManifestWriter = StreamingManifestWriter(handle)
writer.write_record({"pk": 1})
writer.close()
assert json.loads((target / "manifest.json").read_text()) == [{"pk": 1}]
assert not (target / "manifest.json.tmp").exists()
def test_no_delete_keeps_unwritten_files(
self,
tmp_path: Path,
+112
View File
@@ -12,6 +12,7 @@ from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentType
from documents.models import OcrTemplate
from documents.models import StoragePath
from documents.models import Tag
from paperless_testing.dirs import DirectoriesMixin
@@ -583,6 +584,87 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {"remote_ocr": True})
@mock.patch("documents.views.bulk_edit.run_zone_ocr")
def test_run_zone_ocr_documents_endpoint(self, m) -> None:
"""
GIVEN:
- API data to run zone OCR on a document
WHEN:
- API is called
THEN:
- run_zone_ocr is called with the document
"""
self.setup_mock(m, "run_zone_ocr")
response = self.client.post(
"/api/documents/run_zone_ocr/",
json.dumps({"documents": [self.doc1.id]}),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {})
@mock.patch("documents.views.bulk_edit.run_zone_ocr")
def test_run_zone_ocr_documents_endpoint_with_template(self, m) -> None:
"""
GIVEN:
- API data to run a chosen OCR template on a document
WHEN:
- API is called
THEN:
- run_zone_ocr is called with the template
"""
self.setup_mock(m, "run_zone_ocr")
template = OcrTemplate.objects.create(
name="Bills",
source_width=100,
source_height=100,
)
response = self.client.post(
"/api/documents/run_zone_ocr/",
json.dumps({"documents": [self.doc1.id], "template": template.pk}),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {"template": template})
@mock.patch("documents.views.bulk_edit.run_zone_ocr")
def test_run_zone_ocr_documents_endpoint_rejects_private_template(
self,
m,
) -> None:
"""
GIVEN:
- A user who can change the documents but not view the template
WHEN:
- API is called with that template
THEN:
- The whole request is refused
"""
user = UserFactory(username="editor")
grant_global(user, "view_document", "change_document")
template = OcrTemplate.objects.create(
name="Private",
owner=UserFactory(username="other"),
source_width=100,
source_height=100,
)
self.client.force_authenticate(user=user)
response = self.client.post(
"/api/documents/run_zone_ocr/",
json.dumps({"documents": [self.doc1.id], "template": template.pk}),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.set_storage_path")
def test_api_set_storage_path(self, m) -> None:
"""
@@ -1844,6 +1926,36 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m.assert_called_once()
self.assertEqual(m.call_args.kwargs["pages"], [[1], [2, 3, 4], [5]])
@mock.patch("documents.serialisers.bulk_edit.delete_pages")
def test_bulk_edit_delete_pages_rejects_pages_below_one(self, m) -> None:
"""
GIVEN:
- A legacy delete_pages bulk edit
WHEN:
- API to bulk edit is called with a page number below 1
THEN:
- API returns HTTP 400
- delete_pages is not called
"""
self.setup_mock(m, "delete_pages")
for pages in ([0], [-1], [1, 0]):
with self.subTest(pages=pages):
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc2.id],
"method": "delete_pages",
"parameters": {"pages": pages},
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn(b"pages must be positive integers", response.content)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.rotate")
def test_rotate_insufficient_permissions(self, m) -> None:
self.doc1.owner = User.objects.get(username="temp_admin")
File diff suppressed because it is too large. Load diff
+259 -231
View File
@@ -1,9 +1,9 @@
import shutil
from collections.abc import Callable
from datetime import date
from pathlib import Path
from unittest import mock
import pikepdf
import pytest
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
@@ -21,9 +21,12 @@ from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentType
from documents.models import OcrTemplate
from documents.models import StoragePath
from documents.models import Tag
from documents.pdf_ops import PageSpec
from documents.permissions import set_permissions_for_objects
from documents.tasks import extract_zones
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.permissions import grant_object
@@ -796,16 +799,14 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.img_doc.save()
@staticmethod
def mock_password_required_pdf(
mock_open: mock.Mock,
fake_pdf: mock.Mock,
) -> None:
password_context = mock.MagicMock()
password_context.__enter__.return_value = fake_pdf
mock_open.side_effect = [
pikepdf.PasswordError("password required"),
password_context,
]
def fake_decrypt(
src: Path,
make_dst: Callable[[], Path],
password: str,
) -> Path:
dst = make_dst()
dst.write_bytes(b"password removed")
return dst
@mock.patch("documents.tasks.consume_file.s")
def test_merge(self, mock_consume_file) -> None:
@@ -850,12 +851,12 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(result, "OK")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.PdfMerger")
@mock.patch("documents.tasks.consume_file.s")
def test_merge_uses_latest_version_source_for_root_selection(
self,
mock_consume_file,
mock_open_pdf,
mock_merger,
) -> None:
version_file = self.dirs.scratch_dir / "sample2_version_merge.pdf"
shutil.copy(self.doc2.source_path, version_file)
@@ -866,16 +867,14 @@ class TestPDFActions(DirectoriesMixin, TestCase):
filename=version_file,
mime_type="application/pdf",
)
fake_pdf = mock.MagicMock()
fake_pdf.pdf_version = "1.7"
fake_pdf.pages = [mock.Mock()]
mock_open_pdf.return_value.__enter__.return_value = fake_pdf
merger = mock_merger.return_value.__enter__.return_value
merger.save.side_effect = lambda dst: shutil.copy(version.source_path, dst)
result = bulk_edit.merge([self.doc2.id])
self.assertEqual(result, "OK")
mock_open_pdf.assert_called_once_with(str(version.source_path))
mock_consume_file.assert_not_called()
merger.add.assert_called_once_with(version.source_path)
mock_consume_file.assert_called_once()
@mock.patch("documents.bulk_edit.delete.si")
@mock.patch("documents.tasks.consume_file.s")
@@ -1037,18 +1036,18 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
@mock.patch("documents.tasks.consume_file.delay")
@mock.patch("pikepdf.open")
def test_merge_with_errors(self, mock_open_pdf, mock_consume_file) -> None:
@mock.patch("documents.pdf_ops.PdfMerger.add")
def test_merge_with_errors(self, mock_add, mock_consume_file) -> None:
"""
GIVEN:
- Existing documents
WHEN:
- Merge action is called with 2 documents
- Error occurs when opening both files
- Error occurs when adding both files
THEN:
- Consume file should not be called
"""
mock_open_pdf.side_effect = Exception("Error opening PDF")
mock_add.side_effect = Exception("Error opening PDF")
doc_ids = [self.doc2.id, self.doc3.id]
with self.assertLogs("paperless.bulk_edit", level="ERROR") as cm:
@@ -1085,12 +1084,12 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(result, "OK")
@mock.patch("documents.bulk_edit.group")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.build_pdfs")
@mock.patch("documents.tasks.consume_file.s")
def test_split_uses_latest_version_source_for_root_selection(
self,
mock_consume_file,
mock_open_pdf,
mock_build_pdfs,
mock_group,
) -> None:
version_file = self.dirs.scratch_dir / "sample2_version_split.pdf"
@@ -1102,17 +1101,15 @@ class TestPDFActions(DirectoriesMixin, TestCase):
filename=version_file,
mime_type="application/pdf",
)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock(), mock.Mock()]
mock_open_pdf.return_value.__enter__.return_value = fake_pdf
mock_build_pdfs.return_value = [version.source_path, version.source_path]
mock_group.return_value.delay.return_value = None
result = bulk_edit.split([self.doc2.id], [[1], [2]])
self.assertEqual(result, "OK")
mock_open_pdf.assert_called_once_with(version.source_path)
mock_consume_file.assert_not_called()
mock_group.return_value.delay.assert_not_called()
self.assertEqual(mock_build_pdfs.call_args.args[0], version.source_path)
self.assertEqual(mock_consume_file.call_count, 2)
mock_group.return_value.delay.assert_called_once()
@mock.patch("documents.bulk_edit.delete.si")
@mock.patch("documents.tasks.consume_file.s")
@@ -1200,18 +1197,18 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(self.doc2.archive_serial_number, 222)
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.Pdf.save")
def test_split_with_errors(self, mock_save_pdf, mock_consume_file) -> None:
@mock.patch("documents.pdf_ops.build_pdfs")
def test_split_with_errors(self, mock_build_pdfs, mock_consume_file) -> None:
"""
GIVEN:
- Existing documents
WHEN:
- Split action is called with 1 document and 2 page groups
- Error occurs when saving the files
- Error occurs when building the files
THEN:
- Consume file should not be called
"""
mock_save_pdf.side_effect = Exception("Error saving PDF")
mock_build_pdfs.side_effect = Exception("Error building PDFs")
doc_ids = [self.doc2.id]
pages = [[1, 2], [3]]
@@ -1246,10 +1243,10 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(result, "OK")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.Pdf.save")
@mock.patch("documents.pdf_ops.rotate_pdf")
def test_rotate_with_error(
self,
mock_pdf_save,
mock_rotate_pdf,
mock_consume_delay,
) -> None:
"""
@@ -1257,11 +1254,11 @@ class TestPDFActions(DirectoriesMixin, TestCase):
- Existing documents
WHEN:
- Rotate action is called with 2 documents
- PikePDF raises an error
- Rotating the PDF raises an error
THEN:
- Rotate action should be called 0 times
"""
mock_pdf_save.side_effect = Exception("Error saving PDF")
mock_rotate_pdf.side_effect = Exception("Error rotating PDF")
doc_ids = [self.doc2.id, self.doc3.id]
with self.assertLogs("paperless.bulk_edit", level="ERROR") as cm:
@@ -1296,10 +1293,10 @@ class TestPDFActions(DirectoriesMixin, TestCase):
@mock.patch("documents.data_models.magic.from_file", return_value="application/pdf")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.rotate_pdf")
def test_rotate_explicit_selection_uses_root_source_when_root_selected(
self,
mock_open,
mock_rotate_pdf,
mock_consume_delay,
mock_magic,
) -> None:
@@ -1308,9 +1305,6 @@ class TestPDFActions(DirectoriesMixin, TestCase):
title="B version 1",
root_document=self.doc2,
)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock()]
mock_open.return_value.__enter__.return_value = fake_pdf
result = bulk_edit.rotate(
[self.doc2.id],
@@ -1319,26 +1313,35 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
mock_open.assert_called_once_with(self.doc2.source_path)
self.assertEqual(mock_rotate_pdf.call_args.args[0], self.doc2.source_path)
mock_consume_delay.assert_called_once()
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.Pdf.save")
@mock.patch("documents.pdf_ops.remove_pages")
@mock.patch("documents.data_models.magic.from_file", return_value="application/pdf")
def test_delete_pages(self, mock_magic, mock_pdf_save, mock_consume_delay) -> None:
def test_delete_pages(
self,
mock_magic,
mock_remove_pages,
mock_consume_delay,
) -> None:
"""
GIVEN:
- Existing documents
WHEN:
- Delete pages action is called with 1 document and 2 pages
THEN:
- Save should be called once
- The pages are removed from the document's source PDF
- A new version should be enqueued via consume_file
"""
doc_ids = [self.doc2.id]
pages = [1, 3]
result = bulk_edit.delete_pages(doc_ids, pages)
mock_pdf_save.assert_called_once()
mock_remove_pages.assert_called_once_with(
self.doc2.source_path,
mock.ANY,
[1, 3],
)
mock_consume_delay.assert_called_once()
task_kwargs = mock_consume_delay.call_args.kwargs["kwargs"]
self.assertEqual(task_kwargs["input_doc"].root_document_id, self.doc2.id)
@@ -1350,10 +1353,10 @@ class TestPDFActions(DirectoriesMixin, TestCase):
@mock.patch("documents.data_models.magic.from_file", return_value="application/pdf")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.remove_pages")
def test_delete_pages_explicit_selection_uses_root_source_when_root_selected(
self,
mock_open,
mock_remove_pages,
mock_consume_delay,
mock_magic,
) -> None:
@@ -1362,9 +1365,6 @@ class TestPDFActions(DirectoriesMixin, TestCase):
title="B version 1",
root_document=self.doc2,
)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock(), mock.Mock()]
mock_open.return_value.__enter__.return_value = fake_pdf
result = bulk_edit.delete_pages(
[self.doc2.id],
@@ -1373,23 +1373,26 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
mock_open.assert_called_once_with(self.doc2.source_path)
self.assertEqual(mock_remove_pages.call_args.args[0], self.doc2.source_path)
mock_consume_delay.assert_called_once()
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.Pdf.save")
def test_delete_pages_with_error(self, mock_pdf_save, mock_consume_delay) -> None:
@mock.patch("documents.pdf_ops.remove_pages")
def test_delete_pages_with_error(
self,
mock_remove_pages,
mock_consume_delay,
) -> None:
"""
GIVEN:
- Existing documents
WHEN:
- Delete pages action is called with 1 document and 2 pages
- PikePDF raises an error
- Removing the pages raises an error
THEN:
- Save should be called once
- No new version should be enqueued
"""
mock_pdf_save.side_effect = Exception("Error saving PDF")
mock_remove_pages.side_effect = Exception("Error removing pages")
doc_ids = [self.doc2.id]
pages = [1, 3]
@@ -1419,6 +1422,41 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(result, "OK")
mock_group.return_value.delay.assert_called_once()
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.pdf_ops.build_pdfs")
@mock.patch("documents.tasks.consume_file.s")
def test_edit_pdf_maps_operations_to_outputs(
self,
mock_consume_file: mock.Mock,
mock_build_pdfs: mock.Mock,
mock_group: mock.Mock,
) -> None:
"""
GIVEN:
- Existing document
WHEN:
- edit_pdf is called with operations interleaved across two outputs,
some of them rotated
THEN:
- Each output is built from its own operations, in operation order,
with the requested rotation
"""
mock_build_pdfs.return_value = [self.doc2.source_path, self.doc2.source_path]
mock_group.return_value.delay.return_value = None
operations = [
{"page": 3, "doc": 1},
{"page": 1, "doc": 0, "rotate": 90},
{"page": 2, "doc": 1, "rotate": 180},
]
bulk_edit.edit_pdf([self.doc2.id], operations)
outputs = mock_build_pdfs.call_args.args[1]
self.assertEqual(
[specs for specs, _ in outputs],
[[PageSpec(1, 90)], [PageSpec(3), PageSpec(2, 180)]],
)
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
def test_edit_pdf_with_user_override(self, mock_consume_file, mock_group) -> None:
@@ -1534,12 +1572,10 @@ class TestPDFActions(DirectoriesMixin, TestCase):
@mock.patch("documents.data_models.magic.from_file", return_value="application/pdf")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.new")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.build_pdfs")
def test_edit_pdf_explicit_selection_uses_root_source_when_root_selected(
self,
mock_open,
mock_new,
mock_build_pdfs,
mock_consume_delay,
mock_magic,
) -> None:
@@ -1548,12 +1584,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
title="B version 1",
root_document=self.doc2,
)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock()]
mock_open.return_value.__enter__.return_value = fake_pdf
output_pdf = mock.MagicMock()
output_pdf.pages = []
mock_new.return_value = output_pdf
mock_build_pdfs.return_value = [Path("edited.pdf")]
result = bulk_edit.edit_pdf(
[self.doc2.id],
@@ -1563,7 +1594,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
mock_open.assert_called_once_with(self.doc2.source_path)
self.assertEqual(mock_build_pdfs.call_args.args[0], self.doc2.source_path)
mock_consume_delay.assert_called_once()
@mock.patch("documents.bulk_edit.group")
@@ -1589,31 +1620,6 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(result, "OK")
mock_group.return_value.delay.assert_called_once()
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
def test_edit_pdf_open_failure(
self,
mock_consume_file: mock.Mock,
mock_group: mock.Mock,
) -> None:
"""
GIVEN:
- Existing document
WHEN:
- edit_pdf fails to open PDF
THEN:
- Task group is not called
"""
doc_ids = [self.doc2.id]
operations = [
{"page": 9999}, # invalid page, forces error during PDF load
]
with self.assertLogs("paperless.bulk_edit", level="ERROR"):
with self.assertRaises(Exception):
bulk_edit.edit_pdf(doc_ids, operations)
mock_group.assert_not_called()
mock_consume_file.assert_not_called()
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
def test_edit_pdf_multiple_outputs_with_update_flag_errors(
@@ -1640,23 +1646,15 @@ class TestPDFActions(DirectoriesMixin, TestCase):
mock_group.assert_not_called()
mock_consume_file.assert_not_called()
@mock.patch("pikepdf.open")
def test_edit_pdf_rejects_invalid_operations(self, mock_open) -> None:
for operations in ([], [{"page": 1, "doc": 2**32}]):
with self.subTest(operations=operations):
with self.assertLogs("paperless.bulk_edit", level="ERROR"):
with self.assertRaisesRegex(ValueError, "index is out of bounds"):
bulk_edit.edit_pdf([self.doc2.id], operations)
mock_open.assert_not_called()
@mock.patch("documents.bulk_edit.update_document_content_maybe_archive_file.delay")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("documents.bulk_edit.tempfile.mkdtemp")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.decrypt_pdf")
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=True)
def test_remove_password_update_document(
self,
mock_open,
mock_needs_decrypt,
mock_decrypt,
mock_mkdtemp,
mock_consume_delay,
mock_update_document,
@@ -1665,16 +1663,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
temp_dir = self.dirs.scratch_dir / "remove-password-update"
temp_dir.mkdir(parents=True, exist_ok=True)
mock_mkdtemp.return_value = str(temp_dir)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock(), mock.Mock(), mock.Mock()]
fake_pdf.is_encrypted = True
def save_side_effect(target_path):
Path(target_path).write_bytes(b"new pdf content")
fake_pdf.save.side_effect = save_side_effect
mock_open.return_value.__enter__.return_value = fake_pdf
mock_decrypt.side_effect = self.fake_decrypt
result = bulk_edit.remove_password(
[doc.id],
@@ -1683,14 +1672,8 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
self.assertEqual(
mock_open.call_args_list,
[
mock.call(doc.source_path),
mock.call(doc.source_path, password="secret"),
],
)
fake_pdf.remove_unreferenced_resources.assert_called_once()
mock_needs_decrypt.assert_called_once_with(doc.source_path)
mock_decrypt.assert_called_once_with(doc.source_path, mock.ANY, "secret")
mock_update_document.assert_not_called()
mock_consume_delay.assert_called_once()
task_kwargs = mock_consume_delay.call_args.kwargs["kwargs"]
@@ -1703,40 +1686,15 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertEqual(task_kwargs["input_doc"].root_document_id, doc.id)
self.assertIsNotNone(task_kwargs["overrides"])
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("documents.bulk_edit.tempfile.mkdtemp")
@mock.patch("pikepdf.open")
def test_remove_password_update_document_skips_unencrypted_pdf(
self,
mock_open,
mock_mkdtemp,
mock_consume_delay,
) -> None:
doc = self.doc1
fake_pdf = mock.MagicMock()
fake_pdf.is_encrypted = False
mock_open.return_value.__enter__.return_value = fake_pdf
result = bulk_edit.remove_password(
[doc.id],
password="secret",
update_document=True,
)
self.assertEqual(result, "OK")
mock_open.assert_called_once_with(doc.source_path)
fake_pdf.remove_unreferenced_resources.assert_not_called()
fake_pdf.save.assert_not_called()
mock_mkdtemp.assert_not_called()
mock_consume_delay.assert_not_called()
@mock.patch("documents.bulk_edit.update_document_content_maybe_archive_file.delay")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("documents.bulk_edit.tempfile.mkdtemp")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.decrypt_pdf")
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=True)
def test_remove_password_update_document_uses_source_paths(
self,
mock_open,
mock_needs_decrypt,
mock_decrypt,
mock_mkdtemp,
mock_consume_delay,
mock_update_document,
@@ -1747,14 +1705,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
temp_dir = self.dirs.scratch_dir / "remove-password-source-file"
temp_dir.mkdir(parents=True, exist_ok=True)
mock_mkdtemp.return_value = str(temp_dir)
fake_pdf = mock.MagicMock()
self.mock_password_required_pdf(mock_open, fake_pdf)
def save_side_effect(target_path):
Path(target_path).write_bytes(b"new pdf content")
fake_pdf.save.side_effect = save_side_effect
mock_decrypt.side_effect = self.fake_decrypt
result = bulk_edit.remove_password(
[doc.id],
@@ -1764,22 +1715,19 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
self.assertEqual(
mock_open.call_args_list,
[
mock.call(source_file),
mock.call(source_file, password="secret"),
],
)
mock_needs_decrypt.assert_called_once_with(source_file)
mock_decrypt.assert_called_once_with(source_file, mock.ANY, "secret")
mock_update_document.assert_not_called()
mock_consume_delay.assert_called_once()
@mock.patch("documents.data_models.magic.from_file", return_value="application/pdf")
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.decrypt_pdf")
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=True)
def test_remove_password_explicit_selection_uses_root_source_when_root_selected(
self,
mock_open,
mock_needs_decrypt,
mock_decrypt,
mock_consume_delay,
mock_magic,
) -> None:
@@ -1788,8 +1736,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
title="A version 1",
root_document=self.doc1,
)
fake_pdf = mock.MagicMock()
self.mock_password_required_pdf(mock_open, fake_pdf)
mock_decrypt.return_value = Path("unprotected.pdf")
result = bulk_edit.remove_password(
[self.doc1.id],
@@ -1799,12 +1746,11 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
self.assertEqual(
mock_open.call_args_list,
[
mock.call(self.doc1.source_path),
mock.call(self.doc1.source_path, password="secret"),
],
mock_needs_decrypt.assert_called_once_with(self.doc1.source_path)
mock_decrypt.assert_called_once_with(
self.doc1.source_path,
mock.ANY,
"secret",
)
mock_consume_delay.assert_called_once()
@@ -1812,10 +1758,12 @@ class TestPDFActions(DirectoriesMixin, TestCase):
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
@mock.patch("documents.bulk_edit.tempfile.mkdtemp")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.decrypt_pdf")
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=True)
def test_remove_password_creates_consumable_document(
self,
mock_open: mock.Mock,
mock_needs_decrypt: mock.Mock,
mock_decrypt: mock.Mock,
mock_mkdtemp: mock.Mock,
mock_consume_file: mock.Mock,
mock_group: mock.Mock,
@@ -1825,15 +1773,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
temp_dir = self.dirs.scratch_dir / "remove-password"
temp_dir.mkdir(parents=True, exist_ok=True)
mock_mkdtemp.return_value = str(temp_dir)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock(), mock.Mock()]
self.mock_password_required_pdf(mock_open, fake_pdf)
def save_side_effect(target_path: Path) -> None:
target_path.write_bytes(b"password removed")
fake_pdf.save.side_effect = save_side_effect
mock_decrypt.side_effect = self.fake_decrypt
mock_group.return_value.delay.return_value = None
user = User.objects.create(username="owner")
@@ -1848,13 +1788,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
self.assertEqual(
mock_open.call_args_list,
[
mock.call(doc.source_path),
mock.call(doc.source_path, password="secret"),
],
)
mock_decrypt.assert_called_once_with(doc.source_path, mock.ANY, "secret")
mock_consume_file.assert_called_once()
call_kwargs = mock_consume_file.call_args.kwargs
consumable_document = call_kwargs["input_doc"]
@@ -1876,21 +1810,18 @@ class TestPDFActions(DirectoriesMixin, TestCase):
@mock.patch("documents.bulk_edit.chord")
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
@mock.patch("documents.bulk_edit.tempfile.mkdtemp")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.decrypt_pdf")
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=False)
def test_remove_password_skips_unencrypted_pdf_without_queueing(
self,
mock_open: mock.Mock,
mock_mkdtemp: mock.Mock,
mock_needs_decrypt: mock.Mock,
mock_decrypt: mock.Mock,
mock_consume_file: mock.Mock,
mock_group: mock.Mock,
mock_chord: mock.Mock,
mock_delete: mock.Mock,
) -> None:
doc = self.doc2
fake_pdf = mock.MagicMock()
fake_pdf.is_encrypted = False
mock_open.return_value.__enter__.return_value = fake_pdf
result = bulk_edit.remove_password(
[doc.id],
@@ -1900,10 +1831,8 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
mock_open.assert_called_once_with(doc.source_path)
fake_pdf.remove_unreferenced_resources.assert_not_called()
fake_pdf.save.assert_not_called()
mock_mkdtemp.assert_not_called()
mock_needs_decrypt.assert_called_once_with(doc.source_path)
mock_decrypt.assert_not_called()
mock_consume_file.assert_not_called()
mock_group.assert_not_called()
mock_chord.assert_not_called()
@@ -1914,10 +1843,12 @@ class TestPDFActions(DirectoriesMixin, TestCase):
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
@mock.patch("documents.bulk_edit.tempfile.mkdtemp")
@mock.patch("pikepdf.open")
@mock.patch("documents.pdf_ops.decrypt_pdf")
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=True)
def test_remove_password_deletes_original(
self,
mock_open: mock.Mock,
mock_needs_decrypt: mock.Mock,
mock_decrypt: mock.Mock,
mock_mkdtemp: mock.Mock,
mock_consume_file: mock.Mock,
mock_group: mock.Mock,
@@ -1928,15 +1859,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
temp_dir = self.dirs.scratch_dir / "remove-password-delete"
temp_dir.mkdir(parents=True, exist_ok=True)
mock_mkdtemp.return_value = str(temp_dir)
fake_pdf = mock.MagicMock()
fake_pdf.pages = [mock.Mock(), mock.Mock()]
self.mock_password_required_pdf(mock_open, fake_pdf)
def save_side_effect(target_path: Path) -> None:
target_path.write_bytes(b"password removed")
fake_pdf.save.side_effect = save_side_effect
mock_decrypt.side_effect = self.fake_decrypt
mock_chord.return_value.delay.return_value = None
result = bulk_edit.remove_password(
@@ -1948,23 +1871,23 @@ class TestPDFActions(DirectoriesMixin, TestCase):
)
self.assertEqual(result, "OK")
self.assertEqual(
mock_open.call_args_list,
[
mock.call(doc.source_path),
mock.call(doc.source_path, password="secret"),
],
)
mock_decrypt.assert_called_once_with(doc.source_path, mock.ANY, "secret")
mock_consume_file.assert_called_once()
mock_group.assert_not_called()
mock_chord.assert_called_once()
mock_chord.return_value.delay.assert_called_once()
mock_delete.si.assert_called_once_with([doc.id])
@mock.patch("pikepdf.open")
def test_remove_password_open_failure(self, mock_open: mock.Mock) -> None:
mock_open.side_effect = RuntimeError("wrong password")
@mock.patch(
"documents.pdf_ops.decrypt_pdf",
side_effect=RuntimeError("wrong password"),
)
@mock.patch("documents.pdf_ops.needs_decrypt", return_value=True)
def test_remove_password_failure_raises_value_error(
self,
mock_needs_decrypt: mock.Mock,
mock_decrypt: mock.Mock,
) -> None:
with self.assertLogs("paperless.bulk_edit", level="ERROR") as cm:
with self.assertRaises(ValueError) as exc:
bulk_edit.remove_password([self.doc1.id], password="secret")
@@ -1973,6 +1896,111 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertIn("Error removing password from document", cm.output[0])
@pytest.mark.django_db
class TestBulkEditRunZoneOcr:
@pytest.fixture
def mock_task(self, mocker: MockerFixture) -> mock.MagicMock:
return mocker.patch("documents.bulk_edit.extract_zones")
def test_queues_every_selected_document(
self,
mock_task: mock.MagicMock,
) -> None:
"""
GIVEN:
- Selected documents
WHEN:
- run_zone_ocr is called with them
THEN:
- Each document is queued once, templates are matched when its
task runs
"""
docs = DocumentFactory.create_batch(2)
result = bulk_edit.run_zone_ocr([doc.id for doc in docs])
assert result == "OK"
assert [
call.kwargs["kwargs"] for call in mock_task.apply_async.call_args_list
] == [{"document_id": doc.id, "template_id": None} for doc in docs]
def test_queues_the_chosen_template(self, mock_task: mock.MagicMock) -> None:
"""
GIVEN:
- A chosen OCR template
WHEN:
- run_zone_ocr is called with it
THEN:
- The template is queued with the document
"""
doc = DocumentFactory()
template = OcrTemplate.objects.create(
name="Bills",
source_width=100,
source_height=100,
)
bulk_edit.run_zone_ocr([doc.id], template=template)
assert mock_task.apply_async.call_args.kwargs["kwargs"] == {
"document_id": doc.id,
"template_id": template.pk,
}
def test_task_runs_extraction(self, mocker: MockerFixture) -> None:
"""
GIVEN:
- A queued zone OCR task
WHEN:
- The task runs
THEN:
- Zone extraction runs on the stored document
"""
extract = mocker.patch("documents.tasks.run_zone_extraction")
doc = DocumentFactory()
extract_zones(doc.id)
extract.assert_called_once_with(doc, None)
def test_task_runs_the_chosen_template(self, mocker: MockerFixture) -> None:
"""
GIVEN:
- A queued zone OCR task with a chosen template
WHEN:
- The task runs
THEN:
- Zone extraction runs with only that template
"""
extract = mocker.patch("documents.tasks.run_zone_extraction")
doc = DocumentFactory()
template = OcrTemplate.objects.create(
name="Bills",
source_width=100,
source_height=100,
)
extract_zones(doc.id, template.pk)
extract.assert_called_once_with(doc, None, templates=[template])
def test_task_skips_a_deleted_template(self, mocker: MockerFixture) -> None:
"""
GIVEN:
- A template deleted between queueing and running
WHEN:
- The task runs
THEN:
- Nothing is extracted
"""
extract = mocker.patch("documents.tasks.run_zone_extraction")
doc = DocumentFactory()
extract_zones(doc.id, 999)
extract.assert_not_called()
@pytest.mark.django_db
class TestBulkEditReprocess:
@pytest.fixture
+520
View File
@@ -0,0 +1,520 @@
"""
Tests for documents.pdf_ops.
These use real PDFs from the sample directories. No database, Celery or mocks.
Pages are compared by a hash of their content stream, so page identity and order
are easy to assert.
"""
import hashlib
from collections.abc import Callable
from pathlib import Path
import pikepdf
import pytest
from documents import pdf_ops
from documents.pdf_ops import PageSpec
SRC_ROOT = Path(__file__).parents[2]
SAMPLES = Path(__file__).parent / "samples"
THREE_PAGES = SAMPLES / "documents" / "originals" / "0000002.pdf"
TWELVE_PAGES = SAMPLES / "barcodes" / "split-by-asn-2.pdf"
ENCRYPTED = SAMPLES / "password-is-test.pdf"
SIGNED = SRC_ROOT / "paperless" / "tests" / "samples" / "tesseract" / "signed.pdf"
def _page_fingerprint(page: pikepdf.Page) -> str:
contents = page.obj.get("/Contents")
assert contents is not None, "sample page has no /Contents"
streams = list(contents) if isinstance(contents, pikepdf.Array) else [contents]
return hashlib.sha256(b"".join(s.read_bytes() for s in streams)).hexdigest()
def fingerprints(path: Path) -> list[str]:
with pikepdf.open(path) as pdf:
return [_page_fingerprint(page) for page in pdf.pages]
def rotations(path: Path) -> list[int]:
with pikepdf.open(path) as pdf:
return [int(page.obj.get("/Rotate", 0)) for page in pdf.pages]
def docinfo_keys(path: Path) -> set[str]:
with pikepdf.open(path) as pdf:
return set(pdf.docinfo.keys())
def constant(path: Path) -> Callable[[], Path]:
return lambda: path
@pytest.fixture
def source_fingerprints() -> list[str]:
fps = fingerprints(THREE_PAGES)
assert len(set(fps)) == 3, "sample must have three distinct pages"
return fps
class TestRotatePdf:
def test_rotation_is_relative_and_applies_to_every_page(
self,
tmp_path: Path,
) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- It is rotated by 90 degrees, then the result is rotated by 90 again
THEN:
- Every page is rotated relative to its current rotation
- The page content itself is unchanged
"""
once = tmp_path / "once.pdf"
twice = tmp_path / "twice.pdf"
pdf_ops.rotate_pdf(THREE_PAGES, once, 90)
pdf_ops.rotate_pdf(once, twice, 90)
assert rotations(once) == [90, 90, 90]
assert rotations(twice) == [180, 180, 180]
assert fingerprints(twice) == fingerprints(THREE_PAGES)
def test_keeps_document_info(self, tmp_path: Path) -> None:
"""
GIVEN:
- A PDF with document info
WHEN:
- It is rotated
THEN:
- The document info is still present in the output
"""
dst = tmp_path / "out.pdf"
pdf_ops.rotate_pdf(THREE_PAGES, dst, 90)
assert "/Creator" in docinfo_keys(dst)
class TestRemovePages:
@pytest.mark.parametrize(
("pages", "kept"),
[
pytest.param([2], [0, 2], id="single"),
pytest.param([3, 1], [1], id="unordered"),
pytest.param([2, 2], [0, 2], id="duplicates-remove-once"),
pytest.param([], [0, 1, 2], id="empty-keeps-everything"),
pytest.param([1, 2, 3], [], id="every-page"),
],
)
def test_removes_only_the_requested_pages(
self,
tmp_path: Path,
source_fingerprints: list[str],
pages: list[int],
kept: list[int],
) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- Pages are removed, in any order and possibly repeated
THEN:
- Exactly the other pages remain, in their original order
"""
dst = tmp_path / "out.pdf"
pdf_ops.remove_pages(THREE_PAGES, dst, pages)
assert fingerprints(dst) == [source_fingerprints[i] for i in kept]
def test_keeps_document_info(self, tmp_path: Path) -> None:
"""
GIVEN:
- A PDF with document info
WHEN:
- A page is removed
THEN:
- The document info is still present in the output
"""
dst = tmp_path / "out.pdf"
pdf_ops.remove_pages(THREE_PAGES, dst, [1])
assert "/Creator" in docinfo_keys(dst)
@pytest.mark.parametrize(
"bad_page",
[
pytest.param(0, id="zero"),
pytest.param(-1, id="negative"),
],
)
def test_rejects_pages_below_one(self, tmp_path: Path, bad_page: int) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- Pages are removed and one of them is below 1
THEN:
- ValueError is raised
- No output file is written
"""
dst = tmp_path / "out.pdf"
with pytest.raises(ValueError, match="start at 1"):
pdf_ops.remove_pages(THREE_PAGES, dst, [1, bad_page])
assert not dst.exists()
class TestBuildPdfs:
def test_selects_and_orders_pages(
self,
tmp_path: Path,
source_fingerprints: list[str],
) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- One output is built from pages 3 then 1
THEN:
- The output has those pages in that order
- Its path is returned
"""
dst = tmp_path / "out.pdf"
written = pdf_ops.build_pdfs(
THREE_PAGES,
[([PageSpec(3), PageSpec(1)], constant(dst))],
)
assert written == [dst]
assert fingerprints(dst) == [source_fingerprints[2], source_fingerprints[0]]
def test_rotates_only_the_requested_pages(self, tmp_path: Path) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- One output is built with a different rotation on each page
THEN:
- Each output page has exactly the rotation requested for it
"""
dst = tmp_path / "out.pdf"
pdf_ops.build_pdfs(
THREE_PAGES,
[([PageSpec(1), PageSpec(2, 90), PageSpec(3, 180)], constant(dst))],
)
assert rotations(dst) == [0, 90, 180]
def test_writes_one_file_per_output_in_order(self, tmp_path: Path) -> None:
"""
GIVEN:
- A twelve page PDF
WHEN:
- Two outputs are built from different page ranges
THEN:
- Two files are written and returned in output order
- Each holds exactly its own pages
"""
first = tmp_path / "first.pdf"
second = tmp_path / "second.pdf"
source = fingerprints(TWELVE_PAGES)
written = pdf_ops.build_pdfs(
TWELVE_PAGES,
[
([PageSpec(p) for p in (1, 2, 3)], constant(first)),
([PageSpec(p) for p in range(4, 13)], constant(second)),
],
)
assert written == [first, second]
assert fingerprints(first) == source[:3]
assert fingerprints(second) == source[3:]
def test_empty_page_list_writes_a_zero_page_file(self, tmp_path: Path) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- An output with no pages is built
THEN:
- A PDF with zero pages is written
"""
dst = tmp_path / "out.pdf"
pdf_ops.build_pdfs(THREE_PAGES, [([], constant(dst))])
assert fingerprints(dst) == []
def test_destination_is_not_requested_when_a_page_is_out_of_range(
self,
tmp_path: Path,
) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- Several outputs are built
- A later output refers to a page past the end
THEN:
- IndexError is raised
- No destination was requested for any output, including earlier valid ones
"""
requested: list[Path] = []
def make_dst() -> Path:
requested.append(tmp_path / "out.pdf")
return requested[-1]
with pytest.raises(IndexError):
pdf_ops.build_pdfs(
THREE_PAGES,
[([PageSpec(1)], make_dst), ([PageSpec(99)], make_dst)],
)
assert requested == []
@pytest.mark.parametrize(
"bad_page",
[
pytest.param(0, id="zero"),
pytest.param(-1, id="negative"),
],
)
def test_rejects_pages_below_one_before_opening_anything(
self,
tmp_path: Path,
bad_page: int,
) -> None:
"""
GIVEN:
- A three page PDF
WHEN:
- Several outputs are built and a later one has a page below 1
THEN:
- ValueError is raised
- No destination was requested for any output
"""
requested: list[Path] = []
def make_dst() -> Path:
requested.append(tmp_path / "out.pdf")
return requested[-1]
with pytest.raises(ValueError, match="start at 1"):
pdf_ops.build_pdfs(
THREE_PAGES,
[([PageSpec(1)], make_dst), ([PageSpec(bad_page)], make_dst)],
)
assert requested == []
class TestValidatePageOperations:
def test_returns_the_output_count(self) -> None:
"""
GIVEN:
- Operations that all target the default output
WHEN:
- They are validated
THEN:
- One output document is reported
"""
operations = [{"page": 1}, {"page": 2}, {"page": 3}]
assert pdf_ops.validate_page_operations(operations, single_output=True) == 1
def test_gap_in_output_indices_counts_up_to_the_highest(self) -> None:
"""
GIVEN:
- Operations that target outputs 0 and 2 but never 1
WHEN:
- They are validated
THEN:
- Three output documents are reported
"""
operations = [
{"page": 1, "doc": 0},
{"page": 2, "doc": 2},
{"page": 3, "doc": 0},
]
count = pdf_ops.validate_page_operations(operations, single_output=False)
assert count == 3
def test_empty_operations_are_rejected(self) -> None:
"""
GIVEN:
- No operations
WHEN:
- They are validated
THEN:
- ValueError is raised
"""
with pytest.raises(ValueError, match="index is out of bounds"):
pdf_ops.validate_page_operations([], single_output=False)
def test_multiple_outputs_rejected_when_single_output_required(self) -> None:
"""
GIVEN:
- Operations that target two outputs
WHEN:
- They are validated with a single output required
THEN:
- ValueError is raised
"""
operations = [{"page": 1, "doc": 0}, {"page": 2, "doc": 1}]
with pytest.raises(ValueError, match="Multiple output documents"):
pdf_ops.validate_page_operations(operations, single_output=True)
@pytest.mark.parametrize(
"doc",
[
pytest.param(-1, id="negative"),
pytest.param(2, id="equal-to-operation-count"),
pytest.param(2**32, id="huge"),
],
)
def test_output_index_out_of_bounds(self, doc: int) -> None:
"""
GIVEN:
- Two operations, one with an output index that is out of bounds
WHEN:
- They are validated
THEN:
- ValueError is raised
"""
operations = [{"page": 1, "doc": 0}, {"page": 2, "doc": doc}]
with pytest.raises(ValueError, match="index is out of bounds"):
pdf_ops.validate_page_operations(operations, single_output=False)
class TestDecrypt:
@pytest.mark.parametrize(
("path", "expected"),
[
pytest.param(ENCRYPTED, True, id="password-required"),
pytest.param(SIGNED, True, id="opens-without-password-but-encrypted"),
pytest.param(THREE_PAGES, False, id="not-encrypted"),
],
)
def test_needs_decrypt(self, path: Path, *, expected: bool) -> None:
"""
GIVEN:
- A PDF that is encrypted, or encrypted but openable, or plain
WHEN:
- needs_decrypt is asked about it
THEN:
- Only the unencrypted PDF reports False
"""
assert pdf_ops.needs_decrypt(path) is expected
def test_decrypt_writes_an_unencrypted_copy(self, tmp_path: Path) -> None:
"""
GIVEN:
- A password protected PDF
WHEN:
- It is decrypted with the correct password
THEN:
- The path from make_dst is returned
- The written copy no longer needs decrypting
"""
dst = tmp_path / "out.pdf"
result = pdf_ops.decrypt_pdf(ENCRYPTED, constant(dst), "test")
assert result == dst
assert pdf_ops.needs_decrypt(dst) is False
def test_wrong_password_raises_and_never_requests_a_destination(
self,
tmp_path: Path,
) -> None:
"""
GIVEN:
- A password protected PDF
WHEN:
- It is decrypted with the wrong password
THEN:
- PasswordError is raised
- No destination was requested
"""
requested: list[Path] = []
def make_dst() -> Path:
requested.append(tmp_path / "out.pdf")
return requested[-1]
with pytest.raises(pikepdf.PasswordError):
pdf_ops.decrypt_pdf(ENCRYPTED, make_dst, "wrong")
assert requested == []
class TestPdfMerger:
def test_pages_are_appended_in_the_order_added(
self,
tmp_path: Path,
source_fingerprints: list[str],
) -> None:
"""
GIVEN:
- A reordered PDF and the original three page PDF
WHEN:
- Both are added to a merger in that order and saved
THEN:
- The output holds all pages in the order they were added
"""
reordered = tmp_path / "reordered.pdf"
merged = tmp_path / "merged.pdf"
pdf_ops.build_pdfs(
THREE_PAGES,
[([PageSpec(3), PageSpec(1)], constant(reordered))],
)
with pdf_ops.PdfMerger() as merger:
merger.add(reordered)
merger.add(THREE_PAGES)
merger.save(merged)
assert fingerprints(merged) == [
source_fingerprints[2],
source_fingerprints[0],
*source_fingerprints,
]
def test_output_version_is_at_least_the_highest_source_version(
self,
tmp_path: Path,
) -> None:
"""
GIVEN:
- Two PDFs with different PDF versions
WHEN:
- Both are added to a merger and saved
THEN:
- The output version is at least the highest source version
"""
merged = tmp_path / "merged.pdf"
with pikepdf.open(TWELVE_PAGES) as pdf:
source_versions = [pdf.pdf_version]
with pikepdf.open(THREE_PAGES) as pdf:
source_versions.append(pdf.pdf_version)
with pdf_ops.PdfMerger() as merger:
merger.add(TWELVE_PAGES)
merger.add(THREE_PAGES)
merger.save(merged)
with pikepdf.open(merged) as pdf:
assert pdf.pdf_version >= max(source_versions)
+18 -1
View File
@@ -4,6 +4,7 @@ from django.conf import settings
from pytest_mock import MockerFixture
from documents.regex import safe_regex_finditer
from documents.regex import safe_regex_fullmatch
from documents.regex import safe_regex_match
from documents.regex import safe_regex_search
from documents.regex import safe_regex_sub
@@ -26,7 +27,7 @@ class TestValidateRegexPattern:
class TestSafeRegexSearchAndMatch:
"""Tests for safe_regex_search and safe_regex_match (same contract)."""
"""Tests for safe_regex_search, _match and _fullmatch (same contract)."""
@pytest.mark.parametrize(
("func", "pattern", "text", "expected_group"),
@@ -45,6 +46,13 @@ class TestSafeRegexSearchAndMatch:
"123",
id="match-match-found",
),
pytest.param(
safe_regex_fullmatch,
r"\d+",
"123",
"123",
id="fullmatch-match-found",
),
],
)
def test_match_found(self, func, pattern, text, expected_group) -> None:
@@ -57,6 +65,12 @@ class TestSafeRegexSearchAndMatch:
[
pytest.param(safe_regex_search, r"\d+", "abcdef", id="search-no-match"),
pytest.param(safe_regex_match, r"\d+", "abc123", id="match-no-match"),
pytest.param(
safe_regex_fullmatch,
r"\d+",
"123abc",
id="fullmatch-no-match",
),
],
)
def test_no_match(self, func, pattern, text) -> None:
@@ -67,6 +81,7 @@ class TestSafeRegexSearchAndMatch:
[
pytest.param(safe_regex_search, id="search"),
pytest.param(safe_regex_match, id="match"),
pytest.param(safe_regex_fullmatch, id="fullmatch"),
],
)
def test_invalid_pattern_returns_none(self, func) -> None:
@@ -77,6 +92,7 @@ class TestSafeRegexSearchAndMatch:
[
pytest.param(safe_regex_search, id="search"),
pytest.param(safe_regex_match, id="match"),
pytest.param(safe_regex_fullmatch, id="fullmatch"),
],
)
def test_flags_respected(self, func) -> None:
@@ -87,6 +103,7 @@ class TestSafeRegexSearchAndMatch:
[
pytest.param(safe_regex_search, "search", id="search"),
pytest.param(safe_regex_match, "match", id="match"),
pytest.param(safe_regex_fullmatch, "fullmatch", id="fullmatch"),
],
)
def test_timeout_returns_none(
+38
View File
@@ -13,14 +13,52 @@ from documents.data_models import DocumentMetadataOverrides
from documents.data_models import DocumentSource
from documents.models import PaperlessTask
from documents.signals.handlers import before_task_publish_handler
from documents.signals.handlers import run_zone_ocr_extraction
from documents.signals.handlers import task_failure_handler
from documents.signals.handlers import task_postrun_handler
from documents.signals.handlers import task_prerun_handler
from documents.signals.handlers import task_revoked_handler
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import PaperlessTaskFactory
from paperless_testing.factories import UserFactory
@pytest.mark.django_db
@pytest.mark.parametrize(
"original_file",
[None, "/tmp/original.pdf", Path("/tmp/original.pdf")],
)
def test_consumption_runs_zone_extraction(
original_file,
mocker: pytest_mock.MockerFixture,
):
extract = mocker.patch("documents.zone_ocr.run_zone_extraction")
document = DocumentFactory()
run_zone_ocr_extraction(sender=None, document=document, original_file=original_file)
extract.assert_called_once_with(
document,
Path(original_file) if original_file else None,
)
@pytest.mark.django_db
def test_zone_extraction_failure_does_not_fail_consumption(
mocker: pytest_mock.MockerFixture,
caplog,
):
mocker.patch(
"documents.zone_ocr.run_zone_extraction",
side_effect=RuntimeError("OCR failed"),
)
document = DocumentFactory()
run_zone_ocr_extraction(sender=None, document=document)
assert f"Zone OCR extraction failed for document {document.pk}" in caplog.text
@pytest.fixture
def consume_input_doc():
doc = mock.MagicMock(spec=ConsumableDocument)
File diff suppressed because it is too large. Load diff
+265
View File
@@ -3,11 +3,13 @@ import logging
import os
import platform
import re
import subprocess
import tempfile
import zipfile
from collections import defaultdict
from collections import deque
from collections.abc import Callable
from dataclasses import asdict
from datetime import datetime
from datetime import timedelta
from http import HTTPStatus
@@ -137,6 +139,7 @@ from documents.filters import DocumentFilterSet
from documents.filters import DocumentsOrderingFilter
from documents.filters import DocumentTypeFilterSet
from documents.filters import EffectiveContentFilter
from documents.filters import ObjectFilter
from documents.filters import PaperlessTaskFilterSet
from documents.filters import PermittedObjectsFilter
from documents.filters import ShareLinkBundleFilterSet
@@ -156,6 +159,8 @@ from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentType
from documents.models import Note
from documents.models import OcrTemplate
from documents.models import OcrTemplateZone
from documents.models import PaperlessTask
from documents.models import SavedView
from documents.models import ShareLink
@@ -184,6 +189,7 @@ from documents.permissions import restrict_queryset_to_visible
from documents.permissions import set_permissions_for_objects
from documents.permissions import user_is_unrestricted
from documents.plugins.date_parsing import get_date_parser
from documents.regex import safe_regex_fullmatch
from documents.schema import generate_object_with_permissions_schema
from documents.search import SearchHit
from documents.serialisers import AcknowledgeTasksViewSerializer
@@ -204,11 +210,15 @@ from documents.serialisers import EmailSerializer
from documents.serialisers import MergeDocumentsAsVersionsSerializer
from documents.serialisers import MergeDocumentsSerializer
from documents.serialisers import NotesSerializer
from documents.serialisers import OcrTemplateSerializer
from documents.serialisers import OcrTemplateTestSerializer
from documents.serialisers import PostDocumentSerializer
from documents.serialisers import RemovePasswordDocumentsSerializer
from documents.serialisers import ReprocessDocumentsSerializer
from documents.serialisers import RotateDocumentsSerializer
from documents.serialisers import RunTaskSerializer
from documents.serialisers import RunZoneOcrDocumentsSerializer
from documents.serialisers import RunZoneOcrSerializer
from documents.serialisers import SavedViewSerializer
from documents.serialisers import SearchResultSerializer
from documents.serialisers import SerializerWithPerms
@@ -761,6 +771,19 @@ class EmailDocumentDetailSchema(EmailSerializer):
@extend_schema_view(
list=extend_schema(
parameters=[
OpenApiParameter(
name=name,
type=OpenApiTypes.INT,
many=True,
style="form",
explode=False,
)
for name, field in DocumentFilterSet.base_filters.items()
if isinstance(field, ObjectFilter)
],
),
retrieve=extend_schema(
description="Retrieve a single document",
responses={
@@ -2279,6 +2302,76 @@ class DocumentViewSet(
raise Http404
return version_doc
@extend_schema(
operation_id="documents_run_zone_ocr",
request=RunZoneOcrSerializer,
responses=inline_serializer(
name="RunZoneOcrResult",
fields={
"status": serializers.ChoiceField(choices=["completed", "skipped"]),
"reason": serializers.ChoiceField(
choices=["no_template"],
required=False,
),
"results": serializers.ListField(child=serializers.DictField()),
},
),
)
@action(
methods=["post"],
detail=True,
url_path="run-zone-ocr",
permission_classes=[IsAuthenticated],
)
def run_zone_ocr(self, request, pk=None):
"""
Run zone-based OCR extraction on this document, with the given template
or with every template that matches it.
"""
serializer = RunZoneOcrSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
template = serializer.validated_data.get("template")
document = get_object_or_404(
Document.objects.select_related("root_document__owner"),
pk=pk,
)
if not request.user.has_perm(
"documents.change_document",
) or not has_perms_owner_aware(request.user, "change_document", document):
return HttpResponseForbidden("Insufficient permissions")
if template is not None and not has_perms_owner_aware(
request.user,
"view_ocrtemplate",
template,
):
return HttpResponseForbidden("Insufficient permissions")
from documents.zone_ocr import get_templates_for_document
templates = (
[template] if template is not None else get_templates_for_document(document)
)
if not templates:
return Response(
{"status": "skipped", "reason": "no_template", "results": []},
)
doc_path = document.archive_path or document.source_path
if not doc_path or not Path(doc_path).is_file():
return Response(
{"error": "Document file not found"},
status=status.HTTP_404_NOT_FOUND,
)
from documents.zone_ocr import run_zone_extraction
results = run_zone_extraction(document, None, templates=templates)
return Response(
{"status": "completed", "results": [asdict(result) for result in results]},
)
@extend_schema(
operation_id="documents_delete_version",
parameters=[
@@ -3431,6 +3524,40 @@ class ReprocessDocumentsView(DocumentOperationPermissionMixin):
)
@extend_schema_view(
post=extend_schema(
operation_id="documents_run_zone_ocr_bulk",
description="Run OCR template zone extraction on selected documents",
responses={
200: inline_serializer(
name="RunZoneOcrDocumentsResult",
fields={
"result": serializers.CharField(),
},
),
},
),
)
class RunZoneOcrDocumentsView(DocumentOperationPermissionMixin):
serializer_class = RunZoneOcrDocumentsSerializer
def post(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
template = serializer.validated_data.get("template")
if template is not None and not has_perms_owner_aware(
request.user,
"view_ocrtemplate",
template,
):
return HttpResponseForbidden("Insufficient permissions")
return self._execute_document_action(
method=bulk_edit.run_zone_ocr,
validated_data=serializer.validated_data,
operation_label="document zone OCR",
)
@extend_schema_view(
post=extend_schema(
operation_id="documents_edit_pdf",
@@ -5723,3 +5850,141 @@ def serve_logo(request: HttpRequest, filename: str | None = None) -> FileRespons
filename=logo_name,
as_attachment=True,
)
class OcrTemplateViewSet(BulkPermissionMixin, PassUserMixin, ModelViewSet[OcrTemplate]):
"""CRUD for OCR templates with zone definitions."""
model = OcrTemplate
queryset = (
OcrTemplate.objects.select_related("owner")
.prefetch_related(
"zones",
"zones__custom_field",
)
.order_by("name")
)
serializer_class = OcrTemplateSerializer
pagination_class = StandardPagination
permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
filter_backends = (
OrderingFilter,
PermittedObjectsFilter,
)
ordering_fields = ("name",)
@action(
detail=False,
methods=["get"],
url_path=r"document-page-image/(?P<doc_id>[0-9]+)/(?P<page>[0-9]+)",
)
def document_page_image(self, request, doc_id=None, page=None):
"""Render a page for the template editor's zone overlay."""
document = get_object_or_404(
Document.objects.select_related("root_document__owner"),
pk=doc_id,
)
if not has_perms_owner_aware(request.user, "view_document", document):
return HttpResponseForbidden("Insufficient permissions")
page_num = int(page)
if document.page_count and page_num >= document.page_count:
raise Http404(
f"Page {page_num} out of range (document has {document.page_count} pages)",
)
doc_path = document.archive_path or document.source_path
if not doc_path or not Path(doc_path).is_file():
raise Http404("Document file not found")
if document.mime_type and document.mime_type.startswith("image/"):
content = Path(doc_path).read_bytes()
return HttpResponse(content, content_type=document.mime_type)
with tempfile.TemporaryDirectory(dir=settings.SCRATCH_DIR) as tmp_dir:
output_prefix = Path(tmp_dir) / "page"
try:
subprocess.run(
[
"pdftoppm",
"-png",
"-r",
"150",
"-f",
str(page_num + 1),
"-l",
str(page_num + 1),
str(doc_path),
str(output_prefix),
],
check=True,
capture_output=True,
timeout=30,
)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as error:
logger.warning(
"Failed to render page %s of document %s: %s",
page_num,
document.pk,
error,
)
raise Http404("Failed to render page") from error
except FileNotFoundError as error:
logger.warning("pdftoppm is not available")
raise Http404("Failed to render page") from error
rendered = sorted(Path(tmp_dir).glob("page-*.png"))
if not rendered:
raise Http404("No rendered page found")
content = rendered[0].read_bytes()
return HttpResponse(content, content_type="image/png")
@extend_schema(request=OcrTemplateTestSerializer)
@action(detail=False, methods=["post"], url_path="test-zone")
def test_zone(self, request):
"""Extract an ad-hoc zone without changing the document."""
from documents.zone_ocr import extract_zone_preview
serializer = OcrTemplateTestSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
document = serializer.validated_data["document"]
if not has_perms_owner_aware(request.user, "view_document", document):
return HttpResponseForbidden("Insufficient permissions")
doc_path = document.archive_path or document.source_path
if not doc_path or not Path(doc_path).is_file():
return Response(
{"error": "Document file not found"},
status=status.HTTP_404_NOT_FOUND,
)
zone = OcrTemplateZone(**serializer.validated_data["zone"])
result = extract_zone_preview(
Path(doc_path),
zone,
zone.zone_source_width or 0,
zone.zone_source_height or 0,
document.page_count,
)
regex_match = None
if zone.validation_regex and result.get("value") is not None:
regex_match = (
safe_regex_fullmatch(zone.validation_regex, result["value"]) is not None
)
return Response(
{
"raw_text": result.get("raw_text"),
"value": result.get("value"),
"regex": zone.validation_regex,
"regex_match": regex_match,
},
)
+672
View File
@@ -0,0 +1,672 @@
from __future__ import annotations
import logging
import math
import re
import string
import subprocess
import tempfile
from dataclasses import dataclass
from datetime import date
from datetime import datetime
from decimal import Decimal
from decimal import InvalidOperation
from pathlib import Path
import dateparser
import magic
import zxingcpp
from django.conf import settings
from django.core.exceptions import ValidationError
from django.db import transaction
from PIL import Image
from documents.matching import matches
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import OcrTemplate
from documents.models import OcrTemplateZone
from documents.permissions import permitted_object_ids
from documents.regex import safe_regex_fullmatch
from documents.versioning import get_root_document
logger = logging.getLogger("paperless.zone_ocr")
@dataclass(frozen=True)
class ZoneExtraction:
raw_text: str | None = None
value: str | None = None
@dataclass(frozen=True)
class FieldUpdate:
field_name: str
value: object
custom_field: CustomField | None = None
@dataclass
class ZoneOcrResult:
template: str
zone: str
custom_field: str
value: object = None
reason: str | None = None
def _zone_result(
template: OcrTemplate,
zone: OcrTemplateZone,
*,
value: object = None,
reason: str | None = None,
) -> ZoneOcrResult:
if zone.target == OcrTemplateZone.TargetType.CUSTOM_FIELD:
label = zone.custom_field.name if zone.custom_field_id else "(no field)"
else:
label = {"title": "Title", "asn": "ASN", "created": "Created"}[zone.target]
return ZoneOcrResult(template.name, zone.name, label, value, reason)
def get_templates_for_document(document: Document) -> list[OcrTemplate]:
"""
The enabled templates that match the document's content and that its owner
can view, the same rule as matching. A version is owned through its root
document.
"""
owner = get_root_document(document).owner
templates = (
OcrTemplate.objects.filter(
enabled=True,
id__in=permitted_object_ids(owner, OcrTemplate, "view_ocrtemplate"),
)
.order_by("name", "pk")
.prefetch_related("zones", "zones__custom_field")
)
return [template for template in templates if matches(template, document)]
def run_zone_extraction(
document: Document,
original_file: Path | None,
templates: list[OcrTemplate] | None = None,
) -> list[ZoneOcrResult]:
"""
Run zone-based OCR extraction for the given templates, or for every
template that matches the document when none are given.
"""
if templates is None:
templates = get_templates_for_document(document)
if not templates:
return []
doc_path = _resolve_doc_path(document, original_file)
if doc_path is None:
logger.warning(
"Zone OCR: no accessible file for document %d",
document.pk,
)
return [
_zone_result(template, zone, reason="missing_file")
for template in templates
for zone in template.zones.all()
]
template_zones = [(template, list(template.zones.all())) for template in templates]
pages = {
_resolve_page_idx(zone.page, document.page_count)
for _, zones in template_zones
for zone in zones
}
if not pages:
return []
results = []
with tempfile.TemporaryDirectory(dir=settings.SCRATCH_DIR) as tmp_dir:
tmp_path = Path(tmp_dir)
page_images = _render_pages(doc_path, pages, tmp_path, document.page_count)
for template, zones in template_zones:
try:
results.extend(
_process_template(document, page_images, template, zones, tmp_path),
)
except Exception:
# One failed template must not prevent the remaining templates from running.
logger.exception(
"Zone OCR: error processing template '%s' for document %d",
template.name,
document.pk,
)
results.extend(
_zone_result(template, zone, reason="template_failed")
for zone in zones
)
return results
def _resolve_doc_path(
document: Document,
original_file: Path | None,
) -> Path | None:
"""Prefer the archive, then the source, then the ingestion signal's file."""
candidates = []
if document.has_archive_version:
candidates.append(document.archive_path)
candidates.append(document.source_path)
if original_file is not None:
candidates.append(original_file)
for path in candidates:
if path is not None and Path(path).is_file():
return Path(path)
return None
def _resolve_page_idx(page_value: int | None, page_count: int | None) -> int:
"""Resolve a 1-based page or -1 (last page), defaulting to the first page."""
if page_value == -1:
return max((page_count or 1) - 1, 0)
return max((page_value or 1) - 1, 0)
def _process_template(
document: Document,
page_images: dict[int, Path],
template: OcrTemplate,
zones: list[OcrTemplateZone],
tmp_path: Path,
) -> list[ZoneOcrResult]:
"""Extract and combine zone values before applying a template's updates."""
grouped: dict[str, list[tuple[OcrTemplateZone, ZoneOcrResult]]] = {}
results = []
for zone in zones:
page_idx = _resolve_page_idx(zone.page, document.page_count)
extracted = None
reason = "missing_page"
if page_idx in page_images:
extraction = _extract_zone(
page_images[page_idx],
zone,
zone.zone_source_width or template.source_width,
zone.zone_source_height or template.source_height,
tmp_path,
)
extracted = extraction.value
reason = None if extracted else "no_value"
if (
extracted is not None
and zone.validation_regex
and not safe_regex_fullmatch(zone.validation_regex, extracted)
):
reason = "regex_mismatch"
result = _zone_result(
template,
zone,
value=extracted if reason is None else None,
reason=reason,
)
results.append(result)
grouped.setdefault(_field_key(zone), []).append((zone, result))
updates = []
combine_formats = template.combine_formats or {}
for key, field_zones in grouped.items():
accepted = [result for _, result in field_zones if result.reason is None]
if not accepted:
continue
value = _combine_field_value(
combine_formats.get(key, ""),
[(zone.name, result.value) for zone, result in field_zones],
)
target_zone = field_zones[0][0]
update = _prepare_field_update(target_zone, value) if value else None
if update is not None:
updates.append(update)
for result in accepted:
result.value = update.value if update else None
if update is None:
result.reason = (
"unassigned" if key == "custom_field" else "invalid_value"
)
_apply_field_updates(document, updates)
return results
def _field_key(zone: OcrTemplateZone) -> str:
"""Use the same target keys as the editor's combination formats."""
target = zone.target
if target == OcrTemplateZone.TargetType.CUSTOM_FIELD and zone.custom_field_id:
return str(zone.custom_field_id)
return target
def _combine_field_value(
fmt: str,
field_zones: list[tuple[str, str | None]],
) -> str:
"""Combine the OCR values of all zones targeting one field.
With a format string, `{Zone Name}` tokens are replaced by that zone's value
and literal text is kept; separators left dangling by an empty token are
cleaned up. Without a format, the zone values are joined in order by a space.
"""
values = {name: value or "" for name, value in field_zones}
if not fmt:
return " ".join(value for _, value in field_zones if value).strip()
def _replace(match: re.Match) -> str:
return values.get(match.group(1).strip(), "")
combined = re.sub(r"\{([^{}]+)\}", _replace, fmt)
# Tidy up separators an empty token may have left behind.
combined = re.sub(r"\s{2,}", " ", combined)
combined = re.sub(r"([^\w\s])\s*\1+", r"\1", combined)
return combined.strip().strip("-/.,;:| \t")
def _render_pages(
doc_path: Path,
pages: set[int],
tmp_dir: Path,
page_count: int | None,
) -> dict[int, Path]:
"""Render specific PDF pages as PNG images using pdftoppm (poppler-utils)."""
result: dict[int, Path] = {}
mime = _detect_mime(doc_path)
if mime and mime.startswith("image/"):
if 0 in pages:
result[0] = doc_path
return result
for actual_page in sorted(pages):
if actual_page < 0 or (page_count and actual_page >= page_count):
continue
output_prefix = tmp_dir / f"page_{actual_page}"
try:
subprocess.run(
[
"pdftoppm",
"-png",
"-singlefile",
"-r",
"300",
"-f",
str(actual_page + 1), # pdftoppm is 1-indexed
"-l",
str(actual_page + 1),
str(doc_path),
str(output_prefix),
],
check=True,
capture_output=True,
timeout=60,
)
except subprocess.TimeoutExpired:
logger.error("Zone OCR: pdftoppm timed out for page %d", actual_page)
continue
except subprocess.CalledProcessError as e:
logger.error(
"Zone OCR: pdftoppm failed for page %d: %s",
actual_page,
e.stderr.decode(errors="replace") if e.stderr else str(e),
)
continue
except FileNotFoundError:
logger.error("Zone OCR: pdftoppm not found")
return result
rendered = output_prefix.with_suffix(".png")
if rendered.is_file():
result[actual_page] = rendered
return result
def _crop_zone(
page_img: Path,
zone: OcrTemplateZone,
source_width: int,
source_height: int,
) -> Image.Image | None:
"""Crop a zone from the page image and return the PIL Image."""
try:
with Image.open(page_img) as img:
img_width, img_height = img.size
if not source_width or not source_height:
source_width, source_height = img.size
scale_x = img_width / source_width
scale_y = img_height / source_height
crop_left = int(zone.x * scale_x)
crop_top = int(zone.y * scale_y)
crop_right = int((zone.x + zone.width) * scale_x)
crop_bottom = int((zone.y + zone.height) * scale_y)
# Clamp to the image so an oversized zone can't crop out of bounds.
crop_left = max(0, min(crop_left, img_width))
crop_top = max(0, min(crop_top, img_height))
crop_right = min(crop_right, img_width)
crop_bottom = min(crop_bottom, img_height)
if crop_right - crop_left < 2 or crop_bottom - crop_top < 2:
logger.warning("Zone OCR: crop too small for zone '%s'", zone.name)
return None
return img.crop((crop_left, crop_top, crop_right, crop_bottom))
except (OSError, ValueError, Image.DecompressionBombError):
logger.exception("Zone OCR: crop failed for zone '%s'", zone.name)
return None
def _read_barcode(cropped: Image.Image, zone_name: str) -> str | None:
"""Read QR/barcode from a cropped image using zxingcpp."""
try:
results = zxingcpp.read_barcodes(cropped)
return results[0].text if results else None
except (RuntimeError, ValueError, TypeError):
logger.exception("Zone OCR: barcode read failed for zone '%s'", zone_name)
return None
def _ocr_text(cropped: Image.Image, zone: OcrTemplateZone, tmp_dir: Path) -> str | None:
"""OCR a cropped image with Tesseract."""
crop_path = tmp_dir / f"zone_{zone.pk}.png"
cropped.save(crop_path)
try:
proc = subprocess.run(
[
"tesseract",
str(crop_path),
"stdout",
"-l",
zone.ocr_language,
"--psm",
"6", # Assume uniform block of text
],
capture_output=True,
text=True,
timeout=30,
check=True,
)
return proc.stdout.strip() or None
except subprocess.TimeoutExpired:
logger.error("Zone OCR: Tesseract timed out for zone '%s'", zone.name)
return None
except subprocess.CalledProcessError as e:
logger.error(
"Zone OCR: Tesseract failed for zone '%s': %s",
zone.name,
e.stderr[:200] if e.stderr else str(e),
)
return None
except FileNotFoundError:
logger.error("Zone OCR: Tesseract not found")
return None
def _extract_zone(
page_img: Path,
zone: OcrTemplateZone,
source_width: int,
source_height: int,
tmp_dir: Path,
) -> ZoneExtraction:
"""Read and transform a zone, retaining the text for the editor's preview."""
cropped = _crop_zone(page_img, zone, source_width, source_height)
if cropped is None:
return ZoneExtraction()
try:
raw_text = (
_read_barcode(cropped, zone.name)
if zone.transform == OcrTemplateZone.TransformType.QR_CODE
else _ocr_text(cropped, zone, tmp_dir)
)
finally:
cropped.close()
return ZoneExtraction(
raw_text=raw_text,
value=_apply_transform(raw_text, zone.transform, zone.date_format)
if raw_text
else None,
)
def extract_zone_preview(
doc_path: Path,
zone: OcrTemplateZone,
source_width: int,
source_height: int,
page_count: int | None,
) -> dict:
"""Extract a zone without writing its value to the document."""
page_idx = _resolve_page_idx(zone.page, page_count)
with tempfile.TemporaryDirectory(dir=settings.SCRATCH_DIR) as tmp_dir:
tmp_path = Path(tmp_dir)
page_images = _render_pages(doc_path, {page_idx}, tmp_path, page_count)
if page_idx not in page_images:
return {"raw_text": None, "value": None}
result = _extract_zone(
page_images[page_idx],
zone,
source_width,
source_height,
tmp_path,
)
return {"raw_text": result.raw_text, "value": result.value}
def _parse_iso_date(text: str) -> date | None:
match = re.search(r"\d{4}-\d{2}-\d{2}", text)
return date.fromisoformat(match[0]) if match else None
def _parse_date(text: str, fmt: str = "") -> date | None:
"""Try the zone's date format, ISO dates, then automatic date detection."""
if fmt:
try:
return datetime.strptime(text, fmt).date()
except ValueError:
pass
try:
iso_date = _parse_iso_date(text)
except ValueError:
iso_date = None
if iso_date is not None:
return iso_date
try:
parsed = dateparser.parse(
text,
settings={
"PREFER_DAY_OF_MONTH": "first",
"RETURN_AS_TIMEZONE_AWARE": False,
},
)
if parsed:
return parsed.date()
except (ValueError, OverflowError):
logger.debug("Zone OCR: date parsing failed")
return None
def _apply_transform(text: str, transform: str, date_format: str = "") -> str:
"""Apply post-processing transform to extracted text."""
text = text.strip()
if not text:
return text
if transform in ("strip", "none"):
return text
elif transform == "date":
parsed = _parse_date(text, date_format)
return parsed.isoformat() if parsed else text
elif transform == "uppercase":
return text.upper()
elif transform == "lowercase":
return text.lower()
elif transform == "numeric":
result = re.sub(r"[^\d.,\-]", "", text)
return result if result else text
elif transform == "strip_punctuation":
return text.strip(string.punctuation + " \t\r\n")
return text
def _prepare_field_update(
zone: OcrTemplateZone,
value: str,
) -> FieldUpdate | None:
"""Convert extracted text and enforce the destination field's constraints."""
custom_field = None
if zone.target == OcrTemplateZone.TargetType.CUSTOM_FIELD:
if not zone.custom_field_id:
return None
custom_field = zone.custom_field
field_name = CustomFieldInstance.get_value_field_name(custom_field.data_type)
typed_value = _convert_value(value, custom_field.data_type)
model = CustomFieldInstance
elif zone.target == OcrTemplateZone.TargetType.TITLE:
field_name, typed_value = "title", value[:128]
model = Document
elif zone.target == OcrTemplateZone.TargetType.ASN:
digits = re.sub(r"[^\d]", "", value)
field_name = "archive_serial_number"
typed_value = int(digits) if digits else None
model = Document
elif zone.target == OcrTemplateZone.TargetType.CREATED:
field_name, typed_value = "created", _parse_date(value)
model = Document
else:
return None
if typed_value is None:
return None
try:
typed_value = model._meta.get_field(field_name).clean(typed_value, None)
except ValidationError:
logger.info("Zone OCR: invalid value for zone '%s'", zone.name)
return None
return FieldUpdate(field_name, typed_value, custom_field)
def _apply_field_updates(document: Document, updates: list[FieldUpdate]) -> None:
builtin_values = {
update.field_name: update.value
for update in updates
if update.custom_field is None
}
original_values = {name: getattr(document, name) for name in builtin_values}
try:
with transaction.atomic():
for update in updates:
if update.custom_field is not None:
CustomFieldInstance.objects.update_or_create(
document=document,
field=update.custom_field,
defaults={update.field_name: update.value},
)
if builtin_values:
for name, value in builtin_values.items():
setattr(document, name, value)
document.save(update_fields=list(builtin_values))
except Exception:
# Database rollback does not restore attributes on the model instance.
for name, value in original_values.items():
setattr(document, name, value)
raise
def _normalize_number(value: str) -> str:
cleaned = re.sub(r"[^\d.,\-]", "", value)
# When both separators occur, the rightmost one is the decimal separator.
if "," in cleaned and "." in cleaned:
if cleaned.rindex(",") > cleaned.rindex("."):
cleaned = cleaned.replace(".", "").replace(",", ".")
else:
cleaned = cleaned.replace(",", "")
else:
cleaned = cleaned.replace(",", ".")
Decimal(cleaned)
return cleaned
def _convert_value(value: str, data_type: str) -> object | None:
"""Convert an extracted OCR string to the appropriate type for the custom field."""
if not value:
return None
try:
if data_type in (
CustomField.FieldDataType.STRING,
CustomField.FieldDataType.URL,
):
return value[:128]
elif data_type == CustomField.FieldDataType.LONG_TEXT:
return value
elif data_type == CustomField.FieldDataType.INT:
digits = re.sub(r"[^\d\-]", "", value)
digits = digits.lstrip("-")
if not digits:
return None
if value.strip().startswith("-"):
digits = "-" + digits
return int(digits)
elif data_type == CustomField.FieldDataType.FLOAT:
number = float(_normalize_number(value))
return number if math.isfinite(number) else None
elif data_type == CustomField.FieldDataType.DATE:
parsed = _parse_iso_date(value)
return parsed.isoformat() if parsed else None
elif data_type == CustomField.FieldDataType.MONETARY:
return _normalize_number(value)
elif data_type == CustomField.FieldDataType.BOOL:
lower = value.lower().strip()
if lower in ("true", "yes", "1", "ja", "oui", "si", "x"):
return True
elif lower in ("false", "no", "0", "nein", "non"):
return False
return None
else:
logger.debug(
"Zone OCR: unsupported custom field type %s for OCR extraction",
data_type,
)
return None
except (ValueError, InvalidOperation):
logger.info("Zone OCR: could not convert value to %s", data_type)
return None
def _detect_mime(path: Path) -> str | None:
"""Detect MIME type of a file."""
try:
return magic.from_file(str(path), mime=True)
except (OSError, magic.MagicException):
logger.debug("Zone OCR: magic failed for %s, falling back to extension", path)
suffix = path.suffix.lower()
return {
".pdf": "application/pdf",
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".tiff": "image/tiff",
".tif": "image/tiff",
".webp": "image/webp",
".bmp": "image/bmp",
".gif": "image/gif",
}.get(suffix)
+19 -19
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr ""
"Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-10-21 15:47+0000\n"
"POT-Creation-Date: 2026-10-22 19:29+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n"
"Language-Team: English\n"
@@ -1653,7 +1653,7 @@ msgid "workflow runs"
msgstr ""
#: documents/serialisers.py:515 documents/serialisers.py:872
#: documents/serialisers.py:2902 documents/views.py:363 documents/views.py:2773
#: documents/serialisers.py:2904 documents/views.py:364 documents/views.py:2787
#: paperless_mail/serialisers.py:156
msgid "Insufficient permissions."
msgstr ""
@@ -1662,39 +1662,39 @@ msgstr ""
msgid "Invalid color."
msgstr ""
#: documents/serialisers.py:2369
#: documents/serialisers.py:2371
#, python-format
msgid "File type %(type)s not supported"
msgstr ""
#: documents/serialisers.py:2413
#: documents/serialisers.py:2415
#, python-format
msgid "Custom field id must be an integer: %(id)s"
msgstr ""
#: documents/serialisers.py:2420
#: documents/serialisers.py:2422
#, python-format
msgid "Custom field with id %(id)s does not exist"
msgstr ""
#: documents/serialisers.py:2437 documents/serialisers.py:2447
#: documents/serialisers.py:2439 documents/serialisers.py:2449
msgid ""
"Custom fields must be a list of integers or an object mapping ids to values."
msgstr ""
#: documents/serialisers.py:2442
#: documents/serialisers.py:2444
msgid "Some custom fields don't exist or were specified twice."
msgstr ""
#: documents/serialisers.py:2589
#: documents/serialisers.py:2591
msgid "Invalid variable detected."
msgstr ""
#: documents/serialisers.py:2958
#: documents/serialisers.py:2960
msgid "Duplicate document identifiers are not allowed."
msgstr ""
#: documents/serialisers.py:2988 documents/views.py:4823
#: documents/serialisers.py:2990 documents/views.py:4837
#, python-format
msgid "Documents not found: %(ids)s"
msgstr ""
@@ -1945,40 +1945,40 @@ msgstr ""
msgid ", "
msgstr ""
#: documents/views.py:318
#: documents/views.py:319
msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr ""
#: documents/views.py:356 documents/views.py:2770
#: documents/views.py:357 documents/views.py:2784
msgid "Invalid more_like_id"
msgstr ""
#: documents/views.py:1708
#: documents/views.py:1722
msgid "Invalid AI configuration."
msgstr ""
#: documents/views.py:1719
#: documents/views.py:1733
msgid "AI backend request timed out."
msgstr ""
#: documents/views.py:1731
#: documents/views.py:1745
msgid "AI backend rejected the request. Check logs for details."
msgstr ""
#: documents/views.py:4839
#: documents/views.py:4853
#, python-format
msgid "Insufficient permissions to share document %(id)s."
msgstr ""
#: documents/views.py:4886
#: documents/views.py:4900
msgid "Bundle is already being processed."
msgstr ""
#: documents/views.py:4952
#: documents/views.py:4966
msgid "The share link bundle is still being prepared. Please try again later."
msgstr ""
#: documents/views.py:4966
#: documents/views.py:4980
msgid "The share link bundle is unavailable."
msgstr ""
+8
View File
@@ -30,11 +30,13 @@ from documents.views import IndexView
from documents.views import LogViewSet
from documents.views import MergeDocumentsAsVersionsView
from documents.views import MergeDocumentsView
from documents.views import OcrTemplateViewSet
from documents.views import PostDocumentView
from documents.views import RemoteVersionView
from documents.views import RemovePasswordDocumentsView
from documents.views import ReprocessDocumentsView
from documents.views import RotateDocumentsView
from documents.views import RunZoneOcrDocumentsView
from documents.views import SavedViewViewSet
from documents.views import SearchAutoCompleteView
from documents.views import SelectionDataView
@@ -90,6 +92,7 @@ api_router.register(r"workflow_triggers", WorkflowTriggerViewSet)
api_router.register(r"workflow_actions", WorkflowActionViewSet)
api_router.register(r"workflows", WorkflowViewSet)
api_router.register(r"custom_fields", CustomFieldViewSet)
api_router.register(r"ocr_templates", OcrTemplateViewSet)
api_router.register(r"config", ApplicationConfigurationViewSet)
api_router.register(r"processed_mail", ProcessedMailViewSet)
@@ -166,6 +169,11 @@ urlpatterns = [
ReprocessDocumentsView.as_view(),
name="reprocess_documents",
),
re_path(
"^run_zone_ocr/",
RunZoneOcrDocumentsView.as_view(),
name="run_zone_ocr_documents",
),
re_path(
"^rotate/",
RotateDocumentsView.as_view(),
+1 -1
View File
@@ -1,6 +1,6 @@
from typing import Final
__version__: Final[tuple[int, int, int]] = (3, 3, 3)
__version__: Final[tuple[int, int, int]] = (3, 3, 4)
# Version string like X.Y.Z
__full_version_str__: Final[str] = ".".join(map(str, __version__))
# Version string like X.Y
Generated
+72 -68
View File
@@ -1,5 +1,5 @@
version = 1
revision = 3
revision = 5
requires-python = ">=3.11"
resolution-markers = [
"python_full_version >= '3.15' and sys_platform == 'darwin'",
@@ -9,7 +9,7 @@ resolution-markers = [
"python_full_version >= '3.12' and python_full_version < '3.15' and sys_platform == 'darwin'",
"python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux'",
"python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux'",
"(python_full_version >= '3.12' and python_full_version < '3.15' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and platform_machine == 'aarch64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and platform_machine == 'x86_64' and sys_platform == 'linux')",
"(python_full_version == '3.12.*' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.14.*' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and sys_platform == 'linux')",
"python_full_version < '3.12' and sys_platform == 'darwin'",
"python_full_version < '3.12' and sys_platform == 'linux'",
]
@@ -530,11 +530,11 @@ wheels = [
[[package]]
name = "certifi"
version = "2026.1.4"
version = "2026.10.19"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e0/2d/a891ca51311197f6ad14a7ef42e2399f36cf2f9bd44752b3dc4eab60fdc5/certifi-2026.1.4.tar.gz", hash = "sha256:ac726dd470482006e014ad384921ed6438c457018f4b3d204aea4281258b2120", size = 154268, upload-time = "2026-01-04T02:42:41.825Z" }
sdist = { url = "https://files.pythonhosted.org/packages/a6/bd/1436c50cc46699de0230fbaacc1d1de7bfec413176099949e9d5112e88f2/certifi-2026.10.19.tar.gz", hash = "sha256:035c1e85572b141bf1c3a3eff0e33a8eb74917522f2559075ff8d2fea780a1ae", size = 137598, upload-time = "2026-10-20T02:24:16.604Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e6/ad/3cc14f097111b4de0040c83a525973216457bbeeb63739ef1ed275c1c021/certifi-2026.1.4-py3-none-any.whl", hash = "sha256:9943707519e4add1115f44c2bc244f782c0249876bf51b6599fee1ffbedd685c", size = 152900, upload-time = "2026-01-04T02:42:40.15Z" },
{ url = "https://files.pythonhosted.org/packages/c4/51/29046e48f693c54e5b7412216d447a1d8362c9653148d98c02cdf65fa733/certifi-2026.10.19-py3-none-any.whl", hash = "sha256:42d35340cc5b9ba78e818ddbff0c5b1c2b779de448a340fd1b989f5fa8670e1b", size = 136464, upload-time = "2026-10-20T02:24:15.072Z" },
]
[[package]]
@@ -3040,7 +3040,7 @@ wheels = [
[[package]]
name = "paperless-ngx"
version = "3.3.3"
version = "3.3.4"
source = { virtual = "." }
dependencies = [
{ name = "azure-ai-documentintelligence" },
@@ -3114,7 +3114,7 @@ mariadb = [
]
postgres = [
{ name = "psycopg", extra = ["c", "pool"] },
{ name = "psycopg-c", version = "3.3.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.12' and platform_machine == 'aarch64') or (python_full_version == '3.13.*' and platform_machine == 'aarch64') or (python_full_version >= '3.15' and platform_machine == 'aarch64') or (python_full_version < '3.12' and platform_machine == 'x86_64') or (python_full_version == '3.13.*' and platform_machine == 'x86_64') or (python_full_version >= '3.15' and platform_machine == 'x86_64') or (platform_machine != 'aarch64' and platform_machine != 'x86_64') or sys_platform != 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12' or python_full_version == '3.13.*' or python_full_version >= '3.15' or (platform_machine != 'aarch64' and platform_machine != 'x86_64') or sys_platform != 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl" }, marker = "python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" }, marker = "python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" }, marker = "python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
@@ -3243,7 +3243,7 @@ requires-dist = [
{ name = "psycopg-c", marker = "python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.14.*' and platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl" },
{ name = "psycopg-c", marker = "(python_full_version < '3.12' and platform_machine == 'aarch64' and extra == 'postgres') or (python_full_version == '3.13.*' and platform_machine == 'aarch64' and extra == 'postgres') or (python_full_version >= '3.15' and platform_machine == 'aarch64' and extra == 'postgres') or (python_full_version < '3.12' and platform_machine == 'x86_64' and extra == 'postgres') or (python_full_version == '3.13.*' and platform_machine == 'x86_64' and extra == 'postgres') or (python_full_version >= '3.15' and platform_machine == 'x86_64' and extra == 'postgres') or (platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'postgres') or (sys_platform != 'linux' and extra == 'postgres')", specifier = "==3.3.4" },
{ name = "psycopg-c", marker = "(python_full_version < '3.12' and extra == 'postgres') or (python_full_version == '3.13.*' and extra == 'postgres') or (python_full_version >= '3.15' and extra == 'postgres') or (platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'postgres') or (sys_platform != 'linux' and extra == 'postgres')", specifier = "==3.3.4" },
{ name = "psycopg-pool", marker = "extra == 'postgres'", specifier = "==3.3.3" },
{ name = "python-dateutil", specifier = "~=2.9.0" },
{ name = "python-dotenv", specifier = "~=1.2.1" },
@@ -3421,65 +3421,69 @@ wheels = [
[[package]]
name = "pillow"
version = "12.3.0"
version = "13.0.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/1c/3d/bb7fca845737cf9d7dbde16ed1843984665ff2e0a518f5db43e77ec540b9/pillow-12.3.0.tar.gz", hash = "sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce", size = 47025035, upload-time = "2026-07-01T11:56:38.965Z" }
sdist = { url = "https://files.pythonhosted.org/packages/ce/a4/9cc30690a394a8305e8be4c9036aa16d0dfddadd09079860b582f562496d/pillow-13.0.0.tar.gz", hash = "sha256:cdcd1ba258178f7356d74170c9a81ba4eab6a5d55748bc402a5a2a6fe6be3510", size = 47058631, upload-time = "2026-10-16T22:18:09.718Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/c8/0a78b0e02d7ac54bc03e5321c9220da52f0c2ea83b21f7c40e7f3169c502/pillow-12.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756", size = 5392415, upload-time = "2026-07-01T11:53:47.162Z" },
{ url = "https://files.pythonhosted.org/packages/b2/5b/a02d30018abd97ced9f5a6c63d28597694a00d066516b9c1c6de45859fc9/pillow-12.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6", size = 4785266, upload-time = "2026-07-01T11:53:49.079Z" },
{ url = "https://files.pythonhosted.org/packages/c8/98/766667a4be768150a202836acd9fad19c06824ca86c4286d3cf6b274964e/pillow-12.3.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd", size = 6263814, upload-time = "2026-07-01T11:53:51.32Z" },
{ url = "https://files.pythonhosted.org/packages/3b/2d/ede717bc1144f63886c21fd349bb95860b0d1a21149ff16f2bb362b612b6/pillow-12.3.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd", size = 6934408, upload-time = "2026-07-01T11:53:53.487Z" },
{ url = "https://files.pythonhosted.org/packages/a3/48/9c58b685e69d49c31af6c8eb9012055fab7e665785165c84796e2c73ce72/pillow-12.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c", size = 6337160, upload-time = "2026-07-01T11:53:55.457Z" },
{ url = "https://files.pythonhosted.org/packages/ff/fa/dc2a5c0ba6df93f67c31d34b808b7ce440b40cdbf96f0b81cde1d1e6fa93/pillow-12.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5", size = 7045172, upload-time = "2026-07-01T11:53:57.736Z" },
{ url = "https://files.pythonhosted.org/packages/37/bf/fb3ebff8ddcb76aac5a01389251bbbb9519922a9b520d8247c1ca864a25d/pillow-12.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965", size = 5345969, upload-time = "2026-07-01T11:54:06.397Z" },
{ url = "https://files.pythonhosted.org/packages/d8/66/9a386a92561f402389a4fc70c18838bf6d35eb5eb5c6850b4b2dc64f5048/pillow-12.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7", size = 4780323, upload-time = "2026-07-01T11:54:09.351Z" },
{ url = "https://files.pythonhosted.org/packages/25/27/ac8f99618ffd3dde21db0f4d4b1d2ab00c0880595bfd17df103f7f39fd0c/pillow-12.3.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9", size = 6266838, upload-time = "2026-07-01T11:54:11.71Z" },
{ url = "https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91", size = 6940830, upload-time = "2026-07-01T11:54:13.732Z" },
{ url = "https://files.pythonhosted.org/packages/eb/51/8b08617af3ad95e33ce6d7dd2c99ed6c8298f7fb131636303956be022e25/pillow-12.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c", size = 6344383, upload-time = "2026-07-01T11:54:15.756Z" },
{ url = "https://files.pythonhosted.org/packages/1d/72/cf78ac9780bb93c28328f408973845a309d4d145041665f734572ced1b52/pillow-12.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df", size = 7052934, upload-time = "2026-07-01T11:54:17.721Z" },
{ url = "https://files.pythonhosted.org/packages/9d/ac/31fb64e1e7efb5a4b50cd3d92049ba89ac6e4d8d3bb6a74e15048ca3353e/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89", size = 4161684, upload-time = "2026-07-01T11:54:25.934Z" },
{ url = "https://files.pythonhosted.org/packages/87/b4/9805e23d2b4d77842b468513841fda254ee42f0289d25088340e4ff46e2d/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace", size = 4255487, upload-time = "2026-07-01T11:54:27.935Z" },
{ url = "https://files.pythonhosted.org/packages/df/39/ecf519435a200c693fe053a6ee4d835b41cf963a4dfc2551c4e637cb2a71/pillow-12.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec", size = 3696433, upload-time = "2026-07-01T11:54:29.813Z" },
{ url = "https://files.pythonhosted.org/packages/42/92/2fc3ffad878ae8dd5469ec1bc8eb83b71f48e13efdf68f02709003982a32/pillow-12.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66", size = 5345889, upload-time = "2026-07-01T11:54:31.97Z" },
{ url = "https://files.pythonhosted.org/packages/10/76/8803c13605b763d33d156c4678fc77f8443389c0c51c8aef707bb02015f4/pillow-12.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35", size = 4780109, upload-time = "2026-07-01T11:54:34.026Z" },
{ url = "https://files.pythonhosted.org/packages/1f/01/e18aff37cb0b4aac47ac90f016d347a49aca667ef97f190b06ac2aabc928/pillow-12.3.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65", size = 6263736, upload-time = "2026-07-01T11:54:36.131Z" },
{ url = "https://files.pythonhosted.org/packages/f7/62/de5bdd77d935331f4f802edc11e4d82950f642caad6cb2f949837b8560e2/pillow-12.3.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3", size = 6937129, upload-time = "2026-07-01T11:54:38.216Z" },
{ url = "https://files.pythonhosted.org/packages/70/4d/105627a13300c5e0df1d174230b32fd1273062c96f7745fd552b945d1e1d/pillow-12.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a", size = 6339562, upload-time = "2026-07-01T11:54:40.354Z" },
{ url = "https://files.pythonhosted.org/packages/6b/1d/f13de01a553988ab895ba1c722e06cf3144d4f57656fd5b81b6d881f1179/pillow-12.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e", size = 7049439, upload-time = "2026-07-01T11:54:42.489Z" },
{ url = "https://files.pythonhosted.org/packages/dc/01/001f65b68192f0228cc1dbbc8d2530ab5d58b61037ba0587f946fea607cd/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330", size = 4161736, upload-time = "2026-07-01T11:54:51.156Z" },
{ url = "https://files.pythonhosted.org/packages/1a/d2/0219746d0fd16fc8a84498e79452375be3797d3ce4044596ce565164b84f/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217", size = 4255435, upload-time = "2026-07-01T11:54:53.414Z" },
{ url = "https://files.pythonhosted.org/packages/c8/02/8d0bc62ef0302318c46ff2a512822d2610e81c7aa46c9b3abe6cbaca5ad0/pillow-12.3.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930", size = 3696262, upload-time = "2026-07-01T11:54:55.739Z" },
{ url = "https://files.pythonhosted.org/packages/85/e2/73c77d218410b14f5f2d565e8a998d5317b7b9c75368d29985139f7a46f0/pillow-12.3.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8", size = 5350344, upload-time = "2026-07-01T11:54:57.657Z" },
{ url = "https://files.pythonhosted.org/packages/c7/da/32c752228ae345f489e3a42499d817b6c3996da7e8a3bc7a04fc806b243b/pillow-12.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0", size = 4780131, upload-time = "2026-07-01T11:54:59.713Z" },
{ url = "https://files.pythonhosted.org/packages/b1/9d/8b2c807dbef61a5197c047afe99823787eb66f63daf9fb2432f91d6f0462/pillow-12.3.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321", size = 6263757, upload-time = "2026-07-01T11:55:01.778Z" },
{ url = "https://files.pythonhosted.org/packages/5c/44/c85361f65dbe00eea8576ee467c768d25129989efb76e94f205e9ca9bb46/pillow-12.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b", size = 6936962, upload-time = "2026-07-01T11:55:03.93Z" },
{ url = "https://files.pythonhosted.org/packages/18/7e/e483414b35800b86b6f08dbbc7803fb5cd52c4d6f897f47d53ea2c7e6f65/pillow-12.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198", size = 6339171, upload-time = "2026-07-01T11:55:05.989Z" },
{ url = "https://files.pythonhosted.org/packages/f0/f4/68c491844841ede6bed70189546b3ee9731cf9f2cbad396faff5e1ccba45/pillow-12.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130", size = 7048116, upload-time = "2026-07-01T11:55:08.131Z" },
{ url = "https://files.pythonhosted.org/packages/52/69/c2208e56af9bfc1913afb24020297a691eb1d4ef688474c8a04913f65e04/pillow-12.3.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e", size = 5352503, upload-time = "2026-07-01T11:55:17.076Z" },
{ url = "https://files.pythonhosted.org/packages/07/70/e5686d753e898a45d778ff1718dba8516ead6ab6b95d85fc8c4b70650cf2/pillow-12.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17", size = 4782956, upload-time = "2026-07-01T11:55:19.448Z" },
{ url = "https://files.pythonhosted.org/packages/d5/37/25c6692f06927ee973ff18c8d9ee98ad0b4d84ee67a09610c2dd1447958e/pillow-12.3.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385", size = 6322855, upload-time = "2026-07-01T11:55:21.613Z" },
{ url = "https://files.pythonhosted.org/packages/cc/91/420637fcb8f1bc11029e403b4538e6694744428d8246118e45719f944556/pillow-12.3.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c", size = 6989642, upload-time = "2026-07-01T11:55:24.006Z" },
{ url = "https://files.pythonhosted.org/packages/10/08/b94d7811281ccf0d143a1cf768d1c49e1e54af63e7b708ab2ee3eb87face/pillow-12.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d", size = 6391281, upload-time = "2026-07-01T11:55:26.252Z" },
{ url = "https://files.pythonhosted.org/packages/d2/87/24233f785f55474dc02ce3e739c5528a77e3a862e9333d1dd7a25cc31f70/pillow-12.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931", size = 7096716, upload-time = "2026-07-01T11:55:28.318Z" },
{ url = "https://files.pythonhosted.org/packages/5d/dc/8fdce34ec725a33c81c6ba122b904d6b9024e50ea9ac7bede62fab54506c/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139", size = 4162063, upload-time = "2026-07-01T11:55:37.941Z" },
{ url = "https://files.pythonhosted.org/packages/76/66/2044b9a63d3b84ff048228dfcb7cd9bf0df983e8470971bf7d4c57b693de/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402", size = 4255549, upload-time = "2026-07-01T11:55:40.022Z" },
{ url = "https://files.pythonhosted.org/packages/52/7e/1f67e6f4ece6b582ee4b539decbcc9f848dc245a93ed8cd7338bafef72f1/pillow-12.3.0-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c", size = 3696331, upload-time = "2026-07-01T11:55:41.98Z" },
{ url = "https://files.pythonhosted.org/packages/12/40/d306fc2c8e4d45d7f175c77edca7063be7b86fe7fe6e68f4353bf71d808c/pillow-12.3.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f", size = 5350370, upload-time = "2026-07-01T11:55:44.028Z" },
{ url = "https://files.pythonhosted.org/packages/dd/44/668fb1437e8ce420f62d6106eb66e44a5971602a4d794615bdf79315d82d/pillow-12.3.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701", size = 4780147, upload-time = "2026-07-01T11:55:46.073Z" },
{ url = "https://files.pythonhosted.org/packages/0c/08/93fa2e70e30a2d81547e481b6ee2bb9522117221fb1e0ce4b5df70967677/pillow-12.3.0-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace", size = 6273659, upload-time = "2026-07-01T11:55:48.264Z" },
{ url = "https://files.pythonhosted.org/packages/f8/6d/043e96ff814fc31a33077e4cba86082167db520c93632afdf2042febbb0c/pillow-12.3.0-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4", size = 6947439, upload-time = "2026-07-01T11:55:50.503Z" },
{ url = "https://files.pythonhosted.org/packages/af/92/ba71d2ee2ac0edf3fa33bd9d5ee9ee080da70b1766f3ca3934f9938ddac9/pillow-12.3.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39", size = 6353577, upload-time = "2026-07-01T11:55:52.697Z" },
{ url = "https://files.pythonhosted.org/packages/0f/ce/e63064e2122923ff687c8ad792d0d736a7b3920a56a46982e81a7fdd25d6/pillow-12.3.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71", size = 7060394, upload-time = "2026-07-01T11:55:55.149Z" },
{ url = "https://files.pythonhosted.org/packages/30/88/707027ba09942dfa2c28759b5c222d769290a41c6d20ea60ec250801941f/pillow-12.3.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf", size = 5352509, upload-time = "2026-07-01T11:56:04.2Z" },
{ url = "https://files.pythonhosted.org/packages/b0/6d/00352fa25332c2569cd387851f568cc5a4b75a9adbfb37ac4fbce4c02eec/pillow-12.3.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64", size = 4783167, upload-time = "2026-07-01T11:56:06.631Z" },
{ url = "https://files.pythonhosted.org/packages/13/4f/9e049dfa21af7c22427275720e2490267ba8138120add5c4c574deb69782/pillow-12.3.0-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e", size = 6329237, upload-time = "2026-07-01T11:56:08.868Z" },
{ url = "https://files.pythonhosted.org/packages/36/16/cf6eeaae8d0fce8dd390a33437cf68c5d5bd73834a2bc6e2f14efda0ab45/pillow-12.3.0-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777", size = 6997047, upload-time = "2026-07-01T11:56:11.379Z" },
{ url = "https://files.pythonhosted.org/packages/1e/69/dbf769bdd55f48bf5733cac28edc6364ffaa072ec9ba336266e4fe66be55/pillow-12.3.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1", size = 6400440, upload-time = "2026-07-01T11:56:13.908Z" },
{ url = "https://files.pythonhosted.org/packages/a0/e1/ffc9cfc2eea0d178da8018e18e959301ad9d6bc9f3edb7181e748a474b97/pillow-12.3.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9", size = 7105895, upload-time = "2026-07-01T11:56:16.575Z" },
{ url = "https://files.pythonhosted.org/packages/75/18/2e8b40223153ccbc60df07f9e8928dc0c76202aa4e55ae9f53962b6510d6/pillow-12.3.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468", size = 5302510, upload-time = "2026-07-01T11:56:25.736Z" },
{ url = "https://files.pythonhosted.org/packages/46/3e/51fabf59d5ab801ceab709453d3ab6b180083496579549de4c45ced6528a/pillow-12.3.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94", size = 4736058, upload-time = "2026-07-01T11:56:28.041Z" },
{ url = "https://files.pythonhosted.org/packages/bf/20/22fe9384b7949e25fb1293bcfc84fb82590ff4ea6b37c95b24d26d793d86/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e", size = 5237776, upload-time = "2026-07-01T11:56:30.263Z" },
{ url = "https://files.pythonhosted.org/packages/08/14/f6ba68107680ffa74b39985f3f30884e41318fbc4250caa423c79b4788bb/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3", size = 5860358, upload-time = "2026-07-01T11:56:32.68Z" },
{ url = "https://files.pythonhosted.org/packages/6c/d7/af6af1b8cc55e07cb0489d5b0950bf5a9bb1461eeba825700b6d8d0e26ea/pillow-13.0.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:6f38f3f885a09cafc93fd667b1173a92f379e43785d635e7983c893f0da319fb", size = 5592324, upload-time = "2026-10-16T22:15:24.672Z" },
{ url = "https://files.pythonhosted.org/packages/0b/d2/2c12cb4fbb5d95f38d1001ff3d95fafcb830077018cba40e56a8f932dfbc/pillow-13.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:37290fbef0c73e2d51d594eb78fded7dc7170d1c3ad0a890269e6fd8c8c4e061", size = 4856298, upload-time = "2026-10-16T22:15:27.669Z" },
{ url = "https://files.pythonhosted.org/packages/5c/ee/229b27472483a4801ea94990815e710391836afdf987b4b722b62bd75db6/pillow-13.0.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7c88c10dd432f3b39ccdafc9ed81d0c21b8fd634502574da76d1bfa4fe2c76d0", size = 6517304, upload-time = "2026-10-16T22:15:29.43Z" },
{ url = "https://files.pythonhosted.org/packages/36/46/f509b74787f40ce0627501089d630b2c741b63b244c1f40b640b3b01e181/pillow-13.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a625df9a2a5597fa5e5266e01aefc82604bb85d291e3547981e863560d75da6c", size = 7140667, upload-time = "2026-10-16T22:15:31.368Z" },
{ url = "https://files.pythonhosted.org/packages/aa/a4/5a398b8ec2c37145397d6461c0b551c23ec03072aa64f357aed25e582b00/pillow-13.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:10b5019883c98f1dd819ca48ffc6f33a91cee04948fe637c605f4345c7cafd6f", size = 6610583, upload-time = "2026-10-16T22:15:34.025Z" },
{ url = "https://files.pythonhosted.org/packages/4f/7b/f2f1109b230fd4e2d57ab2305de31adc37ea9b4ff8719436c623cb484415/pillow-13.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f391dc92a015c478745192fe4f2e120c2af2aba18d9fe0616341cbf537599806", size = 7218241, upload-time = "2026-10-16T22:15:36.214Z" },
{ url = "https://files.pythonhosted.org/packages/7a/b8/58d79332229482eb0d27c82973850808c19d37e5bdc14443d3417bd373f5/pillow-13.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:67c8d44ad89fa7eb88de5d6ef6199b1e03dd24974f85a67639fc8badb801fe79", size = 5568643, upload-time = "2026-10-16T22:15:44.474Z" },
{ url = "https://files.pythonhosted.org/packages/39/4b/f2ed53d83aa44ac19599ac0a54897f867fafb95f406980d1e24083f8cbe3/pillow-13.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc481f742ccda20e9ed760b8f978dff50f2a4e9bc253ac167fb0a05b0783ef52", size = 4852329, upload-time = "2026-10-16T22:15:46.284Z" },
{ url = "https://files.pythonhosted.org/packages/3b/bb/9e30e8ed18c20800ee327a12107d5249ad8dc444867b3adfcc21654e3c71/pillow-13.0.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8bf295fe459343bc6093c8e5e0df60bfa189e0fcb775a22e2c2488fdeccc6c0d", size = 6519309, upload-time = "2026-10-16T22:15:48.14Z" },
{ url = "https://files.pythonhosted.org/packages/e6/99/fdf65cc539e266ab93546c261bdf98411fe165c6e4e13f9dbf597097192b/pillow-13.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:42d63be3af4f9b951b199b0a9d23365d37ab14687e6d77b74940432c8515c4ca", size = 7144542, upload-time = "2026-10-16T22:15:50.093Z" },
{ url = "https://files.pythonhosted.org/packages/26/9b/f38b7c4d8f599537e2d1a6d546cec9beef7536ff8f5c49a3f9c78179d8ac/pillow-13.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:4bd024d567eca931b668978d672dba914d4b8e2628e2c3073a2ad178de8a2620", size = 6618042, upload-time = "2026-10-16T22:15:52.054Z" },
{ url = "https://files.pythonhosted.org/packages/a8/7c/45ee28531f89ee4582967d8a17f7a26d71ca64cd902b93c269590cd60a00/pillow-13.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:23edf4ec4b6f4f240e8df6e4985e9ee2ced86474632433e55fb1e2548744bf40", size = 7228137, upload-time = "2026-10-16T22:15:54.009Z" },
{ url = "https://files.pythonhosted.org/packages/82/e2/9132d38df320e8099e2642d9fa538f18a7dfebf68a875a31725dc2a115aa/pillow-13.0.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:e98b4f91f99b24b1da72e10621ba181f7281922384a7403094debb4199de6c00", size = 4190620, upload-time = "2026-10-16T22:16:02.527Z" },
{ url = "https://files.pythonhosted.org/packages/cb/be/436dac0996876f65ef1916ffb711c5ef4af2ebc57414b6f16642e07f6b47/pillow-13.0.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:34fb3db1149c879343861831728c243c432d4a5afc8f1878ca7612ef8de2a033", size = 4290286, upload-time = "2026-10-16T22:16:04.861Z" },
{ url = "https://files.pythonhosted.org/packages/60/16/7c1007162e4169ea0d5bec47e2e0b2174f56c50cf5b8c02a1dc23e41037e/pillow-13.0.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:2d50500a4a3bc6469adcdb319cf05d688b08ca3bf0f22a05339f3b25b433a599", size = 3723169, upload-time = "2026-10-16T22:16:06.956Z" },
{ url = "https://files.pythonhosted.org/packages/08/bc/578ab2eaa04705a8978bd9be0fe15c3d43a8fc95097c4f7e39540e79e4fa/pillow-13.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ef5fe6b7db3f501c56000ba8fe7c7e1a35ca1d45824fbeb95a67daa85cff4dbe", size = 5568469, upload-time = "2026-10-16T22:16:08.664Z" },
{ url = "https://files.pythonhosted.org/packages/98/c8/ccede3dcc4f274b80fb49199eed1ef34ba05566c8b71976b9d1f111da2e3/pillow-13.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:f11a3c1358bb3c68012a7b1f41b223c3db9f7bfde90b481df028cfe1c3363313", size = 4852187, upload-time = "2026-10-16T22:16:10.693Z" },
{ url = "https://files.pythonhosted.org/packages/fb/3c/cd0770e96d20bfd990db024bcbd43897ff3e88da1f4782cf90ea231b1d4f/pillow-13.0.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:19b66ca0a4d25cc0200b381ce824c7ea090db61427315e73cfb05dbf1b35c369", size = 6515908, upload-time = "2026-10-16T22:16:12.896Z" },
{ url = "https://files.pythonhosted.org/packages/ef/fe/f5cdaa2d1c4086e757430f368c271a58ee29cfe7378e7ae7b9d964015c25/pillow-13.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5f56499c1a74d2b862beb0c33b8016f99d095858455df9d21583bbbc1c15b582", size = 7142544, upload-time = "2026-10-16T22:16:14.844Z" },
{ url = "https://files.pythonhosted.org/packages/fb/b5/a54f6d44adbcd23737c8c12953f42be107e4cca64e247c0ca3a5b563061c/pillow-13.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a2a14cd73d5e6c927f644b0110ee81f59c4ef506259ec9183904747f2caacc81", size = 6613097, upload-time = "2026-10-16T22:16:16.893Z" },
{ url = "https://files.pythonhosted.org/packages/e6/b4/9e6c1207c21b2e91db0d15c16d30f3701ce5add1bd4da0e787761e66cb6e/pillow-13.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:08b826a9bf1265c76de04a42061d61ea6b109418b7826f28ff1ecdfaa26c5502", size = 7224846, upload-time = "2026-10-16T22:16:19.195Z" },
{ url = "https://files.pythonhosted.org/packages/8e/45/fbad9c7f5e2f1745c373a434d46072bc84eddfefaa28b852425ac957fda6/pillow-13.0.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:44629d4498c9cb6b2051806155f22141df0d8a929c56134b166655232dfb2f8c", size = 4190721, upload-time = "2026-10-16T22:16:27.524Z" },
{ url = "https://files.pythonhosted.org/packages/a2/bf/2074902a259a90692c5dbf132885d21671644cbbd3f690047516243a55db/pillow-13.0.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:bf97ba1b5542050e9b0c74e632323b6d8136cc99d64e193abebee22fe6013c76", size = 4290236, upload-time = "2026-10-16T22:16:29.562Z" },
{ url = "https://files.pythonhosted.org/packages/ff/a5/3d1bb93b8ea38d15d21356c1b8e9f50971ac592aaa248f7fb71739094c0e/pillow-13.0.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:aa694c49787040943570d61c9b8fc01271bd2a2cfdd55f0d896f8aecdce1a981", size = 3723093, upload-time = "2026-10-16T22:16:31.249Z" },
{ url = "https://files.pythonhosted.org/packages/84/a1/be060135d85187b25fa8b2573786c0e45c62ae27527fff5257675eb97f4a/pillow-13.0.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:49c12af515073e9bf511e4ef736743bf70d531a643668722279314b937742ba4", size = 5572253, upload-time = "2026-10-16T22:16:33.103Z" },
{ url = "https://files.pythonhosted.org/packages/b5/f4/0bf6f1ac9ba7ee2ffe9914aeb410a1035a0c1a3c42f122388d7fc950bc4b/pillow-13.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:d9b0a2bee3f3e03e9f29057aedb807dbf1e1d96ee1b9d0f422d9db6c8583271d", size = 4852269, upload-time = "2026-10-16T22:16:35.102Z" },
{ url = "https://files.pythonhosted.org/packages/83/16/b603274617b3e6728da3a0133d4fba1ea8d7fbe0c4ea5fef9cbffa398b8b/pillow-13.0.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5933fcea5a5ba64c686f55c886a115cbfd92f2900460784a1e79909fdbafd7b6", size = 6516058, upload-time = "2026-10-16T22:16:37.228Z" },
{ url = "https://files.pythonhosted.org/packages/1d/f2/1371707ce438ba9a43f9d24089828494ec6f9396e28cd7903aa704dd6306/pillow-13.0.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e91c2b1cbeae768c98752e67e717e283d68dae45717b41f2d91eaee648e37b7d", size = 7142574, upload-time = "2026-10-16T22:16:39.172Z" },
{ url = "https://files.pythonhosted.org/packages/b9/a9/62cf68587f910bc87c5951bcb6e9430c7643a65076adf5f18b4bf350e2bd/pillow-13.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7f6299e494e46253d40c34653cac369412f44ca53b5f85d848d3f95b1ed0546c", size = 6614295, upload-time = "2026-10-16T22:16:41.117Z" },
{ url = "https://files.pythonhosted.org/packages/6c/c9/9e9b48178649730299be0bafbf5f810a5b6977538f5a92d47e4ba5f5d608/pillow-13.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:f9bd7d260d986f3553518c5d723544346fa48031f8aef3265d786aeb59845bc0", size = 7224841, upload-time = "2026-10-16T22:16:43.118Z" },
{ url = "https://files.pythonhosted.org/packages/97/73/a53502f731f6c84805a4ce87a8881e86d07a65aee64728b900d112dc703e/pillow-13.0.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7b999816b5850a35ada08443e6e03945f1f6488144a83afe2d37b240a552aafa", size = 5578196, upload-time = "2026-10-16T22:16:51.291Z" },
{ url = "https://files.pythonhosted.org/packages/21/50/7e949959a56b5559c78ef172b70fc6025706acb289f3f648a15329e4fa6f/pillow-13.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:67871a13cf47275201553b536c025da777f997fa562a19eb798af4ad4b97c18f", size = 4857276, upload-time = "2026-10-16T22:16:53.399Z" },
{ url = "https://files.pythonhosted.org/packages/c9/52/43ded0ac08e77e39d17bd6d894162572201b5e8fdc4a022b14c55a1960a4/pillow-13.0.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e8ca8f69b7c77fa96dad99f8638c1fc4f1c0bed4acaa1baf7a3c31bdb80a0ab1", size = 6607008, upload-time = "2026-10-16T22:16:55.525Z" },
{ url = "https://files.pythonhosted.org/packages/0d/2e/4a57ed216d5bf877387cba4d44c02aa87abe51ad2e06404aa6b597c4da39/pillow-13.0.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6bce6a29c3898191f4c9998e4d4e8f69790caeabde46e54da4bc235ce5bde683", size = 7224281, upload-time = "2026-10-16T22:16:57.738Z" },
{ url = "https://files.pythonhosted.org/packages/5c/3d/2929b248e135752d63aaeabf53f6de797e905b7b6263f9c1e2adade20aba/pillow-13.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:816752a5ef76a80484c8c441c85593b7ea9304f6231fc5776ad1fbff7e773058", size = 6695600, upload-time = "2026-10-16T22:16:59.842Z" },
{ url = "https://files.pythonhosted.org/packages/bc/33/fb0da992cf084ed1ee6ee05e96b6191e35c351eb707ca2df874feb24ce47/pillow-13.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5bdb38ba220b61b4ec31dd1046dea6b016d3c6850a23f43ed8b58332b2fc668e", size = 7300589, upload-time = "2026-10-16T22:17:01.686Z" },
{ url = "https://files.pythonhosted.org/packages/6e/eb/e86a41962a3a31ad495a3d6a912d9002b01bba4a6fa71c28703abdca7f69/pillow-13.0.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7d5e097eabd5d4915c27ac9f6e9325bb1e01a7c94d7348b5c4451d3fc30e8521", size = 4191213, upload-time = "2026-10-16T22:17:09.449Z" },
{ url = "https://files.pythonhosted.org/packages/dd/04/7e7445e706afec84ef06dc25b2c77f9cecff242c23730c62928ad1089d4a/pillow-13.0.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:51ebb529fc5e8abdd55302f6c3f8943037348ec2384329a3e55677f69ae2fdfc", size = 4290650, upload-time = "2026-10-16T22:17:11.196Z" },
{ url = "https://files.pythonhosted.org/packages/46/85/d534089068ac40c0f61f650389a296c812eab745a8a733f81feec39ae955/pillow-13.0.0-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:c87a04e7f152ee4db5dbe15f0bfa3fe628a1aeedbe66b81c43b87f52c813f9ac", size = 3723078, upload-time = "2026-10-16T22:17:12.772Z" },
{ url = "https://files.pythonhosted.org/packages/ce/2b/00b5c3b401df538b5dbe311d3899730fae5ba1e21cb8b80a128887ac6747/pillow-13.0.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:20722cd5375533ad754053363a80b8fb30c79d37930a6f7228030e237ce3bd5d", size = 5572292, upload-time = "2026-10-16T22:17:14.539Z" },
{ url = "https://files.pythonhosted.org/packages/72/f5/a6b0610f946e9b54ae888ba7ad29d6e2f8d9b11f69e212d70416b7186c09/pillow-13.0.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:04a63720f79e22fd54d897c233a68314c50ca0603360f54ce2bd5db8f0f43782", size = 4852222, upload-time = "2026-10-16T22:17:16.521Z" },
{ url = "https://files.pythonhosted.org/packages/1e/d0/c0ff25bb0427ec70f32aac995453321fc3c4494a46c7292ff02d9d260b04/pillow-13.0.0-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4f8db54177035a2b5f717f94475b74907e12a96dc74f6d487f3966b0a4694071", size = 6527349, upload-time = "2026-10-16T22:17:18.534Z" },
{ url = "https://files.pythonhosted.org/packages/85/11/e46fa0bcc0fc117f1eacc206b3d51f04d583fab328b4b740672d0438d83f/pillow-13.0.0-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8f36aa77f74b14a2103622b196402c71330e4e7cf4122ee45c854e9a01e0eecf", size = 7152873, upload-time = "2026-10-16T22:17:20.415Z" },
{ url = "https://files.pythonhosted.org/packages/7c/53/2bdeca705f3a58679fa9201233193c86a30238340ca29b3a152c3fcc719b/pillow-13.0.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:149a1c323ba366f90b48e6f1a6db6582eb50512df374588de56fc909abc0f0f3", size = 6626871, upload-time = "2026-10-16T22:17:22.361Z" },
{ url = "https://files.pythonhosted.org/packages/14/de/3829030f209722ade3753bb456b805a0e7f885cf885b483407365c82e9ca/pillow-13.0.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e892354a424b1e82a78ba16785822bdd40de06f19446a3d4a28047df1a4f8932", size = 7236439, upload-time = "2026-10-16T22:17:24.352Z" },
{ url = "https://files.pythonhosted.org/packages/23/16/e9120defdb8bff693b582d035312acc6f530cdc2bc5238a87f410efccd82/pillow-13.0.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:21c32061a5512dc13f87cc7303e120e4d77c2f70df89a35af0e3ce246e5ac57f", size = 5578129, upload-time = "2026-10-16T22:17:31.642Z" },
{ url = "https://files.pythonhosted.org/packages/47/22/76faf2bb2448f48a84cf894c1e2bb1090d7d2b1aa521ece131cb2f05d631/pillow-13.0.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:1c774b9c0ba0d0b356f71530677b39028878a2cae5a93e0a13779cfd17120f88", size = 4857428, upload-time = "2026-10-16T22:17:33.43Z" },
{ url = "https://files.pythonhosted.org/packages/b3/64/0861ea4e03c50b14af9ed9dd8b9f388ab42f445c623a9fcbab19aef7aa55/pillow-13.0.0-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:20822140c9a2ba9c5e42df342b0a892ab2e102fc0e3db967a133bb9ce3f898bb", size = 6612582, upload-time = "2026-10-16T22:17:35.415Z" },
{ url = "https://files.pythonhosted.org/packages/5c/20/cd1e9b3f60a9f3d5ebe46da36955c26004092a501f7f2c42f9fcc26611bc/pillow-13.0.0-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ffea6b2986a797c84b6d1d27d4f43e0b83c73fc16281a930891cd237cf0d08fe", size = 7230176, upload-time = "2026-10-16T22:17:37.362Z" },
{ url = "https://files.pythonhosted.org/packages/b6/ef/ffb0a2fc85690714fb7ac662956949645d57f15bf4426f520da361e7b6e3/pillow-13.0.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:05cd2cd39055d0b9650060e4513ef4a9a0aea870083950bd046a0d6490f93a22", size = 6703640, upload-time = "2026-10-16T22:17:39.313Z" },
{ url = "https://files.pythonhosted.org/packages/e0/c2/6402ee3ccb3363aeed8fdb3f9fe56703ac5377a539f0b81b53999d7e121f/pillow-13.0.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:c6fd031333dedcad7ce3e03d244093f43504f209e86925ebdb2b868fa4037630", size = 7309231, upload-time = "2026-10-16T22:17:41.021Z" },
{ url = "https://files.pythonhosted.org/packages/68/c0/fb7b1caf25ed5ac5d46116f09b8b0706b9559833c65c667cecb1a53153f2/pillow-13.0.0-pp311-pypy311_pp80-macosx_10_15_x86_64.whl", hash = "sha256:405be6058e206a86fedce2f715ecc2343f94072d7925251632dffa2c05f70dca", size = 5521630, upload-time = "2026-10-16T22:17:50.008Z" },
{ url = "https://files.pythonhosted.org/packages/1a/e6/4d11d5b1137f4fc080605cce410cfaae4f303ad400bdec75d27d3083fa29/pillow-13.0.0-pp311-pypy311_pp80-macosx_11_0_arm64.whl", hash = "sha256:b84be41387fc00db28c13c02e30d42b02d8d8f6a4a741a1aee6012ca7f1f6039", size = 4806023, upload-time = "2026-10-16T22:17:51.932Z" },
{ url = "https://files.pythonhosted.org/packages/f1/45/0923202a658e8ff6aafd72c1a4395ebadcdebaa182c437d8595d3dd3db32/pillow-13.0.0-pp311-pypy311_pp80-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a192360d512305fe92d3053e4f7627bbad1f7708f1f4ad38cccd9336acabe9ad", size = 5464831, upload-time = "2026-10-16T22:17:53.742Z" },
{ url = "https://files.pythonhosted.org/packages/07/52/d3e38f0cad1dadea2a658b9a7d6ba8a04048f5ae14f011b806e5c3bd6bcb/pillow-13.0.0-pp311-pypy311_pp80-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:83b37abd87b590acd8f47ca8a61f70893322b11834d36e07c2298dacad4d14d2", size = 6043133, upload-time = "2026-10-16T22:17:55.625Z" },
{ url = "https://files.pythonhosted.org/packages/60/da/f6d644e631c5b370c5d0b899f3ccadd2780aab070b29a110527cf4e1fd2c/pillow-13.0.0-pp312-pypy312_pp80-macosx_10_15_x86_64.whl", hash = "sha256:8619685cad8efc4c38c66d3b36aad854523297c43d748b3b6f0cc03c90ba3463", size = 5523188, upload-time = "2026-10-16T22:17:59.334Z" },
{ url = "https://files.pythonhosted.org/packages/e4/b0/2594f202d743a35d4fe80682bfd96f277e5ca7aecb27cf64ef5bfc1d000b/pillow-13.0.0-pp312-pypy312_pp80-macosx_11_0_arm64.whl", hash = "sha256:7e91e97a929232cdb828889ee6ad0e48d7f40f4ec38818f4e4a8968b2b1af22e", size = 4806808, upload-time = "2026-10-16T22:18:01.188Z" },
{ url = "https://files.pythonhosted.org/packages/8e/ed/d504165ef79af6fafe25c76726cf30f544cc236002f7ed054f1d51adc4ff/pillow-13.0.0-pp312-pypy312_pp80-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:93ff9c9c3663ce23aaa5d0b6b6abac55359e2a0c06c7e30f3771829af816be63", size = 5464358, upload-time = "2026-10-16T22:18:02.779Z" },
{ url = "https://files.pythonhosted.org/packages/7b/fe/a10aba5503ca13e9e009b8bd0a1a4ce5584bdce246245fd6f3a83701369c/pillow-13.0.0-pp312-pypy312_pp80-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e65dd6e75de67b2362a4fa3e16aa627595f65054dc4ec73d64680c84d592fa61", size = 6043514, upload-time = "2026-10-16T22:18:04.538Z" },
]
[[package]]
@@ -3696,7 +3700,7 @@ wheels = [
[package.optional-dependencies]
c = [
{ name = "psycopg-c", version = "3.3.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.12' and implementation_name != 'pypy' and platform_machine == 'aarch64') or (python_full_version == '3.13.*' and implementation_name != 'pypy' and platform_machine == 'aarch64') or (python_full_version >= '3.15' and implementation_name != 'pypy' and platform_machine == 'aarch64') or (python_full_version < '3.12' and implementation_name != 'pypy' and platform_machine == 'x86_64') or (python_full_version == '3.13.*' and implementation_name != 'pypy' and platform_machine == 'x86_64') or (python_full_version >= '3.15' and implementation_name != 'pypy' and platform_machine == 'x86_64') or (implementation_name != 'pypy' and platform_machine != 'aarch64' and platform_machine != 'x86_64') or (implementation_name != 'pypy' and sys_platform != 'linux')" },
{ name = "psycopg-c", version = "3.3.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.12' and implementation_name != 'pypy') or (python_full_version == '3.13.*' and implementation_name != 'pypy') or (python_full_version >= '3.15' and implementation_name != 'pypy') or (implementation_name != 'pypy' and platform_machine != 'aarch64' and platform_machine != 'x86_64') or (implementation_name != 'pypy' and sys_platform != 'linux')" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl" }, marker = "python_full_version == '3.12.*' and implementation_name != 'pypy' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" }, marker = "python_full_version == '3.12.*' and implementation_name != 'pypy' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" }, marker = "python_full_version == '3.14.*' and implementation_name != 'pypy' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
@@ -3714,7 +3718,7 @@ resolution-markers = [
"python_full_version >= '3.15' and sys_platform == 'darwin'",
"python_full_version >= '3.15' and sys_platform == 'linux'",
"python_full_version >= '3.12' and python_full_version < '3.15' and sys_platform == 'darwin'",
"(python_full_version >= '3.12' and python_full_version < '3.15' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and platform_machine == 'aarch64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and platform_machine == 'x86_64' and sys_platform == 'linux')",
"(python_full_version == '3.12.*' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.14.*' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and sys_platform == 'linux')",
"python_full_version < '3.12' and sys_platform == 'darwin'",
"python_full_version < '3.12' and sys_platform == 'linux'",
]
@@ -5256,7 +5260,7 @@ resolution-markers = [
"python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux'",
"python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux'",
"python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux'",
"(python_full_version >= '3.12' and python_full_version < '3.15' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and platform_machine == 'aarch64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and platform_machine == 'x86_64' and sys_platform == 'linux')",
"(python_full_version == '3.12.*' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.14.*' and platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux') or (python_full_version == '3.13.*' and sys_platform == 'linux')",
"python_full_version < '3.12' and sys_platform == 'linux'",
]
dependencies = [