Compare commits

..
Author SHA1 Message Date
stumpylog d7084ee1ab perf: use imap_tools uid_list fetch to skip manual UID batching
imap_tools 1.14.0 adds a uid_list fetch arg that fetches by UID
directly without issuing a SEARCH command, and does its own
bulk-fetch batching. Use it in place of the manual per-batch
AND(uid=...) fetch loop.
2026-07-24 12:57:14 -07:00
stumpylog 8391b4c524 perf: batch body fetches when many new UIDs are pending 2026-07-21 10:32:05 -07:00
stumpylog c53a336ccd fix: diff UIDs against ProcessedMail before fetching message bodies 2026-07-21 10:01:14 -07:00
stumpylogandClaude Sonnet 5 ed37f667e4 fix: record ProcessedMail for attachment-less mail under attachments-only rules
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 09:52:01 -07:00
stumpylogandClaude Sonnet 5 9ede2fbd06 test: add BogusMailBox.uids() and UID criteria support
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 09:35:22 -07:00
Trenton Holmes a22b9fdbe4 Store ideas for making mail fetch more efficient 2026-07-20 19:31:36 -07:00
d8d8872414 Fix (beta): compute num_notes via a subquery instead of Count()+distinct() (#13182)
DocumentViewSet.get_queryset() annotated num_notes via a LEFT JOIN to
documents_note plus Count(), which requires the database to aggregate
every matching document's note count before it can even sort or paginate
the result -- on every list request, not just filtered ones.

Switched to the same correlated-subquery pattern already used two lines
above for effective_content (Subquery + OuterRef), which Django compiles
to portable SQL across sqlite/postgresql/mariadb rather than a join-based
aggregate.

Benchmarked against a 100k synthetic document corpus (Postgres): the
plain document list request dropped from ~2.2-2.4s to ~0.9-1.0s. Note
that a separate, larger cost remains in the same queryset's plain
.distinct() call, which still forces a full sort over every matching
row regardless of this fix -- tracked separately, not addressed here.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 13:59:44 -07:00
shamoonandGitHub ff609c2987 Change: update root modified timestamp on version changes (#13170) 2026-07-20 15:45:24 +00:00
Trenton HandGitHub 2b784e709b Fix: prevent tag assignment from reverting other pending workflow assignments (#13178)
apply_assignment_to_document() mutated tags directly on the shared document instance via add_nested_tags(), whose m2m_changed signal triggers update_filename_and_move_files() -> instance.refresh_from_db(), discarding any not-yet-saved fields (e.g. storage_path) staged by an earlier-ordered action in the same workflow. Apply tag changes to a freshly-fetched instance instead, matching the pattern already used in apply_removal_to_document().
2026-07-20 08:22:30 -07:00
Trenton HandGitHub c9716252f0 Performance: Add DB indexes for common query/sort patterns (#13167)
Adds indexes on Document.checksum, Document.page_count, and a composite
(owner, created) index on Document, plus composite (field, value_*)
indexes on CustomFieldInstance for each typed value column.

Benchmarked against a 100k-document / 200k-custom-field-instance SQLite
dataset: Document.checksum lookups ~69x faster, page_count sort ~40x,
owner+created list queries ~6x. CustomFieldInstance composite indexes
show large gains (10-24x) on realistic narrow-selectivity queries
matching the workflow scheduler and custom-field search code paths.
2026-07-20 13:12:54 +00:00
shamoon 5cf9152a40 Merge branch 'dev' into beta 2026-07-19 20:20:48 -07:00
shamoonandGitHub 80210bd3bf Performance: use NgOptimizedImage for thumbnail lazy loading (#13169) 2026-07-19 18:05:51 -07:00
4e52dd5710 Fix: cache per-request effective-document resolution for thumb/metadata/preview (#13166)
Django's condition() decorator invokes etag_func and last_modified_func
separately, and the view itself may resolve again -- each call to
resolve_effective_document_by_pk() was redoing the same root/version
DB lookups. Memoize the resolution on the request object so a single
thumb/metadata/preview request resolves the effective document once
instead of up to three times.

Related to paperless-ngx/paperless-ngx#13161.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-19 18:04:16 -07:00
GitHub Actions d34ef75786 Auto translate strings 2026-07-19 22:48:24 +00:00
shamoon b8c424c8e1 Update document-detail.component.html 2026-07-19 15:46:17 -07:00
GitHub Actions ec8991c2ec Auto translate strings 2026-07-19 20:56:43 +00:00
shamoonandGitHub 4f6d9fa93f Fix: dont allow archive version for share links when one doesnt exist (#13163) 2026-07-19 20:54:58 +00:00
GitHub Actions 2180b21c41 Auto translate strings 2026-07-19 14:45:46 +00:00
shamoonandGitHub 99bdfdfe7a Fix: better handle saved view errors in dashboard widgets (#13164) 2026-07-19 07:44:12 -07:00
GitHub Actions 73a0586a75 Auto translate strings 2026-07-18 06:00:02 +00:00
shamoon dcba44ad78 Merge branch 'beta' into dev 2026-07-17 22:57:47 -07:00
shamoon ed54f1a8b9 Fix: more signal-backed conversions 2026-07-17 22:57:37 -07:00
shamoonandGitHub f8a641b402 Fix (beta): convert chat component to signal-backed (#13152) 2026-07-17 22:26:07 -07:00
stumpylog e85223929c Fix: increase checks-discovery-timeout for Wait for Docker Build
The Merge and Push Manifest check isn't materialized until the
build-arch matrix completes (its `if` gates on a job output), which
routinely takes longer than the action's 60s discovery window,
causing the release workflow to bail before Docker builds finish.
2026-07-17 15:27:55 -07:00
stumpylog f2e3304576 Merge remote-tracking branch 'origin/dev' into beta 2026-07-17 15:16:36 -07:00
6938fb0e01 New Crowdin translations by GitHub Action (#13134)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-17 15:16:21 -07:00
Trenton HandGitHub 255c214d0b Fix: avoid HAVING clause conflicts in custom field query filter on MariaDB (#13143) 2026-07-17 13:44:58 -07:00
Trenton HandGitHub a0ab05a9a6 Chore: Upgrade more vulerable packages (#13020)
* Few more before

* Bump django to 5.2.16

* Bump daphne to 4.2.2

* Bump transformers to 5.13.1

* Bump nltk to 3.10.0
2026-07-17 12:01:30 -07:00
df1ddb15cc Performance: Tantivy indexing optimization (#13053)
* Tantivy: get permissions by chunks

-40% indexing time compared to previous commit

* Make progress bar process one by one with chunk

-15% indexing time compared to previous commit

* Prefetch FK + iterate over chunk from SQL

Prefetch additional needed data (note user, custom field content)

-20% indexing time compared to previous commit

* Reindex: increase Tantivy heap size from 128 to 512MB

Gains probably vary depending on the machine,
but it seems a sweet spot compatible with low-end hardware.

* Reindex: optimization on permission fetching and autocomplete word set

-10% indexing time compared to previous commit

* Autocomplete analyzer python->rust

Splits words with underscore compared to the python analyzer.
E.g.: "blue_print" -> ["blue", "print"]
It can still be found with the "blue_print" keyword,
as the search string is also split in two words.

-50% indexing time compared to previous commit (indexing is twice faster!)

* Index bigram for CJK content only

Inedxing time slightly longer (~3%),
but since the non-CJK content is not indexed,
bigram searchs will be slightly optimized.

* Fix group-based view_document permissions missing from bulk rebuild

_bulk_get_viewer_ids only queried UserObjectPermission, dropping the
group-permission expansion that get_users_with_perms(with_group_users=True)
performs for the non-batched per-document indexing path. A user who could
only see a document via group membership would lose search access to it
after any full reindex.

Also query GroupObjectPermission and expand group membership to user ids,
matching the existing single-document behavior.

* Yield (document, viewer_ids) pairs from _DocumentViewerStream

Previously _DocumentViewerStream.__iter__ yielded plain Document objects
while the matching viewer ids were exposed through a separate mutable
attribute (viewer_ids_by_pk), overwritten each time the generator crossed
a chunk boundary. rebuild() read that attribute out-of-band per document.

This only worked because the current iter_wrapper (a plain progress-bar
passthrough) happens to consume the stream in strict lock-step with no
lookahead. Any wrapper that buffers, batches, or reorders would silently
pair a document with the wrong chunk's viewer ids. Yield the pair directly
so the association travels with the document regardless of how iter_wrapper
consumes the stream, and drop the now-unneeded viewer_ids_by_pk attribute.

* Add --heap-size-mb CLI arg to document_index reindex

writer_heap_bytes was hardcoded at 512MB with no way to tune it. Expose it
as a manual-rebuild-only CLI arg rather than a settings/env var, per review
feedback, so lower-memory hosts can reduce it without a wider config
surface. Defaults to unset so TantivyBackend.rebuild's own default stays
the single source of truth.

---------

Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-17 11:33:09 -07:00
GitHub Actions 71557d7c64 Auto translate strings 2026-07-16 17:43:31 +00:00
shamoon 66f5111dc1 Chore: some frontend sonar fixes 2026-07-16 10:42:00 -07:00
dependabot[bot]andGitHub 0addb448f5 Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 4 updates (#13137) 2026-07-15 23:52:20 +00:00
dependabot[bot]andGitHub 5428ea5caa Chore(deps-dev): Bump the development group with 2 updates (#13068)
Signed-off-by: dependabot[bot] <support@github.com>
2026-07-15 21:04:28 +00:00
GitHub Actions 2de2dfdb81 Auto translate strings 2026-07-15 20:45:19 +00:00
Hugo Beauzée-LuyssenandGitHub f0d934259c Fix: scope ProcessedMail UID deduplication by IMAP UIDVALIDITY (#12839) 2026-07-15 13:43:16 -07:00
Albert MikaelyanandGitHub 735c3b073a Fix: allow setting any UID and GID when running rootless (#13090) 2026-07-15 10:19:22 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e2cf3633fe docker-compose(deps): bump greenmail/standalone in /docker/compose (#13067)
Bumps greenmail/standalone from 2.1.8 to 2.1.9.

---
updated-dependencies:
- dependency-name: greenmail/standalone
  dependency-version: 2.1.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 09:23:10 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
c0589a9ce9 docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#13066)
Bumps gotenberg/gotenberg from 8.33 to 8.34.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.34'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 09:09:30 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2135572adb docker-compose(deps): bump nginx in /docker/compose (#13065)
Bumps nginx from 1.31.1-alpine to 1.31.2-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.2-alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 15:55:33 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
697e262b5d docker(deps): Bump astral-sh/uv (#13070)
Bumps [astral-sh/uv](https://github.com/astral-sh/uv) from 0.11.19-python3.12-trixie-slim to 0.11.28-python3.12-trixie-slim.
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/uv/compare/0.11.19...0.11.28)

---
updated-dependencies:
- dependency-name: astral-sh/uv
  dependency-version: 0.11.26-python3.12-trixie-slim
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 15:37:35 +00:00
GitHub Actions 5a65c4448d Auto translate strings 2026-07-15 15:14:34 +00:00
shamoonandGitHub 370a0a29ba Fix (beta): fix sidebar document close button (#13136) 2026-07-15 08:12:56 -07:00
536007f42b New Crowdin translations by GitHub Action (#12722)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-14 09:29:42 -07:00
shamoonandGitHub 71e2dd4b08 Fix: include wasm assets in pdf.js-based viewer (#13122) 2026-07-11 22:14:38 -07:00
GitHub Actions dabd7826d0 Auto translate strings 2026-07-11 23:14:15 +00:00
shamoonandGitHub 054b234739 Fix: fix ui labels from md5 to sha256 (#13119) 2026-07-11 16:12:47 -07:00
GitHub Actions 77dd4d1b0c Auto translate strings 2026-07-11 23:05:28 +00:00
shamoon 56dc3e426e Merge branch 'dev' into beta 2026-07-10 11:34:15 -07:00
GitHub Actions be8890cfaf Auto translate strings 2026-07-10 18:24:51 +00:00
shamoonandGitHub e03b09e2ea Chore: pnpm lockfile cleanup (#13116) 2026-07-10 11:23:19 -07:00
shamoon 19960cc4fa Merge branch 'dev' into beta 2026-07-10 00:44:47 -07:00
shamoonandGitHub e145fe8cc7 Fix: clamp mailrule max age before migration (#13093) 2026-07-07 10:56:45 -07:00
shamoonandGitHub fce55a1609 Fix: use latest document version for bulk download (#13103) 2026-07-07 10:55:40 -07:00
shamoonandGitHub 1dabd2601d Fix: handle relative date with comma query (#13087) 2026-07-03 12:44:33 -07:00
shamoon 88f69841b3 Merge branch 'dev' into beta 2026-07-02 15:06:19 -07:00
shamoon 5b8fbdcec7 Merge branch 'dev' into beta 2026-06-27 15:56:20 -07:00
shamoon beb048b94a Merge branch 'dev' into beta 2026-06-27 06:53:09 -07:00
shamoonandGitHub b33d11778a Fix: change tantivy token limit to title field max length (#13046) 2026-06-23 08:02:26 -07:00
240 changed files with 89123 additions and 55335 deletions
+1
View File
@@ -26,6 +26,7 @@ jobs:
check-name: 'Merge and Push Manifest'
repo-token: ${{ secrets.GITHUB_TOKEN }}
wait-interval: 60
checks-discovery-timeout: 1800
build-release:
name: Build Release
needs: wait-for-docker
+4 -4
View File
@@ -38,7 +38,7 @@ repos:
- json
# See https://github.com/prettier/prettier/issues/15742 for the fork reason
- repo: https://github.com/rbubley/mirrors-prettier
rev: 'v3.8.4'
rev: 'v3.9.4'
hooks:
- id: prettier
types_or:
@@ -46,16 +46,16 @@ repos:
- ts
- markdown
additional_dependencies:
- prettier@3.8.3
- prettier@3.9.4
- 'prettier-plugin-organize-imports@4.3.0'
# Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.17
rev: v0.15.20
hooks:
- id: ruff-check
- id: ruff-format
- repo: https://github.com/tox-dev/pyproject-fmt
rev: "v2.24.1"
rev: "v2.25.1"
hooks:
- id: pyproject-fmt
additional_dependencies: [tomli]
+5 -1
View File
@@ -30,7 +30,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs
# Comments:
# - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.11.19-python3.12-trixie-slim AS s6-overlay-base
FROM ghcr.io/astral-sh/uv:0.11.28-python3.12-trixie-slim AS s6-overlay-base
WORKDIR /usr/src/s6
@@ -238,6 +238,10 @@ RUN set -eux \
&& chown --from root:root --changes --recursive paperless:paperless /usr/src/paperless \
&& echo "Making fontconfig cache writable for arbitrary container UIDs" \
&& chmod 1777 /var/cache/fontconfig \
&& echo "Making /run world-writable for rootless operation" \
&& chmod 1777 /run \
&& echo "Removing setuid from s6-overlay-suexec for rootless compat" \
&& chmod u-s /command/s6-overlay-suexec \
&& echo "Collecting static files" \
&& PAPERLESS_SECRET_KEY=build-time-dummy s6-setuidgid paperless python3 manage.py collectstatic --clear --no-input --link \
&& PAPERLESS_SECRET_KEY=build-time-dummy s6-setuidgid paperless python3 manage.py compilemessages \
+3 -3
View File
@@ -4,7 +4,7 @@
# correct networking for the tests
services:
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
hostname: gotenberg
container_name: gotenberg
network_mode: host
@@ -24,7 +24,7 @@ services:
network_mode: host
restart: unless-stopped
greenmail:
image: docker.io/greenmail/standalone:2.1.8
image: docker.io/greenmail/standalone:2.1.9
hostname: greenmail
container_name: greenmail
environment:
@@ -35,7 +35,7 @@ services:
- "3143:3143" # IMAP
restart: unless-stopped
nginx:
image: docker.io/nginx:1.31.1-alpine
image: docker.io/nginx:1.31.2-alpine
hostname: nginx
container_name: nginx
ports:
@@ -72,7 +72,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.
@@ -67,7 +67,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.
@@ -56,7 +56,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.
+143
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -326,3 +326,11 @@ behind a reverse proxy may need to set
[`PAPERLESS_TRUSTED_PROXIES`](configuration.md#PAPERLESS_TRUSTED_PROXIES),
[`PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER`](configuration.md#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER),
or both, to avoid `403 Forbidden` errors on login.
## Database Migrations
Some integer fields have been changed to smaller types to reduce database size. If you have any `MailRule` records with a `maximum_age` greater than 32767, they will be clamped to 32767 during the migration to avoid errors during migration.
### Action Required
No user action is required. The migration will automatically clamp any `MailRule.maximum_age` values greater than 32767 to 32767 during the migration process.
+1 -1
View File
@@ -142,7 +142,7 @@ a [superuser](usage.md#superusers) account.
It is not possible to run the container rootless if additional languages are specified via `PAPERLESS_OCR_LANGUAGES`.
If you want to run Paperless as a rootless container, set `user:` in `docker-compose.yml` to the UID and GID of your host user (use `id -u` and `id -g` to find these values). The container process starts directly as that user with no internal privilege remapping:
If you want to run Paperless as a rootless container, set `user:` in `docker-compose.yml` to the UID and GID of your host user (use `id -u` and `id -g` to find these values). The container process starts directly as that user with no internal privilege remapping. Any UID and GID is supported:
```yaml
webserver:
@@ -0,0 +1,662 @@
# Mail Fetch Re-download Fix Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stop `paperless_mail` from re-downloading full IMAP message bodies for mail it has already handled, and make sure attachment-less mail under attachments-only rules is recorded so it stops matching the search forever.
**Architecture:** In `_handle_mail_rule`, replace the single "fetch every matching message's full body" call with a cheap `UID SEARCH` first, subtract UIDs already in `ProcessedMail`, and fetch bodies only for the remainder (batched). Separately, extract the existing "record `PROCESSED_WO_CONSUMPTION`" block into a shared helper and call it from the no-attachments-under-attachments-only-scope early return in `_handle_message`, which currently skips it entirely.
**Tech Stack:** Django (pytest-django), `imap_tools` (`MailBox.uids()`, `AND` query builder), existing `ProcessedMail` model — no new dependencies, no migrations.
## Global Constraints
- No new required settings and no DB migrations for existing installs (spec: "No new required settings/migrations for existing installs").
- Batch size for body fetches is a hardcoded module constant (`MAIL_FETCH_BATCH_SIZE = 500`), not user-configurable.
- No mailbox action (mark-read/flag/tag/move/delete) is applied for the no-attachment/no-consumption case — only a `ProcessedMail` row is written, matching the existing precedent for other no-consumption outcomes.
- Every path that currently produces a document or applies a mail action must behave exactly as before (no change to `_process_attachments`'/`_process_eml`'s consuming behavior, action application, or the existing per-message `already_processed` dedup check, which stays in place as a safety net).
- Follow the existing per-message `uid_validity` matching semantics exactly: when `self._current_uid_validity is not None`, only match `ProcessedMail` rows with the same `uid_validity` or `uid_validity IS NULL`; when it is `None` (server didn't report UIDVALIDITY), fall back to matching on `(rule, uid, folder)` alone, ignoring `uid_validity`. (This exact conditional already exists at `mail.py:722-726` for the per-message check — the new UID-diff query must replicate it or several existing uidvalidity tests will break.)
---
### Task 1: Add UID search support to the `BogusMailBox` test double
**Files:**
- Modify: `src/paperless_mail/tests/test_mail.py:137-171` (`BogusMailBox.fetch`)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Produces: `BogusMailBox.uids(self, criteria, charset="") -> list[str]`, filtering with the same rules as `fetch()`, returning only UIDs (no message bodies). `BogusMailBox.fetch()` gains support for a `UID <comma-list>` criteria token (used by real code's `AND(uid=[...])` queries).
- Consumes: nothing new — this is test infrastructure only, no production code changes in this task.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py` (near the other `BogusMailBox`-adjacent tests, e.g. after `test_handle_message`):
```python
def test_bogus_mailbox_uids_and_uid_criteria(self) -> None:
mailbox = self.mailMocker.bogus_mailbox
all_messages = list(mailbox.messages)
# uids() returns the UIDs of unseen messages, no bodies needed to call it
unseen_uids = mailbox.uids("(UNSEEN)")
self.assertEqual(
set(unseen_uids),
{m.uid for m in all_messages if not m.seen},
)
# fetch() with an explicit UID criteria returns only the matching messages
target_uid = all_messages[0].uid
from imap_tools import AND
fetched = mailbox.fetch(AND(uid=[target_uid]), mark_seen=False)
self.assertEqual([m.uid for m in fetched], [target_uid])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_bogus_mailbox_uids_and_uid_criteria -v`
Expected: FAIL with `AttributeError: 'BogusMailBox' object has no attribute 'uids'` (and/or the UID-criteria fetch returning all 3 default messages instead of 1, since `fetch()` doesn't understand a `UID` token yet).
- [ ] **Step 3: Implement `uids()` and `UID` criteria support**
Replace `src/paperless_mail/tests/test_mail.py:137-171`:
```python
def fetch(self, criteria, mark_seen, charset="", *, bulk=True):
return self._filter_messages(criteria)
def uids(self, criteria, charset="") -> list[str]:
return [m.uid for m in self._filter_messages(criteria)]
def _filter_messages(self, criteria):
msg = self.messages
criteria = str(criteria).strip("()").split(" ")
if "UNSEEN" in criteria:
msg = filter(lambda m: not m.seen, msg)
if "SUBJECT" in criteria:
subject = criteria[criteria.index("SUBJECT") + 1].strip('"')
msg = filter(lambda m: subject in m.subject, msg)
if "BODY" in criteria:
body = criteria[criteria.index("BODY") + 1].strip('"')
msg = filter(lambda m: body in m.text, msg)
if "FROM" in criteria:
from_ = criteria[criteria.index("FROM") + 1].strip('"')
msg = filter(lambda m: from_ in m.from_, msg)
if "TO" in criteria:
to_ = criteria[criteria.index("TO") + 1].strip('"')
msg = filter(lambda m: any(to_ in to_addr for to_addr in m.to), msg)
if "UNFLAGGED" in criteria:
msg = filter(lambda m: not m.flagged, msg)
if "UNKEYWORD" in criteria:
tag = criteria[criteria.index("UNKEYWORD") + 1].strip("'")
msg = filter(lambda m: tag not in m.flags, msg)
if "(X-GM-LABELS" in criteria: # ['NOT', '(X-GM-LABELS', '"processed"']
msg = filter(lambda m: "processed" not in m.flags, msg)
if "UID" in criteria:
uid_list = criteria[criteria.index("UID") + 1].split(",")
msg = filter(lambda m: m.uid in uid_list, msg)
return list(msg)
```
This is a pure refactor of the existing filtering logic into `_filter_messages`, reused by both `fetch()` (unchanged behavior) and the new `uids()`, plus one new `if "UID" in criteria` branch.
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_bogus_mailbox_uids_and_uid_criteria -v`
Expected: PASS
- [ ] **Step 5: Run the full existing `test_mail.py` suite to confirm the refactor didn't change `fetch()` behavior**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all tests PASS (same pass count as before this task)
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/tests/test_mail.py
git commit -m "test: add BogusMailBox.uids() and UID criteria support"
```
---
### Task 2: Record `ProcessedMail` for attachment-less mail under attachments-only rules
**Files:**
- Modify: `src/paperless_mail/mail.py:749-793` (`_handle_message`)
- Modify: `src/paperless_mail/mail.py:952-979` (`_process_attachments`, tail)
- Test: `src/paperless_mail/tests/test_mail.py:540-548` (`test_handle_empty_message`, rewritten)
**Interfaces:**
- Produces: `MailAccountHandler._record_processed_without_consumption(self, message: MailMessage, rule: MailRule) -> None` — idempotently writes a `ProcessedMail(status="PROCESSED_WO_CONSUMPTION")` row for `(rule, message.uid, rule.folder)` if one doesn't already exist for the current `self._current_uid_validity`. No mailbox action is applied.
- Consumes: `self._current_uid_validity` (already set by `_handle_mail_rule` before `_handle_message` is called; `None` when called directly, e.g. in unit tests).
- [ ] **Step 1: Write the failing test**
Replace `src/paperless_mail/tests/test_mail.py:540-548` (`test_handle_empty_message`):
```python
def test_handle_empty_message(self) -> None:
message = self.mailMocker.messageBuilder.create_message(
subject="No attachments here",
attachments=[],
)
account = MailAccount.objects.create()
rule = MailRule.objects.create(
account=account,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
result = self.mail_account_handler._handle_message(message, rule)
self.mailMocker._queue_consumption_tasks_mock.assert_not_called()
self.assertEqual(result, 0)
processed = ProcessedMail.objects.get(
rule=rule,
uid=message.uid,
folder=rule.folder,
)
self.assertEqual(processed.status, "PROCESSED_WO_CONSUMPTION")
# Calling it again must not create a second row
self.mail_account_handler._handle_message(message, rule)
self.assertEqual(
ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
).count(),
1,
)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_empty_message -v`
Expected: FAIL — `ProcessedMail.DoesNotExist` (no row is created by the current early return).
- [ ] **Step 3: Add the shared helper and call it from both places**
Insert a new method right after `_handle_message` ends, i.e. after `src/paperless_mail/mail.py:793` (`return processed_elements`), before `def filename_inclusion_matches`:
```python
def _record_processed_without_consumption(
self,
message: MailMessage,
rule: MailRule,
) -> None:
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date)
if is_naive(message.date)
else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
```
Then modify `_handle_message`'s early return at `src/paperless_mail/mail.py:756-760`:
```python
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements
```
becomes:
```python
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
self._record_processed_without_consumption(message, rule)
return processed_elements
```
Then replace the tail of `_process_attachments` at `src/paperless_mail/mail.py:952-979`:
```python
if len(consume_tasks) > 0:
queue_consumption_tasks(
consume_tasks=consume_tasks,
rule=rule,
message=message,
uid_validity=self._current_uid_validity,
)
else:
# No files to consume, just mark as processed if it wasn't by .eml processing
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date)
if is_naive(message.date)
else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
return processed_attachments
```
with:
```python
if len(consume_tasks) > 0:
queue_consumption_tasks(
consume_tasks=consume_tasks,
rule=rule,
message=message,
uid_validity=self._current_uid_validity,
)
else:
# No files to consume, just mark as processed if it wasn't by .eml processing
self._record_processed_without_consumption(message, rule)
return processed_attachments
```
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_empty_message -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite (regression check for the `_process_attachments` refactor)**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "fix: record ProcessedMail for attachment-less mail under attachments-only rules"
```
---
### Task 3: Diff UIDs against `ProcessedMail` before fetching message bodies
**Files:**
- Modify: `src/paperless_mail/mail.py:641-693` (`_handle_mail_rule`, the fetch section)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Consumes: `MailAccountHandler._record_processed_without_consumption` (Task 2), `ProcessedMail` model, `imap_tools.AND`, `M.uids()` (Task 1's `BogusMailBox.uids()` in tests; `imap_tools.MailBox.uids()` in production).
- Produces: no new public interface — `_handle_mail_rule`'s external behavior (return value, exceptions raised) is unchanged; only its internal fetch strategy changes. `_handle_mail_rule` now returns `0` immediately, without calling `M.fetch()` at all, when every UID matching the search criteria already has a `ProcessedMail` row.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py`:
```python
def test_handle_mail_account_skips_body_fetch_for_already_processed_mail(
self,
) -> None:
"""
GIVEN:
- An attachment-less mail under an attachments-only mark-read rule,
already recorded as PROCESSED_WO_CONSUMPTION
WHEN:
- The mail account is processed again and the mail still matches the
search criteria (it was never marked read, since no mail action is
applied for the no-consumption case)
THEN:
- No IMAP body fetch happens for that mail; only the cheap UID search runs.
"""
account = MailAccount.objects.create(
name="test",
imap_server="",
username="admin",
password="secret",
)
rule = MailRule.objects.create(
name="testrule",
account=account,
action=MailRule.MailAction.MARK_READ,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
message = self.mailMocker.messageBuilder.create_message(
subject="No attachment",
attachments=[],
)
self.mailMocker.bogus_mailbox.messages = [message]
self.mailMocker.bogus_mailbox.updateClient()
# First run: records ProcessedMail without consuming anything.
self.mail_account_handler.handle_mail_account(account)
self.assertTrue(
ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
).exists(),
)
self.mailMocker._queue_consumption_tasks_mock.assert_not_called()
# Second run: message still matches UNSEEN (mark-read action never ran),
# but its body must not be downloaded again.
with mock.patch.object(
self.mailMocker.bogus_mailbox,
"fetch",
wraps=self.mailMocker.bogus_mailbox.fetch,
) as fetch_spy:
self.mail_account_handler.handle_mail_account(account)
fetch_spy.assert_not_called()
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_skips_body_fetch_for_already_processed_mail -v`
Expected: FAIL — `fetch_spy.assert_not_called()` fails because the current code always calls `M.fetch()` for every message matching the search criteria, regardless of `ProcessedMail`.
- [ ] **Step 3: Implement the UID diff**
Replace `src/paperless_mail/mail.py:683-693`:
```python
try:
messages = M.fetch(
criteria=criterias,
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
with:
```python
try:
all_uids = set(
M.uids(criteria=criterias, charset=rule.account.character_set),
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while searching folder {rule.folder}",
) from err
processed_uids_qs = ProcessedMail.objects.filter(
rule=rule,
folder=rule.folder,
uid__in=all_uids,
)
if self._current_uid_validity is not None:
processed_uids_qs = processed_uids_qs.filter(
Q(uid_validity=self._current_uid_validity)
| Q(uid_validity__isnull=True),
)
processed_uids = set(processed_uids_qs.values_list("uid", flat=True))
new_uids = all_uids - processed_uids
if not new_uids:
self.log.debug(
f"Rule {rule}: No new mail matching criteria {criterias}",
)
return 0
try:
messages = M.fetch(
criteria=AND(uid=list(new_uids)),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
Note the `uid_validity` handling here deliberately mirrors the existing per-message check at `mail.py:722-726` exactly (see Global Constraints) — the extra `Q(...)` filter is applied only when `self._current_uid_validity is not None`.
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_skips_body_fetch_for_already_processed_mail -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite (regression check, especially the uidvalidity tests)**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS, including:
- `test_handle_mail_account_skip_duplicate_uids_from_fetch`
- `test_handle_mail_account_skips_mail_already_processed_in_same_uidvalidity`
- `test_handle_mail_account_processes_mail_after_uidvalidity_change`
- `test_handle_mail_account_skips_mail_processed_before_uidvalidity_tracking`
- `test_handle_mail_account_processes_mail_when_uidvalidity_unavailable`
- `test_handle_mail_account_skips_mail_when_uidvalidity_unavailable_but_prior_record_exists`
- `test_handle_mail_account_overlapping_rules_only_first_consumes`
If any of these fail, the `uid_validity` branching in Step 3 does not match `mail.py:722-726` closely enough — re-check against the Global Constraints note before changing test expectations.
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "fix: diff UIDs against ProcessedMail before fetching message bodies"
```
---
### Task 4: Batch the body fetch for large backlogs
**Files:**
- Modify: `src/paperless_mail/mail.py:74-76` (module-level constant)
- Modify: `src/paperless_mail/mail.py` (the `M.fetch(...)` block added in Task 3)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Produces: module-level constant `paperless_mail.mail.MAIL_FETCH_BATCH_SIZE: int = 500`.
- Consumes: `itertools` (already imported at `mail.py:2`), `sorted_new_uids` derived from Task 3's `new_uids` set.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py`. Add `from paperless_mail.mail import MAIL_FETCH_BATCH_SIZE` to the imports at the top of the file (alongside the other `from paperless_mail.mail import ...` lines at `test_mail.py:35-38`).
```python
@mock.patch("paperless_mail.mail.MAIL_FETCH_BATCH_SIZE", 5)
def test_handle_mail_account_batches_body_fetch_for_large_backlog(self) -> None:
"""
GIVEN:
- More new/unprocessed mail than MAIL_FETCH_BATCH_SIZE
WHEN:
- The mail account is processed
THEN:
- The body fetch is issued in multiple batches
- Every message is still processed (none dropped at a batch boundary)
"""
account = MailAccount.objects.create(
name="test",
imap_server="",
username="admin",
password="secret",
)
rule = MailRule.objects.create(
name="testrule",
account=account,
action=MailRule.MailAction.MARK_READ,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
message_count = 12 # more than the patched batch size of 5
self.mailMocker.bogus_mailbox.messages = [
self.mailMocker.messageBuilder.create_message(
subject=f"No attachment {i}",
attachments=[],
)
for i in range(message_count)
]
self.mailMocker.bogus_mailbox.updateClient()
with mock.patch.object(
self.mailMocker.bogus_mailbox,
"fetch",
wraps=self.mailMocker.bogus_mailbox.fetch,
) as fetch_spy:
self.mail_account_handler.handle_mail_account(account)
# ceil(12 / 5) == 3 batches
self.assertEqual(fetch_spy.call_count, 3)
self.assertEqual(
ProcessedMail.objects.filter(rule=rule).count(),
message_count,
)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_batches_body_fetch_for_large_backlog -v`
Expected: FAIL — either an `ImportError` for `MAIL_FETCH_BATCH_SIZE` (doesn't exist yet) or, once that import is stubbed out, `fetch_spy.call_count == 1` instead of `3` (Task 3's implementation fetches all new UIDs in one call).
- [ ] **Step 3: Add the constant and batch the fetch**
Insert after `src/paperless_mail/mail.py:74` (right after the `APPLE_MAIL_TAG_COLORS` dict closes, before `class MailError`):
```python
MAIL_FETCH_BATCH_SIZE = 500
```
Replace the `M.fetch(...)` block added in Task 3 (Task 3 Step 3's final `try/except`):
```python
try:
messages = M.fetch(
criteria=AND(uid=list(new_uids)),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
with:
```python
sorted_new_uids = sorted(new_uids, key=int)
message_batches = []
for batch_start in range(0, len(sorted_new_uids), MAIL_FETCH_BATCH_SIZE):
batch = sorted_new_uids[batch_start : batch_start + MAIL_FETCH_BATCH_SIZE]
try:
message_batches.append(
M.fetch(
criteria=AND(uid=batch),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
),
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
messages = itertools.chain(*message_batches)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_batches_body_fetch_for_large_backlog -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "perf: batch body fetches when many new UIDs are pending"
```
---
### Task 5: Full regression pass and lint
**Files:** none (verification only, plus any fixups this step surfaces)
- [ ] **Step 1: Run the full backend test suite with coverage/parallelism as configured**
Run: `uv run pytest src/paperless_mail/`
Expected: all `paperless_mail` tests PASS
- [ ] **Step 2: Run the full project test suite**
Run: `uv run pytest`
Expected: all tests PASS (no regressions outside `paperless_mail`)
- [ ] **Step 3: Lint and format**
Run: `uv run ruff check src/paperless_mail/` and `uv run ruff format --check src/paperless_mail/`
Expected: no errors. If `ruff format --check` reports files needing formatting, run `uv run ruff format src/paperless_mail/` and re-check.
- [ ] **Step 4: Type check against the frozen baselines**
Run: `uv run mypy src/paperless_mail/mail.py` (or the project's standard mypy invocation) and confirm no new violations beyond `.mypy-baseline.txt`.
Expected: no new errors introduced by this change.
- [ ] **Step 5: Commit any fixups**
If Steps 1-4 required changes:
```bash
git add -u
git commit -m "chore: fix lint/type fallout from mail fetch fix"
```
If no changes were needed, skip this step — nothing to commit.
@@ -0,0 +1,216 @@
# Mail fetch fix: avoid re-downloading already-handled IMAP mail
## Problem
`paperless_mail`'s `_handle_mail_rule` (`src/paperless_mail/mail.py:641`) fetches
the full RFC822 body of every message an IMAP `SEARCH` matches (via
`M.fetch(criteria=criterias, ..., bulk=True)`), then does de-duplication in
Python afterward. Two things compound into a real bug (reported upstream in
[paperless-ngx#13175](https://github.com/paperless-ngx/paperless-ngx/issues/13175)):
1. **Full-body fetch happens before dedup.** The de-dup check against
`ProcessedMail` (`mail.py:717-731`) happens per-message, after the body has
already been downloaded for every matching message. Steady-state cost is
_O(everything the search matches)_, not _O(new since last run)_.
2. **Attachment-less mail under an attachments-only rule is never recorded.**
`_handle_message` (`mail.py:749`) returns early, before any attachment
processing, when a message has no attachments and
`rule.consumption_scope == ATTACHMENTS_ONLY`:
```python
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements # 0, no ProcessedMail row, no mail action applied
```
No `ProcessedMail` row is written and no mail action (mark-read/flag/tag)
is applied, since both only happen via `queue_consumption_tasks`, which is
only reached from the attachment/eml processing paths. The message keeps
matching the search (e.g. `UNSEEN`) forever.
Combined: an inbox where most mail has no attachments gets fully re-downloaded
on every scheduled run (default every 10 minutes), indefinitely, for large
mailboxes -- confirmed against the current codebase, not user
misconfiguration.
## Goals
- Stop re-downloading full message bodies for mail that has already been
handled (processed into a document, or determined to produce nothing).
- Record `ProcessedMail` for the no-attachment / attachments-only early-return
case, so it participates in dedup like every other terminal outcome.
- No new required settings or DB migrations for existing installs.
- Graceful behavior on first run against a large existing mailbox (no single
giant IMAP command, no giant in-memory batch).
- Preserve existing action/consumption semantics exactly for every path that
currently produces a document or applies a mail action.
## Non-goals
- Changing what mail actions (mark-read/flag/tag/move/delete) get applied, or
when.
- Applying a mail action to the no-attachment early-return case (explicitly
out of scope -- see Decisions).
- Making batch size user-configurable.
- Touching EML_ONLY / EVERYTHING consumption scopes' semantics (they already
don't hit the early-return branch described above).
## Decisions
These were settled during brainstorming and are load-bearing for the design
below:
- **Fix both halves together.** Recording `ProcessedMail` alone does not fix
the bandwidth problem: without a mail action applied, an attachment-less
message stays unseen and keeps matching the search criteria, so its body
gets re-downloaded every run regardless of whether it gets reprocessed.
Fixing only the bandwidth side without recording `ProcessedMail` would mean
the UID-diff step never has anything to exclude for these messages. They
must ship together.
- **No mailbox action for the no-attachment case.** Only a `ProcessedMail`
row is written; the message is not marked read/flagged/moved/deleted. This
matches the existing precedent for other no-consumption outcomes
(`mail.py:960-977`, e.g. an attachment present but filtered out or an
unsupported mime type) and avoids changing the user's mailbox state for
mail paperless previously never touched.
- **UID-diff-before-fetch, not a BODYSTRUCTURE probe or SEARCH-side
exclusion.** Considered and rejected:
- BODYSTRUCTURE probing (fetch structure only, decide whether to fetch full
body) only helps the attachments-only scope, not the general
already-processed case, and `imap_tools` doesn't cleanly expose a
structure-only fetch.
- Excluding already-processed UIDs directly in the IMAP `SEARCH` criteria
(`NOT UID (...)`) was rejected because for tens of thousands of
already-processed UIDs the excluded-UID list itself blows up the command
size -- worse than the two-step approach.
- **Batch size is a hardcoded constant**, not a new setting, per the
"no new required settings" goal.
## Design
### `_handle_mail_rule` (`mail.py:641`)
Replace the single `M.fetch(criteria=criterias, ...)` call with:
1. `self._current_uid_validity` computed as today (unchanged, already first).
2. `criterias = make_criterias(...)` (unchanged).
3. `all_uids = set(M.uids(criteria=criterias, charset=rule.account.character_set))`
-- a `UID SEARCH`, no bodies.
4. Query already-processed UIDs in one DB round trip, reusing the same
uid_validity matching already used per-message:
```python
processed_uids = set(
ProcessedMail.objects.filter(
rule=rule, folder=rule.folder, uid__in=all_uids,
).filter(
Q(uid_validity=self._current_uid_validity) | Q(uid_validity__isnull=True),
).values_list("uid", flat=True)
)
```
5. `new_uids = all_uids - processed_uids`. If empty: log at debug level and
`return 0` -- no body fetch at all. This is the steady-state case.
6. Otherwise, iterate `new_uids` in fixed-size batches of
`MAIL_FETCH_BATCH_SIZE = 500` (module-level constant), calling
`M.fetch(criteria=AND(uid=batch), mark_seen=False, charset=rule.account.character_set, bulk=True)`
per batch. Chain the resulting message iterators into the existing
per-message loop (`mail.py:699+`) unchanged.
7. The existing per-message `already_processed` DB check
(`mail.py:717-731`) stays in place unchanged, as a safety net against
races (e.g. concurrent rule runs against the same folder in this pass) and
to keep behavior identical if steps 3/4 ever disagree with it.
### New helper: `_record_processed_without_consumption`
Extract the existing dedup-and-create block at `mail.py:960-977` into a
method on the same class:
```python
def _record_processed_without_consumption(self, message, rule) -> None:
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date) if is_naive(message.date) else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
```
Call sites:
- `_process_attachments`'s existing no-consumables branch (replaces the
inline block, same behavior).
- `_handle_message`'s early return (`mail.py:756-760`), newly, for the
no-attachments-under-attachments-only-scope case.
### Data flow (steady state, nothing new)
search criteria (unchanged) -> `UID SEARCH` for matching UIDs (cheap) ->
subtract UIDs already in `ProcessedMail` (cheap, DB-only) -> zero new UIDs ->
no fetch, no body download.
### Data flow (new mail present)
... same as above -> some UIDs remain -> fetch bodies only for those,
batched -> existing per-message processing, action application, and
`ProcessedMail` recording, unchanged except the no-attachment early return
now also calls `_record_processed_without_consumption`.
## Error handling
- `M.uids(...)` raising is wrapped in the same `try/except` that currently
wraps `M.fetch(...)` (`mail.py:683-693`), surfacing as `MailError`
identically to today's search-failure behavior.
- A batch's `M.fetch(...)` raising mid-loop is caught by the same
`try/except`, applied per-batch. One failing batch fails the whole rule
run for this pass -- matching today's all-or-nothing semantics (currently
a single failed fetch already fails the whole rule).
- Empty `new_uids` short-circuits before any fetch, so it can't hit fetch
error paths at all for the common case.
- `_record_processed_without_consumption` reuses the existing
`ProcessedMail.objects.create` call already in production use; no new
failure mode introduced.
## Testing
In `src/paperless_mail/tests/test_mail.py`:
- Extend the `BogusMailBox` test double with a `uids()` method that mirrors
its existing `fetch()` criteria matching but returns only UIDs.
- New test: an attachment-less message under an attachments-only mark-read
rule is recorded as `PROCESSED_WO_CONSUMPTION` on the first run. On a
second run, with the same message still unseen server-side, assert no
body fetch happens (only `M.uids()` is called) -- i.e. steady state costs
one search and zero body downloads.
- New test: a folder with more new UIDs than `MAIL_FETCH_BATCH_SIZE` results
in multiple batched `fetch` calls, and all messages are still processed
(no messages dropped at a batch boundary).
- Update existing `test_handle_empty_message` to assert a
`PROCESSED_WO_CONSUMPTION` row now exists for the no-attachment case.
- Regression: existing tests for `EML_ONLY`/`EVERYTHING` scopes and the
multi-rule-same-folder dedup (`consumed_messages` set) should pass
unchanged, since neither the per-message `already_processed` check nor the
`consumed_messages` set logic is touched by this design.
## Open questions / risks
- `imap_tools`'s `AND(uid=batch)` criteria builder needs to be confirmed to
produce a valid `UID FETCH <list>` command at the batch sizes used here;
covered by the batching test above.
- If a mail server doesn't support `UID SEARCH` the same way it supports the
existing `fetch`'s implicit search, `M.uids()` could behave differently
from today's `M.fetch()` on some edge-case server. Existing project test
coverage (via `BogusMailBox`) won't catch server-specific quirks; this is
the same class of risk as any IMAP-behavior assumption already baked into
this module.
+5 -5
View File
@@ -16,7 +16,7 @@ classifiers = [
dependencies = [
"azure-ai-documentintelligence>=1.0.2",
"babel>=2.17",
"bleach~=6.3.0",
"bleach~=6.4.0",
"celery[redis]~=5.6.2",
"channels~=4.2",
"channels-redis~=4.2",
@@ -47,7 +47,7 @@ dependencies = [
"gotenberg-client~=0.14.0",
"httpx-oauth~=0.16",
"ijson>=3.2",
"imap-tools~=1.13.0",
"imap-tools~=1.14.0",
"jinja2~=3.1.5",
"langdetect~=1.0.9",
"llama-index-core>=0.14.22",
@@ -56,7 +56,7 @@ dependencies = [
"llama-index-embeddings-openai-like>=0.2.2",
"llama-index-llms-ollama>=0.9.1",
"llama-index-llms-openai-like>=0.7.1",
"nltk~=3.9.1",
"nltk~=3.10.0",
"ocrmypdf~=17.4.2",
"openai>=2.32",
"pathvalidate~=3.3.1",
@@ -101,11 +101,11 @@ dev = [
{ include-group = "testing" },
]
docs = [
"zensical>=0.0.43",
"zensical>=0.0.47",
]
lint = [
"prek~=0.3.10",
"ruff~=0.15.15",
"ruff~=0.15.20",
]
testing = [
"daphne",
+10
View File
@@ -79,6 +79,16 @@
"glob": "{pdf.worker.min.mjs,pdf.min.mjs}",
"input": "node_modules/pdfjs-dist/legacy/build/",
"output": "/assets/js/"
},
{
"glob": "**/*",
"input": "node_modules/pdfjs-dist/wasm/",
"output": "/assets/wasm/"
},
{
"glob": "**/*",
"input": "node_modules/pdfjs-dist/iccs/",
"output": "/assets/iccs/"
}
],
"styles": [
+291 -258
View File
File diff suppressed because it is too large Load Diff
-1
View File
@@ -18,7 +18,6 @@
"@angular/forms": "~22.0.5",
"@angular/localize": "~22.0.5",
"@angular/platform-browser": "~22.0.5",
"@angular/platform-browser-dynamic": "~22.0.5",
"@angular/router": "~22.0.5",
"@ng-bootstrap/ng-bootstrap": "^21.0.0",
"@ng-select/ng-select": "^23.2.0",
+1295 -1844
View File
File diff suppressed because it is too large Load Diff
-38
View File
@@ -13,8 +13,6 @@ import { DocumentDetailComponent } from './components/document-detail/document-d
import { DocumentListComponent } from './components/document-list/document-list.component'
import { DocumentAttributesComponent } from './components/manage/document-attributes/document-attributes.component'
import { MailComponent } from './components/manage/mail/mail.component'
import { OcrTemplateEditorComponent } from './components/manage/ocr-templates/ocr-template-editor/ocr-template-editor.component'
import { OcrTemplatesComponent } from './components/manage/ocr-templates/ocr-templates.component'
import { SavedViewsComponent } from './components/manage/saved-views/saved-views.component'
import { WorkflowsComponent } from './components/manage/workflows/workflows.component'
import { NotFoundComponent } from './components/not-found/not-found.component'
@@ -276,42 +274,6 @@ export const routes: Routes = [
componentName: 'WorkflowsComponent',
},
},
{
path: 'ocr-templates',
component: OcrTemplatesComponent,
canActivate: [PermissionsGuard],
data: {
requiredPermission: {
action: PermissionAction.View,
type: PermissionType.OcrTemplate,
},
componentName: 'OcrTemplatesComponent',
},
},
{
path: 'ocr-templates/new',
component: OcrTemplateEditorComponent,
canActivate: [PermissionsGuard],
data: {
requiredPermission: {
action: PermissionAction.Add,
type: PermissionType.OcrTemplate,
},
componentName: 'OcrTemplateEditorComponent',
},
},
{
path: 'ocr-templates/:id',
component: OcrTemplateEditorComponent,
canActivate: [PermissionsGuard],
data: {
requiredPermission: {
action: PermissionAction.Change,
type: PermissionType.OcrTemplate,
},
componentName: 'OcrTemplateEditorComponent',
},
},
{
path: 'mail',
component: MailComponent,
@@ -5,8 +5,9 @@
i18n-info>
<div class="input-group input-group-sm align-items-center">
<div class="input-group input-group-sm me-3">
<span class="input-group-text text-muted" i18n>Show</span>
<label class="input-group-text text-muted" for="logLineLimit" i18n>Show</label>
<input
id="logLineLimit"
class="form-control"
type="number"
min="100"
@@ -17,7 +18,7 @@
<span class="input-group-text text-muted" i18n>lines</span>
</div>
<div class="form-check form-switch mt-1">
<input class="form-check-input" type="checkbox" role="switch" [ngModel]="autoRefreshEnabled()" (ngModelChange)="autoRefreshEnabled.set($event)">
<input id="autoRefreshSwitch" class="form-check-input" type="checkbox" role="switch" [ngModel]="autoRefreshEnabled()" (ngModelChange)="autoRefreshEnabled.set($event)">
<label class="form-check-label" for="autoRefreshSwitch" i18n>Auto refresh</label>
</div>
</div>
@@ -100,9 +100,7 @@ export class LogsComponent
parsed.some((log, idx) => {
const current = this.logs()[idx]
return (
!current ||
current.message !== log.message ||
current.level !== log.level
current?.message !== log.message || current?.level !== log.level
)
})
if (hasChanges) {
@@ -37,7 +37,7 @@
<form [formGroup]="settingsForm" (ngSubmit)="saveSettings()">
<ul ngbNav #nav="ngbNav" (navChange)="onNavChange($event)" [(activeId)]="activeNavID" class="nav-tabs">
<ul ngbNav #nav="ngbNav" (navChange)="onNavChange($event)" [activeId]="activeNavID()" (activeIdChange)="activeNavID.set($event)" class="nav-tabs">
<li [ngbNavItem]="SettingsNavIDs.General">
<a ngbNavLink i18n>General</a>
<ng-template ngbNavContent>
@@ -150,10 +150,10 @@
<div class="row mb-3">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Full search links to</span>
<label for="searchLink" i18n>Full search links to</label>
</div>
<div class="col mb-3">
<select class="form-select" formControlName="searchLink">
<select id="searchLink" class="form-select" formControlName="searchLink">
<option [ngValue]="GlobalSearchType.TITLE_CONTENT" i18n>Title and content search</option>
<option [ngValue]="GlobalSearchType.ADVANCED" i18n>Advanced search</option>
</select>
@@ -199,10 +199,10 @@
<h5 i18n>Documents</h5>
<div class="row mb-3">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Items per page</span>
<label for="documentListItemPerPage" i18n>Items per page</label>
</div>
<div class="col">
<select class="form-select" formControlName="documentListItemPerPage">
<select id="documentListItemPerPage" class="form-select" formControlName="documentListItemPerPage">
<option [ngValue]="10">10</option>
<option [ngValue]="25">25</option>
<option [ngValue]="50">50</option>
@@ -274,10 +274,10 @@
<h5 class="mt-3" i18n>PDF Editor</h5>
<div class="row">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Default editing mode</span>
<label for="pdfEditorDefaultEditMode" i18n>Default editing mode</label>
</div>
<div class="col">
<select class="form-select" formControlName="pdfEditorDefaultEditMode">
<select id="pdfEditorDefaultEditMode" class="form-select" formControlName="pdfEditorDefaultEditMode">
<option [ngValue]="PdfEditorEditMode.Create" i18n>Create new document(s)</option>
<option [ngValue]="PdfEditorEditMode.Update" i18n>Add document version</option>
</select>
@@ -242,7 +242,7 @@ describe('SettingsComponent', () => {
activatedRoute.snapshot.fragment = '#notifications'
const scrollSpy = jest.spyOn(viewportScroller, 'scrollToAnchor')
component.ngOnInit()
expect(component.activeNavID).toEqual(4) // Notifications
expect(component.activeNavID()).toEqual(4) // Notifications
component.ngAfterViewInit()
expect(scrollSpy).toHaveBeenCalledWith('#notifications')
})
@@ -399,17 +399,17 @@ describe('SettingsComponent', () => {
completeSetup()
const field = 'storage_path'
expect(
component.settingsForm.get('documentDetailsHiddenFields').value.length
).toEqual(0)
component.settingsForm.get('documentDetailsHiddenFields').value
).toHaveLength(0)
component.toggleDocumentDetailField(field, false)
expect(
component.settingsForm.get('documentDetailsHiddenFields').value.length
).toEqual(1)
component.settingsForm.get('documentDetailsHiddenFields').value
).toHaveLength(1)
expect(component.isDocumentDetailFieldShown(field)).toBeFalsy()
component.toggleDocumentDetailField(field, true)
expect(
component.settingsForm.get('documentDetailsHiddenFields').value.length
).toEqual(0)
component.settingsForm.get('documentDetailsHiddenFields').value
).toHaveLength(0)
expect(component.isDocumentDetailFieldShown(field)).toBeTruthy()
})
})
@@ -142,7 +142,7 @@ export class SettingsComponent
private systemStatusService = inject(SystemStatusService)
private savedViewsService = inject(SavedViewService)
activeNavID: number
readonly activeNavID = signal<number>(undefined)
settingsForm = new FormGroup({
bulkEditConfirmationDialogs: new FormControl(null),
@@ -283,7 +283,7 @@ export class SettingsComponent
(navID) => navID.toLowerCase() == section
)
if (navIDKey) {
this.activeNavID = SettingsNavIDs[navIDKey]
this.activeNavID.set(SettingsNavIDs[navIDKey])
}
}
})
@@ -386,7 +386,7 @@ export class SettingsComponent
.navigate(['settings', foundNavIDkey.toLowerCase()])
.then((navigated) => {
if (!navigated && this.isDirty) {
this.activeNavID = navChangeEvent.activeId
this.activeNavID.set(navChangeEvent.activeId)
} else if (navigated && this.isDirty) {
this.initialize()
}
@@ -15,7 +15,7 @@
<i-bs name="check2-all" class="me-1"></i-bs><ng-container i18n>Dismiss all</ng-container>
</button>
<div class="form-check form-switch mb-0 ms-2">
<input class="form-check-input" type="checkbox" role="switch" [(ngModel)]="autoRefreshEnabled">
<input id="autoRefreshSwitch" class="form-check-input" type="checkbox" role="switch" [(ngModel)]="autoRefreshEnabled">
<label class="form-check-label" for="autoRefreshSwitch" i18n>Auto refresh</label>
</div>
</div>
@@ -93,7 +93,8 @@
<i-bs width="1em" height="1em" name="x"></i-bs>
</button>
}
<input #filterInput class="form-control form-control-sm" type="text"
<label class="visually-hidden" for="taskSearch" i18n>Search tasks</label>
<input #filterInput id="taskSearch" class="form-control form-control-sm" type="text"
(keyup)="filterInputKeyup($event)"
[(ngModel)]="filterText">
</div>
@@ -211,7 +212,7 @@
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
</button>
</td>
<td scope="row" class="actions-col">
<td class="actions-col">
<div class="btn-group" role="group">
<button class="btn btn-sm btn-outline-secondary" (click)="dismissTask(task); $event.stopPropagation();" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.PaperlessTask }">
<i-bs name="check" class="me-1"></i-bs><ng-container i18n>Dismiss</ng-container>
@@ -150,7 +150,8 @@
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" name="file-text"></i-bs><span>{{d.title | documentTitle}}</span>
<span class="close flex-column justify-content-center" (click)="closeDocument(d); $event.preventDefault()">
<span class="close flex-column justify-content-center"
(click)="$event.preventDefault(); $event.stopPropagation(); closeDocument(d)">
<i-bs name="x"></i-bs>
</span>
</a>
@@ -243,14 +244,6 @@
<i-bs class="me-2" name="boxes"></i-bs><span><ng-container i18n>Workflows</ng-container></span>
</a>
</li>
<li class="nav-item app-link"
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.OcrTemplate }">
<a class="nav-link" routerLink="ocr-templates" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="OCR Templates" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="file-earmark-break"></i-bs><span><ng-container i18n>OCR Templates</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
tourAnchor="tour.mail">
<a class="nav-link" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
@@ -6,7 +6,7 @@
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="chatDropdown">
<div class="chat-container bg-light p-2">
<div class="chat-messages font-monospace small">
@for (message of messages; track message) {
@for (message of messages(); track message) {
<div class="message d-flex flex-row small" [class.justify-content-end]="message.role === 'user'">
<div class="p-2 m-2" [class.bg-body]="message.role === 'user'">
<span>
@@ -29,16 +29,19 @@
</div>
<form class="chat-input">
<label class="visually-hidden" for="chatInput" i18n>Message</label>
<div class="input-group">
<input
#chatInput
id="chatInput"
class="form-control form-control-sm" name="chatInput" type="text"
[placeholder]="placeholder"
[disabled]="loading"
[(ngModel)]="input"
[disabled]="loading()"
[ngModel]="input()"
(ngModelChange)="input.set($event)"
(keydown)="searchInputKeyDown($event)"
/>
<button class="btn btn-sm btn-secondary" type="button" (click)="sendMessage()" [disabled]="loading">Send</button>
<button class="btn btn-sm btn-secondary" type="button" (click)="sendMessage()" [disabled]="loading()">Send</button>
</div>
</form>
</div>
@@ -56,44 +56,53 @@ describe('ChatComponent', () => {
it('should update documentId on initialization', () => {
jest.spyOn(router, 'url', 'get').mockReturnValue('/documents/123')
component.ngOnInit()
expect(component.documentId).toBe(123)
expect(component.documentId()).toBe(123)
})
it('should update documentId on navigation', () => {
component.ngOnInit()
routerEvents$.next(new NavigationEnd(1, '/documents/456', '/documents/456'))
expect(component.documentId).toBe(456)
expect(component.documentId()).toBe(456)
})
it('should return correct placeholder based on documentId', () => {
component.documentId = 123
component.documentId.set(123)
expect(component.placeholder).toBe('Ask a question about this document...')
component.documentId = undefined
component.documentId.set(undefined)
expect(component.placeholder).toBe('Ask a question about a document...')
})
it('should send a message and handle streaming response', () => {
component.input = 'Hello'
it('should send a message and render the streaming response', async () => {
component.input.set('Hello')
component.sendMessage()
expect(component.messages.length).toBe(2)
expect(component.messages[0].content).toBe('Hello')
expect(component.loading).toBe(true)
expect(component.messages()).toHaveLength(2)
expect(component.messages()[0].content).toBe('Hello')
expect(component.loading()).toBe(true)
mockStream$.next('Hi')
expect(component.messages[1].content).toBe('H')
expect(component.messages()[1].content).toBe('H')
mockStream$.next('Hi there')
// advance time to process the typewriter effect
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe('Hi there')
await jest.runAllTimersAsync()
await fixture.whenStable()
expect(component.messages()[1].content).toBe('Hi there')
expect(
fixture.nativeElement.querySelector('.chat-messages').textContent
).toContain('Hi there')
mockStream$.complete()
expect(component.loading).toBe(false)
expect(component.messages[1].isStreaming).toBe(false)
await jest.runAllTimersAsync()
await fixture.whenStable()
expect(component.loading()).toBe(false)
expect(component.messages()[1].isStreaming).toBe(false)
expect(fixture.nativeElement.querySelector('#chatInput').disabled).toBe(
false
)
})
it('should parse references from the metadata trailer without showing it', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.next(
@@ -101,14 +110,14 @@ describe('ChatComponent', () => {
)
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe('Hi there')
expect(component.messages[1].references).toEqual([
expect(component.messages()[1].content).toBe('Hi there')
expect(component.messages()[1].references).toEqual([
{ id: 42, title: 'Bread Recipe' },
])
})
it('should render document reference links under assistant messages', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.next(
@@ -123,12 +132,12 @@ describe('ChatComponent', () => {
})
it('should remove delimiter fragments that were already streamed', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.next(`Hi there${CHAT_METADATA_DELIMITER.slice(0, 8)}`)
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe(
expect(component.messages()[1].content).toBe(
`Hi there${CHAT_METADATA_DELIMITER.slice(0, 8)}`
)
@@ -137,28 +146,28 @@ describe('ChatComponent', () => {
)
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe('Hi there')
expect(component.messages[1].references).toEqual([
expect(component.messages()[1].content).toBe('Hi there')
expect(component.messages()[1].references).toEqual([
{ id: 42, title: 'Bread Recipe' },
])
})
it('should handle errors during streaming', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.error('Error')
expect(component.messages[1].content).toContain(
expect(component.messages()[1].content).toContain(
'⚠️ Error receiving response.'
)
expect(component.loading).toBe(false)
expect(component.loading()).toBe(false)
})
it('should enqueue typewriter chunks correctly', () => {
const message = { content: '', role: 'assistant', isStreaming: true }
component.enqueueTypewriter(null, message as any) // coverage for null
component.enqueueTypewriter('Hello', message as any)
expect(component['typewriterBuffer'].length).toBe(4)
expect(component['typewriterBuffer']).toHaveLength(4)
})
it('should scroll to bottom after sending a message', () => {
@@ -166,7 +175,7 @@ describe('ChatComponent', () => {
ChatComponent.prototype as any,
'scrollToBottom'
)
component.input = 'Test'
component.input.set('Test')
component.sendMessage()
expect(scrollSpy).toHaveBeenCalled()
})
@@ -1,4 +1,11 @@
import { Component, ElementRef, inject, OnInit, ViewChild } from '@angular/core'
import {
Component,
ElementRef,
inject,
OnInit,
signal,
ViewChild,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NavigationEnd, Router, RouterModule } from '@angular/router'
import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
@@ -23,10 +30,10 @@ import {
styleUrl: './chat.component.scss',
})
export class ChatComponent implements OnInit {
public messages: ChatMessage[] = []
public loading = false
public input: string = ''
public documentId!: number
readonly messages = signal<ChatMessage[]>([])
readonly loading = signal(false)
readonly input = signal('')
readonly documentId = signal<number>(undefined)
private chatService: ChatService = inject(ChatService)
private router: Router = inject(Router)
@@ -38,7 +45,7 @@ export class ChatComponent implements OnInit {
private typewriterActive = false
public get placeholder(): string {
return this.documentId
return this.documentId()
? $localize`Ask a question about this document...`
: $localize`Ask a question about a document...`
}
@@ -57,14 +64,14 @@ export class ChatComponent implements OnInit {
private updateDocumentId(url: string): void {
const docIdRe = url.match(/^\/documents\/(\d+)/)
this.documentId = docIdRe ? +docIdRe[1] : undefined
this.documentId.set(docIdRe ? +docIdRe[1] : undefined)
}
sendMessage(): void {
if (!this.input.trim()) return
if (!this.input().trim()) return
const userMessage: ChatMessage = { role: 'user', content: this.input }
this.messages.push(userMessage)
const userMessage: ChatMessage = { role: 'user', content: this.input() }
this.messages.update((messages) => [...messages, userMessage])
this.scrollToBottom()
const assistantMessage: ChatMessage = {
@@ -72,12 +79,12 @@ export class ChatComponent implements OnInit {
content: '',
isStreaming: true,
}
this.messages.push(assistantMessage)
this.loading = true
this.messages.update((messages) => [...messages, assistantMessage])
this.loading.set(true)
let lastVisibleContent = ''
this.chatService.streamChat(this.documentId, this.input).subscribe({
this.chatService.streamChat(this.documentId(), this.input()).subscribe({
next: (chunk) => {
const nextResponse = parseChatResponse(chunk)
@@ -93,26 +100,30 @@ export class ChatComponent implements OnInit {
}
assistantMessage.references = nextResponse.references
this.notifyMessagesChanged()
},
error: () => {
assistantMessage.content += '\n\n⚠️ Error receiving response.'
assistantMessage.isStreaming = false
this.loading = false
this.notifyMessagesChanged()
this.loading.set(false)
},
complete: () => {
assistantMessage.isStreaming = false
this.loading = false
this.notifyMessagesChanged()
this.loading.set(false)
this.scrollToBottom()
},
})
this.input = ''
this.input.set('')
}
private resetTypewriter(message: ChatMessage, content: string): void {
this.typewriterBuffer = []
this.typewriterActive = false
message.content = content
this.notifyMessagesChanged()
this.scrollToBottom()
}
@@ -135,11 +146,16 @@ export class ChatComponent implements OnInit {
const nextChar = this.typewriterBuffer.shift()
message.content += nextChar
this.notifyMessagesChanged()
this.scrollToBottom()
setTimeout(() => this.playTypewriter(message), 10) // 10ms per character
}
private notifyMessagesChanged(): void {
this.messages.update((messages) => [...messages])
}
private scrollToBottom(): void {
setTimeout(() => {
this.scrollAnchor?.nativeElement?.scrollIntoView({ behavior: 'smooth' })
@@ -1,5 +1,12 @@
import { CurrencyPipe, getLocaleCurrencyCode, SlicePipe } from '@angular/common'
import { Component, inject, Input, LOCALE_ID, OnInit } from '@angular/core'
import {
ChangeDetectorRef,
Component,
inject,
Input,
LOCALE_ID,
OnInit,
} from '@angular/core'
import { NgbTooltipModule } from '@ng-bootstrap/ng-bootstrap'
import { takeUntil } from 'rxjs'
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
@@ -22,6 +29,7 @@ export class CustomFieldDisplayComponent
{
private customFieldService = inject(CustomFieldsService)
private documentService = inject(DocumentService)
private changeDetector = inject(ChangeDetectorRef)
CustomFieldDataType = CustomFieldDataType
@@ -74,6 +82,7 @@ export class CustomFieldDisplayComponent
this.customFieldService.listAll().subscribe((r) => {
this.customFields = r.results
this.init()
this.changeDetector.markForCheck()
})
}
@@ -111,6 +120,7 @@ export class CustomFieldDisplayComponent
this.docLinkDocuments = this.value
.map((id) => result.results.find((d) => d.id === id))
.filter((d) => d)
this.changeDetector.markForCheck()
})
}
@@ -95,8 +95,8 @@ describe('CustomFieldsDropdownComponent', () => {
it('should support update unused fields', () => {
component.existingFields = [{ field: fields[0].id } as any]
component['updateUnusedFields']()
expect(component['unusedFields'].length).toEqual(1)
expect(component['unusedFields'][0].name).toEqual('Field 2')
expect(component['unusedFields']().length).toEqual(1)
expect(component['unusedFields']()[0].name).toEqual('Field 2')
})
it('should support getting data type label', () => {
@@ -8,6 +8,7 @@ import {
ViewChild,
ViewChildren,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbDropdownModule, NgbModal } from '@ng-bootstrap/ng-bootstrap'
@@ -65,11 +66,11 @@ export class CustomFieldsDropdownComponent extends LoadingComponentWithPermissio
@ViewChildren('button') buttons: QueryList<ElementRef>
private customFields: CustomField[] = []
private unusedFields: CustomField[] = []
private readonly unusedFields = signal<CustomField[]>([])
private keyboardIndex: number
public get filteredFields(): CustomField[] {
return this.unusedFields.filter(
return this.unusedFields().filter(
(f) => !this.filterText || matchesSearchText(f.name, this.filterText)
)
}
@@ -99,8 +100,10 @@ export class CustomFieldsDropdownComponent extends LoadingComponentWithPermissio
}
private updateUnusedFields() {
this.unusedFields = this.customFields.filter(
(f) => !this.existingFields?.find((e) => e.field === f.id)
this.unusedFields.set(
this.customFields.filter(
(f) => !this.existingFields?.find((e) => e.field === f.id)
)
)
}
@@ -77,7 +77,7 @@
<div class="input-group input-group-sm">
<ng-select #fieldSelects
class="paperless-input-select"
[items]="customFields"
[items]="customFields()"
[(ngModel)]="atom.field"
[disabled]="disabled"
bindLabel="name"
@@ -79,7 +79,7 @@ describe('CustomFieldsQueryDropdownComponent', () => {
})
it('should initialize custom fields on creation', () => {
expect(component.customFields).toEqual(customFields)
expect(component.customFields()).toEqual(customFields)
})
it('should add an expression when opened if queries are empty', () => {
@@ -101,7 +101,7 @@ describe('CustomFieldsQueryDropdownComponent', () => {
data_type: CustomFieldDataType.String,
extra_data: {},
}
component.customFields = [field]
component.customFields.set([field])
const operators = component.getOperatorsForField(1)
expect(operators.length).toEqual(
[
@@ -138,7 +138,7 @@ describe('CustomFieldsQueryDropdownComponent', () => {
],
},
}
component.customFields = [field]
component.customFields.set([field])
const options = component.getSelectOptionsForField(1)
expect(options).toEqual([
{ label: 'Option 1', id: 'abc-123' },
@@ -6,6 +6,7 @@ import {
Input,
Output,
QueryList,
signal,
ViewChild,
ViewChildren,
} from '@angular/core'
@@ -278,7 +279,7 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
@Output()
selectionModelChange = new EventEmitter<CustomFieldQueriesModel>()
customFields: CustomField[] = []
readonly customFields = signal<CustomField[]>([])
public readonly today: string = new Date().toLocaleDateString('en-CA')
@@ -325,12 +326,12 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
.listAll()
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe((result) => {
this.customFields = result.results
this.customFields.set(result.results)
})
}
public getCustomFieldByID(id: number): CustomField {
return this.customFields.find((field) => field.id === id)
return this.customFields().find((field) => field.id === id)
}
public addAtom(expression: CustomFieldQueryExpression) {
@@ -353,7 +354,7 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
getOperatorsForField(
fieldID: number
): Array<{ value: string; label: string }> {
const field = this.customFields.find((field) => field.id === fieldID)
const field = this.customFields().find((field) => field.id === fieldID)
const groups: CustomFieldQueryOperatorGroups[] = field
? CUSTOM_FIELD_QUERY_OPERATOR_GROUPS_BY_TYPE[field.data_type]
: [CustomFieldQueryOperatorGroups.Basic]
@@ -369,7 +370,7 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
getSelectOptionsForField(
fieldID: number
): Array<{ label: string; id: string }> {
const field = this.customFields.find((field) => field.id === fieldID)
const field = this.customFields().find((field) => field.id === fieldID)
if (field) {
return field.extra_data['select_options']
}
@@ -22,11 +22,11 @@
<ng-template>
<div class="card mb-2">
<div class="card-body p-2">
@if (testLoading) {
@if (testLoading()) {
<ng-container [ngTemplateOutlet]="loadingTemplate"></ng-container>
} @else if (testResult) {
<code>{{testResult}}</code>
} @else if (testFailed) {
} @else if (testResult()) {
<code>{{testResult()}}</code>
} @else if (testFailed()) {
<div class="text-danger" i18n>Path test failed</div>
} @else {
<div class="text-muted small" i18n>No document selected</div>
@@ -42,7 +42,7 @@
[compareWith]="compareDocuments"
[trackByFn]="trackByFn"
[minTermLength]="2"
[loading]="loading"
[loading]="loading()"
[typeahead]="documentsInput$"
(change)="testPath($event)">
<ng-template #loadingTemplate ng-loadingspinner-tmp>
@@ -58,17 +58,17 @@ describe('StoragePathEditDialogComponent', () => {
fixture.detectChanges()
component.testPath({ id: 1 })
expect(testSpy).toHaveBeenCalledWith('test/{{title}}', 1)
expect(component.testResult).toBe('test/abc123')
expect(component.testFailed).toBeFalsy()
expect(component.testResult()).toBe('test/abc123')
expect(component.testFailed()).toBeFalsy()
// test failed
testSpy.mockReturnValueOnce(of(''))
component.testPath({ id: 1 })
expect(component.testResult).toBeNull()
expect(component.testFailed).toBeTruthy()
expect(component.testResult()).toBeNull()
expect(component.testFailed()).toBeTruthy()
component.testPath(null)
expect(component.testResult).toBeNull()
expect(component.testResult()).toBeNull()
})
it('should compare two documents by id', () => {
@@ -1,5 +1,5 @@
import { AsyncPipe, NgTemplateOutlet } from '@angular/common'
import { Component, OnDestroy, inject } from '@angular/core'
import { Component, OnDestroy, inject, signal } from '@angular/core'
import {
FormControl,
FormGroup,
@@ -65,9 +65,9 @@ export class StoragePathEditDialogComponent
public documentsInput$ = new Subject<string>()
public foundDocuments$: Observable<Document[]>
private testDocument: Document
public testResult: string
public testFailed: boolean = false
public testLoading = false
readonly testResult = signal<string>(undefined)
readonly testFailed = signal(false)
readonly testLoading = signal(false)
constructor() {
super()
@@ -99,22 +99,22 @@ export class StoragePathEditDialogComponent
public testPath(document: Document) {
if (!document) {
this.testResult = null
this.testResult.set(null)
return
}
this.testDocument = document
this.testLoading = true
this.testLoading.set(true)
;(this.service as StoragePathService)
.testPath(this.objectForm.get('path').value, document.id)
.subscribe((result) => {
if (result?.length) {
this.testResult = result
this.testFailed = false
this.testResult.set(result)
this.testFailed.set(false)
} else {
this.testResult = null
this.testFailed = true
this.testResult.set(null)
this.testFailed.set(true)
}
this.testLoading = false
this.testLoading.set(false)
})
}
@@ -12,7 +12,7 @@
<pngx-input-color i18n-title title="Color" formControlName="color" [error]="error?.color"></pngx-input-color>
<pngx-input-select i18n-title title="Parent" formControlName="parent" [items]="tags" [allowNull]="true" [error]="error?.parent"></pngx-input-select>
<pngx-input-select i18n-title title="Parent" formControlName="parent" [items]="tags()" [allowNull]="true" [error]="error?.parent"></pngx-input-select>
<pngx-input-check i18n-title title="Inbox tag" formControlName="is_inbox_tag" i18n-hint hint="Inbox tags are automatically assigned to all consumed documents."></pngx-input-check>
<pngx-input-select i18n-title title="Matching algorithm" [items]="getMatchingAlgorithms()" formControlName="matching_algorithm"></pngx-input-select>
@@ -1,4 +1,4 @@
import { Component, inject } from '@angular/core'
import { Component, inject, signal } from '@angular/core'
import {
FormControl,
FormGroup,
@@ -35,7 +35,7 @@ import { TextComponent } from '../../input/text/text.component'
],
})
export class TagEditDialogComponent extends EditDialogComponent<Tag> {
tags: Tag[]
readonly tags = signal<Tag[]>([])
constructor() {
super()
@@ -43,7 +43,7 @@ export class TagEditDialogComponent extends EditDialogComponent<Tag> {
this.userService = inject(UserService)
this.settingsService = inject(SettingsService)
this.service.listAll().subscribe((result) => {
this.tags = result.results
this.tags.set(result.results)
})
}
@@ -180,7 +180,7 @@
<div class="col">
<div class="trigger-filters mb-3">
<div class="d-flex align-items-center">
<label class="form-label mb-0" i18n>Advanced Filters</label>
<span class="form-label mb-0" i18n>Advanced Filters</span>
<button
type="button"
class="btn btn-sm btn-outline-primary ms-auto"
@@ -401,7 +401,7 @@ describe('WorkflowEditDialogComponent', () => {
component.addFilter(triggerGroup as FormGroup)
const filters = component.getFiltersFormArray(triggerGroup as FormGroup)
expect(filters.length).toBe(3)
expect(filters).toHaveLength(3)
filters.at(0).get('values').setValue([1])
filters.at(1).get('values').setValue([2, 3])
@@ -672,7 +672,7 @@ describe('WorkflowEditDialogComponent', () => {
} as any
const filters = component['buildFiltersFormArray'](trigger)
expect(filters.length).toBe(0)
expect(filters).toHaveLength(0)
component.filterDefinitions = originalDefinitions
})
@@ -739,7 +739,7 @@ describe('WorkflowEditDialogComponent', () => {
component.ngOnInit()
const triggerGroup = component.triggerFields.at(0) as FormGroup
const filters = component.getFiltersFormArray(triggerGroup)
expect(filters.length).toBe(13)
expect(filters).toHaveLength(13)
const customFieldFilter = filters.at(12) as FormGroup
expect(customFieldFilter.get('type').value).toBe(
TriggerFilterType.CustomFieldQuery
@@ -969,12 +969,12 @@ describe('WorkflowEditDialogComponent', () => {
component.addFilter(triggerGroup)
component.removeFilter(triggerGroup, 0)
expect(component.getFiltersFormArray(triggerGroup).length).toBe(0)
expect(component.getFiltersFormArray(triggerGroup)).toHaveLength(0)
component.addFilter(triggerGroup)
const filterArrayAfterAdd = component.getFiltersFormArray(triggerGroup)
filterArrayAfterAdd.at(0).get('type').setValue(TriggerFilterType.TagsAll)
expect(component.getFiltersFormArray(triggerGroup).length).toBe(1)
expect(component.getFiltersFormArray(triggerGroup)).toHaveLength(1)
})
it('should remove selected custom field from the form group', () => {
@@ -933,7 +933,7 @@ export class WorkflowEditDialogComponent
getFilterSelectItems(type: TriggerFilterType) {
const definition = this.getFilterDefinition(type)
if (!definition || definition.inputType !== 'select') {
if (definition?.inputType !== 'select') {
return []
}
@@ -58,8 +58,8 @@
</button>
}
@if ((selectionModel.items | filter: filterText:'name').length > 0) {
<button class="list-group-item list-group-item-action bg-light d-flex align-items-center" (click)="applyClicked()" [disabled]="!modelIsDirty || disabled">
<small class="ms-2" [ngClass]="{'fw-bold': modelIsDirty}" i18n>Apply</small>
<button class="list-group-item list-group-item-action bg-light d-flex align-items-center" (click)="applyClicked()" [disabled]="!modelIsDirty() || disabled">
<small class="ms-2" [ngClass]="{'fw-bold': modelIsDirty()}" i18n>Apply</small>
<i-bs width="1.5em" height="1em" name="arrow-right"></i-bs>
</button>
}
@@ -221,7 +221,7 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
.dispatchEvent(new MouseEvent('click')) // open
selectionModel.toggle(items[0].id)
fixture.detectChanges()
expect(component.modelIsDirty).toBeTruthy()
expect(component.modelIsDirty()).toBeTruthy()
let applyResult: ChangedItems
const closeSpy = jest.spyOn(component.dropdown, 'close')
component.apply.subscribe((result) => (applyResult = result))
@@ -244,7 +244,7 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
.dispatchEvent(new MouseEvent('click')) // open
selectionModel.toggle(items[0].id)
fixture.detectChanges()
expect(component.modelIsDirty).toBeTruthy()
expect(component.modelIsDirty()).toBeTruthy()
let applyResult: ChangedItems
component.apply.subscribe((result) => (applyResult = result))
component.dropdown.close()
@@ -12,6 +12,7 @@ import {
Output,
ViewChild,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbDropdown, NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
@@ -796,7 +797,7 @@ export class FilterableDropdownComponent
return this.title ? this.title.replace(/\s/g, '_').toLowerCase() : null
}
modelIsDirty: boolean = false
readonly modelIsDirty = signal(false)
private keyboardIndex: number
@@ -811,7 +812,7 @@ export class FilterableDropdownComponent
constructor() {
super()
this.selectionModelChange.subscribe((updatedModel) => {
this.modelIsDirty = updatedModel.isDirty()
this.modelIsDirty.set(updatedModel.isDirty())
})
}
@@ -858,7 +859,7 @@ export class FilterableDropdownComponent
}, 0)
if (this.editing) {
this.selectionModel.reset()
this.modelIsDirty = false
this.modelIsDirty.set(false)
}
this.selectionModel.singleSelect =
this.editing && !this.selectionModel.manyToOne
@@ -5,6 +5,7 @@ import {
inject,
Input,
Output,
signal,
} from '@angular/core'
import {
FormsModule,
@@ -63,11 +64,11 @@ export class CustomFieldsValuesComponent extends AbstractInputComponent<Object>
super()
customFieldsService.listAll().subscribe((items) => {
this.fields = items.results
this.fields.set(items.results)
})
}
private fields: CustomField[]
private readonly fields = signal<CustomField[]>([])
private _selectedFields: number[]
@@ -90,6 +91,6 @@ export class CustomFieldsValuesComponent extends AbstractInputComponent<Object>
public removeSelectedField: EventEmitter<number> = new EventEmitter<number>()
public getCustomField(id: number): CustomField {
return this.fields.find((field) => field.id === id)
return this.fields().find((field) => field.id === id)
}
}
@@ -33,7 +33,7 @@
[compareWith]="compareDocuments"
[trackByFn]="trackByFn"
[minTermLength]="2"
[loading]="loading"
[loading]="loading()"
[typeahead]="documentsInput$"
(mousedown)="$event.stopImmediatePropagation()"
(change)="onChange(selectedDocumentIDs)">
@@ -2,7 +2,11 @@ import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NG_VALUE_ACCESSOR } from '@angular/forms'
import { of, throwError } from 'rxjs'
import { By } from '@angular/platform-browser'
import { provideRouter } from '@angular/router'
import { NgSelectComponent } from '@ng-select/ng-select'
import { allIcons, NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { of, Subject, throwError } from 'rxjs'
import { FILTER_SIMPLE_TITLE } from 'src/app/data/filter-rule-type'
import { DocumentService } from 'src/app/services/rest/document.service'
import { DocumentLinkComponent } from './document-link.component'
@@ -33,10 +37,11 @@ describe('DocumentLinkComponent', () => {
beforeEach(() => {
TestBed.configureTestingModule({
imports: [DocumentLinkComponent],
imports: [DocumentLinkComponent, NgxBootstrapIconsModule.pick(allIcons)],
providers: [
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
provideRouter([]),
],
})
documentService = TestBed.inject(DocumentService)
@@ -60,6 +65,25 @@ describe('DocumentLinkComponent', () => {
expect(getSpy).toHaveBeenCalled()
})
it('should render loading and selected documents after async updates', async () => {
const result$ = new Subject<any>()
jest.spyOn(documentService, 'getFew').mockReturnValue(result$)
component.writeValue([1])
await fixture.whenStable()
const select = fixture.debugElement.query(By.directive(NgSelectComponent))
.componentInstance as NgSelectComponent
expect(select.loading()).toBe(true)
result$.next({ count: 1, all: [1], results: [documents[0]] })
result$.complete()
await fixture.whenStable()
expect(select.loading()).toBe(false)
expect(fixture.nativeElement.textContent).toContain(documents[0].title)
})
it('shoud maintain ordering of selected documents', () => {
const getSpy = jest.spyOn(documentService, 'getFew')
getSpy.mockImplementation((ids) => {
@@ -6,6 +6,7 @@ import {
Input,
OnDestroy,
OnInit,
signal,
} from '@angular/core'
import {
FormsModule,
@@ -63,7 +64,7 @@ export class DocumentLinkComponent
documentsInput$ = new Subject<string>()
foundDocuments$: Observable<Document[]>
loading = false
readonly loading = signal(false)
selectedDocuments: Document[] = []
private unsubscribeNotifier: Subject<any> = new Subject()
@@ -93,12 +94,12 @@ export class DocumentLinkComponent
this.selectedDocuments = []
super.writeValue([])
} else {
this.loading = true
this.loading.set(true)
this.documentsService
.getFew(documentIDs, { fields: 'id,title' })
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((documentResults) => {
this.loading = false
this.loading.set(false)
this.selectedDocuments = documentIDs.map(
(id) => documentResults.results.find((d) => d.id === id) ?? {}
)
@@ -113,7 +114,7 @@ export class DocumentLinkComponent
this.documentsInput$.pipe(
distinctUntilChanged(),
takeUntil(this.unsubscribeNotifier),
tap(() => (this.loading = true)),
tap(() => this.loading.set(true)),
switchMap((title) =>
this.documentsService
.listFiltered(
@@ -133,7 +134,7 @@ export class DocumentLinkComponent
)
),
catchError(() => of([])), // empty on error
tap(() => (this.loading = false))
tap(() => this.loading.set(false))
)
)
)
@@ -37,6 +37,7 @@ export class NumberComponent extends AbstractInputComponent<number> {
this.documentService.getNextAsn().subscribe((nextAsn) => {
this.value = nextAsn
this.onChange(this.value)
this.changeDetector.markForCheck()
})
}
@@ -157,8 +157,8 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
private removeChildren(tagIDs: number[], tag: Tag) {
if (tag.children?.length) {
const childIDs = tag.children.map((child) => child.id)
tagIDs = tagIDs.filter((id) => !childIDs.includes(id))
const childIDs = new Set(tag.children.map((child) => child.id))
tagIDs = tagIDs.filter((id) => !childIDs.has(id))
for (const child of tag.children) {
tagIDs = this.removeChildren(tagIDs, child)
}
@@ -14,7 +14,7 @@ import { environment } from 'src/environments/environment'
})
export class PageHeaderComponent {
private titleService = inject(Title)
private clipboard = inject(Clipboard)
private readonly clipboard = inject(Clipboard)
readonly id = input<number>(undefined)
readonly subTitle = input('')
@@ -49,8 +49,7 @@ describe('PngxPdfViewerComponent', () => {
new URL('assets/js/pdf.worker.min.mjs', document.baseURI).toString()
)
const isVisible = (component as any).findController.onIsPageVisible as
| (() => boolean)
| undefined
(() => boolean) | undefined
expect(isVisible?.()).toBe(true)
expect(loadSpy).toHaveBeenCalledWith(
expect.objectContaining({ numPages: 1 })
@@ -61,6 +60,7 @@ describe('PngxPdfViewerComponent', () => {
it('resolves the worker source relative to the document base URI', async () => {
setBaseHref('/paperless/')
const getDocumentSpy = jest.spyOn(pdfjs, 'getDocument')
await initComponent()
@@ -70,6 +70,13 @@ describe('PngxPdfViewerComponent', () => {
expect(pdfjs.GlobalWorkerOptions.workerSrc).toContain(
'/paperless/assets/js/pdf.worker.min.mjs'
)
expect(getDocumentSpy).toHaveBeenCalledWith({
url: 'test.pdf',
password: undefined,
withCredentials: true,
wasmUrl: expect.stringContaining('/paperless/assets/wasm/'),
iccUrl: expect.stringContaining('/paperless/assets/iccs/'),
})
})
it('initializes single-page viewer and disables text layer', async () => {
@@ -174,10 +174,12 @@ export class PngxPdfViewerComponent
'assets/js/pdf.worker.min.mjs',
this.document.baseURI
).toString()
let initOptions = {
const initOptions = {
url: this.src,
password: this.password,
withCredentials: true,
wasmUrl: new URL('assets/wasm/', this.document.baseURI).toString(),
iccUrl: new URL('assets/iccs/', this.document.baseURI).toString(),
}
this.loadingTask = getDocument(initOptions)
try {
@@ -68,7 +68,7 @@
[(ngModel)]="selectionModel.includeUsers"
[disabled]="disabled"
[clearable]="false"
[items]="users"
[items]="users()"
bindLabel="username"
multiple="true"
bindValue="id"
@@ -1,5 +1,12 @@
import { NgClass } from '@angular/common'
import { Component, EventEmitter, Input, Output, inject } from '@angular/core'
import {
Component,
EventEmitter,
Input,
Output,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectComponent } from '@ng-select/ng-select'
@@ -75,7 +82,7 @@ export class PermissionsFilterDropdownComponent extends ComponentWithPermissions
@Output()
ownerFilterSet = new EventEmitter<PermissionsSelectionModel>()
users: User[]
readonly users = signal<User[]>([])
hideUnowned: boolean
@@ -102,7 +109,7 @@ export class PermissionsFilterDropdownComponent extends ComponentWithPermissions
.listAll()
.pipe(first())
.subscribe({
next: (result) => (this.users = result.results),
next: (result) => this.users.set(result.results),
})
}
}
@@ -57,15 +57,15 @@
</div>
</div>
<div class="col-12 col-md-6">
@if (socialAccounts?.length > 0) {
@if (socialAccounts().length > 0) {
<div class="mb-3">
<p i18n>Connected social accounts</p>
<ul class="list-group">
@for (account of socialAccounts; track account.id) {
@for (account of socialAccounts(); track account.id) {
<li class="list-group-item"
ngbPopover="Set a password before disconnecting social account."
i18n-ngbPopover
[disablePopover]="hasUsablePassword"
[disablePopover]="hasUsablePassword()"
triggers="mouseenter:mouseleave">
{{account.name}} ({{account.provider}})
<pngx-confirm-button
@@ -75,7 +75,7 @@
i18n-title
buttonClasses="btn-outline-danger btn-sm ms-2 align-baseline"
iconName="trash"
[disabled]="!hasUsablePassword"
[disabled]="!hasUsablePassword()"
(confirm)="disconnectSocialAccount(account.id)">
</pngx-confirm-button>
</li>
@@ -84,11 +84,11 @@
<div class="form-text text-muted text-end fst-italic" i18n>Warning: disconnecting social accounts cannot be undone</div>
</div>
}
@if (socialAccountProviders?.length > 0) {
@if (socialAccountProviders().length > 0) {
<div class="mb-3">
<p i18n>Connect new social account</p>
<div class="list-group">
@for (provider of socialAccountProviders; track provider.name) {
@for (provider of socialAccountProviders(); track provider.name) {
<a class="list-group-item list-group-item-action text-primary d-flex align-items-center" href="{{ provider.login_url }}" rel="noopener noreferrer">
{{provider.name}}<i-bs class="pb-1 ms-2" name="box-arrow-up-right"></i-bs>
</a>
@@ -96,7 +96,7 @@
</div>
</div>
}
@if (!isTotpEnabled) {
@if (!isTotpEnabled()) {
<div ngbAccordion>
<div ngbAccordionItem>
<h2 ngbAccordionHeader>
@@ -105,10 +105,10 @@
<div ngbAccordionCollapse>
<div ngbAccordionBody>
<ng-template>
@if (totpSettingsLoading) {
@if (totpSettingsLoading()) {
<div class="spinner-border spinner-border-sm fw-normal ms-2 me-auto" role="status"></div>
<div class="visually-hidden" i18n>Loading...</div>
} @else if (totpSettings) {
} @else if (totpSettings()) {
<figure class="figure">
@if (qrSvgDataUrl) {
<img class="bg-white d-inline-block" [src]="qrSvgDataUrl" alt="Authenticator QR code">
@@ -116,14 +116,14 @@
<figcaption class="figure-caption text-end mt-2" i18n>Scan the QR code with your authenticator app and then enter the code below</figcaption>
</figure>
<p>
<ng-container i18n>Authenticator secret</ng-container>: <code>{{totpSettings.secret}}</code>.
<ng-container i18n>Authenticator secret</ng-container>: <code>{{totpSettings().secret}}</code>.
<ng-container i18n>You can store this secret and use it to reinstall your authenticator app at a later time.</ng-container>
</p>
<div class="input-group mb-3">
<input type="text" class="form-control" formControlName="totp_code" placeholder="Code" i18n-placeholder>
<button type="button" class="btn btn-primary ml-auto" (click)="activateTotp()" [disabled]="totpLoading">
<button type="button" class="btn btn-primary ml-auto" (click)="activateTotp()" [disabled]="totpLoading()">
<ng-container i18n>Enable</ng-container>
@if (totpLoading) {
@if (totpLoading()) {
<div class="spinner-border spinner-border-sm fw-normal ms-2" role="status"></div>
<div class="visually-hidden" i18n>Loading...</div>
}
@@ -137,18 +137,18 @@
</div>
} @else {
<label class="d-block mb-2" i18n>Two-factor Authentication</label>
@if (recoveryCodes) {
@if (recoveryCodes()) {
<div class="alert alert-warning" role="alert">
<i-bs name="exclamation-triangle" class="me-1"></i-bs><ng-container i18n>Recovery codes will not be shown again, make sure to save them.</ng-container>
</div>
<div class="d-flex flex-row align-items-start mb-3">
<ul class="list-group w-50">
@for (code of recoveryCodes; track code; let i = $index) {
@for (code of recoveryCodes(); track code; let i = $index) {
@if (i % 2 === 0) {
<li class="list-group-item d-flex justify-content-around align-items-center">
<code>{{code}}</code>
@if (recoveryCodes[i + 1]) {
<code>{{recoveryCodes[i + 1]}}</code>
@if (recoveryCodes()[i + 1]) {
<code>{{recoveryCodes()[i + 1]}}</code>
}
</li>
}
@@ -171,7 +171,7 @@
i18n-title
buttonClasses="btn-outline-danger btn-sm"
iconName="trash"
[disabled]="totpLoading"
[disabled]="totpLoading()"
(confirm)="deactivateTotp()">
</pngx-confirm-button>
}
@@ -10,7 +10,7 @@ import {
NgbPopoverModule,
} from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { NEVER, of, Subject, throwError } from 'rxjs'
import { ProfileService } from 'src/app/services/profile.service'
import { ToastService } from 'src/app/services/toast.service'
import * as navUtils from 'src/app/utils/navigation'
@@ -60,6 +60,10 @@ describe('ProfileEditDialogComponent', () => {
providers: [NgbActiveModal, provideHttpClient(withInterceptorsFromDi())],
})
profileService = TestBed.inject(ProfileService)
jest.spyOn(profileService, 'get').mockReturnValue(NEVER)
jest
.spyOn(profileService, 'getSocialAccountProviders')
.mockReturnValue(of([]))
toastService = TestBed.inject(ToastService)
clipboard = TestBed.inject(Clipboard)
fixture = TestBed.createComponent(ProfileEditDialogComponent)
@@ -155,7 +159,7 @@ describe('ProfileEditDialogComponent', () => {
'getSocialAccountProviders'
)
getProvidersSpy.mockReturnValue(of(socialAccountProviders))
component.hasUsablePassword = true
component.hasUsablePassword.set(true)
component.ngOnInit()
component.form.get('password').patchValue('new*pass')
component.onPasswordKeyUp({
@@ -268,6 +272,27 @@ describe('ProfileEditDialogComponent', () => {
expect(getProvidersSpy).toHaveBeenCalled()
})
it('should render social account providers after an async update', async () => {
const providers$ = new Subject<typeof socialAccountProviders>()
jest.spyOn(profileService, 'get').mockReturnValue(of(profile))
jest
.spyOn(profileService, 'getSocialAccountProviders')
.mockReturnValue(providers$)
component.ngOnInit()
await fixture.whenStable()
expect(
fixture.nativeElement.querySelector('a[href="https://example.com"]')
).toBeNull()
providers$.next(socialAccountProviders)
await fixture.whenStable()
expect(
fixture.nativeElement.querySelector('a[href="https://example.com"]')
).not.toBeNull()
})
it('should remove disconnected social account from component, show error if needed', () => {
const disconnectSpy = jest.spyOn(profileService, 'disconnectSocialAccount')
const getSpy = jest.spyOn(profileService, 'get')
@@ -276,7 +301,7 @@ describe('ProfileEditDialogComponent', () => {
const errorSpy = jest.spyOn(toastService, 'showError')
expect(component.socialAccounts).toContainEqual(socialAccount)
expect(component.socialAccounts()).toContainEqual(socialAccount)
// fail first
disconnectSpy.mockReturnValueOnce(
@@ -289,7 +314,7 @@ describe('ProfileEditDialogComponent', () => {
disconnectSpy.mockReturnValue(of(socialAccount.id))
component.disconnectSocialAccount(socialAccount.id)
expect(disconnectSpy).toHaveBeenCalled()
expect(component.socialAccounts).not.toContainEqual(socialAccount)
expect(component.socialAccounts()).not.toContainEqual(socialAccount)
})
it('should get totp settings', () => {
@@ -310,7 +335,7 @@ describe('ProfileEditDialogComponent', () => {
getSpy.mockReturnValue(of(settings))
component.gettotpSettings()
expect(getSpy).toHaveBeenCalled()
expect(component.totpSettings).toEqual(settings)
expect(component.totpSettings()).toEqual(settings)
})
it('should activate totp', () => {
@@ -319,15 +344,15 @@ describe('ProfileEditDialogComponent', () => {
const toastInfoSpy = jest.spyOn(toastService, 'showInfo')
const error = new Error('failed to activate totp')
activateSpy.mockReturnValueOnce(throwError(() => error))
component.totpSettings = {
component.totpSettings.set({
url: 'http://localhost/',
qr_svg: 'svg',
secret: 'secret',
}
})
component.form.get('totp_code').patchValue('123456')
component.activateTotp()
expect(activateSpy).toHaveBeenCalledWith(
component.totpSettings.secret,
component.totpSettings().secret,
component.form.get('totp_code').value
)
expect(toastErrorSpy).toHaveBeenCalled()
@@ -341,8 +366,8 @@ describe('ProfileEditDialogComponent', () => {
)
component.activateTotp()
expect(toastInfoSpy).toHaveBeenCalled()
expect(component.isTotpEnabled).toBeTruthy()
expect(component.recoveryCodes).toEqual(['1', '2', '3'])
expect(component.isTotpEnabled()).toBeTruthy()
expect(component.recoveryCodes()).toEqual(['1', '2', '3'])
})
it('should deactivate totp', () => {
@@ -362,13 +387,13 @@ describe('ProfileEditDialogComponent', () => {
deactivateSpy.mockReturnValueOnce(of(true))
component.deactivateTotp()
expect(toastInfoSpy).toHaveBeenCalled()
expect(component.isTotpEnabled).toBeFalsy()
expect(component.isTotpEnabled()).toBeFalsy()
})
it('should copy recovery codes', () => {
jest.useFakeTimers()
const copySpy = jest.spyOn(clipboard, 'copy')
component.recoveryCodes = ['1', '2', '3']
component.recoveryCodes.set(['1', '2', '3'])
component.copyRecoveryCodes()
expect(copySpy).toHaveBeenCalledWith('1\n2\n3')
jest.advanceTimersByTime(3000)
@@ -56,6 +56,14 @@ export class ProfileEditDialogComponent
readonly showEmailConfirm = signal(false)
readonly copied = signal(false)
readonly codesCopied = signal(false)
readonly hasUsablePassword = signal(false)
readonly isTotpEnabled = signal(false)
readonly totpSettings = signal<TotpSettings>(undefined)
readonly totpSettingsLoading = signal(false)
readonly totpLoading = signal(false)
readonly recoveryCodes = signal<string[]>(undefined)
readonly socialAccounts = signal<SocialAccount[]>([])
readonly socialAccountProviders = signal<SocialAccountProvider[]>([])
public form = new FormGroup({
email: new FormControl(''),
@@ -72,25 +80,15 @@ export class ProfileEditDialogComponent
private newPassword: string
private passwordConfirm: string
public hasUsablePassword: boolean = false
private currentEmail: string
private newEmail: string
private emailConfirm: string
public isTotpEnabled: boolean = false
public totpSettings: TotpSettings
public totpSettingsLoading: boolean = false
public totpLoading: boolean = false
public recoveryCodes: string[]
public socialAccounts: SocialAccount[] = []
public socialAccountProviders: SocialAccountProvider[] = []
get qrSvgDataUrl(): string | null {
if (!this.totpSettings?.qr_svg) {
if (!this.totpSettings()?.qr_svg) {
return null
}
return `data:image/svg+xml;utf8,${encodeURIComponent(this.totpSettings.qr_svg)}`
return `data:image/svg+xml;utf8,${encodeURIComponent(this.totpSettings().qr_svg)}`
}
ngOnInit(): void {
@@ -107,20 +105,20 @@ export class ProfileEditDialogComponent
this.onEmailChange()
})
this.currentPassword = profile.password
this.hasUsablePassword = profile.has_usable_password
this.hasUsablePassword.set(profile.has_usable_password)
this.form.get('password').valueChanges.subscribe((newPassword) => {
this.newPassword = newPassword
this.onPasswordChange()
})
this.socialAccounts = profile.social_accounts
this.isTotpEnabled = profile.is_mfa_enabled
this.socialAccounts.set(profile.social_accounts ?? [])
this.isTotpEnabled.set(profile.is_mfa_enabled)
})
this.profileService
.getSocialAccountProviders()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((providers) => {
this.socialAccountProviders = providers
this.socialAccountProviders.set(providers ?? [])
})
}
@@ -259,7 +257,9 @@ export class ProfileEditDialogComponent
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (id: number) => {
this.socialAccounts = this.socialAccounts.filter((a) => a.id != id)
this.socialAccounts.update((accounts) =>
accounts.filter((account) => account.id != id)
)
},
error: (error) => {
this.toastService.showError(
@@ -271,36 +271,39 @@ export class ProfileEditDialogComponent
}
public gettotpSettings(): void {
this.totpSettingsLoading = true
this.totpSettingsLoading.set(true)
this.profileService
.getTotpSettings()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (totpSettings) => {
this.totpSettingsLoading = false
this.totpSettings = totpSettings
this.totpSettingsLoading.set(false)
this.totpSettings.set(totpSettings)
},
error: (error) => {
this.toastService.showError(
$localize`Error fetching TOTP settings`,
error
)
this.totpSettingsLoading = false
this.totpSettingsLoading.set(false)
},
})
}
public activateTotp(): void {
this.totpLoading = true
this.totpLoading.set(true)
this.form.get('totp_code').disable()
this.profileService
.activateTotp(this.totpSettings.secret, this.form.get('totp_code').value)
.activateTotp(
this.totpSettings().secret,
this.form.get('totp_code').value
)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (activationResponse) => {
this.totpLoading = false
this.isTotpEnabled = activationResponse.success
this.recoveryCodes = activationResponse.recovery_codes
this.totpLoading.set(false)
this.isTotpEnabled.set(activationResponse.success)
this.recoveryCodes.set(activationResponse.recovery_codes)
this.form.get('totp_code').enable()
if (activationResponse.success) {
this.toastService.showInfo($localize`TOTP activated successfully`)
@@ -309,7 +312,7 @@ export class ProfileEditDialogComponent
}
},
error: (error) => {
this.totpLoading = false
this.totpLoading.set(false)
this.form.get('totp_code').enable()
this.toastService.showError($localize`Error activating TOTP`, error)
},
@@ -317,15 +320,15 @@ export class ProfileEditDialogComponent
}
public deactivateTotp(): void {
this.totpLoading = true
this.totpLoading.set(true)
this.profileService
.deactivateTotp()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (success) => {
this.totpLoading = false
this.isTotpEnabled = !success
this.recoveryCodes = null
this.totpLoading.set(false)
this.isTotpEnabled.set(!success)
this.recoveryCodes.set(null)
if (success) {
this.toastService.showInfo($localize`TOTP deactivated successfully`)
} else {
@@ -333,14 +336,14 @@ export class ProfileEditDialogComponent
}
},
error: (error) => {
this.totpLoading = false
this.totpLoading.set(false)
this.toastService.showError($localize`Error deactivating TOTP`, error)
},
})
}
public copyRecoveryCodes(): void {
this.clipboard.copy(this.recoveryCodes.join('\n'))
this.clipboard.copy(this.recoveryCodes().join('\n'))
this.codesCopied.set(true)
setTimeout(() => {
this.codesCopied.set(false)
@@ -65,8 +65,9 @@
<dd class="col-sm-8"><code>{{ createdBundle.slug }}</code></dd>
<dt class="col-sm-4" i18n>Link</dt>
<dd class="col-sm-8">
<label class="visually-hidden" for="shareBundleLink" i18n>Share link</label>
<div class="input-group input-group-sm">
<input class="form-control" type="text" [value]="getShareUrl(createdBundle)" readonly>
<input id="shareBundleLink" class="form-control" type="text" [value]="getShareUrl(createdBundle)" readonly>
<button
class="btn btn-outline-primary"
type="button"
@@ -14,7 +14,9 @@
}
<div content class="wrapper fade" [class.show]="show()">
@if (displayMode() === DisplayMode.TABLE) {
@if (error()) {
<div class="alert alert-danger mb-0" role="alert"><ng-container i18n>Error while loading documents</ng-container>: {{error()}}</div>
} @else if (displayMode() === DisplayMode.TABLE) {
<table class="table table-hover mb-0 mt-n2 align-middle">
<thead>
<tr>
@@ -1,6 +1,10 @@
import { DragDropModule } from '@angular/cdk/drag-drop'
import { DatePipe } from '@angular/common'
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import {
HttpErrorResponse,
provideHttpClient,
withInterceptorsFromDi,
} from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
@@ -8,7 +12,7 @@ import { Router } from '@angular/router'
import { RouterTestingModule } from '@angular/router/testing'
import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { Subject, of } from 'rxjs'
import { Subject, of, throwError } from 'rxjs'
import { routes } from 'src/app/app-routing.module'
import { CustomFieldDisplayComponent } from 'src/app/components/common/custom-field-display/custom-field-display.component'
import { PreviewPopupComponent } from 'src/app/components/common/preview-popup/preview-popup.component'
@@ -230,6 +234,27 @@ describe('SavedViewWidgetComponent', () => {
expect(component.documents()).toEqual(documentResults)
})
it('should show an error if documents fail to load', () => {
jest.spyOn(documentService, 'listFiltered').mockReturnValue(
throwError(
() =>
new HttpErrorResponse({
error: { added__date__lte: ['Enter a valid date.'] },
status: 400,
})
)
)
component.reload()
fixture.detectChanges()
expect(component.loading()).toBe(false)
expect(component.error()).toEqual('Added: Enter a valid date.')
expect(fixture.debugElement.nativeElement.textContent).toContain(
'Error while loading documents: Added: Enter a valid date.'
)
})
it('should reload on document consumption finished', () => {
const fileStatusSubject = new Subject<FileStatus>()
jest
@@ -125,6 +125,8 @@ export class SavedViewWidgetComponent
readonly count = signal<number>(null)
readonly error = signal<string | null>(null)
placeholderRows: number[] = []
ngOnInit(): void {
@@ -180,6 +182,7 @@ export class SavedViewWidgetComponent
reload() {
this.loading.set(this.documents().length == 0)
this.error.set(null)
this.show.set(true)
this.documentService
.listFiltered(
@@ -191,12 +194,40 @@ export class SavedViewWidgetComponent
{ truncate_content: true }
)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((result) => {
this.documents.set(result.results)
this.count.set(result.count)
this.savedViewService.setDocumentCount(this.savedView, result.count)
this.loading.set(false)
this.show.set(true)
.subscribe({
next: (result) => {
this.documents.set(result.results)
this.count.set(result.count)
this.savedViewService.setDocumentCount(this.savedView, result.count)
this.loading.set(false)
this.show.set(true)
},
error: (error) => {
this.documents.set([])
this.count.set(null)
let errorMessage
if (
typeof error.error === 'object' &&
Object.keys(error.error).length > 0
) {
errorMessage = Object.keys(error.error)
.map((fieldName) => {
const fieldNameBase = fieldName.split('__')[0]
const fieldError: Array<string> = error.error[fieldName]
return `${
this.documentService.sortFields.find(
(f) => f.field?.split('__')[0] == fieldNameBase
)?.name ?? fieldNameBase
}: ${fieldError[0]}`
})
.join(', ')
} else {
errorMessage = error.error
}
this.error.set(errorMessage)
this.loading.set(false)
this.show.set(true)
},
})
}
@@ -1 +1 @@
<p i18n>Searching document with asn {{asn}}</p>
<p i18n>Searching document with asn {{asn()}}</p>
@@ -1,4 +1,4 @@
import { Component, OnInit, inject } from '@angular/core'
import { Component, OnInit, inject, signal } from '@angular/core'
import { ActivatedRoute, Router } from '@angular/router'
import { FILTER_ASN } from '../../data/filter-rule-type'
import { DocumentService } from '../../services/rest/document.service'
@@ -13,13 +13,13 @@ export class DocumentAsnComponent implements OnInit {
private route = inject(ActivatedRoute)
private router = inject(Router)
asn: string
readonly asn = signal<string>(undefined)
ngOnInit(): void {
this.route.paramMap.subscribe((paramMap) => {
this.asn = paramMap.get('id')
this.asn.set(paramMap.get('id'))
this.documentsService
.listAllFilteredIds([{ rule_type: FILTER_ASN, value: this.asn }])
.listAllFilteredIds([{ rule_type: FILTER_ASN, value: this.asn() }])
.subscribe((documentId) => {
if (documentId.length == 1) {
this.router.navigate(['documents', documentId[0]])
@@ -82,23 +82,6 @@
<i-bs name="pencil" class="me-1"></i-bs><ng-container i18n>PDF Editor</ng-container>
</button>
<button
ngbDropdownItem
(click)="runZoneOcr()"
[disabled]="!userCanEdit || !document?.document_type"
*pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.OcrTemplate }"
>
<i-bs width="1em" height="1em" name="file-earmark-ruled" class="me-1"></i-bs><span i18n>Run Zone OCR</span>
</button>
<button
ngbDropdownItem
(click)="createOcrTemplate()"
*pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.OcrTemplate }"
>
<i-bs width="1em" height="1em" name="file-earmark-medical" class="me-1"></i-bs><span i18n>Create OCR Template</span>
</button>
@if (userIsOwner && (requiresPassword || password)) {
<button ngbDropdownItem (click)="removePassword()" [disabled]="!password">
<i-bs name="unlock" class="me-1"></i-bs><ng-container i18n>Remove Password</ng-container>
@@ -332,7 +315,7 @@
<td>{{metadata()?.original_filename}}</td>
</tr>
<tr>
<td i18n>Original MD5 checksum</td>
<td i18n>Original SHA256 checksum</td>
<td>{{metadata()?.original_checksum}}</td>
</tr>
<tr>
@@ -345,7 +328,7 @@
</tr>
@if (metadata()?.has_archive_version) {
<tr>
<td i18n>Archive MD5 checksum</td>
<td i18n>Archive SHA256 checksum</td>
<td>{{metadata()?.archive_checksum}}</td>
</tr>
}
@@ -499,9 +482,7 @@
</pngx-pdf-viewer>
</div>
} @else {
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%">
<span>Preview is unavailable.</span>
</object>
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%"></object>
}
}
@case (ContentRenderType.Text) {
@@ -522,9 +503,7 @@
}
}
@case (ContentRenderType.Other) {
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%">
<span>Preview is unavailable.</span>
</object>
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%"></object>
}
}
@if (requiresPassword) {
@@ -510,7 +510,7 @@ describe('DocumentDetailComponent', () => {
const deleteButtons = fixture.debugElement.queryAll(
By.css('pngx-confirm-button')
)
expect(deleteButtons.length).toEqual(1)
expect(deleteButtons).toHaveLength(1)
})
it('should fall back to details tab when duplicates tab is active but no duplicates', () => {
@@ -1429,48 +1429,6 @@ export class DocumentDetailComponent
})
}
runZoneOcr() {
this.documentsService.runZoneOcr(this.document.id).subscribe({
next: (res) => {
const results = res.results ?? []
if (results.length) {
const failed = results.filter(
(r) =>
r.value === null ||
r.value === undefined ||
`${r.value}`.trim() === ''
)
const filled = results.length - failed.length
let msg = $localize`Filled ${filled} of ${results.length} fields`
if (failed.length) {
const names = failed.map((r) => r.zone).join(', ')
msg = `${msg}. ${$localize`Failed to match zones: ${names}`}`
}
this.toastService.showInfo(msg)
} else {
this.toastService.showInfo(
$localize`Zone OCR ran but no results extracted.`
)
}
this.documentsService
.get(this.documentId)
.subscribe((doc) => this.updateComponent(doc))
},
error: (error) => {
this.toastService.showError($localize`Zone OCR failed`, error)
},
})
}
createOcrTemplate() {
this.router.navigate(['/ocr-templates', 'new'], {
queryParams: {
document_type: this.document.document_type,
sample_document: this.document.id,
},
})
}
private getSelectedNonLatestVersionId(): number | null {
const versions = this.document()?.versions ?? []
if (!versions.length || !this.selectedVersionId()) {
@@ -1,6 +1,6 @@
@if (loading()) {
<div class="d-flex">
<output class="spinner-border spinner-border-sm fw-normal" role="status"></output>
<output class="spinner-border spinner-border-sm fw-normal"></output>
</div>
} @else {
<ul class="list-group">
@@ -26,11 +26,11 @@ import { UserService } from 'src/app/services/rest/user.service'
],
})
export class DocumentHistoryComponent implements OnInit {
private documentService = inject(DocumentService)
private correspondentService = inject(CorrespondentService)
private storagePathService = inject(StoragePathService)
private documentTypeService = inject(DocumentTypeService)
private userService = inject(UserService)
private readonly documentService = inject(DocumentService)
private readonly correspondentService = inject(CorrespondentService)
private readonly storagePathService = inject(StoragePathService)
private readonly documentTypeService = inject(DocumentTypeService)
private readonly userService = inject(UserService)
public AuditLogAction = AuditLogAction
@@ -70,7 +70,7 @@ export class DocumentHistoryComponent implements OnInit {
return cached
}
const idInt = parseInt(id, 10)
const idInt = Number.parseInt(id, 10)
const fallback$ = of(id)
let result$: Observable<string>
@@ -5,10 +5,11 @@
</button>
<div class="dropdown-menu shadow" ngbDropdownMenu>
<div class="px-3 py-2 mb-2">
@if (versionUploadState === UploadState.Idle) {
@if (versionUploadState() === UploadState.Idle) {
<div class="input-group input-group-sm mb-2">
<span class="input-group-text" i18n>Label</span>
<label class="input-group-text" for="newVersionLabel" i18n>Label</label>
<input
id="newVersionLabel"
class="form-control"
type="text"
[(ngModel)]="newVersionLabel"
@@ -31,7 +32,7 @@
<i-bs name="file-earmark-plus"></i-bs><span class="ps-1" i18n>Add new version</span>
</button>
} @else {
@switch (versionUploadState) {
@switch (versionUploadState()) {
@case (UploadState.Uploading) {
<div class="small text-muted mt-1 d-flex align-items-center">
<output class="spinner-border spinner-border-sm me-2" aria-hidden="true"></output>
@@ -49,8 +50,8 @@
<span i18n>Version upload failed.</span>
<button type="button" class="btn btn-link btn-sm p-0 ms-2" (click)="clearVersionUploadStatus()" i18n>Dismiss</button>
</div>
@if (versionUploadError) {
<div class="small text-muted mt-1">{{ versionUploadError }}</div>
@if (versionUploadError()) {
<div class="small text-muted mt-1">{{ versionUploadError() }}</div>
}
}
}
@@ -70,15 +71,19 @@
}
</div>
<div class="d-flex flex-column">
@if (isEditingVersion(version.id)) {
<label class="visually-hidden" for="versionLabel{{ version.id }}" i18n>Version label</label>
}
<div class="input-group input-group-sm mb-1">
@if (isEditingVersion(version.id)) {
<input
id="versionLabel{{ version.id }}"
class="form-control"
type="text"
[(ngModel)]="versionLabelDraft"
i18n-placeholder
placeholder="Version label"
[disabled]="savingVersionLabelId !== null"
[disabled]="savingVersionLabelId() !== null"
(keydown.enter)="submitEditedVersionLabel(version, $event)"
(keydown.escape)="cancelEditingVersion($event)"
(click)="$event.stopPropagation()"
@@ -96,7 +101,7 @@
<button
type="button"
class="btn btn-outline-secondary"
[disabled]="savingVersionLabelId !== null"
[disabled]="savingVersionLabelId() !== null"
(click)="isEditingVersion(version.id) ? submitEditedVersionLabel(version, $event) : beginEditingVersion(version, $event)"
>
@if (isEditingVersion(version.id)) {
@@ -205,7 +205,7 @@ describe('DocumentVersionDropdownComponent', () => {
{ id: 3, is_root: true, checksum: 'aaaa' },
{ id: 10, is_root: false, checksum: 'bbbb', version_label: 'Updated' },
])
expect(component.savingVersionLabelId).toBeNull()
expect(component.savingVersionLabelId()).toBeNull()
})
it('saveVersionLabel should show error toast on failure', () => {
@@ -218,7 +218,7 @@ describe('DocumentVersionDropdownComponent', () => {
'Error updating version label',
error
)
expect(component.savingVersionLabelId).toBeNull()
expect(component.savingVersionLabelId()).toBeNull()
})
it('onVersionFileSelected should upload and update versions after websocket success', () => {
@@ -252,11 +252,11 @@ describe('DocumentVersionDropdownComponent', () => {
expect(versionsEmitSpy).toHaveBeenCalledWith(versions)
expect(selectedEmitSpy).toHaveBeenCalledWith(20)
expect(component.newVersionLabel).toEqual('')
expect(component.versionUploadState).toEqual(UploadState.Idle)
expect(component.versionUploadError).toBeNull()
expect(component.versionUploadState()).toEqual(UploadState.Idle)
expect(component.versionUploadError()).toBeNull()
})
it('onVersionFileSelected should set failed state after websocket failure', () => {
it('onVersionFileSelected should render failed state after websocket failure', async () => {
const file = new File(['test'], 'new-version.pdf', {
type: 'application/pdf',
})
@@ -266,9 +266,11 @@ describe('DocumentVersionDropdownComponent', () => {
component.onVersionFileSelected({ target: input } as Event)
failed$.next({ taskId: 'task-1', message: 'processing failed' })
await fixture.whenStable()
expect(component.versionUploadState).toEqual(UploadState.Failed)
expect(component.versionUploadError).toEqual('processing failed')
expect(component.versionUploadState()).toEqual(UploadState.Failed)
expect(component.versionUploadError()).toEqual('processing failed')
expect(fixture.nativeElement.textContent).toContain('processing failed')
expect(documentService.getVersions).not.toHaveBeenCalled()
})
@@ -282,8 +284,8 @@ describe('DocumentVersionDropdownComponent', () => {
component.onVersionFileSelected({ target: input } as Event)
expect(component.versionUploadState).toEqual(UploadState.Failed)
expect(component.versionUploadError).toEqual('Missing task ID.')
expect(component.versionUploadState()).toEqual(UploadState.Failed)
expect(component.versionUploadError()).toEqual('Missing task ID.')
expect(documentService.getVersions).not.toHaveBeenCalled()
})
@@ -298,8 +300,8 @@ describe('DocumentVersionDropdownComponent', () => {
component.onVersionFileSelected({ target: input } as Event)
expect(component.versionUploadState).toEqual(UploadState.Failed)
expect(component.versionUploadError).toEqual('upload failed')
expect(component.versionUploadState()).toEqual(UploadState.Failed)
expect(component.versionUploadError()).toEqual('upload failed')
expect(toastService.showError).toHaveBeenCalledWith(
'Error uploading new version',
error
@@ -307,8 +309,8 @@ describe('DocumentVersionDropdownComponent', () => {
})
it('ngOnChanges should clear upload status on document switch', () => {
component.versionUploadState = UploadState.Failed
component.versionUploadError = 'something failed'
component.versionUploadState.set(UploadState.Failed)
component.versionUploadError.set('something failed')
component.editingVersionId = 10
component.versionLabelDraft = 'draft'
@@ -316,8 +318,8 @@ describe('DocumentVersionDropdownComponent', () => {
documentId: new SimpleChange(3, 4, false),
})
expect(component.versionUploadState).toEqual(UploadState.Idle)
expect(component.versionUploadError).toBeNull()
expect(component.versionUploadState()).toEqual(UploadState.Idle)
expect(component.versionUploadError()).toBeNull()
expect(component.editingVersionId).toBeNull()
expect(component.versionLabelDraft).toEqual('')
})
@@ -7,6 +7,7 @@ import {
OnChanges,
OnDestroy,
Output,
signal,
SimpleChanges,
} from '@angular/core'
import { FormsModule } from '@angular/forms'
@@ -59,9 +60,9 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
@Output() versionsUpdated = new EventEmitter<DocumentVersionInfo[]>()
newVersionLabel: string = ''
versionUploadState: UploadState = UploadState.Idle
versionUploadError: string | null = null
savingVersionLabelId: number | null = null
readonly versionUploadState = signal(UploadState.Idle)
readonly versionUploadError = signal<string | null>(null)
readonly savingVersionLabelId = signal<number | null>(null)
editingVersionId: number | null = null
versionLabelDraft: string = ''
@@ -101,7 +102,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
beginEditingVersion(version: DocumentVersionInfo, event?: Event): void {
event?.preventDefault()
event?.stopPropagation()
if (!this.canEditLabels || this.savingVersionLabelId !== null) return
if (!this.canEditLabels || this.savingVersionLabelId() !== null) return
this.editingVersionId = version.id
this.versionLabelDraft = version.version_label ?? ''
}
@@ -116,7 +117,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
submitEditedVersionLabel(version: DocumentVersionInfo, event?: Event): void {
event?.preventDefault()
event?.stopPropagation()
if (this.savingVersionLabelId !== null) return
if (this.savingVersionLabelId() !== null) return
const nextLabel = this.versionLabelDraft?.trim() || null
const currentLabel = version.version_label?.trim() || null
if (nextLabel === currentLabel) {
@@ -158,15 +159,15 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
}
saveVersionLabel(versionId: number, versionLabel: string | null): void {
if (this.savingVersionLabelId !== null) return
this.savingVersionLabelId = versionId
if (this.savingVersionLabelId() !== null) return
this.savingVersionLabelId.set(versionId)
this.documentsService
.updateVersionLabel(this.documentId, versionId, versionLabel)
.pipe(
first(),
finalize(() => {
if (this.savingVersionLabelId === versionId) {
this.savingVersionLabelId = null
if (this.savingVersionLabelId() === versionId) {
this.savingVersionLabelId.set(null)
}
}),
takeUntil(this.destroy$)
@@ -199,8 +200,8 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
const file = input.files[0]
input.value = ''
const label = this.newVersionLabel?.trim()
this.versionUploadState = UploadState.Uploading
this.versionUploadError = null
this.versionUploadState.set(UploadState.Uploading)
this.versionUploadError.set(null)
this.documentsService
.uploadVersion(uploadDocumentId, file, label)
.pipe(
@@ -210,7 +211,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
$localize`Uploading new version. Processing will happen in the background.`
)
this.newVersionLabel = ''
this.versionUploadState = UploadState.Processing
this.versionUploadState.set(UploadState.Processing)
}),
map((taskId) =>
typeof taskId === 'string'
@@ -219,8 +220,8 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
),
switchMap((taskId) => {
if (!taskId) {
this.versionUploadState = UploadState.Failed
this.versionUploadError = $localize`Missing task ID.`
this.versionUploadState.set(UploadState.Failed)
this.versionUploadError.set($localize`Missing task ID.`)
return of(null)
}
return merge(
@@ -240,9 +241,10 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
switchMap((result) => {
if (result?.state !== 'success') {
if (result?.state === 'failed') {
this.versionUploadState = UploadState.Failed
this.versionUploadError =
this.versionUploadState.set(UploadState.Failed)
this.versionUploadError.set(
result.message || $localize`Upload failed.`
)
}
return of(null)
}
@@ -264,8 +266,10 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
},
error: (error) => {
if (uploadDocumentId !== this.documentId) return
this.versionUploadState = UploadState.Failed
this.versionUploadError = error?.message || $localize`Upload failed.`
this.versionUploadState.set(UploadState.Failed)
this.versionUploadError.set(
error?.message || $localize`Upload failed.`
)
this.toastService.showError(
$localize`Error uploading new version`,
error
@@ -275,7 +279,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
}
clearVersionUploadStatus(): void {
this.versionUploadState = UploadState.Idle
this.versionUploadError = null
this.versionUploadState.set(UploadState.Idle)
this.versionUploadError.set(null)
}
}
@@ -10,7 +10,7 @@
[createRef]="createTag.bind(this)"
(opened)="openTagsDropdown()"
[(selectionModel)]="tagSelectionModel"
[documentCounts]="tagDocumentCounts"
[documentCounts]="tagDocumentCounts()"
(apply)="setTags($event)"
shortcutKey="t">
</pngx-filterable-dropdown>
@@ -24,7 +24,7 @@
[createRef]="createCorrespondent.bind(this)"
(opened)="openCorrespondentDropdown()"
[(selectionModel)]="correspondentSelectionModel"
[documentCounts]="correspondentDocumentCounts"
[documentCounts]="correspondentDocumentCounts()"
(apply)="setCorrespondents($event)"
shortcutKey="y">
</pngx-filterable-dropdown>
@@ -38,7 +38,7 @@
[createRef]="createDocumentType.bind(this)"
(opened)="openDocumentTypeDropdown()"
[(selectionModel)]="documentTypeSelectionModel"
[documentCounts]="documentTypeDocumentCounts"
[documentCounts]="documentTypeDocumentCounts()"
(apply)="setDocumentTypes($event)"
shortcutKey="u">
</pngx-filterable-dropdown>
@@ -52,7 +52,7 @@
[createRef]="createStoragePath.bind(this)"
(opened)="openStoragePathDropdown()"
[(selectionModel)]="storagePathsSelectionModel"
[documentCounts]="storagePathDocumentCounts"
[documentCounts]="storagePathDocumentCounts()"
(apply)="setStoragePaths($event)"
shortcutKey="i">
</pngx-filterable-dropdown>
@@ -66,7 +66,7 @@
[createRef]="createCustomField.bind(this)"
(opened)="openCustomFieldsDropdown()"
[(selectionModel)]="customFieldsSelectionModel"
[documentCounts]="customFieldDocumentCounts"
[documentCounts]="customFieldDocumentCounts()"
extraButtonTitle="Set values"
i18n-extraButtonTitle
(extraButton)="setCustomFieldValues($event)"
@@ -95,9 +95,6 @@
<button ngbDropdownItem (click)="mergeSelected()" [disabled]="!userCanAdd || list.allSelected || list.selectedCount < 2">
<i-bs name="journals" class="me-1"></i-bs><ng-container i18n>Merge</ng-container>
</button>
<button ngbDropdownItem (click)="runZoneOcrSelected()" [disabled]="!userCanEditAll || list.allSelected">
<i-bs name="file-earmark-ruled" class="me-1"></i-bs><ng-container i18n>Run Zone OCR</ng-container>
</button>
</div>
</div>
</div>
@@ -127,11 +124,11 @@
</div>
</div>
<div class="btn-group btn-group-sm">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload" (click)="downloadSelected()">
@if (!awaitingDownload) {
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()">
@if (!awaitingDownload()) {
<i-bs name="arrow-down"></i-bs>
}
@if (awaitingDownload) {
@if (awaitingDownload()) {
<div class="spinner-border spinner-border-sm" role="status">
<span class="visually-hidden">Preparing download...</span>
</div>
@@ -303,7 +303,7 @@ describe('BulkEditorComponent', () => {
component.openDocumentTypeDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.documentTypeDocumentCounts).toEqual(
expect(component.documentTypeDocumentCounts()).toEqual(
selectionData.selected_document_types
)
})
@@ -320,7 +320,7 @@ describe('BulkEditorComponent', () => {
component.openCorrespondentDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.correspondentDocumentCounts).toEqual(
expect(component.correspondentDocumentCounts()).toEqual(
selectionData.selected_correspondents
)
})
@@ -337,7 +337,7 @@ describe('BulkEditorComponent', () => {
component.openStoragePathDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.storagePathDocumentCounts).toEqual(
expect(component.storagePathDocumentCounts()).toEqual(
selectionData.selected_storage_paths
)
})
@@ -354,7 +354,7 @@ describe('BulkEditorComponent', () => {
component.openCustomFieldsDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.customFieldDocumentCounts).toEqual(
expect(component.customFieldDocumentCounts()).toEqual(
selectionData.selected_custom_fields
)
})
@@ -1,4 +1,11 @@
import { Component, inject, Input, OnDestroy, OnInit } from '@angular/core'
import {
Component,
inject,
Input,
OnDestroy,
OnInit,
signal,
} from '@angular/core'
import {
FormControl,
FormGroup,
@@ -12,15 +19,7 @@ import {
} from '@ng-bootstrap/ng-bootstrap'
import { saveAs } from 'file-saver'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import {
first,
forkJoin,
map,
Observable,
Subject,
switchMap,
takeUntil,
} from 'rxjs'
import { first, map, Observable, Subject, switchMap, takeUntil } from 'rxjs'
import { ConfirmDialogComponent } from 'src/app/components/common/confirm-dialog/confirm-dialog.component'
import { CustomField } from 'src/app/data/custom-field'
import { MatchingModel } from 'src/app/data/matching-model'
@@ -109,12 +108,12 @@ export class BulkEditorComponent
documentTypeSelectionModel = new FilterableDropdownSelectionModel()
storagePathsSelectionModel = new FilterableDropdownSelectionModel()
customFieldsSelectionModel = new FilterableDropdownSelectionModel(true)
tagDocumentCounts: SelectionDataItem[]
correspondentDocumentCounts: SelectionDataItem[]
documentTypeDocumentCounts: SelectionDataItem[]
storagePathDocumentCounts: SelectionDataItem[]
customFieldDocumentCounts: SelectionDataItem[]
awaitingDownload: boolean
readonly tagDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly correspondentDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly documentTypeDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly storagePathDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly customFieldDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly awaitingDownload = signal(false)
unsubscribeNotifier: Subject<any> = new Subject()
@@ -373,8 +372,8 @@ export class BulkEditorComponent
openTagsDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.tagDocumentCounts = selectionData?.selected_tags ?? []
this.applySelectionData(this.tagDocumentCounts, this.tagSelectionModel)
this.tagDocumentCounts.set(selectionData?.selected_tags ?? [])
this.applySelectionData(this.tagDocumentCounts(), this.tagSelectionModel)
return
}
@@ -382,7 +381,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.tagDocumentCounts = s.selected_tags
this.tagDocumentCounts.set(s.selected_tags)
this.applySelectionData(s.selected_tags, this.tagSelectionModel)
})
}
@@ -390,10 +389,11 @@ export class BulkEditorComponent
openDocumentTypeDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.documentTypeDocumentCounts =
this.documentTypeDocumentCounts.set(
selectionData?.selected_document_types ?? []
)
this.applySelectionData(
this.documentTypeDocumentCounts,
this.documentTypeDocumentCounts(),
this.documentTypeSelectionModel
)
return
@@ -403,7 +403,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.documentTypeDocumentCounts = s.selected_document_types
this.documentTypeDocumentCounts.set(s.selected_document_types)
this.applySelectionData(
s.selected_document_types,
this.documentTypeSelectionModel
@@ -414,10 +414,11 @@ export class BulkEditorComponent
openCorrespondentDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.correspondentDocumentCounts =
this.correspondentDocumentCounts.set(
selectionData?.selected_correspondents ?? []
)
this.applySelectionData(
this.correspondentDocumentCounts,
this.correspondentDocumentCounts(),
this.correspondentSelectionModel
)
return
@@ -427,7 +428,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.correspondentDocumentCounts = s.selected_correspondents
this.correspondentDocumentCounts.set(s.selected_correspondents)
this.applySelectionData(
s.selected_correspondents,
this.correspondentSelectionModel
@@ -438,10 +439,11 @@ export class BulkEditorComponent
openStoragePathDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.storagePathDocumentCounts =
this.storagePathDocumentCounts.set(
selectionData?.selected_storage_paths ?? []
)
this.applySelectionData(
this.storagePathDocumentCounts,
this.storagePathDocumentCounts(),
this.storagePathsSelectionModel
)
return
@@ -451,7 +453,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.storagePathDocumentCounts = s.selected_storage_paths
this.storagePathDocumentCounts.set(s.selected_storage_paths)
this.applySelectionData(
s.selected_storage_paths,
this.storagePathsSelectionModel
@@ -462,10 +464,11 @@ export class BulkEditorComponent
openCustomFieldsDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.customFieldDocumentCounts =
this.customFieldDocumentCounts.set(
selectionData?.selected_custom_fields ?? []
)
this.applySelectionData(
this.customFieldDocumentCounts,
this.customFieldDocumentCounts(),
this.customFieldsSelectionModel
)
return
@@ -475,7 +478,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.customFieldDocumentCounts = s.selected_custom_fields
this.customFieldDocumentCounts.set(s.selected_custom_fields)
this.applySelectionData(
s.selected_custom_fields,
this.customFieldsSelectionModel
@@ -884,7 +887,7 @@ export class BulkEditorComponent
}
downloadSelected() {
this.awaitingDownload = true
this.awaitingDownload.set(true)
let downloadFileType: string =
this.downloadForm.get('downloadFileTypeArchive').value &&
this.downloadForm.get('downloadFileTypeOriginals').value
@@ -901,7 +904,7 @@ export class BulkEditorComponent
.pipe(first())
.subscribe((result: any) => {
saveAs(result, 'documents.zip')
this.awaitingDownload = false
this.awaitingDownload.set(false)
})
}
@@ -925,27 +928,6 @@ export class BulkEditorComponent
})
}
runZoneOcrSelected() {
const ids = Array.from(this.list.selected)
if (!ids.length) return
const modal = this.modalService.open(ConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Run Zone OCR`
modal.componentInstance.messageBold = $localize`Run zone OCR on ${this.getSelectionSize()} selected document(s)?`
modal.componentInstance.message = $localize`Each document's type template (if it has one) is applied, overwriting the mapped fields.`
modal.componentInstance.btnCaption = $localize`Proceed`
modal.componentInstance.confirmClicked
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
modal.componentInstance.buttonsEnabled = false
this.executeDocumentAction(
modal,
forkJoin(ids.map((id) => this.documentService.runZoneOcr(id)))
)
})
}
setPermissions() {
let modal = this.modalService.open(PermissionsDialogComponent, {
backdrop: 'static',
@@ -2,7 +2,7 @@
<div class="row g-0">
<div class="col-md-2 doc-img-container rounded-start" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit()">
@if (document()) {
<img [src]="getThumbUrl()" class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()">
<img [ngSrc]="getThumbUrl()" fill class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()">
<div class="border-end border-bottom bg-light document-card-check">
<div class="form-check">
@@ -88,6 +88,12 @@ describe('DocumentCardLargeComponent', () => {
expect(fixture.nativeElement.textContent).toContain('8 pages')
})
it('should lazy load the thumbnail', () => {
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should trim content', () => {
expect(component.contentTrimmed).toHaveLength(503) // includes ...
})
@@ -1,4 +1,4 @@
import { AsyncPipe } from '@angular/common'
import { AsyncPipe, NgOptimizedImage } from '@angular/common'
import {
AfterViewInit,
Component,
@@ -46,6 +46,7 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
DocumentTypeNamePipe,
@@ -2,7 +2,7 @@
<div class="card h-100 shadow-sm document-card" [class.placeholder-glow]="!document()" [class.card-selected]="selected()" (mouseleave)="mouseLeaveCard()">
<div class="border-bottom doc-img-container rounded-top" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit(this)">
@if (document()) {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [src]="getThumbUrl()">
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill>
<div class="border-end border-bottom bg-light py-1 px-2 document-card-check">
<div class="form-check">
@@ -22,10 +22,14 @@
}
}
.doc-img-container {
position: relative;
height: 180px;
}
.doc-img {
object-fit: cover;
object-position: top left;
height: 180px;
}
.document-card-check {
@@ -61,6 +61,12 @@ describe('DocumentCardSmallComponent', () => {
expect(fixture.nativeElement.textContent).toContain('12 pages')
})
it('should lazy load the thumbnail', () => {
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should display a document, limit tags to 5', () => {
expect(fixture.nativeElement.textContent).toContain('Document 10')
expect(
@@ -1,4 +1,4 @@
import { AsyncPipe } from '@angular/common'
import { AsyncPipe, NgOptimizedImage } from '@angular/common'
import {
AfterViewInit,
Component,
@@ -46,6 +46,7 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
DocumentTypeNamePipe,
@@ -115,7 +115,7 @@
</pngx-page-header>
<div class="row sticky-top py-3 mt-n2 mt-md-n3 bg-body">
<pngx-filter-editor [hidden]="isBulkEditing" [disabled]="isBulkEditing" [filterRules]="list.filterRules" (filterRulesChange)="onFilterRulesChange($event)" (resetFilterRules)="onFilterRulesReset($event)" [unmodifiedFilterRules]="unmodifiedFilterRules" [selectionData]="list.selectionData" #filterEditor></pngx-filter-editor>
<pngx-filter-editor [hidden]="isBulkEditing" [disabled]="isBulkEditing" [filterRules]="list.filterRules" (filterRulesChange)="onFilterRulesChange($event)" (resetFilterRules)="onFilterRulesReset($event)" [unmodifiedFilterRules]="unmodifiedFilterRules()" [selectionData]="list.selectionData" #filterEditor></pngx-filter-editor>
<pngx-bulk-editor [hidden]="!isBulkEditing" [disabled]="!isBulkEditing"></pngx-bulk-editor>
</div>
@@ -5,6 +5,7 @@ import {
OnDestroy,
OnInit,
QueryList,
signal,
ViewChild,
ViewChildren,
} from '@angular/core'
@@ -149,7 +150,7 @@ export class DocumentListComponent
)
}
unmodifiedFilterRules: FilterRule[] = []
readonly unmodifiedFilterRules = signal<FilterRule[]>([])
private unmodifiedSavedView: SavedView
private activeSavedView: SavedView | null = null
@@ -299,7 +300,7 @@ export class DocumentListComponent
this.savedViewService.setDocumentCount(view, this.list.collectionSize)
})
this.updateDisplayCustomFields()
this.unmodifiedFilterRules = view.filter_rules
this.unmodifiedFilterRules.set(view.filter_rules)
})
this.route.queryParamMap
@@ -316,7 +317,7 @@ export class DocumentListComponent
this.activeSavedView = null
this.list.activateSavedView(null)
this.list.loadFromQueryParams(queryParams)
this.unmodifiedFilterRules = []
this.unmodifiedFilterRules.set([])
}
})
@@ -415,7 +416,7 @@ export class DocumentListComponent
this.toastService.showInfo(
$localize`View "${this.list.activeSavedViewTitle}" saved successfully.`
)
this.unmodifiedFilterRules = this.list.filterRules
this.unmodifiedFilterRules.set(this.list.filterRules)
},
error: (err) => {
this.toastService.showError(
@@ -80,7 +80,7 @@
shortcutKey="i"></pngx-filterable-dropdown>
}
@if (permissionsService.currentUserCan(PermissionAction.View, PermissionType.CustomField) && customFields.length > 0) {
@if (permissionsService.currentUserCan(PermissionAction.View, PermissionType.CustomField) && customFields().length > 0) {
<pngx-custom-fields-query-dropdown class="flex-fill fade" [class.show]="show()" title="Custom fields" icon="ui-radios" i18n-title
[(selectionModel)]="customFieldQueriesModel"
(selectionModelChange)="updateRules()"
@@ -328,7 +328,7 @@ describe('FilterEditorComponent', () => {
})
it('should ingest legacy text filter rules for doc title + content', () => {
expect(component.textFilter).toEqual(null)
expect(component.textFilter).toBeNull()
component.filterRules = [
{
rule_type: FILTER_TITLE_CONTENT,
@@ -1,5 +1,6 @@
import {
AfterViewInit,
ChangeDetectorRef,
Component,
ElementRef,
EventEmitter,
@@ -9,6 +10,7 @@ import {
Output,
ViewChild,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import {
@@ -349,7 +351,7 @@ export class FilterEditorComponent
@ViewChild('textFilterInput')
textFilterInput: ElementRef
customFields: CustomField[] = []
readonly customFields = signal<CustomField[]>([])
tagDocumentCounts: SelectionDataItem[]
correspondentDocumentCounts: SelectionDataItem[]
@@ -514,6 +516,7 @@ export class FilterEditorComponent
this.documentService.get(this._moreLikeId).subscribe((result) => {
this._moreLikeDoc = result
this._textFilter = result.title
this.changeDetector.markForCheck()
})
break
case FILTER_CREATED_AFTER:
@@ -1162,6 +1165,7 @@ export class FilterEditorComponent
private loadingCountTotal: number = 0
private loadingCount: number = 0
private readonly changeDetector = inject(ChangeDetectorRef)
private maybeCompleteLoading() {
this.loadingCount++
@@ -1229,7 +1233,7 @@ export class FilterEditorComponent
) {
this.loadingCountTotal++
this.customFieldService.listAll().subscribe((result) => {
this.customFields = result.results
this.customFields.set(result.results)
this.maybeCompleteLoading()
})
}
@@ -1,18 +1,18 @@
<div class="row mb-3">
<div class="col mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<label class="text-muted me-2 mb-0" i18n>Filter by:</label>
<input class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
<label class="text-muted me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<input id="managementNameFilter" class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
</div>
</div>
<div class="col-auto mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<div class="input-group input-group-sm w-auto d-none d-md-flex">
<span class="input-group-text border-0" i18n>Show:</span>
<label class="input-group-text border-0" for="managementPageSize" i18n>Show:</label>
</div>
<div class="input-group input-group-sm w-auto me-3">
<select class="form-select form-select-sm small" [(ngModel)]="pageSize">
<select id="managementPageSize" class="form-select form-select-sm small" [(ngModel)]="pageSize">
<option [ngValue]="25">25</option>
<option [ngValue]="50">50</option>
<option [ngValue]="100">100</option>
@@ -83,12 +83,12 @@ describe('TagListComponent', () => {
]
component['_nameFilter'] = null // Simulate empty name filter
const filtered = component.filterData(tags as any)
expect(filtered.length).toBe(2)
expect(filtered).toHaveLength(2)
expect(filtered.find((t) => t.id === 2)).toBeUndefined()
component['_nameFilter'] = 'Tag2' // Simulate non-empty name filter
const filteredWithName = component.filterData(tags as any)
expect(filteredWithName.length).toBe(2)
expect(filteredWithName).toHaveLength(2)
expect(filteredWithName.find((t) => t.id === 2)).toBeUndefined()
expect(
filteredWithName
@@ -26,7 +26,7 @@
<th scope="col" style="width: 40px;">
<div class="form-check m-0 ms-2 me-n2">
<input type="checkbox" class="form-check-input" id="all-objects" [ngModel]="toggleAllEnabled()" (ngModelChange)="toggleAllEnabled.set($event)" [disabled]="processedMails().length === 0" (click)="toggleAll($event); $event.stopPropagation();">
<label class="form-check-label" for="all-objects"></label>
<label class="form-check-label" for="all-objects"><span class="visually-hidden" i18n>Select all processed email messages</span></label>
</div>
</th>
<th scope="col" i18n>Subject</th>
@@ -5,6 +5,6 @@
pre {
white-space: pre-wrap;
word-break: break-word;
overflow-wrap: break-word;
}
}
@@ -1,34 +0,0 @@
@if (zones.length === 0) {
<p class="text-muted" i18n>
No zones defined. Load a document preview and draw rectangles to add zones.
</p>
}
<div class="list-group">
@for (zone of zones; track $index; let i = $index) {
<div
class="list-group-item list-group-item-action d-flex justify-content-between align-items-center"
[style.box-shadow]="selectedZoneIndex === i ? 'inset 3px 0 0 0 var(--bs-primary)' : null"
>
<div class="flex-grow-1" role="button" style="cursor: pointer;" (click)="zoneSelected.emit(i)">
<div>
<strong [class.text-primary]="selectedZoneIndex === i">
{{ zone.name }}
</strong>
</div>
<div class="small text-muted">
{{ getZoneTargetName(zone) }} - {{ zone.width }}x{{ zone.height }}px
<ng-container i18n>p.</ng-container>{{ zonePage(zone) }}
</div>
</div>
<div class="btn-group">
<button class="btn btn-sm btn-outline-secondary" type="button" (click)="zoneSelected.emit(i)" title="Edit" i18n-title>
<i-bs name="pencil"></i-bs>
</button>
<button class="btn btn-sm btn-outline-danger" type="button" (click)="zoneRemoved.emit(i)" title="Delete" i18n-title>
<i-bs name="trash"></i-bs>
</button>
</div>
</div>
}
</div>
@@ -1,72 +0,0 @@
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { CustomField } from 'src/app/data/custom-field'
import { OcrTemplateZone } from 'src/app/data/ocr-template'
import { OcrTemplateEditorZoneListComponent } from './ocr-template-editor-zone-list.component'
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
return {
name: 'Zone 1',
target: 'custom_field',
custom_field: 7,
x: 10,
y: 20,
width: 30,
height: 40,
page: 1,
ocr_language: 'eng',
transform: 'strip',
validation_regex: '',
order: 0,
...overrides,
}
}
describe('OcrTemplateEditorZoneListComponent', () => {
let fixture: ComponentFixture<OcrTemplateEditorZoneListComponent>
let component: OcrTemplateEditorZoneListComponent
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
OcrTemplateEditorZoneListComponent,
NgxBootstrapIconsModule.pick(allIcons),
],
}).compileComponents()
fixture = TestBed.createComponent(OcrTemplateEditorZoneListComponent)
component = fixture.componentInstance
})
it('shows empty state when no zones are defined', () => {
fixture.detectChanges()
expect(fixture.nativeElement.textContent).toContain('No zones defined')
})
it('renders zone target, size, and page', () => {
component.zones = [zone()]
component.customFields = [{ id: 7, name: 'Invoice Number' } as CustomField]
fixture.detectChanges()
const text = fixture.nativeElement.textContent
expect(text).toContain('Zone 1')
expect(text).toContain('Invoice Number')
expect(text).toContain('30x40px')
expect(text).toContain('p.1')
})
it('emits select and remove events', () => {
component.zones = [zone()]
const selectSpy = jest.spyOn(component.zoneSelected, 'emit')
const removeSpy = jest.spyOn(component.zoneRemoved, 'emit')
fixture.detectChanges()
const buttons = fixture.nativeElement.querySelectorAll('button')
buttons[0].click()
buttons[1].click()
expect(selectSpy).toHaveBeenCalledWith(0)
expect(removeSpy).toHaveBeenCalledWith(0)
})
})
@@ -1,41 +0,0 @@
import { Component, EventEmitter, Input, Output } from '@angular/core'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { CustomField } from 'src/app/data/custom-field'
import { OCR_BUILTIN_TARGETS, OcrTemplateZone } from 'src/app/data/ocr-template'
import { getZonePage } from '../zone-geometry'
@Component({
selector: 'pngx-ocr-template-zone-list',
imports: [NgxBootstrapIconsModule],
templateUrl: './ocr-template-editor-zone-list.component.html',
})
export class OcrTemplateEditorZoneListComponent {
@Input() zones: OcrTemplateZone[] = []
@Input() selectedZoneIndex: number | null = null
@Input() previewPage = 0
@Input() previewPageCount: number | null = null
@Input() customFields: CustomField[] = []
@Output() zoneSelected = new EventEmitter<number>()
@Output() zoneRemoved = new EventEmitter<number>()
zonePage(zone: OcrTemplateZone): number {
return getZonePage(zone, this.previewPage, this.previewPageCount)
}
getZoneTargetName(zone: OcrTemplateZone): string {
const target = zone.target || 'custom_field'
if (target === 'custom_field') {
return zone.custom_field
? this.getCustomFieldName(zone.custom_field)
: $localize`(no field)`
}
return OCR_BUILTIN_TARGETS.find((t) => t.id === target)?.name ?? target
}
private getCustomFieldName(id: number): string {
return (
this.customFields.find((field) => field.id === id)?.name ?? `Field #${id}`
)
}
}
@@ -1,442 +0,0 @@
<pngx-page-header [title]="pageTitle" [id]="template.id">
<div class="input-group input-group-sm me-5 align-items-center">
<div class="input-group-text">
<i-bs name="file-text"></i-bs>
</div>
<input
type="text"
class="form-control"
[(ngModel)]="previewDocModel"
[ngbTypeahead]="searchDocuments"
[inputFormatter]="documentFormatter"
[resultFormatter]="documentFormatter"
(selectItem)="onPreviewDocSelected($event)"
[editable]="false"
placeholder="Search documents by title..."
i18n-placeholder
/>
</div>
<div class="d-flex align-items-center flex-wrap gap-2">
<div class="input-group input-group-sm ms-2 d-none d-md-flex">
<div class="input-group-text" i18n>Page</div>
<input class="form-control flex-grow-0 w-auto" type="number" min="1" [max]="previewPageCount" [(ngModel)]="previewPageDisplay" />
<div class="input-group-text" i18n>of {{previewPageCount}}</div>
</div>
<button type="button" class="btn btn-sm btn-outline-secondary" i18n-title title="Previous" (click)="prevPage()" [disabled]="!pageImageUrl || previewPage <= 0">
<i-bs width="1.2em" height="1.2em" name="arrow-left"></i-bs>
</button>
<button type="button" class="btn btn-sm btn-outline-secondary" i18n-title title="Next" (click)="nextPage()" [disabled]="!pageImageUrl || previewPage >= (previewPageCount ?? 1) - 1">
<i-bs width="1.2em" height="1.2em" name="arrow-right"></i-bs>
</button>
<div class="input-group input-group-sm">
<button class="btn btn-outline-secondary" (click)="zoomOut()" i18n>-</button>
<span class="input-group-text">{{ zoom * 100 | number: '1.0-0' }}%</span>
<button class="btn btn-outline-secondary" (click)="zoomIn()" i18n>+</button>
</div>
</div>
</pngx-page-header>
<div class="row">
<div class="col-md-4">
<div class="btn-toolbar mb-1 border-bottom">
<div class="btn-group pb-3">
<a routerLink="/ocr-templates" class="btn btn-sm btn-outline-secondary">
<i-bs width="1.2em" height="1.2em" name="x"></i-bs>
<span class="ms-1" i18n>Close</span>
</a>
</div>
<div class="btn-group ms-auto pb-3">
<button class="btn btn-sm btn-primary" (click)="save()" [disabled]="saving">
@if (saving) {
<span class="spinner-border spinner-border-sm me-1"></span>
}
<span i18n>Save</span>
</button>
</div>
</div>
<ul ngbNav #nav="ngbNav" [(activeId)]="activeTab" class="nav-underline flex-nowrap flex-md-wrap overflow-auto">
<li ngbNavItem="settings">
<a ngbNavLink i18n>Settings</a>
<ng-template ngbNavContent>
<div class="row mb-3">
<div class="col-9">
<pngx-input-text [(ngModel)]="template.name" title="Template name" i18n-title></pngx-input-text>
</div>
<div class="col-3">
<pngx-input-switch [(ngModel)]="template.enabled" title="Enabled" i18n-title></pngx-input-switch>
</div>
</div>
<pngx-input-select [(ngModel)]="template.document_type" [items]="documentTypes" bindLabel="name" bindValue="id" title="Document type" i18n-title></pngx-input-select>
<small class="text-muted" i18n>
Draw rectangles on the preview to define extraction zones. Use the
page controls above the preview to add zones on different pages.
</small>
</ng-template>
</li>
<li ngbNavItem="zones">
<a ngbNavLink><ng-container i18n>Zones</ng-container> <span class="badge bg-primary ms-2">{{ template.zones.length }}</span></a>
<ng-template ngbNavContent>
<pngx-ocr-template-zone-list
[zones]="template.zones"
[selectedZoneIndex]="selectedZoneIndex"
[previewPage]="previewPage"
[previewPageCount]="previewPageCount"
[customFields]="customFields"
(zoneSelected)="selectZone($event)"
(zoneRemoved)="removeZone($event)"
></pngx-ocr-template-zone-list>
</ng-template>
</li>
<li ngbNavItem="zone">
<a ngbNavLink i18n>Zone</a>
<ng-template ngbNavContent>
@if (selectedZone; as zone) {
<div class="d-flex justify-content-between align-items-center mb-3">
<strong>{{ zone.name }}</strong>
<div class="d-flex gap-2">
<button class="btn btn-sm btn-primary" (click)="save()" [disabled]="saving">
@if (saving) {
<span class="spinner-border spinner-border-sm me-1"></span>
}
<span i18n>Save</span>
</button>
<button class="btn btn-sm btn-outline-danger" (click)="deleteSelectedZone()">
<i-bs name="trash" class="me-1"></i-bs><ng-container i18n>Delete zone</ng-container>
</button>
</div>
</div>
<div class="mb-3">
<label class="form-label" i18n>Zone Name</label>
<input
type="text"
class="form-control"
[(ngModel)]="zone.name"
/>
</div>
<div class="mb-3">
<label class="form-label" i18n>Page</label>
<input
type="number"
class="form-control"
[(ngModel)]="zone.page"
min="-1"
/>
<small class="text-muted" i18n>Page this zone is on. Use -1 for the last page. Set automatically when you draw it.</small>
</div>
<div class="mb-3">
<label class="form-label" i18n>Field</label>
<div class="input-group">
<select class="form-select" [ngModel]="zoneFieldValue(zone)" (ngModelChange)="setZoneField(zone, $event)">
<optgroup label="Built-in fields" i18n-label>
@for (t of builtinTargets; track t.id) {
<option [ngValue]="t.id">{{ t.name }}</option>
}
</optgroup>
<optgroup label="Custom fields" i18n-label>
@for (cf of customFields; track cf.id) {
<option [ngValue]="cf.id">{{ cf.name }} ({{ cf.data_type }})</option>
}
</optgroup>
</select>
<button
class="btn btn-outline-secondary"
type="button"
(click)="openQuickCreate(selectedZoneIndex)"
title="Create new custom field"
i18n-title
>
<i-bs name="plus"></i-bs>
</button>
</div>
<small class="text-muted" i18n>Write the extracted value to a custom field, or to a built-in field (Title, ASN, Date created).</small>
</div>
@if (isFieldShared(zone)) {
<div class="card mb-3 border-info">
<div class="card-body">
<h6 class="card-title d-flex align-items-center gap-2">
<i-bs name="braces"></i-bs>
<span i18n>Combine zones into this field</span>
</h6>
<p class="small text-muted mb-2" i18n>
More than one zone writes to this field. Build the combined
value below: click a zone to insert its token, and type any
separators or literal text between tokens.
</p>
<div class="d-flex flex-wrap gap-1 mb-2">
@for (z of zonesForField(zone); track $index) {
<button
type="button"
class="btn btn-sm btn-outline-info"
(click)="insertCombineToken(zone, z)"
title="Insert token"
i18n-title
>
+ {{ z.name || 'Zone' }}
</button>
}
</div>
<input
type="text"
class="form-control font-monospace"
[ngModel]="getCombineFormat(zone)"
(ngModelChange)="setCombineFormat(zone, $event)"
placeholder="{Zone 1} - {Zone 2}"
/>
<small class="text-muted" i18n>
Tokens are matched by zone name. An empty zone leaves its
token blank and the stray separator is trimmed. Leave empty
to just join the zones in order with a space.
</small>
</div>
</div>
}
@if (showQuickCreate) {
<div class="card mb-3 border-primary">
<div class="card-body">
<h6 class="card-title" i18n>Create Custom Field</h6>
<div class="mb-2">
<label class="form-label small" i18n>Field Name</label>
<input type="text" class="form-control form-control-sm"
[(ngModel)]="quickCreateName" placeholder="e.g. Invoice Number" />
</div>
<div class="mb-2">
<label class="form-label small" i18n>Field Type</label>
<select class="form-select form-select-sm" [(ngModel)]="quickCreateType">
@for (t of quickCreateTypes; track t.id) {
<option [ngValue]="t.id">{{ t.name }}</option>
}
</select>
</div>
<div class="d-flex gap-2">
<button class="btn btn-primary btn-sm" (click)="submitQuickCreate()"
[disabled]="!quickCreateName.trim()" i18n>
Create & Assign
</button>
<button class="btn btn-outline-secondary btn-sm" (click)="cancelQuickCreate()" i18n>
Cancel
</button>
</div>
</div>
</div>
}
<div class="mb-3">
<label class="form-label" i18n>OCR Language</label>
<ng-select
[items]="ocrLanguageOptions"
bindLabel="name"
bindValue="id"
[multiple]="true"
[closeOnSelect]="false"
[ngModel]="ocrLanguageArray(zone)"
(ngModelChange)="setOcrLanguages(zone, $event)"
placeholder="Select languages"
i18n-placeholder
></ng-select>
</div>
<div class="mb-3">
<label class="form-label" i18n>Transform</label>
<select class="form-select" [(ngModel)]="zone.transform">
@for (opt of transformOptions; track opt.id) {
<option [ngValue]="opt.id">{{ opt.name }}</option>
}
</select>
</div>
@if (zone.transform === dateTransform) {
<div class="mb-3">
<label class="form-label" i18n>Date format</label>
<select class="form-select" [ngModel]="dateFormatChoice(zone)" (ngModelChange)="setDateFormatChoice(zone, $event)">
@for (opt of dateFormatOptions; track opt.id) {
<option [ngValue]="opt.id">{{ opt.name }}</option>
}
<option [ngValue]="customDateFormatChoice" i18n>Custom...</option>
</select>
@if (usesCustomDateFormat(zone)) {
<div class="input-group mt-2">
<input type="text" class="form-control font-monospace" [(ngModel)]="zone.date_format" placeholder="%d.%m.%Y" />
<button class="btn btn-outline-secondary" type="button" [ngbPopover]="dateFmtHelp" [autoClose]="true" title="Date format help" i18n-title>
<i-bs name="question-circle"></i-bs>
</button>
</div>
<ng-template #dateFmtHelp>
<p class="mb-1" i18n>Python date codes:</p>
<ul class="mb-1 ps-3">
<li><code>%d</code> <ng-container i18n>day (01-31)</ng-container></li>
<li><code>%m</code> <ng-container i18n>month (01-12)</ng-container></li>
<li><code>%Y</code> <ng-container i18n>year, 4-digit</ng-container></li>
<li><code>%y</code> <ng-container i18n>year, 2-digit</ng-container></li>
<li><code>%b</code> <ng-container i18n>month name (Jan)</ng-container></li>
</ul>
<span i18n>Example:</span> <code>%d.%m.%Y</code> -> 03.03.2026
</ng-template>
}
</div>
}
<div class="mb-3">
<label class="form-label" i18n>Validation Regex</label>
<input
type="text"
class="form-control font-monospace"
[(ngModel)]="zone.validation_regex"
placeholder="e.g. \d{2}\.\d{2}\.\d{4}"
>
</div>
<div class="text-muted small">
{{ zone.x }}, {{ zone.y }} - {{ zone.width }}x{{ zone.height }}px
</div>
<hr class="my-3" />
<h6 i18n>Test</h6>
@if (!previewDocId) {
<p class="text-muted small mb-0" i18n>
Load a document in the Settings tab to test this zone.
</p>
} @else {
<button class="btn btn-sm btn-outline-secondary" (click)="testZone()" [disabled]="zoneTesting">
@if (zoneTesting) {
<span class="spinner-border spinner-border-sm me-1"></span>
}
<span i18n>Test this zone</span>
</button>
@if (zoneTestResult) {
@if (zoneTestResult.error) {
<div class="alert alert-warning py-2 mt-2 mb-0 small">{{ zoneTestResult.error }}</div>
} @else {
<dl class="row small mt-2 mb-0">
<dt class="col-sm-4" i18n>OCR text</dt>
<dd class="col-sm-8"><code>{{ zoneTestResult.raw_text || '(nothing detected)' }}</code></dd>
<dt class="col-sm-4" i18n>Value</dt>
<dd class="col-sm-8"><code>{{ zoneTestResult.value || '(empty)' }}</code></dd>
@if (zoneTestResult.regex) {
<dt class="col-sm-4" i18n>Validation</dt>
<dd class="col-sm-8">
@if (zoneTestResult.regex_match) {
<span class="badge bg-success" i18n>Regex matches</span>
} @else {
<span class="badge bg-danger" i18n>Regex does not match</span>
}
</dd>
}
</dl>
}
}
}
} @else {
<p class="text-muted" i18n>
Select a zone from the Zones tab, or draw a rectangle on the document to create one.
</p>
}
</ng-template>
</li>
</ul>
<div [ngbNavOutlet]="nav" class="mt-3"></div>
</div>
<!-- Right column: Document preview with zone overlay -->
<div class="col-md-8">
@if (pageImageUrl) {
<div class="zone-preview-scroll border">
<div class="zone-preview-stage" [style.width.%]="zoom * 100">
<img
#pageImage
[src]="pageImageUrl"
(load)="onImageLoad()"
class="zone-preview-image"
[style.visibility]="imageLoaded ? 'visible' : 'hidden'"
crossorigin="use-credentials"
/>
@if (imageLoaded) {
<svg
#zoneOverlay
class="zone-overlay"
[attr.viewBox]="overlayViewBox()"
preserveAspectRatio="none"
[style.cursor]="overlayCursor"
(mousedown)="onOverlayMouseDown($event)"
(mousemove)="onOverlayMouseMove($event)"
(mouseup)="onOverlayMouseUp($event)"
>
@for (zone of template.zones; track $index; let i = $index) {
@if (zoneDisplayRect(i); as rect) {
<g>
<rect
class="zone-rect"
[class.zone-rect-selected]="selectedZoneIndex === i"
[attr.x]="rect.x"
[attr.y]="rect.y"
[attr.width]="rect.w"
[attr.height]="rect.h"
[attr.stroke]="zoneColor(i)"
[attr.fill]="zoneFill(i)"
></rect>
<text
class="zone-label"
[attr.x]="rect.x + overlayUnitSize(6)"
[attr.y]="zoneLabelY(rect)"
[attr.font-size]="overlayFontSize()"
[attr.fill]="zoneColor(i)"
>{{ zoneLabel(zone, i) }}</text>
@if (selectedZoneIndex === i) {
@for (handle of resizeHandles(rect); track handle.handle) {
<rect
class="zone-resize-handle"
[attr.x]="handle.x - overlayHandleSize() / 2"
[attr.y]="handle.y - overlayHandleSize() / 2"
[attr.width]="overlayHandleSize()"
[attr.height]="overlayHandleSize()"
[attr.fill]="zoneColor(i)"
></rect>
}
}
</g>
}
}
@if (drawingRect(); as rect) {
<rect
class="zone-drawing-rect"
[attr.x]="rect.x"
[attr.y]="rect.y"
[attr.width]="rect.w"
[attr.height]="rect.h"
></rect>
}
</svg>
}
@if (!imageLoaded) {
<div class="d-flex justify-content-center p-5">
<div class="spinner-border" role="status">
<span class="visually-hidden" i18n>Loading page...</span>
</div>
</div>
}
</div>
</div>
} @else {
<div class="border rounded p-5 text-center text-muted">
<i-bs name="file-earmark-image" width="48" height="48"></i-bs>
<p class="mt-3" i18n>
Enter a document ID and click "Load" to preview a page and draw extraction zones.
</p>
</div>
}
</div>
</div>
@@ -1,63 +0,0 @@
:host {
display: block;
}
.zone-preview-scroll {
max-height: 78vh;
overflow: auto;
}
.zone-preview-stage {
display: inline-block;
position: relative;
}
.zone-preview-image {
display: block;
width: 100%;
}
.zone-overlay {
height: 100%;
inset: 0;
position: absolute;
touch-action: none;
width: 100%;
}
.zone-rect,
.zone-drawing-rect {
vector-effect: non-scaling-stroke;
}
.zone-rect {
stroke-width: 2;
}
.zone-rect-selected {
stroke-width: 3;
}
.zone-label {
font-family: var(--bs-font-sans-serif);
font-weight: 600;
paint-order: stroke;
pointer-events: none;
stroke: #fff;
stroke-linejoin: round;
stroke-width: 4px;
vector-effect: non-scaling-stroke;
}
.zone-resize-handle {
stroke: #fff;
stroke-width: 1;
vector-effect: non-scaling-stroke;
}
.zone-drawing-rect {
fill: rgba(105, 219, 124, 0.25);
stroke: #69db7c;
stroke-dasharray: 5 5;
stroke-width: 2;
}
@@ -1,962 +0,0 @@
import { CommonModule } from '@angular/common'
import {
Component,
ElementRef,
HostListener,
inject,
OnDestroy,
OnInit,
ViewChild,
} from '@angular/core'
import { FormsModule } from '@angular/forms'
import { ActivatedRoute, Router, RouterModule } from '@angular/router'
import {
NgbNavModule,
NgbPopoverModule,
NgbTypeaheadModule,
NgbTypeaheadSelectItemEvent,
} from '@ng-bootstrap/ng-bootstrap'
import { NgSelectModule } from '@ng-select/ng-select'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import {
catchError,
debounceTime,
distinctUntilChanged,
map,
Observable,
of,
Subject,
switchMap,
takeUntil,
} from 'rxjs'
import { SelectComponent } from 'src/app/components/common/input/select/select.component'
import { SwitchComponent } from 'src/app/components/common/input/switch/switch.component'
import { TextComponent } from 'src/app/components/common/input/text/text.component'
import { PageHeaderComponent } from 'src/app/components/common/page-header/page-header.component'
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
import { Document } from 'src/app/data/document'
import { DocumentType } from 'src/app/data/document-type'
import {
DATE_FORMAT_OPTIONS,
DEFAULT_OCR_ZONE_LANGUAGE,
DEFAULT_OCR_ZONE_TARGET,
DEFAULT_OCR_ZONE_TRANSFORM,
isOcrBuiltinTarget,
OCR_BUILTIN_TARGETS,
OCR_LANGUAGE_OPTIONS,
OCR_ZONE_TARGET,
OCR_ZONE_TRANSFORM,
OcrBuiltinTarget,
OcrTemplate,
OcrTemplateZone,
OcrZoneTestResult,
TRANSFORM_OPTIONS,
ZoneTestRequest,
} from 'src/app/data/ocr-template'
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
import { DocumentTypeService } from 'src/app/services/rest/document-type.service'
import { DocumentService } from 'src/app/services/rest/document.service'
import { OcrTemplateService } from 'src/app/services/rest/ocr-template.service'
import { ToastService } from 'src/app/services/toast.service'
import { OcrTemplateEditorZoneListComponent } from './ocr-template-editor-zone-list/ocr-template-editor-zone-list.component'
import {
DisplayRect,
DrawingRect,
findHandleAt,
findZoneAt,
getZoneDisplayRect,
getZonePage,
HANDLE_SIZE,
isZoneOnPage,
MoveStart,
moveZone,
Point,
ResizeHandle,
resizeZone,
} from './zone-geometry'
type ActiveTab = 'settings' | 'zones' | 'zone'
type ZoneFieldSelection = OcrBuiltinTarget | number | null
type OverlayInteraction =
| { kind: 'idle' }
| { kind: 'drawing'; rect: DrawingRect }
| { kind: 'moving'; zoneIndex: number; start: MoveStart }
| { kind: 'resizing'; zoneIndex: number; handle: ResizeHandle }
interface ResizeHandleMarker extends Point {
handle: ResizeHandle
}
const CUSTOM_DATE_FORMAT_CHOICE = 'custom'
const MIN_DRAWN_ZONE_SIZE = 10
const NO_OVERLAY_INTERACTION: OverlayInteraction = { kind: 'idle' }
const ZONE_COLORS = [
'#4f8ff7',
'#ff6b6b',
'#51cf66',
'#ffd43b',
'#cc5de8',
'#ff922b',
'#20c997',
'#e599f7',
]
const RESIZE_CURSOR: Record<ResizeHandle, string> = {
nw: 'nw-resize',
ne: 'ne-resize',
sw: 'sw-resize',
se: 'se-resize',
n: 'n-resize',
s: 's-resize',
w: 'w-resize',
e: 'e-resize',
}
@Component({
selector: 'pngx-ocr-template-editor',
standalone: true,
imports: [
PageHeaderComponent,
TextComponent,
SelectComponent,
SwitchComponent,
CommonModule,
FormsModule,
RouterModule,
NgbNavModule,
NgbPopoverModule,
NgbTypeaheadModule,
NgSelectModule,
NgxBootstrapIconsModule,
OcrTemplateEditorZoneListComponent,
],
templateUrl: './ocr-template-editor.component.html',
styleUrls: ['./ocr-template-editor.component.scss'],
})
export class OcrTemplateEditorComponent implements OnInit, OnDestroy {
private readonly route = inject(ActivatedRoute)
private readonly router = inject(Router)
private readonly templateService = inject(OcrTemplateService)
private readonly customFieldsService = inject(CustomFieldsService)
private readonly documentTypeService = inject(DocumentTypeService)
private readonly correspondentService = inject(CorrespondentService)
private readonly documentService = inject(DocumentService)
private readonly toastService = inject(ToastService)
private readonly destroy$ = new Subject<void>()
private readonly customDateFormatZones = new WeakSet<OcrTemplateZone>()
@ViewChild('zoneOverlay') overlayRef: ElementRef<SVGSVGElement>
@ViewChild('pageImage') imageRef: ElementRef<HTMLImageElement>
template: OcrTemplate = {
id: null,
name: '',
document_type: null,
sample_document: null,
source_width: 0,
source_height: 0,
enabled: true,
combine_formats: {},
zones: [],
}
customFields: CustomField[] = []
documentTypes: DocumentType[] = []
transformOptions = TRANSFORM_OPTIONS
builtinTargets = OCR_BUILTIN_TARGETS
dateFormatOptions = DATE_FORMAT_OPTIONS
ocrLanguageOptions = OCR_LANGUAGE_OPTIONS
dateTransform = OCR_ZONE_TRANSFORM.Date
customDateFormatChoice = CUSTOM_DATE_FORMAT_CHOICE
isNew = true
saving = false
previewDocId: number | null = null
previewPage = 0
previewPageCount: number | null = null
private pageCountForDoc: number | null = null
pageImageUrl: string | null = null
imageLoaded = false
zoom = 1
previewDocModel: Document | string = ''
private correspondentNames = new Map<number, string>()
public get previewPageDisplay(): number {
return this.previewPage + 1
}
public set previewPageDisplay(value: number) {
this.goToPage(value - 1)
}
activeTab: ActiveTab = 'settings'
selectedZoneIndex: number | null = null
private overlayInteraction: OverlayInteraction = NO_OVERLAY_INTERACTION
overlayCursor = 'crosshair'
zoneTestResult: OcrZoneTestResult | null = null
zoneTesting = false
showQuickCreate = false
quickCreateName = ''
quickCreateType = CustomFieldDataType.String
quickCreateForZoneIndex: number | null = null
quickCreateTypes = [
{ id: CustomFieldDataType.String, name: $localize`String` },
{ id: CustomFieldDataType.Integer, name: $localize`Integer` },
{ id: CustomFieldDataType.Float, name: $localize`Float` },
{ id: CustomFieldDataType.Date, name: $localize`Date` },
{ id: CustomFieldDataType.Monetary, name: $localize`Monetary` },
{ id: CustomFieldDataType.Boolean, name: $localize`Boolean` },
{ id: CustomFieldDataType.Url, name: $localize`URL` },
{ id: CustomFieldDataType.LongText, name: $localize`Long Text` },
]
get selectedZone(): OcrTemplateZone | null {
return this.selectedZoneIndex !== null
? (this.template.zones[this.selectedZoneIndex] ?? null)
: null
}
get pageTitle(): string {
return this.isNew
? $localize`New OCR Template`
: $localize`Edit OCR Template`
}
ngOnInit() {
this.customFieldsService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => (this.customFields = r.results))
this.documentTypeService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => (this.documentTypes = r.results))
this.correspondentService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => {
this.correspondentNames = new Map(r.results.map((c) => [c.id, c.name]))
})
const id = this.route.snapshot.paramMap.get('id')
if (id && id !== 'new') {
this.isNew = false
this.templateService
.get(parseInt(id))
.pipe(takeUntil(this.destroy$))
.subscribe((t) => {
this.template = t
this.template.combine_formats ??= {}
if (t.sample_document) {
this.previewDocId = t.sample_document
this.loadPreview()
}
})
} else {
const qp = this.route.snapshot.queryParams
if (qp['document_type']) {
this.template.document_type = parseInt(qp['document_type'])
}
if (qp['sample_document']) {
const docId = parseInt(qp['sample_document'])
this.template.sample_document = docId
this.previewDocId = docId
this.loadPreview()
}
}
}
searchDocuments = (text$: Observable<string>): Observable<Document[]> =>
text$.pipe(
debounceTime(250),
distinctUntilChanged(),
switchMap((term) => {
if (!term || term.trim().length < 2) return of([])
const params: { title__icontains: string; document_type__id?: number } =
{ title__icontains: term.trim() }
if (this.template.document_type) {
params['document_type__id'] = this.template.document_type
}
return this.documentService.list(1, 10, 'created', true, params).pipe(
map((r) => r.results),
catchError(() => of([]))
)
})
)
documentFormatter = (doc: Document | string): string => {
if (typeof doc === 'string') return doc
const corr = doc.correspondent
? this.correspondentNames.get(doc.correspondent)
: null
return corr
? `#${doc.id} ${doc.title} (${corr})`
: `#${doc.id} ${doc.title}`
}
onPreviewDocSelected(event: NgbTypeaheadSelectItemEvent<Document>) {
event.preventDefault()
const doc: Document = event.item
this.previewDocModel = doc
this.previewDocId = doc.id
if (!this.template.document_type && doc.document_type) {
this.template.document_type = doc.document_type
}
this.previewPage = 0
this.loadPreview()
}
clearPreviewDoc() {
this.previewDocModel = ''
this.previewDocId = null
this.previewPageCount = null
this.pageCountForDoc = null
this.previewPage = 0
this.pageImageUrl = null
this.imageLoaded = false
}
loadPreview() {
if (!this.previewDocId) return
if (this.pageCountForDoc !== this.previewDocId) {
this.pageCountForDoc = this.previewDocId
this.previewPageCount = null
this.documentService
.get(this.previewDocId)
.pipe(takeUntil(this.destroy$))
.subscribe({
next: (doc) => {
this.previewPageCount = doc?.page_count ?? null
if (doc && !this.previewDocModel) this.previewDocModel = doc
},
error: () => (this.previewPageCount = null),
})
}
this.pageImageUrl = this.templateService.getPageImageUrl(
this.previewDocId,
this.previewPage
)
this.imageLoaded = false
}
goToPage(page: number) {
if (!Number.isFinite(page)) return
const max = this.previewPageCount ? this.previewPageCount - 1 : page
const clamped = Math.max(0, Math.min(page, max))
if (clamped === this.previewPage) return
this.previewPage = clamped
this.loadPreview()
}
prevPage() {
this.goToPage(this.previewPage - 1)
}
nextPage() {
this.goToPage(this.previewPage + 1)
}
zoomIn() {
this.zoom = Math.min(4, Math.round((this.zoom + 0.25) * 100) / 100)
}
zoomOut() {
this.zoom = Math.max(0.5, Math.round((this.zoom - 0.25) * 100) / 100)
}
resetZoom() {
this.zoom = 1
}
zonePage(zone: OcrTemplateZone): number {
return getZonePage(zone, this.previewPage, this.previewPageCount)
}
private isOnCurrentPage(zone: OcrTemplateZone): boolean {
return isZoneOnPage(zone, this.previewPage, this.previewPageCount)
}
onImageLoad() {
this.imageLoaded = true
const img = this.imageRef.nativeElement
this.template.source_width = img.naturalWidth
this.template.source_height = img.naturalHeight
}
onOverlayMouseDown(event: MouseEvent) {
const point = this.svgPointFromEvent(event)
if (!point) return
event.preventDefault()
if (this.selectedZoneIndex !== null) {
const handle = this.findHandleAt(point, this.selectedZoneIndex)
if (handle) {
this.overlayInteraction = {
kind: 'resizing',
zoneIndex: this.selectedZoneIndex,
handle,
}
return
}
}
const clickedIdx = this.findZoneAt(point)
if (clickedIdx !== null && !event.shiftKey) {
this.selectZone(clickedIdx)
const zone = this.template.zones[clickedIdx]
this.overlayInteraction = {
kind: 'moving',
zoneIndex: clickedIdx,
start: {
mouseX: point.x,
mouseY: point.y,
zoneX: zone.x,
zoneY: zone.y,
},
}
return
}
// Shift+click or click on empty area starts a new zone.
this.overlayInteraction = {
kind: 'drawing',
rect: {
startX: point.x,
startY: point.y,
endX: point.x,
endY: point.y,
},
}
this.selectedZoneIndex = null
}
onOverlayMouseMove(event: MouseEvent) {
const point = this.svgPointFromEvent(event)
if (!point) return
if (this.overlayInteraction.kind === 'resizing') {
this.applyResize(
this.overlayInteraction.zoneIndex,
this.overlayInteraction.handle,
point
)
return
}
if (this.overlayInteraction.kind === 'moving') {
moveZone(
this.template.zones[this.overlayInteraction.zoneIndex],
point,
this.overlayInteraction.start,
this.imageNaturalSize(),
this.imageNaturalSize()
)
return
}
if (this.overlayInteraction.kind === 'drawing') {
this.overlayInteraction.rect.endX = point.x
this.overlayInteraction.rect.endY = point.y
return
}
this.updateOverlayCursor(point)
}
private updateOverlayCursor(point: Point) {
if (this.selectedZoneIndex !== null) {
const handle = this.findHandleAt(point, this.selectedZoneIndex)
if (handle) {
this.overlayCursor = RESIZE_CURSOR[handle] || 'crosshair'
return
}
}
this.overlayCursor = this.findZoneAt(point) !== null ? 'move' : 'crosshair'
}
onOverlayMouseUp(_event: MouseEvent) {
if (
this.overlayInteraction.kind === 'moving' ||
this.overlayInteraction.kind === 'resizing'
) {
this.stopOverlayInteraction()
return
}
if (this.overlayInteraction.kind !== 'drawing') return
const drawingRect = this.overlayInteraction.rect
this.stopOverlayInteraction()
const rect = this.sourceRectFromDrawing(drawingRect)
// Ignore tiny accidental clicks.
if (rect.w < MIN_DRAWN_ZONE_SIZE || rect.h < MIN_DRAWN_ZONE_SIZE) {
return
}
this.template.zones.push(this.createZoneFromRect(rect))
this.selectZone(this.template.zones.length - 1)
}
private createZoneFromRect(rect: DisplayRect): OcrTemplateZone {
const imageSize = this.imageNaturalSize()
return {
name: `Zone ${this.template.zones.length + 1}`,
target: DEFAULT_OCR_ZONE_TARGET,
custom_field: this.defaultCustomFieldId(),
x: rect.x,
y: rect.y,
width: rect.w,
height: rect.h,
page: this.previewPageDisplay,
ocr_language: DEFAULT_OCR_ZONE_LANGUAGE,
transform: DEFAULT_OCR_ZONE_TRANSFORM,
date_format: '',
validation_regex: '',
order: this.template.zones.length,
zone_source_width: imageSize.width,
zone_source_height: imageSize.height,
}
}
private defaultCustomFieldId(): number | null {
return this.customFields[0]?.id ?? null
}
@HostListener('document:mouseup')
onDocumentMouseUp() {
if (this.overlayInteraction.kind === 'idle') return
this.stopOverlayInteraction()
}
private stopOverlayInteraction() {
this.overlayInteraction = NO_OVERLAY_INTERACTION
this.overlayCursor = 'crosshair'
}
drawingRect(): DisplayRect | null {
return this.overlayInteraction.kind === 'drawing'
? this.displayRectFromDrawing(this.overlayInteraction.rect)
: null
}
zoneDisplayRect(zoneIdx: number): DisplayRect | null {
const img = this.imageRef?.nativeElement
if (!img || !img.naturalWidth) return null
const zone = this.template.zones[zoneIdx]
if (!zone) return null
if (!this.isOnCurrentPage(zone)) return null
return getZoneDisplayRect(
zone,
this.imageNaturalSize(),
this.imageNaturalSize()
)
}
private findHandleAt(point: Point, zoneIdx: number): ResizeHandle | null {
const r = this.zoneDisplayRect(zoneIdx)
if (!r) return null
return findHandleAt(point, r, this.overlayHandleSize())
}
private applyResize(zoneIndex: number, handle: ResizeHandle, point: Point) {
const zone = this.template.zones[zoneIndex]
if (!zone) return
resizeZone(
zone,
handle,
point,
this.imageNaturalSize(),
this.imageNaturalSize()
)
}
private findZoneAt(point: Point): number | null {
const img = this.imageRef.nativeElement
if (!img.naturalWidth) return null
return findZoneAt(
point,
this.template.zones,
this.previewPage,
this.previewPageCount,
this.imageNaturalSize(),
this.imageNaturalSize()
)
}
overlayViewBox(): string {
const imageSize = this.imageNaturalSize()
return `0 0 ${imageSize.width} ${imageSize.height}`
}
zoneColor(index: number): string {
return ZONE_COLORS[index % ZONE_COLORS.length]
}
zoneFill(index: number): string {
return `${this.zoneColor(index)}33`
}
zoneLabel(zone: OcrTemplateZone, index: number): string {
return zone.name || `Zone ${index + 1}`
}
zoneLabelY(rect: DisplayRect): number {
return Math.max(this.overlayUnitSize(14), rect.y - this.overlayUnitSize(4))
}
resizeHandles(rect: DisplayRect): ResizeHandleMarker[] {
return [
{ handle: 'nw', x: rect.x, y: rect.y },
{ handle: 'n', x: rect.x + rect.w / 2, y: rect.y },
{ handle: 'ne', x: rect.x + rect.w, y: rect.y },
{ handle: 'w', x: rect.x, y: rect.y + rect.h / 2 },
{ handle: 'e', x: rect.x + rect.w, y: rect.y + rect.h / 2 },
{ handle: 'sw', x: rect.x, y: rect.y + rect.h },
{ handle: 's', x: rect.x + rect.w / 2, y: rect.y + rect.h },
{ handle: 'se', x: rect.x + rect.w, y: rect.y + rect.h },
]
}
overlayHandleSize(): number {
return this.overlayUnitSize(HANDLE_SIZE)
}
overlayFontSize(): number {
return this.overlayUnitSize(12)
}
overlayUnitSize(screenPixels: number): number {
const img = this.imageRef?.nativeElement
if (!img?.naturalWidth || !img.clientWidth) return screenPixels
return (screenPixels * img.naturalWidth) / img.clientWidth
}
private svgPointFromEvent(event: MouseEvent): Point | null {
const svg = this.overlayRef?.nativeElement
const matrix = svg?.getScreenCTM()
if (!svg || !matrix) return null
const point = svg.createSVGPoint()
point.x = event.clientX
point.y = event.clientY
const svgPoint = point.matrixTransform(matrix.inverse())
return { x: svgPoint.x, y: svgPoint.y }
}
private displayRectFromDrawing(rect: DrawingRect): DisplayRect {
return {
x: Math.min(rect.startX, rect.endX),
y: Math.min(rect.startY, rect.endY),
w: Math.abs(rect.endX - rect.startX),
h: Math.abs(rect.endY - rect.startY),
}
}
private sourceRectFromDrawing(rect: DrawingRect): DisplayRect {
const displayRect = this.displayRectFromDrawing(rect)
return {
x: Math.round(displayRect.x),
y: Math.round(displayRect.y),
w: Math.round(displayRect.w),
h: Math.round(displayRect.h),
}
}
private imageNaturalSize() {
const img = this.imageRef.nativeElement
return { width: img.naturalWidth, height: img.naturalHeight }
}
removeZone(index: number) {
this.template.zones.splice(index, 1)
if (this.selectedZoneIndex === index) {
this.selectedZoneIndex = null
} else if (this.selectedZoneIndex > index) {
this.selectedZoneIndex--
}
}
selectZone(index: number) {
this.selectedZoneIndex = index
this.activeTab = 'zone'
this.zoneTestResult = null
const zone = this.template.zones[index]
if (zone) {
this.seedCombineDefault(zone)
this.goToPage(this.zonePage(zone) - 1)
}
}
testZone() {
const zone = this.selectedZone
if (!zone || !this.previewDocId) return
this.zoneTesting = true
this.zoneTestResult = null
this.templateService
.testZone(this.previewDocId, this.zoneTestRequest(zone))
.pipe(takeUntil(this.destroy$))
.subscribe({
next: (res) => {
this.zoneTestResult = res
this.zoneTesting = false
},
error: (err) => {
this.zoneTestResult = {
error: err.error?.error || $localize`Test failed`,
}
this.zoneTesting = false
},
})
}
private zoneTestRequest(zone: OcrTemplateZone): ZoneTestRequest {
return {
name: zone.name,
x: zone.x,
y: zone.y,
width: zone.width,
height: zone.height,
page: zone.page ?? 1,
ocr_language: zone.ocr_language,
transform: zone.transform,
date_format: zone.date_format,
validation_regex: zone.validation_regex,
zone_source_width: zone.zone_source_width,
zone_source_height: zone.zone_source_height,
}
}
deleteSelectedZone() {
if (this.selectedZoneIndex === null) return
this.removeZone(this.selectedZoneIndex)
this.activeTab = 'zones'
}
save() {
this.saving = true
this.pruneCombineFormats()
this.template.sample_document = this.previewDocId
const obs = this.isNew
? this.templateService.create(this.template)
: this.templateService.update(this.template)
obs.pipe(takeUntil(this.destroy$)).subscribe({
next: (saved) => {
const idx = this.selectedZoneIndex
this.template = saved
this.isNew = false
this.selectedZoneIndex = idx
this.saving = false
this.toastService.showInfo($localize`OCR template saved.`)
},
error: (e) => {
this.saving = false
this.toastService.showError($localize`Error saving OCR template.`, e)
},
})
}
private ocrLangCache = new WeakMap<
OcrTemplateZone,
{ src: string; arr: string[] }
>()
ocrLanguageArray(zone: OcrTemplateZone): string[] {
const src = zone.ocr_language || ''
const cached = this.ocrLangCache.get(zone)
if (cached && cached.src === src) return cached.arr
const arr = src ? src.split('+').filter(Boolean) : []
this.ocrLangCache.set(zone, { src, arr })
return arr
}
setOcrLanguages(zone: OcrTemplateZone, langs: string[]) {
zone.ocr_language = (langs || []).join('+')
this.ocrLangCache.set(zone, {
src: zone.ocr_language,
arr: langs ? [...langs] : [],
})
}
getCustomFieldName(id: number): string {
const cf = this.customFields.find((f) => f.id === id)
return cf ? cf.name : `Field #${id}`
}
/** Value bound to the field select: a built-in id string or a custom-field id. */
zoneFieldValue(zone: OcrTemplateZone): ZoneFieldSelection {
const target = zone.target || DEFAULT_OCR_ZONE_TARGET
return target === OCR_ZONE_TARGET.CustomField ? zone.custom_field : target
}
setZoneField(zone: OcrTemplateZone, value: ZoneFieldSelection) {
if (isOcrBuiltinTarget(value)) {
zone.target = value
zone.custom_field = null
} else {
zone.target = OCR_ZONE_TARGET.CustomField
zone.custom_field = typeof value === 'number' ? value : null
}
this.seedCombineDefault(zone)
}
fieldKeyFor(zone: OcrTemplateZone): string | null {
const v = this.zoneFieldValue(zone)
return v === null || v === undefined ? null : String(v)
}
zonesForField(zone: OcrTemplateZone): OcrTemplateZone[] {
const key = this.fieldKeyFor(zone)
if (!key) return []
return this.template.zones.filter((z) => this.fieldKeyFor(z) === key)
}
isFieldShared(zone: OcrTemplateZone): boolean {
return this.zonesForField(zone).length > 1
}
getCombineFormat(zone: OcrTemplateZone): string {
const key = this.fieldKeyFor(zone)
return (key && this.template.combine_formats?.[key]) || ''
}
setCombineFormat(zone: OcrTemplateZone, value: string) {
const key = this.fieldKeyFor(zone)
if (!key) return
this.template.combine_formats ??= {}
this.template.combine_formats[key] = value
}
insertCombineToken(zone: OcrTemplateZone, tokenZone: OcrTemplateZone) {
const token = `{${tokenZone.name}}`
const current = this.getCombineFormat(zone)
const sep = current && !current.endsWith(' ') ? ' ' : ''
this.setCombineFormat(zone, `${current}${sep}${token}`)
}
private seedCombineDefault(zone: OcrTemplateZone) {
const key = this.fieldKeyFor(zone)
if (!key) return
const shared = this.zonesForField(zone)
if (shared.length <= 1) return
this.template.combine_formats ??= {}
if (!this.template.combine_formats[key]) {
this.template.combine_formats[key] = shared
.map((z) => `{${z.name}}`)
.join(' ')
}
}
private pruneCombineFormats() {
const formats = this.template.combine_formats
if (!formats) return
const counts = new Map<string, number>()
for (const z of this.template.zones) {
const key = this.fieldKeyFor(z)
if (key) counts.set(key, (counts.get(key) ?? 0) + 1)
}
for (const key of Object.keys(formats)) {
if ((counts.get(key) ?? 0) <= 1) delete formats[key]
}
}
/** Value bound to the date-format select: a preset, '' (auto), or 'custom'. */
dateFormatChoice(zone: OcrTemplateZone): string {
return this.usesCustomDateFormat(zone)
? CUSTOM_DATE_FORMAT_CHOICE
: zone.date_format || ''
}
setDateFormatChoice(zone: OcrTemplateZone, value: string) {
if (value === CUSTOM_DATE_FORMAT_CHOICE) {
this.customDateFormatZones.add(zone)
zone.date_format ||= ''
} else {
this.customDateFormatZones.delete(zone)
zone.date_format = value
}
}
usesCustomDateFormat(zone: OcrTemplateZone): boolean {
return (
this.customDateFormatZones.has(zone) ||
(!!zone.date_format &&
!this.dateFormatOptions.some(
(option) => option.id === zone.date_format
))
)
}
getZoneTargetName(zone: OcrTemplateZone): string {
const target = zone.target || DEFAULT_OCR_ZONE_TARGET
if (target === OCR_ZONE_TARGET.CustomField) {
return zone.custom_field
? this.getCustomFieldName(zone.custom_field)
: $localize`(no field)`
}
return this.builtinTargets.find((t) => t.id === target)?.name ?? target
}
getDocumentTypeName(id: number): string {
const dt = this.documentTypes.find((d) => d.id === id)
return dt ? dt.name : `Type #${id}`
}
openQuickCreate(zoneIndex: number | null) {
if (zoneIndex === null) return
this.quickCreateForZoneIndex = zoneIndex
this.quickCreateName = this.template.zones[zoneIndex]?.name || ''
this.quickCreateType = CustomFieldDataType.String
this.showQuickCreate = true
}
cancelQuickCreate() {
this.showQuickCreate = false
this.quickCreateForZoneIndex = null
}
submitQuickCreate() {
if (!this.quickCreateName.trim()) return
this.templateService
.quickCreateField(this.quickCreateName.trim(), this.quickCreateType)
.pipe(takeUntil(this.destroy$))
.subscribe({
next: (result) => {
this.customFieldsService.clearCache()
this.customFieldsService
.listAll()
.pipe(takeUntil(this.destroy$))
.subscribe((r) => {
this.customFields = r.results
if (this.quickCreateForZoneIndex !== null) {
this.template.zones[this.quickCreateForZoneIndex].custom_field =
result.id
this.template.zones[this.quickCreateForZoneIndex].target =
OCR_ZONE_TARGET.CustomField
}
this.showQuickCreate = false
this.quickCreateForZoneIndex = null
})
},
error: (err) => {
this.toastService.showError(
$localize`Failed to create custom field.`,
err
)
},
})
}
ngOnDestroy() {
this.destroy$.next()
this.destroy$.complete()
}
}
@@ -1,140 +0,0 @@
import { OcrTemplateZone } from 'src/app/data/ocr-template'
import {
findHandleAt,
findZoneAt,
getZoneDisplayRect,
getZonePage,
isZoneOnPage,
moveZone,
resizeZone,
sourceRectFromDrawing,
} from './zone-geometry'
function zone(overrides: Partial<OcrTemplateZone> = {}): OcrTemplateZone {
return {
name: 'Zone',
target: 'custom_field',
custom_field: 1,
x: 100,
y: 200,
width: 300,
height: 400,
page: 1,
ocr_language: 'eng',
transform: 'strip',
validation_regex: '',
order: 0,
...overrides,
}
}
describe('OCR template editor geometry', () => {
it('normalizes zone pages', () => {
expect(getZonePage(zone({ page: 2 }), 0, 5)).toBe(2)
expect(getZonePage(zone({ page: -1 }), 0, 5)).toBe(5)
expect(getZonePage(zone({ page: -1 }), 2, null)).toBe(3)
expect(getZonePage(zone({ page: 0 }), 0, 5)).toBe(1)
expect(getZonePage(zone({ page: undefined }), 0, 5)).toBe(1)
})
it('checks whether a zone is on the current preview page', () => {
expect(isZoneOnPage(zone({ page: 2 }), 1, 5)).toBe(true)
expect(isZoneOnPage(zone({ page: 2 }), 0, 5)).toBe(false)
expect(isZoneOnPage(zone({ page: -1 }), 4, 5)).toBe(true)
})
it('scales source coordinates to canvas display coordinates', () => {
expect(
getZoneDisplayRect(
zone({ x: 100, y: 200, width: 300, height: 400 }),
{ width: 500, height: 1000 },
{ width: 1000, height: 2000 }
)
).toEqual({ x: 50, y: 100, w: 150, h: 200 })
})
it('uses per-zone source dimensions when present', () => {
expect(
getZoneDisplayRect(
zone({
x: 100,
y: 100,
width: 100,
height: 100,
zone_source_width: 1000,
zone_source_height: 1000,
}),
{ width: 500, height: 500 },
{ width: 2000, height: 2000 }
)
).toEqual({ x: 50, y: 50, w: 50, h: 50 })
})
it('finds zones from topmost to bottommost on the current page', () => {
const zones = [
zone({ name: 'first', x: 0, y: 0, width: 100, height: 100, page: 1 }),
zone({ name: 'second', x: 0, y: 0, width: 50, height: 50, page: 1 }),
zone({ name: 'third', x: 0, y: 0, width: 50, height: 50, page: 2 }),
]
expect(
findZoneAt(
{ x: 25, y: 25 },
zones,
0,
2,
{ width: 100, height: 100 },
{ width: 100, height: 100 }
)
).toBe(1)
})
it('finds resize handles around a display rect', () => {
const rect = { x: 10, y: 20, w: 100, h: 200 }
expect(findHandleAt({ x: 10, y: 20 }, rect)).toBe('nw')
expect(findHandleAt({ x: 110, y: 220 }, rect)).toBe('se')
expect(findHandleAt({ x: 60, y: 20 }, rect)).toBe('n')
expect(findHandleAt({ x: 90, y: 160 }, rect)).toBeNull()
})
it('moves zones without leaving source image bounds', () => {
const z = zone({ x: 50, y: 50, width: 100, height: 100 })
moveZone(
z,
{ x: 500, y: 500 },
{ mouseX: 50, mouseY: 50, zoneX: 50, zoneY: 50 },
{ width: 500, height: 500 },
{ width: 500, height: 500 }
)
expect(z.x).toBe(400)
expect(z.y).toBe(400)
})
it('resizes zones without leaving source image bounds', () => {
const z = zone({ x: 50, y: 50, width: 100, height: 100 })
resizeZone(
z,
'se',
{ x: 500, y: 500 },
{ width: 500, height: 500 },
{ width: 200, height: 200 }
)
expect(z.width).toBe(150)
expect(z.height).toBe(150)
})
it('converts drawn canvas rectangles to source rectangles', () => {
expect(
sourceRectFromDrawing(
{ startX: 100, startY: 200, endX: 50, endY: 100 },
{ width: 500, height: 1000 },
{ width: 1000, height: 2000 }
)
).toEqual({ x: 100, y: 200, w: 100, h: 200 })
})
})
@@ -1,201 +0,0 @@
import { OcrTemplateZone } from 'src/app/data/ocr-template'
export interface DrawingRect {
startX: number
startY: number
endX: number
endY: number
}
export interface Dimensions {
width: number
height: number
}
export interface Point {
x: number
y: number
}
export interface DisplayRect {
x: number
y: number
w: number
h: number
}
export interface MoveStart {
mouseX: number
mouseY: number
zoneX: number
zoneY: number
}
export type ResizeHandle = 'n' | 's' | 'e' | 'w' | 'ne' | 'nw' | 'se' | 'sw'
export const HANDLE_SIZE = 8
export const MIN_ZONE_SIZE = 10
export function getZonePage(
zone: OcrTemplateZone,
previewPage: number,
previewPageCount: number | null
): number {
const page = zone.page ?? 1
if (page === -1) return previewPageCount ?? previewPage + 1
return page >= 1 ? page : 1
}
export function isZoneOnPage(
zone: OcrTemplateZone,
previewPage: number,
previewPageCount: number | null
): boolean {
return getZonePage(zone, previewPage, previewPageCount) === previewPage + 1
}
export function getZoneSourceSize(
zone: OcrTemplateZone,
imageSize: Dimensions
): Dimensions {
return {
width: zone.zone_source_width || imageSize.width,
height: zone.zone_source_height || imageSize.height,
}
}
export function getZoneDisplayRect(
zone: OcrTemplateZone,
canvasSize: Dimensions,
imageSize: Dimensions
): DisplayRect {
const sourceSize = getZoneSourceSize(zone, imageSize)
const scaleX = canvasSize.width / sourceSize.width
const scaleY = canvasSize.height / sourceSize.height
return {
x: zone.x * scaleX,
y: zone.y * scaleY,
w: zone.width * scaleX,
h: zone.height * scaleY,
}
}
export function findHandleAt(
point: Point,
rect: DisplayRect,
handleSize = HANDLE_SIZE
): ResizeHandle | null {
const handles: [ResizeHandle, number, number][] = [
['nw', rect.x, rect.y],
['n', rect.x + rect.w / 2, rect.y],
['ne', rect.x + rect.w, rect.y],
['w', rect.x, rect.y + rect.h / 2],
['e', rect.x + rect.w, rect.y + rect.h / 2],
['sw', rect.x, rect.y + rect.h],
['s', rect.x + rect.w / 2, rect.y + rect.h],
['se', rect.x + rect.w, rect.y + rect.h],
]
return (
handles.find(
([, x, y]) =>
Math.abs(point.x - x) <= handleSize &&
Math.abs(point.y - y) <= handleSize
)?.[0] ?? null
)
}
export function findZoneAt(
point: Point,
zones: OcrTemplateZone[],
previewPage: number,
previewPageCount: number | null,
canvasSize: Dimensions,
imageSize: Dimensions
): number | null {
for (let i = zones.length - 1; i >= 0; i--) {
const zone = zones[i]
if (!isZoneOnPage(zone, previewPage, previewPageCount)) continue
const rect = getZoneDisplayRect(zone, canvasSize, imageSize)
if (
point.x >= rect.x &&
point.x <= rect.x + rect.w &&
point.y >= rect.y &&
point.y <= rect.y + rect.h
) {
return i
}
}
return null
}
export function moveZone(
zone: OcrTemplateZone,
point: Point,
moveStart: MoveStart,
canvasSize: Dimensions,
imageSize: Dimensions
) {
const sourceSize = getZoneSourceSize(zone, imageSize)
const scaleX = sourceSize.width / canvasSize.width
const scaleY = sourceSize.height / canvasSize.height
const dx = Math.round((point.x - moveStart.mouseX) * scaleX)
const dy = Math.round((point.y - moveStart.mouseY) * scaleY)
zone.x = clamp(moveStart.zoneX + dx, 0, sourceSize.width - zone.width)
zone.y = clamp(moveStart.zoneY + dy, 0, sourceSize.height - zone.height)
}
export function resizeZone(
zone: OcrTemplateZone,
handle: ResizeHandle,
point: Point,
canvasSize: Dimensions,
imageSize: Dimensions
) {
const sourceSize = getZoneSourceSize(zone, imageSize)
const scaleX = sourceSize.width / canvasSize.width
const scaleY = sourceSize.height / canvasSize.height
const imageX = clamp(Math.round(point.x * scaleX), 0, sourceSize.width)
const imageY = clamp(Math.round(point.y * scaleY), 0, sourceSize.height)
if (handle.includes('w')) {
const right = Math.min(zone.x + zone.width, sourceSize.width)
zone.x = clamp(imageX, 0, right - MIN_ZONE_SIZE)
zone.width = right - zone.x
}
if (handle.includes('e')) {
zone.width = Math.max(MIN_ZONE_SIZE, imageX - zone.x)
}
if (handle.includes('n')) {
const bottom = Math.min(zone.y + zone.height, sourceSize.height)
zone.y = clamp(imageY, 0, bottom - MIN_ZONE_SIZE)
zone.height = bottom - zone.y
}
if (handle.includes('s')) {
zone.height = Math.max(MIN_ZONE_SIZE, imageY - zone.y)
}
}
export function sourceRectFromDrawing(
rect: DrawingRect,
canvasSize: Dimensions,
imageSize: Dimensions
): DisplayRect {
const scaleX = imageSize.width / canvasSize.width
const scaleY = imageSize.height / canvasSize.height
return {
x: Math.round(Math.min(rect.startX, rect.endX) * scaleX),
y: Math.round(Math.min(rect.startY, rect.endY) * scaleY),
w: Math.round(Math.abs(rect.endX - rect.startX) * scaleX),
h: Math.round(Math.abs(rect.endY - rect.startY) * scaleY),
}
}
function clamp(value: number, min: number, max: number): number {
return Math.max(min, Math.min(value, max))
}

Some files were not shown because too many files have changed in this diff Show More