Compare commits

..
Author SHA1 Message Date
Trenton Holmes a22b9fdbe4 Store ideas for making mail fetch more efficient 2026-07-20 19:31:36 -07:00
d8d8872414 Fix (beta): compute num_notes via a subquery instead of Count()+distinct() (#13182)
DocumentViewSet.get_queryset() annotated num_notes via a LEFT JOIN to
documents_note plus Count(), which requires the database to aggregate
every matching document's note count before it can even sort or paginate
the result -- on every list request, not just filtered ones.

Switched to the same correlated-subquery pattern already used two lines
above for effective_content (Subquery + OuterRef), which Django compiles
to portable SQL across sqlite/postgresql/mariadb rather than a join-based
aggregate.

Benchmarked against a 100k synthetic document corpus (Postgres): the
plain document list request dropped from ~2.2-2.4s to ~0.9-1.0s. Note
that a separate, larger cost remains in the same queryset's plain
.distinct() call, which still forces a full sort over every matching
row regardless of this fix -- tracked separately, not addressed here.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 13:59:44 -07:00
shamoonandGitHub ff609c2987 Change: update root modified timestamp on version changes (#13170) 2026-07-20 15:45:24 +00:00
Trenton HandGitHub 2b784e709b Fix: prevent tag assignment from reverting other pending workflow assignments (#13178)
apply_assignment_to_document() mutated tags directly on the shared document instance via add_nested_tags(), whose m2m_changed signal triggers update_filename_and_move_files() -> instance.refresh_from_db(), discarding any not-yet-saved fields (e.g. storage_path) staged by an earlier-ordered action in the same workflow. Apply tag changes to a freshly-fetched instance instead, matching the pattern already used in apply_removal_to_document().
2026-07-20 08:22:30 -07:00
Trenton HandGitHub c9716252f0 Performance: Add DB indexes for common query/sort patterns (#13167)
Adds indexes on Document.checksum, Document.page_count, and a composite
(owner, created) index on Document, plus composite (field, value_*)
indexes on CustomFieldInstance for each typed value column.

Benchmarked against a 100k-document / 200k-custom-field-instance SQLite
dataset: Document.checksum lookups ~69x faster, page_count sort ~40x,
owner+created list queries ~6x. CustomFieldInstance composite indexes
show large gains (10-24x) on realistic narrow-selectivity queries
matching the workflow scheduler and custom-field search code paths.
2026-07-20 13:12:54 +00:00
shamoon 5cf9152a40 Merge branch 'dev' into beta 2026-07-19 20:20:48 -07:00
shamoonandGitHub 80210bd3bf Performance: use NgOptimizedImage for thumbnail lazy loading (#13169) 2026-07-19 18:05:51 -07:00
4e52dd5710 Fix: cache per-request effective-document resolution for thumb/metadata/preview (#13166)
Django's condition() decorator invokes etag_func and last_modified_func
separately, and the view itself may resolve again -- each call to
resolve_effective_document_by_pk() was redoing the same root/version
DB lookups. Memoize the resolution on the request object so a single
thumb/metadata/preview request resolves the effective document once
instead of up to three times.

Related to paperless-ngx/paperless-ngx#13161.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-19 18:04:16 -07:00
GitHub Actions d34ef75786 Auto translate strings 2026-07-19 22:48:24 +00:00
shamoon b8c424c8e1 Update document-detail.component.html 2026-07-19 15:46:17 -07:00
GitHub Actions ec8991c2ec Auto translate strings 2026-07-19 20:56:43 +00:00
shamoonandGitHub 4f6d9fa93f Fix: dont allow archive version for share links when one doesnt exist (#13163) 2026-07-19 20:54:58 +00:00
GitHub Actions 2180b21c41 Auto translate strings 2026-07-19 14:45:46 +00:00
shamoonandGitHub 99bdfdfe7a Fix: better handle saved view errors in dashboard widgets (#13164) 2026-07-19 07:44:12 -07:00
GitHub Actions 73a0586a75 Auto translate strings 2026-07-18 06:00:02 +00:00
shamoon dcba44ad78 Merge branch 'beta' into dev 2026-07-17 22:57:47 -07:00
shamoon ed54f1a8b9 Fix: more signal-backed conversions 2026-07-17 22:57:37 -07:00
shamoonandGitHub f8a641b402 Fix (beta): convert chat component to signal-backed (#13152) 2026-07-17 22:26:07 -07:00
stumpylog e85223929c Fix: increase checks-discovery-timeout for Wait for Docker Build
The Merge and Push Manifest check isn't materialized until the
build-arch matrix completes (its `if` gates on a job output), which
routinely takes longer than the action's 60s discovery window,
causing the release workflow to bail before Docker builds finish.
2026-07-17 15:27:55 -07:00
stumpylog f2e3304576 Merge remote-tracking branch 'origin/dev' into beta 2026-07-17 15:16:36 -07:00
6938fb0e01 New Crowdin translations by GitHub Action (#13134)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-17 15:16:21 -07:00
Trenton HandGitHub 255c214d0b Fix: avoid HAVING clause conflicts in custom field query filter on MariaDB (#13143) 2026-07-17 13:44:58 -07:00
Trenton HandGitHub a0ab05a9a6 Chore: Upgrade more vulerable packages (#13020)
* Few more before

* Bump django to 5.2.16

* Bump daphne to 4.2.2

* Bump transformers to 5.13.1

* Bump nltk to 3.10.0
2026-07-17 12:01:30 -07:00
df1ddb15cc Performance: Tantivy indexing optimization (#13053)
* Tantivy: get permissions by chunks

-40% indexing time compared to previous commit

* Make progress bar process one by one with chunk

-15% indexing time compared to previous commit

* Prefetch FK + iterate over chunk from SQL

Prefetch additional needed data (note user, custom field content)

-20% indexing time compared to previous commit

* Reindex: increase Tantivy heap size from 128 to 512MB

Gains probably vary depending on the machine,
but it seems a sweet spot compatible with low-end hardware.

* Reindex: optimization on permission fetching and autocomplete word set

-10% indexing time compared to previous commit

* Autocomplete analyzer python->rust

Splits words with underscore compared to the python analyzer.
E.g.: "blue_print" -> ["blue", "print"]
It can still be found with the "blue_print" keyword,
as the search string is also split in two words.

-50% indexing time compared to previous commit (indexing is twice faster!)

* Index bigram for CJK content only

Inedxing time slightly longer (~3%),
but since the non-CJK content is not indexed,
bigram searchs will be slightly optimized.

* Fix group-based view_document permissions missing from bulk rebuild

_bulk_get_viewer_ids only queried UserObjectPermission, dropping the
group-permission expansion that get_users_with_perms(with_group_users=True)
performs for the non-batched per-document indexing path. A user who could
only see a document via group membership would lose search access to it
after any full reindex.

Also query GroupObjectPermission and expand group membership to user ids,
matching the existing single-document behavior.

* Yield (document, viewer_ids) pairs from _DocumentViewerStream

Previously _DocumentViewerStream.__iter__ yielded plain Document objects
while the matching viewer ids were exposed through a separate mutable
attribute (viewer_ids_by_pk), overwritten each time the generator crossed
a chunk boundary. rebuild() read that attribute out-of-band per document.

This only worked because the current iter_wrapper (a plain progress-bar
passthrough) happens to consume the stream in strict lock-step with no
lookahead. Any wrapper that buffers, batches, or reorders would silently
pair a document with the wrong chunk's viewer ids. Yield the pair directly
so the association travels with the document regardless of how iter_wrapper
consumes the stream, and drop the now-unneeded viewer_ids_by_pk attribute.

* Add --heap-size-mb CLI arg to document_index reindex

writer_heap_bytes was hardcoded at 512MB with no way to tune it. Expose it
as a manual-rebuild-only CLI arg rather than a settings/env var, per review
feedback, so lower-memory hosts can reduce it without a wider config
surface. Defaults to unset so TantivyBackend.rebuild's own default stays
the single source of truth.

---------

Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-17 11:33:09 -07:00
GitHub Actions 71557d7c64 Auto translate strings 2026-07-16 17:43:31 +00:00
shamoon 66f5111dc1 Chore: some frontend sonar fixes 2026-07-16 10:42:00 -07:00
dependabot[bot]andGitHub 0addb448f5 Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 4 updates (#13137) 2026-07-15 23:52:20 +00:00
dependabot[bot]andGitHub 5428ea5caa Chore(deps-dev): Bump the development group with 2 updates (#13068)
Signed-off-by: dependabot[bot] <support@github.com>
2026-07-15 21:04:28 +00:00
GitHub Actions 2de2dfdb81 Auto translate strings 2026-07-15 20:45:19 +00:00
Hugo Beauzée-LuyssenandGitHub f0d934259c Fix: scope ProcessedMail UID deduplication by IMAP UIDVALIDITY (#12839) 2026-07-15 13:43:16 -07:00
Albert MikaelyanandGitHub 735c3b073a Fix: allow setting any UID and GID when running rootless (#13090) 2026-07-15 10:19:22 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e2cf3633fe docker-compose(deps): bump greenmail/standalone in /docker/compose (#13067)
Bumps greenmail/standalone from 2.1.8 to 2.1.9.

---
updated-dependencies:
- dependency-name: greenmail/standalone
  dependency-version: 2.1.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 09:23:10 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
c0589a9ce9 docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#13066)
Bumps gotenberg/gotenberg from 8.33 to 8.34.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.34'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 09:09:30 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2135572adb docker-compose(deps): bump nginx in /docker/compose (#13065)
Bumps nginx from 1.31.1-alpine to 1.31.2-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.2-alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 15:55:33 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
697e262b5d docker(deps): Bump astral-sh/uv (#13070)
Bumps [astral-sh/uv](https://github.com/astral-sh/uv) from 0.11.19-python3.12-trixie-slim to 0.11.28-python3.12-trixie-slim.
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/uv/compare/0.11.19...0.11.28)

---
updated-dependencies:
- dependency-name: astral-sh/uv
  dependency-version: 0.11.26-python3.12-trixie-slim
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-15 15:37:35 +00:00
GitHub Actions 5a65c4448d Auto translate strings 2026-07-15 15:14:34 +00:00
shamoonandGitHub 370a0a29ba Fix (beta): fix sidebar document close button (#13136) 2026-07-15 08:12:56 -07:00
536007f42b New Crowdin translations by GitHub Action (#12722)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-14 09:29:42 -07:00
shamoonandGitHub 71e2dd4b08 Fix: include wasm assets in pdf.js-based viewer (#13122) 2026-07-11 22:14:38 -07:00
GitHub Actions dabd7826d0 Auto translate strings 2026-07-11 23:14:15 +00:00
shamoonandGitHub 054b234739 Fix: fix ui labels from md5 to sha256 (#13119) 2026-07-11 16:12:47 -07:00
GitHub Actions 77dd4d1b0c Auto translate strings 2026-07-11 23:05:28 +00:00
shamoon 56dc3e426e Merge branch 'dev' into beta 2026-07-10 11:34:15 -07:00
GitHub Actions be8890cfaf Auto translate strings 2026-07-10 18:24:51 +00:00
shamoonandGitHub e03b09e2ea Chore: pnpm lockfile cleanup (#13116) 2026-07-10 11:23:19 -07:00
shamoon 19960cc4fa Merge branch 'dev' into beta 2026-07-10 00:44:47 -07:00
shamoonandGitHub e145fe8cc7 Fix: clamp mailrule max age before migration (#13093) 2026-07-07 10:56:45 -07:00
shamoonandGitHub fce55a1609 Fix: use latest document version for bulk download (#13103) 2026-07-07 10:55:40 -07:00
shamoonandGitHub 1dabd2601d Fix: handle relative date with comma query (#13087) 2026-07-03 12:44:33 -07:00
shamoon 88f69841b3 Merge branch 'dev' into beta 2026-07-02 15:06:19 -07:00
shamoon 5b8fbdcec7 Merge branch 'dev' into beta 2026-06-27 15:56:20 -07:00
shamoon beb048b94a Merge branch 'dev' into beta 2026-06-27 06:53:09 -07:00
shamoonandGitHub b33d11778a Fix: change tantivy token limit to title field max length (#13046) 2026-06-23 08:02:26 -07:00
214 changed files with 88328 additions and 49638 deletions
+1
View File
@@ -26,6 +26,7 @@ jobs:
check-name: 'Merge and Push Manifest'
repo-token: ${{ secrets.GITHUB_TOKEN }}
wait-interval: 60
checks-discovery-timeout: 1800
build-release:
name: Build Release
needs: wait-for-docker
+4 -4
View File
@@ -38,7 +38,7 @@ repos:
- json
# See https://github.com/prettier/prettier/issues/15742 for the fork reason
- repo: https://github.com/rbubley/mirrors-prettier
rev: 'v3.8.4'
rev: 'v3.9.4'
hooks:
- id: prettier
types_or:
@@ -46,16 +46,16 @@ repos:
- ts
- markdown
additional_dependencies:
- prettier@3.8.3
- prettier@3.9.4
- 'prettier-plugin-organize-imports@4.3.0'
# Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.17
rev: v0.15.20
hooks:
- id: ruff-check
- id: ruff-format
- repo: https://github.com/tox-dev/pyproject-fmt
rev: "v2.24.1"
rev: "v2.25.1"
hooks:
- id: pyproject-fmt
additional_dependencies: [tomli]
+5 -1
View File
@@ -30,7 +30,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs
# Comments:
# - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.11.19-python3.12-trixie-slim AS s6-overlay-base
FROM ghcr.io/astral-sh/uv:0.11.28-python3.12-trixie-slim AS s6-overlay-base
WORKDIR /usr/src/s6
@@ -238,6 +238,10 @@ RUN set -eux \
&& chown --from root:root --changes --recursive paperless:paperless /usr/src/paperless \
&& echo "Making fontconfig cache writable for arbitrary container UIDs" \
&& chmod 1777 /var/cache/fontconfig \
&& echo "Making /run world-writable for rootless operation" \
&& chmod 1777 /run \
&& echo "Removing setuid from s6-overlay-suexec for rootless compat" \
&& chmod u-s /command/s6-overlay-suexec \
&& echo "Collecting static files" \
&& PAPERLESS_SECRET_KEY=build-time-dummy s6-setuidgid paperless python3 manage.py collectstatic --clear --no-input --link \
&& PAPERLESS_SECRET_KEY=build-time-dummy s6-setuidgid paperless python3 manage.py compilemessages \
+3 -3
View File
@@ -4,7 +4,7 @@
# correct networking for the tests
services:
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
hostname: gotenberg
container_name: gotenberg
network_mode: host
@@ -24,7 +24,7 @@ services:
network_mode: host
restart: unless-stopped
greenmail:
image: docker.io/greenmail/standalone:2.1.8
image: docker.io/greenmail/standalone:2.1.9
hostname: greenmail
container_name: greenmail
environment:
@@ -35,7 +35,7 @@ services:
- "3143:3143" # IMAP
restart: unless-stopped
nginx:
image: docker.io/nginx:1.31.1-alpine
image: docker.io/nginx:1.31.2-alpine
hostname: nginx
container_name: nginx
ports:
@@ -72,7 +72,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.
@@ -67,7 +67,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.
@@ -56,7 +56,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.33
image: docker.io/gotenberg/gotenberg:8.34
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.
+143
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -326,3 +326,11 @@ behind a reverse proxy may need to set
[`PAPERLESS_TRUSTED_PROXIES`](configuration.md#PAPERLESS_TRUSTED_PROXIES),
[`PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER`](configuration.md#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER),
or both, to avoid `403 Forbidden` errors on login.
## Database Migrations
Some integer fields have been changed to smaller types to reduce database size. If you have any `MailRule` records with a `maximum_age` greater than 32767, they will be clamped to 32767 during the migration to avoid errors during migration.
### Action Required
No user action is required. The migration will automatically clamp any `MailRule.maximum_age` values greater than 32767 to 32767 during the migration process.
+1 -1
View File
@@ -142,7 +142,7 @@ a [superuser](usage.md#superusers) account.
It is not possible to run the container rootless if additional languages are specified via `PAPERLESS_OCR_LANGUAGES`.
If you want to run Paperless as a rootless container, set `user:` in `docker-compose.yml` to the UID and GID of your host user (use `id -u` and `id -g` to find these values). The container process starts directly as that user with no internal privilege remapping:
If you want to run Paperless as a rootless container, set `user:` in `docker-compose.yml` to the UID and GID of your host user (use `id -u` and `id -g` to find these values). The container process starts directly as that user with no internal privilege remapping. Any UID and GID is supported:
```yaml
webserver:
@@ -0,0 +1,662 @@
# Mail Fetch Re-download Fix Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stop `paperless_mail` from re-downloading full IMAP message bodies for mail it has already handled, and make sure attachment-less mail under attachments-only rules is recorded so it stops matching the search forever.
**Architecture:** In `_handle_mail_rule`, replace the single "fetch every matching message's full body" call with a cheap `UID SEARCH` first, subtract UIDs already in `ProcessedMail`, and fetch bodies only for the remainder (batched). Separately, extract the existing "record `PROCESSED_WO_CONSUMPTION`" block into a shared helper and call it from the no-attachments-under-attachments-only-scope early return in `_handle_message`, which currently skips it entirely.
**Tech Stack:** Django (pytest-django), `imap_tools` (`MailBox.uids()`, `AND` query builder), existing `ProcessedMail` model — no new dependencies, no migrations.
## Global Constraints
- No new required settings and no DB migrations for existing installs (spec: "No new required settings/migrations for existing installs").
- Batch size for body fetches is a hardcoded module constant (`MAIL_FETCH_BATCH_SIZE = 500`), not user-configurable.
- No mailbox action (mark-read/flag/tag/move/delete) is applied for the no-attachment/no-consumption case — only a `ProcessedMail` row is written, matching the existing precedent for other no-consumption outcomes.
- Every path that currently produces a document or applies a mail action must behave exactly as before (no change to `_process_attachments`'/`_process_eml`'s consuming behavior, action application, or the existing per-message `already_processed` dedup check, which stays in place as a safety net).
- Follow the existing per-message `uid_validity` matching semantics exactly: when `self._current_uid_validity is not None`, only match `ProcessedMail` rows with the same `uid_validity` or `uid_validity IS NULL`; when it is `None` (server didn't report UIDVALIDITY), fall back to matching on `(rule, uid, folder)` alone, ignoring `uid_validity`. (This exact conditional already exists at `mail.py:722-726` for the per-message check — the new UID-diff query must replicate it or several existing uidvalidity tests will break.)
---
### Task 1: Add UID search support to the `BogusMailBox` test double
**Files:**
- Modify: `src/paperless_mail/tests/test_mail.py:137-171` (`BogusMailBox.fetch`)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Produces: `BogusMailBox.uids(self, criteria, charset="") -> list[str]`, filtering with the same rules as `fetch()`, returning only UIDs (no message bodies). `BogusMailBox.fetch()` gains support for a `UID <comma-list>` criteria token (used by real code's `AND(uid=[...])` queries).
- Consumes: nothing new — this is test infrastructure only, no production code changes in this task.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py` (near the other `BogusMailBox`-adjacent tests, e.g. after `test_handle_message`):
```python
def test_bogus_mailbox_uids_and_uid_criteria(self) -> None:
mailbox = self.mailMocker.bogus_mailbox
all_messages = list(mailbox.messages)
# uids() returns the UIDs of unseen messages, no bodies needed to call it
unseen_uids = mailbox.uids("(UNSEEN)")
self.assertEqual(
set(unseen_uids),
{m.uid for m in all_messages if not m.seen},
)
# fetch() with an explicit UID criteria returns only the matching messages
target_uid = all_messages[0].uid
from imap_tools import AND
fetched = mailbox.fetch(AND(uid=[target_uid]), mark_seen=False)
self.assertEqual([m.uid for m in fetched], [target_uid])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_bogus_mailbox_uids_and_uid_criteria -v`
Expected: FAIL with `AttributeError: 'BogusMailBox' object has no attribute 'uids'` (and/or the UID-criteria fetch returning all 3 default messages instead of 1, since `fetch()` doesn't understand a `UID` token yet).
- [ ] **Step 3: Implement `uids()` and `UID` criteria support**
Replace `src/paperless_mail/tests/test_mail.py:137-171`:
```python
def fetch(self, criteria, mark_seen, charset="", *, bulk=True):
return self._filter_messages(criteria)
def uids(self, criteria, charset="") -> list[str]:
return [m.uid for m in self._filter_messages(criteria)]
def _filter_messages(self, criteria):
msg = self.messages
criteria = str(criteria).strip("()").split(" ")
if "UNSEEN" in criteria:
msg = filter(lambda m: not m.seen, msg)
if "SUBJECT" in criteria:
subject = criteria[criteria.index("SUBJECT") + 1].strip('"')
msg = filter(lambda m: subject in m.subject, msg)
if "BODY" in criteria:
body = criteria[criteria.index("BODY") + 1].strip('"')
msg = filter(lambda m: body in m.text, msg)
if "FROM" in criteria:
from_ = criteria[criteria.index("FROM") + 1].strip('"')
msg = filter(lambda m: from_ in m.from_, msg)
if "TO" in criteria:
to_ = criteria[criteria.index("TO") + 1].strip('"')
msg = filter(lambda m: any(to_ in to_addr for to_addr in m.to), msg)
if "UNFLAGGED" in criteria:
msg = filter(lambda m: not m.flagged, msg)
if "UNKEYWORD" in criteria:
tag = criteria[criteria.index("UNKEYWORD") + 1].strip("'")
msg = filter(lambda m: tag not in m.flags, msg)
if "(X-GM-LABELS" in criteria: # ['NOT', '(X-GM-LABELS', '"processed"']
msg = filter(lambda m: "processed" not in m.flags, msg)
if "UID" in criteria:
uid_list = criteria[criteria.index("UID") + 1].split(",")
msg = filter(lambda m: m.uid in uid_list, msg)
return list(msg)
```
This is a pure refactor of the existing filtering logic into `_filter_messages`, reused by both `fetch()` (unchanged behavior) and the new `uids()`, plus one new `if "UID" in criteria` branch.
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_bogus_mailbox_uids_and_uid_criteria -v`
Expected: PASS
- [ ] **Step 5: Run the full existing `test_mail.py` suite to confirm the refactor didn't change `fetch()` behavior**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all tests PASS (same pass count as before this task)
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/tests/test_mail.py
git commit -m "test: add BogusMailBox.uids() and UID criteria support"
```
---
### Task 2: Record `ProcessedMail` for attachment-less mail under attachments-only rules
**Files:**
- Modify: `src/paperless_mail/mail.py:749-793` (`_handle_message`)
- Modify: `src/paperless_mail/mail.py:952-979` (`_process_attachments`, tail)
- Test: `src/paperless_mail/tests/test_mail.py:540-548` (`test_handle_empty_message`, rewritten)
**Interfaces:**
- Produces: `MailAccountHandler._record_processed_without_consumption(self, message: MailMessage, rule: MailRule) -> None` — idempotently writes a `ProcessedMail(status="PROCESSED_WO_CONSUMPTION")` row for `(rule, message.uid, rule.folder)` if one doesn't already exist for the current `self._current_uid_validity`. No mailbox action is applied.
- Consumes: `self._current_uid_validity` (already set by `_handle_mail_rule` before `_handle_message` is called; `None` when called directly, e.g. in unit tests).
- [ ] **Step 1: Write the failing test**
Replace `src/paperless_mail/tests/test_mail.py:540-548` (`test_handle_empty_message`):
```python
def test_handle_empty_message(self) -> None:
message = self.mailMocker.messageBuilder.create_message(
subject="No attachments here",
attachments=[],
)
account = MailAccount.objects.create()
rule = MailRule.objects.create(
account=account,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
result = self.mail_account_handler._handle_message(message, rule)
self.mailMocker._queue_consumption_tasks_mock.assert_not_called()
self.assertEqual(result, 0)
processed = ProcessedMail.objects.get(
rule=rule,
uid=message.uid,
folder=rule.folder,
)
self.assertEqual(processed.status, "PROCESSED_WO_CONSUMPTION")
# Calling it again must not create a second row
self.mail_account_handler._handle_message(message, rule)
self.assertEqual(
ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
).count(),
1,
)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_empty_message -v`
Expected: FAIL — `ProcessedMail.DoesNotExist` (no row is created by the current early return).
- [ ] **Step 3: Add the shared helper and call it from both places**
Insert a new method right after `_handle_message` ends, i.e. after `src/paperless_mail/mail.py:793` (`return processed_elements`), before `def filename_inclusion_matches`:
```python
def _record_processed_without_consumption(
self,
message: MailMessage,
rule: MailRule,
) -> None:
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date)
if is_naive(message.date)
else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
```
Then modify `_handle_message`'s early return at `src/paperless_mail/mail.py:756-760`:
```python
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements
```
becomes:
```python
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
self._record_processed_without_consumption(message, rule)
return processed_elements
```
Then replace the tail of `_process_attachments` at `src/paperless_mail/mail.py:952-979`:
```python
if len(consume_tasks) > 0:
queue_consumption_tasks(
consume_tasks=consume_tasks,
rule=rule,
message=message,
uid_validity=self._current_uid_validity,
)
else:
# No files to consume, just mark as processed if it wasn't by .eml processing
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date)
if is_naive(message.date)
else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
return processed_attachments
```
with:
```python
if len(consume_tasks) > 0:
queue_consumption_tasks(
consume_tasks=consume_tasks,
rule=rule,
message=message,
uid_validity=self._current_uid_validity,
)
else:
# No files to consume, just mark as processed if it wasn't by .eml processing
self._record_processed_without_consumption(message, rule)
return processed_attachments
```
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_empty_message -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite (regression check for the `_process_attachments` refactor)**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "fix: record ProcessedMail for attachment-less mail under attachments-only rules"
```
---
### Task 3: Diff UIDs against `ProcessedMail` before fetching message bodies
**Files:**
- Modify: `src/paperless_mail/mail.py:641-693` (`_handle_mail_rule`, the fetch section)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Consumes: `MailAccountHandler._record_processed_without_consumption` (Task 2), `ProcessedMail` model, `imap_tools.AND`, `M.uids()` (Task 1's `BogusMailBox.uids()` in tests; `imap_tools.MailBox.uids()` in production).
- Produces: no new public interface — `_handle_mail_rule`'s external behavior (return value, exceptions raised) is unchanged; only its internal fetch strategy changes. `_handle_mail_rule` now returns `0` immediately, without calling `M.fetch()` at all, when every UID matching the search criteria already has a `ProcessedMail` row.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py`:
```python
def test_handle_mail_account_skips_body_fetch_for_already_processed_mail(
self,
) -> None:
"""
GIVEN:
- An attachment-less mail under an attachments-only mark-read rule,
already recorded as PROCESSED_WO_CONSUMPTION
WHEN:
- The mail account is processed again and the mail still matches the
search criteria (it was never marked read, since no mail action is
applied for the no-consumption case)
THEN:
- No IMAP body fetch happens for that mail; only the cheap UID search runs.
"""
account = MailAccount.objects.create(
name="test",
imap_server="",
username="admin",
password="secret",
)
rule = MailRule.objects.create(
name="testrule",
account=account,
action=MailRule.MailAction.MARK_READ,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
message = self.mailMocker.messageBuilder.create_message(
subject="No attachment",
attachments=[],
)
self.mailMocker.bogus_mailbox.messages = [message]
self.mailMocker.bogus_mailbox.updateClient()
# First run: records ProcessedMail without consuming anything.
self.mail_account_handler.handle_mail_account(account)
self.assertTrue(
ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
).exists(),
)
self.mailMocker._queue_consumption_tasks_mock.assert_not_called()
# Second run: message still matches UNSEEN (mark-read action never ran),
# but its body must not be downloaded again.
with mock.patch.object(
self.mailMocker.bogus_mailbox,
"fetch",
wraps=self.mailMocker.bogus_mailbox.fetch,
) as fetch_spy:
self.mail_account_handler.handle_mail_account(account)
fetch_spy.assert_not_called()
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_skips_body_fetch_for_already_processed_mail -v`
Expected: FAIL — `fetch_spy.assert_not_called()` fails because the current code always calls `M.fetch()` for every message matching the search criteria, regardless of `ProcessedMail`.
- [ ] **Step 3: Implement the UID diff**
Replace `src/paperless_mail/mail.py:683-693`:
```python
try:
messages = M.fetch(
criteria=criterias,
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
with:
```python
try:
all_uids = set(
M.uids(criteria=criterias, charset=rule.account.character_set),
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while searching folder {rule.folder}",
) from err
processed_uids_qs = ProcessedMail.objects.filter(
rule=rule,
folder=rule.folder,
uid__in=all_uids,
)
if self._current_uid_validity is not None:
processed_uids_qs = processed_uids_qs.filter(
Q(uid_validity=self._current_uid_validity)
| Q(uid_validity__isnull=True),
)
processed_uids = set(processed_uids_qs.values_list("uid", flat=True))
new_uids = all_uids - processed_uids
if not new_uids:
self.log.debug(
f"Rule {rule}: No new mail matching criteria {criterias}",
)
return 0
try:
messages = M.fetch(
criteria=AND(uid=list(new_uids)),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
Note the `uid_validity` handling here deliberately mirrors the existing per-message check at `mail.py:722-726` exactly (see Global Constraints) — the extra `Q(...)` filter is applied only when `self._current_uid_validity is not None`.
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_skips_body_fetch_for_already_processed_mail -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite (regression check, especially the uidvalidity tests)**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS, including:
- `test_handle_mail_account_skip_duplicate_uids_from_fetch`
- `test_handle_mail_account_skips_mail_already_processed_in_same_uidvalidity`
- `test_handle_mail_account_processes_mail_after_uidvalidity_change`
- `test_handle_mail_account_skips_mail_processed_before_uidvalidity_tracking`
- `test_handle_mail_account_processes_mail_when_uidvalidity_unavailable`
- `test_handle_mail_account_skips_mail_when_uidvalidity_unavailable_but_prior_record_exists`
- `test_handle_mail_account_overlapping_rules_only_first_consumes`
If any of these fail, the `uid_validity` branching in Step 3 does not match `mail.py:722-726` closely enough — re-check against the Global Constraints note before changing test expectations.
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "fix: diff UIDs against ProcessedMail before fetching message bodies"
```
---
### Task 4: Batch the body fetch for large backlogs
**Files:**
- Modify: `src/paperless_mail/mail.py:74-76` (module-level constant)
- Modify: `src/paperless_mail/mail.py` (the `M.fetch(...)` block added in Task 3)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Produces: module-level constant `paperless_mail.mail.MAIL_FETCH_BATCH_SIZE: int = 500`.
- Consumes: `itertools` (already imported at `mail.py:2`), `sorted_new_uids` derived from Task 3's `new_uids` set.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py`. Add `from paperless_mail.mail import MAIL_FETCH_BATCH_SIZE` to the imports at the top of the file (alongside the other `from paperless_mail.mail import ...` lines at `test_mail.py:35-38`).
```python
@mock.patch("paperless_mail.mail.MAIL_FETCH_BATCH_SIZE", 5)
def test_handle_mail_account_batches_body_fetch_for_large_backlog(self) -> None:
"""
GIVEN:
- More new/unprocessed mail than MAIL_FETCH_BATCH_SIZE
WHEN:
- The mail account is processed
THEN:
- The body fetch is issued in multiple batches
- Every message is still processed (none dropped at a batch boundary)
"""
account = MailAccount.objects.create(
name="test",
imap_server="",
username="admin",
password="secret",
)
rule = MailRule.objects.create(
name="testrule",
account=account,
action=MailRule.MailAction.MARK_READ,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
message_count = 12 # more than the patched batch size of 5
self.mailMocker.bogus_mailbox.messages = [
self.mailMocker.messageBuilder.create_message(
subject=f"No attachment {i}",
attachments=[],
)
for i in range(message_count)
]
self.mailMocker.bogus_mailbox.updateClient()
with mock.patch.object(
self.mailMocker.bogus_mailbox,
"fetch",
wraps=self.mailMocker.bogus_mailbox.fetch,
) as fetch_spy:
self.mail_account_handler.handle_mail_account(account)
# ceil(12 / 5) == 3 batches
self.assertEqual(fetch_spy.call_count, 3)
self.assertEqual(
ProcessedMail.objects.filter(rule=rule).count(),
message_count,
)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_batches_body_fetch_for_large_backlog -v`
Expected: FAIL — either an `ImportError` for `MAIL_FETCH_BATCH_SIZE` (doesn't exist yet) or, once that import is stubbed out, `fetch_spy.call_count == 1` instead of `3` (Task 3's implementation fetches all new UIDs in one call).
- [ ] **Step 3: Add the constant and batch the fetch**
Insert after `src/paperless_mail/mail.py:74` (right after the `APPLE_MAIL_TAG_COLORS` dict closes, before `class MailError`):
```python
MAIL_FETCH_BATCH_SIZE = 500
```
Replace the `M.fetch(...)` block added in Task 3 (Task 3 Step 3's final `try/except`):
```python
try:
messages = M.fetch(
criteria=AND(uid=list(new_uids)),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
with:
```python
sorted_new_uids = sorted(new_uids, key=int)
message_batches = []
for batch_start in range(0, len(sorted_new_uids), MAIL_FETCH_BATCH_SIZE):
batch = sorted_new_uids[batch_start : batch_start + MAIL_FETCH_BATCH_SIZE]
try:
message_batches.append(
M.fetch(
criteria=AND(uid=batch),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
),
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
messages = itertools.chain(*message_batches)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_batches_body_fetch_for_large_backlog -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "perf: batch body fetches when many new UIDs are pending"
```
---
### Task 5: Full regression pass and lint
**Files:** none (verification only, plus any fixups this step surfaces)
- [ ] **Step 1: Run the full backend test suite with coverage/parallelism as configured**
Run: `uv run pytest src/paperless_mail/`
Expected: all `paperless_mail` tests PASS
- [ ] **Step 2: Run the full project test suite**
Run: `uv run pytest`
Expected: all tests PASS (no regressions outside `paperless_mail`)
- [ ] **Step 3: Lint and format**
Run: `uv run ruff check src/paperless_mail/` and `uv run ruff format --check src/paperless_mail/`
Expected: no errors. If `ruff format --check` reports files needing formatting, run `uv run ruff format src/paperless_mail/` and re-check.
- [ ] **Step 4: Type check against the frozen baselines**
Run: `uv run mypy src/paperless_mail/mail.py` (or the project's standard mypy invocation) and confirm no new violations beyond `.mypy-baseline.txt`.
Expected: no new errors introduced by this change.
- [ ] **Step 5: Commit any fixups**
If Steps 1-4 required changes:
```bash
git add -u
git commit -m "chore: fix lint/type fallout from mail fetch fix"
```
If no changes were needed, skip this step — nothing to commit.
@@ -0,0 +1,216 @@
# Mail fetch fix: avoid re-downloading already-handled IMAP mail
## Problem
`paperless_mail`'s `_handle_mail_rule` (`src/paperless_mail/mail.py:641`) fetches
the full RFC822 body of every message an IMAP `SEARCH` matches (via
`M.fetch(criteria=criterias, ..., bulk=True)`), then does de-duplication in
Python afterward. Two things compound into a real bug (reported upstream in
[paperless-ngx#13175](https://github.com/paperless-ngx/paperless-ngx/issues/13175)):
1. **Full-body fetch happens before dedup.** The de-dup check against
`ProcessedMail` (`mail.py:717-731`) happens per-message, after the body has
already been downloaded for every matching message. Steady-state cost is
_O(everything the search matches)_, not _O(new since last run)_.
2. **Attachment-less mail under an attachments-only rule is never recorded.**
`_handle_message` (`mail.py:749`) returns early, before any attachment
processing, when a message has no attachments and
`rule.consumption_scope == ATTACHMENTS_ONLY`:
```python
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements # 0, no ProcessedMail row, no mail action applied
```
No `ProcessedMail` row is written and no mail action (mark-read/flag/tag)
is applied, since both only happen via `queue_consumption_tasks`, which is
only reached from the attachment/eml processing paths. The message keeps
matching the search (e.g. `UNSEEN`) forever.
Combined: an inbox where most mail has no attachments gets fully re-downloaded
on every scheduled run (default every 10 minutes), indefinitely, for large
mailboxes -- confirmed against the current codebase, not user
misconfiguration.
## Goals
- Stop re-downloading full message bodies for mail that has already been
handled (processed into a document, or determined to produce nothing).
- Record `ProcessedMail` for the no-attachment / attachments-only early-return
case, so it participates in dedup like every other terminal outcome.
- No new required settings or DB migrations for existing installs.
- Graceful behavior on first run against a large existing mailbox (no single
giant IMAP command, no giant in-memory batch).
- Preserve existing action/consumption semantics exactly for every path that
currently produces a document or applies a mail action.
## Non-goals
- Changing what mail actions (mark-read/flag/tag/move/delete) get applied, or
when.
- Applying a mail action to the no-attachment early-return case (explicitly
out of scope -- see Decisions).
- Making batch size user-configurable.
- Touching EML_ONLY / EVERYTHING consumption scopes' semantics (they already
don't hit the early-return branch described above).
## Decisions
These were settled during brainstorming and are load-bearing for the design
below:
- **Fix both halves together.** Recording `ProcessedMail` alone does not fix
the bandwidth problem: without a mail action applied, an attachment-less
message stays unseen and keeps matching the search criteria, so its body
gets re-downloaded every run regardless of whether it gets reprocessed.
Fixing only the bandwidth side without recording `ProcessedMail` would mean
the UID-diff step never has anything to exclude for these messages. They
must ship together.
- **No mailbox action for the no-attachment case.** Only a `ProcessedMail`
row is written; the message is not marked read/flagged/moved/deleted. This
matches the existing precedent for other no-consumption outcomes
(`mail.py:960-977`, e.g. an attachment present but filtered out or an
unsupported mime type) and avoids changing the user's mailbox state for
mail paperless previously never touched.
- **UID-diff-before-fetch, not a BODYSTRUCTURE probe or SEARCH-side
exclusion.** Considered and rejected:
- BODYSTRUCTURE probing (fetch structure only, decide whether to fetch full
body) only helps the attachments-only scope, not the general
already-processed case, and `imap_tools` doesn't cleanly expose a
structure-only fetch.
- Excluding already-processed UIDs directly in the IMAP `SEARCH` criteria
(`NOT UID (...)`) was rejected because for tens of thousands of
already-processed UIDs the excluded-UID list itself blows up the command
size -- worse than the two-step approach.
- **Batch size is a hardcoded constant**, not a new setting, per the
"no new required settings" goal.
## Design
### `_handle_mail_rule` (`mail.py:641`)
Replace the single `M.fetch(criteria=criterias, ...)` call with:
1. `self._current_uid_validity` computed as today (unchanged, already first).
2. `criterias = make_criterias(...)` (unchanged).
3. `all_uids = set(M.uids(criteria=criterias, charset=rule.account.character_set))`
-- a `UID SEARCH`, no bodies.
4. Query already-processed UIDs in one DB round trip, reusing the same
uid_validity matching already used per-message:
```python
processed_uids = set(
ProcessedMail.objects.filter(
rule=rule, folder=rule.folder, uid__in=all_uids,
).filter(
Q(uid_validity=self._current_uid_validity) | Q(uid_validity__isnull=True),
).values_list("uid", flat=True)
)
```
5. `new_uids = all_uids - processed_uids`. If empty: log at debug level and
`return 0` -- no body fetch at all. This is the steady-state case.
6. Otherwise, iterate `new_uids` in fixed-size batches of
`MAIL_FETCH_BATCH_SIZE = 500` (module-level constant), calling
`M.fetch(criteria=AND(uid=batch), mark_seen=False, charset=rule.account.character_set, bulk=True)`
per batch. Chain the resulting message iterators into the existing
per-message loop (`mail.py:699+`) unchanged.
7. The existing per-message `already_processed` DB check
(`mail.py:717-731`) stays in place unchanged, as a safety net against
races (e.g. concurrent rule runs against the same folder in this pass) and
to keep behavior identical if steps 3/4 ever disagree with it.
### New helper: `_record_processed_without_consumption`
Extract the existing dedup-and-create block at `mail.py:960-977` into a
method on the same class:
```python
def _record_processed_without_consumption(self, message, rule) -> None:
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date) if is_naive(message.date) else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
```
Call sites:
- `_process_attachments`'s existing no-consumables branch (replaces the
inline block, same behavior).
- `_handle_message`'s early return (`mail.py:756-760`), newly, for the
no-attachments-under-attachments-only-scope case.
### Data flow (steady state, nothing new)
search criteria (unchanged) -> `UID SEARCH` for matching UIDs (cheap) ->
subtract UIDs already in `ProcessedMail` (cheap, DB-only) -> zero new UIDs ->
no fetch, no body download.
### Data flow (new mail present)
... same as above -> some UIDs remain -> fetch bodies only for those,
batched -> existing per-message processing, action application, and
`ProcessedMail` recording, unchanged except the no-attachment early return
now also calls `_record_processed_without_consumption`.
## Error handling
- `M.uids(...)` raising is wrapped in the same `try/except` that currently
wraps `M.fetch(...)` (`mail.py:683-693`), surfacing as `MailError`
identically to today's search-failure behavior.
- A batch's `M.fetch(...)` raising mid-loop is caught by the same
`try/except`, applied per-batch. One failing batch fails the whole rule
run for this pass -- matching today's all-or-nothing semantics (currently
a single failed fetch already fails the whole rule).
- Empty `new_uids` short-circuits before any fetch, so it can't hit fetch
error paths at all for the common case.
- `_record_processed_without_consumption` reuses the existing
`ProcessedMail.objects.create` call already in production use; no new
failure mode introduced.
## Testing
In `src/paperless_mail/tests/test_mail.py`:
- Extend the `BogusMailBox` test double with a `uids()` method that mirrors
its existing `fetch()` criteria matching but returns only UIDs.
- New test: an attachment-less message under an attachments-only mark-read
rule is recorded as `PROCESSED_WO_CONSUMPTION` on the first run. On a
second run, with the same message still unseen server-side, assert no
body fetch happens (only `M.uids()` is called) -- i.e. steady state costs
one search and zero body downloads.
- New test: a folder with more new UIDs than `MAIL_FETCH_BATCH_SIZE` results
in multiple batched `fetch` calls, and all messages are still processed
(no messages dropped at a batch boundary).
- Update existing `test_handle_empty_message` to assert a
`PROCESSED_WO_CONSUMPTION` row now exists for the no-attachment case.
- Regression: existing tests for `EML_ONLY`/`EVERYTHING` scopes and the
multi-rule-same-folder dedup (`consumed_messages` set) should pass
unchanged, since neither the per-message `already_processed` check nor the
`consumed_messages` set logic is touched by this design.
## Open questions / risks
- `imap_tools`'s `AND(uid=batch)` criteria builder needs to be confirmed to
produce a valid `UID FETCH <list>` command at the batch sizes used here;
covered by the batching test above.
- If a mail server doesn't support `UID SEARCH` the same way it supports the
existing `fetch`'s implicit search, `M.uids()` could behave differently
from today's `M.fetch()` on some edge-case server. Existing project test
coverage (via `BogusMailBox`) won't catch server-specific quirks; this is
the same class of risk as any IMAP-behavior assumption already baked into
this module.
+4 -4
View File
@@ -16,7 +16,7 @@ classifiers = [
dependencies = [
"azure-ai-documentintelligence>=1.0.2",
"babel>=2.17",
"bleach~=6.3.0",
"bleach~=6.4.0",
"celery[redis]~=5.6.2",
"channels~=4.2",
"channels-redis~=4.2",
@@ -56,7 +56,7 @@ dependencies = [
"llama-index-embeddings-openai-like>=0.2.2",
"llama-index-llms-ollama>=0.9.1",
"llama-index-llms-openai-like>=0.7.1",
"nltk~=3.9.1",
"nltk~=3.10.0",
"ocrmypdf~=17.4.2",
"openai>=2.32",
"pathvalidate~=3.3.1",
@@ -101,11 +101,11 @@ dev = [
{ include-group = "testing" },
]
docs = [
"zensical>=0.0.43",
"zensical>=0.0.47",
]
lint = [
"prek~=0.3.10",
"ruff~=0.15.15",
"ruff~=0.15.20",
]
testing = [
"daphne",
+10
View File
@@ -79,6 +79,16 @@
"glob": "{pdf.worker.min.mjs,pdf.min.mjs}",
"input": "node_modules/pdfjs-dist/legacy/build/",
"output": "/assets/js/"
},
{
"glob": "**/*",
"input": "node_modules/pdfjs-dist/wasm/",
"output": "/assets/wasm/"
},
{
"glob": "**/*",
"input": "node_modules/pdfjs-dist/iccs/",
"output": "/assets/iccs/"
}
],
"styles": [
+291 -258
View File
File diff suppressed because it is too large Load Diff
+1254 -1854
View File
File diff suppressed because it is too large Load Diff
@@ -5,8 +5,9 @@
i18n-info>
<div class="input-group input-group-sm align-items-center">
<div class="input-group input-group-sm me-3">
<span class="input-group-text text-muted" i18n>Show</span>
<label class="input-group-text text-muted" for="logLineLimit" i18n>Show</label>
<input
id="logLineLimit"
class="form-control"
type="number"
min="100"
@@ -17,7 +18,7 @@
<span class="input-group-text text-muted" i18n>lines</span>
</div>
<div class="form-check form-switch mt-1">
<input class="form-check-input" type="checkbox" role="switch" [ngModel]="autoRefreshEnabled()" (ngModelChange)="autoRefreshEnabled.set($event)">
<input id="autoRefreshSwitch" class="form-check-input" type="checkbox" role="switch" [ngModel]="autoRefreshEnabled()" (ngModelChange)="autoRefreshEnabled.set($event)">
<label class="form-check-label" for="autoRefreshSwitch" i18n>Auto refresh</label>
</div>
</div>
@@ -100,9 +100,7 @@ export class LogsComponent
parsed.some((log, idx) => {
const current = this.logs()[idx]
return (
!current ||
current.message !== log.message ||
current.level !== log.level
current?.message !== log.message || current?.level !== log.level
)
})
if (hasChanges) {
@@ -37,7 +37,7 @@
<form [formGroup]="settingsForm" (ngSubmit)="saveSettings()">
<ul ngbNav #nav="ngbNav" (navChange)="onNavChange($event)" [(activeId)]="activeNavID" class="nav-tabs">
<ul ngbNav #nav="ngbNav" (navChange)="onNavChange($event)" [activeId]="activeNavID()" (activeIdChange)="activeNavID.set($event)" class="nav-tabs">
<li [ngbNavItem]="SettingsNavIDs.General">
<a ngbNavLink i18n>General</a>
<ng-template ngbNavContent>
@@ -150,10 +150,10 @@
<div class="row mb-3">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Full search links to</span>
<label for="searchLink" i18n>Full search links to</label>
</div>
<div class="col mb-3">
<select class="form-select" formControlName="searchLink">
<select id="searchLink" class="form-select" formControlName="searchLink">
<option [ngValue]="GlobalSearchType.TITLE_CONTENT" i18n>Title and content search</option>
<option [ngValue]="GlobalSearchType.ADVANCED" i18n>Advanced search</option>
</select>
@@ -199,10 +199,10 @@
<h5 i18n>Documents</h5>
<div class="row mb-3">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Items per page</span>
<label for="documentListItemPerPage" i18n>Items per page</label>
</div>
<div class="col">
<select class="form-select" formControlName="documentListItemPerPage">
<select id="documentListItemPerPage" class="form-select" formControlName="documentListItemPerPage">
<option [ngValue]="10">10</option>
<option [ngValue]="25">25</option>
<option [ngValue]="50">50</option>
@@ -274,10 +274,10 @@
<h5 class="mt-3" i18n>PDF Editor</h5>
<div class="row">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Default editing mode</span>
<label for="pdfEditorDefaultEditMode" i18n>Default editing mode</label>
</div>
<div class="col">
<select class="form-select" formControlName="pdfEditorDefaultEditMode">
<select id="pdfEditorDefaultEditMode" class="form-select" formControlName="pdfEditorDefaultEditMode">
<option [ngValue]="PdfEditorEditMode.Create" i18n>Create new document(s)</option>
<option [ngValue]="PdfEditorEditMode.Update" i18n>Add document version</option>
</select>
@@ -242,7 +242,7 @@ describe('SettingsComponent', () => {
activatedRoute.snapshot.fragment = '#notifications'
const scrollSpy = jest.spyOn(viewportScroller, 'scrollToAnchor')
component.ngOnInit()
expect(component.activeNavID).toEqual(4) // Notifications
expect(component.activeNavID()).toEqual(4) // Notifications
component.ngAfterViewInit()
expect(scrollSpy).toHaveBeenCalledWith('#notifications')
})
@@ -399,17 +399,17 @@ describe('SettingsComponent', () => {
completeSetup()
const field = 'storage_path'
expect(
component.settingsForm.get('documentDetailsHiddenFields').value.length
).toEqual(0)
component.settingsForm.get('documentDetailsHiddenFields').value
).toHaveLength(0)
component.toggleDocumentDetailField(field, false)
expect(
component.settingsForm.get('documentDetailsHiddenFields').value.length
).toEqual(1)
component.settingsForm.get('documentDetailsHiddenFields').value
).toHaveLength(1)
expect(component.isDocumentDetailFieldShown(field)).toBeFalsy()
component.toggleDocumentDetailField(field, true)
expect(
component.settingsForm.get('documentDetailsHiddenFields').value.length
).toEqual(0)
component.settingsForm.get('documentDetailsHiddenFields').value
).toHaveLength(0)
expect(component.isDocumentDetailFieldShown(field)).toBeTruthy()
})
})
@@ -142,7 +142,7 @@ export class SettingsComponent
private systemStatusService = inject(SystemStatusService)
private savedViewsService = inject(SavedViewService)
activeNavID: number
readonly activeNavID = signal<number>(undefined)
settingsForm = new FormGroup({
bulkEditConfirmationDialogs: new FormControl(null),
@@ -283,7 +283,7 @@ export class SettingsComponent
(navID) => navID.toLowerCase() == section
)
if (navIDKey) {
this.activeNavID = SettingsNavIDs[navIDKey]
this.activeNavID.set(SettingsNavIDs[navIDKey])
}
}
})
@@ -386,7 +386,7 @@ export class SettingsComponent
.navigate(['settings', foundNavIDkey.toLowerCase()])
.then((navigated) => {
if (!navigated && this.isDirty) {
this.activeNavID = navChangeEvent.activeId
this.activeNavID.set(navChangeEvent.activeId)
} else if (navigated && this.isDirty) {
this.initialize()
}
@@ -15,7 +15,7 @@
<i-bs name="check2-all" class="me-1"></i-bs><ng-container i18n>Dismiss all</ng-container>
</button>
<div class="form-check form-switch mb-0 ms-2">
<input class="form-check-input" type="checkbox" role="switch" [(ngModel)]="autoRefreshEnabled">
<input id="autoRefreshSwitch" class="form-check-input" type="checkbox" role="switch" [(ngModel)]="autoRefreshEnabled">
<label class="form-check-label" for="autoRefreshSwitch" i18n>Auto refresh</label>
</div>
</div>
@@ -93,7 +93,8 @@
<i-bs width="1em" height="1em" name="x"></i-bs>
</button>
}
<input #filterInput class="form-control form-control-sm" type="text"
<label class="visually-hidden" for="taskSearch" i18n>Search tasks</label>
<input #filterInput id="taskSearch" class="form-control form-control-sm" type="text"
(keyup)="filterInputKeyup($event)"
[(ngModel)]="filterText">
</div>
@@ -211,7 +212,7 @@
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
</button>
</td>
<td scope="row" class="actions-col">
<td class="actions-col">
<div class="btn-group" role="group">
<button class="btn btn-sm btn-outline-secondary" (click)="dismissTask(task); $event.stopPropagation();" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.PaperlessTask }">
<i-bs name="check" class="me-1"></i-bs><ng-container i18n>Dismiss</ng-container>
@@ -150,7 +150,8 @@
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" name="file-text"></i-bs><span>{{d.title | documentTitle}}</span>
<span class="close flex-column justify-content-center" (click)="closeDocument(d); $event.preventDefault()">
<span class="close flex-column justify-content-center"
(click)="$event.preventDefault(); $event.stopPropagation(); closeDocument(d)">
<i-bs name="x"></i-bs>
</span>
</a>
@@ -6,7 +6,7 @@
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="chatDropdown">
<div class="chat-container bg-light p-2">
<div class="chat-messages font-monospace small">
@for (message of messages; track message) {
@for (message of messages(); track message) {
<div class="message d-flex flex-row small" [class.justify-content-end]="message.role === 'user'">
<div class="p-2 m-2" [class.bg-body]="message.role === 'user'">
<span>
@@ -29,16 +29,19 @@
</div>
<form class="chat-input">
<label class="visually-hidden" for="chatInput" i18n>Message</label>
<div class="input-group">
<input
#chatInput
id="chatInput"
class="form-control form-control-sm" name="chatInput" type="text"
[placeholder]="placeholder"
[disabled]="loading"
[(ngModel)]="input"
[disabled]="loading()"
[ngModel]="input()"
(ngModelChange)="input.set($event)"
(keydown)="searchInputKeyDown($event)"
/>
<button class="btn btn-sm btn-secondary" type="button" (click)="sendMessage()" [disabled]="loading">Send</button>
<button class="btn btn-sm btn-secondary" type="button" (click)="sendMessage()" [disabled]="loading()">Send</button>
</div>
</form>
</div>
@@ -56,44 +56,53 @@ describe('ChatComponent', () => {
it('should update documentId on initialization', () => {
jest.spyOn(router, 'url', 'get').mockReturnValue('/documents/123')
component.ngOnInit()
expect(component.documentId).toBe(123)
expect(component.documentId()).toBe(123)
})
it('should update documentId on navigation', () => {
component.ngOnInit()
routerEvents$.next(new NavigationEnd(1, '/documents/456', '/documents/456'))
expect(component.documentId).toBe(456)
expect(component.documentId()).toBe(456)
})
it('should return correct placeholder based on documentId', () => {
component.documentId = 123
component.documentId.set(123)
expect(component.placeholder).toBe('Ask a question about this document...')
component.documentId = undefined
component.documentId.set(undefined)
expect(component.placeholder).toBe('Ask a question about a document...')
})
it('should send a message and handle streaming response', () => {
component.input = 'Hello'
it('should send a message and render the streaming response', async () => {
component.input.set('Hello')
component.sendMessage()
expect(component.messages.length).toBe(2)
expect(component.messages[0].content).toBe('Hello')
expect(component.loading).toBe(true)
expect(component.messages()).toHaveLength(2)
expect(component.messages()[0].content).toBe('Hello')
expect(component.loading()).toBe(true)
mockStream$.next('Hi')
expect(component.messages[1].content).toBe('H')
expect(component.messages()[1].content).toBe('H')
mockStream$.next('Hi there')
// advance time to process the typewriter effect
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe('Hi there')
await jest.runAllTimersAsync()
await fixture.whenStable()
expect(component.messages()[1].content).toBe('Hi there')
expect(
fixture.nativeElement.querySelector('.chat-messages').textContent
).toContain('Hi there')
mockStream$.complete()
expect(component.loading).toBe(false)
expect(component.messages[1].isStreaming).toBe(false)
await jest.runAllTimersAsync()
await fixture.whenStable()
expect(component.loading()).toBe(false)
expect(component.messages()[1].isStreaming).toBe(false)
expect(fixture.nativeElement.querySelector('#chatInput').disabled).toBe(
false
)
})
it('should parse references from the metadata trailer without showing it', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.next(
@@ -101,14 +110,14 @@ describe('ChatComponent', () => {
)
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe('Hi there')
expect(component.messages[1].references).toEqual([
expect(component.messages()[1].content).toBe('Hi there')
expect(component.messages()[1].references).toEqual([
{ id: 42, title: 'Bread Recipe' },
])
})
it('should render document reference links under assistant messages', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.next(
@@ -123,12 +132,12 @@ describe('ChatComponent', () => {
})
it('should remove delimiter fragments that were already streamed', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.next(`Hi there${CHAT_METADATA_DELIMITER.slice(0, 8)}`)
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe(
expect(component.messages()[1].content).toBe(
`Hi there${CHAT_METADATA_DELIMITER.slice(0, 8)}`
)
@@ -137,28 +146,28 @@ describe('ChatComponent', () => {
)
jest.advanceTimersByTime(1000)
expect(component.messages[1].content).toBe('Hi there')
expect(component.messages[1].references).toEqual([
expect(component.messages()[1].content).toBe('Hi there')
expect(component.messages()[1].references).toEqual([
{ id: 42, title: 'Bread Recipe' },
])
})
it('should handle errors during streaming', () => {
component.input = 'Hello'
component.input.set('Hello')
component.sendMessage()
mockStream$.error('Error')
expect(component.messages[1].content).toContain(
expect(component.messages()[1].content).toContain(
'⚠️ Error receiving response.'
)
expect(component.loading).toBe(false)
expect(component.loading()).toBe(false)
})
it('should enqueue typewriter chunks correctly', () => {
const message = { content: '', role: 'assistant', isStreaming: true }
component.enqueueTypewriter(null, message as any) // coverage for null
component.enqueueTypewriter('Hello', message as any)
expect(component['typewriterBuffer'].length).toBe(4)
expect(component['typewriterBuffer']).toHaveLength(4)
})
it('should scroll to bottom after sending a message', () => {
@@ -166,7 +175,7 @@ describe('ChatComponent', () => {
ChatComponent.prototype as any,
'scrollToBottom'
)
component.input = 'Test'
component.input.set('Test')
component.sendMessage()
expect(scrollSpy).toHaveBeenCalled()
})
@@ -1,4 +1,11 @@
import { Component, ElementRef, inject, OnInit, ViewChild } from '@angular/core'
import {
Component,
ElementRef,
inject,
OnInit,
signal,
ViewChild,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NavigationEnd, Router, RouterModule } from '@angular/router'
import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
@@ -23,10 +30,10 @@ import {
styleUrl: './chat.component.scss',
})
export class ChatComponent implements OnInit {
public messages: ChatMessage[] = []
public loading = false
public input: string = ''
public documentId!: number
readonly messages = signal<ChatMessage[]>([])
readonly loading = signal(false)
readonly input = signal('')
readonly documentId = signal<number>(undefined)
private chatService: ChatService = inject(ChatService)
private router: Router = inject(Router)
@@ -38,7 +45,7 @@ export class ChatComponent implements OnInit {
private typewriterActive = false
public get placeholder(): string {
return this.documentId
return this.documentId()
? $localize`Ask a question about this document...`
: $localize`Ask a question about a document...`
}
@@ -57,14 +64,14 @@ export class ChatComponent implements OnInit {
private updateDocumentId(url: string): void {
const docIdRe = url.match(/^\/documents\/(\d+)/)
this.documentId = docIdRe ? +docIdRe[1] : undefined
this.documentId.set(docIdRe ? +docIdRe[1] : undefined)
}
sendMessage(): void {
if (!this.input.trim()) return
if (!this.input().trim()) return
const userMessage: ChatMessage = { role: 'user', content: this.input }
this.messages.push(userMessage)
const userMessage: ChatMessage = { role: 'user', content: this.input() }
this.messages.update((messages) => [...messages, userMessage])
this.scrollToBottom()
const assistantMessage: ChatMessage = {
@@ -72,12 +79,12 @@ export class ChatComponent implements OnInit {
content: '',
isStreaming: true,
}
this.messages.push(assistantMessage)
this.loading = true
this.messages.update((messages) => [...messages, assistantMessage])
this.loading.set(true)
let lastVisibleContent = ''
this.chatService.streamChat(this.documentId, this.input).subscribe({
this.chatService.streamChat(this.documentId(), this.input()).subscribe({
next: (chunk) => {
const nextResponse = parseChatResponse(chunk)
@@ -93,26 +100,30 @@ export class ChatComponent implements OnInit {
}
assistantMessage.references = nextResponse.references
this.notifyMessagesChanged()
},
error: () => {
assistantMessage.content += '\n\n⚠️ Error receiving response.'
assistantMessage.isStreaming = false
this.loading = false
this.notifyMessagesChanged()
this.loading.set(false)
},
complete: () => {
assistantMessage.isStreaming = false
this.loading = false
this.notifyMessagesChanged()
this.loading.set(false)
this.scrollToBottom()
},
})
this.input = ''
this.input.set('')
}
private resetTypewriter(message: ChatMessage, content: string): void {
this.typewriterBuffer = []
this.typewriterActive = false
message.content = content
this.notifyMessagesChanged()
this.scrollToBottom()
}
@@ -135,11 +146,16 @@ export class ChatComponent implements OnInit {
const nextChar = this.typewriterBuffer.shift()
message.content += nextChar
this.notifyMessagesChanged()
this.scrollToBottom()
setTimeout(() => this.playTypewriter(message), 10) // 10ms per character
}
private notifyMessagesChanged(): void {
this.messages.update((messages) => [...messages])
}
private scrollToBottom(): void {
setTimeout(() => {
this.scrollAnchor?.nativeElement?.scrollIntoView({ behavior: 'smooth' })
@@ -1,5 +1,12 @@
import { CurrencyPipe, getLocaleCurrencyCode, SlicePipe } from '@angular/common'
import { Component, inject, Input, LOCALE_ID, OnInit } from '@angular/core'
import {
ChangeDetectorRef,
Component,
inject,
Input,
LOCALE_ID,
OnInit,
} from '@angular/core'
import { NgbTooltipModule } from '@ng-bootstrap/ng-bootstrap'
import { takeUntil } from 'rxjs'
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
@@ -22,6 +29,7 @@ export class CustomFieldDisplayComponent
{
private customFieldService = inject(CustomFieldsService)
private documentService = inject(DocumentService)
private changeDetector = inject(ChangeDetectorRef)
CustomFieldDataType = CustomFieldDataType
@@ -74,6 +82,7 @@ export class CustomFieldDisplayComponent
this.customFieldService.listAll().subscribe((r) => {
this.customFields = r.results
this.init()
this.changeDetector.markForCheck()
})
}
@@ -111,6 +120,7 @@ export class CustomFieldDisplayComponent
this.docLinkDocuments = this.value
.map((id) => result.results.find((d) => d.id === id))
.filter((d) => d)
this.changeDetector.markForCheck()
})
}
@@ -95,8 +95,8 @@ describe('CustomFieldsDropdownComponent', () => {
it('should support update unused fields', () => {
component.existingFields = [{ field: fields[0].id } as any]
component['updateUnusedFields']()
expect(component['unusedFields'].length).toEqual(1)
expect(component['unusedFields'][0].name).toEqual('Field 2')
expect(component['unusedFields']().length).toEqual(1)
expect(component['unusedFields']()[0].name).toEqual('Field 2')
})
it('should support getting data type label', () => {
@@ -8,6 +8,7 @@ import {
ViewChild,
ViewChildren,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbDropdownModule, NgbModal } from '@ng-bootstrap/ng-bootstrap'
@@ -65,11 +66,11 @@ export class CustomFieldsDropdownComponent extends LoadingComponentWithPermissio
@ViewChildren('button') buttons: QueryList<ElementRef>
private customFields: CustomField[] = []
private unusedFields: CustomField[] = []
private readonly unusedFields = signal<CustomField[]>([])
private keyboardIndex: number
public get filteredFields(): CustomField[] {
return this.unusedFields.filter(
return this.unusedFields().filter(
(f) => !this.filterText || matchesSearchText(f.name, this.filterText)
)
}
@@ -99,8 +100,10 @@ export class CustomFieldsDropdownComponent extends LoadingComponentWithPermissio
}
private updateUnusedFields() {
this.unusedFields = this.customFields.filter(
(f) => !this.existingFields?.find((e) => e.field === f.id)
this.unusedFields.set(
this.customFields.filter(
(f) => !this.existingFields?.find((e) => e.field === f.id)
)
)
}
@@ -77,7 +77,7 @@
<div class="input-group input-group-sm">
<ng-select #fieldSelects
class="paperless-input-select"
[items]="customFields"
[items]="customFields()"
[(ngModel)]="atom.field"
[disabled]="disabled"
bindLabel="name"
@@ -79,7 +79,7 @@ describe('CustomFieldsQueryDropdownComponent', () => {
})
it('should initialize custom fields on creation', () => {
expect(component.customFields).toEqual(customFields)
expect(component.customFields()).toEqual(customFields)
})
it('should add an expression when opened if queries are empty', () => {
@@ -101,7 +101,7 @@ describe('CustomFieldsQueryDropdownComponent', () => {
data_type: CustomFieldDataType.String,
extra_data: {},
}
component.customFields = [field]
component.customFields.set([field])
const operators = component.getOperatorsForField(1)
expect(operators.length).toEqual(
[
@@ -138,7 +138,7 @@ describe('CustomFieldsQueryDropdownComponent', () => {
],
},
}
component.customFields = [field]
component.customFields.set([field])
const options = component.getSelectOptionsForField(1)
expect(options).toEqual([
{ label: 'Option 1', id: 'abc-123' },
@@ -6,6 +6,7 @@ import {
Input,
Output,
QueryList,
signal,
ViewChild,
ViewChildren,
} from '@angular/core'
@@ -278,7 +279,7 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
@Output()
selectionModelChange = new EventEmitter<CustomFieldQueriesModel>()
customFields: CustomField[] = []
readonly customFields = signal<CustomField[]>([])
public readonly today: string = new Date().toLocaleDateString('en-CA')
@@ -325,12 +326,12 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
.listAll()
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe((result) => {
this.customFields = result.results
this.customFields.set(result.results)
})
}
public getCustomFieldByID(id: number): CustomField {
return this.customFields.find((field) => field.id === id)
return this.customFields().find((field) => field.id === id)
}
public addAtom(expression: CustomFieldQueryExpression) {
@@ -353,7 +354,7 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
getOperatorsForField(
fieldID: number
): Array<{ value: string; label: string }> {
const field = this.customFields.find((field) => field.id === fieldID)
const field = this.customFields().find((field) => field.id === fieldID)
const groups: CustomFieldQueryOperatorGroups[] = field
? CUSTOM_FIELD_QUERY_OPERATOR_GROUPS_BY_TYPE[field.data_type]
: [CustomFieldQueryOperatorGroups.Basic]
@@ -369,7 +370,7 @@ export class CustomFieldsQueryDropdownComponent extends LoadingComponentWithPerm
getSelectOptionsForField(
fieldID: number
): Array<{ label: string; id: string }> {
const field = this.customFields.find((field) => field.id === fieldID)
const field = this.customFields().find((field) => field.id === fieldID)
if (field) {
return field.extra_data['select_options']
}
@@ -22,11 +22,11 @@
<ng-template>
<div class="card mb-2">
<div class="card-body p-2">
@if (testLoading) {
@if (testLoading()) {
<ng-container [ngTemplateOutlet]="loadingTemplate"></ng-container>
} @else if (testResult) {
<code>{{testResult}}</code>
} @else if (testFailed) {
} @else if (testResult()) {
<code>{{testResult()}}</code>
} @else if (testFailed()) {
<div class="text-danger" i18n>Path test failed</div>
} @else {
<div class="text-muted small" i18n>No document selected</div>
@@ -42,7 +42,7 @@
[compareWith]="compareDocuments"
[trackByFn]="trackByFn"
[minTermLength]="2"
[loading]="loading"
[loading]="loading()"
[typeahead]="documentsInput$"
(change)="testPath($event)">
<ng-template #loadingTemplate ng-loadingspinner-tmp>
@@ -58,17 +58,17 @@ describe('StoragePathEditDialogComponent', () => {
fixture.detectChanges()
component.testPath({ id: 1 })
expect(testSpy).toHaveBeenCalledWith('test/{{title}}', 1)
expect(component.testResult).toBe('test/abc123')
expect(component.testFailed).toBeFalsy()
expect(component.testResult()).toBe('test/abc123')
expect(component.testFailed()).toBeFalsy()
// test failed
testSpy.mockReturnValueOnce(of(''))
component.testPath({ id: 1 })
expect(component.testResult).toBeNull()
expect(component.testFailed).toBeTruthy()
expect(component.testResult()).toBeNull()
expect(component.testFailed()).toBeTruthy()
component.testPath(null)
expect(component.testResult).toBeNull()
expect(component.testResult()).toBeNull()
})
it('should compare two documents by id', () => {
@@ -1,5 +1,5 @@
import { AsyncPipe, NgTemplateOutlet } from '@angular/common'
import { Component, OnDestroy, inject } from '@angular/core'
import { Component, OnDestroy, inject, signal } from '@angular/core'
import {
FormControl,
FormGroup,
@@ -65,9 +65,9 @@ export class StoragePathEditDialogComponent
public documentsInput$ = new Subject<string>()
public foundDocuments$: Observable<Document[]>
private testDocument: Document
public testResult: string
public testFailed: boolean = false
public testLoading = false
readonly testResult = signal<string>(undefined)
readonly testFailed = signal(false)
readonly testLoading = signal(false)
constructor() {
super()
@@ -99,22 +99,22 @@ export class StoragePathEditDialogComponent
public testPath(document: Document) {
if (!document) {
this.testResult = null
this.testResult.set(null)
return
}
this.testDocument = document
this.testLoading = true
this.testLoading.set(true)
;(this.service as StoragePathService)
.testPath(this.objectForm.get('path').value, document.id)
.subscribe((result) => {
if (result?.length) {
this.testResult = result
this.testFailed = false
this.testResult.set(result)
this.testFailed.set(false)
} else {
this.testResult = null
this.testFailed = true
this.testResult.set(null)
this.testFailed.set(true)
}
this.testLoading = false
this.testLoading.set(false)
})
}
@@ -12,7 +12,7 @@
<pngx-input-color i18n-title title="Color" formControlName="color" [error]="error?.color"></pngx-input-color>
<pngx-input-select i18n-title title="Parent" formControlName="parent" [items]="tags" [allowNull]="true" [error]="error?.parent"></pngx-input-select>
<pngx-input-select i18n-title title="Parent" formControlName="parent" [items]="tags()" [allowNull]="true" [error]="error?.parent"></pngx-input-select>
<pngx-input-check i18n-title title="Inbox tag" formControlName="is_inbox_tag" i18n-hint hint="Inbox tags are automatically assigned to all consumed documents."></pngx-input-check>
<pngx-input-select i18n-title title="Matching algorithm" [items]="getMatchingAlgorithms()" formControlName="matching_algorithm"></pngx-input-select>
@@ -1,4 +1,4 @@
import { Component, inject } from '@angular/core'
import { Component, inject, signal } from '@angular/core'
import {
FormControl,
FormGroup,
@@ -35,7 +35,7 @@ import { TextComponent } from '../../input/text/text.component'
],
})
export class TagEditDialogComponent extends EditDialogComponent<Tag> {
tags: Tag[]
readonly tags = signal<Tag[]>([])
constructor() {
super()
@@ -43,7 +43,7 @@ export class TagEditDialogComponent extends EditDialogComponent<Tag> {
this.userService = inject(UserService)
this.settingsService = inject(SettingsService)
this.service.listAll().subscribe((result) => {
this.tags = result.results
this.tags.set(result.results)
})
}
@@ -180,7 +180,7 @@
<div class="col">
<div class="trigger-filters mb-3">
<div class="d-flex align-items-center">
<label class="form-label mb-0" i18n>Advanced Filters</label>
<span class="form-label mb-0" i18n>Advanced Filters</span>
<button
type="button"
class="btn btn-sm btn-outline-primary ms-auto"
@@ -401,7 +401,7 @@ describe('WorkflowEditDialogComponent', () => {
component.addFilter(triggerGroup as FormGroup)
const filters = component.getFiltersFormArray(triggerGroup as FormGroup)
expect(filters.length).toBe(3)
expect(filters).toHaveLength(3)
filters.at(0).get('values').setValue([1])
filters.at(1).get('values').setValue([2, 3])
@@ -672,7 +672,7 @@ describe('WorkflowEditDialogComponent', () => {
} as any
const filters = component['buildFiltersFormArray'](trigger)
expect(filters.length).toBe(0)
expect(filters).toHaveLength(0)
component.filterDefinitions = originalDefinitions
})
@@ -739,7 +739,7 @@ describe('WorkflowEditDialogComponent', () => {
component.ngOnInit()
const triggerGroup = component.triggerFields.at(0) as FormGroup
const filters = component.getFiltersFormArray(triggerGroup)
expect(filters.length).toBe(13)
expect(filters).toHaveLength(13)
const customFieldFilter = filters.at(12) as FormGroup
expect(customFieldFilter.get('type').value).toBe(
TriggerFilterType.CustomFieldQuery
@@ -969,12 +969,12 @@ describe('WorkflowEditDialogComponent', () => {
component.addFilter(triggerGroup)
component.removeFilter(triggerGroup, 0)
expect(component.getFiltersFormArray(triggerGroup).length).toBe(0)
expect(component.getFiltersFormArray(triggerGroup)).toHaveLength(0)
component.addFilter(triggerGroup)
const filterArrayAfterAdd = component.getFiltersFormArray(triggerGroup)
filterArrayAfterAdd.at(0).get('type').setValue(TriggerFilterType.TagsAll)
expect(component.getFiltersFormArray(triggerGroup).length).toBe(1)
expect(component.getFiltersFormArray(triggerGroup)).toHaveLength(1)
})
it('should remove selected custom field from the form group', () => {
@@ -933,7 +933,7 @@ export class WorkflowEditDialogComponent
getFilterSelectItems(type: TriggerFilterType) {
const definition = this.getFilterDefinition(type)
if (!definition || definition.inputType !== 'select') {
if (definition?.inputType !== 'select') {
return []
}
@@ -58,8 +58,8 @@
</button>
}
@if ((selectionModel.items | filter: filterText:'name').length > 0) {
<button class="list-group-item list-group-item-action bg-light d-flex align-items-center" (click)="applyClicked()" [disabled]="!modelIsDirty || disabled">
<small class="ms-2" [ngClass]="{'fw-bold': modelIsDirty}" i18n>Apply</small>
<button class="list-group-item list-group-item-action bg-light d-flex align-items-center" (click)="applyClicked()" [disabled]="!modelIsDirty() || disabled">
<small class="ms-2" [ngClass]="{'fw-bold': modelIsDirty()}" i18n>Apply</small>
<i-bs width="1.5em" height="1em" name="arrow-right"></i-bs>
</button>
}
@@ -221,7 +221,7 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
.dispatchEvent(new MouseEvent('click')) // open
selectionModel.toggle(items[0].id)
fixture.detectChanges()
expect(component.modelIsDirty).toBeTruthy()
expect(component.modelIsDirty()).toBeTruthy()
let applyResult: ChangedItems
const closeSpy = jest.spyOn(component.dropdown, 'close')
component.apply.subscribe((result) => (applyResult = result))
@@ -244,7 +244,7 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
.dispatchEvent(new MouseEvent('click')) // open
selectionModel.toggle(items[0].id)
fixture.detectChanges()
expect(component.modelIsDirty).toBeTruthy()
expect(component.modelIsDirty()).toBeTruthy()
let applyResult: ChangedItems
component.apply.subscribe((result) => (applyResult = result))
component.dropdown.close()
@@ -12,6 +12,7 @@ import {
Output,
ViewChild,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbDropdown, NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
@@ -796,7 +797,7 @@ export class FilterableDropdownComponent
return this.title ? this.title.replace(/\s/g, '_').toLowerCase() : null
}
modelIsDirty: boolean = false
readonly modelIsDirty = signal(false)
private keyboardIndex: number
@@ -811,7 +812,7 @@ export class FilterableDropdownComponent
constructor() {
super()
this.selectionModelChange.subscribe((updatedModel) => {
this.modelIsDirty = updatedModel.isDirty()
this.modelIsDirty.set(updatedModel.isDirty())
})
}
@@ -858,7 +859,7 @@ export class FilterableDropdownComponent
}, 0)
if (this.editing) {
this.selectionModel.reset()
this.modelIsDirty = false
this.modelIsDirty.set(false)
}
this.selectionModel.singleSelect =
this.editing && !this.selectionModel.manyToOne
@@ -5,6 +5,7 @@ import {
inject,
Input,
Output,
signal,
} from '@angular/core'
import {
FormsModule,
@@ -63,11 +64,11 @@ export class CustomFieldsValuesComponent extends AbstractInputComponent<Object>
super()
customFieldsService.listAll().subscribe((items) => {
this.fields = items.results
this.fields.set(items.results)
})
}
private fields: CustomField[]
private readonly fields = signal<CustomField[]>([])
private _selectedFields: number[]
@@ -90,6 +91,6 @@ export class CustomFieldsValuesComponent extends AbstractInputComponent<Object>
public removeSelectedField: EventEmitter<number> = new EventEmitter<number>()
public getCustomField(id: number): CustomField {
return this.fields.find((field) => field.id === id)
return this.fields().find((field) => field.id === id)
}
}
@@ -33,7 +33,7 @@
[compareWith]="compareDocuments"
[trackByFn]="trackByFn"
[minTermLength]="2"
[loading]="loading"
[loading]="loading()"
[typeahead]="documentsInput$"
(mousedown)="$event.stopImmediatePropagation()"
(change)="onChange(selectedDocumentIDs)">
@@ -2,7 +2,11 @@ import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NG_VALUE_ACCESSOR } from '@angular/forms'
import { of, throwError } from 'rxjs'
import { By } from '@angular/platform-browser'
import { provideRouter } from '@angular/router'
import { NgSelectComponent } from '@ng-select/ng-select'
import { allIcons, NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { of, Subject, throwError } from 'rxjs'
import { FILTER_SIMPLE_TITLE } from 'src/app/data/filter-rule-type'
import { DocumentService } from 'src/app/services/rest/document.service'
import { DocumentLinkComponent } from './document-link.component'
@@ -33,10 +37,11 @@ describe('DocumentLinkComponent', () => {
beforeEach(() => {
TestBed.configureTestingModule({
imports: [DocumentLinkComponent],
imports: [DocumentLinkComponent, NgxBootstrapIconsModule.pick(allIcons)],
providers: [
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
provideRouter([]),
],
})
documentService = TestBed.inject(DocumentService)
@@ -60,6 +65,25 @@ describe('DocumentLinkComponent', () => {
expect(getSpy).toHaveBeenCalled()
})
it('should render loading and selected documents after async updates', async () => {
const result$ = new Subject<any>()
jest.spyOn(documentService, 'getFew').mockReturnValue(result$)
component.writeValue([1])
await fixture.whenStable()
const select = fixture.debugElement.query(By.directive(NgSelectComponent))
.componentInstance as NgSelectComponent
expect(select.loading()).toBe(true)
result$.next({ count: 1, all: [1], results: [documents[0]] })
result$.complete()
await fixture.whenStable()
expect(select.loading()).toBe(false)
expect(fixture.nativeElement.textContent).toContain(documents[0].title)
})
it('shoud maintain ordering of selected documents', () => {
const getSpy = jest.spyOn(documentService, 'getFew')
getSpy.mockImplementation((ids) => {
@@ -6,6 +6,7 @@ import {
Input,
OnDestroy,
OnInit,
signal,
} from '@angular/core'
import {
FormsModule,
@@ -63,7 +64,7 @@ export class DocumentLinkComponent
documentsInput$ = new Subject<string>()
foundDocuments$: Observable<Document[]>
loading = false
readonly loading = signal(false)
selectedDocuments: Document[] = []
private unsubscribeNotifier: Subject<any> = new Subject()
@@ -93,12 +94,12 @@ export class DocumentLinkComponent
this.selectedDocuments = []
super.writeValue([])
} else {
this.loading = true
this.loading.set(true)
this.documentsService
.getFew(documentIDs, { fields: 'id,title' })
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((documentResults) => {
this.loading = false
this.loading.set(false)
this.selectedDocuments = documentIDs.map(
(id) => documentResults.results.find((d) => d.id === id) ?? {}
)
@@ -113,7 +114,7 @@ export class DocumentLinkComponent
this.documentsInput$.pipe(
distinctUntilChanged(),
takeUntil(this.unsubscribeNotifier),
tap(() => (this.loading = true)),
tap(() => this.loading.set(true)),
switchMap((title) =>
this.documentsService
.listFiltered(
@@ -133,7 +134,7 @@ export class DocumentLinkComponent
)
),
catchError(() => of([])), // empty on error
tap(() => (this.loading = false))
tap(() => this.loading.set(false))
)
)
)
@@ -37,6 +37,7 @@ export class NumberComponent extends AbstractInputComponent<number> {
this.documentService.getNextAsn().subscribe((nextAsn) => {
this.value = nextAsn
this.onChange(this.value)
this.changeDetector.markForCheck()
})
}
@@ -157,8 +157,8 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
private removeChildren(tagIDs: number[], tag: Tag) {
if (tag.children?.length) {
const childIDs = tag.children.map((child) => child.id)
tagIDs = tagIDs.filter((id) => !childIDs.includes(id))
const childIDs = new Set(tag.children.map((child) => child.id))
tagIDs = tagIDs.filter((id) => !childIDs.has(id))
for (const child of tag.children) {
tagIDs = this.removeChildren(tagIDs, child)
}
@@ -14,7 +14,7 @@ import { environment } from 'src/environments/environment'
})
export class PageHeaderComponent {
private titleService = inject(Title)
private clipboard = inject(Clipboard)
private readonly clipboard = inject(Clipboard)
readonly id = input<number>(undefined)
readonly subTitle = input('')
@@ -49,8 +49,7 @@ describe('PngxPdfViewerComponent', () => {
new URL('assets/js/pdf.worker.min.mjs', document.baseURI).toString()
)
const isVisible = (component as any).findController.onIsPageVisible as
| (() => boolean)
| undefined
(() => boolean) | undefined
expect(isVisible?.()).toBe(true)
expect(loadSpy).toHaveBeenCalledWith(
expect.objectContaining({ numPages: 1 })
@@ -61,6 +60,7 @@ describe('PngxPdfViewerComponent', () => {
it('resolves the worker source relative to the document base URI', async () => {
setBaseHref('/paperless/')
const getDocumentSpy = jest.spyOn(pdfjs, 'getDocument')
await initComponent()
@@ -70,6 +70,13 @@ describe('PngxPdfViewerComponent', () => {
expect(pdfjs.GlobalWorkerOptions.workerSrc).toContain(
'/paperless/assets/js/pdf.worker.min.mjs'
)
expect(getDocumentSpy).toHaveBeenCalledWith({
url: 'test.pdf',
password: undefined,
withCredentials: true,
wasmUrl: expect.stringContaining('/paperless/assets/wasm/'),
iccUrl: expect.stringContaining('/paperless/assets/iccs/'),
})
})
it('initializes single-page viewer and disables text layer', async () => {
@@ -174,10 +174,12 @@ export class PngxPdfViewerComponent
'assets/js/pdf.worker.min.mjs',
this.document.baseURI
).toString()
let initOptions = {
const initOptions = {
url: this.src,
password: this.password,
withCredentials: true,
wasmUrl: new URL('assets/wasm/', this.document.baseURI).toString(),
iccUrl: new URL('assets/iccs/', this.document.baseURI).toString(),
}
this.loadingTask = getDocument(initOptions)
try {
@@ -68,7 +68,7 @@
[(ngModel)]="selectionModel.includeUsers"
[disabled]="disabled"
[clearable]="false"
[items]="users"
[items]="users()"
bindLabel="username"
multiple="true"
bindValue="id"
@@ -1,5 +1,12 @@
import { NgClass } from '@angular/common'
import { Component, EventEmitter, Input, Output, inject } from '@angular/core'
import {
Component,
EventEmitter,
Input,
Output,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectComponent } from '@ng-select/ng-select'
@@ -75,7 +82,7 @@ export class PermissionsFilterDropdownComponent extends ComponentWithPermissions
@Output()
ownerFilterSet = new EventEmitter<PermissionsSelectionModel>()
users: User[]
readonly users = signal<User[]>([])
hideUnowned: boolean
@@ -102,7 +109,7 @@ export class PermissionsFilterDropdownComponent extends ComponentWithPermissions
.listAll()
.pipe(first())
.subscribe({
next: (result) => (this.users = result.results),
next: (result) => this.users.set(result.results),
})
}
}
@@ -57,15 +57,15 @@
</div>
</div>
<div class="col-12 col-md-6">
@if (socialAccounts?.length > 0) {
@if (socialAccounts().length > 0) {
<div class="mb-3">
<p i18n>Connected social accounts</p>
<ul class="list-group">
@for (account of socialAccounts; track account.id) {
@for (account of socialAccounts(); track account.id) {
<li class="list-group-item"
ngbPopover="Set a password before disconnecting social account."
i18n-ngbPopover
[disablePopover]="hasUsablePassword"
[disablePopover]="hasUsablePassword()"
triggers="mouseenter:mouseleave">
{{account.name}} ({{account.provider}})
<pngx-confirm-button
@@ -75,7 +75,7 @@
i18n-title
buttonClasses="btn-outline-danger btn-sm ms-2 align-baseline"
iconName="trash"
[disabled]="!hasUsablePassword"
[disabled]="!hasUsablePassword()"
(confirm)="disconnectSocialAccount(account.id)">
</pngx-confirm-button>
</li>
@@ -84,11 +84,11 @@
<div class="form-text text-muted text-end fst-italic" i18n>Warning: disconnecting social accounts cannot be undone</div>
</div>
}
@if (socialAccountProviders?.length > 0) {
@if (socialAccountProviders().length > 0) {
<div class="mb-3">
<p i18n>Connect new social account</p>
<div class="list-group">
@for (provider of socialAccountProviders; track provider.name) {
@for (provider of socialAccountProviders(); track provider.name) {
<a class="list-group-item list-group-item-action text-primary d-flex align-items-center" href="{{ provider.login_url }}" rel="noopener noreferrer">
{{provider.name}}<i-bs class="pb-1 ms-2" name="box-arrow-up-right"></i-bs>
</a>
@@ -96,7 +96,7 @@
</div>
</div>
}
@if (!isTotpEnabled) {
@if (!isTotpEnabled()) {
<div ngbAccordion>
<div ngbAccordionItem>
<h2 ngbAccordionHeader>
@@ -105,10 +105,10 @@
<div ngbAccordionCollapse>
<div ngbAccordionBody>
<ng-template>
@if (totpSettingsLoading) {
@if (totpSettingsLoading()) {
<div class="spinner-border spinner-border-sm fw-normal ms-2 me-auto" role="status"></div>
<div class="visually-hidden" i18n>Loading...</div>
} @else if (totpSettings) {
} @else if (totpSettings()) {
<figure class="figure">
@if (qrSvgDataUrl) {
<img class="bg-white d-inline-block" [src]="qrSvgDataUrl" alt="Authenticator QR code">
@@ -116,14 +116,14 @@
<figcaption class="figure-caption text-end mt-2" i18n>Scan the QR code with your authenticator app and then enter the code below</figcaption>
</figure>
<p>
<ng-container i18n>Authenticator secret</ng-container>: <code>{{totpSettings.secret}}</code>.
<ng-container i18n>Authenticator secret</ng-container>: <code>{{totpSettings().secret}}</code>.
<ng-container i18n>You can store this secret and use it to reinstall your authenticator app at a later time.</ng-container>
</p>
<div class="input-group mb-3">
<input type="text" class="form-control" formControlName="totp_code" placeholder="Code" i18n-placeholder>
<button type="button" class="btn btn-primary ml-auto" (click)="activateTotp()" [disabled]="totpLoading">
<button type="button" class="btn btn-primary ml-auto" (click)="activateTotp()" [disabled]="totpLoading()">
<ng-container i18n>Enable</ng-container>
@if (totpLoading) {
@if (totpLoading()) {
<div class="spinner-border spinner-border-sm fw-normal ms-2" role="status"></div>
<div class="visually-hidden" i18n>Loading...</div>
}
@@ -137,18 +137,18 @@
</div>
} @else {
<label class="d-block mb-2" i18n>Two-factor Authentication</label>
@if (recoveryCodes) {
@if (recoveryCodes()) {
<div class="alert alert-warning" role="alert">
<i-bs name="exclamation-triangle" class="me-1"></i-bs><ng-container i18n>Recovery codes will not be shown again, make sure to save them.</ng-container>
</div>
<div class="d-flex flex-row align-items-start mb-3">
<ul class="list-group w-50">
@for (code of recoveryCodes; track code; let i = $index) {
@for (code of recoveryCodes(); track code; let i = $index) {
@if (i % 2 === 0) {
<li class="list-group-item d-flex justify-content-around align-items-center">
<code>{{code}}</code>
@if (recoveryCodes[i + 1]) {
<code>{{recoveryCodes[i + 1]}}</code>
@if (recoveryCodes()[i + 1]) {
<code>{{recoveryCodes()[i + 1]}}</code>
}
</li>
}
@@ -171,7 +171,7 @@
i18n-title
buttonClasses="btn-outline-danger btn-sm"
iconName="trash"
[disabled]="totpLoading"
[disabled]="totpLoading()"
(confirm)="deactivateTotp()">
</pngx-confirm-button>
}
@@ -10,7 +10,7 @@ import {
NgbPopoverModule,
} from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { NEVER, of, Subject, throwError } from 'rxjs'
import { ProfileService } from 'src/app/services/profile.service'
import { ToastService } from 'src/app/services/toast.service'
import * as navUtils from 'src/app/utils/navigation'
@@ -60,6 +60,10 @@ describe('ProfileEditDialogComponent', () => {
providers: [NgbActiveModal, provideHttpClient(withInterceptorsFromDi())],
})
profileService = TestBed.inject(ProfileService)
jest.spyOn(profileService, 'get').mockReturnValue(NEVER)
jest
.spyOn(profileService, 'getSocialAccountProviders')
.mockReturnValue(of([]))
toastService = TestBed.inject(ToastService)
clipboard = TestBed.inject(Clipboard)
fixture = TestBed.createComponent(ProfileEditDialogComponent)
@@ -155,7 +159,7 @@ describe('ProfileEditDialogComponent', () => {
'getSocialAccountProviders'
)
getProvidersSpy.mockReturnValue(of(socialAccountProviders))
component.hasUsablePassword = true
component.hasUsablePassword.set(true)
component.ngOnInit()
component.form.get('password').patchValue('new*pass')
component.onPasswordKeyUp({
@@ -268,6 +272,27 @@ describe('ProfileEditDialogComponent', () => {
expect(getProvidersSpy).toHaveBeenCalled()
})
it('should render social account providers after an async update', async () => {
const providers$ = new Subject<typeof socialAccountProviders>()
jest.spyOn(profileService, 'get').mockReturnValue(of(profile))
jest
.spyOn(profileService, 'getSocialAccountProviders')
.mockReturnValue(providers$)
component.ngOnInit()
await fixture.whenStable()
expect(
fixture.nativeElement.querySelector('a[href="https://example.com"]')
).toBeNull()
providers$.next(socialAccountProviders)
await fixture.whenStable()
expect(
fixture.nativeElement.querySelector('a[href="https://example.com"]')
).not.toBeNull()
})
it('should remove disconnected social account from component, show error if needed', () => {
const disconnectSpy = jest.spyOn(profileService, 'disconnectSocialAccount')
const getSpy = jest.spyOn(profileService, 'get')
@@ -276,7 +301,7 @@ describe('ProfileEditDialogComponent', () => {
const errorSpy = jest.spyOn(toastService, 'showError')
expect(component.socialAccounts).toContainEqual(socialAccount)
expect(component.socialAccounts()).toContainEqual(socialAccount)
// fail first
disconnectSpy.mockReturnValueOnce(
@@ -289,7 +314,7 @@ describe('ProfileEditDialogComponent', () => {
disconnectSpy.mockReturnValue(of(socialAccount.id))
component.disconnectSocialAccount(socialAccount.id)
expect(disconnectSpy).toHaveBeenCalled()
expect(component.socialAccounts).not.toContainEqual(socialAccount)
expect(component.socialAccounts()).not.toContainEqual(socialAccount)
})
it('should get totp settings', () => {
@@ -310,7 +335,7 @@ describe('ProfileEditDialogComponent', () => {
getSpy.mockReturnValue(of(settings))
component.gettotpSettings()
expect(getSpy).toHaveBeenCalled()
expect(component.totpSettings).toEqual(settings)
expect(component.totpSettings()).toEqual(settings)
})
it('should activate totp', () => {
@@ -319,15 +344,15 @@ describe('ProfileEditDialogComponent', () => {
const toastInfoSpy = jest.spyOn(toastService, 'showInfo')
const error = new Error('failed to activate totp')
activateSpy.mockReturnValueOnce(throwError(() => error))
component.totpSettings = {
component.totpSettings.set({
url: 'http://localhost/',
qr_svg: 'svg',
secret: 'secret',
}
})
component.form.get('totp_code').patchValue('123456')
component.activateTotp()
expect(activateSpy).toHaveBeenCalledWith(
component.totpSettings.secret,
component.totpSettings().secret,
component.form.get('totp_code').value
)
expect(toastErrorSpy).toHaveBeenCalled()
@@ -341,8 +366,8 @@ describe('ProfileEditDialogComponent', () => {
)
component.activateTotp()
expect(toastInfoSpy).toHaveBeenCalled()
expect(component.isTotpEnabled).toBeTruthy()
expect(component.recoveryCodes).toEqual(['1', '2', '3'])
expect(component.isTotpEnabled()).toBeTruthy()
expect(component.recoveryCodes()).toEqual(['1', '2', '3'])
})
it('should deactivate totp', () => {
@@ -362,13 +387,13 @@ describe('ProfileEditDialogComponent', () => {
deactivateSpy.mockReturnValueOnce(of(true))
component.deactivateTotp()
expect(toastInfoSpy).toHaveBeenCalled()
expect(component.isTotpEnabled).toBeFalsy()
expect(component.isTotpEnabled()).toBeFalsy()
})
it('should copy recovery codes', () => {
jest.useFakeTimers()
const copySpy = jest.spyOn(clipboard, 'copy')
component.recoveryCodes = ['1', '2', '3']
component.recoveryCodes.set(['1', '2', '3'])
component.copyRecoveryCodes()
expect(copySpy).toHaveBeenCalledWith('1\n2\n3')
jest.advanceTimersByTime(3000)
@@ -56,6 +56,14 @@ export class ProfileEditDialogComponent
readonly showEmailConfirm = signal(false)
readonly copied = signal(false)
readonly codesCopied = signal(false)
readonly hasUsablePassword = signal(false)
readonly isTotpEnabled = signal(false)
readonly totpSettings = signal<TotpSettings>(undefined)
readonly totpSettingsLoading = signal(false)
readonly totpLoading = signal(false)
readonly recoveryCodes = signal<string[]>(undefined)
readonly socialAccounts = signal<SocialAccount[]>([])
readonly socialAccountProviders = signal<SocialAccountProvider[]>([])
public form = new FormGroup({
email: new FormControl(''),
@@ -72,25 +80,15 @@ export class ProfileEditDialogComponent
private newPassword: string
private passwordConfirm: string
public hasUsablePassword: boolean = false
private currentEmail: string
private newEmail: string
private emailConfirm: string
public isTotpEnabled: boolean = false
public totpSettings: TotpSettings
public totpSettingsLoading: boolean = false
public totpLoading: boolean = false
public recoveryCodes: string[]
public socialAccounts: SocialAccount[] = []
public socialAccountProviders: SocialAccountProvider[] = []
get qrSvgDataUrl(): string | null {
if (!this.totpSettings?.qr_svg) {
if (!this.totpSettings()?.qr_svg) {
return null
}
return `data:image/svg+xml;utf8,${encodeURIComponent(this.totpSettings.qr_svg)}`
return `data:image/svg+xml;utf8,${encodeURIComponent(this.totpSettings().qr_svg)}`
}
ngOnInit(): void {
@@ -107,20 +105,20 @@ export class ProfileEditDialogComponent
this.onEmailChange()
})
this.currentPassword = profile.password
this.hasUsablePassword = profile.has_usable_password
this.hasUsablePassword.set(profile.has_usable_password)
this.form.get('password').valueChanges.subscribe((newPassword) => {
this.newPassword = newPassword
this.onPasswordChange()
})
this.socialAccounts = profile.social_accounts
this.isTotpEnabled = profile.is_mfa_enabled
this.socialAccounts.set(profile.social_accounts ?? [])
this.isTotpEnabled.set(profile.is_mfa_enabled)
})
this.profileService
.getSocialAccountProviders()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((providers) => {
this.socialAccountProviders = providers
this.socialAccountProviders.set(providers ?? [])
})
}
@@ -259,7 +257,9 @@ export class ProfileEditDialogComponent
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (id: number) => {
this.socialAccounts = this.socialAccounts.filter((a) => a.id != id)
this.socialAccounts.update((accounts) =>
accounts.filter((account) => account.id != id)
)
},
error: (error) => {
this.toastService.showError(
@@ -271,36 +271,39 @@ export class ProfileEditDialogComponent
}
public gettotpSettings(): void {
this.totpSettingsLoading = true
this.totpSettingsLoading.set(true)
this.profileService
.getTotpSettings()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (totpSettings) => {
this.totpSettingsLoading = false
this.totpSettings = totpSettings
this.totpSettingsLoading.set(false)
this.totpSettings.set(totpSettings)
},
error: (error) => {
this.toastService.showError(
$localize`Error fetching TOTP settings`,
error
)
this.totpSettingsLoading = false
this.totpSettingsLoading.set(false)
},
})
}
public activateTotp(): void {
this.totpLoading = true
this.totpLoading.set(true)
this.form.get('totp_code').disable()
this.profileService
.activateTotp(this.totpSettings.secret, this.form.get('totp_code').value)
.activateTotp(
this.totpSettings().secret,
this.form.get('totp_code').value
)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (activationResponse) => {
this.totpLoading = false
this.isTotpEnabled = activationResponse.success
this.recoveryCodes = activationResponse.recovery_codes
this.totpLoading.set(false)
this.isTotpEnabled.set(activationResponse.success)
this.recoveryCodes.set(activationResponse.recovery_codes)
this.form.get('totp_code').enable()
if (activationResponse.success) {
this.toastService.showInfo($localize`TOTP activated successfully`)
@@ -309,7 +312,7 @@ export class ProfileEditDialogComponent
}
},
error: (error) => {
this.totpLoading = false
this.totpLoading.set(false)
this.form.get('totp_code').enable()
this.toastService.showError($localize`Error activating TOTP`, error)
},
@@ -317,15 +320,15 @@ export class ProfileEditDialogComponent
}
public deactivateTotp(): void {
this.totpLoading = true
this.totpLoading.set(true)
this.profileService
.deactivateTotp()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe({
next: (success) => {
this.totpLoading = false
this.isTotpEnabled = !success
this.recoveryCodes = null
this.totpLoading.set(false)
this.isTotpEnabled.set(!success)
this.recoveryCodes.set(null)
if (success) {
this.toastService.showInfo($localize`TOTP deactivated successfully`)
} else {
@@ -333,14 +336,14 @@ export class ProfileEditDialogComponent
}
},
error: (error) => {
this.totpLoading = false
this.totpLoading.set(false)
this.toastService.showError($localize`Error deactivating TOTP`, error)
},
})
}
public copyRecoveryCodes(): void {
this.clipboard.copy(this.recoveryCodes.join('\n'))
this.clipboard.copy(this.recoveryCodes().join('\n'))
this.codesCopied.set(true)
setTimeout(() => {
this.codesCopied.set(false)
@@ -65,8 +65,9 @@
<dd class="col-sm-8"><code>{{ createdBundle.slug }}</code></dd>
<dt class="col-sm-4" i18n>Link</dt>
<dd class="col-sm-8">
<label class="visually-hidden" for="shareBundleLink" i18n>Share link</label>
<div class="input-group input-group-sm">
<input class="form-control" type="text" [value]="getShareUrl(createdBundle)" readonly>
<input id="shareBundleLink" class="form-control" type="text" [value]="getShareUrl(createdBundle)" readonly>
<button
class="btn btn-outline-primary"
type="button"
@@ -14,7 +14,9 @@
}
<div content class="wrapper fade" [class.show]="show()">
@if (displayMode() === DisplayMode.TABLE) {
@if (error()) {
<div class="alert alert-danger mb-0" role="alert"><ng-container i18n>Error while loading documents</ng-container>: {{error()}}</div>
} @else if (displayMode() === DisplayMode.TABLE) {
<table class="table table-hover mb-0 mt-n2 align-middle">
<thead>
<tr>
@@ -1,6 +1,10 @@
import { DragDropModule } from '@angular/cdk/drag-drop'
import { DatePipe } from '@angular/common'
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import {
HttpErrorResponse,
provideHttpClient,
withInterceptorsFromDi,
} from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
@@ -8,7 +12,7 @@ import { Router } from '@angular/router'
import { RouterTestingModule } from '@angular/router/testing'
import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { Subject, of } from 'rxjs'
import { Subject, of, throwError } from 'rxjs'
import { routes } from 'src/app/app-routing.module'
import { CustomFieldDisplayComponent } from 'src/app/components/common/custom-field-display/custom-field-display.component'
import { PreviewPopupComponent } from 'src/app/components/common/preview-popup/preview-popup.component'
@@ -230,6 +234,27 @@ describe('SavedViewWidgetComponent', () => {
expect(component.documents()).toEqual(documentResults)
})
it('should show an error if documents fail to load', () => {
jest.spyOn(documentService, 'listFiltered').mockReturnValue(
throwError(
() =>
new HttpErrorResponse({
error: { added__date__lte: ['Enter a valid date.'] },
status: 400,
})
)
)
component.reload()
fixture.detectChanges()
expect(component.loading()).toBe(false)
expect(component.error()).toEqual('Added: Enter a valid date.')
expect(fixture.debugElement.nativeElement.textContent).toContain(
'Error while loading documents: Added: Enter a valid date.'
)
})
it('should reload on document consumption finished', () => {
const fileStatusSubject = new Subject<FileStatus>()
jest
@@ -125,6 +125,8 @@ export class SavedViewWidgetComponent
readonly count = signal<number>(null)
readonly error = signal<string | null>(null)
placeholderRows: number[] = []
ngOnInit(): void {
@@ -180,6 +182,7 @@ export class SavedViewWidgetComponent
reload() {
this.loading.set(this.documents().length == 0)
this.error.set(null)
this.show.set(true)
this.documentService
.listFiltered(
@@ -191,12 +194,40 @@ export class SavedViewWidgetComponent
{ truncate_content: true }
)
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((result) => {
this.documents.set(result.results)
this.count.set(result.count)
this.savedViewService.setDocumentCount(this.savedView, result.count)
this.loading.set(false)
this.show.set(true)
.subscribe({
next: (result) => {
this.documents.set(result.results)
this.count.set(result.count)
this.savedViewService.setDocumentCount(this.savedView, result.count)
this.loading.set(false)
this.show.set(true)
},
error: (error) => {
this.documents.set([])
this.count.set(null)
let errorMessage
if (
typeof error.error === 'object' &&
Object.keys(error.error).length > 0
) {
errorMessage = Object.keys(error.error)
.map((fieldName) => {
const fieldNameBase = fieldName.split('__')[0]
const fieldError: Array<string> = error.error[fieldName]
return `${
this.documentService.sortFields.find(
(f) => f.field?.split('__')[0] == fieldNameBase
)?.name ?? fieldNameBase
}: ${fieldError[0]}`
})
.join(', ')
} else {
errorMessage = error.error
}
this.error.set(errorMessage)
this.loading.set(false)
this.show.set(true)
},
})
}
@@ -1 +1 @@
<p i18n>Searching document with asn {{asn}}</p>
<p i18n>Searching document with asn {{asn()}}</p>
@@ -1,4 +1,4 @@
import { Component, OnInit, inject } from '@angular/core'
import { Component, OnInit, inject, signal } from '@angular/core'
import { ActivatedRoute, Router } from '@angular/router'
import { FILTER_ASN } from '../../data/filter-rule-type'
import { DocumentService } from '../../services/rest/document.service'
@@ -13,13 +13,13 @@ export class DocumentAsnComponent implements OnInit {
private route = inject(ActivatedRoute)
private router = inject(Router)
asn: string
readonly asn = signal<string>(undefined)
ngOnInit(): void {
this.route.paramMap.subscribe((paramMap) => {
this.asn = paramMap.get('id')
this.asn.set(paramMap.get('id'))
this.documentsService
.listAllFilteredIds([{ rule_type: FILTER_ASN, value: this.asn }])
.listAllFilteredIds([{ rule_type: FILTER_ASN, value: this.asn() }])
.subscribe((documentId) => {
if (documentId.length == 1) {
this.router.navigate(['documents', documentId[0]])
@@ -315,7 +315,7 @@
<td>{{metadata()?.original_filename}}</td>
</tr>
<tr>
<td i18n>Original MD5 checksum</td>
<td i18n>Original SHA256 checksum</td>
<td>{{metadata()?.original_checksum}}</td>
</tr>
<tr>
@@ -328,7 +328,7 @@
</tr>
@if (metadata()?.has_archive_version) {
<tr>
<td i18n>Archive MD5 checksum</td>
<td i18n>Archive SHA256 checksum</td>
<td>{{metadata()?.archive_checksum}}</td>
</tr>
}
@@ -482,9 +482,7 @@
</pngx-pdf-viewer>
</div>
} @else {
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%">
<span>Preview is unavailable.</span>
</object>
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%"></object>
}
}
@case (ContentRenderType.Text) {
@@ -505,9 +503,7 @@
}
}
@case (ContentRenderType.Other) {
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%">
<span>Preview is unavailable.</span>
</object>
<object [data]="previewUrl() | safeUrl" class="preview-sticky" width="100%"></object>
}
}
@if (requiresPassword) {
@@ -510,7 +510,7 @@ describe('DocumentDetailComponent', () => {
const deleteButtons = fixture.debugElement.queryAll(
By.css('pngx-confirm-button')
)
expect(deleteButtons.length).toEqual(1)
expect(deleteButtons).toHaveLength(1)
})
it('should fall back to details tab when duplicates tab is active but no duplicates', () => {
@@ -1,6 +1,6 @@
@if (loading()) {
<div class="d-flex">
<output class="spinner-border spinner-border-sm fw-normal" role="status"></output>
<output class="spinner-border spinner-border-sm fw-normal"></output>
</div>
} @else {
<ul class="list-group">
@@ -26,11 +26,11 @@ import { UserService } from 'src/app/services/rest/user.service'
],
})
export class DocumentHistoryComponent implements OnInit {
private documentService = inject(DocumentService)
private correspondentService = inject(CorrespondentService)
private storagePathService = inject(StoragePathService)
private documentTypeService = inject(DocumentTypeService)
private userService = inject(UserService)
private readonly documentService = inject(DocumentService)
private readonly correspondentService = inject(CorrespondentService)
private readonly storagePathService = inject(StoragePathService)
private readonly documentTypeService = inject(DocumentTypeService)
private readonly userService = inject(UserService)
public AuditLogAction = AuditLogAction
@@ -70,7 +70,7 @@ export class DocumentHistoryComponent implements OnInit {
return cached
}
const idInt = parseInt(id, 10)
const idInt = Number.parseInt(id, 10)
const fallback$ = of(id)
let result$: Observable<string>
@@ -5,10 +5,11 @@
</button>
<div class="dropdown-menu shadow" ngbDropdownMenu>
<div class="px-3 py-2 mb-2">
@if (versionUploadState === UploadState.Idle) {
@if (versionUploadState() === UploadState.Idle) {
<div class="input-group input-group-sm mb-2">
<span class="input-group-text" i18n>Label</span>
<label class="input-group-text" for="newVersionLabel" i18n>Label</label>
<input
id="newVersionLabel"
class="form-control"
type="text"
[(ngModel)]="newVersionLabel"
@@ -31,7 +32,7 @@
<i-bs name="file-earmark-plus"></i-bs><span class="ps-1" i18n>Add new version</span>
</button>
} @else {
@switch (versionUploadState) {
@switch (versionUploadState()) {
@case (UploadState.Uploading) {
<div class="small text-muted mt-1 d-flex align-items-center">
<output class="spinner-border spinner-border-sm me-2" aria-hidden="true"></output>
@@ -49,8 +50,8 @@
<span i18n>Version upload failed.</span>
<button type="button" class="btn btn-link btn-sm p-0 ms-2" (click)="clearVersionUploadStatus()" i18n>Dismiss</button>
</div>
@if (versionUploadError) {
<div class="small text-muted mt-1">{{ versionUploadError }}</div>
@if (versionUploadError()) {
<div class="small text-muted mt-1">{{ versionUploadError() }}</div>
}
}
}
@@ -70,15 +71,19 @@
}
</div>
<div class="d-flex flex-column">
@if (isEditingVersion(version.id)) {
<label class="visually-hidden" for="versionLabel{{ version.id }}" i18n>Version label</label>
}
<div class="input-group input-group-sm mb-1">
@if (isEditingVersion(version.id)) {
<input
id="versionLabel{{ version.id }}"
class="form-control"
type="text"
[(ngModel)]="versionLabelDraft"
i18n-placeholder
placeholder="Version label"
[disabled]="savingVersionLabelId !== null"
[disabled]="savingVersionLabelId() !== null"
(keydown.enter)="submitEditedVersionLabel(version, $event)"
(keydown.escape)="cancelEditingVersion($event)"
(click)="$event.stopPropagation()"
@@ -96,7 +101,7 @@
<button
type="button"
class="btn btn-outline-secondary"
[disabled]="savingVersionLabelId !== null"
[disabled]="savingVersionLabelId() !== null"
(click)="isEditingVersion(version.id) ? submitEditedVersionLabel(version, $event) : beginEditingVersion(version, $event)"
>
@if (isEditingVersion(version.id)) {
@@ -205,7 +205,7 @@ describe('DocumentVersionDropdownComponent', () => {
{ id: 3, is_root: true, checksum: 'aaaa' },
{ id: 10, is_root: false, checksum: 'bbbb', version_label: 'Updated' },
])
expect(component.savingVersionLabelId).toBeNull()
expect(component.savingVersionLabelId()).toBeNull()
})
it('saveVersionLabel should show error toast on failure', () => {
@@ -218,7 +218,7 @@ describe('DocumentVersionDropdownComponent', () => {
'Error updating version label',
error
)
expect(component.savingVersionLabelId).toBeNull()
expect(component.savingVersionLabelId()).toBeNull()
})
it('onVersionFileSelected should upload and update versions after websocket success', () => {
@@ -252,11 +252,11 @@ describe('DocumentVersionDropdownComponent', () => {
expect(versionsEmitSpy).toHaveBeenCalledWith(versions)
expect(selectedEmitSpy).toHaveBeenCalledWith(20)
expect(component.newVersionLabel).toEqual('')
expect(component.versionUploadState).toEqual(UploadState.Idle)
expect(component.versionUploadError).toBeNull()
expect(component.versionUploadState()).toEqual(UploadState.Idle)
expect(component.versionUploadError()).toBeNull()
})
it('onVersionFileSelected should set failed state after websocket failure', () => {
it('onVersionFileSelected should render failed state after websocket failure', async () => {
const file = new File(['test'], 'new-version.pdf', {
type: 'application/pdf',
})
@@ -266,9 +266,11 @@ describe('DocumentVersionDropdownComponent', () => {
component.onVersionFileSelected({ target: input } as Event)
failed$.next({ taskId: 'task-1', message: 'processing failed' })
await fixture.whenStable()
expect(component.versionUploadState).toEqual(UploadState.Failed)
expect(component.versionUploadError).toEqual('processing failed')
expect(component.versionUploadState()).toEqual(UploadState.Failed)
expect(component.versionUploadError()).toEqual('processing failed')
expect(fixture.nativeElement.textContent).toContain('processing failed')
expect(documentService.getVersions).not.toHaveBeenCalled()
})
@@ -282,8 +284,8 @@ describe('DocumentVersionDropdownComponent', () => {
component.onVersionFileSelected({ target: input } as Event)
expect(component.versionUploadState).toEqual(UploadState.Failed)
expect(component.versionUploadError).toEqual('Missing task ID.')
expect(component.versionUploadState()).toEqual(UploadState.Failed)
expect(component.versionUploadError()).toEqual('Missing task ID.')
expect(documentService.getVersions).not.toHaveBeenCalled()
})
@@ -298,8 +300,8 @@ describe('DocumentVersionDropdownComponent', () => {
component.onVersionFileSelected({ target: input } as Event)
expect(component.versionUploadState).toEqual(UploadState.Failed)
expect(component.versionUploadError).toEqual('upload failed')
expect(component.versionUploadState()).toEqual(UploadState.Failed)
expect(component.versionUploadError()).toEqual('upload failed')
expect(toastService.showError).toHaveBeenCalledWith(
'Error uploading new version',
error
@@ -307,8 +309,8 @@ describe('DocumentVersionDropdownComponent', () => {
})
it('ngOnChanges should clear upload status on document switch', () => {
component.versionUploadState = UploadState.Failed
component.versionUploadError = 'something failed'
component.versionUploadState.set(UploadState.Failed)
component.versionUploadError.set('something failed')
component.editingVersionId = 10
component.versionLabelDraft = 'draft'
@@ -316,8 +318,8 @@ describe('DocumentVersionDropdownComponent', () => {
documentId: new SimpleChange(3, 4, false),
})
expect(component.versionUploadState).toEqual(UploadState.Idle)
expect(component.versionUploadError).toBeNull()
expect(component.versionUploadState()).toEqual(UploadState.Idle)
expect(component.versionUploadError()).toBeNull()
expect(component.editingVersionId).toBeNull()
expect(component.versionLabelDraft).toEqual('')
})
@@ -7,6 +7,7 @@ import {
OnChanges,
OnDestroy,
Output,
signal,
SimpleChanges,
} from '@angular/core'
import { FormsModule } from '@angular/forms'
@@ -59,9 +60,9 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
@Output() versionsUpdated = new EventEmitter<DocumentVersionInfo[]>()
newVersionLabel: string = ''
versionUploadState: UploadState = UploadState.Idle
versionUploadError: string | null = null
savingVersionLabelId: number | null = null
readonly versionUploadState = signal(UploadState.Idle)
readonly versionUploadError = signal<string | null>(null)
readonly savingVersionLabelId = signal<number | null>(null)
editingVersionId: number | null = null
versionLabelDraft: string = ''
@@ -101,7 +102,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
beginEditingVersion(version: DocumentVersionInfo, event?: Event): void {
event?.preventDefault()
event?.stopPropagation()
if (!this.canEditLabels || this.savingVersionLabelId !== null) return
if (!this.canEditLabels || this.savingVersionLabelId() !== null) return
this.editingVersionId = version.id
this.versionLabelDraft = version.version_label ?? ''
}
@@ -116,7 +117,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
submitEditedVersionLabel(version: DocumentVersionInfo, event?: Event): void {
event?.preventDefault()
event?.stopPropagation()
if (this.savingVersionLabelId !== null) return
if (this.savingVersionLabelId() !== null) return
const nextLabel = this.versionLabelDraft?.trim() || null
const currentLabel = version.version_label?.trim() || null
if (nextLabel === currentLabel) {
@@ -158,15 +159,15 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
}
saveVersionLabel(versionId: number, versionLabel: string | null): void {
if (this.savingVersionLabelId !== null) return
this.savingVersionLabelId = versionId
if (this.savingVersionLabelId() !== null) return
this.savingVersionLabelId.set(versionId)
this.documentsService
.updateVersionLabel(this.documentId, versionId, versionLabel)
.pipe(
first(),
finalize(() => {
if (this.savingVersionLabelId === versionId) {
this.savingVersionLabelId = null
if (this.savingVersionLabelId() === versionId) {
this.savingVersionLabelId.set(null)
}
}),
takeUntil(this.destroy$)
@@ -199,8 +200,8 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
const file = input.files[0]
input.value = ''
const label = this.newVersionLabel?.trim()
this.versionUploadState = UploadState.Uploading
this.versionUploadError = null
this.versionUploadState.set(UploadState.Uploading)
this.versionUploadError.set(null)
this.documentsService
.uploadVersion(uploadDocumentId, file, label)
.pipe(
@@ -210,7 +211,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
$localize`Uploading new version. Processing will happen in the background.`
)
this.newVersionLabel = ''
this.versionUploadState = UploadState.Processing
this.versionUploadState.set(UploadState.Processing)
}),
map((taskId) =>
typeof taskId === 'string'
@@ -219,8 +220,8 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
),
switchMap((taskId) => {
if (!taskId) {
this.versionUploadState = UploadState.Failed
this.versionUploadError = $localize`Missing task ID.`
this.versionUploadState.set(UploadState.Failed)
this.versionUploadError.set($localize`Missing task ID.`)
return of(null)
}
return merge(
@@ -240,9 +241,10 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
switchMap((result) => {
if (result?.state !== 'success') {
if (result?.state === 'failed') {
this.versionUploadState = UploadState.Failed
this.versionUploadError =
this.versionUploadState.set(UploadState.Failed)
this.versionUploadError.set(
result.message || $localize`Upload failed.`
)
}
return of(null)
}
@@ -264,8 +266,10 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
},
error: (error) => {
if (uploadDocumentId !== this.documentId) return
this.versionUploadState = UploadState.Failed
this.versionUploadError = error?.message || $localize`Upload failed.`
this.versionUploadState.set(UploadState.Failed)
this.versionUploadError.set(
error?.message || $localize`Upload failed.`
)
this.toastService.showError(
$localize`Error uploading new version`,
error
@@ -275,7 +279,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
}
clearVersionUploadStatus(): void {
this.versionUploadState = UploadState.Idle
this.versionUploadError = null
this.versionUploadState.set(UploadState.Idle)
this.versionUploadError.set(null)
}
}
@@ -10,7 +10,7 @@
[createRef]="createTag.bind(this)"
(opened)="openTagsDropdown()"
[(selectionModel)]="tagSelectionModel"
[documentCounts]="tagDocumentCounts"
[documentCounts]="tagDocumentCounts()"
(apply)="setTags($event)"
shortcutKey="t">
</pngx-filterable-dropdown>
@@ -24,7 +24,7 @@
[createRef]="createCorrespondent.bind(this)"
(opened)="openCorrespondentDropdown()"
[(selectionModel)]="correspondentSelectionModel"
[documentCounts]="correspondentDocumentCounts"
[documentCounts]="correspondentDocumentCounts()"
(apply)="setCorrespondents($event)"
shortcutKey="y">
</pngx-filterable-dropdown>
@@ -38,7 +38,7 @@
[createRef]="createDocumentType.bind(this)"
(opened)="openDocumentTypeDropdown()"
[(selectionModel)]="documentTypeSelectionModel"
[documentCounts]="documentTypeDocumentCounts"
[documentCounts]="documentTypeDocumentCounts()"
(apply)="setDocumentTypes($event)"
shortcutKey="u">
</pngx-filterable-dropdown>
@@ -52,7 +52,7 @@
[createRef]="createStoragePath.bind(this)"
(opened)="openStoragePathDropdown()"
[(selectionModel)]="storagePathsSelectionModel"
[documentCounts]="storagePathDocumentCounts"
[documentCounts]="storagePathDocumentCounts()"
(apply)="setStoragePaths($event)"
shortcutKey="i">
</pngx-filterable-dropdown>
@@ -66,7 +66,7 @@
[createRef]="createCustomField.bind(this)"
(opened)="openCustomFieldsDropdown()"
[(selectionModel)]="customFieldsSelectionModel"
[documentCounts]="customFieldDocumentCounts"
[documentCounts]="customFieldDocumentCounts()"
extraButtonTitle="Set values"
i18n-extraButtonTitle
(extraButton)="setCustomFieldValues($event)"
@@ -124,11 +124,11 @@
</div>
</div>
<div class="btn-group btn-group-sm">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload" (click)="downloadSelected()">
@if (!awaitingDownload) {
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()">
@if (!awaitingDownload()) {
<i-bs name="arrow-down"></i-bs>
}
@if (awaitingDownload) {
@if (awaitingDownload()) {
<div class="spinner-border spinner-border-sm" role="status">
<span class="visually-hidden">Preparing download...</span>
</div>
@@ -303,7 +303,7 @@ describe('BulkEditorComponent', () => {
component.openDocumentTypeDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.documentTypeDocumentCounts).toEqual(
expect(component.documentTypeDocumentCounts()).toEqual(
selectionData.selected_document_types
)
})
@@ -320,7 +320,7 @@ describe('BulkEditorComponent', () => {
component.openCorrespondentDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.correspondentDocumentCounts).toEqual(
expect(component.correspondentDocumentCounts()).toEqual(
selectionData.selected_correspondents
)
})
@@ -337,7 +337,7 @@ describe('BulkEditorComponent', () => {
component.openStoragePathDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.storagePathDocumentCounts).toEqual(
expect(component.storagePathDocumentCounts()).toEqual(
selectionData.selected_storage_paths
)
})
@@ -354,7 +354,7 @@ describe('BulkEditorComponent', () => {
component.openCustomFieldsDropdown()
expect(getSelectionDataSpy).not.toHaveBeenCalled()
expect(component.customFieldDocumentCounts).toEqual(
expect(component.customFieldDocumentCounts()).toEqual(
selectionData.selected_custom_fields
)
})
@@ -1,4 +1,11 @@
import { Component, inject, Input, OnDestroy, OnInit } from '@angular/core'
import {
Component,
inject,
Input,
OnDestroy,
OnInit,
signal,
} from '@angular/core'
import {
FormControl,
FormGroup,
@@ -101,12 +108,12 @@ export class BulkEditorComponent
documentTypeSelectionModel = new FilterableDropdownSelectionModel()
storagePathsSelectionModel = new FilterableDropdownSelectionModel()
customFieldsSelectionModel = new FilterableDropdownSelectionModel(true)
tagDocumentCounts: SelectionDataItem[]
correspondentDocumentCounts: SelectionDataItem[]
documentTypeDocumentCounts: SelectionDataItem[]
storagePathDocumentCounts: SelectionDataItem[]
customFieldDocumentCounts: SelectionDataItem[]
awaitingDownload: boolean
readonly tagDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly correspondentDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly documentTypeDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly storagePathDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly customFieldDocumentCounts = signal<SelectionDataItem[]>(undefined)
readonly awaitingDownload = signal(false)
unsubscribeNotifier: Subject<any> = new Subject()
@@ -365,8 +372,8 @@ export class BulkEditorComponent
openTagsDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.tagDocumentCounts = selectionData?.selected_tags ?? []
this.applySelectionData(this.tagDocumentCounts, this.tagSelectionModel)
this.tagDocumentCounts.set(selectionData?.selected_tags ?? [])
this.applySelectionData(this.tagDocumentCounts(), this.tagSelectionModel)
return
}
@@ -374,7 +381,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.tagDocumentCounts = s.selected_tags
this.tagDocumentCounts.set(s.selected_tags)
this.applySelectionData(s.selected_tags, this.tagSelectionModel)
})
}
@@ -382,10 +389,11 @@ export class BulkEditorComponent
openDocumentTypeDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.documentTypeDocumentCounts =
this.documentTypeDocumentCounts.set(
selectionData?.selected_document_types ?? []
)
this.applySelectionData(
this.documentTypeDocumentCounts,
this.documentTypeDocumentCounts(),
this.documentTypeSelectionModel
)
return
@@ -395,7 +403,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.documentTypeDocumentCounts = s.selected_document_types
this.documentTypeDocumentCounts.set(s.selected_document_types)
this.applySelectionData(
s.selected_document_types,
this.documentTypeSelectionModel
@@ -406,10 +414,11 @@ export class BulkEditorComponent
openCorrespondentDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.correspondentDocumentCounts =
this.correspondentDocumentCounts.set(
selectionData?.selected_correspondents ?? []
)
this.applySelectionData(
this.correspondentDocumentCounts,
this.correspondentDocumentCounts(),
this.correspondentSelectionModel
)
return
@@ -419,7 +428,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.correspondentDocumentCounts = s.selected_correspondents
this.correspondentDocumentCounts.set(s.selected_correspondents)
this.applySelectionData(
s.selected_correspondents,
this.correspondentSelectionModel
@@ -430,10 +439,11 @@ export class BulkEditorComponent
openStoragePathDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.storagePathDocumentCounts =
this.storagePathDocumentCounts.set(
selectionData?.selected_storage_paths ?? []
)
this.applySelectionData(
this.storagePathDocumentCounts,
this.storagePathDocumentCounts(),
this.storagePathsSelectionModel
)
return
@@ -443,7 +453,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.storagePathDocumentCounts = s.selected_storage_paths
this.storagePathDocumentCounts.set(s.selected_storage_paths)
this.applySelectionData(
s.selected_storage_paths,
this.storagePathsSelectionModel
@@ -454,10 +464,11 @@ export class BulkEditorComponent
openCustomFieldsDropdown() {
if (this.list.allSelected) {
const selectionData = this.list.selectionData
this.customFieldDocumentCounts =
this.customFieldDocumentCounts.set(
selectionData?.selected_custom_fields ?? []
)
this.applySelectionData(
this.customFieldDocumentCounts,
this.customFieldDocumentCounts(),
this.customFieldsSelectionModel
)
return
@@ -467,7 +478,7 @@ export class BulkEditorComponent
.getSelectionData(Array.from(this.list.selected))
.pipe(first())
.subscribe((s) => {
this.customFieldDocumentCounts = s.selected_custom_fields
this.customFieldDocumentCounts.set(s.selected_custom_fields)
this.applySelectionData(
s.selected_custom_fields,
this.customFieldsSelectionModel
@@ -876,7 +887,7 @@ export class BulkEditorComponent
}
downloadSelected() {
this.awaitingDownload = true
this.awaitingDownload.set(true)
let downloadFileType: string =
this.downloadForm.get('downloadFileTypeArchive').value &&
this.downloadForm.get('downloadFileTypeOriginals').value
@@ -893,7 +904,7 @@ export class BulkEditorComponent
.pipe(first())
.subscribe((result: any) => {
saveAs(result, 'documents.zip')
this.awaitingDownload = false
this.awaitingDownload.set(false)
})
}
@@ -2,7 +2,7 @@
<div class="row g-0">
<div class="col-md-2 doc-img-container rounded-start" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit()">
@if (document()) {
<img [src]="getThumbUrl()" class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()">
<img [ngSrc]="getThumbUrl()" fill class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()">
<div class="border-end border-bottom bg-light document-card-check">
<div class="form-check">
@@ -88,6 +88,12 @@ describe('DocumentCardLargeComponent', () => {
expect(fixture.nativeElement.textContent).toContain('8 pages')
})
it('should lazy load the thumbnail', () => {
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should trim content', () => {
expect(component.contentTrimmed).toHaveLength(503) // includes ...
})
@@ -1,4 +1,4 @@
import { AsyncPipe } from '@angular/common'
import { AsyncPipe, NgOptimizedImage } from '@angular/common'
import {
AfterViewInit,
Component,
@@ -46,6 +46,7 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
DocumentTypeNamePipe,
@@ -2,7 +2,7 @@
<div class="card h-100 shadow-sm document-card" [class.placeholder-glow]="!document()" [class.card-selected]="selected()" (mouseleave)="mouseLeaveCard()">
<div class="border-bottom doc-img-container rounded-top" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit(this)">
@if (document()) {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [src]="getThumbUrl()">
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill>
<div class="border-end border-bottom bg-light py-1 px-2 document-card-check">
<div class="form-check">
@@ -22,10 +22,14 @@
}
}
.doc-img-container {
position: relative;
height: 180px;
}
.doc-img {
object-fit: cover;
object-position: top left;
height: 180px;
}
.document-card-check {
@@ -61,6 +61,12 @@ describe('DocumentCardSmallComponent', () => {
expect(fixture.nativeElement.textContent).toContain('12 pages')
})
it('should lazy load the thumbnail', () => {
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should display a document, limit tags to 5', () => {
expect(fixture.nativeElement.textContent).toContain('Document 10')
expect(
@@ -1,4 +1,4 @@
import { AsyncPipe } from '@angular/common'
import { AsyncPipe, NgOptimizedImage } from '@angular/common'
import {
AfterViewInit,
Component,
@@ -46,6 +46,7 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
DocumentTypeNamePipe,
@@ -115,7 +115,7 @@
</pngx-page-header>
<div class="row sticky-top py-3 mt-n2 mt-md-n3 bg-body">
<pngx-filter-editor [hidden]="isBulkEditing" [disabled]="isBulkEditing" [filterRules]="list.filterRules" (filterRulesChange)="onFilterRulesChange($event)" (resetFilterRules)="onFilterRulesReset($event)" [unmodifiedFilterRules]="unmodifiedFilterRules" [selectionData]="list.selectionData" #filterEditor></pngx-filter-editor>
<pngx-filter-editor [hidden]="isBulkEditing" [disabled]="isBulkEditing" [filterRules]="list.filterRules" (filterRulesChange)="onFilterRulesChange($event)" (resetFilterRules)="onFilterRulesReset($event)" [unmodifiedFilterRules]="unmodifiedFilterRules()" [selectionData]="list.selectionData" #filterEditor></pngx-filter-editor>
<pngx-bulk-editor [hidden]="!isBulkEditing" [disabled]="!isBulkEditing"></pngx-bulk-editor>
</div>
@@ -5,6 +5,7 @@ import {
OnDestroy,
OnInit,
QueryList,
signal,
ViewChild,
ViewChildren,
} from '@angular/core'
@@ -149,7 +150,7 @@ export class DocumentListComponent
)
}
unmodifiedFilterRules: FilterRule[] = []
readonly unmodifiedFilterRules = signal<FilterRule[]>([])
private unmodifiedSavedView: SavedView
private activeSavedView: SavedView | null = null
@@ -299,7 +300,7 @@ export class DocumentListComponent
this.savedViewService.setDocumentCount(view, this.list.collectionSize)
})
this.updateDisplayCustomFields()
this.unmodifiedFilterRules = view.filter_rules
this.unmodifiedFilterRules.set(view.filter_rules)
})
this.route.queryParamMap
@@ -316,7 +317,7 @@ export class DocumentListComponent
this.activeSavedView = null
this.list.activateSavedView(null)
this.list.loadFromQueryParams(queryParams)
this.unmodifiedFilterRules = []
this.unmodifiedFilterRules.set([])
}
})
@@ -415,7 +416,7 @@ export class DocumentListComponent
this.toastService.showInfo(
$localize`View "${this.list.activeSavedViewTitle}" saved successfully.`
)
this.unmodifiedFilterRules = this.list.filterRules
this.unmodifiedFilterRules.set(this.list.filterRules)
},
error: (err) => {
this.toastService.showError(
@@ -80,7 +80,7 @@
shortcutKey="i"></pngx-filterable-dropdown>
}
@if (permissionsService.currentUserCan(PermissionAction.View, PermissionType.CustomField) && customFields.length > 0) {
@if (permissionsService.currentUserCan(PermissionAction.View, PermissionType.CustomField) && customFields().length > 0) {
<pngx-custom-fields-query-dropdown class="flex-fill fade" [class.show]="show()" title="Custom fields" icon="ui-radios" i18n-title
[(selectionModel)]="customFieldQueriesModel"
(selectionModelChange)="updateRules()"
@@ -328,7 +328,7 @@ describe('FilterEditorComponent', () => {
})
it('should ingest legacy text filter rules for doc title + content', () => {
expect(component.textFilter).toEqual(null)
expect(component.textFilter).toBeNull()
component.filterRules = [
{
rule_type: FILTER_TITLE_CONTENT,
@@ -1,5 +1,6 @@
import {
AfterViewInit,
ChangeDetectorRef,
Component,
ElementRef,
EventEmitter,
@@ -9,6 +10,7 @@ import {
Output,
ViewChild,
inject,
signal,
} from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import {
@@ -349,7 +351,7 @@ export class FilterEditorComponent
@ViewChild('textFilterInput')
textFilterInput: ElementRef
customFields: CustomField[] = []
readonly customFields = signal<CustomField[]>([])
tagDocumentCounts: SelectionDataItem[]
correspondentDocumentCounts: SelectionDataItem[]
@@ -514,6 +516,7 @@ export class FilterEditorComponent
this.documentService.get(this._moreLikeId).subscribe((result) => {
this._moreLikeDoc = result
this._textFilter = result.title
this.changeDetector.markForCheck()
})
break
case FILTER_CREATED_AFTER:
@@ -1162,6 +1165,7 @@ export class FilterEditorComponent
private loadingCountTotal: number = 0
private loadingCount: number = 0
private readonly changeDetector = inject(ChangeDetectorRef)
private maybeCompleteLoading() {
this.loadingCount++
@@ -1229,7 +1233,7 @@ export class FilterEditorComponent
) {
this.loadingCountTotal++
this.customFieldService.listAll().subscribe((result) => {
this.customFields = result.results
this.customFields.set(result.results)
this.maybeCompleteLoading()
})
}
@@ -1,18 +1,18 @@
<div class="row mb-3">
<div class="col mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<label class="text-muted me-2 mb-0" i18n>Filter by:</label>
<input class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
<label class="text-muted me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<input id="managementNameFilter" class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
</div>
</div>
<div class="col-auto mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<div class="input-group input-group-sm w-auto d-none d-md-flex">
<span class="input-group-text border-0" i18n>Show:</span>
<label class="input-group-text border-0" for="managementPageSize" i18n>Show:</label>
</div>
<div class="input-group input-group-sm w-auto me-3">
<select class="form-select form-select-sm small" [(ngModel)]="pageSize">
<select id="managementPageSize" class="form-select form-select-sm small" [(ngModel)]="pageSize">
<option [ngValue]="25">25</option>
<option [ngValue]="50">50</option>
<option [ngValue]="100">100</option>
@@ -83,12 +83,12 @@ describe('TagListComponent', () => {
]
component['_nameFilter'] = null // Simulate empty name filter
const filtered = component.filterData(tags as any)
expect(filtered.length).toBe(2)
expect(filtered).toHaveLength(2)
expect(filtered.find((t) => t.id === 2)).toBeUndefined()
component['_nameFilter'] = 'Tag2' // Simulate non-empty name filter
const filteredWithName = component.filterData(tags as any)
expect(filteredWithName.length).toBe(2)
expect(filteredWithName).toHaveLength(2)
expect(filteredWithName.find((t) => t.id === 2)).toBeUndefined()
expect(
filteredWithName
@@ -26,7 +26,7 @@
<th scope="col" style="width: 40px;">
<div class="form-check m-0 ms-2 me-n2">
<input type="checkbox" class="form-check-input" id="all-objects" [ngModel]="toggleAllEnabled()" (ngModelChange)="toggleAllEnabled.set($event)" [disabled]="processedMails().length === 0" (click)="toggleAll($event); $event.stopPropagation();">
<label class="form-check-label" for="all-objects"></label>
<label class="form-check-label" for="all-objects"><span class="visually-hidden" i18n>Select all processed email messages</span></label>
</div>
</th>
<th scope="col" i18n>Subject</th>
@@ -5,6 +5,6 @@
pre {
white-space: pre-wrap;
word-break: break-word;
overflow-wrap: break-word;
}
}
+1 -6
View File
@@ -1,12 +1,7 @@
import { normalizeSync } from 'normalize-diacritics'
export type SearchTextValue =
| string
| number
| boolean
| bigint
| null
| undefined
string | number | boolean | bigint | null | undefined
export function normalizeSearchText(value: SearchTextValue): string {
return normalizeSync(String(value ?? '')).toLocaleLowerCase()
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More