Compare commits

..
Author SHA1 Message Date
shamoonandGitHub 2ab1ac7cb8 [Beta] Paperless-ngx v3.0.0 Beta (#12713) 2026-07-22 10:55:13 -07:00
shamoon c2d9c52cde Merge branch 'main' into beta 2026-07-22 10:30:57 -07:00
cdb11b94b5 New Crowdin translations by GitHub Action (#13153)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-22 10:30:33 -07:00
GitHub Actions cfd7a3a7f0 Auto translate strings 2026-07-22 17:18:01 +00:00
shamoon c6329f4b76 Merge branch 'beta' into dev 2026-07-22 10:16:33 -07:00
shamoonandGitHub 5e06987102 Chore (beta): mark first batch of thumbnails as priorty for LCP warning (#13196) 2026-07-22 10:15:56 -07:00
shamoonandGitHub 3f93557934 Fix (beta): prevent pdfjs offsetParent warning (#13197) 2026-07-22 10:06:41 -07:00
shamoon afe6ad2192 Chore: remove unused import 2026-07-22 09:23:18 -07:00
GitHub Actions a3f9fa0369 Auto translate strings 2026-07-22 15:45:25 +00:00
shamoonandGitHub e106618c39 Fix (beta): dont include hidden buttons in virtual scroll container height (#13194) 2026-07-22 08:42:31 -07:00
shamoonandGitHub d113826cf8 Tweakhancement: allow some bulk operations with all (#13193) 2026-07-22 08:28:49 -07:00
shamoonandGitHub c063b3799f Fix (beta): fix custom field bulk editing for 'all' (#13191) 2026-07-22 08:14:53 -07:00
81795bc93a Security (beta): enforce current permissions in autocomplete (#13188)
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-22 08:07:43 -07:00
shamoonandGitHub 6664f05543 Security (beta): bound email linkification (#13187) 2026-07-22 07:41:12 -07:00
shamoonandGitHub 8aab34ea73 Security (beta): two small permission check fixes (#13186) 2026-07-21 16:50:28 +00:00
8404198ec8 Fix (beta): short-circuit ObjectOwnedOrGrantedPermissionsFilter for superusers (#13183)
Mirrors the existing short-circuit already present in its sibling class,
ObjectOwnedPermissionsFilter, which this one lacked.

Benchmarked as a superuser against a 100k-document corpus: no measurable
difference. django-guardian's own get_objects_for_user() (called via
super().filter_queryset()) already returns the queryset unrestricted for
superusers before this change, so the guardian permission tables were
never actually queried for superusers either way. This change only skips
building a redundant 3-way OR of conditions that are all subsets of the
already-unrestricted queryset -- worth keeping for clarity/consistency
with the sibling filter, not as a performance claim.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 07:31:58 -07:00
shamoon 5589a584b6 Documentation: clarify PAPERLESS_URL requirement for pw reset 2026-07-10 11:05:40 -07:00
shamoonandGitHub fd0168ac6f Documentation: fix PAPERLESS_DATE_PARSER_LANGUAGES formatting (#12997) 2026-06-12 23:26:45 -07:00
131 changed files with 26359 additions and 25893 deletions
+11 -9
View File
@@ -1159,19 +1159,21 @@ still perform some basic text pre-processing before matching.
#### [`PAPERLESS_DATE_PARSER_LANGUAGES=<lang>`](#PAPERLESS_DATE_PARSER_LANGUAGES) {#PAPERLESS_DATE_PARSER_LANGUAGES}
Specifies which language Paperless should use when parsing dates from documents.
: Specifies which language Paperless should use when parsing dates from documents.
This should be a language code supported by the dateparser library,
for example: "en", or a combination such as "en+de".
Locales are also supported (e.g., "en-AU").
Multiple languages can be combined using "+", for example: "en+de" or "en-AU+de".
For valid values, refer to the list of supported languages and locales in the [dateparser documentation](https://dateparser.readthedocs.io/en/latest/supported_locales.html).
: This should be a language code supported by the dateparser library,
for example: "en", or a combination such as "en+de".
Locales are also supported (e.g., "en-AU").
Multiple languages can be combined using "+", for example: "en+de" or "en-AU+de".
For valid values, refer to the list of supported languages and locales in the [dateparser documentation](https://dateparser.readthedocs.io/en/latest/supported_locales.html).
: Set this to match the languages in which most of your documents are written.
Set this to match the languages in which most of your documents are written.
If not set, Paperless will attempt to infer the language(s) from the OCR configuration (`PAPERLESS_OCR_LANGUAGE`).
!!! note
This format differs from the `PAPERLESS_OCR_LANGUAGE` setting, which uses ISO 639-2 codes (3 letters, e.g., "eng+deu" for Tesseract OCR).
!!! note
This format differs from the `PAPERLESS_OCR_LANGUAGE` setting, which uses ISO 639-2 codes (3 letters, e.g., "eng+deu" for Tesseract OCR).
#### [`PAPERLESS_EMAIL_TASK_CRON=<cron expression>`](#PAPERLESS_EMAIL_TASK_CRON) {#PAPERLESS_EMAIL_TASK_CRON}
@@ -1,662 +0,0 @@
# Mail Fetch Re-download Fix Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stop `paperless_mail` from re-downloading full IMAP message bodies for mail it has already handled, and make sure attachment-less mail under attachments-only rules is recorded so it stops matching the search forever.
**Architecture:** In `_handle_mail_rule`, replace the single "fetch every matching message's full body" call with a cheap `UID SEARCH` first, subtract UIDs already in `ProcessedMail`, and fetch bodies only for the remainder (batched). Separately, extract the existing "record `PROCESSED_WO_CONSUMPTION`" block into a shared helper and call it from the no-attachments-under-attachments-only-scope early return in `_handle_message`, which currently skips it entirely.
**Tech Stack:** Django (pytest-django), `imap_tools` (`MailBox.uids()`, `AND` query builder), existing `ProcessedMail` model — no new dependencies, no migrations.
## Global Constraints
- No new required settings and no DB migrations for existing installs (spec: "No new required settings/migrations for existing installs").
- Batch size for body fetches is a hardcoded module constant (`MAIL_FETCH_BATCH_SIZE = 500`), not user-configurable.
- No mailbox action (mark-read/flag/tag/move/delete) is applied for the no-attachment/no-consumption case — only a `ProcessedMail` row is written, matching the existing precedent for other no-consumption outcomes.
- Every path that currently produces a document or applies a mail action must behave exactly as before (no change to `_process_attachments`'/`_process_eml`'s consuming behavior, action application, or the existing per-message `already_processed` dedup check, which stays in place as a safety net).
- Follow the existing per-message `uid_validity` matching semantics exactly: when `self._current_uid_validity is not None`, only match `ProcessedMail` rows with the same `uid_validity` or `uid_validity IS NULL`; when it is `None` (server didn't report UIDVALIDITY), fall back to matching on `(rule, uid, folder)` alone, ignoring `uid_validity`. (This exact conditional already exists at `mail.py:722-726` for the per-message check — the new UID-diff query must replicate it or several existing uidvalidity tests will break.)
---
### Task 1: Add UID search support to the `BogusMailBox` test double
**Files:**
- Modify: `src/paperless_mail/tests/test_mail.py:137-171` (`BogusMailBox.fetch`)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Produces: `BogusMailBox.uids(self, criteria, charset="") -> list[str]`, filtering with the same rules as `fetch()`, returning only UIDs (no message bodies). `BogusMailBox.fetch()` gains support for a `UID <comma-list>` criteria token (used by real code's `AND(uid=[...])` queries).
- Consumes: nothing new — this is test infrastructure only, no production code changes in this task.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py` (near the other `BogusMailBox`-adjacent tests, e.g. after `test_handle_message`):
```python
def test_bogus_mailbox_uids_and_uid_criteria(self) -> None:
mailbox = self.mailMocker.bogus_mailbox
all_messages = list(mailbox.messages)
# uids() returns the UIDs of unseen messages, no bodies needed to call it
unseen_uids = mailbox.uids("(UNSEEN)")
self.assertEqual(
set(unseen_uids),
{m.uid for m in all_messages if not m.seen},
)
# fetch() with an explicit UID criteria returns only the matching messages
target_uid = all_messages[0].uid
from imap_tools import AND
fetched = mailbox.fetch(AND(uid=[target_uid]), mark_seen=False)
self.assertEqual([m.uid for m in fetched], [target_uid])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_bogus_mailbox_uids_and_uid_criteria -v`
Expected: FAIL with `AttributeError: 'BogusMailBox' object has no attribute 'uids'` (and/or the UID-criteria fetch returning all 3 default messages instead of 1, since `fetch()` doesn't understand a `UID` token yet).
- [ ] **Step 3: Implement `uids()` and `UID` criteria support**
Replace `src/paperless_mail/tests/test_mail.py:137-171`:
```python
def fetch(self, criteria, mark_seen, charset="", *, bulk=True):
return self._filter_messages(criteria)
def uids(self, criteria, charset="") -> list[str]:
return [m.uid for m in self._filter_messages(criteria)]
def _filter_messages(self, criteria):
msg = self.messages
criteria = str(criteria).strip("()").split(" ")
if "UNSEEN" in criteria:
msg = filter(lambda m: not m.seen, msg)
if "SUBJECT" in criteria:
subject = criteria[criteria.index("SUBJECT") + 1].strip('"')
msg = filter(lambda m: subject in m.subject, msg)
if "BODY" in criteria:
body = criteria[criteria.index("BODY") + 1].strip('"')
msg = filter(lambda m: body in m.text, msg)
if "FROM" in criteria:
from_ = criteria[criteria.index("FROM") + 1].strip('"')
msg = filter(lambda m: from_ in m.from_, msg)
if "TO" in criteria:
to_ = criteria[criteria.index("TO") + 1].strip('"')
msg = filter(lambda m: any(to_ in to_addr for to_addr in m.to), msg)
if "UNFLAGGED" in criteria:
msg = filter(lambda m: not m.flagged, msg)
if "UNKEYWORD" in criteria:
tag = criteria[criteria.index("UNKEYWORD") + 1].strip("'")
msg = filter(lambda m: tag not in m.flags, msg)
if "(X-GM-LABELS" in criteria: # ['NOT', '(X-GM-LABELS', '"processed"']
msg = filter(lambda m: "processed" not in m.flags, msg)
if "UID" in criteria:
uid_list = criteria[criteria.index("UID") + 1].split(",")
msg = filter(lambda m: m.uid in uid_list, msg)
return list(msg)
```
This is a pure refactor of the existing filtering logic into `_filter_messages`, reused by both `fetch()` (unchanged behavior) and the new `uids()`, plus one new `if "UID" in criteria` branch.
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_bogus_mailbox_uids_and_uid_criteria -v`
Expected: PASS
- [ ] **Step 5: Run the full existing `test_mail.py` suite to confirm the refactor didn't change `fetch()` behavior**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all tests PASS (same pass count as before this task)
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/tests/test_mail.py
git commit -m "test: add BogusMailBox.uids() and UID criteria support"
```
---
### Task 2: Record `ProcessedMail` for attachment-less mail under attachments-only rules
**Files:**
- Modify: `src/paperless_mail/mail.py:749-793` (`_handle_message`)
- Modify: `src/paperless_mail/mail.py:952-979` (`_process_attachments`, tail)
- Test: `src/paperless_mail/tests/test_mail.py:540-548` (`test_handle_empty_message`, rewritten)
**Interfaces:**
- Produces: `MailAccountHandler._record_processed_without_consumption(self, message: MailMessage, rule: MailRule) -> None` — idempotently writes a `ProcessedMail(status="PROCESSED_WO_CONSUMPTION")` row for `(rule, message.uid, rule.folder)` if one doesn't already exist for the current `self._current_uid_validity`. No mailbox action is applied.
- Consumes: `self._current_uid_validity` (already set by `_handle_mail_rule` before `_handle_message` is called; `None` when called directly, e.g. in unit tests).
- [ ] **Step 1: Write the failing test**
Replace `src/paperless_mail/tests/test_mail.py:540-548` (`test_handle_empty_message`):
```python
def test_handle_empty_message(self) -> None:
message = self.mailMocker.messageBuilder.create_message(
subject="No attachments here",
attachments=[],
)
account = MailAccount.objects.create()
rule = MailRule.objects.create(
account=account,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
result = self.mail_account_handler._handle_message(message, rule)
self.mailMocker._queue_consumption_tasks_mock.assert_not_called()
self.assertEqual(result, 0)
processed = ProcessedMail.objects.get(
rule=rule,
uid=message.uid,
folder=rule.folder,
)
self.assertEqual(processed.status, "PROCESSED_WO_CONSUMPTION")
# Calling it again must not create a second row
self.mail_account_handler._handle_message(message, rule)
self.assertEqual(
ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
).count(),
1,
)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_empty_message -v`
Expected: FAIL — `ProcessedMail.DoesNotExist` (no row is created by the current early return).
- [ ] **Step 3: Add the shared helper and call it from both places**
Insert a new method right after `_handle_message` ends, i.e. after `src/paperless_mail/mail.py:793` (`return processed_elements`), before `def filename_inclusion_matches`:
```python
def _record_processed_without_consumption(
self,
message: MailMessage,
rule: MailRule,
) -> None:
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date)
if is_naive(message.date)
else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
```
Then modify `_handle_message`'s early return at `src/paperless_mail/mail.py:756-760`:
```python
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements
```
becomes:
```python
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
self._record_processed_without_consumption(message, rule)
return processed_elements
```
Then replace the tail of `_process_attachments` at `src/paperless_mail/mail.py:952-979`:
```python
if len(consume_tasks) > 0:
queue_consumption_tasks(
consume_tasks=consume_tasks,
rule=rule,
message=message,
uid_validity=self._current_uid_validity,
)
else:
# No files to consume, just mark as processed if it wasn't by .eml processing
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date)
if is_naive(message.date)
else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
return processed_attachments
```
with:
```python
if len(consume_tasks) > 0:
queue_consumption_tasks(
consume_tasks=consume_tasks,
rule=rule,
message=message,
uid_validity=self._current_uid_validity,
)
else:
# No files to consume, just mark as processed if it wasn't by .eml processing
self._record_processed_without_consumption(message, rule)
return processed_attachments
```
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_empty_message -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite (regression check for the `_process_attachments` refactor)**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "fix: record ProcessedMail for attachment-less mail under attachments-only rules"
```
---
### Task 3: Diff UIDs against `ProcessedMail` before fetching message bodies
**Files:**
- Modify: `src/paperless_mail/mail.py:641-693` (`_handle_mail_rule`, the fetch section)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Consumes: `MailAccountHandler._record_processed_without_consumption` (Task 2), `ProcessedMail` model, `imap_tools.AND`, `M.uids()` (Task 1's `BogusMailBox.uids()` in tests; `imap_tools.MailBox.uids()` in production).
- Produces: no new public interface — `_handle_mail_rule`'s external behavior (return value, exceptions raised) is unchanged; only its internal fetch strategy changes. `_handle_mail_rule` now returns `0` immediately, without calling `M.fetch()` at all, when every UID matching the search criteria already has a `ProcessedMail` row.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py`:
```python
def test_handle_mail_account_skips_body_fetch_for_already_processed_mail(
self,
) -> None:
"""
GIVEN:
- An attachment-less mail under an attachments-only mark-read rule,
already recorded as PROCESSED_WO_CONSUMPTION
WHEN:
- The mail account is processed again and the mail still matches the
search criteria (it was never marked read, since no mail action is
applied for the no-consumption case)
THEN:
- No IMAP body fetch happens for that mail; only the cheap UID search runs.
"""
account = MailAccount.objects.create(
name="test",
imap_server="",
username="admin",
password="secret",
)
rule = MailRule.objects.create(
name="testrule",
account=account,
action=MailRule.MailAction.MARK_READ,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
message = self.mailMocker.messageBuilder.create_message(
subject="No attachment",
attachments=[],
)
self.mailMocker.bogus_mailbox.messages = [message]
self.mailMocker.bogus_mailbox.updateClient()
# First run: records ProcessedMail without consuming anything.
self.mail_account_handler.handle_mail_account(account)
self.assertTrue(
ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
).exists(),
)
self.mailMocker._queue_consumption_tasks_mock.assert_not_called()
# Second run: message still matches UNSEEN (mark-read action never ran),
# but its body must not be downloaded again.
with mock.patch.object(
self.mailMocker.bogus_mailbox,
"fetch",
wraps=self.mailMocker.bogus_mailbox.fetch,
) as fetch_spy:
self.mail_account_handler.handle_mail_account(account)
fetch_spy.assert_not_called()
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_skips_body_fetch_for_already_processed_mail -v`
Expected: FAIL — `fetch_spy.assert_not_called()` fails because the current code always calls `M.fetch()` for every message matching the search criteria, regardless of `ProcessedMail`.
- [ ] **Step 3: Implement the UID diff**
Replace `src/paperless_mail/mail.py:683-693`:
```python
try:
messages = M.fetch(
criteria=criterias,
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
with:
```python
try:
all_uids = set(
M.uids(criteria=criterias, charset=rule.account.character_set),
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while searching folder {rule.folder}",
) from err
processed_uids_qs = ProcessedMail.objects.filter(
rule=rule,
folder=rule.folder,
uid__in=all_uids,
)
if self._current_uid_validity is not None:
processed_uids_qs = processed_uids_qs.filter(
Q(uid_validity=self._current_uid_validity)
| Q(uid_validity__isnull=True),
)
processed_uids = set(processed_uids_qs.values_list("uid", flat=True))
new_uids = all_uids - processed_uids
if not new_uids:
self.log.debug(
f"Rule {rule}: No new mail matching criteria {criterias}",
)
return 0
try:
messages = M.fetch(
criteria=AND(uid=list(new_uids)),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
Note the `uid_validity` handling here deliberately mirrors the existing per-message check at `mail.py:722-726` exactly (see Global Constraints) — the extra `Q(...)` filter is applied only when `self._current_uid_validity is not None`.
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_skips_body_fetch_for_already_processed_mail -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite (regression check, especially the uidvalidity tests)**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS, including:
- `test_handle_mail_account_skip_duplicate_uids_from_fetch`
- `test_handle_mail_account_skips_mail_already_processed_in_same_uidvalidity`
- `test_handle_mail_account_processes_mail_after_uidvalidity_change`
- `test_handle_mail_account_skips_mail_processed_before_uidvalidity_tracking`
- `test_handle_mail_account_processes_mail_when_uidvalidity_unavailable`
- `test_handle_mail_account_skips_mail_when_uidvalidity_unavailable_but_prior_record_exists`
- `test_handle_mail_account_overlapping_rules_only_first_consumes`
If any of these fail, the `uid_validity` branching in Step 3 does not match `mail.py:722-726` closely enough — re-check against the Global Constraints note before changing test expectations.
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "fix: diff UIDs against ProcessedMail before fetching message bodies"
```
---
### Task 4: Batch the body fetch for large backlogs
**Files:**
- Modify: `src/paperless_mail/mail.py:74-76` (module-level constant)
- Modify: `src/paperless_mail/mail.py` (the `M.fetch(...)` block added in Task 3)
- Test: `src/paperless_mail/tests/test_mail.py` (new test in `class TestMail`)
**Interfaces:**
- Produces: module-level constant `paperless_mail.mail.MAIL_FETCH_BATCH_SIZE: int = 500`.
- Consumes: `itertools` (already imported at `mail.py:2`), `sorted_new_uids` derived from Task 3's `new_uids` set.
- [ ] **Step 1: Write the failing test**
Add to `class TestMail` in `src/paperless_mail/tests/test_mail.py`. Add `from paperless_mail.mail import MAIL_FETCH_BATCH_SIZE` to the imports at the top of the file (alongside the other `from paperless_mail.mail import ...` lines at `test_mail.py:35-38`).
```python
@mock.patch("paperless_mail.mail.MAIL_FETCH_BATCH_SIZE", 5)
def test_handle_mail_account_batches_body_fetch_for_large_backlog(self) -> None:
"""
GIVEN:
- More new/unprocessed mail than MAIL_FETCH_BATCH_SIZE
WHEN:
- The mail account is processed
THEN:
- The body fetch is issued in multiple batches
- Every message is still processed (none dropped at a batch boundary)
"""
account = MailAccount.objects.create(
name="test",
imap_server="",
username="admin",
password="secret",
)
rule = MailRule.objects.create(
name="testrule",
account=account,
action=MailRule.MailAction.MARK_READ,
consumption_scope=MailRule.ConsumptionScope.ATTACHMENTS_ONLY,
)
message_count = 12 # more than the patched batch size of 5
self.mailMocker.bogus_mailbox.messages = [
self.mailMocker.messageBuilder.create_message(
subject=f"No attachment {i}",
attachments=[],
)
for i in range(message_count)
]
self.mailMocker.bogus_mailbox.updateClient()
with mock.patch.object(
self.mailMocker.bogus_mailbox,
"fetch",
wraps=self.mailMocker.bogus_mailbox.fetch,
) as fetch_spy:
self.mail_account_handler.handle_mail_account(account)
# ceil(12 / 5) == 3 batches
self.assertEqual(fetch_spy.call_count, 3)
self.assertEqual(
ProcessedMail.objects.filter(rule=rule).count(),
message_count,
)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_batches_body_fetch_for_large_backlog -v`
Expected: FAIL — either an `ImportError` for `MAIL_FETCH_BATCH_SIZE` (doesn't exist yet) or, once that import is stubbed out, `fetch_spy.call_count == 1` instead of `3` (Task 3's implementation fetches all new UIDs in one call).
- [ ] **Step 3: Add the constant and batch the fetch**
Insert after `src/paperless_mail/mail.py:74` (right after the `APPLE_MAIL_TAG_COLORS` dict closes, before `class MailError`):
```python
MAIL_FETCH_BATCH_SIZE = 500
```
Replace the `M.fetch(...)` block added in Task 3 (Task 3 Step 3's final `try/except`):
```python
try:
messages = M.fetch(
criteria=AND(uid=list(new_uids)),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
```
with:
```python
sorted_new_uids = sorted(new_uids, key=int)
message_batches = []
for batch_start in range(0, len(sorted_new_uids), MAIL_FETCH_BATCH_SIZE):
batch = sorted_new_uids[batch_start : batch_start + MAIL_FETCH_BATCH_SIZE]
try:
message_batches.append(
M.fetch(
criteria=AND(uid=batch),
mark_seen=False,
charset=rule.account.character_set,
bulk=True,
),
)
except Exception as err:
raise MailError(
f"Rule {rule}: Error while fetching folder {rule.folder}",
) from err
messages = itertools.chain(*message_batches)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py::TestMail::test_handle_mail_account_batches_body_fetch_for_large_backlog -v`
Expected: PASS
- [ ] **Step 5: Run the full `test_mail.py` suite**
Run: `uv run pytest --override-ini="addopts=" src/paperless_mail/tests/test_mail.py -v`
Expected: all PASS
- [ ] **Step 6: Commit**
```bash
git add src/paperless_mail/mail.py src/paperless_mail/tests/test_mail.py
git commit -m "perf: batch body fetches when many new UIDs are pending"
```
---
### Task 5: Full regression pass and lint
**Files:** none (verification only, plus any fixups this step surfaces)
- [ ] **Step 1: Run the full backend test suite with coverage/parallelism as configured**
Run: `uv run pytest src/paperless_mail/`
Expected: all `paperless_mail` tests PASS
- [ ] **Step 2: Run the full project test suite**
Run: `uv run pytest`
Expected: all tests PASS (no regressions outside `paperless_mail`)
- [ ] **Step 3: Lint and format**
Run: `uv run ruff check src/paperless_mail/` and `uv run ruff format --check src/paperless_mail/`
Expected: no errors. If `ruff format --check` reports files needing formatting, run `uv run ruff format src/paperless_mail/` and re-check.
- [ ] **Step 4: Type check against the frozen baselines**
Run: `uv run mypy src/paperless_mail/mail.py` (or the project's standard mypy invocation) and confirm no new violations beyond `.mypy-baseline.txt`.
Expected: no new errors introduced by this change.
- [ ] **Step 5: Commit any fixups**
If Steps 1-4 required changes:
```bash
git add -u
git commit -m "chore: fix lint/type fallout from mail fetch fix"
```
If no changes were needed, skip this step — nothing to commit.
@@ -1,216 +0,0 @@
# Mail fetch fix: avoid re-downloading already-handled IMAP mail
## Problem
`paperless_mail`'s `_handle_mail_rule` (`src/paperless_mail/mail.py:641`) fetches
the full RFC822 body of every message an IMAP `SEARCH` matches (via
`M.fetch(criteria=criterias, ..., bulk=True)`), then does de-duplication in
Python afterward. Two things compound into a real bug (reported upstream in
[paperless-ngx#13175](https://github.com/paperless-ngx/paperless-ngx/issues/13175)):
1. **Full-body fetch happens before dedup.** The de-dup check against
`ProcessedMail` (`mail.py:717-731`) happens per-message, after the body has
already been downloaded for every matching message. Steady-state cost is
_O(everything the search matches)_, not _O(new since last run)_.
2. **Attachment-less mail under an attachments-only rule is never recorded.**
`_handle_message` (`mail.py:749`) returns early, before any attachment
processing, when a message has no attachments and
`rule.consumption_scope == ATTACHMENTS_ONLY`:
```python
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements # 0, no ProcessedMail row, no mail action applied
```
No `ProcessedMail` row is written and no mail action (mark-read/flag/tag)
is applied, since both only happen via `queue_consumption_tasks`, which is
only reached from the attachment/eml processing paths. The message keeps
matching the search (e.g. `UNSEEN`) forever.
Combined: an inbox where most mail has no attachments gets fully re-downloaded
on every scheduled run (default every 10 minutes), indefinitely, for large
mailboxes -- confirmed against the current codebase, not user
misconfiguration.
## Goals
- Stop re-downloading full message bodies for mail that has already been
handled (processed into a document, or determined to produce nothing).
- Record `ProcessedMail` for the no-attachment / attachments-only early-return
case, so it participates in dedup like every other terminal outcome.
- No new required settings or DB migrations for existing installs.
- Graceful behavior on first run against a large existing mailbox (no single
giant IMAP command, no giant in-memory batch).
- Preserve existing action/consumption semantics exactly for every path that
currently produces a document or applies a mail action.
## Non-goals
- Changing what mail actions (mark-read/flag/tag/move/delete) get applied, or
when.
- Applying a mail action to the no-attachment early-return case (explicitly
out of scope -- see Decisions).
- Making batch size user-configurable.
- Touching EML_ONLY / EVERYTHING consumption scopes' semantics (they already
don't hit the early-return branch described above).
## Decisions
These were settled during brainstorming and are load-bearing for the design
below:
- **Fix both halves together.** Recording `ProcessedMail` alone does not fix
the bandwidth problem: without a mail action applied, an attachment-less
message stays unseen and keeps matching the search criteria, so its body
gets re-downloaded every run regardless of whether it gets reprocessed.
Fixing only the bandwidth side without recording `ProcessedMail` would mean
the UID-diff step never has anything to exclude for these messages. They
must ship together.
- **No mailbox action for the no-attachment case.** Only a `ProcessedMail`
row is written; the message is not marked read/flagged/moved/deleted. This
matches the existing precedent for other no-consumption outcomes
(`mail.py:960-977`, e.g. an attachment present but filtered out or an
unsupported mime type) and avoids changing the user's mailbox state for
mail paperless previously never touched.
- **UID-diff-before-fetch, not a BODYSTRUCTURE probe or SEARCH-side
exclusion.** Considered and rejected:
- BODYSTRUCTURE probing (fetch structure only, decide whether to fetch full
body) only helps the attachments-only scope, not the general
already-processed case, and `imap_tools` doesn't cleanly expose a
structure-only fetch.
- Excluding already-processed UIDs directly in the IMAP `SEARCH` criteria
(`NOT UID (...)`) was rejected because for tens of thousands of
already-processed UIDs the excluded-UID list itself blows up the command
size -- worse than the two-step approach.
- **Batch size is a hardcoded constant**, not a new setting, per the
"no new required settings" goal.
## Design
### `_handle_mail_rule` (`mail.py:641`)
Replace the single `M.fetch(criteria=criterias, ...)` call with:
1. `self._current_uid_validity` computed as today (unchanged, already first).
2. `criterias = make_criterias(...)` (unchanged).
3. `all_uids = set(M.uids(criteria=criterias, charset=rule.account.character_set))`
-- a `UID SEARCH`, no bodies.
4. Query already-processed UIDs in one DB round trip, reusing the same
uid_validity matching already used per-message:
```python
processed_uids = set(
ProcessedMail.objects.filter(
rule=rule, folder=rule.folder, uid__in=all_uids,
).filter(
Q(uid_validity=self._current_uid_validity) | Q(uid_validity__isnull=True),
).values_list("uid", flat=True)
)
```
5. `new_uids = all_uids - processed_uids`. If empty: log at debug level and
`return 0` -- no body fetch at all. This is the steady-state case.
6. Otherwise, iterate `new_uids` in fixed-size batches of
`MAIL_FETCH_BATCH_SIZE = 500` (module-level constant), calling
`M.fetch(criteria=AND(uid=batch), mark_seen=False, charset=rule.account.character_set, bulk=True)`
per batch. Chain the resulting message iterators into the existing
per-message loop (`mail.py:699+`) unchanged.
7. The existing per-message `already_processed` DB check
(`mail.py:717-731`) stays in place unchanged, as a safety net against
races (e.g. concurrent rule runs against the same folder in this pass) and
to keep behavior identical if steps 3/4 ever disagree with it.
### New helper: `_record_processed_without_consumption`
Extract the existing dedup-and-create block at `mail.py:960-977` into a
method on the same class:
```python
def _record_processed_without_consumption(self, message, rule) -> None:
if not ProcessedMail.objects.filter(
rule=rule,
uid=message.uid,
folder=rule.folder,
uid_validity=self._current_uid_validity,
).exists():
ProcessedMail.objects.create(
rule=rule,
folder=rule.folder,
uid=message.uid,
uid_validity=self._current_uid_validity,
subject=message.subject,
received=make_aware(message.date) if is_naive(message.date) else message.date,
status="PROCESSED_WO_CONSUMPTION",
)
```
Call sites:
- `_process_attachments`'s existing no-consumables branch (replaces the
inline block, same behavior).
- `_handle_message`'s early return (`mail.py:756-760`), newly, for the
no-attachments-under-attachments-only-scope case.
### Data flow (steady state, nothing new)
search criteria (unchanged) -> `UID SEARCH` for matching UIDs (cheap) ->
subtract UIDs already in `ProcessedMail` (cheap, DB-only) -> zero new UIDs ->
no fetch, no body download.
### Data flow (new mail present)
... same as above -> some UIDs remain -> fetch bodies only for those,
batched -> existing per-message processing, action application, and
`ProcessedMail` recording, unchanged except the no-attachment early return
now also calls `_record_processed_without_consumption`.
## Error handling
- `M.uids(...)` raising is wrapped in the same `try/except` that currently
wraps `M.fetch(...)` (`mail.py:683-693`), surfacing as `MailError`
identically to today's search-failure behavior.
- A batch's `M.fetch(...)` raising mid-loop is caught by the same
`try/except`, applied per-batch. One failing batch fails the whole rule
run for this pass -- matching today's all-or-nothing semantics (currently
a single failed fetch already fails the whole rule).
- Empty `new_uids` short-circuits before any fetch, so it can't hit fetch
error paths at all for the common case.
- `_record_processed_without_consumption` reuses the existing
`ProcessedMail.objects.create` call already in production use; no new
failure mode introduced.
## Testing
In `src/paperless_mail/tests/test_mail.py`:
- Extend the `BogusMailBox` test double with a `uids()` method that mirrors
its existing `fetch()` criteria matching but returns only UIDs.
- New test: an attachment-less message under an attachments-only mark-read
rule is recorded as `PROCESSED_WO_CONSUMPTION` on the first run. On a
second run, with the same message still unseen server-side, assert no
body fetch happens (only `M.uids()` is called) -- i.e. steady state costs
one search and zero body downloads.
- New test: a folder with more new UIDs than `MAIL_FETCH_BATCH_SIZE` results
in multiple batched `fetch` calls, and all messages are still processed
(no messages dropped at a batch boundary).
- Update existing `test_handle_empty_message` to assert a
`PROCESSED_WO_CONSUMPTION` row now exists for the no-attachment case.
- Regression: existing tests for `EML_ONLY`/`EVERYTHING` scopes and the
multi-rule-same-folder dedup (`consumed_messages` set) should pass
unchanged, since neither the per-message `already_processed` check nor the
`consumed_messages` set logic is touched by this design.
## Open questions / risks
- `imap_tools`'s `AND(uid=batch)` criteria builder needs to be confirmed to
produce a valid `UID FETCH <list>` command at the batch sizes used here;
covered by the batching test above.
- If a mail server doesn't support `UID SEARCH` the same way it supports the
existing `fetch`'s implicit search, `M.uids()` could behave differently
from today's `M.fetch()` on some edge-case server. Existing project test
coverage (via `BogusMailBox`) won't catch server-specific quirks; this is
the same class of risk as any IMAP-behavior assumption already baked into
this module.
+2 -2
View File
@@ -458,8 +458,8 @@ For related metadata such as tags, correspondents, document types, and storage p
### Password reset
In order to enable the password reset feature you will need to setup an SMTP backend, see
[`PAPERLESS_EMAIL_HOST`](configuration.md#PAPERLESS_EMAIL_HOST). If your installation does not have
[`PAPERLESS_URL`](configuration.md#PAPERLESS_URL) set, the reset link included in emails will use the server host.
[`PAPERLESS_EMAIL_HOST`](configuration.md#PAPERLESS_EMAIL_HOST). You should also set
[`PAPERLESS_URL`](configuration.md#PAPERLESS_URL) and / or its corresponding configuration settings.
### Two-factor authentication
+87 -68
View File
@@ -1317,7 +1317,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">236</context>
<context context-type="linenumber">237</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -1615,7 +1615,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">205</context>
<context context-type="linenumber">206</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -1646,7 +1646,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">245</context>
<context context-type="linenumber">246</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -1677,7 +1677,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">254</context>
<context context-type="linenumber">255</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -1716,7 +1716,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">218</context>
<context context-type="linenumber">219</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.html</context>
@@ -2065,7 +2065,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">263</context>
<context context-type="linenumber">264</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -2236,11 +2236,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">72</context>
<context context-type="linenumber">76</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">143</context>
<context context-type="linenumber">147</context>
</context-group>
</trans-unit>
<trans-unit id="3278307631146748151" datatype="html">
@@ -3477,11 +3477,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">78</context>
<context context-type="linenumber">82</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">149</context>
<context context-type="linenumber">153</context>
</context-group>
</trans-unit>
<trans-unit id="searchResults.noResults" datatype="html">
@@ -3833,7 +3833,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">75</context>
<context context-type="linenumber">79</context>
</context-group>
</trans-unit>
<trans-unit id="3188389494264426470" datatype="html">
@@ -4067,7 +4067,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">272</context>
<context context-type="linenumber">273</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -5906,7 +5906,7 @@
<source>Create</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.html</context>
<context context-type="linenumber">56</context>
<context context-type="linenumber">54</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/share-links-dialog/share-links-dialog.component.html</context>
@@ -5921,14 +5921,14 @@
<source>Apply</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.html</context>
<context context-type="linenumber">62</context>
<context context-type="linenumber">60</context>
</context-group>
</trans-unit>
<trans-unit id="7780041345210191160" datatype="html">
<source>Click again to exclude items.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.html</context>
<context context-type="linenumber">75</context>
<context context-type="linenumber">73</context>
</context-group>
</trans-unit>
<trans-unit id="7593728289020204896" datatype="html">
@@ -5943,7 +5943,7 @@
<source>Open <x id="PH" equiv-text="this.title"/> filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/common/filterable-dropdown/filterable-dropdown.component.ts</context>
<context context-type="linenumber">824</context>
<context context-type="linenumber">828</context>
</context-group>
</trans-unit>
<trans-unit id="7005745151564974365" datatype="html">
@@ -7400,11 +7400,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">25</context>
<context context-type="linenumber">29</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">317</context>
<context context-type="linenumber">318</context>
</context-group>
</trans-unit>
<trans-unit id="78870852467682010" datatype="html">
@@ -7415,11 +7415,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">96</context>
<context context-type="linenumber">100</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">357</context>
<context context-type="linenumber">358</context>
</context-group>
</trans-unit>
<trans-unit id="157572966557284263" datatype="html">
@@ -7430,11 +7430,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">102</context>
<context context-type="linenumber">106</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">364</context>
<context context-type="linenumber">365</context>
</context-group>
</trans-unit>
<trans-unit id="883965278435032344" datatype="html">
@@ -7452,7 +7452,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">385</context>
<context context-type="linenumber">386</context>
</context-group>
</trans-unit>
<trans-unit id="3542042671420335679" datatype="html">
@@ -7463,7 +7463,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">385</context>
<context context-type="linenumber">386</context>
</context-group>
</trans-unit>
<trans-unit id="872092479747931526" datatype="html">
@@ -7713,7 +7713,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">69</context>
<context context-type="linenumber">73</context>
</context-group>
</trans-unit>
<trans-unit id="2336375155355449543" datatype="html">
@@ -7767,7 +7767,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">215</context>
<context context-type="linenumber">216</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
@@ -8739,28 +8739,28 @@
<source>Custom fields updated.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1024</context>
<context context-type="linenumber">1025</context>
</context-group>
</trans-unit>
<trans-unit id="3873496751167944011" datatype="html">
<source>Error updating custom fields.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1033</context>
<context context-type="linenumber">1034</context>
</context-group>
</trans-unit>
<trans-unit id="6144801143088984138" datatype="html">
<source>Share link bundle creation requested.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1073</context>
<context context-type="linenumber">1082</context>
</context-group>
</trans-unit>
<trans-unit id="46019676931295023" datatype="html">
<source>Share link bundle creation is not available yet.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/bulk-editor/bulk-editor.component.ts</context>
<context context-type="linenumber">1080</context>
<context context-type="linenumber">1089</context>
</context-group>
</trans-unit>
<trans-unit id="6307402210351946694" datatype="html">
@@ -8785,89 +8785,108 @@
<context context-type="linenumber">73,78</context>
</context-group>
</trans-unit>
<trans-unit id="7575628227893972164" datatype="html">
<source><x id="INTERPOLATION" equiv-text="document().title | documentTitle }}"/> thumbnail</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">6</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">8</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">6</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">8</context>
</context-group>
</trans-unit>
<trans-unit id="2784168796433474565" datatype="html">
<source>Filter by tag</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">36</context>
<context context-type="linenumber">40</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">333</context>
<context context-type="linenumber">334</context>
</context-group>
</trans-unit>
<trans-unit id="106713086593101376" datatype="html">
<source>View notes</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">91</context>
<context context-type="linenumber">95</context>
</context-group>
</trans-unit>
<trans-unit id="3727324658595204357" datatype="html">
<source>Created: <x id="INTERPOLATION" equiv-text="{{ document().created | customDate }}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">115,116</context>
<context context-type="linenumber">119,120</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">76,77</context>
<context context-type="linenumber">80,81</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">91,92</context>
<context context-type="linenumber">95,96</context>
</context-group>
</trans-unit>
<trans-unit id="2030261243264601523" datatype="html">
<source>Added: <x id="INTERPOLATION" equiv-text="{{ document().added | customDate }}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">116,117</context>
<context context-type="linenumber">120,121</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">77,78</context>
<context context-type="linenumber">81,82</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">92,93</context>
<context context-type="linenumber">96,97</context>
</context-group>
</trans-unit>
<trans-unit id="4235671847487610290" datatype="html">
<source>Modified: <x id="INTERPOLATION" equiv-text="{{ document().modified | customDate }}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">117,118</context>
<context context-type="linenumber">121,122</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">78,79</context>
<context context-type="linenumber">82,83</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">93,94</context>
<context context-type="linenumber">97,98</context>
</context-group>
</trans-unit>
<trans-unit id="197162226430950645" datatype="html">
<source>{VAR_PLURAL, plural, =1 {1 page} other {<x id="INTERPOLATION"/> pages}}</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">134</context>
<context context-type="linenumber">138</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">106</context>
<context context-type="linenumber">110</context>
</context-group>
</trans-unit>
<trans-unit id="5739581984228459958" datatype="html">
<source>Shared</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">144</context>
<context context-type="linenumber">148</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">125</context>
<context context-type="linenumber">129</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -8882,35 +8901,35 @@
<source>Score:</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-large/document-card-large.component.html</context>
<context context-type="linenumber">149</context>
<context context-type="linenumber">153</context>
</context-group>
</trans-unit>
<trans-unit id="3661756380991326939" datatype="html">
<source>Toggle tag filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">20</context>
<context context-type="linenumber">24</context>
</context-group>
</trans-unit>
<trans-unit id="4648526799630820486" datatype="html">
<source>Toggle correspondent filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">43</context>
<context context-type="linenumber">47</context>
</context-group>
</trans-unit>
<trans-unit id="5319701482646590642" datatype="html">
<source>Toggle document type filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">59</context>
<context context-type="linenumber">63</context>
</context-group>
</trans-unit>
<trans-unit id="8950368321707344185" datatype="html">
<source>Toggle storage path filter</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-card-small/document-card-small.component.html</context>
<context context-type="linenumber">66</context>
<context context-type="linenumber">70</context>
</context-group>
</trans-unit>
<trans-unit id="3797570084942068182" datatype="html">
@@ -9059,14 +9078,14 @@
<source>Sort by ASN</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">192</context>
<context context-type="linenumber">193</context>
</context-group>
</trans-unit>
<trans-unit id="7517688192215738656" datatype="html">
<source>ASN</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">196</context>
<context context-type="linenumber">197</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
@@ -9085,28 +9104,28 @@
<source>Sort by correspondent</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">201</context>
<context context-type="linenumber">202</context>
</context-group>
</trans-unit>
<trans-unit id="2066713941761361709" datatype="html">
<source>Sort by title</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">210</context>
<context context-type="linenumber">211</context>
</context-group>
</trans-unit>
<trans-unit id="6232673011753681091" datatype="html">
<source>Sort by owner</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">223</context>
<context context-type="linenumber">224</context>
</context-group>
</trans-unit>
<trans-unit id="3715596725146409911" datatype="html">
<source>Owner</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">227</context>
<context context-type="linenumber">228</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -9121,49 +9140,49 @@
<source>Sort by notes</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">232</context>
<context context-type="linenumber">233</context>
</context-group>
</trans-unit>
<trans-unit id="5499001829734502606" datatype="html">
<source>Sort by document type</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">241</context>
<context context-type="linenumber">242</context>
</context-group>
</trans-unit>
<trans-unit id="6213829731736042759" datatype="html">
<source>Sort by storage path</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">250</context>
<context context-type="linenumber">251</context>
</context-group>
</trans-unit>
<trans-unit id="3406167410329973166" datatype="html">
<source>Sort by created date</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">259</context>
<context context-type="linenumber">260</context>
</context-group>
</trans-unit>
<trans-unit id="3769035778779263084" datatype="html">
<source>Sort by added date</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">268</context>
<context context-type="linenumber">269</context>
</context-group>
</trans-unit>
<trans-unit id="4874754501044009042" datatype="html">
<source>Sort by number of pages</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">277</context>
<context context-type="linenumber">278</context>
</context-group>
</trans-unit>
<trans-unit id="3817498941817715969" datatype="html">
<source>Pages</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">281</context>
<context context-type="linenumber">282</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/document.ts</context>
@@ -9182,28 +9201,28 @@
<source> Shared </source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">284,286</context>
<context context-type="linenumber">285,287</context>
</context-group>
</trans-unit>
<trans-unit id="5083658411133224968" datatype="html">
<source>Sort by <x id="INTERPOLATION" equiv-text="{{getDisplayCustomFieldTitle(field_id)}}"/></source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">291,292</context>
<context context-type="linenumber">292,293</context>
</context-group>
</trans-unit>
<trans-unit id="2179847500064178686" datatype="html">
<source>Edit document</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">325</context>
<context context-type="linenumber">326</context>
</context-group>
</trans-unit>
<trans-unit id="3420321797707163677" datatype="html">
<source>Preview document</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.html</context>
<context context-type="linenumber">326</context>
<context context-type="linenumber">327</context>
</context-group>
</trans-unit>
<trans-unit id="4512084577073831437" datatype="html">
@@ -34,30 +34,28 @@
</div>
@if (selectionModel.items) {
<cdk-virtual-scroll-viewport class="items" [itemSize]="FILTERABLE_BUTTON_HEIGHT_PX" #buttonsViewport [style.height.px]="scrollViewportHeight">
<div *cdkVirtualFor="let item of selectionModel.items | filter: filterText:'name'; trackBy: trackByItem; let i = index">
@if (allowSelectNone || item.id) {
<pngx-toggleable-dropdown-button
[item]="item"
[hideCount]="hideCount(item)"
[opacifyCount]="!editing"
[state]="selectionModel.get(item.id)"
[count]="getUpdatedDocumentCount(item.id)"
(toggled)="selectionModel.toggle(item.id)"
(exclude)="excludeClicked(item.id)"
[disabled]="disabled">
</pngx-toggleable-dropdown-button>
}
<div *cdkVirtualFor="let item of filteredItems; trackBy: trackByItem; let i = index">
<pngx-toggleable-dropdown-button
[item]="item"
[hideCount]="hideCount(item)"
[opacifyCount]="!editing"
[state]="selectionModel.get(item.id)"
[count]="getUpdatedDocumentCount(item.id)"
(toggled)="selectionModel.toggle(item.id)"
(exclude)="excludeClicked(item.id)"
[disabled]="disabled">
</pngx-toggleable-dropdown-button>
</div>
</cdk-virtual-scroll-viewport>
}
@if (editing) {
@if ((selectionModel.items | filter: filterText:'name').length === 0 && createRef !== undefined) {
@if (filteredItems.length === 0 && createRef !== undefined) {
<button class="list-group-item list-group-item-action bg-light" (click)="createClicked()" [disabled]="disabled">
<small class="ms-2"><ng-container i18n>Create</ng-container> "{{filterText}}"</small>
<i-bs width="1.5em" height="1em" name="plus"></i-bs>
</button>
}
@if ((selectionModel.items | filter: filterText:'name').length > 0) {
@if (filteredItems.length > 0) {
<button class="list-group-item list-group-item-action bg-light d-flex align-items-center" (click)="applyClicked()" [disabled]="!modelIsDirty() || disabled">
<small class="ms-2" [ngClass]="{'fw-bold': modelIsDirty()}" i18n>Apply</small>
<i-bs width="1.5em" height="1em" name="arrow-right"></i-bs>
@@ -265,7 +265,9 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
expect(document.activeElement).toEqual(
component.listFilterTextInput.nativeElement
)
expect(component.buttonsViewport.getRenderedRange().end).toEqual(3) // all items shown
expect(component.buttonsViewport.getRenderedRange().end).toEqual(
items.length
) // all selectable items shown
component.filterText = 'Tag2'
fixture.detectChanges()
@@ -278,6 +280,29 @@ describe('FilterableDropdownComponent & FilterableDropdownSelectionModel', () =>
expect(component.filterText).toHaveLength(0)
})
it('should omit disallowed null items from the virtual scroll viewport', async () => {
component.selectionModel.items = items
component.icon = 'tag-fill'
fixture.nativeElement
.querySelector('button')
.dispatchEvent(new MouseEvent('click')) // open
fixture.detectChanges()
await wait(100)
fixture.detectChanges()
component.buttonsViewport?.checkViewportSize()
fixture.detectChanges()
expect(component.filteredItems).toEqual(items)
expect(component.scrollViewportHeight).toEqual(
items.length * component.FILTERABLE_BUTTON_HEIGHT_PX
)
expect(
component.buttonsViewport.elementRef.nativeElement.querySelectorAll(
'.cdk-virtual-scroll-content-wrapper > div'
)
).toHaveLength(items.length)
})
it('should toggle & close on enter inside filter field if 1 item remains', async () => {
component.selectionModel.items = items
component.icon = 'tag-fill'
@@ -661,7 +661,6 @@ export class FilterableDropdownSelectionModel {
imports: [
ClearableBadgeComponent,
ToggleableDropdownButtonComponent,
FilterPipe,
FormsModule,
ReactiveFormsModule,
NgxBootstrapIconsModule,
@@ -801,12 +800,17 @@ export class FilterableDropdownComponent
private keyboardIndex: number
public get filteredItems(): MatchingModel[] {
return this.filterPipe
.transform(this.items, this.filterText, 'name')
.filter((item) => this.allowSelectNone || Boolean(item.id))
}
public get scrollViewportHeight(): number {
const filteredLength = this.filterPipe.transform(
this.items,
this.filterText
).length
return Math.min(filteredLength * this.FILTERABLE_BUTTON_HEIGHT_PX, 400)
return Math.min(
this.filteredItems.length * this.FILTERABLE_BUTTON_HEIGHT_PX,
400
)
}
constructor() {
@@ -878,7 +882,7 @@ export class FilterableDropdownComponent
}
listFilterEnter(): void {
let filtered = this.filterPipe.transform(this.items, this.filterText)
const filtered = this.filteredItems
if (filtered.length == 1) {
this.selectionModel.toggle(filtered[0].id)
setTimeout(() => {
@@ -138,6 +138,18 @@ describe('PngxPdfViewerComponent', () => {
expect(applyScaleSpy).toHaveBeenCalled()
})
it('does not reset the viewer when it is already on the requested page', async () => {
await initComponent()
const viewer = (component as any).pdfViewer as PDFViewer
const currentPageSpy = jest.spyOn(viewer, 'currentPageNumber', 'set')
component.page = viewer.currentPageNumber
;(component as any).applyViewerState()
expect(currentPageSpy).not.toHaveBeenCalled()
})
it('dispatches find when search query changes after render', async () => {
await initComponent()
@@ -256,7 +256,9 @@ export class PngxPdfViewerComponent
Math.max(Math.trunc(this.page), 1),
this.pdfViewer.pagesCount
)
this.pdfViewer.currentPageNumber = nextPage
if (nextPage !== this.pdfViewer.currentPageNumber) {
this.pdfViewer.currentPageNumber = nextPage
}
}
if (this.page === this.lastViewerPage) {
this.lastViewerPage = undefined
@@ -103,13 +103,13 @@
class="btn btn-sm btn-outline-primary"
id="dropdownSend"
ngbDropdownToggle
[disabled]="disabled || !list.hasSelection || list.allSelected"
[disabled]="disabled || !canSendSelection"
>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container>
</div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownSend" class="shadow">
<button ngbDropdownItem (click)="createShareLinkBundle()" [disabled]="list.allSelected">
<button ngbDropdownItem (click)="createShareLinkBundle()" [disabled]="!canSendSelection">
<i-bs name="link" class="me-1"></i-bs><ng-container i18n>Create a share link bundle</ng-container>
</button>
<button ngbDropdownItem (click)="manageShareLinkBundles()">
@@ -117,7 +117,7 @@
</button>
<div class="dropdown-divider"></div>
@if (emailEnabled) {
<button ngbDropdownItem (click)="emailSelected()" [disabled]="list.allSelected">
<button ngbDropdownItem (click)="emailSelected()" [disabled]="!canSendSelection">
<i-bs name="envelope" class="me-1"></i-bs><ng-container i18n>Email</ng-container>
</button>
}
@@ -208,6 +208,50 @@ describe('BulkEditorComponent', () => {
expect(component.tagSelectionModel.selectionSize()).toEqual(1)
})
it('should allow sending an all-filtered selection that fits on the current page', () => {
jest
.spyOn(documentListViewService, 'hasSelection', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'allSelected', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'selectedCount', 'get')
.mockReturnValue(5)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([1, 2, 3, 4, 5]))
fixture.detectChanges()
expect(component.canSendSelection).toBe(true)
expect(
fixture.debugElement.query(By.css('#dropdownSend')).nativeElement.disabled
).toBe(false)
})
it('should prevent sending an all-filtered selection spanning multiple pages', () => {
jest
.spyOn(documentListViewService, 'hasSelection', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'allSelected', 'get')
.mockReturnValue(true)
jest
.spyOn(documentListViewService, 'selectedCount', 'get')
.mockReturnValue(6)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([1, 2, 3, 4, 5]))
fixture.detectChanges()
expect(component.canSendSelection).toBe(false)
expect(
fixture.debugElement.query(By.css('#dropdownSend')).nativeElement.disabled
).toBe(true)
})
it('should apply selection data to correspondents menu', () => {
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
fixture.detectChanges()
@@ -1597,6 +1641,7 @@ describe('BulkEditorComponent', () => {
expect(modal.componentInstance.customFields.length).toEqual(2)
expect(modal.componentInstance.fieldsToAddIds).toEqual([1, 2])
expect(modal.componentInstance.selection).toEqual({ documents: [3, 4] })
expect(modal.componentInstance.selectionCount).toEqual(2)
expect(modal.componentInstance.documents).toEqual([3, 4])
modal.componentInstance.failed.emit()
@@ -1020,6 +1020,7 @@ export class BulkEditorComponent
)
dialog.selection = this.getSelectionQuery()
dialog.selectionCount = this.getSelectionSize()
dialog.succeeded.subscribe((result) => {
this.toastService.showInfo($localize`Custom fields updated.`)
this.list.reload()
@@ -1040,6 +1041,14 @@ export class BulkEditorComponent
return this.settings.get(SETTINGS_KEYS.EMAIL_ENABLED)
}
public get canSendSelection(): boolean {
return (
this.list.hasSelection &&
(!this.list.allSelected ||
this.list.selectedCount === this.list.selected.size)
)
}
createShareLinkBundle() {
const modal = this.modalService.open(ShareLinkBundleDialogComponent, {
backdrop: 'static',
@@ -1,9 +1,9 @@
<form [formGroup]="form" (ngSubmit)="save()" autocomplete="off">
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title" i8n>{
documents.length,
documentCount,
plural,
=1 {Set custom fields for 1 document} other {Set custom fields for {{documents.length}} documents}
=1 {Set custom fields for 1 document} other {Set custom fields for {{documentCount}} documents}
}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
@@ -42,6 +42,20 @@ describe('CustomFieldsBulkEditDialogComponent', () => {
expect(component.form.contains('2')).toBeTruthy()
})
it('should render the document count for a filtered selection', () => {
component.selection = {
all: true,
filters: { title__icontains: 'invoice' },
}
component.selectionCount = 42
fixture.detectChanges()
expect(component.documents).toEqual([])
expect(
fixture.nativeElement.querySelector('.modal-title').textContent
).toContain('Set custom fields for 42 documents')
})
it('should emit succeeded event and close modal on successful save', () => {
const editSpy = jest
.spyOn(documentService, 'bulkEdit')
@@ -81,8 +81,14 @@ export class CustomFieldsBulkEditDialogComponent {
public selection: DocumentSelectionQuery = { documents: [] }
public selectionCount: number
public get documents(): number[] {
return this.selection.documents
return this.selection.documents ?? []
}
public get documentCount(): number {
return this.selectionCount ?? this.documents.length
}
initForm() {
@@ -2,7 +2,11 @@
<div class="row g-0">
<div class="col-md-2 doc-img-container rounded-start" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit()">
@if (document()) {
<img [ngSrc]="getThumbUrl()" fill class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()">
@if (priority()) {
<img [ngSrc]="getThumbUrl()" fill priority class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()" alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
} @else {
<img [ngSrc]="getThumbUrl()" fill class="card-img doc-img border-end rounded-start" [class.inverted]="getIsThumbInverted()" alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
}
<div class="border-end border-bottom bg-light document-card-check">
<div class="form-check">
@@ -94,6 +94,15 @@ describe('DocumentCardLargeComponent', () => {
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should prioritize the thumbnail when requested', () => {
fixture.componentRef.setInput('priority', true)
fixture.detectChanges()
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('eager')
expect(thumbnail.getAttribute('fetchpriority')).toEqual('high')
})
it('should trim content', () => {
expect(component.contentTrimmed).toHaveLength(503) // includes ...
})
@@ -66,6 +66,7 @@ export class DocumentCardLargeComponent
private documentService = inject(DocumentService)
settingsService = inject(SettingsService)
readonly selected = input(false)
readonly priority = input(false)
readonly displayFields = input<string[]>(
DEFAULT_DISPLAY_FIELDS.map((f) => f.id)
)
@@ -2,7 +2,11 @@
<div class="card h-100 shadow-sm document-card" [class.placeholder-glow]="!document()" [class.card-selected]="selected()" (mouseleave)="mouseLeaveCard()">
<div class="border-bottom doc-img-container rounded-top" (click)="this.toggleSelected.emit($event)" (dblclick)="dblClickDocument.emit(this)">
@if (document()) {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill>
@if (priority()) {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill priority alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
} @else {
<img class="card-img doc-img" [class.inverted]="getIsThumbInverted()" [ngSrc]="getThumbUrl()" fill alt="{{ document().title | documentTitle }} thumbnail" i18n-alt>
}
<div class="border-end border-bottom bg-light py-1 px-2 document-card-check">
<div class="form-check">
@@ -67,6 +67,15 @@ describe('DocumentCardSmallComponent', () => {
expect(thumbnail.getAttribute('loading')).toEqual('lazy')
})
it('should prioritize the thumbnail when requested', () => {
fixture.componentRef.setInput('priority', true)
fixture.detectChanges()
const thumbnail: HTMLImageElement =
fixture.nativeElement.querySelector('img.doc-img')
expect(thumbnail.getAttribute('loading')).toEqual('eager')
expect(thumbnail.getAttribute('fetchpriority')).toEqual('high')
})
it('should display a document, limit tags to 5', () => {
expect(fixture.nativeElement.textContent).toContain('Document 10')
expect(
@@ -66,6 +66,7 @@ export class DocumentCardSmallComponent
private documentService = inject(DocumentService)
settingsService = inject(SettingsService)
readonly selected = input(false)
readonly priority = input(false)
readonly document = input<Document>(undefined)
readonly displayFields = input<string[]>(
DEFAULT_DISPLAY_FIELDS.map((f) => f.id)
@@ -164,9 +164,10 @@
} @else {
@if (list.displayMode === DisplayMode.LARGE_CARDS) {
<div>
@for (d of list.documents; track d.id) {
@for (d of list.documents; track d.id; let i = $index) {
<pngx-document-card-large
[selected]="list.isSelected(d)"
[priority]="i < 2"
(toggleSelected)="toggleSelected(d, $event)"
(dblClickDocument)="openDocumentDetail(d)"
[document]="d"
@@ -398,9 +399,10 @@
}
@if (list.displayMode === DisplayMode.SMALL_CARDS) {
<div class="row row-cols-paperless-cards">
@for (d of list.documents; track d.id) {
@for (d of list.documents; track d.id; let i = $index) {
<pngx-document-card-small class="p-0"
[selected]="list.isSelected(d)"
[priority]="i < 6"
(toggleSelected)="toggleSelected(d, $event)"
(dblClickDocument)="openDocumentDetail(d)"
[document]="d"
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+2
View File
@@ -1024,6 +1024,8 @@ class ObjectOwnedOrGrantedPermissionsFilter(ObjectPermissionsFilter):
"""
def filter_queryset(self, request, queryset, view):
if request.user.is_superuser:
return queryset
objects_with_perms = super().filter_queryset(request, queryset, view)
objects_owned = queryset.filter(owner=request.user)
objects_unowned = queryset.filter(owner__isnull=True)
+88 -35
View File
@@ -11,6 +11,7 @@ from enum import StrEnum
from itertools import islice
from typing import TYPE_CHECKING
from typing import Final
from typing import NamedTuple
from typing import Self
from typing import TypedDict
from typing import TypeVar
@@ -21,6 +22,7 @@ import tantivy
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.utils.timezone import get_current_timezone
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
from documents.search._query import build_permission_filter
@@ -45,6 +47,8 @@ if TYPE_CHECKING:
from pathlib import Path
from django.contrib.auth.models import AbstractUser
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from django.db.models import QuerySet
from documents.models import Document
@@ -59,6 +63,18 @@ _LOCK_BACKOFF_CAP: Final[float] = 10.0 # seconds
T = TypeVar("T")
class ViewerGrant(NamedTuple):
"""Direct user and group view grants for a single document.
Named fields (rather than a bare 2-tuple) so ``viewer_ids`` and
``viewer_group_ids`` can't be silently transposed at a call site — both
are ``list[int]``, so a positional swap would type-check cleanly.
"""
viewer_ids: list[int]
viewer_group_ids: list[int]
class SearchMode(StrEnum):
QUERY = "query"
TEXT = "text"
@@ -367,7 +383,7 @@ class TantivyBackend:
) -> tantivy.Query:
"""Wrap a query with a permission filter if the user is not a superuser."""
if user is not None:
permission_filter = build_permission_filter(self._schema, user)
permission_filter = self._build_permission_filter(user)
return tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, query),
@@ -376,11 +392,21 @@ class TantivyBackend:
)
return query
def _build_permission_filter(self, user: AbstractUser) -> tantivy.Query:
"""Build a filter using the user's current group memberships."""
group_ids = user.groups.values_list("pk", flat=True)
return build_permission_filter(
self._schema,
user,
viewer_group_ids=group_ids,
)
def _build_tantivy_doc(
self,
document: Document,
effective_content: str | None = None,
viewer_ids: list[int] | None = None,
viewer_group_ids: list[int] | None = None,
) -> tantivy.Document:
"""Build a tantivy Document from a Django Document instance.
@@ -497,10 +523,26 @@ class TantivyBackend:
users_with_perms = get_users_with_perms(
document,
only_with_perms_in=["view_document"],
with_group_users=False,
)
viewer_ids = list(
cast("QuerySet[User]", users_with_perms).values_list("id", flat=True),
)
viewer_ids = [int(u.id) for u in users_with_perms]
for viewer_id in viewer_ids:
doc.add_unsigned("viewer_id", viewer_id)
if viewer_group_ids is None:
groups_with_perms = get_groups_with_perms(
document,
only_with_perms_in=["view_document"],
)
viewer_group_ids = list(
cast("QuerySet[Group]", groups_with_perms).values_list(
"id",
flat=True,
),
)
for viewer_group_id in viewer_group_ids:
doc.add_unsigned("viewer_group_id", viewer_group_id)
# Autocomplete words
text_sources = [document.title, content]
@@ -813,7 +855,7 @@ class TantivyBackend:
# Intersect with permission filter so autocomplete words from
# invisible documents don't leak to other users.
if user is not None and not user.is_superuser:
permission_query = build_permission_filter(self._schema, user)
permission_query = self._build_permission_filter(user)
matches = searcher.terms_with_prefix(
"autocomplete_word",
@@ -915,7 +957,7 @@ class TantivyBackend:
def rebuild(
self,
documents: QuerySet[Document],
iter_wrapper: IterWrapper[tuple[Document, list[int]]] = identity,
iter_wrapper: IterWrapper[tuple[Document, ViewerGrant]] = identity,
writer_heap_bytes: int = 512_000_000,
) -> None:
"""
@@ -928,8 +970,8 @@ class TantivyBackend:
documents: QuerySet of Document instances to index
iter_wrapper: Optional wrapper function for progress tracking
(e.g., progress bar). Wraps an iterable of
``(document, viewer_ids)`` pairs and should yield each pair
unchanged, advancing one step per document.
``(document, (viewer_ids, viewer_group_ids))`` pairs and should yield
each unchanged, advancing one step per document.
writer_heap_bytes: Tantivy writer memory budget (split across the
writer's threads). Larger values buffer more docs in RAM before
flushing a segment, deferring merge work; they do not avoid it.
@@ -953,11 +995,14 @@ class TantivyBackend:
documents_stream = _DocumentViewerStream(documents, chunk_size=1000)
try:
writer = new_index.writer(heap_size=writer_heap_bytes)
for document, viewer_ids in iter_wrapper(documents_stream):
for document, (viewer_ids, viewer_group_ids) in iter_wrapper(
documents_stream,
):
doc = self._build_tantivy_doc(
document,
document.get_effective_content(),
viewer_ids=viewer_ids,
viewer_group_ids=viewer_group_ids,
)
writer.add_document(doc)
writer.commit()
@@ -978,19 +1023,26 @@ def chunked(iterable, size):
yield chunk
class _DocumentViewerStream:
"""Yield (document, viewer_ids) pairs while batch-loading viewer ids.
_EMPTY_VIEWER_GRANT: Final[ViewerGrant] = ViewerGrant(
viewer_ids=[],
viewer_group_ids=[],
)
Viewer permissions are fetched one SQL query per chunk (see
``_bulk_get_viewer_ids``), but documents are yielded individually so a
class _DocumentViewerStream:
"""Yield document permission data while batch-loading grants.
Viewer permissions are fetched in batches (see
``_bulk_get_viewer_permissions``), but documents are yielded individually so a
progress bar wrapped around this stream advances per document rather than
jumping a whole chunk at a time. ``__len__`` lets the progress helper still
discover the total (it inspects ``QuerySet``/``Sized``).
The viewer ids travel with each document in the yielded pair rather than
through a separate mutable attribute, so the pairing survives regardless
of how ``iter_wrapper`` consumes the stream (buffering, batching, etc.) —
there is no reliance on the caller advancing this generator in lock-step.
The viewer and group ids travel with each document in the yielded pair
rather than through a separate mutable attribute, so the pairing survives
regardless of how ``iter_wrapper`` consumes the stream (buffering,
batching, etc.) — there is no reliance on the caller advancing this
generator in lock-step.
"""
def __init__(self, documents: QuerySet[Document], *, chunk_size: int) -> None:
@@ -1000,25 +1052,25 @@ class _DocumentViewerStream:
def __len__(self) -> int:
return self._documents.count()
def __iter__(self) -> Iterator[tuple[Document, list[int]]]:
def __iter__(self) -> Iterator[tuple[Document, ViewerGrant]]:
# iterator(chunk_size=…) streams from a server-side cursor instead of
# materialising the whole queryset in memory; since Django 4.1 it still
# honours prefetch_related, running the prefetches one batch at a time.
documents = self._documents.iterator(chunk_size=self._chunk_size)
for chunk in chunked(documents, self._chunk_size):
viewer_ids_by_pk = _bulk_get_viewer_ids([doc.pk for doc in chunk])
grants_by_pk = _bulk_get_viewer_permissions([doc.pk for doc in chunk])
for doc in chunk:
yield doc, viewer_ids_by_pk.get(doc.pk, [])
yield doc, grants_by_pk.get(doc.pk, _EMPTY_VIEWER_GRANT)
def _bulk_get_viewer_ids(doc_pks: Sequence[int]) -> dict[int, list[int]]:
"""Fetch all view_document permissions for a batch of documents in one query.
def _bulk_get_viewer_permissions(
doc_pks: Sequence[int],
) -> dict[int, ViewerGrant]:
"""Fetch direct user and group view grants for a batch of documents, keyed by pk.
Mirrors get_users_with_perms(doc, only_with_perms_in=["view_document"])
(with_group_users defaults to True there): a user counts as a viewer if
they hold the permission directly, OR via membership in a group that
holds the permission. Missing the group case would silently drop search
access for group-only viewers.
Group grants remain group IDs in the index so permission checks use the
requesting user's current memberships. Expanding groups to user IDs here
would leave stale access behind after a user is removed from a group.
"""
from collections import defaultdict
@@ -1032,6 +1084,7 @@ def _bulk_get_viewer_ids(doc_pks: Sequence[int]) -> dict[int, list[int]]:
str_pks = [str(pk) for pk in doc_pks]
viewer_map: dict[int, set[int]] = defaultdict(set)
viewer_group_map: dict[int, set[int]] = defaultdict(set)
# Fold the permission lookup into the query via a join on codename instead
# of a separate Permission.objects.get(), which would otherwise run once per
@@ -1045,22 +1098,22 @@ def _bulk_get_viewer_ids(doc_pks: Sequence[int]) -> dict[int, list[int]]:
for object_pk, user_id in user_qs:
viewer_map[int(object_pk)].add(user_id)
# User.groups has related_query_name="user", so group__user__id joins
# through the group membership m2m to the member users' ids in the same
# single-query fashion as user_qs above (values_list compiles to one SQL
# JOIN; no per-row Python-side lookups follow, so select_related /
# prefetch_related do not apply here).
group_qs = GroupObjectPermission.objects.filter(
content_type=ct,
permission__content_type=ct,
permission__codename="view_document",
object_pk__in=str_pks,
).values_list("object_pk", "group__user__id")
for object_pk, user_id in group_qs:
if user_id is not None:
viewer_map[int(object_pk)].add(user_id)
).values_list("object_pk", "group_id")
for object_pk, group_id in group_qs:
viewer_group_map[int(object_pk)].add(group_id)
return {object_pk: list(user_ids) for object_pk, user_ids in viewer_map.items()}
return {
object_pk: ViewerGrant(
viewer_ids=list(viewer_map.get(object_pk, ())),
viewer_group_ids=list(viewer_group_map.get(object_pk, ())),
)
for object_pk in viewer_map.keys() | viewer_group_map.keys()
}
# Module-level singleton with proper thread safety
+11 -1
View File
@@ -20,6 +20,7 @@ from documents.search._translate import SearchQueryError
from documents.search._translate import translate_query
if TYPE_CHECKING:
from collections.abc import Iterable
from datetime import tzinfo
from django.contrib.auth.base_user import AbstractBaseUser
@@ -114,6 +115,7 @@ def normalize_query(query: str) -> str:
def build_permission_filter(
schema: tantivy.Schema,
user: AbstractBaseUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
@@ -123,10 +125,12 @@ def build_permission_filter(
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
@@ -140,7 +144,13 @@ def build_permission_filter(
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
return tantivy.Query.disjunction_max_query([no_owner, owned, shared])
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
DEFAULT_SEARCH_FIELDS = [
+1
View File
@@ -102,6 +102,7 @@ def build_schema() -> tantivy.Schema:
"tag_id",
"owner_id",
"viewer_id",
"viewer_group_id",
):
sb.add_unsigned_field(field, stored=False, indexed=True, fast=True)
+92
View File
@@ -1419,6 +1419,30 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
@mock.patch("documents.views.bulk_edit.merge")
def test_merge_and_delete_requires_change_permission(self, m) -> None:
self.setup_mock(m, "merge")
user = User.objects.create_user(username="no-change")
user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="delete_document"),
)
self.client.force_authenticate(user=user)
response = self.client.post(
"/api/documents/merge/",
json.dumps(
{
"documents": [self.doc2.id, self.doc3.id],
"delete_originals": True,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.merge")
def test_merge_invalid_parameters(self, m) -> None:
self.setup_mock(m, "merge")
@@ -1668,6 +1692,74 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
@mock.patch("documents.views.bulk_edit.edit_pdf")
def test_edit_pdf_update_requires_change_permission(self, m) -> None:
self.setup_mock(m, "edit_pdf")
user = User.objects.create_user(username="no-change")
self.client.force_authenticate(user=user)
response = self.client.post(
"/api/documents/edit_pdf/",
json.dumps(
{
"documents": [self.doc2.id],
"operations": [{"page": 1}],
"update_document": True,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.remove_password")
@mock.patch("documents.views.bulk_edit.edit_pdf")
def test_delete_original_requires_delete_permission(
self,
edit_pdf_mock,
remove_password_mock,
) -> None:
self.setup_mock(edit_pdf_mock, "edit_pdf")
self.setup_mock(remove_password_mock, "remove_password")
user = User.objects.create_user(username="no-delete")
user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="change_document"),
)
self.client.force_authenticate(user=user)
cases = [
(
"/api/documents/edit_pdf/",
{
"documents": [self.doc2.id],
"operations": [{"page": 1}],
"delete_original": True,
},
edit_pdf_mock,
),
(
"/api/documents/remove_password/",
{
"documents": [self.doc2.id],
"password": "secret",
"delete_original": True,
},
remove_password_mock,
),
]
for endpoint, payload, operation_mock in cases:
with self.subTest(endpoint=endpoint):
response = self.client.post(
endpoint,
json.dumps(payload),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
operation_mock.assert_not_called()
@mock.patch("documents.views.bulk_edit.remove_password")
def test_remove_password(self, m) -> None:
self.setup_mock(m, "remove_password")
@@ -669,6 +669,26 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
def test_update_version_requires_global_change_permission(self) -> None:
user = User.objects.create_user(username="add-only")
user.user_permissions.add(Permission.objects.get(codename="add_document"))
root = Document.objects.create(
title="root",
checksum="root",
mime_type="application/pdf",
)
self.client.force_authenticate(user=user)
with mock.patch("documents.views.consume_file") as consume_mock:
resp = self.client.post(
f"/api/documents/{root.id}/update_version/",
{"document": self._make_pdf_upload()},
format="multipart",
)
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
consume_mock.apply_async.assert_not_called()
def test_update_version_returns_404_for_missing_document(self) -> None:
resp = self.client.post(
"/api/documents/9999/update_version/",
@@ -131,6 +131,10 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.client.get("/api/saved_views/").status_code,
status.HTTP_403_FORBIDDEN,
)
self.assertEqual(
self.client.get("/api/search/autocomplete/?term=test").status_code,
status.HTTP_403_FORBIDDEN,
)
def test_api_sufficient_permissions(self) -> None:
user = User.objects.create_user(username="test")
+25
View File
@@ -832,6 +832,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
"""
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1.user_permissions.add(Permission.objects.get(codename="view_document"))
self.client.force_authenticate(user=u1)
@@ -878,6 +879,30 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data, ["applebaum", "apples", "appletini"])
def test_search_autocomplete_group_revocation_is_immediate(self) -> None:
user = User.objects.create_user("group-user")
owner = User.objects.create_user("document-owner")
group = Group.objects.create(name="temporary-viewers")
user.user_permissions.add(Permission.objects.get(codename="view_document"))
user.groups.add(group)
document = Document.objects.create(
title="private",
content="canarysecretautocomplete",
checksum="group-revocation",
owner=owner,
)
assign_perm("view_document", group, document)
get_backend().add_or_update(document)
self.client.force_authenticate(user=user)
response = self.client.get("/api/search/autocomplete/?term=canarysecret")
self.assertEqual(response.data, ["canarysecretautocomplete"])
user.groups.remove(group)
response = self.client.get("/api/search/autocomplete/?term=canarysecret")
self.assertEqual(response.data, [])
def test_search_autocomplete_field_name_match(self) -> None:
"""
GIVEN:
+14 -6
View File
@@ -1947,10 +1947,13 @@ class DocumentViewSet(
"root_document",
).get(pk=pk)
root_doc = get_root_document(request_doc)
if request.user is not None and not has_perms_owner_aware(
request.user,
"change_document",
root_doc,
if request.user is not None and (
not request.user.has_perm("documents.change_document")
or not has_perms_owner_aware(
request.user,
"change_document",
root_doc,
)
):
return HttpResponseForbidden("Insufficient permissions")
except Document.DoesNotExist:
@@ -2756,7 +2759,7 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
)
or (method == bulk_edit.edit_pdf and parameters.get("update_document"))
):
has_perms = user_is_owner_of_all_documents
has_perms = has_perms and user_is_owner_of_all_documents
# check global add permissions for methods that create documents
if (
@@ -2781,6 +2784,11 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
method in [bulk_edit.merge, bulk_edit.split]
and parameters.get("delete_originals")
)
or (
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
and parameters.get("delete_original")
and not parameters.get("update_document")
)
)
and not user.has_perm("documents.delete_document")
):
@@ -3381,7 +3389,7 @@ class SelectionDataView(GenericAPIView[Any]):
),
)
class SearchAutoCompleteView(GenericAPIView[Any]):
permission_classes = (IsAuthenticated,)
permission_classes = (IsAuthenticated, ViewDocumentsPermissions)
def get(self, request, format=None):
user = self.request.user if hasattr(self.request, "user") else None
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More