Compare commits

..
Author SHA1 Message Date
stumpylog 74abdf8ff8 Docs: remove stale drf-writable-nested references, DRY up audit-actor update()
Final-review cleanup on refactor/remove-drf-writable-nested: several
comments/docstrings still described drf-writable-nested's removed
NestedUpdateMixin behavior in the present tense. Rewrote them to explain
the shared context-cache and delete-query-count guarantees in terms of
the current code (bulk-edit's per-field CustomFieldInstanceSerializer
construction, and _sync_custom_fields' single hard-delete query).

Also collapsed the duplicated `if custom_fields_data is not None:
self._sync_custom_fields(...)` line in DocumentSerializer.update()'s
audit-log branches into a single code path using
contextlib.nullcontext(), removing the drift risk that caused the
earlier audit-actor bug.

No behavior change.
2026-08-25 11:53:42 -07:00
stumpylogandClaude Sonnet 5 ce5d5c33b0 chore: remove drf-writable-nested dependency
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 11:07:11 -07:00
stumpylog d9d44520bd test: add query-count regression coverage for custom_fields sync on update 2026-08-25 10:59:47 -07:00
stumpylog 500ff563cf refactor: replace drf-writable-nested's NestedUpdateMixin with explicit custom_fields sync 2026-08-25 10:38:51 -07:00
stumpylog 7f314b5149 test: characterize custom field instance delete-on-omit as a hard delete 2026-08-25 10:28:24 -07:00
stumpylog 132918821d Merge remote-tracking branch 'origin/perf/batch-custom-field-lookup' into tmp/perf-integration 2026-08-25 10:15:52 -07:00
stumpylog 512a3fe196 Merge remote-tracking branch 'origin/perf/batch-modify-custom-fields' into tmp/perf-integration 2026-08-25 10:15:48 -07:00
stumpylog b01e0368b7 Merge remote-tracking branch 'origin/perf/batch-tags-field-lookup' into tmp/perf-integration 2026-08-25 10:15:44 -07:00
stumpylog 1c82bd15c5 Merge branch 'perf/batch-set-permissions' into tmp/perf-integration 2026-08-25 10:15:39 -07:00
stumpylog cbac71c165 Perf: avoid unnecessary full-row fetches in batch permission assignment
set_permissions_for_objects now takes a model + pks instead of instances,
and identity filtering resolves straight to ids, so bulk-editing
permissions no longer materializes full Document/User/Group rows just to
read their pk/id. Row construction for bulk_create is also chunked to
bound peak memory for very large "apply to all" operations.
2026-08-25 09:27:24 -07:00
stumpylog 8780bcd5c7 Perf: batch guardian permission assignment in bulk-edit
bulk_edit.set_permissions and BulkEditObjectPermissionsView both
looped documents/objects and called set_permissions_for_object per
object, which itself calls guardian's assign_perm/remove_perm once
per (object, user) pair -- ~10-20+ queries per object, scaling with
selection size.

Added set_permissions_for_objects, a bulk equivalent that resolves
existing permission holders once across the whole batch (not once per
object) and applies changes with a small, batch-size-independent
number of queries per action instead of one per (object, user) pair.

Deliberately does not use guardian's queryset-aware assign_perm:
passing a list as the target routes to bulk_assign_perm, which skips
creating a direct permission row for anyone who already has the
permission via ANY group membership (checked via
ObjectPermissionChecker.has_perm, which is group-inheritance-aware) --
unlike the single-object assign_perm this replaces, which always
ensures a direct row via get_or_create. Losing that guarantee would
mean a later revocation of the group's grant silently strips access
an admin explicitly asked to be direct. Bulk-creates rows straight
against UserObjectPermission/GroupObjectPermission instead
(ignore_conflicts=True, relying on the existing (identity, permission,
object_pk) unique constraint), which preserves the original semantics
exactly while still batching every object and identity into one query
per action. Also raises Permission.DoesNotExist for an unrecognized
action name instead of silently no-op-ing, matching the original
per-object path -- BulkEditObjectsSerializer never actually validates
action keys against the raw client-supplied permissions dict, so this
is reachable from client input, not just internal callers.

Verified via CaptureQueriesContext: query count is now identical at 5
vs. 50 documents/objects (was 1,123 queries for 20 documents on the
Document path, 2,806 for 50 tags on the BulkEditObjectPermissionsView
path, both now flat). Full documents test suite green (2,148 passed,
1 skipped).
2026-08-24 20:04:58 -07:00
GitHub Actions 492424f7f2 Auto translate strings 2026-08-25 00:07:03 +00:00
shamoonandGitHub cd525819d7 Tweak: more misc UI tweaks (#13783) 2026-08-24 17:05:29 -07:00
stumpylog 9d2416c435 Perf: batch id resolution for TagsField and friends
TagsField/CorrespondentField/DocumentTypeField/StoragePathField were
plain PrimaryKeyRelatedField subclasses with no batching. When used
with many=True (only tags today: DocumentSerializer.tags,
WorkflowActionSerializer.assign_tags), DRF's ManyRelatedField resolves
each submitted id with its own query -- one query per tag on every
PATCH/PUT that sets tags.

Added BatchResolvingPrimaryKeyRelatedField as the shared base for all
four field classes and overrode many_init so the many=True form
(_BatchingManyRelatedField) resolves the whole id list with one
pk__in query, falling back to the child relation's normal per-item
validation for anything not found in that batch. Only TagsField uses
many=True today, but the fix isn't tag-specific -- if a future PR puts
many=True on one of the others, it inherits the same batching instead
of reintroducing this as a new bug.

Independent review caught a real regression: Django's IntegerFieldOverflow
guard (out-of-range int -> EmptyResultSet) only covers exact/gt/gte/lt/lte
lookups, not `in`, so an absurdly large tag id reached the batched
pk__in= query as-is and raised an unhandled OverflowError (SQLite) /
DataError (Postgres) instead of the normal 400 the original per-item
`exact` lookup produced. Guarded the batch query and fall through to
per-item resolution (which goes through the protected `exact` lookup)
on failure.

Verified via CaptureQueriesContext against a real API PATCH: 20 tags
dropped from 54 to 35 queries per request (exactly the 19 saved by
collapsing 20 individual lookups into one batched query). Full
documents/workflows/bulk-edit/retagger/custom-fields suites green
(443 passed).
2026-08-24 15:15:00 -07:00
GitHub Actions 2609327e9c Auto translate strings 2026-08-24 21:44:25 +00:00
shamoonandGitHub b90ccf910f Finally, the remote ocr workflow (#13637)
* Ok! Backend stuff for the remote ocr workflow

* Frotnend workflow stuff

* And docs

* Fix dynamic action fields thing

* Actually, fix the action dropdown thing

* Fix this validation thing, and we have to check existing actions

* Fix migration
2026-08-24 14:43:05 -07:00
shamoonandGitHub c93c996edf Remote ocr reprocess (#13636)
* Backend stuff for remote ocr reprocess, add to bulk edit pass in from ui settings

* Ok, frontend reprocess remote option

* Docs
2026-08-24 14:43:05 -07:00
shamoonandGitHub 7f1609332a Allow parsers to declare uses remote, and remote ocr_mode (#13634)
* uses_remote_service + allow_remote to allow opt-in / out of remote OCR

* Add to parser dev docs

* remote_ocr_mode config setting

* Checks for remote_ocr_mode and fix import

* Update config.component.spec.ts

* More tests for remote_ocr_mode

* Docs for remote_ocr_mode

* Ok, wire up the remote_ocr_mode with allow_remote for consumer

* Update consumer.py

* Format remote OCR mode check tests

* Use get_choice_from_env
2026-08-24 14:43:04 -07:00
stumpylog a98d0669e4 Perf: batch CustomField/Document lookups in modify_custom_fields
modify_custom_fields looped documents x fields, re-.get()-ing the
CustomField queryset per iteration and Document.objects.get() per doc
for DOCUMENTLINK fields -- same shape as the earlier custom_fields
serializer N+1 (#13779), just nested one level deeper. Resolve both
into dicts once up front instead. Also pass the resolved objects
(not bare ids) to update_or_create so newly-created CustomFieldInstance
rows cache their field/document FK, avoiding a re-fetch when auditlog's
post_save receiver calls str(instance) (which touches .field.name).

docs_by_id defers `content` (the one field guaranteed both large and
unused by this function or its receivers) rather than using .only(),
since .only() would just turn the filename-generation signal's other
field access into a deferred-reload N+1.

Verified via CaptureQueriesContext: 6 docs x 4 fields dropped from 48
CustomField queries to 1; DOCUMENTLINK per-doc Document lookups dropped
from N to 0 (single batched query instead).
2026-08-24 14:21:59 -07:00
GitHub Actions a1f20c9fe7 Auto translate strings 2026-08-24 21:19:19 +00:00
shamoonandGitHub 4fd1c60731 Enhancement: support using remote OCR engines selectively (#13633)
* Backend changes and migration for remote OCR Config

* Backend tests

* Frontend stuff, with sections

* Docs

* Update test_tesseract_parser.py

* Actually we cant use this any more, in case settings are in app config

* Dont mark entire test file for db, use a mock for empty engine settings
2026-08-24 14:17:52 -07:00
stumpylog bda506968b Handles a bad client sending malformed JSON or non-int primary keys 2026-08-24 12:35:06 -07:00
GitHub Actions ba83e5b39a Auto translate strings 2026-08-24 18:28:51 +00:00
shamoonandGitHub 1497bd33a1 Fix: fix bottom mobile nav buttons on Android (#13780) 2026-08-24 11:27:26 -07:00
shamoon 01c40aa0bd Tweak: subtler shadow, tweak against light color 2026-08-24 08:51:51 -07:00
shamoon 5f70b6eee7 Fix navbar button alignment + hover consistency 2026-08-24 08:47:27 -07:00
GitHub Actions e3e4b26944 Auto translate strings 2026-08-24 15:16:51 +00:00
a0908f6b4a Enhancement: websocket heartbeat (#13739)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-08-24 15:15:21 +00:00
bab9129ff8 Fix: lazy import guardian modules to fix search language setting (#13768)
Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2026-08-24 13:46:20 +00:00
GitHub Actions 1e13f86174 Auto translate strings 2026-08-24 09:13:06 +00:00
shamoonandGitHub 22c57fb392 Enhancement: more v3 ui tweaks (#13774) 2026-08-24 02:11:34 -07:00
GitHub Actions 08a7f6ccc0 Auto translate strings 2026-08-24 04:56:30 +00:00
shamoonandGitHub 3f4d327f64 Chore: add some missing UI accessibility labels (#13772) 2026-08-23 21:55:07 -07:00
GitHub Actions bd326540fa Auto translate strings 2026-08-24 02:43:34 +00:00
shamoonandGitHub 33f9adb05a Chore: refactor permission checkbox live changes (#13771) 2026-08-23 19:42:18 -07:00
Trenton Holmes 4ccb34a70b Perf: avoid per-instance CustomField reload in DocumentMetadataOverrides
send_websocket_document_updated calls document.refresh_from_db()
before building overrides, which drops the custom_fields prefetch
(and its select_related("field")) set up by the view's queryset.
DocumentMetadataOverrides.from_document() then lazily reloads field
once per custom field instance. Since from_document() can't rely on
the caller having a prefetched document, select_related explicitly at
the point of use instead.
2026-08-23 17:33:05 -07:00
Trenton Holmes 0a466c9fcf Perf: reuse resolved CustomField objects across drf-writable-nested's per-item revalidation
drf-writable-nested's update_or_create_reverse_relations rebuilds a
fresh serializer -- and fresh field instances -- per custom_fields item
while matching existing vs. new instances during save(), so the
per-instance lookup cache alone only helped the first validation pass.
It passes the same context dict (by reference) to every one of those
serializers, so stash resolved CustomField objects there instead:
later passes reuse them for free rather than re-querying.
2026-08-23 17:12:25 -07:00
shamoonandGitHub d24606a03b Tweak: better support long list of views in documents list (#13769) 2026-08-23 16:40:55 -07:00
shamoonandGitHub 294328f174 Fix: version indexing fixes (#13737) 2026-08-23 23:04:47 +00:00
Trenton Holmes c9cc4f427d Perf: batch CustomField lookups when validating a document's custom_fields
DocumentSerializer.custom_fields validates each item's field id via a
plain PrimaryKeyRelatedField, which issues one SELECT per custom field
per validation pass (discussion #13690). Batch-resolve all field ids in
one query and cache them on the field instance so per-item validation
is free instead of re-querying.
2026-08-23 14:53:46 -07:00
shamoonandGitHub 0458bad5f2 Fix: append charset to file response for text files (#13759) 2026-08-22 06:15:53 -07:00
shamoon a510d03c77 Merge branch 'main' into dev 2026-08-22 05:27:10 -07:00
shamoonandGitHub d7b3612a41 Chore: pin Apache Tika images to 3.3.1 (#13758) 2026-08-22 05:25:37 -07:00
shamoonandGitHub 7e4a644714 Fix: align bulk edit perms with document model (#13757) 2026-08-22 05:24:16 -07:00
GitHub Actions bbcd6af2fe Auto translate strings 2026-08-21 15:25:24 +00:00
shamoonandGitHub 0431939f18 QoL: add count badge to versions dropdown (#13753) 2026-08-21 08:23:47 -07:00
shamoonandGitHub bed95ea301 Tweakhancement: add jitter to IMAP polling schedule (#13734) 2026-08-20 17:19:47 +00:00
GitHub Actions 42034c3c77 Auto translate strings 2026-08-20 04:52:42 +00:00
shamoonandGitHub 705220fb5a Fix: use selected version for doc detail emailing (#13738) 2026-08-19 21:51:13 -07:00
shamoonandGitHub 2d084983c8 Documentation: clarify OCR mode changes in v3 (#13666) 2026-08-12 21:46:52 -07:00
shamoonandGitHub e2c284f64e Documentation: add wiki links for AI stuff and parser plugins (#13626) 2026-08-09 19:05:32 -07:00
178 changed files with 7432 additions and 6530 deletions
@@ -68,7 +68,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
-3
View File
@@ -115,6 +115,3 @@ celerybeat-schedule*
# Git worktree local folder
.worktrees
# Agent workflow scratch (ledgers, briefs, review packages)
.superpowers/
@@ -81,7 +81,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
@@ -76,7 +76,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
@@ -65,7 +65,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
+1 -2
View File
@@ -521,8 +521,7 @@ Pass `--recreate` to wipe the existing index before rebuilding. Use this when th
index is corrupted or you want a fully clean rebuild.
Pass `--if-needed` to skip the rebuild if the index is already up to date (schema
version, schema fingerprint and search language all match). Safe to run on every
startup or upgrade.
version and search language match). Safe to run on every startup or upgrade.
Specify `optimize` to optimize the index. This command is regularly invoked by the
task scheduler.
+6 -1
View File
@@ -129,6 +129,10 @@ At a minimum you need to enable AI and choose an LLM backend:
and/or [`PAPERLESS_AI_LLM_ENDPOINT`](configuration.md#PAPERLESS_AI_LLM_ENDPOINT). Ollama
requires `PAPERLESS_AI_LLM_ENDPOINT` pointing at your Ollama server.
See the community-maintained wiki page on
[choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for suggested generation and embedding models.
### AI-assisted suggestions
With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type,
@@ -808,7 +812,8 @@ Third-party parser plugins extend Paperless-ngx to support additional file
formats. A plugin is a Python package that advertises itself under the
`paperless_ngx.parsers` entry point group. Refer to the
[developer documentation](development.md#making-custom-parsers) for how to
create one.
create one, or see the wiki for a community-maintained list of
[parser plugins](https://github.com/paperless-ngx/paperless-ngx/wiki/Related-Projects#parser-plugins).
!!! warning "Third-party plugins are not officially supported"
+2 -1
View File
@@ -302,7 +302,8 @@ The following methods are supported:
- `delete`
- No `parameters` required
- `reprocess`
- No `parameters` required
- Optional `parameters`: `{ "remote_ocr": true }` to send the documents to the
remote OCR engine, see [Remote OCR](usage.md#remote-ocr). Defaults to false.
- `set_permissions`
- Requires `parameters`:
- `"set_permissions": PERMISSIONS_OBJ` (see format [above](#permissions)) and / or
+17 -1
View File
@@ -1215,7 +1215,7 @@ should be a valid crontab(5) expression describing when to run.
: If set to the string "disable", no emails will be fetched automatically.
Defaults to `*/10 * * * *` or every ten minutes.
Defaults to every ten minutes, with an installation-specific minute offset.
#### [`PAPERLESS_TRAIN_TASK_CRON=<cron expression>`](#PAPERLESS_TRAIN_TASK_CRON) {#PAPERLESS_TRAIN_TASK_CRON}
@@ -2066,6 +2066,18 @@ password. All of these options come from their similarly-named [Django settings]
Defaults to None.
#### [`PAPERLESS_REMOTE_OCR_MODE=<str>`](#PAPERLESS_REMOTE_OCR_MODE) {#PAPERLESS_REMOTE_OCR_MODE}
: Which documents are sent to the remote OCR engine.
- `always`: every document of a supported file type is sent to the remote
engine, bypassing the local OCR engine.
- `workflow_only`: documents are processed locally unless a workflow
explicitly enables remote OCR for them, letting you use the remote engine
selectively.
Defaults to "always".
## AI {#ai}
#### [`PAPERLESS_AI_ENABLED=<bool>`](#PAPERLESS_AI_ENABLED) {#PAPERLESS_AI_ENABLED}
@@ -2088,6 +2100,8 @@ suggestions. This setting is required to be set to true in order to use the AI f
models supported by the current embedding backend. If not supplied, defaults to
"text-embedding-3-small" for the OpenAI-compatible backend,
"sentence-transformers/all-MiniLM-L6-v2" for Huggingface, and "embeddinggemma" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for language and resource considerations.
Defaults to None.
@@ -2144,6 +2158,8 @@ setting is required to be set to use the AI features.
: The model to use for the AI backend, i.e. "gpt-3.5-turbo", "gpt-4" or any of the models supported
by the current backend. If not supplied, defaults to "gpt-3.5-turbo" for the OpenAI-compatible
backend and "llama3.1" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for local versus remote and model-size considerations.
Defaults to None.
+14
View File
@@ -456,6 +456,20 @@ def score(
return 10
```
**Remote services**
If your parser sends document content to a remote service, declare it:
```python
class MyCustomParser:
uses_remote_service = True
```
Paperless-ngx excludes such parsers when the document being consumed has not
been marked for remote processing, so users can keep remote OCR off by default
and enable it selectively with a workflow. Parsers that do not declare the
attribute are treated as fully local and are always considered.
**Archive and rendition flags**
```python
+4 -3
View File
@@ -156,7 +156,7 @@ The new settings are independent:
### Database configuration
If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and `skip_archive_file` values are converted to the new `archive_file_generation` field. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent.
If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and explicit `skip_archive_file` values are converted to the new `archive_file_generation` field. Users who relied on the old defaults must set `archive_file_generation` to `always` to preserve the v2 behaviour of always creating an archive. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent.
### Action Required
@@ -165,8 +165,9 @@ Remove any `PAPERLESS_OCR_SKIP_ARCHIVE_FILE` variable from your environment. If
```bash
# v2: skip OCR when text present, always archive
PAPERLESS_OCR_MODE=skip
# v3: equivalent (auto is the new default)
# No change needed - auto is the default
# v3: equivalent
PAPERLESS_OCR_MODE=auto
PAPERLESS_ARCHIVE_FILE_GENERATION=always
# v2: skip OCR when text present, skip archive too
PAPERLESS_OCR_MODE=skip_noarchive
+26 -91
View File
@@ -654,6 +654,19 @@ happened while it was still encrypted, that original version will likewise be mi
**Current limitation**: Passwords are stored as a simple list without descriptions. To handle
multiple PDF types with different passwords, create separate workflows for each use case.
##### Remote OCR {#workflow-action-remote-ocr}
"Remote OCR" actions send the document to the configured remote OCR engine instead of processing it
locally. To use remote OCR selectively, set the [remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE)
to `workflow_only` then add this action to a workflow that matches only the documents you
want sent to the remote engine. See [Remote OCR](#remote-ocr) for the engine setup. The action only works with
a **Consumption Started** trigger.
The action takes no options, its presence is what enables remote OCR for a matching document.
If the remote engine is not configured, or does not support the document's file type, the document is
processed locally instead and a warning is written to the log.
#### Workflow placeholders
Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
@@ -886,19 +899,6 @@ Matching documents with logical expressions:
```
shopname AND (product1 OR product2)
invoice NOT draft
```
`AND`, `OR` and `NOT` must be written in capitals, and parentheses group sub-expressions. Terms written next to each other with no operator between them are combined with `AND`.
!!! warning
A leading `-` does **not** exclude a term. Separators are stripped during indexing, so `invoice -secret` searches for `invoice` and `secret`, which is the opposite of what you probably intended. Use `NOT` to exclude a term: `invoice NOT secret`.
Matching an exact phrase, in order, by quoting it:
```
"quick brown fox"
```
Matching specific tags, correspondents or types:
@@ -906,12 +906,8 @@ Matching specific tags, correspondents or types:
```
type:invoice tag:unpaid
correspondent:university certificate
tag:bills,unpaid
```
- `document_type` may be abbreviated to `type`, and `storage_path` to `path`.
- A comma-separated list after `tag:` requires **all** of the listed tags, so `tag:bills,unpaid` matches only documents tagged both `bills` and `unpaid`.
Matching dates:
```
@@ -920,58 +916,14 @@ added:yesterday
modified:today
```
Matching by archive metadata:
```
asn:100
page_count:12
num_notes:0
checksum:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
original_filename:invoice.pdf
```
- `asn` matches a document's Archive Serial Number.
- `page_count` matches a document's page count.
- `num_notes` matches how many notes a document has.
- `checksum` matches the checksum of the original document file (not the archived/processed version). Unlike the text fields, this one is stored verbatim rather than tokenized, so only a complete, lowercase checksum matches. To search by the first few characters instead, use a wildcard: `checksum:9f86d081*`. Wildcard patterns on the text fields are also tried stemmed, to line up with the stemmed index, but `checksum` is indexed without stemming, so its patterns are not stemmed either: a wildcard prefix is matched literally, apart from being lowercased first. `checksum:9F86D081*` therefore does find the document, even though the plain uppercase term does not.
- `original_filename` matches the filename of the document as originally consumed.
`asn`, `page_count` and `num_notes` are numeric and also accept ranges, for example `asn:[50 to 150]`.
Matching inexact words:
```
invoice*
title:Invoice*
produ*name
```
Wildcards are matched against the _stemmed_ terms stored in the index, not
against the words as they appear in the document. Each literal part of a
pattern is tried both as you typed it and in its stemmed form, so a trailing
`*` matches a word and its inflections (`invoice*` finds "invoice", "invoices"
and "invoiced") as well as longer words whose stored term still begins with
what you typed (`copy*` finds "copyright" alongside "copy" and "copies").
It is still not a plain prefix search over the original text. A trailing `*`
matches a stored term when either the run you typed or its stemmed form is a
prefix of that term, so a fragment that stops part-way between the two matches
neither: `universities*` finds "university" and "universities", which are both
stored as `univers`, while the shorter `universit*` finds nothing at all. For
the same reason `happine*` does not find "happiness", which is stored as
`happi`. And a pattern that requires letters after the wildcard which stemming
has removed cannot match either: `productname` is stored as `productnam`, so
`produ*name` finds nothing.
Matching natural date keywords:
The multi-word date keywords listed below work quoted or unquoted after a
date field (`added:"previous month"` and `added:previous month` are
equivalent); elsewhere in a query the same words are treated as ordinary
search text. Other date expressions the parser accepts (relative offsets
like `-1 week`, or specific dates like `12 december 2019`) must be quoted when
they stand alone as a value; inside a range's brackets they work unquoted, as
in `added:[-1 week to now]`.
```
added:today
modified:yesterday
@@ -984,30 +936,6 @@ Supported date keywords: `today`, `yesterday`, `previous week`,
`this month`, `previous month`, `this year`, `previous year`,
`previous quarter`.
These other date forms also work after a date field:
```
added:tomorrow
created:2005-03-04
added:january
modified:"next monday"
added:"last monday"
added:"2005-01-01T00:00:00Z"
created:[2005-01-01 to 2005-01-31]
added:[2005-06-15T09:00:00Z to 2005-06-15T17:00:00Z]
```
- `tomorrow`, like `today` and `yesterday`, covers that whole day.
- An ISO date such as `2005-03-04` covers that whole day, and `2005-01` covers that whole month.
- A month name such as `january` covers that whole month in the current year.
- `next <weekday>` and `last <weekday>` each cover that whole day and must be quoted. A bare weekday name such as `monday` is not accepted.
- A full timestamp such as `2005-01-01T00:00:00Z` matches that exact instant. Like the other expressions above, it has to be quoted when it stands on its own: `added:"2005-01-01T00:00:00Z"`. The unquoted spelling is rejected with an error rather than searched, because only part of it can be read as a date.
- A range takes two of the above as its bounds, for example `created:[2005 to 2009]` or `added:[2005-01-01 to 2005-01-31]`. Bounds may carry a time of day. A bound is normally written without quotes; if you do quote one, use single quotes (`added:['-1 week' to now]`), because a double-quoted bound is rejected with an error.
!!! warning
As a value on its own, `now`, `noon`, `midnight` and relative offsets such as `"-3 days"` or `"-1 week"` are accepted by the parser but resolve to a single instant rather than to a span of time, so they match only a document whose timestamp is exactly that instant, which in practice means no documents at all. Quoting does not change this. As a *range bound* they are the opposite of a trap and are what you want: `added:['-1 week' to now]` covers the whole of the last seven days. Spellings like `now-3days` and `"3 days ago"` are rejected outright wherever they appear.
#### Searching custom fields
Custom field names and values are included in the full-text index, but they
@@ -1023,7 +951,6 @@ custom_fields.name:Insurance custom_fields.value:policy
- `custom_fields.value` matches against the value of any custom field.
- `custom_fields.name` matches the name of the field (use quotes for multi-word names).
- Combine both to find documents where a specific named field contains a specific value.
- The bare `custom_fields:` prefix is shorthand for `custom_fields.value:`.
Because separators are stripped during indexing, individual parts of formatted
codes are searchable on their own. A value stored as `A-1312/99.50` produces the
@@ -1051,9 +978,9 @@ notes.note:reminder
notes.user:alice notes.note:insurance
```
The bare `notes:` prefix is shorthand for `notes.note:`.
All of these constructs can be combined as you see fit. What is described above is the whole of the query language paperless supports. It resembles other search query languages without being identical to any of them, so a construct that is not documented here is most likely treated as ordinary search text rather than as syntax, and an unrecognized field name is searched as text too.
All of these constructs can be combined as you see fit. If you want to
learn more about the query language used by paperless, see the
[Tantivy query language documentation](https://docs.rs/tantivy/latest/tantivy/query/struct.QueryParser.html).
!!! note
@@ -1176,11 +1103,19 @@ Paperless-ngx supports performing OCR on documents using remote services. At the
[Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence).
This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with
Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details.
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details. These
settings can be supplied as environment variables or via **Application Configuration**.
Additionally, when using a commercial service with this feature, consider both potential costs as well as any associated file size
or page limitations (e.g. with a free tier).
By default, every document of a supported file type is sent to the remote engine. To use it more selectively, set the
[remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE) to `workflow_only`. Documents are then processed locally
unless a [remote OCR workflow action](#workflow-action-remote-ocr) enables it for them, so you can limit the remote
engine to particular documents.
Setting the mode to `workflow_only` also allows the **Reprocess** actions to selectively use remote OCR for individual documents.
## Architecture
Paperless-ngx consists of the following components:
-5
View File
@@ -40,7 +40,6 @@ dependencies = [
"djangorestframework~=3.16",
"drf-spectacular~=0.30",
"drf-spectacular-sidecar~=2026.7.1",
"drf-writable-nested~=0.7.1",
"filelock~=3.32.0",
"flower~=2.0.1",
"gotenberg-client~=0.14.0",
@@ -77,7 +76,6 @@ dependencies = [
"torch~=2.13.0",
"watchfiles>=1.2",
"whitenoise~=6.11",
"whoosh-compat[tantivy]",
"zxing-cpp~=3.1.0",
]
[project.optional-dependencies]
@@ -169,9 +167,6 @@ psycopg-c = [
torch = [
{ index = "pytorch-cpu" },
]
# TODO: switch to a pinned PyPI version once whoosh-compat releases; fall back
# to a pinned git commit SHA if that release slips.
whoosh-compat = { path = "../whoosh-compat" }
[tool.ruff]
target-version = "py311"
@@ -33,7 +33,7 @@ test('should warn on unsaved changes', async ({ page }) => {
await page.getByRole('button', { name: 'Close', exact: true }).click()
await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/)
await page.getByRole('button', { name: 'Cancel' }).click()
await page.getByRole('link', { name: 'Close all' }).click()
await page.getByRole('button', { name: 'Close all' }).click()
await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/)
})
+726 -335
View File
File diff suppressed because it is too large Load Diff
@@ -14,43 +14,48 @@
<a ngbNavLink>{{category}}</a>
<ng-template ngbNavContent>
<div class="p-3">
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
@for (option of getCategoryOptions(category); track option.key) {
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
@for (section of getCategorySections(category); track section) {
@if (section) {
<h5 class="mt-4 mb-3">{{section}}</h5>
}
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
@for (option of getCategoryOptions(category, section); track option.key) {
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div>
</div>
</div>
}
</div>
}
</div>
}
</div>
</ng-template>
</li>
@@ -8,7 +8,11 @@ import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectModule } from '@ng-select/ng-select'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { OutputTypeConfig } from 'src/app/data/paperless-config'
import {
ConfigCategory,
ConfigSection,
OutputTypeConfig,
} from 'src/app/data/paperless-config'
import { ConfigService } from 'src/app/services/config.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
@@ -158,4 +162,24 @@ describe('ConfigComponent', () => {
component.resetOption('barcodes_enabled')
expect(component.configForm.get('barcodes_enabled').value).toBeNull()
})
it('should group options into sections within a category, or not', () => {
const sections = component.getCategorySections(ConfigCategory.OCR)
expect(sections).toEqual([null, ConfigSection.RemoteOCR])
expect(
component
.getCategoryOptions(ConfigCategory.OCR)
.map((option) => option.key)
).toContain('output_type')
expect(
component
.getCategoryOptions(ConfigCategory.OCR, ConfigSection.RemoteOCR)
.map((option) => option.key)
).toEqual([
'remote_ocr_engine',
'remote_ocr_api_key',
'remote_ocr_endpoint',
'remote_ocr_mode',
])
})
})
@@ -74,8 +74,20 @@ export class ConfigComponent
return Object.values(ConfigCategory)
}
getCategoryOptions(category: string): ConfigOption[] {
return PaperlessConfigOptions.filter((o) => o.category === category)
getCategorySections(category: string): string[] {
return [
...new Set(
PaperlessConfigOptions.filter((o) => o.category === category).map(
(o) => o.section ?? null // null means no section
)
),
]
}
getCategoryOptions(category: string, section: string = null): ConfigOption[] {
return PaperlessConfigOptions.filter(
(o) => o.category === category && (o.section ?? null) === section
)
}
initialConfig: PaperlessConfig
@@ -89,7 +89,7 @@
</div>
</div>
@if (filterText?.length) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetFilter()">
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetFilter()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs>
</button>
}
@@ -208,7 +208,7 @@
</td>
}
<td class="d-lg-none">
<button class="btn btn-link" (click)="expandTask(task); $event.stopPropagation();">
<button class="btn btn-link" (click)="expandTask(task); $event.stopPropagation();" aria-label="View task details" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
</button>
</td>
@@ -18,9 +18,11 @@
</button>
</pngx-page-header>
<div class="row mb-3">
<ngb-pagination class="col-auto" [pageSize]="25" [collectionSize]="totalDocuments()" [page]="page()" [maxSize]="5" (pageChange)="page.set($event); reload()" size="sm" aria-label="Pagination"></ngb-pagination>
</div>
@if (totalDocuments() > 25) {
<div class="row mb-3">
<ngb-pagination class="col-auto" [pageSize]="25" [collectionSize]="totalDocuments()" [page]="page()" [maxSize]="5" (pageChange)="page.set($event); reload()" size="sm" aria-label="Pagination"></ngb-pagination>
</div>
}
<div class="card border table-responsive mb-3">
<table class="table table-striped align-middle shadow-sm mb-0">
@@ -64,7 +66,7 @@
<td scope="row">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -2,29 +2,36 @@
<button class="navbar-toggler d-md-none collapsed border-0" type="button" data-toggle="collapse"
data-target="#sidebarMenu" aria-controls="sidebarMenu" aria-expanded="false" aria-label="Toggle navigation"
(click)="closeMobileSearch(); toggleMenuCollapsed()">
<span class="navbar-toggler-icon"></span>
<i-bs width="1.5em" height="1.5em" name="list"></i-bs>
</button>
<a class="navbar-brand d-flex align-items-center me-0 px-3 py-3 order-sm-0"
[ngClass]="{ 'slim': slimSidebarEnabled, 'col-auto col-md-3 col-lg-2 col-xxxl-1' : !slimSidebarEnabled, 'py-3' : !customAppTitle?.length || slimSidebarEnabled, 'py-2': customAppTitle?.length }"
<a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0 order-sm-0"
[ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
routerLink="/dashboard"
tourAnchor="tour.intro">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
<div class="ms-2 ms-md-3 d-inline-block" [class.d-md-none]="slimSidebarEnabled">
@if (customAppTitle?.length) {
<div class="d-flex flex-column align-items-start custom-title">
<span class="title">{{customAppTitle}}</span>
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
</div>
@if (!hasCustomBranding) {
<pngx-logo extra_classes="navbar-official-logo px-1" height="2.4rem"></pngx-logo>
<svg class="brand-mark brand-mark-slim d-none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
} @else {
@if (customAppLogo) {
<img class="brand-logo" [src]="customAppLogo" alt="" />
} @else {
Paperless-ngx
<svg class="brand-mark" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
}
</div>
<div class="brand-copy ms-2 text-truncate" [class.d-md-none]="slimSidebarEnabled">
<span class="brand-title text-truncate">{{ appTitle }}</span>
@if (customAppTitle) {
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
}
</div>
}
</a>
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 px-3 ps-md-4 me-sm-auto order-3 order-sm-1"
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 me-sm-auto order-3 order-sm-1"
[class.mobile-hidden]="mobileSearchHidden()">
<div class="col-12 col-md-7">
<div class="col-12 header-search mx-auto">
<pngx-global-search></pngx-global-search>
</div>
</div>
@@ -34,7 +41,7 @@
}
<pngx-toasts-dropdown></pngx-toasts-dropdown>
<li ngbDropdown class="nav-item dropdown">
<button class="btn ps-1 border-0" id="userDropdown" ngbDropdownToggle>
<button class="btn navbar-action border-0" id="userDropdown" ngbDropdownToggle aria-label="User menu" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="person-circle"></i-bs>
<span class="small ms-2 d-none d-sm-inline">
{{this.settingsService.displayName}}
@@ -71,7 +78,7 @@
[ngClass]="slimSidebarEnabled ? 'slim' : 'col-md-3 col-lg-2 col-xxxl-1'" [class.animating]="slimSidebarAnimating()"
[ngbCollapse]="isMenuCollapsed()">
@if (canSaveSettings) {
<button class="btn btn-sm btn-dark sidebar-slim-toggler" (click)="toggleSlimSidebar()">
<button class="btn btn-sm btn-dark sidebar-slim-toggler" (click)="toggleSlimSidebar()" [aria-label]="slimSidebarEnabled ? 'Expand sidebar' : 'Collapse sidebar'" i18n-aria-label>
@if (slimSidebarEnabled) {
<i-bs width="0.9em" height="0.9em" name="chevron-double-right"></i-bs>
} @else {
@@ -79,18 +86,20 @@
}
</button>
}
<div class="sidebar-sticky pt-3 d-flex flex-column justify-space-around">
<div class="sidebar-sticky pt-3 pb-1 d-flex flex-column justify-space-around">
<ul class="nav flex-column">
<li class="nav-item app-link">
<a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="house"></i-bs><span><ng-container i18n>Dashboard</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }">
<a class="nav-link" routerLink="documents" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
<a class="nav-link" routerLink="documents" routerLinkActive="active"
[routerLinkActiveOptions]="{ paths: 'exact', queryParams: 'ignored', matrixParams: 'ignored', fragment: 'ignored' }"
(click)="closeMenu()"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="files"></i-bs><span><ng-container i18n>Documents</ng-container></span>
</a>
@@ -109,7 +118,7 @@
(cdkDragEnded)="onDragEnd($event)">
<a class="nav-link" routerLink="view/{{view.id}}"
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
@if (showSidebarCounts && !slimSidebarEnabled) {
@@ -147,7 +156,7 @@
<li class="nav-item w-100 app-link">
<a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" routerLink="documents/{{d.id}}"
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="d.title | documentTitle"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" name="file-text"></i-bs><span>{{d.title | documentTitle}}</span>
<span class="close flex-column justify-content-center"
@@ -159,11 +168,12 @@
}
@if (openDocuments.length >= 1) {
<li class="nav-item w-100 app-link">
<a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" [routerLink]="[]" (click)="closeAll()"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
<button type="button" class="nav-link nav-link-action app-link w-100 text-start"
[class.text-truncate]="!slimSidebarEnabled" (click)="closeAll()"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="x"></i-bs><span><ng-container i18n>Close all</ng-container></span>
</a>
</button>
</li>
}
</ul>
@@ -177,10 +187,11 @@
@if (canManageAttributes) {
<li class="nav-item app-link" tourAnchor="tour.tags">
<div class="d-flex align-items-center attributes-row">
<a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
<a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active"
[routerLinkActiveOptions]="{ exact: !(slimSidebarEnabled || attributesSectionsCollapsed) }" (click)="closeMenu()"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="stack"></i-bs><span><ng-container i18n>Attributes</ng-container></span>
<i-bs name="stack"></i-bs><span class="ms-2"><ng-container i18n>Attributes</ng-container></span>
</a>
@if (!slimSidebarEnabled && canSaveSettings) {
<button
@@ -230,7 +241,7 @@
}
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
<a class="nav-link" routerLink="savedviews" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="window-stack"></i-bs><span><ng-container i18n>Saved Views</ng-container></span>
</a>
@@ -239,7 +250,7 @@
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Workflow }"
tourAnchor="tour.workflows">
<a class="nav-link" routerLink="workflows" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="boxes"></i-bs><span><ng-container i18n>Workflows</ng-container></span>
</a>
@@ -247,14 +258,14 @@
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
tourAnchor="tour.mail">
<a class="nav-link" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="envelope"></i-bs><span><ng-container i18n>Mail</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
<a class="nav-link" routerLink="trash" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Trash"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="trash"></i-bs><span><ng-container i18n>Trash</ng-container></span>
</a>
@@ -270,21 +281,21 @@
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }"
tourAnchor="tour.settings">
<a class="nav-link" routerLink="settings" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="gear"></i-bs><span><ng-container i18n>Settings</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.AppConfig }">
<a class="nav-link" routerLink="config" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="sliders2-vertical"></i-bs><span><ng-container i18n>Configuration</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.User }">
<a class="nav-link" routerLink="usersgroups" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="people"></i-bs><span><ng-container i18n>Users & Groups</ng-container></span>
</a>
@@ -293,7 +304,7 @@
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.PaperlessTask }"
tourAnchor="tour.file-tasks">
<a class="nav-link" routerLink="tasks" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="list-task"></i-bs><span><ng-container i18n>Tasks</ng-container>@if (tasksService.needsAttentionTasks.length > 0) {
<span><span class="badge bg-danger ms-2 d-inline">{{tasksService.needsAttentionTasks.length}}</span></span>
@@ -306,26 +317,26 @@
@if (permissionsService.isAdmin()) {
<li class="nav-item app-link">
<a class="nav-link" routerLink="logs" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Logs"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="text-left"></i-bs><span><ng-container i18n>Logs</ng-container></span>
</a>
</li>
}
<li class="nav-item mt-2" tourAnchor="tour.outro">
<a class="px-3 py-2 text-muted small d-flex align-items-center flex-wrap text-decoration-none"
<a class="text-muted small d-flex align-items-center flex-wrap text-decoration-none nav-anchor"
target="_blank" rel="noopener noreferrer" href="https://docs.paperless-ngx.com" ngbPopover="Documentation"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="d-flex me-2" name="question-circle"></i-bs><span><ng-container i18n>Documentation</ng-container></span>
</a>
</li>
<li class="nav-item" [class.visually-hidden]="slimSidebarEnabled">
<div class="px-3 py-0 text-muted small d-flex align-items-center flex-wrap">
<div class="text-muted small d-flex align-items-center flex-wrap nav-label">
<div class="me-3">
<a class="text-muted text-decoration-none" target="_blank" rel="noopener noreferrer"
href="https://github.com/paperless-ngx/paperless-ngx" ngbPopover="GitHub" i18n-ngbPopover
[disablePopover]="!slimSidebarEnabled" placement="end" container="body"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
{{ versionString }}
</a>
@@ -363,7 +374,7 @@
</a>
}
} @else {
<a *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }" class="small text-decoration-none" routerLink="/settings" fragment="update-checking"
<a *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }" class="small text-decoration-none" routerLink="/settings" fragment="update-checking" aria-label="Configure update checking" i18n-aria-label
[ngbPopover]="updateCheckingNotEnabledPopContent" popoverClass="shadow" triggers="mouseenter"
container="body">
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
@@ -7,8 +7,8 @@
bottom: 0;
left: 0;
z-index: 995; /* Behind the navbar */
padding: 50px 0 0; /* Height of navbar */
box-shadow: inset -1px 0 0 rgba(0, 0, 0, .1);
padding: 64px 0 0; /* Height of navbar */
border-right: 1px solid color-mix(in srgb, var(--bs-border-color) 65%, transparent);
overflow-y: auto;
--pngx-sidebar-width: 100%;
max-width: var(--pngx-sidebar-width);
@@ -25,7 +25,7 @@
}
.view-name {
max-width: calc(100% - 50px)
max-width: calc(100% - 55px)
}
.nav-group:not(:has(.app-link)) .sidebar-heading {
@@ -47,7 +47,7 @@
}
@media (max-width: 767.98px) {
.sidebar {
top: 3.5rem;
top: 4rem;
}
.search-container {
@@ -67,16 +67,35 @@
main.mobile-search-hidden {
padding-top: 56px;
}
.navbar-toggler {
padding-left: calc(12px - (1.5em * 2.5 / 16));
}
}
.search-container {
padding-left: 1rem;
padding-right: 1rem;
}
// Below sm the search gets its own full-width row, so line it up with main's content edge
@media (max-width: 575.98px) {
.search-container {
padding-left: 12px;
padding-right: 12px;
}
}
main {
transition: all .2s ease;
padding-top: 110px;
padding-top: 118px;
background: var(--pngx-bg-darker);
min-height: 100vh;
}
@media (min-width: 768px) {
main {
padding-top: 56px;
padding-top: 64px;
}
}
@@ -91,11 +110,11 @@ main {
@media(min-width: 768px) {
.sidebar.slim {
max-width: 50px;
max-width: 55px;
li.nav-item span.badge {
display: inline-block;
margin-right: 2px;
margin-right: -4px;
}
}
@@ -139,18 +158,11 @@ main {
}
}
.sidebar.slim {
li.nav-item span.badge {
display: inline-block;
margin-right: 2px;
}
}
.sidebar-slim-toggler {
display: block;
position: fixed;
left: calc(var(--pngx-sidebar-width) - 12px);
top: 60px;
bottom: 16px;
z-index: 996;
--bs-btn-padding-x: 0.35rem;
--bs-btn-padding-y: 0.125rem;
@@ -158,7 +170,7 @@ main {
}
.sidebar.slim .sidebar-slim-toggler {
--pngx-sidebar-width: 50px !important;
--pngx-sidebar-width: 56px !important;
}
}
@@ -171,7 +183,7 @@ main {
position: relative;
top: 0;
height: 100%;
padding-top: 0.5rem;
padding: .75rem .5rem 1rem;
overflow-x: hidden;
overflow-y: auto; /* Scrollable contents if viewport is shorter than content. */
min-height: min-content;
@@ -186,11 +198,14 @@ main {
.sidebar .nav-link {
font-weight: 500;
white-space: nowrap;
border-left: 2px solid transparent;
transition: color .15s ease-in-out;
border-radius: .55rem;
margin: 1px 0;
padding: .55rem .7rem;
transition: color .15s ease-in-out, background-color .15s ease-in-out;
&:hover, &.active, &:focus {
&:hover, &:focus {
color: var(--bs-primary);
background-color: color-mix(in srgb, var(--bs-primary) 8%, transparent);
}
&:focus-visible {
@@ -199,8 +214,21 @@ main {
}
&.active {
font-weight: bold;
border-left-color: var(--bs-primary);
font-weight: 600;
color: var(--bs-primary);
background-color: color-mix(in srgb, var(--bs-primary) 13%, transparent);
}
&.nav-link-action {
color: var(--bs-secondary-color);
font-weight: 400;
background-color: transparent;
&:hover,
&:focus {
color: var(--bs-primary);
background-color: transparent;
}
}
i-bs {
@@ -209,20 +237,49 @@ main {
}
}
// sub-page gets marker only
.nav-item:has(.attributes-submenu.show .nav-link.active) > .attributes-row > .nav-link.active {
border-left-color: transparent;
.sidebar .nav-anchor, .sidebar .nav-label {
padding: .25rem .7rem;
}
// bring sub-menu markers back out to L edge
.attributes-submenu .nav-link {
margin-left: -1rem;
padding-left: calc(var(--bs-nav-link-padding-x) + 1rem);
.attributes-row {
border-radius: .55rem;
margin: .1rem 0;
transition: color .15s ease-in-out, background-color .15s ease-in-out;
> .nav-link {
margin: 0;
&:hover,
&:focus,
&.active {
background-color: transparent;
}
}
&:hover,
&:has(> .nav-link:focus-visible) {
background-color: color-mix(in srgb, var(--bs-primary) 8%, transparent);
}
&:has(> .nav-link.active) {
background-color: color-mix(in srgb, var(--bs-primary) 13%, transparent);
}
}
.attributes-row .attributes-expand-btn {
opacity: 0.2;
width: 1.75rem;
height: 1.75rem;
margin-right: .35rem !important;
border-radius: 50%;
box-shadow: none !important;
transition: opacity 0.15s ease-in-out;
&:focus-visible {
outline: 2px solid color-mix(in srgb, var(--bs-primary) 55%, transparent);
outline-offset: 1px;
opacity: 1;
}
}
.attributes-row:hover .attributes-expand-btn {
@@ -230,8 +287,10 @@ main {
}
.sidebar-heading {
font-size: 0.75rem;
font-size: 0.68rem;
text-transform: uppercase;
font-weight: 700;
opacity: .75;
}
.nav {
@@ -288,16 +347,118 @@ main {
*/
.navbar-brand {
font-size: 1rem;
--pngx-navbar-brand-shadow-rgb: 0, 0, 0;
font-size: 1.0625rem;
min-height: 64px;
letter-spacing: -0.015em;
.flex-column {
padding: 0.15rem 0;
&:hover,
&:focus-visible {
::ng-deep .navbar-official-logo,
.brand-mark,
.brand-logo {
filter: drop-shadow(0 2px 3px rgba(var(--pngx-navbar-brand-shadow-rgb), .5));
}
}
::ng-deep .navbar-official-logo {
filter: drop-shadow(0 1px 2px rgba(var(--pngx-navbar-brand-shadow-rgb), .3));
transition: filter .15s ease-in-out;
@media screen and (max-width: 575.98px) {
max-height: 2rem;
}
}
.brand-mark {
width: 1.65rem;
height: 1.65rem;
flex: 0 0 auto;
transition: filter .15s ease-in-out;
}
.brand-copy {
display: flex;
flex-direction: column;
align-items: flex-start;
min-width: 0;
line-height: 1.1;
transition: transform .15s ease-in-out;
}
.brand-title {
font-weight: 600;
max-width: 100%;
min-width: 0;
}
.byline {
font-size: 0.5rem;
letter-spacing: 0.1rem;
margin-top: .15rem;
font-size: .5rem;
font-weight: 500;
letter-spacing: .1rem;
opacity: .8;
}
.brand-logo {
width: auto;
height: 2.75rem;
max-width: 5rem;
flex: 0 0 auto;
object-fit: contain;
transition: filter .15s ease-in-out, transform .15s ease-in-out;
}
}
:host-context(.primary-light) .navbar-brand {
--pngx-navbar-brand-shadow-rgb: 255, 255, 255; // Light app color, use white shadow for dark text
}
:host ::ng-deep .navbar-official-logo {
.leaf {
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 70%, var(--bs-primary)) !important;
}
.text {
fill: var(--pngx-primary-text-contrast) !important;
}
}
.navbar {
min-height: 64px;
box-shadow: 0 1px 0 rgba(0, 0, 0, .12), 0 4px 18px rgba(0, 0, 0, .08) !important;
}
.navbar > ul {
align-items: center;
gap: .125rem;
padding-right: .5rem;
}
:host ::ng-deep .navbar-action {
display: inline-flex;
align-items: center;
justify-content: center;
min-width: 2.5rem;
min-height: 2.5rem;
padding: .45rem .55rem;
border-radius: .6rem;
transition: background-color .15s ease-in-out, opacity .15s ease-in-out;
&:hover,
&:focus-visible {
background-color: rgba(0, 0, 0, .14);
}
}
#userDropdown {
padding-left: .75rem;
}
.header-search {
width: 100%;
max-width: 55rem;
}
@media screen and (max-width: 575.98px) {
@@ -311,6 +472,7 @@ main {
.navbar-toggler {
grid-area: toggler;
color: var(--pngx-primary-text-contrast);
text-align: left;
}
@@ -346,18 +508,44 @@ main {
grid-area: actions;
justify-self: end;
flex-wrap: nowrap;
gap: 0;
padding-right: .25rem;
}
:host ::ng-deep .navbar-action {
min-width: 2.25rem;
padding-right: .4rem;
padding-left: .4rem;
}
#userDropdown {
padding-right: .35rem;
padding-left: .45rem;
}
}
@media screen and (min-width: 768px) {
.navbar-brand.slim {
max-width: 50px;
max-width: 55px;
.brand-logo {
width: 1.65rem;
max-width: 1.65rem;
}
.brand-mark-slim {
display: block !important;
}
}
:host ::ng-deep .navbar-brand.slim .navbar-official-logo {
display: none;
}
}
:host ::ng-deep .dropdown.show .dropdown-toggle,
:host ::ng-deep .dropdown-toggle:hover {
opacity: 0.7;
opacity: 1;
}
.dropdown-toggle::after {
@@ -45,6 +45,7 @@ import { TasksService } from 'src/app/services/tasks.service'
import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment'
import { ChatComponent } from '../chat/chat/chat.component'
import { LogoComponent } from '../common/logo/logo.component'
import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
import { DocumentDetailComponent } from '../document-detail/document-detail.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
@@ -59,6 +60,7 @@ const SCROLL_THRESHOLD = 16
styleUrls: ['./app-frame.component.scss'],
imports: [
GlobalSearchComponent,
LogoComponent,
DocumentTitlePipe,
IfPermissionsDirective,
ToastsDropdownComponent,
@@ -190,11 +192,34 @@ export class AppFrameComponent
return `${environment.appTitle} v${this.settingsService.get(SETTINGS_KEYS.VERSION)}${environment.tag === 'prod' ? '' : ` #${environment.tag}`}`
}
get appTitle(): string {
this.settingsService.trackChanges()
return (
this.settingsService.get(SETTINGS_KEYS.APP_TITLE) || environment.appTitle
)
}
get customAppTitle(): string {
this.settingsService.trackChanges()
return this.settingsService.get(SETTINGS_KEYS.APP_TITLE)
}
get hasCustomBranding(): boolean {
this.settingsService.trackChanges()
return !!(
this.settingsService.get(SETTINGS_KEYS.APP_TITLE)?.length ||
this.settingsService.get(SETTINGS_KEYS.APP_LOGO)?.length
)
}
get customAppLogo(): string {
this.settingsService.trackChanges()
const logo = this.settingsService.get(SETTINGS_KEYS.APP_LOGO)
return logo?.length
? environment.apiBaseUrl.replace(/\/api\/$/, logo)
: null
}
get canSaveSettings(): boolean {
return (
this.permissionsService.currentUserCan(
@@ -253,6 +278,10 @@ export class AppFrameComponent
})
}
get slimSidebarPopoversEnabled(): boolean {
return this.slimSidebarEnabled && !this.isMobileViewport()
}
get attributesSectionsCollapsed(): boolean {
this.settingsService.trackChanges()
return this.settingsService
@@ -4,12 +4,12 @@ form {
> i-bs[name="search"] {
position: absolute;
left: 0.6rem;
top: .35rem;
top: .25rem;
color: rgba(255, 255, 255, 0.6);
@media screen and (min-width: 768px) {
// adjust for smaller font size on non-mobile
top: 0.25rem;
top: .15rem;
}
}
@@ -37,8 +37,9 @@ form {
}
.form-control {
color: rgba(255, 255, 255, 0.3);
background-color: rgba(0, 0, 0, 0.15);
min-height: 2.25rem;
color: rgba(255, 255, 255, 0.55);
background-color: rgba(0, 0, 0, 0.16);
padding-left: 1.8rem;
border-color: rgba(255, 255, 255, 0.2);
transition: all .3s ease, padding-left 0s ease, background-color 0s ease; // Safari requires all
@@ -52,7 +53,7 @@ form {
}
&:focus-within {
background-color: rgba(0, 0, 0, 0.3);
background-color: rgba(0, 0, 0, 0.26);
color: var(--bs-light);
flex-grow: 1;
padding-left: 0.5rem;
@@ -1,9 +1,9 @@
<li ngbDropdown class="nav-item mx-1" (openChange)="onOpenChange($event)">
<li ngbDropdown class="nav-item position-relative" (openChange)="onOpenChange($event)">
@if (toasts().length) {
<span class="badge rounded-pill z-3 pe-none bg-secondary me-2 position-absolute top-0 left-0">{{ toasts().length }}</span>
<span class="notification-count badge rounded-pill z-3 pe-none bg-secondary position-absolute">{{ toasts().length }}</span>
}
<button class="btn border-0" id="notificationsDropdown" ngbDropdownToggle>
<button class="btn navbar-action border-0" id="notificationsDropdown" ngbDropdownToggle aria-label="Notifications" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="bell"></i-bs>
</button>
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="notificationsDropdown">
@@ -11,6 +11,16 @@
display: none;
}
.notification-count {
top: -.2rem;
right: -.2rem;
min-width: 1.15rem;
height: 1.15rem;
padding: .18rem .32rem;
font-size: .68rem;
line-height: 1;
}
.dropdown-item {
white-space: initial;
}
@@ -1,6 +1,6 @@
<li ngbDropdown class="nav-item me-n2" (openChange)="onOpenChange($event)">
<button class="btn border-0" id="chatDropdown" ngbDropdownToggle>
<li ngbDropdown class="nav-item" (openChange)="onOpenChange($event)">
<button class="btn navbar-action border-0" id="chatDropdown" ngbDropdownToggle aria-label="Chat" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="chatSquareDots"></i-bs>
</button>
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="chatDropdown">
@@ -0,0 +1,28 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
</div>
<div class="modal-body">
@if (messageBold) {
<p class="text-break"><b>{{messageBold}}</b></p>
}
@if (message) {
<p class="mb-0 text-break" [innerHTML]="message"></p>
}
@if (showRemoteOcr) {
<div class="form-check mt-3">
<input class="form-check-input" type="checkbox" id="reprocessRemoteOcr" [(ngModel)]="remoteOcr" />
<label class="form-check-label" for="reprocessRemoteOcr" i18n>Use remote OCR</label>
<div class="form-text" i18n>Sends the document to the configured remote OCR service, which may incur costs.</div>
</div>
}
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,72 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { RemoteOCRModeConfig } from 'src/app/data/paperless-config'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { SettingsService } from 'src/app/services/settings.service'
import { ReprocessConfirmDialogComponent } from './reprocess-confirm-dialog.component'
describe('ReprocessConfirmDialogComponent', () => {
let component: ReprocessConfirmDialogComponent
let fixture: ComponentFixture<ReprocessConfirmDialogComponent>
let settingsService: SettingsService
const createComponent = (configured: boolean, mode: string) => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, configured)
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_MODE, mode)
fixture = TestBed.createComponent(ReprocessConfirmDialogComponent)
component = fixture.componentInstance
fixture.detectChanges()
}
beforeEach(async () => {
TestBed.configureTestingModule({
providers: [
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
imports: [ReprocessConfirmDialogComponent],
}).compileComponents()
settingsService = TestBed.inject(SettingsService)
})
it('should not request remote OCR by default', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.remoteOcr).toBeFalsy()
})
it('should not offer remote OCR when no engine is configured', () => {
createComponent(false, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should not offer remote OCR when it already handles every document', () => {
createComponent(true, RemoteOCRModeConfig.ALWAYS)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should offer remote OCR when configured and selective', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeTruthy()
const checkbox = fixture.nativeElement.querySelector('#reprocessRemoteOcr')
expect(checkbox).not.toBeNull()
checkbox.click()
fixture.detectChanges()
expect(component.remoteOcr).toBeTruthy()
})
})
@@ -0,0 +1,20 @@
import { Component, inject } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { SettingsService } from 'src/app/services/settings.service'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-reprocess-confirm-dialog',
templateUrl: './reprocess-confirm-dialog.component.html',
imports: [FormsModule],
})
export class ReprocessConfirmDialogComponent extends ConfirmDialogComponent {
private settings = inject(SettingsService)
remoteOcr: boolean = false
public get showRemoteOcr(): boolean {
// Hidden when it is not configured, or when it already handles every document anyway.
return this.settings.remoteOCRIsSelectable
}
}
@@ -6,7 +6,7 @@
<div class="modal-body">
<div class="row">
<div class="col-2 d-flex justify-content-end">
<button class="btn btn-secondary mt-auto" (click)="rotate(false)">
<button class="btn btn-secondary mt-auto" (click)="rotate(false)" aria-label="Rotate counterclockwise" i18n-aria-label>
<i-bs name="arrow-counterclockwise"></i-bs>
</button>
</div>
@@ -16,7 +16,7 @@
}
</div>
<div class="col-2 d-flex">
<button class="btn btn-secondary mt-auto" (click)="rotate()">
<button class="btn btn-secondary mt-auto" (click)="rotate()" aria-label="Rotate clockwise" i18n-aria-label>
<i-bs name="arrow-clockwise"></i-bs>
</button>
</div>
@@ -1,5 +1,5 @@
<div ngbDropdown #fieldDropdown="ngbDropdown" (openChange)="onOpenClose($event)" [popperOptions]="popperOptions">
<button type="button" class="btn btn-sm btn-outline-primary" id="customFieldsDropdown" [disabled]="disabled" ngbDropdownToggle>
<button type="button" class="btn btn-sm btn-outline-primary" id="customFieldsDropdown" [disabled]="disabled" ngbDropdownToggle aria-label="Custom Fields" i18n-aria-label>
<i-bs name="ui-radios"></i-bs><div class="d-none d-lg-inline ms-1"><ng-container i18n>Custom Fields</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="customFieldsDropdown" class="shadow custom-fields-dropdown">
@@ -1,6 +1,6 @@
@if (useDropdown) {
<div class="btn-group w-100" role="group" ngbDropdown #dropdown="ngbDropdown" (openChange)="onOpenChange($event)" [popperOptions]="popperOptions">
<button class="btn btn-sm btn-outline-primary" id="dropdown_toggle" ngbDropdownToggle [disabled]="disabled">
<button class="btn btn-sm btn-outline-primary" id="dropdown_toggle" ngbDropdownToggle [disabled]="disabled" [aria-label]="title">
<i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
@if (isActive) {
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge>
@@ -38,7 +38,7 @@
ngbDatepicker
#d="ngbDatepicker"
[footerTemplate]="datePickerFooterTemplate" />
<button class="btn btn-sm btn-outline-secondary rounded-end" (click)="d.toggle()" type="button">
<button class="btn btn-sm btn-outline-secondary rounded-end" (click)="d.toggle()" type="button" aria-label="Open date picker" i18n-aria-label>
<i-bs name="calendar-event"></i-bs>
</button>
<ng-template #datePickerFooterTemplate>
@@ -143,7 +143,7 @@
<input class="w-25 form-control rounded-end" type="text" [(ngModel)]="atom.value" [disabled]="disabled">
}
}
<button class="btn btn-link btn-sm text-danger pe-0" type="button" (click)="removeElement(atom)" [disabled]="disabled">
<button class="btn btn-link btn-sm text-danger pe-0" type="button" (click)="removeElement(atom)" [disabled]="disabled" aria-label="Remove query" i18n-aria-label>
<i-bs name="x-circle"></i-bs>
</button>
</div>
@@ -185,7 +185,7 @@
<i-bs name="braces"></i-bs>
</button>
@if (expression.depth > 0) {
<button type="button" class="btn btn-sm btn-outline-secondary text-danger" (click)="removeElement(expression)" [disabled]="disabled">
<button type="button" class="btn btn-sm btn-outline-secondary text-danger" (click)="removeElement(expression)" [disabled]="disabled" aria-label="Remove expression" i18n-aria-label>
<i-bs name="x-circle"></i-bs>
</button>
}
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group" [popperOptions]="popperOptions" [placement]="placement">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="createdDateTo || createdDateFrom ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="createdDateTo || createdDateFrom ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs width="1em" height="1em" name="calendar-event-fill"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
</button>
@@ -9,7 +9,7 @@
<div class="list-group-item d-flex p-2 select-item" role="menuitem">
<div class="selected-icon">
@if (createdRelativeDate) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedRelativeDate()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedRelativeDate()" aria-label="Clear created relative date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -33,7 +33,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (createdDateFrom) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedFrom()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedFrom()" aria-label="Clear created from date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -43,7 +43,7 @@
<span class="input-group-text w-25 small text-muted" i18n>From</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="createdDateFrom" ngbDatepicker #createdDateFromPicker="ngbDatepicker" [footerTemplate]="createdFromFooterTemplate">
<button class="btn btn-outline-secondary" (click)="createdDateFromPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="createdDateFromPicker.toggle()" type="button" aria-label="Open created from date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #createdFromFooterTemplate>
@@ -57,7 +57,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (createdDateTo) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedTo()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedTo()" aria-label="Clear created to date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -67,7 +67,7 @@
<span class="input-group-text w-25 small text-muted" i18n>To</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="createdDateTo" ngbDatepicker #createdDateToPicker="ngbDatepicker" [footerTemplate]="createdToFooterTemplate">
<button class="btn btn-outline-secondary" (click)="createdDateToPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="createdDateToPicker.toggle()" type="button" aria-label="Open created to date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #createdToFooterTemplate>
@@ -85,7 +85,7 @@
<div class="list-group-item d-flex p-2 select-item" role="menuitem">
<div class="selected-icon">
@if (addedRelativeDate) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedRelativeDate()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedRelativeDate()" aria-label="Clear added relative date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -109,7 +109,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (addedDateFrom) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedFrom()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedFrom()" aria-label="Clear added from date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -119,7 +119,7 @@
<span class="input-group-text w-25 small text-muted" i18n>From</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="addedDateFrom" ngbDatepicker #addedDateFromPicker="ngbDatepicker" [footerTemplate]="addedFromFooterTemplate">
<button class="btn btn-outline-secondary" (click)="addedDateFromPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="addedDateFromPicker.toggle()" type="button" aria-label="Open added from date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #addedFromFooterTemplate>
@@ -133,7 +133,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (addedDateTo) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedTo()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedTo()" aria-label="Clear added to date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -143,7 +143,7 @@
<span class="input-group-text w-25 small text-muted" i18n>To</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="addedDateTo" ngbDatepicker #addedDateToPicker="ngbDatepicker" [footerTemplate]="addedToFooterTemplate">
<button class="btn btn-outline-secondary" (click)="addedDateToPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="addedDateToPicker.toggle()" type="button" aria-label="Open added to date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #addedToFooterTemplate>
@@ -455,6 +455,13 @@
</div>
</div>
}
@case (WorkflowActionType.RemoteOcr) {
<div class="row">
<div class="col">
<p class="text-muted small" i18n>The document will be sent to the configured remote OCR service. May incur costs.</p>
</div>
</div>
}
}
</div>
</ng-template>
@@ -29,6 +29,7 @@ import {
DocumentSource,
WorkflowTriggerType,
} from 'src/app/data/workflow-trigger'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
@@ -224,7 +225,12 @@ describe('WorkflowEditDialogComponent', () => {
).toEqual('Document Added')
expect(component.getTriggerTypeOptionName(null)).toEqual('')
expect(component.sourceOptions).toEqual(DOCUMENT_SOURCE_OPTIONS)
expect(component.actionTypeOptions).toEqual(WORKFLOW_ACTION_OPTIONS)
// Remote OCR is absent until the workflow has a consumption trigger
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.RemoteOcr
)
)
expect(
component.getActionTypeOptionName(WorkflowActionType.Assignment)
).toEqual('Assignment')
@@ -237,7 +243,104 @@ describe('WorkflowEditDialogComponent', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(false)
component.ngOnInit()
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter((a) => a.id !== WorkflowActionType.Email)
WORKFLOW_ACTION_OPTIONS.filter(
(a) =>
a.id !== WorkflowActionType.Email &&
a.id !== WorkflowActionType.RemoteOcr
)
)
})
it('should offer remote OCR only for consumption workflows', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// A consumption trigger makes the action reachable
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Any other trigger type runs after the document has been parsed
component.object = {
name: 'Workflow 2',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should offer remote OCR on a trigger added to a new workflow', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
component.ngOnInit()
// Nothing for the action to apply to yet
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
// addTrigger creates the form field with emitEvent false, so the options
// have to be computed on read rather than cached from valueChanges
component.addTrigger()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Switching that trigger to a type that runs after parsing removes it
component.triggerFields
.at(0)
.get('type')
.setValue(WorkflowTriggerType.DocumentAdded)
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should keep remote OCR listed when an action already uses it', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// Otherwise changing the trigger would silently blank the selection
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [{ type: WorkflowActionType.RemoteOcr }],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
})
it('should not offer remote OCR when no engine is configured', () => {
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) => key !== SETTINGS_KEYS.REMOTE_OCR_CONFIGURED)
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
@@ -148,6 +148,10 @@ export const WORKFLOW_ACTION_OPTIONS = [
id: WorkflowActionType.MoveToTrash,
name: $localize`Move to trash`,
},
{
id: WorkflowActionType.RemoteOcr,
name: $localize`Remote OCR`,
},
]
export enum TriggerFilterType {
@@ -504,8 +508,6 @@ export class WorkflowEditDialogComponent
expandedItem: number = null
readonly allowedActionTypes = signal([])
private readonly triggerFilterOptionsMap = new WeakMap<
FormArray,
TriggerFilterOption[]
@@ -548,13 +550,40 @@ export class WorkflowEditDialogComponent
this.checkRemovalActionFields.bind(this)
)
this.checkRemovalActionFields(this.objectForm.value)
this.allowedActionTypes.set(
this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)
? WORKFLOW_ACTION_OPTIONS
: WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.Email
)
)
}
private allowedActionTypes: typeof WORKFLOW_ACTION_OPTIONS = null
private getAllowedActionTypes() {
let allowed = WORKFLOW_ACTION_OPTIONS
if (!this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.Email)
}
// Remote OCR is decided before the document is parsed, so it is only
// offered for workflows that run at consumption.
const formWorkflow: Workflow = this.objectForm?.value
const remoteOcrUsable =
this.settingsService.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
(formWorkflow?.triggers?.some(
(trigger) => trigger.type === WorkflowTriggerType.Consumption
) ||
formWorkflow?.actions?.some(
(action) => action.type === WorkflowActionType.RemoteOcr
))
if (!remoteOcrUsable) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
}
if (
this.allowedActionTypes?.length === allowed.length &&
this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
) {
return this.allowedActionTypes
}
this.allowedActionTypes = allowed
return allowed
}
private checkRemovalActionFields(formWorkflow: Workflow) {
@@ -1279,7 +1308,8 @@ export class WorkflowEditDialogComponent
get actionTypeOptions() {
this.settingsService.trackChanges()
return this.allowedActionTypes()
// Computed on read rather than cached
return this.getAllowedActionTypes()
}
getActionTypeOptionName(type: WorkflowActionType): string {
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group" (openChange)="dropdownOpenChange($event)" #dropdown="ngbDropdown" (keydown)="listKeyDown($event)" [popperOptions]="popperOptions" [autoClose]="!creating()">
<button class="btn btn-sm" id="dropdown_{{name}}" ngbDropdownToggle [ngClass]="!editing && selectionModel.selectionSize() > 0 ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled">
<button class="btn btn-sm" id="dropdown_{{name}}" ngbDropdownToggle [ngClass]="!editing && selectionModel.selectionSize() > 0 ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
@if (!editing && selectionModel.totalCount > 0) {
<pngx-clearable-badge [number]="selectionModel.totalCount" [selected]="selectionModel.selectionSize() > 0" (cleared)="reset()"></pngx-clearable-badge>
@@ -4,7 +4,7 @@
}
<div class="input-group" [class.is-invalid]="error">
<button type="button" class="input-group-text" [style.background-color]="value" (click)="colorPicker.toggle()">&nbsp;&nbsp;&nbsp;</button>
<button type="button" class="input-group-text" [style.background-color]="value" (click)="colorPicker.toggle()" aria-label="Open color picker" i18n-aria-label>&nbsp;&nbsp;&nbsp;</button>
<ng-template #popContent>
<div style="min-width: 200px;" class="pb-3">
@@ -14,7 +14,7 @@
<input #inputField class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" (change)="onChange(value)" [autoClose]="'outside'" [ngbPopover]="popContent" #colorPicker="ngbPopover" placement="bottom" popoverClass="shadow">
<button class="btn btn-outline-secondary" type="button" (click)="randomize()">
<button class="btn btn-outline-secondary" type="button" (click)="randomize()" aria-label="Choose a random color" i18n-aria-label>
<i-bs name="dice5"></i-bs>
</button>
@@ -74,7 +74,7 @@
class="flex-grow-1"></pngx-input-textarea>
}
}
<button type="button" class="btn btn-link text-danger" (click)="removeSelectedField.next(fieldId)">
<button type="button" class="btn btn-link text-danger" (click)="removeSelectedField.next(fieldId)" aria-label="Remove custom field" i18n-aria-label>
<i-bs name="trash"></i-bs>
</button>
</div>
@@ -13,7 +13,7 @@
<input #inputField class="form-control" [class.is-invalid]="error" [placeholder]="placeholder" [id]="inputId" maxlength="10"
(dateSelect)="onChange(value)" (change)="onChange(value)" (keypress)="onKeyPress($event)" (paste)="onPaste($event)"
name="dp" [(ngModel)]="value" ngbDatepicker #datePicker="ngbDatepicker" #datePickerContent="ngModel" [disabled]="disabled" [footerTemplate]="datePickerFooterTemplate">
<button class="btn btn-outline-secondary calendar" (click)="datePicker.toggle()" type="button" [disabled]="disabled">
<button class="btn btn-outline-secondary calendar" (click)="datePicker.toggle()" type="button" [disabled]="disabled" aria-label="Open date picker" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="calendar"></i-bs>
</button>
<ng-template #datePickerFooterTemplate>
@@ -13,7 +13,7 @@
<div class="input-group mb-3">
<input type="text" class="form-control" [(ngModel)]="entry[0]" (change)="inputChange()" [disabled]="disabled" autocomplete="off">
<input type="text" class="form-control" [(ngModel)]="entry[1]" (change)="inputChange()" [disabled]="disabled" autocomplete="off">
<button type="button" class="btn btn-outline-secondary" (click)="removeEntry(i)">
<button type="button" class="btn btn-outline-secondary" (click)="removeEntry(i)" aria-label="Remove entry" i18n-aria-label>
<i-bs class="text-danger" name="trash"></i-bs>
</button>
</div>
@@ -50,7 +50,7 @@
</ng-template>
</ng-select>
@if (allowCreateNew && !hideAddButton) {
<button class="btn btn-outline-secondary" type="button" (click)="addItem()" [disabled]="disabled">
<button class="btn btn-outline-secondary" type="button" (click)="addItem()" [disabled]="disabled" aria-label="Create new item" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="plus"></i-bs>
</button>
}
@@ -48,7 +48,7 @@
</ng-template>
</ng-select>
@if (allowCreate && !hideAddButton) {
<button class="btn btn-outline-secondary" type="button" (click)="createTag(null, true)" [disabled]="disabled">
<button class="btn btn-outline-secondary" type="button" (click)="createTag(null, true)" [disabled]="disabled" aria-label="Create new tag" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="plus"></i-bs>
</button>
}
@@ -1,6 +1,6 @@
<div class="row pt-3 pb-3 pb-md-2 align-items-center">
<div class="row pt-3 pb-2 align-items-center">
<div class="col-md text-truncate">
<h3 class="d-flex align-items-center mb-1" style="line-height: 1.4">
<h3 class="d-flex align-items-center mb-2 mb-md-1" style="line-height: 1.4">
<span class="text-truncate">{{title()}}</span>
@if (id()) {
<span class="badge bg-primary text-primary-text-contrast ms-3 small fs-normal cursor-pointer" (click)="copyID()">
@@ -12,10 +12,10 @@
</span>
}
@if (subTitle()) {
<span class="h6 mb-0 mt-1 d-block d-md-inline fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
<span class="page-subtitle h6 mb-0 mt-1 fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
}
@if (info()) {
<button class="btn btn-sm btn-link text-muted p-0 p-md-2" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<button class="btn btn-sm btn-link text-muted p-0 ms-2 p-md-2 ms-md-0" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<i-bs name="question-circle"></i-bs>
</button>
<ng-template #infoPopover>
@@ -1,5 +1,6 @@
h3 {
min-height: calc(1.325rem + 0.9vw);
flex-wrap: wrap;
.badge {
font-size: 0.65rem;
@@ -7,6 +8,25 @@ h3 {
}
}
// Drop the subtitle onto its own and shrink it
@media (max-width: 767.98px) {
h3 > .page-subtitle {
flex: 0 0 100%;
margin-top: 0 !important;
font-size: .7rem;
line-height: 1.3 !important;
}
}
:host {
display: block;
margin-bottom: .35rem;
}
h3 > .h6 {
color: var(--bs-secondary-color);
}
@media (min-width: 1200px) {
h3 {
min-height: 2.8rem;
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="isActive ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="isActive ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
</button>
@@ -107,6 +107,25 @@ describe('PermissionsSelectComponent', () => {
expect(component.form.get('Tag').get('Change').disabled).toBeTruthy()
})
it('should update checkboxes when inherited permissions change', () => {
component.ngOnInit()
component.inheritedPermissions = ['documents.change_document']
component.writeValue(['delete_document'])
expect(component.form.get('Document').get('Change').value).toBeTruthy()
expect(component.form.get('Document').get('Change').disabled).toBeTruthy()
// swap for a group with a different permission, but the same number of them
component.inheritedPermissions = ['documents.view_document']
// the no-longer-inherited permission is unchecked, the explicit one is kept
expect(component.permissions).toEqual(['delete_document'])
expect(component.form.get('Document').get('Change').value).toBeFalsy()
expect(component.form.get('Document').get('Change').disabled).toBeFalsy()
expect(component.form.get('Document').get('Delete').value).toBeTruthy()
expect(component.form.get('Document').get('View').value).toBeTruthy()
expect(component.form.get('Document').get('View').disabled).toBeTruthy()
})
it('should exclude history permissions if disabled', () => {
settingsService.set(SETTINGS_KEYS.AUDITLOG_ENABLED, false)
fixture = TestBed.createComponent(PermissionsSelectComponent)
@@ -74,12 +74,22 @@ export class PermissionsSelectComponent
? inherited.map((p) => p.replace(/^\w+\./g, ''))
: []
if (this._inheritedPermissions !== newInheritedPermissions) {
this._inheritedPermissions = newInheritedPermissions
this.writeValue(this.permissions) // updates visual checks etc.
}
const changed =
newInheritedPermissions.length !== this._inheritedPermissions.length ||
newInheritedPermissions.some(
(p) => !this._inheritedPermissions.includes(p)
)
this.updateDisabledStates()
if (changed) {
// skip inherited permissions, these are the explicitly set ones
this.permissions = this.getSelectedPermissions(
this.form.getRawValue()
).filter((p) => !this._inheritedPermissions.includes(p))
this._inheritedPermissions = newInheritedPermissions
this.applyCheckedState()
} else {
this.updateDisabledStates()
}
}
inheritedWarning: string = $localize`Inherited from group`
@@ -106,20 +116,29 @@ export class PermissionsSelectComponent
}
this.permissions = permissions ?? []
const allPerms = this._inheritedPermissions.concat(this.permissions)
this.applyCheckedState()
}
allPerms.forEach((permissionStr) => {
const { actionKey, typeKey } =
this.permissionsService.getPermissionKeys(permissionStr)
// sets every checkbox from inherited + own perms
private applyCheckedState(): void {
const allPerms = new Set(
this._inheritedPermissions.concat(this.permissions)
)
if (actionKey && typeKey) {
this.form
.get(typeKey)
?.get(actionKey)
?.patchValue(true, { emitEvent: false })
}
})
this.allowedTypes.forEach((type) => {
const typeGroup = this.form.get(type)
for (const action of Object.keys(PermissionAction)) {
typeGroup.get(action)?.patchValue(
allPerms.has(
this.permissionsService.getPermissionCode(
PermissionAction[action],
PermissionType[type]
)
),
{ emitEvent: false } // don't trigger valueChanges now
)
}
if (this.typeHasAllActionsSelected(type)) {
this.typesWithAllActions.add(type)
} else {
@@ -150,26 +169,9 @@ export class PermissionsSelectComponent
ngOnInit(): void {
this.form.valueChanges.subscribe((newValue) => {
let permissions = []
Object.entries(newValue).forEach(([typeKey, typeValue]) => {
const selectedActions = Object.entries(typeValue).filter(
([actionKey, actionValue]) =>
actionValue &&
this.isActionSupported(
PermissionType[typeKey],
PermissionAction[actionKey]
)
)
selectedActions.forEach(([actionKey]) => {
permissions.push(
(PermissionType[typeKey] as string).replace(
'%s',
PermissionAction[actionKey]
)
)
})
const permissions = this.getSelectedPermissions(newValue)
Object.keys(newValue).forEach((typeKey) => {
if (this.typeHasAllActionsSelected(typeKey)) {
this.typesWithAllActions.add(typeKey)
} else {
@@ -269,6 +271,30 @@ export class PermissionsSelectComponent
return true
}
private getSelectedPermissions(formValue: object): string[] {
const permissions = []
Object.entries(formValue).forEach(([typeKey, typeValue]) => {
Object.entries(typeValue)
.filter(
([actionKey, actionValue]) =>
actionValue &&
this.isActionSupported(
PermissionType[typeKey],
PermissionAction[actionKey]
)
)
.forEach(([actionKey]) => {
permissions.push(
this.permissionsService.getPermissionCode(
PermissionAction[actionKey],
PermissionType[typeKey]
)
)
})
})
return permissions
}
private typeHasAllActionsSelected(typeKey: string): boolean {
return Object.keys(PermissionAction)
.filter((action) =>
@@ -1,5 +1,5 @@
<div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())">
<button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())" [aria-label]="noSuggestions ? 'No suggestions' : 'Suggest'" i18n-aria-label>
@if (loading()) {
<div class="spinner-border spinner-border-sm" role="status"></div>
} @else if (noSuggestions) {
@@ -23,7 +23,7 @@
<dd>
{{status().pngx_version}}
@if (versionMismatch()) {
<button class="btn btn-sm d-inline align-items-center btn-dark text-uppercase small" [ngbPopover]="versionPopover" triggers="click mouseenter:mouseleave">
<button class="btn btn-sm d-inline align-items-center btn-dark text-uppercase small" [ngbPopover]="versionPopover" triggers="click mouseenter:mouseleave" aria-label="View version mismatch details" i18n-aria-label>
<i-bs name="exclamation-triangle-fill" class="text-danger lh-1"></i-bs>
</button>
}
@@ -1,9 +1,7 @@
<pngx-page-header title="Dashboard" [subTitle]="subtitle" i18n-title tourAnchor="tour.dashboard">
<pngx-logo extra_classes="d-none d-md-block mt-n2" height="3rem"></pngx-logo>
</pngx-page-header>
<pngx-page-header title="Dashboard" [subTitle]="subtitle" i18n-title tourAnchor="tour.dashboard"></pngx-page-header>
<div class="row">
<div class="col-12 col-lg-8 col-xl-9 mb-4">
<div class="row dashboard-grid g-4 pb-0">
<div class="col-12 col-lg-8 col-xl-9 mb-4 dashboard-main">
<div class="row row-cols-1 g-4"
cdkDropList
[cdkDropListDisabled]="settingsService.globalDropzoneActive()"
@@ -58,7 +56,7 @@
</ng-container>
</div>
</div>
<div class="col-12 col-lg-4 col-xl-3 col-sidebar">
<div class="col-12 col-lg-4 col-xl-3 col-sidebar dashboard-aside">
<div class="row row-cols-1 g-4 mb-4 sticky-lg-top z-0">
<pngx-upload-file-widget></pngx-upload-file-widget>
<pngx-statistics-widget *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.UISettings }"></pngx-statistics-widget>
@@ -1,3 +1,13 @@
.col-sidebar .row {
top: 3.5rem;
top: 4.75rem;
}
.dashboard-grid {
padding-bottom: 2rem;
}
@media (min-width: 1200px) {
.dashboard-main {
padding-right: 1rem;
}
}
@@ -15,7 +15,6 @@ import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment'
import { LogoComponent } from '../common/logo/logo.component'
import { PageHeaderComponent } from '../common/page-header/page-header.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
import { SavedViewWidgetComponent } from './widgets/saved-view-widget/saved-view-widget.component'
@@ -28,7 +27,6 @@ import { WelcomeWidgetComponent } from './widgets/welcome-widget/welcome-widget.
templateUrl: './dashboard.component.html',
styleUrls: ['./dashboard.component.scss'],
imports: [
LogoComponent,
PageHeaderComponent,
SavedViewWidgetComponent,
StatisticsWidgetComponent,
@@ -8,3 +8,14 @@
width: 0.6rem;
}
}
.list-group {
--bs-list-group-border-color: color-mix(in srgb, var(--bs-border-color) 58%, transparent);
--bs-list-group-bg: transparent;
border-radius: .6rem;
overflow: hidden;
}
.list-group-item {
padding: .7rem .8rem;
}
@@ -4,6 +4,16 @@
.btn-outline-dark {
--bs-btn-border-color: var(--bs-border-color-translucent);
border-style: dashed;
border-width: 1px;
min-height: 5.25rem;
background: color-mix(in srgb, var(--bs-primary) 4%, var(--bs-light)) !important;
&:hover,
&:focus {
border-color: var(--bs-primary);
background: color-mix(in srgb, var(--bs-primary) 9%, var(--bs-light)) !important;
}
}
.smaller {
@@ -1,5 +1,5 @@
@if (!cardless()) {
<div class="card shadow-sm bg-light fade" [class.show]="show()" cdkDrag [cdkDragDisabled]="!draggable()" cdkDragPreviewContainer="parent">
<div class="card bg-light fade" [class.show]="show()" cdkDrag [cdkDragDisabled]="!draggable()" cdkDragPreviewContainer="parent">
<div class="card-header">
<div class="d-flex justify-content-between align-items-center">
<div class="d-flex align-items-center">
@@ -2,6 +2,20 @@ i-bs {
cursor: move;
}
.card {
overflow: hidden;
border-color: color-mix(in srgb, var(--bs-border-color) 68%, transparent);
.card-header {
padding: .9rem 1rem;
border-bottom-color: color-mix(in srgb, var(--bs-border-color) 55%, transparent);
}
.card-body {
padding: 1rem;
}
}
.fade.show {
animation: pngx-entry-fade 160ms ease-out;
}
@@ -20,7 +20,7 @@
</div>
}
<button type="button" class="btn btn-sm btn-outline-danger me-md-4" (click)="delete()" [disabled]="!userIsOwner" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
<button type="button" class="btn btn-sm btn-outline-danger me-md-4" (click)="delete()" [disabled]="!userIsOwner" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }" aria-label="Delete" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="trash"></i-bs><span class="d-none d-lg-inline ps-1" i18n>Delete</span>
</button>
@@ -35,7 +35,7 @@
/>
<div class="btn-group">
<button (click)="download()" class="btn btn-sm btn-outline-primary" [disabled]="downloading()">
<button (click)="download()" class="btn btn-sm btn-outline-primary" [disabled]="downloading()" aria-label="Download" i18n-aria-label>
@if (downloading()) {
<div class="spinner-border spinner-border-sm" role="status"></div>
} @else {
@@ -45,7 +45,7 @@
</button>
<div class="btn-group" ngbDropdown role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle" [disabled]="downloading()" ngbDropdownToggle></button>
<button class="btn btn-sm btn-outline-primary dropdown-toggle" [disabled]="downloading()" ngbDropdownToggle aria-label="Download options" i18n-aria-label></button>
<div class="dropdown-menu shadow" ngbDropdownMenu>
@if (metadata()?.has_archive_version) {
<button ngbDropdownItem (click)="download(true)" [disabled]="downloading()" i18n>Download original</button>
@@ -62,7 +62,7 @@
</div>
<div class="ms-auto" ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="actionsDropdown" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="actionsDropdown" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow">
@@ -91,7 +91,7 @@
</div>
<div class="ms-auto" ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="sendDropdown" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="sendDropdown" ngbDropdownToggle aria-label="Send" i18n-aria-label>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow">
@@ -963,12 +963,24 @@ describe('DocumentDetailComponent', () => {
component.reprocess()
const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] })
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, false)
expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
})
it('should pass remote OCR choice when reprocessing', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
reprocessSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component.reprocess()
openModal.componentInstance.remoteOcr = true
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, true)
})
it('should show error if redo ocr call fails', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
@@ -2171,6 +2183,11 @@ describe('DocumentDetailComponent', () => {
).toBe(10)
component.openEmailDocument()
expect(modalSpy).toHaveBeenCalled()
expect(
(
modalSpy.mock.results[1].value as NgbModalRef
).componentInstance.documentIds()
).toEqual([10])
})
it('should set previewText', () => {
@@ -97,6 +97,7 @@ import { ISODateAdapter } from 'src/app/utils/ngb-iso-date-adapter'
import * as UTIF from 'utif'
import { DocumentDetailFieldID } from '../admin/settings/settings.component'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
@@ -1398,7 +1399,7 @@ export class DocumentDetailComponent
}
reprocess() {
let modal = this.modalService.open(ConfirmDialogComponent, {
let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Reprocess confirm`
@@ -1409,7 +1410,10 @@ export class DocumentDetailComponent
modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled.set(false)
this.documentsService
.reprocessDocuments({ documents: [this.document().id] })
.reprocessDocuments(
{ documents: [this.document().id] },
modal.componentInstance.remoteOcr
)
.subscribe({
next: () => {
this.toastService.showInfo(
@@ -1973,7 +1977,9 @@ export class DocumentDetailComponent
const modal = this.modalService.open(EmailDocumentDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.documentIds.set([this.document().id])
modal.componentInstance.documentIds.set([
this.selectedVersionId() ?? this.document().id,
])
modal.componentInstance.hasArchiveVersion.set(
this.metadata()?.has_archive_version ??
!!this.document()?.archived_file_name
@@ -1,7 +1,10 @@
<div class="btn-group" ngbDropdown autoClose="outside">
<button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle aria-label="Versions" i18n-aria-label>
<i-bs name="file-earmark-diff"></i-bs>
<span class="d-none d-lg-inline ps-1" i18n>Versions</span>
@if (versions.length > 1) {
<span class="badge text-bg-secondary ms-1">{{ versions.length }}</span>
}
</button>
<div class="dropdown-menu shadow" ngbDropdownMenu>
<div class="px-3 py-2 mb-2">
@@ -1,6 +1,6 @@
<h6>
<button type="button" class="btn btn-outline-secondary btn-sm me-2"
(click)="expand = !expand">
(click)="expand = !expand" aria-label="Toggle document metadata" i18n-aria-label>
@if (!expand) {
<i-bs width="1.2em" height="1.2em" name="caret-down"></i-bs>
}
@@ -74,7 +74,7 @@
</pngx-filterable-dropdown>
}
<div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll">
<button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll" aria-label="Permissions" i18n-aria-label>
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Permissions</ng-container></div>
</button>
</div>
@@ -82,7 +82,7 @@
<div class="d-flex align-items-center gap-2 ms-auto">
<div class="btn-toolbar">
<div ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
@@ -107,6 +107,8 @@
id="dropdownSend"
ngbDropdownToggle
[disabled]="disabled || !canSendSelection"
aria-label="Send"
i18n-aria-label
>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container>
</div>
@@ -127,7 +129,7 @@
</div>
</div>
<div class="btn-group btn-group-sm">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()" aria-label="Download" i18n-aria-label>
@if (!awaitingDownload()) {
<i-bs name="arrow-down"></i-bs>
}
@@ -139,7 +141,7 @@
<div class="d-none d-sm-inline ms-1"><ng-container i18n>Download</ng-container></div>
</button>
<div ngbDropdown class="me-2 d-flex btn-group" role="group">
<button type="button" class="btn btn-sm btn-outline-primary dropdown-toggle-split rounded-end" ngbDropdownToggle></button>
<button type="button" class="btn btn-sm btn-outline-primary dropdown-toggle-split rounded-end" ngbDropdownToggle aria-label="Download options" i18n-aria-label></button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
<form [formGroup]="downloadForm" class="px-3 py-1">
<p class="mb-1" i18n>Include:</p>
@@ -1,5 +1,5 @@
.dropdown-toggle-split {
--bs-border-radius: .25rem;
--bs-border-radius: .45rem;
}
.dropdown-menu{
@@ -1122,6 +1122,7 @@ describe('BulkEditorComponent', () => {
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
remote_ocr: false,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -52,6 +52,7 @@ import { flattenTags } from 'src/app/utils/flatten-tags'
import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dialog/merge-as-versions-confirm-dialog/merge-as-versions-confirm-dialog.component'
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
@@ -916,7 +917,7 @@ export class BulkEditorComponent
}
reprocessSelected() {
let modal = this.modalService.open(ConfirmDialogComponent, {
let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Reprocess confirm`
@@ -930,7 +931,10 @@ export class BulkEditorComponent
modal.componentInstance.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
this.documentService.reprocessDocuments(this.getSelectionQuery())
this.documentService.reprocessDocuments(
this.getSelectionQuery(),
modal.componentInstance.remoteOcr
)
)
})
}
@@ -61,7 +61,7 @@
</pngx-input-textarea>
}
}
<button type="button" class="btn btn-outline-danger mb-3" (click)="removeField(field.id)">
<button type="button" class="btn btn-outline-danger mb-3" (click)="removeField(field.id)" aria-label="Remove custom field" i18n-aria-label>
<i-bs name="x"></i-bs>
</button>
</div>
@@ -154,9 +154,9 @@
<i-bs name="download"></i-bs>
</a>
} @else {
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button>
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button>
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button>
<span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
<span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
<span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
}
</div>
</div>
@@ -1,6 +1,6 @@
<pngx-page-header [title]="getTitle()">
<div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle aria-label="Select" i18n-aria-label>
<i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div>
@if (list.hasSelection) {
<pngx-clearable-badge [selected]="list.hasSelection" [number]="list.selectedCount" (cleared)="list.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
@@ -31,7 +31,7 @@
</div>
</div>
<div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle aria-label="Show" i18n-aria-label>
<i-bs name="card-heading"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Show</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownDisplayFields" class="shadow">
@@ -61,7 +61,7 @@
</div>
<div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle>
<button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle aria-label="Sort" i18n-aria-label>
<i-bs name="arrow-down-up"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Sort</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownBasic1" class="shadow dropdown-menu-right">
@@ -85,8 +85,8 @@
</div>
</div>
<div class="btn-group flex-fill" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" ngbDropdown role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle flex-fill" tourAnchor="tour.documents-views" ngbDropdownToggle>
<div class="btn-group flex-fill" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" ngbDropdown #viewsDropdown="ngbDropdown" role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle flex-fill" tourAnchor="tour.documents-views" ngbDropdownToggle aria-label="Views" i18n-aria-label>
<i-bs name="window-stack"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Views</ng-container></div>
@if (savedViewIsModified) {
<div class="position-absolute top-0 start-100 p-2 translate-middle badge bg-secondary border border-light rounded-circle">
@@ -95,15 +95,15 @@
}
</button>
<div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu>
@if (!list.activeSavedViewId) {
@for (view of savedViewService.allViews; track view) {
<button ngbDropdownItem (click)="loadViewConfig(view.id)">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
</button>
}
@if (savedViewService.allViews.length > 0) {
<div class="dropdown-divider"></div>
}
@if (viewsDropdown.isOpen() && !list.activeSavedViewId && savedViewService.allViews.length > 0) {
<div class="views-list overflow-y-auto">
@for (view of savedViewService.allViews; track view.id) {
<button ngbDropdownItem (click)="loadViewConfig(view.id)">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
</button>
}
</div>
<div class="dropdown-divider"></div>
}
@if (list.activeSavedViewId && activeSavedViewCanChange) {
@@ -157,7 +157,7 @@
</div>
</ng-template>
<div tourAnchor="tour.documents">
<div class="mt-3 mb-n2" tourAnchor="tour.documents">
<ng-container *ngTemplateOutlet="pagination"></ng-container>
</div>
@@ -56,11 +56,11 @@ $paperless-card-breakpoints: (
.sticky-top {
z-index: 990; // below main navbar
top: calc(7rem - 2px); // height of navbar + search row (mobile)
top: calc(7.5rem - 2px); // height of navbar + search row (mobile)
transition: top 0.2s ease;
@media (min-width: 580px) {
top: 3.5rem; // height of navbar
top: 4.5em; // height of navbar
}
}
@@ -73,7 +73,7 @@ $paperless-card-breakpoints: (
@media (max-width: 579.98px) {
:host-context(main.mobile-search-hidden) .sticky-top {
top: calc(3.5rem - 2px); // height of navbar only when search is hidden
top: calc(4rem - 2px); // height of navbar only when search is hidden
}
}
@@ -93,4 +93,8 @@ a {
pngx-page-header .dropdown-menu {
--bs-dropdown-min-width: 12em;
.views-list {
max-height: min(400px, calc(100vh - 260px)); // leave room for the header above and the actions below
}
}
@@ -18,7 +18,7 @@
</select>
}
@if (_textFilter) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()">
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs>
</button>
}
@@ -24,7 +24,7 @@
<div class="btn-toolbar gap-2">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -7,7 +7,7 @@
>
@if (activeManagementList) {
<div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle aria-label="Select" i18n-aria-label>
<i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div>
@if (activeManagementList.hasSelection) {
<pngx-clearable-badge [selected]="activeManagementList.hasSelection" [number]="activeManagementList.selectedCount" (cleared)="activeManagementList.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
@@ -1,7 +1,7 @@
<div class="row mb-3">
<div class="col mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<label class="text-muted me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<label class="me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<input id="managementNameFilter" class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
</div>
</div>
@@ -9,7 +9,7 @@
<div class="col-auto mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<div class="input-group input-group-sm w-auto d-none d-md-flex">
<label class="input-group-text border-0" for="managementPageSize" i18n>Show:</label>
<label class="input-group-text bg-transparent border-0" for="managementPageSize" i18n>Show:</label>
</div>
<div class="input-group input-group-sm w-auto me-3">
<select id="managementPageSize" class="form-select form-select-sm small" [(ngModel)]="pageSize">
@@ -113,7 +113,7 @@
<div class="btn-toolbar gap-2">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -58,7 +58,7 @@
<div class="col">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -146,7 +146,7 @@
<div class="col-3">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -47,7 +47,7 @@
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
+55
View File
@@ -54,6 +54,10 @@ export const ConfigCategory = {
AI: $localize`AI Settings`,
}
export const ConfigSection = {
RemoteOCR: $localize`Remote OCR`,
}
export const LLMEmbeddingBackendConfig = {
OPENAI_LIKE: 'openai-like',
HUGGINGFACE: 'huggingface',
@@ -65,6 +69,15 @@ export const LLMBackendConfig = {
OLLAMA: 'ollama',
}
export const RemoteOCREngineConfig = {
AZURE_AI: 'azureai',
}
export const RemoteOCRModeConfig = {
ALWAYS: 'always',
WORKFLOW_ONLY: 'workflow_only',
}
export interface ConfigOption {
key: string
title: string
@@ -72,6 +85,7 @@ export interface ConfigOption {
choices?: Array<{ id: string; name: string }>
config_key?: string
category: string
section?: string
note?: string
}
@@ -181,6 +195,43 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_OCR_USER_ARGS',
category: ConfigCategory.OCR,
},
{
key: 'remote_ocr_engine',
title: $localize`Remote OCR Engine`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCREngineConfig),
config_key: 'PAPERLESS_REMOTE_OCR_ENGINE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Enabling remote OCR sends documents to a third-party service for processing. Consider the privacy implications as well as potential costs before enabling.`,
},
{
key: 'remote_ocr_api_key',
title: $localize`Remote OCR API Key`,
type: ConfigOptionType.Password,
config_key: 'PAPERLESS_REMOTE_OCR_API_KEY',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
},
{
key: 'remote_ocr_endpoint',
title: $localize`Remote OCR Endpoint`,
type: ConfigOptionType.String,
config_key: 'PAPERLESS_REMOTE_OCR_ENDPOINT',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Required when using the Azure AI engine.`,
},
{
key: 'remote_ocr_mode',
title: $localize`Remote OCR Mode`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCRModeConfig),
config_key: 'PAPERLESS_REMOTE_OCR_MODE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Which documents are sent to the remote engine. Use 'workflow_only' to keep remote OCR off unless a workflow enables it for a document.`,
},
{
key: 'app_logo',
title: $localize`Application Logo`,
@@ -398,6 +449,10 @@ export interface PaperlessConfig extends ObjectWithId {
barcode_enable_tag: boolean
barcode_tag_mapping: object
barcode_tag_split: boolean
remote_ocr_engine: string
remote_ocr_api_key: string
remote_ocr_endpoint: string
remote_ocr_mode: string
ai_enabled: boolean
llm_embedding_backend: string
llm_embedding_model: string
+13
View File
@@ -1,5 +1,6 @@
import { PdfEditorEditMode } from '../components/common/pdf-editor/pdf-editor-edit-mode'
import { PdfZoomScale } from '../components/common/pdf-viewer/pdf-viewer.types'
import { RemoteOCRModeConfig } from './paperless-config'
import { User } from './user'
export interface UiSettings {
@@ -94,6 +95,8 @@ export const SETTINGS_KEYS = {
OUTLOOK_OAUTH_URL: 'outlook_oauth_url',
EMAIL_ENABLED: 'email_enabled',
AI_ENABLED: 'ai_enabled',
REMOTE_OCR_CONFIGURED: 'remote_ocr:configured',
REMOTE_OCR_MODE: 'remote_ocr:mode',
}
export const SETTINGS: UiSetting[] = [
@@ -347,4 +350,14 @@ export const SETTINGS: UiSetting[] = [
type: 'string',
default: PdfEditorEditMode.Create,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_CONFIGURED,
type: 'boolean',
default: false,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_MODE,
type: 'string',
default: RemoteOCRModeConfig.ALWAYS,
},
]
+1
View File
@@ -7,6 +7,7 @@ export enum WorkflowActionType {
Webhook = 4,
PasswordRemoval = 5,
MoveToTrash = 6,
RemoteOcr = 7,
}
export interface WorkflowActionEmail extends ObjectWithId {
@@ -284,6 +284,21 @@ describe(`DocumentService`, () => {
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: false,
})
})
it('should request remote OCR when reprocessing with it enabled', () => {
const ids = [1, 2, 3]
subscription = service
.reprocessDocuments({ documents: ids }, true)
.subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/reprocess/`
)
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: true,
})
})
@@ -349,9 +349,13 @@ export class DocumentService extends AbstractPaperlessService<Document> {
})
}
reprocessDocuments(selection: DocumentSelectionQuery) {
reprocessDocuments(
selection: DocumentSelectionQuery,
remoteOcr: boolean = false
) {
return this.http.post(this.getResourceUrl(null, 'reprocess'), {
...selection,
remote_ocr: remoteOcr,
})
}
@@ -13,6 +13,7 @@ import { environment } from 'src/environments/environment'
import { CustomFieldDataType } from '../data/custom-field'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { SavedView } from '../data/saved-view'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings'
import { PermissionsService } from './permissions.service'
import { CustomFieldsService } from './rest/custom-fields.service'
@@ -434,4 +435,26 @@ describe('SettingsService', () => {
).name
).toEqual(customFields[0].name)
})
it('should offer remote OCR only when configured and selective', () => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, false)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
// configured, but already handling every document
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, true)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.ALWAYS
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeTruthy()
})
})
@@ -19,6 +19,7 @@ import {
} from 'src/app/utils/color'
import { DEFAULT_APP_TITLE, environment } from 'src/environments/environment'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SavedView } from '../data/saved-view'
import {
PAPERLESS_GREEN_HEX,
@@ -687,6 +688,17 @@ export class SettingsService {
return this.settingIsSet(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)
}
/**
* Offering remote OCR as a choice only makes sense when an engine
* is configured but is not already handling every document.
*/
get remoteOCRIsSelectable(): boolean {
return (
this.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
this.get(SETTINGS_KEYS.REMOTE_OCR_MODE) !== RemoteOCRModeConfig.ALWAYS
)
}
offerTour(): boolean {
return this.dashboardIsEmpty() && !this.get(SETTINGS_KEYS.TOUR_COMPLETE)
}
@@ -438,6 +438,21 @@ describe('ConsumerStatusService', () => {
expect(updated).toBeTruthy()
})
it('should ignore keep-alive heartbeat messages from the server', () => {
let updated = false
let deleted = false
websocketStatusService.onDocumentUpdated().subscribe(() => (updated = true))
websocketStatusService.onDocumentDeleted().subscribe(() => (deleted = true))
websocketStatusService.connect()
server.send({ type: WebsocketStatusType.HEARTBEAT })
expect(updated).toBeFalsy()
expect(deleted).toBeFalsy()
expect(websocketStatusService.getConsumerStatus()).toHaveLength(0)
websocketStatusService.disconnect()
})
it('should ignore document updated events the user cannot view', () => {
let updated = false
websocketStatusService.onDocumentUpdated().subscribe(() => {
@@ -11,6 +11,7 @@ export enum WebsocketStatusType {
STATUS_UPDATE = 'status_update',
DOCUMENTS_DELETED = 'documents_deleted',
DOCUMENT_UPDATED = 'document_updated',
HEARTBEAT = 'heartbeat',
}
// see ProgressStatusOptions in src/documents/plugins/helpers.py
@@ -207,6 +208,10 @@ export class WebsocketStatusService {
case WebsocketStatusType.STATUS_UPDATE:
this.handleProgressUpdate(messageData as WebsocketProgressMessage)
break
case WebsocketStatusType.HEARTBEAT:
// keep-alive from the server, see paperless.consumers.StatusConsumer
break
}
}
}
+2
View File
@@ -118,6 +118,7 @@ import {
journalBookmarkFill,
journals,
link,
list,
listNested,
listTask,
listUl,
@@ -365,6 +366,7 @@ const icons = {
journalBookmarkFill,
journals,
link,
list,
listNested,
listTask,
listUl,
+38 -2
View File
@@ -1,7 +1,7 @@
@use "sass:meta";
// bs variables
$grid-gutter-width: 1.5rem;
$border-radius: .375rem;
$border-radius: .425rem;
$btn-border-width: var(--bs-border-width);
$form-file-button-bg: var(--bs-body-bg);
@@ -68,15 +68,44 @@ body {
--pngx-body-font-size: 0.875rem;
font-size: var(--pngx-body-font-size);
height: 100vh;
letter-spacing: -0.005em;
}
* {
transition: background-color 0.3s ease, border-color 0.3s ease;
}
.card,
.dropdown-menu,
.modal-content,
.popover {
--bs-card-border-color: color-mix(in srgb, var(--bs-border-color) 72%, transparent);
border-radius: .55rem;
}
.card {
box-shadow: 0 1px 2px rgba(0, 0, 0, .04), 0 8px 24px rgba(0, 0, 0, .035);
}
.btn {
--bs-btn-border-radius: .425rem;
--bs-border-radius-sm: .425rem;
font-weight: 500;
}
.form-control,
.form-select,
.input-group-text {
border-radius: .425rem;
}
.pagination, .input-group {
--bs-border-radius-sm: .425rem;
}
@media(min-width: 768px) {
.col-slim {
padding-left: calc(50px + $grid-gutter-width) !important;
padding-left: calc(56px + $grid-gutter-width) !important;
}
}
@@ -507,6 +536,7 @@ ul.pagination {
}
.nav-underline {
gap: calc(0.5 * var(--bs-nav-underline-gap)) var(--bs-nav-underline-gap);
.nav-link {
&.active, &:hover, &:focus {
color: var(--bs-primary);
@@ -660,6 +690,10 @@ table.table {
--bs-toast-max-width: var(--pngx-toast-max-width);
}
.alert {
--bs-border-radius: .425rem;
}
.alert-primary {
--bs-alert-color: var(--bs-primary);
--bs-alert-bg: var(--pngx-primary-faded);
@@ -791,6 +825,8 @@ code {
--bs-accordion-bg: var(--bs-light);
--bs-accordion-active-color: var(--bs-primary);
--bs-accordion-active-bg: var(--pngx-bg-alt);
--bs-border-radius: .425rem;
--bs-accordion-inner-border-radius: calc(.425rem - 1px);
}
.accordion-button::after {
+3 -3
View File
@@ -20,7 +20,7 @@
--pngx-primary-darken-27: hsl(var(--pngx-primary), calc(var(--pngx-primary-lightness) - 27%));
--pngx-success-darken-10: hsl(152, 69%, 11%); // based on success #198754
--pngx-bg-alt: #fff;
--pngx-bg-darker: var(--bs-gray-100);
--pngx-bg-darker: #f4f6f5;
--pngx-bg-alt2: var(--bs-gray-200); // #e9ecef
--pngx-bg-disabled: #f7f7f7;
--pngx-card-hover-border: var(--bs-tertiary-color);
@@ -91,7 +91,7 @@ $form-check-radio-checked-bg-image-dark: url("data:image/svg+xml,%3csvg xmlns='h
--pngx-body-color-accent: #{$text-color-dark-bg-accent};
--pngx-bg-alt: #242529;
--pngx-bg-alt2: #232323;
--pngx-bg-darker: #101216;
--pngx-bg-darker: #121315;
--pngx-bg-disabled: var(--pngx-bg-alt);
--pngx-card-hover-border: var(--bs-border-color);
--pngx-focus-alpha: 0.6;
@@ -107,7 +107,7 @@ $form-check-radio-checked-bg-image-dark: url("data:image/svg+xml,%3csvg xmlns='h
--bs-light-rgb: 28, 28, 31;
--bs-info: var(--pngx-bg-alt);
--bs-info-rgb: 36, 36, 39;
--bs-border-color: #47494f;
--bs-border-color: #34373d;
--bs-tertiary-bg: var(--pngx-bg-darker);
--bs-dark-border-subtle: var(--pngx-bg-darker);
--bs-border-color-translucent: rgba(0, 0, 0, .175); // override bs
+54 -31
View File
@@ -27,7 +27,7 @@ from documents.models import DocumentType
from documents.models import PaperlessTask
from documents.models import StoragePath
from documents.models import Tag
from documents.permissions import set_permissions_for_object
from documents.permissions import set_permissions_for_objects
from documents.plugins.helpers import DocumentsStatusManager
from documents.tasks import bulk_update_documents
from documents.tasks import consume_file
@@ -305,33 +305,49 @@ def modify_custom_fields(
else [(field, None) for field in add_custom_fields]
)
custom_fields = CustomField.objects.filter(
id__in=[int(field) for field, _ in add_custom_fields],
).distinct()
custom_fields_by_id: dict[int, CustomField] = {
cf.id: cf
for cf in CustomField.objects.filter(
id__in=[int(field) for field, _ in add_custom_fields],
)
}
# Deferred, not `.only()`: these objects get cached onto the FK
# descriptor of newly-created CustomFieldInstance rows below, and
# downstream post_save receivers (e.g. the filename-generation signal)
# touch other Document fields -- `.only("pk")` would just turn that into
# a deferred-field reload per document, trading one N+1 for another.
# `content` is the one field guaranteed to be both large (full OCR text)
# and unused by anything this function or its receivers touch.
docs_by_id: dict[int, Document] = {
doc.id: doc
for doc in Document.objects.filter(id__in=affected_docs).defer("content")
}
for field_id, value in add_custom_fields:
custom_field = custom_fields_by_id[field_id]
value_field = CustomFieldInstance.TYPE_TO_DATA_STORE_NAME_MAP[
custom_field.data_type
]
for doc_id in affected_docs:
defaults = {}
custom_field = custom_fields.get(id=field_id)
if custom_field:
value_field = CustomFieldInstance.TYPE_TO_DATA_STORE_NAME_MAP[
custom_field.data_type
]
defaults[value_field] = value
if (
custom_field.data_type == CustomField.FieldDataType.DOCUMENTLINK
and value
and doc_id in value
):
# Prevent self-linking
continue
defaults = {value_field: value}
if (
custom_field.data_type == CustomField.FieldDataType.DOCUMENTLINK
and value
and doc_id in value
):
# Prevent self-linking
continue
# Pass the already-resolved objects, not bare ids: this caches
# them on the FK descriptor of any newly-created instance, so a
# later `.field`/`.document` access (e.g. auditlog's post_save
# receiver calling `str(instance)`, which touches `.field.name`)
# doesn't trigger its own per-instance re-fetch.
CustomFieldInstance.objects.update_or_create(
document_id=doc_id,
field_id=field_id,
document=docs_by_id[doc_id],
field=custom_field,
defaults=defaults,
)
if custom_field.data_type == CustomField.FieldDataType.DOCUMENTLINK:
doc = Document.objects.get(id=doc_id)
reflect_doclinks(doc, custom_field, value)
reflect_doclinks(docs_by_id[doc_id], custom_field, value)
# For doc link fields that are being removed, remove symmetrical links
for doclink_being_removed_instance in CustomFieldInstance.objects.filter(
@@ -339,12 +355,10 @@ def modify_custom_fields(
field__id__in=remove_custom_fields,
field__data_type=CustomField.FieldDataType.DOCUMENTLINK,
value_document_ids__isnull=False,
):
).select_related("field"):
for target_doc_id in doclink_being_removed_instance.value:
remove_doclink(
document=Document.objects.get(
id=doclink_being_removed_instance.document.id,
),
document=docs_by_id[doclink_being_removed_instance.document_id],
field=doclink_being_removed_instance.field,
target_doc_id=target_doc_id,
)
@@ -399,10 +413,16 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
return "OK"
def reprocess(doc_ids: list[int]) -> Literal["OK"]:
def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
"""
Re-run parsing for the given documents.
Consumption workflows do not run here, so ``remote_ocr`` is how the user
asks for the remote engine when it is not configured to handle everything.
"""
for document_id in doc_ids:
update_document_content_maybe_archive_file.apply_async(
kwargs={"document_id": document_id},
kwargs={"document_id": document_id, "remote_ocr": remote_ocr},
headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
)
@@ -424,10 +444,13 @@ def set_permissions(
else:
qs.update(owner=owner)
for doc in qs:
set_permissions_for_object(permissions=set_permissions, object=doc, merge=merge)
affected_docs = list(qs.values_list("pk", flat=True))
set_permissions_for_objects(
permissions=set_permissions,
model=Document,
pks=affected_docs,
merge=merge,
)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
+18
View File
@@ -53,6 +53,7 @@ from documents.utils import copy_basic_file_stats
from documents.utils import copy_file_with_basic_stats
from documents.utils import run_subprocess
from paperless.config import OcrConfig
from paperless.config import RemoteOCRConfig
from paperless.models import ArchiveFileGenerationChoices
from paperless.parsers import ParserContext
from paperless.parsers import ParserProtocol
@@ -451,12 +452,19 @@ class ConsumerPlugin(
except Exception as e:
self.log.error(f"Error attempting to clean PDF: {e}")
# Workflows have already run at this point, so the metadata knows
# whether this document was singled out for remote OCR
allow_remote = (
self.metadata.remote_ocr or RemoteOCRConfig().remote_ocr_by_default
)
# Based on the mime type, get the parser for that type
parser_class: type[ParserProtocol] | None = (
get_parser_registry().get_parser_for_file(
mime_type,
self.filename,
self.working_copy,
allow_remote=allow_remote,
)
)
if not parser_class:
@@ -465,6 +473,16 @@ class ConsumerPlugin(
f"Unsupported mime type {mime_type}",
)
if self.metadata.remote_ocr and not getattr(
parser_class,
"uses_remote_service",
False,
):
self.log.warning(
"Remote OCR was requested for this document but no remote "
"parser is available for it, processing locally instead.",
)
# Notify all listeners that we're going to do some work.
document_consumption_started.send(
+4 -1
View File
@@ -34,6 +34,7 @@ class DocumentMetadataOverrides:
skip_asn_if_exists: bool = False
version_label: str | None = None
actor_id: int | None = None
remote_ocr: bool = False
def update(self, other: "DocumentMetadataOverrides") -> "DocumentMetadataOverrides":
"""
@@ -57,6 +58,8 @@ class DocumentMetadataOverrides:
self.actor_id = other.actor_id
if other.skip_asn_if_exists:
self.skip_asn_if_exists = True
if other.remote_ocr:
self.remote_ocr = True
if other.version_label is not None:
self.version_label = other.version_label
@@ -126,7 +129,7 @@ class DocumentMetadataOverrides:
)
overrides.custom_fields = {
custom_field.field.id: custom_field.value
for custom_field in doc.custom_fields.all()
for custom_field in doc.custom_fields.select_related("field").all()
}
groups_with_perms = get_groups_with_perms(
@@ -0,0 +1,30 @@
# Generated by Django 5.2.16 on 2026-08-10 17:27
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0023_savedview_icon"),
]
operations = [
migrations.AlterField(
model_name="workflowaction",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Assignment"),
(2, "Removal"),
(3, "Email"),
(4, "Webhook"),
(5, "Password removal"),
(6, "Move to trash"),
(7, "Remote OCR"),
],
default=1,
verbose_name="Workflow Action Type",
),
),
]
+4
View File
@@ -1670,6 +1670,10 @@ class WorkflowAction(models.Model):
6,
_("Move to trash"),
)
REMOTE_OCR = (
7,
_("Remote OCR"),
)
type = models.PositiveSmallIntegerField(
_("Workflow Action Type"),
+176
View File
@@ -173,6 +173,182 @@ def set_permissions_for_object(
)
def _resolve_permissions(codenames: set[str], ctype: ContentType) -> list[Permission]:
"""
Resolves `codenames` to Permission rows, raising like the single-object
assign_perm() this bulk path replaces does (via a `.get()` internally)
if any codename doesn't exist -- e.g. a client-supplied action name that
was never validated (BulkEditObjectsSerializer._validate_permissions
calls validate_set_permissions() only for its side-effecting id checks
and discards the filtered dict it returns, so an unrecognized action key
reaches this function as-is). A plain `.filter()` with no existence
check would otherwise silently build zero rows and no-op instead of
reporting the bad input.
"""
permission_objs = list(
Permission.objects.filter(content_type=ctype, codename__in=codenames),
)
missing = codenames - {p.codename for p in permission_objs}
if missing:
raise Permission.DoesNotExist(
f"Permission matching query does not exist for codename(s): "
f"{', '.join(sorted(missing))}",
)
return permission_objs
# Target number of permission rows to build in Python before handing them to
# bulk_create -- keeps peak memory bounded for a large "apply to all" call,
# independent of bulk_create's own batch_size (which only caps the size of
# each INSERT statement, not how many row objects exist in memory at once).
_PERMISSION_ROW_CHUNK_SIZE = 5000
def _apply_bulk_permission_entry(
*,
perm_model: type[UserObjectPermission] | type[GroupObjectPermission],
identity_model: type[User] | type[Group],
identity_field: str,
ids: list[int],
codename: str,
permission_objs: list[Permission],
ctype: ContentType,
object_pks: list[str],
merge: bool,
) -> None:
# Only the ids are needed to build permission rows (via `<field>_id=`),
# so avoid fetching full User/Group rows for identities that may not
# even end up being granted anything new.
add_ids = set(
identity_model.objects.filter(id__in=ids).values_list("id", flat=True),
)
if not merge:
existing_ids = set(
perm_model.objects.filter(
content_type=ctype,
object_pk__in=object_pks,
permission__codename=codename,
).values_list(f"{identity_field}_id", flat=True),
)
remove_ids = existing_ids - add_ids
if remove_ids:
perm_model.objects.filter(
content_type=ctype,
object_pk__in=object_pks,
permission__codename=codename,
**{f"{identity_field}_id__in": remove_ids},
).delete()
if not add_ids:
return
rows_per_pk = len(permission_objs) * len(add_ids)
pks_per_chunk = max(1, _PERMISSION_ROW_CHUNK_SIZE // rows_per_pk)
for start in range(0, len(object_pks), pks_per_chunk):
pk_chunk = object_pks[start : start + pks_per_chunk]
rows = [
perm_model(
content_type=ctype,
object_pk=pk,
permission=permission_obj,
**{f"{identity_field}_id": identity_id},
)
for permission_obj in permission_objs
for pk in pk_chunk
for identity_id in add_ids
]
# ignore_conflicts skips only rows that already exist as an exact
# (identity, permission, object) match -- the same de-dup the
# underlying (user|group, permission, object_pk) unique constraint
# already enforces for the single-object assign_perm() this
# replaces, so it doesn't change what counts as "already granted".
# batch_size caps how many rows go into a single INSERT so a huge
# chunk doesn't build one enormous statement.
perm_model.objects.bulk_create(rows, ignore_conflicts=True, batch_size=1000)
def set_permissions_for_objects(
permissions: dict,
model: type[Model],
pks: QuerySet | list,
*,
merge: bool = False,
) -> None:
"""
Bulk equivalent of set_permissions_for_object: applies the same
permission changes to every object identified by `pks` at once.
Takes a model + pks (rather than model instances) deliberately -- the
permission rows built below only ever need `pk`, `content_type`, and
identity ids, so callers shouldn't have to fetch full rows (with every
other field) just to hand them to this function.
Deliberately does not use guardian's queryset/list-aware assign_perm:
passing a list as the object routes to bulk_assign_perm, which skips
creating a direct permission row for anyone who already has the
permission via ANY group membership (it checks
ObjectPermissionChecker.has_perm, which is group-inheritance-aware) --
unlike the single-object assign_perm this replaces, which always
ensures a direct row via get_or_create regardless of group-derived
access. Losing that guarantee would mean a later revocation of the
group's grant silently strips access an admin explicitly asked to be
direct. Bulk-creating rows straight against the permission models
instead (see _apply_bulk_permission_entry) preserves the original
always-create-a-direct-row semantics while still batching every object
and every identity into one query per action, rather than one query per
(object, user) pair.
"""
object_pks = [str(pk) for pk in pks]
if not object_pks:
return
model_name = model.__name__.lower()
ctype = ContentType.objects.get_for_model(model)
for action, entry in permissions.items():
codename = f"{action}_{model_name}"
implied_codenames = {codename}
if action == "change":
# change gives view too
implied_codenames.add(f"view_{model_name}")
# Resolved once per action (not once per users/groups branch) and
# shared between both below -- also where an unrecognized action
# name (see _resolve_permissions) is caught.
permission_objs = (
_resolve_permissions(implied_codenames, ctype)
if "users" in entry or "groups" in entry
else []
)
if "users" in entry:
_apply_bulk_permission_entry(
perm_model=UserObjectPermission,
identity_model=User,
identity_field="user",
ids=entry["users"],
codename=codename,
permission_objs=permission_objs,
ctype=ctype,
object_pks=object_pks,
merge=merge,
)
if "groups" in entry:
_apply_bulk_permission_entry(
perm_model=GroupObjectPermission,
identity_model=Group,
identity_field="group",
ids=entry["groups"],
codename=codename,
permission_objs=permission_objs,
ctype=ctype,
object_pks=object_pks,
merge=merge,
)
def permitted_object_ids(
user: User | None,
model: type[Model],
+2 -10
View File
@@ -6,20 +6,13 @@ from documents.search._backend import TantivyRelevanceList
from documents.search._backend import WriteBatch
from documents.search._backend import get_backend
from documents.search._backend import reset_backend
from documents.search._errors import InvalidDateQuery
from documents.search._errors import InvalidNumberQuery
from documents.search._errors import MultipleSearchQueryErrors
from documents.search._errors import QueryTooLongError
from documents.search._errors import SearchQueryError
from documents.search._errors import search_query_error_messages
from documents.search._schema import needs_rebuild
from documents.search._schema import wipe_index
from documents.search._translate import InvalidDateQuery
from documents.search._translate import SearchQueryError
__all__ = [
"InvalidDateQuery",
"InvalidNumberQuery",
"MultipleSearchQueryErrors",
"QueryTooLongError",
"SearchHit",
"SearchIndexLockError",
"SearchMode",
@@ -30,6 +23,5 @@ __all__ = [
"get_backend",
"needs_rebuild",
"reset_backend",
"search_query_error_messages",
"wipe_index",
]
+22 -84
View File
@@ -20,11 +20,9 @@ from typing import cast
import filelock
import tantivy
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.utils.timezone import get_current_timezone
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
from documents.search._query import build_permission_filter
from documents.search._query import extract_cjk_text
from documents.search._query import parse_simple_text_highlight_query
from documents.search._query import parse_simple_text_query
@@ -42,7 +40,6 @@ from documents.utils import QuerySetStream
from documents.utils import identity
if TYPE_CHECKING:
from collections.abc import Iterable
from collections.abc import Iterator
from collections.abc import Sequence
from pathlib import Path
@@ -266,11 +263,7 @@ class WriteBatch:
if self._lock is not None:
self._lock.release()
def add_or_update(
self,
document: Document,
effective_content: str | None = None,
) -> None:
def add_or_update(self, document: Document) -> None:
"""
Add or update a document in the batch.
@@ -280,11 +273,9 @@ class WriteBatch:
Args:
document: Django Document instance to index
effective_content: Override document.content for indexing (used when
re-indexing with newer OCR text from document versions)
"""
self.remove(document.pk)
doc = self._backend._build_tantivy_doc(document, effective_content)
doc = self._backend._build_tantivy_doc(document)
self._writer.add_document(doc)
def remove(self, doc_id: int) -> None:
@@ -294,47 +285,6 @@ class WriteBatch:
)
def build_permission_filter(
schema: tantivy.Schema,
user: AbstractUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
Creates a query that matches only documents visible to the specified user
according to paperless-ngx permission rules:
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
"""
owner_any = tantivy.Query.exists_query("owner_id")
no_owner = tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, tantivy.Query.all_query()),
(tantivy.Occur.MustNot, owner_any),
],
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
class TantivyBackend:
"""
Tantivy search backend with explicit lifecycle management.
@@ -466,18 +416,20 @@ class TantivyBackend:
def _build_tantivy_doc(
self,
document: Document,
effective_content: str | None = None,
viewer_ids: list[int] | None = None,
viewer_group_ids: list[int] | None = None,
) -> tantivy.Document:
"""Build a tantivy Document from a Django Document instance.
``effective_content`` overrides ``document.content`` for indexing
used when re-indexing a root document with a newer version's OCR text.
A root document is indexed with its effective content, i.e. the newest
version's OCR text, so it is never indexed with its own outdated text.
Annotate the queryset with ``annotate_effective_content`` when indexing
more than a couple of documents, to resolve that without a query each.
"""
content = (
effective_content if effective_content is not None else document.content
)
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
content = document.get_effective_content() or ""
doc = tantivy.Document()
@@ -506,6 +458,7 @@ class TantivyBackend:
doc.add_text("correspondent_sort", document.correspondent.name)
if cjk_corr := extract_cjk_text(document.correspondent.name):
doc.add_text("bigram_correspondent", cjk_corr)
doc.add_unsigned("correspondent_id", document.correspondent_id)
# Document type
if document.document_type:
@@ -513,10 +466,12 @@ class TantivyBackend:
doc.add_text("type_sort", document.document_type.name)
if cjk_type := extract_cjk_text(document.document_type.name):
doc.add_text("bigram_document_type", cjk_type)
doc.add_unsigned("document_type_id", document.document_type_id)
# Storage path
if document.storage_path:
doc.add_text("storage_path", document.storage_path.name)
doc.add_unsigned("storage_path_id", document.storage_path_id)
# Tags — collect names for autocomplete in the same pass
tag_names: list[str] = []
@@ -524,13 +479,12 @@ class TantivyBackend:
doc.add_text("tag", tag.name)
if cjk_tag := extract_cjk_text(tag.name):
doc.add_text("bigram_tag", cjk_tag)
doc.add_unsigned("tag_id", tag.pk)
tag_names.append(tag.name)
# Notes — JSON for structured queries (notes.user:alice, notes.note:text).
# notes_text is a plain-text companion for snippet/highlight generation;
# tantivy's SnippetGenerator does not support JSON fields. It is not in
# _DEFAULT_SEARCH_FIELDS, so an unqualified query never searches it: a
# note matches through the JSON field or not at all.
# tantivy's SnippetGenerator does not support JSON fields.
num_notes = 0
note_texts: list[str] = []
for note in document.notes.all():
@@ -546,9 +500,8 @@ class TantivyBackend:
if note_texts:
doc.add_text("notes_text", " ".join(note_texts))
# Custom fields — JSON for structured queries (custom_fields.name:x,
# custom_fields.value:y). There is no companion text field here, unlike
# notes: custom field values are reachable only through the JSON field.
# Custom fields — JSON for structured queries (custom_fields.name:x, custom_fields.value:y),
# companion text field for default full-text search.
for cfi in document.custom_fields.all():
search_value = cfi.value_for_search
# Skip fields where there is no value yet
@@ -624,11 +577,7 @@ class TantivyBackend:
return doc
def add_or_update(
self,
document: Document,
effective_content: str | None = None,
) -> None:
def add_or_update(self, document: Document) -> None:
"""
Add or update a single document with file locking.
@@ -641,12 +590,11 @@ class TantivyBackend:
Args:
document: Django Document instance to index
effective_content: Override document.content for indexing
"""
self._ensure_open()
try:
with self.batch_update(lock_timeout=_LOCK_TIMEOUT_SECONDS) as batch:
batch.add_or_update(document, effective_content)
batch.add_or_update(document)
except SearchIndexLockError:
logger.error(
"Search index lock exhausted for document %d after %d attempts; "
@@ -720,17 +668,7 @@ class TantivyBackend:
user_query = self._parse_query(query, search_mode)
highlight_query = user_query
if search_mode is SearchMode.TEXT:
try:
highlight_query = parse_simple_text_highlight_query(
self._index,
query,
)
except ValueError:
logger.debug(
"Skipping simple text highlight query: token string is not "
"valid tantivy query syntax: %r",
query,
)
highlight_query = parse_simple_text_highlight_query(self._index, query)
# For notes_text snippet generation, we need a query that targets the
# notes_text field directly. user_query may contain JSON-field terms
@@ -1077,7 +1015,6 @@ class TantivyBackend:
):
doc = self._build_tantivy_doc(
document,
document.get_effective_content(),
viewer_ids=viewer_ids,
viewer_group_ids=viewer_group_ids,
)
@@ -1145,6 +1082,7 @@ def _bulk_get_viewer_permissions(
"""
from collections import defaultdict
from django.contrib.contenttypes.models import ContentType
from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission
+171
View File
@@ -0,0 +1,171 @@
from __future__ import annotations
from datetime import UTC
from datetime import date
from datetime import datetime
from datetime import timedelta
from typing import TYPE_CHECKING
from typing import Final
from dateutil.relativedelta import relativedelta
if TYPE_CHECKING:
from datetime import tzinfo
_DATE_ONLY_FIELDS = frozenset({"created"})
_TODAY: Final[str] = "today"
_YESTERDAY: Final[str] = "yesterday"
_PREVIOUS_WEEK: Final[str] = "previous week"
_THIS_MONTH: Final[str] = "this month"
_PREVIOUS_MONTH: Final[str] = "previous month"
_THIS_YEAR: Final[str] = "this year"
_PREVIOUS_YEAR: Final[str] = "previous year"
_PREVIOUS_QUARTER: Final[str] = "previous quarter"
_DATE_KEYWORDS = frozenset(
{
_TODAY,
_YESTERDAY,
_PREVIOUS_WEEK,
_THIS_MONTH,
_PREVIOUS_MONTH,
_THIS_YEAR,
_PREVIOUS_YEAR,
_PREVIOUS_QUARTER,
},
)
def _fmt(dt: datetime) -> str:
"""Format a datetime as an ISO 8601 UTC string for use in Tantivy range queries."""
return dt.astimezone(UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
def _iso_range(lo: datetime, hi: datetime) -> str:
"""
Format a half-open ``[lo TO hi)`` range in ISO 8601 for Tantivy query syntax.
``hi`` is always the exclusive ceiling of a computed period (the start of
the *next* day/week/month/quarter/year), so the closing bracket must be
the Tantivy exclusive-range brace ``}`` rather than ``]`` otherwise the
first instant of the following period (e.g. the 1st of next month) is
incorrectly included in the match.
"""
return f"[{_fmt(lo)} TO {_fmt(hi)}}}"
def _quarter_start(d: date) -> date:
"""Return the first day of the calendar quarter containing ``d``."""
return date(d.year, ((d.month - 1) // 3) * 3 + 1, 1)
def _midnight(d: date, tz: tzinfo) -> datetime:
"""Convert a calendar date at local-timezone midnight to a UTC datetime."""
return datetime(d.year, d.month, d.day, tzinfo=tz).astimezone(UTC)
def _keyword_bounds(keyword: str, tz: tzinfo) -> tuple[date, date]:
"""
Map a relative date keyword to ``(start, exclusive_end)`` calendar dates.
``tz`` only determines what "today" is; the caller decides how the returned
dates become UTC datetime boundaries (date-only vs. local-midnight offset).
"""
today = datetime.now(tz).date()
if keyword == _TODAY:
return today, today + timedelta(days=1)
if keyword == _YESTERDAY:
return today - timedelta(days=1), today
if keyword == _PREVIOUS_WEEK:
this_monday = today - timedelta(days=today.weekday())
return this_monday - timedelta(weeks=1), this_monday
if keyword == _THIS_MONTH:
first = today.replace(day=1)
return first, first + relativedelta(months=1)
if keyword == _PREVIOUS_MONTH:
this_first = today.replace(day=1)
return this_first - relativedelta(months=1), this_first
if keyword == _THIS_YEAR:
return date(today.year, 1, 1), date(today.year + 1, 1, 1)
if keyword == _PREVIOUS_YEAR:
return date(today.year - 1, 1, 1), date(today.year, 1, 1)
if keyword == _PREVIOUS_QUARTER:
this_quarter = _quarter_start(today)
return this_quarter - relativedelta(months=3), this_quarter
raise ValueError(f"Unknown keyword: {keyword}")
def _date_only_range(keyword: str, tz: tzinfo) -> str:
"""
For `created` (DateField): use the local calendar date, converted to
midnight UTC boundaries. No offset arithmetic date only.
"""
start, end = _keyword_bounds(keyword, tz)
lo = datetime(start.year, start.month, start.day, tzinfo=UTC)
hi = datetime(end.year, end.month, end.day, tzinfo=UTC)
return _iso_range(lo, hi)
def _datetime_range(keyword: str, tz: tzinfo) -> str:
"""
For `added` / `modified` (DateTimeField, stored as UTC): convert local day
boundaries to UTC full offset arithmetic required.
"""
start, end = _keyword_bounds(keyword, tz)
return _iso_range(_midnight(start, tz), _midnight(end, tz))
def _precision_bounds(digits: str) -> tuple[date, date] | None:
"""
Map a 4/6/8-digit date token to (start, exclusive_end) calendar dates.
YYYY -> whole year, YYYYMM -> whole month, YYYYMMDD -> single day.
Returns None for any unparsable or out-of-range value (e.g. month 23),
so callers can emit a no-match clause instead of erroring (Whoosh parity).
"""
try:
if len(digits) == 4:
year = int(digits)
return date(year, 1, 1), date(year + 1, 1, 1)
if len(digits) == 6:
year, month = int(digits[:4]), int(digits[4:6])
start = date(year, month, 1)
end = date(year + 1, 1, 1) if month == 12 else date(year, month + 1, 1)
return start, end
if len(digits) == 8:
start = date(int(digits[:4]), int(digits[4:6]), int(digits[6:8]))
return start, start + timedelta(days=1)
except ValueError:
return None
return None
def _utc_bounds_for_field(
field: str,
start: date,
end: date,
tz: tzinfo,
) -> tuple[datetime, datetime]:
"""
Convert calendar-date bounds to UTC datetimes per the field's storage type.
For DateField (``created``) the bounds are UTC midnight (no offset). For
DateTimeField (``added``/``modified``) the bounds are local-tz midnight
converted to UTC, matching how each field is indexed.
"""
if field in _DATE_ONLY_FIELDS:
return (
datetime(start.year, start.month, start.day, tzinfo=UTC),
datetime(end.year, end.month, end.day, tzinfo=UTC),
)
return (
datetime(start.year, start.month, start.day, tzinfo=tz).astimezone(UTC),
datetime(end.year, end.month, end.day, tzinfo=tz).astimezone(UTC),
)
def _field_range_from_dates(field: str, start: date, end: date, tz: tzinfo) -> str:
"""Build a Tantivy ``field:[lo TO hi]`` ISO range from calendar-date bounds."""
lo, hi = _utc_bounds_for_field(field, start, end, tz)
return f"{field}:{_iso_range(lo, hi)}"
-71
View File
@@ -1,71 +0,0 @@
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from collections.abc import Sequence
class SearchQueryError(ValueError):
"""
Base for user-fixable search query errors.
Carries a message safe to surface to the user (no internal details). The
view layer catches this and returns an HTTP 400, so any future subclass
gets the same treatment.
"""
class InvalidDateQuery(SearchQueryError):
"""Raised when a date field value or range bound cannot be parsed."""
def __init__(self, field: str | None, value: str | None) -> None:
self.field = field
self.value = value
super().__init__(f"Invalid date value {value!r} for field {field!r}.")
class InvalidNumberQuery(SearchQueryError):
"""Raised when a numeric field value or range bound cannot be parsed."""
def __init__(self, field: str | None, value: str | None) -> None:
self.field = field
self.value = value
super().__init__(f"Invalid numeric value {value!r} for field {field!r}.")
class QueryTooLongError(SearchQueryError):
"""Raised when a query string exceeds the maximum allowed length.
whoosh-compat's fieldname tagger is O(n^2) in plain word characters, so an
unbounded query is a CPU-exhaustion vector against a single request
handler. This is a hard boundary, not a validation nicety.
"""
def __init__(self, length: int, limit: int) -> None:
self.length = length
self.limit = limit
super().__init__(
f"The search query is too long ({length} characters). "
f"The maximum allowed length is {limit} characters.",
)
class MultipleSearchQueryErrors(SearchQueryError):
"""Aggregates every user-fixable error from one parse, not just the first."""
def __init__(self, errors: Sequence[SearchQueryError]) -> None:
self.errors = tuple(errors)
super().__init__("; ".join(str(e) for e in self.errors))
def search_query_error_messages(e: SearchQueryError) -> list[str]:
"""The user-facing message list for a SearchQueryError.
Every offending value's message, not just the first, so the user can
fix them all in one round-trip. Shared by every view that maps
SearchQueryError to an HTTP 400.
"""
if isinstance(e, MultipleSearchQueryErrors):
return [str(sub) for sub in e.errors]
return [str(e)]
-42
View File
@@ -1,42 +0,0 @@
from __future__ import annotations
from whoosh_compat import FieldKind
from whoosh_compat import FieldSpec
from whoosh_compat import SubpathSpec
# Internal-only schema fields with no query-syntax meaning of their own
# (sort shadow fields, bigram CJK fields, simple_title/simple_content,
# autocomplete_word, notes_text) are NOT represented here — they are
# declared in _schema.py's field_descriptors().
#
# analyzer/pattern_normalizer are deliberately left at FieldSpec's default
# (None): they're language-specific and only meaningful to whoosh-compat's
# parser, so _registry.py attaches them per-language via dataclasses.replace()
# rather than PUBLIC_FIELDS declaring them itself. _schema.py only reads
# name/kind/fast and never sees the analyzer at all.
PUBLIC_FIELDS: tuple[FieldSpec, ...] = (
FieldSpec("title", FieldKind.TEXT),
FieldSpec("content", FieldKind.TEXT),
FieldSpec("correspondent", FieldKind.TEXT),
FieldSpec("document_type", FieldKind.TEXT, aliases=("type",)),
FieldSpec("storage_path", FieldKind.TEXT, aliases=("path",)),
FieldSpec("original_filename", FieldKind.TEXT),
FieldSpec("tag", FieldKind.TEXT, comma_values=True),
FieldSpec("checksum", FieldKind.KEYWORD),
FieldSpec("asn", FieldKind.U64, fast=True),
FieldSpec("page_count", FieldKind.U64, fast=True),
FieldSpec("num_notes", FieldKind.U64, fast=True),
FieldSpec("created", FieldKind.DATE, date_only=True, fast=True),
FieldSpec("modified", FieldKind.DATETIME, fast=True),
FieldSpec("added", FieldKind.DATETIME, fast=True),
FieldSpec(
"notes",
FieldKind.JSON,
subpaths={"user": SubpathSpec(), "note": SubpathSpec(default=True)},
),
FieldSpec(
"custom_fields",
FieldKind.JSON,
subpaths={"name": SubpathSpec(), "value": SubpathSpec(default=True)},
),
)

Some files were not shown because too many files have changed in this diff Show More