Compare commits

..
Author SHA1 Message Date
stumpylog 5865e96a06 Handles a bad client sending malformed JSON or non-int primary keys 2026-08-25 14:26:33 -07:00
Trenton Holmes 172af8362d Perf: avoid per-instance CustomField reload in DocumentMetadataOverrides
send_websocket_document_updated calls document.refresh_from_db()
before building overrides, which drops the custom_fields prefetch
(and its select_related("field")) set up by the view's queryset.
DocumentMetadataOverrides.from_document() then lazily reloads field
once per custom field instance. Since from_document() can't rely on
the caller having a prefetched document, select_related explicitly at
the point of use instead.
2026-08-25 14:26:33 -07:00
Trenton Holmes e46163bfdf Perf: reuse resolved CustomField objects across drf-writable-nested's per-item revalidation
drf-writable-nested's update_or_create_reverse_relations rebuilds a
fresh serializer -- and fresh field instances -- per custom_fields item
while matching existing vs. new instances during save(), so the
per-instance lookup cache alone only helped the first validation pass.
It passes the same context dict (by reference) to every one of those
serializers, so stash resolved CustomField objects there instead:
later passes reuse them for free rather than re-querying.
2026-08-25 14:26:33 -07:00
Trenton Holmes f8e5aea8dd Perf: batch CustomField lookups when validating a document's custom_fields
DocumentSerializer.custom_fields validates each item's field id via a
plain PrimaryKeyRelatedField, which issues one SELECT per custom field
per validation pass (discussion #13690). Batch-resolve all field ids in
one query and cache them on the field instance so per-item validation
is free instead of re-querying.
2026-08-25 14:26:33 -07:00
shamoon e24db7023a Tweak: adjust brand leaf opacity 2026-08-25 10:44:49 -07:00
shamoon f5ddc14588 Fix: also check global change_mailaccount with test 2026-08-25 10:44:49 -07:00
GitHub Actions 492424f7f2 Auto translate strings 2026-08-25 00:07:03 +00:00
shamoonandGitHub cd525819d7 Tweak: more misc UI tweaks (#13783) 2026-08-24 17:05:29 -07:00
GitHub Actions 2609327e9c Auto translate strings 2026-08-24 21:44:25 +00:00
shamoonandGitHub b90ccf910f Finally, the remote ocr workflow (#13637)
* Ok! Backend stuff for the remote ocr workflow

* Frotnend workflow stuff

* And docs

* Fix dynamic action fields thing

* Actually, fix the action dropdown thing

* Fix this validation thing, and we have to check existing actions

* Fix migration
2026-08-24 14:43:05 -07:00
shamoonandGitHub c93c996edf Remote ocr reprocess (#13636)
* Backend stuff for remote ocr reprocess, add to bulk edit pass in from ui settings

* Ok, frontend reprocess remote option

* Docs
2026-08-24 14:43:05 -07:00
shamoonandGitHub 7f1609332a Allow parsers to declare uses remote, and remote ocr_mode (#13634)
* uses_remote_service + allow_remote to allow opt-in / out of remote OCR

* Add to parser dev docs

* remote_ocr_mode config setting

* Checks for remote_ocr_mode and fix import

* Update config.component.spec.ts

* More tests for remote_ocr_mode

* Docs for remote_ocr_mode

* Ok, wire up the remote_ocr_mode with allow_remote for consumer

* Update consumer.py

* Format remote OCR mode check tests

* Use get_choice_from_env
2026-08-24 14:43:04 -07:00
GitHub Actions a1f20c9fe7 Auto translate strings 2026-08-24 21:19:19 +00:00
shamoonandGitHub 4fd1c60731 Enhancement: support using remote OCR engines selectively (#13633)
* Backend changes and migration for remote OCR Config

* Backend tests

* Frontend stuff, with sections

* Docs

* Update test_tesseract_parser.py

* Actually we cant use this any more, in case settings are in app config

* Dont mark entire test file for db, use a mock for empty engine settings
2026-08-24 14:17:52 -07:00
GitHub Actions ba83e5b39a Auto translate strings 2026-08-24 18:28:51 +00:00
shamoonandGitHub 1497bd33a1 Fix: fix bottom mobile nav buttons on Android (#13780) 2026-08-24 11:27:26 -07:00
shamoon 01c40aa0bd Tweak: subtler shadow, tweak against light color 2026-08-24 08:51:51 -07:00
shamoon 5f70b6eee7 Fix navbar button alignment + hover consistency 2026-08-24 08:47:27 -07:00
GitHub Actions e3e4b26944 Auto translate strings 2026-08-24 15:16:51 +00:00
a0908f6b4a Enhancement: websocket heartbeat (#13739)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-08-24 15:15:21 +00:00
bab9129ff8 Fix: lazy import guardian modules to fix search language setting (#13768)
Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2026-08-24 13:46:20 +00:00
GitHub Actions 1e13f86174 Auto translate strings 2026-08-24 09:13:06 +00:00
shamoonandGitHub 22c57fb392 Enhancement: more v3 ui tweaks (#13774) 2026-08-24 02:11:34 -07:00
GitHub Actions 08a7f6ccc0 Auto translate strings 2026-08-24 04:56:30 +00:00
shamoonandGitHub 3f4d327f64 Chore: add some missing UI accessibility labels (#13772) 2026-08-23 21:55:07 -07:00
GitHub Actions bd326540fa Auto translate strings 2026-08-24 02:43:34 +00:00
shamoonandGitHub 33f9adb05a Chore: refactor permission checkbox live changes (#13771) 2026-08-23 19:42:18 -07:00
shamoonandGitHub d24606a03b Tweak: better support long list of views in documents list (#13769) 2026-08-23 16:40:55 -07:00
shamoonandGitHub 294328f174 Fix: version indexing fixes (#13737) 2026-08-23 23:04:47 +00:00
shamoonandGitHub 0458bad5f2 Fix: append charset to file response for text files (#13759) 2026-08-22 06:15:53 -07:00
shamoon a510d03c77 Merge branch 'main' into dev 2026-08-22 05:27:10 -07:00
shamoonandGitHub d7b3612a41 Chore: pin Apache Tika images to 3.3.1 (#13758) 2026-08-22 05:25:37 -07:00
shamoonandGitHub 7e4a644714 Fix: align bulk edit perms with document model (#13757) 2026-08-22 05:24:16 -07:00
GitHub Actions bbcd6af2fe Auto translate strings 2026-08-21 15:25:24 +00:00
shamoonandGitHub 0431939f18 QoL: add count badge to versions dropdown (#13753) 2026-08-21 08:23:47 -07:00
shamoonandGitHub bed95ea301 Tweakhancement: add jitter to IMAP polling schedule (#13734) 2026-08-20 17:19:47 +00:00
GitHub Actions 42034c3c77 Auto translate strings 2026-08-20 04:52:42 +00:00
shamoonandGitHub 705220fb5a Fix: use selected version for doc detail emailing (#13738) 2026-08-19 21:51:13 -07:00
shamoonandGitHub 2d084983c8 Documentation: clarify OCR mode changes in v3 (#13666) 2026-08-12 21:46:52 -07:00
shamoonandGitHub e2c284f64e Documentation: add wiki links for AI stuff and parser plugins (#13626) 2026-08-09 19:05:32 -07:00
179 changed files with 6650 additions and 6477 deletions
@@ -68,7 +68,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
-3
View File
@@ -115,6 +115,3 @@ celerybeat-schedule*
# Git worktree local folder
.worktrees
# Agent workflow scratch (ledgers, briefs, review packages)
.superpowers/
@@ -81,7 +81,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
@@ -76,7 +76,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
@@ -65,7 +65,7 @@ services:
- "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*"
tika:
image: docker.io/apache/tika:latest
image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped
volumes:
data:
+1 -2
View File
@@ -521,8 +521,7 @@ Pass `--recreate` to wipe the existing index before rebuilding. Use this when th
index is corrupted or you want a fully clean rebuild.
Pass `--if-needed` to skip the rebuild if the index is already up to date (schema
version, schema fingerprint and search language all match). Safe to run on every
startup or upgrade.
version and search language match). Safe to run on every startup or upgrade.
Specify `optimize` to optimize the index. This command is regularly invoked by the
task scheduler.
+6 -1
View File
@@ -129,6 +129,10 @@ At a minimum you need to enable AI and choose an LLM backend:
and/or [`PAPERLESS_AI_LLM_ENDPOINT`](configuration.md#PAPERLESS_AI_LLM_ENDPOINT). Ollama
requires `PAPERLESS_AI_LLM_ENDPOINT` pointing at your Ollama server.
See the community-maintained wiki page on
[choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for suggested generation and embedding models.
### AI-assisted suggestions
With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type,
@@ -808,7 +812,8 @@ Third-party parser plugins extend Paperless-ngx to support additional file
formats. A plugin is a Python package that advertises itself under the
`paperless_ngx.parsers` entry point group. Refer to the
[developer documentation](development.md#making-custom-parsers) for how to
create one.
create one, or see the wiki for a community-maintained list of
[parser plugins](https://github.com/paperless-ngx/paperless-ngx/wiki/Related-Projects#parser-plugins).
!!! warning "Third-party plugins are not officially supported"
+2 -1
View File
@@ -302,7 +302,8 @@ The following methods are supported:
- `delete`
- No `parameters` required
- `reprocess`
- No `parameters` required
- Optional `parameters`: `{ "remote_ocr": true }` to send the documents to the
remote OCR engine, see [Remote OCR](usage.md#remote-ocr). Defaults to false.
- `set_permissions`
- Requires `parameters`:
- `"set_permissions": PERMISSIONS_OBJ` (see format [above](#permissions)) and / or
+17 -1
View File
@@ -1215,7 +1215,7 @@ should be a valid crontab(5) expression describing when to run.
: If set to the string "disable", no emails will be fetched automatically.
Defaults to `*/10 * * * *` or every ten minutes.
Defaults to every ten minutes, with an installation-specific minute offset.
#### [`PAPERLESS_TRAIN_TASK_CRON=<cron expression>`](#PAPERLESS_TRAIN_TASK_CRON) {#PAPERLESS_TRAIN_TASK_CRON}
@@ -2066,6 +2066,18 @@ password. All of these options come from their similarly-named [Django settings]
Defaults to None.
#### [`PAPERLESS_REMOTE_OCR_MODE=<str>`](#PAPERLESS_REMOTE_OCR_MODE) {#PAPERLESS_REMOTE_OCR_MODE}
: Which documents are sent to the remote OCR engine.
- `always`: every document of a supported file type is sent to the remote
engine, bypassing the local OCR engine.
- `workflow_only`: documents are processed locally unless a workflow
explicitly enables remote OCR for them, letting you use the remote engine
selectively.
Defaults to "always".
## AI {#ai}
#### [`PAPERLESS_AI_ENABLED=<bool>`](#PAPERLESS_AI_ENABLED) {#PAPERLESS_AI_ENABLED}
@@ -2088,6 +2100,8 @@ suggestions. This setting is required to be set to true in order to use the AI f
models supported by the current embedding backend. If not supplied, defaults to
"text-embedding-3-small" for the OpenAI-compatible backend,
"sentence-transformers/all-MiniLM-L6-v2" for Huggingface, and "embeddinggemma" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for language and resource considerations.
Defaults to None.
@@ -2144,6 +2158,8 @@ setting is required to be set to use the AI features.
: The model to use for the AI backend, i.e. "gpt-3.5-turbo", "gpt-4" or any of the models supported
by the current backend. If not supplied, defaults to "gpt-3.5-turbo" for the OpenAI-compatible
backend and "llama3.1" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for local versus remote and model-size considerations.
Defaults to None.
+14
View File
@@ -456,6 +456,20 @@ def score(
return 10
```
**Remote services**
If your parser sends document content to a remote service, declare it:
```python
class MyCustomParser:
uses_remote_service = True
```
Paperless-ngx excludes such parsers when the document being consumed has not
been marked for remote processing, so users can keep remote OCR off by default
and enable it selectively with a workflow. Parsers that do not declare the
attribute are treated as fully local and are always considered.
**Archive and rendition flags**
```python
+4 -3
View File
@@ -156,7 +156,7 @@ The new settings are independent:
### Database configuration
If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and `skip_archive_file` values are converted to the new `archive_file_generation` field. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent.
If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and explicit `skip_archive_file` values are converted to the new `archive_file_generation` field. Users who relied on the old defaults must set `archive_file_generation` to `always` to preserve the v2 behaviour of always creating an archive. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent.
### Action Required
@@ -165,8 +165,9 @@ Remove any `PAPERLESS_OCR_SKIP_ARCHIVE_FILE` variable from your environment. If
```bash
# v2: skip OCR when text present, always archive
PAPERLESS_OCR_MODE=skip
# v3: equivalent (auto is the new default)
# No change needed - auto is the default
# v3: equivalent
PAPERLESS_OCR_MODE=auto
PAPERLESS_ARCHIVE_FILE_GENERATION=always
# v2: skip OCR when text present, skip archive too
PAPERLESS_OCR_MODE=skip_noarchive
+26 -91
View File
@@ -654,6 +654,19 @@ happened while it was still encrypted, that original version will likewise be mi
**Current limitation**: Passwords are stored as a simple list without descriptions. To handle
multiple PDF types with different passwords, create separate workflows for each use case.
##### Remote OCR {#workflow-action-remote-ocr}
"Remote OCR" actions send the document to the configured remote OCR engine instead of processing it
locally. To use remote OCR selectively, set the [remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE)
to `workflow_only` then add this action to a workflow that matches only the documents you
want sent to the remote engine. See [Remote OCR](#remote-ocr) for the engine setup. The action only works with
a **Consumption Started** trigger.
The action takes no options, its presence is what enables remote OCR for a matching document.
If the remote engine is not configured, or does not support the document's file type, the document is
processed locally instead and a warning is written to the log.
#### Workflow placeholders
Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
@@ -886,19 +899,6 @@ Matching documents with logical expressions:
```
shopname AND (product1 OR product2)
invoice NOT draft
```
`AND`, `OR` and `NOT` must be written in capitals, and parentheses group sub-expressions. Terms written next to each other with no operator between them are combined with `AND`.
!!! warning
A leading `-` does **not** exclude a term. Separators are stripped during indexing, so `invoice -secret` searches for `invoice` and `secret`, which is the opposite of what you probably intended. Use `NOT` to exclude a term: `invoice NOT secret`.
Matching an exact phrase, in order, by quoting it:
```
"quick brown fox"
```
Matching specific tags, correspondents or types:
@@ -906,12 +906,8 @@ Matching specific tags, correspondents or types:
```
type:invoice tag:unpaid
correspondent:university certificate
tag:bills,unpaid
```
- `document_type` may be abbreviated to `type`, and `storage_path` to `path`.
- A comma-separated list after `tag:` requires **all** of the listed tags, so `tag:bills,unpaid` matches only documents tagged both `bills` and `unpaid`.
Matching dates:
```
@@ -920,58 +916,14 @@ added:yesterday
modified:today
```
Matching by archive metadata:
```
asn:100
page_count:12
num_notes:0
checksum:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
original_filename:invoice.pdf
```
- `asn` matches a document's Archive Serial Number.
- `page_count` matches a document's page count.
- `num_notes` matches how many notes a document has.
- `checksum` matches the checksum of the original document file (not the archived/processed version). Unlike the text fields, this one is stored verbatim rather than tokenized, so only a complete, lowercase checksum matches. To search by the first few characters instead, use a wildcard: `checksum:9f86d081*`. Wildcard patterns on the text fields are also tried stemmed, to line up with the stemmed index, but `checksum` is indexed without stemming, so its patterns are not stemmed either: a wildcard prefix is matched literally, apart from being lowercased first. `checksum:9F86D081*` therefore does find the document, even though the plain uppercase term does not.
- `original_filename` matches the filename of the document as originally consumed.
`asn`, `page_count` and `num_notes` are numeric and also accept ranges, for example `asn:[50 to 150]`.
Matching inexact words:
```
invoice*
title:Invoice*
produ*name
```
Wildcards are matched against the _stemmed_ terms stored in the index, not
against the words as they appear in the document. Each literal part of a
pattern is tried both as you typed it and in its stemmed form, so a trailing
`*` matches a word and its inflections (`invoice*` finds "invoice", "invoices"
and "invoiced") as well as longer words whose stored term still begins with
what you typed (`copy*` finds "copyright" alongside "copy" and "copies").
It is still not a plain prefix search over the original text. A trailing `*`
matches a stored term when either the run you typed or its stemmed form is a
prefix of that term, so a fragment that stops part-way between the two matches
neither: `universities*` finds "university" and "universities", which are both
stored as `univers`, while the shorter `universit*` finds nothing at all. For
the same reason `happine*` does not find "happiness", which is stored as
`happi`. And a pattern that requires letters after the wildcard which stemming
has removed cannot match either: `productname` is stored as `productnam`, so
`produ*name` finds nothing.
Matching natural date keywords:
The multi-word date keywords listed below work quoted or unquoted after a
date field (`added:"previous month"` and `added:previous month` are
equivalent); elsewhere in a query the same words are treated as ordinary
search text. Other date expressions the parser accepts (relative offsets
like `-1 week`, or specific dates like `12 december 2019`) must be quoted when
they stand alone as a value; inside a range's brackets they work unquoted, as
in `added:[-1 week to now]`.
```
added:today
modified:yesterday
@@ -984,30 +936,6 @@ Supported date keywords: `today`, `yesterday`, `previous week`,
`this month`, `previous month`, `this year`, `previous year`,
`previous quarter`.
These other date forms also work after a date field:
```
added:tomorrow
created:2005-03-04
added:january
modified:"next monday"
added:"last monday"
added:"2005-01-01T00:00:00Z"
created:[2005-01-01 to 2005-01-31]
added:[2005-06-15T09:00:00Z to 2005-06-15T17:00:00Z]
```
- `tomorrow`, like `today` and `yesterday`, covers that whole day.
- An ISO date such as `2005-03-04` covers that whole day, and `2005-01` covers that whole month.
- A month name such as `january` covers that whole month in the current year.
- `next <weekday>` and `last <weekday>` each cover that whole day and must be quoted. A bare weekday name such as `monday` is not accepted.
- A full timestamp such as `2005-01-01T00:00:00Z` matches that exact instant. Like the other expressions above, it has to be quoted when it stands on its own: `added:"2005-01-01T00:00:00Z"`. The unquoted spelling is rejected with an error rather than searched, because only part of it can be read as a date.
- A range takes two of the above as its bounds, for example `created:[2005 to 2009]` or `added:[2005-01-01 to 2005-01-31]`. Bounds may carry a time of day. A bound is normally written without quotes; if you do quote one, use single quotes (`added:['-1 week' to now]`), because a double-quoted bound is rejected with an error.
!!! warning
As a value on its own, `now`, `noon`, `midnight` and relative offsets such as `"-3 days"` or `"-1 week"` are accepted by the parser but resolve to a single instant rather than to a span of time, so they match only a document whose timestamp is exactly that instant, which in practice means no documents at all. Quoting does not change this. As a *range bound* they are the opposite of a trap and are what you want: `added:['-1 week' to now]` covers the whole of the last seven days. Spellings like `now-3days` and `"3 days ago"` are rejected outright wherever they appear.
#### Searching custom fields
Custom field names and values are included in the full-text index, but they
@@ -1023,7 +951,6 @@ custom_fields.name:Insurance custom_fields.value:policy
- `custom_fields.value` matches against the value of any custom field.
- `custom_fields.name` matches the name of the field (use quotes for multi-word names).
- Combine both to find documents where a specific named field contains a specific value.
- The bare `custom_fields:` prefix is shorthand for `custom_fields.value:`.
Because separators are stripped during indexing, individual parts of formatted
codes are searchable on their own. A value stored as `A-1312/99.50` produces the
@@ -1051,9 +978,9 @@ notes.note:reminder
notes.user:alice notes.note:insurance
```
The bare `notes:` prefix is shorthand for `notes.note:`.
All of these constructs can be combined as you see fit. What is described above is the whole of the query language paperless supports. It resembles other search query languages without being identical to any of them, so a construct that is not documented here is most likely treated as ordinary search text rather than as syntax, and an unrecognized field name is searched as text too.
All of these constructs can be combined as you see fit. If you want to
learn more about the query language used by paperless, see the
[Tantivy query language documentation](https://docs.rs/tantivy/latest/tantivy/query/struct.QueryParser.html).
!!! note
@@ -1176,11 +1103,19 @@ Paperless-ngx supports performing OCR on documents using remote services. At the
[Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence).
This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with
Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details.
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details. These
settings can be supplied as environment variables or via **Application Configuration**.
Additionally, when using a commercial service with this feature, consider both potential costs as well as any associated file size
or page limitations (e.g. with a free tier).
By default, every document of a supported file type is sent to the remote engine. To use it more selectively, set the
[remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE) to `workflow_only`. Documents are then processed locally
unless a [remote OCR workflow action](#workflow-action-remote-ocr) enables it for them, so you can limit the remote
engine to particular documents.
Setting the mode to `workflow_only` also allows the **Reprocess** actions to selectively use remote OCR for individual documents.
## Architecture
Paperless-ngx consists of the following components:
-4
View File
@@ -77,7 +77,6 @@ dependencies = [
"torch~=2.13.0",
"watchfiles>=1.2",
"whitenoise~=6.11",
"whoosh-compat[tantivy]",
"zxing-cpp~=3.1.0",
]
[project.optional-dependencies]
@@ -169,9 +168,6 @@ psycopg-c = [
torch = [
{ index = "pytorch-cpu" },
]
# TODO: switch to a pinned PyPI version once whoosh-compat releases; fall back
# to a pinned git commit SHA if that release slips.
whoosh-compat = { path = "../whoosh-compat" }
[tool.ruff]
target-version = "py311"
@@ -33,7 +33,7 @@ test('should warn on unsaved changes', async ({ page }) => {
await page.getByRole('button', { name: 'Close', exact: true }).click()
await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/)
await page.getByRole('button', { name: 'Cancel' }).click()
await page.getByRole('link', { name: 'Close all' }).click()
await page.getByRole('button', { name: 'Close all' }).click()
await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/)
})
+726 -335
View File
File diff suppressed because it is too large Load Diff
@@ -14,43 +14,48 @@
<a ngbNavLink>{{category}}</a>
<ng-template ngbNavContent>
<div class="p-3">
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
@for (option of getCategoryOptions(category); track option.key) {
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
@for (section of getCategorySections(category); track section) {
@if (section) {
<h5 class="mt-4 mb-3">{{section}}</h5>
}
<div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
@for (option of getCategoryOptions(category, section); track option.key) {
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div>
<div class="mb-n3">
@switch (option.type) {
@case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
@case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
@case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
@case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
@case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
@case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
}
</div>
@if (option.note) {
<div class="form-text fst-italic">{{option.note}}</div>
}
</div>
</div>
</div>
}
</div>
}
</div>
}
</div>
</ng-template>
</li>
@@ -8,7 +8,11 @@ import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectModule } from '@ng-select/ng-select'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { OutputTypeConfig } from 'src/app/data/paperless-config'
import {
ConfigCategory,
ConfigSection,
OutputTypeConfig,
} from 'src/app/data/paperless-config'
import { ConfigService } from 'src/app/services/config.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
@@ -158,4 +162,24 @@ describe('ConfigComponent', () => {
component.resetOption('barcodes_enabled')
expect(component.configForm.get('barcodes_enabled').value).toBeNull()
})
it('should group options into sections within a category, or not', () => {
const sections = component.getCategorySections(ConfigCategory.OCR)
expect(sections).toEqual([null, ConfigSection.RemoteOCR])
expect(
component
.getCategoryOptions(ConfigCategory.OCR)
.map((option) => option.key)
).toContain('output_type')
expect(
component
.getCategoryOptions(ConfigCategory.OCR, ConfigSection.RemoteOCR)
.map((option) => option.key)
).toEqual([
'remote_ocr_engine',
'remote_ocr_api_key',
'remote_ocr_endpoint',
'remote_ocr_mode',
])
})
})
@@ -74,8 +74,20 @@ export class ConfigComponent
return Object.values(ConfigCategory)
}
getCategoryOptions(category: string): ConfigOption[] {
return PaperlessConfigOptions.filter((o) => o.category === category)
getCategorySections(category: string): string[] {
return [
...new Set(
PaperlessConfigOptions.filter((o) => o.category === category).map(
(o) => o.section ?? null // null means no section
)
),
]
}
getCategoryOptions(category: string, section: string = null): ConfigOption[] {
return PaperlessConfigOptions.filter(
(o) => o.category === category && (o.section ?? null) === section
)
}
initialConfig: PaperlessConfig
@@ -89,7 +89,7 @@
</div>
</div>
@if (filterText?.length) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetFilter()">
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetFilter()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs>
</button>
}
@@ -208,7 +208,7 @@
</td>
}
<td class="d-lg-none">
<button class="btn btn-link" (click)="expandTask(task); $event.stopPropagation();">
<button class="btn btn-link" (click)="expandTask(task); $event.stopPropagation();" aria-label="View task details" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
</button>
</td>
@@ -18,9 +18,11 @@
</button>
</pngx-page-header>
<div class="row mb-3">
<ngb-pagination class="col-auto" [pageSize]="25" [collectionSize]="totalDocuments()" [page]="page()" [maxSize]="5" (pageChange)="page.set($event); reload()" size="sm" aria-label="Pagination"></ngb-pagination>
</div>
@if (totalDocuments() > 25) {
<div class="row mb-3">
<ngb-pagination class="col-auto" [pageSize]="25" [collectionSize]="totalDocuments()" [page]="page()" [maxSize]="5" (pageChange)="page.set($event); reload()" size="sm" aria-label="Pagination"></ngb-pagination>
</div>
}
<div class="card border table-responsive mb-3">
<table class="table table-striped align-middle shadow-sm mb-0">
@@ -64,7 +66,7 @@
<td scope="row">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -2,29 +2,36 @@
<button class="navbar-toggler d-md-none collapsed border-0" type="button" data-toggle="collapse"
data-target="#sidebarMenu" aria-controls="sidebarMenu" aria-expanded="false" aria-label="Toggle navigation"
(click)="closeMobileSearch(); toggleMenuCollapsed()">
<span class="navbar-toggler-icon"></span>
<i-bs width="1.5em" height="1.5em" name="list"></i-bs>
</button>
<a class="navbar-brand d-flex align-items-center me-0 px-3 py-3 order-sm-0"
[ngClass]="{ 'slim': slimSidebarEnabled, 'col-auto col-md-3 col-lg-2 col-xxxl-1' : !slimSidebarEnabled, 'py-3' : !customAppTitle?.length || slimSidebarEnabled, 'py-2': customAppTitle?.length }"
<a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0 order-sm-0"
[ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
routerLink="/dashboard"
tourAnchor="tour.intro">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
<div class="ms-2 ms-md-3 d-inline-block" [class.d-md-none]="slimSidebarEnabled">
@if (customAppTitle?.length) {
<div class="d-flex flex-column align-items-start custom-title">
<span class="title">{{customAppTitle}}</span>
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
</div>
@if (!hasCustomBranding) {
<pngx-logo extra_classes="navbar-official-logo px-1" height="2.4rem"></pngx-logo>
<svg class="brand-mark brand-mark-slim d-none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
} @else {
@if (customAppLogo) {
<img class="brand-logo" [src]="customAppLogo" alt="" />
} @else {
Paperless-ngx
<svg class="brand-mark" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
}
</div>
<div class="brand-copy ms-2 text-truncate" [class.d-md-none]="slimSidebarEnabled">
<span class="brand-title text-truncate">{{ appTitle }}</span>
@if (customAppTitle) {
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
}
</div>
}
</a>
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 px-3 ps-md-4 me-sm-auto order-3 order-sm-1"
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 me-sm-auto order-3 order-sm-1"
[class.mobile-hidden]="mobileSearchHidden()">
<div class="col-12 col-md-7">
<div class="col-12 header-search mx-auto">
<pngx-global-search></pngx-global-search>
</div>
</div>
@@ -34,7 +41,7 @@
}
<pngx-toasts-dropdown></pngx-toasts-dropdown>
<li ngbDropdown class="nav-item dropdown">
<button class="btn ps-1 border-0" id="userDropdown" ngbDropdownToggle>
<button class="btn navbar-action border-0" id="userDropdown" ngbDropdownToggle aria-label="User menu" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="person-circle"></i-bs>
<span class="small ms-2 d-none d-sm-inline">
{{this.settingsService.displayName}}
@@ -71,7 +78,7 @@
[ngClass]="slimSidebarEnabled ? 'slim' : 'col-md-3 col-lg-2 col-xxxl-1'" [class.animating]="slimSidebarAnimating()"
[ngbCollapse]="isMenuCollapsed()">
@if (canSaveSettings) {
<button class="btn btn-sm btn-dark sidebar-slim-toggler" (click)="toggleSlimSidebar()">
<button class="btn btn-sm btn-dark sidebar-slim-toggler" (click)="toggleSlimSidebar()" [aria-label]="slimSidebarEnabled ? 'Expand sidebar' : 'Collapse sidebar'" i18n-aria-label>
@if (slimSidebarEnabled) {
<i-bs width="0.9em" height="0.9em" name="chevron-double-right"></i-bs>
} @else {
@@ -79,18 +86,20 @@
}
</button>
}
<div class="sidebar-sticky pt-3 d-flex flex-column justify-space-around">
<div class="sidebar-sticky pt-3 pb-1 d-flex flex-column justify-space-around">
<ul class="nav flex-column">
<li class="nav-item app-link">
<a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="house"></i-bs><span><ng-container i18n>Dashboard</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }">
<a class="nav-link" routerLink="documents" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
<a class="nav-link" routerLink="documents" routerLinkActive="active"
[routerLinkActiveOptions]="{ paths: 'exact', queryParams: 'ignored', matrixParams: 'ignored', fragment: 'ignored' }"
(click)="closeMenu()"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="files"></i-bs><span><ng-container i18n>Documents</ng-container></span>
</a>
@@ -109,7 +118,7 @@
(cdkDragEnded)="onDragEnd($event)">
<a class="nav-link" routerLink="view/{{view.id}}"
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
@if (showSidebarCounts && !slimSidebarEnabled) {
@@ -147,7 +156,7 @@
<li class="nav-item w-100 app-link">
<a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" routerLink="documents/{{d.id}}"
routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="d.title | documentTitle"
[disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
popoverClass="popover-slim">
<i-bs class="me-2" name="file-text"></i-bs><span>{{d.title | documentTitle}}</span>
<span class="close flex-column justify-content-center"
@@ -159,11 +168,12 @@
}
@if (openDocuments.length >= 1) {
<li class="nav-item w-100 app-link">
<a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" [routerLink]="[]" (click)="closeAll()"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
<button type="button" class="nav-link nav-link-action app-link w-100 text-start"
[class.text-truncate]="!slimSidebarEnabled" (click)="closeAll()"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="x"></i-bs><span><ng-container i18n>Close all</ng-container></span>
</a>
</button>
</li>
}
</ul>
@@ -177,10 +187,11 @@
@if (canManageAttributes) {
<li class="nav-item app-link" tourAnchor="tour.tags">
<div class="d-flex align-items-center attributes-row">
<a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
<a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active"
[routerLinkActiveOptions]="{ exact: !(slimSidebarEnabled || attributesSectionsCollapsed) }" (click)="closeMenu()"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="stack"></i-bs><span><ng-container i18n>Attributes</ng-container></span>
<i-bs name="stack"></i-bs><span class="ms-2"><ng-container i18n>Attributes</ng-container></span>
</a>
@if (!slimSidebarEnabled && canSaveSettings) {
<button
@@ -230,7 +241,7 @@
}
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
<a class="nav-link" routerLink="savedviews" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="window-stack"></i-bs><span><ng-container i18n>Saved Views</ng-container></span>
</a>
@@ -239,7 +250,7 @@
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Workflow }"
tourAnchor="tour.workflows">
<a class="nav-link" routerLink="workflows" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="boxes"></i-bs><span><ng-container i18n>Workflows</ng-container></span>
</a>
@@ -247,14 +258,14 @@
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
tourAnchor="tour.mail">
<a class="nav-link" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="envelope"></i-bs><span><ng-container i18n>Mail</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
<a class="nav-link" routerLink="trash" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Trash"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="trash"></i-bs><span><ng-container i18n>Trash</ng-container></span>
</a>
@@ -270,21 +281,21 @@
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }"
tourAnchor="tour.settings">
<a class="nav-link" routerLink="settings" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="gear"></i-bs><span><ng-container i18n>Settings</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.AppConfig }">
<a class="nav-link" routerLink="config" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="sliders2-vertical"></i-bs><span><ng-container i18n>Configuration</ng-container></span>
</a>
</li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.User }">
<a class="nav-link" routerLink="usersgroups" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="people"></i-bs><span><ng-container i18n>Users & Groups</ng-container></span>
</a>
@@ -293,7 +304,7 @@
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.PaperlessTask }"
tourAnchor="tour.file-tasks">
<a class="nav-link" routerLink="tasks" routerLinkActive="active" (click)="closeMenu()"
ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="list-task"></i-bs><span><ng-container i18n>Tasks</ng-container>@if (tasksService.needsAttentionTasks.length > 0) {
<span><span class="badge bg-danger ms-2 d-inline">{{tasksService.needsAttentionTasks.length}}</span></span>
@@ -306,26 +317,26 @@
@if (permissionsService.isAdmin()) {
<li class="nav-item app-link">
<a class="nav-link" routerLink="logs" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Logs"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="text-left"></i-bs><span><ng-container i18n>Logs</ng-container></span>
</a>
</li>
}
<li class="nav-item mt-2" tourAnchor="tour.outro">
<a class="px-3 py-2 text-muted small d-flex align-items-center flex-wrap text-decoration-none"
<a class="text-muted small d-flex align-items-center flex-wrap text-decoration-none nav-anchor"
target="_blank" rel="noopener noreferrer" href="https://docs.paperless-ngx.com" ngbPopover="Documentation"
i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="d-flex me-2" name="question-circle"></i-bs><span><ng-container i18n>Documentation</ng-container></span>
</a>
</li>
<li class="nav-item" [class.visually-hidden]="slimSidebarEnabled">
<div class="px-3 py-0 text-muted small d-flex align-items-center flex-wrap">
<div class="text-muted small d-flex align-items-center flex-wrap nav-label">
<div class="me-3">
<a class="text-muted text-decoration-none" target="_blank" rel="noopener noreferrer"
href="https://github.com/paperless-ngx/paperless-ngx" ngbPopover="GitHub" i18n-ngbPopover
[disablePopover]="!slimSidebarEnabled" placement="end" container="body"
[disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
triggers="mouseenter:mouseleave" popoverClass="popover-slim">
{{ versionString }}
</a>
@@ -363,7 +374,7 @@
</a>
}
} @else {
<a *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }" class="small text-decoration-none" routerLink="/settings" fragment="update-checking"
<a *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }" class="small text-decoration-none" routerLink="/settings" fragment="update-checking" aria-label="Configure update checking" i18n-aria-label
[ngbPopover]="updateCheckingNotEnabledPopContent" popoverClass="shadow" triggers="mouseenter"
container="body">
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
@@ -7,8 +7,8 @@
bottom: 0;
left: 0;
z-index: 995; /* Behind the navbar */
padding: 50px 0 0; /* Height of navbar */
box-shadow: inset -1px 0 0 rgba(0, 0, 0, .1);
padding: 64px 0 0; /* Height of navbar */
border-right: 1px solid color-mix(in srgb, var(--bs-border-color) 65%, transparent);
overflow-y: auto;
--pngx-sidebar-width: 100%;
max-width: var(--pngx-sidebar-width);
@@ -25,7 +25,7 @@
}
.view-name {
max-width: calc(100% - 50px)
max-width: calc(100% - 55px)
}
.nav-group:not(:has(.app-link)) .sidebar-heading {
@@ -47,7 +47,7 @@
}
@media (max-width: 767.98px) {
.sidebar {
top: 3.5rem;
top: 4rem;
}
.search-container {
@@ -67,16 +67,35 @@
main.mobile-search-hidden {
padding-top: 56px;
}
.navbar-toggler {
padding-left: calc(12px - (1.5em * 2.5 / 16));
}
}
.search-container {
padding-left: 1rem;
padding-right: 1rem;
}
// Below sm the search gets its own full-width row, so line it up with main's content edge
@media (max-width: 575.98px) {
.search-container {
padding-left: 12px;
padding-right: 12px;
}
}
main {
transition: all .2s ease;
padding-top: 110px;
padding-top: 118px;
background: var(--pngx-bg-darker);
min-height: 100vh;
}
@media (min-width: 768px) {
main {
padding-top: 56px;
padding-top: 64px;
}
}
@@ -91,11 +110,11 @@ main {
@media(min-width: 768px) {
.sidebar.slim {
max-width: 50px;
max-width: 55px;
li.nav-item span.badge {
display: inline-block;
margin-right: 2px;
margin-right: -4px;
}
}
@@ -139,18 +158,11 @@ main {
}
}
.sidebar.slim {
li.nav-item span.badge {
display: inline-block;
margin-right: 2px;
}
}
.sidebar-slim-toggler {
display: block;
position: fixed;
left: calc(var(--pngx-sidebar-width) - 12px);
top: 60px;
bottom: 16px;
z-index: 996;
--bs-btn-padding-x: 0.35rem;
--bs-btn-padding-y: 0.125rem;
@@ -158,7 +170,7 @@ main {
}
.sidebar.slim .sidebar-slim-toggler {
--pngx-sidebar-width: 50px !important;
--pngx-sidebar-width: 56px !important;
}
}
@@ -171,7 +183,7 @@ main {
position: relative;
top: 0;
height: 100%;
padding-top: 0.5rem;
padding: .75rem .5rem 1rem;
overflow-x: hidden;
overflow-y: auto; /* Scrollable contents if viewport is shorter than content. */
min-height: min-content;
@@ -186,11 +198,14 @@ main {
.sidebar .nav-link {
font-weight: 500;
white-space: nowrap;
border-left: 2px solid transparent;
transition: color .15s ease-in-out;
border-radius: .55rem;
margin: 1px 0;
padding: .55rem .7rem;
transition: color .15s ease-in-out, background-color .15s ease-in-out;
&:hover, &.active, &:focus {
&:hover, &:focus {
color: var(--bs-primary);
background-color: color-mix(in srgb, var(--bs-primary) 8%, transparent);
}
&:focus-visible {
@@ -199,8 +214,21 @@ main {
}
&.active {
font-weight: bold;
border-left-color: var(--bs-primary);
font-weight: 600;
color: var(--bs-primary);
background-color: color-mix(in srgb, var(--bs-primary) 13%, transparent);
}
&.nav-link-action {
color: var(--bs-secondary-color);
font-weight: 400;
background-color: transparent;
&:hover,
&:focus {
color: var(--bs-primary);
background-color: transparent;
}
}
i-bs {
@@ -209,20 +237,49 @@ main {
}
}
// sub-page gets marker only
.nav-item:has(.attributes-submenu.show .nav-link.active) > .attributes-row > .nav-link.active {
border-left-color: transparent;
.sidebar .nav-anchor, .sidebar .nav-label {
padding: .25rem .7rem;
}
// bring sub-menu markers back out to L edge
.attributes-submenu .nav-link {
margin-left: -1rem;
padding-left: calc(var(--bs-nav-link-padding-x) + 1rem);
.attributes-row {
border-radius: .55rem;
margin: .1rem 0;
transition: color .15s ease-in-out, background-color .15s ease-in-out;
> .nav-link {
margin: 0;
&:hover,
&:focus,
&.active {
background-color: transparent;
}
}
&:hover,
&:has(> .nav-link:focus-visible) {
background-color: color-mix(in srgb, var(--bs-primary) 8%, transparent);
}
&:has(> .nav-link.active) {
background-color: color-mix(in srgb, var(--bs-primary) 13%, transparent);
}
}
.attributes-row .attributes-expand-btn {
opacity: 0.2;
width: 1.75rem;
height: 1.75rem;
margin-right: .35rem !important;
border-radius: 50%;
box-shadow: none !important;
transition: opacity 0.15s ease-in-out;
&:focus-visible {
outline: 2px solid color-mix(in srgb, var(--bs-primary) 55%, transparent);
outline-offset: 1px;
opacity: 1;
}
}
.attributes-row:hover .attributes-expand-btn {
@@ -230,8 +287,10 @@ main {
}
.sidebar-heading {
font-size: 0.75rem;
font-size: 0.68rem;
text-transform: uppercase;
font-weight: 700;
opacity: .75;
}
.nav {
@@ -288,16 +347,118 @@ main {
*/
.navbar-brand {
font-size: 1rem;
--pngx-navbar-brand-shadow-rgb: 0, 0, 0;
font-size: 1.0625rem;
min-height: 64px;
letter-spacing: -0.015em;
.flex-column {
padding: 0.15rem 0;
&:hover,
&:focus-visible {
::ng-deep .navbar-official-logo,
.brand-mark,
.brand-logo {
filter: drop-shadow(0 2px 3px rgba(var(--pngx-navbar-brand-shadow-rgb), .5));
}
}
::ng-deep .navbar-official-logo {
filter: drop-shadow(0 1px 2px rgba(var(--pngx-navbar-brand-shadow-rgb), .3));
transition: filter .15s ease-in-out;
@media screen and (max-width: 575.98px) {
max-height: 2rem;
}
}
.brand-mark {
width: 1.65rem;
height: 1.65rem;
flex: 0 0 auto;
transition: filter .15s ease-in-out;
}
.brand-copy {
display: flex;
flex-direction: column;
align-items: flex-start;
min-width: 0;
line-height: 1.1;
transition: transform .15s ease-in-out;
}
.brand-title {
font-weight: 600;
max-width: 100%;
min-width: 0;
}
.byline {
font-size: 0.5rem;
letter-spacing: 0.1rem;
margin-top: .15rem;
font-size: .5rem;
font-weight: 500;
letter-spacing: .1rem;
opacity: .8;
}
.brand-logo {
width: auto;
height: 2.75rem;
max-width: 5rem;
flex: 0 0 auto;
object-fit: contain;
transition: filter .15s ease-in-out, transform .15s ease-in-out;
}
}
:host-context(.primary-light) .navbar-brand {
--pngx-navbar-brand-shadow-rgb: 255, 255, 255; // Light app color, use white shadow for dark text
}
:host ::ng-deep .navbar-official-logo {
.leaf {
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 85%, var(--bs-primary)) !important;
}
.text {
fill: var(--pngx-primary-text-contrast) !important;
}
}
.navbar {
min-height: 64px;
box-shadow: 0 1px 0 rgba(0, 0, 0, .12), 0 4px 18px rgba(0, 0, 0, .08) !important;
}
.navbar > ul {
align-items: center;
gap: .125rem;
padding-right: .5rem;
}
:host ::ng-deep .navbar-action {
display: inline-flex;
align-items: center;
justify-content: center;
min-width: 2.5rem;
min-height: 2.5rem;
padding: .45rem .55rem;
border-radius: .6rem;
transition: background-color .15s ease-in-out, opacity .15s ease-in-out;
&:hover,
&:focus-visible {
background-color: rgba(0, 0, 0, .14);
}
}
#userDropdown {
padding-left: .75rem;
}
.header-search {
width: 100%;
max-width: 55rem;
}
@media screen and (max-width: 575.98px) {
@@ -311,6 +472,7 @@ main {
.navbar-toggler {
grid-area: toggler;
color: var(--pngx-primary-text-contrast);
text-align: left;
}
@@ -346,18 +508,44 @@ main {
grid-area: actions;
justify-self: end;
flex-wrap: nowrap;
gap: 0;
padding-right: .25rem;
}
:host ::ng-deep .navbar-action {
min-width: 2.25rem;
padding-right: .4rem;
padding-left: .4rem;
}
#userDropdown {
padding-right: .35rem;
padding-left: .45rem;
}
}
@media screen and (min-width: 768px) {
.navbar-brand.slim {
max-width: 50px;
max-width: 55px;
.brand-logo {
width: 1.65rem;
max-width: 1.65rem;
}
.brand-mark-slim {
display: block !important;
}
}
:host ::ng-deep .navbar-brand.slim .navbar-official-logo {
display: none;
}
}
:host ::ng-deep .dropdown.show .dropdown-toggle,
:host ::ng-deep .dropdown-toggle:hover {
opacity: 0.7;
opacity: 1;
}
.dropdown-toggle::after {
@@ -45,6 +45,7 @@ import { TasksService } from 'src/app/services/tasks.service'
import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment'
import { ChatComponent } from '../chat/chat/chat.component'
import { LogoComponent } from '../common/logo/logo.component'
import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
import { DocumentDetailComponent } from '../document-detail/document-detail.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
@@ -59,6 +60,7 @@ const SCROLL_THRESHOLD = 16
styleUrls: ['./app-frame.component.scss'],
imports: [
GlobalSearchComponent,
LogoComponent,
DocumentTitlePipe,
IfPermissionsDirective,
ToastsDropdownComponent,
@@ -190,11 +192,34 @@ export class AppFrameComponent
return `${environment.appTitle} v${this.settingsService.get(SETTINGS_KEYS.VERSION)}${environment.tag === 'prod' ? '' : ` #${environment.tag}`}`
}
get appTitle(): string {
this.settingsService.trackChanges()
return (
this.settingsService.get(SETTINGS_KEYS.APP_TITLE) || environment.appTitle
)
}
get customAppTitle(): string {
this.settingsService.trackChanges()
return this.settingsService.get(SETTINGS_KEYS.APP_TITLE)
}
get hasCustomBranding(): boolean {
this.settingsService.trackChanges()
return !!(
this.settingsService.get(SETTINGS_KEYS.APP_TITLE)?.length ||
this.settingsService.get(SETTINGS_KEYS.APP_LOGO)?.length
)
}
get customAppLogo(): string {
this.settingsService.trackChanges()
const logo = this.settingsService.get(SETTINGS_KEYS.APP_LOGO)
return logo?.length
? environment.apiBaseUrl.replace(/\/api\/$/, logo)
: null
}
get canSaveSettings(): boolean {
return (
this.permissionsService.currentUserCan(
@@ -253,6 +278,10 @@ export class AppFrameComponent
})
}
get slimSidebarPopoversEnabled(): boolean {
return this.slimSidebarEnabled && !this.isMobileViewport()
}
get attributesSectionsCollapsed(): boolean {
this.settingsService.trackChanges()
return this.settingsService
@@ -4,12 +4,12 @@ form {
> i-bs[name="search"] {
position: absolute;
left: 0.6rem;
top: .35rem;
top: .25rem;
color: rgba(255, 255, 255, 0.6);
@media screen and (min-width: 768px) {
// adjust for smaller font size on non-mobile
top: 0.25rem;
top: .15rem;
}
}
@@ -37,8 +37,9 @@ form {
}
.form-control {
color: rgba(255, 255, 255, 0.3);
background-color: rgba(0, 0, 0, 0.15);
min-height: 2.25rem;
color: rgba(255, 255, 255, 0.55);
background-color: rgba(0, 0, 0, 0.16);
padding-left: 1.8rem;
border-color: rgba(255, 255, 255, 0.2);
transition: all .3s ease, padding-left 0s ease, background-color 0s ease; // Safari requires all
@@ -52,7 +53,7 @@ form {
}
&:focus-within {
background-color: rgba(0, 0, 0, 0.3);
background-color: rgba(0, 0, 0, 0.26);
color: var(--bs-light);
flex-grow: 1;
padding-left: 0.5rem;
@@ -1,9 +1,9 @@
<li ngbDropdown class="nav-item mx-1" (openChange)="onOpenChange($event)">
<li ngbDropdown class="nav-item position-relative" (openChange)="onOpenChange($event)">
@if (toasts().length) {
<span class="badge rounded-pill z-3 pe-none bg-secondary me-2 position-absolute top-0 left-0">{{ toasts().length }}</span>
<span class="notification-count badge rounded-pill z-3 pe-none bg-secondary position-absolute">{{ toasts().length }}</span>
}
<button class="btn border-0" id="notificationsDropdown" ngbDropdownToggle>
<button class="btn navbar-action border-0" id="notificationsDropdown" ngbDropdownToggle aria-label="Notifications" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="bell"></i-bs>
</button>
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="notificationsDropdown">
@@ -11,6 +11,16 @@
display: none;
}
.notification-count {
top: -.2rem;
right: -.2rem;
min-width: 1.15rem;
height: 1.15rem;
padding: .18rem .32rem;
font-size: .68rem;
line-height: 1;
}
.dropdown-item {
white-space: initial;
}
@@ -1,6 +1,6 @@
<li ngbDropdown class="nav-item me-n2" (openChange)="onOpenChange($event)">
<button class="btn border-0" id="chatDropdown" ngbDropdownToggle>
<li ngbDropdown class="nav-item" (openChange)="onOpenChange($event)">
<button class="btn navbar-action border-0" id="chatDropdown" ngbDropdownToggle aria-label="Chat" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="chatSquareDots"></i-bs>
</button>
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="chatDropdown">
@@ -0,0 +1,28 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
</div>
<div class="modal-body">
@if (messageBold) {
<p class="text-break"><b>{{messageBold}}</b></p>
}
@if (message) {
<p class="mb-0 text-break" [innerHTML]="message"></p>
}
@if (showRemoteOcr) {
<div class="form-check mt-3">
<input class="form-check-input" type="checkbox" id="reprocessRemoteOcr" [(ngModel)]="remoteOcr" />
<label class="form-check-label" for="reprocessRemoteOcr" i18n>Use remote OCR</label>
<div class="form-text" i18n>Sends the document to the configured remote OCR service, which may incur costs.</div>
</div>
}
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,72 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { RemoteOCRModeConfig } from 'src/app/data/paperless-config'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { SettingsService } from 'src/app/services/settings.service'
import { ReprocessConfirmDialogComponent } from './reprocess-confirm-dialog.component'
describe('ReprocessConfirmDialogComponent', () => {
let component: ReprocessConfirmDialogComponent
let fixture: ComponentFixture<ReprocessConfirmDialogComponent>
let settingsService: SettingsService
const createComponent = (configured: boolean, mode: string) => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, configured)
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_MODE, mode)
fixture = TestBed.createComponent(ReprocessConfirmDialogComponent)
component = fixture.componentInstance
fixture.detectChanges()
}
beforeEach(async () => {
TestBed.configureTestingModule({
providers: [
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
imports: [ReprocessConfirmDialogComponent],
}).compileComponents()
settingsService = TestBed.inject(SettingsService)
})
it('should not request remote OCR by default', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.remoteOcr).toBeFalsy()
})
it('should not offer remote OCR when no engine is configured', () => {
createComponent(false, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should not offer remote OCR when it already handles every document', () => {
createComponent(true, RemoteOCRModeConfig.ALWAYS)
expect(component.showRemoteOcr).toBeFalsy()
expect(
fixture.nativeElement.querySelector('#reprocessRemoteOcr')
).toBeNull()
})
it('should offer remote OCR when configured and selective', () => {
createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
expect(component.showRemoteOcr).toBeTruthy()
const checkbox = fixture.nativeElement.querySelector('#reprocessRemoteOcr')
expect(checkbox).not.toBeNull()
checkbox.click()
fixture.detectChanges()
expect(component.remoteOcr).toBeTruthy()
})
})
@@ -0,0 +1,20 @@
import { Component, inject } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { SettingsService } from 'src/app/services/settings.service'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-reprocess-confirm-dialog',
templateUrl: './reprocess-confirm-dialog.component.html',
imports: [FormsModule],
})
export class ReprocessConfirmDialogComponent extends ConfirmDialogComponent {
private settings = inject(SettingsService)
remoteOcr: boolean = false
public get showRemoteOcr(): boolean {
// Hidden when it is not configured, or when it already handles every document anyway.
return this.settings.remoteOCRIsSelectable
}
}
@@ -6,7 +6,7 @@
<div class="modal-body">
<div class="row">
<div class="col-2 d-flex justify-content-end">
<button class="btn btn-secondary mt-auto" (click)="rotate(false)">
<button class="btn btn-secondary mt-auto" (click)="rotate(false)" aria-label="Rotate counterclockwise" i18n-aria-label>
<i-bs name="arrow-counterclockwise"></i-bs>
</button>
</div>
@@ -16,7 +16,7 @@
}
</div>
<div class="col-2 d-flex">
<button class="btn btn-secondary mt-auto" (click)="rotate()">
<button class="btn btn-secondary mt-auto" (click)="rotate()" aria-label="Rotate clockwise" i18n-aria-label>
<i-bs name="arrow-clockwise"></i-bs>
</button>
</div>
@@ -1,5 +1,5 @@
<div ngbDropdown #fieldDropdown="ngbDropdown" (openChange)="onOpenClose($event)" [popperOptions]="popperOptions">
<button type="button" class="btn btn-sm btn-outline-primary" id="customFieldsDropdown" [disabled]="disabled" ngbDropdownToggle>
<button type="button" class="btn btn-sm btn-outline-primary" id="customFieldsDropdown" [disabled]="disabled" ngbDropdownToggle aria-label="Custom Fields" i18n-aria-label>
<i-bs name="ui-radios"></i-bs><div class="d-none d-lg-inline ms-1"><ng-container i18n>Custom Fields</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="customFieldsDropdown" class="shadow custom-fields-dropdown">
@@ -1,6 +1,6 @@
@if (useDropdown) {
<div class="btn-group w-100" role="group" ngbDropdown #dropdown="ngbDropdown" (openChange)="onOpenChange($event)" [popperOptions]="popperOptions">
<button class="btn btn-sm btn-outline-primary" id="dropdown_toggle" ngbDropdownToggle [disabled]="disabled">
<button class="btn btn-sm btn-outline-primary" id="dropdown_toggle" ngbDropdownToggle [disabled]="disabled" [aria-label]="title">
<i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
@if (isActive) {
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge>
@@ -38,7 +38,7 @@
ngbDatepicker
#d="ngbDatepicker"
[footerTemplate]="datePickerFooterTemplate" />
<button class="btn btn-sm btn-outline-secondary rounded-end" (click)="d.toggle()" type="button">
<button class="btn btn-sm btn-outline-secondary rounded-end" (click)="d.toggle()" type="button" aria-label="Open date picker" i18n-aria-label>
<i-bs name="calendar-event"></i-bs>
</button>
<ng-template #datePickerFooterTemplate>
@@ -143,7 +143,7 @@
<input class="w-25 form-control rounded-end" type="text" [(ngModel)]="atom.value" [disabled]="disabled">
}
}
<button class="btn btn-link btn-sm text-danger pe-0" type="button" (click)="removeElement(atom)" [disabled]="disabled">
<button class="btn btn-link btn-sm text-danger pe-0" type="button" (click)="removeElement(atom)" [disabled]="disabled" aria-label="Remove query" i18n-aria-label>
<i-bs name="x-circle"></i-bs>
</button>
</div>
@@ -185,7 +185,7 @@
<i-bs name="braces"></i-bs>
</button>
@if (expression.depth > 0) {
<button type="button" class="btn btn-sm btn-outline-secondary text-danger" (click)="removeElement(expression)" [disabled]="disabled">
<button type="button" class="btn btn-sm btn-outline-secondary text-danger" (click)="removeElement(expression)" [disabled]="disabled" aria-label="Remove expression" i18n-aria-label>
<i-bs name="x-circle"></i-bs>
</button>
}
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group" [popperOptions]="popperOptions" [placement]="placement">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="createdDateTo || createdDateFrom ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="createdDateTo || createdDateFrom ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs width="1em" height="1em" name="calendar-event-fill"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
</button>
@@ -9,7 +9,7 @@
<div class="list-group-item d-flex p-2 select-item" role="menuitem">
<div class="selected-icon">
@if (createdRelativeDate) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedRelativeDate()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedRelativeDate()" aria-label="Clear created relative date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -33,7 +33,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (createdDateFrom) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedFrom()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedFrom()" aria-label="Clear created from date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -43,7 +43,7 @@
<span class="input-group-text w-25 small text-muted" i18n>From</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="createdDateFrom" ngbDatepicker #createdDateFromPicker="ngbDatepicker" [footerTemplate]="createdFromFooterTemplate">
<button class="btn btn-outline-secondary" (click)="createdDateFromPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="createdDateFromPicker.toggle()" type="button" aria-label="Open created from date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #createdFromFooterTemplate>
@@ -57,7 +57,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (createdDateTo) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedTo()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedTo()" aria-label="Clear created to date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -67,7 +67,7 @@
<span class="input-group-text w-25 small text-muted" i18n>To</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="createdDateTo" ngbDatepicker #createdDateToPicker="ngbDatepicker" [footerTemplate]="createdToFooterTemplate">
<button class="btn btn-outline-secondary" (click)="createdDateToPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="createdDateToPicker.toggle()" type="button" aria-label="Open created to date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #createdToFooterTemplate>
@@ -85,7 +85,7 @@
<div class="list-group-item d-flex p-2 select-item" role="menuitem">
<div class="selected-icon">
@if (addedRelativeDate) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedRelativeDate()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedRelativeDate()" aria-label="Clear added relative date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -109,7 +109,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (addedDateFrom) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedFrom()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedFrom()" aria-label="Clear added from date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -119,7 +119,7 @@
<span class="input-group-text w-25 small text-muted" i18n>From</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="addedDateFrom" ngbDatepicker #addedDateFromPicker="ngbDatepicker" [footerTemplate]="addedFromFooterTemplate">
<button class="btn btn-outline-secondary" (click)="addedDateFromPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="addedDateFromPicker.toggle()" type="button" aria-label="Open added from date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #addedFromFooterTemplate>
@@ -133,7 +133,7 @@
<div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon">
@if (addedDateTo) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedTo()">
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedTo()" aria-label="Clear added to date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a>
@@ -143,7 +143,7 @@
<span class="input-group-text w-25 small text-muted" i18n>To</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="addedDateTo" ngbDatepicker #addedDateToPicker="ngbDatepicker" [footerTemplate]="addedToFooterTemplate">
<button class="btn btn-outline-secondary" (click)="addedDateToPicker.toggle()" type="button">
<button class="btn btn-outline-secondary" (click)="addedDateToPicker.toggle()" type="button" aria-label="Open added to date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs>
</button>
<ng-template #addedToFooterTemplate>
@@ -455,6 +455,13 @@
</div>
</div>
}
@case (WorkflowActionType.RemoteOcr) {
<div class="row">
<div class="col">
<p class="text-muted small" i18n>The document will be sent to the configured remote OCR service. May incur costs.</p>
</div>
</div>
}
}
</div>
</ng-template>
@@ -29,6 +29,7 @@ import {
DocumentSource,
WorkflowTriggerType,
} from 'src/app/data/workflow-trigger'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
@@ -224,7 +225,12 @@ describe('WorkflowEditDialogComponent', () => {
).toEqual('Document Added')
expect(component.getTriggerTypeOptionName(null)).toEqual('')
expect(component.sourceOptions).toEqual(DOCUMENT_SOURCE_OPTIONS)
expect(component.actionTypeOptions).toEqual(WORKFLOW_ACTION_OPTIONS)
// Remote OCR is absent until the workflow has a consumption trigger
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.RemoteOcr
)
)
expect(
component.getActionTypeOptionName(WorkflowActionType.Assignment)
).toEqual('Assignment')
@@ -237,7 +243,104 @@ describe('WorkflowEditDialogComponent', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(false)
component.ngOnInit()
expect(component.actionTypeOptions).toEqual(
WORKFLOW_ACTION_OPTIONS.filter((a) => a.id !== WorkflowActionType.Email)
WORKFLOW_ACTION_OPTIONS.filter(
(a) =>
a.id !== WorkflowActionType.Email &&
a.id !== WorkflowActionType.RemoteOcr
)
)
})
it('should offer remote OCR only for consumption workflows', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// A consumption trigger makes the action reachable
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Any other trigger type runs after the document has been parsed
component.object = {
name: 'Workflow 2',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should offer remote OCR on a trigger added to a new workflow', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
component.ngOnInit()
// Nothing for the action to apply to yet
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
// addTrigger creates the form field with emitEvent false, so the options
// have to be computed on read rather than cached from valueChanges
component.addTrigger()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
// Switching that trigger to a type that runs after parsing removes it
component.triggerFields
.at(0)
.get('type')
.setValue(WorkflowTriggerType.DocumentAdded)
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
it('should keep remote OCR listed when an action already uses it', () => {
jest.spyOn(settingsService, 'get').mockReturnValue(true)
// Otherwise changing the trigger would silently blank the selection
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
actions: [{ type: WorkflowActionType.RemoteOcr }],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).toContain(
WorkflowActionType.RemoteOcr
)
})
it('should not offer remote OCR when no engine is configured', () => {
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) => key !== SETTINGS_KEYS.REMOTE_OCR_CONFIGURED)
component.object = {
name: 'Workflow 1',
order: 0,
enabled: true,
triggers: [{ type: WorkflowTriggerType.Consumption }],
actions: [],
} as Workflow
component.ngOnInit()
expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
WorkflowActionType.RemoteOcr
)
})
@@ -148,6 +148,10 @@ export const WORKFLOW_ACTION_OPTIONS = [
id: WorkflowActionType.MoveToTrash,
name: $localize`Move to trash`,
},
{
id: WorkflowActionType.RemoteOcr,
name: $localize`Remote OCR`,
},
]
export enum TriggerFilterType {
@@ -504,8 +508,6 @@ export class WorkflowEditDialogComponent
expandedItem: number = null
readonly allowedActionTypes = signal([])
private readonly triggerFilterOptionsMap = new WeakMap<
FormArray,
TriggerFilterOption[]
@@ -548,13 +550,40 @@ export class WorkflowEditDialogComponent
this.checkRemovalActionFields.bind(this)
)
this.checkRemovalActionFields(this.objectForm.value)
this.allowedActionTypes.set(
this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)
? WORKFLOW_ACTION_OPTIONS
: WORKFLOW_ACTION_OPTIONS.filter(
(a) => a.id !== WorkflowActionType.Email
)
)
}
private allowedActionTypes: typeof WORKFLOW_ACTION_OPTIONS = null
private getAllowedActionTypes() {
let allowed = WORKFLOW_ACTION_OPTIONS
if (!this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.Email)
}
// Remote OCR is decided before the document is parsed, so it is only
// offered for workflows that run at consumption.
const formWorkflow: Workflow = this.objectForm?.value
const remoteOcrUsable =
this.settingsService.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
(formWorkflow?.triggers?.some(
(trigger) => trigger.type === WorkflowTriggerType.Consumption
) ||
formWorkflow?.actions?.some(
(action) => action.type === WorkflowActionType.RemoteOcr
))
if (!remoteOcrUsable) {
allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
}
if (
this.allowedActionTypes?.length === allowed.length &&
this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
) {
return this.allowedActionTypes
}
this.allowedActionTypes = allowed
return allowed
}
private checkRemovalActionFields(formWorkflow: Workflow) {
@@ -1279,7 +1308,8 @@ export class WorkflowEditDialogComponent
get actionTypeOptions() {
this.settingsService.trackChanges()
return this.allowedActionTypes()
// Computed on read rather than cached
return this.getAllowedActionTypes()
}
getActionTypeOptionName(type: WorkflowActionType): string {
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group" (openChange)="dropdownOpenChange($event)" #dropdown="ngbDropdown" (keydown)="listKeyDown($event)" [popperOptions]="popperOptions" [autoClose]="!creating()">
<button class="btn btn-sm" id="dropdown_{{name}}" ngbDropdownToggle [ngClass]="!editing && selectionModel.selectionSize() > 0 ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled">
<button class="btn btn-sm" id="dropdown_{{name}}" ngbDropdownToggle [ngClass]="!editing && selectionModel.selectionSize() > 0 ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
@if (!editing && selectionModel.totalCount > 0) {
<pngx-clearable-badge [number]="selectionModel.totalCount" [selected]="selectionModel.selectionSize() > 0" (cleared)="reset()"></pngx-clearable-badge>
@@ -4,7 +4,7 @@
}
<div class="input-group" [class.is-invalid]="error">
<button type="button" class="input-group-text" [style.background-color]="value" (click)="colorPicker.toggle()">&nbsp;&nbsp;&nbsp;</button>
<button type="button" class="input-group-text" [style.background-color]="value" (click)="colorPicker.toggle()" aria-label="Open color picker" i18n-aria-label>&nbsp;&nbsp;&nbsp;</button>
<ng-template #popContent>
<div style="min-width: 200px;" class="pb-3">
@@ -14,7 +14,7 @@
<input #inputField class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" (change)="onChange(value)" [autoClose]="'outside'" [ngbPopover]="popContent" #colorPicker="ngbPopover" placement="bottom" popoverClass="shadow">
<button class="btn btn-outline-secondary" type="button" (click)="randomize()">
<button class="btn btn-outline-secondary" type="button" (click)="randomize()" aria-label="Choose a random color" i18n-aria-label>
<i-bs name="dice5"></i-bs>
</button>
@@ -74,7 +74,7 @@
class="flex-grow-1"></pngx-input-textarea>
}
}
<button type="button" class="btn btn-link text-danger" (click)="removeSelectedField.next(fieldId)">
<button type="button" class="btn btn-link text-danger" (click)="removeSelectedField.next(fieldId)" aria-label="Remove custom field" i18n-aria-label>
<i-bs name="trash"></i-bs>
</button>
</div>
@@ -13,7 +13,7 @@
<input #inputField class="form-control" [class.is-invalid]="error" [placeholder]="placeholder" [id]="inputId" maxlength="10"
(dateSelect)="onChange(value)" (change)="onChange(value)" (keypress)="onKeyPress($event)" (paste)="onPaste($event)"
name="dp" [(ngModel)]="value" ngbDatepicker #datePicker="ngbDatepicker" #datePickerContent="ngModel" [disabled]="disabled" [footerTemplate]="datePickerFooterTemplate">
<button class="btn btn-outline-secondary calendar" (click)="datePicker.toggle()" type="button" [disabled]="disabled">
<button class="btn btn-outline-secondary calendar" (click)="datePicker.toggle()" type="button" [disabled]="disabled" aria-label="Open date picker" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="calendar"></i-bs>
</button>
<ng-template #datePickerFooterTemplate>
@@ -13,7 +13,7 @@
<div class="input-group mb-3">
<input type="text" class="form-control" [(ngModel)]="entry[0]" (change)="inputChange()" [disabled]="disabled" autocomplete="off">
<input type="text" class="form-control" [(ngModel)]="entry[1]" (change)="inputChange()" [disabled]="disabled" autocomplete="off">
<button type="button" class="btn btn-outline-secondary" (click)="removeEntry(i)">
<button type="button" class="btn btn-outline-secondary" (click)="removeEntry(i)" aria-label="Remove entry" i18n-aria-label>
<i-bs class="text-danger" name="trash"></i-bs>
</button>
</div>
@@ -50,7 +50,7 @@
</ng-template>
</ng-select>
@if (allowCreateNew && !hideAddButton) {
<button class="btn btn-outline-secondary" type="button" (click)="addItem()" [disabled]="disabled">
<button class="btn btn-outline-secondary" type="button" (click)="addItem()" [disabled]="disabled" aria-label="Create new item" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="plus"></i-bs>
</button>
}
@@ -48,7 +48,7 @@
</ng-template>
</ng-select>
@if (allowCreate && !hideAddButton) {
<button class="btn btn-outline-secondary" type="button" (click)="createTag(null, true)" [disabled]="disabled">
<button class="btn btn-outline-secondary" type="button" (click)="createTag(null, true)" [disabled]="disabled" aria-label="Create new tag" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="plus"></i-bs>
</button>
}
@@ -1,6 +1,6 @@
<div class="row pt-3 pb-3 pb-md-2 align-items-center">
<div class="row pt-3 pb-2 align-items-center">
<div class="col-md text-truncate">
<h3 class="d-flex align-items-center mb-1" style="line-height: 1.4">
<h3 class="d-flex align-items-center mb-2 mb-md-1" style="line-height: 1.4">
<span class="text-truncate">{{title()}}</span>
@if (id()) {
<span class="badge bg-primary text-primary-text-contrast ms-3 small fs-normal cursor-pointer" (click)="copyID()">
@@ -12,10 +12,10 @@
</span>
}
@if (subTitle()) {
<span class="h6 mb-0 mt-1 d-block d-md-inline fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
<span class="page-subtitle h6 mb-0 mt-1 fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
}
@if (info()) {
<button class="btn btn-sm btn-link text-muted p-0 p-md-2" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<button class="btn btn-sm btn-link text-muted p-0 ms-2 p-md-2 ms-md-0" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<i-bs name="question-circle"></i-bs>
</button>
<ng-template #infoPopover>
@@ -1,5 +1,6 @@
h3 {
min-height: calc(1.325rem + 0.9vw);
flex-wrap: wrap;
.badge {
font-size: 0.65rem;
@@ -7,6 +8,25 @@ h3 {
}
}
// Drop the subtitle onto its own and shrink it
@media (max-width: 767.98px) {
h3 > .page-subtitle {
flex: 0 0 100%;
margin-top: 0 !important;
font-size: .7rem;
line-height: 1.3 !important;
}
}
:host {
display: block;
margin-bottom: .35rem;
}
h3 > .h6 {
color: var(--bs-secondary-color);
}
@media (min-width: 1200px) {
h3 {
min-height: 2.8rem;
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="isActive ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="isActive ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
</button>
@@ -107,6 +107,25 @@ describe('PermissionsSelectComponent', () => {
expect(component.form.get('Tag').get('Change').disabled).toBeTruthy()
})
it('should update checkboxes when inherited permissions change', () => {
component.ngOnInit()
component.inheritedPermissions = ['documents.change_document']
component.writeValue(['delete_document'])
expect(component.form.get('Document').get('Change').value).toBeTruthy()
expect(component.form.get('Document').get('Change').disabled).toBeTruthy()
// swap for a group with a different permission, but the same number of them
component.inheritedPermissions = ['documents.view_document']
// the no-longer-inherited permission is unchecked, the explicit one is kept
expect(component.permissions).toEqual(['delete_document'])
expect(component.form.get('Document').get('Change').value).toBeFalsy()
expect(component.form.get('Document').get('Change').disabled).toBeFalsy()
expect(component.form.get('Document').get('Delete').value).toBeTruthy()
expect(component.form.get('Document').get('View').value).toBeTruthy()
expect(component.form.get('Document').get('View').disabled).toBeTruthy()
})
it('should exclude history permissions if disabled', () => {
settingsService.set(SETTINGS_KEYS.AUDITLOG_ENABLED, false)
fixture = TestBed.createComponent(PermissionsSelectComponent)
@@ -74,12 +74,22 @@ export class PermissionsSelectComponent
? inherited.map((p) => p.replace(/^\w+\./g, ''))
: []
if (this._inheritedPermissions !== newInheritedPermissions) {
this._inheritedPermissions = newInheritedPermissions
this.writeValue(this.permissions) // updates visual checks etc.
}
const changed =
newInheritedPermissions.length !== this._inheritedPermissions.length ||
newInheritedPermissions.some(
(p) => !this._inheritedPermissions.includes(p)
)
this.updateDisabledStates()
if (changed) {
// skip inherited permissions, these are the explicitly set ones
this.permissions = this.getSelectedPermissions(
this.form.getRawValue()
).filter((p) => !this._inheritedPermissions.includes(p))
this._inheritedPermissions = newInheritedPermissions
this.applyCheckedState()
} else {
this.updateDisabledStates()
}
}
inheritedWarning: string = $localize`Inherited from group`
@@ -106,20 +116,29 @@ export class PermissionsSelectComponent
}
this.permissions = permissions ?? []
const allPerms = this._inheritedPermissions.concat(this.permissions)
this.applyCheckedState()
}
allPerms.forEach((permissionStr) => {
const { actionKey, typeKey } =
this.permissionsService.getPermissionKeys(permissionStr)
// sets every checkbox from inherited + own perms
private applyCheckedState(): void {
const allPerms = new Set(
this._inheritedPermissions.concat(this.permissions)
)
if (actionKey && typeKey) {
this.form
.get(typeKey)
?.get(actionKey)
?.patchValue(true, { emitEvent: false })
}
})
this.allowedTypes.forEach((type) => {
const typeGroup = this.form.get(type)
for (const action of Object.keys(PermissionAction)) {
typeGroup.get(action)?.patchValue(
allPerms.has(
this.permissionsService.getPermissionCode(
PermissionAction[action],
PermissionType[type]
)
),
{ emitEvent: false } // don't trigger valueChanges now
)
}
if (this.typeHasAllActionsSelected(type)) {
this.typesWithAllActions.add(type)
} else {
@@ -150,26 +169,9 @@ export class PermissionsSelectComponent
ngOnInit(): void {
this.form.valueChanges.subscribe((newValue) => {
let permissions = []
Object.entries(newValue).forEach(([typeKey, typeValue]) => {
const selectedActions = Object.entries(typeValue).filter(
([actionKey, actionValue]) =>
actionValue &&
this.isActionSupported(
PermissionType[typeKey],
PermissionAction[actionKey]
)
)
selectedActions.forEach(([actionKey]) => {
permissions.push(
(PermissionType[typeKey] as string).replace(
'%s',
PermissionAction[actionKey]
)
)
})
const permissions = this.getSelectedPermissions(newValue)
Object.keys(newValue).forEach((typeKey) => {
if (this.typeHasAllActionsSelected(typeKey)) {
this.typesWithAllActions.add(typeKey)
} else {
@@ -269,6 +271,30 @@ export class PermissionsSelectComponent
return true
}
private getSelectedPermissions(formValue: object): string[] {
const permissions = []
Object.entries(formValue).forEach(([typeKey, typeValue]) => {
Object.entries(typeValue)
.filter(
([actionKey, actionValue]) =>
actionValue &&
this.isActionSupported(
PermissionType[typeKey],
PermissionAction[actionKey]
)
)
.forEach(([actionKey]) => {
permissions.push(
this.permissionsService.getPermissionCode(
PermissionAction[actionKey],
PermissionType[typeKey]
)
)
})
})
return permissions
}
private typeHasAllActionsSelected(typeKey: string): boolean {
return Object.keys(PermissionAction)
.filter((action) =>
@@ -1,5 +1,5 @@
<div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())">
<button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())" [aria-label]="noSuggestions ? 'No suggestions' : 'Suggest'" i18n-aria-label>
@if (loading()) {
<div class="spinner-border spinner-border-sm" role="status"></div>
} @else if (noSuggestions) {
@@ -23,7 +23,7 @@
<dd>
{{status().pngx_version}}
@if (versionMismatch()) {
<button class="btn btn-sm d-inline align-items-center btn-dark text-uppercase small" [ngbPopover]="versionPopover" triggers="click mouseenter:mouseleave">
<button class="btn btn-sm d-inline align-items-center btn-dark text-uppercase small" [ngbPopover]="versionPopover" triggers="click mouseenter:mouseleave" aria-label="View version mismatch details" i18n-aria-label>
<i-bs name="exclamation-triangle-fill" class="text-danger lh-1"></i-bs>
</button>
}
@@ -1,9 +1,7 @@
<pngx-page-header title="Dashboard" [subTitle]="subtitle" i18n-title tourAnchor="tour.dashboard">
<pngx-logo extra_classes="d-none d-md-block mt-n2" height="3rem"></pngx-logo>
</pngx-page-header>
<pngx-page-header title="Dashboard" [subTitle]="subtitle" i18n-title tourAnchor="tour.dashboard"></pngx-page-header>
<div class="row">
<div class="col-12 col-lg-8 col-xl-9 mb-4">
<div class="row dashboard-grid g-4 pb-0">
<div class="col-12 col-lg-8 col-xl-9 mb-4 dashboard-main">
<div class="row row-cols-1 g-4"
cdkDropList
[cdkDropListDisabled]="settingsService.globalDropzoneActive()"
@@ -58,7 +56,7 @@
</ng-container>
</div>
</div>
<div class="col-12 col-lg-4 col-xl-3 col-sidebar">
<div class="col-12 col-lg-4 col-xl-3 col-sidebar dashboard-aside">
<div class="row row-cols-1 g-4 mb-4 sticky-lg-top z-0">
<pngx-upload-file-widget></pngx-upload-file-widget>
<pngx-statistics-widget *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.UISettings }"></pngx-statistics-widget>
@@ -1,3 +1,13 @@
.col-sidebar .row {
top: 3.5rem;
top: 4.75rem;
}
.dashboard-grid {
padding-bottom: 2rem;
}
@media (min-width: 1200px) {
.dashboard-main {
padding-right: 1rem;
}
}
@@ -15,7 +15,6 @@ import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment'
import { LogoComponent } from '../common/logo/logo.component'
import { PageHeaderComponent } from '../common/page-header/page-header.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
import { SavedViewWidgetComponent } from './widgets/saved-view-widget/saved-view-widget.component'
@@ -28,7 +27,6 @@ import { WelcomeWidgetComponent } from './widgets/welcome-widget/welcome-widget.
templateUrl: './dashboard.component.html',
styleUrls: ['./dashboard.component.scss'],
imports: [
LogoComponent,
PageHeaderComponent,
SavedViewWidgetComponent,
StatisticsWidgetComponent,
@@ -8,3 +8,14 @@
width: 0.6rem;
}
}
.list-group {
--bs-list-group-border-color: color-mix(in srgb, var(--bs-border-color) 58%, transparent);
--bs-list-group-bg: transparent;
border-radius: .6rem;
overflow: hidden;
}
.list-group-item {
padding: .7rem .8rem;
}
@@ -4,6 +4,16 @@
.btn-outline-dark {
--bs-btn-border-color: var(--bs-border-color-translucent);
border-style: dashed;
border-width: 1px;
min-height: 5.25rem;
background: color-mix(in srgb, var(--bs-primary) 4%, var(--bs-light)) !important;
&:hover,
&:focus {
border-color: var(--bs-primary);
background: color-mix(in srgb, var(--bs-primary) 9%, var(--bs-light)) !important;
}
}
.smaller {
@@ -1,5 +1,5 @@
@if (!cardless()) {
<div class="card shadow-sm bg-light fade" [class.show]="show()" cdkDrag [cdkDragDisabled]="!draggable()" cdkDragPreviewContainer="parent">
<div class="card bg-light fade" [class.show]="show()" cdkDrag [cdkDragDisabled]="!draggable()" cdkDragPreviewContainer="parent">
<div class="card-header">
<div class="d-flex justify-content-between align-items-center">
<div class="d-flex align-items-center">
@@ -2,6 +2,20 @@ i-bs {
cursor: move;
}
.card {
overflow: hidden;
border-color: color-mix(in srgb, var(--bs-border-color) 68%, transparent);
.card-header {
padding: .9rem 1rem;
border-bottom-color: color-mix(in srgb, var(--bs-border-color) 55%, transparent);
}
.card-body {
padding: 1rem;
}
}
.fade.show {
animation: pngx-entry-fade 160ms ease-out;
}
@@ -20,7 +20,7 @@
</div>
}
<button type="button" class="btn btn-sm btn-outline-danger me-md-4" (click)="delete()" [disabled]="!userIsOwner" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
<button type="button" class="btn btn-sm btn-outline-danger me-md-4" (click)="delete()" [disabled]="!userIsOwner" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }" aria-label="Delete" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="trash"></i-bs><span class="d-none d-lg-inline ps-1" i18n>Delete</span>
</button>
@@ -35,7 +35,7 @@
/>
<div class="btn-group">
<button (click)="download()" class="btn btn-sm btn-outline-primary" [disabled]="downloading()">
<button (click)="download()" class="btn btn-sm btn-outline-primary" [disabled]="downloading()" aria-label="Download" i18n-aria-label>
@if (downloading()) {
<div class="spinner-border spinner-border-sm" role="status"></div>
} @else {
@@ -45,7 +45,7 @@
</button>
<div class="btn-group" ngbDropdown role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle" [disabled]="downloading()" ngbDropdownToggle></button>
<button class="btn btn-sm btn-outline-primary dropdown-toggle" [disabled]="downloading()" ngbDropdownToggle aria-label="Download options" i18n-aria-label></button>
<div class="dropdown-menu shadow" ngbDropdownMenu>
@if (metadata()?.has_archive_version) {
<button ngbDropdownItem (click)="download(true)" [disabled]="downloading()" i18n>Download original</button>
@@ -62,7 +62,7 @@
</div>
<div class="ms-auto" ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="actionsDropdown" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="actionsDropdown" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow">
@@ -91,7 +91,7 @@
</div>
<div class="ms-auto" ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="sendDropdown" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="sendDropdown" ngbDropdownToggle aria-label="Send" i18n-aria-label>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow">
@@ -963,12 +963,24 @@ describe('DocumentDetailComponent', () => {
component.reprocess()
const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] })
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, false)
expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
})
it('should pass remote OCR choice when reprocessing', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
reprocessSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component.reprocess()
openModal.componentInstance.remoteOcr = true
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, true)
})
it('should show error if redo ocr call fails', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
@@ -2171,6 +2183,11 @@ describe('DocumentDetailComponent', () => {
).toBe(10)
component.openEmailDocument()
expect(modalSpy).toHaveBeenCalled()
expect(
(
modalSpy.mock.results[1].value as NgbModalRef
).componentInstance.documentIds()
).toEqual([10])
})
it('should set previewText', () => {
@@ -97,6 +97,7 @@ import { ISODateAdapter } from 'src/app/utils/ngb-iso-date-adapter'
import * as UTIF from 'utif'
import { DocumentDetailFieldID } from '../admin/settings/settings.component'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
@@ -1398,7 +1399,7 @@ export class DocumentDetailComponent
}
reprocess() {
let modal = this.modalService.open(ConfirmDialogComponent, {
let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Reprocess confirm`
@@ -1409,7 +1410,10 @@ export class DocumentDetailComponent
modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled.set(false)
this.documentsService
.reprocessDocuments({ documents: [this.document().id] })
.reprocessDocuments(
{ documents: [this.document().id] },
modal.componentInstance.remoteOcr
)
.subscribe({
next: () => {
this.toastService.showInfo(
@@ -1973,7 +1977,9 @@ export class DocumentDetailComponent
const modal = this.modalService.open(EmailDocumentDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.documentIds.set([this.document().id])
modal.componentInstance.documentIds.set([
this.selectedVersionId() ?? this.document().id,
])
modal.componentInstance.hasArchiveVersion.set(
this.metadata()?.has_archive_version ??
!!this.document()?.archived_file_name
@@ -1,7 +1,10 @@
<div class="btn-group" ngbDropdown autoClose="outside">
<button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle aria-label="Versions" i18n-aria-label>
<i-bs name="file-earmark-diff"></i-bs>
<span class="d-none d-lg-inline ps-1" i18n>Versions</span>
@if (versions.length > 1) {
<span class="badge text-bg-secondary ms-1">{{ versions.length }}</span>
}
</button>
<div class="dropdown-menu shadow" ngbDropdownMenu>
<div class="px-3 py-2 mb-2">
@@ -1,6 +1,6 @@
<h6>
<button type="button" class="btn btn-outline-secondary btn-sm me-2"
(click)="expand = !expand">
(click)="expand = !expand" aria-label="Toggle document metadata" i18n-aria-label>
@if (!expand) {
<i-bs width="1.2em" height="1.2em" name="caret-down"></i-bs>
}
@@ -74,7 +74,7 @@
</pngx-filterable-dropdown>
}
<div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll">
<button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll" aria-label="Permissions" i18n-aria-label>
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Permissions</ng-container></div>
</button>
</div>
@@ -82,7 +82,7 @@
<div class="d-flex align-items-center gap-2 ms-auto">
<div class="btn-toolbar">
<div ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
@@ -107,6 +107,8 @@
id="dropdownSend"
ngbDropdownToggle
[disabled]="disabled || !canSendSelection"
aria-label="Send"
i18n-aria-label
>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container>
</div>
@@ -127,7 +129,7 @@
</div>
</div>
<div class="btn-group btn-group-sm">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()" aria-label="Download" i18n-aria-label>
@if (!awaitingDownload()) {
<i-bs name="arrow-down"></i-bs>
}
@@ -139,7 +141,7 @@
<div class="d-none d-sm-inline ms-1"><ng-container i18n>Download</ng-container></div>
</button>
<div ngbDropdown class="me-2 d-flex btn-group" role="group">
<button type="button" class="btn btn-sm btn-outline-primary dropdown-toggle-split rounded-end" ngbDropdownToggle></button>
<button type="button" class="btn btn-sm btn-outline-primary dropdown-toggle-split rounded-end" ngbDropdownToggle aria-label="Download options" i18n-aria-label></button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
<form [formGroup]="downloadForm" class="px-3 py-1">
<p class="mb-1" i18n>Include:</p>
@@ -1,5 +1,5 @@
.dropdown-toggle-split {
--bs-border-radius: .25rem;
--bs-border-radius: .45rem;
}
.dropdown-menu{
@@ -1122,6 +1122,7 @@ describe('BulkEditorComponent', () => {
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
remote_ocr: false,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -52,6 +52,7 @@ import { flattenTags } from 'src/app/utils/flatten-tags'
import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dialog/merge-as-versions-confirm-dialog/merge-as-versions-confirm-dialog.component'
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
@@ -916,7 +917,7 @@ export class BulkEditorComponent
}
reprocessSelected() {
let modal = this.modalService.open(ConfirmDialogComponent, {
let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Reprocess confirm`
@@ -930,7 +931,10 @@ export class BulkEditorComponent
modal.componentInstance.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
this.documentService.reprocessDocuments(this.getSelectionQuery())
this.documentService.reprocessDocuments(
this.getSelectionQuery(),
modal.componentInstance.remoteOcr
)
)
})
}
@@ -61,7 +61,7 @@
</pngx-input-textarea>
}
}
<button type="button" class="btn btn-outline-danger mb-3" (click)="removeField(field.id)">
<button type="button" class="btn btn-outline-danger mb-3" (click)="removeField(field.id)" aria-label="Remove custom field" i18n-aria-label>
<i-bs name="x"></i-bs>
</button>
</div>
@@ -154,9 +154,9 @@
<i-bs name="download"></i-bs>
</a>
} @else {
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button>
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button>
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button>
<span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
<span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
<span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
}
</div>
</div>
@@ -1,6 +1,6 @@
<pngx-page-header [title]="getTitle()">
<div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle aria-label="Select" i18n-aria-label>
<i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div>
@if (list.hasSelection) {
<pngx-clearable-badge [selected]="list.hasSelection" [number]="list.selectedCount" (cleared)="list.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
@@ -31,7 +31,7 @@
</div>
</div>
<div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle aria-label="Show" i18n-aria-label>
<i-bs name="card-heading"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Show</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownDisplayFields" class="shadow">
@@ -61,7 +61,7 @@
</div>
<div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle>
<button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle aria-label="Sort" i18n-aria-label>
<i-bs name="arrow-down-up"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Sort</ng-container></div>
</button>
<div ngbDropdownMenu aria-labelledby="dropdownBasic1" class="shadow dropdown-menu-right">
@@ -85,8 +85,8 @@
</div>
</div>
<div class="btn-group flex-fill" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" ngbDropdown role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle flex-fill" tourAnchor="tour.documents-views" ngbDropdownToggle>
<div class="btn-group flex-fill" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" ngbDropdown #viewsDropdown="ngbDropdown" role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle flex-fill" tourAnchor="tour.documents-views" ngbDropdownToggle aria-label="Views" i18n-aria-label>
<i-bs name="window-stack"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Views</ng-container></div>
@if (savedViewIsModified) {
<div class="position-absolute top-0 start-100 p-2 translate-middle badge bg-secondary border border-light rounded-circle">
@@ -95,15 +95,15 @@
}
</button>
<div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu>
@if (!list.activeSavedViewId) {
@for (view of savedViewService.allViews; track view) {
<button ngbDropdownItem (click)="loadViewConfig(view.id)">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
</button>
}
@if (savedViewService.allViews.length > 0) {
<div class="dropdown-divider"></div>
}
@if (viewsDropdown.isOpen() && !list.activeSavedViewId && savedViewService.allViews.length > 0) {
<div class="views-list overflow-y-auto">
@for (view of savedViewService.allViews; track view.id) {
<button ngbDropdownItem (click)="loadViewConfig(view.id)">
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
</button>
}
</div>
<div class="dropdown-divider"></div>
}
@if (list.activeSavedViewId && activeSavedViewCanChange) {
@@ -157,7 +157,7 @@
</div>
</ng-template>
<div tourAnchor="tour.documents">
<div class="mt-3 mb-n2" tourAnchor="tour.documents">
<ng-container *ngTemplateOutlet="pagination"></ng-container>
</div>
@@ -56,11 +56,11 @@ $paperless-card-breakpoints: (
.sticky-top {
z-index: 990; // below main navbar
top: calc(7rem - 2px); // height of navbar + search row (mobile)
top: calc(7.5rem - 2px); // height of navbar + search row (mobile)
transition: top 0.2s ease;
@media (min-width: 580px) {
top: 3.5rem; // height of navbar
top: 4.5em; // height of navbar
}
}
@@ -73,7 +73,7 @@ $paperless-card-breakpoints: (
@media (max-width: 579.98px) {
:host-context(main.mobile-search-hidden) .sticky-top {
top: calc(3.5rem - 2px); // height of navbar only when search is hidden
top: calc(4rem - 2px); // height of navbar only when search is hidden
}
}
@@ -93,4 +93,8 @@ a {
pngx-page-header .dropdown-menu {
--bs-dropdown-min-width: 12em;
.views-list {
max-height: min(400px, calc(100vh - 260px)); // leave room for the header above and the actions below
}
}
@@ -18,7 +18,7 @@
</select>
}
@if (_textFilter) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()">
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs>
</button>
}
@@ -24,7 +24,7 @@
<div class="btn-toolbar gap-2">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -7,7 +7,7 @@
>
@if (activeManagementList) {
<div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle aria-label="Select" i18n-aria-label>
<i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div>
@if (activeManagementList.hasSelection) {
<pngx-clearable-badge [selected]="activeManagementList.hasSelection" [number]="activeManagementList.selectedCount" (cleared)="activeManagementList.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
@@ -1,7 +1,7 @@
<div class="row mb-3">
<div class="col mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<label class="text-muted me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<label class="me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<input id="managementNameFilter" class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
</div>
</div>
@@ -9,7 +9,7 @@
<div class="col-auto mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center">
<div class="input-group input-group-sm w-auto d-none d-md-flex">
<label class="input-group-text border-0" for="managementPageSize" i18n>Show:</label>
<label class="input-group-text bg-transparent border-0" for="managementPageSize" i18n>Show:</label>
</div>
<div class="input-group input-group-sm w-auto me-3">
<select id="managementPageSize" class="form-select form-select-sm small" [(ngModel)]="pageSize">
@@ -113,7 +113,7 @@
<div class="btn-toolbar gap-2">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -58,7 +58,7 @@
<div class="col">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -146,7 +146,7 @@
<div class="col-3">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -47,7 +47,7 @@
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs>
</button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
+55
View File
@@ -54,6 +54,10 @@ export const ConfigCategory = {
AI: $localize`AI Settings`,
}
export const ConfigSection = {
RemoteOCR: $localize`Remote OCR`,
}
export const LLMEmbeddingBackendConfig = {
OPENAI_LIKE: 'openai-like',
HUGGINGFACE: 'huggingface',
@@ -65,6 +69,15 @@ export const LLMBackendConfig = {
OLLAMA: 'ollama',
}
export const RemoteOCREngineConfig = {
AZURE_AI: 'azureai',
}
export const RemoteOCRModeConfig = {
ALWAYS: 'always',
WORKFLOW_ONLY: 'workflow_only',
}
export interface ConfigOption {
key: string
title: string
@@ -72,6 +85,7 @@ export interface ConfigOption {
choices?: Array<{ id: string; name: string }>
config_key?: string
category: string
section?: string
note?: string
}
@@ -181,6 +195,43 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_OCR_USER_ARGS',
category: ConfigCategory.OCR,
},
{
key: 'remote_ocr_engine',
title: $localize`Remote OCR Engine`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCREngineConfig),
config_key: 'PAPERLESS_REMOTE_OCR_ENGINE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Enabling remote OCR sends documents to a third-party service for processing. Consider the privacy implications as well as potential costs before enabling.`,
},
{
key: 'remote_ocr_api_key',
title: $localize`Remote OCR API Key`,
type: ConfigOptionType.Password,
config_key: 'PAPERLESS_REMOTE_OCR_API_KEY',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
},
{
key: 'remote_ocr_endpoint',
title: $localize`Remote OCR Endpoint`,
type: ConfigOptionType.String,
config_key: 'PAPERLESS_REMOTE_OCR_ENDPOINT',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Required when using the Azure AI engine.`,
},
{
key: 'remote_ocr_mode',
title: $localize`Remote OCR Mode`,
type: ConfigOptionType.Select,
choices: mapToItems(RemoteOCRModeConfig),
config_key: 'PAPERLESS_REMOTE_OCR_MODE',
category: ConfigCategory.OCR,
section: ConfigSection.RemoteOCR,
note: $localize`Which documents are sent to the remote engine. Use 'workflow_only' to keep remote OCR off unless a workflow enables it for a document.`,
},
{
key: 'app_logo',
title: $localize`Application Logo`,
@@ -398,6 +449,10 @@ export interface PaperlessConfig extends ObjectWithId {
barcode_enable_tag: boolean
barcode_tag_mapping: object
barcode_tag_split: boolean
remote_ocr_engine: string
remote_ocr_api_key: string
remote_ocr_endpoint: string
remote_ocr_mode: string
ai_enabled: boolean
llm_embedding_backend: string
llm_embedding_model: string
+13
View File
@@ -1,5 +1,6 @@
import { PdfEditorEditMode } from '../components/common/pdf-editor/pdf-editor-edit-mode'
import { PdfZoomScale } from '../components/common/pdf-viewer/pdf-viewer.types'
import { RemoteOCRModeConfig } from './paperless-config'
import { User } from './user'
export interface UiSettings {
@@ -94,6 +95,8 @@ export const SETTINGS_KEYS = {
OUTLOOK_OAUTH_URL: 'outlook_oauth_url',
EMAIL_ENABLED: 'email_enabled',
AI_ENABLED: 'ai_enabled',
REMOTE_OCR_CONFIGURED: 'remote_ocr:configured',
REMOTE_OCR_MODE: 'remote_ocr:mode',
}
export const SETTINGS: UiSetting[] = [
@@ -347,4 +350,14 @@ export const SETTINGS: UiSetting[] = [
type: 'string',
default: PdfEditorEditMode.Create,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_CONFIGURED,
type: 'boolean',
default: false,
},
{
key: SETTINGS_KEYS.REMOTE_OCR_MODE,
type: 'string',
default: RemoteOCRModeConfig.ALWAYS,
},
]
+1
View File
@@ -7,6 +7,7 @@ export enum WorkflowActionType {
Webhook = 4,
PasswordRemoval = 5,
MoveToTrash = 6,
RemoteOcr = 7,
}
export interface WorkflowActionEmail extends ObjectWithId {
@@ -284,6 +284,21 @@ describe(`DocumentService`, () => {
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: false,
})
})
it('should request remote OCR when reprocessing with it enabled', () => {
const ids = [1, 2, 3]
subscription = service
.reprocessDocuments({ documents: ids }, true)
.subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/reprocess/`
)
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: true,
})
})
@@ -349,9 +349,13 @@ export class DocumentService extends AbstractPaperlessService<Document> {
})
}
reprocessDocuments(selection: DocumentSelectionQuery) {
reprocessDocuments(
selection: DocumentSelectionQuery,
remoteOcr: boolean = false
) {
return this.http.post(this.getResourceUrl(null, 'reprocess'), {
...selection,
remote_ocr: remoteOcr,
})
}
@@ -13,6 +13,7 @@ import { environment } from 'src/environments/environment'
import { CustomFieldDataType } from '../data/custom-field'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { SavedView } from '../data/saved-view'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings'
import { PermissionsService } from './permissions.service'
import { CustomFieldsService } from './rest/custom-fields.service'
@@ -434,4 +435,26 @@ describe('SettingsService', () => {
).name
).toEqual(customFields[0].name)
})
it('should offer remote OCR only when configured and selective', () => {
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, false)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
// configured, but already handling every document
settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, true)
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.ALWAYS
)
expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
settingsService.set(
SETTINGS_KEYS.REMOTE_OCR_MODE,
RemoteOCRModeConfig.WORKFLOW_ONLY
)
expect(settingsService.remoteOCRIsSelectable).toBeTruthy()
})
})
@@ -19,6 +19,7 @@ import {
} from 'src/app/utils/color'
import { DEFAULT_APP_TITLE, environment } from 'src/environments/environment'
import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
import { RemoteOCRModeConfig } from '../data/paperless-config'
import { SavedView } from '../data/saved-view'
import {
PAPERLESS_GREEN_HEX,
@@ -687,6 +688,17 @@ export class SettingsService {
return this.settingIsSet(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)
}
/**
* Offering remote OCR as a choice only makes sense when an engine
* is configured but is not already handling every document.
*/
get remoteOCRIsSelectable(): boolean {
return (
this.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
this.get(SETTINGS_KEYS.REMOTE_OCR_MODE) !== RemoteOCRModeConfig.ALWAYS
)
}
offerTour(): boolean {
return this.dashboardIsEmpty() && !this.get(SETTINGS_KEYS.TOUR_COMPLETE)
}
@@ -438,6 +438,21 @@ describe('ConsumerStatusService', () => {
expect(updated).toBeTruthy()
})
it('should ignore keep-alive heartbeat messages from the server', () => {
let updated = false
let deleted = false
websocketStatusService.onDocumentUpdated().subscribe(() => (updated = true))
websocketStatusService.onDocumentDeleted().subscribe(() => (deleted = true))
websocketStatusService.connect()
server.send({ type: WebsocketStatusType.HEARTBEAT })
expect(updated).toBeFalsy()
expect(deleted).toBeFalsy()
expect(websocketStatusService.getConsumerStatus()).toHaveLength(0)
websocketStatusService.disconnect()
})
it('should ignore document updated events the user cannot view', () => {
let updated = false
websocketStatusService.onDocumentUpdated().subscribe(() => {
@@ -11,6 +11,7 @@ export enum WebsocketStatusType {
STATUS_UPDATE = 'status_update',
DOCUMENTS_DELETED = 'documents_deleted',
DOCUMENT_UPDATED = 'document_updated',
HEARTBEAT = 'heartbeat',
}
// see ProgressStatusOptions in src/documents/plugins/helpers.py
@@ -207,6 +208,10 @@ export class WebsocketStatusService {
case WebsocketStatusType.STATUS_UPDATE:
this.handleProgressUpdate(messageData as WebsocketProgressMessage)
break
case WebsocketStatusType.HEARTBEAT:
// keep-alive from the server, see paperless.consumers.StatusConsumer
break
}
}
}
+2
View File
@@ -118,6 +118,7 @@ import {
journalBookmarkFill,
journals,
link,
list,
listNested,
listTask,
listUl,
@@ -365,6 +366,7 @@ const icons = {
journalBookmarkFill,
journals,
link,
list,
listNested,
listTask,
listUl,
+38 -2
View File
@@ -1,7 +1,7 @@
@use "sass:meta";
// bs variables
$grid-gutter-width: 1.5rem;
$border-radius: .375rem;
$border-radius: .425rem;
$btn-border-width: var(--bs-border-width);
$form-file-button-bg: var(--bs-body-bg);
@@ -68,15 +68,44 @@ body {
--pngx-body-font-size: 0.875rem;
font-size: var(--pngx-body-font-size);
height: 100vh;
letter-spacing: -0.005em;
}
* {
transition: background-color 0.3s ease, border-color 0.3s ease;
}
.card,
.dropdown-menu,
.modal-content,
.popover {
--bs-card-border-color: color-mix(in srgb, var(--bs-border-color) 72%, transparent);
border-radius: .55rem;
}
.card {
box-shadow: 0 1px 2px rgba(0, 0, 0, .04), 0 8px 24px rgba(0, 0, 0, .035);
}
.btn {
--bs-btn-border-radius: .425rem;
--bs-border-radius-sm: .425rem;
font-weight: 500;
}
.form-control,
.form-select,
.input-group-text {
border-radius: .425rem;
}
.pagination, .input-group {
--bs-border-radius-sm: .425rem;
}
@media(min-width: 768px) {
.col-slim {
padding-left: calc(50px + $grid-gutter-width) !important;
padding-left: calc(56px + $grid-gutter-width) !important;
}
}
@@ -507,6 +536,7 @@ ul.pagination {
}
.nav-underline {
gap: calc(0.5 * var(--bs-nav-underline-gap)) var(--bs-nav-underline-gap);
.nav-link {
&.active, &:hover, &:focus {
color: var(--bs-primary);
@@ -660,6 +690,10 @@ table.table {
--bs-toast-max-width: var(--pngx-toast-max-width);
}
.alert {
--bs-border-radius: .425rem;
}
.alert-primary {
--bs-alert-color: var(--bs-primary);
--bs-alert-bg: var(--pngx-primary-faded);
@@ -791,6 +825,8 @@ code {
--bs-accordion-bg: var(--bs-light);
--bs-accordion-active-color: var(--bs-primary);
--bs-accordion-active-bg: var(--pngx-bg-alt);
--bs-border-radius: .425rem;
--bs-accordion-inner-border-radius: calc(.425rem - 1px);
}
.accordion-button::after {
+3 -3
View File
@@ -20,7 +20,7 @@
--pngx-primary-darken-27: hsl(var(--pngx-primary), calc(var(--pngx-primary-lightness) - 27%));
--pngx-success-darken-10: hsl(152, 69%, 11%); // based on success #198754
--pngx-bg-alt: #fff;
--pngx-bg-darker: var(--bs-gray-100);
--pngx-bg-darker: #f4f6f5;
--pngx-bg-alt2: var(--bs-gray-200); // #e9ecef
--pngx-bg-disabled: #f7f7f7;
--pngx-card-hover-border: var(--bs-tertiary-color);
@@ -91,7 +91,7 @@ $form-check-radio-checked-bg-image-dark: url("data:image/svg+xml,%3csvg xmlns='h
--pngx-body-color-accent: #{$text-color-dark-bg-accent};
--pngx-bg-alt: #242529;
--pngx-bg-alt2: #232323;
--pngx-bg-darker: #101216;
--pngx-bg-darker: #121315;
--pngx-bg-disabled: var(--pngx-bg-alt);
--pngx-card-hover-border: var(--bs-border-color);
--pngx-focus-alpha: 0.6;
@@ -107,7 +107,7 @@ $form-check-radio-checked-bg-image-dark: url("data:image/svg+xml,%3csvg xmlns='h
--bs-light-rgb: 28, 28, 31;
--bs-info: var(--pngx-bg-alt);
--bs-info-rgb: 36, 36, 39;
--bs-border-color: #47494f;
--bs-border-color: #34373d;
--bs-tertiary-bg: var(--pngx-bg-darker);
--bs-dark-border-subtle: var(--pngx-bg-darker);
--bs-border-color-translucent: rgba(0, 0, 0, .175); // override bs
+8 -2
View File
@@ -399,10 +399,16 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
return "OK"
def reprocess(doc_ids: list[int]) -> Literal["OK"]:
def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
"""
Re-run parsing for the given documents.
Consumption workflows do not run here, so ``remote_ocr`` is how the user
asks for the remote engine when it is not configured to handle everything.
"""
for document_id in doc_ids:
update_document_content_maybe_archive_file.apply_async(
kwargs={"document_id": document_id},
kwargs={"document_id": document_id, "remote_ocr": remote_ocr},
headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
)
+18
View File
@@ -53,6 +53,7 @@ from documents.utils import copy_basic_file_stats
from documents.utils import copy_file_with_basic_stats
from documents.utils import run_subprocess
from paperless.config import OcrConfig
from paperless.config import RemoteOCRConfig
from paperless.models import ArchiveFileGenerationChoices
from paperless.parsers import ParserContext
from paperless.parsers import ParserProtocol
@@ -451,12 +452,19 @@ class ConsumerPlugin(
except Exception as e:
self.log.error(f"Error attempting to clean PDF: {e}")
# Workflows have already run at this point, so the metadata knows
# whether this document was singled out for remote OCR
allow_remote = (
self.metadata.remote_ocr or RemoteOCRConfig().remote_ocr_by_default
)
# Based on the mime type, get the parser for that type
parser_class: type[ParserProtocol] | None = (
get_parser_registry().get_parser_for_file(
mime_type,
self.filename,
self.working_copy,
allow_remote=allow_remote,
)
)
if not parser_class:
@@ -465,6 +473,16 @@ class ConsumerPlugin(
f"Unsupported mime type {mime_type}",
)
if self.metadata.remote_ocr and not getattr(
parser_class,
"uses_remote_service",
False,
):
self.log.warning(
"Remote OCR was requested for this document but no remote "
"parser is available for it, processing locally instead.",
)
# Notify all listeners that we're going to do some work.
document_consumption_started.send(
+4 -1
View File
@@ -34,6 +34,7 @@ class DocumentMetadataOverrides:
skip_asn_if_exists: bool = False
version_label: str | None = None
actor_id: int | None = None
remote_ocr: bool = False
def update(self, other: "DocumentMetadataOverrides") -> "DocumentMetadataOverrides":
"""
@@ -57,6 +58,8 @@ class DocumentMetadataOverrides:
self.actor_id = other.actor_id
if other.skip_asn_if_exists:
self.skip_asn_if_exists = True
if other.remote_ocr:
self.remote_ocr = True
if other.version_label is not None:
self.version_label = other.version_label
@@ -126,7 +129,7 @@ class DocumentMetadataOverrides:
)
overrides.custom_fields = {
custom_field.field.id: custom_field.value
for custom_field in doc.custom_fields.all()
for custom_field in doc.custom_fields.select_related("field").all()
}
groups_with_perms = get_groups_with_perms(
@@ -0,0 +1,30 @@
# Generated by Django 5.2.16 on 2026-08-10 17:27
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0023_savedview_icon"),
]
operations = [
migrations.AlterField(
model_name="workflowaction",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Assignment"),
(2, "Removal"),
(3, "Email"),
(4, "Webhook"),
(5, "Password removal"),
(6, "Move to trash"),
(7, "Remote OCR"),
],
default=1,
verbose_name="Workflow Action Type",
),
),
]
+4
View File
@@ -1670,6 +1670,10 @@ class WorkflowAction(models.Model):
6,
_("Move to trash"),
)
REMOTE_OCR = (
7,
_("Remote OCR"),
)
type = models.PositiveSmallIntegerField(
_("Workflow Action Type"),
+2 -10
View File
@@ -6,20 +6,13 @@ from documents.search._backend import TantivyRelevanceList
from documents.search._backend import WriteBatch
from documents.search._backend import get_backend
from documents.search._backend import reset_backend
from documents.search._errors import InvalidDateQuery
from documents.search._errors import InvalidNumberQuery
from documents.search._errors import MultipleSearchQueryErrors
from documents.search._errors import QueryTooLongError
from documents.search._errors import SearchQueryError
from documents.search._errors import search_query_error_messages
from documents.search._schema import needs_rebuild
from documents.search._schema import wipe_index
from documents.search._translate import InvalidDateQuery
from documents.search._translate import SearchQueryError
__all__ = [
"InvalidDateQuery",
"InvalidNumberQuery",
"MultipleSearchQueryErrors",
"QueryTooLongError",
"SearchHit",
"SearchIndexLockError",
"SearchMode",
@@ -30,6 +23,5 @@ __all__ = [
"get_backend",
"needs_rebuild",
"reset_backend",
"search_query_error_messages",
"wipe_index",
]
+22 -84
View File
@@ -20,11 +20,9 @@ from typing import cast
import filelock
import tantivy
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.utils.timezone import get_current_timezone
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
from documents.search._query import build_permission_filter
from documents.search._query import extract_cjk_text
from documents.search._query import parse_simple_text_highlight_query
from documents.search._query import parse_simple_text_query
@@ -42,7 +40,6 @@ from documents.utils import QuerySetStream
from documents.utils import identity
if TYPE_CHECKING:
from collections.abc import Iterable
from collections.abc import Iterator
from collections.abc import Sequence
from pathlib import Path
@@ -266,11 +263,7 @@ class WriteBatch:
if self._lock is not None:
self._lock.release()
def add_or_update(
self,
document: Document,
effective_content: str | None = None,
) -> None:
def add_or_update(self, document: Document) -> None:
"""
Add or update a document in the batch.
@@ -280,11 +273,9 @@ class WriteBatch:
Args:
document: Django Document instance to index
effective_content: Override document.content for indexing (used when
re-indexing with newer OCR text from document versions)
"""
self.remove(document.pk)
doc = self._backend._build_tantivy_doc(document, effective_content)
doc = self._backend._build_tantivy_doc(document)
self._writer.add_document(doc)
def remove(self, doc_id: int) -> None:
@@ -294,47 +285,6 @@ class WriteBatch:
)
def build_permission_filter(
schema: tantivy.Schema,
user: AbstractUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
Creates a query that matches only documents visible to the specified user
according to paperless-ngx permission rules:
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
"""
owner_any = tantivy.Query.exists_query("owner_id")
no_owner = tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, tantivy.Query.all_query()),
(tantivy.Occur.MustNot, owner_any),
],
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
class TantivyBackend:
"""
Tantivy search backend with explicit lifecycle management.
@@ -466,18 +416,20 @@ class TantivyBackend:
def _build_tantivy_doc(
self,
document: Document,
effective_content: str | None = None,
viewer_ids: list[int] | None = None,
viewer_group_ids: list[int] | None = None,
) -> tantivy.Document:
"""Build a tantivy Document from a Django Document instance.
``effective_content`` overrides ``document.content`` for indexing
used when re-indexing a root document with a newer version's OCR text.
A root document is indexed with its effective content, i.e. the newest
version's OCR text, so it is never indexed with its own outdated text.
Annotate the queryset with ``annotate_effective_content`` when indexing
more than a couple of documents, to resolve that without a query each.
"""
content = (
effective_content if effective_content is not None else document.content
)
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
content = document.get_effective_content() or ""
doc = tantivy.Document()
@@ -506,6 +458,7 @@ class TantivyBackend:
doc.add_text("correspondent_sort", document.correspondent.name)
if cjk_corr := extract_cjk_text(document.correspondent.name):
doc.add_text("bigram_correspondent", cjk_corr)
doc.add_unsigned("correspondent_id", document.correspondent_id)
# Document type
if document.document_type:
@@ -513,10 +466,12 @@ class TantivyBackend:
doc.add_text("type_sort", document.document_type.name)
if cjk_type := extract_cjk_text(document.document_type.name):
doc.add_text("bigram_document_type", cjk_type)
doc.add_unsigned("document_type_id", document.document_type_id)
# Storage path
if document.storage_path:
doc.add_text("storage_path", document.storage_path.name)
doc.add_unsigned("storage_path_id", document.storage_path_id)
# Tags — collect names for autocomplete in the same pass
tag_names: list[str] = []
@@ -524,13 +479,12 @@ class TantivyBackend:
doc.add_text("tag", tag.name)
if cjk_tag := extract_cjk_text(tag.name):
doc.add_text("bigram_tag", cjk_tag)
doc.add_unsigned("tag_id", tag.pk)
tag_names.append(tag.name)
# Notes — JSON for structured queries (notes.user:alice, notes.note:text).
# notes_text is a plain-text companion for snippet/highlight generation;
# tantivy's SnippetGenerator does not support JSON fields. It is not in
# _DEFAULT_SEARCH_FIELDS, so an unqualified query never searches it: a
# note matches through the JSON field or not at all.
# tantivy's SnippetGenerator does not support JSON fields.
num_notes = 0
note_texts: list[str] = []
for note in document.notes.all():
@@ -546,9 +500,8 @@ class TantivyBackend:
if note_texts:
doc.add_text("notes_text", " ".join(note_texts))
# Custom fields — JSON for structured queries (custom_fields.name:x,
# custom_fields.value:y). There is no companion text field here, unlike
# notes: custom field values are reachable only through the JSON field.
# Custom fields — JSON for structured queries (custom_fields.name:x, custom_fields.value:y),
# companion text field for default full-text search.
for cfi in document.custom_fields.all():
search_value = cfi.value_for_search
# Skip fields where there is no value yet
@@ -624,11 +577,7 @@ class TantivyBackend:
return doc
def add_or_update(
self,
document: Document,
effective_content: str | None = None,
) -> None:
def add_or_update(self, document: Document) -> None:
"""
Add or update a single document with file locking.
@@ -641,12 +590,11 @@ class TantivyBackend:
Args:
document: Django Document instance to index
effective_content: Override document.content for indexing
"""
self._ensure_open()
try:
with self.batch_update(lock_timeout=_LOCK_TIMEOUT_SECONDS) as batch:
batch.add_or_update(document, effective_content)
batch.add_or_update(document)
except SearchIndexLockError:
logger.error(
"Search index lock exhausted for document %d after %d attempts; "
@@ -720,17 +668,7 @@ class TantivyBackend:
user_query = self._parse_query(query, search_mode)
highlight_query = user_query
if search_mode is SearchMode.TEXT:
try:
highlight_query = parse_simple_text_highlight_query(
self._index,
query,
)
except ValueError:
logger.debug(
"Skipping simple text highlight query: token string is not "
"valid tantivy query syntax: %r",
query,
)
highlight_query = parse_simple_text_highlight_query(self._index, query)
# For notes_text snippet generation, we need a query that targets the
# notes_text field directly. user_query may contain JSON-field terms
@@ -1077,7 +1015,6 @@ class TantivyBackend:
):
doc = self._build_tantivy_doc(
document,
document.get_effective_content(),
viewer_ids=viewer_ids,
viewer_group_ids=viewer_group_ids,
)
@@ -1145,6 +1082,7 @@ def _bulk_get_viewer_permissions(
"""
from collections import defaultdict
from django.contrib.contenttypes.models import ContentType
from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission
+171
View File
@@ -0,0 +1,171 @@
from __future__ import annotations
from datetime import UTC
from datetime import date
from datetime import datetime
from datetime import timedelta
from typing import TYPE_CHECKING
from typing import Final
from dateutil.relativedelta import relativedelta
if TYPE_CHECKING:
from datetime import tzinfo
_DATE_ONLY_FIELDS = frozenset({"created"})
_TODAY: Final[str] = "today"
_YESTERDAY: Final[str] = "yesterday"
_PREVIOUS_WEEK: Final[str] = "previous week"
_THIS_MONTH: Final[str] = "this month"
_PREVIOUS_MONTH: Final[str] = "previous month"
_THIS_YEAR: Final[str] = "this year"
_PREVIOUS_YEAR: Final[str] = "previous year"
_PREVIOUS_QUARTER: Final[str] = "previous quarter"
_DATE_KEYWORDS = frozenset(
{
_TODAY,
_YESTERDAY,
_PREVIOUS_WEEK,
_THIS_MONTH,
_PREVIOUS_MONTH,
_THIS_YEAR,
_PREVIOUS_YEAR,
_PREVIOUS_QUARTER,
},
)
def _fmt(dt: datetime) -> str:
"""Format a datetime as an ISO 8601 UTC string for use in Tantivy range queries."""
return dt.astimezone(UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
def _iso_range(lo: datetime, hi: datetime) -> str:
"""
Format a half-open ``[lo TO hi)`` range in ISO 8601 for Tantivy query syntax.
``hi`` is always the exclusive ceiling of a computed period (the start of
the *next* day/week/month/quarter/year), so the closing bracket must be
the Tantivy exclusive-range brace ``}`` rather than ``]`` otherwise the
first instant of the following period (e.g. the 1st of next month) is
incorrectly included in the match.
"""
return f"[{_fmt(lo)} TO {_fmt(hi)}}}"
def _quarter_start(d: date) -> date:
"""Return the first day of the calendar quarter containing ``d``."""
return date(d.year, ((d.month - 1) // 3) * 3 + 1, 1)
def _midnight(d: date, tz: tzinfo) -> datetime:
"""Convert a calendar date at local-timezone midnight to a UTC datetime."""
return datetime(d.year, d.month, d.day, tzinfo=tz).astimezone(UTC)
def _keyword_bounds(keyword: str, tz: tzinfo) -> tuple[date, date]:
"""
Map a relative date keyword to ``(start, exclusive_end)`` calendar dates.
``tz`` only determines what "today" is; the caller decides how the returned
dates become UTC datetime boundaries (date-only vs. local-midnight offset).
"""
today = datetime.now(tz).date()
if keyword == _TODAY:
return today, today + timedelta(days=1)
if keyword == _YESTERDAY:
return today - timedelta(days=1), today
if keyword == _PREVIOUS_WEEK:
this_monday = today - timedelta(days=today.weekday())
return this_monday - timedelta(weeks=1), this_monday
if keyword == _THIS_MONTH:
first = today.replace(day=1)
return first, first + relativedelta(months=1)
if keyword == _PREVIOUS_MONTH:
this_first = today.replace(day=1)
return this_first - relativedelta(months=1), this_first
if keyword == _THIS_YEAR:
return date(today.year, 1, 1), date(today.year + 1, 1, 1)
if keyword == _PREVIOUS_YEAR:
return date(today.year - 1, 1, 1), date(today.year, 1, 1)
if keyword == _PREVIOUS_QUARTER:
this_quarter = _quarter_start(today)
return this_quarter - relativedelta(months=3), this_quarter
raise ValueError(f"Unknown keyword: {keyword}")
def _date_only_range(keyword: str, tz: tzinfo) -> str:
"""
For `created` (DateField): use the local calendar date, converted to
midnight UTC boundaries. No offset arithmetic date only.
"""
start, end = _keyword_bounds(keyword, tz)
lo = datetime(start.year, start.month, start.day, tzinfo=UTC)
hi = datetime(end.year, end.month, end.day, tzinfo=UTC)
return _iso_range(lo, hi)
def _datetime_range(keyword: str, tz: tzinfo) -> str:
"""
For `added` / `modified` (DateTimeField, stored as UTC): convert local day
boundaries to UTC full offset arithmetic required.
"""
start, end = _keyword_bounds(keyword, tz)
return _iso_range(_midnight(start, tz), _midnight(end, tz))
def _precision_bounds(digits: str) -> tuple[date, date] | None:
"""
Map a 4/6/8-digit date token to (start, exclusive_end) calendar dates.
YYYY -> whole year, YYYYMM -> whole month, YYYYMMDD -> single day.
Returns None for any unparsable or out-of-range value (e.g. month 23),
so callers can emit a no-match clause instead of erroring (Whoosh parity).
"""
try:
if len(digits) == 4:
year = int(digits)
return date(year, 1, 1), date(year + 1, 1, 1)
if len(digits) == 6:
year, month = int(digits[:4]), int(digits[4:6])
start = date(year, month, 1)
end = date(year + 1, 1, 1) if month == 12 else date(year, month + 1, 1)
return start, end
if len(digits) == 8:
start = date(int(digits[:4]), int(digits[4:6]), int(digits[6:8]))
return start, start + timedelta(days=1)
except ValueError:
return None
return None
def _utc_bounds_for_field(
field: str,
start: date,
end: date,
tz: tzinfo,
) -> tuple[datetime, datetime]:
"""
Convert calendar-date bounds to UTC datetimes per the field's storage type.
For DateField (``created``) the bounds are UTC midnight (no offset). For
DateTimeField (``added``/``modified``) the bounds are local-tz midnight
converted to UTC, matching how each field is indexed.
"""
if field in _DATE_ONLY_FIELDS:
return (
datetime(start.year, start.month, start.day, tzinfo=UTC),
datetime(end.year, end.month, end.day, tzinfo=UTC),
)
return (
datetime(start.year, start.month, start.day, tzinfo=tz).astimezone(UTC),
datetime(end.year, end.month, end.day, tzinfo=tz).astimezone(UTC),
)
def _field_range_from_dates(field: str, start: date, end: date, tz: tzinfo) -> str:
"""Build a Tantivy ``field:[lo TO hi]`` ISO range from calendar-date bounds."""
lo, hi = _utc_bounds_for_field(field, start, end, tz)
return f"{field}:{_iso_range(lo, hi)}"
-71
View File
@@ -1,71 +0,0 @@
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from collections.abc import Sequence
class SearchQueryError(ValueError):
"""
Base for user-fixable search query errors.
Carries a message safe to surface to the user (no internal details). The
view layer catches this and returns an HTTP 400, so any future subclass
gets the same treatment.
"""
class InvalidDateQuery(SearchQueryError):
"""Raised when a date field value or range bound cannot be parsed."""
def __init__(self, field: str | None, value: str | None) -> None:
self.field = field
self.value = value
super().__init__(f"Invalid date value {value!r} for field {field!r}.")
class InvalidNumberQuery(SearchQueryError):
"""Raised when a numeric field value or range bound cannot be parsed."""
def __init__(self, field: str | None, value: str | None) -> None:
self.field = field
self.value = value
super().__init__(f"Invalid numeric value {value!r} for field {field!r}.")
class QueryTooLongError(SearchQueryError):
"""Raised when a query string exceeds the maximum allowed length.
whoosh-compat's fieldname tagger is O(n^2) in plain word characters, so an
unbounded query is a CPU-exhaustion vector against a single request
handler. This is a hard boundary, not a validation nicety.
"""
def __init__(self, length: int, limit: int) -> None:
self.length = length
self.limit = limit
super().__init__(
f"The search query is too long ({length} characters). "
f"The maximum allowed length is {limit} characters.",
)
class MultipleSearchQueryErrors(SearchQueryError):
"""Aggregates every user-fixable error from one parse, not just the first."""
def __init__(self, errors: Sequence[SearchQueryError]) -> None:
self.errors = tuple(errors)
super().__init__("; ".join(str(e) for e in self.errors))
def search_query_error_messages(e: SearchQueryError) -> list[str]:
"""The user-facing message list for a SearchQueryError.
Every offending value's message, not just the first, so the user can
fix them all in one round-trip. Shared by every view that maps
SearchQueryError to an HTTP 400.
"""
if isinstance(e, MultipleSearchQueryErrors):
return [str(sub) for sub in e.errors]
return [str(e)]
-42
View File
@@ -1,42 +0,0 @@
from __future__ import annotations
from whoosh_compat import FieldKind
from whoosh_compat import FieldSpec
from whoosh_compat import SubpathSpec
# Internal-only schema fields with no query-syntax meaning of their own
# (sort shadow fields, bigram CJK fields, simple_title/simple_content,
# autocomplete_word, notes_text) are NOT represented here — they are
# declared in _schema.py's field_descriptors().
#
# analyzer/pattern_normalizer are deliberately left at FieldSpec's default
# (None): they're language-specific and only meaningful to whoosh-compat's
# parser, so _registry.py attaches them per-language via dataclasses.replace()
# rather than PUBLIC_FIELDS declaring them itself. _schema.py only reads
# name/kind/fast and never sees the analyzer at all.
PUBLIC_FIELDS: tuple[FieldSpec, ...] = (
FieldSpec("title", FieldKind.TEXT),
FieldSpec("content", FieldKind.TEXT),
FieldSpec("correspondent", FieldKind.TEXT),
FieldSpec("document_type", FieldKind.TEXT, aliases=("type",)),
FieldSpec("storage_path", FieldKind.TEXT, aliases=("path",)),
FieldSpec("original_filename", FieldKind.TEXT),
FieldSpec("tag", FieldKind.TEXT, comma_values=True),
FieldSpec("checksum", FieldKind.KEYWORD),
FieldSpec("asn", FieldKind.U64, fast=True),
FieldSpec("page_count", FieldKind.U64, fast=True),
FieldSpec("num_notes", FieldKind.U64, fast=True),
FieldSpec("created", FieldKind.DATE, date_only=True, fast=True),
FieldSpec("modified", FieldKind.DATETIME, fast=True),
FieldSpec("added", FieldKind.DATETIME, fast=True),
FieldSpec(
"notes",
FieldKind.JSON,
subpaths={"user": SubpathSpec(), "note": SubpathSpec(default=True)},
),
FieldSpec(
"custom_fields",
FieldKind.JSON,
subpaths={"name": SubpathSpec(), "value": SubpathSpec(default=True)},
),
)
+145 -442
View File
@@ -6,30 +6,22 @@ from typing import Final
import regex
import tantivy
import whoosh_compat as wc
from django.conf import settings
from whoosh_compat.emitters.tantivy_ import emit as tantivy_emit
from whoosh_compat.errors import Cause
from whoosh_compat.errors import Diagnostic
from whoosh_compat.errors import DiagnosticKind
from whoosh_compat.errors import QueryError
from documents.search._errors import InvalidDateQuery
from documents.search._errors import InvalidNumberQuery
from documents.search._errors import MultipleSearchQueryErrors
from documents.search._errors import SearchQueryError
from documents.search._registry import get_field_registry
from documents.search._tokenizer import simple_search_tokens
from documents.search._translate import SearchQueryError
from documents.search._translate import translate_query
if TYPE_CHECKING:
from collections.abc import Iterable
from datetime import tzinfo
from django.contrib.auth.base_user import AbstractBaseUser
logger = logging.getLogger("paperless.search")
# Maximum seconds any single regex substitution over user-supplied query text
# may run. The one remaining use is a character class, which cannot backtrack,
# so the bound is an upper limit on that substitution's cost, not the ReDoS
# guard it was originally written as.
# Maximum seconds any single regex substitution may run.
# Prevents ReDoS on adversarial user-supplied query strings.
_REGEX_TIMEOUT: Final[float] = 1.0
# Matches CJK/Hangul characters so queries can be routed to bigram fields.
@@ -37,64 +29,6 @@ _REGEX_TIMEOUT: Final[float] = 1.0
_CJK_RE: Final = regex.compile(r"[\p{Han}\p{Hiragana}\p{Katakana}\p{Hangul}]+")
def _user_facing_emit_message(d: Diagnostic) -> str:
"""A user-safe message for an emit-time QueryError's Diagnostic.
Built from the Diagnostic's structured fields (kind, field), never from
d.message: whoosh-compat documents that as developer/log output with no
stability guarantee, and PATTERN_TOO_COMPLEX embeds the raw backend
error text in it.
"""
field = str(d.field) if d.field is not None else None
if d.kind is DiagnosticKind.EXISTS_REQUIRES_FAST:
return f"Existence searches (field:*) are not supported for field {field!r}."
if d.kind is DiagnosticKind.TEXT_RANGE:
return f"Range searches are not supported for field {field!r}."
if d.kind is DiagnosticKind.PATTERN_TOO_COMPLEX:
return f"The wildcard pattern for field {field!r} is too complex."
if d.kind is DiagnosticKind.SCHEMA_FIELD_MISSING:
return f"Field {field!r} is not available in the search index."
logger.warning("Unmapped emit diagnostic %s: %s", d.kind, d.message)
return "The search query could not be executed."
def _map_emit_error(e: QueryError) -> SearchQueryError:
"""Route an emit-time QueryError by its Diagnostic's Cause.
INVALID_INPUT/UNSUPPORTED are user-input errors, exactly like a parse
diagnostic, and map to a 400. INTERNAL means a defect in whoosh-compat
or in our own AST handling, never the user's query, so the QueryError is
re-raised rather than converted, reaching the generic 500 handler instead
of blaming the query. MISCONFIGURED is deliberately both: the registry and
the index schema disagree, which only an operator can fix, so it is logged
as an error, but a request is still waiting and the query cannot run
either way, so it also returns a 400.
EXISTS_REQUIRES_FAST is the one MISCONFIGURED kind that is not a
disagreement. whoosh-compat derives it from the registry's own FieldSpec
(kind plus fast) without ever consulting the index schema, so it fires
whenever a non-fast field of a kind that cannot answer "exists" is asked
to: for us that is only the JSON fields, which field_descriptors() builds
non-fast on purpose. "notes:*" and the five other spellings of it are
ordinary user error that no operator action can clear, so they get the
400 without the alert.
"""
d = e.diagnostic
if d.cause is Cause.INTERNAL:
raise e
if (
d.cause is Cause.MISCONFIGURED
and d.kind is not DiagnosticKind.EXISTS_REQUIRES_FAST
):
logger.error(
"Search index misconfiguration for field %s (%s): %s",
d.field,
d.kind.name,
d.message,
)
return SearchQueryError(_user_facing_emit_message(d))
def _has_cjk(text: str) -> bool:
"""Return True if text contains any CJK characters."""
return bool(_CJK_RE.search(text))
@@ -103,36 +37,14 @@ def _has_cjk(text: str) -> bool:
def extract_cjk_text(text: str) -> str:
"""Join the CJK runs in ``text`` for indexing into bigram (char-ngram) fields.
Mirrors the query side, which extracts the CJK runs of whatever it is
about to search for (the raw string in simple modes, the parsed query's
free-text tokens in query mode): only CJK runs are ever searched against
the bigram fields, so only CJK runs are worth indexing there. Latin text
fed to a character-bigram field is never matched and only bloats the
Mirrors the query side (``_build_cjk_query``): only CJK runs are ever searched
against the bigram fields, so only CJK runs are worth indexing there. Latin
text fed to a character-bigram field is never matched and only bloats the
index and slows indexing/merge. Returns "" when there is no CJK text.
"""
return " ".join(_CJK_RE.findall(text))
def _parse_cjk_text(
index: tantivy.Index,
cjk_text: str,
fields: list[str],
) -> tantivy.Query | None:
"""Parse a plain CJK run string against ``fields``, or None if it won't parse."""
try:
return index.parse_query(cjk_text, fields)
except Exception:
# Broad on purpose, unlike _try_parse_fuzzy_query's narrower
# ValueError: cjk_text isn't filtered to a guaranteed-safe token
# set the way the fuzzy blend's word string is, so the exact
# failure mode tantivy could raise here isn't pinned down.
logger.debug(
"Skipping CJK search clause: could not parse CJK text: %r",
cjk_text,
)
return None
def _build_cjk_query(
index: tantivy.Index,
raw_query: str,
@@ -140,259 +52,91 @@ def _build_cjk_query(
) -> tantivy.Query | None:
"""Build a bigram-field query from the CJK runs in ``raw_query``.
For the simple (TEXT/TITLE) modes, whose input is plain text and carries
no query grammar to respect. Only the CJK character runs are extracted, so
a stray ``field:`` prefix or ``-``/``+`` in the input can neither leak
field semantics nor fail the parse, and no Latin token reaches the
character-bigram matcher (where it would produce spurious matches against
unrelated Latin text). Returns None when there is no CJK text or the parse
fails.
Only the CJK character runs are extracted and parsed; ASCII field prefixes,
boolean operators and date keywords are discarded. This keeps the CJK clause
plain-text and consistent across query/simple modes (no leaked ``field:``
semantics, no parse failures from spaced ``-``/``+``), and avoids feeding
Latin tokens into the character-bigram matcher (which would produce spurious
matches against unrelated Latin text). Returns None when there is no CJK
text or the parse fails.
"""
cjk_text = extract_cjk_text(raw_query)
cjk_text = " ".join(_CJK_RE.findall(raw_query))
if not cjk_text:
return None
return _parse_cjk_text(index, cjk_text, fields)
def _build_ast_cjk_query(
index: tantivy.Index,
ast: wc.ast.Node,
registry: wc.FieldRegistry,
) -> tantivy.Query | None:
"""Build the bigram clause of a QUERY-mode search from the parsed AST.
Same discipline as the fuzzy clause (see _try_parse_fuzzy_query): the CJK
runs come from whoosh_compat's ``free_text_tokens`` over the parsed tree,
never from the raw query string, so a term the user negated or restricted
to a field outside the default search fields contributes nothing, instead
of resurfacing as a top-level clause matching every bigram field.
``free_text_tokens`` reports no field of its own, so the tokens are
collected one default field at a time: a bare term, which the parser has
already copied onto every default field, is therefore searched across
every bigram field, while ``title:東京`` reaches ``bigram_title`` alone.
Fields whose CJK text is identical (the bare-term case) share a single
parse over all of their bigram fields at once.
Raw (``analyzed=False``) tokens are used because the bigram fields have
their own character-ngram analyzer: the default fields' word analyzers
have no useful say over a CJK run, and running them first would only
risk dropping it (remove_long) before the run is ever extracted.
Returns None when the query has no CJK free text.
"""
fields_by_text: dict[str, list[str]] = {}
for field, bigram_field in _CJK_BIGRAM_FIELDS.items():
tokens = wc.free_text_tokens(
ast,
registry=registry,
fields=[field],
analyzed=False,
)
cjk_text = extract_cjk_text(" ".join(tokens))
if cjk_text:
fields_by_text.setdefault(cjk_text, []).append(bigram_field)
clauses: list[tuple[tantivy.Occur, tantivy.Query]] = [
(tantivy.Occur.Should, query)
for cjk_text, bigram_fields in fields_by_text.items()
if (query := _parse_cjk_text(index, cjk_text, bigram_fields)) is not None
]
return _any_of(clauses) if clauses else None
# A joined fuzzy word string must stay plain words: it goes back through
# tantivy's own query parser, and the raw query text the clause collects
# routinely carries characters that parser reads as grammar (a colon, a
# bracket, a quote, a leading -). Each token is cut into its word runs and
# only those are kept, so no field syntax, pattern, range or grouping can
# reach the parser. Cutting rather than dropping the whole token is what
# keeps ordinary hyphenated, dotted and quoted input ("COVID-19",
# "hello@example.com", "tax reports") contributing to the clause at all.
_WORD_RUN_RE = regex.compile(r"\w+")
# The one piece of tantivy grammar that survives the cut: its boolean
# keywords are themselves word runs. Only these exact spellings are
# grammar there ("And"/"and" are ordinary terms), so lowercasing exactly
# these turns them back into the ordinary terms the field analyzer used to
# make of them, before the clause switched to raw text. Left alone, a
# quoted phrase would silently restructure the clause ("tax AND reports"
# becoming a conjunction) or fail to parse and drop it entirely
# ("tax AND", or "IN" anywhere).
#
# Only these words are touched: tantivy lowercases query terms with the
# field's own analyzer, and doing it ourselves first is not always the
# same operation (Python folds a final sigma to a different letter than
# tantivy does, and turns Turkish 'İ' into a sequence tantivy then splits
# in two), which would search for terms the index does not contain.
_TANTIVY_KEYWORDS: Final[frozenset[str]] = frozenset({"AND", "OR", "NOT", "IN"})
def _try_parse_fuzzy_query(
index: tantivy.Index,
ast: wc.ast.Node,
registry: wc.FieldRegistry,
) -> tantivy.Query | None:
"""Build the fuzzy blend clause from the parsed query's free-text
words, or None if it has none.
The clause is built by handing tantivy's own query parser a plain
word string (there's no clean AST-level fuzzy equivalent to
whoosh-compat's parse tree, and fuzzy matching was always an
approximate, secondary, 0.1-boosted clause). The words come from
whoosh_compat's ``free_text_tokens`` over the already-parsed AST,
never from the raw query string: raw whoosh grammar (date keywords,
``[2005 to 2009]`` ranges, bracket-class wildcards) is not tantivy
syntax, and feeding it here used to knock the fuzzy clause out for
the whole query the moment any such construct appeared alongside a
typo'd word. The helper also keeps excluded terms out: a ``NOT``'d
word must not resurface through the fuzzy clause.
Chosen trade-off: a term explicitly fielded on one of the default
search fields (``correspondent:acme``) contributes its text to the
word string UNFIELDED, so the fuzzy clause searches it across all
default fields rather than just the one the user named. That is
recall-only widening on a secondary 0.1-boosted clause the score
threshold already disciplines, accepted in exchange for never feeding
field syntax to tantivy's parser. What the word string guarantees is
exactly that: no field prefix, pattern, range, grouping or quoting
survives, and the boolean keywords that do survive (they are word
runs) are lowercased into ordinary terms; see _TANTIVY_KEYWORDS.
The words are the query's RAW text, not the analyzer's output
(``analyzed=False``), because ``index.parse_query`` analyzes whatever
it is given and analysis is not idempotent: ``universities`` stems to
``univers``, and handing that back stems it again to ``univ``, a term
the index does not contain. ``prefix=True`` hid this as over-broad
matching (``univ`` also prefixes ``unicycle``) rather than as no
matches at all. Raw text is untokenized, which is why it is cut into
word runs above rather than taken whole.
The ValueError guard stays as insurance (the word string is plain
tokens, so tantivy accepting it is expected, not assumed): on a parse
failure the fuzzy clause is skipped and the exact/CJK clauses stand,
rather than the whole query failing.
"""
tokens = wc.free_text_tokens(
ast,
registry=registry,
fields=_DEFAULT_SEARCH_FIELDS,
analyzed=False,
)
words = list(
dict.fromkeys(
word.lower() if word in _TANTIVY_KEYWORDS else word
for token in tokens
for word in _WORD_RUN_RE.findall(token)
),
)
if not words:
return None
fuzzy_text = " ".join(words)
try:
return index.parse_query(
fuzzy_text,
_DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
fuzzy_fields={f: (True, 1, True) for f in _DEFAULT_SEARCH_FIELDS},
)
except ValueError:
logger.debug(
"Skipping fuzzy search clause: token string is not valid "
"tantivy query syntax: %r",
fuzzy_text,
)
return index.parse_query(cjk_text, fields)
except Exception:
return None
_DEFAULT_SEARCH_FIELDS: Final[list[str]] = [
def build_permission_filter(
schema: tantivy.Schema,
user: AbstractBaseUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
Creates a query that matches only documents visible to the specified user
according to paperless-ngx permission rules:
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
"""
owner_any = tantivy.Query.exists_query("owner_id")
no_owner = tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, tantivy.Query.all_query()),
(tantivy.Occur.MustNot, owner_any),
],
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
DEFAULT_SEARCH_FIELDS = [
"title",
"content",
"correspondent",
"document_type",
"tag",
]
_SIMPLE_SEARCH_FIELDS: Final[list[str]] = ["simple_title", "simple_content"]
_TITLE_SEARCH_FIELDS: Final[list[str]] = ["simple_title"]
# The bigram (character-ngram) companion of each default search field.
_CJK_BIGRAM_FIELDS: Final[dict[str, str]] = {
field: f"bigram_{field}" for field in _DEFAULT_SEARCH_FIELDS
}
SIMPLE_SEARCH_FIELDS = ["simple_title", "simple_content"]
TITLE_SEARCH_FIELDS = ["simple_title"]
_CJK_ALL_FIELDS: Final[list[str]] = [
"bigram_content",
"bigram_title",
"bigram_correspondent",
"bigram_document_type",
"bigram_tag",
]
_CJK_CONTENT_FIELDS: Final[list[str]] = ["bigram_content"]
_CJK_TITLE_FIELDS: Final[list[str]] = ["bigram_title"]
_FIELD_BOOSTS = {"title": 2.0}
_SIMPLE_FIELD_BOOSTS = {"simple_title": 2.0}
class _ConjunctiveNegations(wc.ast.Visitor[tuple["wc.ast.Node", ...]]):
"""Collect the subtrees an AST excludes from every document it matches.
A negation reached through ``And``/``AndNot``/``Require`` (and through
the required half of an ``AndMaybe``) constrains the whole query, so it
can be re-stated above the blend. ``Or`` is deliberately not descended
into: in ``invoice OR NOT secret`` the negation is one branch's own
condition, and hoisting it would throw away documents the other branch
matches. Nor is a collected subtree descended into, since a negation
inside a negation is not an exclusion.
Node types with no negation to contribute (every leaf, ``Or``) fall
through to ``generic_visit``.
"""
def generic_visit(self, node: wc.ast.Node) -> tuple[wc.ast.Node, ...]:
return ()
def visit_not(self, node: wc.ast.Not) -> tuple[wc.ast.Node, ...]:
return (node.child,)
def visit_andnot(self, node: wc.ast.AndNot) -> tuple[wc.ast.Node, ...]:
return (*self.visit(node.positive), node.negative)
def visit_and(self, node: wc.ast.And) -> tuple[wc.ast.Node, ...]:
return tuple(
negation for child in node.children for negation in self.visit(child)
)
def visit_boosted(self, node: wc.ast.Boosted) -> tuple[wc.ast.Node, ...]:
return self.visit(node.child)
def visit_andmaybe(self, node: wc.ast.AndMaybe) -> tuple[wc.ast.Node, ...]:
return self.visit(node.required)
def visit_require(self, node: wc.ast.Require) -> tuple[wc.ast.Node, ...]:
return (*self.visit(node.scored), *self.visit(node.filter_only))
def _negation_clauses(
index: tantivy.Index,
ast: wc.ast.Node,
registry: wc.FieldRegistry,
) -> list[tuple[tantivy.Occur, tantivy.Query]]:
"""MustNot clauses for everything ``ast`` excludes conjunctively.
Each excluded subtree is emitted as its own positive query and attached
with ``MustNot``, rather than emitting a negative query and hoping
tantivy accepts a bare one.
"""
try:
return [
(
tantivy.Occur.MustNot,
tantivy_emit(negation, index=index, registry=registry),
)
for negation in _ConjunctiveNegations().visit(ast)
]
except QueryError as e:
raise _map_emit_error(e) from e
def _any_of(clauses: list[tuple[tantivy.Occur, tantivy.Query]]) -> tantivy.Query:
"""Collapse a clause list: none -> empty, one -> itself (no wasted
single-clause boolean_query wrapping), many -> boolean_query(clauses)."""
if not clauses:
return tantivy.Query.empty_query()
if len(clauses) == 1:
return clauses[0][1]
return tantivy.Query.boolean_query(clauses)
def _simple_query_tokens(raw_query: str) -> list[str]:
# Tokenize and fold via the same analyzer used to index simple_title /
# simple_content, so query terms fold identically to the indexed terms
# (single source of truth for ASCII folding).
return simple_search_tokens(raw_query)
def _build_simple_token_query(
@@ -424,7 +168,9 @@ def _build_simple_token_query(
query = tantivy.Query.boost_query(query, boost)
field_queries.append((tantivy.Occur.Should, query))
return _any_of(field_queries)
if len(field_queries) == 1:
return field_queries[0][1]
return tantivy.Query.boolean_query(field_queries)
def parse_user_query(
@@ -433,53 +179,52 @@ def parse_user_query(
tz: tzinfo,
) -> tantivy.Query:
"""
Parse user query through whoosh-compat, then blend in fuzzy/CJK clauses.
Parse user query through the complete preprocessing pipeline.
1. wc.parse() against the shared FieldRegistry (whoosh grammar -> AST).
Bare notes:/custom_fields: prefixes resolve to their default subpath
(notes.note:/custom_fields.value:) directly in the registry, via
each JSON field's SubpathSpec(default=True).
2. Any diagnostics (bad dates/numbers) map to SearchQueryError subclasses
and raise the view returns HTTP 400 with every offending field
listed, not just the first.
3. emit() turns the AST into a tantivy.Query directly (no string
round-trip). A QueryError is routed by its Diagnostic's Cause
(_map_emit_error): a construct that parses but can't execute against
tantivy (e.g. a text-field range) is a 400, a registry/schema
mismatch is logged and a 400, and an INTERNAL defect is re-raised.
4. Optional fuzzy blend (ADVANCED_FUZZY_SEARCH_THRESHOLD) builds a
plain word string from the parsed AST's free-text tokens
(whoosh_compat.free_text_tokens) and feeds THAT to
index.parse_query never raw_query, whose whoosh grammar (date
keywords, bracket-class wildcards, etc.) tantivy's parser rejects,
which used to silently knock the fuzzy clause out of any mixed
query (see _try_parse_fuzzy_query).
5. Optional CJK bigram clause, built from the same parsed AST for the
same reason (see _build_ast_cjk_query): a CJK term the query negated
or fielded must not resurface through it.
6. When any optional clause was added, the query's conjunctive
exclusions are restated as MustNot above the blend
(_negation_clauses): a clause built from positive terms cannot
express them, and as a bare Should it would undo them.
Transforms the raw user query through multiple stages:
1. Date keyword rewriting (today ISO 8601 ranges)
2. Query normalization (comma expansion, whitespace cleanup)
3. Tantivy parsing with field boosts
4. Optional fuzzy query blending (if ADVANCED_FUZZY_SEARCH_THRESHOLD set)
Args:
index: Tantivy index with registered tokenizers
raw_query: Original user query string
tz: Timezone for date boundary calculations
Returns:
Parsed Tantivy query ready for execution
Note:
When ADVANCED_FUZZY_SEARCH_THRESHOLD is configured, adds a low-priority
fuzzy query as a Should clause (0.1 boost) to catch approximate matches
while keeping exact matches ranked higher. The threshold value is applied
as a post-search score filter, not during query construction.
"""
registry = get_field_registry(settings.SEARCH_LANGUAGE)
result = wc.parse(
raw_query,
registry=registry,
default_fields=_DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
tz=tz,
)
if result.diagnostics:
raise _diagnostics_to_error(result.diagnostics)
try:
exact = tantivy_emit(result.ast, index=index, registry=registry)
except QueryError as e:
raise _map_emit_error(e) from e
query_str = translate_query(raw_query, tz)
except SearchQueryError:
# Intentional, user-fixable error (e.g. an unparsable date). Propagate so
# the view can return a 400 with a helpful message rather than falling
# back to the raw (still-invalid) query.
raise
except Exception: # pragma: no cover - defensive
logger.warning("Query translation failed; using raw query", exc_info=True)
query_str = raw_query
exact = index.parse_query(
query_str,
DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
)
# The standard analyzer keeps a whitespace-free CJK run as a single token,
# so substring queries can't match content/title (and long runs are dropped
# by remove_long). Route CJK queries to the bigram fields, whose ngram
# tokenizer indexes overlapping 2-grams for substring matching.
cjk_query = (
_build_ast_cjk_query(index, result.ast, registry)
_build_cjk_query(index, raw_query, _CJK_ALL_FIELDS)
if _has_cjk(raw_query)
else None
)
@@ -490,65 +235,22 @@ def parse_user_query(
threshold = settings.ADVANCED_FUZZY_SEARCH_THRESHOLD
if threshold is not None:
fuzzy = _try_parse_fuzzy_query(index, result.ast, registry)
if fuzzy is not None:
clauses.append(
(tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)),
)
fuzzy = index.parse_query(
query_str,
DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
# (prefix=True, distance=1, transposition_cost_one=True) — edit-distance fuzziness
fuzzy_fields={f: (True, 1, True) for f in DEFAULT_SEARCH_FIELDS},
)
# 0.1 boost keeps fuzzy hits ranked below exact matches (intentional)
clauses.append((tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)))
if cjk_query is not None:
clauses.append((tantivy.Occur.Should, cjk_query))
if len(clauses) == 1:
return exact
# The fuzzy and CJK clauses are built from positive terms only, so as
# plain Shoulds beside the exact clause they re-admit exactly the
# documents the query excluded. Restate the exclusions once, above the
# whole blend. Redundant against the exact clause, which already
# carries them, but idempotently so.
negations = _negation_clauses(index, result.ast, registry)
if not negations:
return _any_of(clauses)
return tantivy.Query.boolean_query(
[(tantivy.Occur.Must, _any_of(clauses)), *negations],
)
# The three whoosh-compat kinds for a wildcard on a field that cannot
# carry one. d.field_kind supplies the discriminator, so naming the field's
# type needs no second trip through the registry.
_PATTERN_ON_KINDS: Final = frozenset(
{
DiagnosticKind.PATTERN_ON_NUMERIC,
DiagnosticKind.PATTERN_ON_BOOLEAN_EXISTS,
DiagnosticKind.PATTERN_ON_SUBPATH,
},
)
def _diagnostics_to_error(diagnostics: tuple[Diagnostic, ...]) -> SearchQueryError:
errors = [_single_diagnostic_to_error(d) for d in diagnostics]
return errors[0] if len(errors) == 1 else MultipleSearchQueryErrors(errors)
def _single_diagnostic_to_error(d: Diagnostic) -> SearchQueryError:
# d.field is a FieldRef, not a str: str(d.field) gives the canonical
# dotted name (an aliased query, e.g. type:, reports document_type).
field_name = str(d.field) if d.field is not None else None
if d.kind is DiagnosticKind.BAD_DATE:
return InvalidDateQuery(field_name, d.raw_value)
if d.kind is DiagnosticKind.BAD_NUMBER:
return InvalidNumberQuery(field_name, d.raw_value)
if d.kind is DiagnosticKind.TOO_DEEP:
return SearchQueryError("The search query is nested too deeply.")
if d.kind in _PATTERN_ON_KINDS:
kind_label = f" ({d.field_kind.name.lower()})" if d.field_kind else ""
return SearchQueryError(
f"Wildcard patterns are not supported for field "
f"{field_name!r}{kind_label}.",
)
logger.warning("Unmapped parse diagnostic %s: %s", d.kind, d.message)
return SearchQueryError("The search query could not be executed.")
return tantivy.Query.boolean_query(clauses)
def parse_simple_query(
@@ -566,7 +268,7 @@ def parse_simple_query(
CJK substrings the simple analyzer can't (long whitespace-free runs are
dropped by remove_long).
"""
tokens = simple_search_tokens(raw_query)
tokens = _simple_query_tokens(raw_query)
clauses: list[tuple[tantivy.Occur, tantivy.Query]] = []
if tokens:
@@ -589,14 +291,23 @@ def parse_simple_query(
)
for token in tokens
]
clauses.append((tantivy.Occur.Should, _any_of(token_queries)))
simple_query = (
token_queries[0][1]
if len(token_queries) == 1
else tantivy.Query.boolean_query(token_queries)
)
clauses.append((tantivy.Occur.Should, simple_query))
if cjk_fields and _has_cjk(raw_query):
cjk_q = _build_cjk_query(index, raw_query, cjk_fields)
if cjk_q is not None:
clauses.append((tantivy.Occur.Should, cjk_q))
return _any_of(clauses)
if not clauses:
return tantivy.Query.empty_query()
if len(clauses) == 1:
return clauses[0][1]
return tantivy.Query.boolean_query(clauses)
def parse_simple_text_highlight_query(
@@ -611,21 +322,13 @@ def parse_simple_text_highlight_query(
# Strip Tantivy operator chars before tokenizing: this is a plain-text
# highlight query, not a structured boolean query, so +/- are separators.
tokens = simple_search_tokens(
tokens = _simple_query_tokens(
regex.sub(r"[-+]", " ", raw_query, timeout=_REGEX_TIMEOUT),
)
if not tokens:
return tantivy.Query.empty_query()
# Quote each token as its own phrase, escaping backslashes and embedded
# quotes. simple search tokens can carry arbitrary Tantivy syntax
# characters (`"`, `:`, `(`, `[`, `/`, ...) that the query-string parser
# would otherwise interpret as query grammar rather than literal text.
quoted_tokens = [
'"' + token.replace("\\", "\\\\").replace('"', '\\"') + '"' for token in tokens
]
return index.parse_query(" ".join(quoted_tokens), ["content"])
return index.parse_query(" ".join(tokens), ["content"])
def parse_simple_text_query(
@@ -639,7 +342,7 @@ def parse_simple_text_query(
return parse_simple_query(
index,
raw_query,
_SIMPLE_SEARCH_FIELDS,
SIMPLE_SEARCH_FIELDS,
cjk_fields=_CJK_CONTENT_FIELDS,
)
@@ -655,6 +358,6 @@ def parse_simple_title_query(
return parse_simple_query(
index,
raw_query,
_TITLE_SEARCH_FIELDS,
TITLE_SEARCH_FIELDS,
cjk_fields=_CJK_TITLE_FIELDS,
)

Some files were not shown because too many files have changed in this diff Show More