Compare commits

...
Author SHA1 Message Date
stumpylog ca42c27455 fix(search): cap the advanced search query length 2026-08-28 09:48:14 -07:00
stumpylog a626ad4627 style(search): remove em dashes from PR2 comments/docstrings 2026-08-28 09:47:14 -07:00
stumpylog 9029b27aeb feat(search): parse advanced search with whoosh-compat and delete the hand-written translator 2026-08-28 09:47:14 -07:00
stumpylog 03e05e0804 test(search): move schema fingerprint/version tests from PR2
Both files exercise documents.search._schema exclusively (build_schema,
schema_fingerprint, needs_rebuild, open_or_rebuild_index) with no
dependency on whoosh-compat query routing, so they belong with the
schema/field-registry work rather than PR2's query rewrite.
2026-08-28 09:46:57 -07:00
stumpylog 2f79549f92 style(search): remove em dashes from PR1 comments/docstrings 2026-08-28 09:35:24 -07:00
stumpylog 9e9f3f4e09 refactor(search): dispatch table for _public_field_descriptors
Collapse the TEXT/KEYWORD/U64/DATE/DATETIME if/elif chain into a
FieldKind -> (schema kind, tokenizer) lookup table. JSON stays an
explicit branch since it can emit a second, synthetic notes_text
descriptor.
2026-08-28 09:30:06 -07:00
stumpylog d0869bec01 feat(search): add whoosh-compat, the shared field table and the field registry 2026-08-28 08:42:14 -07:00
shamoon 01c40aa0bd Tweak: subtler shadow, tweak against light color 2026-08-24 08:51:51 -07:00
shamoon 5f70b6eee7 Fix navbar button alignment + hover consistency 2026-08-24 08:47:27 -07:00
GitHub Actions e3e4b26944 Auto translate strings 2026-08-24 15:16:51 +00:00
a0908f6b4a Enhancement: websocket heartbeat (#13739)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-08-24 15:15:21 +00:00
bab9129ff8 Fix: lazy import guardian modules to fix search language setting (#13768)
Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2026-08-24 13:46:20 +00:00
GitHub Actions 1e13f86174 Auto translate strings 2026-08-24 09:13:06 +00:00
shamoonandGitHub 22c57fb392 Enhancement: more v3 ui tweaks (#13774) 2026-08-24 02:11:34 -07:00
GitHub Actions 08a7f6ccc0 Auto translate strings 2026-08-24 04:56:30 +00:00
shamoonandGitHub 3f4d327f64 Chore: add some missing UI accessibility labels (#13772) 2026-08-23 21:55:07 -07:00
GitHub Actions bd326540fa Auto translate strings 2026-08-24 02:43:34 +00:00
shamoonandGitHub 33f9adb05a Chore: refactor permission checkbox live changes (#13771) 2026-08-23 19:42:18 -07:00
shamoonandGitHub d24606a03b Tweak: better support long list of views in documents list (#13769) 2026-08-23 16:40:55 -07:00
shamoonandGitHub 294328f174 Fix: version indexing fixes (#13737) 2026-08-23 23:04:47 +00:00
shamoonandGitHub 0458bad5f2 Fix: append charset to file response for text files (#13759) 2026-08-22 06:15:53 -07:00
shamoon a510d03c77 Merge branch 'main' into dev 2026-08-22 05:27:10 -07:00
shamoonandGitHub d7b3612a41 Chore: pin Apache Tika images to 3.3.1 (#13758) 2026-08-22 05:25:37 -07:00
shamoonandGitHub 7e4a644714 Fix: align bulk edit perms with document model (#13757) 2026-08-22 05:24:16 -07:00
GitHub Actions bbcd6af2fe Auto translate strings 2026-08-21 15:25:24 +00:00
shamoonandGitHub 0431939f18 QoL: add count badge to versions dropdown (#13753) 2026-08-21 08:23:47 -07:00
shamoonandGitHub bed95ea301 Tweakhancement: add jitter to IMAP polling schedule (#13734) 2026-08-20 17:19:47 +00:00
GitHub Actions 42034c3c77 Auto translate strings 2026-08-20 04:52:42 +00:00
shamoonandGitHub 705220fb5a Fix: use selected version for doc detail emailing (#13738) 2026-08-19 21:51:13 -07:00
GitHub Actions a424dace43 Auto translate strings 2026-08-19 18:20:04 +00:00
shamoonandGitHub 751299895e Fix: hide version delete button without global perms (#13735) 2026-08-19 11:17:19 -07:00
Trenton HandGitHub 5f9bc5de88 Chore: Upgrade Docker image to Python 3.14 (#13721)
* Upgrades our base image to uv 0.12 branch and Python 3.14

* Upgrades our workflows to uv 0.12.x as well

* Updates these locked wheels too
2026-08-19 09:43:14 -07:00
f1c8a72f26 Enhancement: sync OIDC groups to superuser and staff roles (#13060)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
Co-authored-by: SoleroTG <github-29h@solero.quietmail.eu>
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-08-19 15:27:56 +00:00
GitHub Actions fd3c525f03 Auto translate strings 2026-08-19 14:24:36 +00:00
shamoonandGitHub e389298aab Enhancement: merge documents as versions (#13515) 2026-08-19 07:20:14 -07:00
shamoonandGitHub c5c5cc0b1d Tweak: adjust modal proportions for small screens (#13728) 2026-08-18 19:34:18 -07:00
b17a512539 Refactor: render paperless_ai prompts via Jinja2 templates instead of f-strings (#13698)
* Refactor: render paperless_ai prompts via Jinja2 templates instead of f-strings

* Apply suggestions from code review

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-08-18 18:32:21 +00:00
shamoon 4cf027de40 Tweak: space out the side menu sub-nav a bit 2026-08-18 09:44:03 -07:00
shamoon 41953c7846 Chore: harden ImageMagick policy 2026-08-18 09:38:54 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e5abe5cf32 Chore(deps): Bump the uv group across 1 directory with 2 updates (#13709)
Bumps the uv group with 2 updates in the / directory: [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) and [sqlparse](https://github.com/andialbrecht/sqlparse).


Updates `pymdown-extensions` from 11.0 to 11.0.1
- [Release notes](https://github.com/facelessuser/pymdown-extensions/releases)
- [Commits](https://github.com/facelessuser/pymdown-extensions/compare/11.0...11.0.1)

Updates `sqlparse` from 0.5.5 to 0.6.0
- [Changelog](https://github.com/andialbrecht/sqlparse/blob/master/CHANGELOG)
- [Commits](https://github.com/andialbrecht/sqlparse/compare/0.5.5...0.6.0)

---
updated-dependencies:
- dependency-name: pymdown-extensions
  dependency-version: 11.0.1
  dependency-type: indirect
- dependency-name: sqlparse
  dependency-version: 0.6.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-18 15:11:24 +00:00
shamoonandGitHub 2d084983c8 Documentation: clarify OCR mode changes in v3 (#13666) 2026-08-12 21:46:52 -07:00
shamoonandGitHub e2c284f64e Documentation: add wiki links for AI stuff and parser plugins (#13626) 2026-08-09 19:05:32 -07:00
169 changed files with 10037 additions and 4215 deletions
@@ -68,7 +68,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
+1 -1
View File
@@ -11,7 +11,7 @@ concurrency:
group: backend-${{ github.event.pull_request.number || github.ref }} group: backend-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true cancel-in-progress: true
env: env:
DEFAULT_UV_VERSION: "0.11.x" DEFAULT_UV_VERSION: "0.12.x"
NLTK_DATA: "/usr/share/nltk_data" NLTK_DATA: "/usr/share/nltk_data"
permissions: {} permissions: {}
jobs: jobs:
+1 -1
View File
@@ -11,7 +11,7 @@ concurrency:
permissions: permissions:
contents: read contents: read
env: env:
DEFAULT_UV_VERSION: "0.11.x" DEFAULT_UV_VERSION: "0.12.x"
DEFAULT_PYTHON_VERSION: "3.12" DEFAULT_PYTHON_VERSION: "3.12"
jobs: jobs:
changes: changes:
+1 -1
View File
@@ -8,7 +8,7 @@ concurrency:
group: release-${{ github.ref }} group: release-${{ github.ref }}
cancel-in-progress: false cancel-in-progress: false
env: env:
DEFAULT_UV_VERSION: "0.11.x" DEFAULT_UV_VERSION: "0.12.x"
DEFAULT_PYTHON_VERSION: "3.12" DEFAULT_PYTHON_VERSION: "3.12"
permissions: {} permissions: {}
jobs: jobs:
+1 -1
View File
@@ -4,7 +4,7 @@ on:
branches: branches:
- dev - dev
env: env:
DEFAULT_UV_VERSION: "0.11.x" DEFAULT_UV_VERSION: "0.12.x"
jobs: jobs:
generate-translate-strings: generate-translate-strings:
name: Generate Translation Strings name: Generate Translation Strings
+1 -1
View File
@@ -30,7 +30,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs # Purpose: Installs s6-overlay and rootfs
# Comments: # Comments:
# - Don't leave anything extra in here either # - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.11.32-python3.12-trixie-slim AS s6-overlay-base FROM ghcr.io/astral-sh/uv:0.12.5-python3.14-trixie-slim AS s6-overlay-base
WORKDIR /usr/src/s6 WORKDIR /usr/src/s6
@@ -81,7 +81,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
@@ -76,7 +76,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
@@ -65,7 +65,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
@@ -68,7 +68,14 @@
<!-- <policy domain="resource" name="thread" value="4"/> --> <!-- <policy domain="resource" name="thread" value="4"/> -->
<!-- <policy domain="resource" name="throttle" value="0"/> --> <!-- <policy domain="resource" name="throttle" value="0"/> -->
<!-- <policy domain="resource" name="time" value="3600"/> --> <!-- <policy domain="resource" name="time" value="3600"/> -->
<!-- <policy domain="coder" rights="none" pattern="MVG" /> --> <!-- Paperless does not process SVG or ImageMagick scripting formats. -->
<policy domain="coder" rights="none" pattern="SVG" />
<policy domain="coder" rights="none" pattern="SVGZ" />
<policy domain="coder" rights="none" pattern="MSVG" />
<policy domain="coder" rights="none" pattern="RSVG" />
<policy domain="coder" rights="none" pattern="MSL" />
<policy domain="coder" rights="none" pattern="MVG" />
<policy domain="coder" rights="none" pattern="EPHEMERAL" />
<!-- <policy domain="module" rights="none" pattern="{PS,PDF,XPS}" /> --> <!-- <policy domain="module" rights="none" pattern="{PS,PDF,XPS}" /> -->
<!-- <policy domain="delegate" rights="none" pattern="HTTPS" /> --> <!-- <policy domain="delegate" rights="none" pattern="HTTPS" /> -->
<!-- <policy domain="path" rights="none" pattern="@*" /> --> <!-- <policy domain="path" rights="none" pattern="@*" /> -->
@@ -78,8 +85,6 @@
<!-- <policy domain="system" name="pixel-cache-memory" value="anonymous"/> --> <!-- <policy domain="system" name="pixel-cache-memory" value="anonymous"/> -->
<!-- <policy domain="system" name="shred" value="2"/> --> <!-- <policy domain="system" name="shred" value="2"/> -->
<!-- <policy domain="system" name="precision" value="6"/> --> <!-- <policy domain="system" name="precision" value="6"/> -->
<!-- not needed due to the need to use explicitly by mvg: -->
<!-- <policy domain="delegate" rights="none" pattern="MVG" /> -->
<!-- use curl --> <!-- use curl -->
<policy domain="delegate" rights="none" pattern="URL" /> <policy domain="delegate" rights="none" pattern="URL" />
<policy domain="delegate" rights="none" pattern="HTTPS" /> <policy domain="delegate" rights="none" pattern="HTTPS" />
+6 -1
View File
@@ -129,6 +129,10 @@ At a minimum you need to enable AI and choose an LLM backend:
and/or [`PAPERLESS_AI_LLM_ENDPOINT`](configuration.md#PAPERLESS_AI_LLM_ENDPOINT). Ollama and/or [`PAPERLESS_AI_LLM_ENDPOINT`](configuration.md#PAPERLESS_AI_LLM_ENDPOINT). Ollama
requires `PAPERLESS_AI_LLM_ENDPOINT` pointing at your Ollama server. requires `PAPERLESS_AI_LLM_ENDPOINT` pointing at your Ollama server.
See the community-maintained wiki page on
[choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for suggested generation and embedding models.
### AI-assisted suggestions ### AI-assisted suggestions
With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type, With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type,
@@ -808,7 +812,8 @@ Third-party parser plugins extend Paperless-ngx to support additional file
formats. A plugin is a Python package that advertises itself under the formats. A plugin is a Python package that advertises itself under the
`paperless_ngx.parsers` entry point group. Refer to the `paperless_ngx.parsers` entry point group. Refer to the
[developer documentation](development.md#making-custom-parsers) for how to [developer documentation](development.md#making-custom-parsers) for how to
create one. create one, or see the wiki for a community-maintained list of
[parser plugins](https://github.com/paperless-ngx/paperless-ngx/wiki/Related-Projects#parser-plugins).
!!! warning "Third-party plugins are not officially supported" !!! warning "Third-party plugins are not officially supported"
+1
View File
@@ -227,6 +227,7 @@ Version-aware endpoints:
- `PATCH /api/documents/{id}/`: content updates target the selected version (`?version={version_id}`) or latest version by default; non-content metadata updates target the root document. - `PATCH /api/documents/{id}/`: content updates target the selected version (`?version={version_id}`) or latest version by default; non-content metadata updates target the root document.
- `GET /api/documents/{id}/download/`, `GET /api/documents/{id}/preview/`, `GET /api/documents/{id}/thumb/`, `GET /api/documents/{id}/metadata/`: accept `?version={version_id}`. - `GET /api/documents/{id}/download/`, `GET /api/documents/{id}/preview/`, `GET /api/documents/{id}/thumb/`, `GET /api/documents/{id}/metadata/`: accept `?version={version_id}`.
- `POST /api/documents/{id}/update_version/`: uploads a new version using multipart form field `document` and optional `version_label`. - `POST /api/documents/{id}/update_version/`: uploads a new version using multipart form field `document` and optional `version_label`.
- `POST /api/documents/merge_as_versions/`: merges existing top-level documents as versions of a selected root. The JSON body must contain `documents` (at least two document IDs) and `root_document_id` (one of those IDs). When merging one source document, an optional `version_label` may be provided.
- `PATCH /api/documents/{id}/versions/{version_id}/`: updates the `version_label` of a specific version. - `PATCH /api/documents/{id}/versions/{version_id}/`: updates the `version_label` of a specific version.
- `DELETE /api/documents/{root_id}/versions/{version_id}/`: deletes a non-root version. - `DELETE /api/documents/{root_id}/versions/{version_id}/`: deletes a non-root version.
+23 -1
View File
@@ -776,6 +776,24 @@ system. See the corresponding
Defaults to "groups" Defaults to "groups"
#### [`PAPERLESS_SOCIAL_ACCOUNT_SYNC_SUPERUSER_GROUP=<str>`](#PAPERLESS_SOCIAL_ACCOUNT_SYNC_SUPERUSER_GROUP) {#PAPERLESS_SOCIAL_ACCOUNT_SYNC_SUPERUSER_GROUP}
: Allows you to define a group name that, if present in the third-party authentication system's groups claim, will grant the user superuser (admin) and staff status in Paperless-ngx. If the group is not present in the claim, superuser status will be revoked upon next login.
!!! warning
This is a direct reflection of the claim on every login, including the connecting user, with no exemption for the last remaining admin. If the group is missing or misconfigured on the identity provider side, the logged-in user will immediately lose their own superuser access. Fix the group membership or claim mapping on the identity provider to restore it. If the identity provider itself is unreachable or misconfigured and you are locked out, you can recover admin access locally with `manage.py createsuperuser`.
Defaults to None
#### [`PAPERLESS_SOCIAL_ACCOUNT_SYNC_STAFF_GROUP=<str>`](#PAPERLESS_SOCIAL_ACCOUNT_SYNC_STAFF_GROUP) {#PAPERLESS_SOCIAL_ACCOUNT_SYNC_STAFF_GROUP}
: Allows you to define a group name that, if present in the third-party authentication system's groups claim, will grant the user staff status in Paperless-ngx. If the group is not present in the claim and the user is not a superuser, staff status will be revoked upon next login.
!!! warning
As with [`PAPERLESS_SOCIAL_ACCOUNT_SYNC_SUPERUSER_GROUP`](#PAPERLESS_SOCIAL_ACCOUNT_SYNC_SUPERUSER_GROUP), this is applied on every login unconditionally, including for the connecting user themselves.
Defaults to None
#### [`PAPERLESS_SOCIAL_ACCOUNT_DEFAULT_GROUPS=<comma-separated-list>`](#PAPERLESS_SOCIAL_ACCOUNT_DEFAULT_GROUPS) {#PAPERLESS_SOCIAL_ACCOUNT_DEFAULT_GROUPS} #### [`PAPERLESS_SOCIAL_ACCOUNT_DEFAULT_GROUPS=<comma-separated-list>`](#PAPERLESS_SOCIAL_ACCOUNT_DEFAULT_GROUPS) {#PAPERLESS_SOCIAL_ACCOUNT_DEFAULT_GROUPS}
: A list of group names that users who signup via social accounts will be added to upon signup. Groups listed here must already exist. : A list of group names that users who signup via social accounts will be added to upon signup. Groups listed here must already exist.
@@ -1197,7 +1215,7 @@ should be a valid crontab(5) expression describing when to run.
: If set to the string "disable", no emails will be fetched automatically. : If set to the string "disable", no emails will be fetched automatically.
Defaults to `*/10 * * * *` or every ten minutes. Defaults to every ten minutes, with an installation-specific minute offset.
#### [`PAPERLESS_TRAIN_TASK_CRON=<cron expression>`](#PAPERLESS_TRAIN_TASK_CRON) {#PAPERLESS_TRAIN_TASK_CRON} #### [`PAPERLESS_TRAIN_TASK_CRON=<cron expression>`](#PAPERLESS_TRAIN_TASK_CRON) {#PAPERLESS_TRAIN_TASK_CRON}
@@ -2070,6 +2088,8 @@ suggestions. This setting is required to be set to true in order to use the AI f
models supported by the current embedding backend. If not supplied, defaults to models supported by the current embedding backend. If not supplied, defaults to
"text-embedding-3-small" for the OpenAI-compatible backend, "text-embedding-3-small" for the OpenAI-compatible backend,
"sentence-transformers/all-MiniLM-L6-v2" for Huggingface, and "embeddinggemma" for Ollama. "sentence-transformers/all-MiniLM-L6-v2" for Huggingface, and "embeddinggemma" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for language and resource considerations.
Defaults to None. Defaults to None.
@@ -2126,6 +2146,8 @@ setting is required to be set to use the AI features.
: The model to use for the AI backend, i.e. "gpt-3.5-turbo", "gpt-4" or any of the models supported : The model to use for the AI backend, i.e. "gpt-3.5-turbo", "gpt-4" or any of the models supported
by the current backend. If not supplied, defaults to "gpt-3.5-turbo" for the OpenAI-compatible by the current backend. If not supplied, defaults to "gpt-3.5-turbo" for the OpenAI-compatible
backend and "llama3.1" for Ollama. backend and "llama3.1" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for local versus remote and model-size considerations.
Defaults to None. Defaults to None.
+4 -3
View File
@@ -156,7 +156,7 @@ The new settings are independent:
### Database configuration ### Database configuration
If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and `skip_archive_file` values are converted to the new `archive_file_generation` field. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent. If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and explicit `skip_archive_file` values are converted to the new `archive_file_generation` field. Users who relied on the old defaults must set `archive_file_generation` to `always` to preserve the v2 behaviour of always creating an archive. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent.
### Action Required ### Action Required
@@ -165,8 +165,9 @@ Remove any `PAPERLESS_OCR_SKIP_ARCHIVE_FILE` variable from your environment. If
```bash ```bash
# v2: skip OCR when text present, always archive # v2: skip OCR when text present, always archive
PAPERLESS_OCR_MODE=skip PAPERLESS_OCR_MODE=skip
# v3: equivalent (auto is the new default) # v3: equivalent
# No change needed - auto is the default PAPERLESS_OCR_MODE=auto
PAPERLESS_ARCHIVE_FILE_GENERATION=always
# v2: skip OCR when text present, skip archive too # v2: skip OCR when text present, skip archive too
PAPERLESS_OCR_MODE=skip_noarchive PAPERLESS_OCR_MODE=skip_noarchive
+8 -15
View File
@@ -416,22 +416,15 @@ to a positive number to enable polling and disable native filesystem notificatio
You may need to change the path in the files. Example: You may need to change the path in the files. Example:
`ExecStart=/opt/paperless/.local/bin/celery --app paperless worker --loglevel INFO` `ExecStart=/opt/paperless/.local/bin/celery --app paperless worker --loglevel INFO`
12. Configure ImageMagick to allow processing of PDF documents. Most 12. Configure ImageMagick to allow processing of PDF documents and disable
distributions have this disabled by default, since PDF documents can formats that Paperless-ngx does not use. Most distributions disable PDF
contain malware. If you don't do this, Paperless-ngx will fall back to processing by default, since PDF documents can contain malware. If you
Ghostscript for certain steps such as thumbnail generation. don't enable it, Paperless-ngx will fall back to Ghostscript for certain
steps such as thumbnail generation.
Edit `/etc/ImageMagick-6/policy.xml` and adjust Configure the active ImageMagick policy file (commonly
`/etc/ImageMagick-6/policy.xml` or `/etc/ImageMagick-7/policy.xml`) and
``` adjust similar to [the docker policy file](https://raw.githubusercontent.com/paperless-ngx/paperless-ngx/refs/heads/main/docker/rootfs/etc/ImageMagick-6/paperless-policy.xml). You should also include restrictions as noted there.
<policy domain="coder" rights="none" pattern="PDF" />
```
to
```
<policy domain="coder" rights="read|write" pattern="PDF" />
```
**Optional: Install the [jbig2enc](https://ocrmypdf.readthedocs.io/en/latest/jbig2.html) encoder.** **Optional: Install the [jbig2enc](https://ocrmypdf.readthedocs.io/en/latest/jbig2.html) encoder.**
This will reduce the size of generated PDF documents. You'll most likely need to compile this yourself, because this This will reduce the size of generated PDF documents. You'll most likely need to compile this yourself, because this
+4
View File
@@ -99,6 +99,10 @@ Think of versions as **file history** for a document.
- By default, search and document content use the latest version. - By default, search and document content use the latest version.
- In document detail, selecting a version switches the preview, file metadata and content (and download etc buttons) to that version. - In document detail, selecting a version switches the preview, file metadata and content (and download etc buttons) to that version.
- Deleting a non-root version keeps metadata and falls back to the latest remaining version. - Deleting a non-root version keeps metadata and falls back to the latest remaining version.
- From the document list, select two or more documents and choose **Merge as versions** to combine them under one entry. Select the root document whose metadata and permissions should be retained; the other selected documents become file versions. The root may already have versions, but documents being added as versions must not have version histories of their own.
- From a document's **Versions** menu, choose **Existing** to search for another document and add it as a version of the current document.
- Documents merged as versions give up their archive serial number. If the root has no ASN of its own it takes the first one, otherwise the ASNs are released and the removal is logged.
- Merging as versions cannot be undone from the UI, and deleting the root document moves its versions to the trash as well.
### Management Lists ### Management Lists
+2
View File
@@ -34,6 +34,8 @@ PAPERLESS_SECRET_KEY=change-me
#PAPERLESS_AUTO_LOGIN_USERNAME= #PAPERLESS_AUTO_LOGIN_USERNAME=
#PAPERLESS_COOKIE_PREFIX= #PAPERLESS_COOKIE_PREFIX=
#PAPERLESS_ENABLE_HTTP_REMOTE_USER=false #PAPERLESS_ENABLE_HTTP_REMOTE_USER=false
#PAPERLESS_SOCIAL_ACCOUNT_SYNC_SUPERUSER_GROUP=
#PAPERLESS_SOCIAL_ACCOUNT_SYNC_STAFF_GROUP=
# OCR settings # OCR settings
+7 -4
View File
@@ -77,6 +77,7 @@ dependencies = [
"torch~=2.13.0", "torch~=2.13.0",
"watchfiles>=1.2", "watchfiles>=1.2",
"whitenoise~=6.11", "whitenoise~=6.11",
"whoosh-compat[tantivy]==0.1",
"zxing-cpp~=3.1.0", "zxing-cpp~=3.1.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]
@@ -84,9 +85,9 @@ mariadb = [
"mysqlclient~=2.2.7", "mysqlclient~=2.2.7",
] ]
postgres = [ postgres = [
"psycopg[c,pool]==3.3", "psycopg[c,pool]==3.3.4",
# Direct dependency for proper resolution of the pre-built wheels # Direct dependency for proper resolution of the pre-built wheels
"psycopg-c==3.3", "psycopg-c==3.3.4",
"psycopg-pool==3.3.1", "psycopg-pool==3.3.1",
] ]
webserver = [ webserver = [
@@ -160,8 +161,10 @@ explicit = true
[tool.uv.sources] [tool.uv.sources]
# Markers are chosen to select these almost exclusively when building the Docker image # Markers are chosen to select these almost exclusively when building the Docker image
psycopg-c = [ psycopg-c = [
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.0/psycopg_c-3.3.0-cp312-cp312-linux_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.12'" }, { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.12'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.0/psycopg_c-3.3.0-cp312-cp312-linux_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.12'" }, { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.12'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.14'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.14'" },
] ]
torch = [ torch = [
{ index = "pytorch-cpu" }, { index = "pytorch-cpu" },
@@ -33,7 +33,7 @@ test('should warn on unsaved changes', async ({ page }) => {
await page.getByRole('button', { name: 'Close', exact: true }).click() await page.getByRole('button', { name: 'Close', exact: true }).click()
await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/) await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/)
await page.getByRole('button', { name: 'Cancel' }).click() await page.getByRole('button', { name: 'Cancel' }).click()
await page.getByRole('link', { name: 'Close all' }).click() await page.getByRole('button', { name: 'Close all' }).click()
await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/) await expect(page.getByRole('dialog')).toHaveText(/unsaved changes/)
}) })
+653 -221
View File
File diff suppressed because it is too large Load Diff
@@ -89,7 +89,7 @@
</div> </div>
</div> </div>
@if (filterText?.length) { @if (filterText?.length) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetFilter()"> <button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetFilter()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs> <i-bs width="1em" height="1em" name="x"></i-bs>
</button> </button>
} }
@@ -208,7 +208,7 @@
</td> </td>
} }
<td class="d-lg-none"> <td class="d-lg-none">
<button class="btn btn-link" (click)="expandTask(task); $event.stopPropagation();"> <button class="btn btn-link" (click)="expandTask(task); $event.stopPropagation();" aria-label="View task details" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs> <i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
</button> </button>
</td> </td>
@@ -18,9 +18,11 @@
</button> </button>
</pngx-page-header> </pngx-page-header>
<div class="row mb-3"> @if (totalDocuments() > 25) {
<ngb-pagination class="col-auto" [pageSize]="25" [collectionSize]="totalDocuments()" [page]="page()" [maxSize]="5" (pageChange)="page.set($event); reload()" size="sm" aria-label="Pagination"></ngb-pagination> <div class="row mb-3">
</div> <ngb-pagination class="col-auto" [pageSize]="25" [collectionSize]="totalDocuments()" [page]="page()" [maxSize]="5" (pageChange)="page.set($event); reload()" size="sm" aria-label="Pagination"></ngb-pagination>
</div>
}
<div class="card border table-responsive mb-3"> <div class="card border table-responsive mb-3">
<table class="table table-striped align-middle shadow-sm mb-0"> <table class="table table-striped align-middle shadow-sm mb-0">
@@ -64,7 +66,7 @@
<td scope="row"> <td scope="row">
<div class="btn-group d-block d-sm-none"> <div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block"> <div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle> <button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs> <i-bs name="three-dots-vertical"></i-bs>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile"> <div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -4,27 +4,34 @@
(click)="closeMobileSearch(); toggleMenuCollapsed()"> (click)="closeMobileSearch(); toggleMenuCollapsed()">
<span class="navbar-toggler-icon"></span> <span class="navbar-toggler-icon"></span>
</button> </button>
<a class="navbar-brand d-flex align-items-center me-0 px-3 py-3 order-sm-0" <a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0 order-sm-0"
[ngClass]="{ 'slim': slimSidebarEnabled, 'col-auto col-md-3 col-lg-2 col-xxxl-1' : !slimSidebarEnabled, 'py-3' : !customAppTitle?.length || slimSidebarEnabled, 'py-2': customAppTitle?.length }" [ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
routerLink="/dashboard" routerLink="/dashboard"
tourAnchor="tour.intro"> tourAnchor="tour.intro">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor"> @if (!hasCustomBranding) {
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/> <pngx-logo extra_classes="navbar-official-logo px-1" height="2.4rem"></pngx-logo>
</svg> <svg class="brand-mark brand-mark-slim d-none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<div class="ms-2 ms-md-3 d-inline-block" [class.d-md-none]="slimSidebarEnabled"> <path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
@if (customAppTitle?.length) { </svg>
<div class="d-flex flex-column align-items-start custom-title"> } @else {
<span class="title">{{customAppTitle}}</span> @if (customAppLogo) {
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span> <img class="brand-logo" [src]="customAppLogo" alt="" />
</div>
} @else { } @else {
Paperless-ngx <svg class="brand-mark" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
<path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
</svg>
} }
</div> <div class="brand-copy ms-2 text-truncate" [class.d-md-none]="slimSidebarEnabled">
<span class="brand-title text-truncate">{{ appTitle }}</span>
@if (customAppTitle) {
<span class="byline text-uppercase font-monospace" i18n>by Paperless-ngx</span>
}
</div>
}
</a> </a>
<div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 px-3 ps-md-4 me-sm-auto order-3 order-sm-1" <div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 px-3 ps-md-3 me-sm-auto order-3 order-sm-1"
[class.mobile-hidden]="mobileSearchHidden()"> [class.mobile-hidden]="mobileSearchHidden()">
<div class="col-12 col-md-7"> <div class="col-12 header-search">
<pngx-global-search></pngx-global-search> <pngx-global-search></pngx-global-search>
</div> </div>
</div> </div>
@@ -34,7 +41,7 @@
} }
<pngx-toasts-dropdown></pngx-toasts-dropdown> <pngx-toasts-dropdown></pngx-toasts-dropdown>
<li ngbDropdown class="nav-item dropdown"> <li ngbDropdown class="nav-item dropdown">
<button class="btn ps-1 border-0" id="userDropdown" ngbDropdownToggle> <button class="btn navbar-action border-0" id="userDropdown" ngbDropdownToggle aria-label="User menu" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="person-circle"></i-bs> <i-bs width="1.3em" height="1.3em" name="person-circle"></i-bs>
<span class="small ms-2 d-none d-sm-inline"> <span class="small ms-2 d-none d-sm-inline">
{{this.settingsService.displayName}} {{this.settingsService.displayName}}
@@ -71,7 +78,7 @@
[ngClass]="slimSidebarEnabled ? 'slim' : 'col-md-3 col-lg-2 col-xxxl-1'" [class.animating]="slimSidebarAnimating()" [ngClass]="slimSidebarEnabled ? 'slim' : 'col-md-3 col-lg-2 col-xxxl-1'" [class.animating]="slimSidebarAnimating()"
[ngbCollapse]="isMenuCollapsed()"> [ngbCollapse]="isMenuCollapsed()">
@if (canSaveSettings) { @if (canSaveSettings) {
<button class="btn btn-sm btn-dark sidebar-slim-toggler" (click)="toggleSlimSidebar()"> <button class="btn btn-sm btn-dark sidebar-slim-toggler" (click)="toggleSlimSidebar()" [aria-label]="slimSidebarEnabled ? 'Expand sidebar' : 'Collapse sidebar'" i18n-aria-label>
@if (slimSidebarEnabled) { @if (slimSidebarEnabled) {
<i-bs width="0.9em" height="0.9em" name="chevron-double-right"></i-bs> <i-bs width="0.9em" height="0.9em" name="chevron-double-right"></i-bs>
} @else { } @else {
@@ -79,7 +86,7 @@
} }
</button> </button>
} }
<div class="sidebar-sticky pt-3 d-flex flex-column justify-space-around"> <div class="sidebar-sticky pt-3 pb-1 d-flex flex-column justify-space-around">
<ul class="nav flex-column"> <ul class="nav flex-column">
<li class="nav-item app-link"> <li class="nav-item app-link">
<a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()" <a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
@@ -89,7 +96,9 @@
</a> </a>
</li> </li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }"> <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }">
<a class="nav-link" routerLink="documents" routerLinkActive="active" (click)="closeMenu()" <a class="nav-link" routerLink="documents" routerLinkActive="active"
[routerLinkActiveOptions]="{ paths: 'exact', queryParams: 'ignored', matrixParams: 'ignored', fragment: 'ignored' }"
(click)="closeMenu()"
ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim"> container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="files"></i-bs><span><ng-container i18n>Documents</ng-container></span> <i-bs class="me-2" name="files"></i-bs><span><ng-container i18n>Documents</ng-container></span>
@@ -159,11 +168,12 @@
} }
@if (openDocuments.length >= 1) { @if (openDocuments.length >= 1) {
<li class="nav-item w-100 app-link"> <li class="nav-item w-100 app-link">
<a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" [routerLink]="[]" (click)="closeAll()" <button type="button" class="nav-link nav-link-action app-link w-100 text-start"
[class.text-truncate]="!slimSidebarEnabled" (click)="closeAll()"
ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim"> container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="x"></i-bs><span><ng-container i18n>Close all</ng-container></span> <i-bs class="me-2" name="x"></i-bs><span><ng-container i18n>Close all</ng-container></span>
</a> </button>
</li> </li>
} }
</ul> </ul>
@@ -177,10 +187,11 @@
@if (canManageAttributes) { @if (canManageAttributes) {
<li class="nav-item app-link" tourAnchor="tour.tags"> <li class="nav-item app-link" tourAnchor="tour.tags">
<div class="d-flex align-items-center attributes-row"> <div class="d-flex align-items-center attributes-row">
<a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active" (click)="closeMenu()" <a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active"
[routerLinkActiveOptions]="{ exact: !(slimSidebarEnabled || attributesSectionsCollapsed) }" (click)="closeMenu()"
ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim"> container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
<i-bs class="me-2" name="stack"></i-bs><span><ng-container i18n>Attributes</ng-container></span> <i-bs name="stack"></i-bs><span class="ms-2"><ng-container i18n>Attributes</ng-container></span>
</a> </a>
@if (!slimSidebarEnabled && canSaveSettings) { @if (!slimSidebarEnabled && canSaveSettings) {
<button <button
@@ -195,32 +206,32 @@
} }
</div> </div>
<div <div
class="attributes-submenu ms-2" class="attributes-submenu ms-3"
[ngbCollapse]="slimSidebarEnabled || attributesSectionsCollapsed" [ngbCollapse]="slimSidebarEnabled || attributesSectionsCollapsed"
> >
<ul class="nav flex-column"> <ul class="nav flex-column">
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Tag }"> <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Tag }">
<a class="nav-link py-1" routerLink="attributes/tags" routerLinkActive="active" (click)="closeMenu()"> <a class="nav-link" routerLink="attributes/tags" routerLinkActive="active" (click)="closeMenu()">
<i-bs class="me-2" name="tags"></i-bs><span><ng-container i18n>Tags</ng-container></span> <i-bs class="me-2" name="tags"></i-bs><span><ng-container i18n>Tags</ng-container></span>
</a> </a>
</li> </li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Correspondent }"> <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Correspondent }">
<a class="nav-link py-1" routerLink="attributes/correspondents" routerLinkActive="active" (click)="closeMenu()"> <a class="nav-link" routerLink="attributes/correspondents" routerLinkActive="active" (click)="closeMenu()">
<i-bs class="me-2" name="person"></i-bs><span><ng-container i18n>Correspondents</ng-container></span> <i-bs class="me-2" name="person"></i-bs><span><ng-container i18n>Correspondents</ng-container></span>
</a> </a>
</li> </li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.DocumentType }"> <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.DocumentType }">
<a class="nav-link py-1" routerLink="attributes/documenttypes" routerLinkActive="active" (click)="closeMenu()"> <a class="nav-link" routerLink="attributes/documenttypes" routerLinkActive="active" (click)="closeMenu()">
<i-bs class="me-2" name="hash"></i-bs><span><ng-container i18n>Document types</ng-container></span> <i-bs class="me-2" name="hash"></i-bs><span><ng-container i18n>Document types</ng-container></span>
</a> </a>
</li> </li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.StoragePath }"> <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.StoragePath }">
<a class="nav-link py-1" routerLink="attributes/storagepaths" routerLinkActive="active" (click)="closeMenu()"> <a class="nav-link" routerLink="attributes/storagepaths" routerLinkActive="active" (click)="closeMenu()">
<i-bs class="me-2" name="folder"></i-bs><span><ng-container i18n>Storage paths</ng-container></span> <i-bs class="me-2" name="folder"></i-bs><span><ng-container i18n>Storage paths</ng-container></span>
</a> </a>
</li> </li>
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.CustomField }"> <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.CustomField }">
<a class="nav-link py-1" routerLink="attributes/customfields" routerLinkActive="active" (click)="closeMenu()"> <a class="nav-link" routerLink="attributes/customfields" routerLinkActive="active" (click)="closeMenu()">
<i-bs class="me-2" name="ui-radios"></i-bs><span><ng-container i18n>Custom fields</ng-container></span> <i-bs class="me-2" name="ui-radios"></i-bs><span><ng-container i18n>Custom fields</ng-container></span>
</a> </a>
</li> </li>
@@ -363,7 +374,7 @@
</a> </a>
} }
} @else { } @else {
<a *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }" class="small text-decoration-none" routerLink="/settings" fragment="update-checking" <a *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }" class="small text-decoration-none" routerLink="/settings" fragment="update-checking" aria-label="Configure update checking" i18n-aria-label
[ngbPopover]="updateCheckingNotEnabledPopContent" popoverClass="shadow" triggers="mouseenter" [ngbPopover]="updateCheckingNotEnabledPopContent" popoverClass="shadow" triggers="mouseenter"
container="body"> container="body">
<i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs> <i-bs width="1.2em" height="1.2em" name="info-circle"></i-bs>
@@ -7,8 +7,8 @@
bottom: 0; bottom: 0;
left: 0; left: 0;
z-index: 995; /* Behind the navbar */ z-index: 995; /* Behind the navbar */
padding: 50px 0 0; /* Height of navbar */ padding: 64px 0 0; /* Height of navbar */
box-shadow: inset -1px 0 0 rgba(0, 0, 0, .1); border-right: 1px solid color-mix(in srgb, var(--bs-border-color) 65%, transparent);
overflow-y: auto; overflow-y: auto;
--pngx-sidebar-width: 100%; --pngx-sidebar-width: 100%;
max-width: var(--pngx-sidebar-width); max-width: var(--pngx-sidebar-width);
@@ -25,7 +25,7 @@
} }
.view-name { .view-name {
max-width: calc(100% - 50px) max-width: calc(100% - 55px)
} }
.nav-group:not(:has(.app-link)) .sidebar-heading { .nav-group:not(:has(.app-link)) .sidebar-heading {
@@ -47,7 +47,7 @@
} }
@media (max-width: 767.98px) { @media (max-width: 767.98px) {
.sidebar { .sidebar {
top: 3.5rem; top: 4rem;
} }
.search-container { .search-container {
@@ -71,12 +71,14 @@
main { main {
transition: all .2s ease; transition: all .2s ease;
padding-top: 110px; padding-top: 118px;
background: var(--pngx-bg-darker);
min-height: 100vh;
} }
@media (min-width: 768px) { @media (min-width: 768px) {
main { main {
padding-top: 56px; padding-top: 64px;
} }
} }
@@ -91,7 +93,7 @@ main {
@media(min-width: 768px) { @media(min-width: 768px) {
.sidebar.slim { .sidebar.slim {
max-width: 50px; max-width: 55px;
li.nav-item span.badge { li.nav-item span.badge {
display: inline-block; display: inline-block;
@@ -150,7 +152,7 @@ main {
display: block; display: block;
position: fixed; position: fixed;
left: calc(var(--pngx-sidebar-width) - 12px); left: calc(var(--pngx-sidebar-width) - 12px);
top: 60px; top: 72px;
z-index: 996; z-index: 996;
--bs-btn-padding-x: 0.35rem; --bs-btn-padding-x: 0.35rem;
--bs-btn-padding-y: 0.125rem; --bs-btn-padding-y: 0.125rem;
@@ -158,7 +160,7 @@ main {
} }
.sidebar.slim .sidebar-slim-toggler { .sidebar.slim .sidebar-slim-toggler {
--pngx-sidebar-width: 50px !important; --pngx-sidebar-width: 56px !important;
} }
} }
@@ -171,7 +173,7 @@ main {
position: relative; position: relative;
top: 0; top: 0;
height: 100%; height: 100%;
padding-top: 0.5rem; padding: .75rem .5rem 1rem;
overflow-x: hidden; overflow-x: hidden;
overflow-y: auto; /* Scrollable contents if viewport is shorter than content. */ overflow-y: auto; /* Scrollable contents if viewport is shorter than content. */
min-height: min-content; min-height: min-content;
@@ -186,11 +188,14 @@ main {
.sidebar .nav-link { .sidebar .nav-link {
font-weight: 500; font-weight: 500;
white-space: nowrap; white-space: nowrap;
border-left: 2px solid transparent; border-radius: .55rem;
transition: color .15s ease-in-out; margin: 1px 0;
padding: .55rem .7rem;
transition: color .15s ease-in-out, background-color .15s ease-in-out;
&:hover, &.active, &:focus { &:hover, &:focus {
color: var(--bs-primary); color: var(--bs-primary);
background-color: color-mix(in srgb, var(--bs-primary) 8%, transparent);
} }
&:focus-visible { &:focus-visible {
@@ -199,8 +204,21 @@ main {
} }
&.active { &.active {
font-weight: bold; font-weight: 600;
border-left-color: var(--bs-primary); color: var(--bs-primary);
background-color: color-mix(in srgb, var(--bs-primary) 13%, transparent);
}
&.nav-link-action {
color: var(--bs-secondary-color);
font-weight: 400;
background-color: transparent;
&:hover,
&:focus {
color: var(--bs-primary);
background-color: transparent;
}
} }
i-bs { i-bs {
@@ -209,20 +227,45 @@ main {
} }
} }
// sub-page gets marker only .attributes-row {
.nav-item:has(.attributes-submenu.show .nav-link.active) > .attributes-row > .nav-link.active { border-radius: .55rem;
border-left-color: transparent; margin: .1rem 0;
} transition: color .15s ease-in-out, background-color .15s ease-in-out;
// bring sub-menu markers back out to L edge > .nav-link {
.attributes-submenu .nav-link { margin: 0;
margin-left: -0.5rem;
padding-left: calc(var(--bs-nav-link-padding-x) + 0.5rem); &:hover,
&:focus,
&.active {
background-color: transparent;
}
}
&:hover,
&:has(> .nav-link:focus-visible) {
background-color: color-mix(in srgb, var(--bs-primary) 8%, transparent);
}
&:has(> .nav-link.active) {
background-color: color-mix(in srgb, var(--bs-primary) 13%, transparent);
}
} }
.attributes-row .attributes-expand-btn { .attributes-row .attributes-expand-btn {
opacity: 0.2; opacity: 0.2;
width: 1.75rem;
height: 1.75rem;
margin-right: .35rem !important;
border-radius: 50%;
box-shadow: none !important;
transition: opacity 0.15s ease-in-out; transition: opacity 0.15s ease-in-out;
&:focus-visible {
outline: 2px solid color-mix(in srgb, var(--bs-primary) 55%, transparent);
outline-offset: 1px;
opacity: 1;
}
} }
.attributes-row:hover .attributes-expand-btn { .attributes-row:hover .attributes-expand-btn {
@@ -230,8 +273,10 @@ main {
} }
.sidebar-heading { .sidebar-heading {
font-size: 0.75rem; font-size: 0.68rem;
text-transform: uppercase; text-transform: uppercase;
font-weight: 700;
opacity: .75;
} }
.nav { .nav {
@@ -288,16 +333,118 @@ main {
*/ */
.navbar-brand { .navbar-brand {
font-size: 1rem; --pngx-navbar-brand-shadow-rgb: 0, 0, 0;
font-size: 1.0625rem;
min-height: 64px;
letter-spacing: -0.015em;
.flex-column { &:hover,
padding: 0.15rem 0; &:focus-visible {
::ng-deep .navbar-official-logo,
.brand-mark,
.brand-logo {
filter: drop-shadow(0 2px 3px rgba(var(--pngx-navbar-brand-shadow-rgb), .5));
}
}
::ng-deep .navbar-official-logo {
filter: drop-shadow(0 1px 2px rgba(var(--pngx-navbar-brand-shadow-rgb), .3));
transition: filter .15s ease-in-out;
@media screen and (max-width: 575.98px) {
max-height: 2rem;
}
}
.brand-mark {
width: 1.65rem;
height: 1.65rem;
flex: 0 0 auto;
transition: filter .15s ease-in-out;
}
.brand-copy {
display: flex;
flex-direction: column;
align-items: flex-start;
min-width: 0;
line-height: 1.1;
transition: transform .15s ease-in-out;
}
.brand-title {
font-weight: 600;
max-width: 100%;
min-width: 0;
} }
.byline { .byline {
font-size: 0.5rem; margin-top: .15rem;
letter-spacing: 0.1rem; font-size: .5rem;
font-weight: 500;
letter-spacing: .1rem;
opacity: .8;
} }
.brand-logo {
width: auto;
height: 2.75rem;
max-width: 5rem;
flex: 0 0 auto;
object-fit: contain;
transition: filter .15s ease-in-out, transform .15s ease-in-out;
}
}
:host-context(.primary-light) .navbar-brand {
--pngx-navbar-brand-shadow-rgb: 255, 255, 255; // Light app color, use white shadow for dark text
}
:host ::ng-deep .navbar-official-logo {
.leaf {
fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 70%, var(--bs-primary)) !important;
}
.text {
fill: var(--pngx-primary-text-contrast) !important;
}
}
.navbar {
min-height: 64px;
box-shadow: 0 1px 0 rgba(0, 0, 0, .12), 0 4px 18px rgba(0, 0, 0, .08) !important;
}
.navbar > ul {
align-items: center;
gap: .125rem;
padding-right: .5rem;
}
:host ::ng-deep .navbar-action {
display: inline-flex;
align-items: center;
justify-content: center;
min-width: 2.5rem;
min-height: 2.5rem;
padding: .45rem .55rem;
border-radius: .6rem;
transition: background-color .15s ease-in-out, opacity .15s ease-in-out;
&:hover,
&:focus-visible {
background-color: rgba(0, 0, 0, .14);
}
}
#userDropdown {
padding-left: .75rem;
}
.header-search {
width: 100%;
max-width: 44rem;
} }
@media screen and (max-width: 575.98px) { @media screen and (max-width: 575.98px) {
@@ -346,18 +493,44 @@ main {
grid-area: actions; grid-area: actions;
justify-self: end; justify-self: end;
flex-wrap: nowrap; flex-wrap: nowrap;
gap: 0;
padding-right: .25rem;
}
:host ::ng-deep .navbar-action {
min-width: 2.25rem;
padding-right: .4rem;
padding-left: .4rem;
}
#userDropdown {
padding-right: .35rem;
padding-left: .45rem;
} }
} }
@media screen and (min-width: 768px) { @media screen and (min-width: 768px) {
.navbar-brand.slim { .navbar-brand.slim {
max-width: 50px; max-width: 55px;
.brand-logo {
width: 1.65rem;
max-width: 1.65rem;
}
.brand-mark-slim {
display: block !important;
}
}
:host ::ng-deep .navbar-brand.slim .navbar-official-logo {
display: none;
} }
} }
:host ::ng-deep .dropdown.show .dropdown-toggle, :host ::ng-deep .dropdown.show .dropdown-toggle,
:host ::ng-deep .dropdown-toggle:hover { :host ::ng-deep .dropdown-toggle:hover {
opacity: 0.7; opacity: 1;
} }
.dropdown-toggle::after { .dropdown-toggle::after {
@@ -45,6 +45,7 @@ import { TasksService } from 'src/app/services/tasks.service'
import { ToastService } from 'src/app/services/toast.service' import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment' import { environment } from 'src/environments/environment'
import { ChatComponent } from '../chat/chat/chat.component' import { ChatComponent } from '../chat/chat/chat.component'
import { LogoComponent } from '../common/logo/logo.component'
import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component' import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
import { DocumentDetailComponent } from '../document-detail/document-detail.component' import { DocumentDetailComponent } from '../document-detail/document-detail.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component' import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
@@ -59,6 +60,7 @@ const SCROLL_THRESHOLD = 16
styleUrls: ['./app-frame.component.scss'], styleUrls: ['./app-frame.component.scss'],
imports: [ imports: [
GlobalSearchComponent, GlobalSearchComponent,
LogoComponent,
DocumentTitlePipe, DocumentTitlePipe,
IfPermissionsDirective, IfPermissionsDirective,
ToastsDropdownComponent, ToastsDropdownComponent,
@@ -190,11 +192,34 @@ export class AppFrameComponent
return `${environment.appTitle} v${this.settingsService.get(SETTINGS_KEYS.VERSION)}${environment.tag === 'prod' ? '' : ` #${environment.tag}`}` return `${environment.appTitle} v${this.settingsService.get(SETTINGS_KEYS.VERSION)}${environment.tag === 'prod' ? '' : ` #${environment.tag}`}`
} }
get appTitle(): string {
this.settingsService.trackChanges()
return (
this.settingsService.get(SETTINGS_KEYS.APP_TITLE) || environment.appTitle
)
}
get customAppTitle(): string { get customAppTitle(): string {
this.settingsService.trackChanges() this.settingsService.trackChanges()
return this.settingsService.get(SETTINGS_KEYS.APP_TITLE) return this.settingsService.get(SETTINGS_KEYS.APP_TITLE)
} }
get hasCustomBranding(): boolean {
this.settingsService.trackChanges()
return !!(
this.settingsService.get(SETTINGS_KEYS.APP_TITLE)?.length ||
this.settingsService.get(SETTINGS_KEYS.APP_LOGO)?.length
)
}
get customAppLogo(): string {
this.settingsService.trackChanges()
const logo = this.settingsService.get(SETTINGS_KEYS.APP_LOGO)
return logo?.length
? environment.apiBaseUrl.replace(/\/api\/$/, logo)
: null
}
get canSaveSettings(): boolean { get canSaveSettings(): boolean {
return ( return (
this.permissionsService.currentUserCan( this.permissionsService.currentUserCan(
@@ -4,12 +4,12 @@ form {
> i-bs[name="search"] { > i-bs[name="search"] {
position: absolute; position: absolute;
left: 0.6rem; left: 0.6rem;
top: .35rem; top: .25rem;
color: rgba(255, 255, 255, 0.6); color: rgba(255, 255, 255, 0.6);
@media screen and (min-width: 768px) { @media screen and (min-width: 768px) {
// adjust for smaller font size on non-mobile // adjust for smaller font size on non-mobile
top: 0.25rem; top: .15rem;
} }
} }
@@ -37,8 +37,9 @@ form {
} }
.form-control { .form-control {
color: rgba(255, 255, 255, 0.3); min-height: 2.25rem;
background-color: rgba(0, 0, 0, 0.15); color: rgba(255, 255, 255, 0.55);
background-color: rgba(0, 0, 0, 0.16);
padding-left: 1.8rem; padding-left: 1.8rem;
border-color: rgba(255, 255, 255, 0.2); border-color: rgba(255, 255, 255, 0.2);
transition: all .3s ease, padding-left 0s ease, background-color 0s ease; // Safari requires all transition: all .3s ease, padding-left 0s ease, background-color 0s ease; // Safari requires all
@@ -52,7 +53,7 @@ form {
} }
&:focus-within { &:focus-within {
background-color: rgba(0, 0, 0, 0.3); background-color: rgba(0, 0, 0, 0.26);
color: var(--bs-light); color: var(--bs-light);
flex-grow: 1; flex-grow: 1;
padding-left: 0.5rem; padding-left: 0.5rem;
@@ -1,9 +1,9 @@
<li ngbDropdown class="nav-item mx-1" (openChange)="onOpenChange($event)"> <li ngbDropdown class="nav-item position-relative" (openChange)="onOpenChange($event)">
@if (toasts().length) { @if (toasts().length) {
<span class="badge rounded-pill z-3 pe-none bg-secondary me-2 position-absolute top-0 left-0">{{ toasts().length }}</span> <span class="notification-count badge rounded-pill z-3 pe-none bg-secondary position-absolute">{{ toasts().length }}</span>
} }
<button class="btn border-0" id="notificationsDropdown" ngbDropdownToggle> <button class="btn navbar-action border-0" id="notificationsDropdown" ngbDropdownToggle aria-label="Notifications" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="bell"></i-bs> <i-bs width="1.3em" height="1.3em" name="bell"></i-bs>
</button> </button>
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="notificationsDropdown"> <div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="notificationsDropdown">
@@ -11,6 +11,16 @@
display: none; display: none;
} }
.notification-count {
top: -.2rem;
right: -.2rem;
min-width: 1.15rem;
height: 1.15rem;
padding: .18rem .32rem;
font-size: .68rem;
line-height: 1;
}
.dropdown-item { .dropdown-item {
white-space: initial; white-space: initial;
} }
@@ -1,6 +1,6 @@
<li ngbDropdown class="nav-item me-n2" (openChange)="onOpenChange($event)"> <li ngbDropdown class="nav-item" (openChange)="onOpenChange($event)">
<button class="btn border-0" id="chatDropdown" ngbDropdownToggle> <button class="btn navbar-action border-0" id="chatDropdown" ngbDropdownToggle aria-label="Chat" i18n-aria-label>
<i-bs width="1.3em" height="1.3em" name="chatSquareDots"></i-bs> <i-bs width="1.3em" height="1.3em" name="chatSquareDots"></i-bs>
</button> </button>
<div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="chatDropdown"> <div ngbDropdownMenu class="dropdown-menu-end shadow p-3" aria-labelledby="chatDropdown">
@@ -0,0 +1,58 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()"></button>
</div>
<div class="modal-body">
<p>{{message}}</p>
<div class="form-group">
<label class="form-label" for="rootDocumentID" i18n>Root document:</label>
<select id="rootDocumentID" class="form-select" [ngModel]="rootDocumentID()" (ngModelChange)="rootDocumentID.set($event)">
@for (document of documents(); track document.id) {
<option [ngValue]="document.id">{{document.title}}</option>
}
</select>
</div>
<div class="form-group mt-4">
<span class="form-label d-inline-block" i18n>Versions (oldest first):</span>
<ul class="list-group"
cdkDropList
[cdkDropListData]="versionDocumentIDs()"
(cdkDropListDropped)="onDrop($event)">
@for (documentID of versionDocumentIDs(); track documentID) {
@let document = getDocument(documentID);
@if (document) {
<li class="list-group-item d-flex align-items-center" cdkDrag>
<i-bs name="grip-vertical" class="me-2"></i-bs>
<div class="d-flex flex-column">
<div>
@if (document.correspondent) {
<b>{{document.correspondent | correspondentName | async}}: </b>
}{{document.title}}
</div>
<small class="text-muted">
{{document.created | customDate:'mediumDate'}}
@if (document.page_count) {
| {document.page_count, plural, =1 {One page} other {{{document.page_count}} pages}}
}
</small>
</div>
@if ($last) {
<span class="badge bg-primary ms-auto" i18n>Current version</span>
}
</li>
}
}
</ul>
@if (versionDocumentIDs().length > 1) {
<div class="form-text" i18n>Drag to reorder.</div>
}
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled()">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled()">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,70 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of } from 'rxjs'
import { DocumentService } from 'src/app/services/rest/document.service'
import { MergeAsVersionsConfirmDialogComponent } from './merge-as-versions-confirm-dialog.component'
describe('MergeAsVersionsConfirmDialogComponent', () => {
let component: MergeAsVersionsConfirmDialogComponent
let fixture: ComponentFixture<MergeAsVersionsConfirmDialogComponent>
let documentService: DocumentService
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
NgxBootstrapIconsModule.pick(allIcons),
MergeAsVersionsConfirmDialogComponent,
],
providers: [
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
}).compileComponents()
fixture = TestBed.createComponent(MergeAsVersionsConfirmDialogComponent)
documentService = TestBed.inject(DocumentService)
component = fixture.componentInstance
fixture.detectChanges()
})
it('should fetch selected documents', () => {
const documents = [
{ id: 1, title: 'Document 1' },
{ id: 2, title: 'Document 2' },
]
jest.spyOn(documentService, 'getFew').mockReturnValue(
of({
all: [1, 2],
count: 2,
results: documents,
})
)
component.documentIDs.set([1, 2])
component.ngOnInit()
expect(component.documents()).toEqual(documents)
expect(documentService.getFew).toHaveBeenCalledWith([1, 2])
})
it('should exclude the root from the draggable documents', () => {
component.documentIDs.set([1, 2, 3])
component.rootDocumentID.set(2)
expect(component.versionDocumentIDs()).toEqual([1, 3])
})
it('should move draggable documents while keeping the root fixed', () => {
component.documentIDs.set([1, 2, 3])
component.rootDocumentID.set(1)
component.onDrop({ previousIndex: 1, currentIndex: 0 } as any)
expect(component.documentIDs()).toEqual([1, 3, 2])
expect(component.versionDocumentIDs()).toEqual([3, 2])
})
})
@@ -0,0 +1,70 @@
import {
CdkDragDrop,
DragDropModule,
moveItemInArray,
} from '@angular/cdk/drag-drop'
import { AsyncPipe } from '@angular/common'
import { Component, OnInit, computed, inject, signal } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { takeUntil } from 'rxjs'
import { Document } from 'src/app/data/document'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentService } from 'src/app/services/rest/document.service'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-merge-as-versions-confirm-dialog',
templateUrl: './merge-as-versions-confirm-dialog.component.html',
styleUrl: './merge-as-versions-confirm-dialog.component.scss',
imports: [
AsyncPipe,
CorrespondentNamePipe,
CustomDatePipe,
DragDropModule,
FormsModule,
NgxBootstrapIconsModule,
],
})
export class MergeAsVersionsConfirmDialogComponent
extends ConfirmDialogComponent
implements OnInit
{
private readonly documentService = inject(DocumentService)
readonly documentIDs = signal<number[]>([])
readonly documents = signal<Document[]>([])
readonly rootDocumentID = signal(-1)
readonly versionDocumentIDs = computed(() =>
this.documentIDs().filter(
(documentID) => documentID !== this.rootDocumentID()
)
)
ngOnInit() {
this.documentService
.getFew(this.documentIDs())
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((response) => this.documents.set(response.results))
}
onDrop(event: CdkDragDrop<number[]>) {
const versionDocumentIDs = this.versionDocumentIDs().concat()
moveItemInArray(versionDocumentIDs, event.previousIndex, event.currentIndex)
// The root keeps its place in the list, only the versions move around it
let versionIndex = 0
this.documentIDs.update((documentIDs) =>
documentIDs.map((documentID) =>
documentID === this.rootDocumentID()
? documentID
: versionDocumentIDs[versionIndex++]
)
)
}
getDocument(documentID: number): Document | undefined {
return this.documents().find((document) => document.id === documentID)
}
}
@@ -36,7 +36,7 @@
</div> </div>
<div class="form-group mt-4"> <div class="form-group mt-4">
<label class="form-label" for="metadataDocumentID" i18n>Use metadata from:</label> <label class="form-label" for="metadataDocumentID" i18n>Use metadata from:</label>
<select class="form-select" [ngModel]="metadataDocumentID()" (ngModelChange)="metadataDocumentID.set($event)"> <select id="metadataDocumentID" class="form-select" [ngModel]="metadataDocumentID()" (ngModelChange)="metadataDocumentID.set($event)">
<option [ngValue]="-1" i18n>Regenerate all metadata</option> <option [ngValue]="-1" i18n>Regenerate all metadata</option>
@for (document of documents(); track document.id) { @for (document of documents(); track document.id) {
<option [ngValue]="document.id">{{document.title}}</option> <option [ngValue]="document.id">{{document.title}}</option>
@@ -6,7 +6,7 @@
<div class="modal-body"> <div class="modal-body">
<div class="row"> <div class="row">
<div class="col-2 d-flex justify-content-end"> <div class="col-2 d-flex justify-content-end">
<button class="btn btn-secondary mt-auto" (click)="rotate(false)"> <button class="btn btn-secondary mt-auto" (click)="rotate(false)" aria-label="Rotate counterclockwise" i18n-aria-label>
<i-bs name="arrow-counterclockwise"></i-bs> <i-bs name="arrow-counterclockwise"></i-bs>
</button> </button>
</div> </div>
@@ -16,7 +16,7 @@
} }
</div> </div>
<div class="col-2 d-flex"> <div class="col-2 d-flex">
<button class="btn btn-secondary mt-auto" (click)="rotate()"> <button class="btn btn-secondary mt-auto" (click)="rotate()" aria-label="Rotate clockwise" i18n-aria-label>
<i-bs name="arrow-clockwise"></i-bs> <i-bs name="arrow-clockwise"></i-bs>
</button> </button>
</div> </div>
@@ -1,5 +1,5 @@
<div ngbDropdown #fieldDropdown="ngbDropdown" (openChange)="onOpenClose($event)" [popperOptions]="popperOptions"> <div ngbDropdown #fieldDropdown="ngbDropdown" (openChange)="onOpenClose($event)" [popperOptions]="popperOptions">
<button type="button" class="btn btn-sm btn-outline-primary" id="customFieldsDropdown" [disabled]="disabled" ngbDropdownToggle> <button type="button" class="btn btn-sm btn-outline-primary" id="customFieldsDropdown" [disabled]="disabled" ngbDropdownToggle aria-label="Custom Fields" i18n-aria-label>
<i-bs name="ui-radios"></i-bs><div class="d-none d-lg-inline ms-1"><ng-container i18n>Custom Fields</ng-container></div> <i-bs name="ui-radios"></i-bs><div class="d-none d-lg-inline ms-1"><ng-container i18n>Custom Fields</ng-container></div>
</button> </button>
<div ngbDropdownMenu aria-labelledby="customFieldsDropdown" class="shadow custom-fields-dropdown"> <div ngbDropdownMenu aria-labelledby="customFieldsDropdown" class="shadow custom-fields-dropdown">
@@ -1,6 +1,6 @@
@if (useDropdown) { @if (useDropdown) {
<div class="btn-group w-100" role="group" ngbDropdown #dropdown="ngbDropdown" (openChange)="onOpenChange($event)" [popperOptions]="popperOptions"> <div class="btn-group w-100" role="group" ngbDropdown #dropdown="ngbDropdown" (openChange)="onOpenChange($event)" [popperOptions]="popperOptions">
<button class="btn btn-sm btn-outline-primary" id="dropdown_toggle" ngbDropdownToggle [disabled]="disabled"> <button class="btn btn-sm btn-outline-primary" id="dropdown_toggle" ngbDropdownToggle [disabled]="disabled" [aria-label]="title">
<i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div> <i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
@if (isActive) { @if (isActive) {
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge> <pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge>
@@ -38,7 +38,7 @@
ngbDatepicker ngbDatepicker
#d="ngbDatepicker" #d="ngbDatepicker"
[footerTemplate]="datePickerFooterTemplate" /> [footerTemplate]="datePickerFooterTemplate" />
<button class="btn btn-sm btn-outline-secondary rounded-end" (click)="d.toggle()" type="button"> <button class="btn btn-sm btn-outline-secondary rounded-end" (click)="d.toggle()" type="button" aria-label="Open date picker" i18n-aria-label>
<i-bs name="calendar-event"></i-bs> <i-bs name="calendar-event"></i-bs>
</button> </button>
<ng-template #datePickerFooterTemplate> <ng-template #datePickerFooterTemplate>
@@ -143,7 +143,7 @@
<input class="w-25 form-control rounded-end" type="text" [(ngModel)]="atom.value" [disabled]="disabled"> <input class="w-25 form-control rounded-end" type="text" [(ngModel)]="atom.value" [disabled]="disabled">
} }
} }
<button class="btn btn-link btn-sm text-danger pe-0" type="button" (click)="removeElement(atom)" [disabled]="disabled"> <button class="btn btn-link btn-sm text-danger pe-0" type="button" (click)="removeElement(atom)" [disabled]="disabled" aria-label="Remove query" i18n-aria-label>
<i-bs name="x-circle"></i-bs> <i-bs name="x-circle"></i-bs>
</button> </button>
</div> </div>
@@ -185,7 +185,7 @@
<i-bs name="braces"></i-bs> <i-bs name="braces"></i-bs>
</button> </button>
@if (expression.depth > 0) { @if (expression.depth > 0) {
<button type="button" class="btn btn-sm btn-outline-secondary text-danger" (click)="removeElement(expression)" [disabled]="disabled"> <button type="button" class="btn btn-sm btn-outline-secondary text-danger" (click)="removeElement(expression)" [disabled]="disabled" aria-label="Remove expression" i18n-aria-label>
<i-bs name="x-circle"></i-bs> <i-bs name="x-circle"></i-bs>
</button> </button>
} }
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group" [popperOptions]="popperOptions" [placement]="placement"> <div class="btn-group w-100" ngbDropdown role="group" [popperOptions]="popperOptions" [placement]="placement">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="createdDateTo || createdDateFrom ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled"> <button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="createdDateTo || createdDateFrom ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs width="1em" height="1em" name="calendar-event-fill"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div> <i-bs width="1em" height="1em" name="calendar-event-fill"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span> <pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
</button> </button>
@@ -9,7 +9,7 @@
<div class="list-group-item d-flex p-2 select-item" role="menuitem"> <div class="list-group-item d-flex p-2 select-item" role="menuitem">
<div class="selected-icon"> <div class="selected-icon">
@if (createdRelativeDate) { @if (createdRelativeDate) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedRelativeDate()"> <a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedRelativeDate()" aria-label="Clear created relative date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs> <i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs> <i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a> </a>
@@ -33,7 +33,7 @@
<div class="list-group-item d-flex p-2" role="menuitem"> <div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon"> <div class="selected-icon">
@if (createdDateFrom) { @if (createdDateFrom) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedFrom()"> <a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedFrom()" aria-label="Clear created from date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs> <i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs> <i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a> </a>
@@ -43,7 +43,7 @@
<span class="input-group-text w-25 small text-muted" i18n>From</span> <span class="input-group-text w-25 small text-muted" i18n>From</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)" <input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="createdDateFrom" ngbDatepicker #createdDateFromPicker="ngbDatepicker" [footerTemplate]="createdFromFooterTemplate"> maxlength="10" [(ngModel)]="createdDateFrom" ngbDatepicker #createdDateFromPicker="ngbDatepicker" [footerTemplate]="createdFromFooterTemplate">
<button class="btn btn-outline-secondary" (click)="createdDateFromPicker.toggle()" type="button"> <button class="btn btn-outline-secondary" (click)="createdDateFromPicker.toggle()" type="button" aria-label="Open created from date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs> <i-bs width="1em" height="1em" name="calendar"></i-bs>
</button> </button>
<ng-template #createdFromFooterTemplate> <ng-template #createdFromFooterTemplate>
@@ -57,7 +57,7 @@
<div class="list-group-item d-flex p-2" role="menuitem"> <div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon"> <div class="selected-icon">
@if (createdDateTo) { @if (createdDateTo) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedTo()"> <a class="text-light focus-variants" href="javascript:void(0)" (click)="clearCreatedTo()" aria-label="Clear created to date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs> <i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs> <i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a> </a>
@@ -67,7 +67,7 @@
<span class="input-group-text w-25 small text-muted" i18n>To</span> <span class="input-group-text w-25 small text-muted" i18n>To</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)" <input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="createdDateTo" ngbDatepicker #createdDateToPicker="ngbDatepicker" [footerTemplate]="createdToFooterTemplate"> maxlength="10" [(ngModel)]="createdDateTo" ngbDatepicker #createdDateToPicker="ngbDatepicker" [footerTemplate]="createdToFooterTemplate">
<button class="btn btn-outline-secondary" (click)="createdDateToPicker.toggle()" type="button"> <button class="btn btn-outline-secondary" (click)="createdDateToPicker.toggle()" type="button" aria-label="Open created to date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs> <i-bs width="1em" height="1em" name="calendar"></i-bs>
</button> </button>
<ng-template #createdToFooterTemplate> <ng-template #createdToFooterTemplate>
@@ -85,7 +85,7 @@
<div class="list-group-item d-flex p-2 select-item" role="menuitem"> <div class="list-group-item d-flex p-2 select-item" role="menuitem">
<div class="selected-icon"> <div class="selected-icon">
@if (addedRelativeDate) { @if (addedRelativeDate) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedRelativeDate()"> <a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedRelativeDate()" aria-label="Clear added relative date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs> <i-bs width="1em" height="1em" name="check" class="variant-unfocused text-dark"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs> <i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a> </a>
@@ -109,7 +109,7 @@
<div class="list-group-item d-flex p-2" role="menuitem"> <div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon"> <div class="selected-icon">
@if (addedDateFrom) { @if (addedDateFrom) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedFrom()"> <a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedFrom()" aria-label="Clear added from date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs> <i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs> <i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a> </a>
@@ -119,7 +119,7 @@
<span class="input-group-text w-25 small text-muted" i18n>From</span> <span class="input-group-text w-25 small text-muted" i18n>From</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)" <input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="addedDateFrom" ngbDatepicker #addedDateFromPicker="ngbDatepicker" [footerTemplate]="addedFromFooterTemplate"> maxlength="10" [(ngModel)]="addedDateFrom" ngbDatepicker #addedDateFromPicker="ngbDatepicker" [footerTemplate]="addedFromFooterTemplate">
<button class="btn btn-outline-secondary" (click)="addedDateFromPicker.toggle()" type="button"> <button class="btn btn-outline-secondary" (click)="addedDateFromPicker.toggle()" type="button" aria-label="Open added from date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs> <i-bs width="1em" height="1em" name="calendar"></i-bs>
</button> </button>
<ng-template #addedFromFooterTemplate> <ng-template #addedFromFooterTemplate>
@@ -133,7 +133,7 @@
<div class="list-group-item d-flex p-2" role="menuitem"> <div class="list-group-item d-flex p-2" role="menuitem">
<div class="selected-icon"> <div class="selected-icon">
@if (addedDateTo) { @if (addedDateTo) {
<a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedTo()"> <a class="text-light focus-variants" href="javascript:void(0)" (click)="clearAddedTo()" aria-label="Clear added to date" i18n-aria-label>
<i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs> <i-bs width="1em" height="1em" name="check" class="variant-unfocused"></i-bs>
<i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs> <i-bs width="1em" height="1em" name="x" class="variant-focused text-primary"></i-bs>
</a> </a>
@@ -143,7 +143,7 @@
<span class="input-group-text w-25 small text-muted" i18n>To</span> <span class="input-group-text w-25 small text-muted" i18n>To</span>
<input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)" <input class="form-control small" [placeholder]="datePlaceHolder" (dateSelect)="onChangeDebounce()" (change)="onChangeDebounce()" (keypress)="onKeyPress($event)"
maxlength="10" [(ngModel)]="addedDateTo" ngbDatepicker #addedDateToPicker="ngbDatepicker" [footerTemplate]="addedToFooterTemplate"> maxlength="10" [(ngModel)]="addedDateTo" ngbDatepicker #addedDateToPicker="ngbDatepicker" [footerTemplate]="addedToFooterTemplate">
<button class="btn btn-outline-secondary" (click)="addedDateToPicker.toggle()" type="button"> <button class="btn btn-outline-secondary" (click)="addedDateToPicker.toggle()" type="button" aria-label="Open added to date picker" i18n-aria-label>
<i-bs width="1em" height="1em" name="calendar"></i-bs> <i-bs width="1em" height="1em" name="calendar"></i-bs>
</button> </button>
<ng-template #addedToFooterTemplate> <ng-template #addedToFooterTemplate>
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group" (openChange)="dropdownOpenChange($event)" #dropdown="ngbDropdown" (keydown)="listKeyDown($event)" [popperOptions]="popperOptions" [autoClose]="!creating()"> <div class="btn-group w-100" ngbDropdown role="group" (openChange)="dropdownOpenChange($event)" #dropdown="ngbDropdown" (keydown)="listKeyDown($event)" [popperOptions]="popperOptions" [autoClose]="!creating()">
<button class="btn btn-sm" id="dropdown_{{name}}" ngbDropdownToggle [ngClass]="!editing && selectionModel.selectionSize() > 0 ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled"> <button class="btn btn-sm" id="dropdown_{{name}}" ngbDropdownToggle [ngClass]="!editing && selectionModel.selectionSize() > 0 ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div> <i-bs name="{{icon}}"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
@if (!editing && selectionModel.totalCount > 0) { @if (!editing && selectionModel.totalCount > 0) {
<pngx-clearable-badge [number]="selectionModel.totalCount" [selected]="selectionModel.selectionSize() > 0" (cleared)="reset()"></pngx-clearable-badge> <pngx-clearable-badge [number]="selectionModel.totalCount" [selected]="selectionModel.selectionSize() > 0" (cleared)="reset()"></pngx-clearable-badge>
@@ -4,7 +4,7 @@
} }
<div class="input-group" [class.is-invalid]="error"> <div class="input-group" [class.is-invalid]="error">
<button type="button" class="input-group-text" [style.background-color]="value" (click)="colorPicker.toggle()">&nbsp;&nbsp;&nbsp;</button> <button type="button" class="input-group-text" [style.background-color]="value" (click)="colorPicker.toggle()" aria-label="Open color picker" i18n-aria-label>&nbsp;&nbsp;&nbsp;</button>
<ng-template #popContent> <ng-template #popContent>
<div style="min-width: 200px;" class="pb-3"> <div style="min-width: 200px;" class="pb-3">
@@ -14,7 +14,7 @@
<input #inputField class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" (change)="onChange(value)" [autoClose]="'outside'" [ngbPopover]="popContent" #colorPicker="ngbPopover" placement="bottom" popoverClass="shadow"> <input #inputField class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" (change)="onChange(value)" [autoClose]="'outside'" [ngbPopover]="popContent" #colorPicker="ngbPopover" placement="bottom" popoverClass="shadow">
<button class="btn btn-outline-secondary" type="button" (click)="randomize()"> <button class="btn btn-outline-secondary" type="button" (click)="randomize()" aria-label="Choose a random color" i18n-aria-label>
<i-bs name="dice5"></i-bs> <i-bs name="dice5"></i-bs>
</button> </button>
@@ -74,7 +74,7 @@
class="flex-grow-1"></pngx-input-textarea> class="flex-grow-1"></pngx-input-textarea>
} }
} }
<button type="button" class="btn btn-link text-danger" (click)="removeSelectedField.next(fieldId)"> <button type="button" class="btn btn-link text-danger" (click)="removeSelectedField.next(fieldId)" aria-label="Remove custom field" i18n-aria-label>
<i-bs name="trash"></i-bs> <i-bs name="trash"></i-bs>
</button> </button>
</div> </div>
@@ -13,7 +13,7 @@
<input #inputField class="form-control" [class.is-invalid]="error" [placeholder]="placeholder" [id]="inputId" maxlength="10" <input #inputField class="form-control" [class.is-invalid]="error" [placeholder]="placeholder" [id]="inputId" maxlength="10"
(dateSelect)="onChange(value)" (change)="onChange(value)" (keypress)="onKeyPress($event)" (paste)="onPaste($event)" (dateSelect)="onChange(value)" (change)="onChange(value)" (keypress)="onKeyPress($event)" (paste)="onPaste($event)"
name="dp" [(ngModel)]="value" ngbDatepicker #datePicker="ngbDatepicker" #datePickerContent="ngModel" [disabled]="disabled" [footerTemplate]="datePickerFooterTemplate"> name="dp" [(ngModel)]="value" ngbDatepicker #datePicker="ngbDatepicker" #datePickerContent="ngModel" [disabled]="disabled" [footerTemplate]="datePickerFooterTemplate">
<button class="btn btn-outline-secondary calendar" (click)="datePicker.toggle()" type="button" [disabled]="disabled"> <button class="btn btn-outline-secondary calendar" (click)="datePicker.toggle()" type="button" [disabled]="disabled" aria-label="Open date picker" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="calendar"></i-bs> <i-bs width="1.2em" height="1.2em" name="calendar"></i-bs>
</button> </button>
<ng-template #datePickerFooterTemplate> <ng-template #datePickerFooterTemplate>
@@ -13,7 +13,7 @@
<div class="input-group mb-3"> <div class="input-group mb-3">
<input type="text" class="form-control" [(ngModel)]="entry[0]" (change)="inputChange()" [disabled]="disabled" autocomplete="off"> <input type="text" class="form-control" [(ngModel)]="entry[0]" (change)="inputChange()" [disabled]="disabled" autocomplete="off">
<input type="text" class="form-control" [(ngModel)]="entry[1]" (change)="inputChange()" [disabled]="disabled" autocomplete="off"> <input type="text" class="form-control" [(ngModel)]="entry[1]" (change)="inputChange()" [disabled]="disabled" autocomplete="off">
<button type="button" class="btn btn-outline-secondary" (click)="removeEntry(i)"> <button type="button" class="btn btn-outline-secondary" (click)="removeEntry(i)" aria-label="Remove entry" i18n-aria-label>
<i-bs class="text-danger" name="trash"></i-bs> <i-bs class="text-danger" name="trash"></i-bs>
</button> </button>
</div> </div>
@@ -50,7 +50,7 @@
</ng-template> </ng-template>
</ng-select> </ng-select>
@if (allowCreateNew && !hideAddButton) { @if (allowCreateNew && !hideAddButton) {
<button class="btn btn-outline-secondary" type="button" (click)="addItem()" [disabled]="disabled"> <button class="btn btn-outline-secondary" type="button" (click)="addItem()" [disabled]="disabled" aria-label="Create new item" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="plus"></i-bs> <i-bs width="1.2em" height="1.2em" name="plus"></i-bs>
</button> </button>
} }
@@ -48,7 +48,7 @@
</ng-template> </ng-template>
</ng-select> </ng-select>
@if (allowCreate && !hideAddButton) { @if (allowCreate && !hideAddButton) {
<button class="btn btn-outline-secondary" type="button" (click)="createTag(null, true)" [disabled]="disabled"> <button class="btn btn-outline-secondary" type="button" (click)="createTag(null, true)" [disabled]="disabled" aria-label="Create new tag" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="plus"></i-bs> <i-bs width="1.2em" height="1.2em" name="plus"></i-bs>
</button> </button>
} }
@@ -7,6 +7,15 @@ h3 {
} }
} }
:host {
display: block;
margin-bottom: .35rem;
}
h3 > .h6 {
color: var(--bs-secondary-color);
}
@media (min-width: 1200px) { @media (min-width: 1200px) {
h3 { h3 {
min-height: 2.8rem; min-height: 2.8rem;
@@ -1,5 +1,5 @@
<div class="btn-group w-100" ngbDropdown role="group"> <div class="btn-group w-100" ngbDropdown role="group">
<button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="isActive ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled"> <button class="btn btn-sm" id="dropdown{{title}}" ngbDropdownToggle [ngClass]="isActive ? 'btn-primary' : 'btn-outline-primary'" [disabled]="disabled" [aria-label]="title">
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div> <i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1">{{title}}</div>
<pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span> <pngx-clearable-badge [selected]="isActive" (cleared)="reset()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
</button> </button>
@@ -107,6 +107,25 @@ describe('PermissionsSelectComponent', () => {
expect(component.form.get('Tag').get('Change').disabled).toBeTruthy() expect(component.form.get('Tag').get('Change').disabled).toBeTruthy()
}) })
it('should update checkboxes when inherited permissions change', () => {
component.ngOnInit()
component.inheritedPermissions = ['documents.change_document']
component.writeValue(['delete_document'])
expect(component.form.get('Document').get('Change').value).toBeTruthy()
expect(component.form.get('Document').get('Change').disabled).toBeTruthy()
// swap for a group with a different permission, but the same number of them
component.inheritedPermissions = ['documents.view_document']
// the no-longer-inherited permission is unchecked, the explicit one is kept
expect(component.permissions).toEqual(['delete_document'])
expect(component.form.get('Document').get('Change').value).toBeFalsy()
expect(component.form.get('Document').get('Change').disabled).toBeFalsy()
expect(component.form.get('Document').get('Delete').value).toBeTruthy()
expect(component.form.get('Document').get('View').value).toBeTruthy()
expect(component.form.get('Document').get('View').disabled).toBeTruthy()
})
it('should exclude history permissions if disabled', () => { it('should exclude history permissions if disabled', () => {
settingsService.set(SETTINGS_KEYS.AUDITLOG_ENABLED, false) settingsService.set(SETTINGS_KEYS.AUDITLOG_ENABLED, false)
fixture = TestBed.createComponent(PermissionsSelectComponent) fixture = TestBed.createComponent(PermissionsSelectComponent)
@@ -74,12 +74,22 @@ export class PermissionsSelectComponent
? inherited.map((p) => p.replace(/^\w+\./g, '')) ? inherited.map((p) => p.replace(/^\w+\./g, ''))
: [] : []
if (this._inheritedPermissions !== newInheritedPermissions) { const changed =
this._inheritedPermissions = newInheritedPermissions newInheritedPermissions.length !== this._inheritedPermissions.length ||
this.writeValue(this.permissions) // updates visual checks etc. newInheritedPermissions.some(
} (p) => !this._inheritedPermissions.includes(p)
)
this.updateDisabledStates() if (changed) {
// skip inherited permissions, these are the explicitly set ones
this.permissions = this.getSelectedPermissions(
this.form.getRawValue()
).filter((p) => !this._inheritedPermissions.includes(p))
this._inheritedPermissions = newInheritedPermissions
this.applyCheckedState()
} else {
this.updateDisabledStates()
}
} }
inheritedWarning: string = $localize`Inherited from group` inheritedWarning: string = $localize`Inherited from group`
@@ -106,20 +116,29 @@ export class PermissionsSelectComponent
} }
this.permissions = permissions ?? [] this.permissions = permissions ?? []
const allPerms = this._inheritedPermissions.concat(this.permissions) this.applyCheckedState()
}
allPerms.forEach((permissionStr) => { // sets every checkbox from inherited + own perms
const { actionKey, typeKey } = private applyCheckedState(): void {
this.permissionsService.getPermissionKeys(permissionStr) const allPerms = new Set(
this._inheritedPermissions.concat(this.permissions)
)
if (actionKey && typeKey) {
this.form
.get(typeKey)
?.get(actionKey)
?.patchValue(true, { emitEvent: false })
}
})
this.allowedTypes.forEach((type) => { this.allowedTypes.forEach((type) => {
const typeGroup = this.form.get(type)
for (const action of Object.keys(PermissionAction)) {
typeGroup.get(action)?.patchValue(
allPerms.has(
this.permissionsService.getPermissionCode(
PermissionAction[action],
PermissionType[type]
)
),
{ emitEvent: false } // don't trigger valueChanges now
)
}
if (this.typeHasAllActionsSelected(type)) { if (this.typeHasAllActionsSelected(type)) {
this.typesWithAllActions.add(type) this.typesWithAllActions.add(type)
} else { } else {
@@ -150,26 +169,9 @@ export class PermissionsSelectComponent
ngOnInit(): void { ngOnInit(): void {
this.form.valueChanges.subscribe((newValue) => { this.form.valueChanges.subscribe((newValue) => {
let permissions = [] const permissions = this.getSelectedPermissions(newValue)
Object.entries(newValue).forEach(([typeKey, typeValue]) => {
const selectedActions = Object.entries(typeValue).filter(
([actionKey, actionValue]) =>
actionValue &&
this.isActionSupported(
PermissionType[typeKey],
PermissionAction[actionKey]
)
)
selectedActions.forEach(([actionKey]) => {
permissions.push(
(PermissionType[typeKey] as string).replace(
'%s',
PermissionAction[actionKey]
)
)
})
Object.keys(newValue).forEach((typeKey) => {
if (this.typeHasAllActionsSelected(typeKey)) { if (this.typeHasAllActionsSelected(typeKey)) {
this.typesWithAllActions.add(typeKey) this.typesWithAllActions.add(typeKey)
} else { } else {
@@ -269,6 +271,30 @@ export class PermissionsSelectComponent
return true return true
} }
private getSelectedPermissions(formValue: object): string[] {
const permissions = []
Object.entries(formValue).forEach(([typeKey, typeValue]) => {
Object.entries(typeValue)
.filter(
([actionKey, actionValue]) =>
actionValue &&
this.isActionSupported(
PermissionType[typeKey],
PermissionAction[actionKey]
)
)
.forEach(([actionKey]) => {
permissions.push(
this.permissionsService.getPermissionCode(
PermissionAction[actionKey],
PermissionType[typeKey]
)
)
})
})
return permissions
}
private typeHasAllActionsSelected(typeKey: string): boolean { private typeHasAllActionsSelected(typeKey: string): boolean {
return Object.keys(PermissionAction) return Object.keys(PermissionAction)
.filter((action) => .filter((action) =>
@@ -1,5 +1,5 @@
<div class="btn-group"> <div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())"> <button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())" [aria-label]="noSuggestions ? 'No suggestions' : 'Suggest'" i18n-aria-label>
@if (loading()) { @if (loading()) {
<div class="spinner-border spinner-border-sm" role="status"></div> <div class="spinner-border spinner-border-sm" role="status"></div>
} @else if (noSuggestions) { } @else if (noSuggestions) {
@@ -23,7 +23,7 @@
<dd> <dd>
{{status().pngx_version}} {{status().pngx_version}}
@if (versionMismatch()) { @if (versionMismatch()) {
<button class="btn btn-sm d-inline align-items-center btn-dark text-uppercase small" [ngbPopover]="versionPopover" triggers="click mouseenter:mouseleave"> <button class="btn btn-sm d-inline align-items-center btn-dark text-uppercase small" [ngbPopover]="versionPopover" triggers="click mouseenter:mouseleave" aria-label="View version mismatch details" i18n-aria-label>
<i-bs name="exclamation-triangle-fill" class="text-danger lh-1"></i-bs> <i-bs name="exclamation-triangle-fill" class="text-danger lh-1"></i-bs>
</button> </button>
} }
@@ -1,9 +1,7 @@
<pngx-page-header title="Dashboard" [subTitle]="subtitle" i18n-title tourAnchor="tour.dashboard"> <pngx-page-header title="Dashboard" [subTitle]="subtitle" i18n-title tourAnchor="tour.dashboard"></pngx-page-header>
<pngx-logo extra_classes="d-none d-md-block mt-n2" height="3rem"></pngx-logo>
</pngx-page-header>
<div class="row"> <div class="row dashboard-grid g-4 pb-0">
<div class="col-12 col-lg-8 col-xl-9 mb-4"> <div class="col-12 col-lg-8 col-xl-9 mb-4 dashboard-main">
<div class="row row-cols-1 g-4" <div class="row row-cols-1 g-4"
cdkDropList cdkDropList
[cdkDropListDisabled]="settingsService.globalDropzoneActive()" [cdkDropListDisabled]="settingsService.globalDropzoneActive()"
@@ -58,7 +56,7 @@
</ng-container> </ng-container>
</div> </div>
</div> </div>
<div class="col-12 col-lg-4 col-xl-3 col-sidebar"> <div class="col-12 col-lg-4 col-xl-3 col-sidebar dashboard-aside">
<div class="row row-cols-1 g-4 mb-4 sticky-lg-top z-0"> <div class="row row-cols-1 g-4 mb-4 sticky-lg-top z-0">
<pngx-upload-file-widget></pngx-upload-file-widget> <pngx-upload-file-widget></pngx-upload-file-widget>
<pngx-statistics-widget *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.UISettings }"></pngx-statistics-widget> <pngx-statistics-widget *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.UISettings }"></pngx-statistics-widget>
@@ -1,3 +1,13 @@
.col-sidebar .row { .col-sidebar .row {
top: 3.5rem; top: 4.75rem;
}
.dashboard-grid {
padding-bottom: 2rem;
}
@media (min-width: 1200px) {
.dashboard-main {
padding-right: 1rem;
}
} }
@@ -15,7 +15,6 @@ import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service' import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service' import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment' import { environment } from 'src/environments/environment'
import { LogoComponent } from '../common/logo/logo.component'
import { PageHeaderComponent } from '../common/page-header/page-header.component' import { PageHeaderComponent } from '../common/page-header/page-header.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component' import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
import { SavedViewWidgetComponent } from './widgets/saved-view-widget/saved-view-widget.component' import { SavedViewWidgetComponent } from './widgets/saved-view-widget/saved-view-widget.component'
@@ -28,7 +27,6 @@ import { WelcomeWidgetComponent } from './widgets/welcome-widget/welcome-widget.
templateUrl: './dashboard.component.html', templateUrl: './dashboard.component.html',
styleUrls: ['./dashboard.component.scss'], styleUrls: ['./dashboard.component.scss'],
imports: [ imports: [
LogoComponent,
PageHeaderComponent, PageHeaderComponent,
SavedViewWidgetComponent, SavedViewWidgetComponent,
StatisticsWidgetComponent, StatisticsWidgetComponent,
@@ -8,3 +8,14 @@
width: 0.6rem; width: 0.6rem;
} }
} }
.list-group {
--bs-list-group-border-color: color-mix(in srgb, var(--bs-border-color) 58%, transparent);
--bs-list-group-bg: transparent;
border-radius: .6rem;
overflow: hidden;
}
.list-group-item {
padding: .7rem .8rem;
}
@@ -4,6 +4,16 @@
.btn-outline-dark { .btn-outline-dark {
--bs-btn-border-color: var(--bs-border-color-translucent); --bs-btn-border-color: var(--bs-border-color-translucent);
border-style: dashed;
border-width: 1px;
min-height: 5.25rem;
background: color-mix(in srgb, var(--bs-primary) 4%, var(--bs-light)) !important;
&:hover,
&:focus {
border-color: var(--bs-primary);
background: color-mix(in srgb, var(--bs-primary) 9%, var(--bs-light)) !important;
}
} }
.smaller { .smaller {
@@ -1,5 +1,5 @@
@if (!cardless()) { @if (!cardless()) {
<div class="card shadow-sm bg-light fade" [class.show]="show()" cdkDrag [cdkDragDisabled]="!draggable()" cdkDragPreviewContainer="parent"> <div class="card bg-light fade" [class.show]="show()" cdkDrag [cdkDragDisabled]="!draggable()" cdkDragPreviewContainer="parent">
<div class="card-header"> <div class="card-header">
<div class="d-flex justify-content-between align-items-center"> <div class="d-flex justify-content-between align-items-center">
<div class="d-flex align-items-center"> <div class="d-flex align-items-center">
@@ -2,6 +2,20 @@ i-bs {
cursor: move; cursor: move;
} }
.card {
overflow: hidden;
border-color: color-mix(in srgb, var(--bs-border-color) 68%, transparent);
.card-header {
padding: .9rem 1rem;
border-bottom-color: color-mix(in srgb, var(--bs-border-color) 55%, transparent);
}
.card-body {
padding: 1rem;
}
}
.fade.show { .fade.show {
animation: pngx-entry-fade 160ms ease-out; animation: pngx-entry-fade 160ms ease-out;
} }
@@ -20,7 +20,7 @@
</div> </div>
} }
<button type="button" class="btn btn-sm btn-outline-danger me-md-4" (click)="delete()" [disabled]="!userIsOwner" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }"> <button type="button" class="btn btn-sm btn-outline-danger me-md-4" (click)="delete()" [disabled]="!userIsOwner" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }" aria-label="Delete" i18n-aria-label>
<i-bs width="1.2em" height="1.2em" name="trash"></i-bs><span class="d-none d-lg-inline ps-1" i18n>Delete</span> <i-bs width="1.2em" height="1.2em" name="trash"></i-bs><span class="d-none d-lg-inline ps-1" i18n>Delete</span>
</button> </button>
@@ -35,7 +35,7 @@
/> />
<div class="btn-group"> <div class="btn-group">
<button (click)="download()" class="btn btn-sm btn-outline-primary" [disabled]="downloading()"> <button (click)="download()" class="btn btn-sm btn-outline-primary" [disabled]="downloading()" aria-label="Download" i18n-aria-label>
@if (downloading()) { @if (downloading()) {
<div class="spinner-border spinner-border-sm" role="status"></div> <div class="spinner-border spinner-border-sm" role="status"></div>
} @else { } @else {
@@ -45,7 +45,7 @@
</button> </button>
<div class="btn-group" ngbDropdown role="group"> <div class="btn-group" ngbDropdown role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle" [disabled]="downloading()" ngbDropdownToggle></button> <button class="btn btn-sm btn-outline-primary dropdown-toggle" [disabled]="downloading()" ngbDropdownToggle aria-label="Download options" i18n-aria-label></button>
<div class="dropdown-menu shadow" ngbDropdownMenu> <div class="dropdown-menu shadow" ngbDropdownMenu>
@if (metadata()?.has_archive_version) { @if (metadata()?.has_archive_version) {
<button ngbDropdownItem (click)="download(true)" [disabled]="downloading()" i18n>Download original</button> <button ngbDropdownItem (click)="download(true)" [disabled]="downloading()" i18n>Download original</button>
@@ -62,7 +62,7 @@
</div> </div>
<div class="ms-auto" ngbDropdown> <div class="ms-auto" ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="actionsDropdown" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary" id="actionsDropdown" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div> <i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow"> <div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow">
@@ -91,7 +91,7 @@
</div> </div>
<div class="ms-auto" ngbDropdown> <div class="ms-auto" ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="sendDropdown" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary" id="sendDropdown" ngbDropdownToggle aria-label="Send" i18n-aria-label>
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container></div> <i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container></div>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow"> <div ngbDropdownMenu aria-labelledby="actionsDropdown" class="shadow">
@@ -467,13 +467,6 @@ describe('DocumentDetailComponent', () => {
const docWithVersions = { const docWithVersions = {
...doc, ...doc,
versions: [ versions: [
{
id: doc.id,
added: new Date('2024-01-01T00:00:00Z'),
version_label: 'Original',
checksum: 'aaaa',
is_root: true,
},
{ {
id: 10, id: 10,
added: new Date('2024-01-02T00:00:00Z'), added: new Date('2024-01-02T00:00:00Z'),
@@ -481,6 +474,13 @@ describe('DocumentDetailComponent', () => {
checksum: 'bbbb', checksum: 'bbbb',
is_root: false, is_root: false,
}, },
{
id: doc.id,
added: new Date('2024-01-01T00:00:00Z'),
version_label: 'Original',
checksum: 'aaaa',
is_root: true,
},
], ],
} as Document } as Document
@@ -1232,8 +1232,8 @@ describe('DocumentDetailComponent', () => {
metadataSpy.mockClear() metadataSpy.mockClear()
component.document().versions = [ component.document().versions = [
{ id: doc.id, is_root: true },
{ id: 10, is_root: false }, { id: 10, is_root: false },
{ id: doc.id, is_root: true },
] as any ] as any
jest.spyOn(documentService, 'getPreviewUrl').mockReturnValue('preview-root') jest.spyOn(documentService, 'getPreviewUrl').mockReturnValue('preview-root')
jest.spyOn(documentService, 'getThumbUrl').mockReturnValue('thumb-root') jest.spyOn(documentService, 'getThumbUrl').mockReturnValue('thumb-root')
@@ -1929,8 +1929,8 @@ describe('DocumentDetailComponent', () => {
component.documentId.set(doc.id) component.documentId.set(doc.id)
component.document.set({ ...doc, versions: [] } as Document) component.document.set({ ...doc, versions: [] } as Document)
const updatedVersions = [ const updatedVersions = [
{ id: doc.id, is_root: true },
{ id: 10, is_root: false }, { id: 10, is_root: false },
{ id: doc.id, is_root: true },
] as any ] as any
const openDoc = { ...doc, versions: [] } as Document const openDoc = { ...doc, versions: [] } as Document
jest.spyOn(openDocumentsService, 'getOpenDocument').mockReturnValue(openDoc) jest.spyOn(openDocumentsService, 'getOpenDocument').mockReturnValue(openDoc)
@@ -2046,8 +2046,8 @@ describe('DocumentDetailComponent', () => {
it('should include version in download and print only for non-latest selected version', () => { it('should include version in download and print only for non-latest selected version', () => {
initNormally() initNormally()
component.document().versions = [ component.document().versions = [
{ id: doc.id, is_root: true },
{ id: 10, is_root: false }, { id: 10, is_root: false },
{ id: doc.id, is_root: true },
] as any ] as any
const getDownloadUrlSpy = jest const getDownloadUrlSpy = jest
@@ -2171,6 +2171,11 @@ describe('DocumentDetailComponent', () => {
).toBe(10) ).toBe(10)
component.openEmailDocument() component.openEmailDocument()
expect(modalSpy).toHaveBeenCalled() expect(modalSpy).toHaveBeenCalled()
expect(
(
modalSpy.mock.results[1].value as NgbModalRef
).componentInstance.documentIds()
).toEqual([10])
}) })
it('should set previewText', () => { it('should set previewText', () => {
@@ -889,13 +889,9 @@ export class DocumentDetailComponent
updateComponent(doc: Document) { updateComponent(doc: Document) {
this.document.set(doc) this.document.set(doc)
// Default selected version is the newest version // Default selected version is the newest version, which the API returns first
const versions = doc.versions ?? [] const versions = doc.versions ?? []
this.selectedVersionId.set( this.selectedVersionId.set(versions.length ? versions[0].id : doc.id)
versions.length
? Math.max(...versions.map((version) => version.id))
: doc.id
)
this.previewLoaded.set(false) this.previewLoaded.set(false)
this.requiresPassword = false this.requiresPassword = false
this.updateFormForCustomFields() this.updateFormForCustomFields()
@@ -1441,7 +1437,8 @@ export class DocumentDetailComponent
if (!versions.length || !this.selectedVersionId()) { if (!versions.length || !this.selectedVersionId()) {
return null return null
} }
const latestVersionId = Math.max(...versions.map((version) => version.id)) // The API returns versions newest first
const latestVersionId = versions[0].id
return this.selectedVersionId() === latestVersionId return this.selectedVersionId() === latestVersionId
? null ? null
: this.selectedVersionId() : this.selectedVersionId()
@@ -1976,7 +1973,9 @@ export class DocumentDetailComponent
const modal = this.modalService.open(EmailDocumentDialogComponent, { const modal = this.modalService.open(EmailDocumentDialogComponent, {
backdrop: 'static', backdrop: 'static',
}) })
modal.componentInstance.documentIds.set([this.document().id]) modal.componentInstance.documentIds.set([
this.selectedVersionId() ?? this.document().id,
])
modal.componentInstance.hasArchiveVersion.set( modal.componentInstance.hasArchiveVersion.set(
this.metadata()?.has_archive_version ?? this.metadata()?.has_archive_version ??
!!this.document()?.archived_file_name !!this.document()?.archived_file_name
@@ -0,0 +1,18 @@
<div class="modal-header">
<h4 class="modal-title" i18n>Add existing document as version</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()"></button>
</div>
<div class="modal-body">
<pngx-input-document-link
[(ngModel)]="selectedDocumentIDs"
[parentDocumentID]="rootDocumentID"
[minimal]="true"
placeholder="Search for a document"
i18n-placeholder
></pngx-input-document-link>
<div class="form-text mt-2" i18n>Select one document to add as a version.</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-outline-secondary" (click)="cancel()" [disabled]="!buttonsEnabled()" i18n>Cancel</button>
<button type="button" class="btn btn-primary" (click)="confirm()" [disabled]="!buttonsEnabled() || selectedDocumentIDs.length !== 1" i18n>Add version</button>
</div>
@@ -0,0 +1,73 @@
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { DocumentService } from 'src/app/services/rest/document.service'
import { AddExistingDocumentVersionDialogComponent } from './add-existing-document-version-dialog.component'
describe('AddExistingDocumentVersionDialogComponent', () => {
let component: AddExistingDocumentVersionDialogComponent
let fixture: ComponentFixture<AddExistingDocumentVersionDialogComponent>
let activeModal: jest.Mocked<Pick<NgbActiveModal, 'dismiss'>>
beforeEach(async () => {
activeModal = { dismiss: jest.fn() }
await TestBed.configureTestingModule({
imports: [AddExistingDocumentVersionDialogComponent],
providers: [
{
provide: NgbActiveModal,
useValue: activeModal,
},
{
provide: DocumentService,
useValue: {},
},
],
}).compileComponents()
fixture = TestBed.createComponent(AddExistingDocumentVersionDialogComponent)
component = fixture.componentInstance
component.rootDocumentID = 3
fixture.detectChanges()
})
it('should emit the single selected document', () => {
const emitSpy = jest.spyOn(component.confirmClicked, 'emit')
component.selectedDocumentIDs = [20]
component.confirm()
expect(emitSpy).toHaveBeenCalledWith(20)
})
it('should require exactly one selected document', () => {
const emitSpy = jest.spyOn(component.confirmClicked, 'emit')
component.selectedDocumentIDs = [20, 21]
component.confirm()
expect(emitSpy).not.toHaveBeenCalled()
})
it('should dismiss on cancel', () => {
component.cancel()
expect(activeModal.dismiss).toHaveBeenCalled()
})
it('should re-render the buttons when they are toggled from outside', async () => {
const cancelButton: HTMLButtonElement = fixture.nativeElement.querySelector(
'.modal-footer button'
)
expect(cancelButton.disabled).toBeFalsy()
// No detectChanges: the dropdown toggling this from a request callback is
// all that happens, and nothing else schedules a render for the modal
component.buttonsEnabled.set(false)
await fixture.whenStable()
expect(cancelButton.disabled).toBeTruthy()
component.buttonsEnabled.set(true)
await fixture.whenStable()
expect(cancelButton.disabled).toBeFalsy()
})
})
@@ -0,0 +1,35 @@
import {
Component,
EventEmitter,
Input,
Output,
inject,
signal,
} from '@angular/core'
import { FormsModule } from '@angular/forms'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { DocumentLinkComponent } from 'src/app/components/common/input/document-link/document-link.component'
@Component({
selector: 'pngx-add-existing-document-version-dialog',
templateUrl: './add-existing-document-version-dialog.component.html',
imports: [DocumentLinkComponent, FormsModule],
})
export class AddExistingDocumentVersionDialogComponent {
private readonly activeModal = inject(NgbActiveModal)
@Input() rootDocumentID: number
@Output() confirmClicked = new EventEmitter<number>()
selectedDocumentIDs: number[] = []
readonly buttonsEnabled = signal(true)
confirm(): void {
if (this.selectedDocumentIDs.length !== 1) return
this.confirmClicked.emit(this.selectedDocumentIDs[0])
}
cancel(): void {
this.activeModal.dismiss()
}
}
@@ -1,7 +1,10 @@
<div class="btn-group" ngbDropdown autoClose="outside"> <div class="btn-group" ngbDropdown autoClose="outside">
<button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle> <button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle aria-label="Versions" i18n-aria-label>
<i-bs name="file-earmark-diff"></i-bs> <i-bs name="file-earmark-diff"></i-bs>
<span class="d-none d-lg-inline ps-1" i18n>Versions</span> <span class="d-none d-lg-inline ps-1" i18n>Versions</span>
@if (versions.length > 1) {
<span class="badge text-bg-secondary ms-1">{{ versions.length }}</span>
}
</button> </button>
<div class="dropdown-menu shadow" ngbDropdownMenu> <div class="dropdown-menu shadow" ngbDropdownMenu>
<div class="px-3 py-2 mb-2"> <div class="px-3 py-2 mb-2">
@@ -24,13 +27,26 @@
class="visually-hidden" class="visually-hidden"
(change)="onVersionFileSelected($event)" (change)="onVersionFileSelected($event)"
/> />
<button <div class="btn-group btn-group-sm w-100">
class="btn btn-sm btn-outline-secondary w-100" <button
(click)="versionFileInput.click()" class="btn btn-sm btn-outline-secondary w-100"
[disabled]="!userIsOwner || !userCanEdit" (click)="versionFileInput.click()"
> [disabled]="!userIsOwner || !userCanEdit"
<i-bs name="file-earmark-plus"></i-bs><span class="ps-1" i18n>Add new version</span> title="Upload a new version"
</button> i18n-title
>
<i-bs name="file-earmark-plus"></i-bs><span class="ps-1" i18n>Upload</span>
</button>
<button
class="btn btn-sm btn-outline-secondary w-100"
(click)="addExistingDocumentAsVersion()"
[disabled]="!userIsOwner || !userCanEdit"
title="Use an existing document"
i18n-title
>
<i-bs name="file-earmark"></i-bs><span class="ps-1" i18n>Existing</span>
</button>
</div>
} @else { } @else {
@switch (versionUploadState()) { @switch (versionUploadState()) {
@case (UploadState.Uploading) { @case (UploadState.Uploading) {
@@ -128,6 +144,7 @@
i18n-confirmMessage i18n-confirmMessage
[disabled]="!userIsOwner || !userCanEdit" [disabled]="!userIsOwner || !userCanEdit"
(confirm)="deleteVersion(version.id)" (confirm)="deleteVersion(version.id)"
*pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }"
> >
<span class="visually-hidden" i18n>Delete version</span> <span class="visually-hidden" i18n>Delete version</span>
</pngx-confirm-button> </pngx-confirm-button>
@@ -1,9 +1,16 @@
import { DatePipe } from '@angular/common' import { DatePipe } from '@angular/common'
import { SimpleChange } from '@angular/core' import { SimpleChange, signal } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing' import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { Subject, of, throwError } from 'rxjs' import { Subject, of, throwError } from 'rxjs'
import { DocumentVersionInfo } from 'src/app/data/document' import { DocumentVersionInfo } from 'src/app/data/document'
import {
PermissionAction,
PermissionsService,
PermissionType,
} from 'src/app/services/permissions.service'
import { DocumentService } from 'src/app/services/rest/document.service' import { DocumentService } from 'src/app/services/rest/document.service'
import { SettingsService } from 'src/app/services/settings.service' import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service' import { ToastService } from 'src/app/services/toast.service'
@@ -19,12 +26,20 @@ describe('DocumentVersionDropdownComponent', () => {
let documentService: jest.Mocked< let documentService: jest.Mocked<
Pick< Pick<
DocumentService, DocumentService,
'deleteVersion' | 'getVersions' | 'uploadVersion' | 'updateVersionLabel' | 'deleteVersion'
| 'getVersions'
| 'mergeDocumentsAsVersions'
| 'uploadVersion'
| 'updateVersionLabel'
> >
> >
let toastService: jest.Mocked<Pick<ToastService, 'showError' | 'showInfo'>> let toastService: jest.Mocked<Pick<ToastService, 'showError' | 'showInfo'>>
let finished$: Subject<{ taskId: string }> let finished$: Subject<{ taskId: string }>
let failed$: Subject<{ taskId: string; message?: string }> let failed$: Subject<{ taskId: string; message?: string }>
let modalService: jest.Mocked<Pick<NgbModal, 'open'>>
let permissionsService: jest.Mocked<
Pick<PermissionsService, 'currentUserCan'>
>
beforeEach(async () => { beforeEach(async () => {
finished$ = new Subject<{ taskId: string }>() finished$ = new Subject<{ taskId: string }>()
@@ -32,13 +47,18 @@ describe('DocumentVersionDropdownComponent', () => {
documentService = { documentService = {
deleteVersion: jest.fn(), deleteVersion: jest.fn(),
getVersions: jest.fn(), getVersions: jest.fn(),
mergeDocumentsAsVersions: jest.fn(),
uploadVersion: jest.fn(), uploadVersion: jest.fn(),
updateVersionLabel: jest.fn(), updateVersionLabel: jest.fn(),
} }
modalService = { open: jest.fn() }
toastService = { toastService = {
showError: jest.fn(), showError: jest.fn(),
showInfo: jest.fn(), showInfo: jest.fn(),
} }
permissionsService = {
currentUserCan: jest.fn().mockReturnValue(true),
}
await TestBed.configureTestingModule({ await TestBed.configureTestingModule({
imports: [ imports: [
@@ -61,6 +81,14 @@ describe('DocumentVersionDropdownComponent', () => {
provide: ToastService, provide: ToastService,
useValue: toastService, useValue: toastService,
}, },
{
provide: NgbModal,
useValue: modalService,
},
{
provide: PermissionsService,
useValue: permissionsService,
},
{ {
provide: WebsocketStatusService, provide: WebsocketStatusService,
useValue: { useValue: {
@@ -131,6 +159,31 @@ describe('DocumentVersionDropdownComponent', () => {
) )
}) })
it('should not show version delete buttons without document delete permission', () => {
fixture.destroy()
permissionsService.currentUserCan.mockReturnValue(false)
fixture = TestBed.createComponent(DocumentVersionDropdownComponent)
component = fixture.componentInstance
component.documentId = 3
component.selectedVersionId = 3
component.userIsOwner = true
component.userCanEdit = true
component.versions = [
{ id: 3, is_root: true, checksum: 'aaaa' },
{ id: 10, is_root: false, checksum: 'bbbb' },
]
fixture.detectChanges()
expect(permissionsService.currentUserCan).toHaveBeenCalledWith(
PermissionAction.Delete,
PermissionType.Document
)
expect(
fixture.debugElement.queryAll(By.css('pngx-confirm-button'))
).toHaveLength(0)
})
it('beginEditingVersion should set active row and draft label', () => { it('beginEditingVersion should set active row and draft label', () => {
component.userCanEdit = true component.userCanEdit = true
component.userIsOwner = true component.userIsOwner = true
@@ -222,9 +275,10 @@ describe('DocumentVersionDropdownComponent', () => {
}) })
it('onVersionFileSelected should upload and update versions after websocket success', () => { it('onVersionFileSelected should upload and update versions after websocket success', () => {
// Newest first, as the API returns them
const versions: DocumentVersionInfo[] = [ const versions: DocumentVersionInfo[] = [
{ id: 3, is_root: true, checksum: 'aaaa' },
{ id: 20, is_root: false, checksum: 'cccc' }, { id: 20, is_root: false, checksum: 'cccc' },
{ id: 3, is_root: true, checksum: 'aaaa' },
] ]
const file = new File(['test'], 'new-version.pdf', { const file = new File(['test'], 'new-version.pdf', {
type: 'application/pdf', type: 'application/pdf',
@@ -323,4 +377,45 @@ describe('DocumentVersionDropdownComponent', () => {
expect(component.editingVersionId).toBeNull() expect(component.editingVersionId).toBeNull()
expect(component.versionLabelDraft).toEqual('') expect(component.versionLabelDraft).toEqual('')
}) })
it('addExistingDocumentAsVersion should merge with a label and refresh versions', () => {
const confirmClicked = new Subject<number>()
const modal = {
componentInstance: {
rootDocumentID: null,
buttonsEnabled: signal(true),
confirmClicked,
},
close: jest.fn(),
}
modalService.open.mockReturnValue(modal as any)
documentService.mergeDocumentsAsVersions.mockReturnValue(of({} as any))
// Newest first, as the API returns them. The merged document has a lower id
// than the root, which is the whole point of merging an existing document.
const versions: DocumentVersionInfo[] = [
{ id: 2, is_root: false, checksum: 'cccc' },
{ id: 3, is_root: true, checksum: 'aaaa' },
]
documentService.getVersions.mockReturnValue(of({ id: 3, versions } as any))
component.newVersionLabel = ' Imported '
const versionsEmitSpy = jest.spyOn(component.versionsUpdated, 'emit')
const selectedEmitSpy = jest.spyOn(component.versionSelected, 'emit')
component.addExistingDocumentAsVersion()
expect(modal.componentInstance.rootDocumentID).toEqual(3)
confirmClicked.next(2)
expect(documentService.mergeDocumentsAsVersions).toHaveBeenCalledWith(
[3, 2],
3,
'Imported'
)
expect(documentService.updateVersionLabel).not.toHaveBeenCalled()
expect(documentService.getVersions).toHaveBeenCalledWith(3)
expect(versionsEmitSpy).toHaveBeenCalledWith(versions)
expect(selectedEmitSpy).toHaveBeenCalledWith(2)
expect(component.newVersionLabel).toEqual('')
expect(modal.close).toHaveBeenCalled()
expect(toastService.showInfo).toHaveBeenCalled()
})
}) })
@@ -11,7 +11,7 @@ import {
SimpleChanges, SimpleChanges,
} from '@angular/core' } from '@angular/core'
import { FormsModule } from '@angular/forms' import { FormsModule } from '@angular/forms'
import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap' import { NgbDropdownModule, NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { merge, of, Subject } from 'rxjs' import { merge, of, Subject } from 'rxjs'
import { import {
@@ -25,6 +25,7 @@ import {
tap, tap,
} from 'rxjs/operators' } from 'rxjs/operators'
import { DocumentVersionInfo } from 'src/app/data/document' import { DocumentVersionInfo } from 'src/app/data/document'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe' import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentService } from 'src/app/services/rest/document.service' import { DocumentService } from 'src/app/services/rest/document.service'
import { ToastService } from 'src/app/services/toast.service' import { ToastService } from 'src/app/services/toast.service'
@@ -33,6 +34,8 @@ import {
WebsocketStatusService, WebsocketStatusService,
} from 'src/app/services/websocket-status.service' } from 'src/app/services/websocket-status.service'
import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-button.component' import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-button.component'
import { ComponentWithPermissions } from '../../with-permissions/with-permissions.component'
import { AddExistingDocumentVersionDialogComponent } from './add-existing-document-version-dialog/add-existing-document-version-dialog.component'
@Component({ @Component({
selector: 'pngx-document-version-dropdown', selector: 'pngx-document-version-dropdown',
@@ -43,11 +46,15 @@ import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-butt
NgbDropdownModule, NgbDropdownModule,
NgxBootstrapIconsModule, NgxBootstrapIconsModule,
ConfirmButtonComponent, ConfirmButtonComponent,
IfPermissionsDirective,
SlicePipe, SlicePipe,
CustomDatePipe, CustomDatePipe,
], ],
}) })
export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy { export class DocumentVersionDropdownComponent
extends ComponentWithPermissions
implements OnChanges, OnDestroy
{
UploadState = UploadState UploadState = UploadState
@Input() documentId: number @Input() documentId: number
@@ -69,6 +76,7 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
private readonly documentsService = inject(DocumentService) private readonly documentsService = inject(DocumentService)
private readonly toastService = inject(ToastService) private readonly toastService = inject(ToastService)
private readonly websocketStatusService = inject(WebsocketStatusService) private readonly websocketStatusService = inject(WebsocketStatusService)
private readonly modalService = inject(NgbModal)
private readonly destroy$ = new Subject<void>() private readonly destroy$ = new Subject<void>()
private readonly documentChange$ = new Subject<void>() private readonly documentChange$ = new Subject<void>()
@@ -256,11 +264,10 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
.subscribe({ .subscribe({
next: (doc) => { next: (doc) => {
if (uploadDocumentId !== this.documentId) return if (uploadDocumentId !== this.documentId) return
if (doc?.versions) { if (doc?.versions?.length) {
this.versionsUpdated.emit(doc.versions) this.versionsUpdated.emit(doc.versions)
this.versionSelected.emit( // The API returns versions newest first
Math.max(...doc.versions.map((version) => version.id)) this.versionSelected.emit(doc.versions[0].id)
)
this.clearVersionUploadStatus() this.clearVersionUploadStatus()
} }
}, },
@@ -278,6 +285,55 @@ export class DocumentVersionDropdownComponent implements OnChanges, OnDestroy {
}) })
} }
addExistingDocumentAsVersion(): void {
const modal = this.modalService.open(
AddExistingDocumentVersionDialogComponent,
{ backdrop: 'static' }
)
const dialog =
modal.componentInstance as AddExistingDocumentVersionDialogComponent
dialog.rootDocumentID = this.documentId
dialog.confirmClicked
.pipe(takeUntil(this.destroy$), takeUntil(this.documentChange$))
.subscribe((existingDocumentID) => {
dialog.buttonsEnabled.set(false)
const versionLabel = this.newVersionLabel?.trim()
this.documentsService
.mergeDocumentsAsVersions(
[this.documentId, existingDocumentID],
this.documentId,
versionLabel
)
.pipe(
switchMap(() => this.documentsService.getVersions(this.documentId)),
first(),
finalize(() => dialog.buttonsEnabled.set(true)),
takeUntil(this.destroy$),
takeUntil(this.documentChange$)
)
.subscribe({
next: (document) => {
if (document?.versions?.length) {
this.versionsUpdated.emit(document.versions)
// The API returns versions newest first
this.versionSelected.emit(document.versions[0].id)
}
this.newVersionLabel = ''
modal.close()
this.toastService.showInfo(
$localize`Existing document added as a version.`
)
},
error: (error) => {
this.toastService.showError(
$localize`Error adding existing document as a version`,
error
)
},
})
})
}
clearVersionUploadStatus(): void { clearVersionUploadStatus(): void {
this.versionUploadState.set(UploadState.Idle) this.versionUploadState.set(UploadState.Idle)
this.versionUploadError.set(null) this.versionUploadError.set(null)
@@ -1,6 +1,6 @@
<h6> <h6>
<button type="button" class="btn btn-outline-secondary btn-sm me-2" <button type="button" class="btn btn-outline-secondary btn-sm me-2"
(click)="expand = !expand"> (click)="expand = !expand" aria-label="Toggle document metadata" i18n-aria-label>
@if (!expand) { @if (!expand) {
<i-bs width="1.2em" height="1.2em" name="caret-down"></i-bs> <i-bs width="1.2em" height="1.2em" name="caret-down"></i-bs>
} }
@@ -74,7 +74,7 @@
</pngx-filterable-dropdown> </pngx-filterable-dropdown>
} }
<div class="btn-group"> <div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll"> <button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll" aria-label="Permissions" i18n-aria-label>
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Permissions</ng-container></div> <i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Permissions</ng-container></div>
</button> </button>
</div> </div>
@@ -82,7 +82,7 @@
<div class="d-flex align-items-center gap-2 ms-auto"> <div class="d-flex align-items-center gap-2 ms-auto">
<div class="btn-toolbar"> <div class="btn-toolbar">
<div ngbDropdown> <div ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div> <i-bs name="three-dots"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Actions</ng-container></div>
</button> </button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow"> <div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
@@ -95,6 +95,9 @@
<button ngbDropdownItem (click)="mergeSelected()" [disabled]="!userCanAdd || list.allSelected || list.selectedCount < 2"> <button ngbDropdownItem (click)="mergeSelected()" [disabled]="!userCanAdd || list.allSelected || list.selectedCount < 2">
<i-bs name="journals" class="me-1"></i-bs><ng-container i18n>Merge</ng-container> <i-bs name="journals" class="me-1"></i-bs><ng-container i18n>Merge</ng-container>
</button> </button>
<button ngbDropdownItem (click)="mergeSelectedAsVersions()" [disabled]="!userOwnsAll || !userCanEditAll || !userCanDelete || list.allSelected || list.selectedCount < 2">
<i-bs name="journal-bookmark-fill" class="me-1"></i-bs><ng-container i18n>Merge as versions</ng-container>
</button>
</div> </div>
</div> </div>
</div> </div>
@@ -104,6 +107,8 @@
id="dropdownSend" id="dropdownSend"
ngbDropdownToggle ngbDropdownToggle
[disabled]="disabled || !canSendSelection" [disabled]="disabled || !canSendSelection"
aria-label="Send"
i18n-aria-label
> >
<i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container> <i-bs name="send"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Send</ng-container>
</div> </div>
@@ -124,7 +129,7 @@
</div> </div>
</div> </div>
<div class="btn-group btn-group-sm"> <div class="btn-group btn-group-sm">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()"> <button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload()" (click)="downloadSelected()" aria-label="Download" i18n-aria-label>
@if (!awaitingDownload()) { @if (!awaitingDownload()) {
<i-bs name="arrow-down"></i-bs> <i-bs name="arrow-down"></i-bs>
} }
@@ -136,7 +141,7 @@
<div class="d-none d-sm-inline ms-1"><ng-container i18n>Download</ng-container></div> <div class="d-none d-sm-inline ms-1"><ng-container i18n>Download</ng-container></div>
</button> </button>
<div ngbDropdown class="me-2 d-flex btn-group" role="group"> <div ngbDropdown class="me-2 d-flex btn-group" role="group">
<button type="button" class="btn btn-sm btn-outline-primary dropdown-toggle-split rounded-end" ngbDropdownToggle></button> <button type="button" class="btn btn-sm btn-outline-primary dropdown-toggle-split rounded-end" ngbDropdownToggle aria-label="Download options" i18n-aria-label></button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow"> <div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
<form [formGroup]="downloadForm" class="px-3 py-1"> <form [formGroup]="downloadForm" class="px-3 py-1">
<p class="mb-1" i18n>Include:</p> <p class="mb-1" i18n>Include:</p>
@@ -1,5 +1,5 @@
.dropdown-toggle-split { .dropdown-toggle-split {
--bs-border-radius: .25rem; --bs-border-radius: .45rem;
} }
.dropdown-menu{ .dropdown-menu{
@@ -1248,6 +1248,89 @@ describe('BulkEditorComponent', () => {
expect(documentListViewService.selected.size).toEqual(0) expect(documentListViewService.selected.size).toEqual(0)
}) })
it('should support merging documents as versions', () => {
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest
.spyOn(documentListViewService, 'documents', 'get')
.mockReturnValue([{ id: 3 }, { id: 4 }])
jest.spyOn(documentService, 'getFew').mockReturnValue(
of({
all: [3, 4],
count: 2,
results: [
{ id: 3, title: 'Document 3' },
{ id: 4, title: 'Document 4' },
],
})
)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([3, 4]))
jest
.spyOn(permissionsService, 'currentUserHasObjectPermissions')
.mockReturnValue(true)
jest
.spyOn(permissionsService, 'currentUserOwnsObject')
.mockReturnValue(true)
const mergeAsVersionsSpy = jest
.spyOn(documentService, 'mergeDocumentsAsVersions')
.mockReturnValue(of(true))
const toastInfoSpy = jest.spyOn(toastService, 'showInfo')
fixture.detectChanges()
component.mergeSelectedAsVersions()
expect(modal).not.toBeUndefined()
modal.componentInstance.rootDocumentID.set(4)
modal.componentInstance.confirm()
expect(mergeAsVersionsSpy).toHaveBeenCalledWith([3, 4], 4)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
expect(documentListViewService.selected.size).toEqual(0)
expect(toastInfoSpy).toHaveBeenCalledWith('Documents merged as versions.')
})
it('should not report success when merging documents as versions fails', () => {
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest
.spyOn(documentListViewService, 'documents', 'get')
.mockReturnValue([{ id: 3 }, { id: 4 }])
jest.spyOn(documentService, 'getFew').mockReturnValue(
of({
all: [3, 4],
count: 2,
results: [
{ id: 3, title: 'Document 3' },
{ id: 4, title: 'Document 4' },
],
})
)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([3, 4]))
jest
.spyOn(documentService, 'mergeDocumentsAsVersions')
.mockReturnValue(throwError(() => new Error('failed')))
const toastInfoSpy = jest.spyOn(toastService, 'showInfo')
const toastErrorSpy = jest.spyOn(toastService, 'showError')
fixture.detectChanges()
component.mergeSelectedAsVersions()
modal.componentInstance.rootDocumentID.set(4)
modal.componentInstance.confirm()
expect(toastErrorSpy).toHaveBeenCalled()
expect(toastInfoSpy).not.toHaveBeenCalled()
})
it('should support bulk download with archive, originals or both and file formatting', () => { it('should support bulk download with archive, originals or both and file formatting', () => {
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true) jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest jest
@@ -50,6 +50,7 @@ import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service' import { ToastService } from 'src/app/services/toast.service'
import { flattenTags } from 'src/app/utils/flatten-tags' import { flattenTags } from 'src/app/utils/flatten-tags'
import { queryParamsFromFilterRules } from 'src/app/utils/query-params' import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dialog/merge-as-versions-confirm-dialog/merge-as-versions-confirm-dialog.component'
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component' import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component' import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component' import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
@@ -171,6 +172,13 @@ export class BulkEditorComponent
) )
} }
get userCanDelete(): boolean {
return this.permissionService.currentUserCan(
PermissionAction.Delete,
PermissionType.Document
)
}
ngOnInit() { ngOnInit() {
if ( if (
this.permissionService.currentUserCan( this.permissionService.currentUserCan(
@@ -287,14 +295,17 @@ export class BulkEditorComponent
private executeDocumentAction( private executeDocumentAction(
modal: NgbModalRef, modal: NgbModalRef,
request: Observable<any>, request: Observable<any>,
options: { deleteOriginals?: boolean } = {} options: { clearSelection?: boolean; successMessage?: string } = {}
) { ) {
if (modal) { if (modal) {
modal.componentInstance.buttonsEnabled.set(false) modal.componentInstance.buttonsEnabled.set(false)
} }
request.pipe(first()).subscribe({ request.pipe(first()).subscribe({
next: () => { next: () => {
this.handleOperationSuccess(modal, options.deleteOriginals ?? false) this.handleOperationSuccess(modal, options.clearSelection ?? false)
if (options.successMessage) {
this.toastService.showInfo(options.successMessage)
}
}, },
error: (error) => this.handleOperationError(modal, error), error: (error) => this.handleOperationError(modal, error),
}) })
@@ -990,7 +1001,7 @@ export class BulkEditorComponent
this.executeDocumentAction( this.executeDocumentAction(
modal, modal,
this.documentService.mergeDocuments(mergeDialog.documentIDs(), args), this.documentService.mergeDocuments(mergeDialog.documentIDs(), args),
{ deleteOriginals: !!args.delete_originals } { clearSelection: !!args.delete_originals }
) )
this.toastService.showInfo( this.toastService.showInfo(
$localize`Merged document will be queued for consumption.` $localize`Merged document will be queued for consumption.`
@@ -998,6 +1009,35 @@ export class BulkEditorComponent
}) })
} }
mergeSelectedAsVersions() {
let modal = this.modalService.open(MergeAsVersionsConfirmDialogComponent, {
backdrop: 'static',
})
const mergeDialog =
modal.componentInstance as MergeAsVersionsConfirmDialogComponent
const documentIDs = Array.from(this.list.selected)
mergeDialog.title = $localize`Merge as versions`
mergeDialog.message = $localize`The selected documents will become versions of the root document.`
mergeDialog.btnCaption = $localize`Proceed`
mergeDialog.documentIDs.set(documentIDs)
mergeDialog.rootDocumentID.set(documentIDs[0])
mergeDialog.confirmClicked
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
this.executeDocumentAction(
modal,
this.documentService.mergeDocumentsAsVersions(
mergeDialog.documentIDs(),
mergeDialog.rootDocumentID()
),
{
clearSelection: true,
successMessage: $localize`Documents merged as versions.`,
}
)
})
}
public setCustomFieldValues(changedCustomFields: ChangedItems) { public setCustomFieldValues(changedCustomFields: ChangedItems) {
const modal = this.modalService.open(CustomFieldsBulkEditDialogComponent, { const modal = this.modalService.open(CustomFieldsBulkEditDialogComponent, {
backdrop: 'static', backdrop: 'static',
@@ -61,7 +61,7 @@
</pngx-input-textarea> </pngx-input-textarea>
} }
} }
<button type="button" class="btn btn-outline-danger mb-3" (click)="removeField(field.id)"> <button type="button" class="btn btn-outline-danger mb-3" (click)="removeField(field.id)" aria-label="Remove custom field" i18n-aria-label>
<i-bs name="x"></i-bs> <i-bs name="x"></i-bs>
</button> </button>
</div> </div>
@@ -154,9 +154,9 @@
<i-bs name="download"></i-bs> <i-bs name="download"></i-bs>
</a> </a>
} @else { } @else {
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button> <span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button> <span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
<button class="btn btn-sm btn-outline-secondary placeholder bg-secondary"></button> <span class="btn btn-sm btn-outline-secondary placeholder bg-secondary" aria-hidden="true"></span>
} }
</div> </div>
</div> </div>
@@ -1,6 +1,6 @@
<pngx-page-header [title]="getTitle()"> <pngx-page-header [title]="getTitle()">
<div ngbDropdown class="btn-group flex-fill d-sm-none"> <div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle aria-label="Select" i18n-aria-label>
<i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div> <i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div>
@if (list.hasSelection) { @if (list.hasSelection) {
<pngx-clearable-badge [selected]="list.hasSelection" [number]="list.selectedCount" (cleared)="list.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span> <pngx-clearable-badge [selected]="list.hasSelection" [number]="list.selectedCount" (cleared)="list.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
@@ -31,7 +31,7 @@
</div> </div>
</div> </div>
<div ngbDropdown class="btn-group flex-fill"> <div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle aria-label="Show" i18n-aria-label>
<i-bs name="card-heading"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Show</ng-container></div> <i-bs name="card-heading"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Show</ng-container></div>
</button> </button>
<div ngbDropdownMenu aria-labelledby="dropdownDisplayFields" class="shadow"> <div ngbDropdownMenu aria-labelledby="dropdownDisplayFields" class="shadow">
@@ -61,7 +61,7 @@
</div> </div>
<div ngbDropdown class="btn-group flex-fill"> <div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle> <button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle aria-label="Sort" i18n-aria-label>
<i-bs name="arrow-down-up"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Sort</ng-container></div> <i-bs name="arrow-down-up"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Sort</ng-container></div>
</button> </button>
<div ngbDropdownMenu aria-labelledby="dropdownBasic1" class="shadow dropdown-menu-right"> <div ngbDropdownMenu aria-labelledby="dropdownBasic1" class="shadow dropdown-menu-right">
@@ -85,8 +85,8 @@
</div> </div>
</div> </div>
<div class="btn-group flex-fill" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" ngbDropdown role="group"> <div class="btn-group flex-fill" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" ngbDropdown #viewsDropdown="ngbDropdown" role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle flex-fill" tourAnchor="tour.documents-views" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary dropdown-toggle flex-fill" tourAnchor="tour.documents-views" ngbDropdownToggle aria-label="Views" i18n-aria-label>
<i-bs name="window-stack"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Views</ng-container></div> <i-bs name="window-stack"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Views</ng-container></div>
@if (savedViewIsModified) { @if (savedViewIsModified) {
<div class="position-absolute top-0 start-100 p-2 translate-middle badge bg-secondary border border-light rounded-circle"> <div class="position-absolute top-0 start-100 p-2 translate-middle badge bg-secondary border border-light rounded-circle">
@@ -95,15 +95,15 @@
} }
</button> </button>
<div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu> <div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu>
@if (!list.activeSavedViewId) { @if (viewsDropdown.isOpen() && !list.activeSavedViewId && savedViewService.allViews.length > 0) {
@for (view of savedViewService.allViews; track view) { <div class="views-list overflow-y-auto">
<button ngbDropdownItem (click)="loadViewConfig(view.id)"> @for (view of savedViewService.allViews; track view.id) {
<i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}} <button ngbDropdownItem (click)="loadViewConfig(view.id)">
</button> <i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
} </button>
@if (savedViewService.allViews.length > 0) { }
<div class="dropdown-divider"></div> </div>
} <div class="dropdown-divider"></div>
} }
@if (list.activeSavedViewId && activeSavedViewCanChange) { @if (list.activeSavedViewId && activeSavedViewCanChange) {
@@ -157,7 +157,7 @@
</div> </div>
</ng-template> </ng-template>
<div tourAnchor="tour.documents"> <div class="mt-3 mb-n2" tourAnchor="tour.documents">
<ng-container *ngTemplateOutlet="pagination"></ng-container> <ng-container *ngTemplateOutlet="pagination"></ng-container>
</div> </div>
@@ -56,11 +56,11 @@ $paperless-card-breakpoints: (
.sticky-top { .sticky-top {
z-index: 990; // below main navbar z-index: 990; // below main navbar
top: calc(7rem - 2px); // height of navbar + search row (mobile) top: calc(7.5rem - 2px); // height of navbar + search row (mobile)
transition: top 0.2s ease; transition: top 0.2s ease;
@media (min-width: 580px) { @media (min-width: 580px) {
top: 3.5rem; // height of navbar top: 4.5em; // height of navbar
} }
} }
@@ -73,7 +73,7 @@ $paperless-card-breakpoints: (
@media (max-width: 579.98px) { @media (max-width: 579.98px) {
:host-context(main.mobile-search-hidden) .sticky-top { :host-context(main.mobile-search-hidden) .sticky-top {
top: calc(3.5rem - 2px); // height of navbar only when search is hidden top: calc(4rem - 2px); // height of navbar only when search is hidden
} }
} }
@@ -93,4 +93,8 @@ a {
pngx-page-header .dropdown-menu { pngx-page-header .dropdown-menu {
--bs-dropdown-min-width: 12em; --bs-dropdown-min-width: 12em;
.views-list {
max-height: min(400px, calc(100vh - 260px)); // leave room for the header above and the actions below
}
} }
@@ -18,7 +18,7 @@
</select> </select>
} }
@if (_textFilter) { @if (_textFilter) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()"> <button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs> <i-bs width="1em" height="1em" name="x"></i-bs>
</button> </button>
} }
@@ -24,7 +24,7 @@
<div class="btn-toolbar gap-2"> <div class="btn-toolbar gap-2">
<div class="btn-group d-block d-sm-none"> <div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block"> <div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle> <button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs> <i-bs name="three-dots-vertical"></i-bs>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile"> <div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -7,7 +7,7 @@
> >
@if (activeManagementList) { @if (activeManagementList) {
<div ngbDropdown class="btn-group flex-fill d-sm-none"> <div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle> <button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle aria-label="Select" i18n-aria-label>
<i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div> <i-bs name="text-indent-left"></i-bs><div class="d-none d-sm-inline ms-1"><ng-container i18n>Select</ng-container></div>
@if (activeManagementList.hasSelection) { @if (activeManagementList.hasSelection) {
<pngx-clearable-badge [selected]="activeManagementList.hasSelection" [number]="activeManagementList.selectedCount" (cleared)="activeManagementList.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span> <pngx-clearable-badge [selected]="activeManagementList.hasSelection" [number]="activeManagementList.selectedCount" (cleared)="activeManagementList.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
@@ -1,7 +1,7 @@
<div class="row mb-3"> <div class="row mb-3">
<div class="col mb-2 mb-xl-0"> <div class="col mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center"> <div class="form-inline d-flex align-items-center">
<label class="text-muted me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label> <label class="me-2 mb-0" for="managementNameFilter" i18n>Filter by:</label>
<input id="managementNameFilter" class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder> <input id="managementNameFilter" class="form-control form-control-sm flex-fill w-auto" type="text" autofocus [(ngModel)]="nameFilter" (keyup)="onNameFilterKeyUp($event)" placeholder="Name" i18n-placeholder>
</div> </div>
</div> </div>
@@ -9,7 +9,7 @@
<div class="col-auto mb-2 mb-xl-0"> <div class="col-auto mb-2 mb-xl-0">
<div class="form-inline d-flex align-items-center"> <div class="form-inline d-flex align-items-center">
<div class="input-group input-group-sm w-auto d-none d-md-flex"> <div class="input-group input-group-sm w-auto d-none d-md-flex">
<label class="input-group-text border-0" for="managementPageSize" i18n>Show:</label> <label class="input-group-text bg-transparent border-0" for="managementPageSize" i18n>Show:</label>
</div> </div>
<div class="input-group input-group-sm w-auto me-3"> <div class="input-group input-group-sm w-auto me-3">
<select id="managementPageSize" class="form-select form-select-sm small" [(ngModel)]="pageSize"> <select id="managementPageSize" class="form-select form-select-sm small" [(ngModel)]="pageSize">
@@ -113,7 +113,7 @@
<div class="btn-toolbar gap-2"> <div class="btn-toolbar gap-2">
<div class="btn-group d-block d-sm-none"> <div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block"> <div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle> <button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs> <i-bs name="three-dots-vertical"></i-bs>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile"> <div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -58,7 +58,7 @@
<div class="col"> <div class="col">
<div class="btn-group d-block d-sm-none"> <div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block"> <div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle> <button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs> <i-bs name="three-dots-vertical"></i-bs>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile"> <div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -146,7 +146,7 @@
<div class="col-3"> <div class="col-3">
<div class="btn-group d-block d-sm-none"> <div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block"> <div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle> <button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs> <i-bs name="three-dots-vertical"></i-bs>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile"> <div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -47,7 +47,7 @@
<div class="btn-group d-block d-sm-none"> <div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block"> <div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle> <button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle aria-label="Actions" i18n-aria-label>
<i-bs name="three-dots-vertical"></i-bs> <i-bs name="three-dots-vertical"></i-bs>
</button> </button>
<div ngbDropdownMenu aria-labelledby="actionsMenuMobile"> <div ngbDropdownMenu aria-labelledby="actionsMenuMobile">
@@ -316,6 +316,34 @@ describe(`DocumentService`, () => {
}) })
}) })
it('should call appropriate api endpoint for merging documents as versions', () => {
const ids = [1, 2, 3]
subscription = service.mergeDocumentsAsVersions(ids, 2).subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/merge_as_versions/`
)
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
documents: ids,
root_document_id: 2,
})
})
it('should include an optional label when merging one document as a version', () => {
const ids = [1, 2]
subscription = service
.mergeDocumentsAsVersions(ids, 2, 'Imported')
.subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/merge_as_versions/`
)
expect(req.request.body).toEqual({
documents: ids,
root_document_id: 2,
version_label: 'Imported',
})
})
it('should call appropriate api endpoint for edit pdf', () => { it('should call appropriate api endpoint for edit pdf', () => {
const ids = [1] const ids = [1]
const args = { operations: [{ page: 1, rotate: 90, doc: 0 }] } const args = { operations: [{ page: 1, rotate: 90, doc: 0 }] }
@@ -374,6 +374,18 @@ export class DocumentService extends AbstractPaperlessService<Document> {
}) })
} }
mergeDocumentsAsVersions(
ids: number[],
rootDocumentId: number,
versionLabel?: string
) {
return this.http.post(this.getResourceUrl(null, 'merge_as_versions'), {
documents: ids,
root_document_id: rootDocumentId,
...(versionLabel ? { version_label: versionLabel } : {}),
})
}
editPdfDocuments(ids: number[], request: EditPdfDocumentsRequest) { editPdfDocuments(ids: number[], request: EditPdfDocumentsRequest) {
return this.http.post(this.getResourceUrl(null, 'edit_pdf'), { return this.http.post(this.getResourceUrl(null, 'edit_pdf'), {
documents: ids, documents: ids,
@@ -438,6 +438,21 @@ describe('ConsumerStatusService', () => {
expect(updated).toBeTruthy() expect(updated).toBeTruthy()
}) })
it('should ignore keep-alive heartbeat messages from the server', () => {
let updated = false
let deleted = false
websocketStatusService.onDocumentUpdated().subscribe(() => (updated = true))
websocketStatusService.onDocumentDeleted().subscribe(() => (deleted = true))
websocketStatusService.connect()
server.send({ type: WebsocketStatusType.HEARTBEAT })
expect(updated).toBeFalsy()
expect(deleted).toBeFalsy()
expect(websocketStatusService.getConsumerStatus()).toHaveLength(0)
websocketStatusService.disconnect()
})
it('should ignore document updated events the user cannot view', () => { it('should ignore document updated events the user cannot view', () => {
let updated = false let updated = false
websocketStatusService.onDocumentUpdated().subscribe(() => { websocketStatusService.onDocumentUpdated().subscribe(() => {
@@ -11,6 +11,7 @@ export enum WebsocketStatusType {
STATUS_UPDATE = 'status_update', STATUS_UPDATE = 'status_update',
DOCUMENTS_DELETED = 'documents_deleted', DOCUMENTS_DELETED = 'documents_deleted',
DOCUMENT_UPDATED = 'document_updated', DOCUMENT_UPDATED = 'document_updated',
HEARTBEAT = 'heartbeat',
} }
// see ProgressStatusOptions in src/documents/plugins/helpers.py // see ProgressStatusOptions in src/documents/plugins/helpers.py
@@ -207,6 +208,10 @@ export class WebsocketStatusService {
case WebsocketStatusType.STATUS_UPDATE: case WebsocketStatusType.STATUS_UPDATE:
this.handleProgressUpdate(messageData as WebsocketProgressMessage) this.handleProgressUpdate(messageData as WebsocketProgressMessage)
break break
case WebsocketStatusType.HEARTBEAT:
// keep-alive from the server, see paperless.consumers.StatusConsumer
break
} }
} }
} }
+2
View File
@@ -115,6 +115,7 @@ import {
house, house,
inbox, inbox,
infoCircle, infoCircle,
journalBookmarkFill,
journals, journals,
link, link,
listNested, listNested,
@@ -361,6 +362,7 @@ const icons = {
house, house,
inbox, inbox,
infoCircle, infoCircle,
journalBookmarkFill,
journals, journals,
link, link,
listNested, listNested,
+86 -2
View File
@@ -1,7 +1,7 @@
@use "sass:meta"; @use "sass:meta";
// bs variables // bs variables
$grid-gutter-width: 1.5rem; $grid-gutter-width: 1.5rem;
$border-radius: .375rem; $border-radius: .425rem;
$btn-border-width: var(--bs-border-width); $btn-border-width: var(--bs-border-width);
$form-file-button-bg: var(--bs-body-bg); $form-file-button-bg: var(--bs-body-bg);
@@ -68,15 +68,44 @@ body {
--pngx-body-font-size: 0.875rem; --pngx-body-font-size: 0.875rem;
font-size: var(--pngx-body-font-size); font-size: var(--pngx-body-font-size);
height: 100vh; height: 100vh;
letter-spacing: -0.005em;
} }
* { * {
transition: background-color 0.3s ease, border-color 0.3s ease; transition: background-color 0.3s ease, border-color 0.3s ease;
} }
.card,
.dropdown-menu,
.modal-content,
.popover {
--bs-card-border-color: color-mix(in srgb, var(--bs-border-color) 72%, transparent);
border-radius: .55rem;
}
.card {
box-shadow: 0 1px 2px rgba(0, 0, 0, .04), 0 8px 24px rgba(0, 0, 0, .035);
}
.btn {
--bs-btn-border-radius: .425rem;
--bs-border-radius-sm: .425rem;
font-weight: 500;
}
.form-control,
.form-select,
.input-group-text {
border-radius: .425rem;
}
.pagination, .input-group {
--bs-border-radius-sm: .425rem;
}
@media(min-width: 768px) { @media(min-width: 768px) {
.col-slim { .col-slim {
padding-left: calc(50px + $grid-gutter-width) !important; padding-left: calc(56px + $grid-gutter-width) !important;
} }
} }
@@ -607,10 +636,63 @@ table.table {
color: var(--bs-body-color); color: var(--bs-body-color);
} }
// Tighten horizontal spacing in modals on small viewports
@media (max-width: 575.98px) {
.modal {
--bs-modal-margin: 0.25rem;
--bs-modal-header-padding-x: 0.5rem;
--bs-modal-header-padding: var(--bs-modal-header-padding-y) var(--bs-modal-header-padding-x);
.modal-body {
padding-inline: 0.5rem;
}
.modal-footer {
padding-inline: 0.25rem;
}
.accordion {
--bs-accordion-btn-padding-x: 0.75rem;
--bs-accordion-body-padding-x: 0.5rem;
}
.card {
--bs-card-spacer-x: 0.5rem;
}
.list-group {
--bs-list-group-item-padding-x: 0.5rem;
}
}
}
// Wider-width modals on small viewports when the content is wide (e.g. landscape)
@media (min-width: 576px) and (max-height: 700px) {
.modal {
--bs-modal-margin: 0.5rem;
}
.modal-dialog {
max-width: min(var(--bs-modal-width), calc(100% - 1rem));
}
.modal-lg {
--bs-modal-width: 800px;
}
.modal-xl {
--bs-modal-width: 1140px;
}
}
.toast { .toast {
--bs-toast-max-width: var(--pngx-toast-max-width); --bs-toast-max-width: var(--pngx-toast-max-width);
} }
.alert {
--bs-border-radius: .425rem;
}
.alert-primary { .alert-primary {
--bs-alert-color: var(--bs-primary); --bs-alert-color: var(--bs-primary);
--bs-alert-bg: var(--pngx-primary-faded); --bs-alert-bg: var(--pngx-primary-faded);
@@ -742,6 +824,8 @@ code {
--bs-accordion-bg: var(--bs-light); --bs-accordion-bg: var(--bs-light);
--bs-accordion-active-color: var(--bs-primary); --bs-accordion-active-color: var(--bs-primary);
--bs-accordion-active-bg: var(--pngx-bg-alt); --bs-accordion-active-bg: var(--pngx-bg-alt);
--bs-border-radius: .425rem;
--bs-accordion-inner-border-radius: calc(.425rem - 1px);
} }
.accordion-button::after { .accordion-button::after {
+3 -3
View File
@@ -20,7 +20,7 @@
--pngx-primary-darken-27: hsl(var(--pngx-primary), calc(var(--pngx-primary-lightness) - 27%)); --pngx-primary-darken-27: hsl(var(--pngx-primary), calc(var(--pngx-primary-lightness) - 27%));
--pngx-success-darken-10: hsl(152, 69%, 11%); // based on success #198754 --pngx-success-darken-10: hsl(152, 69%, 11%); // based on success #198754
--pngx-bg-alt: #fff; --pngx-bg-alt: #fff;
--pngx-bg-darker: var(--bs-gray-100); --pngx-bg-darker: #f4f6f5;
--pngx-bg-alt2: var(--bs-gray-200); // #e9ecef --pngx-bg-alt2: var(--bs-gray-200); // #e9ecef
--pngx-bg-disabled: #f7f7f7; --pngx-bg-disabled: #f7f7f7;
--pngx-card-hover-border: var(--bs-tertiary-color); --pngx-card-hover-border: var(--bs-tertiary-color);
@@ -91,7 +91,7 @@ $form-check-radio-checked-bg-image-dark: url("data:image/svg+xml,%3csvg xmlns='h
--pngx-body-color-accent: #{$text-color-dark-bg-accent}; --pngx-body-color-accent: #{$text-color-dark-bg-accent};
--pngx-bg-alt: #242529; --pngx-bg-alt: #242529;
--pngx-bg-alt2: #232323; --pngx-bg-alt2: #232323;
--pngx-bg-darker: #101216; --pngx-bg-darker: #121315;
--pngx-bg-disabled: var(--pngx-bg-alt); --pngx-bg-disabled: var(--pngx-bg-alt);
--pngx-card-hover-border: var(--bs-border-color); --pngx-card-hover-border: var(--bs-border-color);
--pngx-focus-alpha: 0.6; --pngx-focus-alpha: 0.6;
@@ -107,7 +107,7 @@ $form-check-radio-checked-bg-image-dark: url("data:image/svg+xml,%3csvg xmlns='h
--bs-light-rgb: 28, 28, 31; --bs-light-rgb: 28, 28, 31;
--bs-info: var(--pngx-bg-alt); --bs-info: var(--pngx-bg-alt);
--bs-info-rgb: 36, 36, 39; --bs-info-rgb: 36, 36, 39;
--bs-border-color: #47494f; --bs-border-color: #34373d;
--bs-tertiary-bg: var(--pngx-bg-darker); --bs-tertiary-bg: var(--pngx-bg-darker);
--bs-dark-border-subtle: var(--pngx-bg-darker); --bs-dark-border-subtle: var(--pngx-bg-darker);
--bs-border-color-translucent: rgba(0, 0, 0, .175); // override bs --bs-border-color-translucent: rgba(0, 0, 0, .175); // override bs
+114
View File
@@ -12,6 +12,7 @@ from celery import group
from celery import shared_task from celery import shared_task
from django.conf import settings from django.conf import settings
from django.db import transaction from django.db import transaction
from django.db.models import Max
from django.db.models import Q from django.db.models import Q
from django.utils import timezone from django.utils import timezone
@@ -30,6 +31,7 @@ from documents.permissions import set_permissions_for_object
from documents.plugins.helpers import DocumentsStatusManager from documents.plugins.helpers import DocumentsStatusManager
from documents.tasks import bulk_update_documents from documents.tasks import bulk_update_documents
from documents.tasks import consume_file from documents.tasks import consume_file
from documents.tasks import remove_document_from_index
from documents.tasks import update_document_content_maybe_archive_file from documents.tasks import update_document_content_maybe_archive_file
from documents.versioning import get_latest_version_for_root from documents.versioning import get_latest_version_for_root
from documents.versioning import get_root_document from documents.versioning import get_root_document
@@ -39,6 +41,9 @@ if TYPE_CHECKING:
from django.contrib.auth.models import User from django.contrib.auth.models import User
if settings.AUDIT_LOG_ENABLED:
from auditlog.models import LogEntry
logger: logging.Logger = logging.getLogger("paperless.bulk_edit") logger: logging.Logger = logging.getLogger("paperless.bulk_edit")
SourceMode = Literal["latest_version", "explicit_selection"] SourceMode = Literal["latest_version", "explicit_selection"]
@@ -612,6 +617,115 @@ def merge(
return "OK" return "OK"
def merge_as_versions(
doc_ids: list[int],
*,
root_document_id: int,
version_label: str | None = None,
user: User | None = None,
) -> Literal["OK"]:
with transaction.atomic():
documents = list(
# Ordered by pk so concurrent merges take the row locks in the same order
Document.objects.select_for_update()
.filter(id__in=doc_ids)
.order_by("id")
.defer("content"),
)
documents_by_id = {document.id: document for document in documents}
source_ids = [doc_id for doc_id in doc_ids if doc_id != root_document_id]
root_document = documents_by_id[root_document_id]
next_version_index = (
Document.global_objects.filter(
root_document_id=root_document_id,
).aggregate(max_index=Max("version_index"))["max_index"]
or 0
)
# A version gives up its ASN
source_asns = [
documents_by_id[source_id].archive_serial_number
for source_id in source_ids
if documents_by_id[source_id].archive_serial_number is not None
]
updated_fields = ["root_document", "version_index", "archive_serial_number"]
if version_label is not None:
updated_fields.append("version_label")
for source_id in source_ids:
next_version_index += 1
source_document = documents_by_id[source_id]
source_document.root_document_id = root_document.pk
source_document.version_index = next_version_index
source_document.archive_serial_number = None
if version_label is not None:
source_document.version_label = version_label
# bulk_update and not save() to avoid post_save now
Document.objects.bulk_update(
[documents_by_id[source_id] for source_id in source_ids],
updated_fields,
)
root_updates = {"modified": timezone.now()}
if source_asns and root_document.archive_serial_number is None:
# If a version had one, hand the ASN over, the same as merge() does
root_updates["archive_serial_number"] = source_asns.pop(0)
logger.info(
f"Document {root_document.id} took archive serial number "
f"{root_updates['archive_serial_number']} from a document merged into it",
)
if source_asns:
logger.warning(
f"Archive serial number(s) {source_asns} were removed by merging "
f"those documents as versions of document {root_document.id}",
)
Document.objects.filter(pk=root_document.pk).update(**root_updates)
if settings.AUDIT_LOG_ENABLED:
# update() doesn't fire auditlog signals, so manual
LogEntry.objects.log_create(
instance=root_document,
changes={"Merged As Versions": ["None", source_ids]},
action=LogEntry.Action.UPDATE,
actor=user,
additional_data={
"reason": "Merged as versions",
"version_ids": source_ids,
},
)
# One batch rather than a task each
from documents.search import SearchIndexLockError
from documents.search import get_backend
try:
with get_backend().batch_update() as batch:
for source_id in source_ids:
batch.remove(source_id)
except SearchIndexLockError:
logger.error(
f"Search index lock exhausted removing {source_ids}, "
f"scheduling deferred index removal",
)
for source_id in source_ids:
remove_document_from_index.apply_async(args=[source_id], countdown=60)
bulk_update_documents.apply_async(
kwargs={"document_ids": [root_document_id]},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
# And as far as the frontend is concerned, they're deleted
status_mgr = DocumentsStatusManager()
status_mgr.send_documents_deleted(source_ids)
return "OK"
def split( def split(
doc_ids: list[int], doc_ids: list[int],
pages: list[list[int]], pages: list[list[int]],
+6 -4
View File
@@ -372,6 +372,10 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
For version documents, this is always the document's own content. For version documents, this is always the document's own content.
If the queryset already annotated ``effective_content``, that value is used. If the queryset already annotated ``effective_content``, that value is used.
""" """
# Here to avoid circular import
from documents.versioning import sort_versions_newest_first
from documents.versioning import versions_newest_first
if hasattr(self, "effective_content"): if hasattr(self, "effective_content"):
return getattr(self, "effective_content") return getattr(self, "effective_content")
@@ -388,12 +392,10 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
# Empty list means prefetch ran and found no versions — use own content. # Empty list means prefetch ran and found no versions — use own content.
if not prefetched_versions: if not prefetched_versions:
return self.content return self.content
latest_prefetched = max(prefetched_versions, key=lambda doc: doc.id) return sort_versions_newest_first(prefetched_versions)[0].content
return latest_prefetched.content
latest_version_content = ( latest_version_content = (
Document.objects.filter(root_document=self) versions_newest_first(Document.objects.filter(root_document=self))
.order_by("-id")
.values_list("content", flat=True) .values_list("content", flat=True)
.first() .first()
) )
+10 -2
View File
@@ -6,13 +6,20 @@ from documents.search._backend import TantivyRelevanceList
from documents.search._backend import WriteBatch from documents.search._backend import WriteBatch
from documents.search._backend import get_backend from documents.search._backend import get_backend
from documents.search._backend import reset_backend from documents.search._backend import reset_backend
from documents.search._errors import InvalidDateQuery
from documents.search._errors import InvalidNumberQuery
from documents.search._errors import MultipleSearchQueryErrors
from documents.search._errors import QueryTooLongError
from documents.search._errors import SearchQueryError
from documents.search._errors import search_query_error_messages
from documents.search._schema import needs_rebuild from documents.search._schema import needs_rebuild
from documents.search._schema import wipe_index from documents.search._schema import wipe_index
from documents.search._translate import InvalidDateQuery
from documents.search._translate import SearchQueryError
__all__ = [ __all__ = [
"InvalidDateQuery", "InvalidDateQuery",
"InvalidNumberQuery",
"MultipleSearchQueryErrors",
"QueryTooLongError",
"SearchHit", "SearchHit",
"SearchIndexLockError", "SearchIndexLockError",
"SearchMode", "SearchMode",
@@ -23,5 +30,6 @@ __all__ = [
"get_backend", "get_backend",
"needs_rebuild", "needs_rebuild",
"reset_backend", "reset_backend",
"search_query_error_messages",
"wipe_index", "wipe_index",
] ]
+72 -34
View File
@@ -20,12 +20,8 @@ from typing import cast
import filelock import filelock
import tantivy import tantivy
from django.conf import settings from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.utils.timezone import get_current_timezone from django.utils.timezone import get_current_timezone
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
from documents.search._query import build_permission_filter
from documents.search._query import extract_cjk_text from documents.search._query import extract_cjk_text
from documents.search._query import parse_simple_text_highlight_query from documents.search._query import parse_simple_text_highlight_query
from documents.search._query import parse_simple_text_query from documents.search._query import parse_simple_text_query
@@ -43,6 +39,7 @@ from documents.utils import QuerySetStream
from documents.utils import identity from documents.utils import identity
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Iterable
from collections.abc import Iterator from collections.abc import Iterator
from collections.abc import Sequence from collections.abc import Sequence
from pathlib import Path from pathlib import Path
@@ -266,11 +263,7 @@ class WriteBatch:
if self._lock is not None: if self._lock is not None:
self._lock.release() self._lock.release()
def add_or_update( def add_or_update(self, document: Document) -> None:
self,
document: Document,
effective_content: str | None = None,
) -> None:
""" """
Add or update a document in the batch. Add or update a document in the batch.
@@ -280,11 +273,9 @@ class WriteBatch:
Args: Args:
document: Django Document instance to index document: Django Document instance to index
effective_content: Override document.content for indexing (used when
re-indexing with newer OCR text from document versions)
""" """
self.remove(document.pk) self.remove(document.pk)
doc = self._backend._build_tantivy_doc(document, effective_content) doc = self._backend._build_tantivy_doc(document)
self._writer.add_document(doc) self._writer.add_document(doc)
def remove(self, doc_id: int) -> None: def remove(self, doc_id: int) -> None:
@@ -294,6 +285,47 @@ class WriteBatch:
) )
def build_permission_filter(
schema: tantivy.Schema,
user: AbstractUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
Creates a query that matches only documents visible to the specified user
according to paperless-ngx permission rules:
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
"""
owner_any = tantivy.Query.exists_query("owner_id")
no_owner = tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, tantivy.Query.all_query()),
(tantivy.Occur.MustNot, owner_any),
],
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
class TantivyBackend: class TantivyBackend:
""" """
Tantivy search backend with explicit lifecycle management. Tantivy search backend with explicit lifecycle management.
@@ -425,18 +457,20 @@ class TantivyBackend:
def _build_tantivy_doc( def _build_tantivy_doc(
self, self,
document: Document, document: Document,
effective_content: str | None = None,
viewer_ids: list[int] | None = None, viewer_ids: list[int] | None = None,
viewer_group_ids: list[int] | None = None, viewer_group_ids: list[int] | None = None,
) -> tantivy.Document: ) -> tantivy.Document:
"""Build a tantivy Document from a Django Document instance. """Build a tantivy Document from a Django Document instance.
``effective_content`` overrides ``document.content`` for indexing A root document is indexed with its effective content, i.e. the newest
used when re-indexing a root document with a newer version's OCR text. version's OCR text, so it is never indexed with its own outdated text.
Annotate the queryset with ``annotate_effective_content`` when indexing
more than a couple of documents, to resolve that without a query each.
""" """
content = ( from guardian.shortcuts import get_groups_with_perms
effective_content if effective_content is not None else document.content from guardian.shortcuts import get_users_with_perms
)
content = document.get_effective_content() or ""
doc = tantivy.Document() doc = tantivy.Document()
@@ -465,7 +499,6 @@ class TantivyBackend:
doc.add_text("correspondent_sort", document.correspondent.name) doc.add_text("correspondent_sort", document.correspondent.name)
if cjk_corr := extract_cjk_text(document.correspondent.name): if cjk_corr := extract_cjk_text(document.correspondent.name):
doc.add_text("bigram_correspondent", cjk_corr) doc.add_text("bigram_correspondent", cjk_corr)
doc.add_unsigned("correspondent_id", document.correspondent_id)
# Document type # Document type
if document.document_type: if document.document_type:
@@ -473,12 +506,10 @@ class TantivyBackend:
doc.add_text("type_sort", document.document_type.name) doc.add_text("type_sort", document.document_type.name)
if cjk_type := extract_cjk_text(document.document_type.name): if cjk_type := extract_cjk_text(document.document_type.name):
doc.add_text("bigram_document_type", cjk_type) doc.add_text("bigram_document_type", cjk_type)
doc.add_unsigned("document_type_id", document.document_type_id)
# Storage path # Storage path
if document.storage_path: if document.storage_path:
doc.add_text("storage_path", document.storage_path.name) doc.add_text("storage_path", document.storage_path.name)
doc.add_unsigned("storage_path_id", document.storage_path_id)
# Tags — collect names for autocomplete in the same pass # Tags — collect names for autocomplete in the same pass
tag_names: list[str] = [] tag_names: list[str] = []
@@ -486,12 +517,13 @@ class TantivyBackend:
doc.add_text("tag", tag.name) doc.add_text("tag", tag.name)
if cjk_tag := extract_cjk_text(tag.name): if cjk_tag := extract_cjk_text(tag.name):
doc.add_text("bigram_tag", cjk_tag) doc.add_text("bigram_tag", cjk_tag)
doc.add_unsigned("tag_id", tag.pk)
tag_names.append(tag.name) tag_names.append(tag.name)
# Notes — JSON for structured queries (notes.user:alice, notes.note:text). # Notes — JSON for structured queries (notes.user:alice, notes.note:text).
# notes_text is a plain-text companion for snippet/highlight generation; # notes_text is a plain-text companion for snippet/highlight generation;
# tantivy's SnippetGenerator does not support JSON fields. # tantivy's SnippetGenerator does not support JSON fields. It is not in
# _DEFAULT_SEARCH_FIELDS, so an unqualified query never searches it: a
# note matches through the JSON field or not at all.
num_notes = 0 num_notes = 0
note_texts: list[str] = [] note_texts: list[str] = []
for note in document.notes.all(): for note in document.notes.all():
@@ -507,8 +539,9 @@ class TantivyBackend:
if note_texts: if note_texts:
doc.add_text("notes_text", " ".join(note_texts)) doc.add_text("notes_text", " ".join(note_texts))
# Custom fields JSON for structured queries (custom_fields.name:x, custom_fields.value:y), # Custom fields: JSON for structured queries (custom_fields.name:x,
# companion text field for default full-text search. # custom_fields.value:y). There is no companion text field here, unlike
# notes: custom field values are reachable only through the JSON field.
for cfi in document.custom_fields.all(): for cfi in document.custom_fields.all():
search_value = cfi.value_for_search search_value = cfi.value_for_search
# Skip fields where there is no value yet # Skip fields where there is no value yet
@@ -584,11 +617,7 @@ class TantivyBackend:
return doc return doc
def add_or_update( def add_or_update(self, document: Document) -> None:
self,
document: Document,
effective_content: str | None = None,
) -> None:
""" """
Add or update a single document with file locking. Add or update a single document with file locking.
@@ -601,12 +630,11 @@ class TantivyBackend:
Args: Args:
document: Django Document instance to index document: Django Document instance to index
effective_content: Override document.content for indexing
""" """
self._ensure_open() self._ensure_open()
try: try:
with self.batch_update(lock_timeout=_LOCK_TIMEOUT_SECONDS) as batch: with self.batch_update(lock_timeout=_LOCK_TIMEOUT_SECONDS) as batch:
batch.add_or_update(document, effective_content) batch.add_or_update(document)
except SearchIndexLockError: except SearchIndexLockError:
logger.error( logger.error(
"Search index lock exhausted for document %d after %d attempts; " "Search index lock exhausted for document %d after %d attempts; "
@@ -680,7 +708,17 @@ class TantivyBackend:
user_query = self._parse_query(query, search_mode) user_query = self._parse_query(query, search_mode)
highlight_query = user_query highlight_query = user_query
if search_mode is SearchMode.TEXT: if search_mode is SearchMode.TEXT:
highlight_query = parse_simple_text_highlight_query(self._index, query) try:
highlight_query = parse_simple_text_highlight_query(
self._index,
query,
)
except ValueError:
logger.debug(
"Skipping simple text highlight query: token string is not "
"valid tantivy query syntax: %r",
query,
)
# For notes_text snippet generation, we need a query that targets the # For notes_text snippet generation, we need a query that targets the
# notes_text field directly. user_query may contain JSON-field terms # notes_text field directly. user_query may contain JSON-field terms
@@ -1027,7 +1065,6 @@ class TantivyBackend:
): ):
doc = self._build_tantivy_doc( doc = self._build_tantivy_doc(
document, document,
document.get_effective_content(),
viewer_ids=viewer_ids, viewer_ids=viewer_ids,
viewer_group_ids=viewer_group_ids, viewer_group_ids=viewer_group_ids,
) )
@@ -1095,6 +1132,7 @@ def _bulk_get_viewer_permissions(
""" """
from collections import defaultdict from collections import defaultdict
from django.contrib.contenttypes.models import ContentType
from guardian.models import GroupObjectPermission from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission from guardian.models import UserObjectPermission
-171
View File
@@ -1,171 +0,0 @@
from __future__ import annotations
from datetime import UTC
from datetime import date
from datetime import datetime
from datetime import timedelta
from typing import TYPE_CHECKING
from typing import Final
from dateutil.relativedelta import relativedelta
if TYPE_CHECKING:
from datetime import tzinfo
_DATE_ONLY_FIELDS = frozenset({"created"})
_TODAY: Final[str] = "today"
_YESTERDAY: Final[str] = "yesterday"
_PREVIOUS_WEEK: Final[str] = "previous week"
_THIS_MONTH: Final[str] = "this month"
_PREVIOUS_MONTH: Final[str] = "previous month"
_THIS_YEAR: Final[str] = "this year"
_PREVIOUS_YEAR: Final[str] = "previous year"
_PREVIOUS_QUARTER: Final[str] = "previous quarter"
_DATE_KEYWORDS = frozenset(
{
_TODAY,
_YESTERDAY,
_PREVIOUS_WEEK,
_THIS_MONTH,
_PREVIOUS_MONTH,
_THIS_YEAR,
_PREVIOUS_YEAR,
_PREVIOUS_QUARTER,
},
)
def _fmt(dt: datetime) -> str:
"""Format a datetime as an ISO 8601 UTC string for use in Tantivy range queries."""
return dt.astimezone(UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
def _iso_range(lo: datetime, hi: datetime) -> str:
"""
Format a half-open ``[lo TO hi)`` range in ISO 8601 for Tantivy query syntax.
``hi`` is always the exclusive ceiling of a computed period (the start of
the *next* day/week/month/quarter/year), so the closing bracket must be
the Tantivy exclusive-range brace ``}`` rather than ``]`` otherwise the
first instant of the following period (e.g. the 1st of next month) is
incorrectly included in the match.
"""
return f"[{_fmt(lo)} TO {_fmt(hi)}}}"
def _quarter_start(d: date) -> date:
"""Return the first day of the calendar quarter containing ``d``."""
return date(d.year, ((d.month - 1) // 3) * 3 + 1, 1)
def _midnight(d: date, tz: tzinfo) -> datetime:
"""Convert a calendar date at local-timezone midnight to a UTC datetime."""
return datetime(d.year, d.month, d.day, tzinfo=tz).astimezone(UTC)
def _keyword_bounds(keyword: str, tz: tzinfo) -> tuple[date, date]:
"""
Map a relative date keyword to ``(start, exclusive_end)`` calendar dates.
``tz`` only determines what "today" is; the caller decides how the returned
dates become UTC datetime boundaries (date-only vs. local-midnight offset).
"""
today = datetime.now(tz).date()
if keyword == _TODAY:
return today, today + timedelta(days=1)
if keyword == _YESTERDAY:
return today - timedelta(days=1), today
if keyword == _PREVIOUS_WEEK:
this_monday = today - timedelta(days=today.weekday())
return this_monday - timedelta(weeks=1), this_monday
if keyword == _THIS_MONTH:
first = today.replace(day=1)
return first, first + relativedelta(months=1)
if keyword == _PREVIOUS_MONTH:
this_first = today.replace(day=1)
return this_first - relativedelta(months=1), this_first
if keyword == _THIS_YEAR:
return date(today.year, 1, 1), date(today.year + 1, 1, 1)
if keyword == _PREVIOUS_YEAR:
return date(today.year - 1, 1, 1), date(today.year, 1, 1)
if keyword == _PREVIOUS_QUARTER:
this_quarter = _quarter_start(today)
return this_quarter - relativedelta(months=3), this_quarter
raise ValueError(f"Unknown keyword: {keyword}")
def _date_only_range(keyword: str, tz: tzinfo) -> str:
"""
For `created` (DateField): use the local calendar date, converted to
midnight UTC boundaries. No offset arithmetic date only.
"""
start, end = _keyword_bounds(keyword, tz)
lo = datetime(start.year, start.month, start.day, tzinfo=UTC)
hi = datetime(end.year, end.month, end.day, tzinfo=UTC)
return _iso_range(lo, hi)
def _datetime_range(keyword: str, tz: tzinfo) -> str:
"""
For `added` / `modified` (DateTimeField, stored as UTC): convert local day
boundaries to UTC full offset arithmetic required.
"""
start, end = _keyword_bounds(keyword, tz)
return _iso_range(_midnight(start, tz), _midnight(end, tz))
def _precision_bounds(digits: str) -> tuple[date, date] | None:
"""
Map a 4/6/8-digit date token to (start, exclusive_end) calendar dates.
YYYY -> whole year, YYYYMM -> whole month, YYYYMMDD -> single day.
Returns None for any unparsable or out-of-range value (e.g. month 23),
so callers can emit a no-match clause instead of erroring (Whoosh parity).
"""
try:
if len(digits) == 4:
year = int(digits)
return date(year, 1, 1), date(year + 1, 1, 1)
if len(digits) == 6:
year, month = int(digits[:4]), int(digits[4:6])
start = date(year, month, 1)
end = date(year + 1, 1, 1) if month == 12 else date(year, month + 1, 1)
return start, end
if len(digits) == 8:
start = date(int(digits[:4]), int(digits[4:6]), int(digits[6:8]))
return start, start + timedelta(days=1)
except ValueError:
return None
return None
def _utc_bounds_for_field(
field: str,
start: date,
end: date,
tz: tzinfo,
) -> tuple[datetime, datetime]:
"""
Convert calendar-date bounds to UTC datetimes per the field's storage type.
For DateField (``created``) the bounds are UTC midnight (no offset). For
DateTimeField (``added``/``modified``) the bounds are local-tz midnight
converted to UTC, matching how each field is indexed.
"""
if field in _DATE_ONLY_FIELDS:
return (
datetime(start.year, start.month, start.day, tzinfo=UTC),
datetime(end.year, end.month, end.day, tzinfo=UTC),
)
return (
datetime(start.year, start.month, start.day, tzinfo=tz).astimezone(UTC),
datetime(end.year, end.month, end.day, tzinfo=tz).astimezone(UTC),
)
def _field_range_from_dates(field: str, start: date, end: date, tz: tzinfo) -> str:
"""Build a Tantivy ``field:[lo TO hi]`` ISO range from calendar-date bounds."""
lo, hi = _utc_bounds_for_field(field, start, end, tz)
return f"{field}:{_iso_range(lo, hi)}"
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from collections.abc import Sequence
class SearchQueryError(ValueError):
"""
Base for user-fixable search query errors.
Carries a message safe to surface to the user (no internal details). The
view layer catches this and returns an HTTP 400, so any future subclass
gets the same treatment.
"""
class InvalidDateQuery(SearchQueryError):
"""Raised when a date field value or range bound cannot be parsed."""
def __init__(self, field: str | None, value: str | None) -> None:
self.field = field
self.value = value
super().__init__(f"Invalid date value {value!r} for field {field!r}.")
class InvalidNumberQuery(SearchQueryError):
"""Raised when a numeric field value or range bound cannot be parsed."""
def __init__(self, field: str | None, value: str | None) -> None:
self.field = field
self.value = value
super().__init__(f"Invalid numeric value {value!r} for field {field!r}.")
class QueryTooLongError(SearchQueryError):
"""Raised when a query string exceeds the maximum allowed length.
whoosh-compat's fieldname tagger is O(n^2) in plain word characters, so an
unbounded query is a CPU-exhaustion vector against a single request
handler. This is a hard boundary, not a validation nicety.
"""
def __init__(self, length: int, limit: int) -> None:
self.length = length
self.limit = limit
super().__init__(
f"The search query is too long ({length} characters). "
f"The maximum allowed length is {limit} characters.",
)
class MultipleSearchQueryErrors(SearchQueryError):
"""Aggregates every user-fixable error from one parse, not just the first."""
def __init__(self, errors: Sequence[SearchQueryError]) -> None:
self.errors = tuple(errors)
super().__init__("; ".join(str(e) for e in self.errors))
def search_query_error_messages(e: SearchQueryError) -> list[str]:
"""The user-facing message list for a SearchQueryError.
Every offending value's message, not just the first, so the user can
fix them all in one round-trip. Shared by every view that maps
SearchQueryError to an HTTP 400.
"""
if isinstance(e, MultipleSearchQueryErrors):
return [str(sub) for sub in e.errors]
return [str(e)]
+42
View File
@@ -0,0 +1,42 @@
from __future__ import annotations
from whoosh_compat import FieldKind
from whoosh_compat import FieldSpec
from whoosh_compat import SubpathSpec
# Internal-only schema fields with no query-syntax meaning of their own
# (sort shadow fields, bigram CJK fields, simple_title/simple_content,
# autocomplete_word, notes_text) are NOT represented here, they are
# declared in _schema.py's field_descriptors().
#
# analyzer/pattern_normalizer are deliberately left at FieldSpec's default
# (None): they're language-specific and only meaningful to whoosh-compat's
# parser, so _registry.py attaches them per-language via dataclasses.replace()
# rather than PUBLIC_FIELDS declaring them itself. _schema.py only reads
# name/kind/fast and never sees the analyzer at all.
PUBLIC_FIELDS: tuple[FieldSpec, ...] = (
FieldSpec("title", FieldKind.TEXT),
FieldSpec("content", FieldKind.TEXT),
FieldSpec("correspondent", FieldKind.TEXT),
FieldSpec("document_type", FieldKind.TEXT, aliases=("type",)),
FieldSpec("storage_path", FieldKind.TEXT, aliases=("path",)),
FieldSpec("original_filename", FieldKind.TEXT),
FieldSpec("tag", FieldKind.TEXT, comma_values=True),
FieldSpec("checksum", FieldKind.KEYWORD),
FieldSpec("asn", FieldKind.U64, fast=True),
FieldSpec("page_count", FieldKind.U64, fast=True),
FieldSpec("num_notes", FieldKind.U64, fast=True),
FieldSpec("created", FieldKind.DATE, date_only=True, fast=True),
FieldSpec("modified", FieldKind.DATETIME, fast=True),
FieldSpec("added", FieldKind.DATETIME, fast=True),
FieldSpec(
"notes",
FieldKind.JSON,
subpaths={"user": SubpathSpec(), "note": SubpathSpec(default=True)},
),
FieldSpec(
"custom_fields",
FieldKind.JSON,
subpaths={"name": SubpathSpec(), "value": SubpathSpec(default=True)},
),
)
+450 -145
View File
@@ -6,22 +6,30 @@ from typing import Final
import regex import regex
import tantivy import tantivy
import whoosh_compat as wc
from django.conf import settings from django.conf import settings
from whoosh_compat.emitters.tantivy_ import emit as tantivy_emit
from whoosh_compat.errors import Cause
from whoosh_compat.errors import Diagnostic
from whoosh_compat.errors import DiagnosticKind
from whoosh_compat.errors import QueryError
from documents.search._errors import InvalidDateQuery
from documents.search._errors import InvalidNumberQuery
from documents.search._errors import MultipleSearchQueryErrors
from documents.search._errors import SearchQueryError
from documents.search._registry import get_field_registry
from documents.search._tokenizer import simple_search_tokens from documents.search._tokenizer import simple_search_tokens
from documents.search._translate import SearchQueryError
from documents.search._translate import translate_query
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Iterable
from datetime import tzinfo from datetime import tzinfo
from django.contrib.auth.base_user import AbstractBaseUser
logger = logging.getLogger("paperless.search") logger = logging.getLogger("paperless.search")
# Maximum seconds any single regex substitution may run. # Maximum seconds any single regex substitution over user-supplied query text
# Prevents ReDoS on adversarial user-supplied query strings. # may run. The one remaining use is a character class, which cannot backtrack,
# so the bound is an upper limit on that substitution's cost, not the ReDoS
# guard it was originally written as.
_REGEX_TIMEOUT: Final[float] = 1.0 _REGEX_TIMEOUT: Final[float] = 1.0
# Matches CJK/Hangul characters so queries can be routed to bigram fields. # Matches CJK/Hangul characters so queries can be routed to bigram fields.
@@ -29,6 +37,64 @@ _REGEX_TIMEOUT: Final[float] = 1.0
_CJK_RE: Final = regex.compile(r"[\p{Han}\p{Hiragana}\p{Katakana}\p{Hangul}]+") _CJK_RE: Final = regex.compile(r"[\p{Han}\p{Hiragana}\p{Katakana}\p{Hangul}]+")
def _user_facing_emit_message(d: Diagnostic) -> str:
"""A user-safe message for an emit-time QueryError's Diagnostic.
Built from the Diagnostic's structured fields (kind, field), never from
d.message: whoosh-compat documents that as developer/log output with no
stability guarantee, and PATTERN_TOO_COMPLEX embeds the raw backend
error text in it.
"""
field = str(d.field) if d.field is not None else None
if d.kind is DiagnosticKind.EXISTS_REQUIRES_FAST:
return f"Existence searches (field:*) are not supported for field {field!r}."
if d.kind is DiagnosticKind.TEXT_RANGE:
return f"Range searches are not supported for field {field!r}."
if d.kind is DiagnosticKind.PATTERN_TOO_COMPLEX:
return f"The wildcard pattern for field {field!r} is too complex."
if d.kind is DiagnosticKind.SCHEMA_FIELD_MISSING:
return f"Field {field!r} is not available in the search index."
logger.warning("Unmapped emit diagnostic %s: %s", d.kind, d.message)
return "The search query could not be executed."
def _map_emit_error(e: QueryError) -> SearchQueryError:
"""Route an emit-time QueryError by its Diagnostic's Cause.
INVALID_INPUT/UNSUPPORTED are user-input errors, exactly like a parse
diagnostic, and map to a 400. INTERNAL means a defect in whoosh-compat
or in our own AST handling, never the user's query, so the QueryError is
re-raised rather than converted, reaching the generic 500 handler instead
of blaming the query. MISCONFIGURED is deliberately both: the registry and
the index schema disagree, which only an operator can fix, so it is logged
as an error, but a request is still waiting and the query cannot run
either way, so it also returns a 400.
EXISTS_REQUIRES_FAST is the one MISCONFIGURED kind that is not a
disagreement. whoosh-compat derives it from the registry's own FieldSpec
(kind plus fast) without ever consulting the index schema, so it fires
whenever a non-fast field of a kind that cannot answer "exists" is asked
to: for us that is only the JSON fields, which field_descriptors() builds
non-fast on purpose. "notes:*" and the five other spellings of it are
ordinary user error that no operator action can clear, so they get the
400 without the alert.
"""
d = e.diagnostic
if d.cause is Cause.INTERNAL:
raise e
if (
d.cause is Cause.MISCONFIGURED
and d.kind is not DiagnosticKind.EXISTS_REQUIRES_FAST
):
logger.error(
"Search index misconfiguration for field %s (%s): %s",
d.field,
d.kind.name,
d.message,
)
return SearchQueryError(_user_facing_emit_message(d))
def _has_cjk(text: str) -> bool: def _has_cjk(text: str) -> bool:
"""Return True if text contains any CJK characters.""" """Return True if text contains any CJK characters."""
return bool(_CJK_RE.search(text)) return bool(_CJK_RE.search(text))
@@ -37,14 +103,36 @@ def _has_cjk(text: str) -> bool:
def extract_cjk_text(text: str) -> str: def extract_cjk_text(text: str) -> str:
"""Join the CJK runs in ``text`` for indexing into bigram (char-ngram) fields. """Join the CJK runs in ``text`` for indexing into bigram (char-ngram) fields.
Mirrors the query side (``_build_cjk_query``): only CJK runs are ever searched Mirrors the query side, which extracts the CJK runs of whatever it is
against the bigram fields, so only CJK runs are worth indexing there. Latin about to search for (the raw string in simple modes, the parsed query's
text fed to a character-bigram field is never matched and only bloats the free-text tokens in query mode): only CJK runs are ever searched against
the bigram fields, so only CJK runs are worth indexing there. Latin text
fed to a character-bigram field is never matched and only bloats the
index and slows indexing/merge. Returns "" when there is no CJK text. index and slows indexing/merge. Returns "" when there is no CJK text.
""" """
return " ".join(_CJK_RE.findall(text)) return " ".join(_CJK_RE.findall(text))
def _parse_cjk_text(
index: tantivy.Index,
cjk_text: str,
fields: list[str],
) -> tantivy.Query | None:
"""Parse a plain CJK run string against ``fields``, or None if it won't parse."""
try:
return index.parse_query(cjk_text, fields)
except Exception:
# Broad on purpose, unlike _try_parse_fuzzy_query's narrower
# ValueError: cjk_text isn't filtered to a guaranteed-safe token
# set the way the fuzzy blend's word string is, so the exact
# failure mode tantivy could raise here isn't pinned down.
logger.debug(
"Skipping CJK search clause: could not parse CJK text: %r",
cjk_text,
)
return None
def _build_cjk_query( def _build_cjk_query(
index: tantivy.Index, index: tantivy.Index,
raw_query: str, raw_query: str,
@@ -52,91 +140,259 @@ def _build_cjk_query(
) -> tantivy.Query | None: ) -> tantivy.Query | None:
"""Build a bigram-field query from the CJK runs in ``raw_query``. """Build a bigram-field query from the CJK runs in ``raw_query``.
Only the CJK character runs are extracted and parsed; ASCII field prefixes, For the simple (TEXT/TITLE) modes, whose input is plain text and carries
boolean operators and date keywords are discarded. This keeps the CJK clause no query grammar to respect. Only the CJK character runs are extracted, so
plain-text and consistent across query/simple modes (no leaked ``field:`` a stray ``field:`` prefix or ``-``/``+`` in the input can neither leak
semantics, no parse failures from spaced ``-``/``+``), and avoids feeding field semantics nor fail the parse, and no Latin token reaches the
Latin tokens into the character-bigram matcher (which would produce spurious character-bigram matcher (where it would produce spurious matches against
matches against unrelated Latin text). Returns None when there is no CJK unrelated Latin text). Returns None when there is no CJK text or the parse
text or the parse fails. fails.
""" """
cjk_text = " ".join(_CJK_RE.findall(raw_query)) cjk_text = extract_cjk_text(raw_query)
if not cjk_text: if not cjk_text:
return None return None
return _parse_cjk_text(index, cjk_text, fields)
def _build_ast_cjk_query(
index: tantivy.Index,
ast: wc.ast.Node,
registry: wc.FieldRegistry,
) -> tantivy.Query | None:
"""Build the bigram clause of a QUERY-mode search from the parsed AST.
Same discipline as the fuzzy clause (see _try_parse_fuzzy_query): the CJK
runs come from whoosh_compat's ``free_text_tokens`` over the parsed tree,
never from the raw query string, so a term the user negated or restricted
to a field outside the default search fields contributes nothing, instead
of resurfacing as a top-level clause matching every bigram field.
``free_text_tokens`` reports no field of its own, so the tokens are
collected one default field at a time: a bare term, which the parser has
already copied onto every default field, is therefore searched across
every bigram field, while ``title:東京`` reaches ``bigram_title`` alone.
Fields whose CJK text is identical (the bare-term case) share a single
parse over all of their bigram fields at once.
Raw (``analyzed=False``) tokens are used because the bigram fields have
their own character-ngram analyzer: the default fields' word analyzers
have no useful say over a CJK run, and running them first would only
risk dropping it (remove_long) before the run is ever extracted.
Returns None when the query has no CJK free text.
"""
fields_by_text: dict[str, list[str]] = {}
for field, bigram_field in _CJK_BIGRAM_FIELDS.items():
tokens = wc.free_text_tokens(
ast,
registry=registry,
fields=[field],
analyzed=False,
)
cjk_text = extract_cjk_text(" ".join(tokens))
if cjk_text:
fields_by_text.setdefault(cjk_text, []).append(bigram_field)
clauses: list[tuple[tantivy.Occur, tantivy.Query]] = [
(tantivy.Occur.Should, query)
for cjk_text, bigram_fields in fields_by_text.items()
if (query := _parse_cjk_text(index, cjk_text, bigram_fields)) is not None
]
return _any_of(clauses) if clauses else None
# A joined fuzzy word string must stay plain words: it goes back through
# tantivy's own query parser, and the raw query text the clause collects
# routinely carries characters that parser reads as grammar (a colon, a
# bracket, a quote, a leading -). Each token is cut into its word runs and
# only those are kept, so no field syntax, pattern, range or grouping can
# reach the parser. Cutting rather than dropping the whole token is what
# keeps ordinary hyphenated, dotted and quoted input ("COVID-19",
# "hello@example.com", "tax reports") contributing to the clause at all.
_WORD_RUN_RE = regex.compile(r"\w+")
# The one piece of tantivy grammar that survives the cut: its boolean
# keywords are themselves word runs. Only these exact spellings are
# grammar there ("And"/"and" are ordinary terms), so lowercasing exactly
# these turns them back into the ordinary terms the field analyzer used to
# make of them, before the clause switched to raw text. Left alone, a
# quoted phrase would silently restructure the clause ("tax AND reports"
# becoming a conjunction) or fail to parse and drop it entirely
# ("tax AND", or "IN" anywhere).
#
# Only these words are touched: tantivy lowercases query terms with the
# field's own analyzer, and doing it ourselves first is not always the
# same operation (Python folds a final sigma to a different letter than
# tantivy does, and turns Turkish 'İ' into a sequence tantivy then splits
# in two), which would search for terms the index does not contain.
_TANTIVY_KEYWORDS: Final[frozenset[str]] = frozenset({"AND", "OR", "NOT", "IN"})
def _try_parse_fuzzy_query(
index: tantivy.Index,
ast: wc.ast.Node,
registry: wc.FieldRegistry,
) -> tantivy.Query | None:
"""Build the fuzzy blend clause from the parsed query's free-text
words, or None if it has none.
The clause is built by handing tantivy's own query parser a plain
word string (there's no clean AST-level fuzzy equivalent to
whoosh-compat's parse tree, and fuzzy matching was always an
approximate, secondary, 0.1-boosted clause). The words come from
whoosh_compat's ``free_text_tokens`` over the already-parsed AST,
never from the raw query string: raw whoosh grammar (date keywords,
``[2005 to 2009]`` ranges, bracket-class wildcards) is not tantivy
syntax, and feeding it here used to knock the fuzzy clause out for
the whole query the moment any such construct appeared alongside a
typo'd word. The helper also keeps excluded terms out: a ``NOT``'d
word must not resurface through the fuzzy clause.
Chosen trade-off: a term explicitly fielded on one of the default
search fields (``correspondent:acme``) contributes its text to the
word string UNFIELDED, so the fuzzy clause searches it across all
default fields rather than just the one the user named. That is
recall-only widening on a secondary 0.1-boosted clause the score
threshold already disciplines, accepted in exchange for never feeding
field syntax to tantivy's parser. What the word string guarantees is
exactly that: no field prefix, pattern, range, grouping or quoting
survives, and the boolean keywords that do survive (they are word
runs) are lowercased into ordinary terms; see _TANTIVY_KEYWORDS.
The words are the query's RAW text, not the analyzer's output
(``analyzed=False``), because ``index.parse_query`` analyzes whatever
it is given and analysis is not idempotent: ``universities`` stems to
``univers``, and handing that back stems it again to ``univ``, a term
the index does not contain. ``prefix=True`` hid this as over-broad
matching (``univ`` also prefixes ``unicycle``) rather than as no
matches at all. Raw text is untokenized, which is why it is cut into
word runs above rather than taken whole.
The ValueError guard stays as insurance (the word string is plain
tokens, so tantivy accepting it is expected, not assumed): on a parse
failure the fuzzy clause is skipped and the exact/CJK clauses stand,
rather than the whole query failing.
"""
tokens = wc.free_text_tokens(
ast,
registry=registry,
fields=_DEFAULT_SEARCH_FIELDS,
analyzed=False,
)
words = list(
dict.fromkeys(
word.lower() if word in _TANTIVY_KEYWORDS else word
for token in tokens
for word in _WORD_RUN_RE.findall(token)
),
)
if not words:
return None
fuzzy_text = " ".join(words)
try: try:
return index.parse_query(cjk_text, fields) return index.parse_query(
except Exception: fuzzy_text,
_DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
fuzzy_fields={f: (True, 1, True) for f in _DEFAULT_SEARCH_FIELDS},
)
except ValueError:
logger.debug(
"Skipping fuzzy search clause: token string is not valid "
"tantivy query syntax: %r",
fuzzy_text,
)
return None return None
def build_permission_filter( _DEFAULT_SEARCH_FIELDS: Final[list[str]] = [
schema: tantivy.Schema,
user: AbstractBaseUser,
viewer_group_ids: Iterable[int] = (),
) -> tantivy.Query:
"""
Build a query filter for user document permissions.
Creates a query that matches only documents visible to the specified user
according to paperless-ngx permission rules:
- Public documents (no owner) are visible to all users
- Private documents are visible to their owner
- Documents explicitly shared with the user are visible
- Documents shared with one of the user's current groups are visible
Args:
schema: Tantivy schema for field validation
user: User to check permissions for
viewer_group_ids: Current group memberships for the user
Returns:
Tantivy query that filters results to visible documents
"""
owner_any = tantivy.Query.exists_query("owner_id")
no_owner = tantivy.Query.boolean_query(
[
(tantivy.Occur.Must, tantivy.Query.all_query()),
(tantivy.Occur.MustNot, owner_any),
],
)
owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
group_shared = [
tantivy.Query.term_query(schema, "viewer_group_id", group_id)
for group_id in viewer_group_ids
]
return tantivy.Query.disjunction_max_query(
[no_owner, owned, shared, *group_shared],
)
DEFAULT_SEARCH_FIELDS = [
"title", "title",
"content", "content",
"correspondent", "correspondent",
"document_type", "document_type",
"tag", "tag",
] ]
SIMPLE_SEARCH_FIELDS = ["simple_title", "simple_content"] _SIMPLE_SEARCH_FIELDS: Final[list[str]] = ["simple_title", "simple_content"]
TITLE_SEARCH_FIELDS = ["simple_title"] _TITLE_SEARCH_FIELDS: Final[list[str]] = ["simple_title"]
_CJK_ALL_FIELDS: Final[list[str]] = [ # The bigram (character-ngram) companion of each default search field.
"bigram_content", _CJK_BIGRAM_FIELDS: Final[dict[str, str]] = {
"bigram_title", field: f"bigram_{field}" for field in _DEFAULT_SEARCH_FIELDS
"bigram_correspondent", }
"bigram_document_type",
"bigram_tag",
]
_CJK_CONTENT_FIELDS: Final[list[str]] = ["bigram_content"] _CJK_CONTENT_FIELDS: Final[list[str]] = ["bigram_content"]
_CJK_TITLE_FIELDS: Final[list[str]] = ["bigram_title"] _CJK_TITLE_FIELDS: Final[list[str]] = ["bigram_title"]
_FIELD_BOOSTS = {"title": 2.0} _FIELD_BOOSTS = {"title": 2.0}
_SIMPLE_FIELD_BOOSTS = {"simple_title": 2.0} _SIMPLE_FIELD_BOOSTS = {"simple_title": 2.0}
def _simple_query_tokens(raw_query: str) -> list[str]: class _ConjunctiveNegations(wc.ast.Visitor[tuple["wc.ast.Node", ...]]):
# Tokenize and fold via the same analyzer used to index simple_title / """Collect the subtrees an AST excludes from every document it matches.
# simple_content, so query terms fold identically to the indexed terms
# (single source of truth for ASCII folding). A negation reached through ``And``/``AndNot``/``Require`` (and through
return simple_search_tokens(raw_query) the required half of an ``AndMaybe``) constrains the whole query, so it
can be re-stated above the blend. ``Or`` is deliberately not descended
into: in ``invoice OR NOT secret`` the negation is one branch's own
condition, and hoisting it would throw away documents the other branch
matches. Nor is a collected subtree descended into, since a negation
inside a negation is not an exclusion.
Node types with no negation to contribute (every leaf, ``Or``) fall
through to ``generic_visit``.
"""
def generic_visit(self, node: wc.ast.Node) -> tuple[wc.ast.Node, ...]:
return ()
def visit_not(self, node: wc.ast.Not) -> tuple[wc.ast.Node, ...]:
return (node.child,)
def visit_andnot(self, node: wc.ast.AndNot) -> tuple[wc.ast.Node, ...]:
return (*self.visit(node.positive), node.negative)
def visit_and(self, node: wc.ast.And) -> tuple[wc.ast.Node, ...]:
return tuple(
negation for child in node.children for negation in self.visit(child)
)
def visit_boosted(self, node: wc.ast.Boosted) -> tuple[wc.ast.Node, ...]:
return self.visit(node.child)
def visit_andmaybe(self, node: wc.ast.AndMaybe) -> tuple[wc.ast.Node, ...]:
return self.visit(node.required)
def visit_require(self, node: wc.ast.Require) -> tuple[wc.ast.Node, ...]:
return (*self.visit(node.scored), *self.visit(node.filter_only))
def _negation_clauses(
index: tantivy.Index,
ast: wc.ast.Node,
registry: wc.FieldRegistry,
) -> list[tuple[tantivy.Occur, tantivy.Query]]:
"""MustNot clauses for everything ``ast`` excludes conjunctively.
Each excluded subtree is emitted as its own positive query and attached
with ``MustNot``, rather than emitting a negative query and hoping
tantivy accepts a bare one.
"""
try:
return [
(
tantivy.Occur.MustNot,
tantivy_emit(negation, index=index, registry=registry),
)
for negation in _ConjunctiveNegations().visit(ast)
]
except QueryError as e:
raise _map_emit_error(e) from e
def _any_of(clauses: list[tuple[tantivy.Occur, tantivy.Query]]) -> tantivy.Query:
"""Collapse a clause list: none -> empty, one -> itself (no wasted
single-clause boolean_query wrapping), many -> boolean_query(clauses)."""
if not clauses:
return tantivy.Query.empty_query()
if len(clauses) == 1:
return clauses[0][1]
return tantivy.Query.boolean_query(clauses)
def _build_simple_token_query( def _build_simple_token_query(
@@ -168,9 +424,7 @@ def _build_simple_token_query(
query = tantivy.Query.boost_query(query, boost) query = tantivy.Query.boost_query(query, boost)
field_queries.append((tantivy.Occur.Should, query)) field_queries.append((tantivy.Occur.Should, query))
if len(field_queries) == 1: return _any_of(field_queries)
return field_queries[0][1]
return tantivy.Query.boolean_query(field_queries)
def parse_user_query( def parse_user_query(
@@ -179,52 +433,53 @@ def parse_user_query(
tz: tzinfo, tz: tzinfo,
) -> tantivy.Query: ) -> tantivy.Query:
""" """
Parse user query through the complete preprocessing pipeline. Parse user query through whoosh-compat, then blend in fuzzy/CJK clauses.
Transforms the raw user query through multiple stages: 1. wc.parse() against the shared FieldRegistry (whoosh grammar -> AST).
1. Date keyword rewriting (today ISO 8601 ranges) Bare notes:/custom_fields: prefixes resolve to their default subpath
2. Query normalization (comma expansion, whitespace cleanup) (notes.note:/custom_fields.value:) directly in the registry, via
3. Tantivy parsing with field boosts each JSON field's SubpathSpec(default=True).
4. Optional fuzzy query blending (if ADVANCED_FUZZY_SEARCH_THRESHOLD set) 2. Any diagnostics (bad dates/numbers) map to SearchQueryError subclasses
and raise, the view returns HTTP 400 with every offending field
Args: listed, not just the first.
index: Tantivy index with registered tokenizers 3. emit() turns the AST into a tantivy.Query directly (no string
raw_query: Original user query string round-trip). A QueryError is routed by its Diagnostic's Cause
tz: Timezone for date boundary calculations (_map_emit_error): a construct that parses but can't execute against
tantivy (e.g. a text-field range) is a 400, a registry/schema
Returns: mismatch is logged and a 400, and an INTERNAL defect is re-raised.
Parsed Tantivy query ready for execution 4. Optional fuzzy blend (ADVANCED_FUZZY_SEARCH_THRESHOLD) builds a
plain word string from the parsed AST's free-text tokens
Note: (whoosh_compat.free_text_tokens) and feeds THAT to
When ADVANCED_FUZZY_SEARCH_THRESHOLD is configured, adds a low-priority index.parse_query, never raw_query, whose whoosh grammar (date
fuzzy query as a Should clause (0.1 boost) to catch approximate matches keywords, bracket-class wildcards, etc.) tantivy's parser rejects,
while keeping exact matches ranked higher. The threshold value is applied which used to silently knock the fuzzy clause out of any mixed
as a post-search score filter, not during query construction. query (see _try_parse_fuzzy_query).
5. Optional CJK bigram clause, built from the same parsed AST for the
same reason (see _build_ast_cjk_query): a CJK term the query negated
or fielded must not resurface through it.
6. When any optional clause was added, the query's conjunctive
exclusions are restated as MustNot above the blend
(_negation_clauses): a clause built from positive terms cannot
express them, and as a bare Should it would undo them.
""" """
registry = get_field_registry(settings.SEARCH_LANGUAGE)
result = wc.parse(
raw_query,
registry=registry,
default_fields=_DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
tz=tz,
)
if result.diagnostics:
raise _diagnostics_to_error(result.diagnostics)
try: try:
query_str = translate_query(raw_query, tz) exact = tantivy_emit(result.ast, index=index, registry=registry)
except SearchQueryError: except QueryError as e:
# Intentional, user-fixable error (e.g. an unparsable date). Propagate so raise _map_emit_error(e) from e
# the view can return a 400 with a helpful message rather than falling
# back to the raw (still-invalid) query.
raise
except Exception: # pragma: no cover - defensive
logger.warning("Query translation failed; using raw query", exc_info=True)
query_str = raw_query
exact = index.parse_query(
query_str,
DEFAULT_SEARCH_FIELDS,
field_boosts=_FIELD_BOOSTS,
)
# The standard analyzer keeps a whitespace-free CJK run as a single token,
# so substring queries can't match content/title (and long runs are dropped
# by remove_long). Route CJK queries to the bigram fields, whose ngram
# tokenizer indexes overlapping 2-grams for substring matching.
cjk_query = ( cjk_query = (
_build_cjk_query(index, raw_query, _CJK_ALL_FIELDS) _build_ast_cjk_query(index, result.ast, registry)
if _has_cjk(raw_query) if _has_cjk(raw_query)
else None else None
) )
@@ -235,22 +490,73 @@ def parse_user_query(
threshold = settings.ADVANCED_FUZZY_SEARCH_THRESHOLD threshold = settings.ADVANCED_FUZZY_SEARCH_THRESHOLD
if threshold is not None: if threshold is not None:
fuzzy = index.parse_query( fuzzy = _try_parse_fuzzy_query(index, result.ast, registry)
query_str, if fuzzy is not None:
DEFAULT_SEARCH_FIELDS, clauses.append(
field_boosts=_FIELD_BOOSTS, (tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)),
# (prefix=True, distance=1, transposition_cost_one=True) — edit-distance fuzziness )
fuzzy_fields={f: (True, 1, True) for f in DEFAULT_SEARCH_FIELDS},
)
# 0.1 boost keeps fuzzy hits ranked below exact matches (intentional)
clauses.append((tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)))
if cjk_query is not None: if cjk_query is not None:
clauses.append((tantivy.Occur.Should, cjk_query)) clauses.append((tantivy.Occur.Should, cjk_query))
if len(clauses) == 1: if len(clauses) == 1:
return exact return exact
return tantivy.Query.boolean_query(clauses) # The fuzzy and CJK clauses are built from positive terms only, so as
# plain Shoulds beside the exact clause they re-admit exactly the
# documents the query excluded. Restate the exclusions once, above the
# whole blend. Redundant against the exact clause, which already
# carries them, but idempotently so.
negations = _negation_clauses(index, result.ast, registry)
if not negations:
return _any_of(clauses)
return tantivy.Query.boolean_query(
[(tantivy.Occur.Must, _any_of(clauses)), *negations],
)
# The three whoosh-compat kinds for a wildcard on a field that cannot
# carry one. d.field_kind supplies the discriminator, so naming the field's
# type needs no second trip through the registry.
_PATTERN_ON_KINDS: Final = frozenset(
{
DiagnosticKind.PATTERN_ON_NUMERIC,
DiagnosticKind.PATTERN_ON_BOOLEAN_EXISTS,
DiagnosticKind.PATTERN_ON_SUBPATH,
},
)
def _diagnostics_to_error(diagnostics: tuple[Diagnostic, ...]) -> SearchQueryError:
errors = [_single_diagnostic_to_error(d) for d in diagnostics]
return errors[0] if len(errors) == 1 else MultipleSearchQueryErrors(errors)
def _single_diagnostic_to_error(d: Diagnostic) -> SearchQueryError:
# d.field is a FieldRef, not a str: str(d.field) gives the canonical
# dotted name (an aliased query, e.g. type:, reports document_type).
field_name = str(d.field) if d.field is not None else None
if d.kind is DiagnosticKind.BAD_DATE:
return InvalidDateQuery(field_name, d.raw_value)
if d.kind is DiagnosticKind.BAD_NUMBER:
return InvalidNumberQuery(field_name, d.raw_value)
if d.kind is DiagnosticKind.TOO_DEEP:
return SearchQueryError("The search query is nested too deeply.")
if d.kind in _PATTERN_ON_KINDS:
kind_label = f" ({d.field_kind.name.lower()})" if d.field_kind else ""
return SearchQueryError(
f"Wildcard patterns are not supported for field "
f"{field_name!r}{kind_label}.",
)
if d.kind is DiagnosticKind.SINGLE_CHAR_BRACKET_RANGE:
field_label = f" for field {field_name!r}" if field_name else ""
return SearchQueryError(
f"{d.raw_value!r} looks like a bracket range{field_label}, but "
"'[' is not a wildcard character on its own. Combine it with a "
"wildcard, e.g. a trailing '*', or double-quote the value to "
"search it as literal text.",
)
logger.warning("Unmapped parse diagnostic %s: %s", d.kind, d.message)
return SearchQueryError("The search query could not be executed.")
def parse_simple_query( def parse_simple_query(
@@ -268,7 +574,7 @@ def parse_simple_query(
CJK substrings the simple analyzer can't (long whitespace-free runs are CJK substrings the simple analyzer can't (long whitespace-free runs are
dropped by remove_long). dropped by remove_long).
""" """
tokens = _simple_query_tokens(raw_query) tokens = simple_search_tokens(raw_query)
clauses: list[tuple[tantivy.Occur, tantivy.Query]] = [] clauses: list[tuple[tantivy.Occur, tantivy.Query]] = []
if tokens: if tokens:
@@ -291,23 +597,14 @@ def parse_simple_query(
) )
for token in tokens for token in tokens
] ]
simple_query = ( clauses.append((tantivy.Occur.Should, _any_of(token_queries)))
token_queries[0][1]
if len(token_queries) == 1
else tantivy.Query.boolean_query(token_queries)
)
clauses.append((tantivy.Occur.Should, simple_query))
if cjk_fields and _has_cjk(raw_query): if cjk_fields and _has_cjk(raw_query):
cjk_q = _build_cjk_query(index, raw_query, cjk_fields) cjk_q = _build_cjk_query(index, raw_query, cjk_fields)
if cjk_q is not None: if cjk_q is not None:
clauses.append((tantivy.Occur.Should, cjk_q)) clauses.append((tantivy.Occur.Should, cjk_q))
if not clauses: return _any_of(clauses)
return tantivy.Query.empty_query()
if len(clauses) == 1:
return clauses[0][1]
return tantivy.Query.boolean_query(clauses)
def parse_simple_text_highlight_query( def parse_simple_text_highlight_query(
@@ -322,13 +619,21 @@ def parse_simple_text_highlight_query(
# Strip Tantivy operator chars before tokenizing: this is a plain-text # Strip Tantivy operator chars before tokenizing: this is a plain-text
# highlight query, not a structured boolean query, so +/- are separators. # highlight query, not a structured boolean query, so +/- are separators.
tokens = _simple_query_tokens( tokens = simple_search_tokens(
regex.sub(r"[-+]", " ", raw_query, timeout=_REGEX_TIMEOUT), regex.sub(r"[-+]", " ", raw_query, timeout=_REGEX_TIMEOUT),
) )
if not tokens: if not tokens:
return tantivy.Query.empty_query() return tantivy.Query.empty_query()
return index.parse_query(" ".join(tokens), ["content"]) # Quote each token as its own phrase, escaping backslashes and embedded
# quotes. simple search tokens can carry arbitrary Tantivy syntax
# characters (`"`, `:`, `(`, `[`, `/`, ...) that the query-string parser
# would otherwise interpret as query grammar rather than literal text.
quoted_tokens = [
'"' + token.replace("\\", "\\\\").replace('"', '\\"') + '"' for token in tokens
]
return index.parse_query(" ".join(quoted_tokens), ["content"])
def parse_simple_text_query( def parse_simple_text_query(
@@ -342,7 +647,7 @@ def parse_simple_text_query(
return parse_simple_query( return parse_simple_query(
index, index,
raw_query, raw_query,
SIMPLE_SEARCH_FIELDS, _SIMPLE_SEARCH_FIELDS,
cjk_fields=_CJK_CONTENT_FIELDS, cjk_fields=_CJK_CONTENT_FIELDS,
) )
@@ -358,6 +663,6 @@ def parse_simple_title_query(
return parse_simple_query( return parse_simple_query(
index, index,
raw_query, raw_query,
TITLE_SEARCH_FIELDS, _TITLE_SEARCH_FIELDS,
cjk_fields=_CJK_TITLE_FIELDS, cjk_fields=_CJK_TITLE_FIELDS,
) )
+91
View File
@@ -0,0 +1,91 @@
from __future__ import annotations
import dataclasses
from typing import TYPE_CHECKING
from whoosh_compat import FieldKind
from whoosh_compat import FieldRegistry
from documents.search._fields import PUBLIC_FIELDS
from documents.search._tokenizer import ascii_fold
from documents.search._tokenizer import paperless_text_analyzer
from documents.search._tokenizer import stem_pattern_text
if TYPE_CHECKING:
from whoosh_compat import PatternNormalizer
_registry_cache: dict[str | None, FieldRegistry] = {}
def _identity_analyzer(text: str) -> list[str]:
"""Analyzer for KEYWORD fields indexed with the raw tokenizer (no splitting)."""
return [text]
def _fold_normalizer(text: str) -> str:
"""Wildcard/regex literal-run normalizer for fields indexed without stemming."""
return ascii_fold(text.lower())
def _make_pattern_normalizer(language: str | None) -> PatternNormalizer:
"""Build the wildcard/regex literal-run normalizer for a search language."""
def _pattern_normalizer(text: str) -> tuple[str, ...]:
"""Normalize a literal run into the forms a term may match.
TEXT index terms go through lowercase -> ascii_fold -> stem, so a
pattern that skips stemming can never match one: "invoice*" would look
for a term starting with "invoice" while the index holds "invoic". The
run is therefore offered stemmed as well. KEYWORD fields are indexed
raw and get _fold_normalizer instead, so their patterns stay literal.
Both forms are returned, as alternatives, because neither is a prefix
of the other in general: English stemming substitutes as well as
truncates ("copy" -> "copi"), so the stem alone loses the compounds
the typed run reaches ("copyright") while the typed run alone loses
the inflections the stem reaches ("copies"). whoosh-compat ORs the
alternatives per literal run and deduplicates them, so a run the
stemmer leaves alone costs exactly the one branch it did before.
Inside a bracket class the emitter calls this once per character and
uses the answer only if it is a single one-character form; two forms
there leave the character as typed. A stemmer does not change a lone
character, so the two forms deduplicate to one and the class body is
folded as before.
"""
folded = ascii_fold(text.lower())
stemmed = stem_pattern_text(folded, language)
return (folded, stemmed)
return _pattern_normalizer
def get_field_registry(language: str | None) -> FieldRegistry:
"""Build (or return the cached) FieldRegistry for the given search language.
Cached keyed by language, rebuilt on the same trigger register_tokenizers()
uses (settings.SEARCH_LANGUAGE change). A fresh call with a new language
builds and caches a new registry rather than mutating the old one.
"""
if language in _registry_cache:
return _registry_cache[language]
text_analyzer = paperless_text_analyzer(language).analyze
pattern_normalizer = _make_pattern_normalizer(language)
specs = [
dataclasses.replace(
field,
analyzer=_identity_analyzer
if field.kind is FieldKind.KEYWORD
else text_analyzer,
pattern_normalizer=_fold_normalizer
if field.kind is FieldKind.KEYWORD
else pattern_normalizer,
)
for field in PUBLIC_FIELDS
]
registry = FieldRegistry(specs)
_registry_cache[language] = registry
return registry
+222 -83
View File
@@ -1,14 +1,19 @@
from __future__ import annotations from __future__ import annotations
import hashlib
import json import json
import logging import logging
import shutil import shutil
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from typing import Final from typing import Final
from typing import NamedTuple
from typing import cast from typing import cast
import tantivy import tantivy
from django.conf import settings from django.conf import settings
from whoosh_compat import FieldKind
from documents.search._fields import PUBLIC_FIELDS
if TYPE_CHECKING: if TYPE_CHECKING:
from pathlib import Path from pathlib import Path
@@ -16,7 +21,185 @@ if TYPE_CHECKING:
logger = logging.getLogger("paperless.search") logger = logging.getLogger("paperless.search")
# v1 - Initial tantivy schema format # v1 - Initial tantivy schema format
SCHEMA_VERSION: Final[int] = 1 # v2 - build_schema() derived from PUBLIC_FIELDS, changing the field declaration
# order, and the write-only correspondent/document_type/storage_path/tag id
# columns dropped. tantivy compares schemas by ordered field list, so an
# index built by v1 rejects every write against the v2 schema.
SCHEMA_VERSION: Final[int] = 2
class FieldDescriptor(NamedTuple):
"""One tantivy field, in declaration order.
The descriptor vocabulary is paperless', not tantivy-py's: it is both the
input to the SchemaBuilder and the input to schema_fingerprint(), so the
persisted fingerprint cannot move under a tantivy-py upgrade.
"""
name: str
kind: str
stored: bool
indexed: bool
fast: bool
tokenizer: str | None
# (schema kind, tokenizer) for the FieldKind -> FieldDescriptor mapping that
# doesn't need special-casing. JSON is handled separately below since it can
# emit a second, synthetic descriptor.
_KIND_TABLE: Final[dict[FieldKind, tuple[str, str | None]]] = {
FieldKind.TEXT: ("text", "paperless_text"),
FieldKind.KEYWORD: ("text", "raw"),
FieldKind.U64: ("u64", None),
FieldKind.DATE: ("date", None),
FieldKind.DATETIME: ("date", None),
}
# Kinds whose fast-field flag follows FieldSpec.fast rather than always False.
_FAST_FROM_FIELD: Final[frozenset[FieldKind]] = frozenset(
{FieldKind.U64, FieldKind.DATE, FieldKind.DATETIME},
)
def _public_field_descriptors() -> list[FieldDescriptor]:
"""Descriptors for the query-visible fields declared in PUBLIC_FIELDS."""
descriptors: list[FieldDescriptor] = []
for field in PUBLIC_FIELDS:
if field.kind is FieldKind.JSON:
descriptors.append(
FieldDescriptor(
field.name,
"json",
stored=True,
indexed=True,
fast=False,
tokenizer="paperless_text",
),
)
if field.name == "notes":
# Plain-text companion for snippet generation: tantivy's
# SnippetGenerator does not support JSON fields. Schema-only,
# no query-syntax meaning, not in PUBLIC_FIELDS.
descriptors.append(
FieldDescriptor(
"notes_text",
"text",
stored=True,
indexed=True,
fast=False,
tokenizer="paperless_text",
),
)
continue
schema_kind, tokenizer = _KIND_TABLE[field.kind]
descriptors.append(
FieldDescriptor(
field.name,
schema_kind,
stored=True,
indexed=True,
fast=field.fast if field.kind in _FAST_FROM_FIELD else False,
tokenizer=tokenizer,
),
)
return descriptors
def field_descriptors() -> list[FieldDescriptor]:
"""Every field of the document index, in the order tantivy declares them.
tantivy compares schemas by *ordered* field list, so the order here is
part of the on-disk contract: schema_fingerprint() hashes it and
needs_rebuild() acts on the result.
"""
return [
FieldDescriptor(
"id",
"u64",
stored=True,
indexed=True,
fast=True,
tokenizer=None,
),
*_public_field_descriptors(),
# Shadow sort fields - fast, not stored
*(
FieldDescriptor(
name,
"text",
stored=False,
indexed=True,
fast=True,
tokenizer="simple_analyzer",
)
for name in ("title_sort", "correspondent_sort", "type_sort")
),
# CJK support - not stored, indexed only
*(
FieldDescriptor(
name,
"text",
stored=False,
indexed=True,
fast=False,
tokenizer="bigram_analyzer",
)
for name in (
"bigram_content",
"bigram_title",
"bigram_correspondent",
"bigram_document_type",
"bigram_tag",
)
),
# Simple substring search support for title/content - not stored,
# indexed only
*(
FieldDescriptor(
name,
"text",
stored=False,
indexed=True,
fast=False,
tokenizer="simple_search_analyzer",
)
for name in ("simple_title", "simple_content")
),
# Autocomplete prefix scan via terms_with_prefix, which walks the
# field's term dictionary - so the field must be indexed (term dict),
# not stored. The stored value is never read back, so storing it only
# wastes space.
FieldDescriptor(
"autocomplete_word",
"text",
stored=False,
indexed=True,
fast=False,
tokenizer="raw",
),
# Permission filter columns, read by build_permission_filter.
*(
FieldDescriptor(
name,
"u64",
stored=False,
indexed=True,
fast=True,
tokenizer=None,
)
for name in ("owner_id", "viewer_id", "viewer_group_id")
),
]
def schema_fingerprint() -> str:
"""Hash of the field descriptors, stamped into .index_settings.json.
Changes whenever a field is added, removed, retyped, re-optioned or
reordered, so an index built from a different schema shape is detected
even when SCHEMA_VERSION was not bumped.
"""
payload = json.dumps([list(descriptor) for descriptor in field_descriptors()])
return hashlib.blake2b(payload.encode()).hexdigest()
def build_schema() -> tantivy.Schema: def build_schema() -> tantivy.Schema:
@@ -32,85 +215,37 @@ def build_schema() -> tantivy.Schema:
""" """
sb = tantivy.SchemaBuilder() sb = tantivy.SchemaBuilder()
sb.add_unsigned_field("id", stored=True, indexed=True, fast=True) for descriptor in field_descriptors():
sb.add_text_field("checksum", stored=True, tokenizer_name="raw") if descriptor.kind == "text":
sb.add_text_field(
for field in ( descriptor.name,
"title", stored=descriptor.stored,
"correspondent", fast=descriptor.fast,
"document_type", tokenizer_name=cast("str", descriptor.tokenizer),
"storage_path", )
"original_filename", elif descriptor.kind == "json":
"content", sb.add_json_field(
): descriptor.name,
sb.add_text_field(field, stored=True, tokenizer_name="paperless_text") stored=descriptor.stored,
fast=descriptor.fast,
# Shadow sort fields - fast, not stored/indexed tokenizer_name=cast("str", descriptor.tokenizer),
for field in ("title_sort", "correspondent_sort", "type_sort"): )
sb.add_text_field( elif descriptor.kind == "u64":
field, sb.add_unsigned_field(
stored=False, descriptor.name,
tokenizer_name="simple_analyzer", stored=descriptor.stored,
fast=True, indexed=descriptor.indexed,
) fast=descriptor.fast,
)
# CJK support - not stored, indexed only elif descriptor.kind == "date":
sb.add_text_field("bigram_content", stored=False, tokenizer_name="bigram_analyzer") sb.add_date_field(
sb.add_text_field("bigram_title", stored=False, tokenizer_name="bigram_analyzer") descriptor.name,
sb.add_text_field( stored=descriptor.stored,
"bigram_correspondent", indexed=descriptor.indexed,
stored=False, fast=descriptor.fast,
tokenizer_name="bigram_analyzer", )
) else:
sb.add_text_field( raise ValueError(f"Unknown schema field kind: {descriptor.kind}")
"bigram_document_type",
stored=False,
tokenizer_name="bigram_analyzer",
)
sb.add_text_field("bigram_tag", stored=False, tokenizer_name="bigram_analyzer")
# Simple substring search support for title/content - not stored, indexed only
sb.add_text_field(
"simple_title",
stored=False,
tokenizer_name="simple_search_analyzer",
)
sb.add_text_field(
"simple_content",
stored=False,
tokenizer_name="simple_search_analyzer",
)
# Autocomplete prefix scan via terms_with_prefix, which walks the field's
# term dictionary - so the field must be indexed (term dict), not stored.
# The stored value is never read back, so storing it only wastes space.
sb.add_text_field("autocomplete_word", stored=False, tokenizer_name="raw")
sb.add_text_field("tag", stored=True, tokenizer_name="paperless_text")
# JSON fields — structured queries: notes.user:alice, custom_fields.name:invoice
sb.add_json_field("notes", stored=True, tokenizer_name="paperless_text")
# Plain-text companion for notes — tantivy's SnippetGenerator does not support
# JSON fields, so highlights require a text field with the same content.
sb.add_text_field("notes_text", stored=True, tokenizer_name="paperless_text")
sb.add_json_field("custom_fields", stored=True, tokenizer_name="paperless_text")
for field in (
"correspondent_id",
"document_type_id",
"storage_path_id",
"tag_id",
"owner_id",
"viewer_id",
"viewer_group_id",
):
sb.add_unsigned_field(field, stored=False, indexed=True, fast=True)
for field in ("created", "modified", "added"):
sb.add_date_field(field, stored=True, indexed=True, fast=True)
for field in ("asn", "page_count", "num_notes"):
sb.add_unsigned_field(field, stored=True, indexed=True, fast=True)
return sb.build() return sb.build()
@@ -119,9 +254,9 @@ def needs_rebuild(index_dir: Path) -> bool:
""" """
Check if the search index needs rebuilding. Check if the search index needs rebuilding.
Reads .index_settings.json to compare the stored schema version and Reads .index_settings.json to compare the stored schema version, search
search language against the current configuration. Returns True if the language and schema fingerprint against the current configuration. Returns
file is missing, unparsable, or either value mismatches. True if the file is missing, unparsable, or any value mismatches.
Args: Args:
index_dir: Path to the search index directory index_dir: Path to the search index directory
@@ -140,6 +275,9 @@ def needs_rebuild(index_dir: Path) -> bool:
if "language" not in data or data["language"] != settings.SEARCH_LANGUAGE: if "language" not in data or data["language"] != settings.SEARCH_LANGUAGE:
logger.info("Search index language changed - rebuilding.") logger.info("Search index language changed - rebuilding.")
return True return True
if data.get("schema_fingerprint") != schema_fingerprint():
logger.info("Search index schema fingerprint mismatch - rebuilding.")
return True
except ValueError: except ValueError:
return True return True
return False return False
@@ -170,6 +308,7 @@ def _write_sentinels(index_dir: Path) -> None:
{ {
"schema_version": SCHEMA_VERSION, "schema_version": SCHEMA_VERSION,
"language": settings.SEARCH_LANGUAGE, "language": settings.SEARCH_LANGUAGE,
"schema_fingerprint": schema_fingerprint(),
}, },
), ),
) )
+51 -2
View File
@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import logging import logging
from functools import cache
from typing import Final from typing import Final
import tantivy import tantivy
@@ -71,7 +72,7 @@ def register_tokenizers(index: tantivy.Index, language: str | None) -> None:
use fast=True and Tantivy requires fast-field tokenizers to exist use fast=True and Tantivy requires fast-field tokenizers to exist
even for documents that omit those fields. even for documents that omit those fields.
""" """
index.register_tokenizer("paperless_text", _paperless_text(language)) index.register_tokenizer("paperless_text", paperless_text_analyzer(language))
index.register_tokenizer("simple_analyzer", _simple_analyzer()) index.register_tokenizer("simple_analyzer", _simple_analyzer())
index.register_tokenizer("bigram_analyzer", _bigram_analyzer()) index.register_tokenizer("bigram_analyzer", _bigram_analyzer())
index.register_tokenizer("simple_search_analyzer", _simple_search_analyzer()) index.register_tokenizer("simple_search_analyzer", _simple_search_analyzer())
@@ -79,7 +80,7 @@ def register_tokenizers(index: tantivy.Index, language: str | None) -> None:
index.register_fast_field_tokenizer("simple_analyzer", _simple_analyzer()) index.register_fast_field_tokenizer("simple_analyzer", _simple_analyzer())
def _paperless_text(language: str | None) -> tantivy.TextAnalyzer: def paperless_text_analyzer(language: str | None) -> tantivy.TextAnalyzer:
"""Main full-text tokenizer for content, title, etc: simple -> remove_long(129) -> lowercase -> ascii_fold [-> stemmer]""" """Main full-text tokenizer for content, title, etc: simple -> remove_long(129) -> lowercase -> ascii_fold [-> stemmer]"""
builder = ( builder = (
tantivy.TextAnalyzerBuilder(tantivy.Tokenizer.simple()) tantivy.TextAnalyzerBuilder(tantivy.Tokenizer.simple())
@@ -100,6 +101,54 @@ def _paperless_text(language: str | None) -> tantivy.TextAnalyzer:
return builder.build() return builder.build()
@cache
def _pattern_stemmer(language: str | None) -> tantivy.TextAnalyzer | None:
"""The stemming tail of paperless_text_analyzer, over a whole literal run.
Same language gate and same Snowball stemmer paperless_text_analyzer
applies at index time, so query patterns follow SEARCH_LANGUAGE. Returns
None when that gate disables stemming; paperless_text_analyzer already
warns about an unsupported language, so this stays quiet.
The raw tokenizer keeps the run whole (a wildcard literal is a fragment,
not necessarily a word), and remove_long is kept so an over-long run is
treated the same way the index treats it.
"""
if not language:
return None
tantivy_lang = _LANGUAGE_MAP.get(language.lower())
if tantivy_lang is None:
return None
return (
tantivy.TextAnalyzerBuilder(tantivy.Tokenizer.raw())
.filter(tantivy.Filter.remove_long(_TOKEN_REMOVE_LONG_LIMIT))
.filter(tantivy.Filter.stemmer(tantivy_lang))
.build()
)
def stem_pattern_text(text: str, language: str | None) -> str:
"""Stem an already lowercased/ascii-folded run the way index terms are.
Returns text unchanged when stemming is disabled for language, and also
when the stem step does not yield exactly one token: remove_long drops a run
past the length limit, leaving no stem to substitute. Falling back to the
text as typed is the safe direction for a pattern prefix, since it can only
be as narrow as it was before stemming was considered.
The raw tokenizer emits one token whatever the input and the stemmer is
1-to-1, so only the zero-token case can fire today; the guard covers both
counts so a tokenizer change cannot turn this into an IndexError.
"""
analyzer = _pattern_stemmer(language)
if analyzer is None:
return text
tokens = analyzer.analyze(text)
if len(tokens) != 1:
return text
return tokens[0]
def _simple_analyzer() -> tantivy.TextAnalyzer: def _simple_analyzer() -> tantivy.TextAnalyzer:
"""Tokenizer for shadow sort fields (title_sort, correspondent_sort, type_sort): simple -> lowercase -> ascii_fold.""" """Tokenizer for shadow sort fields (title_sort, correspondent_sort, type_sort): simple -> lowercase -> ascii_fold."""
return ( return (

Some files were not shown because too many files have changed in this diff Show More