Compare commits

..
Author SHA1 Message Date
Trenton H 4f777d7438 Feature: propagate resolved secrets to interactive container shells (#14254)
docker exec bash bypasses s6, so *_FILE secrets resolved by init-env-file
into /run/s6/container_environment aren't visible even though s6 services
and management wrappers already see them via with-contenv.

Add /etc/profile.d/contenv.sh, sourced by both login and non-login
interactive bash shells, to export those resolved values into the shell.
2026-09-24 13:07:41 -07:00
Trenton H 2a44d8b5ba Fix: During a move to the trash directory, attempt to copy metadata, but don't let it fail the move (#14250) 2026-09-23 21:51:22 -07:00
Trenton H abf5050ea7 Fix: convert file mtime to the configured time zone directly (#14249)
The `created` date fallback derived a naive datetime from a file's
mtime using the OS-local zone, then labeled it as the configured
TIME_ZONE without converting. When the OS-local zone and TIME_ZONE
disagree, or when the C library can't resolve zoneinfo at all (as in
some sandboxed environments, where it silently falls back to UTC),
the resulting date can land on the wrong calendar day.

Convert the timestamp directly into the target zone with `tz=` on
fromtimestamp() instead of a naive conversion plus make_aware().
2026-09-23 15:00:48 -07:00
GitHub Actions 091ddf7c45 Auto translate strings 2026-09-23 19:13:55 +00:00
shamoon c9f7f2cfbe Fix: ensure documentDeleted subscription is discarded (#14247) 2026-09-23 12:12:26 -07:00
Trenton H b457610ffb Chore: Fix bugs in the test suite (#14244)
* Fix: redirect SHARE_LINK_BUNDLE_DIR to the test temp layout instead of the real media directory

* Fix: include f_to in test_filters subTest labels so each of the 8 cases reports distinctly

* Fix: run the post_consume error-log assertion after the raising call and match the actual paperless_mail logger name

* Fix: rename the blank-password workflow test to match its behavior and add a real wrong-password-fails test

* Fix: use the created social account's actual pk and remove an accidental tuple wrapping the mock provider

* Fix: assert against the created documents' actual pks instead of hardcoded 1 and 2

* Fix: assert test_compression actually produces a valid LZMA-compressed zip

* Fix: clear os.environ when patching PAPERLESS_ADMIN_* vars so a host-set value can't leak into the no-user test

* Fix: restore MIDDLEWARE, AUTHENTICATION_BACKENDS and REST_FRAMEWORK auth classes after each remote-user settings test instead of leaking the mutation into later tests

* Fix: use a guaranteed-nonexistent temp path instead of hardcoded /tmp/foo/bar in test_export_target_not_exists
2026-09-23 19:11:58 +00:00
GitHub Actions 969c2ea0e2 Auto translate strings 2026-09-23 19:00:47 +00:00
shamoon 31b806a285 Fix: ensure bulk operations are checked against version root (#14246) 2026-09-23 18:59:27 +00:00
GitHub Actions 99851b418c Auto translate strings 2026-09-23 18:46:43 +00:00
e34eda07bb Enhancement: support separate embedding API key (#14067)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-09-23 11:44:37 -07:00
GitHub Actions 793459b416 Auto translate strings 2026-09-23 18:22:05 +00:00
shamoon 04297fd02c Enhancement: support passthrough extra params for LLMs (#14202) 2026-09-23 18:20:43 +00:00
dependabot[bot] 1b277dd8e1 Chore(deps): Bump autobahn in the uv group across 1 directory (#14231)
Bumps the uv group with 1 update in the / directory: [autobahn](https://github.com/crossbario/autobahn-python).


Updates `autobahn` from 25.12.2 to 26.7.1
- [Release notes](https://github.com/crossbario/autobahn-python/releases)
- [Changelog](https://github.com/crossbario/autobahn-python/blob/master/docs/changelog.rst)
- [Commits](https://github.com/crossbario/autobahn-python/compare/v25.12.2...v26.7.1)

---
updated-dependencies:
- dependency-name: autobahn
  dependency-version: 26.7.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-23 15:33:01 +00:00
shamoon 3c20abeb4c Fix: indexing after document-added workflows signal (#14242) 2026-09-23 14:50:48 +00:00
GitHub Actions b11f1f8459 Auto translate strings 2026-09-23 03:19:06 +00:00
Trenton H 7424e7ce0b Fix: Record full tag and custom field lists in bulk edit audit log (#14236)
Snapshot only the edited field, before and after the operation, gathering tags
and custom field instances into sorted id lists per document (empty when there
are none).
2026-09-23 03:17:40 +00:00
shamoon a53a3d3769 Chore: update pikepdf for ocrmypdf requirement (#14235) 2026-09-22 19:01:21 -07:00
Trenton H 15b73b890c Chore: Ban cross-app imports of documents.tests (#14224)
Nothing outside the documents app imports documents.tests any more.

Ruff now bans the module everywhere except src/documents/tests, where in-app imports remain fine.
2026-09-22 08:02:18 -07:00
Trenton H 02d355061f Chore: Move helpers that other test modules imported out of test files (#14223)
The mail message and mailbox builders, the fake libmagic and the classifier preprocessor stub lived inside test_mail.py and test_classifier.py, so other test modules imported them by importing a test module.

They now live in helpers modules beside the tests that use them.
2026-09-22 08:02:18 -07:00
Trenton H d8b5b4d447 Chore: Stop the live-service retry helper sleeping after a success (#14222)
util_call_with_backoff made every call wait 20 seconds even when the first attempt succeeded

The helper now sleeps only after a failure that has another attempt left.
2026-09-22 08:02:18 -07:00
Trenton H 03ac4aed7e Chore: Move the cross-app test helpers into the shared layer and add a progress fixture (#14221)
Test modules in paperless, paperless_mail and documents imported filesystem assertions, the migration test base, the retry helper and the streaming-response reader out of documents/tests/utils.py, which kept each app's tests coupled to another app's test package.

They now live in paperless_testing, and the progress manager fake is renamed FakeProgressManager and now subclasses the real ProgressManager, overriding only the transport, so the payload it records is built by the production code. The twenty places that patched documents.tasks.ProgressManager by hand now use a fake_progress_manager fixture.
2026-09-22 08:02:17 -07:00
Trenton H 90d23bad9c Chore: Give the search index directory one owner in the tests (#14217)
Two fixtures created a temporary index directory and pointed INDEX_DIR at it, and paperless_dirs did the same, so a test that requested more than one got whichever assignment ran last. The search conftest no longer defines its own index_dir fixture, the tests that took it read paperless_dirs.index_dir instead, and _search_index is now a thin wrapper that requests paperless_dirs. The fixture that yields a Document is renamed from indexed_document to searchable_document so it no longer differs by one character from the index_document factory next to it.
2026-09-22 08:02:17 -07:00
Trenton H e4367b5648 Chore: Delete the in-memory Tantivy backend and run the search tests on disk (#14216)
TantivyBackend(path=None) built an in-memory index that is not used by production ever used. The backend now requires a path, the open, write-batch and rebuild branches are gone, and the shared backend fixture and the fulltext similar-documents fixture build a real index under the per-test directory layout.
2026-09-22 08:02:16 -07:00
Trenton H cb85441c2f Chore: Decrease backend test suite time (a little) (#14213)
* Chore: Speed up test setup by hashing passwords with MD5 and batching index writes

Don't use Django's default PBKDF2, about 600 ms per use and 110 uses across the suite. Switches to MD5 instead.

Also fixes a test that didn't batch update the search index

* Chore: Stop the invalid webhook params test from waiting on a Celery broker

test_workflow_webhook_action_url_invalid_params_headers left send_webhook.apply_async unpatched, so it tried to actually enqueu and waited for the timeout.
2026-09-21 20:17:40 +00:00
Trenton H cceaa559d4 Chore: Rename sample directory fixtures and drop unused parser ones (#14215)
Nineteen single-file fixtures in the parsers conftest had no consumers anywhere in the test tree.

Two fixtures were both called samples_dir and resolved one directory apart They are now document_samples_dir and parser_samples_dir
2026-09-21 12:59:32 -07:00
GitHub Actions a748d4c64f Auto translate strings 2026-09-21 19:01:49 +00:00
shamoon 452ed005bd Fix: handle legacy bulk edit page range with missing page_count (#14212) 2026-09-21 19:00:14 +00:00
Trenton H 40058ff7d5 Chore: Give the test suite a larger regex timeout (#14211)
Maybe the random ordering sometimes causes heavier tests to run alongside timed regex ones?
2026-09-21 11:38:54 -07:00
Trenton H f502cd5e34 Chore: Add shared helpers for granting test permissions and use them (#14200) 2026-09-21 10:57:14 -07:00
Trenton H 99ce6b5db3 Chore: Share the API client fixtures and build test users cheaply (#14199) 2026-09-21 10:57:14 -07:00
Trenton H 2d955e9697 Chore: Randomize test order and seed Faker per run (#14173)
Enables pytest-randomly, which has sat commented out in pyproject.toml
since the Pytest 9 upgrade. Tests now run in a different order every
session, so a test cannot quietly depend on another having run first.
2026-09-21 10:57:13 -07:00
Trenton H f440e8d33c Chore: Move unittest directory setup onto the shared fixture (#14172)
The unittest side of the suite built its temp directory tree with
tempfile.mkdtemp and a manually enabled override_settings, cleaned up only if
tearDown ran. That is now gone. DirectoriesMixin lives alongside the layout it
bridges and does nothing but hand the paperless_dirs fixture to TestCase
subclasses as self.dirs, so both halves of the suite get the same twelve
settings, the same directory shapes and cleanup owned by tmp_path.

The mixin moves to paperless_testing.dirs rather than staying in the documents
test utilities, because modules in paperless and paperless_mail import it
across the app boundary. The thirty-eight consuming modules change only their
import line; self.dirs.scratch_dir and its siblings keep working.
2026-09-21 10:57:12 -07:00
Trenton H 4c264651e8 Chore: State the test directory layout in one place (#14171)
The temp directory layout used by the tests was written out four separate
times: once in the documents conftest, once in the paperless checks tests,
once in a fixture local to the NFC upload tests, and once in the helper
behind the old paperless_environment context manager. Each copy covered a
different subset of the settings, so which directories a test actually got
depended on which copy it happened to reach.

The layout now lives in paperless_testing.dirs. build_paperless_dirs owns
where things go and creates them, dirs_settings maps them onto Django
setting names and is pure, and a paperless_dirs fixture in the root conftest
applies that mapping through pytest-django's settings fixture so every app
can reach it. Tests that need a second environment part way through a test
body use the paperless_environment context manager from the same module,
which expresses the identical layout through override_settings. The three
redundant implementations and the old media settings fixture are gone, and
their consumers now take paperless_dirs.
2026-09-21 10:57:11 -07:00
Trenton H 48d97b78bb Chore: Move model factories to the shared test layer (#14170)
The model factories lived in the documents test package, but three other
apps needed them. The AI, mail and testing suites all reached across an app
boundary to import from documents.tests.factories, which made a private test
package into a shared dependency.

The factories now live in the shared testing package, where cross-app use is
the intended use.
2026-09-21 10:57:11 -07:00
Trenton H 659a0cb2ef Chore: Add a shared test support layer (#14168)
All four Django apps have test code in common, but the only place to put it
was the documents app's own tests package, so paperless, paperless_ai and
paperless_mail each reached across an app boundary to import helpers and
relied on fixtures that were only defined for the documents test path.

This adds a root src/conftest.py holding the fixtures every app needs and an
new src/paperless_testing package for shared helpers a test names
2026-09-21 10:57:11 -07:00
Trenton H d02d1e1711 Chore: Remove unused test helpers (#14166) 2026-09-21 10:57:10 -07:00
dependabot[bot]andstumpylog 3b41810e7b Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 2 updates (#14133)
* Chore(deps): Bump the pre-commit-dependencies group across 1 directory with 2 updates

Bumps the pre-commit-dependencies group with 2 updates in the / directory: [https://github.com/astral-sh/ruff-pre-commit](https://github.com/astral-sh/ruff-pre-commit) and [https://github.com/tox-dev/pyproject-fmt](https://github.com/tox-dev/pyproject-fmt).


Updates `https://github.com/astral-sh/ruff-pre-commit` from v0.16.5 to 0.16.7
- [Release notes](https://github.com/astral-sh/ruff-pre-commit/releases)
- [Commits](https://github.com/astral-sh/ruff-pre-commit/compare/v0.16.5...v0.16.7)

Updates `https://github.com/tox-dev/pyproject-fmt` from v2.28.1 to 2.29.4
- [Release notes](https://github.com/tox-dev/pyproject-fmt/releases)
- [Commits](https://github.com/tox-dev/pyproject-fmt/compare/v2.28.1...v2.29.4)

---
updated-dependencies:
- dependency-name: https://github.com/astral-sh/ruff-pre-commit
  dependency-version: 0.16.6
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
- dependency-name: https://github.com/tox-dev/pyproject-fmt
  dependency-version: 2.29.4
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Runs new formatting

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-09-21 07:49:12 -07:00
Zhiyuan Zhengandzhzy0077 12314fcaa8 Fix: ignore invalid EXIF orientation when generating image archives (#14203)
Images with an out-of-spec EXIF orientation value (e.g. 0) fail
conversion with img2pdf.ExifOrientationError, which aborts archive
generation and, during consumption with OCR disabled, fails the whole
document.

Pass rotation=img2pdf.Rotation.ifvalid so invalid orientation values
are ignored while valid values (1, 3, 6, 8) are still applied.

Co-authored-by: zhzy0077 <zhzy0077@users.noreply.github.com>
2026-09-21 07:21:24 -07:00
github-actions[bot] 8705bd510a Changelog v3.2.1 - GHA (#14198)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-20 15:00:56 -07:00
Trenton H 7575d60782 Bump version to 3.2.1 2026-09-20 14:29:37 -07:00
github-actions[bot]andCrowdin Bot 8b02a23b86 New Crowdin translations by GitHub Action (#14179)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-09-20 21:24:50 +00:00
dependabot[bot] 87d157096e Chore(deps): Bump anyio in the uv group across 1 directory (#14175)
Bumps the uv group with 1 update in the / directory: [anyio](https://github.com/agronholm/anyio).


Updates `anyio` from 4.12.1 to 4.14.2
- [Release notes](https://github.com/agronholm/anyio/releases)
- [Commits](https://github.com/agronholm/anyio/compare/4.12.1...4.14.2)

---
updated-dependencies:
- dependency-name: anyio
  dependency-version: 4.14.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-20 13:21:27 +00:00
Bitfoo 54e332d259 Fix: only pass --conf to flower when flowerconfig.py exists (#14182)
* Fix: only pass --conf to flower when flowerconfig.py exists

The service passes --conf=${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py
unconditionally, but the image does not ship that file. flower 2.0.1 tolerated
this because it ignored a missing config whose basename matched its own default
name; 2.1.0 dropped that basename() call (mher/flower#1391) and now raises
FileNotFoundError, so flower crash-loops and never binds 5555.

Pass the flag only when the file is present.
2026-09-19 21:10:52 -07:00
164 changed files with 3412 additions and 5184 deletions
+2
View File
@@ -15,6 +15,8 @@
# Test related
**/.pytest_cache
**/tests
src/paperless_testing
src/conftest.py
**/*.spec.ts
**/htmlcov
# Local folders
+2 -2
View File
@@ -50,12 +50,12 @@ repos:
- 'prettier-plugin-organize-imports@4.3.0'
# Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.5
rev: v0.16.7
hooks:
- id: ruff-check
- id: ruff-format
- repo: https://github.com/tox-dev/pyproject-fmt
rev: "v2.28.1"
rev: "v2.29.4"
hooks:
- id: pyproject-fmt
additional_dependencies: [tomli]
+3 -1
View File
@@ -171,7 +171,9 @@ RUN set -eux \
&& cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \
&& echo "Cleaning up image layer" \
&& rm --force --verbose *.deb \
&& rm --recursive --force --verbose /var/lib/apt/lists/*
&& rm --recursive --force --verbose /var/lib/apt/lists/* \
&& echo "Configuring interactive shells to source the s6 container environment" \
&& echo '. /etc/profile.d/contenv.sh' >> /etc/bash.bashrc
WORKDIR /usr/src/paperless/src/
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
# Source s6 container environment for interactive shells.
# Ensures variables resolved from *_FILE secret injection are visible
# when using 'docker exec bash'. Does not affect s6 services (those
# use with-contenv directly). Has no effect in non-container contexts
# because the directory will not exist.
# Note: sh/dash shells opened via 'docker exec sh' are not covered;
# only bash-based sessions benefit from this file.
_pngx_contenv="/run/s6/container_environment"
if [ -d "${_pngx_contenv}" ]; then
for _pngx_f in "${_pngx_contenv}"/*; do
[ -f "${_pngx_f}" ] || continue
_pngx_name=$(basename "${_pngx_f}")
_pngx_val=$(cat "${_pngx_f}")
export "${_pngx_name}=${_pngx_val}"
done
fi
unset _pngx_contenv _pngx_f _pngx_name _pngx_val
@@ -2,6 +2,7 @@
# shellcheck shell=bash
declare -r log_prefix="[svc-flower]"
declare -r flower_config="${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py"
echo "${log_prefix} Checking if we should start flower..."
@@ -9,12 +10,20 @@ if [[ -n "${PAPERLESS_ENABLE_FLOWER}" ]]; then
# Small delay to allow celery to be up first
echo "${log_prefix} Starting flower in 5s"
sleep 5
cd ${PAPERLESS_SRC_DIR}
cd "${PAPERLESS_SRC_DIR}" || exit 1
# Only pass --conf if the file is actually there. The image does not ship one, and
# flower >= 2.1.0 exits with FileNotFoundError when an explicitly given --conf path
# does not exist (mher/flower#1391). Earlier versions silently ignored it.
declare -a conf_args=()
if [[ -f "${flower_config}" ]]; then
conf_args=(--conf="${flower_config}")
fi
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec /usr/local/bin/celery --app paperless flower --conf=${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py
exec /usr/local/bin/celery --app paperless flower "${conf_args[@]}"
else
exec s6-setuidgid paperless /usr/local/bin/celery --app paperless flower --conf=${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py
exec s6-setuidgid paperless /usr/local/bin/celery --app paperless flower "${conf_args[@]}"
fi
else
+5 -2
View File
@@ -153,8 +153,11 @@ in similar existing documents, and the document chat can retrieve relevant conte
Enable it by setting
[`PAPERLESS_AI_LLM_EMBEDDING_BACKEND`](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_BACKEND)
(`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). The index is only
built when AI is enabled **and** an embedding backend is set.
(`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). By default, the main
LLM API key and endpoint are used, but an optional embedding-specific[API key](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_API_KEY)
and [endpoint](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) can be configured.
The index is only built when AI is enabled **and** an embedding backend is set.
The index is updated automatically on a schedule controlled by
[`PAPERLESS_LLM_INDEX_TASK_CRON`](configuration.md#PAPERLESS_LLM_INDEX_TASK_CRON) (daily by
+25
View File
@@ -1,5 +1,30 @@
# Changelog
## paperless-ngx 3.2.1
### Bug Fixes
- Fix: only pass --conf to flower when flowerconfig.py exists [@bitfoo1](https://github.com/bitfoo1) ([#14182](https://github.com/paperless-ngx/paperless-ngx/pull/14182))
- Fix: replace stale mail-fetch overlap check with a self-expiring lock [@stumpylog](https://github.com/stumpylog) ([#14189](https://github.com/paperless-ngx/paperless-ngx/pull/14189))
- Fix: bump ocrmypdf to 17.12 to pick up the ligature text-layer fix [@stumpylog](https://github.com/stumpylog) ([#14190](https://github.com/paperless-ngx/paperless-ngx/pull/14190))
- Fix: rebuild the search index automatically when it is missing Tantivy files [@stumpylog](https://github.com/stumpylog) ([#14180](https://github.com/paperless-ngx/paperless-ngx/pull/14180))
### Dependencies
- Chore(deps): Bump anyio from 4.12.1 to 4.14.2 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#14175](https://github.com/paperless-ngx/paperless-ngx/pull/14175))
### All App Changes
<details>
<summary>4 changes</summary>
- Chore(deps): Bump anyio from 4.12.1 to 4.14.2 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#14175](https://github.com/paperless-ngx/paperless-ngx/pull/14175))
- Fix: replace stale mail-fetch overlap check with a self-expiring lock [@stumpylog](https://github.com/stumpylog) ([#14189](https://github.com/paperless-ngx/paperless-ngx/pull/14189))
- Fix: bump ocrmypdf to 17.12 to pick up the ligature text-layer fix [@stumpylog](https://github.com/stumpylog) ([#14190](https://github.com/paperless-ngx/paperless-ngx/pull/14190))
- Fix: rebuild the search index automatically when it is missing Tantivy files [@stumpylog](https://github.com/stumpylog) ([#14180](https://github.com/paperless-ngx/paperless-ngx/pull/14180))
</details>
## paperless-ngx 3.2.0
### Features / Enhancements
+20
View File
@@ -2133,6 +2133,13 @@ for language and resource considerations.
Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_API_KEY=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_API_KEY) {#PAPERLESS_AI_LLM_EMBEDDING_API_KEY}
: The API key to use for the embedding backend. If not supplied, embeddings use
`PAPERLESS_AI_LLM_API_KEY`.
Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) {#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT}
: The endpoint / url to use for the embedding backend. If not supplied, embeddings use
@@ -2217,6 +2224,19 @@ used with the OpenAI-compatible backend to target a custom provider or local gat
Defaults to true, which allows internal endpoints.
#### [`PAPERLESS_AI_LLM_EXTRA_PARAMS=<json>`](#PAPERLESS_AI_LLM_EXTRA_PARAMS) {#PAPERLESS_AI_LLM_EXTRA_PARAMS}
: A JSON object of extra parameters sent with every LLM request, for providers that require a parameter Paperless does not
set itself. Values here override Paperless' own, and no validation is performed. Whatever you put here is passed to the
backend as-is, so an invalid parameter will simply be rejected by your provider. For example, current OpenAI reasoning
models refuse tool calls on the chat completions API unless reasoning is off:
```
PAPERLESS_AI_LLM_EXTRA_PARAMS={"reasoning_effort": "none"}
```
Defaults to empty, which adds nothing to requests.
#### [`PAPERLESS_LLM_INDEX_TASK_CRON=<cron expression>`](#PAPERLESS_LLM_INDEX_TASK_CRON) {#PAPERLESS_LLM_INDEX_TASK_CRON}
: Configures the schedule to update the AI embeddings of text content and metadata for all documents. Only performed if
+1
View File
@@ -150,6 +150,7 @@ pnpm ng build --configuration production
is loaded as well. However, the tests rely on the default
configuration. This is not ideal. But for now, make sure no settings
except for DEBUG are overridden when testing.
- Tests run in a random order each session, so that one test cannot quietly depend on another having run first. The seed is printed at the top of the run; pass `--randomly-seed=<seed>` to replay that exact order, or `--randomly-seed=last` to repeat the previous run.
!!! note
+17 -6
View File
@@ -1,7 +1,9 @@
[project]
name = "paperless-ngx"
version = "3.2.0"
description = "A community-supported supercharged document management system: scan, index and archive all your physical documents"
version = "3.2.1"
description = """\
A community-supported supercharged document management system: scan, index and archive all your physical documents\
"""
readme = "README.md"
requires-python = ">=3.11"
classifiers = [
@@ -10,6 +12,7 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Programming Language :: Python :: 3.15",
]
# TODO: Move certain things to groups and then utilize that further
# This will allow testing to not install a webserver, mysql, etc
@@ -117,7 +120,7 @@ testing = [
"pytest-env~=1.7.0",
"pytest-httpx",
"pytest-mock~=3.15.1",
# "pytest-randomly~=4.0.1",
"pytest-randomly~=5.0.0",
"pytest-rerunfailures~=16.4",
"pytest-sugar",
"pytest-xdist~=3.8.0",
@@ -244,6 +247,10 @@ per-file-ignores."docker/wait-for-redis.py" = [
per-file-ignores."src/documents/models.py" = [
"SIM115",
]
per-file-ignores."src/documents/tests/*.py" = [
"TID251",
]
flake8-tidy-imports.banned-api."documents.tests".msg = "Shared test infrastructure lives in src/paperless_testing/."
isort.force-single-line = true
[tool.codespell]
@@ -271,9 +278,9 @@ plugins = [
]
[tool.pyrefly]
baseline = ".pyrefly-baseline.json"
python-platform = "linux"
search-path = [ "src" ]
baseline = ".pyrefly-baseline.json"
[tool.django-stubs]
django_settings_module = "paperless.settings"
@@ -326,6 +333,8 @@ PAPERLESS_CACHE_BACKEND = "django.core.cache.backends.locmem.LocMemCache"
PAPERLESS_CHANNELS_BACKEND = "channels.layers.InMemoryChannelLayer"
# I don't think anything hits this, but just in case, basically infinite
PAPERLESS_TOKEN_THROTTLE_RATE = "1000/min"
# The 0.1s production default trips on a stalled CI runner, the date parsing tests then find no dates
PAPERLESS_MATCH_REGEX_TIMEOUT_SECONDS = "5"
[tool.coverage.run]
source = [
@@ -334,13 +343,15 @@ source = [
omit = [
"*/tests/*",
"manage.py",
"paperless/wsgi.py",
"paperless/auth.py",
"paperless/wsgi.py",
"src/conftest.py",
"src/paperless_testing/*",
]
[tool.coverage.report]
exclude_also = [
"if settings.AUDIT_LOG_ENABLED:",
"if AUDIT_LOG_ENABLED:",
"if settings.AUDIT_LOG_ENABLED:",
"if TYPE_CHECKING:",
]
+35 -21
View File
@@ -9745,7 +9745,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">348</context>
<context context-type="linenumber">351</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -9760,7 +9760,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">341</context>
<context context-type="linenumber">344</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -10016,56 +10016,56 @@
<source>Reset filters / selection</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">329</context>
<context context-type="linenumber">332</context>
</context-group>
</trans-unit>
<trans-unit id="4135055128446167640" datatype="html">
<source>Open first [selected] document</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">357</context>
<context context-type="linenumber">360</context>
</context-group>
</trans-unit>
<trans-unit id="3629960544875360046" datatype="html">
<source>Previous page</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">373</context>
<context context-type="linenumber">376</context>
</context-group>
</trans-unit>
<trans-unit id="3337301694210287595" datatype="html">
<source>Next page</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">385</context>
<context context-type="linenumber">388</context>
</context-group>
</trans-unit>
<trans-unit id="2155249406916744630" datatype="html">
<source>View &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot; saved successfully.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">419</context>
<context context-type="linenumber">422</context>
</context-group>
</trans-unit>
<trans-unit id="4646273665293421938" datatype="html">
<source>Failed to save view &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot;.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">425</context>
<context context-type="linenumber">428</context>
</context-group>
</trans-unit>
<trans-unit id="6837554170707123455" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">494</context>
<context context-type="linenumber">497</context>
</context-group>
</trans-unit>
<trans-unit id="6028096992841030074" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully, but could not update visibility settings.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">500</context>
<context context-type="linenumber">503</context>
</context-group>
</trans-unit>
<trans-unit id="739880801667335279" datatype="html">
@@ -12018,81 +12018,95 @@
<context context-type="linenumber">351</context>
</context-group>
</trans-unit>
<trans-unit id="861068592166833023" datatype="html">
<source>LLM Embedding API Key</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">358</context>
</context-group>
</trans-unit>
<trans-unit id="2929108042259892948" datatype="html">
<source>Used for embeddings when set, otherwise LLM API key is used.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">360</context>
</context-group>
</trans-unit>
<trans-unit id="3554114880473286122" datatype="html">
<source>LLM Embedding Endpoint</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">358</context>
<context context-type="linenumber">366</context>
</context-group>
</trans-unit>
<trans-unit id="1044242175651289991" datatype="html">
<source>LLM Embedding Chunk Size</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">365</context>
<context context-type="linenumber">373</context>
</context-group>
</trans-unit>
<trans-unit id="7218245223139363113" datatype="html">
<source>LLM Context Size</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">372</context>
<context context-type="linenumber">380</context>
</context-group>
</trans-unit>
<trans-unit id="4234495692726214397" datatype="html">
<source>LLM Backend</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">379</context>
<context context-type="linenumber">387</context>
</context-group>
</trans-unit>
<trans-unit id="7935234833834000002" datatype="html">
<source>LLM Model</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">387</context>
<context context-type="linenumber">395</context>
</context-group>
</trans-unit>
<trans-unit id="1980550530387803165" datatype="html">
<source>LLM API Key</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">394</context>
<context context-type="linenumber">402</context>
</context-group>
</trans-unit>
<trans-unit id="6126617860376156501" datatype="html">
<source>LLM Endpoint</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">401</context>
<context context-type="linenumber">409</context>
</context-group>
</trans-unit>
<trans-unit id="6572826277249350975" datatype="html">
<source>LLM Output Language</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">408</context>
<context context-type="linenumber">416</context>
</context-group>
</trans-unit>
<trans-unit id="3284403507172415792" datatype="html">
<source>Language to use for generated AI suggestions. When unset, AI suggestions use the user&apos;s display language if explicitly set.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">412</context>
<context context-type="linenumber">420</context>
</context-group>
</trans-unit>
<trans-unit id="4493921125434706859" datatype="html">
<source>LLM Request Timeout</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">416</context>
<context context-type="linenumber">424</context>
</context-group>
</trans-unit>
<trans-unit id="483994032066441287" datatype="html">
<source>Timeout in seconds for LLM requests.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">420</context>
<context context-type="linenumber">428</context>
</context-group>
</trans-unit>
<trans-unit id="1055686627716339120" datatype="html">
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "paperless-ngx-ui",
"version": "3.2.0",
"version": "3.2.1",
"scripts": {
"preinstall": "npx only-allow pnpm",
"ng": "ng",
@@ -1,347 +0,0 @@
<div class="modal-header">
<h4 class="modal-title" id="advanced-search-dialog-title" i18n>
Advanced search
</h4>
<button
type="button"
class="btn-close"
aria-label="Close"
i18n-aria-label
(click)="cancel()"
></button>
</div>
<div class="modal-body">
@if (unreadable) {
<div class="alert alert-warning d-flex flex-wrap gap-2 align-items-center">
<div class="flex-grow-1">
<span i18n
>The current query uses syntax this editor can't show. It still works
as typed.</span
>
</div>
<button
type="button"
class="btn btn-sm btn-outline-secondary"
(click)="startOver()"
i18n
>
Start a new query
</button>
</div>
}
<ng-container
*ngTemplateOutlet="
groupTemplate;
context: { group: root, parent: null, depth: 0 }
"
></ng-container>
<div class="mt-4">
<label
class="form-label small text-muted"
for="advanced-search-preview"
i18n
>Query</label
>
<pre
id="advanced-search-preview"
class="query-preview mb-0 p-2 rounded border"
>@if (generatedQuery) {{{ generatedQuery }}} @else {<span class="text-muted fst-italic" i18n>Nothing to search for yet</span>}</pre>
</div>
</div>
<div class="modal-footer">
<button
type="button"
class="btn btn-outline-secondary"
(click)="cancel()"
i18n
>
Cancel
</button>
<button
type="button"
class="btn btn-primary"
(click)="apply()"
[disabled]="!generatedQuery"
i18n
>
Apply
</button>
</div>
<ng-template
#groupTemplate
let-group="group"
let-parent="parent"
let-depth="depth"
>
<div class="d-flex w-100 gap-2">
<div class="d-flex flex-grow-1 flex-column">
<div class="d-flex align-items-center flex-wrap">
<div
class="btn-group btn-group-xs"
role="group"
aria-label="Match"
i18n-aria-label
>
<input
type="radio"
class="btn-check"
[(ngModel)]="group.operator"
[ngModelOptions]="{ standalone: true }"
[value]="LogicalOperator.Or"
id="advancedSearchAny_{{ idFor(group) }}"
name="advancedSearchAny_{{ idFor(group) }}"
/>
<label
class="btn btn-outline-primary"
for="advancedSearchAny_{{ idFor(group) }}"
i18n
>Any</label
>
<input
type="radio"
class="btn-check"
[(ngModel)]="group.operator"
[ngModelOptions]="{ standalone: true }"
[value]="LogicalOperator.And"
id="advancedSearchAll_{{ idFor(group) }}"
name="advancedSearchAll_{{ idFor(group) }}"
/>
<label
class="btn btn-outline-primary"
for="advancedSearchAll_{{ idFor(group) }}"
i18n
>All</label
>
<input
type="radio"
class="btn-check"
[(ngModel)]="group.operator"
[ngModelOptions]="{ standalone: true }"
[value]="LogicalOperator.Not"
id="advancedSearchNot_{{ idFor(group) }}"
name="advancedSearchNot_{{ idFor(group) }}"
/>
<label
class="btn btn-outline-secondary"
for="advancedSearchNot_{{ idFor(group) }}"
i18n
>Not</label
>
</div>
<span class="small text-muted ms-2">
@switch (group.operator) {
@case (LogicalOperator.And) {
<ng-container i18n>match all of these</ng-container>
}
@case (LogicalOperator.Or) {
<ng-container i18n>match any of these</ng-container>
}
@case (LogicalOperator.Not) {
<ng-container i18n>match none of these</ng-container>
}
}
</span>
</div>
<div class="list-group list-group-flush">
@for (element of group.children; track element) {
<div class="list-group-item px-0 d-flex flex-nowrap">
@if (element.type === ElementType.Group) {
<ng-container
*ngTemplateOutlet="
groupTemplate;
context: { group: element, parent: group, depth: depth + 1 }
"
></ng-container>
} @else {
<ng-container
*ngTemplateOutlet="
atomTemplate;
context: { atom: element, parent: group }
"
></ng-container>
}
</div>
}
</div>
</div>
<div
class="btn-group-vertical align-self-start ms-2 ps-2 border-start"
role="group"
aria-label="Group actions"
i18n-aria-label
>
<button
type="button"
class="btn btn-sm btn-outline-secondary text-primary"
title="Add condition"
i18n-title
(click)="addAtom(group)"
[disabled]="group.children.length >= maxAtoms"
>
<i-bs name="node-plus"></i-bs>
</button>
<button
type="button"
class="btn btn-sm btn-outline-secondary text-primary"
title="Add group"
i18n-title
(click)="addGroup(group)"
[disabled]="depth >= maxDepth"
>
<i-bs name="braces"></i-bs>
</button>
@if (parent) {
<button
type="button"
class="btn btn-sm btn-outline-secondary text-danger"
aria-label="Remove group"
i18n-aria-label
(click)="remove(parent, group)"
>
<i-bs name="x-circle"></i-bs>
</button>
}
</div>
</div>
</ng-template>
<ng-template #atomTemplate let-atom="atom" let-parent="parent">
<div class="d-flex align-items-center gap-1 w-100">
<div class="input-group input-group-sm flex-wrap">
<select
class="form-select flex-grow-0 w-auto"
[(ngModel)]="atom.field"
[ngModelOptions]="{ standalone: true }"
(ngModelChange)="onFieldChange(atom)"
aria-label="Field"
i18n-aria-label
>
@for (fieldGroup of fieldGroups; track fieldGroup.label) {
<optgroup [label]="fieldGroup.label">
@for (field of fieldGroup.fields; track field) {
<option [ngValue]="field">{{ fieldLabels[field] }}</option>
}
</optgroup>
}
</select>
<select
class="form-select flex-grow-0 w-auto"
[(ngModel)]="atom.operator"
[ngModelOptions]="{ standalone: true }"
(ngModelChange)="onOperatorChange(atom)"
aria-label="Condition"
i18n-aria-label
>
@for (operator of operatorsFor(atom); track operator) {
<option [ngValue]="operator">
{{ operatorLabel(atom, operator) }}
</option>
}
</select>
@switch (atom.operator) {
@case (Operator.DateKeyword) {
<select
class="form-select"
[(ngModel)]="atom.value"
[ngModelOptions]="{ standalone: true }"
aria-label="Period"
i18n-aria-label
>
<option [ngValue]="''" disabled i18n>Choose a period</option>
@for (keyword of dateKeywords; track keyword) {
<option [ngValue]="keyword">
{{ dateKeywordLabels[keyword] }}
</option>
}
</select>
}
@case (Operator.WithinLast) {
<input
class="form-control amount"
type="number"
min="1"
[(ngModel)]="atom.value"
[ngModelOptions]="{ standalone: true }"
aria-label="Amount"
i18n-aria-label
/>
<select
class="form-select"
[(ngModel)]="atom.unit"
[ngModelOptions]="{ standalone: true }"
aria-label="Unit"
i18n-aria-label
>
@for (unit of dateUnits; track unit) {
<option [ngValue]="unit">{{ dateUnitLabels[unit] }}</option>
}
</select>
}
@case (Operator.Between) {
<input
class="form-control"
[type]="kindOf(atom) === FieldKind.Date ? 'date' : 'number'"
[(ngModel)]="atom.value"
[ngModelOptions]="{ standalone: true }"
aria-label="From"
i18n-aria-label
/>
<span class="input-group-text" i18n>and</span>
<input
class="form-control"
[type]="kindOf(atom) === FieldKind.Date ? 'date' : 'number'"
[(ngModel)]="atom.valueTo"
[ngModelOptions]="{ standalone: true }"
aria-label="To"
i18n-aria-label
/>
}
@default {
@if (kindOf(atom) === FieldKind.Date) {
<input
class="form-control"
type="date"
[(ngModel)]="atom.value"
[ngModelOptions]="{ standalone: true }"
aria-label="Date"
i18n-aria-label
/>
} @else if (kindOf(atom) === FieldKind.Number) {
<input
class="form-control"
type="number"
min="0"
[(ngModel)]="atom.value"
[ngModelOptions]="{ standalone: true }"
aria-label="Number"
i18n-aria-label
/>
} @else {
<input
class="form-control"
type="text"
[(ngModel)]="atom.value"
[ngModelOptions]="{ standalone: true }"
[placeholder]="placeholderFor(atom)"
aria-label="Value"
i18n-aria-label
/>
}
}
}
</div>
<button
class="btn btn-link btn-sm text-danger px-1"
type="button"
(click)="remove(parent, atom)"
aria-label="Remove condition"
i18n-aria-label
>
<i-bs name="x-circle"></i-bs>
</button>
</div>
</ng-template>
@@ -1,11 +0,0 @@
.query-preview {
font-size: 0.8125rem;
white-space: pre-wrap;
word-break: break-word;
background-color: var(--pngx-bg-darker);
border-color: var(--bs-border-color) !important;
}
.input-group .amount {
flex: 0 1 5rem;
}
@@ -1,186 +0,0 @@
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { allIcons, NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import {
AdvancedSearchDateUnit,
AdvancedSearchField,
AdvancedSearchLogicalOperator,
AdvancedSearchOperator,
AdvancedSearchQueryAtom,
AdvancedSearchQueryElementType,
AdvancedSearchQueryGroup,
} from 'src/app/data/advanced-search-query'
import { AdvancedSearchDialogComponent } from './advanced-search-dialog.component'
describe('AdvancedSearchDialogComponent', () => {
let component: AdvancedSearchDialogComponent
let fixture: ComponentFixture<AdvancedSearchDialogComponent>
let activeModal: NgbActiveModal
const firstAtom = (): AdvancedSearchQueryAtom =>
component.root.children[0] as AdvancedSearchQueryAtom
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
AdvancedSearchDialogComponent,
NgxBootstrapIconsModule.pick(allIcons),
],
providers: [NgbActiveModal],
}).compileComponents()
fixture = TestBed.createComponent(AdvancedSearchDialogComponent)
activeModal = TestBed.inject(NgbActiveModal)
component = fixture.componentInstance
fixture.detectChanges()
})
it('should start with one empty condition and nothing to search for', () => {
expect(component.root.children).toHaveLength(1)
expect(component.generatedQuery).toBe('')
expect(component.unreadable).toBeFalsy()
})
it('should show an existing query as conditions', () => {
component.query = 'title:invoice AND NOT tag:paid'
expect(component.unreadable).toBeFalsy()
expect(component.root.children).toHaveLength(2)
expect(component.generatedQuery).toBe('title:invoice AND NOT tag:paid')
})
it('should flag a query it cannot show and start empty', () => {
component.query = 'title:invoice^2'
expect(component.unreadable).toBeTruthy()
expect(component.generatedQuery).toBe('')
})
it('should clear the warning when starting a new query', () => {
component.query = 'title:invoice^2'
component.startOver()
expect(component.unreadable).toBeFalsy()
expect(component.root.children).toHaveLength(1)
})
it('should treat an empty query as a fresh start', () => {
component.query = ' '
expect(component.unreadable).toBeFalsy()
expect(component.root.children).toHaveLength(1)
})
it('should write the query as conditions are filled in', () => {
const atom = firstAtom()
atom.field = AdvancedSearchField.Title
atom.value = 'unpaid invoice'
expect(component.generatedQuery).toBe('title:unpaid AND title:invoice')
})
it('should offer the conditions of the chosen field', () => {
const atom = firstAtom()
atom.field = AdvancedSearchField.Added
component.onFieldChange(atom)
expect(component.operatorsFor(atom)).toContain(
AdvancedSearchOperator.WithinLast
)
expect(component.operatorsFor(atom)).not.toContain(
AdvancedSearchOperator.Phrase
)
})
it('should keep a condition the new field still offers', () => {
const atom = firstAtom()
atom.operator = AdvancedSearchOperator.Phrase
atom.field = AdvancedSearchField.Correspondent
component.onFieldChange(atom)
expect(atom.operator).toBe(AdvancedSearchOperator.Phrase)
})
it('should replace a condition the new field does not offer, and clear the value', () => {
const atom = firstAtom()
atom.operator = AdvancedSearchOperator.Phrase
atom.value = 'invoice'
atom.field = AdvancedSearchField.ASN
component.onFieldChange(atom)
expect(atom.operator).toBe(AdvancedSearchOperator.Equals)
expect(atom.value).toBe('')
})
it('should give a within-the-last condition a unit to start from', () => {
const atom = firstAtom()
atom.field = AdvancedSearchField.Added
atom.operator = AdvancedSearchOperator.WithinLast
component.onOperatorChange(atom)
expect(atom.unit).toBe(AdvancedSearchDateUnit.Day)
atom.value = '3'
expect(component.generatedQuery).toBe('added:[-3 days to now]')
})
it('should label date comparisons as dates', () => {
const atom = firstAtom()
atom.field = AdvancedSearchField.Created
expect(
component.operatorLabel(atom, AdvancedSearchOperator.AtLeast)
).toEqual('is on or after')
atom.field = AdvancedSearchField.ASN
expect(
component.operatorLabel(atom, AdvancedSearchOperator.AtLeast)
).toEqual('is at least')
})
it('should add and remove conditions', () => {
component.addAtom(component.root)
expect(component.root.children).toHaveLength(2)
component.remove(component.root, component.root.children[1])
expect(component.root.children).toHaveLength(1)
})
it('should add a group, which starts as Any', () => {
component.addGroup(component.root)
const group = component.root.children[1] as AdvancedSearchQueryGroup
expect(group.type).toBe(AdvancedSearchQueryElementType.Group)
expect(group.operator).toBe(AdvancedSearchLogicalOperator.Or)
expect(group.children).toHaveLength(1)
})
it('should give every group its own id, once', () => {
component.addGroup(component.root)
const group = component.root.children[1] as AdvancedSearchQueryGroup
expect(component.idFor(component.root)).not.toEqual(component.idFor(group))
expect(component.idFor(group)).toEqual(component.idFor(group))
})
it('should apply the query and close', () => {
const emitSpy = jest.spyOn(component.queryApplied, 'emit')
const closeSpy = jest.spyOn(activeModal, 'close')
const atom = firstAtom()
atom.value = 'invoice'
component.apply()
expect(emitSpy).toHaveBeenCalledWith('content:invoice')
expect(closeSpy).toHaveBeenCalled()
})
it('should close without applying on cancel', () => {
const emitSpy = jest.spyOn(component.queryApplied, 'emit')
const closeSpy = jest.spyOn(activeModal, 'close')
component.cancel()
expect(emitSpy).not.toHaveBeenCalled()
expect(closeSpy).toHaveBeenCalled()
})
it('should show the query it will apply', () => {
component.query = 'content:invoice OR content:receipt'
fixture.detectChanges()
const preview = fixture.nativeElement.querySelector(
'#advanced-search-preview'
)
expect(preview.textContent).toContain('content:invoice OR content:receipt')
})
it('should not offer to apply an empty query', () => {
fixture.detectChanges()
const apply = Array.from(
fixture.nativeElement.querySelectorAll('.modal-footer button')
).pop() as HTMLButtonElement
expect(apply.disabled).toBeTruthy()
})
})
@@ -1,197 +0,0 @@
import { NgTemplateOutlet } from '@angular/common'
import { Component, EventEmitter, inject, Input, Output } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import {
ADVANCED_SEARCH_DATE_KEYWORD_LABELS,
ADVANCED_SEARCH_DATE_KEYWORDS,
ADVANCED_SEARCH_DATE_OPERATOR_LABELS,
ADVANCED_SEARCH_DATE_UNIT_LABELS,
ADVANCED_SEARCH_FIELD_GROUPS,
ADVANCED_SEARCH_FIELD_KINDS,
ADVANCED_SEARCH_FIELD_LABELS,
ADVANCED_SEARCH_MAX_ATOMS,
ADVANCED_SEARCH_MAX_DEPTH,
ADVANCED_SEARCH_OPERATOR_LABELS,
ADVANCED_SEARCH_OPERATORS_BY_KIND,
AdvancedSearchDateUnit,
AdvancedSearchField,
AdvancedSearchFieldKind,
AdvancedSearchLogicalOperator,
AdvancedSearchOperator,
AdvancedSearchQueryAtom,
AdvancedSearchQueryElement,
AdvancedSearchQueryElementType,
AdvancedSearchQueryGroup,
} from 'src/app/data/advanced-search-query'
import {
parseAdvancedSearchQuery,
serializeAdvancedSearchQuery,
} from 'src/app/utils/advanced-search-query'
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
@Component({
selector: 'pngx-advanced-search-dialog',
templateUrl: './advanced-search-dialog.component.html',
styleUrl: './advanced-search-dialog.component.scss',
imports: [FormsModule, NgTemplateOutlet, NgxBootstrapIconsModule],
})
export class AdvancedSearchDialogComponent extends LoadingComponentWithPermissions {
private activeModal = inject(NgbActiveModal)
public readonly ElementType = AdvancedSearchQueryElementType
public readonly LogicalOperator = AdvancedSearchLogicalOperator
public readonly Operator = AdvancedSearchOperator
public readonly FieldKind = AdvancedSearchFieldKind
public readonly fieldGroups = ADVANCED_SEARCH_FIELD_GROUPS
public readonly fieldLabels = ADVANCED_SEARCH_FIELD_LABELS
public readonly dateKeywords = ADVANCED_SEARCH_DATE_KEYWORDS
public readonly dateKeywordLabels = ADVANCED_SEARCH_DATE_KEYWORD_LABELS
public readonly dateUnits = Object.values(AdvancedSearchDateUnit)
public readonly dateUnitLabels = ADVANCED_SEARCH_DATE_UNIT_LABELS
public readonly maxDepth = ADVANCED_SEARCH_MAX_DEPTH
public readonly maxAtoms = ADVANCED_SEARCH_MAX_ATOMS
@Output()
public queryApplied = new EventEmitter<string>()
public root: AdvancedSearchQueryGroup = this.emptyRoot()
// True when the query in the search box uses syntax the editor cannot show
public unreadable: boolean = false
private _query: string = ''
@Input()
set query(query: string) {
this._query = query ?? ''
const parsed = parseAdvancedSearchQuery(this._query)
this.unreadable = !!this._query.trim() && !parsed
this.root = parsed ?? this.emptyRoot()
}
get query(): string {
return this._query
}
constructor() {
super()
this.loading.set(false)
}
// Stable ids for the radio groups, without putting them in the query model
private ids = new WeakMap<object, number>()
private nextId = 0
public idFor(element: AdvancedSearchQueryElement): number {
if (!this.ids.has(element)) {
this.ids.set(element, this.nextId++)
}
return this.ids.get(element)
}
private emptyRoot(): AdvancedSearchQueryGroup {
return {
type: AdvancedSearchQueryElementType.Group,
operator: AdvancedSearchLogicalOperator.And,
children: [this.newAtom()],
}
}
private newAtom(): AdvancedSearchQueryAtom {
return {
type: AdvancedSearchQueryElementType.Atom,
field: AdvancedSearchField.Content,
operator: AdvancedSearchOperator.AllWords,
value: '',
}
}
public get generatedQuery(): string {
return serializeAdvancedSearchQuery(this.root)
}
public kindOf(atom: AdvancedSearchQueryAtom): AdvancedSearchFieldKind {
return ADVANCED_SEARCH_FIELD_KINDS[atom.field]
}
public operatorsFor(atom: AdvancedSearchQueryAtom): AdvancedSearchOperator[] {
return ADVANCED_SEARCH_OPERATORS_BY_KIND[this.kindOf(atom)]
}
public operatorLabel(
atom: AdvancedSearchQueryAtom,
operator: AdvancedSearchOperator
): string {
return this.kindOf(atom) === AdvancedSearchFieldKind.Date
? (ADVANCED_SEARCH_DATE_OPERATOR_LABELS[operator] ??
ADVANCED_SEARCH_OPERATOR_LABELS[operator])
: ADVANCED_SEARCH_OPERATOR_LABELS[operator]
}
public placeholderFor(atom: AdvancedSearchQueryAtom): string {
switch (atom.operator) {
case AdvancedSearchOperator.Phrase:
return $localize`phrase`
case AdvancedSearchOperator.StartsWith:
return $localize`beginning of a word`
default:
return $localize`words`
}
}
public onFieldChange(atom: AdvancedSearchQueryAtom) {
// Keep the condition only if the new field still offers it
if (!this.operatorsFor(atom).includes(atom.operator)) {
atom.operator = this.operatorsFor(atom)[0]
}
this.clearValues(atom)
}
public onOperatorChange(atom: AdvancedSearchQueryAtom) {
this.clearValues(atom)
}
private clearValues(atom: AdvancedSearchQueryAtom) {
atom.value = ''
atom.valueTo = undefined
atom.unit =
atom.operator === AdvancedSearchOperator.WithinLast
? AdvancedSearchDateUnit.Day
: undefined
}
public addAtom(group: AdvancedSearchQueryGroup) {
group.children.push(this.newAtom())
}
public addGroup(group: AdvancedSearchQueryGroup) {
group.children.push({
type: AdvancedSearchQueryElementType.Group,
operator: AdvancedSearchLogicalOperator.Or,
children: [this.newAtom()],
})
}
public remove(
parent: AdvancedSearchQueryGroup,
element: AdvancedSearchQueryElement
) {
parent.children = parent.children.filter((child) => child !== element)
}
public startOver() {
this.unreadable = false
this.root = this.emptyRoot()
}
public apply() {
this.queryApplied.emit(this.generatedQuery)
this.activeModal.close()
}
public cancel() {
this.activeModal.close()
}
}
@@ -146,6 +146,19 @@ describe('DocumentListComponent', () => {
expect(reloadSpy).toHaveBeenCalled()
})
it('should stop reloading on document deleted after destroy', () => {
const reloadSpy = jest.spyOn(documentListService, 'reload')
const documentDeletedSubject = new Subject<boolean>()
jest
.spyOn(websocketStatusService, 'onDocumentDeleted')
.mockReturnValue(documentDeletedSubject)
fixture.detectChanges()
fixture.destroy()
reloadSpy.mockClear()
documentDeletedSubject.next(true)
expect(reloadSpy).not.toHaveBeenCalled()
})
it('should show score sort fields on fulltext queries', () => {
documentListService.setFilterRules([
{
@@ -270,9 +270,12 @@ export class DocumentListComponent
this.list.reload()
})
this.websocketStatusService.onDocumentDeleted().subscribe(() => {
this.list.reload()
})
this.websocketStatusService
.onDocumentDeleted()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
this.list.reload()
})
this.route.paramMap
.pipe(
@@ -17,12 +17,6 @@
}
</select>
}
@if (advancedSearchEditorAvailable) {
<button class="btn btn-sm btn-outline-primary" type="button" (click)="openAdvancedSearchEditor()"
title="Edit query" i18n-title [disabled]="disabled">
<i-bs name="sliders"></i-bs>
</button>
}
@if (_textFilter) {
<button class="btn btn-link btn-sm px-2 position-absolute top-0 end-0 z-10" (click)="resetTextField()" aria-label="Clear search" i18n-aria-label>
<i-bs width="1em" height="1em" name="x"></i-bs>
@@ -12,8 +12,6 @@ import {
NgbDatepickerModule,
NgbDropdownItem,
NgbDropdownModule,
NgbModal,
NgbModalRef,
NgbTypeaheadModule,
} from '@ng-bootstrap/ng-bootstrap'
import { NgSelectComponent, NgSelectModule } from '@ng-select/ng-select'
@@ -2517,45 +2515,4 @@ describe('FilterEditorComponent', () => {
expect(component.textFilter).toEqual('help ')
})
it('should open the advanced search editor with the current query and apply the result', () => {
const modalService: NgbModal = TestBed.inject(NgbModal)
let modal: NgbModalRef
modalService.activeInstances.subscribe(
(instances) => (modal = instances[0])
)
component.textFilterTarget = 'fulltext-query'
component.updateTextFilter('title:invoice')
fixture.detectChanges()
const editorButton = fixture.debugElement.query(
By.css('button[title="Edit query"]')
)
expect(editorButton).not.toBeNull()
editorButton.triggerEventHandler('click')
fixture.detectChanges()
expect(modal.componentInstance.query).toEqual('title:invoice')
const rulesSpy = jest.spyOn(component.filterRulesChange, 'next')
modal.componentInstance.queryApplied.emit('title:invoice AND NOT tag:paid')
expect(component.textFilter).toEqual('title:invoice AND NOT tag:paid')
expect(documentService.searchQuery).toEqual(
'title:invoice AND NOT tag:paid'
)
expect(rulesSpy).toHaveBeenCalledWith([
{
rule_type: FILTER_FULLTEXT_QUERY,
value: 'title:invoice AND NOT tag:paid',
},
])
})
it('should not offer the advanced search editor for other targets', () => {
component.textFilterTarget = 'title-content'
fixture.detectChanges()
expect(
fixture.debugElement.query(By.css('button[title="Edit query"]'))
).toBeNull()
})
})
@@ -15,13 +15,12 @@ import {
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import {
NgbDropdownModule,
NgbModal,
NgbTypeahead,
NgbTypeaheadModule,
} from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { TourNgBootstrap } from 'ngx-ui-tour-ng-bootstrap'
import { first, Observable, Subject, from } from 'rxjs'
import { Observable, Subject, from } from 'rxjs'
import {
catchError,
debounceTime,
@@ -122,7 +121,6 @@ import {
PermissionsFilterDropdownComponent,
PermissionsSelectionModel,
} from '../../common/permissions-filter-dropdown/permissions-filter-dropdown.component'
import { AdvancedSearchDialogComponent } from '../../common/advanced-search-dialog/advanced-search-dialog.component'
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
const TEXT_FILTER_TARGET_TITLE = 'title'
@@ -288,7 +286,6 @@ export class FilterEditorComponent
permissionsService = inject(PermissionsService)
private customFieldService = inject(CustomFieldsService)
private searchService = inject(SearchService)
private modalService = inject(NgbModal)
generateFilterName() {
if (this.filterRules.length == 1) {
@@ -1375,23 +1372,6 @@ export class FilterEditorComponent
}
}
get advancedSearchEditorAvailable(): boolean {
return this.textFilterTarget === TEXT_FILTER_TARGET_FULLTEXT_QUERY
}
openAdvancedSearchEditor() {
const modal = this.modalService.open(AdvancedSearchDialogComponent, {
backdrop: 'static',
size: 'lg',
})
modal.componentInstance.query = this._textFilter ?? ''
modal.componentInstance.queryApplied
.pipe(first())
.subscribe((query: string) => {
this.updateTextFilter(query)
})
}
textFilterKeydown(event: KeyboardEvent) {
if (event.key == 'Enter') {
if (event.defaultPrevented) {
@@ -1,262 +0,0 @@
// Fields and forms documented in docs/usage.md > "Document searches"
export enum AdvancedSearchField {
Any = '',
Title = 'title',
Content = 'content',
OriginalFilename = 'original_filename',
NoteText = 'notes.note',
NoteAuthor = 'notes.user',
CustomFieldName = 'custom_fields.name',
CustomFieldValue = 'custom_fields.value',
Correspondent = 'correspondent',
DocumentType = 'document_type',
StoragePath = 'storage_path',
Tag = 'tag',
ASN = 'asn',
PageCount = 'page_count',
NumNotes = 'num_notes',
Created = 'created',
Added = 'added',
Modified = 'modified',
Checksum = 'checksum',
}
export enum AdvancedSearchFieldKind {
Text = 'text',
Number = 'number',
Date = 'date',
Checksum = 'checksum',
}
export const ADVANCED_SEARCH_FIELD_KINDS: Record<
AdvancedSearchField,
AdvancedSearchFieldKind
> = {
[AdvancedSearchField.Any]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.Title]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.Content]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.OriginalFilename]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.NoteText]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.NoteAuthor]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.CustomFieldName]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.CustomFieldValue]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.Correspondent]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.DocumentType]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.StoragePath]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.Tag]: AdvancedSearchFieldKind.Text,
[AdvancedSearchField.ASN]: AdvancedSearchFieldKind.Number,
[AdvancedSearchField.PageCount]: AdvancedSearchFieldKind.Number,
[AdvancedSearchField.NumNotes]: AdvancedSearchFieldKind.Number,
[AdvancedSearchField.Created]: AdvancedSearchFieldKind.Date,
[AdvancedSearchField.Added]: AdvancedSearchFieldKind.Date,
[AdvancedSearchField.Modified]: AdvancedSearchFieldKind.Date,
[AdvancedSearchField.Checksum]: AdvancedSearchFieldKind.Checksum,
}
export enum AdvancedSearchOperator {
AllWords = 'all',
AnyWord = 'any',
Phrase = 'phrase',
StartsWith = 'prefix',
Equals = 'eq',
AtLeast = 'gte',
AtMost = 'lte',
Between = 'between',
DateKeyword = 'keyword',
WithinLast = 'within',
}
export const ADVANCED_SEARCH_OPERATORS_BY_KIND: Record<
AdvancedSearchFieldKind,
AdvancedSearchOperator[]
> = {
[AdvancedSearchFieldKind.Text]: [
AdvancedSearchOperator.AllWords,
AdvancedSearchOperator.AnyWord,
AdvancedSearchOperator.Phrase,
AdvancedSearchOperator.StartsWith,
],
[AdvancedSearchFieldKind.Number]: [
AdvancedSearchOperator.Equals,
AdvancedSearchOperator.AtLeast,
AdvancedSearchOperator.AtMost,
AdvancedSearchOperator.Between,
],
[AdvancedSearchFieldKind.Date]: [
AdvancedSearchOperator.DateKeyword,
AdvancedSearchOperator.WithinLast,
AdvancedSearchOperator.AtLeast,
AdvancedSearchOperator.AtMost,
AdvancedSearchOperator.Between,
],
[AdvancedSearchFieldKind.Checksum]: [AdvancedSearchOperator.StartsWith],
}
export const ADVANCED_SEARCH_DATE_KEYWORDS = [
'today',
'yesterday',
'tomorrow',
'previous week',
'this month',
'previous month',
'previous quarter',
'this year',
'previous year',
] as const
export type AdvancedSearchDateKeyword =
(typeof ADVANCED_SEARCH_DATE_KEYWORDS)[number]
export enum AdvancedSearchDateUnit {
Day = 'day',
Week = 'week',
Month = 'month',
Year = 'year',
}
export enum AdvancedSearchLogicalOperator {
And = 'AND',
Or = 'OR',
Not = 'NOT',
}
export enum AdvancedSearchQueryElementType {
Atom = 'atom',
Group = 'group',
}
export interface AdvancedSearchQueryAtom {
type: AdvancedSearchQueryElementType.Atom
field: AdvancedSearchField
operator: AdvancedSearchOperator
value?: string
valueTo?: string
unit?: AdvancedSearchDateUnit // for WithinLast
}
export interface AdvancedSearchQueryGroup {
type: AdvancedSearchQueryElementType.Group
operator: AdvancedSearchLogicalOperator
children: AdvancedSearchQueryElement[]
}
export type AdvancedSearchQueryElement =
AdvancedSearchQueryAtom | AdvancedSearchQueryGroup
export const ADVANCED_SEARCH_MAX_DEPTH = 2
export const ADVANCED_SEARCH_MAX_ATOMS = 10
export const ADVANCED_SEARCH_FIELD_LABELS: Record<AdvancedSearchField, string> =
{
[AdvancedSearchField.Any]: $localize`Any field`,
[AdvancedSearchField.Title]: $localize`Title`,
[AdvancedSearchField.Content]: $localize`Content`,
[AdvancedSearchField.OriginalFilename]: $localize`File name`,
[AdvancedSearchField.NoteText]: $localize`Note text`,
[AdvancedSearchField.NoteAuthor]: $localize`Note author`,
[AdvancedSearchField.CustomFieldName]: $localize`Custom field name`,
[AdvancedSearchField.CustomFieldValue]: $localize`Custom field value`,
[AdvancedSearchField.Correspondent]: $localize`Correspondent name`,
[AdvancedSearchField.DocumentType]: $localize`Document type name`,
[AdvancedSearchField.StoragePath]: $localize`Storage path name`,
[AdvancedSearchField.Tag]: $localize`Tag name`,
[AdvancedSearchField.ASN]: $localize`ASN`,
[AdvancedSearchField.PageCount]: $localize`Pages`,
[AdvancedSearchField.NumNotes]: $localize`Number of notes`,
[AdvancedSearchField.Created]: $localize`Created`,
[AdvancedSearchField.Added]: $localize`Added`,
[AdvancedSearchField.Modified]: $localize`Modified`,
[AdvancedSearchField.Checksum]: $localize`Checksum`,
}
export const ADVANCED_SEARCH_FIELD_GROUPS: {
label: string
fields: AdvancedSearchField[]
}[] = [
{
label: $localize`Text`,
fields: [
AdvancedSearchField.Any,
AdvancedSearchField.Title,
AdvancedSearchField.Content,
AdvancedSearchField.OriginalFilename,
AdvancedSearchField.NoteText,
AdvancedSearchField.NoteAuthor,
AdvancedSearchField.CustomFieldName,
AdvancedSearchField.CustomFieldValue,
],
},
{
label: $localize`Names`,
fields: [
AdvancedSearchField.Correspondent,
AdvancedSearchField.DocumentType,
AdvancedSearchField.StoragePath,
AdvancedSearchField.Tag,
],
},
{
label: $localize`Numbers`,
fields: [
AdvancedSearchField.ASN,
AdvancedSearchField.PageCount,
AdvancedSearchField.NumNotes,
],
},
{
label: $localize`Dates`,
fields: [
AdvancedSearchField.Created,
AdvancedSearchField.Added,
AdvancedSearchField.Modified,
],
},
{ label: $localize`Other`, fields: [AdvancedSearchField.Checksum] },
]
export const ADVANCED_SEARCH_OPERATOR_LABELS: Record<
AdvancedSearchOperator,
string
> = {
[AdvancedSearchOperator.AllWords]: $localize`contains all words`,
[AdvancedSearchOperator.AnyWord]: $localize`contains any word`,
[AdvancedSearchOperator.Phrase]: $localize`contains the phrase`,
[AdvancedSearchOperator.StartsWith]: $localize`starts with`,
[AdvancedSearchOperator.Equals]: $localize`is`,
[AdvancedSearchOperator.AtLeast]: $localize`is at least`,
[AdvancedSearchOperator.AtMost]: $localize`is at most`,
[AdvancedSearchOperator.Between]: $localize`is between`,
[AdvancedSearchOperator.DateKeyword]: $localize`is`,
[AdvancedSearchOperator.WithinLast]: $localize`is within the last`,
}
// Comparing dates reads differently than comparing counts
export const ADVANCED_SEARCH_DATE_OPERATOR_LABELS: Partial<
Record<AdvancedSearchOperator, string>
> = {
[AdvancedSearchOperator.AtLeast]: $localize`is on or after`,
[AdvancedSearchOperator.AtMost]: $localize`is on or before`,
}
export const ADVANCED_SEARCH_DATE_KEYWORD_LABELS: Record<string, string> = {
today: $localize`today`,
yesterday: $localize`yesterday`,
tomorrow: $localize`tomorrow`,
'previous week': $localize`previous week`,
'this month': $localize`this month`,
'previous month': $localize`previous month`,
'previous quarter': $localize`previous quarter`,
'this year': $localize`this year`,
'previous year': $localize`previous year`,
}
export const ADVANCED_SEARCH_DATE_UNIT_LABELS: Record<
AdvancedSearchDateUnit,
string
> = {
[AdvancedSearchDateUnit.Day]: $localize`days`,
[AdvancedSearchDateUnit.Week]: $localize`weeks`,
[AdvancedSearchDateUnit.Month]: $localize`months`,
[AdvancedSearchDateUnit.Year]: $localize`years`,
}
+9
View File
@@ -353,6 +353,14 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_MODEL',
category: ConfigCategory.AI,
},
{
key: 'llm_embedding_api_key',
title: $localize`LLM Embedding API Key`,
type: ConfigOptionType.Password,
note: $localize`Used for embeddings when set, otherwise LLM API key is used.`,
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_API_KEY',
category: ConfigCategory.AI,
},
{
key: 'llm_embedding_endpoint',
title: $localize`LLM Embedding Endpoint`,
@@ -457,6 +465,7 @@ export interface PaperlessConfig extends ObjectWithId {
ai_enabled: boolean
llm_embedding_backend: string
llm_embedding_model: string
llm_embedding_api_key: string
llm_embedding_endpoint: string
llm_embedding_chunk_size: number
llm_context_size: number
@@ -1,515 +0,0 @@
import {
AdvancedSearchDateUnit,
AdvancedSearchField,
AdvancedSearchLogicalOperator,
AdvancedSearchOperator,
AdvancedSearchQueryAtom,
AdvancedSearchQueryElement,
AdvancedSearchQueryElementType,
AdvancedSearchQueryGroup,
} from '../data/advanced-search-query'
import {
parseAdvancedSearchQuery,
serializeAdvancedSearchQuery,
} from './advanced-search-query'
const atom = (
field: AdvancedSearchField,
operator: AdvancedSearchOperator,
value?: string,
extra: Partial<AdvancedSearchQueryAtom> = {}
): AdvancedSearchQueryAtom => ({
type: AdvancedSearchQueryElementType.Atom,
field,
operator,
value,
...extra,
})
const group = (
operator: AdvancedSearchLogicalOperator,
...children: AdvancedSearchQueryElement[]
): AdvancedSearchQueryGroup => ({
type: AdvancedSearchQueryElementType.Group,
operator,
children,
})
const { And, Or, Not } = AdvancedSearchLogicalOperator
describe('serializeAdvancedSearchQuery', () => {
describe('text fields', () => {
it.each([
[AdvancedSearchOperator.AllWords, 'invoice', 'title:invoice'],
[
AdvancedSearchOperator.AllWords,
' invoice unpaid ',
'title:invoice AND title:unpaid',
],
[
AdvancedSearchOperator.AnyWord,
'invoice unpaid',
'title:invoice OR title:unpaid',
],
[
AdvancedSearchOperator.Phrase,
'quick brown fox',
'title:"quick brown fox"',
],
[AdvancedSearchOperator.Phrase, 'say "hi"', 'title:"say hi"'],
[AdvancedSearchOperator.StartsWith, 'invoi', 'title:invoi*'],
[AdvancedSearchOperator.StartsWith, 'in*v?oi', 'title:invoi*'],
])('%s %j writes %s', (operator, value, expected) => {
expect(
serializeAdvancedSearchQuery(
atom(AdvancedSearchField.Title, operator, value)
)
).toBe(expected)
})
it('writes bare words for the Any field', () => {
expect(
serializeAdvancedSearchQuery(
atom(AdvancedSearchField.Any, AdvancedSearchOperator.AllWords, 'a b')
)
).toBe('a AND b')
})
it.each([
['A-1312/99', 'custom_fields.value:A-1312/99'],
["O'Brien", "custom_fields.value:O'Brien"],
["'quoted'", `custom_fields.value:"'quoted'"`],
['foo:bar', 'custom_fields.value:"foo:bar"'],
['(x)', 'custom_fields.value:"(x)"'],
['2024*', 'custom_fields.value:"2024*"'],
['a,b', 'custom_fields.value:"a,b"'],
['OR', 'custom_fields.value:"OR"'],
['or', 'custom_fields.value:or'],
])(
'quotes %j only when the grammar would read it as syntax',
(value, expected) => {
expect(
serializeAdvancedSearchQuery(
atom(
AdvancedSearchField.CustomFieldValue,
AdvancedSearchOperator.AllWords,
value
)
)
).toBe(expected)
}
)
it('uses the dotted names for custom fields', () => {
expect(
serializeAdvancedSearchQuery(
group(
And,
atom(
AdvancedSearchField.CustomFieldName,
AdvancedSearchOperator.Phrase,
'status'
),
atom(
AdvancedSearchField.CustomFieldValue,
AdvancedSearchOperator.AllWords,
'paid'
)
)
)
).toBe('custom_fields.name:"status" AND custom_fields.value:paid')
})
it('uses the dotted names for notes', () => {
expect(
serializeAdvancedSearchQuery(
group(
And,
atom(
AdvancedSearchField.NoteText,
AdvancedSearchOperator.AllWords,
'call'
),
atom(
AdvancedSearchField.NoteAuthor,
AdvancedSearchOperator.AllWords,
'alice'
)
)
)
).toBe('notes.note:call AND notes.user:alice')
})
it.each([
[AdvancedSearchOperator.AllWords, ''],
[AdvancedSearchOperator.AllWords, ' '],
[AdvancedSearchOperator.AllWords, '!! --'],
[AdvancedSearchOperator.Phrase, '""'],
[AdvancedSearchOperator.StartsWith, 'two words'],
[AdvancedSearchOperator.StartsWith, '***'],
[AdvancedSearchOperator.StartsWith, undefined],
])('leaves out %s %j', (operator, value) => {
expect(
serializeAdvancedSearchQuery(
atom(AdvancedSearchField.Title, operator, value)
)
).toBe('')
})
})
describe('checksum', () => {
it('lowercases the prefix', () => {
expect(
serializeAdvancedSearchQuery(
atom(
AdvancedSearchField.Checksum,
AdvancedSearchOperator.StartsWith,
'9F86D081'
)
)
).toBe('checksum:9f86d081*')
})
})
describe('number fields', () => {
it.each([
[AdvancedSearchOperator.Equals, '42', undefined, 'asn:42'],
[AdvancedSearchOperator.AtLeast, '50', undefined, 'asn:[50 to]'],
[AdvancedSearchOperator.AtMost, '50', undefined, 'asn:[to 50]'],
[AdvancedSearchOperator.Between, '50', '150', 'asn:[50 to 150]'],
[AdvancedSearchOperator.Equals, '4.2', undefined, ''],
[AdvancedSearchOperator.Equals, '-1', undefined, ''],
[AdvancedSearchOperator.Equals, '2024-01-01', undefined, ''],
[AdvancedSearchOperator.Between, '50', '', ''],
])('%s %j %j writes %j', (operator, value, valueTo, expected) => {
expect(
serializeAdvancedSearchQuery(
atom(AdvancedSearchField.ASN, operator, value, { valueTo })
)
).toBe(expected)
})
})
describe('date fields', () => {
it.each([
['today', 'added:today'],
['previous month', 'added:"previous month"'],
['last tuesday', ''],
['', ''],
])('keyword %j writes %j', (value, expected) => {
expect(
serializeAdvancedSearchQuery(
atom(
AdvancedSearchField.Added,
AdvancedSearchOperator.DateKeyword,
value
)
)
).toBe(expected)
})
it.each([
['1', AdvancedSearchDateUnit.Day, 'added:[-1 day to now]'],
['3', AdvancedSearchDateUnit.Month, 'added:[-3 months to now]'],
['2', AdvancedSearchDateUnit.Week, 'added:[-2 weeks to now]'],
['0', AdvancedSearchDateUnit.Year, ''],
['1.5', AdvancedSearchDateUnit.Year, ''],
['3', undefined, ''],
['3', 'fortnight' as AdvancedSearchDateUnit, ''],
])('within the last %j %j writes %j', (value, unit, expected) => {
expect(
serializeAdvancedSearchQuery(
atom(
AdvancedSearchField.Added,
AdvancedSearchOperator.WithinLast,
value,
{
unit,
}
)
)
).toBe(expected)
})
it.each([
[
AdvancedSearchOperator.AtLeast,
'2024-01-01',
undefined,
'created:[2024-01-01 to]',
],
[
AdvancedSearchOperator.AtMost,
'2024-01-01',
undefined,
'created:[to 2024-01-01]',
],
[
AdvancedSearchOperator.Between,
'2024-01-01',
'2024-03-31',
'created:[2024-01-01 to 2024-03-31]',
],
[AdvancedSearchOperator.AtLeast, '2024', undefined, ''],
[AdvancedSearchOperator.Between, '2024-01-01', 'now', ''],
])('%s %j %j writes %j', (operator, value, valueTo, expected) => {
expect(
serializeAdvancedSearchQuery(
atom(AdvancedSearchField.Created, operator, value, { valueTo })
)
).toBe(expected)
})
})
describe('groups', () => {
const invoice = atom(
AdvancedSearchField.Content,
AdvancedSearchOperator.AllWords,
'invoice'
)
const letter = atom(
AdvancedSearchField.Title,
AdvancedSearchOperator.AllWords,
'letter'
)
const paid = atom(
AdvancedSearchField.Tag,
AdvancedSearchOperator.AllWords,
'paid'
)
const twoWords = atom(
AdvancedSearchField.Title,
AdvancedSearchOperator.AllWords,
'a b'
)
const anyWords = atom(
AdvancedSearchField.Title,
AdvancedSearchOperator.AnyWord,
'a b'
)
const empty = atom(
AdvancedSearchField.Title,
AdvancedSearchOperator.AllWords,
''
)
it.each([
['an empty group', group(And), ''],
['a group of empty atoms', group(Or, empty, group(And, empty)), ''],
[
'a single child without parentheses',
group(Or, invoice),
'content:invoice',
],
['All', group(And, invoice, letter), 'content:invoice AND title:letter'],
['Any', group(Or, invoice, letter), 'content:invoice OR title:letter'],
[
'skipped empty atoms',
group(And, empty, invoice, empty),
'content:invoice',
],
['Not with one child', group(Not, paid), 'NOT tag:paid'],
[
'Not as none of',
group(Not, paid, letter),
'NOT (tag:paid OR title:letter)',
],
[
'Not with a compound child',
group(Not, twoWords),
'NOT (title:a AND title:b)',
],
[
'Not inside All',
group(And, invoice, group(Not, paid)),
'content:invoice AND NOT tag:paid',
],
[
'Not inside Any',
group(Or, invoice, group(Not, paid)),
'content:invoice OR NOT tag:paid',
],
[
'Any inside All',
group(And, invoice, group(Or, letter, paid)),
'content:invoice AND (title:letter OR tag:paid)',
],
[
'All inside Any',
group(Or, invoice, group(And, letter, paid)),
'content:invoice OR (title:letter AND tag:paid)',
],
[
'All inside All flattened',
group(And, invoice, group(And, letter, paid)),
'content:invoice AND title:letter AND tag:paid',
],
[
'an all-words atom inside Any',
group(Or, invoice, twoWords),
'content:invoice OR (title:a AND title:b)',
],
[
'an any-word atom inside All',
group(And, invoice, anyWords),
'content:invoice AND (title:a OR title:b)',
],
[
'an all-words atom inside Not with siblings',
group(Not, paid, twoWords),
'NOT (tag:paid OR (title:a AND title:b))',
],
])('writes %s', (_, tree, expected) => {
expect(serializeAdvancedSearchQuery(tree)).toBe(expected)
})
it('writes the mockup example', () => {
expect(
serializeAdvancedSearchQuery(
group(
And,
invoice,
group(
Or,
atom(
AdvancedSearchField.Correspondent,
AdvancedSearchOperator.Phrase,
'acme corp'
),
atom(
AdvancedSearchField.Content,
AdvancedSearchOperator.Phrase,
'acme corporation'
)
),
atom(
AdvancedSearchField.Added,
AdvancedSearchOperator.WithinLast,
'3',
{
unit: AdvancedSearchDateUnit.Month,
}
),
group(Not, paid)
)
)
).toBe(
'content:invoice AND (correspondent:"acme corp" OR content:"acme corporation") AND added:[-3 months to now] AND NOT tag:paid'
)
})
})
})
describe('parseAdvancedSearchQuery', () => {
const canonical = [
'title:invoice',
'title:invoice AND title:unpaid',
'content:invoice OR content:receipt',
'a AND b',
'title:"quick brown fox"',
'title:invoi*',
"custom_fields.value:O'Brien",
'custom_fields.value:"foo:bar"',
'custom_fields.name:"status" AND custom_fields.value:paid',
'notes.note:call AND notes.user:alice',
'checksum:9f86d081*',
'asn:42',
'asn:[50 to 150]',
'asn:[50 to]',
'asn:[to 50]',
'page_count:[10 to]',
'added:today',
'added:"previous month"',
'added:[-1 day to now]',
'added:[-3 months to now]',
'created:[2024-01-01 to 2024-03-31]',
'created:[2024-01-01 to]',
'created:[to 2024-01-01]',
'NOT tag:paid',
'NOT (tag:paid OR title:letter)',
'NOT (title:a AND title:b)',
'content:invoice OR NOT tag:paid',
'content:invoice AND (title:letter OR tag:paid)',
'content:invoice OR (title:letter AND tag:paid)',
'content:invoice AND (correspondent:"acme corp" OR content:"acme corporation") AND added:[-3 months to now] AND NOT tag:paid',
]
it.each(canonical)('reads back %s unchanged', (query) => {
const tree = parseAdvancedSearchQuery(query)
expect(tree).not.toBeNull()
expect(serializeAdvancedSearchQuery(tree)).toBe(query)
})
it('reads surrounding whitespace', () => {
expect(
serializeAdvancedSearchQuery(
parseAdvancedSearchQuery(' title:invoice ')
)
).toBe('title:invoice')
})
it('puts the words of one condition back together', () => {
expect(parseAdvancedSearchQuery('title:invoice AND title:unpaid')).toEqual(
group(
And,
atom(
AdvancedSearchField.Title,
AdvancedSearchOperator.AllWords,
'invoice unpaid'
)
)
)
})
it('keeps words of different fields apart', () => {
expect(parseAdvancedSearchQuery('title:a OR content:b')).toEqual(
group(
Or,
atom(AdvancedSearchField.Title, AdvancedSearchOperator.AllWords, 'a'),
atom(AdvancedSearchField.Content, AdvancedSearchOperator.AllWords, 'b')
)
)
})
it('reads a range as its condition', () => {
expect(parseAdvancedSearchQuery('added:[-3 months to now]')).toEqual(
group(
And,
atom(
AdvancedSearchField.Added,
AdvancedSearchOperator.WithinLast,
'3',
{
unit: AdvancedSearchDateUnit.Month,
}
)
)
)
})
it.each([
['', 'nothing'],
[' ', 'whitespace'],
['type:invoice', 'a field alias'],
['notes:call', 'a bare notes prefix'],
['unknown:x', 'an unknown field'],
['title:a AND content:b OR title:c', 'AND and OR mixed at one level'],
['title:a b', 'an implicit AND'],
['title:"unterminated', 'an unterminated phrase'],
['asn:[50 to', 'an unterminated range'],
['title:a*b', 'a wildcard in the middle'],
['title:invoice^2', 'a boost'],
['asn:[50 to abc]', 'a non-numeric bound'],
['asn:invoice', 'a word on a number field'],
['created:[2024 to 2025]', 'a year-only date range'],
['added:"last tuesday"', 'a date keyword paperless does not have'],
['title:[a to b]', 'a range on a text field'],
['checksum:9f86d081', 'a checksum without a wildcard'],
['(title:invoice)', 'parentheses the editor would not write'],
['title:invoice AND', 'a trailing operator'],
['title:"a"b', 'a value running into the next'],
['ADDED:today', 'an uppercase field name'],
])('leaves %s alone, having %s', (query) => {
expect(parseAdvancedSearchQuery(query)).toBeNull()
})
})
@@ -1,500 +0,0 @@
import {
ADVANCED_SEARCH_DATE_KEYWORDS,
ADVANCED_SEARCH_FIELD_KINDS,
AdvancedSearchDateUnit,
AdvancedSearchField,
AdvancedSearchFieldKind,
AdvancedSearchLogicalOperator,
AdvancedSearchOperator,
AdvancedSearchQueryAtom,
AdvancedSearchQueryElement,
AdvancedSearchQueryElementType,
AdvancedSearchQueryGroup,
} from '../data/advanced-search-query'
// Anything the query grammar would read as syntax rather than as a word
const SYNTAX_CHARS = /[\s():"[\]*?,{}^~\\]/
const SYNTAX_CHARS_GLOBAL = new RegExp(SYNTAX_CHARS, 'g')
// A word wrapped in single quotes is also syntax, an apostrophe inside one is not
const EDGE_SINGLE_QUOTE = /^'|'$/
const RESERVED_WORDS = /^(AND|OR|NOT|TO)$/
const HAS_WORD_CHAR = /[\p{L}\p{N}]/u
const WHOLE_NUMBER = /^\d+$/
const ISO_DATE = /^\d{4}-\d{2}-\d{2}$/
interface Serialized {
text: string
// The operator joining the top level of `text`, null when it is self-delimiting
join:
AdvancedSearchLogicalOperator.And | AdvancedSearchLogicalOperator.Or | null
}
const prefix = (field: AdvancedSearchField) => (field ? `${field}:` : '')
const quote = (text: string) => `"${text.replace(/"/g, '')}"`
const words = (value: string) =>
value
.trim()
.split(/\s+/)
.filter((word) => HAS_WORD_CHAR.test(word))
const word = (w: string) =>
SYNTAX_CHARS.test(w) || EDGE_SINGLE_QUOTE.test(w) || RESERVED_WORDS.test(w)
? quote(w)
: w
const atomic = (text: string): Serialized => ({ text, join: null })
function serializeWords(
atom: AdvancedSearchQueryAtom,
join: AdvancedSearchLogicalOperator.And | AdvancedSearchLogicalOperator.Or
): Serialized {
// A field applies only to the word right after it, so repeat it per word
const terms = words(atom.value ?? '').map(
(w) => `${prefix(atom.field)}${word(w)}`
)
if (terms.length === 0) return null
if (terms.length === 1) return atomic(terms[0])
return { text: terms.join(` ${join} `), join }
}
function serializeRange(
atom: AdvancedSearchQueryAtom,
isValid: (v: string) => boolean
): Serialized {
const lo = atom.value?.trim() ?? ''
const hi = atom.valueTo?.trim() ?? ''
const field = prefix(atom.field)
switch (atom.operator) {
case AdvancedSearchOperator.Equals:
return isValid(lo) ? atomic(`${field}${lo}`) : null
case AdvancedSearchOperator.AtLeast:
return isValid(lo) ? atomic(`${field}[${lo} to]`) : null
case AdvancedSearchOperator.AtMost:
return isValid(lo) ? atomic(`${field}[to ${lo}]`) : null
case AdvancedSearchOperator.Between:
return isValid(lo) && isValid(hi)
? atomic(`${field}[${lo} to ${hi}]`)
: null
}
return null
}
function serializeAtom(atom: AdvancedSearchQueryAtom): Serialized {
const field = prefix(atom.field)
const value = atom.value?.trim() ?? ''
switch (atom.operator) {
case AdvancedSearchOperator.AllWords:
return serializeWords(atom, AdvancedSearchLogicalOperator.And)
case AdvancedSearchOperator.AnyWord:
return serializeWords(atom, AdvancedSearchLogicalOperator.Or)
case AdvancedSearchOperator.Phrase:
return HAS_WORD_CHAR.test(value)
? atomic(`${field}${quote(value)}`)
: null
case AdvancedSearchOperator.StartsWith: {
if (/\s/.test(value)) return null
let stem = value.replace(SYNTAX_CHARS_GLOBAL, '')
if (!HAS_WORD_CHAR.test(stem)) return null
// checksum is indexed as-is, in lowercase
if (atom.field === AdvancedSearchField.Checksum) {
stem = stem.toLowerCase()
}
return atomic(`${field}${stem}*`)
}
case AdvancedSearchOperator.DateKeyword:
return (ADVANCED_SEARCH_DATE_KEYWORDS as readonly string[]).includes(
value
)
? atomic(`${field}${word(value)}`)
: null
case AdvancedSearchOperator.WithinLast: {
const amount = parseInt(value, 10)
const units = Object.values(AdvancedSearchDateUnit) as string[]
if (!WHOLE_NUMBER.test(value) || amount < 1) return null
if (!units.includes(atom.unit)) return null
const unit = amount === 1 ? atom.unit : `${atom.unit}s`
return atomic(`${field}[-${amount} ${unit} to now]`)
}
case AdvancedSearchOperator.Equals:
case AdvancedSearchOperator.AtLeast:
case AdvancedSearchOperator.AtMost:
case AdvancedSearchOperator.Between:
return serializeRange(
atom,
ADVANCED_SEARCH_FIELD_KINDS[atom.field] === AdvancedSearchFieldKind.Date
? (v) => ISO_DATE.test(v)
: (v) => WHOLE_NUMBER.test(v)
)
}
return null
}
function wrap(
child: Serialized,
parentJoin: AdvancedSearchLogicalOperator
): string {
return child.join === null || child.join === parentJoin
? child.text
: `(${child.text})`
}
function serializeGroup(group: AdvancedSearchQueryGroup): Serialized {
const children = group.children.map(serializeElement).filter(Boolean)
if (children.length === 0) return null
if (group.operator === AdvancedSearchLogicalOperator.Not) {
// A Not group matches documents matching none of its children
if (children.length === 1 && children[0].join === null) {
return atomic(`NOT ${children[0].text}`)
}
const inner =
children.length === 1
? children[0].text
: children
.map((c) => wrap(c, AdvancedSearchLogicalOperator.Or))
.join(' OR ')
return atomic(`NOT (${inner})`)
}
if (children.length === 1) return children[0]
return {
text: children
.map((c) => wrap(c, group.operator))
.join(` ${group.operator} `),
join: group.operator,
}
}
function serializeElement(element: AdvancedSearchQueryElement): Serialized {
return element.type === AdvancedSearchQueryElementType.Group
? serializeGroup(element)
: serializeAtom(element)
}
/**
* Writes an editor tree as a full-text query. Atoms that are not filled in
* (or not valid) are left out, and empty groups with them.
*/
export function serializeAdvancedSearchQuery(
element: AdvancedSearchQueryElement
): string {
return serializeElement(element)?.text ?? ''
}
// --- Reading a query back into the editor --------------------------------
//
// Deliberately narrow: this reads the forms serializeAdvancedSearchQuery
// writes, and nothing else. A query it cannot read is not a failure, it just
// stays text, so there is never a lossy or surprising conversion. The final
// round-trip check below is what holds that promise: a tree is only returned
// when writing it out again reproduces the query exactly.
const RELATIVE_BOUND = /^-(\d+) (day|week|month|year)s?$/
const FIELD_PREFIX = /^([a-z_]+(?:\.[a-z_]+)?):/
const KEYWORD_TOKEN = /^(AND|OR|NOT)(?=[\s(]|$)/
// Either bound may be missing: [50 to 150], [50 to], [to 50]
const RANGE_BOUNDS = /^(?:(.+?) )?to(?: (.+))?$/
const TRAILING_WILDCARD = /^([^*?]+)\*$/
class UnreadableQuery extends Error {}
interface Token {
type: 'term' | 'AND' | 'OR' | 'NOT' | '(' | ')'
field?: string
value?: string
quoted?: boolean
range?: boolean
}
// A parsed element, plus what it takes to merge the per-word terms the
// serializer writes for "contains all words" back into a single condition
interface Parsed {
element: AdvancedSearchQueryElement
word?: { field: AdvancedSearchField; text: string }
}
function tokenize(query: string): Token[] {
const tokens: Token[] = []
let i = 0
while (i < query.length) {
const rest = query.slice(i)
if (/^\s/.test(rest)) {
i++
continue
}
if (rest[0] === '(' || rest[0] === ')') {
tokens.push({ type: rest[0] as '(' | ')' })
i++
continue
}
const keyword = KEYWORD_TOKEN.exec(rest)
if (keyword) {
tokens.push({ type: keyword[1] as 'AND' | 'OR' | 'NOT' })
i += keyword[1].length
continue
}
const fieldMatch = FIELD_PREFIX.exec(rest)
const field = fieldMatch ? fieldMatch[1] : ''
i += fieldMatch ? fieldMatch[0].length : 0
const value = query.slice(i)
if (value.startsWith('"')) {
const end = query.indexOf('"', i + 1)
if (end < 0) throw new UnreadableQuery()
tokens.push({
type: 'term',
field,
value: query.slice(i + 1, end),
quoted: true,
})
i = end + 1
} else if (value.startsWith('[')) {
const end = query.indexOf(']', i + 1)
if (end < 0) throw new UnreadableQuery()
tokens.push({
type: 'term',
field,
value: query.slice(i + 1, end),
range: true,
})
i = end + 1
} else {
const bare = /^[^\s()]+/.exec(value)
if (!bare) throw new UnreadableQuery()
tokens.push({ type: 'term', field, value: bare[0] })
i += bare[0].length
}
// Nothing may run on directly after a value, e.g. title:"a"b
if (i < query.length && !/[\s)]/.test(query[i])) throw new UnreadableQuery()
}
return tokens
}
function resolveField(name: string): AdvancedSearchField {
const fields = Object.values(AdvancedSearchField) as string[]
// Aliases (type:, path:, notes:) are left to the text box on purpose:
// reading one would mean rewriting the user's query as it was read
if (!fields.includes(name)) throw new UnreadableQuery()
return name as AdvancedSearchField
}
function atomFrom(
field: AdvancedSearchField,
operator: AdvancedSearchOperator,
value: string,
extra: Partial<AdvancedSearchQueryAtom> = {}
): AdvancedSearchQueryAtom {
return {
type: AdvancedSearchQueryElementType.Atom,
field,
operator,
value,
...extra,
}
}
function parseRange(
field: AdvancedSearchField,
kind: AdvancedSearchFieldKind,
body: string
): AdvancedSearchQueryAtom {
const bounds = RANGE_BOUNDS.exec(body)
if (!bounds) throw new UnreadableQuery()
const lo = bounds[1] ?? ''
const hi = bounds[2] ?? ''
if (kind === AdvancedSearchFieldKind.Date) {
const relative = RELATIVE_BOUND.exec(lo)
if (relative && hi === 'now') {
return atomFrom(field, AdvancedSearchOperator.WithinLast, relative[1], {
unit: relative[2] as AdvancedSearchDateUnit,
})
}
} else if (kind !== AdvancedSearchFieldKind.Number) {
throw new UnreadableQuery()
}
const isValid =
kind === AdvancedSearchFieldKind.Date
? (v: string) => ISO_DATE.test(v)
: (v: string) => WHOLE_NUMBER.test(v)
if (lo && hi) {
if (!isValid(lo) || !isValid(hi)) throw new UnreadableQuery()
return atomFrom(field, AdvancedSearchOperator.Between, lo, { valueTo: hi })
}
if (lo && isValid(lo))
return atomFrom(field, AdvancedSearchOperator.AtLeast, lo)
if (hi && isValid(hi))
return atomFrom(field, AdvancedSearchOperator.AtMost, hi)
throw new UnreadableQuery()
}
function parseTerm(token: Token): Parsed {
const field = resolveField(token.field)
const kind = ADVANCED_SEARCH_FIELD_KINDS[field]
const value = token.value
const isKeyword = (
ADVANCED_SEARCH_DATE_KEYWORDS as readonly string[]
).includes(value)
if (token.range) {
return { element: parseRange(field, kind, value) }
}
if (kind === AdvancedSearchFieldKind.Date) {
if (!isKeyword) throw new UnreadableQuery()
return {
element: atomFrom(field, AdvancedSearchOperator.DateKeyword, value),
}
}
if (token.quoted) {
if (kind !== AdvancedSearchFieldKind.Text) throw new UnreadableQuery()
return { element: atomFrom(field, AdvancedSearchOperator.Phrase, value) }
}
const wildcard = TRAILING_WILDCARD.exec(value)
if (wildcard) {
if (kind === AdvancedSearchFieldKind.Number) throw new UnreadableQuery()
return {
element: atomFrom(field, AdvancedSearchOperator.StartsWith, wildcard[1]),
}
}
if (kind === AdvancedSearchFieldKind.Number) {
if (!WHOLE_NUMBER.test(value)) throw new UnreadableQuery()
return { element: atomFrom(field, AdvancedSearchOperator.Equals, value) }
}
// A checksum is only ever searched by its first characters
if (kind === AdvancedSearchFieldKind.Checksum) throw new UnreadableQuery()
return {
element: atomFrom(field, AdvancedSearchOperator.AllWords, value),
word: { field, text: value },
}
}
// The serializer repeats the field for every word, because a field applies
// only to the word after it. Put those back together into one condition.
function mergeWords(
parts: Parsed[],
operator: AdvancedSearchLogicalOperator.And | AdvancedSearchLogicalOperator.Or
): AdvancedSearchQueryElement[] {
const merged: AdvancedSearchQueryElement[] = []
for (let i = 0; i < parts.length; i++) {
const run = [parts[i]]
while (
parts[i].word &&
parts[i + 1]?.word &&
parts[i + 1].word.field === parts[i].word.field
) {
run.push(parts[++i])
}
if (run.length === 1) {
merged.push(run[0].element)
continue
}
merged.push(
atomFrom(
run[0].word.field,
operator === AdvancedSearchLogicalOperator.And
? AdvancedSearchOperator.AllWords
: AdvancedSearchOperator.AnyWord,
run.map((part) => part.word.text).join(' ')
)
)
}
return merged
}
interface Cursor {
tokens: Token[]
at: number
}
function parseExpression(cursor: Cursor): Parsed {
const parts: Parsed[] = [parseOperand(cursor)]
let operator:
AdvancedSearchLogicalOperator.And | AdvancedSearchLogicalOperator.Or
while (
cursor.tokens[cursor.at]?.type === 'AND' ||
cursor.tokens[cursor.at]?.type === 'OR'
) {
const next = cursor.tokens[cursor.at++].type as
AdvancedSearchLogicalOperator.And | AdvancedSearchLogicalOperator.Or
// One level mixing AND and OR is never something the editor wrote
if (operator && next !== operator) throw new UnreadableQuery()
operator = next
parts.push(parseOperand(cursor))
}
if (parts.length === 1) return parts[0]
const children = mergeWords(parts, operator)
if (children.length === 1) return { element: children[0] }
return {
element: {
type: AdvancedSearchQueryElementType.Group,
operator,
children,
},
}
}
function parseOperand(cursor: Cursor): Parsed {
const token = cursor.tokens[cursor.at++]
if (!token) throw new UnreadableQuery()
if (token.type === 'NOT') {
const child = parseOperand(cursor)
return {
element: {
type: AdvancedSearchQueryElementType.Group,
operator: AdvancedSearchLogicalOperator.Not,
children: [child.element],
},
}
}
if (token.type === '(') {
const inner = parseExpression(cursor)
if (cursor.tokens[cursor.at++]?.type !== ')') throw new UnreadableQuery()
return { element: inner.element }
}
if (token.type !== 'term') throw new UnreadableQuery()
return parseTerm(token)
}
/**
* Reads a query the editor could have written back into an editor tree, or
* returns null when the editor cannot show it, in which case the query stays
* text. Never returns a tree that would be written back differently.
*/
export function parseAdvancedSearchQuery(
query: string
): AdvancedSearchQueryGroup | null {
const trimmed = query?.trim() ?? ''
if (!trimmed) return null
let parsed: Parsed
try {
const cursor: Cursor = { tokens: tokenize(trimmed), at: 0 }
parsed = parseExpression(cursor)
if (cursor.at !== cursor.tokens.length) throw new UnreadableQuery()
} catch {
return null
}
const root =
parsed.element.type === AdvancedSearchQueryElementType.Group
? parsed.element
: {
type: AdvancedSearchQueryElementType.Group as const,
operator: AdvancedSearchLogicalOperator.And,
children: [parsed.element],
}
return serializeAdvancedSearchQuery(root) === trimmed ? root : null
}
+1 -1
View File
@@ -8,7 +8,7 @@ export const environment = {
apiVersion: '10', // match src/paperless/settings.py
appTitle: DEFAULT_APP_TITLE,
tag: 'prod',
version: '3.2.0',
version: '3.2.1',
webSocketHost: window.location.host,
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
webSocketBaseUrl: base_url.pathname + 'ws/',
+5 -5
View File
@@ -2855,7 +2855,7 @@
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
<context context-type="linenumber">104</context>
</context-group>
<target state="needs-translation">Apply AI Suggestions</target>
<target state="translated">KI-Vorschläge anwenden</target>
</trans-unit>
<trans-unit id="9172233176401579786" datatype="html" approved="yes">
<source>Scheduled</source>
@@ -3907,7 +3907,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">2,3</context>
</context-group>
<target state="needs-translation">Share links</target>
<target state="translated">Share Links</target>
</trans-unit>
<trans-unit id="7844706011418789951" datatype="html" approved="yes">
<source>Administration</source>
@@ -5835,7 +5835,7 @@
<context context-type="sourcefile">src/app/components/common/edit-dialog/mail-rule-edit-dialog/mail-rule-edit-dialog.component.ts</context>
<context context-type="linenumber">182</context>
</context-group>
<target state="needs-translation">Error retrieving correspondents</target>
<target state="translated">Fehler beim Abrufen der Korrespondenten</target>
</trans-unit>
<trans-unit id="3407606224361082860" datatype="html">
<source>Error retrieving document types</source>
@@ -9623,7 +9623,7 @@
<context context-type="sourcefile">src/app/components/document-detail/document-detail.component.ts</context>
<context context-type="linenumber">1051</context>
</context-group>
<target state="needs-translation">You have unsaved changes to the content of this version.</target>
<target state="translated">Sie haben ungespeicherte Änderungen am Inhalt dieser Version.</target>
</trans-unit>
<trans-unit id="85184271222513014" datatype="html">
<source>Switching versions will discard them.</source>
@@ -12574,7 +12574,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">4,5</context>
</context-group>
<target state="needs-translation">Manage public links to individual documents and document bundles.</target>
<target state="translated">Verwalten Sie öffentliche Links zu einzelnen Dokumenten und Dokumentenbündeln.</target>
</trans-unit>
<trans-unit id="3740311348735113099" datatype="html">
<source>Document links</source>
+14 -14
View File
@@ -3907,7 +3907,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">2,3</context>
</context-group>
<target state="needs-translation">Share links</target>
<target state="translated">Condividi link</target>
</trans-unit>
<trans-unit id="7844706011418789951" datatype="html">
<source>Administration</source>
@@ -12411,7 +12411,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">40,41</context>
</context-group>
<target state="needs-translation">Expired</target>
<target state="translated">Scaduto</target>
</trans-unit>
<trans-unit id="2057574872309338088" datatype="html">
<source>Copy share link</source>
@@ -12455,7 +12455,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-bundle-list/share-link-bundle-list.component.html</context>
<context context-type="linenumber">164,165</context>
</context-group>
<target state="needs-translation">Share link bundles pagination</target>
<target state="translated">Paginazione dei pacchetti di link di condivisione</target>
</trans-unit>
<trans-unit id="1541995860059283227" datatype="html">
<source>Failed to load share link bundles.</source>
@@ -12511,7 +12511,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">7,8</context>
</context-group>
<target state="needs-translation"> No document share links currently exist. </target>
<target state="translated"> Attualmente non sono presenti link per la condivisione dei documenti. </target>
</trans-unit>
<trans-unit id="894965666917873756" datatype="html">
<source>Document</source>
@@ -12519,7 +12519,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">15,16</context>
</context-group>
<target state="needs-translation">Document</target>
<target state="translated">Documento</target>
</trans-unit>
<trans-unit id="7420713064779772535" datatype="html">
<source>Delete share link</source>
@@ -12527,7 +12527,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">75,76</context>
</context-group>
<target state="needs-translation">Delete share link</target>
<target state="translated">Elimina link di condivisione</target>
</trans-unit>
<trans-unit id="5198271726831144068" datatype="html">
<source>Share links pagination</source>
@@ -12535,7 +12535,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">104,105</context>
</context-group>
<target state="needs-translation">Share links pagination</target>
<target state="translated">Paginazione link condivisi</target>
</trans-unit>
<trans-unit id="2976350639944567244" datatype="html">
<source>Failed to load share links.</source>
@@ -12543,7 +12543,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">105</context>
</context-group>
<target state="needs-translation">Failed to load share links.</target>
<target state="translated">Impossibile caricare i link di condivisione.</target>
</trans-unit>
<trans-unit id="4100863040928608880" datatype="html">
<source>Error retrieving share links.</source>
@@ -12551,7 +12551,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">107</context>
</context-group>
<target state="needs-translation">Error retrieving share links.</target>
<target state="translated">Errore nel recuperare i link di condivisione.</target>
</trans-unit>
<trans-unit id="326120283142363299" datatype="html">
<source>Share link deleted.</source>
@@ -12559,7 +12559,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">154</context>
</context-group>
<target state="needs-translation">Share link deleted.</target>
<target state="translated">Link di condivisione eliminato.</target>
</trans-unit>
<trans-unit id="4964931417181298145" datatype="html">
<source>Error deleting share link.</source>
@@ -12567,7 +12567,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">159</context>
</context-group>
<target state="needs-translation">Error deleting share link.</target>
<target state="translated">Errore nell'eliminare il link condivisione.</target>
</trans-unit>
<trans-unit id="3439104787814858626" datatype="html">
<source>Manage public links to individual documents and document bundles.</source>
@@ -12575,7 +12575,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">4,5</context>
</context-group>
<target state="needs-translation">Manage public links to individual documents and document bundles.</target>
<target state="translated">Gestisci i link pubblici ai singoli documenti e pacchetti di documenti.</target>
</trans-unit>
<trans-unit id="3740311348735113099" datatype="html">
<source>Document links</source>
@@ -12583,7 +12583,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">18,19</context>
</context-group>
<target state="needs-translation">Document links</target>
<target state="translated">Link al documento</target>
</trans-unit>
<trans-unit id="8216780576869093031" datatype="html">
<source>Bundles</source>
@@ -12591,7 +12591,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">26,27</context>
</context-group>
<target state="needs-translation">Bundles</target>
<target state="translated">Pacchetti</target>
</trans-unit>
<trans-unit id="1229748338333965418" datatype="html">
<source>Use workflows to customize the behavior of Paperless-ngx when events &apos;trigger&apos; a workflow.</source>
+14 -14
View File
@@ -3907,7 +3907,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">2,3</context>
</context-group>
<target state="needs-translation">Share links</target>
<target state="translated">Ссылки для обмена</target>
</trans-unit>
<trans-unit id="7844706011418789951" datatype="html">
<source>Administration</source>
@@ -12410,7 +12410,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">40,41</context>
</context-group>
<target state="needs-translation">Expired</target>
<target state="translated">Истёкшие</target>
</trans-unit>
<trans-unit id="2057574872309338088" datatype="html">
<source>Copy share link</source>
@@ -12454,7 +12454,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-bundle-list/share-link-bundle-list.component.html</context>
<context context-type="linenumber">164,165</context>
</context-group>
<target state="needs-translation">Share link bundles pagination</target>
<target state="translated">Пагинация ссылок</target>
</trans-unit>
<trans-unit id="1541995860059283227" datatype="html">
<source>Failed to load share link bundles.</source>
@@ -12510,7 +12510,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">7,8</context>
</context-group>
<target state="needs-translation"> No document share links currently exist. </target>
<target state="translated"> В настоящее время нет общих ссылок. </target>
</trans-unit>
<trans-unit id="894965666917873756" datatype="html">
<source>Document</source>
@@ -12518,7 +12518,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">15,16</context>
</context-group>
<target state="needs-translation">Document</target>
<target state="translated">Документ</target>
</trans-unit>
<trans-unit id="7420713064779772535" datatype="html">
<source>Delete share link</source>
@@ -12526,7 +12526,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">75,76</context>
</context-group>
<target state="needs-translation">Delete share link</target>
<target state="translated">Удалить ссылку для обмена</target>
</trans-unit>
<trans-unit id="5198271726831144068" datatype="html">
<source>Share links pagination</source>
@@ -12534,7 +12534,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">104,105</context>
</context-group>
<target state="needs-translation">Share links pagination</target>
<target state="translated">Пагинация ссылок</target>
</trans-unit>
<trans-unit id="2976350639944567244" datatype="html">
<source>Failed to load share links.</source>
@@ -12542,7 +12542,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">105</context>
</context-group>
<target state="needs-translation">Failed to load share links.</target>
<target state="translated">Не удалось загрузить ссылки.</target>
</trans-unit>
<trans-unit id="4100863040928608880" datatype="html">
<source>Error retrieving share links.</source>
@@ -12550,7 +12550,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">107</context>
</context-group>
<target state="needs-translation">Error retrieving share links.</target>
<target state="translated">Ошибка при получении ссылок.</target>
</trans-unit>
<trans-unit id="326120283142363299" datatype="html">
<source>Share link deleted.</source>
@@ -12558,7 +12558,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">154</context>
</context-group>
<target state="needs-translation">Share link deleted.</target>
<target state="translated">Ссылка для обмена удалена.</target>
</trans-unit>
<trans-unit id="4964931417181298145" datatype="html">
<source>Error deleting share link.</source>
@@ -12566,7 +12566,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">159</context>
</context-group>
<target state="needs-translation">Error deleting share link.</target>
<target state="translated">Ошибка при удалении ссылки.</target>
</trans-unit>
<trans-unit id="3439104787814858626" datatype="html">
<source>Manage public links to individual documents and document bundles.</source>
@@ -12574,7 +12574,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">4,5</context>
</context-group>
<target state="needs-translation">Manage public links to individual documents and document bundles.</target>
<target state="translated">Управление публичными ссылками на отдельные документы и комплекты документов.</target>
</trans-unit>
<trans-unit id="3740311348735113099" datatype="html">
<source>Document links</source>
@@ -12582,7 +12582,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">18,19</context>
</context-group>
<target state="needs-translation">Document links</target>
<target state="translated">Ссылки на документы</target>
</trans-unit>
<trans-unit id="8216780576869093031" datatype="html">
<source>Bundles</source>
@@ -12590,7 +12590,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">26,27</context>
</context-group>
<target state="needs-translation">Bundles</target>
<target state="translated">Комплекты</target>
</trans-unit>
<trans-unit id="1229748338333965418" datatype="html">
<source>Use workflows to customize the behavior of Paperless-ngx when events &apos;trigger&apos; a workflow.</source>
+15 -15
View File
@@ -2855,7 +2855,7 @@
<context context-type="sourcefile">src/app/components/admin/tasks/tasks.component.ts</context>
<context context-type="linenumber">104</context>
</context-group>
<target state="needs-translation">Apply AI Suggestions</target>
<target state="translated">Uporabi predloge umetne inteligence</target>
</trans-unit>
<trans-unit id="9172233176401579786" datatype="html">
<source>Scheduled</source>
@@ -3907,7 +3907,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">2,3</context>
</context-group>
<target state="needs-translation">Share links</target>
<target state="translated">Delite povezave</target>
</trans-unit>
<trans-unit id="7844706011418789951" datatype="html">
<source>Administration</source>
@@ -12411,7 +12411,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">40,41</context>
</context-group>
<target state="needs-translation">Expired</target>
<target state="translated">Poteklo</target>
</trans-unit>
<trans-unit id="2057574872309338088" datatype="html">
<source>Copy share link</source>
@@ -12455,7 +12455,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-bundle-list/share-link-bundle-list.component.html</context>
<context context-type="linenumber">164,165</context>
</context-group>
<target state="needs-translation">Share link bundles pagination</target>
<target state="translated">Deljenje povezav – strani v sklopih</target>
</trans-unit>
<trans-unit id="1541995860059283227" datatype="html">
<source>Failed to load share link bundles.</source>
@@ -12511,7 +12511,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">7,8</context>
</context-group>
<target state="needs-translation"> No document share links currently exist. </target>
<target state="translated"> Trenutno ni povezav za skupno rabo dokumentov. </target>
</trans-unit>
<trans-unit id="894965666917873756" datatype="html">
<source>Document</source>
@@ -12519,7 +12519,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">15,16</context>
</context-group>
<target state="needs-translation">Document</target>
<target state="translated">Dokument</target>
</trans-unit>
<trans-unit id="7420713064779772535" datatype="html">
<source>Delete share link</source>
@@ -12527,7 +12527,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">75,76</context>
</context-group>
<target state="needs-translation">Delete share link</target>
<target state="translated">Izbriši povezavo za deljenje</target>
</trans-unit>
<trans-unit id="5198271726831144068" datatype="html">
<source>Share links pagination</source>
@@ -12535,7 +12535,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.html</context>
<context context-type="linenumber">104,105</context>
</context-group>
<target state="needs-translation">Share links pagination</target>
<target state="translated">Deljenje povezav – strani v sklopih</target>
</trans-unit>
<trans-unit id="2976350639944567244" datatype="html">
<source>Failed to load share links.</source>
@@ -12543,7 +12543,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">105</context>
</context-group>
<target state="needs-translation">Failed to load share links.</target>
<target state="translated">Povezav za deljenje ni bilo mogoče naložiti.</target>
</trans-unit>
<trans-unit id="4100863040928608880" datatype="html">
<source>Error retrieving share links.</source>
@@ -12551,7 +12551,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">107</context>
</context-group>
<target state="needs-translation">Error retrieving share links.</target>
<target state="translated">Napaka pri pridobivanju povezav za deljenje.</target>
</trans-unit>
<trans-unit id="326120283142363299" datatype="html">
<source>Share link deleted.</source>
@@ -12559,7 +12559,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">154</context>
</context-group>
<target state="needs-translation">Share link deleted.</target>
<target state="translated">Povezava za deljenje je bila izbrisana.</target>
</trans-unit>
<trans-unit id="4964931417181298145" datatype="html">
<source>Error deleting share link.</source>
@@ -12567,7 +12567,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-link-list/share-link-list.component.ts</context>
<context context-type="linenumber">159</context>
</context-group>
<target state="needs-translation">Error deleting share link.</target>
<target state="translated">Napaka pri brisanju povezave za deljenje.</target>
</trans-unit>
<trans-unit id="3439104787814858626" datatype="html">
<source>Manage public links to individual documents and document bundles.</source>
@@ -12575,7 +12575,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">4,5</context>
</context-group>
<target state="needs-translation">Manage public links to individual documents and document bundles.</target>
<target state="translated">Upravljajte javne povezave do posameznih dokumentov in paketov dokumentov.</target>
</trans-unit>
<trans-unit id="3740311348735113099" datatype="html">
<source>Document links</source>
@@ -12583,7 +12583,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">18,19</context>
</context-group>
<target state="needs-translation">Document links</target>
<target state="translated">Povezave do dokumentov</target>
</trans-unit>
<trans-unit id="8216780576869093031" datatype="html">
<source>Bundles</source>
@@ -12591,7 +12591,7 @@
<context context-type="sourcefile">src/app/components/manage/share-links/share-links.component.html</context>
<context context-type="linenumber">26,27</context>
</context-group>
<target state="needs-translation">Bundles</target>
<target state="translated">Paketi</target>
</trans-unit>
<trans-unit id="1229748338333965418" datatype="html">
<source>Use workflows to customize the behavior of Paperless-ngx when events &apos;trigger&apos; a workflow.</source>
+1 -1
View File
@@ -11518,7 +11518,7 @@
<context context-type="sourcefile">src/app/components/manage/document-attributes/custom-fields/custom-fields.component.html</context>
<context context-type="linenumber">70,72</context>
</context-group>
<target state="translated"/>
<target state="translated">Inga fält har definierats.</target>
</trans-unit>
<trans-unit id="3032792139967609806" datatype="html">
<source>Confirm delete field</source>
-2
View File
@@ -148,7 +148,6 @@ import {
send,
shop,
slashCircle,
sliders,
sliders2Vertical,
sortAlphaDown,
sortAlphaUpAlt,
@@ -398,7 +397,6 @@ const icons = {
send,
slashCircle,
shop,
sliders,
sliders2Vertical,
sortAlphaDown,
sortAlphaUpAlt,
+151
View File
@@ -0,0 +1,151 @@
"""Fixtures available to every Paperless-ngx app.
Loaded automatically for every test path. Keep module-scope imports minimal:
this file is imported for every session, so anything heavy belongs inside
the fixture body that needs it.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
if TYPE_CHECKING:
from collections.abc import Generator
from pathlib import Path
from django.contrib.auth.models import User
from pytest_django.fixtures import Settings
from rest_framework.test import APIClient
from paperless_testing.dirs import PaperlessDirs
from paperless_testing.fakes.progress import FakeProgressManager
@pytest.fixture(scope="session", autouse=True)
def faker_session_locale() -> str:
"""Pin Faker's locale so generated data does not follow the host locale.
The seed itself is left to pytest-randomly, which derives one per run.
"""
return "en_US"
@pytest.fixture(autouse=True)
def _fast_password_hasher(settings: Settings) -> None:
"""Hash test passwords with MD5 instead of Django's default PBKDF2.
PBKDF2 is deliberately slow, and every ``admin_user`` or
``create_superuser`` call pays for it: about 600 ms each. No test depends
on the hash format, only on ``check_password`` and on the stored value
changing when the password does.
"""
settings.PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
@pytest.fixture(autouse=True)
def _clear_content_type_caches() -> None:
"""Clear Django's ContentType cache and guardian's lru_cache before each test.
Tests that delete and reinsert ContentType/Permission rows (e.g. the
importer) corrupt both caches. Without this fixture a subsequent test on
the same xdist worker sees stale ContentType objects and guardian raises
MixedContentTypeError.
"""
from django.contrib.contenttypes.models import ContentType
from guardian.shortcuts import clear_ct_cache
ContentType.objects.clear_cache()
clear_ct_cache()
@pytest.fixture(autouse=True)
def _clear_django_caches() -> None:
"""Clear every configured cache before each test.
Cached values outlive the test that wrote them: the classifier keys its
vectorized content on a hash of the content itself, so a second test
generating the same fixture data takes the cache-hit path and never calls
the code it is asserting against.
"""
from django.core.cache import caches
for cache in caches.all(initialized_only=False):
cache.clear()
@pytest.fixture
def paperless_dirs(
tmp_path: Path,
settings: Settings,
) -> Generator[PaperlessDirs, None, None]:
"""The standard temp directory layout, applied to Django settings."""
from documents.search import reset_backend
from paperless_testing.dirs import build_paperless_dirs
from paperless_testing.dirs import dirs_settings
dirs = build_paperless_dirs(tmp_path)
for name, value in dirs_settings(dirs).items():
setattr(settings, name, value)
# Not directory settings, but they are needed alongside the layout by the
# sanity checker tests.
settings.IGNORABLE_FILES = {".DS_Store", "Thumbs.db", "desktop.ini"}
settings.APP_LOGO = ""
reset_backend()
yield dirs
reset_backend()
@pytest.fixture
def rest_api_client() -> APIClient:
"""The basic DRF APIClient, unauthenticated."""
from rest_framework.test import APIClient
return APIClient()
@pytest.fixture
def regular_user(db: None) -> User:
"""Unprivileged user for permission boundary tests."""
from paperless_testing.factories import UserFactory
return UserFactory(username="regular")
@pytest.fixture
def admin_client(rest_api_client: APIClient, admin_user: User) -> APIClient:
"""Admin client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture
def v9_client(rest_api_client: APIClient, admin_user: User) -> APIClient:
"""Admin client pre-authenticated and sending the v9 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=9")
return rest_api_client
@pytest.fixture
def user_client(rest_api_client: APIClient, regular_user: User) -> APIClient:
"""Regular-user client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=regular_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture
def fake_progress_manager(
monkeypatch: pytest.MonkeyPatch,
) -> type[FakeProgressManager]:
"""Replace documents.tasks.ProgressManager with the fake, so consuming a file
in a test never tries to reach a broker."""
from paperless_testing.fakes.progress import FakeProgressManager
monkeypatch.setattr("documents.tasks.ProgressManager", FakeProgressManager)
return FakeProgressManager
-1
View File
@@ -26,7 +26,6 @@ class DocumentsConfig(AppConfig):
document_consumption_finished.connect(set_document_type)
document_consumption_finished.connect(set_tags)
document_consumption_finished.connect(set_storage_path)
document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(run_workflows_added)
document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(add_or_update_document_in_llm_index)
+3 -2
View File
@@ -857,8 +857,9 @@ class ConsumerPlugin(
self.log.debug(f"Creation date from parse_date: {create_date}")
else:
stats = Path(self.input_doc.original_file).stat()
create_date = timezone.make_aware(
datetime.datetime.fromtimestamp(stats.st_mtime),
create_date = datetime.datetime.fromtimestamp(
stats.st_mtime,
tz=timezone.get_current_timezone(),
)
self.log.debug(f"Creation date from st_mtime: {create_date}")
+50 -63
View File
@@ -196,52 +196,49 @@ class WriteBatch:
return self._raw_writer
def __enter__(self) -> Self:
if self._backend._path is not None:
lock_path = self._backend._path / ".tantivy.lock"
self._lock = filelock.FileLock(str(lock_path))
for attempt in range(_LOCK_RETRY_ATTEMPTS):
try:
self._lock.acquire(timeout=self._lock_timeout)
break
except filelock.Timeout:
if attempt == _LOCK_RETRY_ATTEMPTS - 1:
raise SearchIndexLockError(
f"Could not acquire index lock after {_LOCK_RETRY_ATTEMPTS} "
f"attempts (timeout={self._lock_timeout}s each)",
)
sleep_s = random.uniform(
0,
min(_LOCK_BACKOFF_CAP, _LOCK_BACKOFF_BASE * (2**attempt)),
lock_path = self._backend._path / ".tantivy.lock"
self._lock = filelock.FileLock(str(lock_path))
for attempt in range(_LOCK_RETRY_ATTEMPTS):
try:
self._lock.acquire(timeout=self._lock_timeout)
break
except filelock.Timeout:
if attempt == _LOCK_RETRY_ATTEMPTS - 1:
raise SearchIndexLockError(
f"Could not acquire index lock after {_LOCK_RETRY_ATTEMPTS} "
f"attempts (timeout={self._lock_timeout}s each)",
)
logger.debug(
"Index lock contention; retrying in %.2fs (attempt %d/%d)",
sleep_s,
attempt + 1,
_LOCK_RETRY_ATTEMPTS,
)
time.sleep(sleep_s)
sleep_s = random.uniform(
0,
min(_LOCK_BACKOFF_CAP, _LOCK_BACKOFF_BASE * (2**attempt)),
)
logger.debug(
"Index lock contention; retrying in %.2fs (attempt %d/%d)",
sleep_s,
attempt + 1,
_LOCK_RETRY_ATTEMPTS,
)
time.sleep(sleep_s)
# Open a fresh Index (and thus a fresh Tantivy ManagedDirectory)
# for the write, rather than reusing the process-local cached
# index. ManagedDirectory loads its GC bookkeeping (.managed.json)
# once, at construction, and never re-reads it; paperless runs
# several long-lived processes (Granian workers, Celery workers)
# that take turns writing under the file lock above. A cached,
# long-lived writer index would carry a stale managed-files view
# and, on commit, overwrite .managed.json with that stale view -
# permanently losing track of segment files other processes
# registered in the meantime, so they can never be garbage
# collected. Reopening fresh here always picks up the current
# on-disk state. The long-lived self._backend._index is used for
# reads only and is reloaded (not reopened) after commit below.
write_index = tantivy.Index(
build_schema(),
path=str(self._backend._path),
)
register_tokenizers(write_index, settings.SEARCH_LANGUAGE)
self._raw_writer = write_index.writer()
else:
self._raw_writer = self._backend._index.writer()
# Open a fresh Index (and thus a fresh Tantivy ManagedDirectory)
# for the write, rather than reusing the process-local cached
# index. ManagedDirectory loads its GC bookkeeping (.managed.json)
# once, at construction, and never re-reads it; paperless runs
# several long-lived processes (Granian workers, Celery workers)
# that take turns writing under the file lock above. A cached,
# long-lived writer index would carry a stale managed-files view
# and, on commit, overwrite .managed.json with that stale view -
# permanently losing track of segment files other processes
# registered in the meantime, so they can never be garbage
# collected. Reopening fresh here always picks up the current
# on-disk state. The long-lived self._backend._index is used for
# reads only and is reloaded (not reopened) after commit below.
write_index = tantivy.Index(
build_schema(),
path=str(self._backend._path),
)
register_tokenizers(write_index, settings.SEARCH_LANGUAGE)
self._raw_writer = write_index.writer()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
@@ -372,9 +369,8 @@ class TantivyBackend:
Tantivy search backend with explicit lifecycle management.
Provides full-text search capabilities using the Tantivy search engine.
Supports in-memory indexes (for testing) and persistent on-disk indexes
(for production use). Handles document indexing, search queries, autocompletion,
and "more like this" functionality.
Keeps a persistent on-disk index. Handles document indexing, search queries,
autocompletion, and "more like this" functionality.
The backend manages its own connection lifecycle and can be reset when
the underlying index directory changes (e.g., during test isolation).
@@ -408,9 +404,7 @@ class TantivyBackend:
},
)
def __init__(self, path: Path | None = None):
# path=None → in-memory index (for tests)
# path=some_dir → on-disk index (for production)
def __init__(self, path: Path):
self._path = path
self._raw_index: tantivy.Index | None = None
self._raw_schema: tantivy.Schema | None = None
@@ -429,16 +423,13 @@ class TantivyBackend:
"""
Open or rebuild the index as needed.
For disk-based indexes, checks if rebuilding is needed due to schema
version or language changes. Registers custom tokenizers after opening.
Checks if rebuilding is needed due to schema version or language
changes. Registers custom tokenizers after opening.
Safe to call multiple times - subsequent calls are no-ops.
"""
if self._raw_index is not None:
return # pragma: no cover
if self._path is not None:
self._raw_index = open_or_rebuild_index(self._path)
else:
self._raw_index = tantivy.Index(build_schema())
self._raw_index = open_or_rebuild_index(self._path)
register_tokenizers(self._raw_index, settings.SEARCH_LANGUAGE)
self._raw_schema = self._raw_index.schema
@@ -1102,13 +1093,9 @@ class TantivyBackend:
writer's threads). Larger values buffer more docs in RAM before
flushing a segment, deferring merge work; they do not avoid it.
"""
# Create new index (on-disk or in-memory)
if self._path is not None:
wipe_index(self._path)
new_index = tantivy.Index(build_schema(), path=str(self._path))
_write_sentinels(self._path)
else:
new_index = tantivy.Index(build_schema())
wipe_index(self._path)
new_index = tantivy.Index(build_schema(), path=str(self._path))
_write_sentinels(self._path)
register_tokenizers(new_index, settings.SEARCH_LANGUAGE)
# Point instance at the new index so _build_tantivy_doc uses it
+3 -1
View File
@@ -2098,6 +2098,8 @@ class BulkEditSerializer(
if not isinstance(parameters["pages"], str):
raise serializers.ValidationError("invalid pages specified")
page_count = Document.objects.get(id=document_id).page_count
if not page_count:
raise serializers.ValidationError("document page count is unknown")
pages = []
for group in parameters["pages"].split(","):
start, is_range, end = group.partition("-")
@@ -2107,7 +2109,7 @@ class BulkEditSerializer(
except ValueError as e:
raise serializers.ValidationError("invalid pages specified") from e
# Bound the range before building it, a huge one would exhaust memory
if not 1 <= first <= last or (page_count and last > page_count):
if not 1 <= first <= last <= page_count:
raise serializers.ValidationError("invalid pages specified")
pages.append(list(range(first, last + 1)))
parameters["pages"] = pages
+6 -1
View File
@@ -56,6 +56,7 @@ from documents.permissions import get_objects_for_user_owner_aware
from documents.plugins.helpers import DocumentsStatusManager
from documents.templating.utils import convert_format_str_to_template_format
from documents.utils import compute_checksum
from documents.utils import copy_file_with_basic_stats
from documents.workflows.actions import build_workflow_action_context
from documents.workflows.actions import execute_email_action
from documents.workflows.actions import execute_move_to_trash_action
@@ -363,7 +364,11 @@ def cleanup_document_deletion(sender, instance, **kwargs) -> None:
logger.debug(f"Moving {instance.source_path} to trash at {new_file_path}")
try:
shutil.move(instance.source_path, new_file_path)
shutil.move(
instance.source_path,
new_file_path,
copy_function=copy_file_with_basic_stats,
)
except OSError as e:
logger.error(
f"Failed to move {instance.source_path} to trash at "
+18 -143
View File
@@ -1,88 +1,41 @@
import shutil
import zoneinfo
from collections.abc import Generator
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
import filelock
import pytest
from django.contrib.auth import get_user_model
from django.contrib.contenttypes.models import ContentType
from guardian.shortcuts import clear_ct_cache
from pytest_django.fixtures import Settings
from rest_framework.test import APIClient
from documents.tests.factories import DocumentFactory
UserModelT = get_user_model()
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from documents.models import Document
@dataclass(frozen=True, slots=True)
class PaperlessDirs:
"""Standard Paperless-ngx directory layout for tests."""
media: Path
originals: Path
archive: Path
thumbnails: Path
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture(scope="session")
def samples_dir() -> Path:
def document_samples_dir() -> Path:
"""Path to the shared test sample documents."""
return Path(__file__).parent / "samples" / "documents"
@pytest.fixture()
def paperless_dirs(tmp_path: Path) -> PaperlessDirs:
"""Create and return the directory structure for testing."""
media = tmp_path / "media"
dirs = PaperlessDirs(
media=media,
originals=media / "documents" / "originals",
archive=media / "documents" / "archive",
thumbnails=media / "documents" / "thumbnails",
)
for d in (dirs.originals, dirs.archive, dirs.thumbnails):
d.mkdir(parents=True)
return dirs
@pytest.fixture()
def _media_settings(paperless_dirs: PaperlessDirs, settings) -> None:
"""Configure Django settings to point at temp directories."""
settings.MEDIA_ROOT = paperless_dirs.media
settings.ORIGINALS_DIR = paperless_dirs.originals
settings.ARCHIVE_DIR = paperless_dirs.archive
settings.THUMBNAIL_DIR = paperless_dirs.thumbnails
settings.MEDIA_LOCK = paperless_dirs.media / "media.lock"
settings.IGNORABLE_FILES = {".DS_Store", "Thumbs.db", "desktop.ini"}
settings.APP_LOGO = ""
@pytest.fixture()
def sample_doc(
paperless_dirs: PaperlessDirs,
_media_settings: None,
samples_dir: Path,
paperless_dirs: "PaperlessDirs",
document_samples_dir: Path,
) -> "Document":
"""Create a document with valid files and matching checksums."""
with filelock.FileLock(paperless_dirs.media / "media.lock"):
with filelock.FileLock(paperless_dirs.media_lock):
shutil.copy(
samples_dir / "originals" / "0000001.pdf",
paperless_dirs.originals / "0000001.pdf",
document_samples_dir / "originals" / "0000001.pdf",
paperless_dirs.originals_dir / "0000001.pdf",
)
shutil.copy(
samples_dir / "archive" / "0000001.pdf",
paperless_dirs.archive / "0000001.pdf",
document_samples_dir / "archive" / "0000001.pdf",
paperless_dirs.archive_dir / "0000001.pdf",
)
shutil.copy(
samples_dir / "thumbnails" / "0000001.webp",
paperless_dirs.thumbnails / "0000001.webp",
document_samples_dir / "thumbnails" / "0000001.webp",
paperless_dirs.thumbnail_dir / "0000001.webp",
)
return DocumentFactory(
@@ -97,95 +50,17 @@ def sample_doc(
)
@pytest.fixture()
def _search_index(
tmp_path: Path,
settings: Settings,
) -> Generator[None, None, None]:
"""Create a temp index directory and point INDEX_DIR at it.
@pytest.fixture
def _search_index(paperless_dirs: "PaperlessDirs") -> None:
"""Point the search backend at a fresh, empty index directory.
Resets the backend singleton before and after so each test gets a clean
index rather than reusing a stale singleton from another test.
paperless_dirs owns INDEX_DIR and resets the backend singleton on both
sides of the test, so requesting it is all that is needed.
"""
from documents.search import reset_backend
index_dir = tmp_path / "index"
index_dir.mkdir()
settings.INDEX_DIR = index_dir
reset_backend()
yield
reset_backend()
@pytest.fixture()
def settings_timezone(settings: Settings) -> zoneinfo.ZoneInfo:
return zoneinfo.ZoneInfo(settings.TIME_ZONE)
@pytest.fixture
def rest_api_client():
"""
The basic DRF ApiClient
"""
yield APIClient()
@pytest.fixture()
def regular_user(django_user_model: type[UserModelT]) -> UserModelT:
"""Unprivileged authenticated user for permission boundary tests."""
return django_user_model.objects.create_user(username="regular", password="regular")
@pytest.fixture()
def admin_client(rest_api_client: APIClient, admin_user: UserModelT) -> APIClient:
"""Admin client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture()
def v9_client(rest_api_client: APIClient, admin_user: UserModelT) -> APIClient:
"""Admin client pre-authenticated and sending the v9 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=9")
return rest_api_client
@pytest.fixture()
def user_client(rest_api_client: APIClient, regular_user: UserModelT) -> APIClient:
"""Regular-user client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=regular_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture(autouse=True)
def _clear_content_type_caches() -> None:
"""Clear Django's ContentType cache and guardian's lru_cache before each test.
Tests that delete and reinsert ContentType/Permission rows (e.g. the
importer) corrupt both caches. Without this fixture a subsequent test on
the same xdist worker sees stale ContentType objects and guardian raises
MixedContentTypeError.
"""
ContentType.objects.clear_cache()
clear_ct_cache()
@pytest.fixture(scope="session", autouse=True)
def faker_session_locale():
"""Set Faker locale for reproducibility."""
return "en_US"
@pytest.fixture(scope="session", autouse=True)
def faker_seed():
return 12345
@pytest.fixture
def indexed_document(_search_index: None) -> "Document":
def searchable_document(_search_index: None) -> "Document":
"""One searchable document, for tests about what the search endpoint
returns rather than about what it finds.
"""
+10
View File
@@ -0,0 +1,10 @@
import re
def dummy_preprocess(content: str) -> str:
"""
Simpler, faster pre-processing for testing purposes
"""
content = content.lower().strip()
content = re.sub(r"\s+", " ", content)
return content
@@ -15,11 +15,11 @@ from rich.console import Console
from documents.management.commands.document_sanity_checker import Command
from documents.sanity_checker import SanityCheckMessages
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from documents.models import Document
from documents.tests.conftest import PaperlessDirs
from paperless_testing.dirs import PaperlessDirs
def _render_to_string(messages: SanityCheckMessages) -> str:
@@ -71,7 +71,7 @@ class TestRenderResultsWithIssues:
assert "INFO" in output
assert "No OCR data" in output
@pytest.mark.usefixtures("_media_settings")
@pytest.mark.usefixtures("paperless_dirs")
def test_global_message(self) -> None:
msgs = SanityCheckMessages()
msgs.warning(None, "Orphaned file: /tmp/stray.pdf")
@@ -87,7 +87,7 @@ class TestRenderResultsWithIssues:
assert "Thumbnail missing" in output
assert "Checksum mismatch" in output
@pytest.mark.usefixtures("_media_settings")
@pytest.mark.usefixtures("paperless_dirs")
def test_unknown_doc_pk(self) -> None:
msgs = SanityCheckMessages()
msgs.error(99999, "Ghost document")
@@ -184,7 +184,6 @@ class TestDocumentSanityCheckerCommand:
assert "ERROR" in output
assert "Original of document does not exist" in output
@pytest.mark.usefixtures("_media_settings")
def test_checksum_mismatch(self, paperless_dirs: PaperlessDirs) -> None:
"""Lightweight document with zero-byte files triggers checksum mismatch."""
doc = DocumentFactory(
+4 -12
View File
@@ -9,29 +9,21 @@ from documents.search._backend import TantivyBackend
from documents.search._backend import reset_backend
from documents.search._schema import build_schema
from documents.search._tokenizer import register_tokenizers
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from collections.abc import Callable
from collections.abc import Generator
from pathlib import Path
from pytest_django.fixtures import Settings
from documents.models import Document
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture
def index_dir(tmp_path: Path, settings: Settings) -> Path:
path = tmp_path / "index"
path.mkdir()
settings.INDEX_DIR = path
return path
@pytest.fixture
def backend() -> Generator[TantivyBackend, None, None]:
b = TantivyBackend() # path=None → in-memory index
def backend(paperless_dirs: PaperlessDirs) -> Generator[TantivyBackend, None, None]:
b = TantivyBackend(path=paperless_dirs.index_dir)
b.open()
try:
yield b
@@ -14,7 +14,6 @@ from typing import TYPE_CHECKING
import pytest
import time_machine
from django.contrib.auth.models import User
from documents.models import CustomField
from documents.models import CustomFieldInstance
@@ -22,7 +21,8 @@ from documents.models import DocumentType
from documents.models import Note
from documents.models import StoragePath
from documents.search._query import parse_user_query
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
if TYPE_CHECKING:
from collections.abc import Callable
@@ -148,7 +148,7 @@ class TestJsonSubpaths:
THEN:
- Only the document with alice's note matches
"""
alice = User.objects.create_user(username="alice")
alice = UserFactory(username="alice")
doc_with_note = DocumentFactory(
title="Has note",
content="x",
+17 -16
View File
@@ -3,10 +3,8 @@ from pathlib import Path
import pytest
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from django.db import connection
from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from pytest_mock import MockerFixture
from documents.models import CustomField
@@ -19,11 +17,12 @@ from documents.search._backend import WriteBatch
from documents.search._backend import get_backend
from documents.search._backend import reset_backend
from documents.signals.handlers import add_to_index
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory
from documents.tests.factories import DocumentTypeFactory
from documents.tests.factories import TagFactory
from documents.tests.factories import UserFactory
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_object
pytestmark = [pytest.mark.search, pytest.mark.django_db]
@@ -189,7 +188,7 @@ class TestAddOrUpdateIds:
pk=1,
owner=owner,
)
assign_perm("view_document", user, doc)
grant_object(user, doc, "view_document")
with backend.batch_update() as batch:
batch.add_or_update_ids([doc.pk])
@@ -209,7 +208,7 @@ class TestAddOrUpdateIds:
pk=1,
owner=owner,
)
assign_perm("view_document", group, doc)
grant_object(group, doc, "view_document")
with backend.batch_update() as batch:
batch.add_or_update_ids([doc.pk])
@@ -763,8 +762,8 @@ class TestSearchIds:
def test_respects_permission_filter(self, backend: TantivyBackend) -> None:
"""search_ids must respect user permission filtering."""
owner = User.objects.create_user("ids_owner")
other = User.objects.create_user("ids_other")
owner = UserFactory(username="ids_owner")
other = UserFactory(username="ids_other")
doc = Document.objects.create(
title="private doc",
content="secret keyword",
@@ -843,7 +842,7 @@ class TestRebuild:
content="group secret keyword",
owner=owner,
)
assign_perm("view_document", group, doc)
grant_object(group, doc, "view_document")
backend.rebuild(Document.objects.all())
@@ -948,7 +947,8 @@ class TestSingleton:
yield
reset_backend()
def test_returns_same_instance_on_repeated_calls(self, index_dir) -> None:
@pytest.mark.usefixtures("paperless_dirs")
def test_returns_same_instance_on_repeated_calls(self) -> None:
"""Singleton pattern: repeated calls to get_backend() must return the same instance."""
assert get_backend() is get_backend()
@@ -965,7 +965,8 @@ class TestSingleton:
assert b1 is not b2
assert b2._path == tmp_path / "b"
def test_reset_forces_new_instance(self, index_dir) -> None:
@pytest.mark.usefixtures("paperless_dirs")
def test_reset_forces_new_instance(self) -> None:
"""reset_backend() must force creation of a new backend instance on next get_backend() call."""
b1 = get_backend()
reset_backend()
@@ -1071,7 +1072,7 @@ class TestFieldHandling:
def test_notes_include_user_information(self, backend: TantivyBackend) -> None:
"""Notes must be indexed with user information when available for structured queries."""
user = User.objects.create_user("notewriter")
user = UserFactory(username="notewriter")
doc = Document.objects.create(
title="Doc with notes",
content="test",
@@ -1173,7 +1174,7 @@ class TestHighlightHits:
notes.note: prefix so the query targets notes content directly, but
the snippet is generated from notes_text which stores the same text.
"""
user = User.objects.create_user("hl_noteuser")
user = UserFactory(username="hl_noteuser")
doc = Document.objects.create(
title="Doc with matching note",
content="unrelated content",
@@ -27,7 +27,7 @@ import time_machine
from documents.models import Note
from documents.models import Tag
from documents.search._errors import InvalidDateQuery
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from collections.abc import Callable
@@ -269,7 +269,7 @@ class TestDocumentedDateForms:
yield
@pytest.fixture
def dated(self, index_document: Callable[..., Document]) -> dict[str, int]:
def dated(self, backend: TantivyBackend) -> dict[str, int]:
stamps = {
"today": datetime(2026, 6, 15, 9, 0, tzinfo=UTC),
"yesterday": datetime(2026, 6, 14, 9, 0, tzinfo=UTC),
@@ -279,14 +279,14 @@ class TestDocumentedDateForms:
"january": datetime(2026, 1, 10, 10, 0, tzinfo=UTC),
"old": datetime(2005, 3, 4, 15, 30, tzinfo=UTC),
}
return {
label: index_document(
title=label,
content="dated body",
added=stamp,
).pk
docs = {
label: DocumentFactory(title=label, content="dated body", added=stamp)
for label, stamp in stamps.items()
}
with backend.batch_update() as batch:
for doc in docs.values():
batch.add_or_update(doc)
return {label: doc.pk for label, doc in docs.items()}
@pytest.mark.parametrize(
("query", "label"),
@@ -29,7 +29,7 @@ from rest_framework import status
from documents.search._backend import SearchMode
from documents.search._query import parse_simple_text_highlight_query
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from rest_framework.test import APIClient
@@ -17,12 +17,12 @@ from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
from django.contrib.auth.models import User
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Note
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
if TYPE_CHECKING:
from collections.abc import Callable
@@ -53,7 +53,7 @@ class TestBareJsonFieldPrefixes:
decoy's content match does not resurface through a demoted
text search
"""
alice = User.objects.create_user(username="alice")
alice = UserFactory(username="alice")
with_note = DocumentFactory(title="Has note", content="x")
Note.objects.create(document=with_note, user=alice, note="crocodile")
backend.add_or_update(with_note)
@@ -116,7 +116,7 @@ class TestBareJsonFieldPrefixes:
document; the default-subpath resolution for the bare
prefix does not interfere with explicit subpath addressing
"""
bob = User.objects.create_user(username="bob")
bob = UserFactory(username="bob")
doc = DocumentFactory(title="Bob note", content="x")
Note.objects.create(document=doc, user=bob, note="remark")
backend.add_or_update(doc)
@@ -20,7 +20,6 @@ from typing import TYPE_CHECKING
import pytest
import tantivy
from django.contrib.auth.models import User
from whoosh_compat import FieldKind
from documents.models import CustomField
@@ -28,6 +27,7 @@ from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import Note
from documents.search._fields import PUBLIC_FIELDS
from paperless_testing.factories import UserFactory
if TYPE_CHECKING:
from documents.search._backend import TantivyBackend
@@ -49,7 +49,7 @@ class TestJsonSubpathsAreWrittenAtIndexTime:
- Every subpath PUBLIC_FIELDS declares for notes/custom_fields
is present as a key in the document's stored JSON payload
"""
user = User.objects.create_user(username="completeness-user")
user = UserFactory(username="completeness-user")
field = CustomField.objects.create(
name="Completeness Field",
data_type=CustomField.FieldDataType.STRING,
@@ -15,7 +15,7 @@ from documents.search._backend import SearchIndexLockError
from documents.search._backend import TantivyBackend
from documents.tasks import index_document
from documents.tasks import remove_document_from_index
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from collections.abc import Generator
@@ -1,6 +1,6 @@
import pytest
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
pytestmark = pytest.mark.search
@@ -18,13 +18,14 @@ from typing import TYPE_CHECKING
import pytest
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from documents.models import Correspondent
from documents.models import Document
from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_object
if TYPE_CHECKING:
from documents.search._backend import TantivyBackend
@@ -34,22 +35,22 @@ pytestmark = [pytest.mark.search, pytest.mark.django_db]
@pytest.fixture
def owner() -> User:
return User.objects.create_user(username="owner")
return UserFactory(username="owner")
@pytest.fixture
def stranger() -> User:
return User.objects.create_user(username="stranger")
return UserFactory(username="stranger")
@pytest.fixture
def viewer() -> User:
return User.objects.create_user(username="viewer")
return UserFactory(username="viewer")
@pytest.fixture
def group_member() -> User:
user = User.objects.create_user(username="group_member")
user = UserFactory(username="group_member")
user.groups.add(Group.objects.create(name="accounting"))
return user
@@ -127,7 +128,7 @@ class TestPermissionFilteringOnIndexedDocuments:
checksum="perm-shared-user",
owner=owner,
)
assign_perm("view_document", viewer, doc)
grant_object(viewer, doc, "view_document")
backend.add_or_update(doc)
assert backend.search_ids("invoice", user=viewer) == [doc.pk]
@@ -157,7 +158,7 @@ class TestPermissionFilteringOnIndexedDocuments:
checksum="perm-shared-group",
owner=owner,
)
assign_perm("view_document", group_member.groups.first(), doc)
grant_object(group_member.groups.first(), doc, "view_document")
backend.add_or_update(doc)
assert backend.search_ids("invoice", user=group_member) == [doc.pk]
+22 -19
View File
@@ -13,11 +13,11 @@ from documents.search._schema import needs_rebuild
from documents.search._schema import schema_fingerprint
if TYPE_CHECKING:
from pathlib import Path
import tantivy
from pytest_django.fixtures import Settings
from paperless_testing.dirs import PaperlessDirs
pytestmark = pytest.mark.search
@@ -25,16 +25,19 @@ pytestmark = pytest.mark.search
class TestNeedsRebuild:
"""needs_rebuild covers all sentinel-file states that require a full reindex."""
def test_returns_true_when_settings_file_missing(self, index_dir: Path) -> None:
assert needs_rebuild(index_dir) is True
def test_returns_true_when_settings_file_missing(
self,
paperless_dirs: PaperlessDirs,
) -> None:
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_false_when_version_and_language_match(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = "en"
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps(
{
"schema_version": SCHEMA_VERSION,
@@ -43,51 +46,51 @@ class TestNeedsRebuild:
},
),
)
assert needs_rebuild(index_dir) is False
assert needs_rebuild(paperless_dirs.index_dir) is False
def test_returns_true_on_schema_version_mismatch(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = None
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION - 1, "language": None}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_true_when_version_is_not_an_integer(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = None
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": "not-a-number", "language": None}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_true_when_language_key_missing(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = "en"
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_true_when_language_differs(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = "de"
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION, "language": "en"}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def _schema_fields(schema: tantivy.Schema) -> dict[str, dict]:
@@ -35,6 +35,8 @@ if TYPE_CHECKING:
from pytest_django.fixtures import SettingsWrapper
from paperless_testing.dirs import PaperlessDirs
pytestmark = pytest.mark.search
# The on-disk field layout of a v2 index, pinned as data. Any edit here is an
@@ -469,7 +471,7 @@ def _fingerprint_of(descriptors: list[FieldDescriptor]) -> str:
class TestNeedsRebuildOnFingerprint:
def test_matching_fingerprint_does_not_rebuild(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
) -> None:
"""
@@ -482,13 +484,13 @@ class TestNeedsRebuildOnFingerprint:
- It returns False
"""
settings.SEARCH_LANGUAGE = None
_sentinels(index_dir)
_sentinels(paperless_dirs.index_dir)
assert needs_rebuild(index_dir) is False
assert needs_rebuild(paperless_dirs.index_dir) is False
def test_stale_fingerprint_rebuilds_despite_a_matching_version(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -505,7 +507,7 @@ class TestNeedsRebuildOnFingerprint:
every subsequent write would raise
"""
settings.SEARCH_LANGUAGE = None
_sentinels(index_dir)
_sentinels(paperless_dirs.index_dir)
extended = [
*field_descriptors(),
FieldDescriptor(
@@ -519,11 +521,11 @@ class TestNeedsRebuildOnFingerprint:
]
monkeypatch.setattr(_schema, "field_descriptors", lambda: extended)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_reordered_schema_rebuilds(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -538,16 +540,16 @@ class TestNeedsRebuildOnFingerprint:
- It returns True
"""
settings.SEARCH_LANGUAGE = None
_sentinels(index_dir)
_sentinels(paperless_dirs.index_dir)
reordered = field_descriptors()
reordered[1], reordered[2] = reordered[2], reordered[1]
monkeypatch.setattr(_schema, "field_descriptors", lambda: reordered)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_missing_fingerprint_rebuilds(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
) -> None:
"""
@@ -561,15 +563,15 @@ class TestNeedsRebuildOnFingerprint:
is rebuilt rather than trusted
"""
settings.SEARCH_LANGUAGE = None
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION, "language": None}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_written_sentinels_satisfy_the_check(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
) -> None:
"""
@@ -582,6 +584,6 @@ class TestNeedsRebuildOnFingerprint:
- It returns False
"""
settings.SEARCH_LANGUAGE = "en"
_write_sentinels(index_dir)
_write_sentinels(paperless_dirs.index_dir)
assert needs_rebuild(index_dir) is False
assert needs_rebuild(paperless_dirs.index_dir) is False
+4 -4
View File
@@ -16,11 +16,11 @@ from documents.models import Document
from documents.models import Tag
from documents.search import get_backend
from documents.search import reset_backend
from documents.tests.factories import DocumentFactory
from documents.tests.factories import TagFactory
from documents.tests.factories import UserFactory
from documents.tests.utils import DirectoriesMixin
from paperless.admin import PaperlessUserAdmin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
@pytest.fixture
+49 -5
View File
@@ -3,7 +3,6 @@ from io import BytesIO
from pathlib import Path
from unittest.mock import patch
from django.contrib.auth.models import User
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import override_settings
from PIL import Image
@@ -11,10 +10,11 @@ from PIL.PngImagePlugin import PngInfo
from rest_framework import status
from rest_framework.test import APITestCase
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import read_streaming_response
from paperless.models import ApplicationConfiguration
from paperless.models import ColorConvertChoices
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
class TestApiAppConfig(DirectoriesMixin, APITestCase):
@@ -23,7 +23,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
user = User.objects.create_superuser(username="temp_admin")
user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=user)
def test_api_get_config(self) -> None:
@@ -81,6 +81,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
"ai_enabled": None,
"llm_embedding_backend": None,
"llm_embedding_model": None,
"llm_embedding_api_key": None,
"llm_embedding_endpoint": None,
"llm_embedding_chunk_size": None,
"llm_context_size": None,
@@ -267,7 +268,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
THEN:
- old app_logo file is deleted
"""
admin = User.objects.create_superuser(username="admin")
admin = UserFactory(username="admin", superuser=True)
self.client.force_login(user=admin)
response = self.client.get("/logo/")
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
@@ -922,6 +923,49 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED)
self.assertEqual(ApplicationConfiguration.objects.count(), 1)
def test_update_llm_embedding_api_key(self) -> None:
"""
GIVEN:
- Existing config with llm_embedding_api_key specified
WHEN:
- API to update llm_embedding_api_key is called with all *s
- API to update llm_embedding_api_key is called with empty string
THEN:
- llm_embedding_api_key is unchanged
- llm_embedding_api_key is set to None
"""
config = ApplicationConfiguration.objects.first()
assert config is not None
config.llm_embedding_api_key = "1234567890"
config.save()
# Test with all *
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"llm_embedding_api_key": "*" * 32,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.llm_embedding_api_key, "1234567890")
# Test with empty string
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"llm_embedding_api_key": "",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.llm_embedding_api_key, None)
def test_update_llm_api_key(self) -> None:
"""
GIVEN:
+16 -10
View File
@@ -4,8 +4,6 @@ import json
import shutil
import zipfile
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from django.utils import timezone
from rest_framework import status
@@ -14,9 +12,11 @@ from rest_framework.test import APITestCase
from documents.models import Correspondent
from documents.models import Document
from documents.models import DocumentType
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import SampleDirMixin
from documents.tests.utils import read_streaming_response
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
@@ -25,7 +25,7 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
self.doc1 = Document.objects.create(title="unrelated", checksum="A")
@@ -166,7 +166,15 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
),
content_type="application/json",
)
response.close()
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response["Content-Type"], "application/zip")
with zipfile.ZipFile(io.BytesIO(read_streaming_response(response))) as zipf:
self.assertEqual(zipf.infolist()[0].compress_type, zipfile.ZIP_LZMA)
with self.doc2.source_file as f:
self.assertEqual(f.read(), zipf.read("2021-01-01 document A.pdf"))
@override_settings(FILENAME_FORMAT="{correspondent}/{title}")
def test_formatted_download_originals(self) -> None:
@@ -326,10 +334,8 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
)
def test_download_insufficient_permissions(self) -> None:
user = User.objects.create_user(username="temp_user")
user.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
user = UserFactory(username="temp_user")
grant_global(user, "view_document")
self.client.force_authenticate(user=user)
self.doc2.owner = self.user
+90 -43
View File
@@ -2,27 +2,30 @@ import json
from unittest import mock
from auditlog.models import LogEntry
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APITestCase
from documents.models import Correspondent
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestBulkEditAPI(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
user = User.objects.create_superuser(username="temp_admin")
user = UserFactory(username="temp_admin", superuser=True)
self.user = user
self.client.force_authenticate(user=user)
@@ -284,9 +287,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m,
) -> None:
self.setup_mock(m, "modify_custom_fields")
user = User.objects.create_user(username="doc-owner")
user.user_permissions.add(Permission.objects.get(codename="change_document"))
other_user = User.objects.create_user(username="other-user")
user = UserFactory(username="doc-owner")
grant_global(user, "change_document")
other_user = UserFactory(username="other-user")
source_doc = Document.objects.create(
checksum="source",
title="Source",
@@ -787,10 +790,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
@mock.patch("documents.serialisers.bulk_edit.set_storage_path")
def test_api_bulk_edit_with_all_true_resolves_owned_duplicates(self, m) -> None:
self.setup_mock(m, "set_storage_path")
user = User.objects.create_user(username="duplicate-owner")
user.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
user = UserFactory(username="duplicate-owner")
grant_global(user, "change_document")
first_duplicate = Document.objects.create(
checksum="owned-duplicate",
title="First duplicate",
@@ -1178,7 +1179,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
user1 = User.objects.create(username="user1")
self.client.force_authenticate(user=user1)
assign_perm("view_document", user1, self.doc2)
grant_object(user1, self.doc2, "view_document")
response = self.client.post(
"/api/documents/selection_data/",
@@ -1188,9 +1189,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
user1.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_global(user1, "view_document")
user1 = User.objects.get(pk=user1.pk)
self.client.force_authenticate(user=user1)
response = self.client.post(
@@ -1533,7 +1532,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save()
user1 = User.objects.create(username="user1")
user1.user_permissions.add(*Permission.objects.all())
grant_all_global(user1)
user1.save()
self.client.force_authenticate(user=user1)
@@ -1587,8 +1586,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save()
user1 = User.objects.create(username="user1")
assign_perm("view_document", user1, self.doc1)
user1.user_permissions.add(*Permission.objects.all())
grant_object(user1, self.doc1, "view_document")
grant_all_global(user1)
user1.save()
self.client.force_authenticate(user=user1)
@@ -1609,7 +1608,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m.assert_not_called()
self.assertEqual(response.content, b"Insufficient permissions")
assign_perm("change_document", user1, self.doc1)
grant_object(user1, self.doc1, "change_document")
response = self.client.post(
"/api/documents/bulk_edit/",
@@ -1786,6 +1785,36 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertIn(b"invalid pages specified", response.content)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.split")
def test_bulk_edit_split_rejects_unknown_page_count(self, m) -> None:
"""
GIVEN:
- A legacy split bulk edit of a document without a page count
WHEN:
- API to bulk edit is called
THEN:
- API returns HTTP 400
- split is not called
"""
self.setup_mock(m, "split")
for pages in ("1", "1-5000000"):
with self.subTest(pages=pages):
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc1.id],
"method": "split",
"parameters": {"pages": pages},
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn(b"document page count is unknown", response.content)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.split")
def test_bulk_edit_split_parses_pages(self, m) -> None:
"""
@@ -1819,7 +1848,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save()
user1 = User.objects.create(username="user1")
user1.user_permissions.add(*Permission.objects.all())
grant_all_global(user1)
user1.save()
self.client.force_authenticate(user=user1)
@@ -1880,7 +1909,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save()
user1 = User.objects.create(username="user1")
user1.user_permissions.add(*Permission.objects.all())
grant_all_global(user1)
user1.save()
self.client.force_authenticate(user=user1)
@@ -1919,11 +1948,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
@mock.patch("documents.views.bulk_edit.merge")
def test_merge_and_delete_requires_change_permission(self, m) -> None:
self.setup_mock(m, "merge")
user = User.objects.create_user(username="no-change")
user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="delete_document"),
)
user = UserFactory(username="no-change")
grant_global(user, "add_document", "delete_document")
self.client.force_authenticate(user=user)
response = self.client.post(
@@ -2310,7 +2336,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save()
user1 = User.objects.create(username="user1")
user1.user_permissions.add(*Permission.objects.all())
grant_all_global(user1)
user1.save()
self.client.force_authenticate(user=user1)
@@ -2345,7 +2371,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
@mock.patch("documents.views.bulk_edit.edit_pdf")
def test_edit_pdf_update_requires_change_permission(self, m) -> None:
self.setup_mock(m, "edit_pdf")
user = User.objects.create_user(username="no-change")
user = UserFactory(username="no-change")
self.client.force_authenticate(user=user)
response = self.client.post(
@@ -2372,11 +2398,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
) -> None:
self.setup_mock(edit_pdf_mock, "edit_pdf")
self.setup_mock(remove_password_mock, "remove_password")
user = User.objects.create_user(username="no-delete")
user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="change_document"),
)
user = UserFactory(username="no-delete")
grant_global(user, "add_document", "change_document")
self.client.force_authenticate(user=user)
cases = [
@@ -2463,7 +2486,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save()
user1 = User.objects.create(username="user1")
user1.user_permissions.add(*Permission.objects.all())
grant_all_global(user1)
user1.save()
self.client.force_authenticate(user=user1)
@@ -2503,7 +2526,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
WHEN:
- API to bulk edit documents is called
THEN:
- Audit log is created
- Audit log is created with the old and new correspondent
"""
LogEntry.objects.all().delete()
response = self.client.post(
@@ -2519,7 +2542,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 1)
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(entry.changes, {"correspondent": [None, self.c2.id]})
@override_settings(AUDIT_LOG_ENABLED=True)
def test_bulk_edit_audit_log_enabled_tags(self) -> None:
@@ -2527,16 +2551,18 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
GIVEN:
- Audit log is enabled
WHEN:
- API to bulk edit tags is called
- API to bulk edit tags is called on an untagged document and a
document with several tags
THEN:
- Audit log is created
- Audit log is created for each document with its full tag list
before and after the edit
"""
LogEntry.objects.all().delete()
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc1.id],
"documents": [self.doc1.id, self.doc4.id],
"method": "modify_tags",
"parameters": {
"add_tags": [self.t1.id],
@@ -2548,18 +2574,32 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 1)
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(entry.changes, {"tags": [[], [self.t1.id]]})
entry = LogEntry.objects.get_for_object(self.doc4).get()
self.assertEqual(
entry.changes,
{"tags": [[self.t1.id, self.t2.id], [self.t1.id]]},
)
@override_settings(AUDIT_LOG_ENABLED=True)
def test_bulk_edit_audit_log_enabled_custom_fields(self) -> None:
"""
GIVEN:
- Audit log is enabled
- A document with two custom fields
WHEN:
- API to bulk edit custom fields is called
- API to bulk edit custom fields is called to add a third
THEN:
- Audit log is created
- Audit log is created with every custom field instance before and
after the edit
- Audit log is created for the new custom field instance
"""
cf3 = CustomField.objects.create(name="cf3", data_type="string")
existing = [
CustomFieldInstance.objects.create(document=self.doc1, field=field)
for field in (self.cf2, cf3)
]
LogEntry.objects.all().delete()
response = self.client.post(
"/api/documents/bulk_edit/",
@@ -2577,7 +2617,14 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 2)
added = CustomFieldInstance.objects.get(document=self.doc1, field=self.cf1)
existing_ids = [instance.id for instance in existing]
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(
entry.changes,
{"custom_fields": [existing_ids, [*existing_ids, added.id]]},
)
self.assertEqual(LogEntry.objects.get_for_object(added).count(), 1)
def test_api_bulk_edit_with_bad_search_query_returns_400(self) -> None:
"""
+6 -6
View File
@@ -4,20 +4,22 @@ from typing import TYPE_CHECKING
from unittest import mock
import pytest
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from rest_framework import status
from rest_framework.test import APIClient
from rest_framework.test import APITestCase
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
if TYPE_CHECKING:
from django.contrib.auth.models import User
from pytest_mock import MockerFixture
class TestChatStreamingViewInputValidation(APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
def _mock_ai_enabled(self) -> mock.MagicMock:
@@ -113,9 +115,7 @@ class TestChatStreamingViewUnrestrictedFlag:
needs to reach the view at all. Model-level only: says nothing
about which documents (if any) this user can actually see.
"""
regular_user.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
grant_global(regular_user, "view_document")
return user_client
@pytest.mark.parametrize(
+13 -16
View File
@@ -4,23 +4,24 @@ from unittest import mock
from unittest.mock import ANY
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APITestCase
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
ENDPOINT = "/api/custom_fields/"
def setUp(self) -> None:
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
return super().setUp()
@@ -1174,11 +1175,9 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
def test_documentlink_patch_requires_change_permission_on_target_documents(
self,
) -> None:
source_owner = User.objects.create_user(username="source-owner")
source_owner.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
other_user = User.objects.create_user(username="other-user")
source_owner = UserFactory(username="source-owner")
grant_global(source_owner, "change_document")
other_user = UserFactory(username="other-user")
source_doc = Document.objects.create(
title="Source",
@@ -1221,11 +1220,9 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
def test_documentlink_patch_allowed_with_change_permission_on_target_documents(
self,
) -> None:
source_owner = User.objects.create_user(username="source-owner")
source_owner.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
other_user = User.objects.create_user(username="other-user")
source_owner = UserFactory(username="source-owner")
grant_global(source_owner, "change_document")
other_user = UserFactory(username="other-user")
source_doc = Document.objects.create(
title="Source",
@@ -1244,7 +1241,7 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
data_type=CustomField.FieldDataType.DOCUMENTLINK,
)
assign_perm("change_document", source_owner, target_doc)
grant_object(source_owner, target_doc, "change_document")
self.client.force_authenticate(user=source_owner)
resp = self.client.patch(
@@ -1337,7 +1334,7 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
self.assertEqual(results[0]["document_count"], 1)
# Test as user without access to the document
non_superuser = User.objects.create_user(username="non_superuser")
non_superuser = UserFactory(username="non_superuser")
non_superuser.user_permissions.add(
*Permission.objects.exclude(codename="view_global_statistics"),
)
@@ -5,8 +5,6 @@ from typing import TYPE_CHECKING
from unittest import mock
from auditlog.models import LogEntry # type: ignore[import-untyped]
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase as DjangoTestCase
@@ -18,10 +16,12 @@ from documents.data_models import DocumentSource
from documents.filters import EffectiveContentFilter
from documents.filters import TitleContentFilter
from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import read_streaming_response
from documents.versioning import annotate_effective_content
from documents.views import DocumentSelectionMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
if TYPE_CHECKING:
from pathlib import Path
@@ -31,7 +31,7 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
def _make_pdf_upload(self, name: str = "version.pdf") -> SimpleUploadedFile:
@@ -89,11 +89,9 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND)
def test_root_endpoint_returns_403_when_user_lacks_permission(self) -> None:
owner = User.objects.create_user(username="owner")
viewer = User.objects.create_user(username="viewer")
viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="owner")
viewer = UserFactory(username="viewer")
grant_global(viewer, "view_document")
root = Document.objects.create(
title="root",
checksum="root",
@@ -283,11 +281,9 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(mock_backend.add_or_update.call_args[0][0].id, root.id)
def test_delete_version_returns_403_without_permission(self) -> None:
owner = User.objects.create_user(username="owner")
other = User.objects.create_user(username="other")
other.user_permissions.add(
Permission.objects.get(codename="delete_document"),
)
owner = UserFactory(username="owner")
other = UserFactory(username="other")
grant_global(other, "delete_document")
root = Document.objects.create(
title="root",
checksum="root",
@@ -371,11 +367,9 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertTrue(resp.data["is_root"])
def test_update_version_label_returns_403_without_permission(self) -> None:
owner = User.objects.create_user(username="owner")
other = User.objects.create_user(username="other")
other.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
owner = UserFactory(username="owner")
other = UserFactory(username="other")
grant_global(other, "change_document")
root = Document.objects.create(
title="root",
checksum="root",
@@ -553,11 +547,9 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND)
def test_metadata_returns_403_when_user_lacks_permission(self) -> None:
owner = User.objects.create_user(username="owner")
other = User.objects.create_user(username="other")
other.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="owner")
other = UserFactory(username="other")
grant_global(other, "view_document")
doc = Document.objects.create(
title="root",
checksum="root",
@@ -653,8 +645,8 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_500_INTERNAL_SERVER_ERROR)
def test_update_version_returns_403_without_permission(self) -> None:
owner = User.objects.create_user(username="owner")
other = User.objects.create_user(username="other")
owner = UserFactory(username="owner")
other = UserFactory(username="other")
root = Document.objects.create(
title="root",
checksum="root",
@@ -672,8 +664,8 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
def test_update_version_requires_global_change_permission(self) -> None:
user = User.objects.create_user(username="add-only")
user.user_permissions.add(Permission.objects.get(codename="add_document"))
user = UserFactory(username="add-only")
grant_global(user, "add_document")
root = Document.objects.create(
title="root",
checksum="root",
@@ -978,7 +970,7 @@ class TestVersionAwareFilters(DjangoTestCase):
superseded content -- selecting documents the list view, filtered by
the same term, does not show.
"""
user = User.objects.create_superuser(username="bulk_selection")
user = UserFactory(username="bulk_selection", superuser=True)
selected = DocumentSelectionMixin()._resolve_document_ids(
user=user,
@@ -1005,7 +997,7 @@ class TestBulkSelectionExcludesVersions(DjangoTestCase):
"Select all matching" reconstructs the document list, which never
contains version documents as rows of their own.
"""
user = User.objects.create_superuser(username="bulk_versions")
user = UserFactory(username="bulk_versions", superuser=True)
root = Document.objects.create(
title="shared-title root",
checksum="bulk-root",
+87 -100
View File
@@ -23,7 +23,6 @@ from django.core.files.uploadedfile import SimpleUploadedFile
from django.db import DataError
from django.test import override_settings
from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APITestCase
@@ -48,18 +47,22 @@ from documents.models import Workflow
from documents.models import WorkflowAction
from documents.models import WorkflowTrigger
from documents.signals.handlers import run_workflows
from documents.tests.factories import DocumentFactory
from documents.tests.factories import TagFactory
from documents.tests.utils import ConsumeTaskMixin
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import read_streaming_response
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
cache.clear()
@@ -357,10 +360,10 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
with Path(filename).open("wb") as f:
f.write(content)
user1 = User.objects.create_user(username="test1")
user2 = User.objects.create_user(username="test2")
user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user1 = UserFactory(username="test1")
user2 = UserFactory(username="test2")
grant_global(user1, "view_document")
grant_global(user2, "view_document")
self.client.force_authenticate(user2)
@@ -383,7 +386,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
response = self.client.get(f"/api/documents/{doc.pk}/thumb/")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
assign_perm("view_document", user2, doc)
grant_object(user2, doc, "view_document")
response = self.client.get(f"/api/documents/{doc.pk}/download/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
@@ -760,8 +763,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- History is returned
"""
# No auditlog permissions
user = User.objects.create_user(username="test")
user.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user = UserFactory(username="test")
grant_global(user, "view_document")
self.client.force_authenticate(user=user)
doc = Document.objects.create(
title="First title",
@@ -776,7 +779,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
# superuser
user.is_superuser = True
user.save()
user2 = User.objects.create_user(username="test2")
user2 = UserFactory(username="test2")
doc2 = Document.objects.create(
title="Second title",
checksum="456",
@@ -1073,11 +1076,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["duplicate_documents"], [])
def test_has_duplicates_filter_respects_document_permissions(self) -> None:
owner = User.objects.create_user(username="duplicate-owner")
requester = User.objects.create_user(username="duplicate-requester")
requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="duplicate-owner")
requester = UserFactory(username="duplicate-requester")
grant_global(requester, "view_document")
visible_document = Document.objects.create(
title="visible document",
checksum="permission-match",
@@ -1096,7 +1097,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
[document["id"] for document in response.data["results"]],
)
assign_perm("view_document", requester, hidden_duplicate)
grant_object(requester, hidden_duplicate, "view_document")
response = self.client.get("/api/documents/?has_duplicates=true")
self.assertIn(
visible_document.id,
@@ -1317,10 +1318,10 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Owner filters work correctly but still respect permissions
"""
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
u2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
grant_global(u1, "view_document")
grant_global(u2, "view_document")
u1_doc1 = Document.objects.create(
title="none1",
@@ -1353,7 +1354,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
)
self.client.force_authenticate(user=u1)
assign_perm("view_document", u1, u2_doc2)
grant_object(u1, u2_doc2, "view_document")
# Will not show any u1 docs or u2_doc1 which isn't shared
response = self.client.get(f"/api/documents/?owner__id__none={u1.id}")
@@ -1400,7 +1401,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
[u1_doc1.id, u1_doc2.id, u2_doc2.id],
)
assign_perm("view_document", u2, u1_doc1)
grant_object(u2, u1_doc1, "view_document")
# Will show only documents shared by user
response = self.client.get(f"/api/documents/?shared_by__id={u1.id}")
@@ -1424,8 +1425,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- The document is returned exactly once, not once per permission path
(regression test for https://github.com/paperless-ngx/paperless-ngx/issues/13331)
"""
user = User.objects.create_user("user1")
user.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user = UserFactory(username="user1")
grant_global(user, "view_document")
group = Group.objects.create(name="group1")
user.groups.add(group)
@@ -1433,7 +1434,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
tag2 = TagFactory()
doc = DocumentFactory(title="shared", owner=user)
doc.tags.add(tag1, tag2)
assign_perm("view_document", group, doc)
grant_object(group, doc, "view_document")
self.client.force_authenticate(user=user)
response = self.client.get(
@@ -1452,11 +1453,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- The document does not appear in their results
"""
owner = User.objects.create_user("owner1")
stranger = User.objects.create_user("stranger1")
stranger.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
owner = UserFactory(username="owner1")
stranger = UserFactory(username="stranger1")
grant_global(stranger, "view_document")
DocumentFactory(title="private", owner=owner)
@@ -1474,17 +1473,17 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Only the group member sees the document
"""
owner = User.objects.create_user("owner2")
member = User.objects.create_user("member1")
non_member = User.objects.create_user("nonmember1")
owner = UserFactory(username="owner2")
member = UserFactory(username="member1")
non_member = UserFactory(username="nonmember1")
for u in (member, non_member):
u.user_permissions.add(*Permission.objects.filter(codename="view_document"))
grant_global(u, "view_document")
group = Group.objects.create(name="group2")
member.groups.add(group)
doc = DocumentFactory(title="shared2", owner=owner)
assign_perm("view_document", group, doc)
grant_object(group, doc, "view_document")
self.client.force_authenticate(user=member)
response = self.client.get("/api/documents/")
@@ -1785,8 +1784,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Statistics only include inbox counts for tags accessible by the user
"""
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
inbox_tag_u1 = Tag.objects.create(name="inbox_u1", is_inbox_tag=True, owner=u1)
Tag.objects.create(name="inbox_u2", is_inbox_tag=True, owner=u2)
doc_u1 = Document.objects.create(
@@ -1816,11 +1815,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["documents_inbox"], 0)
def test_statistics_with_statistics_permission(self) -> None:
owner = User.objects.create_user("owner")
stats_user = User.objects.create_user("stats-user")
stats_user.user_permissions.add(
Permission.objects.get(codename="view_global_statistics"),
)
owner = UserFactory(username="owner")
stats_user = UserFactory(username="stats-user")
grant_global(stats_user, "view_global_statistics")
inbox_tag = Tag.objects.create(
name="stats_inbox",
@@ -1986,7 +1983,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED)
def test_upload_insufficient_permissions(self) -> None:
self.client.force_authenticate(user=User.objects.create_user("testuser2"))
self.client.force_authenticate(user=UserFactory(username="testuser2"))
with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
response = self.client.post(
@@ -2782,9 +2779,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
mock_get_date_parser.assert_not_called()
def test_saved_views(self) -> None:
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u3 = User.objects.create_user("user3")
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
u3 = UserFactory(username="user3")
view_perm = Permission.objects.get(codename="view_savedview")
change_perm = Permission.objects.get(codename="change_savedview")
@@ -2807,9 +2804,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
sort_field="",
)
assign_perm("view_savedview", u1, v2)
assign_perm("change_savedview", u1, v2)
assign_perm("view_savedview", u1, v3)
grant_object(u1, v2, "view_savedview")
grant_object(u1, v2, "change_savedview")
grant_object(u1, v3, "view_savedview")
self.client.force_authenticate(user=u1)
@@ -3064,7 +3061,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertListEqual(saved_view_settings["sidebar_views_visible_ids"], [v2.id])
def test_saved_view_create_update_patch(self) -> None:
User.objects.create_user("user1")
UserFactory(username="user1")
view = {
"name": "test",
@@ -3127,7 +3124,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- Display options are updated
- Display fields are validated
"""
User.objects.create_user("user1")
UserFactory(username="user1")
view = {
"name": "test",
@@ -3568,11 +3565,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Notes are neither created nor deleted
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.all())
user1 = UserFactory(username="test1")
grant_all_global(user1)
user1.save()
user2 = User.objects.create_user(username="test2")
user2 = UserFactory(username="test2")
user2.save()
doc = Document.objects.create(
@@ -3592,7 +3589,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(resp.content, b"Insufficient permissions to view notes")
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
assign_perm("view_document", user1, doc)
grant_object(user1, doc, "view_document")
resp = self.client.post(
f"/api/documents/{doc.pk}/notes/",
@@ -3616,12 +3613,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_notes_require_global_document_permissions(self) -> None:
user = User.objects.create_user(username="note_editor")
user.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_note", "add_note", "delete_note"],
),
)
user = UserFactory(username="note_editor")
grant_global(user, "view_note", "add_note", "delete_note")
doc = Document.objects.create(
title="test",
mime_type="application/pdf",
@@ -3634,9 +3627,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
response = self.client.get(f"/api/documents/{doc.pk}/notes/")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
user.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_global(user, "view_document")
user = User.objects.get(pk=user.pk)
self.client.force_authenticate(user)
response = self.client.get(f"/api/documents/{doc.pk}/notes/")
@@ -3648,9 +3639,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
user.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
grant_global(user, "change_document")
user = User.objects.get(pk=user.pk)
self.client.force_authenticate(user)
response = self.client.post(
@@ -3797,12 +3786,12 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- Unique items are created
- Non-unique items are not allowed
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.filter(codename="add_tag"))
user1 = UserFactory(username="test1")
grant_global(user1, "add_tag")
user1.save()
user2 = User.objects.create_user(username="test2")
user2.user_permissions.add(*Permission.objects.filter(codename="add_tag"))
user2 = UserFactory(username="test2")
grant_global(user2, "add_tag")
user2.save()
# User 1 creates tag 1 owned by user 1 by default
@@ -3857,12 +3846,12 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- Unique items are created
- Non-unique items are not allowed on update
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.filter(codename="change_tag"))
user1 = UserFactory(username="test1")
grant_global(user1, "change_tag")
user1.save()
user2 = User.objects.create_user(username="test2")
user2.user_permissions.add(*Permission.objects.filter(codename="change_tag"))
user2 = UserFactory(username="test2")
grant_global(user2, "change_tag")
user2.save()
# Create name tag 1 owned by user 1
@@ -3993,11 +3982,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Links only shown if user has permissions
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.all())
user1 = UserFactory(username="test1")
grant_all_global(user1)
user1.save()
user2 = User.objects.create_user(username="test2")
user2 = UserFactory(username="test2")
user2.save()
doc = Document.objects.create(
@@ -4017,7 +4006,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(resp.content, b"Insufficient permissions to add share link")
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
assign_perm("change_document", user1, doc)
grant_object(user1, doc, "change_document")
resp = self.client.get(
f"/api/documents/{doc.pk}/share_links/",
@@ -4034,11 +4023,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Share link creation is denied until view permission is granted
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.filter(codename="add_sharelink"))
user1 = UserFactory(username="test1")
grant_global(user1, "add_sharelink")
user1.save()
user2 = User.objects.create_user(username="test2")
user2 = UserFactory(username="test2")
user2.save()
doc = Document.objects.create(
@@ -4060,7 +4049,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
)
self.assertEqual(create_resp.status_code, status.HTTP_403_FORBIDDEN)
assign_perm("view_document", user1, doc)
grant_object(user1, doc, "view_document")
create_resp = self.client.post(
"/api/share_links/",
@@ -4072,9 +4061,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
)
self.assertEqual(create_resp.status_code, status.HTTP_403_FORBIDDEN)
user1.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_global(user1, "view_document")
user1 = User.objects.get(pk=user1.pk)
self.client.force_authenticate(user1)
create_resp = self.client.post(
@@ -4097,11 +4084,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- ASN +1 from user2's doc is returned for user1
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.all())
user1 = UserFactory(username="test1")
grant_all_global(user1)
user1.save()
user2 = User.objects.create_user(username="test2")
user2 = UserFactory(username="test2")
user2.save()
doc1 = Document.objects.create(
@@ -4141,8 +4128,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- ASN 1 is returned
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.all())
user1 = UserFactory(username="test1")
grant_all_global(user1)
user1.save()
doc1 = Document.objects.create(
@@ -4170,7 +4157,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Explicit error is returned
"""
user1 = User.objects.create_superuser(username="test1")
user1 = UserFactory(username="test1", superuser=True)
self.client.force_authenticate(user1)
@@ -4348,8 +4335,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN:
- Error response is returned
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.all())
user1 = UserFactory(username="test1")
grant_all_global(user1)
user1.save()
doc = Document.objects.create(
@@ -4454,7 +4441,7 @@ class TestDocumentApiTagColors(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
@@ -4534,7 +4521,7 @@ class TestDocumentApiCustomFieldsSorting(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
self.doc1 = Document.objects.create(
+8 -8
View File
@@ -2,16 +2,16 @@ import json
import shutil
from unittest import mock
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.core import mail
from django.test import override_settings
from rest_framework import status
from rest_framework.test import APITestCase
from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import SampleDirMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
@@ -20,7 +20,7 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
self.doc1 = Document.objects.create(
@@ -303,8 +303,8 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
THEN:
- Forbidden response is returned
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user1 = UserFactory(username="test1")
grant_global(user1, "view_document")
doc_owned = Document.objects.create(
title="owned_doc",
@@ -338,8 +338,8 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
THEN:
- Request succeeds
"""
user1 = User.objects.create_user(username="test1")
user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user1 = UserFactory(username="test1")
grant_global(user1, "view_document")
self.client.force_authenticate(user1)
@@ -4,7 +4,6 @@ from collections.abc import Callable
from datetime import date
from urllib.parse import quote
from django.contrib.auth.models import User
from rest_framework.test import APITestCase
from documents.models import CustomField
@@ -13,7 +12,8 @@ from documents.models import Document
from documents.models import SavedView
from documents.models import SavedViewFilterRule
from documents.serialisers import DocumentSerializer
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
class DocumentWrapper:
@@ -35,7 +35,7 @@ class TestCustomFieldsSearch(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
# Create one custom field per type. The fields are called f"{type}_field".
+35 -54
View File
@@ -3,12 +3,10 @@ import json
from unittest import mock
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.db import connection
from django.test import override_settings
from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
from rest_framework import status
@@ -21,14 +19,17 @@ from documents.models import Document
from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestApiObjects(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
user = User.objects.create_superuser(username="temp_admin")
user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=user)
self.tag1 = Tag.objects.create(name="t1", is_inbox_tag=True)
@@ -153,7 +154,7 @@ class TestApiObjects(DirectoriesMixin, APITestCase):
# A newer document owned by another user must not leak through the
# aggregate for a non-superuser who cannot view it
other = User.objects.create_user(username="other")
other = UserFactory(username="other")
Document.objects.create(
mime_type="application/pdf",
correspondent=self.c1,
@@ -162,10 +163,8 @@ class TestApiObjects(DirectoriesMixin, APITestCase):
owner=other,
)
user = User.objects.create_user(username="regular")
user.user_permissions.add(
Permission.objects.get(codename="view_correspondent"),
)
user = UserFactory(username="regular")
grant_global(user, "view_correspondent")
self.client.force_authenticate(user=user)
response = self.client.get("/api/correspondents/?last_correspondence=true")
@@ -200,7 +199,7 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
user = User.objects.create_superuser(username="temp_admin")
user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=user)
self.sp1 = StoragePath.objects.create(name="sp1", path="Something/{checksum}")
@@ -455,11 +454,9 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
self.assertEqual(response.data, "folder/Something.pdf")
def test_test_storage_path_requires_document_view_permission(self) -> None:
owner = User.objects.create_user(username="owner")
unprivileged = User.objects.create_user(username="unprivileged")
unprivileged.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="owner")
unprivileged = UserFactory(username="unprivileged")
grant_global(unprivileged, "view_document")
document = Document.objects.create(
mime_type="application/pdf",
owner=owner,
@@ -481,15 +478,15 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
self.assertIn("document", response.data)
def test_test_storage_path_allows_shared_document_view_permission(self) -> None:
owner = User.objects.create_user(username="owner")
viewer = User.objects.create_user(username="viewer")
owner = UserFactory(username="owner")
viewer = UserFactory(username="viewer")
document = Document.objects.create(
mime_type="application/pdf",
owner=owner,
title="Shared",
checksum="123",
)
assign_perm("view_document", viewer, document)
grant_object(viewer, document, "view_document")
self.client.force_authenticate(user=viewer)
response = self.client.post(
@@ -504,9 +501,7 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_global(viewer, "view_document")
viewer = User.objects.get(pk=viewer.pk)
self.client.force_authenticate(user=viewer)
response = self.client.post(
@@ -545,14 +540,12 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_exposes_basic_document_context_but_not_sensitive_owner_data(
self,
) -> None:
owner = User.objects.create_user(
owner = UserFactory(
username="owner",
password="password",
email="owner@example.com",
)
owner.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_global(owner, "view_document")
document = Document.objects.create(
mime_type="application/pdf",
owner=owner,
@@ -614,8 +607,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_includes_related_objects_for_visible_document(
self,
) -> None:
owner = User.objects.create_user(username="owner")
viewer = User.objects.create_user(username="viewer")
owner = UserFactory(username="owner")
viewer = UserFactory(username="viewer")
private_correspondent = Correspondent.objects.create(
name="Private Correspondent",
owner=owner,
@@ -627,10 +620,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
title="Document",
checksum="123",
)
assign_perm("view_document", viewer, document)
viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_object(viewer, document, "view_document")
grant_global(viewer, "view_document")
self.client.force_authenticate(user=viewer)
response = self.client.post(
@@ -662,7 +653,7 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
self.assertEqual(response.data, "Private Correspondent.pdf")
def test_test_storage_path_superuser_can_view_private_related_objects(self) -> None:
owner = User.objects.create_user(username="owner")
owner = UserFactory(username="owner")
private_correspondent = Correspondent.objects.create(
name="Private Correspondent",
owner=owner,
@@ -693,8 +684,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_includes_doc_type_storage_path_and_tags(
self,
) -> None:
owner = User.objects.create_user(username="owner")
viewer = User.objects.create_user(username="viewer")
owner = UserFactory(username="owner")
viewer = UserFactory(username="viewer")
private_document_type = DocumentType.objects.create(
name="Private Type",
owner=owner,
@@ -717,10 +708,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
checksum="123",
)
document.tags.add(private_tag)
assign_perm("view_document", viewer, document)
viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_object(viewer, document, "view_document")
grant_global(viewer, "view_document")
self.client.force_authenticate(user=viewer)
response = self.client.post(
@@ -756,8 +745,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_includes_custom_fields_for_visible_document(
self,
) -> None:
owner = User.objects.create_user(username="owner")
viewer = User.objects.create_user(username="viewer")
owner = UserFactory(username="owner")
viewer = UserFactory(username="viewer")
document = Document.objects.create(
mime_type="application/pdf",
owner=owner,
@@ -773,10 +762,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
field=custom_field,
value_int=42,
)
assign_perm("view_document", viewer, document)
viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
grant_object(viewer, document, "view_document")
grant_global(viewer, "view_document")
self.client.force_authenticate(user=viewer)
response = self.client.post(
@@ -798,7 +785,7 @@ class TestBulkEditObjects(APITestCase):
def setUp(self) -> None:
super().setUp()
self.temp_admin = User.objects.create_superuser(username="temp_admin")
self.temp_admin = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.temp_admin)
self.t1 = Tag.objects.create(name="t1")
@@ -1030,9 +1017,7 @@ class TestBulkEditObjects(APITestCase):
THEN:
- User is able to delete objects
"""
self.user1.user_permissions.add(
*Permission.objects.filter(codename="delete_tag"),
)
grant_global(self.user1, "delete_tag")
self.user1.save()
self.client.force_authenticate(user=self.user1)
@@ -1062,9 +1047,7 @@ class TestBulkEditObjects(APITestCase):
self.t2.owner = User.objects.get(username="temp_admin")
self.t2.save()
self.user1.user_permissions.add(
*Permission.objects.filter(codename="delete_tag"),
)
grant_global(self.user1, "delete_tag")
self.user1.save()
self.client.force_authenticate(user=self.user1)
@@ -1097,9 +1080,7 @@ class TestBulkEditObjects(APITestCase):
self.t2.owner = User.objects.get(username="temp_admin")
self.t2.save()
self.user1.user_permissions.add(
*Permission.objects.filter(codename="delete_tag"),
)
grant_global(self.user1, "delete_tag")
self.user1.save()
self.client.force_authenticate(user=self.user1)
+62 -76
View File
@@ -5,9 +5,7 @@ from unittest import mock
from allauth.mfa.models import Authenticator
from allauth.mfa.totp.internal import auth as totp_auth
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from guardian.shortcuts import get_perms
from guardian.shortcuts import get_users_with_perms
from rest_framework import status
@@ -19,7 +17,11 @@ from documents.models import DocumentType
from documents.models import MatchingModel
from documents.models import StoragePath
from documents.models import Tag
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestApiAuth(DirectoriesMixin, APITestCase):
@@ -93,14 +95,14 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.assertNotIn("X-Version", response)
def test_api_version_with_auth(self) -> None:
user = User.objects.create_superuser(username="test")
user = UserFactory(username="test", superuser=True)
self.client.force_authenticate(user)
response = self.client.get("/api/documents/")
self.assertIn("X-Api-Version", response)
self.assertIn("X-Version", response)
def test_api_insufficient_permissions(self) -> None:
user = User.objects.create_user(username="test")
user = UserFactory(username="test")
self.client.force_authenticate(user)
Document.objects.create(title="Test")
@@ -137,8 +139,8 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
)
def test_api_sufficient_permissions(self) -> None:
user = User.objects.create_user(username="test")
user.user_permissions.add(*Permission.objects.all())
user = UserFactory(username="test")
grant_all_global(user)
user.is_staff = True
self.client.force_authenticate(user)
@@ -166,9 +168,9 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
)
def test_api_get_object_permissions(self) -> None:
user1 = User.objects.create_user(username="test1")
user2 = User.objects.create_user(username="test2")
user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user1 = UserFactory(username="test1")
user2 = UserFactory(username="test2")
grant_global(user1, "view_document")
self.client.force_authenticate(user1)
self.assertEqual(
@@ -205,7 +207,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN:
- Object created with current user as owner
"""
user1 = User.objects.create_superuser(username="user1")
user1 = UserFactory(username="user1", superuser=True)
self.client.force_authenticate(user1)
@@ -234,7 +236,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN:
- Object created with no owner
"""
user1 = User.objects.create_superuser(username="user1")
user1 = UserFactory(username="user1", superuser=True)
self.client.force_authenticate(user1)
@@ -265,7 +267,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN:
- Object permissions are set appropriately
"""
user1 = User.objects.create_superuser(username="user1")
user1 = UserFactory(username="user1", superuser=True)
user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1")
@@ -313,7 +315,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN:
- Object permissions are set appropriately
"""
user1 = User.objects.create_superuser(username="user1")
user1 = UserFactory(username="user1", superuser=True)
user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1")
@@ -363,7 +365,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
mime_type="application/pdf",
content="this is a document",
)
user1 = User.objects.create_superuser(username="user1")
user1 = UserFactory(username="user1", superuser=True)
user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1")
@@ -413,16 +415,16 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
mime_type="application/pdf",
content="this is a document",
)
user1 = User.objects.create_superuser(username="user1")
user1 = UserFactory(username="user1", superuser=True)
user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1")
doc.owner = user1
doc.save()
assign_perm("view_document", user2, doc)
assign_perm("change_document", user2, doc)
assign_perm("view_document", group1, doc)
assign_perm("change_document", group1, doc)
grant_object(user2, doc, "view_document")
grant_object(user2, doc, "change_document")
grant_object(group1, doc, "view_document")
grant_object(group1, doc, "change_document")
self.client.force_authenticate(user1)
@@ -446,11 +448,9 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.assertIn("change_document", get_perms(group1, doc))
def test_document_permissions_change_requires_owner(self) -> None:
owner = User.objects.create_user(username="owner")
editor = User.objects.create_user(username="editor")
editor.user_permissions.add(
*Permission.objects.all(),
)
owner = UserFactory(username="owner")
editor = UserFactory(username="editor")
grant_all_global(editor)
doc = Document.objects.create(
title="Ownered doc",
@@ -460,8 +460,8 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
owner=owner,
)
assign_perm("view_document", editor, doc)
assign_perm("change_document", editor, doc)
grant_object(editor, doc, "view_document")
grant_object(editor, doc, "change_document")
self.client.force_authenticate(editor)
response = self.client.patch(
@@ -499,9 +499,9 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_dynamic_permissions_fields(self) -> None:
user1 = User.objects.create_user(username="user1")
user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user2 = User.objects.create_user(username="user2")
user1 = UserFactory(username="user1")
grant_global(user1, "view_document")
user2 = UserFactory(username="user2")
Document.objects.create(title="Test", content="content 1", checksum="1")
doc2 = Document.objects.create(
@@ -523,10 +523,10 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
owner=user1,
)
assign_perm("view_document", user1, doc2)
assign_perm("view_document", user1, doc3)
assign_perm("change_document", user1, doc3)
assign_perm("view_document", user2, doc4)
grant_object(user1, doc2, "view_document")
grant_object(user1, doc3, "view_document")
grant_object(user1, doc3, "change_document")
grant_object(user2, doc4, "view_document")
self.client.force_authenticate(user1)
@@ -574,8 +574,8 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
owned by someone else with no explicit guardian grant -- mirrors
guardian's own ObjectPermissionChecker.has_perm() superuser shortcut.
"""
superuser = User.objects.create_superuser(username="admin")
other_user = User.objects.create_user(username="user2")
superuser = UserFactory(username="admin", superuser=True)
other_user = UserFactory(username="user2")
Document.objects.create(
title="Test",
content="content",
@@ -602,7 +602,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN:
- MFA required error is returned
"""
user1 = User.objects.create_user(username="user1")
user1 = UserFactory(username="user1")
user1.set_password("password")
user1.save()
@@ -626,7 +626,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN:
- MFA code is required
"""
user1 = User.objects.create_user(username="user1")
user1 = UserFactory(username="user1")
user1.set_password("password")
user1.save()
@@ -688,7 +688,7 @@ class TestApiUser(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
def test_get_users(self) -> None:
@@ -858,10 +858,8 @@ class TestApiUser(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
regular_user = User.objects.create_user(username="regular_user")
regular_user.user_permissions.add(
*Permission.objects.all(),
)
regular_user = UserFactory(username="regular_user")
grant_all_global(regular_user)
self.client.force_authenticate(regular_user)
Authenticator.objects.create(
user=user1,
@@ -885,9 +883,9 @@ class TestApiUser(DirectoriesMixin, APITestCase):
- Only superusers can change superuser status
"""
user1 = User.objects.create_user(username="user1")
user1.user_permissions.add(*Permission.objects.all())
user2 = User.objects.create_superuser(username="user2")
user1 = UserFactory(username="user1")
grant_all_global(user1)
user2 = UserFactory(username="user2", superuser=True)
self.client.force_authenticate(user1)
@@ -972,9 +970,9 @@ class TestApiUser(DirectoriesMixin, APITestCase):
- Only superusers can change staff status
"""
user1 = User.objects.create_user(username="user1")
user1.user_permissions.add(*Permission.objects.all())
user2 = User.objects.create_superuser(username="user2")
user1 = UserFactory(username="user1")
grant_all_global(user1)
user2 = UserFactory(username="user2", superuser=True)
self.client.force_authenticate(user1)
@@ -1027,7 +1025,7 @@ class TestApiGroup(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
def test_get_groups(self) -> None:
@@ -1128,7 +1126,7 @@ class TestBulkEditObjectPermissions(APITestCase):
def setUp(self) -> None:
super().setUp()
self.temp_admin = User.objects.create_superuser(username="temp_admin")
self.temp_admin = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.temp_admin)
self.t1 = Tag.objects.create(name="t1")
@@ -1276,7 +1274,7 @@ class TestBulkEditObjectPermissions(APITestCase):
},
}
assign_perm("view_tag", self.user3, self.t1)
grant_object(self.user3, self.t1, "view_tag")
self.t1.owner = self.user3
self.t1.save()
@@ -1373,13 +1371,9 @@ class TestBulkEditObjectPermissions(APITestCase):
"""
self.t1.owner = self.user2
self.t1.save()
assign_perm("view_tag", self.user1, self.t1)
assign_perm("change_tag", self.user1, self.t1)
self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag"],
),
)
grant_object(self.user1, self.t1, "view_tag")
grant_object(self.user1, self.t1, "change_tag")
grant_global(self.user1, "view_tag", "change_tag")
user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1)
@@ -1426,13 +1420,9 @@ class TestBulkEditObjectPermissions(APITestCase):
"""
owned = Tag.objects.create(name="owned", owner=self.user1)
shared = Tag.objects.create(name="shared", owner=self.user2)
assign_perm("view_tag", self.user1, shared)
assign_perm("change_tag", self.user1, shared)
self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag"],
),
)
grant_object(self.user1, shared, "view_tag")
grant_object(self.user1, shared, "change_tag")
grant_global(self.user1, "view_tag", "change_tag")
user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1)
@@ -1473,14 +1463,10 @@ class TestBulkEditObjectPermissions(APITestCase):
"""
self.t1.owner = self.user2
self.t1.save()
assign_perm("view_tag", self.user1, self.t1)
assign_perm("change_tag", self.user1, self.t1)
assign_perm("delete_tag", self.user1, self.t1)
self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag", "delete_tag"],
),
)
grant_object(self.user1, self.t1, "view_tag")
grant_object(self.user1, self.t1, "change_tag")
grant_object(self.user1, self.t1, "delete_tag")
grant_global(self.user1, "view_tag", "change_tag", "delete_tag")
user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1)
@@ -1585,7 +1571,7 @@ class TestBulkEditObjectPermissions(APITestCase):
- Request succeeds and null is treated as an empty user list,
so the existing view permission is removed
"""
assign_perm("view_tag", self.user1, self.t1)
grant_object(self.user1, self.t1, "view_tag")
response = self.client.post(
"/api/bulk_edit_objects/",
@@ -1680,7 +1666,7 @@ class TestFullPermissionsFlag(APITestCase):
def setUp(self) -> None:
super().setUp()
self.admin = User.objects.create_superuser(username="admin")
self.admin = UserFactory(username="admin", superuser=True)
def test_full_perms_flag(self) -> None:
"""
@@ -1,3 +1,5 @@
from __future__ import annotations
import unicodedata
from typing import TYPE_CHECKING
from unittest import mock
@@ -7,8 +9,11 @@ import pytest
from django.core.files.uploadedfile import SimpleUploadedFile
if TYPE_CHECKING:
from rest_framework.test import APIClient
from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture()
@@ -18,22 +23,14 @@ def consume_file_mock():
yield m
@pytest.fixture()
def directories(tmp_path, settings, _media_settings):
scratch = tmp_path / "scratch"
scratch.mkdir()
settings.SCRATCH_DIR = scratch
return scratch
@pytest.mark.django_db
class TestPostDocumentNFCNormalization:
def test_nfd_filename_normalized_to_nfc(
self,
admin_client,
admin_client: APIClient,
consume_file_mock: mock.MagicMock,
directories,
):
paperless_dirs: PaperlessDirs,
) -> None:
"""Uploaded file with NFD filename must have its name stored as NFC."""
nfd = unicodedata.normalize("NFD", "Rechnung März.pdf")
nfc = unicodedata.normalize("NFC", "Rechnung März.pdf")
@@ -69,10 +66,10 @@ class TestPostDocumentNFCNormalization:
def test_already_nfc_filename_unchanged(
self,
admin_client,
admin_client: APIClient,
consume_file_mock: mock.MagicMock,
directories,
):
paperless_dirs: PaperlessDirs,
) -> None:
"""Uploaded file with already-NFC filename must pass through unchanged."""
nfc = unicodedata.normalize("NFC", "Invoice_2024.pdf")
+16 -17
View File
@@ -8,7 +8,8 @@ from rest_framework import status
from rest_framework.authtoken.models import Token
from rest_framework.test import APITestCase
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
# see allauth.socialaccount.providers.openid.provider.OpenIDProvider
@@ -55,23 +56,23 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(
self.user = UserFactory(
username="temp_admin",
first_name="firstname",
last_name="surname",
superuser=True,
)
self.client.force_authenticate(user=self.user)
def setupSocialAccount(self) -> None:
def setupSocialAccount(self) -> SocialAccount:
SocialApp.objects.create(
name="Keycloak",
provider="openid_connect",
provider_id="keycloak-test",
)
self.user.socialaccount_set.add(
SocialAccount(uid="123456789", provider="keycloak-test"),
bulk=False,
)
social_account = SocialAccount(uid="123456789", provider="keycloak-test")
self.user.socialaccount_set.add(social_account, bulk=False)
return social_account
def test_get_profile(self) -> None:
"""
@@ -109,19 +110,17 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
THEN:
- Profile is returned with social accounts
"""
self.setupSocialAccount()
social_account = self.setupSocialAccount()
openid_provider = (
MockOpenIDConnectProvider(
app=SocialApp.objects.get(provider_id="keycloak-test"),
),
openid_provider = MockOpenIDConnectProvider(
app=SocialApp.objects.get(provider_id="keycloak-test"),
)
mock_list_providers.return_value = [
openid_provider,
]
mock_get_provider_account.return_value = MockOpenIDConnectProviderAccount(
mock_social_account_dict={
"name": openid_provider[0].name,
"name": openid_provider.name,
},
)
@@ -133,7 +132,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
response.data["social_accounts"],
[
{
"id": 1,
"id": social_account.pk,
"provider": "keycloak-test",
"name": "Keycloak",
},
@@ -150,7 +149,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
THEN:
- Profile is returned with "Unknown App" as name
"""
self.setupSocialAccount()
social_account = self.setupSocialAccount()
# Remove the social app
SocialApp.objects.get(provider_id="keycloak-test").delete()
@@ -163,7 +162,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
response.data["social_accounts"],
[
{
"id": 1,
"id": social_account.pk,
"provider": "keycloak-test",
"name": "Unknown App",
},
@@ -401,7 +400,7 @@ class TestApiTOTPViews(APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
def test_get_totp(self) -> None:
+40 -46
View File
@@ -7,10 +7,8 @@ import time_machine
from dateutil.relativedelta import relativedelta
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APITestCase
@@ -27,10 +25,13 @@ from documents.models import Tag
from documents.models import Workflow
from documents.search import get_backend
from documents.search import reset_backend
from documents.tests.factories import DocumentFactory
from documents.tests.utils import DirectoriesMixin
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
pytestmark = pytest.mark.search
@@ -40,7 +41,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
super().setUp()
reset_backend()
self.user = User.objects.create_superuser(username="temp_admin")
self.user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=self.user)
def tearDown(self) -> None:
@@ -949,9 +950,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN:
- Terms only within docs user has access to are returned
"""
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1.user_permissions.add(Permission.objects.get(codename="view_document"))
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
grant_global(u1, "view_document")
self.client.force_authenticate(user=u1)
@@ -991,7 +992,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data, ["applebaum", "apples"])
assign_perm("view_document", u1, d3)
grant_object(u1, d3, "view_document")
backend.add_or_update(d3)
response = self.client.get("/api/search/autocomplete/?term=app")
@@ -999,10 +1000,10 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(response.data, ["applebaum", "apples", "appletini"])
def test_search_autocomplete_group_revocation_is_immediate(self) -> None:
user = User.objects.create_user("group-user")
owner = User.objects.create_user("document-owner")
user = UserFactory(username="group-user")
owner = UserFactory(username="document-owner")
group = Group.objects.create(name="temporary-viewers")
user.user_permissions.add(Permission.objects.get(codename="view_document"))
grant_global(user, "view_document")
user.groups.add(group)
document = Document.objects.create(
@@ -1011,7 +1012,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
checksum="group-revocation",
owner=owner,
)
assign_perm("view_document", group, document)
grant_object(group, document, "view_document")
get_backend().add_or_update(document)
self.client.force_authenticate(user=user)
@@ -1091,11 +1092,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertIsNone(response.data["corrected_query"])
def test_search_spelling_suggestion_suppressed_for_private_terms(self) -> None:
owner = User.objects.create_user("owner")
attacker = User.objects.create_user("attacker")
attacker.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="owner")
attacker = UserFactory(username="attacker")
grant_global(attacker, "view_document")
backend = get_backend()
for i in range(5):
@@ -1222,11 +1221,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN:
- The request is rejected
"""
owner = User.objects.create_user("owner")
attacker = User.objects.create_user("attacker")
attacker.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="owner")
attacker = UserFactory(username="attacker")
grant_global(attacker, "view_document")
private_seed = Document.objects.create(
title="private bank statement",
@@ -1534,11 +1531,11 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
- Only owned docs are returned for regular users
- All docs are returned for superuser
"""
superuser = User.objects.create_superuser("superuser")
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
u2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
superuser = UserFactory(username="superuser", superuser=True)
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
grant_global(u1, "view_document")
grant_global(u2, "view_document")
Document.objects.create(checksum="1", content="test 1", owner=u1)
Document.objects.create(checksum="2", content="test 2", owner=u2)
@@ -1588,10 +1585,10 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN:
- Only docs with granted view permissions are returned
"""
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
u2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
grant_global(u1, "view_document")
grant_global(u2, "view_document")
d1 = Document.objects.create(checksum="1", content="test 1", owner=u1)
d2 = Document.objects.create(checksum="2", content="test 2", owner=u2)
@@ -1616,9 +1613,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
r = self.client.get("/api/documents/?query=test&owner__isnull=true")
self.assertEqual(r.data["count"], 1)
assign_perm("view_document", u1, d2)
assign_perm("view_document", u1, d3)
assign_perm("view_document", u2, d1)
grant_object(u1, d2, "view_document")
grant_object(u1, d3, "view_document")
grant_object(u2, d1, "view_document")
backend.add_or_update(d1)
backend.add_or_update(d2)
@@ -1641,8 +1638,8 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(r.data["count"], 1)
def test_search_sorting(self) -> None:
u1 = User.objects.create_user("user1")
u2 = User.objects.create_user("user2")
u1 = UserFactory(username="user1")
u2 = UserFactory(username="user2")
c1 = Correspondent.objects.create(name="corres Ax")
c2 = Correspondent.objects.create(name="corres Cx")
c3 = Correspondent.objects.create(name="corres Bx")
@@ -1892,8 +1889,8 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
pk=5,
)
user1 = User.objects.create_user("bank user1")
user2 = User.objects.create_superuser("user2")
user1 = UserFactory(username="bank user1")
user2 = UserFactory(username="user2", superuser=True)
group1 = Group.objects.create(name="bank group1")
Group.objects.create(name="group2")
@@ -1925,7 +1922,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
sort_field="",
owner=user2,
)
assign_perm("view_savedview", user1, shared_view)
grant_object(user1, shared_view, "view_savedview")
mail_account1 = MailAccount.objects.create(name="bank mail account 1")
mail_account2 = MailAccount.objects.create(name="mail account 2")
mail_rule1 = MailRule.objects.create(
@@ -2018,12 +2015,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(returned.get(root.id), "latest content")
def test_global_search_filters_owned_mail_objects(self) -> None:
user1 = User.objects.create_user("mail-search-user")
user2 = User.objects.create_user("other-mail-search-user")
user1.user_permissions.add(
Permission.objects.get(codename="view_mailaccount"),
Permission.objects.get(codename="view_mailrule"),
)
user1 = UserFactory(username="mail-search-user")
user2 = UserFactory(username="other-mail-search-user")
grant_global(user1, "view_mailaccount", "view_mailrule")
own_account = MailAccount.objects.create(
name="bank owned account",
+10 -10
View File
@@ -33,7 +33,7 @@ class TestSearchQueryErrorStillBecomesA400:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -68,7 +68,7 @@ class TestLibraryDefectsPropagate:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -98,7 +98,7 @@ class TestLibraryDefectsPropagate:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -141,7 +141,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -181,7 +181,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -221,7 +221,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -259,7 +259,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -287,7 +287,7 @@ class TestSelectionPathsAgreeWithSearch:
{
"documents": [],
"all": True,
"filters": {"more_like_id": indexed_document.pk},
"filters": {"more_like_id": searchable_document.pk},
},
format="json",
)
@@ -298,7 +298,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -328,7 +328,7 @@ class TestSelectionPathsAgreeWithSearch:
{
"documents": [],
"all": True,
"filters": {"more_like_id": indexed_document.pk},
"filters": {"more_like_id": searchable_document.pk},
},
format="json",
)
@@ -36,7 +36,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -66,7 +66,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_query_at_exactly_the_cap_is_accepted(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -86,7 +86,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_an_ordinary_query_is_unaffected(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -112,7 +112,7 @@ class TestPostSelectionPathsEnforceTheCap:
def test_bulk_edit_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -154,7 +154,7 @@ class TestPostSelectionPathsEnforceTheCap:
self,
bulk_update_task_mock: mock.MagicMock,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -187,7 +187,7 @@ class TestPostSelectionPathsEnforceTheCap:
def test_bulk_download_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -236,7 +236,7 @@ class TestGlobalSearchEnforcesTheCapToo:
def test_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -260,7 +260,7 @@ class TestGlobalSearchEnforcesTheCapToo:
def test_query_at_exactly_the_cap_is_accepted(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -38,7 +38,7 @@ class TestUnterminatedBracketReturnsA400:
def test_unterminated_bracket_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
query: str,
) -> None:
"""
@@ -59,7 +59,7 @@ class TestUnterminatedBracketReturnsA400:
def test_properly_closed_bracket_still_searches_cleanly(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
+7 -11
View File
@@ -5,8 +5,6 @@ from datetime import timedelta
from pathlib import Path
from unittest import mock
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from django.utils import timezone
from rest_framework import status
@@ -14,8 +12,10 @@ from rest_framework.test import APITestCase
from documents.models import PaperlessTask
from documents.permissions import has_system_status_permission
from documents.tests.factories import PaperlessTaskFactory
from paperless import version
from paperless_testing.factories import PaperlessTaskFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestSystemStatus(APITestCase):
@@ -23,9 +23,7 @@ class TestSystemStatus(APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_superuser(
username="temp_admin",
)
self.user = UserFactory(username="temp_admin", superuser=True)
self.tmp_dir = Path(tempfile.mkdtemp())
self.override = override_settings(MEDIA_ROOT=self.tmp_dir)
self.override.enable()
@@ -96,7 +94,7 @@ class TestSystemStatus(APITestCase):
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
self.assertEqual(response["WWW-Authenticate"], "Token")
normal_user = User.objects.create_user(username="normal_user")
normal_user = UserFactory(username="normal_user")
self.client.force_login(normal_user)
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
@@ -107,10 +105,8 @@ class TestSystemStatus(APITestCase):
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
user = User.objects.create_user(username="status_user")
user.user_permissions.add(
Permission.objects.get(codename="view_system_monitoring"),
)
user = UserFactory(username="status_user")
grant_global(user, "view_system_monitoring")
self.client.force_login(user)
response = self.client.get(self.ENDPOINT)
+20 -38
View File
@@ -11,22 +11,21 @@ from datetime import timedelta
from unittest import mock
import pytest
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APIClient
from documents.filters import PaperlessTaskFilterSet
from documents.models import PaperlessTask
from documents.tests.factories import DocumentFactory
from documents.tests.factories import PaperlessTaskFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import PaperlessTaskFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
pytestmark = pytest.mark.api
ENDPOINT = "/api/tasks/"
ACCEPT_V10 = "application/json; version=10"
ACCEPT_V9 = "application/json; version=9"
@@ -346,21 +345,16 @@ class TestGetTasksV10:
self,
admin_user: User,
regular_user: User,
user_client: APIClient,
) -> None:
"""Regular users see their own tasks and unowned (system) tasks; other users' tasks are hidden."""
regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
client = APIClient()
client.force_authenticate(user=regular_user)
client.credentials(HTTP_ACCEPT=ACCEPT_V10)
grant_global(regular_user, "view_paperlesstask")
PaperlessTaskFactory(owner=admin_user) # other user — not visible
unowned_task = PaperlessTaskFactory() # unowned (system task) — visible
own_task = PaperlessTaskFactory(owner=regular_user)
response = client.get(ENDPOINT)
response = user_client.get(ENDPOINT)
assert response.status_code == status.HTTP_200_OK
assert response.data["count"] == 2
@@ -590,9 +584,7 @@ class TestGetTasksV9:
regular_user: User,
) -> None:
"""Non-staff users see their own tasks plus unowned tasks via v9 API."""
regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
grant_global(regular_user, "view_paperlesstask")
client = APIClient()
client.force_authenticate(user=regular_user)
@@ -732,19 +724,17 @@ class TestAcknowledge:
assert response.status_code == status.HTTP_403_FORBIDDEN
def test_succeeds_with_change_permission(self, regular_user: User) -> None:
def test_succeeds_with_change_permission(
self,
regular_user: User,
user_client: APIClient,
) -> None:
"""Users granted change_paperlesstask permission can acknowledge tasks."""
regular_user.user_permissions.add(
Permission.objects.get(codename="change_paperlesstask"),
)
grant_global(regular_user, "change_paperlesstask")
regular_user.save()
client = APIClient()
client.force_authenticate(user=regular_user)
client.credentials(HTTP_ACCEPT=ACCEPT_V10)
task = PaperlessTaskFactory()
response = client.post(
response = user_client.post(
ENDPOINT + "acknowledge/",
{"tasks": [task.id]},
format="json",
@@ -807,9 +797,7 @@ class TestSummaryPermissions:
regular_user,
) -> None:
"""A user with view_system_monitoring but no document permissions can access summary/."""
regular_user.user_permissions.add(
Permission.objects.get(codename="view_system_monitoring"),
)
grant_global(regular_user, "view_system_monitoring")
response = user_client.get(ENDPOINT + "summary/")
@@ -822,9 +810,7 @@ class TestSummaryPermissions:
admin_user,
) -> None:
"""Monitoring user sees aggregate data for all tasks, not just unowned ones."""
regular_user.user_permissions.add(
Permission.objects.get(codename="view_system_monitoring"),
)
grant_global(regular_user, "view_system_monitoring")
PaperlessTaskFactory(
owner=admin_user,
task_type=PaperlessTask.TaskType.CONSUME_FILE,
@@ -845,9 +831,7 @@ class TestSummaryPermissions:
) -> None:
"""A regular user with view_paperlesstask but not view_system_monitoring sees only
their own tasks and unowned tasks in the summary, not other users' tasks."""
regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
grant_global(regular_user, "view_paperlesstask")
PaperlessTaskFactory(
owner=regular_user,
@@ -1012,9 +996,7 @@ class TestDuplicateDocumentsPermissions:
@pytest.fixture()
def user_v9_client(self, regular_user: User) -> APIClient:
regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
grant_global(regular_user, "view_paperlesstask")
client = APIClient()
client.force_authenticate(user=regular_user)
client.credentials(HTTP_ACCEPT=ACCEPT_V9)
@@ -1085,7 +1067,7 @@ class TestDuplicateDocumentsPermissions:
) -> None:
"""A user with explicit guardian view_document permission sees the duplicate_of document."""
doc = DocumentFactory(owner=admin_user, title="Granted Doc")
assign_perm("view_document", regular_user, doc)
grant_object(regular_user, doc, "view_document")
PaperlessTaskFactory(
owner=regular_user,
status=PaperlessTask.Status.SUCCESS,
+9 -9
View File
@@ -1,21 +1,21 @@
from datetime import date
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.core.cache import cache
from rest_framework import status
from rest_framework.test import APITestCase
from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
class TestTrashAPI(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.user = User.objects.create_user(username="temp_admin")
self.user.user_permissions.add(*Permission.objects.all())
self.user = UserFactory(username="temp_admin")
grant_all_global(self.user)
self.client.force_authenticate(user=self.user)
cache.clear()
@@ -70,7 +70,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
self.assertEqual(Document.global_objects.count(), 0)
def test_trash_list_requires_global_document_view_permission(self) -> None:
user = User.objects.create_user(username="trash_owner")
user = UserFactory(username="trash_owner")
document = Document.objects.create(title="Owned", owner=user)
document.delete()
self.client.force_authenticate(user)
@@ -140,7 +140,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
created=date(2023, 1, 2),
)
document_not_owned.delete()
user2 = User.objects.create_user(username="user2")
user2 = UserFactory(username="user2")
document_u2 = Document.objects.create(
title="Title3",
content="content3",
@@ -158,7 +158,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
self.assertEqual(resp.data["results"][1]["id"], document_u1.pk)
# superuser sees all documents
superuser = User.objects.create_superuser(username="superuser")
superuser = UserFactory(username="superuser", superuser=True)
self.client.force_authenticate(user=superuser)
resp = self.client.get("/api/trash/")
self.assertEqual(resp.status_code, status.HTTP_200_OK)
@@ -174,7 +174,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
- 403 Forbidden
"""
user2 = User.objects.create_user(username="user2")
user2 = UserFactory(username="user2")
document = Document.objects.create(
title="Title",
content="content",
+5 -5
View File
@@ -1,13 +1,13 @@
import json
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from rest_framework import status
from rest_framework.test import APITestCase
from documents.tests.utils import DirectoriesMixin
from paperless.version import __full_version_str__
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
class TestApiUiSettings(DirectoriesMixin, APITestCase):
@@ -15,7 +15,7 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
self.test_user = User.objects.create_superuser(username="test")
self.test_user = UserFactory(username="test", superuser=True)
self.test_user.first_name = "Test"
self.test_user.last_name = "User"
self.test_user.save()
@@ -91,7 +91,7 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
)
def test_api_set_ui_settings_insufficient_global_permissions(self) -> None:
not_superuser = User.objects.create_user(username="test_not_superuser")
not_superuser = UserFactory(username="test_not_superuser")
self.client.force_authenticate(user=not_superuser)
settings = {
@@ -111,7 +111,7 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_api_set_ui_settings_sufficient_global_permissions(self) -> None:
not_superuser = User.objects.create_user(username="test_not_superuser")
not_superuser = UserFactory(username="test_not_superuser")
not_superuser.user_permissions.add(
*Permission.objects.filter(codename__contains="uisettings"),
)
+3 -2
View File
@@ -14,7 +14,8 @@ from documents.models import Tag
from documents.models import Workflow
from documents.models import WorkflowAction
from documents.models import WorkflowTrigger
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
class TestApiWorkflows(DirectoriesMixin, APITestCase):
@@ -25,7 +26,7 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
def setUp(self) -> None:
super().setUp()
user = User.objects.create_superuser(username="temp_admin")
user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=user)
self.user2 = User.objects.create(username="user2")
self.user3 = User.objects.create(username="user3")
+75 -75
View File
@@ -2,8 +2,8 @@ import shutil
from collections.abc import Generator
from contextlib import contextmanager
from pathlib import Path
from unittest import mock
import pytest
from django.conf import settings
from django.test import TestCase
from django.test import override_settings
@@ -18,11 +18,11 @@ from documents.models import Document
from documents.models import Tag
from documents.plugins.base import StopConsumeTaskError
from documents.tests.utils import ConsumeTaskMixin
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless.models import ApplicationConfiguration
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.fakes.progress import FakeProgressManager
class GetReaderPluginMixin:
@@ -31,7 +31,7 @@ class GetReaderPluginMixin:
reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(),
DummyProgressManager(filepath.name, None),
FakeProgressManager(filepath.name, None),
self.dirs.scratch_dir,
"task-id",
)
@@ -86,6 +86,7 @@ class TestBarcode(
self.assertDictEqual(separator_page_numbers, {1: False})
@override_settings(CONSUMER_ENABLE_ASN_BARCODE=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_asn_barcode_duplicate_in_trash_fails(self) -> None:
"""
GIVEN:
@@ -110,15 +111,14 @@ class TestBarcode(
dupe_asn = settings.SCRATCH_DIR / "barcode-39-asn-123-second.pdf"
shutil.copy(test_file, dupe_asn)
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
with self.assertRaisesRegex(ConsumerError, r"ASN 123.*trash"):
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dupe_asn,
),
None,
)
with self.assertRaisesRegex(ConsumerError, r"ASN 123.*trash"):
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dupe_asn,
),
None,
)
@override_settings(
CONSUMER_BARCODE_TIFF_SUPPORT=True,
@@ -606,6 +606,7 @@ class TestBarcodeNewConsume(
TestCase,
):
@override_settings(CONSUMER_ENABLE_BARCODES=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file(self) -> None:
"""
GIVEN:
@@ -624,34 +625,33 @@ class TestBarcodeNewConsume(
overrides = DocumentMetadataOverrides(tag_ids=[1, 2, 9])
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
self.assertEqual(
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=temp_copy,
),
overrides,
self.assertEqual(
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=temp_copy,
),
{"reason": "Barcode splitting complete!"},
)
# 2 new document consume tasks created
self.assertEqual(self.consume_file_mock.call_count, 2)
overrides,
),
{"reason": "Barcode splitting complete!"},
)
# 2 new document consume tasks created
self.assertEqual(self.consume_file_mock.call_count, 2)
self.assertIsNotFile(temp_copy)
self.assertIsNotFile(temp_copy)
# Check the split files exist
# Check the original_path is set
# Check the source is unchanged
# Check the overrides are unchanged
for (
new_input_doc,
new_doc_overrides,
) in self.get_all_consume_task_call_args():
self.assertIsFile(new_input_doc.original_file)
self.assertEqual(new_input_doc.original_path, temp_copy)
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertEqual(overrides, new_doc_overrides)
# Check the split files exist
# Check the original_path is set
# Check the source is unchanged
# Check the overrides are unchanged
for (
new_input_doc,
new_doc_overrides,
) in self.get_all_consume_task_call_args():
self.assertIsFile(new_input_doc.original_file)
self.assertEqual(new_input_doc.original_path, temp_copy)
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertEqual(overrides, new_doc_overrides)
class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, TestCase):
@@ -660,7 +660,7 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(),
DummyProgressManager(filepath.name, None),
FakeProgressManager(filepath.name, None),
self.dirs.scratch_dir,
"task-id",
)
@@ -745,6 +745,7 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
self.assertEqual(asn, None)
@override_settings(CONSUMER_ENABLE_ASN_BARCODE=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file_asn_assignment(self) -> None:
"""
GIVEN:
@@ -762,19 +763,18 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
dst = settings.SCRATCH_DIR / "barcode-39-asn-123.pdf"
shutil.copy(test_file, dst)
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
document = Document.objects.first()
assert document is not None
document = Document.objects.first()
assert document is not None
self.assertEqual(document.archive_serial_number, 123)
self.assertEqual(document.archive_serial_number, 123)
def test_scan_file_for_qrcode_without_upscale(self) -> None:
"""
@@ -819,7 +819,7 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(),
DummyProgressManager(filepath.name, None),
FakeProgressManager(filepath.name, None),
self.dirs.scratch_dir,
"task-id",
)
@@ -1024,6 +1024,7 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
CELERY_TASK_ALWAYS_EAGER=True,
OCR_MODE="auto",
)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file_tag_split_and_assignment(self) -> None:
"""
GIVEN:
@@ -1042,34 +1043,33 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
dst = settings.SCRATCH_DIR / "split-by-tag-basic.pdf"
shutil.copy(test_file, dst)
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
result = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
result = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
self.assertEqual(result, {"reason": "Barcode splitting complete!"})
self.assertEqual(result, {"reason": "Barcode splitting complete!"})
documents = Document.objects.all().order_by("id")
self.assertEqual(documents.count(), 3)
documents = Document.objects.all().order_by("id")
self.assertEqual(documents.count(), 3)
doc1 = documents[0]
self.assertEqual(doc1.tags.count(), 0)
doc1 = documents[0]
self.assertEqual(doc1.tags.count(), 0)
doc2 = documents[1]
self.assertEqual(doc2.tags.count(), 1)
_tag_1 = doc2.tags.first()
assert _tag_1 is not None
self.assertEqual(_tag_1.name, "invoice")
doc2 = documents[1]
self.assertEqual(doc2.tags.count(), 1)
_tag_1 = doc2.tags.first()
assert _tag_1 is not None
self.assertEqual(_tag_1.name, "invoice")
doc3 = documents[2]
self.assertEqual(doc3.tags.count(), 1)
_tag_2 = doc3.tags.first()
assert _tag_2 is not None
self.assertEqual(_tag_2.name, "receipt")
doc3 = documents[2]
self.assertEqual(doc3.tags.count(), 1)
_tag_2 = doc3.tags.first()
assert _tag_2 is not None
self.assertEqual(_tag_2.name, "receipt")
@override_settings(
CONSUMER_ENABLE_TAG_BARCODE=True,
+6 -6
View File
@@ -10,7 +10,6 @@ from django.contrib.auth.models import User
from django.db import connection
from django.test import TestCase
from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms
@@ -23,7 +22,8 @@ from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
from documents.permissions import set_permissions_for_objects
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.permissions import grant_object
class TestBulkEdit(DirectoriesMixin, TestCase):
@@ -440,7 +440,7 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
def test_set_permissions(self, m) -> None:
doc_ids = [self.doc1.id, self.doc2.id, self.doc3.id]
assign_perm("view_document", self.group1, self.doc1)
grant_object(self.group1, self.doc1, "view_document")
permissions = {
"view": {
@@ -482,8 +482,8 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
self.doc1.owner = self.user1
self.doc1.save()
assign_perm("view_document", self.user1, self.doc1)
assign_perm("view_document", self.group1, self.doc1)
grant_object(self.user1, self.doc1, "view_document")
grant_object(self.group1, self.doc1, "view_document")
permissions = {
"view": {
@@ -609,7 +609,7 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
self.doc1.owner = self.user1
self.doc1.save()
self.user1.groups.add(self.group1)
assign_perm("view_document", self.group1, self.doc1)
grant_object(self.group1, self.doc1, "view_document")
bulk_edit.set_permissions(
[self.doc1.id],
+4 -13
View File
@@ -1,5 +1,4 @@
import pickle
import re
import warnings
from datetime import UTC
from datetime import datetime
@@ -28,21 +27,13 @@ from documents.models import DocumentType
from documents.models import MatchingModel
from documents.models import StoragePath
from documents.models import Tag
from documents.tests.factories import DocumentFactory
from documents.tests.factories import TagFactory
from documents.tests.utils import DirectoriesMixin
from documents.tests.helpers import dummy_preprocess
from paperless.settings import CLASSIFIER_LANGUAGES
from paperless.signed_pickle import HMAC_SIZE
from paperless.signed_pickle import signed_pickle_dumps
def dummy_preprocess(content: str) -> str:
"""
Simpler, faster pre-processing for testing purposes
"""
content = content.lower().strip()
content = re.sub(r"\s+", " ", content)
return content
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
class TestClassifier(DirectoriesMixin, TestCase):
+8 -7
View File
@@ -30,11 +30,12 @@ from documents.models import Tag
from documents.parsers import ParseError
from documents.plugins.helpers import ProgressStatusOptions
from documents.tasks import sanity_check
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import GetConsumerMixin
from paperless_mail.models import MailRule
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.fakes.progress import FakeProgressManager
class _BaseNewStyleParser:
@@ -769,14 +770,14 @@ class TestConsumer(
original_modified = timezone.now() - datetime.timedelta(days=1)
Document.objects.filter(pk=root_doc.pk).update(modified=original_modified)
actor = User.objects.create_user(
actor = UserFactory(
username="actor",
email="actor@example.com",
password="password",
)
version_file = self.get_test_file2()
status = DummyProgressManager(version_file.name, None)
status = FakeProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides(
version_label="v2",
actor_id=actor.pk,
@@ -839,7 +840,7 @@ class TestConsumer(
assert root_doc is not None
version_file = self.get_test_file2()
status = DummyProgressManager(version_file.name, None)
status = FakeProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides(
filename="valid_pdf_version-upload",
actor_id=999999,
@@ -896,7 +897,7 @@ class TestConsumer(
assert root_doc is not None
def consume_version(version_file: Path) -> Document:
status = DummyProgressManager(version_file.name, None)
status = FakeProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides()
doc = ConsumableDocument(
DocumentSource.ApiUpload,
@@ -9,11 +9,11 @@ from django.test.utils import CaptureQueriesContext
from rest_framework import status
from documents.models import Document
from documents.tests.factories import DocumentFactory
from documents.versioning import LATEST_VERSION_CONTENT_PREFETCH_ATTR
from documents.versioning import has_prefetched_effective_content
from documents.versioning import latest_version_content_prefetch
from documents.views import DocumentViewSet
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from rest_framework.test import APIClient
+49 -39
View File
@@ -2,8 +2,8 @@ import datetime as dt
import os
import shutil
from pathlib import Path
from unittest import mock
import pytest
from django.test import TestCase
from django.test import override_settings
from pdfminer.high_level import extract_text
@@ -15,22 +15,26 @@ from documents.data_models import ConsumableDocument
from documents.data_models import DocumentSource
from documents.double_sided import STAGING_FILE_NAME
from documents.double_sided import TIMEOUT_MINUTES
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@pytest.mark.usefixtures("fake_progress_manager")
@override_settings(
CONSUMER_RECURSIVE=True,
CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=True,
)
class TestDoubleSided(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
SAMPLE_DIR = Path(__file__).parent / "samples"
class TestDoubleSided(
DirectoriesMixin,
FileSystemAssertsMixin,
SampleDirMixin,
TestCase,
):
def setUp(self) -> None:
super().setUp()
self.dirs.double_sided_dir = self.dirs.consumption_dir / "double-sided"
self.dirs.double_sided_dir.mkdir()
self.double_sided_dir = self.dirs.consumption_dir / "double-sided"
self.double_sided_dir.mkdir()
self.staging_file = self.dirs.scratch_dir / STAGING_FILE_NAME
def consume_file(self, srcname, dstname: str | Path = "foo.pdf"):
@@ -39,20 +43,16 @@ class TestDoubleSided(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
destination file does not exist afterwards
"""
src = self.SAMPLE_DIR / srcname
dst = self.dirs.double_sided_dir / dstname
dst = self.double_sided_dir / dstname
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy(src, dst)
with mock.patch(
"documents.tasks.ProgressManager",
DummyProgressManager,
):
msg = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
msg = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
self.assertIsNotFile(dst)
return msg
@@ -214,31 +214,41 @@ class TestDoubleSided(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
self.assertIsNotFile(self.staging_file)
self.assertIsInstance(msg.get("document_id"), int)
def test_subdirectory_upload(self) -> None:
def test_consume_double_sided_in_nested_dir(self) -> None:
"""
GIVEN:
- A staging file exists
WHEN:
- A file gets uploaded into foo/bar/double-sided
or double-sided/foo/bar
- A file is uploaded into foo/bar/double-sided
THEN:
- The collated file gets put into foo/bar
- The collated file is placed into foo/bar
"""
# TODO: parameterize this instead
for path in [
Path("foo") / "bar" / "double-sided",
Path("double-sided") / "foo" / "bar",
]:
with self.subTest(path=str(path)):
# Ensure we get fresh directories for each run
self.tearDown()
self.setUp()
self.create_staging_file()
self.consume_file(
"double-sided-odd.pdf",
Path("foo") / "bar" / "double-sided" / "foo.pdf",
)
self.assertIsFile(
self.dirs.consumption_dir / "foo" / "bar" / "foo-collated.pdf",
)
self.create_staging_file()
self.consume_file("double-sided-odd.pdf", Path(path) / "foo.pdf")
self.assertIsFile(
self.dirs.consumption_dir / "foo" / "bar" / "foo-collated.pdf",
)
def test_consume_double_sided_with_nested_subdir(self) -> None:
"""
GIVEN:
- A staging file exists
WHEN:
- A file is uploaded into double-sided/foo/bar
THEN:
- The collated file is placed into foo/bar
"""
self.create_staging_file()
self.consume_file(
"double-sided-odd.pdf",
Path("double-sided") / "foo" / "bar" / "foo.pdf",
)
self.assertIsFile(
self.dirs.consumption_dir / "foo" / "bar" / "foo-collated.pdf",
)
@override_settings(CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=False)
def test_disabled_double_sided_dir_upload(self) -> None:
+5 -5
View File
@@ -8,7 +8,6 @@ from unittest import mock
import pytest
from auditlog.context import disable_auditlog
from django.conf import settings
from django.contrib.auth.models import User
from django.db import DatabaseError
from django.db import connection
from django.test import TestCase
@@ -30,9 +29,10 @@ from documents.models import DocumentType
from documents.models import StoragePath
from documents.serialisers import DocumentSerializer
from documents.tasks import empty_trash
from documents.tests.factories import DocumentFactory
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
class TestFileHandling(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
@@ -1323,7 +1323,7 @@ class TestFilenameGeneration(DirectoriesMixin, TestCase):
- Document without owner returns "none"
"""
u1 = User.objects.create_user("user1")
u1 = UserFactory(username="user1")
owned_doc = Document.objects.create(
title="The Title",
+4 -4
View File
@@ -13,10 +13,10 @@ import pytest
from documents.file_handling import generate_filename
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory
from documents.tests.factories import StoragePathFactory
from documents.tests.factories import TagFactory
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import StoragePathFactory
from paperless_testing.factories import TagFactory
@pytest.mark.django_db
+2 -2
View File
@@ -20,8 +20,8 @@ if TYPE_CHECKING:
from documents.file_handling import generate_filename
from documents.models import Document
from documents.tasks import update_document_content_maybe_archive_file
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
sample_file: Path = Path(__file__).parent / "samples" / "simple.pdf"
+11 -10
View File
@@ -26,7 +26,6 @@ from django.test import override_settings
from django.utils import timezone
from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission
from guardian.shortcuts import assign_perm
from documents.management.commands import document_exporter
from documents.models import Correspondent
@@ -46,11 +45,12 @@ from documents.models import WorkflowTrigger
from documents.sanity_checker import check_sanity
from documents.settings import EXPORTER_FILE_NAME
from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from documents.tests.utils import paperless_environment
from paperless_mail.models import MailAccount
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.dirs import paperless_environment
from paperless_testing.permissions import grant_object
@pytest.mark.management
@@ -105,8 +105,8 @@ class TestExportImport(
user=self.user,
)
assign_perm("view_document", self.user2, self.d2)
assign_perm("view_document", self.group1, self.d3)
grant_object(self.user2, self.d2, "view_document")
grant_object(self.group1, self.d3, "view_document")
self.t1 = Tag.objects.create(name="t")
self.dt1 = DocumentType.objects.create(name="dt")
@@ -677,12 +677,13 @@ class TestExportImport(
THEN:
- Error is raised
"""
args = ["document_exporter", "/tmp/foo/bar"]
with tempfile.TemporaryDirectory() as tmp_dir:
args = ["document_exporter", str(Path(tmp_dir) / "does-not-exist")]
with self.assertRaises(CommandError) as e:
call_command(*args, skip_checks=True)
with self.assertRaises(CommandError) as e:
call_command(*args, skip_checks=True)
self.assertEqual("That path doesn't exist", str(e.exception))
self.assertEqual("That path doesn't exist", str(e.exception))
def test_export_target_exists_but_is_file(self) -> None:
"""
+9 -9
View File
@@ -8,7 +8,7 @@ from django.core.management import call_command
from django.test import TestCase
from documents.models import Document
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory
@pytest.mark.management
@@ -123,14 +123,14 @@ class TestFuzzyMatchCommand(TestCase):
- Output contains clickable links to the documents instead of titles
"""
# Content similarity is 86.667
Document.objects.create(
doc1 = Document.objects.create(
checksum="BEEFCAFE",
title="A",
content="first document scanned by bob",
mime_type="application/pdf",
filename="test.pdf",
)
Document.objects.create(
doc2 = Document.objects.create(
checksum="DEADBEAF",
title="A",
content="first document scanned by alice",
@@ -145,8 +145,8 @@ class TestFuzzyMatchCommand(TestCase):
"http://localhost:8000",
)
self.assertIn("Found 1 matching pair(s)", stdout)
self.assertIn("http://localhost:8000/documents/1/details", stdout)
self.assertIn("http://localhost:8000/documents/2/details", stdout)
self.assertIn(f"http://localhost:8000/documents/{doc1.pk}/details", stdout)
self.assertIn(f"http://localhost:8000/documents/{doc2.pk}/details", stdout)
def test_with_3_matches(self) -> None:
"""
@@ -198,14 +198,14 @@ class TestFuzzyMatchCommand(TestCase):
- Documents 1 and 2 remain
"""
# Content similarity is 86.667
Document.objects.create(
doc1 = Document.objects.create(
checksum="BEEFCAFE",
title="A",
content="first document scanned by bob",
mime_type="application/pdf",
filename="test.pdf",
)
Document.objects.create(
doc2 = Document.objects.create(
checksum="DEADBEAF",
title="A",
content="second document scanned by alice",
@@ -235,8 +235,8 @@ class TestFuzzyMatchCommand(TestCase):
self.assertIn("Deleting 1 document(s)", stdout)
self.assertEqual(Document.objects.count(), 2)
self.assertIsNotNone(Document.objects.get(pk=1))
self.assertIsNotNone(Document.objects.get(pk=2))
self.assertIsNotNone(Document.objects.get(pk=doc1.pk))
self.assertIsNotNone(Document.objects.get(pk=doc2.pk))
def test_document_deletion_cancelled(self) -> None:
"""
@@ -15,9 +15,9 @@ from documents.management.commands.document_importer import _deserialize_record
from documents.models import Document
from documents.settings import EXPORTER_ARCHIVE_NAME
from documents.settings import EXPORTER_FILE_NAME
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@pytest.mark.management
@@ -14,12 +14,12 @@ from documents.models import DocumentType
from documents.models import MatchingModel
from documents.models import StoragePath
from documents.models import Tag
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory
from documents.tests.factories import DocumentTypeFactory
from documents.tests.factories import StoragePathFactory
from documents.tests.factories import TagFactory
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory
from paperless_testing.factories import StoragePathFactory
from paperless_testing.factories import TagFactory
# ---------------------------------------------------------------------------
# Module-level type aliases
@@ -7,14 +7,14 @@ from django.contrib.auth.models import User
from django.core.management import call_command
from django.test import TestCase
from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin
@pytest.mark.management
class TestManageSuperUser(DirectoriesMixin, TestCase):
def call_command(self, environ):
out = StringIO()
with mock.patch.dict(os.environ, environ):
with mock.patch.dict(os.environ, environ, clear=True):
call_command(
"manage_superuser",
"--no-color",
@@ -9,8 +9,8 @@ from django.test import TestCase
from documents.management.commands.document_thumbnails import _process_document
from documents.models import Document
from documents.parsers import get_default_thumbnail
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@pytest.mark.management
+4 -4
View File
@@ -7,10 +7,10 @@ from documents import matching
from documents.models import Document
from documents.models import MatchingModel
from documents.signals import document_consumption_finished
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory
from documents.tests.factories import DocumentTypeFactory
from documents.tests.factories import TagFactory
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory
from paperless_testing.factories import TagFactory
@pytest.fixture(
@@ -2,8 +2,6 @@ import json
from unittest import mock
from auditlog.models import LogEntry
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType
from django.test import TestCase
from rest_framework import status
@@ -12,6 +10,8 @@ from rest_framework.test import APITestCase
from documents.bulk_edit import merge_as_versions
from documents.models import Document
from documents.serialisers import MergeDocumentsAsVersionsSerializer
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestMergeDocumentsAsVersionsSerializer(TestCase):
@@ -294,7 +294,7 @@ class TestMergeDocumentsAsVersions(TestCase):
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
@mock.patch("documents.search.get_backend")
def test_writes_audit_log_entry(self, *_mocks) -> None:
user = User.objects.create_user(username="merger")
user = UserFactory(username="merger")
root = Document.objects.create(checksum="A", title="Root")
source = Document.objects.create(checksum="B", title="Source")
LogEntry.objects.all().delete()
@@ -335,12 +335,8 @@ class TestMergeDocumentsAsVersions(TestCase):
class TestMergeDocumentsAsVersionsAPI(APITestCase):
def setUp(self) -> None:
self.user = User.objects.create_user(username="user")
self.user.user_permissions.add(
Permission.objects.get(codename="change_document"),
Permission.objects.get(codename="view_document"),
Permission.objects.get(codename="delete_document"),
)
self.user = UserFactory(username="user")
grant_global(self.user, "change_document", "view_document", "delete_document")
self.doc1 = Document.objects.create(
checksum="A",
title="A",
@@ -382,7 +378,7 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
@mock.patch("documents.views.bulk_edit.merge_as_versions")
def test_requires_change_permission(self, merge_mock) -> None:
merge_mock.__name__ = "merge_as_versions"
user = User.objects.create_user(username="no-change")
user = UserFactory(username="no-change")
self.doc1.owner = user
self.doc1.save()
self.doc2.owner = user
@@ -405,11 +401,8 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
def test_requires_delete_permission(self, merge_mock) -> None:
merge_mock.__name__ = "merge_as_versions"
# Owns them and may change them, but may not make them stop being documents
user = User.objects.create_user(username="no-delete")
user.user_permissions.add(
Permission.objects.get(codename="change_document"),
Permission.objects.get(codename="view_document"),
)
user = UserFactory(username="no-delete")
grant_global(user, "change_document", "view_document")
for doc in (self.doc1, self.doc2):
doc.owner = user
doc.save()
@@ -1,4 +1,4 @@
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
SAVED_VIEWS_KEY = "saved_views"
DASHBOARD_VIEWS_VISIBLE_IDS_KEY = "dashboard_views_visible_ids"
@@ -7,7 +7,7 @@ from django.conf import settings
from django.db import connection
from django.test import override_settings
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
def _sha256(data: bytes) -> str:
@@ -1,4 +1,4 @@
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
class TestMigrateShareLinkBundlePermissions(TestMigrations):
+2 -2
View File
@@ -2,8 +2,8 @@ import pytest
from documents.models import Correspondent
from documents.models import Document
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory
@pytest.mark.django_db
@@ -1,15 +1,13 @@
from __future__ import annotations
from http import HTTPStatus
from typing import TYPE_CHECKING
from unittest.mock import patch
import pytest
from django.contrib.auth.models import AnonymousUser
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from guardian.shortcuts import assign_perm
from rest_framework.test import APIClient
from documents.matching import match_correspondents
@@ -24,11 +22,17 @@ from documents.permissions import permitted_document_ids
from documents.permissions import permitted_object_ids
from documents.permissions import restrict_queryset_to_visible
from documents.serialisers import _get_viewable_duplicates
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory
from documents.tests.factories import DocumentTypeFactory
from documents.tests.factories import StoragePathFactory
from documents.tests.factories import TagFactory
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory
from paperless_testing.factories import StoragePathFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
if TYPE_CHECKING:
from paperless_testing.dirs import PaperlessDirs
def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden):
@@ -47,8 +51,8 @@ def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden
@pytest.mark.django_db
class TestPermittedDocumentIdsSecurity:
def test_owner_sees_own_document(self):
user = User.objects.create_user(username="alice")
stranger = User.objects.create_user(username="mallory")
user = UserFactory(username="alice")
stranger = UserFactory(username="mallory")
owned = DocumentFactory(owner=user)
strangers_doc = DocumentFactory(owner=stranger)
@@ -61,7 +65,7 @@ class TestPermittedDocumentIdsSecurity:
)
def test_unowned_document_visible_to_everyone(self):
user = User.objects.create_user(username="alice")
user = UserFactory(username="alice")
unowned = DocumentFactory(owner=None)
assert_visible_document_ids(
@@ -71,12 +75,12 @@ class TestPermittedDocumentIdsSecurity:
)
def test_explicit_user_permission_grants_visibility(self):
grantee = User.objects.create_user(username="alice")
stranger = User.objects.create_user(username="mallory")
owner = User.objects.create_user(username="owner")
grantee = UserFactory(username="alice")
stranger = UserFactory(username="mallory")
owner = UserFactory(username="owner")
shared = DocumentFactory(owner=owner)
not_shared = DocumentFactory(owner=owner)
assign_perm("view_document", grantee, shared)
grant_object(grantee, shared, "view_document")
assert_visible_document_ids(
permitted_document_ids(grantee),
@@ -90,13 +94,13 @@ class TestPermittedDocumentIdsSecurity:
)
def test_explicit_group_permission_grants_visibility_to_members_only(self):
owner = User.objects.create_user(username="owner")
member = User.objects.create_user(username="member")
non_member = User.objects.create_user(username="non_member")
owner = UserFactory(username="owner")
member = UserFactory(username="member")
non_member = UserFactory(username="non_member")
group = Group.objects.create(name="finance")
member.groups.add(group)
shared = DocumentFactory(owner=owner)
assign_perm("view_document", group, shared)
grant_object(group, shared, "view_document")
assert_visible_document_ids(
permitted_document_ids(member),
@@ -110,7 +114,7 @@ class TestPermittedDocumentIdsSecurity:
)
def test_soft_deleted_document_excluded_by_default(self):
owner = User.objects.create_user(username="owner")
owner = UserFactory(username="owner")
doc = DocumentFactory(owner=owner)
doc.delete() # soft delete
doc.refresh_from_db()
@@ -126,8 +130,8 @@ class TestPermittedDocumentIdsSecurity:
)
def test_superuser_sees_everything_including_no_perm_documents(self):
superuser = User.objects.create_superuser(username="root")
owner = User.objects.create_user(username="owner")
superuser = UserFactory(username="root", superuser=True)
owner = UserFactory(username="owner")
doc = DocumentFactory(owner=owner)
assert_visible_document_ids(
@@ -137,7 +141,7 @@ class TestPermittedDocumentIdsSecurity:
)
def test_anonymous_user_sees_only_unowned_documents(self):
owner = User.objects.create_user(username="owner")
owner = UserFactory(username="owner")
owned = DocumentFactory(owner=owner)
unowned = DocumentFactory(owner=None)
@@ -151,7 +155,7 @@ class TestPermittedDocumentIdsSecurity:
@pytest.mark.django_db
class TestPermittedDocumentIdsIncludeDeleted:
def test_include_deleted_true_reveals_soft_deleted_owned_document(self):
owner = User.objects.create_user(username="owner")
owner = UserFactory(username="owner")
doc = DocumentFactory(owner=owner)
doc.delete()
@@ -162,8 +166,8 @@ class TestPermittedDocumentIdsIncludeDeleted:
)
def test_include_deleted_true_still_respects_permission_boundary(self):
owner = User.objects.create_user(username="owner")
stranger = User.objects.create_user(username="mallory")
owner = UserFactory(username="owner")
stranger = UserFactory(username="mallory")
doc = DocumentFactory(owner=owner)
doc.delete()
@@ -191,14 +195,12 @@ class TestAiChatAllDocumentsPermissionBoundary:
def test_chat_all_documents_excludes_unshared_document(self, mock_stream_chat):
mock_stream_chat.return_value = iter([b"data"])
owner = User.objects.create_user(username="owner")
asker = User.objects.create_user(username="asker")
asker.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
owner = UserFactory(username="owner")
asker = UserFactory(username="asker")
grant_global(asker, "view_document")
shared = DocumentFactory(owner=owner)
not_shared = DocumentFactory(owner=owner)
assign_perm("view_document", asker, shared)
grant_object(asker, shared, "view_document")
client = APIClient()
client.force_authenticate(user=asker)
@@ -219,13 +221,13 @@ class TestAiChatAllDocumentsPermissionBoundary:
@pytest.mark.django_db
class TestDuplicateDocumentsPermissionBoundary:
def test_get_viewable_duplicates_includes_soft_deleted_but_respects_perms(self):
owner = User.objects.create_user(username="owner")
stranger = User.objects.create_user(username="mallory")
owner = UserFactory(username="owner")
stranger = UserFactory(username="mallory")
original = DocumentFactory(owner=owner, checksum="dupe-checksum")
dup_visible = DocumentFactory(owner=owner, checksum="dupe-checksum")
dup_hidden = DocumentFactory(owner=owner, checksum="dupe-checksum")
dup_hidden.delete() # soft delete, should still be found (include_deleted=True)
assign_perm("view_document", stranger, dup_visible)
grant_object(stranger, dup_visible, "view_document")
result_owner = _get_viewable_duplicates(original, owner)
assert {d.pk for d in result_owner} == {dup_visible.pk, dup_hidden.pk}
@@ -237,13 +239,13 @@ class TestDuplicateDocumentsPermissionBoundary:
@pytest.mark.django_db
class TestPermittedDocumentIdsArbitraryPermission:
def test_change_document_permission_is_distinct_from_view(self):
owner = User.objects.create_user(username="owner")
viewer_only = User.objects.create_user(username="viewer")
editor = User.objects.create_user(username="editor")
owner = UserFactory(username="owner")
viewer_only = UserFactory(username="viewer")
editor = UserFactory(username="editor")
doc = DocumentFactory(owner=owner)
assign_perm("view_document", viewer_only, doc)
assign_perm("change_document", editor, doc)
assign_perm("view_document", editor, doc)
grant_object(viewer_only, doc, "view_document")
grant_object(editor, doc, "change_document")
grant_object(editor, doc, "view_document")
assert_visible_document_ids(
permitted_document_ids(editor, perm="change_document"),
@@ -257,10 +259,10 @@ class TestPermittedDocumentIdsArbitraryPermission:
)
def test_qualified_permission_string_is_normalized_to_codename(self):
owner = User.objects.create_user(username="owner")
editor = User.objects.create_user(username="editor")
owner = UserFactory(username="owner")
editor = UserFactory(username="editor")
doc = DocumentFactory(owner=owner)
assign_perm("change_document", editor, doc)
grant_object(editor, doc, "change_document")
assert_visible_document_ids(
permitted_document_ids(editor, perm="documents.change_document"),
@@ -269,11 +271,11 @@ class TestPermittedDocumentIdsArbitraryPermission:
)
def test_delete_permission_with_include_deleted_for_trash_restore(self):
owner = User.objects.create_user(username="owner")
stranger = User.objects.create_user(username="mallory")
view_only = User.objects.create_user(username="viewer")
owner = UserFactory(username="owner")
stranger = UserFactory(username="mallory")
view_only = UserFactory(username="viewer")
doc = DocumentFactory(owner=owner)
assign_perm("view_document", view_only, doc)
grant_object(view_only, doc, "view_document")
doc.delete()
assert_visible_document_ids(
@@ -307,11 +309,9 @@ class TestEmailDocumentPermissionBoundary:
self,
rest_api_client,
):
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "view_document")
rest_api_client.force_authenticate(user=requester)
hidden = DocumentFactory(owner=owner)
@@ -339,19 +339,17 @@ class TestBulkEditChangePermissionBoundary:
# permitted document must not be partially applied just because it
# was bundled with a forbidden one, proving the endpoint checks
# every document in the batch rather than only the first/last.
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
# grant the global change_document permission so the object-level
# check (not the global has_perm check) is what's under test
requester.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
grant_global(requester, "change_document")
rest_api_client.force_authenticate(user=requester)
changeable = DocumentFactory(owner=owner)
assign_perm("view_document", requester, changeable)
assign_perm("change_document", requester, changeable) # fully permitted
grant_object(requester, changeable, "view_document")
grant_object(requester, changeable, "change_document") # fully permitted
target = DocumentFactory(owner=owner)
assign_perm("view_document", requester, target) # view only, NOT change
grant_object(requester, target, "view_document") # view only, NOT change
response = rest_api_client.post(
"/api/documents/bulk_edit/",
@@ -369,15 +367,14 @@ class TestBulkEditChangePermissionBoundary:
class TestBulkDownloadPermissionChecksRootDocument:
def test_download_requires_global_view_permission(
self,
rest_api_client,
paperless_dirs,
_media_settings,
):
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
rest_api_client: APIClient,
paperless_dirs: PaperlessDirs,
) -> None:
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
root = DocumentFactory(owner=owner)
root.source_path.write_bytes(b"%PDF-1.4 test")
assign_perm("view_document", requester, root)
grant_object(requester, root, "view_document")
rest_api_client.force_authenticate(user=requester)
response = rest_api_client.post(
@@ -390,21 +387,18 @@ class TestBulkDownloadPermissionChecksRootDocument:
def test_permission_checked_on_root_not_on_version(
self,
rest_api_client,
paperless_dirs,
_media_settings,
):
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
rest_api_client: APIClient,
paperless_dirs: PaperlessDirs,
) -> None:
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "view_document")
rest_api_client.force_authenticate(user=requester)
root = DocumentFactory(owner=owner)
# a version of root that the requester has NOT been individually granted
version = DocumentFactory(owner=owner, root_document=root, version_index=1)
version.source_path.write_bytes(b"%PDF-1.4 test")
assign_perm("view_document", requester, root) # granted on ROOT only
grant_object(requester, root, "view_document") # granted on ROOT only
response = rest_api_client.post(
"/api/documents/bulk_download/",
@@ -422,11 +416,9 @@ class TestBulkDownloadPermissionChecksRootDocument:
# root-or-version bug; a user with no grant at all (the old
# `stranger` case) can't tell the two apart, since they're denied
# either way.
version_only_grantee = User.objects.create_user(username="version_only_grantee")
version_only_grantee.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
assign_perm("view_document", version_only_grantee, version)
version_only_grantee = UserFactory(username="version_only_grantee")
grant_global(version_only_grantee, "view_document")
grant_object(version_only_grantee, version, "view_document")
rest_api_client.force_authenticate(user=version_only_grantee)
response = rest_api_client.post(
"/api/documents/bulk_download/",
@@ -438,6 +430,53 @@ class TestBulkDownloadPermissionChecksRootDocument:
) # version-only grant must not substitute for root permission
@pytest.mark.django_db
class TestDocumentOperationPermissionChecksRootDocument:
@pytest.mark.parametrize(
("endpoint", "payload"),
[
pytest.param("/api/documents/merge/", {}, id="merge"),
pytest.param("/api/documents/rotate/", {"degrees": 90}, id="rotate"),
],
)
@pytest.mark.parametrize("version_owner", ["none", "requester"])
def test_version_operation_acts_on_root(
self,
rest_api_client: APIClient,
endpoint: str,
payload: dict,
version_owner: str,
) -> None:
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "change_document")
grant_global(requester, "add_document")
rest_api_client.force_authenticate(user=requester)
root = DocumentFactory(owner=owner)
# A version whose owner went stale, e.g. created before the root changed hands
version = DocumentFactory(
owner=requester if version_owner == "requester" else None,
root_document=root,
version_index=1,
)
with (
patch("documents.views.bulk_edit.merge") as mock_merge,
patch("documents.views.bulk_edit.rotate") as mock_rotate,
):
mock_merge.__name__ = "merge"
mock_rotate.__name__ = "rotate"
response = rest_api_client.post(
endpoint,
{"documents": [version.pk], **payload},
format="json",
)
assert response.status_code == HTTPStatus.FORBIDDEN
mock_merge.assert_not_called()
mock_rotate.assert_not_called()
@pytest.mark.django_db
@pytest.mark.usefixtures("_search_index")
class TestTrashRestorePermissionBoundary:
@@ -445,14 +484,12 @@ class TestTrashRestorePermissionBoundary:
self,
rest_api_client,
):
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
requester.user_permissions.add(
Permission.objects.get(codename="delete_document"),
)
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "delete_document")
rest_api_client.force_authenticate(user=requester)
doc = DocumentFactory(owner=owner)
assign_perm("view_document", requester, doc) # view only, NOT delete
grant_object(requester, doc, "view_document") # view only, NOT delete
doc.delete()
response = rest_api_client.post(
@@ -466,14 +503,12 @@ class TestTrashRestorePermissionBoundary:
self,
rest_api_client,
):
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
requester.user_permissions.add(
Permission.objects.get(codename="delete_document"),
)
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "delete_document")
rest_api_client.force_authenticate(user=requester)
doc = DocumentFactory(owner=owner)
assign_perm("delete_document", requester, doc)
grant_object(requester, doc, "delete_document")
doc.delete()
response = rest_api_client.post(
@@ -484,11 +519,11 @@ class TestTrashRestorePermissionBoundary:
assert response.status_code == HTTPStatus.OK
def test_restore_requires_global_delete_permission(self, rest_api_client):
owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
rest_api_client.force_authenticate(user=requester)
doc = DocumentFactory(owner=owner)
assign_perm("delete_document", requester, doc)
grant_object(requester, doc, "delete_document")
doc.delete()
response = rest_api_client.post(
@@ -513,14 +548,12 @@ class TestTrashViewExcludesExplicitlyGrantedDocuments:
"""
def test_explicit_grant_does_not_leak_trashed_document(self, rest_api_client):
owner = User.objects.create_user(username="trash_owner")
grantee = User.objects.create_user(username="trash_grantee")
grantee.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
owner = UserFactory(username="trash_owner")
grantee = UserFactory(username="trash_grantee")
grant_global(grantee, "view_document")
doc = DocumentFactory(owner=owner)
doc.delete() # soft delete
assign_perm("view_document", grantee, doc)
grant_object(grantee, doc, "view_document")
rest_api_client.force_authenticate(user=grantee)
response = rest_api_client.get("/api/trash/")
@@ -542,8 +575,8 @@ class TestTrashViewExcludesExplicitlyGrantedDocuments:
)
class TestPermittedObjectIdsGenericModels:
def test_owner_sees_own_object(self, model, factory, perm):
owner = User.objects.create_user(username=f"owner_{model.__name__}")
stranger = User.objects.create_user(username=f"stranger_{model.__name__}")
owner = UserFactory(username=f"owner_{model.__name__}")
stranger = UserFactory(username=f"stranger_{model.__name__}")
owned = factory(owner=owner)
strangers = factory(owner=stranger)
@@ -556,14 +589,14 @@ class TestPermittedObjectIdsGenericModels:
@pytest.mark.parametrize("is_superuser", [False, True])
def test_inactive_user_sees_nothing(self, model, factory, perm, is_superuser):
suffix = f"{model.__name__}_{is_superuser}"
user = User.objects.create_user(
user = UserFactory(
username=f"inactive_{suffix}",
is_active=False,
is_superuser=is_superuser,
)
other = User.objects.create_user(username=f"other_{suffix}")
other = UserFactory(username=f"other_{suffix}")
granted = factory(owner=other)
assign_perm(perm, user, granted)
grant_object(user, granted, perm)
assert_visible_document_ids(
permitted_object_ids(user, model, perm),
@@ -576,7 +609,7 @@ class TestPermittedObjectIdsGenericModels:
)
def test_unowned_object_visible_to_everyone(self, model, factory, perm):
user = User.objects.create_user(username=f"user_{model.__name__}")
user = UserFactory(username=f"user_{model.__name__}")
unowned = factory(owner=None)
assert_visible_document_ids(
@@ -586,12 +619,12 @@ class TestPermittedObjectIdsGenericModels:
)
def test_explicit_permission_grants_visibility(self, model, factory, perm):
owner = User.objects.create_user(username=f"owner2_{model.__name__}")
grantee = User.objects.create_user(username=f"grantee_{model.__name__}")
stranger = User.objects.create_user(username=f"stranger2_{model.__name__}")
owner = UserFactory(username=f"owner2_{model.__name__}")
grantee = UserFactory(username=f"grantee_{model.__name__}")
stranger = UserFactory(username=f"stranger2_{model.__name__}")
shared = factory(owner=owner)
not_shared = factory(owner=owner)
assign_perm(perm, grantee, shared)
grant_object(grantee, shared, perm)
assert_visible_document_ids(
permitted_object_ids(grantee, model, perm),
@@ -610,13 +643,13 @@ class TestPermittedObjectIdsGenericModels:
factory,
perm,
):
owner = User.objects.create_user(username=f"owner3_{model.__name__}")
member = User.objects.create_user(username=f"member_{model.__name__}")
non_member = User.objects.create_user(username=f"nonmember_{model.__name__}")
owner = UserFactory(username=f"owner3_{model.__name__}")
member = UserFactory(username=f"member_{model.__name__}")
non_member = UserFactory(username=f"nonmember_{model.__name__}")
group = Group.objects.create(name=f"group_{model.__name__}")
member.groups.add(group)
shared = factory(owner=owner)
assign_perm(perm, group, shared)
grant_object(group, shared, perm)
assert_visible_document_ids(
permitted_object_ids(member, model, perm),
@@ -630,8 +663,8 @@ class TestPermittedObjectIdsGenericModels:
)
def test_superuser_sees_everything(self, model, factory, perm):
superuser = User.objects.create_superuser(username=f"root_{model.__name__}")
owner = User.objects.create_user(username=f"owner4_{model.__name__}")
superuser = UserFactory(username=f"root_{model.__name__}", superuser=True)
owner = UserFactory(username=f"owner4_{model.__name__}")
obj = factory(owner=owner)
assert_visible_document_ids(
@@ -644,8 +677,8 @@ class TestPermittedObjectIdsGenericModels:
@pytest.mark.django_db
class TestMatchingRespectsObjectPermissions:
def test_match_tags_only_considers_tags_visible_to_user(self):
owner = User.objects.create_user(username="tag_owner")
classifying_user = User.objects.create_user(username="classifier_user")
owner = UserFactory(username="tag_owner")
classifying_user = UserFactory(username="classifier_user")
visible_tag = TagFactory(
owner=owner,
match="invoice",
@@ -656,7 +689,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice",
matching_algorithm=Tag.MATCH_LITERAL,
)
assign_perm("view_tag", classifying_user, visible_tag)
grant_object(classifying_user, visible_tag, "view_tag")
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_tags(doc, classifier=None, user=classifying_user)
@@ -665,8 +698,8 @@ class TestMatchingRespectsObjectPermissions:
assert hidden_tag.pk not in matched_ids
def test_match_correspondents_only_considers_correspondents_visible_to_user(self):
owner = User.objects.create_user(username="correspondent_owner")
classifying_user = User.objects.create_user(username="classifier_user2")
owner = UserFactory(username="correspondent_owner")
classifying_user = UserFactory(username="classifier_user2")
visible_correspondent = CorrespondentFactory(
owner=owner,
match="invoice",
@@ -677,7 +710,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice",
matching_algorithm=Correspondent.MATCH_LITERAL,
)
assign_perm("view_correspondent", classifying_user, visible_correspondent)
grant_object(classifying_user, visible_correspondent, "view_correspondent")
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_correspondents(doc, classifier=None, user=classifying_user)
@@ -686,8 +719,8 @@ class TestMatchingRespectsObjectPermissions:
assert hidden_correspondent.pk not in matched_ids
def test_match_document_types_only_considers_document_types_visible_to_user(self):
owner = User.objects.create_user(username="document_type_owner")
classifying_user = User.objects.create_user(username="classifier_user3")
owner = UserFactory(username="document_type_owner")
classifying_user = UserFactory(username="classifier_user3")
visible_document_type = DocumentTypeFactory(
owner=owner,
match="invoice",
@@ -698,7 +731,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice",
matching_algorithm=DocumentType.MATCH_LITERAL,
)
assign_perm("view_documenttype", classifying_user, visible_document_type)
grant_object(classifying_user, visible_document_type, "view_documenttype")
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_document_types(doc, classifier=None, user=classifying_user)
@@ -707,8 +740,8 @@ class TestMatchingRespectsObjectPermissions:
assert hidden_document_type.pk not in matched_ids
def test_match_storage_paths_only_considers_storage_paths_visible_to_user(self):
owner = User.objects.create_user(username="storage_path_owner")
classifying_user = User.objects.create_user(username="classifier_user4")
owner = UserFactory(username="storage_path_owner")
classifying_user = UserFactory(username="classifier_user4")
visible_storage_path = StoragePathFactory(
owner=owner,
match="invoice",
@@ -719,7 +752,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice",
matching_algorithm=StoragePath.MATCH_LITERAL,
)
assign_perm("view_storagepath", classifying_user, visible_storage_path)
grant_object(classifying_user, visible_storage_path, "view_storagepath")
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_storage_paths(doc, classifier=None, user=classifying_user)
@@ -731,14 +764,12 @@ class TestMatchingRespectsObjectPermissions:
@pytest.mark.django_db
class TestBulkEditObjectsApplyToAllPermissionBoundary:
def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client):
owner = User.objects.create_user(username="tags_owner")
requester = User.objects.create_user(username="tags_requester")
new_owner = User.objects.create_user(username="tags_new_owner")
owner = UserFactory(username="tags_owner")
requester = UserFactory(username="tags_requester")
new_owner = UserFactory(username="tags_new_owner")
# grant the global change_tag permission so the object-level
# filtering (not the global has_perm check) is what's under test
requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
grant_global(requester, "change_tag")
rest_api_client.force_authenticate(user=requester)
visible = TagFactory(owner=requester)
hidden = TagFactory(owner=owner)
@@ -771,16 +802,14 @@ class TestBulkEditObjectsApplyToAllPermissionBoundary:
request rather than being silently skipped. Editing permissions is
limited to the owner, same as documents.
"""
owner = User.objects.create_user(username="shared_tags_owner")
requester = User.objects.create_user(username="shared_tags_requester")
requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
owner = UserFactory(username="shared_tags_owner")
requester = UserFactory(username="shared_tags_requester")
grant_global(requester, "change_tag")
rest_api_client.force_authenticate(user=requester)
owned = TagFactory(owner=requester)
shared = TagFactory(owner=owner)
assign_perm("view_tag", requester, shared)
assign_perm("change_tag", requester, shared)
grant_object(requester, shared, "view_tag")
grant_object(requester, shared, "change_tag")
response = rest_api_client.post(
"/api/bulk_edit_objects/",
@@ -831,14 +860,12 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
would pass/fail based on FK cascade behavior, not on whether the
descendant-expansion logic itself respected per-object permissions.
"""
owner = User.objects.create_user(username="tag_hierarchy_owner")
requester = User.objects.create_user(username="tag_hierarchy_requester")
new_owner = User.objects.create_user(username="tag_hierarchy_new_owner")
owner = UserFactory(username="tag_hierarchy_owner")
requester = UserFactory(username="tag_hierarchy_requester")
new_owner = UserFactory(username="tag_hierarchy_new_owner")
# global change_tag permission so the has_perm() gate passes and the
# object-level permitted_object_ids filtering is what's under test
requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
grant_global(requester, "change_tag")
rest_api_client.force_authenticate(user=requester)
parent = TagFactory(owner=requester, name="parent-tag")
@@ -890,7 +917,7 @@ class TestRestrictQuerysetToVisible:
- The queryset is returned unfiltered, rather than
permitted_object_ids(None, ...)'s narrower "unowned rows only"
"""
owner = User.objects.create_user(username="vis_none_owner")
owner = UserFactory(username="vis_none_owner")
tag = TagFactory(owner=owner)
visible = restrict_queryset_to_visible(Tag.objects.all(), None, "view_tag")
@@ -907,8 +934,8 @@ class TestRestrictQuerysetToVisible:
- The queryset is returned unfiltered, skipping the permission
lookup entirely
"""
superuser = User.objects.create_superuser(username="vis_active_super")
owner = User.objects.create_user(username="vis_active_super_owner")
superuser = UserFactory(username="vis_active_super", superuser=True)
owner = UserFactory(username="vis_active_super_owner")
tag = TagFactory(owner=owner)
visible = restrict_queryset_to_visible(
@@ -930,7 +957,7 @@ class TestRestrictQuerysetToVisible:
deactivation has to win over the superuser shortcut, matching
permitted_object_ids's own ordering
"""
user = User.objects.create_user(
user = UserFactory(
username="vis_inactive_super",
is_active=False,
is_superuser=True,
@@ -951,8 +978,8 @@ class TestRestrictQuerysetToVisible:
THEN:
- Only the rows permitted_object_ids() reports are visible
"""
user = User.objects.create_user(username="vis_regular")
other = User.objects.create_user(username="vis_regular_other")
user = UserFactory(username="vis_regular")
other = UserFactory(username="vis_regular_other")
own = TagFactory(owner=user)
hidden = TagFactory(owner=other)
@@ -1,11 +1,11 @@
import pytest
from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from rest_framework.test import APIRequestFactory
from documents.filters import PermittedObjectsFilter
from documents.models import Tag
from documents.tests.factories import TagFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_object
class _DummyView:
@@ -15,8 +15,8 @@ class _DummyView:
@pytest.mark.django_db
class TestPermittedObjectsFilter:
def test_superuser_bypasses_filtering_entirely(self):
superuser = User.objects.create_superuser(username="root")
owner = User.objects.create_user(username="owner")
superuser = UserFactory(username="root", superuser=True)
owner = UserFactory(username="owner")
TagFactory(owner=owner)
request = APIRequestFactory().get("/")
request.user = superuser
@@ -29,13 +29,13 @@ class TestPermittedObjectsFilter:
assert result.count() == Tag.objects.count()
def test_non_superuser_sees_only_owned_unowned_and_granted(self):
owner = User.objects.create_user(username="owner")
grantee = User.objects.create_user(username="grantee")
owner = UserFactory(username="owner")
grantee = UserFactory(username="grantee")
owned = TagFactory(owner=grantee)
unowned = TagFactory(owner=None)
granted = TagFactory(owner=owner)
hidden = TagFactory(owner=owner)
assign_perm("view_tag", grantee, granted)
grant_object(grantee, granted, "view_tag")
request = APIRequestFactory().get("/")
request.user = grantee
@@ -49,11 +49,11 @@ class TestPermittedObjectsFilter:
assert hidden.pk not in visible_ids
def test_include_granted_false_excludes_explicitly_shared_objects(self):
owner = User.objects.create_user(username="owner2")
grantee = User.objects.create_user(username="grantee2")
owner = UserFactory(username="owner2")
grantee = UserFactory(username="grantee2")
owned = TagFactory(owner=grantee)
granted = TagFactory(owner=owner)
assign_perm("view_tag", grantee, granted)
grant_object(grantee, granted, "view_tag")
request = APIRequestFactory().get("/")
request.user = grantee
@@ -74,15 +74,15 @@ class TestPermittedObjectsFilter:
[("inactive", False), ("inactive_super", True)],
)
def test_inactive_user_sees_nothing(self, username: str, *, is_superuser: bool):
user = User.objects.create_user(
user = UserFactory(
username=username,
is_active=False,
is_superuser=is_superuser,
)
TagFactory(owner=None)
TagFactory(owner=user)
granted = TagFactory(owner=User.objects.create_user(username=f"o_{username}"))
assign_perm("view_tag", user, granted)
granted = TagFactory(owner=UserFactory(username=f"o_{username}"))
grant_object(user, granted, "view_tag")
request = APIRequestFactory().get("/")
request.user = user
@@ -94,7 +94,7 @@ class TestPermittedObjectsFilter:
assert result.count() == 0
def test_inactive_user_sees_nothing_with_include_granted_false(self):
user = User.objects.create_user(username="inactive_owner", is_active=False)
user = UserFactory(username="inactive_owner", is_active=False)
TagFactory(owner=user)
TagFactory(owner=None)
request = APIRequestFactory().get("/")
+8 -9
View File
@@ -19,7 +19,7 @@ if TYPE_CHECKING:
from collections.abc import Iterable
from documents.models import Document
from documents.tests.conftest import PaperlessDirs
from paperless_testing.dirs import PaperlessDirs
class TestSanityCheckMessages:
@@ -46,14 +46,14 @@ class TestSanityCheckMessages:
class TestCheckSanityNoDocuments:
"""Sanity checks against an empty archive."""
@pytest.mark.usefixtures("_media_settings")
@pytest.mark.usefixtures("paperless_dirs")
def test_no_documents(self) -> None:
messages = check_sanity()
assert not messages.has_error
assert not messages.has_warning
assert messages.total_issue_count == 0
@pytest.mark.usefixtures("_media_settings")
@pytest.mark.usefixtures("paperless_dirs")
def test_no_issues_logs_clean(self, caplog: pytest.LogCaptureFixture) -> None:
messages = check_sanity()
with caplog.at_level(logging.INFO, logger="paperless.sanity_checker"):
@@ -214,18 +214,17 @@ class TestCheckSanityOrphans:
sample_doc: Document,
paperless_dirs: PaperlessDirs,
) -> None:
(paperless_dirs.originals / "orphan.pdf").touch()
(paperless_dirs.originals_dir / "orphan.pdf").touch()
messages = check_sanity()
assert messages.has_warning
assert any("Orphaned file" in m["message"] for m in messages[None])
@pytest.mark.usefixtures("_media_settings")
def test_ignorable_files_not_flagged(
self,
paperless_dirs: PaperlessDirs,
) -> None:
(paperless_dirs.media / ".DS_Store").touch()
(paperless_dirs.media / "desktop.ini").touch()
(paperless_dirs.media_dir / ".DS_Store").touch()
(paperless_dirs.media_dir / "desktop.ini").touch()
messages = check_sanity()
assert not messages.has_warning
@@ -269,13 +268,13 @@ class TestCheckSanityLogMessages:
paperless_dirs: PaperlessDirs,
caplog: pytest.LogCaptureFixture,
) -> None:
(paperless_dirs.originals / "orphan.pdf").touch()
(paperless_dirs.originals_dir / "orphan.pdf").touch()
messages = check_sanity()
with caplog.at_level(logging.WARNING, logger="paperless.sanity_checker"):
messages.log_messages()
assert "Orphaned file" in caplog.text
@pytest.mark.usefixtures("_media_settings")
@pytest.mark.usefixtures("paperless_dirs")
def test_logs_unknown_doc_pk(self, caplog: pytest.LogCaptureFixture) -> None:
"""A doc PK not in the DB logs 'Unknown' as the title."""
messages = check_sanity()

Some files were not shown because too many files have changed in this diff Show More