Compare commits

..
Author SHA1 Message Date
shamoon c63afb47b2 Documentation: correct duplicates info (#14243) 2026-09-23 08:27:18 -07:00
135 changed files with 3586 additions and 3613 deletions
-2
View File
@@ -15,8 +15,6 @@
# Test related # Test related
**/.pytest_cache **/.pytest_cache
**/tests **/tests
src/paperless_testing
src/conftest.py
**/*.spec.ts **/*.spec.ts
**/htmlcov **/htmlcov
# Local folders # Local folders
+2 -2
View File
@@ -50,12 +50,12 @@ repos:
- 'prettier-plugin-organize-imports@4.3.0' - 'prettier-plugin-organize-imports@4.3.0'
# Python hooks # Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit - repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.7 rev: v0.16.5
hooks: hooks:
- id: ruff-check - id: ruff-check
- id: ruff-format - id: ruff-format
- repo: https://github.com/tox-dev/pyproject-fmt - repo: https://github.com/tox-dev/pyproject-fmt
rev: "v2.29.4" rev: "v2.28.1"
hooks: hooks:
- id: pyproject-fmt - id: pyproject-fmt
additional_dependencies: [tomli] additional_dependencies: [tomli]
+2 -5
View File
@@ -153,11 +153,8 @@ in similar existing documents, and the document chat can retrieve relevant conte
Enable it by setting Enable it by setting
[`PAPERLESS_AI_LLM_EMBEDDING_BACKEND`](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_BACKEND) [`PAPERLESS_AI_LLM_EMBEDDING_BACKEND`](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_BACKEND)
(`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). By default, the main (`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). The index is only
LLM API key and endpoint are used, but an optional embedding-specific[API key](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_API_KEY) built when AI is enabled **and** an embedding backend is set.
and [endpoint](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) can be configured.
The index is only built when AI is enabled **and** an embedding backend is set.
The index is updated automatically on a schedule controlled by The index is updated automatically on a schedule controlled by
[`PAPERLESS_LLM_INDEX_TASK_CRON`](configuration.md#PAPERLESS_LLM_INDEX_TASK_CRON) (daily by [`PAPERLESS_LLM_INDEX_TASK_CRON`](configuration.md#PAPERLESS_LLM_INDEX_TASK_CRON) (daily by
-20
View File
@@ -2133,13 +2133,6 @@ for language and resource considerations.
Defaults to None. Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_API_KEY=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_API_KEY) {#PAPERLESS_AI_LLM_EMBEDDING_API_KEY}
: The API key to use for the embedding backend. If not supplied, embeddings use
`PAPERLESS_AI_LLM_API_KEY`.
Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) {#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT} #### [`PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) {#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT}
: The endpoint / url to use for the embedding backend. If not supplied, embeddings use : The endpoint / url to use for the embedding backend. If not supplied, embeddings use
@@ -2224,19 +2217,6 @@ used with the OpenAI-compatible backend to target a custom provider or local gat
Defaults to true, which allows internal endpoints. Defaults to true, which allows internal endpoints.
#### [`PAPERLESS_AI_LLM_EXTRA_PARAMS=<json>`](#PAPERLESS_AI_LLM_EXTRA_PARAMS) {#PAPERLESS_AI_LLM_EXTRA_PARAMS}
: A JSON object of extra parameters sent with every LLM request, for providers that require a parameter Paperless does not
set itself. Values here override Paperless' own, and no validation is performed. Whatever you put here is passed to the
backend as-is, so an invalid parameter will simply be rejected by your provider. For example, current OpenAI reasoning
models refuse tool calls on the chat completions API unless reasoning is off:
```
PAPERLESS_AI_LLM_EXTRA_PARAMS={"reasoning_effort": "none"}
```
Defaults to empty, which adds nothing to requests.
#### [`PAPERLESS_LLM_INDEX_TASK_CRON=<cron expression>`](#PAPERLESS_LLM_INDEX_TASK_CRON) {#PAPERLESS_LLM_INDEX_TASK_CRON} #### [`PAPERLESS_LLM_INDEX_TASK_CRON=<cron expression>`](#PAPERLESS_LLM_INDEX_TASK_CRON) {#PAPERLESS_LLM_INDEX_TASK_CRON}
: Configures the schedule to update the AI embeddings of text content and metadata for all documents. Only performed if : Configures the schedule to update the AI embeddings of text content and metadata for all documents. Only performed if
-1
View File
@@ -150,7 +150,6 @@ pnpm ng build --configuration production
is loaded as well. However, the tests rely on the default is loaded as well. However, the tests rely on the default
configuration. This is not ideal. But for now, make sure no settings configuration. This is not ideal. But for now, make sure no settings
except for DEBUG are overridden when testing. except for DEBUG are overridden when testing.
- Tests run in a random order each session, so that one test cannot quietly depend on another having run first. The seed is printed at the top of the run; pass `--randomly-seed=<seed>` to replay that exact order, or `--randomly-seed=last` to repeat the previous run.
!!! note !!! note
+3 -1
View File
@@ -76,7 +76,9 @@ is not supported by any of the available parsers.
**A:** Not by default. As of v3, a file whose contents match an existing document is still **A:** Not by default. As of v3, a file whose contents match an existing document is still
consumed, and the duplicate is flagged in the UI — open the document and check the consumed, and the duplicate is flagged in the UI — open the document and check the
**Duplicates** tab to review documents that share the same content. If you prefer the old **Duplicates** tab to review documents that share the same content, or filter the document
list by **Duplicates** to find all of them (see
[Duplicate documents](usage.md#duplicate-documents)). If you prefer the old
behavior of rejecting duplicates during consumption, set behavior of rejecting duplicates during consumption, set
[`PAPERLESS_CONSUMER_DELETE_DUPLICATES`](configuration.md#PAPERLESS_CONSUMER_DELETE_DUPLICATES) [`PAPERLESS_CONSUMER_DELETE_DUPLICATES`](configuration.md#PAPERLESS_CONSUMER_DELETE_DUPLICATES)
to `true`. to `true`.
+7 -8
View File
@@ -299,19 +299,18 @@ for details.
### Duplicate documents ### Duplicate documents
By default, Paperless-ngx **does not reject duplicates**. If you consume a file whose By default, Paperless-ngx **does not reject duplicates**. If you consume a file whose
contents exactly match an existing document (same checksum), the new copy is still contents match an existing document (same original or archive checksum), the new copy is
consumed and a warning is logged. The task entry for the upload also flags that a still consumed and a warning is logged.
duplicate was detected and links to the existing document(s).
To review duplicates, open a document and switch to the **Duplicates** tab on the When a document has duplicates, a **Duplicates** tab appears on its detail page, listing
document detail page. It lists other documents that share the same content, including any the other documents you can view that share the same content (including any in the trash).
that are in the trash (shown with a badge), and links to each so you can decide which to To find all documents with duplicates, choose **Duplicates** in the document list's text
keep. filter dropdown, or use `has_duplicates=true` in the REST API.
If you would rather reject duplicates at consumption time (the pre-v3 behavior), set If you would rather reject duplicates at consumption time (the pre-v3 behavior), set
[`PAPERLESS_CONSUMER_DELETE_DUPLICATES`](configuration.md#PAPERLESS_CONSUMER_DELETE_DUPLICATES) [`PAPERLESS_CONSUMER_DELETE_DUPLICATES`](configuration.md#PAPERLESS_CONSUMER_DELETE_DUPLICATES)
to `true`. The duplicate file is then deleted instead of consumed, and the task fails with to `true`. The duplicate file is then deleted instead of consumed, and the task fails with
a "document already exists" message. a "Document already exists" message linking to the existing document.
## Document Suggestions ## Document Suggestions
+5 -16
View File
@@ -1,9 +1,7 @@
[project] [project]
name = "paperless-ngx" name = "paperless-ngx"
version = "3.2.1" version = "3.2.1"
description = """\ description = "A community-supported supercharged document management system: scan, index and archive all your physical documents"
A community-supported supercharged document management system: scan, index and archive all your physical documents\
"""
readme = "README.md" readme = "README.md"
requires-python = ">=3.11" requires-python = ">=3.11"
classifiers = [ classifiers = [
@@ -12,7 +10,6 @@ classifiers = [
"Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14", "Programming Language :: Python :: 3.14",
"Programming Language :: Python :: 3.15",
] ]
# TODO: Move certain things to groups and then utilize that further # TODO: Move certain things to groups and then utilize that further
# This will allow testing to not install a webserver, mysql, etc # This will allow testing to not install a webserver, mysql, etc
@@ -120,7 +117,7 @@ testing = [
"pytest-env~=1.7.0", "pytest-env~=1.7.0",
"pytest-httpx", "pytest-httpx",
"pytest-mock~=3.15.1", "pytest-mock~=3.15.1",
"pytest-randomly~=5.0.0", # "pytest-randomly~=4.0.1",
"pytest-rerunfailures~=16.4", "pytest-rerunfailures~=16.4",
"pytest-sugar", "pytest-sugar",
"pytest-xdist~=3.8.0", "pytest-xdist~=3.8.0",
@@ -247,10 +244,6 @@ per-file-ignores."docker/wait-for-redis.py" = [
per-file-ignores."src/documents/models.py" = [ per-file-ignores."src/documents/models.py" = [
"SIM115", "SIM115",
] ]
per-file-ignores."src/documents/tests/*.py" = [
"TID251",
]
flake8-tidy-imports.banned-api."documents.tests".msg = "Shared test infrastructure lives in src/paperless_testing/."
isort.force-single-line = true isort.force-single-line = true
[tool.codespell] [tool.codespell]
@@ -278,9 +271,9 @@ plugins = [
] ]
[tool.pyrefly] [tool.pyrefly]
baseline = ".pyrefly-baseline.json"
python-platform = "linux" python-platform = "linux"
search-path = [ "src" ] search-path = [ "src" ]
baseline = ".pyrefly-baseline.json"
[tool.django-stubs] [tool.django-stubs]
django_settings_module = "paperless.settings" django_settings_module = "paperless.settings"
@@ -333,8 +326,6 @@ PAPERLESS_CACHE_BACKEND = "django.core.cache.backends.locmem.LocMemCache"
PAPERLESS_CHANNELS_BACKEND = "channels.layers.InMemoryChannelLayer" PAPERLESS_CHANNELS_BACKEND = "channels.layers.InMemoryChannelLayer"
# I don't think anything hits this, but just in case, basically infinite # I don't think anything hits this, but just in case, basically infinite
PAPERLESS_TOKEN_THROTTLE_RATE = "1000/min" PAPERLESS_TOKEN_THROTTLE_RATE = "1000/min"
# The 0.1s production default trips on a stalled CI runner, the date parsing tests then find no dates
PAPERLESS_MATCH_REGEX_TIMEOUT_SECONDS = "5"
[tool.coverage.run] [tool.coverage.run]
source = [ source = [
@@ -343,15 +334,13 @@ source = [
omit = [ omit = [
"*/tests/*", "*/tests/*",
"manage.py", "manage.py",
"paperless/auth.py",
"paperless/wsgi.py", "paperless/wsgi.py",
"src/conftest.py", "paperless/auth.py",
"src/paperless_testing/*",
] ]
[tool.coverage.report] [tool.coverage.report]
exclude_also = [ exclude_also = [
"if AUDIT_LOG_ENABLED:",
"if settings.AUDIT_LOG_ENABLED:", "if settings.AUDIT_LOG_ENABLED:",
"if AUDIT_LOG_ENABLED:",
"if TYPE_CHECKING:", "if TYPE_CHECKING:",
] ]
+21 -35
View File
@@ -9745,7 +9745,7 @@
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">351</context> <context context-type="linenumber">348</context>
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context> <context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -9760,7 +9760,7 @@
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">344</context> <context context-type="linenumber">341</context>
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context> <context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -10016,56 +10016,56 @@
<source>Reset filters / selection</source> <source>Reset filters / selection</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">332</context> <context context-type="linenumber">329</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4135055128446167640" datatype="html"> <trans-unit id="4135055128446167640" datatype="html">
<source>Open first [selected] document</source> <source>Open first [selected] document</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">360</context> <context context-type="linenumber">357</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3629960544875360046" datatype="html"> <trans-unit id="3629960544875360046" datatype="html">
<source>Previous page</source> <source>Previous page</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">376</context> <context context-type="linenumber">373</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3337301694210287595" datatype="html"> <trans-unit id="3337301694210287595" datatype="html">
<source>Next page</source> <source>Next page</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">388</context> <context context-type="linenumber">385</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="2155249406916744630" datatype="html"> <trans-unit id="2155249406916744630" datatype="html">
<source>View &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot; saved successfully.</source> <source>View &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot; saved successfully.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">422</context> <context context-type="linenumber">419</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4646273665293421938" datatype="html"> <trans-unit id="4646273665293421938" datatype="html">
<source>Failed to save view &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot;.</source> <source>Failed to save view &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot;.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">428</context> <context context-type="linenumber">425</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6837554170707123455" datatype="html"> <trans-unit id="6837554170707123455" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully.</source> <source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">497</context> <context context-type="linenumber">494</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6028096992841030074" datatype="html"> <trans-unit id="6028096992841030074" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully, but could not update visibility settings.</source> <source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully, but could not update visibility settings.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">503</context> <context context-type="linenumber">500</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="739880801667335279" datatype="html"> <trans-unit id="739880801667335279" datatype="html">
@@ -12018,95 +12018,81 @@
<context context-type="linenumber">351</context> <context context-type="linenumber">351</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="861068592166833023" datatype="html">
<source>LLM Embedding API Key</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">358</context>
</context-group>
</trans-unit>
<trans-unit id="2929108042259892948" datatype="html">
<source>Used for embeddings when set, otherwise LLM API key is used.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">360</context>
</context-group>
</trans-unit>
<trans-unit id="3554114880473286122" datatype="html"> <trans-unit id="3554114880473286122" datatype="html">
<source>LLM Embedding Endpoint</source> <source>LLM Embedding Endpoint</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">366</context> <context context-type="linenumber">358</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="1044242175651289991" datatype="html"> <trans-unit id="1044242175651289991" datatype="html">
<source>LLM Embedding Chunk Size</source> <source>LLM Embedding Chunk Size</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">373</context> <context context-type="linenumber">365</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="7218245223139363113" datatype="html"> <trans-unit id="7218245223139363113" datatype="html">
<source>LLM Context Size</source> <source>LLM Context Size</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">380</context> <context context-type="linenumber">372</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4234495692726214397" datatype="html"> <trans-unit id="4234495692726214397" datatype="html">
<source>LLM Backend</source> <source>LLM Backend</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">387</context> <context context-type="linenumber">379</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="7935234833834000002" datatype="html"> <trans-unit id="7935234833834000002" datatype="html">
<source>LLM Model</source> <source>LLM Model</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">395</context> <context context-type="linenumber">387</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="1980550530387803165" datatype="html"> <trans-unit id="1980550530387803165" datatype="html">
<source>LLM API Key</source> <source>LLM API Key</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">402</context> <context context-type="linenumber">394</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6126617860376156501" datatype="html"> <trans-unit id="6126617860376156501" datatype="html">
<source>LLM Endpoint</source> <source>LLM Endpoint</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">409</context> <context context-type="linenumber">401</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6572826277249350975" datatype="html"> <trans-unit id="6572826277249350975" datatype="html">
<source>LLM Output Language</source> <source>LLM Output Language</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">416</context> <context context-type="linenumber">408</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3284403507172415792" datatype="html"> <trans-unit id="3284403507172415792" datatype="html">
<source>Language to use for generated AI suggestions. When unset, AI suggestions use the user&apos;s display language if explicitly set.</source> <source>Language to use for generated AI suggestions. When unset, AI suggestions use the user&apos;s display language if explicitly set.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">420</context> <context context-type="linenumber">412</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4493921125434706859" datatype="html"> <trans-unit id="4493921125434706859" datatype="html">
<source>LLM Request Timeout</source> <source>LLM Request Timeout</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">424</context> <context context-type="linenumber">416</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="483994032066441287" datatype="html"> <trans-unit id="483994032066441287" datatype="html">
<source>Timeout in seconds for LLM requests.</source> <source>Timeout in seconds for LLM requests.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context> <context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">428</context> <context context-type="linenumber">420</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="1055686627716339120" datatype="html"> <trans-unit id="1055686627716339120" datatype="html">
@@ -146,19 +146,6 @@ describe('DocumentListComponent', () => {
expect(reloadSpy).toHaveBeenCalled() expect(reloadSpy).toHaveBeenCalled()
}) })
it('should stop reloading on document deleted after destroy', () => {
const reloadSpy = jest.spyOn(documentListService, 'reload')
const documentDeletedSubject = new Subject<boolean>()
jest
.spyOn(websocketStatusService, 'onDocumentDeleted')
.mockReturnValue(documentDeletedSubject)
fixture.detectChanges()
fixture.destroy()
reloadSpy.mockClear()
documentDeletedSubject.next(true)
expect(reloadSpy).not.toHaveBeenCalled()
})
it('should show score sort fields on fulltext queries', () => { it('should show score sort fields on fulltext queries', () => {
documentListService.setFilterRules([ documentListService.setFilterRules([
{ {
@@ -270,12 +270,9 @@ export class DocumentListComponent
this.list.reload() this.list.reload()
}) })
this.websocketStatusService this.websocketStatusService.onDocumentDeleted().subscribe(() => {
.onDocumentDeleted() this.list.reload()
.pipe(takeUntil(this.unsubscribeNotifier)) })
.subscribe(() => {
this.list.reload()
})
this.route.paramMap this.route.paramMap
.pipe( .pipe(
-9
View File
@@ -353,14 +353,6 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_MODEL', config_key: 'PAPERLESS_AI_LLM_EMBEDDING_MODEL',
category: ConfigCategory.AI, category: ConfigCategory.AI,
}, },
{
key: 'llm_embedding_api_key',
title: $localize`LLM Embedding API Key`,
type: ConfigOptionType.Password,
note: $localize`Used for embeddings when set, otherwise LLM API key is used.`,
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_API_KEY',
category: ConfigCategory.AI,
},
{ {
key: 'llm_embedding_endpoint', key: 'llm_embedding_endpoint',
title: $localize`LLM Embedding Endpoint`, title: $localize`LLM Embedding Endpoint`,
@@ -465,7 +457,6 @@ export interface PaperlessConfig extends ObjectWithId {
ai_enabled: boolean ai_enabled: boolean
llm_embedding_backend: string llm_embedding_backend: string
llm_embedding_model: string llm_embedding_model: string
llm_embedding_api_key: string
llm_embedding_endpoint: string llm_embedding_endpoint: string
llm_embedding_chunk_size: number llm_embedding_chunk_size: number
llm_context_size: number llm_context_size: number
-151
View File
@@ -1,151 +0,0 @@
"""Fixtures available to every Paperless-ngx app.
Loaded automatically for every test path. Keep module-scope imports minimal:
this file is imported for every session, so anything heavy belongs inside
the fixture body that needs it.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
if TYPE_CHECKING:
from collections.abc import Generator
from pathlib import Path
from django.contrib.auth.models import User
from pytest_django.fixtures import Settings
from rest_framework.test import APIClient
from paperless_testing.dirs import PaperlessDirs
from paperless_testing.fakes.progress import FakeProgressManager
@pytest.fixture(scope="session", autouse=True)
def faker_session_locale() -> str:
"""Pin Faker's locale so generated data does not follow the host locale.
The seed itself is left to pytest-randomly, which derives one per run.
"""
return "en_US"
@pytest.fixture(autouse=True)
def _fast_password_hasher(settings: Settings) -> None:
"""Hash test passwords with MD5 instead of Django's default PBKDF2.
PBKDF2 is deliberately slow, and every ``admin_user`` or
``create_superuser`` call pays for it: about 600 ms each. No test depends
on the hash format, only on ``check_password`` and on the stored value
changing when the password does.
"""
settings.PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
@pytest.fixture(autouse=True)
def _clear_content_type_caches() -> None:
"""Clear Django's ContentType cache and guardian's lru_cache before each test.
Tests that delete and reinsert ContentType/Permission rows (e.g. the
importer) corrupt both caches. Without this fixture a subsequent test on
the same xdist worker sees stale ContentType objects and guardian raises
MixedContentTypeError.
"""
from django.contrib.contenttypes.models import ContentType
from guardian.shortcuts import clear_ct_cache
ContentType.objects.clear_cache()
clear_ct_cache()
@pytest.fixture(autouse=True)
def _clear_django_caches() -> None:
"""Clear every configured cache before each test.
Cached values outlive the test that wrote them: the classifier keys its
vectorized content on a hash of the content itself, so a second test
generating the same fixture data takes the cache-hit path and never calls
the code it is asserting against.
"""
from django.core.cache import caches
for cache in caches.all(initialized_only=False):
cache.clear()
@pytest.fixture
def paperless_dirs(
tmp_path: Path,
settings: Settings,
) -> Generator[PaperlessDirs, None, None]:
"""The standard temp directory layout, applied to Django settings."""
from documents.search import reset_backend
from paperless_testing.dirs import build_paperless_dirs
from paperless_testing.dirs import dirs_settings
dirs = build_paperless_dirs(tmp_path)
for name, value in dirs_settings(dirs).items():
setattr(settings, name, value)
# Not directory settings, but they are needed alongside the layout by the
# sanity checker tests.
settings.IGNORABLE_FILES = {".DS_Store", "Thumbs.db", "desktop.ini"}
settings.APP_LOGO = ""
reset_backend()
yield dirs
reset_backend()
@pytest.fixture
def rest_api_client() -> APIClient:
"""The basic DRF APIClient, unauthenticated."""
from rest_framework.test import APIClient
return APIClient()
@pytest.fixture
def regular_user(db: None) -> User:
"""Unprivileged user for permission boundary tests."""
from paperless_testing.factories import UserFactory
return UserFactory(username="regular")
@pytest.fixture
def admin_client(rest_api_client: APIClient, admin_user: User) -> APIClient:
"""Admin client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture
def v9_client(rest_api_client: APIClient, admin_user: User) -> APIClient:
"""Admin client pre-authenticated and sending the v9 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=9")
return rest_api_client
@pytest.fixture
def user_client(rest_api_client: APIClient, regular_user: User) -> APIClient:
"""Regular-user client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=regular_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture
def fake_progress_manager(
monkeypatch: pytest.MonkeyPatch,
) -> type[FakeProgressManager]:
"""Replace documents.tasks.ProgressManager with the fake, so consuming a file
in a test never tries to reach a broker."""
from paperless_testing.fakes.progress import FakeProgressManager
monkeypatch.setattr("documents.tasks.ProgressManager", FakeProgressManager)
return FakeProgressManager
+1
View File
@@ -26,6 +26,7 @@ class DocumentsConfig(AppConfig):
document_consumption_finished.connect(set_document_type) document_consumption_finished.connect(set_document_type)
document_consumption_finished.connect(set_tags) document_consumption_finished.connect(set_tags)
document_consumption_finished.connect(set_storage_path) document_consumption_finished.connect(set_storage_path)
document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(run_workflows_added) document_consumption_finished.connect(run_workflows_added)
document_consumption_finished.connect(add_to_index) document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(add_or_update_document_in_llm_index) document_consumption_finished.connect(add_or_update_document_in_llm_index)
+63 -50
View File
@@ -196,49 +196,52 @@ class WriteBatch:
return self._raw_writer return self._raw_writer
def __enter__(self) -> Self: def __enter__(self) -> Self:
lock_path = self._backend._path / ".tantivy.lock" if self._backend._path is not None:
self._lock = filelock.FileLock(str(lock_path)) lock_path = self._backend._path / ".tantivy.lock"
for attempt in range(_LOCK_RETRY_ATTEMPTS): self._lock = filelock.FileLock(str(lock_path))
try: for attempt in range(_LOCK_RETRY_ATTEMPTS):
self._lock.acquire(timeout=self._lock_timeout) try:
break self._lock.acquire(timeout=self._lock_timeout)
except filelock.Timeout: break
if attempt == _LOCK_RETRY_ATTEMPTS - 1: except filelock.Timeout:
raise SearchIndexLockError( if attempt == _LOCK_RETRY_ATTEMPTS - 1:
f"Could not acquire index lock after {_LOCK_RETRY_ATTEMPTS} " raise SearchIndexLockError(
f"attempts (timeout={self._lock_timeout}s each)", f"Could not acquire index lock after {_LOCK_RETRY_ATTEMPTS} "
f"attempts (timeout={self._lock_timeout}s each)",
)
sleep_s = random.uniform(
0,
min(_LOCK_BACKOFF_CAP, _LOCK_BACKOFF_BASE * (2**attempt)),
) )
sleep_s = random.uniform( logger.debug(
0, "Index lock contention; retrying in %.2fs (attempt %d/%d)",
min(_LOCK_BACKOFF_CAP, _LOCK_BACKOFF_BASE * (2**attempt)), sleep_s,
) attempt + 1,
logger.debug( _LOCK_RETRY_ATTEMPTS,
"Index lock contention; retrying in %.2fs (attempt %d/%d)", )
sleep_s, time.sleep(sleep_s)
attempt + 1,
_LOCK_RETRY_ATTEMPTS,
)
time.sleep(sleep_s)
# Open a fresh Index (and thus a fresh Tantivy ManagedDirectory) # Open a fresh Index (and thus a fresh Tantivy ManagedDirectory)
# for the write, rather than reusing the process-local cached # for the write, rather than reusing the process-local cached
# index. ManagedDirectory loads its GC bookkeeping (.managed.json) # index. ManagedDirectory loads its GC bookkeeping (.managed.json)
# once, at construction, and never re-reads it; paperless runs # once, at construction, and never re-reads it; paperless runs
# several long-lived processes (Granian workers, Celery workers) # several long-lived processes (Granian workers, Celery workers)
# that take turns writing under the file lock above. A cached, # that take turns writing under the file lock above. A cached,
# long-lived writer index would carry a stale managed-files view # long-lived writer index would carry a stale managed-files view
# and, on commit, overwrite .managed.json with that stale view - # and, on commit, overwrite .managed.json with that stale view -
# permanently losing track of segment files other processes # permanently losing track of segment files other processes
# registered in the meantime, so they can never be garbage # registered in the meantime, so they can never be garbage
# collected. Reopening fresh here always picks up the current # collected. Reopening fresh here always picks up the current
# on-disk state. The long-lived self._backend._index is used for # on-disk state. The long-lived self._backend._index is used for
# reads only and is reloaded (not reopened) after commit below. # reads only and is reloaded (not reopened) after commit below.
write_index = tantivy.Index( write_index = tantivy.Index(
build_schema(), build_schema(),
path=str(self._backend._path), path=str(self._backend._path),
) )
register_tokenizers(write_index, settings.SEARCH_LANGUAGE) register_tokenizers(write_index, settings.SEARCH_LANGUAGE)
self._raw_writer = write_index.writer() self._raw_writer = write_index.writer()
else:
self._raw_writer = self._backend._index.writer()
return self return self
def __exit__(self, exc_type, exc_val, exc_tb): def __exit__(self, exc_type, exc_val, exc_tb):
@@ -369,8 +372,9 @@ class TantivyBackend:
Tantivy search backend with explicit lifecycle management. Tantivy search backend with explicit lifecycle management.
Provides full-text search capabilities using the Tantivy search engine. Provides full-text search capabilities using the Tantivy search engine.
Keeps a persistent on-disk index. Handles document indexing, search queries, Supports in-memory indexes (for testing) and persistent on-disk indexes
autocompletion, and "more like this" functionality. (for production use). Handles document indexing, search queries, autocompletion,
and "more like this" functionality.
The backend manages its own connection lifecycle and can be reset when The backend manages its own connection lifecycle and can be reset when
the underlying index directory changes (e.g., during test isolation). the underlying index directory changes (e.g., during test isolation).
@@ -404,7 +408,9 @@ class TantivyBackend:
}, },
) )
def __init__(self, path: Path): def __init__(self, path: Path | None = None):
# path=None → in-memory index (for tests)
# path=some_dir → on-disk index (for production)
self._path = path self._path = path
self._raw_index: tantivy.Index | None = None self._raw_index: tantivy.Index | None = None
self._raw_schema: tantivy.Schema | None = None self._raw_schema: tantivy.Schema | None = None
@@ -423,13 +429,16 @@ class TantivyBackend:
""" """
Open or rebuild the index as needed. Open or rebuild the index as needed.
Checks if rebuilding is needed due to schema version or language For disk-based indexes, checks if rebuilding is needed due to schema
changes. Registers custom tokenizers after opening. version or language changes. Registers custom tokenizers after opening.
Safe to call multiple times - subsequent calls are no-ops. Safe to call multiple times - subsequent calls are no-ops.
""" """
if self._raw_index is not None: if self._raw_index is not None:
return # pragma: no cover return # pragma: no cover
self._raw_index = open_or_rebuild_index(self._path) if self._path is not None:
self._raw_index = open_or_rebuild_index(self._path)
else:
self._raw_index = tantivy.Index(build_schema())
register_tokenizers(self._raw_index, settings.SEARCH_LANGUAGE) register_tokenizers(self._raw_index, settings.SEARCH_LANGUAGE)
self._raw_schema = self._raw_index.schema self._raw_schema = self._raw_index.schema
@@ -1093,9 +1102,13 @@ class TantivyBackend:
writer's threads). Larger values buffer more docs in RAM before writer's threads). Larger values buffer more docs in RAM before
flushing a segment, deferring merge work; they do not avoid it. flushing a segment, deferring merge work; they do not avoid it.
""" """
wipe_index(self._path) # Create new index (on-disk or in-memory)
new_index = tantivy.Index(build_schema(), path=str(self._path)) if self._path is not None:
_write_sentinels(self._path) wipe_index(self._path)
new_index = tantivy.Index(build_schema(), path=str(self._path))
_write_sentinels(self._path)
else:
new_index = tantivy.Index(build_schema())
register_tokenizers(new_index, settings.SEARCH_LANGUAGE) register_tokenizers(new_index, settings.SEARCH_LANGUAGE)
# Point instance at the new index so _build_tantivy_doc uses it # Point instance at the new index so _build_tantivy_doc uses it
+1 -3
View File
@@ -2098,8 +2098,6 @@ class BulkEditSerializer(
if not isinstance(parameters["pages"], str): if not isinstance(parameters["pages"], str):
raise serializers.ValidationError("invalid pages specified") raise serializers.ValidationError("invalid pages specified")
page_count = Document.objects.get(id=document_id).page_count page_count = Document.objects.get(id=document_id).page_count
if not page_count:
raise serializers.ValidationError("document page count is unknown")
pages = [] pages = []
for group in parameters["pages"].split(","): for group in parameters["pages"].split(","):
start, is_range, end = group.partition("-") start, is_range, end = group.partition("-")
@@ -2109,7 +2107,7 @@ class BulkEditSerializer(
except ValueError as e: except ValueError as e:
raise serializers.ValidationError("invalid pages specified") from e raise serializers.ValidationError("invalid pages specified") from e
# Bound the range before building it, a huge one would exhaust memory # Bound the range before building it, a huge one would exhaust memory
if not 1 <= first <= last <= page_count: if not 1 <= first <= last or (page_count and last > page_count):
raise serializers.ValidationError("invalid pages specified") raise serializers.ValidationError("invalid pages specified")
pages.append(list(range(first, last + 1))) pages.append(list(range(first, last + 1)))
parameters["pages"] = pages parameters["pages"] = pages
+143 -18
View File
@@ -1,41 +1,88 @@
import shutil import shutil
import zoneinfo
from collections.abc import Generator
from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
import filelock import filelock
import pytest import pytest
from django.contrib.auth import get_user_model
from django.contrib.contenttypes.models import ContentType
from guardian.shortcuts import clear_ct_cache
from pytest_django.fixtures import Settings
from rest_framework.test import APIClient
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
UserModelT = get_user_model()
if TYPE_CHECKING: if TYPE_CHECKING:
from documents.models import Document from documents.models import Document
from paperless_testing.dirs import PaperlessDirs
@dataclass(frozen=True, slots=True)
class PaperlessDirs:
"""Standard Paperless-ngx directory layout for tests."""
media: Path
originals: Path
archive: Path
thumbnails: Path
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def document_samples_dir() -> Path: def samples_dir() -> Path:
"""Path to the shared test sample documents.""" """Path to the shared test sample documents."""
return Path(__file__).parent / "samples" / "documents" return Path(__file__).parent / "samples" / "documents"
@pytest.fixture()
def paperless_dirs(tmp_path: Path) -> PaperlessDirs:
"""Create and return the directory structure for testing."""
media = tmp_path / "media"
dirs = PaperlessDirs(
media=media,
originals=media / "documents" / "originals",
archive=media / "documents" / "archive",
thumbnails=media / "documents" / "thumbnails",
)
for d in (dirs.originals, dirs.archive, dirs.thumbnails):
d.mkdir(parents=True)
return dirs
@pytest.fixture()
def _media_settings(paperless_dirs: PaperlessDirs, settings) -> None:
"""Configure Django settings to point at temp directories."""
settings.MEDIA_ROOT = paperless_dirs.media
settings.ORIGINALS_DIR = paperless_dirs.originals
settings.ARCHIVE_DIR = paperless_dirs.archive
settings.THUMBNAIL_DIR = paperless_dirs.thumbnails
settings.MEDIA_LOCK = paperless_dirs.media / "media.lock"
settings.IGNORABLE_FILES = {".DS_Store", "Thumbs.db", "desktop.ini"}
settings.APP_LOGO = ""
@pytest.fixture() @pytest.fixture()
def sample_doc( def sample_doc(
paperless_dirs: "PaperlessDirs", paperless_dirs: PaperlessDirs,
document_samples_dir: Path, _media_settings: None,
samples_dir: Path,
) -> "Document": ) -> "Document":
"""Create a document with valid files and matching checksums.""" """Create a document with valid files and matching checksums."""
with filelock.FileLock(paperless_dirs.media_lock): with filelock.FileLock(paperless_dirs.media / "media.lock"):
shutil.copy( shutil.copy(
document_samples_dir / "originals" / "0000001.pdf", samples_dir / "originals" / "0000001.pdf",
paperless_dirs.originals_dir / "0000001.pdf", paperless_dirs.originals / "0000001.pdf",
) )
shutil.copy( shutil.copy(
document_samples_dir / "archive" / "0000001.pdf", samples_dir / "archive" / "0000001.pdf",
paperless_dirs.archive_dir / "0000001.pdf", paperless_dirs.archive / "0000001.pdf",
) )
shutil.copy( shutil.copy(
document_samples_dir / "thumbnails" / "0000001.webp", samples_dir / "thumbnails" / "0000001.webp",
paperless_dirs.thumbnail_dir / "0000001.webp", paperless_dirs.thumbnails / "0000001.webp",
) )
return DocumentFactory( return DocumentFactory(
@@ -50,17 +97,95 @@ def sample_doc(
) )
@pytest.fixture @pytest.fixture()
def _search_index(paperless_dirs: "PaperlessDirs") -> None: def _search_index(
"""Point the search backend at a fresh, empty index directory. tmp_path: Path,
settings: Settings,
) -> Generator[None, None, None]:
"""Create a temp index directory and point INDEX_DIR at it.
paperless_dirs owns INDEX_DIR and resets the backend singleton on both Resets the backend singleton before and after so each test gets a clean
sides of the test, so requesting it is all that is needed. index rather than reusing a stale singleton from another test.
""" """
from documents.search import reset_backend
index_dir = tmp_path / "index"
index_dir.mkdir()
settings.INDEX_DIR = index_dir
reset_backend()
yield
reset_backend()
@pytest.fixture()
def settings_timezone(settings: Settings) -> zoneinfo.ZoneInfo:
return zoneinfo.ZoneInfo(settings.TIME_ZONE)
@pytest.fixture @pytest.fixture
def searchable_document(_search_index: None) -> "Document": def rest_api_client():
"""
The basic DRF ApiClient
"""
yield APIClient()
@pytest.fixture()
def regular_user(django_user_model: type[UserModelT]) -> UserModelT:
"""Unprivileged authenticated user for permission boundary tests."""
return django_user_model.objects.create_user(username="regular", password="regular")
@pytest.fixture()
def admin_client(rest_api_client: APIClient, admin_user: UserModelT) -> APIClient:
"""Admin client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture()
def v9_client(rest_api_client: APIClient, admin_user: UserModelT) -> APIClient:
"""Admin client pre-authenticated and sending the v9 Accept header."""
rest_api_client.force_authenticate(user=admin_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=9")
return rest_api_client
@pytest.fixture()
def user_client(rest_api_client: APIClient, regular_user: UserModelT) -> APIClient:
"""Regular-user client pre-authenticated and sending the v10 Accept header."""
rest_api_client.force_authenticate(user=regular_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture(autouse=True)
def _clear_content_type_caches() -> None:
"""Clear Django's ContentType cache and guardian's lru_cache before each test.
Tests that delete and reinsert ContentType/Permission rows (e.g. the
importer) corrupt both caches. Without this fixture a subsequent test on
the same xdist worker sees stale ContentType objects and guardian raises
MixedContentTypeError.
"""
ContentType.objects.clear_cache()
clear_ct_cache()
@pytest.fixture(scope="session", autouse=True)
def faker_session_locale():
"""Set Faker locale for reproducibility."""
return "en_US"
@pytest.fixture(scope="session", autouse=True)
def faker_seed():
return 12345
@pytest.fixture
def indexed_document(_search_index: None) -> "Document":
"""One searchable document, for tests about what the search endpoint """One searchable document, for tests about what the search endpoint
returns rather than about what it finds. returns rather than about what it finds.
""" """
@@ -6,7 +6,6 @@ from __future__ import annotations
import factory import factory
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.contrib.auth.models import Group
from factory.django import DjangoModelFactory from factory.django import DjangoModelFactory
from documents.models import Correspondent from documents.models import Correspondent
@@ -72,13 +71,6 @@ class DocumentFactory(DjangoModelFactory[Document]):
storage_path = None storage_path = None
class GroupFactory(DjangoModelFactory[Group]):
class Meta:
model = Group
name = factory.Sequence(lambda n: f"group{n}")
class UserFactory(DjangoModelFactory[UserModelT]): class UserFactory(DjangoModelFactory[UserModelT]):
class Meta: class Meta:
model = UserModelT model = UserModelT
@@ -86,10 +78,7 @@ class UserFactory(DjangoModelFactory[UserModelT]):
username = factory.Sequence(lambda n: f"user{n}") username = factory.Sequence(lambda n: f"user{n}")
is_staff = False is_staff = False
is_superuser = False is_superuser = False
# Hashing a real password costs about half a second per user, and no test password = factory.django.Password("test")
# authenticates with its password. None gives the unusable password
# create_user() gives.
password = factory.django.Password(None)
class Params: class Params:
superuser = factory.Trait(is_staff=True, is_superuser=True) superuser = factory.Trait(is_staff=True, is_superuser=True)
-10
View File
@@ -1,10 +0,0 @@
import re
def dummy_preprocess(content: str) -> str:
"""
Simpler, faster pre-processing for testing purposes
"""
content = content.lower().strip()
content = re.sub(r"\s+", " ", content)
return content
@@ -15,11 +15,11 @@ from rich.console import Console
from documents.management.commands.document_sanity_checker import Command from documents.management.commands.document_sanity_checker import Command
from documents.sanity_checker import SanityCheckMessages from documents.sanity_checker import SanityCheckMessages
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from documents.models import Document from documents.models import Document
from paperless_testing.dirs import PaperlessDirs from documents.tests.conftest import PaperlessDirs
def _render_to_string(messages: SanityCheckMessages) -> str: def _render_to_string(messages: SanityCheckMessages) -> str:
@@ -71,7 +71,7 @@ class TestRenderResultsWithIssues:
assert "INFO" in output assert "INFO" in output
assert "No OCR data" in output assert "No OCR data" in output
@pytest.mark.usefixtures("paperless_dirs") @pytest.mark.usefixtures("_media_settings")
def test_global_message(self) -> None: def test_global_message(self) -> None:
msgs = SanityCheckMessages() msgs = SanityCheckMessages()
msgs.warning(None, "Orphaned file: /tmp/stray.pdf") msgs.warning(None, "Orphaned file: /tmp/stray.pdf")
@@ -87,7 +87,7 @@ class TestRenderResultsWithIssues:
assert "Thumbnail missing" in output assert "Thumbnail missing" in output
assert "Checksum mismatch" in output assert "Checksum mismatch" in output
@pytest.mark.usefixtures("paperless_dirs") @pytest.mark.usefixtures("_media_settings")
def test_unknown_doc_pk(self) -> None: def test_unknown_doc_pk(self) -> None:
msgs = SanityCheckMessages() msgs = SanityCheckMessages()
msgs.error(99999, "Ghost document") msgs.error(99999, "Ghost document")
@@ -184,6 +184,7 @@ class TestDocumentSanityCheckerCommand:
assert "ERROR" in output assert "ERROR" in output
assert "Original of document does not exist" in output assert "Original of document does not exist" in output
@pytest.mark.usefixtures("_media_settings")
def test_checksum_mismatch(self, paperless_dirs: PaperlessDirs) -> None: def test_checksum_mismatch(self, paperless_dirs: PaperlessDirs) -> None:
"""Lightweight document with zero-byte files triggers checksum mismatch.""" """Lightweight document with zero-byte files triggers checksum mismatch."""
doc = DocumentFactory( doc = DocumentFactory(
+12 -4
View File
@@ -9,21 +9,29 @@ from documents.search._backend import TantivyBackend
from documents.search._backend import reset_backend from documents.search._backend import reset_backend
from documents.search._schema import build_schema from documents.search._schema import build_schema
from documents.search._tokenizer import register_tokenizers from documents.search._tokenizer import register_tokenizers
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
from collections.abc import Generator from collections.abc import Generator
from pathlib import Path
from pytest_django.fixtures import Settings from pytest_django.fixtures import Settings
from documents.models import Document from documents.models import Document
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture @pytest.fixture
def backend(paperless_dirs: PaperlessDirs) -> Generator[TantivyBackend, None, None]: def index_dir(tmp_path: Path, settings: Settings) -> Path:
b = TantivyBackend(path=paperless_dirs.index_dir) path = tmp_path / "index"
path.mkdir()
settings.INDEX_DIR = path
return path
@pytest.fixture
def backend() -> Generator[TantivyBackend, None, None]:
b = TantivyBackend() # path=None → in-memory index
b.open() b.open()
try: try:
yield b yield b
@@ -14,6 +14,7 @@ from typing import TYPE_CHECKING
import pytest import pytest
import time_machine import time_machine
from django.contrib.auth.models import User
from documents.models import CustomField from documents.models import CustomField
from documents.models import CustomFieldInstance from documents.models import CustomFieldInstance
@@ -21,8 +22,7 @@ from documents.models import DocumentType
from documents.models import Note from documents.models import Note
from documents.models import StoragePath from documents.models import StoragePath
from documents.search._query import parse_user_query from documents.search._query import parse_user_query
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import UserFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
@@ -148,7 +148,7 @@ class TestJsonSubpaths:
THEN: THEN:
- Only the document with alice's note matches - Only the document with alice's note matches
""" """
alice = UserFactory(username="alice") alice = User.objects.create_user(username="alice")
doc_with_note = DocumentFactory( doc_with_note = DocumentFactory(
title="Has note", title="Has note",
content="x", content="x",
+16 -17
View File
@@ -3,8 +3,10 @@ from pathlib import Path
import pytest import pytest
from django.contrib.auth.models import Group from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from django.db import connection from django.db import connection
from django.test.utils import CaptureQueriesContext from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from pytest_mock import MockerFixture from pytest_mock import MockerFixture
from documents.models import CustomField from documents.models import CustomField
@@ -17,12 +19,11 @@ from documents.search._backend import WriteBatch
from documents.search._backend import get_backend from documents.search._backend import get_backend
from documents.search._backend import reset_backend from documents.search._backend import reset_backend
from documents.signals.handlers import add_to_index from documents.signals.handlers import add_to_index
from paperless_testing.factories import CorrespondentFactory from documents.tests.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory from documents.tests.factories import DocumentTypeFactory
from paperless_testing.factories import TagFactory from documents.tests.factories import TagFactory
from paperless_testing.factories import UserFactory from documents.tests.factories import UserFactory
from paperless_testing.permissions import grant_object
pytestmark = [pytest.mark.search, pytest.mark.django_db] pytestmark = [pytest.mark.search, pytest.mark.django_db]
@@ -188,7 +189,7 @@ class TestAddOrUpdateIds:
pk=1, pk=1,
owner=owner, owner=owner,
) )
grant_object(user, doc, "view_document") assign_perm("view_document", user, doc)
with backend.batch_update() as batch: with backend.batch_update() as batch:
batch.add_or_update_ids([doc.pk]) batch.add_or_update_ids([doc.pk])
@@ -208,7 +209,7 @@ class TestAddOrUpdateIds:
pk=1, pk=1,
owner=owner, owner=owner,
) )
grant_object(group, doc, "view_document") assign_perm("view_document", group, doc)
with backend.batch_update() as batch: with backend.batch_update() as batch:
batch.add_or_update_ids([doc.pk]) batch.add_or_update_ids([doc.pk])
@@ -762,8 +763,8 @@ class TestSearchIds:
def test_respects_permission_filter(self, backend: TantivyBackend) -> None: def test_respects_permission_filter(self, backend: TantivyBackend) -> None:
"""search_ids must respect user permission filtering.""" """search_ids must respect user permission filtering."""
owner = UserFactory(username="ids_owner") owner = User.objects.create_user("ids_owner")
other = UserFactory(username="ids_other") other = User.objects.create_user("ids_other")
doc = Document.objects.create( doc = Document.objects.create(
title="private doc", title="private doc",
content="secret keyword", content="secret keyword",
@@ -842,7 +843,7 @@ class TestRebuild:
content="group secret keyword", content="group secret keyword",
owner=owner, owner=owner,
) )
grant_object(group, doc, "view_document") assign_perm("view_document", group, doc)
backend.rebuild(Document.objects.all()) backend.rebuild(Document.objects.all())
@@ -947,8 +948,7 @@ class TestSingleton:
yield yield
reset_backend() reset_backend()
@pytest.mark.usefixtures("paperless_dirs") def test_returns_same_instance_on_repeated_calls(self, index_dir) -> None:
def test_returns_same_instance_on_repeated_calls(self) -> None:
"""Singleton pattern: repeated calls to get_backend() must return the same instance.""" """Singleton pattern: repeated calls to get_backend() must return the same instance."""
assert get_backend() is get_backend() assert get_backend() is get_backend()
@@ -965,8 +965,7 @@ class TestSingleton:
assert b1 is not b2 assert b1 is not b2
assert b2._path == tmp_path / "b" assert b2._path == tmp_path / "b"
@pytest.mark.usefixtures("paperless_dirs") def test_reset_forces_new_instance(self, index_dir) -> None:
def test_reset_forces_new_instance(self) -> None:
"""reset_backend() must force creation of a new backend instance on next get_backend() call.""" """reset_backend() must force creation of a new backend instance on next get_backend() call."""
b1 = get_backend() b1 = get_backend()
reset_backend() reset_backend()
@@ -1072,7 +1071,7 @@ class TestFieldHandling:
def test_notes_include_user_information(self, backend: TantivyBackend) -> None: def test_notes_include_user_information(self, backend: TantivyBackend) -> None:
"""Notes must be indexed with user information when available for structured queries.""" """Notes must be indexed with user information when available for structured queries."""
user = UserFactory(username="notewriter") user = User.objects.create_user("notewriter")
doc = Document.objects.create( doc = Document.objects.create(
title="Doc with notes", title="Doc with notes",
content="test", content="test",
@@ -1174,7 +1173,7 @@ class TestHighlightHits:
notes.note: prefix so the query targets notes content directly, but notes.note: prefix so the query targets notes content directly, but
the snippet is generated from notes_text which stores the same text. the snippet is generated from notes_text which stores the same text.
""" """
user = UserFactory(username="hl_noteuser") user = User.objects.create_user("hl_noteuser")
doc = Document.objects.create( doc = Document.objects.create(
title="Doc with matching note", title="Doc with matching note",
content="unrelated content", content="unrelated content",
@@ -27,7 +27,7 @@ import time_machine
from documents.models import Note from documents.models import Note
from documents.models import Tag from documents.models import Tag
from documents.search._errors import InvalidDateQuery from documents.search._errors import InvalidDateQuery
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
@@ -269,7 +269,7 @@ class TestDocumentedDateForms:
yield yield
@pytest.fixture @pytest.fixture
def dated(self, backend: TantivyBackend) -> dict[str, int]: def dated(self, index_document: Callable[..., Document]) -> dict[str, int]:
stamps = { stamps = {
"today": datetime(2026, 6, 15, 9, 0, tzinfo=UTC), "today": datetime(2026, 6, 15, 9, 0, tzinfo=UTC),
"yesterday": datetime(2026, 6, 14, 9, 0, tzinfo=UTC), "yesterday": datetime(2026, 6, 14, 9, 0, tzinfo=UTC),
@@ -279,14 +279,14 @@ class TestDocumentedDateForms:
"january": datetime(2026, 1, 10, 10, 0, tzinfo=UTC), "january": datetime(2026, 1, 10, 10, 0, tzinfo=UTC),
"old": datetime(2005, 3, 4, 15, 30, tzinfo=UTC), "old": datetime(2005, 3, 4, 15, 30, tzinfo=UTC),
} }
docs = { return {
label: DocumentFactory(title=label, content="dated body", added=stamp) label: index_document(
title=label,
content="dated body",
added=stamp,
).pk
for label, stamp in stamps.items() for label, stamp in stamps.items()
} }
with backend.batch_update() as batch:
for doc in docs.values():
batch.add_or_update(doc)
return {label: doc.pk for label, doc in docs.items()}
@pytest.mark.parametrize( @pytest.mark.parametrize(
("query", "label"), ("query", "label"),
@@ -29,7 +29,7 @@ from rest_framework import status
from documents.search._backend import SearchMode from documents.search._backend import SearchMode
from documents.search._query import parse_simple_text_highlight_query from documents.search._query import parse_simple_text_highlight_query
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from rest_framework.test import APIClient from rest_framework.test import APIClient
@@ -17,12 +17,12 @@ from __future__ import annotations
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
import pytest import pytest
from django.contrib.auth.models import User
from documents.models import CustomField from documents.models import CustomField
from documents.models import CustomFieldInstance from documents.models import CustomFieldInstance
from documents.models import Note from documents.models import Note
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import UserFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
@@ -53,7 +53,7 @@ class TestBareJsonFieldPrefixes:
decoy's content match does not resurface through a demoted decoy's content match does not resurface through a demoted
text search text search
""" """
alice = UserFactory(username="alice") alice = User.objects.create_user(username="alice")
with_note = DocumentFactory(title="Has note", content="x") with_note = DocumentFactory(title="Has note", content="x")
Note.objects.create(document=with_note, user=alice, note="crocodile") Note.objects.create(document=with_note, user=alice, note="crocodile")
backend.add_or_update(with_note) backend.add_or_update(with_note)
@@ -116,7 +116,7 @@ class TestBareJsonFieldPrefixes:
document; the default-subpath resolution for the bare document; the default-subpath resolution for the bare
prefix does not interfere with explicit subpath addressing prefix does not interfere with explicit subpath addressing
""" """
bob = UserFactory(username="bob") bob = User.objects.create_user(username="bob")
doc = DocumentFactory(title="Bob note", content="x") doc = DocumentFactory(title="Bob note", content="x")
Note.objects.create(document=doc, user=bob, note="remark") Note.objects.create(document=doc, user=bob, note="remark")
backend.add_or_update(doc) backend.add_or_update(doc)
@@ -20,6 +20,7 @@ from typing import TYPE_CHECKING
import pytest import pytest
import tantivy import tantivy
from django.contrib.auth.models import User
from whoosh_compat import FieldKind from whoosh_compat import FieldKind
from documents.models import CustomField from documents.models import CustomField
@@ -27,7 +28,6 @@ from documents.models import CustomFieldInstance
from documents.models import Document from documents.models import Document
from documents.models import Note from documents.models import Note
from documents.search._fields import PUBLIC_FIELDS from documents.search._fields import PUBLIC_FIELDS
from paperless_testing.factories import UserFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from documents.search._backend import TantivyBackend from documents.search._backend import TantivyBackend
@@ -49,7 +49,7 @@ class TestJsonSubpathsAreWrittenAtIndexTime:
- Every subpath PUBLIC_FIELDS declares for notes/custom_fields - Every subpath PUBLIC_FIELDS declares for notes/custom_fields
is present as a key in the document's stored JSON payload is present as a key in the document's stored JSON payload
""" """
user = UserFactory(username="completeness-user") user = User.objects.create_user(username="completeness-user")
field = CustomField.objects.create( field = CustomField.objects.create(
name="Completeness Field", name="Completeness Field",
data_type=CustomField.FieldDataType.STRING, data_type=CustomField.FieldDataType.STRING,
@@ -15,7 +15,7 @@ from documents.search._backend import SearchIndexLockError
from documents.search._backend import TantivyBackend from documents.search._backend import TantivyBackend
from documents.tasks import index_document from documents.tasks import index_document
from documents.tasks import remove_document_from_index from documents.tasks import remove_document_from_index
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Generator from collections.abc import Generator
@@ -1,6 +1,6 @@
import pytest import pytest
from paperless_testing.migrations import TestMigrations from documents.tests.utils import TestMigrations
pytestmark = pytest.mark.search pytestmark = pytest.mark.search
@@ -18,14 +18,13 @@ from typing import TYPE_CHECKING
import pytest import pytest
from django.contrib.auth.models import Group from django.contrib.auth.models import Group
from django.contrib.auth.models import User from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from documents.models import Correspondent from documents.models import Correspondent
from documents.models import Document from documents.models import Document
from documents.models import DocumentType from documents.models import DocumentType
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_object
if TYPE_CHECKING: if TYPE_CHECKING:
from documents.search._backend import TantivyBackend from documents.search._backend import TantivyBackend
@@ -35,22 +34,22 @@ pytestmark = [pytest.mark.search, pytest.mark.django_db]
@pytest.fixture @pytest.fixture
def owner() -> User: def owner() -> User:
return UserFactory(username="owner") return User.objects.create_user(username="owner")
@pytest.fixture @pytest.fixture
def stranger() -> User: def stranger() -> User:
return UserFactory(username="stranger") return User.objects.create_user(username="stranger")
@pytest.fixture @pytest.fixture
def viewer() -> User: def viewer() -> User:
return UserFactory(username="viewer") return User.objects.create_user(username="viewer")
@pytest.fixture @pytest.fixture
def group_member() -> User: def group_member() -> User:
user = UserFactory(username="group_member") user = User.objects.create_user(username="group_member")
user.groups.add(Group.objects.create(name="accounting")) user.groups.add(Group.objects.create(name="accounting"))
return user return user
@@ -128,7 +127,7 @@ class TestPermissionFilteringOnIndexedDocuments:
checksum="perm-shared-user", checksum="perm-shared-user",
owner=owner, owner=owner,
) )
grant_object(viewer, doc, "view_document") assign_perm("view_document", viewer, doc)
backend.add_or_update(doc) backend.add_or_update(doc)
assert backend.search_ids("invoice", user=viewer) == [doc.pk] assert backend.search_ids("invoice", user=viewer) == [doc.pk]
@@ -158,7 +157,7 @@ class TestPermissionFilteringOnIndexedDocuments:
checksum="perm-shared-group", checksum="perm-shared-group",
owner=owner, owner=owner,
) )
grant_object(group_member.groups.first(), doc, "view_document") assign_perm("view_document", group_member.groups.first(), doc)
backend.add_or_update(doc) backend.add_or_update(doc)
assert backend.search_ids("invoice", user=group_member) == [doc.pk] assert backend.search_ids("invoice", user=group_member) == [doc.pk]
+19 -22
View File
@@ -13,11 +13,11 @@ from documents.search._schema import needs_rebuild
from documents.search._schema import schema_fingerprint from documents.search._schema import schema_fingerprint
if TYPE_CHECKING: if TYPE_CHECKING:
from pathlib import Path
import tantivy import tantivy
from pytest_django.fixtures import Settings from pytest_django.fixtures import Settings
from paperless_testing.dirs import PaperlessDirs
pytestmark = pytest.mark.search pytestmark = pytest.mark.search
@@ -25,19 +25,16 @@ pytestmark = pytest.mark.search
class TestNeedsRebuild: class TestNeedsRebuild:
"""needs_rebuild covers all sentinel-file states that require a full reindex.""" """needs_rebuild covers all sentinel-file states that require a full reindex."""
def test_returns_true_when_settings_file_missing( def test_returns_true_when_settings_file_missing(self, index_dir: Path) -> None:
self, assert needs_rebuild(index_dir) is True
paperless_dirs: PaperlessDirs,
) -> None:
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_false_when_version_and_language_match( def test_returns_false_when_version_and_language_match(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: Settings, settings: Settings,
) -> None: ) -> None:
settings.SEARCH_LANGUAGE = "en" settings.SEARCH_LANGUAGE = "en"
(paperless_dirs.index_dir / ".index_settings.json").write_text( (index_dir / ".index_settings.json").write_text(
json.dumps( json.dumps(
{ {
"schema_version": SCHEMA_VERSION, "schema_version": SCHEMA_VERSION,
@@ -46,51 +43,51 @@ class TestNeedsRebuild:
}, },
), ),
) )
assert needs_rebuild(paperless_dirs.index_dir) is False assert needs_rebuild(index_dir) is False
def test_returns_true_on_schema_version_mismatch( def test_returns_true_on_schema_version_mismatch(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: Settings, settings: Settings,
) -> None: ) -> None:
settings.SEARCH_LANGUAGE = None settings.SEARCH_LANGUAGE = None
(paperless_dirs.index_dir / ".index_settings.json").write_text( (index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION - 1, "language": None}), json.dumps({"schema_version": SCHEMA_VERSION - 1, "language": None}),
) )
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def test_returns_true_when_version_is_not_an_integer( def test_returns_true_when_version_is_not_an_integer(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: Settings, settings: Settings,
) -> None: ) -> None:
settings.SEARCH_LANGUAGE = None settings.SEARCH_LANGUAGE = None
(paperless_dirs.index_dir / ".index_settings.json").write_text( (index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": "not-a-number", "language": None}), json.dumps({"schema_version": "not-a-number", "language": None}),
) )
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def test_returns_true_when_language_key_missing( def test_returns_true_when_language_key_missing(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: Settings, settings: Settings,
) -> None: ) -> None:
settings.SEARCH_LANGUAGE = "en" settings.SEARCH_LANGUAGE = "en"
(paperless_dirs.index_dir / ".index_settings.json").write_text( (index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION}), json.dumps({"schema_version": SCHEMA_VERSION}),
) )
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def test_returns_true_when_language_differs( def test_returns_true_when_language_differs(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: Settings, settings: Settings,
) -> None: ) -> None:
settings.SEARCH_LANGUAGE = "de" settings.SEARCH_LANGUAGE = "de"
(paperless_dirs.index_dir / ".index_settings.json").write_text( (index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION, "language": "en"}), json.dumps({"schema_version": SCHEMA_VERSION, "language": "en"}),
) )
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def _schema_fields(schema: tantivy.Schema) -> dict[str, dict]: def _schema_fields(schema: tantivy.Schema) -> dict[str, dict]:
@@ -35,8 +35,6 @@ if TYPE_CHECKING:
from pytest_django.fixtures import SettingsWrapper from pytest_django.fixtures import SettingsWrapper
from paperless_testing.dirs import PaperlessDirs
pytestmark = pytest.mark.search pytestmark = pytest.mark.search
# The on-disk field layout of a v2 index, pinned as data. Any edit here is an # The on-disk field layout of a v2 index, pinned as data. Any edit here is an
@@ -471,7 +469,7 @@ def _fingerprint_of(descriptors: list[FieldDescriptor]) -> str:
class TestNeedsRebuildOnFingerprint: class TestNeedsRebuildOnFingerprint:
def test_matching_fingerprint_does_not_rebuild( def test_matching_fingerprint_does_not_rebuild(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: SettingsWrapper, settings: SettingsWrapper,
) -> None: ) -> None:
""" """
@@ -484,13 +482,13 @@ class TestNeedsRebuildOnFingerprint:
- It returns False - It returns False
""" """
settings.SEARCH_LANGUAGE = None settings.SEARCH_LANGUAGE = None
_sentinels(paperless_dirs.index_dir) _sentinels(index_dir)
assert needs_rebuild(paperless_dirs.index_dir) is False assert needs_rebuild(index_dir) is False
def test_stale_fingerprint_rebuilds_despite_a_matching_version( def test_stale_fingerprint_rebuilds_despite_a_matching_version(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: SettingsWrapper, settings: SettingsWrapper,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
@@ -507,7 +505,7 @@ class TestNeedsRebuildOnFingerprint:
every subsequent write would raise every subsequent write would raise
""" """
settings.SEARCH_LANGUAGE = None settings.SEARCH_LANGUAGE = None
_sentinels(paperless_dirs.index_dir) _sentinels(index_dir)
extended = [ extended = [
*field_descriptors(), *field_descriptors(),
FieldDescriptor( FieldDescriptor(
@@ -521,11 +519,11 @@ class TestNeedsRebuildOnFingerprint:
] ]
monkeypatch.setattr(_schema, "field_descriptors", lambda: extended) monkeypatch.setattr(_schema, "field_descriptors", lambda: extended)
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def test_reordered_schema_rebuilds( def test_reordered_schema_rebuilds(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: SettingsWrapper, settings: SettingsWrapper,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
@@ -540,16 +538,16 @@ class TestNeedsRebuildOnFingerprint:
- It returns True - It returns True
""" """
settings.SEARCH_LANGUAGE = None settings.SEARCH_LANGUAGE = None
_sentinels(paperless_dirs.index_dir) _sentinels(index_dir)
reordered = field_descriptors() reordered = field_descriptors()
reordered[1], reordered[2] = reordered[2], reordered[1] reordered[1], reordered[2] = reordered[2], reordered[1]
monkeypatch.setattr(_schema, "field_descriptors", lambda: reordered) monkeypatch.setattr(_schema, "field_descriptors", lambda: reordered)
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def test_missing_fingerprint_rebuilds( def test_missing_fingerprint_rebuilds(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: SettingsWrapper, settings: SettingsWrapper,
) -> None: ) -> None:
""" """
@@ -563,15 +561,15 @@ class TestNeedsRebuildOnFingerprint:
is rebuilt rather than trusted is rebuilt rather than trusted
""" """
settings.SEARCH_LANGUAGE = None settings.SEARCH_LANGUAGE = None
(paperless_dirs.index_dir / ".index_settings.json").write_text( (index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION, "language": None}), json.dumps({"schema_version": SCHEMA_VERSION, "language": None}),
) )
assert needs_rebuild(paperless_dirs.index_dir) is True assert needs_rebuild(index_dir) is True
def test_written_sentinels_satisfy_the_check( def test_written_sentinels_satisfy_the_check(
self, self,
paperless_dirs: PaperlessDirs, index_dir: Path,
settings: SettingsWrapper, settings: SettingsWrapper,
) -> None: ) -> None:
""" """
@@ -584,6 +582,6 @@ class TestNeedsRebuildOnFingerprint:
- It returns False - It returns False
""" """
settings.SEARCH_LANGUAGE = "en" settings.SEARCH_LANGUAGE = "en"
_write_sentinels(paperless_dirs.index_dir) _write_sentinels(index_dir)
assert needs_rebuild(paperless_dirs.index_dir) is False assert needs_rebuild(index_dir) is False
+4 -4
View File
@@ -16,11 +16,11 @@ from documents.models import Document
from documents.models import Tag from documents.models import Tag
from documents.search import get_backend from documents.search import get_backend
from documents.search import reset_backend from documents.search import reset_backend
from documents.tests.factories import DocumentFactory
from documents.tests.factories import TagFactory
from documents.tests.factories import UserFactory
from documents.tests.utils import DirectoriesMixin
from paperless.admin import PaperlessUserAdmin from paperless.admin import PaperlessUserAdmin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
@pytest.fixture @pytest.fixture
+5 -49
View File
@@ -3,6 +3,7 @@ from io import BytesIO
from pathlib import Path from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
from django.contrib.auth.models import User
from django.core.files.uploadedfile import SimpleUploadedFile from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import override_settings from django.test import override_settings
from PIL import Image from PIL import Image
@@ -10,11 +11,10 @@ from PIL.PngImagePlugin import PngInfo
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import read_streaming_response
from paperless.models import ApplicationConfiguration from paperless.models import ApplicationConfiguration
from paperless.models import ColorConvertChoices from paperless.models import ColorConvertChoices
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
class TestApiAppConfig(DirectoriesMixin, APITestCase): class TestApiAppConfig(DirectoriesMixin, APITestCase):
@@ -23,7 +23,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
user = UserFactory(username="temp_admin", superuser=True) user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
def test_api_get_config(self) -> None: def test_api_get_config(self) -> None:
@@ -81,7 +81,6 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
"ai_enabled": None, "ai_enabled": None,
"llm_embedding_backend": None, "llm_embedding_backend": None,
"llm_embedding_model": None, "llm_embedding_model": None,
"llm_embedding_api_key": None,
"llm_embedding_endpoint": None, "llm_embedding_endpoint": None,
"llm_embedding_chunk_size": None, "llm_embedding_chunk_size": None,
"llm_context_size": None, "llm_context_size": None,
@@ -268,7 +267,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
THEN: THEN:
- old app_logo file is deleted - old app_logo file is deleted
""" """
admin = UserFactory(username="admin", superuser=True) admin = User.objects.create_superuser(username="admin")
self.client.force_login(user=admin) self.client.force_login(user=admin)
response = self.client.get("/logo/") response = self.client.get("/logo/")
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND) self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
@@ -923,49 +922,6 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED) self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED)
self.assertEqual(ApplicationConfiguration.objects.count(), 1) self.assertEqual(ApplicationConfiguration.objects.count(), 1)
def test_update_llm_embedding_api_key(self) -> None:
"""
GIVEN:
- Existing config with llm_embedding_api_key specified
WHEN:
- API to update llm_embedding_api_key is called with all *s
- API to update llm_embedding_api_key is called with empty string
THEN:
- llm_embedding_api_key is unchanged
- llm_embedding_api_key is set to None
"""
config = ApplicationConfiguration.objects.first()
assert config is not None
config.llm_embedding_api_key = "1234567890"
config.save()
# Test with all *
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"llm_embedding_api_key": "*" * 32,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.llm_embedding_api_key, "1234567890")
# Test with empty string
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"llm_embedding_api_key": "",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.llm_embedding_api_key, None)
def test_update_llm_api_key(self) -> None: def test_update_llm_api_key(self) -> None:
""" """
GIVEN: GIVEN:
+10 -16
View File
@@ -4,6 +4,8 @@ import json
import shutil import shutil
import zipfile import zipfile
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from django.utils import timezone from django.utils import timezone
from rest_framework import status from rest_framework import status
@@ -12,11 +14,9 @@ from rest_framework.test import APITestCase
from documents.models import Correspondent from documents.models import Correspondent
from documents.models import Document from documents.models import Document
from documents.models import DocumentType from documents.models import DocumentType
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import SampleDirMixin from documents.tests.utils import SampleDirMixin
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import read_streaming_response
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase): class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
@@ -25,7 +25,7 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
self.doc1 = Document.objects.create(title="unrelated", checksum="A") self.doc1 = Document.objects.create(title="unrelated", checksum="A")
@@ -166,15 +166,7 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
), ),
content_type="application/json", content_type="application/json",
) )
response.close()
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response["Content-Type"], "application/zip")
with zipfile.ZipFile(io.BytesIO(read_streaming_response(response))) as zipf:
self.assertEqual(zipf.infolist()[0].compress_type, zipfile.ZIP_LZMA)
with self.doc2.source_file as f:
self.assertEqual(f.read(), zipf.read("2021-01-01 document A.pdf"))
@override_settings(FILENAME_FORMAT="{correspondent}/{title}") @override_settings(FILENAME_FORMAT="{correspondent}/{title}")
def test_formatted_download_originals(self) -> None: def test_formatted_download_originals(self) -> None:
@@ -334,8 +326,10 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
) )
def test_download_insufficient_permissions(self) -> None: def test_download_insufficient_permissions(self) -> None:
user = UserFactory(username="temp_user") user = User.objects.create_user(username="temp_user")
grant_global(user, "view_document") user.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
self.doc2.owner = self.user self.doc2.owner = self.user
+43 -90
View File
@@ -2,30 +2,27 @@ import json
from unittest import mock from unittest import mock
from auditlog.models import LogEntry from auditlog.models import LogEntry
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from guardian.shortcuts import assign_perm
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.models import Correspondent from documents.models import Correspondent
from documents.models import CustomField from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document from documents.models import Document
from documents.models import DocumentType from documents.models import DocumentType
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestBulkEditAPI(DirectoriesMixin, APITestCase): class TestBulkEditAPI(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
user = UserFactory(username="temp_admin", superuser=True) user = User.objects.create_superuser(username="temp_admin")
self.user = user self.user = user
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
@@ -287,9 +284,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m, m,
) -> None: ) -> None:
self.setup_mock(m, "modify_custom_fields") self.setup_mock(m, "modify_custom_fields")
user = UserFactory(username="doc-owner") user = User.objects.create_user(username="doc-owner")
grant_global(user, "change_document") user.user_permissions.add(Permission.objects.get(codename="change_document"))
other_user = UserFactory(username="other-user") other_user = User.objects.create_user(username="other-user")
source_doc = Document.objects.create( source_doc = Document.objects.create(
checksum="source", checksum="source",
title="Source", title="Source",
@@ -790,8 +787,10 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
@mock.patch("documents.serialisers.bulk_edit.set_storage_path") @mock.patch("documents.serialisers.bulk_edit.set_storage_path")
def test_api_bulk_edit_with_all_true_resolves_owned_duplicates(self, m) -> None: def test_api_bulk_edit_with_all_true_resolves_owned_duplicates(self, m) -> None:
self.setup_mock(m, "set_storage_path") self.setup_mock(m, "set_storage_path")
user = UserFactory(username="duplicate-owner") user = User.objects.create_user(username="duplicate-owner")
grant_global(user, "change_document") user.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
first_duplicate = Document.objects.create( first_duplicate = Document.objects.create(
checksum="owned-duplicate", checksum="owned-duplicate",
title="First duplicate", title="First duplicate",
@@ -1179,7 +1178,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
grant_object(user1, self.doc2, "view_document") assign_perm("view_document", user1, self.doc2)
response = self.client.post( response = self.client.post(
"/api/documents/selection_data/", "/api/documents/selection_data/",
@@ -1189,7 +1188,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
grant_global(user1, "view_document") user1.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
user1 = User.objects.get(pk=user1.pk) user1 = User.objects.get(pk=user1.pk)
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
response = self.client.post( response = self.client.post(
@@ -1532,7 +1533,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin") self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save() self.doc1.save()
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1586,8 +1587,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin") self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save() self.doc1.save()
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
grant_object(user1, self.doc1, "view_document") assign_perm("view_document", user1, self.doc1)
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1608,7 +1609,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m.assert_not_called() m.assert_not_called()
self.assertEqual(response.content, b"Insufficient permissions") self.assertEqual(response.content, b"Insufficient permissions")
grant_object(user1, self.doc1, "change_document") assign_perm("change_document", user1, self.doc1)
response = self.client.post( response = self.client.post(
"/api/documents/bulk_edit/", "/api/documents/bulk_edit/",
@@ -1785,36 +1786,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertIn(b"invalid pages specified", response.content) self.assertIn(b"invalid pages specified", response.content)
m.assert_not_called() m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.split")
def test_bulk_edit_split_rejects_unknown_page_count(self, m) -> None:
"""
GIVEN:
- A legacy split bulk edit of a document without a page count
WHEN:
- API to bulk edit is called
THEN:
- API returns HTTP 400
- split is not called
"""
self.setup_mock(m, "split")
for pages in ("1", "1-5000000"):
with self.subTest(pages=pages):
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc1.id],
"method": "split",
"parameters": {"pages": pages},
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn(b"document page count is unknown", response.content)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.split") @mock.patch("documents.serialisers.bulk_edit.split")
def test_bulk_edit_split_parses_pages(self, m) -> None: def test_bulk_edit_split_parses_pages(self, m) -> None:
""" """
@@ -1848,7 +1819,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin") self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save() self.doc1.save()
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1909,7 +1880,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin") self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save() self.doc1.save()
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1948,8 +1919,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
@mock.patch("documents.views.bulk_edit.merge") @mock.patch("documents.views.bulk_edit.merge")
def test_merge_and_delete_requires_change_permission(self, m) -> None: def test_merge_and_delete_requires_change_permission(self, m) -> None:
self.setup_mock(m, "merge") self.setup_mock(m, "merge")
user = UserFactory(username="no-change") user = User.objects.create_user(username="no-change")
grant_global(user, "add_document", "delete_document") user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="delete_document"),
)
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
response = self.client.post( response = self.client.post(
@@ -2336,7 +2310,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin") self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save() self.doc1.save()
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -2371,7 +2345,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
@mock.patch("documents.views.bulk_edit.edit_pdf") @mock.patch("documents.views.bulk_edit.edit_pdf")
def test_edit_pdf_update_requires_change_permission(self, m) -> None: def test_edit_pdf_update_requires_change_permission(self, m) -> None:
self.setup_mock(m, "edit_pdf") self.setup_mock(m, "edit_pdf")
user = UserFactory(username="no-change") user = User.objects.create_user(username="no-change")
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
response = self.client.post( response = self.client.post(
@@ -2398,8 +2372,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
) -> None: ) -> None:
self.setup_mock(edit_pdf_mock, "edit_pdf") self.setup_mock(edit_pdf_mock, "edit_pdf")
self.setup_mock(remove_password_mock, "remove_password") self.setup_mock(remove_password_mock, "remove_password")
user = UserFactory(username="no-delete") user = User.objects.create_user(username="no-delete")
grant_global(user, "add_document", "change_document") user.user_permissions.add(
Permission.objects.get(codename="add_document"),
Permission.objects.get(codename="change_document"),
)
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
cases = [ cases = [
@@ -2486,7 +2463,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.owner = User.objects.get(username="temp_admin") self.doc1.owner = User.objects.get(username="temp_admin")
self.doc1.save() self.doc1.save()
user1 = User.objects.create(username="user1") user1 = User.objects.create(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -2526,7 +2503,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
WHEN: WHEN:
- API to bulk edit documents is called - API to bulk edit documents is called
THEN: THEN:
- Audit log is created with the old and new correspondent - Audit log is created
""" """
LogEntry.objects.all().delete() LogEntry.objects.all().delete()
response = self.client.post( response = self.client.post(
@@ -2542,8 +2519,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
) )
self.assertEqual(response.status_code, status.HTTP_200_OK) self.assertEqual(response.status_code, status.HTTP_200_OK)
entry = LogEntry.objects.get_for_object(self.doc1).get() self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 1)
self.assertEqual(entry.changes, {"correspondent": [None, self.c2.id]})
@override_settings(AUDIT_LOG_ENABLED=True) @override_settings(AUDIT_LOG_ENABLED=True)
def test_bulk_edit_audit_log_enabled_tags(self) -> None: def test_bulk_edit_audit_log_enabled_tags(self) -> None:
@@ -2551,18 +2527,16 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
GIVEN: GIVEN:
- Audit log is enabled - Audit log is enabled
WHEN: WHEN:
- API to bulk edit tags is called on an untagged document and a - API to bulk edit tags is called
document with several tags
THEN: THEN:
- Audit log is created for each document with its full tag list - Audit log is created
before and after the edit
""" """
LogEntry.objects.all().delete() LogEntry.objects.all().delete()
response = self.client.post( response = self.client.post(
"/api/documents/bulk_edit/", "/api/documents/bulk_edit/",
json.dumps( json.dumps(
{ {
"documents": [self.doc1.id, self.doc4.id], "documents": [self.doc1.id],
"method": "modify_tags", "method": "modify_tags",
"parameters": { "parameters": {
"add_tags": [self.t1.id], "add_tags": [self.t1.id],
@@ -2574,32 +2548,18 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
) )
self.assertEqual(response.status_code, status.HTTP_200_OK) self.assertEqual(response.status_code, status.HTTP_200_OK)
entry = LogEntry.objects.get_for_object(self.doc1).get() self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 1)
self.assertEqual(entry.changes, {"tags": [[], [self.t1.id]]})
entry = LogEntry.objects.get_for_object(self.doc4).get()
self.assertEqual(
entry.changes,
{"tags": [[self.t1.id, self.t2.id], [self.t1.id]]},
)
@override_settings(AUDIT_LOG_ENABLED=True) @override_settings(AUDIT_LOG_ENABLED=True)
def test_bulk_edit_audit_log_enabled_custom_fields(self) -> None: def test_bulk_edit_audit_log_enabled_custom_fields(self) -> None:
""" """
GIVEN: GIVEN:
- Audit log is enabled - Audit log is enabled
- A document with two custom fields
WHEN: WHEN:
- API to bulk edit custom fields is called to add a third - API to bulk edit custom fields is called
THEN: THEN:
- Audit log is created with every custom field instance before and - Audit log is created
after the edit
- Audit log is created for the new custom field instance
""" """
cf3 = CustomField.objects.create(name="cf3", data_type="string")
existing = [
CustomFieldInstance.objects.create(document=self.doc1, field=field)
for field in (self.cf2, cf3)
]
LogEntry.objects.all().delete() LogEntry.objects.all().delete()
response = self.client.post( response = self.client.post(
"/api/documents/bulk_edit/", "/api/documents/bulk_edit/",
@@ -2617,14 +2577,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
) )
self.assertEqual(response.status_code, status.HTTP_200_OK) self.assertEqual(response.status_code, status.HTTP_200_OK)
added = CustomFieldInstance.objects.get(document=self.doc1, field=self.cf1) self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 2)
existing_ids = [instance.id for instance in existing]
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(
entry.changes,
{"custom_fields": [existing_ids, [*existing_ids, added.id]]},
)
self.assertEqual(LogEntry.objects.get_for_object(added).count(), 1)
def test_api_bulk_edit_with_bad_search_query_returns_400(self) -> None: def test_api_bulk_edit_with_bad_search_query_returns_400(self) -> None:
""" """
+6 -6
View File
@@ -4,22 +4,20 @@ from typing import TYPE_CHECKING
from unittest import mock from unittest import mock
import pytest import pytest
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from rest_framework import status from rest_framework import status
from rest_framework.test import APIClient from rest_framework.test import APIClient
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
if TYPE_CHECKING: if TYPE_CHECKING:
from django.contrib.auth.models import User
from pytest_mock import MockerFixture from pytest_mock import MockerFixture
class TestChatStreamingViewInputValidation(APITestCase): class TestChatStreamingViewInputValidation(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def _mock_ai_enabled(self) -> mock.MagicMock: def _mock_ai_enabled(self) -> mock.MagicMock:
@@ -115,7 +113,9 @@ class TestChatStreamingViewUnrestrictedFlag:
needs to reach the view at all. Model-level only: says nothing needs to reach the view at all. Model-level only: says nothing
about which documents (if any) this user can actually see. about which documents (if any) this user can actually see.
""" """
grant_global(regular_user, "view_document") regular_user.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
return user_client return user_client
@pytest.mark.parametrize( @pytest.mark.parametrize(
+16 -13
View File
@@ -4,24 +4,23 @@ from unittest import mock
from unittest.mock import ANY from unittest.mock import ANY
from django.contrib.auth.models import Permission from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from guardian.shortcuts import assign_perm
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.models import CustomField from documents.models import CustomField
from documents.models import CustomFieldInstance from documents.models import CustomFieldInstance
from documents.models import Document from documents.models import Document
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestCustomFieldsAPI(DirectoriesMixin, APITestCase): class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
ENDPOINT = "/api/custom_fields/" ENDPOINT = "/api/custom_fields/"
def setUp(self) -> None: def setUp(self) -> None:
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
return super().setUp() return super().setUp()
@@ -1175,9 +1174,11 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
def test_documentlink_patch_requires_change_permission_on_target_documents( def test_documentlink_patch_requires_change_permission_on_target_documents(
self, self,
) -> None: ) -> None:
source_owner = UserFactory(username="source-owner") source_owner = User.objects.create_user(username="source-owner")
grant_global(source_owner, "change_document") source_owner.user_permissions.add(
other_user = UserFactory(username="other-user") Permission.objects.get(codename="change_document"),
)
other_user = User.objects.create_user(username="other-user")
source_doc = Document.objects.create( source_doc = Document.objects.create(
title="Source", title="Source",
@@ -1220,9 +1221,11 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
def test_documentlink_patch_allowed_with_change_permission_on_target_documents( def test_documentlink_patch_allowed_with_change_permission_on_target_documents(
self, self,
) -> None: ) -> None:
source_owner = UserFactory(username="source-owner") source_owner = User.objects.create_user(username="source-owner")
grant_global(source_owner, "change_document") source_owner.user_permissions.add(
other_user = UserFactory(username="other-user") Permission.objects.get(codename="change_document"),
)
other_user = User.objects.create_user(username="other-user")
source_doc = Document.objects.create( source_doc = Document.objects.create(
title="Source", title="Source",
@@ -1241,7 +1244,7 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
data_type=CustomField.FieldDataType.DOCUMENTLINK, data_type=CustomField.FieldDataType.DOCUMENTLINK,
) )
grant_object(source_owner, target_doc, "change_document") assign_perm("change_document", source_owner, target_doc)
self.client.force_authenticate(user=source_owner) self.client.force_authenticate(user=source_owner)
resp = self.client.patch( resp = self.client.patch(
@@ -1334,7 +1337,7 @@ class TestCustomFieldsAPI(DirectoriesMixin, APITestCase):
self.assertEqual(results[0]["document_count"], 1) self.assertEqual(results[0]["document_count"], 1)
# Test as user without access to the document # Test as user without access to the document
non_superuser = UserFactory(username="non_superuser") non_superuser = User.objects.create_user(username="non_superuser")
non_superuser.user_permissions.add( non_superuser.user_permissions.add(
*Permission.objects.exclude(codename="view_global_statistics"), *Permission.objects.exclude(codename="view_global_statistics"),
) )
@@ -5,6 +5,8 @@ from typing import TYPE_CHECKING
from unittest import mock from unittest import mock
from auditlog.models import LogEntry # type: ignore[import-untyped] from auditlog.models import LogEntry # type: ignore[import-untyped]
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from django.core.files.uploadedfile import SimpleUploadedFile from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase as DjangoTestCase from django.test import TestCase as DjangoTestCase
@@ -16,12 +18,10 @@ from documents.data_models import DocumentSource
from documents.filters import EffectiveContentFilter from documents.filters import EffectiveContentFilter
from documents.filters import TitleContentFilter from documents.filters import TitleContentFilter
from documents.models import Document from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import read_streaming_response
from documents.versioning import annotate_effective_content from documents.versioning import annotate_effective_content
from documents.views import DocumentSelectionMixin from documents.views import DocumentSelectionMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
if TYPE_CHECKING: if TYPE_CHECKING:
from pathlib import Path from pathlib import Path
@@ -31,7 +31,7 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def _make_pdf_upload(self, name: str = "version.pdf") -> SimpleUploadedFile: def _make_pdf_upload(self, name: str = "version.pdf") -> SimpleUploadedFile:
@@ -89,9 +89,11 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND) self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND)
def test_root_endpoint_returns_403_when_user_lacks_permission(self) -> None: def test_root_endpoint_returns_403_when_user_lacks_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
viewer = UserFactory(username="viewer") viewer = User.objects.create_user(username="viewer")
grant_global(viewer, "view_document") viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
root = Document.objects.create( root = Document.objects.create(
title="root", title="root",
checksum="root", checksum="root",
@@ -281,9 +283,11 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(mock_backend.add_or_update.call_args[0][0].id, root.id) self.assertEqual(mock_backend.add_or_update.call_args[0][0].id, root.id)
def test_delete_version_returns_403_without_permission(self) -> None: def test_delete_version_returns_403_without_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
other = UserFactory(username="other") other = User.objects.create_user(username="other")
grant_global(other, "delete_document") other.user_permissions.add(
Permission.objects.get(codename="delete_document"),
)
root = Document.objects.create( root = Document.objects.create(
title="root", title="root",
checksum="root", checksum="root",
@@ -367,9 +371,11 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertTrue(resp.data["is_root"]) self.assertTrue(resp.data["is_root"])
def test_update_version_label_returns_403_without_permission(self) -> None: def test_update_version_label_returns_403_without_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
other = UserFactory(username="other") other = User.objects.create_user(username="other")
grant_global(other, "change_document") other.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
root = Document.objects.create( root = Document.objects.create(
title="root", title="root",
checksum="root", checksum="root",
@@ -547,9 +553,11 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND) self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND)
def test_metadata_returns_403_when_user_lacks_permission(self) -> None: def test_metadata_returns_403_when_user_lacks_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
other = UserFactory(username="other") other = User.objects.create_user(username="other")
grant_global(other, "view_document") other.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
doc = Document.objects.create( doc = Document.objects.create(
title="root", title="root",
checksum="root", checksum="root",
@@ -645,8 +653,8 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_500_INTERNAL_SERVER_ERROR) self.assertEqual(resp.status_code, status.HTTP_500_INTERNAL_SERVER_ERROR)
def test_update_version_returns_403_without_permission(self) -> None: def test_update_version_returns_403_without_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
other = UserFactory(username="other") other = User.objects.create_user(username="other")
root = Document.objects.create( root = Document.objects.create(
title="root", title="root",
checksum="root", checksum="root",
@@ -664,8 +672,8 @@ class TestDocumentVersioningApi(DirectoriesMixin, APITestCase):
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
def test_update_version_requires_global_change_permission(self) -> None: def test_update_version_requires_global_change_permission(self) -> None:
user = UserFactory(username="add-only") user = User.objects.create_user(username="add-only")
grant_global(user, "add_document") user.user_permissions.add(Permission.objects.get(codename="add_document"))
root = Document.objects.create( root = Document.objects.create(
title="root", title="root",
checksum="root", checksum="root",
@@ -970,7 +978,7 @@ class TestVersionAwareFilters(DjangoTestCase):
superseded content -- selecting documents the list view, filtered by superseded content -- selecting documents the list view, filtered by
the same term, does not show. the same term, does not show.
""" """
user = UserFactory(username="bulk_selection", superuser=True) user = User.objects.create_superuser(username="bulk_selection")
selected = DocumentSelectionMixin()._resolve_document_ids( selected = DocumentSelectionMixin()._resolve_document_ids(
user=user, user=user,
@@ -997,7 +1005,7 @@ class TestBulkSelectionExcludesVersions(DjangoTestCase):
"Select all matching" reconstructs the document list, which never "Select all matching" reconstructs the document list, which never
contains version documents as rows of their own. contains version documents as rows of their own.
""" """
user = UserFactory(username="bulk_versions", superuser=True) user = User.objects.create_superuser(username="bulk_versions")
root = Document.objects.create( root = Document.objects.create(
title="shared-title root", title="shared-title root",
checksum="bulk-root", checksum="bulk-root",
+100 -87
View File
@@ -23,6 +23,7 @@ from django.core.files.uploadedfile import SimpleUploadedFile
from django.db import DataError from django.db import DataError
from django.test import override_settings from django.test import override_settings
from django.utils import timezone from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
@@ -47,22 +48,18 @@ from documents.models import Workflow
from documents.models import WorkflowAction from documents.models import WorkflowAction
from documents.models import WorkflowTrigger from documents.models import WorkflowTrigger
from documents.signals.handlers import run_workflows from documents.signals.handlers import run_workflows
from documents.tests.factories import DocumentFactory
from documents.tests.factories import TagFactory
from documents.tests.utils import ConsumeTaskMixin from documents.tests.utils import ConsumeTaskMixin
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import DocumentFactory from documents.tests.utils import read_streaming_response
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase): class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
cache.clear() cache.clear()
@@ -360,10 +357,10 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
with Path(filename).open("wb") as f: with Path(filename).open("wb") as f:
f.write(content) f.write(content)
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
grant_global(user1, "view_document") user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
grant_global(user2, "view_document") user2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
self.client.force_authenticate(user2) self.client.force_authenticate(user2)
@@ -386,7 +383,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
response = self.client.get(f"/api/documents/{doc.pk}/thumb/") response = self.client.get(f"/api/documents/{doc.pk}/thumb/")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
grant_object(user2, doc, "view_document") assign_perm("view_document", user2, doc)
response = self.client.get(f"/api/documents/{doc.pk}/download/") response = self.client.get(f"/api/documents/{doc.pk}/download/")
self.assertEqual(response.status_code, status.HTTP_200_OK) self.assertEqual(response.status_code, status.HTTP_200_OK)
@@ -763,8 +760,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- History is returned - History is returned
""" """
# No auditlog permissions # No auditlog permissions
user = UserFactory(username="test") user = User.objects.create_user(username="test")
grant_global(user, "view_document") user.user_permissions.add(*Permission.objects.filter(codename="view_document"))
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
doc = Document.objects.create( doc = Document.objects.create(
title="First title", title="First title",
@@ -779,7 +776,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
# superuser # superuser
user.is_superuser = True user.is_superuser = True
user.save() user.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
doc2 = Document.objects.create( doc2 = Document.objects.create(
title="Second title", title="Second title",
checksum="456", checksum="456",
@@ -1076,9 +1073,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["duplicate_documents"], []) self.assertEqual(response.data["duplicate_documents"], [])
def test_has_duplicates_filter_respects_document_permissions(self) -> None: def test_has_duplicates_filter_respects_document_permissions(self) -> None:
owner = UserFactory(username="duplicate-owner") owner = User.objects.create_user(username="duplicate-owner")
requester = UserFactory(username="duplicate-requester") requester = User.objects.create_user(username="duplicate-requester")
grant_global(requester, "view_document") requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
visible_document = Document.objects.create( visible_document = Document.objects.create(
title="visible document", title="visible document",
checksum="permission-match", checksum="permission-match",
@@ -1097,7 +1096,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
[document["id"] for document in response.data["results"]], [document["id"] for document in response.data["results"]],
) )
grant_object(requester, hidden_duplicate, "view_document") assign_perm("view_document", requester, hidden_duplicate)
response = self.client.get("/api/documents/?has_duplicates=true") response = self.client.get("/api/documents/?has_duplicates=true")
self.assertIn( self.assertIn(
visible_document.id, visible_document.id,
@@ -1318,10 +1317,10 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Owner filters work correctly but still respect permissions - Owner filters work correctly but still respect permissions
""" """
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
grant_global(u1, "view_document") u1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
grant_global(u2, "view_document") u2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
u1_doc1 = Document.objects.create( u1_doc1 = Document.objects.create(
title="none1", title="none1",
@@ -1354,7 +1353,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
) )
self.client.force_authenticate(user=u1) self.client.force_authenticate(user=u1)
grant_object(u1, u2_doc2, "view_document") assign_perm("view_document", u1, u2_doc2)
# Will not show any u1 docs or u2_doc1 which isn't shared # Will not show any u1 docs or u2_doc1 which isn't shared
response = self.client.get(f"/api/documents/?owner__id__none={u1.id}") response = self.client.get(f"/api/documents/?owner__id__none={u1.id}")
@@ -1401,7 +1400,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
[u1_doc1.id, u1_doc2.id, u2_doc2.id], [u1_doc1.id, u1_doc2.id, u2_doc2.id],
) )
grant_object(u2, u1_doc1, "view_document") assign_perm("view_document", u2, u1_doc1)
# Will show only documents shared by user # Will show only documents shared by user
response = self.client.get(f"/api/documents/?shared_by__id={u1.id}") response = self.client.get(f"/api/documents/?shared_by__id={u1.id}")
@@ -1425,8 +1424,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- The document is returned exactly once, not once per permission path - The document is returned exactly once, not once per permission path
(regression test for https://github.com/paperless-ngx/paperless-ngx/issues/13331) (regression test for https://github.com/paperless-ngx/paperless-ngx/issues/13331)
""" """
user = UserFactory(username="user1") user = User.objects.create_user("user1")
grant_global(user, "view_document") user.user_permissions.add(*Permission.objects.filter(codename="view_document"))
group = Group.objects.create(name="group1") group = Group.objects.create(name="group1")
user.groups.add(group) user.groups.add(group)
@@ -1434,7 +1433,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
tag2 = TagFactory() tag2 = TagFactory()
doc = DocumentFactory(title="shared", owner=user) doc = DocumentFactory(title="shared", owner=user)
doc.tags.add(tag1, tag2) doc.tags.add(tag1, tag2)
grant_object(group, doc, "view_document") assign_perm("view_document", group, doc)
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
response = self.client.get( response = self.client.get(
@@ -1453,9 +1452,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- The document does not appear in their results - The document does not appear in their results
""" """
owner = UserFactory(username="owner1") owner = User.objects.create_user("owner1")
stranger = UserFactory(username="stranger1") stranger = User.objects.create_user("stranger1")
grant_global(stranger, "view_document") stranger.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
DocumentFactory(title="private", owner=owner) DocumentFactory(title="private", owner=owner)
@@ -1473,17 +1474,17 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Only the group member sees the document - Only the group member sees the document
""" """
owner = UserFactory(username="owner2") owner = User.objects.create_user("owner2")
member = UserFactory(username="member1") member = User.objects.create_user("member1")
non_member = UserFactory(username="nonmember1") non_member = User.objects.create_user("nonmember1")
for u in (member, non_member): for u in (member, non_member):
grant_global(u, "view_document") u.user_permissions.add(*Permission.objects.filter(codename="view_document"))
group = Group.objects.create(name="group2") group = Group.objects.create(name="group2")
member.groups.add(group) member.groups.add(group)
doc = DocumentFactory(title="shared2", owner=owner) doc = DocumentFactory(title="shared2", owner=owner)
grant_object(group, doc, "view_document") assign_perm("view_document", group, doc)
self.client.force_authenticate(user=member) self.client.force_authenticate(user=member)
response = self.client.get("/api/documents/") response = self.client.get("/api/documents/")
@@ -1784,8 +1785,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Statistics only include inbox counts for tags accessible by the user - Statistics only include inbox counts for tags accessible by the user
""" """
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
inbox_tag_u1 = Tag.objects.create(name="inbox_u1", is_inbox_tag=True, owner=u1) inbox_tag_u1 = Tag.objects.create(name="inbox_u1", is_inbox_tag=True, owner=u1)
Tag.objects.create(name="inbox_u2", is_inbox_tag=True, owner=u2) Tag.objects.create(name="inbox_u2", is_inbox_tag=True, owner=u2)
doc_u1 = Document.objects.create( doc_u1 = Document.objects.create(
@@ -1815,9 +1816,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["documents_inbox"], 0) self.assertEqual(response.data["documents_inbox"], 0)
def test_statistics_with_statistics_permission(self) -> None: def test_statistics_with_statistics_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user("owner")
stats_user = UserFactory(username="stats-user") stats_user = User.objects.create_user("stats-user")
grant_global(stats_user, "view_global_statistics") stats_user.user_permissions.add(
Permission.objects.get(codename="view_global_statistics"),
)
inbox_tag = Tag.objects.create( inbox_tag = Tag.objects.create(
name="stats_inbox", name="stats_inbox",
@@ -1983,7 +1986,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED) self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED)
def test_upload_insufficient_permissions(self) -> None: def test_upload_insufficient_permissions(self) -> None:
self.client.force_authenticate(user=UserFactory(username="testuser2")) self.client.force_authenticate(user=User.objects.create_user("testuser2"))
with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f: with (Path(__file__).parent / "samples" / "simple.pdf").open("rb") as f:
response = self.client.post( response = self.client.post(
@@ -2779,9 +2782,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
mock_get_date_parser.assert_not_called() mock_get_date_parser.assert_not_called()
def test_saved_views(self) -> None: def test_saved_views(self) -> None:
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
u3 = UserFactory(username="user3") u3 = User.objects.create_user("user3")
view_perm = Permission.objects.get(codename="view_savedview") view_perm = Permission.objects.get(codename="view_savedview")
change_perm = Permission.objects.get(codename="change_savedview") change_perm = Permission.objects.get(codename="change_savedview")
@@ -2804,9 +2807,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
sort_field="", sort_field="",
) )
grant_object(u1, v2, "view_savedview") assign_perm("view_savedview", u1, v2)
grant_object(u1, v2, "change_savedview") assign_perm("change_savedview", u1, v2)
grant_object(u1, v3, "view_savedview") assign_perm("view_savedview", u1, v3)
self.client.force_authenticate(user=u1) self.client.force_authenticate(user=u1)
@@ -3061,7 +3064,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertListEqual(saved_view_settings["sidebar_views_visible_ids"], [v2.id]) self.assertListEqual(saved_view_settings["sidebar_views_visible_ids"], [v2.id])
def test_saved_view_create_update_patch(self) -> None: def test_saved_view_create_update_patch(self) -> None:
UserFactory(username="user1") User.objects.create_user("user1")
view = { view = {
"name": "test", "name": "test",
@@ -3124,7 +3127,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- Display options are updated - Display options are updated
- Display fields are validated - Display fields are validated
""" """
UserFactory(username="user1") User.objects.create_user("user1")
view = { view = {
"name": "test", "name": "test",
@@ -3565,11 +3568,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Notes are neither created nor deleted - Notes are neither created nor deleted
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
user2.save() user2.save()
doc = Document.objects.create( doc = Document.objects.create(
@@ -3589,7 +3592,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(resp.content, b"Insufficient permissions to view notes") self.assertEqual(resp.content, b"Insufficient permissions to view notes")
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
grant_object(user1, doc, "view_document") assign_perm("view_document", user1, doc)
resp = self.client.post( resp = self.client.post(
f"/api/documents/{doc.pk}/notes/", f"/api/documents/{doc.pk}/notes/",
@@ -3613,8 +3616,12 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_notes_require_global_document_permissions(self) -> None: def test_notes_require_global_document_permissions(self) -> None:
user = UserFactory(username="note_editor") user = User.objects.create_user(username="note_editor")
grant_global(user, "view_note", "add_note", "delete_note") user.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_note", "add_note", "delete_note"],
),
)
doc = Document.objects.create( doc = Document.objects.create(
title="test", title="test",
mime_type="application/pdf", mime_type="application/pdf",
@@ -3627,7 +3634,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
response = self.client.get(f"/api/documents/{doc.pk}/notes/") response = self.client.get(f"/api/documents/{doc.pk}/notes/")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
grant_global(user, "view_document") user.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
user = User.objects.get(pk=user.pk) user = User.objects.get(pk=user.pk)
self.client.force_authenticate(user) self.client.force_authenticate(user)
response = self.client.get(f"/api/documents/{doc.pk}/notes/") response = self.client.get(f"/api/documents/{doc.pk}/notes/")
@@ -3639,7 +3648,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
) )
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
grant_global(user, "change_document") user.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
user = User.objects.get(pk=user.pk) user = User.objects.get(pk=user.pk)
self.client.force_authenticate(user) self.client.force_authenticate(user)
response = self.client.post( response = self.client.post(
@@ -3786,12 +3797,12 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- Unique items are created - Unique items are created
- Non-unique items are not allowed - Non-unique items are not allowed
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_global(user1, "add_tag") user1.user_permissions.add(*Permission.objects.filter(codename="add_tag"))
user1.save() user1.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
grant_global(user2, "add_tag") user2.user_permissions.add(*Permission.objects.filter(codename="add_tag"))
user2.save() user2.save()
# User 1 creates tag 1 owned by user 1 by default # User 1 creates tag 1 owned by user 1 by default
@@ -3846,12 +3857,12 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
- Unique items are created - Unique items are created
- Non-unique items are not allowed on update - Non-unique items are not allowed on update
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_global(user1, "change_tag") user1.user_permissions.add(*Permission.objects.filter(codename="change_tag"))
user1.save() user1.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
grant_global(user2, "change_tag") user2.user_permissions.add(*Permission.objects.filter(codename="change_tag"))
user2.save() user2.save()
# Create name tag 1 owned by user 1 # Create name tag 1 owned by user 1
@@ -3982,11 +3993,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Links only shown if user has permissions - Links only shown if user has permissions
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
user2.save() user2.save()
doc = Document.objects.create( doc = Document.objects.create(
@@ -4006,7 +4017,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(resp.content, b"Insufficient permissions to add share link") self.assertEqual(resp.content, b"Insufficient permissions to add share link")
self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(resp.status_code, status.HTTP_403_FORBIDDEN)
grant_object(user1, doc, "change_document") assign_perm("change_document", user1, doc)
resp = self.client.get( resp = self.client.get(
f"/api/documents/{doc.pk}/share_links/", f"/api/documents/{doc.pk}/share_links/",
@@ -4023,11 +4034,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Share link creation is denied until view permission is granted - Share link creation is denied until view permission is granted
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_global(user1, "add_sharelink") user1.user_permissions.add(*Permission.objects.filter(codename="add_sharelink"))
user1.save() user1.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
user2.save() user2.save()
doc = Document.objects.create( doc = Document.objects.create(
@@ -4049,7 +4060,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
) )
self.assertEqual(create_resp.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(create_resp.status_code, status.HTTP_403_FORBIDDEN)
grant_object(user1, doc, "view_document") assign_perm("view_document", user1, doc)
create_resp = self.client.post( create_resp = self.client.post(
"/api/share_links/", "/api/share_links/",
@@ -4061,7 +4072,9 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
) )
self.assertEqual(create_resp.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(create_resp.status_code, status.HTTP_403_FORBIDDEN)
grant_global(user1, "view_document") user1.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
user1 = User.objects.get(pk=user1.pk) user1 = User.objects.get(pk=user1.pk)
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
create_resp = self.client.post( create_resp = self.client.post(
@@ -4084,11 +4097,11 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- ASN +1 from user2's doc is returned for user1 - ASN +1 from user2's doc is returned for user1
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
user2.save() user2.save()
doc1 = Document.objects.create( doc1 = Document.objects.create(
@@ -4128,8 +4141,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- ASN 1 is returned - ASN 1 is returned
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
doc1 = Document.objects.create( doc1 = Document.objects.create(
@@ -4157,7 +4170,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Explicit error is returned - Explicit error is returned
""" """
user1 = UserFactory(username="test1", superuser=True) user1 = User.objects.create_superuser(username="test1")
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -4335,8 +4348,8 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
THEN: THEN:
- Error response is returned - Error response is returned
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user1.save() user1.save()
doc = Document.objects.create( doc = Document.objects.create(
@@ -4441,7 +4454,7 @@ class TestDocumentApiTagColors(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
@@ -4521,7 +4534,7 @@ class TestDocumentApiCustomFieldsSorting(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
self.doc1 = Document.objects.create( self.doc1 = Document.objects.create(
+8 -8
View File
@@ -2,16 +2,16 @@ import json
import shutil import shutil
from unittest import mock from unittest import mock
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.core import mail from django.core import mail
from django.test import override_settings from django.test import override_settings
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.models import Document from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import SampleDirMixin from documents.tests.utils import SampleDirMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase): class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
@@ -20,7 +20,7 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
self.doc1 = Document.objects.create( self.doc1 = Document.objects.create(
@@ -303,8 +303,8 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
THEN: THEN:
- Forbidden response is returned - Forbidden response is returned
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_global(user1, "view_document") user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
doc_owned = Document.objects.create( doc_owned = Document.objects.create(
title="owned_doc", title="owned_doc",
@@ -338,8 +338,8 @@ class TestEmail(DirectoriesMixin, SampleDirMixin, APITestCase):
THEN: THEN:
- Request succeeds - Request succeeds
""" """
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
grant_global(user1, "view_document") user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -4,6 +4,7 @@ from collections.abc import Callable
from datetime import date from datetime import date
from urllib.parse import quote from urllib.parse import quote
from django.contrib.auth.models import User
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.models import CustomField from documents.models import CustomField
@@ -12,8 +13,7 @@ from documents.models import Document
from documents.models import SavedView from documents.models import SavedView
from documents.models import SavedViewFilterRule from documents.models import SavedViewFilterRule
from documents.serialisers import DocumentSerializer from documents.serialisers import DocumentSerializer
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
class DocumentWrapper: class DocumentWrapper:
@@ -35,7 +35,7 @@ class TestCustomFieldsSearch(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
# Create one custom field per type. The fields are called f"{type}_field". # Create one custom field per type. The fields are called f"{type}_field".
+54 -35
View File
@@ -3,10 +3,12 @@ import json
from unittest import mock from unittest import mock
from django.contrib.auth.models import Group from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.db import connection from django.db import connection
from django.test import override_settings from django.test import override_settings
from django.test.utils import CaptureQueriesContext from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from guardian.shortcuts import get_groups_with_perms from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms from guardian.shortcuts import get_users_with_perms
from rest_framework import status from rest_framework import status
@@ -19,17 +21,14 @@ from documents.models import Document
from documents.models import DocumentType from documents.models import DocumentType
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestApiObjects(DirectoriesMixin, APITestCase): class TestApiObjects(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
user = UserFactory(username="temp_admin", superuser=True) user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
self.tag1 = Tag.objects.create(name="t1", is_inbox_tag=True) self.tag1 = Tag.objects.create(name="t1", is_inbox_tag=True)
@@ -154,7 +153,7 @@ class TestApiObjects(DirectoriesMixin, APITestCase):
# A newer document owned by another user must not leak through the # A newer document owned by another user must not leak through the
# aggregate for a non-superuser who cannot view it # aggregate for a non-superuser who cannot view it
other = UserFactory(username="other") other = User.objects.create_user(username="other")
Document.objects.create( Document.objects.create(
mime_type="application/pdf", mime_type="application/pdf",
correspondent=self.c1, correspondent=self.c1,
@@ -163,8 +162,10 @@ class TestApiObjects(DirectoriesMixin, APITestCase):
owner=other, owner=other,
) )
user = UserFactory(username="regular") user = User.objects.create_user(username="regular")
grant_global(user, "view_correspondent") user.user_permissions.add(
Permission.objects.get(codename="view_correspondent"),
)
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
response = self.client.get("/api/correspondents/?last_correspondence=true") response = self.client.get("/api/correspondents/?last_correspondence=true")
@@ -199,7 +200,7 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
user = UserFactory(username="temp_admin", superuser=True) user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
self.sp1 = StoragePath.objects.create(name="sp1", path="Something/{checksum}") self.sp1 = StoragePath.objects.create(name="sp1", path="Something/{checksum}")
@@ -454,9 +455,11 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
self.assertEqual(response.data, "folder/Something.pdf") self.assertEqual(response.data, "folder/Something.pdf")
def test_test_storage_path_requires_document_view_permission(self) -> None: def test_test_storage_path_requires_document_view_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
unprivileged = UserFactory(username="unprivileged") unprivileged = User.objects.create_user(username="unprivileged")
grant_global(unprivileged, "view_document") unprivileged.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
document = Document.objects.create( document = Document.objects.create(
mime_type="application/pdf", mime_type="application/pdf",
owner=owner, owner=owner,
@@ -478,15 +481,15 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
self.assertIn("document", response.data) self.assertIn("document", response.data)
def test_test_storage_path_allows_shared_document_view_permission(self) -> None: def test_test_storage_path_allows_shared_document_view_permission(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
viewer = UserFactory(username="viewer") viewer = User.objects.create_user(username="viewer")
document = Document.objects.create( document = Document.objects.create(
mime_type="application/pdf", mime_type="application/pdf",
owner=owner, owner=owner,
title="Shared", title="Shared",
checksum="123", checksum="123",
) )
grant_object(viewer, document, "view_document") assign_perm("view_document", viewer, document)
self.client.force_authenticate(user=viewer) self.client.force_authenticate(user=viewer)
response = self.client.post( response = self.client.post(
@@ -501,7 +504,9 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
) )
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
grant_global(viewer, "view_document") viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
viewer = User.objects.get(pk=viewer.pk) viewer = User.objects.get(pk=viewer.pk)
self.client.force_authenticate(user=viewer) self.client.force_authenticate(user=viewer)
response = self.client.post( response = self.client.post(
@@ -540,12 +545,14 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_exposes_basic_document_context_but_not_sensitive_owner_data( def test_test_storage_path_exposes_basic_document_context_but_not_sensitive_owner_data(
self, self,
) -> None: ) -> None:
owner = UserFactory( owner = User.objects.create_user(
username="owner", username="owner",
password="password", password="password",
email="owner@example.com", email="owner@example.com",
) )
grant_global(owner, "view_document") owner.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
document = Document.objects.create( document = Document.objects.create(
mime_type="application/pdf", mime_type="application/pdf",
owner=owner, owner=owner,
@@ -607,8 +614,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_includes_related_objects_for_visible_document( def test_test_storage_path_includes_related_objects_for_visible_document(
self, self,
) -> None: ) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
viewer = UserFactory(username="viewer") viewer = User.objects.create_user(username="viewer")
private_correspondent = Correspondent.objects.create( private_correspondent = Correspondent.objects.create(
name="Private Correspondent", name="Private Correspondent",
owner=owner, owner=owner,
@@ -620,8 +627,10 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
title="Document", title="Document",
checksum="123", checksum="123",
) )
grant_object(viewer, document, "view_document") assign_perm("view_document", viewer, document)
grant_global(viewer, "view_document") viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
self.client.force_authenticate(user=viewer) self.client.force_authenticate(user=viewer)
response = self.client.post( response = self.client.post(
@@ -653,7 +662,7 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
self.assertEqual(response.data, "Private Correspondent.pdf") self.assertEqual(response.data, "Private Correspondent.pdf")
def test_test_storage_path_superuser_can_view_private_related_objects(self) -> None: def test_test_storage_path_superuser_can_view_private_related_objects(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
private_correspondent = Correspondent.objects.create( private_correspondent = Correspondent.objects.create(
name="Private Correspondent", name="Private Correspondent",
owner=owner, owner=owner,
@@ -684,8 +693,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_includes_doc_type_storage_path_and_tags( def test_test_storage_path_includes_doc_type_storage_path_and_tags(
self, self,
) -> None: ) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
viewer = UserFactory(username="viewer") viewer = User.objects.create_user(username="viewer")
private_document_type = DocumentType.objects.create( private_document_type = DocumentType.objects.create(
name="Private Type", name="Private Type",
owner=owner, owner=owner,
@@ -708,8 +717,10 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
checksum="123", checksum="123",
) )
document.tags.add(private_tag) document.tags.add(private_tag)
grant_object(viewer, document, "view_document") assign_perm("view_document", viewer, document)
grant_global(viewer, "view_document") viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
self.client.force_authenticate(user=viewer) self.client.force_authenticate(user=viewer)
response = self.client.post( response = self.client.post(
@@ -745,8 +756,8 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
def test_test_storage_path_includes_custom_fields_for_visible_document( def test_test_storage_path_includes_custom_fields_for_visible_document(
self, self,
) -> None: ) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
viewer = UserFactory(username="viewer") viewer = User.objects.create_user(username="viewer")
document = Document.objects.create( document = Document.objects.create(
mime_type="application/pdf", mime_type="application/pdf",
owner=owner, owner=owner,
@@ -762,8 +773,10 @@ class TestApiStoragePaths(DirectoriesMixin, APITestCase):
field=custom_field, field=custom_field,
value_int=42, value_int=42,
) )
grant_object(viewer, document, "view_document") assign_perm("view_document", viewer, document)
grant_global(viewer, "view_document") viewer.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
self.client.force_authenticate(user=viewer) self.client.force_authenticate(user=viewer)
response = self.client.post( response = self.client.post(
@@ -785,7 +798,7 @@ class TestBulkEditObjects(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.temp_admin = UserFactory(username="temp_admin", superuser=True) self.temp_admin = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.temp_admin) self.client.force_authenticate(user=self.temp_admin)
self.t1 = Tag.objects.create(name="t1") self.t1 = Tag.objects.create(name="t1")
@@ -1017,7 +1030,9 @@ class TestBulkEditObjects(APITestCase):
THEN: THEN:
- User is able to delete objects - User is able to delete objects
""" """
grant_global(self.user1, "delete_tag") self.user1.user_permissions.add(
*Permission.objects.filter(codename="delete_tag"),
)
self.user1.save() self.user1.save()
self.client.force_authenticate(user=self.user1) self.client.force_authenticate(user=self.user1)
@@ -1047,7 +1062,9 @@ class TestBulkEditObjects(APITestCase):
self.t2.owner = User.objects.get(username="temp_admin") self.t2.owner = User.objects.get(username="temp_admin")
self.t2.save() self.t2.save()
grant_global(self.user1, "delete_tag") self.user1.user_permissions.add(
*Permission.objects.filter(codename="delete_tag"),
)
self.user1.save() self.user1.save()
self.client.force_authenticate(user=self.user1) self.client.force_authenticate(user=self.user1)
@@ -1080,7 +1097,9 @@ class TestBulkEditObjects(APITestCase):
self.t2.owner = User.objects.get(username="temp_admin") self.t2.owner = User.objects.get(username="temp_admin")
self.t2.save() self.t2.save()
grant_global(self.user1, "delete_tag") self.user1.user_permissions.add(
*Permission.objects.filter(codename="delete_tag"),
)
self.user1.save() self.user1.save()
self.client.force_authenticate(user=self.user1) self.client.force_authenticate(user=self.user1)
+76 -62
View File
@@ -5,7 +5,9 @@ from unittest import mock
from allauth.mfa.models import Authenticator from allauth.mfa.models import Authenticator
from allauth.mfa.totp.internal import auth as totp_auth from allauth.mfa.totp.internal import auth as totp_auth
from django.contrib.auth.models import Group from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from guardian.shortcuts import get_perms from guardian.shortcuts import get_perms
from guardian.shortcuts import get_users_with_perms from guardian.shortcuts import get_users_with_perms
from rest_framework import status from rest_framework import status
@@ -17,11 +19,7 @@ from documents.models import DocumentType
from documents.models import MatchingModel from documents.models import MatchingModel
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestApiAuth(DirectoriesMixin, APITestCase): class TestApiAuth(DirectoriesMixin, APITestCase):
@@ -95,14 +93,14 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.assertNotIn("X-Version", response) self.assertNotIn("X-Version", response)
def test_api_version_with_auth(self) -> None: def test_api_version_with_auth(self) -> None:
user = UserFactory(username="test", superuser=True) user = User.objects.create_superuser(username="test")
self.client.force_authenticate(user) self.client.force_authenticate(user)
response = self.client.get("/api/documents/") response = self.client.get("/api/documents/")
self.assertIn("X-Api-Version", response) self.assertIn("X-Api-Version", response)
self.assertIn("X-Version", response) self.assertIn("X-Version", response)
def test_api_insufficient_permissions(self) -> None: def test_api_insufficient_permissions(self) -> None:
user = UserFactory(username="test") user = User.objects.create_user(username="test")
self.client.force_authenticate(user) self.client.force_authenticate(user)
Document.objects.create(title="Test") Document.objects.create(title="Test")
@@ -139,8 +137,8 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
) )
def test_api_sufficient_permissions(self) -> None: def test_api_sufficient_permissions(self) -> None:
user = UserFactory(username="test") user = User.objects.create_user(username="test")
grant_all_global(user) user.user_permissions.add(*Permission.objects.all())
user.is_staff = True user.is_staff = True
self.client.force_authenticate(user) self.client.force_authenticate(user)
@@ -168,9 +166,9 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
) )
def test_api_get_object_permissions(self) -> None: def test_api_get_object_permissions(self) -> None:
user1 = UserFactory(username="test1") user1 = User.objects.create_user(username="test1")
user2 = UserFactory(username="test2") user2 = User.objects.create_user(username="test2")
grant_global(user1, "view_document") user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
self.assertEqual( self.assertEqual(
@@ -207,7 +205,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN: THEN:
- Object created with current user as owner - Object created with current user as owner
""" """
user1 = UserFactory(username="user1", superuser=True) user1 = User.objects.create_superuser(username="user1")
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -236,7 +234,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN: THEN:
- Object created with no owner - Object created with no owner
""" """
user1 = UserFactory(username="user1", superuser=True) user1 = User.objects.create_superuser(username="user1")
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -267,7 +265,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN: THEN:
- Object permissions are set appropriately - Object permissions are set appropriately
""" """
user1 = UserFactory(username="user1", superuser=True) user1 = User.objects.create_superuser(username="user1")
user2 = User.objects.create(username="user2") user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1") group1 = Group.objects.create(name="group1")
@@ -315,7 +313,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN: THEN:
- Object permissions are set appropriately - Object permissions are set appropriately
""" """
user1 = UserFactory(username="user1", superuser=True) user1 = User.objects.create_superuser(username="user1")
user2 = User.objects.create(username="user2") user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1") group1 = Group.objects.create(name="group1")
@@ -365,7 +363,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
mime_type="application/pdf", mime_type="application/pdf",
content="this is a document", content="this is a document",
) )
user1 = UserFactory(username="user1", superuser=True) user1 = User.objects.create_superuser(username="user1")
user2 = User.objects.create(username="user2") user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1") group1 = Group.objects.create(name="group1")
@@ -415,16 +413,16 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
mime_type="application/pdf", mime_type="application/pdf",
content="this is a document", content="this is a document",
) )
user1 = UserFactory(username="user1", superuser=True) user1 = User.objects.create_superuser(username="user1")
user2 = User.objects.create(username="user2") user2 = User.objects.create(username="user2")
group1 = Group.objects.create(name="group1") group1 = Group.objects.create(name="group1")
doc.owner = user1 doc.owner = user1
doc.save() doc.save()
grant_object(user2, doc, "view_document") assign_perm("view_document", user2, doc)
grant_object(user2, doc, "change_document") assign_perm("change_document", user2, doc)
grant_object(group1, doc, "view_document") assign_perm("view_document", group1, doc)
grant_object(group1, doc, "change_document") assign_perm("change_document", group1, doc)
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -448,9 +446,11 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.assertIn("change_document", get_perms(group1, doc)) self.assertIn("change_document", get_perms(group1, doc))
def test_document_permissions_change_requires_owner(self) -> None: def test_document_permissions_change_requires_owner(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
editor = UserFactory(username="editor") editor = User.objects.create_user(username="editor")
grant_all_global(editor) editor.user_permissions.add(
*Permission.objects.all(),
)
doc = Document.objects.create( doc = Document.objects.create(
title="Ownered doc", title="Ownered doc",
@@ -460,8 +460,8 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
owner=owner, owner=owner,
) )
grant_object(editor, doc, "view_document") assign_perm("view_document", editor, doc)
grant_object(editor, doc, "change_document") assign_perm("change_document", editor, doc)
self.client.force_authenticate(editor) self.client.force_authenticate(editor)
response = self.client.patch( response = self.client.patch(
@@ -499,9 +499,9 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_dynamic_permissions_fields(self) -> None: def test_dynamic_permissions_fields(self) -> None:
user1 = UserFactory(username="user1") user1 = User.objects.create_user(username="user1")
grant_global(user1, "view_document") user1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
user2 = UserFactory(username="user2") user2 = User.objects.create_user(username="user2")
Document.objects.create(title="Test", content="content 1", checksum="1") Document.objects.create(title="Test", content="content 1", checksum="1")
doc2 = Document.objects.create( doc2 = Document.objects.create(
@@ -523,10 +523,10 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
owner=user1, owner=user1,
) )
grant_object(user1, doc2, "view_document") assign_perm("view_document", user1, doc2)
grant_object(user1, doc3, "view_document") assign_perm("view_document", user1, doc3)
grant_object(user1, doc3, "change_document") assign_perm("change_document", user1, doc3)
grant_object(user2, doc4, "view_document") assign_perm("view_document", user2, doc4)
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -574,8 +574,8 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
owned by someone else with no explicit guardian grant -- mirrors owned by someone else with no explicit guardian grant -- mirrors
guardian's own ObjectPermissionChecker.has_perm() superuser shortcut. guardian's own ObjectPermissionChecker.has_perm() superuser shortcut.
""" """
superuser = UserFactory(username="admin", superuser=True) superuser = User.objects.create_superuser(username="admin")
other_user = UserFactory(username="user2") other_user = User.objects.create_user(username="user2")
Document.objects.create( Document.objects.create(
title="Test", title="Test",
content="content", content="content",
@@ -602,7 +602,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN: THEN:
- MFA required error is returned - MFA required error is returned
""" """
user1 = UserFactory(username="user1") user1 = User.objects.create_user(username="user1")
user1.set_password("password") user1.set_password("password")
user1.save() user1.save()
@@ -626,7 +626,7 @@ class TestApiAuth(DirectoriesMixin, APITestCase):
THEN: THEN:
- MFA code is required - MFA code is required
""" """
user1 = UserFactory(username="user1") user1 = User.objects.create_user(username="user1")
user1.set_password("password") user1.set_password("password")
user1.save() user1.save()
@@ -688,7 +688,7 @@ class TestApiUser(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def test_get_users(self) -> None: def test_get_users(self) -> None:
@@ -858,8 +858,10 @@ class TestApiUser(DirectoriesMixin, APITestCase):
) )
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND) self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
regular_user = UserFactory(username="regular_user") regular_user = User.objects.create_user(username="regular_user")
grant_all_global(regular_user) regular_user.user_permissions.add(
*Permission.objects.all(),
)
self.client.force_authenticate(regular_user) self.client.force_authenticate(regular_user)
Authenticator.objects.create( Authenticator.objects.create(
user=user1, user=user1,
@@ -883,9 +885,9 @@ class TestApiUser(DirectoriesMixin, APITestCase):
- Only superusers can change superuser status - Only superusers can change superuser status
""" """
user1 = UserFactory(username="user1") user1 = User.objects.create_user(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user2 = UserFactory(username="user2", superuser=True) user2 = User.objects.create_superuser(username="user2")
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -970,9 +972,9 @@ class TestApiUser(DirectoriesMixin, APITestCase):
- Only superusers can change staff status - Only superusers can change staff status
""" """
user1 = UserFactory(username="user1") user1 = User.objects.create_user(username="user1")
grant_all_global(user1) user1.user_permissions.add(*Permission.objects.all())
user2 = UserFactory(username="user2", superuser=True) user2 = User.objects.create_superuser(username="user2")
self.client.force_authenticate(user1) self.client.force_authenticate(user1)
@@ -1025,7 +1027,7 @@ class TestApiGroup(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def test_get_groups(self) -> None: def test_get_groups(self) -> None:
@@ -1126,7 +1128,7 @@ class TestBulkEditObjectPermissions(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.temp_admin = UserFactory(username="temp_admin", superuser=True) self.temp_admin = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.temp_admin) self.client.force_authenticate(user=self.temp_admin)
self.t1 = Tag.objects.create(name="t1") self.t1 = Tag.objects.create(name="t1")
@@ -1274,7 +1276,7 @@ class TestBulkEditObjectPermissions(APITestCase):
}, },
} }
grant_object(self.user3, self.t1, "view_tag") assign_perm("view_tag", self.user3, self.t1)
self.t1.owner = self.user3 self.t1.owner = self.user3
self.t1.save() self.t1.save()
@@ -1371,9 +1373,13 @@ class TestBulkEditObjectPermissions(APITestCase):
""" """
self.t1.owner = self.user2 self.t1.owner = self.user2
self.t1.save() self.t1.save()
grant_object(self.user1, self.t1, "view_tag") assign_perm("view_tag", self.user1, self.t1)
grant_object(self.user1, self.t1, "change_tag") assign_perm("change_tag", self.user1, self.t1)
grant_global(self.user1, "view_tag", "change_tag") self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag"],
),
)
user1 = User.objects.get(pk=self.user1.pk) user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1420,9 +1426,13 @@ class TestBulkEditObjectPermissions(APITestCase):
""" """
owned = Tag.objects.create(name="owned", owner=self.user1) owned = Tag.objects.create(name="owned", owner=self.user1)
shared = Tag.objects.create(name="shared", owner=self.user2) shared = Tag.objects.create(name="shared", owner=self.user2)
grant_object(self.user1, shared, "view_tag") assign_perm("view_tag", self.user1, shared)
grant_object(self.user1, shared, "change_tag") assign_perm("change_tag", self.user1, shared)
grant_global(self.user1, "view_tag", "change_tag") self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag"],
),
)
user1 = User.objects.get(pk=self.user1.pk) user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1463,10 +1473,14 @@ class TestBulkEditObjectPermissions(APITestCase):
""" """
self.t1.owner = self.user2 self.t1.owner = self.user2
self.t1.save() self.t1.save()
grant_object(self.user1, self.t1, "view_tag") assign_perm("view_tag", self.user1, self.t1)
grant_object(self.user1, self.t1, "change_tag") assign_perm("change_tag", self.user1, self.t1)
grant_object(self.user1, self.t1, "delete_tag") assign_perm("delete_tag", self.user1, self.t1)
grant_global(self.user1, "view_tag", "change_tag", "delete_tag") self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag", "delete_tag"],
),
)
user1 = User.objects.get(pk=self.user1.pk) user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1) self.client.force_authenticate(user=user1)
@@ -1571,7 +1585,7 @@ class TestBulkEditObjectPermissions(APITestCase):
- Request succeeds and null is treated as an empty user list, - Request succeeds and null is treated as an empty user list,
so the existing view permission is removed so the existing view permission is removed
""" """
grant_object(self.user1, self.t1, "view_tag") assign_perm("view_tag", self.user1, self.t1)
response = self.client.post( response = self.client.post(
"/api/bulk_edit_objects/", "/api/bulk_edit_objects/",
@@ -1666,7 +1680,7 @@ class TestFullPermissionsFlag(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.admin = UserFactory(username="admin", superuser=True) self.admin = User.objects.create_superuser(username="admin")
def test_full_perms_flag(self) -> None: def test_full_perms_flag(self) -> None:
""" """
@@ -1,5 +1,3 @@
from __future__ import annotations
import unicodedata import unicodedata
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from unittest import mock from unittest import mock
@@ -9,11 +7,8 @@ import pytest
from django.core.files.uploadedfile import SimpleUploadedFile from django.core.files.uploadedfile import SimpleUploadedFile
if TYPE_CHECKING: if TYPE_CHECKING:
from rest_framework.test import APIClient
from documents.data_models import ConsumableDocument from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides from documents.data_models import DocumentMetadataOverrides
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture() @pytest.fixture()
@@ -23,14 +18,22 @@ def consume_file_mock():
yield m yield m
@pytest.fixture()
def directories(tmp_path, settings, _media_settings):
scratch = tmp_path / "scratch"
scratch.mkdir()
settings.SCRATCH_DIR = scratch
return scratch
@pytest.mark.django_db @pytest.mark.django_db
class TestPostDocumentNFCNormalization: class TestPostDocumentNFCNormalization:
def test_nfd_filename_normalized_to_nfc( def test_nfd_filename_normalized_to_nfc(
self, self,
admin_client: APIClient, admin_client,
consume_file_mock: mock.MagicMock, consume_file_mock: mock.MagicMock,
paperless_dirs: PaperlessDirs, directories,
) -> None: ):
"""Uploaded file with NFD filename must have its name stored as NFC.""" """Uploaded file with NFD filename must have its name stored as NFC."""
nfd = unicodedata.normalize("NFD", "Rechnung März.pdf") nfd = unicodedata.normalize("NFD", "Rechnung März.pdf")
nfc = unicodedata.normalize("NFC", "Rechnung März.pdf") nfc = unicodedata.normalize("NFC", "Rechnung März.pdf")
@@ -66,10 +69,10 @@ class TestPostDocumentNFCNormalization:
def test_already_nfc_filename_unchanged( def test_already_nfc_filename_unchanged(
self, self,
admin_client: APIClient, admin_client,
consume_file_mock: mock.MagicMock, consume_file_mock: mock.MagicMock,
paperless_dirs: PaperlessDirs, directories,
) -> None: ):
"""Uploaded file with already-NFC filename must pass through unchanged.""" """Uploaded file with already-NFC filename must pass through unchanged."""
nfc = unicodedata.normalize("NFC", "Invoice_2024.pdf") nfc = unicodedata.normalize("NFC", "Invoice_2024.pdf")
+17 -16
View File
@@ -8,8 +8,7 @@ from rest_framework import status
from rest_framework.authtoken.models import Token from rest_framework.authtoken.models import Token
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
# see allauth.socialaccount.providers.openid.provider.OpenIDProvider # see allauth.socialaccount.providers.openid.provider.OpenIDProvider
@@ -56,23 +55,23 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory( self.user = User.objects.create_superuser(
username="temp_admin", username="temp_admin",
first_name="firstname", first_name="firstname",
last_name="surname", last_name="surname",
superuser=True,
) )
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def setupSocialAccount(self) -> SocialAccount: def setupSocialAccount(self) -> None:
SocialApp.objects.create( SocialApp.objects.create(
name="Keycloak", name="Keycloak",
provider="openid_connect", provider="openid_connect",
provider_id="keycloak-test", provider_id="keycloak-test",
) )
social_account = SocialAccount(uid="123456789", provider="keycloak-test") self.user.socialaccount_set.add(
self.user.socialaccount_set.add(social_account, bulk=False) SocialAccount(uid="123456789", provider="keycloak-test"),
return social_account bulk=False,
)
def test_get_profile(self) -> None: def test_get_profile(self) -> None:
""" """
@@ -110,17 +109,19 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
THEN: THEN:
- Profile is returned with social accounts - Profile is returned with social accounts
""" """
social_account = self.setupSocialAccount() self.setupSocialAccount()
openid_provider = MockOpenIDConnectProvider( openid_provider = (
app=SocialApp.objects.get(provider_id="keycloak-test"), MockOpenIDConnectProvider(
app=SocialApp.objects.get(provider_id="keycloak-test"),
),
) )
mock_list_providers.return_value = [ mock_list_providers.return_value = [
openid_provider, openid_provider,
] ]
mock_get_provider_account.return_value = MockOpenIDConnectProviderAccount( mock_get_provider_account.return_value = MockOpenIDConnectProviderAccount(
mock_social_account_dict={ mock_social_account_dict={
"name": openid_provider.name, "name": openid_provider[0].name,
}, },
) )
@@ -132,7 +133,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
response.data["social_accounts"], response.data["social_accounts"],
[ [
{ {
"id": social_account.pk, "id": 1,
"provider": "keycloak-test", "provider": "keycloak-test",
"name": "Keycloak", "name": "Keycloak",
}, },
@@ -149,7 +150,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
THEN: THEN:
- Profile is returned with "Unknown App" as name - Profile is returned with "Unknown App" as name
""" """
social_account = self.setupSocialAccount() self.setupSocialAccount()
# Remove the social app # Remove the social app
SocialApp.objects.get(provider_id="keycloak-test").delete() SocialApp.objects.get(provider_id="keycloak-test").delete()
@@ -162,7 +163,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
response.data["social_accounts"], response.data["social_accounts"],
[ [
{ {
"id": social_account.pk, "id": 1,
"provider": "keycloak-test", "provider": "keycloak-test",
"name": "Unknown App", "name": "Unknown App",
}, },
@@ -400,7 +401,7 @@ class TestApiTOTPViews(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def test_get_totp(self) -> None: def test_get_totp(self) -> None:
+46 -40
View File
@@ -7,8 +7,10 @@ import time_machine
from dateutil.relativedelta import relativedelta from dateutil.relativedelta import relativedelta
from django.contrib.auth.models import Group from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from django.utils import timezone from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
@@ -25,13 +27,10 @@ from documents.models import Tag
from documents.models import Workflow from documents.models import Workflow
from documents.search import get_backend from documents.search import get_backend
from documents.search import reset_backend from documents.search import reset_backend
from documents.tests.factories import DocumentFactory
from documents.tests.utils import DirectoriesMixin
from paperless_mail.models import MailAccount from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule from paperless_mail.models import MailRule
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
pytestmark = pytest.mark.search pytestmark = pytest.mark.search
@@ -41,7 +40,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
super().setUp() super().setUp()
reset_backend() reset_backend()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
def tearDown(self) -> None: def tearDown(self) -> None:
@@ -950,9 +949,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN: THEN:
- Terms only within docs user has access to are returned - Terms only within docs user has access to are returned
""" """
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
grant_global(u1, "view_document") u1.user_permissions.add(Permission.objects.get(codename="view_document"))
self.client.force_authenticate(user=u1) self.client.force_authenticate(user=u1)
@@ -992,7 +991,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK) self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data, ["applebaum", "apples"]) self.assertEqual(response.data, ["applebaum", "apples"])
grant_object(u1, d3, "view_document") assign_perm("view_document", u1, d3)
backend.add_or_update(d3) backend.add_or_update(d3)
response = self.client.get("/api/search/autocomplete/?term=app") response = self.client.get("/api/search/autocomplete/?term=app")
@@ -1000,10 +999,10 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(response.data, ["applebaum", "apples", "appletini"]) self.assertEqual(response.data, ["applebaum", "apples", "appletini"])
def test_search_autocomplete_group_revocation_is_immediate(self) -> None: def test_search_autocomplete_group_revocation_is_immediate(self) -> None:
user = UserFactory(username="group-user") user = User.objects.create_user("group-user")
owner = UserFactory(username="document-owner") owner = User.objects.create_user("document-owner")
group = Group.objects.create(name="temporary-viewers") group = Group.objects.create(name="temporary-viewers")
grant_global(user, "view_document") user.user_permissions.add(Permission.objects.get(codename="view_document"))
user.groups.add(group) user.groups.add(group)
document = Document.objects.create( document = Document.objects.create(
@@ -1012,7 +1011,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
checksum="group-revocation", checksum="group-revocation",
owner=owner, owner=owner,
) )
grant_object(group, document, "view_document") assign_perm("view_document", group, document)
get_backend().add_or_update(document) get_backend().add_or_update(document)
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
@@ -1092,9 +1091,11 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertIsNone(response.data["corrected_query"]) self.assertIsNone(response.data["corrected_query"])
def test_search_spelling_suggestion_suppressed_for_private_terms(self) -> None: def test_search_spelling_suggestion_suppressed_for_private_terms(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user("owner")
attacker = UserFactory(username="attacker") attacker = User.objects.create_user("attacker")
grant_global(attacker, "view_document") attacker.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
backend = get_backend() backend = get_backend()
for i in range(5): for i in range(5):
@@ -1221,9 +1222,11 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN: THEN:
- The request is rejected - The request is rejected
""" """
owner = UserFactory(username="owner") owner = User.objects.create_user("owner")
attacker = UserFactory(username="attacker") attacker = User.objects.create_user("attacker")
grant_global(attacker, "view_document") attacker.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
private_seed = Document.objects.create( private_seed = Document.objects.create(
title="private bank statement", title="private bank statement",
@@ -1531,11 +1534,11 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
- Only owned docs are returned for regular users - Only owned docs are returned for regular users
- All docs are returned for superuser - All docs are returned for superuser
""" """
superuser = UserFactory(username="superuser", superuser=True) superuser = User.objects.create_superuser("superuser")
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
grant_global(u1, "view_document") u1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
grant_global(u2, "view_document") u2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
Document.objects.create(checksum="1", content="test 1", owner=u1) Document.objects.create(checksum="1", content="test 1", owner=u1)
Document.objects.create(checksum="2", content="test 2", owner=u2) Document.objects.create(checksum="2", content="test 2", owner=u2)
@@ -1585,10 +1588,10 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN: THEN:
- Only docs with granted view permissions are returned - Only docs with granted view permissions are returned
""" """
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
grant_global(u1, "view_document") u1.user_permissions.add(*Permission.objects.filter(codename="view_document"))
grant_global(u2, "view_document") u2.user_permissions.add(*Permission.objects.filter(codename="view_document"))
d1 = Document.objects.create(checksum="1", content="test 1", owner=u1) d1 = Document.objects.create(checksum="1", content="test 1", owner=u1)
d2 = Document.objects.create(checksum="2", content="test 2", owner=u2) d2 = Document.objects.create(checksum="2", content="test 2", owner=u2)
@@ -1613,9 +1616,9 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
r = self.client.get("/api/documents/?query=test&owner__isnull=true") r = self.client.get("/api/documents/?query=test&owner__isnull=true")
self.assertEqual(r.data["count"], 1) self.assertEqual(r.data["count"], 1)
grant_object(u1, d2, "view_document") assign_perm("view_document", u1, d2)
grant_object(u1, d3, "view_document") assign_perm("view_document", u1, d3)
grant_object(u2, d1, "view_document") assign_perm("view_document", u2, d1)
backend.add_or_update(d1) backend.add_or_update(d1)
backend.add_or_update(d2) backend.add_or_update(d2)
@@ -1638,8 +1641,8 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(r.data["count"], 1) self.assertEqual(r.data["count"], 1)
def test_search_sorting(self) -> None: def test_search_sorting(self) -> None:
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
u2 = UserFactory(username="user2") u2 = User.objects.create_user("user2")
c1 = Correspondent.objects.create(name="corres Ax") c1 = Correspondent.objects.create(name="corres Ax")
c2 = Correspondent.objects.create(name="corres Cx") c2 = Correspondent.objects.create(name="corres Cx")
c3 = Correspondent.objects.create(name="corres Bx") c3 = Correspondent.objects.create(name="corres Bx")
@@ -1889,8 +1892,8 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
pk=5, pk=5,
) )
user1 = UserFactory(username="bank user1") user1 = User.objects.create_user("bank user1")
user2 = UserFactory(username="user2", superuser=True) user2 = User.objects.create_superuser("user2")
group1 = Group.objects.create(name="bank group1") group1 = Group.objects.create(name="bank group1")
Group.objects.create(name="group2") Group.objects.create(name="group2")
@@ -1922,7 +1925,7 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
sort_field="", sort_field="",
owner=user2, owner=user2,
) )
grant_object(user1, shared_view, "view_savedview") assign_perm("view_savedview", user1, shared_view)
mail_account1 = MailAccount.objects.create(name="bank mail account 1") mail_account1 = MailAccount.objects.create(name="bank mail account 1")
mail_account2 = MailAccount.objects.create(name="mail account 2") mail_account2 = MailAccount.objects.create(name="mail account 2")
mail_rule1 = MailRule.objects.create( mail_rule1 = MailRule.objects.create(
@@ -2015,9 +2018,12 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
self.assertEqual(returned.get(root.id), "latest content") self.assertEqual(returned.get(root.id), "latest content")
def test_global_search_filters_owned_mail_objects(self) -> None: def test_global_search_filters_owned_mail_objects(self) -> None:
user1 = UserFactory(username="mail-search-user") user1 = User.objects.create_user("mail-search-user")
user2 = UserFactory(username="other-mail-search-user") user2 = User.objects.create_user("other-mail-search-user")
grant_global(user1, "view_mailaccount", "view_mailrule") user1.user_permissions.add(
Permission.objects.get(codename="view_mailaccount"),
Permission.objects.get(codename="view_mailrule"),
)
own_account = MailAccount.objects.create( own_account = MailAccount.objects.create(
name="bank owned account", name="bank owned account",
+10 -10
View File
@@ -33,7 +33,7 @@ class TestSearchQueryErrorStillBecomesA400:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -68,7 +68,7 @@ class TestLibraryDefectsPropagate:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -98,7 +98,7 @@ class TestLibraryDefectsPropagate:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -141,7 +141,7 @@ class TestSelectionPathsAgreeWithSearch:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -181,7 +181,7 @@ class TestSelectionPathsAgreeWithSearch:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -221,7 +221,7 @@ class TestSelectionPathsAgreeWithSearch:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -259,7 +259,7 @@ class TestSelectionPathsAgreeWithSearch:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -287,7 +287,7 @@ class TestSelectionPathsAgreeWithSearch:
{ {
"documents": [], "documents": [],
"all": True, "all": True,
"filters": {"more_like_id": searchable_document.pk}, "filters": {"more_like_id": indexed_document.pk},
}, },
format="json", format="json",
) )
@@ -298,7 +298,7 @@ class TestSelectionPathsAgreeWithSearch:
self, self,
admin_client: APIClient, admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -328,7 +328,7 @@ class TestSelectionPathsAgreeWithSearch:
{ {
"documents": [], "documents": [],
"all": True, "all": True,
"filters": {"more_like_id": searchable_document.pk}, "filters": {"more_like_id": indexed_document.pk},
}, },
format="json", format="json",
) )
@@ -36,7 +36,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_query_one_over_the_cap_is_a_400( def test_query_one_over_the_cap_is_a_400(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -66,7 +66,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_query_at_exactly_the_cap_is_accepted( def test_query_at_exactly_the_cap_is_accepted(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -86,7 +86,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_an_ordinary_query_is_unaffected( def test_an_ordinary_query_is_unaffected(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -112,7 +112,7 @@ class TestPostSelectionPathsEnforceTheCap:
def test_bulk_edit_query_one_over_the_cap_is_a_400( def test_bulk_edit_query_one_over_the_cap_is_a_400(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -154,7 +154,7 @@ class TestPostSelectionPathsEnforceTheCap:
self, self,
bulk_update_task_mock: mock.MagicMock, bulk_update_task_mock: mock.MagicMock,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -187,7 +187,7 @@ class TestPostSelectionPathsEnforceTheCap:
def test_bulk_download_query_one_over_the_cap_is_a_400( def test_bulk_download_query_one_over_the_cap_is_a_400(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -236,7 +236,7 @@ class TestGlobalSearchEnforcesTheCapToo:
def test_query_one_over_the_cap_is_a_400( def test_query_one_over_the_cap_is_a_400(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -260,7 +260,7 @@ class TestGlobalSearchEnforcesTheCapToo:
def test_query_at_exactly_the_cap_is_accepted( def test_query_at_exactly_the_cap_is_accepted(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
@@ -38,7 +38,7 @@ class TestUnterminatedBracketReturnsA400:
def test_unterminated_bracket_is_a_400( def test_unterminated_bracket_is_a_400(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
query: str, query: str,
) -> None: ) -> None:
""" """
@@ -59,7 +59,7 @@ class TestUnterminatedBracketReturnsA400:
def test_properly_closed_bracket_still_searches_cleanly( def test_properly_closed_bracket_still_searches_cleanly(
self, self,
admin_client: APIClient, admin_client: APIClient,
searchable_document: Document, indexed_document: Document,
) -> None: ) -> None:
""" """
GIVEN: GIVEN:
+11 -7
View File
@@ -5,6 +5,8 @@ from datetime import timedelta
from pathlib import Path from pathlib import Path
from unittest import mock from unittest import mock
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from django.utils import timezone from django.utils import timezone
from rest_framework import status from rest_framework import status
@@ -12,10 +14,8 @@ from rest_framework.test import APITestCase
from documents.models import PaperlessTask from documents.models import PaperlessTask
from documents.permissions import has_system_status_permission from documents.permissions import has_system_status_permission
from documents.tests.factories import PaperlessTaskFactory
from paperless import version from paperless import version
from paperless_testing.factories import PaperlessTaskFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestSystemStatus(APITestCase): class TestSystemStatus(APITestCase):
@@ -23,7 +23,9 @@ class TestSystemStatus(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin", superuser=True) self.user = User.objects.create_superuser(
username="temp_admin",
)
self.tmp_dir = Path(tempfile.mkdtemp()) self.tmp_dir = Path(tempfile.mkdtemp())
self.override = override_settings(MEDIA_ROOT=self.tmp_dir) self.override = override_settings(MEDIA_ROOT=self.tmp_dir)
self.override.enable() self.override.enable()
@@ -94,7 +96,7 @@ class TestSystemStatus(APITestCase):
response = self.client.get(self.ENDPOINT) response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED) self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
self.assertEqual(response["WWW-Authenticate"], "Token") self.assertEqual(response["WWW-Authenticate"], "Token")
normal_user = UserFactory(username="normal_user") normal_user = User.objects.create_user(username="normal_user")
self.client.force_login(normal_user) self.client.force_login(normal_user)
response = self.client.get(self.ENDPOINT) response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
@@ -105,8 +107,10 @@ class TestSystemStatus(APITestCase):
response = self.client.get(self.ENDPOINT) response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED) self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
user = UserFactory(username="status_user") user = User.objects.create_user(username="status_user")
grant_global(user, "view_system_monitoring") user.user_permissions.add(
Permission.objects.get(codename="view_system_monitoring"),
)
self.client.force_login(user) self.client.force_login(user)
response = self.client.get(self.ENDPOINT) response = self.client.get(self.ENDPOINT)
+38 -20
View File
@@ -11,21 +11,22 @@ from datetime import timedelta
from unittest import mock from unittest import mock
import pytest import pytest
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.utils import timezone from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status from rest_framework import status
from rest_framework.test import APIClient from rest_framework.test import APIClient
from documents.filters import PaperlessTaskFilterSet from documents.filters import PaperlessTaskFilterSet
from documents.models import PaperlessTask from documents.models import PaperlessTask
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import PaperlessTaskFactory from documents.tests.factories import PaperlessTaskFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
pytestmark = pytest.mark.api pytestmark = pytest.mark.api
ENDPOINT = "/api/tasks/" ENDPOINT = "/api/tasks/"
ACCEPT_V10 = "application/json; version=10"
ACCEPT_V9 = "application/json; version=9" ACCEPT_V9 = "application/json; version=9"
@@ -345,16 +346,21 @@ class TestGetTasksV10:
self, self,
admin_user: User, admin_user: User,
regular_user: User, regular_user: User,
user_client: APIClient,
) -> None: ) -> None:
"""Regular users see their own tasks and unowned (system) tasks; other users' tasks are hidden.""" """Regular users see their own tasks and unowned (system) tasks; other users' tasks are hidden."""
grant_global(regular_user, "view_paperlesstask") regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
client = APIClient()
client.force_authenticate(user=regular_user)
client.credentials(HTTP_ACCEPT=ACCEPT_V10)
PaperlessTaskFactory(owner=admin_user) # other user — not visible PaperlessTaskFactory(owner=admin_user) # other user — not visible
unowned_task = PaperlessTaskFactory() # unowned (system task) — visible unowned_task = PaperlessTaskFactory() # unowned (system task) — visible
own_task = PaperlessTaskFactory(owner=regular_user) own_task = PaperlessTaskFactory(owner=regular_user)
response = user_client.get(ENDPOINT) response = client.get(ENDPOINT)
assert response.status_code == status.HTTP_200_OK assert response.status_code == status.HTTP_200_OK
assert response.data["count"] == 2 assert response.data["count"] == 2
@@ -584,7 +590,9 @@ class TestGetTasksV9:
regular_user: User, regular_user: User,
) -> None: ) -> None:
"""Non-staff users see their own tasks plus unowned tasks via v9 API.""" """Non-staff users see their own tasks plus unowned tasks via v9 API."""
grant_global(regular_user, "view_paperlesstask") regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
client = APIClient() client = APIClient()
client.force_authenticate(user=regular_user) client.force_authenticate(user=regular_user)
@@ -724,17 +732,19 @@ class TestAcknowledge:
assert response.status_code == status.HTTP_403_FORBIDDEN assert response.status_code == status.HTTP_403_FORBIDDEN
def test_succeeds_with_change_permission( def test_succeeds_with_change_permission(self, regular_user: User) -> None:
self,
regular_user: User,
user_client: APIClient,
) -> None:
"""Users granted change_paperlesstask permission can acknowledge tasks.""" """Users granted change_paperlesstask permission can acknowledge tasks."""
grant_global(regular_user, "change_paperlesstask") regular_user.user_permissions.add(
Permission.objects.get(codename="change_paperlesstask"),
)
regular_user.save() regular_user.save()
client = APIClient()
client.force_authenticate(user=regular_user)
client.credentials(HTTP_ACCEPT=ACCEPT_V10)
task = PaperlessTaskFactory() task = PaperlessTaskFactory()
response = user_client.post( response = client.post(
ENDPOINT + "acknowledge/", ENDPOINT + "acknowledge/",
{"tasks": [task.id]}, {"tasks": [task.id]},
format="json", format="json",
@@ -797,7 +807,9 @@ class TestSummaryPermissions:
regular_user, regular_user,
) -> None: ) -> None:
"""A user with view_system_monitoring but no document permissions can access summary/.""" """A user with view_system_monitoring but no document permissions can access summary/."""
grant_global(regular_user, "view_system_monitoring") regular_user.user_permissions.add(
Permission.objects.get(codename="view_system_monitoring"),
)
response = user_client.get(ENDPOINT + "summary/") response = user_client.get(ENDPOINT + "summary/")
@@ -810,7 +822,9 @@ class TestSummaryPermissions:
admin_user, admin_user,
) -> None: ) -> None:
"""Monitoring user sees aggregate data for all tasks, not just unowned ones.""" """Monitoring user sees aggregate data for all tasks, not just unowned ones."""
grant_global(regular_user, "view_system_monitoring") regular_user.user_permissions.add(
Permission.objects.get(codename="view_system_monitoring"),
)
PaperlessTaskFactory( PaperlessTaskFactory(
owner=admin_user, owner=admin_user,
task_type=PaperlessTask.TaskType.CONSUME_FILE, task_type=PaperlessTask.TaskType.CONSUME_FILE,
@@ -831,7 +845,9 @@ class TestSummaryPermissions:
) -> None: ) -> None:
"""A regular user with view_paperlesstask but not view_system_monitoring sees only """A regular user with view_paperlesstask but not view_system_monitoring sees only
their own tasks and unowned tasks in the summary, not other users' tasks.""" their own tasks and unowned tasks in the summary, not other users' tasks."""
grant_global(regular_user, "view_paperlesstask") regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
PaperlessTaskFactory( PaperlessTaskFactory(
owner=regular_user, owner=regular_user,
@@ -996,7 +1012,9 @@ class TestDuplicateDocumentsPermissions:
@pytest.fixture() @pytest.fixture()
def user_v9_client(self, regular_user: User) -> APIClient: def user_v9_client(self, regular_user: User) -> APIClient:
grant_global(regular_user, "view_paperlesstask") regular_user.user_permissions.add(
Permission.objects.get(codename="view_paperlesstask"),
)
client = APIClient() client = APIClient()
client.force_authenticate(user=regular_user) client.force_authenticate(user=regular_user)
client.credentials(HTTP_ACCEPT=ACCEPT_V9) client.credentials(HTTP_ACCEPT=ACCEPT_V9)
@@ -1067,7 +1085,7 @@ class TestDuplicateDocumentsPermissions:
) -> None: ) -> None:
"""A user with explicit guardian view_document permission sees the duplicate_of document.""" """A user with explicit guardian view_document permission sees the duplicate_of document."""
doc = DocumentFactory(owner=admin_user, title="Granted Doc") doc = DocumentFactory(owner=admin_user, title="Granted Doc")
grant_object(regular_user, doc, "view_document") assign_perm("view_document", regular_user, doc)
PaperlessTaskFactory( PaperlessTaskFactory(
owner=regular_user, owner=regular_user,
status=PaperlessTask.Status.SUCCESS, status=PaperlessTask.Status.SUCCESS,
+9 -9
View File
@@ -1,21 +1,21 @@
from datetime import date from datetime import date
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.core.cache import cache from django.core.cache import cache
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.models import Document from documents.models import Document
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
class TestTrashAPI(DirectoriesMixin, APITestCase): class TestTrashAPI(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="temp_admin") self.user = User.objects.create_user(username="temp_admin")
grant_all_global(self.user) self.user.user_permissions.add(*Permission.objects.all())
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
cache.clear() cache.clear()
@@ -70,7 +70,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
self.assertEqual(Document.global_objects.count(), 0) self.assertEqual(Document.global_objects.count(), 0)
def test_trash_list_requires_global_document_view_permission(self) -> None: def test_trash_list_requires_global_document_view_permission(self) -> None:
user = UserFactory(username="trash_owner") user = User.objects.create_user(username="trash_owner")
document = Document.objects.create(title="Owned", owner=user) document = Document.objects.create(title="Owned", owner=user)
document.delete() document.delete()
self.client.force_authenticate(user) self.client.force_authenticate(user)
@@ -140,7 +140,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
created=date(2023, 1, 2), created=date(2023, 1, 2),
) )
document_not_owned.delete() document_not_owned.delete()
user2 = UserFactory(username="user2") user2 = User.objects.create_user(username="user2")
document_u2 = Document.objects.create( document_u2 = Document.objects.create(
title="Title3", title="Title3",
content="content3", content="content3",
@@ -158,7 +158,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
self.assertEqual(resp.data["results"][1]["id"], document_u1.pk) self.assertEqual(resp.data["results"][1]["id"], document_u1.pk)
# superuser sees all documents # superuser sees all documents
superuser = UserFactory(username="superuser", superuser=True) superuser = User.objects.create_superuser(username="superuser")
self.client.force_authenticate(user=superuser) self.client.force_authenticate(user=superuser)
resp = self.client.get("/api/trash/") resp = self.client.get("/api/trash/")
self.assertEqual(resp.status_code, status.HTTP_200_OK) self.assertEqual(resp.status_code, status.HTTP_200_OK)
@@ -174,7 +174,7 @@ class TestTrashAPI(DirectoriesMixin, APITestCase):
- 403 Forbidden - 403 Forbidden
""" """
user2 = UserFactory(username="user2") user2 = User.objects.create_user(username="user2")
document = Document.objects.create( document = Document.objects.create(
title="Title", title="Title",
content="content", content="content",
+5 -5
View File
@@ -1,13 +1,13 @@
import json import json
from django.contrib.auth.models import Permission from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
from documents.tests.utils import DirectoriesMixin
from paperless.version import __full_version_str__ from paperless.version import __full_version_str__
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
class TestApiUiSettings(DirectoriesMixin, APITestCase): class TestApiUiSettings(DirectoriesMixin, APITestCase):
@@ -15,7 +15,7 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.test_user = UserFactory(username="test", superuser=True) self.test_user = User.objects.create_superuser(username="test")
self.test_user.first_name = "Test" self.test_user.first_name = "Test"
self.test_user.last_name = "User" self.test_user.last_name = "User"
self.test_user.save() self.test_user.save()
@@ -91,7 +91,7 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
) )
def test_api_set_ui_settings_insufficient_global_permissions(self) -> None: def test_api_set_ui_settings_insufficient_global_permissions(self) -> None:
not_superuser = UserFactory(username="test_not_superuser") not_superuser = User.objects.create_user(username="test_not_superuser")
self.client.force_authenticate(user=not_superuser) self.client.force_authenticate(user=not_superuser)
settings = { settings = {
@@ -111,7 +111,7 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_api_set_ui_settings_sufficient_global_permissions(self) -> None: def test_api_set_ui_settings_sufficient_global_permissions(self) -> None:
not_superuser = UserFactory(username="test_not_superuser") not_superuser = User.objects.create_user(username="test_not_superuser")
not_superuser.user_permissions.add( not_superuser.user_permissions.add(
*Permission.objects.filter(codename__contains="uisettings"), *Permission.objects.filter(codename__contains="uisettings"),
) )
+2 -3
View File
@@ -14,8 +14,7 @@ from documents.models import Tag
from documents.models import Workflow from documents.models import Workflow
from documents.models import WorkflowAction from documents.models import WorkflowAction
from documents.models import WorkflowTrigger from documents.models import WorkflowTrigger
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
class TestApiWorkflows(DirectoriesMixin, APITestCase): class TestApiWorkflows(DirectoriesMixin, APITestCase):
@@ -26,7 +25,7 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
user = UserFactory(username="temp_admin", superuser=True) user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=user) self.client.force_authenticate(user=user)
self.user2 = User.objects.create(username="user2") self.user2 = User.objects.create(username="user2")
self.user3 = User.objects.create(username="user3") self.user3 = User.objects.create(username="user3")
+75 -75
View File
@@ -2,8 +2,8 @@ import shutil
from collections.abc import Generator from collections.abc import Generator
from contextlib import contextmanager from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from unittest import mock
import pytest
from django.conf import settings from django.conf import settings
from django.test import TestCase from django.test import TestCase
from django.test import override_settings from django.test import override_settings
@@ -18,11 +18,11 @@ from documents.models import Document
from documents.models import Tag from documents.models import Tag
from documents.plugins.base import StopConsumeTaskError from documents.plugins.base import StopConsumeTaskError
from documents.tests.utils import ConsumeTaskMixin from documents.tests.utils import ConsumeTaskMixin
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin from documents.tests.utils import SampleDirMixin
from paperless.models import ApplicationConfiguration from paperless.models import ApplicationConfiguration
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.fakes.progress import FakeProgressManager
class GetReaderPluginMixin: class GetReaderPluginMixin:
@@ -31,7 +31,7 @@ class GetReaderPluginMixin:
reader = BarcodePlugin( reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath), ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(), DocumentMetadataOverrides(),
FakeProgressManager(filepath.name, None), DummyProgressManager(filepath.name, None),
self.dirs.scratch_dir, self.dirs.scratch_dir,
"task-id", "task-id",
) )
@@ -86,7 +86,6 @@ class TestBarcode(
self.assertDictEqual(separator_page_numbers, {1: False}) self.assertDictEqual(separator_page_numbers, {1: False})
@override_settings(CONSUMER_ENABLE_ASN_BARCODE=True) @override_settings(CONSUMER_ENABLE_ASN_BARCODE=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_asn_barcode_duplicate_in_trash_fails(self) -> None: def test_asn_barcode_duplicate_in_trash_fails(self) -> None:
""" """
GIVEN: GIVEN:
@@ -111,14 +110,15 @@ class TestBarcode(
dupe_asn = settings.SCRATCH_DIR / "barcode-39-asn-123-second.pdf" dupe_asn = settings.SCRATCH_DIR / "barcode-39-asn-123-second.pdf"
shutil.copy(test_file, dupe_asn) shutil.copy(test_file, dupe_asn)
with self.assertRaisesRegex(ConsumerError, r"ASN 123.*trash"): with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
tasks.consume_file( with self.assertRaisesRegex(ConsumerError, r"ASN 123.*trash"):
ConsumableDocument( tasks.consume_file(
source=DocumentSource.ConsumeFolder, ConsumableDocument(
original_file=dupe_asn, source=DocumentSource.ConsumeFolder,
), original_file=dupe_asn,
None, ),
) None,
)
@override_settings( @override_settings(
CONSUMER_BARCODE_TIFF_SUPPORT=True, CONSUMER_BARCODE_TIFF_SUPPORT=True,
@@ -606,7 +606,6 @@ class TestBarcodeNewConsume(
TestCase, TestCase,
): ):
@override_settings(CONSUMER_ENABLE_BARCODES=True) @override_settings(CONSUMER_ENABLE_BARCODES=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file(self) -> None: def test_consume_barcode_file(self) -> None:
""" """
GIVEN: GIVEN:
@@ -625,33 +624,34 @@ class TestBarcodeNewConsume(
overrides = DocumentMetadataOverrides(tag_ids=[1, 2, 9]) overrides = DocumentMetadataOverrides(tag_ids=[1, 2, 9])
self.assertEqual( with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
tasks.consume_file( self.assertEqual(
ConsumableDocument( tasks.consume_file(
source=DocumentSource.ConsumeFolder, ConsumableDocument(
original_file=temp_copy, source=DocumentSource.ConsumeFolder,
original_file=temp_copy,
),
overrides,
), ),
overrides, {"reason": "Barcode splitting complete!"},
), )
{"reason": "Barcode splitting complete!"}, # 2 new document consume tasks created
) self.assertEqual(self.consume_file_mock.call_count, 2)
# 2 new document consume tasks created
self.assertEqual(self.consume_file_mock.call_count, 2)
self.assertIsNotFile(temp_copy) self.assertIsNotFile(temp_copy)
# Check the split files exist # Check the split files exist
# Check the original_path is set # Check the original_path is set
# Check the source is unchanged # Check the source is unchanged
# Check the overrides are unchanged # Check the overrides are unchanged
for ( for (
new_input_doc, new_input_doc,
new_doc_overrides, new_doc_overrides,
) in self.get_all_consume_task_call_args(): ) in self.get_all_consume_task_call_args():
self.assertIsFile(new_input_doc.original_file) self.assertIsFile(new_input_doc.original_file)
self.assertEqual(new_input_doc.original_path, temp_copy) self.assertEqual(new_input_doc.original_path, temp_copy)
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder) self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertEqual(overrides, new_doc_overrides) self.assertEqual(overrides, new_doc_overrides)
class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, TestCase): class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, TestCase):
@@ -660,7 +660,7 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
reader = BarcodePlugin( reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath), ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(), DocumentMetadataOverrides(),
FakeProgressManager(filepath.name, None), DummyProgressManager(filepath.name, None),
self.dirs.scratch_dir, self.dirs.scratch_dir,
"task-id", "task-id",
) )
@@ -745,7 +745,6 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
self.assertEqual(asn, None) self.assertEqual(asn, None)
@override_settings(CONSUMER_ENABLE_ASN_BARCODE=True) @override_settings(CONSUMER_ENABLE_ASN_BARCODE=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file_asn_assignment(self) -> None: def test_consume_barcode_file_asn_assignment(self) -> None:
""" """
GIVEN: GIVEN:
@@ -763,18 +762,19 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
dst = settings.SCRATCH_DIR / "barcode-39-asn-123.pdf" dst = settings.SCRATCH_DIR / "barcode-39-asn-123.pdf"
shutil.copy(test_file, dst) shutil.copy(test_file, dst)
tasks.consume_file( with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
ConsumableDocument( tasks.consume_file(
source=DocumentSource.ConsumeFolder, ConsumableDocument(
original_file=dst, source=DocumentSource.ConsumeFolder,
), original_file=dst,
None, ),
) None,
)
document = Document.objects.first() document = Document.objects.first()
assert document is not None assert document is not None
self.assertEqual(document.archive_serial_number, 123) self.assertEqual(document.archive_serial_number, 123)
def test_scan_file_for_qrcode_without_upscale(self) -> None: def test_scan_file_for_qrcode_without_upscale(self) -> None:
""" """
@@ -819,7 +819,7 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
reader = BarcodePlugin( reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath), ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(), DocumentMetadataOverrides(),
FakeProgressManager(filepath.name, None), DummyProgressManager(filepath.name, None),
self.dirs.scratch_dir, self.dirs.scratch_dir,
"task-id", "task-id",
) )
@@ -1024,7 +1024,6 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
CELERY_TASK_ALWAYS_EAGER=True, CELERY_TASK_ALWAYS_EAGER=True,
OCR_MODE="auto", OCR_MODE="auto",
) )
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file_tag_split_and_assignment(self) -> None: def test_consume_barcode_file_tag_split_and_assignment(self) -> None:
""" """
GIVEN: GIVEN:
@@ -1043,33 +1042,34 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
dst = settings.SCRATCH_DIR / "split-by-tag-basic.pdf" dst = settings.SCRATCH_DIR / "split-by-tag-basic.pdf"
shutil.copy(test_file, dst) shutil.copy(test_file, dst)
result = tasks.consume_file( with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
ConsumableDocument( result = tasks.consume_file(
source=DocumentSource.ConsumeFolder, ConsumableDocument(
original_file=dst, source=DocumentSource.ConsumeFolder,
), original_file=dst,
None, ),
) None,
)
self.assertEqual(result, {"reason": "Barcode splitting complete!"}) self.assertEqual(result, {"reason": "Barcode splitting complete!"})
documents = Document.objects.all().order_by("id") documents = Document.objects.all().order_by("id")
self.assertEqual(documents.count(), 3) self.assertEqual(documents.count(), 3)
doc1 = documents[0] doc1 = documents[0]
self.assertEqual(doc1.tags.count(), 0) self.assertEqual(doc1.tags.count(), 0)
doc2 = documents[1] doc2 = documents[1]
self.assertEqual(doc2.tags.count(), 1) self.assertEqual(doc2.tags.count(), 1)
_tag_1 = doc2.tags.first() _tag_1 = doc2.tags.first()
assert _tag_1 is not None assert _tag_1 is not None
self.assertEqual(_tag_1.name, "invoice") self.assertEqual(_tag_1.name, "invoice")
doc3 = documents[2] doc3 = documents[2]
self.assertEqual(doc3.tags.count(), 1) self.assertEqual(doc3.tags.count(), 1)
_tag_2 = doc3.tags.first() _tag_2 = doc3.tags.first()
assert _tag_2 is not None assert _tag_2 is not None
self.assertEqual(_tag_2.name, "receipt") self.assertEqual(_tag_2.name, "receipt")
@override_settings( @override_settings(
CONSUMER_ENABLE_TAG_BARCODE=True, CONSUMER_ENABLE_TAG_BARCODE=True,
+6 -6
View File
@@ -10,6 +10,7 @@ from django.contrib.auth.models import User
from django.db import connection from django.db import connection
from django.test import TestCase from django.test import TestCase
from django.test.utils import CaptureQueriesContext from django.test.utils import CaptureQueriesContext
from guardian.shortcuts import assign_perm
from guardian.shortcuts import get_groups_with_perms from guardian.shortcuts import get_groups_with_perms
from guardian.shortcuts import get_users_with_perms from guardian.shortcuts import get_users_with_perms
@@ -22,8 +23,7 @@ from documents.models import DocumentType
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from documents.permissions import set_permissions_for_objects from documents.permissions import set_permissions_for_objects
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.permissions import grant_object
class TestBulkEdit(DirectoriesMixin, TestCase): class TestBulkEdit(DirectoriesMixin, TestCase):
@@ -440,7 +440,7 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
def test_set_permissions(self, m) -> None: def test_set_permissions(self, m) -> None:
doc_ids = [self.doc1.id, self.doc2.id, self.doc3.id] doc_ids = [self.doc1.id, self.doc2.id, self.doc3.id]
grant_object(self.group1, self.doc1, "view_document") assign_perm("view_document", self.group1, self.doc1)
permissions = { permissions = {
"view": { "view": {
@@ -482,8 +482,8 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
self.doc1.owner = self.user1 self.doc1.owner = self.user1
self.doc1.save() self.doc1.save()
grant_object(self.user1, self.doc1, "view_document") assign_perm("view_document", self.user1, self.doc1)
grant_object(self.group1, self.doc1, "view_document") assign_perm("view_document", self.group1, self.doc1)
permissions = { permissions = {
"view": { "view": {
@@ -609,7 +609,7 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
self.doc1.owner = self.user1 self.doc1.owner = self.user1
self.doc1.save() self.doc1.save()
self.user1.groups.add(self.group1) self.user1.groups.add(self.group1)
grant_object(self.group1, self.doc1, "view_document") assign_perm("view_document", self.group1, self.doc1)
bulk_edit.set_permissions( bulk_edit.set_permissions(
[self.doc1.id], [self.doc1.id],
+13 -4
View File
@@ -1,4 +1,5 @@
import pickle import pickle
import re
import warnings import warnings
from datetime import UTC from datetime import UTC
from datetime import datetime from datetime import datetime
@@ -27,13 +28,21 @@ from documents.models import DocumentType
from documents.models import MatchingModel from documents.models import MatchingModel
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from documents.tests.helpers import dummy_preprocess from documents.tests.factories import DocumentFactory
from documents.tests.factories import TagFactory
from documents.tests.utils import DirectoriesMixin
from paperless.settings import CLASSIFIER_LANGUAGES from paperless.settings import CLASSIFIER_LANGUAGES
from paperless.signed_pickle import HMAC_SIZE from paperless.signed_pickle import HMAC_SIZE
from paperless.signed_pickle import signed_pickle_dumps from paperless.signed_pickle import signed_pickle_dumps
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory def dummy_preprocess(content: str) -> str:
"""
Simpler, faster pre-processing for testing purposes
"""
content = content.lower().strip()
content = re.sub(r"\s+", " ", content)
return content
class TestClassifier(DirectoriesMixin, TestCase): class TestClassifier(DirectoriesMixin, TestCase):
+7 -8
View File
@@ -30,12 +30,11 @@ from documents.models import Tag
from documents.parsers import ParseError from documents.parsers import ParseError
from documents.plugins.helpers import ProgressStatusOptions from documents.plugins.helpers import ProgressStatusOptions
from documents.tasks import sanity_check from documents.tasks import sanity_check
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import GetConsumerMixin from documents.tests.utils import GetConsumerMixin
from paperless_mail.models import MailRule from paperless_mail.models import MailRule
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.fakes.progress import FakeProgressManager
class _BaseNewStyleParser: class _BaseNewStyleParser:
@@ -770,14 +769,14 @@ class TestConsumer(
original_modified = timezone.now() - datetime.timedelta(days=1) original_modified = timezone.now() - datetime.timedelta(days=1)
Document.objects.filter(pk=root_doc.pk).update(modified=original_modified) Document.objects.filter(pk=root_doc.pk).update(modified=original_modified)
actor = UserFactory( actor = User.objects.create_user(
username="actor", username="actor",
email="actor@example.com", email="actor@example.com",
password="password", password="password",
) )
version_file = self.get_test_file2() version_file = self.get_test_file2()
status = FakeProgressManager(version_file.name, None) status = DummyProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides( overrides = DocumentMetadataOverrides(
version_label="v2", version_label="v2",
actor_id=actor.pk, actor_id=actor.pk,
@@ -840,7 +839,7 @@ class TestConsumer(
assert root_doc is not None assert root_doc is not None
version_file = self.get_test_file2() version_file = self.get_test_file2()
status = FakeProgressManager(version_file.name, None) status = DummyProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides( overrides = DocumentMetadataOverrides(
filename="valid_pdf_version-upload", filename="valid_pdf_version-upload",
actor_id=999999, actor_id=999999,
@@ -897,7 +896,7 @@ class TestConsumer(
assert root_doc is not None assert root_doc is not None
def consume_version(version_file: Path) -> Document: def consume_version(version_file: Path) -> Document:
status = FakeProgressManager(version_file.name, None) status = DummyProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides() overrides = DocumentMetadataOverrides()
doc = ConsumableDocument( doc = ConsumableDocument(
DocumentSource.ApiUpload, DocumentSource.ApiUpload,
@@ -9,11 +9,11 @@ from django.test.utils import CaptureQueriesContext
from rest_framework import status from rest_framework import status
from documents.models import Document from documents.models import Document
from documents.tests.factories import DocumentFactory
from documents.versioning import LATEST_VERSION_CONTENT_PREFETCH_ATTR from documents.versioning import LATEST_VERSION_CONTENT_PREFETCH_ATTR
from documents.versioning import has_prefetched_effective_content from documents.versioning import has_prefetched_effective_content
from documents.versioning import latest_version_content_prefetch from documents.versioning import latest_version_content_prefetch
from documents.views import DocumentViewSet from documents.views import DocumentViewSet
from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING: if TYPE_CHECKING:
from rest_framework.test import APIClient from rest_framework.test import APIClient
+39 -49
View File
@@ -2,8 +2,8 @@ import datetime as dt
import os import os
import shutil import shutil
from pathlib import Path from pathlib import Path
from unittest import mock
import pytest
from django.test import TestCase from django.test import TestCase
from django.test import override_settings from django.test import override_settings
from pdfminer.high_level import extract_text from pdfminer.high_level import extract_text
@@ -15,26 +15,22 @@ from documents.data_models import ConsumableDocument
from documents.data_models import DocumentSource from documents.data_models import DocumentSource
from documents.double_sided import STAGING_FILE_NAME from documents.double_sided import STAGING_FILE_NAME
from documents.double_sided import TIMEOUT_MINUTES from documents.double_sided import TIMEOUT_MINUTES
from documents.tests.utils import SampleDirMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.assertions import FileSystemAssertsMixin from documents.tests.utils import DummyProgressManager
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import FileSystemAssertsMixin
@pytest.mark.usefixtures("fake_progress_manager")
@override_settings( @override_settings(
CONSUMER_RECURSIVE=True, CONSUMER_RECURSIVE=True,
CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=True, CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=True,
) )
class TestDoubleSided( class TestDoubleSided(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
DirectoriesMixin, SAMPLE_DIR = Path(__file__).parent / "samples"
FileSystemAssertsMixin,
SampleDirMixin,
TestCase,
):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.double_sided_dir = self.dirs.consumption_dir / "double-sided" self.dirs.double_sided_dir = self.dirs.consumption_dir / "double-sided"
self.double_sided_dir.mkdir() self.dirs.double_sided_dir.mkdir()
self.staging_file = self.dirs.scratch_dir / STAGING_FILE_NAME self.staging_file = self.dirs.scratch_dir / STAGING_FILE_NAME
def consume_file(self, srcname, dstname: str | Path = "foo.pdf"): def consume_file(self, srcname, dstname: str | Path = "foo.pdf"):
@@ -43,16 +39,20 @@ class TestDoubleSided(
destination file does not exist afterwards destination file does not exist afterwards
""" """
src = self.SAMPLE_DIR / srcname src = self.SAMPLE_DIR / srcname
dst = self.double_sided_dir / dstname dst = self.dirs.double_sided_dir / dstname
dst.parent.mkdir(parents=True, exist_ok=True) dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy(src, dst) shutil.copy(src, dst)
msg = tasks.consume_file( with mock.patch(
ConsumableDocument( "documents.tasks.ProgressManager",
source=DocumentSource.ConsumeFolder, DummyProgressManager,
original_file=dst, ):
), msg = tasks.consume_file(
None, ConsumableDocument(
) source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
self.assertIsNotFile(dst) self.assertIsNotFile(dst)
return msg return msg
@@ -214,41 +214,31 @@ class TestDoubleSided(
self.assertIsNotFile(self.staging_file) self.assertIsNotFile(self.staging_file)
self.assertIsInstance(msg.get("document_id"), int) self.assertIsInstance(msg.get("document_id"), int)
def test_consume_double_sided_in_nested_dir(self) -> None: def test_subdirectory_upload(self) -> None:
""" """
GIVEN: GIVEN:
- A staging file exists - A staging file exists
WHEN: WHEN:
- A file is uploaded into foo/bar/double-sided - A file gets uploaded into foo/bar/double-sided
or double-sided/foo/bar
THEN: THEN:
- The collated file is placed into foo/bar - The collated file gets put into foo/bar
""" """
self.create_staging_file() # TODO: parameterize this instead
self.consume_file( for path in [
"double-sided-odd.pdf", Path("foo") / "bar" / "double-sided",
Path("foo") / "bar" / "double-sided" / "foo.pdf", Path("double-sided") / "foo" / "bar",
) ]:
self.assertIsFile( with self.subTest(path=str(path)):
self.dirs.consumption_dir / "foo" / "bar" / "foo-collated.pdf", # Ensure we get fresh directories for each run
) self.tearDown()
self.setUp()
def test_consume_double_sided_with_nested_subdir(self) -> None: self.create_staging_file()
""" self.consume_file("double-sided-odd.pdf", Path(path) / "foo.pdf")
GIVEN: self.assertIsFile(
- A staging file exists self.dirs.consumption_dir / "foo" / "bar" / "foo-collated.pdf",
WHEN: )
- A file is uploaded into double-sided/foo/bar
THEN:
- The collated file is placed into foo/bar
"""
self.create_staging_file()
self.consume_file(
"double-sided-odd.pdf",
Path("double-sided") / "foo" / "bar" / "foo.pdf",
)
self.assertIsFile(
self.dirs.consumption_dir / "foo" / "bar" / "foo-collated.pdf",
)
@override_settings(CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=False) @override_settings(CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=False)
def test_disabled_double_sided_dir_upload(self) -> None: def test_disabled_double_sided_dir_upload(self) -> None:
+5 -5
View File
@@ -8,6 +8,7 @@ from unittest import mock
import pytest import pytest
from auditlog.context import disable_auditlog from auditlog.context import disable_auditlog
from django.conf import settings from django.conf import settings
from django.contrib.auth.models import User
from django.db import DatabaseError from django.db import DatabaseError
from django.db import connection from django.db import connection
from django.test import TestCase from django.test import TestCase
@@ -29,10 +30,9 @@ from documents.models import DocumentType
from documents.models import StoragePath from documents.models import StoragePath
from documents.serialisers import DocumentSerializer from documents.serialisers import DocumentSerializer
from documents.tasks import empty_trash from documents.tasks import empty_trash
from paperless_testing.assertions import FileSystemAssertsMixin from documents.tests.factories import DocumentFactory
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import DocumentFactory from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.factories import UserFactory
class TestFileHandling(DirectoriesMixin, FileSystemAssertsMixin, TestCase): class TestFileHandling(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
@@ -1323,7 +1323,7 @@ class TestFilenameGeneration(DirectoriesMixin, TestCase):
- Document without owner returns "none" - Document without owner returns "none"
""" """
u1 = UserFactory(username="user1") u1 = User.objects.create_user("user1")
owned_doc = Document.objects.create( owned_doc = Document.objects.create(
title="The Title", title="The Title",
+4 -4
View File
@@ -13,10 +13,10 @@ import pytest
from documents.file_handling import generate_filename from documents.file_handling import generate_filename
from documents.models import CustomField from documents.models import CustomField
from documents.models import CustomFieldInstance from documents.models import CustomFieldInstance
from paperless_testing.factories import CorrespondentFactory from documents.tests.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import StoragePathFactory from documents.tests.factories import StoragePathFactory
from paperless_testing.factories import TagFactory from documents.tests.factories import TagFactory
@pytest.mark.django_db @pytest.mark.django_db
+2 -2
View File
@@ -20,8 +20,8 @@ if TYPE_CHECKING:
from documents.file_handling import generate_filename from documents.file_handling import generate_filename
from documents.models import Document from documents.models import Document
from documents.tasks import update_document_content_maybe_archive_file from documents.tasks import update_document_content_maybe_archive_file
from paperless_testing.assertions import FileSystemAssertsMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import FileSystemAssertsMixin
sample_file: Path = Path(__file__).parent / "samples" / "simple.pdf" sample_file: Path = Path(__file__).parent / "samples" / "simple.pdf"
+10 -11
View File
@@ -26,6 +26,7 @@ from django.test import override_settings
from django.utils import timezone from django.utils import timezone
from guardian.models import GroupObjectPermission from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission from guardian.models import UserObjectPermission
from guardian.shortcuts import assign_perm
from documents.management.commands import document_exporter from documents.management.commands import document_exporter
from documents.models import Correspondent from documents.models import Correspondent
@@ -45,12 +46,11 @@ from documents.models import WorkflowTrigger
from documents.sanity_checker import check_sanity from documents.sanity_checker import check_sanity
from documents.settings import EXPORTER_FILE_NAME from documents.settings import EXPORTER_FILE_NAME
from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin from documents.tests.utils import SampleDirMixin
from documents.tests.utils import paperless_environment
from paperless_mail.models import MailAccount from paperless_mail.models import MailAccount
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.dirs import paperless_environment
from paperless_testing.permissions import grant_object
@pytest.mark.management @pytest.mark.management
@@ -105,8 +105,8 @@ class TestExportImport(
user=self.user, user=self.user,
) )
grant_object(self.user2, self.d2, "view_document") assign_perm("view_document", self.user2, self.d2)
grant_object(self.group1, self.d3, "view_document") assign_perm("view_document", self.group1, self.d3)
self.t1 = Tag.objects.create(name="t") self.t1 = Tag.objects.create(name="t")
self.dt1 = DocumentType.objects.create(name="dt") self.dt1 = DocumentType.objects.create(name="dt")
@@ -677,13 +677,12 @@ class TestExportImport(
THEN: THEN:
- Error is raised - Error is raised
""" """
with tempfile.TemporaryDirectory() as tmp_dir: args = ["document_exporter", "/tmp/foo/bar"]
args = ["document_exporter", str(Path(tmp_dir) / "does-not-exist")]
with self.assertRaises(CommandError) as e: with self.assertRaises(CommandError) as e:
call_command(*args, skip_checks=True) call_command(*args, skip_checks=True)
self.assertEqual("That path doesn't exist", str(e.exception)) self.assertEqual("That path doesn't exist", str(e.exception))
def test_export_target_exists_but_is_file(self) -> None: def test_export_target_exists_but_is_file(self) -> None:
""" """
+9 -9
View File
@@ -8,7 +8,7 @@ from django.core.management import call_command
from django.test import TestCase from django.test import TestCase
from documents.models import Document from documents.models import Document
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
@pytest.mark.management @pytest.mark.management
@@ -123,14 +123,14 @@ class TestFuzzyMatchCommand(TestCase):
- Output contains clickable links to the documents instead of titles - Output contains clickable links to the documents instead of titles
""" """
# Content similarity is 86.667 # Content similarity is 86.667
doc1 = Document.objects.create( Document.objects.create(
checksum="BEEFCAFE", checksum="BEEFCAFE",
title="A", title="A",
content="first document scanned by bob", content="first document scanned by bob",
mime_type="application/pdf", mime_type="application/pdf",
filename="test.pdf", filename="test.pdf",
) )
doc2 = Document.objects.create( Document.objects.create(
checksum="DEADBEAF", checksum="DEADBEAF",
title="A", title="A",
content="first document scanned by alice", content="first document scanned by alice",
@@ -145,8 +145,8 @@ class TestFuzzyMatchCommand(TestCase):
"http://localhost:8000", "http://localhost:8000",
) )
self.assertIn("Found 1 matching pair(s)", stdout) self.assertIn("Found 1 matching pair(s)", stdout)
self.assertIn(f"http://localhost:8000/documents/{doc1.pk}/details", stdout) self.assertIn("http://localhost:8000/documents/1/details", stdout)
self.assertIn(f"http://localhost:8000/documents/{doc2.pk}/details", stdout) self.assertIn("http://localhost:8000/documents/2/details", stdout)
def test_with_3_matches(self) -> None: def test_with_3_matches(self) -> None:
""" """
@@ -198,14 +198,14 @@ class TestFuzzyMatchCommand(TestCase):
- Documents 1 and 2 remain - Documents 1 and 2 remain
""" """
# Content similarity is 86.667 # Content similarity is 86.667
doc1 = Document.objects.create( Document.objects.create(
checksum="BEEFCAFE", checksum="BEEFCAFE",
title="A", title="A",
content="first document scanned by bob", content="first document scanned by bob",
mime_type="application/pdf", mime_type="application/pdf",
filename="test.pdf", filename="test.pdf",
) )
doc2 = Document.objects.create( Document.objects.create(
checksum="DEADBEAF", checksum="DEADBEAF",
title="A", title="A",
content="second document scanned by alice", content="second document scanned by alice",
@@ -235,8 +235,8 @@ class TestFuzzyMatchCommand(TestCase):
self.assertIn("Deleting 1 document(s)", stdout) self.assertIn("Deleting 1 document(s)", stdout)
self.assertEqual(Document.objects.count(), 2) self.assertEqual(Document.objects.count(), 2)
self.assertIsNotNone(Document.objects.get(pk=doc1.pk)) self.assertIsNotNone(Document.objects.get(pk=1))
self.assertIsNotNone(Document.objects.get(pk=doc2.pk)) self.assertIsNotNone(Document.objects.get(pk=2))
def test_document_deletion_cancelled(self) -> None: def test_document_deletion_cancelled(self) -> None:
""" """
@@ -15,9 +15,9 @@ from documents.management.commands.document_importer import _deserialize_record
from documents.models import Document from documents.models import Document
from documents.settings import EXPORTER_ARCHIVE_NAME from documents.settings import EXPORTER_ARCHIVE_NAME
from documents.settings import EXPORTER_FILE_NAME from documents.settings import EXPORTER_FILE_NAME
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin from documents.tests.utils import SampleDirMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@pytest.mark.management @pytest.mark.management
@@ -14,12 +14,12 @@ from documents.models import DocumentType
from documents.models import MatchingModel from documents.models import MatchingModel
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from paperless_testing.dirs import DirectoriesMixin from documents.tests.factories import CorrespondentFactory
from paperless_testing.factories import CorrespondentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentTypeFactory
from paperless_testing.factories import DocumentTypeFactory from documents.tests.factories import StoragePathFactory
from paperless_testing.factories import StoragePathFactory from documents.tests.factories import TagFactory
from paperless_testing.factories import TagFactory from documents.tests.utils import DirectoriesMixin
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Module-level type aliases # Module-level type aliases
@@ -7,14 +7,14 @@ from django.contrib.auth.models import User
from django.core.management import call_command from django.core.management import call_command
from django.test import TestCase from django.test import TestCase
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
@pytest.mark.management @pytest.mark.management
class TestManageSuperUser(DirectoriesMixin, TestCase): class TestManageSuperUser(DirectoriesMixin, TestCase):
def call_command(self, environ): def call_command(self, environ):
out = StringIO() out = StringIO()
with mock.patch.dict(os.environ, environ, clear=True): with mock.patch.dict(os.environ, environ):
call_command( call_command(
"manage_superuser", "manage_superuser",
"--no-color", "--no-color",
@@ -9,8 +9,8 @@ from django.test import TestCase
from documents.management.commands.document_thumbnails import _process_document from documents.management.commands.document_thumbnails import _process_document
from documents.models import Document from documents.models import Document
from documents.parsers import get_default_thumbnail from documents.parsers import get_default_thumbnail
from paperless_testing.assertions import FileSystemAssertsMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import FileSystemAssertsMixin
@pytest.mark.management @pytest.mark.management
+4 -4
View File
@@ -7,10 +7,10 @@ from documents import matching
from documents.models import Document from documents.models import Document
from documents.models import MatchingModel from documents.models import MatchingModel
from documents.signals import document_consumption_finished from documents.signals import document_consumption_finished
from paperless_testing.factories import CorrespondentFactory from documents.tests.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory from documents.tests.factories import DocumentTypeFactory
from paperless_testing.factories import TagFactory from documents.tests.factories import TagFactory
@pytest.fixture( @pytest.fixture(
@@ -2,6 +2,8 @@ import json
from unittest import mock from unittest import mock
from auditlog.models import LogEntry from auditlog.models import LogEntry
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from django.test import TestCase from django.test import TestCase
from rest_framework import status from rest_framework import status
@@ -10,8 +12,6 @@ from rest_framework.test import APITestCase
from documents.bulk_edit import merge_as_versions from documents.bulk_edit import merge_as_versions
from documents.models import Document from documents.models import Document
from documents.serialisers import MergeDocumentsAsVersionsSerializer from documents.serialisers import MergeDocumentsAsVersionsSerializer
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestMergeDocumentsAsVersionsSerializer(TestCase): class TestMergeDocumentsAsVersionsSerializer(TestCase):
@@ -294,7 +294,7 @@ class TestMergeDocumentsAsVersions(TestCase):
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async") @mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
@mock.patch("documents.search.get_backend") @mock.patch("documents.search.get_backend")
def test_writes_audit_log_entry(self, *_mocks) -> None: def test_writes_audit_log_entry(self, *_mocks) -> None:
user = UserFactory(username="merger") user = User.objects.create_user(username="merger")
root = Document.objects.create(checksum="A", title="Root") root = Document.objects.create(checksum="A", title="Root")
source = Document.objects.create(checksum="B", title="Source") source = Document.objects.create(checksum="B", title="Source")
LogEntry.objects.all().delete() LogEntry.objects.all().delete()
@@ -335,8 +335,12 @@ class TestMergeDocumentsAsVersions(TestCase):
class TestMergeDocumentsAsVersionsAPI(APITestCase): class TestMergeDocumentsAsVersionsAPI(APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
self.user = UserFactory(username="user") self.user = User.objects.create_user(username="user")
grant_global(self.user, "change_document", "view_document", "delete_document") self.user.user_permissions.add(
Permission.objects.get(codename="change_document"),
Permission.objects.get(codename="view_document"),
Permission.objects.get(codename="delete_document"),
)
self.doc1 = Document.objects.create( self.doc1 = Document.objects.create(
checksum="A", checksum="A",
title="A", title="A",
@@ -378,7 +382,7 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
@mock.patch("documents.views.bulk_edit.merge_as_versions") @mock.patch("documents.views.bulk_edit.merge_as_versions")
def test_requires_change_permission(self, merge_mock) -> None: def test_requires_change_permission(self, merge_mock) -> None:
merge_mock.__name__ = "merge_as_versions" merge_mock.__name__ = "merge_as_versions"
user = UserFactory(username="no-change") user = User.objects.create_user(username="no-change")
self.doc1.owner = user self.doc1.owner = user
self.doc1.save() self.doc1.save()
self.doc2.owner = user self.doc2.owner = user
@@ -401,8 +405,11 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
def test_requires_delete_permission(self, merge_mock) -> None: def test_requires_delete_permission(self, merge_mock) -> None:
merge_mock.__name__ = "merge_as_versions" merge_mock.__name__ = "merge_as_versions"
# Owns them and may change them, but may not make them stop being documents # Owns them and may change them, but may not make them stop being documents
user = UserFactory(username="no-delete") user = User.objects.create_user(username="no-delete")
grant_global(user, "change_document", "view_document") user.user_permissions.add(
Permission.objects.get(codename="change_document"),
Permission.objects.get(codename="view_document"),
)
for doc in (self.doc1, self.doc2): for doc in (self.doc1, self.doc2):
doc.owner = user doc.owner = user
doc.save() doc.save()
@@ -1,4 +1,4 @@
from paperless_testing.migrations import TestMigrations from documents.tests.utils import TestMigrations
SAVED_VIEWS_KEY = "saved_views" SAVED_VIEWS_KEY = "saved_views"
DASHBOARD_VIEWS_VISIBLE_IDS_KEY = "dashboard_views_visible_ids" DASHBOARD_VIEWS_VISIBLE_IDS_KEY = "dashboard_views_visible_ids"
@@ -7,7 +7,7 @@ from django.conf import settings
from django.db import connection from django.db import connection
from django.test import override_settings from django.test import override_settings
from paperless_testing.migrations import TestMigrations from documents.tests.utils import TestMigrations
def _sha256(data: bytes) -> str: def _sha256(data: bytes) -> str:
@@ -1,4 +1,4 @@
from paperless_testing.migrations import TestMigrations from documents.tests.utils import TestMigrations
class TestMigrateShareLinkBundlePermissions(TestMigrations): class TestMigrateShareLinkBundlePermissions(TestMigrations):
+2 -2
View File
@@ -2,8 +2,8 @@ import pytest
from documents.models import Correspondent from documents.models import Correspondent
from documents.models import Document from documents.models import Document
from paperless_testing.factories import CorrespondentFactory from documents.tests.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
@pytest.mark.django_db @pytest.mark.django_db
@@ -1,13 +1,15 @@
from __future__ import annotations from __future__ import annotations
from http import HTTPStatus from http import HTTPStatus
from typing import TYPE_CHECKING
from unittest.mock import patch from unittest.mock import patch
import pytest import pytest
from django.contrib.auth.models import AnonymousUser from django.contrib.auth.models import AnonymousUser
from django.contrib.auth.models import Group from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings from django.test import override_settings
from guardian.shortcuts import assign_perm
from rest_framework.test import APIClient from rest_framework.test import APIClient
from documents.matching import match_correspondents from documents.matching import match_correspondents
@@ -22,17 +24,11 @@ from documents.permissions import permitted_document_ids
from documents.permissions import permitted_object_ids from documents.permissions import permitted_object_ids
from documents.permissions import restrict_queryset_to_visible from documents.permissions import restrict_queryset_to_visible
from documents.serialisers import _get_viewable_duplicates from documents.serialisers import _get_viewable_duplicates
from paperless_testing.factories import CorrespondentFactory from documents.tests.factories import CorrespondentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentTypeFactory from documents.tests.factories import DocumentTypeFactory
from paperless_testing.factories import StoragePathFactory from documents.tests.factories import StoragePathFactory
from paperless_testing.factories import TagFactory from documents.tests.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
if TYPE_CHECKING:
from paperless_testing.dirs import PaperlessDirs
def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden): def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden):
@@ -51,8 +47,8 @@ def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden
@pytest.mark.django_db @pytest.mark.django_db
class TestPermittedDocumentIdsSecurity: class TestPermittedDocumentIdsSecurity:
def test_owner_sees_own_document(self): def test_owner_sees_own_document(self):
user = UserFactory(username="alice") user = User.objects.create_user(username="alice")
stranger = UserFactory(username="mallory") stranger = User.objects.create_user(username="mallory")
owned = DocumentFactory(owner=user) owned = DocumentFactory(owner=user)
strangers_doc = DocumentFactory(owner=stranger) strangers_doc = DocumentFactory(owner=stranger)
@@ -65,7 +61,7 @@ class TestPermittedDocumentIdsSecurity:
) )
def test_unowned_document_visible_to_everyone(self): def test_unowned_document_visible_to_everyone(self):
user = UserFactory(username="alice") user = User.objects.create_user(username="alice")
unowned = DocumentFactory(owner=None) unowned = DocumentFactory(owner=None)
assert_visible_document_ids( assert_visible_document_ids(
@@ -75,12 +71,12 @@ class TestPermittedDocumentIdsSecurity:
) )
def test_explicit_user_permission_grants_visibility(self): def test_explicit_user_permission_grants_visibility(self):
grantee = UserFactory(username="alice") grantee = User.objects.create_user(username="alice")
stranger = UserFactory(username="mallory") stranger = User.objects.create_user(username="mallory")
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
shared = DocumentFactory(owner=owner) shared = DocumentFactory(owner=owner)
not_shared = DocumentFactory(owner=owner) not_shared = DocumentFactory(owner=owner)
grant_object(grantee, shared, "view_document") assign_perm("view_document", grantee, shared)
assert_visible_document_ids( assert_visible_document_ids(
permitted_document_ids(grantee), permitted_document_ids(grantee),
@@ -94,13 +90,13 @@ class TestPermittedDocumentIdsSecurity:
) )
def test_explicit_group_permission_grants_visibility_to_members_only(self): def test_explicit_group_permission_grants_visibility_to_members_only(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
member = UserFactory(username="member") member = User.objects.create_user(username="member")
non_member = UserFactory(username="non_member") non_member = User.objects.create_user(username="non_member")
group = Group.objects.create(name="finance") group = Group.objects.create(name="finance")
member.groups.add(group) member.groups.add(group)
shared = DocumentFactory(owner=owner) shared = DocumentFactory(owner=owner)
grant_object(group, shared, "view_document") assign_perm("view_document", group, shared)
assert_visible_document_ids( assert_visible_document_ids(
permitted_document_ids(member), permitted_document_ids(member),
@@ -114,7 +110,7 @@ class TestPermittedDocumentIdsSecurity:
) )
def test_soft_deleted_document_excluded_by_default(self): def test_soft_deleted_document_excluded_by_default(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
doc.delete() # soft delete doc.delete() # soft delete
doc.refresh_from_db() doc.refresh_from_db()
@@ -130,8 +126,8 @@ class TestPermittedDocumentIdsSecurity:
) )
def test_superuser_sees_everything_including_no_perm_documents(self): def test_superuser_sees_everything_including_no_perm_documents(self):
superuser = UserFactory(username="root", superuser=True) superuser = User.objects.create_superuser(username="root")
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
assert_visible_document_ids( assert_visible_document_ids(
@@ -141,7 +137,7 @@ class TestPermittedDocumentIdsSecurity:
) )
def test_anonymous_user_sees_only_unowned_documents(self): def test_anonymous_user_sees_only_unowned_documents(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
owned = DocumentFactory(owner=owner) owned = DocumentFactory(owner=owner)
unowned = DocumentFactory(owner=None) unowned = DocumentFactory(owner=None)
@@ -155,7 +151,7 @@ class TestPermittedDocumentIdsSecurity:
@pytest.mark.django_db @pytest.mark.django_db
class TestPermittedDocumentIdsIncludeDeleted: class TestPermittedDocumentIdsIncludeDeleted:
def test_include_deleted_true_reveals_soft_deleted_owned_document(self): def test_include_deleted_true_reveals_soft_deleted_owned_document(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
doc.delete() doc.delete()
@@ -166,8 +162,8 @@ class TestPermittedDocumentIdsIncludeDeleted:
) )
def test_include_deleted_true_still_respects_permission_boundary(self): def test_include_deleted_true_still_respects_permission_boundary(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
stranger = UserFactory(username="mallory") stranger = User.objects.create_user(username="mallory")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
doc.delete() doc.delete()
@@ -195,12 +191,14 @@ class TestAiChatAllDocumentsPermissionBoundary:
def test_chat_all_documents_excludes_unshared_document(self, mock_stream_chat): def test_chat_all_documents_excludes_unshared_document(self, mock_stream_chat):
mock_stream_chat.return_value = iter([b"data"]) mock_stream_chat.return_value = iter([b"data"])
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
asker = UserFactory(username="asker") asker = User.objects.create_user(username="asker")
grant_global(asker, "view_document") asker.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
shared = DocumentFactory(owner=owner) shared = DocumentFactory(owner=owner)
not_shared = DocumentFactory(owner=owner) not_shared = DocumentFactory(owner=owner)
grant_object(asker, shared, "view_document") assign_perm("view_document", asker, shared)
client = APIClient() client = APIClient()
client.force_authenticate(user=asker) client.force_authenticate(user=asker)
@@ -221,13 +219,13 @@ class TestAiChatAllDocumentsPermissionBoundary:
@pytest.mark.django_db @pytest.mark.django_db
class TestDuplicateDocumentsPermissionBoundary: class TestDuplicateDocumentsPermissionBoundary:
def test_get_viewable_duplicates_includes_soft_deleted_but_respects_perms(self): def test_get_viewable_duplicates_includes_soft_deleted_but_respects_perms(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
stranger = UserFactory(username="mallory") stranger = User.objects.create_user(username="mallory")
original = DocumentFactory(owner=owner, checksum="dupe-checksum") original = DocumentFactory(owner=owner, checksum="dupe-checksum")
dup_visible = DocumentFactory(owner=owner, checksum="dupe-checksum") dup_visible = DocumentFactory(owner=owner, checksum="dupe-checksum")
dup_hidden = DocumentFactory(owner=owner, checksum="dupe-checksum") dup_hidden = DocumentFactory(owner=owner, checksum="dupe-checksum")
dup_hidden.delete() # soft delete, should still be found (include_deleted=True) dup_hidden.delete() # soft delete, should still be found (include_deleted=True)
grant_object(stranger, dup_visible, "view_document") assign_perm("view_document", stranger, dup_visible)
result_owner = _get_viewable_duplicates(original, owner) result_owner = _get_viewable_duplicates(original, owner)
assert {d.pk for d in result_owner} == {dup_visible.pk, dup_hidden.pk} assert {d.pk for d in result_owner} == {dup_visible.pk, dup_hidden.pk}
@@ -239,13 +237,13 @@ class TestDuplicateDocumentsPermissionBoundary:
@pytest.mark.django_db @pytest.mark.django_db
class TestPermittedDocumentIdsArbitraryPermission: class TestPermittedDocumentIdsArbitraryPermission:
def test_change_document_permission_is_distinct_from_view(self): def test_change_document_permission_is_distinct_from_view(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
viewer_only = UserFactory(username="viewer") viewer_only = User.objects.create_user(username="viewer")
editor = UserFactory(username="editor") editor = User.objects.create_user(username="editor")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
grant_object(viewer_only, doc, "view_document") assign_perm("view_document", viewer_only, doc)
grant_object(editor, doc, "change_document") assign_perm("change_document", editor, doc)
grant_object(editor, doc, "view_document") assign_perm("view_document", editor, doc)
assert_visible_document_ids( assert_visible_document_ids(
permitted_document_ids(editor, perm="change_document"), permitted_document_ids(editor, perm="change_document"),
@@ -259,10 +257,10 @@ class TestPermittedDocumentIdsArbitraryPermission:
) )
def test_qualified_permission_string_is_normalized_to_codename(self): def test_qualified_permission_string_is_normalized_to_codename(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
editor = UserFactory(username="editor") editor = User.objects.create_user(username="editor")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
grant_object(editor, doc, "change_document") assign_perm("change_document", editor, doc)
assert_visible_document_ids( assert_visible_document_ids(
permitted_document_ids(editor, perm="documents.change_document"), permitted_document_ids(editor, perm="documents.change_document"),
@@ -271,11 +269,11 @@ class TestPermittedDocumentIdsArbitraryPermission:
) )
def test_delete_permission_with_include_deleted_for_trash_restore(self): def test_delete_permission_with_include_deleted_for_trash_restore(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
stranger = UserFactory(username="mallory") stranger = User.objects.create_user(username="mallory")
view_only = UserFactory(username="viewer") view_only = User.objects.create_user(username="viewer")
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
grant_object(view_only, doc, "view_document") assign_perm("view_document", view_only, doc)
doc.delete() doc.delete()
assert_visible_document_ids( assert_visible_document_ids(
@@ -309,9 +307,11 @@ class TestEmailDocumentPermissionBoundary:
self, self,
rest_api_client, rest_api_client,
): ):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
grant_global(requester, "view_document") requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
hidden = DocumentFactory(owner=owner) hidden = DocumentFactory(owner=owner)
@@ -339,17 +339,19 @@ class TestBulkEditChangePermissionBoundary:
# permitted document must not be partially applied just because it # permitted document must not be partially applied just because it
# was bundled with a forbidden one, proving the endpoint checks # was bundled with a forbidden one, proving the endpoint checks
# every document in the batch rather than only the first/last. # every document in the batch rather than only the first/last.
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
# grant the global change_document permission so the object-level # grant the global change_document permission so the object-level
# check (not the global has_perm check) is what's under test # check (not the global has_perm check) is what's under test
grant_global(requester, "change_document") requester.user_permissions.add(
Permission.objects.get(codename="change_document"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
changeable = DocumentFactory(owner=owner) changeable = DocumentFactory(owner=owner)
grant_object(requester, changeable, "view_document") assign_perm("view_document", requester, changeable)
grant_object(requester, changeable, "change_document") # fully permitted assign_perm("change_document", requester, changeable) # fully permitted
target = DocumentFactory(owner=owner) target = DocumentFactory(owner=owner)
grant_object(requester, target, "view_document") # view only, NOT change assign_perm("view_document", requester, target) # view only, NOT change
response = rest_api_client.post( response = rest_api_client.post(
"/api/documents/bulk_edit/", "/api/documents/bulk_edit/",
@@ -367,14 +369,15 @@ class TestBulkEditChangePermissionBoundary:
class TestBulkDownloadPermissionChecksRootDocument: class TestBulkDownloadPermissionChecksRootDocument:
def test_download_requires_global_view_permission( def test_download_requires_global_view_permission(
self, self,
rest_api_client: APIClient, rest_api_client,
paperless_dirs: PaperlessDirs, paperless_dirs,
) -> None: _media_settings,
owner = UserFactory(username="owner") ):
requester = UserFactory(username="requester") owner = User.objects.create_user(username="owner")
requester = User.objects.create_user(username="requester")
root = DocumentFactory(owner=owner) root = DocumentFactory(owner=owner)
root.source_path.write_bytes(b"%PDF-1.4 test") root.source_path.write_bytes(b"%PDF-1.4 test")
grant_object(requester, root, "view_document") assign_perm("view_document", requester, root)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
response = rest_api_client.post( response = rest_api_client.post(
@@ -387,18 +390,21 @@ class TestBulkDownloadPermissionChecksRootDocument:
def test_permission_checked_on_root_not_on_version( def test_permission_checked_on_root_not_on_version(
self, self,
rest_api_client: APIClient, rest_api_client,
paperless_dirs: PaperlessDirs, paperless_dirs,
) -> None: _media_settings,
owner = UserFactory(username="owner") ):
requester = UserFactory(username="requester") owner = User.objects.create_user(username="owner")
grant_global(requester, "view_document") requester = User.objects.create_user(username="requester")
requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
root = DocumentFactory(owner=owner) root = DocumentFactory(owner=owner)
# a version of root that the requester has NOT been individually granted # a version of root that the requester has NOT been individually granted
version = DocumentFactory(owner=owner, root_document=root, version_index=1) version = DocumentFactory(owner=owner, root_document=root, version_index=1)
version.source_path.write_bytes(b"%PDF-1.4 test") version.source_path.write_bytes(b"%PDF-1.4 test")
grant_object(requester, root, "view_document") # granted on ROOT only assign_perm("view_document", requester, root) # granted on ROOT only
response = rest_api_client.post( response = rest_api_client.post(
"/api/documents/bulk_download/", "/api/documents/bulk_download/",
@@ -416,9 +422,11 @@ class TestBulkDownloadPermissionChecksRootDocument:
# root-or-version bug; a user with no grant at all (the old # root-or-version bug; a user with no grant at all (the old
# `stranger` case) can't tell the two apart, since they're denied # `stranger` case) can't tell the two apart, since they're denied
# either way. # either way.
version_only_grantee = UserFactory(username="version_only_grantee") version_only_grantee = User.objects.create_user(username="version_only_grantee")
grant_global(version_only_grantee, "view_document") version_only_grantee.user_permissions.add(
grant_object(version_only_grantee, version, "view_document") Permission.objects.get(codename="view_document"),
)
assign_perm("view_document", version_only_grantee, version)
rest_api_client.force_authenticate(user=version_only_grantee) rest_api_client.force_authenticate(user=version_only_grantee)
response = rest_api_client.post( response = rest_api_client.post(
"/api/documents/bulk_download/", "/api/documents/bulk_download/",
@@ -430,53 +438,6 @@ class TestBulkDownloadPermissionChecksRootDocument:
) # version-only grant must not substitute for root permission ) # version-only grant must not substitute for root permission
@pytest.mark.django_db
class TestDocumentOperationPermissionChecksRootDocument:
@pytest.mark.parametrize(
("endpoint", "payload"),
[
pytest.param("/api/documents/merge/", {}, id="merge"),
pytest.param("/api/documents/rotate/", {"degrees": 90}, id="rotate"),
],
)
@pytest.mark.parametrize("version_owner", ["none", "requester"])
def test_version_operation_acts_on_root(
self,
rest_api_client: APIClient,
endpoint: str,
payload: dict,
version_owner: str,
) -> None:
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "change_document")
grant_global(requester, "add_document")
rest_api_client.force_authenticate(user=requester)
root = DocumentFactory(owner=owner)
# A version whose owner went stale, e.g. created before the root changed hands
version = DocumentFactory(
owner=requester if version_owner == "requester" else None,
root_document=root,
version_index=1,
)
with (
patch("documents.views.bulk_edit.merge") as mock_merge,
patch("documents.views.bulk_edit.rotate") as mock_rotate,
):
mock_merge.__name__ = "merge"
mock_rotate.__name__ = "rotate"
response = rest_api_client.post(
endpoint,
{"documents": [version.pk], **payload},
format="json",
)
assert response.status_code == HTTPStatus.FORBIDDEN
mock_merge.assert_not_called()
mock_rotate.assert_not_called()
@pytest.mark.django_db @pytest.mark.django_db
@pytest.mark.usefixtures("_search_index") @pytest.mark.usefixtures("_search_index")
class TestTrashRestorePermissionBoundary: class TestTrashRestorePermissionBoundary:
@@ -484,12 +445,14 @@ class TestTrashRestorePermissionBoundary:
self, self,
rest_api_client, rest_api_client,
): ):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
grant_global(requester, "delete_document") requester.user_permissions.add(
Permission.objects.get(codename="delete_document"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
grant_object(requester, doc, "view_document") # view only, NOT delete assign_perm("view_document", requester, doc) # view only, NOT delete
doc.delete() doc.delete()
response = rest_api_client.post( response = rest_api_client.post(
@@ -503,12 +466,14 @@ class TestTrashRestorePermissionBoundary:
self, self,
rest_api_client, rest_api_client,
): ):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
grant_global(requester, "delete_document") requester.user_permissions.add(
Permission.objects.get(codename="delete_document"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
grant_object(requester, doc, "delete_document") assign_perm("delete_document", requester, doc)
doc.delete() doc.delete()
response = rest_api_client.post( response = rest_api_client.post(
@@ -519,11 +484,11 @@ class TestTrashRestorePermissionBoundary:
assert response.status_code == HTTPStatus.OK assert response.status_code == HTTPStatus.OK
def test_restore_requires_global_delete_permission(self, rest_api_client): def test_restore_requires_global_delete_permission(self, rest_api_client):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
grant_object(requester, doc, "delete_document") assign_perm("delete_document", requester, doc)
doc.delete() doc.delete()
response = rest_api_client.post( response = rest_api_client.post(
@@ -548,12 +513,14 @@ class TestTrashViewExcludesExplicitlyGrantedDocuments:
""" """
def test_explicit_grant_does_not_leak_trashed_document(self, rest_api_client): def test_explicit_grant_does_not_leak_trashed_document(self, rest_api_client):
owner = UserFactory(username="trash_owner") owner = User.objects.create_user(username="trash_owner")
grantee = UserFactory(username="trash_grantee") grantee = User.objects.create_user(username="trash_grantee")
grant_global(grantee, "view_document") grantee.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
doc = DocumentFactory(owner=owner) doc = DocumentFactory(owner=owner)
doc.delete() # soft delete doc.delete() # soft delete
grant_object(grantee, doc, "view_document") assign_perm("view_document", grantee, doc)
rest_api_client.force_authenticate(user=grantee) rest_api_client.force_authenticate(user=grantee)
response = rest_api_client.get("/api/trash/") response = rest_api_client.get("/api/trash/")
@@ -575,8 +542,8 @@ class TestTrashViewExcludesExplicitlyGrantedDocuments:
) )
class TestPermittedObjectIdsGenericModels: class TestPermittedObjectIdsGenericModels:
def test_owner_sees_own_object(self, model, factory, perm): def test_owner_sees_own_object(self, model, factory, perm):
owner = UserFactory(username=f"owner_{model.__name__}") owner = User.objects.create_user(username=f"owner_{model.__name__}")
stranger = UserFactory(username=f"stranger_{model.__name__}") stranger = User.objects.create_user(username=f"stranger_{model.__name__}")
owned = factory(owner=owner) owned = factory(owner=owner)
strangers = factory(owner=stranger) strangers = factory(owner=stranger)
@@ -589,14 +556,14 @@ class TestPermittedObjectIdsGenericModels:
@pytest.mark.parametrize("is_superuser", [False, True]) @pytest.mark.parametrize("is_superuser", [False, True])
def test_inactive_user_sees_nothing(self, model, factory, perm, is_superuser): def test_inactive_user_sees_nothing(self, model, factory, perm, is_superuser):
suffix = f"{model.__name__}_{is_superuser}" suffix = f"{model.__name__}_{is_superuser}"
user = UserFactory( user = User.objects.create_user(
username=f"inactive_{suffix}", username=f"inactive_{suffix}",
is_active=False, is_active=False,
is_superuser=is_superuser, is_superuser=is_superuser,
) )
other = UserFactory(username=f"other_{suffix}") other = User.objects.create_user(username=f"other_{suffix}")
granted = factory(owner=other) granted = factory(owner=other)
grant_object(user, granted, perm) assign_perm(perm, user, granted)
assert_visible_document_ids( assert_visible_document_ids(
permitted_object_ids(user, model, perm), permitted_object_ids(user, model, perm),
@@ -609,7 +576,7 @@ class TestPermittedObjectIdsGenericModels:
) )
def test_unowned_object_visible_to_everyone(self, model, factory, perm): def test_unowned_object_visible_to_everyone(self, model, factory, perm):
user = UserFactory(username=f"user_{model.__name__}") user = User.objects.create_user(username=f"user_{model.__name__}")
unowned = factory(owner=None) unowned = factory(owner=None)
assert_visible_document_ids( assert_visible_document_ids(
@@ -619,12 +586,12 @@ class TestPermittedObjectIdsGenericModels:
) )
def test_explicit_permission_grants_visibility(self, model, factory, perm): def test_explicit_permission_grants_visibility(self, model, factory, perm):
owner = UserFactory(username=f"owner2_{model.__name__}") owner = User.objects.create_user(username=f"owner2_{model.__name__}")
grantee = UserFactory(username=f"grantee_{model.__name__}") grantee = User.objects.create_user(username=f"grantee_{model.__name__}")
stranger = UserFactory(username=f"stranger2_{model.__name__}") stranger = User.objects.create_user(username=f"stranger2_{model.__name__}")
shared = factory(owner=owner) shared = factory(owner=owner)
not_shared = factory(owner=owner) not_shared = factory(owner=owner)
grant_object(grantee, shared, perm) assign_perm(perm, grantee, shared)
assert_visible_document_ids( assert_visible_document_ids(
permitted_object_ids(grantee, model, perm), permitted_object_ids(grantee, model, perm),
@@ -643,13 +610,13 @@ class TestPermittedObjectIdsGenericModels:
factory, factory,
perm, perm,
): ):
owner = UserFactory(username=f"owner3_{model.__name__}") owner = User.objects.create_user(username=f"owner3_{model.__name__}")
member = UserFactory(username=f"member_{model.__name__}") member = User.objects.create_user(username=f"member_{model.__name__}")
non_member = UserFactory(username=f"nonmember_{model.__name__}") non_member = User.objects.create_user(username=f"nonmember_{model.__name__}")
group = Group.objects.create(name=f"group_{model.__name__}") group = Group.objects.create(name=f"group_{model.__name__}")
member.groups.add(group) member.groups.add(group)
shared = factory(owner=owner) shared = factory(owner=owner)
grant_object(group, shared, perm) assign_perm(perm, group, shared)
assert_visible_document_ids( assert_visible_document_ids(
permitted_object_ids(member, model, perm), permitted_object_ids(member, model, perm),
@@ -663,8 +630,8 @@ class TestPermittedObjectIdsGenericModels:
) )
def test_superuser_sees_everything(self, model, factory, perm): def test_superuser_sees_everything(self, model, factory, perm):
superuser = UserFactory(username=f"root_{model.__name__}", superuser=True) superuser = User.objects.create_superuser(username=f"root_{model.__name__}")
owner = UserFactory(username=f"owner4_{model.__name__}") owner = User.objects.create_user(username=f"owner4_{model.__name__}")
obj = factory(owner=owner) obj = factory(owner=owner)
assert_visible_document_ids( assert_visible_document_ids(
@@ -677,8 +644,8 @@ class TestPermittedObjectIdsGenericModels:
@pytest.mark.django_db @pytest.mark.django_db
class TestMatchingRespectsObjectPermissions: class TestMatchingRespectsObjectPermissions:
def test_match_tags_only_considers_tags_visible_to_user(self): def test_match_tags_only_considers_tags_visible_to_user(self):
owner = UserFactory(username="tag_owner") owner = User.objects.create_user(username="tag_owner")
classifying_user = UserFactory(username="classifier_user") classifying_user = User.objects.create_user(username="classifier_user")
visible_tag = TagFactory( visible_tag = TagFactory(
owner=owner, owner=owner,
match="invoice", match="invoice",
@@ -689,7 +656,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice", match="invoice",
matching_algorithm=Tag.MATCH_LITERAL, matching_algorithm=Tag.MATCH_LITERAL,
) )
grant_object(classifying_user, visible_tag, "view_tag") assign_perm("view_tag", classifying_user, visible_tag)
doc = DocumentFactory(owner=classifying_user, content="an invoice document") doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_tags(doc, classifier=None, user=classifying_user) matched = match_tags(doc, classifier=None, user=classifying_user)
@@ -698,8 +665,8 @@ class TestMatchingRespectsObjectPermissions:
assert hidden_tag.pk not in matched_ids assert hidden_tag.pk not in matched_ids
def test_match_correspondents_only_considers_correspondents_visible_to_user(self): def test_match_correspondents_only_considers_correspondents_visible_to_user(self):
owner = UserFactory(username="correspondent_owner") owner = User.objects.create_user(username="correspondent_owner")
classifying_user = UserFactory(username="classifier_user2") classifying_user = User.objects.create_user(username="classifier_user2")
visible_correspondent = CorrespondentFactory( visible_correspondent = CorrespondentFactory(
owner=owner, owner=owner,
match="invoice", match="invoice",
@@ -710,7 +677,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice", match="invoice",
matching_algorithm=Correspondent.MATCH_LITERAL, matching_algorithm=Correspondent.MATCH_LITERAL,
) )
grant_object(classifying_user, visible_correspondent, "view_correspondent") assign_perm("view_correspondent", classifying_user, visible_correspondent)
doc = DocumentFactory(owner=classifying_user, content="an invoice document") doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_correspondents(doc, classifier=None, user=classifying_user) matched = match_correspondents(doc, classifier=None, user=classifying_user)
@@ -719,8 +686,8 @@ class TestMatchingRespectsObjectPermissions:
assert hidden_correspondent.pk not in matched_ids assert hidden_correspondent.pk not in matched_ids
def test_match_document_types_only_considers_document_types_visible_to_user(self): def test_match_document_types_only_considers_document_types_visible_to_user(self):
owner = UserFactory(username="document_type_owner") owner = User.objects.create_user(username="document_type_owner")
classifying_user = UserFactory(username="classifier_user3") classifying_user = User.objects.create_user(username="classifier_user3")
visible_document_type = DocumentTypeFactory( visible_document_type = DocumentTypeFactory(
owner=owner, owner=owner,
match="invoice", match="invoice",
@@ -731,7 +698,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice", match="invoice",
matching_algorithm=DocumentType.MATCH_LITERAL, matching_algorithm=DocumentType.MATCH_LITERAL,
) )
grant_object(classifying_user, visible_document_type, "view_documenttype") assign_perm("view_documenttype", classifying_user, visible_document_type)
doc = DocumentFactory(owner=classifying_user, content="an invoice document") doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_document_types(doc, classifier=None, user=classifying_user) matched = match_document_types(doc, classifier=None, user=classifying_user)
@@ -740,8 +707,8 @@ class TestMatchingRespectsObjectPermissions:
assert hidden_document_type.pk not in matched_ids assert hidden_document_type.pk not in matched_ids
def test_match_storage_paths_only_considers_storage_paths_visible_to_user(self): def test_match_storage_paths_only_considers_storage_paths_visible_to_user(self):
owner = UserFactory(username="storage_path_owner") owner = User.objects.create_user(username="storage_path_owner")
classifying_user = UserFactory(username="classifier_user4") classifying_user = User.objects.create_user(username="classifier_user4")
visible_storage_path = StoragePathFactory( visible_storage_path = StoragePathFactory(
owner=owner, owner=owner,
match="invoice", match="invoice",
@@ -752,7 +719,7 @@ class TestMatchingRespectsObjectPermissions:
match="invoice", match="invoice",
matching_algorithm=StoragePath.MATCH_LITERAL, matching_algorithm=StoragePath.MATCH_LITERAL,
) )
grant_object(classifying_user, visible_storage_path, "view_storagepath") assign_perm("view_storagepath", classifying_user, visible_storage_path)
doc = DocumentFactory(owner=classifying_user, content="an invoice document") doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_storage_paths(doc, classifier=None, user=classifying_user) matched = match_storage_paths(doc, classifier=None, user=classifying_user)
@@ -764,12 +731,14 @@ class TestMatchingRespectsObjectPermissions:
@pytest.mark.django_db @pytest.mark.django_db
class TestBulkEditObjectsApplyToAllPermissionBoundary: class TestBulkEditObjectsApplyToAllPermissionBoundary:
def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client): def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client):
owner = UserFactory(username="tags_owner") owner = User.objects.create_user(username="tags_owner")
requester = UserFactory(username="tags_requester") requester = User.objects.create_user(username="tags_requester")
new_owner = UserFactory(username="tags_new_owner") new_owner = User.objects.create_user(username="tags_new_owner")
# grant the global change_tag permission so the object-level # grant the global change_tag permission so the object-level
# filtering (not the global has_perm check) is what's under test # filtering (not the global has_perm check) is what's under test
grant_global(requester, "change_tag") requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
visible = TagFactory(owner=requester) visible = TagFactory(owner=requester)
hidden = TagFactory(owner=owner) hidden = TagFactory(owner=owner)
@@ -802,14 +771,16 @@ class TestBulkEditObjectsApplyToAllPermissionBoundary:
request rather than being silently skipped. Editing permissions is request rather than being silently skipped. Editing permissions is
limited to the owner, same as documents. limited to the owner, same as documents.
""" """
owner = UserFactory(username="shared_tags_owner") owner = User.objects.create_user(username="shared_tags_owner")
requester = UserFactory(username="shared_tags_requester") requester = User.objects.create_user(username="shared_tags_requester")
grant_global(requester, "change_tag") requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
owned = TagFactory(owner=requester) owned = TagFactory(owner=requester)
shared = TagFactory(owner=owner) shared = TagFactory(owner=owner)
grant_object(requester, shared, "view_tag") assign_perm("view_tag", requester, shared)
grant_object(requester, shared, "change_tag") assign_perm("change_tag", requester, shared)
response = rest_api_client.post( response = rest_api_client.post(
"/api/bulk_edit_objects/", "/api/bulk_edit_objects/",
@@ -860,12 +831,14 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
would pass/fail based on FK cascade behavior, not on whether the would pass/fail based on FK cascade behavior, not on whether the
descendant-expansion logic itself respected per-object permissions. descendant-expansion logic itself respected per-object permissions.
""" """
owner = UserFactory(username="tag_hierarchy_owner") owner = User.objects.create_user(username="tag_hierarchy_owner")
requester = UserFactory(username="tag_hierarchy_requester") requester = User.objects.create_user(username="tag_hierarchy_requester")
new_owner = UserFactory(username="tag_hierarchy_new_owner") new_owner = User.objects.create_user(username="tag_hierarchy_new_owner")
# global change_tag permission so the has_perm() gate passes and the # global change_tag permission so the has_perm() gate passes and the
# object-level permitted_object_ids filtering is what's under test # object-level permitted_object_ids filtering is what's under test
grant_global(requester, "change_tag") requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
parent = TagFactory(owner=requester, name="parent-tag") parent = TagFactory(owner=requester, name="parent-tag")
@@ -917,7 +890,7 @@ class TestRestrictQuerysetToVisible:
- The queryset is returned unfiltered, rather than - The queryset is returned unfiltered, rather than
permitted_object_ids(None, ...)'s narrower "unowned rows only" permitted_object_ids(None, ...)'s narrower "unowned rows only"
""" """
owner = UserFactory(username="vis_none_owner") owner = User.objects.create_user(username="vis_none_owner")
tag = TagFactory(owner=owner) tag = TagFactory(owner=owner)
visible = restrict_queryset_to_visible(Tag.objects.all(), None, "view_tag") visible = restrict_queryset_to_visible(Tag.objects.all(), None, "view_tag")
@@ -934,8 +907,8 @@ class TestRestrictQuerysetToVisible:
- The queryset is returned unfiltered, skipping the permission - The queryset is returned unfiltered, skipping the permission
lookup entirely lookup entirely
""" """
superuser = UserFactory(username="vis_active_super", superuser=True) superuser = User.objects.create_superuser(username="vis_active_super")
owner = UserFactory(username="vis_active_super_owner") owner = User.objects.create_user(username="vis_active_super_owner")
tag = TagFactory(owner=owner) tag = TagFactory(owner=owner)
visible = restrict_queryset_to_visible( visible = restrict_queryset_to_visible(
@@ -957,7 +930,7 @@ class TestRestrictQuerysetToVisible:
deactivation has to win over the superuser shortcut, matching deactivation has to win over the superuser shortcut, matching
permitted_object_ids's own ordering permitted_object_ids's own ordering
""" """
user = UserFactory( user = User.objects.create_user(
username="vis_inactive_super", username="vis_inactive_super",
is_active=False, is_active=False,
is_superuser=True, is_superuser=True,
@@ -978,8 +951,8 @@ class TestRestrictQuerysetToVisible:
THEN: THEN:
- Only the rows permitted_object_ids() reports are visible - Only the rows permitted_object_ids() reports are visible
""" """
user = UserFactory(username="vis_regular") user = User.objects.create_user(username="vis_regular")
other = UserFactory(username="vis_regular_other") other = User.objects.create_user(username="vis_regular_other")
own = TagFactory(owner=user) own = TagFactory(owner=user)
hidden = TagFactory(owner=other) hidden = TagFactory(owner=other)
@@ -1,11 +1,11 @@
import pytest import pytest
from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from rest_framework.test import APIRequestFactory from rest_framework.test import APIRequestFactory
from documents.filters import PermittedObjectsFilter from documents.filters import PermittedObjectsFilter
from documents.models import Tag from documents.models import Tag
from paperless_testing.factories import TagFactory from documents.tests.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_object
class _DummyView: class _DummyView:
@@ -15,8 +15,8 @@ class _DummyView:
@pytest.mark.django_db @pytest.mark.django_db
class TestPermittedObjectsFilter: class TestPermittedObjectsFilter:
def test_superuser_bypasses_filtering_entirely(self): def test_superuser_bypasses_filtering_entirely(self):
superuser = UserFactory(username="root", superuser=True) superuser = User.objects.create_superuser(username="root")
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
TagFactory(owner=owner) TagFactory(owner=owner)
request = APIRequestFactory().get("/") request = APIRequestFactory().get("/")
request.user = superuser request.user = superuser
@@ -29,13 +29,13 @@ class TestPermittedObjectsFilter:
assert result.count() == Tag.objects.count() assert result.count() == Tag.objects.count()
def test_non_superuser_sees_only_owned_unowned_and_granted(self): def test_non_superuser_sees_only_owned_unowned_and_granted(self):
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
grantee = UserFactory(username="grantee") grantee = User.objects.create_user(username="grantee")
owned = TagFactory(owner=grantee) owned = TagFactory(owner=grantee)
unowned = TagFactory(owner=None) unowned = TagFactory(owner=None)
granted = TagFactory(owner=owner) granted = TagFactory(owner=owner)
hidden = TagFactory(owner=owner) hidden = TagFactory(owner=owner)
grant_object(grantee, granted, "view_tag") assign_perm("view_tag", grantee, granted)
request = APIRequestFactory().get("/") request = APIRequestFactory().get("/")
request.user = grantee request.user = grantee
@@ -49,11 +49,11 @@ class TestPermittedObjectsFilter:
assert hidden.pk not in visible_ids assert hidden.pk not in visible_ids
def test_include_granted_false_excludes_explicitly_shared_objects(self): def test_include_granted_false_excludes_explicitly_shared_objects(self):
owner = UserFactory(username="owner2") owner = User.objects.create_user(username="owner2")
grantee = UserFactory(username="grantee2") grantee = User.objects.create_user(username="grantee2")
owned = TagFactory(owner=grantee) owned = TagFactory(owner=grantee)
granted = TagFactory(owner=owner) granted = TagFactory(owner=owner)
grant_object(grantee, granted, "view_tag") assign_perm("view_tag", grantee, granted)
request = APIRequestFactory().get("/") request = APIRequestFactory().get("/")
request.user = grantee request.user = grantee
@@ -74,15 +74,15 @@ class TestPermittedObjectsFilter:
[("inactive", False), ("inactive_super", True)], [("inactive", False), ("inactive_super", True)],
) )
def test_inactive_user_sees_nothing(self, username: str, *, is_superuser: bool): def test_inactive_user_sees_nothing(self, username: str, *, is_superuser: bool):
user = UserFactory( user = User.objects.create_user(
username=username, username=username,
is_active=False, is_active=False,
is_superuser=is_superuser, is_superuser=is_superuser,
) )
TagFactory(owner=None) TagFactory(owner=None)
TagFactory(owner=user) TagFactory(owner=user)
granted = TagFactory(owner=UserFactory(username=f"o_{username}")) granted = TagFactory(owner=User.objects.create_user(username=f"o_{username}"))
grant_object(user, granted, "view_tag") assign_perm("view_tag", user, granted)
request = APIRequestFactory().get("/") request = APIRequestFactory().get("/")
request.user = user request.user = user
@@ -94,7 +94,7 @@ class TestPermittedObjectsFilter:
assert result.count() == 0 assert result.count() == 0
def test_inactive_user_sees_nothing_with_include_granted_false(self): def test_inactive_user_sees_nothing_with_include_granted_false(self):
user = UserFactory(username="inactive_owner", is_active=False) user = User.objects.create_user(username="inactive_owner", is_active=False)
TagFactory(owner=user) TagFactory(owner=user)
TagFactory(owner=None) TagFactory(owner=None)
request = APIRequestFactory().get("/") request = APIRequestFactory().get("/")
+9 -8
View File
@@ -19,7 +19,7 @@ if TYPE_CHECKING:
from collections.abc import Iterable from collections.abc import Iterable
from documents.models import Document from documents.models import Document
from paperless_testing.dirs import PaperlessDirs from documents.tests.conftest import PaperlessDirs
class TestSanityCheckMessages: class TestSanityCheckMessages:
@@ -46,14 +46,14 @@ class TestSanityCheckMessages:
class TestCheckSanityNoDocuments: class TestCheckSanityNoDocuments:
"""Sanity checks against an empty archive.""" """Sanity checks against an empty archive."""
@pytest.mark.usefixtures("paperless_dirs") @pytest.mark.usefixtures("_media_settings")
def test_no_documents(self) -> None: def test_no_documents(self) -> None:
messages = check_sanity() messages = check_sanity()
assert not messages.has_error assert not messages.has_error
assert not messages.has_warning assert not messages.has_warning
assert messages.total_issue_count == 0 assert messages.total_issue_count == 0
@pytest.mark.usefixtures("paperless_dirs") @pytest.mark.usefixtures("_media_settings")
def test_no_issues_logs_clean(self, caplog: pytest.LogCaptureFixture) -> None: def test_no_issues_logs_clean(self, caplog: pytest.LogCaptureFixture) -> None:
messages = check_sanity() messages = check_sanity()
with caplog.at_level(logging.INFO, logger="paperless.sanity_checker"): with caplog.at_level(logging.INFO, logger="paperless.sanity_checker"):
@@ -214,17 +214,18 @@ class TestCheckSanityOrphans:
sample_doc: Document, sample_doc: Document,
paperless_dirs: PaperlessDirs, paperless_dirs: PaperlessDirs,
) -> None: ) -> None:
(paperless_dirs.originals_dir / "orphan.pdf").touch() (paperless_dirs.originals / "orphan.pdf").touch()
messages = check_sanity() messages = check_sanity()
assert messages.has_warning assert messages.has_warning
assert any("Orphaned file" in m["message"] for m in messages[None]) assert any("Orphaned file" in m["message"] for m in messages[None])
@pytest.mark.usefixtures("_media_settings")
def test_ignorable_files_not_flagged( def test_ignorable_files_not_flagged(
self, self,
paperless_dirs: PaperlessDirs, paperless_dirs: PaperlessDirs,
) -> None: ) -> None:
(paperless_dirs.media_dir / ".DS_Store").touch() (paperless_dirs.media / ".DS_Store").touch()
(paperless_dirs.media_dir / "desktop.ini").touch() (paperless_dirs.media / "desktop.ini").touch()
messages = check_sanity() messages = check_sanity()
assert not messages.has_warning assert not messages.has_warning
@@ -268,13 +269,13 @@ class TestCheckSanityLogMessages:
paperless_dirs: PaperlessDirs, paperless_dirs: PaperlessDirs,
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
(paperless_dirs.originals_dir / "orphan.pdf").touch() (paperless_dirs.originals / "orphan.pdf").touch()
messages = check_sanity() messages = check_sanity()
with caplog.at_level(logging.WARNING, logger="paperless.sanity_checker"): with caplog.at_level(logging.WARNING, logger="paperless.sanity_checker"):
messages.log_messages() messages.log_messages()
assert "Orphaned file" in caplog.text assert "Orphaned file" in caplog.text
@pytest.mark.usefixtures("paperless_dirs") @pytest.mark.usefixtures("_media_settings")
def test_logs_unknown_doc_pk(self, caplog: pytest.LogCaptureFixture) -> None: def test_logs_unknown_doc_pk(self, caplog: pytest.LogCaptureFixture) -> None:
"""A doc PK not in the DB logs 'Unknown' as the title.""" """A doc PK not in the DB logs 'Unknown' as the title."""
messages = check_sanity() messages = check_sanity()
+23 -11
View File
@@ -6,8 +6,10 @@ from pathlib import Path
from unittest import mock from unittest import mock
from django.conf import settings from django.conf import settings
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.utils import timezone from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import serializers from rest_framework import serializers
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
@@ -18,11 +20,8 @@ from documents.models import ShareLinkBundle
from documents.serialisers import ShareLinkBundleSerializer from documents.serialisers import ShareLinkBundleSerializer
from documents.tasks import build_share_link_bundle from documents.tasks import build_share_link_bundle
from documents.tasks import cleanup_expired_share_link_bundles from documents.tasks import cleanup_expired_share_link_bundles
from paperless_testing.dirs import DirectoriesMixin from documents.tests.factories import DocumentFactory
from paperless_testing.factories import DocumentFactory from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class ShareLinkBundleAPITests(DirectoriesMixin, APITestCase): class ShareLinkBundleAPITests(DirectoriesMixin, APITestCase):
@@ -30,7 +29,7 @@ class ShareLinkBundleAPITests(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="bundle_admin", superuser=True) self.user = User.objects.create_superuser(username="bundle_admin")
self.client.force_authenticate(self.user) self.client.force_authenticate(self.user)
self.document = DocumentFactory.create() self.document = DocumentFactory.create()
@@ -56,11 +55,13 @@ class ShareLinkBundleAPITests(DirectoriesMixin, APITestCase):
self, self,
delay_mock, delay_mock,
) -> None: ) -> None:
owner = UserFactory(username="document_owner") owner = User.objects.create_user(username="document_owner")
requester = UserFactory(username="bundle_creator") requester = User.objects.create_user(username="bundle_creator")
grant_global(requester, "add_sharelinkbundle") requester.user_permissions.add(
Permission.objects.get(codename="add_sharelinkbundle"),
)
document = DocumentFactory.create(owner=owner) document = DocumentFactory.create(owner=owner)
grant_object(requester, document, "view_document") assign_perm("view_document", requester, document)
self.client.force_authenticate(requester) self.client.force_authenticate(requester)
payload = { payload = {
"document_ids": [document.pk], "document_ids": [document.pk],
@@ -71,7 +72,9 @@ class ShareLinkBundleAPITests(DirectoriesMixin, APITestCase):
response = self.client.post(self.ENDPOINT, payload, format="json") response = self.client.post(self.ENDPOINT, payload, format="json")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
grant_global(requester, "view_document") requester.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
requester = User.objects.get(pk=requester.pk) requester = User.objects.get(pk=requester.pk)
self.client.force_authenticate(requester) self.client.force_authenticate(requester)
response = self.client.post(self.ENDPOINT, payload, format="json") response = self.client.post(self.ENDPOINT, payload, format="json")
@@ -339,6 +342,15 @@ class ShareLinkBundleBuildTaskTests(DirectoriesMixin, APITestCase):
) )
self.document.archive_checksum = "" self.document.archive_checksum = ""
self.document.save() self.document.save()
self.addCleanup(
setattr,
settings,
"SHARE_LINK_BUNDLE_DIR",
settings.SHARE_LINK_BUNDLE_DIR,
)
settings.SHARE_LINK_BUNDLE_DIR = (
Path(settings.MEDIA_ROOT) / "documents" / "share_link_bundles"
)
def _write_document_file(self, *, archive: bool, content: bytes) -> Path: def _write_document_file(self, *, archive: bool, content: bytes) -> Path:
if archive: if archive:
+9 -7
View File
@@ -1,5 +1,7 @@
from unittest import mock from unittest import mock
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from rest_framework import status from rest_framework import status
from rest_framework.test import APITestCase from rest_framework.test import APITestCase
@@ -11,16 +13,16 @@ from documents.models import WorkflowAction
from documents.models import WorkflowTrigger from documents.models import WorkflowTrigger
from documents.serialisers import TagSerializer from documents.serialisers import TagSerializer
from documents.signals.handlers import run_workflows from documents.signals.handlers import run_workflows
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
class TestTagHierarchyPermissions(APITestCase): class TestTagHierarchyPermissions(APITestCase):
def test_children_only_include_visible_tags(self) -> None: def test_children_only_include_visible_tags(self) -> None:
owner = UserFactory(username="owner") owner = User.objects.create_user(username="owner")
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
grant_global(requester, "view_tag") requester.user_permissions.add(
Permission.objects.get(codename="view_tag"),
)
parent = Tag.objects.create(name="Visible parent", owner=requester) parent = Tag.objects.create(name="Visible parent", owner=requester)
hidden_child = Tag.objects.create( hidden_child = Tag.objects.create(
name="Hidden child", name="Hidden child",
@@ -47,7 +49,7 @@ class TestTagHierarchyPermissions(APITestCase):
class TestTagHierarchy(DirectoriesMixin, APITestCase): class TestTagHierarchy(DirectoriesMixin, APITestCase):
def setUp(self) -> None: def setUp(self) -> None:
super().setUp() super().setUp()
self.user = UserFactory(username="admin", superuser=True) self.user = User.objects.create_superuser(username="admin")
self.client.force_authenticate(user=self.user) self.client.force_authenticate(user=self.user)
self.parent = Tag.objects.create(name="Parent") self.parent = Tag.objects.create(name="Parent")
+3 -4
View File
@@ -17,8 +17,7 @@ from documents.signals.handlers import task_failure_handler
from documents.signals.handlers import task_postrun_handler from documents.signals.handlers import task_postrun_handler
from documents.signals.handlers import task_prerun_handler from documents.signals.handlers import task_prerun_handler
from documents.signals.handlers import task_revoked_handler from documents.signals.handlers import task_revoked_handler
from paperless_testing.factories import PaperlessTaskFactory from documents.tests.factories import PaperlessTaskFactory
from paperless_testing.factories import UserFactory
@pytest.fixture @pytest.fixture
@@ -35,8 +34,8 @@ def consume_input_doc():
@pytest.fixture @pytest.fixture
def consume_overrides(): def consume_overrides(django_user_model):
user = UserFactory(username="testuser") user = django_user_model.objects.create_user(username="testuser")
overrides = mock.MagicMock(spec=DocumentMetadataOverrides) overrides = mock.MagicMock(spec=DocumentMetadataOverrides)
overrides.owner_id = user.id overrides.owner_id = user.id
return overrides return overrides
+3 -3
View File
@@ -17,9 +17,9 @@ from documents.models import Tag
from documents.models import WorkflowAction from documents.models import WorkflowAction
from documents.sanity_checker import SanityCheckFailedException from documents.sanity_checker import SanityCheckFailedException
from documents.sanity_checker import SanityCheckMessages from documents.sanity_checker import SanityCheckMessages
from documents.tests.helpers import dummy_preprocess from documents.tests.test_classifier import dummy_preprocess
from paperless_testing.assertions import FileSystemAssertsMixin from documents.tests.utils import DirectoriesMixin
from paperless_testing.dirs import DirectoriesMixin from documents.tests.utils import FileSystemAssertsMixin
@pytest.mark.django_db @pytest.mark.django_db
@@ -14,8 +14,8 @@ from documents.conditionals import preview_etag
from documents.conditionals import thumbnail_etag from documents.conditionals import thumbnail_etag
from documents.conditionals import thumbnail_last_modified from documents.conditionals import thumbnail_last_modified
from documents.models import Document from documents.models import Document
from documents.tests.utils import DirectoriesMixin
from documents.versioning import resolve_effective_document_by_pk from documents.versioning import resolve_effective_document_by_pk
from paperless_testing.dirs import DirectoriesMixin
if TYPE_CHECKING: if TYPE_CHECKING:
from rest_framework.request import Request from rest_framework.request import Request
+40 -28
View File
@@ -15,6 +15,7 @@ from django.test import TestCase
from django.test import override_settings from django.test import override_settings
from django.test.utils import CaptureQueriesContext from django.test.utils import CaptureQueriesContext
from django.utils import timezone from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status from rest_framework import status
from documents.caching import get_llm_suggestion_cache from documents.caching import get_llm_suggestion_cache
@@ -28,14 +29,11 @@ from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from documents.models import UiSettings from documents.models import UiSettings
from documents.signals.handlers import update_llm_suggestions_cache from documents.signals.handlers import update_llm_suggestions_cache
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import read_streaming_response
from paperless.models import ApplicationConfiguration from paperless.models import ApplicationConfiguration
from paperless_ai.exceptions import LLMProviderError from paperless_ai.exceptions import LLMProviderError
from paperless_ai.exceptions import LLMTimeoutError from paperless_ai.exceptions import LLMTimeoutError
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
class TestViews(DirectoriesMixin, TestCase): class TestViews(DirectoriesMixin, TestCase):
@@ -45,7 +43,7 @@ class TestViews(DirectoriesMixin, TestCase):
ApplicationConfiguration.objects.get_or_create() ApplicationConfiguration.objects.get_or_create()
def setUp(self) -> None: def setUp(self) -> None:
self.user = UserFactory(username="testuser") self.user = User.objects.create_user("testuser")
super().setUp() super().setUp()
def test_login_redirect(self) -> None: def test_login_redirect(self) -> None:
@@ -143,7 +141,9 @@ class TestViews(DirectoriesMixin, TestCase):
codename__contains="sharelink", codename__contains="sharelink",
) )
self.user.user_permissions.add(*sharelink_permissions) self.user.user_permissions.add(*sharelink_permissions)
grant_global(self.user, "view_document") self.user.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
self.user.save() self.user.save()
self.client.force_login(self.user) self.client.force_login(self.user)
@@ -205,7 +205,9 @@ class TestViews(DirectoriesMixin, TestCase):
codename__contains="sharelink", codename__contains="sharelink",
) )
self.user.user_permissions.add(*sharelink_permissions) self.user.user_permissions.add(*sharelink_permissions)
grant_global(self.user, "view_document") self.user.user_permissions.add(
Permission.objects.get(codename="view_document"),
)
self.client.force_login(self.user) self.client.force_login(self.user)
create_response = self.client.post( create_response = self.client.post(
@@ -238,16 +240,16 @@ class TestViews(DirectoriesMixin, TestCase):
group2 = Group.objects.create(name="group2") group2 = Group.objects.create(name="group2")
group3 = Group.objects.create(name="group3") group3 = Group.objects.create(name="group3")
t1 = Tag.objects.create(name="invoice", pk=1) t1 = Tag.objects.create(name="invoice", pk=1)
grant_object(self.user, t1, "view_tag") assign_perm("view_tag", self.user, t1)
grant_object(user2, t1, "view_tag") assign_perm("view_tag", user2, t1)
grant_object(user3, t1, "view_tag") assign_perm("view_tag", user3, t1)
grant_object(group1, t1, "view_tag") assign_perm("view_tag", group1, t1)
grant_object(group2, t1, "view_tag") assign_perm("view_tag", group2, t1)
grant_object(group3, t1, "view_tag") assign_perm("view_tag", group3, t1)
grant_object(self.user, t1, "change_tag") assign_perm("change_tag", self.user, t1)
grant_object(user2, t1, "change_tag") assign_perm("change_tag", user2, t1)
grant_object(group1, t1, "change_tag") assign_perm("change_tag", group1, t1)
grant_object(group2, t1, "change_tag") assign_perm("change_tag", group2, t1)
Tag.objects.create(name="bank statement", pk=2) Tag.objects.create(name="bank statement", pk=2)
d1 = Document.objects.create( d1 = Document.objects.create(
@@ -336,7 +338,7 @@ class TestViews(DirectoriesMixin, TestCase):
class TestAISuggestions(DirectoriesMixin, TestCase): class TestAISuggestions(DirectoriesMixin, TestCase):
def setUp(self) -> None: def setUp(self) -> None:
self.user = UserFactory(username="testuser", superuser=True) self.user = User.objects.create_superuser(username="testuser")
self.document = Document.objects.create( self.document = Document.objects.create(
title="Test Document", title="Test Document",
filename="test.pdf", filename="test.pdf",
@@ -423,10 +425,14 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
requester; the invisible tag id does not leak into either the requester; the invisible tag id does not leak into either the
matched or suggested tags matched or suggested tags
""" """
tag_owner = UserFactory(username="cache_tag_owner") tag_owner = User.objects.create_user(username="cache_tag_owner")
invisible_tag = Tag.objects.create(name="cache_restricted", owner=tag_owner) invisible_tag = Tag.objects.create(name="cache_restricted", owner=tag_owner)
requester = UserFactory(username="cache_requester") requester = User.objects.create_user(username="cache_requester")
grant_global(requester, "view_document", "change_document", "view_tag") requester.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_document", "change_document", "view_tag"],
),
)
mock_get_cache.return_value = MagicMock( mock_get_cache.return_value = MagicMock(
suggestions={ suggestions={
"title": "Untitled", "title": "Untitled",
@@ -631,7 +637,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
- The classification runs with the second user's visibility - The classification runs with the second user's visibility
context without evicting the first user's result context without evicting the first user's result
""" """
second_user = UserFactory(username="second_user", superuser=True) second_user = User.objects.create_superuser(username="second_user")
empty_choices = { empty_choices = {
"tags": {"existing_ids": [], "new_names": []}, "tags": {"existing_ids": [], "new_names": []},
"correspondents": {"existing_ids": [], "new_names": []}, "correspondents": {"existing_ids": [], "new_names": []},
@@ -869,10 +875,14 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
permission filtering survives the full request path permission filtering survives the full request path
- it does not appear in either the matched or suggested tags - it does not appear in either the matched or suggested tags
""" """
tag_owner = UserFactory(username="tagowner") tag_owner = User.objects.create_user(username="tagowner")
invisible_tag = Tag.objects.create(name="restricted", owner=tag_owner) invisible_tag = Tag.objects.create(name="restricted", owner=tag_owner)
requester = UserFactory(username="requester") requester = User.objects.create_user(username="requester")
grant_global(requester, "view_document", "change_document", "view_tag") requester.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_document", "change_document", "view_tag"],
),
)
mock_get_ai_classification.return_value = { mock_get_ai_classification.return_value = {
"title": "Untitled", "title": "Untitled",
@@ -946,7 +956,7 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
ENDPOINT = "/api/documents/chat/" ENDPOINT = "/api/documents/chat/"
def setUp(self) -> None: def setUp(self) -> None:
self.user = UserFactory(username="testuser", password="pass") self.user = User.objects.create_user(username="testuser", password="pass")
self.client.force_login(user=self.user) self.client.force_login(user=self.user)
self.document = Document.objects.create( self.document = Document.objects.create(
title="Test Document", title="Test Document",
@@ -956,7 +966,9 @@ class TestAIChatStreamingView(DirectoriesMixin, TestCase):
super().setUp() super().setUp()
def grant_view_document_permission(self) -> None: def grant_view_document_permission(self) -> None:
grant_global(self.user, "view_document") self.user.user_permissions.add(
*Permission.objects.filter(codename="view_document"),
)
@override_settings(AI_ENABLED=False) @override_settings(AI_ENABLED=False)
def test_post_ai_disabled(self) -> None: def test_post_ai_disabled(self) -> None:
File diff suppressed because it is too large Load Diff
+346 -2
View File
@@ -1,16 +1,242 @@
import shutil
import tempfile
import time
import warnings
from collections import namedtuple
from collections.abc import Callable
from collections.abc import Generator from collections.abc import Generator
from collections.abc import Iterator from collections.abc import Iterator
from contextlib import contextmanager from contextlib import contextmanager
from os import PathLike
from pathlib import Path from pathlib import Path
from typing import Any
from unittest import mock from unittest import mock
import httpx
import pytest
from django.apps import apps
from django.db import connection
from django.db.migrations.executor import MigrationExecutor
from django.http import StreamingHttpResponse
from django.test import TransactionTestCase
from django.test import override_settings
from documents.consumer import AsnCheckPlugin from documents.consumer import AsnCheckPlugin
from documents.consumer import ConsumerPlugin from documents.consumer import ConsumerPlugin
from documents.consumer import ConsumerPreflightPlugin from documents.consumer import ConsumerPreflightPlugin
from documents.data_models import ConsumableDocument from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides from documents.data_models import DocumentMetadataOverrides
from documents.data_models import DocumentSource from documents.data_models import DocumentSource
from paperless_testing.fakes.progress import FakeProgressManager from documents.parsers import ParseError
from documents.plugins.helpers import ProgressStatusOptions
def setup_directories():
dirs = namedtuple("Dirs", ())
dirs.data_dir = Path(tempfile.mkdtemp()).resolve()
dirs.scratch_dir = Path(tempfile.mkdtemp()).resolve()
dirs.media_dir = Path(tempfile.mkdtemp()).resolve()
dirs.consumption_dir = Path(tempfile.mkdtemp()).resolve()
dirs.static_dir = Path(tempfile.mkdtemp()).resolve()
dirs.index_dir = dirs.data_dir / "index"
dirs.originals_dir = dirs.media_dir / "documents" / "originals"
dirs.thumbnail_dir = dirs.media_dir / "documents" / "thumbnails"
dirs.archive_dir = dirs.media_dir / "documents" / "archive"
dirs.logging_dir = dirs.data_dir / "log"
dirs.index_dir.mkdir(parents=True, exist_ok=True)
dirs.originals_dir.mkdir(parents=True, exist_ok=True)
dirs.thumbnail_dir.mkdir(parents=True, exist_ok=True)
dirs.archive_dir.mkdir(parents=True, exist_ok=True)
dirs.logging_dir.mkdir(parents=True, exist_ok=True)
dirs.settings_override = override_settings(
DATA_DIR=dirs.data_dir,
SCRATCH_DIR=dirs.scratch_dir,
MEDIA_ROOT=dirs.media_dir,
ORIGINALS_DIR=dirs.originals_dir,
THUMBNAIL_DIR=dirs.thumbnail_dir,
ARCHIVE_DIR=dirs.archive_dir,
CONSUMPTION_DIR=dirs.consumption_dir,
LOGGING_DIR=dirs.logging_dir,
INDEX_DIR=dirs.index_dir,
STATIC_ROOT=dirs.static_dir,
MODEL_FILE=dirs.data_dir / "classification_model.pickle",
MEDIA_LOCK=dirs.media_dir / "media.lock",
)
dirs.settings_override.enable()
return dirs
def remove_dirs(dirs) -> None:
shutil.rmtree(dirs.media_dir, ignore_errors=True)
shutil.rmtree(dirs.data_dir, ignore_errors=True)
shutil.rmtree(dirs.scratch_dir, ignore_errors=True)
shutil.rmtree(dirs.consumption_dir, ignore_errors=True)
shutil.rmtree(dirs.static_dir, ignore_errors=True)
dirs.settings_override.disable()
@contextmanager
def paperless_environment():
dirs = None
try:
dirs = setup_directories()
yield dirs
finally:
if dirs:
remove_dirs(dirs)
def util_call_with_backoff(
method_or_callable: Callable,
args: list | tuple,
*,
skip_on_50x_err=True,
) -> tuple[bool, Any]:
"""
For whatever reason, the images started during the test pipeline like to
segfault sometimes, crash and otherwise fail randomly, when run with the
exact files that usually pass.
So, this function will retry the given method/function up to 3 times, with larger backoff
periods between each attempt, in hopes the issue resolves itself during
one attempt to parse.
This will wait the following:
- Attempt 1 - 20s following failure
- Attempt 2 - 40s following failure
- Attempt 3 - 80s following failure
"""
result = None
succeeded = False
retry_time = 20.0
retry_count = 0
status_codes = []
max_retry_count = 3
while retry_count < max_retry_count and not succeeded:
try:
result = method_or_callable(*args)
succeeded = True
except ParseError as e: # pragma: no cover
cause_exec = e.__cause__
if cause_exec is not None and isinstance(cause_exec, httpx.HTTPStatusError):
status_codes.append(cause_exec.response.status_code)
warnings.warn(
f"HTTP Exception for {cause_exec.request.url} - {cause_exec}",
)
else:
warnings.warn(f"Unexpected error: {e}")
except Exception as e: # pragma: no cover
warnings.warn(f"Unexpected error: {e}")
retry_count = retry_count + 1
time.sleep(retry_time)
retry_time = retry_time * 2.0
if (
not succeeded
and status_codes
and skip_on_50x_err
and all(httpx.codes.is_server_error(code) for code in status_codes)
):
pytest.skip("Repeated HTTP 50x for service") # pragma: no cover
return succeeded, result
def read_streaming_response(response: StreamingHttpResponse) -> bytes:
"""Consume a StreamingHttpResponse/FileResponse and close it."""
content = b"".join(response.streaming_content)
response.close()
return content
class DirectoriesMixin:
"""
Creates and overrides settings for all folders and paths, then ensures
they are cleaned up on exit
"""
def setUp(self) -> None:
from documents.search import reset_backend
reset_backend()
self.dirs = setup_directories()
super().setUp()
def tearDown(self) -> None:
from documents.search import reset_backend
super().tearDown()
reset_backend()
remove_dirs(self.dirs)
class FileSystemAssertsMixin:
"""
Utilities for checks various state information of the file system
"""
def assertIsFile(self, path: PathLike[str] | str) -> None:
self.assertTrue(Path(path).resolve().is_file(), f"File does not exist: {path}")
def assertIsNotFile(self, path: PathLike[str] | str) -> None:
self.assertFalse(Path(path).resolve().is_file(), f"File does exist: {path}")
def assertIsDir(self, path: PathLike[str] | str) -> None:
self.assertTrue(Path(path).resolve().is_dir(), f"Dir does not exist: {path}")
def assertIsNotDir(self, path: PathLike[str] | str) -> None:
self.assertFalse(Path(path).resolve().is_dir(), f"Dir does exist: {path}")
def assertFilesEqual(
self,
path1: PathLike[str] | str,
path2: PathLike[str] | str,
) -> None:
path1 = Path(path1)
path2 = Path(path2)
import hashlib
hash1 = hashlib.sha256(path1.read_bytes()).hexdigest()
hash2 = hashlib.sha256(path2.read_bytes()).hexdigest()
self.assertEqual(hash1, hash2, "File SHA256 mismatch")
def assertFileCountInDir(self, path: PathLike[str] | str, count: int) -> None:
path = Path(path).resolve()
self.assertTrue(path.is_dir(), f"Path {path} is not a directory")
files = [x for x in path.iterdir() if x.is_file()]
self.assertEqual(
len(files),
count,
f"Path {path} contains {len(files)} files instead of {count} files",
)
class ConsumerProgressMixin:
"""
Mocks the Consumer _send_progress, preventing attempts to connect to Redis
and allowing access to its calls for verification
"""
def setUp(self) -> None:
self.send_progress_patcher = mock.patch(
"documents.consumer.Consumer._send_progress",
)
self.send_progress_mock = self.send_progress_patcher.start()
super().setUp()
def tearDown(self) -> None:
super().tearDown()
self.send_progress_patcher.stop()
class ConsumeTaskMixin: class ConsumeTaskMixin:
@@ -48,6 +274,64 @@ class ConsumeTaskMixin:
yield (task_kwargs["input_doc"], task_kwargs["overrides"]) yield (task_kwargs["input_doc"], task_kwargs["overrides"])
class TestMigrations(TransactionTestCase):
@property
def app(self):
return apps.get_containing_app_config(type(self).__module__).name
migrate_from = None
dependencies = None
migrate_to = None
auto_migrate = True
def setUp(self) -> None:
super().setUp()
assert self.migrate_from and self.migrate_to, (
f"TestCase '{type(self).__name__}' must define migrate_from and migrate_to properties"
)
self.migrate_from = [(self.app, self.migrate_from)]
if self.dependencies is not None:
self.migrate_from.extend(self.dependencies)
self.migrate_to = [(self.app, self.migrate_to)]
executor = MigrationExecutor(connection)
old_apps = executor.loader.project_state(self.migrate_from).apps
# Reverse to the original migration
executor.migrate(self.migrate_from)
self.setUpBeforeMigration(old_apps)
self.apps = old_apps
if self.auto_migrate:
self.performMigration()
def performMigration(self) -> None:
# Run the migration to test
executor = MigrationExecutor(connection)
executor.loader.build_graph() # reload.
executor.migrate(self.migrate_to)
self.apps = executor.loader.project_state(self.migrate_to).apps
def setUpBeforeMigration(self, apps) -> None:
pass
def tearDown(self) -> None:
"""
Ensure the database schema is restored to the latest migration after
each migration test, so subsequent tests run against HEAD.
"""
try:
executor = MigrationExecutor(connection)
executor.loader.build_graph()
targets = executor.loader.graph.leaf_nodes()
executor.migrate(targets)
finally:
super().tearDown()
class SampleDirMixin: class SampleDirMixin:
SAMPLE_DIR = Path(__file__).parent / "samples" SAMPLE_DIR = Path(__file__).parent / "samples"
@@ -64,7 +348,7 @@ class GetConsumerMixin:
mailrule_id: int | None = None, mailrule_id: int | None = None,
) -> Generator[ConsumerPlugin, None, None]: ) -> Generator[ConsumerPlugin, None, None]:
# Store this for verification # Store this for verification
self.status = FakeProgressManager(filepath.name, None) self.status = DummyProgressManager(filepath.name, None)
doc = ConsumableDocument( doc = ConsumableDocument(
source, source,
original_file=filepath, original_file=filepath,
@@ -100,3 +384,63 @@ class GetConsumerMixin:
yield reader yield reader
finally: finally:
reader.cleanup() reader.cleanup()
class DummyProgressManager:
"""
A dummy handler for progress management that doesn't actually try to
connect to Redis. Payloads are stored for test assertions if needed.
Use it with
mock.patch("documents.tasks.ProgressManager", DummyProgressManager)
"""
def __init__(self, filename: str, task_id: str | None = None) -> None:
self.filename = filename
self.task_id = task_id
self.payloads = []
def __enter__(self):
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
def open(self) -> None:
pass
def close(self) -> None:
pass
def send_progress(
self,
status: ProgressStatusOptions,
message: str,
current_progress: int,
max_progress: int,
*,
document_id: int | None = None,
owner_id: int | None = None,
users_can_view: list[int] | None = None,
groups_can_view: list[int] | None = None,
) -> None:
# Ensure the layer is open
self.open()
payload = {
"type": "status_update",
"data": {
"filename": self.filename,
"task_id": self.task_id,
"current_progress": current_progress,
"max_progress": max_progress,
"status": status,
"message": message,
"document_id": document_id,
"owner_id": owner_id,
"users_can_view": users_can_view or [],
"groups_can_view": groups_can_view or [],
},
}
self.payloads.append(payload)
+34 -51
View File
@@ -49,6 +49,7 @@ from django.db.models import Sum
from django.db.models import When from django.db.models import When
from django.db.models.functions import Coalesce from django.db.models.functions import Coalesce
from django.db.models.functions import Lower from django.db.models.functions import Lower
from django.db.models.manager import Manager
from django.http import FileResponse from django.http import FileResponse
from django.http import Http404 from django.http import Http404
from django.http import HttpRequest from django.http import HttpRequest
@@ -2967,15 +2968,11 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
if user.is_superuser: if user.is_superuser:
return True return True
root_docs = { document_objs = Document.objects.select_related("owner").filter(
get_root_document(doc) pk__in=documents,
for doc in Document.objects.select_related( )
"owner",
"root_document__owner",
).filter(pk__in=documents)
}
user_is_owner_of_all_documents = all( user_is_owner_of_all_documents = all(
(doc.owner == user or doc.owner is None) for doc in root_docs (doc.owner == user or doc.owner is None) for doc in document_objs
) )
# check global and object permissions for all documents # check global and object permissions for all documents
@@ -2983,13 +2980,9 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
user.has_perm( user.has_perm(
"documents.change_document", "documents.change_document",
) )
and not Document.global_objects.filter( and not document_objs.exclude(
pk__in=[doc.pk for doc in root_docs],
)
.exclude(
pk__in=permitted_document_ids(user, perm="change_document"), pk__in=permitted_document_ids(user, perm="change_document"),
) ).exists()
.exists()
) )
# check ownership for methods that change original document # check ownership for methods that change original document
@@ -3148,38 +3141,6 @@ class BulkEditView(DocumentOperationPermissionMixin):
serializer_class = BulkEditSerializer serializer_class = BulkEditSerializer
@staticmethod
def _snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
"""
Returns each document's current value of field, for the audit log.
Tags and custom fields are one row per value, so they are gathered
into a sorted list of pks per document (empty when there are none).
Reading them through Document.values() instead would join those rows
and return one arbitrary value per document.
"""
if field == "tags":
rows = (
Document.tags.through.objects.filter(document_id__in=doc_ids)
.order_by("tag_id")
.values_list("document_id", "tag_id")
)
elif field == "custom_fields":
rows = (
CustomFieldInstance.objects.filter(document_id__in=doc_ids)
.order_by("pk")
.values_list("document_id", "pk")
)
else:
return dict(
Document.objects.filter(pk__in=doc_ids).values_list("pk", field),
)
values: dict[int, list[int]] = {doc_id: [] for doc_id in doc_ids}
for doc_id, pk in rows:
values[doc_id].append(pk)
return values
def post(self, request, *args, **kwargs): def post(self, request, *args, **kwargs):
request_method = request.data.get("method") request_method = request.data.get("method")
api_version = int(request.version or settings.REST_FRAMEWORK["DEFAULT_VERSION"]) api_version = int(request.version or settings.REST_FRAMEWORK["DEFAULT_VERSION"])
@@ -3226,19 +3187,41 @@ class BulkEditView(DocumentOperationPermissionMixin):
try: try:
modified_field = self.MODIFIED_FIELD_BY_METHOD.get(method.__name__, None) modified_field = self.MODIFIED_FIELD_BY_METHOD.get(method.__name__, None)
if settings.AUDIT_LOG_ENABLED and modified_field: if settings.AUDIT_LOG_ENABLED and modified_field:
old_values = self._snapshot_field(documents, modified_field) old_documents = {
obj["pk"]: obj
for obj in Document.objects.filter(pk__in=documents).values(
"pk",
"correspondent",
"document_type",
"storage_path",
"tags",
"custom_fields",
"deleted_at",
"checksum",
)
}
result = method(documents, **parameters) result = method(documents, **parameters)
if settings.AUDIT_LOG_ENABLED and modified_field: if settings.AUDIT_LOG_ENABLED and modified_field:
new_values = self._snapshot_field(documents, modified_field) new_documents = Document.objects.filter(pk__in=documents)
for doc in Document.objects.filter(pk__in=documents): for doc in new_documents:
old_value = old_documents[doc.pk][modified_field]
new_value = getattr(doc, modified_field)
if isinstance(new_value, Model):
# correspondent, document type, etc.
new_value = new_value.pk
elif isinstance(new_value, Manager):
# tags, custom fields
new_value = list(new_value.values_list("pk", flat=True))
LogEntry.objects.log_create( LogEntry.objects.log_create(
instance=doc, instance=doc,
changes={ changes={
modified_field: [ modified_field: [
old_values[doc.pk], old_value,
new_values[doc.pk], new_value,
], ],
}, },
action=LogEntry.Action.UPDATE, action=LogEntry.Action.UPDATE,
+65 -69
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr "" msgstr ""
"Project-Id-Version: paperless-ngx\n" "Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n" "Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-09-23 19:00+0000\n" "POT-Creation-Date: 2026-09-18 01:29+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n" "PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n" "Last-Translator: \n"
"Language-Team: English\n" "Language-Team: English\n"
@@ -1632,7 +1632,7 @@ msgid "workflow runs"
msgstr "" msgstr ""
#: documents/serialisers.py:514 documents/serialisers.py:871 #: documents/serialisers.py:514 documents/serialisers.py:871
#: documents/serialisers.py:2885 documents/views.py:342 documents/views.py:2725 #: documents/serialisers.py:2883 documents/views.py:343 documents/views.py:2726
#: paperless_mail/serialisers.py:156 #: paperless_mail/serialisers.py:156
msgid "Insufficient permissions." msgid "Insufficient permissions."
msgstr "" msgstr ""
@@ -1641,39 +1641,39 @@ msgstr ""
msgid "Invalid color." msgid "Invalid color."
msgstr "" msgstr ""
#: documents/serialisers.py:2352 #: documents/serialisers.py:2350
#, python-format #, python-format
msgid "File type %(type)s not supported" msgid "File type %(type)s not supported"
msgstr "" msgstr ""
#: documents/serialisers.py:2396 #: documents/serialisers.py:2394
#, python-format #, python-format
msgid "Custom field id must be an integer: %(id)s" msgid "Custom field id must be an integer: %(id)s"
msgstr "" msgstr ""
#: documents/serialisers.py:2403 #: documents/serialisers.py:2401
#, python-format #, python-format
msgid "Custom field with id %(id)s does not exist" msgid "Custom field with id %(id)s does not exist"
msgstr "" msgstr ""
#: documents/serialisers.py:2420 documents/serialisers.py:2430 #: documents/serialisers.py:2418 documents/serialisers.py:2428
msgid "" msgid ""
"Custom fields must be a list of integers or an object mapping ids to values." "Custom fields must be a list of integers or an object mapping ids to values."
msgstr "" msgstr ""
#: documents/serialisers.py:2425 #: documents/serialisers.py:2423
msgid "Some custom fields don't exist or were specified twice." msgid "Some custom fields don't exist or were specified twice."
msgstr "" msgstr ""
#: documents/serialisers.py:2572 #: documents/serialisers.py:2570
msgid "Invalid variable detected." msgid "Invalid variable detected."
msgstr "" msgstr ""
#: documents/serialisers.py:2941 #: documents/serialisers.py:2939
msgid "Duplicate document identifiers are not allowed." msgid "Duplicate document identifiers are not allowed."
msgstr "" msgstr ""
#: documents/serialisers.py:2971 documents/views.py:4780 #: documents/serialisers.py:2969 documents/views.py:4763
#, python-format #, python-format
msgid "Documents not found: %(ids)s" msgid "Documents not found: %(ids)s"
msgstr "" msgstr ""
@@ -1941,40 +1941,40 @@ msgstr ""
msgid "Unable to parse URI {value}" msgid "Unable to parse URI {value}"
msgstr "" msgstr ""
#: documents/views.py:335 documents/views.py:2722 #: documents/views.py:336 documents/views.py:2723
msgid "Invalid more_like_id" msgid "Invalid more_like_id"
msgstr "" msgstr ""
#: documents/views.py:1669 #: documents/views.py:1670
msgid "Invalid AI configuration." msgid "Invalid AI configuration."
msgstr "" msgstr ""
#: documents/views.py:1680 #: documents/views.py:1681
msgid "AI backend request timed out." msgid "AI backend request timed out."
msgstr "" msgstr ""
#: documents/views.py:1692 #: documents/views.py:1693
msgid "AI backend rejected the request. Check logs for details." msgid "AI backend rejected the request. Check logs for details."
msgstr "" msgstr ""
#: documents/views.py:2547 documents/views.py:2863 #: documents/views.py:2548 documents/views.py:2864
msgid "Specify only one of text, title_search, query, or more_like_id." msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr "" msgstr ""
#: documents/views.py:4793 #: documents/views.py:4776
#, python-format #, python-format
msgid "Insufficient permissions to share document %(id)s." msgid "Insufficient permissions to share document %(id)s."
msgstr "" msgstr ""
#: documents/views.py:4839 #: documents/views.py:4822
msgid "Bundle is already being processed." msgid "Bundle is already being processed."
msgstr "" msgstr ""
#: documents/views.py:4903 #: documents/views.py:4886
msgid "The share link bundle is still being prepared. Please try again later." msgid "The share link bundle is still being prepared. Please try again later."
msgstr "" msgstr ""
#: documents/views.py:4917 #: documents/views.py:4900
msgid "The share link bundle is unavailable." msgid "The share link bundle is unavailable."
msgstr "" msgstr ""
@@ -2219,194 +2219,190 @@ msgid "Sets the LLM embedding model"
msgstr "" msgstr ""
#: paperless/models.py:369 #: paperless/models.py:369
msgid "Sets the LLM embedding API key"
msgstr ""
#: paperless/models.py:376
msgid "Sets the LLM embedding endpoint, optional" msgid "Sets the LLM embedding endpoint, optional"
msgstr "" msgstr ""
#: paperless/models.py:383 #: paperless/models.py:376
msgid "Sets the LLM embedding chunk size" msgid "Sets the LLM embedding chunk size"
msgstr "" msgstr ""
#: paperless/models.py:389 #: paperless/models.py:382
msgid "Sets the LLM context size" msgid "Sets the LLM context size"
msgstr "" msgstr ""
#: paperless/models.py:395 #: paperless/models.py:388
msgid "Sets the LLM backend" msgid "Sets the LLM backend"
msgstr "" msgstr ""
#: paperless/models.py:403 #: paperless/models.py:396
msgid "Sets the LLM model" msgid "Sets the LLM model"
msgstr "" msgstr ""
#: paperless/models.py:410 #: paperless/models.py:403
msgid "Sets the LLM API key" msgid "Sets the LLM API key"
msgstr "" msgstr ""
#: paperless/models.py:417 #: paperless/models.py:410
msgid "Sets the LLM endpoint, optional" msgid "Sets the LLM endpoint, optional"
msgstr "" msgstr ""
#: paperless/models.py:424 #: paperless/models.py:417
msgid "Sets the LLM output language" msgid "Sets the LLM output language"
msgstr "" msgstr ""
#: paperless/models.py:431 #: paperless/models.py:424
msgid "Sets the LLM timeout in seconds" msgid "Sets the LLM timeout in seconds"
msgstr "" msgstr ""
#: paperless/models.py:437 #: paperless/models.py:430
msgid "paperless application settings" msgid "paperless application settings"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:559 #: paperless/settings/__init__.py:558
msgid "English (US)" msgid "English (US)"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:560 #: paperless/settings/__init__.py:559
msgid "Arabic" msgid "Arabic"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:561 #: paperless/settings/__init__.py:560
msgid "Afrikaans" msgid "Afrikaans"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:562 #: paperless/settings/__init__.py:561
msgid "Belarusian" msgid "Belarusian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:563 #: paperless/settings/__init__.py:562
msgid "Bulgarian" msgid "Bulgarian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:564 #: paperless/settings/__init__.py:563
msgid "Catalan" msgid "Catalan"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:565 #: paperless/settings/__init__.py:564
msgid "Czech" msgid "Czech"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:566 #: paperless/settings/__init__.py:565
msgid "Danish" msgid "Danish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:567 #: paperless/settings/__init__.py:566
msgid "German" msgid "German"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:568 #: paperless/settings/__init__.py:567
msgid "Greek" msgid "Greek"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:569 #: paperless/settings/__init__.py:568
msgid "English (GB)" msgid "English (GB)"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:570 #: paperless/settings/__init__.py:569
msgid "Spanish" msgid "Spanish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:571 #: paperless/settings/__init__.py:570
msgid "Persian" msgid "Persian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:572 #: paperless/settings/__init__.py:571
msgid "Finnish" msgid "Finnish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:573 #: paperless/settings/__init__.py:572
msgid "French" msgid "French"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:574 #: paperless/settings/__init__.py:573
msgid "Hungarian" msgid "Hungarian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:575 #: paperless/settings/__init__.py:574
msgid "Indonesian" msgid "Indonesian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:576 #: paperless/settings/__init__.py:575
msgid "Italian" msgid "Italian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:577 #: paperless/settings/__init__.py:576
msgid "Japanese" msgid "Japanese"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:578 #: paperless/settings/__init__.py:577
msgid "Korean" msgid "Korean"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:579 #: paperless/settings/__init__.py:578
msgid "Luxembourgish" msgid "Luxembourgish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:580 #: paperless/settings/__init__.py:579
msgid "Norwegian" msgid "Norwegian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:581 #: paperless/settings/__init__.py:580
msgid "Dutch" msgid "Dutch"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:582 #: paperless/settings/__init__.py:581
msgid "Polish" msgid "Polish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:583 #: paperless/settings/__init__.py:582
msgid "Portuguese (Brazil)" msgid "Portuguese (Brazil)"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:584 #: paperless/settings/__init__.py:583
msgid "Portuguese" msgid "Portuguese"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:585 #: paperless/settings/__init__.py:584
msgid "Romanian" msgid "Romanian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:586 #: paperless/settings/__init__.py:585
msgid "Russian" msgid "Russian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:587 #: paperless/settings/__init__.py:586
msgid "Slovak" msgid "Slovak"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:588 #: paperless/settings/__init__.py:587
msgid "Slovenian" msgid "Slovenian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:589 #: paperless/settings/__init__.py:588
msgid "Serbian" msgid "Serbian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:590 #: paperless/settings/__init__.py:589
msgid "Swedish" msgid "Swedish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:591 #: paperless/settings/__init__.py:590
msgid "Turkish" msgid "Turkish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:592 #: paperless/settings/__init__.py:591
msgid "Ukrainian" msgid "Ukrainian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:593 #: paperless/settings/__init__.py:592
msgid "Vietnamese" msgid "Vietnamese"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:594 #: paperless/settings/__init__.py:593
msgid "Chinese Simplified" msgid "Chinese Simplified"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:595 #: paperless/settings/__init__.py:594
msgid "Chinese Traditional" msgid "Chinese Traditional"
msgstr "" msgstr ""
-7
View File
@@ -1,6 +1,5 @@
import dataclasses import dataclasses
import json import json
from typing import Any
from django.conf import settings from django.conf import settings
@@ -245,7 +244,6 @@ class AIConfig(BaseConfig):
ai_enabled: bool = dataclasses.field(init=False) ai_enabled: bool = dataclasses.field(init=False)
llm_embedding_backend: str = dataclasses.field(init=False) llm_embedding_backend: str = dataclasses.field(init=False)
llm_embedding_model: str = dataclasses.field(init=False) llm_embedding_model: str = dataclasses.field(init=False)
llm_embedding_api_key: str = dataclasses.field(init=False)
llm_embedding_endpoint: str = dataclasses.field(init=False) llm_embedding_endpoint: str = dataclasses.field(init=False)
llm_embedding_chunk_size: int = dataclasses.field(init=False) llm_embedding_chunk_size: int = dataclasses.field(init=False)
llm_context_size: int = dataclasses.field(init=False) llm_context_size: int = dataclasses.field(init=False)
@@ -256,7 +254,6 @@ class AIConfig(BaseConfig):
llm_endpoint: str = dataclasses.field(init=False) llm_endpoint: str = dataclasses.field(init=False)
llm_output_language: str = dataclasses.field(init=False) llm_output_language: str = dataclasses.field(init=False)
llm_allow_internal_endpoints: bool = dataclasses.field(init=False) llm_allow_internal_endpoints: bool = dataclasses.field(init=False)
llm_extra_params: dict[str, Any] = dataclasses.field(init=False)
def __post_init__(self) -> None: def __post_init__(self) -> None:
app_config = self._get_config_instance() app_config = self._get_config_instance()
@@ -272,9 +269,6 @@ class AIConfig(BaseConfig):
self.llm_embedding_model = ( self.llm_embedding_model = (
app_config.llm_embedding_model or settings.LLM_EMBEDDING_MODEL app_config.llm_embedding_model or settings.LLM_EMBEDDING_MODEL
) )
self.llm_embedding_api_key = (
app_config.llm_embedding_api_key or settings.LLM_EMBEDDING_API_KEY
)
self.llm_embedding_endpoint = ( self.llm_embedding_endpoint = (
app_config.llm_embedding_endpoint or settings.LLM_EMBEDDING_ENDPOINT app_config.llm_embedding_endpoint or settings.LLM_EMBEDDING_ENDPOINT
) )
@@ -293,7 +287,6 @@ class AIConfig(BaseConfig):
app_config.llm_output_language or settings.LLM_OUTPUT_LANGUAGE app_config.llm_output_language or settings.LLM_OUTPUT_LANGUAGE
) )
self.llm_allow_internal_endpoints = settings.LLM_ALLOW_INTERNAL_ENDPOINTS self.llm_allow_internal_endpoints = settings.LLM_ALLOW_INTERNAL_ENDPOINTS
self.llm_extra_params = settings.LLM_EXTRA_PARAMS
@property @property
def llm_index_enabled(self) -> bool: def llm_index_enabled(self) -> bool:
@@ -1,23 +0,0 @@
# Generated by Django 5.2.16 on 2026-09-11 09:32
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("paperless", "0016_alter_applicationconfiguration_ai_enabled"),
]
operations = [
migrations.AddField(
model_name="applicationconfiguration",
name="llm_embedding_api_key",
field=models.CharField(
blank=True,
max_length=1024,
null=True,
verbose_name="Sets the LLM embedding API key",
),
),
]
-7
View File
@@ -365,13 +365,6 @@ class ApplicationConfiguration(AbstractSingletonModel):
max_length=128, max_length=128,
) )
llm_embedding_api_key = models.CharField(
verbose_name=_("Sets the LLM embedding API key"),
blank=True,
null=True,
max_length=1024,
)
llm_embedding_endpoint = models.CharField( llm_embedding_endpoint = models.CharField(
verbose_name=_("Sets the LLM embedding endpoint, optional"), verbose_name=_("Sets the LLM embedding endpoint, optional"),
blank=True, blank=True,
+1 -5
View File
@@ -394,11 +394,7 @@ class RasterisedDocumentParser:
plain_pdf_path = Path(self.tempdir) / "image_plain.pdf" plain_pdf_path = Path(self.tempdir) / "image_plain.pdf"
try: try:
convert_kwargs: dict = { convert_kwargs: dict = {}
# Ignore invalid EXIF orientation values (e.g. 0) instead of
# aborting the conversion; valid values are still applied
"rotation": img2pdf.Rotation.ifvalid,
}
if self.settings.image_dpi is not None: if self.settings.image_dpi is not None:
convert_kwargs["layout_fun"] = img2pdf.get_fixed_dpi_layout_fun( convert_kwargs["layout_fun"] = img2pdf.get_fixed_dpi_layout_fun(
(self.settings.image_dpi, self.settings.image_dpi), (self.settings.image_dpi, self.settings.image_dpi),
+1 -10
View File
@@ -216,11 +216,6 @@ class ApplicationConfigurationSerializer(
externally_configured_variables = serializers.SerializerMethodField() externally_configured_variables = serializers.SerializerMethodField()
user_args = serializers.JSONField(binary=True, allow_null=True) user_args = serializers.JSONField(binary=True, allow_null=True)
barcode_tag_mapping = serializers.JSONField(binary=True, allow_null=True) barcode_tag_mapping = serializers.JSONField(binary=True, allow_null=True)
llm_embedding_api_key = ObfuscatedPasswordField(
required=False,
allow_null=True,
max_length=1024,
)
llm_api_key = ObfuscatedPasswordField( llm_api_key = ObfuscatedPasswordField(
required=False, required=False,
allow_null=True, allow_null=True,
@@ -232,11 +227,7 @@ class ApplicationConfigurationSerializer(
max_length=1024, max_length=1024,
) )
OBFUSCATED_FIELDS = ( OBFUSCATED_FIELDS = ("llm_api_key", "remote_ocr_api_key")
"llm_embedding_api_key",
"llm_api_key",
"remote_ocr_api_key",
)
def get_externally_configured_variables( def get_externally_configured_variables(
self, self,
-22
View File
@@ -7,7 +7,6 @@ import multiprocessing
import os import os
import tempfile import tempfile
from pathlib import Path from pathlib import Path
from typing import Any
from typing import Final from typing import Final
from urllib.parse import urlparse from urllib.parse import urlparse
@@ -1082,25 +1081,6 @@ CLASSIFIER_LANGUAGES: Final[dict[str, str]] = {
} }
def _get_llm_extra_params() -> dict[str, Any]:
"""
Parse PAPERLESS_AI_LLM_EXTRA_PARAMS, a JSON object passed straight through
to the LLM backend's request body.
"""
raw = os.getenv("PAPERLESS_AI_LLM_EXTRA_PARAMS", "{}")
try:
parsed = json.loads(raw)
except json.JSONDecodeError as e:
raise ImproperlyConfigured(
"PAPERLESS_AI_LLM_EXTRA_PARAMS must be valid JSON",
) from e
if not isinstance(parsed, dict):
raise ImproperlyConfigured(
"PAPERLESS_AI_LLM_EXTRA_PARAMS must be a JSON object",
)
return parsed
def _get_classifier_language_setting(ocr_lang: str) -> str | None: def _get_classifier_language_setting(ocr_lang: str) -> str | None:
""" """
Maps the primary Tesseract language to the classifier's stemming Maps the primary Tesseract language to the classifier's stemming
@@ -1236,7 +1216,6 @@ LLM_EMBEDDING_BACKEND = get_choice_from_env(
{"huggingface", "openai-like", "ollama"}, {"huggingface", "openai-like", "ollama"},
) )
LLM_EMBEDDING_MODEL = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_MODEL") LLM_EMBEDDING_MODEL = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_MODEL")
LLM_EMBEDDING_API_KEY = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_API_KEY")
LLM_EMBEDDING_ENDPOINT = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT") LLM_EMBEDDING_ENDPOINT = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT")
LLM_EMBEDDING_CHUNK_SIZE = get_int_from_env( LLM_EMBEDDING_CHUNK_SIZE = get_int_from_env(
"PAPERLESS_AI_LLM_EMBEDDING_CHUNK_SIZE", "PAPERLESS_AI_LLM_EMBEDDING_CHUNK_SIZE",
@@ -1262,4 +1241,3 @@ LLM_ALLOW_INTERNAL_ENDPOINTS = get_bool_from_env(
"PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS", "PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS",
"true", "true",
) )
LLM_EXTRA_PARAMS = _get_llm_extra_params()
+4 -4
View File
@@ -22,11 +22,11 @@ if TYPE_CHECKING:
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def parser_samples_dir() -> Path: def samples_dir() -> Path:
"""Absolute path to the shared parser sample files directory. """Absolute path to the shared parser sample files directory.
Sub-package conftest files derive format-specific paths from this root, Sub-package conftest files derive format-specific paths from this root,
e.g. ``parser_samples_dir / "text" / "test.txt"``. e.g. ``samples_dir / "text" / "test.txt"``.
Returns Returns
------- -------
@@ -37,7 +37,7 @@ def parser_samples_dir() -> Path:
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def tagged_no_text_pdf_file(parser_samples_dir: Path) -> Path: def tagged_no_text_pdf_file(samples_dir: Path) -> Path:
"""Path to a tagged PDF whose only "text" is pdftotext layout padding. """Path to a tagged PDF whose only "text" is pdftotext layout padding.
Reproduces GH #13387: ``/MarkInfo /Marked true`` is set, but the only Reproduces GH #13387: ``/MarkInfo /Marked true`` is set, but the only
@@ -50,7 +50,7 @@ def tagged_no_text_pdf_file(parser_samples_dir: Path) -> Path:
Path Path
Absolute path to ``tesseract/tagged-but-no-text.pdf``. Absolute path to ``tesseract/tagged-but-no-text.pdf``.
""" """
return parser_samples_dir / "tesseract" / "tagged-but-no-text.pdf" return samples_dir / "tesseract" / "tagged-but-no-text.pdf"
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
+240 -12
View File
@@ -37,15 +37,15 @@ if TYPE_CHECKING:
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def text_samples_dir(parser_samples_dir: Path) -> Path: def text_samples_dir(samples_dir: Path) -> Path:
"""Absolute path to the text parser sample files directory. """Absolute path to the text parser sample files directory.
Returns Returns
------- -------
Path Path
``<parser_samples_dir>/text/`` ``<samples_dir>/text/``
""" """
return parser_samples_dir / "text" return samples_dir / "text"
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
@@ -175,15 +175,15 @@ def no_engine_settings(
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def tika_samples_dir(parser_samples_dir: Path) -> Path: def tika_samples_dir(samples_dir: Path) -> Path:
"""Absolute path to the Tika parser sample files directory. """Absolute path to the Tika parser sample files directory.
Returns Returns
------- -------
Path Path
``<parser_samples_dir>/tika/`` ``<samples_dir>/tika/``
""" """
return parser_samples_dir / "tika" return samples_dir / "tika"
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
@@ -258,15 +258,15 @@ def tika_parser() -> Generator[TikaDocumentParser, None, None]:
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def mail_samples_dir(parser_samples_dir: Path) -> Path: def mail_samples_dir(samples_dir: Path) -> Path:
"""Absolute path to the mail parser sample files directory. """Absolute path to the mail parser sample files directory.
Returns Returns
------- -------
Path Path
``<parser_samples_dir>/mail/`` ``<samples_dir>/mail/``
""" """
return parser_samples_dir / "mail" return samples_dir / "mail"
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
@@ -421,15 +421,75 @@ def nginx_base_url() -> Generator[str, None, None]:
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def tesseract_samples_dir(parser_samples_dir: Path) -> Path: def tesseract_samples_dir(samples_dir: Path) -> Path:
"""Absolute path to the tesseract parser sample files directory. """Absolute path to the tesseract parser sample files directory.
Returns Returns
------- -------
Path Path
``<parser_samples_dir>/tesseract/`` ``<samples_dir>/tesseract/``
""" """
return parser_samples_dir / "tesseract" return samples_dir / "tesseract"
@pytest.fixture(scope="session")
def document_webp_file(tesseract_samples_dir: Path) -> Path:
"""Path to a WebP document sample file.
Returns
-------
Path
Absolute path to ``tesseract/document.webp``.
"""
return tesseract_samples_dir / "document.webp"
@pytest.fixture(scope="session")
def encrypted_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to an encrypted PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/encrypted.pdf``.
"""
return tesseract_samples_dir / "encrypted.pdf"
@pytest.fixture(scope="session")
def multi_page_digital_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page digital PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-digital.pdf``.
"""
return tesseract_samples_dir / "multi-page-digital.pdf"
@pytest.fixture(scope="session")
def multi_page_images_alpha_rgb_tiff_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page TIFF with alpha channel in RGB.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-images-alpha-rgb.tiff``.
"""
return tesseract_samples_dir / "multi-page-images-alpha-rgb.tiff"
@pytest.fixture(scope="session")
def multi_page_images_alpha_tiff_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page TIFF with alpha channel.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-images-alpha.tiff``.
"""
return tesseract_samples_dir / "multi-page-images-alpha.tiff"
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
@@ -444,6 +504,90 @@ def multi_page_images_pdf_file(tesseract_samples_dir: Path) -> Path:
return tesseract_samples_dir / "multi-page-images.pdf" return tesseract_samples_dir / "multi-page-images.pdf"
@pytest.fixture(scope="session")
def multi_page_images_tiff_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page TIFF sample file.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-images.tiff``.
"""
return tesseract_samples_dir / "multi-page-images.tiff"
@pytest.fixture(scope="session")
def multi_page_mixed_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page mixed PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-mixed.pdf``.
"""
return tesseract_samples_dir / "multi-page-mixed.pdf"
@pytest.fixture(scope="session")
def no_text_alpha_png_file(tesseract_samples_dir: Path) -> Path:
"""Path to a PNG with alpha channel and no text.
Returns
-------
Path
Absolute path to ``tesseract/no-text-alpha.png``.
"""
return tesseract_samples_dir / "no-text-alpha.png"
@pytest.fixture(scope="session")
def rotated_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a rotated PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/rotated.pdf``.
"""
return tesseract_samples_dir / "rotated.pdf"
@pytest.fixture(scope="session")
def rtl_test_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to an RTL test PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/rtl-test.pdf``.
"""
return tesseract_samples_dir / "rtl-test.pdf"
@pytest.fixture(scope="session")
def signed_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a signed PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/signed.pdf``.
"""
return tesseract_samples_dir / "signed.pdf"
@pytest.fixture(scope="session")
def simple_alpha_png_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple PNG with alpha channel.
Returns
-------
Path
Absolute path to ``tesseract/simple-alpha.png``.
"""
return tesseract_samples_dir / "simple-alpha.png"
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def simple_digital_pdf_file(tesseract_samples_dir: Path) -> Path: def simple_digital_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple digital PDF sample file. """Path to a simple digital PDF sample file.
@@ -468,6 +612,54 @@ def simple_no_dpi_png_file(tesseract_samples_dir: Path) -> Path:
return tesseract_samples_dir / "simple-no-dpi.png" return tesseract_samples_dir / "simple-no-dpi.png"
@pytest.fixture(scope="session")
def simple_bmp_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple BMP sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.bmp``.
"""
return tesseract_samples_dir / "simple.bmp"
@pytest.fixture(scope="session")
def simple_gif_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple GIF sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.gif``.
"""
return tesseract_samples_dir / "simple.gif"
@pytest.fixture(scope="session")
def simple_heic_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple HEIC sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.heic``.
"""
return tesseract_samples_dir / "simple.heic"
@pytest.fixture(scope="session")
def simple_jpg_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple JPG sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.jpg``.
"""
return tesseract_samples_dir / "simple.jpg"
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def simple_png_file(tesseract_samples_dir: Path) -> Path: def simple_png_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple PNG sample file. """Path to a simple PNG sample file.
@@ -480,6 +672,42 @@ def simple_png_file(tesseract_samples_dir: Path) -> Path:
return tesseract_samples_dir / "simple.png" return tesseract_samples_dir / "simple.png"
@pytest.fixture(scope="session")
def simple_tif_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple TIF sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.tif``.
"""
return tesseract_samples_dir / "simple.tif"
@pytest.fixture(scope="session")
def single_page_mixed_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a single-page mixed PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/single-page-mixed.pdf``.
"""
return tesseract_samples_dir / "single-page-mixed.pdf"
@pytest.fixture(scope="session")
def with_form_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a PDF with form sample file.
Returns
-------
Path
Absolute path to ``tesseract/with-form.pdf``.
"""
return tesseract_samples_dir / "with-form.pdf"
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Tesseract parser instance and settings helpers # Tesseract parser instance and settings helpers
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@@ -18,7 +18,6 @@ import img2pdf
import magic import magic
import pikepdf import pikepdf
import pytest import pytest
from PIL import Image
from documents.parsers import ParseError from documents.parsers import ParseError
@@ -140,24 +139,3 @@ class TestConvertImageToPdfa:
tesseract_parser._convert_image_to_pdfa(simple_png_file) tesseract_parser._convert_image_to_pdfa(simple_png_file)
spy.assert_not_called() spy.assert_not_called()
def test_invalid_exif_orientation_is_ignored(
self,
tesseract_parser: RasterisedDocumentParser,
tmp_path: Path,
) -> None:
"""
GIVEN: a JPEG with an invalid EXIF orientation value (0)
WHEN: _convert_image_to_pdfa is called
THEN: the invalid value is ignored and a valid PDF is produced
"""
image_path = tmp_path / "invalid_orientation.jpg"
with Image.new("RGB", (120, 80), "white") as image:
exif = image.getexif()
exif[274] = 0 # EXIF tag 274: Orientation
image.save(image_path, exif=exif)
result = tesseract_parser._convert_image_to_pdfa(image_path)
assert result.exists()
assert magic.from_file(str(result), mime=True) == "application/pdf"
@@ -10,8 +10,8 @@ from imagehash import average_hash
from PIL import Image from PIL import Image
from pytest_mock import MockerFixture from pytest_mock import MockerFixture
from documents.tests.utils import util_call_with_backoff
from paperless.parsers.mail import MailDocumentParser from paperless.parsers.mail import MailDocumentParser
from paperless_testing.retry import util_call_with_backoff
def extract_text(pdf_path: Path) -> str: def extract_text(pdf_path: Path) -> str:
@@ -3,14 +3,14 @@ import json
from django.test import TestCase from django.test import TestCase
from django.test import override_settings from django.test import override_settings
from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin
from paperless.models import ApplicationConfiguration from paperless.models import ApplicationConfiguration
from paperless.models import CleanChoices from paperless.models import CleanChoices
from paperless.models import ColorConvertChoices from paperless.models import ColorConvertChoices
from paperless.models import ModeChoices from paperless.models import ModeChoices
from paperless.models import OutputTypeChoices from paperless.models import OutputTypeChoices
from paperless.parsers.tesseract import RasterisedDocumentParser from paperless.parsers.tesseract import RasterisedDocumentParser
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
class TestParserSettingsFromDb(DirectoriesMixin, FileSystemAssertsMixin, TestCase): class TestParserSettingsFromDb(DirectoriesMixin, FileSystemAssertsMixin, TestCase):

Some files were not shown because too many files have changed in this diff Show More