Compare commits

...
11 Commits
17 changed files with 317 additions and 52 deletions
@@ -68,7 +68,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
@@ -81,7 +81,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
@@ -76,7 +76,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
@@ -65,7 +65,7 @@ services:
- "--chromium-disable-javascript=true" - "--chromium-disable-javascript=true"
- "--chromium-allow-list=file:///tmp/.*" - "--chromium-allow-list=file:///tmp/.*"
tika: tika:
image: docker.io/apache/tika:latest image: docker.io/apache/tika:3.3.1.0
restart: unless-stopped restart: unless-stopped
volumes: volumes:
data: data:
+6 -1
View File
@@ -129,6 +129,10 @@ At a minimum you need to enable AI and choose an LLM backend:
and/or [`PAPERLESS_AI_LLM_ENDPOINT`](configuration.md#PAPERLESS_AI_LLM_ENDPOINT). Ollama and/or [`PAPERLESS_AI_LLM_ENDPOINT`](configuration.md#PAPERLESS_AI_LLM_ENDPOINT). Ollama
requires `PAPERLESS_AI_LLM_ENDPOINT` pointing at your Ollama server. requires `PAPERLESS_AI_LLM_ENDPOINT` pointing at your Ollama server.
See the community-maintained wiki page on
[choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for suggested generation and embedding models.
### AI-assisted suggestions ### AI-assisted suggestions
With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type, With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type,
@@ -808,7 +812,8 @@ Third-party parser plugins extend Paperless-ngx to support additional file
formats. A plugin is a Python package that advertises itself under the formats. A plugin is a Python package that advertises itself under the
`paperless_ngx.parsers` entry point group. Refer to the `paperless_ngx.parsers` entry point group. Refer to the
[developer documentation](development.md#making-custom-parsers) for how to [developer documentation](development.md#making-custom-parsers) for how to
create one. create one, or see the wiki for a community-maintained list of
[parser plugins](https://github.com/paperless-ngx/paperless-ngx/wiki/Related-Projects#parser-plugins).
!!! warning "Third-party plugins are not officially supported" !!! warning "Third-party plugins are not officially supported"
+5 -1
View File
@@ -1215,7 +1215,7 @@ should be a valid crontab(5) expression describing when to run.
: If set to the string "disable", no emails will be fetched automatically. : If set to the string "disable", no emails will be fetched automatically.
Defaults to `*/10 * * * *` or every ten minutes. Defaults to every ten minutes, with an installation-specific minute offset.
#### [`PAPERLESS_TRAIN_TASK_CRON=<cron expression>`](#PAPERLESS_TRAIN_TASK_CRON) {#PAPERLESS_TRAIN_TASK_CRON} #### [`PAPERLESS_TRAIN_TASK_CRON=<cron expression>`](#PAPERLESS_TRAIN_TASK_CRON) {#PAPERLESS_TRAIN_TASK_CRON}
@@ -2088,6 +2088,8 @@ suggestions. This setting is required to be set to true in order to use the AI f
models supported by the current embedding backend. If not supplied, defaults to models supported by the current embedding backend. If not supplied, defaults to
"text-embedding-3-small" for the OpenAI-compatible backend, "text-embedding-3-small" for the OpenAI-compatible backend,
"sentence-transformers/all-MiniLM-L6-v2" for Huggingface, and "embeddinggemma" for Ollama. "sentence-transformers/all-MiniLM-L6-v2" for Huggingface, and "embeddinggemma" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for language and resource considerations.
Defaults to None. Defaults to None.
@@ -2144,6 +2146,8 @@ setting is required to be set to use the AI features.
: The model to use for the AI backend, i.e. "gpt-3.5-turbo", "gpt-4" or any of the models supported : The model to use for the AI backend, i.e. "gpt-3.5-turbo", "gpt-4" or any of the models supported
by the current backend. If not supplied, defaults to "gpt-3.5-turbo" for the OpenAI-compatible by the current backend. If not supplied, defaults to "gpt-3.5-turbo" for the OpenAI-compatible
backend and "llama3.1" for Ollama. backend and "llama3.1" for Ollama.
See [choosing AI models](https://github.com/paperless-ngx/paperless-ngx/wiki/AI-Model-Recommendations)
for local versus remote and model-size considerations.
Defaults to None. Defaults to None.
+4 -3
View File
@@ -156,7 +156,7 @@ The new settings are independent:
### Database configuration ### Database configuration
If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and `skip_archive_file` values are converted to the new `archive_file_generation` field. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent. If you changed OCR settings via the admin UI (ApplicationConfiguration), the database values are **migrated automatically** during the upgrade. `mode` values (`skip` / `skip_noarchive`) are mapped to their new equivalents and explicit `skip_archive_file` values are converted to the new `archive_file_generation` field. Users who relied on the old defaults must set `archive_file_generation` to `always` to preserve the v2 behaviour of always creating an archive. After upgrading, review the OCR settings in the admin UI to confirm the migrated values match your intent.
### Action Required ### Action Required
@@ -165,8 +165,9 @@ Remove any `PAPERLESS_OCR_SKIP_ARCHIVE_FILE` variable from your environment. If
```bash ```bash
# v2: skip OCR when text present, always archive # v2: skip OCR when text present, always archive
PAPERLESS_OCR_MODE=skip PAPERLESS_OCR_MODE=skip
# v3: equivalent (auto is the new default) # v3: equivalent
# No change needed - auto is the default PAPERLESS_OCR_MODE=auto
PAPERLESS_ARCHIVE_FILE_GENERATION=always
# v2: skip OCR when text present, skip archive too # v2: skip OCR when text present, skip archive too
PAPERLESS_OCR_MODE=skip_noarchive PAPERLESS_OCR_MODE=skip_noarchive
+17 -17
View File
@@ -2195,7 +2195,7 @@
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">64,65</context> <context context-type="linenumber">67,68</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="2134950584701094962" datatype="html"> <trans-unit id="2134950584701094962" datatype="html">
@@ -6180,7 +6180,7 @@
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">35,36</context> <context context-type="linenumber">38,39</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="5554528553553249088" datatype="html"> <trans-unit id="5554528553553249088" datatype="html">
@@ -8361,11 +8361,11 @@
<source>An error occurred loading tiff: <x id="PH" equiv-text="err.toString()"/></source> <source>An error occurred loading tiff: <x id="PH" equiv-text="err.toString()"/></source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-detail.component.ts</context> <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.ts</context>
<context context-type="linenumber">2023</context> <context context-type="linenumber">2025</context>
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-detail.component.ts</context> <context context-type="sourcefile">src/app/components/document-detail/document-detail.component.ts</context>
<context context-type="linenumber">2029</context> <context context-type="linenumber">2031</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4958946940233632319" datatype="html"> <trans-unit id="4958946940233632319" datatype="html">
@@ -8407,88 +8407,88 @@
<source>Label</source> <source>Label</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">10,11</context> <context context-type="linenumber">13,14</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3915966149686975421" datatype="html"> <trans-unit id="3915966149686975421" datatype="html">
<source>Optional</source> <source>Optional</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">17,18</context> <context context-type="linenumber">20,21</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3009569956897708604" datatype="html"> <trans-unit id="3009569956897708604" datatype="html">
<source>Upload a new version</source> <source>Upload a new version</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">32,33</context> <context context-type="linenumber">35,36</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="278763911843514709" datatype="html"> <trans-unit id="278763911843514709" datatype="html">
<source>Use an existing document</source> <source>Use an existing document</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">41,42</context> <context context-type="linenumber">44,45</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="1906507263389090131" datatype="html"> <trans-unit id="1906507263389090131" datatype="html">
<source>Existing</source> <source>Existing</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">44,45</context> <context context-type="linenumber">47,48</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3119565892291077820" datatype="html"> <trans-unit id="3119565892291077820" datatype="html">
<source>Uploading version...</source> <source>Uploading version...</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">52,53</context> <context context-type="linenumber">55,56</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="8538204518127762311" datatype="html"> <trans-unit id="8538204518127762311" datatype="html">
<source>Processing version...</source> <source>Processing version...</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">58,59</context> <context context-type="linenumber">61,62</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6587183611950398104" datatype="html"> <trans-unit id="6587183611950398104" datatype="html">
<source>Version upload failed.</source> <source>Version upload failed.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">63,64</context> <context context-type="linenumber">66,67</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6328986243040954523" datatype="html"> <trans-unit id="6328986243040954523" datatype="html">
<source>Version label</source> <source>Version label</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">88,89</context> <context context-type="linenumber">91,92</context>
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">98,99</context> <context context-type="linenumber">101,102</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="2724055831234181057" datatype="html"> <trans-unit id="2724055831234181057" datatype="html">
<source>Version</source> <source>Version</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">109</context> <context context-type="linenumber">112</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="2491751767883178298" datatype="html"> <trans-unit id="2491751767883178298" datatype="html">
<source>Delete this version?</source> <source>Delete this version?</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">140,141</context> <context context-type="linenumber">143,144</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="894108684749537200" datatype="html"> <trans-unit id="894108684749537200" datatype="html">
<source>Delete version</source> <source>Delete version</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context> <context context-type="sourcefile">src/app/components/document-detail/document-version-dropdown/document-version-dropdown.component.html</context>
<context context-type="linenumber">146,147</context> <context context-type="linenumber">149,150</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="9111284310237278846" datatype="html"> <trans-unit id="9111284310237278846" datatype="html">
@@ -2171,6 +2171,11 @@ describe('DocumentDetailComponent', () => {
).toBe(10) ).toBe(10)
component.openEmailDocument() component.openEmailDocument()
expect(modalSpy).toHaveBeenCalled() expect(modalSpy).toHaveBeenCalled()
expect(
(
modalSpy.mock.results[1].value as NgbModalRef
).componentInstance.documentIds()
).toEqual([10])
}) })
it('should set previewText', () => { it('should set previewText', () => {
@@ -1973,7 +1973,9 @@ export class DocumentDetailComponent
const modal = this.modalService.open(EmailDocumentDialogComponent, { const modal = this.modalService.open(EmailDocumentDialogComponent, {
backdrop: 'static', backdrop: 'static',
}) })
modal.componentInstance.documentIds.set([this.document().id]) modal.componentInstance.documentIds.set([
this.selectedVersionId() ?? this.document().id,
])
modal.componentInstance.hasArchiveVersion.set( modal.componentInstance.hasArchiveVersion.set(
this.metadata()?.has_archive_version ?? this.metadata()?.has_archive_version ??
!!this.document()?.archived_file_name !!this.document()?.archived_file_name
@@ -2,6 +2,9 @@
<button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle> <button class="btn btn-sm btn-outline-secondary dropdown-toggle" ngbDropdownToggle>
<i-bs name="file-earmark-diff"></i-bs> <i-bs name="file-earmark-diff"></i-bs>
<span class="d-none d-lg-inline ps-1" i18n>Versions</span> <span class="d-none d-lg-inline ps-1" i18n>Versions</span>
@if (versions.length > 1) {
<span class="badge text-bg-secondary ms-1">{{ versions.length }}</span>
}
</button> </button>
<div class="dropdown-menu shadow" ngbDropdownMenu> <div class="dropdown-menu shadow" ngbDropdownMenu>
<div class="px-3 py-2 mb-2"> <div class="px-3 py-2 mb-2">
+50
View File
@@ -497,6 +497,56 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
) )
response.close() response.close()
@override_settings(FILENAME_FORMAT="")
def test_serve_text_file_declares_utf8_charset(self) -> None:
"""
GIVEN:
- A UTF-8 encoded text document
WHEN:
- The file is served for preview or download
THEN:
- The Content-Type declares the UTF-8 charset, so the browser does
not fall back to its locale default and mangle non-ASCII text
"""
doc = Document.objects.create(
title="none",
filename="my_document.txt",
mime_type="text/plain",
)
Path(doc.source_path).write_bytes("für Grüße München".encode())
for endpoint in ("preview", "download"):
with self.subTest(endpoint=endpoint):
response = self.client.get(f"/api/documents/{doc.pk}/{endpoint}/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response["Content-Type"], "text/plain; charset=utf-8")
self.assertEqual(
read_streaming_response(response).decode("utf-8"),
"für Grüße München",
)
@override_settings(FILENAME_FORMAT="")
def test_serve_pdf_file_has_no_charset(self) -> None:
"""
GIVEN:
- A PDF document
WHEN:
- The file is served for preview
THEN:
- No charset is added to the binary content type
"""
doc = Document.objects.create(
title="none",
filename="my_document.pdf",
mime_type="application/pdf",
)
Path(doc.source_path).write_bytes(b"This is a test")
response = self.client.get(f"/api/documents/{doc.pk}/preview/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response["Content-Type"], "application/pdf")
response.close()
def test_document_actions_not_existing_file(self) -> None: def test_document_actions_not_existing_file(self) -> None:
doc = Document.objects.create( doc = Document.objects.create(
title="none", title="none",
+139
View File
@@ -1360,6 +1360,145 @@ class TestBulkEditObjectPermissions(APITestCase):
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertEqual(response.content, b"Insufficient permissions") self.assertEqual(response.content, b"Insufficient permissions")
def test_bulk_edit_object_permissions_shared_object_not_owner(self) -> None:
"""
GIVEN:
- Object owned by another user, shared with the logged in user with
change permissions
WHEN:
- bulk_edit_objects API endpoint is called with set_permissions operation
THEN:
- User is not able to take ownership or change permissions, consistent
with the single object API
"""
self.t1.owner = self.user2
self.t1.save()
assign_perm("view_tag", self.user1, self.t1)
assign_perm("change_tag", self.user1, self.t1)
self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag"],
),
)
user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1)
response = self.client.post(
"/api/bulk_edit_objects/",
json.dumps(
{
"objects": [self.t1.id],
"object_type": "tags",
"operation": "set_permissions",
"owner": user1.id,
"permissions": {
"view": {"users": [user1.id], "groups": []},
"change": {"users": [user1.id], "groups": []},
},
"merge": False,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertEqual(Tag.objects.get(pk=self.t1.id).owner, self.user2)
# the single object endpoint refuses the same request
response = self.client.patch(
f"/api/tags/{self.t1.id}/",
{"owner": user1.id},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertEqual(Tag.objects.get(pk=self.t1.id).owner, self.user2)
def test_bulk_edit_object_permissions_all_with_shared_objects(self) -> None:
"""
GIVEN:
- Objects owned by the logged in user, unowned objects and objects owned
by another user but shared with the logged in user
WHEN:
- bulk_edit_objects API endpoint is called with set_permissions operation
and all = True
THEN:
- The request is refused and no objects are changed
"""
owned = Tag.objects.create(name="owned", owner=self.user1)
shared = Tag.objects.create(name="shared", owner=self.user2)
assign_perm("view_tag", self.user1, shared)
assign_perm("change_tag", self.user1, shared)
self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag"],
),
)
user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1)
response = self.client.post(
"/api/bulk_edit_objects/",
json.dumps(
{
"objects": [],
"all": True,
"object_type": "tags",
"operation": "set_permissions",
"permissions": {
"view": {"users": [self.user3.id], "groups": []},
"change": {"users": [self.user3.id], "groups": []},
},
"merge": False,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
# nothing was changed, including the objects the user does own
self.assertNotIn(self.user3, get_users_with_perms(owned))
self.assertNotIn(self.user3, get_users_with_perms(self.t1))
self.assertNotIn(self.user3, get_users_with_perms(shared))
self.assertEqual(Tag.objects.get(pk=shared.pk).owner, self.user2)
def test_bulk_edit_object_delete_shared_object_not_owner(self) -> None:
"""
GIVEN:
- Object owned by another user, shared with the logged in user with
change and delete permissions
WHEN:
- bulk_edit_objects API endpoint is called with delete operation
THEN:
- User is not able to delete the object, consistent with documents
"""
self.t1.owner = self.user2
self.t1.save()
assign_perm("view_tag", self.user1, self.t1)
assign_perm("change_tag", self.user1, self.t1)
assign_perm("delete_tag", self.user1, self.t1)
self.user1.user_permissions.add(
*Permission.objects.filter(
codename__in=["view_tag", "change_tag", "delete_tag"],
),
)
user1 = User.objects.get(pk=self.user1.pk)
self.client.force_authenticate(user=user1)
response = self.client.post(
"/api/bulk_edit_objects/",
json.dumps(
{
"objects": [self.t1.id],
"object_type": "tags",
"operation": "delete",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertTrue(Tag.objects.filter(pk=self.t1.id).exists())
def test_bulk_edit_object_permissions_validation(self) -> None: def test_bulk_edit_object_permissions_validation(self) -> None:
""" """
GIVEN: GIVEN:
@@ -677,16 +677,54 @@ class TestBulkEditObjectsApplyToAllPermissionBoundary:
def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client): def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client):
owner = User.objects.create_user(username="tags_owner") owner = User.objects.create_user(username="tags_owner")
requester = User.objects.create_user(username="tags_requester") requester = User.objects.create_user(username="tags_requester")
new_owner = User.objects.create_user(username="tags_new_owner")
# grant the global change_tag permission so the object-level # grant the global change_tag permission so the object-level
# filtering (not the global has_perm check) is what's under test # filtering (not the global has_perm check) is what's under test
requester.user_permissions.add( requester.user_permissions.add(
Permission.objects.get(codename="change_tag"), Permission.objects.get(codename="change_tag"),
) )
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
visible = TagFactory(owner=owner) visible = TagFactory(owner=requester)
hidden = TagFactory(owner=owner) hidden = TagFactory(owner=owner)
assign_perm("view_tag", requester, visible)
assign_perm("change_tag", requester, visible) response = rest_api_client.post(
"/api/bulk_edit_objects/",
{
"object_type": "tags",
"operation": "set_permissions",
"all": True,
"filters": {},
"owner": new_owner.pk,
},
format="json",
)
assert response.status_code == HTTPStatus.OK
# The apply_to_all dispatch must resolve permitted objects up front:
# the requester's own tag gets its owner reassigned, while the tag
# owned by someone else is excluded entirely and keeps its owner.
visible.refresh_from_db()
hidden.refresh_from_db()
assert visible.owner == new_owner
assert hidden.owner == owner
def test_apply_to_all_tags_refuses_shared_but_unowned_tag(self, rest_api_client):
"""
A tag owned by someone else but shared with the requester is inside the
permitted set, so it reaches the ownership gate and fails the whole
request rather than being silently skipped. Editing permissions is
limited to the owner, same as documents.
"""
owner = User.objects.create_user(username="shared_tags_owner")
requester = User.objects.create_user(username="shared_tags_requester")
requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
rest_api_client.force_authenticate(user=requester)
owned = TagFactory(owner=requester)
shared = TagFactory(owner=owner)
assign_perm("view_tag", requester, shared)
assign_perm("change_tag", requester, shared)
response = rest_api_client.post( response = rest_api_client.post(
"/api/bulk_edit_objects/", "/api/bulk_edit_objects/",
@@ -699,16 +737,12 @@ class TestBulkEditObjectsApplyToAllPermissionBoundary:
}, },
format="json", format="json",
) )
assert response.status_code == HTTPStatus.OK assert response.status_code == HTTPStatus.FORBIDDEN
# The apply_to_all dispatch must resolve permitted objects up front: owned.refresh_from_db()
# the visible tag (object-level change_tag granted) gets its owner shared.refresh_from_db()
# reassigned, while the hidden tag (no object-level grant) is assert shared.owner == owner
# excluded entirely and keeps its original owner. assert owned.owner == requester
visible.refresh_from_db()
hidden.refresh_from_db()
assert visible.owner == requester
assert hidden.owner == owner
@pytest.mark.django_db @pytest.mark.django_db
@@ -720,8 +754,8 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
""" """
GIVEN: GIVEN:
- A tag hierarchy (parent -> permitted_child, unpermitted_child) - A tag hierarchy (parent -> permitted_child, unpermitted_child)
- A non-superuser requester with object-level change_tag granted - A non-superuser requester who owns the parent and only ONE of
on the parent and on only ONE of the two children the two children
WHEN: WHEN:
- bulk_edit_objects is called with all=True and a filter that - bulk_edit_objects is called with all=True and a filter that
matches only the root (parent) tag, engaging the matches only the root (parent) tag, engaging the
@@ -743,6 +777,7 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
""" """
owner = User.objects.create_user(username="tag_hierarchy_owner") owner = User.objects.create_user(username="tag_hierarchy_owner")
requester = User.objects.create_user(username="tag_hierarchy_requester") requester = User.objects.create_user(username="tag_hierarchy_requester")
new_owner = User.objects.create_user(username="tag_hierarchy_new_owner")
# global change_tag permission so the has_perm() gate passes and the # global change_tag permission so the has_perm() gate passes and the
# object-level permitted_object_ids filtering is what's under test # object-level permitted_object_ids filtering is what's under test
requester.user_permissions.add( requester.user_permissions.add(
@@ -750,9 +785,9 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
) )
rest_api_client.force_authenticate(user=requester) rest_api_client.force_authenticate(user=requester)
parent = TagFactory(owner=owner, name="parent-tag") parent = TagFactory(owner=requester, name="parent-tag")
permitted_child = TagFactory( permitted_child = TagFactory(
owner=owner, owner=requester,
name="permitted-child-tag", name="permitted-child-tag",
tn_parent=parent, tn_parent=parent,
) )
@@ -761,9 +796,6 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
name="unpermitted-child-tag", name="unpermitted-child-tag",
tn_parent=parent, tn_parent=parent,
) )
assign_perm("change_tag", requester, parent)
assign_perm("change_tag", requester, permitted_child)
# unpermitted_child is intentionally NOT granted change_tag
response = rest_api_client.post( response = rest_api_client.post(
"/api/bulk_edit_objects/", "/api/bulk_edit_objects/",
@@ -772,7 +804,7 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
"operation": "set_permissions", "operation": "set_permissions",
"all": True, "all": True,
"filters": {"is_root": True}, "filters": {"is_root": True},
"owner": requester.pk, "owner": new_owner.pk,
}, },
format="json", format="json",
) )
@@ -781,8 +813,8 @@ class TestBulkEditObjectsTagDescendantPartialPermission:
parent.refresh_from_db() parent.refresh_from_db()
permitted_child.refresh_from_db() permitted_child.refresh_from_db()
unpermitted_child.refresh_from_db() unpermitted_child.refresh_from_db()
assert parent.owner == requester assert parent.owner == new_owner
assert permitted_child.owner == requester assert permitted_child.owner == new_owner
assert unpermitted_child.owner == owner assert unpermitted_child.owner == owner
+5 -1
View File
@@ -4778,6 +4778,9 @@ def serve_file(
# Support browser previewing csv files by using text mime type # Support browser previewing csv files by using text mime type
if mime_type in {"application/csv", "text/csv"} and disposition == "inline": if mime_type in {"application/csv", "text/csv"} and disposition == "inline":
mime_type = "text/plain" mime_type = "text/plain"
# Tell browsers to use UTF-8 for the text files we parse as UTF-8
if mime_type in {"text/plain", "text/csv", "application/csv"}:
mime_type = f"{mime_type}; charset=utf-8"
response = FileResponse(file_handle, content_type=mime_type) response = FileResponse(file_handle, content_type=mime_type)
# Firefox is not able to handle unicode characters in filename field # Firefox is not able to handle unicode characters in filename field
@@ -4879,10 +4882,11 @@ class BulkEditObjectsView(PassUserMixin):
if not user.is_superuser: if not user.is_superuser:
perm = f"documents.{perm_codename}" perm = f"documents.{perm_codename}"
# Limited to the owner (or unowned), same as documents, see BulkEditView
has_perms = ( has_perms = (
user.has_perm(perm) user.has_perm(perm)
and not objs.exclude( and not objs.exclude(
pk__in=permitted_object_ids(user, object_class, perm_codename), Q(owner=user) | Q(owner__isnull=True),
).exists() ).exists()
) )
+10
View File
@@ -1,6 +1,7 @@
import datetime import datetime
import logging import logging
import os import os
from hashlib import sha256
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@@ -172,6 +173,15 @@ def parse_beat_schedule() -> dict:
# Don't add disabled tasks to the schedule # Don't add disabled tasks to the schedule
if value == "disable": if value == "disable":
continue continue
if (
task["env_key"] == "PAPERLESS_EMAIL_TASK_CRON"
and task["env_key"] not in os.environ
):
# Spread default polling across the ten-minute interval.
secret = os.environ["PAPERLESS_SECRET_KEY"].encode()
offset = int.from_bytes(sha256(secret).digest()) % 10
minutes = ",".join(str(minute) for minute in range(offset, 60, 10))
value = f"{minutes} * * * *"
# I find https://crontab.guru/ super helpful # I find https://crontab.guru/ super helpful
# crontab(5) format # crontab(5) format
# - five time-and-date fields # - five time-and-date fields
@@ -168,6 +168,7 @@ class TestParseHostingSettings:
def make_expected_schedule( def make_expected_schedule(
overrides: dict[str, dict[str, Any]] | None = None, overrides: dict[str, dict[str, Any]] | None = None,
disabled: set[str] | None = None, disabled: set[str] | None = None,
email_minute: str = "6,16,26,36,46,56",
) -> dict[str, Any]: ) -> dict[str, Any]:
""" """
Build the expected schedule with optional overrides and disabled tasks. Build the expected schedule with optional overrides and disabled tasks.
@@ -185,7 +186,7 @@ def make_expected_schedule(
schedule: dict[str, Any] = { schedule: dict[str, Any] = {
"Check all e-mail accounts": { "Check all e-mail accounts": {
"task": "paperless_mail.tasks.process_mail_accounts", "task": "paperless_mail.tasks.process_mail_accounts",
"schedule": crontab(minute="*/10"), "schedule": crontab(minute=email_minute),
"options": { "options": {
"expires": mail_expire, "expires": mail_expire,
"headers": {"trigger_source": "scheduled"}, "headers": {"trigger_source": "scheduled"},
@@ -266,6 +267,11 @@ class TestParseBeatSchedule:
("env", "expected"), ("env", "expected"),
[ [
pytest.param({}, make_expected_schedule(), id="defaults"), pytest.param({}, make_expected_schedule(), id="defaults"),
pytest.param(
{"PAPERLESS_EMAIL_TASK_CRON": "*/10 * * * *"},
make_expected_schedule(email_minute="*/10"),
id="email-explicit-default",
),
pytest.param( pytest.param(
{"PAPERLESS_EMAIL_TASK_CRON": "*/50 * * * mon"}, {"PAPERLESS_EMAIL_TASK_CRON": "*/50 * * * mon"},
make_expected_schedule( make_expected_schedule(
@@ -304,7 +310,11 @@ class TestParseBeatSchedule:
expected: dict[str, Any], expected: dict[str, Any],
mocker: MockerFixture, mocker: MockerFixture,
) -> None: ) -> None:
mocker.patch.dict(os.environ, env, clear=False) mocker.patch.dict(
os.environ,
{"PAPERLESS_SECRET_KEY": "test-secret", **env},
clear=False,
)
schedule = parse_beat_schedule() schedule = parse_beat_schedule()
assert schedule == expected assert schedule == expected