Compare commits

...
Author SHA1 Message Date
Trenton H 4f777d7438 Feature: propagate resolved secrets to interactive container shells (#14254)
docker exec bash bypasses s6, so *_FILE secrets resolved by init-env-file
into /run/s6/container_environment aren't visible even though s6 services
and management wrappers already see them via with-contenv.

Add /etc/profile.d/contenv.sh, sourced by both login and non-login
interactive bash shells, to export those resolved values into the shell.
2026-09-24 13:07:41 -07:00
Trenton H 2a44d8b5ba Fix: During a move to the trash directory, attempt to copy metadata, but don't let it fail the move (#14250) 2026-09-23 21:51:22 -07:00
Trenton H abf5050ea7 Fix: convert file mtime to the configured time zone directly (#14249)
The `created` date fallback derived a naive datetime from a file's
mtime using the OS-local zone, then labeled it as the configured
TIME_ZONE without converting. When the OS-local zone and TIME_ZONE
disagree, or when the C library can't resolve zoneinfo at all (as in
some sandboxed environments, where it silently falls back to UTC),
the resulting date can land on the wrong calendar day.

Convert the timestamp directly into the target zone with `tz=` on
fromtimestamp() instead of a naive conversion plus make_aware().
2026-09-23 15:00:48 -07:00
GitHub Actions 091ddf7c45 Auto translate strings 2026-09-23 19:13:55 +00:00
shamoon c9f7f2cfbe Fix: ensure documentDeleted subscription is discarded (#14247) 2026-09-23 12:12:26 -07:00
Trenton H b457610ffb Chore: Fix bugs in the test suite (#14244)
* Fix: redirect SHARE_LINK_BUNDLE_DIR to the test temp layout instead of the real media directory

* Fix: include f_to in test_filters subTest labels so each of the 8 cases reports distinctly

* Fix: run the post_consume error-log assertion after the raising call and match the actual paperless_mail logger name

* Fix: rename the blank-password workflow test to match its behavior and add a real wrong-password-fails test

* Fix: use the created social account's actual pk and remove an accidental tuple wrapping the mock provider

* Fix: assert against the created documents' actual pks instead of hardcoded 1 and 2

* Fix: assert test_compression actually produces a valid LZMA-compressed zip

* Fix: clear os.environ when patching PAPERLESS_ADMIN_* vars so a host-set value can't leak into the no-user test

* Fix: restore MIDDLEWARE, AUTHENTICATION_BACKENDS and REST_FRAMEWORK auth classes after each remote-user settings test instead of leaking the mutation into later tests

* Fix: use a guaranteed-nonexistent temp path instead of hardcoded /tmp/foo/bar in test_export_target_not_exists
2026-09-23 19:11:58 +00:00
GitHub Actions 969c2ea0e2 Auto translate strings 2026-09-23 19:00:47 +00:00
shamoon 31b806a285 Fix: ensure bulk operations are checked against version root (#14246) 2026-09-23 18:59:27 +00:00
GitHub Actions 99851b418c Auto translate strings 2026-09-23 18:46:43 +00:00
e34eda07bb Enhancement: support separate embedding API key (#14067)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-09-23 11:44:37 -07:00
GitHub Actions 793459b416 Auto translate strings 2026-09-23 18:22:05 +00:00
shamoon 04297fd02c Enhancement: support passthrough extra params for LLMs (#14202) 2026-09-23 18:20:43 +00:00
dependabot[bot] 1b277dd8e1 Chore(deps): Bump autobahn in the uv group across 1 directory (#14231)
Bumps the uv group with 1 update in the / directory: [autobahn](https://github.com/crossbario/autobahn-python).


Updates `autobahn` from 25.12.2 to 26.7.1
- [Release notes](https://github.com/crossbario/autobahn-python/releases)
- [Changelog](https://github.com/crossbario/autobahn-python/blob/master/docs/changelog.rst)
- [Commits](https://github.com/crossbario/autobahn-python/compare/v25.12.2...v26.7.1)

---
updated-dependencies:
- dependency-name: autobahn
  dependency-version: 26.7.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-23 15:33:01 +00:00
shamoon 3c20abeb4c Fix: indexing after document-added workflows signal (#14242) 2026-09-23 14:50:48 +00:00
GitHub Actions b11f1f8459 Auto translate strings 2026-09-23 03:19:06 +00:00
Trenton H 7424e7ce0b Fix: Record full tag and custom field lists in bulk edit audit log (#14236)
Snapshot only the edited field, before and after the operation, gathering tags
and custom field instances into sorted id lists per document (empty when there
are none).
2026-09-23 03:17:40 +00:00
shamoon a53a3d3769 Chore: update pikepdf for ocrmypdf requirement (#14235) 2026-09-22 19:01:21 -07:00
Trenton H 15b73b890c Chore: Ban cross-app imports of documents.tests (#14224)
Nothing outside the documents app imports documents.tests any more.

Ruff now bans the module everywhere except src/documents/tests, where in-app imports remain fine.
2026-09-22 08:02:18 -07:00
Trenton H 02d355061f Chore: Move helpers that other test modules imported out of test files (#14223)
The mail message and mailbox builders, the fake libmagic and the classifier preprocessor stub lived inside test_mail.py and test_classifier.py, so other test modules imported them by importing a test module.

They now live in helpers modules beside the tests that use them.
2026-09-22 08:02:18 -07:00
Trenton H d8b5b4d447 Chore: Stop the live-service retry helper sleeping after a success (#14222)
util_call_with_backoff made every call wait 20 seconds even when the first attempt succeeded

The helper now sleeps only after a failure that has another attempt left.
2026-09-22 08:02:18 -07:00
Trenton H 03ac4aed7e Chore: Move the cross-app test helpers into the shared layer and add a progress fixture (#14221)
Test modules in paperless, paperless_mail and documents imported filesystem assertions, the migration test base, the retry helper and the streaming-response reader out of documents/tests/utils.py, which kept each app's tests coupled to another app's test package.

They now live in paperless_testing, and the progress manager fake is renamed FakeProgressManager and now subclasses the real ProgressManager, overriding only the transport, so the payload it records is built by the production code. The twenty places that patched documents.tasks.ProgressManager by hand now use a fake_progress_manager fixture.
2026-09-22 08:02:17 -07:00
Trenton H 90d23bad9c Chore: Give the search index directory one owner in the tests (#14217)
Two fixtures created a temporary index directory and pointed INDEX_DIR at it, and paperless_dirs did the same, so a test that requested more than one got whichever assignment ran last. The search conftest no longer defines its own index_dir fixture, the tests that took it read paperless_dirs.index_dir instead, and _search_index is now a thin wrapper that requests paperless_dirs. The fixture that yields a Document is renamed from indexed_document to searchable_document so it no longer differs by one character from the index_document factory next to it.
2026-09-22 08:02:17 -07:00
Trenton H e4367b5648 Chore: Delete the in-memory Tantivy backend and run the search tests on disk (#14216)
TantivyBackend(path=None) built an in-memory index that is not used by production ever used. The backend now requires a path, the open, write-batch and rebuild branches are gone, and the shared backend fixture and the fulltext similar-documents fixture build a real index under the per-test directory layout.
2026-09-22 08:02:16 -07:00
Trenton H cb85441c2f Chore: Decrease backend test suite time (a little) (#14213)
* Chore: Speed up test setup by hashing passwords with MD5 and batching index writes

Don't use Django's default PBKDF2, about 600 ms per use and 110 uses across the suite. Switches to MD5 instead.

Also fixes a test that didn't batch update the search index

* Chore: Stop the invalid webhook params test from waiting on a Celery broker

test_workflow_webhook_action_url_invalid_params_headers left send_webhook.apply_async unpatched, so it tried to actually enqueu and waited for the timeout.
2026-09-21 20:17:40 +00:00
Trenton H cceaa559d4 Chore: Rename sample directory fixtures and drop unused parser ones (#14215)
Nineteen single-file fixtures in the parsers conftest had no consumers anywhere in the test tree.

Two fixtures were both called samples_dir and resolved one directory apart They are now document_samples_dir and parser_samples_dir
2026-09-21 12:59:32 -07:00
GitHub Actions a748d4c64f Auto translate strings 2026-09-21 19:01:49 +00:00
shamoon 452ed005bd Fix: handle legacy bulk edit page range with missing page_count (#14212) 2026-09-21 19:00:14 +00:00
Trenton H 40058ff7d5 Chore: Give the test suite a larger regex timeout (#14211)
Maybe the random ordering sometimes causes heavier tests to run alongside timed regex ones?
2026-09-21 11:38:54 -07:00
86 changed files with 2066 additions and 1700 deletions
+3 -1
View File
@@ -171,7 +171,9 @@ RUN set -eux \
&& cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \
&& echo "Cleaning up image layer" \
&& rm --force --verbose *.deb \
&& rm --recursive --force --verbose /var/lib/apt/lists/*
&& rm --recursive --force --verbose /var/lib/apt/lists/* \
&& echo "Configuring interactive shells to source the s6 container environment" \
&& echo '. /etc/profile.d/contenv.sh' >> /etc/bash.bashrc
WORKDIR /usr/src/paperless/src/
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
# Source s6 container environment for interactive shells.
# Ensures variables resolved from *_FILE secret injection are visible
# when using 'docker exec bash'. Does not affect s6 services (those
# use with-contenv directly). Has no effect in non-container contexts
# because the directory will not exist.
# Note: sh/dash shells opened via 'docker exec sh' are not covered;
# only bash-based sessions benefit from this file.
_pngx_contenv="/run/s6/container_environment"
if [ -d "${_pngx_contenv}" ]; then
for _pngx_f in "${_pngx_contenv}"/*; do
[ -f "${_pngx_f}" ] || continue
_pngx_name=$(basename "${_pngx_f}")
_pngx_val=$(cat "${_pngx_f}")
export "${_pngx_name}=${_pngx_val}"
done
fi
unset _pngx_contenv _pngx_f _pngx_name _pngx_val
+5 -2
View File
@@ -153,8 +153,11 @@ in similar existing documents, and the document chat can retrieve relevant conte
Enable it by setting
[`PAPERLESS_AI_LLM_EMBEDDING_BACKEND`](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_BACKEND)
(`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). The index is only
built when AI is enabled **and** an embedding backend is set.
(`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). By default, the main
LLM API key and endpoint are used, but an optional embedding-specific[API key](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_API_KEY)
and [endpoint](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) can be configured.
The index is only built when AI is enabled **and** an embedding backend is set.
The index is updated automatically on a schedule controlled by
[`PAPERLESS_LLM_INDEX_TASK_CRON`](configuration.md#PAPERLESS_LLM_INDEX_TASK_CRON) (daily by
+20
View File
@@ -2133,6 +2133,13 @@ for language and resource considerations.
Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_API_KEY=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_API_KEY) {#PAPERLESS_AI_LLM_EMBEDDING_API_KEY}
: The API key to use for the embedding backend. If not supplied, embeddings use
`PAPERLESS_AI_LLM_API_KEY`.
Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) {#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT}
: The endpoint / url to use for the embedding backend. If not supplied, embeddings use
@@ -2217,6 +2224,19 @@ used with the OpenAI-compatible backend to target a custom provider or local gat
Defaults to true, which allows internal endpoints.
#### [`PAPERLESS_AI_LLM_EXTRA_PARAMS=<json>`](#PAPERLESS_AI_LLM_EXTRA_PARAMS) {#PAPERLESS_AI_LLM_EXTRA_PARAMS}
: A JSON object of extra parameters sent with every LLM request, for providers that require a parameter Paperless does not
set itself. Values here override Paperless' own, and no validation is performed. Whatever you put here is passed to the
backend as-is, so an invalid parameter will simply be rejected by your provider. For example, current OpenAI reasoning
models refuse tool calls on the chat completions API unless reasoning is off:
```
PAPERLESS_AI_LLM_EXTRA_PARAMS={"reasoning_effort": "none"}
```
Defaults to empty, which adds nothing to requests.
#### [`PAPERLESS_LLM_INDEX_TASK_CRON=<cron expression>`](#PAPERLESS_LLM_INDEX_TASK_CRON) {#PAPERLESS_LLM_INDEX_TASK_CRON}
: Configures the schedule to update the AI embeddings of text content and metadata for all documents. Only performed if
+6
View File
@@ -247,6 +247,10 @@ per-file-ignores."docker/wait-for-redis.py" = [
per-file-ignores."src/documents/models.py" = [
"SIM115",
]
per-file-ignores."src/documents/tests/*.py" = [
"TID251",
]
flake8-tidy-imports.banned-api."documents.tests".msg = "Shared test infrastructure lives in src/paperless_testing/."
isort.force-single-line = true
[tool.codespell]
@@ -329,6 +333,8 @@ PAPERLESS_CACHE_BACKEND = "django.core.cache.backends.locmem.LocMemCache"
PAPERLESS_CHANNELS_BACKEND = "channels.layers.InMemoryChannelLayer"
# I don't think anything hits this, but just in case, basically infinite
PAPERLESS_TOKEN_THROTTLE_RATE = "1000/min"
# The 0.1s production default trips on a stalled CI runner, the date parsing tests then find no dates
PAPERLESS_MATCH_REGEX_TIMEOUT_SECONDS = "5"
[tool.coverage.run]
source = [
+35 -21
View File
@@ -9745,7 +9745,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">348</context>
<context context-type="linenumber">351</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -9760,7 +9760,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">341</context>
<context context-type="linenumber">344</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -10016,56 +10016,56 @@
<source>Reset filters / selection</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">329</context>
<context context-type="linenumber">332</context>
</context-group>
</trans-unit>
<trans-unit id="4135055128446167640" datatype="html">
<source>Open first [selected] document</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">357</context>
<context context-type="linenumber">360</context>
</context-group>
</trans-unit>
<trans-unit id="3629960544875360046" datatype="html">
<source>Previous page</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">373</context>
<context context-type="linenumber">376</context>
</context-group>
</trans-unit>
<trans-unit id="3337301694210287595" datatype="html">
<source>Next page</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">385</context>
<context context-type="linenumber">388</context>
</context-group>
</trans-unit>
<trans-unit id="2155249406916744630" datatype="html">
<source>View &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot; saved successfully.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">419</context>
<context context-type="linenumber">422</context>
</context-group>
</trans-unit>
<trans-unit id="4646273665293421938" datatype="html">
<source>Failed to save view &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot;.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">425</context>
<context context-type="linenumber">428</context>
</context-group>
</trans-unit>
<trans-unit id="6837554170707123455" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">494</context>
<context context-type="linenumber">497</context>
</context-group>
</trans-unit>
<trans-unit id="6028096992841030074" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully, but could not update visibility settings.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">500</context>
<context context-type="linenumber">503</context>
</context-group>
</trans-unit>
<trans-unit id="739880801667335279" datatype="html">
@@ -12018,81 +12018,95 @@
<context context-type="linenumber">351</context>
</context-group>
</trans-unit>
<trans-unit id="861068592166833023" datatype="html">
<source>LLM Embedding API Key</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">358</context>
</context-group>
</trans-unit>
<trans-unit id="2929108042259892948" datatype="html">
<source>Used for embeddings when set, otherwise LLM API key is used.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">360</context>
</context-group>
</trans-unit>
<trans-unit id="3554114880473286122" datatype="html">
<source>LLM Embedding Endpoint</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">358</context>
<context context-type="linenumber">366</context>
</context-group>
</trans-unit>
<trans-unit id="1044242175651289991" datatype="html">
<source>LLM Embedding Chunk Size</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">365</context>
<context context-type="linenumber">373</context>
</context-group>
</trans-unit>
<trans-unit id="7218245223139363113" datatype="html">
<source>LLM Context Size</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">372</context>
<context context-type="linenumber">380</context>
</context-group>
</trans-unit>
<trans-unit id="4234495692726214397" datatype="html">
<source>LLM Backend</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">379</context>
<context context-type="linenumber">387</context>
</context-group>
</trans-unit>
<trans-unit id="7935234833834000002" datatype="html">
<source>LLM Model</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">387</context>
<context context-type="linenumber">395</context>
</context-group>
</trans-unit>
<trans-unit id="1980550530387803165" datatype="html">
<source>LLM API Key</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">394</context>
<context context-type="linenumber">402</context>
</context-group>
</trans-unit>
<trans-unit id="6126617860376156501" datatype="html">
<source>LLM Endpoint</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">401</context>
<context context-type="linenumber">409</context>
</context-group>
</trans-unit>
<trans-unit id="6572826277249350975" datatype="html">
<source>LLM Output Language</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">408</context>
<context context-type="linenumber">416</context>
</context-group>
</trans-unit>
<trans-unit id="3284403507172415792" datatype="html">
<source>Language to use for generated AI suggestions. When unset, AI suggestions use the user&apos;s display language if explicitly set.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">412</context>
<context context-type="linenumber">420</context>
</context-group>
</trans-unit>
<trans-unit id="4493921125434706859" datatype="html">
<source>LLM Request Timeout</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">416</context>
<context context-type="linenumber">424</context>
</context-group>
</trans-unit>
<trans-unit id="483994032066441287" datatype="html">
<source>Timeout in seconds for LLM requests.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/data/paperless-config.ts</context>
<context context-type="linenumber">420</context>
<context context-type="linenumber">428</context>
</context-group>
</trans-unit>
<trans-unit id="1055686627716339120" datatype="html">
@@ -146,6 +146,19 @@ describe('DocumentListComponent', () => {
expect(reloadSpy).toHaveBeenCalled()
})
it('should stop reloading on document deleted after destroy', () => {
const reloadSpy = jest.spyOn(documentListService, 'reload')
const documentDeletedSubject = new Subject<boolean>()
jest
.spyOn(websocketStatusService, 'onDocumentDeleted')
.mockReturnValue(documentDeletedSubject)
fixture.detectChanges()
fixture.destroy()
reloadSpy.mockClear()
documentDeletedSubject.next(true)
expect(reloadSpy).not.toHaveBeenCalled()
})
it('should show score sort fields on fulltext queries', () => {
documentListService.setFilterRules([
{
@@ -270,9 +270,12 @@ export class DocumentListComponent
this.list.reload()
})
this.websocketStatusService.onDocumentDeleted().subscribe(() => {
this.list.reload()
})
this.websocketStatusService
.onDocumentDeleted()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
this.list.reload()
})
this.route.paramMap
.pipe(
+9
View File
@@ -353,6 +353,14 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_MODEL',
category: ConfigCategory.AI,
},
{
key: 'llm_embedding_api_key',
title: $localize`LLM Embedding API Key`,
type: ConfigOptionType.Password,
note: $localize`Used for embeddings when set, otherwise LLM API key is used.`,
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_API_KEY',
category: ConfigCategory.AI,
},
{
key: 'llm_embedding_endpoint',
title: $localize`LLM Embedding Endpoint`,
@@ -457,6 +465,7 @@ export interface PaperlessConfig extends ObjectWithId {
ai_enabled: boolean
llm_embedding_backend: string
llm_embedding_model: string
llm_embedding_api_key: string
llm_embedding_endpoint: string
llm_embedding_chunk_size: number
llm_context_size: number
+25
View File
@@ -20,6 +20,7 @@ if TYPE_CHECKING:
from rest_framework.test import APIClient
from paperless_testing.dirs import PaperlessDirs
from paperless_testing.fakes.progress import FakeProgressManager
@pytest.fixture(scope="session", autouse=True)
@@ -31,6 +32,18 @@ def faker_session_locale() -> str:
return "en_US"
@pytest.fixture(autouse=True)
def _fast_password_hasher(settings: Settings) -> None:
"""Hash test passwords with MD5 instead of Django's default PBKDF2.
PBKDF2 is deliberately slow, and every ``admin_user`` or
``create_superuser`` call pays for it: about 600 ms each. No test depends
on the hash format, only on ``check_password`` and on the stored value
changing when the password does.
"""
settings.PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
@pytest.fixture(autouse=True)
def _clear_content_type_caches() -> None:
"""Clear Django's ContentType cache and guardian's lru_cache before each test.
@@ -124,3 +137,15 @@ def user_client(rest_api_client: APIClient, regular_user: User) -> APIClient:
rest_api_client.force_authenticate(user=regular_user)
rest_api_client.credentials(HTTP_ACCEPT="application/json; version=10")
return rest_api_client
@pytest.fixture
def fake_progress_manager(
monkeypatch: pytest.MonkeyPatch,
) -> type[FakeProgressManager]:
"""Replace documents.tasks.ProgressManager with the fake, so consuming a file
in a test never tries to reach a broker."""
from paperless_testing.fakes.progress import FakeProgressManager
monkeypatch.setattr("documents.tasks.ProgressManager", FakeProgressManager)
return FakeProgressManager
-1
View File
@@ -26,7 +26,6 @@ class DocumentsConfig(AppConfig):
document_consumption_finished.connect(set_document_type)
document_consumption_finished.connect(set_tags)
document_consumption_finished.connect(set_storage_path)
document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(run_workflows_added)
document_consumption_finished.connect(add_to_index)
document_consumption_finished.connect(add_or_update_document_in_llm_index)
+3 -2
View File
@@ -857,8 +857,9 @@ class ConsumerPlugin(
self.log.debug(f"Creation date from parse_date: {create_date}")
else:
stats = Path(self.input_doc.original_file).stat()
create_date = timezone.make_aware(
datetime.datetime.fromtimestamp(stats.st_mtime),
create_date = datetime.datetime.fromtimestamp(
stats.st_mtime,
tz=timezone.get_current_timezone(),
)
self.log.debug(f"Creation date from st_mtime: {create_date}")
+50 -63
View File
@@ -196,52 +196,49 @@ class WriteBatch:
return self._raw_writer
def __enter__(self) -> Self:
if self._backend._path is not None:
lock_path = self._backend._path / ".tantivy.lock"
self._lock = filelock.FileLock(str(lock_path))
for attempt in range(_LOCK_RETRY_ATTEMPTS):
try:
self._lock.acquire(timeout=self._lock_timeout)
break
except filelock.Timeout:
if attempt == _LOCK_RETRY_ATTEMPTS - 1:
raise SearchIndexLockError(
f"Could not acquire index lock after {_LOCK_RETRY_ATTEMPTS} "
f"attempts (timeout={self._lock_timeout}s each)",
)
sleep_s = random.uniform(
0,
min(_LOCK_BACKOFF_CAP, _LOCK_BACKOFF_BASE * (2**attempt)),
lock_path = self._backend._path / ".tantivy.lock"
self._lock = filelock.FileLock(str(lock_path))
for attempt in range(_LOCK_RETRY_ATTEMPTS):
try:
self._lock.acquire(timeout=self._lock_timeout)
break
except filelock.Timeout:
if attempt == _LOCK_RETRY_ATTEMPTS - 1:
raise SearchIndexLockError(
f"Could not acquire index lock after {_LOCK_RETRY_ATTEMPTS} "
f"attempts (timeout={self._lock_timeout}s each)",
)
logger.debug(
"Index lock contention; retrying in %.2fs (attempt %d/%d)",
sleep_s,
attempt + 1,
_LOCK_RETRY_ATTEMPTS,
)
time.sleep(sleep_s)
sleep_s = random.uniform(
0,
min(_LOCK_BACKOFF_CAP, _LOCK_BACKOFF_BASE * (2**attempt)),
)
logger.debug(
"Index lock contention; retrying in %.2fs (attempt %d/%d)",
sleep_s,
attempt + 1,
_LOCK_RETRY_ATTEMPTS,
)
time.sleep(sleep_s)
# Open a fresh Index (and thus a fresh Tantivy ManagedDirectory)
# for the write, rather than reusing the process-local cached
# index. ManagedDirectory loads its GC bookkeeping (.managed.json)
# once, at construction, and never re-reads it; paperless runs
# several long-lived processes (Granian workers, Celery workers)
# that take turns writing under the file lock above. A cached,
# long-lived writer index would carry a stale managed-files view
# and, on commit, overwrite .managed.json with that stale view -
# permanently losing track of segment files other processes
# registered in the meantime, so they can never be garbage
# collected. Reopening fresh here always picks up the current
# on-disk state. The long-lived self._backend._index is used for
# reads only and is reloaded (not reopened) after commit below.
write_index = tantivy.Index(
build_schema(),
path=str(self._backend._path),
)
register_tokenizers(write_index, settings.SEARCH_LANGUAGE)
self._raw_writer = write_index.writer()
else:
self._raw_writer = self._backend._index.writer()
# Open a fresh Index (and thus a fresh Tantivy ManagedDirectory)
# for the write, rather than reusing the process-local cached
# index. ManagedDirectory loads its GC bookkeeping (.managed.json)
# once, at construction, and never re-reads it; paperless runs
# several long-lived processes (Granian workers, Celery workers)
# that take turns writing under the file lock above. A cached,
# long-lived writer index would carry a stale managed-files view
# and, on commit, overwrite .managed.json with that stale view -
# permanently losing track of segment files other processes
# registered in the meantime, so they can never be garbage
# collected. Reopening fresh here always picks up the current
# on-disk state. The long-lived self._backend._index is used for
# reads only and is reloaded (not reopened) after commit below.
write_index = tantivy.Index(
build_schema(),
path=str(self._backend._path),
)
register_tokenizers(write_index, settings.SEARCH_LANGUAGE)
self._raw_writer = write_index.writer()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
@@ -372,9 +369,8 @@ class TantivyBackend:
Tantivy search backend with explicit lifecycle management.
Provides full-text search capabilities using the Tantivy search engine.
Supports in-memory indexes (for testing) and persistent on-disk indexes
(for production use). Handles document indexing, search queries, autocompletion,
and "more like this" functionality.
Keeps a persistent on-disk index. Handles document indexing, search queries,
autocompletion, and "more like this" functionality.
The backend manages its own connection lifecycle and can be reset when
the underlying index directory changes (e.g., during test isolation).
@@ -408,9 +404,7 @@ class TantivyBackend:
},
)
def __init__(self, path: Path | None = None):
# path=None → in-memory index (for tests)
# path=some_dir → on-disk index (for production)
def __init__(self, path: Path):
self._path = path
self._raw_index: tantivy.Index | None = None
self._raw_schema: tantivy.Schema | None = None
@@ -429,16 +423,13 @@ class TantivyBackend:
"""
Open or rebuild the index as needed.
For disk-based indexes, checks if rebuilding is needed due to schema
version or language changes. Registers custom tokenizers after opening.
Checks if rebuilding is needed due to schema version or language
changes. Registers custom tokenizers after opening.
Safe to call multiple times - subsequent calls are no-ops.
"""
if self._raw_index is not None:
return # pragma: no cover
if self._path is not None:
self._raw_index = open_or_rebuild_index(self._path)
else:
self._raw_index = tantivy.Index(build_schema())
self._raw_index = open_or_rebuild_index(self._path)
register_tokenizers(self._raw_index, settings.SEARCH_LANGUAGE)
self._raw_schema = self._raw_index.schema
@@ -1102,13 +1093,9 @@ class TantivyBackend:
writer's threads). Larger values buffer more docs in RAM before
flushing a segment, deferring merge work; they do not avoid it.
"""
# Create new index (on-disk or in-memory)
if self._path is not None:
wipe_index(self._path)
new_index = tantivy.Index(build_schema(), path=str(self._path))
_write_sentinels(self._path)
else:
new_index = tantivy.Index(build_schema())
wipe_index(self._path)
new_index = tantivy.Index(build_schema(), path=str(self._path))
_write_sentinels(self._path)
register_tokenizers(new_index, settings.SEARCH_LANGUAGE)
# Point instance at the new index so _build_tantivy_doc uses it
+3 -1
View File
@@ -2098,6 +2098,8 @@ class BulkEditSerializer(
if not isinstance(parameters["pages"], str):
raise serializers.ValidationError("invalid pages specified")
page_count = Document.objects.get(id=document_id).page_count
if not page_count:
raise serializers.ValidationError("document page count is unknown")
pages = []
for group in parameters["pages"].split(","):
start, is_range, end = group.partition("-")
@@ -2107,7 +2109,7 @@ class BulkEditSerializer(
except ValueError as e:
raise serializers.ValidationError("invalid pages specified") from e
# Bound the range before building it, a huge one would exhaust memory
if not 1 <= first <= last or (page_count and last > page_count):
if not 1 <= first <= last <= page_count:
raise serializers.ValidationError("invalid pages specified")
pages.append(list(range(first, last + 1)))
parameters["pages"] = pages
+6 -1
View File
@@ -56,6 +56,7 @@ from documents.permissions import get_objects_for_user_owner_aware
from documents.plugins.helpers import DocumentsStatusManager
from documents.templating.utils import convert_format_str_to_template_format
from documents.utils import compute_checksum
from documents.utils import copy_file_with_basic_stats
from documents.workflows.actions import build_workflow_action_context
from documents.workflows.actions import execute_email_action
from documents.workflows.actions import execute_move_to_trash_action
@@ -363,7 +364,11 @@ def cleanup_document_deletion(sender, instance, **kwargs) -> None:
logger.debug(f"Moving {instance.source_path} to trash at {new_file_path}")
try:
shutil.move(instance.source_path, new_file_path)
shutil.move(
instance.source_path,
new_file_path,
copy_function=copy_file_with_basic_stats,
)
except OSError as e:
logger.error(
f"Failed to move {instance.source_path} to trash at "
+11 -24
View File
@@ -1,11 +1,9 @@
import shutil
from collections.abc import Generator
from pathlib import Path
from typing import TYPE_CHECKING
import filelock
import pytest
from pytest_django.fixtures import Settings
from paperless_testing.factories import DocumentFactory
@@ -15,7 +13,7 @@ if TYPE_CHECKING:
@pytest.fixture(scope="session")
def samples_dir() -> Path:
def document_samples_dir() -> Path:
"""Path to the shared test sample documents."""
return Path(__file__).parent / "samples" / "documents"
@@ -23,20 +21,20 @@ def samples_dir() -> Path:
@pytest.fixture()
def sample_doc(
paperless_dirs: "PaperlessDirs",
samples_dir: Path,
document_samples_dir: Path,
) -> "Document":
"""Create a document with valid files and matching checksums."""
with filelock.FileLock(paperless_dirs.media_lock):
shutil.copy(
samples_dir / "originals" / "0000001.pdf",
document_samples_dir / "originals" / "0000001.pdf",
paperless_dirs.originals_dir / "0000001.pdf",
)
shutil.copy(
samples_dir / "archive" / "0000001.pdf",
document_samples_dir / "archive" / "0000001.pdf",
paperless_dirs.archive_dir / "0000001.pdf",
)
shutil.copy(
samples_dir / "thumbnails" / "0000001.webp",
document_samples_dir / "thumbnails" / "0000001.webp",
paperless_dirs.thumbnail_dir / "0000001.webp",
)
@@ -52,28 +50,17 @@ def sample_doc(
)
@pytest.fixture()
def _search_index(
tmp_path: Path,
settings: Settings,
) -> Generator[None, None, None]:
"""Create a temp index directory and point INDEX_DIR at it.
@pytest.fixture
def _search_index(paperless_dirs: "PaperlessDirs") -> None:
"""Point the search backend at a fresh, empty index directory.
Resets the backend singleton before and after so each test gets a clean
index rather than reusing a stale singleton from another test.
paperless_dirs owns INDEX_DIR and resets the backend singleton on both
sides of the test, so requesting it is all that is needed.
"""
from documents.search import reset_backend
index_dir = tmp_path / "index"
index_dir.mkdir()
settings.INDEX_DIR = index_dir
reset_backend()
yield
reset_backend()
@pytest.fixture
def indexed_document(_search_index: None) -> "Document":
def searchable_document(_search_index: None) -> "Document":
"""One searchable document, for tests about what the search endpoint
returns rather than about what it finds.
"""
+10
View File
@@ -0,0 +1,10 @@
import re
def dummy_preprocess(content: str) -> str:
"""
Simpler, faster pre-processing for testing purposes
"""
content = content.lower().strip()
content = re.sub(r"\s+", " ", content)
return content
+3 -11
View File
@@ -14,24 +14,16 @@ from paperless_testing.factories import DocumentFactory
if TYPE_CHECKING:
from collections.abc import Callable
from collections.abc import Generator
from pathlib import Path
from pytest_django.fixtures import Settings
from documents.models import Document
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture
def index_dir(tmp_path: Path, settings: Settings) -> Path:
path = tmp_path / "index"
path.mkdir()
settings.INDEX_DIR = path
return path
@pytest.fixture
def backend() -> Generator[TantivyBackend, None, None]:
b = TantivyBackend() # path=None → in-memory index
def backend(paperless_dirs: PaperlessDirs) -> Generator[TantivyBackend, None, None]:
b = TantivyBackend(path=paperless_dirs.index_dir)
b.open()
try:
yield b
+4 -2
View File
@@ -947,7 +947,8 @@ class TestSingleton:
yield
reset_backend()
def test_returns_same_instance_on_repeated_calls(self, index_dir) -> None:
@pytest.mark.usefixtures("paperless_dirs")
def test_returns_same_instance_on_repeated_calls(self) -> None:
"""Singleton pattern: repeated calls to get_backend() must return the same instance."""
assert get_backend() is get_backend()
@@ -964,7 +965,8 @@ class TestSingleton:
assert b1 is not b2
assert b2._path == tmp_path / "b"
def test_reset_forces_new_instance(self, index_dir) -> None:
@pytest.mark.usefixtures("paperless_dirs")
def test_reset_forces_new_instance(self) -> None:
"""reset_backend() must force creation of a new backend instance on next get_backend() call."""
b1 = get_backend()
reset_backend()
@@ -269,7 +269,7 @@ class TestDocumentedDateForms:
yield
@pytest.fixture
def dated(self, index_document: Callable[..., Document]) -> dict[str, int]:
def dated(self, backend: TantivyBackend) -> dict[str, int]:
stamps = {
"today": datetime(2026, 6, 15, 9, 0, tzinfo=UTC),
"yesterday": datetime(2026, 6, 14, 9, 0, tzinfo=UTC),
@@ -279,14 +279,14 @@ class TestDocumentedDateForms:
"january": datetime(2026, 1, 10, 10, 0, tzinfo=UTC),
"old": datetime(2005, 3, 4, 15, 30, tzinfo=UTC),
}
return {
label: index_document(
title=label,
content="dated body",
added=stamp,
).pk
docs = {
label: DocumentFactory(title=label, content="dated body", added=stamp)
for label, stamp in stamps.items()
}
with backend.batch_update() as batch:
for doc in docs.values():
batch.add_or_update(doc)
return {label: doc.pk for label, doc in docs.items()}
@pytest.mark.parametrize(
("query", "label"),
@@ -1,6 +1,6 @@
import pytest
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
pytestmark = pytest.mark.search
+22 -19
View File
@@ -13,11 +13,11 @@ from documents.search._schema import needs_rebuild
from documents.search._schema import schema_fingerprint
if TYPE_CHECKING:
from pathlib import Path
import tantivy
from pytest_django.fixtures import Settings
from paperless_testing.dirs import PaperlessDirs
pytestmark = pytest.mark.search
@@ -25,16 +25,19 @@ pytestmark = pytest.mark.search
class TestNeedsRebuild:
"""needs_rebuild covers all sentinel-file states that require a full reindex."""
def test_returns_true_when_settings_file_missing(self, index_dir: Path) -> None:
assert needs_rebuild(index_dir) is True
def test_returns_true_when_settings_file_missing(
self,
paperless_dirs: PaperlessDirs,
) -> None:
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_false_when_version_and_language_match(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = "en"
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps(
{
"schema_version": SCHEMA_VERSION,
@@ -43,51 +46,51 @@ class TestNeedsRebuild:
},
),
)
assert needs_rebuild(index_dir) is False
assert needs_rebuild(paperless_dirs.index_dir) is False
def test_returns_true_on_schema_version_mismatch(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = None
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION - 1, "language": None}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_true_when_version_is_not_an_integer(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = None
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": "not-a-number", "language": None}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_true_when_language_key_missing(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = "en"
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_returns_true_when_language_differs(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: Settings,
) -> None:
settings.SEARCH_LANGUAGE = "de"
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION, "language": "en"}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def _schema_fields(schema: tantivy.Schema) -> dict[str, dict]:
@@ -35,6 +35,8 @@ if TYPE_CHECKING:
from pytest_django.fixtures import SettingsWrapper
from paperless_testing.dirs import PaperlessDirs
pytestmark = pytest.mark.search
# The on-disk field layout of a v2 index, pinned as data. Any edit here is an
@@ -469,7 +471,7 @@ def _fingerprint_of(descriptors: list[FieldDescriptor]) -> str:
class TestNeedsRebuildOnFingerprint:
def test_matching_fingerprint_does_not_rebuild(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
) -> None:
"""
@@ -482,13 +484,13 @@ class TestNeedsRebuildOnFingerprint:
- It returns False
"""
settings.SEARCH_LANGUAGE = None
_sentinels(index_dir)
_sentinels(paperless_dirs.index_dir)
assert needs_rebuild(index_dir) is False
assert needs_rebuild(paperless_dirs.index_dir) is False
def test_stale_fingerprint_rebuilds_despite_a_matching_version(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -505,7 +507,7 @@ class TestNeedsRebuildOnFingerprint:
every subsequent write would raise
"""
settings.SEARCH_LANGUAGE = None
_sentinels(index_dir)
_sentinels(paperless_dirs.index_dir)
extended = [
*field_descriptors(),
FieldDescriptor(
@@ -519,11 +521,11 @@ class TestNeedsRebuildOnFingerprint:
]
monkeypatch.setattr(_schema, "field_descriptors", lambda: extended)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_reordered_schema_rebuilds(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -538,16 +540,16 @@ class TestNeedsRebuildOnFingerprint:
- It returns True
"""
settings.SEARCH_LANGUAGE = None
_sentinels(index_dir)
_sentinels(paperless_dirs.index_dir)
reordered = field_descriptors()
reordered[1], reordered[2] = reordered[2], reordered[1]
monkeypatch.setattr(_schema, "field_descriptors", lambda: reordered)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_missing_fingerprint_rebuilds(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
) -> None:
"""
@@ -561,15 +563,15 @@ class TestNeedsRebuildOnFingerprint:
is rebuilt rather than trusted
"""
settings.SEARCH_LANGUAGE = None
(index_dir / ".index_settings.json").write_text(
(paperless_dirs.index_dir / ".index_settings.json").write_text(
json.dumps({"schema_version": SCHEMA_VERSION, "language": None}),
)
assert needs_rebuild(index_dir) is True
assert needs_rebuild(paperless_dirs.index_dir) is True
def test_written_sentinels_satisfy_the_check(
self,
index_dir: Path,
paperless_dirs: PaperlessDirs,
settings: SettingsWrapper,
) -> None:
"""
@@ -582,6 +584,6 @@ class TestNeedsRebuildOnFingerprint:
- It returns False
"""
settings.SEARCH_LANGUAGE = "en"
_write_sentinels(index_dir)
_write_sentinels(paperless_dirs.index_dir)
assert needs_rebuild(index_dir) is False
assert needs_rebuild(paperless_dirs.index_dir) is False
+45 -1
View File
@@ -10,11 +10,11 @@ from PIL.PngImagePlugin import PngInfo
from rest_framework import status
from rest_framework.test import APITestCase
from documents.tests.utils import read_streaming_response
from paperless.models import ApplicationConfiguration
from paperless.models import ColorConvertChoices
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
class TestApiAppConfig(DirectoriesMixin, APITestCase):
@@ -81,6 +81,7 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
"ai_enabled": None,
"llm_embedding_backend": None,
"llm_embedding_model": None,
"llm_embedding_api_key": None,
"llm_embedding_endpoint": None,
"llm_embedding_chunk_size": None,
"llm_context_size": None,
@@ -922,6 +923,49 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_405_METHOD_NOT_ALLOWED)
self.assertEqual(ApplicationConfiguration.objects.count(), 1)
def test_update_llm_embedding_api_key(self) -> None:
"""
GIVEN:
- Existing config with llm_embedding_api_key specified
WHEN:
- API to update llm_embedding_api_key is called with all *s
- API to update llm_embedding_api_key is called with empty string
THEN:
- llm_embedding_api_key is unchanged
- llm_embedding_api_key is set to None
"""
config = ApplicationConfiguration.objects.first()
assert config is not None
config.llm_embedding_api_key = "1234567890"
config.save()
# Test with all *
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"llm_embedding_api_key": "*" * 32,
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.llm_embedding_api_key, "1234567890")
# Test with empty string
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps(
{
"llm_embedding_api_key": "",
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
config.refresh_from_db()
self.assertEqual(config.llm_embedding_api_key, None)
def test_update_llm_api_key(self) -> None:
"""
GIVEN:
+10 -2
View File
@@ -13,9 +13,9 @@ from documents.models import Correspondent
from documents.models import Document
from documents.models import DocumentType
from documents.tests.utils import SampleDirMixin
from documents.tests.utils import read_streaming_response
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
@@ -166,7 +166,15 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
),
content_type="application/json",
)
response.close()
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response["Content-Type"], "application/zip")
with zipfile.ZipFile(io.BytesIO(read_streaming_response(response))) as zipf:
self.assertEqual(zipf.infolist()[0].compress_type, zipfile.ZIP_LZMA)
with self.doc2.source_file as f:
self.assertEqual(f.read(), zipf.read("2021-01-01 document A.pdf"))
@override_settings(FILENAME_FORMAT="{correspondent}/{title}")
def test_formatted_download_originals(self) -> None:
+64 -9
View File
@@ -9,6 +9,7 @@ from rest_framework.test import APITestCase
from documents.models import Correspondent
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import DocumentType
from documents.models import StoragePath
@@ -1784,6 +1785,36 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertIn(b"invalid pages specified", response.content)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.split")
def test_bulk_edit_split_rejects_unknown_page_count(self, m) -> None:
"""
GIVEN:
- A legacy split bulk edit of a document without a page count
WHEN:
- API to bulk edit is called
THEN:
- API returns HTTP 400
- split is not called
"""
self.setup_mock(m, "split")
for pages in ("1", "1-5000000"):
with self.subTest(pages=pages):
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc1.id],
"method": "split",
"parameters": {"pages": pages},
},
),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn(b"document page count is unknown", response.content)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.split")
def test_bulk_edit_split_parses_pages(self, m) -> None:
"""
@@ -2495,7 +2526,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
WHEN:
- API to bulk edit documents is called
THEN:
- Audit log is created
- Audit log is created with the old and new correspondent
"""
LogEntry.objects.all().delete()
response = self.client.post(
@@ -2511,7 +2542,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 1)
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(entry.changes, {"correspondent": [None, self.c2.id]})
@override_settings(AUDIT_LOG_ENABLED=True)
def test_bulk_edit_audit_log_enabled_tags(self) -> None:
@@ -2519,16 +2551,18 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
GIVEN:
- Audit log is enabled
WHEN:
- API to bulk edit tags is called
- API to bulk edit tags is called on an untagged document and a
document with several tags
THEN:
- Audit log is created
- Audit log is created for each document with its full tag list
before and after the edit
"""
LogEntry.objects.all().delete()
response = self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": [self.doc1.id],
"documents": [self.doc1.id, self.doc4.id],
"method": "modify_tags",
"parameters": {
"add_tags": [self.t1.id],
@@ -2540,18 +2574,32 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 1)
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(entry.changes, {"tags": [[], [self.t1.id]]})
entry = LogEntry.objects.get_for_object(self.doc4).get()
self.assertEqual(
entry.changes,
{"tags": [[self.t1.id, self.t2.id], [self.t1.id]]},
)
@override_settings(AUDIT_LOG_ENABLED=True)
def test_bulk_edit_audit_log_enabled_custom_fields(self) -> None:
"""
GIVEN:
- Audit log is enabled
- A document with two custom fields
WHEN:
- API to bulk edit custom fields is called
- API to bulk edit custom fields is called to add a third
THEN:
- Audit log is created
- Audit log is created with every custom field instance before and
after the edit
- Audit log is created for the new custom field instance
"""
cf3 = CustomField.objects.create(name="cf3", data_type="string")
existing = [
CustomFieldInstance.objects.create(document=self.doc1, field=field)
for field in (self.cf2, cf3)
]
LogEntry.objects.all().delete()
response = self.client.post(
"/api/documents/bulk_edit/",
@@ -2569,7 +2617,14 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 2)
added = CustomFieldInstance.objects.get(document=self.doc1, field=self.cf1)
existing_ids = [instance.id for instance in existing]
entry = LogEntry.objects.get_for_object(self.doc1).get()
self.assertEqual(
entry.changes,
{"custom_fields": [existing_ids, [*existing_ids, added.id]]},
)
self.assertEqual(LogEntry.objects.get_for_object(added).count(), 1)
def test_api_bulk_edit_with_bad_search_query_returns_400(self) -> None:
"""
@@ -16,11 +16,11 @@ from documents.data_models import DocumentSource
from documents.filters import EffectiveContentFilter
from documents.filters import TitleContentFilter
from documents.models import Document
from documents.tests.utils import read_streaming_response
from documents.versioning import annotate_effective_content
from documents.views import DocumentSelectionMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
if TYPE_CHECKING:
+1 -1
View File
@@ -48,11 +48,11 @@ from documents.models import WorkflowAction
from documents.models import WorkflowTrigger
from documents.signals.handlers import run_workflows
from documents.tests.utils import ConsumeTaskMixin
from documents.tests.utils import read_streaming_response
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
+11 -14
View File
@@ -64,16 +64,15 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
)
self.client.force_authenticate(user=self.user)
def setupSocialAccount(self) -> None:
def setupSocialAccount(self) -> SocialAccount:
SocialApp.objects.create(
name="Keycloak",
provider="openid_connect",
provider_id="keycloak-test",
)
self.user.socialaccount_set.add(
SocialAccount(uid="123456789", provider="keycloak-test"),
bulk=False,
)
social_account = SocialAccount(uid="123456789", provider="keycloak-test")
self.user.socialaccount_set.add(social_account, bulk=False)
return social_account
def test_get_profile(self) -> None:
"""
@@ -111,19 +110,17 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
THEN:
- Profile is returned with social accounts
"""
self.setupSocialAccount()
social_account = self.setupSocialAccount()
openid_provider = (
MockOpenIDConnectProvider(
app=SocialApp.objects.get(provider_id="keycloak-test"),
),
openid_provider = MockOpenIDConnectProvider(
app=SocialApp.objects.get(provider_id="keycloak-test"),
)
mock_list_providers.return_value = [
openid_provider,
]
mock_get_provider_account.return_value = MockOpenIDConnectProviderAccount(
mock_social_account_dict={
"name": openid_provider[0].name,
"name": openid_provider.name,
},
)
@@ -135,7 +132,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
response.data["social_accounts"],
[
{
"id": 1,
"id": social_account.pk,
"provider": "keycloak-test",
"name": "Keycloak",
},
@@ -152,7 +149,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
THEN:
- Profile is returned with "Unknown App" as name
"""
self.setupSocialAccount()
social_account = self.setupSocialAccount()
# Remove the social app
SocialApp.objects.get(provider_id="keycloak-test").delete()
@@ -165,7 +162,7 @@ class TestApiProfile(DirectoriesMixin, APITestCase):
response.data["social_accounts"],
[
{
"id": 1,
"id": social_account.pk,
"provider": "keycloak-test",
"name": "Unknown App",
},
+10 -10
View File
@@ -33,7 +33,7 @@ class TestSearchQueryErrorStillBecomesA400:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -68,7 +68,7 @@ class TestLibraryDefectsPropagate:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -98,7 +98,7 @@ class TestLibraryDefectsPropagate:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -141,7 +141,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -181,7 +181,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -221,7 +221,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -259,7 +259,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -287,7 +287,7 @@ class TestSelectionPathsAgreeWithSearch:
{
"documents": [],
"all": True,
"filters": {"more_like_id": indexed_document.pk},
"filters": {"more_like_id": searchable_document.pk},
},
format="json",
)
@@ -298,7 +298,7 @@ class TestSelectionPathsAgreeWithSearch:
self,
admin_client: APIClient,
monkeypatch: pytest.MonkeyPatch,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -328,7 +328,7 @@ class TestSelectionPathsAgreeWithSearch:
{
"documents": [],
"all": True,
"filters": {"more_like_id": indexed_document.pk},
"filters": {"more_like_id": searchable_document.pk},
},
format="json",
)
@@ -36,7 +36,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -66,7 +66,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_query_at_exactly_the_cap_is_accepted(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -86,7 +86,7 @@ class TestGetSearchEndpointEnforcesTheCap:
def test_an_ordinary_query_is_unaffected(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -112,7 +112,7 @@ class TestPostSelectionPathsEnforceTheCap:
def test_bulk_edit_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -154,7 +154,7 @@ class TestPostSelectionPathsEnforceTheCap:
self,
bulk_update_task_mock: mock.MagicMock,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -187,7 +187,7 @@ class TestPostSelectionPathsEnforceTheCap:
def test_bulk_download_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -236,7 +236,7 @@ class TestGlobalSearchEnforcesTheCapToo:
def test_query_one_over_the_cap_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -260,7 +260,7 @@ class TestGlobalSearchEnforcesTheCapToo:
def test_query_at_exactly_the_cap_is_accepted(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
@@ -38,7 +38,7 @@ class TestUnterminatedBracketReturnsA400:
def test_unterminated_bracket_is_a_400(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
query: str,
) -> None:
"""
@@ -59,7 +59,7 @@ class TestUnterminatedBracketReturnsA400:
def test_properly_closed_bracket_still_searches_cleanly(
self,
admin_client: APIClient,
indexed_document: Document,
searchable_document: Document,
) -> None:
"""
GIVEN:
+74 -74
View File
@@ -2,8 +2,8 @@ import shutil
from collections.abc import Generator
from contextlib import contextmanager
from pathlib import Path
from unittest import mock
import pytest
from django.conf import settings
from django.test import TestCase
from django.test import override_settings
@@ -18,11 +18,11 @@ from documents.models import Document
from documents.models import Tag
from documents.plugins.base import StopConsumeTaskError
from documents.tests.utils import ConsumeTaskMixin
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless.models import ApplicationConfiguration
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.fakes.progress import FakeProgressManager
class GetReaderPluginMixin:
@@ -31,7 +31,7 @@ class GetReaderPluginMixin:
reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(),
DummyProgressManager(filepath.name, None),
FakeProgressManager(filepath.name, None),
self.dirs.scratch_dir,
"task-id",
)
@@ -86,6 +86,7 @@ class TestBarcode(
self.assertDictEqual(separator_page_numbers, {1: False})
@override_settings(CONSUMER_ENABLE_ASN_BARCODE=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_asn_barcode_duplicate_in_trash_fails(self) -> None:
"""
GIVEN:
@@ -110,15 +111,14 @@ class TestBarcode(
dupe_asn = settings.SCRATCH_DIR / "barcode-39-asn-123-second.pdf"
shutil.copy(test_file, dupe_asn)
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
with self.assertRaisesRegex(ConsumerError, r"ASN 123.*trash"):
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dupe_asn,
),
None,
)
with self.assertRaisesRegex(ConsumerError, r"ASN 123.*trash"):
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dupe_asn,
),
None,
)
@override_settings(
CONSUMER_BARCODE_TIFF_SUPPORT=True,
@@ -606,6 +606,7 @@ class TestBarcodeNewConsume(
TestCase,
):
@override_settings(CONSUMER_ENABLE_BARCODES=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file(self) -> None:
"""
GIVEN:
@@ -624,34 +625,33 @@ class TestBarcodeNewConsume(
overrides = DocumentMetadataOverrides(tag_ids=[1, 2, 9])
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
self.assertEqual(
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=temp_copy,
),
overrides,
self.assertEqual(
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=temp_copy,
),
{"reason": "Barcode splitting complete!"},
)
# 2 new document consume tasks created
self.assertEqual(self.consume_file_mock.call_count, 2)
overrides,
),
{"reason": "Barcode splitting complete!"},
)
# 2 new document consume tasks created
self.assertEqual(self.consume_file_mock.call_count, 2)
self.assertIsNotFile(temp_copy)
self.assertIsNotFile(temp_copy)
# Check the split files exist
# Check the original_path is set
# Check the source is unchanged
# Check the overrides are unchanged
for (
new_input_doc,
new_doc_overrides,
) in self.get_all_consume_task_call_args():
self.assertIsFile(new_input_doc.original_file)
self.assertEqual(new_input_doc.original_path, temp_copy)
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertEqual(overrides, new_doc_overrides)
# Check the split files exist
# Check the original_path is set
# Check the source is unchanged
# Check the overrides are unchanged
for (
new_input_doc,
new_doc_overrides,
) in self.get_all_consume_task_call_args():
self.assertIsFile(new_input_doc.original_file)
self.assertEqual(new_input_doc.original_path, temp_copy)
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertEqual(overrides, new_doc_overrides)
class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, TestCase):
@@ -660,7 +660,7 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(),
DummyProgressManager(filepath.name, None),
FakeProgressManager(filepath.name, None),
self.dirs.scratch_dir,
"task-id",
)
@@ -745,6 +745,7 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
self.assertEqual(asn, None)
@override_settings(CONSUMER_ENABLE_ASN_BARCODE=True)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file_asn_assignment(self) -> None:
"""
GIVEN:
@@ -762,19 +763,18 @@ class TestAsnBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
dst = settings.SCRATCH_DIR / "barcode-39-asn-123.pdf"
shutil.copy(test_file, dst)
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
document = Document.objects.first()
assert document is not None
document = Document.objects.first()
assert document is not None
self.assertEqual(document.archive_serial_number, 123)
self.assertEqual(document.archive_serial_number, 123)
def test_scan_file_for_qrcode_without_upscale(self) -> None:
"""
@@ -819,7 +819,7 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
reader = BarcodePlugin(
ConsumableDocument(DocumentSource.ConsumeFolder, original_file=filepath),
DocumentMetadataOverrides(),
DummyProgressManager(filepath.name, None),
FakeProgressManager(filepath.name, None),
self.dirs.scratch_dir,
"task-id",
)
@@ -1024,6 +1024,7 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
CELERY_TASK_ALWAYS_EAGER=True,
OCR_MODE="auto",
)
@pytest.mark.usefixtures("fake_progress_manager")
def test_consume_barcode_file_tag_split_and_assignment(self) -> None:
"""
GIVEN:
@@ -1042,34 +1043,33 @@ class TestTagBarcode(DirectoriesMixin, SampleDirMixin, GetReaderPluginMixin, Tes
dst = settings.SCRATCH_DIR / "split-by-tag-basic.pdf"
shutil.copy(test_file, dst)
with mock.patch("documents.tasks.ProgressManager", DummyProgressManager):
result = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
result = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
self.assertEqual(result, {"reason": "Barcode splitting complete!"})
self.assertEqual(result, {"reason": "Barcode splitting complete!"})
documents = Document.objects.all().order_by("id")
self.assertEqual(documents.count(), 3)
documents = Document.objects.all().order_by("id")
self.assertEqual(documents.count(), 3)
doc1 = documents[0]
self.assertEqual(doc1.tags.count(), 0)
doc1 = documents[0]
self.assertEqual(doc1.tags.count(), 0)
doc2 = documents[1]
self.assertEqual(doc2.tags.count(), 1)
_tag_1 = doc2.tags.first()
assert _tag_1 is not None
self.assertEqual(_tag_1.name, "invoice")
doc2 = documents[1]
self.assertEqual(doc2.tags.count(), 1)
_tag_1 = doc2.tags.first()
assert _tag_1 is not None
self.assertEqual(_tag_1.name, "invoice")
doc3 = documents[2]
self.assertEqual(doc3.tags.count(), 1)
_tag_2 = doc3.tags.first()
assert _tag_2 is not None
self.assertEqual(_tag_2.name, "receipt")
doc3 = documents[2]
self.assertEqual(doc3.tags.count(), 1)
_tag_2 = doc3.tags.first()
assert _tag_2 is not None
self.assertEqual(_tag_2.name, "receipt")
@override_settings(
CONSUMER_ENABLE_TAG_BARCODE=True,
+1 -10
View File
@@ -1,5 +1,4 @@
import pickle
import re
import warnings
from datetime import UTC
from datetime import datetime
@@ -28,6 +27,7 @@ from documents.models import DocumentType
from documents.models import MatchingModel
from documents.models import StoragePath
from documents.models import Tag
from documents.tests.helpers import dummy_preprocess
from paperless.settings import CLASSIFIER_LANGUAGES
from paperless.signed_pickle import HMAC_SIZE
from paperless.signed_pickle import signed_pickle_dumps
@@ -36,15 +36,6 @@ from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
def dummy_preprocess(content: str) -> str:
"""
Simpler, faster pre-processing for testing purposes
"""
content = content.lower().strip()
content = re.sub(r"\s+", " ", content)
return content
class TestClassifier(DirectoriesMixin, TestCase):
def setUp(self) -> None:
super().setUp()
+5 -5
View File
@@ -30,12 +30,12 @@ from documents.models import Tag
from documents.parsers import ParseError
from documents.plugins.helpers import ProgressStatusOptions
from documents.tasks import sanity_check
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import GetConsumerMixin
from paperless_mail.models import MailRule
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.fakes.progress import FakeProgressManager
class _BaseNewStyleParser:
@@ -777,7 +777,7 @@ class TestConsumer(
)
version_file = self.get_test_file2()
status = DummyProgressManager(version_file.name, None)
status = FakeProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides(
version_label="v2",
actor_id=actor.pk,
@@ -840,7 +840,7 @@ class TestConsumer(
assert root_doc is not None
version_file = self.get_test_file2()
status = DummyProgressManager(version_file.name, None)
status = FakeProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides(
filename="valid_pdf_version-upload",
actor_id=999999,
@@ -897,7 +897,7 @@ class TestConsumer(
assert root_doc is not None
def consume_version(version_file: Path) -> Document:
status = DummyProgressManager(version_file.name, None)
status = FakeProgressManager(version_file.name, None)
overrides = DocumentMetadataOverrides()
doc = ConsumableDocument(
DocumentSource.ApiUpload,
+17 -17
View File
@@ -2,8 +2,8 @@ import datetime as dt
import os
import shutil
from pathlib import Path
from unittest import mock
import pytest
from django.test import TestCase
from django.test import override_settings
from pdfminer.high_level import extract_text
@@ -15,18 +15,22 @@ from documents.data_models import ConsumableDocument
from documents.data_models import DocumentSource
from documents.double_sided import STAGING_FILE_NAME
from documents.double_sided import TIMEOUT_MINUTES
from documents.tests.utils import DummyProgressManager
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@pytest.mark.usefixtures("fake_progress_manager")
@override_settings(
CONSUMER_RECURSIVE=True,
CONSUMER_ENABLE_COLLATE_DOUBLE_SIDED=True,
)
class TestDoubleSided(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
SAMPLE_DIR = Path(__file__).parent / "samples"
class TestDoubleSided(
DirectoriesMixin,
FileSystemAssertsMixin,
SampleDirMixin,
TestCase,
):
def setUp(self) -> None:
super().setUp()
self.double_sided_dir = self.dirs.consumption_dir / "double-sided"
@@ -42,17 +46,13 @@ class TestDoubleSided(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
dst = self.double_sided_dir / dstname
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy(src, dst)
with mock.patch(
"documents.tasks.ProgressManager",
DummyProgressManager,
):
msg = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
msg = tasks.consume_file(
ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=dst,
),
None,
)
self.assertIsNotFile(dst)
return msg
+1 -1
View File
@@ -29,7 +29,7 @@ from documents.models import DocumentType
from documents.models import StoragePath
from documents.serialisers import DocumentSerializer
from documents.tasks import empty_trash
from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
+1 -1
View File
@@ -20,7 +20,7 @@ if TYPE_CHECKING:
from documents.file_handling import generate_filename
from documents.models import Document
from documents.tasks import update_document_content_maybe_archive_file
from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
sample_file: Path = Path(__file__).parent / "samples" / "simple.pdf"
@@ -45,9 +45,9 @@ from documents.models import WorkflowTrigger
from documents.sanity_checker import check_sanity
from documents.settings import EXPORTER_FILE_NAME
from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless_mail.models import MailAccount
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.dirs import paperless_environment
from paperless_testing.permissions import grant_object
@@ -677,12 +677,13 @@ class TestExportImport(
THEN:
- Error is raised
"""
args = ["document_exporter", "/tmp/foo/bar"]
with tempfile.TemporaryDirectory() as tmp_dir:
args = ["document_exporter", str(Path(tmp_dir) / "does-not-exist")]
with self.assertRaises(CommandError) as e:
call_command(*args, skip_checks=True)
with self.assertRaises(CommandError) as e:
call_command(*args, skip_checks=True)
self.assertEqual("That path doesn't exist", str(e.exception))
self.assertEqual("That path doesn't exist", str(e.exception))
def test_export_target_exists_but_is_file(self) -> None:
"""
+8 -8
View File
@@ -123,14 +123,14 @@ class TestFuzzyMatchCommand(TestCase):
- Output contains clickable links to the documents instead of titles
"""
# Content similarity is 86.667
Document.objects.create(
doc1 = Document.objects.create(
checksum="BEEFCAFE",
title="A",
content="first document scanned by bob",
mime_type="application/pdf",
filename="test.pdf",
)
Document.objects.create(
doc2 = Document.objects.create(
checksum="DEADBEAF",
title="A",
content="first document scanned by alice",
@@ -145,8 +145,8 @@ class TestFuzzyMatchCommand(TestCase):
"http://localhost:8000",
)
self.assertIn("Found 1 matching pair(s)", stdout)
self.assertIn("http://localhost:8000/documents/1/details", stdout)
self.assertIn("http://localhost:8000/documents/2/details", stdout)
self.assertIn(f"http://localhost:8000/documents/{doc1.pk}/details", stdout)
self.assertIn(f"http://localhost:8000/documents/{doc2.pk}/details", stdout)
def test_with_3_matches(self) -> None:
"""
@@ -198,14 +198,14 @@ class TestFuzzyMatchCommand(TestCase):
- Documents 1 and 2 remain
"""
# Content similarity is 86.667
Document.objects.create(
doc1 = Document.objects.create(
checksum="BEEFCAFE",
title="A",
content="first document scanned by bob",
mime_type="application/pdf",
filename="test.pdf",
)
Document.objects.create(
doc2 = Document.objects.create(
checksum="DEADBEAF",
title="A",
content="second document scanned by alice",
@@ -235,8 +235,8 @@ class TestFuzzyMatchCommand(TestCase):
self.assertIn("Deleting 1 document(s)", stdout)
self.assertEqual(Document.objects.count(), 2)
self.assertIsNotNone(Document.objects.get(pk=1))
self.assertIsNotNone(Document.objects.get(pk=2))
self.assertIsNotNone(Document.objects.get(pk=doc1.pk))
self.assertIsNotNone(Document.objects.get(pk=doc2.pk))
def test_document_deletion_cancelled(self) -> None:
"""
@@ -15,8 +15,8 @@ from documents.management.commands.document_importer import _deserialize_record
from documents.models import Document
from documents.settings import EXPORTER_ARCHIVE_NAME
from documents.settings import EXPORTER_FILE_NAME
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.utils import SampleDirMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@@ -14,7 +14,7 @@ from paperless_testing.dirs import DirectoriesMixin
class TestManageSuperUser(DirectoriesMixin, TestCase):
def call_command(self, environ):
out = StringIO()
with mock.patch.dict(os.environ, environ):
with mock.patch.dict(os.environ, environ, clear=True):
call_command(
"manage_superuser",
"--no-color",
@@ -9,7 +9,7 @@ from django.test import TestCase
from documents.management.commands.document_thumbnails import _process_document
from documents.models import Document
from documents.parsers import get_default_thumbnail
from documents.tests.utils import FileSystemAssertsMixin
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@@ -1,4 +1,4 @@
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
SAVED_VIEWS_KEY = "saved_views"
DASHBOARD_VIEWS_VISIBLE_IDS_KEY = "dashboard_views_visible_ids"
@@ -7,7 +7,7 @@ from django.conf import settings
from django.db import connection
from django.test import override_settings
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
def _sha256(data: bytes) -> str:
@@ -1,4 +1,4 @@
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
class TestMigrateShareLinkBundlePermissions(TestMigrations):
@@ -430,6 +430,53 @@ class TestBulkDownloadPermissionChecksRootDocument:
) # version-only grant must not substitute for root permission
@pytest.mark.django_db
class TestDocumentOperationPermissionChecksRootDocument:
@pytest.mark.parametrize(
("endpoint", "payload"),
[
pytest.param("/api/documents/merge/", {}, id="merge"),
pytest.param("/api/documents/rotate/", {"degrees": 90}, id="rotate"),
],
)
@pytest.mark.parametrize("version_owner", ["none", "requester"])
def test_version_operation_acts_on_root(
self,
rest_api_client: APIClient,
endpoint: str,
payload: dict,
version_owner: str,
) -> None:
owner = UserFactory(username="owner")
requester = UserFactory(username="requester")
grant_global(requester, "change_document")
grant_global(requester, "add_document")
rest_api_client.force_authenticate(user=requester)
root = DocumentFactory(owner=owner)
# A version whose owner went stale, e.g. created before the root changed hands
version = DocumentFactory(
owner=requester if version_owner == "requester" else None,
root_document=root,
version_index=1,
)
with (
patch("documents.views.bulk_edit.merge") as mock_merge,
patch("documents.views.bulk_edit.rotate") as mock_rotate,
):
mock_merge.__name__ = "merge"
mock_rotate.__name__ = "rotate"
response = rest_api_client.post(
endpoint,
{"documents": [version.pk], **payload},
format="json",
)
assert response.status_code == HTTPStatus.FORBIDDEN
mock_merge.assert_not_called()
mock_rotate.assert_not_called()
@pytest.mark.django_db
@pytest.mark.usefixtures("_search_index")
class TestTrashRestorePermissionBoundary:
@@ -339,15 +339,6 @@ class ShareLinkBundleBuildTaskTests(DirectoriesMixin, APITestCase):
)
self.document.archive_checksum = ""
self.document.save()
self.addCleanup(
setattr,
settings,
"SHARE_LINK_BUNDLE_DIR",
settings.SHARE_LINK_BUNDLE_DIR,
)
settings.SHARE_LINK_BUNDLE_DIR = (
Path(settings.MEDIA_ROOT) / "documents" / "share_link_bundles"
)
def _write_document_file(self, *, archive: bool, content: bytes) -> Path:
if archive:
+2 -2
View File
@@ -17,8 +17,8 @@ from documents.models import Tag
from documents.models import WorkflowAction
from documents.sanity_checker import SanityCheckFailedException
from documents.sanity_checker import SanityCheckMessages
from documents.tests.test_classifier import dummy_preprocess
from documents.tests.utils import FileSystemAssertsMixin
from documents.tests.helpers import dummy_preprocess
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
+1 -1
View File
@@ -28,12 +28,12 @@ from documents.models import StoragePath
from documents.models import Tag
from documents.models import UiSettings
from documents.signals.handlers import update_llm_suggestions_cache
from documents.tests.utils import read_streaming_response
from paperless.models import ApplicationConfiguration
from paperless_ai.exceptions import LLMProviderError
from paperless_ai.exceptions import LLMTimeoutError
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
from paperless_testing.permissions import grant_object
File diff suppressed because it is too large Load Diff
+2 -225
View File
@@ -1,126 +1,16 @@
import time
import warnings
from collections.abc import Callable
from collections.abc import Generator
from collections.abc import Iterator
from contextlib import contextmanager
from os import PathLike
from pathlib import Path
from typing import Any
from unittest import mock
import httpx
import pytest
from django.apps import apps
from django.db import connection
from django.db.migrations.executor import MigrationExecutor
from django.http import StreamingHttpResponse
from django.test import TransactionTestCase
from documents.consumer import AsnCheckPlugin
from documents.consumer import ConsumerPlugin
from documents.consumer import ConsumerPreflightPlugin
from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
from documents.data_models import DocumentSource
from documents.parsers import ParseError
from documents.plugins.helpers import ProgressStatusOptions
def util_call_with_backoff(
method_or_callable: Callable,
args: list | tuple,
*,
skip_on_50x_err=True,
) -> tuple[bool, Any]:
"""
For whatever reason, the images started during the test pipeline like to
segfault sometimes, crash and otherwise fail randomly, when run with the
exact files that usually pass.
So, this function will retry the given method/function up to 3 times, with larger backoff
periods between each attempt, in hopes the issue resolves itself during
one attempt to parse.
This will wait the following:
- Attempt 1 - 20s following failure
- Attempt 2 - 40s following failure
- Attempt 3 - 80s following failure
"""
result = None
succeeded = False
retry_time = 20.0
retry_count = 0
status_codes = []
max_retry_count = 3
while retry_count < max_retry_count and not succeeded:
try:
result = method_or_callable(*args)
succeeded = True
except ParseError as e: # pragma: no cover
cause_exec = e.__cause__
if cause_exec is not None and isinstance(cause_exec, httpx.HTTPStatusError):
status_codes.append(cause_exec.response.status_code)
warnings.warn(
f"HTTP Exception for {cause_exec.request.url} - {cause_exec}",
)
else:
warnings.warn(f"Unexpected error: {e}")
except Exception as e: # pragma: no cover
warnings.warn(f"Unexpected error: {e}")
retry_count = retry_count + 1
time.sleep(retry_time)
retry_time = retry_time * 2.0
if (
not succeeded
and status_codes
and skip_on_50x_err
and all(httpx.codes.is_server_error(code) for code in status_codes)
):
pytest.skip("Repeated HTTP 50x for service") # pragma: no cover
return succeeded, result
def read_streaming_response(response: StreamingHttpResponse) -> bytes:
"""Consume a StreamingHttpResponse/FileResponse and close it."""
content = b"".join(response.streaming_content)
response.close()
return content
class FileSystemAssertsMixin:
"""
Utilities for checks various state information of the file system
"""
def assertIsFile(self, path: PathLike[str] | str) -> None:
self.assertTrue(Path(path).resolve().is_file(), f"File does not exist: {path}")
def assertIsNotFile(self, path: PathLike[str] | str) -> None:
self.assertFalse(Path(path).resolve().is_file(), f"File does exist: {path}")
def assertIsDir(self, path: PathLike[str] | str) -> None:
self.assertTrue(Path(path).resolve().is_dir(), f"Dir does not exist: {path}")
def assertIsNotDir(self, path: PathLike[str] | str) -> None:
self.assertFalse(Path(path).resolve().is_dir(), f"Dir does exist: {path}")
def assertFileCountInDir(self, path: PathLike[str] | str, count: int) -> None:
path = Path(path).resolve()
self.assertTrue(path.is_dir(), f"Path {path} is not a directory")
files = [x for x in path.iterdir() if x.is_file()]
self.assertEqual(
len(files),
count,
f"Path {path} contains {len(files)} files instead of {count} files",
)
from paperless_testing.fakes.progress import FakeProgressManager
class ConsumeTaskMixin:
@@ -158,59 +48,6 @@ class ConsumeTaskMixin:
yield (task_kwargs["input_doc"], task_kwargs["overrides"])
class TestMigrations(TransactionTestCase):
@property
def app(self):
return apps.get_containing_app_config(type(self).__module__).name
migrate_from = None
dependencies = None
migrate_to = None
def setUp(self) -> None:
super().setUp()
assert self.migrate_from and self.migrate_to, (
f"TestCase '{type(self).__name__}' must define migrate_from and migrate_to properties"
)
self.migrate_from = [(self.app, self.migrate_from)]
if self.dependencies is not None:
self.migrate_from.extend(self.dependencies)
self.migrate_to = [(self.app, self.migrate_to)]
executor = MigrationExecutor(connection)
old_apps = executor.loader.project_state(self.migrate_from).apps
# Reverse to the original migration
executor.migrate(self.migrate_from)
self.setUpBeforeMigration(old_apps)
self.apps = old_apps
# Run the migration to test
executor = MigrationExecutor(connection)
executor.loader.build_graph() # reload.
executor.migrate(self.migrate_to)
self.apps = executor.loader.project_state(self.migrate_to).apps
def setUpBeforeMigration(self, apps) -> None:
pass
def tearDown(self) -> None:
"""
Ensure the database schema is restored to the latest migration after
each migration test, so subsequent tests run against HEAD.
"""
try:
executor = MigrationExecutor(connection)
executor.loader.build_graph()
targets = executor.loader.graph.leaf_nodes()
executor.migrate(targets)
finally:
super().tearDown()
class SampleDirMixin:
SAMPLE_DIR = Path(__file__).parent / "samples"
@@ -227,7 +64,7 @@ class GetConsumerMixin:
mailrule_id: int | None = None,
) -> Generator[ConsumerPlugin, None, None]:
# Store this for verification
self.status = DummyProgressManager(filepath.name, None)
self.status = FakeProgressManager(filepath.name, None)
doc = ConsumableDocument(
source,
original_file=filepath,
@@ -263,63 +100,3 @@ class GetConsumerMixin:
yield reader
finally:
reader.cleanup()
class DummyProgressManager:
"""
A dummy handler for progress management that doesn't actually try to
connect to Redis. Payloads are stored for test assertions if needed.
Use it with
mock.patch("documents.tasks.ProgressManager", DummyProgressManager)
"""
def __init__(self, filename: str, task_id: str | None = None) -> None:
self.filename = filename
self.task_id = task_id
self.payloads = []
def __enter__(self):
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
def open(self) -> None:
pass
def close(self) -> None:
pass
def send_progress(
self,
status: ProgressStatusOptions,
message: str,
current_progress: int,
max_progress: int,
*,
document_id: int | None = None,
owner_id: int | None = None,
users_can_view: list[int] | None = None,
groups_can_view: list[int] | None = None,
) -> None:
# Ensure the layer is open
self.open()
payload = {
"type": "status_update",
"data": {
"filename": self.filename,
"task_id": self.task_id,
"current_progress": current_progress,
"max_progress": max_progress,
"status": status,
"message": message,
"document_id": document_id,
"owner_id": owner_id,
"users_can_view": users_can_view or [],
"groups_can_view": groups_can_view or [],
},
}
self.payloads.append(payload)
+51 -34
View File
@@ -49,7 +49,6 @@ from django.db.models import Sum
from django.db.models import When
from django.db.models.functions import Coalesce
from django.db.models.functions import Lower
from django.db.models.manager import Manager
from django.http import FileResponse
from django.http import Http404
from django.http import HttpRequest
@@ -2968,11 +2967,15 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
if user.is_superuser:
return True
document_objs = Document.objects.select_related("owner").filter(
pk__in=documents,
)
root_docs = {
get_root_document(doc)
for doc in Document.objects.select_related(
"owner",
"root_document__owner",
).filter(pk__in=documents)
}
user_is_owner_of_all_documents = all(
(doc.owner == user or doc.owner is None) for doc in document_objs
(doc.owner == user or doc.owner is None) for doc in root_docs
)
# check global and object permissions for all documents
@@ -2980,9 +2983,13 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
user.has_perm(
"documents.change_document",
)
and not document_objs.exclude(
and not Document.global_objects.filter(
pk__in=[doc.pk for doc in root_docs],
)
.exclude(
pk__in=permitted_document_ids(user, perm="change_document"),
).exists()
)
.exists()
)
# check ownership for methods that change original document
@@ -3141,6 +3148,38 @@ class BulkEditView(DocumentOperationPermissionMixin):
serializer_class = BulkEditSerializer
@staticmethod
def _snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
"""
Returns each document's current value of field, for the audit log.
Tags and custom fields are one row per value, so they are gathered
into a sorted list of pks per document (empty when there are none).
Reading them through Document.values() instead would join those rows
and return one arbitrary value per document.
"""
if field == "tags":
rows = (
Document.tags.through.objects.filter(document_id__in=doc_ids)
.order_by("tag_id")
.values_list("document_id", "tag_id")
)
elif field == "custom_fields":
rows = (
CustomFieldInstance.objects.filter(document_id__in=doc_ids)
.order_by("pk")
.values_list("document_id", "pk")
)
else:
return dict(
Document.objects.filter(pk__in=doc_ids).values_list("pk", field),
)
values: dict[int, list[int]] = {doc_id: [] for doc_id in doc_ids}
for doc_id, pk in rows:
values[doc_id].append(pk)
return values
def post(self, request, *args, **kwargs):
request_method = request.data.get("method")
api_version = int(request.version or settings.REST_FRAMEWORK["DEFAULT_VERSION"])
@@ -3187,41 +3226,19 @@ class BulkEditView(DocumentOperationPermissionMixin):
try:
modified_field = self.MODIFIED_FIELD_BY_METHOD.get(method.__name__, None)
if settings.AUDIT_LOG_ENABLED and modified_field:
old_documents = {
obj["pk"]: obj
for obj in Document.objects.filter(pk__in=documents).values(
"pk",
"correspondent",
"document_type",
"storage_path",
"tags",
"custom_fields",
"deleted_at",
"checksum",
)
}
old_values = self._snapshot_field(documents, modified_field)
result = method(documents, **parameters)
if settings.AUDIT_LOG_ENABLED and modified_field:
new_documents = Document.objects.filter(pk__in=documents)
for doc in new_documents:
old_value = old_documents[doc.pk][modified_field]
new_value = getattr(doc, modified_field)
if isinstance(new_value, Model):
# correspondent, document type, etc.
new_value = new_value.pk
elif isinstance(new_value, Manager):
# tags, custom fields
new_value = list(new_value.values_list("pk", flat=True))
new_values = self._snapshot_field(documents, modified_field)
for doc in Document.objects.filter(pk__in=documents):
LogEntry.objects.log_create(
instance=doc,
changes={
modified_field: [
old_value,
new_value,
old_values[doc.pk],
new_values[doc.pk],
],
},
action=LogEntry.Action.UPDATE,
+69 -65
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr ""
"Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-09-18 01:29+0000\n"
"POT-Creation-Date: 2026-09-23 19:00+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n"
"Language-Team: English\n"
@@ -1632,7 +1632,7 @@ msgid "workflow runs"
msgstr ""
#: documents/serialisers.py:514 documents/serialisers.py:871
#: documents/serialisers.py:2883 documents/views.py:343 documents/views.py:2726
#: documents/serialisers.py:2885 documents/views.py:342 documents/views.py:2725
#: paperless_mail/serialisers.py:156
msgid "Insufficient permissions."
msgstr ""
@@ -1641,39 +1641,39 @@ msgstr ""
msgid "Invalid color."
msgstr ""
#: documents/serialisers.py:2350
#: documents/serialisers.py:2352
#, python-format
msgid "File type %(type)s not supported"
msgstr ""
#: documents/serialisers.py:2394
#: documents/serialisers.py:2396
#, python-format
msgid "Custom field id must be an integer: %(id)s"
msgstr ""
#: documents/serialisers.py:2401
#: documents/serialisers.py:2403
#, python-format
msgid "Custom field with id %(id)s does not exist"
msgstr ""
#: documents/serialisers.py:2418 documents/serialisers.py:2428
#: documents/serialisers.py:2420 documents/serialisers.py:2430
msgid ""
"Custom fields must be a list of integers or an object mapping ids to values."
msgstr ""
#: documents/serialisers.py:2423
#: documents/serialisers.py:2425
msgid "Some custom fields don't exist or were specified twice."
msgstr ""
#: documents/serialisers.py:2570
#: documents/serialisers.py:2572
msgid "Invalid variable detected."
msgstr ""
#: documents/serialisers.py:2939
#: documents/serialisers.py:2941
msgid "Duplicate document identifiers are not allowed."
msgstr ""
#: documents/serialisers.py:2969 documents/views.py:4763
#: documents/serialisers.py:2971 documents/views.py:4780
#, python-format
msgid "Documents not found: %(ids)s"
msgstr ""
@@ -1941,40 +1941,40 @@ msgstr ""
msgid "Unable to parse URI {value}"
msgstr ""
#: documents/views.py:336 documents/views.py:2723
#: documents/views.py:335 documents/views.py:2722
msgid "Invalid more_like_id"
msgstr ""
#: documents/views.py:1670
#: documents/views.py:1669
msgid "Invalid AI configuration."
msgstr ""
#: documents/views.py:1681
#: documents/views.py:1680
msgid "AI backend request timed out."
msgstr ""
#: documents/views.py:1693
#: documents/views.py:1692
msgid "AI backend rejected the request. Check logs for details."
msgstr ""
#: documents/views.py:2548 documents/views.py:2864
#: documents/views.py:2547 documents/views.py:2863
msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr ""
#: documents/views.py:4776
#: documents/views.py:4793
#, python-format
msgid "Insufficient permissions to share document %(id)s."
msgstr ""
#: documents/views.py:4822
#: documents/views.py:4839
msgid "Bundle is already being processed."
msgstr ""
#: documents/views.py:4886
#: documents/views.py:4903
msgid "The share link bundle is still being prepared. Please try again later."
msgstr ""
#: documents/views.py:4900
#: documents/views.py:4917
msgid "The share link bundle is unavailable."
msgstr ""
@@ -2219,190 +2219,194 @@ msgid "Sets the LLM embedding model"
msgstr ""
#: paperless/models.py:369
msgid "Sets the LLM embedding endpoint, optional"
msgid "Sets the LLM embedding API key"
msgstr ""
#: paperless/models.py:376
msgid "Sets the LLM embedding endpoint, optional"
msgstr ""
#: paperless/models.py:383
msgid "Sets the LLM embedding chunk size"
msgstr ""
#: paperless/models.py:382
#: paperless/models.py:389
msgid "Sets the LLM context size"
msgstr ""
#: paperless/models.py:388
#: paperless/models.py:395
msgid "Sets the LLM backend"
msgstr ""
#: paperless/models.py:396
#: paperless/models.py:403
msgid "Sets the LLM model"
msgstr ""
#: paperless/models.py:403
#: paperless/models.py:410
msgid "Sets the LLM API key"
msgstr ""
#: paperless/models.py:410
#: paperless/models.py:417
msgid "Sets the LLM endpoint, optional"
msgstr ""
#: paperless/models.py:417
#: paperless/models.py:424
msgid "Sets the LLM output language"
msgstr ""
#: paperless/models.py:424
#: paperless/models.py:431
msgid "Sets the LLM timeout in seconds"
msgstr ""
#: paperless/models.py:430
#: paperless/models.py:437
msgid "paperless application settings"
msgstr ""
#: paperless/settings/__init__.py:558
#: paperless/settings/__init__.py:559
msgid "English (US)"
msgstr ""
#: paperless/settings/__init__.py:559
#: paperless/settings/__init__.py:560
msgid "Arabic"
msgstr ""
#: paperless/settings/__init__.py:560
#: paperless/settings/__init__.py:561
msgid "Afrikaans"
msgstr ""
#: paperless/settings/__init__.py:561
#: paperless/settings/__init__.py:562
msgid "Belarusian"
msgstr ""
#: paperless/settings/__init__.py:562
#: paperless/settings/__init__.py:563
msgid "Bulgarian"
msgstr ""
#: paperless/settings/__init__.py:563
#: paperless/settings/__init__.py:564
msgid "Catalan"
msgstr ""
#: paperless/settings/__init__.py:564
#: paperless/settings/__init__.py:565
msgid "Czech"
msgstr ""
#: paperless/settings/__init__.py:565
#: paperless/settings/__init__.py:566
msgid "Danish"
msgstr ""
#: paperless/settings/__init__.py:566
#: paperless/settings/__init__.py:567
msgid "German"
msgstr ""
#: paperless/settings/__init__.py:567
#: paperless/settings/__init__.py:568
msgid "Greek"
msgstr ""
#: paperless/settings/__init__.py:568
#: paperless/settings/__init__.py:569
msgid "English (GB)"
msgstr ""
#: paperless/settings/__init__.py:569
#: paperless/settings/__init__.py:570
msgid "Spanish"
msgstr ""
#: paperless/settings/__init__.py:570
#: paperless/settings/__init__.py:571
msgid "Persian"
msgstr ""
#: paperless/settings/__init__.py:571
#: paperless/settings/__init__.py:572
msgid "Finnish"
msgstr ""
#: paperless/settings/__init__.py:572
#: paperless/settings/__init__.py:573
msgid "French"
msgstr ""
#: paperless/settings/__init__.py:573
#: paperless/settings/__init__.py:574
msgid "Hungarian"
msgstr ""
#: paperless/settings/__init__.py:574
#: paperless/settings/__init__.py:575
msgid "Indonesian"
msgstr ""
#: paperless/settings/__init__.py:575
#: paperless/settings/__init__.py:576
msgid "Italian"
msgstr ""
#: paperless/settings/__init__.py:576
#: paperless/settings/__init__.py:577
msgid "Japanese"
msgstr ""
#: paperless/settings/__init__.py:577
#: paperless/settings/__init__.py:578
msgid "Korean"
msgstr ""
#: paperless/settings/__init__.py:578
#: paperless/settings/__init__.py:579
msgid "Luxembourgish"
msgstr ""
#: paperless/settings/__init__.py:579
#: paperless/settings/__init__.py:580
msgid "Norwegian"
msgstr ""
#: paperless/settings/__init__.py:580
#: paperless/settings/__init__.py:581
msgid "Dutch"
msgstr ""
#: paperless/settings/__init__.py:581
#: paperless/settings/__init__.py:582
msgid "Polish"
msgstr ""
#: paperless/settings/__init__.py:582
#: paperless/settings/__init__.py:583
msgid "Portuguese (Brazil)"
msgstr ""
#: paperless/settings/__init__.py:583
#: paperless/settings/__init__.py:584
msgid "Portuguese"
msgstr ""
#: paperless/settings/__init__.py:584
#: paperless/settings/__init__.py:585
msgid "Romanian"
msgstr ""
#: paperless/settings/__init__.py:585
#: paperless/settings/__init__.py:586
msgid "Russian"
msgstr ""
#: paperless/settings/__init__.py:586
#: paperless/settings/__init__.py:587
msgid "Slovak"
msgstr ""
#: paperless/settings/__init__.py:587
#: paperless/settings/__init__.py:588
msgid "Slovenian"
msgstr ""
#: paperless/settings/__init__.py:588
#: paperless/settings/__init__.py:589
msgid "Serbian"
msgstr ""
#: paperless/settings/__init__.py:589
#: paperless/settings/__init__.py:590
msgid "Swedish"
msgstr ""
#: paperless/settings/__init__.py:590
#: paperless/settings/__init__.py:591
msgid "Turkish"
msgstr ""
#: paperless/settings/__init__.py:591
#: paperless/settings/__init__.py:592
msgid "Ukrainian"
msgstr ""
#: paperless/settings/__init__.py:592
#: paperless/settings/__init__.py:593
msgid "Vietnamese"
msgstr ""
#: paperless/settings/__init__.py:593
#: paperless/settings/__init__.py:594
msgid "Chinese Simplified"
msgstr ""
#: paperless/settings/__init__.py:594
#: paperless/settings/__init__.py:595
msgid "Chinese Traditional"
msgstr ""
+7
View File
@@ -1,5 +1,6 @@
import dataclasses
import json
from typing import Any
from django.conf import settings
@@ -244,6 +245,7 @@ class AIConfig(BaseConfig):
ai_enabled: bool = dataclasses.field(init=False)
llm_embedding_backend: str = dataclasses.field(init=False)
llm_embedding_model: str = dataclasses.field(init=False)
llm_embedding_api_key: str = dataclasses.field(init=False)
llm_embedding_endpoint: str = dataclasses.field(init=False)
llm_embedding_chunk_size: int = dataclasses.field(init=False)
llm_context_size: int = dataclasses.field(init=False)
@@ -254,6 +256,7 @@ class AIConfig(BaseConfig):
llm_endpoint: str = dataclasses.field(init=False)
llm_output_language: str = dataclasses.field(init=False)
llm_allow_internal_endpoints: bool = dataclasses.field(init=False)
llm_extra_params: dict[str, Any] = dataclasses.field(init=False)
def __post_init__(self) -> None:
app_config = self._get_config_instance()
@@ -269,6 +272,9 @@ class AIConfig(BaseConfig):
self.llm_embedding_model = (
app_config.llm_embedding_model or settings.LLM_EMBEDDING_MODEL
)
self.llm_embedding_api_key = (
app_config.llm_embedding_api_key or settings.LLM_EMBEDDING_API_KEY
)
self.llm_embedding_endpoint = (
app_config.llm_embedding_endpoint or settings.LLM_EMBEDDING_ENDPOINT
)
@@ -287,6 +293,7 @@ class AIConfig(BaseConfig):
app_config.llm_output_language or settings.LLM_OUTPUT_LANGUAGE
)
self.llm_allow_internal_endpoints = settings.LLM_ALLOW_INTERNAL_ENDPOINTS
self.llm_extra_params = settings.LLM_EXTRA_PARAMS
@property
def llm_index_enabled(self) -> bool:
@@ -0,0 +1,23 @@
# Generated by Django 5.2.16 on 2026-09-11 09:32
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("paperless", "0016_alter_applicationconfiguration_ai_enabled"),
]
operations = [
migrations.AddField(
model_name="applicationconfiguration",
name="llm_embedding_api_key",
field=models.CharField(
blank=True,
max_length=1024,
null=True,
verbose_name="Sets the LLM embedding API key",
),
),
]
+7
View File
@@ -365,6 +365,13 @@ class ApplicationConfiguration(AbstractSingletonModel):
max_length=128,
)
llm_embedding_api_key = models.CharField(
verbose_name=_("Sets the LLM embedding API key"),
blank=True,
null=True,
max_length=1024,
)
llm_embedding_endpoint = models.CharField(
verbose_name=_("Sets the LLM embedding endpoint, optional"),
blank=True,
+10 -1
View File
@@ -216,6 +216,11 @@ class ApplicationConfigurationSerializer(
externally_configured_variables = serializers.SerializerMethodField()
user_args = serializers.JSONField(binary=True, allow_null=True)
barcode_tag_mapping = serializers.JSONField(binary=True, allow_null=True)
llm_embedding_api_key = ObfuscatedPasswordField(
required=False,
allow_null=True,
max_length=1024,
)
llm_api_key = ObfuscatedPasswordField(
required=False,
allow_null=True,
@@ -227,7 +232,11 @@ class ApplicationConfigurationSerializer(
max_length=1024,
)
OBFUSCATED_FIELDS = ("llm_api_key", "remote_ocr_api_key")
OBFUSCATED_FIELDS = (
"llm_embedding_api_key",
"llm_api_key",
"remote_ocr_api_key",
)
def get_externally_configured_variables(
self,
+22
View File
@@ -7,6 +7,7 @@ import multiprocessing
import os
import tempfile
from pathlib import Path
from typing import Any
from typing import Final
from urllib.parse import urlparse
@@ -1081,6 +1082,25 @@ CLASSIFIER_LANGUAGES: Final[dict[str, str]] = {
}
def _get_llm_extra_params() -> dict[str, Any]:
"""
Parse PAPERLESS_AI_LLM_EXTRA_PARAMS, a JSON object passed straight through
to the LLM backend's request body.
"""
raw = os.getenv("PAPERLESS_AI_LLM_EXTRA_PARAMS", "{}")
try:
parsed = json.loads(raw)
except json.JSONDecodeError as e:
raise ImproperlyConfigured(
"PAPERLESS_AI_LLM_EXTRA_PARAMS must be valid JSON",
) from e
if not isinstance(parsed, dict):
raise ImproperlyConfigured(
"PAPERLESS_AI_LLM_EXTRA_PARAMS must be a JSON object",
)
return parsed
def _get_classifier_language_setting(ocr_lang: str) -> str | None:
"""
Maps the primary Tesseract language to the classifier's stemming
@@ -1216,6 +1236,7 @@ LLM_EMBEDDING_BACKEND = get_choice_from_env(
{"huggingface", "openai-like", "ollama"},
)
LLM_EMBEDDING_MODEL = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_MODEL")
LLM_EMBEDDING_API_KEY = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_API_KEY")
LLM_EMBEDDING_ENDPOINT = os.getenv("PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT")
LLM_EMBEDDING_CHUNK_SIZE = get_int_from_env(
"PAPERLESS_AI_LLM_EMBEDDING_CHUNK_SIZE",
@@ -1241,3 +1262,4 @@ LLM_ALLOW_INTERNAL_ENDPOINTS = get_bool_from_env(
"PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS",
"true",
)
LLM_EXTRA_PARAMS = _get_llm_extra_params()
+4 -4
View File
@@ -22,11 +22,11 @@ if TYPE_CHECKING:
@pytest.fixture(scope="session")
def samples_dir() -> Path:
def parser_samples_dir() -> Path:
"""Absolute path to the shared parser sample files directory.
Sub-package conftest files derive format-specific paths from this root,
e.g. ``samples_dir / "text" / "test.txt"``.
e.g. ``parser_samples_dir / "text" / "test.txt"``.
Returns
-------
@@ -37,7 +37,7 @@ def samples_dir() -> Path:
@pytest.fixture(scope="session")
def tagged_no_text_pdf_file(samples_dir: Path) -> Path:
def tagged_no_text_pdf_file(parser_samples_dir: Path) -> Path:
"""Path to a tagged PDF whose only "text" is pdftotext layout padding.
Reproduces GH #13387: ``/MarkInfo /Marked true`` is set, but the only
@@ -50,7 +50,7 @@ def tagged_no_text_pdf_file(samples_dir: Path) -> Path:
Path
Absolute path to ``tesseract/tagged-but-no-text.pdf``.
"""
return samples_dir / "tesseract" / "tagged-but-no-text.pdf"
return parser_samples_dir / "tesseract" / "tagged-but-no-text.pdf"
@pytest.fixture(autouse=True)
+12 -240
View File
@@ -37,15 +37,15 @@ if TYPE_CHECKING:
@pytest.fixture(scope="session")
def text_samples_dir(samples_dir: Path) -> Path:
def text_samples_dir(parser_samples_dir: Path) -> Path:
"""Absolute path to the text parser sample files directory.
Returns
-------
Path
``<samples_dir>/text/``
``<parser_samples_dir>/text/``
"""
return samples_dir / "text"
return parser_samples_dir / "text"
@pytest.fixture(scope="session")
@@ -175,15 +175,15 @@ def no_engine_settings(
@pytest.fixture(scope="session")
def tika_samples_dir(samples_dir: Path) -> Path:
def tika_samples_dir(parser_samples_dir: Path) -> Path:
"""Absolute path to the Tika parser sample files directory.
Returns
-------
Path
``<samples_dir>/tika/``
``<parser_samples_dir>/tika/``
"""
return samples_dir / "tika"
return parser_samples_dir / "tika"
@pytest.fixture(scope="session")
@@ -258,15 +258,15 @@ def tika_parser() -> Generator[TikaDocumentParser, None, None]:
@pytest.fixture(scope="session")
def mail_samples_dir(samples_dir: Path) -> Path:
def mail_samples_dir(parser_samples_dir: Path) -> Path:
"""Absolute path to the mail parser sample files directory.
Returns
-------
Path
``<samples_dir>/mail/``
``<parser_samples_dir>/mail/``
"""
return samples_dir / "mail"
return parser_samples_dir / "mail"
@pytest.fixture(scope="session")
@@ -421,75 +421,15 @@ def nginx_base_url() -> Generator[str, None, None]:
@pytest.fixture(scope="session")
def tesseract_samples_dir(samples_dir: Path) -> Path:
def tesseract_samples_dir(parser_samples_dir: Path) -> Path:
"""Absolute path to the tesseract parser sample files directory.
Returns
-------
Path
``<samples_dir>/tesseract/``
``<parser_samples_dir>/tesseract/``
"""
return samples_dir / "tesseract"
@pytest.fixture(scope="session")
def document_webp_file(tesseract_samples_dir: Path) -> Path:
"""Path to a WebP document sample file.
Returns
-------
Path
Absolute path to ``tesseract/document.webp``.
"""
return tesseract_samples_dir / "document.webp"
@pytest.fixture(scope="session")
def encrypted_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to an encrypted PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/encrypted.pdf``.
"""
return tesseract_samples_dir / "encrypted.pdf"
@pytest.fixture(scope="session")
def multi_page_digital_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page digital PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-digital.pdf``.
"""
return tesseract_samples_dir / "multi-page-digital.pdf"
@pytest.fixture(scope="session")
def multi_page_images_alpha_rgb_tiff_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page TIFF with alpha channel in RGB.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-images-alpha-rgb.tiff``.
"""
return tesseract_samples_dir / "multi-page-images-alpha-rgb.tiff"
@pytest.fixture(scope="session")
def multi_page_images_alpha_tiff_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page TIFF with alpha channel.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-images-alpha.tiff``.
"""
return tesseract_samples_dir / "multi-page-images-alpha.tiff"
return parser_samples_dir / "tesseract"
@pytest.fixture(scope="session")
@@ -504,90 +444,6 @@ def multi_page_images_pdf_file(tesseract_samples_dir: Path) -> Path:
return tesseract_samples_dir / "multi-page-images.pdf"
@pytest.fixture(scope="session")
def multi_page_images_tiff_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page TIFF sample file.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-images.tiff``.
"""
return tesseract_samples_dir / "multi-page-images.tiff"
@pytest.fixture(scope="session")
def multi_page_mixed_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a multi-page mixed PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/multi-page-mixed.pdf``.
"""
return tesseract_samples_dir / "multi-page-mixed.pdf"
@pytest.fixture(scope="session")
def no_text_alpha_png_file(tesseract_samples_dir: Path) -> Path:
"""Path to a PNG with alpha channel and no text.
Returns
-------
Path
Absolute path to ``tesseract/no-text-alpha.png``.
"""
return tesseract_samples_dir / "no-text-alpha.png"
@pytest.fixture(scope="session")
def rotated_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a rotated PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/rotated.pdf``.
"""
return tesseract_samples_dir / "rotated.pdf"
@pytest.fixture(scope="session")
def rtl_test_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to an RTL test PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/rtl-test.pdf``.
"""
return tesseract_samples_dir / "rtl-test.pdf"
@pytest.fixture(scope="session")
def signed_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a signed PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/signed.pdf``.
"""
return tesseract_samples_dir / "signed.pdf"
@pytest.fixture(scope="session")
def simple_alpha_png_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple PNG with alpha channel.
Returns
-------
Path
Absolute path to ``tesseract/simple-alpha.png``.
"""
return tesseract_samples_dir / "simple-alpha.png"
@pytest.fixture(scope="session")
def simple_digital_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple digital PDF sample file.
@@ -612,54 +468,6 @@ def simple_no_dpi_png_file(tesseract_samples_dir: Path) -> Path:
return tesseract_samples_dir / "simple-no-dpi.png"
@pytest.fixture(scope="session")
def simple_bmp_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple BMP sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.bmp``.
"""
return tesseract_samples_dir / "simple.bmp"
@pytest.fixture(scope="session")
def simple_gif_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple GIF sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.gif``.
"""
return tesseract_samples_dir / "simple.gif"
@pytest.fixture(scope="session")
def simple_heic_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple HEIC sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.heic``.
"""
return tesseract_samples_dir / "simple.heic"
@pytest.fixture(scope="session")
def simple_jpg_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple JPG sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.jpg``.
"""
return tesseract_samples_dir / "simple.jpg"
@pytest.fixture(scope="session")
def simple_png_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple PNG sample file.
@@ -672,42 +480,6 @@ def simple_png_file(tesseract_samples_dir: Path) -> Path:
return tesseract_samples_dir / "simple.png"
@pytest.fixture(scope="session")
def simple_tif_file(tesseract_samples_dir: Path) -> Path:
"""Path to a simple TIF sample file.
Returns
-------
Path
Absolute path to ``tesseract/simple.tif``.
"""
return tesseract_samples_dir / "simple.tif"
@pytest.fixture(scope="session")
def single_page_mixed_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a single-page mixed PDF sample file.
Returns
-------
Path
Absolute path to ``tesseract/single-page-mixed.pdf``.
"""
return tesseract_samples_dir / "single-page-mixed.pdf"
@pytest.fixture(scope="session")
def with_form_pdf_file(tesseract_samples_dir: Path) -> Path:
"""Path to a PDF with form sample file.
Returns
-------
Path
Absolute path to ``tesseract/with-form.pdf``.
"""
return tesseract_samples_dir / "with-form.pdf"
# ------------------------------------------------------------------
# Tesseract parser instance and settings helpers
# ------------------------------------------------------------------
@@ -10,8 +10,8 @@ from imagehash import average_hash
from PIL import Image
from pytest_mock import MockerFixture
from documents.tests.utils import util_call_with_backoff
from paperless.parsers.mail import MailDocumentParser
from paperless_testing.retry import util_call_with_backoff
def extract_text(pdf_path: Path) -> str:
@@ -3,13 +3,13 @@ import json
from django.test import TestCase
from django.test import override_settings
from documents.tests.utils import FileSystemAssertsMixin
from paperless.models import ApplicationConfiguration
from paperless.models import CleanChoices
from paperless.models import ColorConvertChoices
from paperless.models import ModeChoices
from paperless.models import OutputTypeChoices
from paperless.parsers.tesseract import RasterisedDocumentParser
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
@@ -3,8 +3,8 @@ from pathlib import Path
import pytest
from documents.tests.utils import util_call_with_backoff
from paperless.parsers.tika import TikaDocumentParser
from paperless_testing.retry import util_call_with_backoff
@pytest.mark.skipif(
@@ -17,6 +17,24 @@ class TestRemoteUser(DirectoriesMixin, APITestCase):
self.user = UserFactory(username="temp_admin", superuser=True)
# _parse_remote_user_settings() mutates these shared lists in place,
# so undo that after the test instead of leaking remote-user auth
# into every test that runs afterward.
original_middleware = list(settings.MIDDLEWARE)
original_auth_backends = list(settings.AUTHENTICATION_BACKENDS)
original_auth_classes = list(
settings.REST_FRAMEWORK["DEFAULT_AUTHENTICATION_CLASSES"],
)
def _restore_remote_user_settings() -> None:
settings.MIDDLEWARE[:] = original_middleware
settings.AUTHENTICATION_BACKENDS[:] = original_auth_backends
settings.REST_FRAMEWORK["DEFAULT_AUTHENTICATION_CLASSES"][:] = (
original_auth_classes
)
self.addCleanup(_restore_remote_user_settings)
def test_remote_user(self) -> None:
"""
GIVEN:
@@ -7,6 +7,7 @@ from django.core.exceptions import ImproperlyConfigured
from paperless.settings import _get_allauth_trusted_proxy_count
from paperless.settings import _get_classifier_language_setting
from paperless.settings import _get_llm_extra_params
from paperless.settings import _get_search_language_setting
from paperless.settings import _parse_paperless_url
from paperless.settings import default_threads_per_worker
@@ -166,3 +167,45 @@ class TestPaperlessURLSettings(TestCase):
self.assertIn(url, settings.CSRF_TRUSTED_ORIGINS)
self.assertIn(url, settings.CORS_ALLOWED_ORIGINS)
class TestLlmExtraParams:
@pytest.mark.parametrize(
("env_value", "expected"),
[
pytest.param(None, {}, id="unset"),
pytest.param(
'{"reasoning_effort": "none"}',
{"reasoning_effort": "none"},
id="json-object",
),
],
)
def test_parses(
self,
monkeypatch,
env_value,
expected,
):
if env_value is None:
monkeypatch.delenv("PAPERLESS_AI_LLM_EXTRA_PARAMS", raising=False)
else:
monkeypatch.setenv("PAPERLESS_AI_LLM_EXTRA_PARAMS", env_value)
assert _get_llm_extra_params() == expected
@pytest.mark.parametrize(
("env_value", "match"),
[
pytest.param("reasoning_effort=none", "valid JSON", id="invalid-json"),
pytest.param('["none"]', "JSON object", id="not-an-object"),
],
)
def test_invalid_raises(
self,
monkeypatch,
env_value,
match,
):
monkeypatch.setenv("PAPERLESS_AI_LLM_EXTRA_PARAMS", env_value)
with pytest.raises(ImproperlyConfigured, match=match):
_get_llm_extra_params()
@@ -30,3 +30,27 @@ class TestBooleanConfigPrecedence(TestCase):
config.save()
self.assertTrue(AIConfig().ai_enabled)
class TestAIConfigPrecedence(TestCase):
@override_settings(LLM_EMBEDDING_API_KEY="environment-embedding-key")
def test_database_embedding_api_key_overrides_environment_setting(self) -> None:
config, _ = ApplicationConfiguration.objects.get_or_create()
config.llm_embedding_api_key = "database-embedding-key"
config.save()
self.assertEqual(
AIConfig().llm_embedding_api_key,
"database-embedding-key",
)
@override_settings(LLM_EMBEDDING_API_KEY="environment-embedding-key")
def test_null_embedding_api_key_uses_environment_setting(self) -> None:
config, _ = ApplicationConfiguration.objects.get_or_create()
config.llm_embedding_api_key = None
config.save()
self.assertEqual(
AIConfig().llm_embedding_api_key,
"environment-embedding-key",
)
@@ -1,4 +1,4 @@
from documents.tests.utils import TestMigrations
from paperless_testing.migrations import TestMigrations
class TestMigrateSkipArchiveFile(TestMigrations):
+2
View File
@@ -75,6 +75,7 @@ class AIClient:
context_window=self.settings.llm_context_size,
request_timeout=self.settings.llm_request_timeout,
system_prompt=LLM_SYSTEM_PROMPT,
additional_kwargs=self.settings.llm_extra_params,
client=Client(
host=endpoint,
timeout=self.settings.llm_request_timeout,
@@ -111,6 +112,7 @@ class AIClient:
is_chat_model=True,
is_function_calling_model=True,
system_prompt=LLM_SYSTEM_PROMPT,
additional_kwargs=self.settings.llm_extra_params,
http_client=http_client,
async_http_client=async_http_client,
)
+3 -1
View File
@@ -41,7 +41,9 @@ def get_embedding_model(config: AIConfig) -> "BaseEmbedding":
)
return OpenAILikeEmbedding(
model_name=config.llm_embedding_model or "text-embedding-3-small",
api_key=config.llm_api_key or PLACEHOLDER_API_KEY,
api_key=config.llm_embedding_api_key
or config.llm_api_key
or PLACEHOLDER_API_KEY,
api_base=endpoint,
timeout=config.llm_request_timeout,
http_client=http_client,
+7 -2
View File
@@ -1,6 +1,7 @@
import datetime
from collections.abc import Generator
from types import SimpleNamespace
from typing import TYPE_CHECKING
from unittest.mock import MagicMock
from unittest.mock import patch
@@ -28,6 +29,9 @@ from paperless_testing.factories import TagFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_object
if TYPE_CHECKING:
from paperless_testing.dirs import PaperlessDirs
@pytest.fixture
def mock_document():
@@ -630,10 +634,11 @@ class TestFulltextSimilarDocuments:
def fulltext_backend(
self,
mocker: pytest_mock.MockerFixture,
paperless_dirs: "PaperlessDirs",
) -> Generator[TantivyBackend, None, None]:
"""An in-memory Tantivy backend, wired up as the module-level
"""An on-disk Tantivy backend, wired up as the module-level
singleton _fulltext_similar_documents resolves via get_backend()."""
backend = TantivyBackend(path=None)
backend = TantivyBackend(path=paperless_dirs.index_dir)
backend.open()
mocker.patch("documents.search.get_backend", return_value=backend)
try:
+33
View File
@@ -23,6 +23,7 @@ def mock_ai_config():
mock_config.llm_allow_internal_endpoints = True
mock_config.llm_context_size = 8192
mock_config.llm_request_timeout = 120
mock_config.llm_extra_params = {}
MockAIConfig.return_value = mock_config
yield mock_config
@@ -52,6 +53,7 @@ def test_get_llm_ollama(mock_ai_config, mock_ollama_llm):
context_window=8192,
request_timeout=120,
system_prompt=LLM_SYSTEM_PROMPT,
additional_kwargs={},
client=ANY,
async_client=ANY,
)
@@ -74,6 +76,7 @@ def test_get_llm_openai(mock_ai_config, mock_openai_llm):
is_chat_model=True,
is_function_calling_model=True,
system_prompt=LLM_SYSTEM_PROMPT,
additional_kwargs={},
http_client=ANY,
async_http_client=ANY,
)
@@ -196,6 +199,36 @@ def test_run_llm_query_openai_uses_tools(mock_ai_config, mock_openai_llm):
)
@pytest.mark.parametrize(
("backend", "llm_fixture"),
[
pytest.param("openai-like", "mock_openai_llm", id="openai-like"),
pytest.param("ollama", "mock_ollama_llm", id="ollama"),
],
)
def test_get_llm_passes_extra_params(request, mock_ai_config, backend, llm_fixture):
"""
GIVEN:
- Extra LLM params configured, e.g. for a provider that needs a
parameter we do not set ourselves
WHEN:
- The client builds the LLM
THEN:
- They are handed to the backend as additional_kwargs
"""
llm_mock = request.getfixturevalue(llm_fixture)
mock_ai_config.llm_backend = backend
mock_ai_config.llm_model = "gpt-5.6-luna"
mock_ai_config.llm_endpoint = "http://test-url"
mock_ai_config.llm_extra_params = {"reasoning_effort": "none"}
AIClient()
assert llm_mock.call_args.kwargs["additional_kwargs"] == {
"reasoning_effort": "none",
}
def test_run_llm_query_openai_timeout_raises_local_error(
mock_ai_config,
mock_openai_llm,
+18 -1
View File
@@ -17,6 +17,7 @@ from paperless_ai.embedding import get_embedding_model
@pytest.fixture
def mock_ai_config():
with patch("paperless_ai.embedding.AIConfig") as MockAIConfig:
MockAIConfig.return_value.llm_embedding_api_key = None
MockAIConfig.return_value.llm_embedding_endpoint = None
MockAIConfig.return_value.llm_allow_internal_endpoints = True
MockAIConfig.return_value.llm_context_size = 8192
@@ -63,6 +64,7 @@ def mock_document():
def test_get_embedding_model_openai(mock_ai_config):
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.OPENAI_LIKE
mock_ai_config.return_value.llm_embedding_model = "text-embedding-3-small"
mock_ai_config.return_value.llm_embedding_api_key = "test_embedding_api_key"
mock_ai_config.return_value.llm_api_key = "test_api_key"
mock_ai_config.return_value.llm_endpoint = "http://test-url"
@@ -72,7 +74,7 @@ def test_get_embedding_model_openai(mock_ai_config):
model = get_embedding_model(mock_ai_config.return_value)
MockOpenAIEmbedding.assert_called_once_with(
model_name="text-embedding-3-small",
api_key="test_api_key",
api_key="test_embedding_api_key",
api_base="http://test-url",
timeout=120,
http_client=ANY,
@@ -81,6 +83,20 @@ def test_get_embedding_model_openai(mock_ai_config):
assert model == MockOpenAIEmbedding.return_value
def test_get_embedding_model_openai_falls_back_to_llm_api_key(mock_ai_config):
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.OPENAI_LIKE
mock_ai_config.return_value.llm_embedding_model = "text-embedding-3-small"
mock_ai_config.return_value.llm_api_key = "test_api_key"
mock_ai_config.return_value.llm_endpoint = "http://test-url"
with patch(
"llama_index.embeddings.openai_like.OpenAILikeEmbedding",
) as MockOpenAIEmbedding:
get_embedding_model(mock_ai_config.return_value)
assert MockOpenAIEmbedding.call_args.kwargs["api_key"] == "test_api_key"
@pytest.mark.parametrize("configured_key", [None, ""])
def test_get_embedding_model_openai_without_api_key_sends_placeholder(
mock_ai_config,
@@ -89,6 +105,7 @@ def test_get_embedding_model_openai_without_api_key_sends_placeholder(
"""Same required key handling as the LLM client, see #13831."""
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.OPENAI_LIKE
mock_ai_config.return_value.llm_embedding_model = "text-embedding-3-small"
mock_ai_config.return_value.llm_embedding_api_key = configured_key
mock_ai_config.return_value.llm_api_key = configured_key
mock_ai_config.return_value.llm_endpoint = "http://test-url"
+262
View File
@@ -0,0 +1,262 @@
import dataclasses
import email.message
import uuid
from contextlib import AbstractContextManager
from imap_tools import MailboxFolderSelectError
from imap_tools import MailboxLoginError
from imap_tools import MailMessage
from imap_tools import MailMessageFlags
@dataclasses.dataclass
class _AttachmentDef:
filename: str = "a_file.pdf"
maintype: str = "application/pdf"
subtype: str = "pdf"
disposition: str = "attachment"
content: bytes = b"a PDF document"
class BogusFolderManager:
current_folder = "INBOX"
uidvalidity = "1"
def set(self, new_folder) -> None:
if new_folder not in ["INBOX", "spam"]:
raise MailboxFolderSelectError(None, "uhm")
self.current_folder = new_folder
def status(self, folder, options):
return {"UIDVALIDITY": self.uidvalidity}
class BogusClient:
def __init__(self, messages) -> None:
self.messages: list[MailMessage] = messages
self.capabilities: list[str] = []
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
pass
def authenticate(self, mechanism, authobject) -> None:
# authobject must be a callable object
auth_bytes = authobject(None)
if auth_bytes != b"\x00admin\x00w57\xc3\xa4\xc3\xb6\xc3\xbcw4b6huwb6nhu":
raise MailboxLoginError("BAD", "OK")
def uid(self, command, *args) -> None:
if command == "STORE":
for message in self.messages:
if message.uid == args[0]:
flag = args[2]
if flag == "processed":
message._raw_flag_data.append(b"+FLAGS (processed)")
if hasattr(message, "flags"):
del message.flags
class BogusMailBox(AbstractContextManager):
# Common values so tests don't need to remember an accepted login
USERNAME: str = "admin"
ASCII_PASSWORD: str = "secret"
# Note the non-ascii characters here
UTF_PASSWORD: str = "w57äöüw4b6huwb6nhu"
# A dummy access token
ACCESS_TOKEN = "ea7e075cd3acf2c54c48e600398d5d5a"
def __init__(self) -> None:
self.messages: list[MailMessage] = []
self.messages_spam: list[MailMessage] = []
self.folder = BogusFolderManager()
self.client = BogusClient(self.messages)
self._host = ""
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
pass
def updateClient(self) -> None:
self.client = BogusClient(self.messages)
def login(self, username, password) -> None:
# This will raise a UnicodeEncodeError if the password is not ASCII only
password.encode("ascii")
# Otherwise, check for correct values
if username != self.USERNAME or password != self.ASCII_PASSWORD:
raise MailboxLoginError("BAD", "OK")
def login_utf8(self, username, password) -> None:
# Expected to only be called with the UTF-8 password
if username != self.USERNAME or password != self.UTF_PASSWORD:
raise MailboxLoginError("BAD", "OK")
def xoauth2(self, username: str, access_token: str) -> None:
if username != self.USERNAME or access_token != self.ACCESS_TOKEN:
raise MailboxLoginError("BAD", "OK")
def fetch(
self,
criteria="ALL",
charset="",
*,
mark_seen=True,
bulk=True,
uid_list=None,
):
if uid_list is not None:
return [m for m in self.messages if m.uid in uid_list]
return self._filter_messages(criteria)
def uids(self, criteria, charset="") -> list[str]:
return [m.uid for m in self._filter_messages(criteria)]
def _filter_messages(self, criteria):
msg = self.messages
criteria = str(criteria).strip("()").split(" ")
if "UNSEEN" in criteria:
msg = filter(lambda m: not m.seen, msg)
if "SUBJECT" in criteria:
subject = criteria[criteria.index("SUBJECT") + 1].strip('"')
msg = filter(lambda m: subject in m.subject, msg)
if "BODY" in criteria:
body = criteria[criteria.index("BODY") + 1].strip('"')
msg = filter(lambda m: body in m.text, msg)
if "FROM" in criteria:
from_ = criteria[criteria.index("FROM") + 1].strip('"')
msg = filter(lambda m: from_ in m.from_, msg)
if "TO" in criteria:
to_ = criteria[criteria.index("TO") + 1].strip('"')
msg = filter(lambda m: any(to_ in to_addr for to_addr in m.to), msg)
if "UNFLAGGED" in criteria:
msg = filter(lambda m: not m.flagged, msg)
if "UNKEYWORD" in criteria:
tag = criteria[criteria.index("UNKEYWORD") + 1].strip("'")
msg = filter(lambda m: tag not in m.flags, msg)
if "(X-GM-LABELS" in criteria: # ['NOT', '(X-GM-LABELS', '"processed"']
msg = filter(lambda m: "processed" not in m.flags, msg)
if "UID" in criteria:
uid_list = criteria[criteria.index("UID") + 1].split(",")
msg = filter(lambda m: m.uid in uid_list, msg)
return list(msg)
def delete(self, uid_list) -> None:
self.messages = list(filter(lambda m: m.uid not in uid_list, self.messages))
def flag(self, uid_list, flag_set, value) -> None:
for message in self.messages:
if message.uid in uid_list:
for flag in flag_set:
if flag == MailMessageFlags.FLAGGED:
message.flagged = value
if flag == MailMessageFlags.SEEN:
message.seen = value
if flag == "processed":
message._raw_flag_data.append(b"+FLAGS (processed)")
if hasattr(message, "flags"):
del message.flags
def move(self, uid_list, folder) -> None:
if folder == "spam":
self.messages_spam += list(
filter(lambda m: m.uid in uid_list, self.messages),
)
self.messages = list(filter(lambda m: m.uid not in uid_list, self.messages))
else:
raise Exception
def fake_magic_from_buffer(buffer, *, mime=False):
if mime:
if "PDF" in str(buffer):
return "application/pdf"
else:
return "unknown/type"
else:
return "Some verbose file description"
class MessageBuilder:
def __init__(self) -> None:
self._next_uid = 1
def create_message(
self,
*,
attachments: int | list[_AttachmentDef] = 1,
body: str = "",
subject: str = "the subject",
from_: str = "no_one@mail.com",
to: list[str] | None = None,
seen: bool = False,
flagged: bool = False,
processed: bool = False,
) -> MailMessage:
if to is None:
to = ["tosomeone@somewhere.com"]
email_msg = email.message.EmailMessage()
# TODO: This does NOT set the UID
email_msg["Message-ID"] = str(uuid.uuid4())
email_msg["Subject"] = subject
email_msg["From"] = from_
email_msg["To"] = str(" ,".join(to))
email_msg.set_content(body)
# Either add some default number of attachments
# or the provided attachments
if isinstance(attachments, int):
for i in range(attachments):
attachment = _AttachmentDef(filename=f"file_{i}.pdf")
email_msg.add_attachment(
attachment.content,
maintype=attachment.maintype,
subtype=attachment.subtype,
disposition=attachment.disposition,
filename=attachment.filename,
)
else:
for attachment in attachments:
email_msg.add_attachment(
attachment.content,
maintype=attachment.maintype,
subtype=attachment.subtype,
disposition=attachment.disposition,
filename=attachment.filename,
)
# Convert the EmailMessage to an imap_tools MailMessage
imap_msg = MailMessage.from_bytes(email_msg.as_bytes())
# TODO: Unsure how to add a uid to the actual EmailMessage. This hacks it in,
# based on how imap_tools uses regex to extract it.
# This should be a large enough pool
uid = self._next_uid
self._next_uid += 1
imap_msg._raw_uid_data = f"UID {uid}".encode()
imap_msg.seen = seen
imap_msg.flagged = flagged
if processed:
imap_msg._raw_flag_data.append(b"+FLAGS (processed)")
if hasattr(imap_msg, "flags"):
del imap_msg.flags
return imap_msg
+1 -1
View File
@@ -11,7 +11,7 @@ from paperless_mail.models import ProcessedMail
from paperless_mail.tests.factories import MailAccountFactory
from paperless_mail.tests.factories import MailRuleFactory
from paperless_mail.tests.factories import ProcessedMailFactory
from paperless_mail.tests.test_mail import BogusMailBox
from paperless_mail.tests.helpers import BogusMailBox
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import DocumentTypeFactory
+16 -264
View File
@@ -1,9 +1,7 @@
import dataclasses
import email.contentmanager
import time
import uuid
from collections import namedtuple
from contextlib import AbstractContextManager
from datetime import timedelta
from unittest import mock
@@ -18,16 +16,12 @@ from imap_tools import NOT
from imap_tools import EmailAddress
from imap_tools import FolderInfo
from imap_tools import MailboxFolderSelectError
from imap_tools import MailboxLoginError
from imap_tools import MailMessage
from imap_tools import MailMessageFlags
from imap_tools import errors
from rest_framework import status
from rest_framework.test import APITestCase
from documents.models import Correspondent
from documents.models import MatchingModel
from documents.tests.utils import FileSystemAssertsMixin
from paperless_mail import tasks
from paperless_mail.mail import MailAccountHandler
from paperless_mail.mail import MailError
@@ -40,265 +34,17 @@ from paperless_mail.models import MailRule
from paperless_mail.models import ProcessedMail
from paperless_mail.tests.factories import MailAccountFactory
from paperless_mail.tests.factories import MailRuleFactory
from paperless_mail.tests.helpers import BogusMailBox
from paperless_mail.tests.helpers import MessageBuilder
from paperless_mail.tests.helpers import _AttachmentDef
from paperless_mail.tests.helpers import fake_magic_from_buffer
from paperless_testing.assertions import FileSystemAssertsMixin
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import CorrespondentFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
@dataclasses.dataclass
class _AttachmentDef:
filename: str = "a_file.pdf"
maintype: str = "application/pdf"
subtype: str = "pdf"
disposition: str = "attachment"
content: bytes = b"a PDF document"
class BogusFolderManager:
current_folder = "INBOX"
uidvalidity = "1"
def set(self, new_folder) -> None:
if new_folder not in ["INBOX", "spam"]:
raise MailboxFolderSelectError(None, "uhm")
self.current_folder = new_folder
def status(self, folder, options):
return {"UIDVALIDITY": self.uidvalidity}
class BogusClient:
def __init__(self, messages) -> None:
self.messages: list[MailMessage] = messages
self.capabilities: list[str] = []
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
pass
def authenticate(self, mechanism, authobject) -> None:
# authobject must be a callable object
auth_bytes = authobject(None)
if auth_bytes != b"\x00admin\x00w57\xc3\xa4\xc3\xb6\xc3\xbcw4b6huwb6nhu":
raise MailboxLoginError("BAD", "OK")
def uid(self, command, *args) -> None:
if command == "STORE":
for message in self.messages:
if message.uid == args[0]:
flag = args[2]
if flag == "processed":
message._raw_flag_data.append(b"+FLAGS (processed)")
if hasattr(message, "flags"):
del message.flags
class BogusMailBox(AbstractContextManager):
# Common values so tests don't need to remember an accepted login
USERNAME: str = "admin"
ASCII_PASSWORD: str = "secret"
# Note the non-ascii characters here
UTF_PASSWORD: str = "w57äöüw4b6huwb6nhu"
# A dummy access token
ACCESS_TOKEN = "ea7e075cd3acf2c54c48e600398d5d5a"
def __init__(self) -> None:
self.messages: list[MailMessage] = []
self.messages_spam: list[MailMessage] = []
self.folder = BogusFolderManager()
self.client = BogusClient(self.messages)
self._host = ""
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
pass
def updateClient(self) -> None:
self.client = BogusClient(self.messages)
def login(self, username, password) -> None:
# This will raise a UnicodeEncodeError if the password is not ASCII only
password.encode("ascii")
# Otherwise, check for correct values
if username != self.USERNAME or password != self.ASCII_PASSWORD:
raise MailboxLoginError("BAD", "OK")
def login_utf8(self, username, password) -> None:
# Expected to only be called with the UTF-8 password
if username != self.USERNAME or password != self.UTF_PASSWORD:
raise MailboxLoginError("BAD", "OK")
def xoauth2(self, username: str, access_token: str) -> None:
if username != self.USERNAME or access_token != self.ACCESS_TOKEN:
raise MailboxLoginError("BAD", "OK")
def fetch(
self,
criteria="ALL",
charset="",
*,
mark_seen=True,
bulk=True,
uid_list=None,
):
if uid_list is not None:
return [m for m in self.messages if m.uid in uid_list]
return self._filter_messages(criteria)
def uids(self, criteria, charset="") -> list[str]:
return [m.uid for m in self._filter_messages(criteria)]
def _filter_messages(self, criteria):
msg = self.messages
criteria = str(criteria).strip("()").split(" ")
if "UNSEEN" in criteria:
msg = filter(lambda m: not m.seen, msg)
if "SUBJECT" in criteria:
subject = criteria[criteria.index("SUBJECT") + 1].strip('"')
msg = filter(lambda m: subject in m.subject, msg)
if "BODY" in criteria:
body = criteria[criteria.index("BODY") + 1].strip('"')
msg = filter(lambda m: body in m.text, msg)
if "FROM" in criteria:
from_ = criteria[criteria.index("FROM") + 1].strip('"')
msg = filter(lambda m: from_ in m.from_, msg)
if "TO" in criteria:
to_ = criteria[criteria.index("TO") + 1].strip('"')
msg = filter(lambda m: any(to_ in to_addr for to_addr in m.to), msg)
if "UNFLAGGED" in criteria:
msg = filter(lambda m: not m.flagged, msg)
if "UNKEYWORD" in criteria:
tag = criteria[criteria.index("UNKEYWORD") + 1].strip("'")
msg = filter(lambda m: tag not in m.flags, msg)
if "(X-GM-LABELS" in criteria: # ['NOT', '(X-GM-LABELS', '"processed"']
msg = filter(lambda m: "processed" not in m.flags, msg)
if "UID" in criteria:
uid_list = criteria[criteria.index("UID") + 1].split(",")
msg = filter(lambda m: m.uid in uid_list, msg)
return list(msg)
def delete(self, uid_list) -> None:
self.messages = list(filter(lambda m: m.uid not in uid_list, self.messages))
def flag(self, uid_list, flag_set, value) -> None:
for message in self.messages:
if message.uid in uid_list:
for flag in flag_set:
if flag == MailMessageFlags.FLAGGED:
message.flagged = value
if flag == MailMessageFlags.SEEN:
message.seen = value
if flag == "processed":
message._raw_flag_data.append(b"+FLAGS (processed)")
if hasattr(message, "flags"):
del message.flags
def move(self, uid_list, folder) -> None:
if folder == "spam":
self.messages_spam += list(
filter(lambda m: m.uid in uid_list, self.messages),
)
self.messages = list(filter(lambda m: m.uid not in uid_list, self.messages))
else:
raise Exception
def fake_magic_from_buffer(buffer, *, mime=False):
if mime:
if "PDF" in str(buffer):
return "application/pdf"
else:
return "unknown/type"
else:
return "Some verbose file description"
class MessageBuilder:
def __init__(self) -> None:
self._next_uid = 1
def create_message(
self,
*,
attachments: int | list[_AttachmentDef] = 1,
body: str = "",
subject: str = "the subject",
from_: str = "no_one@mail.com",
to: list[str] | None = None,
seen: bool = False,
flagged: bool = False,
processed: bool = False,
) -> MailMessage:
if to is None:
to = ["tosomeone@somewhere.com"]
email_msg = email.message.EmailMessage()
# TODO: This does NOT set the UID
email_msg["Message-ID"] = str(uuid.uuid4())
email_msg["Subject"] = subject
email_msg["From"] = from_
email_msg["To"] = str(" ,".join(to))
email_msg.set_content(body)
# Either add some default number of attachments
# or the provided attachments
if isinstance(attachments, int):
for i in range(attachments):
attachment = _AttachmentDef(filename=f"file_{i}.pdf")
email_msg.add_attachment(
attachment.content,
maintype=attachment.maintype,
subtype=attachment.subtype,
disposition=attachment.disposition,
filename=attachment.filename,
)
else:
for attachment in attachments:
email_msg.add_attachment(
attachment.content,
maintype=attachment.maintype,
subtype=attachment.subtype,
disposition=attachment.disposition,
filename=attachment.filename,
)
# Convert the EmailMessage to an imap_tools MailMessage
imap_msg = MailMessage.from_bytes(email_msg.as_bytes())
# TODO: Unsure how to add a uid to the actual EmailMessage. This hacks it in,
# based on how imap_tools uses regex to extract it.
# This should be a large enough pool
uid = self._next_uid
self._next_uid += 1
imap_msg._raw_uid_data = f"UID {uid}".encode()
imap_msg.seen = seen
imap_msg.flagged = flagged
if processed:
imap_msg._raw_flag_data.append(b"+FLAGS (processed)")
if hasattr(imap_msg, "flags"):
del imap_msg.flags
return imap_msg
def reset_bogus_mailbox(
bogus_mailbox: BogusMailBox,
message_builder: MessageBuilder,
@@ -1819,7 +1565,12 @@ class TestMail(
("electronic", None, "invoices@mycompany.com", None, 1),
(None, "amazon", "me@myselfandi.com", None, 1),
]:
with self.subTest(f_body=f_body, f_from=f_from, f_subject=f_subject):
with self.subTest(
f_body=f_body,
f_from=f_from,
f_to=f_to,
f_subject=f_subject,
):
MailRule.objects.all().delete()
_ = MailRule.objects.create(
name="testrule3",
@@ -2060,7 +1811,7 @@ class TestPostConsumeAction(TestCase):
with (
self.assertRaises(errors.ImapToolsError),
self.assertLogs("paperless.mail", level="ERROR") as cm,
self.assertLogs("paperless_mail", level="ERROR") as cm,
):
apply_mail_action(
result=[],
@@ -2069,9 +1820,10 @@ class TestPostConsumeAction(TestCase):
message_subject=self.message_subject,
message_date=self.message_date,
)
error_str = cm.output[0]
expected_str = "Error while processing mail action during post_consume"
self.assertIn(expected_str, error_str)
error_str = cm.output[0]
expected_str = "Error while processing mail action during post_consume"
self.assertIn(expected_str, error_str)
processed_mail = ProcessedMail.objects.get(uid=self.message_uid)
self.assertEqual(processed_mail.status, "FAILED")
+3 -3
View File
@@ -15,9 +15,9 @@ import pytest
from paperless_mail.models import MailRule
from paperless_mail.tests.factories import MailAccountFactory
from paperless_mail.tests.test_mail import MessageBuilder
from paperless_mail.tests.test_mail import _AttachmentDef
from paperless_mail.tests.test_mail import fake_magic_from_buffer
from paperless_mail.tests.helpers import MessageBuilder
from paperless_mail.tests.helpers import _AttachmentDef
from paperless_mail.tests.helpers import fake_magic_from_buffer
@pytest.fixture()
@@ -16,8 +16,8 @@ from paperless_mail.mail import MailAccountHandler
from paperless_mail.models import MailRule
from paperless_mail.preprocessor import MailMessageDecryptor
from paperless_mail.tests.factories import MailAccountFactory
from paperless_mail.tests.helpers import _AttachmentDef
from paperless_mail.tests.test_mail import TestMail
from paperless_mail.tests.test_mail import _AttachmentDef
class MessageEncryptor:
+37
View File
@@ -0,0 +1,37 @@
"""Filesystem assertions for unittest-style tests."""
from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from os import PathLike
class FileSystemAssertsMixin:
def assertIsFile(self, path: PathLike[str] | str) -> None:
if not Path(path).resolve().is_file():
raise AssertionError(f"File does not exist: {path}")
def assertIsNotFile(self, path: PathLike[str] | str) -> None:
if Path(path).resolve().is_file():
raise AssertionError(f"File does exist: {path}")
def assertIsDir(self, path: PathLike[str] | str) -> None:
if not Path(path).resolve().is_dir():
raise AssertionError(f"Dir does not exist: {path}")
def assertIsNotDir(self, path: PathLike[str] | str) -> None:
if Path(path).resolve().is_dir():
raise AssertionError(f"Dir does exist: {path}")
def assertFileCountInDir(self, path: PathLike[str] | str, count: int) -> None:
path = Path(path).resolve()
if not path.is_dir():
raise AssertionError(f"Path {path} is not a directory")
found = len([x for x in path.iterdir() if x.is_file()])
if found != count:
raise AssertionError(
f"Path {path} contains {found} files instead of {count} files",
)
+4
View File
@@ -37,6 +37,7 @@ class PaperlessDirs:
logging_dir: Path
model_file: Path
media_lock: Path
share_link_bundle_dir: Path
class DirSettings(TypedDict):
@@ -54,6 +55,7 @@ class DirSettings(TypedDict):
STATIC_ROOT: Path
MODEL_FILE: Path
MEDIA_LOCK: Path
SHARE_LINK_BUNDLE_DIR: Path
def build_paperless_dirs(root: Path) -> PaperlessDirs:
@@ -75,6 +77,7 @@ def build_paperless_dirs(root: Path) -> PaperlessDirs:
logging_dir=data_dir / "log",
model_file=data_dir / "classification_model.pickle",
media_lock=media_dir / "media.lock",
share_link_bundle_dir=documents_dir / "share_link_bundles",
)
for directory in (
@@ -109,6 +112,7 @@ def dirs_settings(dirs: PaperlessDirs) -> DirSettings:
STATIC_ROOT=dirs.static_dir,
MODEL_FILE=dirs.model_file,
MEDIA_LOCK=dirs.media_lock,
SHARE_LINK_BUNDLE_DIR=dirs.share_link_bundle_dir,
)
+31
View File
@@ -0,0 +1,31 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from documents.plugins.helpers import ProgressManager
if TYPE_CHECKING:
from documents.plugins.helpers import WebsocketPayload
class FakeProgressManager(ProgressManager):
"""
The real ProgressManager with the channel layer cut out: send_progress still
builds the payload, so it cannot drift, and the payloads are recorded instead
of being sent to Redis.
Use it through the `fake_progress_manager` fixture, or construct it directly.
"""
def __init__(self, filename: str | None = None, task_id: str | None = None) -> None:
super().__init__(filename, task_id)
self.payloads: list[WebsocketPayload] = []
def open(self) -> None:
pass
def close(self) -> None:
pass
def send(self, payload: WebsocketPayload) -> None:
self.payloads.append(payload)
+13
View File
@@ -0,0 +1,13 @@
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from django.http import StreamingHttpResponse
def read_streaming_response(response: StreamingHttpResponse) -> bytes:
"""Consume a StreamingHttpResponse/FileResponse and close it."""
content = b"".join(response.streaming_content)
response.close()
return content
+65
View File
@@ -0,0 +1,65 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from typing import Any
from django.apps import apps
from django.db import connection
from django.db.migrations.executor import MigrationExecutor
from django.test import TransactionTestCase
if TYPE_CHECKING:
from django.apps.registry import Apps
class TestMigrations(TransactionTestCase):
@property
def app(self) -> str:
return apps.get_containing_app_config(type(self).__module__).name
migrate_from: Any = None
dependencies: list[tuple[str, str]] | None = None
migrate_to: Any = None
def setUp(self) -> None:
super().setUp()
assert self.migrate_from and self.migrate_to, (
f"TestCase '{type(self).__name__}' must define migrate_from and migrate_to properties"
)
self.migrate_from = [(self.app, self.migrate_from)]
if self.dependencies is not None:
self.migrate_from.extend(self.dependencies)
self.migrate_to = [(self.app, self.migrate_to)]
executor = MigrationExecutor(connection)
old_apps = executor.loader.project_state(self.migrate_from).apps
# Reverse to the original migration
executor.migrate(self.migrate_from)
self.setUpBeforeMigration(old_apps)
self.apps = old_apps
# Run the migration to test
executor = MigrationExecutor(connection)
executor.loader.build_graph() # reload.
executor.migrate(self.migrate_to)
self.apps = executor.loader.project_state(self.migrate_to).apps
def setUpBeforeMigration(self, apps: Apps) -> None:
pass
def tearDown(self) -> None:
"""
Ensure the database schema is restored to the latest migration after
each migration test, so subsequent tests run against HEAD.
"""
try:
executor = MigrationExecutor(connection)
executor.loader.build_graph()
targets = executor.loader.graph.leaf_nodes()
executor.migrate(targets)
finally:
super().tearDown()
+76
View File
@@ -0,0 +1,76 @@
from __future__ import annotations
import time
import warnings
from typing import TYPE_CHECKING
from typing import Any
import httpx
import pytest
from documents.parsers import ParseError
if TYPE_CHECKING:
from collections.abc import Callable
def util_call_with_backoff(
method_or_callable: Callable,
args: list | tuple,
*,
skip_on_50x_err: bool = True,
) -> tuple[bool, Any]:
"""
For whatever reason, the images started during the test pipeline like to
segfault sometimes, crash and otherwise fail randomly, when run with the
exact files that usually pass.
So, this function will retry the given method/function up to 3 times, with larger backoff
periods between each attempt, in hopes the issue resolves itself during
one attempt to parse.
This will wait the following:
- Attempt 1 - 20s following failure
- Attempt 2 - 40s following failure
- Attempt 3 - 80s following failure
"""
result = None
succeeded = False
retry_time = 20.0
retry_count = 0
status_codes = []
max_retry_count = 3
while retry_count < max_retry_count and not succeeded:
try:
result = method_or_callable(*args)
succeeded = True
except ParseError as e: # pragma: no cover
cause_exec = e.__cause__
if cause_exec is not None and isinstance(cause_exec, httpx.HTTPStatusError):
status_codes.append(cause_exec.response.status_code)
warnings.warn(
f"HTTP Exception for {cause_exec.request.url} - {cause_exec}",
)
else:
warnings.warn(f"Unexpected error: {e}")
except Exception as e: # pragma: no cover
warnings.warn(f"Unexpected error: {e}")
retry_count = retry_count + 1
if not succeeded and retry_count < max_retry_count:
time.sleep(retry_time)
retry_time = retry_time * 2.0
if (
not succeeded
and status_codes
and skip_on_50x_err
and all(httpx.codes.is_server_error(code) for code in status_codes)
):
pytest.skip("Repeated HTTP 50x for service") # pragma: no cover
return succeeded, result
Generated
+57 -47
View File
@@ -220,7 +220,7 @@ wheels = [
[[package]]
name = "autobahn"
version = "25.12.2"
version = "26.7.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cbor2" },
@@ -228,24 +228,35 @@ dependencies = [
{ name = "cryptography" },
{ name = "hyperlink" },
{ name = "msgpack", marker = "platform_python_implementation == 'CPython'" },
{ name = "py-ubjson" },
{ name = "txaio" },
{ name = "u-msgpack-python", marker = "platform_python_implementation != 'CPython'" },
{ name = "ujson" },
]
sdist = { url = "https://files.pythonhosted.org/packages/54/d5/9adf0f5b9eb244e58e898e9f3db4b00c09835ef4b6c37d491886e0376b4f/autobahn-25.12.2.tar.gz", hash = "sha256:754c06a54753aeb7e8d10c5cbf03249ad9e2a1a32bca8be02865c6f00628a98c", size = 13893652, upload-time = "2025-12-15T11:13:19.086Z" }
sdist = { url = "https://files.pythonhosted.org/packages/de/73/f109f563c27e048e45d135d81af19e6ca391e24905550b06bd1c9d674c57/autobahn-26.7.1.tar.gz", hash = "sha256:c6949a2c6eb95fb1c218837dbda0a59abbbebafb8b11098551c01a7061dfd245", size = 14056542, upload-time = "2026-07-15T19:14:01.246Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ae/23/923e4f11dc9d12b9f5a014f36d591c479d623d54dda3bdcbd688cd12f052/autobahn-25.12.2-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:16df879672c60f1f3fe452138c80f0fd221b3cb2ee5a14390c80f33b994104c1", size = 2053413, upload-time = "2025-12-15T11:12:58.167Z" },
{ url = "https://files.pythonhosted.org/packages/b3/0d/3d39637a1e32f555ce5fabec4a723a035556ef918b14140faea05e7de902/autobahn-25.12.2-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6ffe28048ef96eb0f925f24c2569bd72332e120f4cb31cd6c40dd66718a5f85e", size = 2224850, upload-time = "2025-12-15T11:13:00.089Z" },
{ url = "https://files.pythonhosted.org/packages/64/8d/36452c06cbcad6d04587aeb87dfa987ef94be4a427b9f2155783d166bd97/autobahn-25.12.2-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:220748f21e91bd4a538d2d3de640cc17ee30b79f1c04a6c3dcdef321d531ee1c", size = 2225453, upload-time = "2025-12-15T11:13:02.865Z" },
{ url = "https://files.pythonhosted.org/packages/83/30/ef9c47038e4e9257319d6e1b87668b3df360a0c488d66ccff9d11aaff6ba/autobahn-25.12.2-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:bc17f6cab9438156d2701c293c76fd02a144f9be0a992c065dfee1935ce4845b", size = 1960447, upload-time = "2025-12-15T11:13:05.007Z" },
{ url = "https://files.pythonhosted.org/packages/e2/e4/f3d5cb70bc0b9b5523d940734b2e0a251510d051a50d2e723f321e890859/autobahn-25.12.2-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b5297a782fc7d0a26842438ef1342549ceee29496cda52672ac44635c79eeb94", size = 2053955, upload-time = "2025-12-15T11:13:06.052Z" },
{ url = "https://files.pythonhosted.org/packages/ea/49/4e592a19ae58fd9c796821a882b22598fac295ede50f899cc9d14a0282b6/autobahn-25.12.2-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c0c3f1d5dafda52f8dc962ab583b6f3473b7b7186cab082d05372ed43a8261a5", size = 2225441, upload-time = "2025-12-15T11:13:07.527Z" },
{ url = "https://files.pythonhosted.org/packages/54/b7/0a0e3ecb2af7e452f5f359d19bdc647cbc8658f3f498bfa3bf8545cf4768/autobahn-25.12.2-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:c840ee136bfaf6560467160129b0b25a0e33c9a51e2b251e98c5474f27583915", size = 1960463, upload-time = "2025-12-15T11:13:10.183Z" },
{ url = "https://files.pythonhosted.org/packages/19/8b/4215ac49d6b793b592fb08698f3a0e21a59eb3520be7f7ed288fcb52d919/autobahn-25.12.2-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9abda5cf817c0f8a19a55a67a031adf2fc70ed351719b5bd9e6fa0f5f4bc8f89", size = 2225590, upload-time = "2025-12-15T11:13:11.367Z" },
{ url = "https://files.pythonhosted.org/packages/d6/99/b4a3da42471d3ec36e2dca0c1a5368a079fed9f73b159ce3f049c4a4983b/autobahn-25.12.2-pp311-pypy311_pp73-macosx_15_0_arm64.whl", hash = "sha256:0c226329ddec154c6f3b491ea3e4713035f0326c96ebfd6b305bf90f27a2fba1", size = 1955357, upload-time = "2025-12-15T11:13:13.581Z" },
{ url = "https://files.pythonhosted.org/packages/89/81/67f19dd7395a9f1123a1f071314f8d1c4879c1869adeb8d99a236e756ac0/autobahn-25.12.2-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:5f079393a7626eb448c8accf21151f5f206d02f8e9cee4313d62a5ca30a3aaed", size = 623173, upload-time = "2025-12-15T11:13:14.945Z" },
{ url = "https://files.pythonhosted.org/packages/71/eb/857eab3d25e3b9cc9e7e741d6193808ad91de0befb38cf10658bd339c205/autobahn-25.12.2-pp311-pypy311_pp73-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9b3a6c7d54a9f0434a435d88b86555510e5d0a84aa87042e292f29f707cab237", size = 2178008, upload-time = "2025-12-15T11:13:15.881Z" },
{ url = "https://files.pythonhosted.org/packages/44/8c/381cdcab8016df2177adc93d25f84ca3a5fb8f8be4f9d784336416c7bee8/autobahn-26.7.1-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:3fe80550707f0affb5cb10f3e0f66ec7e6e52abb29edc66dd76734c2d7d51bf4", size = 1997747, upload-time = "2026-07-15T19:13:21.998Z" },
{ url = "https://files.pythonhosted.org/packages/79/a9/9293c6c6bc8970f42c9675942de78f306e18eafa47edba52fd27f9dc71bd/autobahn-26.7.1-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:00fb9acd8775eaa0e272f36b76db903f10de56478f6a72f0bd07ee882ae1f2b8", size = 2082284, upload-time = "2026-07-15T19:13:23.582Z" },
{ url = "https://files.pythonhosted.org/packages/a9/ba/7396cb42a9c59df20c350ea05f75e6f25f582b474dee82b8e32823b2711e/autobahn-26.7.1-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:97c9674eddd55ad3ebd733824789175e5fb90c88afd523de507569ba0fcd6853", size = 2254260, upload-time = "2026-07-15T19:13:24.894Z" },
{ url = "https://files.pythonhosted.org/packages/b7/64/19753442770662ff45c4fe48db6345ac6fa3100fbbd989241c074e38ea6f/autobahn-26.7.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:30fa714de5c9903ef64084d3a938d8a3bac0bb42f1532d5de22f34b04a1c4819", size = 3173653, upload-time = "2026-07-15T19:13:26.261Z" },
{ url = "https://files.pythonhosted.org/packages/04/a4/b690f272427acf1e8ea03b146e559dc67ade10dd4e0cccacc1d4c011b141/autobahn-26.7.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:c3362197f3b9d5b0df7f3365bd00dedba7ee8b649d652941377abe690d1c8b14", size = 3402880, upload-time = "2026-07-15T19:13:27.683Z" },
{ url = "https://files.pythonhosted.org/packages/6e/23/0769ef39e1cfb0bec15bacdd7f407aaedfda14c0ca3f7e818b856f2ed1a1/autobahn-26.7.1-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:6c9013e9aa9ea8a561c89d7be2709546b51fc7ac8fdf6cd71bc12a634672d9a8", size = 2000605, upload-time = "2026-07-15T19:13:30.391Z" },
{ url = "https://files.pythonhosted.org/packages/f9/ef/26833f38ecf3aef3ff0aa09feb12f5d472f7370104148a6b78e3c7afc286/autobahn-26.7.1-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:abd9ebe577dd1030f9a0c41d20dc00eca90d5cf338531abcb510d978825feeea", size = 2082844, upload-time = "2026-07-15T19:13:31.559Z" },
{ url = "https://files.pythonhosted.org/packages/b2/4c/00553ee9d57ee11df47bc9867d120cfe721a73bec0b58fb3a3b91cd7c797/autobahn-26.7.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:de491baa4cf52fb6d7f542e445c72d94ce52dc7aabb47b0b4d5191e452dd2c74", size = 2254852, upload-time = "2026-07-15T19:13:32.8Z" },
{ url = "https://files.pythonhosted.org/packages/c3/ca/7884f6ffb8410882df98cb939dea24225dd79e4f091ceb59f4b826e54f2f/autobahn-26.7.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:9723561c820ed032fe5a8f1530cb6d5f91dc595eea6009f3b2de7044a3df892a", size = 3174235, upload-time = "2026-07-15T19:13:34.134Z" },
{ url = "https://files.pythonhosted.org/packages/1f/e7/c6704e8f6bef3aa552a851a34908a06d91317d35ca55ec04b5db14385c33/autobahn-26.7.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e81c86cf41adca8a56ca5621ecd9ba40037f6d90d338c334bd529c8ac94bd7b6", size = 3403687, upload-time = "2026-07-15T19:13:35.633Z" },
{ url = "https://files.pythonhosted.org/packages/36/92/2f6e57d9f9e6b86b9db362f58aaa6cfeadc2f3a6901ec95aab27ef232b5c/autobahn-26.7.1-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:2ce48214b28f73338fabe0c7fd13d222cfab9e1dd2ef11660293522f64e76727", size = 1987052, upload-time = "2026-07-15T19:13:38.543Z" },
{ url = "https://files.pythonhosted.org/packages/38/6d/f170134468e276fa9ea57eb1ae41f9cc0dcd0228e9d501f370fa50c0ee31/autobahn-26.7.1-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a5f285dce9b3dff3eb2ef6c818ac8ede24d96bd1edac340855170fc9825c38a9", size = 2082813, upload-time = "2026-07-15T19:13:39.686Z" },
{ url = "https://files.pythonhosted.org/packages/0d/ce/b735fa933e9ba4fa8c3f9aa9ae68b4e2d4aadb0a92b38ade30e37a7d4795/autobahn-26.7.1-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:66ab6e034e54f8c473df1a6b8031a3db46c16deebaf0c9b66db3e9137d2fab5e", size = 2254818, upload-time = "2026-07-15T19:13:40.951Z" },
{ url = "https://files.pythonhosted.org/packages/df/3e/57855f4f52aa0ee64c6d8210637d0d9847de4c1082e03ddd2ffafd853d2d/autobahn-26.7.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:20b3eab7d483e93278f9d7345592eb6b465883a6e88c2823aad13c3f943452db", size = 682494, upload-time = "2026-07-15T19:13:42.193Z" },
{ url = "https://files.pythonhosted.org/packages/3b/3c/3944f17dd2a06aee7d0d9f1c37b5a94518434d13ba2c38e738d33ca10daf/autobahn-26.7.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:cd840524ff190aee695a58e8acd8740ee66b4a0e6a58ccb41416ed2cbe48d43f", size = 3403666, upload-time = "2026-07-15T19:13:43.508Z" },
{ url = "https://files.pythonhosted.org/packages/51/3e/200471878093a502f8e8078c1ca19fd82acc68ae9ac363e395170da6dbe2/autobahn-26.7.1-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:f3d1be925e3fb33fff5280c1bd02027047519812c400d3efa5477d3968686c94", size = 1987070, upload-time = "2026-07-15T19:13:47.112Z" },
{ url = "https://files.pythonhosted.org/packages/61/d1/704f881fd2c52b056dc0f14e6d0d640b1f3ff43f3b84cf85d3631e3243f4/autobahn-26.7.1-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:44f8094b0c0fa29a4963ce12130b7932469f89afa0242ff858d2b26541a81005", size = 2082939, upload-time = "2026-07-15T19:13:48.569Z" },
{ url = "https://files.pythonhosted.org/packages/a8/27/84e76aec7abbcb502d4cd34ef5c859eaa19a3b707cda68ab7ce68478dd92/autobahn-26.7.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a0fcf9c3ff6b9b2bc85d6e1814a94d1d941d62f7df36d350b523d77df85d66ea", size = 2254983, upload-time = "2026-07-15T19:13:49.802Z" },
{ url = "https://files.pythonhosted.org/packages/e6/81/a810732a10342c5d6b90d19f83fa2bc9b6126e7c0cda7c4df867e311aa2e/autobahn-26.7.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:4f82e5a113f6c1ff14cec99aa411f7da8fceec3dcd4647d7ebdfc0278811e14d", size = 3174295, upload-time = "2026-07-15T19:13:51.227Z" },
{ url = "https://files.pythonhosted.org/packages/a6/c6/4886fdaecfeda013e085288a9d83bad6f1ded9995b8088ca933d9ec37201/autobahn-26.7.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:919309cbe41b28b0a3028e7c6fed52aca9fd21639619f372d3270c44341b6bdc", size = 3403713, upload-time = "2026-07-15T19:13:52.744Z" },
{ url = "https://files.pythonhosted.org/packages/94/14/6485c29ad06a6bd7b3017558f99f6f89dcaa6ef6641930b25d9247adba4c/autobahn-26.7.1-pp311-pypy311_pp73-macosx_15_0_arm64.whl", hash = "sha256:9088acf790caf8cfd86590cb2b749279256ee210198f41d9858a38d1346e56c9", size = 1981962, upload-time = "2026-07-15T19:13:55.79Z" },
{ url = "https://files.pythonhosted.org/packages/8d/46/cb6d09604417beacdf485b414a05efa18511b0e78ac5451b3655bee711fa/autobahn-26.7.1-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ea4548ee15c6bdf8aa0a1e81bf47b42db350f2bef69f83bace27a95ed0d21276", size = 655967, upload-time = "2026-07-15T19:13:57.19Z" },
{ url = "https://files.pythonhosted.org/packages/00/d9/b846bc5a37f25ac147879d6451c466968a54efbf9d0467f0732f37c6f3f3/autobahn-26.7.1-pp311-pypy311_pp73-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d4ee0fe13a5218831d60863becd8c3cf6558e6c5ccc5d9d0e722012bdb1459bf", size = 2207401, upload-time = "2026-07-15T19:13:58.392Z" },
]
[[package]]
@@ -3211,40 +3222,45 @@ wheels = [
[[package]]
name = "pikepdf"
version = "10.2.0"
version = "10.13.0.post1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "deprecated" },
{ name = "lxml" },
{ name = "packaging" },
{ name = "pillow" },
]
sdist = { url = "https://files.pythonhosted.org/packages/6e/e9/a1462d6160805ca80c8f4aafc941aaf410a92d0fcc683706e94f499c2fac/pikepdf-10.2.0.tar.gz", hash = "sha256:0f398b0daeb2ffd2358f75c06f1dd47b9ba76f1a77dfe938cccf7080c58227d7", size = 4568506, upload-time = "2026-01-09T22:54:25.847Z" }
sdist = { url = "https://files.pythonhosted.org/packages/1c/0e/6e74dd213537b71c945743a4b3112dbb430896ad68b8a6ad22e4468455d4/pikepdf-10.13.0.post1.tar.gz", hash = "sha256:4b73f926ebae81f04bf14527af330bd00bb268be767e0f189f7c4c3e4ad7ae0a", size = 4973186, upload-time = "2026-09-05T06:49:20.825Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/10/dc/aa7293763b603a9080ffab7ab87c7b571d637a389e9fb2ba839b864ca283/pikepdf-10.2.0-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:fb93732127d5183a91300af39e1cda5ded309e8439daec93536331a472b5e190", size = 4727891, upload-time = "2026-01-09T22:53:26.262Z" },
{ url = "https://files.pythonhosted.org/packages/d2/dc/700c31f2c14f94d92483b10e1918390948ed20f6f572d82beb78ac5f94d0/pikepdf-10.2.0-cp311-cp311-macosx_15_0_x86_64.whl", hash = "sha256:ab7bd4629539cf2136a799dc3eaa2dfda59937035a97b0c5e22a7a3a4033cc49", size = 5030510, upload-time = "2026-01-09T22:53:28.023Z" },
{ url = "https://files.pythonhosted.org/packages/23/46/dc63364b05aa1913f2d7480cad62676bfb473065ba4b02d314dfd482f7dd/pikepdf-10.2.0-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2f5623a5ba456d69dfeb86dc3bb3ec31ec1d120382d8c24804d1b430fce715ea", size = 2439498, upload-time = "2026-01-09T22:53:30.122Z" },
{ url = "https://files.pythonhosted.org/packages/59/b6/1f9b8ca588fd34d9e3df49a80c62016e0b42ce6e580146c46d9728fdb6e8/pikepdf-10.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:dec4d12f294df378d122ae441c27c1e76fb0d15b1e9d7374ae70c26604559bab", size = 2666945, upload-time = "2026-01-09T22:53:32.309Z" },
{ url = "https://files.pythonhosted.org/packages/c1/31/b1e61fac59f0b807edde655a821ff83bb041ae1500234c52ed1a2403c44a/pikepdf-10.2.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c0cebe3235232f1bd3c5f7956218ce92241c94223cb80eba837d372a40c61765", size = 3638109, upload-time = "2026-01-09T22:53:34.144Z" },
{ url = "https://files.pythonhosted.org/packages/02/e9/a99bbf503c9d55e54553edff84ec67cac49d335fc33f3d5516c4746b6340/pikepdf-10.2.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:d0908e845c9140e245ad89a19fdfc6e5a6d82fcb505b8cc2c0ce81439ac4f064", size = 3829538, upload-time = "2026-01-09T22:53:36.341Z" },
{ url = "https://files.pythonhosted.org/packages/73/18/598383493a0f0f0c4eecd09b8fe06dddb9d326a89e2623a134d43e051485/pikepdf-10.2.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:18c35d00baff72bfae82d67028bedb02ea2b208e1af5545c23cd681f2487a279", size = 4737716, upload-time = "2026-01-09T22:53:39.973Z" },
{ url = "https://files.pythonhosted.org/packages/a9/f5/bec04784ba07d44f03b52ea524bcb7409bf7185ee8abec7ae29e3ac9e9ae/pikepdf-10.2.0-cp312-cp312-macosx_15_0_x86_64.whl", hash = "sha256:dd849d033b95de15965c095ebc4d78983099a11bb7b7897801dfaf3cb4083a35", size = 5042152, upload-time = "2026-01-09T22:53:41.808Z" },
{ url = "https://files.pythonhosted.org/packages/38/3e/148b3c8e101c8ac3a33f41e86c5739413575495e471bce45ee228aafcbd6/pikepdf-10.2.0-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9910efdc7907af3da9e7b2a125a1f67d512165ffa623f62825deeb642669a7a", size = 2445796, upload-time = "2026-01-09T22:53:44.027Z" },
{ url = "https://files.pythonhosted.org/packages/c2/ef/b06f8fd68c34fed631cb8e3520dd955e59987de0eee6960dbc94bed11711/pikepdf-10.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f0ec947e6429d7a3306153d32a0142462fdd8f905c5fe08c8a8e8c53b9c28a5c", size = 2693908, upload-time = "2026-01-09T22:53:46.039Z" },
{ url = "https://files.pythonhosted.org/packages/3e/bf/e5c40e9210e2ae8da7cad2cf6ae7d1db3b63a2916e6040645958e9ab4054/pikepdf-10.2.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:b6383219a1cd31400403a69737a4e2a0c5d2a2c4cb9f380bcf45e33e8de802ea", size = 3643423, upload-time = "2026-01-09T22:53:48.333Z" },
{ url = "https://files.pythonhosted.org/packages/d4/1b/969dfb29dc9fd7b82fa7bc065df498e8a3e7ddb81e982140634ee539a8db/pikepdf-10.2.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:46d2f9ef5a84949bfc11152a323558f94cf85d9d97e9c510c061c7f803028f3f", size = 3854816, upload-time = "2026-01-09T22:53:50.158Z" },
{ url = "https://files.pythonhosted.org/packages/f4/c5/e6f9e3407dd73ec570000a64747ff84e2f57b06b0477d1da6eaca5038162/pikepdf-10.2.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:09ff28d1de7fc7711a7ef8dfc40396d9243b64ee24c37cd1ab2a9f9827895caa", size = 4737680, upload-time = "2026-01-09T22:53:54.905Z" },
{ url = "https://files.pythonhosted.org/packages/eb/de/dffb785235ac2d930db86b215c1848d7258e625fa1949dd0633f8b72ab0a/pikepdf-10.2.0-cp313-cp313-macosx_15_0_x86_64.whl", hash = "sha256:62348b66e1401a4db0c64976b72dd74bb1a9eb3a33007a661500f4f8a64436bd", size = 5042150, upload-time = "2026-01-09T22:53:57.722Z" },
{ url = "https://files.pythonhosted.org/packages/1a/f0/4d883f57304d98650ade30a8c73fe593582d9afd9a7dada1f5f3f4cce362/pikepdf-10.2.0-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b9e91780cb9ea3c6a350ffbcf03d5d95c30084d238afbd1d4b927cdb9e3649d", size = 2445446, upload-time = "2026-01-09T22:53:59.993Z" },
{ url = "https://files.pythonhosted.org/packages/62/65/ffe2555812a152d616accacea7c1c617c27a75590379ea7d9cc3a26bd92d/pikepdf-10.2.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:52360a49a22e9353ec9a08ff5713cec8aacaf3ef960c704bc0a89ca8f050bdad", size = 2696242, upload-time = "2026-01-09T22:54:01.687Z" },
{ url = "https://files.pythonhosted.org/packages/ed/8c/2f937b0e2867cd48b523122e08753571fc9847978e239d7b5db9bd46879c/pikepdf-10.2.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4ba4c1046939eb22c24c396deb37f8e0500caaa66b73114be55377d5554b4167", size = 3643730, upload-time = "2026-01-09T22:54:04.177Z" },
{ url = "https://files.pythonhosted.org/packages/2e/17/f2919e4085c399e938bb945ea712dea70b3849e17cae6403f0cc1100e9ef/pikepdf-10.2.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:3dcd8957a08e0a47f7a138904dca8cf73962fa17a096a47cd8bc33eb83a4f0a7", size = 3856645, upload-time = "2026-01-09T22:54:07.887Z" },
{ url = "https://files.pythonhosted.org/packages/1d/6e/846902abe8286d3b4ab70893e9ffbeec99aadd93ba1536cf471b222bb910/pikepdf-10.2.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:194c9a81ecb49e425a5cd5162621270b5e42cf05709d87eac018bd6f9ce98f80", size = 4733930, upload-time = "2026-01-09T22:54:11.931Z" },
{ url = "https://files.pythonhosted.org/packages/8b/6d/abdbb794d2a512d4e828ef2014cc47ca263ad3fbd1b65f25f791b9c0bb1e/pikepdf-10.2.0-cp314-cp314-macosx_15_0_x86_64.whl", hash = "sha256:5adcf87dbfff4e1cd0a850db487274f474c94a6bf6347f3842c53da8d0eaa8df", size = 5042477, upload-time = "2026-01-09T22:54:13.708Z" },
{ url = "https://files.pythonhosted.org/packages/52/6c/6c42694fe1574a37aa2a40b4ba29a6713b4226436155ef7aa0bef649c117/pikepdf-10.2.0-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5de3cecbb35c4bc651e9326932974217be1d450d4a9840d77a592062eb507e27", size = 2448419, upload-time = "2026-01-09T22:54:15.6Z" },
{ url = "https://files.pythonhosted.org/packages/45/f4/aca3286aa37ace581afc8e3e0644a0cc55b9f9ceb31f28219d12ca11536c/pikepdf-10.2.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:77868fd25182a45a4f3dec3c461aea8c696ef9565894c5cde4394bc8c32fb069", size = 2697600, upload-time = "2026-01-09T22:54:18.123Z" },
{ url = "https://files.pythonhosted.org/packages/eb/a6/9135f9f0189634de61410573a0712d849e0157e3902e6b867339cc7dbf1b/pikepdf-10.2.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a9a10e15e2f4d0bba36a2b4328342d00eff1a5a31399e1d1a93483c70d3c2b0e", size = 3647720, upload-time = "2026-01-09T22:54:20.158Z" },
{ url = "https://files.pythonhosted.org/packages/83/60/f282077773a3321fad4cbfb16fe73ee3f8dd93b408df65c24779f12227c5/pikepdf-10.2.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:a8f80ecf00fb15a760f218432a1046e7797cd14eaa6ccb52c8814ae8852745d8", size = 3859133, upload-time = "2026-01-09T22:54:22.358Z" },
{ url = "https://files.pythonhosted.org/packages/4d/c1/48c9c0ed2ed88ca5d9cdd7f16075385a05a812d781b61bfa7f2d5182b247/pikepdf-10.13.0.post1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:98a7305e330f797da02b543d3ad57a134c4a14c6ec6f8d86d91aa9dd130c425b", size = 1846300, upload-time = "2026-09-05T06:48:11.563Z" },
{ url = "https://files.pythonhosted.org/packages/e9/20/484a3a61664132dc8c4bd97e0b8291fa79f9f4a3b1e1ffd5b67ac41ed98a/pikepdf-10.13.0.post1-cp311-cp311-macosx_15_0_x86_64.whl", hash = "sha256:f3dedd02795626f17ee42d5c02ec4ec94e28aa47046ef430d4478454a8fbd07f", size = 1944179, upload-time = "2026-09-05T06:48:14.01Z" },
{ url = "https://files.pythonhosted.org/packages/66/38/797df7d60352fc5ec3943c425acaa15d032cd2e673b516861f449536db57/pikepdf-10.13.0.post1-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7ba09ef5a5f26e38ee558d2a08223fee08a5ef1868962ae2d8590d4de3c8c92f", size = 2105054, upload-time = "2026-09-05T06:48:15.547Z" },
{ url = "https://files.pythonhosted.org/packages/a8/8a/1f003558c5c05cecf182af775839ad674fe23e06b314d0219b9d8422680a/pikepdf-10.13.0.post1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:365b94f2be7e2857c6cb5445b56dc52dc7417ba9f06c8a4282d9f521cb2d0fb8", size = 2308617, upload-time = "2026-09-05T06:48:17.216Z" },
{ url = "https://files.pythonhosted.org/packages/55/5b/0e7193ee8c7ca5b15f478918033a0fa78917b01249e1d4a4a65754644cd3/pikepdf-10.13.0.post1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:f515a31c76cce043bbb7b781e77a343a4e26fa8f520ba337d30ddea0f7a0ce50", size = 3742337, upload-time = "2026-09-05T06:48:18.971Z" },
{ url = "https://files.pythonhosted.org/packages/b6/f5/519e8728c04d05dcbd44d03b266a3f6acf8de3a0a6ed5ec0fa39ececddd7/pikepdf-10.13.0.post1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:b63577c44fedf7ed6b971076f7c7ed0ff95a8bac627ac93b79e8b542214861a7", size = 3952988, upload-time = "2026-09-05T06:48:20.984Z" },
{ url = "https://files.pythonhosted.org/packages/6c/a5/598e72c72ed46046e297f15763dffda88424870724a4a22f599b815cb774/pikepdf-10.13.0.post1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:2c6e83f8a1828ec79cdec4df8cc07209eaf10ed7e4f5a90a7356b254bacc07d5", size = 1845515, upload-time = "2026-09-05T06:48:24.477Z" },
{ url = "https://files.pythonhosted.org/packages/f9/b3/29691a5e9ee915357c081730d8cc02f35f19b4155857556fbe562a4d83ba/pikepdf-10.13.0.post1-cp312-cp312-macosx_15_0_x86_64.whl", hash = "sha256:f2463f650efab46905b9e279f5c776faf96c65bb45c1acf9f2de0e8a6eec5fb7", size = 1944755, upload-time = "2026-09-05T06:48:26.332Z" },
{ url = "https://files.pythonhosted.org/packages/04/4e/201f553b9405424d7aefa3be997f0a1c787ac3a089a844f1fc42f412fe0e/pikepdf-10.13.0.post1-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9613505f5b22203465d4224a3fd8cf69876ce8278442c6478ac6849f54724a30", size = 2102202, upload-time = "2026-09-05T06:48:28.816Z" },
{ url = "https://files.pythonhosted.org/packages/3e/a3/bd7e7b321e8bfe4b8530da57d12c557a259bbd4b40e739960a1f2ea507cb/pikepdf-10.13.0.post1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01f80ca046d984752cf6f08093debc193884bee91c01c104aa0236767711a20f", size = 2307194, upload-time = "2026-09-05T06:48:31.558Z" },
{ url = "https://files.pythonhosted.org/packages/23/b0/ca630f56015dfc6c4c8c81fdeb5f5099e7fca354f54a41dfe7f1382a5316/pikepdf-10.13.0.post1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:1f76fcbe5d86f2ae6f231ba542cd04793d4c89e75bd5f62526b7412926ff2100", size = 3740157, upload-time = "2026-09-05T06:48:33.285Z" },
{ url = "https://files.pythonhosted.org/packages/be/48/7a84adc2fd14ec35e4b3d007575914de1ce0518e8c69b35ebee62fa2b142/pikepdf-10.13.0.post1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:092a9bf15739e931ecab15ec3baee5d9629dad90ea4e42b779f6b439d2d1e462", size = 3951601, upload-time = "2026-09-05T06:48:35.468Z" },
{ url = "https://files.pythonhosted.org/packages/10/f4/3636368760840cbc3ee512330024dd6f518d583c1bbbb1b551ca8e18f5e8/pikepdf-10.13.0.post1-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:87141ada970386ff6640db54f0bda734d3bde7960d3ba04a76768b48e25028ca", size = 1845524, upload-time = "2026-09-05T06:48:39.309Z" },
{ url = "https://files.pythonhosted.org/packages/ce/dc/7bbfba253a0394a237b81be371a64f904f99636d579ec78ef5d92025fd2d/pikepdf-10.13.0.post1-cp313-cp313-macosx_15_0_x86_64.whl", hash = "sha256:55e53b4d8a4b1700f686f76e3a68411e421e962a4c8b1b90d00aab3f3e494a55", size = 1944829, upload-time = "2026-09-05T06:48:41.601Z" },
{ url = "https://files.pythonhosted.org/packages/b3/a3/10367bfb93501a151cbb96b6973f7c049fef04040aea35cf6cedf579c3e4/pikepdf-10.13.0.post1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8fb8f82dc43056a4b4f891e78ee1db4e3ced75ba3e87b836f8e28c8771228928", size = 2102269, upload-time = "2026-09-05T06:48:43.222Z" },
{ url = "https://files.pythonhosted.org/packages/2d/bd/a68b5d9b4aef4d4b9c374cfdfd941623f52303d31adea13554568df42abe/pikepdf-10.13.0.post1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d3b58ccb30b93ba400e6a6a83315b4830eea49f6f19e18d78241fe6b1c49fec2", size = 2307160, upload-time = "2026-09-05T06:48:45.192Z" },
{ url = "https://files.pythonhosted.org/packages/9a/59/47bd86d9e338d301c28416d52d0e154a0d6322cf6b957c9df5f3d7828aa2/pikepdf-10.13.0.post1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6b038cd5bcbb6c1952bcc271695eaf24c4199d72e45606ee5e467f837760820e", size = 3739722, upload-time = "2026-09-05T06:48:47.105Z" },
{ url = "https://files.pythonhosted.org/packages/49/40/87fb6dddc9dce110429c72449e942174fd500f6fc4c9ff518a6b73057aa7/pikepdf-10.13.0.post1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b7b0cbb135de32ec3f41651a08ab294e3c18ae9fec32516a48d27e64a53a47f0", size = 3951549, upload-time = "2026-09-05T06:48:49.263Z" },
{ url = "https://files.pythonhosted.org/packages/14/0e/86897bf5325824c1f2d9d8be89839baf11011b5392d92620cb0273fb9af5/pikepdf-10.13.0.post1-cp314-abi3-macosx_14_0_arm64.whl", hash = "sha256:51fae4a4a3c6549aa4c405896ff7010f3e43e0c4f407c0bcee071ef13d271202", size = 1845245, upload-time = "2026-09-05T06:48:53.575Z" },
{ url = "https://files.pythonhosted.org/packages/f0/ed/923846b7511627f8564d09345e083f14674dfb193de92d27f1cc4650602e/pikepdf-10.13.0.post1-cp314-abi3-macosx_15_0_x86_64.whl", hash = "sha256:8cb976331cb8b03ec3465e06d9e7a3eadbadb7e622be888e70f918ac732a105e", size = 1943659, upload-time = "2026-09-05T06:48:55.823Z" },
{ url = "https://files.pythonhosted.org/packages/58/bb/fcb09ad4bd227bbb37a7e5b24de86f9ce9d462aa0c7ee18781899bfa378a/pikepdf-10.13.0.post1-cp314-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3e18d5a009bbe5f3ab18f916fb9e28f0c5b0d920736e3a85cc10c627ec633596", size = 2099423, upload-time = "2026-09-05T06:48:57.671Z" },
{ url = "https://files.pythonhosted.org/packages/3b/c9/707f9ba96727fa366a650237e46f0e20a73b244592eb6b97a49e401e2b43/pikepdf-10.13.0.post1-cp314-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:414f42c83e5e6029870de1a988625dafc95781ebff10e15d82caeb5e69a83c9c", size = 2303539, upload-time = "2026-09-05T06:48:59.506Z" },
{ url = "https://files.pythonhosted.org/packages/22/15/79a2ccc354514a1321a161867a011fc917be158fc01a88da8c78ad18399d/pikepdf-10.13.0.post1-cp314-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:f0eb89f06cad9231b9db54d81a22592b03b63924824a6b850febd2b75daa6546", size = 3737772, upload-time = "2026-09-05T06:49:01.245Z" },
{ url = "https://files.pythonhosted.org/packages/6e/85/a17440c2de64da71dc012b42e644b30ee4d98eb540c14d4e9f3538b73a9e/pikepdf-10.13.0.post1-cp314-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:544f1be1b1e5630a79cd182a8663c439504099eb9eae0d342a50173d9825bdf3", size = 3948342, upload-time = "2026-09-05T06:49:03.27Z" },
{ url = "https://files.pythonhosted.org/packages/e9/65/15a796a3cf9fb17d41acc1ab6719e7d3dcdf1260e76909d0dd32ecc97ba7/pikepdf-10.13.0.post1-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:ef4ed47d40aa44deb063feb4a88e8bcf1c8fa0183ce526dc4295f7cbdb1292f8", size = 1853638, upload-time = "2026-09-05T06:49:07.184Z" },
{ url = "https://files.pythonhosted.org/packages/aa/f3/5d49a511fd13b59b94c5ad673695d331fce5d2846ab1501646c2a3b35b5f/pikepdf-10.13.0.post1-cp314-cp314t-macosx_15_0_x86_64.whl", hash = "sha256:571efcd1d54e0dd817973c76c253feb6fb758c93bb0c16a893cc68f2a178d404", size = 1951768, upload-time = "2026-09-05T06:49:09.048Z" },
{ url = "https://files.pythonhosted.org/packages/06/de/f6bbd9653695f6e2ed3494a439f11f3a89450c004fe9bf8ee583201ea759/pikepdf-10.13.0.post1-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2db9a18074ba112e7c517e8c21dfd8894cc13b8a37ccf49a5841192170fb68eb", size = 2107137, upload-time = "2026-09-05T06:49:10.788Z" },
{ url = "https://files.pythonhosted.org/packages/c0/aa/43b355681f05ea0b5808a26cba9e8e764686fa8dcebe0875db612664de40/pikepdf-10.13.0.post1-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f7962a75cf22d0d683b49ab19b8966e94dc7014d9aa6806f3c0d2b37fb9ae607", size = 2310991, upload-time = "2026-09-05T06:49:12.455Z" },
{ url = "https://files.pythonhosted.org/packages/a9/46/77574e9c4bded01afd7a3fe538f5432e396c3772c9bc1eab5d287aed00df/pikepdf-10.13.0.post1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:b948e11f7dd3710f939194f00b4d75b0df87a30d53b31414128768ac25da77d8", size = 3744358, upload-time = "2026-09-05T06:49:14.428Z" },
{ url = "https://files.pythonhosted.org/packages/89/a8/4857df72cf4773553c2e6a82f93ee5e98c02f1c4e4877379e84d27384982/pikepdf-10.13.0.post1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:99f6afccd6119233e7133bd4c2ade48461de3ddd4269cc28a4f90cdf7c1372f5", size = 3955964, upload-time = "2026-09-05T06:49:16.656Z" },
]
[[package]]
@@ -3604,12 +3620,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/37/ed/89c2c620af0e1660354cd8aabf9f5b21f911597ce22acb37c805d6c86bc8/psycopg_pool-3.3.1-py3-none-any.whl", hash = "sha256:2af5b432941c4c9ad5c87b3fa410aec910ec8f7c122855897983a06c45f2e4b5", size = 40023, upload-time = "2026-05-01T23:31:53.136Z" },
]
[[package]]
name = "py-ubjson"
version = "0.16.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/1d/c7/28220d37e041fe1df03e857fe48f768dcd30cd151480bf6f00da8713214a/py-ubjson-0.16.1.tar.gz", hash = "sha256:b9bfb8695a1c7e3632e800fb83c943bf67ed45ddd87cd0344851610c69a5a482", size = 50316, upload-time = "2020-04-18T15:05:57.698Z" }
[[package]]
name = "pyasn1"
version = "0.6.4"